2024-03-15T12:36:31.140 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-15T12:36:31.140 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/norstats}} --owner scheduled_leonidus@teuthology --name leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi -- /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993/orig.config.yaml 2024-03-15T12:36:31.198 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-15T12:36:31.297 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993 branch: wip-lusov-qdb-squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/norstats}} email: null first_in_suite: false job_id: '7601993' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-qdb-squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client dirsize rbytes: false client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b kclient: client.0: mount_subvol_num: 1 rbytes: false selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-qdb-squid sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b owner: scheduled_leonidus@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6918 sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b sleep_before_teardown: 0 subset: 111/10000 suite: fs suite_branch: wip-lusov-qdb-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi002.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBABhPollIpRhiA6TCjJYoOStcnohLcq0SKbzqF8cdvZiE78+44hG40KL3rAizf6AxipHJBQT3p0J83/O8OIc8xQ= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKjHaaL6R+WYZzUH2uBY+xOq0sK99jhOfHkoTIQulkLjcwuEqznoVQ+7uZai3AURfz7ONA9oorFRnVNPvJYpG6o= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG4pdgFTELesbcqVJl1lUCmcR/+y2/2gIX5pJK68t9x5P0B5uq40G7EtX1gxARnox/nBDZ/Yvjfb5ZWL5O6sPZo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - fs/norstats teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-15_11:44:52 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-15T12:36:31.297 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa; will attempt to use it 2024-03-15T12:36:31.298 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa/tasks 2024-03-15T12:36:31.298 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-15T12:36:31.300 INFO:teuthology.task.internal:Checking packages... 2024-03-15T12:36:31.322 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b' 2024-03-15T12:36:31.323 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-15T12:36:31.323 INFO:teuthology.packaging:ref: None 2024-03-15T12:36:31.323 INFO:teuthology.packaging:tag: None 2024-03-15T12:36:31.323 INFO:teuthology.packaging:branch: wip-lusov-qdb-squid 2024-03-15T12:36:31.323 INFO:teuthology.packaging:sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-15T12:36:31.323 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-qdb-squid 2024-03-15T12:36:31.467 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1681.gaba3c1d2 2024-03-15T12:36:31.468 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-15T12:36:31.475 INFO:teuthology.task.internal:no buildpackages task found 2024-03-15T12:36:31.475 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-15T12:36:31.487 INFO:teuthology.task.internal:Saving configuration 2024-03-15T12:36:31.508 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-15T12:36:31.514 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-15T12:36:31.538 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-15 12:31:22.484210', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBABhPollIpRhiA6TCjJYoOStcnohLcq0SKbzqF8cdvZiE78+44hG40KL3rAizf6AxipHJBQT3p0J83/O8OIc8xQ='} 2024-03-15T12:36:31.563 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-15 12:31:22.482281', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKjHaaL6R+WYZzUH2uBY+xOq0sK99jhOfHkoTIQulkLjcwuEqznoVQ+7uZai3AURfz7ONA9oorFRnVNPvJYpG6o='} 2024-03-15T12:36:31.585 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-15 12:31:22.486691', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG4pdgFTELesbcqVJl1lUCmcR/+y2/2gIX5pJK68t9x5P0B5uq40G7EtX1gxARnox/nBDZ/Yvjfb5ZWL5O6sPZo='} 2024-03-15T12:36:31.585 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-15T12:36:31.591 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-15T12:36:31.591 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-15T12:36:31.592 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-15T12:36:31.592 INFO:teuthology.run_tasks:Running task console_log... 2024-03-15T12:36:31.703 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f924c5010d0>, signals=[15]) 2024-03-15T12:36:31.703 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-15T12:36:31.710 INFO:teuthology.task.internal:Opening connections... 2024-03-15T12:36:31.710 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi002.front.sepia.ceph.com 2024-03-15T12:36:31.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:36:31.816 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-03-15T12:36:31.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:36:31.905 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2024-03-15T12:36:31.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:36:31.994 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-15T12:36:32.002 DEBUG:teuthology.orchestra.run.smithi002:> uname -m 2024-03-15T12:36:32.019 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2024-03-15T12:36:32.020 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2024-03-15T12:36:32.078 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2024-03-15T12:36:32.078 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="9" 2024-03-15T12:36:32.078 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2024-03-15T12:36:32.078 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2024-03-15T12:36:32.078 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="9" 2024-03-15T12:36:32.078 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el9" 2024-03-15T12:36:32.078 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-15T12:36:32.078 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2024-03-15T12:36:32.078 INFO:teuthology.orchestra.run.smithi002.stdout:LOGO="fedora-logo-icon" 2024-03-15T12:36:32.079 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-15T12:36:32.079 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2024-03-15T12:36:32.079 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-15T12:36:32.079 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-15T12:36:32.079 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-15T12:36:32.080 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2024-03-15T12:36:32.107 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-03-15T12:36:32.125 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-03-15T12:36:32.125 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-03-15T12:36:32.182 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-03-15T12:36:32.182 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-03-15T12:36:32.182 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-03-15T12:36:32.182 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-03-15T12:36:32.182 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-03-15T12:36:32.182 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-03-15T12:36:32.182 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-15T12:36:32.182 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-03-15T12:36:32.182 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-03-15T12:36:32.182 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-15T12:36:32.182 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-03-15T12:36:32.183 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-15T12:36:32.183 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-15T12:36:32.183 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-15T12:36:32.183 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-03-15T12:36:32.215 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2024-03-15T12:36:32.232 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2024-03-15T12:36:32.233 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2024-03-15T12:36:32.289 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2024-03-15T12:36:32.289 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2024-03-15T12:36:32.289 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2024-03-15T12:36:32.290 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2024-03-15T12:36:32.290 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2024-03-15T12:36:32.290 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2024-03-15T12:36:32.290 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-15T12:36:32.290 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2024-03-15T12:36:32.290 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2024-03-15T12:36:32.290 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-15T12:36:32.290 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2024-03-15T12:36:32.290 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-15T12:36:32.290 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-15T12:36:32.290 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-15T12:36:32.291 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2024-03-15T12:36:32.315 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-15T12:36:32.326 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-15T12:36:32.333 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-15T12:36:32.333 DEBUG:teuthology.orchestra.run.smithi002:> test '!' -e /home/ubuntu/cephtest 2024-03-15T12:36:32.335 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-03-15T12:36:32.338 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2024-03-15T12:36:32.353 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-15T12:36:32.359 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-15T12:36:32.359 DEBUG:teuthology.orchestra.run.smithi002:> test -z $(ls -A /var/lib/ceph) 2024-03-15T12:36:32.392 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-03-15T12:36:32.395 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2024-03-15T12:36:32.506 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-15T12:36:32.579 INFO:teuthology.run_tasks:Running task kernel... 2024-03-15T12:36:32.593 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-15T12:36:32.593 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-15T12:36:32.594 DEBUG:teuthology.orchestra.run.smithi002:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-15T12:36:32.594 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-15T12:36:32.595 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-15T12:36:32.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-15T12:36:32.611 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-03-15T12:36:32.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-15T12:36:32.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-15T12:36:32.613 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-03-15T12:36:32.613 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-03-15T12:36:32.667 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-361.el9.x86_64 2024-03-15T12:36:32.668 INFO:teuthology.task.kernel:Running kernel on smithi184: 5.14.0-361.el9.x86_64 2024-03-15T12:36:32.668 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum install -y kernel 2024-03-15T12:36:32.671 INFO:teuthology.orchestra.run.smithi002.stdout:5.14.0-361.el9.x86_64 2024-03-15T12:36:32.671 INFO:teuthology.task.kernel:Running kernel on smithi002: 5.14.0-361.el9.x86_64 2024-03-15T12:36:32.671 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum install -y kernel 2024-03-15T12:36:32.672 INFO:teuthology.orchestra.run.smithi149.stdout:5.14.0-361.el9.x86_64 2024-03-15T12:36:32.672 INFO:teuthology.task.kernel:Running kernel on smithi149: 5.14.0-361.el9.x86_64 2024-03-15T12:36:32.672 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum install -y kernel 2024-03-15T12:36:34.932 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-15T12:36:35.071 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - AppStream 9.1 MB/s | 6.7 MB 00:00 2024-03-15T12:36:35.180 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - AppStream 9.6 MB/s | 6.7 MB 00:00 2024-03-15T12:36:36.591 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-03-15T12:36:36.830 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - BaseOS 3.1 MB/s | 2.0 MB 00:00 2024-03-15T12:36:36.939 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - BaseOS 3.1 MB/s | 2.0 MB 00:00 2024-03-15T12:36:37.729 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-15T12:36:38.514 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-03-15T12:36:39.805 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-03-15T12:36:41.557 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 19 MB 00:01 2024-03-15T12:36:43.864 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.1 MB 00:00 2024-03-15T12:36:45.442 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 15 kB 00:00 2024-03-15T12:36:46.541 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-03-15T12:36:46.616 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.1 MB 00:01 2024-03-15T12:36:47.803 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 781 kB/s | 8.0 MB 00:10 2024-03-15T12:36:48.208 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 15 kB 00:00 2024-03-15T12:36:49.685 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-15T12:36:51.532 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-15T12:36:53.731 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-15T12:36:53.731 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-15T12:36:53.860 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-15T12:36:53.870 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-15T12:36:53.870 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-03-15T12:36:53.870 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-15T12:36:53.870 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-15T12:36:53.870 INFO:teuthology.orchestra.run.smithi184.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-15T12:36:53.870 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-03-15T12:36:53.870 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-15T12:36:53.871 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-15T12:36:53.871 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-15T12:36:53.871 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:36:53.871 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-15T12:36:53.871 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-15T12:36:53.871 INFO:teuthology.orchestra.run.smithi184.stdout:Install 4 Packages 2024-03-15T12:36:53.871 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:36:53.872 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 100 M 2024-03-15T12:36:53.872 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 123 M 2024-03-15T12:36:53.872 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-15T12:36:54.793 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-03-15T12:36:55.659 INFO:teuthology.orchestra.run.smithi184.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 3.5 MB/s | 6.0 MB 00:01 2024-03-15T12:36:57.083 INFO:teuthology.orchestra.run.smithi002.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-15T12:36:57.083 INFO:teuthology.orchestra.run.smithi002.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-15T12:36:57.173 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-15T12:36:57.180 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-15T12:36:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: Package Architecture Version Repository Size 2024-03-15T12:36:57.180 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-15T12:36:57.180 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-15T12:36:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-15T12:36:57.180 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-03-15T12:36:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-15T12:36:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-15T12:36:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-15T12:36:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:36:57.180 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-15T12:36:57.180 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-15T12:36:57.181 INFO:teuthology.orchestra.run.smithi002.stdout:Install 4 Packages 2024-03-15T12:36:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:36:57.181 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 100 M 2024-03-15T12:36:57.181 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 123 M 2024-03-15T12:36:57.181 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-15T12:36:57.684 INFO:teuthology.orchestra.run.smithi184.stdout:(2/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 10 MB/s | 39 MB 00:03 2024-03-15T12:36:58.126 INFO:teuthology.orchestra.run.smithi184.stdout:(3/4): kernel-modules-core-5.14.0-427.el9.x86_6 14 MB/s | 33 MB 00:02 2024-03-15T12:36:58.876 INFO:teuthology.orchestra.run.smithi184.stdout:(4/4): kernel-core-5.14.0-427.el9.x86_64.rpm 4.2 MB/s | 21 MB 00:04 2024-03-15T12:36:58.878 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-15T12:36:58.879 INFO:teuthology.orchestra.run.smithi184.stdout:Total 20 MB/s | 100 MB 00:05 2024-03-15T12:36:59.760 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-15T12:37:00.117 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-15T12:37:00.117 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-15T12:37:01.479 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-15T12:37:01.480 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-15T12:37:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-15T12:37:05.115 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-15T12:37:05.252 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-15T12:37:06.176 INFO:teuthology.orchestra.run.smithi002.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 696 kB/s | 6.0 MB 00:08 2024-03-15T12:37:08.017 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-15T12:37:08.197 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-15T12:37:12.577 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 814 kB/s | 19 MB 00:23 2024-03-15T12:37:12.941 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-15T12:37:13.149 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:37:17.885 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.1 MB 00:01 2024-03-15T12:37:17.905 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:37:19.962 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 15 kB 00:00 2024-03-15T12:37:21.898 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-15T12:37:26.962 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-15T12:37:27.145 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:01 ago on Fri 15 Mar 2024 12:37:26 PM UTC. 2024-03-15T12:37:29.169 INFO:teuthology.orchestra.run.smithi149.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-15T12:37:29.169 INFO:teuthology.orchestra.run.smithi149.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-15T12:37:29.262 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-15T12:37:29.269 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-15T12:37:29.269 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-03-15T12:37:29.269 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-15T12:37:29.269 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-15T12:37:29.269 INFO:teuthology.orchestra.run.smithi149.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-15T12:37:29.269 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-03-15T12:37:29.269 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-15T12:37:29.269 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-15T12:37:29.269 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-15T12:37:29.269 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:37:29.269 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-15T12:37:29.270 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-15T12:37:29.270 INFO:teuthology.orchestra.run.smithi149.stdout:Install 4 Packages 2024-03-15T12:37:29.270 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:37:29.270 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 100 M 2024-03-15T12:37:29.270 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 123 M 2024-03-15T12:37:29.271 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-15T12:37:29.282 INFO:teuthology.orchestra.run.smithi002.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 669 kB/s | 21 MB 00:31 2024-03-15T12:37:29.789 INFO:teuthology.orchestra.run.smithi149.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 13 MB/s | 6.0 MB 00:00 2024-03-15T12:37:30.689 INFO:teuthology.orchestra.run.smithi149.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 15 MB/s | 21 MB 00:01 2024-03-15T12:37:31.164 INFO:teuthology.orchestra.run.smithi149.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 21 MB/s | 39 MB 00:01 2024-03-15T12:37:31.605 INFO:teuthology.orchestra.run.smithi149.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 18 MB/s | 33 MB 00:01 2024-03-15T12:37:31.606 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-15T12:37:31.606 INFO:teuthology.orchestra.run.smithi149.stdout:Total 43 MB/s | 100 MB 00:02 2024-03-15T12:37:32.495 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-15T12:37:32.885 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-15T12:37:32.885 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-15T12:37:34.214 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-15T12:37:34.215 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-15T12:37:36.938 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-15T12:37:37.578 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-15T12:37:37.705 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-15T12:37:40.487 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-15T12:37:40.675 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-15T12:37:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-15T12:37:45.836 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:37:47.094 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:37:47.094 INFO:teuthology.orchestra.run.smithi184.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-15T12:37:47.095 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:37:47.181 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:37:47.625 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:37:47.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-15T12:37:47.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-15T12:37:47.625 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-15T12:37:48.055 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:37:48.055 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:37:48.055 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-15T12:37:48.056 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-15T12:37:48.056 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-15T12:37:48.056 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-15T12:37:48.056 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-15T12:37:48.056 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:37:48.056 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-15T12:37:48.592 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2024-03-15T12:37:48.735 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-15T12:37:48.735 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-15T12:37:48.735 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-15T12:37:48.736 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-15T12:37:48.736 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-15T12:37:48.736 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-03-15T12:37:48.736 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi184.front.sepia.ceph.com, path=None, version=distro) 2024-03-15T12:37:48.736 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2024-03-15T12:37:48.857 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-15T12:37:48.857 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-15T12:37:48.858 DEBUG:teuthology.orchestra.run.smithi184:> sudo rpm -qi grub2-tools 2024-03-15T12:37:48.898 INFO:teuthology.orchestra.run.smithi184.stdout:Name : grub2-tools 2024-03-15T12:37:48.898 INFO:teuthology.orchestra.run.smithi184.stdout:Epoch : 1 2024-03-15T12:37:48.898 INFO:teuthology.orchestra.run.smithi184.stdout:Version : 2.06 2024-03-15T12:37:48.898 INFO:teuthology.orchestra.run.smithi184.stdout:Release : 68.el9 2024-03-15T12:37:48.898 INFO:teuthology.orchestra.run.smithi184.stdout:Architecture: x86_64 2024-03-15T12:37:48.898 INFO:teuthology.orchestra.run.smithi184.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-15T12:37:48.898 INFO:teuthology.orchestra.run.smithi184.stdout:Group : Unspecified 2024-03-15T12:37:48.898 INFO:teuthology.orchestra.run.smithi184.stdout:Size : 8264254 2024-03-15T12:37:48.898 INFO:teuthology.orchestra.run.smithi184.stdout:License : GPLv3+ 2024-03-15T12:37:48.899 INFO:teuthology.orchestra.run.smithi184.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-15T12:37:48.899 INFO:teuthology.orchestra.run.smithi184.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-15T12:37:48.899 INFO:teuthology.orchestra.run.smithi184.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-15T12:37:48.899 INFO:teuthology.orchestra.run.smithi184.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-15T12:37:48.899 INFO:teuthology.orchestra.run.smithi184.stdout:Packager : builder@centos.org 2024-03-15T12:37:48.899 INFO:teuthology.orchestra.run.smithi184.stdout:Vendor : CentOS 2024-03-15T12:37:48.899 INFO:teuthology.orchestra.run.smithi184.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-15T12:37:48.899 INFO:teuthology.orchestra.run.smithi184.stdout:Summary : Support tools for GRUB. 2024-03-15T12:37:48.899 INFO:teuthology.orchestra.run.smithi184.stdout:Description : 2024-03-15T12:37:48.899 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:37:48.899 INFO:teuthology.orchestra.run.smithi184.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-15T12:37:48.899 INFO:teuthology.orchestra.run.smithi184.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-15T12:37:48.899 INFO:teuthology.orchestra.run.smithi184.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-15T12:37:48.900 INFO:teuthology.orchestra.run.smithi184.stdout:hardware devices. 2024-03-15T12:37:48.900 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:37:48.900 INFO:teuthology.orchestra.run.smithi184.stdout:This subpackage provides tools for support of all platforms. 2024-03-15T12:37:48.900 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-15T12:37:48.901 INFO:teuthology.task.kernel:Updating grub on smithi184 to boot 5.14.0-427.el9.x86_64 2024-03-15T12:37:48.901 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-15T12:37:49.527 INFO:teuthology.orchestra.run.smithi184.stderr:Generating grub configuration file ... 2024-03-15T12:37:50.582 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:37:50.906 INFO:teuthology.orchestra.run.smithi184.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-15T12:37:50.946 INFO:teuthology.orchestra.run.smithi184.stderr:done 2024-03-15T12:37:50.948 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-03-15T12:37:50.963 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.jG6dAzY8Nn 2024-03-15T12:37:50.964 DEBUG:teuthology.orchestra.run.smithi184:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jG6dAzY8Nn 2024-03-15T12:37:51.037 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.jG6dAzY8Nn 2024-03-15T12:37:51.175 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.jG6dAzY8Nn is 6KB 2024-03-15T12:37:51.224 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.jG6dAzY8Nn 2024-03-15T12:37:51.239 DEBUG:teuthology.orchestra.run.smithi184:> sudo /bin/ls /boot/loader/entries || true 2024-03-15T12:37:51.311 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-15T12:37:51.311 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-15T12:37:51.311 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-15T12:37:51.311 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-15T12:37:51.312 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-15T12:37:51.470 DEBUG:teuthology.orchestra.run.smithi184:> sudo shutdown -r now 2024-03-15T12:37:53.946 INFO:teuthology.orchestra.run.smithi002.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 711 kB/s | 39 MB 00:56 2024-03-15T12:37:54.130 INFO:teuthology.orchestra.run.smithi002.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 713 kB/s | 33 MB 00:47 2024-03-15T12:37:54.130 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-15T12:37:54.130 INFO:teuthology.orchestra.run.smithi002.stdout:Total 1.7 MB/s | 100 MB 00:56 2024-03-15T12:37:55.046 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-15T12:37:55.510 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-15T12:37:55.510 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-15T12:37:56.902 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-15T12:37:56.903 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-15T12:37:59.774 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-15T12:38:00.453 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-15T12:38:00.596 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-15T12:38:03.496 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-15T12:38:03.674 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-15T12:38:08.772 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-15T12:38:08.894 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:38:14.029 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:38:19.794 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:38:19.794 INFO:teuthology.orchestra.run.smithi149.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-15T12:38:19.794 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:38:19.881 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:38:20.297 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:38:20.297 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-15T12:38:20.297 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-15T12:38:20.297 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-15T12:38:20.734 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:38:20.734 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:38:20.734 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-15T12:38:20.734 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-15T12:38:20.734 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-15T12:38:20.735 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-15T12:38:20.735 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-15T12:38:20.735 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:38:20.735 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-15T12:38:21.240 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q kernel | sort -rV | head -n 1 2024-03-15T12:38:21.410 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-15T12:38:21.410 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-15T12:38:21.411 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-15T12:38:21.411 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-15T12:38:21.411 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-15T12:38:21.411 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-15T12:38:21.411 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi149.front.sepia.ceph.com, path=None, version=distro) 2024-03-15T12:38:21.411 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q kernel | sort -rV | head -n 1 2024-03-15T12:38:21.473 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-03-15T12:38:21.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-03-15T12:38:21.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:38:21.534 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-15T12:38:21.535 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-15T12:38:21.535 DEBUG:teuthology.orchestra.run.smithi149:> sudo rpm -qi grub2-tools 2024-03-15T12:38:21.569 INFO:teuthology.orchestra.run.smithi149.stdout:Name : grub2-tools 2024-03-15T12:38:21.569 INFO:teuthology.orchestra.run.smithi149.stdout:Epoch : 1 2024-03-15T12:38:21.569 INFO:teuthology.orchestra.run.smithi149.stdout:Version : 2.06 2024-03-15T12:38:21.569 INFO:teuthology.orchestra.run.smithi149.stdout:Release : 68.el9 2024-03-15T12:38:21.569 INFO:teuthology.orchestra.run.smithi149.stdout:Architecture: x86_64 2024-03-15T12:38:21.570 INFO:teuthology.orchestra.run.smithi149.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-15T12:38:21.570 INFO:teuthology.orchestra.run.smithi149.stdout:Group : Unspecified 2024-03-15T12:38:21.570 INFO:teuthology.orchestra.run.smithi149.stdout:Size : 8264254 2024-03-15T12:38:21.570 INFO:teuthology.orchestra.run.smithi149.stdout:License : GPLv3+ 2024-03-15T12:38:21.570 INFO:teuthology.orchestra.run.smithi149.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-15T12:38:21.570 INFO:teuthology.orchestra.run.smithi149.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-15T12:38:21.570 INFO:teuthology.orchestra.run.smithi149.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-15T12:38:21.570 INFO:teuthology.orchestra.run.smithi149.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-15T12:38:21.570 INFO:teuthology.orchestra.run.smithi149.stdout:Packager : builder@centos.org 2024-03-15T12:38:21.570 INFO:teuthology.orchestra.run.smithi149.stdout:Vendor : CentOS 2024-03-15T12:38:21.570 INFO:teuthology.orchestra.run.smithi149.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-15T12:38:21.570 INFO:teuthology.orchestra.run.smithi149.stdout:Summary : Support tools for GRUB. 2024-03-15T12:38:21.571 INFO:teuthology.orchestra.run.smithi149.stdout:Description : 2024-03-15T12:38:21.571 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:38:21.571 INFO:teuthology.orchestra.run.smithi149.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-15T12:38:21.571 INFO:teuthology.orchestra.run.smithi149.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-15T12:38:21.571 INFO:teuthology.orchestra.run.smithi149.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-15T12:38:21.571 INFO:teuthology.orchestra.run.smithi149.stdout:hardware devices. 2024-03-15T12:38:21.571 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:38:21.571 INFO:teuthology.orchestra.run.smithi149.stdout:This subpackage provides tools for support of all platforms. 2024-03-15T12:38:21.572 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-15T12:38:21.572 INFO:teuthology.task.kernel:Updating grub on smithi149 to boot 5.14.0-427.el9.x86_64 2024-03-15T12:38:21.572 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-15T12:38:22.179 INFO:teuthology.orchestra.run.smithi149.stderr:Generating grub configuration file ... 2024-03-15T12:38:23.566 INFO:teuthology.orchestra.run.smithi149.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-15T12:38:23.601 INFO:teuthology.orchestra.run.smithi149.stderr:done 2024-03-15T12:38:23.603 DEBUG:teuthology.orchestra.run.smithi149:> mktemp 2024-03-15T12:38:23.618 INFO:teuthology.orchestra.run.smithi149.stdout:/tmp/tmp.LCf0yy7tzK 2024-03-15T12:38:23.618 DEBUG:teuthology.orchestra.run.smithi149:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LCf0yy7tzK 2024-03-15T12:38:23.687 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0666 /tmp/tmp.LCf0yy7tzK 2024-03-15T12:38:23.826 DEBUG:teuthology.orchestra.remote:smithi149:/tmp/tmp.LCf0yy7tzK is 6KB 2024-03-15T12:38:23.874 DEBUG:teuthology.orchestra.run.smithi149:> rm -fr /tmp/tmp.LCf0yy7tzK 2024-03-15T12:38:23.889 DEBUG:teuthology.orchestra.run.smithi149:> sudo /bin/ls /boot/loader/entries || true 2024-03-15T12:38:23.954 INFO:teuthology.orchestra.run.smithi149.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-15T12:38:23.955 INFO:teuthology.orchestra.run.smithi149.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-15T12:38:23.955 INFO:teuthology.orchestra.run.smithi149.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-15T12:38:23.955 INFO:teuthology.orchestra.run.smithi149.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-15T12:38:23.956 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-15T12:38:24.106 DEBUG:teuthology.orchestra.run.smithi149:> sudo shutdown -r now 2024-03-15T12:38:39.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2024-03-15T12:38:44.897 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:38:44.897 INFO:teuthology.orchestra.run.smithi002.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-15T12:38:44.897 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:38:44.984 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:38:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:38:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-15T12:38:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-15T12:38:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-15T12:38:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-15T12:38:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:38:45.893 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-15T12:38:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-15T12:38:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-15T12:38:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-15T12:38:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-15T12:38:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:38:45.894 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-15T12:38:46.415 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q kernel | sort -rV | head -n 1 2024-03-15T12:38:46.566 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-15T12:38:46.567 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-15T12:38:46.567 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-15T12:38:46.567 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-15T12:38:46.567 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-15T12:38:46.567 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-15T12:38:46.567 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi002.front.sepia.ceph.com, path=None, version=distro) 2024-03-15T12:38:46.567 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q kernel | sort -rV | head -n 1 2024-03-15T12:38:46.694 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-15T12:38:46.694 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-15T12:38:46.694 DEBUG:teuthology.orchestra.run.smithi002:> sudo rpm -qi grub2-tools 2024-03-15T12:38:46.742 INFO:teuthology.orchestra.run.smithi002.stdout:Name : grub2-tools 2024-03-15T12:38:46.742 INFO:teuthology.orchestra.run.smithi002.stdout:Epoch : 1 2024-03-15T12:38:46.742 INFO:teuthology.orchestra.run.smithi002.stdout:Version : 2.06 2024-03-15T12:38:46.742 INFO:teuthology.orchestra.run.smithi002.stdout:Release : 68.el9 2024-03-15T12:38:46.742 INFO:teuthology.orchestra.run.smithi002.stdout:Architecture: x86_64 2024-03-15T12:38:46.742 INFO:teuthology.orchestra.run.smithi002.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-15T12:38:46.742 INFO:teuthology.orchestra.run.smithi002.stdout:Group : Unspecified 2024-03-15T12:38:46.742 INFO:teuthology.orchestra.run.smithi002.stdout:Size : 8264254 2024-03-15T12:38:46.743 INFO:teuthology.orchestra.run.smithi002.stdout:License : GPLv3+ 2024-03-15T12:38:46.743 INFO:teuthology.orchestra.run.smithi002.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-15T12:38:46.743 INFO:teuthology.orchestra.run.smithi002.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-15T12:38:46.743 INFO:teuthology.orchestra.run.smithi002.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-15T12:38:46.743 INFO:teuthology.orchestra.run.smithi002.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-15T12:38:46.743 INFO:teuthology.orchestra.run.smithi002.stdout:Packager : builder@centos.org 2024-03-15T12:38:46.743 INFO:teuthology.orchestra.run.smithi002.stdout:Vendor : CentOS 2024-03-15T12:38:46.743 INFO:teuthology.orchestra.run.smithi002.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-15T12:38:46.743 INFO:teuthology.orchestra.run.smithi002.stdout:Summary : Support tools for GRUB. 2024-03-15T12:38:46.743 INFO:teuthology.orchestra.run.smithi002.stdout:Description : 2024-03-15T12:38:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:38:46.743 INFO:teuthology.orchestra.run.smithi002.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-15T12:38:46.743 INFO:teuthology.orchestra.run.smithi002.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-15T12:38:46.744 INFO:teuthology.orchestra.run.smithi002.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-15T12:38:46.744 INFO:teuthology.orchestra.run.smithi002.stdout:hardware devices. 2024-03-15T12:38:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:38:46.744 INFO:teuthology.orchestra.run.smithi002.stdout:This subpackage provides tools for support of all platforms. 2024-03-15T12:38:46.745 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-15T12:38:46.745 INFO:teuthology.task.kernel:Updating grub on smithi002 to boot 5.14.0-427.el9.x86_64 2024-03-15T12:38:46.745 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-15T12:38:47.337 INFO:teuthology.orchestra.run.smithi002.stderr:Generating grub configuration file ... 2024-03-15T12:38:48.695 INFO:teuthology.orchestra.run.smithi002.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-15T12:38:48.727 INFO:teuthology.orchestra.run.smithi002.stderr:done 2024-03-15T12:38:48.729 DEBUG:teuthology.orchestra.run.smithi002:> mktemp 2024-03-15T12:38:48.745 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.nQ95n0sYKA 2024-03-15T12:38:48.745 DEBUG:teuthology.orchestra.run.smithi002:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nQ95n0sYKA 2024-03-15T12:38:48.817 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.nQ95n0sYKA 2024-03-15T12:38:48.903 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-03-15T12:38:48.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:38:48.963 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.nQ95n0sYKA is 6KB 2024-03-15T12:38:49.011 DEBUG:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.nQ95n0sYKA 2024-03-15T12:38:49.025 DEBUG:teuthology.orchestra.run.smithi002:> sudo /bin/ls /boot/loader/entries || true 2024-03-15T12:38:49.094 INFO:teuthology.orchestra.run.smithi002.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-15T12:38:49.094 INFO:teuthology.orchestra.run.smithi002.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-15T12:38:49.094 INFO:teuthology.orchestra.run.smithi002.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-15T12:38:49.094 INFO:teuthology.orchestra.run.smithi002.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-15T12:38:49.095 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-15T12:38:49.246 DEBUG:teuthology.orchestra.run.smithi002:> sudo shutdown -r now 2024-03-15T12:38:51.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2024-03-15T12:38:54.111 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-15T12:38:54.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-15T12:38:54.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:39:03.977 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-03-15T12:39:03.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:39:12.670 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-03-15T12:39:19.251 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-15T12:39:19.251 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-03-15T12:39:19.252 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:39:21.674 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-15T12:39:21.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:39:24.734 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-03-15T12:39:35.839 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-03-15T12:39:36.351 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-03-15T12:39:36.351 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-427.el9.x86_64"... 2024-03-15T12:39:36.352 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-03-15T12:39:36.379 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-427.el9.x86_64 2024-03-15T12:39:36.379 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-15T12:39:36.379 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-15T12:39:36.380 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-15T12:39:36.735 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-15T12:39:36.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:39:37.380 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-03-15T12:39:37.381 DEBUG:teuthology.orchestra.run.smithi184:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-15T12:39:37.432 INFO:teuthology.orchestra.run.smithi184.stdout:ttyS1 2024-03-15T12:39:37.605 DEBUG:teuthology.parallel:result is None 2024-03-15T12:39:37.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-03-15T12:39:46.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-03-15T12:39:46.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:39:55.170 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-03-15T12:40:10.171 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-15T12:40:10.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:40:10.529 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-03-15T12:40:10.959 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-15T12:40:10.959 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-427.el9.x86_64"... 2024-03-15T12:40:10.959 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-03-15T12:40:10.973 INFO:teuthology.orchestra.run.smithi149.stdout:5.14.0-427.el9.x86_64 2024-03-15T12:40:10.973 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-15T12:40:10.973 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-15T12:40:10.974 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-15T12:40:11.974 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-15T12:40:11.975 DEBUG:teuthology.orchestra.run.smithi149:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-15T12:40:12.062 INFO:teuthology.orchestra.run.smithi149.stdout:ttyS1 2024-03-15T12:40:12.094 DEBUG:teuthology.parallel:result is None 2024-03-15T12:40:46.763 DEBUG:teuthology.orchestra.remote:timed out 2024-03-15T12:40:58.764 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-03-15T12:40:58.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:40:59.098 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-03-15T12:40:59.492 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-03-15T12:40:59.492 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-427.el9.x86_64"... 2024-03-15T12:40:59.493 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-03-15T12:40:59.508 INFO:teuthology.orchestra.run.smithi002.stdout:5.14.0-427.el9.x86_64 2024-03-15T12:40:59.508 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-15T12:40:59.508 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-15T12:40:59.508 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-15T12:41:00.509 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-15T12:41:00.509 DEBUG:teuthology.orchestra.run.smithi002:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-15T12:41:00.578 INFO:teuthology.orchestra.run.smithi002.stdout:ttyS1 2024-03-15T12:41:00.619 DEBUG:teuthology.parallel:result is None 2024-03-15T12:41:00.619 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-15T12:41:00.627 INFO:teuthology.task.internal:Creating test directory... 2024-03-15T12:41:00.627 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-15T12:41:00.630 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-15T12:41:00.633 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-15T12:41:00.651 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-15T12:41:00.695 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-15T12:41:00.704 INFO:teuthology.task.internal:Creating archive directory... 2024-03-15T12:41:00.704 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-15T12:41:00.707 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-15T12:41:00.709 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-15T12:41:00.763 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-15T12:41:00.769 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-15T12:41:00.770 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-15T12:41:00.773 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-15T12:41:00.781 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-15T12:41:00.814 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-15T12:41:00.815 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-15T12:41:00.820 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-15T12:41:00.827 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-15T12:41:00.829 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-15T12:41:00.834 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-15T12:41:00.836 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-15T12:41:00.842 INFO:teuthology.task.internal:Configuring sudo... 2024-03-15T12:41:00.843 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-15T12:41:00.872 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-15T12:41:00.878 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-15T12:41:00.908 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-15T12:41:00.917 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-15T12:41:00.917 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-15T12:41:00.941 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-15T12:41:00.947 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-15T12:41:00.964 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-15T12:41:01.118 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-15T12:41:01.170 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-15T12:41:01.258 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-15T12:41:01.304 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:41:01.304 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-15T12:41:01.373 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-15T12:41:01.416 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-15T12:41:01.489 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-15T12:41:01.531 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-15T12:41:01.576 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-15T12:41:01.576 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-15T12:41:01.645 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-15T12:41:01.692 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-15T12:41:01.744 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-15T12:41:01.828 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-15T12:41:01.874 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-15T12:41:01.874 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-15T12:41:01.943 DEBUG:teuthology.orchestra.run.smithi002:> sudo service rsyslog restart 2024-03-15T12:41:01.946 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-03-15T12:41:01.948 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2024-03-15T12:41:01.990 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-15T12:41:01.999 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-15T12:41:02.034 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-15T12:41:02.404 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-15T12:41:02.411 INFO:teuthology.task.internal:Starting timer... 2024-03-15T12:41:02.411 INFO:teuthology.run_tasks:Running task pcp... 2024-03-15T12:41:02.421 INFO:teuthology.run_tasks:Running task selinux... 2024-03-15T12:41:02.430 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-15T12:41:02.430 DEBUG:teuthology.orchestra.run.smithi002:> sudo service auditd rotate 2024-03-15T12:41:02.473 INFO:teuthology.orchestra.run.smithi002.stdout:Rotating logs: 2024-03-15T12:41:02.475 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2024-03-15T12:41:02.523 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2024-03-15T12:41:02.525 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2024-03-15T12:41:02.593 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: 2024-03-15T12:41:02.594 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-15T12:41:02.594 DEBUG:teuthology.orchestra.run.smithi002:> /usr/sbin/getenforce 2024-03-15T12:41:02.620 INFO:teuthology.orchestra.run.smithi002.stdout:Permissive 2024-03-15T12:41:02.620 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2024-03-15T12:41:02.649 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2024-03-15T12:41:02.649 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2024-03-15T12:41:02.681 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2024-03-15T12:41:02.681 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi002.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2024-03-15T12:41:02.681 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-15T12:41:02.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-15T12:41:02.710 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-15T12:41:02.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-15T12:41:02.739 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-15T12:41:02.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-15T12:41:02.768 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-15T12:41:02.768 DEBUG:teuthology.orchestra.run.smithi002:> sudo /usr/sbin/setenforce permissive 2024-03-15T12:41:02.806 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2024-03-15T12:41:02.843 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2024-03-15T12:41:02.874 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-15T12:41:02.884 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-15T12:41:02.890 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-15T12:41:03.043 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-15T12:41:03.060 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-15T12:41:03.061 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi002.front.sepia.ceph.com,smithi149.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-15T12:45:19.062 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi002.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2024-03-15T12:45:19.063 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-03-15T12:45:19.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:45:19.155 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-03-15T12:45:19.232 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-03-15T12:45:19.233 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-15T12:45:19.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:45:19.321 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-03-15T12:45:19.401 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-15T12:45:19.401 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-03-15T12:45:19.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-15T12:45:19.491 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-03-15T12:45:19.572 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-03-15T12:45:19.572 INFO:teuthology.run_tasks:Running task clock... 2024-03-15T12:45:19.583 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-15T12:45:19.584 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-15T12:45:19.584 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-15T12:45:19.587 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-15T12:45:19.587 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-15T12:45:19.589 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-15T12:45:19.589 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-15T12:45:19.616 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-15T12:45:19.620 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-15T12:45:19.632 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-15T12:45:19.637 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-15T12:45:19.646 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-15T12:45:19.662 INFO:teuthology.orchestra.run.smithi002.stderr:sudo: ntpd: command not found 2024-03-15T12:45:19.665 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-15T12:45:19.666 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2024-03-15T12:45:19.675 INFO:teuthology.orchestra.run.smithi002.stdout:506 Cannot talk to daemon 2024-03-15T12:45:19.680 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2024-03-15T12:45:19.692 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-15T12:45:19.696 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-15T12:45:19.697 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2024-03-15T12:45:19.708 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-15T12:45:19.712 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2024-03-15T12:45:19.713 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-15T12:45:19.730 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-15T12:45:19.746 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-15T12:45:19.761 INFO:teuthology.orchestra.run.smithi002.stderr:bash: line 1: ntpq: command not found 2024-03-15T12:45:19.763 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-03-15T12:45:19.765 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-15T12:45:19.765 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2024-03-15T12:45:19.765 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-15T12:45:19.765 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-15T12:45:19.765 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-15T12:45:19.765 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-15T12:45:19.767 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-15T12:45:19.767 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-03-15T12:45:19.767 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-15T12:45:19.767 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-15T12:45:19.767 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-15T12:45:19.767 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-15T12:45:19.801 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-03-15T12:45:19.804 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-15T12:45:19.804 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-03-15T12:45:19.804 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-15T12:45:19.804 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-15T12:45:19.804 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-15T12:45:19.804 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-15T12:45:19.805 INFO:teuthology.run_tasks:Running task install... 2024-03-15T12:45:19.813 DEBUG:teuthology.task.install:project ceph 2024-03-15T12:45:19.814 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b'}} 2024-03-15T12:45:19.814 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b'} 2024-03-15T12:45:19.814 INFO:teuthology.task.install:Using flavor: default 2024-03-15T12:45:19.820 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-15T12:45:19.820 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-15T12:45:19.820 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b', 'tag': None, 'wait_for_package': False} 2024-03-15T12:45:19.821 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-15T12:45:19.823 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b', 'tag': None, 'wait_for_package': False} 2024-03-15T12:45:19.823 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-15T12:45:19.824 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b', 'tag': None, 'wait_for_package': False} 2024-03-15T12:45:19.824 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-15T12:45:19.978 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/ 2024-03-15T12:45:19.978 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1681.gaba3c1d2 2024-03-15T12:45:20.127 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/ 2024-03-15T12:45:20.127 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1681.gaba3c1d2 2024-03-15T12:45:20.131 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/ 2024-03-15T12:45:20.131 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1681.gaba3c1d2 2024-03-15T12:45:20.188 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-15T12:45:20.189 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-15T12:45:20.189 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-15T12:45:20.217 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-15T12:45:20.218 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-15T12:45:20.241 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-15T12:45:20.242 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-15T12:45:20.242 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-15T12:45:20.273 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-15T12:45:20.274 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-15T12:45:20.287 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-15T12:45:20.298 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-15T12:45:20.298 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:45:20.298 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-15T12:45:20.326 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-15T12:45:20.326 DEBUG:teuthology.orchestra.run.smithi002:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-15T12:45:20.341 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-15T12:45:20.367 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-15T12:45:20.394 DEBUG:teuthology.orchestra.run.smithi002:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-15T12:45:20.421 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-15T12:45:20.434 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2024-03-15T12:45:20.435 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2024-03-15T12:45:20.475 DEBUG:teuthology.orchestra.run.smithi002:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-15T12:45:20.488 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2024-03-15T12:45:20.489 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2024-03-15T12:45:20.541 INFO:teuthology.orchestra.run.smithi002.stdout:check_obsoletes = 1 2024-03-15T12:45:20.542 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum clean all 2024-03-15T12:45:20.675 INFO:teuthology.orchestra.run.smithi149.stdout:74 files removed 2024-03-15T12:45:20.702 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-15T12:45:20.728 INFO:teuthology.orchestra.run.smithi184.stdout:74 files removed 2024-03-15T12:45:20.757 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-15T12:45:20.791 INFO:teuthology.orchestra.run.smithi002.stdout:74 files removed 2024-03-15T12:45:20.818 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-15T12:45:21.453 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 220 kB/s | 83 kB 00:00 2024-03-15T12:45:21.486 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 182 kB/s | 83 kB 00:00 2024-03-15T12:45:21.545 INFO:teuthology.orchestra.run.smithi002.stdout:ceph packages for x86_64 205 kB/s | 83 kB 00:00 2024-03-15T12:45:21.829 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 31 kB/s | 10 kB 00:00 2024-03-15T12:45:21.920 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 26 kB/s | 10 kB 00:00 2024-03-15T12:45:21.966 INFO:teuthology.orchestra.run.smithi002.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-15T12:45:22.246 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-03-15T12:45:22.315 INFO:teuthology.orchestra.run.smithi002.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-03-15T12:45:22.354 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-03-15T12:45:22.976 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-15T12:45:23.182 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - AppStream 7.4 MB/s | 6.7 MB 00:00 2024-03-15T12:45:23.257 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - AppStream 7.7 MB/s | 6.7 MB 00:00 2024-03-15T12:45:24.509 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.0 MB 00:00 2024-03-15T12:45:24.582 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-15T12:45:24.774 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-03-15T12:45:25.389 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-15T12:45:26.306 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.0 MB 00:01 2024-03-15T12:45:26.425 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-03-15T12:45:27.099 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-03-15T12:45:28.823 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-03-15T12:45:29.076 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-03-15T12:45:31.169 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 6.1 MB 00:00 2024-03-15T12:45:32.526 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 15 kB 00:00 2024-03-15T12:45:32.824 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.1 MB 00:00 2024-03-15T12:45:33.972 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-15T12:45:34.093 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.1 MB 00:01 2024-03-15T12:45:34.200 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 15 kB 00:00 2024-03-15T12:45:35.479 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-03-15T12:45:35.794 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 15 kB 00:00 2024-03-15T12:45:38.919 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-15T12:45:40.560 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-15T12:45:40.857 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 4.2 MB/s | 21 MB 00:04 2024-03-15T12:45:41.432 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-15T12:45:41.433 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-15T12:45:41.447 INFO:teuthology.orchestra.run.smithi184.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-15T12:45:41.448 INFO:teuthology.orchestra.run.smithi184.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-15T12:45:41.449 INFO:teuthology.orchestra.run.smithi184.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-15T12:45:41.450 INFO:teuthology.orchestra.run.smithi184.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-15T12:45:41.452 INFO:teuthology.orchestra.run.smithi184.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-15T12:45:41.455 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-15T12:45:41.456 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-15T12:45:41.536 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-15T12:45:41.557 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================ 2024-03-15T12:45:41.557 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-03-15T12:45:41.557 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================ 2024-03-15T12:45:41.557 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-15T12:45:41.557 INFO:teuthology.orchestra.run.smithi184.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-15T12:45:41.557 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 6.6 k 2024-03-15T12:45:41.557 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 5.3 M 2024-03-15T12:45:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 887 k 2024-03-15T12:45:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 145 k 2024-03-15T12:45:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 1.5 M 2024-03-15T12:45:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 138 k 2024-03-15T12:45:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 1.8 M 2024-03-15T12:45:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 7.4 M 2024-03-15T12:45:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 49 k 2024-03-15T12:45:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 9.8 M 2024-03-15T12:45:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 47 M 2024-03-15T12:45:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 264 k 2024-03-15T12:45:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 748 k 2024-03-15T12:45:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 218 k 2024-03-15T12:45:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 27 k 2024-03-15T12:45:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 18 k 2024-03-15T12:45:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-15T12:45:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-15T12:45:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-15T12:45:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 31 k 2024-03-15T12:45:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 744 k 2024-03-15T12:45:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 127 k 2024-03-15T12:45:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-15T12:45:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-15T12:45:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-15T12:45:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-15T12:45:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-15T12:45:41.560 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-15T12:45:41.560 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 161 k 2024-03-15T12:45:41.560 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 321 k 2024-03-15T12:45:41.560 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 298 k 2024-03-15T12:45:41.560 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 99 k 2024-03-15T12:45:41.560 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 85 k 2024-03-15T12:45:41.560 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 3.1 M 2024-03-15T12:45:41.560 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 171 k 2024-03-15T12:45:41.560 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-15T12:45:41.560 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2024-03-15T12:45:41.560 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-15T12:45:41.560 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-15T12:45:41.560 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-15T12:45:41.561 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-15T12:45:41.561 INFO:teuthology.orchestra.run.smithi184.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-15T12:45:41.561 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-15T12:45:41.561 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-15T12:45:41.561 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-15T12:45:41.561 INFO:teuthology.orchestra.run.smithi184.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-15T12:45:41.561 INFO:teuthology.orchestra.run.smithi184.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-15T12:45:41.561 INFO:teuthology.orchestra.run.smithi184.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-15T12:45:41.561 INFO:teuthology.orchestra.run.smithi184.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-15T12:45:41.561 INFO:teuthology.orchestra.run.smithi184.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-15T12:45:41.561 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 3.4 M 2024-03-15T12:45:41.561 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 3.1 M 2024-03-15T12:45:41.561 INFO:teuthology.orchestra.run.smithi184.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-15T12:45:41.562 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-15T12:45:41.562 INFO:teuthology.orchestra.run.smithi184.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-15T12:45:41.562 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-15T12:45:41.562 INFO:teuthology.orchestra.run.smithi184.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-15T12:45:41.562 INFO:teuthology.orchestra.run.smithi184.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-15T12:45:41.562 INFO:teuthology.orchestra.run.smithi184.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-15T12:45:41.562 INFO:teuthology.orchestra.run.smithi184.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-15T12:45:41.562 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-15T12:45:41.562 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-03-15T12:45:41.562 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-15T12:45:41.562 INFO:teuthology.orchestra.run.smithi184.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-15T12:45:41.562 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-15T12:45:41.562 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 20 M 2024-03-15T12:45:41.563 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 31 k 2024-03-15T12:45:41.563 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 2.3 M 2024-03-15T12:45:41.563 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 248 k 2024-03-15T12:45:41.563 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 4.7 M 2024-03-15T12:45:41.563 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 17 M 2024-03-15T12:45:41.563 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 15 k 2024-03-15T12:45:41.563 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 25 k 2024-03-15T12:45:41.563 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-15T12:45:41.563 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-15T12:45:41.563 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-15T12:45:41.563 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-15T12:45:41.563 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-15T12:45:41.563 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-15T12:45:41.563 INFO:teuthology.orchestra.run.smithi184.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-15T12:45:41.564 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-15T12:45:41.564 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-15T12:45:41.564 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 163 k 2024-03-15T12:45:41.564 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-15T12:45:41.564 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-15T12:45:41.564 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-15T12:45:41.564 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-15T12:45:41.564 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-15T12:45:41.564 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 498 k 2024-03-15T12:45:41.564 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-15T12:45:41.564 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 5.1 M 2024-03-15T12:45:41.564 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-15T12:45:41.564 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-15T12:45:41.564 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-15T12:45:41.565 INFO:teuthology.orchestra.run.smithi184.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-15T12:45:41.565 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-15T12:45:41.565 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-15T12:45:41.565 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-15T12:45:41.565 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-15T12:45:41.565 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-15T12:45:41.565 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-15T12:45:41.565 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-15T12:45:41.565 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-15T12:45:41.565 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-15T12:45:41.565 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-15T12:45:41.565 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-15T12:45:41.565 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-15T12:45:41.566 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-15T12:45:41.566 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-15T12:45:41.566 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 45 k 2024-03-15T12:45:41.566 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 131 k 2024-03-15T12:45:41.566 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-15T12:45:41.566 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-15T12:45:41.566 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-15T12:45:41.566 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-15T12:45:41.566 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-15T12:45:41.566 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-15T12:45:41.566 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-15T12:45:41.566 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-15T12:45:41.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-15T12:45:41.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-15T12:45:41.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-15T12:45:41.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-15T12:45:41.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-15T12:45:41.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-15T12:45:41.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-15T12:45:41.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-15T12:45:41.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-15T12:45:41.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-15T12:45:41.567 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-15T12:45:41.568 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-15T12:45:41.568 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-15T12:45:41.568 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-15T12:45:41.568 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-15T12:45:41.568 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-15T12:45:41.568 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-15T12:45:41.568 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-15T12:45:41.568 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-15T12:45:41.568 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-15T12:45:41.568 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-15T12:45:41.568 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-15T12:45:41.569 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-15T12:45:41.569 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-15T12:45:41.569 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-15T12:45:41.569 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-15T12:45:41.569 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-15T12:45:41.569 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-15T12:45:41.569 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-15T12:45:41.569 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-15T12:45:41.569 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-15T12:45:41.569 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-15T12:45:41.569 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-15T12:45:41.569 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-15T12:45:41.569 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-15T12:45:41.569 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-15T12:45:41.570 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-15T12:45:41.570 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-15T12:45:41.570 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-15T12:45:41.570 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-15T12:45:41.570 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-15T12:45:41.570 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-15T12:45:41.570 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-15T12:45:41.570 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-15T12:45:41.570 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-15T12:45:41.570 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-15T12:45:41.570 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-15T12:45:41.570 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-15T12:45:41.571 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-15T12:45:41.571 INFO:teuthology.orchestra.run.smithi184.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-15T12:45:41.571 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-15T12:45:41.571 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-15T12:45:41.571 INFO:teuthology.orchestra.run.smithi184.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-15T12:45:41.571 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-15T12:45:41.571 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-15T12:45:41.571 INFO:teuthology.orchestra.run.smithi184.stdout:Installing weak dependencies: 2024-03-15T12:45:41.571 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-15T12:45:41.571 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:45:41.571 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-15T12:45:41.571 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================ 2024-03-15T12:45:41.571 INFO:teuthology.orchestra.run.smithi184.stdout:Install 150 Packages 2024-03-15T12:45:41.571 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 24 Packages 2024-03-15T12:45:41.572 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:45:41.572 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 224 M 2024-03-15T12:45:41.572 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-15T12:45:42.320 INFO:teuthology.orchestra.run.smithi184.stdout:(1/174): ceph-19.0.0-1681.gaba3c1d2.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-03-15T12:45:42.745 INFO:teuthology.orchestra.run.smithi184.stdout:(2/174): ceph-fuse-19.0.0-1681.gaba3c1d2.el9.x8 2.0 MB/s | 887 kB 00:00 2024-03-15T12:45:42.871 INFO:teuthology.orchestra.run.smithi184.stdout:(3/174): ceph-immutable-object-cache-19.0.0-168 1.1 MB/s | 145 kB 00:00 2024-03-15T12:45:43.104 INFO:teuthology.orchestra.run.smithi184.stdout:(4/174): ceph-base-19.0.0-1681.gaba3c1d2.el9.x8 5.3 MB/s | 5.3 MB 00:01 2024-03-15T12:45:43.172 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-15T12:45:43.173 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-15T12:45:43.185 INFO:teuthology.orchestra.run.smithi149.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-15T12:45:43.186 INFO:teuthology.orchestra.run.smithi149.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-15T12:45:43.187 INFO:teuthology.orchestra.run.smithi149.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-15T12:45:43.188 INFO:teuthology.orchestra.run.smithi149.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-15T12:45:43.190 INFO:teuthology.orchestra.run.smithi149.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-15T12:45:43.192 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-15T12:45:43.193 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-15T12:45:43.273 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-15T12:45:43.288 INFO:teuthology.orchestra.run.smithi184.stdout:(5/174): ceph-mgr-19.0.0-1681.gaba3c1d2.el9.x86 8.1 MB/s | 1.5 MB 00:00 2024-03-15T12:45:43.302 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================ 2024-03-15T12:45:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-03-15T12:45:43.303 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================ 2024-03-15T12:45:43.303 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-15T12:45:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-15T12:45:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 6.6 k 2024-03-15T12:45:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 5.3 M 2024-03-15T12:45:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 887 k 2024-03-15T12:45:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 145 k 2024-03-15T12:45:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 1.5 M 2024-03-15T12:45:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 138 k 2024-03-15T12:45:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 1.8 M 2024-03-15T12:45:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 7.4 M 2024-03-15T12:45:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 49 k 2024-03-15T12:45:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 9.8 M 2024-03-15T12:45:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 47 M 2024-03-15T12:45:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 264 k 2024-03-15T12:45:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 748 k 2024-03-15T12:45:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 218 k 2024-03-15T12:45:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 27 k 2024-03-15T12:45:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 18 k 2024-03-15T12:45:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-15T12:45:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-15T12:45:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-15T12:45:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 31 k 2024-03-15T12:45:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 744 k 2024-03-15T12:45:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 127 k 2024-03-15T12:45:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-15T12:45:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-15T12:45:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 161 k 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 321 k 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 298 k 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 99 k 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 85 k 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 3.1 M 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 171 k 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-15T12:45:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-15T12:45:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-15T12:45:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-15T12:45:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-15T12:45:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-15T12:45:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-15T12:45:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-15T12:45:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-15T12:45:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-15T12:45:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-15T12:45:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 3.4 M 2024-03-15T12:45:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 3.1 M 2024-03-15T12:45:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-15T12:45:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-15T12:45:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-15T12:45:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-15T12:45:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-15T12:45:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-15T12:45:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-15T12:45:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-15T12:45:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-15T12:45:43.307 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-03-15T12:45:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-15T12:45:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-15T12:45:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-15T12:45:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 20 M 2024-03-15T12:45:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 31 k 2024-03-15T12:45:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 2.3 M 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 248 k 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 4.7 M 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 17 M 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 15 k 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 25 k 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 163 k 2024-03-15T12:45:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 498 k 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 5.1 M 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-15T12:45:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-15T12:45:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-15T12:45:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-15T12:45:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-15T12:45:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-15T12:45:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-15T12:45:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-15T12:45:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-15T12:45:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-15T12:45:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-15T12:45:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 45 k 2024-03-15T12:45:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 131 k 2024-03-15T12:45:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-15T12:45:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-15T12:45:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-15T12:45:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-15T12:45:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-15T12:45:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-15T12:45:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-15T12:45:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-15T12:45:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-15T12:45:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-15T12:45:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-15T12:45:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-15T12:45:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-15T12:45:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-15T12:45:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-15T12:45:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-15T12:45:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-15T12:45:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-15T12:45:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-15T12:45:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-15T12:45:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-15T12:45:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-15T12:45:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-15T12:45:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-15T12:45:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-15T12:45:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-15T12:45:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-15T12:45:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-15T12:45:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-15T12:45:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-15T12:45:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-15T12:45:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-15T12:45:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-15T12:45:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-15T12:45:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-15T12:45:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-15T12:45:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-15T12:45:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-15T12:45:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-15T12:45:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-15T12:45:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-15T12:45:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-15T12:45:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-15T12:45:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-15T12:45:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-15T12:45:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-15T12:45:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-15T12:45:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-15T12:45:43.315 INFO:teuthology.orchestra.run.smithi149.stdout:Installing weak dependencies: 2024-03-15T12:45:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-15T12:45:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:45:43.315 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-15T12:45:43.315 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================ 2024-03-15T12:45:43.315 INFO:teuthology.orchestra.run.smithi149.stdout:Install 150 Packages 2024-03-15T12:45:43.315 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 24 Packages 2024-03-15T12:45:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:45:43.315 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 224 M 2024-03-15T12:45:43.316 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-15T12:45:43.622 INFO:teuthology.orchestra.run.smithi184.stdout:(6/174): ceph-mds-19.0.0-1681.gaba3c1d2.el9.x86 3.1 MB/s | 2.3 MB 00:00 2024-03-15T12:45:43.780 INFO:teuthology.orchestra.run.smithi184.stdout:(7/174): ceph-mon-19.0.0-1681.gaba3c1d2.el9.x86 9.5 MB/s | 4.7 MB 00:00 2024-03-15T12:45:44.261 INFO:teuthology.orchestra.run.smithi149.stdout:(1/174): ceph-19.0.0-1681.gaba3c1d2.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-03-15T12:45:44.381 INFO:teuthology.orchestra.run.smithi184.stdout:(8/174): ceph-common-19.0.0-1681.gaba3c1d2.el9. 9.0 MB/s | 20 MB 00:02 2024-03-15T12:45:44.456 INFO:teuthology.orchestra.run.smithi184.stdout:(9/174): ceph-selinux-19.0.0-1681.gaba3c1d2.el9 336 kB/s | 25 kB 00:00 2024-03-15T12:45:44.650 INFO:teuthology.orchestra.run.smithi184.stdout:(10/174): ceph-radosgw-19.0.0-1681.gaba3c1d2.el 11 MB/s | 9.8 MB 00:00 2024-03-15T12:45:44.756 INFO:teuthology.orchestra.run.smithi184.stdout:(11/174): cephfs-mirror-19.0.0-1681.gaba3c1d2.e 2.0 MB/s | 218 kB 00:00 2024-03-15T12:45:44.848 INFO:teuthology.orchestra.run.smithi184.stdout:(12/174): cephfs-shell-19.0.0-1681.gaba3c1d2.el 295 kB/s | 27 kB 00:00 2024-03-15T12:45:44.932 INFO:teuthology.orchestra.run.smithi184.stdout:(13/174): libcephfs-devel-19.0.0-1681.gaba3c1d2 376 kB/s | 31 kB 00:00 2024-03-15T12:45:45.170 INFO:teuthology.orchestra.run.smithi149.stdout:(2/174): ceph-fuse-19.0.0-1681.gaba3c1d2.el9.x8 977 kB/s | 887 kB 00:00 2024-03-15T12:45:45.345 INFO:teuthology.orchestra.run.smithi149.stdout:(3/174): ceph-immutable-object-cache-19.0.0-168 832 kB/s | 145 kB 00:00 2024-03-15T12:45:45.512 INFO:teuthology.orchestra.run.smithi149.stdout:(4/174): ceph-base-19.0.0-1681.gaba3c1d2.el9.x8 3.4 MB/s | 5.3 MB 00:01 2024-03-15T12:45:45.887 INFO:teuthology.orchestra.run.smithi149.stdout:(5/174): ceph-mgr-19.0.0-1681.gaba3c1d2.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-03-15T12:45:45.916 INFO:teuthology.orchestra.run.smithi184.stdout:(14/174): ceph-osd-19.0.0-1681.gaba3c1d2.el9.x8 7.3 MB/s | 17 MB 00:02 2024-03-15T12:45:45.941 INFO:teuthology.orchestra.run.smithi184.stdout:(15/174): libcephfs2-19.0.0-1681.gaba3c1d2.el9. 738 kB/s | 744 kB 00:01 2024-03-15T12:45:45.962 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-03-15T12:45:45.991 INFO:teuthology.orchestra.run.smithi184.stdout:(16/174): libcephsqlite-19.0.0-1681.gaba3c1d2.e 2.1 MB/s | 163 kB 00:00 2024-03-15T12:45:46.074 INFO:teuthology.orchestra.run.smithi184.stdout:(17/174): libradosstriper1-19.0.0-1681.gaba3c1d 5.8 MB/s | 498 kB 00:00 2024-03-15T12:45:46.150 INFO:teuthology.orchestra.run.smithi184.stdout:(18/174): librados-devel-19.0.0-1681.gaba3c1d2. 606 kB/s | 127 kB 00:00 2024-03-15T12:45:46.258 INFO:teuthology.orchestra.run.smithi184.stdout:(19/174): python3-ceph-argparse-19.0.0-1681.gab 417 kB/s | 45 kB 00:00 2024-03-15T12:45:46.475 INFO:teuthology.orchestra.run.smithi184.stdout:(20/174): python3-ceph-common-19.0.0-1681.gaba3 606 kB/s | 131 kB 00:00 2024-03-15T12:45:46.487 INFO:teuthology.orchestra.run.smithi149.stdout:(6/174): ceph-mds-19.0.0-1681.gaba3c1d2.el9.x86 2.0 MB/s | 2.3 MB 00:01 2024-03-15T12:45:46.576 INFO:teuthology.orchestra.run.smithi184.stdout:(21/174): librgw2-19.0.0-1681.gaba3c1d2.el9.x86 10 MB/s | 5.1 MB 00:00 2024-03-15T12:45:46.643 INFO:teuthology.orchestra.run.smithi184.stdout:(22/174): python3-cephfs-19.0.0-1681.gaba3c1d2. 968 kB/s | 161 kB 00:00 2024-03-15T12:45:46.668 INFO:teuthology.orchestra.run.smithi184.stdout:(23/174): python3-rados-19.0.0-1681.gaba3c1d2.e 3.4 MB/s | 321 kB 00:00 2024-03-15T12:45:46.744 INFO:teuthology.orchestra.run.smithi184.stdout:(24/174): python3-rgw-19.0.0-1681.gaba3c1d2.el9 1.3 MB/s | 99 kB 00:00 2024-03-15T12:45:46.787 INFO:teuthology.orchestra.run.smithi149.stdout:(7/174): ceph-mon-19.0.0-1681.gaba3c1d2.el9.x86 5.2 MB/s | 4.7 MB 00:00 2024-03-15T12:45:46.810 INFO:teuthology.orchestra.run.smithi184.stdout:(25/174): rbd-fuse-19.0.0-1681.gaba3c1d2.el9.x8 1.2 MB/s | 85 kB 00:00 2024-03-15T12:45:46.852 INFO:teuthology.orchestra.run.smithi184.stdout:(26/174): python3-rbd-19.0.0-1681.gaba3c1d2.el9 1.4 MB/s | 298 kB 00:00 2024-03-15T12:45:47.044 INFO:teuthology.orchestra.run.smithi184.stdout:(27/174): rbd-nbd-19.0.0-1681.gaba3c1d2.el9.x86 890 kB/s | 171 kB 00:00 2024-03-15T12:45:47.194 INFO:teuthology.orchestra.run.smithi184.stdout:(28/174): rbd-mirror-19.0.0-1681.gaba3c1d2.el9. 8.0 MB/s | 3.1 MB 00:00 2024-03-15T12:45:47.220 INFO:teuthology.orchestra.run.smithi184.stdout:(29/174): ceph-grafana-dashboards-19.0.0-1681.g 179 kB/s | 31 kB 00:00 2024-03-15T12:45:47.287 INFO:teuthology.orchestra.run.smithi184.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1681.gaba3c1d 1.5 MB/s | 138 kB 00:00 2024-03-15T12:45:47.987 INFO:teuthology.orchestra.run.smithi184.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1681.gaba3c 2.3 MB/s | 1.8 MB 00:00 2024-03-15T12:45:48.071 INFO:teuthology.orchestra.run.smithi184.stdout:(32/174): ceph-mgr-diskprediction-local-19.0.0- 9.5 MB/s | 7.4 MB 00:00 2024-03-15T12:45:48.104 INFO:teuthology.orchestra.run.smithi184.stdout:(33/174): ceph-mgr-modules-core-19.0.0-1681.gab 2.1 MB/s | 248 kB 00:00 2024-03-15T12:45:48.130 INFO:teuthology.orchestra.run.smithi184.stdout:(34/174): ceph-mgr-rook-19.0.0-1681.gaba3c1d2.e 842 kB/s | 49 kB 00:00 2024-03-15T12:45:48.180 INFO:teuthology.orchestra.run.smithi184.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1681.ga 204 kB/s | 15 kB 00:00 2024-03-15T12:45:48.205 INFO:teuthology.orchestra.run.smithi184.stdout:(36/174): ceph-volume-19.0.0-1681.gaba3c1d2.el9 3.4 MB/s | 264 kB 00:00 2024-03-15T12:45:48.306 INFO:teuthology.orchestra.run.smithi184.stdout:(37/174): cephfs-top-19.0.0-1681.gaba3c1d2.el9. 182 kB/s | 18 kB 00:00 2024-03-15T12:45:48.389 INFO:teuthology.orchestra.run.smithi184.stdout:(38/174): cephadm-19.0.0-1681.gaba3c1d2.el9.noa 3.5 MB/s | 748 kB 00:00 2024-03-15T12:45:48.523 INFO:teuthology.orchestra.run.smithi184.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 662 kB 00:00 2024-03-15T12:45:48.560 INFO:teuthology.orchestra.run.smithi002.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-15T12:45:48.561 INFO:teuthology.orchestra.run.smithi002.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-15T12:45:48.573 INFO:teuthology.orchestra.run.smithi184.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.9 MB/s | 920 kB 00:00 2024-03-15T12:45:48.576 INFO:teuthology.orchestra.run.smithi002.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-15T12:45:48.577 INFO:teuthology.orchestra.run.smithi002.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-15T12:45:48.578 INFO:teuthology.orchestra.run.smithi002.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-15T12:45:48.579 INFO:teuthology.orchestra.run.smithi002.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-15T12:45:48.581 INFO:teuthology.orchestra.run.smithi002.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-15T12:45:48.584 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-15T12:45:48.585 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-15T12:45:48.599 INFO:teuthology.orchestra.run.smithi184.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 106 kB 00:00 2024-03-15T12:45:48.624 INFO:teuthology.orchestra.run.smithi184.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-15T12:45:48.641 INFO:teuthology.orchestra.run.smithi184.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 762 kB/s | 32 kB 00:00 2024-03-15T12:45:48.672 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-15T12:45:48.692 INFO:teuthology.orchestra.run.smithi002.stdout:============================================================================================ 2024-03-15T12:45:48.692 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-03-15T12:45:48.692 INFO:teuthology.orchestra.run.smithi002.stdout:============================================================================================ 2024-03-15T12:45:48.692 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-15T12:45:48.692 INFO:teuthology.orchestra.run.smithi002.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-15T12:45:48.693 INFO:teuthology.orchestra.run.smithi002.stdout: ceph x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 6.6 k 2024-03-15T12:45:48.693 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 5.3 M 2024-03-15T12:45:48.693 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 887 k 2024-03-15T12:45:48.693 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 145 k 2024-03-15T12:45:48.693 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 1.5 M 2024-03-15T12:45:48.693 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 138 k 2024-03-15T12:45:48.693 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 1.8 M 2024-03-15T12:45:48.693 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 7.4 M 2024-03-15T12:45:48.693 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 49 k 2024-03-15T12:45:48.693 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 9.8 M 2024-03-15T12:45:48.693 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 47 M 2024-03-15T12:45:48.693 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 264 k 2024-03-15T12:45:48.693 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 748 k 2024-03-15T12:45:48.694 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 218 k 2024-03-15T12:45:48.694 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-shell x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 27 k 2024-03-15T12:45:48.694 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 18 k 2024-03-15T12:45:48.694 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-15T12:45:48.694 INFO:teuthology.orchestra.run.smithi002.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-15T12:45:48.694 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-15T12:45:48.694 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 31 k 2024-03-15T12:45:48.694 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 744 k 2024-03-15T12:45:48.694 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 127 k 2024-03-15T12:45:48.694 INFO:teuthology.orchestra.run.smithi002.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-15T12:45:48.694 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-15T12:45:48.694 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-15T12:45:48.695 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-15T12:45:48.695 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-15T12:45:48.695 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-15T12:45:48.695 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 161 k 2024-03-15T12:45:48.695 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 321 k 2024-03-15T12:45:48.695 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 298 k 2024-03-15T12:45:48.695 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 99 k 2024-03-15T12:45:48.695 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 85 k 2024-03-15T12:45:48.695 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 3.1 M 2024-03-15T12:45:48.695 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 171 k 2024-03-15T12:45:48.695 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-15T12:45:48.695 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrading: 2024-03-15T12:45:48.695 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-15T12:45:48.695 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-15T12:45:48.696 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-15T12:45:48.696 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-15T12:45:48.696 INFO:teuthology.orchestra.run.smithi002.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-15T12:45:48.696 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-15T12:45:48.696 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-15T12:45:48.696 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-15T12:45:48.696 INFO:teuthology.orchestra.run.smithi002.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-15T12:45:48.696 INFO:teuthology.orchestra.run.smithi002.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-15T12:45:48.696 INFO:teuthology.orchestra.run.smithi002.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-15T12:45:48.696 INFO:teuthology.orchestra.run.smithi002.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-15T12:45:48.696 INFO:teuthology.orchestra.run.smithi002.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-15T12:45:48.696 INFO:teuthology.orchestra.run.smithi002.stdout: librados2 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 3.4 M 2024-03-15T12:45:48.697 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 3.1 M 2024-03-15T12:45:48.697 INFO:teuthology.orchestra.run.smithi002.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-15T12:45:48.697 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-15T12:45:48.697 INFO:teuthology.orchestra.run.smithi002.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-15T12:45:48.697 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-15T12:45:48.697 INFO:teuthology.orchestra.run.smithi002.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-15T12:45:48.697 INFO:teuthology.orchestra.run.smithi002.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-15T12:45:48.697 INFO:teuthology.orchestra.run.smithi002.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-15T12:45:48.697 INFO:teuthology.orchestra.run.smithi002.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-15T12:45:48.697 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-15T12:45:48.697 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-03-15T12:45:48.697 INFO:teuthology.orchestra.run.smithi002.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-15T12:45:48.697 INFO:teuthology.orchestra.run.smithi002.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-15T12:45:48.698 INFO:teuthology.orchestra.run.smithi002.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-15T12:45:48.698 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 20 M 2024-03-15T12:45:48.698 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 31 k 2024-03-15T12:45:48.698 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 2.3 M 2024-03-15T12:45:48.698 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 248 k 2024-03-15T12:45:48.698 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 4.7 M 2024-03-15T12:45:48.698 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 17 M 2024-03-15T12:45:48.698 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 15 k 2024-03-15T12:45:48.698 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 25 k 2024-03-15T12:45:48.698 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-15T12:45:48.699 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-15T12:45:48.699 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-15T12:45:48.699 INFO:teuthology.orchestra.run.smithi002.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-15T12:45:48.699 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-15T12:45:48.699 INFO:teuthology.orchestra.run.smithi002.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-15T12:45:48.699 INFO:teuthology.orchestra.run.smithi002.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-15T12:45:48.699 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-15T12:45:48.699 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-15T12:45:48.699 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 163 k 2024-03-15T12:45:48.699 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-15T12:45:48.699 INFO:teuthology.orchestra.run.smithi002.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-15T12:45:48.699 INFO:teuthology.orchestra.run.smithi002.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-15T12:45:48.700 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-15T12:45:48.700 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-15T12:45:48.700 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 498 k 2024-03-15T12:45:48.700 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-15T12:45:48.700 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 5.1 M 2024-03-15T12:45:48.700 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-15T12:45:48.700 INFO:teuthology.orchestra.run.smithi002.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-15T12:45:48.700 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-15T12:45:48.700 INFO:teuthology.orchestra.run.smithi002.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-15T12:45:48.700 INFO:teuthology.orchestra.run.smithi002.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-15T12:45:48.700 INFO:teuthology.orchestra.run.smithi002.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-15T12:45:48.701 INFO:teuthology.orchestra.run.smithi002.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-15T12:45:48.701 INFO:teuthology.orchestra.run.smithi002.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-15T12:45:48.701 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-15T12:45:48.701 INFO:teuthology.orchestra.run.smithi002.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-15T12:45:48.701 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-15T12:45:48.701 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-15T12:45:48.701 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-15T12:45:48.701 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-15T12:45:48.701 INFO:teuthology.orchestra.run.smithi002.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-15T12:45:48.701 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-15T12:45:48.701 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-15T12:45:48.701 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-15T12:45:48.701 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 45 k 2024-03-15T12:45:48.701 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 131 k 2024-03-15T12:45:48.702 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-15T12:45:48.702 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-15T12:45:48.702 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-15T12:45:48.702 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-15T12:45:48.702 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-15T12:45:48.702 INFO:teuthology.orchestra.run.smithi002.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-15T12:45:48.702 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-15T12:45:48.702 INFO:teuthology.orchestra.run.smithi002.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-15T12:45:48.702 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-15T12:45:48.702 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-15T12:45:48.702 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-15T12:45:48.702 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-15T12:45:48.703 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-15T12:45:48.703 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-15T12:45:48.703 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-15T12:45:48.703 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-15T12:45:48.703 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-15T12:45:48.703 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-15T12:45:48.703 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-15T12:45:48.703 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-15T12:45:48.703 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-15T12:45:48.703 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-15T12:45:48.704 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-15T12:45:48.704 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-15T12:45:48.704 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-15T12:45:48.704 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-15T12:45:48.704 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-15T12:45:48.704 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-15T12:45:48.704 INFO:teuthology.orchestra.run.smithi002.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-15T12:45:48.704 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-15T12:45:48.704 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-15T12:45:48.704 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-15T12:45:48.704 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-15T12:45:48.704 INFO:teuthology.orchestra.run.smithi002.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-15T12:45:48.704 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-15T12:45:48.705 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-15T12:45:48.705 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-15T12:45:48.705 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-15T12:45:48.705 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-15T12:45:48.705 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-15T12:45:48.705 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-15T12:45:48.705 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-15T12:45:48.705 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-15T12:45:48.705 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-15T12:45:48.705 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-15T12:45:48.705 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-15T12:45:48.705 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-15T12:45:48.705 INFO:teuthology.orchestra.run.smithi002.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-15T12:45:48.706 INFO:teuthology.orchestra.run.smithi002.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-15T12:45:48.706 INFO:teuthology.orchestra.run.smithi002.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-15T12:45:48.706 INFO:teuthology.orchestra.run.smithi002.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-15T12:45:48.706 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-15T12:45:48.706 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-15T12:45:48.706 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-15T12:45:48.706 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-15T12:45:48.706 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-15T12:45:48.706 INFO:teuthology.orchestra.run.smithi002.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-15T12:45:48.706 INFO:teuthology.orchestra.run.smithi002.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-15T12:45:48.706 INFO:teuthology.orchestra.run.smithi002.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-15T12:45:48.706 INFO:teuthology.orchestra.run.smithi002.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-15T12:45:48.706 INFO:teuthology.orchestra.run.smithi002.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-15T12:45:48.707 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-15T12:45:48.707 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-15T12:45:48.707 INFO:teuthology.orchestra.run.smithi002.stdout:Installing weak dependencies: 2024-03-15T12:45:48.707 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-15T12:45:48.707 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:45:48.707 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-15T12:45:48.707 INFO:teuthology.orchestra.run.smithi002.stdout:============================================================================================ 2024-03-15T12:45:48.707 INFO:teuthology.orchestra.run.smithi002.stdout:Install 150 Packages 2024-03-15T12:45:48.707 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrade 24 Packages 2024-03-15T12:45:48.707 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:45:48.708 INFO:teuthology.orchestra.run.smithi184.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 293 kB/s | 17 kB 00:00 2024-03-15T12:45:48.708 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 224 M 2024-03-15T12:45:48.708 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-15T12:45:48.767 INFO:teuthology.orchestra.run.smithi184.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-03-15T12:45:48.783 INFO:teuthology.orchestra.run.smithi184.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 120 kB/s | 10 kB 00:00 2024-03-15T12:45:48.804 INFO:teuthology.orchestra.run.smithi149.stdout:(8/174): ceph-radosgw-19.0.0-1681.gaba3c1d2.el9 4.8 MB/s | 9.8 MB 00:02 2024-03-15T12:45:48.809 INFO:teuthology.orchestra.run.smithi184.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-15T12:45:48.834 INFO:teuthology.orchestra.run.smithi184.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 882 kB/s | 44 kB 00:00 2024-03-15T12:45:48.855 INFO:teuthology.orchestra.run.smithi149.stdout:(9/174): ceph-selinux-19.0.0-1681.gaba3c1d2.el9 505 kB/s | 25 kB 00:00 2024-03-15T12:45:48.859 INFO:teuthology.orchestra.run.smithi184.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-15T12:45:48.893 INFO:teuthology.orchestra.run.smithi184.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-03-15T12:45:48.918 INFO:teuthology.orchestra.run.smithi184.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 788 kB/s | 46 kB 00:00 2024-03-15T12:45:48.969 INFO:teuthology.orchestra.run.smithi184.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-15T12:45:48.994 INFO:teuthology.orchestra.run.smithi184.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-15T12:45:49.019 INFO:teuthology.orchestra.run.smithi184.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-15T12:45:49.044 INFO:teuthology.orchestra.run.smithi184.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 682 kB/s | 34 kB 00:00 2024-03-15T12:45:49.178 INFO:teuthology.orchestra.run.smithi184.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-03-15T12:45:49.212 INFO:teuthology.orchestra.run.smithi184.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 9.2 MB/s | 1.5 MB 00:00 2024-03-15T12:45:49.237 INFO:teuthology.orchestra.run.smithi184.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 14 MB/s | 811 kB 00:00 2024-03-15T12:45:49.263 INFO:teuthology.orchestra.run.smithi184.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.6 MB/s | 134 kB 00:00 2024-03-15T12:45:49.305 INFO:teuthology.orchestra.run.smithi002.stdout:(1/174): ceph-19.0.0-1681.gaba3c1d2.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-03-15T12:45:49.413 INFO:teuthology.orchestra.run.smithi184.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 32 MB/s | 5.7 MB 00:00 2024-03-15T12:45:49.446 INFO:teuthology.orchestra.run.smithi184.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2024-03-15T12:45:49.472 INFO:teuthology.orchestra.run.smithi184.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-15T12:45:49.622 INFO:teuthology.orchestra.run.smithi184.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 5.8 MB 00:00 2024-03-15T12:45:49.647 INFO:teuthology.orchestra.run.smithi184.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 140 kB/s | 24 kB 00:00 2024-03-15T12:45:49.673 INFO:teuthology.orchestra.run.smithi184.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2024-03-15T12:45:49.688 INFO:teuthology.orchestra.run.smithi149.stdout:(10/174): ceph-osd-19.0.0-1681.gaba3c1d2.el9.x8 5.2 MB/s | 17 MB 00:03 2024-03-15T12:45:49.698 INFO:teuthology.orchestra.run.smithi184.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-15T12:45:49.723 INFO:teuthology.orchestra.run.smithi184.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-03-15T12:45:49.771 INFO:teuthology.orchestra.run.smithi149.stdout:(11/174): cephfs-mirror-19.0.0-1681.gaba3c1d2.e 2.6 MB/s | 218 kB 00:00 2024-03-15T12:45:49.838 INFO:teuthology.orchestra.run.smithi149.stdout:(12/174): cephfs-shell-19.0.0-1681.gaba3c1d2.el 407 kB/s | 27 kB 00:00 2024-03-15T12:45:49.897 INFO:teuthology.orchestra.run.smithi149.stdout:(13/174): libcephfs-devel-19.0.0-1681.gaba3c1d2 536 kB/s | 31 kB 00:00 2024-03-15T12:45:50.021 INFO:teuthology.orchestra.run.smithi002.stdout:(2/174): ceph-fuse-19.0.0-1681.gaba3c1d2.el9.x8 1.2 MB/s | 887 kB 00:00 2024-03-15T12:45:50.047 INFO:teuthology.orchestra.run.smithi149.stdout:(14/174): libcephfs2-19.0.0-1681.gaba3c1d2.el9. 4.8 MB/s | 744 kB 00:00 2024-03-15T12:45:50.131 INFO:teuthology.orchestra.run.smithi149.stdout:(15/174): libcephsqlite-19.0.0-1681.gaba3c1d2.e 1.9 MB/s | 163 kB 00:00 2024-03-15T12:45:50.147 INFO:teuthology.orchestra.run.smithi002.stdout:(3/174): ceph-immutable-object-cache-19.0.0-168 1.1 MB/s | 145 kB 00:00 2024-03-15T12:45:50.182 INFO:teuthology.orchestra.run.smithi184.stdout:(68/174): ceph-test-19.0.0-1681.gaba3c1d2.el9.x 8.3 MB/s | 47 MB 00:05 2024-03-15T12:45:50.189 INFO:teuthology.orchestra.run.smithi149.stdout:(16/174): librados-devel-19.0.0-1681.gaba3c1d2. 2.1 MB/s | 127 kB 00:00 2024-03-15T12:45:50.257 INFO:teuthology.orchestra.run.smithi184.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.5 MB/s | 4.8 MB 00:00 2024-03-15T12:45:50.282 INFO:teuthology.orchestra.run.smithi184.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 390 kB/s | 217 kB 00:00 2024-03-15T12:45:50.308 INFO:teuthology.orchestra.run.smithi184.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-03-15T12:45:50.333 INFO:teuthology.orchestra.run.smithi184.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.4 MB/s | 69 kB 00:00 2024-03-15T12:45:50.340 INFO:teuthology.orchestra.run.smithi149.stdout:(17/174): libradosstriper1-19.0.0-1681.gaba3c1d 3.2 MB/s | 498 kB 00:00 2024-03-15T12:45:50.358 INFO:teuthology.orchestra.run.smithi184.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 551 kB/s | 41 kB 00:00 2024-03-15T12:45:50.392 INFO:teuthology.orchestra.run.smithi184.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-03-15T12:45:50.425 INFO:teuthology.orchestra.run.smithi184.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2024-03-15T12:45:50.492 INFO:teuthology.orchestra.run.smithi184.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 211 kB 00:00 2024-03-15T12:45:50.509 INFO:teuthology.orchestra.run.smithi184.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 406 kB/s | 47 kB 00:00 2024-03-15T12:45:50.526 INFO:teuthology.orchestra.run.smithi184.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-03-15T12:45:50.652 INFO:teuthology.orchestra.run.smithi184.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 273 kB/s | 39 kB 00:00 2024-03-15T12:45:50.760 INFO:teuthology.orchestra.run.smithi184.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 177 kB/s | 41 kB 00:00 2024-03-15T12:45:50.919 INFO:teuthology.orchestra.run.smithi184.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-03-15T12:45:50.952 INFO:teuthology.orchestra.run.smithi184.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 300 kB 00:00 2024-03-15T12:45:50.977 INFO:teuthology.orchestra.run.smithi184.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 263 kB/s | 56 kB 00:00 2024-03-15T12:45:51.003 INFO:teuthology.orchestra.run.smithi184.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-15T12:45:51.020 INFO:teuthology.orchestra.run.smithi184.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 663 kB/s | 44 kB 00:00 2024-03-15T12:45:51.045 INFO:teuthology.orchestra.run.smithi184.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2024-03-15T12:45:51.070 INFO:teuthology.orchestra.run.smithi184.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-15T12:45:51.104 INFO:teuthology.orchestra.run.smithi184.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-15T12:45:51.129 INFO:teuthology.orchestra.run.smithi184.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-15T12:45:51.163 INFO:teuthology.orchestra.run.smithi184.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-15T12:45:51.182 INFO:teuthology.orchestra.run.smithi149.stdout:(18/174): ceph-common-19.0.0-1681.gaba3c1d2.el9 2.8 MB/s | 20 MB 00:07 2024-03-15T12:45:51.188 INFO:teuthology.orchestra.run.smithi184.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 187 kB 00:00 2024-03-15T12:45:51.282 INFO:teuthology.orchestra.run.smithi149.stdout:(19/174): python3-ceph-argparse-19.0.0-1681.gab 452 kB/s | 45 kB 00:00 2024-03-15T12:45:51.372 INFO:teuthology.orchestra.run.smithi184.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 774 kB/s | 187 kB 00:00 2024-03-15T12:45:51.397 INFO:teuthology.orchestra.run.smithi002.stdout:(4/174): ceph-common-19.0.0-1681.gaba3c1d2.el9. 8.6 MB/s | 20 MB 00:02 2024-03-15T12:45:51.407 INFO:teuthology.orchestra.run.smithi149.stdout:(20/174): librgw2-19.0.0-1681.gaba3c1d2.el9.x86 4.8 MB/s | 5.1 MB 00:01 2024-03-15T12:45:51.422 INFO:teuthology.orchestra.run.smithi184.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 836 kB/s | 195 kB 00:00 2024-03-15T12:45:51.424 INFO:teuthology.orchestra.run.smithi149.stdout:(21/174): python3-ceph-common-19.0.0-1681.gaba3 926 kB/s | 131 kB 00:00 2024-03-15T12:45:51.447 INFO:teuthology.orchestra.run.smithi002.stdout:(5/174): ceph-mds-19.0.0-1681.gaba3c1d2.el9.x86 1.8 MB/s | 2.3 MB 00:01 2024-03-15T12:45:51.448 INFO:teuthology.orchestra.run.smithi184.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-03-15T12:45:51.481 INFO:teuthology.orchestra.run.smithi184.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-03-15T12:45:51.516 INFO:teuthology.orchestra.run.smithi149.stdout:(22/174): python3-cephfs-19.0.0-1681.gaba3c1d2. 1.5 MB/s | 161 kB 00:00 2024-03-15T12:45:51.530 INFO:teuthology.orchestra.run.smithi002.stdout:(6/174): ceph-mgr-19.0.0-1681.gaba3c1d2.el9.x86 11 MB/s | 1.5 MB 00:00 2024-03-15T12:45:51.583 INFO:teuthology.orchestra.run.smithi149.stdout:(23/174): python3-rados-19.0.0-1681.gaba3c1d2.e 2.0 MB/s | 321 kB 00:00 2024-03-15T12:45:51.597 INFO:teuthology.orchestra.run.smithi002.stdout:(7/174): ceph-base-19.0.0-1681.gaba3c1d2.el9.x8 2.1 MB/s | 5.3 MB 00:02 2024-03-15T12:45:51.608 INFO:teuthology.orchestra.run.smithi149.stdout:(24/174): python3-rbd-19.0.0-1681.gaba3c1d2.el9 3.2 MB/s | 298 kB 00:00 2024-03-15T12:45:51.650 INFO:teuthology.orchestra.run.smithi149.stdout:(25/174): python3-rgw-19.0.0-1681.gaba3c1d2.el9 1.5 MB/s | 99 kB 00:00 2024-03-15T12:45:51.706 INFO:teuthology.orchestra.run.smithi184.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 76 kB/s | 21 kB 00:00 2024-03-15T12:45:51.717 INFO:teuthology.orchestra.run.smithi149.stdout:(26/174): rbd-fuse-19.0.0-1681.gaba3c1d2.el9.x8 785 kB/s | 85 kB 00:00 2024-03-15T12:45:51.732 INFO:teuthology.orchestra.run.smithi184.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 90 kB/s | 25 kB 00:00 2024-03-15T12:45:52.074 INFO:teuthology.orchestra.run.smithi184.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 669 kB/s | 245 kB 00:00 2024-03-15T12:45:52.174 INFO:teuthology.orchestra.run.smithi184.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 561 kB/s | 248 kB 00:00 2024-03-15T12:45:52.199 INFO:teuthology.orchestra.run.smithi184.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.7 MB/s | 216 kB 00:00 2024-03-15T12:45:52.283 INFO:teuthology.orchestra.run.smithi184.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 322 kB/s | 35 kB 00:00 2024-03-15T12:45:52.308 INFO:teuthology.orchestra.run.smithi184.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 410 kB/s | 44 kB 00:00 2024-03-15T12:45:52.392 INFO:teuthology.orchestra.run.smithi184.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-03-15T12:45:52.417 INFO:teuthology.orchestra.run.smithi184.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 701 kB/s | 94 kB 00:00 2024-03-15T12:45:52.542 INFO:teuthology.orchestra.run.smithi184.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-03-15T12:45:52.559 INFO:teuthology.orchestra.run.smithi149.stdout:(27/174): rbd-nbd-19.0.0-1681.gaba3c1d2.el9.x86 203 kB/s | 171 kB 00:00 2024-03-15T12:45:52.667 INFO:teuthology.orchestra.run.smithi149.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1681.g 290 kB/s | 31 kB 00:00 2024-03-15T12:45:52.701 INFO:teuthology.orchestra.run.smithi184.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.4 MB/s | 4.1 MB 00:01 2024-03-15T12:45:52.743 INFO:teuthology.orchestra.run.smithi149.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1681.gaba3c1d 1.8 MB/s | 138 kB 00:00 2024-03-15T12:45:52.764 INFO:teuthology.orchestra.run.smithi002.stdout:(8/174): ceph-osd-19.0.0-1681.gaba3c1d2.el9.x86 14 MB/s | 17 MB 00:01 2024-03-15T12:45:52.839 INFO:teuthology.orchestra.run.smithi002.stdout:(9/174): ceph-mon-19.0.0-1681.gaba3c1d2.el9.x86 3.4 MB/s | 4.7 MB 00:01 2024-03-15T12:45:52.851 INFO:teuthology.orchestra.run.smithi149.stdout:(30/174): rbd-mirror-19.0.0-1681.gaba3c1d2.el9. 2.6 MB/s | 3.1 MB 00:01 2024-03-15T12:45:52.865 INFO:teuthology.orchestra.run.smithi002.stdout:(10/174): ceph-selinux-19.0.0-1681.gaba3c1d2.el 252 kB/s | 25 kB 00:00 2024-03-15T12:45:52.940 INFO:teuthology.orchestra.run.smithi002.stdout:(11/174): cephfs-mirror-19.0.0-1681.gaba3c1d2.e 2.8 MB/s | 218 kB 00:00 2024-03-15T12:45:52.968 INFO:teuthology.orchestra.run.smithi184.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-03-15T12:45:53.007 INFO:teuthology.orchestra.run.smithi002.stdout:(12/174): cephfs-shell-19.0.0-1681.gaba3c1d2.el 406 kB/s | 27 kB 00:00 2024-03-15T12:45:53.074 INFO:teuthology.orchestra.run.smithi002.stdout:(13/174): libcephfs-devel-19.0.0-1681.gaba3c1d2 471 kB/s | 31 kB 00:00 2024-03-15T12:45:53.085 INFO:teuthology.orchestra.run.smithi184.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 821 kB/s | 548 kB 00:00 2024-03-15T12:45:53.152 INFO:teuthology.orchestra.run.smithi184.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 507 kB/s | 309 kB 00:00 2024-03-15T12:45:53.174 INFO:teuthology.orchestra.run.smithi002.stdout:(14/174): libcephfs2-19.0.0-1681.gaba3c1d2.el9. 7.3 MB/s | 744 kB 00:00 2024-03-15T12:45:53.177 INFO:teuthology.orchestra.run.smithi184.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 282 kB/s | 26 kB 00:00 2024-03-15T12:45:53.218 INFO:teuthology.orchestra.run.smithi149.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1681.gaba3c 3.7 MB/s | 1.8 MB 00:00 2024-03-15T12:45:53.249 INFO:teuthology.orchestra.run.smithi002.stdout:(15/174): libcephsqlite-19.0.0-1681.gaba3c1d2.e 2.1 MB/s | 163 kB 00:00 2024-03-15T12:45:53.260 INFO:teuthology.orchestra.run.smithi184.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 447 kB/s | 49 kB 00:00 2024-03-15T12:45:53.286 INFO:teuthology.orchestra.run.smithi184.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-03-15T12:45:53.319 INFO:teuthology.orchestra.run.smithi149.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1681.gab 2.4 MB/s | 248 kB 00:00 2024-03-15T12:45:53.333 INFO:teuthology.orchestra.run.smithi002.stdout:(16/174): librados-devel-19.0.0-1681.gaba3c1d2. 1.5 MB/s | 127 kB 00:00 2024-03-15T12:45:53.394 INFO:teuthology.orchestra.run.smithi149.stdout:(33/174): ceph-mgr-rook-19.0.0-1681.gaba3c1d2.e 657 kB/s | 49 kB 00:00 2024-03-15T12:45:53.417 INFO:teuthology.orchestra.run.smithi002.stdout:(17/174): libradosstriper1-19.0.0-1681.gaba3c1d 5.8 MB/s | 498 kB 00:00 2024-03-15T12:45:53.436 INFO:teuthology.orchestra.run.smithi184.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 862 kB/s | 151 kB 00:00 2024-03-15T12:45:53.469 INFO:teuthology.orchestra.run.smithi149.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1681.ga 204 kB/s | 15 kB 00:00 2024-03-15T12:45:53.666 INFO:teuthology.orchestra.run.smithi184.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-03-15T12:45:53.695 INFO:teuthology.orchestra.run.smithi184.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.0 MB/s | 838 kB 00:00 2024-03-15T12:45:53.779 INFO:teuthology.orchestra.run.smithi184.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 385 kB/s | 43 kB 00:00 2024-03-15T12:45:53.804 INFO:teuthology.orchestra.run.smithi184.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-03-15T12:45:53.888 INFO:teuthology.orchestra.run.smithi184.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-03-15T12:45:53.916 INFO:teuthology.orchestra.run.smithi002.stdout:(18/174): librgw2-19.0.0-1681.gaba3c1d2.el9.x86 10 MB/s | 5.1 MB 00:00 2024-03-15T12:45:53.921 INFO:teuthology.orchestra.run.smithi184.stdout:(119/174): python3-asyncssh-2.13.2-4.el9.noarch 1.1 MB/s | 548 kB 00:00 2024-03-15T12:45:53.947 INFO:teuthology.orchestra.run.smithi184.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-03-15T12:45:53.967 INFO:teuthology.orchestra.run.smithi002.stdout:(19/174): python3-ceph-argparse-19.0.0-1681.gab 901 kB/s | 45 kB 00:00 2024-03-15T12:45:53.997 INFO:teuthology.orchestra.run.smithi184.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-03-15T12:45:54.026 INFO:teuthology.orchestra.run.smithi002.stdout:(20/174): python3-ceph-common-19.0.0-1681.gaba3 2.2 MB/s | 131 kB 00:00 2024-03-15T12:45:54.047 INFO:teuthology.orchestra.run.smithi184.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-03-15T12:45:54.092 INFO:teuthology.orchestra.run.smithi002.stdout:(21/174): python3-cephfs-19.0.0-1681.gaba3c1d2. 2.4 MB/s | 161 kB 00:00 2024-03-15T12:45:54.113 INFO:teuthology.orchestra.run.smithi184.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-03-15T12:45:54.168 INFO:teuthology.orchestra.run.smithi002.stdout:(22/174): python3-rados-19.0.0-1681.gaba3c1d2.e 4.2 MB/s | 321 kB 00:00 2024-03-15T12:45:54.181 INFO:teuthology.orchestra.run.smithi184.stdout:(124/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.1 MB/s | 303 kB 00:00 2024-03-15T12:45:54.206 INFO:teuthology.orchestra.run.smithi149.stdout:(35/174): ceph-volume-19.0.0-1681.gaba3c1d2.el9 358 kB/s | 264 kB 00:00 2024-03-15T12:45:54.206 INFO:teuthology.orchestra.run.smithi184.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-03-15T12:45:54.235 INFO:teuthology.orchestra.run.smithi002.stdout:(23/174): python3-rbd-19.0.0-1681.gaba3c1d2.el9 4.4 MB/s | 298 kB 00:00 2024-03-15T12:45:54.282 INFO:teuthology.orchestra.run.smithi184.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-03-15T12:45:54.302 INFO:teuthology.orchestra.run.smithi002.stdout:(24/174): python3-rgw-19.0.0-1681.gaba3c1d2.el9 1.5 MB/s | 99 kB 00:00 2024-03-15T12:45:54.307 INFO:teuthology.orchestra.run.smithi184.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-03-15T12:45:54.382 INFO:teuthology.orchestra.run.smithi184.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-03-15T12:45:54.402 INFO:teuthology.orchestra.run.smithi002.stdout:(25/174): ceph-radosgw-19.0.0-1681.gaba3c1d2.el 3.5 MB/s | 9.8 MB 00:02 2024-03-15T12:45:54.408 INFO:teuthology.orchestra.run.smithi184.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-03-15T12:45:54.427 INFO:teuthology.orchestra.run.smithi002.stdout:(26/174): rbd-fuse-19.0.0-1681.gaba3c1d2.el9.x8 681 kB/s | 85 kB 00:00 2024-03-15T12:45:54.444 INFO:teuthology.orchestra.run.smithi149.stdout:(36/174): cephadm-19.0.0-1681.gaba3c1d2.el9.noa 3.1 MB/s | 748 kB 00:00 2024-03-15T12:45:54.486 INFO:teuthology.orchestra.run.smithi002.stdout:(27/174): rbd-nbd-19.0.0-1681.gaba3c1d2.el9.x86 2.9 MB/s | 171 kB 00:00 2024-03-15T12:45:54.491 INFO:teuthology.orchestra.run.smithi184.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-03-15T12:45:54.512 INFO:teuthology.orchestra.run.smithi149.stdout:(37/174): cephfs-top-19.0.0-1681.gaba3c1d2.el9. 272 kB/s | 18 kB 00:00 2024-03-15T12:45:54.525 INFO:teuthology.orchestra.run.smithi184.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-03-15T12:45:54.536 INFO:teuthology.orchestra.run.smithi002.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1681.g 630 kB/s | 31 kB 00:00 2024-03-15T12:45:54.592 INFO:teuthology.orchestra.run.smithi184.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 461 kB/s | 46 kB 00:00 2024-03-15T12:45:54.595 INFO:teuthology.orchestra.run.smithi002.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1681.gaba3c1d 2.3 MB/s | 138 kB 00:00 2024-03-15T12:45:54.625 INFO:teuthology.orchestra.run.smithi184.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 786 kB/s | 79 kB 00:00 2024-03-15T12:45:54.667 INFO:teuthology.orchestra.run.smithi184.stdout:(134/174): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-03-15T12:45:54.693 INFO:teuthology.orchestra.run.smithi184.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 575 kB/s | 58 kB 00:00 2024-03-15T12:45:54.704 INFO:teuthology.orchestra.run.smithi149.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 662 kB 00:00 2024-03-15T12:45:54.735 INFO:teuthology.orchestra.run.smithi184.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-03-15T12:45:54.768 INFO:teuthology.orchestra.run.smithi184.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-03-15T12:45:54.771 INFO:teuthology.orchestra.run.smithi149.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 13 MB/s | 920 kB 00:00 2024-03-15T12:45:54.793 INFO:teuthology.orchestra.run.smithi184.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 894 kB/s | 90 kB 00:00 2024-03-15T12:45:54.803 INFO:teuthology.orchestra.run.smithi002.stdout:(30/174): ceph-mgr-dashboard-19.0.0-1681.gaba3c 8.6 MB/s | 1.8 MB 00:00 2024-03-15T12:45:54.804 INFO:teuthology.orchestra.run.smithi149.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-03-15T12:45:54.836 INFO:teuthology.orchestra.run.smithi184.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 280 kB/s | 28 kB 00:00 2024-03-15T12:45:54.846 INFO:teuthology.orchestra.run.smithi149.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-15T12:45:54.869 INFO:teuthology.orchestra.run.smithi184.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-03-15T12:45:54.880 INFO:teuthology.orchestra.run.smithi149.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 952 kB/s | 32 kB 00:00 2024-03-15T12:45:54.944 INFO:teuthology.orchestra.run.smithi184.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-03-15T12:45:54.978 INFO:teuthology.orchestra.run.smithi184.stdout:(142/174): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-03-15T12:45:55.003 INFO:teuthology.orchestra.run.smithi184.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 267 kB/s | 36 kB 00:00 2024-03-15T12:45:55.005 INFO:teuthology.orchestra.run.smithi149.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-03-15T12:45:55.039 INFO:teuthology.orchestra.run.smithi149.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 513 kB/s | 17 kB 00:00 2024-03-15T12:45:55.045 INFO:teuthology.orchestra.run.smithi184.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-03-15T12:45:55.064 INFO:teuthology.orchestra.run.smithi149.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 407 kB/s | 10 kB 00:00 2024-03-15T12:45:55.106 INFO:teuthology.orchestra.run.smithi149.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-15T12:45:55.112 INFO:teuthology.orchestra.run.smithi184.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 823 kB/s | 90 kB 00:00 2024-03-15T12:45:55.140 INFO:teuthology.orchestra.run.smithi149.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-15T12:45:55.154 INFO:teuthology.orchestra.run.smithi184.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-03-15T12:45:55.180 INFO:teuthology.orchestra.run.smithi184.stdout:(147/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2024-03-15T12:45:55.190 INFO:teuthology.orchestra.run.smithi149.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-15T12:45:55.205 INFO:teuthology.orchestra.run.smithi184.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 216 kB/s | 20 kB 00:00 2024-03-15T12:45:55.248 INFO:teuthology.orchestra.run.smithi149.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 578 kB 00:00 2024-03-15T12:45:55.255 INFO:teuthology.orchestra.run.smithi184.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-03-15T12:45:55.290 INFO:teuthology.orchestra.run.smithi149.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-03-15T12:45:55.341 INFO:teuthology.orchestra.run.smithi149.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-15T12:45:55.383 INFO:teuthology.orchestra.run.smithi149.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2024-03-15T12:45:55.425 INFO:teuthology.orchestra.run.smithi149.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-03-15T12:45:55.437 INFO:teuthology.orchestra.run.smithi002.stdout:(31/174): rbd-mirror-19.0.0-1681.gaba3c1d2.el9. 3.0 MB/s | 3.1 MB 00:01 2024-03-15T12:45:55.450 INFO:teuthology.orchestra.run.smithi149.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 1.3 MB/s | 34 kB 00:00 2024-03-15T12:45:55.529 INFO:teuthology.orchestra.run.smithi002.stdout:(32/174): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-03-15T12:45:55.554 INFO:teuthology.orchestra.run.smithi002.stdout:(33/174): ceph-mgr-modules-core-19.0.0-1681.gab 2.1 MB/s | 248 kB 00:00 2024-03-15T12:45:55.571 INFO:teuthology.orchestra.run.smithi002.stdout:(34/174): ceph-mgr-rook-19.0.0-1681.gaba3c1d2.e 1.2 MB/s | 49 kB 00:00 2024-03-15T12:45:55.617 INFO:teuthology.orchestra.run.smithi149.stdout:(55/174): ceph-mgr-diskprediction-local-19.0.0- 2.7 MB/s | 7.4 MB 00:02 2024-03-15T12:45:55.621 INFO:teuthology.orchestra.run.smithi002.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1681.ga 229 kB/s | 15 kB 00:00 2024-03-15T12:45:55.655 INFO:teuthology.orchestra.run.smithi002.stdout:(36/174): ceph-volume-19.0.0-1681.gaba3c1d2.el9 3.1 MB/s | 264 kB 00:00 2024-03-15T12:45:55.692 INFO:teuthology.orchestra.run.smithi149.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-03-15T12:45:55.730 INFO:teuthology.orchestra.run.smithi002.stdout:(37/174): cephfs-top-19.0.0-1681.gaba3c1d2.el9. 243 kB/s | 18 kB 00:00 2024-03-15T12:45:55.734 INFO:teuthology.orchestra.run.smithi149.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 13 MB/s | 1.5 MB 00:00 2024-03-15T12:45:55.767 INFO:teuthology.orchestra.run.smithi149.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-03-15T12:45:55.793 INFO:teuthology.orchestra.run.smithi149.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-03-15T12:45:55.881 INFO:teuthology.orchestra.run.smithi184.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 2.3 MB/s | 1.6 MB 00:00 2024-03-15T12:45:55.960 INFO:teuthology.orchestra.run.smithi149.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 30 MB/s | 5.7 MB 00:00 2024-03-15T12:45:55.993 INFO:teuthology.orchestra.run.smithi149.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-03-15T12:45:56.005 INFO:teuthology.orchestra.run.smithi002.stdout:(38/174): cephadm-19.0.0-1681.gaba3c1d2.el9.noa 1.9 MB/s | 748 kB 00:00 2024-03-15T12:45:56.019 INFO:teuthology.orchestra.run.smithi149.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-15T12:45:56.022 INFO:teuthology.orchestra.run.smithi184.stdout:(151/174): librbd1-19.0.0-1681.gaba3c1d2.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-03-15T12:45:56.097 INFO:teuthology.orchestra.run.smithi184.stdout:(152/174): librados2-19.0.0-1681.gaba3c1d2.el9. 3.8 MB/s | 3.4 MB 00:00 2024-03-15T12:45:56.111 INFO:teuthology.orchestra.run.smithi149.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 267 kB/s | 24 kB 00:00 2024-03-15T12:45:56.131 INFO:teuthology.orchestra.run.smithi002.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 662 kB 00:00 2024-03-15T12:45:56.186 INFO:teuthology.orchestra.run.smithi149.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 5.8 MB 00:00 2024-03-15T12:45:56.211 INFO:teuthology.orchestra.run.smithi149.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 185 kB/s | 18 kB 00:00 2024-03-15T12:45:56.223 INFO:teuthology.orchestra.run.smithi002.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 920 kB 00:00 2024-03-15T12:45:56.245 INFO:teuthology.orchestra.run.smithi149.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-15T12:45:56.248 INFO:teuthology.orchestra.run.smithi002.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 908 kB/s | 106 kB 00:00 2024-03-15T12:45:56.248 INFO:teuthology.orchestra.run.smithi184.stdout:(153/174): xfsdump-3.1.12-4.el9.x86_64.rpm 876 kB/s | 321 kB 00:00 2024-03-15T12:45:56.262 INFO:teuthology.orchestra.run.smithi149.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 623 kB/s | 32 kB 00:00 2024-03-15T12:45:56.281 INFO:teuthology.orchestra.run.smithi002.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2024-03-15T12:45:56.315 INFO:teuthology.orchestra.run.smithi002.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 472 kB/s | 32 kB 00:00 2024-03-15T12:45:56.515 INFO:teuthology.orchestra.run.smithi184.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 159 kB/s | 42 kB 00:00 2024-03-15T12:45:56.565 INFO:teuthology.orchestra.run.smithi184.stdout:(155/174): NetworkManager-libnm-1.46.0-1.el9.x8 3.9 MB/s | 1.8 MB 00:00 2024-03-15T12:45:56.582 INFO:teuthology.orchestra.run.smithi002.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.9 MB/s | 3.0 MB 00:00 2024-03-15T12:45:56.607 INFO:teuthology.orchestra.run.smithi002.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 59 kB/s | 17 kB 00:00 2024-03-15T12:45:56.624 INFO:teuthology.orchestra.run.smithi184.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-15T12:45:56.649 INFO:teuthology.orchestra.run.smithi002.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 244 kB/s | 10 kB 00:00 2024-03-15T12:45:56.674 INFO:teuthology.orchestra.run.smithi002.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 159 kB 00:00 2024-03-15T12:45:56.682 INFO:teuthology.orchestra.run.smithi184.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 626 kB/s | 36 kB 00:00 2024-03-15T12:45:56.700 INFO:teuthology.orchestra.run.smithi002.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 884 kB/s | 44 kB 00:00 2024-03-15T12:45:56.708 INFO:teuthology.orchestra.run.smithi184.stdout:(158/174): NetworkManager-tui-1.46.0-1.el9.x86_ 1.3 MB/s | 248 kB 00:00 2024-03-15T12:45:56.754 INFO:teuthology.orchestra.run.smithi149.stdout:(68/174): ceph-test-19.0.0-1681.gaba3c1d2.el9.x 6.0 MB/s | 47 MB 00:07 2024-03-15T12:45:56.758 INFO:teuthology.orchestra.run.smithi184.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-03-15T12:45:56.767 INFO:teuthology.orchestra.run.smithi002.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 661 kB 00:00 2024-03-15T12:45:56.801 INFO:teuthology.orchestra.run.smithi002.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 5.6 MB/s | 578 kB 00:00 2024-03-15T12:45:56.817 INFO:teuthology.orchestra.run.smithi184.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-03-15T12:45:56.821 INFO:teuthology.orchestra.run.smithi149.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.3 MB/s | 4.8 MB 00:00 2024-03-15T12:45:56.826 INFO:teuthology.orchestra.run.smithi002.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 806 kB/s | 46 kB 00:00 2024-03-15T12:45:56.842 INFO:teuthology.orchestra.run.smithi184.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 276 kB/s | 23 kB 00:00 2024-03-15T12:45:56.846 INFO:teuthology.orchestra.run.smithi149.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 372 kB/s | 217 kB 00:00 2024-03-15T12:45:56.868 INFO:teuthology.orchestra.run.smithi002.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.1 MB/s | 282 kB 00:00 2024-03-15T12:45:56.871 INFO:teuthology.orchestra.run.smithi149.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 174 kB 00:00 2024-03-15T12:45:56.875 INFO:teuthology.orchestra.run.smithi184.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-03-15T12:45:56.893 INFO:teuthology.orchestra.run.smithi002.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2024-03-15T12:45:56.897 INFO:teuthology.orchestra.run.smithi149.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-15T12:45:56.901 INFO:teuthology.orchestra.run.smithi184.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-03-15T12:45:56.913 INFO:teuthology.orchestra.run.smithi149.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 619 kB/s | 41 kB 00:00 2024-03-15T12:45:56.927 INFO:teuthology.orchestra.run.smithi002.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-03-15T12:45:56.942 INFO:teuthology.orchestra.run.smithi149.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-15T12:45:56.951 INFO:teuthology.orchestra.run.smithi184.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 136 kB 00:00 2024-03-15T12:45:56.952 INFO:teuthology.orchestra.run.smithi002.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 583 kB/s | 34 kB 00:00 2024-03-15T12:45:56.964 INFO:teuthology.orchestra.run.smithi149.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-15T12:45:56.977 INFO:teuthology.orchestra.run.smithi184.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 849 kB/s | 64 kB 00:00 2024-03-15T12:45:56.989 INFO:teuthology.orchestra.run.smithi149.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2024-03-15T12:45:57.006 INFO:teuthology.orchestra.run.smithi149.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 711 kB/s | 47 kB 00:00 2024-03-15T12:45:57.023 INFO:teuthology.orchestra.run.smithi149.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 733 kB/s | 43 kB 00:00 2024-03-15T12:45:57.027 INFO:teuthology.orchestra.run.smithi184.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 381 kB/s | 29 kB 00:00 2024-03-15T12:45:57.090 INFO:teuthology.orchestra.run.smithi149.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 464 kB/s | 39 kB 00:00 2024-03-15T12:45:57.186 INFO:teuthology.orchestra.run.smithi184.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 5.8 MB/s | 1.2 MB 00:00 2024-03-15T12:45:57.319 INFO:teuthology.orchestra.run.smithi002.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-03-15T12:45:57.332 INFO:teuthology.orchestra.run.smithi149.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 133 kB/s | 41 kB 00:00 2024-03-15T12:45:57.461 INFO:teuthology.orchestra.run.smithi002.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 3.0 MB/s | 1.5 MB 00:00 2024-03-15T12:45:57.482 INFO:teuthology.orchestra.run.smithi149.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 768 kB/s | 300 kB 00:00 2024-03-15T12:45:57.494 INFO:teuthology.orchestra.run.smithi002.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 4.6 MB/s | 811 kB 00:00 2024-03-15T12:45:57.545 INFO:teuthology.orchestra.run.smithi002.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 1.6 MB/s | 134 kB 00:00 2024-03-15T12:45:57.653 INFO:teuthology.orchestra.run.smithi184.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 4.9 MB/s | 2.3 MB 00:00 2024-03-15T12:45:57.682 INFO:teuthology.orchestra.run.smithi149.stdout:(82/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-03-15T12:45:57.699 INFO:teuthology.orchestra.run.smithi149.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 154 kB/s | 56 kB 00:00 2024-03-15T12:45:57.716 INFO:teuthology.orchestra.run.smithi149.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 271 kB/s | 63 kB 00:00 2024-03-15T12:45:57.733 INFO:teuthology.orchestra.run.smithi149.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-15T12:45:57.767 INFO:teuthology.orchestra.run.smithi149.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-15T12:45:57.792 INFO:teuthology.orchestra.run.smithi149.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-15T12:45:57.794 INFO:teuthology.orchestra.run.smithi184.stdout:(169/174): NetworkManager-1.46.0-1.el9.x86_64.r 1.3 MB/s | 2.3 MB 00:01 2024-03-15T12:45:57.820 INFO:teuthology.orchestra.run.smithi184.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 2.7 MB/s | 465 kB 00:00 2024-03-15T12:45:57.842 INFO:teuthology.orchestra.run.smithi149.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-15T12:45:57.867 INFO:teuthology.orchestra.run.smithi149.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-03-15T12:45:57.870 INFO:teuthology.orchestra.run.smithi184.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 414 kB/s | 31 kB 00:00 2024-03-15T12:45:57.887 INFO:teuthology.orchestra.run.smithi002.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.0 MB/s | 1.0 MB 00:00 2024-03-15T12:45:57.909 INFO:teuthology.orchestra.run.smithi149.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 124 kB 00:00 2024-03-15T12:45:57.935 INFO:teuthology.orchestra.run.smithi149.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-03-15T12:45:57.953 INFO:teuthology.orchestra.run.smithi002.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 12 MB/s | 5.7 MB 00:00 2024-03-15T12:45:57.979 INFO:teuthology.orchestra.run.smithi002.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 852 kB/s | 78 kB 00:00 2024-03-15T12:45:58.020 INFO:teuthology.orchestra.run.smithi184.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.4 MB/s | 1.1 MB 00:00 2024-03-15T12:45:58.212 INFO:teuthology.orchestra.run.smithi002.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 105 kB/s | 24 kB 00:00 2024-03-15T12:45:58.254 INFO:teuthology.orchestra.run.smithi184.stdout:(173/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 1.8 MB/s | 2.2 MB 00:01 2024-03-15T12:45:58.279 INFO:teuthology.orchestra.run.smithi184.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 182 kB/s | 74 kB 00:00 2024-03-15T12:45:58.283 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-15T12:45:58.284 INFO:teuthology.orchestra.run.smithi184.stdout:Total 13 MB/s | 224 MB 00:16 2024-03-15T12:45:58.287 INFO:teuthology.orchestra.run.smithi002.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2024-03-15T12:45:58.313 INFO:teuthology.orchestra.run.smithi002.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 182 kB/s | 18 kB 00:00 2024-03-15T12:45:58.338 INFO:teuthology.orchestra.run.smithi002.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-15T12:45:58.363 INFO:teuthology.orchestra.run.smithi002.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 631 kB/s | 32 kB 00:00 2024-03-15T12:45:58.468 INFO:teuthology.orchestra.run.smithi149.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 335 kB/s | 187 kB 00:00 2024-03-15T12:45:58.493 INFO:teuthology.orchestra.run.smithi149.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 350 kB/s | 195 kB 00:00 2024-03-15T12:45:58.577 INFO:teuthology.orchestra.run.smithi149.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-03-15T12:45:58.580 INFO:teuthology.orchestra.run.smithi002.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-03-15T12:45:58.594 INFO:teuthology.orchestra.run.smithi149.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 215 kB/s | 21 kB 00:00 2024-03-15T12:45:58.605 INFO:teuthology.orchestra.run.smithi002.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 900 kB/s | 217 kB 00:00 2024-03-15T12:45:58.631 INFO:teuthology.orchestra.run.smithi002.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-15T12:45:58.656 INFO:teuthology.orchestra.run.smithi002.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-15T12:45:58.669 INFO:teuthology.orchestra.run.smithi149.stdout:(96/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-03-15T12:45:58.673 INFO:teuthology.orchestra.run.smithi002.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 989 kB/s | 41 kB 00:00 2024-03-15T12:45:58.694 INFO:teuthology.orchestra.run.smithi149.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 217 kB/s | 25 kB 00:00 2024-03-15T12:45:58.698 INFO:teuthology.orchestra.run.smithi002.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 5.6 MB/s | 238 kB 00:00 2024-03-15T12:45:58.723 INFO:teuthology.orchestra.run.smithi002.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-15T12:45:58.749 INFO:teuthology.orchestra.run.smithi002.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-15T12:45:58.761 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-15T12:45:58.787 INFO:teuthology.orchestra.run.smithi149.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-03-15T12:45:58.799 INFO:teuthology.orchestra.run.smithi002.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 631 kB/s | 47 kB 00:00 2024-03-15T12:45:58.824 INFO:teuthology.orchestra.run.smithi002.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 571 kB/s | 43 kB 00:00 2024-03-15T12:45:58.837 INFO:teuthology.orchestra.run.smithi149.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 4.3 MB/s | 216 kB 00:00 2024-03-15T12:45:58.887 INFO:teuthology.orchestra.run.smithi149.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 697 kB/s | 35 kB 00:00 2024-03-15T12:45:58.912 INFO:teuthology.orchestra.run.smithi149.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.1 MB/s | 248 kB 00:00 2024-03-15T12:45:58.937 INFO:teuthology.orchestra.run.smithi149.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 888 kB/s | 44 kB 00:00 2024-03-15T12:45:58.982 INFO:teuthology.orchestra.run.smithi149.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-03-15T12:45:59.008 INFO:teuthology.orchestra.run.smithi002.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 211 kB/s | 39 kB 00:00 2024-03-15T12:45:59.070 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-15T12:45:59.070 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-15T12:45:59.130 INFO:teuthology.orchestra.run.smithi149.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 116 kB/s | 22 kB 00:00 2024-03-15T12:45:59.230 INFO:teuthology.orchestra.run.smithi149.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-03-15T12:45:59.467 INFO:teuthology.orchestra.run.smithi002.stdout:(79/174): ceph-test-19.0.0-1681.gaba3c1d2.el9.x 7.1 MB/s | 47 MB 00:06 2024-03-15T12:45:59.500 INFO:teuthology.orchestra.run.smithi002.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 84 kB/s | 41 kB 00:00 2024-03-15T12:45:59.597 INFO:teuthology.orchestra.run.smithi149.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 843 kB/s | 309 kB 00:00 2024-03-15T12:45:59.656 INFO:teuthology.orchestra.run.smithi149.stdout:(107/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.9 MB/s | 4.1 MB 00:01 2024-03-15T12:45:59.681 INFO:teuthology.orchestra.run.smithi149.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 995 kB/s | 548 kB 00:00 2024-03-15T12:45:59.734 INFO:teuthology.orchestra.run.smithi002.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-03-15T12:45:59.739 INFO:teuthology.orchestra.run.smithi149.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 137 kB/s | 19 kB 00:00 2024-03-15T12:45:59.759 INFO:teuthology.orchestra.run.smithi002.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 300 kB 00:00 2024-03-15T12:45:59.785 INFO:teuthology.orchestra.run.smithi002.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 200 kB/s | 56 kB 00:00 2024-03-15T12:45:59.789 INFO:teuthology.orchestra.run.smithi149.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-03-15T12:45:59.810 INFO:teuthology.orchestra.run.smithi002.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 841 kB/s | 63 kB 00:00 2024-03-15T12:45:59.835 INFO:teuthology.orchestra.run.smithi002.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-15T12:45:59.852 INFO:teuthology.orchestra.run.smithi002.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2024-03-15T12:45:59.877 INFO:teuthology.orchestra.run.smithi002.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-03-15T12:45:59.894 INFO:teuthology.orchestra.run.smithi002.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 2.4 MB/s | 103 kB 00:00 2024-03-15T12:45:59.898 INFO:teuthology.orchestra.run.smithi149.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 306 kB/s | 49 kB 00:00 2024-03-15T12:45:59.936 INFO:teuthology.orchestra.run.smithi002.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-15T12:45:59.949 INFO:teuthology.orchestra.run.smithi149.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 424 kB/s | 67 kB 00:00 2024-03-15T12:45:59.961 INFO:teuthology.orchestra.run.smithi002.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-15T12:45:59.986 INFO:teuthology.orchestra.run.smithi002.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-03-15T12:46:00.120 INFO:teuthology.orchestra.run.smithi002.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-03-15T12:46:00.145 INFO:teuthology.orchestra.run.smithi002.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-03-15T12:46:00.170 INFO:teuthology.orchestra.run.smithi002.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-15T12:46:00.212 INFO:teuthology.orchestra.run.smithi002.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-03-15T12:46:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 161 kB/s | 21 kB 00:00 2024-03-15T12:46:00.304 INFO:teuthology.orchestra.run.smithi002.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 190 kB/s | 25 kB 00:00 2024-03-15T12:46:00.373 INFO:teuthology.orchestra.run.smithi149.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 318 kB/s | 151 kB 00:00 2024-03-15T12:46:00.455 INFO:teuthology.orchestra.run.smithi002.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2024-03-15T12:46:00.496 INFO:teuthology.orchestra.run.smithi002.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-03-15T12:46:00.532 INFO:teuthology.orchestra.run.smithi149.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.4 MB/s | 838 kB 00:00 2024-03-15T12:46:00.563 INFO:teuthology.orchestra.run.smithi002.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.9 MB/s | 216 kB 00:00 2024-03-15T12:46:00.589 INFO:teuthology.orchestra.run.smithi002.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 379 kB/s | 35 kB 00:00 2024-03-15T12:46:00.630 INFO:teuthology.orchestra.run.smithi002.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 667 kB/s | 44 kB 00:00 2024-03-15T12:46:00.656 INFO:teuthology.orchestra.run.smithi002.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-03-15T12:46:00.749 INFO:teuthology.orchestra.run.smithi149.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 201 kB/s | 43 kB 00:00 2024-03-15T12:46:00.798 INFO:teuthology.orchestra.run.smithi002.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-03-15T12:46:00.874 INFO:teuthology.orchestra.run.smithi149.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 258 kB/s | 32 kB 00:00 2024-03-15T12:46:00.900 INFO:teuthology.orchestra.run.smithi149.stdout:(117/174): python3-asyncssh-2.13.2-4.el9.noarch 1.0 MB/s | 548 kB 00:00 2024-03-15T12:46:01.033 INFO:teuthology.orchestra.run.smithi149.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 94 kB/s | 15 kB 00:00 2024-03-15T12:46:01.086 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-15T12:46:01.087 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-15T12:46:01.131 INFO:teuthology.orchestra.run.smithi002.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.5 MB/s | 4.1 MB 00:00 2024-03-15T12:46:01.150 INFO:teuthology.orchestra.run.smithi149.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 687 kB/s | 172 kB 00:00 2024-03-15T12:46:01.157 INFO:teuthology.orchestra.run.smithi002.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-03-15T12:46:01.215 INFO:teuthology.orchestra.run.smithi002.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-03-15T12:46:01.241 INFO:teuthology.orchestra.run.smithi002.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.8 MB/s | 309 kB 00:00 2024-03-15T12:46:01.274 INFO:teuthology.orchestra.run.smithi002.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 166 kB/s | 19 kB 00:00 2024-03-15T12:46:01.350 INFO:teuthology.orchestra.run.smithi149.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 1.1 MB/s | 358 kB 00:00 2024-03-15T12:46:01.467 INFO:teuthology.orchestra.run.smithi149.stdout:(121/174): libarrow-9.0.0-10.el9.x86_64.rpm 2.4 MB/s | 4.4 MB 00:01 2024-03-15T12:46:01.568 INFO:teuthology.orchestra.run.smithi149.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 162 kB/s | 35 kB 00:00 2024-03-15T12:46:01.593 INFO:teuthology.orchestra.run.smithi149.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 685 kB/s | 303 kB 00:00 2024-03-15T12:46:01.783 INFO:teuthology.orchestra.run.smithi002.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 48 kB/s | 26 kB 00:00 2024-03-15T12:46:01.808 INFO:teuthology.orchestra.run.smithi002.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 91 kB/s | 49 kB 00:00 2024-03-15T12:46:01.868 INFO:teuthology.orchestra.run.smithi149.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 525 kB/s | 144 kB 00:00 2024-03-15T12:46:01.902 INFO:teuthology.orchestra.run.smithi149.stdout:(125/174): python3-google-auth-2.28.0-1.el9.noa 467 kB/s | 203 kB 00:00 2024-03-15T12:46:02.010 INFO:teuthology.orchestra.run.smithi149.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-03-15T12:46:02.033 INFO:teuthology.orchestra.run.smithi002.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 269 kB/s | 67 kB 00:00 2024-03-15T12:46:02.036 INFO:teuthology.orchestra.run.smithi149.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 132 kB/s | 18 kB 00:00 2024-03-15T12:46:02.161 INFO:teuthology.orchestra.run.smithi149.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 154 kB/s | 23 kB 00:00 2024-03-15T12:46:02.345 INFO:teuthology.orchestra.run.smithi149.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 63 kB/s | 19 kB 00:00 2024-03-15T12:46:02.437 INFO:teuthology.orchestra.run.smithi149.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 71 kB/s | 20 kB 00:00 2024-03-15T12:46:02.642 INFO:teuthology.orchestra.run.smithi002.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 181 kB/s | 151 kB 00:00 2024-03-15T12:46:02.687 INFO:teuthology.orchestra.run.smithi149.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 185 kB/s | 46 kB 00:00 2024-03-15T12:46:02.842 INFO:teuthology.orchestra.run.smithi002.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 4.4 MB 00:01 2024-03-15T12:46:02.909 INFO:teuthology.orchestra.run.smithi002.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 958 kB/s | 838 kB 00:00 2024-03-15T12:46:02.959 INFO:teuthology.orchestra.run.smithi002.stdout:(116/174): python3-asyncssh-2.13.2-4.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-03-15T12:46:02.984 INFO:teuthology.orchestra.run.smithi002.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 307 kB/s | 43 kB 00:00 2024-03-15T12:46:03.010 INFO:teuthology.orchestra.run.smithi002.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-03-15T12:46:03.035 INFO:teuthology.orchestra.run.smithi002.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-03-15T12:46:03.077 INFO:teuthology.orchestra.run.smithi002.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-03-15T12:46:03.127 INFO:teuthology.orchestra.run.smithi002.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-03-15T12:46:03.153 INFO:teuthology.orchestra.run.smithi002.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-03-15T12:46:03.178 INFO:teuthology.orchestra.run.smithi002.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 354 kB/s | 35 kB 00:00 2024-03-15T12:46:03.245 INFO:teuthology.orchestra.run.smithi149.stdout:(132/174): python3-grpcio-1.46.7-10.el9.x86_64. 1.2 MB/s | 2.0 MB 00:01 2024-03-15T12:46:03.279 INFO:teuthology.orchestra.run.smithi149.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 1.1 MB/s | 1.0 MB 00:00 2024-03-15T12:46:03.338 INFO:teuthology.orchestra.run.smithi149.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 121 kB/s | 79 kB 00:00 2024-03-15T12:46:03.379 INFO:teuthology.orchestra.run.smithi149.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 432 kB/s | 58 kB 00:00 2024-03-15T12:46:03.505 INFO:teuthology.orchestra.run.smithi149.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 98 kB/s | 16 kB 00:00 2024-03-15T12:46:03.630 INFO:teuthology.orchestra.run.smithi149.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 226 kB/s | 28 kB 00:00 2024-03-15T12:46:03.772 INFO:teuthology.orchestra.run.smithi149.stdout:(138/174): python3-repoze-lru-0.7-16.el9.noarch 217 kB/s | 31 kB 00:00 2024-03-15T12:46:03.778 INFO:teuthology.orchestra.run.smithi002.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 312 kB/s | 203 kB 00:00 2024-03-15T12:46:03.797 INFO:teuthology.orchestra.run.smithi149.stdout:(139/174): python3-pecan-1.4.2-3.el9.noarch.rpm 526 kB/s | 272 kB 00:00 2024-03-15T12:46:03.847 INFO:teuthology.orchestra.run.smithi149.stdout:(140/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 192 kB/s | 90 kB 00:00 2024-03-15T12:46:04.039 INFO:teuthology.orchestra.run.smithi149.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 244 kB/s | 59 kB 00:00 2024-03-15T12:46:04.131 INFO:teuthology.orchestra.run.smithi149.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 126 kB/s | 36 kB 00:00 2024-03-15T12:46:04.223 INFO:teuthology.orchestra.run.smithi149.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 278 kB/s | 51 kB 00:00 2024-03-15T12:46:04.301 INFO:teuthology.orchestra.run.smithi149.stdout:(144/174): python3-routes-2.5.1-5.el9.noarch.rp 358 kB/s | 188 kB 00:00 2024-03-15T12:46:04.457 INFO:teuthology.orchestra.run.smithi149.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 705 kB/s | 230 kB 00:00 2024-03-15T12:46:04.482 INFO:teuthology.orchestra.run.smithi149.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 347 kB/s | 90 kB 00:00 2024-03-15T12:46:04.599 INFO:teuthology.orchestra.run.smithi149.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 141 kB/s | 20 kB 00:00 2024-03-15T12:46:04.741 INFO:teuthology.orchestra.run.smithi149.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 966 kB/s | 427 kB 00:00 2024-03-15T12:46:04.766 INFO:teuthology.orchestra.run.smithi149.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 675 kB/s | 192 kB 00:00 2024-03-15T12:46:05.134 INFO:teuthology.orchestra.run.smithi149.stdout:(150/174): librados2-19.0.0-1681.gaba3c1d2.el9. 8.6 MB/s | 3.4 MB 00:00 2024-03-15T12:46:05.153 INFO:teuthology.orchestra.run.smithi002.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 7.8 kB/s | 11 kB 00:01 2024-03-15T12:46:05.200 INFO:teuthology.orchestra.run.smithi149.stdout:(151/174): librbd1-19.0.0-1681.gaba3c1d2.el9.x8 7.2 MB/s | 3.1 MB 00:00 2024-03-15T12:46:05.225 INFO:teuthology.orchestra.run.smithi149.stdout:(152/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.4 MB/s | 321 kB 00:00 2024-03-15T12:46:05.795 INFO:teuthology.orchestra.run.smithi002.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 55 kB/s | 144 kB 00:02 2024-03-15T12:46:06.112 INFO:teuthology.orchestra.run.smithi002.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 18 kB/s | 18 kB 00:00 2024-03-15T12:46:06.312 INFO:teuthology.orchestra.run.smithi002.stdout:(128/174): python3-grpcio-1.46.7-10.el9.x86_64. 666 kB/s | 2.0 MB 00:03 2024-03-15T12:46:06.337 INFO:teuthology.orchestra.run.smithi002.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 43 kB/s | 23 kB 00:00 2024-03-15T12:46:06.362 INFO:teuthology.orchestra.run.smithi002.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 78 kB/s | 19 kB 00:00 2024-03-15T12:46:06.404 INFO:teuthology.orchestra.run.smithi002.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-03-15T12:46:06.817 INFO:teuthology.orchestra.run.smithi149.stdout:(153/174): thrift-0.15.0-2.el9.x86_64.rpm 730 kB/s | 1.6 MB 00:02 2024-03-15T12:46:07.268 INFO:teuthology.orchestra.run.smithi149.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 94 kB/s | 42 kB 00:00 2024-03-15T12:46:07.463 INFO:teuthology.orchestra.run.smithi002.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 950 kB/s | 1.0 MB 00:01 2024-03-15T12:46:07.605 INFO:teuthology.orchestra.run.smithi002.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 37 kB/s | 46 kB 00:01 2024-03-15T12:46:07.634 INFO:teuthology.orchestra.run.smithi149.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 677 kB/s | 248 kB 00:00 2024-03-15T12:46:07.680 INFO:teuthology.orchestra.run.smithi002.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 62 kB/s | 79 kB 00:01 2024-03-15T12:46:07.739 INFO:teuthology.orchestra.run.smithi002.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 210 kB/s | 58 kB 00:00 2024-03-15T12:46:07.784 INFO:teuthology.orchestra.run.smithi149.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 476 kB/s | 71 kB 00:00 2024-03-15T12:46:07.918 INFO:teuthology.orchestra.run.smithi149.stdout:(157/174): NetworkManager-libnm-1.46.0-1.el9.x8 691 kB/s | 1.8 MB 00:02 2024-03-15T12:46:07.943 INFO:teuthology.orchestra.run.smithi149.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 230 kB/s | 36 kB 00:00 2024-03-15T12:46:08.014 INFO:teuthology.orchestra.run.smithi002.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 666 kB/s | 272 kB 00:00 2024-03-15T12:46:08.036 INFO:teuthology.orchestra.run.smithi149.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.6 MB/s | 192 kB 00:00 2024-03-15T12:46:08.039 INFO:teuthology.orchestra.run.smithi002.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 46 kB/s | 16 kB 00:00 2024-03-15T12:46:08.061 INFO:teuthology.orchestra.run.smithi149.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.1 MB/s | 256 kB 00:00 2024-03-15T12:46:08.106 INFO:teuthology.orchestra.run.smithi002.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 245 kB/s | 90 kB 00:00 2024-03-15T12:46:08.131 INFO:teuthology.orchestra.run.smithi002.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 241 kB/s | 28 kB 00:00 2024-03-15T12:46:08.144 INFO:teuthology.orchestra.run.smithi149.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 212 kB/s | 23 kB 00:00 2024-03-15T12:46:08.156 INFO:teuthology.orchestra.run.smithi002.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2024-03-15T12:46:08.178 INFO:teuthology.orchestra.run.smithi149.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 928 kB/s | 108 kB 00:00 2024-03-15T12:46:08.270 INFO:teuthology.orchestra.run.smithi149.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 155 kB 00:00 2024-03-15T12:46:08.304 INFO:teuthology.orchestra.run.smithi149.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-03-15T12:46:08.306 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-15T12:46:08.379 INFO:teuthology.orchestra.run.smithi149.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 588 kB/s | 64 kB 00:00 2024-03-15T12:46:08.398 INFO:teuthology.orchestra.run.smithi002.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 646 kB/s | 188 kB 00:00 2024-03-15T12:46:08.404 INFO:teuthology.orchestra.run.smithi149.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 286 kB/s | 29 kB 00:00 2024-03-15T12:46:08.409 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-15T12:46:08.443 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-15T12:46:08.457 INFO:teuthology.orchestra.run.smithi002.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 182 kB/s | 59 kB 00:00 2024-03-15T12:46:08.482 INFO:teuthology.orchestra.run.smithi002.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 110 kB/s | 36 kB 00:00 2024-03-15T12:46:08.499 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-15T12:46:08.509 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-15T12:46:08.549 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-15T12:46:08.550 INFO:teuthology.orchestra.run.smithi002.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 338 kB/s | 51 kB 00:00 2024-03-15T12:46:08.603 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-15T12:46:08.644 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-15T12:46:08.670 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-15T12:46:08.713 INFO:teuthology.orchestra.run.smithi149.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 7.0 MB/s | 2.2 MB 00:00 2024-03-15T12:46:08.730 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-15T12:46:08.746 INFO:teuthology.orchestra.run.smithi149.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.3 MB/s | 1.2 MB 00:00 2024-03-15T12:46:08.871 INFO:teuthology.orchestra.run.smithi149.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.6 MB/s | 465 kB 00:00 2024-03-15T12:46:08.950 INFO:teuthology.orchestra.run.smithi002.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 466 kB/s | 230 kB 00:00 2024-03-15T12:46:08.980 INFO:teuthology.orchestra.run.smithi149.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 287 kB/s | 31 kB 00:00 2024-03-15T12:46:09.057 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-15T12:46:09.108 INFO:teuthology.orchestra.run.smithi002.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 143 kB/s | 90 kB 00:00 2024-03-15T12:46:09.121 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 10/198 2024-03-15T12:46:09.243 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 10/198 2024-03-15T12:46:09.264 INFO:teuthology.orchestra.run.smithi149.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.8 MB/s | 1.1 MB 00:00 2024-03-15T12:46:09.280 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 11/198 2024-03-15T12:46:09.364 INFO:teuthology.orchestra.run.smithi149.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 743 kB/s | 74 kB 00:00 2024-03-15T12:46:09.376 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 11/198 2024-03-15T12:46:09.407 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:19.0.0-1681.gaba3c1d2.el9.x86_64 12/198 2024-03-15T12:46:09.449 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-15T12:46:09.497 INFO:teuthology.orchestra.run.smithi149.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 2.9 MB/s | 2.3 MB 00:00 2024-03-15T12:46:09.641 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-15T12:46:09.691 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-15T12:46:09.719 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 16/198 2024-03-15T12:46:09.739 INFO:teuthology.orchestra.run.smithi149.stdout:(174/174): NetworkManager-1.46.0-1.el9.x86_64.r 516 kB/s | 2.3 MB 00:04 2024-03-15T12:46:09.744 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-15T12:46:09.744 INFO:teuthology.orchestra.run.smithi149.stdout:Total 8.5 MB/s | 224 MB 00:26 2024-03-15T12:46:09.822 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 16/198 2024-03-15T12:46:09.845 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86 17/198 2024-03-15T12:46:09.910 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86 17/198 2024-03-15T12:46:09.949 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-15T12:46:09.980 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-15T12:46:09.996 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-15T12:46:10.160 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-15T12:46:10.192 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-15T12:46:10.205 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-15T12:46:10.240 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-15T12:46:10.316 INFO:teuthology.orchestra.run.smithi002.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 242 kB/s | 427 kB 00:01 2024-03-15T12:46:10.350 INFO:teuthology.orchestra.run.smithi002.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 14 kB/s | 20 kB 00:01 2024-03-15T12:46:10.445 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-15T12:46:10.519 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-15T12:46:10.519 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-15T12:46:10.583 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-15T12:46:10.635 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-15T12:46:10.742 INFO:teuthology.orchestra.run.smithi002.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 117 kB/s | 192 kB 00:01 2024-03-15T12:46:10.749 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-15T12:46:10.820 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-15T12:46:11.017 INFO:teuthology.orchestra.run.smithi002.stdout:(150/174): librados2-19.0.0-1681.gaba3c1d2.el9. 5.1 MB/s | 3.4 MB 00:00 2024-03-15T12:46:11.025 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-15T12:46:11.059 INFO:teuthology.orchestra.run.smithi002.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-15T12:46:11.136 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-15T12:46:11.228 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-15T12:46:11.242 INFO:teuthology.orchestra.run.smithi002.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-03-15T12:46:11.266 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-15T12:46:11.293 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-15T12:46:11.318 INFO:teuthology.orchestra.run.smithi002.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-03-15T12:46:11.337 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-15T12:46:11.360 INFO:teuthology.orchestra.run.smithi002.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-15T12:46:11.401 INFO:teuthology.orchestra.run.smithi002.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-03-15T12:46:11.410 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-15T12:46:11.443 INFO:teuthology.orchestra.run.smithi002.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-03-15T12:46:11.464 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-15T12:46:11.477 INFO:teuthology.orchestra.run.smithi002.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 1.1 MB/s | 36 kB 00:00 2024-03-15T12:46:11.511 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-15T12:46:11.519 INFO:teuthology.orchestra.run.smithi002.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2024-03-15T12:46:11.561 INFO:teuthology.orchestra.run.smithi002.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 256 kB 00:00 2024-03-15T12:46:11.583 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-15T12:46:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 691 kB/s | 23 kB 00:00 2024-03-15T12:46:11.627 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-15T12:46:11.637 INFO:teuthology.orchestra.run.smithi002.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 108 kB 00:00 2024-03-15T12:46:11.687 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-15T12:46:11.695 INFO:teuthology.orchestra.run.smithi002.stdout:(162/174): librbd1-19.0.0-1681.gaba3c1d2.el9.x8 3.3 MB/s | 3.1 MB 00:00 2024-03-15T12:46:11.721 INFO:teuthology.orchestra.run.smithi002.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 155 kB 00:00 2024-03-15T12:46:11.745 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-15T12:46:11.762 INFO:teuthology.orchestra.run.smithi002.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 64 kB 00:00 2024-03-15T12:46:11.772 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-15T12:46:11.829 INFO:teuthology.orchestra.run.smithi002.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 136 kB 00:00 2024-03-15T12:46:11.841 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-15T12:46:11.854 INFO:teuthology.orchestra.run.smithi002.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 312 kB/s | 29 kB 00:00 2024-03-15T12:46:11.878 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:19.0.0-1681.gaba3c1d2.el9. 39/198 2024-03-15T12:46:11.921 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:19.0.0-1681.gaba3c1d2.el 40/198 2024-03-15T12:46:11.938 INFO:teuthology.orchestra.run.smithi002.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 26 MB/s | 2.2 MB 00:00 2024-03-15T12:46:11.949 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:19.0.0-1681.gaba3c1d2.el9.x86_6 41/198 2024-03-15T12:46:11.972 INFO:teuthology.orchestra.run.smithi002.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 8.5 MB/s | 1.2 MB 00:00 2024-03-15T12:46:12.017 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-15T12:46:12.030 INFO:teuthology.orchestra.run.smithi002.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 25 MB/s | 2.3 MB 00:00 2024-03-15T12:46:12.055 INFO:teuthology.orchestra.run.smithi002.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 5.4 MB/s | 465 kB 00:00 2024-03-15T12:46:12.064 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-15T12:46:12.081 INFO:teuthology.orchestra.run.smithi002.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 622 kB/s | 31 kB 00:00 2024-03-15T12:46:12.114 INFO:teuthology.orchestra.run.smithi002.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-03-15T12:46:12.145 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-15T12:46:12.165 INFO:teuthology.orchestra.run.smithi002.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 889 kB/s | 74 kB 00:00 2024-03-15T12:46:12.286 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-15T12:46:12.341 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-15T12:46:12.709 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-15T12:46:12.710 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-15T12:46:12.859 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-15T12:46:12.945 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-15T12:46:13.034 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-15T12:46:13.090 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-15T12:46:13.098 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-15T12:46:13.141 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-15T12:46:13.187 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-15T12:46:13.225 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_6 53/198 2024-03-15T12:46:13.255 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-15T12:46:13.545 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-15T12:46:13.648 INFO:teuthology.orchestra.run.smithi002.stdout:(174/174): thrift-0.15.0-2.el9.x86_64.rpm 486 kB/s | 1.6 MB 00:03 2024-03-15T12:46:13.653 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-15T12:46:13.653 INFO:teuthology.orchestra.run.smithi002.stdout:Total 9.0 MB/s | 224 MB 00:24 2024-03-15T12:46:13.952 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-15T12:46:14.049 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-15T12:46:14.111 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-15T12:46:14.125 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-15T12:46:14.261 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-15T12:46:14.312 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-15T12:46:14.366 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-15T12:46:14.440 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-15T12:46:14.459 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-15T12:46:14.459 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-15T12:46:14.471 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-15T12:46:14.507 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-15T12:46:14.554 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-15T12:46:14.598 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-15T12:46:14.650 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-15T12:46:14.692 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-15T12:46:14.841 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-15T12:46:14.897 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-15T12:46:14.928 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-15T12:46:14.969 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-15T12:46:15.062 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-15T12:46:15.094 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-15T12:46:15.144 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-15T12:46:15.168 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-15T12:46:15.193 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-15T12:46:15.294 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-15T12:46:15.428 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-15T12:46:15.525 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-15T12:46:15.568 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-15T12:46:15.617 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-15T12:46:15.672 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-15T12:46:15.863 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-15T12:46:15.914 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-15T12:46:16.042 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-15T12:46:16.079 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-15T12:46:16.111 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-15T12:46:16.150 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-15T12:46:16.186 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-15T12:46:16.221 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-15T12:46:16.260 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-15T12:46:16.291 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-15T12:46:16.321 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-15T12:46:16.405 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-15T12:46:16.440 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-15T12:46:16.474 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-15T12:46:16.515 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-15T12:46:16.539 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-15T12:46:16.540 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-15T12:46:16.550 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-15T12:46:16.638 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-15T12:46:16.728 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-15T12:46:16.772 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-15T12:46:16.804 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-15T12:46:16.851 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-15T12:46:16.948 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-15T12:46:16.997 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-15T12:46:17.722 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-15T12:46:17.777 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-15T12:46:17.811 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-15T12:46:17.859 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-15T12:46:17.938 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-15T12:46:17.970 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-15T12:46:18.147 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-15T12:46:18.177 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-15T12:46:18.254 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-15T12:46:18.910 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-15T12:46:19.099 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-15T12:46:19.742 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-15T12:46:19.837 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-15T12:46:19.862 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-15T12:46:19.908 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-15T12:46:19.918 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-15T12:46:19.958 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-15T12:46:20.013 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-15T12:46:20.045 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-15T12:46:20.079 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-15T12:46:20.134 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-15T12:46:20.335 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-15T12:46:20.427 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-15T12:46:20.459 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-15T12:46:20.490 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-15T12:46:20.517 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-15T12:46:20.520 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-15T12:46:20.550 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 10/198 2024-03-15T12:46:20.656 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 10/198 2024-03-15T12:46:20.684 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 11/198 2024-03-15T12:46:20.769 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 11/198 2024-03-15T12:46:20.802 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:19.0.0-1681.gaba3c1d2.el9.x86_64 12/198 2024-03-15T12:46:20.836 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-15T12:46:20.843 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-15T12:46:20.989 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 124/198 2024-03-15T12:46:21.028 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-15T12:46:21.077 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-15T12:46:21.106 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 16/198 2024-03-15T12:46:21.208 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 16/198 2024-03-15T12:46:21.232 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86 17/198 2024-03-15T12:46:21.280 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86 17/198 2024-03-15T12:46:21.306 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 124/198 2024-03-15T12:46:21.320 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-15T12:46:21.342 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-15T12:46:21.357 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-15T12:46:21.377 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 125/198 2024-03-15T12:46:21.422 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-15T12:46:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-15T12:46:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-15T12:46:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-15T12:46:21.809 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-15T12:46:21.885 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-15T12:46:21.917 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 128/198 2024-03-15T12:46:21.949 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-15T12:46:21.971 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 128/198 2024-03-15T12:46:22.002 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 129/198 2024-03-15T12:46:22.009 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-15T12:46:22.152 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-15T12:46:22.210 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-15T12:46:22.422 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-15T12:46:22.532 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-15T12:46:22.636 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-15T12:46:22.674 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-15T12:46:22.701 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-15T12:46:22.744 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-15T12:46:22.819 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-15T12:46:22.872 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-15T12:46:22.911 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-15T12:46:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-15T12:46:23.010 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-15T12:46:23.053 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-15T12:46:23.086 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-15T12:46:23.114 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-15T12:46:23.183 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-15T12:46:23.218 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:19.0.0-1681.gaba3c1d2.el9. 39/198 2024-03-15T12:46:23.262 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:19.0.0-1681.gaba3c1d2.el 40/198 2024-03-15T12:46:23.291 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:19.0.0-1681.gaba3c1d2.el9.x86_6 41/198 2024-03-15T12:46:23.352 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-15T12:46:23.406 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-15T12:46:23.479 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-15T12:46:23.613 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-15T12:46:23.666 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-15T12:46:23.884 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-15T12:46:23.969 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-15T12:46:24.004 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-15T12:46:24.042 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-15T12:46:24.052 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-15T12:46:24.117 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-15T12:46:24.157 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 130/198 2024-03-15T12:46:24.172 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-15T12:46:24.190 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-15T12:46:24.198 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 130/198 2024-03-15T12:46:24.213 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-15T12:46:24.247 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-15T12:46:24.270 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-15T12:46:24.299 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-15T12:46:24.360 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-15T12:46:24.417 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-15T12:46:24.425 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-15T12:46:24.461 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-15T12:46:24.506 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-15T12:46:24.544 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_6 53/198 2024-03-15T12:46:24.574 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-15T12:46:24.643 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-15T12:46:24.681 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 10/198 2024-03-15T12:46:24.737 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 130/198 2024-03-15T12:46:24.776 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 131/198 2024-03-15T12:46:24.789 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 10/198 2024-03-15T12:46:24.824 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 11/198 2024-03-15T12:46:24.853 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 131/198 2024-03-15T12:46:24.853 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-15T12:46:24.853 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-15T12:46:24.854 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:46:24.857 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-15T12:46:24.894 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 132/198 2024-03-15T12:46:24.920 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 11/198 2024-03-15T12:46:24.958 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rados-2:19.0.0-1681.gaba3c1d2.el9.x86_64 12/198 2024-03-15T12:46:25.001 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-15T12:46:25.039 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-15T12:46:25.110 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-15T12:46:25.154 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-15T12:46:25.188 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-15T12:46:25.235 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-15T12:46:25.262 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 16/198 2024-03-15T12:46:25.294 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-15T12:46:25.348 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-15T12:46:25.367 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 16/198 2024-03-15T12:46:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86 17/198 2024-03-15T12:46:25.403 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-15T12:46:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86 17/198 2024-03-15T12:46:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-15T12:46:25.501 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-15T12:46:25.523 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-15T12:46:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-15T12:46:25.539 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-15T12:46:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-15T12:46:25.607 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-15T12:46:25.643 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-15T12:46:25.686 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-15T12:46:25.689 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-15T12:46:25.721 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-15T12:46:25.728 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-15T12:46:25.761 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-15T12:46:25.871 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-15T12:46:25.917 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-15T12:46:25.944 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-15T12:46:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-15T12:46:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-15T12:46:26.079 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-15T12:46:26.084 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-15T12:46:26.123 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-15T12:46:26.139 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-15T12:46:26.164 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-15T12:46:26.189 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-15T12:46:26.213 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-15T12:46:26.250 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-15T12:46:26.293 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-15T12:46:26.299 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-15T12:46:26.415 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-15T12:46:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-15T12:46:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-15T12:46:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-15T12:46:26.604 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-15T12:46:26.624 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-15T12:46:26.658 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-15T12:46:26.729 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-15T12:46:26.762 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-15T12:46:26.789 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-15T12:46:26.825 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-15T12:46:26.843 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-15T12:46:26.892 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-15T12:46:26.894 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-15T12:46:26.954 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-15T12:46:26.991 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-15T12:46:27.019 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-15T12:46:27.069 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-15T12:46:27.084 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-15T12:46:27.106 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-15T12:46:27.124 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-15T12:46:27.136 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-15T12:46:27.165 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-15T12:46:27.167 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-15T12:46:27.199 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-15T12:46:27.200 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-15T12:46:27.227 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-15T12:46:27.238 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-15T12:46:27.270 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-15T12:46:27.291 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-15T12:46:27.300 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-15T12:46:27.324 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-common-2:19.0.0-1681.gaba3c1d2.el9. 39/198 2024-03-15T12:46:27.359 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-argparse-2:19.0.0-1681.gaba3c1d2.el 40/198 2024-03-15T12:46:27.377 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-15T12:46:27.387 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cephfs-2:19.0.0-1681.gaba3c1d2.el9.x86_6 41/198 2024-03-15T12:46:27.410 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-15T12:46:27.444 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-15T12:46:27.461 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-15T12:46:27.477 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-15T12:46:27.511 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-15T12:46:27.511 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-15T12:46:27.588 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-15T12:46:27.602 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-15T12:46:27.691 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-15T12:46:27.726 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-15T12:46:27.742 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-15T12:46:27.773 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-15T12:46:27.781 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-15T12:46:27.821 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-15T12:46:27.919 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-15T12:46:27.975 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-15T12:46:28.343 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-15T12:46:28.427 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-15T12:46:28.510 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-15T12:46:28.577 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-15T12:46:28.585 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-15T12:46:28.633 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-15T12:46:28.681 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-15T12:46:28.708 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-15T12:46:28.721 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librados-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_6 53/198 2024-03-15T12:46:28.747 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-15T12:46:28.751 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-15T12:46:28.781 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-15T12:46:28.829 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-15T12:46:28.917 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-15T12:46:28.952 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-15T12:46:29.047 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-15T12:46:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-15T12:46:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-15T12:46:29.213 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-15T12:46:29.236 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-15T12:46:29.286 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-15T12:46:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-15T12:46:29.475 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-15T12:46:29.542 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-15T12:46:29.598 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-15T12:46:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-15T12:46:29.709 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-15T12:46:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-15T12:46:29.795 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-15T12:46:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-15T12:46:29.873 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-15T12:46:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-15T12:46:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-15T12:46:29.989 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-15T12:46:30.057 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-15T12:46:30.105 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-15T12:46:30.134 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-15T12:46:30.177 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-15T12:46:30.274 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-15T12:46:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-15T12:46:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-15T12:46:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-15T12:46:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-15T12:46:30.491 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-15T12:46:30.621 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-15T12:46:30.739 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-15T12:46:30.783 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-15T12:46:30.840 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-15T12:46:30.897 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-15T12:46:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-15T12:46:31.137 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-15T12:46:31.234 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-15T12:46:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-15T12:46:31.317 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-15T12:46:31.328 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-15T12:46:31.349 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-15T12:46:31.358 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-15T12:46:31.379 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-15T12:46:31.389 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-15T12:46:31.407 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-15T12:46:31.409 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-15T12:46:31.443 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-15T12:46:31.482 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-15T12:46:31.514 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-15T12:46:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-15T12:46:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-15T12:46:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-15T12:46:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-15T12:46:31.716 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-15T12:46:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-15T12:46:31.745 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 124/198 2024-03-15T12:46:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-15T12:46:31.835 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 124/198 2024-03-15T12:46:31.858 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-15T12:46:31.867 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 125/198 2024-03-15T12:46:31.903 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-15T12:46:31.964 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-15T12:46:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-15T12:46:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-15T12:46:32.090 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-15T12:46:32.192 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-15T12:46:32.245 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-15T12:46:32.348 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-15T12:46:32.374 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 128/198 2024-03-15T12:46:32.461 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 128/198 2024-03-15T12:46:32.492 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 129/198 2024-03-15T12:46:33.030 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-15T12:46:33.074 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-15T12:46:33.108 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-15T12:46:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-15T12:46:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-15T12:46:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-15T12:46:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-15T12:46:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-15T12:46:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-15T12:46:34.256 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-15T12:46:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-15T12:46:34.646 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 130/198 2024-03-15T12:46:34.680 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 130/198 2024-03-15T12:46:35.223 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 130/198 2024-03-15T12:46:35.258 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 131/198 2024-03-15T12:46:35.351 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 131/198 2024-03-15T12:46:35.351 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-15T12:46:35.351 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-15T12:46:35.351 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:46:35.384 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 132/198 2024-03-15T12:46:35.676 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-15T12:46:35.765 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-15T12:46:35.796 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-15T12:46:35.827 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-15T12:46:35.854 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-15T12:46:36.179 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-15T12:46:36.217 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 124/198 2024-03-15T12:46:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 124/198 2024-03-15T12:46:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 125/198 2024-03-15T12:46:36.339 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 132/198 2024-03-15T12:46:36.339 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2024-03-15T12:46:36.340 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2024-03-15T12:46:36.340 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2024-03-15T12:46:36.340 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2024-03-15T12:46:36.340 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2024-03-15T12:46:36.340 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /root 2024-03-15T12:46:36.340 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2024-03-15T12:46:36.340 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:46:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-15T12:46:36.506 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 133/198 2024-03-15T12:46:36.524 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 133/198 2024-03-15T12:46:36.524 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:36.524 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-15T12:46:36.524 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:46:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-15T12:46:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 128/198 2024-03-15T12:46:36.958 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 128/198 2024-03-15T12:46:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 129/198 2024-03-15T12:46:38.001 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 134/198 2024-03-15T12:46:38.026 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 134/198 2024-03-15T12:46:38.026 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:38.026 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-15T12:46:38.026 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-15T12:46:38.027 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-15T12:46:38.027 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:46:38.251 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 135/198 2024-03-15T12:46:38.274 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 135/198 2024-03-15T12:46:38.274 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:38.274 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-15T12:46:38.274 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-15T12:46:38.274 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-15T12:46:38.274 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:46:38.727 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 136/198 2024-03-15T12:46:38.750 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 136/198 2024-03-15T12:46:38.750 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:38.750 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-15T12:46:38.750 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-15T12:46:38.750 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-15T12:46:38.750 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:46:38.854 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-15T12:46:39.254 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 138/198 2024-03-15T12:46:39.259 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 130/198 2024-03-15T12:46:39.293 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 130/198 2024-03-15T12:46:39.294 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 138/198 2024-03-15T12:46:39.334 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1681.gaba3c1d2.e 139/198 2024-03-15T12:46:39.394 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1681.gaba3c1d2. 140/198 2024-03-15T12:46:39.417 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noa 141/198 2024-03-15T12:46:39.650 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noa 141/198 2024-03-15T12:46:39.685 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.n 142/198 2024-03-15T12:46:39.860 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 130/198 2024-03-15T12:46:39.896 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 131/198 2024-03-15T12:46:39.988 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 131/198 2024-03-15T12:46:39.988 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-15T12:46:39.988 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-15T12:46:39.988 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:46:40.022 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 132/198 2024-03-15T12:46:40.650 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.n 142/198 2024-03-15T12:46:40.677 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba 143/198 2024-03-15T12:46:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba 143/198 2024-03-15T12:46:40.941 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1681.gaba3c1d2.el 144/198 2024-03-15T12:46:40.971 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 145/198 2024-03-15T12:46:40.994 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 145/198 2024-03-15T12:46:40.994 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:40.994 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-15T12:46:40.994 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-15T12:46:40.994 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-15T12:46:40.994 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:46:41.039 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 146/198 2024-03-15T12:46:41.050 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 146/198 2024-03-15T12:46:41.097 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:19.0.0-1681.gaba3c1d2.el9.x86_64 147/198 2024-03-15T12:46:41.947 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-15T12:46:41.982 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 149/198 2024-03-15T12:46:42.005 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 149/198 2024-03-15T12:46:42.005 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:42.005 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-15T12:46:42.005 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-15T12:46:42.005 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-15T12:46:42.005 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:46:42.058 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1681.gaba3c 150/198 2024-03-15T12:46:42.080 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1681.gaba3c 150/198 2024-03-15T12:46:42.080 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:42.080 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-15T12:46:42.080 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:46:42.142 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 151/198 2024-03-15T12:46:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 151/198 2024-03-15T12:46:42.164 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:42.164 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-15T12:46:42.164 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-15T12:46:42.164 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-15T12:46:42.164 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:46:42.484 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 152/198 2024-03-15T12:46:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 152/198 2024-03-15T12:46:42.507 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:42.507 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-15T12:46:42.507 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-15T12:46:42.507 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-15T12:46:42.507 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:46:46.930 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:19.0.0-1681.gaba3c1d2.el9.x86_64 153/198 2024-03-15T12:46:46.979 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 154/198 2024-03-15T12:46:47.010 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 155/198 2024-03-15T12:46:47.037 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-15T12:46:47.229 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-shell-2:19.0.0-1681.gaba3c1d2.el9.x86_64 157/198 2024-03-15T12:46:47.344 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-15T12:46:47.385 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 159/198 2024-03-15T12:46:47.446 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-15T12:46:47.468 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 132/198 2024-03-15T12:46:47.468 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2024-03-15T12:46:47.468 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2024-03-15T12:46:47.468 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2024-03-15T12:46:47.468 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2024-03-15T12:46:47.468 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2024-03-15T12:46:47.469 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /root 2024-03-15T12:46:47.469 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2024-03-15T12:46:47.469 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:46:47.510 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-15T12:46:47.553 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-15T12:46:47.578 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_ 163/198 2024-03-15T12:46:47.611 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-15T12:46:47.640 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-15T12:46:47.642 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 133/198 2024-03-15T12:46:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 133/198 2024-03-15T12:46:47.657 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:47.657 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-15T12:46:47.657 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:46:47.708 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-15T12:46:47.779 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-15T12:46:47.815 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-15T12:46:48.470 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-15T12:46:48.513 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-15T12:46:48.533 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-15T12:46:49.142 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 134/198 2024-03-15T12:46:49.167 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 134/198 2024-03-15T12:46:49.167 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:49.167 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-15T12:46:49.167 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-15T12:46:49.168 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-15T12:46:49.168 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:46:49.342 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-top-2:19.0.0-1681.gaba3c1d2.el9.noarch 170/198 2024-03-15T12:46:49.385 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 135/198 2024-03-15T12:46:49.408 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 135/198 2024-03-15T12:46:49.408 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:49.408 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-15T12:46:49.408 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-15T12:46:49.408 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-15T12:46:49.408 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:46:49.424 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-15T12:46:49.465 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-15T12:46:49.556 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-15T12:46:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-15T12:46:49.611 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-15T12:46:49.637 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-15T12:46:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-15T12:46:49.663 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-15T12:46:49.751 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-15T12:46:49.789 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-15T12:46:49.789 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-15T12:46:49.862 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 136/198 2024-03-15T12:46:49.874 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-15T12:46:49.884 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 136/198 2024-03-15T12:46:49.884 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:49.884 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-15T12:46:49.884 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-15T12:46:49.885 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-15T12:46:49.885 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:46:49.901 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-15T12:46:49.933 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-15T12:46:49.959 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-15T12:46:49.972 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-15T12:46:49.983 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-15T12:46:50.017 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-15T12:46:50.062 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-15T12:46:50.106 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-15T12:46:50.147 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-15T12:46:50.172 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-15T12:46:50.198 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-15T12:46:50.223 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-15T12:46:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-15T12:46:50.230 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-15T12:46:50.406 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 138/198 2024-03-15T12:46:50.409 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-15T12:46:50.409 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-15T12:46:50.437 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 138/198 2024-03-15T12:46:50.468 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1681.gaba3c1d2.e 139/198 2024-03-15T12:46:50.492 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-15T12:46:50.517 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-15T12:46:50.528 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1681.gaba3c1d2. 140/198 2024-03-15T12:46:50.542 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-15T12:46:50.552 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noa 141/198 2024-03-15T12:46:50.575 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-15T12:46:50.601 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-15T12:46:50.669 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-15T12:46:50.782 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noa 141/198 2024-03-15T12:46:50.811 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.n 142/198 2024-03-15T12:46:51.770 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.n 142/198 2024-03-15T12:46:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba 143/198 2024-03-15T12:46:51.884 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 132/198 2024-03-15T12:46:51.884 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /sys 2024-03-15T12:46:51.884 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /proc 2024-03-15T12:46:51.884 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /mnt 2024-03-15T12:46:51.884 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /var/tmp 2024-03-15T12:46:51.885 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /home 2024-03-15T12:46:51.885 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /root 2024-03-15T12:46:51.885 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /tmp 2024-03-15T12:46:51.885 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:46:51.896 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba 143/198 2024-03-15T12:46:52.051 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 133/198 2024-03-15T12:46:52.056 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1681.gaba3c1d2.el 144/198 2024-03-15T12:46:52.067 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 133/198 2024-03-15T12:46:52.067 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:52.067 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-15T12:46:52.067 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:46:52.080 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 145/198 2024-03-15T12:46:52.103 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 145/198 2024-03-15T12:46:52.103 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:52.103 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-15T12:46:52.103 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-15T12:46:52.103 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-15T12:46:52.104 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:46:52.148 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 146/198 2024-03-15T12:46:52.158 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 146/198 2024-03-15T12:46:52.198 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:19.0.0-1681.gaba3c1d2.el9.x86_64 147/198 2024-03-15T12:46:53.057 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-15T12:46:53.091 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 149/198 2024-03-15T12:46:53.114 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 149/198 2024-03-15T12:46:53.114 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:53.114 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-15T12:46:53.114 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-15T12:46:53.114 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-15T12:46:53.114 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:46:53.167 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1681.gaba3c 150/198 2024-03-15T12:46:53.189 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1681.gaba3c 150/198 2024-03-15T12:46:53.189 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:53.189 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-15T12:46:53.189 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:46:53.243 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 151/198 2024-03-15T12:46:53.265 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 151/198 2024-03-15T12:46:53.265 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:53.265 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-15T12:46:53.265 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-15T12:46:53.265 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-15T12:46:53.265 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:46:53.577 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 152/198 2024-03-15T12:46:53.600 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 152/198 2024-03-15T12:46:53.600 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:53.600 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-15T12:46:53.600 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-15T12:46:53.600 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-15T12:46:53.600 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:46:53.621 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 134/198 2024-03-15T12:46:53.646 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 134/198 2024-03-15T12:46:53.646 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:53.646 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-15T12:46:53.646 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-15T12:46:53.646 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-15T12:46:53.647 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:46:53.880 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 135/198 2024-03-15T12:46:53.902 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 135/198 2024-03-15T12:46:53.902 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:53.902 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-15T12:46:53.902 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-15T12:46:53.902 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-15T12:46:53.902 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:46:54.364 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 136/198 2024-03-15T12:46:54.386 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 136/198 2024-03-15T12:46:54.386 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:54.386 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-15T12:46:54.386 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-15T12:46:54.387 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-15T12:46:54.387 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:46:54.474 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-15T12:46:54.909 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 138/198 2024-03-15T12:46:54.939 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 138/198 2024-03-15T12:46:54.971 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1681.gaba3c1d2.e 139/198 2024-03-15T12:46:55.033 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1681.gaba3c1d2. 140/198 2024-03-15T12:46:55.062 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noa 141/198 2024-03-15T12:46:55.304 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noa 141/198 2024-03-15T12:46:55.338 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.n 142/198 2024-03-15T12:46:55.588 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-15T12:46:55.589 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:19.0.0-1681.gaba3c1d2.el9.x86_64 1/198 2024-03-15T12:46:55.589 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2/198 2024-03-15T12:46:55.589 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 3/198 2024-03-15T12:46:55.589 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 4/198 2024-03-15T12:46:55.589 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1681.gaba3c 5/198 2024-03-15T12:46:55.589 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 6/198 2024-03-15T12:46:55.589 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 7/198 2024-03-15T12:46:55.589 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 8/198 2024-03-15T12:46:55.589 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 9/198 2024-03-15T12:46:55.589 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 10/198 2024-03-15T12:46:55.589 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 11/198 2024-03-15T12:46:55.589 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:19.0.0-1681.gaba3c1d2.el9.x86_64 12/198 2024-03-15T12:46:55.589 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 13/198 2024-03-15T12:46:55.589 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-shell-2:19.0.0-1681.gaba3c1d2.el9.x86_64 14/198 2024-03-15T12:46:55.590 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_ 15/198 2024-03-15T12:46:55.590 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 16/198 2024-03-15T12:46:55.590 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 17/198 2024-03-15T12:46:55.590 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_6 18/198 2024-03-15T12:46:55.590 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86 19/198 2024-03-15T12:46:55.590 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 20/198 2024-03-15T12:46:55.590 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1681.gaba3c1d2.el 21/198 2024-03-15T12:46:55.590 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:19.0.0-1681.gaba3c1d2.el9. 22/198 2024-03-15T12:46:55.590 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:19.0.0-1681.gaba3c1d2.el9.x86_6 23/198 2024-03-15T12:46:55.590 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:19.0.0-1681.gaba3c1d2.el9.x86_64 24/198 2024-03-15T12:46:55.590 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 25/198 2024-03-15T12:46:55.590 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 26/198 2024-03-15T12:46:55.590 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 27/198 2024-03-15T12:46:55.591 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 28/198 2024-03-15T12:46:55.591 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 29/198 2024-03-15T12:46:55.591 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1681.gaba3c1d2. 30/198 2024-03-15T12:46:55.591 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noa 31/198 2024-03-15T12:46:55.591 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.n 32/198 2024-03-15T12:46:55.591 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba 33/198 2024-03-15T12:46:55.591 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1681.gaba3c1d2.el 34/198 2024-03-15T12:46:55.591 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 35/198 2024-03-15T12:46:55.591 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1681.gaba3c1d2.e 36/198 2024-03-15T12:46:55.591 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 37/198 2024-03-15T12:46:55.591 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 38/198 2024-03-15T12:46:55.591 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-top-2:19.0.0-1681.gaba3c1d2.el9.noarch 39/198 2024-03-15T12:46:55.591 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-15T12:46:55.592 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-15T12:46:55.592 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-15T12:46:55.592 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-15T12:46:55.592 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-15T12:46:55.592 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-15T12:46:55.592 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-15T12:46:55.592 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-15T12:46:55.592 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-15T12:46:55.592 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-15T12:46:55.592 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-15T12:46:55.592 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-15T12:46:55.592 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-15T12:46:55.592 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-15T12:46:55.592 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-15T12:46:55.593 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-15T12:46:55.593 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-15T12:46:55.593 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-15T12:46:55.593 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-15T12:46:55.593 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-15T12:46:55.593 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-15T12:46:55.593 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-15T12:46:55.593 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-15T12:46:55.593 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-15T12:46:55.593 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-15T12:46:55.593 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-15T12:46:55.593 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-15T12:46:55.593 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-15T12:46:55.593 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-15T12:46:55.594 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-15T12:46:55.594 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-15T12:46:55.594 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-15T12:46:55.594 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-15T12:46:55.594 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-15T12:46:55.594 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-15T12:46:55.594 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-15T12:46:55.594 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-15T12:46:55.594 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-15T12:46:55.594 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-15T12:46:55.594 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-15T12:46:55.594 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-15T12:46:55.594 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-15T12:46:55.594 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-15T12:46:55.595 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-15T12:46:55.595 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-15T12:46:55.595 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-15T12:46:55.595 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-15T12:46:55.595 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-15T12:46:55.595 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-15T12:46:55.595 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-15T12:46:55.595 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-15T12:46:55.595 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-15T12:46:55.595 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-15T12:46:55.595 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-15T12:46:55.595 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-15T12:46:55.595 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-15T12:46:55.596 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-15T12:46:55.596 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-15T12:46:55.596 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-15T12:46:55.596 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-15T12:46:55.596 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-15T12:46:55.596 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-15T12:46:55.596 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-15T12:46:55.596 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-15T12:46:55.596 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-15T12:46:55.596 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-15T12:46:55.596 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-15T12:46:55.596 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-15T12:46:55.596 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-15T12:46:55.597 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-15T12:46:55.597 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-15T12:46:55.597 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-15T12:46:55.597 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-15T12:46:55.597 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-15T12:46:55.597 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-15T12:46:55.597 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-15T12:46:55.597 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-15T12:46:55.597 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-15T12:46:55.597 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-15T12:46:55.597 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-15T12:46:55.597 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-15T12:46:55.597 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-15T12:46:55.597 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-15T12:46:55.598 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-15T12:46:55.598 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-15T12:46:55.598 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-15T12:46:55.598 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-15T12:46:55.598 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-15T12:46:55.598 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-15T12:46:55.598 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-15T12:46:55.598 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-15T12:46:55.598 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-15T12:46:55.598 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-15T12:46:55.598 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-15T12:46:55.598 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-15T12:46:55.598 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-15T12:46:55.599 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-15T12:46:55.599 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-15T12:46:55.599 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-15T12:46:55.599 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-15T12:46:55.599 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-15T12:46:55.599 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-15T12:46:55.599 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-15T12:46:55.599 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-15T12:46:55.599 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-15T12:46:55.599 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-15T12:46:55.599 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-15T12:46:55.599 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-15T12:46:55.599 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-15T12:46:55.599 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-15T12:46:55.600 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-15T12:46:55.600 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 151/198 2024-03-15T12:46:55.600 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-15T12:46:55.600 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 153/198 2024-03-15T12:46:55.600 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-15T12:46:55.600 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-15T12:46:55.600 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-15T12:46:55.600 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-15T12:46:55.600 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-15T12:46:55.600 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-15T12:46:55.600 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-15T12:46:55.600 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-15T12:46:55.600 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-15T12:46:55.601 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-15T12:46:55.601 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-15T12:46:55.601 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-15T12:46:55.601 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-15T12:46:55.601 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-15T12:46:55.601 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-15T12:46:55.601 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-15T12:46:55.601 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-15T12:46:55.601 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-15T12:46:55.601 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-15T12:46:55.601 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-15T12:46:55.601 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-15T12:46:55.601 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-15T12:46:55.601 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-15T12:46:55.602 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-15T12:46:55.602 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-15T12:46:55.602 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-15T12:46:55.602 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-15T12:46:55.602 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-15T12:46:55.602 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-15T12:46:55.602 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-15T12:46:55.602 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-15T12:46:55.602 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-15T12:46:55.602 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-15T12:46:55.602 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-15T12:46:55.602 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-15T12:46:55.602 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-15T12:46:55.602 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-15T12:46:55.603 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-15T12:46:55.603 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-15T12:46:55.603 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-15T12:46:55.603 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-15T12:46:55.603 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-15T12:46:55.603 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-15T12:46:55.603 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-15T12:46:56.353 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.n 142/198 2024-03-15T12:46:56.371 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba 143/198 2024-03-15T12:46:56.478 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba 143/198 2024-03-15T12:46:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1681.gaba3c1d2.el 144/198 2024-03-15T12:46:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 145/198 2024-03-15T12:46:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 145/198 2024-03-15T12:46:56.705 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:56.705 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-15T12:46:56.705 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-15T12:46:56.705 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-15T12:46:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:46:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 146/198 2024-03-15T12:46:56.760 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 146/198 2024-03-15T12:46:56.812 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-2:19.0.0-1681.gaba3c1d2.el9.x86_64 147/198 2024-03-15T12:46:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-15T12:46:57.768 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 149/198 2024-03-15T12:46:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 149/198 2024-03-15T12:46:57.790 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:57.790 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-15T12:46:57.790 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-15T12:46:57.790 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-15T12:46:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:46:57.835 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1681.gaba3c 150/198 2024-03-15T12:46:57.857 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1681.gaba3c 150/198 2024-03-15T12:46:57.857 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:57.857 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-15T12:46:57.857 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:46:57.911 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 151/198 2024-03-15T12:46:57.933 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 151/198 2024-03-15T12:46:57.933 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:57.933 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-15T12:46:57.933 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-15T12:46:57.933 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-15T12:46:57.933 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:46:58.023 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:19.0.0-1681.gaba3c1d2.el9.x86_64 153/198 2024-03-15T12:46:58.064 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 154/198 2024-03-15T12:46:58.095 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 155/198 2024-03-15T12:46:58.139 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-15T12:46:58.253 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 152/198 2024-03-15T12:46:58.275 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 152/198 2024-03-15T12:46:58.275 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-15T12:46:58.275 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-15T12:46:58.275 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-15T12:46:58.276 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-15T12:46:58.276 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:46:58.331 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-shell-2:19.0.0-1681.gaba3c1d2.el9.x86_64 157/198 2024-03-15T12:46:58.446 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-15T12:46:58.487 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 159/198 2024-03-15T12:46:58.538 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-15T12:46:58.595 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-15T12:46:58.638 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-15T12:46:58.663 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_ 163/198 2024-03-15T12:46:58.695 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-15T12:46:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-15T12:46:58.734 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-15T12:46:58.734 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:46:58.735 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2024-03-15T12:46:58.735 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-15T12:46:58.735 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-15T12:46:58.735 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-15T12:46:58.735 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-15T12:46:58.735 INFO:teuthology.orchestra.run.smithi184.stdout: acl-2.3.1-4.el9.x86_64 2024-03-15T12:46:58.735 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-15T12:46:58.735 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-15T12:46:58.735 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-15T12:46:58.735 INFO:teuthology.orchestra.run.smithi184.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-15T12:46:58.735 INFO:teuthology.orchestra.run.smithi184.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-15T12:46:58.735 INFO:teuthology.orchestra.run.smithi184.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-15T12:46:58.735 INFO:teuthology.orchestra.run.smithi184.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-15T12:46:58.736 INFO:teuthology.orchestra.run.smithi184.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-15T12:46:58.736 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.736 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.736 INFO:teuthology.orchestra.run.smithi184.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-15T12:46:58.736 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-15T12:46:58.736 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-15T12:46:58.736 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-15T12:46:58.736 INFO:teuthology.orchestra.run.smithi184.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-15T12:46:58.736 INFO:teuthology.orchestra.run.smithi184.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-15T12:46:58.736 INFO:teuthology.orchestra.run.smithi184.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-15T12:46:58.736 INFO:teuthology.orchestra.run.smithi184.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-15T12:46:58.736 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-15T12:46:58.737 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-15T12:46:58.737 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-15T12:46:58.737 INFO:teuthology.orchestra.run.smithi184.stdout: automake-1.16.2-8.el9.noarch 2024-03-15T12:46:58.737 INFO:teuthology.orchestra.run.smithi184.stdout: bison-3.7.4-5.el9.x86_64 2024-03-15T12:46:58.737 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-15T12:46:58.737 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.737 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.737 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.737 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.737 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:46:58.737 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.737 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.737 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.738 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:46:58.738 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:46:58.738 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:46:58.738 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:46:58.738 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:46:58.738 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.738 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.738 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:46:58.738 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.738 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.738 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.738 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:46:58.738 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:46:58.739 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.739 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.739 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:46:58.739 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-15T12:46:58.739 INFO:teuthology.orchestra.run.smithi184.stdout: flex-2.6.4-9.el9.x86_64 2024-03-15T12:46:58.739 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-15T12:46:58.739 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-15T12:46:58.739 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-15T12:46:58.739 INFO:teuthology.orchestra.run.smithi184.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-15T12:46:58.739 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-15T12:46:58.739 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-15T12:46:58.739 INFO:teuthology.orchestra.run.smithi184.stdout: jq-1.6-16.el9.x86_64 2024-03-15T12:46:58.739 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-15T12:46:58.739 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-15T12:46:58.740 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-15T12:46:58.740 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.740 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.740 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.740 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-15T12:46:58.740 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-15T12:46:58.740 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-15T12:46:58.740 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-15T12:46:58.740 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-15T12:46:58.740 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.740 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.740 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-15T12:46:58.740 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.740 INFO:teuthology.orchestra.run.smithi184.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-15T12:46:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-15T12:46:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-15T12:46:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-15T12:46:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-15T12:46:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: lua-5.4.4-4.el9.x86_64 2024-03-15T12:46:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-15T12:46:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-15T12:46:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-15T12:46:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-15T12:46:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-15T12:46:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-15T12:46:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-15T12:46:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-13.11-1.el9.x86_64 2024-03-15T12:46:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-15T12:46:58.741 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-15T12:46:58.742 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-15T12:46:58.742 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-15T12:46:58.742 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-15T12:46:58.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-15T12:46:58.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-15T12:46:58.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-15T12:46:58.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-15T12:46:58.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-15T12:46:58.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-15T12:46:58.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-15T12:46:58.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-15T12:46:58.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-15T12:46:58.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-15T12:46:58.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-15T12:46:58.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-15T12:46:58.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-15T12:46:58.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-15T12:46:58.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-15T12:46:58.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-15T12:46:58.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-15T12:46:58.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-15T12:46:58.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-15T12:46:58.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-15T12:46:58.743 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-15T12:46:58.744 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-15T12:46:58.744 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-15T12:46:58.744 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-15T12:46:58.744 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-15T12:46:58.744 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-15T12:46:58.744 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-15T12:46:58.744 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-15T12:46:58.744 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-15T12:46:58.744 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-15T12:46:58.744 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-15T12:46:58.744 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-15T12:46:58.744 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-15T12:46:58.744 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-15T12:46:58.744 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-15T12:46:58.744 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply-3.11-14.el9.noarch 2024-03-15T12:46:58.745 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-15T12:46:58.745 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-15T12:46:58.745 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-15T12:46:58.745 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-15T12:46:58.745 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-15T12:46:58.745 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-15T12:46:58.745 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-15T12:46:58.745 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-15T12:46:58.745 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-15T12:46:58.745 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.745 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.745 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-15T12:46:58.745 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-15T12:46:58.745 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-15T12:46:58.746 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.746 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-15T12:46:58.746 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-15T12:46:58.746 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-15T12:46:58.746 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-15T12:46:58.746 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-15T12:46:58.746 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-15T12:46:58.746 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-15T12:46:58.746 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-15T12:46:58.746 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-15T12:46:58.746 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-15T12:46:58.746 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-15T12:46:58.746 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-15T12:46:58.746 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-15T12:46:58.747 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-15T12:46:58.747 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-15T12:46:58.747 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.747 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.747 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:46:58.747 INFO:teuthology.orchestra.run.smithi184.stdout: re2-1:20211101-3.el9.x86_64 2024-03-15T12:46:58.747 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-15T12:46:58.747 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-15T12:46:58.747 INFO:teuthology.orchestra.run.smithi184.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-15T12:46:58.747 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-15T12:46:58.747 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-15T12:46:58.748 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-15T12:46:58.748 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:46:58.748 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-15T12:46:58.795 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-15T12:46:58.853 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-15T12:46:58.890 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-15T12:46:59.266 DEBUG:teuthology.parallel:result is None 2024-03-15T12:46:59.496 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-15T12:46:59.530 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-15T12:46:59.549 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-15T12:47:00.389 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-top-2:19.0.0-1681.gaba3c1d2.el9.noarch 170/198 2024-03-15T12:47:00.467 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-15T12:47:00.509 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-15T12:47:00.598 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-15T12:47:00.654 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-15T12:47:00.679 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-15T12:47:00.704 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-15T12:47:00.731 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-15T12:47:00.731 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-15T12:47:00.844 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-15T12:47:00.882 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-15T12:47:00.882 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-15T12:47:00.967 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-15T12:47:00.998 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-15T12:47:01.026 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-15T12:47:01.051 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-15T12:47:01.076 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-15T12:47:01.110 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-15T12:47:01.161 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-15T12:47:01.200 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-15T12:47:01.225 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-15T12:47:01.250 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-15T12:47:01.275 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-15T12:47:01.301 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-15T12:47:01.307 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-15T12:47:01.307 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-15T12:47:01.487 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-15T12:47:01.487 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-15T12:47:01.570 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-15T12:47:01.595 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-15T12:47:01.620 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-15T12:47:01.653 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-15T12:47:01.679 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-15T12:47:01.746 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-15T12:47:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-test-2:19.0.0-1681.gaba3c1d2.el9.x86_64 153/198 2024-03-15T12:47:03.007 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 154/198 2024-03-15T12:47:03.038 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-nbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 155/198 2024-03-15T12:47:03.065 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-15T12:47:03.277 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-shell-2:19.0.0-1681.gaba3c1d2.el9.x86_64 157/198 2024-03-15T12:47:03.393 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-15T12:47:03.438 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 159/198 2024-03-15T12:47:03.486 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-15T12:47:03.546 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-15T12:47:03.581 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-15T12:47:03.605 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_ 163/198 2024-03-15T12:47:03.638 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-15T12:47:03.667 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-15T12:47:03.728 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-15T12:47:03.784 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-15T12:47:03.815 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-15T12:47:04.437 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-15T12:47:04.479 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-15T12:47:04.498 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-15T12:47:05.377 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-top-2:19.0.0-1681.gaba3c1d2.el9.noarch 170/198 2024-03-15T12:47:05.450 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-15T12:47:05.483 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-15T12:47:05.570 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-15T12:47:05.603 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-15T12:47:05.628 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-15T12:47:05.653 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-15T12:47:05.680 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-15T12:47:05.680 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-15T12:47:05.767 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-15T12:47:05.797 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-15T12:47:05.797 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-15T12:47:05.882 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-15T12:47:05.911 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-15T12:47:05.941 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-15T12:47:05.967 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-15T12:47:05.992 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-15T12:47:06.027 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-15T12:47:06.073 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-15T12:47:06.104 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-15T12:47:06.130 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-15T12:47:06.155 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-15T12:47:06.180 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-15T12:47:06.206 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-15T12:47:06.213 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-15T12:47:06.213 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-15T12:47:06.392 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-15T12:47:06.392 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-15T12:47:06.491 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-15T12:47:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-15T12:47:06.541 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-15T12:47:06.575 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-15T12:47:06.601 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-15T12:47:06.669 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-15T12:47:06.672 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-15T12:47:06.672 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:19.0.0-1681.gaba3c1d2.el9.x86_64 1/198 2024-03-15T12:47:06.672 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2/198 2024-03-15T12:47:06.673 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 3/198 2024-03-15T12:47:06.673 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 4/198 2024-03-15T12:47:06.673 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1681.gaba3c 5/198 2024-03-15T12:47:06.673 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 6/198 2024-03-15T12:47:06.673 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 7/198 2024-03-15T12:47:06.673 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 8/198 2024-03-15T12:47:06.673 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 9/198 2024-03-15T12:47:06.673 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 10/198 2024-03-15T12:47:06.673 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 11/198 2024-03-15T12:47:06.673 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:19.0.0-1681.gaba3c1d2.el9.x86_64 12/198 2024-03-15T12:47:06.673 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 13/198 2024-03-15T12:47:06.673 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-shell-2:19.0.0-1681.gaba3c1d2.el9.x86_64 14/198 2024-03-15T12:47:06.673 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_ 15/198 2024-03-15T12:47:06.673 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 16/198 2024-03-15T12:47:06.674 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 17/198 2024-03-15T12:47:06.674 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_6 18/198 2024-03-15T12:47:06.674 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86 19/198 2024-03-15T12:47:06.674 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 20/198 2024-03-15T12:47:06.674 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1681.gaba3c1d2.el 21/198 2024-03-15T12:47:06.674 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:19.0.0-1681.gaba3c1d2.el9. 22/198 2024-03-15T12:47:06.674 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:19.0.0-1681.gaba3c1d2.el9.x86_6 23/198 2024-03-15T12:47:06.674 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:19.0.0-1681.gaba3c1d2.el9.x86_64 24/198 2024-03-15T12:47:06.674 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 25/198 2024-03-15T12:47:06.677 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 26/198 2024-03-15T12:47:06.677 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 27/198 2024-03-15T12:47:06.692 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 28/198 2024-03-15T12:47:06.692 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 29/198 2024-03-15T12:47:06.692 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1681.gaba3c1d2. 30/198 2024-03-15T12:47:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noa 31/198 2024-03-15T12:47:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.n 32/198 2024-03-15T12:47:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba 33/198 2024-03-15T12:47:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1681.gaba3c1d2.el 34/198 2024-03-15T12:47:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 35/198 2024-03-15T12:47:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1681.gaba3c1d2.e 36/198 2024-03-15T12:47:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 37/198 2024-03-15T12:47:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 38/198 2024-03-15T12:47:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-top-2:19.0.0-1681.gaba3c1d2.el9.noarch 39/198 2024-03-15T12:47:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-15T12:47:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-15T12:47:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-15T12:47:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-15T12:47:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-15T12:47:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-15T12:47:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-15T12:47:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-15T12:47:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-15T12:47:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-15T12:47:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-15T12:47:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-15T12:47:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-15T12:47:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-15T12:47:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-15T12:47:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-15T12:47:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-15T12:47:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-15T12:47:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-15T12:47:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-15T12:47:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-15T12:47:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-15T12:47:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-15T12:47:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-15T12:47:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-15T12:47:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-15T12:47:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-15T12:47:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-15T12:47:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-15T12:47:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-15T12:47:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-15T12:47:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-15T12:47:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-15T12:47:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-15T12:47:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-15T12:47:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-15T12:47:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-15T12:47:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-15T12:47:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-15T12:47:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-15T12:47:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-15T12:47:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-15T12:47:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-15T12:47:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-15T12:47:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-15T12:47:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-15T12:47:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-15T12:47:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-15T12:47:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-15T12:47:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-15T12:47:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-15T12:47:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-15T12:47:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-15T12:47:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-15T12:47:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-15T12:47:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-15T12:47:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-15T12:47:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-15T12:47:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-15T12:47:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-15T12:47:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-15T12:47:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-15T12:47:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-15T12:47:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-15T12:47:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-15T12:47:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-15T12:47:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-15T12:47:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-15T12:47:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-15T12:47:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-15T12:47:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-15T12:47:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-15T12:47:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-15T12:47:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-15T12:47:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-15T12:47:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-15T12:47:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-15T12:47:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-15T12:47:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-15T12:47:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-15T12:47:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-15T12:47:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-15T12:47:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-15T12:47:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-15T12:47:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-15T12:47:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-15T12:47:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-15T12:47:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-15T12:47:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-15T12:47:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-15T12:47:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-15T12:47:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-15T12:47:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-15T12:47:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-15T12:47:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-15T12:47:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-15T12:47:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-15T12:47:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-15T12:47:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-15T12:47:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-15T12:47:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-15T12:47:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-15T12:47:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-15T12:47:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-15T12:47:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-15T12:47:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-15T12:47:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-15T12:47:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-15T12:47:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-15T12:47:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-15T12:47:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-15T12:47:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 151/198 2024-03-15T12:47:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-15T12:47:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 153/198 2024-03-15T12:47:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-15T12:47:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-15T12:47:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-15T12:47:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-15T12:47:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-15T12:47:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-15T12:47:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-15T12:47:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-15T12:47:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-15T12:47:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-15T12:47:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-15T12:47:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-15T12:47:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-15T12:47:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-15T12:47:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-15T12:47:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-15T12:47:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-15T12:47:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-15T12:47:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-15T12:47:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-15T12:47:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-15T12:47:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-15T12:47:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-15T12:47:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-15T12:47:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-15T12:47:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-15T12:47:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-15T12:47:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-15T12:47:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-15T12:47:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-15T12:47:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-15T12:47:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-15T12:47:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-15T12:47:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-15T12:47:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-15T12:47:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-15T12:47:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-15T12:47:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-15T12:47:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-15T12:47:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-15T12:47:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-15T12:47:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-15T12:47:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-15T12:47:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-15T12:47:09.751 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-15T12:47:09.751 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:47:09.751 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-03-15T12:47:09.751 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-15T12:47:09.751 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-15T12:47:09.751 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-15T12:47:09.751 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-15T12:47:09.751 INFO:teuthology.orchestra.run.smithi149.stdout: acl-2.3.1-4.el9.x86_64 2024-03-15T12:47:09.751 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-15T12:47:09.751 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-15T12:47:09.752 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-15T12:47:09.752 INFO:teuthology.orchestra.run.smithi149.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-15T12:47:09.752 INFO:teuthology.orchestra.run.smithi149.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-15T12:47:09.752 INFO:teuthology.orchestra.run.smithi149.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-15T12:47:09.752 INFO:teuthology.orchestra.run.smithi149.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-15T12:47:09.752 INFO:teuthology.orchestra.run.smithi149.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-15T12:47:09.752 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.752 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.752 INFO:teuthology.orchestra.run.smithi149.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-15T12:47:09.752 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-15T12:47:09.752 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-15T12:47:09.752 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-15T12:47:09.752 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-15T12:47:09.752 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-15T12:47:09.753 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-15T12:47:09.753 INFO:teuthology.orchestra.run.smithi149.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-15T12:47:09.753 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-15T12:47:09.753 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-15T12:47:09.753 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-15T12:47:09.753 INFO:teuthology.orchestra.run.smithi149.stdout: automake-1.16.2-8.el9.noarch 2024-03-15T12:47:09.753 INFO:teuthology.orchestra.run.smithi149.stdout: bison-3.7.4-5.el9.x86_64 2024-03-15T12:47:09.753 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-15T12:47:09.753 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.753 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.753 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.753 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.753 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:09.754 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.754 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.754 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.754 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:09.754 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:09.754 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:09.754 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:09.754 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:09.754 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.754 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.754 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:09.754 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.754 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.754 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.755 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:09.755 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:09.755 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.755 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.755 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:09.755 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-15T12:47:09.755 INFO:teuthology.orchestra.run.smithi149.stdout: flex-2.6.4-9.el9.x86_64 2024-03-15T12:47:09.755 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-15T12:47:09.755 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-15T12:47:09.755 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-15T12:47:09.755 INFO:teuthology.orchestra.run.smithi149.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-15T12:47:09.755 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-15T12:47:09.755 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-15T12:47:09.755 INFO:teuthology.orchestra.run.smithi149.stdout: jq-1.6-16.el9.x86_64 2024-03-15T12:47:09.756 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-15T12:47:09.756 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-15T12:47:09.756 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-15T12:47:09.756 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.756 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.756 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.756 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-15T12:47:09.756 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-15T12:47:09.756 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-15T12:47:09.756 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-15T12:47:09.756 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-15T12:47:09.756 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.756 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.756 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-15T12:47:09.757 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.757 INFO:teuthology.orchestra.run.smithi149.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-15T12:47:09.757 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-15T12:47:09.757 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-15T12:47:09.757 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-15T12:47:09.757 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-15T12:47:09.757 INFO:teuthology.orchestra.run.smithi149.stdout: lua-5.4.4-4.el9.x86_64 2024-03-15T12:47:09.757 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-15T12:47:09.757 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-15T12:47:09.757 INFO:teuthology.orchestra.run.smithi149.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-15T12:47:09.757 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-15T12:47:09.757 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-15T12:47:09.757 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-15T12:47:09.757 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-15T12:47:09.758 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-13.11-1.el9.x86_64 2024-03-15T12:47:09.758 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-15T12:47:09.758 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-15T12:47:09.758 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-15T12:47:09.758 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-15T12:47:09.758 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-15T12:47:09.758 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-15T12:47:09.758 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-15T12:47:09.758 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-15T12:47:09.758 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-15T12:47:09.758 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-15T12:47:09.758 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.758 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.759 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.759 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-15T12:47:09.759 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-15T12:47:09.759 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-15T12:47:09.759 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-15T12:47:09.759 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-15T12:47:09.759 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-15T12:47:09.759 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-15T12:47:09.759 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-15T12:47:09.759 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-15T12:47:09.759 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-15T12:47:09.759 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-15T12:47:09.759 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-15T12:47:09.759 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-15T12:47:09.760 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-15T12:47:09.760 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-15T12:47:09.760 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-15T12:47:09.760 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-15T12:47:09.760 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-15T12:47:09.760 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-15T12:47:09.760 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-15T12:47:09.760 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-15T12:47:09.760 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-15T12:47:09.760 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-15T12:47:09.760 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-15T12:47:09.760 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-15T12:47:09.760 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-15T12:47:09.760 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-15T12:47:09.761 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-15T12:47:09.761 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-15T12:47:09.761 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-15T12:47:09.761 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply-3.11-14.el9.noarch 2024-03-15T12:47:09.761 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-15T12:47:09.761 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-15T12:47:09.761 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-15T12:47:09.761 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-15T12:47:09.761 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-15T12:47:09.761 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-15T12:47:09.761 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-15T12:47:09.761 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-15T12:47:09.761 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-15T12:47:09.761 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.762 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.762 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-15T12:47:09.762 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-15T12:47:09.762 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-15T12:47:09.762 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.762 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-15T12:47:09.762 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-15T12:47:09.762 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-15T12:47:09.762 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-15T12:47:09.762 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-15T12:47:09.762 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-15T12:47:09.762 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-15T12:47:09.762 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-15T12:47:09.763 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-15T12:47:09.763 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-15T12:47:09.763 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-15T12:47:09.763 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-15T12:47:09.763 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-15T12:47:09.763 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-15T12:47:09.763 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-15T12:47:09.763 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.763 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.763 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:09.763 INFO:teuthology.orchestra.run.smithi149.stdout: re2-1:20211101-3.el9.x86_64 2024-03-15T12:47:09.763 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-15T12:47:09.763 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-15T12:47:09.763 INFO:teuthology.orchestra.run.smithi149.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-15T12:47:09.763 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-15T12:47:09.764 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-15T12:47:09.764 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-15T12:47:09.764 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-15T12:47:09.764 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-15T12:47:10.235 DEBUG:teuthology.parallel:result is None 2024-03-15T12:47:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-15T12:47:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-2:19.0.0-1681.gaba3c1d2.el9.x86_64 1/198 2024-03-15T12:47:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2/198 2024-03-15T12:47:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 3/198 2024-03-15T12:47:11.463 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 4/198 2024-03-15T12:47:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1681.gaba3c 5/198 2024-03-15T12:47:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 6/198 2024-03-15T12:47:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 7/198 2024-03-15T12:47:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 8/198 2024-03-15T12:47:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 9/198 2024-03-15T12:47:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 10/198 2024-03-15T12:47:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 11/198 2024-03-15T12:47:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-test-2:19.0.0-1681.gaba3c1d2.el9.x86_64 12/198 2024-03-15T12:47:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 13/198 2024-03-15T12:47:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-shell-2:19.0.0-1681.gaba3c1d2.el9.x86_64 14/198 2024-03-15T12:47:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_ 15/198 2024-03-15T12:47:11.464 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 16/198 2024-03-15T12:47:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 17/198 2024-03-15T12:47:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_6 18/198 2024-03-15T12:47:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86 19/198 2024-03-15T12:47:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 20/198 2024-03-15T12:47:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1681.gaba3c1d2.el 21/198 2024-03-15T12:47:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-common-2:19.0.0-1681.gaba3c1d2.el9. 22/198 2024-03-15T12:47:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cephfs-2:19.0.0-1681.gaba3c1d2.el9.x86_6 23/198 2024-03-15T12:47:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rados-2:19.0.0-1681.gaba3c1d2.el9.x86_64 24/198 2024-03-15T12:47:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 25/198 2024-03-15T12:47:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 26/198 2024-03-15T12:47:11.465 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 27/198 2024-03-15T12:47:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 28/198 2024-03-15T12:47:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-nbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 29/198 2024-03-15T12:47:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1681.gaba3c1d2. 30/198 2024-03-15T12:47:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noa 31/198 2024-03-15T12:47:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.n 32/198 2024-03-15T12:47:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba 33/198 2024-03-15T12:47:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1681.gaba3c1d2.el 34/198 2024-03-15T12:47:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 35/198 2024-03-15T12:47:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1681.gaba3c1d2.e 36/198 2024-03-15T12:47:11.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 37/198 2024-03-15T12:47:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 38/198 2024-03-15T12:47:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-top-2:19.0.0-1681.gaba3c1d2.el9.noarch 39/198 2024-03-15T12:47:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-15T12:47:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-15T12:47:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-15T12:47:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-15T12:47:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-15T12:47:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-15T12:47:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-15T12:47:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-15T12:47:11.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-15T12:47:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-15T12:47:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-15T12:47:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-15T12:47:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-15T12:47:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-15T12:47:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-15T12:47:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-15T12:47:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-15T12:47:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-15T12:47:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-15T12:47:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-15T12:47:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-15T12:47:11.484 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-15T12:47:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-15T12:47:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-15T12:47:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-15T12:47:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-15T12:47:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-15T12:47:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-15T12:47:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-15T12:47:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-15T12:47:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-15T12:47:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-15T12:47:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-15T12:47:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-15T12:47:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-15T12:47:11.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-15T12:47:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-15T12:47:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-15T12:47:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-15T12:47:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-15T12:47:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-15T12:47:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-15T12:47:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-15T12:47:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-15T12:47:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-15T12:47:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-15T12:47:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-15T12:47:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-15T12:47:11.486 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-15T12:47:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-15T12:47:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-15T12:47:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-15T12:47:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-15T12:47:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-15T12:47:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-15T12:47:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-15T12:47:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-15T12:47:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-15T12:47:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-15T12:47:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-15T12:47:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-15T12:47:11.487 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-15T12:47:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-15T12:47:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-15T12:47:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-15T12:47:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-15T12:47:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-15T12:47:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-15T12:47:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-15T12:47:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-15T12:47:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-15T12:47:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-15T12:47:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-15T12:47:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-15T12:47:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-15T12:47:11.488 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-15T12:47:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-15T12:47:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-15T12:47:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-15T12:47:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-15T12:47:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-15T12:47:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-15T12:47:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-15T12:47:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-15T12:47:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-15T12:47:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-15T12:47:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-15T12:47:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-15T12:47:11.489 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-15T12:47:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-15T12:47:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-15T12:47:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-15T12:47:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-15T12:47:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-15T12:47:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-15T12:47:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-15T12:47:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-15T12:47:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-15T12:47:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-15T12:47:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-15T12:47:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-15T12:47:11.490 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-15T12:47:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-15T12:47:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-15T12:47:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-15T12:47:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-15T12:47:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-15T12:47:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-15T12:47:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-15T12:47:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-15T12:47:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-15T12:47:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 151/198 2024-03-15T12:47:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-15T12:47:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 153/198 2024-03-15T12:47:11.491 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-15T12:47:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-15T12:47:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-15T12:47:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-15T12:47:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-15T12:47:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-15T12:47:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-15T12:47:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-15T12:47:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-15T12:47:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-15T12:47:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-15T12:47:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-15T12:47:11.492 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-15T12:47:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-15T12:47:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-15T12:47:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-15T12:47:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-15T12:47:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-15T12:47:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-15T12:47:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-15T12:47:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-15T12:47:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-15T12:47:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-15T12:47:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-15T12:47:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-15T12:47:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-15T12:47:11.493 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-15T12:47:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-15T12:47:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-15T12:47:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-15T12:47:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-15T12:47:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-15T12:47:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-15T12:47:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-15T12:47:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-15T12:47:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-15T12:47:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-15T12:47:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-15T12:47:11.494 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-15T12:47:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-15T12:47:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-15T12:47:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-15T12:47:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-15T12:47:11.495 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-15T12:47:14.586 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-15T12:47:14.586 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:47:14.586 INFO:teuthology.orchestra.run.smithi002.stdout:Upgraded: 2024-03-15T12:47:14.586 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-15T12:47:14.586 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-15T12:47:14.586 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-15T12:47:14.586 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-15T12:47:14.587 INFO:teuthology.orchestra.run.smithi002.stdout: acl-2.3.1-4.el9.x86_64 2024-03-15T12:47:14.587 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-15T12:47:14.587 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-15T12:47:14.587 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-15T12:47:14.587 INFO:teuthology.orchestra.run.smithi002.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-15T12:47:14.587 INFO:teuthology.orchestra.run.smithi002.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-15T12:47:14.587 INFO:teuthology.orchestra.run.smithi002.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-15T12:47:14.587 INFO:teuthology.orchestra.run.smithi002.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-15T12:47:14.587 INFO:teuthology.orchestra.run.smithi002.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-15T12:47:14.587 INFO:teuthology.orchestra.run.smithi002.stdout: librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.587 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.587 INFO:teuthology.orchestra.run.smithi002.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-15T12:47:14.587 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-15T12:47:14.588 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-15T12:47:14.588 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-15T12:47:14.588 INFO:teuthology.orchestra.run.smithi002.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-15T12:47:14.588 INFO:teuthology.orchestra.run.smithi002.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-15T12:47:14.588 INFO:teuthology.orchestra.run.smithi002.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-15T12:47:14.588 INFO:teuthology.orchestra.run.smithi002.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-15T12:47:14.588 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-15T12:47:14.588 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-15T12:47:14.588 INFO:teuthology.orchestra.run.smithi002.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-15T12:47:14.588 INFO:teuthology.orchestra.run.smithi002.stdout: automake-1.16.2-8.el9.noarch 2024-03-15T12:47:14.588 INFO:teuthology.orchestra.run.smithi002.stdout: bison-3.7.4-5.el9.x86_64 2024-03-15T12:47:14.588 INFO:teuthology.orchestra.run.smithi002.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-15T12:47:14.588 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.588 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.589 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.589 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.589 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:14.589 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.589 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.589 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.589 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:14.589 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:14.589 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:14.589 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:14.589 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:14.589 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.589 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.590 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:14.590 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.590 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.590 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.590 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:14.590 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:14.590 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.590 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-shell-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.590 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-15T12:47:14.590 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-15T12:47:14.590 INFO:teuthology.orchestra.run.smithi002.stdout: flex-2.6.4-9.el9.x86_64 2024-03-15T12:47:14.590 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-15T12:47:14.590 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-15T12:47:14.590 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-15T12:47:14.591 INFO:teuthology.orchestra.run.smithi002.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-15T12:47:14.591 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-15T12:47:14.591 INFO:teuthology.orchestra.run.smithi002.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-15T12:47:14.591 INFO:teuthology.orchestra.run.smithi002.stdout: jq-1.6-16.el9.x86_64 2024-03-15T12:47:14.591 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-15T12:47:14.591 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-15T12:47:14.591 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-15T12:47:14.591 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.591 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.591 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.591 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-15T12:47:14.591 INFO:teuthology.orchestra.run.smithi002.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-15T12:47:14.591 INFO:teuthology.orchestra.run.smithi002.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-15T12:47:14.591 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-15T12:47:14.592 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-15T12:47:14.592 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.592 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.592 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-15T12:47:14.592 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.592 INFO:teuthology.orchestra.run.smithi002.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-15T12:47:14.592 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-15T12:47:14.592 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-15T12:47:14.592 INFO:teuthology.orchestra.run.smithi002.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-15T12:47:14.592 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-15T12:47:14.592 INFO:teuthology.orchestra.run.smithi002.stdout: lua-5.4.4-4.el9.x86_64 2024-03-15T12:47:14.592 INFO:teuthology.orchestra.run.smithi002.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-15T12:47:14.592 INFO:teuthology.orchestra.run.smithi002.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-15T12:47:14.592 INFO:teuthology.orchestra.run.smithi002.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-15T12:47:14.593 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-15T12:47:14.593 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-15T12:47:14.593 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-15T12:47:14.593 INFO:teuthology.orchestra.run.smithi002.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-15T12:47:14.593 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-13.11-1.el9.x86_64 2024-03-15T12:47:14.593 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-15T12:47:14.593 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-15T12:47:14.593 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-15T12:47:14.593 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-15T12:47:14.593 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-15T12:47:14.593 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-15T12:47:14.593 INFO:teuthology.orchestra.run.smithi002.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-15T12:47:14.593 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-15T12:47:14.593 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-15T12:47:14.594 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-15T12:47:14.594 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.594 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.594 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.594 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-15T12:47:14.594 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-15T12:47:14.594 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-15T12:47:14.594 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-15T12:47:14.594 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-15T12:47:14.594 INFO:teuthology.orchestra.run.smithi002.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-15T12:47:14.594 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-15T12:47:14.594 INFO:teuthology.orchestra.run.smithi002.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-15T12:47:14.594 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-15T12:47:14.595 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-15T12:47:14.595 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-15T12:47:14.595 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-15T12:47:14.595 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-15T12:47:14.595 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-15T12:47:14.595 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-15T12:47:14.595 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-15T12:47:14.595 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-15T12:47:14.595 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-15T12:47:14.595 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-15T12:47:14.595 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-15T12:47:14.595 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-15T12:47:14.595 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-15T12:47:14.595 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-15T12:47:14.596 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-15T12:47:14.596 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-15T12:47:14.596 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-15T12:47:14.596 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-15T12:47:14.596 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-15T12:47:14.596 INFO:teuthology.orchestra.run.smithi002.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-15T12:47:14.596 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-15T12:47:14.596 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ply-3.11-14.el9.noarch 2024-03-15T12:47:14.596 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-15T12:47:14.596 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-15T12:47:14.596 INFO:teuthology.orchestra.run.smithi002.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-15T12:47:14.596 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-15T12:47:14.596 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-15T12:47:14.596 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-15T12:47:14.597 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-15T12:47:14.597 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-15T12:47:14.597 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-15T12:47:14.597 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.597 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.597 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-15T12:47:14.597 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-15T12:47:14.597 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-15T12:47:14.597 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.597 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-15T12:47:14.597 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-15T12:47:14.597 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-15T12:47:14.597 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-15T12:47:14.598 INFO:teuthology.orchestra.run.smithi002.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-15T12:47:14.598 INFO:teuthology.orchestra.run.smithi002.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-15T12:47:14.598 INFO:teuthology.orchestra.run.smithi002.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-15T12:47:14.598 INFO:teuthology.orchestra.run.smithi002.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-15T12:47:14.598 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-15T12:47:14.598 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-15T12:47:14.598 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-15T12:47:14.598 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-15T12:47:14.598 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-15T12:47:14.598 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-15T12:47:14.598 INFO:teuthology.orchestra.run.smithi002.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-15T12:47:14.598 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.598 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.598 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-15T12:47:14.599 INFO:teuthology.orchestra.run.smithi002.stdout: re2-1:20211101-3.el9.x86_64 2024-03-15T12:47:14.599 INFO:teuthology.orchestra.run.smithi002.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-15T12:47:14.599 INFO:teuthology.orchestra.run.smithi002.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-15T12:47:14.599 INFO:teuthology.orchestra.run.smithi002.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-15T12:47:14.599 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-15T12:47:14.599 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-15T12:47:14.599 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-15T12:47:14.599 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:47:14.599 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-15T12:47:15.077 DEBUG:teuthology.parallel:result is None 2024-03-15T12:47:15.078 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-15T12:47:15.300 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-15T12:47:15.321 INFO:teuthology.orchestra.run.smithi002.stdout:19.0.0-1681.gaba3c1d2.el9 2024-03-15T12:47:15.322 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1681.gaba3c1d2.el9 2024-03-15T12:47:15.322 INFO:teuthology.task.install:The correct ceph version 19.0.0-1681.gaba3c1d2 is installed. 2024-03-15T12:47:15.324 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-15T12:47:15.470 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-15T12:47:15.496 INFO:teuthology.orchestra.run.smithi149.stdout:19.0.0-1681.gaba3c1d2.el9 2024-03-15T12:47:15.496 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1681.gaba3c1d2.el9 2024-03-15T12:47:15.496 INFO:teuthology.task.install:The correct ceph version 19.0.0-1681.gaba3c1d2 is installed. 2024-03-15T12:47:15.498 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-15T12:47:15.635 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-15T12:47:15.661 INFO:teuthology.orchestra.run.smithi184.stdout:19.0.0-1681.gaba3c1d2.el9 2024-03-15T12:47:15.661 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1681.gaba3c1d2.el9 2024-03-15T12:47:15.661 INFO:teuthology.task.install:The correct ceph version 19.0.0-1681.gaba3c1d2 is installed. 2024-03-15T12:47:15.663 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-15T12:47:15.663 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:47:15.663 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-15T12:47:15.695 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-15T12:47:15.695 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-15T12:47:15.731 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-15T12:47:15.731 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-15T12:47:15.765 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-15T12:47:15.765 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:47:15.765 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/daemon-helper 2024-03-15T12:47:15.794 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-15T12:47:15.862 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-15T12:47:15.862 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-03-15T12:47:15.893 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-15T12:47:15.963 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-15T12:47:15.963 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2024-03-15T12:47:15.991 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-15T12:47:16.058 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-15T12:47:16.058 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:47:16.058 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-15T12:47:16.087 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-15T12:47:16.155 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-15T12:47:16.156 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-15T12:47:16.185 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-15T12:47:16.254 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-15T12:47:16.254 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-15T12:47:16.281 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-15T12:47:16.348 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-15T12:47:16.348 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:47:16.348 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/stdin-killer 2024-03-15T12:47:16.377 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-15T12:47:16.445 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-15T12:47:16.446 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-03-15T12:47:16.475 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-15T12:47:16.544 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-15T12:47:16.544 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2024-03-15T12:47:16.570 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-15T12:47:16.638 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-15T12:47:16.742 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client dirsize rbytes': False, 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b', 'subvols': {'create': 2}} 2024-03-15T12:47:16.742 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-15T12:47:16.742 INFO:tasks.cephadm:Cluster fsid is 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:47:16.742 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-15T12:47:16.743 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.2', 'mon.b': '172.21.15.149', 'mon.c': '172.21.15.184'} 2024-03-15T12:47:16.743 INFO:tasks.cephadm:First mon is mon.a on smithi002 2024-03-15T12:47:16.743 INFO:tasks.cephadm:First mgr is x 2024-03-15T12:47:16.743 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-15T12:47:16.743 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname $(hostname -s) 2024-03-15T12:47:16.774 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname $(hostname -s) 2024-03-15T12:47:16.804 DEBUG:teuthology.orchestra.run.smithi184:> sudo hostname $(hostname -s) 2024-03-15T12:47:16.830 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-15T12:47:16.831 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-15T12:47:17.043 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77659/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1681-gaba3c1d2', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1681.gaba3c1d2'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-15 03:39:08.164055', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-qdb-squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-15T12:47:17.264 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-lusov-qdb-squid, sha1 aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-15T12:47:17.266 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/x86_64/flavors/default/cephadm 2024-03-15T12:47:17.266 DEBUG:teuthology.orchestra.run.smithi002:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-15T12:47:17.908 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-r--r--. 1 ubuntu ubuntu 766937 Mar 15 12:47 /home/ubuntu/cephtest/cephadm 2024-03-15T12:47:17.908 DEBUG:teuthology.orchestra.run.smithi149:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-15T12:47:18.339 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r--. 1 ubuntu ubuntu 766937 Mar 15 12:47 /home/ubuntu/cephtest/cephadm 2024-03-15T12:47:18.339 DEBUG:teuthology.orchestra.run.smithi184:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-15T12:47:18.909 INFO:teuthology.orchestra.run.smithi184.stdout:-rw-r--r--. 1 ubuntu ubuntu 766937 Mar 15 12:47 /home/ubuntu/cephtest/cephadm 2024-03-15T12:47:18.909 DEBUG:teuthology.orchestra.run.smithi002:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-15T12:47:18.928 DEBUG:teuthology.orchestra.run.smithi149:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-15T12:47:18.946 DEBUG:teuthology.orchestra.run.smithi184:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-15T12:47:18.980 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b on all hosts... 2024-03-15T12:47:18.980 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b pull 2024-03-15T12:47:18.982 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b pull 2024-03-15T12:47:18.989 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b pull 2024-03-15T12:47:19.785 INFO:teuthology.orchestra.run.smithi149.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b... 2024-03-15T12:47:19.947 INFO:teuthology.orchestra.run.smithi002.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b... 2024-03-15T12:47:19.953 INFO:teuthology.orchestra.run.smithi184.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b... 2024-03-15T12:47:51.773 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-15T12:47:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 19.0.0-1681-gaba3c1d2 (aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b) squid (dev)", 2024-03-15T12:47:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: "image_id": "80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0", 2024-03-15T12:47:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: "repo_digests": [ 2024-03-15T12:47:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3" 2024-03-15T12:47:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-03-15T12:47:51.774 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-15T12:48:09.802 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:48:09.802 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 19.0.0-1681-gaba3c1d2 (aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b) squid (dev)", 2024-03-15T12:48:09.802 INFO:teuthology.orchestra.run.smithi002.stdout: "image_id": "80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0", 2024-03-15T12:48:09.802 INFO:teuthology.orchestra.run.smithi002.stdout: "repo_digests": [ 2024-03-15T12:48:09.803 INFO:teuthology.orchestra.run.smithi002.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3" 2024-03-15T12:48:09.803 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-03-15T12:48:09.803 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:48:11.140 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-03-15T12:48:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_version": "ceph version 19.0.0-1681-gaba3c1d2 (aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b) squid (dev)", 2024-03-15T12:48:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "image_id": "80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0", 2024-03-15T12:48:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "repo_digests": [ 2024-03-15T12:48:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3" 2024-03-15T12:48:11.140 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-03-15T12:48:11.140 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-03-15T12:48:11.168 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /etc/ceph 2024-03-15T12:48:11.204 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph 2024-03-15T12:48:11.239 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph 2024-03-15T12:48:11.271 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 777 /etc/ceph 2024-03-15T12:48:11.301 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /etc/ceph 2024-03-15T12:48:11.331 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /etc/ceph 2024-03-15T12:48:11.359 INFO:tasks.cephadm:Writing seed config... 2024-03-15T12:48:11.360 INFO:tasks.cephadm: override: [client] client dirsize rbytes = False 2024-03-15T12:48:11.360 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-15T12:48:11.361 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-15T12:48:11.361 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-15T12:48:11.361 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-15T12:48:11.361 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-15T12:48:11.361 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-15T12:48:11.361 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-15T12:48:11.361 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-15T12:48:11.361 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-15T12:48:11.361 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-15T12:48:11.361 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-15T12:48:11.361 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-15T12:48:11.361 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-15T12:48:11.361 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-15T12:48:11.362 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-15T12:48:11.362 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-15T12:48:11.362 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-15T12:48:11.362 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-15T12:48:11.362 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-15T12:48:11.362 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-15T12:48:11.362 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-15T12:48:11.362 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-15T12:48:11.362 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-15T12:48:11.362 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-15T12:48:11.362 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-15T12:48:11.362 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-15T12:48:11.362 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-15T12:48:11.363 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-15T12:48:11.363 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-15T12:48:11.363 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-15T12:48:11.363 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-15T12:48:11.363 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-15T12:48:11.363 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-15T12:48:11.363 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-15T12:48:11.363 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-15T12:48:11.363 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-15T12:48:11.363 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-15T12:48:11.363 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-15T12:48:11.363 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-15T12:48:11.363 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-15T12:48:11.364 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-15T12:48:11.364 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-15T12:48:11.364 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-15T12:48:11.364 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:48:11.365 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-15T12:48:11.381 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2767718a-e2ca-11ee-95c9-87774f69a715 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client dirsize rbytes = False client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-15T12:48:11.381 DEBUG:teuthology.orchestra.run.smithi002:mon.a> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mon.a.service 2024-03-15T12:48:11.424 DEBUG:teuthology.orchestra.run.smithi002:mgr.x> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.x.service 2024-03-15T12:48:11.467 INFO:tasks.cephadm:Bootstrapping... 2024-03-15T12:48:11.468 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b -v bootstrap --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.2 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-15T12:48:11.667 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-15T12:48:11.668 INFO:teuthology.orchestra.run.smithi002.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b', '-v', 'bootstrap', '--fsid', '2767718a-e2ca-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.2', '--skip-admin-label'] 2024-03-15T12:48:11.692 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-15T12:48:11.693 INFO:teuthology.orchestra.run.smithi002.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-15T12:48:11.693 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying podman|docker is present... 2024-03-15T12:48:11.716 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-15T12:48:11.716 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying lvm2 is present... 2024-03-15T12:48:11.716 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying time synchronization is in place... 2024-03-15T12:48:11.723 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-15T12:48:11.724 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-15T12:48:11.729 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-15T12:48:11.730 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-03-15T12:48:11.736 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout enabled 2024-03-15T12:48:11.742 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout active 2024-03-15T12:48:11.742 INFO:teuthology.orchestra.run.smithi002.stdout:Unit chronyd.service is enabled and running 2024-03-15T12:48:11.742 INFO:teuthology.orchestra.run.smithi002.stdout:Repeating the final host check... 2024-03-15T12:48:11.765 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-15T12:48:11.765 INFO:teuthology.orchestra.run.smithi002.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-15T12:48:11.765 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl is present 2024-03-15T12:48:11.765 INFO:teuthology.orchestra.run.smithi002.stdout:lvcreate is present 2024-03-15T12:48:11.771 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-15T12:48:11.771 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-15T12:48:11.777 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-15T12:48:11.777 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-03-15T12:48:11.783 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout enabled 2024-03-15T12:48:11.789 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout active 2024-03-15T12:48:11.789 INFO:teuthology.orchestra.run.smithi002.stdout:Unit chronyd.service is enabled and running 2024-03-15T12:48:11.789 INFO:teuthology.orchestra.run.smithi002.stdout:Host looks OK 2024-03-15T12:48:11.789 INFO:teuthology.orchestra.run.smithi002.stdout:Cluster fsid: 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:48:11.789 INFO:teuthology.orchestra.run.smithi002.stdout:Acquiring lock 140492536078288 on /run/cephadm/2767718a-e2ca-11ee-95c9-87774f69a715.lock 2024-03-15T12:48:11.789 INFO:teuthology.orchestra.run.smithi002.stdout:Lock 140492536078288 acquired on /run/cephadm/2767718a-e2ca-11ee-95c9-87774f69a715.lock 2024-03-15T12:48:11.790 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying IP 172.21.15.2 port 3300 ... 2024-03-15T12:48:11.790 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying IP 172.21.15.2 port 6789 ... 2024-03-15T12:48:11.790 INFO:teuthology.orchestra.run.smithi002.stdout:Base mon IP(s) is [172.21.15.2:3300, 172.21.15.2:6789], mon addrv is [v2:172.21.15.2:3300,v1:172.21.15.2:6789] 2024-03-15T12:48:11.793 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.2 metric 100 2024-03-15T12:48:11.793 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.2 metric 100 2024-03-15T12:48:11.796 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-15T12:48:11.796 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-15T12:48:11.796 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-15T12:48:11.799 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-15T12:48:11.799 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-15T12:48:11.799 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-15T12:48:11.799 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-15T12:48:11.799 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1920/64 scope link noprefixroute 2024-03-15T12:48:11.799 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-15T12:48:11.799 INFO:teuthology.orchestra.run.smithi002.stdout:Mon IP `172.21.15.2` is in CIDR network `172.21.0.0/20` 2024-03-15T12:48:11.799 INFO:teuthology.orchestra.run.smithi002.stdout:Mon IP `172.21.15.2` is in CIDR network `172.21.0.0/20` 2024-03-15T12:48:11.799 INFO:teuthology.orchestra.run.smithi002.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-15T12:48:11.800 INFO:teuthology.orchestra.run.smithi002.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-15T12:48:11.800 INFO:teuthology.orchestra.run.smithi002.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b... 2024-03-15T12:48:12.278 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 2024-03-15T12:48:12.278 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b... 2024-03-15T12:48:12.279 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Getting image source signatures 2024-03-15T12:48:12.279 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying blob sha256:70a48e3dafcdedd8abcbc488c0076bda17d489568a5d72d4513d3d6eecb1880d 2024-03-15T12:48:12.279 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-15T12:48:12.279 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying config sha256:80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 2024-03-15T12:48:12.279 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-15T12:48:13.299 INFO:teuthology.orchestra.run.smithi002.stdout:ceph: stdout ceph version 19.0.0-1681-gaba3c1d2 (aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b) squid (dev) 2024-03-15T12:48:13.299 INFO:teuthology.orchestra.run.smithi002.stdout:Ceph version: ceph version 19.0.0-1681-gaba3c1d2 (aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b) squid (dev) 2024-03-15T12:48:13.300 INFO:teuthology.orchestra.run.smithi002.stdout:Extracting ceph user uid/gid from container image... 2024-03-15T12:48:14.214 INFO:teuthology.orchestra.run.smithi002.stdout:stat: stdout 167 167 2024-03-15T12:48:14.214 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial keys... 2024-03-15T12:48:15.062 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQCOQ/Rlcw36IRAANT0i1hCycg+QQE7m1P7O6Q== 2024-03-15T12:48:15.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQCPQ/RlslZ0GxAAVbptmS6BQb5PHTa9vaoZMQ== 2024-03-15T12:48:16.897 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQCQQ/RlESypEhAAxkpwJ1l8m+MrgQMSXfdjmQ== 2024-03-15T12:48:16.898 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial monmap... 2024-03-15T12:48:17.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-15T12:48:17.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-15T12:48:17.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:48:17.721 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-15T12:48:17.722 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool for a [v2:172.21.15.2:3300,v1:172.21.15.2:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-15T12:48:17.722 INFO:teuthology.orchestra.run.smithi002.stdout:setting min_mon_release = quincy 2024-03-15T12:48:17.722 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: set fsid to 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:48:17.722 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-15T12:48:17.722 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:48:17.722 INFO:teuthology.orchestra.run.smithi002.stdout:Creating mon... 2024-03-15T12:48:18.576 INFO:teuthology.orchestra.run.smithi002.stdout:create mon.a on 2024-03-15T12:48:18.781 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-15T12:48:18.985 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-15T12:48:19.156 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2767718a-e2ca-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-2767718a-e2ca-11ee-95c9-87774f69a715.target. 2024-03-15T12:48:19.156 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2767718a-e2ca-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-2767718a-e2ca-11ee-95c9-87774f69a715.target. 2024-03-15T12:48:19.391 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mon.a 2024-03-15T12:48:19.391 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mon.a.service: Unit ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mon.a.service not loaded. 2024-03-15T12:48:19.562 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2767718a-e2ca-11ee-95c9-87774f69a715.target.wants/ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-2767718a-e2ca-11ee-95c9-87774f69a715@.service. 2024-03-15T12:48:20.134 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-15T12:48:20.134 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-03-15T12:48:20.140 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-15T12:48:20.141 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-03-15T12:48:20.141 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-03-15T12:48:20.141 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to enable service . firewalld.service is not available 2024-03-15T12:48:20.141 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mon to start... 2024-03-15T12:48:20.141 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mon... 2024-03-15T12:48:21.187 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:21 smithi002 ceph-mon[31684]: from='client.? 172.21.15.2:0/4048969014' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-15T12:48:21.196 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout cluster: 2024-03-15T12:48:21.196 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout id: 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:48:21.196 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-15T12:48:21.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-15T12:48:21.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout services: 2024-03-15T12:48:21.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.531037s) 2024-03-15T12:48:21.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-15T12:48:21.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-15T12:48:21.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-15T12:48:21.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout data: 2024-03-15T12:48:21.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-15T12:48:21.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-15T12:48:21.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-15T12:48:21.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout pgs: 2024-03-15T12:48:21.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-15T12:48:21.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.545+0000 7f7bb1a64700 1 Processor -- start 2024-03-15T12:48:21.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.545+0000 7f7bb1a64700 1 -- start start 2024-03-15T12:48:21.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.545+0000 7f7bb1a64700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bac1078f0 0x7f7bac107d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:21.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.545+0000 7f7bb1a64700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac1082e0 con 0x7f7bac1078f0 2024-03-15T12:48:21.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.546+0000 7f7bab7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bac1078f0 0x7f7bac107d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:21.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.546+0000 7f7bab7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bac1078f0 0x7f7bac107d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42696/0 (socket says 172.21.15.2:42696) 2024-03-15T12:48:21.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.546+0000 7f7bab7fe700 1 -- 172.21.15.2:0/2338202161 learned_addr learned my addr 172.21.15.2:0/2338202161 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:21.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.546+0000 7f7bab7fe700 1 -- 172.21.15.2:0/2338202161 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac108aa0 con 0x7f7bac1078f0 2024-03-15T12:48:21.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.546+0000 7f7bab7fe700 1 --2- 172.21.15.2:0/2338202161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bac1078f0 0x7f7bac107d10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009e80 tx=0x7f7b9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ebe7116d711479 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:21.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.546+0000 7f7baa7fc700 1 -- 172.21.15.2:0/2338202161 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b9c00e070 con 0x7f7bac1078f0 2024-03-15T12:48:21.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.546+0000 7f7baa7fc700 1 -- 172.21.15.2:0/2338202161 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9c003a20 con 0x7f7bac1078f0 2024-03-15T12:48:21.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.546+0000 7f7baa7fc700 1 -- 172.21.15.2:0/2338202161 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b9c003ce0 con 0x7f7bac1078f0 2024-03-15T12:48:21.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.547+0000 7f7bb1a64700 1 -- 172.21.15.2:0/2338202161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bac1078f0 msgr2=0x7f7bac107d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:21.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.547+0000 7f7bb1a64700 1 --2- 172.21.15.2:0/2338202161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bac1078f0 0x7f7bac107d10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009e80 tx=0x7f7b9c00b330 comp rx=0 tx=0).stop 2024-03-15T12:48:21.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.547+0000 7f7bb1a64700 1 -- 172.21.15.2:0/2338202161 shutdown_connections 2024-03-15T12:48:21.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.547+0000 7f7bb1a64700 1 --2- 172.21.15.2:0/2338202161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bac1078f0 0x7f7bac107d10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:21.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.547+0000 7f7bb1a64700 1 -- 172.21.15.2:0/2338202161 >> 172.21.15.2:0/2338202161 conn(0x7f7bac102f50 msgr2=0x7f7bac105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:21.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.547+0000 7f7bb1a64700 1 -- 172.21.15.2:0/2338202161 shutdown_connections 2024-03-15T12:48:21.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.547+0000 7f7bb1a64700 1 -- 172.21.15.2:0/2338202161 wait complete. 2024-03-15T12:48:21.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.548+0000 7f7bb1a64700 1 Processor -- start 2024-03-15T12:48:21.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.548+0000 7f7bb1a64700 1 -- start start 2024-03-15T12:48:21.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.548+0000 7f7bb1a64700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bac1078f0 0x7f7bac07cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:21.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.548+0000 7f7bb1a64700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac07d0c0 con 0x7f7bac1078f0 2024-03-15T12:48:21.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.548+0000 7f7bab7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bac1078f0 0x7f7bac07cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:21.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.548+0000 7f7bab7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bac1078f0 0x7f7bac07cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42700/0 (socket says 172.21.15.2:42700) 2024-03-15T12:48:21.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.548+0000 7f7bab7fe700 1 -- 172.21.15.2:0/4048969014 learned_addr learned my addr 172.21.15.2:0/4048969014 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:21.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.548+0000 7f7bab7fe700 1 -- 172.21.15.2:0/4048969014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c009b30 con 0x7f7bac1078f0 2024-03-15T12:48:21.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.548+0000 7f7bab7fe700 1 --2- 172.21.15.2:0/4048969014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bac1078f0 0x7f7bac07cb80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c004000 tx=0x7f7b9c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:21.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.549+0000 7f7ba8ff9700 1 -- 172.21.15.2:0/4048969014 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b9c004370 con 0x7f7bac1078f0 2024-03-15T12:48:21.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.549+0000 7f7ba8ff9700 1 -- 172.21.15.2:0/4048969014 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9c004510 con 0x7f7bac1078f0 2024-03-15T12:48:21.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.549+0000 7f7bb1a64700 1 -- 172.21.15.2:0/4048969014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bac07b2b0 con 0x7f7bac1078f0 2024-03-15T12:48:21.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.549+0000 7f7ba8ff9700 1 -- 172.21.15.2:0/4048969014 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b9c0047f0 con 0x7f7bac1078f0 2024-03-15T12:48:21.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.549+0000 7f7bb1a64700 1 -- 172.21.15.2:0/4048969014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bac07b750 con 0x7f7bac1078f0 2024-03-15T12:48:21.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.549+0000 7f7ba8ff9700 1 -- 172.21.15.2:0/4048969014 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7b9c0127c0 con 0x7f7bac1078f0 2024-03-15T12:48:21.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.549+0000 7f7ba8ff9700 1 -- 172.21.15.2:0/4048969014 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7b9c01dbf0 con 0x7f7bac1078f0 2024-03-15T12:48:21.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.550+0000 7f7bb1a64700 1 -- 172.21.15.2:0/4048969014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bac07c160 con 0x7f7bac1078f0 2024-03-15T12:48:21.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.552+0000 7f7ba8ff9700 1 -- 172.21.15.2:0/4048969014 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7b9c02b060 con 0x7f7bac1078f0 2024-03-15T12:48:21.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.625+0000 7f7bb1a64700 1 -- 172.21.15.2:0/4048969014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f7bac108e10 con 0x7f7bac1078f0 2024-03-15T12:48:21.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.625+0000 7f7ba8ff9700 1 -- 172.21.15.2:0/4048969014 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f7b9c021030 con 0x7f7bac1078f0 2024-03-15T12:48:21.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.626+0000 7f7bb1a64700 1 -- 172.21.15.2:0/4048969014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bac1078f0 msgr2=0x7f7bac07cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:21.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.626+0000 7f7bb1a64700 1 --2- 172.21.15.2:0/4048969014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bac1078f0 0x7f7bac07cb80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c004000 tx=0x7f7b9c004ad0 comp rx=0 tx=0).stop 2024-03-15T12:48:21.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.626+0000 7f7bb1a64700 1 -- 172.21.15.2:0/4048969014 shutdown_connections 2024-03-15T12:48:21.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.626+0000 7f7bb1a64700 1 --2- 172.21.15.2:0/4048969014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bac1078f0 0x7f7bac07cb80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:21.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.626+0000 7f7bb1a64700 1 -- 172.21.15.2:0/4048969014 >> 172.21.15.2:0/4048969014 conn(0x7f7bac102f50 msgr2=0x7f7bac191bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:21.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.626+0000 7f7bb1a64700 1 -- 172.21.15.2:0/4048969014 shutdown_connections 2024-03-15T12:48:21.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:20.626+0000 7f7bb1a64700 1 -- 172.21.15.2:0/4048969014 wait complete. 2024-03-15T12:48:21.201 INFO:teuthology.orchestra.run.smithi002.stdout:mon is available 2024-03-15T12:48:21.201 INFO:teuthology.orchestra.run.smithi002.stdout:Assimilating anything we can from ceph.conf... 2024-03-15T12:48:22.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-15T12:48:22.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [global] 2024-03-15T12:48:22.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout fsid = 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:48:22.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.2:3300,v1:172.21.15.2:6789] 2024-03-15T12:48:22.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-15T12:48:22.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-15T12:48:22.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-15T12:48:22.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-15T12:48:22.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-15T12:48:22.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-15T12:48:22.236 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-15T12:48:22.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-15T12:48:22.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [osd] 2024-03-15T12:48:22.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-15T12:48:22.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-15T12:48:22.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-15T12:48:22.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-15T12:48:22.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-15T12:48:22.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.630+0000 7fba19eea700 1 Processor -- start 2024-03-15T12:48:22.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.630+0000 7fba19eea700 1 -- start start 2024-03-15T12:48:22.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.630+0000 7fba19eea700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba141078e0 0x7fba14107d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:22.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.630+0000 7fba19eea700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba141082d0 con 0x7fba141078e0 2024-03-15T12:48:22.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.631+0000 7fba137fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba141078e0 0x7fba14107d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:22.237 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.631+0000 7fba137fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba141078e0 0x7fba14107d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42706/0 (socket says 172.21.15.2:42706) 2024-03-15T12:48:22.238 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.631+0000 7fba137fe700 1 -- 172.21.15.2:0/1540972811 learned_addr learned my addr 172.21.15.2:0/1540972811 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:22.238 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.631+0000 7fba137fe700 1 -- 172.21.15.2:0/1540972811 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba14108a90 con 0x7fba141078e0 2024-03-15T12:48:22.238 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.631+0000 7fba137fe700 1 --2- 172.21.15.2:0/1540972811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba141078e0 0x7fba14107d00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fba040098a0 tx=0x7fba04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb346a94efb4e1e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:22.238 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.631+0000 7fba127fc700 1 -- 172.21.15.2:0/1540972811 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba04004160 con 0x7fba141078e0 2024-03-15T12:48:22.238 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.631+0000 7fba127fc700 1 -- 172.21.15.2:0/1540972811 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba040042c0 con 0x7fba141078e0 2024-03-15T12:48:22.238 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.631+0000 7fba127fc700 1 -- 172.21.15.2:0/1540972811 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba040045a0 con 0x7fba141078e0 2024-03-15T12:48:22.238 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.632+0000 7fba19eea700 1 -- 172.21.15.2:0/1540972811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba141078e0 msgr2=0x7fba14107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:22.238 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.632+0000 7fba19eea700 1 --2- 172.21.15.2:0/1540972811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba141078e0 0x7fba14107d00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fba040098a0 tx=0x7fba04009bb0 comp rx=0 tx=0).stop 2024-03-15T12:48:22.238 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.632+0000 7fba19eea700 1 -- 172.21.15.2:0/1540972811 shutdown_connections 2024-03-15T12:48:22.238 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.632+0000 7fba19eea700 1 --2- 172.21.15.2:0/1540972811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba141078e0 0x7fba14107d00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:22.238 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.632+0000 7fba19eea700 1 -- 172.21.15.2:0/1540972811 >> 172.21.15.2:0/1540972811 conn(0x7fba14102f20 msgr2=0x7fba14105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:22.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.632+0000 7fba19eea700 1 -- 172.21.15.2:0/1540972811 shutdown_connections 2024-03-15T12:48:22.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.632+0000 7fba19eea700 1 -- 172.21.15.2:0/1540972811 wait complete. 2024-03-15T12:48:22.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.632+0000 7fba19eea700 1 Processor -- start 2024-03-15T12:48:22.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.633+0000 7fba19eea700 1 -- start start 2024-03-15T12:48:22.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.633+0000 7fba19eea700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba141078e0 0x7fba1419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:22.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.633+0000 7fba19eea700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1419b240 con 0x7fba141078e0 2024-03-15T12:48:22.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.633+0000 7fba137fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba141078e0 0x7fba1419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:22.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.633+0000 7fba137fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba141078e0 0x7fba1419ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42718/0 (socket says 172.21.15.2:42718) 2024-03-15T12:48:22.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.633+0000 7fba137fe700 1 -- 172.21.15.2:0/1655638034 learned_addr learned my addr 172.21.15.2:0/1655638034 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:22.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.633+0000 7fba137fe700 1 -- 172.21.15.2:0/1655638034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba04009550 con 0x7fba141078e0 2024-03-15T12:48:22.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.633+0000 7fba137fe700 1 --2- 172.21.15.2:0/1655638034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba141078e0 0x7fba1419ad00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fba04004000 tx=0x7fba04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:22.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.633+0000 7fba10ff9700 1 -- 172.21.15.2:0/1655638034 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba04011630 con 0x7fba141078e0 2024-03-15T12:48:22.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.634+0000 7fba10ff9700 1 -- 172.21.15.2:0/1655638034 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba04011790 con 0x7fba141078e0 2024-03-15T12:48:22.239 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.634+0000 7fba19eea700 1 -- 172.21.15.2:0/1655638034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1419b440 con 0x7fba141078e0 2024-03-15T12:48:22.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.634+0000 7fba10ff9700 1 -- 172.21.15.2:0/1655638034 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba04011a30 con 0x7fba141078e0 2024-03-15T12:48:22.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.634+0000 7fba19eea700 1 -- 172.21.15.2:0/1655638034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1419b8e0 con 0x7fba141078e0 2024-03-15T12:48:22.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.634+0000 7fba10ff9700 1 -- 172.21.15.2:0/1655638034 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fba04022070 con 0x7fba141078e0 2024-03-15T12:48:22.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.634+0000 7fba10ff9700 1 -- 172.21.15.2:0/1655638034 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fba0401c450 con 0x7fba141078e0 2024-03-15T12:48:22.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.635+0000 7fba19eea700 1 -- 172.21.15.2:0/1655638034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba141949c0 con 0x7fba141078e0 2024-03-15T12:48:22.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.637+0000 7fba10ff9700 1 -- 172.21.15.2:0/1655638034 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fba04011b90 con 0x7fba141078e0 2024-03-15T12:48:22.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.707+0000 7fba19eea700 1 -- 172.21.15.2:0/1655638034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fba14108e00 con 0x7fba141078e0 2024-03-15T12:48:22.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.710+0000 7fba10ff9700 1 -- 172.21.15.2:0/1655638034 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7fba04011de0 con 0x7fba141078e0 2024-03-15T12:48:22.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.711+0000 7fba10ff9700 1 -- 172.21.15.2:0/1655638034 <== mon.0 v2:172.21.15.2:3300/0 8 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba0401c6b0 con 0x7fba141078e0 2024-03-15T12:48:22.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.712+0000 7fba19eea700 1 -- 172.21.15.2:0/1655638034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba141078e0 msgr2=0x7fba1419ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:22.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.712+0000 7fba19eea700 1 --2- 172.21.15.2:0/1655638034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba141078e0 0x7fba1419ad00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fba04004000 tx=0x7fba04004750 comp rx=0 tx=0).stop 2024-03-15T12:48:22.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.712+0000 7fba19eea700 1 -- 172.21.15.2:0/1655638034 shutdown_connections 2024-03-15T12:48:22.240 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.712+0000 7fba19eea700 1 --2- 172.21.15.2:0/1655638034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba141078e0 0x7fba1419ad00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:22.241 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.712+0000 7fba19eea700 1 -- 172.21.15.2:0/1655638034 >> 172.21.15.2:0/1655638034 conn(0x7fba14102f20 msgr2=0x7fba14191b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:22.241 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.712+0000 7fba19eea700 1 -- 172.21.15.2:0/1655638034 shutdown_connections 2024-03-15T12:48:22.241 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:21.712+0000 7fba19eea700 1 -- 172.21.15.2:0/1655638034 wait complete. 2024-03-15T12:48:22.241 INFO:teuthology.orchestra.run.smithi002.stdout:Generating new minimal ceph.conf... 2024-03-15T12:48:22.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:22 smithi002 ceph-mon[31684]: from='client.? 172.21.15.2:0/1655638034' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-15T12:48:22.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:22 smithi002 ceph-mon[31684]: from='client.? 172.21.15.2:0/1655638034' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-15T12:48:23.175 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:23 smithi002 ceph-mon[31684]: from='client.? 172.21.15.2:0/858895967' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:48:23.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.635+0000 7fa128cfb700 1 Processor -- start 2024-03-15T12:48:23.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.635+0000 7fa128cfb700 1 -- start start 2024-03-15T12:48:23.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.635+0000 7fa128cfb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1241067b0 0x7fa124106bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:23.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.635+0000 7fa128cfb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1241071a0 con 0x7fa1241067b0 2024-03-15T12:48:23.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.635+0000 7fa12259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1241067b0 0x7fa124106bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:23.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.635+0000 7fa12259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1241067b0 0x7fa124106bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42722/0 (socket says 172.21.15.2:42722) 2024-03-15T12:48:23.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.635+0000 7fa12259c700 1 -- 172.21.15.2:0/628060183 learned_addr learned my addr 172.21.15.2:0/628060183 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:23.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.636+0000 7fa12259c700 1 -- 172.21.15.2:0/628060183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1241079b0 con 0x7fa1241067b0 2024-03-15T12:48:23.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.636+0000 7fa12259c700 1 --2- 172.21.15.2:0/628060183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1241067b0 0x7fa124106bd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa1140098a0 tx=0x7fa114009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb8147cbfb2a4b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:23.182 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.636+0000 7fa12159a700 1 -- 172.21.15.2:0/628060183 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa114004160 con 0x7fa1241067b0 2024-03-15T12:48:23.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.636+0000 7fa12159a700 1 -- 172.21.15.2:0/628060183 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1140042c0 con 0x7fa1241067b0 2024-03-15T12:48:23.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.636+0000 7fa12159a700 1 -- 172.21.15.2:0/628060183 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa114011ab0 con 0x7fa1241067b0 2024-03-15T12:48:23.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.636+0000 7fa128cfb700 1 -- 172.21.15.2:0/628060183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1241067b0 msgr2=0x7fa124106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:23.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.636+0000 7fa128cfb700 1 --2- 172.21.15.2:0/628060183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1241067b0 0x7fa124106bd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa1140098a0 tx=0x7fa114009bb0 comp rx=0 tx=0).stop 2024-03-15T12:48:23.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.637+0000 7fa128cfb700 1 -- 172.21.15.2:0/628060183 shutdown_connections 2024-03-15T12:48:23.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.637+0000 7fa128cfb700 1 --2- 172.21.15.2:0/628060183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1241067b0 0x7fa124106bd0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:23.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.637+0000 7fa128cfb700 1 -- 172.21.15.2:0/628060183 >> 172.21.15.2:0/628060183 conn(0x7fa124101e10 msgr2=0x7fa124104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:23.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.637+0000 7fa128cfb700 1 -- 172.21.15.2:0/628060183 shutdown_connections 2024-03-15T12:48:23.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.637+0000 7fa128cfb700 1 -- 172.21.15.2:0/628060183 wait complete. 2024-03-15T12:48:23.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.637+0000 7fa128cfb700 1 Processor -- start 2024-03-15T12:48:23.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.637+0000 7fa128cfb700 1 -- start start 2024-03-15T12:48:23.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.637+0000 7fa128cfb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1241067b0 0x7fa12419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:23.183 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.637+0000 7fa128cfb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12419bcb0 con 0x7fa1241067b0 2024-03-15T12:48:23.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.638+0000 7fa12259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1241067b0 0x7fa12419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:23.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.638+0000 7fa12259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1241067b0 0x7fa12419b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42728/0 (socket says 172.21.15.2:42728) 2024-03-15T12:48:23.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.638+0000 7fa12259c700 1 -- 172.21.15.2:0/858895967 learned_addr learned my addr 172.21.15.2:0/858895967 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:23.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.638+0000 7fa12259c700 1 -- 172.21.15.2:0/858895967 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa114009550 con 0x7fa1241067b0 2024-03-15T12:48:23.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.638+0000 7fa12259c700 1 --2- 172.21.15.2:0/858895967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1241067b0 0x7fa12419b770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa114006af0 tx=0x7fa114011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:23.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.638+0000 7fa1137fe700 1 -- 172.21.15.2:0/858895967 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa114003a60 con 0x7fa1241067b0 2024-03-15T12:48:23.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.638+0000 7fa1137fe700 1 -- 172.21.15.2:0/858895967 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa114003bc0 con 0x7fa1241067b0 2024-03-15T12:48:23.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.638+0000 7fa1137fe700 1 -- 172.21.15.2:0/858895967 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa114019780 con 0x7fa1241067b0 2024-03-15T12:48:23.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.638+0000 7fa128cfb700 1 -- 172.21.15.2:0/858895967 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12419beb0 con 0x7fa1241067b0 2024-03-15T12:48:23.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.638+0000 7fa128cfb700 1 -- 172.21.15.2:0/858895967 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12419c350 con 0x7fa1241067b0 2024-03-15T12:48:23.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.638+0000 7fa1137fe700 1 -- 172.21.15.2:0/858895967 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa114024410 con 0x7fa1241067b0 2024-03-15T12:48:23.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.638+0000 7fa1137fe700 1 -- 172.21.15.2:0/858895967 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa11402c750 con 0x7fa1241067b0 2024-03-15T12:48:23.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.639+0000 7fa128cfb700 1 -- 172.21.15.2:0/858895967 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa124195850 con 0x7fa1241067b0 2024-03-15T12:48:23.184 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.641+0000 7fa1137fe700 1 -- 172.21.15.2:0/858895967 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa114019d80 con 0x7fa1241067b0 2024-03-15T12:48:23.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.712+0000 7fa128cfb700 1 -- 172.21.15.2:0/858895967 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa12410abb0 con 0x7fa1241067b0 2024-03-15T12:48:23.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.712+0000 7fa1137fe700 1 -- 172.21.15.2:0/858895967 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7fa11401ac10 con 0x7fa1241067b0 2024-03-15T12:48:23.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.713+0000 7fa128cfb700 1 -- 172.21.15.2:0/858895967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1241067b0 msgr2=0x7fa12419b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:23.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.713+0000 7fa128cfb700 1 --2- 172.21.15.2:0/858895967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1241067b0 0x7fa12419b770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa114006af0 tx=0x7fa114011b70 comp rx=0 tx=0).stop 2024-03-15T12:48:23.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.713+0000 7fa128cfb700 1 -- 172.21.15.2:0/858895967 shutdown_connections 2024-03-15T12:48:23.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.713+0000 7fa128cfb700 1 --2- 172.21.15.2:0/858895967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1241067b0 0x7fa12419b770 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:23.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.713+0000 7fa128cfb700 1 -- 172.21.15.2:0/858895967 >> 172.21.15.2:0/858895967 conn(0x7fa124101e10 msgr2=0x7fa124102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:23.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.713+0000 7fa128cfb700 1 -- 172.21.15.2:0/858895967 shutdown_connections 2024-03-15T12:48:23.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:22.713+0000 7fa128cfb700 1 -- 172.21.15.2:0/858895967 wait complete. 2024-03-15T12:48:23.185 INFO:teuthology.orchestra.run.smithi002.stdout:Restarting the monitor... 2024-03-15T12:48:23.453 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:23 smithi002 systemd[1]: Stopping Ceph mon.a for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T12:48:23.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:23 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-a[31680]: 2024-03-15T12:48:23.465+0000 7fa377ef7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T12:48:23.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:23 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-a[31680]: 2024-03-15T12:48:23.465+0000 7fa377ef7700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-15T12:48:23.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:23 smithi002 podman[31930]: 2024-03-15 12:48:23.660301979 +0000 UTC m=+0.305293082 container died e77cfc86b12c6db5ccfde9fc2782e111e9b1994455bc0a4f8467b129b80d0c9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T12:48:24.295 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 podman[31930]: 2024-03-15 12:48:24.040584957 +0000 UTC m=+0.685576068 container cleanup e77cfc86b12c6db5ccfde9fc2782e111e9b1994455bc0a4f8467b129b80d0c9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-a, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-15T12:48:24.295 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 bash[31930]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-a 2024-03-15T12:48:24.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 podman[31944]: 2024-03-15 12:48:24.308336084 +0000 UTC m=+0.647101040 container remove e77cfc86b12c6db5ccfde9fc2782e111e9b1994455bc0a4f8467b129b80d0c9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5) 2024-03-15T12:48:24.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 systemd[1]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-15T12:48:24.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 systemd[1]: Stopped Ceph mon.a for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:48:24.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 systemd[1]: Starting Ceph mon.a for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T12:48:24.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 podman[32040]: 2024-03-15 12:48:24.667647887 +0000 UTC m=+0.104040222 container create 8198c386c33e32c3cc1a0a058ec5e97bcdd512ca11e79b0c592e011fcb23501c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-a, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-15T12:48:24.923 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 podman[32040]: 2024-03-15 12:48:24.581481678 +0000 UTC m=+0.017873983 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-15T12:48:24.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 podman[32040]: 2024-03-15 12:48:24.861362554 +0000 UTC m=+0.297754848 container init 8198c386c33e32c3cc1a0a058ec5e97bcdd512ca11e79b0c592e011fcb23501c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-15T12:48:24.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 podman[32040]: 2024-03-15 12:48:24.864306338 +0000 UTC m=+0.300698634 container start 8198c386c33e32c3cc1a0a058ec5e97bcdd512ca11e79b0c592e011fcb23501c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-15T12:48:24.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: set uid:gid to 167:167 (ceph:ceph) 2024-03-15T12:48:24.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: ceph version 19.0.0-1681-gaba3c1d2 (aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b) squid (dev), process ceph-mon, pid 2 2024-03-15T12:48:24.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: pidfile_write: ignore empty --pid-file 2024-03-15T12:48:24.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: load: jerasure load: lrc 2024-03-15T12:48:24.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: RocksDB version: 7.9.2 2024-03-15T12:48:24.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Git sha 0 2024-03-15T12:48:24.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Compile date 2024-03-15 02:27:09 2024-03-15T12:48:24.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: DB SUMMARY 2024-03-15T12:48:24.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: DB Session ID: 7T2L2XBZSFOIJ76SF9SP 2024-03-15T12:48:24.924 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: CURRENT file: CURRENT 2024-03-15T12:48:24.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: IDENTITY file: IDENTITY 2024-03-15T12:48:24.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-15T12:48:24.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-15T12:48:24.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99676 ; 2024-03-15T12:48:24.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.error_if_exists: 0 2024-03-15T12:48:24.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.create_if_missing: 0 2024-03-15T12:48:24.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.paranoid_checks: 1 2024-03-15T12:48:24.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-15T12:48:24.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-15T12:48:24.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-15T12:48:24.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.env: 0x5613026b4ae0 2024-03-15T12:48:24.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.fs: PosixFileSystem 2024-03-15T12:48:24.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.info_log: 0x5613047a5900 2024-03-15T12:48:24.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_file_opening_threads: 16 2024-03-15T12:48:24.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.statistics: (nil) 2024-03-15T12:48:24.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.use_fsync: 0 2024-03-15T12:48:24.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_log_file_size: 0 2024-03-15T12:48:24.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-15T12:48:24.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-15T12:48:24.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.keep_log_file_num: 1000 2024-03-15T12:48:24.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.recycle_log_file_num: 0 2024-03-15T12:48:24.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.allow_fallocate: 1 2024-03-15T12:48:24.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.allow_mmap_reads: 0 2024-03-15T12:48:24.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.allow_mmap_writes: 0 2024-03-15T12:48:24.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.use_direct_reads: 0 2024-03-15T12:48:24.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-15T12:48:24.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.create_missing_column_families: 0 2024-03-15T12:48:24.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.db_log_dir: 2024-03-15T12:48:24.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.wal_dir: 2024-03-15T12:48:24.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-15T12:48:24.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-15T12:48:24.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-15T12:48:24.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-15T12:48:24.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-15T12:48:24.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-15T12:48:24.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.advise_random_on_open: 1 2024-03-15T12:48:24.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.db_write_buffer_size: 0 2024-03-15T12:48:24.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.write_buffer_manager: 0x561304779400 2024-03-15T12:48:24.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-15T12:48:24.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-15T12:48:24.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-15T12:48:24.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.rate_limiter: (nil) 2024-03-15T12:48:24.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-15T12:48:24.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.wal_recovery_mode: 2 2024-03-15T12:48:24.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.enable_thread_tracking: 0 2024-03-15T12:48:24.930 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.enable_pipelined_write: 0 2024-03-15T12:48:24.930 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.unordered_write: 0 2024-03-15T12:48:24.930 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-15T12:48:24.930 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-15T12:48:24.930 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-15T12:48:24.930 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-15T12:48:24.930 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.row_cache: None 2024-03-15T12:48:24.930 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.wal_filter: None 2024-03-15T12:48:24.930 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-15T12:48:24.930 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.allow_ingest_behind: 0 2024-03-15T12:48:24.930 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.two_write_queues: 0 2024-03-15T12:48:24.930 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.manual_wal_flush: 0 2024-03-15T12:48:24.930 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.wal_compression: 0 2024-03-15T12:48:24.931 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.atomic_flush: 0 2024-03-15T12:48:24.931 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-15T12:48:24.931 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-15T12:48:24.931 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-15T12:48:24.931 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.log_readahead_size: 0 2024-03-15T12:48:24.931 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-15T12:48:24.931 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.best_efforts_recovery: 0 2024-03-15T12:48:24.931 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-15T12:48:24.931 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-15T12:48:24.931 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.allow_data_in_errors: 0 2024-03-15T12:48:24.931 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.db_host_id: __hostname__ 2024-03-15T12:48:24.931 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-15T12:48:24.931 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_background_jobs: 2 2024-03-15T12:48:24.931 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_background_compactions: -1 2024-03-15T12:48:24.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_subcompactions: 1 2024-03-15T12:48:24.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-15T12:48:24.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-15T12:48:24.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-15T12:48:24.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_total_wal_size: 0 2024-03-15T12:48:24.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-15T12:48:24.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-15T12:48:24.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-15T12:48:24.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-15T12:48:24.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_open_files: -1 2024-03-15T12:48:24.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.bytes_per_sync: 0 2024-03-15T12:48:24.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-15T12:48:24.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-15T12:48:24.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compaction_readahead_size: 0 2024-03-15T12:48:24.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_background_flushes: -1 2024-03-15T12:48:24.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Compression algorithms supported: 2024-03-15T12:48:24.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-15T12:48:24.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: kZSTD supported: 0 2024-03-15T12:48:24.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: kXpressCompression supported: 0 2024-03-15T12:48:24.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: kLZ4HCCompression supported: 1 2024-03-15T12:48:24.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: kZlibCompression supported: 1 2024-03-15T12:48:24.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: kSnappyCompression supported: 1 2024-03-15T12:48:24.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: kLZ4Compression supported: 1 2024-03-15T12:48:24.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: kBZip2Compression supported: 0 2024-03-15T12:48:24.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-15T12:48:24.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-15T12:48:24.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-15T12:48:24.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-15T12:48:24.934 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-15T12:48:24.934 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.merge_operator: 2024-03-15T12:48:24.934 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compaction_filter: None 2024-03-15T12:48:24.934 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compaction_filter_factory: None 2024-03-15T12:48:24.934 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.sst_partitioner_factory: None 2024-03-15T12:48:24.934 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-15T12:48:24.934 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-15T12:48:24.934 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5613047a5a00) 2024-03-15T12:48:24.934 INFO:journalctl@ceph.mon.a.smithi002.stdout: cache_index_and_filter_blocks: 1 2024-03-15T12:48:24.934 INFO:journalctl@ceph.mon.a.smithi002.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-15T12:48:24.934 INFO:journalctl@ceph.mon.a.smithi002.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-15T12:48:24.934 INFO:journalctl@ceph.mon.a.smithi002.stdout: pin_top_level_index_and_filter: 1 2024-03-15T12:48:24.934 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_type: 0 2024-03-15T12:48:24.935 INFO:journalctl@ceph.mon.a.smithi002.stdout: data_block_index_type: 0 2024-03-15T12:48:24.935 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_shortening: 1 2024-03-15T12:48:24.935 INFO:journalctl@ceph.mon.a.smithi002.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-15T12:48:24.935 INFO:journalctl@ceph.mon.a.smithi002.stdout: checksum: 4 2024-03-15T12:48:24.935 INFO:journalctl@ceph.mon.a.smithi002.stdout: no_block_cache: 0 2024-03-15T12:48:24.935 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache: 0x5613047a9090 2024-03-15T12:48:24.935 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_name: BinnedLRUCache 2024-03-15T12:48:24.935 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_options: 2024-03-15T12:48:24.935 INFO:journalctl@ceph.mon.a.smithi002.stdout: capacity : 536870912 2024-03-15T12:48:24.935 INFO:journalctl@ceph.mon.a.smithi002.stdout: num_shard_bits : 4 2024-03-15T12:48:24.935 INFO:journalctl@ceph.mon.a.smithi002.stdout: strict_capacity_limit : 0 2024-03-15T12:48:24.935 INFO:journalctl@ceph.mon.a.smithi002.stdout: high_pri_pool_ratio: 0.000 2024-03-15T12:48:24.935 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_compressed: (nil) 2024-03-15T12:48:24.935 INFO:journalctl@ceph.mon.a.smithi002.stdout: persistent_cache: (nil) 2024-03-15T12:48:24.936 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_size: 4096 2024-03-15T12:48:24.936 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_size_deviation: 10 2024-03-15T12:48:24.936 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_restart_interval: 16 2024-03-15T12:48:24.936 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_block_restart_interval: 1 2024-03-15T12:48:24.936 INFO:journalctl@ceph.mon.a.smithi002.stdout: metadata_block_size: 4096 2024-03-15T12:48:24.936 INFO:journalctl@ceph.mon.a.smithi002.stdout: partition_filters: 0 2024-03-15T12:48:24.936 INFO:journalctl@ceph.mon.a.smithi002.stdout: use_delta_encoding: 1 2024-03-15T12:48:24.936 INFO:journalctl@ceph.mon.a.smithi002.stdout: filter_policy: bloomfilter 2024-03-15T12:48:24.936 INFO:journalctl@ceph.mon.a.smithi002.stdout: whole_key_filtering: 1 2024-03-15T12:48:24.936 INFO:journalctl@ceph.mon.a.smithi002.stdout: verify_compression: 0 2024-03-15T12:48:24.936 INFO:journalctl@ceph.mon.a.smithi002.stdout: read_amp_bytes_per_bit: 0 2024-03-15T12:48:24.936 INFO:journalctl@ceph.mon.a.smithi002.stdout: format_version: 5 2024-03-15T12:48:24.937 INFO:journalctl@ceph.mon.a.smithi002.stdout: enable_index_compression: 1 2024-03-15T12:48:24.937 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_align: 0 2024-03-15T12:48:24.937 INFO:journalctl@ceph.mon.a.smithi002.stdout: max_auto_readahead_size: 262144 2024-03-15T12:48:24.937 INFO:journalctl@ceph.mon.a.smithi002.stdout: prepopulate_block_cache: 0 2024-03-15T12:48:24.937 INFO:journalctl@ceph.mon.a.smithi002.stdout: initial_auto_readahead_size: 8192 2024-03-15T12:48:24.937 INFO:journalctl@ceph.mon.a.smithi002.stdout: num_file_reads_for_auto_readahead: 2 2024-03-15T12:48:24.937 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.write_buffer_size: 33554432 2024-03-15T12:48:24.937 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_write_buffer_number: 2 2024-03-15T12:48:24.937 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compression: NoCompression 2024-03-15T12:48:24.937 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.bottommost_compression: Disabled 2024-03-15T12:48:24.937 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.prefix_extractor: nullptr 2024-03-15T12:48:24.937 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-15T12:48:24.937 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.num_levels: 7 2024-03-15T12:48:24.937 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-15T12:48:24.938 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-15T12:48:24.938 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-15T12:48:24.938 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-15T12:48:24.938 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-15T12:48:24.938 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-15T12:48:24.938 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-15T12:48:24.938 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-15T12:48:24.938 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-15T12:48:24.938 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-15T12:48:24.938 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-15T12:48:24.938 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-15T12:48:24.938 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-15T12:48:24.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compression_opts.level: 32767 2024-03-15T12:48:24.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compression_opts.strategy: 0 2024-03-15T12:48:24.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-15T12:48:24.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-15T12:48:24.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-15T12:48:24.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-15T12:48:24.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compression_opts.enabled: false 2024-03-15T12:48:24.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-15T12:48:24.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-15T12:48:24.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-15T12:48:24.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-15T12:48:24.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.target_file_size_base: 67108864 2024-03-15T12:48:24.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-15T12:48:24.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-15T12:48:24.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-15T12:48:24.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-15T12:48:24.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-15T12:48:24.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-15T12:48:24.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-15T12:48:24.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-15T12:48:24.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-15T12:48:24.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-15T12:48:24.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-15T12:48:24.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-15T12:48:24.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-15T12:48:24.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-15T12:48:24.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.arena_block_size: 1048576 2024-03-15T12:48:24.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-15T12:48:24.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-15T12:48:24.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.disable_auto_compactions: 0 2024-03-15T12:48:24.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-15T12:48:24.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-15T12:48:24.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-15T12:48:24.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-15T12:48:24.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-15T12:48:24.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-15T12:48:24.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-15T12:48:24.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-15T12:48:24.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-15T12:48:24.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-15T12:48:24.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.table_properties_collectors: 2024-03-15T12:48:24.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.inplace_update_support: 0 2024-03-15T12:48:24.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-15T12:48:24.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-15T12:48:24.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-15T12:48:24.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-15T12:48:24.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.bloom_locality: 0 2024-03-15T12:48:24.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.max_successive_merges: 0 2024-03-15T12:48:24.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-15T12:48:24.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.paranoid_file_checks: 0 2024-03-15T12:48:24.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.force_consistency_checks: 1 2024-03-15T12:48:24.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.report_bg_io_stats: 0 2024-03-15T12:48:24.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.ttl: 2592000 2024-03-15T12:48:24.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-15T12:48:24.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-15T12:48:24.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-15T12:48:24.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.enable_blob_files: false 2024-03-15T12:48:24.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.min_blob_size: 0 2024-03-15T12:48:24.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.blob_file_size: 268435456 2024-03-15T12:48:24.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-15T12:48:24.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-15T12:48:24.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-15T12:48:24.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-15T12:48:24.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-15T12:48:24.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.blob_file_starting_level: 0 2024-03-15T12:48:24.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-15T12:48:24.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-15T12:48:24.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-15T12:48:24.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7f5532a8-0c4d-4260-8fbd-bc09662c1a82 2024-03-15T12:48:24.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710506904892455, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-15T12:48:24.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-15T12:48:24.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710506904893389, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94858, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92856, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13773, "raw_average_key_size": 50, "raw_value_size": 85364, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710506904, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7f5532a8-0c4d-4260-8fbd-bc09662c1a82", "db_session_id": "7T2L2XBZSFOIJ76SF9SP", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-15T12:48:24.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710506904893473, "job": 1, "event": "recovery_finished"} 2024-03-15T12:48:24.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-15T12:48:24.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-15T12:48:24.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56130489a000 2024-03-15T12:48:24.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: DB pointer 0x561304884000 2024-03-15T12:48:24.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-15T12:48:24.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-15T12:48:24.945 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** DB Stats ** 2024-03-15T12:48:24.945 INFO:journalctl@ceph.mon.a.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-15T12:48:24.945 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-15T12:48:24.945 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-15T12:48:24.945 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-15T12:48:24.945 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-15T12:48:24.945 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-15T12:48:24.945 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-15T12:48:24.945 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-03-15T12:48:24.945 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** Compaction Stats [default] ** 2024-03-15T12:48:24.945 INFO:journalctl@ceph.mon.a.smithi002.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-15T12:48:24.945 INFO:journalctl@ceph.mon.a.smithi002.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-15T12:48:24.945 INFO:journalctl@ceph.mon.a.smithi002.stdout: L0 2/0 94.47 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 152.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-15T12:48:24.946 INFO:journalctl@ceph.mon.a.smithi002.stdout: Sum 2/0 94.47 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 152.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-15T12:48:24.946 INFO:journalctl@ceph.mon.a.smithi002.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 152.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-15T12:48:24.946 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-03-15T12:48:24.946 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** Compaction Stats [default] ** 2024-03-15T12:48:24.946 INFO:journalctl@ceph.mon.a.smithi002.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-15T12:48:24.946 INFO:journalctl@ceph.mon.a.smithi002.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-15T12:48:24.946 INFO:journalctl@ceph.mon.a.smithi002.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 152.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-15T12:48:24.946 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-03-15T12:48:24.946 INFO:journalctl@ceph.mon.a.smithi002.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-15T12:48:24.946 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-03-15T12:48:24.946 INFO:journalctl@ceph.mon.a.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-15T12:48:24.946 INFO:journalctl@ceph.mon.a.smithi002.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-15T12:48:24.946 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-15T12:48:24.946 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-15T12:48:24.947 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-15T12:48:24.947 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-15T12:48:24.947 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative compaction: 0.00 GB write, 30.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-15T12:48:24.947 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval compaction: 0.00 GB write, 30.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-15T12:48:24.947 INFO:journalctl@ceph.mon.a.smithi002.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-15T12:48:24.947 INFO:journalctl@ceph.mon.a.smithi002.stdout: Block cache BinnedLRUCache@0x5613047a9090#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-15T12:48:24.947 INFO:journalctl@ceph.mon.a.smithi002.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-03-15T12:48:24.947 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-03-15T12:48:24.947 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-15T12:48:24.947 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: starting mon.a rank 0 at public addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] at bind addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:48:24.948 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: mon.a@-1(???) e1 preinit fsid 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:48:24.948 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: mon.a@-1(???).mds e1 new map 2024-03-15T12:48:24.948 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: mon.a@-1(???).mds e1 print_map 2024-03-15T12:48:24.948 INFO:journalctl@ceph.mon.a.smithi002.stdout: e1 2024-03-15T12:48:24.948 INFO:journalctl@ceph.mon.a.smithi002.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-15T12:48:24.948 INFO:journalctl@ceph.mon.a.smithi002.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-15T12:48:24.948 INFO:journalctl@ceph.mon.a.smithi002.stdout: legacy client fscid: -1 2024-03-15T12:48:24.948 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-03-15T12:48:24.948 INFO:journalctl@ceph.mon.a.smithi002.stdout: No filesystems configured 2024-03-15T12:48:24.948 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-15T12:48:24.948 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-15T12:48:24.948 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-15T12:48:24.949 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-15T12:48:24.949 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-15T12:48:24.949 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 bash[32040]: 8198c386c33e32c3cc1a0a058ec5e97bcdd512ca11e79b0c592e011fcb23501c 2024-03-15T12:48:24.949 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 systemd[1]: Started Ceph mon.a for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:48:24.949 INFO:teuthology.orchestra.run.smithi002.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-15T12:48:25.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-15T12:48:25.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: monmap e1: 1 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-15T12:48:25.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: fsmap 2024-03-15T12:48:25.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: osdmap e1: 0 total, 0 up, 0 in 2024-03-15T12:48:25.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:24 smithi002 ceph-mon[32060]: mgrmap e1: no daemons active 2024-03-15T12:48:26.208 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.327+0000 7f50e11b5700 1 Processor -- start 2024-03-15T12:48:26.208 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.327+0000 7f50e11b5700 1 -- start start 2024-03-15T12:48:26.208 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.327+0000 7f50e11b5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50dc1078e0 0x7f50dc107d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:26.208 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.327+0000 7f50e11b5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc1082d0 con 0x7f50dc1078e0 2024-03-15T12:48:26.208 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.327+0000 7f50dad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50dc1078e0 0x7f50dc107d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:26.208 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.327+0000 7f50dad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50dc1078e0 0x7f50dc107d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42740/0 (socket says 172.21.15.2:42740) 2024-03-15T12:48:26.208 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.327+0000 7f50dad9d700 1 -- 172.21.15.2:0/1304133241 learned_addr learned my addr 172.21.15.2:0/1304133241 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:26.208 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.327+0000 7f50dad9d700 1 -- 172.21.15.2:0/1304133241 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc108a90 con 0x7f50dc1078e0 2024-03-15T12:48:26.208 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.328+0000 7f50dad9d700 1 --2- 172.21.15.2:0/1304133241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50dc1078e0 0x7f50dc107d00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0098a0 tx=0x7f50cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76d7033661088e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:26.208 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.328+0000 7f50d9d9b700 1 -- 172.21.15.2:0/1304133241 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50cc004160 con 0x7f50dc1078e0 2024-03-15T12:48:26.208 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.328+0000 7f50d9d9b700 1 -- 172.21.15.2:0/1304133241 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50cc0042c0 con 0x7f50dc1078e0 2024-03-15T12:48:26.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.328+0000 7f50d9d9b700 1 -- 172.21.15.2:0/1304133241 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50cc011ab0 con 0x7f50dc1078e0 2024-03-15T12:48:26.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.328+0000 7f50e11b5700 1 -- 172.21.15.2:0/1304133241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50dc1078e0 msgr2=0x7f50dc107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:26.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.328+0000 7f50e11b5700 1 --2- 172.21.15.2:0/1304133241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50dc1078e0 0x7f50dc107d00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0098a0 tx=0x7f50cc009bb0 comp rx=0 tx=0).stop 2024-03-15T12:48:26.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.328+0000 7f50e11b5700 1 -- 172.21.15.2:0/1304133241 shutdown_connections 2024-03-15T12:48:26.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.328+0000 7f50e11b5700 1 --2- 172.21.15.2:0/1304133241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50dc1078e0 0x7f50dc107d00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:26.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.328+0000 7f50e11b5700 1 -- 172.21.15.2:0/1304133241 >> 172.21.15.2:0/1304133241 conn(0x7f50dc102f00 msgr2=0x7f50dc105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:26.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.329+0000 7f50e11b5700 1 -- 172.21.15.2:0/1304133241 shutdown_connections 2024-03-15T12:48:26.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.329+0000 7f50e11b5700 1 -- 172.21.15.2:0/1304133241 wait complete. 2024-03-15T12:48:26.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.329+0000 7f50e11b5700 1 Processor -- start 2024-03-15T12:48:26.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.329+0000 7f50e11b5700 1 -- start start 2024-03-15T12:48:26.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.329+0000 7f50e11b5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50dc1078e0 0x7f50dc07cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:26.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.330+0000 7f50e11b5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc07d0c0 con 0x7f50dc1078e0 2024-03-15T12:48:26.209 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.330+0000 7f50dad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50dc1078e0 0x7f50dc07cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:26.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.330+0000 7f50dad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50dc1078e0 0x7f50dc07cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42756/0 (socket says 172.21.15.2:42756) 2024-03-15T12:48:26.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.330+0000 7f50dad9d700 1 -- 172.21.15.2:0/2614402406 learned_addr learned my addr 172.21.15.2:0/2614402406 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:26.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.330+0000 7f50dad9d700 1 -- 172.21.15.2:0/2614402406 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50cc009550 con 0x7f50dc1078e0 2024-03-15T12:48:26.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.330+0000 7f50dad9d700 1 --2- 172.21.15.2:0/2614402406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50dc1078e0 0x7f50dc07cb80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f50cc006af0 tx=0x7f50cc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:26.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.330+0000 7f50c3fff700 1 -- 172.21.15.2:0/2614402406 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50cc003930 con 0x7f50dc1078e0 2024-03-15T12:48:26.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.330+0000 7f50e11b5700 1 -- 172.21.15.2:0/2614402406 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50dc07b2b0 con 0x7f50dc1078e0 2024-03-15T12:48:26.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.330+0000 7f50c3fff700 1 -- 172.21.15.2:0/2614402406 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50cc003a90 con 0x7f50dc1078e0 2024-03-15T12:48:26.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.330+0000 7f50c3fff700 1 -- 172.21.15.2:0/2614402406 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50cc019670 con 0x7f50dc1078e0 2024-03-15T12:48:26.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.330+0000 7f50e11b5700 1 -- 172.21.15.2:0/2614402406 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50dc07b750 con 0x7f50dc1078e0 2024-03-15T12:48:26.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.331+0000 7f50c3fff700 1 -- 172.21.15.2:0/2614402406 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f50cc019cc0 con 0x7f50dc1078e0 2024-03-15T12:48:26.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.331+0000 7f50c3fff700 1 -- 172.21.15.2:0/2614402406 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f50cc024a40 con 0x7f50dc1078e0 2024-03-15T12:48:26.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.331+0000 7f50e11b5700 1 -- 172.21.15.2:0/2614402406 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50dc063610 con 0x7f50dc1078e0 2024-03-15T12:48:26.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.334+0000 7f50c3fff700 1 -- 172.21.15.2:0/2614402406 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f50cc003c00 con 0x7f50dc1078e0 2024-03-15T12:48:26.210 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.404+0000 7f50e11b5700 1 -- 172.21.15.2:0/2614402406 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f50dc10bed0 con 0x7f50dc1078e0 2024-03-15T12:48:26.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.405+0000 7f50c3fff700 1 -- 172.21.15.2:0/2614402406 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f50cc01ab20 con 0x7f50dc1078e0 2024-03-15T12:48:26.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.406+0000 7f50e11b5700 1 -- 172.21.15.2:0/2614402406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50dc1078e0 msgr2=0x7f50dc07cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:26.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.406+0000 7f50e11b5700 1 --2- 172.21.15.2:0/2614402406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50dc1078e0 0x7f50dc07cb80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f50cc006af0 tx=0x7f50cc011b70 comp rx=0 tx=0).stop 2024-03-15T12:48:26.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.406+0000 7f50e11b5700 1 -- 172.21.15.2:0/2614402406 shutdown_connections 2024-03-15T12:48:26.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.406+0000 7f50e11b5700 1 --2- 172.21.15.2:0/2614402406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50dc1078e0 0x7f50dc07cb80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:26.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.406+0000 7f50e11b5700 1 -- 172.21.15.2:0/2614402406 >> 172.21.15.2:0/2614402406 conn(0x7f50dc102f00 msgr2=0x7f50dc103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:26.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.406+0000 7f50e11b5700 1 -- 172.21.15.2:0/2614402406 shutdown_connections 2024-03-15T12:48:26.211 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:25.406+0000 7f50e11b5700 1 -- 172.21.15.2:0/2614402406 wait complete. 2024-03-15T12:48:26.211 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-15T12:48:26.471 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-15T12:48:26.471 INFO:teuthology.orchestra.run.smithi002.stdout:Creating mgr... 2024-03-15T12:48:26.472 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-15T12:48:26.472 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-15T12:48:26.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:26 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2614402406' entity='client.admin' 2024-03-15T12:48:27.557 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.x 2024-03-15T12:48:27.557 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.x.service: Unit ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.x.service not loaded. 2024-03-15T12:48:27.849 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2767718a-e2ca-11ee-95c9-87774f69a715.target.wants/ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-2767718a-e2ca-11ee-95c9-87774f69a715@.service. 2024-03-15T12:48:28.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:28 smithi002 podman[32278]: 2024-03-15 12:48:27.997988218 +0000 UTC m=+0.018587939 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-15T12:48:29.191 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:28 smithi002 podman[32278]: 2024-03-15 12:48:28.950365324 +0000 UTC m=+0.970965033 container create 52c2b5b203c7614e0fc2f11fb21673dcbd10afc47d5e374a51dd406d58a5f067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux ) 2024-03-15T12:48:30.804 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:30 smithi002 podman[32278]: 2024-03-15 12:48:30.502333681 +0000 UTC m=+2.522933392 container init 52c2b5b203c7614e0fc2f11fb21673dcbd10afc47d5e374a51dd406d58a5f067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-15T12:48:30.804 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:30 smithi002 podman[32278]: 2024-03-15 12:48:30.5056346 +0000 UTC m=+2.526234311 container start 52c2b5b203c7614e0fc2f11fb21673dcbd10afc47d5e374a51dd406d58a5f067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-15T12:48:31.442 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:31 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:31.071+0000 7f5844fe1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-15T12:48:31.442 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:31 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:31.166+0000 7f5844fe1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-15T12:48:31.454 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-15T12:48:31.455 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-03-15T12:48:31.462 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-15T12:48:31.462 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-03-15T12:48:31.462 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-03-15T12:48:31.462 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to enable service . firewalld.service is not available 2024-03-15T12:48:31.471 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-15T12:48:31.471 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-03-15T12:48:31.478 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-15T12:48:31.478 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-03-15T12:48:31.479 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-03-15T12:48:31.479 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-15T12:48:31.479 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr to start... 2024-03-15T12:48:31.479 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr... 2024-03-15T12:48:31.804 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:31 smithi002 bash[32278]: 52c2b5b203c7614e0fc2f11fb21673dcbd10afc47d5e374a51dd406d58a5f067 2024-03-15T12:48:31.804 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:31 smithi002 systemd[1]: Started Ceph mgr.x for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:48:32.167 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:31 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:31.843+0000 7f5844fe1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-15T12:48:32.168 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:31 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:31.938+0000 7f5844fe1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-15T12:48:32.168 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:32 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:32.081+0000 7f5844fe1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-15T12:48:32.168 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:32 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:32.180+0000 7f5844fe1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-15T12:48:32.521 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:32 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:32.265+0000 7f5844fe1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-15T12:48:32.521 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:32 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:32.534+0000 7f5844fe1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-15T12:48:32.804 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:32 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:32.648+0000 7f5844fe1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-15T12:48:32.804 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:32 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:32.737+0000 7f5844fe1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-15T12:48:33.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:32 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:32.834+0000 7f5844fe1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-15T12:48:33.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:33 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:33.011+0000 7f5844fe1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-15T12:48:34.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:34 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:34.017+0000 7f5844fe1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-15T12:48:34.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:34 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:34.151+0000 7f5844fe1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-15T12:48:34.632 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:34 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3923371022' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-15T12:48:34.632 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:34 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:34.493+0000 7f5844fe1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-15T12:48:34.632 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:34 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:34.592+0000 7f5844fe1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-15T12:48:34.971 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:34 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:34.793+0000 7f5844fe1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:48:34.972 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:34 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:34.984+0000 7f5844fe1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-15T12:48:35.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-15T12:48:35.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-03-15T12:48:35.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "2767718a-e2ca-11ee-95c9-87774f69a715", 2024-03-15T12:48:35.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-03-15T12:48:35.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-15T12:48:35.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-15T12:48:35.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-15T12:48:35.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:35.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-15T12:48:35.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-15T12:48:35.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-03-15T12:48:35.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-15T12:48:35.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-15T12:48:35.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2024-03-15T12:48:35.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-15T12:48:35.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-03-15T12:48:35.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-15T12:48:35.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-15T12:48:35.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-15T12:48:35.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-15T12:48:35.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:35.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-15T12:48:35.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-15T12:48:35.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-15T12:48:35.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-15T12:48:35.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-15T12:48:35.268 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-15T12:48:35.268 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-15T12:48:35.268 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-15T12:48:35.268 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:35.268 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-15T12:48:35.268 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-15T12:48:35.268 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-15T12:48:35.268 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-15T12:48:35.268 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-15T12:48:35.269 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-15T12:48:35.269 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-15T12:48:35.269 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-15T12:48:35.269 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-15T12:48:35.269 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:35.269 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-15T12:48:35.269 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-15T12:48:35.269 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-15T12:48:35.269 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-15T12:48:35.269 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:35.269 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-15T12:48:35.269 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-15T12:48:35.269 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-15T12:48:35.269 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-15T12:48:35.270 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-15T12:48:35.270 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-15T12:48:35.270 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-03-15T12:48:35.270 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-15T12:48:35.270 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-15T12:48:35.270 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:35.270 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-15T12:48:35.270 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-15T12:48:35.270 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-03-15T12:48:20.096386+0000", 2024-03-15T12:48:35.270 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-15T12:48:35.270 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:35.270 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-15T12:48:35.270 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-03-15T12:48:35.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.477+0000 7fb1d7d46700 1 Processor -- start 2024-03-15T12:48:35.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.477+0000 7fb1d7d46700 1 -- start start 2024-03-15T12:48:35.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.477+0000 7fb1d7d46700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1d0108350 0x7fb1d010a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:35.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.477+0000 7fb1d7d46700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d007aec0 con 0x7fb1d0108350 2024-03-15T12:48:35.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.477+0000 7fb1d5ae2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1d0108350 0x7fb1d010a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:35.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.477+0000 7fb1d5ae2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1d0108350 0x7fb1d010a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44266/0 (socket says 172.21.15.2:44266) 2024-03-15T12:48:35.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.477+0000 7fb1d5ae2700 1 -- 172.21.15.2:0/2642908441 learned_addr learned my addr 172.21.15.2:0/2642908441 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:35.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.477+0000 7fb1d5ae2700 1 -- 172.21.15.2:0/2642908441 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d010ac80 con 0x7fb1d0108350 2024-03-15T12:48:35.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.478+0000 7fb1d5ae2700 1 --2- 172.21.15.2:0/2642908441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1d0108350 0x7fb1d010a740 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00098a0 tx=0x7fb1c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77805def3b2e788e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:35.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.478+0000 7fb1d4ae0700 1 -- 172.21.15.2:0/2642908441 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1c0004160 con 0x7fb1d0108350 2024-03-15T12:48:35.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.478+0000 7fb1d4ae0700 1 -- 172.21.15.2:0/2642908441 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1c00042c0 con 0x7fb1d0108350 2024-03-15T12:48:35.271 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.478+0000 7fb1d4ae0700 1 -- 172.21.15.2:0/2642908441 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1c0011ab0 con 0x7fb1d0108350 2024-03-15T12:48:35.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.478+0000 7fb1d7d46700 1 -- 172.21.15.2:0/2642908441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1d0108350 msgr2=0x7fb1d010a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:35.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.478+0000 7fb1d7d46700 1 --2- 172.21.15.2:0/2642908441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1d0108350 0x7fb1d010a740 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00098a0 tx=0x7fb1c0009bb0 comp rx=0 tx=0).stop 2024-03-15T12:48:35.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.478+0000 7fb1d7d46700 1 -- 172.21.15.2:0/2642908441 shutdown_connections 2024-03-15T12:48:35.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.478+0000 7fb1d7d46700 1 --2- 172.21.15.2:0/2642908441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1d0108350 0x7fb1d010a740 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:35.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.478+0000 7fb1d7d46700 1 -- 172.21.15.2:0/2642908441 >> 172.21.15.2:0/2642908441 conn(0x7fb1d0101e60 msgr2=0x7fb1d0104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:35.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.479+0000 7fb1d7d46700 1 -- 172.21.15.2:0/2642908441 shutdown_connections 2024-03-15T12:48:35.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.479+0000 7fb1d7d46700 1 -- 172.21.15.2:0/2642908441 wait complete. 2024-03-15T12:48:35.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.479+0000 7fb1d7d46700 1 Processor -- start 2024-03-15T12:48:35.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.479+0000 7fb1d7d46700 1 -- start start 2024-03-15T12:48:35.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.479+0000 7fb1d7d46700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1d0108350 0x7fb1d019b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:35.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.479+0000 7fb1d7d46700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d019bbe0 con 0x7fb1d0108350 2024-03-15T12:48:35.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.479+0000 7fb1d5ae2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1d0108350 0x7fb1d019b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:35.272 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.479+0000 7fb1d5ae2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1d0108350 0x7fb1d019b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44278/0 (socket says 172.21.15.2:44278) 2024-03-15T12:48:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.479+0000 7fb1d5ae2700 1 -- 172.21.15.2:0/3923371022 learned_addr learned my addr 172.21.15.2:0/3923371022 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.480+0000 7fb1d5ae2700 1 -- 172.21.15.2:0/3923371022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c0009550 con 0x7fb1d0108350 2024-03-15T12:48:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.480+0000 7fb1d5ae2700 1 --2- 172.21.15.2:0/3923371022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1d0108350 0x7fb1d019b6a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0011670 tx=0x7fb1c0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.480+0000 7fb1c6ffd700 1 -- 172.21.15.2:0/3923371022 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1c00039f0 con 0x7fb1d0108350 2024-03-15T12:48:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.480+0000 7fb1c6ffd700 1 -- 172.21.15.2:0/3923371022 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1c0003b50 con 0x7fb1d0108350 2024-03-15T12:48:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.480+0000 7fb1d7d46700 1 -- 172.21.15.2:0/3923371022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d019bde0 con 0x7fb1d0108350 2024-03-15T12:48:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.480+0000 7fb1c6ffd700 1 -- 172.21.15.2:0/3923371022 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1c0019780 con 0x7fb1d0108350 2024-03-15T12:48:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.480+0000 7fb1d7d46700 1 -- 172.21.15.2:0/3923371022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d019c280 con 0x7fb1d0108350 2024-03-15T12:48:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.480+0000 7fb1c6ffd700 1 -- 172.21.15.2:0/3923371022 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb1c002d430 con 0x7fb1d0108350 2024-03-15T12:48:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.480+0000 7fb1c6ffd700 1 -- 172.21.15.2:0/3923371022 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb1c0024a30 con 0x7fb1d0108350 2024-03-15T12:48:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.481+0000 7fb1d7d46700 1 -- 172.21.15.2:0/3923371022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d010bae0 con 0x7fb1d0108350 2024-03-15T12:48:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.484+0000 7fb1c6ffd700 1 -- 172.21.15.2:0/3923371022 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb1c0019e20 con 0x7fb1d0108350 2024-03-15T12:48:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.557+0000 7fb1d7d46700 1 -- 172.21.15.2:0/3923371022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb1d019edf0 con 0x7fb1d0108350 2024-03-15T12:48:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.558+0000 7fb1c6ffd700 1 -- 172.21.15.2:0/3923371022 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fb1c0024c90 con 0x7fb1d0108350 2024-03-15T12:48:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.559+0000 7fb1d7d46700 1 -- 172.21.15.2:0/3923371022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1d0108350 msgr2=0x7fb1d019b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:35.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.559+0000 7fb1d7d46700 1 --2- 172.21.15.2:0/3923371022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1d0108350 0x7fb1d019b6a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0011670 tx=0x7fb1c0011b70 comp rx=0 tx=0).stop 2024-03-15T12:48:35.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.559+0000 7fb1d7d46700 1 -- 172.21.15.2:0/3923371022 shutdown_connections 2024-03-15T12:48:35.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.559+0000 7fb1d7d46700 1 --2- 172.21.15.2:0/3923371022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1d0108350 0x7fb1d019b6a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:35.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.559+0000 7fb1d7d46700 1 -- 172.21.15.2:0/3923371022 >> 172.21.15.2:0/3923371022 conn(0x7fb1d0101e60 msgr2=0x7fb1d0102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:35.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.559+0000 7fb1d7d46700 1 -- 172.21.15.2:0/3923371022 shutdown_connections 2024-03-15T12:48:35.274 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:34.559+0000 7fb1d7d46700 1 -- 172.21.15.2:0/3923371022 wait complete. 2024-03-15T12:48:35.274 INFO:teuthology.orchestra.run.smithi002.stdout:mgr not available, waiting (1/15)... 2024-03-15T12:48:35.529 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:35 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:35.280+0000 7f5844fe1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:48:35.529 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:35 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:35.541+0000 7f5844fe1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-15T12:48:35.804 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:35 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:35.784+0000 7f5844fe1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-15T12:48:37.054 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:36 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:36.611+0000 7f5844fe1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-15T12:48:37.459 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:37.191+0000 7f5844fe1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-15T12:48:37.460 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:37.293+0000 7f5844fe1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-15T12:48:37.734 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:37.521+0000 7f5844fe1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-15T12:48:37.734 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-mon[32060]: Activating manager daemon x 2024-03-15T12:48:37.734 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-mon[32060]: mgrmap e2: x(active, starting, since 0.00286233s) 2024-03-15T12:48:37.734 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-mon[32060]: from='mgr.14100 172.21.15.2:0/551277271' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-15T12:48:37.734 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-mon[32060]: from='mgr.14100 172.21.15.2:0/551277271' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-15T12:48:37.734 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-mon[32060]: from='mgr.14100 172.21.15.2:0/551277271' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-15T12:48:37.734 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-mon[32060]: from='mgr.14100 172.21.15.2:0/551277271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:48:37.735 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-mon[32060]: from='mgr.14100 172.21.15.2:0/551277271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-15T12:48:37.735 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-mon[32060]: Manager daemon x is now available 2024-03-15T12:48:37.735 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-mon[32060]: from='mgr.14100 172.21.15.2:0/551277271' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-15T12:48:37.735 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-mon[32060]: from='mgr.14100 172.21.15.2:0/551277271' entity='mgr.x' 2024-03-15T12:48:37.735 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-mon[32060]: from='mgr.14100 172.21.15.2:0/551277271' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-15T12:48:37.735 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-mon[32060]: from='mgr.14100 172.21.15.2:0/551277271' entity='mgr.x' 2024-03-15T12:48:37.735 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:37 smithi002 ceph-mon[32060]: from='mgr.14100 172.21.15.2:0/551277271' entity='mgr.x' 2024-03-15T12:48:38.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-15T12:48:38.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-03-15T12:48:38.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "2767718a-e2ca-11ee-95c9-87774f69a715", 2024-03-15T12:48:38.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-03-15T12:48:38.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-15T12:48:38.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-15T12:48:38.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-15T12:48:38.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:38.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-15T12:48:38.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-15T12:48:38.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-03-15T12:48:38.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-15T12:48:38.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-15T12:48:38.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2024-03-15T12:48:38.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-15T12:48:38.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-15T12:48:38.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-15T12:48:38.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-15T12:48:38.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-15T12:48:38.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-15T12:48:38.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:38.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-15T12:48:38.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-15T12:48:38.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-15T12:48:38.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-15T12:48:38.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-15T12:48:38.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-15T12:48:38.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-15T12:48:38.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-15T12:48:38.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:38.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-15T12:48:38.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-15T12:48:38.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-15T12:48:38.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-15T12:48:38.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-15T12:48:38.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-15T12:48:38.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-15T12:48:38.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-15T12:48:38.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-15T12:48:38.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:38.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-15T12:48:38.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-15T12:48:38.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-15T12:48:38.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-15T12:48:38.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:38.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-15T12:48:38.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-15T12:48:38.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-15T12:48:38.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-15T12:48:38.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-15T12:48:38.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-15T12:48:38.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-03-15T12:48:38.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-15T12:48:38.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-15T12:48:38.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:38.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-15T12:48:38.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-15T12:48:38.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-03-15T12:48:20.096386+0000", 2024-03-15T12:48:38.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-15T12:48:38.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:38.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-15T12:48:38.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-03-15T12:48:38.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.667+0000 7f4b290d9700 1 Processor -- start 2024-03-15T12:48:38.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.667+0000 7f4b290d9700 1 -- start start 2024-03-15T12:48:38.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.667+0000 7f4b290d9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b24107910 0x7f4b24107d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:38.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.667+0000 7f4b290d9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b24108300 con 0x7f4b24107910 2024-03-15T12:48:38.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.667+0000 7f4b22d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b24107910 0x7f4b24107d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:38.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.667+0000 7f4b22d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b24107910 0x7f4b24107d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44360/0 (socket says 172.21.15.2:44360) 2024-03-15T12:48:38.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.667+0000 7f4b22d9d700 1 -- 172.21.15.2:0/3630216784 learned_addr learned my addr 172.21.15.2:0/3630216784 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:38.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.668+0000 7f4b22d9d700 1 -- 172.21.15.2:0/3630216784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b24108ac0 con 0x7f4b24107910 2024-03-15T12:48:38.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.668+0000 7f4b22d9d700 1 --2- 172.21.15.2:0/3630216784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b24107910 0x7f4b24107d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4b140098a0 tx=0x7f4b14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b64c8edbdf49f05 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:38.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.668+0000 7f4b21d9b700 1 -- 172.21.15.2:0/3630216784 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b14004160 con 0x7f4b24107910 2024-03-15T12:48:38.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.668+0000 7f4b21d9b700 1 -- 172.21.15.2:0/3630216784 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b140042c0 con 0x7f4b24107910 2024-03-15T12:48:38.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.668+0000 7f4b21d9b700 1 -- 172.21.15.2:0/3630216784 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b14011ab0 con 0x7f4b24107910 2024-03-15T12:48:38.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.668+0000 7f4b290d9700 1 -- 172.21.15.2:0/3630216784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b24107910 msgr2=0x7f4b24107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:38.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.668+0000 7f4b290d9700 1 --2- 172.21.15.2:0/3630216784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b24107910 0x7f4b24107d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4b140098a0 tx=0x7f4b14009bb0 comp rx=0 tx=0).stop 2024-03-15T12:48:38.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.669+0000 7f4b290d9700 1 -- 172.21.15.2:0/3630216784 shutdown_connections 2024-03-15T12:48:38.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.669+0000 7f4b290d9700 1 --2- 172.21.15.2:0/3630216784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b24107910 0x7f4b24107d30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:38.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.669+0000 7f4b290d9700 1 -- 172.21.15.2:0/3630216784 >> 172.21.15.2:0/3630216784 conn(0x7f4b24102f50 msgr2=0x7f4b24105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:38.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.669+0000 7f4b290d9700 1 -- 172.21.15.2:0/3630216784 shutdown_connections 2024-03-15T12:48:38.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.669+0000 7f4b290d9700 1 -- 172.21.15.2:0/3630216784 wait complete. 2024-03-15T12:48:38.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.669+0000 7f4b290d9700 1 Processor -- start 2024-03-15T12:48:38.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.669+0000 7f4b290d9700 1 -- start start 2024-03-15T12:48:38.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.670+0000 7f4b290d9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b24107910 0x7f4b2419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:38.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.670+0000 7f4b290d9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2419bbe0 con 0x7f4b24107910 2024-03-15T12:48:38.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.670+0000 7f4b22d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b24107910 0x7f4b2419b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:38.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.670+0000 7f4b22d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b24107910 0x7f4b2419b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44370/0 (socket says 172.21.15.2:44370) 2024-03-15T12:48:38.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.670+0000 7f4b22d9d700 1 -- 172.21.15.2:0/488468510 learned_addr learned my addr 172.21.15.2:0/488468510 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:38.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.670+0000 7f4b22d9d700 1 -- 172.21.15.2:0/488468510 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b14009550 con 0x7f4b24107910 2024-03-15T12:48:38.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.670+0000 7f4b22d9d700 1 --2- 172.21.15.2:0/488468510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b24107910 0x7f4b2419b6a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4b14006af0 tx=0x7f4b14011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:38.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.670+0000 7f4b0bfff700 1 -- 172.21.15.2:0/488468510 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b14003930 con 0x7f4b24107910 2024-03-15T12:48:38.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.670+0000 7f4b290d9700 1 -- 172.21.15.2:0/488468510 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2419bde0 con 0x7f4b24107910 2024-03-15T12:48:38.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.670+0000 7f4b0bfff700 1 -- 172.21.15.2:0/488468510 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b14003a90 con 0x7f4b24107910 2024-03-15T12:48:38.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.670+0000 7f4b0bfff700 1 -- 172.21.15.2:0/488468510 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b14019670 con 0x7f4b24107910 2024-03-15T12:48:38.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.670+0000 7f4b290d9700 1 -- 172.21.15.2:0/488468510 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2419c280 con 0x7f4b24107910 2024-03-15T12:48:38.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.671+0000 7f4b0bfff700 1 -- 172.21.15.2:0/488468510 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 2) v1 ==== 45525+0+0 (secure 0 0 0) 0x7f4b14019930 con 0x7f4b24107910 2024-03-15T12:48:38.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.671+0000 7f4b290d9700 1 -- 172.21.15.2:0/488468510 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b2410bcf0 con 0x7f4b24107910 2024-03-15T12:48:38.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.671+0000 7f4b0bfff700 1 -- 172.21.15.2:0/488468510 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4b1404ca90 con 0x7f4b24107910 2024-03-15T12:48:38.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.673+0000 7f4b0bfff700 1 -- 172.21.15.2:0/488468510 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4b1402d960 con 0x7f4b24107910 2024-03-15T12:48:38.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.746+0000 7f4b290d9700 1 -- 172.21.15.2:0/488468510 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4b2419f050 con 0x7f4b24107910 2024-03-15T12:48:38.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.746+0000 7f4b0bfff700 1 -- 172.21.15.2:0/488468510 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f4b14041e60 con 0x7f4b24107910 2024-03-15T12:48:38.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.747+0000 7f4b290d9700 1 -- 172.21.15.2:0/488468510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b24107910 msgr2=0x7f4b2419b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:38.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.747+0000 7f4b290d9700 1 --2- 172.21.15.2:0/488468510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b24107910 0x7f4b2419b6a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4b14006af0 tx=0x7f4b14011b70 comp rx=0 tx=0).stop 2024-03-15T12:48:38.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.747+0000 7f4b290d9700 1 -- 172.21.15.2:0/488468510 shutdown_connections 2024-03-15T12:48:38.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.747+0000 7f4b290d9700 1 --2- 172.21.15.2:0/488468510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b24107910 0x7f4b2419b6a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:38.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.747+0000 7f4b290d9700 1 -- 172.21.15.2:0/488468510 >> 172.21.15.2:0/488468510 conn(0x7f4b24102f50 msgr2=0x7f4b24103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:38.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.747+0000 7f4b290d9700 1 -- 172.21.15.2:0/488468510 shutdown_connections 2024-03-15T12:48:38.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:37.747+0000 7f4b290d9700 1 -- 172.21.15.2:0/488468510 wait complete. 2024-03-15T12:48:38.348 INFO:teuthology.orchestra.run.smithi002.stdout:mgr not available, waiting (2/15)... 2024-03-15T12:48:38.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:38 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/488468510' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-15T12:48:38.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:38 smithi002 ceph-mon[32060]: mgrmap e3: x(active, since 1.00737s) 2024-03-15T12:48:40.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:40 smithi002 ceph-mon[32060]: mgrmap e4: x(active, since 2s) 2024-03-15T12:48:42.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:42 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2218290318' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-15T12:48:42.955 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-15T12:48:42.955 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-03-15T12:48:42.955 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "2767718a-e2ca-11ee-95c9-87774f69a715", 2024-03-15T12:48:42.955 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-03-15T12:48:42.955 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-15T12:48:42.955 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-15T12:48:42.955 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-15T12:48:42.955 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:42.955 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-15T12:48:42.956 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-15T12:48:42.956 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-03-15T12:48:42.956 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-15T12:48:42.956 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-15T12:48:42.956 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2024-03-15T12:48:42.956 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-15T12:48:42.956 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-03-15T12:48:42.956 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-15T12:48:42.956 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-15T12:48:42.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-15T12:48:42.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-15T12:48:42.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:42.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-15T12:48:42.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-15T12:48:42.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-15T12:48:42.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-15T12:48:42.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-15T12:48:42.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-15T12:48:42.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-15T12:48:42.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-15T12:48:42.957 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-15T12:48:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-15T12:48:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-15T12:48:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-15T12:48:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-15T12:48:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-15T12:48:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-15T12:48:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-15T12:48:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-15T12:48:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-15T12:48:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-15T12:48:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-15T12:48:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-15T12:48:42.959 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:42.959 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-15T12:48:42.959 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-15T12:48:42.959 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-15T12:48:42.959 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-15T12:48:42.959 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-15T12:48:42.959 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-15T12:48:42.959 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-03-15T12:48:42.959 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-15T12:48:42.960 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-15T12:48:42.960 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:42.960 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-15T12:48:42.960 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-15T12:48:42.960 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-03-15T12:48:20.096386+0000", 2024-03-15T12:48:42.960 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-15T12:48:42.960 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-15T12:48:42.960 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-15T12:48:42.960 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-03-15T12:48:42.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.115+0000 7f34b95bf700 1 Processor -- start 2024-03-15T12:48:42.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.115+0000 7f34b95bf700 1 -- start start 2024-03-15T12:48:42.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.115+0000 7f34b95bf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34b41083b0 0x7f34b410a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:42.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.115+0000 7f34b95bf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b407ae60 con 0x7f34b41083b0 2024-03-15T12:48:42.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.115+0000 7f34b2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34b41083b0 0x7f34b410a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:42.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.115+0000 7f34b2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34b41083b0 0x7f34b410a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56030/0 (socket says 172.21.15.2:56030) 2024-03-15T12:48:42.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.115+0000 7f34b2ffd700 1 -- 172.21.15.2:0/3613674000 learned_addr learned my addr 172.21.15.2:0/3613674000 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:42.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.116+0000 7f34b2ffd700 1 -- 172.21.15.2:0/3613674000 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b410ace0 con 0x7f34b41083b0 2024-03-15T12:48:42.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.116+0000 7f34b2ffd700 1 --2- 172.21.15.2:0/3613674000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34b41083b0 0x7f34b410a7a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f34a40098a0 tx=0x7f34a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ce51a7976dae81 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:42.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.116+0000 7f34b27fc700 1 -- 172.21.15.2:0/3613674000 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34a4004160 con 0x7f34b41083b0 2024-03-15T12:48:42.961 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.116+0000 7f34b27fc700 1 -- 172.21.15.2:0/3613674000 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34a40042c0 con 0x7f34b41083b0 2024-03-15T12:48:42.962 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.116+0000 7f34b27fc700 1 -- 172.21.15.2:0/3613674000 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34a4011ab0 con 0x7f34b41083b0 2024-03-15T12:48:42.962 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.117+0000 7f34b95bf700 1 -- 172.21.15.2:0/3613674000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34b41083b0 msgr2=0x7f34b410a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:42.962 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.117+0000 7f34b95bf700 1 --2- 172.21.15.2:0/3613674000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34b41083b0 0x7f34b410a7a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f34a40098a0 tx=0x7f34a4009bb0 comp rx=0 tx=0).stop 2024-03-15T12:48:42.962 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.117+0000 7f34b95bf700 1 -- 172.21.15.2:0/3613674000 shutdown_connections 2024-03-15T12:48:42.962 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.117+0000 7f34b95bf700 1 --2- 172.21.15.2:0/3613674000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34b41083b0 0x7f34b410a7a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:42.962 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.117+0000 7f34b95bf700 1 -- 172.21.15.2:0/3613674000 >> 172.21.15.2:0/3613674000 conn(0x7f34b4101e00 msgr2=0x7f34b4104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:42.962 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.117+0000 7f34b95bf700 1 -- 172.21.15.2:0/3613674000 shutdown_connections 2024-03-15T12:48:42.962 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.117+0000 7f34b95bf700 1 -- 172.21.15.2:0/3613674000 wait complete. 2024-03-15T12:48:42.962 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.117+0000 7f34b95bf700 1 Processor -- start 2024-03-15T12:48:42.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.117+0000 7f34b95bf700 1 -- start start 2024-03-15T12:48:42.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.118+0000 7f34b95bf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34b41083b0 0x7f34b4107c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:42.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.118+0000 7f34b95bf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b4108170 con 0x7f34b41083b0 2024-03-15T12:48:42.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.118+0000 7f34b2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34b41083b0 0x7f34b4107c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:42.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.118+0000 7f34b2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34b41083b0 0x7f34b4107c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56032/0 (socket says 172.21.15.2:56032) 2024-03-15T12:48:42.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.118+0000 7f34b2ffd700 1 -- 172.21.15.2:0/2218290318 learned_addr learned my addr 172.21.15.2:0/2218290318 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:42.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.118+0000 7f34b2ffd700 1 -- 172.21.15.2:0/2218290318 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a4009550 con 0x7f34b41083b0 2024-03-15T12:48:42.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.118+0000 7f34b2ffd700 1 --2- 172.21.15.2:0/2218290318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34b41083b0 0x7f34b4107c30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f34a4006af0 tx=0x7f34a4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:42.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.118+0000 7f34b0ff9700 1 -- 172.21.15.2:0/2218290318 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34a4003930 con 0x7f34b41083b0 2024-03-15T12:48:42.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.118+0000 7f34b95bf700 1 -- 172.21.15.2:0/2218290318 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b4106360 con 0x7f34b41083b0 2024-03-15T12:48:42.963 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.118+0000 7f34b0ff9700 1 -- 172.21.15.2:0/2218290318 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34a4003a90 con 0x7f34b41083b0 2024-03-15T12:48:42.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.118+0000 7f34b0ff9700 1 -- 172.21.15.2:0/2218290318 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34a4019670 con 0x7f34b41083b0 2024-03-15T12:48:42.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.118+0000 7f34b95bf700 1 -- 172.21.15.2:0/2218290318 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b4106800 con 0x7f34b41083b0 2024-03-15T12:48:42.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.119+0000 7f34b0ff9700 1 -- 172.21.15.2:0/2218290318 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7f34a401a430 con 0x7f34b41083b0 2024-03-15T12:48:42.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.119+0000 7f34b0ff9700 1 --2- 172.21.15.2:0/2218290318 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f34a0038c10 0x7f34a003b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:42.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.119+0000 7f34b95bf700 1 -- 172.21.15.2:0/2218290318 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b410bae0 con 0x7f34b41083b0 2024-03-15T12:48:42.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.119+0000 7f34b0ff9700 1 -- 172.21.15.2:0/2218290318 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f34a404ce40 con 0x7f34b41083b0 2024-03-15T12:48:42.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.120+0000 7f34aa7fc700 1 --2- 172.21.15.2:0/2218290318 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f34a0038c10 0x7f34a003b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:42.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.120+0000 7f34aa7fc700 1 --2- 172.21.15.2:0/2218290318 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f34a0038c10 0x7f34a003b0d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f349c006fd0 tx=0x7f349c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:42.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.125+0000 7f34b0ff9700 1 -- 172.21.15.2:0/2218290318 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f34a4018350 con 0x7f34b41083b0 2024-03-15T12:48:42.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.413+0000 7f34b95bf700 1 -- 172.21.15.2:0/2218290318 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f34b4106e00 con 0x7f34b41083b0 2024-03-15T12:48:42.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.414+0000 7f34b0ff9700 1 -- 172.21.15.2:0/2218290318 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f34a4023c20 con 0x7f34b41083b0 2024-03-15T12:48:42.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.416+0000 7f34b95bf700 1 -- 172.21.15.2:0/2218290318 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f34a0038c10 msgr2=0x7f34a003b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:42.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.416+0000 7f34b95bf700 1 --2- 172.21.15.2:0/2218290318 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f34a0038c10 0x7f34a003b0d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f349c006fd0 tx=0x7f349c006e40 comp rx=0 tx=0).stop 2024-03-15T12:48:42.964 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.416+0000 7f34b95bf700 1 -- 172.21.15.2:0/2218290318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34b41083b0 msgr2=0x7f34b4107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:42.965 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.416+0000 7f34b95bf700 1 --2- 172.21.15.2:0/2218290318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34b41083b0 0x7f34b4107c30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f34a4006af0 tx=0x7f34a4011b70 comp rx=0 tx=0).stop 2024-03-15T12:48:42.965 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.416+0000 7f34b95bf700 1 -- 172.21.15.2:0/2218290318 shutdown_connections 2024-03-15T12:48:42.965 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.416+0000 7f34b95bf700 1 --2- 172.21.15.2:0/2218290318 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f34a0038c10 0x7f34a003b0d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:42.965 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.416+0000 7f34b95bf700 1 --2- 172.21.15.2:0/2218290318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34b41083b0 0x7f34b4107c30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:42.965 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.416+0000 7f34b95bf700 1 -- 172.21.15.2:0/2218290318 >> 172.21.15.2:0/2218290318 conn(0x7f34b4101e00 msgr2=0x7f34b41028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:42.965 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.416+0000 7f34b95bf700 1 -- 172.21.15.2:0/2218290318 shutdown_connections 2024-03-15T12:48:42.965 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:42.416+0000 7f34b95bf700 1 -- 172.21.15.2:0/2218290318 wait complete. 2024-03-15T12:48:42.965 INFO:teuthology.orchestra.run.smithi002.stdout:mgr is available 2024-03-15T12:48:43.862 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:43 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/405366189' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-15T12:48:44.098 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-15T12:48:44.098 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [global] 2024-03-15T12:48:44.098 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout fsid = 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:48:44.098 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-15T12:48:44.098 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-15T12:48:44.098 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-15T12:48:44.098 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-15T12:48:44.098 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-15T12:48:44.098 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-15T12:48:44.098 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-15T12:48:44.098 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-15T12:48:44.099 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [osd] 2024-03-15T12:48:44.099 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-15T12:48:44.099 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-15T12:48:44.099 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-15T12:48:44.099 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-15T12:48:44.099 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-15T12:48:44.099 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.373+0000 7fd1e713f700 1 Processor -- start 2024-03-15T12:48:44.099 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.373+0000 7fd1e713f700 1 -- start start 2024-03-15T12:48:44.099 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.374+0000 7fd1e713f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1e0105bd0 0x7fd1e0105ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:44.099 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.374+0000 7fd1e713f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e01065c0 con 0x7fd1e0105bd0 2024-03-15T12:48:44.099 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.374+0000 7fd1e4edb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1e0105bd0 0x7fd1e0105ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:44.099 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.374+0000 7fd1e4edb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1e0105bd0 0x7fd1e0105ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56034/0 (socket says 172.21.15.2:56034) 2024-03-15T12:48:44.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.374+0000 7fd1e4edb700 1 -- 172.21.15.2:0/2291464812 learned_addr learned my addr 172.21.15.2:0/2291464812 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:44.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.374+0000 7fd1e4edb700 1 -- 172.21.15.2:0/2291464812 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e0108b60 con 0x7fd1e0105bd0 2024-03-15T12:48:44.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.374+0000 7fd1e4edb700 1 --2- 172.21.15.2:0/2291464812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1e0105bd0 0x7fd1e0105ff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40098a0 tx=0x7fd1d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0ca49427e304af server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:44.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.374+0000 7fd1df7fe700 1 -- 172.21.15.2:0/2291464812 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1d4004160 con 0x7fd1e0105bd0 2024-03-15T12:48:44.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.375+0000 7fd1df7fe700 1 -- 172.21.15.2:0/2291464812 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1d40042c0 con 0x7fd1e0105bd0 2024-03-15T12:48:44.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.375+0000 7fd1df7fe700 1 -- 172.21.15.2:0/2291464812 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1d4011ab0 con 0x7fd1e0105bd0 2024-03-15T12:48:44.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.375+0000 7fd1e713f700 1 -- 172.21.15.2:0/2291464812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1e0105bd0 msgr2=0x7fd1e0105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:44.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.375+0000 7fd1e713f700 1 --2- 172.21.15.2:0/2291464812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1e0105bd0 0x7fd1e0105ff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40098a0 tx=0x7fd1d4009bb0 comp rx=0 tx=0).stop 2024-03-15T12:48:44.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.375+0000 7fd1e713f700 1 -- 172.21.15.2:0/2291464812 shutdown_connections 2024-03-15T12:48:44.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.375+0000 7fd1e713f700 1 --2- 172.21.15.2:0/2291464812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1e0105bd0 0x7fd1e0105ff0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:44.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.375+0000 7fd1e713f700 1 -- 172.21.15.2:0/2291464812 >> 172.21.15.2:0/2291464812 conn(0x7fd1e0101560 msgr2=0x7fd1e0103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:44.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.375+0000 7fd1e713f700 1 -- 172.21.15.2:0/2291464812 shutdown_connections 2024-03-15T12:48:44.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.375+0000 7fd1e713f700 1 -- 172.21.15.2:0/2291464812 wait complete. 2024-03-15T12:48:44.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.376+0000 7fd1e713f700 1 Processor -- start 2024-03-15T12:48:44.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.376+0000 7fd1e713f700 1 -- start start 2024-03-15T12:48:44.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.376+0000 7fd1e713f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1e0105bd0 0x7fd1e019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:44.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.376+0000 7fd1e713f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e019bc80 con 0x7fd1e0105bd0 2024-03-15T12:48:44.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.376+0000 7fd1e4edb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1e0105bd0 0x7fd1e019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:44.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.376+0000 7fd1e4edb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1e0105bd0 0x7fd1e019b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56046/0 (socket says 172.21.15.2:56046) 2024-03-15T12:48:44.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.376+0000 7fd1e4edb700 1 -- 172.21.15.2:0/405366189 learned_addr learned my addr 172.21.15.2:0/405366189 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:44.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.376+0000 7fd1e4edb700 1 -- 172.21.15.2:0/405366189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d4009550 con 0x7fd1e0105bd0 2024-03-15T12:48:44.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.376+0000 7fd1e4edb700 1 --2- 172.21.15.2:0/405366189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1e0105bd0 0x7fd1e019b740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4011670 tx=0x7fd1d4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:44.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.377+0000 7fd1ddffb700 1 -- 172.21.15.2:0/405366189 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1d40039f0 con 0x7fd1e0105bd0 2024-03-15T12:48:44.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.377+0000 7fd1ddffb700 1 -- 172.21.15.2:0/405366189 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1d4003b50 con 0x7fd1e0105bd0 2024-03-15T12:48:44.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.377+0000 7fd1e713f700 1 -- 172.21.15.2:0/405366189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e019be80 con 0x7fd1e0105bd0 2024-03-15T12:48:44.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.377+0000 7fd1ddffb700 1 -- 172.21.15.2:0/405366189 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1d40197f0 con 0x7fd1e0105bd0 2024-03-15T12:48:44.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.377+0000 7fd1e713f700 1 -- 172.21.15.2:0/405366189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e019c320 con 0x7fd1e0105bd0 2024-03-15T12:48:44.102 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.378+0000 7fd1ddffb700 1 -- 172.21.15.2:0/405366189 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7fd1d4019a60 con 0x7fd1e0105bd0 2024-03-15T12:48:44.102 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.378+0000 7fd1e713f700 1 -- 172.21.15.2:0/405366189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e010bd90 con 0x7fd1e0105bd0 2024-03-15T12:48:44.102 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.378+0000 7fd1ddffb700 1 --2- 172.21.15.2:0/405366189 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fd1d0038c10 0x7fd1d003b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:44.102 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.378+0000 7fd1ddffb700 1 -- 172.21.15.2:0/405366189 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd1d404cfb0 con 0x7fd1e0105bd0 2024-03-15T12:48:44.102 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.378+0000 7fd1dffff700 1 --2- 172.21.15.2:0/405366189 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fd1d0038c10 0x7fd1d003b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:44.102 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.378+0000 7fd1dffff700 1 --2- 172.21.15.2:0/405366189 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fd1d0038c10 0x7fd1d003b0d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc006fd0 tx=0x7fd1cc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:44.102 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.383+0000 7fd1ddffb700 1 -- 172.21.15.2:0/405366189 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd1d402dcd0 con 0x7fd1e0105bd0 2024-03-15T12:48:44.102 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.586+0000 7fd1e713f700 1 -- 172.21.15.2:0/405366189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd1e010bfa0 con 0x7fd1e0105bd0 2024-03-15T12:48:44.102 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.587+0000 7fd1ddffb700 1 -- 172.21.15.2:0/405366189 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fd1d4023be0 con 0x7fd1e0105bd0 2024-03-15T12:48:44.102 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.590+0000 7fd1e713f700 1 -- 172.21.15.2:0/405366189 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fd1d0038c10 msgr2=0x7fd1d003b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:44.102 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.590+0000 7fd1e713f700 1 --2- 172.21.15.2:0/405366189 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fd1d0038c10 0x7fd1d003b0d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc006fd0 tx=0x7fd1cc006e40 comp rx=0 tx=0).stop 2024-03-15T12:48:44.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.590+0000 7fd1e713f700 1 -- 172.21.15.2:0/405366189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1e0105bd0 msgr2=0x7fd1e019b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:44.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.590+0000 7fd1e713f700 1 --2- 172.21.15.2:0/405366189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1e0105bd0 0x7fd1e019b740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4011670 tx=0x7fd1d4011b70 comp rx=0 tx=0).stop 2024-03-15T12:48:44.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.590+0000 7fd1e713f700 1 -- 172.21.15.2:0/405366189 shutdown_connections 2024-03-15T12:48:44.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.590+0000 7fd1e713f700 1 --2- 172.21.15.2:0/405366189 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fd1d0038c10 0x7fd1d003b0d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:44.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.590+0000 7fd1e713f700 1 --2- 172.21.15.2:0/405366189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1e0105bd0 0x7fd1e019b740 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:44.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.590+0000 7fd1e713f700 1 -- 172.21.15.2:0/405366189 >> 172.21.15.2:0/405366189 conn(0x7fd1e0101560 msgr2=0x7fd1e0102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:44.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.590+0000 7fd1e713f700 1 -- 172.21.15.2:0/405366189 shutdown_connections 2024-03-15T12:48:44.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:43.590+0000 7fd1e713f700 1 -- 172.21.15.2:0/405366189 wait complete. 2024-03-15T12:48:44.103 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling cephadm module... 2024-03-15T12:48:45.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:44 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1095628659' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-15T12:48:45.054 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:44 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: ignoring --setuser ceph since I am not root 2024-03-15T12:48:45.055 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:44 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: ignoring --setgroup ceph since I am not root 2024-03-15T12:48:45.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.472+0000 7f83b9c25700 1 Processor -- start 2024-03-15T12:48:45.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.472+0000 7f83b9c25700 1 -- start start 2024-03-15T12:48:45.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.472+0000 7f83b9c25700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b41078e0 0x7f83b4107d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:45.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.472+0000 7f83b9c25700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b41082d0 con 0x7f83b41078e0 2024-03-15T12:48:45.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.472+0000 7f83b37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b41078e0 0x7f83b4107d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:45.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.472+0000 7f83b37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b41078e0 0x7f83b4107d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56056/0 (socket says 172.21.15.2:56056) 2024-03-15T12:48:45.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.472+0000 7f83b37fe700 1 -- 172.21.15.2:0/1750634130 learned_addr learned my addr 172.21.15.2:0/1750634130 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:45.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.472+0000 7f83b37fe700 1 -- 172.21.15.2:0/1750634130 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b4108a90 con 0x7f83b41078e0 2024-03-15T12:48:45.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.473+0000 7f83b37fe700 1 --2- 172.21.15.2:0/1750634130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b41078e0 0x7f83b4107d00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f83a40098a0 tx=0x7f83a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9c83cd92b1e81e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:45.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.473+0000 7f83b2ffd700 1 -- 172.21.15.2:0/1750634130 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83a4004160 con 0x7f83b41078e0 2024-03-15T12:48:45.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.473+0000 7f83b2ffd700 1 -- 172.21.15.2:0/1750634130 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a40042c0 con 0x7f83b41078e0 2024-03-15T12:48:45.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.473+0000 7f83b2ffd700 1 -- 172.21.15.2:0/1750634130 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83a4011ab0 con 0x7f83b41078e0 2024-03-15T12:48:45.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.473+0000 7f83b9c25700 1 -- 172.21.15.2:0/1750634130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b41078e0 msgr2=0x7f83b4107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:45.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.473+0000 7f83b9c25700 1 --2- 172.21.15.2:0/1750634130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b41078e0 0x7f83b4107d00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f83a40098a0 tx=0x7f83a4009bb0 comp rx=0 tx=0).stop 2024-03-15T12:48:45.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.473+0000 7f83b9c25700 1 -- 172.21.15.2:0/1750634130 shutdown_connections 2024-03-15T12:48:45.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.473+0000 7f83b9c25700 1 --2- 172.21.15.2:0/1750634130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b41078e0 0x7f83b4107d00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:45.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.473+0000 7f83b9c25700 1 -- 172.21.15.2:0/1750634130 >> 172.21.15.2:0/1750634130 conn(0x7f83b4102f00 msgr2=0x7f83b4105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:45.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.473+0000 7f83b9c25700 1 -- 172.21.15.2:0/1750634130 shutdown_connections 2024-03-15T12:48:45.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.473+0000 7f83b9c25700 1 -- 172.21.15.2:0/1750634130 wait complete. 2024-03-15T12:48:45.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.474+0000 7f83b9c25700 1 Processor -- start 2024-03-15T12:48:45.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.474+0000 7f83b9c25700 1 -- start start 2024-03-15T12:48:45.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.474+0000 7f83b9c25700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b41078e0 0x7f83b419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:45.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.474+0000 7f83b9c25700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b419bbe0 con 0x7f83b41078e0 2024-03-15T12:48:45.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.474+0000 7f83b37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b41078e0 0x7f83b419b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:45.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.474+0000 7f83b37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b41078e0 0x7f83b419b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56068/0 (socket says 172.21.15.2:56068) 2024-03-15T12:48:45.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.474+0000 7f83b37fe700 1 -- 172.21.15.2:0/1095628659 learned_addr learned my addr 172.21.15.2:0/1095628659 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:45.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.475+0000 7f83b37fe700 1 -- 172.21.15.2:0/1095628659 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a4009550 con 0x7f83b41078e0 2024-03-15T12:48:45.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.475+0000 7f83b37fe700 1 --2- 172.21.15.2:0/1095628659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b41078e0 0x7f83b419b6a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f83a4006af0 tx=0x7f83a4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:45.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.475+0000 7f83b17fa700 1 -- 172.21.15.2:0/1095628659 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83a4003930 con 0x7f83b41078e0 2024-03-15T12:48:45.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.475+0000 7f83b17fa700 1 -- 172.21.15.2:0/1095628659 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a4003a90 con 0x7f83b41078e0 2024-03-15T12:48:45.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.475+0000 7f83b9c25700 1 -- 172.21.15.2:0/1095628659 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b419bde0 con 0x7f83b41078e0 2024-03-15T12:48:45.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.475+0000 7f83b17fa700 1 -- 172.21.15.2:0/1095628659 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83a4019670 con 0x7f83b41078e0 2024-03-15T12:48:45.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.475+0000 7f83b9c25700 1 -- 172.21.15.2:0/1095628659 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b419c280 con 0x7f83b41078e0 2024-03-15T12:48:45.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.476+0000 7f83b17fa700 1 -- 172.21.15.2:0/1095628659 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7f83a4019930 con 0x7f83b41078e0 2024-03-15T12:48:45.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.476+0000 7f83b17fa700 1 --2- 172.21.15.2:0/1095628659 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f83a0038c10 0x7f83a003b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:45.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.476+0000 7f83b17fa700 1 -- 172.21.15.2:0/1095628659 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f83a404ce60 con 0x7f83b41078e0 2024-03-15T12:48:45.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.476+0000 7f83b9c25700 1 -- 172.21.15.2:0/1095628659 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b410bcc0 con 0x7f83b41078e0 2024-03-15T12:48:45.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.476+0000 7f83aaffd700 1 --2- 172.21.15.2:0/1095628659 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f83a0038c10 0x7f83a003b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:45.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.476+0000 7f83aaffd700 1 --2- 172.21.15.2:0/1095628659 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f83a0038c10 0x7f83a003b0d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f839c006fd0 tx=0x7f839c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:45.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.481+0000 7f83b17fa700 1 -- 172.21.15.2:0/1095628659 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f83a4019be0 con 0x7f83b41078e0 2024-03-15T12:48:45.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.741+0000 7f83b9c25700 1 -- 172.21.15.2:0/1095628659 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f83b419eb60 con 0x7f83b41078e0 2024-03-15T12:48:45.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.793+0000 7f83b17fa700 1 -- 172.21.15.2:0/1095628659 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f83a4023390 con 0x7f83b41078e0 2024-03-15T12:48:45.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.794+0000 7f83b17fa700 1 -- 172.21.15.2:0/1095628659 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f83a402d9b0 con 0x7f83b41078e0 2024-03-15T12:48:45.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.796+0000 7f83b9c25700 1 -- 172.21.15.2:0/1095628659 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f83a0038c10 msgr2=0x7f83a003b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:45.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.796+0000 7f83b9c25700 1 --2- 172.21.15.2:0/1095628659 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f83a0038c10 0x7f83a003b0d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f839c006fd0 tx=0x7f839c006e40 comp rx=0 tx=0).stop 2024-03-15T12:48:45.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.796+0000 7f83b9c25700 1 -- 172.21.15.2:0/1095628659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b41078e0 msgr2=0x7f83b419b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:45.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.796+0000 7f83b9c25700 1 --2- 172.21.15.2:0/1095628659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b41078e0 0x7f83b419b6a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f83a4006af0 tx=0x7f83a4011b70 comp rx=0 tx=0).stop 2024-03-15T12:48:45.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.796+0000 7f83b9c25700 1 -- 172.21.15.2:0/1095628659 shutdown_connections 2024-03-15T12:48:45.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.796+0000 7f83b9c25700 1 --2- 172.21.15.2:0/1095628659 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f83a0038c10 0x7f83a003b0d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:45.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.796+0000 7f83b9c25700 1 --2- 172.21.15.2:0/1095628659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83b41078e0 0x7f83b419b6a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:45.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.796+0000 7f83b9c25700 1 -- 172.21.15.2:0/1095628659 >> 172.21.15.2:0/1095628659 conn(0x7f83b4102f00 msgr2=0x7f83b4103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:45.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.796+0000 7f83b9c25700 1 -- 172.21.15.2:0/1095628659 shutdown_connections 2024-03-15T12:48:45.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:44.796+0000 7f83b9c25700 1 -- 172.21.15.2:0/1095628659 wait complete. 2024-03-15T12:48:45.454 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:45 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:45.363+0000 7fd0ce8c1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-15T12:48:45.455 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:45 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:45.467+0000 7fd0ce8c1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-15T12:48:46.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:45 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1095628659' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-15T12:48:46.019 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:45 smithi002 ceph-mon[32060]: mgrmap e5: x(active, since 7s) 2024-03-15T12:48:46.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:46 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:46.134+0000 7fd0ce8c1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-15T12:48:46.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:46 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:46.230+0000 7fd0ce8c1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-15T12:48:46.650 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:46 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:46.374+0000 7fd0ce8c1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-15T12:48:46.650 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:46 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:46.472+0000 7fd0ce8c1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-15T12:48:46.650 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:46 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:46.558+0000 7fd0ce8c1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-15T12:48:46.658 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-03-15T12:48:46.658 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-15T12:48:46.658 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-15T12:48:46.658 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-15T12:48:46.658 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-15T12:48:46.659 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-03-15T12:48:46.659 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.826+0000 7f7da1aba700 1 Processor -- start 2024-03-15T12:48:46.659 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.826+0000 7f7da1aba700 1 -- start start 2024-03-15T12:48:46.659 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.826+0000 7f7da1aba700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1067b0 0x7f7d9c106bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:46.659 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.826+0000 7f7da1aba700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c1071a0 con 0x7f7d9c1067b0 2024-03-15T12:48:46.659 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.826+0000 7f7d9b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1067b0 0x7f7d9c106bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:46.659 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.826+0000 7f7d9b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1067b0 0x7f7d9c106bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56100/0 (socket says 172.21.15.2:56100) 2024-03-15T12:48:46.659 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.826+0000 7f7d9b7fe700 1 -- 172.21.15.2:0/2039984976 learned_addr learned my addr 172.21.15.2:0/2039984976 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:46.659 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.827+0000 7f7d9b7fe700 1 -- 172.21.15.2:0/2039984976 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c1079b0 con 0x7f7d9c1067b0 2024-03-15T12:48:46.659 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.827+0000 7f7d9b7fe700 1 --2- 172.21.15.2:0/2039984976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1067b0 0x7f7d9c106bd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7d900098a0 tx=0x7f7d90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc204ecc69ad415e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:46.659 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.827+0000 7f7d9a7fc700 1 -- 172.21.15.2:0/2039984976 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d90004160 con 0x7f7d9c1067b0 2024-03-15T12:48:46.659 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.827+0000 7f7d9a7fc700 1 -- 172.21.15.2:0/2039984976 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d900042c0 con 0x7f7d9c1067b0 2024-03-15T12:48:46.659 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.827+0000 7f7d9a7fc700 1 -- 172.21.15.2:0/2039984976 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d90011ab0 con 0x7f7d9c1067b0 2024-03-15T12:48:46.660 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.828+0000 7f7da1aba700 1 -- 172.21.15.2:0/2039984976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1067b0 msgr2=0x7f7d9c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:46.660 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.828+0000 7f7da1aba700 1 --2- 172.21.15.2:0/2039984976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1067b0 0x7f7d9c106bd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7d900098a0 tx=0x7f7d90009bb0 comp rx=0 tx=0).stop 2024-03-15T12:48:46.660 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.828+0000 7f7da1aba700 1 -- 172.21.15.2:0/2039984976 shutdown_connections 2024-03-15T12:48:46.660 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.828+0000 7f7da1aba700 1 --2- 172.21.15.2:0/2039984976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1067b0 0x7f7d9c106bd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:46.660 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.828+0000 7f7da1aba700 1 -- 172.21.15.2:0/2039984976 >> 172.21.15.2:0/2039984976 conn(0x7f7d9c101e10 msgr2=0x7f7d9c104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:46.660 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.828+0000 7f7da1aba700 1 -- 172.21.15.2:0/2039984976 shutdown_connections 2024-03-15T12:48:46.660 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.828+0000 7f7da1aba700 1 -- 172.21.15.2:0/2039984976 wait complete. 2024-03-15T12:48:46.660 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.828+0000 7f7da1aba700 1 Processor -- start 2024-03-15T12:48:46.660 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.829+0000 7f7da1aba700 1 -- start start 2024-03-15T12:48:46.660 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.829+0000 7f7da1aba700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1067b0 0x7f7d9c1918d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:46.660 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.829+0000 7f7da1aba700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c191e10 con 0x7f7d9c1067b0 2024-03-15T12:48:46.660 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.829+0000 7f7d9b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1067b0 0x7f7d9c1918d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:46.660 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.829+0000 7f7d9b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1067b0 0x7f7d9c1918d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56110/0 (socket says 172.21.15.2:56110) 2024-03-15T12:48:46.660 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.829+0000 7f7d9b7fe700 1 -- 172.21.15.2:0/1710881244 learned_addr learned my addr 172.21.15.2:0/1710881244 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:46.661 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.829+0000 7f7d9b7fe700 1 -- 172.21.15.2:0/1710881244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d90009550 con 0x7f7d9c1067b0 2024-03-15T12:48:46.661 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.829+0000 7f7d9b7fe700 1 --2- 172.21.15.2:0/1710881244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1067b0 0x7f7d9c1918d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7d90011670 tx=0x7f7d90011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:46.661 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.829+0000 7f7d98ff9700 1 -- 172.21.15.2:0/1710881244 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d900039f0 con 0x7f7d9c1067b0 2024-03-15T12:48:46.661 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.830+0000 7f7d98ff9700 1 -- 172.21.15.2:0/1710881244 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d90003b50 con 0x7f7d9c1067b0 2024-03-15T12:48:46.661 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.830+0000 7f7da1aba700 1 -- 172.21.15.2:0/1710881244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9c190050 con 0x7f7d9c1067b0 2024-03-15T12:48:46.661 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.830+0000 7f7d98ff9700 1 -- 172.21.15.2:0/1710881244 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d900197f0 con 0x7f7d9c1067b0 2024-03-15T12:48:46.661 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.830+0000 7f7da1aba700 1 -- 172.21.15.2:0/1710881244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9c1904f0 con 0x7f7d9c1067b0 2024-03-15T12:48:46.661 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.830+0000 7f7d98ff9700 1 -- 172.21.15.2:0/1710881244 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f7d90019a60 con 0x7f7d9c1067b0 2024-03-15T12:48:46.661 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.831+0000 7f7d98ff9700 1 --2- 172.21.15.2:0/1710881244 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f7d84038cb0 0x7f7d8403b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:46.661 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.831+0000 7f7d98ff9700 1 -- 172.21.15.2:0/1710881244 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7d9001a430 con 0x7f7d9c1067b0 2024-03-15T12:48:46.661 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.831+0000 7f7d9affd700 1 -- 172.21.15.2:0/1710881244 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f7d84038cb0 msgr2=0x7f7d8403b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3111414481 2024-03-15T12:48:46.661 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.831+0000 7f7d9affd700 1 --2- 172.21.15.2:0/1710881244 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f7d84038cb0 0x7f7d8403b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-15T12:48:46.661 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.831+0000 7f7da1aba700 1 -- 172.21.15.2:0/1710881244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9c063610 con 0x7f7d9c1067b0 2024-03-15T12:48:46.661 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:45.836+0000 7f7d98ff9700 1 -- 172.21.15.2:0/1710881244 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7d90054060 con 0x7f7d9c1067b0 2024-03-15T12:48:46.662 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:46.032+0000 7f7d9affd700 1 -- 172.21.15.2:0/1710881244 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f7d84038cb0 msgr2=0x7f7d8403b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3111414481 2024-03-15T12:48:46.662 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:46.032+0000 7f7d9affd700 1 --2- 172.21.15.2:0/1710881244 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f7d84038cb0 0x7f7d8403b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-15T12:48:46.662 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:46.098+0000 7f7da1aba700 1 -- 172.21.15.2:0/1710881244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f7d9c10abb0 con 0x7f7d9c1067b0 2024-03-15T12:48:46.662 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:46.098+0000 7f7d98ff9700 1 -- 172.21.15.2:0/1710881244 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f7d9002a2b0 con 0x7f7d9c1067b0 2024-03-15T12:48:46.662 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:46.100+0000 7f7da1aba700 1 -- 172.21.15.2:0/1710881244 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f7d84038cb0 msgr2=0x7f7d8403b170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-15T12:48:46.662 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:46.100+0000 7f7da1aba700 1 --2- 172.21.15.2:0/1710881244 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f7d84038cb0 0x7f7d8403b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:46.662 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:46.100+0000 7f7da1aba700 1 -- 172.21.15.2:0/1710881244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1067b0 msgr2=0x7f7d9c1918d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:46.662 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:46.100+0000 7f7da1aba700 1 --2- 172.21.15.2:0/1710881244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1067b0 0x7f7d9c1918d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7d90011670 tx=0x7f7d90011b70 comp rx=0 tx=0).stop 2024-03-15T12:48:46.665 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:46.100+0000 7f7da1aba700 1 -- 172.21.15.2:0/1710881244 shutdown_connections 2024-03-15T12:48:46.665 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:46.100+0000 7f7da1aba700 1 --2- 172.21.15.2:0/1710881244 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7f7d84038cb0 0x7f7d8403b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:46.665 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:46.100+0000 7f7da1aba700 1 --2- 172.21.15.2:0/1710881244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1067b0 0x7f7d9c1918d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:46.665 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:46.100+0000 7f7da1aba700 1 -- 172.21.15.2:0/1710881244 >> 172.21.15.2:0/1710881244 conn(0x7f7d9c101e10 msgr2=0x7f7d9c102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:46.665 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:46.101+0000 7f7da1aba700 1 -- 172.21.15.2:0/1710881244 shutdown_connections 2024-03-15T12:48:46.665 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:46.101+0000 7f7da1aba700 1 -- 172.21.15.2:0/1710881244 wait complete. 2024-03-15T12:48:46.665 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for the mgr to restart... 2024-03-15T12:48:46.665 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr epoch 5... 2024-03-15T12:48:46.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:46 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1710881244' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-15T12:48:46.934 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:46 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:46.831+0000 7fd0ce8c1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-15T12:48:47.301 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:46 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:46.946+0000 7fd0ce8c1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-15T12:48:47.301 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:47 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:47.035+0000 7fd0ce8c1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-15T12:48:47.301 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:47 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:47.136+0000 7fd0ce8c1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-15T12:48:47.554 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:47 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:47.315+0000 7fd0ce8c1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-15T12:48:48.554 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:48.300+0000 7fd0ce8c1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-15T12:48:48.554 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:48.433+0000 7fd0ce8c1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-15T12:48:49.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:48.771+0000 7fd0ce8c1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-15T12:48:49.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:48.863+0000 7fd0ce8c1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-15T12:48:49.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:49 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:49.063+0000 7fd0ce8c1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:48:49.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:49 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:49.253+0000 7fd0ce8c1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-15T12:48:49.792 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:49 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:49.542+0000 7fd0ce8c1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:48:50.054 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:49 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:49.805+0000 7fd0ce8c1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-15T12:48:50.054 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:50 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:50.047+0000 7fd0ce8c1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-15T12:48:51.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:50 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:50.872+0000 7fd0ce8c1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-15T12:48:51.764 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:51.455+0000 7fd0ce8c1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-15T12:48:51.764 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:51.555+0000 7fd0ce8c1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-15T12:48:51.764 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:48:51.776+0000 7fd0ce8c1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-15T12:48:52.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-mon[32060]: Active manager daemon x restarted 2024-03-15T12:48:52.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-mon[32060]: Activating manager daemon x 2024-03-15T12:48:52.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-mon[32060]: osdmap e2: 0 total, 0 up, 0 in 2024-03-15T12:48:52.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-mon[32060]: mgrmap e6: x(active, starting, since 0.00321937s) 2024-03-15T12:48:52.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:48:52.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-15T12:48:52.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-15T12:48:52.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-15T12:48:52.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-15T12:48:52.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-mon[32060]: Manager daemon x is now available 2024-03-15T12:48:52.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' 2024-03-15T12:48:52.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' 2024-03-15T12:48:52.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:48:52.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:51 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:48:53.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:52 smithi002 ceph-mon[32060]: Found migration_current of "None". Setting to last migration. 2024-03-15T12:48:53.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:52 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:48:53.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:52 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-15T12:48:53.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:52 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-15T12:48:53.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:52 smithi002 ceph-mon[32060]: mgrmap e7: x(active, since 1.00531s) 2024-03-15T12:48:53.284 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-03-15T12:48:53.291 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-15T12:48:53.291 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-15T12:48:53.291 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-03-15T12:48:53.291 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.108+0000 7fb5c366d700 1 Processor -- start 2024-03-15T12:48:53.291 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.109+0000 7fb5c366d700 1 -- start start 2024-03-15T12:48:53.291 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.109+0000 7fb5c366d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5bc1078e0 0x7fb5bc107d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:53.291 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.109+0000 7fb5c366d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc1082d0 con 0x7fb5bc1078e0 2024-03-15T12:48:53.291 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.109+0000 7fb5c1409700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5bc1078e0 0x7fb5bc107d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:53.291 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.109+0000 7fb5c1409700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5bc1078e0 0x7fb5bc107d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56114/0 (socket says 172.21.15.2:56114) 2024-03-15T12:48:53.291 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.109+0000 7fb5c1409700 1 -- 172.21.15.2:0/3702882840 learned_addr learned my addr 172.21.15.2:0/3702882840 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:53.291 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.109+0000 7fb5c1409700 1 -- 172.21.15.2:0/3702882840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc108a90 con 0x7fb5bc1078e0 2024-03-15T12:48:53.291 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.109+0000 7fb5c1409700 1 --2- 172.21.15.2:0/3702882840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5bc1078e0 0x7fb5bc107d00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00098a0 tx=0x7fb5b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd31d0ad6e328e08 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:53.291 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.109+0000 7fb5affff700 1 -- 172.21.15.2:0/3702882840 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5b0004160 con 0x7fb5bc1078e0 2024-03-15T12:48:53.291 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.110+0000 7fb5affff700 1 -- 172.21.15.2:0/3702882840 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5b00042c0 con 0x7fb5bc1078e0 2024-03-15T12:48:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.110+0000 7fb5affff700 1 -- 172.21.15.2:0/3702882840 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5b0011ab0 con 0x7fb5bc1078e0 2024-03-15T12:48:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.110+0000 7fb5c366d700 1 -- 172.21.15.2:0/3702882840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5bc1078e0 msgr2=0x7fb5bc107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.110+0000 7fb5c366d700 1 --2- 172.21.15.2:0/3702882840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5bc1078e0 0x7fb5bc107d00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00098a0 tx=0x7fb5b0009bb0 comp rx=0 tx=0).stop 2024-03-15T12:48:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.110+0000 7fb5c366d700 1 -- 172.21.15.2:0/3702882840 shutdown_connections 2024-03-15T12:48:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.110+0000 7fb5c366d700 1 --2- 172.21.15.2:0/3702882840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5bc1078e0 0x7fb5bc107d00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.110+0000 7fb5c366d700 1 -- 172.21.15.2:0/3702882840 >> 172.21.15.2:0/3702882840 conn(0x7fb5bc102f20 msgr2=0x7fb5bc105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.110+0000 7fb5c366d700 1 -- 172.21.15.2:0/3702882840 shutdown_connections 2024-03-15T12:48:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.110+0000 7fb5c366d700 1 -- 172.21.15.2:0/3702882840 wait complete. 2024-03-15T12:48:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.111+0000 7fb5c366d700 1 Processor -- start 2024-03-15T12:48:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.111+0000 7fb5c366d700 1 -- start start 2024-03-15T12:48:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.111+0000 7fb5c366d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5bc1078e0 0x7fb5bc19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.111+0000 7fb5c366d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc19ddb0 con 0x7fb5bc1078e0 2024-03-15T12:48:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.111+0000 7fb5c1409700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5bc1078e0 0x7fb5bc19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.111+0000 7fb5c1409700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5bc1078e0 0x7fb5bc19d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56118/0 (socket says 172.21.15.2:56118) 2024-03-15T12:48:53.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.111+0000 7fb5c1409700 1 -- 172.21.15.2:0/218707689 learned_addr learned my addr 172.21.15.2:0/218707689 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:53.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.111+0000 7fb5c1409700 1 -- 172.21.15.2:0/218707689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b0009550 con 0x7fb5bc1078e0 2024-03-15T12:48:53.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.112+0000 7fb5c1409700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5bc1078e0 0x7fb5bc19d870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0000c00 tx=0x7fb5b0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:53.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.112+0000 7fb5ae7fc700 1 -- 172.21.15.2:0/218707689 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5b0003a70 con 0x7fb5bc1078e0 2024-03-15T12:48:53.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.112+0000 7fb5ae7fc700 1 -- 172.21.15.2:0/218707689 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5b0003bd0 con 0x7fb5bc1078e0 2024-03-15T12:48:53.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.112+0000 7fb5c366d700 1 -- 172.21.15.2:0/218707689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5bc19dfb0 con 0x7fb5bc1078e0 2024-03-15T12:48:53.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.112+0000 7fb5ae7fc700 1 -- 172.21.15.2:0/218707689 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5b001a760 con 0x7fb5bc1078e0 2024-03-15T12:48:53.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.112+0000 7fb5c366d700 1 -- 172.21.15.2:0/218707689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5bc19e450 con 0x7fb5bc1078e0 2024-03-15T12:48:53.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.113+0000 7fb5ae7fc700 1 -- 172.21.15.2:0/218707689 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fb5b001a9d0 con 0x7fb5bc1078e0 2024-03-15T12:48:53.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.113+0000 7fb5ae7fc700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fb5a8038d00 0x7fb5a803b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:53.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.113+0000 7fb5c366d700 1 -- 172.21.15.2:0/218707689 --> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5bc10bcc0 con 0x7fb5a8038d00 2024-03-15T12:48:53.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.113+0000 7fb5ae7fc700 1 -- 172.21.15.2:0/218707689 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb5b004ccc0 con 0x7fb5bc1078e0 2024-03-15T12:48:53.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.113+0000 7fb5c0c08700 1 -- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fb5a8038d00 msgr2=0x7fb5a803b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3111414481 2024-03-15T12:48:53.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.113+0000 7fb5c0c08700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fb5a8038d00 0x7fb5a803b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-15T12:48:53.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.314+0000 7fb5c0c08700 1 -- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fb5a8038d00 msgr2=0x7fb5a803b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3111414481 2024-03-15T12:48:53.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.314+0000 7fb5c0c08700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fb5a8038d00 0x7fb5a803b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-15T12:48:53.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.715+0000 7fb5c0c08700 1 -- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fb5a8038d00 msgr2=0x7fb5a803b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3111414481 2024-03-15T12:48:53.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:47.715+0000 7fb5c0c08700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fb5a8038d00 0x7fb5a803b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-15T12:48:53.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:48.516+0000 7fb5c0c08700 1 -- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fb5a8038d00 msgr2=0x7fb5a803b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3111414481 2024-03-15T12:48:53.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:48.516+0000 7fb5c0c08700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fb5a8038d00 0x7fb5a803b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-15T12:48:53.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:50.117+0000 7fb5c0c08700 1 -- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fb5a8038d00 msgr2=0x7fb5a803b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3111414481 2024-03-15T12:48:53.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:50.117+0000 7fb5c0c08700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fb5a8038d00 0x7fb5a803b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-15T12:48:53.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:51.792+0000 7fb5ae7fc700 1 -- 172.21.15.2:0/218707689 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mgrmap(e 6) v1 ==== 45536+0+0 (secure 0 0 0) 0x7fb5b002d910 con 0x7fb5bc1078e0 2024-03-15T12:48:53.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:51.792+0000 7fb5ae7fc700 1 -- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fb5a8038d00 msgr2=0x7fb5a803b1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-15T12:48:53.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:51.792+0000 7fb5ae7fc700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fb5a8038d00 0x7fb5a803b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:53.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.793+0000 7fb5ae7fc700 1 -- 172.21.15.2:0/218707689 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 7) v1 ==== 45663+0+0 (secure 0 0 0) 0x7fb5b004e7d0 con 0x7fb5bc1078e0 2024-03-15T12:48:53.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.793+0000 7fb5ae7fc700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fb5a803c440 0x7fb5a803e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:53.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.793+0000 7fb5ae7fc700 1 -- 172.21.15.2:0/218707689 --> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5bc10bcc0 con 0x7fb5a803c440 2024-03-15T12:48:53.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.794+0000 7fb5c0c08700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fb5a803c440 0x7fb5a803e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:53.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.794+0000 7fb5c0c08700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fb5a803c440 0x7fb5a803e830 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8003a10 tx=0x7fb5b80092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.795+0000 7fb5ae7fc700 1 -- 172.21.15.2:0/218707689 <== mgr.14118 v2:172.21.15.2:6800/3269509343 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fb5bc10bcc0 con 0x7fb5a803c440 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.801+0000 7fb5c366d700 1 -- 172.21.15.2:0/218707689 --> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fb5bc10be30 con 0x7fb5a803c440 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.802+0000 7fb5ae7fc700 1 -- 172.21.15.2:0/218707689 <== mgr.14118 v2:172.21.15.2:6800/3269509343 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fb5bc10be30 con 0x7fb5a803c440 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.802+0000 7fb5c366d700 1 -- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fb5a803c440 msgr2=0x7fb5a803e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.802+0000 7fb5c366d700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fb5a803c440 0x7fb5a803e830 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8003a10 tx=0x7fb5b80092b0 comp rx=0 tx=0).stop 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.802+0000 7fb5c366d700 1 -- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5bc1078e0 msgr2=0x7fb5bc19d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.802+0000 7fb5c366d700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5bc1078e0 0x7fb5bc19d870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0000c00 tx=0x7fb5b0011b70 comp rx=0 tx=0).stop 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.802+0000 7fb5c366d700 1 -- 172.21.15.2:0/218707689 shutdown_connections 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.802+0000 7fb5c366d700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5bc1078e0 0x7fb5bc19d870 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.802+0000 7fb5c366d700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fb5a803c440 0x7fb5a803e830 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.802+0000 7fb5c366d700 1 --2- 172.21.15.2:0/218707689 >> [v2:172.21.15.2:6800/3111414481,v1:172.21.15.2:6801/3111414481] conn(0x7fb5a8038d00 0x7fb5a803b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.802+0000 7fb5c366d700 1 -- 172.21.15.2:0/218707689 >> 172.21.15.2:0/218707689 conn(0x7fb5bc102f20 msgr2=0x7fb5bc1039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.802+0000 7fb5c366d700 1 -- 172.21.15.2:0/218707689 shutdown_connections 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:52.802+0000 7fb5c366d700 1 -- 172.21.15.2:0/218707689 wait complete. 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:mgr epoch 5 is available 2024-03-15T12:48:53.295 INFO:teuthology.orchestra.run.smithi002.stdout:Setting orchestrator backend to cephadm... 2024-03-15T12:48:53.934 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:53 smithi002 ceph-mon[32060]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-15T12:48:53.934 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:53 smithi002 ceph-mon[32060]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-15T12:48:53.934 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:53 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' 2024-03-15T12:48:53.934 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:53 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' 2024-03-15T12:48:53.934 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:53 smithi002 ceph-mon[32060]: [15/Mar/2024:12:48:53] ENGINE Bus STARTING 2024-03-15T12:48:54.470 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.714+0000 7f034eebc700 1 Processor -- start 2024-03-15T12:48:54.471 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.714+0000 7f034eebc700 1 -- start start 2024-03-15T12:48:54.471 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.714+0000 7f034eebc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03481078e0 0x7f0348107d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:54.471 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.714+0000 7f034eebc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03481082d0 con 0x7f03481078e0 2024-03-15T12:48:54.471 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.714+0000 7f034cc58700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03481078e0 0x7f0348107d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:54.471 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.714+0000 7f034cc58700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03481078e0 0x7f0348107d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57620/0 (socket says 172.21.15.2:57620) 2024-03-15T12:48:54.471 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.714+0000 7f034cc58700 1 -- 172.21.15.2:0/4061195104 learned_addr learned my addr 172.21.15.2:0/4061195104 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:54.471 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.715+0000 7f034cc58700 1 -- 172.21.15.2:0/4061195104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0348108a90 con 0x7f03481078e0 2024-03-15T12:48:54.471 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.715+0000 7f034cc58700 1 --2- 172.21.15.2:0/4061195104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03481078e0 0x7f0348107d00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f033c0098a0 tx=0x7f033c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e74d0e794ad8925 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:54.471 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.715+0000 7f03477fe700 1 -- 172.21.15.2:0/4061195104 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f033c004160 con 0x7f03481078e0 2024-03-15T12:48:54.471 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.715+0000 7f03477fe700 1 -- 172.21.15.2:0/4061195104 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f033c0042c0 con 0x7f03481078e0 2024-03-15T12:48:54.471 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.715+0000 7f03477fe700 1 -- 172.21.15.2:0/4061195104 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f033c011ab0 con 0x7f03481078e0 2024-03-15T12:48:54.472 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.715+0000 7f034eebc700 1 -- 172.21.15.2:0/4061195104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03481078e0 msgr2=0x7f0348107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:54.472 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.715+0000 7f034eebc700 1 --2- 172.21.15.2:0/4061195104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03481078e0 0x7f0348107d00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f033c0098a0 tx=0x7f033c009bb0 comp rx=0 tx=0).stop 2024-03-15T12:48:54.472 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.716+0000 7f034eebc700 1 -- 172.21.15.2:0/4061195104 shutdown_connections 2024-03-15T12:48:54.472 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.716+0000 7f034eebc700 1 --2- 172.21.15.2:0/4061195104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03481078e0 0x7f0348107d00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:54.472 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.716+0000 7f034eebc700 1 -- 172.21.15.2:0/4061195104 >> 172.21.15.2:0/4061195104 conn(0x7f0348102f20 msgr2=0x7f0348105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:54.472 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.716+0000 7f034eebc700 1 -- 172.21.15.2:0/4061195104 shutdown_connections 2024-03-15T12:48:54.472 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.716+0000 7f034eebc700 1 -- 172.21.15.2:0/4061195104 wait complete. 2024-03-15T12:48:54.472 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.716+0000 7f034eebc700 1 Processor -- start 2024-03-15T12:48:54.472 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.716+0000 7f034eebc700 1 -- start start 2024-03-15T12:48:54.472 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.716+0000 7f034eebc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03481078e0 0x7f034819b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:54.472 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.716+0000 7f034eebc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034819bbb0 con 0x7f03481078e0 2024-03-15T12:48:54.472 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.716+0000 7f034cc58700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03481078e0 0x7f034819b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:54.473 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.716+0000 7f034cc58700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03481078e0 0x7f034819b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57628/0 (socket says 172.21.15.2:57628) 2024-03-15T12:48:54.473 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.716+0000 7f034cc58700 1 -- 172.21.15.2:0/2120742347 learned_addr learned my addr 172.21.15.2:0/2120742347 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:54.473 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.717+0000 7f034cc58700 1 -- 172.21.15.2:0/2120742347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c009550 con 0x7f03481078e0 2024-03-15T12:48:54.473 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.717+0000 7f034cc58700 1 --2- 172.21.15.2:0/2120742347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03481078e0 0x7f034819b670 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f033c000c00 tx=0x7f033c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:54.473 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.717+0000 7f0345ffb700 1 -- 172.21.15.2:0/2120742347 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f033c003a70 con 0x7f03481078e0 2024-03-15T12:48:54.473 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.717+0000 7f0345ffb700 1 -- 172.21.15.2:0/2120742347 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f033c003bd0 con 0x7f03481078e0 2024-03-15T12:48:54.473 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.717+0000 7f034eebc700 1 -- 172.21.15.2:0/2120742347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034819bdb0 con 0x7f03481078e0 2024-03-15T12:48:54.473 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.717+0000 7f0345ffb700 1 -- 172.21.15.2:0/2120742347 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f033c01a760 con 0x7f03481078e0 2024-03-15T12:48:54.473 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.717+0000 7f034eebc700 1 -- 172.21.15.2:0/2120742347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034819c250 con 0x7f03481078e0 2024-03-15T12:48:54.473 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.718+0000 7f0345ffb700 1 -- 172.21.15.2:0/2120742347 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 7) v1 ==== 45663+0+0 (secure 0 0 0) 0x7f033c01aa20 con 0x7f03481078e0 2024-03-15T12:48:54.473 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.718+0000 7f0345ffb700 1 --2- 172.21.15.2:0/2120742347 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f0338038ba0 0x7f033803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:54.473 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.718+0000 7f0345ffb700 1 -- 172.21.15.2:0/2120742347 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f033c04cb40 con 0x7f03481078e0 2024-03-15T12:48:54.473 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.718+0000 7f0347fff700 1 --2- 172.21.15.2:0/2120742347 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f0338038ba0 0x7f033803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:54.474 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.718+0000 7f034eebc700 1 -- 172.21.15.2:0/2120742347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034810bcc0 con 0x7f03481078e0 2024-03-15T12:48:54.474 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.719+0000 7f0347fff700 1 --2- 172.21.15.2:0/2120742347 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f0338038ba0 0x7f033803b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0334006fd0 tx=0x7f0334006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:54.474 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.724+0000 7f0345ffb700 1 -- 172.21.15.2:0/2120742347 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f033c04ce60 con 0x7f03481078e0 2024-03-15T12:48:54.474 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.890+0000 7f0345ffb700 1 -- 172.21.15.2:0/2120742347 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f033c02dc90 con 0x7f03481078e0 2024-03-15T12:48:54.474 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.947+0000 7f034eebc700 1 -- 172.21.15.2:0/2120742347 --> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f0348104180 con 0x7f0338038ba0 2024-03-15T12:48:54.474 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.952+0000 7f0345ffb700 1 -- 172.21.15.2:0/2120742347 <== mgr.14118 v2:172.21.15.2:6800/3269509343 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0348104180 con 0x7f0338038ba0 2024-03-15T12:48:54.474 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.955+0000 7f034eebc700 1 -- 172.21.15.2:0/2120742347 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f0338038ba0 msgr2=0x7f033803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:54.474 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.955+0000 7f034eebc700 1 --2- 172.21.15.2:0/2120742347 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f0338038ba0 0x7f033803b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0334006fd0 tx=0x7f0334006e40 comp rx=0 tx=0).stop 2024-03-15T12:48:54.474 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.955+0000 7f034eebc700 1 -- 172.21.15.2:0/2120742347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03481078e0 msgr2=0x7f034819b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:54.474 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.955+0000 7f034eebc700 1 --2- 172.21.15.2:0/2120742347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03481078e0 0x7f034819b670 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f033c000c00 tx=0x7f033c011b70 comp rx=0 tx=0).stop 2024-03-15T12:48:54.474 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.955+0000 7f034eebc700 1 -- 172.21.15.2:0/2120742347 shutdown_connections 2024-03-15T12:48:54.474 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.955+0000 7f034eebc700 1 --2- 172.21.15.2:0/2120742347 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f0338038ba0 0x7f033803b060 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:54.474 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.955+0000 7f034eebc700 1 --2- 172.21.15.2:0/2120742347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03481078e0 0x7f034819b670 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:54.475 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.955+0000 7f034eebc700 1 -- 172.21.15.2:0/2120742347 >> 172.21.15.2:0/2120742347 conn(0x7f0348102f20 msgr2=0x7f0348103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:54.475 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.955+0000 7f034eebc700 1 -- 172.21.15.2:0/2120742347 shutdown_connections 2024-03-15T12:48:54.475 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:53.955+0000 7f034eebc700 1 -- 172.21.15.2:0/2120742347 wait complete. 2024-03-15T12:48:55.057 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:54 smithi002 ceph-mon[32060]: [15/Mar/2024:12:48:53] ENGINE Serving on https://172.21.15.2:7150 2024-03-15T12:48:55.057 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:54 smithi002 ceph-mon[32060]: mgrmap e8: x(active, since 2s) 2024-03-15T12:48:55.058 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:54 smithi002 ceph-mon[32060]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:48:55.058 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:54 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' 2024-03-15T12:48:55.058 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:54 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:48:55.058 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:54 smithi002 ceph-mon[32060]: [15/Mar/2024:12:48:53] ENGINE Serving on http://172.21.15.2:8765 2024-03-15T12:48:55.058 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:54 smithi002 ceph-mon[32060]: [15/Mar/2024:12:48:53] ENGINE Bus STARTED 2024-03-15T12:48:55.058 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:54 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:48:55.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-15T12:48:55.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.875+0000 7f7b1ffe1700 1 Processor -- start 2024-03-15T12:48:55.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.875+0000 7f7b1ffe1700 1 -- start start 2024-03-15T12:48:55.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.875+0000 7f7b1ffe1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b18108380 0x7f7b1810a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:55.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.875+0000 7f7b1ffe1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1807ae60 con 0x7f7b18108380 2024-03-15T12:48:55.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.876+0000 7f7b1dd7d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b18108380 0x7f7b1810a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:55.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.876+0000 7f7b1dd7d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b18108380 0x7f7b1810a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57630/0 (socket says 172.21.15.2:57630) 2024-03-15T12:48:55.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.876+0000 7f7b1dd7d700 1 -- 172.21.15.2:0/3738709588 learned_addr learned my addr 172.21.15.2:0/3738709588 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:55.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.876+0000 7f7b1dd7d700 1 -- 172.21.15.2:0/3738709588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1810acb0 con 0x7f7b18108380 2024-03-15T12:48:55.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.876+0000 7f7b1dd7d700 1 --2- 172.21.15.2:0/3738709588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b18108380 0x7f7b1810a770 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009e80 tx=0x7f7b1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38acb3bcc1e66911 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:55.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.876+0000 7f7b1cd7b700 1 -- 172.21.15.2:0/3738709588 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b1000e070 con 0x7f7b18108380 2024-03-15T12:48:55.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.876+0000 7f7b1cd7b700 1 -- 172.21.15.2:0/3738709588 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b1000b8c0 con 0x7f7b18108380 2024-03-15T12:48:55.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.876+0000 7f7b1cd7b700 1 -- 172.21.15.2:0/3738709588 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b100045c0 con 0x7f7b18108380 2024-03-15T12:48:55.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.877+0000 7f7b1ffe1700 1 -- 172.21.15.2:0/3738709588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b18108380 msgr2=0x7f7b1810a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:55.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.877+0000 7f7b1ffe1700 1 --2- 172.21.15.2:0/3738709588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b18108380 0x7f7b1810a770 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009e80 tx=0x7f7b1000b330 comp rx=0 tx=0).stop 2024-03-15T12:48:55.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.877+0000 7f7b1ffe1700 1 -- 172.21.15.2:0/3738709588 shutdown_connections 2024-03-15T12:48:55.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.877+0000 7f7b1ffe1700 1 --2- 172.21.15.2:0/3738709588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b18108380 0x7f7b1810a770 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:55.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.877+0000 7f7b1ffe1700 1 -- 172.21.15.2:0/3738709588 >> 172.21.15.2:0/3738709588 conn(0x7f7b18101e10 msgr2=0x7f7b18104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:55.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.877+0000 7f7b1ffe1700 1 -- 172.21.15.2:0/3738709588 shutdown_connections 2024-03-15T12:48:55.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.877+0000 7f7b1ffe1700 1 -- 172.21.15.2:0/3738709588 wait complete. 2024-03-15T12:48:55.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.877+0000 7f7b1ffe1700 1 Processor -- start 2024-03-15T12:48:55.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.877+0000 7f7b1ffe1700 1 -- start start 2024-03-15T12:48:55.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.878+0000 7f7b1ffe1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b18108380 0x7f7b1819b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:55.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.878+0000 7f7b1ffe1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1819bb90 con 0x7f7b18108380 2024-03-15T12:48:55.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.878+0000 7f7b1dd7d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b18108380 0x7f7b1819b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:55.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.878+0000 7f7b1dd7d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b18108380 0x7f7b1819b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57642/0 (socket says 172.21.15.2:57642) 2024-03-15T12:48:55.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.878+0000 7f7b1dd7d700 1 -- 172.21.15.2:0/1218877300 learned_addr learned my addr 172.21.15.2:0/1218877300 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:55.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.878+0000 7f7b1dd7d700 1 -- 172.21.15.2:0/1218877300 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b10009b30 con 0x7f7b18108380 2024-03-15T12:48:55.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.878+0000 7f7b1dd7d700 1 --2- 172.21.15.2:0/1218877300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b18108380 0x7f7b1819b650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7b10004350 tx=0x7f7b10004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:55.588 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.878+0000 7f7b16ffd700 1 -- 172.21.15.2:0/1218877300 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b1000e040 con 0x7f7b18108380 2024-03-15T12:48:55.588 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.878+0000 7f7b16ffd700 1 -- 172.21.15.2:0/1218877300 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b10003e60 con 0x7f7b18108380 2024-03-15T12:48:55.588 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.878+0000 7f7b1ffe1700 1 -- 172.21.15.2:0/1218877300 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1819bd90 con 0x7f7b18108380 2024-03-15T12:48:55.588 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.878+0000 7f7b16ffd700 1 -- 172.21.15.2:0/1218877300 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b10012820 con 0x7f7b18108380 2024-03-15T12:48:55.588 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.878+0000 7f7b1ffe1700 1 -- 172.21.15.2:0/1218877300 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1819c290 con 0x7f7b18108380 2024-03-15T12:48:55.588 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.879+0000 7f7b16ffd700 1 -- 172.21.15.2:0/1218877300 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f7b1001c440 con 0x7f7b18108380 2024-03-15T12:48:55.588 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.879+0000 7f7b1ffe1700 1 -- 172.21.15.2:0/1218877300 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1810bab0 con 0x7f7b18108380 2024-03-15T12:48:55.588 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.879+0000 7f7b16ffd700 1 --2- 172.21.15.2:0/1218877300 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7b00038cb0 0x7f7b0003b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:55.588 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.879+0000 7f7b16ffd700 1 -- 172.21.15.2:0/1218877300 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7b1004cdc0 con 0x7f7b18108380 2024-03-15T12:48:55.588 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.880+0000 7f7b1d57c700 1 --2- 172.21.15.2:0/1218877300 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7b00038cb0 0x7f7b0003b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:55.588 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.880+0000 7f7b1d57c700 1 --2- 172.21.15.2:0/1218877300 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7b00038cb0 0x7f7b0003b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c006fd0 tx=0x7f7b0c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:55.588 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:54.884+0000 7f7b16ffd700 1 -- 172.21.15.2:0/1218877300 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7b1001bbf0 con 0x7f7b18108380 2024-03-15T12:48:55.588 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:55.090+0000 7f7b1ffe1700 1 -- 172.21.15.2:0/1218877300 --> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b18103050 con 0x7f7b00038cb0 2024-03-15T12:48:55.589 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:55.091+0000 7f7b16ffd700 1 -- 172.21.15.2:0/1218877300 <== mgr.14118 v2:172.21.15.2:6800/3269509343 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f7b18103050 con 0x7f7b00038cb0 2024-03-15T12:48:55.589 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:55.093+0000 7f7b1ffe1700 1 -- 172.21.15.2:0/1218877300 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7b00038cb0 msgr2=0x7f7b0003b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:55.589 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:55.093+0000 7f7b1ffe1700 1 --2- 172.21.15.2:0/1218877300 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7b00038cb0 0x7f7b0003b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c006fd0 tx=0x7f7b0c006e40 comp rx=0 tx=0).stop 2024-03-15T12:48:55.589 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:55.093+0000 7f7b1ffe1700 1 -- 172.21.15.2:0/1218877300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b18108380 msgr2=0x7f7b1819b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:55.589 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:55.093+0000 7f7b1ffe1700 1 --2- 172.21.15.2:0/1218877300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b18108380 0x7f7b1819b650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7b10004350 tx=0x7f7b10004710 comp rx=0 tx=0).stop 2024-03-15T12:48:55.589 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:55.093+0000 7f7b1ffe1700 1 -- 172.21.15.2:0/1218877300 shutdown_connections 2024-03-15T12:48:55.589 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:55.093+0000 7f7b1ffe1700 1 --2- 172.21.15.2:0/1218877300 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7b00038cb0 0x7f7b0003b170 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:55.589 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:55.093+0000 7f7b1ffe1700 1 --2- 172.21.15.2:0/1218877300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b18108380 0x7f7b1819b650 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:55.589 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:55.093+0000 7f7b1ffe1700 1 -- 172.21.15.2:0/1218877300 >> 172.21.15.2:0/1218877300 conn(0x7f7b18101e10 msgr2=0x7f7b18102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:55.589 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:55.093+0000 7f7b1ffe1700 1 -- 172.21.15.2:0/1218877300 shutdown_connections 2024-03-15T12:48:55.589 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:55.093+0000 7f7b1ffe1700 1 -- 172.21.15.2:0/1218877300 wait complete. 2024-03-15T12:48:55.589 INFO:teuthology.orchestra.run.smithi002.stdout:Generating ssh key... 2024-03-15T12:48:56.231 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:55 smithi002 ceph-mon[32060]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:48:56.967 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: Generating public/private rsa key pair. 2024-03-15T12:48:56.967 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: Your identification has been saved in /tmp/tmppkquuy8k/key. 2024-03-15T12:48:56.967 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: Your public key has been saved in /tmp/tmppkquuy8k/key.pub. 2024-03-15T12:48:56.967 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: The key fingerprint is: 2024-03-15T12:48:56.968 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: SHA256:i4MxEkJBXbvhjNnUZ8Uif/H9mj5UURhiGXsycHrZ8sI ceph-2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:48:56.968 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: The key's randomart image is: 2024-03-15T12:48:56.968 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: +---[RSA 3072]----+ 2024-03-15T12:48:56.968 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: |.+o .. o.=o.oo| 2024-03-15T12:48:56.968 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: |. . o. ..B.=.. | 2024-03-15T12:48:56.968 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: |. . + .ooo X + .| 2024-03-15T12:48:56.968 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: | . .B o o. + B ..| 2024-03-15T12:48:56.968 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: | .oo= S . E ...| 2024-03-15T12:48:56.968 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: | . + . . .. .| 2024-03-15T12:48:56.968 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: | . o . . o | 2024-03-15T12:48:56.968 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: | . + | 2024-03-15T12:48:56.968 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: | ... | 2024-03-15T12:48:56.968 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:48:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: +----[SHA256]-----+ 2024-03-15T12:48:57.242 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.030+0000 7fd153e69700 1 Processor -- start 2024-03-15T12:48:57.242 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.030+0000 7fd153e69700 1 -- start start 2024-03-15T12:48:57.242 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.030+0000 7fd153e69700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd14c107910 0x7fd14c107d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:57.242 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.030+0000 7fd153e69700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c108300 con 0x7fd14c107910 2024-03-15T12:48:57.242 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.030+0000 7fd151c05700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd14c107910 0x7fd14c107d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:57.242 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.031+0000 7fd151c05700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd14c107910 0x7fd14c107d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57658/0 (socket says 172.21.15.2:57658) 2024-03-15T12:48:57.242 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.031+0000 7fd151c05700 1 -- 172.21.15.2:0/3195069582 learned_addr learned my addr 172.21.15.2:0/3195069582 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:57.242 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.031+0000 7fd151c05700 1 -- 172.21.15.2:0/3195069582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c108ac0 con 0x7fd14c107910 2024-03-15T12:48:57.242 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.031+0000 7fd151c05700 1 --2- 172.21.15.2:0/3195069582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd14c107910 0x7fd14c107d30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009e80 tx=0x7fd13c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ca8fc88c22e66c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:57.242 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.031+0000 7fd150c03700 1 -- 172.21.15.2:0/3195069582 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd13c00e070 con 0x7fd14c107910 2024-03-15T12:48:57.242 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.031+0000 7fd150c03700 1 -- 172.21.15.2:0/3195069582 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd13c00b8c0 con 0x7fd14c107910 2024-03-15T12:48:57.242 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.031+0000 7fd150c03700 1 -- 172.21.15.2:0/3195069582 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd13c0045c0 con 0x7fd14c107910 2024-03-15T12:48:57.243 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.032+0000 7fd153e69700 1 -- 172.21.15.2:0/3195069582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd14c107910 msgr2=0x7fd14c107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:57.243 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.032+0000 7fd153e69700 1 --2- 172.21.15.2:0/3195069582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd14c107910 0x7fd14c107d30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009e80 tx=0x7fd13c00b330 comp rx=0 tx=0).stop 2024-03-15T12:48:57.243 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.032+0000 7fd153e69700 1 -- 172.21.15.2:0/3195069582 shutdown_connections 2024-03-15T12:48:57.243 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.032+0000 7fd153e69700 1 --2- 172.21.15.2:0/3195069582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd14c107910 0x7fd14c107d30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:57.243 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.032+0000 7fd153e69700 1 -- 172.21.15.2:0/3195069582 >> 172.21.15.2:0/3195069582 conn(0x7fd14c102f50 msgr2=0x7fd14c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:57.243 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.032+0000 7fd153e69700 1 -- 172.21.15.2:0/3195069582 shutdown_connections 2024-03-15T12:48:57.243 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.032+0000 7fd153e69700 1 -- 172.21.15.2:0/3195069582 wait complete. 2024-03-15T12:48:57.243 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.032+0000 7fd153e69700 1 Processor -- start 2024-03-15T12:48:57.243 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.032+0000 7fd153e69700 1 -- start start 2024-03-15T12:48:57.243 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.033+0000 7fd153e69700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd14c107910 0x7fd14c19b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:57.243 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.033+0000 7fd153e69700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c19bbb0 con 0x7fd14c107910 2024-03-15T12:48:57.243 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.033+0000 7fd151c05700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd14c107910 0x7fd14c19b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:57.243 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.033+0000 7fd151c05700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd14c107910 0x7fd14c19b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57662/0 (socket says 172.21.15.2:57662) 2024-03-15T12:48:57.243 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.033+0000 7fd151c05700 1 -- 172.21.15.2:0/1976197885 learned_addr learned my addr 172.21.15.2:0/1976197885 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:57.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.033+0000 7fd151c05700 1 -- 172.21.15.2:0/1976197885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c009b30 con 0x7fd14c107910 2024-03-15T12:48:57.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.033+0000 7fd151c05700 1 --2- 172.21.15.2:0/1976197885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd14c107910 0x7fd14c19b670 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd13c004350 tx=0x7fd13c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:57.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.033+0000 7fd14affd700 1 -- 172.21.15.2:0/1976197885 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd13c00e040 con 0x7fd14c107910 2024-03-15T12:48:57.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.033+0000 7fd14affd700 1 -- 172.21.15.2:0/1976197885 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd13c0094d0 con 0x7fd14c107910 2024-03-15T12:48:57.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.033+0000 7fd153e69700 1 -- 172.21.15.2:0/1976197885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c19bdb0 con 0x7fd14c107910 2024-03-15T12:48:57.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.033+0000 7fd14affd700 1 -- 172.21.15.2:0/1976197885 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd13c0127b0 con 0x7fd14c107910 2024-03-15T12:48:57.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.033+0000 7fd153e69700 1 -- 172.21.15.2:0/1976197885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c19c2b0 con 0x7fd14c107910 2024-03-15T12:48:57.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.034+0000 7fd14affd700 1 -- 172.21.15.2:0/1976197885 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fd13c012a70 con 0x7fd14c107910 2024-03-15T12:48:57.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.034+0000 7fd153e69700 1 -- 172.21.15.2:0/1976197885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c10bcf0 con 0x7fd14c107910 2024-03-15T12:48:57.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.034+0000 7fd14affd700 1 --2- 172.21.15.2:0/1976197885 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fd134038c60 0x7fd13403b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:57.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.034+0000 7fd14affd700 1 -- 172.21.15.2:0/1976197885 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd13c04cec0 con 0x7fd14c107910 2024-03-15T12:48:57.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.035+0000 7fd151404700 1 --2- 172.21.15.2:0/1976197885 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fd134038c60 0x7fd13403b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:57.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.035+0000 7fd151404700 1 --2- 172.21.15.2:0/1976197885 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fd134038c60 0x7fd13403b120 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd140006fd0 tx=0x7fd140006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:57.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.040+0000 7fd14affd700 1 -- 172.21.15.2:0/1976197885 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd13c01ba90 con 0x7fd14c107910 2024-03-15T12:48:57.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.244+0000 7fd153e69700 1 -- 172.21.15.2:0/1976197885 --> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd14c104100 con 0x7fd134038c60 2024-03-15T12:48:57.245 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.682+0000 7fd14affd700 1 -- 172.21.15.2:0/1976197885 <== mgr.14118 v2:172.21.15.2:6800/3269509343 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd14c104100 con 0x7fd134038c60 2024-03-15T12:48:57.245 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.684+0000 7fd153e69700 1 -- 172.21.15.2:0/1976197885 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fd134038c60 msgr2=0x7fd13403b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:57.245 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.684+0000 7fd153e69700 1 --2- 172.21.15.2:0/1976197885 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fd134038c60 0x7fd13403b120 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd140006fd0 tx=0x7fd140006e40 comp rx=0 tx=0).stop 2024-03-15T12:48:57.245 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.684+0000 7fd153e69700 1 -- 172.21.15.2:0/1976197885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd14c107910 msgr2=0x7fd14c19b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:57.245 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.684+0000 7fd153e69700 1 --2- 172.21.15.2:0/1976197885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd14c107910 0x7fd14c19b670 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd13c004350 tx=0x7fd13c003d00 comp rx=0 tx=0).stop 2024-03-15T12:48:57.245 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.684+0000 7fd153e69700 1 -- 172.21.15.2:0/1976197885 shutdown_connections 2024-03-15T12:48:57.245 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.684+0000 7fd153e69700 1 --2- 172.21.15.2:0/1976197885 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fd134038c60 0x7fd13403b120 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:57.245 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.684+0000 7fd153e69700 1 --2- 172.21.15.2:0/1976197885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd14c107910 0x7fd14c19b670 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:57.245 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.684+0000 7fd153e69700 1 -- 172.21.15.2:0/1976197885 >> 172.21.15.2:0/1976197885 conn(0x7fd14c102f50 msgr2=0x7fd14c1039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:57.245 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.684+0000 7fd153e69700 1 -- 172.21.15.2:0/1976197885 shutdown_connections 2024-03-15T12:48:57.245 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:56.684+0000 7fd153e69700 1 -- 172.21.15.2:0/1976197885 wait complete. 2024-03-15T12:48:57.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:57 smithi002 ceph-mon[32060]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:48:57.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:57 smithi002 ceph-mon[32060]: Generating ssh key... 2024-03-15T12:48:57.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:57 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' 2024-03-15T12:48:57.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:57 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' 2024-03-15T12:48:58.399 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClSVWPIfdS232x37HptOHk6PeuHhgDy/yQM2jr2hgAUrNMcQDnlOf0ZzOU/UmtiwBC466wrpztWU73HxLo7s1GbtebBx8WDYC3k1+mcBcm4cIblmnWSKLd4bQtofJqM4aFu7MhAcRdk1OhDbFj8kWdh7huF78MfmdLiYQRoFtiOem4ah2ONlCd67bFzJVaRCMmhBFNBy6wa4KNIwRJpHZybfce1jusTAOuAmdoTECT50bpOiNAnv6rdrrbhbeb6geYWv807QIHxg+zu9YzZopIanJEDkqAzCHBKfTGhcYRL1lvE1md+zYNAy2mM/XnF+tdY/0PBtoXZBREOgNDhGx4GuaXNzmC0QotvQBa893ofjDWGk/+au+R7Qk3YJaza/XJeyj+SkZnFgxgo+/uVp8kR67dogt28raHjs/dNtFWmnwtvFOp+SAi/PclAZfQK9eDldfoW0c3vVYCiJEHAP8jMPOQZR4fUgQ62nX/M7aXQcdroN8MWGOS5k+Oq+g7gmE= ceph-2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:48:58.399 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.630+0000 7fe47cc62700 1 Processor -- start 2024-03-15T12:48:58.399 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.631+0000 7fe47cc62700 1 -- start start 2024-03-15T12:48:58.399 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.631+0000 7fe47cc62700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4781078f0 0x7fe478107d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:58.399 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.631+0000 7fe47cc62700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4781082e0 con 0x7fe4781078f0 2024-03-15T12:48:58.399 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.631+0000 7fe47659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4781078f0 0x7fe478107d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:58.399 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.631+0000 7fe47659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4781078f0 0x7fe478107d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57676/0 (socket says 172.21.15.2:57676) 2024-03-15T12:48:58.399 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.631+0000 7fe47659c700 1 -- 172.21.15.2:0/1193058309 learned_addr learned my addr 172.21.15.2:0/1193058309 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:58.400 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.631+0000 7fe47659c700 1 -- 172.21.15.2:0/1193058309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe478108aa0 con 0x7fe4781078f0 2024-03-15T12:48:58.400 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.632+0000 7fe47659c700 1 --2- 172.21.15.2:0/1193058309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4781078f0 0x7fe478107d10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe4680098a0 tx=0x7fe468009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c7a39b6a6e96e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:58.400 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.632+0000 7fe47559a700 1 -- 172.21.15.2:0/1193058309 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe468004160 con 0x7fe4781078f0 2024-03-15T12:48:58.400 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.632+0000 7fe47559a700 1 -- 172.21.15.2:0/1193058309 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4680042c0 con 0x7fe4781078f0 2024-03-15T12:48:58.400 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.632+0000 7fe47559a700 1 -- 172.21.15.2:0/1193058309 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe468011ab0 con 0x7fe4781078f0 2024-03-15T12:48:58.400 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.632+0000 7fe47cc62700 1 -- 172.21.15.2:0/1193058309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4781078f0 msgr2=0x7fe478107d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:58.400 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.632+0000 7fe47cc62700 1 --2- 172.21.15.2:0/1193058309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4781078f0 0x7fe478107d10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe4680098a0 tx=0x7fe468009bb0 comp rx=0 tx=0).stop 2024-03-15T12:48:58.400 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.632+0000 7fe47cc62700 1 -- 172.21.15.2:0/1193058309 shutdown_connections 2024-03-15T12:48:58.400 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.632+0000 7fe47cc62700 1 --2- 172.21.15.2:0/1193058309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4781078f0 0x7fe478107d10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:58.400 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.632+0000 7fe47cc62700 1 -- 172.21.15.2:0/1193058309 >> 172.21.15.2:0/1193058309 conn(0x7fe478102f50 msgr2=0x7fe478105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:58.400 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.633+0000 7fe47cc62700 1 -- 172.21.15.2:0/1193058309 shutdown_connections 2024-03-15T12:48:58.400 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.633+0000 7fe47cc62700 1 -- 172.21.15.2:0/1193058309 wait complete. 2024-03-15T12:48:58.400 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.633+0000 7fe47cc62700 1 Processor -- start 2024-03-15T12:48:58.400 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.633+0000 7fe47cc62700 1 -- start start 2024-03-15T12:48:58.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.634+0000 7fe47cc62700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4781078f0 0x7fe47819b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:58.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.634+0000 7fe47cc62700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47819bbc0 con 0x7fe4781078f0 2024-03-15T12:48:58.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.634+0000 7fe47659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4781078f0 0x7fe47819b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:58.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.634+0000 7fe47659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4781078f0 0x7fe47819b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57682/0 (socket says 172.21.15.2:57682) 2024-03-15T12:48:58.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.634+0000 7fe47659c700 1 -- 172.21.15.2:0/983968135 learned_addr learned my addr 172.21.15.2:0/983968135 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:48:58.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.634+0000 7fe47659c700 1 -- 172.21.15.2:0/983968135 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe468009550 con 0x7fe4781078f0 2024-03-15T12:48:58.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.634+0000 7fe47659c700 1 --2- 172.21.15.2:0/983968135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4781078f0 0x7fe47819b680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe468006af0 tx=0x7fe468011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:58.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.634+0000 7fe46f7fe700 1 -- 172.21.15.2:0/983968135 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe468003930 con 0x7fe4781078f0 2024-03-15T12:48:58.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.634+0000 7fe47cc62700 1 -- 172.21.15.2:0/983968135 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47819bdc0 con 0x7fe4781078f0 2024-03-15T12:48:58.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.634+0000 7fe46f7fe700 1 -- 172.21.15.2:0/983968135 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe468003a90 con 0x7fe4781078f0 2024-03-15T12:48:58.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.634+0000 7fe46f7fe700 1 -- 172.21.15.2:0/983968135 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe468019670 con 0x7fe4781078f0 2024-03-15T12:48:58.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.634+0000 7fe47cc62700 1 -- 172.21.15.2:0/983968135 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47819c260 con 0x7fe4781078f0 2024-03-15T12:48:58.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.635+0000 7fe46f7fe700 1 -- 172.21.15.2:0/983968135 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fe46801a430 con 0x7fe4781078f0 2024-03-15T12:48:58.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.635+0000 7fe46f7fe700 1 --2- 172.21.15.2:0/983968135 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fe464038cb0 0x7fe46403b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:48:58.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.635+0000 7fe47cc62700 1 -- 172.21.15.2:0/983968135 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe47810bcd0 con 0x7fe4781078f0 2024-03-15T12:48:58.402 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.636+0000 7fe46f7fe700 1 -- 172.21.15.2:0/983968135 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe46804ce60 con 0x7fe4781078f0 2024-03-15T12:48:58.402 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.636+0000 7fe475d9b700 1 --2- 172.21.15.2:0/983968135 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fe464038cb0 0x7fe46403b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:48:58.402 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.636+0000 7fe475d9b700 1 --2- 172.21.15.2:0/983968135 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fe464038cb0 0x7fe46403b170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe460006fd0 tx=0x7fe460006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:48:58.402 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.640+0000 7fe46f7fe700 1 -- 172.21.15.2:0/983968135 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe468023c20 con 0x7fe4781078f0 2024-03-15T12:48:58.402 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.846+0000 7fe47cc62700 1 -- 172.21.15.2:0/983968135 --> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fe478104170 con 0x7fe464038cb0 2024-03-15T12:48:58.402 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.847+0000 7fe46f7fe700 1 -- 172.21.15.2:0/983968135 <== mgr.14118 v2:172.21.15.2:6800/3269509343 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fe478104170 con 0x7fe464038cb0 2024-03-15T12:48:58.402 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.849+0000 7fe47cc62700 1 -- 172.21.15.2:0/983968135 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fe464038cb0 msgr2=0x7fe46403b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:58.402 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.849+0000 7fe47cc62700 1 --2- 172.21.15.2:0/983968135 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fe464038cb0 0x7fe46403b170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe460006fd0 tx=0x7fe460006e40 comp rx=0 tx=0).stop 2024-03-15T12:48:58.402 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.849+0000 7fe47cc62700 1 -- 172.21.15.2:0/983968135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4781078f0 msgr2=0x7fe47819b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:48:58.402 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.849+0000 7fe47cc62700 1 --2- 172.21.15.2:0/983968135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4781078f0 0x7fe47819b680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe468006af0 tx=0x7fe468011b70 comp rx=0 tx=0).stop 2024-03-15T12:48:58.402 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.849+0000 7fe47cc62700 1 -- 172.21.15.2:0/983968135 shutdown_connections 2024-03-15T12:48:58.402 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.849+0000 7fe47cc62700 1 --2- 172.21.15.2:0/983968135 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fe464038cb0 0x7fe46403b170 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:58.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.849+0000 7fe47cc62700 1 --2- 172.21.15.2:0/983968135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4781078f0 0x7fe47819b680 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:48:58.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.849+0000 7fe47cc62700 1 -- 172.21.15.2:0/983968135 >> 172.21.15.2:0/983968135 conn(0x7fe478102f50 msgr2=0x7fe478103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:48:58.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.849+0000 7fe47cc62700 1 -- 172.21.15.2:0/983968135 shutdown_connections 2024-03-15T12:48:58.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:57.849+0000 7fe47cc62700 1 -- 172.21.15.2:0/983968135 wait complete. 2024-03-15T12:48:58.403 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-15T12:48:58.403 INFO:teuthology.orchestra.run.smithi002.stdout:Adding key to root@localhost authorized_keys... 2024-03-15T12:48:58.403 INFO:teuthology.orchestra.run.smithi002.stdout:Adding host smithi002... 2024-03-15T12:48:59.150 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:58 smithi002 ceph-mon[32060]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:49:00.046 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:59 smithi002 ceph-mon[32060]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi002", "addr": "172.21.15.2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:49:00.046 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:48:59 smithi002 ceph-mon[32060]: Deploying cephadm binary to smithi002 2024-03-15T12:49:03.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Added host 'smithi002' with addr '172.21.15.2' 2024-03-15T12:49:03.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.824+0000 7fbcf0d9c700 1 Processor -- start 2024-03-15T12:49:03.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.824+0000 7fbcf0d9c700 1 -- start start 2024-03-15T12:49:03.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.824+0000 7fbcf0d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcec1067b0 0x7fbcec106bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:03.931 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.824+0000 7fbcf0d9c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec1071a0 con 0x7fbcec1067b0 2024-03-15T12:49:03.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.825+0000 7fbcea59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcec1067b0 0x7fbcec106bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:03.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.825+0000 7fbcea59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcec1067b0 0x7fbcec106bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48668/0 (socket says 172.21.15.2:48668) 2024-03-15T12:49:03.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.825+0000 7fbcea59c700 1 -- 172.21.15.2:0/583057146 learned_addr learned my addr 172.21.15.2:0/583057146 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:03.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.825+0000 7fbcea59c700 1 -- 172.21.15.2:0/583057146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec1079b0 con 0x7fbcec1067b0 2024-03-15T12:49:03.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.825+0000 7fbcea59c700 1 --2- 172.21.15.2:0/583057146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcec1067b0 0x7fbcec106bd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbce40098a0 tx=0x7fbce4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b299b2f4c703de36 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:03.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.825+0000 7fbce959a700 1 -- 172.21.15.2:0/583057146 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbce4004160 con 0x7fbcec1067b0 2024-03-15T12:49:03.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.825+0000 7fbce959a700 1 -- 172.21.15.2:0/583057146 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbce40042c0 con 0x7fbcec1067b0 2024-03-15T12:49:03.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.825+0000 7fbce959a700 1 -- 172.21.15.2:0/583057146 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbce4011ab0 con 0x7fbcec1067b0 2024-03-15T12:49:03.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.826+0000 7fbcf0d9c700 1 -- 172.21.15.2:0/583057146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcec1067b0 msgr2=0x7fbcec106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:03.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.826+0000 7fbcf0d9c700 1 --2- 172.21.15.2:0/583057146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcec1067b0 0x7fbcec106bd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbce40098a0 tx=0x7fbce4009bb0 comp rx=0 tx=0).stop 2024-03-15T12:49:03.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.826+0000 7fbcf0d9c700 1 -- 172.21.15.2:0/583057146 shutdown_connections 2024-03-15T12:49:03.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.826+0000 7fbcf0d9c700 1 --2- 172.21.15.2:0/583057146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcec1067b0 0x7fbcec106bd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:03.932 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.826+0000 7fbcf0d9c700 1 -- 172.21.15.2:0/583057146 >> 172.21.15.2:0/583057146 conn(0x7fbcec101d70 msgr2=0x7fbcec104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:03.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.826+0000 7fbcf0d9c700 1 -- 172.21.15.2:0/583057146 shutdown_connections 2024-03-15T12:49:03.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.826+0000 7fbcf0d9c700 1 -- 172.21.15.2:0/583057146 wait complete. 2024-03-15T12:49:03.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.827+0000 7fbcf0d9c700 1 Processor -- start 2024-03-15T12:49:03.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.827+0000 7fbcf0d9c700 1 -- start start 2024-03-15T12:49:03.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.827+0000 7fbcf0d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcec1067b0 0x7fbcec19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:03.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.827+0000 7fbcf0d9c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec19bc50 con 0x7fbcec1067b0 2024-03-15T12:49:03.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.827+0000 7fbcea59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcec1067b0 0x7fbcec19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:03.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.827+0000 7fbcea59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcec1067b0 0x7fbcec19b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48684/0 (socket says 172.21.15.2:48684) 2024-03-15T12:49:03.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.827+0000 7fbcea59c700 1 -- 172.21.15.2:0/3996464126 learned_addr learned my addr 172.21.15.2:0/3996464126 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:03.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.827+0000 7fbcea59c700 1 -- 172.21.15.2:0/3996464126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce4009550 con 0x7fbcec1067b0 2024-03-15T12:49:03.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.828+0000 7fbcea59c700 1 --2- 172.21.15.2:0/3996464126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcec1067b0 0x7fbcec19b710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbce4011670 tx=0x7fbce4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:03.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.828+0000 7fbce37fe700 1 -- 172.21.15.2:0/3996464126 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbce40039f0 con 0x7fbcec1067b0 2024-03-15T12:49:03.933 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.828+0000 7fbce37fe700 1 -- 172.21.15.2:0/3996464126 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbce4003b50 con 0x7fbcec1067b0 2024-03-15T12:49:03.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.828+0000 7fbcf0d9c700 1 -- 172.21.15.2:0/3996464126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec19be50 con 0x7fbcec1067b0 2024-03-15T12:49:03.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.828+0000 7fbce37fe700 1 -- 172.21.15.2:0/3996464126 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbce40197f0 con 0x7fbcec1067b0 2024-03-15T12:49:03.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.828+0000 7fbcf0d9c700 1 -- 172.21.15.2:0/3996464126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec19c2f0 con 0x7fbcec1067b0 2024-03-15T12:49:03.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.829+0000 7fbce37fe700 1 -- 172.21.15.2:0/3996464126 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fbce4019950 con 0x7fbcec1067b0 2024-03-15T12:49:03.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.829+0000 7fbcf0d9c700 1 -- 172.21.15.2:0/3996464126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec195850 con 0x7fbcec1067b0 2024-03-15T12:49:03.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.829+0000 7fbce37fe700 1 --2- 172.21.15.2:0/3996464126 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fbcd8038cb0 0x7fbcd803b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:03.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.829+0000 7fbce37fe700 1 -- 172.21.15.2:0/3996464126 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbce404cee0 con 0x7fbcec1067b0 2024-03-15T12:49:03.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.829+0000 7fbce9d9b700 1 --2- 172.21.15.2:0/3996464126 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fbcd8038cb0 0x7fbcd803b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:03.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.829+0000 7fbce9d9b700 1 --2- 172.21.15.2:0/3996464126 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fbcd8038cb0 0x7fbcd803b170 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4006fd0 tx=0x7fbcd4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:03.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:58.834+0000 7fbce37fe700 1 -- 172.21.15.2:0/3996464126 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbce402b340 con 0x7fbcec1067b0 2024-03-15T12:49:03.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:48:59.044+0000 7fbcf0d9c700 1 -- 172.21.15.2:0/3996464126 --> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi002", "addr": "172.21.15.2", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcec02db90 con 0x7fbcd8038cb0 2024-03-15T12:49:03.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:03.356+0000 7fbce37fe700 1 -- 172.21.15.2:0/3996464126 <== mgr.14118 v2:172.21.15.2:6800/3269509343 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fbcec02db90 con 0x7fbcd8038cb0 2024-03-15T12:49:03.934 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:03.359+0000 7fbcf0d9c700 1 -- 172.21.15.2:0/3996464126 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fbcd8038cb0 msgr2=0x7fbcd803b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:03.935 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:03.359+0000 7fbcf0d9c700 1 --2- 172.21.15.2:0/3996464126 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fbcd8038cb0 0x7fbcd803b170 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4006fd0 tx=0x7fbcd4006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:03.935 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:03.359+0000 7fbcf0d9c700 1 -- 172.21.15.2:0/3996464126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcec1067b0 msgr2=0x7fbcec19b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:03.935 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:03.359+0000 7fbcf0d9c700 1 --2- 172.21.15.2:0/3996464126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcec1067b0 0x7fbcec19b710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbce4011670 tx=0x7fbce4011b70 comp rx=0 tx=0).stop 2024-03-15T12:49:03.935 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:03.359+0000 7fbcf0d9c700 1 -- 172.21.15.2:0/3996464126 shutdown_connections 2024-03-15T12:49:03.935 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:03.359+0000 7fbcf0d9c700 1 --2- 172.21.15.2:0/3996464126 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7fbcd8038cb0 0x7fbcd803b170 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:03.935 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:03.359+0000 7fbcf0d9c700 1 --2- 172.21.15.2:0/3996464126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcec1067b0 0x7fbcec19b710 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:03.935 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:03.359+0000 7fbcf0d9c700 1 -- 172.21.15.2:0/3996464126 >> 172.21.15.2:0/3996464126 conn(0x7fbcec101d70 msgr2=0x7fbcec102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:03.935 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:03.359+0000 7fbcf0d9c700 1 -- 172.21.15.2:0/3996464126 shutdown_connections 2024-03-15T12:49:03.935 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:03.359+0000 7fbcf0d9c700 1 -- 172.21.15.2:0/3996464126 wait complete. 2024-03-15T12:49:03.935 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying unmanaged mon service... 2024-03-15T12:49:04.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:04 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' 2024-03-15T12:49:04.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:04 smithi002 ceph-mon[32060]: Added host smithi002 2024-03-15T12:49:04.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:04 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:49:05.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-15T12:49:05.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.376+0000 7f4b8faa9700 1 Processor -- start 2024-03-15T12:49:05.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.377+0000 7f4b8faa9700 1 -- start start 2024-03-15T12:49:05.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.377+0000 7f4b8faa9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b881078e0 0x7f4b88107d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:05.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.377+0000 7f4b8faa9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b881082d0 con 0x7f4b881078e0 2024-03-15T12:49:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.377+0000 7f4b8d845700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b881078e0 0x7f4b88107d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.377+0000 7f4b8d845700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b881078e0 0x7f4b88107d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48692/0 (socket says 172.21.15.2:48692) 2024-03-15T12:49:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.377+0000 7f4b8d845700 1 -- 172.21.15.2:0/1952372891 learned_addr learned my addr 172.21.15.2:0/1952372891 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.377+0000 7f4b8d845700 1 -- 172.21.15.2:0/1952372891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b88108a90 con 0x7f4b881078e0 2024-03-15T12:49:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.378+0000 7f4b8d845700 1 --2- 172.21.15.2:0/1952372891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b881078e0 0x7f4b88107d00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4b840098a0 tx=0x7f4b84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ae1ba00879a1bf server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.378+0000 7f4b8c843700 1 -- 172.21.15.2:0/1952372891 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b84004160 con 0x7f4b881078e0 2024-03-15T12:49:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.378+0000 7f4b8c843700 1 -- 172.21.15.2:0/1952372891 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b840042c0 con 0x7f4b881078e0 2024-03-15T12:49:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.378+0000 7f4b8c843700 1 -- 172.21.15.2:0/1952372891 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b84011ab0 con 0x7f4b881078e0 2024-03-15T12:49:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.378+0000 7f4b8faa9700 1 -- 172.21.15.2:0/1952372891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b881078e0 msgr2=0x7f4b88107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.378+0000 7f4b8faa9700 1 --2- 172.21.15.2:0/1952372891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b881078e0 0x7f4b88107d00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4b840098a0 tx=0x7f4b84009bb0 comp rx=0 tx=0).stop 2024-03-15T12:49:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.378+0000 7f4b8faa9700 1 -- 172.21.15.2:0/1952372891 shutdown_connections 2024-03-15T12:49:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.378+0000 7f4b8faa9700 1 --2- 172.21.15.2:0/1952372891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b881078e0 0x7f4b88107d00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.378+0000 7f4b8faa9700 1 -- 172.21.15.2:0/1952372891 >> 172.21.15.2:0/1952372891 conn(0x7f4b88102f00 msgr2=0x7f4b88105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.378+0000 7f4b8faa9700 1 -- 172.21.15.2:0/1952372891 shutdown_connections 2024-03-15T12:49:05.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.378+0000 7f4b8faa9700 1 -- 172.21.15.2:0/1952372891 wait complete. 2024-03-15T12:49:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.379+0000 7f4b8faa9700 1 Processor -- start 2024-03-15T12:49:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.379+0000 7f4b8faa9700 1 -- start start 2024-03-15T12:49:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.379+0000 7f4b8faa9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b881078e0 0x7f4b8819b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.379+0000 7f4b8faa9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8819bbe0 con 0x7f4b881078e0 2024-03-15T12:49:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.379+0000 7f4b8d845700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b881078e0 0x7f4b8819b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.379+0000 7f4b8d845700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b881078e0 0x7f4b8819b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48698/0 (socket says 172.21.15.2:48698) 2024-03-15T12:49:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.379+0000 7f4b8d845700 1 -- 172.21.15.2:0/31833566 learned_addr learned my addr 172.21.15.2:0/31833566 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.379+0000 7f4b8d845700 1 -- 172.21.15.2:0/31833566 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b84009550 con 0x7f4b881078e0 2024-03-15T12:49:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.379+0000 7f4b8d845700 1 --2- 172.21.15.2:0/31833566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b881078e0 0x7f4b8819b6a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4b84000c00 tx=0x7f4b84011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.380+0000 7f4b7effd700 1 -- 172.21.15.2:0/31833566 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b84003a70 con 0x7f4b881078e0 2024-03-15T12:49:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.380+0000 7f4b7effd700 1 -- 172.21.15.2:0/31833566 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b84003bd0 con 0x7f4b881078e0 2024-03-15T12:49:05.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.380+0000 7f4b8faa9700 1 -- 172.21.15.2:0/31833566 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8819bde0 con 0x7f4b881078e0 2024-03-15T12:49:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.380+0000 7f4b7effd700 1 -- 172.21.15.2:0/31833566 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b8401a760 con 0x7f4b881078e0 2024-03-15T12:49:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.380+0000 7f4b8faa9700 1 -- 172.21.15.2:0/31833566 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8819c280 con 0x7f4b881078e0 2024-03-15T12:49:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.380+0000 7f4b7effd700 1 -- 172.21.15.2:0/31833566 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f4b8401aa20 con 0x7f4b881078e0 2024-03-15T12:49:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.381+0000 7f4b7effd700 1 --2- 172.21.15.2:0/31833566 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f4b74038cb0 0x7f4b7403b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.381+0000 7f4b7effd700 1 -- 172.21.15.2:0/31833566 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4b8404cd20 con 0x7f4b881078e0 2024-03-15T12:49:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.381+0000 7f4b8faa9700 1 -- 172.21.15.2:0/31833566 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8810bcc0 con 0x7f4b881078e0 2024-03-15T12:49:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.381+0000 7f4b8d044700 1 --2- 172.21.15.2:0/31833566 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f4b74038cb0 0x7f4b7403b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.381+0000 7f4b8d044700 1 --2- 172.21.15.2:0/31833566 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f4b74038cb0 0x7f4b7403b170 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4b78006fd0 tx=0x7f4b78006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.386+0000 7f4b7effd700 1 -- 172.21.15.2:0/31833566 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4b84018760 con 0x7f4b881078e0 2024-03-15T12:49:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.603+0000 7f4b8faa9700 1 -- 172.21.15.2:0/31833566 --> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4b8819eb60 con 0x7f4b74038cb0 2024-03-15T12:49:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.608+0000 7f4b7effd700 1 -- 172.21.15.2:0/31833566 <== mgr.14118 v2:172.21.15.2:6800/3269509343 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4b8819eb60 con 0x7f4b74038cb0 2024-03-15T12:49:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.610+0000 7f4b8faa9700 1 -- 172.21.15.2:0/31833566 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f4b74038cb0 msgr2=0x7f4b7403b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.610+0000 7f4b8faa9700 1 --2- 172.21.15.2:0/31833566 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f4b74038cb0 0x7f4b7403b170 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4b78006fd0 tx=0x7f4b78006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:05.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.610+0000 7f4b8faa9700 1 -- 172.21.15.2:0/31833566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b881078e0 msgr2=0x7f4b8819b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:05.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.610+0000 7f4b8faa9700 1 --2- 172.21.15.2:0/31833566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b881078e0 0x7f4b8819b6a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4b84000c00 tx=0x7f4b84011b70 comp rx=0 tx=0).stop 2024-03-15T12:49:05.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.610+0000 7f4b8faa9700 1 -- 172.21.15.2:0/31833566 shutdown_connections 2024-03-15T12:49:05.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.610+0000 7f4b8faa9700 1 --2- 172.21.15.2:0/31833566 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f4b74038cb0 0x7f4b7403b170 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:05.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.610+0000 7f4b8faa9700 1 --2- 172.21.15.2:0/31833566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b881078e0 0x7f4b8819b6a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:05.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.610+0000 7f4b8faa9700 1 -- 172.21.15.2:0/31833566 >> 172.21.15.2:0/31833566 conn(0x7f4b88102f00 msgr2=0x7f4b88103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:05.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.610+0000 7f4b8faa9700 1 -- 172.21.15.2:0/31833566 shutdown_connections 2024-03-15T12:49:05.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:04.610+0000 7f4b8faa9700 1 -- 172.21.15.2:0/31833566 wait complete. 2024-03-15T12:49:05.396 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying unmanaged mgr service... 2024-03-15T12:49:06.039 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:05 smithi002 ceph-mon[32060]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:49:06.039 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:05 smithi002 ceph-mon[32060]: Saving service mon spec with placement count:5 2024-03-15T12:49:06.039 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:05 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' 2024-03-15T12:49:07.036 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-15T12:49:07.036 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.160+0000 7f2e03b7b700 1 Processor -- start 2024-03-15T12:49:07.036 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.160+0000 7f2e03b7b700 1 -- start start 2024-03-15T12:49:07.036 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.160+0000 7f2e03b7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dfc108370 0x7f2dfc10a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:07.036 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.160+0000 7f2e03b7b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc07ae80 con 0x7f2dfc108370 2024-03-15T12:49:07.037 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.160+0000 7f2e01917700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dfc108370 0x7f2dfc10a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:07.037 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.160+0000 7f2e01917700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dfc108370 0x7f2dfc10a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48706/0 (socket says 172.21.15.2:48706) 2024-03-15T12:49:07.037 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.160+0000 7f2e01917700 1 -- 172.21.15.2:0/985848146 learned_addr learned my addr 172.21.15.2:0/985848146 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:07.037 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.161+0000 7f2e01917700 1 -- 172.21.15.2:0/985848146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc10aca0 con 0x7f2dfc108370 2024-03-15T12:49:07.037 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.161+0000 7f2e01917700 1 --2- 172.21.15.2:0/985848146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dfc108370 0x7f2dfc10a760 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2df80098a0 tx=0x7f2df8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ae02ebddcbeefb server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:07.037 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.161+0000 7f2e00915700 1 -- 172.21.15.2:0/985848146 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2df8004160 con 0x7f2dfc108370 2024-03-15T12:49:07.037 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.161+0000 7f2e00915700 1 -- 172.21.15.2:0/985848146 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2df80042c0 con 0x7f2dfc108370 2024-03-15T12:49:07.037 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.161+0000 7f2e00915700 1 -- 172.21.15.2:0/985848146 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2df8011ab0 con 0x7f2dfc108370 2024-03-15T12:49:07.037 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.162+0000 7f2e03b7b700 1 -- 172.21.15.2:0/985848146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dfc108370 msgr2=0x7f2dfc10a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:07.037 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.162+0000 7f2e03b7b700 1 --2- 172.21.15.2:0/985848146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dfc108370 0x7f2dfc10a760 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2df80098a0 tx=0x7f2df8009bb0 comp rx=0 tx=0).stop 2024-03-15T12:49:07.037 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.162+0000 7f2e03b7b700 1 -- 172.21.15.2:0/985848146 shutdown_connections 2024-03-15T12:49:07.037 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.162+0000 7f2e03b7b700 1 --2- 172.21.15.2:0/985848146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dfc108370 0x7f2dfc10a760 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:07.037 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.162+0000 7f2e03b7b700 1 -- 172.21.15.2:0/985848146 >> 172.21.15.2:0/985848146 conn(0x7f2dfc101e00 msgr2=0x7f2dfc104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:07.037 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.162+0000 7f2e03b7b700 1 -- 172.21.15.2:0/985848146 shutdown_connections 2024-03-15T12:49:07.038 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.162+0000 7f2e03b7b700 1 -- 172.21.15.2:0/985848146 wait complete. 2024-03-15T12:49:07.038 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.162+0000 7f2e03b7b700 1 Processor -- start 2024-03-15T12:49:07.038 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.162+0000 7f2e03b7b700 1 -- start start 2024-03-15T12:49:07.038 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.162+0000 7f2e03b7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dfc108370 0x7f2dfc19d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:07.038 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.162+0000 7f2e03b7b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc19dd90 con 0x7f2dfc108370 2024-03-15T12:49:07.038 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.163+0000 7f2e01917700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dfc108370 0x7f2dfc19d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:07.038 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.163+0000 7f2e01917700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dfc108370 0x7f2dfc19d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48708/0 (socket says 172.21.15.2:48708) 2024-03-15T12:49:07.038 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.163+0000 7f2e01917700 1 -- 172.21.15.2:0/3564020408 learned_addr learned my addr 172.21.15.2:0/3564020408 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:07.038 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.163+0000 7f2e01917700 1 -- 172.21.15.2:0/3564020408 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df8009550 con 0x7f2dfc108370 2024-03-15T12:49:07.038 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.163+0000 7f2e01917700 1 --2- 172.21.15.2:0/3564020408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dfc108370 0x7f2dfc19d850 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2df8006af0 tx=0x7f2df8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:07.038 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.163+0000 7f2df2ffd700 1 -- 172.21.15.2:0/3564020408 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2df8003930 con 0x7f2dfc108370 2024-03-15T12:49:07.038 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.163+0000 7f2e03b7b700 1 -- 172.21.15.2:0/3564020408 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dfc19df90 con 0x7f2dfc108370 2024-03-15T12:49:07.038 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.163+0000 7f2df2ffd700 1 -- 172.21.15.2:0/3564020408 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2df8003a90 con 0x7f2dfc108370 2024-03-15T12:49:07.039 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.163+0000 7f2df2ffd700 1 -- 172.21.15.2:0/3564020408 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2df8019670 con 0x7f2dfc108370 2024-03-15T12:49:07.039 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.163+0000 7f2e03b7b700 1 -- 172.21.15.2:0/3564020408 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dfc19e430 con 0x7f2dfc108370 2024-03-15T12:49:07.039 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.164+0000 7f2df2ffd700 1 -- 172.21.15.2:0/3564020408 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f2df801aa60 con 0x7f2dfc108370 2024-03-15T12:49:07.039 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.164+0000 7f2e03b7b700 1 -- 172.21.15.2:0/3564020408 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dfc10baa0 con 0x7f2dfc108370 2024-03-15T12:49:07.039 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.164+0000 7f2df2ffd700 1 --2- 172.21.15.2:0/3564020408 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f2de8038c60 0x7f2de803b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:07.039 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.165+0000 7f2df2ffd700 1 -- 172.21.15.2:0/3564020408 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2df804cea0 con 0x7f2dfc108370 2024-03-15T12:49:07.039 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.165+0000 7f2e01116700 1 --2- 172.21.15.2:0/3564020408 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f2de8038c60 0x7f2de803b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:07.039 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.165+0000 7f2e01116700 1 --2- 172.21.15.2:0/3564020408 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f2de8038c60 0x7f2de803b120 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2dec006fd0 tx=0x7f2dec006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:07.039 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.170+0000 7f2df2ffd700 1 -- 172.21.15.2:0/3564020408 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2df8018350 con 0x7f2dfc108370 2024-03-15T12:49:07.039 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.387+0000 7f2e03b7b700 1 -- 172.21.15.2:0/3564020408 --> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2dfc102f90 con 0x7f2de8038c60 2024-03-15T12:49:07.039 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.391+0000 7f2df2ffd700 1 -- 172.21.15.2:0/3564020408 <== mgr.14118 v2:172.21.15.2:6800/3269509343 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2dfc102f90 con 0x7f2de8038c60 2024-03-15T12:49:07.039 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.393+0000 7f2e03b7b700 1 -- 172.21.15.2:0/3564020408 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f2de8038c60 msgr2=0x7f2de803b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:07.040 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.393+0000 7f2e03b7b700 1 --2- 172.21.15.2:0/3564020408 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f2de8038c60 0x7f2de803b120 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2dec006fd0 tx=0x7f2dec006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:07.040 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.393+0000 7f2e03b7b700 1 -- 172.21.15.2:0/3564020408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dfc108370 msgr2=0x7f2dfc19d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:07.040 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.393+0000 7f2e03b7b700 1 --2- 172.21.15.2:0/3564020408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dfc108370 0x7f2dfc19d850 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2df8006af0 tx=0x7f2df8011b70 comp rx=0 tx=0).stop 2024-03-15T12:49:07.040 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.394+0000 7f2e03b7b700 1 -- 172.21.15.2:0/3564020408 shutdown_connections 2024-03-15T12:49:07.040 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.394+0000 7f2e03b7b700 1 --2- 172.21.15.2:0/3564020408 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f2de8038c60 0x7f2de803b120 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:07.040 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.394+0000 7f2e03b7b700 1 --2- 172.21.15.2:0/3564020408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dfc108370 0x7f2dfc19d850 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:07.040 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.394+0000 7f2e03b7b700 1 -- 172.21.15.2:0/3564020408 >> 172.21.15.2:0/3564020408 conn(0x7f2dfc101e00 msgr2=0x7f2dfc102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:07.040 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.394+0000 7f2e03b7b700 1 -- 172.21.15.2:0/3564020408 shutdown_connections 2024-03-15T12:49:07.040 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:06.394+0000 7f2e03b7b700 1 -- 172.21.15.2:0/3564020408 wait complete. 2024-03-15T12:49:07.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:07 smithi002 ceph-mon[32060]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:49:07.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:07 smithi002 ceph-mon[32060]: Saving service mgr spec with placement count:2 2024-03-15T12:49:07.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:07 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' 2024-03-15T12:49:07.730 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:07 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' 2024-03-15T12:49:07.730 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:07 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' 2024-03-15T12:49:08.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.515+0000 7f7b0fa6b700 1 Processor -- start 2024-03-15T12:49:08.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.515+0000 7f7b0fa6b700 1 -- start start 2024-03-15T12:49:08.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.515+0000 7f7b0fa6b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b08108380 0x7f7b0810a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:08.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.515+0000 7f7b0fa6b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0807ae80 con 0x7f7b08108380 2024-03-15T12:49:08.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.516+0000 7f7b0d807700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b08108380 0x7f7b0810a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:08.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.516+0000 7f7b0d807700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b08108380 0x7f7b0810a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48722/0 (socket says 172.21.15.2:48722) 2024-03-15T12:49:08.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.516+0000 7f7b0d807700 1 -- 172.21.15.2:0/139186455 learned_addr learned my addr 172.21.15.2:0/139186455 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:08.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.516+0000 7f7b0d807700 1 -- 172.21.15.2:0/139186455 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0810acb0 con 0x7f7b08108380 2024-03-15T12:49:08.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.516+0000 7f7b0d807700 1 --2- 172.21.15.2:0/139186455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b08108380 0x7f7b0810a770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7af80098a0 tx=0x7f7af8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252c08fc47543645 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:08.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.516+0000 7f7b0c805700 1 -- 172.21.15.2:0/139186455 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7af8004160 con 0x7f7b08108380 2024-03-15T12:49:08.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.516+0000 7f7b0c805700 1 -- 172.21.15.2:0/139186455 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7af80042c0 con 0x7f7b08108380 2024-03-15T12:49:08.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.516+0000 7f7b0c805700 1 -- 172.21.15.2:0/139186455 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7af8011a30 con 0x7f7b08108380 2024-03-15T12:49:08.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.517+0000 7f7b0fa6b700 1 -- 172.21.15.2:0/139186455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b08108380 msgr2=0x7f7b0810a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:08.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.517+0000 7f7b0fa6b700 1 --2- 172.21.15.2:0/139186455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b08108380 0x7f7b0810a770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7af80098a0 tx=0x7f7af8009bb0 comp rx=0 tx=0).stop 2024-03-15T12:49:08.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.517+0000 7f7b0fa6b700 1 -- 172.21.15.2:0/139186455 shutdown_connections 2024-03-15T12:49:08.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.517+0000 7f7b0fa6b700 1 --2- 172.21.15.2:0/139186455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b08108380 0x7f7b0810a770 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:08.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.517+0000 7f7b0fa6b700 1 -- 172.21.15.2:0/139186455 >> 172.21.15.2:0/139186455 conn(0x7f7b08101e50 msgr2=0x7f7b08104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:08.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.517+0000 7f7b0fa6b700 1 -- 172.21.15.2:0/139186455 shutdown_connections 2024-03-15T12:49:08.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.517+0000 7f7b0fa6b700 1 -- 172.21.15.2:0/139186455 wait complete. 2024-03-15T12:49:08.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.518+0000 7f7b0fa6b700 1 Processor -- start 2024-03-15T12:49:08.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.518+0000 7f7b0fa6b700 1 -- start start 2024-03-15T12:49:08.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.518+0000 7f7b0fa6b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b08108380 0x7f7b0819b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:08.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.518+0000 7f7b0fa6b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0819bbc0 con 0x7f7b08108380 2024-03-15T12:49:08.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.519+0000 7f7b0d807700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b08108380 0x7f7b0819b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:08.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.519+0000 7f7b0d807700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b08108380 0x7f7b0819b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48728/0 (socket says 172.21.15.2:48728) 2024-03-15T12:49:08.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.519+0000 7f7b0d807700 1 -- 172.21.15.2:0/1282015498 learned_addr learned my addr 172.21.15.2:0/1282015498 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:08.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.519+0000 7f7b0d807700 1 -- 172.21.15.2:0/1282015498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af8009550 con 0x7f7b08108380 2024-03-15T12:49:08.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.519+0000 7f7b0d807700 1 --2- 172.21.15.2:0/1282015498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b08108380 0x7f7b0819b680 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7af8011670 tx=0x7f7af8011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:08.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.519+0000 7f7afeffd700 1 -- 172.21.15.2:0/1282015498 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7af8003a80 con 0x7f7b08108380 2024-03-15T12:49:08.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.519+0000 7f7afeffd700 1 -- 172.21.15.2:0/1282015498 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7af8003be0 con 0x7f7b08108380 2024-03-15T12:49:08.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.519+0000 7f7b0fa6b700 1 -- 172.21.15.2:0/1282015498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0819bdc0 con 0x7f7b08108380 2024-03-15T12:49:08.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.519+0000 7f7afeffd700 1 -- 172.21.15.2:0/1282015498 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7af8019730 con 0x7f7b08108380 2024-03-15T12:49:08.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.519+0000 7f7b0fa6b700 1 -- 172.21.15.2:0/1282015498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0819c260 con 0x7f7b08108380 2024-03-15T12:49:08.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.520+0000 7f7afeffd700 1 -- 172.21.15.2:0/1282015498 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f7af80199a0 con 0x7f7b08108380 2024-03-15T12:49:08.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.521+0000 7f7b0fa6b700 1 -- 172.21.15.2:0/1282015498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0810baf0 con 0x7f7b08108380 2024-03-15T12:49:08.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.521+0000 7f7afeffd700 1 --2- 172.21.15.2:0/1282015498 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7af4038cb0 0x7f7af403b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:08.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.521+0000 7f7afeffd700 1 -- 172.21.15.2:0/1282015498 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7af804ce40 con 0x7f7b08108380 2024-03-15T12:49:08.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.521+0000 7f7b0d006700 1 --2- 172.21.15.2:0/1282015498 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7af4038cb0 0x7f7af403b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:08.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.521+0000 7f7b0d006700 1 --2- 172.21.15.2:0/1282015498 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7af4038cb0 0x7f7af403b170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7b04006fd0 tx=0x7f7b04006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:08.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.526+0000 7f7afeffd700 1 -- 172.21.15.2:0/1282015498 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7af8018350 con 0x7f7b08108380 2024-03-15T12:49:08.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.742+0000 7f7b0fa6b700 1 -- 172.21.15.2:0/1282015498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f7b0819eba0 con 0x7f7b08108380 2024-03-15T12:49:08.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.744+0000 7f7afeffd700 1 -- 172.21.15.2:0/1282015498 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f7af8018350 con 0x7f7b08108380 2024-03-15T12:49:08.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.748+0000 7f7b0fa6b700 1 -- 172.21.15.2:0/1282015498 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7af4038cb0 msgr2=0x7f7af403b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:08.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.748+0000 7f7b0fa6b700 1 --2- 172.21.15.2:0/1282015498 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7af4038cb0 0x7f7af403b170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7b04006fd0 tx=0x7f7b04006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:08.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.748+0000 7f7b0fa6b700 1 -- 172.21.15.2:0/1282015498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b08108380 msgr2=0x7f7b0819b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:08.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.748+0000 7f7b0fa6b700 1 --2- 172.21.15.2:0/1282015498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b08108380 0x7f7b0819b680 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7af8011670 tx=0x7f7af8011b20 comp rx=0 tx=0).stop 2024-03-15T12:49:08.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.748+0000 7f7b0fa6b700 1 -- 172.21.15.2:0/1282015498 shutdown_connections 2024-03-15T12:49:08.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.748+0000 7f7b0fa6b700 1 --2- 172.21.15.2:0/1282015498 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7af4038cb0 0x7f7af403b170 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:08.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.748+0000 7f7b0fa6b700 1 --2- 172.21.15.2:0/1282015498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b08108380 0x7f7b0819b680 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:08.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.748+0000 7f7b0fa6b700 1 -- 172.21.15.2:0/1282015498 >> 172.21.15.2:0/1282015498 conn(0x7f7b08101e50 msgr2=0x7f7b08102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:08.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.748+0000 7f7b0fa6b700 1 -- 172.21.15.2:0/1282015498 shutdown_connections 2024-03-15T12:49:08.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:07.748+0000 7f7b0fa6b700 1 -- 172.21.15.2:0/1282015498 wait complete. 2024-03-15T12:49:08.950 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:08 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1282015498' entity='client.admin' 2024-03-15T12:49:08.950 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:08 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' 2024-03-15T12:49:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.384+0000 7f6710c49700 1 Processor -- start 2024-03-15T12:49:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.384+0000 7f6710c49700 1 -- start start 2024-03-15T12:49:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.385+0000 7f6710c49700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f670c108320 0x7f670c10a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.385+0000 7f6710c49700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c07aec0 con 0x7f670c108320 2024-03-15T12:49:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.385+0000 7f670a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f670c108320 0x7f670c10a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.385+0000 7f670a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f670c108320 0x7f670c10a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41128/0 (socket says 172.21.15.2:41128) 2024-03-15T12:49:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.385+0000 7f670a59c700 1 -- 172.21.15.2:0/29358779 learned_addr learned my addr 172.21.15.2:0/29358779 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.385+0000 7f670a59c700 1 -- 172.21.15.2:0/29358779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c10ac50 con 0x7f670c108320 2024-03-15T12:49:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.386+0000 7f670a59c700 1 --2- 172.21.15.2:0/29358779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f670c108320 0x7f670c10a710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0098a0 tx=0x7f66fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f163e0b5e41b0b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.386+0000 7f670959a700 1 -- 172.21.15.2:0/29358779 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66fc004160 con 0x7f670c108320 2024-03-15T12:49:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.386+0000 7f670959a700 1 -- 172.21.15.2:0/29358779 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f66fc0042c0 con 0x7f670c108320 2024-03-15T12:49:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.386+0000 7f670959a700 1 -- 172.21.15.2:0/29358779 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66fc011a30 con 0x7f670c108320 2024-03-15T12:49:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.386+0000 7f6710c49700 1 -- 172.21.15.2:0/29358779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f670c108320 msgr2=0x7f670c10a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.386+0000 7f6710c49700 1 --2- 172.21.15.2:0/29358779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f670c108320 0x7f670c10a710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0098a0 tx=0x7f66fc009bb0 comp rx=0 tx=0).stop 2024-03-15T12:49:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.387+0000 7f6710c49700 1 -- 172.21.15.2:0/29358779 shutdown_connections 2024-03-15T12:49:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.387+0000 7f6710c49700 1 --2- 172.21.15.2:0/29358779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f670c108320 0x7f670c10a710 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.387+0000 7f6710c49700 1 -- 172.21.15.2:0/29358779 >> 172.21.15.2:0/29358779 conn(0x7f670c101d70 msgr2=0x7f670c104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.387+0000 7f6710c49700 1 -- 172.21.15.2:0/29358779 shutdown_connections 2024-03-15T12:49:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.387+0000 7f6710c49700 1 -- 172.21.15.2:0/29358779 wait complete. 2024-03-15T12:49:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.387+0000 7f6710c49700 1 Processor -- start 2024-03-15T12:49:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.387+0000 7f6710c49700 1 -- start start 2024-03-15T12:49:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.388+0000 7f6710c49700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f670c108320 0x7f670c19b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.388+0000 7f6710c49700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c19bbf0 con 0x7f670c108320 2024-03-15T12:49:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.388+0000 7f670a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f670c108320 0x7f670c19b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.388+0000 7f670a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f670c108320 0x7f670c19b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41140/0 (socket says 172.21.15.2:41140) 2024-03-15T12:49:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.388+0000 7f670a59c700 1 -- 172.21.15.2:0/3836950432 learned_addr learned my addr 172.21.15.2:0/3836950432 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.388+0000 7f670a59c700 1 -- 172.21.15.2:0/3836950432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc009550 con 0x7f670c108320 2024-03-15T12:49:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.388+0000 7f670a59c700 1 --2- 172.21.15.2:0/3836950432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f670c108320 0x7f670c19b6b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f66fc011670 tx=0x7f66fc011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.388+0000 7f67037fe700 1 -- 172.21.15.2:0/3836950432 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66fc003a80 con 0x7f670c108320 2024-03-15T12:49:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.388+0000 7f67037fe700 1 -- 172.21.15.2:0/3836950432 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f66fc003be0 con 0x7f670c108320 2024-03-15T12:49:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.388+0000 7f6710c49700 1 -- 172.21.15.2:0/3836950432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670c19bdf0 con 0x7f670c108320 2024-03-15T12:49:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.388+0000 7f67037fe700 1 -- 172.21.15.2:0/3836950432 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66fc019730 con 0x7f670c108320 2024-03-15T12:49:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.388+0000 7f6710c49700 1 -- 172.21.15.2:0/3836950432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670c19c290 con 0x7f670c108320 2024-03-15T12:49:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.389+0000 7f67037fe700 1 -- 172.21.15.2:0/3836950432 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f66fc0199f0 con 0x7f670c108320 2024-03-15T12:49:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.389+0000 7f6710c49700 1 -- 172.21.15.2:0/3836950432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670c10baf0 con 0x7f670c108320 2024-03-15T12:49:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.389+0000 7f67037fe700 1 --2- 172.21.15.2:0/3836950432 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f66f8038c60 0x7f66f803b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.390+0000 7f67037fe700 1 -- 172.21.15.2:0/3836950432 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f66fc04cf00 con 0x7f670c108320 2024-03-15T12:49:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.390+0000 7f6709d9b700 1 --2- 172.21.15.2:0/3836950432 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f66f8038c60 0x7f66f803b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.390+0000 7f6709d9b700 1 --2- 172.21.15.2:0/3836950432 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f66f8038c60 0x7f66f803b120 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f66f4006fd0 tx=0x7f66f4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.394+0000 7f67037fe700 1 -- 172.21.15.2:0/3836950432 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f66fc02b750 con 0x7f670c108320 2024-03-15T12:49:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.615+0000 7f6710c49700 1 -- 172.21.15.2:0/3836950432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f670c19eb70 con 0x7f670c108320 2024-03-15T12:49:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.618+0000 7f67037fe700 1 -- 172.21.15.2:0/3836950432 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f66fc02b340 con 0x7f670c108320 2024-03-15T12:49:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.620+0000 7f6710c49700 1 -- 172.21.15.2:0/3836950432 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f66f8038c60 msgr2=0x7f66f803b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.620+0000 7f6710c49700 1 --2- 172.21.15.2:0/3836950432 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f66f8038c60 0x7f66f803b120 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f66f4006fd0 tx=0x7f66f4006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.620+0000 7f6710c49700 1 -- 172.21.15.2:0/3836950432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f670c108320 msgr2=0x7f670c19b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.620+0000 7f6710c49700 1 --2- 172.21.15.2:0/3836950432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f670c108320 0x7f670c19b6b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f66fc011670 tx=0x7f66fc011b20 comp rx=0 tx=0).stop 2024-03-15T12:49:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.620+0000 7f6710c49700 1 -- 172.21.15.2:0/3836950432 shutdown_connections 2024-03-15T12:49:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.620+0000 7f6710c49700 1 --2- 172.21.15.2:0/3836950432 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f66f8038c60 0x7f66f803b120 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.620+0000 7f6710c49700 1 --2- 172.21.15.2:0/3836950432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f670c108320 0x7f670c19b6b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.620+0000 7f6710c49700 1 -- 172.21.15.2:0/3836950432 >> 172.21.15.2:0/3836950432 conn(0x7f670c101d70 msgr2=0x7f670c1028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.620+0000 7f6710c49700 1 -- 172.21.15.2:0/3836950432 shutdown_connections 2024-03-15T12:49:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:09.620+0000 7f6710c49700 1 -- 172.21.15.2:0/3836950432 wait complete. 2024-03-15T12:49:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling the dashboard module... 2024-03-15T12:49:10.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:10 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3836950432' entity='client.admin' 2024-03-15T12:49:10.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:10 smithi002 ceph-mon[32060]: from='mgr.14118 172.21.15.2:0/4015038801' entity='mgr.x' 2024-03-15T12:49:11.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:11 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1161072078' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-15T12:49:11.805 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:11 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: ignoring --setuser ceph since I am not root 2024-03-15T12:49:11.805 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:11 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: ignoring --setgroup ceph since I am not root 2024-03-15T12:49:12.231 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.595+0000 7f7fe8d51700 1 Processor -- start 2024-03-15T12:49:12.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.595+0000 7f7fe8d51700 1 -- start start 2024-03-15T12:49:12.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.595+0000 7f7fe8d51700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe41078e0 0x7f7fe4107d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:12.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.596+0000 7f7fe8d51700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe41082d0 con 0x7f7fe41078e0 2024-03-15T12:49:12.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.596+0000 7f7fe259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe41078e0 0x7f7fe4107d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:12.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.596+0000 7f7fe259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe41078e0 0x7f7fe4107d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41150/0 (socket says 172.21.15.2:41150) 2024-03-15T12:49:12.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.596+0000 7f7fe259c700 1 -- 172.21.15.2:0/1037445235 learned_addr learned my addr 172.21.15.2:0/1037445235 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:12.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.596+0000 7f7fe259c700 1 -- 172.21.15.2:0/1037445235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe4108a90 con 0x7f7fe41078e0 2024-03-15T12:49:12.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.596+0000 7f7fe259c700 1 --2- 172.21.15.2:0/1037445235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe41078e0 0x7f7fe4107d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40098a0 tx=0x7f7fd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d359b24cfe2e5a8a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:12.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.596+0000 7f7fe159a700 1 -- 172.21.15.2:0/1037445235 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fd4004160 con 0x7f7fe41078e0 2024-03-15T12:49:12.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.596+0000 7f7fe159a700 1 -- 172.21.15.2:0/1037445235 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7fd40042c0 con 0x7f7fe41078e0 2024-03-15T12:49:12.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.596+0000 7f7fe159a700 1 -- 172.21.15.2:0/1037445235 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fd4011a30 con 0x7f7fe41078e0 2024-03-15T12:49:12.232 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.597+0000 7f7fe8d51700 1 -- 172.21.15.2:0/1037445235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe41078e0 msgr2=0x7f7fe4107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:12.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.597+0000 7f7fe8d51700 1 --2- 172.21.15.2:0/1037445235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe41078e0 0x7f7fe4107d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40098a0 tx=0x7f7fd4009bb0 comp rx=0 tx=0).stop 2024-03-15T12:49:12.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.597+0000 7f7fe8d51700 1 -- 172.21.15.2:0/1037445235 shutdown_connections 2024-03-15T12:49:12.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.597+0000 7f7fe8d51700 1 --2- 172.21.15.2:0/1037445235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe41078e0 0x7f7fe4107d00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:12.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.597+0000 7f7fe8d51700 1 -- 172.21.15.2:0/1037445235 >> 172.21.15.2:0/1037445235 conn(0x7f7fe4102f00 msgr2=0x7f7fe4105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:12.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.597+0000 7f7fe8d51700 1 -- 172.21.15.2:0/1037445235 shutdown_connections 2024-03-15T12:49:12.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.597+0000 7f7fe8d51700 1 -- 172.21.15.2:0/1037445235 wait complete. 2024-03-15T12:49:12.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.598+0000 7f7fe8d51700 1 Processor -- start 2024-03-15T12:49:12.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.598+0000 7f7fe8d51700 1 -- start start 2024-03-15T12:49:12.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.598+0000 7f7fe8d51700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe41078e0 0x7f7fe407cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:12.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.598+0000 7f7fe8d51700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe407d0c0 con 0x7f7fe41078e0 2024-03-15T12:49:12.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.598+0000 7f7fe259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe41078e0 0x7f7fe407cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:12.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.598+0000 7f7fe259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe41078e0 0x7f7fe407cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41166/0 (socket says 172.21.15.2:41166) 2024-03-15T12:49:12.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.598+0000 7f7fe259c700 1 -- 172.21.15.2:0/1161072078 learned_addr learned my addr 172.21.15.2:0/1161072078 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:12.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.598+0000 7f7fe259c700 1 -- 172.21.15.2:0/1161072078 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd4009550 con 0x7f7fe41078e0 2024-03-15T12:49:12.233 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.598+0000 7f7fe259c700 1 --2- 172.21.15.2:0/1161072078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe41078e0 0x7f7fe407cb80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4011780 tx=0x7f7fd4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:12.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.598+0000 7f7fd37fe700 1 -- 172.21.15.2:0/1161072078 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fd4003970 con 0x7f7fe41078e0 2024-03-15T12:49:12.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.598+0000 7f7fd37fe700 1 -- 172.21.15.2:0/1161072078 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7fd4003ad0 con 0x7f7fe41078e0 2024-03-15T12:49:12.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.598+0000 7f7fe8d51700 1 -- 172.21.15.2:0/1161072078 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe407b2b0 con 0x7f7fe41078e0 2024-03-15T12:49:12.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.598+0000 7f7fd37fe700 1 -- 172.21.15.2:0/1161072078 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fd40196c0 con 0x7f7fe41078e0 2024-03-15T12:49:12.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.599+0000 7f7fe8d51700 1 -- 172.21.15.2:0/1161072078 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe407b750 con 0x7f7fe41078e0 2024-03-15T12:49:12.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.599+0000 7f7fd37fe700 1 -- 172.21.15.2:0/1161072078 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f7fd4019940 con 0x7f7fe41078e0 2024-03-15T12:49:12.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.599+0000 7f7fe8d51700 1 -- 172.21.15.2:0/1161072078 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe410bcc0 con 0x7f7fe41078e0 2024-03-15T12:49:12.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.600+0000 7f7fd37fe700 1 --2- 172.21.15.2:0/1161072078 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7fcc038c60 0x7f7fcc03b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:12.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.600+0000 7f7fd37fe700 1 -- 172.21.15.2:0/1161072078 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7fd404d010 con 0x7f7fe41078e0 2024-03-15T12:49:12.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.600+0000 7f7fe1d9b700 1 --2- 172.21.15.2:0/1161072078 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7fcc038c60 0x7f7fcc03b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:12.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.600+0000 7f7fe1d9b700 1 --2- 172.21.15.2:0/1161072078 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7fcc038c60 0x7f7fcc03b120 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8006fd0 tx=0x7f7fd8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:12.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.605+0000 7f7fd37fe700 1 -- 172.21.15.2:0/1161072078 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7fd4018dc0 con 0x7f7fe41078e0 2024-03-15T12:49:12.234 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:10.881+0000 7f7fe8d51700 1 -- 172.21.15.2:0/1161072078 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f7fe407bd60 con 0x7f7fe41078e0 2024-03-15T12:49:12.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:11.621+0000 7f7fd37fe700 1 -- 172.21.15.2:0/1161072078 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f7fd4018be0 con 0x7f7fe41078e0 2024-03-15T12:49:12.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:11.623+0000 7f7fd37fe700 1 -- 172.21.15.2:0/1161072078 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f7fd402d960 con 0x7f7fe41078e0 2024-03-15T12:49:12.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:11.624+0000 7f7fe8d51700 1 -- 172.21.15.2:0/1161072078 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7fcc038c60 msgr2=0x7f7fcc03b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:12.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:11.624+0000 7f7fe8d51700 1 --2- 172.21.15.2:0/1161072078 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7fcc038c60 0x7f7fcc03b120 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8006fd0 tx=0x7f7fd8006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:12.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:11.624+0000 7f7fe8d51700 1 -- 172.21.15.2:0/1161072078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe41078e0 msgr2=0x7f7fe407cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:12.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:11.624+0000 7f7fe8d51700 1 --2- 172.21.15.2:0/1161072078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe41078e0 0x7f7fe407cb80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4011780 tx=0x7f7fd4011b40 comp rx=0 tx=0).stop 2024-03-15T12:49:12.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:11.624+0000 7f7fe8d51700 1 -- 172.21.15.2:0/1161072078 shutdown_connections 2024-03-15T12:49:12.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:11.624+0000 7f7fe8d51700 1 --2- 172.21.15.2:0/1161072078 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7fcc038c60 0x7f7fcc03b120 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:12.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:11.624+0000 7f7fe8d51700 1 --2- 172.21.15.2:0/1161072078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe41078e0 0x7f7fe407cb80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:12.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:11.624+0000 7f7fe8d51700 1 -- 172.21.15.2:0/1161072078 >> 172.21.15.2:0/1161072078 conn(0x7f7fe4102f00 msgr2=0x7f7fe4103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:12.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:11.624+0000 7f7fe8d51700 1 -- 172.21.15.2:0/1161072078 shutdown_connections 2024-03-15T12:49:12.235 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:11.624+0000 7f7fe8d51700 1 -- 172.21.15.2:0/1161072078 wait complete. 2024-03-15T12:49:12.280 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:12 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:12.210+0000 7fa4c3f29200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-15T12:49:12.554 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:12 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:12.319+0000 7fa4c3f29200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-15T12:49:12.909 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:12 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1161072078' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-15T12:49:12.909 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:12 smithi002 ceph-mon[32060]: mgrmap e9: x(active, since 19s) 2024-03-15T12:49:13.229 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:12 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:12.992+0000 7fa4c3f29200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-15T12:49:13.229 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:13 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:13.097+0000 7fa4c3f29200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-15T12:49:13.229 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:13 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:13.241+0000 7fa4c3f29200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-15T12:49:13.506 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:13 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:13.340+0000 7fa4c3f29200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-15T12:49:13.506 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:13 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:13.426+0000 7fa4c3f29200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-15T12:49:13.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-03-15T12:49:13.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-15T12:49:13.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-15T12:49:13.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-15T12:49:13.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-15T12:49:13.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-03-15T12:49:13.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.717+0000 7f58f68ce700 1 Processor -- start 2024-03-15T12:49:13.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.717+0000 7f58f68ce700 1 -- start start 2024-03-15T12:49:13.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.717+0000 7f58f68ce700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58f01067b0 0x7f58f0106bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:13.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.717+0000 7f58f68ce700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f01071a0 con 0x7f58f01067b0 2024-03-15T12:49:13.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.717+0000 7f58effff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58f01067b0 0x7f58f0106bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:13.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.717+0000 7f58effff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58f01067b0 0x7f58f0106bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41184/0 (socket says 172.21.15.2:41184) 2024-03-15T12:49:13.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.717+0000 7f58effff700 1 -- 172.21.15.2:0/359397452 learned_addr learned my addr 172.21.15.2:0/359397452 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:13.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.718+0000 7f58effff700 1 -- 172.21.15.2:0/359397452 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f01079b0 con 0x7f58f01067b0 2024-03-15T12:49:13.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.718+0000 7f58effff700 1 --2- 172.21.15.2:0/359397452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58f01067b0 0x7f58f0106bd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58e00098a0 tx=0x7f58e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00be245b83c5e7b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:13.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.718+0000 7f58eeffd700 1 -- 172.21.15.2:0/359397452 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58e0004160 con 0x7f58f01067b0 2024-03-15T12:49:13.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.718+0000 7f58eeffd700 1 -- 172.21.15.2:0/359397452 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f58e00042c0 con 0x7f58f01067b0 2024-03-15T12:49:13.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.718+0000 7f58eeffd700 1 -- 172.21.15.2:0/359397452 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58e0011a30 con 0x7f58f01067b0 2024-03-15T12:49:13.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.719+0000 7f58f68ce700 1 -- 172.21.15.2:0/359397452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58f01067b0 msgr2=0x7f58f0106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:13.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.719+0000 7f58f68ce700 1 --2- 172.21.15.2:0/359397452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58f01067b0 0x7f58f0106bd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58e00098a0 tx=0x7f58e0009bb0 comp rx=0 tx=0).stop 2024-03-15T12:49:13.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.719+0000 7f58f68ce700 1 -- 172.21.15.2:0/359397452 shutdown_connections 2024-03-15T12:49:13.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.719+0000 7f58f68ce700 1 --2- 172.21.15.2:0/359397452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58f01067b0 0x7f58f0106bd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:13.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.719+0000 7f58f68ce700 1 -- 172.21.15.2:0/359397452 >> 172.21.15.2:0/359397452 conn(0x7f58f0101e10 msgr2=0x7f58f0104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:13.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.719+0000 7f58f68ce700 1 -- 172.21.15.2:0/359397452 shutdown_connections 2024-03-15T12:49:13.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.719+0000 7f58f68ce700 1 -- 172.21.15.2:0/359397452 wait complete. 2024-03-15T12:49:13.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.719+0000 7f58f68ce700 1 Processor -- start 2024-03-15T12:49:13.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.719+0000 7f58f68ce700 1 -- start start 2024-03-15T12:49:13.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.720+0000 7f58f68ce700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58f01067b0 0x7f58f019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:13.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.720+0000 7f58f68ce700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f019def0 con 0x7f58f01067b0 2024-03-15T12:49:13.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.720+0000 7f58effff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58f01067b0 0x7f58f019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:13.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.720+0000 7f58effff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58f01067b0 0x7f58f019d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41190/0 (socket says 172.21.15.2:41190) 2024-03-15T12:49:13.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.720+0000 7f58effff700 1 -- 172.21.15.2:0/3729121205 learned_addr learned my addr 172.21.15.2:0/3729121205 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:13.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.720+0000 7f58effff700 1 -- 172.21.15.2:0/3729121205 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e0009550 con 0x7f58f01067b0 2024-03-15T12:49:13.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.720+0000 7f58effff700 1 --2- 172.21.15.2:0/3729121205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58f01067b0 0x7f58f019d9b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f58e0006af0 tx=0x7f58e0004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:13.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.720+0000 7f58ed7fa700 1 -- 172.21.15.2:0/3729121205 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58e00039e0 con 0x7f58f01067b0 2024-03-15T12:49:13.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.720+0000 7f58ed7fa700 1 -- 172.21.15.2:0/3729121205 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f58e0003b40 con 0x7f58f01067b0 2024-03-15T12:49:13.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.720+0000 7f58f68ce700 1 -- 172.21.15.2:0/3729121205 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f019e0f0 con 0x7f58f01067b0 2024-03-15T12:49:13.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.720+0000 7f58ed7fa700 1 -- 172.21.15.2:0/3729121205 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58e00246c0 con 0x7f58f01067b0 2024-03-15T12:49:13.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.720+0000 7f58f68ce700 1 -- 172.21.15.2:0/3729121205 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f019e590 con 0x7f58f01067b0 2024-03-15T12:49:13.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.721+0000 7f58ed7fa700 1 -- 172.21.15.2:0/3729121205 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f58e002d3f0 con 0x7f58f01067b0 2024-03-15T12:49:13.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.721+0000 7f58f68ce700 1 -- 172.21.15.2:0/3729121205 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f0197c50 con 0x7f58f01067b0 2024-03-15T12:49:13.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.721+0000 7f58ed7fa700 1 --2- 172.21.15.2:0/3729121205 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f58d8038d00 0x7f58d803b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:13.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.721+0000 7f58ed7fa700 1 -- 172.21.15.2:0/3729121205 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f58e004cf80 con 0x7f58f01067b0 2024-03-15T12:49:13.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.721+0000 7f58ef7fe700 1 -- 172.21.15.2:0/3729121205 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f58d8038d00 msgr2=0x7f58d803b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3269509343 2024-03-15T12:49:13.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.721+0000 7f58ef7fe700 1 --2- 172.21.15.2:0/3729121205 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f58d8038d00 0x7f58d803b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-15T12:49:13.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.731+0000 7f58ed7fa700 1 -- 172.21.15.2:0/3729121205 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f58e0081280 con 0x7f58f01067b0 2024-03-15T12:49:13.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.922+0000 7f58ef7fe700 1 -- 172.21.15.2:0/3729121205 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f58d8038d00 msgr2=0x7f58d803b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3269509343 2024-03-15T12:49:13.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.922+0000 7f58ef7fe700 1 --2- 172.21.15.2:0/3729121205 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f58d8038d00 0x7f58d803b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-15T12:49:13.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.989+0000 7f58f68ce700 1 -- 172.21.15.2:0/3729121205 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f58f002db20 con 0x7f58f01067b0 2024-03-15T12:49:13.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.990+0000 7f58ed7fa700 1 -- 172.21.15.2:0/3729121205 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f58e0010740 con 0x7f58f01067b0 2024-03-15T12:49:13.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.992+0000 7f58f68ce700 1 -- 172.21.15.2:0/3729121205 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f58d8038d00 msgr2=0x7f58d803b1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-15T12:49:13.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.992+0000 7f58f68ce700 1 --2- 172.21.15.2:0/3729121205 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f58d8038d00 0x7f58d803b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:13.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.992+0000 7f58f68ce700 1 -- 172.21.15.2:0/3729121205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58f01067b0 msgr2=0x7f58f019d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:13.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.992+0000 7f58f68ce700 1 --2- 172.21.15.2:0/3729121205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58f01067b0 0x7f58f019d9b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f58e0006af0 tx=0x7f58e0004710 comp rx=0 tx=0).stop 2024-03-15T12:49:13.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.992+0000 7f58f68ce700 1 -- 172.21.15.2:0/3729121205 shutdown_connections 2024-03-15T12:49:13.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.992+0000 7f58f68ce700 1 --2- 172.21.15.2:0/3729121205 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f58d8038d00 0x7f58d803b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:13.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.992+0000 7f58f68ce700 1 --2- 172.21.15.2:0/3729121205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58f01067b0 0x7f58f019d9b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:13.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.992+0000 7f58f68ce700 1 -- 172.21.15.2:0/3729121205 >> 172.21.15.2:0/3729121205 conn(0x7f58f0101e10 msgr2=0x7f58f0102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:13.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.992+0000 7f58f68ce700 1 -- 172.21.15.2:0/3729121205 shutdown_connections 2024-03-15T12:49:13.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:12.992+0000 7f58f68ce700 1 -- 172.21.15.2:0/3729121205 wait complete. 2024-03-15T12:49:13.517 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for the mgr to restart... 2024-03-15T12:49:13.517 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr epoch 9... 2024-03-15T12:49:13.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:13 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3729121205' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-15T12:49:13.800 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:13 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:13.701+0000 7fa4c3f29200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-15T12:49:14.054 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:13 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:13.815+0000 7fa4c3f29200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-15T12:49:14.054 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:13 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:13.906+0000 7fa4c3f29200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-15T12:49:14.054 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:14 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:14.004+0000 7fa4c3f29200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-15T12:49:14.507 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:14 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:14.183+0000 7fa4c3f29200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-15T12:49:15.554 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:15 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:15.165+0000 7fa4c3f29200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-15T12:49:15.554 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:15 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:15.299+0000 7fa4c3f29200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-15T12:49:15.917 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:15 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:15.637+0000 7fa4c3f29200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-15T12:49:15.917 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:15 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:15.729+0000 7fa4c3f29200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-15T12:49:15.917 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:15 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:15.929+0000 7fa4c3f29200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:49:16.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:16 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:16.119+0000 7fa4c3f29200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-15T12:49:16.661 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:16 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:16.411+0000 7fa4c3f29200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:49:16.661 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:16 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:16.673+0000 7fa4c3f29200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-15T12:49:17.054 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:16 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:16.916+0000 7fa4c3f29200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-15T12:49:18.054 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:17 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:17.740+0000 7fa4c3f29200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-15T12:49:18.632 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:18 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:18.323+0000 7fa4c3f29200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-15T12:49:18.632 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:18 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:18.423+0000 7fa4c3f29200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-15T12:49:18.632 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:49:18 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:49:18.644+0000 7fa4c3f29200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-15T12:49:19.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:18 smithi002 ceph-mon[32060]: Active manager daemon x restarted 2024-03-15T12:49:19.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:18 smithi002 ceph-mon[32060]: Activating manager daemon x 2024-03-15T12:49:19.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:18 smithi002 ceph-mon[32060]: osdmap e3: 0 total, 0 up, 0 in 2024-03-15T12:49:19.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:18 smithi002 ceph-mon[32060]: mgrmap e10: x(active, starting, since 0.00331569s) 2024-03-15T12:49:19.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:49:19.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-15T12:49:19.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-15T12:49:19.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-15T12:49:19.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-15T12:49:19.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:18 smithi002 ceph-mon[32060]: Manager daemon x is now available 2024-03-15T12:49:19.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:49:19.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:49:19.990 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:19 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-15T12:49:19.991 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:19 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-15T12:49:19.991 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:19 smithi002 ceph-mon[32060]: [15/Mar/2024:12:49:18] ENGINE Bus STARTING 2024-03-15T12:49:19.991 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:19 smithi002 ceph-mon[32060]: [15/Mar/2024:12:49:19] ENGINE Serving on https://172.21.15.2:7150 2024-03-15T12:49:19.991 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:19 smithi002 ceph-mon[32060]: [15/Mar/2024:12:49:19] ENGINE Serving on http://172.21.15.2:8765 2024-03-15T12:49:19.991 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:19 smithi002 ceph-mon[32060]: [15/Mar/2024:12:49:19] ENGINE Bus STARTED 2024-03-15T12:49:19.991 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:19 smithi002 ceph-mon[32060]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-15T12:49:19.991 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:19 smithi002 ceph-mon[32060]: mgrmap e11: x(active, since 1.00583s) 2024-03-15T12:49:19.991 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:19 smithi002 ceph-mon[32060]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-15T12:49:20.247 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-03-15T12:49:20.247 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-15T12:49:20.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-15T12:49:20.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-03-15T12:49:20.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.913+0000 7f78a43e8700 1 Processor -- start 2024-03-15T12:49:20.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.913+0000 7f78a43e8700 1 -- start start 2024-03-15T12:49:20.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.914+0000 7f78a43e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f789c1078e0 0x7f789c107d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:20.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.914+0000 7f78a43e8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c1082d0 con 0x7f789c1078e0 2024-03-15T12:49:20.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.914+0000 7f78a2184700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f789c1078e0 0x7f789c107d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:20.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.914+0000 7f78a2184700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f789c1078e0 0x7f789c107d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41204/0 (socket says 172.21.15.2:41204) 2024-03-15T12:49:20.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.914+0000 7f78a2184700 1 -- 172.21.15.2:0/2862741432 learned_addr learned my addr 172.21.15.2:0/2862741432 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:20.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.914+0000 7f78a2184700 1 -- 172.21.15.2:0/2862741432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789c108a90 con 0x7f789c1078e0 2024-03-15T12:49:20.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.914+0000 7f78a2184700 1 --2- 172.21.15.2:0/2862741432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f789c1078e0 0x7f789c107d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f788c0098a0 tx=0x7f788c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e7773686ed27a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:20.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.915+0000 7f78a1182700 1 -- 172.21.15.2:0/2862741432 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f788c004160 con 0x7f789c1078e0 2024-03-15T12:49:20.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.915+0000 7f78a1182700 1 -- 172.21.15.2:0/2862741432 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f788c0042c0 con 0x7f789c1078e0 2024-03-15T12:49:20.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.915+0000 7f78a1182700 1 -- 172.21.15.2:0/2862741432 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f788c011a30 con 0x7f789c1078e0 2024-03-15T12:49:20.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.915+0000 7f78a43e8700 1 -- 172.21.15.2:0/2862741432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f789c1078e0 msgr2=0x7f789c107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:20.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.915+0000 7f78a43e8700 1 --2- 172.21.15.2:0/2862741432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f789c1078e0 0x7f789c107d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f788c0098a0 tx=0x7f788c009bb0 comp rx=0 tx=0).stop 2024-03-15T12:49:20.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.915+0000 7f78a43e8700 1 -- 172.21.15.2:0/2862741432 shutdown_connections 2024-03-15T12:49:20.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.915+0000 7f78a43e8700 1 --2- 172.21.15.2:0/2862741432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f789c1078e0 0x7f789c107d00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:20.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.915+0000 7f78a43e8700 1 -- 172.21.15.2:0/2862741432 >> 172.21.15.2:0/2862741432 conn(0x7f789c102f20 msgr2=0x7f789c105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:20.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.915+0000 7f78a43e8700 1 -- 172.21.15.2:0/2862741432 shutdown_connections 2024-03-15T12:49:20.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.915+0000 7f78a43e8700 1 -- 172.21.15.2:0/2862741432 wait complete. 2024-03-15T12:49:20.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.916+0000 7f78a43e8700 1 Processor -- start 2024-03-15T12:49:20.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.916+0000 7f78a43e8700 1 -- start start 2024-03-15T12:49:20.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.916+0000 7f78a43e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f789c1078e0 0x7f789c19d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:20.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.916+0000 7f78a43e8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c19de10 con 0x7f789c1078e0 2024-03-15T12:49:20.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.916+0000 7f78a2184700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f789c1078e0 0x7f789c19d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:20.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.916+0000 7f78a2184700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f789c1078e0 0x7f789c19d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41208/0 (socket says 172.21.15.2:41208) 2024-03-15T12:49:20.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.916+0000 7f78a2184700 1 -- 172.21.15.2:0/1189052202 learned_addr learned my addr 172.21.15.2:0/1189052202 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:20.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.917+0000 7f78a2184700 1 -- 172.21.15.2:0/1189052202 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c009550 con 0x7f789c1078e0 2024-03-15T12:49:20.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.917+0000 7f78a2184700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f789c1078e0 0x7f789c19d8d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f788c011780 tx=0x7f788c004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:20.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.917+0000 7f78937fe700 1 -- 172.21.15.2:0/1189052202 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f788c0039c0 con 0x7f789c1078e0 2024-03-15T12:49:20.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.917+0000 7f78937fe700 1 -- 172.21.15.2:0/1189052202 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f788c003b20 con 0x7f789c1078e0 2024-03-15T12:49:20.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.917+0000 7f78937fe700 1 -- 172.21.15.2:0/1189052202 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f788c01a690 con 0x7f789c1078e0 2024-03-15T12:49:20.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.917+0000 7f78a43e8700 1 -- 172.21.15.2:0/1189052202 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789c19e010 con 0x7f789c1078e0 2024-03-15T12:49:20.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.917+0000 7f78a43e8700 1 -- 172.21.15.2:0/1189052202 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789c19e4b0 con 0x7f789c1078e0 2024-03-15T12:49:20.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.918+0000 7f78937fe700 1 -- 172.21.15.2:0/1189052202 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f788c024a80 con 0x7f789c1078e0 2024-03-15T12:49:20.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.918+0000 7f78937fe700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7888038d50 0x7f788803b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:20.254 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.918+0000 7f78937fe700 1 -- 172.21.15.2:0/1189052202 --> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f788803b920 con 0x7f7888038d50 2024-03-15T12:49:20.255 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.918+0000 7f78937fe700 1 -- 172.21.15.2:0/1189052202 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f788c04cc60 con 0x7f789c1078e0 2024-03-15T12:49:20.255 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.918+0000 7f78a1983700 1 -- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7888038d50 msgr2=0x7f788803b210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3269509343 2024-03-15T12:49:20.255 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:13.918+0000 7f78a1983700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7888038d50 0x7f788803b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-15T12:49:20.255 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:14.119+0000 7f78a1983700 1 -- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7888038d50 msgr2=0x7f788803b210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3269509343 2024-03-15T12:49:20.255 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:14.119+0000 7f78a1983700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7888038d50 0x7f788803b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-15T12:49:20.255 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:14.520+0000 7f78a1983700 1 -- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7888038d50 msgr2=0x7f788803b210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3269509343 2024-03-15T12:49:20.255 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:14.520+0000 7f78a1983700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7888038d50 0x7f788803b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-15T12:49:20.255 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:15.321+0000 7f78a1983700 1 -- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7888038d50 msgr2=0x7f788803b210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3269509343 2024-03-15T12:49:20.256 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:15.321+0000 7f78a1983700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7888038d50 0x7f788803b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-15T12:49:20.256 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:16.922+0000 7f78a1983700 1 -- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7888038d50 msgr2=0x7f788803b210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3269509343 2024-03-15T12:49:20.256 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:16.922+0000 7f78a1983700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7888038d50 0x7f788803b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-15T12:49:20.256 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:18.659+0000 7f78937fe700 1 -- 172.21.15.2:0/1189052202 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mgrmap(e 10) v1 ==== 45549+0+0 (secure 0 0 0) 0x7f788c024440 con 0x7f789c1078e0 2024-03-15T12:49:20.256 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:18.660+0000 7f78937fe700 1 -- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7888038d50 msgr2=0x7f788803b210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-15T12:49:20.256 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:18.660+0000 7f78937fe700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7888038d50 0x7f788803b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:20.256 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.661+0000 7f78937fe700 1 -- 172.21.15.2:0/1189052202 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 11) v1 ==== 45676+0+0 (secure 0 0 0) 0x7f788c04e750 con 0x7f789c1078e0 2024-03-15T12:49:20.256 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.661+0000 7f78937fe700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f788803c7d0 0x7f788803ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:20.256 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.661+0000 7f78937fe700 1 -- 172.21.15.2:0/1189052202 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f788803b920 con 0x7f788803c7d0 2024-03-15T12:49:20.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.662+0000 7f78a1983700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f788803c7d0 0x7f788803ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:20.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.662+0000 7f78a1983700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f788803c7d0 0x7f788803ebc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7898003a10 tx=0x7f7898009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:20.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.663+0000 7f78937fe700 1 -- 172.21.15.2:0/1189052202 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f788803b920 con 0x7f788803c7d0 2024-03-15T12:49:20.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.676+0000 7f78a43e8700 1 -- 172.21.15.2:0/1189052202 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f789c10bcc0 con 0x7f788803c7d0 2024-03-15T12:49:20.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.676+0000 7f78937fe700 1 -- 172.21.15.2:0/1189052202 <== mgr.14150 v2:172.21.15.2:6800/343748862 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f789c10bcc0 con 0x7f788803c7d0 2024-03-15T12:49:20.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.677+0000 7f78a43e8700 1 -- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f788803c7d0 msgr2=0x7f788803ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:20.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.677+0000 7f78a43e8700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f788803c7d0 0x7f788803ebc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7898003a10 tx=0x7f7898009270 comp rx=0 tx=0).stop 2024-03-15T12:49:20.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.677+0000 7f78a43e8700 1 -- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f789c1078e0 msgr2=0x7f789c19d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:20.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.677+0000 7f78a43e8700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f789c1078e0 0x7f789c19d8d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f788c011780 tx=0x7f788c004710 comp rx=0 tx=0).stop 2024-03-15T12:49:20.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.677+0000 7f78a43e8700 1 -- 172.21.15.2:0/1189052202 shutdown_connections 2024-03-15T12:49:20.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.677+0000 7f78a43e8700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f788803c7d0 0x7f788803ebc0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:20.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.677+0000 7f78a43e8700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f789c1078e0 0x7f789c19d8d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:20.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.677+0000 7f78a43e8700 1 --2- 172.21.15.2:0/1189052202 >> [v2:172.21.15.2:6800/3269509343,v1:172.21.15.2:6801/3269509343] conn(0x7f7888038d50 0x7f788803b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:20.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.677+0000 7f78a43e8700 1 -- 172.21.15.2:0/1189052202 >> 172.21.15.2:0/1189052202 conn(0x7f789c102f20 msgr2=0x7f789c1039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:20.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.677+0000 7f78a43e8700 1 -- 172.21.15.2:0/1189052202 shutdown_connections 2024-03-15T12:49:20.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:19.677+0000 7f78a43e8700 1 -- 172.21.15.2:0/1189052202 wait complete. 2024-03-15T12:49:20.258 INFO:teuthology.orchestra.run.smithi002.stdout:mgr epoch 9 is available 2024-03-15T12:49:20.259 INFO:teuthology.orchestra.run.smithi002.stdout:Generating a dashboard self-signed certificate... 2024-03-15T12:49:21.855 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:21 smithi002 ceph-mon[32060]: mgrmap e12: x(active, since 2s) 2024-03-15T12:49:21.855 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:21.855 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:21 smithi002 ceph-mon[32060]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:49:21.855 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:21.855 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:21.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-15T12:49:21.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.925+0000 7f7abbfff700 1 Processor -- start 2024-03-15T12:49:21.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.925+0000 7f7abbfff700 1 -- start start 2024-03-15T12:49:21.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.925+0000 7f7abbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7abc1081b0 0x7f7abc10a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:21.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.925+0000 7f7abbfff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc07ae80 con 0x7f7abc1081b0 2024-03-15T12:49:21.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.925+0000 7f7abaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7abc1081b0 0x7f7abc10a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:21.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.925+0000 7f7abaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7abc1081b0 0x7f7abc10a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43782/0 (socket says 172.21.15.2:43782) 2024-03-15T12:49:21.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.925+0000 7f7abaffd700 1 -- 172.21.15.2:0/1150923424 learned_addr learned my addr 172.21.15.2:0/1150923424 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:21.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.926+0000 7f7abaffd700 1 -- 172.21.15.2:0/1150923424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc10aae0 con 0x7f7abc1081b0 2024-03-15T12:49:21.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.926+0000 7f7abaffd700 1 --2- 172.21.15.2:0/1150923424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7abc1081b0 0x7f7abc10a5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009e80 tx=0x7f7aac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca517dde2efa5c30 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:21.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.926+0000 7f7ab9ffb700 1 -- 172.21.15.2:0/1150923424 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7aac00e070 con 0x7f7abc1081b0 2024-03-15T12:49:21.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.926+0000 7f7ab9ffb700 1 -- 172.21.15.2:0/1150923424 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7aac00b8c0 con 0x7f7abc1081b0 2024-03-15T12:49:21.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.926+0000 7f7ab9ffb700 1 -- 172.21.15.2:0/1150923424 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7aac0044b0 con 0x7f7abc1081b0 2024-03-15T12:49:21.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.926+0000 7f7abbfff700 1 -- 172.21.15.2:0/1150923424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7abc1081b0 msgr2=0x7f7abc10a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:21.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.926+0000 7f7abbfff700 1 --2- 172.21.15.2:0/1150923424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7abc1081b0 0x7f7abc10a5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009e80 tx=0x7f7aac00b330 comp rx=0 tx=0).stop 2024-03-15T12:49:21.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.927+0000 7f7abbfff700 1 -- 172.21.15.2:0/1150923424 shutdown_connections 2024-03-15T12:49:21.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.927+0000 7f7abbfff700 1 --2- 172.21.15.2:0/1150923424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7abc1081b0 0x7f7abc10a5a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:21.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.927+0000 7f7abbfff700 1 -- 172.21.15.2:0/1150923424 >> 172.21.15.2:0/1150923424 conn(0x7f7abc101c00 msgr2=0x7f7abc104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:21.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.927+0000 7f7abbfff700 1 -- 172.21.15.2:0/1150923424 shutdown_connections 2024-03-15T12:49:21.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.927+0000 7f7abbfff700 1 -- 172.21.15.2:0/1150923424 wait complete. 2024-03-15T12:49:21.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.927+0000 7f7abbfff700 1 Processor -- start 2024-03-15T12:49:21.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.927+0000 7f7abbfff700 1 -- start start 2024-03-15T12:49:21.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.927+0000 7f7abbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7abc1081b0 0x7f7abc19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:21.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.928+0000 7f7abbfff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc19b940 con 0x7f7abc1081b0 2024-03-15T12:49:21.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.928+0000 7f7abaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7abc1081b0 0x7f7abc19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:21.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.928+0000 7f7abaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7abc1081b0 0x7f7abc19b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43796/0 (socket says 172.21.15.2:43796) 2024-03-15T12:49:21.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.928+0000 7f7abaffd700 1 -- 172.21.15.2:0/1745652826 learned_addr learned my addr 172.21.15.2:0/1745652826 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:21.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.928+0000 7f7abaffd700 1 -- 172.21.15.2:0/1745652826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac009b30 con 0x7f7abc1081b0 2024-03-15T12:49:21.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.928+0000 7f7abaffd700 1 --2- 172.21.15.2:0/1745652826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7abc1081b0 0x7f7abc19b400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7aac000f80 tx=0x7f7aac0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:21.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.928+0000 7f7ab3fff700 1 -- 172.21.15.2:0/1745652826 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7aac00e040 con 0x7f7abc1081b0 2024-03-15T12:49:21.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.928+0000 7f7abbfff700 1 -- 172.21.15.2:0/1745652826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7abc19bb40 con 0x7f7abc1081b0 2024-03-15T12:49:21.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.928+0000 7f7ab3fff700 1 -- 172.21.15.2:0/1745652826 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7aac00bd70 con 0x7f7abc1081b0 2024-03-15T12:49:21.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.928+0000 7f7ab3fff700 1 -- 172.21.15.2:0/1745652826 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7aac01c880 con 0x7f7abc1081b0 2024-03-15T12:49:21.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.928+0000 7f7abbfff700 1 -- 172.21.15.2:0/1745652826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc19bfe0 con 0x7f7abc1081b0 2024-03-15T12:49:21.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.929+0000 7f7ab3fff700 1 -- 172.21.15.2:0/1745652826 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f7aac01e450 con 0x7f7abc1081b0 2024-03-15T12:49:21.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.929+0000 7f7abbfff700 1 -- 172.21.15.2:0/1745652826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7abc10b920 con 0x7f7abc1081b0 2024-03-15T12:49:21.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.929+0000 7f7ab3fff700 1 --2- 172.21.15.2:0/1745652826 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7aa8038d00 0x7f7aa803b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:21.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.929+0000 7f7ab3fff700 1 -- 172.21.15.2:0/1745652826 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7aac04d250 con 0x7f7abc1081b0 2024-03-15T12:49:21.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.929+0000 7f7aba7fc700 1 --2- 172.21.15.2:0/1745652826 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7aa8038d00 0x7f7aa803b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:21.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.930+0000 7f7aba7fc700 1 --2- 172.21.15.2:0/1745652826 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7aa8038d00 0x7f7aa803b1c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4006fd0 tx=0x7f7aa4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:21.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:20.934+0000 7f7ab3fff700 1 -- 172.21.15.2:0/1745652826 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7aac01b8b0 con 0x7f7abc1081b0 2024-03-15T12:49:21.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:21.149+0000 7f7abbfff700 1 -- 172.21.15.2:0/1745652826 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f7abc19e920 con 0x7f7aa8038d00 2024-03-15T12:49:21.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:21.303+0000 7f7ab3fff700 1 -- 172.21.15.2:0/1745652826 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f7abc19e920 con 0x7f7aa8038d00 2024-03-15T12:49:21.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:21.305+0000 7f7abbfff700 1 -- 172.21.15.2:0/1745652826 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7aa8038d00 msgr2=0x7f7aa803b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:21.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:21.305+0000 7f7abbfff700 1 --2- 172.21.15.2:0/1745652826 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7aa8038d00 0x7f7aa803b1c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4006fd0 tx=0x7f7aa4006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:21.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:21.305+0000 7f7abbfff700 1 -- 172.21.15.2:0/1745652826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7abc1081b0 msgr2=0x7f7abc19b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:21.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:21.305+0000 7f7abbfff700 1 --2- 172.21.15.2:0/1745652826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7abc1081b0 0x7f7abc19b400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7aac000f80 tx=0x7f7aac0093e0 comp rx=0 tx=0).stop 2024-03-15T12:49:21.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:21.306+0000 7f7abbfff700 1 -- 172.21.15.2:0/1745652826 shutdown_connections 2024-03-15T12:49:21.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:21.306+0000 7f7abbfff700 1 --2- 172.21.15.2:0/1745652826 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7aa8038d00 0x7f7aa803b1c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:21.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:21.306+0000 7f7abbfff700 1 --2- 172.21.15.2:0/1745652826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7abc1081b0 0x7f7abc19b400 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:21.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:21.306+0000 7f7abbfff700 1 -- 172.21.15.2:0/1745652826 >> 172.21.15.2:0/1745652826 conn(0x7f7abc101c00 msgr2=0x7f7abc1026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:21.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:21.306+0000 7f7abbfff700 1 -- 172.21.15.2:0/1745652826 shutdown_connections 2024-03-15T12:49:21.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:21.306+0000 7f7abbfff700 1 -- 172.21.15.2:0/1745652826 wait complete. 2024-03-15T12:49:21.868 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial admin user... 2024-03-15T12:49:23.293 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$1mLR5BF3u.P9lUoD.NJtKexjn86Z2sk.APqTCtjhSEmme5elofD2i", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710506962, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-15T12:49:23.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.275+0000 7efc64c9f700 1 Processor -- start 2024-03-15T12:49:23.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.275+0000 7efc64c9f700 1 -- start start 2024-03-15T12:49:23.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.275+0000 7efc64c9f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc601067d0 0x7efc60106bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:23.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.275+0000 7efc64c9f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc601071c0 con 0x7efc601067d0 2024-03-15T12:49:23.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.275+0000 7efc5e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc601067d0 0x7efc60106bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:23.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.275+0000 7efc5e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc601067d0 0x7efc60106bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43812/0 (socket says 172.21.15.2:43812) 2024-03-15T12:49:23.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.275+0000 7efc5e59c700 1 -- 172.21.15.2:0/2115479924 learned_addr learned my addr 172.21.15.2:0/2115479924 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:23.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.275+0000 7efc5e59c700 1 -- 172.21.15.2:0/2115479924 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc601079d0 con 0x7efc601067d0 2024-03-15T12:49:23.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.276+0000 7efc5e59c700 1 --2- 172.21.15.2:0/2115479924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc601067d0 0x7efc60106bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efc5000bb40 tx=0x7efc5000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835afd5c07f5a2e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:23.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.276+0000 7efc5d59a700 1 -- 172.21.15.2:0/2115479924 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc5000dbe0 con 0x7efc601067d0 2024-03-15T12:49:23.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.276+0000 7efc5d59a700 1 -- 172.21.15.2:0/2115479924 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efc50004500 con 0x7efc601067d0 2024-03-15T12:49:23.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.276+0000 7efc5d59a700 1 -- 172.21.15.2:0/2115479924 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc50003e90 con 0x7efc601067d0 2024-03-15T12:49:23.294 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.276+0000 7efc64c9f700 1 -- 172.21.15.2:0/2115479924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc601067d0 msgr2=0x7efc60106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:23.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.276+0000 7efc64c9f700 1 --2- 172.21.15.2:0/2115479924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc601067d0 0x7efc60106bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efc5000bb40 tx=0x7efc5000be50 comp rx=0 tx=0).stop 2024-03-15T12:49:23.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.277+0000 7efc64c9f700 1 -- 172.21.15.2:0/2115479924 shutdown_connections 2024-03-15T12:49:23.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.277+0000 7efc64c9f700 1 --2- 172.21.15.2:0/2115479924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc601067d0 0x7efc60106bf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:23.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.277+0000 7efc64c9f700 1 -- 172.21.15.2:0/2115479924 >> 172.21.15.2:0/2115479924 conn(0x7efc60101e50 msgr2=0x7efc60104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:23.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.277+0000 7efc64c9f700 1 -- 172.21.15.2:0/2115479924 shutdown_connections 2024-03-15T12:49:23.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.277+0000 7efc64c9f700 1 -- 172.21.15.2:0/2115479924 wait complete. 2024-03-15T12:49:23.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.277+0000 7efc64c9f700 1 Processor -- start 2024-03-15T12:49:23.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.277+0000 7efc64c9f700 1 -- start start 2024-03-15T12:49:23.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.278+0000 7efc64c9f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc601067d0 0x7efc60199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:23.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.278+0000 7efc64c9f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc60199aa0 con 0x7efc601067d0 2024-03-15T12:49:23.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.278+0000 7efc5e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc601067d0 0x7efc60199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:23.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.278+0000 7efc5e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc601067d0 0x7efc60199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43826/0 (socket says 172.21.15.2:43826) 2024-03-15T12:49:23.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.278+0000 7efc5e59c700 1 -- 172.21.15.2:0/2752874642 learned_addr learned my addr 172.21.15.2:0/2752874642 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:23.295 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.278+0000 7efc5e59c700 1 -- 172.21.15.2:0/2752874642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5000b7f0 con 0x7efc601067d0 2024-03-15T12:49:23.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.278+0000 7efc5e59c700 1 --2- 172.21.15.2:0/2752874642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc601067d0 0x7efc60199560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b340 tx=0x7efc50003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:23.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.278+0000 7efc4f7fe700 1 -- 172.21.15.2:0/2752874642 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc5001e070 con 0x7efc601067d0 2024-03-15T12:49:23.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.278+0000 7efc4f7fe700 1 -- 172.21.15.2:0/2752874642 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efc50004360 con 0x7efc601067d0 2024-03-15T12:49:23.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.278+0000 7efc64c9f700 1 -- 172.21.15.2:0/2752874642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc60199ca0 con 0x7efc601067d0 2024-03-15T12:49:23.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.278+0000 7efc4f7fe700 1 -- 172.21.15.2:0/2752874642 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc500228d0 con 0x7efc601067d0 2024-03-15T12:49:23.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.278+0000 7efc64c9f700 1 -- 172.21.15.2:0/2752874642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc6019a140 con 0x7efc601067d0 2024-03-15T12:49:23.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.279+0000 7efc4f7fe700 1 -- 172.21.15.2:0/2752874642 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7efc50022ad0 con 0x7efc601067d0 2024-03-15T12:49:23.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.279+0000 7efc4f7fe700 1 --2- 172.21.15.2:0/2752874642 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7efc48038d00 0x7efc4803b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:23.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.280+0000 7efc4f7fe700 1 -- 172.21.15.2:0/2752874642 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efc5004f1f0 con 0x7efc601067d0 2024-03-15T12:49:23.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.280+0000 7efc64c9f700 1 -- 172.21.15.2:0/2752874642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc60063610 con 0x7efc601067d0 2024-03-15T12:49:23.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.280+0000 7efc5dd9b700 1 --2- 172.21.15.2:0/2752874642 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7efc48038d00 0x7efc4803b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:23.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.280+0000 7efc5dd9b700 1 --2- 172.21.15.2:0/2752874642 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7efc48038d00 0x7efc4803b1c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efc54006fd0 tx=0x7efc54006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:23.296 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.285+0000 7efc4f7fe700 1 -- 172.21.15.2:0/2752874642 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efc50027080 con 0x7efc601067d0 2024-03-15T12:49:23.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.514+0000 7efc64c9f700 1 -- 172.21.15.2:0/2752874642 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7efc60063430 con 0x7efc48038d00 2024-03-15T12:49:23.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.763+0000 7efc4f7fe700 1 -- 172.21.15.2:0/2752874642 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7efc60063430 con 0x7efc48038d00 2024-03-15T12:49:23.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.765+0000 7efc64c9f700 1 -- 172.21.15.2:0/2752874642 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7efc48038d00 msgr2=0x7efc4803b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:23.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.765+0000 7efc64c9f700 1 --2- 172.21.15.2:0/2752874642 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7efc48038d00 0x7efc4803b1c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efc54006fd0 tx=0x7efc54006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:23.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.765+0000 7efc64c9f700 1 -- 172.21.15.2:0/2752874642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc601067d0 msgr2=0x7efc60199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:23.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.765+0000 7efc64c9f700 1 --2- 172.21.15.2:0/2752874642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc601067d0 0x7efc60199560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efc5000b340 tx=0x7efc50003d00 comp rx=0 tx=0).stop 2024-03-15T12:49:23.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.766+0000 7efc64c9f700 1 -- 172.21.15.2:0/2752874642 shutdown_connections 2024-03-15T12:49:23.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.766+0000 7efc64c9f700 1 --2- 172.21.15.2:0/2752874642 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7efc48038d00 0x7efc4803b1c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:23.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.766+0000 7efc64c9f700 1 --2- 172.21.15.2:0/2752874642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc601067d0 0x7efc60199560 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:23.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.766+0000 7efc64c9f700 1 -- 172.21.15.2:0/2752874642 >> 172.21.15.2:0/2752874642 conn(0x7efc60101e50 msgr2=0x7efc6018fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:23.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.766+0000 7efc64c9f700 1 -- 172.21.15.2:0/2752874642 shutdown_connections 2024-03-15T12:49:23.297 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:22.766+0000 7efc64c9f700 1 -- 172.21.15.2:0/2752874642 wait complete. 2024-03-15T12:49:23.297 INFO:teuthology.orchestra.run.smithi002.stdout:Fetching dashboard port number... 2024-03-15T12:49:23.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:23 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:23.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:23 smithi002 ceph-mon[32060]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:49:23.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:23 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:24.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:24 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2878677614' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-15T12:49:25.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 8443 2024-03-15T12:49:25.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.738+0000 7ff540b1a700 1 Processor -- start 2024-03-15T12:49:25.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.739+0000 7ff540b1a700 1 -- start start 2024-03-15T12:49:25.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.739+0000 7ff540b1a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff53c108340 0x7ff53c10a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:25.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.739+0000 7ff540b1a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c07aea0 con 0x7ff53c108340 2024-03-15T12:49:25.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.739+0000 7ff53a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff53c108340 0x7ff53c10a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:25.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.739+0000 7ff53a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff53c108340 0x7ff53c10a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43828/0 (socket says 172.21.15.2:43828) 2024-03-15T12:49:25.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.739+0000 7ff53a59c700 1 -- 172.21.15.2:0/723133390 learned_addr learned my addr 172.21.15.2:0/723133390 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:25.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.739+0000 7ff53a59c700 1 -- 172.21.15.2:0/723133390 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c10ac70 con 0x7ff53c108340 2024-03-15T12:49:25.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.740+0000 7ff53a59c700 1 --2- 172.21.15.2:0/723133390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff53c108340 0x7ff53c10a730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5300098a0 tx=0x7ff530009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a350d04b9385b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:25.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.740+0000 7ff53959a700 1 -- 172.21.15.2:0/723133390 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff530004160 con 0x7ff53c108340 2024-03-15T12:49:25.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.740+0000 7ff53959a700 1 -- 172.21.15.2:0/723133390 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff5300042c0 con 0x7ff53c108340 2024-03-15T12:49:25.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.740+0000 7ff53959a700 1 -- 172.21.15.2:0/723133390 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff530011a30 con 0x7ff53c108340 2024-03-15T12:49:25.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.740+0000 7ff540b1a700 1 -- 172.21.15.2:0/723133390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff53c108340 msgr2=0x7ff53c10a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:25.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.740+0000 7ff540b1a700 1 --2- 172.21.15.2:0/723133390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff53c108340 0x7ff53c10a730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5300098a0 tx=0x7ff530009bb0 comp rx=0 tx=0).stop 2024-03-15T12:49:25.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.740+0000 7ff540b1a700 1 -- 172.21.15.2:0/723133390 shutdown_connections 2024-03-15T12:49:25.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.740+0000 7ff540b1a700 1 --2- 172.21.15.2:0/723133390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff53c108340 0x7ff53c10a730 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:25.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.740+0000 7ff540b1a700 1 -- 172.21.15.2:0/723133390 >> 172.21.15.2:0/723133390 conn(0x7ff53c101d70 msgr2=0x7ff53c104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:25.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.741+0000 7ff540b1a700 1 -- 172.21.15.2:0/723133390 shutdown_connections 2024-03-15T12:49:25.250 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.741+0000 7ff540b1a700 1 -- 172.21.15.2:0/723133390 wait complete. 2024-03-15T12:49:25.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.741+0000 7ff540b1a700 1 Processor -- start 2024-03-15T12:49:25.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.741+0000 7ff540b1a700 1 -- start start 2024-03-15T12:49:25.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.741+0000 7ff540b1a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff53c108340 0x7ff53c19b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:25.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.741+0000 7ff540b1a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c19bbf0 con 0x7ff53c108340 2024-03-15T12:49:25.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.742+0000 7ff53a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff53c108340 0x7ff53c19b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:25.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.742+0000 7ff53a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff53c108340 0x7ff53c19b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43836/0 (socket says 172.21.15.2:43836) 2024-03-15T12:49:25.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.742+0000 7ff53a59c700 1 -- 172.21.15.2:0/2878677614 learned_addr learned my addr 172.21.15.2:0/2878677614 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:25.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.742+0000 7ff53a59c700 1 -- 172.21.15.2:0/2878677614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff530009550 con 0x7ff53c108340 2024-03-15T12:49:25.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.742+0000 7ff53a59c700 1 --2- 172.21.15.2:0/2878677614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff53c108340 0x7ff53c19b6b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff530006af0 tx=0x7ff530011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:25.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.742+0000 7ff52b7fe700 1 -- 172.21.15.2:0/2878677614 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff530003b50 con 0x7ff53c108340 2024-03-15T12:49:25.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.742+0000 7ff540b1a700 1 -- 172.21.15.2:0/2878677614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53c19bdf0 con 0x7ff53c108340 2024-03-15T12:49:25.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.742+0000 7ff52b7fe700 1 -- 172.21.15.2:0/2878677614 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff530003cb0 con 0x7ff53c108340 2024-03-15T12:49:25.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.742+0000 7ff52b7fe700 1 -- 172.21.15.2:0/2878677614 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff530024800 con 0x7ff53c108340 2024-03-15T12:49:25.251 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.742+0000 7ff540b1a700 1 -- 172.21.15.2:0/2878677614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53c19c290 con 0x7ff53c108340 2024-03-15T12:49:25.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.743+0000 7ff52b7fe700 1 -- 172.21.15.2:0/2878677614 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7ff53002d3f0 con 0x7ff53c108340 2024-03-15T12:49:25.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.743+0000 7ff52b7fe700 1 --2- 172.21.15.2:0/2878677614 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff524038d00 0x7ff52403b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:25.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.743+0000 7ff52b7fe700 1 -- 172.21.15.2:0/2878677614 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff53001a920 con 0x7ff53c108340 2024-03-15T12:49:25.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.744+0000 7ff540b1a700 1 -- 172.21.15.2:0/2878677614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff53c10bac0 con 0x7ff53c108340 2024-03-15T12:49:25.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.744+0000 7ff539d9b700 1 --2- 172.21.15.2:0/2878677614 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff524038d00 0x7ff52403b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:25.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.744+0000 7ff539d9b700 1 --2- 172.21.15.2:0/2878677614 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff524038d00 0x7ff52403b1c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff52c006fd0 tx=0x7ff52c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:25.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.753+0000 7ff52b7fe700 1 -- 172.21.15.2:0/2878677614 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff53002d740 con 0x7ff53c108340 2024-03-15T12:49:25.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.961+0000 7ff540b1a700 1 -- 172.21.15.2:0/2878677614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7ff53c10bc80 con 0x7ff53c108340 2024-03-15T12:49:25.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.961+0000 7ff52b7fe700 1 -- 172.21.15.2:0/2878677614 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7ff530010ed0 con 0x7ff53c108340 2024-03-15T12:49:25.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.964+0000 7ff540b1a700 1 -- 172.21.15.2:0/2878677614 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff524038d00 msgr2=0x7ff52403b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:25.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.964+0000 7ff540b1a700 1 --2- 172.21.15.2:0/2878677614 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff524038d00 0x7ff52403b1c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff52c006fd0 tx=0x7ff52c006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:25.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.964+0000 7ff540b1a700 1 -- 172.21.15.2:0/2878677614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff53c108340 msgr2=0x7ff53c19b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:25.252 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.964+0000 7ff540b1a700 1 --2- 172.21.15.2:0/2878677614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff53c108340 0x7ff53c19b6b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff530006af0 tx=0x7ff530011b40 comp rx=0 tx=0).stop 2024-03-15T12:49:25.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.964+0000 7ff540b1a700 1 -- 172.21.15.2:0/2878677614 shutdown_connections 2024-03-15T12:49:25.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.964+0000 7ff540b1a700 1 --2- 172.21.15.2:0/2878677614 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff524038d00 0x7ff52403b1c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:25.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.964+0000 7ff540b1a700 1 --2- 172.21.15.2:0/2878677614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff53c108340 0x7ff53c19b6b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:25.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.964+0000 7ff540b1a700 1 -- 172.21.15.2:0/2878677614 >> 172.21.15.2:0/2878677614 conn(0x7ff53c101d70 msgr2=0x7ff53c1028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:25.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.964+0000 7ff540b1a700 1 -- 172.21.15.2:0/2878677614 shutdown_connections 2024-03-15T12:49:25.253 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:23.964+0000 7ff540b1a700 1 -- 172.21.15.2:0/2878677614 wait complete. 2024-03-15T12:49:25.258 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-15T12:49:25.258 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-03-15T12:49:25.266 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-15T12:49:25.266 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-03-15T12:49:25.266 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-03-15T12:49:25.266 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-15T12:49:25.268 INFO:teuthology.orchestra.run.smithi002.stdout:Ceph Dashboard is now available at: 2024-03-15T12:49:25.268 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:49:25.268 INFO:teuthology.orchestra.run.smithi002.stdout: URL: https://smithi002.front.sepia.ceph.com:8443/ 2024-03-15T12:49:25.268 INFO:teuthology.orchestra.run.smithi002.stdout: User: admin 2024-03-15T12:49:25.269 INFO:teuthology.orchestra.run.smithi002.stdout: Password: k0e21lzh2n 2024-03-15T12:49:25.269 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:49:25.269 INFO:teuthology.orchestra.run.smithi002.stdout:Saving cluster configuration to /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config directory 2024-03-15T12:49:25.269 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling autotune for osd_memory_target 2024-03-15T12:49:26.388 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.664+0000 7f6e0fb33700 1 Processor -- start 2024-03-15T12:49:26.388 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.664+0000 7f6e0fb33700 1 -- start start 2024-03-15T12:49:26.389 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.665+0000 7f6e0fb33700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e08108380 0x7f6e0810a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:26.389 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.665+0000 7f6e0fb33700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0807ae60 con 0x7f6e08108380 2024-03-15T12:49:26.389 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.665+0000 7f6e0d8cf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e08108380 0x7f6e0810a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:26.389 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.665+0000 7f6e0d8cf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e08108380 0x7f6e0810a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43852/0 (socket says 172.21.15.2:43852) 2024-03-15T12:49:26.389 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.665+0000 7f6e0d8cf700 1 -- 172.21.15.2:0/3160103456 learned_addr learned my addr 172.21.15.2:0/3160103456 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:26.389 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.665+0000 7f6e0d8cf700 1 -- 172.21.15.2:0/3160103456 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0810acb0 con 0x7f6e08108380 2024-03-15T12:49:26.389 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.665+0000 7f6e0d8cf700 1 --2- 172.21.15.2:0/3160103456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e08108380 0x7f6e0810a770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6e040098a0 tx=0x7f6e04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d312d493263c147d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:26.389 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.665+0000 7f6e0c8cd700 1 -- 172.21.15.2:0/3160103456 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e04004160 con 0x7f6e08108380 2024-03-15T12:49:26.389 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.666+0000 7f6e0c8cd700 1 -- 172.21.15.2:0/3160103456 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6e040042c0 con 0x7f6e08108380 2024-03-15T12:49:26.389 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.666+0000 7f6e0c8cd700 1 -- 172.21.15.2:0/3160103456 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e04011a30 con 0x7f6e08108380 2024-03-15T12:49:26.389 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.666+0000 7f6e0fb33700 1 -- 172.21.15.2:0/3160103456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e08108380 msgr2=0x7f6e0810a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:26.389 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.666+0000 7f6e0fb33700 1 --2- 172.21.15.2:0/3160103456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e08108380 0x7f6e0810a770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6e040098a0 tx=0x7f6e04009bb0 comp rx=0 tx=0).stop 2024-03-15T12:49:26.390 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.666+0000 7f6e0fb33700 1 -- 172.21.15.2:0/3160103456 shutdown_connections 2024-03-15T12:49:26.390 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.666+0000 7f6e0fb33700 1 --2- 172.21.15.2:0/3160103456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e08108380 0x7f6e0810a770 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:26.390 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.666+0000 7f6e0fb33700 1 -- 172.21.15.2:0/3160103456 >> 172.21.15.2:0/3160103456 conn(0x7f6e08101e50 msgr2=0x7f6e08104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:26.390 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.666+0000 7f6e0fb33700 1 -- 172.21.15.2:0/3160103456 shutdown_connections 2024-03-15T12:49:26.390 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.666+0000 7f6e0fb33700 1 -- 172.21.15.2:0/3160103456 wait complete. 2024-03-15T12:49:26.390 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.667+0000 7f6e0fb33700 1 Processor -- start 2024-03-15T12:49:26.390 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.667+0000 7f6e0fb33700 1 -- start start 2024-03-15T12:49:26.390 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.667+0000 7f6e0fb33700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e08108380 0x7f6e0819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:26.390 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.667+0000 7f6e0fb33700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0819bb70 con 0x7f6e08108380 2024-03-15T12:49:26.390 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.667+0000 7f6e0d8cf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e08108380 0x7f6e0819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:26.390 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.667+0000 7f6e0d8cf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e08108380 0x7f6e0819b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43856/0 (socket says 172.21.15.2:43856) 2024-03-15T12:49:26.391 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.667+0000 7f6e0d8cf700 1 -- 172.21.15.2:0/4285680362 learned_addr learned my addr 172.21.15.2:0/4285680362 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:26.391 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.667+0000 7f6e0d8cf700 1 -- 172.21.15.2:0/4285680362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e04009550 con 0x7f6e08108380 2024-03-15T12:49:26.391 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.668+0000 7f6e0d8cf700 1 --2- 172.21.15.2:0/4285680362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e08108380 0x7f6e0819b630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6e04006af0 tx=0x7f6e04011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:26.391 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.668+0000 7f6dfeffd700 1 -- 172.21.15.2:0/4285680362 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e040039e0 con 0x7f6e08108380 2024-03-15T12:49:26.391 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.668+0000 7f6dfeffd700 1 -- 172.21.15.2:0/4285680362 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6e04003b40 con 0x7f6e08108380 2024-03-15T12:49:26.391 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.668+0000 7f6dfeffd700 1 -- 172.21.15.2:0/4285680362 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e040196c0 con 0x7f6e08108380 2024-03-15T12:49:26.391 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.668+0000 7f6e0fb33700 1 -- 172.21.15.2:0/4285680362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0819bd70 con 0x7f6e08108380 2024-03-15T12:49:26.391 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.668+0000 7f6e0fb33700 1 -- 172.21.15.2:0/4285680362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0819c210 con 0x7f6e08108380 2024-03-15T12:49:26.391 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.669+0000 7f6dfeffd700 1 -- 172.21.15.2:0/4285680362 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f6e04019980 con 0x7f6e08108380 2024-03-15T12:49:26.391 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.669+0000 7f6dfeffd700 1 --2- 172.21.15.2:0/4285680362 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6df4038d50 0x7f6df403b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:26.391 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.669+0000 7f6dfeffd700 1 -- 172.21.15.2:0/4285680362 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6e0404d0d0 con 0x7f6e08108380 2024-03-15T12:49:26.391 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.669+0000 7f6e0fb33700 1 -- 172.21.15.2:0/4285680362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0810baf0 con 0x7f6e08108380 2024-03-15T12:49:26.391 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.669+0000 7f6e0d0ce700 1 --2- 172.21.15.2:0/4285680362 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6df4038d50 0x7f6df403b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:26.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.669+0000 7f6e0d0ce700 1 --2- 172.21.15.2:0/4285680362 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6df4038d50 0x7f6df403b210 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6df8006fd0 tx=0x7f6df8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:26.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.674+0000 7f6dfeffd700 1 -- 172.21.15.2:0/4285680362 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6e0402c3a0 con 0x7f6e08108380 2024-03-15T12:49:26.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.878+0000 7f6e0fb33700 1 -- 172.21.15.2:0/4285680362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f6e0810bcb0 con 0x7f6e08108380 2024-03-15T12:49:26.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.879+0000 7f6dfeffd700 1 -- 172.21.15.2:0/4285680362 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f6e040350c0 con 0x7f6e08108380 2024-03-15T12:49:26.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.881+0000 7f6e0fb33700 1 -- 172.21.15.2:0/4285680362 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6df4038d50 msgr2=0x7f6df403b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:26.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.881+0000 7f6e0fb33700 1 --2- 172.21.15.2:0/4285680362 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6df4038d50 0x7f6df403b210 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6df8006fd0 tx=0x7f6df8006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:26.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.881+0000 7f6e0fb33700 1 -- 172.21.15.2:0/4285680362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e08108380 msgr2=0x7f6e0819b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:26.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.881+0000 7f6e0fb33700 1 --2- 172.21.15.2:0/4285680362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e08108380 0x7f6e0819b630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6e04006af0 tx=0x7f6e04011bb0 comp rx=0 tx=0).stop 2024-03-15T12:49:26.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.881+0000 7f6e0fb33700 1 -- 172.21.15.2:0/4285680362 shutdown_connections 2024-03-15T12:49:26.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.881+0000 7f6e0fb33700 1 --2- 172.21.15.2:0/4285680362 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6df4038d50 0x7f6df403b210 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:26.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.881+0000 7f6e0fb33700 1 --2- 172.21.15.2:0/4285680362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e08108380 0x7f6e0819b630 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:26.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.881+0000 7f6e0fb33700 1 -- 172.21.15.2:0/4285680362 >> 172.21.15.2:0/4285680362 conn(0x7f6e08101e50 msgr2=0x7f6e08102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:26.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.881+0000 7f6e0fb33700 1 -- 172.21.15.2:0/4285680362 shutdown_connections 2024-03-15T12:49:26.392 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:25.881+0000 7f6e0fb33700 1 -- 172.21.15.2:0/4285680362 wait complete. 2024-03-15T12:49:26.489 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:26 smithi002 ceph-mon[32060]: mgrmap e13: x(active, since 6s) 2024-03-15T12:49:27.668 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.802+0000 7f6c9b34b700 1 Processor -- start 2024-03-15T12:49:27.669 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.802+0000 7f6c9b34b700 1 -- start start 2024-03-15T12:49:27.669 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.802+0000 7f6c9b34b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c9407ce60 0x7f6c9407b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:27.669 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.802+0000 7f6c9b34b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9407b8e0 con 0x7f6c9407ce60 2024-03-15T12:49:27.669 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.802+0000 7f6c990e7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c9407ce60 0x7f6c9407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:27.669 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.802+0000 7f6c990e7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c9407ce60 0x7f6c9407b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43858/0 (socket says 172.21.15.2:43858) 2024-03-15T12:49:27.669 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.802+0000 7f6c990e7700 1 -- 172.21.15.2:0/4131881548 learned_addr learned my addr 172.21.15.2:0/4131881548 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:27.669 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.803+0000 7f6c990e7700 1 -- 172.21.15.2:0/4131881548 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9407ba20 con 0x7f6c9407ce60 2024-03-15T12:49:27.669 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.803+0000 7f6c990e7700 1 --2- 172.21.15.2:0/4131881548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c9407ce60 0x7f6c9407b310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c900098a0 tx=0x7f6c90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8f567f98b500a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:27.669 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.803+0000 7f6c8bfff700 1 -- 172.21.15.2:0/4131881548 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c90004160 con 0x7f6c9407ce60 2024-03-15T12:49:27.669 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.803+0000 7f6c8bfff700 1 -- 172.21.15.2:0/4131881548 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6c900042c0 con 0x7f6c9407ce60 2024-03-15T12:49:27.669 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.803+0000 7f6c8bfff700 1 -- 172.21.15.2:0/4131881548 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c90011a30 con 0x7f6c9407ce60 2024-03-15T12:49:27.669 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.803+0000 7f6c9b34b700 1 -- 172.21.15.2:0/4131881548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c9407ce60 msgr2=0x7f6c9407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:27.669 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.803+0000 7f6c9b34b700 1 --2- 172.21.15.2:0/4131881548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c9407ce60 0x7f6c9407b310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c900098a0 tx=0x7f6c90009bb0 comp rx=0 tx=0).stop 2024-03-15T12:49:27.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.803+0000 7f6c9b34b700 1 -- 172.21.15.2:0/4131881548 shutdown_connections 2024-03-15T12:49:27.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.803+0000 7f6c9b34b700 1 --2- 172.21.15.2:0/4131881548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c9407ce60 0x7f6c9407b310 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:27.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.803+0000 7f6c9b34b700 1 -- 172.21.15.2:0/4131881548 >> 172.21.15.2:0/4131881548 conn(0x7f6c94102e90 msgr2=0x7f6c941052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:27.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.804+0000 7f6c9b34b700 1 -- 172.21.15.2:0/4131881548 shutdown_connections 2024-03-15T12:49:27.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.804+0000 7f6c9b34b700 1 -- 172.21.15.2:0/4131881548 wait complete. 2024-03-15T12:49:27.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.804+0000 7f6c9b34b700 1 Processor -- start 2024-03-15T12:49:27.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.804+0000 7f6c9b34b700 1 -- start start 2024-03-15T12:49:27.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.804+0000 7f6c9b34b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c9407ce60 0x7f6c9419d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:27.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.804+0000 7f6c9b34b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9419dde0 con 0x7f6c9407ce60 2024-03-15T12:49:27.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.804+0000 7f6c990e7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c9407ce60 0x7f6c9419d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:27.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.804+0000 7f6c990e7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c9407ce60 0x7f6c9419d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43870/0 (socket says 172.21.15.2:43870) 2024-03-15T12:49:27.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.804+0000 7f6c990e7700 1 -- 172.21.15.2:0/1397515169 learned_addr learned my addr 172.21.15.2:0/1397515169 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:27.670 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.805+0000 7f6c990e7700 1 -- 172.21.15.2:0/1397515169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c90009550 con 0x7f6c9407ce60 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.805+0000 7f6c990e7700 1 --2- 172.21.15.2:0/1397515169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c9407ce60 0x7f6c9419d8a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6c90011670 tx=0x7f6c90011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.805+0000 7f6c8a7fc700 1 -- 172.21.15.2:0/1397515169 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c90003a80 con 0x7f6c9407ce60 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.805+0000 7f6c9b34b700 1 -- 172.21.15.2:0/1397515169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9419dfe0 con 0x7f6c9407ce60 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.805+0000 7f6c8a7fc700 1 -- 172.21.15.2:0/1397515169 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6c90003be0 con 0x7f6c9407ce60 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.805+0000 7f6c8a7fc700 1 -- 172.21.15.2:0/1397515169 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c90019730 con 0x7f6c9407ce60 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.805+0000 7f6c9b34b700 1 -- 172.21.15.2:0/1397515169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9419e480 con 0x7f6c9407ce60 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.806+0000 7f6c8a7fc700 1 -- 172.21.15.2:0/1397515169 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f6c900199f0 con 0x7f6c9407ce60 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.806+0000 7f6c9b34b700 1 -- 172.21.15.2:0/1397515169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9410dcf0 con 0x7f6c9407ce60 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.806+0000 7f6c8a7fc700 1 --2- 172.21.15.2:0/1397515169 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6c80038d50 0x7f6c8003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.806+0000 7f6c8a7fc700 1 -- 172.21.15.2:0/1397515169 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6c9004d150 con 0x7f6c9407ce60 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.806+0000 7f6c988e6700 1 --2- 172.21.15.2:0/1397515169 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6c80038d50 0x7f6c8003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.807+0000 7f6c988e6700 1 --2- 172.21.15.2:0/1397515169 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6c80038d50 0x7f6c8003b210 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6c84006fd0 tx=0x7f6c84006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:26.811+0000 7f6c8a7fc700 1 -- 172.21.15.2:0/1397515169 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6c9002b730 con 0x7f6c9407ce60 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:27.126+0000 7f6c9b34b700 1 -- 172.21.15.2:0/1397515169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f6c941a0d60 con 0x7f6c9407ce60 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:27.128+0000 7f6c8a7fc700 1 -- 172.21.15.2:0/1397515169 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f6c9002b340 con 0x7f6c9407ce60 2024-03-15T12:49:27.671 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:27.130+0000 7f6c9b34b700 1 -- 172.21.15.2:0/1397515169 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6c80038d50 msgr2=0x7f6c8003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:27.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:27.130+0000 7f6c9b34b700 1 --2- 172.21.15.2:0/1397515169 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6c80038d50 0x7f6c8003b210 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6c84006fd0 tx=0x7f6c84006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:27.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:27.130+0000 7f6c9b34b700 1 -- 172.21.15.2:0/1397515169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c9407ce60 msgr2=0x7f6c9419d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:27.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:27.130+0000 7f6c9b34b700 1 --2- 172.21.15.2:0/1397515169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c9407ce60 0x7f6c9419d8a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6c90011670 tx=0x7f6c90011b20 comp rx=0 tx=0).stop 2024-03-15T12:49:27.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:27.130+0000 7f6c9b34b700 1 -- 172.21.15.2:0/1397515169 shutdown_connections 2024-03-15T12:49:27.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:27.130+0000 7f6c9b34b700 1 --2- 172.21.15.2:0/1397515169 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6c80038d50 0x7f6c8003b210 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:27.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:27.130+0000 7f6c9b34b700 1 --2- 172.21.15.2:0/1397515169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c9407ce60 0x7f6c9419d8a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:27.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:27.130+0000 7f6c9b34b700 1 -- 172.21.15.2:0/1397515169 >> 172.21.15.2:0/1397515169 conn(0x7f6c94102e90 msgr2=0x7f6c941039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:27.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:27.130+0000 7f6c9b34b700 1 -- 172.21.15.2:0/1397515169 shutdown_connections 2024-03-15T12:49:27.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-15T12:49:27.131+0000 7f6c9b34b700 1 -- 172.21.15.2:0/1397515169 wait complete. 2024-03-15T12:49:27.672 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-15T12:49:27.672 INFO:teuthology.orchestra.run.smithi002.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-15T12:49:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:49:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-15T12:49:27.672 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:49:27.672 INFO:teuthology.orchestra.run.smithi002.stdout:Or, if you are only running a single cluster on this host: 2024-03-15T12:49:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:49:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-15T12:49:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:49:27.673 INFO:teuthology.orchestra.run.smithi002.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-15T12:49:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:49:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: ceph telemetry on 2024-03-15T12:49:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:49:27.673 INFO:teuthology.orchestra.run.smithi002.stdout:For more information see: 2024-03-15T12:49:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:49:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-15T12:49:27.673 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:49:27.673 INFO:teuthology.orchestra.run.smithi002.stdout:Bootstrap complete. 2024-03-15T12:49:27.711 INFO:tasks.cephadm:Fetching config... 2024-03-15T12:49:27.711 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:49:27.711 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-15T12:49:27.728 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-15T12:49:27.728 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:49:27.728 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-15T12:49:27.783 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-15T12:49:27.783 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:49:27.783 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-15T12:49:27.850 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-15T12:49:27.850 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:49:27.850 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-15T12:49:27.904 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-15T12:49:27.905 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClSVWPIfdS232x37HptOHk6PeuHhgDy/yQM2jr2hgAUrNMcQDnlOf0ZzOU/UmtiwBC466wrpztWU73HxLo7s1GbtebBx8WDYC3k1+mcBcm4cIblmnWSKLd4bQtofJqM4aFu7MhAcRdk1OhDbFj8kWdh7huF78MfmdLiYQRoFtiOem4ah2ONlCd67bFzJVaRCMmhBFNBy6wa4KNIwRJpHZybfce1jusTAOuAmdoTECT50bpOiNAnv6rdrrbhbeb6geYWv807QIHxg+zu9YzZopIanJEDkqAzCHBKfTGhcYRL1lvE1md+zYNAy2mM/XnF+tdY/0PBtoXZBREOgNDhGx4GuaXNzmC0QotvQBa893ofjDWGk/+au+R7Qk3YJaza/XJeyj+SkZnFgxgo+/uVp8kR67dogt28raHjs/dNtFWmnwtvFOp+SAi/PclAZfQK9eDldfoW0c3vVYCiJEHAP8jMPOQZR4fUgQ62nX/M7aXQcdroN8MWGOS5k+Oq+g7gmE= ceph-2767718a-e2ca-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-15T12:49:27.983 INFO:teuthology.orchestra.run.smithi002.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClSVWPIfdS232x37HptOHk6PeuHhgDy/yQM2jr2hgAUrNMcQDnlOf0ZzOU/UmtiwBC466wrpztWU73HxLo7s1GbtebBx8WDYC3k1+mcBcm4cIblmnWSKLd4bQtofJqM4aFu7MhAcRdk1OhDbFj8kWdh7huF78MfmdLiYQRoFtiOem4ah2ONlCd67bFzJVaRCMmhBFNBy6wa4KNIwRJpHZybfce1jusTAOuAmdoTECT50bpOiNAnv6rdrrbhbeb6geYWv807QIHxg+zu9YzZopIanJEDkqAzCHBKfTGhcYRL1lvE1md+zYNAy2mM/XnF+tdY/0PBtoXZBREOgNDhGx4GuaXNzmC0QotvQBa893ofjDWGk/+au+R7Qk3YJaza/XJeyj+SkZnFgxgo+/uVp8kR67dogt28raHjs/dNtFWmnwtvFOp+SAi/PclAZfQK9eDldfoW0c3vVYCiJEHAP8jMPOQZR4fUgQ62nX/M7aXQcdroN8MWGOS5k+Oq+g7gmE= ceph-2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:49:27.997 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClSVWPIfdS232x37HptOHk6PeuHhgDy/yQM2jr2hgAUrNMcQDnlOf0ZzOU/UmtiwBC466wrpztWU73HxLo7s1GbtebBx8WDYC3k1+mcBcm4cIblmnWSKLd4bQtofJqM4aFu7MhAcRdk1OhDbFj8kWdh7huF78MfmdLiYQRoFtiOem4ah2ONlCd67bFzJVaRCMmhBFNBy6wa4KNIwRJpHZybfce1jusTAOuAmdoTECT50bpOiNAnv6rdrrbhbeb6geYWv807QIHxg+zu9YzZopIanJEDkqAzCHBKfTGhcYRL1lvE1md+zYNAy2mM/XnF+tdY/0PBtoXZBREOgNDhGx4GuaXNzmC0QotvQBa893ofjDWGk/+au+R7Qk3YJaza/XJeyj+SkZnFgxgo+/uVp8kR67dogt28raHjs/dNtFWmnwtvFOp+SAi/PclAZfQK9eDldfoW0c3vVYCiJEHAP8jMPOQZR4fUgQ62nX/M7aXQcdroN8MWGOS5k+Oq+g7gmE= ceph-2767718a-e2ca-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-15T12:49:28.042 INFO:teuthology.orchestra.run.smithi149.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClSVWPIfdS232x37HptOHk6PeuHhgDy/yQM2jr2hgAUrNMcQDnlOf0ZzOU/UmtiwBC466wrpztWU73HxLo7s1GbtebBx8WDYC3k1+mcBcm4cIblmnWSKLd4bQtofJqM4aFu7MhAcRdk1OhDbFj8kWdh7huF78MfmdLiYQRoFtiOem4ah2ONlCd67bFzJVaRCMmhBFNBy6wa4KNIwRJpHZybfce1jusTAOuAmdoTECT50bpOiNAnv6rdrrbhbeb6geYWv807QIHxg+zu9YzZopIanJEDkqAzCHBKfTGhcYRL1lvE1md+zYNAy2mM/XnF+tdY/0PBtoXZBREOgNDhGx4GuaXNzmC0QotvQBa893ofjDWGk/+au+R7Qk3YJaza/XJeyj+SkZnFgxgo+/uVp8kR67dogt28raHjs/dNtFWmnwtvFOp+SAi/PclAZfQK9eDldfoW0c3vVYCiJEHAP8jMPOQZR4fUgQ62nX/M7aXQcdroN8MWGOS5k+Oq+g7gmE= ceph-2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:49:28.057 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClSVWPIfdS232x37HptOHk6PeuHhgDy/yQM2jr2hgAUrNMcQDnlOf0ZzOU/UmtiwBC466wrpztWU73HxLo7s1GbtebBx8WDYC3k1+mcBcm4cIblmnWSKLd4bQtofJqM4aFu7MhAcRdk1OhDbFj8kWdh7huF78MfmdLiYQRoFtiOem4ah2ONlCd67bFzJVaRCMmhBFNBy6wa4KNIwRJpHZybfce1jusTAOuAmdoTECT50bpOiNAnv6rdrrbhbeb6geYWv807QIHxg+zu9YzZopIanJEDkqAzCHBKfTGhcYRL1lvE1md+zYNAy2mM/XnF+tdY/0PBtoXZBREOgNDhGx4GuaXNzmC0QotvQBa893ofjDWGk/+au+R7Qk3YJaza/XJeyj+SkZnFgxgo+/uVp8kR67dogt28raHjs/dNtFWmnwtvFOp+SAi/PclAZfQK9eDldfoW0c3vVYCiJEHAP8jMPOQZR4fUgQ62nX/M7aXQcdroN8MWGOS5k+Oq+g7gmE= ceph-2767718a-e2ca-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-15T12:49:28.103 INFO:teuthology.orchestra.run.smithi184.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClSVWPIfdS232x37HptOHk6PeuHhgDy/yQM2jr2hgAUrNMcQDnlOf0ZzOU/UmtiwBC466wrpztWU73HxLo7s1GbtebBx8WDYC3k1+mcBcm4cIblmnWSKLd4bQtofJqM4aFu7MhAcRdk1OhDbFj8kWdh7huF78MfmdLiYQRoFtiOem4ah2ONlCd67bFzJVaRCMmhBFNBy6wa4KNIwRJpHZybfce1jusTAOuAmdoTECT50bpOiNAnv6rdrrbhbeb6geYWv807QIHxg+zu9YzZopIanJEDkqAzCHBKfTGhcYRL1lvE1md+zYNAy2mM/XnF+tdY/0PBtoXZBREOgNDhGx4GuaXNzmC0QotvQBa893ofjDWGk/+au+R7Qk3YJaza/XJeyj+SkZnFgxgo+/uVp8kR67dogt28raHjs/dNtFWmnwtvFOp+SAi/PclAZfQK9eDldfoW0c3vVYCiJEHAP8jMPOQZR4fUgQ62nX/M7aXQcdroN8MWGOS5k+Oq+g7gmE= ceph-2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:49:28.118 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-15T12:49:28.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:28 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1397515169' entity='client.admin' 2024-03-15T12:49:28.362 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:49:30.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:30.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:30.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi002", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:49:30.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:30.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:49:30.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.702+0000 7f006dfe1700 1 -- 172.21.15.2:0/547927629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00681024a0 msgr2=0x7f0068104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.702+0000 7f006dfe1700 1 --2- 172.21.15.2:0/547927629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00681024a0 0x7f0068104890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0058009910 tx=0x7f0058009c20 comp rx=0 tx=0).stop 2024-03-15T12:49:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.702+0000 7f006dfe1700 1 -- 172.21.15.2:0/547927629 shutdown_connections 2024-03-15T12:49:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.702+0000 7f006dfe1700 1 --2- 172.21.15.2:0/547927629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00681024a0 0x7f0068104890 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.702+0000 7f006dfe1700 1 -- 172.21.15.2:0/547927629 >> 172.21.15.2:0/547927629 conn(0x7f00680fbeb0 msgr2=0x7f00680fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.702+0000 7f006dfe1700 1 -- 172.21.15.2:0/547927629 shutdown_connections 2024-03-15T12:49:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.702+0000 7f006dfe1700 1 -- 172.21.15.2:0/547927629 wait complete. 2024-03-15T12:49:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.703+0000 7f006dfe1700 1 Processor -- start 2024-03-15T12:49:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.703+0000 7f006dfe1700 1 -- start start 2024-03-15T12:49:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.704+0000 7f006dfe1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00681024a0 0x7f0068196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.704+0000 7f006dfe1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0068197420 con 0x7f00681024a0 2024-03-15T12:49:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.704+0000 7f00677fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00681024a0 0x7f0068196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.704+0000 7f00677fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00681024a0 0x7f0068196ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51150/0 (socket says 172.21.15.2:51150) 2024-03-15T12:49:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.704+0000 7f00677fe700 1 -- 172.21.15.2:0/1160272092 learned_addr learned my addr 172.21.15.2:0/1160272092 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.704+0000 7f00677fe700 1 -- 172.21.15.2:0/1160272092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00580095f0 con 0x7f00681024a0 2024-03-15T12:49:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.704+0000 7f00677fe700 1 --2- 172.21.15.2:0/1160272092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00681024a0 0x7f0068196ee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0058006010 tx=0x7f00580057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.704+0000 7f0064ff9700 1 -- 172.21.15.2:0/1160272092 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f005801d070 con 0x7f00681024a0 2024-03-15T12:49:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.704+0000 7f006dfe1700 1 -- 172.21.15.2:0/1160272092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0068197620 con 0x7f00681024a0 2024-03-15T12:49:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.704+0000 7f0064ff9700 1 -- 172.21.15.2:0/1160272092 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f005800be30 con 0x7f00681024a0 2024-03-15T12:49:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.704+0000 7f0064ff9700 1 -- 172.21.15.2:0/1160272092 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0058022e30 con 0x7f00681024a0 2024-03-15T12:49:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.704+0000 7f006dfe1700 1 -- 172.21.15.2:0/1160272092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0068197ac0 con 0x7f00681024a0 2024-03-15T12:49:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.705+0000 7f006dfe1700 1 -- 172.21.15.2:0/1160272092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0068190860 con 0x7f00681024a0 2024-03-15T12:49:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.705+0000 7f0064ff9700 1 -- 172.21.15.2:0/1160272092 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f005800f540 con 0x7f00681024a0 2024-03-15T12:49:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.706+0000 7f0064ff9700 1 --2- 172.21.15.2:0/1160272092 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0050038d50 0x7f005003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.706+0000 7f0064ff9700 1 -- 172.21.15.2:0/1160272092 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f005804d000 con 0x7f00681024a0 2024-03-15T12:49:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.706+0000 7f0066ffd700 1 --2- 172.21.15.2:0/1160272092 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0050038d50 0x7f005003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.706+0000 7f0066ffd700 1 --2- 172.21.15.2:0/1160272092 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0050038d50 0x7f005003b210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f005c006fd0 tx=0x7f005c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.711+0000 7f0064ff9700 1 -- 172.21.15.2:0/1160272092 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0058020de0 con 0x7f00681024a0 2024-03-15T12:49:30.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.915+0000 7f006dfe1700 1 -- 172.21.15.2:0/1160272092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f0068063610 con 0x7f00681024a0 2024-03-15T12:49:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.918+0000 7f0064ff9700 1 -- 172.21.15.2:0/1160272092 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0058027070 con 0x7f00681024a0 2024-03-15T12:49:30.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.920+0000 7f006dfe1700 1 -- 172.21.15.2:0/1160272092 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0050038d50 msgr2=0x7f005003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:30.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.920+0000 7f006dfe1700 1 --2- 172.21.15.2:0/1160272092 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0050038d50 0x7f005003b210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f005c006fd0 tx=0x7f005c006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.920+0000 7f006dfe1700 1 -- 172.21.15.2:0/1160272092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00681024a0 msgr2=0x7f0068196ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.920+0000 7f006dfe1700 1 --2- 172.21.15.2:0/1160272092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00681024a0 0x7f0068196ee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0058006010 tx=0x7f00580057b0 comp rx=0 tx=0).stop 2024-03-15T12:49:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.920+0000 7f006dfe1700 1 -- 172.21.15.2:0/1160272092 shutdown_connections 2024-03-15T12:49:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.920+0000 7f006dfe1700 1 --2- 172.21.15.2:0/1160272092 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0050038d50 0x7f005003b210 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.920+0000 7f006dfe1700 1 --2- 172.21.15.2:0/1160272092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00681024a0 0x7f0068196ee0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.920+0000 7f006dfe1700 1 -- 172.21.15.2:0/1160272092 >> 172.21.15.2:0/1160272092 conn(0x7f00680fbeb0 msgr2=0x7f00680fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.921+0000 7f006dfe1700 1 -- 172.21.15.2:0/1160272092 shutdown_connections 2024-03-15T12:49:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:30.921+0000 7f006dfe1700 1 -- 172.21.15.2:0/1160272092 wait complete. 2024-03-15T12:49:31.648 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-15T12:49:31.648 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-15T12:49:32.191 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:31 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1160272092' entity='client.admin' 2024-03-15T12:49:32.191 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:31 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:49:32.191 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:31 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:32.322 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:49:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.768+0000 7f821329e700 1 -- 172.21.15.2:0/4143723552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f820c101a30 msgr2=0x7f820c101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.768+0000 7f821329e700 1 --2- 172.21.15.2:0/4143723552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f820c101a30 0x7f820c101e50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8208009910 tx=0x7f8208009c20 comp rx=0 tx=0).stop 2024-03-15T12:49:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.768+0000 7f821329e700 1 -- 172.21.15.2:0/4143723552 shutdown_connections 2024-03-15T12:49:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.768+0000 7f821329e700 1 --2- 172.21.15.2:0/4143723552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f820c101a30 0x7f820c101e50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.769+0000 7f821329e700 1 -- 172.21.15.2:0/4143723552 >> 172.21.15.2:0/4143723552 conn(0x7f820c0fcfd0 msgr2=0x7f820c0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.769+0000 7f821329e700 1 -- 172.21.15.2:0/4143723552 shutdown_connections 2024-03-15T12:49:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.769+0000 7f821329e700 1 -- 172.21.15.2:0/4143723552 wait complete. 2024-03-15T12:49:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.769+0000 7f821329e700 1 Processor -- start 2024-03-15T12:49:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.769+0000 7f821329e700 1 -- start start 2024-03-15T12:49:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.769+0000 7f821329e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f820c101a30 0x7f820c196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.769+0000 7f821329e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c1974d0 con 0x7f820c101a30 2024-03-15T12:49:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.770+0000 7f821103a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f820c101a30 0x7f820c196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.770+0000 7f821103a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f820c101a30 0x7f820c196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51172/0 (socket says 172.21.15.2:51172) 2024-03-15T12:49:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.770+0000 7f821103a700 1 -- 172.21.15.2:0/1087533751 learned_addr learned my addr 172.21.15.2:0/1087533751 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.770+0000 7f821103a700 1 -- 172.21.15.2:0/1087533751 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82080095f0 con 0x7f820c101a30 2024-03-15T12:49:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.770+0000 7f821103a700 1 --2- 172.21.15.2:0/1087533751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f820c101a30 0x7f820c196f90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8208004e90 tx=0x7f8208004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:33.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.770+0000 7f82027fc700 1 -- 172.21.15.2:0/1087533751 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f820801d070 con 0x7f820c101a30 2024-03-15T12:49:33.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.770+0000 7f821329e700 1 -- 172.21.15.2:0/1087533751 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820c1976d0 con 0x7f820c101a30 2024-03-15T12:49:33.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.770+0000 7f82027fc700 1 -- 172.21.15.2:0/1087533751 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f820800be30 con 0x7f820c101a30 2024-03-15T12:49:33.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.770+0000 7f82027fc700 1 -- 172.21.15.2:0/1087533751 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8208022e10 con 0x7f820c101a30 2024-03-15T12:49:33.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.770+0000 7f821329e700 1 -- 172.21.15.2:0/1087533751 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c197b70 con 0x7f820c101a30 2024-03-15T12:49:33.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.771+0000 7f82027fc700 1 -- 172.21.15.2:0/1087533751 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f820800f540 con 0x7f820c101a30 2024-03-15T12:49:33.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.771+0000 7f82027fc700 1 --2- 172.21.15.2:0/1087533751 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f81f8038d00 0x7f81f803b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:33.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.771+0000 7f82027fc700 1 -- 172.21.15.2:0/1087533751 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f820804e070 con 0x7f820c101a30 2024-03-15T12:49:33.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.771+0000 7f8210839700 1 --2- 172.21.15.2:0/1087533751 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f81f8038d00 0x7f81f803b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:33.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.771+0000 7f821329e700 1 -- 172.21.15.2:0/1087533751 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820c190880 con 0x7f820c101a30 2024-03-15T12:49:33.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.772+0000 7f8210839700 1 --2- 172.21.15.2:0/1087533751 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f81f8038d00 0x7f81f803b1c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f81fc006fd0 tx=0x7f81fc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:33.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.776+0000 7f82027fc700 1 -- 172.21.15.2:0/1087533751 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8208027020 con 0x7f820c101a30 2024-03-15T12:49:33.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.985+0000 7f821329e700 1 -- 172.21.15.2:0/1087533751 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f820c062420 con 0x7f81f8038d00 2024-03-15T12:49:33.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.988+0000 7f82027fc700 1 -- 172.21.15.2:0/1087533751 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f820c062420 con 0x7f81f8038d00 2024-03-15T12:49:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.990+0000 7f821329e700 1 -- 172.21.15.2:0/1087533751 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f81f8038d00 msgr2=0x7f81f803b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.990+0000 7f821329e700 1 --2- 172.21.15.2:0/1087533751 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f81f8038d00 0x7f81f803b1c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f81fc006fd0 tx=0x7f81fc006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.990+0000 7f821329e700 1 -- 172.21.15.2:0/1087533751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f820c101a30 msgr2=0x7f820c196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.990+0000 7f821329e700 1 --2- 172.21.15.2:0/1087533751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f820c101a30 0x7f820c196f90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8208004e90 tx=0x7f8208004f70 comp rx=0 tx=0).stop 2024-03-15T12:49:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.990+0000 7f821329e700 1 -- 172.21.15.2:0/1087533751 shutdown_connections 2024-03-15T12:49:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.990+0000 7f821329e700 1 --2- 172.21.15.2:0/1087533751 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f81f8038d00 0x7f81f803b1c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.990+0000 7f821329e700 1 --2- 172.21.15.2:0/1087533751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f820c101a30 0x7f820c196f90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.990+0000 7f821329e700 1 -- 172.21.15.2:0/1087533751 >> 172.21.15.2:0/1087533751 conn(0x7f820c0fcfd0 msgr2=0x7f820c0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.991+0000 7f821329e700 1 -- 172.21.15.2:0/1087533751 shutdown_connections 2024-03-15T12:49:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:33.991+0000 7f821329e700 1 -- 172.21.15.2:0/1087533751 wait complete. 2024-03-15T12:49:34.693 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi149 2024-03-15T12:49:34.693 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-15T12:49:34.693 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.conf 2024-03-15T12:49:34.713 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-15T12:49:34.713 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:49:34.774 INFO:tasks.cephadm:Adding host smithi149 to orchestrator... 2024-03-15T12:49:34.774 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch host add smithi149 2024-03-15T12:49:35.014 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:49:35.231 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:34 smithi002 ceph-mon[32060]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:49:35.231 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:35.232 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:49:35.232 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:49:35.232 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:49:35.232 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:34 smithi002 ceph-mon[32060]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-15T12:49:35.232 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:34 smithi002 ceph-mon[32060]: Updating smithi002:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:49:35.232 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:34 smithi002 ceph-mon[32060]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:49:35.232 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:34 smithi002 ceph-mon[32060]: Updating smithi002:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-15T12:49:35.232 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:35.232 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:35.232 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:36.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.383+0000 7fcc951a0700 1 -- 172.21.15.2:0/3592715974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc901079c0 msgr2=0x7fcc90107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:36.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.383+0000 7fcc951a0700 1 --2- 172.21.15.2:0/3592715974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc901079c0 0x7fcc90107da0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800b0a0 tx=0x7fcc8800b3b0 comp rx=0 tx=0).stop 2024-03-15T12:49:36.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.383+0000 7fcc951a0700 1 -- 172.21.15.2:0/3592715974 shutdown_connections 2024-03-15T12:49:36.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.383+0000 7fcc951a0700 1 --2- 172.21.15.2:0/3592715974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc901079c0 0x7fcc90107da0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:36.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.383+0000 7fcc951a0700 1 -- 172.21.15.2:0/3592715974 >> 172.21.15.2:0/3592715974 conn(0x7fcc900fd4f0 msgr2=0x7fcc900ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:36.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.384+0000 7fcc951a0700 1 -- 172.21.15.2:0/3592715974 shutdown_connections 2024-03-15T12:49:36.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.384+0000 7fcc951a0700 1 -- 172.21.15.2:0/3592715974 wait complete. 2024-03-15T12:49:36.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.384+0000 7fcc951a0700 1 Processor -- start 2024-03-15T12:49:36.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.385+0000 7fcc951a0700 1 -- start start 2024-03-15T12:49:36.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.385+0000 7fcc951a0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc901079c0 0x7fcc90197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:36.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.385+0000 7fcc951a0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc90197880 con 0x7fcc901079c0 2024-03-15T12:49:36.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.385+0000 7fcc8ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc901079c0 0x7fcc90197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:36.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.385+0000 7fcc8ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc901079c0 0x7fcc90197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51188/0 (socket says 172.21.15.2:51188) 2024-03-15T12:49:36.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.385+0000 7fcc8ed9d700 1 -- 172.21.15.2:0/3866061907 learned_addr learned my addr 172.21.15.2:0/3866061907 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:36.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.385+0000 7fcc8ed9d700 1 -- 172.21.15.2:0/3866061907 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc88009d00 con 0x7fcc901079c0 2024-03-15T12:49:36.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.385+0000 7fcc8ed9d700 1 --2- 172.21.15.2:0/3866061907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc901079c0 0x7fcc90197340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcc88006130 tx=0x7fcc8800bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:36.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.385+0000 7fcc87fff700 1 -- 172.21.15.2:0/3866061907 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc8800e070 con 0x7fcc901079c0 2024-03-15T12:49:36.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.386+0000 7fcc951a0700 1 -- 172.21.15.2:0/3866061907 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc90197a80 con 0x7fcc901079c0 2024-03-15T12:49:36.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.386+0000 7fcc87fff700 1 -- 172.21.15.2:0/3866061907 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcc88003c10 con 0x7fcc901079c0 2024-03-15T12:49:36.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.386+0000 7fcc87fff700 1 -- 172.21.15.2:0/3866061907 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc880125c0 con 0x7fcc901079c0 2024-03-15T12:49:36.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.386+0000 7fcc951a0700 1 -- 172.21.15.2:0/3866061907 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc90197f00 con 0x7fcc901079c0 2024-03-15T12:49:36.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.387+0000 7fcc87fff700 1 -- 172.21.15.2:0/3866061907 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7fcc88019040 con 0x7fcc901079c0 2024-03-15T12:49:36.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.387+0000 7fcc87fff700 1 --2- 172.21.15.2:0/3866061907 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcc7c038d50 0x7fcc7c03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:36.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.387+0000 7fcc951a0700 1 -- 172.21.15.2:0/3866061907 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc90063610 con 0x7fcc901079c0 2024-03-15T12:49:36.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.387+0000 7fcc87fff700 1 -- 172.21.15.2:0/3866061907 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcc8804c8a0 con 0x7fcc901079c0 2024-03-15T12:49:36.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.387+0000 7fcc8e59c700 1 --2- 172.21.15.2:0/3866061907 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcc7c038d50 0x7fcc7c03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:36.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.387+0000 7fcc8e59c700 1 --2- 172.21.15.2:0/3866061907 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcc7c038d50 0x7fcc7c03b210 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcc78006fd0 tx=0x7fcc78006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:36.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.392+0000 7fcc87fff700 1 -- 172.21.15.2:0/3866061907 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcc8801f020 con 0x7fcc901079c0 2024-03-15T12:49:36.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:36.602+0000 7fcc951a0700 1 -- 172.21.15.2:0/3866061907 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc9010be90 con 0x7fcc7c038d50 2024-03-15T12:49:37.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:36 smithi002 ceph-mon[32060]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:49:38.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:37 smithi002 ceph-mon[32060]: Deploying cephadm binary to smithi149 2024-03-15T12:49:40.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:39 smithi002 ceph-mon[32060]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:49:40.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:40.971+0000 7fcc87fff700 1 -- 172.21.15.2:0/3866061907 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fcc9010be90 con 0x7fcc7c038d50 2024-03-15T12:49:40.959 INFO:teuthology.orchestra.run.smithi002.stdout:Added host 'smithi149' with addr '172.21.15.149' 2024-03-15T12:49:40.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:40.973+0000 7fcc951a0700 1 -- 172.21.15.2:0/3866061907 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcc7c038d50 msgr2=0x7fcc7c03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:40.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:40.973+0000 7fcc951a0700 1 --2- 172.21.15.2:0/3866061907 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcc7c038d50 0x7fcc7c03b210 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcc78006fd0 tx=0x7fcc78006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:40.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:40.973+0000 7fcc951a0700 1 -- 172.21.15.2:0/3866061907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc901079c0 msgr2=0x7fcc90197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:40.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:40.973+0000 7fcc951a0700 1 --2- 172.21.15.2:0/3866061907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc901079c0 0x7fcc90197340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcc88006130 tx=0x7fcc8800bc90 comp rx=0 tx=0).stop 2024-03-15T12:49:40.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:40.973+0000 7fcc951a0700 1 -- 172.21.15.2:0/3866061907 shutdown_connections 2024-03-15T12:49:40.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:40.973+0000 7fcc951a0700 1 --2- 172.21.15.2:0/3866061907 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcc7c038d50 0x7fcc7c03b210 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:40.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:40.973+0000 7fcc951a0700 1 --2- 172.21.15.2:0/3866061907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc901079c0 0x7fcc90197340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:40.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:40.973+0000 7fcc951a0700 1 -- 172.21.15.2:0/3866061907 >> 172.21.15.2:0/3866061907 conn(0x7fcc900fd4f0 msgr2=0x7fcc90109420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:40.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:40.973+0000 7fcc951a0700 1 -- 172.21.15.2:0/3866061907 shutdown_connections 2024-03-15T12:49:40.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:40.973+0000 7fcc951a0700 1 -- 172.21.15.2:0/3866061907 wait complete. 2024-03-15T12:49:41.478 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-15T12:49:41.722 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:49:41.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:41 smithi002 ceph-mon[32060]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:49:41.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:41.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:49:41.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:42.708 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:42 smithi002 ceph-mon[32060]: Added host smithi149 2024-03-15T12:49:42.708 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:42 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:43.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.175+0000 7f0436ab3700 1 -- 172.21.15.2:0/1077496946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0430102c10 msgr2=0x7f0430102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:43.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.175+0000 7f0436ab3700 1 --2- 172.21.15.2:0/1077496946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0430102c10 0x7f0430102ff0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0418009910 tx=0x7f0418009c20 comp rx=0 tx=0).stop 2024-03-15T12:49:43.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.175+0000 7f0436ab3700 1 -- 172.21.15.2:0/1077496946 shutdown_connections 2024-03-15T12:49:43.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.175+0000 7f0436ab3700 1 --2- 172.21.15.2:0/1077496946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0430102c10 0x7f0430102ff0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:43.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.175+0000 7f0436ab3700 1 -- 172.21.15.2:0/1077496946 >> 172.21.15.2:0/1077496946 conn(0x7f04300fc420 msgr2=0x7f04300fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:43.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.175+0000 7f0436ab3700 1 -- 172.21.15.2:0/1077496946 shutdown_connections 2024-03-15T12:49:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.175+0000 7f0436ab3700 1 -- 172.21.15.2:0/1077496946 wait complete. 2024-03-15T12:49:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.176+0000 7f0436ab3700 1 Processor -- start 2024-03-15T12:49:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.176+0000 7f0436ab3700 1 -- start start 2024-03-15T12:49:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.176+0000 7f0436ab3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0430102c10 0x7f0430100870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.176+0000 7f0436ab3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0430102760 con 0x7f0430102c10 2024-03-15T12:49:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.176+0000 7f043484f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0430102c10 0x7f0430100870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.176+0000 7f043484f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0430102c10 0x7f0430100870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54752/0 (socket says 172.21.15.2:54752) 2024-03-15T12:49:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.176+0000 7f043484f700 1 -- 172.21.15.2:0/1149512948 learned_addr learned my addr 172.21.15.2:0/1149512948 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.176+0000 7f043484f700 1 -- 172.21.15.2:0/1149512948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04180095f0 con 0x7f0430102c10 2024-03-15T12:49:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.177+0000 7f043484f700 1 --2- 172.21.15.2:0/1149512948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0430102c10 0x7f0430100870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0418006010 tx=0x7f0418005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.177+0000 7f042dffb700 1 -- 172.21.15.2:0/1149512948 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f041801d070 con 0x7f0430102c10 2024-03-15T12:49:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.177+0000 7f042dffb700 1 -- 172.21.15.2:0/1149512948 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f04180044a0 con 0x7f0430102c10 2024-03-15T12:49:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.177+0000 7f0436ab3700 1 -- 172.21.15.2:0/1149512948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0430100db0 con 0x7f0430102c10 2024-03-15T12:49:43.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.177+0000 7f042dffb700 1 -- 172.21.15.2:0/1149512948 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0418003a70 con 0x7f0430102c10 2024-03-15T12:49:43.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.177+0000 7f0436ab3700 1 -- 172.21.15.2:0/1149512948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04301011d0 con 0x7f0430102c10 2024-03-15T12:49:43.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.178+0000 7f042dffb700 1 -- 172.21.15.2:0/1149512948 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f0418003c60 con 0x7f0430102c10 2024-03-15T12:49:43.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.178+0000 7f042dffb700 1 --2- 172.21.15.2:0/1149512948 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f041c038d50 0x7f041c03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:43.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.178+0000 7f0436ab3700 1 -- 172.21.15.2:0/1149512948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0430063610 con 0x7f0430102c10 2024-03-15T12:49:43.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.178+0000 7f042dffb700 1 -- 172.21.15.2:0/1149512948 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f041804cb20 con 0x7f0430102c10 2024-03-15T12:49:43.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.178+0000 7f042ffff700 1 --2- 172.21.15.2:0/1149512948 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f041c038d50 0x7f041c03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:43.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.179+0000 7f042ffff700 1 --2- 172.21.15.2:0/1149512948 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f041c038d50 0x7f041c03b210 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0420006fd0 tx=0x7f0420006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:43.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.183+0000 7f042dffb700 1 -- 172.21.15.2:0/1149512948 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0418005c00 con 0x7f0430102c10 2024-03-15T12:49:43.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.391+0000 7f0436ab3700 1 -- 172.21.15.2:0/1149512948 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f043010bf90 con 0x7f041c038d50 2024-03-15T12:49:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.393+0000 7f042dffb700 1 -- 172.21.15.2:0/1149512948 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f043010bf90 con 0x7f041c038d50 2024-03-15T12:49:43.380 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:49:43.380 INFO:teuthology.orchestra.run.smithi002.stdout:[{"addr": "172.21.15.2", "hostname": "smithi002", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}] 2024-03-15T12:49:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.395+0000 7f0436ab3700 1 -- 172.21.15.2:0/1149512948 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f041c038d50 msgr2=0x7f041c03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.395+0000 7f0436ab3700 1 --2- 172.21.15.2:0/1149512948 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f041c038d50 0x7f041c03b210 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0420006fd0 tx=0x7f0420006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.395+0000 7f0436ab3700 1 -- 172.21.15.2:0/1149512948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0430102c10 msgr2=0x7f0430100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.395+0000 7f0436ab3700 1 --2- 172.21.15.2:0/1149512948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0430102c10 0x7f0430100870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0418006010 tx=0x7f0418005780 comp rx=0 tx=0).stop 2024-03-15T12:49:43.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.395+0000 7f0436ab3700 1 -- 172.21.15.2:0/1149512948 shutdown_connections 2024-03-15T12:49:43.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.395+0000 7f0436ab3700 1 --2- 172.21.15.2:0/1149512948 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f041c038d50 0x7f041c03b210 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:43.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.395+0000 7f0436ab3700 1 --2- 172.21.15.2:0/1149512948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0430102c10 0x7f0430100870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:43.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.395+0000 7f0436ab3700 1 -- 172.21.15.2:0/1149512948 >> 172.21.15.2:0/1149512948 conn(0x7f04300fc420 msgr2=0x7f04301039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:43.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.395+0000 7f0436ab3700 1 -- 172.21.15.2:0/1149512948 shutdown_connections 2024-03-15T12:49:43.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:43.395+0000 7f0436ab3700 1 -- 172.21.15.2:0/1149512948 wait complete. 2024-03-15T12:49:43.956 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:43 smithi002 ceph-mon[32060]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:49:43.956 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:43 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:43.984 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi184 2024-03-15T12:49:43.985 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-15T12:49:43.985 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.conf 2024-03-15T12:49:44.004 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-15T12:49:44.005 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:49:44.062 INFO:tasks.cephadm:Adding host smithi184 to orchestrator... 2024-03-15T12:49:44.062 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch host add smithi184 2024-03-15T12:49:44.307 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:49:44.675 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:44 smithi002 ceph-mon[32060]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:49:45.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.784+0000 7f801ce61700 1 -- 172.21.15.2:0/2016872677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80181068c0 msgr2=0x7f8018106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.784+0000 7f801ce61700 1 --2- 172.21.15.2:0/2016872677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80181068c0 0x7f8018106ca0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8008009910 tx=0x7f8008009c20 comp rx=0 tx=0).stop 2024-03-15T12:49:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.784+0000 7f801ce61700 1 -- 172.21.15.2:0/2016872677 shutdown_connections 2024-03-15T12:49:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.784+0000 7f801ce61700 1 --2- 172.21.15.2:0/2016872677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80181068c0 0x7f8018106ca0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.784+0000 7f801ce61700 1 -- 172.21.15.2:0/2016872677 >> 172.21.15.2:0/2016872677 conn(0x7f80180fc3d0 msgr2=0x7f80180fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.784+0000 7f801ce61700 1 -- 172.21.15.2:0/2016872677 shutdown_connections 2024-03-15T12:49:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.784+0000 7f801ce61700 1 -- 172.21.15.2:0/2016872677 wait complete. 2024-03-15T12:49:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.785+0000 7f801ce61700 1 Processor -- start 2024-03-15T12:49:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.785+0000 7f801ce61700 1 -- start start 2024-03-15T12:49:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.785+0000 7f801ce61700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80181068c0 0x7f80181972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.785+0000 7f801ce61700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8018197810 con 0x7f80181068c0 2024-03-15T12:49:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.785+0000 7f801659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80181068c0 0x7f80181972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.785+0000 7f801659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80181068c0 0x7f80181972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54772/0 (socket says 172.21.15.2:54772) 2024-03-15T12:49:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.785+0000 7f801659c700 1 -- 172.21.15.2:0/2952555855 learned_addr learned my addr 172.21.15.2:0/2952555855 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:45.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.785+0000 7f801659c700 1 -- 172.21.15.2:0/2952555855 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080095f0 con 0x7f80181068c0 2024-03-15T12:49:45.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.786+0000 7f801659c700 1 --2- 172.21.15.2:0/2952555855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80181068c0 0x7f80181972d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8008006010 tx=0x7f8008005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:45.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.786+0000 7f80077fe700 1 -- 172.21.15.2:0/2952555855 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f800801d070 con 0x7f80181068c0 2024-03-15T12:49:45.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.786+0000 7f801ce61700 1 -- 172.21.15.2:0/2952555855 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8018197a70 con 0x7f80181068c0 2024-03-15T12:49:45.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.786+0000 7f80077fe700 1 -- 172.21.15.2:0/2952555855 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f800800bbc0 con 0x7f80181068c0 2024-03-15T12:49:45.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.786+0000 7f80077fe700 1 -- 172.21.15.2:0/2952555855 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8008003920 con 0x7f80181068c0 2024-03-15T12:49:45.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.786+0000 7f801ce61700 1 -- 172.21.15.2:0/2952555855 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8018197ef0 con 0x7f80181068c0 2024-03-15T12:49:45.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.787+0000 7f80077fe700 1 -- 172.21.15.2:0/2952555855 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f8008003be0 con 0x7f80181068c0 2024-03-15T12:49:45.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.787+0000 7f80077fe700 1 --2- 172.21.15.2:0/2952555855 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8000038d50 0x7f800003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:45.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.787+0000 7f80077fe700 1 -- 172.21.15.2:0/2952555855 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f800804da70 con 0x7f80181068c0 2024-03-15T12:49:45.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.787+0000 7f8015d9b700 1 --2- 172.21.15.2:0/2952555855 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8000038d50 0x7f800003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:45.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.787+0000 7f801ce61700 1 -- 172.21.15.2:0/2952555855 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8018102000 con 0x7f80181068c0 2024-03-15T12:49:45.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.787+0000 7f8015d9b700 1 --2- 172.21.15.2:0/2952555855 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8000038d50 0x7f800003b210 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f800c006fd0 tx=0x7f800c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:45.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.792+0000 7f80077fe700 1 -- 172.21.15.2:0/2952555855 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8008026020 con 0x7f80181068c0 2024-03-15T12:49:45.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:45.997+0000 7f801ce61700 1 -- 172.21.15.2:0/2952555855 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}) v1 -- 0x7f801810acf0 con 0x7f8000038d50 2024-03-15T12:49:45.985 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:45 smithi002 ceph-mon[32060]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:49:48.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:47 smithi002 ceph-mon[32060]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:49:48.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:47 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:48.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:47 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:48.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:47 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:48.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:47 smithi002 ceph-mon[32060]: Deploying cephadm binary to smithi184 2024-03-15T12:49:48.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:47 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:48.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:47 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:49:48.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:47 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:49:48.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:47 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:49:48.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:47 smithi002 ceph-mon[32060]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-15T12:49:48.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:47 smithi002 ceph-mon[32060]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:49:49.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:48 smithi002 ceph-mon[32060]: Updating smithi149:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:49:49.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:48 smithi002 ceph-mon[32060]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:49:50.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:49 smithi002 ceph-mon[32060]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:49:50.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:50.303+0000 7f80077fe700 1 -- 172.21.15.2:0/2952555855 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f801810acf0 con 0x7f8000038d50 2024-03-15T12:49:50.291 INFO:teuthology.orchestra.run.smithi002.stdout:Added host 'smithi184' with addr '172.21.15.184' 2024-03-15T12:49:50.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:50.306+0000 7f801ce61700 1 -- 172.21.15.2:0/2952555855 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8000038d50 msgr2=0x7f800003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:50.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:50.306+0000 7f801ce61700 1 --2- 172.21.15.2:0/2952555855 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8000038d50 0x7f800003b210 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f800c006fd0 tx=0x7f800c006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:50.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:50.306+0000 7f801ce61700 1 -- 172.21.15.2:0/2952555855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80181068c0 msgr2=0x7f80181972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:50.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:50.306+0000 7f801ce61700 1 --2- 172.21.15.2:0/2952555855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80181068c0 0x7f80181972d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8008006010 tx=0x7f8008005780 comp rx=0 tx=0).stop 2024-03-15T12:49:50.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:50.306+0000 7f801ce61700 1 -- 172.21.15.2:0/2952555855 shutdown_connections 2024-03-15T12:49:50.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:50.306+0000 7f801ce61700 1 --2- 172.21.15.2:0/2952555855 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8000038d50 0x7f800003b210 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:50.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:50.306+0000 7f801ce61700 1 --2- 172.21.15.2:0/2952555855 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80181068c0 0x7f80181972d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:50.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:50.306+0000 7f801ce61700 1 -- 172.21.15.2:0/2952555855 >> 172.21.15.2:0/2952555855 conn(0x7f80180fc3d0 msgr2=0x7f80181043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:50.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:50.306+0000 7f801ce61700 1 -- 172.21.15.2:0/2952555855 shutdown_connections 2024-03-15T12:49:50.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:50.306+0000 7f801ce61700 1 -- 172.21.15.2:0/2952555855 wait complete. 2024-03-15T12:49:51.043 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:50 smithi002 ceph-mon[32060]: Updating smithi149:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-15T12:49:51.044 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:50 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:51.044 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:50 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:51.044 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:50 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:51.044 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:50 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:51.044 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:50 smithi002 ceph-mon[32060]: Added host smithi184 2024-03-15T12:49:51.044 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:50 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:49:51.044 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:50 smithi002 ceph-mon[32060]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:49:51.044 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:50 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:51.077 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-15T12:49:51.322 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:49:52.474 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:52 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.968+0000 7f74355b4700 1 -- 172.21.15.2:0/252346165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74301068b0 msgr2=0x7f7430106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.968+0000 7f74355b4700 1 --2- 172.21.15.2:0/252346165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74301068b0 0x7f7430106c90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f742000b0a0 tx=0x7f742000b3b0 comp rx=0 tx=0).stop 2024-03-15T12:49:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.968+0000 7f74355b4700 1 -- 172.21.15.2:0/252346165 shutdown_connections 2024-03-15T12:49:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.968+0000 7f74355b4700 1 --2- 172.21.15.2:0/252346165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74301068b0 0x7f7430106c90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:52.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.968+0000 7f74355b4700 1 -- 172.21.15.2:0/252346165 >> 172.21.15.2:0/252346165 conn(0x7f7430077520 msgr2=0x7f74300fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:52.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.968+0000 7f74355b4700 1 -- 172.21.15.2:0/252346165 shutdown_connections 2024-03-15T12:49:52.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.968+0000 7f74355b4700 1 -- 172.21.15.2:0/252346165 wait complete. 2024-03-15T12:49:52.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.968+0000 7f74355b4700 1 Processor -- start 2024-03-15T12:49:52.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.969+0000 7f74355b4700 1 -- start start 2024-03-15T12:49:52.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.969+0000 7f74355b4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74301068b0 0x7f7430197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:52.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.969+0000 7f74355b4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7430197880 con 0x7f74301068b0 2024-03-15T12:49:52.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.969+0000 7f742effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74301068b0 0x7f7430197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:52.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.969+0000 7f742effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74301068b0 0x7f7430197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54206/0 (socket says 172.21.15.2:54206) 2024-03-15T12:49:52.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.969+0000 7f742effd700 1 -- 172.21.15.2:0/1189492686 learned_addr learned my addr 172.21.15.2:0/1189492686 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:52.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.969+0000 7f742effd700 1 -- 172.21.15.2:0/1189492686 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7420009d00 con 0x7f74301068b0 2024-03-15T12:49:52.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.970+0000 7f742effd700 1 --2- 172.21.15.2:0/1189492686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74301068b0 0x7f7430197340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7420015040 tx=0x7f742000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:52.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.970+0000 7f7427fff700 1 -- 172.21.15.2:0/1189492686 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f742000e050 con 0x7f74301068b0 2024-03-15T12:49:52.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.970+0000 7f7427fff700 1 -- 172.21.15.2:0/1189492686 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7420003c10 con 0x7f74301068b0 2024-03-15T12:49:52.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.970+0000 7f74355b4700 1 -- 172.21.15.2:0/1189492686 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7430197a80 con 0x7f74301068b0 2024-03-15T12:49:52.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.970+0000 7f7427fff700 1 -- 172.21.15.2:0/1189492686 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f742001d5b0 con 0x7f74301068b0 2024-03-15T12:49:52.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.970+0000 7f74355b4700 1 -- 172.21.15.2:0/1189492686 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7430197f00 con 0x7f74301068b0 2024-03-15T12:49:52.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.971+0000 7f7427fff700 1 -- 172.21.15.2:0/1189492686 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f7420019040 con 0x7f74301068b0 2024-03-15T12:49:52.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.971+0000 7f74355b4700 1 -- 172.21.15.2:0/1189492686 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7430063610 con 0x7f74301068b0 2024-03-15T12:49:52.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.971+0000 7f7427fff700 1 --2- 172.21.15.2:0/1189492686 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f741c038da0 0x7f741c03b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:52.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.971+0000 7f7427fff700 1 -- 172.21.15.2:0/1189492686 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f742004cbd0 con 0x7f74301068b0 2024-03-15T12:49:52.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.971+0000 7f742e7fc700 1 --2- 172.21.15.2:0/1189492686 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f741c038da0 0x7f741c03b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:52.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.971+0000 7f742e7fc700 1 --2- 172.21.15.2:0/1189492686 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f741c038da0 0x7f741c03b260 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7418006fd0 tx=0x7f7418006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:52.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:52.976+0000 7f7427fff700 1 -- 172.21.15.2:0/1189492686 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7420026c50 con 0x7f74301068b0 2024-03-15T12:49:53.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:53.189+0000 7f74355b4700 1 -- 172.21.15.2:0/1189492686 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f743010ae20 con 0x7f741c038da0 2024-03-15T12:49:53.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:53.190+0000 7f7427fff700 1 -- 172.21.15.2:0/1189492686 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f743010ae20 con 0x7f741c038da0 2024-03-15T12:49:53.178 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:49:53.178 INFO:teuthology.orchestra.run.smithi002.stdout:[{"addr": "172.21.15.2", "hostname": "smithi002", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}, {"addr": "172.21.15.184", "hostname": "smithi184", "labels": [], "status": ""}] 2024-03-15T12:49:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:53.192+0000 7f74355b4700 1 -- 172.21.15.2:0/1189492686 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f741c038da0 msgr2=0x7f741c03b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:53.192+0000 7f74355b4700 1 --2- 172.21.15.2:0/1189492686 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f741c038da0 0x7f741c03b260 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7418006fd0 tx=0x7f7418006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:53.192+0000 7f74355b4700 1 -- 172.21.15.2:0/1189492686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74301068b0 msgr2=0x7f7430197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:53.192+0000 7f74355b4700 1 --2- 172.21.15.2:0/1189492686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74301068b0 0x7f7430197340 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7420015040 tx=0x7f742000bd40 comp rx=0 tx=0).stop 2024-03-15T12:49:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:53.192+0000 7f74355b4700 1 -- 172.21.15.2:0/1189492686 shutdown_connections 2024-03-15T12:49:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:53.192+0000 7f74355b4700 1 --2- 172.21.15.2:0/1189492686 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f741c038da0 0x7f741c03b260 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:53.192+0000 7f74355b4700 1 --2- 172.21.15.2:0/1189492686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74301068b0 0x7f7430197340 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:53.192+0000 7f74355b4700 1 -- 172.21.15.2:0/1189492686 >> 172.21.15.2:0/1189492686 conn(0x7f7430077520 msgr2=0x7f74300693d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:53.192+0000 7f74355b4700 1 -- 172.21.15.2:0/1189492686 shutdown_connections 2024-03-15T12:49:53.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:53.192+0000 7f74355b4700 1 -- 172.21.15.2:0/1189492686 wait complete. 2024-03-15T12:49:53.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:53 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:53.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:53 smithi002 ceph-mon[32060]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:49:53.897 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-15T12:49:53.897 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-15T12:49:54.144 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:49:54.542 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:54 smithi002 ceph-mon[32060]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:49:55.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.534+0000 7fcc554a4700 1 -- 172.21.15.2:0/1952230133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc50102b70 msgr2=0x7fcc50102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:55.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.534+0000 7fcc554a4700 1 --2- 172.21.15.2:0/1952230133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc50102b70 0x7fcc50102f50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009910 tx=0x7fcc40009c20 comp rx=0 tx=0).stop 2024-03-15T12:49:55.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.534+0000 7fcc554a4700 1 -- 172.21.15.2:0/1952230133 shutdown_connections 2024-03-15T12:49:55.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.534+0000 7fcc554a4700 1 --2- 172.21.15.2:0/1952230133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc50102b70 0x7fcc50102f50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:55.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.534+0000 7fcc554a4700 1 -- 172.21.15.2:0/1952230133 >> 172.21.15.2:0/1952230133 conn(0x7fcc50077520 msgr2=0x7fcc500fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:55.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.534+0000 7fcc554a4700 1 -- 172.21.15.2:0/1952230133 shutdown_connections 2024-03-15T12:49:55.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.534+0000 7fcc554a4700 1 -- 172.21.15.2:0/1952230133 wait complete. 2024-03-15T12:49:55.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.534+0000 7fcc554a4700 1 Processor -- start 2024-03-15T12:49:55.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.535+0000 7fcc554a4700 1 -- start start 2024-03-15T12:49:55.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.535+0000 7fcc554a4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc50102b70 0x7fcc501007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:55.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.535+0000 7fcc554a4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc50102670 con 0x7fcc50102b70 2024-03-15T12:49:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.535+0000 7fcc4effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc50102b70 0x7fcc501007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.535+0000 7fcc4effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc50102b70 0x7fcc501007d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54224/0 (socket says 172.21.15.2:54224) 2024-03-15T12:49:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.535+0000 7fcc4effd700 1 -- 172.21.15.2:0/2337029896 learned_addr learned my addr 172.21.15.2:0/2337029896 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:49:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.535+0000 7fcc4effd700 1 -- 172.21.15.2:0/2337029896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc400095f0 con 0x7fcc50102b70 2024-03-15T12:49:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.536+0000 7fcc4effd700 1 --2- 172.21.15.2:0/2337029896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc50102b70 0x7fcc501007d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcc40006010 tx=0x7fcc40005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.536+0000 7fcc37fff700 1 -- 172.21.15.2:0/2337029896 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc4001d070 con 0x7fcc50102b70 2024-03-15T12:49:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.536+0000 7fcc37fff700 1 -- 172.21.15.2:0/2337029896 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcc40004370 con 0x7fcc50102b70 2024-03-15T12:49:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.536+0000 7fcc554a4700 1 -- 172.21.15.2:0/2337029896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc50100d10 con 0x7fcc50102b70 2024-03-15T12:49:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.536+0000 7fcc37fff700 1 -- 172.21.15.2:0/2337029896 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc40003a30 con 0x7fcc50102b70 2024-03-15T12:49:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.536+0000 7fcc554a4700 1 -- 172.21.15.2:0/2337029896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc50101130 con 0x7fcc50102b70 2024-03-15T12:49:55.525 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:55 smithi002 ceph-mon[32060]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:49:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.537+0000 7fcc37fff700 1 -- 172.21.15.2:0/2337029896 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7fcc40003cf0 con 0x7fcc50102b70 2024-03-15T12:49:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.537+0000 7fcc554a4700 1 -- 172.21.15.2:0/2337029896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc50069cb0 con 0x7fcc50102b70 2024-03-15T12:49:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.537+0000 7fcc37fff700 1 --2- 172.21.15.2:0/2337029896 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcc38038d50 0x7fcc3803b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.537+0000 7fcc37fff700 1 -- 172.21.15.2:0/2337029896 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcc4004cca0 con 0x7fcc50102b70 2024-03-15T12:49:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.537+0000 7fcc4e7fc700 1 --2- 172.21.15.2:0/2337029896 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcc38038d50 0x7fcc3803b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.538+0000 7fcc4e7fc700 1 --2- 172.21.15.2:0/2337029896 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcc38038d50 0x7fcc3803b210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcc44006fd0 tx=0x7fcc44006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.542+0000 7fcc37fff700 1 -- 172.21.15.2:0/2337029896 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcc40029360 con 0x7fcc50102b70 2024-03-15T12:49:55.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:55.752+0000 7fcc554a4700 1 -- 172.21.15.2:0/2337029896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fcc501013e0 con 0x7fcc50102b70 2024-03-15T12:49:56.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:56.402+0000 7fcc37fff700 1 -- 172.21.15.2:0/2337029896 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fcc40026020 con 0x7fcc50102b70 2024-03-15T12:49:56.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:56.404+0000 7fcc554a4700 1 -- 172.21.15.2:0/2337029896 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcc38038d50 msgr2=0x7fcc3803b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:56.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:56.404+0000 7fcc554a4700 1 --2- 172.21.15.2:0/2337029896 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcc38038d50 0x7fcc3803b210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcc44006fd0 tx=0x7fcc44006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:56.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:56.404+0000 7fcc554a4700 1 -- 172.21.15.2:0/2337029896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc50102b70 msgr2=0x7fcc501007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:56.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:56.404+0000 7fcc554a4700 1 --2- 172.21.15.2:0/2337029896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc50102b70 0x7fcc501007d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcc40006010 tx=0x7fcc40005780 comp rx=0 tx=0).stop 2024-03-15T12:49:56.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:56.404+0000 7fcc554a4700 1 -- 172.21.15.2:0/2337029896 shutdown_connections 2024-03-15T12:49:56.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:56.404+0000 7fcc554a4700 1 --2- 172.21.15.2:0/2337029896 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcc38038d50 0x7fcc3803b210 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:56.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:56.404+0000 7fcc554a4700 1 --2- 172.21.15.2:0/2337029896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc50102b70 0x7fcc501007d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:56.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:56.404+0000 7fcc554a4700 1 -- 172.21.15.2:0/2337029896 >> 172.21.15.2:0/2337029896 conn(0x7fcc50077520 msgr2=0x7fcc50103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:56.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:56.404+0000 7fcc554a4700 1 -- 172.21.15.2:0/2337029896 shutdown_connections 2024-03-15T12:49:56.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:49:56.404+0000 7fcc554a4700 1 -- 172.21.15.2:0/2337029896 wait complete. 2024-03-15T12:49:56.392 INFO:teuthology.orchestra.run.smithi002.stderr:adjusted tunables profile to default 2024-03-15T12:49:56.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:56 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:56.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:56 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:56.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:56 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:56.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:56 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:56.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:56 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:49:56.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:56 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:49:56.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:56 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:49:56.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:56 smithi002 ceph-mon[32060]: Updating smithi184:/etc/ceph/ceph.conf 2024-03-15T12:49:56.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:56 smithi002 ceph-mon[32060]: Updating smithi184:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:49:56.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:56 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2337029896' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-15T12:49:56.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:56 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:56.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:56 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:56.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:56 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:49:57.076 INFO:tasks.cephadm:Adding mon.a on smithi002 2024-03-15T12:49:57.076 INFO:tasks.cephadm:Adding mon.b on smithi149 2024-03-15T12:49:57.077 INFO:tasks.cephadm:Adding mon.c on smithi184 2024-03-15T12:49:57.077 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch apply mon '3;smithi002:172.21.15.2=a;smithi149:172.21.15.149=b;smithi184:172.21.15.184=c' 2024-03-15T12:49:57.329 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:49:57.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:57 smithi002 ceph-mon[32060]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:49:57.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:57 smithi002 ceph-mon[32060]: Updating smithi184:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-15T12:49:57.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:57 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2337029896' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-15T12:49:57.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:57 smithi002 ceph-mon[32060]: osdmap e4: 0 total, 0 up, 0 in 2024-03-15T12:49:57.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:57 smithi002 ceph-mon[32060]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:49:58.632 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.643+0000 7f0d28fdf700 1 -- 172.21.15.184:0/1493992744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d24102bc0 msgr2=0x7f0d24102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:58.632 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.643+0000 7f0d28fdf700 1 --2- 172.21.15.184:0/1493992744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d24102bc0 0x7f0d24102fa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009910 tx=0x7f0d18009c20 comp rx=0 tx=0).stop 2024-03-15T12:49:58.632 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.643+0000 7f0d28fdf700 1 -- 172.21.15.184:0/1493992744 shutdown_connections 2024-03-15T12:49:58.632 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.643+0000 7f0d28fdf700 1 --2- 172.21.15.184:0/1493992744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d24102bc0 0x7f0d24102fa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:58.632 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.643+0000 7f0d28fdf700 1 -- 172.21.15.184:0/1493992744 >> 172.21.15.184:0/1493992744 conn(0x7f0d240fc3b0 msgr2=0x7f0d240fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:58.632 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.643+0000 7f0d28fdf700 1 -- 172.21.15.184:0/1493992744 shutdown_connections 2024-03-15T12:49:58.632 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.643+0000 7f0d28fdf700 1 -- 172.21.15.184:0/1493992744 wait complete. 2024-03-15T12:49:58.632 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.644+0000 7f0d28fdf700 1 Processor -- start 2024-03-15T12:49:58.632 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.644+0000 7f0d28fdf700 1 -- start start 2024-03-15T12:49:58.632 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.644+0000 7f0d28fdf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d24102bc0 0x7f0d241973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:58.632 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.644+0000 7f0d28fdf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d24197910 con 0x7f0d24102bc0 2024-03-15T12:49:58.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.644+0000 7f0d2259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d24102bc0 0x7f0d241973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:58.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.644+0000 7f0d2259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d24102bc0 0x7f0d241973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.184:54294/0 (socket says 172.21.15.184:54294) 2024-03-15T12:49:58.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.644+0000 7f0d2259c700 1 -- 172.21.15.184:0/3732547197 learned_addr learned my addr 172.21.15.184:0/3732547197 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-15T12:49:58.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.644+0000 7f0d2259c700 1 -- 172.21.15.184:0/3732547197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d180095f0 con 0x7f0d24102bc0 2024-03-15T12:49:58.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.645+0000 7f0d2259c700 1 --2- 172.21.15.184:0/3732547197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d24102bc0 0x7f0d241973d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0d18006010 tx=0x7f0d18005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:58.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.645+0000 7f0d137fe700 1 -- 172.21.15.184:0/3732547197 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d1801d070 con 0x7f0d24102bc0 2024-03-15T12:49:58.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.645+0000 7f0d28fdf700 1 -- 172.21.15.184:0/3732547197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d24197b10 con 0x7f0d24102bc0 2024-03-15T12:49:58.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.645+0000 7f0d137fe700 1 -- 172.21.15.184:0/3732547197 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0d180044a0 con 0x7f0d24102bc0 2024-03-15T12:49:58.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.645+0000 7f0d137fe700 1 -- 172.21.15.184:0/3732547197 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d18003a70 con 0x7f0d24102bc0 2024-03-15T12:49:58.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.645+0000 7f0d28fdf700 1 -- 172.21.15.184:0/3732547197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d24197f30 con 0x7f0d24102bc0 2024-03-15T12:49:58.635 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.646+0000 7f0d137fe700 1 -- 172.21.15.184:0/3732547197 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f0d18003c60 con 0x7f0d24102bc0 2024-03-15T12:49:58.635 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.646+0000 7f0d28fdf700 1 -- 172.21.15.184:0/3732547197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2406a6c0 con 0x7f0d24102bc0 2024-03-15T12:49:58.635 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.646+0000 7f0d137fe700 1 --2- 172.21.15.184:0/3732547197 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0d0c038d50 0x7f0d0c03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:49:58.635 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.646+0000 7f0d137fe700 1 -- 172.21.15.184:0/3732547197 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0d1804cbe0 con 0x7f0d24102bc0 2024-03-15T12:49:58.635 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.646+0000 7f0d21d9b700 1 --2- 172.21.15.184:0/3732547197 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0d0c038d50 0x7f0d0c03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:49:58.635 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.647+0000 7f0d21d9b700 1 --2- 172.21.15.184:0/3732547197 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0d0c038d50 0x7f0d0c03b210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0d14006fd0 tx=0x7f0d14006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:49:58.639 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.651+0000 7f0d137fe700 1 -- 172.21.15.184:0/3732547197 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0d18004610 con 0x7f0d24102bc0 2024-03-15T12:49:58.844 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.856+0000 7f0d28fdf700 1 -- 172.21.15.184:0/3732547197 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi149:172.21.15.149=b;smithi184:172.21.15.184=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d2410bf10 con 0x7f0d0c038d50 2024-03-15T12:49:58.850 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.862+0000 7f0d137fe700 1 -- 172.21.15.184:0/3732547197 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0d2410bf10 con 0x7f0d0c038d50 2024-03-15T12:49:58.850 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mon update... 2024-03-15T12:49:58.853 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.865+0000 7f0d28fdf700 1 -- 172.21.15.184:0/3732547197 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0d0c038d50 msgr2=0x7f0d0c03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:58.853 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.865+0000 7f0d28fdf700 1 --2- 172.21.15.184:0/3732547197 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0d0c038d50 0x7f0d0c03b210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0d14006fd0 tx=0x7f0d14006e40 comp rx=0 tx=0).stop 2024-03-15T12:49:58.853 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.865+0000 7f0d28fdf700 1 -- 172.21.15.184:0/3732547197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d24102bc0 msgr2=0x7f0d241973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:49:58.853 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.865+0000 7f0d28fdf700 1 --2- 172.21.15.184:0/3732547197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d24102bc0 0x7f0d241973d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0d18006010 tx=0x7f0d18005780 comp rx=0 tx=0).stop 2024-03-15T12:49:58.853 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.865+0000 7f0d28fdf700 1 -- 172.21.15.184:0/3732547197 shutdown_connections 2024-03-15T12:49:58.853 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.865+0000 7f0d28fdf700 1 --2- 172.21.15.184:0/3732547197 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0d0c038d50 0x7f0d0c03b210 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:58.853 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.865+0000 7f0d28fdf700 1 --2- 172.21.15.184:0/3732547197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d24102bc0 0x7f0d241973d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:49:58.853 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.865+0000 7f0d28fdf700 1 -- 172.21.15.184:0/3732547197 >> 172.21.15.184:0/3732547197 conn(0x7f0d240fc3b0 msgr2=0x7f0d24103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:49:58.853 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.865+0000 7f0d28fdf700 1 -- 172.21.15.184:0/3732547197 shutdown_connections 2024-03-15T12:49:58.853 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:49:58.865+0000 7f0d28fdf700 1 -- 172.21.15.184:0/3732547197 wait complete. 2024-03-15T12:49:59.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:58 smithi002 ceph-mon[32060]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:49:59.443 DEBUG:teuthology.orchestra.run.smithi149:mon.b> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mon.b.service 2024-03-15T12:49:59.446 DEBUG:teuthology.orchestra.run.smithi184:mon.c> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mon.c.service 2024-03-15T12:49:59.448 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-15T12:49:59.449 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-15T12:50:00.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:59 smithi002 ceph-mon[32060]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi149:172.21.15.149=b;smithi184:172.21.15.184=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:50:00.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:59 smithi002 ceph-mon[32060]: Saving service mon spec with placement smithi002:172.21.15.2=a;smithi149:172.21.15.149=b;smithi184:172.21.15.184=c;count:3 2024-03-15T12:50:00.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:00.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:00.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:00.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:00.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:00.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-15T12:50:00.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:00.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:49:59 smithi002 ceph-mon[32060]: Deploying daemon mon.c on smithi184 2024-03-15T12:50:01.074 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:50:01.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:00 smithi002 ceph-mon[32060]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:03.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:02 smithi002 ceph-mon[32060]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:03.444 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.454+0000 7fa2e5a06700 1 -- 172.21.15.184:0/3069141494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e01068b0 msgr2=0x7fa2e0106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:03.445 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.454+0000 7fa2e5a06700 1 --2- 172.21.15.184:0/3069141494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e01068b0 0x7fa2e0106c90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009910 tx=0x7fa2d0009c20 comp rx=0 tx=0).stop 2024-03-15T12:50:03.445 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.454+0000 7fa2e5a06700 1 -- 172.21.15.184:0/3069141494 shutdown_connections 2024-03-15T12:50:03.445 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.454+0000 7fa2e5a06700 1 --2- 172.21.15.184:0/3069141494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e01068b0 0x7fa2e0106c90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:03.445 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.454+0000 7fa2e5a06700 1 -- 172.21.15.184:0/3069141494 >> 172.21.15.184:0/3069141494 conn(0x7fa2e00fc3c0 msgr2=0x7fa2e00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:50:03.445 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.454+0000 7fa2e5a06700 1 -- 172.21.15.184:0/3069141494 shutdown_connections 2024-03-15T12:50:03.445 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.454+0000 7fa2e5a06700 1 -- 172.21.15.184:0/3069141494 wait complete. 2024-03-15T12:50:03.445 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.455+0000 7fa2e5a06700 1 Processor -- start 2024-03-15T12:50:03.445 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.455+0000 7fa2e5a06700 1 -- start start 2024-03-15T12:50:03.445 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.455+0000 7fa2e5a06700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e01068b0 0x7fa2e0197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:03.445 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.455+0000 7fa2e5a06700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e01978c0 con 0x7fa2e01068b0 2024-03-15T12:50:03.445 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.455+0000 7fa2deffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e01068b0 0x7fa2e0197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:03.445 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.455+0000 7fa2deffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e01068b0 0x7fa2e0197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.184:49958/0 (socket says 172.21.15.184:49958) 2024-03-15T12:50:03.446 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.455+0000 7fa2deffd700 1 -- 172.21.15.184:0/2978851932 learned_addr learned my addr 172.21.15.184:0/2978851932 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-15T12:50:03.446 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.456+0000 7fa2deffd700 1 -- 172.21.15.184:0/2978851932 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d00095f0 con 0x7fa2e01068b0 2024-03-15T12:50:03.446 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.456+0000 7fa2deffd700 1 --2- 172.21.15.184:0/2978851932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e01068b0 0x7fa2e0197380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0006010 tx=0x7fa2d0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:50:03.446 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.456+0000 7fa2e4a04700 1 -- 172.21.15.184:0/2978851932 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2d001d070 con 0x7fa2e01068b0 2024-03-15T12:50:03.446 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.456+0000 7fa2e5a06700 1 -- 172.21.15.184:0/2978851932 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e0197b20 con 0x7fa2e01068b0 2024-03-15T12:50:03.446 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.456+0000 7fa2e5a06700 1 -- 172.21.15.184:0/2978851932 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e0197fa0 con 0x7fa2e01068b0 2024-03-15T12:50:03.446 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.456+0000 7fa2e4a04700 1 -- 172.21.15.184:0/2978851932 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa2d000bbc0 con 0x7fa2e01068b0 2024-03-15T12:50:03.446 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.456+0000 7fa2e4a04700 1 -- 172.21.15.184:0/2978851932 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2d0003830 con 0x7fa2e01068b0 2024-03-15T12:50:03.447 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.457+0000 7fa2e5a06700 1 -- 172.21.15.184:0/2978851932 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c0005330 con 0x7fa2e01068b0 2024-03-15T12:50:03.447 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.457+0000 7fa2e4a04700 1 -- 172.21.15.184:0/2978851932 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7fa2d000bd30 con 0x7fa2e01068b0 2024-03-15T12:50:03.447 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.457+0000 7fa2e4a04700 1 --2- 172.21.15.184:0/2978851932 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2cc038d50 0x7fa2cc03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:03.447 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.457+0000 7fa2e4a04700 1 -- 172.21.15.184:0/2978851932 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa2d004cb70 con 0x7fa2e01068b0 2024-03-15T12:50:03.447 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.458+0000 7fa2d7fff700 1 --2- 172.21.15.184:0/2978851932 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2cc038d50 0x7fa2cc03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:03.447 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.458+0000 7fa2d7fff700 1 --2- 172.21.15.184:0/2978851932 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2cc038d50 0x7fa2cc03b210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8006fd0 tx=0x7fa2c8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:50:03.451 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.463+0000 7fa2e4a04700 1 -- 172.21.15.184:0/2978851932 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa2d002a3b0 con 0x7fa2e01068b0 2024-03-15T12:50:03.571 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.583+0000 7fa2e4a04700 1 -- 172.21.15.184:0/2978851932 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa2d0051020 con 0x7fa2e01068b0 2024-03-15T12:50:03.742 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:03.754+0000 7fa2e5a06700 1 -- 172.21.15.184:0/2978851932 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa2c00055e0 con 0x7fa2e01068b0 2024-03-15T12:50:08.618 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:08.631+0000 7fa2e4a04700 1 -- 172.21.15.184:0/2978851932 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1011 (secure 0 0 0) 0x7fa2d0026070 con 0x7fa2e01068b0 2024-03-15T12:50:08.619 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:50:08.619 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":2,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","modified":"2024-03-15T12:50:03.585102Z","created":"2024-03-15T12:48:17.225697Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-15T12:50:08.622 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:08.634+0000 7fa2e5a06700 1 -- 172.21.15.184:0/2978851932 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2cc038d50 msgr2=0x7fa2cc03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:08.622 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:08.634+0000 7fa2e5a06700 1 --2- 172.21.15.184:0/2978851932 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2cc038d50 0x7fa2cc03b210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8006fd0 tx=0x7fa2c8006e40 comp rx=0 tx=0).stop 2024-03-15T12:50:08.622 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:08.634+0000 7fa2e5a06700 1 -- 172.21.15.184:0/2978851932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e01068b0 msgr2=0x7fa2e0197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:08.622 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:08.634+0000 7fa2e5a06700 1 --2- 172.21.15.184:0/2978851932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e01068b0 0x7fa2e0197380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0006010 tx=0x7fa2d0005780 comp rx=0 tx=0).stop 2024-03-15T12:50:08.622 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:08.634+0000 7fa2e5a06700 1 -- 172.21.15.184:0/2978851932 shutdown_connections 2024-03-15T12:50:08.622 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:08.634+0000 7fa2e5a06700 1 --2- 172.21.15.184:0/2978851932 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2cc038d50 0x7fa2cc03b210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:08.622 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:08.634+0000 7fa2e5a06700 1 --2- 172.21.15.184:0/2978851932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e01068b0 0x7fa2e0197380 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:08.622 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:08.634+0000 7fa2e5a06700 1 -- 172.21.15.184:0/2978851932 >> 172.21.15.184:0/2978851932 conn(0x7fa2e00fc3c0 msgr2=0x7fa2e0069e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:50:08.622 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:08.635+0000 7fa2e5a06700 1 -- 172.21.15.184:0/2978851932 shutdown_connections 2024-03-15T12:50:08.622 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:08.635+0000 7fa2e5a06700 1 -- 172.21.15.184:0/2978851932 wait complete. 2024-03-15T12:50:08.623 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 2 2024-03-15T12:50:08.928 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:08 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:09.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:50:09.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: mon.a calling monitor election 2024-03-15T12:50:09.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:50:09.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: from='client.? 172.21.15.184:0/2978851932' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-15T12:50:09.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:50:09.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:09.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:50:09.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: mon.c calling monitor election 2024-03-15T12:50:09.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:50:09.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:09.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:50:09.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:50:09.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-15T12:50:09.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: monmap e2: 2 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-15T12:50:09.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: fsmap 2024-03-15T12:50:09.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: osdmap e4: 0 total, 0 up, 0 in 2024-03-15T12:50:09.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: mgrmap e13: x(active, since 49s) 2024-03-15T12:50:09.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: overall HEALTH_OK 2024-03-15T12:50:09.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:09.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:09.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:09.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-15T12:50:09.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:09 smithi002 ceph-mon[32060]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:09 smithi002 ceph-mon[32060]: Deploying daemon mon.b on smithi149 2024-03-15T12:50:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:50:10.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:09 smithi184 ceph-mon[35993]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:10.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:09 smithi184 ceph-mon[35993]: Deploying daemon mon.b on smithi149 2024-03-15T12:50:10.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:50:10.238 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-15T12:50:10.238 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-15T12:50:10.477 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T12:50:11.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.892+0000 7f21e7a93700 1 -- 172.21.15.184:0/1140752264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21e01069e0 msgr2=0x7f21e0106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:11.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.892+0000 7f21e7a93700 1 --2- 172.21.15.184:0/1140752264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21e01069e0 0x7f21e0106dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009910 tx=0x7f21d0009c20 comp rx=0 tx=0).stop 2024-03-15T12:50:11.881 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.892+0000 7f21e7a93700 1 -- 172.21.15.184:0/1140752264 shutdown_connections 2024-03-15T12:50:11.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.892+0000 7f21e7a93700 1 --2- 172.21.15.184:0/1140752264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21e01069e0 0x7f21e0106dc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:11.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.892+0000 7f21e7a93700 1 -- 172.21.15.184:0/1140752264 >> 172.21.15.184:0/1140752264 conn(0x7f21e00fc4f0 msgr2=0x7f21e00fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:50:11.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.892+0000 7f21e7a93700 1 -- 172.21.15.184:0/1140752264 shutdown_connections 2024-03-15T12:50:11.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.892+0000 7f21e7a93700 1 -- 172.21.15.184:0/1140752264 wait complete. 2024-03-15T12:50:11.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e7a93700 1 Processor -- start 2024-03-15T12:50:11.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e7a93700 1 -- start start 2024-03-15T12:50:11.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e7a93700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21e01069e0 0x7f21e0197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:11.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e7a93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21e0197ab0 0x7f21e019be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:11.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e7a93700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e0198040 con 0x7f21e0197ab0 2024-03-15T12:50:11.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e7a93700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e01981b0 con 0x7f21e01069e0 2024-03-15T12:50:11.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e582f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21e01069e0 0x7f21e0197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:11.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e582f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21e01069e0 0x7f21e0197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:50608/0 (socket says 172.21.15.184:50608) 2024-03-15T12:50:11.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e582f700 1 -- 172.21.15.184:0/968089220 learned_addr learned my addr 172.21.15.184:0/968089220 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-15T12:50:11.882 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e502e700 1 --2- 172.21.15.184:0/968089220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21e0197ab0 0x7f21e019be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:11.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e582f700 1 -- 172.21.15.184:0/968089220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21e01069e0 msgr2=0x7f21e0197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-15T12:50:11.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e582f700 1 -- 172.21.15.184:0/968089220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21e01069e0 msgr2=0x7f21e0197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-15T12:50:11.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e582f700 1 --2- 172.21.15.184:0/968089220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21e01069e0 0x7f21e0197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-15T12:50:11.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e582f700 1 --2- 172.21.15.184:0/968089220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21e01069e0 0x7f21e0197570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-15T12:50:11.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e502e700 1 -- 172.21.15.184:0/968089220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21e01069e0 msgr2=0x7f21e0197570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-15T12:50:11.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.893+0000 7f21e502e700 1 --2- 172.21.15.184:0/968089220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21e01069e0 0x7f21e0197570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:11.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.894+0000 7f21e502e700 1 -- 172.21.15.184:0/968089220 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d00095f0 con 0x7f21e0197ab0 2024-03-15T12:50:11.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.894+0000 7f21e502e700 1 --2- 172.21.15.184:0/968089220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21e0197ab0 0x7f21e019be40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00b4d0 tx=0x7f21dc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:50:11.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.894+0000 7f21d6ffd700 1 -- 172.21.15.184:0/968089220 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f21dc011630 con 0x7f21e0197ab0 2024-03-15T12:50:11.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.894+0000 7f21e7a93700 1 -- 172.21.15.184:0/968089220 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e019c3e0 con 0x7f21e0197ab0 2024-03-15T12:50:11.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.894+0000 7f21d6ffd700 1 -- 172.21.15.184:0/968089220 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f21dc011c60 con 0x7f21e0197ab0 2024-03-15T12:50:11.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.894+0000 7f21d6ffd700 1 -- 172.21.15.184:0/968089220 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f21dc010d50 con 0x7f21e0197ab0 2024-03-15T12:50:11.883 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.894+0000 7f21e7a93700 1 -- 172.21.15.184:0/968089220 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e019c930 con 0x7f21e0197ab0 2024-03-15T12:50:11.884 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.895+0000 7f21e7a93700 1 -- 172.21.15.184:0/968089220 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e0102120 con 0x7f21e0197ab0 2024-03-15T12:50:11.884 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.895+0000 7f21d6ffd700 1 -- 172.21.15.184:0/968089220 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f21dc010510 con 0x7f21e0197ab0 2024-03-15T12:50:11.884 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.895+0000 7f21d6ffd700 1 --2- 172.21.15.184:0/968089220 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f21cc038af0 0x7f21cc03afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:11.884 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.895+0000 7f21d6ffd700 1 -- 172.21.15.184:0/968089220 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f21dc04b4c0 con 0x7f21e0197ab0 2024-03-15T12:50:11.884 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.895+0000 7f21e582f700 1 --2- 172.21.15.184:0/968089220 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f21cc038af0 0x7f21cc03afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:11.884 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.896+0000 7f21e582f700 1 --2- 172.21.15.184:0/968089220 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f21cc038af0 0x7f21cc03afb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f21d00092e0 tx=0x7f21d000b580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:50:11.887 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:11.900+0000 7f21d6ffd700 1 -- 172.21.15.184:0/968089220 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f21dc011dd0 con 0x7f21e0197ab0 2024-03-15T12:50:12.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:11 smithi002 ceph-mon[32060]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:12.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:11 smithi184 ceph-mon[35993]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:12.158 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:12.171+0000 7f21e7a93700 1 -- 172.21.15.184:0/968089220 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f21e0063610 con 0x7f21e0197ab0 2024-03-15T12:50:12.158 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:12.171+0000 7f21d6ffd700 1 -- 172.21.15.184:0/968089220 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1011 (secure 0 0 0) 0x7f21dc00d1c0 con 0x7f21e0197ab0 2024-03-15T12:50:12.158 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:50:12.159 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":2,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","modified":"2024-03-15T12:50:03.585102Z","created":"2024-03-15T12:48:17.225697Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-15T12:50:12.161 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:12.173+0000 7f21e7a93700 1 -- 172.21.15.184:0/968089220 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f21cc038af0 msgr2=0x7f21cc03afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:12.161 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:12.173+0000 7f21e7a93700 1 --2- 172.21.15.184:0/968089220 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f21cc038af0 0x7f21cc03afb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f21d00092e0 tx=0x7f21d000b580 comp rx=0 tx=0).stop 2024-03-15T12:50:12.161 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:12.173+0000 7f21e7a93700 1 -- 172.21.15.184:0/968089220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21e0197ab0 msgr2=0x7f21e019be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:12.161 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:12.173+0000 7f21e7a93700 1 --2- 172.21.15.184:0/968089220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21e0197ab0 0x7f21e019be40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00b4d0 tx=0x7f21dc00b890 comp rx=0 tx=0).stop 2024-03-15T12:50:12.161 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:12.173+0000 7f21e7a93700 1 -- 172.21.15.184:0/968089220 shutdown_connections 2024-03-15T12:50:12.161 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:12.173+0000 7f21e7a93700 1 --2- 172.21.15.184:0/968089220 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f21cc038af0 0x7f21cc03afb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:12.161 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:12.173+0000 7f21e7a93700 1 --2- 172.21.15.184:0/968089220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21e01069e0 0x7f21e0197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:12.161 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:12.173+0000 7f21e7a93700 1 --2- 172.21.15.184:0/968089220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21e0197ab0 0x7f21e019be40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:12.161 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:12.173+0000 7f21e7a93700 1 -- 172.21.15.184:0/968089220 >> 172.21.15.184:0/968089220 conn(0x7f21e00fc4f0 msgr2=0x7f21e0190420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:50:12.161 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:12.174+0000 7f21e7a93700 1 -- 172.21.15.184:0/968089220 shutdown_connections 2024-03-15T12:50:12.161 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:12.174+0000 7f21e7a93700 1 -- 172.21.15.184:0/968089220 wait complete. 2024-03-15T12:50:12.162 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 2 2024-03-15T12:50:12.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:12 smithi149 systemd[1]: Started Ceph mon.b for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:50:13.752 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-15T12:50:13.753 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-15T12:50:13.995 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T12:50:18.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:50:18.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: mon.c calling monitor election 2024-03-15T12:50:18.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: mon.a calling monitor election 2024-03-15T12:50:18.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:18.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:50:18.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:18.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:18.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:18.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:18.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:18.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:18.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:18.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:18.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-15T12:50:18.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: monmap e3: 3 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-15T12:50:18.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: fsmap 2024-03-15T12:50:18.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: osdmap e4: 0 total, 0 up, 0 in 2024-03-15T12:50:18.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: mgrmap e13: x(active, since 59s) 2024-03-15T12:50:18.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-15T12:50:18.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-15T12:50:18.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-15T12:50:18.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: mon.b (rank 2) addr [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] is down (out of quorum) 2024-03-15T12:50:18.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:18.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:18.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:18.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:18.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:50:18.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: mon.c calling monitor election 2024-03-15T12:50:18.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: mon.a calling monitor election 2024-03-15T12:50:18.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:18.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:50:18.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:18.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:18.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:18.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:18.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:18.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:18.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:18.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:18.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-15T12:50:18.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: monmap e3: 3 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-15T12:50:18.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: fsmap 2024-03-15T12:50:18.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: osdmap e4: 0 total, 0 up, 0 in 2024-03-15T12:50:18.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: mgrmap e13: x(active, since 59s) 2024-03-15T12:50:18.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-15T12:50:18.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-15T12:50:18.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-15T12:50:18.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: mon.b (rank 2) addr [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] is down (out of quorum) 2024-03-15T12:50:18.062 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:18.062 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:18.062 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:18.062 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:18.470 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.481+0000 7f75633f8700 1 -- 172.21.15.184:0/122171790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c1069b0 msgr2=0x7f7548005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:18.471 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.481+0000 7f75633f8700 1 --2- 172.21.15.184:0/122171790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c1069b0 0x7f7548005610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f754c005fa0 tx=0x7f754c00ffc0 comp rx=0 tx=0).stop 2024-03-15T12:50:18.471 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.481+0000 7f75633f8700 1 -- 172.21.15.184:0/122171790 shutdown_connections 2024-03-15T12:50:18.471 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.481+0000 7f75633f8700 1 --2- 172.21.15.184:0/122171790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c1069b0 0x7f7548005610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:18.471 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.481+0000 7f75633f8700 1 -- 172.21.15.184:0/122171790 >> 172.21.15.184:0/122171790 conn(0x7f755c0fc4a0 msgr2=0x7f755c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:50:18.471 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.481+0000 7f75633f8700 1 -- 172.21.15.184:0/122171790 shutdown_connections 2024-03-15T12:50:18.471 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.481+0000 7f75633f8700 1 -- 172.21.15.184:0/122171790 wait complete. 2024-03-15T12:50:18.471 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f75633f8700 1 Processor -- start 2024-03-15T12:50:18.471 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f75633f8700 1 -- start start 2024-03-15T12:50:18.471 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f75633f8700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f755c1069b0 0x7f755c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:18.471 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f75633f8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f755c196970 0x7f755c19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:18.471 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f75633f8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c19b240 0x7f755c19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:18.472 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f75633f8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c197060 con 0x7f755c19b240 2024-03-15T12:50:18.472 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f75633f8700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c1a2fb0 con 0x7f755c1069b0 2024-03-15T12:50:18.472 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f75633f8700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c1a30f0 con 0x7f755c196970 2024-03-15T12:50:18.472 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f7560993700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f755c196970 0x7f755c19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:18.472 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f7560993700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f755c196970 0x7f755c19ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:50610/0 (socket says 172.21.15.184:50610) 2024-03-15T12:50:18.472 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f7560993700 1 -- 172.21.15.184:0/504408038 learned_addr learned my addr 172.21.15.184:0/504408038 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-15T12:50:18.472 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f7561194700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f755c1069b0 0x7f755c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:18.472 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f7560993700 1 -- 172.21.15.184:0/504408038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f755c196970 msgr2=0x7f755c19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-15T12:50:18.472 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f7560993700 1 -- 172.21.15.184:0/504408038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f755c196970 msgr2=0x7f755c19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-15T12:50:18.472 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f7560993700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f755c196970 0x7f755c19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-15T12:50:18.472 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f7560993700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f755c196970 0x7f755c19ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-15T12:50:18.472 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f7561995700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c19b240 0x7f755c19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:18.472 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f7561194700 1 -- 172.21.15.184:0/504408038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f755c1069b0 msgr2=0x7f755c196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-15T12:50:18.473 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f7561194700 1 -- 172.21.15.184:0/504408038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f755c1069b0 msgr2=0x7f755c196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-15T12:50:18.473 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f7561194700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f755c1069b0 0x7f755c196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-15T12:50:18.473 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.482+0000 7f7561194700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f755c1069b0 0x7f755c196430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-15T12:50:18.473 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.483+0000 7f7561995700 1 -- 172.21.15.184:0/504408038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f755c196970 msgr2=0x7f755c19ad00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-15T12:50:18.473 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.483+0000 7f7561995700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f755c196970 0x7f755c19ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:18.473 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.483+0000 7f7561995700 1 -- 172.21.15.184:0/504408038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f755c1069b0 msgr2=0x7f755c196430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-15T12:50:18.473 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.483+0000 7f7561995700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f755c1069b0 0x7f755c196430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:18.473 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.483+0000 7f7561995700 1 -- 172.21.15.184:0/504408038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c00f9c0 con 0x7f755c19b240 2024-03-15T12:50:18.473 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.483+0000 7f7561995700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c19b240 0x7f755c19b6f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f755400bd50 tx=0x7f755400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:50:18.473 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.483+0000 7f75527fc700 1 -- 172.21.15.184:0/504408038 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755400cd10 con 0x7f755c19b240 2024-03-15T12:50:18.473 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.483+0000 7f75633f8700 1 -- 172.21.15.184:0/504408038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755c1a33d0 con 0x7f755c19b240 2024-03-15T12:50:18.473 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.483+0000 7f75633f8700 1 -- 172.21.15.184:0/504408038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755c1a3920 con 0x7f755c19b240 2024-03-15T12:50:18.473 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.483+0000 7f75527fc700 1 -- 172.21.15.184:0/504408038 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7554009870 con 0x7f755c19b240 2024-03-15T12:50:18.473 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.483+0000 7f75527fc700 1 -- 172.21.15.184:0/504408038 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755401d850 con 0x7f755c19b240 2024-03-15T12:50:18.474 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.484+0000 7f75527fc700 1 -- 172.21.15.184:0/504408038 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f755401d9b0 con 0x7f755c19b240 2024-03-15T12:50:18.474 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.485+0000 7f75527fc700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f754003b3f0 0x7f754003d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:18.474 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.485+0000 7f75633f8700 1 -- 172.21.15.184:0/504408038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755c063610 con 0x7f755c19b240 2024-03-15T12:50:18.475 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.485+0000 7f75527fc700 1 -- 172.21.15.184:0/504408038 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f755404c440 con 0x7f755c19b240 2024-03-15T12:50:18.475 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.485+0000 7f7561194700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f754003b3f0 0x7f754003d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:18.475 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.486+0000 7f7561194700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f754003b3f0 0x7f754003d8b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f754c005b40 tx=0x7f754c005ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:50:18.477 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.491+0000 7f75527fc700 1 -- 172.21.15.184:0/504408038 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7554007b90 con 0x7f755c19b240 2024-03-15T12:50:18.760 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.773+0000 7f75633f8700 1 -- 172.21.15.184:0/504408038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f755c108800 con 0x7f755c19b240 2024-03-15T12:50:18.761 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.774+0000 7f75527fc700 1 -- 172.21.15.184:0/504408038 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1279 (secure 0 0 0) 0x7f7554018170 con 0x7f755c19b240 2024-03-15T12:50:18.761 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-15T12:50:18.761 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":3,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","modified":"2024-03-15T12:50:12.643366Z","created":"2024-03-15T12:48:17.225697Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-15T12:50:18.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.776+0000 7f75633f8700 1 -- 172.21.15.184:0/504408038 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f754003b3f0 msgr2=0x7f754003d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:18.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.776+0000 7f75633f8700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f754003b3f0 0x7f754003d8b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f754c005b40 tx=0x7f754c005ab0 comp rx=0 tx=0).stop 2024-03-15T12:50:18.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.776+0000 7f75633f8700 1 -- 172.21.15.184:0/504408038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c19b240 msgr2=0x7f755c19b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:18.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.776+0000 7f75633f8700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c19b240 0x7f755c19b6f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f755400bd50 tx=0x7f755400be30 comp rx=0 tx=0).stop 2024-03-15T12:50:18.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.776+0000 7f75633f8700 1 -- 172.21.15.184:0/504408038 shutdown_connections 2024-03-15T12:50:18.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.776+0000 7f75633f8700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f754003b3f0 0x7f754003d8b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:18.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.776+0000 7f75633f8700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f755c19b240 0x7f755c19b6f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:18.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.776+0000 7f75633f8700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f755c1069b0 0x7f755c196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:18.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.776+0000 7f75633f8700 1 --2- 172.21.15.184:0/504408038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f755c196970 0x7f755c19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:18.763 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.776+0000 7f75633f8700 1 -- 172.21.15.184:0/504408038 >> 172.21.15.184:0/504408038 conn(0x7f755c0fc4a0 msgr2=0x7f755c0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:50:18.764 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.776+0000 7f75633f8700 1 -- 172.21.15.184:0/504408038 shutdown_connections 2024-03-15T12:50:18.764 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:18.776+0000 7f75633f8700 1 -- 172.21.15.184:0/504408038 wait complete. 2024-03-15T12:50:18.764 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 3 2024-03-15T12:50:19.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:19.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:19.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:18 smithi002 ceph-mon[32060]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:19.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:19.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:19.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:19.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:18 smithi184 ceph-mon[35993]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:19.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:19.867 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-15T12:50:19.867 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-15T12:50:20.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:50:20.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: mon.c calling monitor election 2024-03-15T12:50:20.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: mon.a calling monitor election 2024-03-15T12:50:20.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:20.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:50:20.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:20.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:20.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:20.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:20.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:20.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:20.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:20.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:20.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-15T12:50:20.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: monmap e3: 3 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-15T12:50:20.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: fsmap 2024-03-15T12:50:20.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: osdmap e4: 0 total, 0 up, 0 in 2024-03-15T12:50:20.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: mgrmap e13: x(active, since 59s) 2024-03-15T12:50:20.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-15T12:50:20.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-15T12:50:20.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-15T12:50:20.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: mon.b (rank 2) addr [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] is down (out of quorum) 2024-03-15T12:50:20.114 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.114 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.114 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.114 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.114 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:20.114 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:20.114 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:20.114 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.117 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:50:20.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: mon.b calling monitor election 2024-03-15T12:50:20.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-15T12:50:20.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-15T12:50:20.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: Updating smithi184:/etc/ceph/ceph.conf 2024-03-15T12:50:20.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: Updating smithi149:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:50:20.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: mon.b calling monitor election 2024-03-15T12:50:20.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: mon.c calling monitor election 2024-03-15T12:50:20.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: mon.a calling monitor election 2024-03-15T12:50:20.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-15T12:50:20.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: monmap e3: 3 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-15T12:50:20.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: fsmap 2024-03-15T12:50:20.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: osdmap e4: 0 total, 0 up, 0 in 2024-03-15T12:50:20.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: mgrmap e13: x(active, since 61s) 2024-03-15T12:50:20.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-15T12:50:20.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: Cluster is now healthy 2024-03-15T12:50:20.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: overall HEALTH_OK 2024-03-15T12:50:20.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.807 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-15T12:50:20.807 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-15T12:50:20.807 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:20.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: mon.b calling monitor election 2024-03-15T12:50:20.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-15T12:50:20.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-15T12:50:20.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: Updating smithi184:/etc/ceph/ceph.conf 2024-03-15T12:50:20.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: Updating smithi149:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:50:20.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: mon.b calling monitor election 2024-03-15T12:50:20.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: mon.c calling monitor election 2024-03-15T12:50:20.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: mon.a calling monitor election 2024-03-15T12:50:20.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-15T12:50:20.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: monmap e3: 3 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-15T12:50:20.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: fsmap 2024-03-15T12:50:20.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: osdmap e4: 0 total, 0 up, 0 in 2024-03-15T12:50:20.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: mgrmap e13: x(active, since 61s) 2024-03-15T12:50:20.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-15T12:50:20.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: Cluster is now healthy 2024-03-15T12:50:20.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: overall HEALTH_OK 2024-03-15T12:50:20.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.812 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.813 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.813 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.813 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.813 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.813 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.814 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.814 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.814 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-15T12:50:20.814 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-15T12:50:20.814 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:20.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: mon.b calling monitor election 2024-03-15T12:50:20.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-15T12:50:20.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-15T12:50:20.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: Updating smithi184:/etc/ceph/ceph.conf 2024-03-15T12:50:20.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: Updating smithi149:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:50:20.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: mon.b calling monitor election 2024-03-15T12:50:20.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: mon.c calling monitor election 2024-03-15T12:50:20.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: mon.a calling monitor election 2024-03-15T12:50:20.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-15T12:50:20.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: monmap e3: 3 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-15T12:50:20.863 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: fsmap 2024-03-15T12:50:20.863 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: osdmap e4: 0 total, 0 up, 0 in 2024-03-15T12:50:20.863 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: mgrmap e13: x(active, since 61s) 2024-03-15T12:50:20.863 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-15T12:50:20.863 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: Cluster is now healthy 2024-03-15T12:50:20.863 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: overall HEALTH_OK 2024-03-15T12:50:20.864 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.864 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.864 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.864 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.864 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.864 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.864 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.864 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.864 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.864 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.864 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:20.864 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-15T12:50:20.865 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-15T12:50:20.865 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:21.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:21 smithi184 ceph-mon[35993]: Updating smithi002:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:50:21.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:21 smithi184 ceph-mon[35993]: Updating smithi184:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:50:21.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:21 smithi184 ceph-mon[35993]: Reconfiguring mon.a (unknown last config time)... 2024-03-15T12:50:21.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:21 smithi184 ceph-mon[35993]: Reconfiguring daemon mon.a on smithi002 2024-03-15T12:50:21.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:21 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:21.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:21 smithi184 ceph-mon[35993]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:21.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:21 smithi149 ceph-mon[35927]: Updating smithi002:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:50:21.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:21 smithi149 ceph-mon[35927]: Updating smithi184:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:50:21.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:21 smithi149 ceph-mon[35927]: Reconfiguring mon.a (unknown last config time)... 2024-03-15T12:50:21.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:21 smithi149 ceph-mon[35927]: Reconfiguring daemon mon.a on smithi002 2024-03-15T12:50:21.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:21 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:21.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:21 smithi149 ceph-mon[35927]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:22.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:21 smithi002 ceph-mon[32060]: Updating smithi002:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:50:22.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:21 smithi002 ceph-mon[32060]: Updating smithi184:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:50:22.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:21 smithi002 ceph-mon[32060]: Reconfiguring mon.a (unknown last config time)... 2024-03-15T12:50:22.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:21 smithi002 ceph-mon[32060]: Reconfiguring daemon mon.a on smithi002 2024-03-15T12:50:22.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:50:22.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:21 smithi002 ceph-mon[32060]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.489+0000 7f1ac80a9700 1 -- 172.21.15.2:0/1272481949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ac0100ee0 msgr2=0x7f1ac0101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.489+0000 7f1ac80a9700 1 --2- 172.21.15.2:0/1272481949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ac0100ee0 0x7f1ac0101340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009990 tx=0x7f1ab4009ca0 comp rx=0 tx=0).stop 2024-03-15T12:50:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.489+0000 7f1ac80a9700 1 -- 172.21.15.2:0/1272481949 shutdown_connections 2024-03-15T12:50:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.489+0000 7f1ac80a9700 1 --2- 172.21.15.2:0/1272481949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac0101880 0x7f1ac010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.489+0000 7f1ac80a9700 1 --2- 172.21.15.2:0/1272481949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac0106f20 0x7f1ac0107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.489+0000 7f1ac80a9700 1 --2- 172.21.15.2:0/1272481949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ac0100ee0 0x7f1ac0101340 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.489+0000 7f1ac80a9700 1 -- 172.21.15.2:0/1272481949 >> 172.21.15.2:0/1272481949 conn(0x7f1ac0077e10 msgr2=0x7f1ac0078220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:50:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.489+0000 7f1ac80a9700 1 -- 172.21.15.2:0/1272481949 shutdown_connections 2024-03-15T12:50:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.489+0000 7f1ac80a9700 1 -- 172.21.15.2:0/1272481949 wait complete. 2024-03-15T12:50:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.490+0000 7f1ac80a9700 1 Processor -- start 2024-03-15T12:50:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.490+0000 7f1ac80a9700 1 -- start start 2024-03-15T12:50:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.490+0000 7f1ac80a9700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac0100ee0 0x7f1ac0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.490+0000 7f1ac80a9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ac0101880 0x7f1ac0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.490+0000 7f1ac80a9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac0106f20 0x7f1ac019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.490+0000 7f1ac80a9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac019d3d0 con 0x7f1ac0101880 2024-03-15T12:50:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.490+0000 7f1ac80a9700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac019d540 con 0x7f1ac0100ee0 2024-03-15T12:50:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.490+0000 7f1ac80a9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac019d700 con 0x7f1ac0106f20 2024-03-15T12:50:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.490+0000 7f1ac5644700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ac0101880 0x7f1ac0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.490+0000 7f1ac5644700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ac0101880 0x7f1ac0198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45706/0 (socket says 172.21.15.2:45706) 2024-03-15T12:50:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.490+0000 7f1ac5644700 1 -- 172.21.15.2:0/3184791983 learned_addr learned my addr 172.21.15.2:0/3184791983 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:50:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.490+0000 7f1ac5e45700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac0100ee0 0x7f1ac0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.491+0000 7f1ac6646700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac0106f20 0x7f1ac019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.491+0000 7f1ac5644700 1 -- 172.21.15.2:0/3184791983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac0106f20 msgr2=0x7f1ac019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.491+0000 7f1ac5644700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac0106f20 0x7f1ac019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.491+0000 7f1ac5644700 1 -- 172.21.15.2:0/3184791983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac0100ee0 msgr2=0x7f1ac0198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.491+0000 7f1ac5644700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac0100ee0 0x7f1ac0198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.491+0000 7f1ac5644700 1 -- 172.21.15.2:0/3184791983 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab40095f0 con 0x7f1ac0101880 2024-03-15T12:50:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.491+0000 7f1ac6646700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac0106f20 0x7f1ac019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T12:50:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.491+0000 7f1ac5e45700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac0100ee0 0x7f1ac0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:50:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.491+0000 7f1ac5644700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ac0101880 0x7f1ac0198ad0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4000c00 tx=0x7f1ab4003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:50:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.491+0000 7f1ab2ffd700 1 -- 172.21.15.2:0/3184791983 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab401d070 con 0x7f1ac0101880 2024-03-15T12:50:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.491+0000 7f1ab2ffd700 1 -- 172.21.15.2:0/3184791983 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1ab400be80 con 0x7f1ac0101880 2024-03-15T12:50:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.491+0000 7f1ac80a9700 1 -- 172.21.15.2:0/3184791983 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac019d980 con 0x7f1ac0101880 2024-03-15T12:50:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.491+0000 7f1ab2ffd700 1 -- 172.21.15.2:0/3184791983 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400fd50 con 0x7f1ac0101880 2024-03-15T12:50:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.491+0000 7f1ac80a9700 1 -- 172.21.15.2:0/3184791983 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac01a5240 con 0x7f1ac0101880 2024-03-15T12:50:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.492+0000 7f1ab2ffd700 1 -- 172.21.15.2:0/3184791983 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f1ab400f4f0 con 0x7f1ac0101880 2024-03-15T12:50:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.492+0000 7f1ac80a9700 1 -- 172.21.15.2:0/3184791983 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac01025d0 con 0x7f1ac0101880 2024-03-15T12:50:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.492+0000 7f1ab2ffd700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1aac038f90 0x7f1aac03b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.492+0000 7f1ab2ffd700 1 -- 172.21.15.2:0/3184791983 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1ab404ca90 con 0x7f1ac0101880 2024-03-15T12:50:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.492+0000 7f1ac5e45700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1aac038f90 0x7f1aac03b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.493+0000 7f1ac5e45700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1aac038f90 0x7f1aac03b450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1abc005950 tx=0x7f1abc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:50:22.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.497+0000 7f1ab2ffd700 1 -- 172.21.15.2:0/3184791983 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1ab4005050 con 0x7f1ac0101880 2024-03-15T12:50:22.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.704+0000 7f1ac80a9700 1 -- 172.21.15.2:0/3184791983 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f1ac0199950 con 0x7f1ac0101880 2024-03-15T12:50:22.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.705+0000 7f1ab2ffd700 1 -- 172.21.15.2:0/3184791983 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f1ab4005050 con 0x7f1ac0101880 2024-03-15T12:50:22.692 INFO:teuthology.orchestra.run.smithi002.stdout:# minimal ceph.conf for 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:50:22.692 INFO:teuthology.orchestra.run.smithi002.stdout:[global] 2024-03-15T12:50:22.692 INFO:teuthology.orchestra.run.smithi002.stdout: fsid = 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:50:22.692 INFO:teuthology.orchestra.run.smithi002.stdout: mon_host = [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] 2024-03-15T12:50:22.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.707+0000 7f1ac80a9700 1 -- 172.21.15.2:0/3184791983 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1aac038f90 msgr2=0x7f1aac03b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:22.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.707+0000 7f1ac80a9700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1aac038f90 0x7f1aac03b450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1abc005950 tx=0x7f1abc00b410 comp rx=0 tx=0).stop 2024-03-15T12:50:22.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.707+0000 7f1ac80a9700 1 -- 172.21.15.2:0/3184791983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ac0101880 msgr2=0x7f1ac0198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:22.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.707+0000 7f1ac80a9700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ac0101880 0x7f1ac0198ad0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4000c00 tx=0x7f1ab4003af0 comp rx=0 tx=0).stop 2024-03-15T12:50:22.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.707+0000 7f1ac80a9700 1 -- 172.21.15.2:0/3184791983 shutdown_connections 2024-03-15T12:50:22.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.707+0000 7f1ac80a9700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1aac038f90 0x7f1aac03b450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:22.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.707+0000 7f1ac80a9700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ac0106f20 0x7f1ac019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:22.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.707+0000 7f1ac80a9700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac0100ee0 0x7f1ac0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:22.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.707+0000 7f1ac80a9700 1 --2- 172.21.15.2:0/3184791983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ac0101880 0x7f1ac0198ad0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:22.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.707+0000 7f1ac80a9700 1 -- 172.21.15.2:0/3184791983 >> 172.21.15.2:0/3184791983 conn(0x7f1ac0077e10 msgr2=0x7f1ac0069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:50:22.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.707+0000 7f1ac80a9700 1 -- 172.21.15.2:0/3184791983 shutdown_connections 2024-03-15T12:50:22.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:22.707+0000 7f1ac80a9700 1 -- 172.21.15.2:0/3184791983 wait complete. 2024-03-15T12:50:22.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:22 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:22.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:22 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:22.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:22 smithi149 ceph-mon[35927]: Reconfiguring mon.b (monmap changed)... 2024-03-15T12:50:22.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:22 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-15T12:50:22.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:22 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-15T12:50:22.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:22 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:22.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:22 smithi149 ceph-mon[35927]: Reconfiguring daemon mon.b on smithi149 2024-03-15T12:50:22.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:22 smithi149 ceph-mon[35927]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:22.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:22 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3184791983' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:23.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:22 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:23.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:22 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:23.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:22 smithi002 ceph-mon[32060]: Reconfiguring mon.b (monmap changed)... 2024-03-15T12:50:23.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:22 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-15T12:50:23.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:22 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-15T12:50:23.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:22 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:23.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:22 smithi002 ceph-mon[32060]: Reconfiguring daemon mon.b on smithi149 2024-03-15T12:50:23.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:22 smithi002 ceph-mon[32060]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:23.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:22 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3184791983' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:23.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:22 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:23.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:22 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:23.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:22 smithi184 ceph-mon[35993]: Reconfiguring mon.b (monmap changed)... 2024-03-15T12:50:23.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:22 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-15T12:50:23.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:22 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-15T12:50:23.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:22 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:23.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:22 smithi184 ceph-mon[35993]: Reconfiguring daemon mon.b on smithi149 2024-03-15T12:50:23.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:22 smithi184 ceph-mon[35993]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:23.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:22 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3184791983' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:23.321 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-15T12:50:23.321 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:50:23.322 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.conf 2024-03-15T12:50:23.352 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:50:23.352 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:50:23.419 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-15T12:50:23.419 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.conf 2024-03-15T12:50:23.448 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-15T12:50:23.449 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:50:23.516 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-15T12:50:23.516 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.conf 2024-03-15T12:50:23.545 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-15T12:50:23.545 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:50:23.613 INFO:tasks.cephadm:Adding mgr.x on smithi002 2024-03-15T12:50:23.613 INFO:tasks.cephadm:Adding mgr.y on smithi149 2024-03-15T12:50:23.613 INFO:tasks.cephadm:Adding mgr.z on smithi184 2024-03-15T12:50:23.614 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch apply mgr '3;smithi002=x;smithi149=y;smithi184=z' 2024-03-15T12:50:24.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:24.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:24.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:24 smithi002 ceph-mon[32060]: Reconfiguring mon.c (monmap changed)... 2024-03-15T12:50:24.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-15T12:50:24.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-15T12:50:24.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:24.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:24 smithi002 ceph-mon[32060]: Reconfiguring daemon mon.c on smithi184 2024-03-15T12:50:24.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:24.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:24.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:24 smithi184 ceph-mon[35993]: Reconfiguring mon.c (monmap changed)... 2024-03-15T12:50:24.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-15T12:50:24.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-15T12:50:24.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:24.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:24 smithi184 ceph-mon[35993]: Reconfiguring daemon mon.c on smithi184 2024-03-15T12:50:24.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:24.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:24.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:24 smithi149 ceph-mon[35927]: Reconfiguring mon.c (monmap changed)... 2024-03-15T12:50:24.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-15T12:50:24.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-15T12:50:24.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:24.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:24 smithi149 ceph-mon[35927]: Reconfiguring daemon mon.c on smithi184 2024-03-15T12:50:24.735 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T12:50:25.291 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:25 smithi184 ceph-mon[35993]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:25.291 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:25 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:25.291 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:25 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:25.291 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:25 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:25.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:25 smithi002 ceph-mon[32060]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:25.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:25 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:25.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:25 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:25.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:25 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:25.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:25 smithi149 ceph-mon[35927]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:25.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:25 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:25.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:25 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:25.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:25 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:26.380 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:26 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:26.380 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:26 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:26.381 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:26 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:26.483 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.496+0000 7fbe5d72a700 1 -- 172.21.15.184:0/1013834423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe58102aa0 msgr2=0x7fbe58104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:26.483 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.496+0000 7fbe5d72a700 1 --2- 172.21.15.184:0/1013834423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe58102aa0 0x7fbe58104e90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009990 tx=0x7fbe48009ca0 comp rx=0 tx=0).stop 2024-03-15T12:50:26.483 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.496+0000 7fbe5d72a700 1 -- 172.21.15.184:0/1013834423 shutdown_connections 2024-03-15T12:50:26.483 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.496+0000 7fbe5d72a700 1 --2- 172.21.15.184:0/1013834423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe58107d00 0x7fbe5810a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:26.483 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.496+0000 7fbe5d72a700 1 --2- 172.21.15.184:0/1013834423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe58102aa0 0x7fbe58104e90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:26.483 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.496+0000 7fbe5d72a700 1 --2- 172.21.15.184:0/1013834423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe581053d0 0x7fbe581077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:26.483 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.496+0000 7fbe5d72a700 1 -- 172.21.15.184:0/1013834423 >> 172.21.15.184:0/1013834423 conn(0x7fbe580fc4d0 msgr2=0x7fbe580fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:50:26.483 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.496+0000 7fbe5d72a700 1 -- 172.21.15.184:0/1013834423 shutdown_connections 2024-03-15T12:50:26.483 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.496+0000 7fbe5d72a700 1 -- 172.21.15.184:0/1013834423 wait complete. 2024-03-15T12:50:26.486 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.496+0000 7fbe5d72a700 1 Processor -- start 2024-03-15T12:50:26.486 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe5d72a700 1 -- start start 2024-03-15T12:50:26.486 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe5d72a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe58102aa0 0x7fbe58197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:26.486 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe5d72a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe581053d0 0x7fbe58198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:26.486 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe5d72a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe58107d00 0x7fbe5819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:26.486 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe5d72a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5819fb60 con 0x7fbe581053d0 2024-03-15T12:50:26.487 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe5d72a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5819fca0 con 0x7fbe58102aa0 2024-03-15T12:50:26.487 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe5d72a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5819fe30 con 0x7fbe58107d00 2024-03-15T12:50:26.487 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe577fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe58107d00 0x7fbe5819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:26.487 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe577fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe58107d00 0x7fbe5819f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:34986/0 (socket says 172.21.15.184:34986) 2024-03-15T12:50:26.487 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe577fe700 1 -- 172.21.15.184:0/665072840 learned_addr learned my addr 172.21.15.184:0/665072840 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-15T12:50:26.487 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe56ffd700 1 --2- 172.21.15.184:0/665072840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe58102aa0 0x7fbe58197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:26.487 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe4e7fc700 1 --2- 172.21.15.184:0/665072840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe581053d0 0x7fbe58198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:26.487 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe577fe700 1 -- 172.21.15.184:0/665072840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe58102aa0 msgr2=0x7fbe58197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:26.487 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe577fe700 1 --2- 172.21.15.184:0/665072840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe58102aa0 0x7fbe58197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:26.487 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe577fe700 1 -- 172.21.15.184:0/665072840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe581053d0 msgr2=0x7fbe58198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:26.487 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe577fe700 1 --2- 172.21.15.184:0/665072840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe581053d0 0x7fbe58198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:26.487 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe577fe700 1 -- 172.21.15.184:0/665072840 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe480095f0 con 0x7fbe58107d00 2024-03-15T12:50:26.487 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.497+0000 7fbe4e7fc700 1 --2- 172.21.15.184:0/665072840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe581053d0 0x7fbe58198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:50:26.487 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.498+0000 7fbe577fe700 1 --2- 172.21.15.184:0/665072840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe58107d00 0x7fbe5819f590 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000add0 tx=0x7fbe5000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:50:26.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.498+0000 7fbe54ff9700 1 -- 172.21.15.184:0/665072840 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5000f070 con 0x7fbe58107d00 2024-03-15T12:50:26.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.498+0000 7fbe54ff9700 1 -- 172.21.15.184:0/665072840 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbe50007b00 con 0x7fbe58107d00 2024-03-15T12:50:26.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.498+0000 7fbe54ff9700 1 -- 172.21.15.184:0/665072840 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe50005360 con 0x7fbe58107d00 2024-03-15T12:50:26.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.498+0000 7fbe5d72a700 1 -- 172.21.15.184:0/665072840 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe581a0110 con 0x7fbe58107d00 2024-03-15T12:50:26.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.498+0000 7fbe5d72a700 1 -- 172.21.15.184:0/665072840 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe581a0660 con 0x7fbe58107d00 2024-03-15T12:50:26.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.499+0000 7fbe54ff9700 1 -- 172.21.15.184:0/665072840 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7fbe500054c0 con 0x7fbe58107d00 2024-03-15T12:50:26.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.499+0000 7fbe5d72a700 1 -- 172.21.15.184:0/665072840 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe580fe050 con 0x7fbe58107d00 2024-03-15T12:50:26.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.499+0000 7fbe54ff9700 1 --2- 172.21.15.184:0/665072840 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbe44038f90 0x7fbe4403b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:26.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.499+0000 7fbe54ff9700 1 -- 172.21.15.184:0/665072840 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbe5004afc0 con 0x7fbe58107d00 2024-03-15T12:50:26.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.499+0000 7fbe56ffd700 1 --2- 172.21.15.184:0/665072840 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbe44038f90 0x7fbe4403b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:26.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.500+0000 7fbe56ffd700 1 --2- 172.21.15.184:0/665072840 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbe44038f90 0x7fbe4403b450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800b5c0 tx=0x7fbe480038c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:50:26.491 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.504+0000 7fbe54ff9700 1 -- 172.21.15.184:0/665072840 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbe5001a310 con 0x7fbe58107d00 2024-03-15T12:50:26.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:26 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:26.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:26 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:26.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:26 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:26.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:26 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:26.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:26 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:26.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:26 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:26.698 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.711+0000 7fbe5d72a700 1 -- 172.21.15.184:0/665072840 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi149=y;smithi184=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe5802d990 con 0x7fbe44038f90 2024-03-15T12:50:26.704 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.717+0000 7fbe54ff9700 1 -- 172.21.15.184:0/665072840 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbe5802d990 con 0x7fbe44038f90 2024-03-15T12:50:26.704 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mgr update... 2024-03-15T12:50:26.706 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.719+0000 7fbe5d72a700 1 -- 172.21.15.184:0/665072840 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbe44038f90 msgr2=0x7fbe4403b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:26.706 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.719+0000 7fbe5d72a700 1 --2- 172.21.15.184:0/665072840 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbe44038f90 0x7fbe4403b450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800b5c0 tx=0x7fbe480038c0 comp rx=0 tx=0).stop 2024-03-15T12:50:26.706 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.719+0000 7fbe5d72a700 1 -- 172.21.15.184:0/665072840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe58107d00 msgr2=0x7fbe5819f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:26.706 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.719+0000 7fbe5d72a700 1 --2- 172.21.15.184:0/665072840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe58107d00 0x7fbe5819f590 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000add0 tx=0x7fbe5000c330 comp rx=0 tx=0).stop 2024-03-15T12:50:26.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.719+0000 7fbe5d72a700 1 -- 172.21.15.184:0/665072840 shutdown_connections 2024-03-15T12:50:26.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.719+0000 7fbe5d72a700 1 --2- 172.21.15.184:0/665072840 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbe44038f90 0x7fbe4403b450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:26.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.719+0000 7fbe5d72a700 1 --2- 172.21.15.184:0/665072840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe58107d00 0x7fbe5819f590 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:26.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.719+0000 7fbe5d72a700 1 --2- 172.21.15.184:0/665072840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe58102aa0 0x7fbe58197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:26.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.719+0000 7fbe5d72a700 1 --2- 172.21.15.184:0/665072840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe581053d0 0x7fbe58198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:26.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.719+0000 7fbe5d72a700 1 -- 172.21.15.184:0/665072840 >> 172.21.15.184:0/665072840 conn(0x7fbe580fc4d0 msgr2=0x7fbe581033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:50:26.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.719+0000 7fbe5d72a700 1 -- 172.21.15.184:0/665072840 shutdown_connections 2024-03-15T12:50:26.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:50:26.719+0000 7fbe5d72a700 1 -- 172.21.15.184:0/665072840 wait complete. 2024-03-15T12:50:27.259 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:27 smithi184 ceph-mon[35993]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:27.259 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:27.259 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:27.259 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:27.259 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:27.260 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:27.260 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-15T12:50:27.260 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-15T12:50:27.260 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-15T12:50:27.260 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:27.284 DEBUG:teuthology.orchestra.run.smithi149:mgr.y> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.y.service 2024-03-15T12:50:27.287 DEBUG:teuthology.orchestra.run.smithi184:mgr.z> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.z.service 2024-03-15T12:50:27.289 INFO:tasks.cephadm:Deploying OSDs... 2024-03-15T12:50:27.289 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:50:27.289 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/scratch_devs of=/dev/stdout 2024-03-15T12:50:27.308 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-15T12:50:27.309 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_1 2024-03-15T12:50:27.366 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-15T12:50:27.366 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-15T12:50:27.366 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-15T12:50:27.366 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-15T12:50:27.366 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-03-15T12:50:27.366 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-03-15 12:49:31.221678452 +0000 2024-03-15T12:50:27.366 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-03-15 12:44:54.856689015 +0000 2024-03-15T12:50:27.366 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-03-15 12:44:54.856689015 +0000 2024-03-15T12:50:27.366 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-03-15 12:44:54.856689015 +0000 2024-03-15T12:50:27.367 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-15T12:50:27.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:27 smithi002 ceph-mon[32060]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:27.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:27.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:27.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:27.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:27.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:27.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-15T12:50:27.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-15T12:50:27.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-15T12:50:27.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:27.436 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-03-15T12:50:27.436 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-03-15T12:50:27.437 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.00011234 s, 4.6 MB/s 2024-03-15T12:50:27.438 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-15T12:50:27.496 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_2 2024-03-15T12:50:27.552 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-15T12:50:27.552 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-15T12:50:27.552 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-15T12:50:27.552 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-15T12:50:27.553 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-03-15T12:50:27.553 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-03-15 12:49:31.251677754 +0000 2024-03-15T12:50:27.553 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-03-15 12:44:55.198681056 +0000 2024-03-15T12:50:27.553 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-03-15 12:44:55.198681056 +0000 2024-03-15T12:50:27.553 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-03-15 12:44:55.198681056 +0000 2024-03-15T12:50:27.553 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-15T12:50:27.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:27 smithi149 ceph-mon[35927]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:27.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:27.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:27.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:27.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:27.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:27.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-15T12:50:27.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-15T12:50:27.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-15T12:50:27.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:27.621 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-03-15T12:50:27.621 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-03-15T12:50:27.621 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000120822 s, 4.2 MB/s 2024-03-15T12:50:27.623 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-15T12:50:27.680 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_3 2024-03-15T12:50:27.736 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-15T12:50:27.736 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-15T12:50:27.736 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-03-15T12:50:27.737 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-15T12:50:27.737 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-03-15T12:50:27.737 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-03-15 12:49:31.279677103 +0000 2024-03-15T12:50:27.737 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-03-15 12:44:55.526673423 +0000 2024-03-15T12:50:27.737 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-03-15 12:44:55.526673423 +0000 2024-03-15T12:50:27.737 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-03-15 12:44:55.526673423 +0000 2024-03-15T12:50:27.737 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-15T12:50:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-03-15T12:50:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-03-15T12:50:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000117298 s, 4.4 MB/s 2024-03-15T12:50:27.807 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-15T12:50:27.864 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_4 2024-03-15T12:50:27.920 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-15T12:50:27.920 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-15T12:50:27.920 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-15T12:50:27.920 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-15T12:50:27.920 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-03-15T12:50:27.920 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-03-15 12:49:31.310676382 +0000 2024-03-15T12:50:27.920 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-03-15 12:44:55.861665627 +0000 2024-03-15T12:50:27.920 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-03-15 12:44:55.861665627 +0000 2024-03-15T12:50:27.921 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-03-15 12:44:55.861665627 +0000 2024-03-15T12:50:27.921 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-15T12:50:27.986 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-03-15T12:50:27.987 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-03-15T12:50:27.987 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000103599 s, 4.9 MB/s 2024-03-15T12:50:27.988 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-15T12:50:28.045 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-15T12:50:28.045 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-03-15T12:50:28.063 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-15T12:50:28.063 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-03-15T12:50:28.121 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-15T12:50:28.121 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-15T12:50:28.122 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-15T12:50:28.122 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-15T12:50:28.122 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-15T12:50:28.122 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-15 12:50:12.085403449 +0000 2024-03-15T12:50:28.122 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-15 12:44:55.663262270 +0000 2024-03-15T12:50:28.122 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-15 12:44:55.663262270 +0000 2024-03-15T12:50:28.122 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-15 12:44:55.663262270 +0000 2024-03-15T12:50:28.122 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-15T12:50:28.191 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-15T12:50:28.191 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-15T12:50:28.191 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.00034998 s, 1.5 MB/s 2024-03-15T12:50:28.193 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-15T12:50:28.250 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-03-15T12:50:28.307 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-15T12:50:28.307 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-15T12:50:28.307 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-03-15T12:50:28.307 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-15T12:50:28.307 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-15T12:50:28.308 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-15 12:50:12.085403449 +0000 2024-03-15T12:50:28.308 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-15 12:44:55.992255401 +0000 2024-03-15T12:50:28.308 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-15 12:44:55.992255401 +0000 2024-03-15T12:50:28.308 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-15 12:44:55.992255401 +0000 2024-03-15T12:50:28.308 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-15T12:50:28.340 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:28 smithi184 ceph-mon[35993]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi149=y;smithi184=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:50:28.340 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:28 smithi184 ceph-mon[35993]: Saving service mgr spec with placement smithi002=x;smithi149=y;smithi184=z;count:3 2024-03-15T12:50:28.340 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:28 smithi184 ceph-mon[35993]: Deploying daemon mgr.z on smithi184 2024-03-15T12:50:28.378 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-15T12:50:28.378 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-15T12:50:28.378 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000195246 s, 2.6 MB/s 2024-03-15T12:50:28.378 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-15T12:50:28.436 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-03-15T12:50:28.491 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-15T12:50:28.492 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-15T12:50:28.492 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-15T12:50:28.492 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-15T12:50:28.492 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-15T12:50:28.492 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-15 12:50:12.086403428 +0000 2024-03-15T12:50:28.492 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-15 12:44:56.335248240 +0000 2024-03-15T12:50:28.492 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-15 12:44:56.335248240 +0000 2024-03-15T12:50:28.492 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-15 12:44:56.335248240 +0000 2024-03-15T12:50:28.492 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-15T12:50:28.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:28 smithi002 ceph-mon[32060]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi149=y;smithi184=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:50:28.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:28 smithi002 ceph-mon[32060]: Saving service mgr spec with placement smithi002=x;smithi149=y;smithi184=z;count:3 2024-03-15T12:50:28.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:28 smithi002 ceph-mon[32060]: Deploying daemon mgr.z on smithi184 2024-03-15T12:50:28.556 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:28 smithi149 ceph-mon[35927]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi149=y;smithi184=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:50:28.556 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:28 smithi149 ceph-mon[35927]: Saving service mgr spec with placement smithi002=x;smithi149=y;smithi184=z;count:3 2024-03-15T12:50:28.556 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:28 smithi149 ceph-mon[35927]: Deploying daemon mgr.z on smithi184 2024-03-15T12:50:28.558 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-15T12:50:28.558 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-15T12:50:28.559 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000199189 s, 2.6 MB/s 2024-03-15T12:50:28.560 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-15T12:50:28.616 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-03-15T12:50:28.671 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-15T12:50:28.671 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-15T12:50:28.671 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-15T12:50:28.672 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-15T12:50:28.672 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-15T12:50:28.672 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-15 12:50:12.086403428 +0000 2024-03-15T12:50:28.672 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-15 12:44:56.672241204 +0000 2024-03-15T12:50:28.672 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-15 12:44:56.672241204 +0000 2024-03-15T12:50:28.672 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-15 12:44:56.672241204 +0000 2024-03-15T12:50:28.672 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-15T12:50:28.738 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-15T12:50:28.738 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-15T12:50:28.738 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000218538 s, 2.3 MB/s 2024-03-15T12:50:28.739 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-15T12:50:28.796 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-15T12:50:28.796 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2024-03-15T12:50:28.812 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-15T12:50:28.812 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2024-03-15T12:50:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-15T12:50:28.868 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-15T12:50:28.868 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-15T12:50:28.868 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-15T12:50:28.868 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-03-15T12:50:28.868 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-03-15 12:50:28.747344411 +0000 2024-03-15T12:50:28.868 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-03-15 12:44:55.714744340 +0000 2024-03-15T12:50:28.868 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-03-15 12:44:55.714744340 +0000 2024-03-15T12:50:28.869 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-03-15 12:44:55.714744340 +0000 2024-03-15T12:50:28.869 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-15T12:50:28.932 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:28 smithi184 systemd[1]: Starting Ceph mgr.z for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T12:50:28.935 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-03-15T12:50:28.936 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-03-15T12:50:28.936 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000266315 s, 1.9 MB/s 2024-03-15T12:50:28.937 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-15T12:50:28.992 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2024-03-15T12:50:29.047 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-15T12:50:29.048 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-15T12:50:29.048 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-15T12:50:29.048 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-15T12:50:29.048 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-03-15T12:50:29.048 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-03-15 12:50:28.748344390 +0000 2024-03-15T12:50:29.048 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-03-15 12:44:56.059736837 +0000 2024-03-15T12:50:29.048 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-03-15 12:44:56.059736837 +0000 2024-03-15T12:50:29.048 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-03-15 12:44:56.059736837 +0000 2024-03-15T12:50:29.048 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-15T12:50:29.114 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-03-15T12:50:29.114 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-03-15T12:50:29.114 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000199359 s, 2.6 MB/s 2024-03-15T12:50:29.115 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-15T12:50:29.172 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2024-03-15T12:50:29.195 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:29 smithi184 ceph-mon[35993]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:29.195 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:29 smithi184 podman[37686]: 2024-03-15 12:50:28.922411295 +0000 UTC m=+0.017230556 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T12:50:29.195 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:29 smithi184 podman[37686]: 2024-03-15 12:50:29.039208819 +0000 UTC m=+0.134028078 container create de6c34942d948b567abe1c09450e3dda095e14d3e2497dfbc6476c92a34a8f2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-15T12:50:29.210 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-15T12:50:29.210 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-15T12:50:29.210 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-03-15T12:50:29.210 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-15T12:50:29.210 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-03-15T12:50:29.210 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-03-15 12:50:28.748344390 +0000 2024-03-15T12:50:29.210 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-03-15 12:44:56.412729162 +0000 2024-03-15T12:50:29.210 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-03-15 12:44:56.412729162 +0000 2024-03-15T12:50:29.210 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-03-15 12:44:56.412729162 +0000 2024-03-15T12:50:29.211 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-15T12:50:29.276 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-03-15T12:50:29.276 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-03-15T12:50:29.276 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000180825 s, 2.8 MB/s 2024-03-15T12:50:29.277 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-15T12:50:29.333 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2024-03-15T12:50:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-15T12:50:29.388 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-15T12:50:29.388 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-15T12:50:29.388 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-15T12:50:29.388 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-03-15T12:50:29.389 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-03-15 12:50:28.748344390 +0000 2024-03-15T12:50:29.389 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-03-15 12:44:56.748721857 +0000 2024-03-15T12:50:29.389 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-03-15 12:44:56.748721857 +0000 2024-03-15T12:50:29.389 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-03-15 12:44:56.748721857 +0000 2024-03-15T12:50:29.389 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-15T12:50:29.453 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:29 smithi184 podman[37686]: 2024-03-15 12:50:29.325796329 +0000 UTC m=+0.420615594 container init de6c34942d948b567abe1c09450e3dda095e14d3e2497dfbc6476c92a34a8f2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=) 2024-03-15T12:50:29.453 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:29 smithi184 podman[37686]: 2024-03-15 12:50:29.329029817 +0000 UTC m=+0.423849086 container start de6c34942d948b567abe1c09450e3dda095e14d3e2497dfbc6476c92a34a8f2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-15T12:50:29.453 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:29 smithi184 ceph-mgr[37830]: -- 172.21.15.184:0/2022670682 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b6d2a901a0 con 0x55b6d2a66800 2024-03-15T12:50:29.453 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:29 smithi184 bash[37686]: de6c34942d948b567abe1c09450e3dda095e14d3e2497dfbc6476c92a34a8f2c 2024-03-15T12:50:29.453 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:29 smithi184 systemd[1]: Started Ceph mgr.z for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:50:29.456 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-03-15T12:50:29.456 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-03-15T12:50:29.456 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.00022011 s, 2.3 MB/s 2024-03-15T12:50:29.458 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-15T12:50:29.513 INFO:tasks.cephadm:Deploying osd.0 on smithi002 with /dev/vg_nvme/lv_4... 2024-03-15T12:50:29.513 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-15T12:50:29.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:29 smithi002 ceph-mon[32060]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:29.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:29 smithi149 ceph-mon[35927]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:29.760 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:50:30.230 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:29 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:29.913+0000 7fcc3ea0e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-15T12:50:30.230 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:30 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:30.010+0000 7fcc3ea0e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-15T12:50:30.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:30.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:30.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:30.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-15T12:50:30.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-15T12:50:30.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-15T12:50:30.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:30.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:30 smithi002 ceph-mon[32060]: Deploying daemon mgr.y on smithi149 2024-03-15T12:50:30.810 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:30 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:30.664+0000 7fcc3ea0e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-15T12:50:30.810 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:30 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:30.755+0000 7fcc3ea0e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-15T12:50:30.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:30 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:30.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:30 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:30.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:30 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:30.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:30 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-15T12:50:30.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:30 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-15T12:50:30.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:30 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-15T12:50:30.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:30 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:30.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:30 smithi184 ceph-mon[35993]: Deploying daemon mgr.y on smithi149 2024-03-15T12:50:30.849 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:30 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:30.849 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:30 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:30.849 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:30 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:30.849 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:30 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-15T12:50:30.849 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:30 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-15T12:50:30.849 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:30 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-15T12:50:30.849 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:30 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:30.849 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:30 smithi149 ceph-mon[35927]: Deploying daemon mgr.y on smithi149 2024-03-15T12:50:31.310 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:30 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:30.896+0000 7fcc3ea0e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-15T12:50:31.310 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:30 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:30.991+0000 7fcc3ea0e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-15T12:50:31.310 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:31 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:31.073+0000 7fcc3ea0e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-15T12:50:31.623 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:31 smithi184 ceph-mon[35993]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:31.624 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:31 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:31.345+0000 7fcc3ea0e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-15T12:50:31.624 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:31 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:31.454+0000 7fcc3ea0e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-15T12:50:31.624 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:31 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:31.542+0000 7fcc3ea0e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-15T12:50:31.624 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:31 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:31.636+0000 7fcc3ea0e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-15T12:50:31.668 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-15T12:50:31.668 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-15T12:50:31.668 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2024-03-15T12:50:31.668 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2024-03-15T12:50:31.668 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00900932 s, 1.2 GB/s 2024-03-15T12:50:31.668 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2024-03-15T12:50:31.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:31 smithi002 ceph-mon[32060]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:31.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:31 smithi149 ceph-mon[35927]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:31.861 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:31 smithi149 systemd[1]: Starting Ceph mgr.y for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T12:50:32.060 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:31 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:31.813+0000 7fcc3ea0e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-15T12:50:32.128 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:31 smithi149 podman[37334]: 2024-03-15 12:50:31.784258776 +0000 UTC m=+0.017595712 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T12:50:32.128 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:31 smithi149 podman[37334]: 2024-03-15 12:50:31.912808748 +0000 UTC m=+0.146145684 container create 1004fc3320ccecc18f7bed77f8d2229e4d9ff9b01628180f09cadbb9d87c0c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-03-15T12:50:32.129 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:32 smithi149 podman[37334]: 2024-03-15 12:50:32.114970983 +0000 UTC m=+0.348307922 container init 1004fc3320ccecc18f7bed77f8d2229e4d9ff9b01628180f09cadbb9d87c0c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-15T12:50:32.129 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:32 smithi149 podman[37334]: 2024-03-15 12:50:32.117466089 +0000 UTC m=+0.350803030 container start 1004fc3320ccecc18f7bed77f8d2229e4d9ff9b01628180f09cadbb9d87c0c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-15T12:50:32.352 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi002:vg_nvme/lv_4 2024-03-15T12:50:32.611 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:32 smithi149 ceph-mgr[37353]: -- 172.21.15.149:0/2159831106 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5614206121a0 con 0x5614205e9000 2024-03-15T12:50:32.611 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:32 smithi149 bash[37334]: 1004fc3320ccecc18f7bed77f8d2229e4d9ff9b01628180f09cadbb9d87c0c18 2024-03-15T12:50:32.611 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:32 smithi149 systemd[1]: Started Ceph mgr.y for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:50:32.628 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:50:33.060 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:32 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:32.805+0000 7fcc3ea0e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-15T12:50:33.060 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:32 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:32.951+0000 7fcc3ea0e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-15T12:50:33.111 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:32 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:32.678+0000 7f2adea50200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-15T12:50:33.111 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:32 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:32.774+0000 7f2adea50200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-15T12:50:33.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:33.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:33.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:33.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:33.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:33.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:33 smithi002 ceph-mon[32060]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:33.544 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:33 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:33.461+0000 7f2adea50200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-15T12:50:33.544 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:33 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:33.556+0000 7f2adea50200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-15T12:50:33.545 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:33.545 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:33.545 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:33.545 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:33.545 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:33.545 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:33 smithi149 ceph-mon[35927]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:33.560 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:33 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:33.309+0000 7fcc3ea0e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-15T12:50:33.560 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:33 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:33.403+0000 7fcc3ea0e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-15T12:50:33.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:33.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:33.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:33.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:33.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:33.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:33 smithi184 ceph-mon[35993]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:33.802 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:33 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:33.715+0000 7f2adea50200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-15T12:50:33.802 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:33 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:33.814+0000 7f2adea50200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-15T12:50:34.060 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:33 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:33.600+0000 7fcc3ea0e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:50:34.060 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:33 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:33.798+0000 7fcc3ea0e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-15T12:50:34.111 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:33 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:33.899+0000 7f2adea50200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-15T12:50:34.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.128+0000 7fd60ae2f700 1 -- 172.21.15.2:0/3716197926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd604102a30 msgr2=0x7fd604109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:34.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.128+0000 7fd60ae2f700 1 --2- 172.21.15.2:0/3716197926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd604102a30 0x7fd604109350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00b0a0 tx=0x7fd5fc00b3b0 comp rx=0 tx=0).stop 2024-03-15T12:50:34.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.128+0000 7fd60ae2f700 1 -- 172.21.15.2:0/3716197926 shutdown_connections 2024-03-15T12:50:34.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.128+0000 7fd60ae2f700 1 --2- 172.21.15.2:0/3716197926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd604102a30 0x7fd604109350 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:34.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.128+0000 7fd60ae2f700 1 --2- 172.21.15.2:0/3716197926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604100f10 0x7fd604101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:34.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.128+0000 7fd60ae2f700 1 --2- 172.21.15.2:0/3716197926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd604102070 0x7fd6041024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:34.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.128+0000 7fd60ae2f700 1 -- 172.21.15.2:0/3716197926 >> 172.21.15.2:0/3716197926 conn(0x7fd6040fc4b0 msgr2=0x7fd6040fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:50:34.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.128+0000 7fd60ae2f700 1 -- 172.21.15.2:0/3716197926 shutdown_connections 2024-03-15T12:50:34.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.128+0000 7fd60ae2f700 1 -- 172.21.15.2:0/3716197926 wait complete. 2024-03-15T12:50:34.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.129+0000 7fd60ae2f700 1 Processor -- start 2024-03-15T12:50:34.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.129+0000 7fd60ae2f700 1 -- start start 2024-03-15T12:50:34.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.129+0000 7fd60ae2f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604100f10 0x7fd604104e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:34.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.129+0000 7fd60ae2f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd604102070 0x7fd6041034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:34.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.129+0000 7fd60ae2f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd604102a30 0x7fd604103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:34.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.129+0000 7fd60ae2f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6041041a0 con 0x7fd604102070 2024-03-15T12:50:34.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.129+0000 7fd60ae2f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd604104310 con 0x7fd604102a30 2024-03-15T12:50:34.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.129+0000 7fd60ae2f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6041a2120 con 0x7fd604100f10 2024-03-15T12:50:34.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.129+0000 7fd603fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd604102070 0x7fd6041034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:34.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.129+0000 7fd6093cc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd604102a30 0x7fd604103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:34.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.129+0000 7fd603fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd604102070 0x7fd6041034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47176/0 (socket says 172.21.15.2:47176) 2024-03-15T12:50:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.130+0000 7fd603fff700 1 -- 172.21.15.2:0/2562641547 learned_addr learned my addr 172.21.15.2:0/2562641547 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:50:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.129+0000 7fd608bcb700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604100f10 0x7fd604104e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.130+0000 7fd603fff700 1 -- 172.21.15.2:0/2562641547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604100f10 msgr2=0x7fd604104e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.130+0000 7fd603fff700 1 --2- 172.21.15.2:0/2562641547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604100f10 0x7fd604104e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.130+0000 7fd603fff700 1 -- 172.21.15.2:0/2562641547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd604102a30 msgr2=0x7fd604103a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.130+0000 7fd603fff700 1 --2- 172.21.15.2:0/2562641547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd604102a30 0x7fd604103a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.130+0000 7fd603fff700 1 -- 172.21.15.2:0/2562641547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc009d00 con 0x7fd604102070 2024-03-15T12:50:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.130+0000 7fd608bcb700 1 --2- 172.21.15.2:0/2562641547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604100f10 0x7fd604104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:50:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.130+0000 7fd603fff700 1 --2- 172.21.15.2:0/2562641547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd604102070 0x7fd6041034e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000d950 tx=0x7fd5f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:50:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.130+0000 7fd601ffb700 1 -- 172.21.15.2:0/2562641547 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000f890 con 0x7fd604102070 2024-03-15T12:50:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.130+0000 7fd601ffb700 1 -- 172.21.15.2:0/2562641547 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd5f0010460 con 0x7fd604102070 2024-03-15T12:50:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.130+0000 7fd60ae2f700 1 -- 172.21.15.2:0/2562641547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6041a2400 con 0x7fd604102070 2024-03-15T12:50:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.130+0000 7fd601ffb700 1 -- 172.21.15.2:0/2562641547 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000e610 con 0x7fd604102070 2024-03-15T12:50:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.131+0000 7fd60ae2f700 1 -- 172.21.15.2:0/2562641547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6041a2950 con 0x7fd604102070 2024-03-15T12:50:34.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.131+0000 7fd601ffb700 1 -- 172.21.15.2:0/2562641547 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7fd5f000e770 con 0x7fd604102070 2024-03-15T12:50:34.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.131+0000 7fd60ae2f700 1 -- 172.21.15.2:0/2562641547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd604068ca0 con 0x7fd604102070 2024-03-15T12:50:34.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.132+0000 7fd601ffb700 1 --2- 172.21.15.2:0/2562641547 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd5f4038f90 0x7fd5f403b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:34.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.132+0000 7fd601ffb700 1 -- 172.21.15.2:0/2562641547 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd5f004b8c0 con 0x7fd604102070 2024-03-15T12:50:34.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.132+0000 7fd608bcb700 1 --2- 172.21.15.2:0/2562641547 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd5f4038f90 0x7fd5f403b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:34.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.132+0000 7fd608bcb700 1 --2- 172.21.15.2:0/2562641547 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd5f4038f90 0x7fd5f403b450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8005950 tx=0x7fd5f800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:50:34.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.137+0000 7fd601ffb700 1 -- 172.21.15.2:0/2562641547 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd5f00224e0 con 0x7fd604102070 2024-03-15T12:50:34.329 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:34.081+0000 7fcc3ea0e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:50:34.329 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:34.342+0000 7fcc3ea0e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-15T12:50:34.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:34.353+0000 7fd60ae2f700 1 -- 172.21.15.2:0/2562641547 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6040fe5c0 con 0x7fd5f4038f90 2024-03-15T12:50:34.441 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:34.164+0000 7f2adea50200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-15T12:50:34.441 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:34.274+0000 7f2adea50200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-15T12:50:34.441 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:34.359+0000 7f2adea50200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-15T12:50:34.441 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:34.453+0000 7f2adea50200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-15T12:50:34.651 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:34.577+0000 7fcc3ea0e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-15T12:50:34.861 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:34.631+0000 7f2adea50200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-15T12:50:34.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:34.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:34.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:34.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:34.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:34.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:34.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:34.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:34.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-15T12:50:34.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-15T12:50:34.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:34.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:50:34.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:50:34.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:34 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:35.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:35.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:35.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-15T12:50:35.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-15T12:50:35.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:35.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:50:35.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:50:35.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:34 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:35.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:35.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:35.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-15T12:50:35.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-15T12:50:35.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:35.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:50:35.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:50:35.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:34 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:35.658 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:35 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:35.372+0000 7fcc3ea0e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-15T12:50:35.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:35 smithi002 ceph-mon[32060]: Reconfiguring mgr.x (unknown last config time)... 2024-03-15T12:50:35.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:35 smithi002 ceph-mon[32060]: Reconfiguring daemon mgr.x on smithi002 2024-03-15T12:50:35.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:35 smithi002 ceph-mon[32060]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:50:35.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:35 smithi002 ceph-mon[32060]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:35.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:35 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:35 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:35 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:35.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:35 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:35.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:35 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:35.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:35 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:35 smithi149 ceph-mon[35927]: Reconfiguring mgr.x (unknown last config time)... 2024-03-15T12:50:35.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:35 smithi149 ceph-mon[35927]: Reconfiguring daemon mgr.x on smithi002 2024-03-15T12:50:35.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:35 smithi149 ceph-mon[35927]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:50:35.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:35 smithi149 ceph-mon[35927]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:35.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:35 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:35 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:35 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:35.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:35 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:35.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:35 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:35.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:35 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.862 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:35 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:35.619+0000 7f2adea50200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-15T12:50:35.862 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:35 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:35.751+0000 7f2adea50200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-15T12:50:35.932 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:35 smithi184 ceph-mon[35993]: Reconfiguring mgr.x (unknown last config time)... 2024-03-15T12:50:35.932 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:35 smithi184 ceph-mon[35993]: Reconfiguring daemon mgr.x on smithi002 2024-03-15T12:50:35.932 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:35 smithi184 ceph-mon[35993]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:50:35.932 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:35 smithi184 ceph-mon[35993]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:35.932 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:35 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.932 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:35 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:35.933 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:35 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:35.933 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:35 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:35.933 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:35 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:35.933 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:35 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:36.245 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:35 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:35.945+0000 7fcc3ea0e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-15T12:50:36.245 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:36 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:36.043+0000 7fcc3ea0e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-15T12:50:36.245 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:50:36 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:50:36.257+0000 7fcc3ea0e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-15T12:50:36.361 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:36 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:36.094+0000 7f2adea50200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-15T12:50:36.361 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:36 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:36.184+0000 7f2adea50200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-15T12:50:36.659 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:36 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:36.382+0000 7f2adea50200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:50:36.659 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:36 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:36.565+0000 7f2adea50200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-15T12:50:36.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:36.680+0000 7fd601ffb700 1 -- 172.21.15.2:0/2562641547 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 14) v1 ==== 91023+0+0 (secure 0 0 0) 0x7fd5f0019360 con 0x7fd604102070 2024-03-15T12:50:36.709 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:36 smithi002 ceph-mon[32060]: Standby manager daemon z started 2024-03-15T12:50:36.709 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:36 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.184:0/2266452212' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-15T12:50:36.709 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:36 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.184:0/2266452212' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-15T12:50:36.709 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:36 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.184:0/2266452212' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-15T12:50:36.709 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:36 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.184:0/2266452212' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-15T12:50:37.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:36 smithi184 ceph-mon[35993]: Standby manager daemon z started 2024-03-15T12:50:37.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:36 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.184:0/2266452212' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-15T12:50:37.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:36 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.184:0/2266452212' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-15T12:50:37.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:36 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.184:0/2266452212' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-15T12:50:37.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:36 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.184:0/2266452212' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-15T12:50:37.103 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:36 smithi149 ceph-mon[35927]: Standby manager daemon z started 2024-03-15T12:50:37.103 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:36 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.184:0/2266452212' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-15T12:50:37.103 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:36 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.184:0/2266452212' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-15T12:50:37.103 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:36 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.184:0/2266452212' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-15T12:50:37.103 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:36 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.184:0/2266452212' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-15T12:50:37.103 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:36 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:36.850+0000 7f2adea50200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:50:37.103 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:37 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:37.116+0000 7f2adea50200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-15T12:50:37.361 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:37 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:37.351+0000 7f2adea50200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-15T12:50:38.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:37 smithi002 ceph-mon[32060]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:38.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:37 smithi002 ceph-mon[32060]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-15T12:50:38.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:37 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-15T12:50:38.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:37 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1782846938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9c5bb62-482c-45e1-b63b-b487ca609054"}]: dispatch 2024-03-15T12:50:38.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:37 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1782846938' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9c5bb62-482c-45e1-b63b-b487ca609054"}]': finished 2024-03-15T12:50:38.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:37 smithi002 ceph-mon[32060]: osdmap e5: 1 total, 0 up, 1 in 2024-03-15T12:50:38.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:37 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:38.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:37 smithi184 ceph-mon[35993]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:38.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:37 smithi184 ceph-mon[35993]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-15T12:50:38.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:37 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-15T12:50:38.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:37 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1782846938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9c5bb62-482c-45e1-b63b-b487ca609054"}]: dispatch 2024-03-15T12:50:38.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:37 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1782846938' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9c5bb62-482c-45e1-b63b-b487ca609054"}]': finished 2024-03-15T12:50:38.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:37 smithi184 ceph-mon[35993]: osdmap e5: 1 total, 0 up, 1 in 2024-03-15T12:50:38.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:37 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:38.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:37 smithi149 ceph-mon[35927]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:38.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:37 smithi149 ceph-mon[35927]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-15T12:50:38.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:37 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-15T12:50:38.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:37 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1782846938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9c5bb62-482c-45e1-b63b-b487ca609054"}]: dispatch 2024-03-15T12:50:38.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:37 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1782846938' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9c5bb62-482c-45e1-b63b-b487ca609054"}]': finished 2024-03-15T12:50:38.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:37 smithi149 ceph-mon[35927]: osdmap e5: 1 total, 0 up, 1 in 2024-03-15T12:50:38.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:37 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:38.611 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:38 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:38.149+0000 7f2adea50200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-15T12:50:39.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:38 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1861712555' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:50:39.025 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:38 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:38.728+0000 7f2adea50200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-15T12:50:39.025 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:38 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:38.825+0000 7f2adea50200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-15T12:50:39.025 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:50:39 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:50:39.037+0000 7f2adea50200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-15T12:50:39.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:38 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1861712555' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:50:39.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:38 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1861712555' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:50:39.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:39.695+0000 7fd601ffb700 1 -- 172.21.15.2:0/2562641547 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mgrmap(e 15) v1 ==== 136222+0+0 (secure 0 0 0) 0x7fd5f0010640 con 0x7fd604102070 2024-03-15T12:50:40.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:39 smithi002 ceph-mon[32060]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:40.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:39 smithi002 ceph-mon[32060]: Standby manager daemon y started 2024-03-15T12:50:40.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:39 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.149:0/386400472' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-15T12:50:40.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:39 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.149:0/386400472' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-15T12:50:40.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:39 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.149:0/386400472' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-15T12:50:40.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:39 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.149:0/386400472' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-15T12:50:40.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:39 smithi184 ceph-mon[35993]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:40.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:39 smithi184 ceph-mon[35993]: Standby manager daemon y started 2024-03-15T12:50:40.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:39 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.149:0/386400472' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-15T12:50:40.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:39 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.149:0/386400472' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-15T12:50:40.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:39 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.149:0/386400472' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-15T12:50:40.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:39 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.149:0/386400472' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-15T12:50:40.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:39 smithi149 ceph-mon[35927]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:40.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:39 smithi149 ceph-mon[35927]: Standby manager daemon y started 2024-03-15T12:50:40.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:39 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.149:0/386400472' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-15T12:50:40.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:39 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.149:0/386400472' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-15T12:50:40.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:39 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.149:0/386400472' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-15T12:50:40.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:39 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.149:0/386400472' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-15T12:50:41.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:40 smithi002 ceph-mon[32060]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-03-15T12:50:41.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:40 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-15T12:50:41.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:40 smithi184 ceph-mon[35993]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-03-15T12:50:41.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:40 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-15T12:50:41.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:40 smithi149 ceph-mon[35927]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-03-15T12:50:41.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:40 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-15T12:50:41.872 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:41 smithi002 ceph-mon[32060]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:42.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:41 smithi184 ceph-mon[35993]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:42.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:41 smithi149 ceph-mon[35927]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:42.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:42 smithi002 ceph-mon[32060]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:43.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:42 smithi184 ceph-mon[35993]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:43.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:42 smithi149 ceph-mon[35927]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:44.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:44 smithi002 ceph-mon[32060]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:45.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:44 smithi184 ceph-mon[35993]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:45.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:44 smithi149 ceph-mon[35927]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:45.787 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:45 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-15T12:50:45.788 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:45 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:45.788 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:45 smithi002 ceph-mon[32060]: Deploying daemon osd.0 on smithi002 2024-03-15T12:50:46.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:45 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-15T12:50:46.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:45 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:46.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:45 smithi184 ceph-mon[35993]: Deploying daemon osd.0 on smithi002 2024-03-15T12:50:46.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:45 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-15T12:50:46.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:45 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:46.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:45 smithi149 ceph-mon[35927]: Deploying daemon osd.0 on smithi002 2024-03-15T12:50:46.836 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:46 smithi002 ceph-mon[32060]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:47.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:46 smithi184 ceph-mon[35993]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:47.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:46 smithi149 ceph-mon[35927]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:48.960 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:48 smithi002 ceph-mon[32060]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:49.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:48 smithi184 ceph-mon[35993]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:49.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:48 smithi149 ceph-mon[35927]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:51.456 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:51.457 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:51.457 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:51.457 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:51 smithi002 ceph-mon[32060]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:51.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:51.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:51.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:51.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:51 smithi184 ceph-mon[35993]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:51 smithi149 ceph-mon[35927]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:52.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:52 smithi002 ceph-mon[32060]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:53.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:52 smithi184 ceph-mon[35993]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:53.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:52 smithi149 ceph-mon[35927]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:53.762 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 0 on host 'smithi002' 2024-03-15T12:50:53.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:53.774+0000 7fd601ffb700 1 -- 172.21.15.2:0/2562641547 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd6040fe5c0 con 0x7fd5f4038f90 2024-03-15T12:50:53.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:53.777+0000 7fd60ae2f700 1 -- 172.21.15.2:0/2562641547 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd5f4038f90 msgr2=0x7fd5f403b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:53.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:53.777+0000 7fd60ae2f700 1 --2- 172.21.15.2:0/2562641547 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd5f4038f90 0x7fd5f403b450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8005950 tx=0x7fd5f800b410 comp rx=0 tx=0).stop 2024-03-15T12:50:53.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:53.777+0000 7fd60ae2f700 1 -- 172.21.15.2:0/2562641547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd604102070 msgr2=0x7fd6041034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:53.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:53.777+0000 7fd60ae2f700 1 --2- 172.21.15.2:0/2562641547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd604102070 0x7fd6041034e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000d950 tx=0x7fd5f000dc60 comp rx=0 tx=0).stop 2024-03-15T12:50:53.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:53.777+0000 7fd60ae2f700 1 -- 172.21.15.2:0/2562641547 shutdown_connections 2024-03-15T12:50:53.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:53.777+0000 7fd60ae2f700 1 --2- 172.21.15.2:0/2562641547 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd5f4038f90 0x7fd5f403b450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:53.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:53.777+0000 7fd60ae2f700 1 --2- 172.21.15.2:0/2562641547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd604102a30 0x7fd604103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:53.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:53.777+0000 7fd60ae2f700 1 --2- 172.21.15.2:0/2562641547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd604100f10 0x7fd604104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:53.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:53.777+0000 7fd60ae2f700 1 --2- 172.21.15.2:0/2562641547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd604102070 0x7fd6041034e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:53.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:53.777+0000 7fd60ae2f700 1 -- 172.21.15.2:0/2562641547 >> 172.21.15.2:0/2562641547 conn(0x7fd6040fc4b0 msgr2=0x7fd6040fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:50:53.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:53.777+0000 7fd60ae2f700 1 -- 172.21.15.2:0/2562641547 shutdown_connections 2024-03-15T12:50:53.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:50:53.777+0000 7fd60ae2f700 1 -- 172.21.15.2:0/2562641547 wait complete. 2024-03-15T12:50:54.032 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:53 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:54.032 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:53 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:54.032 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:53 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:54.032 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:53 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:54.033 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:53 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:54.033 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:53 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:54.033 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:53 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:54.033 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:53 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:54.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:53 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:54.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:53 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:54.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:53 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:54.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:53 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:54.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:53 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:54.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:53 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:54.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:53 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:54.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:53 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:54.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:53 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:54.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:53 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:54.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:53 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:54.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:53 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:54.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:53 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:54.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:53 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:54.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:53 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:50:54.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:53 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:54.404 DEBUG:teuthology.orchestra.run.smithi002:osd.0> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.0.service 2024-03-15T12:50:54.406 INFO:tasks.cephadm:Deploying osd.1 on smithi149 with /dev/vg_nvme/lv_4... 2024-03-15T12:50:54.407 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-15T12:50:54.651 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.b/config 2024-03-15T12:50:55.072 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:54 smithi149 ceph-mon[35927]: from='osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-15T12:50:55.072 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:54 smithi149 ceph-mon[35927]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:55.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:54 smithi002 ceph-mon[32060]: from='osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-15T12:50:55.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:54 smithi002 ceph-mon[32060]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:55.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:54 smithi184 ceph-mon[35993]: from='osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-15T12:50:55.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:54 smithi184 ceph-mon[35993]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:56.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:55 smithi002 ceph-mon[32060]: from='osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-15T12:50:56.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:55 smithi002 ceph-mon[32060]: osdmap e6: 1 total, 0 up, 1 in 2024-03-15T12:50:56.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:55 smithi002 ceph-mon[32060]: from='osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-15T12:50:56.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:56.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:55 smithi184 ceph-mon[35993]: from='osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-15T12:50:56.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:55 smithi184 ceph-mon[35993]: osdmap e6: 1 total, 0 up, 1 in 2024-03-15T12:50:56.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:55 smithi184 ceph-mon[35993]: from='osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-15T12:50:56.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:56.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:55 smithi149 ceph-mon[35927]: from='osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-15T12:50:56.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:55 smithi149 ceph-mon[35927]: osdmap e6: 1 total, 0 up, 1 in 2024-03-15T12:50:56.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:55 smithi149 ceph-mon[35927]: from='osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-15T12:50:56.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:56.408 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-15T12:50:56.408 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-15T12:50:56.408 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-15T12:50:56.408 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-15T12:50:56.408 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111959 s, 937 MB/s 2024-03-15T12:50:56.409 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 2024-03-15T12:50:56.409 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-15T12:50:56.774 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 12:50:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-0[46898]: 2024-03-15T12:50:56.533+0000 7f71b3240700 -1 osd.0 0 waiting for initial osdmap 2024-03-15T12:50:56.774 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 12:50:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-0[46898]: 2024-03-15T12:50:56.537+0000 7f71ab02b700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-15T12:50:57.031 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:56 smithi149 ceph-mon[35927]: from='osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-15T12:50:57.032 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:56 smithi149 ceph-mon[35927]: osdmap e7: 1 total, 0 up, 1 in 2024-03-15T12:50:57.032 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:56 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:57.032 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:56 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:57.032 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:56 smithi149 ceph-mon[35927]: from='osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355]' entity='osd.0' 2024-03-15T12:50:57.032 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:56 smithi149 ceph-mon[35927]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:57.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:56 smithi002 ceph-mon[32060]: from='osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-15T12:50:57.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:56 smithi002 ceph-mon[32060]: osdmap e7: 1 total, 0 up, 1 in 2024-03-15T12:50:57.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:56 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:57.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:56 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:57.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:56 smithi002 ceph-mon[32060]: from='osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355]' entity='osd.0' 2024-03-15T12:50:57.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:56 smithi002 ceph-mon[32060]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:57.059 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_4 2024-03-15T12:50:57.307 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.b/config 2024-03-15T12:50:57.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:56 smithi184 ceph-mon[35993]: from='osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-15T12:50:57.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:56 smithi184 ceph-mon[35993]: osdmap e7: 1 total, 0 up, 1 in 2024-03-15T12:50:57.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:56 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:57.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:56 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:57.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:56 smithi184 ceph-mon[35993]: from='osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355]' entity='osd.0' 2024-03-15T12:50:57.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:56 smithi184 ceph-mon[35993]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-15T12:50:58.212 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:57 smithi184 ceph-mon[35993]: purged_snaps scrub starts 2024-03-15T12:50:58.212 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:57 smithi184 ceph-mon[35993]: purged_snaps scrub ok 2024-03-15T12:50:58.212 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:58.212 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:57 smithi184 ceph-mon[35993]: Detected new or changed devices on smithi002 2024-03-15T12:50:58.212 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:58.212 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:58.212 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:50:58.212 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:57 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi002 to 17093M 2024-03-15T12:50:58.212 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:58.212 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:58.212 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:58.212 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:58.213 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:57 smithi184 ceph-mon[35993]: osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355] boot 2024-03-15T12:50:58.213 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:57 smithi184 ceph-mon[35993]: osdmap e8: 1 total, 1 up, 1 in 2024-03-15T12:50:58.213 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:58.283 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:57 smithi149 ceph-mon[35927]: purged_snaps scrub starts 2024-03-15T12:50:58.283 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:57 smithi149 ceph-mon[35927]: purged_snaps scrub ok 2024-03-15T12:50:58.283 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:58.284 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:57 smithi149 ceph-mon[35927]: Detected new or changed devices on smithi002 2024-03-15T12:50:58.284 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:58.284 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:58.284 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:50:58.284 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:57 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi002 to 17093M 2024-03-15T12:50:58.284 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:58.284 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:58.284 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:58.284 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:58.284 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:57 smithi149 ceph-mon[35927]: osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355] boot 2024-03-15T12:50:58.284 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:57 smithi149 ceph-mon[35927]: osdmap e8: 1 total, 1 up, 1 in 2024-03-15T12:50:58.284 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:58.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:57 smithi002 ceph-mon[32060]: purged_snaps scrub starts 2024-03-15T12:50:58.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:57 smithi002 ceph-mon[32060]: purged_snaps scrub ok 2024-03-15T12:50:58.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:58.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:57 smithi002 ceph-mon[32060]: Detected new or changed devices on smithi002 2024-03-15T12:50:58.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:58.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:58.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:50:58.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:57 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi002 to 17093M 2024-03-15T12:50:58.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:58.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:50:58.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:50:58.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:50:58.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:57 smithi002 ceph-mon[32060]: osd.0 [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355] boot 2024-03-15T12:50:58.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:57 smithi002 ceph-mon[32060]: osdmap e8: 1 total, 1 up, 1 in 2024-03-15T12:50:58.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:50:58.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.800+0000 7fb5cb34f700 1 -- 172.21.15.149:0/624858738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c4103bb0 msgr2=0x7fb5c410a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:58.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.800+0000 7fb5cb34f700 1 --2- 172.21.15.149:0/624858738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c4103bb0 0x7fb5c410a460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0099c0 tx=0x7fb5bc009cd0 comp rx=0 tx=0).stop 2024-03-15T12:50:58.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.800+0000 7fb5cb34f700 1 -- 172.21.15.149:0/624858738 shutdown_connections 2024-03-15T12:50:58.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.800+0000 7fb5cb34f700 1 --2- 172.21.15.149:0/624858738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c4102010 0x7fb5c4102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:58.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.800+0000 7fb5cb34f700 1 --2- 172.21.15.149:0/624858738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c4103bb0 0x7fb5c410a460 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:58.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.800+0000 7fb5cb34f700 1 --2- 172.21.15.149:0/624858738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5c4103210 0x7fb5c4103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:58.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.800+0000 7fb5cb34f700 1 -- 172.21.15.149:0/624858738 >> 172.21.15.149:0/624858738 conn(0x7fb5c40fd5f0 msgr2=0x7fb5c40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:50:58.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.800+0000 7fb5cb34f700 1 -- 172.21.15.149:0/624858738 shutdown_connections 2024-03-15T12:50:58.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.800+0000 7fb5cb34f700 1 -- 172.21.15.149:0/624858738 wait complete. 2024-03-15T12:50:58.790 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.800+0000 7fb5cb34f700 1 Processor -- start 2024-03-15T12:50:58.790 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5cb34f700 1 -- start start 2024-03-15T12:50:58.790 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5cb34f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c4102010 0x7fb5c4197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:58.790 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5cb34f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5c4103210 0x7fb5c4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:58.790 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5cb34f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c4103bb0 0x7fb5c419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:58.790 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5cb34f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c419fa60 con 0x7fb5c4103bb0 2024-03-15T12:50:58.790 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5cb34f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c419fba0 con 0x7fb5c4102010 2024-03-15T12:50:58.790 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5cb34f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c419fd30 con 0x7fb5c4103210 2024-03-15T12:50:58.790 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5c90eb700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c4102010 0x7fb5c4197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:58.790 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5c90eb700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c4102010 0x7fb5c4197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:49688/0 (socket says 172.21.15.149:49688) 2024-03-15T12:50:58.790 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5c90eb700 1 -- 172.21.15.149:0/2641410612 learned_addr learned my addr 172.21.15.149:0/2641410612 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-15T12:50:58.790 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5c98ec700 1 --2- 172.21.15.149:0/2641410612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c4103bb0 0x7fb5c419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:58.790 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5c88ea700 1 --2- 172.21.15.149:0/2641410612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5c4103210 0x7fb5c4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:58.791 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5c90eb700 1 -- 172.21.15.149:0/2641410612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5c4103210 msgr2=0x7fb5c4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:58.791 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5c90eb700 1 --2- 172.21.15.149:0/2641410612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5c4103210 0x7fb5c4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:58.791 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5c90eb700 1 -- 172.21.15.149:0/2641410612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c4103bb0 msgr2=0x7fb5c419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:50:58.791 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5c90eb700 1 --2- 172.21.15.149:0/2641410612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c4103bb0 0x7fb5c419f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:50:58.791 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5c90eb700 1 -- 172.21.15.149:0/2641410612 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc0095f0 con 0x7fb5c4102010 2024-03-15T12:50:58.791 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5c88ea700 1 --2- 172.21.15.149:0/2641410612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5c4103210 0x7fb5c4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:50:58.791 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5c90eb700 1 --2- 172.21.15.149:0/2641410612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c4102010 0x7fb5c4197ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0007f10 tx=0x7fb5c000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:50:58.791 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5ba7fc700 1 -- 172.21.15.149:0/2641410612 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0017070 con 0x7fb5c4102010 2024-03-15T12:50:58.791 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5ba7fc700 1 -- 172.21.15.149:0/2641410612 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb5c000f040 con 0x7fb5c4102010 2024-03-15T12:50:58.791 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5ba7fc700 1 -- 172.21.15.149:0/2641410612 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00046f0 con 0x7fb5c4102010 2024-03-15T12:50:58.791 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.801+0000 7fb5cb34f700 1 -- 172.21.15.149:0/2641410612 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c419ffb0 con 0x7fb5c4102010 2024-03-15T12:50:58.791 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.802+0000 7fb5cb34f700 1 -- 172.21.15.149:0/2641410612 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c41a0470 con 0x7fb5c4102010 2024-03-15T12:50:58.792 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.803+0000 7fb5cb34f700 1 -- 172.21.15.149:0/2641410612 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c41093d0 con 0x7fb5c4102010 2024-03-15T12:50:58.792 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.803+0000 7fb5ba7fc700 1 -- 172.21.15.149:0/2641410612 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 15) v1 ==== 136222+0+0 (secure 0 0 0) 0x7fb5c0003a00 con 0x7fb5c4102010 2024-03-15T12:50:58.792 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.804+0000 7fb5ba7fc700 1 --2- 172.21.15.149:0/2641410612 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb5b00a1ab0 0x7fb5b00a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:50:58.792 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.804+0000 7fb5ba7fc700 1 -- 172.21.15.149:0/2641410612 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fb5c00cafa0 con 0x7fb5c4102010 2024-03-15T12:50:58.792 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.804+0000 7fb5c88ea700 1 --2- 172.21.15.149:0/2641410612 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb5b00a1ab0 0x7fb5b00a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:50:58.793 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.805+0000 7fb5c88ea700 1 --2- 172.21.15.149:0/2641410612 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb5b00a1ab0 0x7fb5b00a3f70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400bc10 tx=0x7fb5b400b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:50:58.796 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:58.809+0000 7fb5ba7fc700 1 -- 172.21.15.149:0/2641410612 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb5c0095c20 con 0x7fb5c4102010 2024-03-15T12:50:59.000 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:50:59.013+0000 7fb5cb34f700 1 -- 172.21.15.149:0/2641410612 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5c402da90 con 0x7fb5b00a1ab0 2024-03-15T12:50:59.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:58 smithi002 ceph-mon[32060]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:50:59.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:58 smithi184 ceph-mon[35993]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:50:59.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:58 smithi149 ceph-mon[35927]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:00.195 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:59 smithi149 ceph-mon[35927]: osdmap e9: 1 total, 1 up, 1 in 2024-03-15T12:51:00.196 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:59 smithi149 ceph-mon[35927]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:51:00.196 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:51:00.196 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:51:00.196 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:50:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:00.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:59 smithi002 ceph-mon[32060]: osdmap e9: 1 total, 1 up, 1 in 2024-03-15T12:51:00.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:59 smithi002 ceph-mon[32060]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:51:00.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:51:00.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:51:00.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:50:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:00.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:59 smithi184 ceph-mon[35993]: osdmap e9: 1 total, 1 up, 1 in 2024-03-15T12:51:00.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:59 smithi184 ceph-mon[35993]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:51:00.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:51:00.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:51:00.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:50:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:01.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:00 smithi002 ceph-mon[32060]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:01.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:00 smithi184 ceph-mon[35993]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:01.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:00 smithi149 ceph-mon[35927]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:02.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:01 smithi002 ceph-mon[32060]: from='client.? 172.21.15.149:0/3560815471' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5182e55d-f2cf-40be-acff-0534c2bce7ac"}]: dispatch 2024-03-15T12:51:02.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:01 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5182e55d-f2cf-40be-acff-0534c2bce7ac"}]: dispatch 2024-03-15T12:51:02.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:01 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5182e55d-f2cf-40be-acff-0534c2bce7ac"}]': finished 2024-03-15T12:51:02.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:01 smithi002 ceph-mon[32060]: osdmap e10: 2 total, 1 up, 2 in 2024-03-15T12:51:02.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:01 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:02.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:01 smithi184 ceph-mon[35993]: from='client.? 172.21.15.149:0/3560815471' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5182e55d-f2cf-40be-acff-0534c2bce7ac"}]: dispatch 2024-03-15T12:51:02.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:01 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5182e55d-f2cf-40be-acff-0534c2bce7ac"}]: dispatch 2024-03-15T12:51:02.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:01 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5182e55d-f2cf-40be-acff-0534c2bce7ac"}]': finished 2024-03-15T12:51:02.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:01 smithi184 ceph-mon[35993]: osdmap e10: 2 total, 1 up, 2 in 2024-03-15T12:51:02.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:01 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:02.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:01 smithi149 ceph-mon[35927]: from='client.? 172.21.15.149:0/3560815471' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5182e55d-f2cf-40be-acff-0534c2bce7ac"}]: dispatch 2024-03-15T12:51:02.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:01 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5182e55d-f2cf-40be-acff-0534c2bce7ac"}]: dispatch 2024-03-15T12:51:02.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:01 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5182e55d-f2cf-40be-acff-0534c2bce7ac"}]': finished 2024-03-15T12:51:02.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:01 smithi149 ceph-mon[35927]: osdmap e10: 2 total, 1 up, 2 in 2024-03-15T12:51:02.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:01 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:03.235 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:02 smithi002 ceph-mon[32060]: from='client.? 172.21.15.149:0/3798702529' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:51:03.235 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:02 smithi002 ceph-mon[32060]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:03.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:02 smithi184 ceph-mon[35993]: from='client.? 172.21.15.149:0/3798702529' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:51:03.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:02 smithi184 ceph-mon[35993]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:03.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:02 smithi149 ceph-mon[35927]: from='client.? 172.21.15.149:0/3798702529' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:51:03.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:02 smithi149 ceph-mon[35927]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:05.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:04 smithi002 ceph-mon[32060]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:05.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:04 smithi184 ceph-mon[35993]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:05.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:04 smithi149 ceph-mon[35927]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:06.806 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:06 smithi149 ceph-mon[35927]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:07.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:06 smithi002 ceph-mon[32060]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:07.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:06 smithi184 ceph-mon[35993]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:08.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:08 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-15T12:51:08.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:08 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:09.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-15T12:51:09.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:09.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:08 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-15T12:51:09.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:08 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:09.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:09 smithi149 ceph-mon[35927]: Deploying daemon osd.1 on smithi149 2024-03-15T12:51:09.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:09 smithi149 ceph-mon[35927]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:09.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:09 smithi184 ceph-mon[35993]: Deploying daemon osd.1 on smithi149 2024-03-15T12:51:09.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:09 smithi184 ceph-mon[35993]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:09 smithi002 ceph-mon[32060]: Deploying daemon osd.1 on smithi149 2024-03-15T12:51:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:09 smithi002 ceph-mon[32060]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:11.980 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:11 smithi149 ceph-mon[35927]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:12.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:11 smithi002 ceph-mon[32060]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:12.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:11 smithi184 ceph-mon[35993]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:13.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:13 smithi149 ceph-mon[35927]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:14.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:13 smithi002 ceph-mon[32060]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:14.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:13 smithi184 ceph-mon[35993]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:15.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:14 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:15.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:14 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:15.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:14 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:51:15.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:14 smithi149 ceph-mon[35927]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:15.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:14 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:15.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:14 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:15.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:14 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:51:15.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:14 smithi002 ceph-mon[32060]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:14 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:14 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:14 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:51:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:14 smithi184 ceph-mon[35993]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:17.215 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:51:17.228+0000 7fb5ba7fc700 1 -- 172.21.15.149:0/2641410612 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb5c402da90 con 0x7fb5b00a1ab0 2024-03-15T12:51:17.216 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 1 on host 'smithi149' 2024-03-15T12:51:17.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:51:17.231+0000 7fb5cb34f700 1 -- 172.21.15.149:0/2641410612 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb5b00a1ab0 msgr2=0x7fb5b00a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:51:17.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:51:17.231+0000 7fb5cb34f700 1 --2- 172.21.15.149:0/2641410612 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb5b00a1ab0 0x7fb5b00a3f70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400bc10 tx=0x7fb5b400b5f0 comp rx=0 tx=0).stop 2024-03-15T12:51:17.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:51:17.231+0000 7fb5cb34f700 1 -- 172.21.15.149:0/2641410612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c4102010 msgr2=0x7fb5c4197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:51:17.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:51:17.231+0000 7fb5cb34f700 1 --2- 172.21.15.149:0/2641410612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c4102010 0x7fb5c4197ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0007f10 tx=0x7fb5c000d3b0 comp rx=0 tx=0).stop 2024-03-15T12:51:17.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:51:17.231+0000 7fb5cb34f700 1 -- 172.21.15.149:0/2641410612 shutdown_connections 2024-03-15T12:51:17.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:51:17.231+0000 7fb5cb34f700 1 --2- 172.21.15.149:0/2641410612 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb5b00a1ab0 0x7fb5b00a3f70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:17.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:51:17.231+0000 7fb5cb34f700 1 --2- 172.21.15.149:0/2641410612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c4103bb0 0x7fb5c419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:17.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:51:17.231+0000 7fb5cb34f700 1 --2- 172.21.15.149:0/2641410612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c4102010 0x7fb5c4197ed0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:17.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:51:17.231+0000 7fb5cb34f700 1 --2- 172.21.15.149:0/2641410612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5c4103210 0x7fb5c4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:17.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:51:17.231+0000 7fb5cb34f700 1 -- 172.21.15.149:0/2641410612 >> 172.21.15.149:0/2641410612 conn(0x7fb5c40fd5f0 msgr2=0x7fb5c4106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:51:17.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:51:17.231+0000 7fb5cb34f700 1 -- 172.21.15.149:0/2641410612 shutdown_connections 2024-03-15T12:51:17.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:51:17.231+0000 7fb5cb34f700 1 -- 172.21.15.149:0/2641410612 wait complete. 2024-03-15T12:51:17.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:17.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:17.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:17.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:51:17.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:17.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:17 smithi149 ceph-mon[35927]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:17.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:17.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:17.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:51:17.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:17.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:17.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:17.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:51:17.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:17.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:17 smithi002 ceph-mon[32060]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:17.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:17.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:17.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:51:17.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:17.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:17.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:17.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:51:17.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:17.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:17 smithi184 ceph-mon[35993]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:17.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:17.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:17.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:51:17.841 DEBUG:teuthology.orchestra.run.smithi149:osd.1> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.1.service 2024-03-15T12:51:17.843 INFO:tasks.cephadm:Deploying osd.2 on smithi184 with /dev/vg_nvme/lv_4... 2024-03-15T12:51:17.844 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-15T12:51:18.091 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T12:51:18.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:18 smithi184 ceph-mon[35993]: from='osd.1 [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-15T12:51:18.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:18 smithi184 ceph-mon[35993]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-15T12:51:18.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:18 smithi149 ceph-mon[35927]: from='osd.1 [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-15T12:51:18.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:18 smithi149 ceph-mon[35927]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-15T12:51:18.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:18 smithi002 ceph-mon[32060]: from='osd.1 [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-15T12:51:18.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:18 smithi002 ceph-mon[32060]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-15T12:51:19.394 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:19 smithi184 ceph-mon[35993]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-15T12:51:19.394 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:19 smithi184 ceph-mon[35993]: from='osd.1 [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:51:19.394 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:19 smithi184 ceph-mon[35993]: osdmap e11: 2 total, 1 up, 2 in 2024-03-15T12:51:19.394 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:19 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:19.394 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:19 smithi184 ceph-mon[35993]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:51:19.394 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:19 smithi184 ceph-mon[35993]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:19.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:19 smithi002 ceph-mon[32060]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-15T12:51:19.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:19 smithi002 ceph-mon[32060]: from='osd.1 [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:51:19.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:19 smithi002 ceph-mon[32060]: osdmap e11: 2 total, 1 up, 2 in 2024-03-15T12:51:19.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:19 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:19.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:19 smithi002 ceph-mon[32060]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:51:19.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:19 smithi002 ceph-mon[32060]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:19.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:19 smithi149 ceph-mon[35927]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-15T12:51:19.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:19 smithi149 ceph-mon[35927]: from='osd.1 [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:51:19.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:19 smithi149 ceph-mon[35927]: osdmap e11: 2 total, 1 up, 2 in 2024-03-15T12:51:19.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:19.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:19 smithi149 ceph-mon[35927]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:51:19.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:19 smithi149 ceph-mon[35927]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:19.979 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-15T12:51:19.980 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-15T12:51:19.980 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-03-15T12:51:19.980 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-03-15T12:51:19.980 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.01113 s, 942 MB/s 2024-03-15T12:51:19.980 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-03-15T12:51:20.294 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 12:51:20 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-1[40532]: 2024-03-15T12:51:20.000+0000 7f6178b3a700 -1 osd.1 0 waiting for initial osdmap 2024-03-15T12:51:20.294 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 12:51:20 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-1[40532]: 2024-03-15T12:51:20.005+0000 7f6170925700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-15T12:51:20.540 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:20 smithi184 ceph-mon[35993]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-15T12:51:20.540 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:20 smithi184 ceph-mon[35993]: osdmap e12: 2 total, 1 up, 2 in 2024-03-15T12:51:20.540 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:20.540 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:20.540 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:20 smithi184 ceph-mon[35993]: from='osd.1 ' entity='osd.1' 2024-03-15T12:51:20.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:20 smithi002 ceph-mon[32060]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-15T12:51:20.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:20 smithi002 ceph-mon[32060]: osdmap e12: 2 total, 1 up, 2 in 2024-03-15T12:51:20.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:20.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:20.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:20 smithi002 ceph-mon[32060]: from='osd.1 ' entity='osd.1' 2024-03-15T12:51:20.569 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi184:vg_nvme/lv_4 2024-03-15T12:51:20.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:20 smithi149 ceph-mon[35927]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-15T12:51:20.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:20 smithi149 ceph-mon[35927]: osdmap e12: 2 total, 1 up, 2 in 2024-03-15T12:51:20.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:20.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:20.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:20 smithi149 ceph-mon[35927]: from='osd.1 ' entity='osd.1' 2024-03-15T12:51:20.833 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T12:51:21.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:21 smithi002 ceph-mon[32060]: purged_snaps scrub starts 2024-03-15T12:51:21.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:21 smithi002 ceph-mon[32060]: purged_snaps scrub ok 2024-03-15T12:51:21.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:21.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:21 smithi002 ceph-mon[32060]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:21.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:21.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:21.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:51:21.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:21.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:21.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:51:21.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:21.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:21 smithi002 ceph-mon[32060]: osd.1 [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987] boot 2024-03-15T12:51:21.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:21 smithi002 ceph-mon[32060]: osdmap e13: 2 total, 2 up, 2 in 2024-03-15T12:51:21.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:21.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:21 smithi184 ceph-mon[35993]: purged_snaps scrub starts 2024-03-15T12:51:21.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:21 smithi184 ceph-mon[35993]: purged_snaps scrub ok 2024-03-15T12:51:21.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:21 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:21.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:21 smithi184 ceph-mon[35993]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:21.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:21 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:21.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:21 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:21.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:21 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:51:21.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:21 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:21.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:21 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:21.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:21 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:51:21.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:21 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:21.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:21 smithi184 ceph-mon[35993]: osd.1 [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987] boot 2024-03-15T12:51:21.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:21 smithi184 ceph-mon[35993]: osdmap e13: 2 total, 2 up, 2 in 2024-03-15T12:51:21.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:21 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:21.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:21 smithi149 ceph-mon[35927]: purged_snaps scrub starts 2024-03-15T12:51:21.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:21 smithi149 ceph-mon[35927]: purged_snaps scrub ok 2024-03-15T12:51:21.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:21 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:21.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:21 smithi149 ceph-mon[35927]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-03-15T12:51:21.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:21 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:21.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:21 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:21.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:21 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:51:21.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:21 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:21.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:21 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:21.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:21 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:51:21.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:21 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:21.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:21 smithi149 ceph-mon[35927]: osd.1 [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987] boot 2024-03-15T12:51:21.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:21 smithi149 ceph-mon[35927]: osdmap e13: 2 total, 2 up, 2 in 2024-03-15T12:51:21.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:21 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:51:22.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.390+0000 7ff7dba12700 1 -- 172.21.15.184:0/3058919937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d41006e0 msgr2=0x7ff7d4100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:51:22.380 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.390+0000 7ff7dba12700 1 --2- 172.21.15.184:0/3058919937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d41006e0 0x7ff7d4100b00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0009990 tx=0x7ff7d0009ca0 comp rx=0 tx=0).stop 2024-03-15T12:51:22.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.391+0000 7ff7dba12700 1 -- 172.21.15.184:0/3058919937 shutdown_connections 2024-03-15T12:51:22.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.391+0000 7ff7dba12700 1 --2- 172.21.15.184:0/3058919937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7d4102280 0x7ff7d4108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:22.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.391+0000 7ff7dba12700 1 --2- 172.21.15.184:0/3058919937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d41006e0 0x7ff7d4100b00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:22.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.391+0000 7ff7dba12700 1 --2- 172.21.15.184:0/3058919937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d41018e0 0x7ff7d4101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:22.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.391+0000 7ff7dba12700 1 -- 172.21.15.184:0/3058919937 >> 172.21.15.184:0/3058919937 conn(0x7ff7d40fbc80 msgr2=0x7ff7d40fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:51:22.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.391+0000 7ff7dba12700 1 -- 172.21.15.184:0/3058919937 shutdown_connections 2024-03-15T12:51:22.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.391+0000 7ff7dba12700 1 -- 172.21.15.184:0/3058919937 wait complete. 2024-03-15T12:51:22.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.391+0000 7ff7dba12700 1 Processor -- start 2024-03-15T12:51:22.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.391+0000 7ff7dba12700 1 -- start start 2024-03-15T12:51:22.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7dba12700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d41006e0 0x7ff7d4196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:51:22.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7dba12700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d41018e0 0x7ff7d41974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:51:22.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7dba12700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7d4102280 0x7ff7d419e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:51:22.381 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7dba12700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d419eb20 con 0x7ff7d41006e0 2024-03-15T12:51:22.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7dba12700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d419ec60 con 0x7ff7d4102280 2024-03-15T12:51:22.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7dba12700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d419edf0 con 0x7ff7d41018e0 2024-03-15T12:51:22.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7d8fad700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d41018e0 0x7ff7d41974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:51:22.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7d8fad700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d41018e0 0x7ff7d41974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:44236/0 (socket says 172.21.15.184:44236) 2024-03-15T12:51:22.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7d8fad700 1 -- 172.21.15.184:0/2120041454 learned_addr learned my addr 172.21.15.184:0/2120041454 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-15T12:51:22.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7d97ae700 1 --2- 172.21.15.184:0/2120041454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d41006e0 0x7ff7d4196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:51:22.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7d9faf700 1 --2- 172.21.15.184:0/2120041454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7d4102280 0x7ff7d419e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:51:22.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7d8fad700 1 -- 172.21.15.184:0/2120041454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7d4102280 msgr2=0x7ff7d419e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:51:22.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7d8fad700 1 --2- 172.21.15.184:0/2120041454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7d4102280 0x7ff7d419e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:22.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7d8fad700 1 -- 172.21.15.184:0/2120041454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d41006e0 msgr2=0x7ff7d4196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:51:22.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7d8fad700 1 --2- 172.21.15.184:0/2120041454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d41006e0 0x7ff7d4196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:22.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7d8fad700 1 -- 172.21.15.184:0/2120041454 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d00095f0 con 0x7ff7d41018e0 2024-03-15T12:51:22.382 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7d9faf700 1 --2- 172.21.15.184:0/2120041454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7d4102280 0x7ff7d419e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:51:22.383 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7d8fad700 1 --2- 172.21.15.184:0/2120041454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d41018e0 0x7ff7d41974d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400b560 tx=0x7ff7c400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:51:22.383 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7ca7fc700 1 -- 172.21.15.184:0/2120041454 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4011830 con 0x7ff7d41018e0 2024-03-15T12:51:22.383 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7ca7fc700 1 -- 172.21.15.184:0/2120041454 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff7c4011e60 con 0x7ff7d41018e0 2024-03-15T12:51:22.383 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7dba12700 1 -- 172.21.15.184:0/2120041454 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d419f0d0 con 0x7ff7d41018e0 2024-03-15T12:51:22.383 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7ca7fc700 1 -- 172.21.15.184:0/2120041454 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400f2e0 con 0x7ff7d41018e0 2024-03-15T12:51:22.383 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.392+0000 7ff7dba12700 1 -- 172.21.15.184:0/2120041454 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d419f650 con 0x7ff7d41018e0 2024-03-15T12:51:22.383 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.393+0000 7ff7dba12700 1 -- 172.21.15.184:0/2120041454 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d4107a40 con 0x7ff7d41018e0 2024-03-15T12:51:22.384 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.394+0000 7ff7ca7fc700 1 -- 172.21.15.184:0/2120041454 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 15) v1 ==== 136222+0+0 (secure 0 0 0) 0x7ff7c4010380 con 0x7ff7d41018e0 2024-03-15T12:51:22.384 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.395+0000 7ff7ca7fc700 1 --2- 172.21.15.184:0/2120041454 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff7c00a1a30 0x7ff7c00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:51:22.384 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.395+0000 7ff7ca7fc700 1 -- 172.21.15.184:0/2120041454 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7ff7c40cb650 con 0x7ff7d41018e0 2024-03-15T12:51:22.384 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.395+0000 7ff7d97ae700 1 --2- 172.21.15.184:0/2120041454 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff7c00a1a30 0x7ff7c00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:51:22.384 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.396+0000 7ff7d97ae700 1 --2- 172.21.15.184:0/2120041454 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff7c00a1a30 0x7ff7c00a3ef0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff7d00037f0 tx=0x7ff7d00038c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:51:22.386 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.400+0000 7ff7ca7fc700 1 -- 172.21.15.184:0/2120041454 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff7c40960b0 con 0x7ff7d41018e0 2024-03-15T12:51:22.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:22 smithi002 ceph-mon[32060]: Detected new or changed devices on smithi149 2024-03-15T12:51:22.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:22 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi149 to 5816M 2024-03-15T12:51:22.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:22 smithi184 ceph-mon[35993]: Detected new or changed devices on smithi149 2024-03-15T12:51:22.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:22 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi149 to 5816M 2024-03-15T12:51:22.586 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:22.599+0000 7ff7dba12700 1 -- 172.21.15.184:0/2120041454 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7d402da30 con 0x7ff7c00a1a30 2024-03-15T12:51:22.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:22 smithi149 ceph-mon[35927]: Detected new or changed devices on smithi149 2024-03-15T12:51:22.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:22 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi149 to 5816M 2024-03-15T12:51:23.510 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:23 smithi184 ceph-mon[35993]: osdmap e14: 2 total, 2 up, 2 in 2024-03-15T12:51:23.510 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:23 smithi184 ceph-mon[35993]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:51:23.510 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:23 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:51:23.510 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:23 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:51:23.510 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:23 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:23.510 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:23 smithi184 ceph-mon[35993]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:23.589 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:23 smithi002 ceph-mon[32060]: osdmap e14: 2 total, 2 up, 2 in 2024-03-15T12:51:23.589 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:23 smithi002 ceph-mon[32060]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:51:23.589 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:23 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:51:23.589 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:23 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:51:23.589 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:23 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:23.589 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:23 smithi002 ceph-mon[32060]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:23.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:23 smithi149 ceph-mon[35927]: osdmap e14: 2 total, 2 up, 2 in 2024-03-15T12:51:23.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:23 smithi149 ceph-mon[35927]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:51:23.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:23 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:51:23.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:23 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:51:23.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:23 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:23.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:23 smithi149 ceph-mon[35927]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:24.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:24 smithi184 ceph-mon[35993]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:25.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:24 smithi002 ceph-mon[32060]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:25.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:24 smithi149 ceph-mon[35927]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:26.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:25 smithi002 ceph-mon[32060]: from='client.? 172.21.15.184:0/1990662602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8db2a1c3-c04d-4b78-ad1c-e587fad5ce88"}]: dispatch 2024-03-15T12:51:26.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:25 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8db2a1c3-c04d-4b78-ad1c-e587fad5ce88"}]: dispatch 2024-03-15T12:51:26.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:25 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8db2a1c3-c04d-4b78-ad1c-e587fad5ce88"}]': finished 2024-03-15T12:51:26.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:25 smithi002 ceph-mon[32060]: osdmap e15: 3 total, 2 up, 3 in 2024-03-15T12:51:26.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:25 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:26.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:25 smithi184 ceph-mon[35993]: from='client.? 172.21.15.184:0/1990662602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8db2a1c3-c04d-4b78-ad1c-e587fad5ce88"}]: dispatch 2024-03-15T12:51:26.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:25 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8db2a1c3-c04d-4b78-ad1c-e587fad5ce88"}]: dispatch 2024-03-15T12:51:26.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:25 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8db2a1c3-c04d-4b78-ad1c-e587fad5ce88"}]': finished 2024-03-15T12:51:26.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:25 smithi184 ceph-mon[35993]: osdmap e15: 3 total, 2 up, 3 in 2024-03-15T12:51:26.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:25 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:26.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:25 smithi149 ceph-mon[35927]: from='client.? 172.21.15.184:0/1990662602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8db2a1c3-c04d-4b78-ad1c-e587fad5ce88"}]: dispatch 2024-03-15T12:51:26.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:25 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8db2a1c3-c04d-4b78-ad1c-e587fad5ce88"}]: dispatch 2024-03-15T12:51:26.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:25 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8db2a1c3-c04d-4b78-ad1c-e587fad5ce88"}]': finished 2024-03-15T12:51:26.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:25 smithi149 ceph-mon[35927]: osdmap e15: 3 total, 2 up, 3 in 2024-03-15T12:51:26.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:25 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:27.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:26 smithi002 ceph-mon[32060]: from='client.? 172.21.15.184:0/98342052' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:51:27.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:26 smithi002 ceph-mon[32060]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:27.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:26 smithi184 ceph-mon[35993]: from='client.? 172.21.15.184:0/98342052' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:51:27.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:26 smithi184 ceph-mon[35993]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:27.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:26 smithi149 ceph-mon[35927]: from='client.? 172.21.15.184:0/98342052' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:51:27.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:26 smithi149 ceph-mon[35927]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:29.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:28 smithi002 ceph-mon[32060]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:29.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:28 smithi184 ceph-mon[35993]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:29.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:28 smithi149 ceph-mon[35927]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:31.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:30 smithi002 ceph-mon[32060]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:31.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:30 smithi184 ceph-mon[35993]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:31.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:30 smithi149 ceph-mon[35927]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:32.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:32 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-15T12:51:32.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:32 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:32.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:32 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-15T12:51:32.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:32 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:32.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:32 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-15T12:51:32.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:32 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:33.531 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:33 smithi184 ceph-mon[35993]: Deploying daemon osd.2 on smithi184 2024-03-15T12:51:33.531 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:33 smithi184 ceph-mon[35993]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:33.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:33 smithi002 ceph-mon[32060]: Deploying daemon osd.2 on smithi184 2024-03-15T12:51:33.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:33 smithi002 ceph-mon[32060]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:33.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:33 smithi149 ceph-mon[35927]: Deploying daemon osd.2 on smithi184 2024-03-15T12:51:33.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:33 smithi149 ceph-mon[35927]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:35.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:34 smithi002 ceph-mon[32060]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:35.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:34 smithi184 ceph-mon[35993]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:35.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:34 smithi149 ceph-mon[35927]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:37.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:36 smithi002 ceph-mon[32060]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:37.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:36 smithi184 ceph-mon[35993]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:37.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:36 smithi149 ceph-mon[35927]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:38.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:38 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:38.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:38 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:38.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:38 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:51:38.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:38 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:38.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:38 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:38.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:38 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:51:38.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:38 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:38.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:38 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:38.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:38 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:51:39.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:39 smithi184 ceph-mon[35993]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:39.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:39 smithi002 ceph-mon[32060]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:39.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:39 smithi149 ceph-mon[35927]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:41.001 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:41.014+0000 7ff7ca7fc700 1 -- 172.21.15.184:0/2120041454 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff7d402da30 con 0x7ff7c00a1a30 2024-03-15T12:51:41.002 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 2 on host 'smithi184' 2024-03-15T12:51:41.003 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:41.016+0000 7ff7dba12700 1 -- 172.21.15.184:0/2120041454 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff7c00a1a30 msgr2=0x7ff7c00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:51:41.003 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:41.016+0000 7ff7dba12700 1 --2- 172.21.15.184:0/2120041454 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff7c00a1a30 0x7ff7c00a3ef0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff7d00037f0 tx=0x7ff7d00038c0 comp rx=0 tx=0).stop 2024-03-15T12:51:41.003 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:41.016+0000 7ff7dba12700 1 -- 172.21.15.184:0/2120041454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d41018e0 msgr2=0x7ff7d41974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:51:41.003 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:41.016+0000 7ff7dba12700 1 --2- 172.21.15.184:0/2120041454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d41018e0 0x7ff7d41974d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400b560 tx=0x7ff7c400b870 comp rx=0 tx=0).stop 2024-03-15T12:51:41.003 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:41.016+0000 7ff7dba12700 1 -- 172.21.15.184:0/2120041454 shutdown_connections 2024-03-15T12:51:41.003 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:41.016+0000 7ff7dba12700 1 --2- 172.21.15.184:0/2120041454 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff7c00a1a30 0x7ff7c00a3ef0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:41.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:41.016+0000 7ff7dba12700 1 --2- 172.21.15.184:0/2120041454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d41006e0 0x7ff7d4196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:41.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:41.016+0000 7ff7dba12700 1 --2- 172.21.15.184:0/2120041454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7d4102280 0x7ff7d419e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:41.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:41.016+0000 7ff7dba12700 1 --2- 172.21.15.184:0/2120041454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7d41018e0 0x7ff7d41974d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:41.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:41.016+0000 7ff7dba12700 1 -- 172.21.15.184:0/2120041454 >> 172.21.15.184:0/2120041454 conn(0x7ff7d40fbc80 msgr2=0x7ff7d4104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:51:41.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:41.016+0000 7ff7dba12700 1 -- 172.21.15.184:0/2120041454 shutdown_connections 2024-03-15T12:51:41.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:51:41.016+0000 7ff7dba12700 1 -- 172.21.15.184:0/2120041454 wait complete. 2024-03-15T12:51:41.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:41.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:41.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:41.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:51:41.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:41.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:41 smithi002 ceph-mon[32060]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:41.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:41.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:41.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:51:41.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:41.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:41.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:41.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:51:41.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:41.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:41 smithi184 ceph-mon[35993]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:41.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:41.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:41.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:51:41.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:41.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:41.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:41.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:51:41.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:41.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:41 smithi149 ceph-mon[35927]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:41.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:41.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:41.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:51:41.659 DEBUG:teuthology.orchestra.run.smithi184:osd.2> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.2.service 2024-03-15T12:51:41.662 INFO:tasks.cephadm:Deploying osd.3 on smithi002 with /dev/vg_nvme/lv_3... 2024-03-15T12:51:41.663 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-15T12:51:41.914 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:51:42.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:42 smithi002 ceph-mon[32060]: from='osd.2 [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-15T12:51:42.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:42 smithi002 ceph-mon[32060]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-15T12:51:42.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:42 smithi184 ceph-mon[35993]: from='osd.2 [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-15T12:51:42.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:42 smithi184 ceph-mon[35993]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-15T12:51:42.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:42 smithi149 ceph-mon[35927]: from='osd.2 [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-15T12:51:42.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:42 smithi149 ceph-mon[35927]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-15T12:51:43.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:43 smithi002 ceph-mon[32060]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-15T12:51:43.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:43 smithi002 ceph-mon[32060]: osdmap e16: 3 total, 2 up, 3 in 2024-03-15T12:51:43.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:43 smithi002 ceph-mon[32060]: from='osd.2 [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:51:43.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:43 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:43.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:43 smithi002 ceph-mon[32060]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:51:43.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:43 smithi002 ceph-mon[32060]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:43.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:43 smithi149 ceph-mon[35927]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-15T12:51:43.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:43 smithi149 ceph-mon[35927]: osdmap e16: 3 total, 2 up, 3 in 2024-03-15T12:51:43.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:43 smithi149 ceph-mon[35927]: from='osd.2 [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:51:43.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:43 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:43.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:43 smithi149 ceph-mon[35927]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:51:43.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:43 smithi149 ceph-mon[35927]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:43.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:43 smithi184 ceph-mon[35993]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-15T12:51:43.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:43 smithi184 ceph-mon[35993]: osdmap e16: 3 total, 2 up, 3 in 2024-03-15T12:51:43.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:43 smithi184 ceph-mon[35993]: from='osd.2 [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:51:43.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:43 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:43.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:43 smithi184 ceph-mon[35993]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:51:43.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:43 smithi184 ceph-mon[35993]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:43.939 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-15T12:51:43.939 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-15T12:51:43.939 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2024-03-15T12:51:43.939 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2024-03-15T12:51:43.939 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00863966 s, 1.2 GB/s 2024-03-15T12:51:43.939 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2024-03-15T12:51:44.060 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 12:51:43 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-2[41183]: 2024-03-15T12:51:43.806+0000 7f1e0cf07700 -1 osd.2 0 waiting for initial osdmap 2024-03-15T12:51:44.061 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 12:51:43 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-2[41183]: 2024-03-15T12:51:43.811+0000 7f1e04cf2700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-15T12:51:44.231 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:44 smithi002 ceph-mon[32060]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-03-15T12:51:44.232 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:44 smithi002 ceph-mon[32060]: osdmap e17: 3 total, 2 up, 3 in 2024-03-15T12:51:44.232 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:44 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:44.232 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:44 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:44.232 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:44 smithi002 ceph-mon[32060]: from='osd.2 ' entity='osd.2' 2024-03-15T12:51:44.332 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:44 smithi184 ceph-mon[35993]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-03-15T12:51:44.332 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:44 smithi184 ceph-mon[35993]: osdmap e17: 3 total, 2 up, 3 in 2024-03-15T12:51:44.332 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:44 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:44.332 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:44 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:44.333 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:44 smithi184 ceph-mon[35993]: from='osd.2 ' entity='osd.2' 2024-03-15T12:51:44.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:44 smithi149 ceph-mon[35927]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-03-15T12:51:44.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:44 smithi149 ceph-mon[35927]: osdmap e17: 3 total, 2 up, 3 in 2024-03-15T12:51:44.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:44 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:44.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:44 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:44.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:44 smithi149 ceph-mon[35927]: from='osd.2 ' entity='osd.2' 2024-03-15T12:51:44.527 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi002:vg_nvme/lv_3 2024-03-15T12:51:44.781 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:51:45.185 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:45 smithi002 ceph-mon[32060]: purged_snaps scrub starts 2024-03-15T12:51:45.185 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:45 smithi002 ceph-mon[32060]: purged_snaps scrub ok 2024-03-15T12:51:45.185 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:45 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:45.185 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:45 smithi002 ceph-mon[32060]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:45.185 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:45 smithi002 ceph-mon[32060]: Detected new or changed devices on smithi184 2024-03-15T12:51:45.185 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:45 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:45.185 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:45 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:45.185 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:45 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:51:45.185 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:45 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:45.185 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:45 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:45.185 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:45 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:51:45.186 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:45 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:45.186 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:45 smithi002 ceph-mon[32060]: osd.2 [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344] boot 2024-03-15T12:51:45.186 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:45 smithi002 ceph-mon[32060]: osdmap e18: 3 total, 3 up, 3 in 2024-03-15T12:51:45.186 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:45 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:45.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:45 smithi149 ceph-mon[35927]: purged_snaps scrub starts 2024-03-15T12:51:45.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:45 smithi149 ceph-mon[35927]: purged_snaps scrub ok 2024-03-15T12:51:45.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:45 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:45.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:45 smithi149 ceph-mon[35927]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:45.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:45 smithi149 ceph-mon[35927]: Detected new or changed devices on smithi184 2024-03-15T12:51:45.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:45 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:45.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:45 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:45.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:45 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:51:45.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:45 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:45.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:45 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:45.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:45 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:51:45.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:45 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:45.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:45 smithi149 ceph-mon[35927]: osd.2 [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344] boot 2024-03-15T12:51:45.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:45 smithi149 ceph-mon[35927]: osdmap e18: 3 total, 3 up, 3 in 2024-03-15T12:51:45.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:45 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:45.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:45 smithi184 ceph-mon[35993]: purged_snaps scrub starts 2024-03-15T12:51:45.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:45 smithi184 ceph-mon[35993]: purged_snaps scrub ok 2024-03-15T12:51:45.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:45 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:45.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:45 smithi184 ceph-mon[35993]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-03-15T12:51:45.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:45 smithi184 ceph-mon[35993]: Detected new or changed devices on smithi184 2024-03-15T12:51:45.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:45 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:45.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:45 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:45.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:45 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:51:45.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:45 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:45.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:45 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:45.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:45 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:51:45.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:45 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:45.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:45 smithi184 ceph-mon[35993]: osd.2 [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344] boot 2024-03-15T12:51:45.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:45 smithi184 ceph-mon[35993]: osdmap e18: 3 total, 3 up, 3 in 2024-03-15T12:51:45.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:45 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:51:46.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.351+0000 7f17b952d700 1 -- 172.21.15.2:0/1670629548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17b4102010 msgr2=0x7f17b4102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:51:46.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.351+0000 7f17b952d700 1 --2- 172.21.15.2:0/1670629548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17b4102010 0x7f17b4102430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f17a400b0d0 tx=0x7f17a400b3e0 comp rx=0 tx=0).stop 2024-03-15T12:51:46.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.351+0000 7f17b952d700 1 -- 172.21.15.2:0/1670629548 shutdown_connections 2024-03-15T12:51:46.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.351+0000 7f17b952d700 1 --2- 172.21.15.2:0/1670629548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17b4102010 0x7f17b4102430 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:46.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.351+0000 7f17b952d700 1 --2- 172.21.15.2:0/1670629548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17b4103bb0 0x7f17b410a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:46.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.351+0000 7f17b952d700 1 --2- 172.21.15.2:0/1670629548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17b4103210 0x7f17b4103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:46.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.351+0000 7f17b952d700 1 -- 172.21.15.2:0/1670629548 >> 172.21.15.2:0/1670629548 conn(0x7f17b40fd5f0 msgr2=0x7f17b40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:51:46.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.351+0000 7f17b952d700 1 -- 172.21.15.2:0/1670629548 shutdown_connections 2024-03-15T12:51:46.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.351+0000 7f17b952d700 1 -- 172.21.15.2:0/1670629548 wait complete. 2024-03-15T12:51:46.340 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:46 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi184 to 17088M 2024-03-15T12:51:46.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.351+0000 7f17b952d700 1 Processor -- start 2024-03-15T12:51:46.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b952d700 1 -- start start 2024-03-15T12:51:46.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b952d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17b4102010 0x7f17b419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:51:46.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b952d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17b4103210 0x7f17b419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:51:46.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b952d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17b4103bb0 0x7f17b41a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:51:46.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b952d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b41a1fb0 con 0x7f17b4103210 2024-03-15T12:51:46.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b952d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b41a20f0 con 0x7f17b4103bb0 2024-03-15T12:51:46.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b952d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b41a2280 con 0x7f17b4102010 2024-03-15T12:51:46.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b27fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17b4103210 0x7f17b419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:51:46.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b27fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17b4103210 0x7f17b419a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50190/0 (socket says 172.21.15.2:50190) 2024-03-15T12:51:46.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b27fc700 1 -- 172.21.15.2:0/706585486 learned_addr learned my addr 172.21.15.2:0/706585486 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:51:46.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b2ffd700 1 --2- 172.21.15.2:0/706585486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17b4102010 0x7f17b419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:51:46.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b37fe700 1 --2- 172.21.15.2:0/706585486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17b4103bb0 0x7f17b41a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:51:46.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b27fc700 1 -- 172.21.15.2:0/706585486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17b4102010 msgr2=0x7f17b419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:51:46.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b27fc700 1 --2- 172.21.15.2:0/706585486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17b4102010 0x7f17b419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:46.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b27fc700 1 -- 172.21.15.2:0/706585486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17b4103bb0 msgr2=0x7f17b41a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:51:46.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b27fc700 1 --2- 172.21.15.2:0/706585486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17b4103bb0 0x7f17b41a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:51:46.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.352+0000 7f17b27fc700 1 -- 172.21.15.2:0/706585486 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a4009d00 con 0x7f17b4103210 2024-03-15T12:51:46.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.353+0000 7f17b27fc700 1 --2- 172.21.15.2:0/706585486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17b4103210 0x7f17b419a960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f179c00b560 tx=0x7f179c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:51:46.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.353+0000 7f17abfff700 1 -- 172.21.15.2:0/706585486 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c011830 con 0x7f17b4103210 2024-03-15T12:51:46.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.353+0000 7f17abfff700 1 -- 172.21.15.2:0/706585486 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f179c011e60 con 0x7f17b4103210 2024-03-15T12:51:46.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.353+0000 7f17b952d700 1 -- 172.21.15.2:0/706585486 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b41a2560 con 0x7f17b4103210 2024-03-15T12:51:46.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.353+0000 7f17abfff700 1 -- 172.21.15.2:0/706585486 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c00f2e0 con 0x7f17b4103210 2024-03-15T12:51:46.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.353+0000 7f17b952d700 1 -- 172.21.15.2:0/706585486 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b41a2a60 con 0x7f17b4103210 2024-03-15T12:51:46.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.354+0000 7f17b952d700 1 -- 172.21.15.2:0/706585486 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b4068ca0 con 0x7f17b4103210 2024-03-15T12:51:46.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.355+0000 7f17abfff700 1 -- 172.21.15.2:0/706585486 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 15) v1 ==== 136222+0+0 (secure 0 0 0) 0x7f179c011990 con 0x7f17b4103210 2024-03-15T12:51:46.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.355+0000 7f17abfff700 1 --2- 172.21.15.2:0/706585486 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f17a00a1ab0 0x7f17a00a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:51:46.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.356+0000 7f17b2ffd700 1 --2- 172.21.15.2:0/706585486 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f17a00a1ab0 0x7f17a00a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:51:46.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.356+0000 7f17abfff700 1 -- 172.21.15.2:0/706585486 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f179c0cb7e0 con 0x7f17b4103210 2024-03-15T12:51:46.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.356+0000 7f17b2ffd700 1 --2- 172.21.15.2:0/706585486 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f17a00a1ab0 0x7f17a00a3f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009fd0 tx=0x7f17a400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:51:46.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.360+0000 7f17abfff700 1 -- 172.21.15.2:0/706585486 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f179c095f80 con 0x7f17b4103210 2024-03-15T12:51:46.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:46 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi184 to 17088M 2024-03-15T12:51:46.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:46 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi184 to 17088M 2024-03-15T12:51:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:46.577+0000 7f17b952d700 1 -- 172.21.15.2:0/706585486 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f17b4075930 con 0x7f17a00a1ab0 2024-03-15T12:51:47.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:47 smithi149 ceph-mon[35927]: osdmap e19: 3 total, 3 up, 3 in 2024-03-15T12:51:47.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:47 smithi149 ceph-mon[35927]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:51:47.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:47 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:51:47.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:47 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:51:47.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:47 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:47.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:47 smithi149 ceph-mon[35927]: pgmap v83: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:51:47.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:47 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-15T12:51:47.362 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:47 smithi002 ceph-mon[32060]: osdmap e19: 3 total, 3 up, 3 in 2024-03-15T12:51:47.362 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:47 smithi002 ceph-mon[32060]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:51:47.362 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:47 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:51:47.362 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:47 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:51:47.362 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:47 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:47.362 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:47 smithi002 ceph-mon[32060]: pgmap v83: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:51:47.362 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:47 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-15T12:51:47.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:47 smithi184 ceph-mon[35993]: osdmap e19: 3 total, 3 up, 3 in 2024-03-15T12:51:47.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:47 smithi184 ceph-mon[35993]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:51:47.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:47 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:51:47.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:47 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:51:47.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:47 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:47.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:47 smithi184 ceph-mon[35993]: pgmap v83: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:51:47.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:47 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-15T12:51:48.216 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:48 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-15T12:51:48.216 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:48 smithi002 ceph-mon[32060]: osdmap e20: 3 total, 3 up, 3 in 2024-03-15T12:51:48.216 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:48 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-15T12:51:48.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:48 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-15T12:51:48.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:48 smithi149 ceph-mon[35927]: osdmap e20: 3 total, 3 up, 3 in 2024-03-15T12:51:48.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:48 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-15T12:51:48.554 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 12:51:48 smithi002 sudo[51606]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-15T12:51:48.554 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 12:51:48 smithi002 sudo[51606]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-15T12:51:48.554 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 12:51:48 smithi002 sudo[51606]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-15T12:51:48.554 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 12:51:48 smithi002 sudo[51606]: pam_unix(sudo:session): session closed for user root 2024-03-15T12:51:48.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:48 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-15T12:51:48.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:48 smithi184 ceph-mon[35993]: osdmap e20: 3 total, 3 up, 3 in 2024-03-15T12:51:48.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:48 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-15T12:51:49.054 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 12:51:48 smithi002 sudo[51631]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-15T12:51:49.054 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 12:51:48 smithi002 sudo[51631]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-15T12:51:49.054 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 12:51:48 smithi002 sudo[51631]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-15T12:51:49.054 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 12:51:48 smithi002 sudo[51631]: pam_unix(sudo:session): session closed for user root 2024-03-15T12:51:49.251 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 12:51:49 smithi149 sudo[44217]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-15T12:51:49.251 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 12:51:49 smithi149 sudo[44217]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-15T12:51:49.251 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 12:51:49 smithi149 sudo[44217]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-15T12:51:49.251 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 12:51:49 smithi149 sudo[44217]: pam_unix(sudo:session): session closed for user root 2024-03-15T12:51:49.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-15T12:51:49.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:49 smithi149 ceph-mon[35927]: osdmap e21: 3 total, 3 up, 3 in 2024-03-15T12:51:49.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:49 smithi149 ceph-mon[35927]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:51:49.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:49.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:49 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1828032273' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2438d20-597b-4510-8cb7-3f0ce89067a6"}]: dispatch 2024-03-15T12:51:49.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:49 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1828032273' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2438d20-597b-4510-8cb7-3f0ce89067a6"}]': finished 2024-03-15T12:51:49.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:49 smithi149 ceph-mon[35927]: osdmap e22: 4 total, 3 up, 4 in 2024-03-15T12:51:49.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:51:49.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-15T12:51:49.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:49 smithi002 ceph-mon[32060]: osdmap e21: 3 total, 3 up, 3 in 2024-03-15T12:51:49.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:49 smithi002 ceph-mon[32060]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:51:49.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:49.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:49 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1828032273' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2438d20-597b-4510-8cb7-3f0ce89067a6"}]: dispatch 2024-03-15T12:51:49.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:49 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1828032273' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2438d20-597b-4510-8cb7-3f0ce89067a6"}]': finished 2024-03-15T12:51:49.403 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:49 smithi002 ceph-mon[32060]: osdmap e22: 4 total, 3 up, 4 in 2024-03-15T12:51:49.403 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:51:49.515 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-15T12:51:49.515 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:49 smithi184 ceph-mon[35993]: osdmap e21: 3 total, 3 up, 3 in 2024-03-15T12:51:49.515 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:49 smithi184 ceph-mon[35993]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:51:49.515 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:49.515 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:49 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1828032273' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b2438d20-597b-4510-8cb7-3f0ce89067a6"}]: dispatch 2024-03-15T12:51:49.516 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:49 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1828032273' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2438d20-597b-4510-8cb7-3f0ce89067a6"}]': finished 2024-03-15T12:51:49.516 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:49 smithi184 ceph-mon[35993]: osdmap e22: 4 total, 3 up, 4 in 2024-03-15T12:51:49.516 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:51:49.611 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 12:51:49 smithi149 sudo[44220]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-15T12:51:49.611 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 12:51:49 smithi149 sudo[44220]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-15T12:51:49.611 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 12:51:49 smithi149 sudo[44220]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-15T12:51:49.611 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 12:51:49 smithi149 sudo[44220]: pam_unix(sudo:session): session closed for user root 2024-03-15T12:51:49.769 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 12:51:49 smithi184 sudo[44787]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-15T12:51:49.769 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 12:51:49 smithi184 sudo[44787]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-15T12:51:49.769 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 12:51:49 smithi184 sudo[44787]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-15T12:51:49.769 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 12:51:49 smithi184 sudo[44787]: pam_unix(sudo:session): session closed for user root 2024-03-15T12:51:50.060 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 12:51:49 smithi184 sudo[44790]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-15T12:51:50.060 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 12:51:49 smithi184 sudo[44790]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-15T12:51:50.060 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 12:51:49 smithi184 sudo[44790]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-15T12:51:50.060 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 12:51:49 smithi184 sudo[44790]: pam_unix(sudo:session): session closed for user root 2024-03-15T12:51:50.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:50 smithi002 sudo[51737]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-15T12:51:50.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:50 smithi002 sudo[51737]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-15T12:51:50.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:50 smithi002 sudo[51737]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-15T12:51:50.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:50 smithi002 sudo[51737]: pam_unix(sudo:session): session closed for user root 2024-03-15T12:51:50.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:51:50.345+0000 7f17abfff700 1 -- 172.21.15.2:0/706585486 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f179c096350 con 0x7f17b4103210 2024-03-15T12:51:50.578 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:50 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:50.578 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:50 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/775050593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:51:50.578 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:50 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:50.578 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:50 smithi184 ceph-mon[35993]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-15T12:51:50.578 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:50 smithi184 sudo[44793]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-15T12:51:50.578 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:50 smithi184 sudo[44793]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-15T12:51:50.578 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:50 smithi184 sudo[44793]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-15T12:51:50.579 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:50 smithi184 sudo[44793]: pam_unix(sudo:session): session closed for user root 2024-03-15T12:51:50.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:50 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:50.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:50 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/775050593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:51:50.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:50 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:50.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:50 smithi149 ceph-mon[35927]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-15T12:51:50.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:50 smithi002 sudo[51924]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-15T12:51:50.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:50 smithi002 sudo[51924]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-15T12:51:50.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:50 smithi002 sudo[51924]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-15T12:51:50.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:50 smithi002 sudo[51924]: pam_unix(sudo:session): session closed for user root 2024-03-15T12:51:50.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:50 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:50.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:50 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/775050593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:51:50.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:50 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:51:50.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:50 smithi002 ceph-mon[32060]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-15T12:51:51.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:50 smithi184 sudo[44796]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-15T12:51:51.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:50 smithi184 sudo[44796]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-15T12:51:51.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:50 smithi184 sudo[44796]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-15T12:51:51.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:50 smithi184 sudo[44796]: pam_unix(sudo:session): session closed for user root 2024-03-15T12:51:51.313 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 sudo[44223]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-15T12:51:51.313 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 sudo[44223]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-15T12:51:51.313 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 sudo[44223]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-15T12:51:51.313 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 sudo[44223]: pam_unix(sudo:session): session closed for user root 2024-03-15T12:51:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 sudo[44226]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-15T12:51:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 sudo[44226]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-15T12:51:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 sudo[44226]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-15T12:51:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 sudo[44226]: pam_unix(sudo:session): session closed for user root 2024-03-15T12:51:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-15T12:51:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:51:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:51:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:51:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-15T12:51:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-15T12:51:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:51:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:51:51.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:51:51.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:51:51.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-15T12:51:51.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-15T12:51:51.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:51:51.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:51:51.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:51:51.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:51 smithi149 ceph-mon[35927]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-15T12:51:51.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-15T12:51:51.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:51:51.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:51:51.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:51:51.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-15T12:51:51.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-15T12:51:51.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:51:51.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:51:51.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:51:51.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:51:51.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-15T12:51:51.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-15T12:51:51.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:51:51.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:51:51.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:51:51.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:51 smithi002 ceph-mon[32060]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-15T12:51:51.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-15T12:51:51.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:51:51.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:51:51.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:51:51.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-15T12:51:51.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-15T12:51:51.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:51:51.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:51:51.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:51:51.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:51:51.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-15T12:51:51.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-15T12:51:51.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:51:51.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:51:51.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:51:51.811 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:51 smithi184 ceph-mon[35993]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-15T12:51:53.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:52 smithi002 ceph-mon[32060]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:51:53.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:52 smithi184 ceph-mon[35993]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:51:53.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:52 smithi149 ceph-mon[35927]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:51:55.049 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:54 smithi002 ceph-mon[32060]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:51:55.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:54 smithi184 ceph-mon[35993]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:51:55.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:54 smithi149 ceph-mon[35927]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:51:56.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:56 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-15T12:51:56.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:56 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:56.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:56 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-15T12:51:56.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:56 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:56.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:56 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-15T12:51:56.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:56 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:51:57.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:57 smithi002 ceph-mon[32060]: Deploying daemon osd.3 on smithi002 2024-03-15T12:51:57.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:57 smithi002 ceph-mon[32060]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-03-15T12:51:57.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:57 smithi184 ceph-mon[35993]: Deploying daemon osd.3 on smithi002 2024-03-15T12:51:57.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:57 smithi184 ceph-mon[35993]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-03-15T12:51:57.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:57 smithi149 ceph-mon[35927]: Deploying daemon osd.3 on smithi002 2024-03-15T12:51:57.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:57 smithi149 ceph-mon[35927]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-03-15T12:51:59.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:51:58 smithi002 ceph-mon[32060]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-15T12:51:59.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:51:58 smithi184 ceph-mon[35993]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-15T12:51:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:51:58 smithi149 ceph-mon[35927]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-03-15T12:52:00.934 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:00 smithi002 ceph-mon[32060]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-15T12:52:01.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:00 smithi184 ceph-mon[35993]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-15T12:52:01.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:00 smithi149 ceph-mon[35927]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-03-15T12:52:03.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:02 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:03.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:02 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:03.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:02 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:03.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:02 smithi002 ceph-mon[32060]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-15T12:52:03.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:02 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:03.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:02 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:03.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:02 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:03.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:02 smithi184 ceph-mon[35993]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-15T12:52:03.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:02 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:03.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:02 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:03.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:02 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:03.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:02 smithi149 ceph-mon[35927]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-15T12:52:05.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:52:05.584+0000 7f17abfff700 1 -- 172.21.15.2:0/706585486 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f17b4075930 con 0x7f17a00a1ab0 2024-03-15T12:52:05.571 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 3 on host 'smithi002' 2024-03-15T12:52:05.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:52:05.587+0000 7f17b952d700 1 -- 172.21.15.2:0/706585486 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f17a00a1ab0 msgr2=0x7f17a00a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:52:05.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:52:05.587+0000 7f17b952d700 1 --2- 172.21.15.2:0/706585486 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f17a00a1ab0 0x7f17a00a3f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009fd0 tx=0x7f17a400b040 comp rx=0 tx=0).stop 2024-03-15T12:52:05.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:52:05.587+0000 7f17b952d700 1 -- 172.21.15.2:0/706585486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17b4103210 msgr2=0x7f17b419a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:52:05.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:52:05.587+0000 7f17b952d700 1 --2- 172.21.15.2:0/706585486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17b4103210 0x7f17b419a960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f179c00b560 tx=0x7f179c00b870 comp rx=0 tx=0).stop 2024-03-15T12:52:05.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:52:05.587+0000 7f17b952d700 1 -- 172.21.15.2:0/706585486 shutdown_connections 2024-03-15T12:52:05.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:52:05.587+0000 7f17b952d700 1 --2- 172.21.15.2:0/706585486 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f17a00a1ab0 0x7f17a00a3f70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:05.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:52:05.587+0000 7f17b952d700 1 --2- 172.21.15.2:0/706585486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17b4103bb0 0x7f17b41a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:05.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:52:05.587+0000 7f17b952d700 1 --2- 172.21.15.2:0/706585486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17b4102010 0x7f17b419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:05.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:52:05.587+0000 7f17b952d700 1 --2- 172.21.15.2:0/706585486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17b4103210 0x7f17b419a960 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:05.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:52:05.587+0000 7f17b952d700 1 -- 172.21.15.2:0/706585486 >> 172.21.15.2:0/706585486 conn(0x7f17b40fd5f0 msgr2=0x7f17b4074210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:52:05.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:52:05.587+0000 7f17b952d700 1 -- 172.21.15.2:0/706585486 shutdown_connections 2024-03-15T12:52:05.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:52:05.587+0000 7f17b952d700 1 -- 172.21.15.2:0/706585486 wait complete. 2024-03-15T12:52:05.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:05 smithi002 ceph-mon[32060]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-15T12:52:05.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:05.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:05.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:05.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:05.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:05.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:05.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:05.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:06.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:05 smithi184 ceph-mon[35993]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-15T12:52:06.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:06.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:06.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:06.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:06.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:06.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:06.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:06.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:06.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:05 smithi149 ceph-mon[35927]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-15T12:52:06.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:06.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:06.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:06.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:06.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:06.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:06.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:06.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:06.228 DEBUG:teuthology.orchestra.run.smithi002:osd.3> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.3.service 2024-03-15T12:52:06.230 INFO:tasks.cephadm:Deploying osd.4 on smithi149 with /dev/vg_nvme/lv_3... 2024-03-15T12:52:06.230 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-15T12:52:06.480 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.b/config 2024-03-15T12:52:06.733 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:06 smithi002 ceph-mon[32060]: from='osd.3 [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-15T12:52:06.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:06 smithi149 ceph-mon[35927]: from='osd.3 [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-15T12:52:07.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:06 smithi184 ceph-mon[35993]: from='osd.3 [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-15T12:52:07.797 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:07 smithi002 ceph-mon[32060]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-15T12:52:07.797 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:07 smithi002 ceph-mon[32060]: from='osd.3 [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-15T12:52:07.797 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:07 smithi002 ceph-mon[32060]: osdmap e23: 4 total, 3 up, 4 in 2024-03-15T12:52:07.797 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:07 smithi002 ceph-mon[32060]: from='osd.3 [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-15T12:52:07.797 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:07 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:52:07.798 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 12:52:07 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-3[53699]: 2024-03-15T12:52:07.705+0000 7fbb2ea24700 -1 osd.3 0 waiting for initial osdmap 2024-03-15T12:52:07.798 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 12:52:07 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-3[53699]: 2024-03-15T12:52:07.711+0000 7fbb2a817700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-15T12:52:07.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:07 smithi149 ceph-mon[35927]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-15T12:52:07.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:07 smithi149 ceph-mon[35927]: from='osd.3 [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-15T12:52:07.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:07 smithi149 ceph-mon[35927]: osdmap e23: 4 total, 3 up, 4 in 2024-03-15T12:52:07.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:07 smithi149 ceph-mon[35927]: from='osd.3 [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-15T12:52:07.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:07 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:52:08.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:07 smithi184 ceph-mon[35993]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-15T12:52:08.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:07 smithi184 ceph-mon[35993]: from='osd.3 [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-15T12:52:08.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:07 smithi184 ceph-mon[35993]: osdmap e23: 4 total, 3 up, 4 in 2024-03-15T12:52:08.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:07 smithi184 ceph-mon[35993]: from='osd.3 [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-15T12:52:08.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:07 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:52:08.407 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-15T12:52:08.407 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-15T12:52:08.407 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-15T12:52:08.407 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-15T12:52:08.408 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112628 s, 931 MB/s 2024-03-15T12:52:08.408 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-15T12:52:08.950 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:08 smithi002 ceph-mon[32060]: from='osd.3 [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-15T12:52:08.950 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:08 smithi002 ceph-mon[32060]: osdmap e24: 4 total, 3 up, 4 in 2024-03-15T12:52:08.951 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:52:08.951 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:52:08.951 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:08 smithi002 ceph-mon[32060]: osd.3 [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302] boot 2024-03-15T12:52:08.951 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:08 smithi002 ceph-mon[32060]: osdmap e25: 4 total, 4 up, 4 in 2024-03-15T12:52:08.951 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:52:09.024 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:08 smithi149 ceph-mon[35927]: from='osd.3 [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-15T12:52:09.024 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:08 smithi149 ceph-mon[35927]: osdmap e24: 4 total, 3 up, 4 in 2024-03-15T12:52:09.024 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:08 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:52:09.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:08 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:52:09.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:08 smithi149 ceph-mon[35927]: osd.3 [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302] boot 2024-03-15T12:52:09.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:08 smithi149 ceph-mon[35927]: osdmap e25: 4 total, 4 up, 4 in 2024-03-15T12:52:09.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:08 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:52:09.050 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_3 2024-03-15T12:52:09.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:08 smithi184 ceph-mon[35993]: from='osd.3 [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-15T12:52:09.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:08 smithi184 ceph-mon[35993]: osdmap e24: 4 total, 3 up, 4 in 2024-03-15T12:52:09.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:08 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:52:09.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:08 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:52:09.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:08 smithi184 ceph-mon[35993]: osd.3 [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302] boot 2024-03-15T12:52:09.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:08 smithi184 ceph-mon[35993]: osdmap e25: 4 total, 4 up, 4 in 2024-03-15T12:52:09.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:08 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:52:09.290 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.b/config 2024-03-15T12:52:10.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:09 smithi149 ceph-mon[35927]: purged_snaps scrub starts 2024-03-15T12:52:10.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:09 smithi149 ceph-mon[35927]: purged_snaps scrub ok 2024-03-15T12:52:10.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:09 smithi149 ceph-mon[35927]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:52:10.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:09 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:10.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:09 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:10.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:09 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:10.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:09 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:10.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:09 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:10.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:09 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:10.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:09 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:10.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:09 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:09 smithi002 ceph-mon[32060]: purged_snaps scrub starts 2024-03-15T12:52:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:09 smithi002 ceph-mon[32060]: purged_snaps scrub ok 2024-03-15T12:52:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:09 smithi002 ceph-mon[32060]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:52:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:10.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:10.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:09 smithi184 ceph-mon[35993]: purged_snaps scrub starts 2024-03-15T12:52:10.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:09 smithi184 ceph-mon[35993]: purged_snaps scrub ok 2024-03-15T12:52:10.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:09 smithi184 ceph-mon[35993]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 268 GiB / 268 GiB avail 2024-03-15T12:52:10.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:10.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:10.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:10.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:10.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:10.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:10.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:10.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:10.781 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.793+0000 7f50babf2700 1 -- 172.21.15.149:0/1705948491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b4103210 msgr2=0x7f50b4103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:52:10.781 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.793+0000 7f50babf2700 1 --2- 172.21.15.149:0/1705948491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b4103210 0x7f50b4103670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f50a40099c0 tx=0x7f50a4009cd0 comp rx=0 tx=0).stop 2024-03-15T12:52:10.781 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.793+0000 7f50babf2700 1 -- 172.21.15.149:0/1705948491 shutdown_connections 2024-03-15T12:52:10.781 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.793+0000 7f50babf2700 1 --2- 172.21.15.149:0/1705948491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50b4102010 0x7f50b4102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:10.781 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.793+0000 7f50babf2700 1 --2- 172.21.15.149:0/1705948491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50b4103bb0 0x7f50b410a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:10.781 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.793+0000 7f50babf2700 1 --2- 172.21.15.149:0/1705948491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b4103210 0x7f50b4103670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:10.781 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.793+0000 7f50babf2700 1 -- 172.21.15.149:0/1705948491 >> 172.21.15.149:0/1705948491 conn(0x7f50b40fd5f0 msgr2=0x7f50b40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:52:10.781 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.793+0000 7f50babf2700 1 -- 172.21.15.149:0/1705948491 shutdown_connections 2024-03-15T12:52:10.781 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.793+0000 7f50babf2700 1 -- 172.21.15.149:0/1705948491 wait complete. 2024-03-15T12:52:10.787 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.793+0000 7f50babf2700 1 Processor -- start 2024-03-15T12:52:10.787 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50babf2700 1 -- start start 2024-03-15T12:52:10.787 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50babf2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b4102010 0x7f50b4075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:52:10.787 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50babf2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50b4103210 0x7f50b40743d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:52:10.787 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50babf2700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50b4103bb0 0x7f50b4074910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:52:10.787 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50babf2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4075090 con 0x7f50b4103210 2024-03-15T12:52:10.787 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50babf2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4075200 con 0x7f50b4102010 2024-03-15T12:52:10.787 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50babf2700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b41a1ff0 con 0x7f50b4103bb0 2024-03-15T12:52:10.788 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50b898e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b4102010 0x7f50b4075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:52:10.788 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50b898e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b4102010 0x7f50b4075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:57760/0 (socket says 172.21.15.149:57760) 2024-03-15T12:52:10.788 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50b898e700 1 -- 172.21.15.149:0/1458087927 learned_addr learned my addr 172.21.15.149:0/1458087927 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-15T12:52:10.788 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50abfff700 1 --2- 172.21.15.149:0/1458087927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50b4103210 0x7f50b40743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:52:10.788 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50b918f700 1 --2- 172.21.15.149:0/1458087927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50b4103bb0 0x7f50b4074910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:52:10.788 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50b898e700 1 -- 172.21.15.149:0/1458087927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50b4103bb0 msgr2=0x7f50b4074910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:52:10.788 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50b898e700 1 --2- 172.21.15.149:0/1458087927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50b4103bb0 0x7f50b4074910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:10.788 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50b898e700 1 -- 172.21.15.149:0/1458087927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50b4103210 msgr2=0x7f50b40743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:52:10.788 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50b898e700 1 --2- 172.21.15.149:0/1458087927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50b4103210 0x7f50b40743d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:10.788 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50b898e700 1 -- 172.21.15.149:0/1458087927 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a40095f0 con 0x7f50b4102010 2024-03-15T12:52:10.788 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50b898e700 1 --2- 172.21.15.149:0/1458087927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b4102010 0x7f50b4075d80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f50b000d980 tx=0x7f50b000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:52:10.788 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50a9ffb700 1 -- 172.21.15.149:0/1458087927 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0004020 con 0x7f50b4102010 2024-03-15T12:52:10.788 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50a9ffb700 1 -- 172.21.15.149:0/1458087927 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f50b000f690 con 0x7f50b4102010 2024-03-15T12:52:10.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.794+0000 7f50babf2700 1 -- 172.21.15.149:0/1458087927 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b41a22d0 con 0x7f50b4102010 2024-03-15T12:52:10.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.795+0000 7f50a9ffb700 1 -- 172.21.15.149:0/1458087927 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0003a80 con 0x7f50b4102010 2024-03-15T12:52:10.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.795+0000 7f50babf2700 1 -- 172.21.15.149:0/1458087927 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b41a27d0 con 0x7f50b4102010 2024-03-15T12:52:10.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.796+0000 7f50babf2700 1 -- 172.21.15.149:0/1458087927 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b41093d0 con 0x7f50b4102010 2024-03-15T12:52:10.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.797+0000 7f50a9ffb700 1 -- 172.21.15.149:0/1458087927 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f50b000f800 con 0x7f50b4102010 2024-03-15T12:52:10.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.797+0000 7f50a9ffb700 1 --2- 172.21.15.149:0/1458087927 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f509c0a1ac0 0x7f509c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:52:10.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.797+0000 7f50a9ffb700 1 -- 172.21.15.149:0/1458087927 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f50b00cbe10 con 0x7f50b4102010 2024-03-15T12:52:10.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.798+0000 7f50abfff700 1 --2- 172.21.15.149:0/1458087927 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f509c0a1ac0 0x7f509c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:52:10.789 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.798+0000 7f50abfff700 1 --2- 172.21.15.149:0/1458087927 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f509c0a1ac0 0x7f509c0a3f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009990 tx=0x7f50a40036c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:52:10.790 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:10.802+0000 7f50a9ffb700 1 -- 172.21.15.149:0/1458087927 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f50b0096220 con 0x7f50b4102010 2024-03-15T12:52:10.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:10 smithi149 ceph-mon[35927]: Detected new or changed devices on smithi002 2024-03-15T12:52:10.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:10 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi002 to 8546M 2024-03-15T12:52:10.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:10 smithi149 ceph-mon[35927]: osdmap e26: 4 total, 4 up, 4 in 2024-03-15T12:52:10.993 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:11.005+0000 7f50babf2700 1 -- 172.21.15.149:0/1458087927 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f50b402da30 con 0x7f509c0a1ac0 2024-03-15T12:52:11.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:10 smithi002 ceph-mon[32060]: Detected new or changed devices on smithi002 2024-03-15T12:52:11.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:10 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi002 to 8546M 2024-03-15T12:52:11.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:10 smithi002 ceph-mon[32060]: osdmap e26: 4 total, 4 up, 4 in 2024-03-15T12:52:11.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:10 smithi184 ceph-mon[35993]: Detected new or changed devices on smithi002 2024-03-15T12:52:11.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:10 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi002 to 8546M 2024-03-15T12:52:11.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:10 smithi184 ceph-mon[35993]: osdmap e26: 4 total, 4 up, 4 in 2024-03-15T12:52:11.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:11 smithi149 ceph-mon[35927]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:11.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:11 smithi149 ceph-mon[35927]: osdmap e27: 4 total, 4 up, 4 in 2024-03-15T12:52:11.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:11 smithi149 ceph-mon[35927]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:52:11.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:11 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:52:11.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:11 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:52:11.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:11 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:11.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:11 smithi002 ceph-mon[32060]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:11.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:11 smithi002 ceph-mon[32060]: osdmap e27: 4 total, 4 up, 4 in 2024-03-15T12:52:11.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:11 smithi002 ceph-mon[32060]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:52:11.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:11 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:52:11.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:11 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:52:11.956 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:11 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:12.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:11 smithi184 ceph-mon[35993]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:12.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:11 smithi184 ceph-mon[35993]: osdmap e27: 4 total, 4 up, 4 in 2024-03-15T12:52:12.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:11 smithi184 ceph-mon[35993]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:52:12.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:11 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:52:12.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:11 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:52:12.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:11 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:12.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:12 smithi149 ceph-mon[35927]: osdmap e28: 4 total, 4 up, 4 in 2024-03-15T12:52:12.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:12 smithi149 ceph-mon[35927]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-15T12:52:13.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:12 smithi002 ceph-mon[32060]: osdmap e28: 4 total, 4 up, 4 in 2024-03-15T12:52:13.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:12 smithi002 ceph-mon[32060]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-15T12:52:13.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:12 smithi184 ceph-mon[35993]: osdmap e28: 4 total, 4 up, 4 in 2024-03-15T12:52:13.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:12 smithi184 ceph-mon[35993]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-15T12:52:14.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:13 smithi149 ceph-mon[35927]: osdmap e29: 4 total, 4 up, 4 in 2024-03-15T12:52:14.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:13 smithi149 ceph-mon[35927]: from='client.? 172.21.15.149:0/3925415627' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65f6e3fc-1a8e-4180-9f3f-7a81b98de439"}]: dispatch 2024-03-15T12:52:14.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:13 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65f6e3fc-1a8e-4180-9f3f-7a81b98de439"}]: dispatch 2024-03-15T12:52:14.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:13 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65f6e3fc-1a8e-4180-9f3f-7a81b98de439"}]': finished 2024-03-15T12:52:14.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:13 smithi149 ceph-mon[35927]: osdmap e30: 5 total, 4 up, 5 in 2024-03-15T12:52:14.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:13 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:52:14.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:13 smithi002 ceph-mon[32060]: osdmap e29: 4 total, 4 up, 4 in 2024-03-15T12:52:14.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:13 smithi002 ceph-mon[32060]: from='client.? 172.21.15.149:0/3925415627' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65f6e3fc-1a8e-4180-9f3f-7a81b98de439"}]: dispatch 2024-03-15T12:52:14.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:13 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65f6e3fc-1a8e-4180-9f3f-7a81b98de439"}]: dispatch 2024-03-15T12:52:14.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:13 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65f6e3fc-1a8e-4180-9f3f-7a81b98de439"}]': finished 2024-03-15T12:52:14.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:13 smithi002 ceph-mon[32060]: osdmap e30: 5 total, 4 up, 5 in 2024-03-15T12:52:14.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:13 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:52:14.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:13 smithi184 ceph-mon[35993]: osdmap e29: 4 total, 4 up, 4 in 2024-03-15T12:52:14.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:13 smithi184 ceph-mon[35993]: from='client.? 172.21.15.149:0/3925415627' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65f6e3fc-1a8e-4180-9f3f-7a81b98de439"}]: dispatch 2024-03-15T12:52:14.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:13 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65f6e3fc-1a8e-4180-9f3f-7a81b98de439"}]: dispatch 2024-03-15T12:52:14.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:13 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65f6e3fc-1a8e-4180-9f3f-7a81b98de439"}]': finished 2024-03-15T12:52:14.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:13 smithi184 ceph-mon[35993]: osdmap e30: 5 total, 4 up, 5 in 2024-03-15T12:52:14.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:13 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:52:15.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:14 smithi002 ceph-mon[32060]: from='client.? 172.21.15.149:0/1125299027' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:52:15.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:14 smithi002 ceph-mon[32060]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-15T12:52:15.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:14 smithi184 ceph-mon[35993]: from='client.? 172.21.15.149:0/1125299027' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:52:15.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:14 smithi184 ceph-mon[35993]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-15T12:52:15.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:14 smithi149 ceph-mon[35927]: from='client.? 172.21.15.149:0/1125299027' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:52:15.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:14 smithi149 ceph-mon[35927]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-15T12:52:17.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:16 smithi002 ceph-mon[32060]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-15T12:52:17.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:16 smithi184 ceph-mon[35993]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-15T12:52:17.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:16 smithi149 ceph-mon[35927]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-15T12:52:19.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:18 smithi002 ceph-mon[32060]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:19.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:18 smithi184 ceph-mon[35993]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:19.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:18 smithi149 ceph-mon[35927]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:20.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:20 smithi149 ceph-mon[35927]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:21.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:20 smithi002 ceph-mon[32060]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:21.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:20 smithi184 ceph-mon[35993]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:22.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-15T12:52:22.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:22.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:21 smithi002 ceph-mon[32060]: Deploying daemon osd.4 on smithi149 2024-03-15T12:52:22.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:21 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-15T12:52:22.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:21 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:22.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:21 smithi184 ceph-mon[35993]: Deploying daemon osd.4 on smithi149 2024-03-15T12:52:22.075 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:21 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-15T12:52:22.075 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:21 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:22.075 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:21 smithi149 ceph-mon[35927]: Deploying daemon osd.4 on smithi149 2024-03-15T12:52:23.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:22 smithi002 ceph-mon[32060]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:23.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:22 smithi184 ceph-mon[35993]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:23.070 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:22 smithi149 ceph-mon[35927]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:25.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:24 smithi002 ceph-mon[32060]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:25.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:24 smithi184 ceph-mon[35993]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:25.079 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:24 smithi149 ceph-mon[35927]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:27.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:27.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:27.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:27.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:27 smithi002 ceph-mon[32060]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:27.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:27.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:27.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:27.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:27 smithi184 ceph-mon[35993]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:27.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:27.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:27.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:27.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:27 smithi149 ceph-mon[35927]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:29.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:28 smithi002 ceph-mon[32060]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:29.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:28 smithi184 ceph-mon[35993]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:29.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:28 smithi149 ceph-mon[35927]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:29.506 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 4 on host 'smithi149' 2024-03-15T12:52:29.507 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:29.519+0000 7f50a9ffb700 1 -- 172.21.15.149:0/1458087927 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f50b402da30 con 0x7f509c0a1ac0 2024-03-15T12:52:29.508 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:29.522+0000 7f50babf2700 1 -- 172.21.15.149:0/1458087927 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f509c0a1ac0 msgr2=0x7f509c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:52:29.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:29.522+0000 7f50babf2700 1 --2- 172.21.15.149:0/1458087927 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f509c0a1ac0 0x7f509c0a3f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009990 tx=0x7f50a40036c0 comp rx=0 tx=0).stop 2024-03-15T12:52:29.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:29.522+0000 7f50babf2700 1 -- 172.21.15.149:0/1458087927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b4102010 msgr2=0x7f50b4075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:52:29.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:29.522+0000 7f50babf2700 1 --2- 172.21.15.149:0/1458087927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b4102010 0x7f50b4075d80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f50b000d980 tx=0x7f50b000dc90 comp rx=0 tx=0).stop 2024-03-15T12:52:29.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:29.522+0000 7f50babf2700 1 -- 172.21.15.149:0/1458087927 shutdown_connections 2024-03-15T12:52:29.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:29.522+0000 7f50babf2700 1 --2- 172.21.15.149:0/1458087927 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f509c0a1ac0 0x7f509c0a3f80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:29.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:29.522+0000 7f50babf2700 1 --2- 172.21.15.149:0/1458087927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50b4103bb0 0x7f50b4074910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:29.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:29.522+0000 7f50babf2700 1 --2- 172.21.15.149:0/1458087927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50b4102010 0x7f50b4075d80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:29.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:29.522+0000 7f50babf2700 1 --2- 172.21.15.149:0/1458087927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50b4103210 0x7f50b40743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:29.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:29.522+0000 7f50babf2700 1 -- 172.21.15.149:0/1458087927 >> 172.21.15.149:0/1458087927 conn(0x7f50b40fd5f0 msgr2=0x7f50b4106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:52:29.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:29.522+0000 7f50babf2700 1 -- 172.21.15.149:0/1458087927 shutdown_connections 2024-03-15T12:52:29.509 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:52:29.522+0000 7f50babf2700 1 -- 172.21.15.149:0/1458087927 wait complete. 2024-03-15T12:52:30.149 DEBUG:teuthology.orchestra.run.smithi149:osd.4> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.4.service 2024-03-15T12:52:30.151 INFO:tasks.cephadm:Deploying osd.5 on smithi184 with /dev/vg_nvme/lv_3... 2024-03-15T12:52:30.151 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-15T12:52:30.387 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T12:52:30.756 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:30 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:30.756 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:30 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:30.757 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:30 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:30.757 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:30 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:30.757 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:30 smithi184 ceph-mon[35993]: from='osd.4 [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-15T12:52:30.757 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:30 smithi184 ceph-mon[35993]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-15T12:52:30.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:30.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:30.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:30.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:30.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:30 smithi002 ceph-mon[32060]: from='osd.4 [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-15T12:52:30.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:30 smithi002 ceph-mon[32060]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-15T12:52:30.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:30 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:30.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:30 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:30.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:30 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:30.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:30 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:30.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:30 smithi149 ceph-mon[35927]: from='osd.4 [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-15T12:52:30.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:30 smithi149 ceph-mon[35927]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-15T12:52:31.677 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:31 smithi184 ceph-mon[35993]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-15T12:52:31.677 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:31 smithi184 ceph-mon[35993]: osdmap e31: 5 total, 4 up, 5 in 2024-03-15T12:52:31.678 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:31 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:52:31.678 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:31 smithi184 ceph-mon[35993]: from='osd.4 [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:52:31.678 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:31 smithi184 ceph-mon[35993]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:52:31.678 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:31 smithi184 ceph-mon[35993]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:31.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:31 smithi002 ceph-mon[32060]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-15T12:52:31.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:31 smithi002 ceph-mon[32060]: osdmap e31: 5 total, 4 up, 5 in 2024-03-15T12:52:31.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:31 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:52:31.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:31 smithi002 ceph-mon[32060]: from='osd.4 [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:52:31.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:31 smithi002 ceph-mon[32060]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:52:31.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:31 smithi002 ceph-mon[32060]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:31.839 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:31 smithi149 ceph-mon[35927]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-15T12:52:31.840 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:31 smithi149 ceph-mon[35927]: osdmap e31: 5 total, 4 up, 5 in 2024-03-15T12:52:31.840 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:31 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:52:31.840 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:31 smithi149 ceph-mon[35927]: from='osd.4 [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:52:31.840 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:31 smithi149 ceph-mon[35927]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:52:31.840 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:31 smithi149 ceph-mon[35927]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:31.840 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 12:52:31 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-4[47022]: 2024-03-15T12:52:31.569+0000 7faf2d3f7700 -1 osd.4 0 waiting for initial osdmap 2024-03-15T12:52:31.840 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 12:52:31 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-4[47022]: 2024-03-15T12:52:31.575+0000 7faf291ea700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-15T12:52:32.567 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:32 smithi184 ceph-mon[35993]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-15T12:52:32.567 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:32 smithi184 ceph-mon[35993]: osdmap e32: 5 total, 4 up, 5 in 2024-03-15T12:52:32.567 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:32 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:52:32.567 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:32 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:52:32.567 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-15T12:52:32.567 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-15T12:52:32.567 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-03-15T12:52:32.567 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-03-15T12:52:32.567 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119313 s, 879 MB/s 2024-03-15T12:52:32.568 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-03-15T12:52:32.686 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:32 smithi149 ceph-mon[35927]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-15T12:52:32.686 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:32 smithi149 ceph-mon[35927]: osdmap e32: 5 total, 4 up, 5 in 2024-03-15T12:52:32.686 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:32 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:52:32.686 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:32 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:52:33.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:32 smithi002 ceph-mon[32060]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-15T12:52:33.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:32 smithi002 ceph-mon[32060]: osdmap e32: 5 total, 4 up, 5 in 2024-03-15T12:52:33.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:32 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:52:33.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:32 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:52:33.204 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi184:vg_nvme/lv_3 2024-03-15T12:52:33.443 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T12:52:33.816 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: purged_snaps scrub starts 2024-03-15T12:52:33.816 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: purged_snaps scrub ok 2024-03-15T12:52:33.816 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: osd.4 [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535] boot 2024-03-15T12:52:33.816 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: osdmap e33: 5 total, 5 up, 5 in 2024-03-15T12:52:33.816 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:52:33.816 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:33.816 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:33.816 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:33.816 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:33.817 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:33.817 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:33.817 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:33.817 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:33.817 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:33.817 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:33.817 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:33.817 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:33.817 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:33.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: purged_snaps scrub starts 2024-03-15T12:52:33.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: purged_snaps scrub ok 2024-03-15T12:52:33.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: osd.4 [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535] boot 2024-03-15T12:52:33.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: osdmap e33: 5 total, 5 up, 5 in 2024-03-15T12:52:33.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:52:33.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:33.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:33.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:33.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:33.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:33.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:33.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:33.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:33.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:33.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:33.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:33.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:33.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:34.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: purged_snaps scrub starts 2024-03-15T12:52:34.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: purged_snaps scrub ok 2024-03-15T12:52:34.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: osd.4 [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535] boot 2024-03-15T12:52:34.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: osdmap e33: 5 total, 5 up, 5 in 2024-03-15T12:52:34.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:52:34.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-03-15T12:52:34.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:34.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:34.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:34.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:34.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:34.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:34.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:34.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:34.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:34.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:34.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:34.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:34.848 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:34 smithi184 ceph-mon[35993]: Detected new or changed devices on smithi149 2024-03-15T12:52:34.848 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:34 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi149 to 2908M 2024-03-15T12:52:34.848 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:34 smithi184 ceph-mon[35993]: osdmap e34: 5 total, 5 up, 5 in 2024-03-15T12:52:34.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:34 smithi149 ceph-mon[35927]: Detected new or changed devices on smithi149 2024-03-15T12:52:34.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:34 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi149 to 2908M 2024-03-15T12:52:34.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:34 smithi149 ceph-mon[35927]: osdmap e34: 5 total, 5 up, 5 in 2024-03-15T12:52:35.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:34 smithi002 ceph-mon[32060]: Detected new or changed devices on smithi149 2024-03-15T12:52:35.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:34 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi149 to 2908M 2024-03-15T12:52:35.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:34 smithi002 ceph-mon[32060]: osdmap e34: 5 total, 5 up, 5 in 2024-03-15T12:52:35.080 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.093+0000 7f8e95543700 1 -- 172.21.15.184:0/2745339260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e90100f10 msgr2=0x7f8e90101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:52:35.081 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.093+0000 7f8e95543700 1 --2- 172.21.15.184:0/2745339260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e90100f10 0x7f8e90101330 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009990 tx=0x7f8e80009ca0 comp rx=0 tx=0).stop 2024-03-15T12:52:35.081 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.094+0000 7f8e95543700 1 -- 172.21.15.184:0/2745339260 shutdown_connections 2024-03-15T12:52:35.081 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.094+0000 7f8e95543700 1 --2- 172.21.15.184:0/2745339260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e90102a30 0x7f8e90109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:35.081 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.094+0000 7f8e95543700 1 --2- 172.21.15.184:0/2745339260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e90100f10 0x7f8e90101330 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:35.081 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.094+0000 7f8e95543700 1 --2- 172.21.15.184:0/2745339260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e90102070 0x7f8e901024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:35.081 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.094+0000 7f8e95543700 1 -- 172.21.15.184:0/2745339260 >> 172.21.15.184:0/2745339260 conn(0x7f8e900fc4b0 msgr2=0x7f8e900fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:52:35.081 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.094+0000 7f8e95543700 1 -- 172.21.15.184:0/2745339260 shutdown_connections 2024-03-15T12:52:35.081 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.094+0000 7f8e95543700 1 -- 172.21.15.184:0/2745339260 wait complete. 2024-03-15T12:52:35.083 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.094+0000 7f8e95543700 1 Processor -- start 2024-03-15T12:52:35.083 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e95543700 1 -- start start 2024-03-15T12:52:35.083 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e95543700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e90100f10 0x7f8e90197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:52:35.083 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e95543700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e90102070 0x7f8e90198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:52:35.083 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e95543700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e90102a30 0x7f8e9019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:52:35.083 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e95543700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9019fb70 con 0x7f8e90102070 2024-03-15T12:52:35.083 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e95543700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9019fcb0 con 0x7f8e90100f10 2024-03-15T12:52:35.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e95543700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9019fe40 con 0x7f8e90102a30 2024-03-15T12:52:35.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e8f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e90102a30 0x7f8e9019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:52:35.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e8f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e90102a30 0x7f8e9019f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:43294/0 (socket says 172.21.15.184:43294) 2024-03-15T12:52:35.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e8f7fe700 1 -- 172.21.15.184:0/1188293259 learned_addr learned my addr 172.21.15.184:0/1188293259 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-15T12:52:35.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e8e7fc700 1 --2- 172.21.15.184:0/1188293259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e90102070 0x7f8e90198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:52:35.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e8effd700 1 --2- 172.21.15.184:0/1188293259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e90100f10 0x7f8e90197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:52:35.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e8f7fe700 1 -- 172.21.15.184:0/1188293259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e90100f10 msgr2=0x7f8e90197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:52:35.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e8f7fe700 1 --2- 172.21.15.184:0/1188293259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e90100f10 0x7f8e90197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:35.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e8f7fe700 1 -- 172.21.15.184:0/1188293259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e90102070 msgr2=0x7f8e90198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:52:35.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e8f7fe700 1 --2- 172.21.15.184:0/1188293259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e90102070 0x7f8e90198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:35.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e8f7fe700 1 -- 172.21.15.184:0/1188293259 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e800095f0 con 0x7f8e90102a30 2024-03-15T12:52:35.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e8f7fe700 1 --2- 172.21.15.184:0/1188293259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e90102a30 0x7f8e9019f5a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800b9a0 tx=0x7f8e8800bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:52:35.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.095+0000 7f8e77fff700 1 -- 172.21.15.184:0/1188293259 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8800d850 con 0x7f8e90102a30 2024-03-15T12:52:35.084 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.096+0000 7f8e77fff700 1 -- 172.21.15.184:0/1188293259 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8e8800de80 con 0x7f8e90102a30 2024-03-15T12:52:35.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.096+0000 7f8e95543700 1 -- 172.21.15.184:0/1188293259 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e901a0120 con 0x7f8e90102a30 2024-03-15T12:52:35.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.096+0000 7f8e77fff700 1 -- 172.21.15.184:0/1188293259 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e88017400 con 0x7f8e90102a30 2024-03-15T12:52:35.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.096+0000 7f8e95543700 1 -- 172.21.15.184:0/1188293259 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e901a0650 con 0x7f8e90102a30 2024-03-15T12:52:35.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.097+0000 7f8e95543700 1 -- 172.21.15.184:0/1188293259 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e901082c0 con 0x7f8e90102a30 2024-03-15T12:52:35.085 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.098+0000 7f8e77fff700 1 -- 172.21.15.184:0/1188293259 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f8e8800e420 con 0x7f8e90102a30 2024-03-15T12:52:35.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.099+0000 7f8e77fff700 1 --2- 172.21.15.184:0/1188293259 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8e780a1ac0 0x7f8e780a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:52:35.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.099+0000 7f8e77fff700 1 -- 172.21.15.184:0/1188293259 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f8e88006e40 con 0x7f8e90102a30 2024-03-15T12:52:35.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.099+0000 7f8e8effd700 1 --2- 172.21.15.184:0/1188293259 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8e780a1ac0 0x7f8e780a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:52:35.086 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.100+0000 7f8e8effd700 1 --2- 172.21.15.184:0/1188293259 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8e780a1ac0 0x7f8e780a3f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8e80006010 tx=0x7f8e800038e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:52:35.089 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.103+0000 7f8e77fff700 1 -- 172.21.15.184:0/1188293259 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8e880fa280 con 0x7f8e90102a30 2024-03-15T12:52:35.292 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:35.305+0000 7f8e95543700 1 -- 172.21.15.184:0/1188293259 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e9002d9c0 con 0x7f8e780a1ac0 2024-03-15T12:52:35.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:35 smithi149 ceph-mon[35927]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:35.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:35 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:52:35.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:35 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:52:35.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:35 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:35.891 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:35 smithi184 ceph-mon[35993]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:35.891 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:35 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:52:35.891 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:35 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:52:35.891 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:35 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:36.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:35 smithi002 ceph-mon[32060]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:36.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:35 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:52:36.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:35 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:52:36.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:35 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:36.842 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:36 smithi184 ceph-mon[35993]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:52:36.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:36 smithi149 ceph-mon[35927]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:52:37.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:36 smithi002 ceph-mon[32060]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:52:37.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:37 smithi149 ceph-mon[35927]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:38.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:37 smithi002 ceph-mon[32060]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:38.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:37 smithi184 ceph-mon[35993]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:38.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:38 smithi149 ceph-mon[35927]: from='client.? 172.21.15.184:0/4200171737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e43c485c-25d6-41d5-8b9b-e22e4cc46727"}]: dispatch 2024-03-15T12:52:38.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:38 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e43c485c-25d6-41d5-8b9b-e22e4cc46727"}]: dispatch 2024-03-15T12:52:38.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:38 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e43c485c-25d6-41d5-8b9b-e22e4cc46727"}]': finished 2024-03-15T12:52:38.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:38 smithi149 ceph-mon[35927]: osdmap e35: 6 total, 5 up, 6 in 2024-03-15T12:52:38.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:38 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:39.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:38 smithi002 ceph-mon[32060]: from='client.? 172.21.15.184:0/4200171737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e43c485c-25d6-41d5-8b9b-e22e4cc46727"}]: dispatch 2024-03-15T12:52:39.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:38 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e43c485c-25d6-41d5-8b9b-e22e4cc46727"}]: dispatch 2024-03-15T12:52:39.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:38 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e43c485c-25d6-41d5-8b9b-e22e4cc46727"}]': finished 2024-03-15T12:52:39.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:38 smithi002 ceph-mon[32060]: osdmap e35: 6 total, 5 up, 6 in 2024-03-15T12:52:39.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:38 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:39.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:38 smithi184 ceph-mon[35993]: from='client.? 172.21.15.184:0/4200171737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e43c485c-25d6-41d5-8b9b-e22e4cc46727"}]: dispatch 2024-03-15T12:52:39.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:38 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e43c485c-25d6-41d5-8b9b-e22e4cc46727"}]: dispatch 2024-03-15T12:52:39.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:38 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e43c485c-25d6-41d5-8b9b-e22e4cc46727"}]': finished 2024-03-15T12:52:39.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:38 smithi184 ceph-mon[35993]: osdmap e35: 6 total, 5 up, 6 in 2024-03-15T12:52:39.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:38 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:39.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:39 smithi149 ceph-mon[35927]: from='client.? 172.21.15.184:0/548679760' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:52:39.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:39 smithi149 ceph-mon[35927]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:40.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:39 smithi002 ceph-mon[32060]: from='client.? 172.21.15.184:0/548679760' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:52:40.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:39 smithi002 ceph-mon[32060]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:40.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:39 smithi184 ceph-mon[35993]: from='client.? 172.21.15.184:0/548679760' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:52:40.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:39 smithi184 ceph-mon[35993]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:41.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:40 smithi002 ceph-mon[32060]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:41.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:40 smithi184 ceph-mon[35993]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:41.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:40 smithi149 ceph-mon[35927]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:42.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:42 smithi184 ceph-mon[35993]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:43.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:42 smithi002 ceph-mon[32060]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:43.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:42 smithi149 ceph-mon[35927]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:44.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:44 smithi184 ceph-mon[35993]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:45.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:44 smithi002 ceph-mon[32060]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:45.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:44 smithi149 ceph-mon[35927]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:46.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:45 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-15T12:52:46.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:45 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:46.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:45 smithi002 ceph-mon[32060]: Deploying daemon osd.5 on smithi184 2024-03-15T12:52:46.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:45 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-15T12:52:46.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:45 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:46.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:45 smithi184 ceph-mon[35993]: Deploying daemon osd.5 on smithi184 2024-03-15T12:52:46.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:45 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-15T12:52:46.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:45 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:46.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:45 smithi149 ceph-mon[35927]: Deploying daemon osd.5 on smithi184 2024-03-15T12:52:46.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:46 smithi184 ceph-mon[35993]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:47.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:46 smithi002 ceph-mon[32060]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:46 smithi149 ceph-mon[35927]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:48.922 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:48 smithi184 ceph-mon[35993]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:49.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:48 smithi002 ceph-mon[32060]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:49.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:48 smithi149 ceph-mon[35927]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:50.943 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:50 smithi184 ceph-mon[35993]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:51.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:50 smithi002 ceph-mon[32060]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:51.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:50 smithi149 ceph-mon[35927]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:52.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:52 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:52.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:52 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:52.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:52 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:52.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:52 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:52.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:52 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:52.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:52 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:52.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:52 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:52.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:52 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:52.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:52 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:53.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:53 smithi002 ceph-mon[32060]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:53.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:53 smithi184 ceph-mon[35993]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:53.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:53 smithi149 ceph-mon[35927]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:55.065 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:55.078+0000 7f8e77fff700 1 -- 172.21.15.184:0/1188293259 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8e9002d9c0 con 0x7f8e780a1ac0 2024-03-15T12:52:55.065 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 5 on host 'smithi184' 2024-03-15T12:52:55.067 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:55.080+0000 7f8e95543700 1 -- 172.21.15.184:0/1188293259 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8e780a1ac0 msgr2=0x7f8e780a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:52:55.067 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:55.080+0000 7f8e95543700 1 --2- 172.21.15.184:0/1188293259 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8e780a1ac0 0x7f8e780a3f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8e80006010 tx=0x7f8e800038e0 comp rx=0 tx=0).stop 2024-03-15T12:52:55.067 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:55.080+0000 7f8e95543700 1 -- 172.21.15.184:0/1188293259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e90102a30 msgr2=0x7f8e9019f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:52:55.067 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:55.080+0000 7f8e95543700 1 --2- 172.21.15.184:0/1188293259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e90102a30 0x7f8e9019f5a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800b9a0 tx=0x7f8e8800bcb0 comp rx=0 tx=0).stop 2024-03-15T12:52:55.067 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:55.080+0000 7f8e95543700 1 -- 172.21.15.184:0/1188293259 shutdown_connections 2024-03-15T12:52:55.067 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:55.080+0000 7f8e95543700 1 --2- 172.21.15.184:0/1188293259 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8e780a1ac0 0x7f8e780a3f80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:55.068 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:55.080+0000 7f8e95543700 1 --2- 172.21.15.184:0/1188293259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e90102a30 0x7f8e9019f5a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:55.068 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:55.080+0000 7f8e95543700 1 --2- 172.21.15.184:0/1188293259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e90100f10 0x7f8e90197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:55.068 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:55.080+0000 7f8e95543700 1 --2- 172.21.15.184:0/1188293259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e90102070 0x7f8e90198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:52:55.068 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:55.080+0000 7f8e95543700 1 -- 172.21.15.184:0/1188293259 >> 172.21.15.184:0/1188293259 conn(0x7f8e900fc4b0 msgr2=0x7f8e90105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:52:55.068 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:55.081+0000 7f8e95543700 1 -- 172.21.15.184:0/1188293259 shutdown_connections 2024-03-15T12:52:55.068 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:52:55.081+0000 7f8e95543700 1 -- 172.21.15.184:0/1188293259 wait complete. 2024-03-15T12:52:55.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:55.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:55.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:55.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:55.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:55.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:55 smithi184 ceph-mon[35993]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:55.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:55.311 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:55.311 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:55.311 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:55 smithi184 ceph-mon[35993]: from='osd.5 [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-15T12:52:55.311 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:55 smithi184 ceph-mon[35993]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-15T12:52:55.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:55.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:55.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:55.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:55.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:55.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:55 smithi002 ceph-mon[32060]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:55.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:55.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:55.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:55.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:55 smithi002 ceph-mon[32060]: from='osd.5 [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-15T12:52:55.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:55 smithi002 ceph-mon[32060]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-15T12:52:55.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:55.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:55.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:55.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:55.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:55.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:55 smithi149 ceph-mon[35927]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:55.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:55.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:55.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:52:55.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:55 smithi149 ceph-mon[35927]: from='osd.5 [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-15T12:52:55.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:55 smithi149 ceph-mon[35927]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-15T12:52:55.714 DEBUG:teuthology.orchestra.run.smithi184:osd.5> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.5.service 2024-03-15T12:52:55.716 INFO:tasks.cephadm:Deploying osd.6 on smithi002 with /dev/vg_nvme/lv_2... 2024-03-15T12:52:55.716 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-15T12:52:55.968 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:52:57.244 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:57 smithi184 ceph-mon[35993]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-15T12:52:57.244 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:57 smithi184 ceph-mon[35993]: osdmap e36: 6 total, 5 up, 6 in 2024-03-15T12:52:57.244 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:57.244 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:57 smithi184 ceph-mon[35993]: from='osd.5 [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:52:57.244 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:57 smithi184 ceph-mon[35993]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:52:57.244 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:57 smithi184 ceph-mon[35993]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:57.244 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 12:52:57 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T12:52:57.076+0000 7ff7d140e700 -1 osd.5 0 waiting for initial osdmap 2024-03-15T12:52:57.245 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 12:52:57 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T12:52:57.082+0000 7ff7cd201700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-15T12:52:57.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:57 smithi149 ceph-mon[35927]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-15T12:52:57.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:57 smithi149 ceph-mon[35927]: osdmap e36: 6 total, 5 up, 6 in 2024-03-15T12:52:57.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:57.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:57 smithi149 ceph-mon[35927]: from='osd.5 [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:52:57.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:57 smithi149 ceph-mon[35927]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:52:57.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:57 smithi149 ceph-mon[35927]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:57.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:57 smithi002 ceph-mon[32060]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-15T12:52:57.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:57 smithi002 ceph-mon[32060]: osdmap e36: 6 total, 5 up, 6 in 2024-03-15T12:52:57.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:57.374 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:57 smithi002 ceph-mon[32060]: from='osd.5 [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:52:57.375 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:57 smithi002 ceph-mon[32060]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:52:57.375 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:57 smithi002 ceph-mon[32060]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:57.815 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-15T12:52:57.815 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-15T12:52:57.815 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2024-03-15T12:52:57.815 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2024-03-15T12:52:57.815 INFO:teuthology.orchestra.run.smithi002.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00885403 s, 1.2 GB/s 2024-03-15T12:52:57.816 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2024-03-15T12:52:58.159 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:58 smithi002 ceph-mon[32060]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-03-15T12:52:58.159 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:58 smithi002 ceph-mon[32060]: osdmap e37: 6 total, 5 up, 6 in 2024-03-15T12:52:58.159 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:58 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:58.159 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:58 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:58 smithi149 ceph-mon[35927]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-03-15T12:52:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:58 smithi149 ceph-mon[35927]: osdmap e37: 6 total, 5 up, 6 in 2024-03-15T12:52:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:58 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:58 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:58.386 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:58 smithi184 ceph-mon[35993]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-03-15T12:52:58.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:58 smithi184 ceph-mon[35993]: osdmap e37: 6 total, 5 up, 6 in 2024-03-15T12:52:58.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:58 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:58.387 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:58 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:58.495 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi002:vg_nvme/lv_2 2024-03-15T12:52:58.743 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:52:59.191 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:59 smithi002 ceph-mon[32060]: purged_snaps scrub starts 2024-03-15T12:52:59.191 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:59 smithi002 ceph-mon[32060]: purged_snaps scrub ok 2024-03-15T12:52:59.191 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:59.191 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:59 smithi002 ceph-mon[32060]: osd.5 [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624] boot 2024-03-15T12:52:59.191 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:59 smithi002 ceph-mon[32060]: osdmap e38: 6 total, 6 up, 6 in 2024-03-15T12:52:59.191 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:59.191 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:59 smithi002 ceph-mon[32060]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:59.192 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:59.192 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:59.192 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:59.192 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:59.192 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:59.192 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:59.192 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:59.192 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:52:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:59.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:59 smithi149 ceph-mon[35927]: purged_snaps scrub starts 2024-03-15T12:52:59.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:59 smithi149 ceph-mon[35927]: purged_snaps scrub ok 2024-03-15T12:52:59.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:59.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:59 smithi149 ceph-mon[35927]: osd.5 [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624] boot 2024-03-15T12:52:59.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:59 smithi149 ceph-mon[35927]: osdmap e38: 6 total, 6 up, 6 in 2024-03-15T12:52:59.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:59.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:59 smithi149 ceph-mon[35927]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:59.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:59.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:59.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:59.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:59.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:59.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:59.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:59.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:52:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:59.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:59 smithi184 ceph-mon[35993]: purged_snaps scrub starts 2024-03-15T12:52:59.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:59 smithi184 ceph-mon[35993]: purged_snaps scrub ok 2024-03-15T12:52:59.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:59.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:59 smithi184 ceph-mon[35993]: osd.5 [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624] boot 2024-03-15T12:52:59.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:59 smithi184 ceph-mon[35993]: osdmap e38: 6 total, 6 up, 6 in 2024-03-15T12:52:59.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:52:59.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:59 smithi184 ceph-mon[35993]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-03-15T12:52:59.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:59.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:59.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:59.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:52:59.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:52:59.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:52:59.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:52:59.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:52:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:00.154 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:00 smithi002 ceph-mon[32060]: Detected new or changed devices on smithi184 2024-03-15T12:53:00.154 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:00 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi184 to 8544M 2024-03-15T12:53:00.154 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:00 smithi002 ceph-mon[32060]: osdmap e39: 6 total, 6 up, 6 in 2024-03-15T12:53:00.341 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:00 smithi184 ceph-mon[35993]: Detected new or changed devices on smithi184 2024-03-15T12:53:00.341 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:00 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi184 to 8544M 2024-03-15T12:53:00.341 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:00 smithi184 ceph-mon[35993]: osdmap e39: 6 total, 6 up, 6 in 2024-03-15T12:53:00.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.354+0000 7fa1f53e8700 1 -- 172.21.15.2:0/387373048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1f0102070 msgr2=0x7fa1f01024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:53:00.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.354+0000 7fa1f53e8700 1 --2- 172.21.15.2:0/387373048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1f0102070 0x7fa1f01024f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009990 tx=0x7fa1e4009ca0 comp rx=0 tx=0).stop 2024-03-15T12:53:00.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.354+0000 7fa1f53e8700 1 -- 172.21.15.2:0/387373048 shutdown_connections 2024-03-15T12:53:00.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.354+0000 7fa1f53e8700 1 --2- 172.21.15.2:0/387373048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1f0102a30 0x7fa1f0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:00.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.354+0000 7fa1f53e8700 1 --2- 172.21.15.2:0/387373048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1f0100f10 0x7fa1f0101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:00.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.354+0000 7fa1f53e8700 1 --2- 172.21.15.2:0/387373048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1f0102070 0x7fa1f01024f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:00.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.354+0000 7fa1f53e8700 1 -- 172.21.15.2:0/387373048 >> 172.21.15.2:0/387373048 conn(0x7fa1f00fc4b0 msgr2=0x7fa1f00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:53:00.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.354+0000 7fa1f53e8700 1 -- 172.21.15.2:0/387373048 shutdown_connections 2024-03-15T12:53:00.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.354+0000 7fa1f53e8700 1 -- 172.21.15.2:0/387373048 wait complete. 2024-03-15T12:53:00.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.355+0000 7fa1f53e8700 1 Processor -- start 2024-03-15T12:53:00.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.355+0000 7fa1f53e8700 1 -- start start 2024-03-15T12:53:00.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.355+0000 7fa1f53e8700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1f0100f10 0x7fa1f006b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:53:00.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.355+0000 7fa1f53e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1f0102070 0x7fa1f019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:53:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.355+0000 7fa1f53e8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1f0102a30 0x7fa1f019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:53:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.355+0000 7fa1f53e8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f019fad0 con 0x7fa1f0102070 2024-03-15T12:53:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.355+0000 7fa1f53e8700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f019fc40 con 0x7fa1f0100f10 2024-03-15T12:53:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.355+0000 7fa1f53e8700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f019fe00 con 0x7fa1f0102a30 2024-03-15T12:53:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.355+0000 7fa1ee7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1f0102070 0x7fa1f019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:53:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.355+0000 7fa1ee7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1f0102070 0x7fa1f019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42810/0 (socket says 172.21.15.2:42810) 2024-03-15T12:53:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.355+0000 7fa1ee7fc700 1 -- 172.21.15.2:0/2073793737 learned_addr learned my addr 172.21.15.2:0/2073793737 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:53:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.355+0000 7fa1eeffd700 1 --2- 172.21.15.2:0/2073793737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1f0100f10 0x7fa1f006b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:53:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.356+0000 7fa1ef7fe700 1 --2- 172.21.15.2:0/2073793737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1f0102a30 0x7fa1f019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:53:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.356+0000 7fa1ee7fc700 1 -- 172.21.15.2:0/2073793737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1f0102a30 msgr2=0x7fa1f019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:53:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.356+0000 7fa1ee7fc700 1 --2- 172.21.15.2:0/2073793737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1f0102a30 0x7fa1f019f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.356+0000 7fa1ee7fc700 1 -- 172.21.15.2:0/2073793737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1f0100f10 msgr2=0x7fa1f006b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:53:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.356+0000 7fa1ee7fc700 1 --2- 172.21.15.2:0/2073793737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1f0100f10 0x7fa1f006b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:00.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.356+0000 7fa1ee7fc700 1 -- 172.21.15.2:0/2073793737 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e40095f0 con 0x7fa1f0102070 2024-03-15T12:53:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.356+0000 7fa1ee7fc700 1 --2- 172.21.15.2:0/2073793737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1f0102070 0x7fa1f019cca0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400b2e0 tx=0x7fa1e400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:53:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.356+0000 7fa1d7fff700 1 -- 172.21.15.2:0/2073793737 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e401d070 con 0x7fa1f0102070 2024-03-15T12:53:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.356+0000 7fa1d7fff700 1 -- 172.21.15.2:0/2073793737 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa1e400be90 con 0x7fa1f0102070 2024-03-15T12:53:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.356+0000 7fa1f53e8700 1 -- 172.21.15.2:0/2073793737 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f01a0080 con 0x7fa1f0102070 2024-03-15T12:53:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.356+0000 7fa1d7fff700 1 -- 172.21.15.2:0/2073793737 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40173f0 con 0x7fa1f0102070 2024-03-15T12:53:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.356+0000 7fa1f53e8700 1 -- 172.21.15.2:0/2073793737 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f01a0490 con 0x7fa1f0102070 2024-03-15T12:53:00.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.357+0000 7fa1f53e8700 1 -- 172.21.15.2:0/2073793737 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f0108340 con 0x7fa1f0102070 2024-03-15T12:53:00.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.358+0000 7fa1d7fff700 1 -- 172.21.15.2:0/2073793737 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fa1e4022470 con 0x7fa1f0102070 2024-03-15T12:53:00.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.359+0000 7fa1d7fff700 1 --2- 172.21.15.2:0/2073793737 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa1d80a1ac0 0x7fa1d80a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:53:00.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.359+0000 7fa1d7fff700 1 -- 172.21.15.2:0/2073793737 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4533+0+0 (secure 0 0 0) 0x7fa1e40cd870 con 0x7fa1f0102070 2024-03-15T12:53:00.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.359+0000 7fa1eeffd700 1 --2- 172.21.15.2:0/2073793737 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa1d80a1ac0 0x7fa1d80a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:53:00.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.359+0000 7fa1eeffd700 1 --2- 172.21.15.2:0/2073793737 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa1d80a1ac0 0x7fa1d80a3f80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0005950 tx=0x7fa1e000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:53:00.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.363+0000 7fa1d7fff700 1 -- 172.21.15.2:0/2073793737 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa1e405e280 con 0x7fa1f0102070 2024-03-15T12:53:00.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:00 smithi149 ceph-mon[35927]: Detected new or changed devices on smithi184 2024-03-15T12:53:00.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:00 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi184 to 8544M 2024-03-15T12:53:00.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:00 smithi149 ceph-mon[35927]: osdmap e39: 6 total, 6 up, 6 in 2024-03-15T12:53:00.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:00.578+0000 7fa1f53e8700 1 -- 172.21.15.2:0/2073793737 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1f002da30 con 0x7fa1d80a1ac0 2024-03-15T12:53:01.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:01 smithi149 ceph-mon[35927]: osdmap e40: 6 total, 6 up, 6 in 2024-03-15T12:53:01.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:01 smithi149 ceph-mon[35927]: osdmap e41: 6 total, 6 up, 6 in 2024-03-15T12:53:01.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:01 smithi149 ceph-mon[35927]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:53:01.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:01 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:53:01.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:01 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:53:01.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:01 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:01.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:01 smithi149 ceph-mon[35927]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:01.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:01 smithi002 ceph-mon[32060]: osdmap e40: 6 total, 6 up, 6 in 2024-03-15T12:53:01.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:01 smithi002 ceph-mon[32060]: osdmap e41: 6 total, 6 up, 6 in 2024-03-15T12:53:01.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:01 smithi002 ceph-mon[32060]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:53:01.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:01 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:53:01.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:01 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:53:01.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:01 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:01.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:01 smithi002 ceph-mon[32060]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:01.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:01 smithi184 ceph-mon[35993]: osdmap e40: 6 total, 6 up, 6 in 2024-03-15T12:53:01.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:01 smithi184 ceph-mon[35993]: osdmap e41: 6 total, 6 up, 6 in 2024-03-15T12:53:01.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:01 smithi184 ceph-mon[35993]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:53:01.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:01 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:53:01.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:01 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:53:01.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:01 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:01.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:01 smithi184 ceph-mon[35993]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:02.461 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:02 smithi002 ceph-mon[32060]: osdmap e42: 6 total, 6 up, 6 in 2024-03-15T12:53:02.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:02 smithi149 ceph-mon[35927]: osdmap e42: 6 total, 6 up, 6 in 2024-03-15T12:53:02.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:02 smithi184 ceph-mon[35993]: osdmap e42: 6 total, 6 up, 6 in 2024-03-15T12:53:03.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:03 smithi149 ceph-mon[35927]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-15T12:53:03.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:03 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1388831407' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27"}]: dispatch 2024-03-15T12:53:03.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:03 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1388831407' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27"}]': finished 2024-03-15T12:53:03.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:03 smithi149 ceph-mon[35927]: osdmap e43: 7 total, 6 up, 7 in 2024-03-15T12:53:03.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:03 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:03.716 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:03 smithi002 ceph-mon[32060]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-15T12:53:03.716 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:03 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1388831407' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27"}]: dispatch 2024-03-15T12:53:03.716 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:03 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1388831407' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27"}]': finished 2024-03-15T12:53:03.716 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:03 smithi002 ceph-mon[32060]: osdmap e43: 7 total, 6 up, 7 in 2024-03-15T12:53:03.716 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:03 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:03.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:03 smithi184 ceph-mon[35993]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-15T12:53:03.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:03 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1388831407' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27"}]: dispatch 2024-03-15T12:53:03.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:03 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1388831407' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27"}]': finished 2024-03-15T12:53:03.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:03 smithi184 ceph-mon[35993]: osdmap e43: 7 total, 6 up, 7 in 2024-03-15T12:53:03.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:03 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:04.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:04 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3589959013' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:53:04.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:04 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3589959013' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:53:04.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:04 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3589959013' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:53:05.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:05 smithi149 ceph-mon[35927]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-03-15T12:53:05.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:05 smithi002 ceph-mon[32060]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-03-15T12:53:05.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:05 smithi184 ceph-mon[35993]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-03-15T12:53:07.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:06 smithi002 ceph-mon[32060]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-15T12:53:07.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:06 smithi184 ceph-mon[35993]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-15T12:53:07.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:06 smithi149 ceph-mon[35927]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-15T12:53:09.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:08 smithi002 ceph-mon[32060]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-15T12:53:09.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:08 smithi184 ceph-mon[35993]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-15T12:53:09.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:08 smithi149 ceph-mon[35927]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-15T12:53:11.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:10 smithi002 ceph-mon[32060]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:11.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:10 smithi184 ceph-mon[35993]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:11.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:10 smithi149 ceph-mon[35927]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:11.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:11 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-15T12:53:11.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:11 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:11.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:11 smithi149 ceph-mon[35927]: Deploying daemon osd.6 on smithi002 2024-03-15T12:53:12.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:11 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-15T12:53:12.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:11 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:12.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:11 smithi002 ceph-mon[32060]: Deploying daemon osd.6 on smithi002 2024-03-15T12:53:12.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:11 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-15T12:53:12.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:11 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:12.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:11 smithi184 ceph-mon[35993]: Deploying daemon osd.6 on smithi002 2024-03-15T12:53:13.040 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:12 smithi002 ceph-mon[32060]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:13.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:12 smithi184 ceph-mon[35993]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:13.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:12 smithi149 ceph-mon[35927]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:14.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:14 smithi002 ceph-mon[32060]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:15.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:14 smithi184 ceph-mon[35993]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:15.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:14 smithi149 ceph-mon[35927]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:18.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:18.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:18.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:53:18.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:17 smithi002 ceph-mon[32060]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:18.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:18.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:18.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:53:18.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:17 smithi184 ceph-mon[35993]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:18.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:18.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:18.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:53:18.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:17 smithi149 ceph-mon[35927]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:18.978 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:18 smithi002 ceph-mon[32060]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:19.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:18 smithi184 ceph-mon[35993]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:19.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:18 smithi149 ceph-mon[35927]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:20.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:20.589+0000 7fa1d7fff700 1 -- 172.21.15.2:0/2073793737 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa1f002da30 con 0x7fa1d80a1ac0 2024-03-15T12:53:20.576 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 6 on host 'smithi002' 2024-03-15T12:53:20.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:20.591+0000 7fa1f53e8700 1 -- 172.21.15.2:0/2073793737 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa1d80a1ac0 msgr2=0x7fa1d80a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:53:20.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:20.591+0000 7fa1f53e8700 1 --2- 172.21.15.2:0/2073793737 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa1d80a1ac0 0x7fa1d80a3f80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0005950 tx=0x7fa1e000b410 comp rx=0 tx=0).stop 2024-03-15T12:53:20.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:20.591+0000 7fa1f53e8700 1 -- 172.21.15.2:0/2073793737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1f0102070 msgr2=0x7fa1f019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:53:20.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:20.591+0000 7fa1f53e8700 1 --2- 172.21.15.2:0/2073793737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1f0102070 0x7fa1f019cca0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400b2e0 tx=0x7fa1e400f690 comp rx=0 tx=0).stop 2024-03-15T12:53:20.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:20.592+0000 7fa1f53e8700 1 -- 172.21.15.2:0/2073793737 shutdown_connections 2024-03-15T12:53:20.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:20.592+0000 7fa1f53e8700 1 --2- 172.21.15.2:0/2073793737 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa1d80a1ac0 0x7fa1d80a3f80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:20.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:20.592+0000 7fa1f53e8700 1 --2- 172.21.15.2:0/2073793737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1f0102a30 0x7fa1f019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:20.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:20.592+0000 7fa1f53e8700 1 --2- 172.21.15.2:0/2073793737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1f0100f10 0x7fa1f006b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:20.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:20.592+0000 7fa1f53e8700 1 --2- 172.21.15.2:0/2073793737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1f0102070 0x7fa1f019cca0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:20.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:20.592+0000 7fa1f53e8700 1 -- 172.21.15.2:0/2073793737 >> 172.21.15.2:0/2073793737 conn(0x7fa1f00fc4b0 msgr2=0x7fa1f0105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:53:20.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:20.592+0000 7fa1f53e8700 1 -- 172.21.15.2:0/2073793737 shutdown_connections 2024-03-15T12:53:20.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:53:20.592+0000 7fa1f53e8700 1 -- 172.21.15.2:0/2073793737 wait complete. 2024-03-15T12:53:20.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:20.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:20.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:20.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:53:20.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:20.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:20.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:53:20.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:20.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:20 smithi002 ceph-mon[32060]: from='osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-15T12:53:21.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:21.110 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:21.110 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:21.110 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:53:21.110 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:21.110 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:21.110 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:53:21.110 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:21.111 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:20 smithi184 ceph-mon[35993]: from='osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-15T12:53:21.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:21.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:21.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:21.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:53:21.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:21.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:21.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:53:21.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:21.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:20 smithi149 ceph-mon[35927]: from='osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-15T12:53:21.165 DEBUG:teuthology.orchestra.run.smithi002:osd.6> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.6.service 2024-03-15T12:53:21.168 INFO:tasks.cephadm:Deploying osd.7 on smithi149 with /dev/vg_nvme/lv_2... 2024-03-15T12:53:21.169 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-15T12:53:21.418 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.b/config 2024-03-15T12:53:21.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:21 smithi149 ceph-mon[35927]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:21.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:21 smithi149 ceph-mon[35927]: from='osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-15T12:53:21.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:21 smithi149 ceph-mon[35927]: osdmap e44: 7 total, 6 up, 7 in 2024-03-15T12:53:21.793 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:21 smithi149 ceph-mon[35927]: from='osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-15T12:53:21.793 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:21 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:21.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:21 smithi002 ceph-mon[32060]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:21.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:21 smithi002 ceph-mon[32060]: from='osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-15T12:53:21.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:21 smithi002 ceph-mon[32060]: osdmap e44: 7 total, 6 up, 7 in 2024-03-15T12:53:21.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:21 smithi002 ceph-mon[32060]: from='osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-15T12:53:21.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:21 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:22.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:21 smithi184 ceph-mon[35993]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:22.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:21 smithi184 ceph-mon[35993]: from='osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-15T12:53:22.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:21 smithi184 ceph-mon[35993]: osdmap e44: 7 total, 6 up, 7 in 2024-03-15T12:53:22.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:21 smithi184 ceph-mon[35993]: from='osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-15T12:53:22.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:21 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:23.470 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-15T12:53:23.471 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-15T12:53:23.471 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-15T12:53:23.471 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-15T12:53:23.471 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0157129 s, 667 MB/s 2024-03-15T12:53:23.471 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-15T12:53:23.554 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 12:53:23 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T12:53:23.120+0000 7f7ef74ff700 -1 osd.6 0 waiting for initial osdmap 2024-03-15T12:53:23.555 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 12:53:23 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T12:53:23.133+0000 7f7eef2ea700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-15T12:53:23.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:23 smithi149 ceph-mon[35927]: purged_snaps scrub starts 2024-03-15T12:53:23.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:23 smithi149 ceph-mon[35927]: purged_snaps scrub ok 2024-03-15T12:53:23.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:23 smithi149 ceph-mon[35927]: from='osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-15T12:53:23.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:23 smithi149 ceph-mon[35927]: osdmap e45: 7 total, 6 up, 7 in 2024-03-15T12:53:23.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:23 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:23.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:23 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:23.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:23 smithi149 ceph-mon[35927]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:23.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:23 smithi149 ceph-mon[35927]: from='osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638]' entity='osd.6' 2024-03-15T12:53:23.964 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:23 smithi002 ceph-mon[32060]: purged_snaps scrub starts 2024-03-15T12:53:23.964 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:23 smithi002 ceph-mon[32060]: purged_snaps scrub ok 2024-03-15T12:53:23.964 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:23 smithi002 ceph-mon[32060]: from='osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-15T12:53:23.964 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:23 smithi002 ceph-mon[32060]: osdmap e45: 7 total, 6 up, 7 in 2024-03-15T12:53:23.964 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:23 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:23.965 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:23 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:23.965 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:23 smithi002 ceph-mon[32060]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:23.965 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:23 smithi002 ceph-mon[32060]: from='osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638]' entity='osd.6' 2024-03-15T12:53:24.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:23 smithi184 ceph-mon[35993]: purged_snaps scrub starts 2024-03-15T12:53:24.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:23 smithi184 ceph-mon[35993]: purged_snaps scrub ok 2024-03-15T12:53:24.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:23 smithi184 ceph-mon[35993]: from='osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-15T12:53:24.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:23 smithi184 ceph-mon[35993]: osdmap e45: 7 total, 6 up, 7 in 2024-03-15T12:53:24.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:23 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:24.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:23 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:24.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:23 smithi184 ceph-mon[35993]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-03-15T12:53:24.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:23 smithi184 ceph-mon[35993]: from='osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638]' entity='osd.6' 2024-03-15T12:53:24.145 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_2 2024-03-15T12:53:24.398 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.b/config 2024-03-15T12:53:24.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:24.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:24 smithi002 ceph-mon[32060]: osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638] boot 2024-03-15T12:53:24.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:24 smithi002 ceph-mon[32060]: osdmap e46: 7 total, 7 up, 7 in 2024-03-15T12:53:24.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:24.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:24.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:24.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:24.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:24.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:24.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:24.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:24.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:53:24.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:24.845 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:24.845 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:24 smithi149 ceph-mon[35927]: osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638] boot 2024-03-15T12:53:24.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:24 smithi149 ceph-mon[35927]: osdmap e46: 7 total, 7 up, 7 in 2024-03-15T12:53:24.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:24.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:24.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:24.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:24.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:24.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:24.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:24.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:24.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:53:24.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:25.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:25.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:24 smithi184 ceph-mon[35993]: osd.6 [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638] boot 2024-03-15T12:53:25.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:24 smithi184 ceph-mon[35993]: osdmap e46: 7 total, 7 up, 7 in 2024-03-15T12:53:25.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:53:25.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:25.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:25.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:25.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:25.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:25.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:25.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:25.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:53:25.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:25.837 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:25 smithi149 ceph-mon[35927]: Detected new or changed devices on smithi002 2024-03-15T12:53:25.838 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:25 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi002 to 5697M 2024-03-15T12:53:25.838 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:25 smithi149 ceph-mon[35927]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:26.050 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.064+0000 7f966579b700 1 -- 172.21.15.149:0/3214963573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660107d60 msgr2=0x7f966010a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:53:26.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.064+0000 7f966579b700 1 --2- 172.21.15.149:0/3214963573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660107d60 0x7f966010a170 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f965800b0a0 tx=0x7f965800b3b0 comp rx=0 tx=0).stop 2024-03-15T12:53:26.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.064+0000 7f966579b700 1 -- 172.21.15.149:0/3214963573 shutdown_connections 2024-03-15T12:53:26.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.064+0000 7f966579b700 1 --2- 172.21.15.149:0/3214963573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660107d60 0x7f966010a170 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:26.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.064+0000 7f966579b700 1 --2- 172.21.15.149:0/3214963573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9660102b00 0x7f9660104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:26.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.064+0000 7f966579b700 1 --2- 172.21.15.149:0/3214963573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9660105430 0x7f9660107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:26.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.064+0000 7f966579b700 1 -- 172.21.15.149:0/3214963573 >> 172.21.15.149:0/3214963573 conn(0x7f96600fc4b0 msgr2=0x7f96600fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:53:26.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.064+0000 7f966579b700 1 -- 172.21.15.149:0/3214963573 shutdown_connections 2024-03-15T12:53:26.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.064+0000 7f966579b700 1 -- 172.21.15.149:0/3214963573 wait complete. 2024-03-15T12:53:26.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.064+0000 7f966579b700 1 Processor -- start 2024-03-15T12:53:26.051 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.064+0000 7f966579b700 1 -- start start 2024-03-15T12:53:26.053 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.064+0000 7f966579b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9660102b00 0x7f96601980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:53:26.053 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f966579b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9660105430 0x7f96601985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:53:26.053 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f966579b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660107d60 0x7f966019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:53:26.053 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f966579b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966019fc30 con 0x7f9660105430 2024-03-15T12:53:26.053 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f966579b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966019fd70 con 0x7f9660107d60 2024-03-15T12:53:26.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f966579b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966019ff00 con 0x7f9660102b00 2024-03-15T12:53:26.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f965f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660107d60 0x7f966019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:53:26.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f965f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660107d60 0x7f966019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:57352/0 (socket says 172.21.15.149:57352) 2024-03-15T12:53:26.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f965f7fe700 1 -- 172.21.15.149:0/3670533830 learned_addr learned my addr 172.21.15.149:0/3670533830 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-15T12:53:26.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f965e7fc700 1 --2- 172.21.15.149:0/3670533830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9660105430 0x7f96601985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:53:26.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f965effd700 1 --2- 172.21.15.149:0/3670533830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9660102b00 0x7f96601980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:53:26.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f965f7fe700 1 -- 172.21.15.149:0/3670533830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9660102b00 msgr2=0x7f96601980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:53:26.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f965f7fe700 1 --2- 172.21.15.149:0/3670533830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9660102b00 0x7f96601980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:26.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f965f7fe700 1 -- 172.21.15.149:0/3670533830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9660105430 msgr2=0x7f96601985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:53:26.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f965f7fe700 1 --2- 172.21.15.149:0/3670533830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9660105430 0x7f96601985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:26.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f965f7fe700 1 -- 172.21.15.149:0/3670533830 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9658009d00 con 0x7f9660107d60 2024-03-15T12:53:26.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f965e7fc700 1 --2- 172.21.15.149:0/3670533830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9660105430 0x7f96601985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:53:26.054 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f965f7fe700 1 --2- 172.21.15.149:0/3670533830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660107d60 0x7f966019f660 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f965800b070 tx=0x7f9658004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:53:26.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f9647fff700 1 -- 172.21.15.149:0/3670533830 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965800e040 con 0x7f9660107d60 2024-03-15T12:53:26.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f9647fff700 1 -- 172.21.15.149:0/3670533830 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f965801db50 con 0x7f9660107d60 2024-03-15T12:53:26.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f9647fff700 1 -- 172.21.15.149:0/3670533830 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965801bca0 con 0x7f9660107d60 2024-03-15T12:53:26.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f966579b700 1 -- 172.21.15.149:0/3670533830 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96601a0180 con 0x7f9660107d60 2024-03-15T12:53:26.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.065+0000 7f966579b700 1 -- 172.21.15.149:0/3670533830 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96601a05a0 con 0x7f9660107d60 2024-03-15T12:53:26.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.066+0000 7f966579b700 1 -- 172.21.15.149:0/3670533830 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96600fe030 con 0x7f9660107d60 2024-03-15T12:53:26.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:25 smithi002 ceph-mon[32060]: Detected new or changed devices on smithi002 2024-03-15T12:53:26.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:25 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi002 to 5697M 2024-03-15T12:53:26.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:25 smithi002 ceph-mon[32060]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:26.056 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.067+0000 7f9647fff700 1 -- 172.21.15.149:0/3670533830 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f965801dcc0 con 0x7f9660107d60 2024-03-15T12:53:26.056 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.068+0000 7f9647fff700 1 --2- 172.21.15.149:0/3670533830 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f96480a1ac0 0x7f96480a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:53:26.056 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.068+0000 7f9647fff700 1 -- 172.21.15.149:0/3670533830 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f96580ce930 con 0x7f9660107d60 2024-03-15T12:53:26.056 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.069+0000 7f965effd700 1 --2- 172.21.15.149:0/3670533830 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f96480a1ac0 0x7f96480a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:53:26.057 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.069+0000 7f965effd700 1 --2- 172.21.15.149:0/3670533830 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f96480a1ac0 0x7f96480a3f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9650006fd0 tx=0x7f9650008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:53:26.059 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.072+0000 7f9647fff700 1 -- 172.21.15.149:0/3670533830 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9658098840 con 0x7f9660107d60 2024-03-15T12:53:26.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:25 smithi184 ceph-mon[35993]: Detected new or changed devices on smithi002 2024-03-15T12:53:26.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:25 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi002 to 5697M 2024-03-15T12:53:26.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:25 smithi184 ceph-mon[35993]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:26.263 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:26.276+0000 7f966579b700 1 -- 172.21.15.149:0/3670533830 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f966002da90 con 0x7f96480a1ac0 2024-03-15T12:53:26.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:26 smithi149 ceph-mon[35927]: osdmap e47: 7 total, 7 up, 7 in 2024-03-15T12:53:26.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:26 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:53:26.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:26 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:53:26.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:26 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:27.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:26 smithi002 ceph-mon[32060]: osdmap e47: 7 total, 7 up, 7 in 2024-03-15T12:53:27.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:26 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:53:27.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:26 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:53:27.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:26 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:27.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:26 smithi184 ceph-mon[35993]: osdmap e47: 7 total, 7 up, 7 in 2024-03-15T12:53:27.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:26 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:53:27.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:26 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:53:27.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:26 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:27.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:27 smithi149 ceph-mon[35927]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:53:27.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:27 smithi149 ceph-mon[35927]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:28.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:27 smithi002 ceph-mon[32060]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:53:28.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:27 smithi002 ceph-mon[32060]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:28.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:27 smithi184 ceph-mon[35993]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:53:28.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:27 smithi184 ceph-mon[35993]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:29.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:28 smithi002 ceph-mon[32060]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:29.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:28 smithi184 ceph-mon[35993]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:29.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:28 smithi149 ceph-mon[35927]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:30.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:29 smithi002 ceph-mon[32060]: from='client.? 172.21.15.149:0/1446717043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "997ac305-e8f7-4ae6-b9ea-a491dfad807a"}]: dispatch 2024-03-15T12:53:30.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:29 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "997ac305-e8f7-4ae6-b9ea-a491dfad807a"}]: dispatch 2024-03-15T12:53:30.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:29 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "997ac305-e8f7-4ae6-b9ea-a491dfad807a"}]': finished 2024-03-15T12:53:30.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:29 smithi002 ceph-mon[32060]: osdmap e48: 8 total, 7 up, 8 in 2024-03-15T12:53:30.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:29 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:53:30.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:29 smithi002 ceph-mon[32060]: from='client.? 172.21.15.149:0/1649833918' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:53:30.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:29 smithi184 ceph-mon[35993]: from='client.? 172.21.15.149:0/1446717043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "997ac305-e8f7-4ae6-b9ea-a491dfad807a"}]: dispatch 2024-03-15T12:53:30.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:29 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "997ac305-e8f7-4ae6-b9ea-a491dfad807a"}]: dispatch 2024-03-15T12:53:30.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:29 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "997ac305-e8f7-4ae6-b9ea-a491dfad807a"}]': finished 2024-03-15T12:53:30.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:29 smithi184 ceph-mon[35993]: osdmap e48: 8 total, 7 up, 8 in 2024-03-15T12:53:30.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:29 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:53:30.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:29 smithi184 ceph-mon[35993]: from='client.? 172.21.15.149:0/1649833918' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:53:30.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:29 smithi149 ceph-mon[35927]: from='client.? 172.21.15.149:0/1446717043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "997ac305-e8f7-4ae6-b9ea-a491dfad807a"}]: dispatch 2024-03-15T12:53:30.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:29 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "997ac305-e8f7-4ae6-b9ea-a491dfad807a"}]: dispatch 2024-03-15T12:53:30.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:29 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "997ac305-e8f7-4ae6-b9ea-a491dfad807a"}]': finished 2024-03-15T12:53:30.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:29 smithi149 ceph-mon[35927]: osdmap e48: 8 total, 7 up, 8 in 2024-03-15T12:53:30.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:29 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:53:30.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:29 smithi149 ceph-mon[35927]: from='client.? 172.21.15.149:0/1649833918' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:53:31.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:30 smithi002 ceph-mon[32060]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:31.062 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:30 smithi184 ceph-mon[35993]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:31.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:30 smithi149 ceph-mon[35927]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:32.960 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:32 smithi149 ceph-mon[35927]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:33.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:32 smithi002 ceph-mon[32060]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:33.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:32 smithi184 ceph-mon[35993]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:35.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:34 smithi149 ceph-mon[35927]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:35.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:34 smithi002 ceph-mon[32060]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:35.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:34 smithi184 ceph-mon[35993]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:36.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:36 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-15T12:53:36.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:36 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:36.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:36 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-15T12:53:36.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:36 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:36.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:36 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-15T12:53:36.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:36 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:37.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:37 smithi149 ceph-mon[35927]: Deploying daemon osd.7 on smithi149 2024-03-15T12:53:37.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:37 smithi149 ceph-mon[35927]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:37.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:37 smithi002 ceph-mon[32060]: Deploying daemon osd.7 on smithi149 2024-03-15T12:53:37.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:37 smithi002 ceph-mon[32060]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:37.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:37 smithi184 ceph-mon[35993]: Deploying daemon osd.7 on smithi149 2024-03-15T12:53:37.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:37 smithi184 ceph-mon[35993]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:39.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:38 smithi002 ceph-mon[32060]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:39.055 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:38 smithi149 ceph-mon[35927]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:39.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:38 smithi184 ceph-mon[35993]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:40.830 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:40 smithi149 ceph-mon[35927]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:41.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:40 smithi002 ceph-mon[32060]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:41.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:40 smithi184 ceph-mon[35993]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:43.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:42 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:43.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:42 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:43.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:42 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:53:43.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:42 smithi002 ceph-mon[32060]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:43.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:42 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:43.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:42 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:43.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:42 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:53:43.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:42 smithi184 ceph-mon[35993]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:43.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:42 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:43.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:42 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:43.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:42 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:53:43.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:42 smithi149 ceph-mon[35927]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:44.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:44 smithi149 ceph-mon[35927]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:45.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:44 smithi002 ceph-mon[32060]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:45.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:44 smithi184 ceph-mon[35993]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:45.180 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 7 on host 'smithi149' 2024-03-15T12:53:45.180 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:45.192+0000 7f9647fff700 1 -- 172.21.15.149:0/3670533830 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f966002da90 con 0x7f96480a1ac0 2024-03-15T12:53:45.182 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:45.195+0000 7f966579b700 1 -- 172.21.15.149:0/3670533830 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f96480a1ac0 msgr2=0x7f96480a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:53:45.182 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:45.195+0000 7f966579b700 1 --2- 172.21.15.149:0/3670533830 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f96480a1ac0 0x7f96480a3f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9650006fd0 tx=0x7f9650008040 comp rx=0 tx=0).stop 2024-03-15T12:53:45.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:45.195+0000 7f966579b700 1 -- 172.21.15.149:0/3670533830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660107d60 msgr2=0x7f966019f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:53:45.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:45.195+0000 7f966579b700 1 --2- 172.21.15.149:0/3670533830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660107d60 0x7f966019f660 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f965800b070 tx=0x7f9658004a50 comp rx=0 tx=0).stop 2024-03-15T12:53:45.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:45.195+0000 7f966579b700 1 -- 172.21.15.149:0/3670533830 shutdown_connections 2024-03-15T12:53:45.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:45.195+0000 7f966579b700 1 --2- 172.21.15.149:0/3670533830 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f96480a1ac0 0x7f96480a3f80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:45.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:45.195+0000 7f966579b700 1 --2- 172.21.15.149:0/3670533830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9660107d60 0x7f966019f660 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:45.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:45.195+0000 7f966579b700 1 --2- 172.21.15.149:0/3670533830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9660102b00 0x7f96601980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:45.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:45.195+0000 7f966579b700 1 --2- 172.21.15.149:0/3670533830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9660105430 0x7f96601985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:45.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:45.195+0000 7f966579b700 1 -- 172.21.15.149:0/3670533830 >> 172.21.15.149:0/3670533830 conn(0x7f96600fc4b0 msgr2=0x7f9660103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:53:45.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:45.195+0000 7f966579b700 1 -- 172.21.15.149:0/3670533830 shutdown_connections 2024-03-15T12:53:45.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:53:45.195+0000 7f966579b700 1 -- 172.21.15.149:0/3670533830 wait complete. 2024-03-15T12:53:45.760 DEBUG:teuthology.orchestra.run.smithi149:osd.7> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.7.service 2024-03-15T12:53:45.763 INFO:tasks.cephadm:Deploying osd.8 on smithi184 with /dev/vg_nvme/lv_2... 2024-03-15T12:53:45.763 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-15T12:53:46.009 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T12:53:46.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:46 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:46.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:46 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:46.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:46 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:46.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:46 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:46.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:46 smithi149 ceph-mon[35927]: from='osd.7 [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-15T12:53:46.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:46 smithi149 ceph-mon[35927]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-15T12:53:46.446 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:46 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:46.446 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:46 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:46.446 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:46 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:46.446 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:46 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:46.446 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:46 smithi184 ceph-mon[35993]: from='osd.7 [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-15T12:53:46.446 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:46 smithi184 ceph-mon[35993]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-15T12:53:46.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:46 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:46.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:46 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:46.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:46 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:46.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:46 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:46.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:46 smithi002 ceph-mon[32060]: from='osd.7 [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-15T12:53:46.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:46 smithi002 ceph-mon[32060]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-15T12:53:47.436 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:47 smithi184 ceph-mon[35993]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-15T12:53:47.437 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:47 smithi184 ceph-mon[35993]: osdmap e49: 8 total, 7 up, 8 in 2024-03-15T12:53:47.437 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:47 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:53:47.437 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:47 smithi184 ceph-mon[35993]: from='osd.7 [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:53:47.437 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:47 smithi184 ceph-mon[35993]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:53:47.437 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:47 smithi184 ceph-mon[35993]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:47.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:47 smithi002 ceph-mon[32060]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-15T12:53:47.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:47 smithi002 ceph-mon[32060]: osdmap e49: 8 total, 7 up, 8 in 2024-03-15T12:53:47.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:47 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:53:47.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:47 smithi002 ceph-mon[32060]: from='osd.7 [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:53:47.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:47 smithi002 ceph-mon[32060]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:53:47.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:47 smithi002 ceph-mon[32060]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:47.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:47 smithi149 ceph-mon[35927]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-15T12:53:47.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:47 smithi149 ceph-mon[35927]: osdmap e49: 8 total, 7 up, 8 in 2024-03-15T12:53:47.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:47 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:53:47.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:47 smithi149 ceph-mon[35927]: from='osd.7 [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:53:47.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:47 smithi149 ceph-mon[35927]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:53:47.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:47 smithi149 ceph-mon[35927]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-03-15T12:53:48.148 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-15T12:53:48.148 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-15T12:53:48.148 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-03-15T12:53:48.148 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-03-15T12:53:48.149 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0135944 s, 771 MB/s 2024-03-15T12:53:48.149 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-03-15T12:53:48.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:48 smithi149 ceph-mon[35927]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-15T12:53:48.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:48 smithi149 ceph-mon[35927]: osdmap e50: 8 total, 7 up, 8 in 2024-03-15T12:53:48.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:48 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:53:48.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:48 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:53:48.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:48 smithi149 ceph-mon[35927]: from='osd.7 ' entity='osd.7' 2024-03-15T12:53:48.361 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 12:53:48 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T12:53:48.090+0000 7f471e415700 -1 osd.7 0 waiting for initial osdmap 2024-03-15T12:53:48.361 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 12:53:48 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T12:53:48.101+0000 7f4716200700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-15T12:53:48.452 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:48 smithi184 ceph-mon[35993]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-15T12:53:48.452 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:48 smithi184 ceph-mon[35993]: osdmap e50: 8 total, 7 up, 8 in 2024-03-15T12:53:48.452 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:48 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:53:48.452 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:48 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:53:48.452 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:48 smithi184 ceph-mon[35993]: from='osd.7 ' entity='osd.7' 2024-03-15T12:53:48.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:48 smithi002 ceph-mon[32060]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-15T12:53:48.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:48 smithi002 ceph-mon[32060]: osdmap e50: 8 total, 7 up, 8 in 2024-03-15T12:53:48.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:48 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:53:48.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:48 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:53:48.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:48 smithi002 ceph-mon[32060]: from='osd.7 ' entity='osd.7' 2024-03-15T12:53:48.820 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi184:vg_nvme/lv_2 2024-03-15T12:53:49.077 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T12:53:49.469 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: purged_snaps scrub starts 2024-03-15T12:53:49.470 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: purged_snaps scrub ok 2024-03-15T12:53:49.470 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: osd.7 [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371] boot 2024-03-15T12:53:49.470 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: osdmap e51: 8 total, 8 up, 8 in 2024-03-15T12:53:49.470 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:53:49.470 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:49.470 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:49.470 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:49.470 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:49.470 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:49.470 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:49.470 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:49.471 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:49.471 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:53:49.471 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:49.471 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:53:49.471 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:49.471 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:53:49.471 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:49.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: purged_snaps scrub starts 2024-03-15T12:53:49.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: purged_snaps scrub ok 2024-03-15T12:53:49.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: osd.7 [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371] boot 2024-03-15T12:53:49.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: osdmap e51: 8 total, 8 up, 8 in 2024-03-15T12:53:49.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:53:49.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:49.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:49.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:49.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:49.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:49.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:49.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:49.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:49.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:53:49.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:49.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:53:49.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:49.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:53:49.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:49.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: purged_snaps scrub starts 2024-03-15T12:53:49.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: purged_snaps scrub ok 2024-03-15T12:53:49.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: osd.7 [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371] boot 2024-03-15T12:53:49.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: osdmap e51: 8 total, 8 up, 8 in 2024-03-15T12:53:49.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:53:49.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:49.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:49.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:49.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:49.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:49.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:53:49.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:49.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:49.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:53:49.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:49.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:53:49.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:49.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:53:49.612 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:53:50.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:50 smithi002 ceph-mon[32060]: Detected new or changed devices on smithi149 2024-03-15T12:53:50.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:50 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi149 to 1938M 2024-03-15T12:53:50.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:50 smithi002 ceph-mon[32060]: osdmap e52: 8 total, 8 up, 8 in 2024-03-15T12:53:50.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:50 smithi184 ceph-mon[35993]: Detected new or changed devices on smithi149 2024-03-15T12:53:50.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:50 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi149 to 1938M 2024-03-15T12:53:50.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:50 smithi184 ceph-mon[35993]: osdmap e52: 8 total, 8 up, 8 in 2024-03-15T12:53:50.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:50 smithi149 ceph-mon[35927]: Detected new or changed devices on smithi149 2024-03-15T12:53:50.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:50 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi149 to 1938M 2024-03-15T12:53:50.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:50 smithi149 ceph-mon[35927]: osdmap e52: 8 total, 8 up, 8 in 2024-03-15T12:53:50.773 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.784+0000 7f1d6a5e1700 1 -- 172.21.15.184:0/3603998178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d64103be0 msgr2=0x7f1d6410a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:53:50.774 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.784+0000 7f1d6a5e1700 1 --2- 172.21.15.184:0/3603998178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d64103be0 0x7f1d6410a490 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00b0a0 tx=0x7f1d5c00b3b0 comp rx=0 tx=0).stop 2024-03-15T12:53:50.774 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.784+0000 7f1d6a5e1700 1 -- 172.21.15.184:0/3603998178 shutdown_connections 2024-03-15T12:53:50.774 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.784+0000 7f1d6a5e1700 1 --2- 172.21.15.184:0/3603998178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d64103be0 0x7f1d6410a490 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:50.774 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.784+0000 7f1d6a5e1700 1 --2- 172.21.15.184:0/3603998178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d64102040 0x7f1d64102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:50.774 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.784+0000 7f1d6a5e1700 1 --2- 172.21.15.184:0/3603998178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d64103240 0x7f1d641036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:50.774 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.784+0000 7f1d6a5e1700 1 -- 172.21.15.184:0/3603998178 >> 172.21.15.184:0/3603998178 conn(0x7f1d640fd5e0 msgr2=0x7f1d640ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:53:50.774 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.784+0000 7f1d6a5e1700 1 -- 172.21.15.184:0/3603998178 shutdown_connections 2024-03-15T12:53:50.774 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.784+0000 7f1d6a5e1700 1 -- 172.21.15.184:0/3603998178 wait complete. 2024-03-15T12:53:50.774 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.784+0000 7f1d6a5e1700 1 Processor -- start 2024-03-15T12:53:50.774 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d6a5e1700 1 -- start start 2024-03-15T12:53:50.774 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d6a5e1700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d64102040 0x7f1d641980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:53:50.775 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d6a5e1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d64103240 0x7f1d641985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:53:50.775 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d6a5e1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d64103be0 0x7f1d6419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:53:50.775 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d6a5e1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6419fc40 con 0x7f1d64103be0 2024-03-15T12:53:50.775 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d6a5e1700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6419fd80 con 0x7f1d64102040 2024-03-15T12:53:50.775 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d6a5e1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6419ff10 con 0x7f1d64103240 2024-03-15T12:53:50.775 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d637fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d64103240 0x7f1d641985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:53:50.775 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d637fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d64103240 0x7f1d641985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:36886/0 (socket says 172.21.15.184:36886) 2024-03-15T12:53:50.775 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d637fe700 1 -- 172.21.15.184:0/4127355115 learned_addr learned my addr 172.21.15.184:0/4127355115 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-15T12:53:50.775 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d68b7e700 1 --2- 172.21.15.184:0/4127355115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d64103be0 0x7f1d6419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:53:50.775 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d63fff700 1 --2- 172.21.15.184:0/4127355115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d64102040 0x7f1d641980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:53:50.775 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d637fe700 1 -- 172.21.15.184:0/4127355115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d64102040 msgr2=0x7f1d641980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:53:50.775 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d637fe700 1 --2- 172.21.15.184:0/4127355115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d64102040 0x7f1d641980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:50.775 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d637fe700 1 -- 172.21.15.184:0/4127355115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d64103be0 msgr2=0x7f1d6419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:53:50.776 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d637fe700 1 --2- 172.21.15.184:0/4127355115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d64103be0 0x7f1d6419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:53:50.776 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.785+0000 7f1d637fe700 1 -- 172.21.15.184:0/4127355115 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c009d00 con 0x7f1d64103240 2024-03-15T12:53:50.776 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.786+0000 7f1d637fe700 1 --2- 172.21.15.184:0/4127355115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d64103240 0x7f1d641985f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c950 tx=0x7f1d5800cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:53:50.776 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.786+0000 7f1d617fa700 1 -- 172.21.15.184:0/4127355115 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58007ac0 con 0x7f1d64103240 2024-03-15T12:53:50.776 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.786+0000 7f1d617fa700 1 -- 172.21.15.184:0/4127355115 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1d58004d00 con 0x7f1d64103240 2024-03-15T12:53:50.776 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.786+0000 7f1d617fa700 1 -- 172.21.15.184:0/4127355115 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d580055f0 con 0x7f1d64103240 2024-03-15T12:53:50.776 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.786+0000 7f1d6a5e1700 1 -- 172.21.15.184:0/4127355115 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d641a01f0 con 0x7f1d64103240 2024-03-15T12:53:50.776 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.786+0000 7f1d6a5e1700 1 -- 172.21.15.184:0/4127355115 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d641a0740 con 0x7f1d64103240 2024-03-15T12:53:50.777 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.787+0000 7f1d6a5e1700 1 -- 172.21.15.184:0/4127355115 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d64109330 con 0x7f1d64103240 2024-03-15T12:53:50.777 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.788+0000 7f1d617fa700 1 -- 172.21.15.184:0/4127355115 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f1d58004750 con 0x7f1d64103240 2024-03-15T12:53:50.777 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.788+0000 7f1d617fa700 1 --2- 172.21.15.184:0/4127355115 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1d4c0a1b40 0x7f1d4c0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:53:50.777 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.789+0000 7f1d617fa700 1 -- 172.21.15.184:0/4127355115 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f1d580cc890 con 0x7f1d64103240 2024-03-15T12:53:50.777 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.789+0000 7f1d63fff700 1 --2- 172.21.15.184:0/4127355115 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1d4c0a1b40 0x7f1d4c0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:53:50.777 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.789+0000 7f1d63fff700 1 --2- 172.21.15.184:0/4127355115 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1d4c0a1b40 0x7f1d4c0a4000 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d54006fd0 tx=0x7f1d54008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:53:50.779 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.792+0000 7f1d617fa700 1 -- 172.21.15.184:0/4127355115 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1d58096660 con 0x7f1d64103240 2024-03-15T12:53:50.981 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:53:50.993+0000 7f1d6a5e1700 1 -- 172.21.15.184:0/4127355115 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d6402da30 con 0x7f1d4c0a1b40 2024-03-15T12:53:51.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:51 smithi002 ceph-mon[32060]: osdmap e53: 8 total, 8 up, 8 in 2024-03-15T12:53:51.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:51 smithi002 ceph-mon[32060]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:51.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:53:51.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:53:51.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:51.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:51 smithi184 ceph-mon[35993]: osdmap e53: 8 total, 8 up, 8 in 2024-03-15T12:53:51.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:51 smithi184 ceph-mon[35993]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:51.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:53:51.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:53:51.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:51.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:51 smithi149 ceph-mon[35927]: osdmap e53: 8 total, 8 up, 8 in 2024-03-15T12:53:51.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:51 smithi149 ceph-mon[35927]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:51.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:53:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:53:51.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:53:52.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:52 smithi002 ceph-mon[32060]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:53:52.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:52 smithi002 ceph-mon[32060]: osdmap e54: 8 total, 8 up, 8 in 2024-03-15T12:53:52.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:52 smithi184 ceph-mon[35993]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:53:52.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:52 smithi184 ceph-mon[35993]: osdmap e54: 8 total, 8 up, 8 in 2024-03-15T12:53:52.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:52 smithi149 ceph-mon[35927]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:53:52.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:52 smithi149 ceph-mon[35927]: osdmap e54: 8 total, 8 up, 8 in 2024-03-15T12:53:53.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:53 smithi002 ceph-mon[32060]: osdmap e55: 8 total, 8 up, 8 in 2024-03-15T12:53:53.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:53 smithi002 ceph-mon[32060]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:53.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:53 smithi184 ceph-mon[35993]: osdmap e55: 8 total, 8 up, 8 in 2024-03-15T12:53:53.562 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:53 smithi184 ceph-mon[35993]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:53.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:53 smithi149 ceph-mon[35927]: osdmap e55: 8 total, 8 up, 8 in 2024-03-15T12:53:53.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:53 smithi149 ceph-mon[35927]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:54.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:54 smithi002 ceph-mon[32060]: from='client.? 172.21.15.184:0/176671053' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99a26e37-2f65-4775-80d0-e8388462928f"}]: dispatch 2024-03-15T12:53:54.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:54 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99a26e37-2f65-4775-80d0-e8388462928f"}]: dispatch 2024-03-15T12:53:54.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:54 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99a26e37-2f65-4775-80d0-e8388462928f"}]': finished 2024-03-15T12:53:54.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:54 smithi002 ceph-mon[32060]: osdmap e56: 9 total, 8 up, 9 in 2024-03-15T12:53:54.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:54 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:53:54.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:54 smithi184 ceph-mon[35993]: from='client.? 172.21.15.184:0/176671053' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99a26e37-2f65-4775-80d0-e8388462928f"}]: dispatch 2024-03-15T12:53:54.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:54 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99a26e37-2f65-4775-80d0-e8388462928f"}]: dispatch 2024-03-15T12:53:54.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:54 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99a26e37-2f65-4775-80d0-e8388462928f"}]': finished 2024-03-15T12:53:54.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:54 smithi184 ceph-mon[35993]: osdmap e56: 9 total, 8 up, 9 in 2024-03-15T12:53:54.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:54 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:53:54.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:54 smithi149 ceph-mon[35927]: from='client.? 172.21.15.184:0/176671053' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99a26e37-2f65-4775-80d0-e8388462928f"}]: dispatch 2024-03-15T12:53:54.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:54 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99a26e37-2f65-4775-80d0-e8388462928f"}]: dispatch 2024-03-15T12:53:54.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:54 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99a26e37-2f65-4775-80d0-e8388462928f"}]': finished 2024-03-15T12:53:54.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:54 smithi149 ceph-mon[35927]: osdmap e56: 9 total, 8 up, 9 in 2024-03-15T12:53:54.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:54 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:53:55.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:55 smithi184 ceph-mon[35993]: from='client.? 172.21.15.184:0/953073883' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:53:55.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:55 smithi184 ceph-mon[35993]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:55.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:55 smithi149 ceph-mon[35927]: from='client.? 172.21.15.184:0/953073883' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:53:55.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:55 smithi149 ceph-mon[35927]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:55.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:55 smithi002 ceph-mon[32060]: from='client.? 172.21.15.184:0/953073883' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:53:55.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:55 smithi002 ceph-mon[32060]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:57.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:56 smithi002 ceph-mon[32060]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:57.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:56 smithi184 ceph-mon[35993]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:57.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:56 smithi149 ceph-mon[35927]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:59.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:53:58 smithi002 ceph-mon[32060]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:53:58 smithi184 ceph-mon[35993]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:53:59.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:53:58 smithi149 ceph-mon[35927]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:01.016 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:00 smithi184 ceph-mon[35993]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:01.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:00 smithi002 ceph-mon[32060]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:01.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:00 smithi149 ceph-mon[35927]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:01.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:01 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-15T12:54:01.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:01 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:01.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:01 smithi184 ceph-mon[35993]: Deploying daemon osd.8 on smithi184 2024-03-15T12:54:02.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:01 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-15T12:54:02.084 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:01 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:02.085 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:01 smithi002 ceph-mon[32060]: Deploying daemon osd.8 on smithi184 2024-03-15T12:54:02.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:01 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-15T12:54:02.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:01 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:02.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:01 smithi149 ceph-mon[35927]: Deploying daemon osd.8 on smithi184 2024-03-15T12:54:03.030 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:02 smithi184 ceph-mon[35993]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:03.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:02 smithi002 ceph-mon[32060]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:03.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:02 smithi149 ceph-mon[35927]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:05.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:04 smithi002 ceph-mon[32060]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:05.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:04 smithi184 ceph-mon[35993]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:05.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:04 smithi149 ceph-mon[35927]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:07.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:06 smithi002 ceph-mon[32060]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:07.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:06 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:07.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:06 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:07.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:06 smithi184 ceph-mon[35993]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:07.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:06 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:07.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:06 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:07.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:06 smithi149 ceph-mon[35927]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:07.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:06 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:07.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:06 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:08.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:07 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:54:08.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:07 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:54:08.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:07 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:54:09.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:08 smithi002 ceph-mon[32060]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:09.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:08 smithi184 ceph-mon[35993]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:09.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:08 smithi149 ceph-mon[35927]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:10.001 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:54:10.015+0000 7f1d617fa700 1 -- 172.21.15.184:0/4127355115 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1d6402da30 con 0x7f1d4c0a1b40 2024-03-15T12:54:10.002 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 8 on host 'smithi184' 2024-03-15T12:54:10.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:54:10.017+0000 7f1d6a5e1700 1 -- 172.21.15.184:0/4127355115 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1d4c0a1b40 msgr2=0x7f1d4c0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:54:10.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:54:10.017+0000 7f1d6a5e1700 1 --2- 172.21.15.184:0/4127355115 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1d4c0a1b40 0x7f1d4c0a4000 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d54006fd0 tx=0x7f1d54008040 comp rx=0 tx=0).stop 2024-03-15T12:54:10.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:54:10.017+0000 7f1d6a5e1700 1 -- 172.21.15.184:0/4127355115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d64103240 msgr2=0x7f1d641985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:54:10.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:54:10.017+0000 7f1d6a5e1700 1 --2- 172.21.15.184:0/4127355115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d64103240 0x7f1d641985f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c950 tx=0x7f1d5800cd10 comp rx=0 tx=0).stop 2024-03-15T12:54:10.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:54:10.017+0000 7f1d6a5e1700 1 -- 172.21.15.184:0/4127355115 shutdown_connections 2024-03-15T12:54:10.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:54:10.017+0000 7f1d6a5e1700 1 --2- 172.21.15.184:0/4127355115 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1d4c0a1b40 0x7f1d4c0a4000 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:10.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:54:10.017+0000 7f1d6a5e1700 1 --2- 172.21.15.184:0/4127355115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d64103be0 0x7f1d6419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:10.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:54:10.017+0000 7f1d6a5e1700 1 --2- 172.21.15.184:0/4127355115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d64102040 0x7f1d641980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:10.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:54:10.017+0000 7f1d6a5e1700 1 --2- 172.21.15.184:0/4127355115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d64103240 0x7f1d641985f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:10.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:54:10.017+0000 7f1d6a5e1700 1 -- 172.21.15.184:0/4127355115 >> 172.21.15.184:0/4127355115 conn(0x7f1d640fd5e0 msgr2=0x7f1d64106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:54:10.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:54:10.017+0000 7f1d6a5e1700 1 -- 172.21.15.184:0/4127355115 shutdown_connections 2024-03-15T12:54:10.004 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:54:10.017+0000 7f1d6a5e1700 1 -- 172.21.15.184:0/4127355115 wait complete. 2024-03-15T12:54:10.887 DEBUG:teuthology.orchestra.run.smithi184:osd.8> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.8.service 2024-03-15T12:54:10.890 INFO:tasks.cephadm:Deploying osd.9 on smithi002 with /dev/vg_nvme/lv_1... 2024-03-15T12:54:10.890 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-15T12:54:11.155 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:54:11.277 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:10 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:11.277 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:10 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:11.277 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:10 smithi002 ceph-mon[32060]: from='osd.8 [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-15T12:54:11.277 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:10 smithi002 ceph-mon[32060]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-15T12:54:11.277 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:10 smithi002 ceph-mon[32060]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:11.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:10 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:11.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:10 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:11.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:10 smithi184 ceph-mon[35993]: from='osd.8 [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-15T12:54:11.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:10 smithi184 ceph-mon[35993]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-15T12:54:11.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:10 smithi184 ceph-mon[35993]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:11.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:10 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:11.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:10 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:11.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:10 smithi149 ceph-mon[35927]: from='osd.8 [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-15T12:54:11.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:10 smithi149 ceph-mon[35927]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-15T12:54:11.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:10 smithi149 ceph-mon[35927]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:12.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:12 smithi002 ceph-mon[32060]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-15T12:54:12.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:12 smithi002 ceph-mon[32060]: from='osd.8 [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:54:12.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:12 smithi002 ceph-mon[32060]: osdmap e57: 9 total, 8 up, 9 in 2024-03-15T12:54:12.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:12 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:54:12.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:12 smithi002 ceph-mon[32060]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:54:12.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:12 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:12.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:12 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:12.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:12 smithi184 ceph-mon[35993]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-15T12:54:12.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:12 smithi184 ceph-mon[35993]: from='osd.8 [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:54:12.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:12 smithi184 ceph-mon[35993]: osdmap e57: 9 total, 8 up, 9 in 2024-03-15T12:54:12.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:12 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:54:12.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:12 smithi184 ceph-mon[35993]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:54:12.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:12 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:12.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:12 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:12.310 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 12:54:12 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T12:54:12.004+0000 7f259c36c700 -1 osd.8 0 waiting for initial osdmap 2024-03-15T12:54:12.310 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 12:54:12 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T12:54:12.012+0000 7f259815f700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-15T12:54:12.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:12 smithi149 ceph-mon[35927]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-15T12:54:12.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:12 smithi149 ceph-mon[35927]: from='osd.8 [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:54:12.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:12 smithi149 ceph-mon[35927]: osdmap e57: 9 total, 8 up, 9 in 2024-03-15T12:54:12.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:12 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:54:12.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:12 smithi149 ceph-mon[35927]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:54:12.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:12 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:12.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:12 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:13.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:13 smithi002 ceph-mon[32060]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-03-15T12:54:13.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:13 smithi002 ceph-mon[32060]: osdmap e58: 9 total, 8 up, 9 in 2024-03-15T12:54:13.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:13 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:54:13.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:13 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:54:13.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:13 smithi002 ceph-mon[32060]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:13.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:13 smithi184 ceph-mon[35993]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-03-15T12:54:13.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:13 smithi184 ceph-mon[35993]: osdmap e58: 9 total, 8 up, 9 in 2024-03-15T12:54:13.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:13 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:54:13.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:13 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:54:13.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:13 smithi184 ceph-mon[35993]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:13.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:13 smithi149 ceph-mon[35927]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-03-15T12:54:13.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:13 smithi149 ceph-mon[35927]: osdmap e58: 9 total, 8 up, 9 in 2024-03-15T12:54:13.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:13 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:54:13.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:13 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:54:13.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:13 smithi149 ceph-mon[35927]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-03-15T12:54:13.475 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-15T12:54:13.475 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-15T12:54:13.475 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2024-03-15T12:54:13.475 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2024-03-15T12:54:13.475 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110359 s, 950 MB/s 2024-03-15T12:54:13.476 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2024-03-15T12:54:14.265 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:14 smithi002 ceph-mon[32060]: purged_snaps scrub starts 2024-03-15T12:54:14.265 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:14 smithi002 ceph-mon[32060]: purged_snaps scrub ok 2024-03-15T12:54:14.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:14 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:54:14.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:14 smithi002 ceph-mon[32060]: osd.8 [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579] boot 2024-03-15T12:54:14.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:14 smithi002 ceph-mon[32060]: osdmap e59: 9 total, 9 up, 9 in 2024-03-15T12:54:14.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:14 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:54:14.293 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi002:vg_nvme/lv_1 2024-03-15T12:54:14.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:14 smithi184 ceph-mon[35993]: purged_snaps scrub starts 2024-03-15T12:54:14.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:14 smithi184 ceph-mon[35993]: purged_snaps scrub ok 2024-03-15T12:54:14.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:14 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:54:14.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:14 smithi184 ceph-mon[35993]: osd.8 [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579] boot 2024-03-15T12:54:14.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:14 smithi184 ceph-mon[35993]: osdmap e59: 9 total, 9 up, 9 in 2024-03-15T12:54:14.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:14 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:54:14.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:14 smithi149 ceph-mon[35927]: purged_snaps scrub starts 2024-03-15T12:54:14.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:14 smithi149 ceph-mon[35927]: purged_snaps scrub ok 2024-03-15T12:54:14.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:14 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:54:14.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:14 smithi149 ceph-mon[35927]: osd.8 [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579] boot 2024-03-15T12:54:14.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:14 smithi149 ceph-mon[35927]: osdmap e59: 9 total, 9 up, 9 in 2024-03-15T12:54:14.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:14 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:54:14.539 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:54:15.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: osdmap e60: 9 total, 9 up, 9 in 2024-03-15T12:54:15.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: Detected new or changed devices on smithi184 2024-03-15T12:54:15.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:15.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:15.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:15.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:15.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:15.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi184 to 5696M 2024-03-15T12:54:15.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:15.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:15.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:54:15.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:15.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:54:15.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:15.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:54:15.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:15.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:15 smithi002 ceph-mon[32060]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:15.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: osdmap e60: 9 total, 9 up, 9 in 2024-03-15T12:54:15.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: Detected new or changed devices on smithi184 2024-03-15T12:54:15.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:15.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi184 to 5696M 2024-03-15T12:54:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:54:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:54:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:54:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:15.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:15 smithi184 ceph-mon[35993]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:15.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: osdmap e60: 9 total, 9 up, 9 in 2024-03-15T12:54:15.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: Detected new or changed devices on smithi184 2024-03-15T12:54:15.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:15.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:15.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:15.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:15.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:15.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi184 to 5696M 2024-03-15T12:54:15.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:15.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:15.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:54:15.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:15.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:54:15.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:15.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:54:15.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:15.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:15 smithi149 ceph-mon[35927]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:16.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.180+0000 7f983cad2700 1 -- 172.21.15.2:0/4143860832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9838101200 msgr2=0x7f9838101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:54:16.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.180+0000 7f983cad2700 1 --2- 172.21.15.2:0/4143860832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9838101200 0x7f9838101660 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f982c009990 tx=0x7f982c009ca0 comp rx=0 tx=0).stop 2024-03-15T12:54:16.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.180+0000 7f983cad2700 1 -- 172.21.15.2:0/4143860832 shutdown_connections 2024-03-15T12:54:16.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.180+0000 7f983cad2700 1 --2- 172.21.15.2:0/4143860832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9838101ba0 0x7f9838109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:16.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.180+0000 7f983cad2700 1 --2- 172.21.15.2:0/4143860832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98381024c0 0x7f98381028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:16.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.180+0000 7f983cad2700 1 --2- 172.21.15.2:0/4143860832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9838101200 0x7f9838101660 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:16.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.180+0000 7f983cad2700 1 -- 172.21.15.2:0/4143860832 >> 172.21.15.2:0/4143860832 conn(0x7f98380fbc70 msgr2=0x7f98380fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:54:16.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.180+0000 7f983cad2700 1 -- 172.21.15.2:0/4143860832 shutdown_connections 2024-03-15T12:54:16.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.180+0000 7f983cad2700 1 -- 172.21.15.2:0/4143860832 wait complete. 2024-03-15T12:54:16.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.181+0000 7f983cad2700 1 Processor -- start 2024-03-15T12:54:16.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.181+0000 7f983cad2700 1 -- start start 2024-03-15T12:54:16.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.181+0000 7f983cad2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9838101200 0x7f9838198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:54:16.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.181+0000 7f983cad2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9838101ba0 0x7f9838198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:54:16.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.181+0000 7f983cad2700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98381024c0 0x7f983819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:54:16.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.181+0000 7f983cad2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983819fbc0 con 0x7f9838101200 2024-03-15T12:54:16.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.181+0000 7f983cad2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983819fd00 con 0x7f9838101ba0 2024-03-15T12:54:16.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.181+0000 7f983cad2700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983819fe90 con 0x7f98381024c0 2024-03-15T12:54:16.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.181+0000 7f983659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9838101200 0x7f9838198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:54:16.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.181+0000 7f983659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9838101200 0x7f9838198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38306/0 (socket says 172.21.15.2:38306) 2024-03-15T12:54:16.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.181+0000 7f983659c700 1 -- 172.21.15.2:0/2891688352 learned_addr learned my addr 172.21.15.2:0/2891688352 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:54:16.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.182+0000 7f9835d9b700 1 --2- 172.21.15.2:0/2891688352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9838101ba0 0x7f9838198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:54:16.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.182+0000 7f9836d9d700 1 --2- 172.21.15.2:0/2891688352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98381024c0 0x7f983819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:54:16.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.182+0000 7f983659c700 1 -- 172.21.15.2:0/2891688352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98381024c0 msgr2=0x7f983819f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:54:16.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.182+0000 7f983659c700 1 --2- 172.21.15.2:0/2891688352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98381024c0 0x7f983819f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:16.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.182+0000 7f983659c700 1 -- 172.21.15.2:0/2891688352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9838101ba0 msgr2=0x7f9838198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:54:16.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.182+0000 7f983659c700 1 --2- 172.21.15.2:0/2891688352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9838101ba0 0x7f9838198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:16.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.182+0000 7f983659c700 1 -- 172.21.15.2:0/2891688352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c0095f0 con 0x7f9838101200 2024-03-15T12:54:16.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.182+0000 7f983659c700 1 --2- 172.21.15.2:0/2891688352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9838101200 0x7f9838198030 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f982800b530 tx=0x7f982800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:54:16.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.182+0000 7f98277fe700 1 -- 172.21.15.2:0/2891688352 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828011830 con 0x7f9838101200 2024-03-15T12:54:16.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.182+0000 7f983cad2700 1 -- 172.21.15.2:0/2891688352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98381a0170 con 0x7f9838101200 2024-03-15T12:54:16.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.182+0000 7f98277fe700 1 -- 172.21.15.2:0/2891688352 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9828011e60 con 0x7f9838101200 2024-03-15T12:54:16.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.182+0000 7f98277fe700 1 -- 172.21.15.2:0/2891688352 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828010e60 con 0x7f9838101200 2024-03-15T12:54:16.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.182+0000 7f983cad2700 1 -- 172.21.15.2:0/2891688352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98381a06f0 con 0x7f9838101200 2024-03-15T12:54:16.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.183+0000 7f983cad2700 1 -- 172.21.15.2:0/2891688352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98381083d0 con 0x7f9838101200 2024-03-15T12:54:16.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.185+0000 7f98277fe700 1 -- 172.21.15.2:0/2891688352 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f9828011990 con 0x7f9838101200 2024-03-15T12:54:16.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.185+0000 7f98277fe700 1 --2- 172.21.15.2:0/2891688352 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f98200a1ac0 0x7f98200a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:54:16.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.185+0000 7f9835d9b700 1 --2- 172.21.15.2:0/2891688352 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f98200a1ac0 0x7f98200a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:54:16.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.185+0000 7f98277fe700 1 -- 172.21.15.2:0/2891688352 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f98280cc820 con 0x7f9838101200 2024-03-15T12:54:16.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.186+0000 7f9835d9b700 1 --2- 172.21.15.2:0/2891688352 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f98200a1ac0 0x7f98200a3f80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f982c00b5c0 tx=0x7f982c0038c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:54:16.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.189+0000 7f98277fe700 1 -- 172.21.15.2:0/2891688352 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f98280963c0 con 0x7f9838101200 2024-03-15T12:54:16.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:16 smithi002 ceph-mon[32060]: osdmap e61: 9 total, 9 up, 9 in 2024-03-15T12:54:16.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:16 smithi184 ceph-mon[35993]: osdmap e61: 9 total, 9 up, 9 in 2024-03-15T12:54:16.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:16 smithi149 ceph-mon[35927]: osdmap e61: 9 total, 9 up, 9 in 2024-03-15T12:54:16.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:16.405+0000 7f983cad2700 1 -- 172.21.15.2:0/2891688352 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f983802d990 con 0x7f98200a1ac0 2024-03-15T12:54:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:17 smithi002 ceph-mon[32060]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:54:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:54:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:54:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:17 smithi002 ceph-mon[32060]: osdmap e62: 9 total, 9 up, 9 in 2024-03-15T12:54:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:17 smithi002 ceph-mon[32060]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:17.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:17 smithi184 ceph-mon[35993]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:54:17.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:54:17.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:54:17.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:17.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:17 smithi184 ceph-mon[35993]: osdmap e62: 9 total, 9 up, 9 in 2024-03-15T12:54:17.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:17 smithi184 ceph-mon[35993]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:17.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:17 smithi149 ceph-mon[35927]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:54:17.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:54:17.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:54:17.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:17.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:17 smithi149 ceph-mon[35927]: osdmap e62: 9 total, 9 up, 9 in 2024-03-15T12:54:17.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:17 smithi149 ceph-mon[35927]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:18.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:18 smithi002 ceph-mon[32060]: osdmap e63: 9 total, 9 up, 9 in 2024-03-15T12:54:19.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:18 smithi184 ceph-mon[35993]: osdmap e63: 9 total, 9 up, 9 in 2024-03-15T12:54:19.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:18 smithi149 ceph-mon[35927]: osdmap e63: 9 total, 9 up, 9 in 2024-03-15T12:54:19.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:19 smithi002 ceph-mon[32060]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-15T12:54:19.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:19 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/77245228' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b52d0d0-43fa-41ac-a775-722ff5480e45"}]: dispatch 2024-03-15T12:54:19.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:19 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/77245228' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b52d0d0-43fa-41ac-a775-722ff5480e45"}]': finished 2024-03-15T12:54:19.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:19 smithi002 ceph-mon[32060]: osdmap e64: 10 total, 9 up, 10 in 2024-03-15T12:54:19.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:19 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:20.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:19 smithi184 ceph-mon[35993]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-15T12:54:20.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:19 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/77245228' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b52d0d0-43fa-41ac-a775-722ff5480e45"}]: dispatch 2024-03-15T12:54:20.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:19 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/77245228' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b52d0d0-43fa-41ac-a775-722ff5480e45"}]': finished 2024-03-15T12:54:20.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:19 smithi184 ceph-mon[35993]: osdmap e64: 10 total, 9 up, 10 in 2024-03-15T12:54:20.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:19 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:20.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:19 smithi149 ceph-mon[35927]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-15T12:54:20.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:19 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/77245228' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b52d0d0-43fa-41ac-a775-722ff5480e45"}]: dispatch 2024-03-15T12:54:20.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:19 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/77245228' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b52d0d0-43fa-41ac-a775-722ff5480e45"}]': finished 2024-03-15T12:54:20.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:19 smithi149 ceph-mon[35927]: osdmap e64: 10 total, 9 up, 10 in 2024-03-15T12:54:20.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:21.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:20 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2538860895' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:54:21.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:20 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2538860895' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:54:21.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:20 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2538860895' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:54:22.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:21 smithi002 ceph-mon[32060]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-15T12:54:22.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:21 smithi184 ceph-mon[35993]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-15T12:54:22.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:21 smithi149 ceph-mon[35927]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-15T12:54:23.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:22 smithi002 ceph-mon[32060]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-15T12:54:23.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:22 smithi184 ceph-mon[35993]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-15T12:54:23.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:22 smithi149 ceph-mon[35927]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-15T12:54:25.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:24 smithi002 ceph-mon[32060]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-15T12:54:25.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:24 smithi184 ceph-mon[35993]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-15T12:54:25.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:24 smithi149 ceph-mon[35927]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-15T12:54:26.869 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:26 smithi002 ceph-mon[32060]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-15T12:54:27.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:26 smithi184 ceph-mon[35993]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-15T12:54:27.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:26 smithi149 ceph-mon[35927]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-15T12:54:27.776 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-15T12:54:27.776 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:27.776 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:27 smithi002 ceph-mon[32060]: Deploying daemon osd.9 on smithi002 2024-03-15T12:54:28.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-15T12:54:28.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:28.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:27 smithi184 ceph-mon[35993]: Deploying daemon osd.9 on smithi002 2024-03-15T12:54:28.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-15T12:54:28.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:28.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:27 smithi149 ceph-mon[35927]: Deploying daemon osd.9 on smithi002 2024-03-15T12:54:28.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:28 smithi002 ceph-mon[32060]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:29.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:28 smithi184 ceph-mon[35993]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:29.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:28 smithi149 ceph-mon[35927]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:31.033 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:30 smithi002 ceph-mon[32060]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:31.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:30 smithi184 ceph-mon[35993]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:31.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:30 smithi149 ceph-mon[35927]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:32.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:32 smithi002 ceph-mon[32060]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:33.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:32 smithi184 ceph-mon[35993]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:33.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:32 smithi149 ceph-mon[35927]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:34.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:34.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:54:34.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:33 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:34.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:34.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:54:34.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:33 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:34.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:34.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:54:34.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:33 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:35.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:34 smithi149 ceph-mon[35927]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:35.165 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:34 smithi002 ceph-mon[32060]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:35.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:34 smithi184 ceph-mon[35993]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:36.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:36.915+0000 7f98277fe700 1 -- 172.21.15.2:0/2891688352 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f983802d990 con 0x7f98200a1ac0 2024-03-15T12:54:36.902 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 9 on host 'smithi002' 2024-03-15T12:54:36.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:36.917+0000 7f983cad2700 1 -- 172.21.15.2:0/2891688352 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f98200a1ac0 msgr2=0x7f98200a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:54:36.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:36.917+0000 7f983cad2700 1 --2- 172.21.15.2:0/2891688352 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f98200a1ac0 0x7f98200a3f80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f982c00b5c0 tx=0x7f982c0038c0 comp rx=0 tx=0).stop 2024-03-15T12:54:36.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:36.918+0000 7f983cad2700 1 -- 172.21.15.2:0/2891688352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9838101200 msgr2=0x7f9838198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:54:36.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:36.918+0000 7f983cad2700 1 --2- 172.21.15.2:0/2891688352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9838101200 0x7f9838198030 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f982800b530 tx=0x7f982800b840 comp rx=0 tx=0).stop 2024-03-15T12:54:36.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:36.918+0000 7f983cad2700 1 -- 172.21.15.2:0/2891688352 shutdown_connections 2024-03-15T12:54:36.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:36.918+0000 7f983cad2700 1 --2- 172.21.15.2:0/2891688352 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f98200a1ac0 0x7f98200a3f80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:36.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:36.918+0000 7f983cad2700 1 --2- 172.21.15.2:0/2891688352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9838101200 0x7f9838198030 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:36.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:36.918+0000 7f983cad2700 1 --2- 172.21.15.2:0/2891688352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98381024c0 0x7f983819f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:36.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:36.918+0000 7f983cad2700 1 --2- 172.21.15.2:0/2891688352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9838101ba0 0x7f9838198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:36.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:36.918+0000 7f983cad2700 1 -- 172.21.15.2:0/2891688352 >> 172.21.15.2:0/2891688352 conn(0x7f98380fbc70 msgr2=0x7f9838105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:54:36.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:36.918+0000 7f983cad2700 1 -- 172.21.15.2:0/2891688352 shutdown_connections 2024-03-15T12:54:36.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:54:36.918+0000 7f983cad2700 1 -- 172.21.15.2:0/2891688352 wait complete. 2024-03-15T12:54:37.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:36 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:37.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:36 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:37.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:36 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:37.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:36 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:54:37.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:36 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:37.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:36 smithi002 ceph-mon[32060]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:37.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:36 smithi002 ceph-mon[32060]: from='osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-15T12:54:37.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:36 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:37.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:36 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:37.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:36 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:54:37.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:36 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:37.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:36 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:37.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:36 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:37.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:36 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:54:37.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:36 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:37.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:36 smithi184 ceph-mon[35993]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:37.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:36 smithi184 ceph-mon[35993]: from='osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-15T12:54:37.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:36 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:37.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:36 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:37.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:36 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:54:37.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:36 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:37.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:36 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:37.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:36 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:37.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:36 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:54:37.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:36 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:37.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:36 smithi149 ceph-mon[35927]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:37.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:36 smithi149 ceph-mon[35927]: from='osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-15T12:54:37.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:36 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:37.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:36 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:37.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:36 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:54:37.590 DEBUG:teuthology.orchestra.run.smithi002:osd.9> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.9.service 2024-03-15T12:54:37.593 INFO:tasks.cephadm:Deploying osd.10 on smithi149 with /dev/vg_nvme/lv_1... 2024-03-15T12:54:37.593 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-15T12:54:37.836 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.b/config 2024-03-15T12:54:38.102 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:38 smithi002 ceph-mon[32060]: from='osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-15T12:54:38.102 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:38 smithi002 ceph-mon[32060]: osdmap e65: 10 total, 9 up, 10 in 2024-03-15T12:54:38.102 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:38 smithi002 ceph-mon[32060]: from='osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-15T12:54:38.102 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:38 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:38.263 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:38 smithi149 ceph-mon[35927]: from='osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-15T12:54:38.263 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:38 smithi149 ceph-mon[35927]: osdmap e65: 10 total, 9 up, 10 in 2024-03-15T12:54:38.263 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:38 smithi149 ceph-mon[35927]: from='osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-15T12:54:38.263 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:38 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:38.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:38 smithi184 ceph-mon[35993]: from='osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-15T12:54:38.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:38 smithi184 ceph-mon[35993]: osdmap e65: 10 total, 9 up, 10 in 2024-03-15T12:54:38.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:38 smithi184 ceph-mon[35993]: from='osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-15T12:54:38.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:38 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:38.874 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 12:54:38 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T12:54:38.567+0000 7fd12b706700 -1 osd.9 0 waiting for initial osdmap 2024-03-15T12:54:38.874 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 12:54:38 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T12:54:38.580+0000 7fd1234f1700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-15T12:54:39.228 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:39 smithi149 ceph-mon[35927]: from='osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-15T12:54:39.228 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:39 smithi149 ceph-mon[35927]: osdmap e66: 10 total, 9 up, 10 in 2024-03-15T12:54:39.229 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:39 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:39.229 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:39 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:39.229 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:39 smithi149 ceph-mon[35927]: from='osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413]' entity='osd.9' 2024-03-15T12:54:39.229 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:39 smithi149 ceph-mon[35927]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:39.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:39 smithi002 ceph-mon[32060]: from='osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-15T12:54:39.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:39 smithi002 ceph-mon[32060]: osdmap e66: 10 total, 9 up, 10 in 2024-03-15T12:54:39.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:39 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:39.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:39 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:39.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:39 smithi002 ceph-mon[32060]: from='osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413]' entity='osd.9' 2024-03-15T12:54:39.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:39 smithi002 ceph-mon[32060]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:39.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:39 smithi184 ceph-mon[35993]: from='osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-15T12:54:39.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:39 smithi184 ceph-mon[35993]: osdmap e66: 10 total, 9 up, 10 in 2024-03-15T12:54:39.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:39 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:39.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:39 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:39.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:39 smithi184 ceph-mon[35993]: from='osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413]' entity='osd.9' 2024-03-15T12:54:39.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:39 smithi184 ceph-mon[35993]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 804 GiB / 805 GiB avail 2024-03-15T12:54:39.781 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-15T12:54:39.782 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-15T12:54:39.782 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-15T12:54:39.782 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-15T12:54:39.782 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123844 s, 847 MB/s 2024-03-15T12:54:39.782 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-15T12:54:40.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:40 smithi002 ceph-mon[32060]: purged_snaps scrub starts 2024-03-15T12:54:40.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:40 smithi002 ceph-mon[32060]: purged_snaps scrub ok 2024-03-15T12:54:40.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:40 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:40.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:40 smithi002 ceph-mon[32060]: osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413] boot 2024-03-15T12:54:40.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:40 smithi002 ceph-mon[32060]: osdmap e67: 10 total, 10 up, 10 in 2024-03-15T12:54:40.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:40 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:40.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:40 smithi184 ceph-mon[35993]: purged_snaps scrub starts 2024-03-15T12:54:40.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:40 smithi184 ceph-mon[35993]: purged_snaps scrub ok 2024-03-15T12:54:40.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:40 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:40.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:40 smithi184 ceph-mon[35993]: osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413] boot 2024-03-15T12:54:40.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:40 smithi184 ceph-mon[35993]: osdmap e67: 10 total, 10 up, 10 in 2024-03-15T12:54:40.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:40 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:40.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:40 smithi149 ceph-mon[35927]: purged_snaps scrub starts 2024-03-15T12:54:40.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:40 smithi149 ceph-mon[35927]: purged_snaps scrub ok 2024-03-15T12:54:40.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:40 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:40.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:40 smithi149 ceph-mon[35927]: osd.9 [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413] boot 2024-03-15T12:54:40.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:40 smithi149 ceph-mon[35927]: osdmap e67: 10 total, 10 up, 10 in 2024-03-15T12:54:40.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:40 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:54:40.462 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_1 2024-03-15T12:54:40.701 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.b/config 2024-03-15T12:54:41.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:41 smithi149 ceph-mon[35927]: osdmap e68: 10 total, 10 up, 10 in 2024-03-15T12:54:41.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:41 smithi149 ceph-mon[35927]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:41.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:41.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:41.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:41.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:41.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:41.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:41.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:41.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:41.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:54:41.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:41 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:42.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:41 smithi002 ceph-mon[32060]: osdmap e68: 10 total, 10 up, 10 in 2024-03-15T12:54:42.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:41 smithi002 ceph-mon[32060]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:42.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:42.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:42.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:42.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:42.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:42.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:42.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:42.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:42.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:54:42.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:41 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:42.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:41 smithi184 ceph-mon[35993]: osdmap e68: 10 total, 10 up, 10 in 2024-03-15T12:54:42.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:41 smithi184 ceph-mon[35993]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:42.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:42.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:42.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:42.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:42.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:42.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:54:42.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:42.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:42.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:54:42.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:41 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:42.308 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.321+0000 7f9387736700 1 -- 172.21.15.149:0/2771839239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380103230 msgr2=0x7f9380103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:54:42.308 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.321+0000 7f9387736700 1 --2- 172.21.15.149:0/2771839239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380103230 0x7f9380103690 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f937c0099c0 tx=0x7f937c009cd0 comp rx=0 tx=0).stop 2024-03-15T12:54:42.308 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.321+0000 7f9387736700 1 -- 172.21.15.149:0/2771839239 shutdown_connections 2024-03-15T12:54:42.308 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.321+0000 7f9387736700 1 --2- 172.21.15.149:0/2771839239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9380102030 0x7f9380102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:42.308 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.321+0000 7f9387736700 1 --2- 172.21.15.149:0/2771839239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9380103bd0 0x7f938010a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:42.308 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.321+0000 7f9387736700 1 --2- 172.21.15.149:0/2771839239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380103230 0x7f9380103690 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:42.308 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.321+0000 7f9387736700 1 -- 172.21.15.149:0/2771839239 >> 172.21.15.149:0/2771839239 conn(0x7f93800fd590 msgr2=0x7f93800ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:54:42.308 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.321+0000 7f9387736700 1 -- 172.21.15.149:0/2771839239 shutdown_connections 2024-03-15T12:54:42.308 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.321+0000 7f9387736700 1 -- 172.21.15.149:0/2771839239 wait complete. 2024-03-15T12:54:42.311 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9387736700 1 Processor -- start 2024-03-15T12:54:42.311 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9387736700 1 -- start start 2024-03-15T12:54:42.311 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9387736700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9380102030 0x7f938019a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:54:42.311 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9387736700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9380103230 0x7f938019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:54:42.311 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9387736700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380103bd0 0x7f93801a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:54:42.311 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9387736700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93801a1cd0 con 0x7f9380103230 2024-03-15T12:54:42.311 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9387736700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93801a1e10 con 0x7f9380103bd0 2024-03-15T12:54:42.312 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9387736700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93801a1fa0 con 0x7f9380102030 2024-03-15T12:54:42.312 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9385cd3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380103bd0 0x7f93801a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:54:42.312 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9385cd3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380103bd0 0x7f93801a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:41984/0 (socket says 172.21.15.149:41984) 2024-03-15T12:54:42.312 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9385cd3700 1 -- 172.21.15.149:0/2031030274 learned_addr learned my addr 172.21.15.149:0/2031030274 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-15T12:54:42.312 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9384cd1700 1 --2- 172.21.15.149:0/2031030274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9380103230 0x7f938019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:54:42.312 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f93854d2700 1 --2- 172.21.15.149:0/2031030274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9380102030 0x7f938019a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:54:42.312 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9385cd3700 1 -- 172.21.15.149:0/2031030274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9380102030 msgr2=0x7f938019a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:54:42.312 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9385cd3700 1 --2- 172.21.15.149:0/2031030274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9380102030 0x7f938019a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:42.312 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9385cd3700 1 -- 172.21.15.149:0/2031030274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9380103230 msgr2=0x7f938019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:54:42.312 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9385cd3700 1 --2- 172.21.15.149:0/2031030274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9380103230 0x7f938019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:54:42.312 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f9385cd3700 1 -- 172.21.15.149:0/2031030274 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c0095f0 con 0x7f9380103bd0 2024-03-15T12:54:42.312 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.322+0000 7f93854d2700 1 --2- 172.21.15.149:0/2031030274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9380102030 0x7f938019a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T12:54:42.312 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.323+0000 7f9385cd3700 1 --2- 172.21.15.149:0/2031030274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380103bd0 0x7f93801a1700 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9378007ee0 tx=0x7f937800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:54:42.313 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.323+0000 7f93767fc700 1 -- 172.21.15.149:0/2031030274 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378017070 con 0x7f9380103bd0 2024-03-15T12:54:42.313 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.323+0000 7f93767fc700 1 -- 172.21.15.149:0/2031030274 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f937800f040 con 0x7f9380103bd0 2024-03-15T12:54:42.313 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.323+0000 7f9387736700 1 -- 172.21.15.149:0/2031030274 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93801a2280 con 0x7f9380103bd0 2024-03-15T12:54:42.313 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.323+0000 7f93767fc700 1 -- 172.21.15.149:0/2031030274 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378005750 con 0x7f9380103bd0 2024-03-15T12:54:42.313 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.323+0000 7f9387736700 1 -- 172.21.15.149:0/2031030274 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93801a27d0 con 0x7f9380103bd0 2024-03-15T12:54:42.313 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.324+0000 7f9387736700 1 -- 172.21.15.149:0/2031030274 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9380068ca0 con 0x7f9380103bd0 2024-03-15T12:54:42.313 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.325+0000 7f93767fc700 1 -- 172.21.15.149:0/2031030274 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f93780058b0 con 0x7f9380103bd0 2024-03-15T12:54:42.314 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.325+0000 7f93767fc700 1 --2- 172.21.15.149:0/2031030274 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f936c0a1b40 0x7f936c0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:54:42.314 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.326+0000 7f93767fc700 1 -- 172.21.15.149:0/2031030274 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f93780cc980 con 0x7f9380103bd0 2024-03-15T12:54:42.314 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.326+0000 7f93854d2700 1 --2- 172.21.15.149:0/2031030274 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f936c0a1b40 0x7f936c0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:54:42.314 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.326+0000 7f93854d2700 1 --2- 172.21.15.149:0/2031030274 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f936c0a1b40 0x7f936c0a4000 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9370005fd0 tx=0x7f9370005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:54:42.315 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.329+0000 7f93767fc700 1 -- 172.21.15.149:0/2031030274 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9378096390 con 0x7f9380103bd0 2024-03-15T12:54:42.518 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:54:42.532+0000 7f9387736700 1 -- 172.21.15.149:0/2031030274 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9380075930 con 0x7f936c0a1b40 2024-03-15T12:54:42.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:42 smithi149 ceph-mon[35927]: Detected new or changed devices on smithi002 2024-03-15T12:54:42.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:42 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi002 to 4273M 2024-03-15T12:54:42.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:42 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:54:42.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:42 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:54:42.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:42 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:43.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:42 smithi002 ceph-mon[32060]: Detected new or changed devices on smithi002 2024-03-15T12:54:43.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:42 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi002 to 4273M 2024-03-15T12:54:43.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:42 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:54:43.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:42 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:54:43.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:42 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:43.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:42 smithi184 ceph-mon[35993]: Detected new or changed devices on smithi002 2024-03-15T12:54:43.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:42 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi002 to 4273M 2024-03-15T12:54:43.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:42 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:54:43.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:42 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:54:43.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:42 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:43.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:43 smithi002 ceph-mon[32060]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:54:43.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:43 smithi002 ceph-mon[32060]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:43.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:43 smithi149 ceph-mon[35927]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:54:43.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:43 smithi149 ceph-mon[35927]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:44.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:43 smithi184 ceph-mon[35993]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:54:44.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:43 smithi184 ceph-mon[35993]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:45.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:44 smithi002 ceph-mon[32060]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:45.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:44 smithi184 ceph-mon[35993]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:45.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:44 smithi149 ceph-mon[35927]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:45.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:45 smithi149 ceph-mon[35927]: from='client.? 172.21.15.149:0/981318560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "792088a5-75ff-44b2-b1b0-09a7a0b84554"}]: dispatch 2024-03-15T12:54:45.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:45 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "792088a5-75ff-44b2-b1b0-09a7a0b84554"}]: dispatch 2024-03-15T12:54:45.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:45 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "792088a5-75ff-44b2-b1b0-09a7a0b84554"}]': finished 2024-03-15T12:54:45.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:45 smithi149 ceph-mon[35927]: osdmap e69: 11 total, 10 up, 11 in 2024-03-15T12:54:45.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:45 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:54:46.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:45 smithi002 ceph-mon[32060]: from='client.? 172.21.15.149:0/981318560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "792088a5-75ff-44b2-b1b0-09a7a0b84554"}]: dispatch 2024-03-15T12:54:46.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:45 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "792088a5-75ff-44b2-b1b0-09a7a0b84554"}]: dispatch 2024-03-15T12:54:46.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:45 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "792088a5-75ff-44b2-b1b0-09a7a0b84554"}]': finished 2024-03-15T12:54:46.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:45 smithi002 ceph-mon[32060]: osdmap e69: 11 total, 10 up, 11 in 2024-03-15T12:54:46.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:45 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:54:46.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:45 smithi184 ceph-mon[35993]: from='client.? 172.21.15.149:0/981318560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "792088a5-75ff-44b2-b1b0-09a7a0b84554"}]: dispatch 2024-03-15T12:54:46.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:45 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "792088a5-75ff-44b2-b1b0-09a7a0b84554"}]: dispatch 2024-03-15T12:54:46.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:45 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "792088a5-75ff-44b2-b1b0-09a7a0b84554"}]': finished 2024-03-15T12:54:46.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:45 smithi184 ceph-mon[35993]: osdmap e69: 11 total, 10 up, 11 in 2024-03-15T12:54:46.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:45 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:54:47.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:46 smithi002 ceph-mon[32060]: from='client.? 172.21.15.149:0/2319213658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:54:47.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:46 smithi002 ceph-mon[32060]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:47.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:46 smithi184 ceph-mon[35993]: from='client.? 172.21.15.149:0/2319213658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:54:47.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:46 smithi184 ceph-mon[35993]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:46 smithi149 ceph-mon[35927]: from='client.? 172.21.15.149:0/2319213658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:54:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:46 smithi149 ceph-mon[35927]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:49.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:48 smithi002 ceph-mon[32060]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:49.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:48 smithi184 ceph-mon[35993]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:49.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:48 smithi149 ceph-mon[35927]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:50.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:50 smithi149 ceph-mon[35927]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:51.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:50 smithi002 ceph-mon[32060]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:51.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:50 smithi184 ceph-mon[35993]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:52.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:52 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-15T12:54:52.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:52 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:52.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:52 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-15T12:54:52.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:52 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:52.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:52 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-15T12:54:52.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:52 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:54:53.724 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:53 smithi149 ceph-mon[35927]: Deploying daemon osd.10 on smithi149 2024-03-15T12:54:53.724 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:53 smithi149 ceph-mon[35927]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:53.782 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:53 smithi002 ceph-mon[32060]: Deploying daemon osd.10 on smithi149 2024-03-15T12:54:53.782 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:53 smithi002 ceph-mon[32060]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:53.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:53 smithi184 ceph-mon[35993]: Deploying daemon osd.10 on smithi149 2024-03-15T12:54:53.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:53 smithi184 ceph-mon[35993]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:55.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:54 smithi002 ceph-mon[32060]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:55.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:54 smithi184 ceph-mon[35993]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:55.065 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:54 smithi149 ceph-mon[35927]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:57.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:56 smithi002 ceph-mon[32060]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:57.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:56 smithi184 ceph-mon[35993]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:57.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:56 smithi149 ceph-mon[35927]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:59.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:59.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:59.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:59 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:54:59.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:54:59 smithi149 ceph-mon[35927]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:59.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:59.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:59.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:59 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:54:59.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:54:59 smithi002 ceph-mon[32060]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:54:59.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:59.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:54:59.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:59 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:54:59.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:54:59 smithi184 ceph-mon[35993]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:55:00.892 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:00 smithi149 ceph-mon[35927]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:55:01.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:00 smithi002 ceph-mon[32060]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:55:01.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:00 smithi184 ceph-mon[35993]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:55:01.936 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 10 on host 'smithi149' 2024-03-15T12:55:01.937 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:55:01.949+0000 7f93767fc700 1 -- 172.21.15.149:0/2031030274 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f9380075930 con 0x7f936c0a1b40 2024-03-15T12:55:01.940 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:55:01.953+0000 7f9387736700 1 -- 172.21.15.149:0/2031030274 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f936c0a1b40 msgr2=0x7f936c0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:01.940 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:55:01.953+0000 7f9387736700 1 --2- 172.21.15.149:0/2031030274 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f936c0a1b40 0x7f936c0a4000 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9370005fd0 tx=0x7f9370005f00 comp rx=0 tx=0).stop 2024-03-15T12:55:01.940 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:55:01.954+0000 7f9387736700 1 -- 172.21.15.149:0/2031030274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380103bd0 msgr2=0x7f93801a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:01.940 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:55:01.954+0000 7f9387736700 1 --2- 172.21.15.149:0/2031030274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380103bd0 0x7f93801a1700 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9378007ee0 tx=0x7f937800d3b0 comp rx=0 tx=0).stop 2024-03-15T12:55:01.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:55:01.954+0000 7f9387736700 1 -- 172.21.15.149:0/2031030274 shutdown_connections 2024-03-15T12:55:01.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:55:01.954+0000 7f9387736700 1 --2- 172.21.15.149:0/2031030274 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f936c0a1b40 0x7f936c0a4000 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:01.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:55:01.954+0000 7f9387736700 1 --2- 172.21.15.149:0/2031030274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380103bd0 0x7f93801a1700 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:01.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:55:01.954+0000 7f9387736700 1 --2- 172.21.15.149:0/2031030274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9380102030 0x7f938019a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:01.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:55:01.954+0000 7f9387736700 1 --2- 172.21.15.149:0/2031030274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9380103230 0x7f938019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:01.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:55:01.954+0000 7f9387736700 1 -- 172.21.15.149:0/2031030274 >> 172.21.15.149:0/2031030274 conn(0x7f93800fd590 msgr2=0x7f9380074210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:01.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:55:01.954+0000 7f9387736700 1 -- 172.21.15.149:0/2031030274 shutdown_connections 2024-03-15T12:55:01.941 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-15T12:55:01.954+0000 7f9387736700 1 -- 172.21.15.149:0/2031030274 wait complete. 2024-03-15T12:55:02.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:02 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:02.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:02 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:02.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:02 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:02.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:02 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:55:02.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:02 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:02.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:02 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:02.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:02 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:02.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:02 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:55:02.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:02 smithi149 ceph-mon[35927]: from='osd.10 [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-15T12:55:02.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:02 smithi149 ceph-mon[35927]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-15T12:55:02.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:02 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:02.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:02 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:02.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:02 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:02.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:02 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:55:02.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:02 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:02.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:02 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:02.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:02 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:02.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:02 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:55:02.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:02 smithi002 ceph-mon[32060]: from='osd.10 [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-15T12:55:02.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:02 smithi002 ceph-mon[32060]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-15T12:55:02.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:02 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:02.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:02 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:02.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:02 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:02.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:02 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:55:02.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:02 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:02.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:02 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:02.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:02 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:02.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:02 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:55:02.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:02 smithi184 ceph-mon[35993]: from='osd.10 [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-15T12:55:02.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:02 smithi184 ceph-mon[35993]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-15T12:55:02.563 DEBUG:teuthology.orchestra.run.smithi149:osd.10> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.10.service 2024-03-15T12:55:02.565 INFO:tasks.cephadm:Deploying osd.11 on smithi184 with /dev/vg_nvme/lv_1... 2024-03-15T12:55:02.565 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-15T12:55:02.816 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T12:55:03.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:03 smithi149 ceph-mon[35927]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:55:03.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:03 smithi149 ceph-mon[35927]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-15T12:55:03.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:03 smithi149 ceph-mon[35927]: osdmap e70: 11 total, 10 up, 11 in 2024-03-15T12:55:03.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:03 smithi149 ceph-mon[35927]: from='osd.10 [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:55:03.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:03 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:55:03.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:03 smithi149 ceph-mon[35927]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:55:03.242 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:03 smithi184 ceph-mon[35993]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:55:03.242 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:03 smithi184 ceph-mon[35993]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-15T12:55:03.243 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:03 smithi184 ceph-mon[35993]: osdmap e70: 11 total, 10 up, 11 in 2024-03-15T12:55:03.243 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:03 smithi184 ceph-mon[35993]: from='osd.10 [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:55:03.243 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:03 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:55:03.243 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:03 smithi184 ceph-mon[35993]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:55:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:03 smithi002 ceph-mon[32060]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:55:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:03 smithi002 ceph-mon[32060]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-15T12:55:03.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:03 smithi002 ceph-mon[32060]: osdmap e70: 11 total, 10 up, 11 in 2024-03-15T12:55:03.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:03 smithi002 ceph-mon[32060]: from='osd.10 [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:55:03.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:03 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:55:03.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:03 smithi002 ceph-mon[32060]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-15T12:55:04.870 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-15T12:55:04.870 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-15T12:55:04.870 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-03-15T12:55:04.870 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-03-15T12:55:04.870 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113619 s, 923 MB/s 2024-03-15T12:55:04.871 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-03-15T12:55:05.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:04 smithi149 ceph-mon[35927]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-15T12:55:05.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:04 smithi149 ceph-mon[35927]: osdmap e71: 11 total, 10 up, 11 in 2024-03-15T12:55:05.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:04 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:55:05.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:04 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:55:05.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:04 smithi149 ceph-mon[35927]: from='osd.10 ' entity='osd.10' 2024-03-15T12:55:05.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:04 smithi149 ceph-mon[35927]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:55:05.111 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 12:55:04 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T12:55:04.655+0000 7f6d670a2700 -1 osd.10 0 waiting for initial osdmap 2024-03-15T12:55:05.111 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 12:55:04 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T12:55:04.667+0000 7f6d5ee8d700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-15T12:55:05.187 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:04 smithi184 ceph-mon[35993]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-15T12:55:05.187 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:04 smithi184 ceph-mon[35993]: osdmap e71: 11 total, 10 up, 11 in 2024-03-15T12:55:05.187 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:04 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:55:05.187 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:04 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:55:05.187 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:04 smithi184 ceph-mon[35993]: from='osd.10 ' entity='osd.10' 2024-03-15T12:55:05.187 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:04 smithi184 ceph-mon[35993]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:55:05.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:04 smithi002 ceph-mon[32060]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-15T12:55:05.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:04 smithi002 ceph-mon[32060]: osdmap e71: 11 total, 10 up, 11 in 2024-03-15T12:55:05.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:04 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:55:05.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:04 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:55:05.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:04 smithi002 ceph-mon[32060]: from='osd.10 ' entity='osd.10' 2024-03-15T12:55:05.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:04 smithi002 ceph-mon[32060]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-03-15T12:55:05.532 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi184:vg_nvme/lv_1 2024-03-15T12:55:05.786 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T12:55:06.159 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: purged_snaps scrub starts 2024-03-15T12:55:06.160 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: purged_snaps scrub ok 2024-03-15T12:55:06.160 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: osd.10 [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202] boot 2024-03-15T12:55:06.160 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: osdmap e72: 11 total, 11 up, 11 in 2024-03-15T12:55:06.160 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:55:06.160 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: osdmap e73: 11 total, 11 up, 11 in 2024-03-15T12:55:06.160 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: Detected new or changed devices on smithi149 2024-03-15T12:55:06.160 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:06.160 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:06.160 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:06.160 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:06.160 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:06.160 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:06.160 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi149 to 1454M 2024-03-15T12:55:06.160 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:06.161 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:06.161 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:55:06.161 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:05 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:06.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: purged_snaps scrub starts 2024-03-15T12:55:06.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: purged_snaps scrub ok 2024-03-15T12:55:06.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: osd.10 [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202] boot 2024-03-15T12:55:06.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: osdmap e72: 11 total, 11 up, 11 in 2024-03-15T12:55:06.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:55:06.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: osdmap e73: 11 total, 11 up, 11 in 2024-03-15T12:55:06.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: Detected new or changed devices on smithi149 2024-03-15T12:55:06.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:06.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:06.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:06.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:06.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:06.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:06.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi149 to 1454M 2024-03-15T12:55:06.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:06.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:06.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:55:06.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:05 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:06.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: purged_snaps scrub starts 2024-03-15T12:55:06.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: purged_snaps scrub ok 2024-03-15T12:55:06.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: osd.10 [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202] boot 2024-03-15T12:55:06.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: osdmap e72: 11 total, 11 up, 11 in 2024-03-15T12:55:06.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:55:06.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: osdmap e73: 11 total, 11 up, 11 in 2024-03-15T12:55:06.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: Detected new or changed devices on smithi149 2024-03-15T12:55:06.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:06.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:06.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:06.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:06.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:06.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:06.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi149 to 1454M 2024-03-15T12:55:06.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:06.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:06.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:55:06.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:05 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:07.363 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.375+0000 7fd61fa7d700 1 -- 172.21.15.184:0/2551651237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618103bb0 msgr2=0x7fd61810a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:07.363 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.375+0000 7fd61fa7d700 1 --2- 172.21.15.184:0/2551651237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618103bb0 0x7fd61810a460 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd61000b0a0 tx=0x7fd61000b3b0 comp rx=0 tx=0).stop 2024-03-15T12:55:07.364 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.375+0000 7fd61fa7d700 1 -- 172.21.15.184:0/2551651237 shutdown_connections 2024-03-15T12:55:07.364 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.375+0000 7fd61fa7d700 1 --2- 172.21.15.184:0/2551651237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd618102010 0x7fd618102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:07.364 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.375+0000 7fd61fa7d700 1 --2- 172.21.15.184:0/2551651237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618103bb0 0x7fd61810a460 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:07.364 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.375+0000 7fd61fa7d700 1 --2- 172.21.15.184:0/2551651237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd618103210 0x7fd618103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:07.364 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.375+0000 7fd61fa7d700 1 -- 172.21.15.184:0/2551651237 >> 172.21.15.184:0/2551651237 conn(0x7fd6180fd5f0 msgr2=0x7fd6180ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:07.364 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.375+0000 7fd61fa7d700 1 -- 172.21.15.184:0/2551651237 shutdown_connections 2024-03-15T12:55:07.364 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.375+0000 7fd61fa7d700 1 -- 172.21.15.184:0/2551651237 wait complete. 2024-03-15T12:55:07.364 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.376+0000 7fd61fa7d700 1 Processor -- start 2024-03-15T12:55:07.364 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.376+0000 7fd61fa7d700 1 -- start start 2024-03-15T12:55:07.364 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.376+0000 7fd61fa7d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd618102010 0x7fd618198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:07.364 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.376+0000 7fd61fa7d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd618103210 0x7fd6181985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:07.365 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.376+0000 7fd61fa7d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618103bb0 0x7fd61819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:07.365 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.376+0000 7fd61fa7d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61819fc00 con 0x7fd618103210 2024-03-15T12:55:07.365 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.376+0000 7fd61fa7d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61819fd40 con 0x7fd618102010 2024-03-15T12:55:07.365 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.376+0000 7fd61fa7d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61819fed0 con 0x7fd618103bb0 2024-03-15T12:55:07.365 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.376+0000 7fd61e01a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618103bb0 0x7fd61819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:07.365 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.376+0000 7fd61e01a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618103bb0 0x7fd61819f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:52034/0 (socket says 172.21.15.184:52034) 2024-03-15T12:55:07.365 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.376+0000 7fd61e01a700 1 -- 172.21.15.184:0/38434624 learned_addr learned my addr 172.21.15.184:0/38434624 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-15T12:55:07.365 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.376+0000 7fd61d819700 1 --2- 172.21.15.184:0/38434624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd618102010 0x7fd618198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:07.365 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.376+0000 7fd61d018700 1 --2- 172.21.15.184:0/38434624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd618103210 0x7fd6181985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:07.365 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.377+0000 7fd61e01a700 1 -- 172.21.15.184:0/38434624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd618102010 msgr2=0x7fd618198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:07.365 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.377+0000 7fd61e01a700 1 --2- 172.21.15.184:0/38434624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd618102010 0x7fd618198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:07.365 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.377+0000 7fd61e01a700 1 -- 172.21.15.184:0/38434624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd618103210 msgr2=0x7fd6181985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:07.365 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.377+0000 7fd61e01a700 1 --2- 172.21.15.184:0/38434624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd618103210 0x7fd6181985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:07.366 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.377+0000 7fd61e01a700 1 -- 172.21.15.184:0/38434624 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd610009d00 con 0x7fd618103bb0 2024-03-15T12:55:07.366 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.377+0000 7fd61e01a700 1 --2- 172.21.15.184:0/38434624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618103bb0 0x7fd61819f630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd610007c90 tx=0x7fd610004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:07.367 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.379+0000 7fd60effd700 1 -- 172.21.15.184:0/38434624 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61000e040 con 0x7fd618103bb0 2024-03-15T12:55:07.367 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.379+0000 7fd60effd700 1 -- 172.21.15.184:0/38434624 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd61000cec0 con 0x7fd618103bb0 2024-03-15T12:55:07.367 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.379+0000 7fd61fa7d700 1 -- 172.21.15.184:0/38434624 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6181a0150 con 0x7fd618103bb0 2024-03-15T12:55:07.367 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.379+0000 7fd60effd700 1 -- 172.21.15.184:0/38434624 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61001ba50 con 0x7fd618103bb0 2024-03-15T12:55:07.367 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.379+0000 7fd61fa7d700 1 -- 172.21.15.184:0/38434624 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6181a0610 con 0x7fd618103bb0 2024-03-15T12:55:07.367 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.380+0000 7fd61fa7d700 1 -- 172.21.15.184:0/38434624 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6181093d0 con 0x7fd618103bb0 2024-03-15T12:55:07.368 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.381+0000 7fd60effd700 1 -- 172.21.15.184:0/38434624 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fd61001bbb0 con 0x7fd618103bb0 2024-03-15T12:55:07.368 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.381+0000 7fd60effd700 1 --2- 172.21.15.184:0/38434624 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd6040a1b40 0x7fd6040a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:07.368 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.382+0000 7fd60effd700 1 -- 172.21.15.184:0/38434624 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fd6100ce910 con 0x7fd618103bb0 2024-03-15T12:55:07.368 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.382+0000 7fd61d819700 1 --2- 172.21.15.184:0/38434624 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd6040a1b40 0x7fd6040a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:07.368 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.382+0000 7fd61d819700 1 --2- 172.21.15.184:0/38434624 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd6040a1b40 0x7fd6040a4000 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd618103070 tx=0x7fd60800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:07.376 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.390+0000 7fd60effd700 1 -- 172.21.15.184:0/38434624 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd6100981e0 con 0x7fd618103bb0 2024-03-15T12:55:07.579 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:07.592+0000 7fd61fa7d700 1 -- 172.21.15.184:0/38434624 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd61802da30 con 0x7fd6040a1b40 2024-03-15T12:55:07.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:07 smithi002 ceph-mon[32060]: osdmap e74: 11 total, 11 up, 11 in 2024-03-15T12:55:07.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:07 smithi002 ceph-mon[32060]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:07.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:07 smithi184 ceph-mon[35993]: osdmap e74: 11 total, 11 up, 11 in 2024-03-15T12:55:07.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:07 smithi184 ceph-mon[35993]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:07.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:07 smithi149 ceph-mon[35927]: osdmap e74: 11 total, 11 up, 11 in 2024-03-15T12:55:07.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:07 smithi149 ceph-mon[35927]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:08.529 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:08 smithi184 ceph-mon[35993]: osdmap e75: 11 total, 11 up, 11 in 2024-03-15T12:55:08.529 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:08 smithi184 ceph-mon[35993]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:55:08.529 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:08 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:55:08.529 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:08 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:55:08.529 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:08 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:08.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:08 smithi002 ceph-mon[32060]: osdmap e75: 11 total, 11 up, 11 in 2024-03-15T12:55:08.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:08 smithi002 ceph-mon[32060]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:55:08.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:55:08.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:55:08.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:08 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:08.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:08 smithi149 ceph-mon[35927]: osdmap e75: 11 total, 11 up, 11 in 2024-03-15T12:55:08.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:08 smithi149 ceph-mon[35927]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:55:08.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:08 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-15T12:55:08.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:08 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-15T12:55:08.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:08 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:09.459 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:09 smithi184 ceph-mon[35993]: osdmap e76: 11 total, 11 up, 11 in 2024-03-15T12:55:09.459 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:09 smithi184 ceph-mon[35993]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-03-15T12:55:09.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:09 smithi002 ceph-mon[32060]: osdmap e76: 11 total, 11 up, 11 in 2024-03-15T12:55:09.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:09 smithi002 ceph-mon[32060]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-03-15T12:55:09.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:09 smithi149 ceph-mon[35927]: osdmap e76: 11 total, 11 up, 11 in 2024-03-15T12:55:09.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:09 smithi149 ceph-mon[35927]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-03-15T12:55:10.706 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:10 smithi184 ceph-mon[35993]: from='client.? 172.21.15.184:0/1936726735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a9260e1-292e-4694-863b-b6710b18feb0"}]: dispatch 2024-03-15T12:55:10.706 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:10 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a9260e1-292e-4694-863b-b6710b18feb0"}]: dispatch 2024-03-15T12:55:10.706 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:10 smithi184 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a9260e1-292e-4694-863b-b6710b18feb0"}]': finished 2024-03-15T12:55:10.706 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:10 smithi184 ceph-mon[35993]: osdmap e77: 12 total, 11 up, 12 in 2024-03-15T12:55:10.706 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:10 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:10.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:10 smithi002 ceph-mon[32060]: from='client.? 172.21.15.184:0/1936726735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a9260e1-292e-4694-863b-b6710b18feb0"}]: dispatch 2024-03-15T12:55:10.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:10 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a9260e1-292e-4694-863b-b6710b18feb0"}]: dispatch 2024-03-15T12:55:10.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:10 smithi002 ceph-mon[32060]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a9260e1-292e-4694-863b-b6710b18feb0"}]': finished 2024-03-15T12:55:10.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:10 smithi002 ceph-mon[32060]: osdmap e77: 12 total, 11 up, 12 in 2024-03-15T12:55:10.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:10 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:10.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:10 smithi149 ceph-mon[35927]: from='client.? 172.21.15.184:0/1936726735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a9260e1-292e-4694-863b-b6710b18feb0"}]: dispatch 2024-03-15T12:55:10.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:10 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a9260e1-292e-4694-863b-b6710b18feb0"}]: dispatch 2024-03-15T12:55:10.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:10 smithi149 ceph-mon[35927]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a9260e1-292e-4694-863b-b6710b18feb0"}]': finished 2024-03-15T12:55:10.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:10 smithi149 ceph-mon[35927]: osdmap e77: 12 total, 11 up, 12 in 2024-03-15T12:55:10.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:10 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:11.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:11 smithi002 ceph-mon[32060]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-03-15T12:55:11.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:11 smithi002 ceph-mon[32060]: from='client.? 172.21.15.184:0/4204235364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:55:11.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:11 smithi184 ceph-mon[35993]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-03-15T12:55:11.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:11 smithi184 ceph-mon[35993]: from='client.? 172.21.15.184:0/4204235364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:55:11.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:11 smithi149 ceph-mon[35927]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-03-15T12:55:11.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:11 smithi149 ceph-mon[35927]: from='client.? 172.21.15.184:0/4204235364' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-15T12:55:13.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:12 smithi002 ceph-mon[32060]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-15T12:55:13.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:12 smithi184 ceph-mon[35993]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-15T12:55:13.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:12 smithi149 ceph-mon[35927]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-15T12:55:15.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:14 smithi002 ceph-mon[32060]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-15T12:55:15.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:14 smithi184 ceph-mon[35993]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-15T12:55:15.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:14 smithi149 ceph-mon[35927]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-15T12:55:16.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:16 smithi184 ceph-mon[35993]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:17.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:16 smithi002 ceph-mon[32060]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:17.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:16 smithi149 ceph-mon[35927]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:18.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-15T12:55:18.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:18.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-15T12:55:18.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:18.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-15T12:55:18.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:19.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:18 smithi184 ceph-mon[35993]: Deploying daemon osd.11 on smithi184 2024-03-15T12:55:19.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:18 smithi184 ceph-mon[35993]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:19.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:18 smithi149 ceph-mon[35927]: Deploying daemon osd.11 on smithi184 2024-03-15T12:55:19.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:18 smithi149 ceph-mon[35927]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:19.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:18 smithi002 ceph-mon[32060]: Deploying daemon osd.11 on smithi184 2024-03-15T12:55:19.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:18 smithi002 ceph-mon[32060]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:21.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:20 smithi002 ceph-mon[32060]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:21.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:20 smithi184 ceph-mon[35993]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:21.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:20 smithi149 ceph-mon[35927]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:22.913 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:22 smithi184 ceph-mon[35993]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:23.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:22 smithi002 ceph-mon[32060]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:23.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:22 smithi149 ceph-mon[35927]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:24.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:24.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:24.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:24 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:55:24.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:24.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:24.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:24 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:55:24.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:24.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:24.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:24 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:55:25.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:25 smithi002 ceph-mon[32060]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:25.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:25 smithi184 ceph-mon[35993]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:25.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:25 smithi149 ceph-mon[35927]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:27.597 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:27.609+0000 7fd60effd700 1 -- 172.21.15.184:0/38434624 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd61802da30 con 0x7fd6040a1b40 2024-03-15T12:55:27.598 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 11 on host 'smithi184' 2024-03-15T12:55:27.600 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:27.613+0000 7fd61fa7d700 1 -- 172.21.15.184:0/38434624 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd6040a1b40 msgr2=0x7fd6040a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:27.600 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:27.613+0000 7fd61fa7d700 1 --2- 172.21.15.184:0/38434624 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd6040a1b40 0x7fd6040a4000 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd618103070 tx=0x7fd60800b410 comp rx=0 tx=0).stop 2024-03-15T12:55:27.600 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:27.613+0000 7fd61fa7d700 1 -- 172.21.15.184:0/38434624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618103bb0 msgr2=0x7fd61819f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:27.600 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:27.613+0000 7fd61fa7d700 1 --2- 172.21.15.184:0/38434624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618103bb0 0x7fd61819f630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd610007c90 tx=0x7fd610004d00 comp rx=0 tx=0).stop 2024-03-15T12:55:27.600 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:27.613+0000 7fd61fa7d700 1 -- 172.21.15.184:0/38434624 shutdown_connections 2024-03-15T12:55:27.600 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:27.613+0000 7fd61fa7d700 1 --2- 172.21.15.184:0/38434624 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd6040a1b40 0x7fd6040a4000 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:27.600 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:27.613+0000 7fd61fa7d700 1 --2- 172.21.15.184:0/38434624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd618103bb0 0x7fd61819f630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:27.600 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:27.613+0000 7fd61fa7d700 1 --2- 172.21.15.184:0/38434624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd618102010 0x7fd618198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:27.600 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:27.613+0000 7fd61fa7d700 1 --2- 172.21.15.184:0/38434624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd618103210 0x7fd6181985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:27.600 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:27.613+0000 7fd61fa7d700 1 -- 172.21.15.184:0/38434624 >> 172.21.15.184:0/38434624 conn(0x7fd6180fd5f0 msgr2=0x7fd618106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:27.600 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:27.613+0000 7fd61fa7d700 1 -- 172.21.15.184:0/38434624 shutdown_connections 2024-03-15T12:55:27.601 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:27.613+0000 7fd61fa7d700 1 -- 172.21.15.184:0/38434624 wait complete. 2024-03-15T12:55:27.601 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:27.601 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:27.601 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:27.601 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:55:27.601 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:27 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:27.601 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:27 smithi184 ceph-mon[35993]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:27.601 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:27 smithi184 ceph-mon[35993]: from='osd.11 [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-15T12:55:27.601 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:27 smithi184 ceph-mon[35993]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-15T12:55:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:27.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:27.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:55:27.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:27 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:27.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:27 smithi002 ceph-mon[32060]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:27.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:27 smithi002 ceph-mon[32060]: from='osd.11 [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-15T12:55:27.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:27 smithi002 ceph-mon[32060]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-15T12:55:27.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:27.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:27.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:27.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:55:27.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:27 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:27.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:27 smithi149 ceph-mon[35927]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:27.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:27 smithi149 ceph-mon[35927]: from='osd.11 [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-15T12:55:27.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:27 smithi149 ceph-mon[35927]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-15T12:55:28.242 DEBUG:teuthology.orchestra.run.smithi184:osd.11> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.11.service 2024-03-15T12:55:28.245 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-15T12:55:28.245 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-15T12:55:28.500 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:55:28.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:28 smithi002 ceph-mon[32060]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-15T12:55:28.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:28 smithi002 ceph-mon[32060]: osdmap e78: 12 total, 11 up, 12 in 2024-03-15T12:55:28.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:28 smithi002 ceph-mon[32060]: from='osd.11 [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:55:28.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:28 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:28.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:28 smithi002 ceph-mon[32060]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:55:28.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:28 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:28.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:28 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:55:28.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:28 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:28.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:28 smithi184 ceph-mon[35993]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-15T12:55:28.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:28 smithi184 ceph-mon[35993]: osdmap e78: 12 total, 11 up, 12 in 2024-03-15T12:55:28.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:28 smithi184 ceph-mon[35993]: from='osd.11 [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:55:28.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:28 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:28.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:28 smithi184 ceph-mon[35993]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:55:28.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:28 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:28.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:28 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:55:28.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:28 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:28.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:28 smithi149 ceph-mon[35927]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-15T12:55:28.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:28 smithi149 ceph-mon[35927]: osdmap e78: 12 total, 11 up, 12 in 2024-03-15T12:55:28.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:28 smithi149 ceph-mon[35927]: from='osd.11 [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:55:28.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:28 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:28.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:28 smithi149 ceph-mon[35927]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-03-15T12:55:28.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:28 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:28.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:28 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:55:28.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:28 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:29.433 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 12:55:29 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T12:55:29.127+0000 7f3319177700 -1 osd.11 0 waiting for initial osdmap 2024-03-15T12:55:29.434 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 12:55:29 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T12:55:29.139+0000 7f3310761700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-15T12:55:29.712 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:29 smithi184 ceph-mon[35993]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-03-15T12:55:29.713 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:29 smithi184 ceph-mon[35993]: osdmap e79: 12 total, 11 up, 12 in 2024-03-15T12:55:29.713 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:29 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:29.713 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:29 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:29.713 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:29 smithi184 ceph-mon[35993]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:29.713 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:29 smithi184 ceph-mon[35993]: from='osd.11 ' entity='osd.11' 2024-03-15T12:55:29.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:29 smithi002 ceph-mon[32060]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-03-15T12:55:29.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:29 smithi002 ceph-mon[32060]: osdmap e79: 12 total, 11 up, 12 in 2024-03-15T12:55:29.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:29 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:29.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:29 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:29.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:29 smithi002 ceph-mon[32060]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:29.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:29 smithi002 ceph-mon[32060]: from='osd.11 ' entity='osd.11' 2024-03-15T12:55:29.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:29 smithi149 ceph-mon[35927]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-03-15T12:55:29.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:29 smithi149 ceph-mon[35927]: osdmap e79: 12 total, 11 up, 12 in 2024-03-15T12:55:29.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:29 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:29.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:29 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:29.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:29 smithi149 ceph-mon[35927]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 983 GiB / 983 GiB avail 2024-03-15T12:55:29.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:29 smithi149 ceph-mon[35927]: from='osd.11 ' entity='osd.11' 2024-03-15T12:55:30.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.409+0000 7f1390ecc700 1 -- 172.21.15.2:0/3777010738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138c101ba0 msgr2=0x7f138c102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:30.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.409+0000 7f1390ecc700 1 --2- 172.21.15.2:0/3777010738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138c101ba0 0x7f138c102020 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f138400b0a0 tx=0x7f138400b3b0 comp rx=0 tx=0).stop 2024-03-15T12:55:30.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.409+0000 7f1390ecc700 1 -- 172.21.15.2:0/3777010738 shutdown_connections 2024-03-15T12:55:30.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.409+0000 7f1390ecc700 1 --2- 172.21.15.2:0/3777010738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138c101ba0 0x7f138c102020 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:30.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.409+0000 7f1390ecc700 1 --2- 172.21.15.2:0/3777010738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138c100880 0x7f138c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:30.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.409+0000 7f1390ecc700 1 --2- 172.21.15.2:0/3777010738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f138c1011e0 0x7f138c101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:30.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.409+0000 7f1390ecc700 1 -- 172.21.15.2:0/3777010738 >> 172.21.15.2:0/3777010738 conn(0x7f138c0fc4a0 msgr2=0x7f138c0fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:30.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.409+0000 7f1390ecc700 1 -- 172.21.15.2:0/3777010738 shutdown_connections 2024-03-15T12:55:30.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.410+0000 7f1390ecc700 1 -- 172.21.15.2:0/3777010738 wait complete. 2024-03-15T12:55:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.411+0000 7f1390ecc700 1 Processor -- start 2024-03-15T12:55:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.411+0000 7f1390ecc700 1 -- start start 2024-03-15T12:55:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.411+0000 7f1390ecc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f138c100880 0x7f138c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.411+0000 7f1390ecc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138c1011e0 0x7f138c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.411+0000 7f1390ecc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138c101ba0 0x7f138c1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.411+0000 7f1390ecc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c1a1de0 con 0x7f138c101ba0 2024-03-15T12:55:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.411+0000 7f1390ecc700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c1a1f20 con 0x7f138c100880 2024-03-15T12:55:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.411+0000 7f1390ecc700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c1a20b0 con 0x7f138c1011e0 2024-03-15T12:55:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.411+0000 7f1389d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138c1011e0 0x7f138c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.411+0000 7f138ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138c101ba0 0x7f138c1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.411+0000 7f1389d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138c1011e0 0x7f138c19a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:38214/0 (socket says 172.21.15.2:38214) 2024-03-15T12:55:30.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.411+0000 7f138ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138c101ba0 0x7f138c1a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37936/0 (socket says 172.21.15.2:37936) 2024-03-15T12:55:30.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.411+0000 7f1389d9b700 1 -- 172.21.15.2:0/3383892320 learned_addr learned my addr 172.21.15.2:0/3383892320 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:55:30.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.411+0000 7f138a59c700 1 --2- 172.21.15.2:0/3383892320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f138c100880 0x7f138c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:30.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.412+0000 7f138ad9d700 1 -- 172.21.15.2:0/3383892320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138c1011e0 msgr2=0x7f138c19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:30.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.412+0000 7f138ad9d700 1 --2- 172.21.15.2:0/3383892320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138c1011e0 0x7f138c19a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:30.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.412+0000 7f138ad9d700 1 -- 172.21.15.2:0/3383892320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f138c100880 msgr2=0x7f138c19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:30.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.412+0000 7f138ad9d700 1 --2- 172.21.15.2:0/3383892320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f138c100880 0x7f138c19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:30.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.412+0000 7f138ad9d700 1 -- 172.21.15.2:0/3383892320 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1384009d00 con 0x7f138c101ba0 2024-03-15T12:55:30.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.412+0000 7f1389d9b700 1 --2- 172.21.15.2:0/3383892320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138c1011e0 0x7f138c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:55:30.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.412+0000 7f138ad9d700 1 --2- 172.21.15.2:0/3383892320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138c101ba0 0x7f138c1a1810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1384007bf0 tx=0x7f138400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:30.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.412+0000 7f137b7fe700 1 -- 172.21.15.2:0/3383892320 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138400e040 con 0x7f138c101ba0 2024-03-15T12:55:30.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.412+0000 7f137b7fe700 1 -- 172.21.15.2:0/3383892320 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1384004d60 con 0x7f138c101ba0 2024-03-15T12:55:30.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.412+0000 7f137b7fe700 1 -- 172.21.15.2:0/3383892320 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138401d480 con 0x7f138c101ba0 2024-03-15T12:55:30.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.412+0000 7f1390ecc700 1 -- 172.21.15.2:0/3383892320 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138c1a2330 con 0x7f138c101ba0 2024-03-15T12:55:30.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.412+0000 7f1390ecc700 1 -- 172.21.15.2:0/3383892320 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138c1a2750 con 0x7f138c101ba0 2024-03-15T12:55:30.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.413+0000 7f1390ecc700 1 -- 172.21.15.2:0/3383892320 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f138c10b660 con 0x7f138c101ba0 2024-03-15T12:55:30.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.415+0000 7f137b7fe700 1 -- 172.21.15.2:0/3383892320 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f13840040b0 con 0x7f138c101ba0 2024-03-15T12:55:30.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.415+0000 7f137b7fe700 1 --2- 172.21.15.2:0/3383892320 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f13740a1ac0 0x7f13740a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:30.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.415+0000 7f138a59c700 1 --2- 172.21.15.2:0/3383892320 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f13740a1ac0 0x7f13740a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:30.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.415+0000 7f137b7fe700 1 -- 172.21.15.2:0/3383892320 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f13840ce900 con 0x7f138c101ba0 2024-03-15T12:55:30.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.416+0000 7f138a59c700 1 --2- 172.21.15.2:0/3383892320 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f13740a1ac0 0x7f13740a3f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f137c007c60 tx=0x7f137c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.419+0000 7f137b7fe700 1 -- 172.21.15.2:0/3383892320 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1384098040 con 0x7f138c101ba0 2024-03-15T12:55:30.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.626+0000 7f1390ecc700 1 -- 172.21.15.2:0/3383892320 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f138c10fb10 con 0x7f138c101ba0 2024-03-15T12:55:30.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.627+0000 7f137b7fe700 1 -- 172.21.15.2:0/3383892320 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f138409b700 con 0x7f138c101ba0 2024-03-15T12:55:30.614 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:55:30.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.629+0000 7f1390ecc700 1 -- 172.21.15.2:0/3383892320 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f13740a1ac0 msgr2=0x7f13740a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:30.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.629+0000 7f1390ecc700 1 --2- 172.21.15.2:0/3383892320 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f13740a1ac0 0x7f13740a3f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f137c007c60 tx=0x7f137c0073d0 comp rx=0 tx=0).stop 2024-03-15T12:55:30.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.629+0000 7f1390ecc700 1 -- 172.21.15.2:0/3383892320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138c101ba0 msgr2=0x7f138c1a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:30.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.629+0000 7f1390ecc700 1 --2- 172.21.15.2:0/3383892320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138c101ba0 0x7f138c1a1810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1384007bf0 tx=0x7f138400be20 comp rx=0 tx=0).stop 2024-03-15T12:55:30.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.629+0000 7f1390ecc700 1 -- 172.21.15.2:0/3383892320 shutdown_connections 2024-03-15T12:55:30.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.629+0000 7f1390ecc700 1 --2- 172.21.15.2:0/3383892320 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f13740a1ac0 0x7f13740a3f80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:30.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.629+0000 7f1390ecc700 1 --2- 172.21.15.2:0/3383892320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138c101ba0 0x7f138c1a1810 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:30.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.629+0000 7f1390ecc700 1 --2- 172.21.15.2:0/3383892320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f138c100880 0x7f138c19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:30.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.629+0000 7f1390ecc700 1 --2- 172.21.15.2:0/3383892320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f138c1011e0 0x7f138c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:30.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.629+0000 7f1390ecc700 1 -- 172.21.15.2:0/3383892320 >> 172.21.15.2:0/3383892320 conn(0x7f138c0fc4a0 msgr2=0x7f138c0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:30.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.629+0000 7f1390ecc700 1 -- 172.21.15.2:0/3383892320 shutdown_connections 2024-03-15T12:55:30.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:30.629+0000 7f1390ecc700 1 -- 172.21.15.2:0/3383892320 wait complete. 2024-03-15T12:55:30.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:30 smithi002 ceph-mon[32060]: purged_snaps scrub starts 2024-03-15T12:55:30.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:30 smithi002 ceph-mon[32060]: purged_snaps scrub ok 2024-03-15T12:55:30.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:30.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:30 smithi002 ceph-mon[32060]: osd.11 [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252] boot 2024-03-15T12:55:30.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:30 smithi002 ceph-mon[32060]: osdmap e80: 12 total, 12 up, 12 in 2024-03-15T12:55:30.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:30 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:30.761 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:30 smithi149 ceph-mon[35927]: purged_snaps scrub starts 2024-03-15T12:55:30.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:30 smithi149 ceph-mon[35927]: purged_snaps scrub ok 2024-03-15T12:55:30.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:30 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:30.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:30 smithi149 ceph-mon[35927]: osd.11 [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252] boot 2024-03-15T12:55:30.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:30 smithi149 ceph-mon[35927]: osdmap e80: 12 total, 12 up, 12 in 2024-03-15T12:55:30.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:30 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:30.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:30 smithi184 ceph-mon[35993]: purged_snaps scrub starts 2024-03-15T12:55:30.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:30 smithi184 ceph-mon[35993]: purged_snaps scrub ok 2024-03-15T12:55:30.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:30 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:30.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:30 smithi184 ceph-mon[35993]: osd.11 [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252] boot 2024-03-15T12:55:30.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:30 smithi184 ceph-mon[35993]: osdmap e80: 12 total, 12 up, 12 in 2024-03-15T12:55:30.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:30 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:55:31.351 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1710507330,"num_in_osds":12,"osd_in_since":1710507310,"num_remapped_pgs":0} 2024-03-15T12:55:31.352 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-15T12:55:31.596 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:55:31.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:31 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3383892320' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-15T12:55:31.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:31 smithi002 ceph-mon[32060]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:31.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:31 smithi002 ceph-mon[32060]: osdmap e81: 12 total, 12 up, 12 in 2024-03-15T12:55:31.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:31 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:31.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:31 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:31.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:31 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:31.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:31 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:31.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:31 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:31.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:31 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:31.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:31 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:31.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:31 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:31.648 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:31 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:55:31.648 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:31 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:31.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:31 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3383892320' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-15T12:55:31.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:31 smithi184 ceph-mon[35993]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:31.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:31 smithi184 ceph-mon[35993]: osdmap e81: 12 total, 12 up, 12 in 2024-03-15T12:55:31.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:31 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:31.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:31 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:31.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:31 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:31.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:31 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:31.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:31 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:31.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:31 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:31.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:31 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:31.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:31 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:31.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:31 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:55:31.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:31 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:31.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:31 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3383892320' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-15T12:55:31.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:31 smithi149 ceph-mon[35927]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:31.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:31 smithi149 ceph-mon[35927]: osdmap e81: 12 total, 12 up, 12 in 2024-03-15T12:55:31.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:31 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:31.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:31 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:31.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:31 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:31.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:31 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:31.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:31 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:31.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:31 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:55:31.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:31 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:31.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:31 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:31.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:31 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:55:31.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:31 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:32.615 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:32 smithi184 ceph-mon[35993]: Detected new or changed devices on smithi184 2024-03-15T12:55:32.615 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:32 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi184 to 4272M 2024-03-15T12:55:32.716 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:32 smithi002 ceph-mon[32060]: Detected new or changed devices on smithi184 2024-03-15T12:55:32.716 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:32 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi184 to 4272M 2024-03-15T12:55:32.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:32 smithi149 ceph-mon[35927]: Detected new or changed devices on smithi184 2024-03-15T12:55:32.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:32 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi184 to 4272M 2024-03-15T12:55:33.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.307+0000 7fd0b7488700 1 -- 172.21.15.2:0/2835907318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b0101b80 msgr2=0x7fd0b010ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:33.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.307+0000 7fd0b7488700 1 --2- 172.21.15.2:0/2835907318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b0101b80 0x7fd0b010ddb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800b0a0 tx=0x7fd0a800b3b0 comp rx=0 tx=0).stop 2024-03-15T12:55:33.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.307+0000 7fd0b7488700 1 -- 172.21.15.2:0/2835907318 shutdown_connections 2024-03-15T12:55:33.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.307+0000 7fd0b7488700 1 --2- 172.21.15.2:0/2835907318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b0101b80 0x7fd0b010ddb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:33.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.307+0000 7fd0b7488700 1 --2- 172.21.15.2:0/2835907318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0b01070d0 0x7fd0b01074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:33.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.307+0000 7fd0b7488700 1 --2- 172.21.15.2:0/2835907318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0b01011c0 0x7fd0b0101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:33.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.307+0000 7fd0b7488700 1 -- 172.21.15.2:0/2835907318 >> 172.21.15.2:0/2835907318 conn(0x7fd0b0077d40 msgr2=0x7fd0b0078150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:33.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.307+0000 7fd0b7488700 1 -- 172.21.15.2:0/2835907318 shutdown_connections 2024-03-15T12:55:33.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.307+0000 7fd0b7488700 1 -- 172.21.15.2:0/2835907318 wait complete. 2024-03-15T12:55:33.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b7488700 1 Processor -- start 2024-03-15T12:55:33.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b7488700 1 -- start start 2024-03-15T12:55:33.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b7488700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0b01011c0 0x7fd0b0198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:33.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b7488700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b0101b80 0x7fd0b0198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:33.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b7488700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0b01070d0 0x7fd0b019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:33.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b7488700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b019d4f0 con 0x7fd0b0101b80 2024-03-15T12:55:33.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b7488700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b019d660 con 0x7fd0b01070d0 2024-03-15T12:55:33.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b7488700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b019d820 con 0x7fd0b01011c0 2024-03-15T12:55:33.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b4a23700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b0101b80 0x7fd0b0198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:33.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b4a23700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b0101b80 0x7fd0b0198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37960/0 (socket says 172.21.15.2:37960) 2024-03-15T12:55:33.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b4a23700 1 -- 172.21.15.2:0/475521411 learned_addr learned my addr 172.21.15.2:0/475521411 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:55:33.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b5224700 1 --2- 172.21.15.2:0/475521411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0b01011c0 0x7fd0b0198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:33.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b5a25700 1 --2- 172.21.15.2:0/475521411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0b01070d0 0x7fd0b019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:33.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b4a23700 1 -- 172.21.15.2:0/475521411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0b01011c0 msgr2=0x7fd0b0198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:33.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b4a23700 1 --2- 172.21.15.2:0/475521411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0b01011c0 0x7fd0b0198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:33.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b4a23700 1 -- 172.21.15.2:0/475521411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0b01070d0 msgr2=0x7fd0b019cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:33.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b4a23700 1 --2- 172.21.15.2:0/475521411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0b01070d0 0x7fd0b019cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:33.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.308+0000 7fd0b4a23700 1 -- 172.21.15.2:0/475521411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a8009d00 con 0x7fd0b0101b80 2024-03-15T12:55:33.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.309+0000 7fd0b4a23700 1 --2- 172.21.15.2:0/475521411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b0101b80 0x7fd0b0198b50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000e9a0 tx=0x7fd0a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:33.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.309+0000 7fd0a67fc700 1 -- 172.21.15.2:0/475521411 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0009790 con 0x7fd0b0101b80 2024-03-15T12:55:33.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.309+0000 7fd0a67fc700 1 -- 172.21.15.2:0/475521411 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd0a000cbd0 con 0x7fd0b0101b80 2024-03-15T12:55:33.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.309+0000 7fd0b7488700 1 -- 172.21.15.2:0/475521411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b019db00 con 0x7fd0b0101b80 2024-03-15T12:55:33.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.309+0000 7fd0a67fc700 1 -- 172.21.15.2:0/475521411 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0018680 con 0x7fd0b0101b80 2024-03-15T12:55:33.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.309+0000 7fd0b7488700 1 -- 172.21.15.2:0/475521411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b01a5360 con 0x7fd0b0101b80 2024-03-15T12:55:33.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.310+0000 7fd0b7488700 1 -- 172.21.15.2:0/475521411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b0102660 con 0x7fd0b0101b80 2024-03-15T12:55:33.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.311+0000 7fd0a67fc700 1 -- 172.21.15.2:0/475521411 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fd0a000cd40 con 0x7fd0b0101b80 2024-03-15T12:55:33.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.311+0000 7fd0a67fc700 1 --2- 172.21.15.2:0/475521411 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd09c0a1b40 0x7fd09c0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:33.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.311+0000 7fd0b5224700 1 --2- 172.21.15.2:0/475521411 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd09c0a1b40 0x7fd09c0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:33.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.312+0000 7fd0a67fc700 1 -- 172.21.15.2:0/475521411 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd0a0014070 con 0x7fd0b0101b80 2024-03-15T12:55:33.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.312+0000 7fd0b5224700 1 --2- 172.21.15.2:0/475521411 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd09c0a1b40 0x7fd09c0a4000 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac005950 tx=0x7fd0ac00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:33.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.315+0000 7fd0a67fc700 1 -- 172.21.15.2:0/475521411 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd0a0096ab0 con 0x7fd0b0101b80 2024-03-15T12:55:33.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.518+0000 7fd0b7488700 1 -- 172.21.15.2:0/475521411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0b0068ca0 con 0x7fd0b0101b80 2024-03-15T12:55:33.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.519+0000 7fd0a67fc700 1 -- 172.21.15.2:0/475521411 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18373 (secure 0 0 0) 0x7fd0a009a170 con 0x7fd0b0101b80 2024-03-15T12:55:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:55:33.507 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":81,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:55:31.131625+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:55:33.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.521+0000 7fd0b7488700 1 -- 172.21.15.2:0/475521411 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd09c0a1b40 msgr2=0x7fd09c0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:33.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.521+0000 7fd0b7488700 1 --2- 172.21.15.2:0/475521411 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd09c0a1b40 0x7fd09c0a4000 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac005950 tx=0x7fd0ac00b410 comp rx=0 tx=0).stop 2024-03-15T12:55:33.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.521+0000 7fd0b7488700 1 -- 172.21.15.2:0/475521411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b0101b80 msgr2=0x7fd0b0198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:33.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.521+0000 7fd0b7488700 1 --2- 172.21.15.2:0/475521411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b0101b80 0x7fd0b0198b50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000e9a0 tx=0x7fd0a000ecb0 comp rx=0 tx=0).stop 2024-03-15T12:55:33.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.522+0000 7fd0b7488700 1 -- 172.21.15.2:0/475521411 shutdown_connections 2024-03-15T12:55:33.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.522+0000 7fd0b7488700 1 --2- 172.21.15.2:0/475521411 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd09c0a1b40 0x7fd09c0a4000 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:33.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.522+0000 7fd0b7488700 1 --2- 172.21.15.2:0/475521411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0b01070d0 0x7fd0b019cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:33.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.522+0000 7fd0b7488700 1 --2- 172.21.15.2:0/475521411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0b01011c0 0x7fd0b0198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:33.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.522+0000 7fd0b7488700 1 --2- 172.21.15.2:0/475521411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b0101b80 0x7fd0b0198b50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:33.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.522+0000 7fd0b7488700 1 -- 172.21.15.2:0/475521411 >> 172.21.15.2:0/475521411 conn(0x7fd0b0077d40 msgr2=0x7fd0b01003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:33.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.522+0000 7fd0b7488700 1 -- 172.21.15.2:0/475521411 shutdown_connections 2024-03-15T12:55:33.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:33.522+0000 7fd0b7488700 1 -- 172.21.15.2:0/475521411 wait complete. 2024-03-15T12:55:33.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:33 smithi002 ceph-mon[32060]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:33.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:33 smithi184 ceph-mon[35993]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:33.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:33 smithi149 ceph-mon[35927]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:34.376 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-15T12:51:47.004843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-15T12:55:34.377 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-15T12:55:34.625 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:55:34.746 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:34 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/475521411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:55:34.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:34 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/475521411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:55:34.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:34 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/475521411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:55:35.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:35 smithi002 ceph-mon[32060]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:35.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:35 smithi184 ceph-mon[35993]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:35.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:35 smithi149 ceph-mon[35927]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:36.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 -- 172.21.15.2:0/2008430071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f257c1073a0 msgr2=0x7f257c10ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:36.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 --2- 172.21.15.2:0/2008430071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f257c1073a0 0x7f257c10ef20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f257400b0a0 tx=0x7f257400b3b0 comp rx=0 tx=0).stop 2024-03-15T12:55:36.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 -- 172.21.15.2:0/2008430071 shutdown_connections 2024-03-15T12:55:36.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 --2- 172.21.15.2:0/2008430071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f257c1073a0 0x7f257c10ef20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:36.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 --2- 172.21.15.2:0/2008430071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f257c100b80 0x7f257c100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:36.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 --2- 172.21.15.2:0/2008430071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f257c101530 0x7f257c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:36.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 -- 172.21.15.2:0/2008430071 >> 172.21.15.2:0/2008430071 conn(0x7f257c076240 msgr2=0x7f257c076650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:36.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 -- 172.21.15.2:0/2008430071 shutdown_connections 2024-03-15T12:55:36.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 -- 172.21.15.2:0/2008430071 wait complete. 2024-03-15T12:55:36.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 Processor -- start 2024-03-15T12:55:36.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 -- start start 2024-03-15T12:55:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f257c100b80 0x7f257c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f257c101530 0x7f257c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f257c1073a0 0x7f257c19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c19d5a0 con 0x7f257c1073a0 2024-03-15T12:55:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c19d710 con 0x7f257c100b80 2024-03-15T12:55:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.476+0000 7f2582c52700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c19d8d0 con 0x7f257c101530 2024-03-15T12:55:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.477+0000 7f25811ef700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f257c1073a0 0x7f257c19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.477+0000 7f25811ef700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f257c1073a0 0x7f257c19cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37970/0 (socket says 172.21.15.2:37970) 2024-03-15T12:55:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.477+0000 7f25811ef700 1 -- 172.21.15.2:0/3890183270 learned_addr learned my addr 172.21.15.2:0/3890183270 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:55:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.477+0000 7f25809ee700 1 --2- 172.21.15.2:0/3890183270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f257c100b80 0x7f257c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.477+0000 7f257bfff700 1 --2- 172.21.15.2:0/3890183270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f257c101530 0x7f257c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.477+0000 7f25811ef700 1 -- 172.21.15.2:0/3890183270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f257c101530 msgr2=0x7f257c198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.477+0000 7f25811ef700 1 --2- 172.21.15.2:0/3890183270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f257c101530 0x7f257c198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.477+0000 7f25811ef700 1 -- 172.21.15.2:0/3890183270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f257c100b80 msgr2=0x7f257c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:36.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.477+0000 7f25811ef700 1 --2- 172.21.15.2:0/3890183270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f257c100b80 0x7f257c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:36.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.477+0000 7f25811ef700 1 -- 172.21.15.2:0/3890183270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2574009d00 con 0x7f257c1073a0 2024-03-15T12:55:36.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.477+0000 7f25811ef700 1 --2- 172.21.15.2:0/3890183270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f257c1073a0 0x7f257c19cf90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f257400b830 tx=0x7f257400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:36.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.477+0000 7f2579ffb700 1 -- 172.21.15.2:0/3890183270 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257400e040 con 0x7f257c1073a0 2024-03-15T12:55:36.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.477+0000 7f2582c52700 1 -- 172.21.15.2:0/3890183270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257c19db50 con 0x7f257c1073a0 2024-03-15T12:55:36.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.477+0000 7f2579ffb700 1 -- 172.21.15.2:0/3890183270 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f257401d950 con 0x7f257c1073a0 2024-03-15T12:55:36.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.477+0000 7f2579ffb700 1 -- 172.21.15.2:0/3890183270 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257401baf0 con 0x7f257c1073a0 2024-03-15T12:55:36.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.478+0000 7f2582c52700 1 -- 172.21.15.2:0/3890183270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257c1a5410 con 0x7f257c1073a0 2024-03-15T12:55:36.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.478+0000 7f2582c52700 1 -- 172.21.15.2:0/3890183270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257c10a5e0 con 0x7f257c1073a0 2024-03-15T12:55:36.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.479+0000 7f2579ffb700 1 -- 172.21.15.2:0/3890183270 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f257401bc50 con 0x7f257c1073a0 2024-03-15T12:55:36.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.480+0000 7f2579ffb700 1 --2- 172.21.15.2:0/3890183270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f256c0a1ac0 0x7f256c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:36.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.480+0000 7f25809ee700 1 --2- 172.21.15.2:0/3890183270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f256c0a1ac0 0x7f256c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:36.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.480+0000 7f2579ffb700 1 -- 172.21.15.2:0/3890183270 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f25740cf090 con 0x7f257c1073a0 2024-03-15T12:55:36.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.480+0000 7f25809ee700 1 --2- 172.21.15.2:0/3890183270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f256c0a1ac0 0x7f256c0a3f80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2570009a10 tx=0x7f2570009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:36.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.484+0000 7f2579ffb700 1 -- 172.21.15.2:0/3890183270 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f25740987d0 con 0x7f257c1073a0 2024-03-15T12:55:36.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.685+0000 7f2582c52700 1 -- 172.21.15.2:0/3890183270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f257c068ca0 con 0x7f257c1073a0 2024-03-15T12:55:36.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.686+0000 7f2579ffb700 1 -- 172.21.15.2:0/3890183270 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f257409be90 con 0x7f257c1073a0 2024-03-15T12:55:36.672 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-15T12:55:36.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.688+0000 7f2582c52700 1 -- 172.21.15.2:0/3890183270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f256c0a1ac0 msgr2=0x7f256c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:36.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.688+0000 7f2582c52700 1 --2- 172.21.15.2:0/3890183270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f256c0a1ac0 0x7f256c0a3f80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2570009a10 tx=0x7f2570009190 comp rx=0 tx=0).stop 2024-03-15T12:55:36.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.688+0000 7f2582c52700 1 -- 172.21.15.2:0/3890183270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f257c1073a0 msgr2=0x7f257c19cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:36.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.688+0000 7f2582c52700 1 --2- 172.21.15.2:0/3890183270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f257c1073a0 0x7f257c19cf90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f257400b830 tx=0x7f257400ced0 comp rx=0 tx=0).stop 2024-03-15T12:55:36.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.688+0000 7f2582c52700 1 -- 172.21.15.2:0/3890183270 shutdown_connections 2024-03-15T12:55:36.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.688+0000 7f2582c52700 1 --2- 172.21.15.2:0/3890183270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f256c0a1ac0 0x7f256c0a3f80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:36.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.688+0000 7f2582c52700 1 --2- 172.21.15.2:0/3890183270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f257c1073a0 0x7f257c19cf90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:36.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.688+0000 7f2582c52700 1 --2- 172.21.15.2:0/3890183270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f257c100b80 0x7f257c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:36.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.688+0000 7f2582c52700 1 --2- 172.21.15.2:0/3890183270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f257c101530 0x7f257c198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:36.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.688+0000 7f2582c52700 1 -- 172.21.15.2:0/3890183270 >> 172.21.15.2:0/3890183270 conn(0x7f257c076240 msgr2=0x7f257c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:36.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.688+0000 7f2582c52700 1 -- 172.21.15.2:0/3890183270 shutdown_connections 2024-03-15T12:55:36.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:36.688+0000 7f2582c52700 1 -- 172.21.15.2:0/3890183270 wait complete. 2024-03-15T12:55:37.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:36 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3890183270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:55:37.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:36 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3890183270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:55:37.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:36 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3890183270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:55:37.381 INFO:tasks.cephadm:Adding mds.a on smithi002 2024-03-15T12:55:37.381 INFO:tasks.cephadm:Adding mds.d on smithi002 2024-03-15T12:55:37.382 INFO:tasks.cephadm:Adding mds.g on smithi002 2024-03-15T12:55:37.382 INFO:tasks.cephadm:Adding mds.j on smithi002 2024-03-15T12:55:37.382 INFO:tasks.cephadm:Adding mds.b on smithi149 2024-03-15T12:55:37.382 INFO:tasks.cephadm:Adding mds.e on smithi149 2024-03-15T12:55:37.382 INFO:tasks.cephadm:Adding mds.h on smithi149 2024-03-15T12:55:37.382 INFO:tasks.cephadm:Adding mds.k on smithi149 2024-03-15T12:55:37.382 INFO:tasks.cephadm:Adding mds.c on smithi184 2024-03-15T12:55:37.382 INFO:tasks.cephadm:Adding mds.f on smithi184 2024-03-15T12:55:37.382 INFO:tasks.cephadm:Adding mds.i on smithi184 2024-03-15T12:55:37.382 INFO:tasks.cephadm:Adding mds.l on smithi184 2024-03-15T12:55:37.382 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph orch apply mds all '12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l' 2024-03-15T12:55:37.633 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T12:55:37.979 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:37 smithi184 ceph-mon[35993]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:38.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:37 smithi002 ceph-mon[32060]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:38.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:37 smithi149 ceph-mon[35927]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:39.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:38 smithi002 ceph-mon[32060]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:39.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:38 smithi184 ceph-mon[35993]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:39.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:38 smithi149 ceph-mon[35927]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:39.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.315+0000 7f539f9ca700 1 -- 172.21.15.184:0/1662435668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5398102a30 msgr2=0x7f5398109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:39.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.315+0000 7f539f9ca700 1 --2- 172.21.15.184:0/1662435668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5398102a30 0x7f5398109350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f539000b0a0 tx=0x7f539000b3b0 comp rx=0 tx=0).stop 2024-03-15T12:55:39.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.315+0000 7f539f9ca700 1 -- 172.21.15.184:0/1662435668 shutdown_connections 2024-03-15T12:55:39.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.315+0000 7f539f9ca700 1 --2- 172.21.15.184:0/1662435668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5398102a30 0x7f5398109350 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:39.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.315+0000 7f539f9ca700 1 --2- 172.21.15.184:0/1662435668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5398100f10 0x7f5398101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:39.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.315+0000 7f539f9ca700 1 --2- 172.21.15.184:0/1662435668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5398102070 0x7f53981024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:39.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.315+0000 7f539f9ca700 1 -- 172.21.15.184:0/1662435668 >> 172.21.15.184:0/1662435668 conn(0x7f53980fc4b0 msgr2=0x7f53980fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:39.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.315+0000 7f539f9ca700 1 -- 172.21.15.184:0/1662435668 shutdown_connections 2024-03-15T12:55:39.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.315+0000 7f539f9ca700 1 -- 172.21.15.184:0/1662435668 wait complete. 2024-03-15T12:55:39.304 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539f9ca700 1 Processor -- start 2024-03-15T12:55:39.305 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539f9ca700 1 -- start start 2024-03-15T12:55:39.305 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539f9ca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5398100f10 0x7f5398197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:39.305 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539f9ca700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5398102070 0x7f53981983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:39.305 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539f9ca700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5398102a30 0x7f539819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:39.305 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539f9ca700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539819fa30 con 0x7f5398100f10 2024-03-15T12:55:39.305 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539f9ca700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539819fb70 con 0x7f5398102a30 2024-03-15T12:55:39.305 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539f9ca700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539819fd00 con 0x7f5398102070 2024-03-15T12:55:39.305 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539cf65700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5398102070 0x7f53981983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:39.305 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539cf65700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5398102070 0x7f53981983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:43602/0 (socket says 172.21.15.184:43602) 2024-03-15T12:55:39.305 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539cf65700 1 -- 172.21.15.184:0/3752099308 learned_addr learned my addr 172.21.15.184:0/3752099308 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-15T12:55:39.305 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539d766700 1 --2- 172.21.15.184:0/3752099308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5398100f10 0x7f5398197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:39.305 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539df67700 1 --2- 172.21.15.184:0/3752099308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5398102a30 0x7f539819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:39.305 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539cf65700 1 -- 172.21.15.184:0/3752099308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5398102a30 msgr2=0x7f539819f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:39.305 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539cf65700 1 --2- 172.21.15.184:0/3752099308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5398102a30 0x7f539819f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:39.306 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539cf65700 1 -- 172.21.15.184:0/3752099308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5398100f10 msgr2=0x7f5398197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:39.306 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539cf65700 1 --2- 172.21.15.184:0/3752099308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5398100f10 0x7f5398197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:39.306 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.316+0000 7f539cf65700 1 -- 172.21.15.184:0/3752099308 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5390009d00 con 0x7f5398102070 2024-03-15T12:55:39.306 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.317+0000 7f539cf65700 1 --2- 172.21.15.184:0/3752099308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5398102070 0x7f53981983e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f539400e9d0 tx=0x7f539400ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:39.306 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.317+0000 7f538e7fc700 1 -- 172.21.15.184:0/3752099308 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394009790 con 0x7f5398102070 2024-03-15T12:55:39.306 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.317+0000 7f539f9ca700 1 -- 172.21.15.184:0/3752099308 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539819ffe0 con 0x7f5398102070 2024-03-15T12:55:39.306 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.317+0000 7f538e7fc700 1 -- 172.21.15.184:0/3752099308 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f539400cbd0 con 0x7f5398102070 2024-03-15T12:55:39.306 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.317+0000 7f538e7fc700 1 -- 172.21.15.184:0/3752099308 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394018680 con 0x7f5398102070 2024-03-15T12:55:39.306 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.317+0000 7f539f9ca700 1 -- 172.21.15.184:0/3752099308 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53981a04a0 con 0x7f5398102070 2024-03-15T12:55:39.306 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.318+0000 7f539f9ca700 1 -- 172.21.15.184:0/3752099308 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53981082c0 con 0x7f5398102070 2024-03-15T12:55:39.307 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.319+0000 7f538e7fc700 1 -- 172.21.15.184:0/3752099308 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f539400cd40 con 0x7f5398102070 2024-03-15T12:55:39.307 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.320+0000 7f538e7fc700 1 --2- 172.21.15.184:0/3752099308 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f53840a1ac0 0x7f53840a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:39.307 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.320+0000 7f538e7fc700 1 -- 172.21.15.184:0/3752099308 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5394014070 con 0x7f5398102070 2024-03-15T12:55:39.307 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.320+0000 7f539d766700 1 --2- 172.21.15.184:0/3752099308 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f53840a1ac0 0x7f53840a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:39.307 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.320+0000 7f539d766700 1 --2- 172.21.15.184:0/3752099308 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f53840a1ac0 0x7f53840a3f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5388005950 tx=0x7f538800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:39.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.324+0000 7f538e7fc700 1 -- 172.21.15.184:0/3752099308 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5394096a90 con 0x7f5398102070 2024-03-15T12:55:39.512 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.526+0000 7f539f9ca700 1 -- 172.21.15.184:0/3752099308 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f539802dba0 con 0x7f53840a1ac0 2024-03-15T12:55:39.520 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.534+0000 7f538e7fc700 1 -- 172.21.15.184:0/3752099308 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f539802dba0 con 0x7f53840a1ac0 2024-03-15T12:55:39.520 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mds.all update... 2024-03-15T12:55:39.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.536+0000 7f539f9ca700 1 -- 172.21.15.184:0/3752099308 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f53840a1ac0 msgr2=0x7f53840a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:39.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.536+0000 7f539f9ca700 1 --2- 172.21.15.184:0/3752099308 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f53840a1ac0 0x7f53840a3f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5388005950 tx=0x7f538800b410 comp rx=0 tx=0).stop 2024-03-15T12:55:39.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.536+0000 7f539f9ca700 1 -- 172.21.15.184:0/3752099308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5398102070 msgr2=0x7f53981983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:39.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.536+0000 7f539f9ca700 1 --2- 172.21.15.184:0/3752099308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5398102070 0x7f53981983e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f539400e9d0 tx=0x7f539400ece0 comp rx=0 tx=0).stop 2024-03-15T12:55:39.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.536+0000 7f539f9ca700 1 -- 172.21.15.184:0/3752099308 shutdown_connections 2024-03-15T12:55:39.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.536+0000 7f539f9ca700 1 --2- 172.21.15.184:0/3752099308 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f53840a1ac0 0x7f53840a3f80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:39.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.536+0000 7f539f9ca700 1 --2- 172.21.15.184:0/3752099308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5398100f10 0x7f5398197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:39.523 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.536+0000 7f539f9ca700 1 --2- 172.21.15.184:0/3752099308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5398102a30 0x7f539819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:39.524 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.536+0000 7f539f9ca700 1 --2- 172.21.15.184:0/3752099308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5398102070 0x7f53981983e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:39.524 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.536+0000 7f539f9ca700 1 -- 172.21.15.184:0/3752099308 >> 172.21.15.184:0/3752099308 conn(0x7f53980fc4b0 msgr2=0x7f5398105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:39.524 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.537+0000 7f539f9ca700 1 -- 172.21.15.184:0/3752099308 shutdown_connections 2024-03-15T12:55:39.524 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-15T12:55:39.537+0000 7f539f9ca700 1 -- 172.21.15.184:0/3752099308 wait complete. 2024-03-15T12:55:40.211 DEBUG:teuthology.orchestra.run.smithi002:mds.a> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.a.service 2024-03-15T12:55:40.214 DEBUG:teuthology.orchestra.run.smithi002:mds.d> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.d.service 2024-03-15T12:55:40.258 DEBUG:teuthology.orchestra.run.smithi002:mds.g> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.g.service 2024-03-15T12:55:40.301 DEBUG:teuthology.orchestra.run.smithi002:mds.j> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.j.service 2024-03-15T12:55:40.343 DEBUG:teuthology.orchestra.run.smithi149:mds.b> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.b.service 2024-03-15T12:55:40.345 DEBUG:teuthology.orchestra.run.smithi149:mds.e> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.e.service 2024-03-15T12:55:40.388 DEBUG:teuthology.orchestra.run.smithi149:mds.h> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.h.service 2024-03-15T12:55:40.431 DEBUG:teuthology.orchestra.run.smithi149:mds.k> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.k.service 2024-03-15T12:55:40.474 DEBUG:teuthology.orchestra.run.smithi184:mds.c> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.c.service 2024-03-15T12:55:40.476 DEBUG:teuthology.orchestra.run.smithi184:mds.f> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.f.service 2024-03-15T12:55:40.519 DEBUG:teuthology.orchestra.run.smithi184:mds.i> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.i.service 2024-03-15T12:55:40.562 DEBUG:teuthology.orchestra.run.smithi184:mds.l> sudo journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.l.service 2024-03-15T12:55:40.605 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-15T12:55:40.605 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:55:40.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.735+0000 7f1ba7548640 1 -- 172.21.15.2:0/4290892558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ba01030d0 msgr2=0x7f1ba0109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:40.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.735+0000 7f1ba7548640 1 --2- 172.21.15.2:0/4290892558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ba01030d0 0x7f1ba0109960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009ca0 tx=0x7f1b9c02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:55:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.735+0000 7f1ba7548640 1 -- 172.21.15.2:0/4290892558 shutdown_connections 2024-03-15T12:55:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.735+0000 7f1ba7548640 1 --2- 172.21.15.2:0/4290892558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ba01030d0 0x7f1ba0109960 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.735+0000 7f1ba7548640 1 --2- 172.21.15.2:0/4290892558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ba0102710 0x7f1ba0102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.735+0000 7f1ba7548640 1 --2- 172.21.15.2:0/4290892558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ba0101520 0x7f1ba0101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.735+0000 7f1ba7548640 1 -- 172.21.15.2:0/4290892558 >> 172.21.15.2:0/4290892558 conn(0x7f1ba00fcad0 msgr2=0x7f1ba00feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.736+0000 7f1ba7548640 1 -- 172.21.15.2:0/4290892558 shutdown_connections 2024-03-15T12:55:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.736+0000 7f1ba7548640 1 -- 172.21.15.2:0/4290892558 wait complete. 2024-03-15T12:55:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.736+0000 7f1ba7548640 1 Processor -- start 2024-03-15T12:55:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.736+0000 7f1ba7548640 1 -- start start 2024-03-15T12:55:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.736+0000 7f1ba7548640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ba0101520 0x7f1ba0197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.736+0000 7f1ba7548640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ba0102710 0x7f1ba0197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.736+0000 7f1ba7548640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ba01030d0 0x7f1ba019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.736+0000 7f1ba7548640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba019f030 con 0x7f1ba0102710 2024-03-15T12:55:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.736+0000 7f1ba7548640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba019f1a0 con 0x7f1ba01030d0 2024-03-15T12:55:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.736+0000 7f1ba7548640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba019f360 con 0x7f1ba0101520 2024-03-15T12:55:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.736+0000 7f1ba4ae3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ba0102710 0x7f1ba0197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.736+0000 7f1ba4ae3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ba0102710 0x7f1ba0197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59892/0 (socket says 172.21.15.2:59892) 2024-03-15T12:55:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.736+0000 7f1ba4ae3640 1 -- 172.21.15.2:0/983076079 learned_addr learned my addr 172.21.15.2:0/983076079 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:55:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.736+0000 7f1ba52e4640 1 --2- 172.21.15.2:0/983076079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ba0101520 0x7f1ba0197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.737+0000 7f1ba5ae5640 1 --2- 172.21.15.2:0/983076079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ba01030d0 0x7f1ba019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.737+0000 7f1ba4ae3640 1 -- 172.21.15.2:0/983076079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ba0101520 msgr2=0x7f1ba0197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.737+0000 7f1ba4ae3640 1 --2- 172.21.15.2:0/983076079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ba0101520 0x7f1ba0197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.737+0000 7f1ba4ae3640 1 -- 172.21.15.2:0/983076079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ba01030d0 msgr2=0x7f1ba019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.737+0000 7f1ba4ae3640 1 --2- 172.21.15.2:0/983076079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ba01030d0 0x7f1ba019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.737+0000 7f1ba4ae3640 1 -- 172.21.15.2:0/983076079 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c009900 con 0x7f1ba0102710 2024-03-15T12:55:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.737+0000 7f1ba5ae5640 1 --2- 172.21.15.2:0/983076079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ba01030d0 0x7f1ba019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:55:40.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.737+0000 7f1ba4ae3640 1 --2- 172.21.15.2:0/983076079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ba0102710 0x7f1ba0197a80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000d770 tx=0x7f1b9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:40.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.737+0000 7f1b967fc640 1 -- 172.21.15.2:0/983076079 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b90009c30 con 0x7f1ba0102710 2024-03-15T12:55:40.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.737+0000 7f1ba7548640 1 -- 172.21.15.2:0/983076079 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba019f640 con 0x7f1ba0102710 2024-03-15T12:55:40.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.737+0000 7f1b967fc640 1 -- 172.21.15.2:0/983076079 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1b9000de80 con 0x7f1ba0102710 2024-03-15T12:55:40.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.737+0000 7f1b967fc640 1 -- 172.21.15.2:0/983076079 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b90004e50 con 0x7f1ba0102710 2024-03-15T12:55:40.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.737+0000 7f1ba7548640 1 -- 172.21.15.2:0/983076079 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba019fb00 con 0x7f1ba0102710 2024-03-15T12:55:40.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.738+0000 7f1ba7548640 1 -- 172.21.15.2:0/983076079 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba01087e0 con 0x7f1ba0102710 2024-03-15T12:55:40.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.739+0000 7f1b967fc640 1 -- 172.21.15.2:0/983076079 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f1b90005000 con 0x7f1ba0102710 2024-03-15T12:55:40.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.740+0000 7f1b967fc640 1 --2- 172.21.15.2:0/983076079 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1b8c0a1a30 0x7f1b8c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:40.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.740+0000 7f1ba52e4640 1 --2- 172.21.15.2:0/983076079 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1b8c0a1a30 0x7f1b8c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:40.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.740+0000 7f1b967fc640 1 -- 172.21.15.2:0/983076079 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1b900ccaa0 con 0x7f1ba0102710 2024-03-15T12:55:40.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.740+0000 7f1ba52e4640 1 --2- 172.21.15.2:0/983076079 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1b8c0a1a30 0x7f1b8c0a3ef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0102570 tx=0x7f1b9800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:40.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.744+0000 7f1b967fc640 1 -- 172.21.15.2:0/983076079 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1b900960b0 con 0x7f1ba0102710 2024-03-15T12:55:40.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:40 smithi002 ceph-mon[32060]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:55:40.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:40 smithi002 ceph-mon[32060]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-03-15T12:55:40.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:40 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:40.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:40 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:55:40.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:40 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:40.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:40 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:55:40.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:40 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:40.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:40 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:40.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:40 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:40.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:40 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:40.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:40 smithi184 ceph-mon[35993]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:55:40.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:40 smithi184 ceph-mon[35993]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-03-15T12:55:40.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:40 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:40.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:40 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:55:40.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:40 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:40.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:40 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:55:40.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:40 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:40.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:40 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:40.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:40 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:40.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:40 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:40.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:40 smithi149 ceph-mon[35927]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:55:40.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:40 smithi149 ceph-mon[35927]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-03-15T12:55:40.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:40 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:40.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:40 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:55:40.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:40 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:40.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:40 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:55:40.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:40 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:40.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:40 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:40.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:40 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:40.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:40 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:40.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.929+0000 7f1ba7548640 1 -- 172.21.15.2:0/983076079 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ba002d000 con 0x7f1ba0102710 2024-03-15T12:55:40.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.930+0000 7f1b967fc640 1 -- 172.21.15.2:0/983076079 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18373 (secure 0 0 0) 0x7f1b90099770 con 0x7f1ba0102710 2024-03-15T12:55:40.917 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:55:40.917 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":81,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:55:31.131625+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:55:40.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.932+0000 7f1ba7548640 1 -- 172.21.15.2:0/983076079 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1b8c0a1a30 msgr2=0x7f1b8c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:40.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.932+0000 7f1ba7548640 1 --2- 172.21.15.2:0/983076079 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1b8c0a1a30 0x7f1b8c0a3ef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0102570 tx=0x7f1b9800b390 comp rx=0 tx=0).stop 2024-03-15T12:55:40.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.932+0000 7f1ba7548640 1 -- 172.21.15.2:0/983076079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ba0102710 msgr2=0x7f1ba0197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:40.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.932+0000 7f1ba7548640 1 --2- 172.21.15.2:0/983076079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ba0102710 0x7f1ba0197a80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000d770 tx=0x7f1b9000dc30 comp rx=0 tx=0).stop 2024-03-15T12:55:40.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.932+0000 7f1ba7548640 1 -- 172.21.15.2:0/983076079 shutdown_connections 2024-03-15T12:55:40.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.932+0000 7f1ba7548640 1 --2- 172.21.15.2:0/983076079 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1b8c0a1a30 0x7f1b8c0a3ef0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:40.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.932+0000 7f1ba7548640 1 --2- 172.21.15.2:0/983076079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ba01030d0 0x7f1ba019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:40.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.932+0000 7f1ba7548640 1 --2- 172.21.15.2:0/983076079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ba0102710 0x7f1ba0197a80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:40.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.932+0000 7f1ba7548640 1 --2- 172.21.15.2:0/983076079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ba0101520 0x7f1ba0197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:40.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.932+0000 7f1ba7548640 1 -- 172.21.15.2:0/983076079 >> 172.21.15.2:0/983076079 conn(0x7f1ba00fcad0 msgr2=0x7f1ba01059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:40.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.932+0000 7f1ba7548640 1 -- 172.21.15.2:0/983076079 shutdown_connections 2024-03-15T12:55:40.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:40.932+0000 7f1ba7548640 1 -- 172.21.15.2:0/983076079 wait complete. 2024-03-15T12:55:40.930 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-15T12:51:47.004843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-15T12:55:40.930 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-15T12:55:41.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.049+0000 7f9e547d7640 1 -- 172.21.15.2:0/2277442233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e4c101500 msgr2=0x7f9e4c101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:41.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.049+0000 7f9e547d7640 1 --2- 172.21.15.2:0/2277442233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e4c101500 0x7f9e4c101920 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009900 tx=0x7f9e4802bc30 comp rx=0 tx=0).stop 2024-03-15T12:55:41.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.049+0000 7f9e547d7640 1 -- 172.21.15.2:0/2277442233 shutdown_connections 2024-03-15T12:55:41.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.049+0000 7f9e547d7640 1 --2- 172.21.15.2:0/2277442233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e4c1030b0 0x7f9e4c109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.049+0000 7f9e547d7640 1 --2- 172.21.15.2:0/2277442233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e4c1026f0 0x7f9e4c102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.049+0000 7f9e547d7640 1 --2- 172.21.15.2:0/2277442233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e4c101500 0x7f9e4c101920 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.049+0000 7f9e547d7640 1 -- 172.21.15.2:0/2277442233 >> 172.21.15.2:0/2277442233 conn(0x7f9e4c0fca90 msgr2=0x7f9e4c0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:41.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.049+0000 7f9e547d7640 1 -- 172.21.15.2:0/2277442233 shutdown_connections 2024-03-15T12:55:41.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.049+0000 7f9e547d7640 1 -- 172.21.15.2:0/2277442233 wait complete. 2024-03-15T12:55:41.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e547d7640 1 Processor -- start 2024-03-15T12:55:41.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e547d7640 1 -- start start 2024-03-15T12:55:41.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e547d7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e4c101500 0x7f9e4c1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:41.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e547d7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e4c1026f0 0x7f9e4c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:41.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e547d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e4c1030b0 0x7f9e4c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:41.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e547d7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c19f0e0 con 0x7f9e4c1030b0 2024-03-15T12:55:41.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e547d7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c19f250 con 0x7f9e4c1026f0 2024-03-15T12:55:41.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e547d7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c19f410 con 0x7f9e4c101500 2024-03-15T12:55:41.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e52d74640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e4c1030b0 0x7f9e4c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:41.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e52d74640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e4c1030b0 0x7f9e4c19eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59914/0 (socket says 172.21.15.2:59914) 2024-03-15T12:55:41.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e52d74640 1 -- 172.21.15.2:0/2106192270 learned_addr learned my addr 172.21.15.2:0/2106192270 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:55:41.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e52573640 1 --2- 172.21.15.2:0/2106192270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e4c101500 0x7f9e4c1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:41.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e51d72640 1 --2- 172.21.15.2:0/2106192270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e4c1026f0 0x7f9e4c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:41.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e52d74640 1 -- 172.21.15.2:0/2106192270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e4c101500 msgr2=0x7f9e4c1975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:41.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e52d74640 1 --2- 172.21.15.2:0/2106192270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e4c101500 0x7f9e4c1975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e52d74640 1 -- 172.21.15.2:0/2106192270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e4c1026f0 msgr2=0x7f9e4c197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:41.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e52d74640 1 --2- 172.21.15.2:0/2106192270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e4c1026f0 0x7f9e4c197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e52d74640 1 -- 172.21.15.2:0/2106192270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e48009560 con 0x7f9e4c1030b0 2024-03-15T12:55:41.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.050+0000 7f9e51d72640 1 --2- 172.21.15.2:0/2106192270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e4c1026f0 0x7f9e4c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T12:55:41.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.051+0000 7f9e52d74640 1 --2- 172.21.15.2:0/2106192270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e4c1030b0 0x7f9e4c19eba0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400b970 tx=0x7f9e4400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:41.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.051+0000 7f9e437fe640 1 -- 172.21.15.2:0/2106192270 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4400cb40 con 0x7f9e4c1030b0 2024-03-15T12:55:41.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.051+0000 7f9e437fe640 1 -- 172.21.15.2:0/2106192270 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9e4400cca0 con 0x7f9e4c1030b0 2024-03-15T12:55:41.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.051+0000 7f9e547d7640 1 -- 172.21.15.2:0/2106192270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4c19f6f0 con 0x7f9e4c1030b0 2024-03-15T12:55:41.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.051+0000 7f9e437fe640 1 -- 172.21.15.2:0/2106192270 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e44007750 con 0x7f9e4c1030b0 2024-03-15T12:55:41.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.051+0000 7f9e547d7640 1 -- 172.21.15.2:0/2106192270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4c19fc80 con 0x7f9e4c1030b0 2024-03-15T12:55:41.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.052+0000 7f9e547d7640 1 -- 172.21.15.2:0/2106192270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4c108820 con 0x7f9e4c1030b0 2024-03-15T12:55:41.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.053+0000 7f9e437fe640 1 -- 172.21.15.2:0/2106192270 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f9e44014440 con 0x7f9e4c1030b0 2024-03-15T12:55:41.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.053+0000 7f9e437fe640 1 --2- 172.21.15.2:0/2106192270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9e380a1ab0 0x7f9e380a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:41.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.053+0000 7f9e52573640 1 --2- 172.21.15.2:0/2106192270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9e380a1ab0 0x7f9e380a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:41.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.054+0000 7f9e437fe640 1 -- 172.21.15.2:0/2106192270 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9e440cc890 con 0x7f9e4c1030b0 2024-03-15T12:55:41.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.054+0000 7f9e52573640 1 --2- 172.21.15.2:0/2106192270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9e380a1ab0 0x7f9e380a3f70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9e4802c740 tx=0x7f9e48000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:41.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.058+0000 7f9e437fe640 1 -- 172.21.15.2:0/2106192270 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9e44095ff0 con 0x7f9e4c1030b0 2024-03-15T12:55:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.240+0000 7f9e547d7640 1 -- 172.21.15.2:0/2106192270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9e4c02d000 con 0x7f9e4c1030b0 2024-03-15T12:55:41.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.241+0000 7f9e437fe640 1 -- 172.21.15.2:0/2106192270 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9e440996b0 con 0x7f9e4c1030b0 2024-03-15T12:55:41.227 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-15T12:55:41.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.243+0000 7f9e547d7640 1 -- 172.21.15.2:0/2106192270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9e380a1ab0 msgr2=0x7f9e380a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:41.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.243+0000 7f9e547d7640 1 --2- 172.21.15.2:0/2106192270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9e380a1ab0 0x7f9e380a3f70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9e4802c740 tx=0x7f9e48000bc0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.243+0000 7f9e547d7640 1 -- 172.21.15.2:0/2106192270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e4c1030b0 msgr2=0x7f9e4c19eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:41.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.243+0000 7f9e547d7640 1 --2- 172.21.15.2:0/2106192270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e4c1030b0 0x7f9e4c19eba0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400b970 tx=0x7f9e4400be30 comp rx=0 tx=0).stop 2024-03-15T12:55:41.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.243+0000 7f9e547d7640 1 -- 172.21.15.2:0/2106192270 shutdown_connections 2024-03-15T12:55:41.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.243+0000 7f9e547d7640 1 --2- 172.21.15.2:0/2106192270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e4c1030b0 0x7f9e4c19eba0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.243+0000 7f9e547d7640 1 --2- 172.21.15.2:0/2106192270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9e380a1ab0 0x7f9e380a3f70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.243+0000 7f9e547d7640 1 --2- 172.21.15.2:0/2106192270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e4c1026f0 0x7f9e4c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.243+0000 7f9e547d7640 1 --2- 172.21.15.2:0/2106192270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e4c101500 0x7f9e4c1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.243+0000 7f9e547d7640 1 -- 172.21.15.2:0/2106192270 >> 172.21.15.2:0/2106192270 conn(0x7f9e4c0fca90 msgr2=0x7f9e4c1059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:41.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.243+0000 7f9e547d7640 1 -- 172.21.15.2:0/2106192270 shutdown_connections 2024-03-15T12:55:41.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.243+0000 7f9e547d7640 1 -- 172.21.15.2:0/2106192270 wait complete. 2024-03-15T12:55:41.241 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:55:41.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.357+0000 7fae88fe2640 1 -- 172.21.15.2:0/3957503972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae84100420 msgr2=0x7fae84100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:41.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.357+0000 7fae88fe2640 1 --2- 172.21.15.2:0/3957503972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae84100420 0x7fae84100840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fae74009900 tx=0x7fae7402bc30 comp rx=0 tx=0).stop 2024-03-15T12:55:41.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.357+0000 7fae88fe2640 1 -- 172.21.15.2:0/3957503972 shutdown_connections 2024-03-15T12:55:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.357+0000 7fae88fe2640 1 --2- 172.21.15.2:0/3957503972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae84101fd0 0x7fae84108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.357+0000 7fae88fe2640 1 --2- 172.21.15.2:0/3957503972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae84101610 0x7fae84101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.357+0000 7fae88fe2640 1 --2- 172.21.15.2:0/3957503972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae84100420 0x7fae84100840 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.357+0000 7fae88fe2640 1 -- 172.21.15.2:0/3957503972 >> 172.21.15.2:0/3957503972 conn(0x7fae840fb9d0 msgr2=0x7fae840fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.357+0000 7fae88fe2640 1 -- 172.21.15.2:0/3957503972 shutdown_connections 2024-03-15T12:55:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.357+0000 7fae88fe2640 1 -- 172.21.15.2:0/3957503972 wait complete. 2024-03-15T12:55:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae88fe2640 1 Processor -- start 2024-03-15T12:55:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae88fe2640 1 -- start start 2024-03-15T12:55:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae88fe2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae84100420 0x7fae841975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae88fe2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae84101610 0x7fae84197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae88fe2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae84101fd0 0x7fae8419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae88fe2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8419f0c0 con 0x7fae84101fd0 2024-03-15T12:55:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae88fe2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8419f230 con 0x7fae84101610 2024-03-15T12:55:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae88fe2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8419f3f0 con 0x7fae84100420 2024-03-15T12:55:41.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae84101fd0 0x7fae8419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:41.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae82d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae84101fd0 0x7fae8419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59934/0 (socket says 172.21.15.2:59934) 2024-03-15T12:55:41.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae82d9d640 1 -- 172.21.15.2:0/3879088148 learned_addr learned my addr 172.21.15.2:0/3879088148 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:55:41.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae81d9b640 1 --2- 172.21.15.2:0/3879088148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae84101610 0x7fae84197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:41.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae8259c640 1 --2- 172.21.15.2:0/3879088148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae84100420 0x7fae841975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:41.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae82d9d640 1 -- 172.21.15.2:0/3879088148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae84100420 msgr2=0x7fae841975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:41.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae82d9d640 1 --2- 172.21.15.2:0/3879088148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae84100420 0x7fae841975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae82d9d640 1 -- 172.21.15.2:0/3879088148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae84101610 msgr2=0x7fae84197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:41.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae82d9d640 1 --2- 172.21.15.2:0/3879088148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae84101610 0x7fae84197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae82d9d640 1 -- 172.21.15.2:0/3879088148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae74009560 con 0x7fae84101fd0 2024-03-15T12:55:41.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.358+0000 7fae82d9d640 1 --2- 172.21.15.2:0/3879088148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae84101fd0 0x7fae8419eb80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00eb10 tx=0x7fae7c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:41.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.359+0000 7fae737fe640 1 -- 172.21.15.2:0/3879088148 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c004b20 con 0x7fae84101fd0 2024-03-15T12:55:41.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.359+0000 7fae737fe640 1 -- 172.21.15.2:0/3879088148 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fae7c004c80 con 0x7fae84101fd0 2024-03-15T12:55:41.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.359+0000 7fae88fe2640 1 -- 172.21.15.2:0/3879088148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8419f6d0 con 0x7fae84101fd0 2024-03-15T12:55:41.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.359+0000 7fae737fe640 1 -- 172.21.15.2:0/3879088148 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c010690 con 0x7fae84101fd0 2024-03-15T12:55:41.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.359+0000 7fae88fe2640 1 -- 172.21.15.2:0/3879088148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8419fbe0 con 0x7fae84101fd0 2024-03-15T12:55:41.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.360+0000 7fae88fe2640 1 -- 172.21.15.2:0/3879088148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8410be00 con 0x7fae84101fd0 2024-03-15T12:55:41.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.361+0000 7fae737fe640 1 -- 172.21.15.2:0/3879088148 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fae7c004df0 con 0x7fae84101fd0 2024-03-15T12:55:41.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.361+0000 7fae737fe640 1 --2- 172.21.15.2:0/3879088148 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fae6c0a1a30 0x7fae6c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:41.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.361+0000 7fae8259c640 1 --2- 172.21.15.2:0/3879088148 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fae6c0a1a30 0x7fae6c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:41.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.361+0000 7fae737fe640 1 -- 172.21.15.2:0/3879088148 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fae7c014070 con 0x7fae84101fd0 2024-03-15T12:55:41.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.362+0000 7fae8259c640 1 --2- 172.21.15.2:0/3879088148 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fae6c0a1a30 0x7fae6c0a3ef0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fae7402c740 tx=0x7fae74005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:41.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.365+0000 7fae737fe640 1 -- 172.21.15.2:0/3879088148 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fae7c0971b0 con 0x7fae84101fd0 2024-03-15T12:55:41.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.585+0000 7fae88fe2640 1 -- 172.21.15.2:0/3879088148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae8410bf90 con 0x7fae84101fd0 2024-03-15T12:55:41.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.586+0000 7fae737fe640 1 -- 172.21.15.2:0/3879088148 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fae7c09a870 con 0x7fae84101fd0 2024-03-15T12:55:41.572 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:55:41.572 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-15T12:55:41.573 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 1 2024-03-15T12:55:41.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.588+0000 7fae88fe2640 1 -- 172.21.15.2:0/3879088148 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fae6c0a1a30 msgr2=0x7fae6c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:41.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.588+0000 7fae88fe2640 1 --2- 172.21.15.2:0/3879088148 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fae6c0a1a30 0x7fae6c0a3ef0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fae7402c740 tx=0x7fae74005fd0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.588+0000 7fae88fe2640 1 -- 172.21.15.2:0/3879088148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae84101fd0 msgr2=0x7fae8419eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:41.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.588+0000 7fae88fe2640 1 --2- 172.21.15.2:0/3879088148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae84101fd0 0x7fae8419eb80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00eb10 tx=0x7fae7c00c2d0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.588+0000 7fae88fe2640 1 -- 172.21.15.2:0/3879088148 shutdown_connections 2024-03-15T12:55:41.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.588+0000 7fae88fe2640 1 --2- 172.21.15.2:0/3879088148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae84101fd0 0x7fae8419eb80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.588+0000 7fae88fe2640 1 --2- 172.21.15.2:0/3879088148 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fae6c0a1a30 0x7fae6c0a3ef0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.588+0000 7fae88fe2640 1 --2- 172.21.15.2:0/3879088148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae84101610 0x7fae84197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.588+0000 7fae88fe2640 1 --2- 172.21.15.2:0/3879088148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae84100420 0x7fae841975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:41.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.588+0000 7fae88fe2640 1 -- 172.21.15.2:0/3879088148 >> 172.21.15.2:0/3879088148 conn(0x7fae840fb9d0 msgr2=0x7fae84069b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:41.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.588+0000 7fae88fe2640 1 -- 172.21.15.2:0/3879088148 shutdown_connections 2024-03-15T12:55:41.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:41.588+0000 7fae88fe2640 1 -- 172.21.15.2:0/3879088148 wait complete. 2024-03-15T12:55:41.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:41 smithi002 ceph-mon[32060]: Deploying daemon mds.l on smithi184 2024-03-15T12:55:41.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:41 smithi002 ceph-mon[32060]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:41.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:41 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/983076079' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:55:41.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:41 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2106192270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:55:41.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:41 smithi184 ceph-mon[35993]: Deploying daemon mds.l on smithi184 2024-03-15T12:55:41.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:41 smithi184 ceph-mon[35993]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:41.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:41 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/983076079' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:55:41.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:41 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2106192270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:55:41.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:41 smithi149 ceph-mon[35927]: Deploying daemon mds.l on smithi184 2024-03-15T12:55:41.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:41 smithi149 ceph-mon[35927]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:41.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:41 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/983076079' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:55:41.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:41 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2106192270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:55:42.513 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 12:55:42 smithi184 systemd[1]: Starting Ceph mds.l for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T12:55:42.789 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:42 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3879088148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:42.789 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 12:55:42 smithi184 podman[66035]: 2024-03-15 12:55:42.430708525 +0000 UTC m=+0.017543914 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T12:55:42.789 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 12:55:42 smithi184 podman[66035]: 2024-03-15 12:55:42.550681127 +0000 UTC m=+0.137516548 container create a8956d8f1c6f7ff5e2a7e4a03088f14110e9b781a3607b34d1522aab1bb0004a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-l, RELEASE=wip-lusov-qdb-squid-aba3c1d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-15T12:55:42.789 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 12:55:42 smithi184 podman[66035]: 2024-03-15 12:55:42.803836526 +0000 UTC m=+0.390671897 container init a8956d8f1c6f7ff5e2a7e4a03088f14110e9b781a3607b34d1522aab1bb0004a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T12:55:42.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:42 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3879088148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:42.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:42 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3879088148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:43.059 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 12:55:42 smithi184 podman[66035]: 2024-03-15 12:55:42.806450822 +0000 UTC m=+0.393286201 container start a8956d8f1c6f7ff5e2a7e4a03088f14110e9b781a3607b34d1522aab1bb0004a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.schema-version=1.0) 2024-03-15T12:55:43.059 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 12:55:42 smithi184 ceph-mds[66054]: -- 172.21.15.184:0/3900009423 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55936e55a1a0 con 0x55936e530800 2024-03-15T12:55:43.059 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 12:55:42 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-l[66050]: starting mds.l at 2024-03-15T12:55:43.060 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 12:55:42 smithi184 bash[66035]: a8956d8f1c6f7ff5e2a7e4a03088f14110e9b781a3607b34d1522aab1bb0004a 2024-03-15T12:55:43.060 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 12:55:42 smithi184 systemd[1]: Started Ceph mds.l for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:55:43.586 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:55:43.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.715+0000 7f2fa69f4640 1 -- 172.21.15.2:0/1847970733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fa0100410 msgr2=0x7f2fa0100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:43.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.715+0000 7f2fa69f4640 1 --2- 172.21.15.2:0/1847970733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fa0100410 0x7f2fa0100830 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009900 tx=0x7f2f9002bc30 comp rx=0 tx=0).stop 2024-03-15T12:55:43.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.715+0000 7f2fa69f4640 1 -- 172.21.15.2:0/1847970733 shutdown_connections 2024-03-15T12:55:43.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.715+0000 7f2fa69f4640 1 --2- 172.21.15.2:0/1847970733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fa0101fc0 0x7f2fa0108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:43.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.716+0000 7f2fa69f4640 1 --2- 172.21.15.2:0/1847970733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fa0101600 0x7f2fa0101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:43.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.716+0000 7f2fa69f4640 1 --2- 172.21.15.2:0/1847970733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fa0100410 0x7f2fa0100830 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:43.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.716+0000 7f2fa69f4640 1 -- 172.21.15.2:0/1847970733 >> 172.21.15.2:0/1847970733 conn(0x7f2fa00fb980 msgr2=0x7f2fa00fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:43.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.716+0000 7f2fa69f4640 1 -- 172.21.15.2:0/1847970733 shutdown_connections 2024-03-15T12:55:43.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.716+0000 7f2fa69f4640 1 -- 172.21.15.2:0/1847970733 wait complete. 2024-03-15T12:55:43.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.716+0000 7f2fa69f4640 1 Processor -- start 2024-03-15T12:55:43.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.716+0000 7f2fa69f4640 1 -- start start 2024-03-15T12:55:43.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.716+0000 7f2fa69f4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fa0100410 0x7f2fa0197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:43.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.716+0000 7f2fa69f4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fa0101600 0x7f2fa0197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:43.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.716+0000 7f2fa69f4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fa0101fc0 0x7f2fa019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:43.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.716+0000 7f2fa69f4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa019f120 con 0x7f2fa0101fc0 2024-03-15T12:55:43.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2fa69f4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa019f290 con 0x7f2fa0101600 2024-03-15T12:55:43.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2fa69f4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa019f450 con 0x7f2fa0100410 2024-03-15T12:55:43.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2fa4f91640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fa0101fc0 0x7f2fa019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:43.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2fa4f91640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fa0101fc0 0x7f2fa019ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59956/0 (socket says 172.21.15.2:59956) 2024-03-15T12:55:43.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2fa4f91640 1 -- 172.21.15.2:0/3300644176 learned_addr learned my addr 172.21.15.2:0/3300644176 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:55:43.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2f9ffff640 1 --2- 172.21.15.2:0/3300644176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fa0100410 0x7f2fa0197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:43.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2f9f7fe640 1 --2- 172.21.15.2:0/3300644176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fa0101600 0x7f2fa0197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:43.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2fa4f91640 1 -- 172.21.15.2:0/3300644176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fa0100410 msgr2=0x7f2fa0197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:43.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2fa4f91640 1 --2- 172.21.15.2:0/3300644176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fa0100410 0x7f2fa0197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:43.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2fa4f91640 1 -- 172.21.15.2:0/3300644176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fa0101600 msgr2=0x7f2fa0197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:43.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2fa4f91640 1 --2- 172.21.15.2:0/3300644176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fa0101600 0x7f2fa0197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:43.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2fa4f91640 1 -- 172.21.15.2:0/3300644176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f90009560 con 0x7f2fa0101fc0 2024-03-15T12:55:43.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2fa4f91640 1 --2- 172.21.15.2:0/3300644176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fa0101fc0 0x7f2fa019ebe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800eb10 tx=0x7f2f9800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:43.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2f9d7fa640 1 -- 172.21.15.2:0/3300644176 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98004b20 con 0x7f2fa0101fc0 2024-03-15T12:55:43.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2f9d7fa640 1 -- 172.21.15.2:0/3300644176 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2f98004c80 con 0x7f2fa0101fc0 2024-03-15T12:55:43.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2fa69f4640 1 -- 172.21.15.2:0/3300644176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa019f730 con 0x7f2fa0101fc0 2024-03-15T12:55:43.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2f9d7fa640 1 -- 172.21.15.2:0/3300644176 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98010690 con 0x7f2fa0101fc0 2024-03-15T12:55:43.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.717+0000 7f2fa69f4640 1 -- 172.21.15.2:0/3300644176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa019fbf0 con 0x7f2fa0101fc0 2024-03-15T12:55:43.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.718+0000 7f2fa69f4640 1 -- 172.21.15.2:0/3300644176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa010bdf0 con 0x7f2fa0101fc0 2024-03-15T12:55:43.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.720+0000 7f2f9d7fa640 1 -- 172.21.15.2:0/3300644176 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f2f98004df0 con 0x7f2fa0101fc0 2024-03-15T12:55:43.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.720+0000 7f2f9d7fa640 1 --2- 172.21.15.2:0/3300644176 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2f880a1a30 0x7f2f880a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:43.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.720+0000 7f2f9ffff640 1 --2- 172.21.15.2:0/3300644176 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2f880a1a30 0x7f2f880a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:43.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.721+0000 7f2f9d7fa640 1 -- 172.21.15.2:0/3300644176 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2f98014070 con 0x7f2fa0101fc0 2024-03-15T12:55:43.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.721+0000 7f2f9ffff640 1 --2- 172.21.15.2:0/3300644176 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2f880a1a30 0x7f2f880a3ef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2f9002c740 tx=0x7f2f90005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:43.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.724+0000 7f2f9d7fa640 1 -- 172.21.15.2:0/3300644176 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2f98096e40 con 0x7f2fa0101fc0 2024-03-15T12:55:43.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:43 smithi002 ceph-mon[32060]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:43 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.184:6832/730881833,v1:172.21.15.184:6833/730881833] up:boot 2024-03-15T12:55:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:43 smithi002 ceph-mon[32060]: fsmap 1 up:standby 2024-03-15T12:55:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:43 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-15T12:55:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:43 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:43 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:43 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:43 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:43 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:43 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:43.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:43 smithi184 ceph-mon[35993]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:43.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:43 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.184:6832/730881833,v1:172.21.15.184:6833/730881833] up:boot 2024-03-15T12:55:43.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:43 smithi184 ceph-mon[35993]: fsmap 1 up:standby 2024-03-15T12:55:43.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:43 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-15T12:55:43.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:43 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:43.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:43 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:43.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:43 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:43.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:43 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:43.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:43 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:43.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:43 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:43.847 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:43 smithi149 ceph-mon[35927]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:43.847 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:43 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.184:6832/730881833,v1:172.21.15.184:6833/730881833] up:boot 2024-03-15T12:55:43.847 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:43 smithi149 ceph-mon[35927]: fsmap 1 up:standby 2024-03-15T12:55:43.847 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:43 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-15T12:55:43.847 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:43 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:43.847 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:43 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:43.848 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:43 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:43.848 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:43 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:43.848 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:43 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:43.848 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:43 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.949+0000 7f2fa69f4640 1 -- 172.21.15.2:0/3300644176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fa010bf80 con 0x7f2fa0101fc0 2024-03-15T12:55:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.949+0000 7f2f9d7fa640 1 -- 172.21.15.2:0/3300644176 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7f2f9809a500 con 0x7f2fa0101fc0 2024-03-15T12:55:43.936 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:55:43.936 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-15T12:55:43.936 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 2 2024-03-15T12:55:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.951+0000 7f2fa69f4640 1 -- 172.21.15.2:0/3300644176 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2f880a1a30 msgr2=0x7f2f880a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.951+0000 7f2fa69f4640 1 --2- 172.21.15.2:0/3300644176 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2f880a1a30 0x7f2f880a3ef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2f9002c740 tx=0x7f2f90005fd0 comp rx=0 tx=0).stop 2024-03-15T12:55:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.951+0000 7f2fa69f4640 1 -- 172.21.15.2:0/3300644176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fa0101fc0 msgr2=0x7f2fa019ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.951+0000 7f2fa69f4640 1 --2- 172.21.15.2:0/3300644176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fa0101fc0 0x7f2fa019ebe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800eb10 tx=0x7f2f9800c2d0 comp rx=0 tx=0).stop 2024-03-15T12:55:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.951+0000 7f2fa69f4640 1 -- 172.21.15.2:0/3300644176 shutdown_connections 2024-03-15T12:55:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.951+0000 7f2fa69f4640 1 --2- 172.21.15.2:0/3300644176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fa0101fc0 0x7f2fa019ebe0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.951+0000 7f2fa69f4640 1 --2- 172.21.15.2:0/3300644176 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2f880a1a30 0x7f2f880a3ef0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.951+0000 7f2fa69f4640 1 --2- 172.21.15.2:0/3300644176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fa0101600 0x7f2fa0197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.951+0000 7f2fa69f4640 1 --2- 172.21.15.2:0/3300644176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fa0100410 0x7f2fa0197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.951+0000 7f2fa69f4640 1 -- 172.21.15.2:0/3300644176 >> 172.21.15.2:0/3300644176 conn(0x7f2fa00fb980 msgr2=0x7f2fa00fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:43.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.951+0000 7f2fa69f4640 1 -- 172.21.15.2:0/3300644176 shutdown_connections 2024-03-15T12:55:43.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:43.951+0000 7f2fa69f4640 1 -- 172.21.15.2:0/3300644176 wait complete. 2024-03-15T12:55:44.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:44 smithi149 ceph-mon[35927]: Deploying daemon mds.k on smithi149 2024-03-15T12:55:44.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:44 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3300644176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:44.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:44 smithi002 ceph-mon[32060]: Deploying daemon mds.k on smithi149 2024-03-15T12:55:44.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:44 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3300644176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:44.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:44 smithi184 ceph-mon[35993]: Deploying daemon mds.k on smithi149 2024-03-15T12:55:44.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:44 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3300644176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:45.360 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 12:55:45 smithi149 systemd[1]: Starting Ceph mds.k for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T12:55:45.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:45 smithi149 ceph-mon[35927]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:45.707 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 12:55:45 smithi149 podman[64894]: 2024-03-15 12:55:45.358411827 +0000 UTC m=+0.017884391 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T12:55:45.707 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 12:55:45 smithi149 podman[64894]: 2024-03-15 12:55:45.47739067 +0000 UTC m=+0.136863215 container create 0103c35aed93e9382c6732c6aa0c11754deed9e116f5211c12e5d3557ba0e35c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-k, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-15T12:55:45.707 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 12:55:45 smithi149 podman[64894]: 2024-03-15 12:55:45.721623234 +0000 UTC m=+0.381095784 container init 0103c35aed93e9382c6732c6aa0c11754deed9e116f5211c12e5d3557ba0e35c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-k, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-15T12:55:45.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:45 smithi002 ceph-mon[32060]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:45.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:45 smithi184 ceph-mon[35993]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:45.950 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:55:46.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.074+0000 7fd23806b640 1 -- 172.21.15.2:0/1340321310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2301048c0 msgr2=0x7fd230106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:46.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.074+0000 7fd23806b640 1 --2- 172.21.15.2:0/1340321310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2301048c0 0x7fd230106cb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd220009900 tx=0x7fd22002bc30 comp rx=0 tx=0).stop 2024-03-15T12:55:46.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.074+0000 7fd23806b640 1 -- 172.21.15.2:0/1340321310 shutdown_connections 2024-03-15T12:55:46.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.074+0000 7fd23806b640 1 --2- 172.21.15.2:0/1340321310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2301071f0 0x7fd230109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:46.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.074+0000 7fd23806b640 1 --2- 172.21.15.2:0/1340321310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2301048c0 0x7fd230106cb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:46.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.074+0000 7fd23806b640 1 --2- 172.21.15.2:0/1340321310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd230101f90 0x7fd230104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:46.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.074+0000 7fd23806b640 1 -- 172.21.15.2:0/1340321310 >> 172.21.15.2:0/1340321310 conn(0x7fd2300fb980 msgr2=0x7fd2300fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:46.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.074+0000 7fd23806b640 1 -- 172.21.15.2:0/1340321310 shutdown_connections 2024-03-15T12:55:46.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.074+0000 7fd23806b640 1 -- 172.21.15.2:0/1340321310 wait complete. 2024-03-15T12:55:46.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.075+0000 7fd23806b640 1 Processor -- start 2024-03-15T12:55:46.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.075+0000 7fd23806b640 1 -- start start 2024-03-15T12:55:46.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.075+0000 7fd23806b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd230101f90 0x7fd230101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:46.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.075+0000 7fd23806b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2301048c0 0x7fd2300fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:46.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.075+0000 7fd23806b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2301071f0 0x7fd230100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:46.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.075+0000 7fd23806b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd230100be0 con 0x7fd230101f90 2024-03-15T12:55:46.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.075+0000 7fd23806b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd230100d50 con 0x7fd2301071f0 2024-03-15T12:55:46.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.075+0000 7fd23806b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2301a1660 con 0x7fd2301048c0 2024-03-15T12:55:46.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.075+0000 7fd235e07640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd230101f90 0x7fd230101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:46.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.075+0000 7fd235e07640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd230101f90 0x7fd230101900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59978/0 (socket says 172.21.15.2:59978) 2024-03-15T12:55:46.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.075+0000 7fd235e07640 1 -- 172.21.15.2:0/3814378435 learned_addr learned my addr 172.21.15.2:0/3814378435 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:55:46.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.075+0000 7fd235606640 1 --2- 172.21.15.2:0/3814378435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2301048c0 0x7fd2300fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:46.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.076+0000 7fd236608640 1 --2- 172.21.15.2:0/3814378435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2301071f0 0x7fd230100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:46.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.076+0000 7fd235e07640 1 -- 172.21.15.2:0/3814378435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2301048c0 msgr2=0x7fd2300fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:46.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.076+0000 7fd235e07640 1 --2- 172.21.15.2:0/3814378435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2301048c0 0x7fd2300fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:46.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.076+0000 7fd235e07640 1 -- 172.21.15.2:0/3814378435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2301071f0 msgr2=0x7fd230100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:46.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.076+0000 7fd235e07640 1 --2- 172.21.15.2:0/3814378435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2301071f0 0x7fd230100480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:46.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.076+0000 7fd235e07640 1 -- 172.21.15.2:0/3814378435 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd220009560 con 0x7fd230101f90 2024-03-15T12:55:46.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.076+0000 7fd236608640 1 --2- 172.21.15.2:0/3814378435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2301071f0 0x7fd230100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T12:55:46.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.076+0000 7fd235e07640 1 --2- 172.21.15.2:0/3814378435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd230101f90 0x7fd230101900 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd22c00d7a0 tx=0x7fd22c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:46.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.076+0000 7fd226ffd640 1 -- 172.21.15.2:0/3814378435 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c009c30 con 0x7fd230101f90 2024-03-15T12:55:46.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.076+0000 7fd23806b640 1 -- 172.21.15.2:0/3814378435 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2301a1910 con 0x7fd230101f90 2024-03-15T12:55:46.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.076+0000 7fd226ffd640 1 -- 172.21.15.2:0/3814378435 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd22c00deb0 con 0x7fd230101f90 2024-03-15T12:55:46.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.076+0000 7fd226ffd640 1 -- 172.21.15.2:0/3814378435 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c004e70 con 0x7fd230101f90 2024-03-15T12:55:46.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.076+0000 7fd23806b640 1 -- 172.21.15.2:0/3814378435 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2301a1da0 con 0x7fd230101f90 2024-03-15T12:55:46.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.077+0000 7fd23806b640 1 -- 172.21.15.2:0/3814378435 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2300fd470 con 0x7fd230101f90 2024-03-15T12:55:46.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.078+0000 7fd226ffd640 1 -- 172.21.15.2:0/3814378435 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fd22c005000 con 0x7fd230101f90 2024-03-15T12:55:46.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.079+0000 7fd226ffd640 1 --2- 172.21.15.2:0/3814378435 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd21c0a1a30 0x7fd21c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:46.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.079+0000 7fd226ffd640 1 -- 172.21.15.2:0/3814378435 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd22c0ccf80 con 0x7fd230101f90 2024-03-15T12:55:46.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.079+0000 7fd235606640 1 --2- 172.21.15.2:0/3814378435 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd21c0a1a30 0x7fd21c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:46.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.079+0000 7fd235606640 1 --2- 172.21.15.2:0/3814378435 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd21c0a1a30 0x7fd21c0a3ef0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd220009230 tx=0x7fd220000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:46.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.083+0000 7fd226ffd640 1 -- 172.21.15.2:0/3814378435 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd22c096590 con 0x7fd230101f90 2024-03-15T12:55:46.111 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 12:55:45 smithi149 podman[64894]: 2024-03-15 12:55:45.724575112 +0000 UTC m=+0.384047667 container start 0103c35aed93e9382c6732c6aa0c11754deed9e116f5211c12e5d3557ba0e35c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-15T12:55:46.111 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 12:55:45 smithi149 ceph-mds[64913]: -- 172.21.15.149:0/2766724989 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560e2fb861a0 con 0x560e2fb5d000 2024-03-15T12:55:46.111 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 12:55:45 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-k[64909]: starting mds.k at 2024-03-15T12:55:46.111 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 12:55:45 smithi149 bash[64894]: 0103c35aed93e9382c6732c6aa0c11754deed9e116f5211c12e5d3557ba0e35c 2024-03-15T12:55:46.111 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 12:55:45 smithi149 systemd[1]: Started Ceph mds.k for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:55:46.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.306+0000 7fd23806b640 1 -- 172.21.15.2:0/3814378435 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd23002cf60 con 0x7fd230101f90 2024-03-15T12:55:46.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.306+0000 7fd226ffd640 1 -- 172.21.15.2:0/3814378435 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7fd22c020090 con 0x7fd230101f90 2024-03-15T12:55:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:55:46.293 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-15T12:55:46.293 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 2 2024-03-15T12:55:46.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.308+0000 7fd23806b640 1 -- 172.21.15.2:0/3814378435 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd21c0a1a30 msgr2=0x7fd21c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:46.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.308+0000 7fd23806b640 1 --2- 172.21.15.2:0/3814378435 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd21c0a1a30 0x7fd21c0a3ef0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd220009230 tx=0x7fd220000bc0 comp rx=0 tx=0).stop 2024-03-15T12:55:46.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.308+0000 7fd23806b640 1 -- 172.21.15.2:0/3814378435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd230101f90 msgr2=0x7fd230101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:46.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.308+0000 7fd23806b640 1 --2- 172.21.15.2:0/3814378435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd230101f90 0x7fd230101900 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd22c00d7a0 tx=0x7fd22c00dc60 comp rx=0 tx=0).stop 2024-03-15T12:55:46.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.308+0000 7fd23806b640 1 -- 172.21.15.2:0/3814378435 shutdown_connections 2024-03-15T12:55:46.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.308+0000 7fd23806b640 1 --2- 172.21.15.2:0/3814378435 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd21c0a1a30 0x7fd21c0a3ef0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:46.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.308+0000 7fd23806b640 1 --2- 172.21.15.2:0/3814378435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2301071f0 0x7fd230100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:46.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.308+0000 7fd23806b640 1 --2- 172.21.15.2:0/3814378435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2301048c0 0x7fd2300fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:46.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.308+0000 7fd23806b640 1 --2- 172.21.15.2:0/3814378435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd230101f90 0x7fd230101900 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:46.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.308+0000 7fd23806b640 1 -- 172.21.15.2:0/3814378435 >> 172.21.15.2:0/3814378435 conn(0x7fd2300fb980 msgr2=0x7fd230102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:46.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.308+0000 7fd23806b640 1 -- 172.21.15.2:0/3814378435 shutdown_connections 2024-03-15T12:55:46.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:46.308+0000 7fd23806b640 1 -- 172.21.15.2:0/3814378435 wait complete. 2024-03-15T12:55:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:46 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:46 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:46 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:46 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:46 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:46 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:47.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:46 smithi149 ceph-mon[35927]: Deploying daemon mds.j on smithi002 2024-03-15T12:55:47.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:46 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3814378435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:47.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:46 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] up:boot 2024-03-15T12:55:47.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:46 smithi149 ceph-mon[35927]: fsmap 2 up:standby 2024-03-15T12:55:47.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:46 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-15T12:55:47.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:46 smithi149 ceph-mon[35927]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:47.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:46 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:47.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:46 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:47.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:46 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:47.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:46 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:47.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:46 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:47.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:46 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:47.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:46 smithi002 ceph-mon[32060]: Deploying daemon mds.j on smithi002 2024-03-15T12:55:47.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:46 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3814378435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:47.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:46 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] up:boot 2024-03-15T12:55:47.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:46 smithi002 ceph-mon[32060]: fsmap 2 up:standby 2024-03-15T12:55:47.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:46 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-15T12:55:47.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:46 smithi002 ceph-mon[32060]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:47.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:46 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:47.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:46 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:47.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:46 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:47.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:46 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:47.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:46 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:47.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:46 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:47.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:46 smithi184 ceph-mon[35993]: Deploying daemon mds.j on smithi002 2024-03-15T12:55:47.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:46 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3814378435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:47.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:46 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] up:boot 2024-03-15T12:55:47.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:46 smithi184 ceph-mon[35993]: fsmap 2 up:standby 2024-03-15T12:55:47.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:46 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-15T12:55:47.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:46 smithi184 ceph-mon[35993]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:48.303 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 12:55:48 smithi002 systemd[1]: Starting Ceph mds.j for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T12:55:48.306 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:55:48.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.426+0000 7f207892c640 1 -- 172.21.15.2:0/645862414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20740fff50 msgr2=0x7f2074109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:48.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.426+0000 7f207892c640 1 --2- 172.21.15.2:0/645862414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20740fff50 0x7f2074109b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f206c009ca0 tx=0x7f206c02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:55:48.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.426+0000 7f207892c640 1 -- 172.21.15.2:0/645862414 shutdown_connections 2024-03-15T12:55:48.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.426+0000 7f207892c640 1 --2- 172.21.15.2:0/645862414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20740fff50 0x7f2074109b00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:48.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.426+0000 7f207892c640 1 --2- 172.21.15.2:0/645862414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20740ff590 0x7f20740ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:48.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.426+0000 7f207892c640 1 --2- 172.21.15.2:0/645862414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2074102b20 0x7f2074102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:48.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.426+0000 7f207892c640 1 -- 172.21.15.2:0/645862414 >> 172.21.15.2:0/645862414 conn(0x7f20740fb110 msgr2=0x7f20740fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:48.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.426+0000 7f207892c640 1 -- 172.21.15.2:0/645862414 shutdown_connections 2024-03-15T12:55:48.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.426+0000 7f207892c640 1 -- 172.21.15.2:0/645862414 wait complete. 2024-03-15T12:55:48.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f207892c640 1 Processor -- start 2024-03-15T12:55:48.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f207892c640 1 -- start start 2024-03-15T12:55:48.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f207892c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20740ff590 0x7f2074197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:48.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f207892c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20740fff50 0x7f2074197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:48.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f207892c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2074102b20 0x7f207419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:48.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f207892c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207419f050 con 0x7f20740fff50 2024-03-15T12:55:48.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f207892c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207419f1c0 con 0x7f2074102b20 2024-03-15T12:55:48.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f207892c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207419f380 con 0x7f20740ff590 2024-03-15T12:55:48.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f206bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20740fff50 0x7f2074197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:48.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f206bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20740fff50 0x7f2074197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46028/0 (socket says 172.21.15.2:46028) 2024-03-15T12:55:48.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f206bfff640 1 -- 172.21.15.2:0/4011668511 learned_addr learned my addr 172.21.15.2:0/4011668511 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:55:48.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f2072d9d640 1 --2- 172.21.15.2:0/4011668511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2074102b20 0x7f207419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:48.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f207259c640 1 --2- 172.21.15.2:0/4011668511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20740ff590 0x7f2074197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:48.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f206bfff640 1 -- 172.21.15.2:0/4011668511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20740ff590 msgr2=0x7f2074197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:48.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f206bfff640 1 --2- 172.21.15.2:0/4011668511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20740ff590 0x7f2074197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:48.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f206bfff640 1 -- 172.21.15.2:0/4011668511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2074102b20 msgr2=0x7f207419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:48.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f206bfff640 1 --2- 172.21.15.2:0/4011668511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2074102b20 0x7f207419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:48.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f206bfff640 1 -- 172.21.15.2:0/4011668511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206c009900 con 0x7f20740fff50 2024-03-15T12:55:48.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f207259c640 1 --2- 172.21.15.2:0/4011668511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20740ff590 0x7f2074197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:55:48.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.427+0000 7f206bfff640 1 --2- 172.21.15.2:0/4011668511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20740fff50 0x7f2074197aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f205c00d770 tx=0x7f205c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:48.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.428+0000 7f206b7fe640 1 -- 172.21.15.2:0/4011668511 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c009c30 con 0x7f20740fff50 2024-03-15T12:55:48.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.428+0000 7f207892c640 1 -- 172.21.15.2:0/4011668511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207419f660 con 0x7f20740fff50 2024-03-15T12:55:48.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.428+0000 7f206b7fe640 1 -- 172.21.15.2:0/4011668511 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f205c00de80 con 0x7f20740fff50 2024-03-15T12:55:48.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.428+0000 7f206b7fe640 1 -- 172.21.15.2:0/4011668511 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c004e50 con 0x7f20740fff50 2024-03-15T12:55:48.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.428+0000 7f207892c640 1 -- 172.21.15.2:0/4011668511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207419fb20 con 0x7f20740fff50 2024-03-15T12:55:48.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.429+0000 7f207892c640 1 -- 172.21.15.2:0/4011668511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20741089e0 con 0x7f20740fff50 2024-03-15T12:55:48.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.430+0000 7f206b7fe640 1 -- 172.21.15.2:0/4011668511 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f205c005000 con 0x7f20740fff50 2024-03-15T12:55:48.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.430+0000 7f206b7fe640 1 --2- 172.21.15.2:0/4011668511 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20600a1a30 0x7f20600a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:48.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.430+0000 7f207259c640 1 --2- 172.21.15.2:0/4011668511 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20600a1a30 0x7f20600a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:48.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.430+0000 7f206b7fe640 1 -- 172.21.15.2:0/4011668511 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f205c0cca00 con 0x7f20740fff50 2024-03-15T12:55:48.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.431+0000 7f207259c640 1 --2- 172.21.15.2:0/4011668511 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20600a1a30 0x7f20600a3ef0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2074069820 tx=0x7f2064005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:48.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.435+0000 7f206b7fe640 1 -- 172.21.15.2:0/4011668511 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f205c0960b0 con 0x7f20740fff50 2024-03-15T12:55:48.577 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 12:55:48 smithi002 podman[72781]: 2024-03-15 12:55:48.242888421 +0000 UTC m=+0.017255243 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T12:55:48.577 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 12:55:48 smithi002 podman[72781]: 2024-03-15 12:55:48.360927253 +0000 UTC m=+0.135294059 container create 8302642d79b05c6abb4bc8de3bc36f98f146dafad0d048ce62febc8b27411c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-j, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-15T12:55:48.577 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 12:55:48 smithi002 podman[72781]: 2024-03-15 12:55:48.588887688 +0000 UTC m=+0.363254508 container init 8302642d79b05c6abb4bc8de3bc36f98f146dafad0d048ce62febc8b27411c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-15T12:55:48.577 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 12:55:48 smithi002 podman[72781]: 2024-03-15 12:55:48.591583635 +0000 UTC m=+0.365950444 container start 8302642d79b05c6abb4bc8de3bc36f98f146dafad0d048ce62febc8b27411c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-j, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-15T12:55:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.658+0000 7f207892c640 1 -- 172.21.15.2:0/4011668511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2074108450 con 0x7f20740fff50 2024-03-15T12:55:48.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.659+0000 7f206b7fe640 1 -- 172.21.15.2:0/4011668511 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f205c020860 con 0x7f20740fff50 2024-03-15T12:55:48.645 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:55:48.646 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-15T12:55:48.646 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 3 2024-03-15T12:55:48.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.661+0000 7f207892c640 1 -- 172.21.15.2:0/4011668511 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20600a1a30 msgr2=0x7f20600a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.661+0000 7f207892c640 1 --2- 172.21.15.2:0/4011668511 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20600a1a30 0x7f20600a3ef0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2074069820 tx=0x7f2064005f90 comp rx=0 tx=0).stop 2024-03-15T12:55:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.661+0000 7f207892c640 1 -- 172.21.15.2:0/4011668511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20740fff50 msgr2=0x7f2074197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.661+0000 7f207892c640 1 --2- 172.21.15.2:0/4011668511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20740fff50 0x7f2074197aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f205c00d770 tx=0x7f205c00dc30 comp rx=0 tx=0).stop 2024-03-15T12:55:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.661+0000 7f207892c640 1 -- 172.21.15.2:0/4011668511 shutdown_connections 2024-03-15T12:55:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.661+0000 7f207892c640 1 --2- 172.21.15.2:0/4011668511 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20600a1a30 0x7f20600a3ef0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.661+0000 7f207892c640 1 --2- 172.21.15.2:0/4011668511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2074102b20 0x7f207419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.661+0000 7f207892c640 1 --2- 172.21.15.2:0/4011668511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20740fff50 0x7f2074197aa0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.661+0000 7f207892c640 1 --2- 172.21.15.2:0/4011668511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20740ff590 0x7f2074197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.661+0000 7f207892c640 1 -- 172.21.15.2:0/4011668511 >> 172.21.15.2:0/4011668511 conn(0x7f20740fb110 msgr2=0x7f2074105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.661+0000 7f207892c640 1 -- 172.21.15.2:0/4011668511 shutdown_connections 2024-03-15T12:55:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:48.661+0000 7f207892c640 1 -- 172.21.15.2:0/4011668511 wait complete. 2024-03-15T12:55:49.051 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:48 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/4011668511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:49.051 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:48 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.2:6834/245353007,v1:172.21.15.2:6835/245353007] up:boot 2024-03-15T12:55:49.051 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:48 smithi149 ceph-mon[35927]: fsmap 3 up:standby 2024-03-15T12:55:49.051 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:48 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-15T12:55:49.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:48 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/4011668511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:49.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:48 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.2:6834/245353007,v1:172.21.15.2:6835/245353007] up:boot 2024-03-15T12:55:49.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:48 smithi002 ceph-mon[32060]: fsmap 3 up:standby 2024-03-15T12:55:49.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:48 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-15T12:55:49.054 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 12:55:48 smithi002 ceph-mds[72839]: -- 172.21.15.2:0/1874872270 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56204b64c1a0 con 0x56204b623000 2024-03-15T12:55:49.054 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 12:55:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-j[72835]: starting mds.j at 2024-03-15T12:55:49.054 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 12:55:48 smithi002 bash[72781]: 8302642d79b05c6abb4bc8de3bc36f98f146dafad0d048ce62febc8b27411c47 2024-03-15T12:55:49.054 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 12:55:48 smithi002 systemd[1]: Started Ceph mds.j for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:55:49.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:48 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/4011668511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:49.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:48 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.2:6834/245353007,v1:172.21.15.2:6835/245353007] up:boot 2024-03-15T12:55:49.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:48 smithi184 ceph-mon[35993]: fsmap 3 up:standby 2024-03-15T12:55:49.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:48 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-15T12:55:50.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:49 smithi002 ceph-mon[32060]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:50.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:50.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:50.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:50.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:50.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:50.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:50.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:49 smithi184 ceph-mon[35993]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:50.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:50.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:50.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:50.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:50.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:50.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:50.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:49 smithi149 ceph-mon[35927]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:50.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:50.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:50.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:50.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:50.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:50.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:50.659 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:55:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.798+0000 7f355a247640 1 -- 172.21.15.2:0/1310861105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35540ffde0 msgr2=0x7f3554100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.798+0000 7f355a247640 1 --2- 172.21.15.2:0/1310861105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35540ffde0 0x7f3554100200 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3544009900 tx=0x7f354402bc30 comp rx=0 tx=0).stop 2024-03-15T12:55:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.798+0000 7f355a247640 1 -- 172.21.15.2:0/1310861105 shutdown_connections 2024-03-15T12:55:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.798+0000 7f355a247640 1 --2- 172.21.15.2:0/1310861105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35541010e0 0x7f3554101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:50.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.798+0000 7f355a247640 1 --2- 172.21.15.2:0/1310861105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3554100740 0x7f3554100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.798+0000 7f355a247640 1 --2- 172.21.15.2:0/1310861105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35540ffde0 0x7f3554100200 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.798+0000 7f355a247640 1 -- 172.21.15.2:0/1310861105 >> 172.21.15.2:0/1310861105 conn(0x7f35540fb980 msgr2=0x7f35540fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.798+0000 7f355a247640 1 -- 172.21.15.2:0/1310861105 shutdown_connections 2024-03-15T12:55:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.798+0000 7f355a247640 1 -- 172.21.15.2:0/1310861105 wait complete. 2024-03-15T12:55:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.798+0000 7f355a247640 1 Processor -- start 2024-03-15T12:55:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.798+0000 7f355a247640 1 -- start start 2024-03-15T12:55:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f355a247640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35540ffde0 0x7f35541997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f355a247640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3554100740 0x7f3554199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f355a247640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35541010e0 0x7f35541a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f355a247640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35541a12c0 con 0x7f35540ffde0 2024-03-15T12:55:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f355a247640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35541a1430 con 0x7f35541010e0 2024-03-15T12:55:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f355a247640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35541a15f0 con 0x7f3554100740 2024-03-15T12:55:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f35537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35540ffde0 0x7f35541997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:50.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f35537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35540ffde0 0x7f35541997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46060/0 (socket says 172.21.15.2:46060) 2024-03-15T12:55:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f35537fe640 1 -- 172.21.15.2:0/2330560151 learned_addr learned my addr 172.21.15.2:0/2330560151 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:55:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f3553fff640 1 --2- 172.21.15.2:0/2330560151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35541010e0 0x7f35541a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f3552ffd640 1 --2- 172.21.15.2:0/2330560151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3554100740 0x7f3554199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f35537fe640 1 -- 172.21.15.2:0/2330560151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3554100740 msgr2=0x7f3554199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f35537fe640 1 --2- 172.21.15.2:0/2330560151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3554100740 0x7f3554199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f35537fe640 1 -- 172.21.15.2:0/2330560151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35541010e0 msgr2=0x7f35541a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f35537fe640 1 --2- 172.21.15.2:0/2330560151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35541010e0 0x7f35541a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f35537fe640 1 -- 172.21.15.2:0/2330560151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3544009560 con 0x7f35540ffde0 2024-03-15T12:55:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f3553fff640 1 --2- 172.21.15.2:0/2330560151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35541010e0 0x7f35541a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T12:55:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f35537fe640 1 --2- 172.21.15.2:0/2330560151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35540ffde0 0x7f35541997d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3544005ab0 tx=0x7f3544004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f3550ff9640 1 -- 172.21.15.2:0/2330560151 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35440042d0 con 0x7f35540ffde0 2024-03-15T12:55:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f355a247640 1 -- 172.21.15.2:0/2330560151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35541a1870 con 0x7f35540ffde0 2024-03-15T12:55:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f3550ff9640 1 -- 172.21.15.2:0/2330560151 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3544004430 con 0x7f35540ffde0 2024-03-15T12:55:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f3550ff9640 1 -- 172.21.15.2:0/2330560151 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544003e20 con 0x7f35540ffde0 2024-03-15T12:55:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.799+0000 7f355a247640 1 -- 172.21.15.2:0/2330560151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35541a1dd0 con 0x7f35540ffde0 2024-03-15T12:55:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.801+0000 7f355a247640 1 -- 172.21.15.2:0/2330560151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355410ad00 con 0x7f35540ffde0 2024-03-15T12:55:50.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.801+0000 7f3550ff9640 1 -- 172.21.15.2:0/2330560151 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f35440035f0 con 0x7f35540ffde0 2024-03-15T12:55:50.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.802+0000 7f3550ff9640 1 --2- 172.21.15.2:0/2330560151 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f353c0a1a30 0x7f353c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:50.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.802+0000 7f3552ffd640 1 --2- 172.21.15.2:0/2330560151 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f353c0a1a30 0x7f353c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:50.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.802+0000 7f3550ff9640 1 -- 172.21.15.2:0/2330560151 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f35440ef2f0 con 0x7f35540ffde0 2024-03-15T12:55:50.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.803+0000 7f3552ffd640 1 --2- 172.21.15.2:0/2330560151 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f353c0a1a30 0x7f353c0a3ef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3548005fd0 tx=0x7f3548005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:50.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:50.811+0000 7f3550ff9640 1 -- 172.21.15.2:0/2330560151 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f35440b89a0 con 0x7f35540ffde0 2024-03-15T12:55:51.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:51.037+0000 7f355a247640 1 -- 172.21.15.2:0/2330560151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f355410f0c0 con 0x7f35540ffde0 2024-03-15T12:55:51.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:51.037+0000 7f3550ff9640 1 -- 172.21.15.2:0/2330560151 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2904 (secure 0 0 0) 0x7f355410f0c0 con 0x7f35540ffde0 2024-03-15T12:55:51.024 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:55:51.024 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-15T12:55:51.024 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 4 2024-03-15T12:55:51.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:51.039+0000 7f355a247640 1 -- 172.21.15.2:0/2330560151 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f353c0a1a30 msgr2=0x7f353c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:51.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:51.039+0000 7f355a247640 1 --2- 172.21.15.2:0/2330560151 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f353c0a1a30 0x7f353c0a3ef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3548005fd0 tx=0x7f3548005c70 comp rx=0 tx=0).stop 2024-03-15T12:55:51.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:51.039+0000 7f355a247640 1 -- 172.21.15.2:0/2330560151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35540ffde0 msgr2=0x7f35541997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:51.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:51.039+0000 7f355a247640 1 --2- 172.21.15.2:0/2330560151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35540ffde0 0x7f35541997d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3544005ab0 tx=0x7f3544004e50 comp rx=0 tx=0).stop 2024-03-15T12:55:51.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:51.039+0000 7f355a247640 1 -- 172.21.15.2:0/2330560151 shutdown_connections 2024-03-15T12:55:51.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:51.039+0000 7f355a247640 1 --2- 172.21.15.2:0/2330560151 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f353c0a1a30 0x7f353c0a3ef0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:51.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:51.039+0000 7f355a247640 1 --2- 172.21.15.2:0/2330560151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35541010e0 0x7f35541a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:51.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:51.039+0000 7f355a247640 1 --2- 172.21.15.2:0/2330560151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3554100740 0x7f3554199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:51.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:51.039+0000 7f355a247640 1 --2- 172.21.15.2:0/2330560151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35540ffde0 0x7f35541997d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:51.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:51.040+0000 7f355a247640 1 -- 172.21.15.2:0/2330560151 >> 172.21.15.2:0/2330560151 conn(0x7f35540fb980 msgr2=0x7f35540fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:51.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:51.040+0000 7f355a247640 1 -- 172.21.15.2:0/2330560151 shutdown_connections 2024-03-15T12:55:51.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:51.040+0000 7f355a247640 1 -- 172.21.15.2:0/2330560151 wait complete. 2024-03-15T12:55:51.036 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:50 smithi002 ceph-mon[32060]: Deploying daemon mds.d on smithi002 2024-03-15T12:55:51.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:50 smithi184 ceph-mon[35993]: Deploying daemon mds.d on smithi002 2024-03-15T12:55:51.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:50 smithi149 ceph-mon[35927]: Deploying daemon mds.d on smithi002 2024-03-15T12:55:51.303 INFO:journalctl@ceph.mds.d.smithi002.stdout:Mar 15 12:55:51 smithi002 podman[73175]: 2024-03-15 12:55:51.203204213 +0000 UTC m=+0.018775030 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T12:55:51.303 INFO:journalctl@ceph.mds.d.smithi002.stdout:Mar 15 12:55:51 smithi002 podman[73175]: 2024-03-15 12:55:51.317554435 +0000 UTC m=+0.133125231 container create 2ad74e4d29bb5f3d6b644f35b237330f4f04babd615a52e2df961e1ec061ae87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.build-date=20240307, ceph=True) 2024-03-15T12:55:51.554 INFO:journalctl@ceph.mds.d.smithi002.stdout:Mar 15 12:55:51 smithi002 podman[73175]: 2024-03-15 12:55:51.55360151 +0000 UTC m=+0.369172321 container init 2ad74e4d29bb5f3d6b644f35b237330f4f04babd615a52e2df961e1ec061ae87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T12:55:51.554 INFO:journalctl@ceph.mds.d.smithi002.stdout:Mar 15 12:55:51 smithi002 podman[73175]: 2024-03-15 12:55:51.556276938 +0000 UTC m=+0.371847736 container start 2ad74e4d29bb5f3d6b644f35b237330f4f04babd615a52e2df961e1ec061ae87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-squid-aba3c1d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-15T12:55:52.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:51 smithi002 ceph-mon[32060]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:52.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:51 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2330560151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:52.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:52.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:51 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:52.054 INFO:journalctl@ceph.mds.d.smithi002.stdout:Mar 15 12:55:51 smithi002 ceph-mds[73194]: -- 172.21.15.2:0/4097748384 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5584958481a0 con 0x55849581e800 2024-03-15T12:55:52.054 INFO:journalctl@ceph.mds.d.smithi002.stdout:Mar 15 12:55:51 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-d[73190]: starting mds.d at 2024-03-15T12:55:52.054 INFO:journalctl@ceph.mds.d.smithi002.stdout:Mar 15 12:55:51 smithi002 bash[73175]: 2ad74e4d29bb5f3d6b644f35b237330f4f04babd615a52e2df961e1ec061ae87 2024-03-15T12:55:52.054 INFO:journalctl@ceph.mds.d.smithi002.stdout:Mar 15 12:55:51 smithi002 systemd[1]: Started Ceph mds.d for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:55:52.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:51 smithi184 ceph-mon[35993]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:52.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:51 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2330560151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:52.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:52.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:51 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:52.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:51 smithi149 ceph-mon[35927]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:52.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:51 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2330560151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:52.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:52.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:51 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:53.037 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:55:53.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:52 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.2:6836/1406254318,v1:172.21.15.2:6837/1406254318] up:boot 2024-03-15T12:55:53.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:52 smithi002 ceph-mon[32060]: fsmap 4 up:standby 2024-03-15T12:55:53.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:52 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-15T12:55:53.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:52 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:53.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:52 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:53.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:52 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:53.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:52 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:53.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:52 smithi002 ceph-mon[32060]: Deploying daemon mds.e on smithi149 2024-03-15T12:55:53.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:52 smithi002 ceph-mon[32060]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:53.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:52 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.2:6836/1406254318,v1:172.21.15.2:6837/1406254318] up:boot 2024-03-15T12:55:53.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:52 smithi184 ceph-mon[35993]: fsmap 4 up:standby 2024-03-15T12:55:53.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:52 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-15T12:55:53.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:52 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:53.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:52 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:53.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:52 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:53.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:52 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:53.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:52 smithi184 ceph-mon[35993]: Deploying daemon mds.e on smithi149 2024-03-15T12:55:53.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:52 smithi184 ceph-mon[35993]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:53.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:52 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.2:6836/1406254318,v1:172.21.15.2:6837/1406254318] up:boot 2024-03-15T12:55:53.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:52 smithi149 ceph-mon[35927]: fsmap 4 up:standby 2024-03-15T12:55:53.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:52 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-15T12:55:53.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:52 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:53.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:52 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:53.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:52 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:53.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:52 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:53.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:52 smithi149 ceph-mon[35927]: Deploying daemon mds.e on smithi149 2024-03-15T12:55:53.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:52 smithi149 ceph-mon[35927]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.170+0000 7ff605823640 1 -- 172.21.15.2:0/4012393077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600101620 msgr2=0x7ff600101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.170+0000 7ff605823640 1 --2- 172.21.15.2:0/4012393077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600101620 0x7ff600101aa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009900 tx=0x7ff5e802bc30 comp rx=0 tx=0).stop 2024-03-15T12:55:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.170+0000 7ff605823640 1 -- 172.21.15.2:0/4012393077 shutdown_connections 2024-03-15T12:55:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.170+0000 7ff605823640 1 --2- 172.21.15.2:0/4012393077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff600101fe0 0x7ff600108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.170+0000 7ff605823640 1 --2- 172.21.15.2:0/4012393077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600101620 0x7ff600101aa0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.170+0000 7ff605823640 1 --2- 172.21.15.2:0/4012393077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff600100430 0x7ff600100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.170+0000 7ff605823640 1 -- 172.21.15.2:0/4012393077 >> 172.21.15.2:0/4012393077 conn(0x7ff6000fb9a0 msgr2=0x7ff6000fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.170+0000 7ff605823640 1 -- 172.21.15.2:0/4012393077 shutdown_connections 2024-03-15T12:55:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.170+0000 7ff605823640 1 -- 172.21.15.2:0/4012393077 wait complete. 2024-03-15T12:55:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.171+0000 7ff605823640 1 Processor -- start 2024-03-15T12:55:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.171+0000 7ff605823640 1 -- start start 2024-03-15T12:55:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.171+0000 7ff605823640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600100430 0x7ff6001964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.171+0000 7ff605823640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff600101620 0x7ff600196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.171+0000 7ff605823640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff600101fe0 0x7ff60019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.171+0000 7ff605823640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60019dfb0 con 0x7ff600100430 2024-03-15T12:55:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.171+0000 7ff605823640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60019e120 con 0x7ff600101fe0 2024-03-15T12:55:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.171+0000 7ff605823640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60019e2e0 con 0x7ff600101620 2024-03-15T12:55:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.171+0000 7ff5feffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600100430 0x7ff6001964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.171+0000 7ff5feffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600100430 0x7ff6001964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46096/0 (socket says 172.21.15.2:46096) 2024-03-15T12:55:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.171+0000 7ff5feffd640 1 -- 172.21.15.2:0/924424094 learned_addr learned my addr 172.21.15.2:0/924424094 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:55:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.171+0000 7ff5ff7fe640 1 --2- 172.21.15.2:0/924424094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff600101fe0 0x7ff60019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.171+0000 7ff5fe7fc640 1 --2- 172.21.15.2:0/924424094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff600101620 0x7ff600196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.172+0000 7ff5feffd640 1 -- 172.21.15.2:0/924424094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff600101620 msgr2=0x7ff600196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.172+0000 7ff5feffd640 1 --2- 172.21.15.2:0/924424094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff600101620 0x7ff600196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.172+0000 7ff5feffd640 1 -- 172.21.15.2:0/924424094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff600101fe0 msgr2=0x7ff60019da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.172+0000 7ff5feffd640 1 --2- 172.21.15.2:0/924424094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff600101fe0 0x7ff60019da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.172+0000 7ff5feffd640 1 -- 172.21.15.2:0/924424094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e8009560 con 0x7ff600100430 2024-03-15T12:55:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.172+0000 7ff5feffd640 1 --2- 172.21.15.2:0/924424094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600100430 0x7ff6001964c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800e9c0 tx=0x7ff5f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.172+0000 7ff604821640 1 -- 172.21.15.2:0/924424094 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80096e0 con 0x7ff600100430 2024-03-15T12:55:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.172+0000 7ff604821640 1 -- 172.21.15.2:0/924424094 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff5f800cd80 con 0x7ff600100430 2024-03-15T12:55:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.172+0000 7ff605823640 1 -- 172.21.15.2:0/924424094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60019e5c0 con 0x7ff600100430 2024-03-15T12:55:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.172+0000 7ff604821640 1 -- 172.21.15.2:0/924424094 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8010430 con 0x7ff600100430 2024-03-15T12:55:53.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.172+0000 7ff605823640 1 -- 172.21.15.2:0/924424094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60019eaa0 con 0x7ff600100430 2024-03-15T12:55:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.173+0000 7ff605823640 1 -- 172.21.15.2:0/924424094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff600107770 con 0x7ff600100430 2024-03-15T12:55:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.174+0000 7ff604821640 1 -- 172.21.15.2:0/924424094 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7ff5f8003fe0 con 0x7ff600100430 2024-03-15T12:55:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.175+0000 7ff604821640 1 --2- 172.21.15.2:0/924424094 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff5ec0a1ab0 0x7ff5ec0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.175+0000 7ff5fe7fc640 1 --2- 172.21.15.2:0/924424094 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff5ec0a1ab0 0x7ff5ec0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.175+0000 7ff604821640 1 -- 172.21.15.2:0/924424094 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff5f8014070 con 0x7ff600100430 2024-03-15T12:55:53.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.175+0000 7ff5fe7fc640 1 --2- 172.21.15.2:0/924424094 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff5ec0a1ab0 0x7ff5ec0a3f70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8000c00 tx=0x7ff5e8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:53.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.180+0000 7ff604821640 1 -- 172.21.15.2:0/924424094 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff5f80969b0 con 0x7ff600100430 2024-03-15T12:55:53.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.403+0000 7ff605823640 1 -- 172.21.15.2:0/924424094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff60010be10 con 0x7ff600100430 2024-03-15T12:55:53.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.404+0000 7ff604821640 1 -- 172.21.15.2:0/924424094 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3692 (secure 0 0 0) 0x7ff5f809a070 con 0x7ff600100430 2024-03-15T12:55:53.390 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:55:53.391 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-15T12:55:53.391 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 5 2024-03-15T12:55:53.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.406+0000 7ff605823640 1 -- 172.21.15.2:0/924424094 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff5ec0a1ab0 msgr2=0x7ff5ec0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:53.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.406+0000 7ff605823640 1 --2- 172.21.15.2:0/924424094 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff5ec0a1ab0 0x7ff5ec0a3f70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8000c00 tx=0x7ff5e8005fd0 comp rx=0 tx=0).stop 2024-03-15T12:55:53.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.406+0000 7ff605823640 1 -- 172.21.15.2:0/924424094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600100430 msgr2=0x7ff6001964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:53.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.406+0000 7ff605823640 1 --2- 172.21.15.2:0/924424094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600100430 0x7ff6001964c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800e9c0 tx=0x7ff5f800ee80 comp rx=0 tx=0).stop 2024-03-15T12:55:53.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.406+0000 7ff605823640 1 -- 172.21.15.2:0/924424094 shutdown_connections 2024-03-15T12:55:53.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.406+0000 7ff605823640 1 --2- 172.21.15.2:0/924424094 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff5ec0a1ab0 0x7ff5ec0a3f70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:53.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.406+0000 7ff605823640 1 --2- 172.21.15.2:0/924424094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff600101fe0 0x7ff60019da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:53.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.406+0000 7ff605823640 1 --2- 172.21.15.2:0/924424094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff600101620 0x7ff600196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:53.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.406+0000 7ff605823640 1 --2- 172.21.15.2:0/924424094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600100430 0x7ff6001964c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:53.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.406+0000 7ff605823640 1 -- 172.21.15.2:0/924424094 >> 172.21.15.2:0/924424094 conn(0x7ff6000fb9a0 msgr2=0x7ff600104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:53.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.406+0000 7ff605823640 1 -- 172.21.15.2:0/924424094 shutdown_connections 2024-03-15T12:55:53.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:53.406+0000 7ff605823640 1 -- 172.21.15.2:0/924424094 wait complete. 2024-03-15T12:55:54.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:53 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/924424094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:54.026 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 12:55:53 smithi149 systemd[1]: Starting Ceph mds.e for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T12:55:54.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:53 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/924424094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:54.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:53 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/924424094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:54.287 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 12:55:54 smithi149 podman[65214]: 2024-03-15 12:55:54.039876943 +0000 UTC m=+0.108024766 container create 9b4a7928af3cb3beff5ce93c0f24365c6c1ef97be77a424f98802eed08cedc75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux ) 2024-03-15T12:55:54.287 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 12:55:54 smithi149 podman[65214]: 2024-03-15 12:55:53.95059793 +0000 UTC m=+0.018745764 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T12:55:54.287 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 12:55:54 smithi149 podman[65214]: 2024-03-15 12:55:54.276076139 +0000 UTC m=+0.344223974 container init 9b4a7928af3cb3beff5ce93c0f24365c6c1ef97be77a424f98802eed08cedc75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-e, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T12:55:54.287 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 12:55:54 smithi149 podman[65214]: 2024-03-15 12:55:54.278694305 +0000 UTC m=+0.346842127 container start 9b4a7928af3cb3beff5ce93c0f24365c6c1ef97be77a424f98802eed08cedc75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-15T12:55:54.610 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 12:55:54 smithi149 ceph-mds[65232]: -- 172.21.15.149:0/1187625368 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5626972de1a0 con 0x5626972b4800 2024-03-15T12:55:54.610 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 12:55:54 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-e[65228]: starting mds.e at 2024-03-15T12:55:54.610 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 12:55:54 smithi149 bash[65214]: 9b4a7928af3cb3beff5ce93c0f24365c6c1ef97be77a424f98802eed08cedc75 2024-03-15T12:55:54.610 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 12:55:54 smithi149 systemd[1]: Started Ceph mds.e for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:55:55.404 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:55:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.534+0000 7fefb1ed4640 1 -- 172.21.15.2:0/1556749246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefac1048c0 msgr2=0x7fefac106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.534+0000 7fefb1ed4640 1 --2- 172.21.15.2:0/1556749246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefac1048c0 0x7fefac106cb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef94009900 tx=0x7fef9402bc30 comp rx=0 tx=0).stop 2024-03-15T12:55:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.534+0000 7fefb1ed4640 1 -- 172.21.15.2:0/1556749246 shutdown_connections 2024-03-15T12:55:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.534+0000 7fefb1ed4640 1 --2- 172.21.15.2:0/1556749246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefac1071f0 0x7fefac109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.534+0000 7fefb1ed4640 1 --2- 172.21.15.2:0/1556749246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefac1048c0 0x7fefac106cb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.534+0000 7fefb1ed4640 1 --2- 172.21.15.2:0/1556749246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefac101f90 0x7fefac104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.534+0000 7fefb1ed4640 1 -- 172.21.15.2:0/1556749246 >> 172.21.15.2:0/1556749246 conn(0x7fefac0fb980 msgr2=0x7fefac0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.534+0000 7fefb1ed4640 1 -- 172.21.15.2:0/1556749246 shutdown_connections 2024-03-15T12:55:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.534+0000 7fefb1ed4640 1 -- 172.21.15.2:0/1556749246 wait complete. 2024-03-15T12:55:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.534+0000 7fefb1ed4640 1 Processor -- start 2024-03-15T12:55:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.534+0000 7fefb1ed4640 1 -- start start 2024-03-15T12:55:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefb1ed4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefac101f90 0x7fefac101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefb1ed4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefac1048c0 0x7fefac0fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefb1ed4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefac1071f0 0x7fefac100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefb1ed4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac100be0 con 0x7fefac1071f0 2024-03-15T12:55:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefb1ed4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac100d50 con 0x7fefac1048c0 2024-03-15T12:55:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefb1ed4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac1a1620 con 0x7fefac101f90 2024-03-15T12:55:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefabfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefac1071f0 0x7fefac100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefabfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefac1071f0 0x7fefac100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46116/0 (socket says 172.21.15.2:46116) 2024-03-15T12:55:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefabfff640 1 -- 172.21.15.2:0/2243372709 learned_addr learned my addr 172.21.15.2:0/2243372709 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:55:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefab7fe640 1 --2- 172.21.15.2:0/2243372709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefac101f90 0x7fefac101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefaaffd640 1 --2- 172.21.15.2:0/2243372709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefac1048c0 0x7fefac0fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefabfff640 1 -- 172.21.15.2:0/2243372709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefac101f90 msgr2=0x7fefac101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefabfff640 1 --2- 172.21.15.2:0/2243372709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefac101f90 0x7fefac101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefabfff640 1 -- 172.21.15.2:0/2243372709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefac1048c0 msgr2=0x7fefac0fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefabfff640 1 --2- 172.21.15.2:0/2243372709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefac1048c0 0x7fefac0fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefabfff640 1 -- 172.21.15.2:0/2243372709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef94009560 con 0x7fefac1071f0 2024-03-15T12:55:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefabfff640 1 --2- 172.21.15.2:0/2243372709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefac1071f0 0x7fefac100480 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fefa400e9a0 tx=0x7fefa400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefa8ff9640 1 -- 172.21.15.2:0/2243372709 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa400ccc0 con 0x7fefac1071f0 2024-03-15T12:55:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefa8ff9640 1 -- 172.21.15.2:0/2243372709 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fefa400ce20 con 0x7fefac1071f0 2024-03-15T12:55:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefb1ed4640 1 -- 172.21.15.2:0/2243372709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefac1a18d0 con 0x7fefac1071f0 2024-03-15T12:55:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.535+0000 7fefa8ff9640 1 -- 172.21.15.2:0/2243372709 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4010490 con 0x7fefac1071f0 2024-03-15T12:55:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.536+0000 7fefb1ed4640 1 -- 172.21.15.2:0/2243372709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefac1a1db0 con 0x7fefac1071f0 2024-03-15T12:55:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.536+0000 7fefb1ed4640 1 -- 172.21.15.2:0/2243372709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefac0fd470 con 0x7fefac1071f0 2024-03-15T12:55:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.538+0000 7fefa8ff9640 1 -- 172.21.15.2:0/2243372709 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fefa40046c0 con 0x7fefac1071f0 2024-03-15T12:55:55.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.539+0000 7fefa8ff9640 1 --2- 172.21.15.2:0/2243372709 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fef980a1ab0 0x7fef980a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:55.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.539+0000 7fefab7fe640 1 --2- 172.21.15.2:0/2243372709 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fef980a1ab0 0x7fef980a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:55.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.539+0000 7fefa8ff9640 1 -- 172.21.15.2:0/2243372709 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fefa4014070 con 0x7fefac1071f0 2024-03-15T12:55:55.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.539+0000 7fefab7fe640 1 --2- 172.21.15.2:0/2243372709 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fef980a1ab0 0x7fef980a3f70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fef9c005fd0 tx=0x7fef9c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:55.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.542+0000 7fefa8ff9640 1 -- 172.21.15.2:0/2243372709 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fefa4096b30 con 0x7fefac1071f0 2024-03-15T12:55:55.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.763+0000 7fefb1ed4640 1 -- 172.21.15.2:0/2243372709 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefac10ced0 con 0x7fefac1071f0 2024-03-15T12:55:55.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.763+0000 7fefa8ff9640 1 -- 172.21.15.2:0/2243372709 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7fefa409a1f0 con 0x7fefac1071f0 2024-03-15T12:55:55.751 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:55:55.751 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-15T12:55:55.752 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 6 2024-03-15T12:55:55.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.765+0000 7fefb1ed4640 1 -- 172.21.15.2:0/2243372709 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fef980a1ab0 msgr2=0x7fef980a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:55.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.765+0000 7fefb1ed4640 1 --2- 172.21.15.2:0/2243372709 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fef980a1ab0 0x7fef980a3f70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fef9c005fd0 tx=0x7fef9c009500 comp rx=0 tx=0).stop 2024-03-15T12:55:55.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.765+0000 7fefb1ed4640 1 -- 172.21.15.2:0/2243372709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefac1071f0 msgr2=0x7fefac100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:55.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.765+0000 7fefb1ed4640 1 --2- 172.21.15.2:0/2243372709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefac1071f0 0x7fefac100480 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fefa400e9a0 tx=0x7fefa400eda0 comp rx=0 tx=0).stop 2024-03-15T12:55:55.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.765+0000 7fefb1ed4640 1 -- 172.21.15.2:0/2243372709 shutdown_connections 2024-03-15T12:55:55.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.765+0000 7fefb1ed4640 1 --2- 172.21.15.2:0/2243372709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefac1071f0 0x7fefac100480 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:55.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.765+0000 7fefb1ed4640 1 --2- 172.21.15.2:0/2243372709 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fef980a1ab0 0x7fef980a3f70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:55.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.766+0000 7fefb1ed4640 1 --2- 172.21.15.2:0/2243372709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefac1048c0 0x7fefac0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:55.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.766+0000 7fefb1ed4640 1 --2- 172.21.15.2:0/2243372709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefac101f90 0x7fefac101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:55.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.766+0000 7fefb1ed4640 1 -- 172.21.15.2:0/2243372709 >> 172.21.15.2:0/2243372709 conn(0x7fefac0fb980 msgr2=0x7fefac102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:55.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.766+0000 7fefb1ed4640 1 -- 172.21.15.2:0/2243372709 shutdown_connections 2024-03-15T12:55:55.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:55.766+0000 7fefb1ed4640 1 -- 172.21.15.2:0/2243372709 wait complete. 2024-03-15T12:55:55.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:55.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:55.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:55.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:55.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:55.763 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:55.763 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:55 smithi002 ceph-mon[32060]: Deploying daemon mds.f on smithi184 2024-03-15T12:55:55.763 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:55 smithi002 ceph-mon[32060]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:55.763 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:55 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] up:boot 2024-03-15T12:55:55.763 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:55 smithi002 ceph-mon[32060]: fsmap 5 up:standby 2024-03-15T12:55:55.763 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:55 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-15T12:55:55.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:55.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:55.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:55.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:55.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:55.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:55.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:55 smithi184 ceph-mon[35993]: Deploying daemon mds.f on smithi184 2024-03-15T12:55:55.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:55 smithi184 ceph-mon[35993]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:55.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:55 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] up:boot 2024-03-15T12:55:55.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:55 smithi184 ceph-mon[35993]: fsmap 5 up:standby 2024-03-15T12:55:55.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:55 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-15T12:55:55.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:55.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:55.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:55.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:55.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:55.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:55.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:55 smithi149 ceph-mon[35927]: Deploying daemon mds.f on smithi184 2024-03-15T12:55:55.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:55 smithi149 ceph-mon[35927]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:55.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:55 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] up:boot 2024-03-15T12:55:55.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:55 smithi149 ceph-mon[35927]: fsmap 5 up:standby 2024-03-15T12:55:55.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:55 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-15T12:55:56.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:56 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2243372709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:56.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:56 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2243372709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:56.809 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 12:55:56 smithi184 systemd[1]: Starting Ceph mds.f for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T12:55:56.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:56 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2243372709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:57.086 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 12:55:56 smithi184 podman[66344]: 2024-03-15 12:55:56.729627258 +0000 UTC m=+0.018156588 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T12:55:57.086 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 12:55:56 smithi184 podman[66344]: 2024-03-15 12:55:56.847274153 +0000 UTC m=+0.135803459 container create eb3f5a7c69a9765d4823cb90b621c14027604fe7671636db9f799716cad44486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-f, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-03-15T12:55:57.086 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 12:55:57 smithi184 podman[66344]: 2024-03-15 12:55:57.100324932 +0000 UTC m=+0.388854243 container init eb3f5a7c69a9765d4823cb90b621c14027604fe7671636db9f799716cad44486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-15T12:55:57.442 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 12:55:57 smithi184 podman[66344]: 2024-03-15 12:55:57.103128511 +0000 UTC m=+0.391657818 container start eb3f5a7c69a9765d4823cb90b621c14027604fe7671636db9f799716cad44486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-f, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-15T12:55:57.443 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 12:55:57 smithi184 ceph-mds[66363]: -- 172.21.15.184:0/3831486352 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558e3854e1a0 con 0x558e38525000 2024-03-15T12:55:57.443 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 12:55:57 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-f[66359]: starting mds.f at 2024-03-15T12:55:57.443 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 12:55:57 smithi184 bash[66344]: eb3f5a7c69a9765d4823cb90b621c14027604fe7671636db9f799716cad44486 2024-03-15T12:55:57.443 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 12:55:57 smithi184 systemd[1]: Started Ceph mds.f for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:55:57.765 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:55:57.793 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:57 smithi002 ceph-mon[32060]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:57.793 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:57.793 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:57.793 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:57.793 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:57.794 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:57.794 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:57 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:57.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:57 smithi184 ceph-mon[35993]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:57.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:57.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:57.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:57.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:57.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:57.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:57 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:57.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:57 smithi149 ceph-mon[35927]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:57.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:57.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:57.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:55:57.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:55:57.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:55:57.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:57 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:55:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.894+0000 7f20c88d4640 1 -- 172.21.15.2:0/1745467099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4101fc0 msgr2=0x7f20c4108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.894+0000 7f20c88d4640 1 --2- 172.21.15.2:0/1745467099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4101fc0 0x7f20c4108850 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009ca0 tx=0x7f20bc02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:55:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.894+0000 7f20c88d4640 1 -- 172.21.15.2:0/1745467099 shutdown_connections 2024-03-15T12:55:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.894+0000 7f20c88d4640 1 --2- 172.21.15.2:0/1745467099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4101fc0 0x7f20c4108850 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.894+0000 7f20c88d4640 1 --2- 172.21.15.2:0/1745467099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c4101600 0x7f20c4101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.894+0000 7f20c88d4640 1 --2- 172.21.15.2:0/1745467099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c4100410 0x7f20c4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:57.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.894+0000 7f20c88d4640 1 -- 172.21.15.2:0/1745467099 >> 172.21.15.2:0/1745467099 conn(0x7f20c40fb9a0 msgr2=0x7f20c40fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.894+0000 7f20c88d4640 1 -- 172.21.15.2:0/1745467099 shutdown_connections 2024-03-15T12:55:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.894+0000 7f20c88d4640 1 -- 172.21.15.2:0/1745467099 wait complete. 2024-03-15T12:55:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.895+0000 7f20c88d4640 1 Processor -- start 2024-03-15T12:55:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.895+0000 7f20c88d4640 1 -- start start 2024-03-15T12:55:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.895+0000 7f20c88d4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c4100410 0x7f20c4196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.895+0000 7f20c88d4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c4101600 0x7f20c41969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.895+0000 7f20c88d4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4101fc0 0x7f20c419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.895+0000 7f20c88d4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c419df80 con 0x7f20c4101fc0 2024-03-15T12:55:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.895+0000 7f20c88d4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c419e0f0 con 0x7f20c4101600 2024-03-15T12:55:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.895+0000 7f20c88d4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c419e2b0 con 0x7f20c4100410 2024-03-15T12:55:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20c2d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4101fc0 0x7f20c419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20c2d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4101fc0 0x7f20c419da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46138/0 (socket says 172.21.15.2:46138) 2024-03-15T12:55:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20c2d9d640 1 -- 172.21.15.2:0/449928570 learned_addr learned my addr 172.21.15.2:0/449928570 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:55:57.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20bbfff640 1 --2- 172.21.15.2:0/449928570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c4101600 0x7f20c41969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20c259c640 1 --2- 172.21.15.2:0/449928570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c4100410 0x7f20c4196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20c2d9d640 1 -- 172.21.15.2:0/449928570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c4100410 msgr2=0x7f20c4196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20c2d9d640 1 --2- 172.21.15.2:0/449928570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c4100410 0x7f20c4196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20c2d9d640 1 -- 172.21.15.2:0/449928570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c4101600 msgr2=0x7f20c41969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20c2d9d640 1 --2- 172.21.15.2:0/449928570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c4101600 0x7f20c41969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20c2d9d640 1 -- 172.21.15.2:0/449928570 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20bc009900 con 0x7f20c4101fc0 2024-03-15T12:55:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20c2d9d640 1 --2- 172.21.15.2:0/449928570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4101fc0 0x7f20c419da40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f20bc005e30 tx=0x7f20bc032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20bb7fe640 1 -- 172.21.15.2:0/449928570 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc003970 con 0x7f20c4101fc0 2024-03-15T12:55:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20bb7fe640 1 -- 172.21.15.2:0/449928570 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f20bc004040 con 0x7f20c4101fc0 2024-03-15T12:55:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20c88d4640 1 -- 172.21.15.2:0/449928570 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c419e530 con 0x7f20c4101fc0 2024-03-15T12:55:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20bb7fe640 1 -- 172.21.15.2:0/449928570 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0406b0 con 0x7f20c4101fc0 2024-03-15T12:55:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.896+0000 7f20c88d4640 1 -- 172.21.15.2:0/449928570 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c419e9e0 con 0x7f20c4101fc0 2024-03-15T12:55:57.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.897+0000 7f20c88d4640 1 -- 172.21.15.2:0/449928570 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c41076d0 con 0x7f20c4101fc0 2024-03-15T12:55:57.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.898+0000 7f20bb7fe640 1 -- 172.21.15.2:0/449928570 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f20bc00a040 con 0x7f20c4101fc0 2024-03-15T12:55:57.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.899+0000 7f20bb7fe640 1 --2- 172.21.15.2:0/449928570 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20b00a1a30 0x7f20b00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:55:57.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.899+0000 7f20c259c640 1 --2- 172.21.15.2:0/449928570 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20b00a1a30 0x7f20b00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:55:57.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.899+0000 7f20bb7fe640 1 -- 172.21.15.2:0/449928570 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f20bc0f0120 con 0x7f20c4101fc0 2024-03-15T12:55:57.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.900+0000 7f20c259c640 1 --2- 172.21.15.2:0/449928570 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20b00a1a30 0x7f20b00a3ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f20b4005fd0 tx=0x7f20b400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:55:57.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:57.904+0000 7f20bb7fe640 1 -- 172.21.15.2:0/449928570 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f20bc0b9730 con 0x7f20c4101fc0 2024-03-15T12:55:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:58.125+0000 7f20c88d4640 1 -- 172.21.15.2:0/449928570 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20c402d000 con 0x7f20c4101fc0 2024-03-15T12:55:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:58.125+0000 7f20bb7fe640 1 -- 172.21.15.2:0/449928570 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5277 (secure 0 0 0) 0x7f20bc0bcdf0 con 0x7f20c4101fc0 2024-03-15T12:55:58.112 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:55:58.112 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-15T12:55:58.112 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 7 2024-03-15T12:55:58.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:58.127+0000 7f20c88d4640 1 -- 172.21.15.2:0/449928570 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20b00a1a30 msgr2=0x7f20b00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:58.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:58.127+0000 7f20c88d4640 1 --2- 172.21.15.2:0/449928570 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20b00a1a30 0x7f20b00a3ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f20b4005fd0 tx=0x7f20b400b410 comp rx=0 tx=0).stop 2024-03-15T12:55:58.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:58.127+0000 7f20c88d4640 1 -- 172.21.15.2:0/449928570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4101fc0 msgr2=0x7f20c419da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:55:58.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:58.127+0000 7f20c88d4640 1 --2- 172.21.15.2:0/449928570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4101fc0 0x7f20c419da40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f20bc005e30 tx=0x7f20bc032a10 comp rx=0 tx=0).stop 2024-03-15T12:55:58.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:58.127+0000 7f20c88d4640 1 -- 172.21.15.2:0/449928570 shutdown_connections 2024-03-15T12:55:58.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:58.127+0000 7f20c88d4640 1 --2- 172.21.15.2:0/449928570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20c4101fc0 0x7f20c419da40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:58.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:58.127+0000 7f20c88d4640 1 --2- 172.21.15.2:0/449928570 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20b00a1a30 0x7f20b00a3ef0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:58.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:58.127+0000 7f20c88d4640 1 --2- 172.21.15.2:0/449928570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c4101600 0x7f20c41969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:58.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:58.127+0000 7f20c88d4640 1 --2- 172.21.15.2:0/449928570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20c4100410 0x7f20c4196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:55:58.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:58.127+0000 7f20c88d4640 1 -- 172.21.15.2:0/449928570 >> 172.21.15.2:0/449928570 conn(0x7f20c40fb9a0 msgr2=0x7f20c41048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:55:58.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:58.127+0000 7f20c88d4640 1 -- 172.21.15.2:0/449928570 shutdown_connections 2024-03-15T12:55:58.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:55:58.127+0000 7f20c88d4640 1 -- 172.21.15.2:0/449928570 wait complete. 2024-03-15T12:55:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:58 smithi184 ceph-mon[35993]: Deploying daemon mds.c on smithi184 2024-03-15T12:55:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:58 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] up:boot 2024-03-15T12:55:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:58 smithi184 ceph-mon[35993]: fsmap 6 up:standby 2024-03-15T12:55:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:58 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-15T12:55:58.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:58 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/449928570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:58.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:58 smithi002 ceph-mon[32060]: Deploying daemon mds.c on smithi184 2024-03-15T12:55:58.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:58 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] up:boot 2024-03-15T12:55:58.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:58 smithi002 ceph-mon[32060]: fsmap 6 up:standby 2024-03-15T12:55:58.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:58 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-15T12:55:58.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:58 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/449928570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:58.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:58 smithi149 ceph-mon[35927]: Deploying daemon mds.c on smithi184 2024-03-15T12:55:58.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:58 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] up:boot 2024-03-15T12:55:58.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:58 smithi149 ceph-mon[35927]: fsmap 6 up:standby 2024-03-15T12:55:58.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:58 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-15T12:55:58.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:58 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/449928570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:55:59.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:55:59 smithi184 ceph-mon[35993]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:59.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:55:59 smithi002 ceph-mon[32060]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:59.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:55:59 smithi149 ceph-mon[35927]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:55:59.916 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 12:55:59 smithi184 podman[66649]: 2024-03-15 12:55:59.908189549 +0000 UTC m=+0.361211085 container start eadf048c5df15920ca296c57d759acfada8611b7b6600c137fd5b485256b40af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-c, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-15T12:56:00.125 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:56:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.252+0000 7f7da0a64640 1 -- 172.21.15.2:0/343889570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c103170 msgr2=0x7f7d9c109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.252+0000 7f7da0a64640 1 --2- 172.21.15.2:0/343889570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c103170 0x7f7d9c109a00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009ca0 tx=0x7f7d9402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.252+0000 7f7da0a64640 1 -- 172.21.15.2:0/343889570 shutdown_connections 2024-03-15T12:56:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.252+0000 7f7da0a64640 1 --2- 172.21.15.2:0/343889570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c103170 0x7f7d9c109a00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.252+0000 7f7da0a64640 1 --2- 172.21.15.2:0/343889570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d9c1027b0 0x7f7d9c102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.252+0000 7f7da0a64640 1 --2- 172.21.15.2:0/343889570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d9c0ff7e0 0x7f7d9c0ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.252+0000 7f7da0a64640 1 -- 172.21.15.2:0/343889570 >> 172.21.15.2:0/343889570 conn(0x7f7d9c0fb110 msgr2=0x7f7d9c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.252+0000 7f7da0a64640 1 -- 172.21.15.2:0/343889570 shutdown_connections 2024-03-15T12:56:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.252+0000 7f7da0a64640 1 -- 172.21.15.2:0/343889570 wait complete. 2024-03-15T12:56:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.253+0000 7f7da0a64640 1 Processor -- start 2024-03-15T12:56:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.253+0000 7f7da0a64640 1 -- start start 2024-03-15T12:56:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.253+0000 7f7da0a64640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d9c0ff7e0 0x7f7d9c197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.253+0000 7f7da0a64640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1027b0 0x7f7d9c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.253+0000 7f7da0a64640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d9c103170 0x7f7d9c19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.253+0000 7f7da0a64640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c19f070 con 0x7f7d9c1027b0 2024-03-15T12:56:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.253+0000 7f7da0a64640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c19f1e0 con 0x7f7d9c103170 2024-03-15T12:56:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.253+0000 7f7da0a64640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c19f3a0 con 0x7f7d9c0ff7e0 2024-03-15T12:56:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.253+0000 7f7d93fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1027b0 0x7f7d9c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.253+0000 7f7d9a59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d9c0ff7e0 0x7f7d9c197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.253+0000 7f7d93fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1027b0 0x7f7d9c197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58344/0 (socket says 172.21.15.2:58344) 2024-03-15T12:56:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.253+0000 7f7d93fff640 1 -- 172.21.15.2:0/3829625235 learned_addr learned my addr 172.21.15.2:0/3829625235 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.254+0000 7f7d9ad9d640 1 --2- 172.21.15.2:0/3829625235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d9c103170 0x7f7d9c19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.254+0000 7f7d93fff640 1 -- 172.21.15.2:0/3829625235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d9c0ff7e0 msgr2=0x7f7d9c197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.254+0000 7f7d93fff640 1 --2- 172.21.15.2:0/3829625235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d9c0ff7e0 0x7f7d9c197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.254+0000 7f7d93fff640 1 -- 172.21.15.2:0/3829625235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d9c103170 msgr2=0x7f7d9c19eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.254+0000 7f7d93fff640 1 --2- 172.21.15.2:0/3829625235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d9c103170 0x7f7d9c19eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.254+0000 7f7d93fff640 1 -- 172.21.15.2:0/3829625235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d94009900 con 0x7f7d9c1027b0 2024-03-15T12:56:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.254+0000 7f7d93fff640 1 --2- 172.21.15.2:0/3829625235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1027b0 0x7f7d9c197ac0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400e9f0 tx=0x7f7d8400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.254+0000 7f7d937fe640 1 -- 172.21.15.2:0/3829625235 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d84009720 con 0x7f7d9c1027b0 2024-03-15T12:56:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.254+0000 7f7d937fe640 1 -- 172.21.15.2:0/3829625235 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7d8400cd80 con 0x7f7d9c1027b0 2024-03-15T12:56:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.254+0000 7f7da0a64640 1 -- 172.21.15.2:0/3829625235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9c19f680 con 0x7f7d9c1027b0 2024-03-15T12:56:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.254+0000 7f7d937fe640 1 -- 172.21.15.2:0/3829625235 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d84010690 con 0x7f7d9c1027b0 2024-03-15T12:56:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.254+0000 7f7da0a64640 1 -- 172.21.15.2:0/3829625235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9c19fb40 con 0x7f7d9c1027b0 2024-03-15T12:56:00.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.255+0000 7f7da0a64640 1 -- 172.21.15.2:0/3829625235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9c1088e0 con 0x7f7d9c1027b0 2024-03-15T12:56:00.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.256+0000 7f7d937fe640 1 -- 172.21.15.2:0/3829625235 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f7d84004e90 con 0x7f7d9c1027b0 2024-03-15T12:56:00.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.257+0000 7f7d937fe640 1 --2- 172.21.15.2:0/3829625235 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7d880a1a30 0x7f7d880a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:00.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.257+0000 7f7d937fe640 1 -- 172.21.15.2:0/3829625235 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7d84014070 con 0x7f7d9c1027b0 2024-03-15T12:56:00.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.257+0000 7f7d9a59c640 1 --2- 172.21.15.2:0/3829625235 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7d880a1a30 0x7f7d880a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:00.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.257+0000 7f7d9a59c640 1 --2- 172.21.15.2:0/3829625235 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7d880a1a30 0x7f7d880a3ef0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c005fd0 tx=0x7f7d8c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:00.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.261+0000 7f7d937fe640 1 -- 172.21.15.2:0/3829625235 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7d840968e0 con 0x7f7d9c1027b0 2024-03-15T12:56:00.309 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 12:55:59 smithi184 ceph-mds[66668]: -- 172.21.15.184:0/589132501 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ef86adc1a0 con 0x55ef86ab2c00 2024-03-15T12:56:00.309 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 12:55:59 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-c[66664]: starting mds.c at 2024-03-15T12:56:00.309 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 12:56:00 smithi184 bash[66649]: eadf048c5df15920ca296c57d759acfada8611b7b6600c137fd5b485256b40af 2024-03-15T12:56:00.310 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 12:56:00 smithi184 systemd[1]: Started Ceph mds.c for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:56:00.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.484+0000 7f7da0a64640 1 -- 172.21.15.2:0/3829625235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d9c10cf10 con 0x7f7d9c1027b0 2024-03-15T12:56:00.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.485+0000 7f7d937fe640 1 -- 172.21.15.2:0/3829625235 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6071 (secure 0 0 0) 0x7f7d84099fa0 con 0x7f7d9c1027b0 2024-03-15T12:56:00.472 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:00.472 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-15T12:56:00.472 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2024-03-15T12:56:00.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.487+0000 7f7da0a64640 1 -- 172.21.15.2:0/3829625235 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7d880a1a30 msgr2=0x7f7d880a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:00.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.487+0000 7f7da0a64640 1 --2- 172.21.15.2:0/3829625235 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7d880a1a30 0x7f7d880a3ef0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c005fd0 tx=0x7f7d8c00b410 comp rx=0 tx=0).stop 2024-03-15T12:56:00.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.487+0000 7f7da0a64640 1 -- 172.21.15.2:0/3829625235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1027b0 msgr2=0x7f7d9c197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:00.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.487+0000 7f7da0a64640 1 --2- 172.21.15.2:0/3829625235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1027b0 0x7f7d9c197ac0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400e9f0 tx=0x7f7d8400edf0 comp rx=0 tx=0).stop 2024-03-15T12:56:00.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.487+0000 7f7da0a64640 1 -- 172.21.15.2:0/3829625235 shutdown_connections 2024-03-15T12:56:00.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.487+0000 7f7da0a64640 1 --2- 172.21.15.2:0/3829625235 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7d880a1a30 0x7f7d880a3ef0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:00.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.487+0000 7f7da0a64640 1 --2- 172.21.15.2:0/3829625235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d9c103170 0x7f7d9c19eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:00.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.487+0000 7f7da0a64640 1 --2- 172.21.15.2:0/3829625235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d9c1027b0 0x7f7d9c197ac0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:00.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.487+0000 7f7da0a64640 1 --2- 172.21.15.2:0/3829625235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d9c0ff7e0 0x7f7d9c197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:00.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.487+0000 7f7da0a64640 1 -- 172.21.15.2:0/3829625235 >> 172.21.15.2:0/3829625235 conn(0x7f7d9c0fb110 msgr2=0x7f7d9c105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:00.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.487+0000 7f7da0a64640 1 -- 172.21.15.2:0/3829625235 shutdown_connections 2024-03-15T12:56:00.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:00.487+0000 7f7da0a64640 1 -- 172.21.15.2:0/3829625235 wait complete. 2024-03-15T12:56:01.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:01 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:01.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:01 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:01.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:01 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:01.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:01 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:56:01.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:01 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:56:01.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:01 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:01.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:01 smithi002 ceph-mon[32060]: Deploying daemon mds.h on smithi149 2024-03-15T12:56:01.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:01 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.184:6836/3698954880,v1:172.21.15.184:6837/3698954880] up:boot 2024-03-15T12:56:01.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:01 smithi002 ceph-mon[32060]: fsmap 7 up:standby 2024-03-15T12:56:01.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:01 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-15T12:56:01.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:01 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3829625235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:01.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:01 smithi002 ceph-mon[32060]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:01.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:01 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:01.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:01 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:01.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:01 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:01.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:01 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:56:01.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:01 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:56:01.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:01 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:01.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:01 smithi184 ceph-mon[35993]: Deploying daemon mds.h on smithi149 2024-03-15T12:56:01.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:01 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.184:6836/3698954880,v1:172.21.15.184:6837/3698954880] up:boot 2024-03-15T12:56:01.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:01 smithi184 ceph-mon[35993]: fsmap 7 up:standby 2024-03-15T12:56:01.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:01 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-15T12:56:01.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:01 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3829625235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:01.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:01 smithi184 ceph-mon[35993]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:01.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:01 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:01.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:01 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:01.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:01 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:01.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:01 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:56:01.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:01 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:56:01.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:01 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:01.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:01 smithi149 ceph-mon[35927]: Deploying daemon mds.h on smithi149 2024-03-15T12:56:01.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:01 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.184:6836/3698954880,v1:172.21.15.184:6837/3698954880] up:boot 2024-03-15T12:56:01.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:01 smithi149 ceph-mon[35927]: fsmap 7 up:standby 2024-03-15T12:56:01.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:01 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-15T12:56:01.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:01 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3829625235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:01.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:01 smithi149 ceph-mon[35927]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:02.485 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:56:02.522 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 12:56:02 smithi149 systemd[1]: Starting Ceph mds.h for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T12:56:02.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.614+0000 7f1a49fb5640 1 -- 172.21.15.2:0/4286667808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a44101520 msgr2=0x7f1a44101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:02.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.614+0000 7f1a49fb5640 1 --2- 172.21.15.2:0/4286667808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a44101520 0x7f1a44101940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009900 tx=0x7f1a3402bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:02.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.614+0000 7f1a49fb5640 1 -- 172.21.15.2:0/4286667808 shutdown_connections 2024-03-15T12:56:02.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.614+0000 7f1a49fb5640 1 --2- 172.21.15.2:0/4286667808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a441030d0 0x7f1a44109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.614+0000 7f1a49fb5640 1 --2- 172.21.15.2:0/4286667808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a44102710 0x7f1a44102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.614+0000 7f1a49fb5640 1 --2- 172.21.15.2:0/4286667808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a44101520 0x7f1a44101940 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.614+0000 7f1a49fb5640 1 -- 172.21.15.2:0/4286667808 >> 172.21.15.2:0/4286667808 conn(0x7f1a440fcad0 msgr2=0x7f1a440feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.614+0000 7f1a49fb5640 1 -- 172.21.15.2:0/4286667808 shutdown_connections 2024-03-15T12:56:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.614+0000 7f1a49fb5640 1 -- 172.21.15.2:0/4286667808 wait complete. 2024-03-15T12:56:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.614+0000 7f1a49fb5640 1 Processor -- start 2024-03-15T12:56:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a49fb5640 1 -- start start 2024-03-15T12:56:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a49fb5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a44101520 0x7f1a441975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a49fb5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a44102710 0x7f1a44197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a49fb5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a441030d0 0x7f1a4419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a49fb5640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4419f0a0 con 0x7f1a44101520 2024-03-15T12:56:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a49fb5640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4419f210 con 0x7f1a44102710 2024-03-15T12:56:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a49fb5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4419f3d0 con 0x7f1a441030d0 2024-03-15T12:56:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a437fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a44101520 0x7f1a441975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a437fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a44101520 0x7f1a441975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58362/0 (socket says 172.21.15.2:58362) 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a437fe640 1 -- 172.21.15.2:0/2495535473 learned_addr learned my addr 172.21.15.2:0/2495535473 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a43fff640 1 --2- 172.21.15.2:0/2495535473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a441030d0 0x7f1a4419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a42ffd640 1 --2- 172.21.15.2:0/2495535473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a44102710 0x7f1a44197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a437fe640 1 -- 172.21.15.2:0/2495535473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a441030d0 msgr2=0x7f1a4419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a437fe640 1 --2- 172.21.15.2:0/2495535473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a441030d0 0x7f1a4419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a437fe640 1 -- 172.21.15.2:0/2495535473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a44102710 msgr2=0x7f1a44197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a437fe640 1 --2- 172.21.15.2:0/2495535473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a44102710 0x7f1a44197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a437fe640 1 -- 172.21.15.2:0/2495535473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a34009560 con 0x7f1a44101520 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a42ffd640 1 --2- 172.21.15.2:0/2495535473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a44102710 0x7f1a44197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.615+0000 7f1a437fe640 1 --2- 172.21.15.2:0/2495535473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a44101520 0x7f1a441975b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1a34000c00 tx=0x7f1a340049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.616+0000 7f1a40ff9640 1 -- 172.21.15.2:0/2495535473 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34004250 con 0x7f1a44101520 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.616+0000 7f1a40ff9640 1 -- 172.21.15.2:0/2495535473 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1a340043b0 con 0x7f1a44101520 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.616+0000 7f1a49fb5640 1 -- 172.21.15.2:0/2495535473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4419f650 con 0x7f1a44101520 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.616+0000 7f1a40ff9640 1 -- 172.21.15.2:0/2495535473 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34003d50 con 0x7f1a44101520 2024-03-15T12:56:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.616+0000 7f1a49fb5640 1 -- 172.21.15.2:0/2495535473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4419fb60 con 0x7f1a44101520 2024-03-15T12:56:02.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.617+0000 7f1a49fb5640 1 -- 172.21.15.2:0/2495535473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a441088c0 con 0x7f1a44101520 2024-03-15T12:56:02.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.618+0000 7f1a40ff9640 1 -- 172.21.15.2:0/2495535473 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f1a34005000 con 0x7f1a44101520 2024-03-15T12:56:02.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.619+0000 7f1a40ff9640 1 --2- 172.21.15.2:0/2495535473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1a2c0a1a30 0x7f1a2c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:02.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.619+0000 7f1a42ffd640 1 --2- 172.21.15.2:0/2495535473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1a2c0a1a30 0x7f1a2c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:02.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.619+0000 7f1a40ff9640 1 -- 172.21.15.2:0/2495535473 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1a340ef230 con 0x7f1a44101520 2024-03-15T12:56:02.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.619+0000 7f1a42ffd640 1 --2- 172.21.15.2:0/2495535473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1a2c0a1a30 0x7f1a2c0a3ef0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a44198ac0 tx=0x7f1a38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:02.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.623+0000 7f1a40ff9640 1 -- 172.21.15.2:0/2495535473 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1a340b88e0 con 0x7f1a44101520 2024-03-15T12:56:02.781 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 12:56:02 smithi149 podman[65539]: 2024-03-15 12:56:02.439950076 +0000 UTC m=+0.018331291 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T12:56:02.781 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 12:56:02 smithi149 podman[65539]: 2024-03-15 12:56:02.545606169 +0000 UTC m=+0.123987423 container create 0e291e77314ffa4e31e0eff7f0ec33f32b0936590b0af89f30288c0bea6d4492 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-15T12:56:02.781 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 12:56:02 smithi149 podman[65539]: 2024-03-15 12:56:02.781995937 +0000 UTC m=+0.360377164 container init 0e291e77314ffa4e31e0eff7f0ec33f32b0936590b0af89f30288c0bea6d4492 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-h, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-15T12:56:02.781 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 12:56:02 smithi149 podman[65539]: 2024-03-15 12:56:02.784667667 +0000 UTC m=+0.363048879 container start 0e291e77314ffa4e31e0eff7f0ec33f32b0936590b0af89f30288c0bea6d4492 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-h, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-15T12:56:02.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.844+0000 7f1a49fb5640 1 -- 172.21.15.2:0/2495535473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a4410cf00 con 0x7f1a44101520 2024-03-15T12:56:02.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.845+0000 7f1a40ff9640 1 -- 172.21.15.2:0/2495535473 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6071 (secure 0 0 0) 0x7f1a4410cf00 con 0x7f1a44101520 2024-03-15T12:56:02.831 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:02.831 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-15T12:56:02.832 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2024-03-15T12:56:02.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.847+0000 7f1a49fb5640 1 -- 172.21.15.2:0/2495535473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1a2c0a1a30 msgr2=0x7f1a2c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:02.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.847+0000 7f1a49fb5640 1 --2- 172.21.15.2:0/2495535473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1a2c0a1a30 0x7f1a2c0a3ef0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a44198ac0 tx=0x7f1a38005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:02.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.847+0000 7f1a49fb5640 1 -- 172.21.15.2:0/2495535473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a44101520 msgr2=0x7f1a441975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:02.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.847+0000 7f1a49fb5640 1 --2- 172.21.15.2:0/2495535473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a44101520 0x7f1a441975b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1a34000c00 tx=0x7f1a340049e0 comp rx=0 tx=0).stop 2024-03-15T12:56:02.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.847+0000 7f1a49fb5640 1 -- 172.21.15.2:0/2495535473 shutdown_connections 2024-03-15T12:56:02.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.847+0000 7f1a49fb5640 1 --2- 172.21.15.2:0/2495535473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a441030d0 0x7f1a4419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:02.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.847+0000 7f1a49fb5640 1 --2- 172.21.15.2:0/2495535473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1a2c0a1a30 0x7f1a2c0a3ef0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:02.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.847+0000 7f1a49fb5640 1 --2- 172.21.15.2:0/2495535473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a44102710 0x7f1a44197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:02.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.847+0000 7f1a49fb5640 1 --2- 172.21.15.2:0/2495535473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a44101520 0x7f1a441975b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:02.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.847+0000 7f1a49fb5640 1 -- 172.21.15.2:0/2495535473 >> 172.21.15.2:0/2495535473 conn(0x7f1a440fcad0 msgr2=0x7f1a44105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:02.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.847+0000 7f1a49fb5640 1 -- 172.21.15.2:0/2495535473 shutdown_connections 2024-03-15T12:56:02.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:02.847+0000 7f1a49fb5640 1 -- 172.21.15.2:0/2495535473 wait complete. 2024-03-15T12:56:03.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:02 smithi002 ceph-mon[32060]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:03.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:02 smithi184 ceph-mon[35993]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:03.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:02 smithi149 ceph-mon[35927]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:03.110 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 12:56:02 smithi149 ceph-mds[65558]: -- 172.21.15.149:0/3030291094 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555bbd6c81a0 con 0x555bbd69ec00 2024-03-15T12:56:03.110 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 12:56:02 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-h[65554]: starting mds.h at 2024-03-15T12:56:03.110 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 12:56:02 smithi149 bash[65539]: 0e291e77314ffa4e31e0eff7f0ec33f32b0936590b0af89f30288c0bea6d4492 2024-03-15T12:56:03.110 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 12:56:02 smithi149 systemd[1]: Started Ceph mds.h for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:56:04.046 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:03 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2495535473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:04.046 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:03 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:04.046 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:03 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:04.046 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:03 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:04.046 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:03 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:56:04.046 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:03 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:56:04.047 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:03 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:04.047 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:03 smithi184 ceph-mon[35993]: Deploying daemon mds.i on smithi184 2024-03-15T12:56:04.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:03 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2495535473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:04.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:03 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:04.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:03 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:04.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:03 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:04.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:03 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:56:04.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:03 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:56:04.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:03 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:04.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:03 smithi002 ceph-mon[32060]: Deploying daemon mds.i on smithi184 2024-03-15T12:56:04.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:03 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2495535473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:04.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:03 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:04.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:03 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:04.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:03 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:04.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:03 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:56:04.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:03 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:56:04.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:03 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:04.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:03 smithi149 ceph-mon[35927]: Deploying daemon mds.i on smithi184 2024-03-15T12:56:04.844 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:56:04.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.973+0000 7fa85858c640 1 -- 172.21.15.2:0/1742961848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8501030c0 msgr2=0x7fa850109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:04.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.973+0000 7fa85858c640 1 --2- 172.21.15.2:0/1742961848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8501030c0 0x7fa850109950 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa840009ca0 tx=0x7fa84002c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:04.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.973+0000 7fa85858c640 1 -- 172.21.15.2:0/1742961848 shutdown_connections 2024-03-15T12:56:04.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.973+0000 7fa85858c640 1 --2- 172.21.15.2:0/1742961848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8501030c0 0x7fa850109950 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:04.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.973+0000 7fa85858c640 1 --2- 172.21.15.2:0/1742961848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa850102700 0x7fa850102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:04.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.973+0000 7fa85858c640 1 --2- 172.21.15.2:0/1742961848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa850101510 0x7fa850101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:04.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.973+0000 7fa85858c640 1 -- 172.21.15.2:0/1742961848 >> 172.21.15.2:0/1742961848 conn(0x7fa8500fcaa0 msgr2=0x7fa8500feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:04.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.973+0000 7fa85858c640 1 -- 172.21.15.2:0/1742961848 shutdown_connections 2024-03-15T12:56:04.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.973+0000 7fa85858c640 1 -- 172.21.15.2:0/1742961848 wait complete. 2024-03-15T12:56:04.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa85858c640 1 Processor -- start 2024-03-15T12:56:04.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa85858c640 1 -- start start 2024-03-15T12:56:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa85858c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa850101510 0x7fa8500752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa85858c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa850102700 0x7fa8500738e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa85858c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8501030c0 0x7fa850073e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa85858c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa850074580 con 0x7fa850101510 2024-03-15T12:56:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa85858c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8500746f0 con 0x7fa850102700 2024-03-15T12:56:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa85858c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8501a1590 con 0x7fa8501030c0 2024-03-15T12:56:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa856328640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa850101510 0x7fa8500752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa856328640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa850101510 0x7fa8500752a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58382/0 (socket says 172.21.15.2:58382) 2024-03-15T12:56:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa856328640 1 -- 172.21.15.2:0/3049495470 learned_addr learned my addr 172.21.15.2:0/3049495470 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa856b29640 1 --2- 172.21.15.2:0/3049495470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8501030c0 0x7fa850073e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa855b27640 1 --2- 172.21.15.2:0/3049495470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa850102700 0x7fa8500738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa856328640 1 -- 172.21.15.2:0/3049495470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8501030c0 msgr2=0x7fa850073e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa856328640 1 --2- 172.21.15.2:0/3049495470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8501030c0 0x7fa850073e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:04.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa856328640 1 -- 172.21.15.2:0/3049495470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa850102700 msgr2=0x7fa8500738e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:04.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa856328640 1 --2- 172.21.15.2:0/3049495470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa850102700 0x7fa8500738e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:04.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa856328640 1 -- 172.21.15.2:0/3049495470 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa840009900 con 0x7fa850101510 2024-03-15T12:56:04.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa855b27640 1 --2- 172.21.15.2:0/3049495470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa850102700 0x7fa8500738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:56:04.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.974+0000 7fa856328640 1 --2- 172.21.15.2:0/3049495470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa850101510 0x7fa8500752a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa84800d7a0 tx=0x7fa84800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:04.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.975+0000 7fa84f7fe640 1 -- 172.21.15.2:0/3049495470 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848009c30 con 0x7fa850101510 2024-03-15T12:56:04.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.975+0000 7fa85858c640 1 -- 172.21.15.2:0/3049495470 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8501a1840 con 0x7fa850101510 2024-03-15T12:56:04.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.975+0000 7fa84f7fe640 1 -- 172.21.15.2:0/3049495470 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa84800deb0 con 0x7fa850101510 2024-03-15T12:56:04.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.975+0000 7fa84f7fe640 1 -- 172.21.15.2:0/3049495470 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848003e50 con 0x7fa850101510 2024-03-15T12:56:04.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.975+0000 7fa85858c640 1 -- 172.21.15.2:0/3049495470 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8501a1d50 con 0x7fa850101510 2024-03-15T12:56:04.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.976+0000 7fa85858c640 1 -- 172.21.15.2:0/3049495470 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8501087d0 con 0x7fa850101510 2024-03-15T12:56:04.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.977+0000 7fa84f7fe640 1 -- 172.21.15.2:0/3049495470 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fa848004000 con 0x7fa850101510 2024-03-15T12:56:04.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.977+0000 7fa84f7fe640 1 --2- 172.21.15.2:0/3049495470 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa8380a1a30 0x7fa8380a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:04.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.977+0000 7fa855b27640 1 --2- 172.21.15.2:0/3049495470 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa8380a1a30 0x7fa8380a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:04.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.978+0000 7fa84f7fe640 1 -- 172.21.15.2:0/3049495470 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa8480ccd50 con 0x7fa850101510 2024-03-15T12:56:04.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.978+0000 7fa855b27640 1 --2- 172.21.15.2:0/3049495470 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa8380a1a30 0x7fa8380a3ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa850075010 tx=0x7fa844005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:04.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:04.981+0000 7fa84f7fe640 1 -- 172.21.15.2:0/3049495470 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa848096360 con 0x7fa850101510 2024-03-15T12:56:05.000 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:04 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] up:boot 2024-03-15T12:56:05.000 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:04 smithi184 ceph-mon[35993]: fsmap 8 up:standby 2024-03-15T12:56:05.000 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:04 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-15T12:56:05.000 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:04 smithi184 ceph-mon[35993]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:05.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:04 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] up:boot 2024-03-15T12:56:05.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:04 smithi002 ceph-mon[32060]: fsmap 8 up:standby 2024-03-15T12:56:05.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:04 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-15T12:56:05.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:04 smithi002 ceph-mon[32060]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:05.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:04 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] up:boot 2024-03-15T12:56:05.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:04 smithi149 ceph-mon[35927]: fsmap 8 up:standby 2024-03-15T12:56:05.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:04 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-15T12:56:05.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:04 smithi149 ceph-mon[35927]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:05.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:05.203+0000 7fa85858c640 1 -- 172.21.15.2:0/3049495470 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa85010d010 con 0x7fa850101510 2024-03-15T12:56:05.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:05.203+0000 7fa84f7fe640 1 -- 172.21.15.2:0/3049495470 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6865 (secure 0 0 0) 0x7fa848020090 con 0x7fa850101510 2024-03-15T12:56:05.190 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:05.190 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-15T12:56:05.190 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 9 2024-03-15T12:56:05.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:05.205+0000 7fa85858c640 1 -- 172.21.15.2:0/3049495470 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa8380a1a30 msgr2=0x7fa8380a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:05.205+0000 7fa85858c640 1 --2- 172.21.15.2:0/3049495470 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa8380a1a30 0x7fa8380a3ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa850075010 tx=0x7fa844005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:05.205+0000 7fa85858c640 1 -- 172.21.15.2:0/3049495470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa850101510 msgr2=0x7fa8500752a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:05.205+0000 7fa85858c640 1 --2- 172.21.15.2:0/3049495470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa850101510 0x7fa8500752a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa84800d7a0 tx=0x7fa84800dc60 comp rx=0 tx=0).stop 2024-03-15T12:56:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:05.205+0000 7fa85858c640 1 -- 172.21.15.2:0/3049495470 shutdown_connections 2024-03-15T12:56:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:05.205+0000 7fa85858c640 1 --2- 172.21.15.2:0/3049495470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8501030c0 0x7fa850073e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:05.205+0000 7fa85858c640 1 --2- 172.21.15.2:0/3049495470 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa8380a1a30 0x7fa8380a3ef0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:05.205+0000 7fa85858c640 1 --2- 172.21.15.2:0/3049495470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa850102700 0x7fa8500738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:05.205+0000 7fa85858c640 1 --2- 172.21.15.2:0/3049495470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa850101510 0x7fa8500752a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:05.205+0000 7fa85858c640 1 -- 172.21.15.2:0/3049495470 >> 172.21.15.2:0/3049495470 conn(0x7fa8500fcaa0 msgr2=0x7fa8501059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:05.205+0000 7fa85858c640 1 -- 172.21.15.2:0/3049495470 shutdown_connections 2024-03-15T12:56:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:05.205+0000 7fa85858c640 1 -- 172.21.15.2:0/3049495470 wait complete. 2024-03-15T12:56:05.309 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 12:56:05 smithi184 systemd[1]: Starting Ceph mds.i for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T12:56:05.655 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 12:56:05 smithi184 podman[66966]: 2024-03-15 12:56:05.31522576 +0000 UTC m=+0.018304995 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T12:56:05.655 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 12:56:05 smithi184 podman[66966]: 2024-03-15 12:56:05.416386795 +0000 UTC m=+0.119466018 container create b78052221147e9aaf1615fe83b2b625a644608d6ee5633d01557d4f3a7e3a06d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-i, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2) 2024-03-15T12:56:05.655 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 12:56:05 smithi184 podman[66966]: 2024-03-15 12:56:05.66915306 +0000 UTC m=+0.372232284 container init b78052221147e9aaf1615fe83b2b625a644608d6ee5633d01557d4f3a7e3a06d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-i, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-15T12:56:06.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:05 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3049495470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:06.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:05 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3049495470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:06.059 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 12:56:05 smithi184 podman[66966]: 2024-03-15 12:56:05.672070952 +0000 UTC m=+0.375150182 container start b78052221147e9aaf1615fe83b2b625a644608d6ee5633d01557d4f3a7e3a06d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-15T12:56:06.059 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 12:56:05 smithi184 ceph-mds[66985]: -- 172.21.15.184:0/3863989415 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d52e5641a0 con 0x55d52e53ac00 2024-03-15T12:56:06.060 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 12:56:05 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-i[66981]: starting mds.i at 2024-03-15T12:56:06.060 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 12:56:05 smithi184 bash[66966]: b78052221147e9aaf1615fe83b2b625a644608d6ee5633d01557d4f3a7e3a06d 2024-03-15T12:56:06.060 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 12:56:05 smithi184 systemd[1]: Started Ceph mds.i for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:56:06.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:05 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3049495470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:07.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:06 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.184:6838/1170793322,v1:172.21.15.184:6839/1170793322] up:boot 2024-03-15T12:56:07.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:06 smithi002 ceph-mon[32060]: fsmap 9 up:standby 2024-03-15T12:56:07.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:06 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-15T12:56:07.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:06 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:07.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:06 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:07.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:06 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:07.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:06 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:56:07.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:06 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:56:07.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:06 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:07.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:06 smithi002 ceph-mon[32060]: Deploying daemon mds.g on smithi002 2024-03-15T12:56:07.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:06 smithi002 ceph-mon[32060]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:07.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:06 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.184:6838/1170793322,v1:172.21.15.184:6839/1170793322] up:boot 2024-03-15T12:56:07.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:06 smithi184 ceph-mon[35993]: fsmap 9 up:standby 2024-03-15T12:56:07.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:06 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-15T12:56:07.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:06 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:07.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:06 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:07.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:06 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:07.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:06 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:56:07.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:06 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:56:07.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:06 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:07.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:06 smithi184 ceph-mon[35993]: Deploying daemon mds.g on smithi002 2024-03-15T12:56:07.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:06 smithi184 ceph-mon[35993]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:07.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:06 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.184:6838/1170793322,v1:172.21.15.184:6839/1170793322] up:boot 2024-03-15T12:56:07.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:06 smithi149 ceph-mon[35927]: fsmap 9 up:standby 2024-03-15T12:56:07.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:06 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-15T12:56:07.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:06 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:07.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:06 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:07.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:06 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:07.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:06 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:56:07.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:06 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:56:07.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:06 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:07.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:06 smithi149 ceph-mon[35927]: Deploying daemon mds.g on smithi002 2024-03-15T12:56:07.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:06 smithi149 ceph-mon[35927]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:07.204 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:56:07.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.328+0000 7fa9a74e1640 1 -- 172.21.15.2:0/2459708514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9a0101fb0 msgr2=0x7fa9a01043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:07.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.328+0000 7fa9a74e1640 1 --2- 172.21.15.2:0/2459708514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9a0101fb0 0x7fa9a01043a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009900 tx=0x7fa99c02bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:07.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.328+0000 7fa9a74e1640 1 -- 172.21.15.2:0/2459708514 shutdown_connections 2024-03-15T12:56:07.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.328+0000 7fa9a74e1640 1 --2- 172.21.15.2:0/2459708514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9a0107210 0x7fa9a0109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:07.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.328+0000 7fa9a74e1640 1 --2- 172.21.15.2:0/2459708514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9a01048e0 0x7fa9a0106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:07.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.328+0000 7fa9a74e1640 1 --2- 172.21.15.2:0/2459708514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9a0101fb0 0x7fa9a01043a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:07.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.328+0000 7fa9a74e1640 1 -- 172.21.15.2:0/2459708514 >> 172.21.15.2:0/2459708514 conn(0x7fa9a00fb9a0 msgr2=0x7fa9a00fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:07.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.328+0000 7fa9a74e1640 1 -- 172.21.15.2:0/2459708514 shutdown_connections 2024-03-15T12:56:07.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.328+0000 7fa9a74e1640 1 -- 172.21.15.2:0/2459708514 wait complete. 2024-03-15T12:56:07.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.329+0000 7fa9a74e1640 1 Processor -- start 2024-03-15T12:56:07.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.329+0000 7fa9a74e1640 1 -- start start 2024-03-15T12:56:07.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.329+0000 7fa9a74e1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9a0101fb0 0x7fa9a0197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:07.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.329+0000 7fa9a74e1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9a01048e0 0x7fa9a0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:07.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.329+0000 7fa9a74e1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9a0107210 0x7fa9a019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:07.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.329+0000 7fa9a74e1640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a019f080 con 0x7fa9a01048e0 2024-03-15T12:56:07.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.329+0000 7fa9a74e1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a019f1f0 con 0x7fa9a0107210 2024-03-15T12:56:07.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.329+0000 7fa9a74e1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a019f3b0 con 0x7fa9a0101fb0 2024-03-15T12:56:07.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.329+0000 7fa9a4a7c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9a01048e0 0x7fa9a0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:07.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.329+0000 7fa9a4a7c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9a01048e0 0x7fa9a0197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58394/0 (socket says 172.21.15.2:58394) 2024-03-15T12:56:07.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.329+0000 7fa9a4a7c640 1 -- 172.21.15.2:0/294530064 learned_addr learned my addr 172.21.15.2:0/294530064 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:07.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.329+0000 7fa9a527d640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9a0101fb0 0x7fa9a0197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:07.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.329+0000 7fa9a5a7e640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9a0107210 0x7fa9a019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:07.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.329+0000 7fa9a4a7c640 1 -- 172.21.15.2:0/294530064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9a0101fb0 msgr2=0x7fa9a0197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:07.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.330+0000 7fa9a4a7c640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9a0101fb0 0x7fa9a0197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:07.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.330+0000 7fa9a4a7c640 1 -- 172.21.15.2:0/294530064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9a0107210 msgr2=0x7fa9a019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:07.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.330+0000 7fa9a4a7c640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9a0107210 0x7fa9a019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:07.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.330+0000 7fa9a4a7c640 1 -- 172.21.15.2:0/294530064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c009560 con 0x7fa9a01048e0 2024-03-15T12:56:07.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.330+0000 7fa9a527d640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9a0101fb0 0x7fa9a0197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:56:07.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.330+0000 7fa9a5a7e640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9a0107210 0x7fa9a019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T12:56:07.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.330+0000 7fa9a4a7c640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9a01048e0 0x7fa9a0197ad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa99000c970 tx=0x7fa99000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:07.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.330+0000 7fa9967fc640 1 -- 172.21.15.2:0/294530064 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990004440 con 0x7fa9a01048e0 2024-03-15T12:56:07.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.330+0000 7fa9967fc640 1 -- 172.21.15.2:0/294530064 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa990007d30 con 0x7fa9a01048e0 2024-03-15T12:56:07.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.330+0000 7fa9a74e1640 1 -- 172.21.15.2:0/294530064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a019f690 con 0x7fa9a01048e0 2024-03-15T12:56:07.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.330+0000 7fa9967fc640 1 -- 172.21.15.2:0/294530064 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99000f450 con 0x7fa9a01048e0 2024-03-15T12:56:07.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.330+0000 7fa9a74e1640 1 -- 172.21.15.2:0/294530064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a019fba0 con 0x7fa9a01048e0 2024-03-15T12:56:07.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.331+0000 7fa9a74e1640 1 -- 172.21.15.2:0/294530064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a00fd490 con 0x7fa9a01048e0 2024-03-15T12:56:07.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.332+0000 7fa9967fc640 1 -- 172.21.15.2:0/294530064 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fa990003780 con 0x7fa9a01048e0 2024-03-15T12:56:07.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.332+0000 7fa9967fc640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa98c0a1a30 0x7fa98c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:07.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.333+0000 7fa9a527d640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa98c0a1a30 0x7fa98c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:07.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.333+0000 7fa9967fc640 1 -- 172.21.15.2:0/294530064 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa9900cd560 con 0x7fa9a01048e0 2024-03-15T12:56:07.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.333+0000 7fa9a527d640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa98c0a1a30 0x7fa98c0a3ef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa99c006010 tx=0x7fa99c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:07.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.336+0000 7fa9967fc640 1 -- 172.21.15.2:0/294530064 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa990096c10 con 0x7fa9a01048e0 2024-03-15T12:56:07.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.559+0000 7fa9a74e1640 1 -- 172.21.15.2:0/294530064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9a010cf10 con 0x7fa9a01048e0 2024-03-15T12:56:07.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.559+0000 7fa9967fc640 1 -- 172.21.15.2:0/294530064 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7661 (secure 0 0 0) 0x7fa99009a2d0 con 0x7fa9a01048e0 2024-03-15T12:56:07.546 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:07.546 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-15T12:56:07.546 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 10 2024-03-15T12:56:07.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.561+0000 7fa9a74e1640 1 -- 172.21.15.2:0/294530064 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa98c0a1a30 msgr2=0x7fa98c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:07.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.561+0000 7fa9a74e1640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa98c0a1a30 0x7fa98c0a3ef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa99c006010 tx=0x7fa99c000bc0 comp rx=0 tx=0).stop 2024-03-15T12:56:07.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.561+0000 7fa9a74e1640 1 -- 172.21.15.2:0/294530064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9a01048e0 msgr2=0x7fa9a0197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:07.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.561+0000 7fa9a74e1640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9a01048e0 0x7fa9a0197ad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa99000c970 tx=0x7fa99000ce30 comp rx=0 tx=0).stop 2024-03-15T12:56:07.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.562+0000 7fa9a74e1640 1 -- 172.21.15.2:0/294530064 shutdown_connections 2024-03-15T12:56:07.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.562+0000 7fa9a74e1640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa98c0a1a30 0x7fa98c0a3ef0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:07.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.562+0000 7fa9a74e1640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9a0107210 0x7fa9a019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:07.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.562+0000 7fa9a74e1640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9a01048e0 0x7fa9a0197ad0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:07.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.562+0000 7fa9a74e1640 1 --2- 172.21.15.2:0/294530064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9a0101fb0 0x7fa9a0197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:07.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.562+0000 7fa9a74e1640 1 -- 172.21.15.2:0/294530064 >> 172.21.15.2:0/294530064 conn(0x7fa9a00fb9a0 msgr2=0x7fa9a01028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:07.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.562+0000 7fa9a74e1640 1 -- 172.21.15.2:0/294530064 shutdown_connections 2024-03-15T12:56:07.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:07.562+0000 7fa9a74e1640 1 -- 172.21.15.2:0/294530064 wait complete. 2024-03-15T12:56:08.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:07 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/294530064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:08.293 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:07 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/294530064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:08.293 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 12:56:08 smithi002 systemd[1]: Starting Ceph mds.g for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T12:56:08.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:07 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/294530064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:08.553 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 12:56:08 smithi002 podman[73770]: 2024-03-15 12:56:08.210065334 +0000 UTC m=+0.017516275 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T12:56:08.554 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 12:56:08 smithi002 podman[73770]: 2024-03-15 12:56:08.307774933 +0000 UTC m=+0.115225871 container create 8b4c087eff7f86355f50c41890884c097498cce1d5145a58dd77c73d140ef5ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-g, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-15T12:56:08.554 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 12:56:08 smithi002 podman[73770]: 2024-03-15 12:56:08.518850014 +0000 UTC m=+0.326300961 container init 8b4c087eff7f86355f50c41890884c097498cce1d5145a58dd77c73d140ef5ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-squid-aba3c1d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-15T12:56:08.554 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 12:56:08 smithi002 podman[73770]: 2024-03-15 12:56:08.521475477 +0000 UTC m=+0.328926422 container start 8b4c087eff7f86355f50c41890884c097498cce1d5145a58dd77c73d140ef5ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-15T12:56:08.554 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 12:56:08 smithi002 ceph-mds[73789]: -- 172.21.15.2:0/898283346 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564a01f941a0 con 0x564a01f6a800 2024-03-15T12:56:08.554 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 12:56:08 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-g[73785]: starting mds.g at 2024-03-15T12:56:09.053 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 12:56:08 smithi002 bash[73770]: 8b4c087eff7f86355f50c41890884c097498cce1d5145a58dd77c73d140ef5ea 2024-03-15T12:56:09.053 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 12:56:08 smithi002 systemd[1]: Started Ceph mds.g for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:56:09.560 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:56:09.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 -- 172.21.15.2:0/3292856424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c801048c0 msgr2=0x7f4c80106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:09.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 --2- 172.21.15.2:0/3292856424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c801048c0 0x7f4c80106cb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009900 tx=0x7f4c7002bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:09.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 -- 172.21.15.2:0/3292856424 shutdown_connections 2024-03-15T12:56:09.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 --2- 172.21.15.2:0/3292856424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c801071f0 0x7f4c80109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 --2- 172.21.15.2:0/3292856424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c801048c0 0x7f4c80106cb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 --2- 172.21.15.2:0/3292856424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c80101f90 0x7f4c80104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 -- 172.21.15.2:0/3292856424 >> 172.21.15.2:0/3292856424 conn(0x7f4c800fb980 msgr2=0x7f4c800fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 -- 172.21.15.2:0/3292856424 shutdown_connections 2024-03-15T12:56:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 -- 172.21.15.2:0/3292856424 wait complete. 2024-03-15T12:56:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 Processor -- start 2024-03-15T12:56:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 -- start start 2024-03-15T12:56:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c80101f90 0x7f4c80197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c801048c0 0x7f4c80197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c801071f0 0x7f4c8019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8019f140 con 0x7f4c80101f90 2024-03-15T12:56:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8019f2b0 con 0x7f4c801071f0 2024-03-15T12:56:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.682+0000 7f4c87e9a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8019f470 con 0x7f4c801048c0 2024-03-15T12:56:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c85c36640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c80101f90 0x7f4c80197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:09.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c85c36640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c80101f90 0x7f4c80197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36538/0 (socket says 172.21.15.2:36538) 2024-03-15T12:56:09.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c85c36640 1 -- 172.21.15.2:0/3459795533 learned_addr learned my addr 172.21.15.2:0/3459795533 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:09.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c85435640 1 --2- 172.21.15.2:0/3459795533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c801048c0 0x7f4c80197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:09.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c86437640 1 --2- 172.21.15.2:0/3459795533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c801071f0 0x7f4c8019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:09.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c85c36640 1 -- 172.21.15.2:0/3459795533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c801048c0 msgr2=0x7f4c80197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:09.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c85c36640 1 --2- 172.21.15.2:0/3459795533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c801048c0 0x7f4c80197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:09.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c85c36640 1 -- 172.21.15.2:0/3459795533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c801071f0 msgr2=0x7f4c8019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:09.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c85c36640 1 --2- 172.21.15.2:0/3459795533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c801071f0 0x7f4c8019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:09.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c85c36640 1 -- 172.21.15.2:0/3459795533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c70009560 con 0x7f4c80101f90 2024-03-15T12:56:09.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c86437640 1 --2- 172.21.15.2:0/3459795533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c801071f0 0x7f4c8019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:56:09.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c85c36640 1 --2- 172.21.15.2:0/3459795533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c80101f90 0x7f4c80197650 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800b4a0 tx=0x7f4c7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:09.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c76ffd640 1 -- 172.21.15.2:0/3459795533 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78010610 con 0x7f4c80101f90 2024-03-15T12:56:09.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c76ffd640 1 -- 172.21.15.2:0/3459795533 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4c78010c40 con 0x7f4c80101f90 2024-03-15T12:56:09.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c87e9a640 1 -- 172.21.15.2:0/3459795533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8019f750 con 0x7f4c80101f90 2024-03-15T12:56:09.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c76ffd640 1 -- 172.21.15.2:0/3459795533 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78017de0 con 0x7f4c80101f90 2024-03-15T12:56:09.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.683+0000 7f4c87e9a640 1 -- 172.21.15.2:0/3459795533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8019fcb0 con 0x7f4c80101f90 2024-03-15T12:56:09.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.684+0000 7f4c87e9a640 1 -- 172.21.15.2:0/3459795533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c800fd470 con 0x7f4c80101f90 2024-03-15T12:56:09.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.686+0000 7f4c76ffd640 1 -- 172.21.15.2:0/3459795533 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f4c78010770 con 0x7f4c80101f90 2024-03-15T12:56:09.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.686+0000 7f4c76ffd640 1 --2- 172.21.15.2:0/3459795533 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4c6c0a1a30 0x7f4c6c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:09.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.687+0000 7f4c85435640 1 --2- 172.21.15.2:0/3459795533 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4c6c0a1a30 0x7f4c6c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:09.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.687+0000 7f4c76ffd640 1 -- 172.21.15.2:0/3459795533 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4c780ccba0 con 0x7f4c80101f90 2024-03-15T12:56:09.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.687+0000 7f4c85435640 1 --2- 172.21.15.2:0/3459795533 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4c6c0a1a30 0x7f4c6c0a3ef0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4c7002c740 tx=0x7f4c70005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:09.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.694+0000 7f4c76ffd640 1 -- 172.21.15.2:0/3459795533 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4c780cce60 con 0x7f4c80101f90 2024-03-15T12:56:09.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.917+0000 7f4c87e9a640 1 -- 172.21.15.2:0/3459795533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c8010cf70 con 0x7f4c80101f90 2024-03-15T12:56:09.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.917+0000 7f4c76ffd640 1 -- 172.21.15.2:0/3459795533 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8450 (secure 0 0 0) 0x7f4c78096250 con 0x7f4c80101f90 2024-03-15T12:56:09.904 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:09.904 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-15T12:56:09.904 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 11 2024-03-15T12:56:09.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.919+0000 7f4c87e9a640 1 -- 172.21.15.2:0/3459795533 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4c6c0a1a30 msgr2=0x7f4c6c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:09.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.919+0000 7f4c87e9a640 1 --2- 172.21.15.2:0/3459795533 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4c6c0a1a30 0x7f4c6c0a3ef0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4c7002c740 tx=0x7f4c70005fd0 comp rx=0 tx=0).stop 2024-03-15T12:56:09.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.919+0000 7f4c87e9a640 1 -- 172.21.15.2:0/3459795533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c80101f90 msgr2=0x7f4c80197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:09.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.919+0000 7f4c87e9a640 1 --2- 172.21.15.2:0/3459795533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c80101f90 0x7f4c80197650 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800b4a0 tx=0x7f4c7800b960 comp rx=0 tx=0).stop 2024-03-15T12:56:09.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.919+0000 7f4c87e9a640 1 -- 172.21.15.2:0/3459795533 shutdown_connections 2024-03-15T12:56:09.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.919+0000 7f4c87e9a640 1 --2- 172.21.15.2:0/3459795533 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4c6c0a1a30 0x7f4c6c0a3ef0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:09.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.919+0000 7f4c87e9a640 1 --2- 172.21.15.2:0/3459795533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c801071f0 0x7f4c8019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:09.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.919+0000 7f4c87e9a640 1 --2- 172.21.15.2:0/3459795533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c801048c0 0x7f4c80197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:09.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.919+0000 7f4c87e9a640 1 --2- 172.21.15.2:0/3459795533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c80101f90 0x7f4c80197650 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:09.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.919+0000 7f4c87e9a640 1 -- 172.21.15.2:0/3459795533 >> 172.21.15.2:0/3459795533 conn(0x7f4c800fb980 msgr2=0x7f4c801029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:09.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.919+0000 7f4c87e9a640 1 -- 172.21.15.2:0/3459795533 shutdown_connections 2024-03-15T12:56:09.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:09.919+0000 7f4c87e9a640 1 -- 172.21.15.2:0/3459795533 wait complete. 2024-03-15T12:56:09.915 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:09.915 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:09.915 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:09.916 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:56:09.916 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:56:09.916 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:09.916 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:09 smithi002 ceph-mon[32060]: Deploying daemon mds.a on smithi002 2024-03-15T12:56:09.916 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:09 smithi002 ceph-mon[32060]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:09.916 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:09 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.2:6838/2513297233,v1:172.21.15.2:6839/2513297233] up:boot 2024-03-15T12:56:09.916 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:09 smithi002 ceph-mon[32060]: fsmap 10 up:standby 2024-03-15T12:56:09.916 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:09 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-15T12:56:10.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:10.076 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:10.076 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:10.076 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:56:10.076 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:56:10.076 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:10.076 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:09 smithi184 ceph-mon[35993]: Deploying daemon mds.a on smithi002 2024-03-15T12:56:10.076 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:09 smithi184 ceph-mon[35993]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:10.076 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:09 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.2:6838/2513297233,v1:172.21.15.2:6839/2513297233] up:boot 2024-03-15T12:56:10.076 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:09 smithi184 ceph-mon[35993]: fsmap 10 up:standby 2024-03-15T12:56:10.076 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:09 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-15T12:56:10.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:09 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:10.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:09 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:10.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:09 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:10.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:09 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:56:10.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:09 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:56:10.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:09 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:10.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:09 smithi149 ceph-mon[35927]: Deploying daemon mds.a on smithi002 2024-03-15T12:56:10.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:09 smithi149 ceph-mon[35927]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:10.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:09 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.2:6838/2513297233,v1:172.21.15.2:6839/2513297233] up:boot 2024-03-15T12:56:10.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:09 smithi149 ceph-mon[35927]: fsmap 10 up:standby 2024-03-15T12:56:10.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:09 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-15T12:56:10.784 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:10 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3459795533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:10.785 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 12:56:10 smithi002 systemd[1]: Starting Ceph mds.a for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T12:56:11.053 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 12:56:10 smithi002 podman[74119]: 2024-03-15 12:56:10.874957844 +0000 UTC m=+0.019195520 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T12:56:11.053 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 12:56:10 smithi002 podman[74119]: 2024-03-15 12:56:10.999040247 +0000 UTC m=+0.143277961 container create 8b4995460d1b5663a41702cde9d280171af47520d697d82bd3713c2ba24bae0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-squid-aba3c1d, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-15T12:56:11.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:10 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3459795533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:11.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:10 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3459795533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:11.553 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 12:56:11 smithi002 podman[74119]: 2024-03-15 12:56:11.226738163 +0000 UTC m=+0.370975835 container init 8b4995460d1b5663a41702cde9d280171af47520d697d82bd3713c2ba24bae0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-15T12:56:11.553 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 12:56:11 smithi002 podman[74119]: 2024-03-15 12:56:11.229427615 +0000 UTC m=+0.373665287 container start 8b4995460d1b5663a41702cde9d280171af47520d697d82bd3713c2ba24bae0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-15T12:56:11.553 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 12:56:11 smithi002 ceph-mds[74147]: -- 172.21.15.2:0/2026823083 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e15cedc1a0 con 0x55e15ceb2800 2024-03-15T12:56:11.553 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 12:56:11 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-a[74141]: starting mds.a at 2024-03-15T12:56:11.553 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 12:56:11 smithi002 bash[74119]: 8b4995460d1b5663a41702cde9d280171af47520d697d82bd3713c2ba24bae0a 2024-03-15T12:56:11.554 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 12:56:11 smithi002 systemd[1]: Started Ceph mds.a for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:56:11.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:11 smithi149 ceph-mon[35927]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:11.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:11 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:11.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:11 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:11.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:11 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:11.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:11 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:56:11.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:11 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:56:11.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:11 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:11.918 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:56:12.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.038+0000 7f6c29a2e640 1 -- 172.21.15.2:0/398327016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c24101510 msgr2=0x7f6c24101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:12.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.038+0000 7f6c29a2e640 1 --2- 172.21.15.2:0/398327016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c24101510 0x7f6c24101930 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009900 tx=0x7f6c1c02bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:12.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c29a2e640 1 -- 172.21.15.2:0/398327016 shutdown_connections 2024-03-15T12:56:12.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c29a2e640 1 --2- 172.21.15.2:0/398327016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c241030c0 0x7f6c24109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:12.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c29a2e640 1 --2- 172.21.15.2:0/398327016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c24102700 0x7f6c24102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:12.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c29a2e640 1 --2- 172.21.15.2:0/398327016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c24101510 0x7f6c24101930 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:12.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c29a2e640 1 -- 172.21.15.2:0/398327016 >> 172.21.15.2:0/398327016 conn(0x7f6c240fcaa0 msgr2=0x7f6c240feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:12.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c29a2e640 1 -- 172.21.15.2:0/398327016 shutdown_connections 2024-03-15T12:56:12.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c29a2e640 1 -- 172.21.15.2:0/398327016 wait complete. 2024-03-15T12:56:12.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c29a2e640 1 Processor -- start 2024-03-15T12:56:12.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c29a2e640 1 -- start start 2024-03-15T12:56:12.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c29a2e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c24101510 0x7f6c241975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:12.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c29a2e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c24102700 0x7f6c24197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:12.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c29a2e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c241030c0 0x7f6c2419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:12.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c29a2e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2419f0b0 con 0x7f6c24102700 2024-03-15T12:56:12.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c29a2e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2419f220 con 0x7f6c241030c0 2024-03-15T12:56:12.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c29a2e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2419f3e0 con 0x7f6c24101510 2024-03-15T12:56:12.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.039+0000 7f6c227fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c24102700 0x7f6c24197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:12.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c227fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c24102700 0x7f6c24197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36568/0 (socket says 172.21.15.2:36568) 2024-03-15T12:56:12.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c227fc640 1 -- 172.21.15.2:0/1091052341 learned_addr learned my addr 172.21.15.2:0/1091052341 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:12.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c237fe640 1 --2- 172.21.15.2:0/1091052341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c241030c0 0x7f6c2419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:12.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c22ffd640 1 --2- 172.21.15.2:0/1091052341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c24101510 0x7f6c241975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:12.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c227fc640 1 -- 172.21.15.2:0/1091052341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c24101510 msgr2=0x7f6c241975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:12.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c227fc640 1 --2- 172.21.15.2:0/1091052341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c24101510 0x7f6c241975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:12.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c227fc640 1 -- 172.21.15.2:0/1091052341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c241030c0 msgr2=0x7f6c2419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:12.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c227fc640 1 --2- 172.21.15.2:0/1091052341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c241030c0 0x7f6c2419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:12.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c227fc640 1 -- 172.21.15.2:0/1091052341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c009560 con 0x7f6c24102700 2024-03-15T12:56:12.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c22ffd640 1 --2- 172.21.15.2:0/1091052341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c24101510 0x7f6c241975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:56:12.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c227fc640 1 --2- 172.21.15.2:0/1091052341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c24102700 0x7f6c24197b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00c970 tx=0x7f6c0c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:12.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c28a2c640 1 -- 172.21.15.2:0/1091052341 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c007bb0 con 0x7f6c24102700 2024-03-15T12:56:12.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c28a2c640 1 -- 172.21.15.2:0/1091052341 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6c0c007d10 con 0x7f6c24102700 2024-03-15T12:56:12.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c29a2e640 1 -- 172.21.15.2:0/1091052341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2419f6c0 con 0x7f6c24102700 2024-03-15T12:56:12.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c28a2c640 1 -- 172.21.15.2:0/1091052341 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c00f450 con 0x7f6c24102700 2024-03-15T12:56:12.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.040+0000 7f6c29a2e640 1 -- 172.21.15.2:0/1091052341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2419fbd0 con 0x7f6c24102700 2024-03-15T12:56:12.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.041+0000 7f6c29a2e640 1 -- 172.21.15.2:0/1091052341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c24108830 con 0x7f6c24102700 2024-03-15T12:56:12.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.042+0000 7f6c28a2c640 1 -- 172.21.15.2:0/1091052341 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f6c0c004f40 con 0x7f6c24102700 2024-03-15T12:56:12.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.043+0000 7f6c28a2c640 1 --2- 172.21.15.2:0/1091052341 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6c100a1ab0 0x7f6c100a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:12.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.043+0000 7f6c22ffd640 1 --2- 172.21.15.2:0/1091052341 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6c100a1ab0 0x7f6c100a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:12.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.043+0000 7f6c28a2c640 1 -- 172.21.15.2:0/1091052341 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6c0c0ccd80 con 0x7f6c24102700 2024-03-15T12:56:12.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.043+0000 7f6c22ffd640 1 --2- 172.21.15.2:0/1091052341 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6c100a1ab0 0x7f6c100a3f70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c02c740 tx=0x7f6c1c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:12.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.047+0000 7f6c28a2c640 1 -- 172.21.15.2:0/1091052341 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6c0c096390 con 0x7f6c24102700 2024-03-15T12:56:12.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:11 smithi002 ceph-mon[32060]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:12.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:11 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:12.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:11 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:12.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:11 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:12.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:11 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:56:12.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:11 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:56:12.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:11 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:12.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:11 smithi184 ceph-mon[35993]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:12.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:11 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:12.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:11 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:12.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:11 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:12.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:11 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-15T12:56:12.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:11 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-15T12:56:12.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:11 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:12.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.267+0000 7f6c29a2e640 1 -- 172.21.15.2:0/1091052341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c2410cef0 con 0x7f6c24102700 2024-03-15T12:56:12.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.268+0000 7f6c28a2c640 1 -- 172.21.15.2:0/1091052341 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9239 (secure 0 0 0) 0x7f6c0c01d090 con 0x7f6c24102700 2024-03-15T12:56:12.254 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:12.254 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-15T12:56:12.255 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 12 2024-03-15T12:56:12.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.270+0000 7f6c29a2e640 1 -- 172.21.15.2:0/1091052341 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6c100a1ab0 msgr2=0x7f6c100a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:12.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.270+0000 7f6c29a2e640 1 --2- 172.21.15.2:0/1091052341 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6c100a1ab0 0x7f6c100a3f70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c02c740 tx=0x7f6c1c005fd0 comp rx=0 tx=0).stop 2024-03-15T12:56:12.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.270+0000 7f6c29a2e640 1 -- 172.21.15.2:0/1091052341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c24102700 msgr2=0x7f6c24197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:12.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.270+0000 7f6c29a2e640 1 --2- 172.21.15.2:0/1091052341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c24102700 0x7f6c24197b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00c970 tx=0x7f6c0c00ce30 comp rx=0 tx=0).stop 2024-03-15T12:56:12.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.270+0000 7f6c29a2e640 1 -- 172.21.15.2:0/1091052341 shutdown_connections 2024-03-15T12:56:12.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.270+0000 7f6c29a2e640 1 --2- 172.21.15.2:0/1091052341 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6c100a1ab0 0x7f6c100a3f70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:12.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.270+0000 7f6c29a2e640 1 --2- 172.21.15.2:0/1091052341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c241030c0 0x7f6c2419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:12.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.270+0000 7f6c29a2e640 1 --2- 172.21.15.2:0/1091052341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c24102700 0x7f6c24197b00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:12.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.270+0000 7f6c29a2e640 1 --2- 172.21.15.2:0/1091052341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c24101510 0x7f6c241975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:12.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.270+0000 7f6c29a2e640 1 -- 172.21.15.2:0/1091052341 >> 172.21.15.2:0/1091052341 conn(0x7f6c240fcaa0 msgr2=0x7f6c24105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:12.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.270+0000 7f6c29a2e640 1 -- 172.21.15.2:0/1091052341 shutdown_connections 2024-03-15T12:56:12.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:12.270+0000 7f6c29a2e640 1 -- 172.21.15.2:0/1091052341 wait complete. 2024-03-15T12:56:12.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:12 smithi184 ceph-mon[35993]: Deploying daemon mds.b on smithi149 2024-03-15T12:56:12.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:12 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.2:6840/3747539896,v1:172.21.15.2:6841/3747539896] up:boot 2024-03-15T12:56:12.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:12 smithi184 ceph-mon[35993]: fsmap 11 up:standby 2024-03-15T12:56:12.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:12 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-15T12:56:12.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:12 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1091052341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:13.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:12 smithi002 ceph-mon[32060]: Deploying daemon mds.b on smithi149 2024-03-15T12:56:13.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:12 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.2:6840/3747539896,v1:172.21.15.2:6841/3747539896] up:boot 2024-03-15T12:56:13.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:12 smithi002 ceph-mon[32060]: fsmap 11 up:standby 2024-03-15T12:56:13.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:12 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-15T12:56:13.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:12 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1091052341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:13.091 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:12 smithi149 ceph-mon[35927]: Deploying daemon mds.b on smithi149 2024-03-15T12:56:13.091 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:12 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.2:6840/3747539896,v1:172.21.15.2:6841/3747539896] up:boot 2024-03-15T12:56:13.091 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:12 smithi149 ceph-mon[35927]: fsmap 11 up:standby 2024-03-15T12:56:13.091 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:12 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-15T12:56:13.091 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:12 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1091052341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:13.710 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 12:56:13 smithi149 systemd[1]: Starting Ceph mds.b for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T12:56:14.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:13 smithi002 ceph-mon[32060]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:14.055 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:13 smithi149 ceph-mon[35927]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:14.055 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 12:56:13 smithi149 podman[65847]: 2024-03-15 12:56:13.703476789 +0000 UTC m=+0.017938148 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T12:56:14.055 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 12:56:13 smithi149 podman[65847]: 2024-03-15 12:56:13.825417181 +0000 UTC m=+0.139878537 container create bfb0b91488884145c0ece4321164f1c86145119d7bf7437b3f7b18d998865016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-b, ceph=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-15T12:56:14.055 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 12:56:14 smithi149 podman[65847]: 2024-03-15 12:56:14.069610642 +0000 UTC m=+0.384072007 container init bfb0b91488884145c0ece4321164f1c86145119d7bf7437b3f7b18d998865016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-15T12:56:14.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:13 smithi184 ceph-mon[35993]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:14.268 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:56:14.360 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 12:56:14 smithi149 podman[65847]: 2024-03-15 12:56:14.072321274 +0000 UTC m=+0.386782640 container start bfb0b91488884145c0ece4321164f1c86145119d7bf7437b3f7b18d998865016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-15T12:56:14.360 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 12:56:14 smithi149 ceph-mds[65865]: -- 172.21.15.149:0/542518702 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b2d73ba1a0 con 0x55b2d7390c00 2024-03-15T12:56:14.360 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 12:56:14 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-b[65861]: starting mds.b at 2024-03-15T12:56:14.360 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 12:56:14 smithi149 bash[65847]: bfb0b91488884145c0ece4321164f1c86145119d7bf7437b3f7b18d998865016 2024-03-15T12:56:14.360 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 12:56:14 smithi149 systemd[1]: Started Ceph mds.b for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T12:56:14.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.392+0000 7f813a2ae640 1 -- 172.21.15.2:0/644490397 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81341030c0 msgr2=0x7f8134109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:14.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.392+0000 7f813a2ae640 1 --2- 172.21.15.2:0/644490397 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81341030c0 0x7f8134109950 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f812c009ca0 tx=0x7f812c02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:14.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.392+0000 7f813a2ae640 1 -- 172.21.15.2:0/644490397 shutdown_connections 2024-03-15T12:56:14.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.392+0000 7f813a2ae640 1 --2- 172.21.15.2:0/644490397 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81341030c0 0x7f8134109950 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:14.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.392+0000 7f813a2ae640 1 --2- 172.21.15.2:0/644490397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8134102700 0x7f8134102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:14.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.392+0000 7f813a2ae640 1 --2- 172.21.15.2:0/644490397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8134101510 0x7f8134101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:14.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.392+0000 7f813a2ae640 1 -- 172.21.15.2:0/644490397 >> 172.21.15.2:0/644490397 conn(0x7f81340fcaa0 msgr2=0x7f81340feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:14.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.392+0000 7f813a2ae640 1 -- 172.21.15.2:0/644490397 shutdown_connections 2024-03-15T12:56:14.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.392+0000 7f813a2ae640 1 -- 172.21.15.2:0/644490397 wait complete. 2024-03-15T12:56:14.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.393+0000 7f813a2ae640 1 Processor -- start 2024-03-15T12:56:14.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.393+0000 7f813a2ae640 1 -- start start 2024-03-15T12:56:14.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.393+0000 7f813a2ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8134101510 0x7f8134197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:14.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.393+0000 7f813a2ae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8134102700 0x7f8134197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:14.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.393+0000 7f813a2ae640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81341030c0 0x7f813419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:14.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.393+0000 7f813a2ae640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813419f060 con 0x7f8134102700 2024-03-15T12:56:14.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.393+0000 7f813a2ae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813419f1d0 con 0x7f8134101510 2024-03-15T12:56:14.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.393+0000 7f813a2ae640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813419f390 con 0x7f81341030c0 2024-03-15T12:56:14.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.393+0000 7f81337fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8134102700 0x7f8134197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:14.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f8133fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8134101510 0x7f8134197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:14.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f81337fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8134102700 0x7f8134197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36574/0 (socket says 172.21.15.2:36574) 2024-03-15T12:56:14.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f81337fe640 1 -- 172.21.15.2:0/1070124103 learned_addr learned my addr 172.21.15.2:0/1070124103 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:14.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f813884b640 1 --2- 172.21.15.2:0/1070124103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81341030c0 0x7f813419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:14.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f81337fe640 1 -- 172.21.15.2:0/1070124103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81341030c0 msgr2=0x7f813419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:14.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f81337fe640 1 --2- 172.21.15.2:0/1070124103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81341030c0 0x7f813419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:14.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f81337fe640 1 -- 172.21.15.2:0/1070124103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8134101510 msgr2=0x7f8134197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:14.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f81337fe640 1 --2- 172.21.15.2:0/1070124103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8134101510 0x7f8134197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:14.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f81337fe640 1 -- 172.21.15.2:0/1070124103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c009900 con 0x7f8134102700 2024-03-15T12:56:14.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f8133fff640 1 --2- 172.21.15.2:0/1070124103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8134101510 0x7f8134197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:14.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f81337fe640 1 --2- 172.21.15.2:0/1070124103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8134102700 0x7f8134197ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f812400e9f0 tx=0x7f812400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:14.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f81317fa640 1 -- 172.21.15.2:0/1070124103 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124009700 con 0x7f8134102700 2024-03-15T12:56:14.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f81317fa640 1 -- 172.21.15.2:0/1070124103 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f812400cd80 con 0x7f8134102700 2024-03-15T12:56:14.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f813a2ae640 1 -- 172.21.15.2:0/1070124103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813419f670 con 0x7f8134102700 2024-03-15T12:56:14.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f81317fa640 1 -- 172.21.15.2:0/1070124103 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124018920 con 0x7f8134102700 2024-03-15T12:56:14.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.394+0000 7f813a2ae640 1 -- 172.21.15.2:0/1070124103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813419fb80 con 0x7f8134102700 2024-03-15T12:56:14.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.395+0000 7f813a2ae640 1 -- 172.21.15.2:0/1070124103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8134108830 con 0x7f8134102700 2024-03-15T12:56:14.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.396+0000 7f81317fa640 1 -- 172.21.15.2:0/1070124103 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f8124018a80 con 0x7f8134102700 2024-03-15T12:56:14.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.397+0000 7f81317fa640 1 --2- 172.21.15.2:0/1070124103 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f811c0a1a30 0x7f811c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:14.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.397+0000 7f8133fff640 1 --2- 172.21.15.2:0/1070124103 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f811c0a1a30 0x7f811c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:14.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.397+0000 7f81317fa640 1 -- 172.21.15.2:0/1070124103 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8124014070 con 0x7f8134102700 2024-03-15T12:56:14.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.397+0000 7f8133fff640 1 --2- 172.21.15.2:0/1070124103 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f811c0a1a30 0x7f811c0a3ef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8134102560 tx=0x7f8128005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:14.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.402+0000 7f81317fa640 1 -- 172.21.15.2:0/1070124103 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8124053280 con 0x7f8134102700 2024-03-15T12:56:14.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.632+0000 7f813a2ae640 1 -- 172.21.15.2:0/1070124103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f813410d0a0 con 0x7f8134102700 2024-03-15T12:56:14.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.633+0000 7f81317fa640 1 -- 172.21.15.2:0/1070124103 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9239 (secure 0 0 0) 0x7f81240969d0 con 0x7f8134102700 2024-03-15T12:56:14.619 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:14.619 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-15T12:56:14.620 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 12 2024-03-15T12:56:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.636+0000 7f813a2ae640 1 -- 172.21.15.2:0/1070124103 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f811c0a1a30 msgr2=0x7f811c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.636+0000 7f813a2ae640 1 --2- 172.21.15.2:0/1070124103 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f811c0a1a30 0x7f811c0a3ef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8134102560 tx=0x7f8128005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.636+0000 7f813a2ae640 1 -- 172.21.15.2:0/1070124103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8134102700 msgr2=0x7f8134197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.636+0000 7f813a2ae640 1 --2- 172.21.15.2:0/1070124103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8134102700 0x7f8134197ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f812400e9f0 tx=0x7f812400eeb0 comp rx=0 tx=0).stop 2024-03-15T12:56:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.636+0000 7f813a2ae640 1 -- 172.21.15.2:0/1070124103 shutdown_connections 2024-03-15T12:56:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.636+0000 7f813a2ae640 1 --2- 172.21.15.2:0/1070124103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81341030c0 0x7f813419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.636+0000 7f813a2ae640 1 --2- 172.21.15.2:0/1070124103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8134102700 0x7f8134197ab0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.636+0000 7f813a2ae640 1 --2- 172.21.15.2:0/1070124103 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f811c0a1a30 0x7f811c0a3ef0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.636+0000 7f813a2ae640 1 --2- 172.21.15.2:0/1070124103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8134101510 0x7f8134197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.636+0000 7f813a2ae640 1 -- 172.21.15.2:0/1070124103 >> 172.21.15.2:0/1070124103 conn(0x7f81340fcaa0 msgr2=0x7f8134105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.636+0000 7f813a2ae640 1 -- 172.21.15.2:0/1070124103 shutdown_connections 2024-03-15T12:56:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:14.636+0000 7f813a2ae640 1 -- 172.21.15.2:0/1070124103 wait complete. 2024-03-15T12:56:15.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:15.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:15.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:15.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:15.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:15.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:56:15.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:15 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1070124103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:15.474 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:15 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] up:boot 2024-03-15T12:56:15.475 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:15 smithi184 ceph-mon[35993]: fsmap 12 up:standby 2024-03-15T12:56:15.475 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:15 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-15T12:56:15.475 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:15 smithi184 ceph-mon[35993]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:15.475 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:15.475 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:15.475 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:15.475 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:15.475 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:15.475 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:56:15.476 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:15 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1070124103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:15.476 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:15 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] up:boot 2024-03-15T12:56:15.476 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:15 smithi002 ceph-mon[32060]: fsmap 12 up:standby 2024-03-15T12:56:15.476 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:15 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-15T12:56:15.476 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:15 smithi002 ceph-mon[32060]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:15.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:15.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:15.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:15.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:15.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:15.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:56:15.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:15 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1070124103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:15.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:15 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] up:boot 2024-03-15T12:56:15.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:15 smithi149 ceph-mon[35927]: fsmap 12 up:standby 2024-03-15T12:56:15.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:15 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-15T12:56:15.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:15 smithi149 ceph-mon[35927]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:16.639 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:56:16.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.763+0000 7f71a935c640 1 -- 172.21.15.2:0/875534738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71a4101fc0 msgr2=0x7f71a4108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:16.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.763+0000 7f71a935c640 1 --2- 172.21.15.2:0/875534738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71a4101fc0 0x7f71a4108850 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f719c009ca0 tx=0x7f719c02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:16.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.763+0000 7f71a935c640 1 -- 172.21.15.2:0/875534738 shutdown_connections 2024-03-15T12:56:16.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.763+0000 7f71a935c640 1 --2- 172.21.15.2:0/875534738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71a4101fc0 0x7f71a4108850 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:16.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.763+0000 7f71a935c640 1 --2- 172.21.15.2:0/875534738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71a4101600 0x7f71a4101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:16.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.763+0000 7f71a935c640 1 --2- 172.21.15.2:0/875534738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71a4100410 0x7f71a4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:16.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.763+0000 7f71a935c640 1 -- 172.21.15.2:0/875534738 >> 172.21.15.2:0/875534738 conn(0x7f71a40fb980 msgr2=0x7f71a40fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:16.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.763+0000 7f71a935c640 1 -- 172.21.15.2:0/875534738 shutdown_connections 2024-03-15T12:56:16.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.763+0000 7f71a935c640 1 -- 172.21.15.2:0/875534738 wait complete. 2024-03-15T12:56:16.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.764+0000 7f71a935c640 1 Processor -- start 2024-03-15T12:56:16.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.764+0000 7f71a935c640 1 -- start start 2024-03-15T12:56:16.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.764+0000 7f71a935c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71a4100410 0x7f71a41043a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:16.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.764+0000 7f71a935c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71a4101600 0x7f71a41029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:16.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.764+0000 7f71a935c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71a4101fc0 0x7f71a4102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.764+0000 7f71a935c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a4103680 con 0x7f71a4101fc0 2024-03-15T12:56:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.764+0000 7f71a935c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a41037f0 con 0x7f71a4100410 2024-03-15T12:56:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.764+0000 7f71a935c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a41a15c0 con 0x7f71a4101600 2024-03-15T12:56:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.764+0000 7f71a37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71a4101fc0 0x7f71a4102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.764+0000 7f71a37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71a4101fc0 0x7f71a4102f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36596/0 (socket says 172.21.15.2:36596) 2024-03-15T12:56:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.764+0000 7f71a37fe640 1 -- 172.21.15.2:0/570805604 learned_addr learned my addr 172.21.15.2:0/570805604 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.764+0000 7f71a27fc640 1 --2- 172.21.15.2:0/570805604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71a4101600 0x7f71a41029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.764+0000 7f71a2ffd640 1 --2- 172.21.15.2:0/570805604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71a4100410 0x7f71a41043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.765+0000 7f71a37fe640 1 -- 172.21.15.2:0/570805604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71a4101600 msgr2=0x7f71a41029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.765+0000 7f71a37fe640 1 --2- 172.21.15.2:0/570805604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71a4101600 0x7f71a41029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.765+0000 7f71a37fe640 1 -- 172.21.15.2:0/570805604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71a4100410 msgr2=0x7f71a41043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.765+0000 7f71a37fe640 1 --2- 172.21.15.2:0/570805604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71a4100410 0x7f71a41043a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.765+0000 7f71a37fe640 1 -- 172.21.15.2:0/570805604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c009900 con 0x7f71a4101fc0 2024-03-15T12:56:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.765+0000 7f71a37fe640 1 --2- 172.21.15.2:0/570805604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71a4101fc0 0x7f71a4102f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f719c005cc0 tx=0x7f719c032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.765+0000 7f718bfff640 1 -- 172.21.15.2:0/570805604 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c003970 con 0x7f71a4101fc0 2024-03-15T12:56:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.765+0000 7f718bfff640 1 -- 172.21.15.2:0/570805604 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f719c004040 con 0x7f71a4101fc0 2024-03-15T12:56:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.765+0000 7f71a935c640 1 -- 172.21.15.2:0/570805604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a41a1810 con 0x7f71a4101fc0 2024-03-15T12:56:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.765+0000 7f718bfff640 1 -- 172.21.15.2:0/570805604 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c0406b0 con 0x7f71a4101fc0 2024-03-15T12:56:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.765+0000 7f71a935c640 1 -- 172.21.15.2:0/570805604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a41a1cc0 con 0x7f71a4101fc0 2024-03-15T12:56:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.766+0000 7f71a935c640 1 -- 172.21.15.2:0/570805604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a410bdf0 con 0x7f71a4101fc0 2024-03-15T12:56:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.767+0000 7f718bfff640 1 -- 172.21.15.2:0/570805604 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f719c00a040 con 0x7f71a4101fc0 2024-03-15T12:56:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.768+0000 7f718bfff640 1 --2- 172.21.15.2:0/570805604 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f718c0a1a30 0x7f718c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.768+0000 7f71a2ffd640 1 --2- 172.21.15.2:0/570805604 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f718c0a1a30 0x7f718c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.768+0000 7f718bfff640 1 -- 172.21.15.2:0/570805604 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f719c0f0120 con 0x7f71a4101fc0 2024-03-15T12:56:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.768+0000 7f71a2ffd640 1 --2- 172.21.15.2:0/570805604 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f718c0a1a30 0x7f718c0a3ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f71a4101460 tx=0x7f7194005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:16.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:16.773+0000 7f718bfff640 1 -- 172.21.15.2:0/570805604 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f719c0b9730 con 0x7f71a4101fc0 2024-03-15T12:56:16.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.011+0000 7f71a935c640 1 -- 172.21.15.2:0/570805604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71a41a2160 con 0x7f71a4101fc0 2024-03-15T12:56:16.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.012+0000 7f718bfff640 1 -- 172.21.15.2:0/570805604 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10034 (secure 0 0 0) 0x7f719c0bcdf0 con 0x7f71a4101fc0 2024-03-15T12:56:16.998 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:16.998 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24539,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-15T12:56:16.999 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 13 2024-03-15T12:56:17.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.014+0000 7f71a935c640 1 -- 172.21.15.2:0/570805604 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f718c0a1a30 msgr2=0x7f718c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.014+0000 7f71a935c640 1 --2- 172.21.15.2:0/570805604 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f718c0a1a30 0x7f718c0a3ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f71a4101460 tx=0x7f7194005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:17.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.014+0000 7f71a935c640 1 -- 172.21.15.2:0/570805604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71a4101fc0 msgr2=0x7f71a4102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.014+0000 7f71a935c640 1 --2- 172.21.15.2:0/570805604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71a4101fc0 0x7f71a4102f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f719c005cc0 tx=0x7f719c032a10 comp rx=0 tx=0).stop 2024-03-15T12:56:17.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.014+0000 7f71a935c640 1 -- 172.21.15.2:0/570805604 shutdown_connections 2024-03-15T12:56:17.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.014+0000 7f71a935c640 1 --2- 172.21.15.2:0/570805604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71a4101fc0 0x7f71a4102f20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.014+0000 7f71a935c640 1 --2- 172.21.15.2:0/570805604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71a4101600 0x7f71a41029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.014+0000 7f71a935c640 1 --2- 172.21.15.2:0/570805604 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f718c0a1a30 0x7f718c0a3ef0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.014+0000 7f71a935c640 1 --2- 172.21.15.2:0/570805604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71a4100410 0x7f71a41043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.014+0000 7f71a935c640 1 -- 172.21.15.2:0/570805604 >> 172.21.15.2:0/570805604 conn(0x7f71a40fb980 msgr2=0x7f71a40fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:17.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.014+0000 7f71a935c640 1 -- 172.21.15.2:0/570805604 shutdown_connections 2024-03-15T12:56:17.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.014+0000 7f71a935c640 1 -- 172.21.15.2:0/570805604 wait complete. 2024-03-15T12:56:17.012 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:56:17.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.130+0000 7f23edf19640 1 -- 172.21.15.2:0/3233200030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e8100410 msgr2=0x7f23e8100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.130+0000 7f23edf19640 1 --2- 172.21.15.2:0/3233200030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e8100410 0x7f23e8100830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009900 tx=0x7f23dc02bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:17.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.130+0000 7f23edf19640 1 -- 172.21.15.2:0/3233200030 shutdown_connections 2024-03-15T12:56:17.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.130+0000 7f23edf19640 1 --2- 172.21.15.2:0/3233200030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e8101fc0 0x7f23e8108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.130+0000 7f23edf19640 1 --2- 172.21.15.2:0/3233200030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e8101600 0x7f23e8101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.130+0000 7f23edf19640 1 --2- 172.21.15.2:0/3233200030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e8100410 0x7f23e8100830 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.130+0000 7f23edf19640 1 -- 172.21.15.2:0/3233200030 >> 172.21.15.2:0/3233200030 conn(0x7f23e80fb980 msgr2=0x7f23e80fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:17.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.130+0000 7f23edf19640 1 -- 172.21.15.2:0/3233200030 shutdown_connections 2024-03-15T12:56:17.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.130+0000 7f23edf19640 1 -- 172.21.15.2:0/3233200030 wait complete. 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.131+0000 7f23edf19640 1 Processor -- start 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.131+0000 7f23edf19640 1 -- start start 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.131+0000 7f23edf19640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e8100410 0x7f23e80691c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.131+0000 7f23edf19640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e8101600 0x7f23e819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.131+0000 7f23edf19640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e8101fc0 0x7f23e81a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.131+0000 7f23edf19640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e81a1320 con 0x7f23e8101600 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.131+0000 7f23edf19640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e81a1490 con 0x7f23e8100410 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.131+0000 7f23edf19640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e81a1650 con 0x7f23e8101fc0 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.131+0000 7f23e6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e8101600 0x7f23e819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.131+0000 7f23e6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e8101600 0x7f23e819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36610/0 (socket says 172.21.15.2:36610) 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.131+0000 7f23e6ffd640 1 -- 172.21.15.2:0/2265726825 learned_addr learned my addr 172.21.15.2:0/2265726825 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.131+0000 7f23e77fe640 1 --2- 172.21.15.2:0/2265726825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e8100410 0x7f23e80691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.131+0000 7f23e7fff640 1 --2- 172.21.15.2:0/2265726825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e8101fc0 0x7f23e81a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.131+0000 7f23e6ffd640 1 -- 172.21.15.2:0/2265726825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e8101fc0 msgr2=0x7f23e81a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.132+0000 7f23e6ffd640 1 --2- 172.21.15.2:0/2265726825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e8101fc0 0x7f23e81a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.132+0000 7f23e6ffd640 1 -- 172.21.15.2:0/2265726825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e8100410 msgr2=0x7f23e80691c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.132+0000 7f23e6ffd640 1 --2- 172.21.15.2:0/2265726825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e8100410 0x7f23e80691c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.132+0000 7f23e6ffd640 1 -- 172.21.15.2:0/2265726825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc009560 con 0x7f23e8101600 2024-03-15T12:56:17.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.132+0000 7f23e6ffd640 1 --2- 172.21.15.2:0/2265726825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e8101600 0x7f23e819e710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f23d800d950 tx=0x7f23d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:17.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.132+0000 7f23e4ff9640 1 -- 172.21.15.2:0/2265726825 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8009a30 con 0x7f23e8101600 2024-03-15T12:56:17.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.132+0000 7f23edf19640 1 -- 172.21.15.2:0/2265726825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e81a18d0 con 0x7f23e8101600 2024-03-15T12:56:17.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.132+0000 7f23e4ff9640 1 -- 172.21.15.2:0/2265726825 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f23d8009b90 con 0x7f23e8101600 2024-03-15T12:56:17.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.132+0000 7f23e4ff9640 1 -- 172.21.15.2:0/2265726825 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d80047a0 con 0x7f23e8101600 2024-03-15T12:56:17.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.132+0000 7f23edf19640 1 -- 172.21.15.2:0/2265726825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e81a1e30 con 0x7f23e8101600 2024-03-15T12:56:17.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.133+0000 7f23edf19640 1 -- 172.21.15.2:0/2265726825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e81077b0 con 0x7f23e8101600 2024-03-15T12:56:17.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.134+0000 7f23e4ff9640 1 -- 172.21.15.2:0/2265726825 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f23d8009d00 con 0x7f23e8101600 2024-03-15T12:56:17.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.135+0000 7f23e4ff9640 1 --2- 172.21.15.2:0/2265726825 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f23d00a1a60 0x7f23d00a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:17.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.135+0000 7f23e77fe640 1 --2- 172.21.15.2:0/2265726825 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f23d00a1a60 0x7f23d00a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:17.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.135+0000 7f23e4ff9640 1 -- 172.21.15.2:0/2265726825 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f23d80cc9b0 con 0x7f23e8101600 2024-03-15T12:56:17.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.135+0000 7f23e77fe640 1 --2- 172.21.15.2:0/2265726825 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f23d00a1a60 0x7f23d00a3f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f23dc02c740 tx=0x7f23dc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:17.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.143+0000 7f23e4ff9640 1 -- 172.21.15.2:0/2265726825 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f23d8095fc0 con 0x7f23e8101600 2024-03-15T12:56:17.295 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.296 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.296 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.296 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.296 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.296 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.296 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:17.296 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:56:17.296 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.296 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:17 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:56:17.296 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:17 smithi002 ceph-mon[32060]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:17.296 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:17 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/570805604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:17.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:17.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:56:17.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:17 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:56:17.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:17 smithi184 ceph-mon[35993]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:17.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:17 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/570805604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:17.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.329+0000 7f23edf19640 1 -- 172.21.15.2:0/2265726825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23e802cf60 con 0x7f23e8101600 2024-03-15T12:56:17.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.330+0000 7f23e4ff9640 1 -- 172.21.15.2:0/2265726825 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18373 (secure 0 0 0) 0x7f23d8099680 con 0x7f23e8101600 2024-03-15T12:56:17.316 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:17.316 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":81,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:55:31.131625+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:56:17.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.332+0000 7f23edf19640 1 -- 172.21.15.2:0/2265726825 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f23d00a1a60 msgr2=0x7f23d00a3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.332+0000 7f23edf19640 1 --2- 172.21.15.2:0/2265726825 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f23d00a1a60 0x7f23d00a3f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f23dc02c740 tx=0x7f23dc005fd0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.332+0000 7f23edf19640 1 -- 172.21.15.2:0/2265726825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e8101600 msgr2=0x7f23e819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.332+0000 7f23edf19640 1 --2- 172.21.15.2:0/2265726825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e8101600 0x7f23e819e710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f23d800d950 tx=0x7f23d800de10 comp rx=0 tx=0).stop 2024-03-15T12:56:17.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.332+0000 7f23edf19640 1 -- 172.21.15.2:0/2265726825 shutdown_connections 2024-03-15T12:56:17.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.332+0000 7f23edf19640 1 --2- 172.21.15.2:0/2265726825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e8101fc0 0x7f23e81a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.332+0000 7f23edf19640 1 --2- 172.21.15.2:0/2265726825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e8101600 0x7f23e819e710 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.332+0000 7f23edf19640 1 --2- 172.21.15.2:0/2265726825 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f23d00a1a60 0x7f23d00a3f20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.332+0000 7f23edf19640 1 --2- 172.21.15.2:0/2265726825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e8100410 0x7f23e80691c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.332+0000 7f23edf19640 1 -- 172.21.15.2:0/2265726825 >> 172.21.15.2:0/2265726825 conn(0x7f23e80fb980 msgr2=0x7f23e8104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:17.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.332+0000 7f23edf19640 1 -- 172.21.15.2:0/2265726825 shutdown_connections 2024-03-15T12:56:17.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.332+0000 7f23edf19640 1 -- 172.21.15.2:0/2265726825 wait complete. 2024-03-15T12:56:17.329 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-15T12:51:47.004843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-15T12:56:17.329 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-15T12:56:17.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:17.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:56:17.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:17.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:17 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:56:17.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:17 smithi149 ceph-mon[35927]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:17.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:17 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/570805604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:17.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.449+0000 7fb085d83640 1 -- 172.21.15.2:0/1199440105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb080107250 msgr2=0x7fb080109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.449+0000 7fb085d83640 1 --2- 172.21.15.2:0/1199440105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb080107250 0x7fb080109640 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb078009ca0 tx=0x7fb07802c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.449+0000 7fb085d83640 1 -- 172.21.15.2:0/1199440105 shutdown_connections 2024-03-15T12:56:17.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.449+0000 7fb085d83640 1 --2- 172.21.15.2:0/1199440105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb080107250 0x7fb080109640 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.449+0000 7fb085d83640 1 --2- 172.21.15.2:0/1199440105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb080104920 0x7fb080106d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.449+0000 7fb085d83640 1 --2- 172.21.15.2:0/1199440105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb080101ff0 0x7fb0801043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.449+0000 7fb085d83640 1 -- 172.21.15.2:0/1199440105 >> 172.21.15.2:0/1199440105 conn(0x7fb0800fb9a0 msgr2=0x7fb0800fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:17.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.449+0000 7fb085d83640 1 -- 172.21.15.2:0/1199440105 shutdown_connections 2024-03-15T12:56:17.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.449+0000 7fb085d83640 1 -- 172.21.15.2:0/1199440105 wait complete. 2024-03-15T12:56:17.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb085d83640 1 Processor -- start 2024-03-15T12:56:17.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb085d83640 1 -- start start 2024-03-15T12:56:17.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb085d83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb080101ff0 0x7fb080197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:17.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb085d83640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb080104920 0x7fb080197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:17.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb085d83640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb080107250 0x7fb08019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:17.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb085d83640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08019f070 con 0x7fb080104920 2024-03-15T12:56:17.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb085d83640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08019f1e0 con 0x7fb080107250 2024-03-15T12:56:17.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb085d83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08019f3a0 con 0x7fb080101ff0 2024-03-15T12:56:17.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb07effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb080104920 0x7fb080197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:17.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb07f7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb080101ff0 0x7fb080197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:17.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb07effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb080104920 0x7fb080197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36618/0 (socket says 172.21.15.2:36618) 2024-03-15T12:56:17.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb07effd640 1 -- 172.21.15.2:0/3132185551 learned_addr learned my addr 172.21.15.2:0/3132185551 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:17.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb07ffff640 1 --2- 172.21.15.2:0/3132185551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb080107250 0x7fb08019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:17.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb07effd640 1 -- 172.21.15.2:0/3132185551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb080101ff0 msgr2=0x7fb080197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb07effd640 1 --2- 172.21.15.2:0/3132185551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb080101ff0 0x7fb080197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb07effd640 1 -- 172.21.15.2:0/3132185551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb080107250 msgr2=0x7fb08019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb07effd640 1 --2- 172.21.15.2:0/3132185551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb080107250 0x7fb08019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.450+0000 7fb07effd640 1 -- 172.21.15.2:0/3132185551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb078009900 con 0x7fb080104920 2024-03-15T12:56:17.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.451+0000 7fb07effd640 1 --2- 172.21.15.2:0/3132185551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb080104920 0x7fb080197ac0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb07400c970 tx=0x7fb07400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:17.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.451+0000 7fb07cff9640 1 -- 172.21.15.2:0/3132185551 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074007bb0 con 0x7fb080104920 2024-03-15T12:56:17.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.451+0000 7fb07cff9640 1 -- 172.21.15.2:0/3132185551 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb074007d10 con 0x7fb080104920 2024-03-15T12:56:17.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.451+0000 7fb085d83640 1 -- 172.21.15.2:0/3132185551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08019f680 con 0x7fb080104920 2024-03-15T12:56:17.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.451+0000 7fb07cff9640 1 -- 172.21.15.2:0/3132185551 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07400f450 con 0x7fb080104920 2024-03-15T12:56:17.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.451+0000 7fb085d83640 1 -- 172.21.15.2:0/3132185551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08019fbe0 con 0x7fb080104920 2024-03-15T12:56:17.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.452+0000 7fb085d83640 1 -- 172.21.15.2:0/3132185551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0800fd490 con 0x7fb080104920 2024-03-15T12:56:17.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.453+0000 7fb07cff9640 1 -- 172.21.15.2:0/3132185551 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fb0740046c0 con 0x7fb080104920 2024-03-15T12:56:17.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.453+0000 7fb07cff9640 1 --2- 172.21.15.2:0/3132185551 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb0680a1ab0 0x7fb0680a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:17.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.454+0000 7fb07f7fe640 1 --2- 172.21.15.2:0/3132185551 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb0680a1ab0 0x7fb0680a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:17.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.454+0000 7fb07cff9640 1 -- 172.21.15.2:0/3132185551 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb0740ccec0 con 0x7fb080104920 2024-03-15T12:56:17.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.454+0000 7fb07f7fe640 1 --2- 172.21.15.2:0/3132185551 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb0680a1ab0 0x7fb0680a3f70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb070005fd0 tx=0x7fb07000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:17.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.458+0000 7fb07cff9640 1 -- 172.21.15.2:0/3132185551 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb074096620 con 0x7fb080104920 2024-03-15T12:56:17.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.645+0000 7fb085d83640 1 -- 172.21.15.2:0/3132185551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb08010d0c0 con 0x7fb080104920 2024-03-15T12:56:17.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.646+0000 7fb07cff9640 1 -- 172.21.15.2:0/3132185551 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb074099ce0 con 0x7fb080104920 2024-03-15T12:56:17.632 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-15T12:56:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.648+0000 7fb085d83640 1 -- 172.21.15.2:0/3132185551 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb0680a1ab0 msgr2=0x7fb0680a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.648+0000 7fb085d83640 1 --2- 172.21.15.2:0/3132185551 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb0680a1ab0 0x7fb0680a3f70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb070005fd0 tx=0x7fb07000b390 comp rx=0 tx=0).stop 2024-03-15T12:56:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.648+0000 7fb085d83640 1 -- 172.21.15.2:0/3132185551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb080104920 msgr2=0x7fb080197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.648+0000 7fb085d83640 1 --2- 172.21.15.2:0/3132185551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb080104920 0x7fb080197ac0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb07400c970 tx=0x7fb07400ce30 comp rx=0 tx=0).stop 2024-03-15T12:56:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.648+0000 7fb085d83640 1 -- 172.21.15.2:0/3132185551 shutdown_connections 2024-03-15T12:56:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.648+0000 7fb085d83640 1 --2- 172.21.15.2:0/3132185551 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb0680a1ab0 0x7fb0680a3f70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.648+0000 7fb085d83640 1 --2- 172.21.15.2:0/3132185551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb080107250 0x7fb08019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.648+0000 7fb085d83640 1 --2- 172.21.15.2:0/3132185551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb080104920 0x7fb080197ac0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.648+0000 7fb085d83640 1 --2- 172.21.15.2:0/3132185551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb080101ff0 0x7fb080197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.648+0000 7fb085d83640 1 -- 172.21.15.2:0/3132185551 >> 172.21.15.2:0/3132185551 conn(0x7fb0800fb9a0 msgr2=0x7fb080102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:17.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.648+0000 7fb085d83640 1 -- 172.21.15.2:0/3132185551 shutdown_connections 2024-03-15T12:56:17.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.648+0000 7fb085d83640 1 -- 172.21.15.2:0/3132185551 wait complete. 2024-03-15T12:56:17.646 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-15T12:56:17.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.777+0000 7ff97cbe2640 1 -- 172.21.15.2:0/3935806701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9781071f0 msgr2=0x7ff978109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.777+0000 7ff97cbe2640 1 --2- 172.21.15.2:0/3935806701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9781071f0 0x7ff978109640 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff970009ca0 tx=0x7ff97002c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.777+0000 7ff97cbe2640 1 -- 172.21.15.2:0/3935806701 shutdown_connections 2024-03-15T12:56:17.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.777+0000 7ff97cbe2640 1 --2- 172.21.15.2:0/3935806701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9781071f0 0x7ff978109640 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.777+0000 7ff97cbe2640 1 --2- 172.21.15.2:0/3935806701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9781048c0 0x7ff978106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.777+0000 7ff97cbe2640 1 --2- 172.21.15.2:0/3935806701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff978101f90 0x7ff978104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.777+0000 7ff97cbe2640 1 -- 172.21.15.2:0/3935806701 >> 172.21.15.2:0/3935806701 conn(0x7ff9780fb980 msgr2=0x7ff9780fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:17.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.777+0000 7ff97cbe2640 1 -- 172.21.15.2:0/3935806701 shutdown_connections 2024-03-15T12:56:17.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.777+0000 7ff97cbe2640 1 -- 172.21.15.2:0/3935806701 wait complete. 2024-03-15T12:56:17.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.778+0000 7ff97cbe2640 1 Processor -- start 2024-03-15T12:56:17.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.778+0000 7ff97cbe2640 1 -- start start 2024-03-15T12:56:17.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.778+0000 7ff97cbe2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff978101f90 0x7ff978197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:17.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.778+0000 7ff97cbe2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9781048c0 0x7ff978197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:17.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.778+0000 7ff97cbe2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9781071f0 0x7ff97819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:17.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.778+0000 7ff97cbe2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97819f040 con 0x7ff978101f90 2024-03-15T12:56:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.778+0000 7ff97cbe2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97819f1b0 con 0x7ff9781071f0 2024-03-15T12:56:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.778+0000 7ff97cbe2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97819f370 con 0x7ff9781048c0 2024-03-15T12:56:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.779+0000 7ff97659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff978101f90 0x7ff978197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.779+0000 7ff97659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff978101f90 0x7ff978197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36638/0 (socket says 172.21.15.2:36638) 2024-03-15T12:56:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.779+0000 7ff97659c640 1 -- 172.21.15.2:0/3622518588 learned_addr learned my addr 172.21.15.2:0/3622518588 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.779+0000 7ff976d9d640 1 --2- 172.21.15.2:0/3622518588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9781071f0 0x7ff97819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.779+0000 7ff975d9b640 1 --2- 172.21.15.2:0/3622518588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9781048c0 0x7ff978197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.779+0000 7ff97659c640 1 -- 172.21.15.2:0/3622518588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9781048c0 msgr2=0x7ff978197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.779+0000 7ff97659c640 1 --2- 172.21.15.2:0/3622518588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9781048c0 0x7ff978197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.779+0000 7ff97659c640 1 -- 172.21.15.2:0/3622518588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9781071f0 msgr2=0x7ff97819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.779+0000 7ff97659c640 1 --2- 172.21.15.2:0/3622518588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9781071f0 0x7ff97819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.779+0000 7ff97659c640 1 -- 172.21.15.2:0/3622518588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff970009900 con 0x7ff978101f90 2024-03-15T12:56:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.779+0000 7ff976d9d640 1 --2- 172.21.15.2:0/3622518588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9781071f0 0x7ff97819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.779+0000 7ff97659c640 1 --2- 172.21.15.2:0/3622518588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff978101f90 0x7ff978197550 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff96800e9f0 tx=0x7ff96800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:17.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.779+0000 7ff9677fe640 1 -- 172.21.15.2:0/3622518588 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968009700 con 0x7ff978101f90 2024-03-15T12:56:17.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.780+0000 7ff97cbe2640 1 -- 172.21.15.2:0/3622518588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9780ffd80 con 0x7ff978101f90 2024-03-15T12:56:17.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.780+0000 7ff9677fe640 1 -- 172.21.15.2:0/3622518588 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff96800cd80 con 0x7ff978101f90 2024-03-15T12:56:17.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.780+0000 7ff9677fe640 1 -- 172.21.15.2:0/3622518588 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968018920 con 0x7ff978101f90 2024-03-15T12:56:17.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.780+0000 7ff97cbe2640 1 -- 172.21.15.2:0/3622518588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9781002e0 con 0x7ff978101f90 2024-03-15T12:56:17.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.781+0000 7ff97cbe2640 1 -- 172.21.15.2:0/3622518588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9780fd470 con 0x7ff978101f90 2024-03-15T12:56:17.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.782+0000 7ff9677fe640 1 -- 172.21.15.2:0/3622518588 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7ff968018a80 con 0x7ff978101f90 2024-03-15T12:56:17.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.782+0000 7ff9677fe640 1 --2- 172.21.15.2:0/3622518588 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff9600a1ab0 0x7ff9600a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:17.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.782+0000 7ff975d9b640 1 --2- 172.21.15.2:0/3622518588 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff9600a1ab0 0x7ff9600a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:17.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.782+0000 7ff9677fe640 1 -- 172.21.15.2:0/3622518588 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff968014070 con 0x7ff978101f90 2024-03-15T12:56:17.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.783+0000 7ff975d9b640 1 --2- 172.21.15.2:0/3622518588 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff9600a1ab0 0x7ff9600a3f70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff978198a60 tx=0x7ff96c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:17.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.789+0000 7ff9677fe640 1 -- 172.21.15.2:0/3622518588 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff968053280 con 0x7ff978101f90 2024-03-15T12:56:17.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.991+0000 7ff97cbe2640 1 -- 172.21.15.2:0/3622518588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff97802d000 con 0x7ff978101f90 2024-03-15T12:56:17.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.991+0000 7ff9677fe640 1 -- 172.21.15.2:0/3622518588 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff968096970 con 0x7ff978101f90 2024-03-15T12:56:17.978 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:17.978 INFO:teuthology.orchestra.run.smithi002.stdout:[ 2024-03-15T12:56:17.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T12:56:17.978 INFO:teuthology.orchestra.run.smithi002.stdout: "poolnum": 1, 2024-03-15T12:56:17.978 INFO:teuthology.orchestra.run.smithi002.stdout: "poolname": ".mgr" 2024-03-15T12:56:17.978 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T12:56:17.978 INFO:teuthology.orchestra.run.smithi002.stdout:] 2024-03-15T12:56:17.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.993+0000 7ff97cbe2640 1 -- 172.21.15.2:0/3622518588 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff9600a1ab0 msgr2=0x7ff9600a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.993+0000 7ff97cbe2640 1 --2- 172.21.15.2:0/3622518588 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff9600a1ab0 0x7ff9600a3f70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff978198a60 tx=0x7ff96c005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:17.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.993+0000 7ff97cbe2640 1 -- 172.21.15.2:0/3622518588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff978101f90 msgr2=0x7ff978197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:17.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.993+0000 7ff97cbe2640 1 --2- 172.21.15.2:0/3622518588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff978101f90 0x7ff978197550 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff96800e9f0 tx=0x7ff96800eeb0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.993+0000 7ff97cbe2640 1 -- 172.21.15.2:0/3622518588 shutdown_connections 2024-03-15T12:56:17.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.993+0000 7ff97cbe2640 1 --2- 172.21.15.2:0/3622518588 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff9600a1ab0 0x7ff9600a3f70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.993+0000 7ff97cbe2640 1 --2- 172.21.15.2:0/3622518588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9781071f0 0x7ff97819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.993+0000 7ff97cbe2640 1 --2- 172.21.15.2:0/3622518588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9781048c0 0x7ff978197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.993+0000 7ff97cbe2640 1 --2- 172.21.15.2:0/3622518588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff978101f90 0x7ff978197550 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:17.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.993+0000 7ff97cbe2640 1 -- 172.21.15.2:0/3622518588 >> 172.21.15.2:0/3622518588 conn(0x7ff9780fb980 msgr2=0x7ff9781028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:17.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.993+0000 7ff97cbe2640 1 -- 172.21.15.2:0/3622518588 shutdown_connections 2024-03-15T12:56:17.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:17.993+0000 7ff97cbe2640 1 -- 172.21.15.2:0/3622518588 wait complete. 2024-03-15T12:56:17.991 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-15T12:56:17.991 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-15T12:56:18.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.112+0000 7fae73a88640 1 -- 172.21.15.2:0/727274789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae6c1030c0 msgr2=0x7fae6c109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:18.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.112+0000 7fae73a88640 1 --2- 172.21.15.2:0/727274789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae6c1030c0 0x7fae6c109950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009ca0 tx=0x7fae5c02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:18.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.112+0000 7fae73a88640 1 -- 172.21.15.2:0/727274789 shutdown_connections 2024-03-15T12:56:18.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.112+0000 7fae73a88640 1 --2- 172.21.15.2:0/727274789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae6c1030c0 0x7fae6c109950 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:18.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.112+0000 7fae73a88640 1 --2- 172.21.15.2:0/727274789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae6c102700 0x7fae6c102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:18.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.112+0000 7fae73a88640 1 --2- 172.21.15.2:0/727274789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae6c101510 0x7fae6c101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:18.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.112+0000 7fae73a88640 1 -- 172.21.15.2:0/727274789 >> 172.21.15.2:0/727274789 conn(0x7fae6c0fca80 msgr2=0x7fae6c0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:18.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.112+0000 7fae73a88640 1 -- 172.21.15.2:0/727274789 shutdown_connections 2024-03-15T12:56:18.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.112+0000 7fae73a88640 1 -- 172.21.15.2:0/727274789 wait complete. 2024-03-15T12:56:18.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.113+0000 7fae73a88640 1 Processor -- start 2024-03-15T12:56:18.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.113+0000 7fae73a88640 1 -- start start 2024-03-15T12:56:18.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.113+0000 7fae73a88640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae6c101510 0x7fae6c1974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:18.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.113+0000 7fae73a88640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae6c102700 0x7fae6c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:18.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.113+0000 7fae73a88640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae6c1030c0 0x7fae6c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:18.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.113+0000 7fae73a88640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c19ef90 con 0x7fae6c1030c0 2024-03-15T12:56:18.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.113+0000 7fae73a88640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c19f100 con 0x7fae6c101510 2024-03-15T12:56:18.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.113+0000 7fae73a88640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c19f2c0 con 0x7fae6c102700 2024-03-15T12:56:18.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.113+0000 7fae72025640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae6c1030c0 0x7fae6c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:18.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.113+0000 7fae72025640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae6c1030c0 0x7fae6c19ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52070/0 (socket says 172.21.15.2:52070) 2024-03-15T12:56:18.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.113+0000 7fae72025640 1 -- 172.21.15.2:0/205628640 learned_addr learned my addr 172.21.15.2:0/205628640 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:18.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.113+0000 7fae71824640 1 --2- 172.21.15.2:0/205628640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae6c101510 0x7fae6c1974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:18.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.113+0000 7fae71023640 1 --2- 172.21.15.2:0/205628640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae6c102700 0x7fae6c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:18.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.114+0000 7fae72025640 1 -- 172.21.15.2:0/205628640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae6c102700 msgr2=0x7fae6c1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:18.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.114+0000 7fae72025640 1 --2- 172.21.15.2:0/205628640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae6c102700 0x7fae6c1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:18.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.114+0000 7fae72025640 1 -- 172.21.15.2:0/205628640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae6c101510 msgr2=0x7fae6c1974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:18.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.114+0000 7fae72025640 1 --2- 172.21.15.2:0/205628640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae6c101510 0x7fae6c1974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:18.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.114+0000 7fae72025640 1 -- 172.21.15.2:0/205628640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c009900 con 0x7fae6c1030c0 2024-03-15T12:56:18.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.114+0000 7fae72025640 1 --2- 172.21.15.2:0/205628640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae6c1030c0 0x7fae6c19ea50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fae5c005e30 tx=0x7fae5c032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:18.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.114+0000 7fae6affd640 1 -- 172.21.15.2:0/205628640 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c003970 con 0x7fae6c1030c0 2024-03-15T12:56:18.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.114+0000 7fae6affd640 1 -- 172.21.15.2:0/205628640 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fae5c004040 con 0x7fae6c1030c0 2024-03-15T12:56:18.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.114+0000 7fae73a88640 1 -- 172.21.15.2:0/205628640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6c19f540 con 0x7fae6c1030c0 2024-03-15T12:56:18.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.114+0000 7fae6affd640 1 -- 172.21.15.2:0/205628640 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c0406b0 con 0x7fae6c1030c0 2024-03-15T12:56:18.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.114+0000 7fae73a88640 1 -- 172.21.15.2:0/205628640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6c19f9f0 con 0x7fae6c1030c0 2024-03-15T12:56:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.115+0000 7fae73a88640 1 -- 172.21.15.2:0/205628640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6c1087d0 con 0x7fae6c1030c0 2024-03-15T12:56:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.116+0000 7fae6affd640 1 -- 172.21.15.2:0/205628640 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fae5c00a040 con 0x7fae6c1030c0 2024-03-15T12:56:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.117+0000 7fae6affd640 1 --2- 172.21.15.2:0/205628640 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fae540a1a30 0x7fae540a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.117+0000 7fae71824640 1 --2- 172.21.15.2:0/205628640 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fae540a1a30 0x7fae540a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.117+0000 7fae6affd640 1 -- 172.21.15.2:0/205628640 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fae5c0f0040 con 0x7fae6c1030c0 2024-03-15T12:56:18.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.117+0000 7fae71824640 1 --2- 172.21.15.2:0/205628640 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fae540a1a30 0x7fae540a3ef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae6c102560 tx=0x7fae64005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:18.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.121+0000 7fae6affd640 1 -- 172.21.15.2:0/205628640 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fae5c0b96f0 con 0x7fae6c1030c0 2024-03-15T12:56:18.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:18.310+0000 7fae73a88640 1 -- 172.21.15.2:0/205628640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fae6c02d000 con 0x7fae6c1030c0 2024-03-15T12:56:18.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:18 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2265726825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:18.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:18 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3132185551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:56:18.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:18 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3622518588' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-15T12:56:18.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:18.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:56:18.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:18 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2265726825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:18.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:18 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3132185551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:56:18.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:18 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3622518588' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-15T12:56:18.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:18.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:56:18.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:18 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2265726825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:18.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:18 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3132185551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:56:18.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:18 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3622518588' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-15T12:56:18.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:18.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:56:18.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:56:18.611 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:19.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.183+0000 7fae6affd640 1 -- 172.21.15.2:0/205628640 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7fae5c0bcdb0 con 0x7fae6c1030c0 2024-03-15T12:56:19.169 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' created 2024-03-15T12:56:19.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.185+0000 7fae73a88640 1 -- 172.21.15.2:0/205628640 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fae540a1a30 msgr2=0x7fae540a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:19.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.185+0000 7fae73a88640 1 --2- 172.21.15.2:0/205628640 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fae540a1a30 0x7fae540a3ef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae6c102560 tx=0x7fae64005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:19.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.185+0000 7fae73a88640 1 -- 172.21.15.2:0/205628640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae6c1030c0 msgr2=0x7fae6c19ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:19.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.185+0000 7fae73a88640 1 --2- 172.21.15.2:0/205628640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae6c1030c0 0x7fae6c19ea50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fae5c005e30 tx=0x7fae5c032a10 comp rx=0 tx=0).stop 2024-03-15T12:56:19.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.185+0000 7fae73a88640 1 -- 172.21.15.2:0/205628640 shutdown_connections 2024-03-15T12:56:19.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.185+0000 7fae73a88640 1 --2- 172.21.15.2:0/205628640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae6c1030c0 0x7fae6c19ea50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:19.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.185+0000 7fae73a88640 1 --2- 172.21.15.2:0/205628640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae6c102700 0x7fae6c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:19.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.185+0000 7fae73a88640 1 --2- 172.21.15.2:0/205628640 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fae540a1a30 0x7fae540a3ef0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:19.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.185+0000 7fae73a88640 1 --2- 172.21.15.2:0/205628640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae6c101510 0x7fae6c1974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:19.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.185+0000 7fae73a88640 1 -- 172.21.15.2:0/205628640 >> 172.21.15.2:0/205628640 conn(0x7fae6c0fca80 msgr2=0x7fae6c1059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:19.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.186+0000 7fae73a88640 1 -- 172.21.15.2:0/205628640 shutdown_connections 2024-03-15T12:56:19.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.186+0000 7fae73a88640 1 -- 172.21.15.2:0/205628640 wait complete. 2024-03-15T12:56:19.187 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-15T12:56:19.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.313+0000 7fcb850c3640 1 -- 172.21.15.2:0/1859897344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb80102780 msgr2=0x7fcb80102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:19.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.313+0000 7fcb850c3640 1 --2- 172.21.15.2:0/1859897344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb80102780 0x7fcb80102c00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009900 tx=0x7fcb6802bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:19.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.313+0000 7fcb850c3640 1 -- 172.21.15.2:0/1859897344 shutdown_connections 2024-03-15T12:56:19.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.313+0000 7fcb850c3640 1 --2- 172.21.15.2:0/1859897344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb80103140 0x7fcb801099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:19.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.313+0000 7fcb850c3640 1 --2- 172.21.15.2:0/1859897344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb80102780 0x7fcb80102c00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:19.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.313+0000 7fcb850c3640 1 --2- 172.21.15.2:0/1859897344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb800ff7b0 0x7fcb800ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:19.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.313+0000 7fcb850c3640 1 -- 172.21.15.2:0/1859897344 >> 172.21.15.2:0/1859897344 conn(0x7fcb800fb100 msgr2=0x7fcb800fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:19.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.313+0000 7fcb850c3640 1 -- 172.21.15.2:0/1859897344 shutdown_connections 2024-03-15T12:56:19.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.313+0000 7fcb850c3640 1 -- 172.21.15.2:0/1859897344 wait complete. 2024-03-15T12:56:19.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb850c3640 1 Processor -- start 2024-03-15T12:56:19.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb850c3640 1 -- start start 2024-03-15T12:56:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb850c3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb800ff7b0 0x7fcb80197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb850c3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb80102780 0x7fcb80197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb850c3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb80103140 0x7fcb8019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb850c3640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8019f040 con 0x7fcb80102780 2024-03-15T12:56:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb850c3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8019f1b0 con 0x7fcb80103140 2024-03-15T12:56:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb850c3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8019f370 con 0x7fcb800ff7b0 2024-03-15T12:56:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb77fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb80102780 0x7fcb80197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb77fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb80102780 0x7fcb80197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52090/0 (socket says 172.21.15.2:52090) 2024-03-15T12:56:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb77fff640 1 -- 172.21.15.2:0/755046461 learned_addr learned my addr 172.21.15.2:0/755046461 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb77fff640 1 -- 172.21.15.2:0/755046461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb800ff7b0 msgr2=0x7fcb80197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb7ed9d640 1 --2- 172.21.15.2:0/755046461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb800ff7b0 0x7fcb80197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb77fff640 1 --2- 172.21.15.2:0/755046461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb800ff7b0 0x7fcb80197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb77fff640 1 -- 172.21.15.2:0/755046461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb80103140 msgr2=0x7fcb8019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb7f59e640 1 --2- 172.21.15.2:0/755046461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb80103140 0x7fcb8019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:19.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb77fff640 1 --2- 172.21.15.2:0/755046461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb80103140 0x7fcb8019eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:19.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.314+0000 7fcb77fff640 1 -- 172.21.15.2:0/755046461 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb68009560 con 0x7fcb80102780 2024-03-15T12:56:19.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.315+0000 7fcb77fff640 1 --2- 172.21.15.2:0/755046461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb80102780 0x7fcb80197a90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb68000c00 tx=0x7fcb68004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:19.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.315+0000 7fcb7cd99640 1 -- 172.21.15.2:0/755046461 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68004440 con 0x7fcb80102780 2024-03-15T12:56:19.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.315+0000 7fcb850c3640 1 -- 172.21.15.2:0/755046461 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8019f5f0 con 0x7fcb80102780 2024-03-15T12:56:19.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.315+0000 7fcb7cd99640 1 -- 172.21.15.2:0/755046461 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcb68005480 con 0x7fcb80102780 2024-03-15T12:56:19.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.315+0000 7fcb7cd99640 1 -- 172.21.15.2:0/755046461 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6803e450 con 0x7fcb80102780 2024-03-15T12:56:19.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.315+0000 7fcb850c3640 1 -- 172.21.15.2:0/755046461 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8019fb50 con 0x7fcb80102780 2024-03-15T12:56:19.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.316+0000 7fcb850c3640 1 -- 172.21.15.2:0/755046461 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb801088b0 con 0x7fcb80102780 2024-03-15T12:56:19.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.318+0000 7fcb7cd99640 1 -- 172.21.15.2:0/755046461 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fcb68003680 con 0x7fcb80102780 2024-03-15T12:56:19.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.318+0000 7fcb7cd99640 1 --2- 172.21.15.2:0/755046461 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcb6c0a1a30 0x7fcb6c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:19.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.318+0000 7fcb7ed9d640 1 --2- 172.21.15.2:0/755046461 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcb6c0a1a30 0x7fcb6c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:19.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.318+0000 7fcb7cd99640 1 -- 172.21.15.2:0/755046461 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fcb680efcb0 con 0x7fcb80102780 2024-03-15T12:56:19.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.319+0000 7fcb7ed9d640 1 --2- 172.21.15.2:0/755046461 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcb6c0a1a30 0x7fcb6c0a3ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcb70005fd0 tx=0x7fcb70005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:19.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.322+0000 7fcb7cd99640 1 -- 172.21.15.2:0/755046461 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcb680b91f0 con 0x7fcb80102780 2024-03-15T12:56:19.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:19.505+0000 7fcb850c3640 1 -- 172.21.15.2:0/755046461 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fcb8019f780 con 0x7fcb80102780 2024-03-15T12:56:19.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:19 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/205628640' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-15T12:56:19.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:19 smithi002 ceph-mon[32060]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:19.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:19 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:19.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:19 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/205628640' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-15T12:56:19.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:19 smithi184 ceph-mon[35993]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:19.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:19 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:19.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:19 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/205628640' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-15T12:56:19.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:19 smithi149 ceph-mon[35927]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:19.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:56:20.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.191+0000 7fcb7cd99640 1 -- 172.21.15.2:0/755046461 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) v1 ==== 171+0+0 (secure 0 0 0) 0x7fcb8019f780 con 0x7fcb80102780 2024-03-15T12:56:20.177 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' created 2024-03-15T12:56:20.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.194+0000 7fcb850c3640 1 -- 172.21.15.2:0/755046461 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcb6c0a1a30 msgr2=0x7fcb6c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:20.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.194+0000 7fcb850c3640 1 --2- 172.21.15.2:0/755046461 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcb6c0a1a30 0x7fcb6c0a3ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcb70005fd0 tx=0x7fcb70005850 comp rx=0 tx=0).stop 2024-03-15T12:56:20.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.194+0000 7fcb850c3640 1 -- 172.21.15.2:0/755046461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb80102780 msgr2=0x7fcb80197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:20.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.194+0000 7fcb850c3640 1 --2- 172.21.15.2:0/755046461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb80102780 0x7fcb80197a90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb68000c00 tx=0x7fcb68004040 comp rx=0 tx=0).stop 2024-03-15T12:56:20.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.194+0000 7fcb850c3640 1 -- 172.21.15.2:0/755046461 shutdown_connections 2024-03-15T12:56:20.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.194+0000 7fcb850c3640 1 --2- 172.21.15.2:0/755046461 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcb6c0a1a30 0x7fcb6c0a3ef0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:20.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.194+0000 7fcb850c3640 1 --2- 172.21.15.2:0/755046461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb80103140 0x7fcb8019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:20.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.194+0000 7fcb850c3640 1 --2- 172.21.15.2:0/755046461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb80102780 0x7fcb80197a90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:20.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.194+0000 7fcb850c3640 1 --2- 172.21.15.2:0/755046461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb800ff7b0 0x7fcb80197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:20.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.194+0000 7fcb850c3640 1 -- 172.21.15.2:0/755046461 >> 172.21.15.2:0/755046461 conn(0x7fcb800fb100 msgr2=0x7fcb80105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:20.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.195+0000 7fcb850c3640 1 -- 172.21.15.2:0/755046461 shutdown_connections 2024-03-15T12:56:20.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.195+0000 7fcb850c3640 1 -- 172.21.15.2:0/755046461 wait complete. 2024-03-15T12:56:20.203 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-15T12:56:20.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.326+0000 7f6948548640 1 -- 172.21.15.2:0/2857372892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69401071d0 msgr2=0x7f6940109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:20.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.326+0000 7f6948548640 1 --2- 172.21.15.2:0/2857372892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69401071d0 0x7f6940109620 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6930009ca0 tx=0x7f693002c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:20.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.326+0000 7f6948548640 1 -- 172.21.15.2:0/2857372892 shutdown_connections 2024-03-15T12:56:20.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.326+0000 7f6948548640 1 --2- 172.21.15.2:0/2857372892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69401071d0 0x7f6940109620 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:20.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.326+0000 7f6948548640 1 --2- 172.21.15.2:0/2857372892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69401048a0 0x7f6940106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:20.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.326+0000 7f6948548640 1 --2- 172.21.15.2:0/2857372892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6940101f70 0x7f6940104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:20.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.326+0000 7f6948548640 1 -- 172.21.15.2:0/2857372892 >> 172.21.15.2:0/2857372892 conn(0x7f69400fb980 msgr2=0x7f69400fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:20.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.327+0000 7f6948548640 1 -- 172.21.15.2:0/2857372892 shutdown_connections 2024-03-15T12:56:20.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.327+0000 7f6948548640 1 -- 172.21.15.2:0/2857372892 wait complete. 2024-03-15T12:56:20.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.327+0000 7f6948548640 1 Processor -- start 2024-03-15T12:56:20.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.327+0000 7f6948548640 1 -- start start 2024-03-15T12:56:20.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.327+0000 7f6948548640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6940101f70 0x7f6940197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:20.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.327+0000 7f6948548640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69401048a0 0x7f6940197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:20.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.327+0000 7f6948548640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69401071d0 0x7f694019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:20.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.327+0000 7f6948548640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694019eff0 con 0x7f69401048a0 2024-03-15T12:56:20.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.327+0000 7f6948548640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694019f160 con 0x7f6940101f70 2024-03-15T12:56:20.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.327+0000 7f6948548640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694019f320 con 0x7f69401071d0 2024-03-15T12:56:20.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.327+0000 7f6945ae3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69401048a0 0x7f6940197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:20.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.327+0000 7f6945ae3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69401048a0 0x7f6940197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52108/0 (socket says 172.21.15.2:52108) 2024-03-15T12:56:20.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.327+0000 7f6945ae3640 1 -- 172.21.15.2:0/862808734 learned_addr learned my addr 172.21.15.2:0/862808734 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:20.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.327+0000 7f69462e4640 1 --2- 172.21.15.2:0/862808734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6940101f70 0x7f6940197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:20.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.328+0000 7f6945ae3640 1 -- 172.21.15.2:0/862808734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69401071d0 msgr2=0x7f694019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:20.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.328+0000 7f6945ae3640 1 --2- 172.21.15.2:0/862808734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69401071d0 0x7f694019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:20.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.328+0000 7f6945ae3640 1 -- 172.21.15.2:0/862808734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6940101f70 msgr2=0x7f6940197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:20.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.328+0000 7f6945ae3640 1 --2- 172.21.15.2:0/862808734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6940101f70 0x7f6940197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:20.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.328+0000 7f6945ae3640 1 -- 172.21.15.2:0/862808734 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6930009900 con 0x7f69401048a0 2024-03-15T12:56:20.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.328+0000 7f69462e4640 1 --2- 172.21.15.2:0/862808734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6940101f70 0x7f6940197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:56:20.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.328+0000 7f6945ae3640 1 --2- 172.21.15.2:0/862808734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69401048a0 0x7f6940197a40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f693400d740 tx=0x7f693400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:20.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.328+0000 7f693f7fe640 1 -- 172.21.15.2:0/862808734 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934009c30 con 0x7f69401048a0 2024-03-15T12:56:20.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.328+0000 7f693f7fe640 1 -- 172.21.15.2:0/862808734 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f693400de50 con 0x7f69401048a0 2024-03-15T12:56:20.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.328+0000 7f6948548640 1 -- 172.21.15.2:0/862808734 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694019f600 con 0x7f69401048a0 2024-03-15T12:56:20.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.328+0000 7f693f7fe640 1 -- 172.21.15.2:0/862808734 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934010460 con 0x7f69401048a0 2024-03-15T12:56:20.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.328+0000 7f6948548640 1 -- 172.21.15.2:0/862808734 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694019fac0 con 0x7f69401048a0 2024-03-15T12:56:20.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.329+0000 7f6948548640 1 -- 172.21.15.2:0/862808734 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69400fd470 con 0x7f69401048a0 2024-03-15T12:56:20.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.330+0000 7f693f7fe640 1 -- 172.21.15.2:0/862808734 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f6934004f10 con 0x7f69401048a0 2024-03-15T12:56:20.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.331+0000 7f693f7fe640 1 --2- 172.21.15.2:0/862808734 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f69280a1a30 0x7f69280a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:20.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.331+0000 7f69462e4640 1 --2- 172.21.15.2:0/862808734 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f69280a1a30 0x7f69280a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:20.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.331+0000 7f693f7fe640 1 -- 172.21.15.2:0/862808734 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f69340cd0b0 con 0x7f69401048a0 2024-03-15T12:56:20.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.331+0000 7f69462e4640 1 --2- 172.21.15.2:0/862808734 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f69280a1a30 0x7f69280a3ef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f693800abe0 tx=0x7f693800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:20.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.335+0000 7f693f7fe640 1 -- 172.21.15.2:0/862808734 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6934096470 con 0x7f69401048a0 2024-03-15T12:56:20.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:20.557+0000 7f6948548640 1 -- 172.21.15.2:0/862808734 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f694002d110 con 0x7f69401048a0 2024-03-15T12:56:20.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:20 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/205628640' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-15T12:56:20.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:20 smithi002 ceph-mon[32060]: osdmap e82: 12 total, 12 up, 12 in 2024-03-15T12:56:20.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:20 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/755046461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-15T12:56:20.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:20 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/205628640' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-15T12:56:20.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:20 smithi184 ceph-mon[35993]: osdmap e82: 12 total, 12 up, 12 in 2024-03-15T12:56:20.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:20 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/755046461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-15T12:56:20.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:20 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/205628640' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-15T12:56:20.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:20 smithi149 ceph-mon[35927]: osdmap e82: 12 total, 12 up, 12 in 2024-03-15T12:56:20.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:20 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/755046461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-15T12:56:21.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.399+0000 7f693f7fe640 1 -- 172.21.15.2:0/862808734 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6934017020 con 0x7f69401048a0 2024-03-15T12:56:21.386 INFO:teuthology.orchestra.run.smithi002.stderr:new fs with metadata pool 2 and data pool 3 2024-03-15T12:56:21.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.403+0000 7f6948548640 1 -- 172.21.15.2:0/862808734 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f69280a1a30 msgr2=0x7f69280a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:21.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.403+0000 7f6948548640 1 --2- 172.21.15.2:0/862808734 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f69280a1a30 0x7f69280a3ef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f693800abe0 tx=0x7f693800a5c0 comp rx=0 tx=0).stop 2024-03-15T12:56:21.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.403+0000 7f6948548640 1 -- 172.21.15.2:0/862808734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69401048a0 msgr2=0x7f6940197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:21.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.403+0000 7f6948548640 1 --2- 172.21.15.2:0/862808734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69401048a0 0x7f6940197a40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f693400d740 tx=0x7f693400dc00 comp rx=0 tx=0).stop 2024-03-15T12:56:21.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.403+0000 7f6948548640 1 -- 172.21.15.2:0/862808734 shutdown_connections 2024-03-15T12:56:21.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.403+0000 7f6948548640 1 --2- 172.21.15.2:0/862808734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69401071d0 0x7f694019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:21.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.404+0000 7f6948548640 1 --2- 172.21.15.2:0/862808734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69401048a0 0x7f6940197a40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:21.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.404+0000 7f6948548640 1 --2- 172.21.15.2:0/862808734 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f69280a1a30 0x7f69280a3ef0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:21.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.404+0000 7f6948548640 1 --2- 172.21.15.2:0/862808734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6940101f70 0x7f6940197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:21.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.404+0000 7f6948548640 1 -- 172.21.15.2:0/862808734 >> 172.21.15.2:0/862808734 conn(0x7f69400fb980 msgr2=0x7f69401028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:21.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.404+0000 7f6948548640 1 -- 172.21.15.2:0/862808734 shutdown_connections 2024-03-15T12:56:21.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.404+0000 7f6948548640 1 -- 172.21.15.2:0/862808734 wait complete. 2024-03-15T12:56:21.406 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-15T12:56:21.406 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-15T12:56:21.431 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:21 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/755046461' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-15T12:56:21.432 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:21 smithi002 ceph-mon[32060]: osdmap e83: 12 total, 12 up, 12 in 2024-03-15T12:56:21.432 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:21 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/862808734' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-15T12:56:21.432 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:21 smithi002 ceph-mon[32060]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-15T12:56:21.432 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:21 smithi002 ceph-mon[32060]: pgmap v284: 97 pgs: 19 creating+peering, 1 active+clean, 77 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:21.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.532+0000 7f6a68d27640 1 -- 172.21.15.2:0/3828133920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a641003f0 msgr2=0x7f6a64100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:21.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.532+0000 7f6a68d27640 1 --2- 172.21.15.2:0/3828133920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a641003f0 0x7f6a64100810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009900 tx=0x7f6a5402bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:21.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.533+0000 7f6a68d27640 1 -- 172.21.15.2:0/3828133920 shutdown_connections 2024-03-15T12:56:21.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.533+0000 7f6a68d27640 1 --2- 172.21.15.2:0/3828133920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a64101fa0 0x7f6a64108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:21.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.533+0000 7f6a68d27640 1 --2- 172.21.15.2:0/3828133920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a641015e0 0x7f6a64101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:21.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.533+0000 7f6a68d27640 1 --2- 172.21.15.2:0/3828133920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a641003f0 0x7f6a64100810 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:21.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.533+0000 7f6a68d27640 1 -- 172.21.15.2:0/3828133920 >> 172.21.15.2:0/3828133920 conn(0x7f6a640fb960 msgr2=0x7f6a640fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:21.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.533+0000 7f6a68d27640 1 -- 172.21.15.2:0/3828133920 shutdown_connections 2024-03-15T12:56:21.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.533+0000 7f6a68d27640 1 -- 172.21.15.2:0/3828133920 wait complete. 2024-03-15T12:56:21.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.533+0000 7f6a68d27640 1 Processor -- start 2024-03-15T12:56:21.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.533+0000 7f6a68d27640 1 -- start start 2024-03-15T12:56:21.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a68d27640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a641003f0 0x7f6a64197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:21.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a68d27640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a641015e0 0x7f6a64197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:21.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a68d27640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a64101fa0 0x7f6a6419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:21.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a68d27640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6419f0f0 con 0x7f6a641003f0 2024-03-15T12:56:21.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a68d27640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6419f260 con 0x7f6a64101fa0 2024-03-15T12:56:21.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a68d27640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6419f420 con 0x7f6a641015e0 2024-03-15T12:56:21.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a6259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a641003f0 0x7f6a64197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:21.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a6259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a641003f0 0x7f6a64197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52130/0 (socket says 172.21.15.2:52130) 2024-03-15T12:56:21.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a6259c640 1 -- 172.21.15.2:0/1536904678 learned_addr learned my addr 172.21.15.2:0/1536904678 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:21.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a62d9d640 1 --2- 172.21.15.2:0/1536904678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a64101fa0 0x7f6a6419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:21.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a5bfff640 1 --2- 172.21.15.2:0/1536904678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a641015e0 0x7f6a64197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:21.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a6259c640 1 -- 172.21.15.2:0/1536904678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a641015e0 msgr2=0x7f6a64197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:21.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a6259c640 1 --2- 172.21.15.2:0/1536904678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a641015e0 0x7f6a64197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:21.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a6259c640 1 -- 172.21.15.2:0/1536904678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a64101fa0 msgr2=0x7f6a6419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:21.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a6259c640 1 --2- 172.21.15.2:0/1536904678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a64101fa0 0x7f6a6419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:21.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a6259c640 1 -- 172.21.15.2:0/1536904678 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a54009560 con 0x7f6a641003f0 2024-03-15T12:56:21.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a5bfff640 1 --2- 172.21.15.2:0/1536904678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a641015e0 0x7f6a64197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T12:56:21.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a6259c640 1 --2- 172.21.15.2:0/1536904678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a641003f0 0x7f6a64197600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6a54000c00 tx=0x7f6a54004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:21.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a5b7fe640 1 -- 172.21.15.2:0/1536904678 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a540042d0 con 0x7f6a641003f0 2024-03-15T12:56:21.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a68d27640 1 -- 172.21.15.2:0/1536904678 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6419f6a0 con 0x7f6a641003f0 2024-03-15T12:56:21.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a5b7fe640 1 -- 172.21.15.2:0/1536904678 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6a54004430 con 0x7f6a641003f0 2024-03-15T12:56:21.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a5b7fe640 1 -- 172.21.15.2:0/1536904678 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54003e20 con 0x7f6a641003f0 2024-03-15T12:56:21.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.534+0000 7f6a68d27640 1 -- 172.21.15.2:0/1536904678 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6419fc00 con 0x7f6a641003f0 2024-03-15T12:56:21.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.535+0000 7f6a68d27640 1 -- 172.21.15.2:0/1536904678 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6410bdd0 con 0x7f6a641003f0 2024-03-15T12:56:21.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.536+0000 7f6a5b7fe640 1 -- 172.21.15.2:0/1536904678 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f6a540035f0 con 0x7f6a641003f0 2024-03-15T12:56:21.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.537+0000 7f6a5b7fe640 1 --2- 172.21.15.2:0/1536904678 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6a500a1a30 0x7f6a500a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:21.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.537+0000 7f6a5b7fe640 1 -- 172.21.15.2:0/1536904678 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6a540ef5b0 con 0x7f6a641003f0 2024-03-15T12:56:21.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.537+0000 7f6a5bfff640 1 --2- 172.21.15.2:0/1536904678 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6a500a1a30 0x7f6a500a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:21.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.537+0000 7f6a5bfff640 1 --2- 172.21.15.2:0/1536904678 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6a500a1a30 0x7f6a500a3ef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6a64198b10 tx=0x7f6a4c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:21.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.542+0000 7f6a5b7fe640 1 -- 172.21.15.2:0/1536904678 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6a540b8910 con 0x7f6a641003f0 2024-03-15T12:56:21.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:21 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/755046461' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-15T12:56:21.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:21 smithi184 ceph-mon[35993]: osdmap e83: 12 total, 12 up, 12 in 2024-03-15T12:56:21.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:21 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/862808734' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-15T12:56:21.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:21 smithi184 ceph-mon[35993]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-15T12:56:21.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:21 smithi184 ceph-mon[35993]: pgmap v284: 97 pgs: 19 creating+peering, 1 active+clean, 77 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:21.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:21 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/755046461' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-15T12:56:21.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:21 smithi149 ceph-mon[35927]: osdmap e83: 12 total, 12 up, 12 in 2024-03-15T12:56:21.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:21 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/862808734' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-15T12:56:21.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:21 smithi149 ceph-mon[35927]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-15T12:56:21.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:21 smithi149 ceph-mon[35927]: pgmap v284: 97 pgs: 19 creating+peering, 1 active+clean, 77 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:21.731+0000 7f6a68d27640 1 -- 172.21.15.2:0/1536904678 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f6a6410bf60 con 0x7f6a641003f0 2024-03-15T12:56:22.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.194+0000 7f6a5b7fe640 1 -- 172.21.15.2:0/1536904678 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) v1 ==== 157+0+0 (secure 0 0 0) 0x7f6a540bbfd0 con 0x7f6a641003f0 2024-03-15T12:56:22.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.197+0000 7f6a68d27640 1 -- 172.21.15.2:0/1536904678 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6a500a1a30 msgr2=0x7f6a500a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:22.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.197+0000 7f6a68d27640 1 --2- 172.21.15.2:0/1536904678 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6a500a1a30 0x7f6a500a3ef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6a64198b10 tx=0x7f6a4c005e50 comp rx=0 tx=0).stop 2024-03-15T12:56:22.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.197+0000 7f6a68d27640 1 -- 172.21.15.2:0/1536904678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a641003f0 msgr2=0x7f6a64197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:22.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.197+0000 7f6a68d27640 1 --2- 172.21.15.2:0/1536904678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a641003f0 0x7f6a64197600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6a54000c00 tx=0x7f6a54004ab0 comp rx=0 tx=0).stop 2024-03-15T12:56:22.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.197+0000 7f6a68d27640 1 -- 172.21.15.2:0/1536904678 shutdown_connections 2024-03-15T12:56:22.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.197+0000 7f6a68d27640 1 --2- 172.21.15.2:0/1536904678 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6a500a1a30 0x7f6a500a3ef0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:22.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.197+0000 7f6a68d27640 1 --2- 172.21.15.2:0/1536904678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a64101fa0 0x7f6a6419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:22.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.197+0000 7f6a68d27640 1 --2- 172.21.15.2:0/1536904678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a641015e0 0x7f6a64197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:22.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.197+0000 7f6a68d27640 1 --2- 172.21.15.2:0/1536904678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a641003f0 0x7f6a64197600 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:22.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.197+0000 7f6a68d27640 1 -- 172.21.15.2:0/1536904678 >> 172.21.15.2:0/1536904678 conn(0x7f6a640fb960 msgr2=0x7f6a640fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:22.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.197+0000 7f6a68d27640 1 -- 172.21.15.2:0/1536904678 shutdown_connections 2024-03-15T12:56:22.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.197+0000 7f6a68d27640 1 -- 172.21.15.2:0/1536904678 wait complete. 2024-03-15T12:56:22.197 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-15T12:56:22.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.366+0000 7f16dcd1e640 1 -- 172.21.15.2:0/2051901766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d8101610 msgr2=0x7f16d8101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:22.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.366+0000 7f16dcd1e640 1 --2- 172.21.15.2:0/2051901766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d8101610 0x7f16d8101a90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009900 tx=0x7f16c002bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:22.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.366+0000 7f16dcd1e640 1 -- 172.21.15.2:0/2051901766 shutdown_connections 2024-03-15T12:56:22.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.366+0000 7f16dcd1e640 1 --2- 172.21.15.2:0/2051901766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16d8101fd0 0x7f16d8108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:22.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.366+0000 7f16dcd1e640 1 --2- 172.21.15.2:0/2051901766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d8101610 0x7f16d8101a90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:22.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.366+0000 7f16dcd1e640 1 --2- 172.21.15.2:0/2051901766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16d8100420 0x7f16d8100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:22.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.366+0000 7f16dcd1e640 1 -- 172.21.15.2:0/2051901766 >> 172.21.15.2:0/2051901766 conn(0x7f16d80fb9d0 msgr2=0x7f16d80fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:22.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.366+0000 7f16dcd1e640 1 -- 172.21.15.2:0/2051901766 shutdown_connections 2024-03-15T12:56:22.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.366+0000 7f16dcd1e640 1 -- 172.21.15.2:0/2051901766 wait complete. 2024-03-15T12:56:22.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16dcd1e640 1 Processor -- start 2024-03-15T12:56:22.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16dcd1e640 1 -- start start 2024-03-15T12:56:22.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16dcd1e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16d8100420 0x7f16d81975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:22.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16dcd1e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16d8101610 0x7f16d8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:22.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16dcd1e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d8101fd0 0x7f16d819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:22.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16dcd1e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d819f0a0 con 0x7f16d8101fd0 2024-03-15T12:56:22.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16dcd1e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d819f210 con 0x7f16d8100420 2024-03-15T12:56:22.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16dcd1e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d819f3d0 con 0x7f16d8101610 2024-03-15T12:56:22.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16d6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d8101fd0 0x7f16d819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:22.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16d6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d8101fd0 0x7f16d819eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52148/0 (socket says 172.21.15.2:52148) 2024-03-15T12:56:22.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16d6d9d640 1 -- 172.21.15.2:0/1025113664 learned_addr learned my addr 172.21.15.2:0/1025113664 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:22.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16d659c640 1 --2- 172.21.15.2:0/1025113664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16d8100420 0x7f16d81975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:22.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16cffff640 1 --2- 172.21.15.2:0/1025113664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16d8101610 0x7f16d8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:22.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16d6d9d640 1 -- 172.21.15.2:0/1025113664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16d8101610 msgr2=0x7f16d8197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:22.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16d6d9d640 1 --2- 172.21.15.2:0/1025113664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16d8101610 0x7f16d8197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:22.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16d6d9d640 1 -- 172.21.15.2:0/1025113664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16d8100420 msgr2=0x7f16d81975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:22.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16d6d9d640 1 --2- 172.21.15.2:0/1025113664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16d8100420 0x7f16d81975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:22.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16d6d9d640 1 -- 172.21.15.2:0/1025113664 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c0009560 con 0x7f16d8101fd0 2024-03-15T12:56:22.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.367+0000 7f16d6d9d640 1 --2- 172.21.15.2:0/1025113664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d8101fd0 0x7f16d819eb60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f16d000eaa0 tx=0x7f16d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:22.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.368+0000 7f16cf7fe640 1 -- 172.21.15.2:0/1025113664 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d000cdb0 con 0x7f16d8101fd0 2024-03-15T12:56:22.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.368+0000 7f16dcd1e640 1 -- 172.21.15.2:0/1025113664 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d819f6b0 con 0x7f16d8101fd0 2024-03-15T12:56:22.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.368+0000 7f16cf7fe640 1 -- 172.21.15.2:0/1025113664 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f16d0005020 con 0x7f16d8101fd0 2024-03-15T12:56:22.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.368+0000 7f16cf7fe640 1 -- 172.21.15.2:0/1025113664 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d00116d0 con 0x7f16d8101fd0 2024-03-15T12:56:22.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.368+0000 7f16dcd1e640 1 -- 172.21.15.2:0/1025113664 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d819fb90 con 0x7f16d8101fd0 2024-03-15T12:56:22.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.369+0000 7f16dcd1e640 1 -- 172.21.15.2:0/1025113664 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d8068730 con 0x7f16d8101fd0 2024-03-15T12:56:22.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.370+0000 7f16cf7fe640 1 -- 172.21.15.2:0/1025113664 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f16d0005590 con 0x7f16d8101fd0 2024-03-15T12:56:22.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.370+0000 7f16cf7fe640 1 --2- 172.21.15.2:0/1025113664 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f16c40a1a30 0x7f16c40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:22.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.370+0000 7f16d659c640 1 --2- 172.21.15.2:0/1025113664 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f16c40a1a30 0x7f16c40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:22.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.370+0000 7f16cf7fe640 1 -- 172.21.15.2:0/1025113664 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f16d00cda30 con 0x7f16d8101fd0 2024-03-15T12:56:22.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.371+0000 7f16d659c640 1 --2- 172.21.15.2:0/1025113664 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f16c40a1a30 0x7f16c40a3ef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f16d8101470 tx=0x7f16c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:22.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.374+0000 7f16cf7fe640 1 -- 172.21.15.2:0/1025113664 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f16d0096c10 con 0x7f16d8101fd0 2024-03-15T12:56:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:22.557+0000 7f16dcd1e640 1 -- 172.21.15.2:0/1025113664 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f16d8107670 con 0x7f16d8101fd0 2024-03-15T12:56:22.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:22 smithi002 ceph-mon[32060]: osdmap e84: 12 total, 12 up, 12 in 2024-03-15T12:56:22.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:22 smithi002 ceph-mon[32060]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-15T12:56:22.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:22 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/862808734' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-15T12:56:22.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:22 smithi002 ceph-mon[32060]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-15T12:56:22.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:22 smithi002 ceph-mon[32060]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-15T12:56:22.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:22 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1536904678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-15T12:56:22.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:22 smithi184 ceph-mon[35993]: osdmap e84: 12 total, 12 up, 12 in 2024-03-15T12:56:22.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:22 smithi184 ceph-mon[35993]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-15T12:56:22.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:22 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/862808734' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-15T12:56:22.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:22 smithi184 ceph-mon[35993]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-15T12:56:22.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:22 smithi184 ceph-mon[35993]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-15T12:56:22.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:22 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1536904678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-15T12:56:22.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:22 smithi149 ceph-mon[35927]: osdmap e84: 12 total, 12 up, 12 in 2024-03-15T12:56:22.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:22 smithi149 ceph-mon[35927]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-15T12:56:22.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:22 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/862808734' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-15T12:56:22.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:22 smithi149 ceph-mon[35927]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-15T12:56:22.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:22 smithi149 ceph-mon[35927]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-15T12:56:22.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:22 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1536904678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-15T12:56:23.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:23 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1536904678' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-15T12:56:23.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:23 smithi002 ceph-mon[32060]: osdmap e85: 12 total, 12 up, 12 in 2024-03-15T12:56:23.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:23 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] up:active 2024-03-15T12:56:23.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:23 smithi002 ceph-mon[32060]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-15T12:56:23.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:23 smithi002 ceph-mon[32060]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-15T12:56:23.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:23 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1025113664' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-15T12:56:23.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:23 smithi002 ceph-mon[32060]: pgmap v287: 97 pgs: 19 creating+peering, 8 active+clean, 70 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:23.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:23 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1536904678' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-15T12:56:23.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:23 smithi184 ceph-mon[35993]: osdmap e85: 12 total, 12 up, 12 in 2024-03-15T12:56:23.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:23 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] up:active 2024-03-15T12:56:23.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:23 smithi184 ceph-mon[35993]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-15T12:56:23.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:23 smithi184 ceph-mon[35993]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-15T12:56:23.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:23 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1025113664' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-15T12:56:23.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:23 smithi184 ceph-mon[35993]: pgmap v287: 97 pgs: 19 creating+peering, 8 active+clean, 70 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:23.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:23 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1536904678' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-15T12:56:23.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:23 smithi149 ceph-mon[35927]: osdmap e85: 12 total, 12 up, 12 in 2024-03-15T12:56:23.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:23 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] up:active 2024-03-15T12:56:23.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:23 smithi149 ceph-mon[35927]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-15T12:56:23.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:23 smithi149 ceph-mon[35927]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-15T12:56:23.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:23 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1025113664' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-15T12:56:23.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:23 smithi149 ceph-mon[35927]: pgmap v287: 97 pgs: 19 creating+peering, 8 active+clean, 70 unknown; 577 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:24.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.412+0000 7f16cf7fe640 1 -- 172.21.15.2:0/1025113664 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) v1 ==== 228+0+0 (secure 0 0 0) 0x7f16d001f070 con 0x7f16d8101fd0 2024-03-15T12:56:24.399 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data_ec' created 2024-03-15T12:56:24.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.415+0000 7f16dcd1e640 1 -- 172.21.15.2:0/1025113664 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f16c40a1a30 msgr2=0x7f16c40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:24.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.416+0000 7f16dcd1e640 1 --2- 172.21.15.2:0/1025113664 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f16c40a1a30 0x7f16c40a3ef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f16d8101470 tx=0x7f16c8005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:24.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.416+0000 7f16dcd1e640 1 -- 172.21.15.2:0/1025113664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d8101fd0 msgr2=0x7f16d819eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:24.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.416+0000 7f16dcd1e640 1 --2- 172.21.15.2:0/1025113664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d8101fd0 0x7f16d819eb60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f16d000eaa0 tx=0x7f16d000ef60 comp rx=0 tx=0).stop 2024-03-15T12:56:24.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.416+0000 7f16dcd1e640 1 -- 172.21.15.2:0/1025113664 shutdown_connections 2024-03-15T12:56:24.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.416+0000 7f16dcd1e640 1 --2- 172.21.15.2:0/1025113664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d8101fd0 0x7f16d819eb60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:24.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.416+0000 7f16dcd1e640 1 --2- 172.21.15.2:0/1025113664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16d8101610 0x7f16d8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:24.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.416+0000 7f16dcd1e640 1 --2- 172.21.15.2:0/1025113664 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f16c40a1a30 0x7f16c40a3ef0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:24.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.416+0000 7f16dcd1e640 1 --2- 172.21.15.2:0/1025113664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16d8100420 0x7f16d81975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:24.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.416+0000 7f16dcd1e640 1 -- 172.21.15.2:0/1025113664 >> 172.21.15.2:0/1025113664 conn(0x7f16d80fb9d0 msgr2=0x7f16d8104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:24.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.416+0000 7f16dcd1e640 1 -- 172.21.15.2:0/1025113664 shutdown_connections 2024-03-15T12:56:24.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.416+0000 7f16dcd1e640 1 -- 172.21.15.2:0/1025113664 wait complete. 2024-03-15T12:56:24.421 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-15T12:56:24.446 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:24 smithi002 ceph-mon[32060]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-15T12:56:24.446 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:24 smithi002 ceph-mon[32060]: Cluster is now healthy 2024-03-15T12:56:24.446 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:24 smithi002 ceph-mon[32060]: osdmap e86: 12 total, 12 up, 12 in 2024-03-15T12:56:24.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.547+0000 7f9b476df640 1 -- 172.21.15.2:0/561319109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b40101fc0 msgr2=0x7f9b40108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:24.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.547+0000 7f9b476df640 1 --2- 172.21.15.2:0/561319109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b40101fc0 0x7f9b40108850 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009ca0 tx=0x7f9b3802c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.547+0000 7f9b476df640 1 -- 172.21.15.2:0/561319109 shutdown_connections 2024-03-15T12:56:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.547+0000 7f9b476df640 1 --2- 172.21.15.2:0/561319109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b40101fc0 0x7f9b40108850 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.547+0000 7f9b476df640 1 --2- 172.21.15.2:0/561319109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b40101600 0x7f9b40101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.547+0000 7f9b476df640 1 --2- 172.21.15.2:0/561319109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b40100410 0x7f9b40100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.547+0000 7f9b476df640 1 -- 172.21.15.2:0/561319109 >> 172.21.15.2:0/561319109 conn(0x7f9b400fb9a0 msgr2=0x7f9b400fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.547+0000 7f9b476df640 1 -- 172.21.15.2:0/561319109 shutdown_connections 2024-03-15T12:56:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.547+0000 7f9b476df640 1 -- 172.21.15.2:0/561319109 wait complete. 2024-03-15T12:56:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b476df640 1 Processor -- start 2024-03-15T12:56:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b476df640 1 -- start start 2024-03-15T12:56:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b476df640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b40100410 0x7f9b40197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b476df640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b40101600 0x7f9b40197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b476df640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b40101fc0 0x7f9b4019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b476df640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4019f000 con 0x7f9b40100410 2024-03-15T12:56:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b476df640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4019f170 con 0x7f9b40101600 2024-03-15T12:56:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b476df640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4019f330 con 0x7f9b40101fc0 2024-03-15T12:56:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b4547b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b40100410 0x7f9b40197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b4547b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b40100410 0x7f9b40197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52164/0 (socket says 172.21.15.2:52164) 2024-03-15T12:56:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b4547b640 1 -- 172.21.15.2:0/3552197270 learned_addr learned my addr 172.21.15.2:0/3552197270 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b44c7a640 1 --2- 172.21.15.2:0/3552197270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b40101600 0x7f9b40197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b45c7c640 1 --2- 172.21.15.2:0/3552197270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b40101fc0 0x7f9b4019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b4547b640 1 -- 172.21.15.2:0/3552197270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b40101fc0 msgr2=0x7f9b4019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b4547b640 1 --2- 172.21.15.2:0/3552197270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b40101fc0 0x7f9b4019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b4547b640 1 -- 172.21.15.2:0/3552197270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b40101600 msgr2=0x7f9b40197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b4547b640 1 --2- 172.21.15.2:0/3552197270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b40101600 0x7f9b40197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b4547b640 1 -- 172.21.15.2:0/3552197270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b38009900 con 0x7f9b40100410 2024-03-15T12:56:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.548+0000 7f9b4547b640 1 --2- 172.21.15.2:0/3552197270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b40100410 0x7f9b40197510 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00d770 tx=0x7f9b3c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.549+0000 7f9b367fc640 1 -- 172.21.15.2:0/3552197270 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c009c30 con 0x7f9b40100410 2024-03-15T12:56:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.549+0000 7f9b367fc640 1 -- 172.21.15.2:0/3552197270 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9b3c00de80 con 0x7f9b40100410 2024-03-15T12:56:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.549+0000 7f9b476df640 1 -- 172.21.15.2:0/3552197270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4019f610 con 0x7f9b40100410 2024-03-15T12:56:24.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.549+0000 7f9b367fc640 1 -- 172.21.15.2:0/3552197270 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c010460 con 0x7f9b40100410 2024-03-15T12:56:24.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.549+0000 7f9b476df640 1 -- 172.21.15.2:0/3552197270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4019fb20 con 0x7f9b40100410 2024-03-15T12:56:24.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.550+0000 7f9b476df640 1 -- 172.21.15.2:0/3552197270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4010bde0 con 0x7f9b40100410 2024-03-15T12:56:24.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.551+0000 7f9b367fc640 1 -- 172.21.15.2:0/3552197270 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f9b3c003ec0 con 0x7f9b40100410 2024-03-15T12:56:24.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.551+0000 7f9b367fc640 1 --2- 172.21.15.2:0/3552197270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9b2c0a1a30 0x7f9b2c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:24.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.551+0000 7f9b44c7a640 1 --2- 172.21.15.2:0/3552197270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9b2c0a1a30 0x7f9b2c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:24.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.551+0000 7f9b367fc640 1 -- 172.21.15.2:0/3552197270 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f9b3c0cd310 con 0x7f9b40100410 2024-03-15T12:56:24.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.552+0000 7f9b44c7a640 1 --2- 172.21.15.2:0/3552197270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9b2c0a1a30 0x7f9b2c0a3ef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b40198a20 tx=0x7f9b30005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:24.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.555+0000 7f9b367fc640 1 -- 172.21.15.2:0/3552197270 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9b3c0963b0 con 0x7f9b40100410 2024-03-15T12:56:24.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:24 smithi184 ceph-mon[35993]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-15T12:56:24.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:24 smithi184 ceph-mon[35993]: Cluster is now healthy 2024-03-15T12:56:24.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:24 smithi184 ceph-mon[35993]: osdmap e86: 12 total, 12 up, 12 in 2024-03-15T12:56:24.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:24 smithi149 ceph-mon[35927]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-15T12:56:24.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:24 smithi149 ceph-mon[35927]: Cluster is now healthy 2024-03-15T12:56:24.610 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:24 smithi149 ceph-mon[35927]: osdmap e86: 12 total, 12 up, 12 in 2024-03-15T12:56:24.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:24.737+0000 7f9b476df640 1 -- 172.21.15.2:0/3552197270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f9b4010bfa0 con 0x7f9b40100410 2024-03-15T12:56:25.110 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 12:56:24 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-b[65861]: 2024-03-15T12:56:24.730+0000 7fb0fe4d8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-15T12:56:25.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.418+0000 7f9b367fc640 1 -- 172.21.15.2:0/3552197270 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9b3c0170c0 con 0x7f9b40100410 2024-03-15T12:56:25.404 INFO:teuthology.orchestra.run.smithi002.stderr:set pool 4 allow_ec_overwrites to true 2024-03-15T12:56:25.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.422+0000 7f9b476df640 1 -- 172.21.15.2:0/3552197270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9b2c0a1a30 msgr2=0x7f9b2c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:25.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.422+0000 7f9b476df640 1 --2- 172.21.15.2:0/3552197270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9b2c0a1a30 0x7f9b2c0a3ef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b40198a20 tx=0x7f9b30005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:25.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.422+0000 7f9b476df640 1 -- 172.21.15.2:0/3552197270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b40100410 msgr2=0x7f9b40197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:25.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.422+0000 7f9b476df640 1 --2- 172.21.15.2:0/3552197270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b40100410 0x7f9b40197510 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00d770 tx=0x7f9b3c00dc30 comp rx=0 tx=0).stop 2024-03-15T12:56:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.422+0000 7f9b476df640 1 -- 172.21.15.2:0/3552197270 shutdown_connections 2024-03-15T12:56:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.422+0000 7f9b476df640 1 --2- 172.21.15.2:0/3552197270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b40101fc0 0x7f9b4019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.422+0000 7f9b476df640 1 --2- 172.21.15.2:0/3552197270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9b2c0a1a30 0x7f9b2c0a3ef0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.422+0000 7f9b476df640 1 --2- 172.21.15.2:0/3552197270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b40101600 0x7f9b40197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.422+0000 7f9b476df640 1 --2- 172.21.15.2:0/3552197270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b40100410 0x7f9b40197510 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.422+0000 7f9b476df640 1 -- 172.21.15.2:0/3552197270 >> 172.21.15.2:0/3552197270 conn(0x7f9b400fb9a0 msgr2=0x7f9b400fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.422+0000 7f9b476df640 1 -- 172.21.15.2:0/3552197270 shutdown_connections 2024-03-15T12:56:25.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.422+0000 7f9b476df640 1 -- 172.21.15.2:0/3552197270 wait complete. 2024-03-15T12:56:25.431 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-15T12:56:25.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.596+0000 7f6529d65640 1 -- 172.21.15.2:0/1816908297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6524107290 msgr2=0x7f6524109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:25.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.596+0000 7f6529d65640 1 --2- 172.21.15.2:0/1816908297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6524107290 0x7f6524109680 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f651c009ca0 tx=0x7f651c02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:25.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.596+0000 7f6529d65640 1 -- 172.21.15.2:0/1816908297 shutdown_connections 2024-03-15T12:56:25.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.597+0000 7f6529d65640 1 --2- 172.21.15.2:0/1816908297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6524107290 0x7f6524109680 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:25.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.597+0000 7f6529d65640 1 --2- 172.21.15.2:0/1816908297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6524104960 0x7f6524106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:25.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.597+0000 7f6529d65640 1 --2- 172.21.15.2:0/1816908297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6524102030 0x7f6524104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:25.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.597+0000 7f6529d65640 1 -- 172.21.15.2:0/1816908297 >> 172.21.15.2:0/1816908297 conn(0x7f65240fb960 msgr2=0x7f65240fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:25.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.597+0000 7f6529d65640 1 -- 172.21.15.2:0/1816908297 shutdown_connections 2024-03-15T12:56:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.597+0000 7f6529d65640 1 -- 172.21.15.2:0/1816908297 wait complete. 2024-03-15T12:56:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.597+0000 7f6529d65640 1 Processor -- start 2024-03-15T12:56:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.597+0000 7f6529d65640 1 -- start start 2024-03-15T12:56:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6529d65640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6524102030 0x7f65241019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6529d65640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6524104960 0x7f65240fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6529d65640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6524107290 0x7f6524100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6529d65640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6524100c80 con 0x7f6524104960 2024-03-15T12:56:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6529d65640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6524100df0 con 0x7f6524102030 2024-03-15T12:56:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6529d65640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65241a15e0 con 0x7f6524107290 2024-03-15T12:56:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6522ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6524104960 0x7f65240fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6522ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6524104960 0x7f65240fffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52174/0 (socket says 172.21.15.2:52174) 2024-03-15T12:56:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6522ffd640 1 -- 172.21.15.2:0/632580841 learned_addr learned my addr 172.21.15.2:0/632580841 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f65237fe640 1 --2- 172.21.15.2:0/632580841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6524102030 0x7f65241019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6523fff640 1 --2- 172.21.15.2:0/632580841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6524107290 0x7f6524100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6522ffd640 1 -- 172.21.15.2:0/632580841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6524107290 msgr2=0x7f6524100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6522ffd640 1 --2- 172.21.15.2:0/632580841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6524107290 0x7f6524100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6522ffd640 1 -- 172.21.15.2:0/632580841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6524102030 msgr2=0x7f65241019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6522ffd640 1 --2- 172.21.15.2:0/632580841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6524102030 0x7f65241019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6522ffd640 1 -- 172.21.15.2:0/632580841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c009900 con 0x7f6524104960 2024-03-15T12:56:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6522ffd640 1 --2- 172.21.15.2:0/632580841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6524104960 0x7f65240fffe0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f651800d770 tx=0x7f651800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6520ff9640 1 -- 172.21.15.2:0/632580841 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518009c30 con 0x7f6524104960 2024-03-15T12:56:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6529d65640 1 -- 172.21.15.2:0/632580841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65241a1890 con 0x7f6524104960 2024-03-15T12:56:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6520ff9640 1 -- 172.21.15.2:0/632580841 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f651800de80 con 0x7f6524104960 2024-03-15T12:56:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6520ff9640 1 -- 172.21.15.2:0/632580841 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518010460 con 0x7f6524104960 2024-03-15T12:56:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.598+0000 7f6529d65640 1 -- 172.21.15.2:0/632580841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65241a1d50 con 0x7f6524104960 2024-03-15T12:56:25.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.599+0000 7f6529d65640 1 -- 172.21.15.2:0/632580841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65240fd450 con 0x7f6524104960 2024-03-15T12:56:25.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.600+0000 7f6520ff9640 1 -- 172.21.15.2:0/632580841 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f6518004ec0 con 0x7f6524104960 2024-03-15T12:56:25.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.601+0000 7f6520ff9640 1 --2- 172.21.15.2:0/632580841 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f650c0a1a30 0x7f650c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:25.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.601+0000 7f65237fe640 1 --2- 172.21.15.2:0/632580841 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f650c0a1a30 0x7f650c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:25.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.601+0000 7f6520ff9640 1 -- 172.21.15.2:0/632580841 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f65180ccf70 con 0x7f6524104960 2024-03-15T12:56:25.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.601+0000 7f65237fe640 1 --2- 172.21.15.2:0/632580841 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f650c0a1a30 0x7f650c0a3ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6514009bd0 tx=0x7f6514005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:25.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.606+0000 7f6520ff9640 1 -- 172.21.15.2:0/632580841 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6518095fb0 con 0x7f6524104960 2024-03-15T12:56:25.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:25 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1025113664' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-15T12:56:25.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:25 smithi002 ceph-mon[32060]: osdmap e87: 12 total, 12 up, 12 in 2024-03-15T12:56:25.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:25 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3552197270' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-15T12:56:25.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:25 smithi002 ceph-mon[32060]: pgmap v290: 129 pgs: 19 creating+peering, 71 active+clean, 39 unknown; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 7 op/s 2024-03-15T12:56:25.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:25 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1025113664' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-15T12:56:25.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:25 smithi184 ceph-mon[35993]: osdmap e87: 12 total, 12 up, 12 in 2024-03-15T12:56:25.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:25 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3552197270' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-15T12:56:25.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:25 smithi184 ceph-mon[35993]: pgmap v290: 129 pgs: 19 creating+peering, 71 active+clean, 39 unknown; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 7 op/s 2024-03-15T12:56:25.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:25.829+0000 7f6529d65640 1 -- 172.21.15.2:0/632580841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f652402d000 con 0x7f6524104960 2024-03-15T12:56:25.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:25 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1025113664' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-15T12:56:25.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:25 smithi149 ceph-mon[35927]: osdmap e87: 12 total, 12 up, 12 in 2024-03-15T12:56:25.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:25 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3552197270' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-15T12:56:25.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:25 smithi149 ceph-mon[35927]: pgmap v290: 129 pgs: 19 creating+peering, 71 active+clean, 39 unknown; 577 KiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 7 op/s 2024-03-15T12:56:26.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.425+0000 7f6520ff9640 1 -- 172.21.15.2:0/632580841 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f65180170b0 con 0x7f6524104960 2024-03-15T12:56:26.412 INFO:teuthology.orchestra.run.smithi002.stderr:added data pool 4 to fsmap 2024-03-15T12:56:26.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.428+0000 7f6529d65640 1 -- 172.21.15.2:0/632580841 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f650c0a1a30 msgr2=0x7f650c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:26.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.428+0000 7f6529d65640 1 --2- 172.21.15.2:0/632580841 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f650c0a1a30 0x7f650c0a3ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6514009bd0 tx=0x7f6514005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:26.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.428+0000 7f6529d65640 1 -- 172.21.15.2:0/632580841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6524104960 msgr2=0x7f65240fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:26.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.428+0000 7f6529d65640 1 --2- 172.21.15.2:0/632580841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6524104960 0x7f65240fffe0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f651800d770 tx=0x7f651800dc30 comp rx=0 tx=0).stop 2024-03-15T12:56:26.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.428+0000 7f6529d65640 1 -- 172.21.15.2:0/632580841 shutdown_connections 2024-03-15T12:56:26.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.428+0000 7f6529d65640 1 --2- 172.21.15.2:0/632580841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6524107290 0x7f6524100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.428+0000 7f6529d65640 1 --2- 172.21.15.2:0/632580841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6524104960 0x7f65240fffe0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.428+0000 7f6529d65640 1 --2- 172.21.15.2:0/632580841 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f650c0a1a30 0x7f650c0a3ef0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.428+0000 7f6529d65640 1 --2- 172.21.15.2:0/632580841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6524102030 0x7f65241019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.428+0000 7f6529d65640 1 -- 172.21.15.2:0/632580841 >> 172.21.15.2:0/632580841 conn(0x7f65240fb960 msgr2=0x7f6524102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:26.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.428+0000 7f6529d65640 1 -- 172.21.15.2:0/632580841 shutdown_connections 2024-03-15T12:56:26.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.428+0000 7f6529d65640 1 -- 172.21.15.2:0/632580841 wait complete. 2024-03-15T12:56:26.428 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:56:26.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.550+0000 7fb66058c640 1 -- 172.21.15.2:0/3192979642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb658100410 msgr2=0x7fb658100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:26.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.550+0000 7fb66058c640 1 --2- 172.21.15.2:0/3192979642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb658100410 0x7fb658100830 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb650009900 tx=0x7fb65002bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:26.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb66058c640 1 -- 172.21.15.2:0/3192979642 shutdown_connections 2024-03-15T12:56:26.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb66058c640 1 --2- 172.21.15.2:0/3192979642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb658101fc0 0x7fb658108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb66058c640 1 --2- 172.21.15.2:0/3192979642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb658101600 0x7fb658101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb66058c640 1 --2- 172.21.15.2:0/3192979642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb658100410 0x7fb658100830 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb66058c640 1 -- 172.21.15.2:0/3192979642 >> 172.21.15.2:0/3192979642 conn(0x7fb6580fb980 msgr2=0x7fb6580fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:26.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb66058c640 1 -- 172.21.15.2:0/3192979642 shutdown_connections 2024-03-15T12:56:26.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb66058c640 1 -- 172.21.15.2:0/3192979642 wait complete. 2024-03-15T12:56:26.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb66058c640 1 Processor -- start 2024-03-15T12:56:26.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb66058c640 1 -- start start 2024-03-15T12:56:26.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb66058c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb658100410 0x7fb6581975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:26.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb66058c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb658101600 0x7fb658197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:26.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb66058c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb658101fc0 0x7fb65819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:26.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb66058c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65819f0d0 con 0x7fb658101fc0 2024-03-15T12:56:26.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb66058c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65819f240 con 0x7fb658100410 2024-03-15T12:56:26.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb66058c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65819f400 con 0x7fb658101600 2024-03-15T12:56:26.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb65eb29640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb658101fc0 0x7fb65819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:26.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb65eb29640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb658101fc0 0x7fb65819eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52180/0 (socket says 172.21.15.2:52180) 2024-03-15T12:56:26.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.551+0000 7fb65eb29640 1 -- 172.21.15.2:0/1239350050 learned_addr learned my addr 172.21.15.2:0/1239350050 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:26.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.552+0000 7fb65e328640 1 --2- 172.21.15.2:0/1239350050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb658100410 0x7fb6581975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:26.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.552+0000 7fb65db27640 1 --2- 172.21.15.2:0/1239350050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb658101600 0x7fb658197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:26.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.552+0000 7fb65eb29640 1 -- 172.21.15.2:0/1239350050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb658101600 msgr2=0x7fb658197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:26.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.552+0000 7fb65eb29640 1 --2- 172.21.15.2:0/1239350050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb658101600 0x7fb658197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.552+0000 7fb65eb29640 1 -- 172.21.15.2:0/1239350050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb658100410 msgr2=0x7fb6581975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:26.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.552+0000 7fb65eb29640 1 --2- 172.21.15.2:0/1239350050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb658100410 0x7fb6581975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.552+0000 7fb65eb29640 1 -- 172.21.15.2:0/1239350050 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb650009560 con 0x7fb658101fc0 2024-03-15T12:56:26.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.552+0000 7fb65db27640 1 --2- 172.21.15.2:0/1239350050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb658101600 0x7fb658197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:56:26.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.552+0000 7fb65eb29640 1 --2- 172.21.15.2:0/1239350050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb658101fc0 0x7fb65819eb90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb64800eaa0 tx=0x7fb64800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:26.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.552+0000 7fb6577fe640 1 -- 172.21.15.2:0/1239350050 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64800cdb0 con 0x7fb658101fc0 2024-03-15T12:56:26.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.552+0000 7fb66058c640 1 -- 172.21.15.2:0/1239350050 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65819f6e0 con 0x7fb658101fc0 2024-03-15T12:56:26.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.552+0000 7fb6577fe640 1 -- 172.21.15.2:0/1239350050 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb648005020 con 0x7fb658101fc0 2024-03-15T12:56:26.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.552+0000 7fb6577fe640 1 -- 172.21.15.2:0/1239350050 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6480116d0 con 0x7fb658101fc0 2024-03-15T12:56:26.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.552+0000 7fb66058c640 1 -- 172.21.15.2:0/1239350050 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65819fc40 con 0x7fb658101fc0 2024-03-15T12:56:26.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.553+0000 7fb66058c640 1 -- 172.21.15.2:0/1239350050 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65810bdf0 con 0x7fb658101fc0 2024-03-15T12:56:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.554+0000 7fb6577fe640 1 -- 172.21.15.2:0/1239350050 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fb648005190 con 0x7fb658101fc0 2024-03-15T12:56:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.555+0000 7fb6577fe640 1 --2- 172.21.15.2:0/1239350050 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb6400a1ab0 0x7fb6400a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.555+0000 7fb65e328640 1 --2- 172.21.15.2:0/1239350050 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb6400a1ab0 0x7fb6400a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.555+0000 7fb6577fe640 1 -- 172.21.15.2:0/1239350050 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb6480cd7f0 con 0x7fb658101fc0 2024-03-15T12:56:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.555+0000 7fb65e328640 1 --2- 172.21.15.2:0/1239350050 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb6400a1ab0 0x7fb6400a3f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb65002c740 tx=0x7fb650005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.560+0000 7fb6577fe640 1 -- 172.21.15.2:0/1239350050 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb6480967c0 con 0x7fb658101fc0 2024-03-15T12:56:26.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.782+0000 7fb66058c640 1 -- 172.21.15.2:0/1239350050 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6581a0140 con 0x7fb658101fc0 2024-03-15T12:56:26.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.782+0000 7fb6577fe640 1 -- 172.21.15.2:0/1239350050 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11318 (secure 0 0 0) 0x7fb648099e80 con 0x7fb658101fc0 2024-03-15T12:56:26.769 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:26.769 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T12:56:25.830585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24539,"qdb_cluster":[24539]},"id":1}]} 2024-03-15T12:56:26.769 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-15T12:56:26.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.784+0000 7fb66058c640 1 -- 172.21.15.2:0/1239350050 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb6400a1ab0 msgr2=0x7fb6400a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:26.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.784+0000 7fb66058c640 1 --2- 172.21.15.2:0/1239350050 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb6400a1ab0 0x7fb6400a3f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb65002c740 tx=0x7fb650005fd0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.784+0000 7fb66058c640 1 -- 172.21.15.2:0/1239350050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb658101fc0 msgr2=0x7fb65819eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:26.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.784+0000 7fb66058c640 1 --2- 172.21.15.2:0/1239350050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb658101fc0 0x7fb65819eb90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb64800eaa0 tx=0x7fb64800ef60 comp rx=0 tx=0).stop 2024-03-15T12:56:26.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.784+0000 7fb66058c640 1 -- 172.21.15.2:0/1239350050 shutdown_connections 2024-03-15T12:56:26.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.784+0000 7fb66058c640 1 --2- 172.21.15.2:0/1239350050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb658101fc0 0x7fb65819eb90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.784+0000 7fb66058c640 1 --2- 172.21.15.2:0/1239350050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb658101600 0x7fb658197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.784+0000 7fb66058c640 1 --2- 172.21.15.2:0/1239350050 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb6400a1ab0 0x7fb6400a3f70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.784+0000 7fb66058c640 1 --2- 172.21.15.2:0/1239350050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb658100410 0x7fb6581975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.784+0000 7fb66058c640 1 -- 172.21.15.2:0/1239350050 >> 172.21.15.2:0/1239350050 conn(0x7fb6580fb980 msgr2=0x7fb6580fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:26.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.784+0000 7fb66058c640 1 -- 172.21.15.2:0/1239350050 shutdown_connections 2024-03-15T12:56:26.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.784+0000 7fb66058c640 1 -- 172.21.15.2:0/1239350050 wait complete. 2024-03-15T12:56:26.780 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:26 smithi002 ceph-mon[32060]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-15T12:56:26.781 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:26 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3552197270' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-15T12:56:26.781 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:26 smithi002 ceph-mon[32060]: osdmap e88: 12 total, 12 up, 12 in 2024-03-15T12:56:26.781 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:26 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/632580841' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-15T12:56:26.782 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:56:26.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:26 smithi184 ceph-mon[35993]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-15T12:56:26.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:26 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3552197270' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-15T12:56:26.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:26 smithi184 ceph-mon[35993]: osdmap e88: 12 total, 12 up, 12 in 2024-03-15T12:56:26.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:26 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/632580841' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-15T12:56:26.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:26 smithi149 ceph-mon[35927]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-15T12:56:26.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:26 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3552197270' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-15T12:56:26.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:26 smithi149 ceph-mon[35927]: osdmap e88: 12 total, 12 up, 12 in 2024-03-15T12:56:26.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:26 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/632580841' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-15T12:56:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.898+0000 7fd5466d0640 1 -- 172.21.15.2:0/158794650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5401071f0 msgr2=0x7fd540109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.898+0000 7fd5466d0640 1 --2- 172.21.15.2:0/158794650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5401071f0 0x7fd540109640 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd538009ca0 tx=0x7fd53802c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.898+0000 7fd5466d0640 1 -- 172.21.15.2:0/158794650 shutdown_connections 2024-03-15T12:56:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.898+0000 7fd5466d0640 1 --2- 172.21.15.2:0/158794650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5401071f0 0x7fd540109640 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.898+0000 7fd5466d0640 1 --2- 172.21.15.2:0/158794650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5401048c0 0x7fd540106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.898+0000 7fd5466d0640 1 --2- 172.21.15.2:0/158794650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd540101f90 0x7fd540104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.898+0000 7fd5466d0640 1 -- 172.21.15.2:0/158794650 >> 172.21.15.2:0/158794650 conn(0x7fd5400fb980 msgr2=0x7fd5400fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.898+0000 7fd5466d0640 1 -- 172.21.15.2:0/158794650 shutdown_connections 2024-03-15T12:56:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.898+0000 7fd5466d0640 1 -- 172.21.15.2:0/158794650 wait complete. 2024-03-15T12:56:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.899+0000 7fd5466d0640 1 Processor -- start 2024-03-15T12:56:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.899+0000 7fd5466d0640 1 -- start start 2024-03-15T12:56:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.899+0000 7fd5466d0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd540101f90 0x7fd540101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.899+0000 7fd5466d0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5401048c0 0x7fd5400fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.899+0000 7fd5466d0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5401071f0 0x7fd540100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.899+0000 7fd5466d0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd540100be0 con 0x7fd5401071f0 2024-03-15T12:56:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.899+0000 7fd5466d0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd540100d50 con 0x7fd5401048c0 2024-03-15T12:56:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.899+0000 7fd5466d0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5401a1520 con 0x7fd540101f90 2024-03-15T12:56:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.899+0000 7fd544c6d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5401071f0 0x7fd540100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.899+0000 7fd544c6d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5401071f0 0x7fd540100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52196/0 (socket says 172.21.15.2:52196) 2024-03-15T12:56:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.899+0000 7fd544c6d640 1 -- 172.21.15.2:0/3864888370 learned_addr learned my addr 172.21.15.2:0/3864888370 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.899+0000 7fd53f7fe640 1 --2- 172.21.15.2:0/3864888370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5401048c0 0x7fd5400fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.900+0000 7fd53ffff640 1 --2- 172.21.15.2:0/3864888370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd540101f90 0x7fd540101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.900+0000 7fd544c6d640 1 -- 172.21.15.2:0/3864888370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd540101f90 msgr2=0x7fd540101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.900+0000 7fd544c6d640 1 --2- 172.21.15.2:0/3864888370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd540101f90 0x7fd540101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.900+0000 7fd544c6d640 1 -- 172.21.15.2:0/3864888370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5401048c0 msgr2=0x7fd5400fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.900+0000 7fd544c6d640 1 --2- 172.21.15.2:0/3864888370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5401048c0 0x7fd5400fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.900+0000 7fd544c6d640 1 -- 172.21.15.2:0/3864888370 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd530009490 con 0x7fd5401071f0 2024-03-15T12:56:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.900+0000 7fd53ffff640 1 --2- 172.21.15.2:0/3864888370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd540101f90 0x7fd540101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T12:56:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.900+0000 7fd544c6d640 1 --2- 172.21.15.2:0/3864888370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5401071f0 0x7fd540100480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd538005cc0 tx=0x7fd538032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.900+0000 7fd53d7fa640 1 -- 172.21.15.2:0/3864888370 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538003970 con 0x7fd5401071f0 2024-03-15T12:56:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.900+0000 7fd53d7fa640 1 -- 172.21.15.2:0/3864888370 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd538004040 con 0x7fd5401071f0 2024-03-15T12:56:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.900+0000 7fd5466d0640 1 -- 172.21.15.2:0/3864888370 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd538009900 con 0x7fd5401071f0 2024-03-15T12:56:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.900+0000 7fd53d7fa640 1 -- 172.21.15.2:0/3864888370 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5380406b0 con 0x7fd5401071f0 2024-03-15T12:56:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.900+0000 7fd5466d0640 1 -- 172.21.15.2:0/3864888370 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5401a1a90 con 0x7fd5401071f0 2024-03-15T12:56:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.901+0000 7fd5466d0640 1 -- 172.21.15.2:0/3864888370 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5400fd470 con 0x7fd5401071f0 2024-03-15T12:56:26.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.902+0000 7fd53d7fa640 1 -- 172.21.15.2:0/3864888370 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fd53800a040 con 0x7fd5401071f0 2024-03-15T12:56:26.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.902+0000 7fd53d7fa640 1 --2- 172.21.15.2:0/3864888370 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd5280a1a30 0x7fd5280a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:26.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.903+0000 7fd53ffff640 1 --2- 172.21.15.2:0/3864888370 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd5280a1a30 0x7fd5280a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:26.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.903+0000 7fd53d7fa640 1 -- 172.21.15.2:0/3864888370 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd5380f0830 con 0x7fd5401071f0 2024-03-15T12:56:26.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.903+0000 7fd53ffff640 1 --2- 172.21.15.2:0/3864888370 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd5280a1a30 0x7fd5280a3ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd530009ce0 tx=0x7fd530005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:26.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:26.907+0000 7fd53d7fa640 1 -- 172.21.15.2:0/3864888370 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd5380b97a0 con 0x7fd5401071f0 2024-03-15T12:56:27.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.090+0000 7fd5466d0640 1 -- 172.21.15.2:0/3864888370 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd54002d000 con 0x7fd5401071f0 2024-03-15T12:56:27.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.091+0000 7fd53d7fa640 1 -- 172.21.15.2:0/3864888370 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24045 (secure 0 0 0) 0x7fd5380bce60 con 0x7fd5401071f0 2024-03-15T12:56:27.078 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:27.078 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":89,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:56:25.830426+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:56:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.093+0000 7fd5466d0640 1 -- 172.21.15.2:0/3864888370 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd5280a1a30 msgr2=0x7fd5280a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.093+0000 7fd5466d0640 1 --2- 172.21.15.2:0/3864888370 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd5280a1a30 0x7fd5280a3ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd530009ce0 tx=0x7fd530005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.093+0000 7fd5466d0640 1 -- 172.21.15.2:0/3864888370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5401071f0 msgr2=0x7fd540100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.093+0000 7fd5466d0640 1 --2- 172.21.15.2:0/3864888370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5401071f0 0x7fd540100480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd538005cc0 tx=0x7fd538032a10 comp rx=0 tx=0).stop 2024-03-15T12:56:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.093+0000 7fd5466d0640 1 -- 172.21.15.2:0/3864888370 shutdown_connections 2024-03-15T12:56:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.093+0000 7fd5466d0640 1 --2- 172.21.15.2:0/3864888370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5401071f0 0x7fd540100480 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.093+0000 7fd5466d0640 1 --2- 172.21.15.2:0/3864888370 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd5280a1a30 0x7fd5280a3ef0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.093+0000 7fd5466d0640 1 --2- 172.21.15.2:0/3864888370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5401048c0 0x7fd5400fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.093+0000 7fd5466d0640 1 --2- 172.21.15.2:0/3864888370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd540101f90 0x7fd540101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.093+0000 7fd5466d0640 1 -- 172.21.15.2:0/3864888370 >> 172.21.15.2:0/3864888370 conn(0x7fd5400fb980 msgr2=0x7fd540102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.093+0000 7fd5466d0640 1 -- 172.21.15.2:0/3864888370 shutdown_connections 2024-03-15T12:56:27.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.093+0000 7fd5466d0640 1 -- 172.21.15.2:0/3864888370 wait complete. 2024-03-15T12:56:27.091 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:56:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.207+0000 7f0a2bf2c640 1 -- 172.21.15.2:0/2462065545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a24101fc0 msgr2=0x7f0a24108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.207+0000 7f0a2bf2c640 1 --2- 172.21.15.2:0/2462065545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a24101fc0 0x7f0a24108850 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009ca0 tx=0x7f0a1c02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.207+0000 7f0a2bf2c640 1 -- 172.21.15.2:0/2462065545 shutdown_connections 2024-03-15T12:56:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.207+0000 7f0a2bf2c640 1 --2- 172.21.15.2:0/2462065545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a24101fc0 0x7f0a24108850 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.207+0000 7f0a2bf2c640 1 --2- 172.21.15.2:0/2462065545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a24101600 0x7f0a24101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.207+0000 7f0a2bf2c640 1 --2- 172.21.15.2:0/2462065545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a24100410 0x7f0a24100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.207+0000 7f0a2bf2c640 1 -- 172.21.15.2:0/2462065545 >> 172.21.15.2:0/2462065545 conn(0x7f0a240fb980 msgr2=0x7f0a240fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.207+0000 7f0a2bf2c640 1 -- 172.21.15.2:0/2462065545 shutdown_connections 2024-03-15T12:56:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.207+0000 7f0a2bf2c640 1 -- 172.21.15.2:0/2462065545 wait complete. 2024-03-15T12:56:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2bf2c640 1 Processor -- start 2024-03-15T12:56:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2bf2c640 1 -- start start 2024-03-15T12:56:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2bf2c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a24100410 0x7f0a2406a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2bf2c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a24101600 0x7f0a2419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2bf2c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a24101fc0 0x7f0a241a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2bf2c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a241a12c0 con 0x7f0a24101fc0 2024-03-15T12:56:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2bf2c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a241a1430 con 0x7f0a24100410 2024-03-15T12:56:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2bf2c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a241a15f0 con 0x7f0a24101600 2024-03-15T12:56:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2a4c9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a24101fc0 0x7f0a241a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2a4c9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a24101fc0 0x7f0a241a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52218/0 (socket says 172.21.15.2:52218) 2024-03-15T12:56:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2a4c9640 1 -- 172.21.15.2:0/1897955584 learned_addr learned my addr 172.21.15.2:0/1897955584 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a29cc8640 1 --2- 172.21.15.2:0/1897955584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a24100410 0x7f0a2406a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a294c7640 1 --2- 172.21.15.2:0/1897955584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a24101600 0x7f0a2419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2a4c9640 1 -- 172.21.15.2:0/1897955584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a24101600 msgr2=0x7f0a2419e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2a4c9640 1 --2- 172.21.15.2:0/1897955584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a24101600 0x7f0a2419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2a4c9640 1 -- 172.21.15.2:0/1897955584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a24100410 msgr2=0x7f0a2406a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2a4c9640 1 --2- 172.21.15.2:0/1897955584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a24100410 0x7f0a2406a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2a4c9640 1 -- 172.21.15.2:0/1897955584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a20009490 con 0x7f0a24101fc0 2024-03-15T12:56:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.208+0000 7f0a2a4c9640 1 --2- 172.21.15.2:0/1897955584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a24101fc0 0x7f0a241a0c00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c005cc0 tx=0x7f0a1c032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.209+0000 7f0a1affd640 1 -- 172.21.15.2:0/1897955584 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c034c90 con 0x7f0a24101fc0 2024-03-15T12:56:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.209+0000 7f0a1affd640 1 -- 172.21.15.2:0/1897955584 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0a1c003e40 con 0x7f0a24101fc0 2024-03-15T12:56:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.209+0000 7f0a2bf2c640 1 -- 172.21.15.2:0/1897955584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1c009900 con 0x7f0a24101fc0 2024-03-15T12:56:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.209+0000 7f0a1affd640 1 -- 172.21.15.2:0/1897955584 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c0048a0 con 0x7f0a24101fc0 2024-03-15T12:56:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.209+0000 7f0a2bf2c640 1 -- 172.21.15.2:0/1897955584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a241a1b80 con 0x7f0a24101fc0 2024-03-15T12:56:27.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.210+0000 7f0a2bf2c640 1 -- 172.21.15.2:0/1897955584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a24107730 con 0x7f0a24101fc0 2024-03-15T12:56:27.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.211+0000 7f0a1affd640 1 -- 172.21.15.2:0/1897955584 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f0a1c00a040 con 0x7f0a24101fc0 2024-03-15T12:56:27.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.211+0000 7f0a1affd640 1 --2- 172.21.15.2:0/1897955584 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0a100a1ae0 0x7f0a100a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:27.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.211+0000 7f0a29cc8640 1 --2- 172.21.15.2:0/1897955584 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0a100a1ae0 0x7f0a100a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:27.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.211+0000 7f0a1affd640 1 -- 172.21.15.2:0/1897955584 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f0a1c0f0760 con 0x7f0a24101fc0 2024-03-15T12:56:27.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.212+0000 7f0a29cc8640 1 --2- 172.21.15.2:0/1897955584 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0a100a1ae0 0x7f0a100a3fa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a24101460 tx=0x7f0a20005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:27.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.215+0000 7f0a1affd640 1 -- 172.21.15.2:0/1897955584 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0a1c0b9850 con 0x7f0a24101fc0 2024-03-15T12:56:27.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.399+0000 7f0a2bf2c640 1 -- 172.21.15.2:0/1897955584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a2410bff0 con 0x7f0a24101fc0 2024-03-15T12:56:27.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.401+0000 7f0a1affd640 1 -- 172.21.15.2:0/1897955584 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24045 (secure 0 0 0) 0x7f0a1c044090 con 0x7f0a24101fc0 2024-03-15T12:56:27.387 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:27.387 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":89,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:56:25.830426+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:56:27.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.403+0000 7f0a2bf2c640 1 -- 172.21.15.2:0/1897955584 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0a100a1ae0 msgr2=0x7f0a100a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:27.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.403+0000 7f0a2bf2c640 1 --2- 172.21.15.2:0/1897955584 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0a100a1ae0 0x7f0a100a3fa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a24101460 tx=0x7f0a20005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:27.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.403+0000 7f0a2bf2c640 1 -- 172.21.15.2:0/1897955584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a24101fc0 msgr2=0x7f0a241a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:27.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.403+0000 7f0a2bf2c640 1 --2- 172.21.15.2:0/1897955584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a24101fc0 0x7f0a241a0c00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c005cc0 tx=0x7f0a1c032a10 comp rx=0 tx=0).stop 2024-03-15T12:56:27.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.403+0000 7f0a2bf2c640 1 -- 172.21.15.2:0/1897955584 shutdown_connections 2024-03-15T12:56:27.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.403+0000 7f0a2bf2c640 1 --2- 172.21.15.2:0/1897955584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a24101fc0 0x7f0a241a0c00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.403+0000 7f0a2bf2c640 1 --2- 172.21.15.2:0/1897955584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a24101600 0x7f0a2419e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.403+0000 7f0a2bf2c640 1 --2- 172.21.15.2:0/1897955584 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0a100a1ae0 0x7f0a100a3fa0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.403+0000 7f0a2bf2c640 1 --2- 172.21.15.2:0/1897955584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a24100410 0x7f0a2406a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.403+0000 7f0a2bf2c640 1 -- 172.21.15.2:0/1897955584 >> 172.21.15.2:0/1897955584 conn(0x7f0a240fb980 msgr2=0x7f0a24104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:27.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.403+0000 7f0a2bf2c640 1 -- 172.21.15.2:0/1897955584 shutdown_connections 2024-03-15T12:56:27.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.403+0000 7f0a2bf2c640 1 -- 172.21.15.2:0/1897955584 wait complete. 2024-03-15T12:56:27.416 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:56:27.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.532+0000 7fa10d2a7640 1 -- 172.21.15.2:0/203162019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa108100430 msgr2=0x7fa108100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.532+0000 7fa10d2a7640 1 --2- 172.21.15.2:0/203162019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa108100430 0x7fa108100850 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009900 tx=0x7fa0f802bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.533+0000 7fa10d2a7640 1 -- 172.21.15.2:0/203162019 shutdown_connections 2024-03-15T12:56:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.533+0000 7fa10d2a7640 1 --2- 172.21.15.2:0/203162019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa108101fe0 0x7fa108108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.533+0000 7fa10d2a7640 1 --2- 172.21.15.2:0/203162019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa108101620 0x7fa108101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.533+0000 7fa10d2a7640 1 --2- 172.21.15.2:0/203162019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa108100430 0x7fa108100850 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.533+0000 7fa10d2a7640 1 -- 172.21.15.2:0/203162019 >> 172.21.15.2:0/203162019 conn(0x7fa1080fb9a0 msgr2=0x7fa1080fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.533+0000 7fa10d2a7640 1 -- 172.21.15.2:0/203162019 shutdown_connections 2024-03-15T12:56:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.533+0000 7fa10d2a7640 1 -- 172.21.15.2:0/203162019 wait complete. 2024-03-15T12:56:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.533+0000 7fa10d2a7640 1 Processor -- start 2024-03-15T12:56:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.533+0000 7fa10d2a7640 1 -- start start 2024-03-15T12:56:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.533+0000 7fa10d2a7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa108100430 0x7fa1081964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.533+0000 7fa10d2a7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa108101620 0x7fa108196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.533+0000 7fa10d2a7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa108101fe0 0x7fa10819daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.533+0000 7fa10d2a7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10819dfe0 con 0x7fa108101620 2024-03-15T12:56:27.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.533+0000 7fa10d2a7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10819e150 con 0x7fa108100430 2024-03-15T12:56:27.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.533+0000 7fa10d2a7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10819e310 con 0x7fa108101fe0 2024-03-15T12:56:27.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa0fe7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa108101620 0x7fa108196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:27.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa106ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa108100430 0x7fa1081964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:27.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa0fe7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa108101620 0x7fa108196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52236/0 (socket says 172.21.15.2:52236) 2024-03-15T12:56:27.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa0fe7fc640 1 -- 172.21.15.2:0/2675365090 learned_addr learned my addr 172.21.15.2:0/2675365090 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:27.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa1077fe640 1 --2- 172.21.15.2:0/2675365090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa108101fe0 0x7fa10819daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:27.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa0fe7fc640 1 -- 172.21.15.2:0/2675365090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa108101fe0 msgr2=0x7fa10819daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:27.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa0fe7fc640 1 --2- 172.21.15.2:0/2675365090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa108101fe0 0x7fa10819daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa0fe7fc640 1 -- 172.21.15.2:0/2675365090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa108100430 msgr2=0x7fa1081964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:27.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa0fe7fc640 1 --2- 172.21.15.2:0/2675365090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa108100430 0x7fa1081964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa0fe7fc640 1 -- 172.21.15.2:0/2675365090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f8009560 con 0x7fa108101620 2024-03-15T12:56:27.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa0fe7fc640 1 --2- 172.21.15.2:0/2675365090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa108101620 0x7fa108196a30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000d950 tx=0x7fa0f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:27.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa104ff9640 1 -- 172.21.15.2:0/2675365090 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0009a30 con 0x7fa108101620 2024-03-15T12:56:27.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa104ff9640 1 -- 172.21.15.2:0/2675365090 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa0f0009b90 con 0x7fa108101620 2024-03-15T12:56:27.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa10d2a7640 1 -- 172.21.15.2:0/2675365090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10819e5f0 con 0x7fa108101620 2024-03-15T12:56:27.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa104ff9640 1 -- 172.21.15.2:0/2675365090 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00047a0 con 0x7fa108101620 2024-03-15T12:56:27.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.534+0000 7fa10d2a7640 1 -- 172.21.15.2:0/2675365090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10819eb00 con 0x7fa108101620 2024-03-15T12:56:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.535+0000 7fa10d2a7640 1 -- 172.21.15.2:0/2675365090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1081077f0 con 0x7fa108101620 2024-03-15T12:56:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.536+0000 7fa104ff9640 1 -- 172.21.15.2:0/2675365090 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fa0f0009d00 con 0x7fa108101620 2024-03-15T12:56:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.537+0000 7fa104ff9640 1 --2- 172.21.15.2:0/2675365090 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa0f40a1a30 0x7fa0f40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.537+0000 7fa106ffd640 1 --2- 172.21.15.2:0/2675365090 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa0f40a1a30 0x7fa0f40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.537+0000 7fa104ff9640 1 -- 172.21.15.2:0/2675365090 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa0f00cd120 con 0x7fa108101620 2024-03-15T12:56:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.537+0000 7fa106ffd640 1 --2- 172.21.15.2:0/2675365090 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa0f40a1a30 0x7fa0f40a3ef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa0f802c740 tx=0x7fa0f8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:27.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.541+0000 7fa104ff9640 1 -- 172.21.15.2:0/2675365090 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa0f00960f0 con 0x7fa108101620 2024-03-15T12:56:27.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.762+0000 7fa10d2a7640 1 -- 172.21.15.2:0/2675365090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa10810be10 con 0x7fa108101620 2024-03-15T12:56:27.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.763+0000 7fa104ff9640 1 -- 172.21.15.2:0/2675365090 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11318 (secure 0 0 0) 0x7fa0f00997b0 con 0x7fa108101620 2024-03-15T12:56:27.749 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:27.749 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T12:56:25.830585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24539,"qdb_cluster":[24539]},"id":1}]} 2024-03-15T12:56:27.750 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-15T12:56:27.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.765+0000 7fa10d2a7640 1 -- 172.21.15.2:0/2675365090 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa0f40a1a30 msgr2=0x7fa0f40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:27.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.765+0000 7fa10d2a7640 1 --2- 172.21.15.2:0/2675365090 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa0f40a1a30 0x7fa0f40a3ef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa0f802c740 tx=0x7fa0f8005fd0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.765+0000 7fa10d2a7640 1 -- 172.21.15.2:0/2675365090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa108101620 msgr2=0x7fa108196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:27.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.765+0000 7fa10d2a7640 1 --2- 172.21.15.2:0/2675365090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa108101620 0x7fa108196a30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000d950 tx=0x7fa0f000de10 comp rx=0 tx=0).stop 2024-03-15T12:56:27.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.765+0000 7fa10d2a7640 1 -- 172.21.15.2:0/2675365090 shutdown_connections 2024-03-15T12:56:27.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.765+0000 7fa10d2a7640 1 --2- 172.21.15.2:0/2675365090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa108101fe0 0x7fa10819daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.765+0000 7fa10d2a7640 1 --2- 172.21.15.2:0/2675365090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa108101620 0x7fa108196a30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.765+0000 7fa10d2a7640 1 --2- 172.21.15.2:0/2675365090 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa0f40a1a30 0x7fa0f40a3ef0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.765+0000 7fa10d2a7640 1 --2- 172.21.15.2:0/2675365090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa108100430 0x7fa1081964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.765+0000 7fa10d2a7640 1 -- 172.21.15.2:0/2675365090 >> 172.21.15.2:0/2675365090 conn(0x7fa1080fb9a0 msgr2=0x7fa1081049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:27.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.765+0000 7fa10d2a7640 1 -- 172.21.15.2:0/2675365090 shutdown_connections 2024-03-15T12:56:27.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.765+0000 7fa10d2a7640 1 -- 172.21.15.2:0/2675365090 wait complete. 2024-03-15T12:56:27.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:27 smithi002 ceph-mon[32060]: osdmap e89: 12 total, 12 up, 12 in 2024-03-15T12:56:27.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:27 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/632580841' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-15T12:56:27.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:27 smithi002 ceph-mon[32060]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-15T12:56:27.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:27 smithi002 ceph-mon[32060]: pgmap v293: 129 pgs: 11 creating+peering, 101 active+clean, 17 unknown; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-03-15T12:56:27.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:27 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1239350050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:27.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:27 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3864888370' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:27.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:27 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1897955584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:27.763 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:56:27.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:27 smithi184 ceph-mon[35993]: osdmap e89: 12 total, 12 up, 12 in 2024-03-15T12:56:27.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:27 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/632580841' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-15T12:56:27.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:27 smithi184 ceph-mon[35993]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-15T12:56:27.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:27 smithi184 ceph-mon[35993]: pgmap v293: 129 pgs: 11 creating+peering, 101 active+clean, 17 unknown; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-03-15T12:56:27.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:27 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1239350050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:27.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:27 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3864888370' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:27.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:27 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1897955584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:27.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:27 smithi149 ceph-mon[35927]: osdmap e89: 12 total, 12 up, 12 in 2024-03-15T12:56:27.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:27 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/632580841' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-15T12:56:27.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:27 smithi149 ceph-mon[35927]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-15T12:56:27.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:27 smithi149 ceph-mon[35927]: pgmap v293: 129 pgs: 11 creating+peering, 101 active+clean, 17 unknown; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-03-15T12:56:27.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:27 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1239350050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:27.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:27 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3864888370' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:27.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:27 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1897955584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:27.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.879+0000 7fb102ab5640 1 -- 172.21.15.2:0/3430865581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0fc102040 msgr2=0x7fb0fc1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:27.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.879+0000 7fb102ab5640 1 --2- 172.21.15.2:0/3430865581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0fc102040 0x7fb0fc1088d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009ca0 tx=0x7fb0f402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.879+0000 7fb102ab5640 1 -- 172.21.15.2:0/3430865581 shutdown_connections 2024-03-15T12:56:27.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.879+0000 7fb102ab5640 1 --2- 172.21.15.2:0/3430865581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0fc102040 0x7fb0fc1088d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.879+0000 7fb102ab5640 1 --2- 172.21.15.2:0/3430865581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0fc101680 0x7fb0fc101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.879+0000 7fb102ab5640 1 --2- 172.21.15.2:0/3430865581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0fc100490 0x7fb0fc1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.879+0000 7fb102ab5640 1 -- 172.21.15.2:0/3430865581 >> 172.21.15.2:0/3430865581 conn(0x7fb0fc0fba20 msgr2=0x7fb0fc0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:27.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.879+0000 7fb102ab5640 1 -- 172.21.15.2:0/3430865581 shutdown_connections 2024-03-15T12:56:27.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.879+0000 7fb102ab5640 1 -- 172.21.15.2:0/3430865581 wait complete. 2024-03-15T12:56:27.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb102ab5640 1 Processor -- start 2024-03-15T12:56:27.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb102ab5640 1 -- start start 2024-03-15T12:56:27.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb102ab5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0fc100490 0x7fb0fc06a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:27.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb102ab5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0fc101680 0x7fb0fc19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:27.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb102ab5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0fc102040 0x7fb0fc19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:27.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb102ab5640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc19f0a0 con 0x7fb0fc102040 2024-03-15T12:56:27.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb102ab5640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc19f210 con 0x7fb0fc101680 2024-03-15T12:56:27.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb102ab5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc19f3d0 con 0x7fb0fc100490 2024-03-15T12:56:27.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb101052640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0fc102040 0x7fb0fc19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:27.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb101052640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0fc102040 0x7fb0fc19e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52242/0 (socket says 172.21.15.2:52242) 2024-03-15T12:56:27.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb101052640 1 -- 172.21.15.2:0/1443304443 learned_addr learned my addr 172.21.15.2:0/1443304443 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:27.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb0fbfff640 1 --2- 172.21.15.2:0/1443304443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0fc101680 0x7fb0fc19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:27.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb100851640 1 --2- 172.21.15.2:0/1443304443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0fc100490 0x7fb0fc06a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:27.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb101052640 1 -- 172.21.15.2:0/1443304443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0fc100490 msgr2=0x7fb0fc06a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:27.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb101052640 1 --2- 172.21.15.2:0/1443304443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0fc100490 0x7fb0fc06a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb101052640 1 -- 172.21.15.2:0/1443304443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0fc101680 msgr2=0x7fb0fc19c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:27.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb101052640 1 --2- 172.21.15.2:0/1443304443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0fc101680 0x7fb0fc19c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:27.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb101052640 1 -- 172.21.15.2:0/1443304443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f4009900 con 0x7fb0fc102040 2024-03-15T12:56:27.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb101052640 1 --2- 172.21.15.2:0/1443304443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0fc102040 0x7fb0fc19e910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4005e30 tx=0x7fb0f4032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:27.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb0f9ffb640 1 -- 172.21.15.2:0/1443304443 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4003970 con 0x7fb0fc102040 2024-03-15T12:56:27.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb0f9ffb640 1 -- 172.21.15.2:0/1443304443 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb0f4004040 con 0x7fb0fc102040 2024-03-15T12:56:27.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.880+0000 7fb102ab5640 1 -- 172.21.15.2:0/1443304443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc19f650 con 0x7fb0fc102040 2024-03-15T12:56:27.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.881+0000 7fb0f9ffb640 1 -- 172.21.15.2:0/1443304443 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f40406b0 con 0x7fb0fc102040 2024-03-15T12:56:27.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.881+0000 7fb102ab5640 1 -- 172.21.15.2:0/1443304443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc19fbd0 con 0x7fb0fc102040 2024-03-15T12:56:27.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.882+0000 7fb102ab5640 1 -- 172.21.15.2:0/1443304443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc1077b0 con 0x7fb0fc102040 2024-03-15T12:56:27.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.883+0000 7fb0f9ffb640 1 -- 172.21.15.2:0/1443304443 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fb0f400a040 con 0x7fb0fc102040 2024-03-15T12:56:27.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.883+0000 7fb0f9ffb640 1 --2- 172.21.15.2:0/1443304443 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb0ec0a1ab0 0x7fb0ec0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:27.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.883+0000 7fb100851640 1 --2- 172.21.15.2:0/1443304443 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb0ec0a1ab0 0x7fb0ec0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:27.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.883+0000 7fb0f9ffb640 1 -- 172.21.15.2:0/1443304443 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb0f40f0720 con 0x7fb0fc102040 2024-03-15T12:56:27.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.884+0000 7fb100851640 1 --2- 172.21.15.2:0/1443304443 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb0ec0a1ab0 0x7fb0ec0a3f70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc1014e0 tx=0x7fb0f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:27.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:27.888+0000 7fb0f9ffb640 1 -- 172.21.15.2:0/1443304443 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb0f40b96f0 con 0x7fb0fc102040 2024-03-15T12:56:28.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.070+0000 7fb102ab5640 1 -- 172.21.15.2:0/1443304443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0fc02d000 con 0x7fb0fc102040 2024-03-15T12:56:28.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.072+0000 7fb0f9ffb640 1 -- 172.21.15.2:0/1443304443 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24045 (secure 0 0 0) 0x7fb0f40bcdb0 con 0x7fb0fc102040 2024-03-15T12:56:28.059 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:28.059 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":89,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:56:25.830426+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:56:28.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.074+0000 7fb102ab5640 1 -- 172.21.15.2:0/1443304443 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb0ec0a1ab0 msgr2=0x7fb0ec0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:28.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.074+0000 7fb102ab5640 1 --2- 172.21.15.2:0/1443304443 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb0ec0a1ab0 0x7fb0ec0a3f70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc1014e0 tx=0x7fb0f0005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:28.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.074+0000 7fb102ab5640 1 -- 172.21.15.2:0/1443304443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0fc102040 msgr2=0x7fb0fc19e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:28.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.074+0000 7fb102ab5640 1 --2- 172.21.15.2:0/1443304443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0fc102040 0x7fb0fc19e910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4005e30 tx=0x7fb0f4032a10 comp rx=0 tx=0).stop 2024-03-15T12:56:28.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.074+0000 7fb102ab5640 1 -- 172.21.15.2:0/1443304443 shutdown_connections 2024-03-15T12:56:28.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.074+0000 7fb102ab5640 1 --2- 172.21.15.2:0/1443304443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0fc102040 0x7fb0fc19e910 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.074+0000 7fb102ab5640 1 --2- 172.21.15.2:0/1443304443 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb0ec0a1ab0 0x7fb0ec0a3f70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.074+0000 7fb102ab5640 1 --2- 172.21.15.2:0/1443304443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0fc101680 0x7fb0fc19c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.074+0000 7fb102ab5640 1 --2- 172.21.15.2:0/1443304443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0fc100490 0x7fb0fc06a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.074+0000 7fb102ab5640 1 -- 172.21.15.2:0/1443304443 >> 172.21.15.2:0/1443304443 conn(0x7fb0fc0fba20 msgr2=0x7fb0fc104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:28.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.074+0000 7fb102ab5640 1 -- 172.21.15.2:0/1443304443 shutdown_connections 2024-03-15T12:56:28.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.074+0000 7fb102ab5640 1 -- 172.21.15.2:0/1443304443 wait complete. 2024-03-15T12:56:28.074 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-15T12:56:20.559317+0000', 'modified': '2024-03-15T12:56:25.830585+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24539}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.149:6839/1715135128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 1715135128}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 1715135128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24539, 'qdb_cluster': [24539]} 2024-03-15T12:56:28.074 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-15T12:56:28.074 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-15T12:56:28.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.190+0000 7fd18b5d7640 1 -- 172.21.15.2:0/2056681374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1841048c0 msgr2=0x7fd184106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:28.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.190+0000 7fd18b5d7640 1 --2- 172.21.15.2:0/2056681374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1841048c0 0x7fd184106cb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd178009900 tx=0x7fd17802bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:28.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.190+0000 7fd18b5d7640 1 -- 172.21.15.2:0/2056681374 shutdown_connections 2024-03-15T12:56:28.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.190+0000 7fd18b5d7640 1 --2- 172.21.15.2:0/2056681374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1841071f0 0x7fd184109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.190+0000 7fd18b5d7640 1 --2- 172.21.15.2:0/2056681374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1841048c0 0x7fd184106cb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.190+0000 7fd18b5d7640 1 --2- 172.21.15.2:0/2056681374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd184101f90 0x7fd184104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.190+0000 7fd18b5d7640 1 -- 172.21.15.2:0/2056681374 >> 172.21.15.2:0/2056681374 conn(0x7fd1840fb980 msgr2=0x7fd1840fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:28.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.191+0000 7fd18b5d7640 1 -- 172.21.15.2:0/2056681374 shutdown_connections 2024-03-15T12:56:28.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.191+0000 7fd18b5d7640 1 -- 172.21.15.2:0/2056681374 wait complete. 2024-03-15T12:56:28.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.191+0000 7fd18b5d7640 1 Processor -- start 2024-03-15T12:56:28.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.191+0000 7fd18b5d7640 1 -- start start 2024-03-15T12:56:28.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.191+0000 7fd18b5d7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd184101f90 0x7fd184197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.191+0000 7fd18b5d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1841048c0 0x7fd184197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.191+0000 7fd18b5d7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1841071f0 0x7fd18419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.191+0000 7fd18b5d7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18419f140 con 0x7fd1841048c0 2024-03-15T12:56:28.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.191+0000 7fd18b5d7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18419f2b0 con 0x7fd184101f90 2024-03-15T12:56:28.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.191+0000 7fd18b5d7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18419f470 con 0x7fd1841071f0 2024-03-15T12:56:28.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.191+0000 7fd188b72640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1841048c0 0x7fd184197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.191+0000 7fd188b72640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1841048c0 0x7fd184197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50646/0 (socket says 172.21.15.2:50646) 2024-03-15T12:56:28.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.191+0000 7fd188b72640 1 -- 172.21.15.2:0/287062169 learned_addr learned my addr 172.21.15.2:0/287062169 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:28.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.191+0000 7fd189373640 1 --2- 172.21.15.2:0/287062169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd184101f90 0x7fd184197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.192+0000 7fd188b72640 1 -- 172.21.15.2:0/287062169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1841071f0 msgr2=0x7fd18419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:28.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.192+0000 7fd188b72640 1 --2- 172.21.15.2:0/287062169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1841071f0 0x7fd18419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.192+0000 7fd188b72640 1 -- 172.21.15.2:0/287062169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd184101f90 msgr2=0x7fd184197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:28.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.192+0000 7fd188b72640 1 --2- 172.21.15.2:0/287062169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd184101f90 0x7fd184197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.192+0000 7fd188b72640 1 -- 172.21.15.2:0/287062169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c009490 con 0x7fd1841048c0 2024-03-15T12:56:28.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.192+0000 7fd188b72640 1 --2- 172.21.15.2:0/287062169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1841048c0 0x7fd184197b90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd178005ab0 tx=0x7fd178004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:28.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.192+0000 7fd1827fc640 1 -- 172.21.15.2:0/287062169 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178004250 con 0x7fd1841048c0 2024-03-15T12:56:28.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.192+0000 7fd1827fc640 1 -- 172.21.15.2:0/287062169 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd1780043b0 con 0x7fd1841048c0 2024-03-15T12:56:28.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.192+0000 7fd18b5d7640 1 -- 172.21.15.2:0/287062169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd178009560 con 0x7fd1841048c0 2024-03-15T12:56:28.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.192+0000 7fd1827fc640 1 -- 172.21.15.2:0/287062169 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178003db0 con 0x7fd1841048c0 2024-03-15T12:56:28.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.192+0000 7fd18b5d7640 1 -- 172.21.15.2:0/287062169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18419fa70 con 0x7fd1841048c0 2024-03-15T12:56:28.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.194+0000 7fd1827fc640 1 -- 172.21.15.2:0/287062169 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fd1780035f0 con 0x7fd1841048c0 2024-03-15T12:56:28.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd18e1b4740 1 Processor -- start 2024-03-15T12:56:28.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd18e1b4740 1 -- start start 2024-03-15T12:56:28.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd18e1b4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd18e1b4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbec970 0x55fa5fbf1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd18e1b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbf3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd18e1b4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5fba7cd0 con 0x55fa5fbec970 2024-03-15T12:56:28.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd18e1b4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5fb5af20 con 0x55fa5fbebf80 2024-03-15T12:56:28.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd18e1b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5fb58450 con 0x55fa5fbf1ae0 2024-03-15T12:56:28.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd1827fc640 1 --2- 172.21.15.2:0/287062169 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd16c0a1a30 0x7fd16c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd189373640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbec970 0x55fa5fbf1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd189373640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbec970 0x55fa5fbf1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50656/0 (socket says 172.21.15.2:50656) 2024-03-15T12:56:28.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd189373640 1 -- 172.21.15.2:0/1785969532 learned_addr learned my addr 172.21.15.2:0/1785969532 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:28.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd189b74640 1 --2- 172.21.15.2:0/1785969532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd189b74640 1 --2- 172.21.15.2:0/287062169 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd16c0a1a30 0x7fd16c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd1827fc640 1 -- 172.21.15.2:0/287062169 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd1780ef8c0 con 0x7fd1841048c0 2024-03-15T12:56:28.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd188b72640 1 --2- 172.21.15.2:0/1785969532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbf3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd189373640 1 -- 172.21.15.2:0/1785969532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbf1ae0 msgr2=0x55fa5fbf3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:56:28.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd189373640 1 --2- 172.21.15.2:0/1785969532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbf3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd189373640 1 -- 172.21.15.2:0/1785969532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 msgr2=0x55fa5fbec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:56:28.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd189373640 1 --2- 172.21.15.2:0/1785969532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd189373640 1 -- 172.21.15.2:0/1785969532 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa5fb08f20 con 0x55fa5fbec970 2024-03-15T12:56:28.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.195+0000 7fd189b74640 1 --2- 172.21.15.2:0/1785969532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:28.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.196+0000 7fd189373640 1 --2- 172.21.15.2:0/1785969532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbec970 0x55fa5fbf1470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009e80 tx=0x7fd17c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ad4be667f43b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:28.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.196+0000 7fd189b74640 1 --2- 172.21.15.2:0/287062169 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd16c0a1a30 0x7fd16c0a3ef0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd174009fd0 tx=0x7fd174005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:28.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.196+0000 7fd16b7fe640 1 -- 172.21.15.2:0/1785969532 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c019410 con 0x55fa5fbec970 2024-03-15T12:56:28.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.196+0000 7fd16b7fe640 1 -- 172.21.15.2:0/1785969532 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd17c019a40 con 0x55fa5fbec970 2024-03-15T12:56:28.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.196+0000 7fd16b7fe640 1 -- 172.21.15.2:0/1785969532 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c018a90 con 0x55fa5fbec970 2024-03-15T12:56:28.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.196+0000 7fd18e1b4740 1 -- 172.21.15.2:0/1785969532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbec970 msgr2=0x55fa5fbf1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:28.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.196+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/1785969532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbec970 0x55fa5fbf1470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009e80 tx=0x7fd17c011810 comp rx=0 tx=0).stop 2024-03-15T12:56:28.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.196+0000 7fd18e1b4740 1 -- 172.21.15.2:0/1785969532 shutdown_connections 2024-03-15T12:56:28.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.196+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/1785969532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbf3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.196+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/1785969532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbec970 0x55fa5fbf1470 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.196+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/1785969532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.196+0000 7fd18e1b4740 1 -- 172.21.15.2:0/1785969532 >> 172.21.15.2:0/1785969532 conn(0x55fa5fbe74f0 msgr2=0x55fa5fbe78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:28.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.196+0000 7fd18e1b4740 1 -- 172.21.15.2:0/1785969532 shutdown_connections 2024-03-15T12:56:28.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.197+0000 7fd18e1b4740 1 -- 172.21.15.2:0/1785969532 wait complete. 2024-03-15T12:56:28.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.197+0000 7fd18e1b4740 1 -- start start 2024-03-15T12:56:28.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.197+0000 7fd18e1b4740 1 Processor -- start 2024-03-15T12:56:28.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.197+0000 7fd18e1b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbebf80 0x55fa5fbf61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.197+0000 7fd18e1b4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbec970 0x55fa5fbf66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.197+0000 7fd18e1b4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbfd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.197+0000 7fd18e1b4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5fb53c30 con 0x55fa5fbec970 2024-03-15T12:56:28.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.197+0000 7fd18e1b4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5faf8d30 con 0x55fa5fbf1ae0 2024-03-15T12:56:28.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.197+0000 7fd18e1b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5f827310 con 0x55fa5fbebf80 2024-03-15T12:56:28.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.197+0000 7fd18e1b4740 10 client.0 _ll_register_callbacks cb 0x55fa5fb00d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:56:28.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.197+0000 7fd188b72640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbec970 0x55fa5fbf66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.197+0000 7fd188b72640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbec970 0x55fa5fbf66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50666/0 (socket says 172.21.15.2:50666) 2024-03-15T12:56:28.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.197+0000 7fd189b74640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbfd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.197+0000 7fd188b72640 1 -- 172.21.15.2:0/169432592 learned_addr learned my addr 172.21.15.2:0/169432592 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:28.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.197+0000 7fd189373640 1 --2- 172.21.15.2:0/169432592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbebf80 0x55fa5fbf61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd188b72640 1 -- 172.21.15.2:0/169432592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbebf80 msgr2=0x55fa5fbf61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:56:28.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd188b72640 1 --2- 172.21.15.2:0/169432592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbebf80 0x55fa5fbf61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd188b72640 1 -- 172.21.15.2:0/169432592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbf1ae0 msgr2=0x55fa5fbfd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:56:28.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd188b72640 1 --2- 172.21.15.2:0/169432592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbfd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd188b72640 1 -- 172.21.15.2:0/169432592 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17c009ae0 con 0x55fa5fbec970 2024-03-15T12:56:28.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd189b74640 1 --2- 172.21.15.2:0/169432592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbfd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:28.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd188b72640 1 --2- 172.21.15.2:0/169432592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbec970 0x55fa5fbf66e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd1780b8a50 tx=0x7fd178009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8510fd7628a24fa server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:28.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd168ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:56:28.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd168ff9640 1 -- 172.21.15.2:0/169432592 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1780bc5f0 con 0x55fa5fbec970 2024-03-15T12:56:28.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd168ff9640 1 -- 172.21.15.2:0/169432592 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd1780c0740 con 0x55fa5fbec970 2024-03-15T12:56:28.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd18e1b4740 1 -- 172.21.15.2:0/169432592 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa5f8536c0 con 0x55fa5fbec970 2024-03-15T12:56:28.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd168ff9640 1 -- 172.21.15.2:0/169432592 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1780c4d90 con 0x55fa5fbec970 2024-03-15T12:56:28.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd18e1b4740 10 client.14775 fetch_fsmap learned FSMap version 17 2024-03-15T12:56:28.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd18e1b4740 1 -- 172.21.15.2:0/169432592 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55fa5fb522b0 con 0x55fa5fbec970 2024-03-15T12:56:28.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd168ff9640 1 -- 172.21.15.2:0/169432592 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55fa5f8536c0 con 0x55fa5fbec970 2024-03-15T12:56:28.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd168ff9640 1 -- 172.21.15.2:0/169432592 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd17803c070 con 0x55fa5fbec970 2024-03-15T12:56:28.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.198+0000 7fd18e1b4740 10 client.14775 fetch_fsmap finished waiting for FSMap version 17 2024-03-15T12:56:28.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.199+0000 7fd18e1b4740 10 client.14775 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-15T12:56:28.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.199+0000 7fd18e1b4740 20 client.14775 populate_metadata read hostname 'smithi002' 2024-03-15T12:56:28.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.199+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/169432592 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55fa5fc02730 0x55fa5fc06bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.199+0000 7fd18e1b4740 4 client.14775 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:56:28.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.199+0000 7fd18e1b4740 1 -- 172.21.15.2:0/169432592 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa5f8a4270 con 0x55fa5fc02730 2024-03-15T12:56:28.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.199+0000 7fd189373640 1 --2- 172.21.15.2:0/169432592 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55fa5fc02730 0x55fa5fc06bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.199+0000 7fd189373640 1 --2- 172.21.15.2:0/169432592 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55fa5fc02730 0x55fa5fc06bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdb39db3dfa21585 server_cookie=8ff64fe6281829b2 in_seq=0 out_seq=0 2024-03-15T12:56:28.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.199+0000 7fd168ff9640 10 client.14775 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:56:28.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.200+0000 7fd168ff9640 1 -- 172.21.15.2:0/169432592 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fa5f8a4270 con 0x55fa5fc02730 2024-03-15T12:56:28.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.200+0000 7fd168ff9640 10 client.14775 handle_command_reply: tid=0 2024-03-15T12:56:28.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.200+0000 7fd18e1b4740 1 client.14775 shutdown 2024-03-15T12:56:28.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.200+0000 7fd18e1b4740 1 -- 172.21.15.2:0/169432592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbec970 msgr2=0x55fa5fbf66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:28.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.200+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/169432592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbec970 0x55fa5fbf66e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd1780b8a50 tx=0x7fd178009260 comp rx=0 tx=0).stop 2024-03-15T12:56:28.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.200+0000 7fd18e1b4740 1 -- 172.21.15.2:0/169432592 shutdown_connections 2024-03-15T12:56:28.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.200+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/169432592 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55fa5fc02730 0x55fa5fc06bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.200+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/169432592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbfd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.200+0000 7fd168ff9640 0 client.14775 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:56:28.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.200+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/169432592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbec970 0x55fa5fbf66e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.200+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/169432592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbebf80 0x55fa5fbf61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.200+0000 7fd18e1b4740 1 -- 172.21.15.2:0/169432592 >> 172.21.15.2:0/169432592 conn(0x55fa5fbe74f0 msgr2=0x55fa5fbe67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:28.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.200+0000 7fd18e1b4740 1 -- 172.21.15.2:0/169432592 shutdown_connections 2024-03-15T12:56:28.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.200+0000 7fd18e1b4740 1 -- 172.21.15.2:0/169432592 wait complete. 2024-03-15T12:56:28.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.200+0000 7fd18e1b4740 20 client.14775 trim_cache size 0 max 16384 2024-03-15T12:56:28.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd18e1b4740 1 Processor -- start 2024-03-15T12:56:28.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd18e1b4740 1 -- start start 2024-03-15T12:56:28.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd18e1b4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbed410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd18e1b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbec970 0x55fa5fbfd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd18e1b4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbe5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd18e1b4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5fad9490 con 0x55fa5fbf1ae0 2024-03-15T12:56:28.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd18e1b4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5fc008c0 con 0x55fa5fbebf80 2024-03-15T12:56:28.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd18e1b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5fbfe5d0 con 0x55fa5fbec970 2024-03-15T12:56:28.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd189b74640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbe5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd189b74640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbe5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50678/0 (socket says 172.21.15.2:50678) 2024-03-15T12:56:28.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd189b74640 1 -- 172.21.15.2:0/3152617047 learned_addr learned my addr 172.21.15.2:0/3152617047 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:28.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd189373640 1 --2- 172.21.15.2:0/3152617047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd188b72640 1 --2- 172.21.15.2:0/3152617047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbec970 0x55fa5fbfd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd189b74640 1 -- 172.21.15.2:0/3152617047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbec970 msgr2=0x55fa5fbfd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:56:28.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd189b74640 1 --2- 172.21.15.2:0/3152617047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbec970 0x55fa5fbfd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd189b74640 1 -- 172.21.15.2:0/3152617047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 msgr2=0x55fa5fbed410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:56:28.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd189b74640 1 --2- 172.21.15.2:0/3152617047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbed410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd189b74640 1 -- 172.21.15.2:0/3152617047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa5faeea20 con 0x55fa5fbf1ae0 2024-03-15T12:56:28.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd189373640 1 --2- 172.21.15.2:0/3152617047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbed410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:28.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.219+0000 7fd189b74640 1 --2- 172.21.15.2:0/3152617047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbe5f40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd17400fe20 tx=0x7fd174014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1327c2dee4a4a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:28.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.220+0000 7fd16affd640 1 -- 172.21.15.2:0/3152617047 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1740275f0 con 0x55fa5fbf1ae0 2024-03-15T12:56:28.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.220+0000 7fd16affd640 1 -- 172.21.15.2:0/3152617047 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd174018070 con 0x55fa5fbf1ae0 2024-03-15T12:56:28.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.220+0000 7fd16affd640 1 -- 172.21.15.2:0/3152617047 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1740119e0 con 0x55fa5fbf1ae0 2024-03-15T12:56:28.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.220+0000 7fd18e1b4740 1 -- 172.21.15.2:0/3152617047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbf1ae0 msgr2=0x55fa5fbe5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:28.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.220+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/3152617047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbe5f40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd17400fe20 tx=0x7fd174014040 comp rx=0 tx=0).stop 2024-03-15T12:56:28.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.220+0000 7fd18e1b4740 1 -- 172.21.15.2:0/3152617047 shutdown_connections 2024-03-15T12:56:28.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.220+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/3152617047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbe5f40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.220+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/3152617047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbec970 0x55fa5fbfd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.220+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/3152617047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbed410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.220+0000 7fd18e1b4740 1 -- 172.21.15.2:0/3152617047 >> 172.21.15.2:0/3152617047 conn(0x55fa5fbe74f0 msgr2=0x55fa5fbe78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:28.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.220+0000 7fd18e1b4740 1 -- 172.21.15.2:0/3152617047 shutdown_connections 2024-03-15T12:56:28.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.220+0000 7fd18e1b4740 1 -- 172.21.15.2:0/3152617047 wait complete. 2024-03-15T12:56:28.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd18e1b4740 1 -- start start 2024-03-15T12:56:28.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd18e1b4740 1 Processor -- start 2024-03-15T12:56:28.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd18e1b4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbfb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd18e1b4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbec970 0x55fa5fbfbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd18e1b4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbfc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd18e1b4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5fbf1700 con 0x55fa5fbf1ae0 2024-03-15T12:56:28.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd18e1b4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5fbfcad0 con 0x55fa5fbebf80 2024-03-15T12:56:28.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd18e1b4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5fbfcc90 con 0x55fa5fbec970 2024-03-15T12:56:28.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd18e1b4740 10 client.0 _ll_register_callbacks cb 0x7fd1780b8940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:56:28.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd189b74640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbfc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd189373640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbfb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd189373640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbfb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:38874/0 (socket says 172.21.15.2:38874) 2024-03-15T12:56:28.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd188b72640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbec970 0x55fa5fbfbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd189b74640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbfc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50680/0 (socket says 172.21.15.2:50680) 2024-03-15T12:56:28.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd189373640 1 -- 172.21.15.2:0/3245162686 learned_addr learned my addr 172.21.15.2:0/3245162686 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:28.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd189373640 1 -- 172.21.15.2:0/3245162686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbec970 msgr2=0x55fa5fbfbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:56:28.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd189373640 1 --2- 172.21.15.2:0/3245162686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbec970 0x55fa5fbfbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd189373640 1 -- 172.21.15.2:0/3245162686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbf1ae0 msgr2=0x55fa5fbfc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:56:28.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd189373640 1 --2- 172.21.15.2:0/3245162686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbfc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd189373640 1 -- 172.21.15.2:0/3245162686 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa5fb08f20 con 0x55fa5fbebf80 2024-03-15T12:56:28.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd189b74640 1 --2- 172.21.15.2:0/3245162686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbfc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:28.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.221+0000 7fd188b72640 1 --2- 172.21.15.2:0/3245162686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbec970 0x55fa5fbfbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:28.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd189373640 1 --2- 172.21.15.2:0/3245162686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbfb920 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd17c01c040 tx=0x7fd17c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f84dbb4e0d54df61 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:28.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd168ff9640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T12:56:28.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd168ff9640 1 -- 172.21.15.2:0/3245162686 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c019410 con 0x55fa5fbebf80 2024-03-15T12:56:28.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd18e1b4740 1 -- 172.21.15.2:0/3245162686 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd17c019a40 con 0x55fa5fbebf80 2024-03-15T12:56:28.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd168ff9640 1 -- 172.21.15.2:0/3245162686 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd17c019570 con 0x55fa5fbebf80 2024-03-15T12:56:28.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd168ff9640 1 -- 172.21.15.2:0/3245162686 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd17c019750 con 0x55fa5fbebf80 2024-03-15T12:56:28.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd18e1b4740 10 client.24650 fetch_fsmap learned FSMap version 17 2024-03-15T12:56:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd18e1b4740 1 -- 172.21.15.2:0/3245162686 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x7fd174012800 con 0x55fa5fbebf80 2024-03-15T12:56:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd168ff9640 1 -- 172.21.15.2:0/3245162686 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd17c019a40 con 0x55fa5fbebf80 2024-03-15T12:56:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd168ff9640 1 -- 172.21.15.2:0/3245162686 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd17c005460 con 0x55fa5fbebf80 2024-03-15T12:56:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd18e1b4740 10 client.24650 fetch_fsmap finished waiting for FSMap version 17 2024-03-15T12:56:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd18e1b4740 10 client.24650 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-15T12:56:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd18e1b4740 20 client.24650 populate_metadata read hostname 'smithi002' 2024-03-15T12:56:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/3245162686 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55fa5fc02730 0x55fa5fc5e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd18e1b4740 4 client.24650 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-15T12:56:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.222+0000 7fd18e1b4740 1 -- 172.21.15.2:0/3245162686 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fa5f890610 con 0x55fa5fc02730 2024-03-15T12:56:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd188b72640 1 --2- 172.21.15.2:0/3245162686 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55fa5fc02730 0x55fa5fc5e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd188b72640 1 --2- 172.21.15.2:0/3245162686 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55fa5fc02730 0x55fa5fc5e360 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aece111dfa560d2b server_cookie=622c6f679ddbd75d in_seq=0 out_seq=0 2024-03-15T12:56:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd168ff9640 10 client.24650 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:56:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd168ff9640 1 -- 172.21.15.2:0/3245162686 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x55fa5f890610 con 0x55fa5fc02730 2024-03-15T12:56:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd168ff9640 10 client.24650 handle_command_reply: tid=0 2024-03-15T12:56:28.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd18e1b4740 1 client.24650 shutdown 2024-03-15T12:56:28.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd18e1b4740 1 -- 172.21.15.2:0/3245162686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 msgr2=0x55fa5fbfb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:28.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/3245162686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbfb920 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd17c01c040 tx=0x7fd17c005d30 comp rx=0 tx=0).stop 2024-03-15T12:56:28.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd18e1b4740 1 -- 172.21.15.2:0/3245162686 shutdown_connections 2024-03-15T12:56:28.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/3245162686 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55fa5fc02730 0x55fa5fc5e360 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/3245162686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa5fbf1ae0 0x55fa5fbfc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd168ff9640 0 client.24650 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:56:28.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/3245162686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fa5fbec970 0x55fa5fbfbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd18e1b4740 1 --2- 172.21.15.2:0/3245162686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fa5fbebf80 0x55fa5fbfb920 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd18e1b4740 1 -- 172.21.15.2:0/3245162686 >> 172.21.15.2:0/3245162686 conn(0x55fa5fbe74f0 msgr2=0x55fa5fbed410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:28.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd18e1b4740 1 -- 172.21.15.2:0/3245162686 shutdown_connections 2024-03-15T12:56:28.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd18e1b4740 1 -- 172.21.15.2:0/3245162686 wait complete. 2024-03-15T12:56:28.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.223+0000 7fd18e1b4740 20 client.24650 trim_cache size 0 max 16384 2024-03-15T12:56:28.218 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:56:28.218 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "2767718a-e2ca-11ee-95c9-87774f69a715", 2024-03-15T12:56:28.218 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2024-03-15T12:56:28.218 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24539, 2024-03-15T12:56:28.218 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-03-15T12:56:28.218 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-03-15T12:56:28.218 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-03-15T12:56:28.218 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 6.821282998, 2024-03-15T12:56:28.218 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 17, 2024-03-15T12:56:28.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 89, 2024-03-15T12:56:28.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 89, 2024-03-15T12:56:28.218 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 14.127667553 2024-03-15T12:56:28.218 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:56:28.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.224+0000 7fd18b5d7640 1 -- 172.21.15.2:0/287062169 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd16c0a1a30 msgr2=0x7fd16c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:28.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.224+0000 7fd18b5d7640 1 --2- 172.21.15.2:0/287062169 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd16c0a1a30 0x7fd16c0a3ef0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd174009fd0 tx=0x7fd174005bd0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.224+0000 7fd18b5d7640 1 -- 172.21.15.2:0/287062169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1841048c0 msgr2=0x7fd184197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:28.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.224+0000 7fd18b5d7640 1 --2- 172.21.15.2:0/287062169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1841048c0 0x7fd184197b90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd178005ab0 tx=0x7fd178004a30 comp rx=0 tx=0).stop 2024-03-15T12:56:28.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.224+0000 7fd18b5d7640 1 -- 172.21.15.2:0/287062169 shutdown_connections 2024-03-15T12:56:28.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.224+0000 7fd18b5d7640 1 --2- 172.21.15.2:0/287062169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1841071f0 0x7fd18419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.224+0000 7fd18b5d7640 1 --2- 172.21.15.2:0/287062169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1841048c0 0x7fd184197b90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.224+0000 7fd18b5d7640 1 --2- 172.21.15.2:0/287062169 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd16c0a1a30 0x7fd16c0a3ef0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.224+0000 7fd18b5d7640 1 --2- 172.21.15.2:0/287062169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd184101f90 0x7fd184197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:28.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.224+0000 7fd18b5d7640 1 -- 172.21.15.2:0/287062169 >> 172.21.15.2:0/287062169 conn(0x7fd1840fb980 msgr2=0x7fd184103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:28.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.224+0000 7fd18b5d7640 1 -- 172.21.15.2:0/287062169 shutdown_connections 2024-03-15T12:56:28.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:28.224+0000 7fd18b5d7640 1 -- 172.21.15.2:0/287062169 wait complete. 2024-03-15T12:56:28.222 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-15T12:56:28.222 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-15T12:56:28.222 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-15T12:56:28.461 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:28.700 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:28 smithi002 ceph-mon[32060]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-15T12:56:28.700 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:28 smithi002 ceph-mon[32060]: Cluster is now healthy 2024-03-15T12:56:28.700 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:28 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2675365090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:28.700 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:28 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1443304443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:28.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:28 smithi184 ceph-mon[35993]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-15T12:56:28.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:28 smithi184 ceph-mon[35993]: Cluster is now healthy 2024-03-15T12:56:28.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:28 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2675365090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:28.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:28 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1443304443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:28.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:28 smithi149 ceph-mon[35927]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-15T12:56:28.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:28 smithi149 ceph-mon[35927]: Cluster is now healthy 2024-03-15T12:56:28.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:28 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2675365090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:28.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:28 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1443304443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:29.752 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:29 smithi002 ceph-mon[32060]: pgmap v294: 129 pgs: 11 creating+peering, 113 active+clean, 5 unknown; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-03-15T12:56:29.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:29 smithi184 ceph-mon[35993]: pgmap v294: 129 pgs: 11 creating+peering, 113 active+clean, 5 unknown; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-03-15T12:56:29.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:29 smithi149 ceph-mon[35927]: pgmap v294: 129 pgs: 11 creating+peering, 113 active+clean, 5 unknown; 579 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-03-15T12:56:30.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.159+0000 7ff270008700 1 -- 172.21.15.2:0/1817631887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff268100b70 msgr2=0x7ff268100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:30.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.159+0000 7ff270008700 1 --2- 172.21.15.2:0/1817631887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff268100b70 0x7ff268100f50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff264009990 tx=0x7ff264009ca0 comp rx=0 tx=0).stop 2024-03-15T12:56:30.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.159+0000 7ff270008700 1 -- 172.21.15.2:0/1817631887 shutdown_connections 2024-03-15T12:56:30.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.159+0000 7ff270008700 1 --2- 172.21.15.2:0/1817631887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff268100b70 0x7ff268100f50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:30.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.159+0000 7ff270008700 1 --2- 172.21.15.2:0/1817631887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff268107390 0x7ff26810ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:30.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.159+0000 7ff270008700 1 --2- 172.21.15.2:0/1817631887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff268101520 0x7ff268106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:30.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.159+0000 7ff270008700 1 -- 172.21.15.2:0/1817631887 >> 172.21.15.2:0/1817631887 conn(0x7ff268076210 msgr2=0x7ff268076620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:30.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.160+0000 7ff270008700 1 -- 172.21.15.2:0/1817631887 shutdown_connections 2024-03-15T12:56:30.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.160+0000 7ff270008700 1 -- 172.21.15.2:0/1817631887 wait complete. 2024-03-15T12:56:30.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.160+0000 7ff270008700 1 Processor -- start 2024-03-15T12:56:30.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.160+0000 7ff270008700 1 -- start start 2024-03-15T12:56:30.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.160+0000 7ff270008700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff268100b70 0x7ff268101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:30.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.160+0000 7ff270008700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff268101520 0x7ff268102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:30.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.160+0000 7ff270008700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff268107390 0x7ff268102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:30.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.160+0000 7ff270008700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2681a6950 con 0x7ff268107390 2024-03-15T12:56:30.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.160+0000 7ff270008700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2681a6ac0 con 0x7ff268100b70 2024-03-15T12:56:30.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.160+0000 7ff270008700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2681a6c80 con 0x7ff268101520 2024-03-15T12:56:30.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.160+0000 7ff26e5a5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff268107390 0x7ff268102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:30.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.161+0000 7ff26e5a5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff268107390 0x7ff268102570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50690/0 (socket says 172.21.15.2:50690) 2024-03-15T12:56:30.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.161+0000 7ff26e5a5700 1 -- 172.21.15.2:0/2356628341 learned_addr learned my addr 172.21.15.2:0/2356628341 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:30.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.161+0000 7ff26dda4700 1 --2- 172.21.15.2:0/2356628341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff268100b70 0x7ff268101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:30.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.161+0000 7ff26d5a3700 1 --2- 172.21.15.2:0/2356628341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff268101520 0x7ff268102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:30.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.161+0000 7ff26e5a5700 1 -- 172.21.15.2:0/2356628341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff268101520 msgr2=0x7ff268102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:30.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.161+0000 7ff26e5a5700 1 --2- 172.21.15.2:0/2356628341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff268101520 0x7ff268102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:30.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.161+0000 7ff26e5a5700 1 -- 172.21.15.2:0/2356628341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff268100b70 msgr2=0x7ff268101af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:30.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.161+0000 7ff26e5a5700 1 --2- 172.21.15.2:0/2356628341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff268100b70 0x7ff268101af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:30.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.161+0000 7ff26e5a5700 1 -- 172.21.15.2:0/2356628341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2640095f0 con 0x7ff268107390 2024-03-15T12:56:30.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.161+0000 7ff26e5a5700 1 --2- 172.21.15.2:0/2356628341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff268107390 0x7ff268102570 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff260007ee0 tx=0x7ff26000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:30.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.161+0000 7ff25effd700 1 -- 172.21.15.2:0/2356628341 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff260017070 con 0x7ff268107390 2024-03-15T12:56:30.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.161+0000 7ff25effd700 1 -- 172.21.15.2:0/2356628341 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff26000f040 con 0x7ff268107390 2024-03-15T12:56:30.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.161+0000 7ff270008700 1 -- 172.21.15.2:0/2356628341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2681a6f60 con 0x7ff268107390 2024-03-15T12:56:30.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.161+0000 7ff25effd700 1 -- 172.21.15.2:0/2356628341 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff260005680 con 0x7ff268107390 2024-03-15T12:56:30.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.161+0000 7ff270008700 1 -- 172.21.15.2:0/2356628341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2681a7430 con 0x7ff268107390 2024-03-15T12:56:30.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.162+0000 7ff270008700 1 -- 172.21.15.2:0/2356628341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26810a5f0 con 0x7ff268107390 2024-03-15T12:56:30.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.163+0000 7ff25effd700 1 -- 172.21.15.2:0/2356628341 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7ff2600057e0 con 0x7ff268107390 2024-03-15T12:56:30.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.164+0000 7ff25effd700 1 --2- 172.21.15.2:0/2356628341 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff2540a1ac0 0x7ff2540a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:30.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.164+0000 7ff26dda4700 1 --2- 172.21.15.2:0/2356628341 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff2540a1ac0 0x7ff2540a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:30.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.164+0000 7ff25effd700 1 -- 172.21.15.2:0/2356628341 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff2600cd340 con 0x7ff268107390 2024-03-15T12:56:30.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.164+0000 7ff26dda4700 1 --2- 172.21.15.2:0/2356628341 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff2540a1ac0 0x7ff2540a3f80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff2640098e0 tx=0x7ff264005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:30.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.167+0000 7ff25effd700 1 -- 172.21.15.2:0/2356628341 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff2600962b0 con 0x7ff268107390 2024-03-15T12:56:30.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.416+0000 7ff270008700 1 -- 172.21.15.2:0/2356628341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff268112080 con 0x7ff268107390 2024-03-15T12:56:30.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.416+0000 7ff25effd700 1 -- 172.21.15.2:0/2356628341 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff260099970 con 0x7ff268107390 2024-03-15T12:56:30.403 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-15T12:56:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.418+0000 7ff270008700 1 -- 172.21.15.2:0/2356628341 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff2540a1ac0 msgr2=0x7ff2540a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.418+0000 7ff270008700 1 --2- 172.21.15.2:0/2356628341 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff2540a1ac0 0x7ff2540a3f80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff2640098e0 tx=0x7ff264005960 comp rx=0 tx=0).stop 2024-03-15T12:56:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.418+0000 7ff270008700 1 -- 172.21.15.2:0/2356628341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff268107390 msgr2=0x7ff268102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.418+0000 7ff270008700 1 --2- 172.21.15.2:0/2356628341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff268107390 0x7ff268102570 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff260007ee0 tx=0x7ff26000d3b0 comp rx=0 tx=0).stop 2024-03-15T12:56:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.419+0000 7ff270008700 1 -- 172.21.15.2:0/2356628341 shutdown_connections 2024-03-15T12:56:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.419+0000 7ff270008700 1 --2- 172.21.15.2:0/2356628341 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff2540a1ac0 0x7ff2540a3f80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.419+0000 7ff270008700 1 --2- 172.21.15.2:0/2356628341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff268107390 0x7ff268102570 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.419+0000 7ff270008700 1 --2- 172.21.15.2:0/2356628341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff268100b70 0x7ff268101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.419+0000 7ff270008700 1 --2- 172.21.15.2:0/2356628341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff268101520 0x7ff268102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:30.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.419+0000 7ff270008700 1 -- 172.21.15.2:0/2356628341 >> 172.21.15.2:0/2356628341 conn(0x7ff268076210 msgr2=0x7ff268100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:30.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.419+0000 7ff270008700 1 -- 172.21.15.2:0/2356628341 shutdown_connections 2024-03-15T12:56:30.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:30.419+0000 7ff270008700 1 -- 172.21.15.2:0/2356628341 wait complete. 2024-03-15T12:56:30.723 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:30 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2356628341' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-15T12:56:30.761 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:30 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2356628341' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-15T12:56:30.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:30 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2356628341' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-15T12:56:31.070 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-15T12:56:31.070 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-15T12:56:31.070 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi002.front.sepia.ceph.com 2024-03-15T12:56:31.070 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-15T12:56:31.070 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-15T12:56:31.070 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-15T12:56:31.070 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-15T12:56:31.071 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-15T12:56:31.071 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-15T12:56:31.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:31.071 DEBUG:teuthology.orchestra.run.smithi002:> ip addr 2024-03-15T12:56:31.088 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-15T12:56:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-15T12:56:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2024-03-15T12:56:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-03-15T12:56:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2024-03-15T12:56:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-03-15T12:56:31.088 INFO:teuthology.orchestra.run.smithi002.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-15T12:56:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:02 brd ff:ff:ff:ff:ff:ff 2024-03-15T12:56:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f0 2024-03-15T12:56:31.088 INFO:teuthology.orchestra.run.smithi002.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-15T12:56:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:03 brd ff:ff:ff:ff:ff:ff 2024-03-15T12:56:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f1 2024-03-15T12:56:31.089 INFO:teuthology.orchestra.run.smithi002.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-15T12:56:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:20 brd ff:ff:ff:ff:ff:ff 2024-03-15T12:56:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f0 2024-03-15T12:56:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: inet 172.21.15.2/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-15T12:56:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft 42234sec preferred_lft 42234sec 2024-03-15T12:56:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::ec4:7aff:febd:1920/64 scope link noprefixroute 2024-03-15T12:56:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-03-15T12:56:31.089 INFO:teuthology.orchestra.run.smithi002.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-15T12:56:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:21 brd ff:ff:ff:ff:ff:ff 2024-03-15T12:56:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f1 2024-03-15T12:56:31.090 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-15T12:56:31.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:31.090 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2024-03-15T12:56:31.090 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-03-15T12:56:31.090 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add name ceph-brx type bridge 2024-03-15T12:56:31.090 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr flush dev ceph-brx 2024-03-15T12:56:31.090 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set ceph-brx up 2024-03-15T12:56:31.090 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-15T12:56:31.090 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-03-15T12:56:31.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:31.256 DEBUG:teuthology.orchestra.run.smithi002:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-15T12:56:31.283 INFO:teuthology.orchestra.run.smithi002.stdout:1 2024-03-15T12:56:31.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:31.284 DEBUG:teuthology.orchestra.run.smithi002:> route 2024-03-15T12:56:31.359 INFO:teuthology.orchestra.run.smithi002.stdout:Kernel IP routing table 2024-03-15T12:56:31.359 INFO:teuthology.orchestra.run.smithi002.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-15T12:56:31.359 INFO:teuthology.orchestra.run.smithi002.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-15T12:56:31.359 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-15T12:56:31.359 INFO:teuthology.orchestra.run.smithi002.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-15T12:56:31.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:31.359 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2024-03-15T12:56:31.359 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-03-15T12:56:31.359 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-15T12:56:31.360 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-15T12:56:31.360 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-15T12:56:31.360 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-03-15T12:56:31.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:31.618 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-03-15T12:56:31.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:31.632 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list-id 2024-03-15T12:56:31.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:31.687 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2024-03-15T12:56:31.687 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-03-15T12:56:31.687 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-15T12:56:31.687 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-15T12:56:31.687 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-03-15T12:56:31.710 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:31 smithi184 ceph-mon[35993]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-03-15T12:56:31.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:31 smithi002 ceph-mon[32060]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-03-15T12:56:31.769 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-15T12:56:31.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:31.770 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2024-03-15T12:56:31.770 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-03-15T12:56:31.770 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-15T12:56:31.770 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-15T12:56:31.770 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-15T12:56:31.770 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-15T12:56:31.770 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-15T12:56:31.770 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-03-15T12:56:31.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:31 smithi149 ceph-mon[35927]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-03-15T12:56:31.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:31.904 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2024-03-15T12:56:31.904 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-03-15T12:56:31.904 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set brx.0 up 2024-03-15T12:56:31.904 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set dev brx.0 master ceph-brx 2024-03-15T12:56:31.904 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-03-15T12:56:31.946 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-15T12:56:31.946 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-15T12:56:31.946 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-15T12:56:32.000 INFO:teuthology.orchestra.run.smithi002.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-15T12:56:32.000 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-15T12:56:32.001 DEBUG:teuthology.orchestra.run.smithi002:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-15T12:56:32.055 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2024-03-15T12:56:32.122 DEBUG:teuthology.orchestra.run.smithi002:> cat /proc/self/mounts | awk '{print $2}' 2024-03-15T12:56:32.177 INFO:teuthology.orchestra.run.smithi002.stdout:/proc 2024-03-15T12:56:32.177 INFO:teuthology.orchestra.run.smithi002.stdout:/sys 2024-03-15T12:56:32.177 INFO:teuthology.orchestra.run.smithi002.stdout:/dev 2024-03-15T12:56:32.177 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/security 2024-03-15T12:56:32.178 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/shm 2024-03-15T12:56:32.178 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/pts 2024-03-15T12:56:32.178 INFO:teuthology.orchestra.run.smithi002.stdout:/run 2024-03-15T12:56:32.178 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup 2024-03-15T12:56:32.178 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/pstore 2024-03-15T12:56:32.178 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/bpf 2024-03-15T12:56:32.178 INFO:teuthology.orchestra.run.smithi002.stdout:/ 2024-03-15T12:56:32.178 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/selinux 2024-03-15T12:56:32.178 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-03-15T12:56:32.178 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/hugepages 2024-03-15T12:56:32.178 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/mqueue 2024-03-15T12:56:32.178 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/debug 2024-03-15T12:56:32.179 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/tracing 2024-03-15T12:56:32.179 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/fuse/connections 2024-03-15T12:56:32.179 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/config 2024-03-15T12:56:32.179 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-sysctl.service 2024-03-15T12:56:32.179 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-15T12:56:32.179 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-15T12:56:32.179 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/nfs/rpc_pipefs 2024-03-15T12:56:32.179 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/1000 2024-03-15T12:56:32.179 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/ceph 2024-03-15T12:56:32.179 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay 2024-03-15T12:56:32.179 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/ddc5a2df6c5f126ea7bcf954c77b16982897aafddcc8a8833a369632c9da48f6/merged 2024-03-15T12:56:32.179 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/c8cab69b1194350f1ab37f22fb9ecd4d572d553dadfa155815b6e3304aafd679/merged 2024-03-15T12:56:32.179 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/0 2024-03-15T12:56:32.179 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-03-15T12:56:32.179 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/de0edcba7fb75af99612c7e9340d1c85cbc048ba4ec53b132164902fc55e8f0d/merged 2024-03-15T12:56:32.180 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/b556dbd452654a7280471de853f40a675f63f2735c33d3f32328c9d6e269b0a4/merged 2024-03-15T12:56:32.180 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/0a20668dbd54c385f33152d7dfc70e158dec0f7a59dac3f9d737104ca8a54b0f/merged 2024-03-15T12:56:32.180 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f822ccf9d7b8f0ed2283a52b0123cd8ca69491f62a68b9dc18cab311c2a61eae/merged 2024-03-15T12:56:32.180 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/e0f808252d75766f7d109b8e9ac8e23500976feed51562045ffe3848dab1aa02/merged 2024-03-15T12:56:32.180 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/209819ff0d0fda5f82c3194ba05b1a73df3bd98ee861e678f17c1c4a8999cfc1/merged 2024-03-15T12:56:32.180 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/1246d9dc4a387cb1fec4afdef821c97b1407c045d315e204017bb0df0323f531/merged 2024-03-15T12:56:32.180 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/1b73366dc835c10d01b7f9eb61231037762c713aaa2c489e3875a3e0e8c64e94/merged 2024-03-15T12:56:32.180 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns 2024-03-15T12:56:32.180 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-15T12:56:32.180 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-15T12:56:32.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:32.180 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2024-03-15T12:56:32.232 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-15T12:56:32.232 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-15T12:56:32.274 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2024-03-15T12:56:32.301 DEBUG:teuthology.orchestra.run.smithi002:> cat /proc/self/mounts | awk '{print $2}' 2024-03-15T12:56:32.357 INFO:teuthology.orchestra.run.smithi002.stdout:/proc 2024-03-15T12:56:32.357 INFO:teuthology.orchestra.run.smithi002.stdout:/sys 2024-03-15T12:56:32.357 INFO:teuthology.orchestra.run.smithi002.stdout:/dev 2024-03-15T12:56:32.357 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/security 2024-03-15T12:56:32.358 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/shm 2024-03-15T12:56:32.358 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/pts 2024-03-15T12:56:32.358 INFO:teuthology.orchestra.run.smithi002.stdout:/run 2024-03-15T12:56:32.358 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup 2024-03-15T12:56:32.358 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/pstore 2024-03-15T12:56:32.358 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/bpf 2024-03-15T12:56:32.358 INFO:teuthology.orchestra.run.smithi002.stdout:/ 2024-03-15T12:56:32.358 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/selinux 2024-03-15T12:56:32.358 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-03-15T12:56:32.358 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/hugepages 2024-03-15T12:56:32.358 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/mqueue 2024-03-15T12:56:32.358 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/debug 2024-03-15T12:56:32.359 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/tracing 2024-03-15T12:56:32.359 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/fuse/connections 2024-03-15T12:56:32.359 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/config 2024-03-15T12:56:32.359 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-sysctl.service 2024-03-15T12:56:32.359 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-15T12:56:32.359 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-15T12:56:32.359 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/nfs/rpc_pipefs 2024-03-15T12:56:32.359 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/1000 2024-03-15T12:56:32.359 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/ceph 2024-03-15T12:56:32.359 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay 2024-03-15T12:56:32.359 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/ddc5a2df6c5f126ea7bcf954c77b16982897aafddcc8a8833a369632c9da48f6/merged 2024-03-15T12:56:32.359 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/c8cab69b1194350f1ab37f22fb9ecd4d572d553dadfa155815b6e3304aafd679/merged 2024-03-15T12:56:32.359 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/0 2024-03-15T12:56:32.360 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-03-15T12:56:32.360 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/de0edcba7fb75af99612c7e9340d1c85cbc048ba4ec53b132164902fc55e8f0d/merged 2024-03-15T12:56:32.360 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/b556dbd452654a7280471de853f40a675f63f2735c33d3f32328c9d6e269b0a4/merged 2024-03-15T12:56:32.360 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/0a20668dbd54c385f33152d7dfc70e158dec0f7a59dac3f9d737104ca8a54b0f/merged 2024-03-15T12:56:32.360 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f822ccf9d7b8f0ed2283a52b0123cd8ca69491f62a68b9dc18cab311c2a61eae/merged 2024-03-15T12:56:32.360 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/e0f808252d75766f7d109b8e9ac8e23500976feed51562045ffe3848dab1aa02/merged 2024-03-15T12:56:32.360 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/209819ff0d0fda5f82c3194ba05b1a73df3bd98ee861e678f17c1c4a8999cfc1/merged 2024-03-15T12:56:32.360 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/1246d9dc4a387cb1fec4afdef821c97b1407c045d315e204017bb0df0323f531/merged 2024-03-15T12:56:32.360 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/1b73366dc835c10d01b7f9eb61231037762c713aaa2c489e3875a3e0e8c64e94/merged 2024-03-15T12:56:32.361 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns 2024-03-15T12:56:32.361 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-15T12:56:32.361 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-15T12:56:32.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:32.361 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2024-03-15T12:56:32.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:2024-03-15T12:56:32.383+0000 7f469543e480 -1 init, newargv = 0x55a92af0c4a0 newargc=15 2024-03-15T12:56:32.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:ceph-fuse[76297]: starting ceph client 2024-03-15T12:56:32.375 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:ceph-fuse[76297]: starting fuse 2024-03-15T12:56:32.384 INFO:teuthology.orchestra.run.smithi002.stdout:119 2024-03-15T12:56:32.384 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-03-15T12:56:32.384 DEBUG:teuthology.orchestra.run.smithi002:> sudo stdin-killer -- python3 -c ' 2024-03-15T12:56:32.384 DEBUG:teuthology.orchestra.run.smithi002:> import glob 2024-03-15T12:56:32.384 DEBUG:teuthology.orchestra.run.smithi002:> import re 2024-03-15T12:56:32.384 DEBUG:teuthology.orchestra.run.smithi002:> import os 2024-03-15T12:56:32.384 DEBUG:teuthology.orchestra.run.smithi002:> import subprocess 2024-03-15T12:56:32.384 DEBUG:teuthology.orchestra.run.smithi002:> 2024-03-15T12:56:32.384 DEBUG:teuthology.orchestra.run.smithi002:> def _find_admin_socket(client_name): 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> return files[0] 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> for f in files: 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> contents = proc_f.read() 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> if mountpoint in contents: 2024-03-15T12:56:32.385 DEBUG:teuthology.orchestra.run.smithi002:> return f 2024-03-15T12:56:32.386 DEBUG:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-15T12:56:32.386 DEBUG:teuthology.orchestra.run.smithi002:> 2024-03-15T12:56:32.386 DEBUG:teuthology.orchestra.run.smithi002:> print(_find_admin_socket("client.admin")) 2024-03-15T12:56:32.386 DEBUG:teuthology.orchestra.run.smithi002:> ' 2024-03-15T12:56:32.505 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.admin.76297.asok 2024-03-15T12:56:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-15T12:56:32.515 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76297.asok 2024-03-15T12:56:32.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:32.516 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76297.asok status 2024-03-15T12:56:32.653 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:56:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2024-03-15T12:56:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b", 2024-03-15T12:56:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 19.0.0-1681-gaba3c1d2 (aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b) squid (dev)", 2024-03-15T12:56:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "admin", 2024-03-15T12:56:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2024-03-15T12:56:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-15T12:56:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "76297", 2024-03-15T12:56:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/" 2024-03-15T12:56:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T12:56:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2024-03-15T12:56:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2024-03-15T12:56:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 14793, 2024-03-15T12:56:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2024-03-15T12:56:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2024-03-15T12:56:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2024-03-15T12:56:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 14793 2024-03-15T12:56:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T12:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2024-03-15T12:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-03-15T12:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "192.168.0.1:0", 2024-03-15T12:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2166214050 2024-03-15T12:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T12:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T12:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2024-03-15T12:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-03-15T12:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "192.168.0.1:0", 2024-03-15T12:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2166214050 2024-03-15T12:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T12:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.14793 192.168.0.1:0/2166214050", 2024-03-15T12:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "192.168.0.1:0/2166214050", 2024-03-15T12:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 1, 2024-03-15T12:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 17, 2024-03-15T12:56:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 89, 2024-03-15T12:56:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2024-03-15T12:56:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklisted": false, 2024-03-15T12:56:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs" 2024-03-15T12:56:32.656 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:56:32.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:32.662 DEBUG:teuthology.orchestra.run.smithi002:> stat --file-system '--printf=%T 2024-03-15T12:56:32.662 DEBUG:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-15T12:56:32.677 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2024-03-15T12:56:32.677 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-15T12:56:32.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:32.677 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-15T12:56:32.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:32.744 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-15T12:56:32.814 INFO:teuthology.orchestra.run.smithi002.stdout:# file: . 2024-03-15T12:56:32.814 INFO:teuthology.orchestra.run.smithi002.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-15T12:56:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:32.815 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:56:32.815 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-15T12:56:32.871 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:56:33.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.027+0000 7eff8cd9a640 1 -- 172.21.15.2:0/836355047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff88100490 msgr2=0x7eff881008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.027+0000 7eff8cd9a640 1 --2- 172.21.15.2:0/836355047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff88100490 0x7eff881008b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7eff80009900 tx=0x7eff8002bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:33.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.027+0000 7eff8cd9a640 1 -- 172.21.15.2:0/836355047 shutdown_connections 2024-03-15T12:56:33.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.027+0000 7eff8cd9a640 1 --2- 172.21.15.2:0/836355047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff88102040 0x7eff881088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.027+0000 7eff8cd9a640 1 --2- 172.21.15.2:0/836355047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff88101680 0x7eff88101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.027+0000 7eff8cd9a640 1 --2- 172.21.15.2:0/836355047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff88100490 0x7eff881008b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.027+0000 7eff8cd9a640 1 -- 172.21.15.2:0/836355047 >> 172.21.15.2:0/836355047 conn(0x7eff880fba20 msgr2=0x7eff880fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:33.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.027+0000 7eff8cd9a640 1 -- 172.21.15.2:0/836355047 shutdown_connections 2024-03-15T12:56:33.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.027+0000 7eff8cd9a640 1 -- 172.21.15.2:0/836355047 wait complete. 2024-03-15T12:56:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff8cd9a640 1 Processor -- start 2024-03-15T12:56:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff8cd9a640 1 -- start start 2024-03-15T12:56:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff8cd9a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff88100490 0x7eff881044f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff8cd9a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff88101680 0x7eff88102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff8cd9a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff88102040 0x7eff88103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff8cd9a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff881037d0 con 0x7eff88101680 2024-03-15T12:56:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff8cd9a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff88103940 con 0x7eff88100490 2024-03-15T12:56:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff8cd9a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff881a1640 con 0x7eff88102040 2024-03-15T12:56:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff85d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff88101680 0x7eff88102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff85d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff88101680 0x7eff88102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50710/0 (socket says 172.21.15.2:50710) 2024-03-15T12:56:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff85d9b640 1 -- 172.21.15.2:0/2152362388 learned_addr learned my addr 172.21.15.2:0/2152362388 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff8659c640 1 --2- 172.21.15.2:0/2152362388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff88100490 0x7eff881044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff86d9d640 1 --2- 172.21.15.2:0/2152362388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff88102040 0x7eff88103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff85d9b640 1 -- 172.21.15.2:0/2152362388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff88102040 msgr2=0x7eff88103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff85d9b640 1 --2- 172.21.15.2:0/2152362388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff88102040 0x7eff88103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff85d9b640 1 -- 172.21.15.2:0/2152362388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff88100490 msgr2=0x7eff881044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff85d9b640 1 --2- 172.21.15.2:0/2152362388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff88100490 0x7eff881044f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff85d9b640 1 -- 172.21.15.2:0/2152362388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff80009560 con 0x7eff88101680 2024-03-15T12:56:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.028+0000 7eff85d9b640 1 --2- 172.21.15.2:0/2152362388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff88101680 0x7eff88102b30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00d980 tx=0x7eff7c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.029+0000 7eff777fe640 1 -- 172.21.15.2:0/2152362388 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c009c30 con 0x7eff88101680 2024-03-15T12:56:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.029+0000 7eff777fe640 1 -- 172.21.15.2:0/2152362388 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7eff7c00b660 con 0x7eff88101680 2024-03-15T12:56:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.029+0000 7eff8cd9a640 1 -- 172.21.15.2:0/2152362388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff881a18f0 con 0x7eff88101680 2024-03-15T12:56:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.029+0000 7eff777fe640 1 -- 172.21.15.2:0/2152362388 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c00f6b0 con 0x7eff88101680 2024-03-15T12:56:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.029+0000 7eff8cd9a640 1 -- 172.21.15.2:0/2152362388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff881a1e00 con 0x7eff88101680 2024-03-15T12:56:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.030+0000 7eff8cd9a640 1 -- 172.21.15.2:0/2152362388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8810be00 con 0x7eff88101680 2024-03-15T12:56:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.031+0000 7eff777fe640 1 -- 172.21.15.2:0/2152362388 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7eff7c010460 con 0x7eff88101680 2024-03-15T12:56:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.031+0000 7eff777fe640 1 --2- 172.21.15.2:0/2152362388 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7eff700a1a30 0x7eff700a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.031+0000 7eff8659c640 1 --2- 172.21.15.2:0/2152362388 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7eff700a1a30 0x7eff700a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.031+0000 7eff777fe640 1 -- 172.21.15.2:0/2152362388 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7eff7c0cd270 con 0x7eff88101680 2024-03-15T12:56:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.032+0000 7eff8659c640 1 --2- 172.21.15.2:0/2152362388 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7eff700a1a30 0x7eff700a3ef0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff8002c740 tx=0x7eff80005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:33.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.036+0000 7eff777fe640 1 -- 172.21.15.2:0/2152362388 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7eff7c096240 con 0x7eff88101680 2024-03-15T12:56:33.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:32 smithi002 ceph-mon[32060]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 1 op/s 2024-03-15T12:56:33.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:32 smithi184 ceph-mon[35993]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 1 op/s 2024-03-15T12:56:33.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:32 smithi149 ceph-mon[35927]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 1 op/s 2024-03-15T12:56:33.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.222+0000 7eff8cd9a640 1 -- 172.21.15.2:0/2152362388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff881a1f90 con 0x7eff88101680 2024-03-15T12:56:33.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.223+0000 7eff777fe640 1 -- 172.21.15.2:0/2152362388 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24045 (secure 0 0 0) 0x7eff881a1f90 con 0x7eff88101680 2024-03-15T12:56:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:33.210 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":89,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:56:25.830426+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:56:33.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.225+0000 7eff8cd9a640 1 -- 172.21.15.2:0/2152362388 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7eff700a1a30 msgr2=0x7eff700a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.225+0000 7eff8cd9a640 1 --2- 172.21.15.2:0/2152362388 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7eff700a1a30 0x7eff700a3ef0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff8002c740 tx=0x7eff80005fd0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.225+0000 7eff8cd9a640 1 -- 172.21.15.2:0/2152362388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff88101680 msgr2=0x7eff88102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.225+0000 7eff8cd9a640 1 --2- 172.21.15.2:0/2152362388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff88101680 0x7eff88102b30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00d980 tx=0x7eff7c00de40 comp rx=0 tx=0).stop 2024-03-15T12:56:33.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.225+0000 7eff8cd9a640 1 -- 172.21.15.2:0/2152362388 shutdown_connections 2024-03-15T12:56:33.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.225+0000 7eff8cd9a640 1 --2- 172.21.15.2:0/2152362388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff88102040 0x7eff88103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.225+0000 7eff8cd9a640 1 --2- 172.21.15.2:0/2152362388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff88101680 0x7eff88102b30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.225+0000 7eff8cd9a640 1 --2- 172.21.15.2:0/2152362388 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7eff700a1a30 0x7eff700a3ef0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.225+0000 7eff8cd9a640 1 --2- 172.21.15.2:0/2152362388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff88100490 0x7eff881044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.225+0000 7eff8cd9a640 1 -- 172.21.15.2:0/2152362388 >> 172.21.15.2:0/2152362388 conn(0x7eff880fba20 msgr2=0x7eff88069b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:33.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.225+0000 7eff8cd9a640 1 -- 172.21.15.2:0/2152362388 shutdown_connections 2024-03-15T12:56:33.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.225+0000 7eff8cd9a640 1 -- 172.21.15.2:0/2152362388 wait complete. 2024-03-15T12:56:33.223 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-15T12:51:47.004843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-15T12:56:18.559882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-15T12:56:19.751767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-15T12:56:23.664879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-15T12:56:33.223 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-15T12:56:33.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.342+0000 7f9400046640 1 -- 172.21.15.2:0/3645270074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f81003f0 msgr2=0x7f93f8100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.342+0000 7f9400046640 1 --2- 172.21.15.2:0/3645270074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f81003f0 0x7f93f8100810 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009900 tx=0x7f93f002bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:33.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f9400046640 1 -- 172.21.15.2:0/3645270074 shutdown_connections 2024-03-15T12:56:33.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f9400046640 1 --2- 172.21.15.2:0/3645270074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93f8101fa0 0x7f93f8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f9400046640 1 --2- 172.21.15.2:0/3645270074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93f81015e0 0x7f93f8101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f9400046640 1 --2- 172.21.15.2:0/3645270074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f81003f0 0x7f93f8100810 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f9400046640 1 -- 172.21.15.2:0/3645270074 >> 172.21.15.2:0/3645270074 conn(0x7f93f80fb980 msgr2=0x7f93f80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:33.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f9400046640 1 -- 172.21.15.2:0/3645270074 shutdown_connections 2024-03-15T12:56:33.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f9400046640 1 -- 172.21.15.2:0/3645270074 wait complete. 2024-03-15T12:56:33.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f9400046640 1 Processor -- start 2024-03-15T12:56:33.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f9400046640 1 -- start start 2024-03-15T12:56:33.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f9400046640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f81003f0 0x7f93f810acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f9400046640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93f81015e0 0x7f93f81092f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f9400046640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93f8101fa0 0x7f93f8109830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f9400046640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f819b470 con 0x7f93f81003f0 2024-03-15T12:56:33.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f9400046640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f8109d70 con 0x7f93f81015e0 2024-03-15T12:56:33.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f9400046640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f8109f30 con 0x7f93f8101fa0 2024-03-15T12:56:33.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.343+0000 7f93fdde2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f81003f0 0x7f93f810acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f93fd5e1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93f81015e0 0x7f93f81092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f93fdde2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f81003f0 0x7f93f810acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50718/0 (socket says 172.21.15.2:50718) 2024-03-15T12:56:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f93fdde2640 1 -- 172.21.15.2:0/1471925563 learned_addr learned my addr 172.21.15.2:0/1471925563 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f93fe5e3640 1 --2- 172.21.15.2:0/1471925563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93f8101fa0 0x7f93f8109830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f93fd5e1640 1 -- 172.21.15.2:0/1471925563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93f8101fa0 msgr2=0x7f93f8109830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f93fd5e1640 1 --2- 172.21.15.2:0/1471925563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93f8101fa0 0x7f93f8109830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f93fd5e1640 1 -- 172.21.15.2:0/1471925563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f81003f0 msgr2=0x7f93f810acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f93fd5e1640 1 --2- 172.21.15.2:0/1471925563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f81003f0 0x7f93f810acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f93fd5e1640 1 -- 172.21.15.2:0/1471925563 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f0009560 con 0x7f93f81015e0 2024-03-15T12:56:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f93fdde2640 1 --2- 172.21.15.2:0/1471925563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f81003f0 0x7f93f810acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f93fd5e1640 1 --2- 172.21.15.2:0/1471925563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93f81015e0 0x7f93f81092f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00e9f0 tx=0x7f93ec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f93f6ffd640 1 -- 172.21.15.2:0/1471925563 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec00ccc0 con 0x7f93f81015e0 2024-03-15T12:56:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f9400046640 1 -- 172.21.15.2:0/1471925563 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f81a5e60 con 0x7f93f81015e0 2024-03-15T12:56:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f93f6ffd640 1 -- 172.21.15.2:0/1471925563 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f93ec00ce20 con 0x7f93f81015e0 2024-03-15T12:56:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f93f6ffd640 1 -- 172.21.15.2:0/1471925563 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec004a00 con 0x7f93f81015e0 2024-03-15T12:56:33.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.344+0000 7f9400046640 1 -- 172.21.15.2:0/1471925563 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f81a61d0 con 0x7f93f81015e0 2024-03-15T12:56:33.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.345+0000 7f9400046640 1 -- 172.21.15.2:0/1471925563 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f80682e0 con 0x7f93f81015e0 2024-03-15T12:56:33.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.346+0000 7f93f6ffd640 1 -- 172.21.15.2:0/1471925563 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f93ec004b60 con 0x7f93f81015e0 2024-03-15T12:56:33.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.347+0000 7f93f6ffd640 1 --2- 172.21.15.2:0/1471925563 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f93e00a1ab0 0x7f93e00a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.347+0000 7f93fdde2640 1 --2- 172.21.15.2:0/1471925563 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f93e00a1ab0 0x7f93e00a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.347+0000 7f93f6ffd640 1 -- 172.21.15.2:0/1471925563 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f93ec014070 con 0x7f93f81015e0 2024-03-15T12:56:33.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.347+0000 7f93fdde2640 1 --2- 172.21.15.2:0/1471925563 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f93e00a1ab0 0x7f93e00a3f70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f93f0006010 tx=0x7f93f0004e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.351+0000 7f93f6ffd640 1 -- 172.21.15.2:0/1471925563 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f93ec0964d0 con 0x7f93f81015e0 2024-03-15T12:56:33.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.537+0000 7f9400046640 1 -- 172.21.15.2:0/1471925563 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f93f810ced0 con 0x7f93f81015e0 2024-03-15T12:56:33.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.537+0000 7f93f6ffd640 1 -- 172.21.15.2:0/1471925563 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f93ec01c020 con 0x7f93f81015e0 2024-03-15T12:56:33.524 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-15T12:56:33.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.539+0000 7f9400046640 1 -- 172.21.15.2:0/1471925563 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f93e00a1ab0 msgr2=0x7f93e00a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.539+0000 7f9400046640 1 --2- 172.21.15.2:0/1471925563 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f93e00a1ab0 0x7f93e00a3f70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f93f0006010 tx=0x7f93f0004e00 comp rx=0 tx=0).stop 2024-03-15T12:56:33.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.539+0000 7f9400046640 1 -- 172.21.15.2:0/1471925563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93f81015e0 msgr2=0x7f93f81092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.539+0000 7f9400046640 1 --2- 172.21.15.2:0/1471925563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93f81015e0 0x7f93f81092f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00e9f0 tx=0x7f93ec00eeb0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.539+0000 7f9400046640 1 -- 172.21.15.2:0/1471925563 shutdown_connections 2024-03-15T12:56:33.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.539+0000 7f9400046640 1 --2- 172.21.15.2:0/1471925563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93f8101fa0 0x7f93f8109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.539+0000 7f9400046640 1 --2- 172.21.15.2:0/1471925563 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f93e00a1ab0 0x7f93e00a3f70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.539+0000 7f9400046640 1 --2- 172.21.15.2:0/1471925563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93f81015e0 0x7f93f81092f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.539+0000 7f9400046640 1 --2- 172.21.15.2:0/1471925563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f81003f0 0x7f93f810acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.539+0000 7f9400046640 1 -- 172.21.15.2:0/1471925563 >> 172.21.15.2:0/1471925563 conn(0x7f93f80fb980 msgr2=0x7f93f8104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:33.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.539+0000 7f9400046640 1 -- 172.21.15.2:0/1471925563 shutdown_connections 2024-03-15T12:56:33.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.539+0000 7f9400046640 1 -- 172.21.15.2:0/1471925563 wait complete. 2024-03-15T12:56:33.536 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-15T12:56:33.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.656+0000 7f2aee963640 1 -- 172.21.15.2:0/327729514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ae8101500 msgr2=0x7f2ae8101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.656+0000 7f2aee963640 1 --2- 172.21.15.2:0/327729514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ae8101500 0x7f2ae8101920 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009900 tx=0x7f2ad802bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:33.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.656+0000 7f2aee963640 1 -- 172.21.15.2:0/327729514 shutdown_connections 2024-03-15T12:56:33.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.656+0000 7f2aee963640 1 --2- 172.21.15.2:0/327729514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ae81030b0 0x7f2ae8109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.656+0000 7f2aee963640 1 --2- 172.21.15.2:0/327729514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ae81026f0 0x7f2ae8102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.656+0000 7f2aee963640 1 --2- 172.21.15.2:0/327729514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ae8101500 0x7f2ae8101920 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.656+0000 7f2aee963640 1 -- 172.21.15.2:0/327729514 >> 172.21.15.2:0/327729514 conn(0x7f2ae80fca90 msgr2=0x7f2ae80feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:33.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.656+0000 7f2aee963640 1 -- 172.21.15.2:0/327729514 shutdown_connections 2024-03-15T12:56:33.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.656+0000 7f2aee963640 1 -- 172.21.15.2:0/327729514 wait complete. 2024-03-15T12:56:33.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.656+0000 7f2aee963640 1 Processor -- start 2024-03-15T12:56:33.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.656+0000 7f2aee963640 1 -- start start 2024-03-15T12:56:33.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2aee963640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ae8101500 0x7f2ae8197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2aee963640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ae81026f0 0x7f2ae8197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2aee963640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ae81030b0 0x7f2ae819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2aee963640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae819f100 con 0x7f2ae8101500 2024-03-15T12:56:33.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2aee963640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae819f270 con 0x7f2ae81030b0 2024-03-15T12:56:33.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2aee963640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae819f430 con 0x7f2ae81026f0 2024-03-15T12:56:33.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2ae7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ae8101500 0x7f2ae8197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2ae7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ae8101500 0x7f2ae8197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50724/0 (socket says 172.21.15.2:50724) 2024-03-15T12:56:33.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2ae7fff640 1 -- 172.21.15.2:0/3093057525 learned_addr learned my addr 172.21.15.2:0/3093057525 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:33.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2aecf00640 1 --2- 172.21.15.2:0/3093057525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ae81030b0 0x7f2ae819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2ae77fe640 1 --2- 172.21.15.2:0/3093057525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ae81026f0 0x7f2ae8197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2ae7fff640 1 -- 172.21.15.2:0/3093057525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ae81026f0 msgr2=0x7f2ae8197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2ae7fff640 1 --2- 172.21.15.2:0/3093057525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ae81026f0 0x7f2ae8197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2ae7fff640 1 -- 172.21.15.2:0/3093057525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ae81030b0 msgr2=0x7f2ae819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2ae7fff640 1 --2- 172.21.15.2:0/3093057525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ae81030b0 0x7f2ae819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2ae7fff640 1 -- 172.21.15.2:0/3093057525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad8009560 con 0x7f2ae8101500 2024-03-15T12:56:33.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2ae7fff640 1 --2- 172.21.15.2:0/3093057525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ae8101500 0x7f2ae8197610 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8000c00 tx=0x7f2ad8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:33.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2ae57fa640 1 -- 172.21.15.2:0/3093057525 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad80042d0 con 0x7f2ae8101500 2024-03-15T12:56:33.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2ae57fa640 1 -- 172.21.15.2:0/3093057525 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2ad8004430 con 0x7f2ae8101500 2024-03-15T12:56:33.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2aee963640 1 -- 172.21.15.2:0/3093057525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae819f6b0 con 0x7f2ae8101500 2024-03-15T12:56:33.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2ae57fa640 1 -- 172.21.15.2:0/3093057525 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8003e20 con 0x7f2ae8101500 2024-03-15T12:56:33.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.657+0000 7f2aee963640 1 -- 172.21.15.2:0/3093057525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae819fbc0 con 0x7f2ae8101500 2024-03-15T12:56:33.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.658+0000 7f2aee963640 1 -- 172.21.15.2:0/3093057525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae81088c0 con 0x7f2ae8101500 2024-03-15T12:56:33.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.659+0000 7f2ae57fa640 1 -- 172.21.15.2:0/3093057525 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f2ad80035f0 con 0x7f2ae8101500 2024-03-15T12:56:33.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.660+0000 7f2ae57fa640 1 --2- 172.21.15.2:0/3093057525 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2ad00a1a30 0x7f2ad00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.660+0000 7f2ae77fe640 1 --2- 172.21.15.2:0/3093057525 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2ad00a1a30 0x7f2ad00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.660+0000 7f2ae57fa640 1 -- 172.21.15.2:0/3093057525 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2ad80ef800 con 0x7f2ae8101500 2024-03-15T12:56:33.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.661+0000 7f2ae77fe640 1 --2- 172.21.15.2:0/3093057525 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2ad00a1a30 0x7f2ad00a3ef0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8198b20 tx=0x7f2adc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:33.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.665+0000 7f2ae57fa640 1 -- 172.21.15.2:0/3093057525 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2ad80b87d0 con 0x7f2ae8101500 2024-03-15T12:56:33.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:33 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2152362388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:33.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:33 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1471925563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:56:33.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.848+0000 7f2aee963640 1 -- 172.21.15.2:0/3093057525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ae802cf90 con 0x7f2ae8101500 2024-03-15T12:56:33.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.849+0000 7f2ae57fa640 1 -- 172.21.15.2:0/3093057525 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ad80bbe90 con 0x7f2ae8101500 2024-03-15T12:56:33.835 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:56:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.851+0000 7f2aee963640 1 -- 172.21.15.2:0/3093057525 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2ad00a1a30 msgr2=0x7f2ad00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.851+0000 7f2aee963640 1 --2- 172.21.15.2:0/3093057525 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2ad00a1a30 0x7f2ad00a3ef0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8198b20 tx=0x7f2adc005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.851+0000 7f2aee963640 1 -- 172.21.15.2:0/3093057525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ae8101500 msgr2=0x7f2ae8197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.851+0000 7f2aee963640 1 --2- 172.21.15.2:0/3093057525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ae8101500 0x7f2ae8197610 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8000c00 tx=0x7f2ad8004ab0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.851+0000 7f2aee963640 1 -- 172.21.15.2:0/3093057525 shutdown_connections 2024-03-15T12:56:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.851+0000 7f2aee963640 1 --2- 172.21.15.2:0/3093057525 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2ad00a1a30 0x7f2ad00a3ef0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.851+0000 7f2aee963640 1 --2- 172.21.15.2:0/3093057525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ae81030b0 0x7f2ae819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.851+0000 7f2aee963640 1 --2- 172.21.15.2:0/3093057525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ae81026f0 0x7f2ae8197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.851+0000 7f2aee963640 1 --2- 172.21.15.2:0/3093057525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ae8101500 0x7f2ae8197610 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.851+0000 7f2aee963640 1 -- 172.21.15.2:0/3093057525 >> 172.21.15.2:0/3093057525 conn(0x7f2ae80fca90 msgr2=0x7f2ae8105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.851+0000 7f2aee963640 1 -- 172.21.15.2:0/3093057525 shutdown_connections 2024-03-15T12:56:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.851+0000 7f2aee963640 1 -- 172.21.15.2:0/3093057525 wait complete. 2024-03-15T12:56:33.848 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-15T12:56:33.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.967+0000 7fde7fb3a640 1 -- 172.21.15.2:0/1443719646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde781003f0 msgr2=0x7fde78100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.967+0000 7fde7fb3a640 1 --2- 172.21.15.2:0/1443719646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde781003f0 0x7fde78100810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fde70009900 tx=0x7fde7002bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:33.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.967+0000 7fde7fb3a640 1 -- 172.21.15.2:0/1443719646 shutdown_connections 2024-03-15T12:56:33.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.967+0000 7fde7fb3a640 1 --2- 172.21.15.2:0/1443719646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde78101fa0 0x7fde78108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.967+0000 7fde7fb3a640 1 --2- 172.21.15.2:0/1443719646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde781015e0 0x7fde78101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.967+0000 7fde7fb3a640 1 --2- 172.21.15.2:0/1443719646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde781003f0 0x7fde78100810 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.967+0000 7fde7fb3a640 1 -- 172.21.15.2:0/1443719646 >> 172.21.15.2:0/1443719646 conn(0x7fde780fb980 msgr2=0x7fde780fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:33.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.967+0000 7fde7fb3a640 1 -- 172.21.15.2:0/1443719646 shutdown_connections 2024-03-15T12:56:33.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.967+0000 7fde7fb3a640 1 -- 172.21.15.2:0/1443719646 wait complete. 2024-03-15T12:56:33.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.968+0000 7fde7fb3a640 1 Processor -- start 2024-03-15T12:56:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.968+0000 7fde7fb3a640 1 -- start start 2024-03-15T12:56:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.968+0000 7fde7fb3a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde781003f0 0x7fde78104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.968+0000 7fde7fb3a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde781015e0 0x7fde781029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.968+0000 7fde7fb3a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde78101fa0 0x7fde78102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.968+0000 7fde7fb3a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde78103660 con 0x7fde78101fa0 2024-03-15T12:56:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.968+0000 7fde7fb3a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde781037d0 con 0x7fde781015e0 2024-03-15T12:56:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.968+0000 7fde7fb3a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde781a1660 con 0x7fde781003f0 2024-03-15T12:56:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.968+0000 7fde7e0d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde78101fa0 0x7fde78102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.968+0000 7fde7e0d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde78101fa0 0x7fde78102f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50734/0 (socket says 172.21.15.2:50734) 2024-03-15T12:56:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.968+0000 7fde7e0d7640 1 -- 172.21.15.2:0/1592353501 learned_addr learned my addr 172.21.15.2:0/1592353501 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.968+0000 7fde7d8d6640 1 --2- 172.21.15.2:0/1592353501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde781003f0 0x7fde78104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.968+0000 7fde7d0d5640 1 --2- 172.21.15.2:0/1592353501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde781015e0 0x7fde781029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.969+0000 7fde7e0d7640 1 -- 172.21.15.2:0/1592353501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde781003f0 msgr2=0x7fde78104380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.969+0000 7fde7e0d7640 1 --2- 172.21.15.2:0/1592353501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde781003f0 0x7fde78104380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.969+0000 7fde7e0d7640 1 -- 172.21.15.2:0/1592353501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde781015e0 msgr2=0x7fde781029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.969+0000 7fde7e0d7640 1 --2- 172.21.15.2:0/1592353501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde781015e0 0x7fde781029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.969+0000 7fde7e0d7640 1 -- 172.21.15.2:0/1592353501 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde70009560 con 0x7fde78101fa0 2024-03-15T12:56:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.969+0000 7fde7e0d7640 1 --2- 172.21.15.2:0/1592353501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde78101fa0 0x7fde78102f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fde7400ead0 tx=0x7fde7400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.969+0000 7fde6effd640 1 -- 172.21.15.2:0/1592353501 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7400cde0 con 0x7fde78101fa0 2024-03-15T12:56:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.969+0000 7fde6effd640 1 -- 172.21.15.2:0/1592353501 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fde74005020 con 0x7fde78101fa0 2024-03-15T12:56:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.969+0000 7fde7fb3a640 1 -- 172.21.15.2:0/1592353501 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde781a1910 con 0x7fde78101fa0 2024-03-15T12:56:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.969+0000 7fde6effd640 1 -- 172.21.15.2:0/1592353501 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde740116d0 con 0x7fde78101fa0 2024-03-15T12:56:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.969+0000 7fde7fb3a640 1 -- 172.21.15.2:0/1592353501 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde781a1dd0 con 0x7fde78101fa0 2024-03-15T12:56:33.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.970+0000 7fde7fb3a640 1 -- 172.21.15.2:0/1592353501 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde7810bdd0 con 0x7fde78101fa0 2024-03-15T12:56:33.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.971+0000 7fde6effd640 1 -- 172.21.15.2:0/1592353501 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fde74005590 con 0x7fde78101fa0 2024-03-15T12:56:33.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.971+0000 7fde6effd640 1 --2- 172.21.15.2:0/1592353501 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fde640a1a30 0x7fde640a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:33.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.972+0000 7fde6effd640 1 -- 172.21.15.2:0/1592353501 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fde740cd7a0 con 0x7fde78101fa0 2024-03-15T12:56:33.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.972+0000 7fde7d8d6640 1 --2- 172.21.15.2:0/1592353501 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fde640a1a30 0x7fde640a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:33.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.972+0000 7fde7d8d6640 1 --2- 172.21.15.2:0/1592353501 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fde640a1a30 0x7fde640a3ef0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fde7002c740 tx=0x7fde70005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:33.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:33.976+0000 7fde6effd640 1 -- 172.21.15.2:0/1592353501 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fde7409a350 con 0x7fde78101fa0 2024-03-15T12:56:34.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:33 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2152362388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:34.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:33 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1471925563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:56:34.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:33 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2152362388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:34.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:33 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1471925563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:56:34.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.159+0000 7fde7fb3a640 1 -- 172.21.15.2:0/1592353501 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fde781a2330 con 0x7fde78101fa0 2024-03-15T12:56:34.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.159+0000 7fde6effd640 1 -- 172.21.15.2:0/1592353501 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fde74005190 con 0x7fde78101fa0 2024-03-15T12:56:34.146 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-03-15T12:56:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.162+0000 7fde7fb3a640 1 -- 172.21.15.2:0/1592353501 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fde640a1a30 msgr2=0x7fde640a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.162+0000 7fde7fb3a640 1 --2- 172.21.15.2:0/1592353501 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fde640a1a30 0x7fde640a3ef0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fde7002c740 tx=0x7fde70005fd0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.162+0000 7fde7fb3a640 1 -- 172.21.15.2:0/1592353501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde78101fa0 msgr2=0x7fde78102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.162+0000 7fde7fb3a640 1 --2- 172.21.15.2:0/1592353501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde78101fa0 0x7fde78102f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fde7400ead0 tx=0x7fde7400ef90 comp rx=0 tx=0).stop 2024-03-15T12:56:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.162+0000 7fde7fb3a640 1 -- 172.21.15.2:0/1592353501 shutdown_connections 2024-03-15T12:56:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.162+0000 7fde7fb3a640 1 --2- 172.21.15.2:0/1592353501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde78101fa0 0x7fde78102f00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.162+0000 7fde7fb3a640 1 --2- 172.21.15.2:0/1592353501 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fde640a1a30 0x7fde640a3ef0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.162+0000 7fde7fb3a640 1 --2- 172.21.15.2:0/1592353501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde781015e0 0x7fde781029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.162+0000 7fde7fb3a640 1 --2- 172.21.15.2:0/1592353501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde781003f0 0x7fde78104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.162+0000 7fde7fb3a640 1 -- 172.21.15.2:0/1592353501 >> 172.21.15.2:0/1592353501 conn(0x7fde780fb980 msgr2=0x7fde78068740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.162+0000 7fde7fb3a640 1 -- 172.21.15.2:0/1592353501 shutdown_connections 2024-03-15T12:56:34.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.162+0000 7fde7fb3a640 1 -- 172.21.15.2:0/1592353501 wait complete. 2024-03-15T12:56:34.158 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-15T12:56:34.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.277+0000 7f264579a640 1 -- 172.21.15.2:0/447774617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2640101fa0 msgr2=0x7f2640108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:34.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.277+0000 7f264579a640 1 --2- 172.21.15.2:0/447774617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2640101fa0 0x7f2640108830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2630009ca0 tx=0x7f263002c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.277+0000 7f264579a640 1 -- 172.21.15.2:0/447774617 shutdown_connections 2024-03-15T12:56:34.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.277+0000 7f264579a640 1 --2- 172.21.15.2:0/447774617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2640101fa0 0x7f2640108830 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.277+0000 7f264579a640 1 --2- 172.21.15.2:0/447774617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26401015e0 0x7f2640101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.277+0000 7f264579a640 1 --2- 172.21.15.2:0/447774617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26401003f0 0x7f2640100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.277+0000 7f264579a640 1 -- 172.21.15.2:0/447774617 >> 172.21.15.2:0/447774617 conn(0x7f26400fb980 msgr2=0x7f26400fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:34.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.277+0000 7f264579a640 1 -- 172.21.15.2:0/447774617 shutdown_connections 2024-03-15T12:56:34.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.277+0000 7f264579a640 1 -- 172.21.15.2:0/447774617 wait complete. 2024-03-15T12:56:34.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.278+0000 7f264579a640 1 Processor -- start 2024-03-15T12:56:34.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.278+0000 7f264579a640 1 -- start start 2024-03-15T12:56:34.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.278+0000 7f264579a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26401003f0 0x7f2640104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:34.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.278+0000 7f264579a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26401015e0 0x7f26401029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:34.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.278+0000 7f264579a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2640101fa0 0x7f2640102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:34.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.278+0000 7f264579a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2640103660 con 0x7f26401003f0 2024-03-15T12:56:34.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.278+0000 7f264579a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26401037d0 con 0x7f2640101fa0 2024-03-15T12:56:34.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.278+0000 7f264579a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26401a15a0 con 0x7f26401015e0 2024-03-15T12:56:34.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.278+0000 7f263effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26401003f0 0x7f2640104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:34.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.278+0000 7f263effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26401003f0 0x7f2640104380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50746/0 (socket says 172.21.15.2:50746) 2024-03-15T12:56:34.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.279+0000 7f263effd640 1 -- 172.21.15.2:0/510740248 learned_addr learned my addr 172.21.15.2:0/510740248 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:34.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.279+0000 7f263f7fe640 1 --2- 172.21.15.2:0/510740248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2640101fa0 0x7f2640102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:34.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.279+0000 7f263e7fc640 1 --2- 172.21.15.2:0/510740248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26401015e0 0x7f26401029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:34.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.279+0000 7f263effd640 1 -- 172.21.15.2:0/510740248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26401015e0 msgr2=0x7f26401029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:34.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.279+0000 7f263effd640 1 --2- 172.21.15.2:0/510740248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26401015e0 0x7f26401029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.279+0000 7f263effd640 1 -- 172.21.15.2:0/510740248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2640101fa0 msgr2=0x7f2640102f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:34.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.279+0000 7f263effd640 1 --2- 172.21.15.2:0/510740248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2640101fa0 0x7f2640102f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.279+0000 7f263effd640 1 -- 172.21.15.2:0/510740248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2628009490 con 0x7f26401003f0 2024-03-15T12:56:34.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.279+0000 7f263effd640 1 --2- 172.21.15.2:0/510740248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26401003f0 0x7f2640104380 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f263800d7a0 tx=0x7f263800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:34.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.279+0000 7f2637fff640 1 -- 172.21.15.2:0/510740248 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2638009c30 con 0x7f26401003f0 2024-03-15T12:56:34.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.279+0000 7f2637fff640 1 -- 172.21.15.2:0/510740248 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f263800deb0 con 0x7f26401003f0 2024-03-15T12:56:34.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.279+0000 7f264579a640 1 -- 172.21.15.2:0/510740248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2630009900 con 0x7f26401003f0 2024-03-15T12:56:34.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.279+0000 7f2637fff640 1 -- 172.21.15.2:0/510740248 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2638003e50 con 0x7f26401003f0 2024-03-15T12:56:34.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.279+0000 7f264579a640 1 -- 172.21.15.2:0/510740248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26401a1bd0 con 0x7f26401003f0 2024-03-15T12:56:34.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.280+0000 7f264579a640 1 -- 172.21.15.2:0/510740248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f264010bdd0 con 0x7f26401003f0 2024-03-15T12:56:34.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.281+0000 7f2637fff640 1 -- 172.21.15.2:0/510740248 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f2638004000 con 0x7f26401003f0 2024-03-15T12:56:34.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.282+0000 7f2637fff640 1 --2- 172.21.15.2:0/510740248 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f262c0a1a30 0x7f262c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:34.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.282+0000 7f263e7fc640 1 --2- 172.21.15.2:0/510740248 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f262c0a1a30 0x7f262c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:34.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.282+0000 7f2637fff640 1 -- 172.21.15.2:0/510740248 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f26380cd340 con 0x7f26401003f0 2024-03-15T12:56:34.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.282+0000 7f263e7fc640 1 --2- 172.21.15.2:0/510740248 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f262c0a1a30 0x7f262c0a3ef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f26401040f0 tx=0x7f2628005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:34.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.285+0000 7f2637fff640 1 -- 172.21.15.2:0/510740248 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2638096360 con 0x7f26401003f0 2024-03-15T12:56:34.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.466+0000 7f264579a640 1 -- 172.21.15.2:0/510740248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f26401a1df0 con 0x7f26401003f0 2024-03-15T12:56:34.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.467+0000 7f2637fff640 1 -- 172.21.15.2:0/510740248 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f26380170a0 con 0x7f26401003f0 2024-03-15T12:56:34.453 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:56:34.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.469+0000 7f264579a640 1 -- 172.21.15.2:0/510740248 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f262c0a1a30 msgr2=0x7f262c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:34.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.469+0000 7f264579a640 1 --2- 172.21.15.2:0/510740248 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f262c0a1a30 0x7f262c0a3ef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f26401040f0 tx=0x7f2628005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:34.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.469+0000 7f264579a640 1 -- 172.21.15.2:0/510740248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26401003f0 msgr2=0x7f2640104380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:34.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.469+0000 7f264579a640 1 --2- 172.21.15.2:0/510740248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26401003f0 0x7f2640104380 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f263800d7a0 tx=0x7f263800dc60 comp rx=0 tx=0).stop 2024-03-15T12:56:34.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.469+0000 7f264579a640 1 -- 172.21.15.2:0/510740248 shutdown_connections 2024-03-15T12:56:34.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.469+0000 7f264579a640 1 --2- 172.21.15.2:0/510740248 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f262c0a1a30 0x7f262c0a3ef0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.469+0000 7f264579a640 1 --2- 172.21.15.2:0/510740248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2640101fa0 0x7f2640102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.469+0000 7f264579a640 1 --2- 172.21.15.2:0/510740248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26401015e0 0x7f26401029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.469+0000 7f264579a640 1 --2- 172.21.15.2:0/510740248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26401003f0 0x7f2640104380 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.469+0000 7f264579a640 1 -- 172.21.15.2:0/510740248 >> 172.21.15.2:0/510740248 conn(0x7f26400fb980 msgr2=0x7f26400fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:34.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.469+0000 7f264579a640 1 -- 172.21.15.2:0/510740248 shutdown_connections 2024-03-15T12:56:34.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.469+0000 7f264579a640 1 -- 172.21.15.2:0/510740248 wait complete. 2024-03-15T12:56:34.466 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-15T12:56:34.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.583+0000 7fe8e25a2640 1 -- 172.21.15.2:0/1530974555 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8dc1014d0 msgr2=0x7fe8dc101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:34.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.583+0000 7fe8e25a2640 1 --2- 172.21.15.2:0/1530974555 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8dc1014d0 0x7fe8dc101950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009900 tx=0x7fe8cc02bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:34.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.583+0000 7fe8e25a2640 1 -- 172.21.15.2:0/1530974555 shutdown_connections 2024-03-15T12:56:34.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.583+0000 7fe8e25a2640 1 --2- 172.21.15.2:0/1530974555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8dc101e90 0x7fe8dc108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.583+0000 7fe8e25a2640 1 --2- 172.21.15.2:0/1530974555 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8dc1014d0 0x7fe8dc101950 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.583+0000 7fe8e25a2640 1 --2- 172.21.15.2:0/1530974555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8dc1002e0 0x7fe8dc100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.583+0000 7fe8e25a2640 1 -- 172.21.15.2:0/1530974555 >> 172.21.15.2:0/1530974555 conn(0x7fe8dc0fb870 msgr2=0x7fe8dc0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:34.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.583+0000 7fe8e25a2640 1 -- 172.21.15.2:0/1530974555 shutdown_connections 2024-03-15T12:56:34.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.583+0000 7fe8e25a2640 1 -- 172.21.15.2:0/1530974555 wait complete. 2024-03-15T12:56:34.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e25a2640 1 Processor -- start 2024-03-15T12:56:34.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e25a2640 1 -- start start 2024-03-15T12:56:34.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e25a2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8dc1002e0 0x7fe8dc197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:34.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e25a2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8dc1014d0 0x7fe8dc1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:34.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e25a2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8dc101e90 0x7fe8dc19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:34.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e25a2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc19ef50 con 0x7fe8dc1014d0 2024-03-15T12:56:34.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e25a2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc19f0c0 con 0x7fe8dc1002e0 2024-03-15T12:56:34.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e25a2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc19f280 con 0x7fe8dc101e90 2024-03-15T12:56:34.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e0d9f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8dc1014d0 0x7fe8dc1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:34.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e0d9f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8dc1014d0 0x7fe8dc1979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50752/0 (socket says 172.21.15.2:50752) 2024-03-15T12:56:34.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e0d9f640 1 -- 172.21.15.2:0/2844767538 learned_addr learned my addr 172.21.15.2:0/2844767538 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:34.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e15a0640 1 --2- 172.21.15.2:0/2844767538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8dc1002e0 0x7fe8dc197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:34.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e1da1640 1 --2- 172.21.15.2:0/2844767538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8dc101e90 0x7fe8dc19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:34.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e0d9f640 1 -- 172.21.15.2:0/2844767538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8dc101e90 msgr2=0x7fe8dc19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:34.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e0d9f640 1 --2- 172.21.15.2:0/2844767538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8dc101e90 0x7fe8dc19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e0d9f640 1 -- 172.21.15.2:0/2844767538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8dc1002e0 msgr2=0x7fe8dc197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:34.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e0d9f640 1 --2- 172.21.15.2:0/2844767538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8dc1002e0 0x7fe8dc197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.584+0000 7fe8e0d9f640 1 -- 172.21.15.2:0/2844767538 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc009560 con 0x7fe8dc1014d0 2024-03-15T12:56:34.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.585+0000 7fe8e0d9f640 1 --2- 172.21.15.2:0/2844767538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8dc1014d0 0x7fe8dc1979a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc005ab0 tx=0x7fe8cc004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:34.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.585+0000 7fe8d27fc640 1 -- 172.21.15.2:0/2844767538 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0042d0 con 0x7fe8dc1014d0 2024-03-15T12:56:34.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.585+0000 7fe8d27fc640 1 -- 172.21.15.2:0/2844767538 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe8cc004430 con 0x7fe8dc1014d0 2024-03-15T12:56:34.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.585+0000 7fe8e25a2640 1 -- 172.21.15.2:0/2844767538 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8dc19f500 con 0x7fe8dc1014d0 2024-03-15T12:56:34.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.585+0000 7fe8d27fc640 1 -- 172.21.15.2:0/2844767538 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc003e20 con 0x7fe8dc1014d0 2024-03-15T12:56:34.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.585+0000 7fe8e25a2640 1 -- 172.21.15.2:0/2844767538 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc19fa10 con 0x7fe8dc1014d0 2024-03-15T12:56:34.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.586+0000 7fe8e25a2640 1 -- 172.21.15.2:0/2844767538 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8dc10bcc0 con 0x7fe8dc1014d0 2024-03-15T12:56:34.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.587+0000 7fe8d27fc640 1 -- 172.21.15.2:0/2844767538 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fe8cc0035f0 con 0x7fe8dc1014d0 2024-03-15T12:56:34.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.587+0000 7fe8d27fc640 1 --2- 172.21.15.2:0/2844767538 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe8c80a1a30 0x7fe8c80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:34.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.587+0000 7fe8e15a0640 1 --2- 172.21.15.2:0/2844767538 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe8c80a1a30 0x7fe8c80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:34.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.587+0000 7fe8d27fc640 1 -- 172.21.15.2:0/2844767538 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe8cc0ef830 con 0x7fe8dc1014d0 2024-03-15T12:56:34.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.588+0000 7fe8e15a0640 1 --2- 172.21.15.2:0/2844767538 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe8c80a1a30 0x7fe8c80a3ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc101330 tx=0x7fe8d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:34.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.592+0000 7fe8d27fc640 1 -- 172.21.15.2:0/2844767538 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe8cc0b8800 con 0x7fe8dc1014d0 2024-03-15T12:56:34.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.787+0000 7fe8e25a2640 1 -- 172.21.15.2:0/2844767538 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe8dc10be50 con 0x7fe8dc1014d0 2024-03-15T12:56:34.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.787+0000 7fe8d27fc640 1 -- 172.21.15.2:0/2844767538 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+579 (secure 0 0 0) 0x7fe8cc0bbec0 con 0x7fe8dc1014d0 2024-03-15T12:56:34.774 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3349070582 2024-03-16T12:49:18.658825+0000 2024-03-15T12:56:34.774 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3269509343 2024-03-16T12:49:18.658825+0000 2024-03-15T12:56:34.774 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3269509343 2024-03-16T12:49:18.658825+0000 2024-03-15T12:56:34.774 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/656434628 2024-03-16T12:49:18.658825+0000 2024-03-15T12:56:34.774 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/84615744 2024-03-16T12:48:51.791072+0000 2024-03-15T12:56:34.774 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1464336909 2024-03-16T12:48:51.791072+0000 2024-03-15T12:56:34.774 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3546094985 2024-03-16T12:48:51.791072+0000 2024-03-15T12:56:34.774 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3111414481 2024-03-16T12:48:51.791072+0000 2024-03-15T12:56:34.774 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3111414481 2024-03-16T12:48:51.791072+0000 2024-03-15T12:56:34.774 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3673716509 2024-03-16T12:49:18.658825+0000 2024-03-15T12:56:34.775 INFO:teuthology.orchestra.run.smithi002.stderr:listed 10 entries 2024-03-15T12:56:34.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.789+0000 7fe8e25a2640 1 -- 172.21.15.2:0/2844767538 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe8c80a1a30 msgr2=0x7fe8c80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:34.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.789+0000 7fe8e25a2640 1 --2- 172.21.15.2:0/2844767538 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe8c80a1a30 0x7fe8c80a3ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc101330 tx=0x7fe8d8005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:34.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.789+0000 7fe8e25a2640 1 -- 172.21.15.2:0/2844767538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8dc1014d0 msgr2=0x7fe8dc1979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:34.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.789+0000 7fe8e25a2640 1 --2- 172.21.15.2:0/2844767538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8dc1014d0 0x7fe8dc1979a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc005ab0 tx=0x7fe8cc004e50 comp rx=0 tx=0).stop 2024-03-15T12:56:34.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.789+0000 7fe8e25a2640 1 -- 172.21.15.2:0/2844767538 shutdown_connections 2024-03-15T12:56:34.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.789+0000 7fe8e25a2640 1 --2- 172.21.15.2:0/2844767538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8dc101e90 0x7fe8dc19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.789+0000 7fe8e25a2640 1 --2- 172.21.15.2:0/2844767538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8dc1014d0 0x7fe8dc1979a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.789+0000 7fe8e25a2640 1 --2- 172.21.15.2:0/2844767538 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe8c80a1a30 0x7fe8c80a3ef0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.789+0000 7fe8e25a2640 1 --2- 172.21.15.2:0/2844767538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8dc1002e0 0x7fe8dc197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.789+0000 7fe8e25a2640 1 -- 172.21.15.2:0/2844767538 >> 172.21.15.2:0/2844767538 conn(0x7fe8dc0fb870 msgr2=0x7fe8dc0fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:34.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.790+0000 7fe8e25a2640 1 -- 172.21.15.2:0/2844767538 shutdown_connections 2024-03-15T12:56:34.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.790+0000 7fe8e25a2640 1 -- 172.21.15.2:0/2844767538 wait complete. 2024-03-15T12:56:34.786 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:56:34.786 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-15T12:56:34.803 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:56:34.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.959+0000 7fc48f1c7640 1 -- 172.21.15.2:0/2508284258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc488101600 msgr2=0x7fc488101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:34.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.959+0000 7fc48f1c7640 1 --2- 172.21.15.2:0/2508284258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc488101600 0x7fc488101a80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009900 tx=0x7fc47c02bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:34.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.959+0000 7fc48f1c7640 1 -- 172.21.15.2:0/2508284258 shutdown_connections 2024-03-15T12:56:34.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.959+0000 7fc48f1c7640 1 --2- 172.21.15.2:0/2508284258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc488101fc0 0x7fc488108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.959+0000 7fc48f1c7640 1 --2- 172.21.15.2:0/2508284258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc488101600 0x7fc488101a80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.959+0000 7fc48f1c7640 1 --2- 172.21.15.2:0/2508284258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc488100410 0x7fc488100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.959+0000 7fc48f1c7640 1 -- 172.21.15.2:0/2508284258 >> 172.21.15.2:0/2508284258 conn(0x7fc4880fb980 msgr2=0x7fc4880fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:34.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.960+0000 7fc48f1c7640 1 -- 172.21.15.2:0/2508284258 shutdown_connections 2024-03-15T12:56:34.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.960+0000 7fc48f1c7640 1 -- 172.21.15.2:0/2508284258 wait complete. 2024-03-15T12:56:34.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.960+0000 7fc48f1c7640 1 Processor -- start 2024-03-15T12:56:34.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.960+0000 7fc48f1c7640 1 -- start start 2024-03-15T12:56:34.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.960+0000 7fc48f1c7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc488100410 0x7fc4881975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:34.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.960+0000 7fc48f1c7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc488101600 0x7fc488197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:34.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.960+0000 7fc48f1c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc488101fc0 0x7fc48819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:34.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.960+0000 7fc48f1c7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48819f090 con 0x7fc488101fc0 2024-03-15T12:56:34.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.960+0000 7fc48f1c7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48819f200 con 0x7fc488100410 2024-03-15T12:56:34.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.960+0000 7fc48f1c7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48819f3c0 con 0x7fc488101600 2024-03-15T12:56:34.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.960+0000 7fc48d764640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc488101fc0 0x7fc48819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:34.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.960+0000 7fc48d764640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc488101fc0 0x7fc48819eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50762/0 (socket says 172.21.15.2:50762) 2024-03-15T12:56:34.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.960+0000 7fc48d764640 1 -- 172.21.15.2:0/1360608270 learned_addr learned my addr 172.21.15.2:0/1360608270 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.960+0000 7fc48cf63640 1 --2- 172.21.15.2:0/1360608270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc488100410 0x7fc4881975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.960+0000 7fc487fff640 1 --2- 172.21.15.2:0/1360608270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc488101600 0x7fc488197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.961+0000 7fc48d764640 1 -- 172.21.15.2:0/1360608270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc488101600 msgr2=0x7fc488197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.961+0000 7fc48d764640 1 --2- 172.21.15.2:0/1360608270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc488101600 0x7fc488197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.961+0000 7fc48d764640 1 -- 172.21.15.2:0/1360608270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc488100410 msgr2=0x7fc4881975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.961+0000 7fc48d764640 1 --2- 172.21.15.2:0/1360608270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc488100410 0x7fc4881975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.961+0000 7fc48d764640 1 -- 172.21.15.2:0/1360608270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c009560 con 0x7fc488101fc0 2024-03-15T12:56:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.961+0000 7fc48d764640 1 --2- 172.21.15.2:0/1360608270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc488101fc0 0x7fc48819eb50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc47800e970 tx=0x7fc47800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.961+0000 7fc485ffb640 1 -- 172.21.15.2:0/1360608270 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478009720 con 0x7fc488101fc0 2024-03-15T12:56:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.961+0000 7fc485ffb640 1 -- 172.21.15.2:0/1360608270 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc4780054f0 con 0x7fc488101fc0 2024-03-15T12:56:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.961+0000 7fc48f1c7640 1 -- 172.21.15.2:0/1360608270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48819f6a0 con 0x7fc488101fc0 2024-03-15T12:56:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.961+0000 7fc485ffb640 1 -- 172.21.15.2:0/1360608270 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478010430 con 0x7fc488101fc0 2024-03-15T12:56:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.961+0000 7fc48f1c7640 1 -- 172.21.15.2:0/1360608270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48819fb80 con 0x7fc488101fc0 2024-03-15T12:56:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.962+0000 7fc48f1c7640 1 -- 172.21.15.2:0/1360608270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc488107730 con 0x7fc488101fc0 2024-03-15T12:56:34.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.963+0000 7fc485ffb640 1 -- 172.21.15.2:0/1360608270 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fc478004f90 con 0x7fc488101fc0 2024-03-15T12:56:34.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.964+0000 7fc485ffb640 1 --2- 172.21.15.2:0/1360608270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fc4700a1ab0 0x7fc4700a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:34.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.964+0000 7fc48cf63640 1 --2- 172.21.15.2:0/1360608270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fc4700a1ab0 0x7fc4700a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:34.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.964+0000 7fc485ffb640 1 -- 172.21.15.2:0/1360608270 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc478014070 con 0x7fc488101fc0 2024-03-15T12:56:34.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.964+0000 7fc48cf63640 1 --2- 172.21.15.2:0/1360608270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fc4700a1ab0 0x7fc4700a3f70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc488101460 tx=0x7fc480009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:34.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:34.968+0000 7fc485ffb640 1 -- 172.21.15.2:0/1360608270 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc478096a90 con 0x7fc488101fc0 2024-03-15T12:56:35.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:34 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3093057525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:56:35.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:34 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1592353501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:56:35.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:34 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/510740248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:56:35.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:34 smithi002 ceph-mon[32060]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:35.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:34 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2844767538' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-15T12:56:35.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:34 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3093057525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:56:35.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:34 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1592353501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:56:35.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:34 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/510740248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:56:35.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:34 smithi184 ceph-mon[35993]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:35.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:34 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2844767538' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-15T12:56:35.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:34 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3093057525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:56:35.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:34 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1592353501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:56:35.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:34 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/510740248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:56:35.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:34 smithi149 ceph-mon[35927]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:56:35.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:34 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2844767538' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-15T12:56:35.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.150+0000 7fc48f1c7640 1 -- 172.21.15.2:0/1360608270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc48802d000 con 0x7fc488101fc0 2024-03-15T12:56:35.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.152+0000 7fc485ffb640 1 -- 172.21.15.2:0/1360608270 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24045 (secure 0 0 0) 0x7fc47809a150 con 0x7fc488101fc0 2024-03-15T12:56:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:35.139 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":89,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:56:25.830426+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:56:35.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.154+0000 7fc48f1c7640 1 -- 172.21.15.2:0/1360608270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fc4700a1ab0 msgr2=0x7fc4700a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:35.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.154+0000 7fc48f1c7640 1 --2- 172.21.15.2:0/1360608270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fc4700a1ab0 0x7fc4700a3f70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc488101460 tx=0x7fc480009500 comp rx=0 tx=0).stop 2024-03-15T12:56:35.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.154+0000 7fc48f1c7640 1 -- 172.21.15.2:0/1360608270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc488101fc0 msgr2=0x7fc48819eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:35.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.154+0000 7fc48f1c7640 1 --2- 172.21.15.2:0/1360608270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc488101fc0 0x7fc48819eb50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc47800e970 tx=0x7fc47800ed70 comp rx=0 tx=0).stop 2024-03-15T12:56:35.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.154+0000 7fc48f1c7640 1 -- 172.21.15.2:0/1360608270 shutdown_connections 2024-03-15T12:56:35.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.154+0000 7fc48f1c7640 1 --2- 172.21.15.2:0/1360608270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc488101fc0 0x7fc48819eb50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.154+0000 7fc48f1c7640 1 --2- 172.21.15.2:0/1360608270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc488101600 0x7fc488197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.154+0000 7fc48f1c7640 1 --2- 172.21.15.2:0/1360608270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fc4700a1ab0 0x7fc4700a3f70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.154+0000 7fc48f1c7640 1 --2- 172.21.15.2:0/1360608270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc488100410 0x7fc4881975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.154+0000 7fc48f1c7640 1 -- 172.21.15.2:0/1360608270 >> 172.21.15.2:0/1360608270 conn(0x7fc4880fb980 msgr2=0x7fc488104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:35.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.154+0000 7fc48f1c7640 1 -- 172.21.15.2:0/1360608270 shutdown_connections 2024-03-15T12:56:35.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.154+0000 7fc48f1c7640 1 -- 172.21.15.2:0/1360608270 wait complete. 2024-03-15T12:56:35.151 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-15T12:51:47.004843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-15T12:56:18.559882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-15T12:56:19.751767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-15T12:56:23.664879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-15T12:56:35.152 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-15T12:56:35.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.271+0000 7f0994c8f640 1 -- 172.21.15.2:0/4104962899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0990101060 msgr2=0x7f09901014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:35.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.271+0000 7f0994c8f640 1 --2- 172.21.15.2:0/4104962899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0990101060 0x7f09901014e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0988009ca0 tx=0x7f098802c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.271+0000 7f0994c8f640 1 -- 172.21.15.2:0/4104962899 shutdown_connections 2024-03-15T12:56:35.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.271+0000 7f0994c8f640 1 --2- 172.21.15.2:0/4104962899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0990101060 0x7f09901014e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.271+0000 7f0994c8f640 1 --2- 172.21.15.2:0/4104962899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09901006c0 0x7f0990100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.271+0000 7f0994c8f640 1 --2- 172.21.15.2:0/4104962899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09900ffd60 0x7f0990100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.271+0000 7f0994c8f640 1 -- 172.21.15.2:0/4104962899 >> 172.21.15.2:0/4104962899 conn(0x7f09900fb980 msgr2=0x7f09900fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:35.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.271+0000 7f0994c8f640 1 -- 172.21.15.2:0/4104962899 shutdown_connections 2024-03-15T12:56:35.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.271+0000 7f0994c8f640 1 -- 172.21.15.2:0/4104962899 wait complete. 2024-03-15T12:56:35.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.271+0000 7f0994c8f640 1 Processor -- start 2024-03-15T12:56:35.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f0994c8f640 1 -- start start 2024-03-15T12:56:35.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f0994c8f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09900ffd60 0x7f09901996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:35.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f0994c8f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09901006c0 0x7f0990199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:35.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f0994c8f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0990101060 0x7f09901a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:35.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f0994c8f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09901a11e0 con 0x7f09900ffd60 2024-03-15T12:56:35.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f0994c8f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09901a1350 con 0x7f09901006c0 2024-03-15T12:56:35.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f0994c8f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09901a1510 con 0x7f0990101060 2024-03-15T12:56:35.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f098e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09900ffd60 0x7f09901996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:35.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f098e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09900ffd60 0x7f09901996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50782/0 (socket says 172.21.15.2:50782) 2024-03-15T12:56:35.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f098e59c640 1 -- 172.21.15.2:0/3501980344 learned_addr learned my addr 172.21.15.2:0/3501980344 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:35.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f0987fff640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09901006c0 0x7f0990199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:35.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f098ed9d640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0990101060 0x7f09901a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:35.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f098e59c640 1 -- 172.21.15.2:0/3501980344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0990101060 msgr2=0x7f09901a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:35.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f098e59c640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0990101060 0x7f09901a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f098e59c640 1 -- 172.21.15.2:0/3501980344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09901006c0 msgr2=0x7f0990199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:35.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f098e59c640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09901006c0 0x7f0990199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f098e59c640 1 -- 172.21.15.2:0/3501980344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0988009900 con 0x7f09900ffd60 2024-03-15T12:56:35.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f098ed9d640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0990101060 0x7f09901a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T12:56:35.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f0987fff640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09901006c0 0x7f0990199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:35.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f098e59c640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09900ffd60 0x7f09901996f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f098000c920 tx=0x7f098000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:35.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.272+0000 7f09877fe640 1 -- 172.21.15.2:0/3501980344 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0980007c70 con 0x7f09900ffd60 2024-03-15T12:56:35.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.273+0000 7f0994c8f640 1 -- 172.21.15.2:0/3501980344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09901a17f0 con 0x7f09900ffd60 2024-03-15T12:56:35.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.273+0000 7f09877fe640 1 -- 172.21.15.2:0/3501980344 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0980007dd0 con 0x7f09900ffd60 2024-03-15T12:56:35.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.273+0000 7f09877fe640 1 -- 172.21.15.2:0/3501980344 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0980018820 con 0x7f09900ffd60 2024-03-15T12:56:35.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.273+0000 7f0994c8f640 1 -- 172.21.15.2:0/3501980344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09901a1d50 con 0x7f09900ffd60 2024-03-15T12:56:35.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.274+0000 7f0994c8f640 1 -- 172.21.15.2:0/3501980344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099010ac20 con 0x7f09900ffd60 2024-03-15T12:56:35.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.275+0000 7f09877fe640 1 -- 172.21.15.2:0/3501980344 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f0980018980 con 0x7f09900ffd60 2024-03-15T12:56:35.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.275+0000 7f09877fe640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f097c0a1a30 0x7f097c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:35.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.275+0000 7f0987fff640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f097c0a1a30 0x7f097c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:35.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.275+0000 7f09877fe640 1 -- 172.21.15.2:0/3501980344 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f09800cda70 con 0x7f09900ffd60 2024-03-15T12:56:35.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.276+0000 7f0987fff640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f097c0a1a30 0x7f097c0a3ef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f099019ac00 tx=0x7f0978005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:35.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.279+0000 7f09877fe640 1 -- 172.21.15.2:0/3501980344 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0980096ad0 con 0x7f09900ffd60 2024-03-15T12:56:35.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.461+0000 7f0994c8f640 1 -- 172.21.15.2:0/3501980344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09901a1980 con 0x7f09900ffd60 2024-03-15T12:56:35.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.462+0000 7f09877fe640 1 -- 172.21.15.2:0/3501980344 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09901a1980 con 0x7f09900ffd60 2024-03-15T12:56:35.448 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-15T12:56:35.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.464+0000 7f0994c8f640 1 -- 172.21.15.2:0/3501980344 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f097c0a1a30 msgr2=0x7f097c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:35.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.464+0000 7f0994c8f640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f097c0a1a30 0x7f097c0a3ef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f099019ac00 tx=0x7f0978005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:35.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.464+0000 7f0994c8f640 1 -- 172.21.15.2:0/3501980344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09900ffd60 msgr2=0x7f09901996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:35.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.464+0000 7f0994c8f640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09900ffd60 0x7f09901996f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f098000c920 tx=0x7f098000cde0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.464+0000 7f0994c8f640 1 -- 172.21.15.2:0/3501980344 shutdown_connections 2024-03-15T12:56:35.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.464+0000 7f0994c8f640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0990101060 0x7f09901a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.464+0000 7f0994c8f640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f097c0a1a30 0x7f097c0a3ef0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.464+0000 7f0994c8f640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09901006c0 0x7f0990199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.464+0000 7f0994c8f640 1 --2- 172.21.15.2:0/3501980344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09900ffd60 0x7f09901996f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.464+0000 7f0994c8f640 1 -- 172.21.15.2:0/3501980344 >> 172.21.15.2:0/3501980344 conn(0x7f09900fb980 msgr2=0x7f09900fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:35.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.464+0000 7f0994c8f640 1 -- 172.21.15.2:0/3501980344 shutdown_connections 2024-03-15T12:56:35.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.464+0000 7f0994c8f640 1 -- 172.21.15.2:0/3501980344 wait complete. 2024-03-15T12:56:35.461 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-15T12:56:35.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.580+0000 7f90a6793640 1 -- 172.21.15.2:0/2493850009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a01015e0 msgr2=0x7f90a0101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:35.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.580+0000 7f90a6793640 1 --2- 172.21.15.2:0/2493850009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a01015e0 0x7f90a0101a60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9094009900 tx=0x7f909402bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:35.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.580+0000 7f90a6793640 1 -- 172.21.15.2:0/2493850009 shutdown_connections 2024-03-15T12:56:35.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.580+0000 7f90a6793640 1 --2- 172.21.15.2:0/2493850009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90a0101fa0 0x7f90a0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.580+0000 7f90a6793640 1 --2- 172.21.15.2:0/2493850009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a01015e0 0x7f90a0101a60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.580+0000 7f90a6793640 1 --2- 172.21.15.2:0/2493850009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90a01003f0 0x7f90a0100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.580+0000 7f90a6793640 1 -- 172.21.15.2:0/2493850009 >> 172.21.15.2:0/2493850009 conn(0x7f90a00fb980 msgr2=0x7f90a00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:35.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.580+0000 7f90a6793640 1 -- 172.21.15.2:0/2493850009 shutdown_connections 2024-03-15T12:56:35.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.580+0000 7f90a6793640 1 -- 172.21.15.2:0/2493850009 wait complete. 2024-03-15T12:56:35.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.580+0000 7f90a6793640 1 Processor -- start 2024-03-15T12:56:35.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.580+0000 7f90a6793640 1 -- start start 2024-03-15T12:56:35.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f90a6793640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a01003f0 0x7f90a0197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:35.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f90a6793640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90a01015e0 0x7f90a0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:35.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f90a6793640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90a0101fa0 0x7f90a019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:35.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f90a6793640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a019f060 con 0x7f90a01003f0 2024-03-15T12:56:35.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f90a6793640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a019f1d0 con 0x7f90a01015e0 2024-03-15T12:56:35.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f90a6793640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a019f390 con 0x7f90a0101fa0 2024-03-15T12:56:35.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f909ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a01003f0 0x7f90a0197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:35.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f909ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a01003f0 0x7f90a0197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50806/0 (socket says 172.21.15.2:50806) 2024-03-15T12:56:35.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f909ffff640 1 -- 172.21.15.2:0/2196660132 learned_addr learned my addr 172.21.15.2:0/2196660132 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:35.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f909f7fe640 1 --2- 172.21.15.2:0/2196660132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90a01015e0 0x7f90a0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:35.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f90a4d30640 1 --2- 172.21.15.2:0/2196660132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90a0101fa0 0x7f90a019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:35.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f909ffff640 1 -- 172.21.15.2:0/2196660132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90a0101fa0 msgr2=0x7f90a019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:35.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f909ffff640 1 --2- 172.21.15.2:0/2196660132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90a0101fa0 0x7f90a019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f909ffff640 1 -- 172.21.15.2:0/2196660132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90a01015e0 msgr2=0x7f90a0197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:35.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f909ffff640 1 --2- 172.21.15.2:0/2196660132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90a01015e0 0x7f90a0197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f909ffff640 1 -- 172.21.15.2:0/2196660132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094009560 con 0x7f90a01003f0 2024-03-15T12:56:35.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f909ffff640 1 --2- 172.21.15.2:0/2196660132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a01003f0 0x7f90a0197570 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f909000e9c0 tx=0x7f909000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:35.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f909d7fa640 1 -- 172.21.15.2:0/2196660132 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090009700 con 0x7f90a01003f0 2024-03-15T12:56:35.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f909d7fa640 1 -- 172.21.15.2:0/2196660132 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f90900044f0 con 0x7f90a01003f0 2024-03-15T12:56:35.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.581+0000 7f909d7fa640 1 -- 172.21.15.2:0/2196660132 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090010430 con 0x7f90a01003f0 2024-03-15T12:56:35.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.582+0000 7f90a6793640 1 -- 172.21.15.2:0/2196660132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a019f670 con 0x7f90a01003f0 2024-03-15T12:56:35.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.582+0000 7f90a6793640 1 -- 172.21.15.2:0/2196660132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a019fba0 con 0x7f90a01003f0 2024-03-15T12:56:35.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.583+0000 7f90a6793640 1 -- 172.21.15.2:0/2196660132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a010bdd0 con 0x7f90a01003f0 2024-03-15T12:56:35.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.584+0000 7f909d7fa640 1 -- 172.21.15.2:0/2196660132 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f9090003fe0 con 0x7f90a01003f0 2024-03-15T12:56:35.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.584+0000 7f909d7fa640 1 --2- 172.21.15.2:0/2196660132 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f90880a1ab0 0x7f90880a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:35.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.584+0000 7f909d7fa640 1 -- 172.21.15.2:0/2196660132 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9090014070 con 0x7f90a01003f0 2024-03-15T12:56:35.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.584+0000 7f909f7fe640 1 --2- 172.21.15.2:0/2196660132 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f90880a1ab0 0x7f90880a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:35.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.585+0000 7f909f7fe640 1 --2- 172.21.15.2:0/2196660132 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f90880a1ab0 0x7f90880a3f70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9094000c00 tx=0x7f9094005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:35.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.589+0000 7f909d7fa640 1 -- 172.21.15.2:0/2196660132 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9090096a00 con 0x7f90a01003f0 2024-03-15T12:56:35.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.768+0000 7f90a6793640 1 -- 172.21.15.2:0/2196660132 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90a01a00c0 con 0x7f90a01003f0 2024-03-15T12:56:35.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.769+0000 7f909d7fa640 1 -- 172.21.15.2:0/2196660132 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f909009a0c0 con 0x7f90a01003f0 2024-03-15T12:56:35.755 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:56:35.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.771+0000 7f90a6793640 1 -- 172.21.15.2:0/2196660132 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f90880a1ab0 msgr2=0x7f90880a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:35.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.771+0000 7f90a6793640 1 --2- 172.21.15.2:0/2196660132 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f90880a1ab0 0x7f90880a3f70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9094000c00 tx=0x7f9094005fd0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.771+0000 7f90a6793640 1 -- 172.21.15.2:0/2196660132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a01003f0 msgr2=0x7f90a0197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:35.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.771+0000 7f90a6793640 1 --2- 172.21.15.2:0/2196660132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a01003f0 0x7f90a0197570 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f909000e9c0 tx=0x7f909000ee80 comp rx=0 tx=0).stop 2024-03-15T12:56:35.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.771+0000 7f90a6793640 1 -- 172.21.15.2:0/2196660132 shutdown_connections 2024-03-15T12:56:35.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.771+0000 7f90a6793640 1 --2- 172.21.15.2:0/2196660132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90a0101fa0 0x7f90a019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.771+0000 7f90a6793640 1 --2- 172.21.15.2:0/2196660132 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f90880a1ab0 0x7f90880a3f70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.771+0000 7f90a6793640 1 --2- 172.21.15.2:0/2196660132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90a01015e0 0x7f90a0197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.771+0000 7f90a6793640 1 --2- 172.21.15.2:0/2196660132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a01003f0 0x7f90a0197570 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.771+0000 7f90a6793640 1 -- 172.21.15.2:0/2196660132 >> 172.21.15.2:0/2196660132 conn(0x7f90a00fb980 msgr2=0x7f90a00fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:35.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.771+0000 7f90a6793640 1 -- 172.21.15.2:0/2196660132 shutdown_connections 2024-03-15T12:56:35.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.771+0000 7f90a6793640 1 -- 172.21.15.2:0/2196660132 wait complete. 2024-03-15T12:56:35.768 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-15T12:56:35.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.888+0000 7fccd1ada640 1 -- 172.21.15.2:0/3688009672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc107290 msgr2=0x7fcccc109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:35.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.888+0000 7fccd1ada640 1 --2- 172.21.15.2:0/3688009672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc107290 0x7fcccc109680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009ca0 tx=0x7fccc402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.888+0000 7fccd1ada640 1 -- 172.21.15.2:0/3688009672 shutdown_connections 2024-03-15T12:56:35.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.888+0000 7fccd1ada640 1 --2- 172.21.15.2:0/3688009672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc107290 0x7fcccc109680 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.888+0000 7fccd1ada640 1 --2- 172.21.15.2:0/3688009672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcccc104960 0x7fcccc106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.888+0000 7fccd1ada640 1 --2- 172.21.15.2:0/3688009672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcccc102030 0x7fcccc104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.888+0000 7fccd1ada640 1 -- 172.21.15.2:0/3688009672 >> 172.21.15.2:0/3688009672 conn(0x7fcccc0fb9a0 msgr2=0x7fcccc0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:35.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.888+0000 7fccd1ada640 1 -- 172.21.15.2:0/3688009672 shutdown_connections 2024-03-15T12:56:35.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.888+0000 7fccd1ada640 1 -- 172.21.15.2:0/3688009672 wait complete. 2024-03-15T12:56:35.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.888+0000 7fccd1ada640 1 Processor -- start 2024-03-15T12:56:35.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.888+0000 7fccd1ada640 1 -- start start 2024-03-15T12:56:35.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fccd1ada640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc102030 0x7fcccc1019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:35.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fccd1ada640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcccc104960 0x7fcccc0fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:35.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fccd1ada640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcccc107290 0x7fcccc100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:35.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fccd1ada640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc100c80 con 0x7fcccc102030 2024-03-15T12:56:35.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fccd1ada640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc100df0 con 0x7fcccc104960 2024-03-15T12:56:35.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fccd1ada640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc1a15a0 con 0x7fcccc107290 2024-03-15T12:56:35.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fcccb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc102030 0x7fcccc1019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:35.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fcccb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc102030 0x7fcccc1019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50830/0 (socket says 172.21.15.2:50830) 2024-03-15T12:56:35.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fcccb7fe640 1 -- 172.21.15.2:0/436245949 learned_addr learned my addr 172.21.15.2:0/436245949 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:35.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fcccaffd640 1 --2- 172.21.15.2:0/436245949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcccc104960 0x7fcccc0fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:35.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fcccbfff640 1 --2- 172.21.15.2:0/436245949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcccc107290 0x7fcccc100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:35.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fcccb7fe640 1 -- 172.21.15.2:0/436245949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcccc107290 msgr2=0x7fcccc100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:35.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fcccb7fe640 1 --2- 172.21.15.2:0/436245949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcccc107290 0x7fcccc100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fcccb7fe640 1 -- 172.21.15.2:0/436245949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcccc104960 msgr2=0x7fcccc0fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:35.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fcccb7fe640 1 --2- 172.21.15.2:0/436245949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcccc104960 0x7fcccc0fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:35.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fcccb7fe640 1 -- 172.21.15.2:0/436245949 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc4009900 con 0x7fcccc102030 2024-03-15T12:56:35.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fcccb7fe640 1 --2- 172.21.15.2:0/436245949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc102030 0x7fcccc1019a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00d7a0 tx=0x7fccbc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:35.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fccc8ff9640 1 -- 172.21.15.2:0/436245949 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc009c30 con 0x7fcccc102030 2024-03-15T12:56:35.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fccc8ff9640 1 -- 172.21.15.2:0/436245949 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fccbc00deb0 con 0x7fcccc102030 2024-03-15T12:56:35.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fccd1ada640 1 -- 172.21.15.2:0/436245949 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcccc1a1850 con 0x7fcccc102030 2024-03-15T12:56:35.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fccc8ff9640 1 -- 172.21.15.2:0/436245949 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc004e70 con 0x7fcccc102030 2024-03-15T12:56:35.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.889+0000 7fccd1ada640 1 -- 172.21.15.2:0/436245949 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcccc1a1d10 con 0x7fcccc102030 2024-03-15T12:56:35.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.890+0000 7fccd1ada640 1 -- 172.21.15.2:0/436245949 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcccc0fd490 con 0x7fcccc102030 2024-03-15T12:56:35.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.891+0000 7fccc8ff9640 1 -- 172.21.15.2:0/436245949 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fccbc005000 con 0x7fcccc102030 2024-03-15T12:56:35.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.892+0000 7fccc8ff9640 1 --2- 172.21.15.2:0/436245949 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fccb40a1a30 0x7fccb40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:35.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.892+0000 7fcccaffd640 1 --2- 172.21.15.2:0/436245949 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fccb40a1a30 0x7fccb40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:35.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.892+0000 7fccc8ff9640 1 -- 172.21.15.2:0/436245949 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fccbc0cd510 con 0x7fcccc102030 2024-03-15T12:56:35.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.892+0000 7fcccaffd640 1 --2- 172.21.15.2:0/436245949 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fccb40a1a30 0x7fccb40a3ef0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcccc101710 tx=0x7fccc0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:35.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:35.896+0000 7fccc8ff9640 1 -- 172.21.15.2:0/436245949 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fccbc096480 con 0x7fcccc102030 2024-03-15T12:56:36.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:35 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1360608270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:36.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:35 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3501980344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:56:36.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:35 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2196660132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:56:36.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:35 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1360608270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:36.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:35 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3501980344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:56:36.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:35 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2196660132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:56:36.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.079+0000 7fccd1ada640 1 -- 172.21.15.2:0/436245949 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcccc10cf10 con 0x7fcccc102030 2024-03-15T12:56:36.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.079+0000 7fccc8ff9640 1 -- 172.21.15.2:0/436245949 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fccbc017020 con 0x7fcccc102030 2024-03-15T12:56:36.066 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-03-15T12:56:36.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.082+0000 7fccd1ada640 1 -- 172.21.15.2:0/436245949 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fccb40a1a30 msgr2=0x7fccb40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:36.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.082+0000 7fccd1ada640 1 --2- 172.21.15.2:0/436245949 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fccb40a1a30 0x7fccb40a3ef0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcccc101710 tx=0x7fccc0005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:36.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.082+0000 7fccd1ada640 1 -- 172.21.15.2:0/436245949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc102030 msgr2=0x7fcccc1019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:36.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.082+0000 7fccd1ada640 1 --2- 172.21.15.2:0/436245949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc102030 0x7fcccc1019a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00d7a0 tx=0x7fccbc00dc60 comp rx=0 tx=0).stop 2024-03-15T12:56:36.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.082+0000 7fccd1ada640 1 -- 172.21.15.2:0/436245949 shutdown_connections 2024-03-15T12:56:36.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.082+0000 7fccd1ada640 1 --2- 172.21.15.2:0/436245949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcccc107290 0x7fcccc100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.082+0000 7fccd1ada640 1 --2- 172.21.15.2:0/436245949 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fccb40a1a30 0x7fccb40a3ef0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.082+0000 7fccd1ada640 1 --2- 172.21.15.2:0/436245949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcccc104960 0x7fcccc0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.082+0000 7fccd1ada640 1 --2- 172.21.15.2:0/436245949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc102030 0x7fcccc1019a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.082+0000 7fccd1ada640 1 -- 172.21.15.2:0/436245949 >> 172.21.15.2:0/436245949 conn(0x7fcccc0fb9a0 msgr2=0x7fcccc1028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:36.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.082+0000 7fccd1ada640 1 -- 172.21.15.2:0/436245949 shutdown_connections 2024-03-15T12:56:36.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.082+0000 7fccd1ada640 1 -- 172.21.15.2:0/436245949 wait complete. 2024-03-15T12:56:36.078 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-15T12:56:36.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:35 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1360608270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:36.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:35 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3501980344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:56:36.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:35 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2196660132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:56:36.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.197+0000 7f296db9a640 1 -- 172.21.15.2:0/3938790772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2968107270 msgr2=0x7f2968109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:36.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.197+0000 7f296db9a640 1 --2- 172.21.15.2:0/3938790772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2968107270 0x7f2968109660 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2960009ca0 tx=0x7f296002c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.197+0000 7f296db9a640 1 -- 172.21.15.2:0/3938790772 shutdown_connections 2024-03-15T12:56:36.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.197+0000 7f296db9a640 1 --2- 172.21.15.2:0/3938790772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2968107270 0x7f2968109660 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.197+0000 7f296db9a640 1 --2- 172.21.15.2:0/3938790772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2968104940 0x7f2968106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.197+0000 7f296db9a640 1 --2- 172.21.15.2:0/3938790772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2968102010 0x7f2968104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.197+0000 7f296db9a640 1 -- 172.21.15.2:0/3938790772 >> 172.21.15.2:0/3938790772 conn(0x7f29680fb980 msgr2=0x7f29680fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:36.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.197+0000 7f296db9a640 1 -- 172.21.15.2:0/3938790772 shutdown_connections 2024-03-15T12:56:36.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.197+0000 7f296db9a640 1 -- 172.21.15.2:0/3938790772 wait complete. 2024-03-15T12:56:36.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.198+0000 7f296db9a640 1 Processor -- start 2024-03-15T12:56:36.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.198+0000 7f296db9a640 1 -- start start 2024-03-15T12:56:36.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.198+0000 7f296db9a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2968102010 0x7f29681975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:36.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.198+0000 7f296db9a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2968104940 0x7f2968197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:36.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.198+0000 7f296db9a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2968107270 0x7f296819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:36.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.198+0000 7f296db9a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296819f090 con 0x7f2968102010 2024-03-15T12:56:36.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.198+0000 7f296db9a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296819f200 con 0x7f2968107270 2024-03-15T12:56:36.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.198+0000 7f296db9a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296819f3c0 con 0x7f2968104940 2024-03-15T12:56:36.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.198+0000 7f29677fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2968102010 0x7f29681975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:36.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.198+0000 7f29677fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2968102010 0x7f29681975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50852/0 (socket says 172.21.15.2:50852) 2024-03-15T12:56:36.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.198+0000 7f29677fe640 1 -- 172.21.15.2:0/2151357338 learned_addr learned my addr 172.21.15.2:0/2151357338 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:36.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.198+0000 7f2967fff640 1 --2- 172.21.15.2:0/2151357338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2968107270 0x7f296819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:36.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.198+0000 7f2966ffd640 1 --2- 172.21.15.2:0/2151357338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2968104940 0x7f2968197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:36.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.199+0000 7f29677fe640 1 -- 172.21.15.2:0/2151357338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2968104940 msgr2=0x7f2968197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:36.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.199+0000 7f29677fe640 1 --2- 172.21.15.2:0/2151357338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2968104940 0x7f2968197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.199+0000 7f29677fe640 1 -- 172.21.15.2:0/2151357338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2968107270 msgr2=0x7f296819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:36.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.199+0000 7f29677fe640 1 --2- 172.21.15.2:0/2151357338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2968107270 0x7f296819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.199+0000 7f29677fe640 1 -- 172.21.15.2:0/2151357338 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c009490 con 0x7f2968102010 2024-03-15T12:56:36.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.199+0000 7f2967fff640 1 --2- 172.21.15.2:0/2151357338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2968107270 0x7f296819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:36.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.199+0000 7f29677fe640 1 --2- 172.21.15.2:0/2151357338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2968102010 0x7f29681975a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f295800d770 tx=0x7f295800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:36.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.199+0000 7f2964ff9640 1 -- 172.21.15.2:0/2151357338 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958009c30 con 0x7f2968102010 2024-03-15T12:56:36.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.199+0000 7f2964ff9640 1 -- 172.21.15.2:0/2151357338 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f295800de80 con 0x7f2968102010 2024-03-15T12:56:36.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.199+0000 7f296db9a640 1 -- 172.21.15.2:0/2151357338 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2960009900 con 0x7f2968102010 2024-03-15T12:56:36.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.199+0000 7f2964ff9640 1 -- 172.21.15.2:0/2151357338 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958004e50 con 0x7f2968102010 2024-03-15T12:56:36.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.199+0000 7f296db9a640 1 -- 172.21.15.2:0/2151357338 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296819f9d0 con 0x7f2968102010 2024-03-15T12:56:36.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.200+0000 7f296db9a640 1 -- 172.21.15.2:0/2151357338 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29680fd470 con 0x7f2968102010 2024-03-15T12:56:36.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.201+0000 7f2964ff9640 1 -- 172.21.15.2:0/2151357338 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f2958005000 con 0x7f2968102010 2024-03-15T12:56:36.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.202+0000 7f2964ff9640 1 --2- 172.21.15.2:0/2151357338 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f29500a1a30 0x7f29500a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:36.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.202+0000 7f2966ffd640 1 --2- 172.21.15.2:0/2151357338 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f29500a1a30 0x7f29500a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:36.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.202+0000 7f2964ff9640 1 -- 172.21.15.2:0/2151357338 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f29580cd450 con 0x7f2968102010 2024-03-15T12:56:36.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.202+0000 7f2966ffd640 1 --2- 172.21.15.2:0/2151357338 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f29500a1a30 0x7f29500a3ef0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2968198ab0 tx=0x7f295c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:36.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.206+0000 7f2964ff9640 1 -- 172.21.15.2:0/2151357338 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f29580964d0 con 0x7f2968102010 2024-03-15T12:56:36.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.388+0000 7f296db9a640 1 -- 172.21.15.2:0/2151357338 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f296810cef0 con 0x7f2968102010 2024-03-15T12:56:36.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.388+0000 7f2964ff9640 1 -- 172.21.15.2:0/2151357338 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2958020070 con 0x7f2968102010 2024-03-15T12:56:36.375 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:56:36.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.390+0000 7f296db9a640 1 -- 172.21.15.2:0/2151357338 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f29500a1a30 msgr2=0x7f29500a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:36.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.390+0000 7f296db9a640 1 --2- 172.21.15.2:0/2151357338 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f29500a1a30 0x7f29500a3ef0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2968198ab0 tx=0x7f295c005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:36.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.390+0000 7f296db9a640 1 -- 172.21.15.2:0/2151357338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2968102010 msgr2=0x7f29681975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:36.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.390+0000 7f296db9a640 1 --2- 172.21.15.2:0/2151357338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2968102010 0x7f29681975a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f295800d770 tx=0x7f295800dc30 comp rx=0 tx=0).stop 2024-03-15T12:56:36.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.390+0000 7f296db9a640 1 -- 172.21.15.2:0/2151357338 shutdown_connections 2024-03-15T12:56:36.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.390+0000 7f296db9a640 1 --2- 172.21.15.2:0/2151357338 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f29500a1a30 0x7f29500a3ef0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.390+0000 7f296db9a640 1 --2- 172.21.15.2:0/2151357338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2968107270 0x7f296819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.390+0000 7f296db9a640 1 --2- 172.21.15.2:0/2151357338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2968104940 0x7f2968197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.390+0000 7f296db9a640 1 --2- 172.21.15.2:0/2151357338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2968102010 0x7f29681975a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.390+0000 7f296db9a640 1 -- 172.21.15.2:0/2151357338 >> 172.21.15.2:0/2151357338 conn(0x7f29680fb980 msgr2=0x7f2968102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:36.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.390+0000 7f296db9a640 1 -- 172.21.15.2:0/2151357338 shutdown_connections 2024-03-15T12:56:36.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.390+0000 7f296db9a640 1 -- 172.21.15.2:0/2151357338 wait complete. 2024-03-15T12:56:36.387 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-15T12:56:36.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.504+0000 7f20d1c8f640 1 -- 172.21.15.2:0/3235752836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20cc101fa0 msgr2=0x7f20cc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:36.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.504+0000 7f20d1c8f640 1 --2- 172.21.15.2:0/3235752836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20cc101fa0 0x7f20cc108830 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009ca0 tx=0x7f20c402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.505+0000 7f20d1c8f640 1 -- 172.21.15.2:0/3235752836 shutdown_connections 2024-03-15T12:56:36.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.505+0000 7f20d1c8f640 1 --2- 172.21.15.2:0/3235752836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20cc101fa0 0x7f20cc108830 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.505+0000 7f20d1c8f640 1 --2- 172.21.15.2:0/3235752836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20cc1015e0 0x7f20cc101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.505+0000 7f20d1c8f640 1 --2- 172.21.15.2:0/3235752836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20cc1003f0 0x7f20cc100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.505+0000 7f20d1c8f640 1 -- 172.21.15.2:0/3235752836 >> 172.21.15.2:0/3235752836 conn(0x7f20cc0fb980 msgr2=0x7f20cc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:36.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.505+0000 7f20d1c8f640 1 -- 172.21.15.2:0/3235752836 shutdown_connections 2024-03-15T12:56:36.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.505+0000 7f20d1c8f640 1 -- 172.21.15.2:0/3235752836 wait complete. 2024-03-15T12:56:36.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.505+0000 7f20d1c8f640 1 Processor -- start 2024-03-15T12:56:36.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.505+0000 7f20d1c8f640 1 -- start start 2024-03-15T12:56:36.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20d1c8f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20cc1003f0 0x7f20cc196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:36.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20d1c8f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20cc1015e0 0x7f20cc1969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:36.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20d1c8f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20cc101fa0 0x7f20cc19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20d1c8f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc19df70 con 0x7f20cc101fa0 2024-03-15T12:56:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20d1c8f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc19e0e0 con 0x7f20cc1003f0 2024-03-15T12:56:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20d1c8f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc19e2a0 con 0x7f20cc1015e0 2024-03-15T12:56:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20cbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20cc101fa0 0x7f20cc19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20cbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20cc101fa0 0x7f20cc19da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50874/0 (socket says 172.21.15.2:50874) 2024-03-15T12:56:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20cbfff640 1 -- 172.21.15.2:0/1451549579 learned_addr learned my addr 172.21.15.2:0/1451549579 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20cb7fe640 1 --2- 172.21.15.2:0/1451549579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20cc1003f0 0x7f20cc196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20c2ffd640 1 --2- 172.21.15.2:0/1451549579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20cc1015e0 0x7f20cc1969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20cbfff640 1 -- 172.21.15.2:0/1451549579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20cc1015e0 msgr2=0x7f20cc1969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20cbfff640 1 --2- 172.21.15.2:0/1451549579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20cc1015e0 0x7f20cc1969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20cbfff640 1 -- 172.21.15.2:0/1451549579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20cc1003f0 msgr2=0x7f20cc196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20cbfff640 1 --2- 172.21.15.2:0/1451549579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20cc1003f0 0x7f20cc196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20cbfff640 1 -- 172.21.15.2:0/1451549579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c4009900 con 0x7f20cc101fa0 2024-03-15T12:56:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20cb7fe640 1 --2- 172.21.15.2:0/1451549579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20cc1003f0 0x7f20cc196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20cbfff640 1 --2- 172.21.15.2:0/1451549579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20cc101fa0 0x7f20cc19da30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005cc0 tx=0x7f20c4032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20c97fa640 1 -- 172.21.15.2:0/1451549579 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4034c90 con 0x7f20cc101fa0 2024-03-15T12:56:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20d1c8f640 1 -- 172.21.15.2:0/1451549579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20cc19e520 con 0x7f20cc101fa0 2024-03-15T12:56:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20c97fa640 1 -- 172.21.15.2:0/1451549579 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f20c4003e40 con 0x7f20cc101fa0 2024-03-15T12:56:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20c97fa640 1 -- 172.21.15.2:0/1451549579 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c40048a0 con 0x7f20cc101fa0 2024-03-15T12:56:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.506+0000 7f20d1c8f640 1 -- 172.21.15.2:0/1451549579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20cc19ea20 con 0x7f20cc101fa0 2024-03-15T12:56:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.507+0000 7f20d1c8f640 1 -- 172.21.15.2:0/1451549579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20cc107710 con 0x7f20cc101fa0 2024-03-15T12:56:36.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.509+0000 7f20c97fa640 1 -- 172.21.15.2:0/1451549579 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f20c400a040 con 0x7f20cc101fa0 2024-03-15T12:56:36.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.509+0000 7f20c97fa640 1 --2- 172.21.15.2:0/1451549579 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20b80a1a30 0x7f20b80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:36.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.509+0000 7f20cb7fe640 1 --2- 172.21.15.2:0/1451549579 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20b80a1a30 0x7f20b80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:36.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.509+0000 7f20c97fa640 1 -- 172.21.15.2:0/1451549579 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f20c40f0830 con 0x7f20cc101fa0 2024-03-15T12:56:36.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.510+0000 7f20cb7fe640 1 --2- 172.21.15.2:0/1451549579 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20b80a1a30 0x7f20b80a3ef0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f20cc101440 tx=0x7f20bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:36.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.513+0000 7f20c97fa640 1 -- 172.21.15.2:0/1451549579 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f20c40b9800 con 0x7f20cc101fa0 2024-03-15T12:56:36.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.708+0000 7f20d1c8f640 1 -- 172.21.15.2:0/1451549579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f20cc10bdd0 con 0x7f20cc101fa0 2024-03-15T12:56:36.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.708+0000 7f20c97fa640 1 -- 172.21.15.2:0/1451549579 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+579 (secure 0 0 0) 0x7f20c40bcec0 con 0x7f20cc101fa0 2024-03-15T12:56:36.695 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3349070582 2024-03-16T12:49:18.658825+0000 2024-03-15T12:56:36.696 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3269509343 2024-03-16T12:49:18.658825+0000 2024-03-15T12:56:36.696 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3269509343 2024-03-16T12:49:18.658825+0000 2024-03-15T12:56:36.696 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/656434628 2024-03-16T12:49:18.658825+0000 2024-03-15T12:56:36.696 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/84615744 2024-03-16T12:48:51.791072+0000 2024-03-15T12:56:36.696 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1464336909 2024-03-16T12:48:51.791072+0000 2024-03-15T12:56:36.696 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3546094985 2024-03-16T12:48:51.791072+0000 2024-03-15T12:56:36.697 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3111414481 2024-03-16T12:48:51.791072+0000 2024-03-15T12:56:36.697 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3111414481 2024-03-16T12:48:51.791072+0000 2024-03-15T12:56:36.697 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3673716509 2024-03-16T12:49:18.658825+0000 2024-03-15T12:56:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:listed 10 entries 2024-03-15T12:56:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.710+0000 7f20d1c8f640 1 -- 172.21.15.2:0/1451549579 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20b80a1a30 msgr2=0x7f20b80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.710+0000 7f20d1c8f640 1 --2- 172.21.15.2:0/1451549579 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20b80a1a30 0x7f20b80a3ef0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f20cc101440 tx=0x7f20bc005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.710+0000 7f20d1c8f640 1 -- 172.21.15.2:0/1451549579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20cc101fa0 msgr2=0x7f20cc19da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.710+0000 7f20d1c8f640 1 --2- 172.21.15.2:0/1451549579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20cc101fa0 0x7f20cc19da30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005cc0 tx=0x7f20c4032a10 comp rx=0 tx=0).stop 2024-03-15T12:56:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.710+0000 7f20d1c8f640 1 -- 172.21.15.2:0/1451549579 shutdown_connections 2024-03-15T12:56:36.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.710+0000 7f20d1c8f640 1 --2- 172.21.15.2:0/1451549579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20cc101fa0 0x7f20cc19da30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.710+0000 7f20d1c8f640 1 --2- 172.21.15.2:0/1451549579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20cc1015e0 0x7f20cc1969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.710+0000 7f20d1c8f640 1 --2- 172.21.15.2:0/1451549579 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20b80a1a30 0x7f20b80a3ef0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.710+0000 7f20d1c8f640 1 --2- 172.21.15.2:0/1451549579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20cc1003f0 0x7f20cc196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.710+0000 7f20d1c8f640 1 -- 172.21.15.2:0/1451549579 >> 172.21.15.2:0/1451549579 conn(0x7f20cc0fb980 msgr2=0x7f20cc1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.710+0000 7f20d1c8f640 1 -- 172.21.15.2:0/1451549579 shutdown_connections 2024-03-15T12:56:36.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.710+0000 7f20d1c8f640 1 -- 172.21.15.2:0/1451549579 wait complete. 2024-03-15T12:56:36.708 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2024-03-15T12:56:36.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:36.708 DEBUG:teuthology.orchestra.run.smithi002:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-15T12:56:36.734 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi002.stderr:ceph-fuse[76297]: fuse finished with error 0 and tester_r 0 2024-03-15T12:56:36.736 INFO:teuthology.orchestra.run:waiting for 300 2024-03-15T12:56:36.754 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi002.front.sepia.ceph.com 2024-03-15T12:56:36.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:56:36.754 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-15T12:56:36.817 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:56:36.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.975+0000 7ff4d6779640 1 -- 172.21.15.2:0/314410619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d0101680 msgr2=0x7ff4d0101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:36.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.975+0000 7ff4d6779640 1 --2- 172.21.15.2:0/314410619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d0101680 0x7ff4d0101b00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009900 tx=0x7ff4c402bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:36.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.975+0000 7ff4d6779640 1 -- 172.21.15.2:0/314410619 shutdown_connections 2024-03-15T12:56:36.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.975+0000 7ff4d6779640 1 --2- 172.21.15.2:0/314410619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4d0102040 0x7ff4d01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.975+0000 7ff4d6779640 1 --2- 172.21.15.2:0/314410619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d0101680 0x7ff4d0101b00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.975+0000 7ff4d6779640 1 --2- 172.21.15.2:0/314410619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d0100490 0x7ff4d01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.975+0000 7ff4d6779640 1 -- 172.21.15.2:0/314410619 >> 172.21.15.2:0/314410619 conn(0x7ff4d00fba20 msgr2=0x7ff4d00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:36.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.975+0000 7ff4d6779640 1 -- 172.21.15.2:0/314410619 shutdown_connections 2024-03-15T12:56:36.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.975+0000 7ff4d6779640 1 -- 172.21.15.2:0/314410619 wait complete. 2024-03-15T12:56:36.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.975+0000 7ff4d6779640 1 Processor -- start 2024-03-15T12:56:36.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4d6779640 1 -- start start 2024-03-15T12:56:36.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4d6779640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d0100490 0x7ff4d01043d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:36.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4d6779640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d0101680 0x7ff4d0102a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:36.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4d6779640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4d0102040 0x7ff4d0102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:36.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4d6779640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0103730 con 0x7ff4d0101680 2024-03-15T12:56:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4d6779640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d01038a0 con 0x7ff4d0100490 2024-03-15T12:56:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4d6779640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d01a1620 con 0x7ff4d0102040 2024-03-15T12:56:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4cf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d0101680 0x7ff4d0102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4cf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d0101680 0x7ff4d0102a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50886/0 (socket says 172.21.15.2:50886) 2024-03-15T12:56:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4cf7fe640 1 -- 172.21.15.2:0/1224082243 learned_addr learned my addr 172.21.15.2:0/1224082243 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4cffff640 1 --2- 172.21.15.2:0/1224082243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d0100490 0x7ff4d01043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4d4d16640 1 --2- 172.21.15.2:0/1224082243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4d0102040 0x7ff4d0102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4cf7fe640 1 -- 172.21.15.2:0/1224082243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4d0102040 msgr2=0x7ff4d0102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4cf7fe640 1 --2- 172.21.15.2:0/1224082243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4d0102040 0x7ff4d0102fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4cf7fe640 1 -- 172.21.15.2:0/1224082243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d0100490 msgr2=0x7ff4d01043d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4cf7fe640 1 --2- 172.21.15.2:0/1224082243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d0100490 0x7ff4d01043d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.976+0000 7ff4cf7fe640 1 -- 172.21.15.2:0/1224082243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c4009560 con 0x7ff4d0101680 2024-03-15T12:56:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.977+0000 7ff4cf7fe640 1 --2- 172.21.15.2:0/1224082243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d0101680 0x7ff4d0102a60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4000c00 tx=0x7ff4c4004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.977+0000 7ff4cd7fa640 1 -- 172.21.15.2:0/1224082243 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c40042d0 con 0x7ff4d0101680 2024-03-15T12:56:36.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.977+0000 7ff4cd7fa640 1 -- 172.21.15.2:0/1224082243 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff4c4004430 con 0x7ff4d0101680 2024-03-15T12:56:36.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.977+0000 7ff4d6779640 1 -- 172.21.15.2:0/1224082243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d01a1820 con 0x7ff4d0101680 2024-03-15T12:56:36.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.977+0000 7ff4cd7fa640 1 -- 172.21.15.2:0/1224082243 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4003e50 con 0x7ff4d0101680 2024-03-15T12:56:36.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.977+0000 7ff4d6779640 1 -- 172.21.15.2:0/1224082243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d01a1d30 con 0x7ff4d0101680 2024-03-15T12:56:36.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.978+0000 7ff4d6779640 1 -- 172.21.15.2:0/1224082243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d010be00 con 0x7ff4d0101680 2024-03-15T12:56:36.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.979+0000 7ff4cd7fa640 1 -- 172.21.15.2:0/1224082243 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7ff4c40035f0 con 0x7ff4d0101680 2024-03-15T12:56:36.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.979+0000 7ff4cd7fa640 1 --2- 172.21.15.2:0/1224082243 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff4b80a1a30 0x7ff4b80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:36.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.979+0000 7ff4cffff640 1 --2- 172.21.15.2:0/1224082243 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff4b80a1a30 0x7ff4b80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:36.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.979+0000 7ff4cd7fa640 1 -- 172.21.15.2:0/1224082243 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff4c40ef810 con 0x7ff4d0101680 2024-03-15T12:56:36.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.980+0000 7ff4cffff640 1 --2- 172.21.15.2:0/1224082243 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff4b80a1a30 0x7ff4b80a3ef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff4d01014e0 tx=0x7ff4c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:36.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:36.984+0000 7ff4cd7fa640 1 -- 172.21.15.2:0/1224082243 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff4c40b87e0 con 0x7ff4d0101680 2024-03-15T12:56:37.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:36 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/436245949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:56:37.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:36 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2151357338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:56:37.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:36 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1451549579' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-15T12:56:37.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:36 smithi002 ceph-mon[32060]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 297 B/s wr, 0 op/s 2024-03-15T12:56:37.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:36 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/436245949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:56:37.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:36 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2151357338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:56:37.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:36 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1451549579' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-15T12:56:37.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:36 smithi184 ceph-mon[35993]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 297 B/s wr, 0 op/s 2024-03-15T12:56:37.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:36 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/436245949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:56:37.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:36 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2151357338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:56:37.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:36 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1451549579' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-15T12:56:37.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:36 smithi149 ceph-mon[35927]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 297 B/s wr, 0 op/s 2024-03-15T12:56:37.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.168+0000 7ff4d6779640 1 -- 172.21.15.2:0/1224082243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4d010bf90 con 0x7ff4d0101680 2024-03-15T12:56:37.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.169+0000 7ff4cd7fa640 1 -- 172.21.15.2:0/1224082243 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24045 (secure 0 0 0) 0x7ff4c40bbea0 con 0x7ff4d0101680 2024-03-15T12:56:37.156 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:37.156 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":89,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:56:25.830426+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:56:37.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.171+0000 7ff4d6779640 1 -- 172.21.15.2:0/1224082243 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff4b80a1a30 msgr2=0x7ff4b80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:37.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.171+0000 7ff4d6779640 1 --2- 172.21.15.2:0/1224082243 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff4b80a1a30 0x7ff4b80a3ef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff4d01014e0 tx=0x7ff4c0005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:37.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.171+0000 7ff4d6779640 1 -- 172.21.15.2:0/1224082243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d0101680 msgr2=0x7ff4d0102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:37.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.171+0000 7ff4d6779640 1 --2- 172.21.15.2:0/1224082243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d0101680 0x7ff4d0102a60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4000c00 tx=0x7ff4c4004e50 comp rx=0 tx=0).stop 2024-03-15T12:56:37.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.171+0000 7ff4d6779640 1 -- 172.21.15.2:0/1224082243 shutdown_connections 2024-03-15T12:56:37.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.171+0000 7ff4d6779640 1 --2- 172.21.15.2:0/1224082243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4d0102040 0x7ff4d0102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.171+0000 7ff4d6779640 1 --2- 172.21.15.2:0/1224082243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d0101680 0x7ff4d0102a60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.171+0000 7ff4d6779640 1 --2- 172.21.15.2:0/1224082243 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff4b80a1a30 0x7ff4b80a3ef0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.171+0000 7ff4d6779640 1 --2- 172.21.15.2:0/1224082243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d0100490 0x7ff4d01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.171+0000 7ff4d6779640 1 -- 172.21.15.2:0/1224082243 >> 172.21.15.2:0/1224082243 conn(0x7ff4d00fba20 msgr2=0x7ff4d00fc030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:37.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.171+0000 7ff4d6779640 1 -- 172.21.15.2:0/1224082243 shutdown_connections 2024-03-15T12:56:37.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.171+0000 7ff4d6779640 1 -- 172.21.15.2:0/1224082243 wait complete. 2024-03-15T12:56:37.169 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:56:37.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.288+0000 7fdd60423640 1 -- 172.21.15.2:0/2296112679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd581072a0 msgr2=0x7fdd58109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:37.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.288+0000 7fdd60423640 1 --2- 172.21.15.2:0/2296112679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd581072a0 0x7fdd58109690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009900 tx=0x7fdd5002bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:37.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.288+0000 7fdd60423640 1 -- 172.21.15.2:0/2296112679 shutdown_connections 2024-03-15T12:56:37.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.288+0000 7fdd60423640 1 --2- 172.21.15.2:0/2296112679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd581072a0 0x7fdd58109690 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.288+0000 7fdd60423640 1 --2- 172.21.15.2:0/2296112679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd58104970 0x7fdd58106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.288+0000 7fdd60423640 1 --2- 172.21.15.2:0/2296112679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd58102040 0x7fdd58104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.288+0000 7fdd60423640 1 -- 172.21.15.2:0/2296112679 >> 172.21.15.2:0/2296112679 conn(0x7fdd580fb990 msgr2=0x7fdd580fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:37.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.288+0000 7fdd60423640 1 -- 172.21.15.2:0/2296112679 shutdown_connections 2024-03-15T12:56:37.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.288+0000 7fdd60423640 1 -- 172.21.15.2:0/2296112679 wait complete. 2024-03-15T12:56:37.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd60423640 1 Processor -- start 2024-03-15T12:56:37.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd60423640 1 -- start start 2024-03-15T12:56:37.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd60423640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd58102040 0x7fdd581019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:37.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd60423640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd58104970 0x7fdd580ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:37.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd60423640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd581072a0 0x7fdd58100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:37.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd60423640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd58100c90 con 0x7fdd58104970 2024-03-15T12:56:37.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd60423640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd58100e00 con 0x7fdd58102040 2024-03-15T12:56:37.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd60423640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd581a1610 con 0x7fdd581072a0 2024-03-15T12:56:37.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd5d9be640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd58104970 0x7fdd580ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:37.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd5d9be640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd58104970 0x7fdd580ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50914/0 (socket says 172.21.15.2:50914) 2024-03-15T12:56:37.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd5d9be640 1 -- 172.21.15.2:0/2792238473 learned_addr learned my addr 172.21.15.2:0/2792238473 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:37.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd5e1bf640 1 --2- 172.21.15.2:0/2792238473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd58102040 0x7fdd581019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:37.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd5e9c0640 1 --2- 172.21.15.2:0/2792238473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd581072a0 0x7fdd58100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:37.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd5d9be640 1 -- 172.21.15.2:0/2792238473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd581072a0 msgr2=0x7fdd58100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:37.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd5d9be640 1 --2- 172.21.15.2:0/2792238473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd581072a0 0x7fdd58100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd5d9be640 1 -- 172.21.15.2:0/2792238473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd58102040 msgr2=0x7fdd581019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:37.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd5d9be640 1 --2- 172.21.15.2:0/2792238473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd58102040 0x7fdd581019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd5d9be640 1 -- 172.21.15.2:0/2792238473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd50009560 con 0x7fdd58104970 2024-03-15T12:56:37.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.289+0000 7fdd5d9be640 1 --2- 172.21.15.2:0/2792238473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd58104970 0x7fdd580ffff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800e9f0 tx=0x7fdd4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:37.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.290+0000 7fdd4f7fe640 1 -- 172.21.15.2:0/2792238473 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48009700 con 0x7fdd58104970 2024-03-15T12:56:37.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.290+0000 7fdd4f7fe640 1 -- 172.21.15.2:0/2792238473 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdd4800cd80 con 0x7fdd58104970 2024-03-15T12:56:37.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.290+0000 7fdd60423640 1 -- 172.21.15.2:0/2792238473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd581a18c0 con 0x7fdd58104970 2024-03-15T12:56:37.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.290+0000 7fdd4f7fe640 1 -- 172.21.15.2:0/2792238473 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48010490 con 0x7fdd58104970 2024-03-15T12:56:37.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.290+0000 7fdd60423640 1 -- 172.21.15.2:0/2792238473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd581a1dd0 con 0x7fdd58104970 2024-03-15T12:56:37.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.291+0000 7fdd60423640 1 -- 172.21.15.2:0/2792238473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd580fd480 con 0x7fdd58104970 2024-03-15T12:56:37.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.292+0000 7fdd4f7fe640 1 -- 172.21.15.2:0/2792238473 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fdd48005000 con 0x7fdd58104970 2024-03-15T12:56:37.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.292+0000 7fdd4f7fe640 1 --2- 172.21.15.2:0/2792238473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fdd440a1a30 0x7fdd440a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:37.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.293+0000 7fdd5e1bf640 1 --2- 172.21.15.2:0/2792238473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fdd440a1a30 0x7fdd440a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:37.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.293+0000 7fdd4f7fe640 1 -- 172.21.15.2:0/2792238473 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fdd48014070 con 0x7fdd58104970 2024-03-15T12:56:37.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.293+0000 7fdd5e1bf640 1 --2- 172.21.15.2:0/2792238473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fdd440a1a30 0x7fdd440a3ef0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdd54005c40 tx=0x7fdd5400a570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:37.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.296+0000 7fdd4f7fe640 1 -- 172.21.15.2:0/2792238473 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdd48096b00 con 0x7fdd58104970 2024-03-15T12:56:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.479+0000 7fdd60423640 1 -- 172.21.15.2:0/2792238473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd5810d160 con 0x7fdd58104970 2024-03-15T12:56:37.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.481+0000 7fdd4f7fe640 1 -- 172.21.15.2:0/2792238473 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24045 (secure 0 0 0) 0x7fdd4809a1c0 con 0x7fdd58104970 2024-03-15T12:56:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:37.467 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":89,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:56:25.830426+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:56:37.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.483+0000 7fdd60423640 1 -- 172.21.15.2:0/2792238473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fdd440a1a30 msgr2=0x7fdd440a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:37.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.483+0000 7fdd60423640 1 --2- 172.21.15.2:0/2792238473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fdd440a1a30 0x7fdd440a3ef0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdd54005c40 tx=0x7fdd5400a570 comp rx=0 tx=0).stop 2024-03-15T12:56:37.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.483+0000 7fdd60423640 1 -- 172.21.15.2:0/2792238473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd58104970 msgr2=0x7fdd580ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:37.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.483+0000 7fdd60423640 1 --2- 172.21.15.2:0/2792238473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd58104970 0x7fdd580ffff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800e9f0 tx=0x7fdd4800eeb0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.483+0000 7fdd60423640 1 -- 172.21.15.2:0/2792238473 shutdown_connections 2024-03-15T12:56:37.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.483+0000 7fdd60423640 1 --2- 172.21.15.2:0/2792238473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd581072a0 0x7fdd58100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.483+0000 7fdd60423640 1 --2- 172.21.15.2:0/2792238473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd58104970 0x7fdd580ffff0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.483+0000 7fdd60423640 1 --2- 172.21.15.2:0/2792238473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fdd440a1a30 0x7fdd440a3ef0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.483+0000 7fdd60423640 1 --2- 172.21.15.2:0/2792238473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd58102040 0x7fdd581019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.483+0000 7fdd60423640 1 -- 172.21.15.2:0/2792238473 >> 172.21.15.2:0/2792238473 conn(0x7fdd580fb990 msgr2=0x7fdd581028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:37.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.483+0000 7fdd60423640 1 -- 172.21.15.2:0/2792238473 shutdown_connections 2024-03-15T12:56:37.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.483+0000 7fdd60423640 1 -- 172.21.15.2:0/2792238473 wait complete. 2024-03-15T12:56:37.481 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-15T12:56:37.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.599+0000 7fbfbcc40640 1 -- 172.21.15.2:0/294783171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb81030d0 msgr2=0x7fbfb8109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:37.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.599+0000 7fbfbcc40640 1 --2- 172.21.15.2:0/294783171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb81030d0 0x7fbfb8109960 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009ca0 tx=0x7fbfa802c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.600+0000 7fbfbcc40640 1 -- 172.21.15.2:0/294783171 shutdown_connections 2024-03-15T12:56:37.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.600+0000 7fbfbcc40640 1 --2- 172.21.15.2:0/294783171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb81030d0 0x7fbfb8109960 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.600+0000 7fbfbcc40640 1 --2- 172.21.15.2:0/294783171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfb8102710 0x7fbfb8102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.600+0000 7fbfbcc40640 1 --2- 172.21.15.2:0/294783171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfb8101520 0x7fbfb8101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.600+0000 7fbfbcc40640 1 -- 172.21.15.2:0/294783171 >> 172.21.15.2:0/294783171 conn(0x7fbfb80fca90 msgr2=0x7fbfb80feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:37.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.600+0000 7fbfbcc40640 1 -- 172.21.15.2:0/294783171 shutdown_connections 2024-03-15T12:56:37.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.600+0000 7fbfbcc40640 1 -- 172.21.15.2:0/294783171 wait complete. 2024-03-15T12:56:37.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.600+0000 7fbfbcc40640 1 Processor -- start 2024-03-15T12:56:37.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.600+0000 7fbfbcc40640 1 -- start start 2024-03-15T12:56:37.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfbcc40640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfb8101520 0x7fbfb8197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:37.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfbcc40640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfb8102710 0x7fbfb8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:37.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfbcc40640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb81030d0 0x7fbfb819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:37.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfbcc40640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb819f030 con 0x7fbfb81030d0 2024-03-15T12:56:37.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfbcc40640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb819f1a0 con 0x7fbfb8102710 2024-03-15T12:56:37.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfbcc40640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb819f360 con 0x7fbfb8101520 2024-03-15T12:56:37.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfb6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb81030d0 0x7fbfb819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:37.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfb6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb81030d0 0x7fbfb819eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50938/0 (socket says 172.21.15.2:50938) 2024-03-15T12:56:37.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfb6d9d640 1 -- 172.21.15.2:0/2888527337 learned_addr learned my addr 172.21.15.2:0/2888527337 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:37.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfb659c640 1 --2- 172.21.15.2:0/2888527337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfb8101520 0x7fbfb8197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:37.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfb5d9b640 1 --2- 172.21.15.2:0/2888527337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfb8102710 0x7fbfb8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:37.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfb6d9d640 1 -- 172.21.15.2:0/2888527337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfb8101520 msgr2=0x7fbfb8197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:37.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfb6d9d640 1 --2- 172.21.15.2:0/2888527337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfb8101520 0x7fbfb8197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfb6d9d640 1 -- 172.21.15.2:0/2888527337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfb8102710 msgr2=0x7fbfb8197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:37.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfb6d9d640 1 --2- 172.21.15.2:0/2888527337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfb8102710 0x7fbfb8197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:37.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfb6d9d640 1 -- 172.21.15.2:0/2888527337 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb0009490 con 0x7fbfb81030d0 2024-03-15T12:56:37.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfb6d9d640 1 --2- 172.21.15.2:0/2888527337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb81030d0 0x7fbfb819eaf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005cc0 tx=0x7fbfa8032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:37.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfaf7fe640 1 -- 172.21.15.2:0/2888527337 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8034c90 con 0x7fbfb81030d0 2024-03-15T12:56:37.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfaf7fe640 1 -- 172.21.15.2:0/2888527337 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbfa8003e40 con 0x7fbfb81030d0 2024-03-15T12:56:37.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfbcc40640 1 -- 172.21.15.2:0/2888527337 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa8009900 con 0x7fbfb81030d0 2024-03-15T12:56:37.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfaf7fe640 1 -- 172.21.15.2:0/2888527337 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80048a0 con 0x7fbfb81030d0 2024-03-15T12:56:37.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.601+0000 7fbfbcc40640 1 -- 172.21.15.2:0/2888527337 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb819f900 con 0x7fbfb81030d0 2024-03-15T12:56:37.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.602+0000 7fbfbcc40640 1 -- 172.21.15.2:0/2888527337 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb81087e0 con 0x7fbfb81030d0 2024-03-15T12:56:37.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.603+0000 7fbfaf7fe640 1 -- 172.21.15.2:0/2888527337 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fbfa800a040 con 0x7fbfb81030d0 2024-03-15T12:56:37.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.604+0000 7fbfaf7fe640 1 --2- 172.21.15.2:0/2888527337 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbfa40a1a30 0x7fbfa40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:37.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.604+0000 7fbfb659c640 1 --2- 172.21.15.2:0/2888527337 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbfa40a1a30 0x7fbfa40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:37.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.604+0000 7fbfaf7fe640 1 -- 172.21.15.2:0/2888527337 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fbfa80f0370 con 0x7fbfb81030d0 2024-03-15T12:56:37.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.605+0000 7fbfb659c640 1 --2- 172.21.15.2:0/2888527337 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbfa40a1a30 0x7fbfa40a3ef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8102570 tx=0x7fbfb0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:37.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.609+0000 7fbfaf7fe640 1 -- 172.21.15.2:0/2888527337 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbfa80b9340 con 0x7fbfb81030d0 2024-03-15T12:56:37.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:37.831+0000 7fbfbcc40640 1 -- 172.21.15.2:0/2888527337 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbfb802d000 con 0x7fbfb81030d0 2024-03-15T12:56:38.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:37 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1224082243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:38.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:37 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2792238473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:38.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:37 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1224082243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:38.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:37 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2792238473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:38.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:37 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1224082243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:38.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:37 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2792238473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:38.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.817+0000 7fbfaf7fe640 1 -- 172.21.15.2:0/2888527337 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbfa80bca00 con 0x7fbfb81030d0 2024-03-15T12:56:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.819+0000 7fbfbcc40640 1 -- 172.21.15.2:0/2888527337 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbfa40a1a30 msgr2=0x7fbfa40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.819+0000 7fbfbcc40640 1 --2- 172.21.15.2:0/2888527337 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbfa40a1a30 0x7fbfa40a3ef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8102570 tx=0x7fbfb0005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.819+0000 7fbfbcc40640 1 -- 172.21.15.2:0/2888527337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb81030d0 msgr2=0x7fbfb819eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.819+0000 7fbfbcc40640 1 --2- 172.21.15.2:0/2888527337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb81030d0 0x7fbfb819eaf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005cc0 tx=0x7fbfa8032a10 comp rx=0 tx=0).stop 2024-03-15T12:56:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.819+0000 7fbfbcc40640 1 -- 172.21.15.2:0/2888527337 shutdown_connections 2024-03-15T12:56:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.819+0000 7fbfbcc40640 1 --2- 172.21.15.2:0/2888527337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb81030d0 0x7fbfb819eaf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.819+0000 7fbfbcc40640 1 --2- 172.21.15.2:0/2888527337 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbfa40a1a30 0x7fbfa40a3ef0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.819+0000 7fbfbcc40640 1 --2- 172.21.15.2:0/2888527337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfb8102710 0x7fbfb8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:38.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.819+0000 7fbfbcc40640 1 --2- 172.21.15.2:0/2888527337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfb8101520 0x7fbfb8197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:38.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.819+0000 7fbfbcc40640 1 -- 172.21.15.2:0/2888527337 >> 172.21.15.2:0/2888527337 conn(0x7fbfb80fca90 msgr2=0x7fbfb81059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:38.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.820+0000 7fbfbcc40640 1 -- 172.21.15.2:0/2888527337 shutdown_connections 2024-03-15T12:56:38.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.820+0000 7fbfbcc40640 1 -- 172.21.15.2:0/2888527337 wait complete. 2024-03-15T12:56:38.819 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-03-15T12:56:38.819 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-15T12:56:38.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.946+0000 7fde83c4e640 1 -- 172.21.15.2:0/2248156271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde7c104900 msgr2=0x7fde7c106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:38.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.946+0000 7fde83c4e640 1 --2- 172.21.15.2:0/2248156271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde7c104900 0x7fde7c106cf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009900 tx=0x7fde6c02bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:38.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.946+0000 7fde83c4e640 1 -- 172.21.15.2:0/2248156271 shutdown_connections 2024-03-15T12:56:38.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.946+0000 7fde83c4e640 1 --2- 172.21.15.2:0/2248156271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde7c107230 0x7fde7c109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:38.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.946+0000 7fde83c4e640 1 --2- 172.21.15.2:0/2248156271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde7c104900 0x7fde7c106cf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:38.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.946+0000 7fde83c4e640 1 --2- 172.21.15.2:0/2248156271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde7c101fd0 0x7fde7c1043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:38.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.946+0000 7fde83c4e640 1 -- 172.21.15.2:0/2248156271 >> 172.21.15.2:0/2248156271 conn(0x7fde7c0fb980 msgr2=0x7fde7c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:38.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.946+0000 7fde83c4e640 1 -- 172.21.15.2:0/2248156271 shutdown_connections 2024-03-15T12:56:38.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.946+0000 7fde83c4e640 1 -- 172.21.15.2:0/2248156271 wait complete. 2024-03-15T12:56:38.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.946+0000 7fde83c4e640 1 Processor -- start 2024-03-15T12:56:38.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.946+0000 7fde83c4e640 1 -- start start 2024-03-15T12:56:38.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.946+0000 7fde83c4e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde7c101fd0 0x7fde7c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:38.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.946+0000 7fde83c4e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde7c104900 0x7fde7c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:38.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.946+0000 7fde83c4e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde7c107230 0x7fde7c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:38.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde83c4e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c19f0d0 con 0x7fde7c107230 2024-03-15T12:56:38.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde83c4e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c19f240 con 0x7fde7c101fd0 2024-03-15T12:56:38.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde83c4e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c19f400 con 0x7fde7c104900 2024-03-15T12:56:38.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde821eb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde7c107230 0x7fde7c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:38.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde819ea640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde7c101fd0 0x7fde7c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:38.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde821eb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde7c107230 0x7fde7c19eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41336/0 (socket says 172.21.15.2:41336) 2024-03-15T12:56:38.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde819ea640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde7c101fd0 0x7fde7c1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:57744/0 (socket says 172.21.15.2:57744) 2024-03-15T12:56:38.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde821eb640 1 -- 172.21.15.2:0/2538499388 learned_addr learned my addr 172.21.15.2:0/2538499388 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:38.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde811e9640 1 --2- 172.21.15.2:0/2538499388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde7c104900 0x7fde7c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:38.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde821eb640 1 -- 172.21.15.2:0/2538499388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde7c104900 msgr2=0x7fde7c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:38.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde821eb640 1 --2- 172.21.15.2:0/2538499388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde7c104900 0x7fde7c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:38.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde821eb640 1 -- 172.21.15.2:0/2538499388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde7c101fd0 msgr2=0x7fde7c1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:38.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde821eb640 1 --2- 172.21.15.2:0/2538499388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde7c101fd0 0x7fde7c1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:38.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde821eb640 1 -- 172.21.15.2:0/2538499388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c009560 con 0x7fde7c107230 2024-03-15T12:56:38.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde819ea640 1 --2- 172.21.15.2:0/2538499388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde7c101fd0 0x7fde7c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:38.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde811e9640 1 --2- 172.21.15.2:0/2538499388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde7c104900 0x7fde7c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:56:38.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde821eb640 1 --2- 172.21.15.2:0/2538499388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde7c107230 0x7fde7c19eb90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fde7800eb10 tx=0x7fde7800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:38.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde72ffd640 1 -- 172.21.15.2:0/2538499388 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78004b20 con 0x7fde7c107230 2024-03-15T12:56:38.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde83c4e640 1 -- 172.21.15.2:0/2538499388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7c19f6e0 con 0x7fde7c107230 2024-03-15T12:56:38.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde72ffd640 1 -- 172.21.15.2:0/2538499388 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fde78004c80 con 0x7fde7c107230 2024-03-15T12:56:38.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde72ffd640 1 -- 172.21.15.2:0/2538499388 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78010690 con 0x7fde7c107230 2024-03-15T12:56:38.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.947+0000 7fde83c4e640 1 -- 172.21.15.2:0/2538499388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7c19fc10 con 0x7fde7c107230 2024-03-15T12:56:38.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.948+0000 7fde83c4e640 1 -- 172.21.15.2:0/2538499388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde7c0fd470 con 0x7fde7c107230 2024-03-15T12:56:38.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.950+0000 7fde72ffd640 1 -- 172.21.15.2:0/2538499388 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fde78004df0 con 0x7fde7c107230 2024-03-15T12:56:38.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.950+0000 7fde72ffd640 1 --2- 172.21.15.2:0/2538499388 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fde680a1a30 0x7fde680a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:38.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.951+0000 7fde819ea640 1 --2- 172.21.15.2:0/2538499388 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fde680a1a30 0x7fde680a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:38.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.951+0000 7fde72ffd640 1 -- 172.21.15.2:0/2538499388 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fde78014070 con 0x7fde7c107230 2024-03-15T12:56:38.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.951+0000 7fde819ea640 1 --2- 172.21.15.2:0/2538499388 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fde680a1a30 0x7fde680a3ef0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fde74009fc0 tx=0x7fde74005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:38.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:38.954+0000 7fde72ffd640 1 -- 172.21.15.2:0/2538499388 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fde78097100 con 0x7fde7c107230 2024-03-15T12:56:39.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:38 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2888527337' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-15T12:56:39.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:38 smithi002 ceph-mon[32060]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-15T12:56:39.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:38 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2888527337' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-15T12:56:39.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:38 smithi184 ceph-mon[35993]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-15T12:56:39.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:38 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2888527337' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-15T12:56:39.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:38 smithi149 ceph-mon[35927]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-15T12:56:39.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.176+0000 7fde83c4e640 1 -- 172.21.15.2:0/2538499388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fde7c02d000 con 0x7fde7c107230 2024-03-15T12:56:39.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.818+0000 7fde72ffd640 1 -- 172.21.15.2:0/2538499388 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7fde7809a7c0 con 0x7fde7c107230 2024-03-15T12:56:39.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.821+0000 7fde83c4e640 1 -- 172.21.15.2:0/2538499388 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fde680a1a30 msgr2=0x7fde680a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:39.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.821+0000 7fde83c4e640 1 --2- 172.21.15.2:0/2538499388 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fde680a1a30 0x7fde680a3ef0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fde74009fc0 tx=0x7fde74005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:39.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.821+0000 7fde83c4e640 1 -- 172.21.15.2:0/2538499388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde7c107230 msgr2=0x7fde7c19eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:39.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.821+0000 7fde83c4e640 1 --2- 172.21.15.2:0/2538499388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde7c107230 0x7fde7c19eb90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fde7800eb10 tx=0x7fde7800c2d0 comp rx=0 tx=0).stop 2024-03-15T12:56:39.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.821+0000 7fde83c4e640 1 -- 172.21.15.2:0/2538499388 shutdown_connections 2024-03-15T12:56:39.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.821+0000 7fde83c4e640 1 --2- 172.21.15.2:0/2538499388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde7c107230 0x7fde7c19eb90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:39.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.821+0000 7fde83c4e640 1 --2- 172.21.15.2:0/2538499388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde7c104900 0x7fde7c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:39.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.821+0000 7fde83c4e640 1 --2- 172.21.15.2:0/2538499388 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fde680a1a30 0x7fde680a3ef0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:39.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.821+0000 7fde83c4e640 1 --2- 172.21.15.2:0/2538499388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde7c101fd0 0x7fde7c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:39.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.821+0000 7fde83c4e640 1 -- 172.21.15.2:0/2538499388 >> 172.21.15.2:0/2538499388 conn(0x7fde7c0fb980 msgr2=0x7fde7c102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:39.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.821+0000 7fde83c4e640 1 -- 172.21.15.2:0/2538499388 shutdown_connections 2024-03-15T12:56:39.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.821+0000 7fde83c4e640 1 -- 172.21.15.2:0/2538499388 wait complete. 2024-03-15T12:56:39.824 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-15T12:56:39.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.950+0000 7fe77e9ac640 1 -- 172.21.15.2:0/322132067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7781048a0 msgr2=0x7fe778106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:39.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.950+0000 7fe77e9ac640 1 --2- 172.21.15.2:0/322132067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7781048a0 0x7fe778106c90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009900 tx=0x7fe76c02bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:39.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.950+0000 7fe77e9ac640 1 -- 172.21.15.2:0/322132067 shutdown_connections 2024-03-15T12:56:39.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.950+0000 7fe77e9ac640 1 --2- 172.21.15.2:0/322132067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7781071d0 0x7fe778109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:39.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.950+0000 7fe77e9ac640 1 --2- 172.21.15.2:0/322132067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7781048a0 0x7fe778106c90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:39.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.950+0000 7fe77e9ac640 1 --2- 172.21.15.2:0/322132067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe778101f70 0x7fe778104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:39.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.950+0000 7fe77e9ac640 1 -- 172.21.15.2:0/322132067 >> 172.21.15.2:0/322132067 conn(0x7fe7780fb980 msgr2=0x7fe7780fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:39.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.950+0000 7fe77e9ac640 1 -- 172.21.15.2:0/322132067 shutdown_connections 2024-03-15T12:56:39.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.950+0000 7fe77e9ac640 1 -- 172.21.15.2:0/322132067 wait complete. 2024-03-15T12:56:39.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.951+0000 7fe77e9ac640 1 Processor -- start 2024-03-15T12:56:39.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.951+0000 7fe77e9ac640 1 -- start start 2024-03-15T12:56:39.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.951+0000 7fe77e9ac640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe778101f70 0x7fe778197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:39.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.951+0000 7fe77e9ac640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7781048a0 0x7fe778197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:39.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.951+0000 7fe77e9ac640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7781071d0 0x7fe77819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:39.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.951+0000 7fe77e9ac640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77819f050 con 0x7fe7781071d0 2024-03-15T12:56:39.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.951+0000 7fe77e9ac640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77819f1c0 con 0x7fe778101f70 2024-03-15T12:56:39.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.951+0000 7fe77e9ac640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77819f380 con 0x7fe7781048a0 2024-03-15T12:56:39.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.951+0000 7fe77cf49640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7781071d0 0x7fe77819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:39.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.951+0000 7fe77cf49640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7781071d0 0x7fe77819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41366/0 (socket says 172.21.15.2:41366) 2024-03-15T12:56:39.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.951+0000 7fe77cf49640 1 -- 172.21.15.2:0/3691167512 learned_addr learned my addr 172.21.15.2:0/3691167512 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:39.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.951+0000 7fe777fff640 1 --2- 172.21.15.2:0/3691167512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe778101f70 0x7fe778197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:39.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.952+0000 7fe7777fe640 1 --2- 172.21.15.2:0/3691167512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7781048a0 0x7fe778197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:39.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.952+0000 7fe77cf49640 1 -- 172.21.15.2:0/3691167512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7781048a0 msgr2=0x7fe778197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:39.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.952+0000 7fe77cf49640 1 --2- 172.21.15.2:0/3691167512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7781048a0 0x7fe778197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:39.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.952+0000 7fe77cf49640 1 -- 172.21.15.2:0/3691167512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe778101f70 msgr2=0x7fe778197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:39.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.952+0000 7fe77cf49640 1 --2- 172.21.15.2:0/3691167512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe778101f70 0x7fe778197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:39.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.952+0000 7fe77cf49640 1 -- 172.21.15.2:0/3691167512 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c009560 con 0x7fe7781071d0 2024-03-15T12:56:39.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.952+0000 7fe777fff640 1 --2- 172.21.15.2:0/3691167512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe778101f70 0x7fe778197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:39.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.952+0000 7fe77cf49640 1 --2- 172.21.15.2:0/3691167512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7781071d0 0x7fe77819eb10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe77000cce0 tx=0x7fe77000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:39.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.952+0000 7fe7757fa640 1 -- 172.21.15.2:0/3691167512 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77000f450 con 0x7fe7781071d0 2024-03-15T12:56:39.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.952+0000 7fe7757fa640 1 -- 172.21.15.2:0/3691167512 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe77000fa80 con 0x7fe7781071d0 2024-03-15T12:56:39.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.952+0000 7fe77e9ac640 1 -- 172.21.15.2:0/3691167512 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77819f660 con 0x7fe7781071d0 2024-03-15T12:56:39.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.952+0000 7fe7757fa640 1 -- 172.21.15.2:0/3691167512 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770018b50 con 0x7fe7781071d0 2024-03-15T12:56:39.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.952+0000 7fe77e9ac640 1 -- 172.21.15.2:0/3691167512 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77819faf0 con 0x7fe7781071d0 2024-03-15T12:56:39.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.953+0000 7fe77e9ac640 1 -- 172.21.15.2:0/3691167512 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7780fd470 con 0x7fe7781071d0 2024-03-15T12:56:39.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.954+0000 7fe7757fa640 1 -- 172.21.15.2:0/3691167512 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fe77000fbf0 con 0x7fe7781071d0 2024-03-15T12:56:39.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.955+0000 7fe7757fa640 1 --2- 172.21.15.2:0/3691167512 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe7600a1a30 0x7fe7600a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:39.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.955+0000 7fe777fff640 1 --2- 172.21.15.2:0/3691167512 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe7600a1a30 0x7fe7600a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:39.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.955+0000 7fe7757fa640 1 -- 172.21.15.2:0/3691167512 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe7700cdd70 con 0x7fe7781071d0 2024-03-15T12:56:39.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.956+0000 7fe777fff640 1 --2- 172.21.15.2:0/3691167512 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe7600a1a30 0x7fe7600a3ef0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe768005d00 tx=0x7fe76800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:39.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:39.960+0000 7fe7757fa640 1 -- 172.21.15.2:0/3691167512 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe770096d40 con 0x7fe7781071d0 2024-03-15T12:56:40.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:39 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2888527337' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-15T12:56:40.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:39 smithi149 ceph-mon[35927]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-15T12:56:40.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:39 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2538499388' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-15T12:56:40.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.180+0000 7fe77e9ac640 1 -- 172.21.15.2:0/3691167512 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fe77802d000 con 0x7fe7781071d0 2024-03-15T12:56:40.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:39 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2888527337' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-15T12:56:40.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:39 smithi002 ceph-mon[32060]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-15T12:56:40.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:39 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2538499388' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-15T12:56:40.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:39 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2888527337' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-15T12:56:40.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:39 smithi184 ceph-mon[35993]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-15T12:56:40.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:39 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2538499388' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-15T12:56:40.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.394+0000 7fe7757fa640 1 -- 172.21.15.2:0/3691167512 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7fe77009a400 con 0x7fe7781071d0 2024-03-15T12:56:40.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.396+0000 7fe77e9ac640 1 -- 172.21.15.2:0/3691167512 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe7600a1a30 msgr2=0x7fe7600a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:40.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.396+0000 7fe77e9ac640 1 --2- 172.21.15.2:0/3691167512 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe7600a1a30 0x7fe7600a3ef0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe768005d00 tx=0x7fe76800b410 comp rx=0 tx=0).stop 2024-03-15T12:56:40.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.396+0000 7fe77e9ac640 1 -- 172.21.15.2:0/3691167512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7781071d0 msgr2=0x7fe77819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:40.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.396+0000 7fe77e9ac640 1 --2- 172.21.15.2:0/3691167512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7781071d0 0x7fe77819eb10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe77000cce0 tx=0x7fe77000a620 comp rx=0 tx=0).stop 2024-03-15T12:56:40.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.396+0000 7fe77e9ac640 1 -- 172.21.15.2:0/3691167512 shutdown_connections 2024-03-15T12:56:40.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.396+0000 7fe77e9ac640 1 --2- 172.21.15.2:0/3691167512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7781071d0 0x7fe77819eb10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:40.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.396+0000 7fe77e9ac640 1 --2- 172.21.15.2:0/3691167512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7781048a0 0x7fe778197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:40.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.396+0000 7fe77e9ac640 1 --2- 172.21.15.2:0/3691167512 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe7600a1a30 0x7fe7600a3ef0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:40.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.396+0000 7fe77e9ac640 1 --2- 172.21.15.2:0/3691167512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe778101f70 0x7fe778197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:40.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.396+0000 7fe77e9ac640 1 -- 172.21.15.2:0/3691167512 >> 172.21.15.2:0/3691167512 conn(0x7fe7780fb980 msgr2=0x7fe778102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:40.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.396+0000 7fe77e9ac640 1 -- 172.21.15.2:0/3691167512 shutdown_connections 2024-03-15T12:56:40.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.396+0000 7fe77e9ac640 1 -- 172.21.15.2:0/3691167512 wait complete. 2024-03-15T12:56:40.397 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-15T12:56:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 -- 172.21.15.2:0/3105734567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb01027b0 msgr2=0x7f1fb0102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 --2- 172.21.15.2:0/3105734567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb01027b0 0x7f1fb0102c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009900 tx=0x7f1fa002bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 -- 172.21.15.2:0/3105734567 shutdown_connections 2024-03-15T12:56:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 --2- 172.21.15.2:0/3105734567 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fb0103170 0x7f1fb0109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 --2- 172.21.15.2:0/3105734567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb01027b0 0x7f1fb0102c30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 --2- 172.21.15.2:0/3105734567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb00ff7e0 0x7f1fb00ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 -- 172.21.15.2:0/3105734567 >> 172.21.15.2:0/3105734567 conn(0x7f1fb00fb0f0 msgr2=0x7f1fb00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 -- 172.21.15.2:0/3105734567 shutdown_connections 2024-03-15T12:56:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 -- 172.21.15.2:0/3105734567 wait complete. 2024-03-15T12:56:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 Processor -- start 2024-03-15T12:56:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 -- start start 2024-03-15T12:56:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb00ff7e0 0x7f1fb01975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb01027b0 0x7f1fb0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fb0103170 0x7f1fb019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb019f0b0 con 0x7f1fb0103170 2024-03-15T12:56:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb019f220 con 0x7f1fb00ff7e0 2024-03-15T12:56:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.517+0000 7f1fb8464640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb019f3e0 con 0x7f1fb01027b0 2024-03-15T12:56:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fb6a01640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fb0103170 0x7f1fb019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fb6200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb00ff7e0 0x7f1fb01975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fb6200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb00ff7e0 0x7f1fb01975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:57788/0 (socket says 172.21.15.2:57788) 2024-03-15T12:56:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fb6200640 1 -- 172.21.15.2:0/4160380749 learned_addr learned my addr 172.21.15.2:0/4160380749 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fb6a01640 1 --2- 172.21.15.2:0/4160380749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fb0103170 0x7f1fb019eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41384/0 (socket says 172.21.15.2:41384) 2024-03-15T12:56:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fb59ff640 1 --2- 172.21.15.2:0/4160380749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb01027b0 0x7f1fb0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fb6a01640 1 -- 172.21.15.2:0/4160380749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb01027b0 msgr2=0x7f1fb0197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fb6a01640 1 --2- 172.21.15.2:0/4160380749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb01027b0 0x7f1fb0197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fb6a01640 1 -- 172.21.15.2:0/4160380749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb00ff7e0 msgr2=0x7f1fb01975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fb6a01640 1 --2- 172.21.15.2:0/4160380749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb00ff7e0 0x7f1fb01975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fb6a01640 1 -- 172.21.15.2:0/4160380749 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa0009560 con 0x7f1fb0103170 2024-03-15T12:56:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fb6a01640 1 --2- 172.21.15.2:0/4160380749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fb0103170 0x7f1fb019eb70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009fd0 tx=0x7f1fa800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fa77fe640 1 -- 172.21.15.2:0/4160380749 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa800ce20 con 0x7f1fb0103170 2024-03-15T12:56:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fb8464640 1 -- 172.21.15.2:0/4160380749 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb019f6c0 con 0x7f1fb0103170 2024-03-15T12:56:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fa77fe640 1 -- 172.21.15.2:0/4160380749 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1fa8005020 con 0x7f1fb0103170 2024-03-15T12:56:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fa77fe640 1 -- 172.21.15.2:0/4160380749 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8010690 con 0x7f1fb0103170 2024-03-15T12:56:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.518+0000 7f1fb8464640 1 -- 172.21.15.2:0/4160380749 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb019fba0 con 0x7f1fb0103170 2024-03-15T12:56:40.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.519+0000 7f1fb8464640 1 -- 172.21.15.2:0/4160380749 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb01088e0 con 0x7f1fb0103170 2024-03-15T12:56:40.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.521+0000 7f1fa77fe640 1 -- 172.21.15.2:0/4160380749 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f1fa8005570 con 0x7f1fb0103170 2024-03-15T12:56:40.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.521+0000 7f1fa77fe640 1 --2- 172.21.15.2:0/4160380749 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1f9c0a1a30 0x7f1f9c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:40.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.521+0000 7f1fb6200640 1 --2- 172.21.15.2:0/4160380749 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1f9c0a1a30 0x7f1f9c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:40.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.521+0000 7f1fa77fe640 1 -- 172.21.15.2:0/4160380749 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f1fa8014070 con 0x7f1fb0103170 2024-03-15T12:56:40.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.522+0000 7f1fb6200640 1 --2- 172.21.15.2:0/4160380749 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1f9c0a1a30 0x7f1f9c0a3ef0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1fac005fd0 tx=0x7f1fac00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:40.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.526+0000 7f1fa77fe640 1 -- 172.21.15.2:0/4160380749 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1fa809a880 con 0x7f1fb0103170 2024-03-15T12:56:40.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:40.749+0000 7f1fb8464640 1 -- 172.21.15.2:0/4160380749 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f1fb002cf90 con 0x7f1fb0103170 2024-03-15T12:56:41.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:40 smithi149 ceph-mon[35927]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-15T12:56:41.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:40 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2538499388' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-15T12:56:41.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:40 smithi149 ceph-mon[35927]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-15T12:56:41.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:40 smithi149 ceph-mon[35927]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-15T12:56:41.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:40 smithi149 ceph-mon[35927]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-15T12:56:41.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:40 smithi149 ceph-mon[35927]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-15T12:56:41.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:40 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3691167512' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-15T12:56:41.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:40 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3691167512' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-15T12:56:41.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:40 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] up:active 2024-03-15T12:56:41.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:40 smithi149 ceph-mon[35927]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-15T12:56:41.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:40 smithi149 ceph-mon[35927]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 2 up:standby-replay 8 up:standby 2024-03-15T12:56:41.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:40 smithi149 ceph-mon[35927]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2024-03-15T12:56:41.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:40 smithi149 ceph-mon[35927]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-15T12:56:41.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:40 smithi149 ceph-mon[35927]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-15T12:56:41.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:40 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/4160380749' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-15T12:56:41.289 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:40 smithi002 ceph-mon[32060]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-15T12:56:41.289 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:40 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2538499388' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-15T12:56:41.290 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:40 smithi002 ceph-mon[32060]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-15T12:56:41.290 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:40 smithi002 ceph-mon[32060]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-15T12:56:41.290 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:40 smithi002 ceph-mon[32060]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-15T12:56:41.290 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:40 smithi002 ceph-mon[32060]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-15T12:56:41.290 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:40 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3691167512' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-15T12:56:41.290 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:40 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3691167512' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-15T12:56:41.290 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:40 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] up:active 2024-03-15T12:56:41.290 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:40 smithi002 ceph-mon[32060]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-15T12:56:41.290 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:40 smithi002 ceph-mon[32060]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 2 up:standby-replay 8 up:standby 2024-03-15T12:56:41.290 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:40 smithi002 ceph-mon[32060]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2024-03-15T12:56:41.290 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:40 smithi002 ceph-mon[32060]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-15T12:56:41.290 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:40 smithi002 ceph-mon[32060]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-15T12:56:41.291 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:40 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/4160380749' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-15T12:56:41.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:40 smithi184 ceph-mon[35993]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-15T12:56:41.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:40 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2538499388' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-15T12:56:41.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:40 smithi184 ceph-mon[35993]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-15T12:56:41.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:40 smithi184 ceph-mon[35993]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-15T12:56:41.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:40 smithi184 ceph-mon[35993]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-15T12:56:41.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:40 smithi184 ceph-mon[35993]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-15T12:56:41.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:40 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3691167512' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-15T12:56:41.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:40 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3691167512' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-15T12:56:41.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:40 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] up:active 2024-03-15T12:56:41.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:40 smithi184 ceph-mon[35993]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-15T12:56:41.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:40 smithi184 ceph-mon[35993]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 2 up:standby-replay 8 up:standby 2024-03-15T12:56:41.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:40 smithi184 ceph-mon[35993]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2024-03-15T12:56:41.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:40 smithi184 ceph-mon[35993]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-15T12:56:41.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:40 smithi184 ceph-mon[35993]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-15T12:56:41.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:40 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/4160380749' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-15T12:56:41.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.405+0000 7f1fa77fe640 1 -- 172.21.15.2:0/4160380749 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f1fa809a410 con 0x7f1fb0103170 2024-03-15T12:56:41.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.407+0000 7f1fb8464640 1 -- 172.21.15.2:0/4160380749 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1f9c0a1a30 msgr2=0x7f1f9c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:41.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.407+0000 7f1fb8464640 1 --2- 172.21.15.2:0/4160380749 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1f9c0a1a30 0x7f1f9c0a3ef0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1fac005fd0 tx=0x7f1fac00b410 comp rx=0 tx=0).stop 2024-03-15T12:56:41.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.407+0000 7f1fb8464640 1 -- 172.21.15.2:0/4160380749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fb0103170 msgr2=0x7f1fb019eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:41.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.407+0000 7f1fb8464640 1 --2- 172.21.15.2:0/4160380749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fb0103170 0x7f1fb019eb70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009fd0 tx=0x7f1fa800ef10 comp rx=0 tx=0).stop 2024-03-15T12:56:41.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.407+0000 7f1fb8464640 1 -- 172.21.15.2:0/4160380749 shutdown_connections 2024-03-15T12:56:41.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.407+0000 7f1fb8464640 1 --2- 172.21.15.2:0/4160380749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fb0103170 0x7f1fb019eb70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.407+0000 7f1fb8464640 1 --2- 172.21.15.2:0/4160380749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1fb01027b0 0x7f1fb0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.407+0000 7f1fb8464640 1 --2- 172.21.15.2:0/4160380749 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1f9c0a1a30 0x7f1f9c0a3ef0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.407+0000 7f1fb8464640 1 --2- 172.21.15.2:0/4160380749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb00ff7e0 0x7f1fb01975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.407+0000 7f1fb8464640 1 -- 172.21.15.2:0/4160380749 >> 172.21.15.2:0/4160380749 conn(0x7f1fb00fb0f0 msgr2=0x7f1fb0105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:41.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.407+0000 7f1fb8464640 1 -- 172.21.15.2:0/4160380749 shutdown_connections 2024-03-15T12:56:41.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.407+0000 7f1fb8464640 1 -- 172.21.15.2:0/4160380749 wait complete. 2024-03-15T12:56:41.408 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-03-15T12:56:41.409 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-15T12:56:41.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.532+0000 7f1e48e57640 1 -- 172.21.15.2:0/3048417794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e44101500 msgr2=0x7f1e44101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:41.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.532+0000 7f1e48e57640 1 --2- 172.21.15.2:0/3048417794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e44101500 0x7f1e44101920 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009900 tx=0x7f1e3c02bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:41.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.532+0000 7f1e48e57640 1 -- 172.21.15.2:0/3048417794 shutdown_connections 2024-03-15T12:56:41.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.532+0000 7f1e48e57640 1 --2- 172.21.15.2:0/3048417794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e441030b0 0x7f1e44109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.532+0000 7f1e48e57640 1 --2- 172.21.15.2:0/3048417794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e441026f0 0x7f1e44102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.532+0000 7f1e48e57640 1 --2- 172.21.15.2:0/3048417794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e44101500 0x7f1e44101920 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.532+0000 7f1e48e57640 1 -- 172.21.15.2:0/3048417794 >> 172.21.15.2:0/3048417794 conn(0x7f1e440fca90 msgr2=0x7f1e440feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:41.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.532+0000 7f1e48e57640 1 -- 172.21.15.2:0/3048417794 shutdown_connections 2024-03-15T12:56:41.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.532+0000 7f1e48e57640 1 -- 172.21.15.2:0/3048417794 wait complete. 2024-03-15T12:56:41.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.532+0000 7f1e48e57640 1 Processor -- start 2024-03-15T12:56:41.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.532+0000 7f1e48e57640 1 -- start start 2024-03-15T12:56:41.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e48e57640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e44101500 0x7f1e441975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:41.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e48e57640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e441026f0 0x7f1e44197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:41.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e48e57640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e441030b0 0x7f1e4419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:41.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e48e57640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4419f0a0 con 0x7f1e441026f0 2024-03-15T12:56:41.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e48e57640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4419f210 con 0x7f1e441030b0 2024-03-15T12:56:41.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e48e57640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4419f3d0 con 0x7f1e44101500 2024-03-15T12:56:41.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e41d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e441026f0 0x7f1e44197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:41.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e41d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e441026f0 0x7f1e44197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41400/0 (socket says 172.21.15.2:41400) 2024-03-15T12:56:41.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e41d9b640 1 -- 172.21.15.2:0/2381095265 learned_addr learned my addr 172.21.15.2:0/2381095265 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:41.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e42d9d640 1 --2- 172.21.15.2:0/2381095265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e441030b0 0x7f1e4419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:41.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e4259c640 1 --2- 172.21.15.2:0/2381095265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e44101500 0x7f1e441975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:41.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e41d9b640 1 -- 172.21.15.2:0/2381095265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e44101500 msgr2=0x7f1e441975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:41.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e41d9b640 1 --2- 172.21.15.2:0/2381095265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e44101500 0x7f1e441975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e41d9b640 1 -- 172.21.15.2:0/2381095265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e441030b0 msgr2=0x7f1e4419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:41.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e41d9b640 1 --2- 172.21.15.2:0/2381095265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e441030b0 0x7f1e4419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e41d9b640 1 -- 172.21.15.2:0/2381095265 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3c009560 con 0x7f1e441026f0 2024-03-15T12:56:41.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e41d9b640 1 --2- 172.21.15.2:0/2381095265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e441026f0 0x7f1e44197af0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00d950 tx=0x7f1e2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:41.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e3b7fe640 1 -- 172.21.15.2:0/2381095265 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c009a30 con 0x7f1e441026f0 2024-03-15T12:56:41.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e3b7fe640 1 -- 172.21.15.2:0/2381095265 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1e2c009b90 con 0x7f1e441026f0 2024-03-15T12:56:41.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e48e57640 1 -- 172.21.15.2:0/2381095265 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4419f6b0 con 0x7f1e441026f0 2024-03-15T12:56:41.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e3b7fe640 1 -- 172.21.15.2:0/2381095265 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0047a0 con 0x7f1e441026f0 2024-03-15T12:56:41.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.533+0000 7f1e48e57640 1 -- 172.21.15.2:0/2381095265 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4419fbc0 con 0x7f1e441026f0 2024-03-15T12:56:41.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.534+0000 7f1e48e57640 1 -- 172.21.15.2:0/2381095265 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e441088a0 con 0x7f1e441026f0 2024-03-15T12:56:41.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.535+0000 7f1e3b7fe640 1 -- 172.21.15.2:0/2381095265 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f1e2c009d00 con 0x7f1e441026f0 2024-03-15T12:56:41.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.536+0000 7f1e3b7fe640 1 --2- 172.21.15.2:0/2381095265 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1e300a1a30 0x7f1e300a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:41.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.536+0000 7f1e4259c640 1 --2- 172.21.15.2:0/2381095265 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1e300a1a30 0x7f1e300a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:41.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.536+0000 7f1e3b7fe640 1 -- 172.21.15.2:0/2381095265 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f1e2c0cd060 con 0x7f1e441026f0 2024-03-15T12:56:41.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.536+0000 7f1e4259c640 1 --2- 172.21.15.2:0/2381095265 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1e300a1a30 0x7f1e300a3ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c02c740 tx=0x7f1e3c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:41.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.540+0000 7f1e3b7fe640 1 -- 172.21.15.2:0/2381095265 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1e2c096030 con 0x7f1e441026f0 2024-03-15T12:56:41.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.749+0000 7f1e48e57640 1 -- 172.21.15.2:0/2381095265 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e4402d070 con 0x7f1e300a1a30 2024-03-15T12:56:41.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.772+0000 7f1e3b7fe640 1 -- 172.21.15.2:0/2381095265 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1e4402d070 con 0x7f1e300a1a30 2024-03-15T12:56:41.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.774+0000 7f1e48e57640 1 -- 172.21.15.2:0/2381095265 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1e300a1a30 msgr2=0x7f1e300a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:41.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.774+0000 7f1e48e57640 1 --2- 172.21.15.2:0/2381095265 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1e300a1a30 0x7f1e300a3ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c02c740 tx=0x7f1e3c005fd0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.774+0000 7f1e48e57640 1 -- 172.21.15.2:0/2381095265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e441026f0 msgr2=0x7f1e44197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:41.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.774+0000 7f1e48e57640 1 --2- 172.21.15.2:0/2381095265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e441026f0 0x7f1e44197af0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00d950 tx=0x7f1e2c00de10 comp rx=0 tx=0).stop 2024-03-15T12:56:41.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.774+0000 7f1e48e57640 1 -- 172.21.15.2:0/2381095265 shutdown_connections 2024-03-15T12:56:41.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.774+0000 7f1e48e57640 1 --2- 172.21.15.2:0/2381095265 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f1e300a1a30 0x7f1e300a3ef0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.774+0000 7f1e48e57640 1 --2- 172.21.15.2:0/2381095265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e441030b0 0x7f1e4419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.774+0000 7f1e48e57640 1 --2- 172.21.15.2:0/2381095265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e441026f0 0x7f1e44197af0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.774+0000 7f1e48e57640 1 --2- 172.21.15.2:0/2381095265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e44101500 0x7f1e441975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.774+0000 7f1e48e57640 1 -- 172.21.15.2:0/2381095265 >> 172.21.15.2:0/2381095265 conn(0x7f1e440fca90 msgr2=0x7f1e44105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:41.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.774+0000 7f1e48e57640 1 -- 172.21.15.2:0/2381095265 shutdown_connections 2024-03-15T12:56:41.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.774+0000 7f1e48e57640 1 -- 172.21.15.2:0/2381095265 wait complete. 2024-03-15T12:56:41.771 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-15T12:56:41.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.892+0000 7f4bd611e640 1 -- 172.21.15.2:0/578895817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bd01071d0 msgr2=0x7f4bd0109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:41.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.892+0000 7f4bd611e640 1 --2- 172.21.15.2:0/578895817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bd01071d0 0x7f4bd0109620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009ca0 tx=0x7f4bc802c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.892+0000 7f4bd611e640 1 -- 172.21.15.2:0/578895817 shutdown_connections 2024-03-15T12:56:41.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.892+0000 7f4bd611e640 1 --2- 172.21.15.2:0/578895817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bd01071d0 0x7f4bd0109620 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.892+0000 7f4bd611e640 1 --2- 172.21.15.2:0/578895817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bd01048a0 0x7f4bd0106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.892+0000 7f4bd611e640 1 --2- 172.21.15.2:0/578895817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bd0101f70 0x7f4bd0104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.892+0000 7f4bd611e640 1 -- 172.21.15.2:0/578895817 >> 172.21.15.2:0/578895817 conn(0x7f4bd00fb980 msgr2=0x7f4bd00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:41.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.892+0000 7f4bd611e640 1 -- 172.21.15.2:0/578895817 shutdown_connections 2024-03-15T12:56:41.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.892+0000 7f4bd611e640 1 -- 172.21.15.2:0/578895817 wait complete. 2024-03-15T12:56:41.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.893+0000 7f4bd611e640 1 Processor -- start 2024-03-15T12:56:41.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.893+0000 7f4bd611e640 1 -- start start 2024-03-15T12:56:41.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.893+0000 7f4bd611e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bd0101f70 0x7f4bd01974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:41.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.893+0000 7f4bd611e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bd01048a0 0x7f4bd0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:41.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.893+0000 7f4bd611e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bd01071d0 0x7f4bd019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:41.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.893+0000 7f4bd611e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd019efd0 con 0x7f4bd0101f70 2024-03-15T12:56:41.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.893+0000 7f4bd611e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd019f140 con 0x7f4bd01048a0 2024-03-15T12:56:41.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.893+0000 7f4bd611e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd019f300 con 0x7f4bd01071d0 2024-03-15T12:56:41.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.893+0000 7f4bcf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bd0101f70 0x7f4bd01974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:41.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.893+0000 7f4bcf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bd0101f70 0x7f4bd01974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41430/0 (socket says 172.21.15.2:41430) 2024-03-15T12:56:41.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.893+0000 7f4bcf7fe640 1 -- 172.21.15.2:0/3266739451 learned_addr learned my addr 172.21.15.2:0/3266739451 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:41.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.893+0000 7f4bceffd640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bd01048a0 0x7f4bd0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:41.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.894+0000 7f4bcffff640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bd01071d0 0x7f4bd019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:41.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.894+0000 7f4bcf7fe640 1 -- 172.21.15.2:0/3266739451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bd01071d0 msgr2=0x7f4bd019ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:41.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.894+0000 7f4bcf7fe640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bd01071d0 0x7f4bd019ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.894+0000 7f4bcf7fe640 1 -- 172.21.15.2:0/3266739451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bd01048a0 msgr2=0x7f4bd0197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:41.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.894+0000 7f4bcf7fe640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bd01048a0 0x7f4bd0197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:41.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.894+0000 7f4bcf7fe640 1 -- 172.21.15.2:0/3266739451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc8009900 con 0x7f4bd0101f70 2024-03-15T12:56:41.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.894+0000 7f4bcffff640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bd01071d0 0x7f4bd019ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T12:56:41.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.894+0000 7f4bceffd640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bd01048a0 0x7f4bd0197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:41.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.894+0000 7f4bcf7fe640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bd0101f70 0x7f4bd01974e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000c920 tx=0x7f4bc000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:41.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.894+0000 7f4bccff9640 1 -- 172.21.15.2:0/3266739451 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0007c70 con 0x7f4bd0101f70 2024-03-15T12:56:41.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.894+0000 7f4bd611e640 1 -- 172.21.15.2:0/3266739451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd019f5e0 con 0x7f4bd0101f70 2024-03-15T12:56:41.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.894+0000 7f4bccff9640 1 -- 172.21.15.2:0/3266739451 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4bc0007dd0 con 0x7f4bd0101f70 2024-03-15T12:56:41.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.894+0000 7f4bccff9640 1 -- 172.21.15.2:0/3266739451 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0018820 con 0x7f4bd0101f70 2024-03-15T12:56:41.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.894+0000 7f4bd611e640 1 -- 172.21.15.2:0/3266739451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd019faf0 con 0x7f4bd0101f70 2024-03-15T12:56:41.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.895+0000 7f4bd611e640 1 -- 172.21.15.2:0/3266739451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd00fd470 con 0x7f4bd0101f70 2024-03-15T12:56:41.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.896+0000 7f4bccff9640 1 -- 172.21.15.2:0/3266739451 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f4bc0018980 con 0x7f4bd0101f70 2024-03-15T12:56:41.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.897+0000 7f4bccff9640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4bb80a1a30 0x7f4bb80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:41.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.897+0000 7f4bceffd640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4bb80a1a30 0x7f4bb80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:41.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.897+0000 7f4bccff9640 1 -- 172.21.15.2:0/3266739451 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4bc00cd7c0 con 0x7f4bd0101f70 2024-03-15T12:56:41.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.897+0000 7f4bceffd640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4bb80a1a30 0x7f4bb80a3ef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4bd01989f0 tx=0x7f4bc4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:41.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:41.901+0000 7f4bccff9640 1 -- 172.21.15.2:0/3266739451 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4bc0096790 con 0x7f4bd0101f70 2024-03-15T12:56:42.053 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:56:41 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:56:41.753+0000 7fa426449700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:56:42.053 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:56:41 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:56:41.753+0000 7fa426449700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:56:42.053 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:56:41 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:56:41.753+0000 7fa426449700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:56:42.053 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:56:41 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:56:41.753+0000 7fa426449700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:56:42.054 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:56:41 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:56:41.753+0000 7fa426449700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:56:42.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.110+0000 7f4bd611e640 1 -- 172.21.15.2:0/3266739451 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bd002d070 con 0x7f4bb80a1a30 2024-03-15T12:56:42.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.132+0000 7f4bccff9640 1 -- 172.21.15.2:0/3266739451 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4bd002d070 con 0x7f4bb80a1a30 2024-03-15T12:56:42.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.134+0000 7f4bd611e640 1 -- 172.21.15.2:0/3266739451 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4bb80a1a30 msgr2=0x7f4bb80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:42.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.134+0000 7f4bd611e640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4bb80a1a30 0x7f4bb80a3ef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4bd01989f0 tx=0x7f4bc4005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:42.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.134+0000 7f4bd611e640 1 -- 172.21.15.2:0/3266739451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bd0101f70 msgr2=0x7f4bd01974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:42.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.134+0000 7f4bd611e640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bd0101f70 0x7f4bd01974e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000c920 tx=0x7f4bc000cde0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.134+0000 7f4bd611e640 1 -- 172.21.15.2:0/3266739451 shutdown_connections 2024-03-15T12:56:42.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.134+0000 7f4bd611e640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bd01071d0 0x7f4bd019ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.134+0000 7f4bd611e640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4bb80a1a30 0x7f4bb80a3ef0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.134+0000 7f4bd611e640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bd01048a0 0x7f4bd0197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.134+0000 7f4bd611e640 1 --2- 172.21.15.2:0/3266739451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bd0101f70 0x7f4bd01974e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.134+0000 7f4bd611e640 1 -- 172.21.15.2:0/3266739451 >> 172.21.15.2:0/3266739451 conn(0x7f4bd00fb980 msgr2=0x7f4bd01028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:42.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.135+0000 7f4bd611e640 1 -- 172.21.15.2:0/3266739451 shutdown_connections 2024-03-15T12:56:42.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.135+0000 7f4bd611e640 1 -- 172.21.15.2:0/3266739451 wait complete. 2024-03-15T12:56:42.132 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-15T12:56:42.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.253+0000 7fcd330c3640 1 -- 172.21.15.2:0/1901342692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd2c1026f0 msgr2=0x7fcd2c102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:42.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.253+0000 7fcd330c3640 1 --2- 172.21.15.2:0/1901342692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd2c1026f0 0x7fcd2c102b70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009900 tx=0x7fcd1802bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:42.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.253+0000 7fcd330c3640 1 -- 172.21.15.2:0/1901342692 shutdown_connections 2024-03-15T12:56:42.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.253+0000 7fcd330c3640 1 --2- 172.21.15.2:0/1901342692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd2c1030b0 0x7fcd2c109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.253+0000 7fcd330c3640 1 --2- 172.21.15.2:0/1901342692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd2c1026f0 0x7fcd2c102b70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.253+0000 7fcd330c3640 1 --2- 172.21.15.2:0/1901342692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd2c101500 0x7fcd2c101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.253+0000 7fcd330c3640 1 -- 172.21.15.2:0/1901342692 >> 172.21.15.2:0/1901342692 conn(0x7fcd2c0fca90 msgr2=0x7fcd2c0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:42.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.253+0000 7fcd330c3640 1 -- 172.21.15.2:0/1901342692 shutdown_connections 2024-03-15T12:56:42.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.253+0000 7fcd330c3640 1 -- 172.21.15.2:0/1901342692 wait complete. 2024-03-15T12:56:42.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.253+0000 7fcd330c3640 1 Processor -- start 2024-03-15T12:56:42.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd330c3640 1 -- start start 2024-03-15T12:56:42.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd330c3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd2c101500 0x7fcd2c197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:42.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd330c3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd2c1026f0 0x7fcd2c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:42.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd330c3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd2c1030b0 0x7fcd2c19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:42.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd330c3640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c19eff0 con 0x7fcd2c1026f0 2024-03-15T12:56:42.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd330c3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c19f160 con 0x7fcd2c101500 2024-03-15T12:56:42.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd330c3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c19f320 con 0x7fcd2c1030b0 2024-03-15T12:56:42.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd2bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd2c1026f0 0x7fcd2c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:42.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd2bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd2c1026f0 0x7fcd2c197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41448/0 (socket says 172.21.15.2:41448) 2024-03-15T12:56:42.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd2bfff640 1 -- 172.21.15.2:0/4038603240 learned_addr learned my addr 172.21.15.2:0/4038603240 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:42.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd30e5f640 1 --2- 172.21.15.2:0/4038603240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd2c101500 0x7fcd2c197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:42.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd31660640 1 --2- 172.21.15.2:0/4038603240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd2c1030b0 0x7fcd2c19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:42.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd2bfff640 1 -- 172.21.15.2:0/4038603240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd2c1030b0 msgr2=0x7fcd2c19eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:42.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd2bfff640 1 --2- 172.21.15.2:0/4038603240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd2c1030b0 0x7fcd2c19eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd2bfff640 1 -- 172.21.15.2:0/4038603240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd2c101500 msgr2=0x7fcd2c197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:42.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd2bfff640 1 --2- 172.21.15.2:0/4038603240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd2c101500 0x7fcd2c197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd2bfff640 1 -- 172.21.15.2:0/4038603240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd18009560 con 0x7fcd2c1026f0 2024-03-15T12:56:42.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd31660640 1 --2- 172.21.15.2:0/4038603240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd2c1030b0 0x7fcd2c19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T12:56:42.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd2bfff640 1 --2- 172.21.15.2:0/4038603240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd2c1026f0 0x7fcd2c197a40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcd18005ab0 tx=0x7fcd18004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:42.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd29ffb640 1 -- 172.21.15.2:0/4038603240 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd180042d0 con 0x7fcd2c1026f0 2024-03-15T12:56:42.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd29ffb640 1 -- 172.21.15.2:0/4038603240 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcd18004430 con 0x7fcd2c1026f0 2024-03-15T12:56:42.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd330c3640 1 -- 172.21.15.2:0/4038603240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2c19f5a0 con 0x7fcd2c1026f0 2024-03-15T12:56:42.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.254+0000 7fcd29ffb640 1 -- 172.21.15.2:0/4038603240 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18003e20 con 0x7fcd2c1026f0 2024-03-15T12:56:42.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.255+0000 7fcd330c3640 1 -- 172.21.15.2:0/4038603240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2c19fb00 con 0x7fcd2c1026f0 2024-03-15T12:56:42.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.256+0000 7fcd330c3640 1 -- 172.21.15.2:0/4038603240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2c1088a0 con 0x7fcd2c1026f0 2024-03-15T12:56:42.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.257+0000 7fcd29ffb640 1 -- 172.21.15.2:0/4038603240 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fcd180035f0 con 0x7fcd2c1026f0 2024-03-15T12:56:42.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.257+0000 7fcd29ffb640 1 --2- 172.21.15.2:0/4038603240 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcd1c0a1ab0 0x7fcd1c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:42.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.257+0000 7fcd30e5f640 1 --2- 172.21.15.2:0/4038603240 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcd1c0a1ab0 0x7fcd1c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:42.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.258+0000 7fcd29ffb640 1 -- 172.21.15.2:0/4038603240 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fcd180ef810 con 0x7fcd2c1026f0 2024-03-15T12:56:42.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.258+0000 7fcd30e5f640 1 --2- 172.21.15.2:0/4038603240 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcd1c0a1ab0 0x7fcd1c0a3f70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c102550 tx=0x7fcd20005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:42.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.262+0000 7fcd29ffb640 1 -- 172.21.15.2:0/4038603240 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcd180b87e0 con 0x7fcd2c1026f0 2024-03-15T12:56:42.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.478+0000 7fcd330c3640 1 -- 172.21.15.2:0/4038603240 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd2c02d070 con 0x7fcd1c0a1ab0 2024-03-15T12:56:42.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.482+0000 7fcd29ffb640 1 -- 172.21.15.2:0/4038603240 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fcd2c02d070 con 0x7fcd1c0a1ab0 2024-03-15T12:56:42.468 INFO:teuthology.orchestra.run.smithi002.stdout:/volumes/qa/sv_0/6cf3dd6a-a2de-45bf-928c-1914473d1145 2024-03-15T12:56:42.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.483+0000 7fcd330c3640 1 -- 172.21.15.2:0/4038603240 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcd1c0a1ab0 msgr2=0x7fcd1c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:42.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.483+0000 7fcd330c3640 1 --2- 172.21.15.2:0/4038603240 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcd1c0a1ab0 0x7fcd1c0a3f70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c102550 tx=0x7fcd20005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:42.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.483+0000 7fcd330c3640 1 -- 172.21.15.2:0/4038603240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd2c1026f0 msgr2=0x7fcd2c197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:42.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.483+0000 7fcd330c3640 1 --2- 172.21.15.2:0/4038603240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd2c1026f0 0x7fcd2c197a40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcd18005ab0 tx=0x7fcd18004e50 comp rx=0 tx=0).stop 2024-03-15T12:56:42.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.484+0000 7fcd330c3640 1 -- 172.21.15.2:0/4038603240 shutdown_connections 2024-03-15T12:56:42.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.484+0000 7fcd330c3640 1 --2- 172.21.15.2:0/4038603240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd2c1030b0 0x7fcd2c19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.484+0000 7fcd330c3640 1 --2- 172.21.15.2:0/4038603240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd2c1026f0 0x7fcd2c197a40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.484+0000 7fcd330c3640 1 --2- 172.21.15.2:0/4038603240 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcd1c0a1ab0 0x7fcd1c0a3f70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.484+0000 7fcd330c3640 1 --2- 172.21.15.2:0/4038603240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd2c101500 0x7fcd2c197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.484+0000 7fcd330c3640 1 -- 172.21.15.2:0/4038603240 >> 172.21.15.2:0/4038603240 conn(0x7fcd2c0fca90 msgr2=0x7fcd2c105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:42.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.484+0000 7fcd330c3640 1 -- 172.21.15.2:0/4038603240 shutdown_connections 2024-03-15T12:56:42.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.484+0000 7fcd330c3640 1 -- 172.21.15.2:0/4038603240 wait complete. 2024-03-15T12:56:42.481 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-15T12:56:42.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.603+0000 7f187daed640 1 -- 172.21.15.2:0/1908507002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1878100530 msgr2=0x7f18781009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:42.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.603+0000 7f187daed640 1 --2- 172.21.15.2:0/1908507002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1878100530 0x7f18781009b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f186c009900 tx=0x7f186c02bc30 comp rx=0 tx=0).stop 2024-03-15T12:56:42.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.603+0000 7f187daed640 1 -- 172.21.15.2:0/1908507002 shutdown_connections 2024-03-15T12:56:42.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.603+0000 7f187daed640 1 --2- 172.21.15.2:0/1908507002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1878100ef0 0x7f1878101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.603+0000 7f187daed640 1 --2- 172.21.15.2:0/1908507002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1878100530 0x7f18781009b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.603+0000 7f187daed640 1 --2- 172.21.15.2:0/1908507002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18780ff7d0 0x7f18780ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.603+0000 7f187daed640 1 -- 172.21.15.2:0/1908507002 >> 172.21.15.2:0/1908507002 conn(0x7f18780fb120 msgr2=0x7f18780fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:42.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.603+0000 7f187daed640 1 -- 172.21.15.2:0/1908507002 shutdown_connections 2024-03-15T12:56:42.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.603+0000 7f187daed640 1 -- 172.21.15.2:0/1908507002 wait complete. 2024-03-15T12:56:42.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.603+0000 7f187daed640 1 Processor -- start 2024-03-15T12:56:42.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.603+0000 7f187daed640 1 -- start start 2024-03-15T12:56:42.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f187daed640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18780ff7d0 0x7f18781997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:42.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f187daed640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1878100530 0x7f1878199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:42.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f187daed640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1878100ef0 0x7f18781a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:42.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f187daed640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18781a12c0 con 0x7f18780ff7d0 2024-03-15T12:56:42.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f187daed640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18781a1430 con 0x7f1878100ef0 2024-03-15T12:56:42.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f187daed640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18781a15f0 con 0x7f1878100530 2024-03-15T12:56:42.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f18777fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18780ff7d0 0x7f18781997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:42.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f18777fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18780ff7d0 0x7f18781997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41454/0 (socket says 172.21.15.2:41454) 2024-03-15T12:56:42.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f18777fe640 1 -- 172.21.15.2:0/1864981463 learned_addr learned my addr 172.21.15.2:0/1864981463 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:42.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f1877fff640 1 --2- 172.21.15.2:0/1864981463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1878100ef0 0x7f18781a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:42.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f1876ffd640 1 --2- 172.21.15.2:0/1864981463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1878100530 0x7f1878199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:42.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f18777fe640 1 -- 172.21.15.2:0/1864981463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1878100530 msgr2=0x7f1878199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:42.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f18777fe640 1 --2- 172.21.15.2:0/1864981463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1878100530 0x7f1878199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f18777fe640 1 -- 172.21.15.2:0/1864981463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1878100ef0 msgr2=0x7f18781a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:42.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f18777fe640 1 --2- 172.21.15.2:0/1864981463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1878100ef0 0x7f18781a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f18777fe640 1 -- 172.21.15.2:0/1864981463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c009560 con 0x7f18780ff7d0 2024-03-15T12:56:42.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f18777fe640 1 --2- 172.21.15.2:0/1864981463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18780ff7d0 0x7f18781997d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f186800d950 tx=0x7f186800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:42.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f1874ff9640 1 -- 172.21.15.2:0/1864981463 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868009a30 con 0x7f18780ff7d0 2024-03-15T12:56:42.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f1874ff9640 1 -- 172.21.15.2:0/1864981463 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1868009b90 con 0x7f18780ff7d0 2024-03-15T12:56:42.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f187daed640 1 -- 172.21.15.2:0/1864981463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18781a18d0 con 0x7f18780ff7d0 2024-03-15T12:56:42.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f1874ff9640 1 -- 172.21.15.2:0/1864981463 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18680047a0 con 0x7f18780ff7d0 2024-03-15T12:56:42.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.604+0000 7f187daed640 1 -- 172.21.15.2:0/1864981463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18781a1db0 con 0x7f18780ff7d0 2024-03-15T12:56:42.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.605+0000 7f187daed640 1 -- 172.21.15.2:0/1864981463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187810b410 con 0x7f18780ff7d0 2024-03-15T12:56:42.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.606+0000 7f1874ff9640 1 -- 172.21.15.2:0/1864981463 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f1868009d00 con 0x7f18780ff7d0 2024-03-15T12:56:42.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.607+0000 7f1874ff9640 1 --2- 172.21.15.2:0/1864981463 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f18600a1a30 0x7f18600a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:42.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.607+0000 7f1876ffd640 1 --2- 172.21.15.2:0/1864981463 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f18600a1a30 0x7f18600a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:42.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.607+0000 7f1874ff9640 1 -- 172.21.15.2:0/1864981463 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f18680cd0c0 con 0x7f18780ff7d0 2024-03-15T12:56:42.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.607+0000 7f1876ffd640 1 --2- 172.21.15.2:0/1864981463 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f18600a1a30 0x7f18600a3ef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f186c009230 tx=0x7f186c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:42.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.612+0000 7f1874ff9640 1 -- 172.21.15.2:0/1864981463 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1868096030 con 0x7f18780ff7d0 2024-03-15T12:56:42.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:42 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/4160380749' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-15T12:56:42.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:42 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] up:active 2024-03-15T12:56:42.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:42 smithi002 ceph-mon[32060]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-15T12:56:42.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:42 smithi002 ceph-mon[32060]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-03-15T12:56:42.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:42 smithi002 ceph-mon[32060]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:creating} 2 up:standby-replay 6 up:standby 2024-03-15T12:56:42.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:42 smithi002 ceph-mon[32060]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-03-15T12:56:42.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.820+0000 7f187daed640 1 -- 172.21.15.2:0/1864981463 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f187802d070 con 0x7f18600a1a30 2024-03-15T12:56:42.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:42 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/4160380749' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-15T12:56:42.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:42 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] up:active 2024-03-15T12:56:42.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:42 smithi184 ceph-mon[35993]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-15T12:56:42.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:42 smithi184 ceph-mon[35993]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-03-15T12:56:42.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:42 smithi184 ceph-mon[35993]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:creating} 2 up:standby-replay 6 up:standby 2024-03-15T12:56:42.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:42 smithi184 ceph-mon[35993]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-03-15T12:56:42.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.843+0000 7f1874ff9640 1 -- 172.21.15.2:0/1864981463 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f187802d070 con 0x7f18600a1a30 2024-03-15T12:56:42.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.845+0000 7f187daed640 1 -- 172.21.15.2:0/1864981463 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f18600a1a30 msgr2=0x7f18600a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:42.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.845+0000 7f187daed640 1 --2- 172.21.15.2:0/1864981463 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f18600a1a30 0x7f18600a3ef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f186c009230 tx=0x7f186c000bc0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.845+0000 7f187daed640 1 -- 172.21.15.2:0/1864981463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18780ff7d0 msgr2=0x7f18781997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:42.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.845+0000 7f187daed640 1 --2- 172.21.15.2:0/1864981463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18780ff7d0 0x7f18781997d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f186800d950 tx=0x7f186800de10 comp rx=0 tx=0).stop 2024-03-15T12:56:42.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.845+0000 7f187daed640 1 -- 172.21.15.2:0/1864981463 shutdown_connections 2024-03-15T12:56:42.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.845+0000 7f187daed640 1 --2- 172.21.15.2:0/1864981463 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f18600a1a30 0x7f18600a3ef0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.845+0000 7f187daed640 1 --2- 172.21.15.2:0/1864981463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1878100ef0 0x7f18781a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.845+0000 7f187daed640 1 --2- 172.21.15.2:0/1864981463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1878100530 0x7f1878199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.845+0000 7f187daed640 1 --2- 172.21.15.2:0/1864981463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18780ff7d0 0x7f18781997d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.845+0000 7f187daed640 1 -- 172.21.15.2:0/1864981463 >> 172.21.15.2:0/1864981463 conn(0x7f18780fb120 msgr2=0x7f18780fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:42.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.845+0000 7f187daed640 1 -- 172.21.15.2:0/1864981463 shutdown_connections 2024-03-15T12:56:42.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.845+0000 7f187daed640 1 -- 172.21.15.2:0/1864981463 wait complete. 2024-03-15T12:56:42.842 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-15T12:56:42.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:42 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/4160380749' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-15T12:56:42.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:42 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] up:active 2024-03-15T12:56:42.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:42 smithi149 ceph-mon[35927]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-15T12:56:42.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:42 smithi149 ceph-mon[35927]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-03-15T12:56:42.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:42 smithi149 ceph-mon[35927]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:creating} 2 up:standby-replay 6 up:standby 2024-03-15T12:56:42.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:42 smithi149 ceph-mon[35927]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-03-15T12:56:42.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.964+0000 7fbd96ad4640 1 -- 172.21.15.2:0/3515940954 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd901030b0 msgr2=0x7fbd90109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:42.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.964+0000 7fbd96ad4640 1 --2- 172.21.15.2:0/3515940954 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd901030b0 0x7fbd90109940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009ca0 tx=0x7fbd8802c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.964+0000 7fbd96ad4640 1 -- 172.21.15.2:0/3515940954 shutdown_connections 2024-03-15T12:56:42.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.964+0000 7fbd96ad4640 1 --2- 172.21.15.2:0/3515940954 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd901030b0 0x7fbd90109940 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.964+0000 7fbd96ad4640 1 --2- 172.21.15.2:0/3515940954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd901026f0 0x7fbd90102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.964+0000 7fbd96ad4640 1 --2- 172.21.15.2:0/3515940954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd90101500 0x7fbd90101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.964+0000 7fbd96ad4640 1 -- 172.21.15.2:0/3515940954 >> 172.21.15.2:0/3515940954 conn(0x7fbd900fca90 msgr2=0x7fbd900feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:42.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.964+0000 7fbd96ad4640 1 -- 172.21.15.2:0/3515940954 shutdown_connections 2024-03-15T12:56:42.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.964+0000 7fbd96ad4640 1 -- 172.21.15.2:0/3515940954 wait complete. 2024-03-15T12:56:42.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd96ad4640 1 Processor -- start 2024-03-15T12:56:42.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd96ad4640 1 -- start start 2024-03-15T12:56:42.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd96ad4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd90101500 0x7fbd90197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:42.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd96ad4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd901026f0 0x7fbd90197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:42.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd96ad4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd901030b0 0x7fbd9019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:42.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd96ad4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9019f070 con 0x7fbd901026f0 2024-03-15T12:56:42.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd96ad4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9019f1e0 con 0x7fbd901030b0 2024-03-15T12:56:42.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd96ad4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9019f3a0 con 0x7fbd90101500 2024-03-15T12:56:42.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd8ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd901026f0 0x7fbd90197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:42.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd8ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd901026f0 0x7fbd90197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41464/0 (socket says 172.21.15.2:41464) 2024-03-15T12:56:42.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd94870640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd90101500 0x7fbd90197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:42.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd8ffff640 1 -- 172.21.15.2:0/212743759 learned_addr learned my addr 172.21.15.2:0/212743759 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:42.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd95071640 1 --2- 172.21.15.2:0/212743759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd901030b0 0x7fbd9019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:42.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd8ffff640 1 -- 172.21.15.2:0/212743759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd90101500 msgr2=0x7fbd90197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:42.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd8ffff640 1 --2- 172.21.15.2:0/212743759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd90101500 0x7fbd90197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd8ffff640 1 -- 172.21.15.2:0/212743759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd901030b0 msgr2=0x7fbd9019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:42.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd8ffff640 1 --2- 172.21.15.2:0/212743759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd901030b0 0x7fbd9019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:42.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd8ffff640 1 -- 172.21.15.2:0/212743759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd84009490 con 0x7fbd901026f0 2024-03-15T12:56:42.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd8ffff640 1 --2- 172.21.15.2:0/212743759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd901026f0 0x7fbd90197ac0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400ea60 tx=0x7fbd8400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:42.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.965+0000 7fbd8dffb640 1 -- 172.21.15.2:0/212743759 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8400cde0 con 0x7fbd901026f0 2024-03-15T12:56:42.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.966+0000 7fbd96ad4640 1 -- 172.21.15.2:0/212743759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd88009900 con 0x7fbd901026f0 2024-03-15T12:56:42.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.966+0000 7fbd8dffb640 1 -- 172.21.15.2:0/212743759 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbd840035f0 con 0x7fbd901026f0 2024-03-15T12:56:42.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.966+0000 7fbd8dffb640 1 -- 172.21.15.2:0/212743759 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84010690 con 0x7fbd901026f0 2024-03-15T12:56:42.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.966+0000 7fbd96ad4640 1 -- 172.21.15.2:0/212743759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9019fa00 con 0x7fbd901026f0 2024-03-15T12:56:42.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.967+0000 7fbd96ad4640 1 -- 172.21.15.2:0/212743759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd90108820 con 0x7fbd901026f0 2024-03-15T12:56:42.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.968+0000 7fbd8dffb640 1 -- 172.21.15.2:0/212743759 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fbd84003760 con 0x7fbd901026f0 2024-03-15T12:56:42.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.968+0000 7fbd8dffb640 1 --2- 172.21.15.2:0/212743759 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbd780a1a30 0x7fbd780a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:42.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.969+0000 7fbd94870640 1 --2- 172.21.15.2:0/212743759 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbd780a1a30 0x7fbd780a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:42.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.969+0000 7fbd8dffb640 1 -- 172.21.15.2:0/212743759 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fbd84014070 con 0x7fbd901026f0 2024-03-15T12:56:42.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.969+0000 7fbd94870640 1 --2- 172.21.15.2:0/212743759 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbd780a1a30 0x7fbd780a3ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd90102550 tx=0x7fbd80005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:42.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:42.973+0000 7fbd8dffb640 1 -- 172.21.15.2:0/212743759 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbd84096a70 con 0x7fbd901026f0 2024-03-15T12:56:43.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.183+0000 7fbd96ad4640 1 -- 172.21.15.2:0/212743759 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd9002d000 con 0x7fbd780a1a30 2024-03-15T12:56:43.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.186+0000 7fbd8dffb640 1 -- 172.21.15.2:0/212743759 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fbd9002d000 con 0x7fbd780a1a30 2024-03-15T12:56:43.172 INFO:teuthology.orchestra.run.smithi002.stdout:/volumes/qa/sv_1/f8ccb93d-c0aa-4d73-a0a7-8c3eb7e59ca6 2024-03-15T12:56:43.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.189+0000 7fbd96ad4640 1 -- 172.21.15.2:0/212743759 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbd780a1a30 msgr2=0x7fbd780a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:43.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.189+0000 7fbd96ad4640 1 --2- 172.21.15.2:0/212743759 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbd780a1a30 0x7fbd780a3ef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd90102550 tx=0x7fbd80005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:43.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.189+0000 7fbd96ad4640 1 -- 172.21.15.2:0/212743759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd901026f0 msgr2=0x7fbd90197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:43.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.189+0000 7fbd96ad4640 1 --2- 172.21.15.2:0/212743759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd901026f0 0x7fbd90197ac0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400ea60 tx=0x7fbd8400ef20 comp rx=0 tx=0).stop 2024-03-15T12:56:43.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.189+0000 7fbd96ad4640 1 -- 172.21.15.2:0/212743759 shutdown_connections 2024-03-15T12:56:43.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.189+0000 7fbd96ad4640 1 --2- 172.21.15.2:0/212743759 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbd780a1a30 0x7fbd780a3ef0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.189+0000 7fbd96ad4640 1 --2- 172.21.15.2:0/212743759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd901030b0 0x7fbd9019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.189+0000 7fbd96ad4640 1 --2- 172.21.15.2:0/212743759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd901026f0 0x7fbd90197ac0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.189+0000 7fbd96ad4640 1 --2- 172.21.15.2:0/212743759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd90101500 0x7fbd90197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.189+0000 7fbd96ad4640 1 -- 172.21.15.2:0/212743759 >> 172.21.15.2:0/212743759 conn(0x7fbd900fca90 msgr2=0x7fbd901059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:43.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.189+0000 7fbd96ad4640 1 -- 172.21.15.2:0/212743759 shutdown_connections 2024-03-15T12:56:43.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.189+0000 7fbd96ad4640 1 -- 172.21.15.2:0/212743759 wait complete. 2024-03-15T12:56:43.186 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:56:43.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.304+0000 7fb611b6c640 1 -- 172.21.15.2:0/723906296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c101fc0 msgr2=0x7fb60c108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:43.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.304+0000 7fb611b6c640 1 --2- 172.21.15.2:0/723906296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c101fc0 0x7fb60c108850 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb604009ca0 tx=0x7fb60402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.304+0000 7fb611b6c640 1 -- 172.21.15.2:0/723906296 shutdown_connections 2024-03-15T12:56:43.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.304+0000 7fb611b6c640 1 --2- 172.21.15.2:0/723906296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c101fc0 0x7fb60c108850 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.304+0000 7fb611b6c640 1 --2- 172.21.15.2:0/723906296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb60c101600 0x7fb60c101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.304+0000 7fb611b6c640 1 --2- 172.21.15.2:0/723906296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb60c100410 0x7fb60c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.304+0000 7fb611b6c640 1 -- 172.21.15.2:0/723906296 >> 172.21.15.2:0/723906296 conn(0x7fb60c0fb980 msgr2=0x7fb60c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:43.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.304+0000 7fb611b6c640 1 -- 172.21.15.2:0/723906296 shutdown_connections 2024-03-15T12:56:43.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.304+0000 7fb611b6c640 1 -- 172.21.15.2:0/723906296 wait complete. 2024-03-15T12:56:43.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb611b6c640 1 Processor -- start 2024-03-15T12:56:43.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb611b6c640 1 -- start start 2024-03-15T12:56:43.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb611b6c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c100410 0x7fb60c104470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:43.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb611b6c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb60c101600 0x7fb60c102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:43.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb611b6c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb60c101fc0 0x7fb60c102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:43.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb611b6c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c103750 con 0x7fb60c100410 2024-03-15T12:56:43.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb611b6c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c1038c0 con 0x7fb60c101fc0 2024-03-15T12:56:43.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb611b6c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c1a15c0 con 0x7fb60c101600 2024-03-15T12:56:43.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb60b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c100410 0x7fb60c104470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:43.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb60b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c100410 0x7fb60c104470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41496/0 (socket says 172.21.15.2:41496) 2024-03-15T12:56:43.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb60b7fe640 1 -- 172.21.15.2:0/3113032203 learned_addr learned my addr 172.21.15.2:0/3113032203 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:43.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb60affd640 1 --2- 172.21.15.2:0/3113032203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb60c101600 0x7fb60c102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:43.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb60bfff640 1 --2- 172.21.15.2:0/3113032203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb60c101fc0 0x7fb60c102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb60b7fe640 1 -- 172.21.15.2:0/3113032203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb60c101600 msgr2=0x7fb60c102ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb60b7fe640 1 --2- 172.21.15.2:0/3113032203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb60c101600 0x7fb60c102ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb60b7fe640 1 -- 172.21.15.2:0/3113032203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb60c101fc0 msgr2=0x7fb60c102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.305+0000 7fb60b7fe640 1 --2- 172.21.15.2:0/3113032203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb60c101fc0 0x7fb60c102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.306+0000 7fb60b7fe640 1 -- 172.21.15.2:0/3113032203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb604009900 con 0x7fb60c100410 2024-03-15T12:56:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.306+0000 7fb60b7fe640 1 --2- 172.21.15.2:0/3113032203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c100410 0x7fb60c104470 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00e9f0 tx=0x7fb5fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.306+0000 7fb608ff9640 1 -- 172.21.15.2:0/3113032203 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc009700 con 0x7fb60c100410 2024-03-15T12:56:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.306+0000 7fb608ff9640 1 -- 172.21.15.2:0/3113032203 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb5fc0044f0 con 0x7fb60c100410 2024-03-15T12:56:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.306+0000 7fb611b6c640 1 -- 172.21.15.2:0/3113032203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60c1a1870 con 0x7fb60c100410 2024-03-15T12:56:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.306+0000 7fb608ff9640 1 -- 172.21.15.2:0/3113032203 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc010450 con 0x7fb60c100410 2024-03-15T12:56:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.306+0000 7fb611b6c640 1 -- 172.21.15.2:0/3113032203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60c1a1d80 con 0x7fb60c100410 2024-03-15T12:56:43.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.307+0000 7fb611b6c640 1 -- 172.21.15.2:0/3113032203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60c10bdf0 con 0x7fb60c100410 2024-03-15T12:56:43.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.308+0000 7fb608ff9640 1 -- 172.21.15.2:0/3113032203 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fb5fc00cd80 con 0x7fb60c100410 2024-03-15T12:56:43.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.309+0000 7fb608ff9640 1 --2- 172.21.15.2:0/3113032203 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb5f40a1a30 0x7fb5f40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:43.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.309+0000 7fb60affd640 1 --2- 172.21.15.2:0/3113032203 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb5f40a1a30 0x7fb5f40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:43.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.309+0000 7fb608ff9640 1 -- 172.21.15.2:0/3113032203 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb5fc014070 con 0x7fb60c100410 2024-03-15T12:56:43.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.309+0000 7fb60affd640 1 --2- 172.21.15.2:0/3113032203 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb5f40a1a30 0x7fb5f40a3ef0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb60c1041e0 tx=0x7fb600005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:43.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.314+0000 7fb608ff9640 1 -- 172.21.15.2:0/3113032203 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb5fc096910 con 0x7fb60c100410 2024-03-15T12:56:43.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.433+0000 7fb608ff9640 1 -- 172.21.15.2:0/3113032203 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7fb5fc096ce0 con 0x7fb60c100410 2024-03-15T12:56:43.469 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:43 smithi002 ceph-mon[32060]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:56:43.469 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:43 smithi002 ceph-mon[32060]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:56:43.469 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:43 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] up:active 2024-03-15T12:56:43.469 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:43 smithi002 ceph-mon[32060]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-15T12:56:43.469 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:43 smithi002 ceph-mon[32060]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-15T12:56:43.469 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:43 smithi002 ceph-mon[32060]: Cluster is now healthy 2024-03-15T12:56:43.469 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:43 smithi002 ceph-mon[32060]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:active} 2 up:standby-replay 6 up:standby 2024-03-15T12:56:43.469 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:43 smithi002 ceph-mon[32060]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-03-15T12:56:43.469 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:43 smithi002 ceph-mon[32060]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-03-15T12:56:43.469 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:43 smithi002 ceph-mon[32060]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:56:43.469 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:43 smithi002 ceph-mon[32060]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-15T12:56:43.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.542+0000 7fb611b6c640 1 -- 172.21.15.2:0/3113032203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb60c1a1fa0 con 0x7fb60c100410 2024-03-15T12:56:43.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.543+0000 7fb608ff9640 1 -- 172.21.15.2:0/3113032203 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11453 (secure 0 0 0) 0x7fb5fc02b4a0 con 0x7fb60c100410 2024-03-15T12:56:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:43.530 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T12:56:43.433836+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24464,24500,24460,24539,24434]},"id":1}]} 2024-03-15T12:56:43.530 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 28 2024-03-15T12:56:43.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.545+0000 7fb611b6c640 1 -- 172.21.15.2:0/3113032203 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb5f40a1a30 msgr2=0x7fb5f40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:43.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.545+0000 7fb611b6c640 1 --2- 172.21.15.2:0/3113032203 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb5f40a1a30 0x7fb5f40a3ef0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb60c1041e0 tx=0x7fb600005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:43.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.545+0000 7fb611b6c640 1 -- 172.21.15.2:0/3113032203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c100410 msgr2=0x7fb60c104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:43.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.545+0000 7fb611b6c640 1 --2- 172.21.15.2:0/3113032203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c100410 0x7fb60c104470 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00e9f0 tx=0x7fb5fc00eeb0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.545+0000 7fb611b6c640 1 -- 172.21.15.2:0/3113032203 shutdown_connections 2024-03-15T12:56:43.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.545+0000 7fb611b6c640 1 --2- 172.21.15.2:0/3113032203 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb5f40a1a30 0x7fb5f40a3ef0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.545+0000 7fb611b6c640 1 --2- 172.21.15.2:0/3113032203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb60c101fc0 0x7fb60c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.545+0000 7fb611b6c640 1 --2- 172.21.15.2:0/3113032203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb60c101600 0x7fb60c102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.545+0000 7fb611b6c640 1 --2- 172.21.15.2:0/3113032203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c100410 0x7fb60c104470 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.545+0000 7fb611b6c640 1 -- 172.21.15.2:0/3113032203 >> 172.21.15.2:0/3113032203 conn(0x7fb60c0fb980 msgr2=0x7fb60c068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:43.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.545+0000 7fb611b6c640 1 -- 172.21.15.2:0/3113032203 shutdown_connections 2024-03-15T12:56:43.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.545+0000 7fb611b6c640 1 -- 172.21.15.2:0/3113032203 wait complete. 2024-03-15T12:56:43.542 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:56:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.661+0000 7f7d5c71e640 1 -- 172.21.15.2:0/1667758641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d54100410 msgr2=0x7f7d54100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.661+0000 7f7d5c71e640 1 --2- 172.21.15.2:0/1667758641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d54100410 0x7f7d54100830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009860 tx=0x7f7d4402bb70 comp rx=0 tx=0).stop 2024-03-15T12:56:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.662+0000 7f7d5c71e640 1 -- 172.21.15.2:0/1667758641 shutdown_connections 2024-03-15T12:56:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.662+0000 7f7d5c71e640 1 --2- 172.21.15.2:0/1667758641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d54101fc0 0x7f7d54108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.662+0000 7f7d5c71e640 1 --2- 172.21.15.2:0/1667758641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d54101600 0x7f7d54101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.662+0000 7f7d5c71e640 1 --2- 172.21.15.2:0/1667758641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d54100410 0x7f7d54100830 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.662+0000 7f7d5c71e640 1 -- 172.21.15.2:0/1667758641 >> 172.21.15.2:0/1667758641 conn(0x7f7d540fb980 msgr2=0x7f7d540fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.662+0000 7f7d5c71e640 1 -- 172.21.15.2:0/1667758641 shutdown_connections 2024-03-15T12:56:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.662+0000 7f7d5c71e640 1 -- 172.21.15.2:0/1667758641 wait complete. 2024-03-15T12:56:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.662+0000 7f7d5c71e640 1 Processor -- start 2024-03-15T12:56:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.662+0000 7f7d5c71e640 1 -- start start 2024-03-15T12:56:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.662+0000 7f7d5c71e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d54100410 0x7f7d54197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.662+0000 7f7d5c71e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d54101600 0x7f7d54197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.662+0000 7f7d5c71e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d54101fc0 0x7f7d5419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.662+0000 7f7d5c71e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5419f130 con 0x7f7d54101fc0 2024-03-15T12:56:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.662+0000 7f7d5c71e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5419f2a0 con 0x7f7d54101600 2024-03-15T12:56:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.662+0000 7f7d5c71e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5419f460 con 0x7f7d54100410 2024-03-15T12:56:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d59cb9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d54101600 0x7f7d54197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d5a4ba640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d54100410 0x7f7d54197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d5acbb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d54101fc0 0x7f7d5419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d59cb9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d54101600 0x7f7d54197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:57936/0 (socket says 172.21.15.2:57936) 2024-03-15T12:56:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d59cb9640 1 -- 172.21.15.2:0/2023522141 learned_addr learned my addr 172.21.15.2:0/2023522141 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d59cb9640 1 -- 172.21.15.2:0/2023522141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d54100410 msgr2=0x7f7d54197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d59cb9640 1 --2- 172.21.15.2:0/2023522141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d54100410 0x7f7d54197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d59cb9640 1 -- 172.21.15.2:0/2023522141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d54101fc0 msgr2=0x7f7d5419ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d59cb9640 1 --2- 172.21.15.2:0/2023522141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d54101fc0 0x7f7d5419ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d59cb9640 1 -- 172.21.15.2:0/2023522141 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4c009900 con 0x7f7d54101600 2024-03-15T12:56:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d5a4ba640 1 --2- 172.21.15.2:0/2023522141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d54100410 0x7f7d54197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d5acbb640 1 --2- 172.21.15.2:0/2023522141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d54101fc0 0x7f7d5419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d59cb9640 1 --2- 172.21.15.2:0/2023522141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d54101600 0x7f7d54197b80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000e9f0 tx=0x7f7d5000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d4b7fe640 1 -- 172.21.15.2:0/2023522141 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50009700 con 0x7f7d54101600 2024-03-15T12:56:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d5c71e640 1 -- 172.21.15.2:0/2023522141 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d44009490 con 0x7f7d54101600 2024-03-15T12:56:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d4b7fe640 1 -- 172.21.15.2:0/2023522141 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7d5000cd80 con 0x7f7d54101600 2024-03-15T12:56:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d4b7fe640 1 -- 172.21.15.2:0/2023522141 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50018920 con 0x7f7d54101600 2024-03-15T12:56:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.663+0000 7f7d5c71e640 1 -- 172.21.15.2:0/2023522141 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5419fac0 con 0x7f7d54101600 2024-03-15T12:56:43.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.664+0000 7f7d5c71e640 1 -- 172.21.15.2:0/2023522141 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5410bde0 con 0x7f7d54101600 2024-03-15T12:56:43.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.666+0000 7f7d4b7fe640 1 -- 172.21.15.2:0/2023522141 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f7d50018a80 con 0x7f7d54101600 2024-03-15T12:56:43.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.666+0000 7f7d4b7fe640 1 --2- 172.21.15.2:0/2023522141 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7d400a1ac0 0x7f7d400a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:43.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.666+0000 7f7d5a4ba640 1 --2- 172.21.15.2:0/2023522141 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7d400a1ac0 0x7f7d400a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:43.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.666+0000 7f7d4b7fe640 1 -- 172.21.15.2:0/2023522141 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f7d50014070 con 0x7f7d54101600 2024-03-15T12:56:43.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.667+0000 7f7d5a4ba640 1 --2- 172.21.15.2:0/2023522141 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7d400a1ac0 0x7f7d400a3f80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d4402c180 tx=0x7f7d44034b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:43.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.674+0000 7f7d4b7fe640 1 -- 172.21.15.2:0/2023522141 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7d50096f00 con 0x7f7d54101600 2024-03-15T12:56:43.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:43 smithi184 ceph-mon[35993]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:56:43.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:43 smithi184 ceph-mon[35993]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:56:43.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:43 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] up:active 2024-03-15T12:56:43.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:43 smithi184 ceph-mon[35993]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-15T12:56:43.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:43 smithi184 ceph-mon[35993]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-15T12:56:43.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:43 smithi184 ceph-mon[35993]: Cluster is now healthy 2024-03-15T12:56:43.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:43 smithi184 ceph-mon[35993]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:active} 2 up:standby-replay 6 up:standby 2024-03-15T12:56:43.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:43 smithi184 ceph-mon[35993]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-03-15T12:56:43.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:43 smithi184 ceph-mon[35993]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-03-15T12:56:43.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:43 smithi184 ceph-mon[35993]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:56:43.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:43 smithi184 ceph-mon[35993]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-15T12:56:43.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.858+0000 7f7d5c71e640 1 -- 172.21.15.2:0/2023522141 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d5410bfa0 con 0x7f7d54101600 2024-03-15T12:56:43.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.861+0000 7f7d4b7fe640 1 -- 172.21.15.2:0/2023522141 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24045 (secure 0 0 0) 0x7f7d5009a5c0 con 0x7f7d54101600 2024-03-15T12:56:43.847 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:43.847 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":89,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:56:25.830426+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:56:43.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.863+0000 7f7d5c71e640 1 -- 172.21.15.2:0/2023522141 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7d400a1ac0 msgr2=0x7f7d400a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:43.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.863+0000 7f7d5c71e640 1 --2- 172.21.15.2:0/2023522141 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7d400a1ac0 0x7f7d400a3f80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d4402c180 tx=0x7f7d44034b60 comp rx=0 tx=0).stop 2024-03-15T12:56:43.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.863+0000 7f7d5c71e640 1 -- 172.21.15.2:0/2023522141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d54101600 msgr2=0x7f7d54197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:43.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.863+0000 7f7d5c71e640 1 --2- 172.21.15.2:0/2023522141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d54101600 0x7f7d54197b80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000e9f0 tx=0x7f7d5000eeb0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.863+0000 7f7d5c71e640 1 -- 172.21.15.2:0/2023522141 shutdown_connections 2024-03-15T12:56:43.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.863+0000 7f7d5c71e640 1 --2- 172.21.15.2:0/2023522141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d54101fc0 0x7f7d5419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.863+0000 7f7d5c71e640 1 --2- 172.21.15.2:0/2023522141 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7d400a1ac0 0x7f7d400a3f80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.863+0000 7f7d5c71e640 1 --2- 172.21.15.2:0/2023522141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d54101600 0x7f7d54197b80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.863+0000 7f7d5c71e640 1 --2- 172.21.15.2:0/2023522141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d54100410 0x7f7d54197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.863+0000 7f7d5c71e640 1 -- 172.21.15.2:0/2023522141 >> 172.21.15.2:0/2023522141 conn(0x7f7d540fb980 msgr2=0x7f7d54068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:43.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.863+0000 7f7d5c71e640 1 -- 172.21.15.2:0/2023522141 shutdown_connections 2024-03-15T12:56:43.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.863+0000 7f7d5c71e640 1 -- 172.21.15.2:0/2023522141 wait complete. 2024-03-15T12:56:43.860 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-15T12:56:43.861 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-15T12:56:43.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:43 smithi149 ceph-mon[35927]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:56:43.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:43 smithi149 ceph-mon[35927]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:56:43.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:43 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] up:active 2024-03-15T12:56:43.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:43 smithi149 ceph-mon[35927]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-15T12:56:43.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:43 smithi149 ceph-mon[35927]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-15T12:56:43.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:43 smithi149 ceph-mon[35927]: Cluster is now healthy 2024-03-15T12:56:43.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:43 smithi149 ceph-mon[35927]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=f=up:active} 2 up:standby-replay 6 up:standby 2024-03-15T12:56:43.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:43 smithi149 ceph-mon[35927]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-03-15T12:56:43.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:43 smithi149 ceph-mon[35927]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-03-15T12:56:43.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:43 smithi149 ceph-mon[35927]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:56:43.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:43 smithi149 ceph-mon[35927]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-15T12:56:43.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.980+0000 7f97e3b47640 1 -- 172.21.15.2:0/4276947688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97dc107270 msgr2=0x7f97dc109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:43.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.980+0000 7f97e3b47640 1 --2- 172.21.15.2:0/4276947688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97dc107270 0x7f97dc109660 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009ca0 tx=0x7f97d402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.980+0000 7f97e3b47640 1 -- 172.21.15.2:0/4276947688 shutdown_connections 2024-03-15T12:56:43.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.980+0000 7f97e3b47640 1 --2- 172.21.15.2:0/4276947688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97dc107270 0x7f97dc109660 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.980+0000 7f97e3b47640 1 --2- 172.21.15.2:0/4276947688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97dc104940 0x7f97dc106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.980+0000 7f97e3b47640 1 --2- 172.21.15.2:0/4276947688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97dc102010 0x7f97dc104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.980+0000 7f97e3b47640 1 -- 172.21.15.2:0/4276947688 >> 172.21.15.2:0/4276947688 conn(0x7f97dc0fb980 msgr2=0x7f97dc0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:43.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.980+0000 7f97e3b47640 1 -- 172.21.15.2:0/4276947688 shutdown_connections 2024-03-15T12:56:43.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.980+0000 7f97e3b47640 1 -- 172.21.15.2:0/4276947688 wait complete. 2024-03-15T12:56:43.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e3b47640 1 Processor -- start 2024-03-15T12:56:43.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e3b47640 1 -- start start 2024-03-15T12:56:43.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e3b47640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97dc102010 0x7f97dc197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:43.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e3b47640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97dc104940 0x7f97dc197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:43.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e3b47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97dc107270 0x7f97dc19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e3b47640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc19f030 con 0x7f97dc102010 2024-03-15T12:56:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e3b47640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc19f1a0 con 0x7f97dc104940 2024-03-15T12:56:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e3b47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc19f360 con 0x7f97dc107270 2024-03-15T12:56:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e18e3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97dc102010 0x7f97dc197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e18e3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97dc102010 0x7f97dc197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41526/0 (socket says 172.21.15.2:41526) 2024-03-15T12:56:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e18e3640 1 -- 172.21.15.2:0/1006015807 learned_addr learned my addr 172.21.15.2:0/1006015807 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e10e2640 1 --2- 172.21.15.2:0/1006015807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97dc104940 0x7f97dc197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e20e4640 1 --2- 172.21.15.2:0/1006015807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97dc107270 0x7f97dc19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e18e3640 1 -- 172.21.15.2:0/1006015807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97dc107270 msgr2=0x7f97dc19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e18e3640 1 --2- 172.21.15.2:0/1006015807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97dc107270 0x7f97dc19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e18e3640 1 -- 172.21.15.2:0/1006015807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97dc104940 msgr2=0x7f97dc197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e18e3640 1 --2- 172.21.15.2:0/1006015807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97dc104940 0x7f97dc197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.981+0000 7f97e18e3640 1 -- 172.21.15.2:0/1006015807 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d4009900 con 0x7f97dc102010 2024-03-15T12:56:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.982+0000 7f97e18e3640 1 --2- 172.21.15.2:0/1006015807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97dc102010 0x7f97dc197540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00e9f0 tx=0x7f97cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.982+0000 7f97d2ffd640 1 -- 172.21.15.2:0/1006015807 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc009700 con 0x7f97dc102010 2024-03-15T12:56:43.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.982+0000 7f97d2ffd640 1 -- 172.21.15.2:0/1006015807 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f97cc00cd80 con 0x7f97dc102010 2024-03-15T12:56:43.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.982+0000 7f97e3b47640 1 -- 172.21.15.2:0/1006015807 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97dc19f640 con 0x7f97dc102010 2024-03-15T12:56:43.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.982+0000 7f97d2ffd640 1 -- 172.21.15.2:0/1006015807 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc018920 con 0x7f97dc102010 2024-03-15T12:56:43.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.982+0000 7f97e3b47640 1 -- 172.21.15.2:0/1006015807 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97dc19fba0 con 0x7f97dc102010 2024-03-15T12:56:43.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.983+0000 7f97e3b47640 1 -- 172.21.15.2:0/1006015807 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97dc0fd470 con 0x7f97dc102010 2024-03-15T12:56:43.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.984+0000 7f97d2ffd640 1 -- 172.21.15.2:0/1006015807 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f97cc018a80 con 0x7f97dc102010 2024-03-15T12:56:43.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.985+0000 7f97d2ffd640 1 --2- 172.21.15.2:0/1006015807 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f97c80a1b40 0x7f97c80a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:43.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.985+0000 7f97e10e2640 1 --2- 172.21.15.2:0/1006015807 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f97c80a1b40 0x7f97c80a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:43.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.985+0000 7f97d2ffd640 1 -- 172.21.15.2:0/1006015807 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f97cc014070 con 0x7f97dc102010 2024-03-15T12:56:43.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.986+0000 7f97e10e2640 1 --2- 172.21.15.2:0/1006015807 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f97c80a1b40 0x7f97c80a4000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f97dc198a50 tx=0x7f97d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:43.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:43.989+0000 7f97d2ffd640 1 -- 172.21.15.2:0/1006015807 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f97cc096e80 con 0x7f97dc102010 2024-03-15T12:56:44.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.174+0000 7f97e3b47640 1 -- 172.21.15.2:0/1006015807 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97dc02d070 con 0x7f97c80a1b40 2024-03-15T12:56:44.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.184+0000 7f97d2ffd640 1 -- 172.21.15.2:0/1006015807 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340194 (secure 0 0 0) 0x7f97dc02d070 con 0x7f97c80a1b40 2024-03-15T12:56:44.175 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:44.176 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-03-15T12:56:44.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.188+0000 7f97e3b47640 1 -- 172.21.15.2:0/1006015807 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f97c80a1b40 msgr2=0x7f97c80a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:44.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.188+0000 7f97e3b47640 1 --2- 172.21.15.2:0/1006015807 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f97c80a1b40 0x7f97c80a4000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f97dc198a50 tx=0x7f97d8005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:44.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.188+0000 7f97e3b47640 1 -- 172.21.15.2:0/1006015807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97dc102010 msgr2=0x7f97dc197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:44.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.188+0000 7f97e3b47640 1 --2- 172.21.15.2:0/1006015807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97dc102010 0x7f97dc197540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00e9f0 tx=0x7f97cc00eeb0 comp rx=0 tx=0).stop 2024-03-15T12:56:44.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.188+0000 7f97e3b47640 1 -- 172.21.15.2:0/1006015807 shutdown_connections 2024-03-15T12:56:44.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.188+0000 7f97e3b47640 1 --2- 172.21.15.2:0/1006015807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97dc107270 0x7f97dc19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:44.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.188+0000 7f97e3b47640 1 --2- 172.21.15.2:0/1006015807 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f97c80a1b40 0x7f97c80a4000 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:44.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.188+0000 7f97e3b47640 1 --2- 172.21.15.2:0/1006015807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97dc104940 0x7f97dc197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:44.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.188+0000 7f97e3b47640 1 --2- 172.21.15.2:0/1006015807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97dc102010 0x7f97dc197540 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:44.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.188+0000 7f97e3b47640 1 -- 172.21.15.2:0/1006015807 >> 172.21.15.2:0/1006015807 conn(0x7f97dc0fb980 msgr2=0x7f97dc102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:44.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.188+0000 7f97e3b47640 1 -- 172.21.15.2:0/1006015807 shutdown_connections 2024-03-15T12:56:44.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.188+0000 7f97e3b47640 1 -- 172.21.15.2:0/1006015807 wait complete. 2024-03-15T12:56:44.184 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-03-15T12:56:42.744908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595098,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":243,"num_write_kb":2958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":189,"ondisk_log_size":189,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400012,"kb_used_data":13356,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124541812,"statfs":{"total":1151940427776,"available":1151530815488,"internally_reserved":0,"allocated":13676544,"data_stored":7506972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2320,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003145"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580936+0000","last_change":"2024-03-15T12:56:21.210343+0000","last_active":"2024-03-15T12:56:27.580936+0000","last_peered":"2024-03-15T12:56:27.580936+0000","last_clean":"2024-03-15T12:56:27.580936+0000","last_became_active":"2024-03-15T12:56:21.209787+0000","last_became_peered":"2024-03-15T12:56:21.209787+0000","last_unstale":"2024-03-15T12:56:27.580936+0000","last_undegraded":"2024-03-15T12:56:27.580936+0000","last_fullsized":"2024-03-15T12:56:27.580936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:32:21.204944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351944+0000","last_change":"2024-03-15T12:56:21.214153+0000","last_active":"2024-03-15T12:56:27.351944+0000","last_peered":"2024-03-15T12:56:27.351944+0000","last_clean":"2024-03-15T12:56:27.351944+0000","last_became_active":"2024-03-15T12:56:21.213819+0000","last_became_peered":"2024-03-15T12:56:21.213819+0000","last_unstale":"2024-03-15T12:56:27.351944+0000","last_undegraded":"2024-03-15T12:56:27.351944+0000","last_fullsized":"2024-03-15T12:56:27.351944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:31:43.264698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351630+0000","last_change":"2024-03-15T12:56:21.221678+0000","last_active":"2024-03-15T12:56:27.351630+0000","last_peered":"2024-03-15T12:56:27.351630+0000","last_clean":"2024-03-15T12:56:27.351630+0000","last_became_active":"2024-03-15T12:56:21.221272+0000","last_became_peered":"2024-03-15T12:56:21.221272+0000","last_unstale":"2024-03-15T12:56:27.351630+0000","last_undegraded":"2024-03-15T12:56:27.351630+0000","last_fullsized":"2024-03-15T12:56:27.351630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:01:50.778707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351395+0000","last_change":"2024-03-15T12:56:21.212939+0000","last_active":"2024-03-15T12:56:27.351395+0000","last_peered":"2024-03-15T12:56:27.351395+0000","last_clean":"2024-03-15T12:56:27.351395+0000","last_became_active":"2024-03-15T12:56:21.212437+0000","last_became_peered":"2024-03-15T12:56:21.212437+0000","last_unstale":"2024-03-15T12:56:27.351395+0000","last_undegraded":"2024-03-15T12:56:27.351395+0000","last_fullsized":"2024-03-15T12:56:27.351395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:34:11.377020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349659+0000","last_change":"2024-03-15T12:56:21.221433+0000","last_active":"2024-03-15T12:56:27.349659+0000","last_peered":"2024-03-15T12:56:27.349659+0000","last_clean":"2024-03-15T12:56:27.349659+0000","last_became_active":"2024-03-15T12:56:21.220605+0000","last_became_peered":"2024-03-15T12:56:21.220605+0000","last_unstale":"2024-03-15T12:56:27.349659+0000","last_undegraded":"2024-03-15T12:56:27.349659+0000","last_fullsized":"2024-03-15T12:56:27.349659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:20:25.169387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350654+0000","last_change":"2024-03-15T12:56:21.216539+0000","last_active":"2024-03-15T12:56:27.350654+0000","last_peered":"2024-03-15T12:56:27.350654+0000","last_clean":"2024-03-15T12:56:27.350654+0000","last_became_active":"2024-03-15T12:56:21.216197+0000","last_became_peered":"2024-03-15T12:56:21.216197+0000","last_unstale":"2024-03-15T12:56:27.350654+0000","last_undegraded":"2024-03-15T12:56:27.350654+0000","last_fullsized":"2024-03-15T12:56:27.350654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:00:23.186467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423278+0000","last_change":"2024-03-15T12:56:21.213103+0000","last_active":"2024-03-15T12:56:26.423278+0000","last_peered":"2024-03-15T12:56:26.423278+0000","last_clean":"2024-03-15T12:56:26.423278+0000","last_became_active":"2024-03-15T12:56:21.212789+0000","last_became_peered":"2024-03-15T12:56:21.212789+0000","last_unstale":"2024-03-15T12:56:26.423278+0000","last_undegraded":"2024-03-15T12:56:26.423278+0000","last_fullsized":"2024-03-15T12:56:26.423278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:41:50.140915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349505+0000","last_change":"2024-03-15T12:56:21.214767+0000","last_active":"2024-03-15T12:56:27.349505+0000","last_peered":"2024-03-15T12:56:27.349505+0000","last_clean":"2024-03-15T12:56:27.349505+0000","last_became_active":"2024-03-15T12:56:21.213896+0000","last_became_peered":"2024-03-15T12:56:21.213896+0000","last_unstale":"2024-03-15T12:56:27.349505+0000","last_undegraded":"2024-03-15T12:56:27.349505+0000","last_fullsized":"2024-03-15T12:56:27.349505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:33:53.819906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351682+0000","last_change":"2024-03-15T12:56:21.218566+0000","last_active":"2024-03-15T12:56:27.351682+0000","last_peered":"2024-03-15T12:56:27.351682+0000","last_clean":"2024-03-15T12:56:27.351682+0000","last_became_active":"2024-03-15T12:56:21.218234+0000","last_became_peered":"2024-03-15T12:56:21.218234+0000","last_unstale":"2024-03-15T12:56:27.351682+0000","last_undegraded":"2024-03-15T12:56:27.351682+0000","last_fullsized":"2024-03-15T12:56:27.351682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:56:46.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350790+0000","last_change":"2024-03-15T12:56:21.219367+0000","last_active":"2024-03-15T12:56:27.350790+0000","last_peered":"2024-03-15T12:56:27.350790+0000","last_clean":"2024-03-15T12:56:27.350790+0000","last_became_active":"2024-03-15T12:56:21.217541+0000","last_became_peered":"2024-03-15T12:56:21.217541+0000","last_unstale":"2024-03-15T12:56:27.350790+0000","last_undegraded":"2024-03-15T12:56:27.350790+0000","last_fullsized":"2024-03-15T12:56:27.350790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:19:12.939394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423882+0000","last_change":"2024-03-15T12:56:21.222246+0000","last_active":"2024-03-15T12:56:26.423882+0000","last_peered":"2024-03-15T12:56:26.423882+0000","last_clean":"2024-03-15T12:56:26.423882+0000","last_became_active":"2024-03-15T12:56:21.221883+0000","last_became_peered":"2024-03-15T12:56:21.221883+0000","last_unstale":"2024-03-15T12:56:26.423882+0000","last_undegraded":"2024-03-15T12:56:26.423882+0000","last_fullsized":"2024-03-15T12:56:26.423882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:42:06.892581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351340+0000","last_change":"2024-03-15T12:56:21.218463+0000","last_active":"2024-03-15T12:56:27.351340+0000","last_peered":"2024-03-15T12:56:27.351340+0000","last_clean":"2024-03-15T12:56:27.351340+0000","last_became_active":"2024-03-15T12:56:21.217934+0000","last_became_peered":"2024-03-15T12:56:21.217934+0000","last_unstale":"2024-03-15T12:56:27.351340+0000","last_undegraded":"2024-03-15T12:56:27.351340+0000","last_fullsized":"2024-03-15T12:56:27.351340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:37:29.355655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350069+0000","last_change":"2024-03-15T12:56:21.199631+0000","last_active":"2024-03-15T12:56:27.350069+0000","last_peered":"2024-03-15T12:56:27.350069+0000","last_clean":"2024-03-15T12:56:27.350069+0000","last_became_active":"2024-03-15T12:56:21.199230+0000","last_became_peered":"2024-03-15T12:56:21.199230+0000","last_unstale":"2024-03-15T12:56:27.350069+0000","last_undegraded":"2024-03-15T12:56:27.350069+0000","last_fullsized":"2024-03-15T12:56:27.350069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:45:51.356803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350582+0000","last_change":"2024-03-15T12:56:21.211449+0000","last_active":"2024-03-15T12:56:27.350582+0000","last_peered":"2024-03-15T12:56:27.350582+0000","last_clean":"2024-03-15T12:56:27.350582+0000","last_became_active":"2024-03-15T12:56:21.211001+0000","last_became_peered":"2024-03-15T12:56:21.211001+0000","last_unstale":"2024-03-15T12:56:27.350582+0000","last_undegraded":"2024-03-15T12:56:27.350582+0000","last_fullsized":"2024-03-15T12:56:27.350582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:32:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351256+0000","last_change":"2024-03-15T12:56:21.212001+0000","last_active":"2024-03-15T12:56:27.351256+0000","last_peered":"2024-03-15T12:56:27.351256+0000","last_clean":"2024-03-15T12:56:27.351256+0000","last_became_active":"2024-03-15T12:56:21.211497+0000","last_became_peered":"2024-03-15T12:56:21.211497+0000","last_unstale":"2024-03-15T12:56:27.351256+0000","last_undegraded":"2024-03-15T12:56:27.351256+0000","last_fullsized":"2024-03-15T12:56:27.351256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:27:16.029202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349369+0000","last_change":"2024-03-15T12:56:21.222368+0000","last_active":"2024-03-15T12:56:27.349369+0000","last_peered":"2024-03-15T12:56:27.349369+0000","last_clean":"2024-03-15T12:56:27.349369+0000","last_became_active":"2024-03-15T12:56:21.221356+0000","last_became_peered":"2024-03-15T12:56:21.221356+0000","last_unstale":"2024-03-15T12:56:27.349369+0000","last_undegraded":"2024-03-15T12:56:27.349369+0000","last_fullsized":"2024-03-15T12:56:27.349369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:15:04.125220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580757+0000","last_change":"2024-03-15T12:56:21.211323+0000","last_active":"2024-03-15T12:56:27.580757+0000","last_peered":"2024-03-15T12:56:27.580757+0000","last_clean":"2024-03-15T12:56:27.580757+0000","last_became_active":"2024-03-15T12:56:21.210983+0000","last_became_peered":"2024-03-15T12:56:21.210983+0000","last_unstale":"2024-03-15T12:56:27.580757+0000","last_undegraded":"2024-03-15T12:56:27.580757+0000","last_fullsized":"2024-03-15T12:56:27.580757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:10:51.487985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423526+0000","last_change":"2024-03-15T12:56:21.209435+0000","last_active":"2024-03-15T12:56:26.423526+0000","last_peered":"2024-03-15T12:56:26.423526+0000","last_clean":"2024-03-15T12:56:26.423526+0000","last_became_active":"2024-03-15T12:56:21.208981+0000","last_became_peered":"2024-03-15T12:56:21.208981+0000","last_unstale":"2024-03-15T12:56:26.423526+0000","last_undegraded":"2024-03-15T12:56:26.423526+0000","last_fullsized":"2024-03-15T12:56:26.423526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:47:59.142040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423775+0000","last_change":"2024-03-15T12:56:21.222014+0000","last_active":"2024-03-15T12:56:26.423775+0000","last_peered":"2024-03-15T12:56:26.423775+0000","last_clean":"2024-03-15T12:56:26.423775+0000","last_became_active":"2024-03-15T12:56:21.221548+0000","last_became_peered":"2024-03-15T12:56:21.221548+0000","last_unstale":"2024-03-15T12:56:26.423775+0000","last_undegraded":"2024-03-15T12:56:26.423775+0000","last_fullsized":"2024-03-15T12:56:26.423775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:12:48.943874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423518+0000","last_change":"2024-03-15T12:56:25.447200+0000","last_active":"2024-03-15T12:56:26.423518+0000","last_peered":"2024-03-15T12:56:26.423518+0000","last_clean":"2024-03-15T12:56:26.423518+0000","last_became_active":"2024-03-15T12:56:25.446516+0000","last_became_peered":"2024-03-15T12:56:25.446516+0000","last_unstale":"2024-03-15T12:56:26.423518+0000","last_undegraded":"2024-03-15T12:56:26.423518+0000","last_fullsized":"2024-03-15T12:56:26.423518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:06:46.221772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423639+0000","last_change":"2024-03-15T12:56:20.210826+0000","last_active":"2024-03-15T12:56:26.423639+0000","last_peered":"2024-03-15T12:56:26.423639+0000","last_clean":"2024-03-15T12:56:26.423639+0000","last_became_active":"2024-03-15T12:56:20.210472+0000","last_became_peered":"2024-03-15T12:56:20.210472+0000","last_unstale":"2024-03-15T12:56:26.423639+0000","last_undegraded":"2024-03-15T12:56:26.423639+0000","last_fullsized":"2024-03-15T12:56:26.423639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:33:22.896058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349871+0000","last_change":"2024-03-15T12:56:21.224665+0000","last_active":"2024-03-15T12:56:27.349871+0000","last_peered":"2024-03-15T12:56:27.349871+0000","last_clean":"2024-03-15T12:56:27.349871+0000","last_became_active":"2024-03-15T12:56:21.223826+0000","last_became_peered":"2024-03-15T12:56:21.223826+0000","last_unstale":"2024-03-15T12:56:27.349871+0000","last_undegraded":"2024-03-15T12:56:27.349871+0000","last_fullsized":"2024-03-15T12:56:27.349871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:11:57.576701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350133+0000","last_change":"2024-03-15T12:56:25.444665+0000","last_active":"2024-03-15T12:56:27.350133+0000","last_peered":"2024-03-15T12:56:27.350133+0000","last_clean":"2024-03-15T12:56:27.350133+0000","last_became_active":"2024-03-15T12:56:25.444275+0000","last_became_peered":"2024-03-15T12:56:25.444275+0000","last_unstale":"2024-03-15T12:56:27.350133+0000","last_undegraded":"2024-03-15T12:56:27.350133+0000","last_fullsized":"2024-03-15T12:56:27.350133+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:29:46.470405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351137+0000","last_change":"2024-03-15T12:56:20.215873+0000","last_active":"2024-03-15T12:56:27.351137+0000","last_peered":"2024-03-15T12:56:27.351137+0000","last_clean":"2024-03-15T12:56:27.351137+0000","last_became_active":"2024-03-15T12:56:20.215505+0000","last_became_peered":"2024-03-15T12:56:20.215505+0000","last_unstale":"2024-03-15T12:56:27.351137+0000","last_undegraded":"2024-03-15T12:56:27.351137+0000","last_fullsized":"2024-03-15T12:56:27.351137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:01:57.317382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351431+0000","last_change":"2024-03-15T12:56:21.218200+0000","last_active":"2024-03-15T12:56:27.351431+0000","last_peered":"2024-03-15T12:56:27.351431+0000","last_clean":"2024-03-15T12:56:27.351431+0000","last_became_active":"2024-03-15T12:56:21.217768+0000","last_became_peered":"2024-03-15T12:56:21.217768+0000","last_unstale":"2024-03-15T12:56:27.351431+0000","last_undegraded":"2024-03-15T12:56:27.351431+0000","last_fullsized":"2024-03-15T12:56:27.351431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:39:42.443542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350867+0000","last_change":"2024-03-15T12:56:25.444220+0000","last_active":"2024-03-15T12:56:27.350867+0000","last_peered":"2024-03-15T12:56:27.350867+0000","last_clean":"2024-03-15T12:56:27.350867+0000","last_became_active":"2024-03-15T12:56:25.443361+0000","last_became_peered":"2024-03-15T12:56:25.443361+0000","last_unstale":"2024-03-15T12:56:27.350867+0000","last_undegraded":"2024-03-15T12:56:27.350867+0000","last_fullsized":"2024-03-15T12:56:27.350867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:21:11.593526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350140+0000","last_change":"2024-03-15T12:56:20.218241+0000","last_active":"2024-03-15T12:56:27.350140+0000","last_peered":"2024-03-15T12:56:27.350140+0000","last_clean":"2024-03-15T12:56:27.350140+0000","last_became_active":"2024-03-15T12:56:20.217339+0000","last_became_peered":"2024-03-15T12:56:20.217339+0000","last_unstale":"2024-03-15T12:56:27.350140+0000","last_undegraded":"2024-03-15T12:56:27.350140+0000","last_fullsized":"2024-03-15T12:56:27.350140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:01:22.156502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351392+0000","last_change":"2024-03-15T12:56:21.217224+0000","last_active":"2024-03-15T12:56:27.351392+0000","last_peered":"2024-03-15T12:56:27.351392+0000","last_clean":"2024-03-15T12:56:27.351392+0000","last_became_active":"2024-03-15T12:56:21.216868+0000","last_became_peered":"2024-03-15T12:56:21.216868+0000","last_unstale":"2024-03-15T12:56:27.351392+0000","last_undegraded":"2024-03-15T12:56:27.351392+0000","last_fullsized":"2024-03-15T12:56:27.351392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:03:32.668463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350199+0000","last_change":"2024-03-15T12:56:25.443861+0000","last_active":"2024-03-15T12:56:27.350199+0000","last_peered":"2024-03-15T12:56:27.350199+0000","last_clean":"2024-03-15T12:56:27.350199+0000","last_became_active":"2024-03-15T12:56:25.443386+0000","last_became_peered":"2024-03-15T12:56:25.443386+0000","last_unstale":"2024-03-15T12:56:27.350199+0000","last_undegraded":"2024-03-15T12:56:27.350199+0000","last_fullsized":"2024-03-15T12:56:27.350199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:00:23.063425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349846+0000","last_change":"2024-03-15T12:56:20.212451+0000","last_active":"2024-03-15T12:56:27.349846+0000","last_peered":"2024-03-15T12:56:27.349846+0000","last_clean":"2024-03-15T12:56:27.349846+0000","last_became_active":"2024-03-15T12:56:20.212118+0000","last_became_peered":"2024-03-15T12:56:20.212118+0000","last_unstale":"2024-03-15T12:56:27.349846+0000","last_undegraded":"2024-03-15T12:56:27.349846+0000","last_fullsized":"2024-03-15T12:56:27.349846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:24:33.667652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.849201+0000","last_change":"2024-03-15T12:56:21.216499+0000","last_active":"2024-03-15T12:56:26.849201+0000","last_peered":"2024-03-15T12:56:26.849201+0000","last_clean":"2024-03-15T12:56:26.849201+0000","last_became_active":"2024-03-15T12:56:21.216170+0000","last_became_peered":"2024-03-15T12:56:21.216170+0000","last_unstale":"2024-03-15T12:56:26.849201+0000","last_undegraded":"2024-03-15T12:56:26.849201+0000","last_fullsized":"2024-03-15T12:56:26.849201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:20:26.675451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423625+0000","last_change":"2024-03-15T12:56:25.438194+0000","last_active":"2024-03-15T12:56:26.423625+0000","last_peered":"2024-03-15T12:56:26.423625+0000","last_clean":"2024-03-15T12:56:26.423625+0000","last_became_active":"2024-03-15T12:56:25.437769+0000","last_became_peered":"2024-03-15T12:56:25.437769+0000","last_unstale":"2024-03-15T12:56:26.423625+0000","last_undegraded":"2024-03-15T12:56:26.423625+0000","last_fullsized":"2024-03-15T12:56:26.423625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:29:44.398731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848873+0000","last_change":"2024-03-15T12:56:20.210552+0000","last_active":"2024-03-15T12:56:26.848873+0000","last_peered":"2024-03-15T12:56:26.848873+0000","last_clean":"2024-03-15T12:56:26.848873+0000","last_became_active":"2024-03-15T12:56:20.210191+0000","last_became_peered":"2024-03-15T12:56:20.210191+0000","last_unstale":"2024-03-15T12:56:26.848873+0000","last_undegraded":"2024-03-15T12:56:26.848873+0000","last_fullsized":"2024-03-15T12:56:26.848873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:25:48.225266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349697+0000","last_change":"2024-03-15T12:56:21.220582+0000","last_active":"2024-03-15T12:56:27.349697+0000","last_peered":"2024-03-15T12:56:27.349697+0000","last_clean":"2024-03-15T12:56:27.349697+0000","last_became_active":"2024-03-15T12:56:21.220212+0000","last_became_peered":"2024-03-15T12:56:21.220212+0000","last_unstale":"2024-03-15T12:56:27.349697+0000","last_undegraded":"2024-03-15T12:56:27.349697+0000","last_fullsized":"2024-03-15T12:56:27.349697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:00:54.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423603+0000","last_change":"2024-03-15T12:56:25.446410+0000","last_active":"2024-03-15T12:56:26.423603+0000","last_peered":"2024-03-15T12:56:26.423603+0000","last_clean":"2024-03-15T12:56:26.423603+0000","last_became_active":"2024-03-15T12:56:25.445554+0000","last_became_peered":"2024-03-15T12:56:25.445554+0000","last_unstale":"2024-03-15T12:56:26.423603+0000","last_undegraded":"2024-03-15T12:56:26.423603+0000","last_fullsized":"2024-03-15T12:56:26.423603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:35:08.049374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350291+0000","last_change":"2024-03-15T12:56:20.212599+0000","last_active":"2024-03-15T12:56:27.350291+0000","last_peered":"2024-03-15T12:56:27.350291+0000","last_clean":"2024-03-15T12:56:27.350291+0000","last_became_active":"2024-03-15T12:56:20.212289+0000","last_became_peered":"2024-03-15T12:56:20.212289+0000","last_unstale":"2024-03-15T12:56:27.350291+0000","last_undegraded":"2024-03-15T12:56:27.350291+0000","last_fullsized":"2024-03-15T12:56:27.350291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:12:51.036882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.849122+0000","last_change":"2024-03-15T12:56:21.219407+0000","last_active":"2024-03-15T12:56:26.849122+0000","last_peered":"2024-03-15T12:56:26.849122+0000","last_clean":"2024-03-15T12:56:26.849122+0000","last_became_active":"2024-03-15T12:56:21.219035+0000","last_became_peered":"2024-03-15T12:56:21.219035+0000","last_unstale":"2024-03-15T12:56:26.849122+0000","last_undegraded":"2024-03-15T12:56:26.849122+0000","last_fullsized":"2024-03-15T12:56:26.849122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:35:43.129767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580755+0000","last_change":"2024-03-15T12:56:25.434128+0000","last_active":"2024-03-15T12:56:27.580755+0000","last_peered":"2024-03-15T12:56:27.580755+0000","last_clean":"2024-03-15T12:56:27.580755+0000","last_became_active":"2024-03-15T12:56:25.433661+0000","last_became_peered":"2024-03-15T12:56:25.433661+0000","last_unstale":"2024-03-15T12:56:27.580755+0000","last_undegraded":"2024-03-15T12:56:27.580755+0000","last_fullsized":"2024-03-15T12:56:27.580755+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:32:05.771698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349605+0000","last_change":"2024-03-15T12:56:20.209933+0000","last_active":"2024-03-15T12:56:27.349605+0000","last_peered":"2024-03-15T12:56:27.349605+0000","last_clean":"2024-03-15T12:56:27.349605+0000","last_became_active":"2024-03-15T12:56:20.209047+0000","last_became_peered":"2024-03-15T12:56:20.209047+0000","last_unstale":"2024-03-15T12:56:27.349605+0000","last_undegraded":"2024-03-15T12:56:27.349605+0000","last_fullsized":"2024-03-15T12:56:27.349605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:24:05.551745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349310+0000","last_change":"2024-03-15T12:56:21.215198+0000","last_active":"2024-03-15T12:56:27.349310+0000","last_peered":"2024-03-15T12:56:27.349310+0000","last_clean":"2024-03-15T12:56:27.349310+0000","last_became_active":"2024-03-15T12:56:21.214844+0000","last_became_peered":"2024-03-15T12:56:21.214844+0000","last_unstale":"2024-03-15T12:56:27.349310+0000","last_undegraded":"2024-03-15T12:56:27.349310+0000","last_fullsized":"2024-03-15T12:56:27.349310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:37:08.398317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351861+0000","last_change":"2024-03-15T12:56:25.437047+0000","last_active":"2024-03-15T12:56:27.351861+0000","last_peered":"2024-03-15T12:56:27.351861+0000","last_clean":"2024-03-15T12:56:27.351861+0000","last_became_active":"2024-03-15T12:56:25.436595+0000","last_became_peered":"2024-03-15T12:56:25.436595+0000","last_unstale":"2024-03-15T12:56:27.351861+0000","last_undegraded":"2024-03-15T12:56:27.351861+0000","last_fullsized":"2024-03-15T12:56:27.351861+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:59:55.147044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848610+0000","last_change":"2024-03-15T12:56:20.206348+0000","last_active":"2024-03-15T12:56:26.848610+0000","last_peered":"2024-03-15T12:56:26.848610+0000","last_clean":"2024-03-15T12:56:26.848610+0000","last_became_active":"2024-03-15T12:56:20.206013+0000","last_became_peered":"2024-03-15T12:56:20.206013+0000","last_unstale":"2024-03-15T12:56:26.848610+0000","last_undegraded":"2024-03-15T12:56:26.848610+0000","last_fullsized":"2024-03-15T12:56:26.848610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:32:26.625982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.422867+0000","last_change":"2024-03-15T12:56:21.216517+0000","last_active":"2024-03-15T12:56:26.422867+0000","last_peered":"2024-03-15T12:56:26.422867+0000","last_clean":"2024-03-15T12:56:26.422867+0000","last_became_active":"2024-03-15T12:56:21.216076+0000","last_became_peered":"2024-03-15T12:56:21.216076+0000","last_unstale":"2024-03-15T12:56:26.422867+0000","last_undegraded":"2024-03-15T12:56:26.422867+0000","last_fullsized":"2024-03-15T12:56:26.422867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:19:35.547273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349813+0000","last_change":"2024-03-15T12:56:25.437367+0000","last_active":"2024-03-15T12:56:27.349813+0000","last_peered":"2024-03-15T12:56:27.349813+0000","last_clean":"2024-03-15T12:56:27.349813+0000","last_became_active":"2024-03-15T12:56:25.436930+0000","last_became_peered":"2024-03-15T12:56:25.436930+0000","last_unstale":"2024-03-15T12:56:27.349813+0000","last_undegraded":"2024-03-15T12:56:27.349813+0000","last_fullsized":"2024-03-15T12:56:27.349813+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:53:56.586786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:39.833875+0000","last_change":"2024-03-15T12:56:20.214067+0000","last_active":"2024-03-15T12:56:39.833875+0000","last_peered":"2024-03-15T12:56:39.833875+0000","last_clean":"2024-03-15T12:56:39.833875+0000","last_became_active":"2024-03-15T12:56:20.213696+0000","last_became_peered":"2024-03-15T12:56:20.213696+0000","last_unstale":"2024-03-15T12:56:39.833875+0000","last_undegraded":"2024-03-15T12:56:39.833875+0000","last_fullsized":"2024-03-15T12:56:39.833875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:35:15.343660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351660+0000","last_change":"2024-03-15T12:56:21.211106+0000","last_active":"2024-03-15T12:56:27.351660+0000","last_peered":"2024-03-15T12:56:27.351660+0000","last_clean":"2024-03-15T12:56:27.351660+0000","last_became_active":"2024-03-15T12:56:21.210756+0000","last_became_peered":"2024-03-15T12:56:21.210756+0000","last_unstale":"2024-03-15T12:56:27.351660+0000","last_undegraded":"2024-03-15T12:56:27.351660+0000","last_fullsized":"2024-03-15T12:56:27.351660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:13:25.232077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.422979+0000","last_change":"2024-03-15T12:56:25.441364+0000","last_active":"2024-03-15T12:56:26.422979+0000","last_peered":"2024-03-15T12:56:26.422979+0000","last_clean":"2024-03-15T12:56:26.422979+0000","last_became_active":"2024-03-15T12:56:25.440951+0000","last_became_peered":"2024-03-15T12:56:25.440951+0000","last_unstale":"2024-03-15T12:56:26.422979+0000","last_undegraded":"2024-03-15T12:56:26.422979+0000","last_fullsized":"2024-03-15T12:56:26.422979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:32:28.651856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423490+0000","last_change":"2024-03-15T12:56:20.208633+0000","last_active":"2024-03-15T12:56:26.423490+0000","last_peered":"2024-03-15T12:56:26.423490+0000","last_clean":"2024-03-15T12:56:26.423490+0000","last_became_active":"2024-03-15T12:56:20.208280+0000","last_became_peered":"2024-03-15T12:56:20.208280+0000","last_unstale":"2024-03-15T12:56:26.423490+0000","last_undegraded":"2024-03-15T12:56:26.423490+0000","last_fullsized":"2024-03-15T12:56:26.423490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:00:15.274090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351532+0000","last_change":"2024-03-15T12:56:21.218998+0000","last_active":"2024-03-15T12:56:27.351532+0000","last_peered":"2024-03-15T12:56:27.351532+0000","last_clean":"2024-03-15T12:56:27.351532+0000","last_became_active":"2024-03-15T12:56:21.218617+0000","last_became_peered":"2024-03-15T12:56:21.218617+0000","last_unstale":"2024-03-15T12:56:27.351532+0000","last_undegraded":"2024-03-15T12:56:27.351532+0000","last_fullsized":"2024-03-15T12:56:27.351532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:20:25.496204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351115+0000","last_change":"2024-03-15T12:56:25.438969+0000","last_active":"2024-03-15T12:56:27.351115+0000","last_peered":"2024-03-15T12:56:27.351115+0000","last_clean":"2024-03-15T12:56:27.351115+0000","last_became_active":"2024-03-15T12:56:25.438603+0000","last_became_peered":"2024-03-15T12:56:25.438603+0000","last_unstale":"2024-03-15T12:56:27.351115+0000","last_undegraded":"2024-03-15T12:56:27.351115+0000","last_fullsized":"2024-03-15T12:56:27.351115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:54:23.992253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.849081+0000","last_change":"2024-03-15T12:56:20.215152+0000","last_active":"2024-03-15T12:56:26.849081+0000","last_peered":"2024-03-15T12:56:26.849081+0000","last_clean":"2024-03-15T12:56:26.849081+0000","last_became_active":"2024-03-15T12:56:20.214762+0000","last_became_peered":"2024-03-15T12:56:20.214762+0000","last_unstale":"2024-03-15T12:56:26.849081+0000","last_undegraded":"2024-03-15T12:56:26.849081+0000","last_fullsized":"2024-03-15T12:56:26.849081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:38:45.414984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351465+0000","last_change":"2024-03-15T12:56:21.214005+0000","last_active":"2024-03-15T12:56:27.351465+0000","last_peered":"2024-03-15T12:56:27.351465+0000","last_clean":"2024-03-15T12:56:27.351465+0000","last_became_active":"2024-03-15T12:56:21.213672+0000","last_became_peered":"2024-03-15T12:56:21.213672+0000","last_unstale":"2024-03-15T12:56:27.351465+0000","last_undegraded":"2024-03-15T12:56:27.351465+0000","last_fullsized":"2024-03-15T12:56:27.351465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:13:09.384245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423234+0000","last_change":"2024-03-15T12:56:25.443145+0000","last_active":"2024-03-15T12:56:26.423234+0000","last_peered":"2024-03-15T12:56:26.423234+0000","last_clean":"2024-03-15T12:56:26.423234+0000","last_became_active":"2024-03-15T12:56:25.442755+0000","last_became_peered":"2024-03-15T12:56:25.442755+0000","last_unstale":"2024-03-15T12:56:26.423234+0000","last_undegraded":"2024-03-15T12:56:26.423234+0000","last_fullsized":"2024-03-15T12:56:26.423234+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:14:47.201919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:39.834429+0000","last_change":"2024-03-15T12:56:20.215063+0000","last_active":"2024-03-15T12:56:39.834429+0000","last_peered":"2024-03-15T12:56:39.834429+0000","last_clean":"2024-03-15T12:56:39.834429+0000","last_became_active":"2024-03-15T12:56:20.214742+0000","last_became_peered":"2024-03-15T12:56:20.214742+0000","last_unstale":"2024-03-15T12:56:39.834429+0000","last_undegraded":"2024-03-15T12:56:39.834429+0000","last_fullsized":"2024-03-15T12:56:39.834429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:38:12.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349955+0000","last_change":"2024-03-15T12:56:21.213519+0000","last_active":"2024-03-15T12:56:27.349955+0000","last_peered":"2024-03-15T12:56:27.349955+0000","last_clean":"2024-03-15T12:56:27.349955+0000","last_became_active":"2024-03-15T12:56:21.213198+0000","last_became_peered":"2024-03-15T12:56:21.213198+0000","last_unstale":"2024-03-15T12:56:27.349955+0000","last_undegraded":"2024-03-15T12:56:27.349955+0000","last_fullsized":"2024-03-15T12:56:27.349955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:22:13.037068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.849551+0000","last_change":"2024-03-15T12:56:25.446144+0000","last_active":"2024-03-15T12:56:26.849551+0000","last_peered":"2024-03-15T12:56:26.849551+0000","last_clean":"2024-03-15T12:56:26.849551+0000","last_became_active":"2024-03-15T12:56:25.445342+0000","last_became_peered":"2024-03-15T12:56:25.445342+0000","last_unstale":"2024-03-15T12:56:26.849551+0000","last_undegraded":"2024-03-15T12:56:26.849551+0000","last_fullsized":"2024-03-15T12:56:26.849551+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:57:28.280731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:39.834162+0000","last_change":"2024-03-15T12:56:20.216124+0000","last_active":"2024-03-15T12:56:39.834162+0000","last_peered":"2024-03-15T12:56:39.834162+0000","last_clean":"2024-03-15T12:56:39.834162+0000","last_became_active":"2024-03-15T12:56:20.215754+0000","last_became_peered":"2024-03-15T12:56:20.215754+0000","last_unstale":"2024-03-15T12:56:39.834162+0000","last_undegraded":"2024-03-15T12:56:39.834162+0000","last_fullsized":"2024-03-15T12:56:39.834162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:11:44.403503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349970+0000","last_change":"2024-03-15T12:56:21.214775+0000","last_active":"2024-03-15T12:56:27.349970+0000","last_peered":"2024-03-15T12:56:27.349970+0000","last_clean":"2024-03-15T12:56:27.349970+0000","last_became_active":"2024-03-15T12:56:21.214445+0000","last_became_peered":"2024-03-15T12:56:21.214445+0000","last_unstale":"2024-03-15T12:56:27.349970+0000","last_undegraded":"2024-03-15T12:56:27.349970+0000","last_fullsized":"2024-03-15T12:56:27.349970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:23:24.045670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580734+0000","last_change":"2024-03-15T12:56:25.438001+0000","last_active":"2024-03-15T12:56:27.580734+0000","last_peered":"2024-03-15T12:56:27.580734+0000","last_clean":"2024-03-15T12:56:27.580734+0000","last_became_active":"2024-03-15T12:56:25.437368+0000","last_became_peered":"2024-03-15T12:56:25.437368+0000","last_unstale":"2024-03-15T12:56:27.580734+0000","last_undegraded":"2024-03-15T12:56:27.580734+0000","last_fullsized":"2024-03-15T12:56:27.580734+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:39:27.648437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848969+0000","last_change":"2024-03-15T12:56:20.217191+0000","last_active":"2024-03-15T12:56:26.848969+0000","last_peered":"2024-03-15T12:56:26.848969+0000","last_clean":"2024-03-15T12:56:26.848969+0000","last_became_active":"2024-03-15T12:56:20.216632+0000","last_became_peered":"2024-03-15T12:56:20.216632+0000","last_unstale":"2024-03-15T12:56:26.848969+0000","last_undegraded":"2024-03-15T12:56:26.848969+0000","last_fullsized":"2024-03-15T12:56:26.848969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:16:51.187920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423724+0000","last_change":"2024-03-15T12:56:21.218650+0000","last_active":"2024-03-15T12:56:26.423724+0000","last_peered":"2024-03-15T12:56:26.423724+0000","last_clean":"2024-03-15T12:56:26.423724+0000","last_became_active":"2024-03-15T12:56:21.218329+0000","last_became_peered":"2024-03-15T12:56:21.218329+0000","last_unstale":"2024-03-15T12:56:26.423724+0000","last_undegraded":"2024-03-15T12:56:26.423724+0000","last_fullsized":"2024-03-15T12:56:26.423724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:10:45.632248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423518+0000","last_change":"2024-03-15T12:56:25.447982+0000","last_active":"2024-03-15T12:56:26.423518+0000","last_peered":"2024-03-15T12:56:26.423518+0000","last_clean":"2024-03-15T12:56:26.423518+0000","last_became_active":"2024-03-15T12:56:25.447514+0000","last_became_peered":"2024-03-15T12:56:25.447514+0000","last_unstale":"2024-03-15T12:56:26.423518+0000","last_undegraded":"2024-03-15T12:56:26.423518+0000","last_fullsized":"2024-03-15T12:56:26.423518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:16:52.567619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349976+0000","last_change":"2024-03-15T12:56:20.209778+0000","last_active":"2024-03-15T12:56:27.349976+0000","last_peered":"2024-03-15T12:56:27.349976+0000","last_clean":"2024-03-15T12:56:27.349976+0000","last_became_active":"2024-03-15T12:56:20.209311+0000","last_became_peered":"2024-03-15T12:56:20.209311+0000","last_unstale":"2024-03-15T12:56:27.349976+0000","last_undegraded":"2024-03-15T12:56:27.349976+0000","last_fullsized":"2024-03-15T12:56:27.349976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:05:21.930210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423470+0000","last_change":"2024-03-15T12:56:21.215576+0000","last_active":"2024-03-15T12:56:26.423470+0000","last_peered":"2024-03-15T12:56:26.423470+0000","last_clean":"2024-03-15T12:56:26.423470+0000","last_became_active":"2024-03-15T12:56:21.215261+0000","last_became_peered":"2024-03-15T12:56:21.215261+0000","last_unstale":"2024-03-15T12:56:26.423470+0000","last_undegraded":"2024-03-15T12:56:26.423470+0000","last_fullsized":"2024-03-15T12:56:26.423470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:35:15.461554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350810+0000","last_change":"2024-03-15T12:56:25.445559+0000","last_active":"2024-03-15T12:56:27.350810+0000","last_peered":"2024-03-15T12:56:27.350810+0000","last_clean":"2024-03-15T12:56:27.350810+0000","last_became_active":"2024-03-15T12:56:25.445092+0000","last_became_peered":"2024-03-15T12:56:25.445092+0000","last_unstale":"2024-03-15T12:56:27.350810+0000","last_undegraded":"2024-03-15T12:56:27.350810+0000","last_fullsized":"2024-03-15T12:56:27.350810+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:48:30.558243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349545+0000","last_change":"2024-03-15T12:56:20.214853+0000","last_active":"2024-03-15T12:56:27.349545+0000","last_peered":"2024-03-15T12:56:27.349545+0000","last_clean":"2024-03-15T12:56:27.349545+0000","last_became_active":"2024-03-15T12:56:20.214510+0000","last_became_peered":"2024-03-15T12:56:20.214510+0000","last_unstale":"2024-03-15T12:56:27.349545+0000","last_undegraded":"2024-03-15T12:56:27.349545+0000","last_fullsized":"2024-03-15T12:56:27.349545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:25:39.316428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848800+0000","last_change":"2024-03-15T12:56:21.216511+0000","last_active":"2024-03-15T12:56:26.848800+0000","last_peered":"2024-03-15T12:56:26.848800+0000","last_clean":"2024-03-15T12:56:26.848800+0000","last_became_active":"2024-03-15T12:56:21.216175+0000","last_became_peered":"2024-03-15T12:56:21.216175+0000","last_unstale":"2024-03-15T12:56:26.848800+0000","last_undegraded":"2024-03-15T12:56:26.848800+0000","last_fullsized":"2024-03-15T12:56:26.848800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:16:56.665841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848937+0000","last_change":"2024-03-15T12:56:25.446009+0000","last_active":"2024-03-15T12:56:26.848937+0000","last_peered":"2024-03-15T12:56:26.848937+0000","last_clean":"2024-03-15T12:56:26.848937+0000","last_became_active":"2024-03-15T12:56:25.445474+0000","last_became_peered":"2024-03-15T12:56:25.445474+0000","last_unstale":"2024-03-15T12:56:26.848937+0000","last_undegraded":"2024-03-15T12:56:26.848937+0000","last_fullsized":"2024-03-15T12:56:26.848937+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:53:46.856905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350147+0000","last_change":"2024-03-15T12:56:20.210146+0000","last_active":"2024-03-15T12:56:27.350147+0000","last_peered":"2024-03-15T12:56:27.350147+0000","last_clean":"2024-03-15T12:56:27.350147+0000","last_became_active":"2024-03-15T12:56:20.209769+0000","last_became_peered":"2024-03-15T12:56:20.209769+0000","last_unstale":"2024-03-15T12:56:27.350147+0000","last_undegraded":"2024-03-15T12:56:27.350147+0000","last_fullsized":"2024-03-15T12:56:27.350147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:47:32.505257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580613+0000","last_change":"2024-03-15T12:56:21.214826+0000","last_active":"2024-03-15T12:56:27.580613+0000","last_peered":"2024-03-15T12:56:27.580613+0000","last_clean":"2024-03-15T12:56:27.580613+0000","last_became_active":"2024-03-15T12:56:21.214160+0000","last_became_peered":"2024-03-15T12:56:21.214160+0000","last_unstale":"2024-03-15T12:56:27.580613+0000","last_undegraded":"2024-03-15T12:56:27.580613+0000","last_fullsized":"2024-03-15T12:56:27.580613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:23:17.800522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349871+0000","last_change":"2024-03-15T12:56:25.443496+0000","last_active":"2024-03-15T12:56:27.349871+0000","last_peered":"2024-03-15T12:56:27.349871+0000","last_clean":"2024-03-15T12:56:27.349871+0000","last_became_active":"2024-03-15T12:56:25.442796+0000","last_became_peered":"2024-03-15T12:56:25.442796+0000","last_unstale":"2024-03-15T12:56:27.349871+0000","last_undegraded":"2024-03-15T12:56:27.349871+0000","last_fullsized":"2024-03-15T12:56:27.349871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:35:23.892693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423277+0000","last_change":"2024-03-15T12:56:20.205207+0000","last_active":"2024-03-15T12:56:26.423277+0000","last_peered":"2024-03-15T12:56:26.423277+0000","last_clean":"2024-03-15T12:56:26.423277+0000","last_became_active":"2024-03-15T12:56:20.204759+0000","last_became_peered":"2024-03-15T12:56:20.204759+0000","last_unstale":"2024-03-15T12:56:26.423277+0000","last_undegraded":"2024-03-15T12:56:26.423277+0000","last_fullsized":"2024-03-15T12:56:26.423277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:47:12.575931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350541+0000","last_change":"2024-03-15T12:56:21.216393+0000","last_active":"2024-03-15T12:56:27.350541+0000","last_peered":"2024-03-15T12:56:27.350541+0000","last_clean":"2024-03-15T12:56:27.350541+0000","last_became_active":"2024-03-15T12:56:21.216043+0000","last_became_peered":"2024-03-15T12:56:21.216043+0000","last_unstale":"2024-03-15T12:56:27.350541+0000","last_undegraded":"2024-03-15T12:56:27.350541+0000","last_fullsized":"2024-03-15T12:56:27.350541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:05:49.843726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350384+0000","last_change":"2024-03-15T12:56:25.443854+0000","last_active":"2024-03-15T12:56:27.350384+0000","last_peered":"2024-03-15T12:56:27.350384+0000","last_clean":"2024-03-15T12:56:27.350384+0000","last_became_active":"2024-03-15T12:56:25.443422+0000","last_became_peered":"2024-03-15T12:56:25.443422+0000","last_unstale":"2024-03-15T12:56:27.350384+0000","last_undegraded":"2024-03-15T12:56:27.350384+0000","last_fullsized":"2024-03-15T12:56:27.350384+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:52:18.999161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423389+0000","last_change":"2024-03-15T12:56:20.211493+0000","last_active":"2024-03-15T12:56:26.423389+0000","last_peered":"2024-03-15T12:56:26.423389+0000","last_clean":"2024-03-15T12:56:26.423389+0000","last_became_active":"2024-03-15T12:56:20.211154+0000","last_became_peered":"2024-03-15T12:56:20.211154+0000","last_unstale":"2024-03-15T12:56:26.423389+0000","last_undegraded":"2024-03-15T12:56:26.423389+0000","last_fullsized":"2024-03-15T12:56:26.423389+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:26:28.026899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350374+0000","last_change":"2024-03-15T12:56:21.215999+0000","last_active":"2024-03-15T12:56:27.350374+0000","last_peered":"2024-03-15T12:56:27.350374+0000","last_clean":"2024-03-15T12:56:27.350374+0000","last_became_active":"2024-03-15T12:56:21.215657+0000","last_became_peered":"2024-03-15T12:56:21.215657+0000","last_unstale":"2024-03-15T12:56:27.350374+0000","last_undegraded":"2024-03-15T12:56:27.350374+0000","last_fullsized":"2024-03-15T12:56:27.350374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:47:59.901886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349712+0000","last_change":"2024-03-15T12:56:25.442246+0000","last_active":"2024-03-15T12:56:27.349712+0000","last_peered":"2024-03-15T12:56:27.349712+0000","last_clean":"2024-03-15T12:56:27.349712+0000","last_became_active":"2024-03-15T12:56:25.441755+0000","last_became_peered":"2024-03-15T12:56:25.441755+0000","last_unstale":"2024-03-15T12:56:27.349712+0000","last_undegraded":"2024-03-15T12:56:27.349712+0000","last_fullsized":"2024-03-15T12:56:27.349712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:47:36.037864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848845+0000","last_change":"2024-03-15T12:56:21.215746+0000","last_active":"2024-03-15T12:56:26.848845+0000","last_peered":"2024-03-15T12:56:26.848845+0000","last_clean":"2024-03-15T12:56:26.848845+0000","last_became_active":"2024-03-15T12:56:21.215424+0000","last_became_peered":"2024-03-15T12:56:21.215424+0000","last_unstale":"2024-03-15T12:56:26.848845+0000","last_undegraded":"2024-03-15T12:56:26.848845+0000","last_fullsized":"2024-03-15T12:56:26.848845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:01:15.498119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350368+0000","last_change":"2024-03-15T12:56:20.219705+0000","last_active":"2024-03-15T12:56:27.350368+0000","last_peered":"2024-03-15T12:56:27.350368+0000","last_clean":"2024-03-15T12:56:27.350368+0000","last_became_active":"2024-03-15T12:56:20.219328+0000","last_became_peered":"2024-03-15T12:56:20.219328+0000","last_unstale":"2024-03-15T12:56:27.350368+0000","last_undegraded":"2024-03-15T12:56:27.350368+0000","last_fullsized":"2024-03-15T12:56:27.350368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:47:52.171116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423843+0000","last_change":"2024-03-15T12:56:25.443094+0000","last_active":"2024-03-15T12:56:26.423843+0000","last_peered":"2024-03-15T12:56:26.423843+0000","last_clean":"2024-03-15T12:56:26.423843+0000","last_became_active":"2024-03-15T12:56:25.442708+0000","last_became_peered":"2024-03-15T12:56:25.442708+0000","last_unstale":"2024-03-15T12:56:26.423843+0000","last_undegraded":"2024-03-15T12:56:26.423843+0000","last_fullsized":"2024-03-15T12:56:26.423843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:03:07.769836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349746+0000","last_change":"2024-03-15T12:56:21.223937+0000","last_active":"2024-03-15T12:56:27.349746+0000","last_peered":"2024-03-15T12:56:27.349746+0000","last_clean":"2024-03-15T12:56:27.349746+0000","last_became_active":"2024-03-15T12:56:21.223583+0000","last_became_peered":"2024-03-15T12:56:21.223583+0000","last_unstale":"2024-03-15T12:56:27.349746+0000","last_undegraded":"2024-03-15T12:56:27.349746+0000","last_fullsized":"2024-03-15T12:56:27.349746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:15:34.287445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350925+0000","last_change":"2024-03-15T12:56:20.210812+0000","last_active":"2024-03-15T12:56:27.350925+0000","last_peered":"2024-03-15T12:56:27.350925+0000","last_clean":"2024-03-15T12:56:27.350925+0000","last_became_active":"2024-03-15T12:56:20.210457+0000","last_became_peered":"2024-03-15T12:56:20.210457+0000","last_unstale":"2024-03-15T12:56:27.350925+0000","last_undegraded":"2024-03-15T12:56:27.350925+0000","last_fullsized":"2024-03-15T12:56:27.350925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:04:37.371637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423245+0000","last_change":"2024-03-15T12:55:08.392548+0000","last_active":"2024-03-15T12:56:26.423245+0000","last_peered":"2024-03-15T12:56:26.423245+0000","last_clean":"2024-03-15T12:56:26.423245+0000","last_became_active":"2024-03-15T12:55:08.391972+0000","last_became_peered":"2024-03-15T12:55:08.391972+0000","last_unstale":"2024-03-15T12:56:26.423245+0000","last_undegraded":"2024-03-15T12:56:26.423245+0000","last_fullsized":"2024-03-15T12:56:26.423245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:51:47.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:51:47.093558+0000","last_clean_scrub_stamp":"2024-03-15T12:51:47.093558+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:30:57.264886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.849345+0000","last_change":"2024-03-15T12:56:25.443212+0000","last_active":"2024-03-15T12:56:26.849345+0000","last_peered":"2024-03-15T12:56:26.849345+0000","last_clean":"2024-03-15T12:56:26.849345+0000","last_became_active":"2024-03-15T12:56:25.442787+0000","last_became_peered":"2024-03-15T12:56:25.442787+0000","last_unstale":"2024-03-15T12:56:26.849345+0000","last_undegraded":"2024-03-15T12:56:26.849345+0000","last_fullsized":"2024-03-15T12:56:26.849345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:59:32.244953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349478+0000","last_change":"2024-03-15T12:56:21.212421+0000","last_active":"2024-03-15T12:56:27.349478+0000","last_peered":"2024-03-15T12:56:27.349478+0000","last_clean":"2024-03-15T12:56:27.349478+0000","last_became_active":"2024-03-15T12:56:21.212095+0000","last_became_peered":"2024-03-15T12:56:21.212095+0000","last_unstale":"2024-03-15T12:56:27.349478+0000","last_undegraded":"2024-03-15T12:56:27.349478+0000","last_fullsized":"2024-03-15T12:56:27.349478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:47:10.397511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423369+0000","last_change":"2024-03-15T12:56:20.210466+0000","last_active":"2024-03-15T12:56:26.423369+0000","last_peered":"2024-03-15T12:56:26.423369+0000","last_clean":"2024-03-15T12:56:26.423369+0000","last_became_active":"2024-03-15T12:56:20.210157+0000","last_became_peered":"2024-03-15T12:56:20.210157+0000","last_unstale":"2024-03-15T12:56:26.423369+0000","last_undegraded":"2024-03-15T12:56:26.423369+0000","last_fullsized":"2024-03-15T12:56:26.423369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:58:03.335826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580639+0000","last_change":"2024-03-15T12:56:25.432641+0000","last_active":"2024-03-15T12:56:27.580639+0000","last_peered":"2024-03-15T12:56:27.580639+0000","last_clean":"2024-03-15T12:56:27.580639+0000","last_became_active":"2024-03-15T12:56:25.432212+0000","last_became_peered":"2024-03-15T12:56:25.432212+0000","last_unstale":"2024-03-15T12:56:27.580639+0000","last_undegraded":"2024-03-15T12:56:27.580639+0000","last_fullsized":"2024-03-15T12:56:27.580639+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:40:20.447886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350069+0000","last_change":"2024-03-15T12:56:20.217693+0000","last_active":"2024-03-15T12:56:27.350069+0000","last_peered":"2024-03-15T12:56:27.350069+0000","last_clean":"2024-03-15T12:56:27.350069+0000","last_became_active":"2024-03-15T12:56:20.217335+0000","last_became_peered":"2024-03-15T12:56:20.217335+0000","last_unstale":"2024-03-15T12:56:27.350069+0000","last_undegraded":"2024-03-15T12:56:27.350069+0000","last_fullsized":"2024-03-15T12:56:27.350069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:05:23.500578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350527+0000","last_change":"2024-03-15T12:56:21.207577+0000","last_active":"2024-03-15T12:56:27.350527+0000","last_peered":"2024-03-15T12:56:27.350527+0000","last_clean":"2024-03-15T12:56:27.350527+0000","last_became_active":"2024-03-15T12:56:21.207220+0000","last_became_peered":"2024-03-15T12:56:21.207220+0000","last_unstale":"2024-03-15T12:56:27.350527+0000","last_undegraded":"2024-03-15T12:56:27.350527+0000","last_fullsized":"2024-03-15T12:56:27.350527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:42:12.104220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349448+0000","last_change":"2024-03-15T12:56:25.439762+0000","last_active":"2024-03-15T12:56:27.349448+0000","last_peered":"2024-03-15T12:56:27.349448+0000","last_clean":"2024-03-15T12:56:27.349448+0000","last_became_active":"2024-03-15T12:56:25.439366+0000","last_became_peered":"2024-03-15T12:56:25.439366+0000","last_unstale":"2024-03-15T12:56:27.349448+0000","last_undegraded":"2024-03-15T12:56:27.349448+0000","last_fullsized":"2024-03-15T12:56:27.349448+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:36:29.638449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423528+0000","last_change":"2024-03-15T12:56:20.218438+0000","last_active":"2024-03-15T12:56:26.423528+0000","last_peered":"2024-03-15T12:56:26.423528+0000","last_clean":"2024-03-15T12:56:26.423528+0000","last_became_active":"2024-03-15T12:56:20.218071+0000","last_became_peered":"2024-03-15T12:56:20.218071+0000","last_unstale":"2024-03-15T12:56:26.423528+0000","last_undegraded":"2024-03-15T12:56:26.423528+0000","last_fullsized":"2024-03-15T12:56:26.423528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:24:16.295781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351246+0000","last_change":"2024-03-15T12:56:21.218345+0000","last_active":"2024-03-15T12:56:27.351246+0000","last_peered":"2024-03-15T12:56:27.351246+0000","last_clean":"2024-03-15T12:56:27.351246+0000","last_became_active":"2024-03-15T12:56:21.216534+0000","last_became_peered":"2024-03-15T12:56:21.216534+0000","last_unstale":"2024-03-15T12:56:27.351246+0000","last_undegraded":"2024-03-15T12:56:27.351246+0000","last_fullsized":"2024-03-15T12:56:27.351246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:56:18.977509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423137+0000","last_change":"2024-03-15T12:56:25.435714+0000","last_active":"2024-03-15T12:56:26.423137+0000","last_peered":"2024-03-15T12:56:26.423137+0000","last_clean":"2024-03-15T12:56:26.423137+0000","last_became_active":"2024-03-15T12:56:25.435315+0000","last_became_peered":"2024-03-15T12:56:25.435315+0000","last_unstale":"2024-03-15T12:56:26.423137+0000","last_undegraded":"2024-03-15T12:56:26.423137+0000","last_fullsized":"2024-03-15T12:56:26.423137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:10:03.509078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350956+0000","last_change":"2024-03-15T12:56:20.209356+0000","last_active":"2024-03-15T12:56:27.350956+0000","last_peered":"2024-03-15T12:56:27.350956+0000","last_clean":"2024-03-15T12:56:27.350956+0000","last_became_active":"2024-03-15T12:56:20.208995+0000","last_became_peered":"2024-03-15T12:56:20.208995+0000","last_unstale":"2024-03-15T12:56:27.350956+0000","last_undegraded":"2024-03-15T12:56:27.350956+0000","last_fullsized":"2024-03-15T12:56:27.350956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:14:08.554567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351946+0000","last_change":"2024-03-15T12:56:21.216315+0000","last_active":"2024-03-15T12:56:27.351946+0000","last_peered":"2024-03-15T12:56:27.351946+0000","last_clean":"2024-03-15T12:56:27.351946+0000","last_became_active":"2024-03-15T12:56:21.215943+0000","last_became_peered":"2024-03-15T12:56:21.215943+0000","last_unstale":"2024-03-15T12:56:27.351946+0000","last_undegraded":"2024-03-15T12:56:27.351946+0000","last_fullsized":"2024-03-15T12:56:27.351946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:47:29.913048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350590+0000","last_change":"2024-03-15T12:56:25.437200+0000","last_active":"2024-03-15T12:56:27.350590+0000","last_peered":"2024-03-15T12:56:27.350590+0000","last_clean":"2024-03-15T12:56:27.350590+0000","last_became_active":"2024-03-15T12:56:25.436786+0000","last_became_peered":"2024-03-15T12:56:25.436786+0000","last_unstale":"2024-03-15T12:56:27.350590+0000","last_undegraded":"2024-03-15T12:56:27.350590+0000","last_fullsized":"2024-03-15T12:56:27.350590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:12:00.286749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:39.834469+0000","last_change":"2024-03-15T12:56:20.202240+0000","last_active":"2024-03-15T12:56:39.834469+0000","last_peered":"2024-03-15T12:56:39.834469+0000","last_clean":"2024-03-15T12:56:39.834469+0000","last_became_active":"2024-03-15T12:56:20.201136+0000","last_became_peered":"2024-03-15T12:56:20.201136+0000","last_unstale":"2024-03-15T12:56:39.834469+0000","last_undegraded":"2024-03-15T12:56:39.834469+0000","last_fullsized":"2024-03-15T12:56:39.834469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:30:35.901237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349546+0000","last_change":"2024-03-15T12:56:21.212095+0000","last_active":"2024-03-15T12:56:27.349546+0000","last_peered":"2024-03-15T12:56:27.349546+0000","last_clean":"2024-03-15T12:56:27.349546+0000","last_became_active":"2024-03-15T12:56:21.211744+0000","last_became_peered":"2024-03-15T12:56:21.211744+0000","last_unstale":"2024-03-15T12:56:27.349546+0000","last_undegraded":"2024-03-15T12:56:27.349546+0000","last_fullsized":"2024-03-15T12:56:27.349546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:27:58.412513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423662+0000","last_change":"2024-03-15T12:56:25.448857+0000","last_active":"2024-03-15T12:56:26.423662+0000","last_peered":"2024-03-15T12:56:26.423662+0000","last_clean":"2024-03-15T12:56:26.423662+0000","last_became_active":"2024-03-15T12:56:25.448461+0000","last_became_peered":"2024-03-15T12:56:25.448461+0000","last_unstale":"2024-03-15T12:56:26.423662+0000","last_undegraded":"2024-03-15T12:56:26.423662+0000","last_fullsized":"2024-03-15T12:56:26.423662+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:21:36.426729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349430+0000","last_change":"2024-03-15T12:56:20.217319+0000","last_active":"2024-03-15T12:56:27.349430+0000","last_peered":"2024-03-15T12:56:27.349430+0000","last_clean":"2024-03-15T12:56:27.349430+0000","last_became_active":"2024-03-15T12:56:20.215450+0000","last_became_peered":"2024-03-15T12:56:20.215450+0000","last_unstale":"2024-03-15T12:56:27.349430+0000","last_undegraded":"2024-03-15T12:56:27.349430+0000","last_fullsized":"2024-03-15T12:56:27.349430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:07:09.085860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580383+0000","last_change":"2024-03-15T12:56:21.209360+0000","last_active":"2024-03-15T12:56:27.580383+0000","last_peered":"2024-03-15T12:56:27.580383+0000","last_clean":"2024-03-15T12:56:27.580383+0000","last_became_active":"2024-03-15T12:56:21.208987+0000","last_became_peered":"2024-03-15T12:56:21.208987+0000","last_unstale":"2024-03-15T12:56:27.580383+0000","last_undegraded":"2024-03-15T12:56:27.580383+0000","last_fullsized":"2024-03-15T12:56:27.580383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:19:13.353400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351291+0000","last_change":"2024-03-15T12:56:25.439173+0000","last_active":"2024-03-15T12:56:27.351291+0000","last_peered":"2024-03-15T12:56:27.351291+0000","last_clean":"2024-03-15T12:56:27.351291+0000","last_became_active":"2024-03-15T12:56:25.438812+0000","last_became_peered":"2024-03-15T12:56:25.438812+0000","last_unstale":"2024-03-15T12:56:27.351291+0000","last_undegraded":"2024-03-15T12:56:27.351291+0000","last_fullsized":"2024-03-15T12:56:27.351291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:10:39.107190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423123+0000","last_change":"2024-03-15T12:56:20.211861+0000","last_active":"2024-03-15T12:56:26.423123+0000","last_peered":"2024-03-15T12:56:26.423123+0000","last_clean":"2024-03-15T12:56:26.423123+0000","last_became_active":"2024-03-15T12:56:20.211481+0000","last_became_peered":"2024-03-15T12:56:20.211481+0000","last_unstale":"2024-03-15T12:56:26.423123+0000","last_undegraded":"2024-03-15T12:56:26.423123+0000","last_fullsized":"2024-03-15T12:56:26.423123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:12:45.558423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350332+0000","last_change":"2024-03-15T12:56:21.216058+0000","last_active":"2024-03-15T12:56:27.350332+0000","last_peered":"2024-03-15T12:56:27.350332+0000","last_clean":"2024-03-15T12:56:27.350332+0000","last_became_active":"2024-03-15T12:56:21.215706+0000","last_became_peered":"2024-03-15T12:56:21.215706+0000","last_unstale":"2024-03-15T12:56:27.350332+0000","last_undegraded":"2024-03-15T12:56:27.350332+0000","last_fullsized":"2024-03-15T12:56:27.350332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:55:02.132976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350367+0000","last_change":"2024-03-15T12:56:25.444134+0000","last_active":"2024-03-15T12:56:27.350367+0000","last_peered":"2024-03-15T12:56:27.350367+0000","last_clean":"2024-03-15T12:56:27.350367+0000","last_became_active":"2024-03-15T12:56:25.443452+0000","last_became_peered":"2024-03-15T12:56:25.443452+0000","last_unstale":"2024-03-15T12:56:27.350367+0000","last_undegraded":"2024-03-15T12:56:27.350367+0000","last_fullsized":"2024-03-15T12:56:27.350367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:06:39.660016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349667+0000","last_change":"2024-03-15T12:56:20.214700+0000","last_active":"2024-03-15T12:56:27.349667+0000","last_peered":"2024-03-15T12:56:27.349667+0000","last_clean":"2024-03-15T12:56:27.349667+0000","last_became_active":"2024-03-15T12:56:20.214348+0000","last_became_peered":"2024-03-15T12:56:20.214348+0000","last_unstale":"2024-03-15T12:56:27.349667+0000","last_undegraded":"2024-03-15T12:56:27.349667+0000","last_fullsized":"2024-03-15T12:56:27.349667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:24:37.551182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423507+0000","last_change":"2024-03-15T12:56:21.220746+0000","last_active":"2024-03-15T12:56:26.423507+0000","last_peered":"2024-03-15T12:56:26.423507+0000","last_clean":"2024-03-15T12:56:26.423507+0000","last_became_active":"2024-03-15T12:56:21.220445+0000","last_became_peered":"2024-03-15T12:56:21.220445+0000","last_unstale":"2024-03-15T12:56:26.423507+0000","last_undegraded":"2024-03-15T12:56:26.423507+0000","last_fullsized":"2024-03-15T12:56:26.423507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:58:44.119476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349618+0000","last_change":"2024-03-15T12:56:25.437611+0000","last_active":"2024-03-15T12:56:27.349618+0000","last_peered":"2024-03-15T12:56:27.349618+0000","last_clean":"2024-03-15T12:56:27.349618+0000","last_became_active":"2024-03-15T12:56:25.437195+0000","last_became_peered":"2024-03-15T12:56:25.437195+0000","last_unstale":"2024-03-15T12:56:27.349618+0000","last_undegraded":"2024-03-15T12:56:27.349618+0000","last_fullsized":"2024-03-15T12:56:27.349618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:26:43.558477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423602+0000","last_change":"2024-03-15T12:56:20.208584+0000","last_active":"2024-03-15T12:56:26.423602+0000","last_peered":"2024-03-15T12:56:26.423602+0000","last_clean":"2024-03-15T12:56:26.423602+0000","last_became_active":"2024-03-15T12:56:20.208155+0000","last_became_peered":"2024-03-15T12:56:20.208155+0000","last_unstale":"2024-03-15T12:56:26.423602+0000","last_undegraded":"2024-03-15T12:56:26.423602+0000","last_fullsized":"2024-03-15T12:56:26.423602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:26:13.148273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423884+0000","last_change":"2024-03-15T12:56:21.220421+0000","last_active":"2024-03-15T12:56:26.423884+0000","last_peered":"2024-03-15T12:56:26.423884+0000","last_clean":"2024-03-15T12:56:26.423884+0000","last_became_active":"2024-03-15T12:56:21.220095+0000","last_became_peered":"2024-03-15T12:56:21.220095+0000","last_unstale":"2024-03-15T12:56:26.423884+0000","last_undegraded":"2024-03-15T12:56:26.423884+0000","last_fullsized":"2024-03-15T12:56:26.423884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:35:33.846506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351851+0000","last_change":"2024-03-15T12:56:25.440408+0000","last_active":"2024-03-15T12:56:27.351851+0000","last_peered":"2024-03-15T12:56:27.351851+0000","last_clean":"2024-03-15T12:56:27.351851+0000","last_became_active":"2024-03-15T12:56:25.439965+0000","last_became_peered":"2024-03-15T12:56:25.439965+0000","last_unstale":"2024-03-15T12:56:27.351851+0000","last_undegraded":"2024-03-15T12:56:27.351851+0000","last_fullsized":"2024-03-15T12:56:27.351851+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:12:57.278202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.849290+0000","last_change":"2024-03-15T12:56:20.216320+0000","last_active":"2024-03-15T12:56:26.849290+0000","last_peered":"2024-03-15T12:56:26.849290+0000","last_clean":"2024-03-15T12:56:26.849290+0000","last_became_active":"2024-03-15T12:56:20.215915+0000","last_became_peered":"2024-03-15T12:56:20.215915+0000","last_unstale":"2024-03-15T12:56:26.849290+0000","last_undegraded":"2024-03-15T12:56:26.849290+0000","last_fullsized":"2024-03-15T12:56:26.849290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:25:45.236269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423496+0000","last_change":"2024-03-15T12:56:21.215822+0000","last_active":"2024-03-15T12:56:26.423496+0000","last_peered":"2024-03-15T12:56:26.423496+0000","last_clean":"2024-03-15T12:56:26.423496+0000","last_became_active":"2024-03-15T12:56:21.215508+0000","last_became_peered":"2024-03-15T12:56:21.215508+0000","last_unstale":"2024-03-15T12:56:26.423496+0000","last_undegraded":"2024-03-15T12:56:26.423496+0000","last_fullsized":"2024-03-15T12:56:26.423496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:56:55.919198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848626+0000","last_change":"2024-03-15T12:56:25.436186+0000","last_active":"2024-03-15T12:56:26.848626+0000","last_peered":"2024-03-15T12:56:26.848626+0000","last_clean":"2024-03-15T12:56:26.848626+0000","last_became_active":"2024-03-15T12:56:25.435586+0000","last_became_peered":"2024-03-15T12:56:25.435586+0000","last_unstale":"2024-03-15T12:56:26.848626+0000","last_undegraded":"2024-03-15T12:56:26.848626+0000","last_fullsized":"2024-03-15T12:56:26.848626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:02:26.850487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350644+0000","last_change":"2024-03-15T12:56:20.217277+0000","last_active":"2024-03-15T12:56:27.350644+0000","last_peered":"2024-03-15T12:56:27.350644+0000","last_clean":"2024-03-15T12:56:27.350644+0000","last_became_active":"2024-03-15T12:56:20.216930+0000","last_became_peered":"2024-03-15T12:56:20.216930+0000","last_unstale":"2024-03-15T12:56:27.350644+0000","last_undegraded":"2024-03-15T12:56:27.350644+0000","last_fullsized":"2024-03-15T12:56:27.350644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:47:51.075545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423705+0000","last_change":"2024-03-15T12:56:21.212066+0000","last_active":"2024-03-15T12:56:26.423705+0000","last_peered":"2024-03-15T12:56:26.423705+0000","last_clean":"2024-03-15T12:56:26.423705+0000","last_became_active":"2024-03-15T12:56:21.211650+0000","last_became_peered":"2024-03-15T12:56:21.211650+0000","last_unstale":"2024-03-15T12:56:26.423705+0000","last_undegraded":"2024-03-15T12:56:26.423705+0000","last_fullsized":"2024-03-15T12:56:26.423705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:40:35.322210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580573+0000","last_change":"2024-03-15T12:56:21.212794+0000","last_active":"2024-03-15T12:56:27.580573+0000","last_peered":"2024-03-15T12:56:27.580573+0000","last_clean":"2024-03-15T12:56:27.580573+0000","last_became_active":"2024-03-15T12:56:21.212470+0000","last_became_peered":"2024-03-15T12:56:21.212470+0000","last_unstale":"2024-03-15T12:56:27.580573+0000","last_undegraded":"2024-03-15T12:56:27.580573+0000","last_fullsized":"2024-03-15T12:56:27.580573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:10:04.614223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423286+0000","last_change":"2024-03-15T12:56:21.211746+0000","last_active":"2024-03-15T12:56:26.423286+0000","last_peered":"2024-03-15T12:56:26.423286+0000","last_clean":"2024-03-15T12:56:26.423286+0000","last_became_active":"2024-03-15T12:56:21.211431+0000","last_became_peered":"2024-03-15T12:56:21.211431+0000","last_unstale":"2024-03-15T12:56:26.423286+0000","last_undegraded":"2024-03-15T12:56:26.423286+0000","last_fullsized":"2024-03-15T12:56:26.423286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:46:31.188408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423138+0000","last_change":"2024-03-15T12:56:21.220997+0000","last_active":"2024-03-15T12:56:26.423138+0000","last_peered":"2024-03-15T12:56:26.423138+0000","last_clean":"2024-03-15T12:56:26.423138+0000","last_became_active":"2024-03-15T12:56:21.220644+0000","last_became_peered":"2024-03-15T12:56:21.220644+0000","last_unstale":"2024-03-15T12:56:26.423138+0000","last_undegraded":"2024-03-15T12:56:26.423138+0000","last_fullsized":"2024-03-15T12:56:26.423138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:47:55.852845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351539+0000","last_change":"2024-03-15T12:56:21.211634+0000","last_active":"2024-03-15T12:56:27.351539+0000","last_peered":"2024-03-15T12:56:27.351539+0000","last_clean":"2024-03-15T12:56:27.351539+0000","last_became_active":"2024-03-15T12:56:21.211303+0000","last_became_peered":"2024-03-15T12:56:21.211303+0000","last_unstale":"2024-03-15T12:56:27.351539+0000","last_undegraded":"2024-03-15T12:56:27.351539+0000","last_fullsized":"2024-03-15T12:56:27.351539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:22:32.358270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350501+0000","last_change":"2024-03-15T12:56:21.211673+0000","last_active":"2024-03-15T12:56:27.350501+0000","last_peered":"2024-03-15T12:56:27.350501+0000","last_clean":"2024-03-15T12:56:27.350501+0000","last_became_active":"2024-03-15T12:56:21.211296+0000","last_became_peered":"2024-03-15T12:56:21.211296+0000","last_unstale":"2024-03-15T12:56:27.350501+0000","last_undegraded":"2024-03-15T12:56:27.350501+0000","last_fullsized":"2024-03-15T12:56:27.350501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:40:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580787+0000","last_change":"2024-03-15T12:56:21.208681+0000","last_active":"2024-03-15T12:56:27.580787+0000","last_peered":"2024-03-15T12:56:27.580787+0000","last_clean":"2024-03-15T12:56:27.580787+0000","last_became_active":"2024-03-15T12:56:21.208130+0000","last_became_peered":"2024-03-15T12:56:21.208130+0000","last_unstale":"2024-03-15T12:56:27.580787+0000","last_undegraded":"2024-03-15T12:56:27.580787+0000","last_fullsized":"2024-03-15T12:56:27.580787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:34:52.971485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423883+0000","last_change":"2024-03-15T12:56:21.216979+0000","last_active":"2024-03-15T12:56:26.423883+0000","last_peered":"2024-03-15T12:56:26.423883+0000","last_clean":"2024-03-15T12:56:26.423883+0000","last_became_active":"2024-03-15T12:56:21.216632+0000","last_became_peered":"2024-03-15T12:56:21.216632+0000","last_unstale":"2024-03-15T12:56:26.423883+0000","last_undegraded":"2024-03-15T12:56:26.423883+0000","last_fullsized":"2024-03-15T12:56:26.423883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:21:57.213512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351438+0000","last_change":"2024-03-15T12:56:21.223064+0000","last_active":"2024-03-15T12:56:27.351438+0000","last_peered":"2024-03-15T12:56:27.351438+0000","last_clean":"2024-03-15T12:56:27.351438+0000","last_became_active":"2024-03-15T12:56:21.220882+0000","last_became_peered":"2024-03-15T12:56:21.220882+0000","last_unstale":"2024-03-15T12:56:27.351438+0000","last_undegraded":"2024-03-15T12:56:27.351438+0000","last_fullsized":"2024-03-15T12:56:27.351438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:45:02.930154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848703+0000","last_change":"2024-03-15T12:56:21.218614+0000","last_active":"2024-03-15T12:56:26.848703+0000","last_peered":"2024-03-15T12:56:26.848703+0000","last_clean":"2024-03-15T12:56:26.848703+0000","last_became_active":"2024-03-15T12:56:21.218224+0000","last_became_peered":"2024-03-15T12:56:21.218224+0000","last_unstale":"2024-03-15T12:56:26.848703+0000","last_undegraded":"2024-03-15T12:56:26.848703+0000","last_fullsized":"2024-03-15T12:56:26.848703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:41:29.790358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349429+0000","last_change":"2024-03-15T12:56:21.219019+0000","last_active":"2024-03-15T12:56:27.349429+0000","last_peered":"2024-03-15T12:56:27.349429+0000","last_clean":"2024-03-15T12:56:27.349429+0000","last_became_active":"2024-03-15T12:56:21.218673+0000","last_became_peered":"2024-03-15T12:56:21.218673+0000","last_unstale":"2024-03-15T12:56:27.349429+0000","last_undegraded":"2024-03-15T12:56:27.349429+0000","last_fullsized":"2024-03-15T12:56:27.349429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:42:11.788598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349737+0000","last_change":"2024-03-15T12:56:21.220355+0000","last_active":"2024-03-15T12:56:27.349737+0000","last_peered":"2024-03-15T12:56:27.349737+0000","last_clean":"2024-03-15T12:56:27.349737+0000","last_became_active":"2024-03-15T12:56:21.220021+0000","last_became_peered":"2024-03-15T12:56:21.220021+0000","last_unstale":"2024-03-15T12:56:27.349737+0000","last_undegraded":"2024-03-15T12:56:27.349737+0000","last_fullsized":"2024-03-15T12:56:27.349737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:40:00.947351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349887+0000","last_change":"2024-03-15T12:56:21.213050+0000","last_active":"2024-03-15T12:56:27.349887+0000","last_peered":"2024-03-15T12:56:27.349887+0000","last_clean":"2024-03-15T12:56:27.349887+0000","last_became_active":"2024-03-15T12:56:21.212730+0000","last_became_peered":"2024-03-15T12:56:21.212730+0000","last_unstale":"2024-03-15T12:56:27.349887+0000","last_undegraded":"2024-03-15T12:56:27.349887+0000","last_fullsized":"2024-03-15T12:56:27.349887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:56:36.396285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580559+0000","last_change":"2024-03-15T12:56:21.213942+0000","last_active":"2024-03-15T12:56:27.580559+0000","last_peered":"2024-03-15T12:56:27.580559+0000","last_clean":"2024-03-15T12:56:27.580559+0000","last_became_active":"2024-03-15T12:56:21.213611+0000","last_became_peered":"2024-03-15T12:56:21.213611+0000","last_unstale":"2024-03-15T12:56:27.580559+0000","last_undegraded":"2024-03-15T12:56:27.580559+0000","last_fullsized":"2024-03-15T12:56:27.580559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:23:44.002970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4730,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":16088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":476829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1601536,"data_stored":1071233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":987136,"data_stored":477411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":477433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36892,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708260,"statfs":{"total":95995035648,"available":95957258240,"internally_reserved":0,"allocated":1581056,"data_stored":1067779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":983040,"data_stored":476829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":480671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":987136,"data_stored":476919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":480705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":476933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":476851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1589248,"data_stored":1067379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-15T12:56:44.190 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-15T12:56:44.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.341+0000 7fedde447640 1 -- 172.21.15.2:0/1522401145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd81030d0 msgr2=0x7fedd8109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:44.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.341+0000 7fedde447640 1 --2- 172.21.15.2:0/1522401145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd81030d0 0x7fedd8109960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009ca0 tx=0x7fedd002c0f0 comp rx=0 tx=0).stop 2024-03-15T12:56:44.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.341+0000 7fedde447640 1 -- 172.21.15.2:0/1522401145 shutdown_connections 2024-03-15T12:56:44.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.341+0000 7fedde447640 1 --2- 172.21.15.2:0/1522401145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd81030d0 0x7fedd8109960 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:44.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.341+0000 7fedde447640 1 --2- 172.21.15.2:0/1522401145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedd8102710 0x7fedd8102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:44.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.341+0000 7fedde447640 1 --2- 172.21.15.2:0/1522401145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedd8101520 0x7fedd8101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:44.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.341+0000 7fedde447640 1 -- 172.21.15.2:0/1522401145 >> 172.21.15.2:0/1522401145 conn(0x7fedd80fca90 msgr2=0x7fedd80feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:44.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.341+0000 7fedde447640 1 -- 172.21.15.2:0/1522401145 shutdown_connections 2024-03-15T12:56:44.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.341+0000 7fedde447640 1 -- 172.21.15.2:0/1522401145 wait complete. 2024-03-15T12:56:44.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedde447640 1 Processor -- start 2024-03-15T12:56:44.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedde447640 1 -- start start 2024-03-15T12:56:44.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedde447640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedd8101520 0x7fedd81975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:44.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedde447640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102710 0x7fedd8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:44.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedde447640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedd81030d0 0x7fedd819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:44.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedde447640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd819f0a0 con 0x7fedd8102710 2024-03-15T12:56:44.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedde447640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd819f210 con 0x7fedd81030d0 2024-03-15T12:56:44.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedde447640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd819f3d0 con 0x7fedd8101520 2024-03-15T12:56:44.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedd77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102710 0x7fedd8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:44.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedd77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102710 0x7fedd8197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41542/0 (socket says 172.21.15.2:41542) 2024-03-15T12:56:44.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedd77fe640 1 -- 172.21.15.2:0/738160279 learned_addr learned my addr 172.21.15.2:0/738160279 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:44.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7feddc9e4640 1 --2- 172.21.15.2:0/738160279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedd81030d0 0x7fedd819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:44.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedd7fff640 1 --2- 172.21.15.2:0/738160279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedd8101520 0x7fedd81975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:44.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedd77fe640 1 -- 172.21.15.2:0/738160279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedd8101520 msgr2=0x7fedd81975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:44.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedd77fe640 1 --2- 172.21.15.2:0/738160279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedd8101520 0x7fedd81975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:44.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedd77fe640 1 -- 172.21.15.2:0/738160279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedd81030d0 msgr2=0x7fedd819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:44.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedd77fe640 1 --2- 172.21.15.2:0/738160279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedd81030d0 0x7fedd819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:44.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7fedd77fe640 1 -- 172.21.15.2:0/738160279 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc009490 con 0x7fedd8102710 2024-03-15T12:56:44.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.342+0000 7feddc9e4640 1 --2- 172.21.15.2:0/738160279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedd81030d0 0x7fedd819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:44.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.343+0000 7fedd77fe640 1 --2- 172.21.15.2:0/738160279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102710 0x7fedd8197af0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00ea60 tx=0x7fedcc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:44.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.343+0000 7fedd57fa640 1 -- 172.21.15.2:0/738160279 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc00cde0 con 0x7fedd8102710 2024-03-15T12:56:44.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.343+0000 7fedd57fa640 1 -- 172.21.15.2:0/738160279 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fedcc0035f0 con 0x7fedd8102710 2024-03-15T12:56:44.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.343+0000 7fedd57fa640 1 -- 172.21.15.2:0/738160279 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc010690 con 0x7fedd8102710 2024-03-15T12:56:44.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.343+0000 7fedde447640 1 -- 172.21.15.2:0/738160279 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd0009900 con 0x7fedd8102710 2024-03-15T12:56:44.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.343+0000 7fedde447640 1 -- 172.21.15.2:0/738160279 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd819fa80 con 0x7fedd8102710 2024-03-15T12:56:44.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.344+0000 7fedde447640 1 -- 172.21.15.2:0/738160279 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd8108960 con 0x7fedd8102710 2024-03-15T12:56:44.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.345+0000 7fedd57fa640 1 -- 172.21.15.2:0/738160279 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7fedcc003760 con 0x7fedd8102710 2024-03-15T12:56:44.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.346+0000 7fedd57fa640 1 --2- 172.21.15.2:0/738160279 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fedc00a1b40 0x7fedc00a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:44.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.346+0000 7fedd57fa640 1 -- 172.21.15.2:0/738160279 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fedcc014070 con 0x7fedd8102710 2024-03-15T12:56:44.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.346+0000 7fedd7fff640 1 --2- 172.21.15.2:0/738160279 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fedc00a1b40 0x7fedc00a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:44.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.346+0000 7fedd7fff640 1 --2- 172.21.15.2:0/738160279 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fedc00a1b40 0x7fedc00a4000 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fedd8102570 tx=0x7fedc8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:44.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.350+0000 7fedd57fa640 1 -- 172.21.15.2:0/738160279 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fedcc096a30 con 0x7fedd8102710 2024-03-15T12:56:44.443 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:44 smithi002 ceph-mon[32060]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:56:44.443 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:44 smithi002 ceph-mon[32060]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:56:44.443 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:44 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] up:active 2024-03-15T12:56:44.443 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:44 smithi002 ceph-mon[32060]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-03-15T12:56:44.443 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:44 smithi002 ceph-mon[32060]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-15T12:56:44.443 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:44 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T12:56:44.444 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:44 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3113032203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:44.444 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:44 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2023522141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:44.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.547+0000 7fedde447640 1 -- 172.21.15.2:0/738160279 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fedd802d070 con 0x7fedc00a1b40 2024-03-15T12:56:44.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.557+0000 7fedd57fa640 1 -- 172.21.15.2:0/738160279 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340194 (secure 0 0 0) 0x7fedd802d070 con 0x7fedc00a1b40 2024-03-15T12:56:44.547 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:44.549 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-03-15T12:56:44.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.560+0000 7fedde447640 1 -- 172.21.15.2:0/738160279 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fedc00a1b40 msgr2=0x7fedc00a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:44.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.560+0000 7fedde447640 1 --2- 172.21.15.2:0/738160279 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fedc00a1b40 0x7fedc00a4000 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fedd8102570 tx=0x7fedc8005f90 comp rx=0 tx=0).stop 2024-03-15T12:56:44.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.560+0000 7fedde447640 1 -- 172.21.15.2:0/738160279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102710 msgr2=0x7fedd8197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:44.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.560+0000 7fedde447640 1 --2- 172.21.15.2:0/738160279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102710 0x7fedd8197af0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00ea60 tx=0x7fedcc00ef20 comp rx=0 tx=0).stop 2024-03-15T12:56:44.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.560+0000 7fedde447640 1 -- 172.21.15.2:0/738160279 shutdown_connections 2024-03-15T12:56:44.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.560+0000 7fedde447640 1 --2- 172.21.15.2:0/738160279 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fedc00a1b40 0x7fedc00a4000 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:44.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.560+0000 7fedde447640 1 --2- 172.21.15.2:0/738160279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedd81030d0 0x7fedd819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:44.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.560+0000 7fedde447640 1 --2- 172.21.15.2:0/738160279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102710 0x7fedd8197af0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:44.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.560+0000 7fedde447640 1 --2- 172.21.15.2:0/738160279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedd8101520 0x7fedd81975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:44.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.560+0000 7fedde447640 1 -- 172.21.15.2:0/738160279 >> 172.21.15.2:0/738160279 conn(0x7fedd80fca90 msgr2=0x7fedd8105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:44.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.560+0000 7fedde447640 1 -- 172.21.15.2:0/738160279 shutdown_connections 2024-03-15T12:56:44.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:44.560+0000 7fedde447640 1 -- 172.21.15.2:0/738160279 wait complete. 2024-03-15T12:56:44.557 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-03-15T12:56:42.744908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595098,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":243,"num_write_kb":2958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":189,"ondisk_log_size":189,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400012,"kb_used_data":13356,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124541812,"statfs":{"total":1151940427776,"available":1151530815488,"internally_reserved":0,"allocated":13676544,"data_stored":7506972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2320,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003145"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580936+0000","last_change":"2024-03-15T12:56:21.210343+0000","last_active":"2024-03-15T12:56:27.580936+0000","last_peered":"2024-03-15T12:56:27.580936+0000","last_clean":"2024-03-15T12:56:27.580936+0000","last_became_active":"2024-03-15T12:56:21.209787+0000","last_became_peered":"2024-03-15T12:56:21.209787+0000","last_unstale":"2024-03-15T12:56:27.580936+0000","last_undegraded":"2024-03-15T12:56:27.580936+0000","last_fullsized":"2024-03-15T12:56:27.580936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:32:21.204944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351944+0000","last_change":"2024-03-15T12:56:21.214153+0000","last_active":"2024-03-15T12:56:27.351944+0000","last_peered":"2024-03-15T12:56:27.351944+0000","last_clean":"2024-03-15T12:56:27.351944+0000","last_became_active":"2024-03-15T12:56:21.213819+0000","last_became_peered":"2024-03-15T12:56:21.213819+0000","last_unstale":"2024-03-15T12:56:27.351944+0000","last_undegraded":"2024-03-15T12:56:27.351944+0000","last_fullsized":"2024-03-15T12:56:27.351944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:31:43.264698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351630+0000","last_change":"2024-03-15T12:56:21.221678+0000","last_active":"2024-03-15T12:56:27.351630+0000","last_peered":"2024-03-15T12:56:27.351630+0000","last_clean":"2024-03-15T12:56:27.351630+0000","last_became_active":"2024-03-15T12:56:21.221272+0000","last_became_peered":"2024-03-15T12:56:21.221272+0000","last_unstale":"2024-03-15T12:56:27.351630+0000","last_undegraded":"2024-03-15T12:56:27.351630+0000","last_fullsized":"2024-03-15T12:56:27.351630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:01:50.778707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351395+0000","last_change":"2024-03-15T12:56:21.212939+0000","last_active":"2024-03-15T12:56:27.351395+0000","last_peered":"2024-03-15T12:56:27.351395+0000","last_clean":"2024-03-15T12:56:27.351395+0000","last_became_active":"2024-03-15T12:56:21.212437+0000","last_became_peered":"2024-03-15T12:56:21.212437+0000","last_unstale":"2024-03-15T12:56:27.351395+0000","last_undegraded":"2024-03-15T12:56:27.351395+0000","last_fullsized":"2024-03-15T12:56:27.351395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:34:11.377020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349659+0000","last_change":"2024-03-15T12:56:21.221433+0000","last_active":"2024-03-15T12:56:27.349659+0000","last_peered":"2024-03-15T12:56:27.349659+0000","last_clean":"2024-03-15T12:56:27.349659+0000","last_became_active":"2024-03-15T12:56:21.220605+0000","last_became_peered":"2024-03-15T12:56:21.220605+0000","last_unstale":"2024-03-15T12:56:27.349659+0000","last_undegraded":"2024-03-15T12:56:27.349659+0000","last_fullsized":"2024-03-15T12:56:27.349659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:20:25.169387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350654+0000","last_change":"2024-03-15T12:56:21.216539+0000","last_active":"2024-03-15T12:56:27.350654+0000","last_peered":"2024-03-15T12:56:27.350654+0000","last_clean":"2024-03-15T12:56:27.350654+0000","last_became_active":"2024-03-15T12:56:21.216197+0000","last_became_peered":"2024-03-15T12:56:21.216197+0000","last_unstale":"2024-03-15T12:56:27.350654+0000","last_undegraded":"2024-03-15T12:56:27.350654+0000","last_fullsized":"2024-03-15T12:56:27.350654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:00:23.186467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423278+0000","last_change":"2024-03-15T12:56:21.213103+0000","last_active":"2024-03-15T12:56:26.423278+0000","last_peered":"2024-03-15T12:56:26.423278+0000","last_clean":"2024-03-15T12:56:26.423278+0000","last_became_active":"2024-03-15T12:56:21.212789+0000","last_became_peered":"2024-03-15T12:56:21.212789+0000","last_unstale":"2024-03-15T12:56:26.423278+0000","last_undegraded":"2024-03-15T12:56:26.423278+0000","last_fullsized":"2024-03-15T12:56:26.423278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:41:50.140915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349505+0000","last_change":"2024-03-15T12:56:21.214767+0000","last_active":"2024-03-15T12:56:27.349505+0000","last_peered":"2024-03-15T12:56:27.349505+0000","last_clean":"2024-03-15T12:56:27.349505+0000","last_became_active":"2024-03-15T12:56:21.213896+0000","last_became_peered":"2024-03-15T12:56:21.213896+0000","last_unstale":"2024-03-15T12:56:27.349505+0000","last_undegraded":"2024-03-15T12:56:27.349505+0000","last_fullsized":"2024-03-15T12:56:27.349505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:33:53.819906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351682+0000","last_change":"2024-03-15T12:56:21.218566+0000","last_active":"2024-03-15T12:56:27.351682+0000","last_peered":"2024-03-15T12:56:27.351682+0000","last_clean":"2024-03-15T12:56:27.351682+0000","last_became_active":"2024-03-15T12:56:21.218234+0000","last_became_peered":"2024-03-15T12:56:21.218234+0000","last_unstale":"2024-03-15T12:56:27.351682+0000","last_undegraded":"2024-03-15T12:56:27.351682+0000","last_fullsized":"2024-03-15T12:56:27.351682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:56:46.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350790+0000","last_change":"2024-03-15T12:56:21.219367+0000","last_active":"2024-03-15T12:56:27.350790+0000","last_peered":"2024-03-15T12:56:27.350790+0000","last_clean":"2024-03-15T12:56:27.350790+0000","last_became_active":"2024-03-15T12:56:21.217541+0000","last_became_peered":"2024-03-15T12:56:21.217541+0000","last_unstale":"2024-03-15T12:56:27.350790+0000","last_undegraded":"2024-03-15T12:56:27.350790+0000","last_fullsized":"2024-03-15T12:56:27.350790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:19:12.939394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423882+0000","last_change":"2024-03-15T12:56:21.222246+0000","last_active":"2024-03-15T12:56:26.423882+0000","last_peered":"2024-03-15T12:56:26.423882+0000","last_clean":"2024-03-15T12:56:26.423882+0000","last_became_active":"2024-03-15T12:56:21.221883+0000","last_became_peered":"2024-03-15T12:56:21.221883+0000","last_unstale":"2024-03-15T12:56:26.423882+0000","last_undegraded":"2024-03-15T12:56:26.423882+0000","last_fullsized":"2024-03-15T12:56:26.423882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:42:06.892581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351340+0000","last_change":"2024-03-15T12:56:21.218463+0000","last_active":"2024-03-15T12:56:27.351340+0000","last_peered":"2024-03-15T12:56:27.351340+0000","last_clean":"2024-03-15T12:56:27.351340+0000","last_became_active":"2024-03-15T12:56:21.217934+0000","last_became_peered":"2024-03-15T12:56:21.217934+0000","last_unstale":"2024-03-15T12:56:27.351340+0000","last_undegraded":"2024-03-15T12:56:27.351340+0000","last_fullsized":"2024-03-15T12:56:27.351340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:37:29.355655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350069+0000","last_change":"2024-03-15T12:56:21.199631+0000","last_active":"2024-03-15T12:56:27.350069+0000","last_peered":"2024-03-15T12:56:27.350069+0000","last_clean":"2024-03-15T12:56:27.350069+0000","last_became_active":"2024-03-15T12:56:21.199230+0000","last_became_peered":"2024-03-15T12:56:21.199230+0000","last_unstale":"2024-03-15T12:56:27.350069+0000","last_undegraded":"2024-03-15T12:56:27.350069+0000","last_fullsized":"2024-03-15T12:56:27.350069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:45:51.356803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350582+0000","last_change":"2024-03-15T12:56:21.211449+0000","last_active":"2024-03-15T12:56:27.350582+0000","last_peered":"2024-03-15T12:56:27.350582+0000","last_clean":"2024-03-15T12:56:27.350582+0000","last_became_active":"2024-03-15T12:56:21.211001+0000","last_became_peered":"2024-03-15T12:56:21.211001+0000","last_unstale":"2024-03-15T12:56:27.350582+0000","last_undegraded":"2024-03-15T12:56:27.350582+0000","last_fullsized":"2024-03-15T12:56:27.350582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:32:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351256+0000","last_change":"2024-03-15T12:56:21.212001+0000","last_active":"2024-03-15T12:56:27.351256+0000","last_peered":"2024-03-15T12:56:27.351256+0000","last_clean":"2024-03-15T12:56:27.351256+0000","last_became_active":"2024-03-15T12:56:21.211497+0000","last_became_peered":"2024-03-15T12:56:21.211497+0000","last_unstale":"2024-03-15T12:56:27.351256+0000","last_undegraded":"2024-03-15T12:56:27.351256+0000","last_fullsized":"2024-03-15T12:56:27.351256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:27:16.029202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349369+0000","last_change":"2024-03-15T12:56:21.222368+0000","last_active":"2024-03-15T12:56:27.349369+0000","last_peered":"2024-03-15T12:56:27.349369+0000","last_clean":"2024-03-15T12:56:27.349369+0000","last_became_active":"2024-03-15T12:56:21.221356+0000","last_became_peered":"2024-03-15T12:56:21.221356+0000","last_unstale":"2024-03-15T12:56:27.349369+0000","last_undegraded":"2024-03-15T12:56:27.349369+0000","last_fullsized":"2024-03-15T12:56:27.349369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:15:04.125220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580757+0000","last_change":"2024-03-15T12:56:21.211323+0000","last_active":"2024-03-15T12:56:27.580757+0000","last_peered":"2024-03-15T12:56:27.580757+0000","last_clean":"2024-03-15T12:56:27.580757+0000","last_became_active":"2024-03-15T12:56:21.210983+0000","last_became_peered":"2024-03-15T12:56:21.210983+0000","last_unstale":"2024-03-15T12:56:27.580757+0000","last_undegraded":"2024-03-15T12:56:27.580757+0000","last_fullsized":"2024-03-15T12:56:27.580757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:10:51.487985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423526+0000","last_change":"2024-03-15T12:56:21.209435+0000","last_active":"2024-03-15T12:56:26.423526+0000","last_peered":"2024-03-15T12:56:26.423526+0000","last_clean":"2024-03-15T12:56:26.423526+0000","last_became_active":"2024-03-15T12:56:21.208981+0000","last_became_peered":"2024-03-15T12:56:21.208981+0000","last_unstale":"2024-03-15T12:56:26.423526+0000","last_undegraded":"2024-03-15T12:56:26.423526+0000","last_fullsized":"2024-03-15T12:56:26.423526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:47:59.142040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423775+0000","last_change":"2024-03-15T12:56:21.222014+0000","last_active":"2024-03-15T12:56:26.423775+0000","last_peered":"2024-03-15T12:56:26.423775+0000","last_clean":"2024-03-15T12:56:26.423775+0000","last_became_active":"2024-03-15T12:56:21.221548+0000","last_became_peered":"2024-03-15T12:56:21.221548+0000","last_unstale":"2024-03-15T12:56:26.423775+0000","last_undegraded":"2024-03-15T12:56:26.423775+0000","last_fullsized":"2024-03-15T12:56:26.423775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:12:48.943874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423518+0000","last_change":"2024-03-15T12:56:25.447200+0000","last_active":"2024-03-15T12:56:26.423518+0000","last_peered":"2024-03-15T12:56:26.423518+0000","last_clean":"2024-03-15T12:56:26.423518+0000","last_became_active":"2024-03-15T12:56:25.446516+0000","last_became_peered":"2024-03-15T12:56:25.446516+0000","last_unstale":"2024-03-15T12:56:26.423518+0000","last_undegraded":"2024-03-15T12:56:26.423518+0000","last_fullsized":"2024-03-15T12:56:26.423518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:06:46.221772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423639+0000","last_change":"2024-03-15T12:56:20.210826+0000","last_active":"2024-03-15T12:56:26.423639+0000","last_peered":"2024-03-15T12:56:26.423639+0000","last_clean":"2024-03-15T12:56:26.423639+0000","last_became_active":"2024-03-15T12:56:20.210472+0000","last_became_peered":"2024-03-15T12:56:20.210472+0000","last_unstale":"2024-03-15T12:56:26.423639+0000","last_undegraded":"2024-03-15T12:56:26.423639+0000","last_fullsized":"2024-03-15T12:56:26.423639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:33:22.896058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349871+0000","last_change":"2024-03-15T12:56:21.224665+0000","last_active":"2024-03-15T12:56:27.349871+0000","last_peered":"2024-03-15T12:56:27.349871+0000","last_clean":"2024-03-15T12:56:27.349871+0000","last_became_active":"2024-03-15T12:56:21.223826+0000","last_became_peered":"2024-03-15T12:56:21.223826+0000","last_unstale":"2024-03-15T12:56:27.349871+0000","last_undegraded":"2024-03-15T12:56:27.349871+0000","last_fullsized":"2024-03-15T12:56:27.349871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:11:57.576701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350133+0000","last_change":"2024-03-15T12:56:25.444665+0000","last_active":"2024-03-15T12:56:27.350133+0000","last_peered":"2024-03-15T12:56:27.350133+0000","last_clean":"2024-03-15T12:56:27.350133+0000","last_became_active":"2024-03-15T12:56:25.444275+0000","last_became_peered":"2024-03-15T12:56:25.444275+0000","last_unstale":"2024-03-15T12:56:27.350133+0000","last_undegraded":"2024-03-15T12:56:27.350133+0000","last_fullsized":"2024-03-15T12:56:27.350133+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:29:46.470405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351137+0000","last_change":"2024-03-15T12:56:20.215873+0000","last_active":"2024-03-15T12:56:27.351137+0000","last_peered":"2024-03-15T12:56:27.351137+0000","last_clean":"2024-03-15T12:56:27.351137+0000","last_became_active":"2024-03-15T12:56:20.215505+0000","last_became_peered":"2024-03-15T12:56:20.215505+0000","last_unstale":"2024-03-15T12:56:27.351137+0000","last_undegraded":"2024-03-15T12:56:27.351137+0000","last_fullsized":"2024-03-15T12:56:27.351137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:01:57.317382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351431+0000","last_change":"2024-03-15T12:56:21.218200+0000","last_active":"2024-03-15T12:56:27.351431+0000","last_peered":"2024-03-15T12:56:27.351431+0000","last_clean":"2024-03-15T12:56:27.351431+0000","last_became_active":"2024-03-15T12:56:21.217768+0000","last_became_peered":"2024-03-15T12:56:21.217768+0000","last_unstale":"2024-03-15T12:56:27.351431+0000","last_undegraded":"2024-03-15T12:56:27.351431+0000","last_fullsized":"2024-03-15T12:56:27.351431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:39:42.443542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350867+0000","last_change":"2024-03-15T12:56:25.444220+0000","last_active":"2024-03-15T12:56:27.350867+0000","last_peered":"2024-03-15T12:56:27.350867+0000","last_clean":"2024-03-15T12:56:27.350867+0000","last_became_active":"2024-03-15T12:56:25.443361+0000","last_became_peered":"2024-03-15T12:56:25.443361+0000","last_unstale":"2024-03-15T12:56:27.350867+0000","last_undegraded":"2024-03-15T12:56:27.350867+0000","last_fullsized":"2024-03-15T12:56:27.350867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:21:11.593526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350140+0000","last_change":"2024-03-15T12:56:20.218241+0000","last_active":"2024-03-15T12:56:27.350140+0000","last_peered":"2024-03-15T12:56:27.350140+0000","last_clean":"2024-03-15T12:56:27.350140+0000","last_became_active":"2024-03-15T12:56:20.217339+0000","last_became_peered":"2024-03-15T12:56:20.217339+0000","last_unstale":"2024-03-15T12:56:27.350140+0000","last_undegraded":"2024-03-15T12:56:27.350140+0000","last_fullsized":"2024-03-15T12:56:27.350140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:01:22.156502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351392+0000","last_change":"2024-03-15T12:56:21.217224+0000","last_active":"2024-03-15T12:56:27.351392+0000","last_peered":"2024-03-15T12:56:27.351392+0000","last_clean":"2024-03-15T12:56:27.351392+0000","last_became_active":"2024-03-15T12:56:21.216868+0000","last_became_peered":"2024-03-15T12:56:21.216868+0000","last_unstale":"2024-03-15T12:56:27.351392+0000","last_undegraded":"2024-03-15T12:56:27.351392+0000","last_fullsized":"2024-03-15T12:56:27.351392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:03:32.668463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350199+0000","last_change":"2024-03-15T12:56:25.443861+0000","last_active":"2024-03-15T12:56:27.350199+0000","last_peered":"2024-03-15T12:56:27.350199+0000","last_clean":"2024-03-15T12:56:27.350199+0000","last_became_active":"2024-03-15T12:56:25.443386+0000","last_became_peered":"2024-03-15T12:56:25.443386+0000","last_unstale":"2024-03-15T12:56:27.350199+0000","last_undegraded":"2024-03-15T12:56:27.350199+0000","last_fullsized":"2024-03-15T12:56:27.350199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:00:23.063425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349846+0000","last_change":"2024-03-15T12:56:20.212451+0000","last_active":"2024-03-15T12:56:27.349846+0000","last_peered":"2024-03-15T12:56:27.349846+0000","last_clean":"2024-03-15T12:56:27.349846+0000","last_became_active":"2024-03-15T12:56:20.212118+0000","last_became_peered":"2024-03-15T12:56:20.212118+0000","last_unstale":"2024-03-15T12:56:27.349846+0000","last_undegraded":"2024-03-15T12:56:27.349846+0000","last_fullsized":"2024-03-15T12:56:27.349846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:24:33.667652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.849201+0000","last_change":"2024-03-15T12:56:21.216499+0000","last_active":"2024-03-15T12:56:26.849201+0000","last_peered":"2024-03-15T12:56:26.849201+0000","last_clean":"2024-03-15T12:56:26.849201+0000","last_became_active":"2024-03-15T12:56:21.216170+0000","last_became_peered":"2024-03-15T12:56:21.216170+0000","last_unstale":"2024-03-15T12:56:26.849201+0000","last_undegraded":"2024-03-15T12:56:26.849201+0000","last_fullsized":"2024-03-15T12:56:26.849201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:20:26.675451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423625+0000","last_change":"2024-03-15T12:56:25.438194+0000","last_active":"2024-03-15T12:56:26.423625+0000","last_peered":"2024-03-15T12:56:26.423625+0000","last_clean":"2024-03-15T12:56:26.423625+0000","last_became_active":"2024-03-15T12:56:25.437769+0000","last_became_peered":"2024-03-15T12:56:25.437769+0000","last_unstale":"2024-03-15T12:56:26.423625+0000","last_undegraded":"2024-03-15T12:56:26.423625+0000","last_fullsized":"2024-03-15T12:56:26.423625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:29:44.398731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848873+0000","last_change":"2024-03-15T12:56:20.210552+0000","last_active":"2024-03-15T12:56:26.848873+0000","last_peered":"2024-03-15T12:56:26.848873+0000","last_clean":"2024-03-15T12:56:26.848873+0000","last_became_active":"2024-03-15T12:56:20.210191+0000","last_became_peered":"2024-03-15T12:56:20.210191+0000","last_unstale":"2024-03-15T12:56:26.848873+0000","last_undegraded":"2024-03-15T12:56:26.848873+0000","last_fullsized":"2024-03-15T12:56:26.848873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:25:48.225266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349697+0000","last_change":"2024-03-15T12:56:21.220582+0000","last_active":"2024-03-15T12:56:27.349697+0000","last_peered":"2024-03-15T12:56:27.349697+0000","last_clean":"2024-03-15T12:56:27.349697+0000","last_became_active":"2024-03-15T12:56:21.220212+0000","last_became_peered":"2024-03-15T12:56:21.220212+0000","last_unstale":"2024-03-15T12:56:27.349697+0000","last_undegraded":"2024-03-15T12:56:27.349697+0000","last_fullsized":"2024-03-15T12:56:27.349697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:00:54.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423603+0000","last_change":"2024-03-15T12:56:25.446410+0000","last_active":"2024-03-15T12:56:26.423603+0000","last_peered":"2024-03-15T12:56:26.423603+0000","last_clean":"2024-03-15T12:56:26.423603+0000","last_became_active":"2024-03-15T12:56:25.445554+0000","last_became_peered":"2024-03-15T12:56:25.445554+0000","last_unstale":"2024-03-15T12:56:26.423603+0000","last_undegraded":"2024-03-15T12:56:26.423603+0000","last_fullsized":"2024-03-15T12:56:26.423603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:35:08.049374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350291+0000","last_change":"2024-03-15T12:56:20.212599+0000","last_active":"2024-03-15T12:56:27.350291+0000","last_peered":"2024-03-15T12:56:27.350291+0000","last_clean":"2024-03-15T12:56:27.350291+0000","last_became_active":"2024-03-15T12:56:20.212289+0000","last_became_peered":"2024-03-15T12:56:20.212289+0000","last_unstale":"2024-03-15T12:56:27.350291+0000","last_undegraded":"2024-03-15T12:56:27.350291+0000","last_fullsized":"2024-03-15T12:56:27.350291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:12:51.036882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.849122+0000","last_change":"2024-03-15T12:56:21.219407+0000","last_active":"2024-03-15T12:56:26.849122+0000","last_peered":"2024-03-15T12:56:26.849122+0000","last_clean":"2024-03-15T12:56:26.849122+0000","last_became_active":"2024-03-15T12:56:21.219035+0000","last_became_peered":"2024-03-15T12:56:21.219035+0000","last_unstale":"2024-03-15T12:56:26.849122+0000","last_undegraded":"2024-03-15T12:56:26.849122+0000","last_fullsized":"2024-03-15T12:56:26.849122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:35:43.129767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580755+0000","last_change":"2024-03-15T12:56:25.434128+0000","last_active":"2024-03-15T12:56:27.580755+0000","last_peered":"2024-03-15T12:56:27.580755+0000","last_clean":"2024-03-15T12:56:27.580755+0000","last_became_active":"2024-03-15T12:56:25.433661+0000","last_became_peered":"2024-03-15T12:56:25.433661+0000","last_unstale":"2024-03-15T12:56:27.580755+0000","last_undegraded":"2024-03-15T12:56:27.580755+0000","last_fullsized":"2024-03-15T12:56:27.580755+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:32:05.771698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349605+0000","last_change":"2024-03-15T12:56:20.209933+0000","last_active":"2024-03-15T12:56:27.349605+0000","last_peered":"2024-03-15T12:56:27.349605+0000","last_clean":"2024-03-15T12:56:27.349605+0000","last_became_active":"2024-03-15T12:56:20.209047+0000","last_became_peered":"2024-03-15T12:56:20.209047+0000","last_unstale":"2024-03-15T12:56:27.349605+0000","last_undegraded":"2024-03-15T12:56:27.349605+0000","last_fullsized":"2024-03-15T12:56:27.349605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:24:05.551745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349310+0000","last_change":"2024-03-15T12:56:21.215198+0000","last_active":"2024-03-15T12:56:27.349310+0000","last_peered":"2024-03-15T12:56:27.349310+0000","last_clean":"2024-03-15T12:56:27.349310+0000","last_became_active":"2024-03-15T12:56:21.214844+0000","last_became_peered":"2024-03-15T12:56:21.214844+0000","last_unstale":"2024-03-15T12:56:27.349310+0000","last_undegraded":"2024-03-15T12:56:27.349310+0000","last_fullsized":"2024-03-15T12:56:27.349310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:37:08.398317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351861+0000","last_change":"2024-03-15T12:56:25.437047+0000","last_active":"2024-03-15T12:56:27.351861+0000","last_peered":"2024-03-15T12:56:27.351861+0000","last_clean":"2024-03-15T12:56:27.351861+0000","last_became_active":"2024-03-15T12:56:25.436595+0000","last_became_peered":"2024-03-15T12:56:25.436595+0000","last_unstale":"2024-03-15T12:56:27.351861+0000","last_undegraded":"2024-03-15T12:56:27.351861+0000","last_fullsized":"2024-03-15T12:56:27.351861+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:59:55.147044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848610+0000","last_change":"2024-03-15T12:56:20.206348+0000","last_active":"2024-03-15T12:56:26.848610+0000","last_peered":"2024-03-15T12:56:26.848610+0000","last_clean":"2024-03-15T12:56:26.848610+0000","last_became_active":"2024-03-15T12:56:20.206013+0000","last_became_peered":"2024-03-15T12:56:20.206013+0000","last_unstale":"2024-03-15T12:56:26.848610+0000","last_undegraded":"2024-03-15T12:56:26.848610+0000","last_fullsized":"2024-03-15T12:56:26.848610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:32:26.625982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.422867+0000","last_change":"2024-03-15T12:56:21.216517+0000","last_active":"2024-03-15T12:56:26.422867+0000","last_peered":"2024-03-15T12:56:26.422867+0000","last_clean":"2024-03-15T12:56:26.422867+0000","last_became_active":"2024-03-15T12:56:21.216076+0000","last_became_peered":"2024-03-15T12:56:21.216076+0000","last_unstale":"2024-03-15T12:56:26.422867+0000","last_undegraded":"2024-03-15T12:56:26.422867+0000","last_fullsized":"2024-03-15T12:56:26.422867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:19:35.547273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349813+0000","last_change":"2024-03-15T12:56:25.437367+0000","last_active":"2024-03-15T12:56:27.349813+0000","last_peered":"2024-03-15T12:56:27.349813+0000","last_clean":"2024-03-15T12:56:27.349813+0000","last_became_active":"2024-03-15T12:56:25.436930+0000","last_became_peered":"2024-03-15T12:56:25.436930+0000","last_unstale":"2024-03-15T12:56:27.349813+0000","last_undegraded":"2024-03-15T12:56:27.349813+0000","last_fullsized":"2024-03-15T12:56:27.349813+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:53:56.586786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:39.833875+0000","last_change":"2024-03-15T12:56:20.214067+0000","last_active":"2024-03-15T12:56:39.833875+0000","last_peered":"2024-03-15T12:56:39.833875+0000","last_clean":"2024-03-15T12:56:39.833875+0000","last_became_active":"2024-03-15T12:56:20.213696+0000","last_became_peered":"2024-03-15T12:56:20.213696+0000","last_unstale":"2024-03-15T12:56:39.833875+0000","last_undegraded":"2024-03-15T12:56:39.833875+0000","last_fullsized":"2024-03-15T12:56:39.833875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:35:15.343660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351660+0000","last_change":"2024-03-15T12:56:21.211106+0000","last_active":"2024-03-15T12:56:27.351660+0000","last_peered":"2024-03-15T12:56:27.351660+0000","last_clean":"2024-03-15T12:56:27.351660+0000","last_became_active":"2024-03-15T12:56:21.210756+0000","last_became_peered":"2024-03-15T12:56:21.210756+0000","last_unstale":"2024-03-15T12:56:27.351660+0000","last_undegraded":"2024-03-15T12:56:27.351660+0000","last_fullsized":"2024-03-15T12:56:27.351660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:13:25.232077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.422979+0000","last_change":"2024-03-15T12:56:25.441364+0000","last_active":"2024-03-15T12:56:26.422979+0000","last_peered":"2024-03-15T12:56:26.422979+0000","last_clean":"2024-03-15T12:56:26.422979+0000","last_became_active":"2024-03-15T12:56:25.440951+0000","last_became_peered":"2024-03-15T12:56:25.440951+0000","last_unstale":"2024-03-15T12:56:26.422979+0000","last_undegraded":"2024-03-15T12:56:26.422979+0000","last_fullsized":"2024-03-15T12:56:26.422979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:32:28.651856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423490+0000","last_change":"2024-03-15T12:56:20.208633+0000","last_active":"2024-03-15T12:56:26.423490+0000","last_peered":"2024-03-15T12:56:26.423490+0000","last_clean":"2024-03-15T12:56:26.423490+0000","last_became_active":"2024-03-15T12:56:20.208280+0000","last_became_peered":"2024-03-15T12:56:20.208280+0000","last_unstale":"2024-03-15T12:56:26.423490+0000","last_undegraded":"2024-03-15T12:56:26.423490+0000","last_fullsized":"2024-03-15T12:56:26.423490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:00:15.274090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351532+0000","last_change":"2024-03-15T12:56:21.218998+0000","last_active":"2024-03-15T12:56:27.351532+0000","last_peered":"2024-03-15T12:56:27.351532+0000","last_clean":"2024-03-15T12:56:27.351532+0000","last_became_active":"2024-03-15T12:56:21.218617+0000","last_became_peered":"2024-03-15T12:56:21.218617+0000","last_unstale":"2024-03-15T12:56:27.351532+0000","last_undegraded":"2024-03-15T12:56:27.351532+0000","last_fullsized":"2024-03-15T12:56:27.351532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:20:25.496204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351115+0000","last_change":"2024-03-15T12:56:25.438969+0000","last_active":"2024-03-15T12:56:27.351115+0000","last_peered":"2024-03-15T12:56:27.351115+0000","last_clean":"2024-03-15T12:56:27.351115+0000","last_became_active":"2024-03-15T12:56:25.438603+0000","last_became_peered":"2024-03-15T12:56:25.438603+0000","last_unstale":"2024-03-15T12:56:27.351115+0000","last_undegraded":"2024-03-15T12:56:27.351115+0000","last_fullsized":"2024-03-15T12:56:27.351115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:54:23.992253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.849081+0000","last_change":"2024-03-15T12:56:20.215152+0000","last_active":"2024-03-15T12:56:26.849081+0000","last_peered":"2024-03-15T12:56:26.849081+0000","last_clean":"2024-03-15T12:56:26.849081+0000","last_became_active":"2024-03-15T12:56:20.214762+0000","last_became_peered":"2024-03-15T12:56:20.214762+0000","last_unstale":"2024-03-15T12:56:26.849081+0000","last_undegraded":"2024-03-15T12:56:26.849081+0000","last_fullsized":"2024-03-15T12:56:26.849081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:38:45.414984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351465+0000","last_change":"2024-03-15T12:56:21.214005+0000","last_active":"2024-03-15T12:56:27.351465+0000","last_peered":"2024-03-15T12:56:27.351465+0000","last_clean":"2024-03-15T12:56:27.351465+0000","last_became_active":"2024-03-15T12:56:21.213672+0000","last_became_peered":"2024-03-15T12:56:21.213672+0000","last_unstale":"2024-03-15T12:56:27.351465+0000","last_undegraded":"2024-03-15T12:56:27.351465+0000","last_fullsized":"2024-03-15T12:56:27.351465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:13:09.384245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423234+0000","last_change":"2024-03-15T12:56:25.443145+0000","last_active":"2024-03-15T12:56:26.423234+0000","last_peered":"2024-03-15T12:56:26.423234+0000","last_clean":"2024-03-15T12:56:26.423234+0000","last_became_active":"2024-03-15T12:56:25.442755+0000","last_became_peered":"2024-03-15T12:56:25.442755+0000","last_unstale":"2024-03-15T12:56:26.423234+0000","last_undegraded":"2024-03-15T12:56:26.423234+0000","last_fullsized":"2024-03-15T12:56:26.423234+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:14:47.201919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:39.834429+0000","last_change":"2024-03-15T12:56:20.215063+0000","last_active":"2024-03-15T12:56:39.834429+0000","last_peered":"2024-03-15T12:56:39.834429+0000","last_clean":"2024-03-15T12:56:39.834429+0000","last_became_active":"2024-03-15T12:56:20.214742+0000","last_became_peered":"2024-03-15T12:56:20.214742+0000","last_unstale":"2024-03-15T12:56:39.834429+0000","last_undegraded":"2024-03-15T12:56:39.834429+0000","last_fullsized":"2024-03-15T12:56:39.834429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:38:12.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349955+0000","last_change":"2024-03-15T12:56:21.213519+0000","last_active":"2024-03-15T12:56:27.349955+0000","last_peered":"2024-03-15T12:56:27.349955+0000","last_clean":"2024-03-15T12:56:27.349955+0000","last_became_active":"2024-03-15T12:56:21.213198+0000","last_became_peered":"2024-03-15T12:56:21.213198+0000","last_unstale":"2024-03-15T12:56:27.349955+0000","last_undegraded":"2024-03-15T12:56:27.349955+0000","last_fullsized":"2024-03-15T12:56:27.349955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:22:13.037068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.849551+0000","last_change":"2024-03-15T12:56:25.446144+0000","last_active":"2024-03-15T12:56:26.849551+0000","last_peered":"2024-03-15T12:56:26.849551+0000","last_clean":"2024-03-15T12:56:26.849551+0000","last_became_active":"2024-03-15T12:56:25.445342+0000","last_became_peered":"2024-03-15T12:56:25.445342+0000","last_unstale":"2024-03-15T12:56:26.849551+0000","last_undegraded":"2024-03-15T12:56:26.849551+0000","last_fullsized":"2024-03-15T12:56:26.849551+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:57:28.280731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:39.834162+0000","last_change":"2024-03-15T12:56:20.216124+0000","last_active":"2024-03-15T12:56:39.834162+0000","last_peered":"2024-03-15T12:56:39.834162+0000","last_clean":"2024-03-15T12:56:39.834162+0000","last_became_active":"2024-03-15T12:56:20.215754+0000","last_became_peered":"2024-03-15T12:56:20.215754+0000","last_unstale":"2024-03-15T12:56:39.834162+0000","last_undegraded":"2024-03-15T12:56:39.834162+0000","last_fullsized":"2024-03-15T12:56:39.834162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:11:44.403503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349970+0000","last_change":"2024-03-15T12:56:21.214775+0000","last_active":"2024-03-15T12:56:27.349970+0000","last_peered":"2024-03-15T12:56:27.349970+0000","last_clean":"2024-03-15T12:56:27.349970+0000","last_became_active":"2024-03-15T12:56:21.214445+0000","last_became_peered":"2024-03-15T12:56:21.214445+0000","last_unstale":"2024-03-15T12:56:27.349970+0000","last_undegraded":"2024-03-15T12:56:27.349970+0000","last_fullsized":"2024-03-15T12:56:27.349970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:23:24.045670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580734+0000","last_change":"2024-03-15T12:56:25.438001+0000","last_active":"2024-03-15T12:56:27.580734+0000","last_peered":"2024-03-15T12:56:27.580734+0000","last_clean":"2024-03-15T12:56:27.580734+0000","last_became_active":"2024-03-15T12:56:25.437368+0000","last_became_peered":"2024-03-15T12:56:25.437368+0000","last_unstale":"2024-03-15T12:56:27.580734+0000","last_undegraded":"2024-03-15T12:56:27.580734+0000","last_fullsized":"2024-03-15T12:56:27.580734+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:39:27.648437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848969+0000","last_change":"2024-03-15T12:56:20.217191+0000","last_active":"2024-03-15T12:56:26.848969+0000","last_peered":"2024-03-15T12:56:26.848969+0000","last_clean":"2024-03-15T12:56:26.848969+0000","last_became_active":"2024-03-15T12:56:20.216632+0000","last_became_peered":"2024-03-15T12:56:20.216632+0000","last_unstale":"2024-03-15T12:56:26.848969+0000","last_undegraded":"2024-03-15T12:56:26.848969+0000","last_fullsized":"2024-03-15T12:56:26.848969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:16:51.187920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423724+0000","last_change":"2024-03-15T12:56:21.218650+0000","last_active":"2024-03-15T12:56:26.423724+0000","last_peered":"2024-03-15T12:56:26.423724+0000","last_clean":"2024-03-15T12:56:26.423724+0000","last_became_active":"2024-03-15T12:56:21.218329+0000","last_became_peered":"2024-03-15T12:56:21.218329+0000","last_unstale":"2024-03-15T12:56:26.423724+0000","last_undegraded":"2024-03-15T12:56:26.423724+0000","last_fullsized":"2024-03-15T12:56:26.423724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:10:45.632248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423518+0000","last_change":"2024-03-15T12:56:25.447982+0000","last_active":"2024-03-15T12:56:26.423518+0000","last_peered":"2024-03-15T12:56:26.423518+0000","last_clean":"2024-03-15T12:56:26.423518+0000","last_became_active":"2024-03-15T12:56:25.447514+0000","last_became_peered":"2024-03-15T12:56:25.447514+0000","last_unstale":"2024-03-15T12:56:26.423518+0000","last_undegraded":"2024-03-15T12:56:26.423518+0000","last_fullsized":"2024-03-15T12:56:26.423518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:16:52.567619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349976+0000","last_change":"2024-03-15T12:56:20.209778+0000","last_active":"2024-03-15T12:56:27.349976+0000","last_peered":"2024-03-15T12:56:27.349976+0000","last_clean":"2024-03-15T12:56:27.349976+0000","last_became_active":"2024-03-15T12:56:20.209311+0000","last_became_peered":"2024-03-15T12:56:20.209311+0000","last_unstale":"2024-03-15T12:56:27.349976+0000","last_undegraded":"2024-03-15T12:56:27.349976+0000","last_fullsized":"2024-03-15T12:56:27.349976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:05:21.930210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423470+0000","last_change":"2024-03-15T12:56:21.215576+0000","last_active":"2024-03-15T12:56:26.423470+0000","last_peered":"2024-03-15T12:56:26.423470+0000","last_clean":"2024-03-15T12:56:26.423470+0000","last_became_active":"2024-03-15T12:56:21.215261+0000","last_became_peered":"2024-03-15T12:56:21.215261+0000","last_unstale":"2024-03-15T12:56:26.423470+0000","last_undegraded":"2024-03-15T12:56:26.423470+0000","last_fullsized":"2024-03-15T12:56:26.423470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:35:15.461554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350810+0000","last_change":"2024-03-15T12:56:25.445559+0000","last_active":"2024-03-15T12:56:27.350810+0000","last_peered":"2024-03-15T12:56:27.350810+0000","last_clean":"2024-03-15T12:56:27.350810+0000","last_became_active":"2024-03-15T12:56:25.445092+0000","last_became_peered":"2024-03-15T12:56:25.445092+0000","last_unstale":"2024-03-15T12:56:27.350810+0000","last_undegraded":"2024-03-15T12:56:27.350810+0000","last_fullsized":"2024-03-15T12:56:27.350810+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:48:30.558243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349545+0000","last_change":"2024-03-15T12:56:20.214853+0000","last_active":"2024-03-15T12:56:27.349545+0000","last_peered":"2024-03-15T12:56:27.349545+0000","last_clean":"2024-03-15T12:56:27.349545+0000","last_became_active":"2024-03-15T12:56:20.214510+0000","last_became_peered":"2024-03-15T12:56:20.214510+0000","last_unstale":"2024-03-15T12:56:27.349545+0000","last_undegraded":"2024-03-15T12:56:27.349545+0000","last_fullsized":"2024-03-15T12:56:27.349545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:25:39.316428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848800+0000","last_change":"2024-03-15T12:56:21.216511+0000","last_active":"2024-03-15T12:56:26.848800+0000","last_peered":"2024-03-15T12:56:26.848800+0000","last_clean":"2024-03-15T12:56:26.848800+0000","last_became_active":"2024-03-15T12:56:21.216175+0000","last_became_peered":"2024-03-15T12:56:21.216175+0000","last_unstale":"2024-03-15T12:56:26.848800+0000","last_undegraded":"2024-03-15T12:56:26.848800+0000","last_fullsized":"2024-03-15T12:56:26.848800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:16:56.665841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848937+0000","last_change":"2024-03-15T12:56:25.446009+0000","last_active":"2024-03-15T12:56:26.848937+0000","last_peered":"2024-03-15T12:56:26.848937+0000","last_clean":"2024-03-15T12:56:26.848937+0000","last_became_active":"2024-03-15T12:56:25.445474+0000","last_became_peered":"2024-03-15T12:56:25.445474+0000","last_unstale":"2024-03-15T12:56:26.848937+0000","last_undegraded":"2024-03-15T12:56:26.848937+0000","last_fullsized":"2024-03-15T12:56:26.848937+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:53:46.856905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350147+0000","last_change":"2024-03-15T12:56:20.210146+0000","last_active":"2024-03-15T12:56:27.350147+0000","last_peered":"2024-03-15T12:56:27.350147+0000","last_clean":"2024-03-15T12:56:27.350147+0000","last_became_active":"2024-03-15T12:56:20.209769+0000","last_became_peered":"2024-03-15T12:56:20.209769+0000","last_unstale":"2024-03-15T12:56:27.350147+0000","last_undegraded":"2024-03-15T12:56:27.350147+0000","last_fullsized":"2024-03-15T12:56:27.350147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:47:32.505257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580613+0000","last_change":"2024-03-15T12:56:21.214826+0000","last_active":"2024-03-15T12:56:27.580613+0000","last_peered":"2024-03-15T12:56:27.580613+0000","last_clean":"2024-03-15T12:56:27.580613+0000","last_became_active":"2024-03-15T12:56:21.214160+0000","last_became_peered":"2024-03-15T12:56:21.214160+0000","last_unstale":"2024-03-15T12:56:27.580613+0000","last_undegraded":"2024-03-15T12:56:27.580613+0000","last_fullsized":"2024-03-15T12:56:27.580613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:23:17.800522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349871+0000","last_change":"2024-03-15T12:56:25.443496+0000","last_active":"2024-03-15T12:56:27.349871+0000","last_peered":"2024-03-15T12:56:27.349871+0000","last_clean":"2024-03-15T12:56:27.349871+0000","last_became_active":"2024-03-15T12:56:25.442796+0000","last_became_peered":"2024-03-15T12:56:25.442796+0000","last_unstale":"2024-03-15T12:56:27.349871+0000","last_undegraded":"2024-03-15T12:56:27.349871+0000","last_fullsized":"2024-03-15T12:56:27.349871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:35:23.892693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423277+0000","last_change":"2024-03-15T12:56:20.205207+0000","last_active":"2024-03-15T12:56:26.423277+0000","last_peered":"2024-03-15T12:56:26.423277+0000","last_clean":"2024-03-15T12:56:26.423277+0000","last_became_active":"2024-03-15T12:56:20.204759+0000","last_became_peered":"2024-03-15T12:56:20.204759+0000","last_unstale":"2024-03-15T12:56:26.423277+0000","last_undegraded":"2024-03-15T12:56:26.423277+0000","last_fullsized":"2024-03-15T12:56:26.423277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:47:12.575931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350541+0000","last_change":"2024-03-15T12:56:21.216393+0000","last_active":"2024-03-15T12:56:27.350541+0000","last_peered":"2024-03-15T12:56:27.350541+0000","last_clean":"2024-03-15T12:56:27.350541+0000","last_became_active":"2024-03-15T12:56:21.216043+0000","last_became_peered":"2024-03-15T12:56:21.216043+0000","last_unstale":"2024-03-15T12:56:27.350541+0000","last_undegraded":"2024-03-15T12:56:27.350541+0000","last_fullsized":"2024-03-15T12:56:27.350541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:05:49.843726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350384+0000","last_change":"2024-03-15T12:56:25.443854+0000","last_active":"2024-03-15T12:56:27.350384+0000","last_peered":"2024-03-15T12:56:27.350384+0000","last_clean":"2024-03-15T12:56:27.350384+0000","last_became_active":"2024-03-15T12:56:25.443422+0000","last_became_peered":"2024-03-15T12:56:25.443422+0000","last_unstale":"2024-03-15T12:56:27.350384+0000","last_undegraded":"2024-03-15T12:56:27.350384+0000","last_fullsized":"2024-03-15T12:56:27.350384+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:52:18.999161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423389+0000","last_change":"2024-03-15T12:56:20.211493+0000","last_active":"2024-03-15T12:56:26.423389+0000","last_peered":"2024-03-15T12:56:26.423389+0000","last_clean":"2024-03-15T12:56:26.423389+0000","last_became_active":"2024-03-15T12:56:20.211154+0000","last_became_peered":"2024-03-15T12:56:20.211154+0000","last_unstale":"2024-03-15T12:56:26.423389+0000","last_undegraded":"2024-03-15T12:56:26.423389+0000","last_fullsized":"2024-03-15T12:56:26.423389+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:26:28.026899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350374+0000","last_change":"2024-03-15T12:56:21.215999+0000","last_active":"2024-03-15T12:56:27.350374+0000","last_peered":"2024-03-15T12:56:27.350374+0000","last_clean":"2024-03-15T12:56:27.350374+0000","last_became_active":"2024-03-15T12:56:21.215657+0000","last_became_peered":"2024-03-15T12:56:21.215657+0000","last_unstale":"2024-03-15T12:56:27.350374+0000","last_undegraded":"2024-03-15T12:56:27.350374+0000","last_fullsized":"2024-03-15T12:56:27.350374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:47:59.901886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349712+0000","last_change":"2024-03-15T12:56:25.442246+0000","last_active":"2024-03-15T12:56:27.349712+0000","last_peered":"2024-03-15T12:56:27.349712+0000","last_clean":"2024-03-15T12:56:27.349712+0000","last_became_active":"2024-03-15T12:56:25.441755+0000","last_became_peered":"2024-03-15T12:56:25.441755+0000","last_unstale":"2024-03-15T12:56:27.349712+0000","last_undegraded":"2024-03-15T12:56:27.349712+0000","last_fullsized":"2024-03-15T12:56:27.349712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:47:36.037864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848845+0000","last_change":"2024-03-15T12:56:21.215746+0000","last_active":"2024-03-15T12:56:26.848845+0000","last_peered":"2024-03-15T12:56:26.848845+0000","last_clean":"2024-03-15T12:56:26.848845+0000","last_became_active":"2024-03-15T12:56:21.215424+0000","last_became_peered":"2024-03-15T12:56:21.215424+0000","last_unstale":"2024-03-15T12:56:26.848845+0000","last_undegraded":"2024-03-15T12:56:26.848845+0000","last_fullsized":"2024-03-15T12:56:26.848845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:01:15.498119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350368+0000","last_change":"2024-03-15T12:56:20.219705+0000","last_active":"2024-03-15T12:56:27.350368+0000","last_peered":"2024-03-15T12:56:27.350368+0000","last_clean":"2024-03-15T12:56:27.350368+0000","last_became_active":"2024-03-15T12:56:20.219328+0000","last_became_peered":"2024-03-15T12:56:20.219328+0000","last_unstale":"2024-03-15T12:56:27.350368+0000","last_undegraded":"2024-03-15T12:56:27.350368+0000","last_fullsized":"2024-03-15T12:56:27.350368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:47:52.171116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423843+0000","last_change":"2024-03-15T12:56:25.443094+0000","last_active":"2024-03-15T12:56:26.423843+0000","last_peered":"2024-03-15T12:56:26.423843+0000","last_clean":"2024-03-15T12:56:26.423843+0000","last_became_active":"2024-03-15T12:56:25.442708+0000","last_became_peered":"2024-03-15T12:56:25.442708+0000","last_unstale":"2024-03-15T12:56:26.423843+0000","last_undegraded":"2024-03-15T12:56:26.423843+0000","last_fullsized":"2024-03-15T12:56:26.423843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:03:07.769836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349746+0000","last_change":"2024-03-15T12:56:21.223937+0000","last_active":"2024-03-15T12:56:27.349746+0000","last_peered":"2024-03-15T12:56:27.349746+0000","last_clean":"2024-03-15T12:56:27.349746+0000","last_became_active":"2024-03-15T12:56:21.223583+0000","last_became_peered":"2024-03-15T12:56:21.223583+0000","last_unstale":"2024-03-15T12:56:27.349746+0000","last_undegraded":"2024-03-15T12:56:27.349746+0000","last_fullsized":"2024-03-15T12:56:27.349746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:15:34.287445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350925+0000","last_change":"2024-03-15T12:56:20.210812+0000","last_active":"2024-03-15T12:56:27.350925+0000","last_peered":"2024-03-15T12:56:27.350925+0000","last_clean":"2024-03-15T12:56:27.350925+0000","last_became_active":"2024-03-15T12:56:20.210457+0000","last_became_peered":"2024-03-15T12:56:20.210457+0000","last_unstale":"2024-03-15T12:56:27.350925+0000","last_undegraded":"2024-03-15T12:56:27.350925+0000","last_fullsized":"2024-03-15T12:56:27.350925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:04:37.371637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423245+0000","last_change":"2024-03-15T12:55:08.392548+0000","last_active":"2024-03-15T12:56:26.423245+0000","last_peered":"2024-03-15T12:56:26.423245+0000","last_clean":"2024-03-15T12:56:26.423245+0000","last_became_active":"2024-03-15T12:55:08.391972+0000","last_became_peered":"2024-03-15T12:55:08.391972+0000","last_unstale":"2024-03-15T12:56:26.423245+0000","last_undegraded":"2024-03-15T12:56:26.423245+0000","last_fullsized":"2024-03-15T12:56:26.423245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:51:47.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:51:47.093558+0000","last_clean_scrub_stamp":"2024-03-15T12:51:47.093558+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:30:57.264886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.849345+0000","last_change":"2024-03-15T12:56:25.443212+0000","last_active":"2024-03-15T12:56:26.849345+0000","last_peered":"2024-03-15T12:56:26.849345+0000","last_clean":"2024-03-15T12:56:26.849345+0000","last_became_active":"2024-03-15T12:56:25.442787+0000","last_became_peered":"2024-03-15T12:56:25.442787+0000","last_unstale":"2024-03-15T12:56:26.849345+0000","last_undegraded":"2024-03-15T12:56:26.849345+0000","last_fullsized":"2024-03-15T12:56:26.849345+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:59:32.244953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349478+0000","last_change":"2024-03-15T12:56:21.212421+0000","last_active":"2024-03-15T12:56:27.349478+0000","last_peered":"2024-03-15T12:56:27.349478+0000","last_clean":"2024-03-15T12:56:27.349478+0000","last_became_active":"2024-03-15T12:56:21.212095+0000","last_became_peered":"2024-03-15T12:56:21.212095+0000","last_unstale":"2024-03-15T12:56:27.349478+0000","last_undegraded":"2024-03-15T12:56:27.349478+0000","last_fullsized":"2024-03-15T12:56:27.349478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:47:10.397511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423369+0000","last_change":"2024-03-15T12:56:20.210466+0000","last_active":"2024-03-15T12:56:26.423369+0000","last_peered":"2024-03-15T12:56:26.423369+0000","last_clean":"2024-03-15T12:56:26.423369+0000","last_became_active":"2024-03-15T12:56:20.210157+0000","last_became_peered":"2024-03-15T12:56:20.210157+0000","last_unstale":"2024-03-15T12:56:26.423369+0000","last_undegraded":"2024-03-15T12:56:26.423369+0000","last_fullsized":"2024-03-15T12:56:26.423369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:58:03.335826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580639+0000","last_change":"2024-03-15T12:56:25.432641+0000","last_active":"2024-03-15T12:56:27.580639+0000","last_peered":"2024-03-15T12:56:27.580639+0000","last_clean":"2024-03-15T12:56:27.580639+0000","last_became_active":"2024-03-15T12:56:25.432212+0000","last_became_peered":"2024-03-15T12:56:25.432212+0000","last_unstale":"2024-03-15T12:56:27.580639+0000","last_undegraded":"2024-03-15T12:56:27.580639+0000","last_fullsized":"2024-03-15T12:56:27.580639+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:40:20.447886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350069+0000","last_change":"2024-03-15T12:56:20.217693+0000","last_active":"2024-03-15T12:56:27.350069+0000","last_peered":"2024-03-15T12:56:27.350069+0000","last_clean":"2024-03-15T12:56:27.350069+0000","last_became_active":"2024-03-15T12:56:20.217335+0000","last_became_peered":"2024-03-15T12:56:20.217335+0000","last_unstale":"2024-03-15T12:56:27.350069+0000","last_undegraded":"2024-03-15T12:56:27.350069+0000","last_fullsized":"2024-03-15T12:56:27.350069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:05:23.500578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350527+0000","last_change":"2024-03-15T12:56:21.207577+0000","last_active":"2024-03-15T12:56:27.350527+0000","last_peered":"2024-03-15T12:56:27.350527+0000","last_clean":"2024-03-15T12:56:27.350527+0000","last_became_active":"2024-03-15T12:56:21.207220+0000","last_became_peered":"2024-03-15T12:56:21.207220+0000","last_unstale":"2024-03-15T12:56:27.350527+0000","last_undegraded":"2024-03-15T12:56:27.350527+0000","last_fullsized":"2024-03-15T12:56:27.350527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:42:12.104220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349448+0000","last_change":"2024-03-15T12:56:25.439762+0000","last_active":"2024-03-15T12:56:27.349448+0000","last_peered":"2024-03-15T12:56:27.349448+0000","last_clean":"2024-03-15T12:56:27.349448+0000","last_became_active":"2024-03-15T12:56:25.439366+0000","last_became_peered":"2024-03-15T12:56:25.439366+0000","last_unstale":"2024-03-15T12:56:27.349448+0000","last_undegraded":"2024-03-15T12:56:27.349448+0000","last_fullsized":"2024-03-15T12:56:27.349448+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:36:29.638449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423528+0000","last_change":"2024-03-15T12:56:20.218438+0000","last_active":"2024-03-15T12:56:26.423528+0000","last_peered":"2024-03-15T12:56:26.423528+0000","last_clean":"2024-03-15T12:56:26.423528+0000","last_became_active":"2024-03-15T12:56:20.218071+0000","last_became_peered":"2024-03-15T12:56:20.218071+0000","last_unstale":"2024-03-15T12:56:26.423528+0000","last_undegraded":"2024-03-15T12:56:26.423528+0000","last_fullsized":"2024-03-15T12:56:26.423528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:24:16.295781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351246+0000","last_change":"2024-03-15T12:56:21.218345+0000","last_active":"2024-03-15T12:56:27.351246+0000","last_peered":"2024-03-15T12:56:27.351246+0000","last_clean":"2024-03-15T12:56:27.351246+0000","last_became_active":"2024-03-15T12:56:21.216534+0000","last_became_peered":"2024-03-15T12:56:21.216534+0000","last_unstale":"2024-03-15T12:56:27.351246+0000","last_undegraded":"2024-03-15T12:56:27.351246+0000","last_fullsized":"2024-03-15T12:56:27.351246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:56:18.977509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423137+0000","last_change":"2024-03-15T12:56:25.435714+0000","last_active":"2024-03-15T12:56:26.423137+0000","last_peered":"2024-03-15T12:56:26.423137+0000","last_clean":"2024-03-15T12:56:26.423137+0000","last_became_active":"2024-03-15T12:56:25.435315+0000","last_became_peered":"2024-03-15T12:56:25.435315+0000","last_unstale":"2024-03-15T12:56:26.423137+0000","last_undegraded":"2024-03-15T12:56:26.423137+0000","last_fullsized":"2024-03-15T12:56:26.423137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:10:03.509078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350956+0000","last_change":"2024-03-15T12:56:20.209356+0000","last_active":"2024-03-15T12:56:27.350956+0000","last_peered":"2024-03-15T12:56:27.350956+0000","last_clean":"2024-03-15T12:56:27.350956+0000","last_became_active":"2024-03-15T12:56:20.208995+0000","last_became_peered":"2024-03-15T12:56:20.208995+0000","last_unstale":"2024-03-15T12:56:27.350956+0000","last_undegraded":"2024-03-15T12:56:27.350956+0000","last_fullsized":"2024-03-15T12:56:27.350956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:14:08.554567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351946+0000","last_change":"2024-03-15T12:56:21.216315+0000","last_active":"2024-03-15T12:56:27.351946+0000","last_peered":"2024-03-15T12:56:27.351946+0000","last_clean":"2024-03-15T12:56:27.351946+0000","last_became_active":"2024-03-15T12:56:21.215943+0000","last_became_peered":"2024-03-15T12:56:21.215943+0000","last_unstale":"2024-03-15T12:56:27.351946+0000","last_undegraded":"2024-03-15T12:56:27.351946+0000","last_fullsized":"2024-03-15T12:56:27.351946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:47:29.913048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350590+0000","last_change":"2024-03-15T12:56:25.437200+0000","last_active":"2024-03-15T12:56:27.350590+0000","last_peered":"2024-03-15T12:56:27.350590+0000","last_clean":"2024-03-15T12:56:27.350590+0000","last_became_active":"2024-03-15T12:56:25.436786+0000","last_became_peered":"2024-03-15T12:56:25.436786+0000","last_unstale":"2024-03-15T12:56:27.350590+0000","last_undegraded":"2024-03-15T12:56:27.350590+0000","last_fullsized":"2024-03-15T12:56:27.350590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:12:00.286749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:39.834469+0000","last_change":"2024-03-15T12:56:20.202240+0000","last_active":"2024-03-15T12:56:39.834469+0000","last_peered":"2024-03-15T12:56:39.834469+0000","last_clean":"2024-03-15T12:56:39.834469+0000","last_became_active":"2024-03-15T12:56:20.201136+0000","last_became_peered":"2024-03-15T12:56:20.201136+0000","last_unstale":"2024-03-15T12:56:39.834469+0000","last_undegraded":"2024-03-15T12:56:39.834469+0000","last_fullsized":"2024-03-15T12:56:39.834469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:30:35.901237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349546+0000","last_change":"2024-03-15T12:56:21.212095+0000","last_active":"2024-03-15T12:56:27.349546+0000","last_peered":"2024-03-15T12:56:27.349546+0000","last_clean":"2024-03-15T12:56:27.349546+0000","last_became_active":"2024-03-15T12:56:21.211744+0000","last_became_peered":"2024-03-15T12:56:21.211744+0000","last_unstale":"2024-03-15T12:56:27.349546+0000","last_undegraded":"2024-03-15T12:56:27.349546+0000","last_fullsized":"2024-03-15T12:56:27.349546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:27:58.412513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423662+0000","last_change":"2024-03-15T12:56:25.448857+0000","last_active":"2024-03-15T12:56:26.423662+0000","last_peered":"2024-03-15T12:56:26.423662+0000","last_clean":"2024-03-15T12:56:26.423662+0000","last_became_active":"2024-03-15T12:56:25.448461+0000","last_became_peered":"2024-03-15T12:56:25.448461+0000","last_unstale":"2024-03-15T12:56:26.423662+0000","last_undegraded":"2024-03-15T12:56:26.423662+0000","last_fullsized":"2024-03-15T12:56:26.423662+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:21:36.426729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349430+0000","last_change":"2024-03-15T12:56:20.217319+0000","last_active":"2024-03-15T12:56:27.349430+0000","last_peered":"2024-03-15T12:56:27.349430+0000","last_clean":"2024-03-15T12:56:27.349430+0000","last_became_active":"2024-03-15T12:56:20.215450+0000","last_became_peered":"2024-03-15T12:56:20.215450+0000","last_unstale":"2024-03-15T12:56:27.349430+0000","last_undegraded":"2024-03-15T12:56:27.349430+0000","last_fullsized":"2024-03-15T12:56:27.349430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:07:09.085860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580383+0000","last_change":"2024-03-15T12:56:21.209360+0000","last_active":"2024-03-15T12:56:27.580383+0000","last_peered":"2024-03-15T12:56:27.580383+0000","last_clean":"2024-03-15T12:56:27.580383+0000","last_became_active":"2024-03-15T12:56:21.208987+0000","last_became_peered":"2024-03-15T12:56:21.208987+0000","last_unstale":"2024-03-15T12:56:27.580383+0000","last_undegraded":"2024-03-15T12:56:27.580383+0000","last_fullsized":"2024-03-15T12:56:27.580383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:19:13.353400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351291+0000","last_change":"2024-03-15T12:56:25.439173+0000","last_active":"2024-03-15T12:56:27.351291+0000","last_peered":"2024-03-15T12:56:27.351291+0000","last_clean":"2024-03-15T12:56:27.351291+0000","last_became_active":"2024-03-15T12:56:25.438812+0000","last_became_peered":"2024-03-15T12:56:25.438812+0000","last_unstale":"2024-03-15T12:56:27.351291+0000","last_undegraded":"2024-03-15T12:56:27.351291+0000","last_fullsized":"2024-03-15T12:56:27.351291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:10:39.107190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423123+0000","last_change":"2024-03-15T12:56:20.211861+0000","last_active":"2024-03-15T12:56:26.423123+0000","last_peered":"2024-03-15T12:56:26.423123+0000","last_clean":"2024-03-15T12:56:26.423123+0000","last_became_active":"2024-03-15T12:56:20.211481+0000","last_became_peered":"2024-03-15T12:56:20.211481+0000","last_unstale":"2024-03-15T12:56:26.423123+0000","last_undegraded":"2024-03-15T12:56:26.423123+0000","last_fullsized":"2024-03-15T12:56:26.423123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:12:45.558423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350332+0000","last_change":"2024-03-15T12:56:21.216058+0000","last_active":"2024-03-15T12:56:27.350332+0000","last_peered":"2024-03-15T12:56:27.350332+0000","last_clean":"2024-03-15T12:56:27.350332+0000","last_became_active":"2024-03-15T12:56:21.215706+0000","last_became_peered":"2024-03-15T12:56:21.215706+0000","last_unstale":"2024-03-15T12:56:27.350332+0000","last_undegraded":"2024-03-15T12:56:27.350332+0000","last_fullsized":"2024-03-15T12:56:27.350332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:55:02.132976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350367+0000","last_change":"2024-03-15T12:56:25.444134+0000","last_active":"2024-03-15T12:56:27.350367+0000","last_peered":"2024-03-15T12:56:27.350367+0000","last_clean":"2024-03-15T12:56:27.350367+0000","last_became_active":"2024-03-15T12:56:25.443452+0000","last_became_peered":"2024-03-15T12:56:25.443452+0000","last_unstale":"2024-03-15T12:56:27.350367+0000","last_undegraded":"2024-03-15T12:56:27.350367+0000","last_fullsized":"2024-03-15T12:56:27.350367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:06:39.660016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349667+0000","last_change":"2024-03-15T12:56:20.214700+0000","last_active":"2024-03-15T12:56:27.349667+0000","last_peered":"2024-03-15T12:56:27.349667+0000","last_clean":"2024-03-15T12:56:27.349667+0000","last_became_active":"2024-03-15T12:56:20.214348+0000","last_became_peered":"2024-03-15T12:56:20.214348+0000","last_unstale":"2024-03-15T12:56:27.349667+0000","last_undegraded":"2024-03-15T12:56:27.349667+0000","last_fullsized":"2024-03-15T12:56:27.349667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:24:37.551182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423507+0000","last_change":"2024-03-15T12:56:21.220746+0000","last_active":"2024-03-15T12:56:26.423507+0000","last_peered":"2024-03-15T12:56:26.423507+0000","last_clean":"2024-03-15T12:56:26.423507+0000","last_became_active":"2024-03-15T12:56:21.220445+0000","last_became_peered":"2024-03-15T12:56:21.220445+0000","last_unstale":"2024-03-15T12:56:26.423507+0000","last_undegraded":"2024-03-15T12:56:26.423507+0000","last_fullsized":"2024-03-15T12:56:26.423507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:58:44.119476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349618+0000","last_change":"2024-03-15T12:56:25.437611+0000","last_active":"2024-03-15T12:56:27.349618+0000","last_peered":"2024-03-15T12:56:27.349618+0000","last_clean":"2024-03-15T12:56:27.349618+0000","last_became_active":"2024-03-15T12:56:25.437195+0000","last_became_peered":"2024-03-15T12:56:25.437195+0000","last_unstale":"2024-03-15T12:56:27.349618+0000","last_undegraded":"2024-03-15T12:56:27.349618+0000","last_fullsized":"2024-03-15T12:56:27.349618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:26:43.558477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423602+0000","last_change":"2024-03-15T12:56:20.208584+0000","last_active":"2024-03-15T12:56:26.423602+0000","last_peered":"2024-03-15T12:56:26.423602+0000","last_clean":"2024-03-15T12:56:26.423602+0000","last_became_active":"2024-03-15T12:56:20.208155+0000","last_became_peered":"2024-03-15T12:56:20.208155+0000","last_unstale":"2024-03-15T12:56:26.423602+0000","last_undegraded":"2024-03-15T12:56:26.423602+0000","last_fullsized":"2024-03-15T12:56:26.423602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:26:13.148273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423884+0000","last_change":"2024-03-15T12:56:21.220421+0000","last_active":"2024-03-15T12:56:26.423884+0000","last_peered":"2024-03-15T12:56:26.423884+0000","last_clean":"2024-03-15T12:56:26.423884+0000","last_became_active":"2024-03-15T12:56:21.220095+0000","last_became_peered":"2024-03-15T12:56:21.220095+0000","last_unstale":"2024-03-15T12:56:26.423884+0000","last_undegraded":"2024-03-15T12:56:26.423884+0000","last_fullsized":"2024-03-15T12:56:26.423884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:35:33.846506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351851+0000","last_change":"2024-03-15T12:56:25.440408+0000","last_active":"2024-03-15T12:56:27.351851+0000","last_peered":"2024-03-15T12:56:27.351851+0000","last_clean":"2024-03-15T12:56:27.351851+0000","last_became_active":"2024-03-15T12:56:25.439965+0000","last_became_peered":"2024-03-15T12:56:25.439965+0000","last_unstale":"2024-03-15T12:56:27.351851+0000","last_undegraded":"2024-03-15T12:56:27.351851+0000","last_fullsized":"2024-03-15T12:56:27.351851+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:12:57.278202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.849290+0000","last_change":"2024-03-15T12:56:20.216320+0000","last_active":"2024-03-15T12:56:26.849290+0000","last_peered":"2024-03-15T12:56:26.849290+0000","last_clean":"2024-03-15T12:56:26.849290+0000","last_became_active":"2024-03-15T12:56:20.215915+0000","last_became_peered":"2024-03-15T12:56:20.215915+0000","last_unstale":"2024-03-15T12:56:26.849290+0000","last_undegraded":"2024-03-15T12:56:26.849290+0000","last_fullsized":"2024-03-15T12:56:26.849290+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:25:45.236269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423496+0000","last_change":"2024-03-15T12:56:21.215822+0000","last_active":"2024-03-15T12:56:26.423496+0000","last_peered":"2024-03-15T12:56:26.423496+0000","last_clean":"2024-03-15T12:56:26.423496+0000","last_became_active":"2024-03-15T12:56:21.215508+0000","last_became_peered":"2024-03-15T12:56:21.215508+0000","last_unstale":"2024-03-15T12:56:26.423496+0000","last_undegraded":"2024-03-15T12:56:26.423496+0000","last_fullsized":"2024-03-15T12:56:26.423496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:56:55.919198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848626+0000","last_change":"2024-03-15T12:56:25.436186+0000","last_active":"2024-03-15T12:56:26.848626+0000","last_peered":"2024-03-15T12:56:26.848626+0000","last_clean":"2024-03-15T12:56:26.848626+0000","last_became_active":"2024-03-15T12:56:25.435586+0000","last_became_peered":"2024-03-15T12:56:25.435586+0000","last_unstale":"2024-03-15T12:56:26.848626+0000","last_undegraded":"2024-03-15T12:56:26.848626+0000","last_fullsized":"2024-03-15T12:56:26.848626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:02:26.850487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350644+0000","last_change":"2024-03-15T12:56:20.217277+0000","last_active":"2024-03-15T12:56:27.350644+0000","last_peered":"2024-03-15T12:56:27.350644+0000","last_clean":"2024-03-15T12:56:27.350644+0000","last_became_active":"2024-03-15T12:56:20.216930+0000","last_became_peered":"2024-03-15T12:56:20.216930+0000","last_unstale":"2024-03-15T12:56:27.350644+0000","last_undegraded":"2024-03-15T12:56:27.350644+0000","last_fullsized":"2024-03-15T12:56:27.350644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:47:51.075545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423705+0000","last_change":"2024-03-15T12:56:21.212066+0000","last_active":"2024-03-15T12:56:26.423705+0000","last_peered":"2024-03-15T12:56:26.423705+0000","last_clean":"2024-03-15T12:56:26.423705+0000","last_became_active":"2024-03-15T12:56:21.211650+0000","last_became_peered":"2024-03-15T12:56:21.211650+0000","last_unstale":"2024-03-15T12:56:26.423705+0000","last_undegraded":"2024-03-15T12:56:26.423705+0000","last_fullsized":"2024-03-15T12:56:26.423705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:40:35.322210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580573+0000","last_change":"2024-03-15T12:56:21.212794+0000","last_active":"2024-03-15T12:56:27.580573+0000","last_peered":"2024-03-15T12:56:27.580573+0000","last_clean":"2024-03-15T12:56:27.580573+0000","last_became_active":"2024-03-15T12:56:21.212470+0000","last_became_peered":"2024-03-15T12:56:21.212470+0000","last_unstale":"2024-03-15T12:56:27.580573+0000","last_undegraded":"2024-03-15T12:56:27.580573+0000","last_fullsized":"2024-03-15T12:56:27.580573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:10:04.614223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423286+0000","last_change":"2024-03-15T12:56:21.211746+0000","last_active":"2024-03-15T12:56:26.423286+0000","last_peered":"2024-03-15T12:56:26.423286+0000","last_clean":"2024-03-15T12:56:26.423286+0000","last_became_active":"2024-03-15T12:56:21.211431+0000","last_became_peered":"2024-03-15T12:56:21.211431+0000","last_unstale":"2024-03-15T12:56:26.423286+0000","last_undegraded":"2024-03-15T12:56:26.423286+0000","last_fullsized":"2024-03-15T12:56:26.423286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:46:31.188408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423138+0000","last_change":"2024-03-15T12:56:21.220997+0000","last_active":"2024-03-15T12:56:26.423138+0000","last_peered":"2024-03-15T12:56:26.423138+0000","last_clean":"2024-03-15T12:56:26.423138+0000","last_became_active":"2024-03-15T12:56:21.220644+0000","last_became_peered":"2024-03-15T12:56:21.220644+0000","last_unstale":"2024-03-15T12:56:26.423138+0000","last_undegraded":"2024-03-15T12:56:26.423138+0000","last_fullsized":"2024-03-15T12:56:26.423138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:47:55.852845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351539+0000","last_change":"2024-03-15T12:56:21.211634+0000","last_active":"2024-03-15T12:56:27.351539+0000","last_peered":"2024-03-15T12:56:27.351539+0000","last_clean":"2024-03-15T12:56:27.351539+0000","last_became_active":"2024-03-15T12:56:21.211303+0000","last_became_peered":"2024-03-15T12:56:21.211303+0000","last_unstale":"2024-03-15T12:56:27.351539+0000","last_undegraded":"2024-03-15T12:56:27.351539+0000","last_fullsized":"2024-03-15T12:56:27.351539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:22:32.358270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.350501+0000","last_change":"2024-03-15T12:56:21.211673+0000","last_active":"2024-03-15T12:56:27.350501+0000","last_peered":"2024-03-15T12:56:27.350501+0000","last_clean":"2024-03-15T12:56:27.350501+0000","last_became_active":"2024-03-15T12:56:21.211296+0000","last_became_peered":"2024-03-15T12:56:21.211296+0000","last_unstale":"2024-03-15T12:56:27.350501+0000","last_undegraded":"2024-03-15T12:56:27.350501+0000","last_fullsized":"2024-03-15T12:56:27.350501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:40:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580787+0000","last_change":"2024-03-15T12:56:21.208681+0000","last_active":"2024-03-15T12:56:27.580787+0000","last_peered":"2024-03-15T12:56:27.580787+0000","last_clean":"2024-03-15T12:56:27.580787+0000","last_became_active":"2024-03-15T12:56:21.208130+0000","last_became_peered":"2024-03-15T12:56:21.208130+0000","last_unstale":"2024-03-15T12:56:27.580787+0000","last_undegraded":"2024-03-15T12:56:27.580787+0000","last_fullsized":"2024-03-15T12:56:27.580787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:34:52.971485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.423883+0000","last_change":"2024-03-15T12:56:21.216979+0000","last_active":"2024-03-15T12:56:26.423883+0000","last_peered":"2024-03-15T12:56:26.423883+0000","last_clean":"2024-03-15T12:56:26.423883+0000","last_became_active":"2024-03-15T12:56:21.216632+0000","last_became_peered":"2024-03-15T12:56:21.216632+0000","last_unstale":"2024-03-15T12:56:26.423883+0000","last_undegraded":"2024-03-15T12:56:26.423883+0000","last_fullsized":"2024-03-15T12:56:26.423883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:21:57.213512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.351438+0000","last_change":"2024-03-15T12:56:21.223064+0000","last_active":"2024-03-15T12:56:27.351438+0000","last_peered":"2024-03-15T12:56:27.351438+0000","last_clean":"2024-03-15T12:56:27.351438+0000","last_became_active":"2024-03-15T12:56:21.220882+0000","last_became_peered":"2024-03-15T12:56:21.220882+0000","last_unstale":"2024-03-15T12:56:27.351438+0000","last_undegraded":"2024-03-15T12:56:27.351438+0000","last_fullsized":"2024-03-15T12:56:27.351438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:45:02.930154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:26.848703+0000","last_change":"2024-03-15T12:56:21.218614+0000","last_active":"2024-03-15T12:56:26.848703+0000","last_peered":"2024-03-15T12:56:26.848703+0000","last_clean":"2024-03-15T12:56:26.848703+0000","last_became_active":"2024-03-15T12:56:21.218224+0000","last_became_peered":"2024-03-15T12:56:21.218224+0000","last_unstale":"2024-03-15T12:56:26.848703+0000","last_undegraded":"2024-03-15T12:56:26.848703+0000","last_fullsized":"2024-03-15T12:56:26.848703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:41:29.790358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349429+0000","last_change":"2024-03-15T12:56:21.219019+0000","last_active":"2024-03-15T12:56:27.349429+0000","last_peered":"2024-03-15T12:56:27.349429+0000","last_clean":"2024-03-15T12:56:27.349429+0000","last_became_active":"2024-03-15T12:56:21.218673+0000","last_became_peered":"2024-03-15T12:56:21.218673+0000","last_unstale":"2024-03-15T12:56:27.349429+0000","last_undegraded":"2024-03-15T12:56:27.349429+0000","last_fullsized":"2024-03-15T12:56:27.349429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:42:11.788598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349737+0000","last_change":"2024-03-15T12:56:21.220355+0000","last_active":"2024-03-15T12:56:27.349737+0000","last_peered":"2024-03-15T12:56:27.349737+0000","last_clean":"2024-03-15T12:56:27.349737+0000","last_became_active":"2024-03-15T12:56:21.220021+0000","last_became_peered":"2024-03-15T12:56:21.220021+0000","last_unstale":"2024-03-15T12:56:27.349737+0000","last_undegraded":"2024-03-15T12:56:27.349737+0000","last_fullsized":"2024-03-15T12:56:27.349737+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:40:00.947351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.349887+0000","last_change":"2024-03-15T12:56:21.213050+0000","last_active":"2024-03-15T12:56:27.349887+0000","last_peered":"2024-03-15T12:56:27.349887+0000","last_clean":"2024-03-15T12:56:27.349887+0000","last_became_active":"2024-03-15T12:56:21.212730+0000","last_became_peered":"2024-03-15T12:56:21.212730+0000","last_unstale":"2024-03-15T12:56:27.349887+0000","last_undegraded":"2024-03-15T12:56:27.349887+0000","last_fullsized":"2024-03-15T12:56:27.349887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:56:36.396285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-15T12:56:27.580559+0000","last_change":"2024-03-15T12:56:21.213942+0000","last_active":"2024-03-15T12:56:27.580559+0000","last_peered":"2024-03-15T12:56:27.580559+0000","last_clean":"2024-03-15T12:56:27.580559+0000","last_became_active":"2024-03-15T12:56:21.213611+0000","last_became_peered":"2024-03-15T12:56:21.213611+0000","last_unstale":"2024-03-15T12:56:27.580559+0000","last_undegraded":"2024-03-15T12:56:27.580559+0000","last_fullsized":"2024-03-15T12:56:27.580559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:23:44.002970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4730,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":16088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":476829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1601536,"data_stored":1071233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":987136,"data_stored":477411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":477433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36892,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708260,"statfs":{"total":95995035648,"available":95957258240,"internally_reserved":0,"allocated":1581056,"data_stored":1067779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":983040,"data_stored":476829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":480671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":987136,"data_stored":476919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":480705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":476933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":476851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1589248,"data_stored":1067379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-15T12:56:44.563 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-15T12:56:44.563 INFO:tasks.cephadm:Setting up client nodes... 2024-03-15T12:56:44.564 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-15T12:56:44.718 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:44 smithi149 ceph-mon[35927]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:56:44.718 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:44 smithi149 ceph-mon[35927]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:56:44.718 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:44 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] up:active 2024-03-15T12:56:44.718 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:44 smithi149 ceph-mon[35927]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-03-15T12:56:44.718 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:44 smithi149 ceph-mon[35927]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-15T12:56:44.718 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:44 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T12:56:44.718 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:44 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3113032203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:44.718 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:44 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2023522141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:44.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:44 smithi184 ceph-mon[35993]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:56:44.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:44 smithi184 ceph-mon[35993]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:56:44.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:44 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] up:active 2024-03-15T12:56:44.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:44 smithi184 ceph-mon[35993]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-03-15T12:56:44.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:44 smithi184 ceph-mon[35993]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-15T12:56:44.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:44 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T12:56:44.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:44 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3113032203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:56:44.810 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:44 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2023522141' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:44.842 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:45.110 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 12:56:44 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-b[65861]: 2024-03-15T12:56:44.731+0000 7fb0fe4d8700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-15T12:56:45.110 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 12:56:44 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-b[65861]: 2024-03-15T12:56:44.731+0000 7fb0fe4d8700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-15T12:56:45.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:45 smithi002 ceph-mon[32060]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:56:45.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:45 smithi002 ceph-mon[32060]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:56:45.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:45 smithi002 ceph-mon[32060]: pgmap v302: 129 pgs: 129 active+clean; 586 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-03-15T12:56:45.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:45 smithi184 ceph-mon[35993]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:56:45.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:45 smithi184 ceph-mon[35993]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:56:45.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:45 smithi184 ceph-mon[35993]: pgmap v302: 129 pgs: 129 active+clean; 586 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-03-15T12:56:45.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:45 smithi149 ceph-mon[35927]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:56:45.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:45 smithi149 ceph-mon[35927]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:56:45.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:45 smithi149 ceph-mon[35927]: pgmap v302: 129 pgs: 129 active+clean; 586 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-03-15T12:56:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.575+0000 7fa14821a700 1 -- 172.21.15.2:0/320107351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa140103230 msgr2=0x7fa140103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.575+0000 7fa14821a700 1 --2- 172.21.15.2:0/320107351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa140103230 0x7fa140103690 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa1300099c0 tx=0x7fa130009cd0 comp rx=0 tx=0).stop 2024-03-15T12:56:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.575+0000 7fa14821a700 1 -- 172.21.15.2:0/320107351 shutdown_connections 2024-03-15T12:56:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.575+0000 7fa14821a700 1 --2- 172.21.15.2:0/320107351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa140103bd0 0x7fa14010a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.575+0000 7fa14821a700 1 --2- 172.21.15.2:0/320107351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa140102030 0x7fa140102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.575+0000 7fa14821a700 1 --2- 172.21.15.2:0/320107351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa140103230 0x7fa140103690 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.575+0000 7fa14821a700 1 -- 172.21.15.2:0/320107351 >> 172.21.15.2:0/320107351 conn(0x7fa1400fd5f0 msgr2=0x7fa1400ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:46.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.575+0000 7fa14821a700 1 -- 172.21.15.2:0/320107351 shutdown_connections 2024-03-15T12:56:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.575+0000 7fa14821a700 1 -- 172.21.15.2:0/320107351 wait complete. 2024-03-15T12:56:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.575+0000 7fa14821a700 1 Processor -- start 2024-03-15T12:56:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.575+0000 7fa14821a700 1 -- start start 2024-03-15T12:56:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.575+0000 7fa14821a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa140102030 0x7fa140197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.575+0000 7fa14821a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa140103230 0x7fa140198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.575+0000 7fa14821a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa140103bd0 0x7fa14019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa14821a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14019fa50 con 0x7fa140102030 2024-03-15T12:56:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa14821a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14019fb90 con 0x7fa140103230 2024-03-15T12:56:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa14821a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14019fd20 con 0x7fa140103bd0 2024-03-15T12:56:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa145fb6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa140102030 0x7fa140197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa145fb6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa140102030 0x7fa140197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41568/0 (socket says 172.21.15.2:41568) 2024-03-15T12:56:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa145fb6700 1 -- 172.21.15.2:0/3067006492 learned_addr learned my addr 172.21.15.2:0/3067006492 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa1467b7700 1 --2- 172.21.15.2:0/3067006492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa140103bd0 0x7fa14019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa1457b5700 1 --2- 172.21.15.2:0/3067006492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa140103230 0x7fa140198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:46.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa145fb6700 1 -- 172.21.15.2:0/3067006492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa140103bd0 msgr2=0x7fa14019f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa145fb6700 1 --2- 172.21.15.2:0/3067006492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa140103bd0 0x7fa14019f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa145fb6700 1 -- 172.21.15.2:0/3067006492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa140103230 msgr2=0x7fa140198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa145fb6700 1 --2- 172.21.15.2:0/3067006492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa140103230 0x7fa140198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa145fb6700 1 -- 172.21.15.2:0/3067006492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1300095f0 con 0x7fa140102030 2024-03-15T12:56:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa145fb6700 1 --2- 172.21.15.2:0/3067006492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa140102030 0x7fa140197ec0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00e9a0 tx=0x7fa13c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa136ffd700 1 -- 172.21.15.2:0/3067006492 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c009790 con 0x7fa140102030 2024-03-15T12:56:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa14821a700 1 -- 172.21.15.2:0/3067006492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14019ffa0 con 0x7fa140102030 2024-03-15T12:56:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa136ffd700 1 -- 172.21.15.2:0/3067006492 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa13c00c810 con 0x7fa140102030 2024-03-15T12:56:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa136ffd700 1 -- 172.21.15.2:0/3067006492 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c003ce0 con 0x7fa140102030 2024-03-15T12:56:46.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.576+0000 7fa14821a700 1 -- 172.21.15.2:0/3067006492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1401a0410 con 0x7fa140102030 2024-03-15T12:56:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.577+0000 7fa14821a700 1 -- 172.21.15.2:0/3067006492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1400504f0 con 0x7fa140102030 2024-03-15T12:56:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.578+0000 7fa136ffd700 1 -- 172.21.15.2:0/3067006492 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7fa13c00c980 con 0x7fa140102030 2024-03-15T12:56:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.579+0000 7fa136ffd700 1 --2- 172.21.15.2:0/3067006492 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa12c0a1c00 0x7fa12c0a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.579+0000 7fa1457b5700 1 --2- 172.21.15.2:0/3067006492 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa12c0a1c00 0x7fa12c0a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.579+0000 7fa136ffd700 1 -- 172.21.15.2:0/3067006492 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa13c014070 con 0x7fa140102030 2024-03-15T12:56:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.579+0000 7fa1457b5700 1 --2- 172.21.15.2:0/3067006492 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa12c0a1c00 0x7fa12c0a40c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa13000b580 tx=0x7fa130005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:46.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.583+0000 7fa136ffd700 1 -- 172.21.15.2:0/3067006492 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa13c096810 con 0x7fa140102030 2024-03-15T12:56:46.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.871+0000 7fa14821a700 1 -- 172.21.15.2:0/3067006492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fa1401093f0 con 0x7fa140102030 2024-03-15T12:56:46.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.873+0000 7fa136ffd700 1 -- 172.21.15.2:0/3067006492 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7fa13c099ed0 con 0x7fa140102030 2024-03-15T12:56:46.860 INFO:teuthology.orchestra.run.smithi002.stdout:[client.0] 2024-03-15T12:56:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCORfRlEw4MNBAA7csHsGjK/pxRr7XTXwSkAw== 2024-03-15T12:56:46.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.875+0000 7fa14821a700 1 -- 172.21.15.2:0/3067006492 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa12c0a1c00 msgr2=0x7fa12c0a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:46.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.875+0000 7fa14821a700 1 --2- 172.21.15.2:0/3067006492 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa12c0a1c00 0x7fa12c0a40c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa13000b580 tx=0x7fa130005fb0 comp rx=0 tx=0).stop 2024-03-15T12:56:46.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.876+0000 7fa14821a700 1 -- 172.21.15.2:0/3067006492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa140102030 msgr2=0x7fa140197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:46.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.876+0000 7fa14821a700 1 --2- 172.21.15.2:0/3067006492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa140102030 0x7fa140197ec0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00e9a0 tx=0x7fa13c00ecb0 comp rx=0 tx=0).stop 2024-03-15T12:56:46.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.876+0000 7fa14821a700 1 -- 172.21.15.2:0/3067006492 shutdown_connections 2024-03-15T12:56:46.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.876+0000 7fa14821a700 1 --2- 172.21.15.2:0/3067006492 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa12c0a1c00 0x7fa12c0a40c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:46.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.876+0000 7fa14821a700 1 --2- 172.21.15.2:0/3067006492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa140102030 0x7fa140197ec0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:46.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.876+0000 7fa14821a700 1 --2- 172.21.15.2:0/3067006492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa140103bd0 0x7fa14019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:46.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.876+0000 7fa14821a700 1 --2- 172.21.15.2:0/3067006492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa140103230 0x7fa140198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:46.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.876+0000 7fa14821a700 1 -- 172.21.15.2:0/3067006492 >> 172.21.15.2:0/3067006492 conn(0x7fa1400fd5f0 msgr2=0x7fa140106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:46.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.876+0000 7fa14821a700 1 -- 172.21.15.2:0/3067006492 shutdown_connections 2024-03-15T12:56:46.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:46.876+0000 7fa14821a700 1 -- 172.21.15.2:0/3067006492 wait complete. 2024-03-15T12:56:47.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:46 smithi002 ceph-mon[32060]: pgmap v303: 129 pgs: 129 active+clean; 628 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.7 KiB/s wr, 20 op/s 2024-03-15T12:56:47.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:46 smithi184 ceph-mon[35993]: pgmap v303: 129 pgs: 129 active+clean; 628 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.7 KiB/s wr, 20 op/s 2024-03-15T12:56:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:46 smithi149 ceph-mon[35927]: pgmap v303: 129 pgs: 129 active+clean; 628 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.7 KiB/s wr, 20 op/s 2024-03-15T12:56:47.634 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T12:56:47.635 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-15T12:56:47.635 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-15T12:56:47.681 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-15T12:56:47.681 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-15T12:56:47.681 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-15T12:56:47.969 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:48.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:47 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3067006492' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-15T12:56:48.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:47 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3067006492' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-15T12:56:48.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:47 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3067006492' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-15T12:56:48.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:47 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3067006492' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-15T12:56:48.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:47 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3067006492' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-15T12:56:48.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:47 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3067006492' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-15T12:56:49.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:48 smithi002 ceph-mon[32060]: pgmap v304: 129 pgs: 129 active+clean; 628 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.5 KiB/s wr, 21 op/s 2024-03-15T12:56:49.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:48 smithi184 ceph-mon[35993]: pgmap v304: 129 pgs: 129 active+clean; 628 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.5 KiB/s wr, 21 op/s 2024-03-15T12:56:49.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:48 smithi149 ceph-mon[35927]: pgmap v304: 129 pgs: 129 active+clean; 628 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.5 KiB/s wr, 21 op/s 2024-03-15T12:56:49.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.726+0000 7f00d75bc700 1 -- 172.21.15.2:0/2622971906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00d0101130 msgr2=0x7f00d0101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:49.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.726+0000 7f00d75bc700 1 --2- 172.21.15.2:0/2622971906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00d0101130 0x7f00d0101510 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009990 tx=0x7f00cc009ca0 comp rx=0 tx=0).stop 2024-03-15T12:56:49.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.726+0000 7f00d75bc700 1 -- 172.21.15.2:0/2622971906 shutdown_connections 2024-03-15T12:56:49.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.726+0000 7f00d75bc700 1 --2- 172.21.15.2:0/2622971906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00d0101130 0x7f00d0101510 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:49.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.726+0000 7f00d75bc700 1 --2- 172.21.15.2:0/2622971906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d01061f0 0x7f00d010ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:49.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.726+0000 7f00d75bc700 1 --2- 172.21.15.2:0/2622971906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00d0101ae0 0x7f00d0105b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:49.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.726+0000 7f00d75bc700 1 -- 172.21.15.2:0/2622971906 >> 172.21.15.2:0/2622971906 conn(0x7f00d0077e50 msgr2=0x7f00d0078260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:49.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.726+0000 7f00d75bc700 1 -- 172.21.15.2:0/2622971906 shutdown_connections 2024-03-15T12:56:49.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.726+0000 7f00d75bc700 1 -- 172.21.15.2:0/2622971906 wait complete. 2024-03-15T12:56:49.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.726+0000 7f00d75bc700 1 Processor -- start 2024-03-15T12:56:49.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d75bc700 1 -- start start 2024-03-15T12:56:49.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d75bc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00d0101130 0x7f00d01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:49.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d75bc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d0101ae0 0x7f00d0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:49.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d75bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00d01061f0 0x7f00d019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:49.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d75bc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d019d3a0 con 0x7f00d01061f0 2024-03-15T12:56:49.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d75bc700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d019d510 con 0x7f00d0101130 2024-03-15T12:56:49.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d75bc700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d019d6d0 con 0x7f00d0101ae0 2024-03-15T12:56:49.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d5b59700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00d01061f0 0x7f00d019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:49.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d4b57700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d0101ae0 0x7f00d0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:49.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d4b57700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d0101ae0 0x7f00d0198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:59558/0 (socket says 172.21.15.2:59558) 2024-03-15T12:56:49.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d4b57700 1 -- 172.21.15.2:0/4026176319 learned_addr learned my addr 172.21.15.2:0/4026176319 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:49.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d5358700 1 --2- 172.21.15.2:0/4026176319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00d0101130 0x7f00d01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:49.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d5b59700 1 -- 172.21.15.2:0/4026176319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d0101ae0 msgr2=0x7f00d0198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:49.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d5b59700 1 --2- 172.21.15.2:0/4026176319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d0101ae0 0x7f00d0198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:49.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d5b59700 1 -- 172.21.15.2:0/4026176319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00d0101130 msgr2=0x7f00d01984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:49.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d5b59700 1 --2- 172.21.15.2:0/4026176319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00d0101130 0x7f00d01984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:49.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d5b59700 1 -- 172.21.15.2:0/4026176319 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc0095f0 con 0x7f00d01061f0 2024-03-15T12:56:49.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d4b57700 1 --2- 172.21.15.2:0/4026176319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d0101ae0 0x7f00d0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:49.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.727+0000 7f00d5b59700 1 --2- 172.21.15.2:0/4026176319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00d01061f0 0x7f00d019cd90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f00c800efc0 tx=0x7f00c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:49.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.728+0000 7f00c67fc700 1 -- 172.21.15.2:0/4026176319 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c8018070 con 0x7f00d01061f0 2024-03-15T12:56:49.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.728+0000 7f00c67fc700 1 -- 172.21.15.2:0/4026176319 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f00c800f040 con 0x7f00d01061f0 2024-03-15T12:56:49.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.728+0000 7f00c67fc700 1 -- 172.21.15.2:0/4026176319 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c800cd80 con 0x7f00d01061f0 2024-03-15T12:56:49.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.728+0000 7f00d75bc700 1 -- 172.21.15.2:0/4026176319 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d019d950 con 0x7f00d01061f0 2024-03-15T12:56:49.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.728+0000 7f00d75bc700 1 -- 172.21.15.2:0/4026176319 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d01a5210 con 0x7f00d01061f0 2024-03-15T12:56:49.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.728+0000 7f00d75bc700 1 -- 172.21.15.2:0/4026176319 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d01025f0 con 0x7f00d01061f0 2024-03-15T12:56:49.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.730+0000 7f00c67fc700 1 -- 172.21.15.2:0/4026176319 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f00c8003a00 con 0x7f00d01061f0 2024-03-15T12:56:49.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.730+0000 7f00c67fc700 1 --2- 172.21.15.2:0/4026176319 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f00bc0a1b50 0x7f00bc0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:49.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.730+0000 7f00d5358700 1 --2- 172.21.15.2:0/4026176319 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f00bc0a1b50 0x7f00bc0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:49.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.730+0000 7f00c67fc700 1 -- 172.21.15.2:0/4026176319 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f00c80cdca0 con 0x7f00d01061f0 2024-03-15T12:56:49.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.731+0000 7f00d5358700 1 --2- 172.21.15.2:0/4026176319 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f00bc0a1b50 0x7f00bc0a4010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f00cc0098e0 tx=0x7f00cc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:49.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:49.734+0000 7f00c67fc700 1 -- 172.21.15.2:0/4026176319 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f00c8096b40 con 0x7f00d01061f0 2024-03-15T12:56:49.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:50.010+0000 7f00d75bc700 1 -- 172.21.15.2:0/4026176319 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f00d0110f20 con 0x7f00d01061f0 2024-03-15T12:56:50.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:50.015+0000 7f00c67fc700 1 -- 172.21.15.2:0/4026176319 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260649 (secure 0 0 0) 0x7f00c809a200 con 0x7f00d01061f0 2024-03-15T12:56:50.004 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:50.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:50.018+0000 7f00d75bc700 1 -- 172.21.15.2:0/4026176319 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f00bc0a1b50 msgr2=0x7f00bc0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:50.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:50.018+0000 7f00d75bc700 1 --2- 172.21.15.2:0/4026176319 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f00bc0a1b50 0x7f00bc0a4010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f00cc0098e0 tx=0x7f00cc005960 comp rx=0 tx=0).stop 2024-03-15T12:56:50.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:50.018+0000 7f00d75bc700 1 -- 172.21.15.2:0/4026176319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00d01061f0 msgr2=0x7f00d019cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:50.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:50.018+0000 7f00d75bc700 1 --2- 172.21.15.2:0/4026176319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00d01061f0 0x7f00d019cd90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f00c800efc0 tx=0x7f00c800c3a0 comp rx=0 tx=0).stop 2024-03-15T12:56:50.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:50.018+0000 7f00d75bc700 1 -- 172.21.15.2:0/4026176319 shutdown_connections 2024-03-15T12:56:50.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:50.018+0000 7f00d75bc700 1 --2- 172.21.15.2:0/4026176319 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f00bc0a1b50 0x7f00bc0a4010 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:50.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:50.018+0000 7f00d75bc700 1 --2- 172.21.15.2:0/4026176319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00d01061f0 0x7f00d019cd90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:50.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:50.018+0000 7f00d75bc700 1 --2- 172.21.15.2:0/4026176319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00d0101130 0x7f00d01984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:50.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:50.018+0000 7f00d75bc700 1 --2- 172.21.15.2:0/4026176319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00d0101ae0 0x7f00d0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:50.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:50.018+0000 7f00d75bc700 1 -- 172.21.15.2:0/4026176319 >> 172.21.15.2:0/4026176319 conn(0x7f00d0077e50 msgr2=0x7f00d0107370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:50.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:50.018+0000 7f00d75bc700 1 -- 172.21.15.2:0/4026176319 shutdown_connections 2024-03-15T12:56:50.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:50.018+0000 7f00d75bc700 1 -- 172.21.15.2:0/4026176319 wait complete. 2024-03-15T12:56:50.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:50 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/4026176319' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-15T12:56:50.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:50 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/4026176319' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-15T12:56:50.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:50 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/4026176319' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-15T12:56:50.656 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":343748862},{"type":"v1","addr":"172.21.15.2:6801","nonce":343748862}]},"active_addr":"172.21.15.2:6801/343748862","active_change":"2024-03-15T12:49:18.658935+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24118,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.2:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":883225361}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":2211760175}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":3173750072}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":1602727020}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":306258972}]}]} 2024-03-15T12:56:50.661 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-15T12:56:50.661 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-15T12:56:50.662 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-15T12:56:50.907 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:51.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:51 smithi002 ceph-mon[32060]: pgmap v305: 129 pgs: 129 active+clean; 643 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 25 op/s 2024-03-15T12:56:51.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:51 smithi184 ceph-mon[35993]: pgmap v305: 129 pgs: 129 active+clean; 643 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 25 op/s 2024-03-15T12:56:51.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:51 smithi149 ceph-mon[35927]: pgmap v305: 129 pgs: 129 active+clean; 643 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 25 op/s 2024-03-15T12:56:52.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.547+0000 7f9aafb55700 1 -- 172.21.15.2:0/3908657137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9aa8100ef0 msgr2=0x7f9aa810ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:52.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.547+0000 7f9aafb55700 1 --2- 172.21.15.2:0/3908657137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9aa8100ef0 0x7f9aa810ef00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00099c0 tx=0x7f9aa0009cd0 comp rx=0 tx=0).stop 2024-03-15T12:56:52.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.547+0000 7f9aafb55700 1 -- 172.21.15.2:0/3908657137 shutdown_connections 2024-03-15T12:56:52.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.547+0000 7f9aafb55700 1 --2- 172.21.15.2:0/3908657137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9aa8100ef0 0x7f9aa810ef00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:52.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.547+0000 7f9aafb55700 1 --2- 172.21.15.2:0/3908657137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa8108130 0x7f9aa8108510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:52.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.547+0000 7f9aafb55700 1 --2- 172.21.15.2:0/3908657137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa8100530 0x7f9aa81009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:52.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.547+0000 7f9aafb55700 1 -- 172.21.15.2:0/3908657137 >> 172.21.15.2:0/3908657137 conn(0x7f9aa80774b0 msgr2=0x7f9aa80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:52.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.547+0000 7f9aafb55700 1 -- 172.21.15.2:0/3908657137 shutdown_connections 2024-03-15T12:56:52.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.547+0000 7f9aafb55700 1 -- 172.21.15.2:0/3908657137 wait complete. 2024-03-15T12:56:52.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.548+0000 7f9aafb55700 1 Processor -- start 2024-03-15T12:56:52.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.548+0000 7f9aafb55700 1 -- start start 2024-03-15T12:56:52.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.548+0000 7f9aafb55700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa8100530 0x7f9aa8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:52.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.548+0000 7f9aafb55700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9aa8100ef0 0x7f9aa8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:52.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.548+0000 7f9aafb55700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa8108130 0x7f9aa819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:52.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.548+0000 7f9aafb55700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa819d450 con 0x7f9aa8100ef0 2024-03-15T12:56:52.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.548+0000 7f9aafb55700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa819d5c0 con 0x7f9aa8100530 2024-03-15T12:56:52.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.548+0000 7f9aafb55700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa819d780 con 0x7f9aa8108130 2024-03-15T12:56:52.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.548+0000 7f9aad0f0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9aa8100ef0 0x7f9aa8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:52.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.548+0000 7f9aad0f0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9aa8100ef0 0x7f9aa8198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41704/0 (socket says 172.21.15.2:41704) 2024-03-15T12:56:52.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.548+0000 7f9aad0f0700 1 -- 172.21.15.2:0/613186613 learned_addr learned my addr 172.21.15.2:0/613186613 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:52.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.548+0000 7f9aad8f1700 1 --2- 172.21.15.2:0/613186613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa8100530 0x7f9aa8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:52.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.549+0000 7f9aae0f2700 1 --2- 172.21.15.2:0/613186613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa8108130 0x7f9aa819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:52.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.549+0000 7f9aad0f0700 1 -- 172.21.15.2:0/613186613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa8108130 msgr2=0x7f9aa819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:52.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.549+0000 7f9aad0f0700 1 --2- 172.21.15.2:0/613186613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa8108130 0x7f9aa819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:52.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.549+0000 7f9aad0f0700 1 -- 172.21.15.2:0/613186613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa8100530 msgr2=0x7f9aa8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:52.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.549+0000 7f9aad0f0700 1 --2- 172.21.15.2:0/613186613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa8100530 0x7f9aa8198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:52.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.549+0000 7f9aad0f0700 1 -- 172.21.15.2:0/613186613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa00095f0 con 0x7f9aa8100ef0 2024-03-15T12:56:52.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.549+0000 7f9aad8f1700 1 --2- 172.21.15.2:0/613186613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa8100530 0x7f9aa8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:56:52.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.549+0000 7f9aad0f0700 1 --2- 172.21.15.2:0/613186613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9aa8100ef0 0x7f9aa8198ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c00e9a0 tx=0x7f9a9c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:52.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.549+0000 7f9aa6ffd700 1 -- 172.21.15.2:0/613186613 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c009790 con 0x7f9aa8100ef0 2024-03-15T12:56:52.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.549+0000 7f9aa6ffd700 1 -- 172.21.15.2:0/613186613 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9a9c010430 con 0x7f9aa8100ef0 2024-03-15T12:56:52.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.549+0000 7f9aafb55700 1 -- 172.21.15.2:0/613186613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa819da60 con 0x7f9aa8100ef0 2024-03-15T12:56:52.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.549+0000 7f9aa6ffd700 1 -- 172.21.15.2:0/613186613 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c018660 con 0x7f9aa8100ef0 2024-03-15T12:56:52.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.549+0000 7f9aafb55700 1 -- 172.21.15.2:0/613186613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa81a52c0 con 0x7f9aa8100ef0 2024-03-15T12:56:52.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.550+0000 7f9aafb55700 1 -- 172.21.15.2:0/613186613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa810a650 con 0x7f9aa8100ef0 2024-03-15T12:56:52.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.551+0000 7f9aa6ffd700 1 -- 172.21.15.2:0/613186613 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f9a9c0105a0 con 0x7f9aa8100ef0 2024-03-15T12:56:52.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.552+0000 7f9aa6ffd700 1 --2- 172.21.15.2:0/613186613 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9a900a1bd0 0x7f9a900a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:52.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.552+0000 7f9aad8f1700 1 --2- 172.21.15.2:0/613186613 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9a900a1bd0 0x7f9a900a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:52.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.552+0000 7f9aa6ffd700 1 -- 172.21.15.2:0/613186613 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9a9c014070 con 0x7f9aa8100ef0 2024-03-15T12:56:52.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.552+0000 7f9aad8f1700 1 --2- 172.21.15.2:0/613186613 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9a900a1bd0 0x7f9a900a4090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009550 tx=0x7f9a980091f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:52.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.556+0000 7f9aa6ffd700 1 -- 172.21.15.2:0/613186613 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9a9c053280 con 0x7f9aa8100ef0 2024-03-15T12:56:52.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.764+0000 7f9aafb55700 1 -- 172.21.15.2:0/613186613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9aa8068ca0 con 0x7f9aa8100ef0 2024-03-15T12:56:52.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.765+0000 7f9aa6ffd700 1 -- 172.21.15.2:0/613186613 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24045 (secure 0 0 0) 0x7f9a9c096770 con 0x7f9aa8100ef0 2024-03-15T12:56:52.752 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:52.752 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":89,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:56:25.830426+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:56:52.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.767+0000 7f9aafb55700 1 -- 172.21.15.2:0/613186613 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9a900a1bd0 msgr2=0x7f9a900a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:52.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.767+0000 7f9aafb55700 1 --2- 172.21.15.2:0/613186613 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9a900a1bd0 0x7f9a900a4090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009550 tx=0x7f9a980091f0 comp rx=0 tx=0).stop 2024-03-15T12:56:52.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.767+0000 7f9aafb55700 1 -- 172.21.15.2:0/613186613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9aa8100ef0 msgr2=0x7f9aa8198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:52.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.767+0000 7f9aafb55700 1 --2- 172.21.15.2:0/613186613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9aa8100ef0 0x7f9aa8198ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c00e9a0 tx=0x7f9a9c00ecb0 comp rx=0 tx=0).stop 2024-03-15T12:56:52.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.767+0000 7f9aafb55700 1 -- 172.21.15.2:0/613186613 shutdown_connections 2024-03-15T12:56:52.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.767+0000 7f9aafb55700 1 --2- 172.21.15.2:0/613186613 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9a900a1bd0 0x7f9a900a4090 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:52.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.767+0000 7f9aafb55700 1 --2- 172.21.15.2:0/613186613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9aa8108130 0x7f9aa819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:52.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.767+0000 7f9aafb55700 1 --2- 172.21.15.2:0/613186613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa8100530 0x7f9aa8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:52.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.767+0000 7f9aafb55700 1 --2- 172.21.15.2:0/613186613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9aa8100ef0 0x7f9aa8198ab0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:52.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.767+0000 7f9aafb55700 1 -- 172.21.15.2:0/613186613 >> 172.21.15.2:0/613186613 conn(0x7f9aa80774b0 msgr2=0x7f9aa8104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:52.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.768+0000 7f9aafb55700 1 -- 172.21.15.2:0/613186613 shutdown_connections 2024-03-15T12:56:52.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:52.768+0000 7f9aafb55700 1 -- 172.21.15.2:0/613186613 wait complete. 2024-03-15T12:56:53.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:52 smithi002 ceph-mon[32060]: pgmap v306: 129 pgs: 129 active+clean; 643 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 24 op/s 2024-03-15T12:56:53.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:52 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/613186613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:53.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:52 smithi184 ceph-mon[35993]: pgmap v306: 129 pgs: 129 active+clean; 643 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 24 op/s 2024-03-15T12:56:53.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:52 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/613186613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:53.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:52 smithi149 ceph-mon[35927]: pgmap v306: 129 pgs: 129 active+clean; 643 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 24 op/s 2024-03-15T12:56:53.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:52 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/613186613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:53.567 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-15T12:56:53.567 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-15T12:56:53.819 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:55.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:54 smithi002 ceph-mon[32060]: pgmap v307: 129 pgs: 129 active+clean; 643 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 25 op/s 2024-03-15T12:56:55.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:54 smithi184 ceph-mon[35993]: pgmap v307: 129 pgs: 129 active+clean; 643 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 25 op/s 2024-03-15T12:56:55.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:54 smithi149 ceph-mon[35927]: pgmap v307: 129 pgs: 129 active+clean; 643 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 25 op/s 2024-03-15T12:56:55.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.707+0000 7f0de6d3b700 1 -- 172.21.15.2:0/368778572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0de0106200 msgr2=0x7f0de010ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:55.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.707+0000 7f0de6d3b700 1 --2- 172.21.15.2:0/368778572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0de0106200 0x7f0de010ddd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800b0a0 tx=0x7f0dd800b3b0 comp rx=0 tx=0).stop 2024-03-15T12:56:55.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.707+0000 7f0de6d3b700 1 -- 172.21.15.2:0/368778572 shutdown_connections 2024-03-15T12:56:55.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.707+0000 7f0de6d3b700 1 --2- 172.21.15.2:0/368778572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0de0106200 0x7f0de010ddd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:55.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.707+0000 7f0de6d3b700 1 --2- 172.21.15.2:0/368778572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0de0101170 0x7f0de0101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:55.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.707+0000 7f0de6d3b700 1 --2- 172.21.15.2:0/368778572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de0101b20 0x7f0de0105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:55.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.707+0000 7f0de6d3b700 1 -- 172.21.15.2:0/368778572 >> 172.21.15.2:0/368778572 conn(0x7f0de0076f00 msgr2=0x7f0de0077310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:55.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.708+0000 7f0de6d3b700 1 -- 172.21.15.2:0/368778572 shutdown_connections 2024-03-15T12:56:55.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.708+0000 7f0de6d3b700 1 -- 172.21.15.2:0/368778572 wait complete. 2024-03-15T12:56:55.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.708+0000 7f0de6d3b700 1 Processor -- start 2024-03-15T12:56:55.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.708+0000 7f0de6d3b700 1 -- start start 2024-03-15T12:56:55.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0de6d3b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0de0101170 0x7f0de01a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:55.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0de6d3b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0de0101b20 0x7f0de01a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:55.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0de6d3b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de0106200 0x7f0de019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:55.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0de6d3b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de019f3f0 con 0x7f0de0101b20 2024-03-15T12:56:55.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0de6d3b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de019f530 con 0x7f0de0101170 2024-03-15T12:56:55.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0de6d3b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de019f6f0 con 0x7f0de0106200 2024-03-15T12:56:55.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0ddffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0de0101b20 0x7f0de01a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:55.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0de4ad7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0de0101170 0x7f0de01a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:55.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0ddffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0de0101b20 0x7f0de01a4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41716/0 (socket says 172.21.15.2:41716) 2024-03-15T12:56:55.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0ddffff700 1 -- 172.21.15.2:0/1963307634 learned_addr learned my addr 172.21.15.2:0/1963307634 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:56:55.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0de52d8700 1 --2- 172.21.15.2:0/1963307634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de0106200 0x7f0de019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:55.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0ddffff700 1 -- 172.21.15.2:0/1963307634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de0106200 msgr2=0x7f0de019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:55.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0ddffff700 1 --2- 172.21.15.2:0/1963307634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de0106200 0x7f0de019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0ddffff700 1 -- 172.21.15.2:0/1963307634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0de0101170 msgr2=0x7f0de01a4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0ddffff700 1 --2- 172.21.15.2:0/1963307634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0de0101170 0x7f0de01a4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0ddffff700 1 -- 172.21.15.2:0/1963307634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd8009d00 con 0x7f0de0101b20 2024-03-15T12:56:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0ddffff700 1 --2- 172.21.15.2:0/1963307634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0de0101b20 0x7f0de01a4d70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00e970 tx=0x7f0dcc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0dddffb700 1 -- 172.21.15.2:0/1963307634 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc009790 con 0x7f0de0101b20 2024-03-15T12:56:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.709+0000 7f0dddffb700 1 -- 172.21.15.2:0/1963307634 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0dcc00c7e0 con 0x7f0de0101b20 2024-03-15T12:56:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.710+0000 7f0de6d3b700 1 -- 172.21.15.2:0/1963307634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de019f9d0 con 0x7f0de0101b20 2024-03-15T12:56:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.710+0000 7f0dddffb700 1 -- 172.21.15.2:0/1963307634 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc0055e0 con 0x7f0de0101b20 2024-03-15T12:56:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.710+0000 7f0de6d3b700 1 -- 172.21.15.2:0/1963307634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de01ab8d0 con 0x7f0de0101b20 2024-03-15T12:56:55.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.711+0000 7f0de6d3b700 1 -- 172.21.15.2:0/1963307634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de0102600 con 0x7f0de0101b20 2024-03-15T12:56:55.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.712+0000 7f0dddffb700 1 -- 172.21.15.2:0/1963307634 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f0dcc00c950 con 0x7f0de0101b20 2024-03-15T12:56:55.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.712+0000 7f0dddffb700 1 --2- 172.21.15.2:0/1963307634 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0dd00a1b50 0x7f0dd00a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:56:55.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.712+0000 7f0de4ad7700 1 --2- 172.21.15.2:0/1963307634 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0dd00a1b50 0x7f0dd00a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:56:55.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.712+0000 7f0dddffb700 1 -- 172.21.15.2:0/1963307634 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f0dcc014070 con 0x7f0de0101b20 2024-03-15T12:56:55.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.713+0000 7f0de4ad7700 1 --2- 172.21.15.2:0/1963307634 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0dd00a1b50 0x7f0dd00a4010 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0de010e0a0 tx=0x7f0dd4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:56:55.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.717+0000 7f0dddffb700 1 -- 172.21.15.2:0/1963307634 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0dcc0968a0 con 0x7f0de0101b20 2024-03-15T12:56:55.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.919+0000 7f0de6d3b700 1 -- 172.21.15.2:0/1963307634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0de0068ca0 con 0x7f0de0101b20 2024-03-15T12:56:55.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.920+0000 7f0dddffb700 1 -- 172.21.15.2:0/1963307634 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24045 (secure 0 0 0) 0x7f0dcc096020 con 0x7f0de0101b20 2024-03-15T12:56:55.907 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:56:55.907 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":89,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:56:25.830426+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:56:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.923+0000 7f0de6d3b700 1 -- 172.21.15.2:0/1963307634 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0dd00a1b50 msgr2=0x7f0dd00a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.923+0000 7f0de6d3b700 1 --2- 172.21.15.2:0/1963307634 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0dd00a1b50 0x7f0dd00a4010 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0de010e0a0 tx=0x7f0dd4009260 comp rx=0 tx=0).stop 2024-03-15T12:56:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.923+0000 7f0de6d3b700 1 -- 172.21.15.2:0/1963307634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0de0101b20 msgr2=0x7f0de01a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:56:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.923+0000 7f0de6d3b700 1 --2- 172.21.15.2:0/1963307634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0de0101b20 0x7f0de01a4d70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00e970 tx=0x7f0dcc00ec80 comp rx=0 tx=0).stop 2024-03-15T12:56:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.923+0000 7f0de6d3b700 1 -- 172.21.15.2:0/1963307634 shutdown_connections 2024-03-15T12:56:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.923+0000 7f0de6d3b700 1 --2- 172.21.15.2:0/1963307634 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0dd00a1b50 0x7f0dd00a4010 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.923+0000 7f0de6d3b700 1 --2- 172.21.15.2:0/1963307634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de0106200 0x7f0de019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.923+0000 7f0de6d3b700 1 --2- 172.21.15.2:0/1963307634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0de0101170 0x7f0de01a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:55.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.923+0000 7f0de6d3b700 1 --2- 172.21.15.2:0/1963307634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0de0101b20 0x7f0de01a4d70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:56:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.923+0000 7f0de6d3b700 1 -- 172.21.15.2:0/1963307634 >> 172.21.15.2:0/1963307634 conn(0x7f0de0076f00 msgr2=0x7f0de01006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:56:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.923+0000 7f0de6d3b700 1 -- 172.21.15.2:0/1963307634 shutdown_connections 2024-03-15T12:56:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:56:55.923+0000 7f0de6d3b700 1 -- 172.21.15.2:0/1963307634 wait complete. 2024-03-15T12:56:56.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:55 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1963307634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:56.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:55 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1963307634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:56.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:55 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1963307634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:56:56.662 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-15T12:56:56.663 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-15T12:56:56.663 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-15T12:56:56.663 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-15T12:56:56.664 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-15T12:56:56.664 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-15T12:56:56.665 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-15T12:56:56.665 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-15T12:56:56.665 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-15T12:56:56.666 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-15T12:56:56.666 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-15T12:56:56.667 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-15T12:56:57.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:56 smithi002 ceph-mon[32060]: pgmap v308: 129 pgs: 129 active+clean; 643 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 17 op/s 2024-03-15T12:56:57.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:56 smithi184 ceph-mon[35993]: pgmap v308: 129 pgs: 129 active+clean; 643 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 17 op/s 2024-03-15T12:56:57.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:56 smithi149 ceph-mon[35927]: pgmap v308: 129 pgs: 129 active+clean; 643 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 17 op/s 2024-03-15T12:56:57.464 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:57.464 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:57.465 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:57.468 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:57.474 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:57.474 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:57.475 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:57.477 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:57.477 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:56:59.988 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:56:59 smithi002 ceph-mon[32060]: pgmap v309: 129 pgs: 129 active+clean; 643 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-03-15T12:57:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:56:59 smithi184 ceph-mon[35993]: pgmap v309: 129 pgs: 129 active+clean; 643 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-03-15T12:57:00.062 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:56:59 smithi149 ceph-mon[35927]: pgmap v309: 129 pgs: 129 active+clean; 643 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-03-15T12:57:02.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:01 smithi002 ceph-mon[32060]: pgmap v310: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-15T12:57:02.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:01 smithi184 ceph-mon[35993]: pgmap v310: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-15T12:57:02.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:01 smithi149 ceph-mon[35927]: pgmap v310: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-15T12:57:04.045 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:03 smithi002 ceph-mon[32060]: pgmap v311: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-15T12:57:04.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:03 smithi184 ceph-mon[35993]: pgmap v311: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-15T12:57:04.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:03 smithi149 ceph-mon[35927]: pgmap v311: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-15T12:57:06.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:05 smithi002 ceph-mon[32060]: pgmap v312: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-15T12:57:06.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:05 smithi184 ceph-mon[35993]: pgmap v312: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-15T12:57:06.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:05 smithi149 ceph-mon[35927]: pgmap v312: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-15T12:57:08.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:07 smithi002 ceph-mon[32060]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-15T12:57:08.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:07 smithi184 ceph-mon[35993]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-15T12:57:08.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:07 smithi149 ceph-mon[35927]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-15T12:57:08.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.918+0000 7f12453fc700 1 -- 172.21.15.2:0/3410524821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1240101940 msgr2=0x7f1240101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:08.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.918+0000 7f12453fc700 1 --2- 172.21.15.2:0/3410524821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1240101940 0x7f1240101d20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1230009990 tx=0x7f1230009ca0 comp rx=0 tx=0).stop 2024-03-15T12:57:08.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.918+0000 7f12453fc700 1 -- 172.21.15.2:0/3410524821 shutdown_connections 2024-03-15T12:57:08.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.918+0000 7f12453fc700 1 --2- 172.21.15.2:0/3410524821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1240101940 0x7f1240101d20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:08.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.918+0000 7f12453fc700 1 --2- 172.21.15.2:0/3410524821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1240106260 0x7f124010de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:08.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.918+0000 7f12453fc700 1 --2- 172.21.15.2:0/3410524821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12401022f0 0x7f1240105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:08.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.918+0000 7f12453fc700 1 -- 172.21.15.2:0/3410524821 >> 172.21.15.2:0/3410524821 conn(0x7f12400774e0 msgr2=0x7f12400fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:08.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.918+0000 7f12453fc700 1 -- 172.21.15.2:0/3410524821 shutdown_connections 2024-03-15T12:57:08.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.918+0000 7f12453fc700 1 -- 172.21.15.2:0/3410524821 wait complete. 2024-03-15T12:57:08.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.918+0000 7f12453fc700 1 Processor -- start 2024-03-15T12:57:08.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f12453fc700 1 -- start start 2024-03-15T12:57:08.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f12453fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1240101940 0x7f12401985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:08.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f12453fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12401022f0 0x7f1240198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:08.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f12453fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1240106260 0x7f124019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:08.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f12453fc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124019d450 con 0x7f12401022f0 2024-03-15T12:57:08.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f12453fc700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124019d5c0 con 0x7f1240101940 2024-03-15T12:57:08.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f12453fc700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124019d780 con 0x7f1240106260 2024-03-15T12:57:08.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f123effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1240101940 0x7f12401985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:08.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f123e7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12401022f0 0x7f1240198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:08.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f123effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1240101940 0x7f12401985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60166/0 (socket says 172.21.15.2:60166) 2024-03-15T12:57:08.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f123effd700 1 -- 172.21.15.2:0/2488072759 learned_addr learned my addr 172.21.15.2:0/2488072759 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:08.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f123f7fe700 1 --2- 172.21.15.2:0/2488072759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1240106260 0x7f124019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:08.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f123effd700 1 -- 172.21.15.2:0/2488072759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1240106260 msgr2=0x7f124019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:08.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f123effd700 1 --2- 172.21.15.2:0/2488072759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1240106260 0x7f124019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:08.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f123effd700 1 -- 172.21.15.2:0/2488072759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12401022f0 msgr2=0x7f1240198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:08.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f123effd700 1 --2- 172.21.15.2:0/2488072759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12401022f0 0x7f1240198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:08.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f123effd700 1 -- 172.21.15.2:0/2488072759 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12300095f0 con 0x7f1240101940 2024-03-15T12:57:08.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.919+0000 7f123effd700 1 --2- 172.21.15.2:0/2488072759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1240101940 0x7f12401985c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1230005770 tx=0x7f12300057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:08.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.920+0000 7f1237fff700 1 -- 172.21.15.2:0/2488072759 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123001d070 con 0x7f1240101940 2024-03-15T12:57:08.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.920+0000 7f1237fff700 1 -- 172.21.15.2:0/2488072759 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1230005a00 con 0x7f1240101940 2024-03-15T12:57:08.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.920+0000 7f1237fff700 1 -- 172.21.15.2:0/2488072759 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230021e20 con 0x7f1240101940 2024-03-15T12:57:08.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.920+0000 7f12453fc700 1 -- 172.21.15.2:0/2488072759 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124019da00 con 0x7f1240101940 2024-03-15T12:57:08.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.920+0000 7f12453fc700 1 -- 172.21.15.2:0/2488072759 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12401a52c0 con 0x7f1240101940 2024-03-15T12:57:08.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.920+0000 7f12453fc700 1 -- 172.21.15.2:0/2488072759 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1240111270 con 0x7f1240101940 2024-03-15T12:57:08.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.922+0000 7f1237fff700 1 -- 172.21.15.2:0/2488072759 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f1230021580 con 0x7f1240101940 2024-03-15T12:57:08.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.922+0000 7f1237fff700 1 --2- 172.21.15.2:0/2488072759 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f122c0a1b50 0x7f122c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:08.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.923+0000 7f123e7fc700 1 --2- 172.21.15.2:0/2488072759 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f122c0a1b50 0x7f122c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:08.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.923+0000 7f1237fff700 1 -- 172.21.15.2:0/2488072759 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f12300cec40 con 0x7f1240101940 2024-03-15T12:57:08.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.923+0000 7f1237fff700 1 --2- 172.21.15.2:0/2488072759 >> [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202] conn(0x7f122c0ac810 0x7f122c0aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:08.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.923+0000 7f1237fff700 1 -- 172.21.15.2:0/2488072759 --> [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f122c0af320 con 0x7f122c0ac810 2024-03-15T12:57:08.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.923+0000 7f1237fff700 1 -- 172.21.15.2:0/2488072759 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12300cefb0 con 0x7f1240101940 2024-03-15T12:57:08.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.923+0000 7f123f7fe700 1 --2- 172.21.15.2:0/2488072759 >> [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202] conn(0x7f122c0ac810 0x7f122c0aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:08.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.923+0000 7f123e7fc700 1 --2- 172.21.15.2:0/2488072759 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f122c0a1b50 0x7f122c0a4010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1240107c30 tx=0x7f1228009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:08.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.923+0000 7f123f7fe700 1 --2- 172.21.15.2:0/2488072759 >> [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202] conn(0x7f122c0ac810 0x7f122c0aec50 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:08.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.924+0000 7f1237fff700 1 -- 172.21.15.2:0/2488072759 <== osd.10 v2:172.21.15.149:6824/432128202 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f122c0af320 con 0x7f122c0ac810 2024-03-15T12:57:08.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.943+0000 7f12453fc700 1 -- 172.21.15.2:0/2488072759 --> [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f12400fedd0 con 0x7f122c0ac810 2024-03-15T12:57:08.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.943+0000 7f1237fff700 1 -- 172.21.15.2:0/2488072759 <== osd.10 v2:172.21.15.149:6824/432128202 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f12400fedd0 con 0x7f122c0ac810 2024-03-15T12:57:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.943+0000 7f12453fc700 1 -- 172.21.15.2:0/2488072759 >> [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202] conn(0x7f122c0ac810 msgr2=0x7f122c0aec50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.943+0000 7f12453fc700 1 --2- 172.21.15.2:0/2488072759 >> [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202] conn(0x7f122c0ac810 0x7f122c0aec50 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.943+0000 7f12453fc700 1 -- 172.21.15.2:0/2488072759 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f122c0a1b50 msgr2=0x7f122c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.943+0000 7f12453fc700 1 --2- 172.21.15.2:0/2488072759 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f122c0a1b50 0x7f122c0a4010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1240107c30 tx=0x7f1228009260 comp rx=0 tx=0).stop 2024-03-15T12:57:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.943+0000 7f12453fc700 1 -- 172.21.15.2:0/2488072759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1240101940 msgr2=0x7f12401985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.943+0000 7f12453fc700 1 --2- 172.21.15.2:0/2488072759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1240101940 0x7f12401985c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1230005770 tx=0x7f12300057a0 comp rx=0 tx=0).stop 2024-03-15T12:57:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.943+0000 7f123f7fe700 1 -- 172.21.15.2:0/2488072759 reap_dead start 2024-03-15T12:57:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.943+0000 7f12453fc700 1 -- 172.21.15.2:0/2488072759 shutdown_connections 2024-03-15T12:57:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.943+0000 7f12453fc700 1 -- 172.21.15.2:0/2488072759 >> 172.21.15.2:0/2488072759 conn(0x7f12400774e0 msgr2=0x7f1240104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.944+0000 7f12453fc700 1 -- 172.21.15.2:0/2488072759 shutdown_connections 2024-03-15T12:57:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:08.944+0000 7f12453fc700 1 -- 172.21.15.2:0/2488072759 wait complete. 2024-03-15T12:57:10.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:09 smithi002 ceph-mon[32060]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-15T12:57:10.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:09 smithi184 ceph-mon[35993]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-15T12:57:10.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:09 smithi149 ceph-mon[35927]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-15T12:57:10.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.553+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2058994845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb744101130 msgr2=0x7fb744101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:10.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.553+0000 7fb74a5e6700 1 --2- 172.21.15.2:0/2058994845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb744101130 0x7fb744101510 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb734009990 tx=0x7fb734009ca0 comp rx=0 tx=0).stop 2024-03-15T12:57:10.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.553+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2058994845 shutdown_connections 2024-03-15T12:57:10.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.553+0000 7fb74a5e6700 1 --2- 172.21.15.2:0/2058994845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb744101130 0x7fb744101510 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:10.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.553+0000 7fb74a5e6700 1 --2- 172.21.15.2:0/2058994845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7441061f0 0x7fb74410ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:10.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.553+0000 7fb74a5e6700 1 --2- 172.21.15.2:0/2058994845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb744101ae0 0x7fb744105b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:10.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.553+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2058994845 >> 172.21.15.2:0/2058994845 conn(0x7fb744077e50 msgr2=0x7fb744078260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:10.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.553+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2058994845 shutdown_connections 2024-03-15T12:57:10.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.553+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2058994845 wait complete. 2024-03-15T12:57:10.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.554+0000 7fb74a5e6700 1 Processor -- start 2024-03-15T12:57:10.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.554+0000 7fb74a5e6700 1 -- start start 2024-03-15T12:57:10.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb74a5e6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb744101130 0x7fb744198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb74a5e6700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb744101ae0 0x7fb7441989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb74a5e6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7441061f0 0x7fb74419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb74a5e6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74419d340 con 0x7fb7441061f0 2024-03-15T12:57:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb74a5e6700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74419d4b0 con 0x7fb744101ae0 2024-03-15T12:57:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb74a5e6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74419d670 con 0x7fb744101130 2024-03-15T12:57:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb7437fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb744101ae0 0x7fb7441989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb743fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb744101130 0x7fb744198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb748b83700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7441061f0 0x7fb74419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb743fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb744101130 0x7fb744198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37576/0 (socket says 172.21.15.2:37576) 2024-03-15T12:57:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb743fff700 1 -- 172.21.15.2:0/2072107509 learned_addr learned my addr 172.21.15.2:0/2072107509 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb7437fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb744101ae0 0x7fb7441989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60202/0 (socket says 172.21.15.2:60202) 2024-03-15T12:57:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb748b83700 1 -- 172.21.15.2:0/2072107509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb744101130 msgr2=0x7fb744198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb748b83700 1 --2- 172.21.15.2:0/2072107509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb744101130 0x7fb744198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:10.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb748b83700 1 -- 172.21.15.2:0/2072107509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb744101ae0 msgr2=0x7fb7441989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:10.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb748b83700 1 --2- 172.21.15.2:0/2072107509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb744101ae0 0x7fb7441989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:10.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb748b83700 1 -- 172.21.15.2:0/2072107509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c009520 con 0x7fb7441061f0 2024-03-15T12:57:10.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb748b83700 1 --2- 172.21.15.2:0/2072107509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7441061f0 0x7fb74419cd30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00efb0 tx=0x7fb73c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:10.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb7417fa700 1 -- 172.21.15.2:0/2072107509 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c003a00 con 0x7fb7441061f0 2024-03-15T12:57:10.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb7417fa700 1 -- 172.21.15.2:0/2072107509 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb73c00f040 con 0x7fb7441061f0 2024-03-15T12:57:10.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.555+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2072107509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73c009c40 con 0x7fb7441061f0 2024-03-15T12:57:10.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.556+0000 7fb7417fa700 1 -- 172.21.15.2:0/2072107509 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c004600 con 0x7fb7441061f0 2024-03-15T12:57:10.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.556+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2072107509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7340095f0 con 0x7fb7441061f0 2024-03-15T12:57:10.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.556+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2072107509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb744111250 con 0x7fb7441061f0 2024-03-15T12:57:10.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.558+0000 7fb7417fa700 1 -- 172.21.15.2:0/2072107509 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7fb73c004760 con 0x7fb7441061f0 2024-03-15T12:57:10.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.558+0000 7fb7417fa700 1 --2- 172.21.15.2:0/2072107509 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb7300a1b50 0x7fb7300a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:10.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.558+0000 7fb743fff700 1 --2- 172.21.15.2:0/2072107509 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb7300a1b50 0x7fb7300a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:10.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.558+0000 7fb7417fa700 1 -- 172.21.15.2:0/2072107509 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb73c0cd850 con 0x7fb7441061f0 2024-03-15T12:57:10.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.558+0000 7fb7417fa700 1 --2- 172.21.15.2:0/2072107509 >> [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413] conn(0x7fb7300ac810 0x7fb7300aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:10.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.558+0000 7fb7417fa700 1 -- 172.21.15.2:0/2072107509 --> [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7300af320 con 0x7fb7300ac810 2024-03-15T12:57:10.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.558+0000 7fb7417fa700 1 -- 172.21.15.2:0/2072107509 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb73c0cdc60 con 0x7fb7441061f0 2024-03-15T12:57:10.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.559+0000 7fb7437fe700 1 --2- 172.21.15.2:0/2072107509 >> [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413] conn(0x7fb7300ac810 0x7fb7300aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:10.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.559+0000 7fb743fff700 1 --2- 172.21.15.2:0/2072107509 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb7300a1b50 0x7fb7300a4010 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb734005cd0 tx=0x7fb734005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:10.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.559+0000 7fb7437fe700 1 --2- 172.21.15.2:0/2072107509 >> [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413] conn(0x7fb7300ac810 0x7fb7300aec50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:10.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.560+0000 7fb7417fa700 1 -- 172.21.15.2:0/2072107509 <== osd.9 v2:172.21.15.2:6826/49148413 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb7300af320 con 0x7fb7300ac810 2024-03-15T12:57:10.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.576+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2072107509 --> [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb7440626f0 con 0x7fb7300ac810 2024-03-15T12:57:10.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.576+0000 7fb7417fa700 1 -- 172.21.15.2:0/2072107509 <== osd.9 v2:172.21.15.2:6826/49148413 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb7440626f0 con 0x7fb7300ac810 2024-03-15T12:57:10.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.576+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2072107509 >> [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413] conn(0x7fb7300ac810 msgr2=0x7fb7300aec50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:10.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.576+0000 7fb74a5e6700 1 --2- 172.21.15.2:0/2072107509 >> [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413] conn(0x7fb7300ac810 0x7fb7300aec50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:10.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.576+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2072107509 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb7300a1b50 msgr2=0x7fb7300a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:10.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.576+0000 7fb74a5e6700 1 --2- 172.21.15.2:0/2072107509 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb7300a1b50 0x7fb7300a4010 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb734005cd0 tx=0x7fb734005c60 comp rx=0 tx=0).stop 2024-03-15T12:57:10.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.576+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2072107509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7441061f0 msgr2=0x7fb74419cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:10.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.576+0000 7fb74a5e6700 1 --2- 172.21.15.2:0/2072107509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7441061f0 0x7fb74419cd30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00efb0 tx=0x7fb73c00c3a0 comp rx=0 tx=0).stop 2024-03-15T12:57:10.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.576+0000 7fb748b83700 1 -- 172.21.15.2:0/2072107509 reap_dead start 2024-03-15T12:57:10.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.576+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2072107509 shutdown_connections 2024-03-15T12:57:10.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.576+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2072107509 >> 172.21.15.2:0/2072107509 conn(0x7fb744077e50 msgr2=0x7fb74406a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:10.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.577+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2072107509 shutdown_connections 2024-03-15T12:57:10.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.577+0000 7fb74a5e6700 1 -- 172.21.15.2:0/2072107509 wait complete. 2024-03-15T12:57:10.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.448+0000 7ff173d4b700 1 -- 172.21.15.2:0/2354660977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff16c101170 msgr2=0x7ff16c101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:10.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.448+0000 7ff173d4b700 1 --2- 172.21.15.2:0/2354660977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff16c101170 0x7ff16c101550 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff168009990 tx=0x7ff168009ca0 comp rx=0 tx=0).stop 2024-03-15T12:57:10.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.448+0000 7ff173d4b700 1 -- 172.21.15.2:0/2354660977 shutdown_connections 2024-03-15T12:57:10.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.448+0000 7ff173d4b700 1 --2- 172.21.15.2:0/2354660977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff16c101170 0x7ff16c101550 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:10.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.448+0000 7ff173d4b700 1 --2- 172.21.15.2:0/2354660977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff16c1061c0 0x7ff16c10dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:10.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.448+0000 7ff173d4b700 1 --2- 172.21.15.2:0/2354660977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff16c101b20 0x7ff16c105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:10.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.448+0000 7ff173d4b700 1 -- 172.21.15.2:0/2354660977 >> 172.21.15.2:0/2354660977 conn(0x7ff16c076f30 msgr2=0x7ff16c077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:10.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.448+0000 7ff173d4b700 1 -- 172.21.15.2:0/2354660977 shutdown_connections 2024-03-15T12:57:10.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.448+0000 7ff173d4b700 1 -- 172.21.15.2:0/2354660977 wait complete. 2024-03-15T12:57:10.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.449+0000 7ff173d4b700 1 Processor -- start 2024-03-15T12:57:10.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.449+0000 7ff173d4b700 1 -- start start 2024-03-15T12:57:10.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.449+0000 7ff173d4b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff16c101170 0x7ff16c102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:10.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.449+0000 7ff173d4b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff16c101b20 0x7ff16c1030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:10.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.449+0000 7ff173d4b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff16c1061c0 0x7ff16c103600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:10.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.449+0000 7ff173d4b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c1a6940 con 0x7ff16c1061c0 2024-03-15T12:57:10.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.449+0000 7ff173d4b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c1a6ab0 con 0x7ff16c101b20 2024-03-15T12:57:10.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.449+0000 7ff173d4b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c1a6c70 con 0x7ff16c101170 2024-03-15T12:57:10.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.449+0000 7ff1722e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff16c1061c0 0x7ff16c103600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:10.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.449+0000 7ff1722e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff16c1061c0 0x7ff16c103600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50960/0 (socket says 172.21.15.2:50960) 2024-03-15T12:57:10.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.449+0000 7ff1722e8700 1 -- 172.21.15.2:0/3548769966 learned_addr learned my addr 172.21.15.2:0/3548769966 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:10.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.449+0000 7ff171ae7700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff16c101170 0x7ff16c102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:10.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.449+0000 7ff1712e6700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff16c101b20 0x7ff16c1030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:10.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.450+0000 7ff171ae7700 1 -- 172.21.15.2:0/3548769966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff16c101b20 msgr2=0x7ff16c1030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:10.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.450+0000 7ff171ae7700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff16c101b20 0x7ff16c1030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:10.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.450+0000 7ff171ae7700 1 -- 172.21.15.2:0/3548769966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff16c1061c0 msgr2=0x7ff16c103600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:10.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.450+0000 7ff171ae7700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff16c1061c0 0x7ff16c103600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:10.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.450+0000 7ff171ae7700 1 -- 172.21.15.2:0/3548769966 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1680095f0 con 0x7ff16c101170 2024-03-15T12:57:10.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.450+0000 7ff1722e8700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff16c1061c0 0x7ff16c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:10.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.450+0000 7ff1712e6700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff16c101b20 0x7ff16c1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:10.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.450+0000 7ff171ae7700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff16c101170 0x7ff16c102b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff168006010 tx=0x7ff168005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:10.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.450+0000 7ff15effd700 1 -- 172.21.15.2:0/3548769966 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16801d070 con 0x7ff16c101170 2024-03-15T12:57:10.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.450+0000 7ff173d4b700 1 -- 172.21.15.2:0/3548769966 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16c1a6ef0 con 0x7ff16c101170 2024-03-15T12:57:10.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.450+0000 7ff15effd700 1 -- 172.21.15.2:0/3548769966 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff16800be80 con 0x7ff16c101170 2024-03-15T12:57:10.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.450+0000 7ff15effd700 1 -- 172.21.15.2:0/3548769966 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff168022e10 con 0x7ff16c101170 2024-03-15T12:57:10.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.450+0000 7ff173d4b700 1 -- 172.21.15.2:0/3548769966 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c1a72b0 con 0x7ff16c101170 2024-03-15T12:57:10.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.451+0000 7ff173d4b700 1 -- 172.21.15.2:0/3548769966 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff16c111150 con 0x7ff16c101170 2024-03-15T12:57:10.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.452+0000 7ff15effd700 1 -- 172.21.15.2:0/3548769966 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7ff168022590 con 0x7ff16c101170 2024-03-15T12:57:10.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.453+0000 7ff15effd700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff1580a1b50 0x7ff1580a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:10.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.453+0000 7ff1712e6700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff1580a1b50 0x7ff1580a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:10.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.453+0000 7ff15effd700 1 -- 172.21.15.2:0/3548769966 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff1680ceef0 con 0x7ff16c101170 2024-03-15T12:57:10.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.453+0000 7ff15effd700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344] conn(0x7ff1580ac810 0x7ff1580aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:10.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.453+0000 7ff15effd700 1 -- 172.21.15.2:0/3548769966 --> [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1580af320 con 0x7ff1580ac810 2024-03-15T12:57:10.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.453+0000 7ff15effd700 1 -- 172.21.15.2:0/3548769966 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1680cf260 con 0x7ff16c101170 2024-03-15T12:57:10.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.453+0000 7ff1722e8700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344] conn(0x7ff1580ac810 0x7ff1580aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:10.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.454+0000 7ff1712e6700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff1580a1b50 0x7ff1580a4010 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff16c105af0 tx=0x7ff160009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:10.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.454+0000 7ff1722e8700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344] conn(0x7ff1580ac810 0x7ff1580aec50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:10.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.455+0000 7ff15effd700 1 -- 172.21.15.2:0/3548769966 <== osd.2 v2:172.21.15.184:6800/1645410344 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff1580af320 con 0x7ff1580ac810 2024-03-15T12:57:10.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.489+0000 7ff173d4b700 1 -- 172.21.15.2:0/3548769966 --> [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff16c0626f0 con 0x7ff1580ac810 2024-03-15T12:57:10.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.489+0000 7ff15effd700 1 -- 172.21.15.2:0/3548769966 <== osd.2 v2:172.21.15.184:6800/1645410344 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff16c0626f0 con 0x7ff1580ac810 2024-03-15T12:57:10.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.490+0000 7ff173d4b700 1 -- 172.21.15.2:0/3548769966 >> [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344] conn(0x7ff1580ac810 msgr2=0x7ff1580aec50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:10.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.490+0000 7ff173d4b700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344] conn(0x7ff1580ac810 0x7ff1580aec50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:10.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.490+0000 7ff173d4b700 1 -- 172.21.15.2:0/3548769966 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff1580a1b50 msgr2=0x7ff1580a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:10.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.490+0000 7ff173d4b700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff1580a1b50 0x7ff1580a4010 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff16c105af0 tx=0x7ff160009260 comp rx=0 tx=0).stop 2024-03-15T12:57:10.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.490+0000 7ff173d4b700 1 -- 172.21.15.2:0/3548769966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff16c101170 msgr2=0x7ff16c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:10.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.490+0000 7ff173d4b700 1 --2- 172.21.15.2:0/3548769966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff16c101170 0x7ff16c102b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff168006010 tx=0x7ff168005740 comp rx=0 tx=0).stop 2024-03-15T12:57:10.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.490+0000 7ff1722e8700 1 -- 172.21.15.2:0/3548769966 reap_dead start 2024-03-15T12:57:10.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.490+0000 7ff173d4b700 1 -- 172.21.15.2:0/3548769966 shutdown_connections 2024-03-15T12:57:10.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.490+0000 7ff173d4b700 1 -- 172.21.15.2:0/3548769966 >> 172.21.15.2:0/3548769966 conn(0x7ff16c076f30 msgr2=0x7ff16c06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:10.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.490+0000 7ff173d4b700 1 -- 172.21.15.2:0/3548769966 shutdown_connections 2024-03-15T12:57:10.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:10.490+0000 7ff173d4b700 1 -- 172.21.15.2:0/3548769966 wait complete. 2024-03-15T12:57:12.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:11 smithi002 ceph-mon[32060]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-15T12:57:12.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:11 smithi184 ceph-mon[35993]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-15T12:57:12.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:11 smithi149 ceph-mon[35927]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-15T12:57:12.929 INFO:teuthology.orchestra.run.smithi002.stdout:287762808866 2024-03-15T12:57:12.929 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-15T12:57:13.231 INFO:teuthology.orchestra.run.smithi002.stdout:77309411397 2024-03-15T12:57:13.232 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-15T12:57:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.221+0000 7f03723e3700 1 -- 172.21.15.2:0/1169220937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f036c1061c0 msgr2=0x7f036c10dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.221+0000 7f03723e3700 1 --2- 172.21.15.2:0/1169220937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f036c1061c0 0x7f036c10dd90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f036400b0a0 tx=0x7f036400b3b0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.222+0000 7f03723e3700 1 -- 172.21.15.2:0/1169220937 shutdown_connections 2024-03-15T12:57:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.222+0000 7f03723e3700 1 --2- 172.21.15.2:0/1169220937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f036c1061c0 0x7f036c10dd90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.222+0000 7f03723e3700 1 --2- 172.21.15.2:0/1169220937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f036c101170 0x7f036c101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.222+0000 7f03723e3700 1 --2- 172.21.15.2:0/1169220937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036c101b20 0x7f036c105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.222+0000 7f03723e3700 1 -- 172.21.15.2:0/1169220937 >> 172.21.15.2:0/1169220937 conn(0x7f036c076f30 msgr2=0x7f036c077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.222+0000 7f03723e3700 1 -- 172.21.15.2:0/1169220937 shutdown_connections 2024-03-15T12:57:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.222+0000 7f03723e3700 1 -- 172.21.15.2:0/1169220937 wait complete. 2024-03-15T12:57:13.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.222+0000 7f03723e3700 1 Processor -- start 2024-03-15T12:57:13.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.222+0000 7f03723e3700 1 -- start start 2024-03-15T12:57:13.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.223+0000 7f03723e3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f036c101170 0x7f036c1a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:13.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.223+0000 7f03723e3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f036c101b20 0x7f036c1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:13.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.223+0000 7f03723e3700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036c1061c0 0x7f036c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:13.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.223+0000 7f03723e3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c19acd0 con 0x7f036c101b20 2024-03-15T12:57:13.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.223+0000 7f03723e3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c19ae10 con 0x7f036c101170 2024-03-15T12:57:13.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.223+0000 7f03723e3700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c19afd0 con 0x7f036c1061c0 2024-03-15T12:57:13.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.223+0000 7f036bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f036c101170 0x7f036c1a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:13.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.223+0000 7f0370980700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036c1061c0 0x7f036c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:13.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.223+0000 7f036bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f036c101170 0x7f036c1a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60244/0 (socket says 172.21.15.2:60244) 2024-03-15T12:57:13.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.223+0000 7f036bfff700 1 -- 172.21.15.2:0/1297045474 learned_addr learned my addr 172.21.15.2:0/1297045474 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:13.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.223+0000 7f0370980700 1 --2- 172.21.15.2:0/1297045474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036c1061c0 0x7f036c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37608/0 (socket says 172.21.15.2:37608) 2024-03-15T12:57:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.223+0000 7f0370980700 1 -- 172.21.15.2:0/1297045474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f036c101170 msgr2=0x7f036c1a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.223+0000 7f0370980700 1 --2- 172.21.15.2:0/1297045474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f036c101170 0x7f036c1a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.223+0000 7f0370980700 1 -- 172.21.15.2:0/1297045474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f036c101b20 msgr2=0x7f036c1a09a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-15T12:57:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.224+0000 7f0370980700 1 --2- 172.21.15.2:0/1297045474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f036c101b20 0x7f036c1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.224+0000 7f0370980700 1 -- 172.21.15.2:0/1297045474 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0360009520 con 0x7f036c1061c0 2024-03-15T12:57:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.224+0000 7f036bfff700 1 --2- 172.21.15.2:0/1297045474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f036c101170 0x7f036c1a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.224+0000 7f0370980700 1 --2- 172.21.15.2:0/1297045474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036c1061c0 0x7f036c19a670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f036400d010 tx=0x7f036400bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.224+0000 7f03697fa700 1 -- 172.21.15.2:0/1297045474 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036400e040 con 0x7f036c1061c0 2024-03-15T12:57:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.224+0000 7f03697fa700 1 -- 172.21.15.2:0/1297045474 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f036401d480 con 0x7f036c1061c0 2024-03-15T12:57:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.224+0000 7f03723e3700 1 -- 172.21.15.2:0/1297045474 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0364009d00 con 0x7f036c1061c0 2024-03-15T12:57:13.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.224+0000 7f03697fa700 1 -- 172.21.15.2:0/1297045474 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03640052a0 con 0x7f036c1061c0 2024-03-15T12:57:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.224+0000 7f03723e3700 1 -- 172.21.15.2:0/1297045474 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036c19b4d0 con 0x7f036c1061c0 2024-03-15T12:57:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.225+0000 7f03723e3700 1 -- 172.21.15.2:0/1297045474 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f036c1025c0 con 0x7f036c1061c0 2024-03-15T12:57:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.227+0000 7f03697fa700 1 -- 172.21.15.2:0/1297045474 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f0364003a00 con 0x7f036c1061c0 2024-03-15T12:57:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.228+0000 7f03697fa700 1 --2- 172.21.15.2:0/1297045474 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f03540a1bd0 0x7f03540a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.228+0000 7f036bfff700 1 --2- 172.21.15.2:0/1297045474 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f03540a1bd0 0x7f03540a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.228+0000 7f03697fa700 1 -- 172.21.15.2:0/1297045474 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f03640ce810 con 0x7f036c1061c0 2024-03-15T12:57:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.228+0000 7f03697fa700 1 --2- 172.21.15.2:0/1297045474 >> [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624] conn(0x7f03540ac890 0x7f03540aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.228+0000 7f03697fa700 1 -- 172.21.15.2:0/1297045474 --> [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03540af3a0 con 0x7f03540ac890 2024-03-15T12:57:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.228+0000 7f03697fa700 1 -- 172.21.15.2:0/1297045474 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03640d32a0 con 0x7f036c1061c0 2024-03-15T12:57:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.228+0000 7f036bfff700 1 --2- 172.21.15.2:0/1297045474 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f03540a1bd0 0x7f03540a4090 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f035c011550 tx=0x7f035c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.228+0000 7f036b7fe700 1 --2- 172.21.15.2:0/1297045474 >> [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624] conn(0x7f03540ac890 0x7f03540aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.229+0000 7f036b7fe700 1 --2- 172.21.15.2:0/1297045474 >> [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624] conn(0x7f03540ac890 0x7f03540aecd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.230+0000 7f03697fa700 1 -- 172.21.15.2:0/1297045474 <== osd.5 v2:172.21.15.184:6808/1659458624 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f03540af3a0 con 0x7f03540ac890 2024-03-15T12:57:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.247+0000 7f03723e3700 1 -- 172.21.15.2:0/1297045474 --> [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f036c111220 con 0x7f03540ac890 2024-03-15T12:57:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.247+0000 7f03697fa700 1 -- 172.21.15.2:0/1297045474 <== osd.5 v2:172.21.15.184:6808/1659458624 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f036c111220 con 0x7f03540ac890 2024-03-15T12:57:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.248+0000 7f03723e3700 1 -- 172.21.15.2:0/1297045474 >> [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624] conn(0x7f03540ac890 msgr2=0x7f03540aecd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.248+0000 7f03723e3700 1 --2- 172.21.15.2:0/1297045474 >> [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624] conn(0x7f03540ac890 0x7f03540aecd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.248+0000 7f03723e3700 1 -- 172.21.15.2:0/1297045474 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f03540a1bd0 msgr2=0x7f03540a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.248+0000 7f03723e3700 1 --2- 172.21.15.2:0/1297045474 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f03540a1bd0 0x7f03540a4090 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f035c011550 tx=0x7f035c009260 comp rx=0 tx=0).stop 2024-03-15T12:57:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.248+0000 7f03723e3700 1 -- 172.21.15.2:0/1297045474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036c1061c0 msgr2=0x7f036c19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.248+0000 7f03723e3700 1 --2- 172.21.15.2:0/1297045474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f036c1061c0 0x7f036c19a670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f036400d010 tx=0x7f036400bdd0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.248+0000 7f0370980700 1 -- 172.21.15.2:0/1297045474 reap_dead start 2024-03-15T12:57:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.248+0000 7f03723e3700 1 -- 172.21.15.2:0/1297045474 shutdown_connections 2024-03-15T12:57:13.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.248+0000 7f03723e3700 1 -- 172.21.15.2:0/1297045474 >> 172.21.15.2:0/1297045474 conn(0x7f036c076f30 msgr2=0x7f036c06a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:13.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.248+0000 7f03723e3700 1 -- 172.21.15.2:0/1297045474 shutdown_connections 2024-03-15T12:57:13.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.248+0000 7f03723e3700 1 -- 172.21.15.2:0/1297045474 wait complete. 2024-03-15T12:57:13.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.139+0000 7faae26c4700 1 -- 172.21.15.2:0/705476585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faadc1022c0 msgr2=0x7faadc1026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:13.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.139+0000 7faae26c4700 1 --2- 172.21.15.2:0/705476585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faadc1022c0 0x7faadc1026a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faacc009990 tx=0x7faacc009ca0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.139+0000 7faae26c4700 1 -- 172.21.15.2:0/705476585 shutdown_connections 2024-03-15T12:57:13.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.139+0000 7faae26c4700 1 --2- 172.21.15.2:0/705476585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faadc1022c0 0x7faadc1026a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.139+0000 7faae26c4700 1 --2- 172.21.15.2:0/705476585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faadc1072f0 0x7faadc10eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.139+0000 7faae26c4700 1 --2- 172.21.15.2:0/705476585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc102c70 0x7faadc106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.139+0000 7faae26c4700 1 -- 172.21.15.2:0/705476585 >> 172.21.15.2:0/705476585 conn(0x7faadc079070 msgr2=0x7faadc079480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:13.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.139+0000 7faae26c4700 1 -- 172.21.15.2:0/705476585 shutdown_connections 2024-03-15T12:57:13.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.139+0000 7faae26c4700 1 -- 172.21.15.2:0/705476585 wait complete. 2024-03-15T12:57:13.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.139+0000 7faae26c4700 1 Processor -- start 2024-03-15T12:57:13.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.139+0000 7faae26c4700 1 -- start start 2024-03-15T12:57:13.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.140+0000 7faae26c4700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc1022c0 0x7faadc0742e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:13.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.140+0000 7faae26c4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faadc102c70 0x7faadc074820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:13.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.140+0000 7faae26c4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faadc1072f0 0x7faadc074d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:13.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.140+0000 7faae26c4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc1a68d0 con 0x7faadc1072f0 2024-03-15T12:57:13.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.140+0000 7faae26c4700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc1a6a40 con 0x7faadc1022c0 2024-03-15T12:57:13.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.140+0000 7faae26c4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc1a6c00 con 0x7faadc102c70 2024-03-15T12:57:13.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.140+0000 7faadbfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc1022c0 0x7faadc0742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:13.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.140+0000 7faae0c61700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faadc1072f0 0x7faadc074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:13.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.140+0000 7faadb7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faadc102c70 0x7faadc074820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:13.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.140+0000 7faadbfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc1022c0 0x7faadc0742e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60226/0 (socket says 172.21.15.2:60226) 2024-03-15T12:57:13.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.140+0000 7faadbfff700 1 -- 172.21.15.2:0/1619038444 learned_addr learned my addr 172.21.15.2:0/1619038444 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:13.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.140+0000 7faadb7fe700 1 -- 172.21.15.2:0/1619038444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc1022c0 msgr2=0x7faadc0742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:13.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.140+0000 7faadb7fe700 1 --2- 172.21.15.2:0/1619038444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc1022c0 0x7faadc0742e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.140+0000 7faadb7fe700 1 -- 172.21.15.2:0/1619038444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faadc1072f0 msgr2=0x7faadc074d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:13.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.140+0000 7faadb7fe700 1 --2- 172.21.15.2:0/1619038444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faadc1072f0 0x7faadc074d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.141+0000 7faadb7fe700 1 -- 172.21.15.2:0/1619038444 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad0009520 con 0x7faadc102c70 2024-03-15T12:57:13.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.141+0000 7faadb7fe700 1 --2- 172.21.15.2:0/1619038444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faadc102c70 0x7faadc074820 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faad000e9c0 tx=0x7faad000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:13.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.141+0000 7faad97fa700 1 -- 172.21.15.2:0/1619038444 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad000cb70 con 0x7faadc102c70 2024-03-15T12:57:13.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.141+0000 7faad97fa700 1 -- 172.21.15.2:0/1619038444 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7faad000ee60 con 0x7faadc102c70 2024-03-15T12:57:13.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.141+0000 7faae26c4700 1 -- 172.21.15.2:0/1619038444 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faacc0095f0 con 0x7faadc102c70 2024-03-15T12:57:13.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.141+0000 7faad97fa700 1 -- 172.21.15.2:0/1619038444 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0003d20 con 0x7faadc102c70 2024-03-15T12:57:13.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.141+0000 7faae26c4700 1 -- 172.21.15.2:0/1619038444 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faadc1a7240 con 0x7faadc102c70 2024-03-15T12:57:13.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.142+0000 7faae26c4700 1 -- 172.21.15.2:0/1619038444 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faadc1122b0 con 0x7faadc102c70 2024-03-15T12:57:13.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.143+0000 7faad97fa700 1 -- 172.21.15.2:0/1619038444 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7faad0003ea0 con 0x7faadc102c70 2024-03-15T12:57:13.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.143+0000 7faad97fa700 1 --2- 172.21.15.2:0/1619038444 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7faac40a1bd0 0x7faac40a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:13.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.144+0000 7faadbfff700 1 --2- 172.21.15.2:0/1619038444 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7faac40a1bd0 0x7faac40a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:13.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.144+0000 7faad97fa700 1 -- 172.21.15.2:0/1619038444 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7faad0014070 con 0x7faadc102c70 2024-03-15T12:57:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.144+0000 7faad97fa700 1 --2- 172.21.15.2:0/1619038444 >> [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355] conn(0x7faac40ac890 0x7faac40aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.144+0000 7faad97fa700 1 -- 172.21.15.2:0/1619038444 --> [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faac40af3a0 con 0x7faac40ac890 2024-03-15T12:57:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.144+0000 7faad97fa700 1 -- 172.21.15.2:0/1619038444 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7faad00ce1a0 con 0x7faadc102c70 2024-03-15T12:57:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.144+0000 7faae0c61700 1 --2- 172.21.15.2:0/1619038444 >> [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355] conn(0x7faac40ac890 0x7faac40aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.144+0000 7faadbfff700 1 --2- 172.21.15.2:0/1619038444 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7faac40a1bd0 0x7faac40a4090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faacc005cd0 tx=0x7faacc005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.144+0000 7faae0c61700 1 --2- 172.21.15.2:0/1619038444 >> [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355] conn(0x7faac40ac890 0x7faac40aecd0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.145+0000 7faad97fa700 1 -- 172.21.15.2:0/1619038444 <== osd.0 v2:172.21.15.2:6802/2123099355 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7faac40af3a0 con 0x7faac40ac890 2024-03-15T12:57:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.162+0000 7faae26c4700 1 -- 172.21.15.2:0/1619038444 --> [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faadc0626a0 con 0x7faac40ac890 2024-03-15T12:57:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.162+0000 7faad97fa700 1 -- 172.21.15.2:0/1619038444 <== osd.0 v2:172.21.15.2:6802/2123099355 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faadc0626a0 con 0x7faac40ac890 2024-03-15T12:57:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.162+0000 7faae26c4700 1 -- 172.21.15.2:0/1619038444 >> [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355] conn(0x7faac40ac890 msgr2=0x7faac40aecd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.162+0000 7faae26c4700 1 --2- 172.21.15.2:0/1619038444 >> [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355] conn(0x7faac40ac890 0x7faac40aecd0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.162+0000 7faae26c4700 1 -- 172.21.15.2:0/1619038444 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7faac40a1bd0 msgr2=0x7faac40a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.162+0000 7faae26c4700 1 --2- 172.21.15.2:0/1619038444 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7faac40a1bd0 0x7faac40a4090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faacc005cd0 tx=0x7faacc005c60 comp rx=0 tx=0).stop 2024-03-15T12:57:13.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.162+0000 7faae26c4700 1 -- 172.21.15.2:0/1619038444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faadc102c70 msgr2=0x7faadc074820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:13.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.162+0000 7faae26c4700 1 --2- 172.21.15.2:0/1619038444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faadc102c70 0x7faadc074820 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faad000e9c0 tx=0x7faad000ecd0 comp rx=0 tx=0).stop 2024-03-15T12:57:13.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.162+0000 7faae0c61700 1 -- 172.21.15.2:0/1619038444 reap_dead start 2024-03-15T12:57:13.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.163+0000 7faae26c4700 1 -- 172.21.15.2:0/1619038444 shutdown_connections 2024-03-15T12:57:13.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.163+0000 7faae26c4700 1 -- 172.21.15.2:0/1619038444 >> 172.21.15.2:0/1619038444 conn(0x7faadc079070 msgr2=0x7faadc1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:13.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.163+0000 7faae26c4700 1 -- 172.21.15.2:0/1619038444 shutdown_connections 2024-03-15T12:57:13.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:13.163+0000 7faae26c4700 1 -- 172.21.15.2:0/1619038444 wait complete. 2024-03-15T12:57:14.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:13 smithi002 ceph-mon[32060]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:57:14.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:13 smithi184 ceph-mon[35993]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:57:14.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:13 smithi149 ceph-mon[35927]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:57:14.870 INFO:teuthology.orchestra.run.smithi002.stdout:309237645339 2024-03-15T12:57:14.871 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-15T12:57:16.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:15 smithi184 ceph-mon[35993]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-15T12:57:16.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:15 smithi149 ceph-mon[35927]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-15T12:57:16.172 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:15 smithi002 ceph-mon[32060]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-03-15T12:57:16.198 INFO:teuthology.orchestra.run.smithi002.stdout:34359738447 2024-03-15T12:57:16.199 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-15T12:57:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.396+0000 7f03e5355700 1 -- 172.21.15.2:0/3329740286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03e0101880 msgr2=0x7f03e010dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.396+0000 7f03e5355700 1 --2- 172.21.15.2:0/3329740286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03e0101880 0x7f03e010dd60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f03d800b0a0 tx=0x7f03d800b3b0 comp rx=0 tx=0).stop 2024-03-15T12:57:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.396+0000 7f03e5355700 1 -- 172.21.15.2:0/3329740286 shutdown_connections 2024-03-15T12:57:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.396+0000 7f03e5355700 1 --2- 172.21.15.2:0/3329740286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03e0101880 0x7f03e010dd60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.396+0000 7f03e5355700 1 --2- 172.21.15.2:0/3329740286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03e0106f20 0x7f03e0107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.396+0000 7f03e5355700 1 --2- 172.21.15.2:0/3329740286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03e0100ee0 0x7f03e0101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.396+0000 7f03e5355700 1 -- 172.21.15.2:0/3329740286 >> 172.21.15.2:0/3329740286 conn(0x7f03e0077e10 msgr2=0x7f03e0078220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.396+0000 7f03e5355700 1 -- 172.21.15.2:0/3329740286 shutdown_connections 2024-03-15T12:57:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.396+0000 7f03e5355700 1 -- 172.21.15.2:0/3329740286 wait complete. 2024-03-15T12:57:16.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.397+0000 7f03e5355700 1 Processor -- start 2024-03-15T12:57:16.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.397+0000 7f03e5355700 1 -- start start 2024-03-15T12:57:16.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.397+0000 7f03e5355700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03e0100ee0 0x7f03e01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:16.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.397+0000 7f03e5355700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03e0101880 0x7f03e0198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:16.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.397+0000 7f03e5355700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03e0106f20 0x7f03e019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:16.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.397+0000 7f03e5355700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e019d5d0 con 0x7f03e0100ee0 2024-03-15T12:57:16.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.397+0000 7f03e5355700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e019d740 con 0x7f03e0106f20 2024-03-15T12:57:16.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.397+0000 7f03e5355700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e019d900 con 0x7f03e0101880 2024-03-15T12:57:16.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03deffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03e0100ee0 0x7f03e01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03de7fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03e0101880 0x7f03e0198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03deffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03e0100ee0 0x7f03e01986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51018/0 (socket says 172.21.15.2:51018) 2024-03-15T12:57:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03deffd700 1 -- 172.21.15.2:0/632932703 learned_addr learned my addr 172.21.15.2:0/632932703 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03df7fe700 1 --2- 172.21.15.2:0/632932703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03e0106f20 0x7f03e019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03de7fc700 1 -- 172.21.15.2:0/632932703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03e0106f20 msgr2=0x7f03e019cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03de7fc700 1 --2- 172.21.15.2:0/632932703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03e0106f20 0x7f03e019cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03de7fc700 1 -- 172.21.15.2:0/632932703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03e0100ee0 msgr2=0x7f03e01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03de7fc700 1 --2- 172.21.15.2:0/632932703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03e0100ee0 0x7f03e01986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03de7fc700 1 -- 172.21.15.2:0/632932703 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d8009d00 con 0x7f03e0101880 2024-03-15T12:57:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03deffd700 1 --2- 172.21.15.2:0/632932703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03e0100ee0 0x7f03e01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03de7fc700 1 --2- 172.21.15.2:0/632932703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03e0101880 0x7f03e0198c30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f03d400e9f0 tx=0x7f03d400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03c7fff700 1 -- 172.21.15.2:0/632932703 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d400c9c0 con 0x7f03e0101880 2024-03-15T12:57:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03e5355700 1 -- 172.21.15.2:0/632932703 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e019dbe0 con 0x7f03e0101880 2024-03-15T12:57:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03c7fff700 1 -- 172.21.15.2:0/632932703 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f03d4010430 con 0x7f03e0101880 2024-03-15T12:57:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.398+0000 7f03c7fff700 1 -- 172.21.15.2:0/632932703 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4018660 con 0x7f03e0101880 2024-03-15T12:57:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.399+0000 7f03e5355700 1 -- 172.21.15.2:0/632932703 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e01a5440 con 0x7f03e0101880 2024-03-15T12:57:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.400+0000 7f03e5355700 1 -- 172.21.15.2:0/632932703 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f03e01111c0 con 0x7f03e0101880 2024-03-15T12:57:16.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.402+0000 7f03c7fff700 1 -- 172.21.15.2:0/632932703 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f03d40105a0 con 0x7f03e0101880 2024-03-15T12:57:16.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.403+0000 7f03c7fff700 1 --2- 172.21.15.2:0/632932703 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f03c80a1b50 0x7f03c80a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:16.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.403+0000 7f03c7fff700 1 -- 172.21.15.2:0/632932703 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f03d4014070 con 0x7f03e0101880 2024-03-15T12:57:16.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.403+0000 7f03deffd700 1 --2- 172.21.15.2:0/632932703 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f03c80a1b50 0x7f03c80a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:16.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.403+0000 7f03c7fff700 1 --2- 172.21.15.2:0/632932703 >> [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371] conn(0x7f03c80ac800 0x7f03c80aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:16.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.403+0000 7f03c7fff700 1 -- 172.21.15.2:0/632932703 --> [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03c80af310 con 0x7f03c80ac800 2024-03-15T12:57:16.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.403+0000 7f03c7fff700 1 -- 172.21.15.2:0/632932703 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03d40ce340 con 0x7f03e0101880 2024-03-15T12:57:16.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.404+0000 7f03df7fe700 1 --2- 172.21.15.2:0/632932703 >> [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371] conn(0x7f03c80ac800 0x7f03c80aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:16.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.404+0000 7f03deffd700 1 --2- 172.21.15.2:0/632932703 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f03c80a1b50 0x7f03c80a4010 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f03d0011550 tx=0x7f03d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:16.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.404+0000 7f03df7fe700 1 --2- 172.21.15.2:0/632932703 >> [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371] conn(0x7f03c80ac800 0x7f03c80aec40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:16.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.405+0000 7f03c7fff700 1 -- 172.21.15.2:0/632932703 <== osd.7 v2:172.21.15.149:6816/4241660371 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f03c80af310 con 0x7f03c80ac800 2024-03-15T12:57:16.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.424+0000 7f03e5355700 1 -- 172.21.15.2:0/632932703 --> [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f03e00ffc50 con 0x7f03c80ac800 2024-03-15T12:57:16.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.425+0000 7f03c7fff700 1 -- 172.21.15.2:0/632932703 <== osd.7 v2:172.21.15.149:6816/4241660371 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f03e00ffc50 con 0x7f03c80ac800 2024-03-15T12:57:16.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.425+0000 7f03e5355700 1 -- 172.21.15.2:0/632932703 >> [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371] conn(0x7f03c80ac800 msgr2=0x7f03c80aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:16.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.425+0000 7f03e5355700 1 --2- 172.21.15.2:0/632932703 >> [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371] conn(0x7f03c80ac800 0x7f03c80aec40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:16.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.425+0000 7f03e5355700 1 -- 172.21.15.2:0/632932703 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f03c80a1b50 msgr2=0x7f03c80a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:16.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.425+0000 7f03e5355700 1 --2- 172.21.15.2:0/632932703 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f03c80a1b50 0x7f03c80a4010 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f03d0011550 tx=0x7f03d0009260 comp rx=0 tx=0).stop 2024-03-15T12:57:16.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.425+0000 7f03e5355700 1 -- 172.21.15.2:0/632932703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03e0101880 msgr2=0x7f03e0198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:16.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.425+0000 7f03e5355700 1 --2- 172.21.15.2:0/632932703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03e0101880 0x7f03e0198c30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f03d400e9f0 tx=0x7f03d400ed00 comp rx=0 tx=0).stop 2024-03-15T12:57:16.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.425+0000 7f03df7fe700 1 -- 172.21.15.2:0/632932703 reap_dead start 2024-03-15T12:57:16.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.426+0000 7f03e5355700 1 -- 172.21.15.2:0/632932703 shutdown_connections 2024-03-15T12:57:16.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.426+0000 7f03e5355700 1 -- 172.21.15.2:0/632932703 >> 172.21.15.2:0/632932703 conn(0x7f03e0077e10 msgr2=0x7f03e0105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:16.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.426+0000 7f03e5355700 1 -- 172.21.15.2:0/632932703 shutdown_connections 2024-03-15T12:57:16.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:16.426+0000 7f03e5355700 1 -- 172.21.15.2:0/632932703 wait complete. 2024-03-15T12:57:17.481 INFO:teuthology.orchestra.run.smithi002.stdout:163208757302 2024-03-15T12:57:17.482 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-15T12:57:17.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.921+0000 7fbbc672f700 1 -- 172.21.15.2:0/79111616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbc0108010 msgr2=0x7fbbc01083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:17.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.921+0000 7fbbc672f700 1 --2- 172.21.15.2:0/79111616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbc0108010 0x7fbbc01083f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00099c0 tx=0x7fbbb0009cd0 comp rx=0 tx=0).stop 2024-03-15T12:57:17.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.921+0000 7fbbc672f700 1 -- 172.21.15.2:0/79111616 shutdown_connections 2024-03-15T12:57:17.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.921+0000 7fbbc672f700 1 --2- 172.21.15.2:0/79111616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbc0108010 0x7fbbc01083f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:17.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.921+0000 7fbbc672f700 1 --2- 172.21.15.2:0/79111616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc006b0f0 0x7fbbc010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:17.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.921+0000 7fbbc672f700 1 --2- 172.21.15.2:0/79111616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbc006a730 0x7fbbc006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:17.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.921+0000 7fbbc672f700 1 -- 172.21.15.2:0/79111616 >> 172.21.15.2:0/79111616 conn(0x7fbbc0077ef0 msgr2=0x7fbbc0078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:17.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.921+0000 7fbbc672f700 1 -- 172.21.15.2:0/79111616 shutdown_connections 2024-03-15T12:57:17.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.921+0000 7fbbc672f700 1 -- 172.21.15.2:0/79111616 wait complete. 2024-03-15T12:57:17.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.921+0000 7fbbc672f700 1 Processor -- start 2024-03-15T12:57:17.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbc672f700 1 -- start start 2024-03-15T12:57:17.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbc672f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbc006a730 0x7fbbc0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:17.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbc672f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc006b0f0 0x7fbbc01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:17.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbc672f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbc0108010 0x7fbbc019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:17.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbc672f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc019d370 con 0x7fbbc006a730 2024-03-15T12:57:17.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbc672f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc019d4e0 con 0x7fbbc006b0f0 2024-03-15T12:57:17.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbc672f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc019d6a0 con 0x7fbbc0108010 2024-03-15T12:57:17.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbbffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbc006a730 0x7fbbc0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:17.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbbffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbc006a730 0x7fbbc0198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50292/0 (socket says 172.21.15.2:50292) 2024-03-15T12:57:17.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbbffff700 1 -- 172.21.15.2:0/2528787157 learned_addr learned my addr 172.21.15.2:0/2528787157 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:17.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbbf7fe700 1 --2- 172.21.15.2:0/2528787157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc006b0f0 0x7fbbc01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:17.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbc4ccc700 1 --2- 172.21.15.2:0/2528787157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbc0108010 0x7fbbc019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:17.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbbffff700 1 -- 172.21.15.2:0/2528787157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbc0108010 msgr2=0x7fbbc019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:17.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbbffff700 1 --2- 172.21.15.2:0/2528787157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbc0108010 0x7fbbc019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:17.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbbffff700 1 -- 172.21.15.2:0/2528787157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc006b0f0 msgr2=0x7fbbc01989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:17.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbbffff700 1 --2- 172.21.15.2:0/2528787157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc006b0f0 0x7fbbc01989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:17.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbbffff700 1 -- 172.21.15.2:0/2528787157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb00095f0 con 0x7fbbc006a730 2024-03-15T12:57:17.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.922+0000 7fbbbf7fe700 1 --2- 172.21.15.2:0/2528787157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc006b0f0 0x7fbbc01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:17.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.923+0000 7fbbbffff700 1 --2- 172.21.15.2:0/2528787157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbc006a730 0x7fbbc0198490 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0006010 tx=0x7fbbb0005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:17.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.923+0000 7fbbbd7fa700 1 -- 172.21.15.2:0/2528787157 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb001d070 con 0x7fbbc006a730 2024-03-15T12:57:17.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.923+0000 7fbbc672f700 1 -- 172.21.15.2:0/2528787157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc019d920 con 0x7fbbc006a730 2024-03-15T12:57:17.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.923+0000 7fbbbd7fa700 1 -- 172.21.15.2:0/2528787157 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbbb0003680 con 0x7fbbc006a730 2024-03-15T12:57:17.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.923+0000 7fbbbd7fa700 1 -- 172.21.15.2:0/2528787157 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb000fab0 con 0x7fbbc006a730 2024-03-15T12:57:17.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.923+0000 7fbbc672f700 1 -- 172.21.15.2:0/2528787157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc01a51e0 con 0x7fbbc006a730 2024-03-15T12:57:17.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.924+0000 7fbbc672f700 1 -- 172.21.15.2:0/2528787157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbbc01036c0 con 0x7fbbc006a730 2024-03-15T12:57:17.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.925+0000 7fbbbd7fa700 1 -- 172.21.15.2:0/2528787157 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7fbbb000b730 con 0x7fbbc006a730 2024-03-15T12:57:17.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.925+0000 7fbbbd7fa700 1 --2- 172.21.15.2:0/2528787157 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbbac0a1b50 0x7fbbac0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:17.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.926+0000 7fbbbd7fa700 1 -- 172.21.15.2:0/2528787157 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fbbb00ceb80 con 0x7fbbc006a730 2024-03-15T12:57:17.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.926+0000 7fbbbf7fe700 1 --2- 172.21.15.2:0/2528787157 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbbac0a1b50 0x7fbbac0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:17.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.926+0000 7fbbbd7fa700 1 --2- 172.21.15.2:0/2528787157 >> [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535] conn(0x7fbbac0ac810 0x7fbbac0aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:17.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.926+0000 7fbbbd7fa700 1 -- 172.21.15.2:0/2528787157 --> [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbac0af320 con 0x7fbbac0ac810 2024-03-15T12:57:17.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.926+0000 7fbbbd7fa700 1 -- 172.21.15.2:0/2528787157 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbb00cef90 con 0x7fbbc006a730 2024-03-15T12:57:17.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.926+0000 7fbbc4ccc700 1 --2- 172.21.15.2:0/2528787157 >> [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535] conn(0x7fbbac0ac810 0x7fbbac0aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:17.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.926+0000 7fbbbf7fe700 1 --2- 172.21.15.2:0/2528787157 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbbac0a1b50 0x7fbbac0a4010 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0078530 tx=0x7fbba8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:17.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.926+0000 7fbbc4ccc700 1 --2- 172.21.15.2:0/2528787157 >> [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535] conn(0x7fbbac0ac810 0x7fbbac0aec50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:17.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.927+0000 7fbbbd7fa700 1 -- 172.21.15.2:0/2528787157 <== osd.4 v2:172.21.15.149:6808/1790191535 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbbac0af320 con 0x7fbbac0ac810 2024-03-15T12:57:17.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.944+0000 7fbbc672f700 1 -- 172.21.15.2:0/2528787157 --> [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbbc0112310 con 0x7fbbac0ac810 2024-03-15T12:57:17.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.944+0000 7fbbbd7fa700 1 -- 172.21.15.2:0/2528787157 <== osd.4 v2:172.21.15.149:6808/1790191535 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbbc0112310 con 0x7fbbac0ac810 2024-03-15T12:57:17.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.944+0000 7fbbc672f700 1 -- 172.21.15.2:0/2528787157 >> [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535] conn(0x7fbbac0ac810 msgr2=0x7fbbac0aec50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:17.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.944+0000 7fbbc672f700 1 --2- 172.21.15.2:0/2528787157 >> [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535] conn(0x7fbbac0ac810 0x7fbbac0aec50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:17.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.944+0000 7fbbc672f700 1 -- 172.21.15.2:0/2528787157 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbbac0a1b50 msgr2=0x7fbbac0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:17.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.944+0000 7fbbc672f700 1 --2- 172.21.15.2:0/2528787157 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbbac0a1b50 0x7fbbac0a4010 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0078530 tx=0x7fbba8009260 comp rx=0 tx=0).stop 2024-03-15T12:57:17.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.944+0000 7fbbc672f700 1 -- 172.21.15.2:0/2528787157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbc006a730 msgr2=0x7fbbc0198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:17.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.944+0000 7fbbc672f700 1 --2- 172.21.15.2:0/2528787157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbc006a730 0x7fbbc0198490 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0006010 tx=0x7fbbb0005280 comp rx=0 tx=0).stop 2024-03-15T12:57:17.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.944+0000 7fbbc4ccc700 1 -- 172.21.15.2:0/2528787157 reap_dead start 2024-03-15T12:57:17.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.944+0000 7fbbc672f700 1 -- 172.21.15.2:0/2528787157 shutdown_connections 2024-03-15T12:57:17.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.944+0000 7fbbc672f700 1 -- 172.21.15.2:0/2528787157 >> 172.21.15.2:0/2528787157 conn(0x7fbbc0077ef0 msgr2=0x7fbbc00ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:17.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.944+0000 7fbbc672f700 1 -- 172.21.15.2:0/2528787157 shutdown_connections 2024-03-15T12:57:17.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:17.944+0000 7fbbc672f700 1 -- 172.21.15.2:0/2528787157 wait complete. 2024-03-15T12:57:18.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:17 smithi002 ceph-mon[32060]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-15T12:57:18.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:17 smithi184 ceph-mon[35993]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-15T12:57:18.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:17 smithi149 ceph-mon[35927]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-15T12:57:18.637 INFO:teuthology.orchestra.run.smithi002.stdout:141733920828 2024-03-15T12:57:18.637 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-15T12:57:18.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.858+0000 7fca5b062700 1 -- 172.21.15.2:0/936493021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca54069c90 msgr2=0x7fca5406a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:18.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.858+0000 7fca5b062700 1 --2- 172.21.15.2:0/936493021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca54069c90 0x7fca5406a0f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0099c0 tx=0x7fca3c009cd0 comp rx=0 tx=0).stop 2024-03-15T12:57:18.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.858+0000 7fca5b062700 1 -- 172.21.15.2:0/936493021 shutdown_connections 2024-03-15T12:57:18.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.858+0000 7fca5b062700 1 --2- 172.21.15.2:0/936493021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca5406a630 0x7fca5410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:18.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.858+0000 7fca5b062700 1 --2- 172.21.15.2:0/936493021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca54108030 0x7fca54108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:18.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.858+0000 7fca5b062700 1 --2- 172.21.15.2:0/936493021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca54069c90 0x7fca5406a0f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:18.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.858+0000 7fca5b062700 1 -- 172.21.15.2:0/936493021 >> 172.21.15.2:0/936493021 conn(0x7fca54077e10 msgr2=0x7fca54078220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:18.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.858+0000 7fca5b062700 1 -- 172.21.15.2:0/936493021 shutdown_connections 2024-03-15T12:57:18.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.858+0000 7fca5b062700 1 -- 172.21.15.2:0/936493021 wait complete. 2024-03-15T12:57:18.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.859+0000 7fca5b062700 1 Processor -- start 2024-03-15T12:57:18.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.859+0000 7fca5b062700 1 -- start start 2024-03-15T12:57:18.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.859+0000 7fca5b062700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca54069c90 0x7fca54198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:18.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.859+0000 7fca5b062700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca5406a630 0x7fca54198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:18.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.859+0000 7fca5b062700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca54108030 0x7fca5419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:18.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.859+0000 7fca5b062700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5419d490 con 0x7fca5406a630 2024-03-15T12:57:18.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.859+0000 7fca5b062700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5419d600 con 0x7fca54069c90 2024-03-15T12:57:18.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.859+0000 7fca5b062700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5419d7c0 con 0x7fca54108030 2024-03-15T12:57:18.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.859+0000 7fca53fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca5406a630 0x7fca54198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:18.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.859+0000 7fca58dfe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca54069c90 0x7fca54198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:18.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.859+0000 7fca53fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca5406a630 0x7fca54198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50304/0 (socket says 172.21.15.2:50304) 2024-03-15T12:57:18.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.859+0000 7fca53fff700 1 -- 172.21.15.2:0/3386297736 learned_addr learned my addr 172.21.15.2:0/3386297736 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:18.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.859+0000 7fca595ff700 1 --2- 172.21.15.2:0/3386297736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca54108030 0x7fca5419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:18.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.860+0000 7fca595ff700 1 -- 172.21.15.2:0/3386297736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca54069c90 msgr2=0x7fca54198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:18.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.860+0000 7fca595ff700 1 --2- 172.21.15.2:0/3386297736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca54069c90 0x7fca54198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:18.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.860+0000 7fca595ff700 1 -- 172.21.15.2:0/3386297736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca5406a630 msgr2=0x7fca54198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:18.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.860+0000 7fca595ff700 1 --2- 172.21.15.2:0/3386297736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca5406a630 0x7fca54198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:18.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.860+0000 7fca595ff700 1 -- 172.21.15.2:0/3386297736 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c0095f0 con 0x7fca54108030 2024-03-15T12:57:18.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.860+0000 7fca58dfe700 1 --2- 172.21.15.2:0/3386297736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca54069c90 0x7fca54198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:18.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.860+0000 7fca595ff700 1 --2- 172.21.15.2:0/3386297736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca54108030 0x7fca5419ce80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00d720 tx=0x7fca4c00da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:18.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.860+0000 7fca51ffb700 1 -- 172.21.15.2:0/3386297736 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c004220 con 0x7fca54108030 2024-03-15T12:57:18.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.860+0000 7fca51ffb700 1 -- 172.21.15.2:0/3386297736 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fca4c004380 con 0x7fca54108030 2024-03-15T12:57:18.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.860+0000 7fca51ffb700 1 -- 172.21.15.2:0/3386297736 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c003b90 con 0x7fca54108030 2024-03-15T12:57:18.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.860+0000 7fca5b062700 1 -- 172.21.15.2:0/3386297736 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5419daa0 con 0x7fca54108030 2024-03-15T12:57:18.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.860+0000 7fca5b062700 1 -- 172.21.15.2:0/3386297736 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca541a5300 con 0x7fca54108030 2024-03-15T12:57:18.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.861+0000 7fca5b062700 1 -- 172.21.15.2:0/3386297736 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca54112300 con 0x7fca54108030 2024-03-15T12:57:18.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.862+0000 7fca51ffb700 1 -- 172.21.15.2:0/3386297736 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7fca4c003cf0 con 0x7fca54108030 2024-03-15T12:57:18.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.863+0000 7fca51ffb700 1 --2- 172.21.15.2:0/3386297736 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fca400a1bd0 0x7fca400a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:18.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.863+0000 7fca58dfe700 1 --2- 172.21.15.2:0/3386297736 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fca400a1bd0 0x7fca400a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:18.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.863+0000 7fca51ffb700 1 -- 172.21.15.2:0/3386297736 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fca4c0ccf40 con 0x7fca54108030 2024-03-15T12:57:18.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.863+0000 7fca51ffb700 1 --2- 172.21.15.2:0/3386297736 >> [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638] conn(0x7fca400accb0 0x7fca400af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:18.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.863+0000 7fca51ffb700 1 -- 172.21.15.2:0/3386297736 --> [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca400af820 con 0x7fca400accb0 2024-03-15T12:57:18.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.863+0000 7fca51ffb700 1 -- 172.21.15.2:0/3386297736 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca4c0cf520 con 0x7fca54108030 2024-03-15T12:57:18.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.863+0000 7fca53fff700 1 --2- 172.21.15.2:0/3386297736 >> [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638] conn(0x7fca400accb0 0x7fca400af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:18.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.863+0000 7fca58dfe700 1 --2- 172.21.15.2:0/3386297736 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fca400a1bd0 0x7fca400a4090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fca44009480 tx=0x7fca4400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:18.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.864+0000 7fca53fff700 1 --2- 172.21.15.2:0/3386297736 >> [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638] conn(0x7fca400accb0 0x7fca400af150 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:18.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.864+0000 7fca51ffb700 1 -- 172.21.15.2:0/3386297736 <== osd.6 v2:172.21.15.2:6818/4103461638 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fca400af820 con 0x7fca400accb0 2024-03-15T12:57:18.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.881+0000 7fca5b062700 1 -- 172.21.15.2:0/3386297736 --> [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fca540626f0 con 0x7fca400accb0 2024-03-15T12:57:18.867 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:57:18.867 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-15T12:57:18.867 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-15T12:57:18.867 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-03-15T12:57:18.867 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-03-15T12:57:18.868 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-15T12:57:18.868 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-03-15T12:57:18.868 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-03-15T12:57:18.868 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-03-15T12:57:18.868 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:18 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2024-03-15T12:57:18.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.881+0000 7fca51ffb700 1 -- 172.21.15.2:0/3386297736 <== osd.6 v2:172.21.15.2:6818/4103461638 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fca540626f0 con 0x7fca400accb0 2024-03-15T12:57:18.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.882+0000 7fca5b062700 1 -- 172.21.15.2:0/3386297736 >> [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638] conn(0x7fca400accb0 msgr2=0x7fca400af150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:18.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.882+0000 7fca5b062700 1 --2- 172.21.15.2:0/3386297736 >> [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638] conn(0x7fca400accb0 0x7fca400af150 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:18.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.882+0000 7fca5b062700 1 -- 172.21.15.2:0/3386297736 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fca400a1bd0 msgr2=0x7fca400a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:18.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.882+0000 7fca5b062700 1 --2- 172.21.15.2:0/3386297736 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fca400a1bd0 0x7fca400a4090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fca44009480 tx=0x7fca4400b040 comp rx=0 tx=0).stop 2024-03-15T12:57:18.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.882+0000 7fca5b062700 1 -- 172.21.15.2:0/3386297736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca54108030 msgr2=0x7fca5419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:18.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.882+0000 7fca5b062700 1 --2- 172.21.15.2:0/3386297736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca54108030 0x7fca5419ce80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00d720 tx=0x7fca4c00da30 comp rx=0 tx=0).stop 2024-03-15T12:57:18.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.882+0000 7fca595ff700 1 -- 172.21.15.2:0/3386297736 reap_dead start 2024-03-15T12:57:18.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.882+0000 7fca5b062700 1 -- 172.21.15.2:0/3386297736 shutdown_connections 2024-03-15T12:57:18.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.882+0000 7fca5b062700 1 -- 172.21.15.2:0/3386297736 >> 172.21.15.2:0/3386297736 conn(0x7fca54077e10 msgr2=0x7fca540ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:18.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.882+0000 7fca5b062700 1 -- 172.21.15.2:0/3386297736 shutdown_connections 2024-03-15T12:57:18.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:18.882+0000 7fca5b062700 1 -- 172.21.15.2:0/3386297736 wait complete. 2024-03-15T12:57:19.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:57:19.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-15T12:57:19.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-15T12:57:19.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-03-15T12:57:19.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-03-15T12:57:19.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-15T12:57:19.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-03-15T12:57:19.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-03-15T12:57:19.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-03-15T12:57:19.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:18 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2024-03-15T12:57:19.138 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:57:19.138 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-15T12:57:19.138 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-15T12:57:19.139 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-03-15T12:57:19.139 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-03-15T12:57:19.139 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-03-15T12:57:19.139 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-03-15T12:57:19.139 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-03-15T12:57:19.139 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-03-15T12:57:19.139 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:18 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2024-03-15T12:57:19.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.525+0000 7fa75edb3700 1 -- 172.21.15.2:0/1282215357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7581029f0 msgr2=0x7fa75810ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:19.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.525+0000 7fa75edb3700 1 --2- 172.21.15.2:0/1282215357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7581029f0 0x7fa75810ee90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa75000b0a0 tx=0x7fa75000b3b0 comp rx=0 tx=0).stop 2024-03-15T12:57:19.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.525+0000 7fa75edb3700 1 -- 172.21.15.2:0/1282215357 shutdown_connections 2024-03-15T12:57:19.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.525+0000 7fa75edb3700 1 --2- 172.21.15.2:0/1282215357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7581029f0 0x7fa75810ee90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:19.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.525+0000 7fa75edb3700 1 --2- 172.21.15.2:0/1282215357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7581080e0 0x7fa7581084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:19.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.525+0000 7fa75edb3700 1 --2- 172.21.15.2:0/1282215357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa758102030 0x7fa7581024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:19.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.525+0000 7fa75edb3700 1 -- 172.21.15.2:0/1282215357 >> 172.21.15.2:0/1282215357 conn(0x7fa758079110 msgr2=0x7fa758079520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:19.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.525+0000 7fa75edb3700 1 -- 172.21.15.2:0/1282215357 shutdown_connections 2024-03-15T12:57:19.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.525+0000 7fa75edb3700 1 -- 172.21.15.2:0/1282215357 wait complete. 2024-03-15T12:57:19.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.526+0000 7fa75edb3700 1 Processor -- start 2024-03-15T12:57:19.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.526+0000 7fa75edb3700 1 -- start start 2024-03-15T12:57:19.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.526+0000 7fa75edb3700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa758102030 0x7fa758198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:19.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.526+0000 7fa75edb3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7581029f0 0x7fa758198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:19.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.526+0000 7fa75edb3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7581080e0 0x7fa75819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:19.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.526+0000 7fa75edb3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75819d4e0 con 0x7fa7581029f0 2024-03-15T12:57:19.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.526+0000 7fa75edb3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75819d650 con 0x7fa7581080e0 2024-03-15T12:57:19.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.526+0000 7fa75edb3700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75819d810 con 0x7fa758102030 2024-03-15T12:57:19.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.526+0000 7fa757fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7581029f0 0x7fa758198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:19.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.526+0000 7fa757fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7581029f0 0x7fa758198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50328/0 (socket says 172.21.15.2:50328) 2024-03-15T12:57:19.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.526+0000 7fa757fff700 1 -- 172.21.15.2:0/1369549136 learned_addr learned my addr 172.21.15.2:0/1369549136 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:19.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.526+0000 7fa75cb4f700 1 --2- 172.21.15.2:0/1369549136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa758102030 0x7fa758198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:19.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.527+0000 7fa75d350700 1 --2- 172.21.15.2:0/1369549136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7581080e0 0x7fa75819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:19.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.527+0000 7fa757fff700 1 -- 172.21.15.2:0/1369549136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa758102030 msgr2=0x7fa758198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:19.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.527+0000 7fa757fff700 1 --2- 172.21.15.2:0/1369549136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa758102030 0x7fa758198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:19.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.527+0000 7fa757fff700 1 -- 172.21.15.2:0/1369549136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7581080e0 msgr2=0x7fa75819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:19.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.527+0000 7fa757fff700 1 --2- 172.21.15.2:0/1369549136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7581080e0 0x7fa75819ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:19.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.527+0000 7fa757fff700 1 -- 172.21.15.2:0/1369549136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa750009d00 con 0x7fa7581029f0 2024-03-15T12:57:19.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.527+0000 7fa757fff700 1 --2- 172.21.15.2:0/1369549136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7581029f0 0x7fa758198b40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa74400e9a0 tx=0x7fa74400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:19.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.527+0000 7fa755ffb700 1 -- 172.21.15.2:0/1369549136 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744009790 con 0x7fa7581029f0 2024-03-15T12:57:19.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.527+0000 7fa755ffb700 1 -- 172.21.15.2:0/1369549136 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa744010430 con 0x7fa7581029f0 2024-03-15T12:57:19.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.527+0000 7fa75edb3700 1 -- 172.21.15.2:0/1369549136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75819daf0 con 0x7fa7581029f0 2024-03-15T12:57:19.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.527+0000 7fa755ffb700 1 -- 172.21.15.2:0/1369549136 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744018660 con 0x7fa7581029f0 2024-03-15T12:57:19.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.527+0000 7fa75edb3700 1 -- 172.21.15.2:0/1369549136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7580743e0 con 0x7fa7581029f0 2024-03-15T12:57:19.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.528+0000 7fa75edb3700 1 -- 172.21.15.2:0/1369549136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa758112380 con 0x7fa7581029f0 2024-03-15T12:57:19.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.529+0000 7fa755ffb700 1 -- 172.21.15.2:0/1369549136 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7fa7440105a0 con 0x7fa7581029f0 2024-03-15T12:57:19.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.530+0000 7fa755ffb700 1 --2- 172.21.15.2:0/1369549136 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa7480a1b50 0x7fa7480a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:19.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.530+0000 7fa755ffb700 1 -- 172.21.15.2:0/1369549136 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fa744014070 con 0x7fa7581029f0 2024-03-15T12:57:19.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.530+0000 7fa75cb4f700 1 --2- 172.21.15.2:0/1369549136 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa7480a1b50 0x7fa7480a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:19.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.530+0000 7fa755ffb700 1 --2- 172.21.15.2:0/1369549136 >> [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252] conn(0x7fa7480acc30 0x7fa7480af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:19.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.530+0000 7fa755ffb700 1 -- 172.21.15.2:0/1369549136 --> [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7480af7a0 con 0x7fa7480acc30 2024-03-15T12:57:19.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.530+0000 7fa755ffb700 1 -- 172.21.15.2:0/1369549136 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7440ce4a0 con 0x7fa7581029f0 2024-03-15T12:57:19.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.530+0000 7fa75d350700 1 --2- 172.21.15.2:0/1369549136 >> [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252] conn(0x7fa7480acc30 0x7fa7480af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:19.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.531+0000 7fa75cb4f700 1 --2- 172.21.15.2:0/1369549136 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa7480a1b50 0x7fa7480a4010 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa74c004090 tx=0x7fa74c00f7a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:19.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.531+0000 7fa75d350700 1 --2- 172.21.15.2:0/1369549136 >> [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252] conn(0x7fa7480acc30 0x7fa7480af0d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:19.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.532+0000 7fa755ffb700 1 -- 172.21.15.2:0/1369549136 <== osd.11 v2:172.21.15.184:6824/1184158252 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa7480af7a0 con 0x7fa7480acc30 2024-03-15T12:57:19.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.553+0000 7fa75edb3700 1 -- 172.21.15.2:0/1369549136 --> [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa7580626f0 con 0x7fa7480acc30 2024-03-15T12:57:19.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.553+0000 7fa755ffb700 1 -- 172.21.15.2:0/1369549136 <== osd.11 v2:172.21.15.184:6824/1184158252 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa7580626f0 con 0x7fa7480acc30 2024-03-15T12:57:19.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.554+0000 7fa75edb3700 1 -- 172.21.15.2:0/1369549136 >> [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252] conn(0x7fa7480acc30 msgr2=0x7fa7480af0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:19.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.554+0000 7fa75edb3700 1 --2- 172.21.15.2:0/1369549136 >> [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252] conn(0x7fa7480acc30 0x7fa7480af0d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:19.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.554+0000 7fa75edb3700 1 -- 172.21.15.2:0/1369549136 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa7480a1b50 msgr2=0x7fa7480a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:19.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.554+0000 7fa75edb3700 1 --2- 172.21.15.2:0/1369549136 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa7480a1b50 0x7fa7480a4010 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa74c004090 tx=0x7fa74c00f7a0 comp rx=0 tx=0).stop 2024-03-15T12:57:19.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.554+0000 7fa75edb3700 1 -- 172.21.15.2:0/1369549136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7581029f0 msgr2=0x7fa758198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:19.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.554+0000 7fa75edb3700 1 --2- 172.21.15.2:0/1369549136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7581029f0 0x7fa758198b40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa74400e9a0 tx=0x7fa74400ecb0 comp rx=0 tx=0).stop 2024-03-15T12:57:19.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.554+0000 7fa75d350700 1 -- 172.21.15.2:0/1369549136 reap_dead start 2024-03-15T12:57:19.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.554+0000 7fa75edb3700 1 -- 172.21.15.2:0/1369549136 shutdown_connections 2024-03-15T12:57:19.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.554+0000 7fa75edb3700 1 -- 172.21.15.2:0/1369549136 >> 172.21.15.2:0/1369549136 conn(0x7fa758079110 msgr2=0x7fa758100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:19.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.555+0000 7fa75edb3700 1 -- 172.21.15.2:0/1369549136 shutdown_connections 2024-03-15T12:57:19.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:19.555+0000 7fa75edb3700 1 -- 172.21.15.2:0/1369549136 wait complete. 2024-03-15T12:57:20.004 INFO:teuthology.orchestra.run.smithi002.stdout:197568495666 2024-03-15T12:57:20.005 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-15T12:57:20.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:19 smithi149 ceph-mon[35927]: pgmap v319: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:57:20.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-15T12:57:20.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-15T12:57:20.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-03-15T12:57:20.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-03-15T12:57:20.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-15T12:57:20.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-03-15T12:57:20.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-03-15T12:57:20.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-03-15T12:57:20.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:19 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-03-15T12:57:20.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:19 smithi149 ceph-mon[35927]: osdmap e90: 12 total, 12 up, 12 in 2024-03-15T12:57:20.250 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:19 smithi002 ceph-mon[32060]: pgmap v319: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:57:20.250 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:19 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-15T12:57:20.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:19 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-15T12:57:20.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:19 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-03-15T12:57:20.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:19 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-03-15T12:57:20.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:19 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-15T12:57:20.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:19 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-03-15T12:57:20.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:19 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-03-15T12:57:20.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:19 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-03-15T12:57:20.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:19 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-03-15T12:57:20.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:19 smithi002 ceph-mon[32060]: osdmap e90: 12 total, 12 up, 12 in 2024-03-15T12:57:20.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:19 smithi184 ceph-mon[35993]: pgmap v319: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:57:20.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:19 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-15T12:57:20.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:19 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-15T12:57:20.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:19 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-03-15T12:57:20.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:19 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-03-15T12:57:20.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:19 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-03-15T12:57:20.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:19 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-03-15T12:57:20.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:19 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-03-15T12:57:20.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:19 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-03-15T12:57:20.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:19 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-03-15T12:57:20.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:19 smithi184 ceph-mon[35993]: osdmap e90: 12 total, 12 up, 12 in 2024-03-15T12:57:20.590 INFO:teuthology.orchestra.run.smithi002.stdout:219043332140 2024-03-15T12:57:20.590 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-15T12:57:21.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:20 smithi149 ceph-mon[35927]: osdmap e91: 12 total, 12 up, 12 in 2024-03-15T12:57:21.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:21.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:21.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:21.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:20 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:21.124 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:20 smithi002 ceph-mon[32060]: osdmap e91: 12 total, 12 up, 12 in 2024-03-15T12:57:21.124 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:21.124 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:21.124 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:21.124 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:20 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:21.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:20 smithi184 ceph-mon[35993]: osdmap e91: 12 total, 12 up, 12 in 2024-03-15T12:57:21.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:21.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:21.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:21.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:20 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:21.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.382+0000 7f9ffbc59700 1 -- 172.21.15.2:0/2219157390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff41029f0 msgr2=0x7f9ff410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:21.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.382+0000 7f9ffbc59700 1 --2- 172.21.15.2:0/2219157390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff41029f0 0x7f9ff410ee30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b0d0 tx=0x7f9fec00b3e0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.382+0000 7f9ffbc59700 1 -- 172.21.15.2:0/2219157390 shutdown_connections 2024-03-15T12:57:21.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.382+0000 7f9ffbc59700 1 --2- 172.21.15.2:0/2219157390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff41029f0 0x7f9ff410ee30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.382+0000 7f9ffbc59700 1 --2- 172.21.15.2:0/2219157390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ff4108010 0x7f9ff41083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.382+0000 7f9ffbc59700 1 --2- 172.21.15.2:0/2219157390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ff4102030 0x7f9ff41024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.382+0000 7f9ffbc59700 1 -- 172.21.15.2:0/2219157390 >> 172.21.15.2:0/2219157390 conn(0x7f9ff4079110 msgr2=0x7f9ff4079520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:21.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.382+0000 7f9ffbc59700 1 -- 172.21.15.2:0/2219157390 shutdown_connections 2024-03-15T12:57:21.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.382+0000 7f9ffbc59700 1 -- 172.21.15.2:0/2219157390 wait complete. 2024-03-15T12:57:21.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.383+0000 7f9ffbc59700 1 Processor -- start 2024-03-15T12:57:21.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.383+0000 7f9ffbc59700 1 -- start start 2024-03-15T12:57:21.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.383+0000 7f9ffbc59700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff4102030 0x7f9ff4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:21.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.383+0000 7f9ffbc59700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ff41029f0 0x7f9ff4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:21.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.383+0000 7f9ffbc59700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ff4108010 0x7f9ff419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:21.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.383+0000 7f9ffbc59700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff419d3f0 con 0x7f9ff4102030 2024-03-15T12:57:21.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.383+0000 7f9ffbc59700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff419d560 con 0x7f9ff41029f0 2024-03-15T12:57:21.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.383+0000 7f9ffbc59700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff419d720 con 0x7f9ff4108010 2024-03-15T12:57:21.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.383+0000 7f9ff99f5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff4102030 0x7f9ff4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:21.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.383+0000 7f9ff99f5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff4102030 0x7f9ff4198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50336/0 (socket says 172.21.15.2:50336) 2024-03-15T12:57:21.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.383+0000 7f9ff99f5700 1 -- 172.21.15.2:0/822919501 learned_addr learned my addr 172.21.15.2:0/822919501 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:21.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.383+0000 7f9ff91f4700 1 --2- 172.21.15.2:0/822919501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ff41029f0 0x7f9ff4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:21.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.383+0000 7f9ffa1f6700 1 --2- 172.21.15.2:0/822919501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ff4108010 0x7f9ff419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:21.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.384+0000 7f9ff99f5700 1 -- 172.21.15.2:0/822919501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ff4108010 msgr2=0x7f9ff419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:21.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.384+0000 7f9ff99f5700 1 --2- 172.21.15.2:0/822919501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ff4108010 0x7f9ff419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.384+0000 7f9ff99f5700 1 -- 172.21.15.2:0/822919501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ff41029f0 msgr2=0x7f9ff4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:21.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.384+0000 7f9ff99f5700 1 --2- 172.21.15.2:0/822919501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ff41029f0 0x7f9ff4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.384+0000 7f9ff99f5700 1 -- 172.21.15.2:0/822919501 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec009d00 con 0x7f9ff4102030 2024-03-15T12:57:21.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.384+0000 7f9ff99f5700 1 --2- 172.21.15.2:0/822919501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff4102030 0x7f9ff4198510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400e9a0 tx=0x7f9fe400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:21.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.384+0000 7f9feaffd700 1 -- 172.21.15.2:0/822919501 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4009790 con 0x7f9ff4102030 2024-03-15T12:57:21.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.384+0000 7f9feaffd700 1 -- 172.21.15.2:0/822919501 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9fe4010430 con 0x7f9ff4102030 2024-03-15T12:57:21.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.384+0000 7f9ffbc59700 1 -- 172.21.15.2:0/822919501 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff419da00 con 0x7f9ff4102030 2024-03-15T12:57:21.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.384+0000 7f9feaffd700 1 -- 172.21.15.2:0/822919501 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4018660 con 0x7f9ff4102030 2024-03-15T12:57:21.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.384+0000 7f9ffbc59700 1 -- 172.21.15.2:0/822919501 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff41a5260 con 0x7f9ff4102030 2024-03-15T12:57:21.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.385+0000 7f9ffbc59700 1 -- 172.21.15.2:0/822919501 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9ff41036c0 con 0x7f9ff4102030 2024-03-15T12:57:21.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.386+0000 7f9feaffd700 1 -- 172.21.15.2:0/822919501 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f9fe40105a0 con 0x7f9ff4102030 2024-03-15T12:57:21.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.387+0000 7f9feaffd700 1 --2- 172.21.15.2:0/822919501 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9fe00a1b50 0x7f9fe00a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:21.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.387+0000 7f9ff91f4700 1 --2- 172.21.15.2:0/822919501 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9fe00a1b50 0x7f9fe00a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:21.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.387+0000 7f9feaffd700 1 -- 172.21.15.2:0/822919501 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f9fe4014070 con 0x7f9ff4102030 2024-03-15T12:57:21.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.387+0000 7f9feaffd700 1 --2- 172.21.15.2:0/822919501 >> [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302] conn(0x7f9fe00acc30 0x7f9fe00af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:21.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.387+0000 7f9feaffd700 1 -- 172.21.15.2:0/822919501 --> [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9fe00af7a0 con 0x7f9fe00acc30 2024-03-15T12:57:21.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.387+0000 7f9feaffd700 1 -- 172.21.15.2:0/822919501 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fe40ce460 con 0x7f9ff4102030 2024-03-15T12:57:21.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.387+0000 7f9ffa1f6700 1 --2- 172.21.15.2:0/822919501 >> [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302] conn(0x7f9fe00acc30 0x7f9fe00af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:21.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.387+0000 7f9ff91f4700 1 --2- 172.21.15.2:0/822919501 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9fe00a1b50 0x7f9fe00a4010 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40796a0 tx=0x7f9ff0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:21.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.387+0000 7f9ffa1f6700 1 --2- 172.21.15.2:0/822919501 >> [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302] conn(0x7f9fe00acc30 0x7f9fe00af0d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:21.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.388+0000 7f9feaffd700 1 -- 172.21.15.2:0/822919501 <== osd.3 v2:172.21.15.2:6810/12565302 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9fe00af7a0 con 0x7f9fe00acc30 2024-03-15T12:57:21.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.405+0000 7f9ffbc59700 1 -- 172.21.15.2:0/822919501 --> [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9ff4112310 con 0x7f9fe00acc30 2024-03-15T12:57:21.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.405+0000 7f9feaffd700 1 -- 172.21.15.2:0/822919501 <== osd.3 v2:172.21.15.2:6810/12565302 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9ff4112310 con 0x7f9fe00acc30 2024-03-15T12:57:21.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.405+0000 7f9ffbc59700 1 -- 172.21.15.2:0/822919501 >> [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302] conn(0x7f9fe00acc30 msgr2=0x7f9fe00af0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:21.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.405+0000 7f9ffbc59700 1 --2- 172.21.15.2:0/822919501 >> [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302] conn(0x7f9fe00acc30 0x7f9fe00af0d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.405+0000 7f9ffbc59700 1 -- 172.21.15.2:0/822919501 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9fe00a1b50 msgr2=0x7f9fe00a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:21.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.405+0000 7f9ffbc59700 1 --2- 172.21.15.2:0/822919501 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9fe00a1b50 0x7f9fe00a4010 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40796a0 tx=0x7f9ff0009260 comp rx=0 tx=0).stop 2024-03-15T12:57:21.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.405+0000 7f9ffbc59700 1 -- 172.21.15.2:0/822919501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff4102030 msgr2=0x7f9ff4198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:21.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.405+0000 7f9ffbc59700 1 --2- 172.21.15.2:0/822919501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff4102030 0x7f9ff4198510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400e9a0 tx=0x7f9fe400ecb0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.405+0000 7f9ffa1f6700 1 -- 172.21.15.2:0/822919501 reap_dead start 2024-03-15T12:57:21.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.405+0000 7f9ffbc59700 1 -- 172.21.15.2:0/822919501 shutdown_connections 2024-03-15T12:57:21.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.405+0000 7f9ffbc59700 1 -- 172.21.15.2:0/822919501 >> 172.21.15.2:0/822919501 conn(0x7f9ff4079110 msgr2=0x7f9ff4100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:21.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.406+0000 7f9ffbc59700 1 -- 172.21.15.2:0/822919501 shutdown_connections 2024-03-15T12:57:21.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.406+0000 7f9ffbc59700 1 -- 172.21.15.2:0/822919501 wait complete. 2024-03-15T12:57:21.777 INFO:teuthology.orchestra.run.smithi002.stdout:343597383705 2024-03-15T12:57:21.778 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-15T12:57:21.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.981+0000 7f0454567700 1 -- 172.21.15.2:0/2786594799 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f044c107300 msgr2=0x7f044c10eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:21.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.981+0000 7f0454567700 1 --2- 172.21.15.2:0/2786594799 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f044c107300 0x7f044c10eed0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f044400b0a0 tx=0x7f044400b3b0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.981+0000 7f0454567700 1 -- 172.21.15.2:0/2786594799 shutdown_connections 2024-03-15T12:57:21.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.981+0000 7f0454567700 1 --2- 172.21.15.2:0/2786594799 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f044c107300 0x7f044c10eed0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.981+0000 7f0454567700 1 --2- 172.21.15.2:0/2786594799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f044c102220 0x7f044c102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.981+0000 7f0454567700 1 --2- 172.21.15.2:0/2786594799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f044c102bd0 0x7f044c106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.981+0000 7f0454567700 1 -- 172.21.15.2:0/2786594799 >> 172.21.15.2:0/2786594799 conn(0x7f044c0fda90 msgr2=0x7f044c0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:21.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.981+0000 7f0454567700 1 -- 172.21.15.2:0/2786594799 shutdown_connections 2024-03-15T12:57:21.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.981+0000 7f0454567700 1 -- 172.21.15.2:0/2786594799 wait complete. 2024-03-15T12:57:21.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0454567700 1 Processor -- start 2024-03-15T12:57:21.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0454567700 1 -- start start 2024-03-15T12:57:21.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0454567700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f044c102220 0x7f044c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:21.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0454567700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f044c102bd0 0x7f044c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:21.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0454567700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f044c107300 0x7f044c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:21.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0454567700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c19d4c0 con 0x7f044c102220 2024-03-15T12:57:21.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0454567700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c19d630 con 0x7f044c107300 2024-03-15T12:57:21.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0454567700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c19d7f0 con 0x7f044c102bd0 2024-03-15T12:57:21.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0452303700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f044c102220 0x7f044c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:21.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0451b02700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f044c102bd0 0x7f044c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:21.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0452b04700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f044c107300 0x7f044c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:21.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0452303700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f044c102220 0x7f044c1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50366/0 (socket says 172.21.15.2:50366) 2024-03-15T12:57:21.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0451b02700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f044c102bd0 0x7f044c198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:39332/0 (socket says 172.21.15.2:39332) 2024-03-15T12:57:21.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0452303700 1 -- 172.21.15.2:0/2688751024 learned_addr learned my addr 172.21.15.2:0/2688751024 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:21.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0451b02700 1 -- 172.21.15.2:0/2688751024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f044c107300 msgr2=0x7f044c19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:21.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0451b02700 1 --2- 172.21.15.2:0/2688751024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f044c107300 0x7f044c19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0451b02700 1 -- 172.21.15.2:0/2688751024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f044c102220 msgr2=0x7f044c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:21.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0451b02700 1 --2- 172.21.15.2:0/2688751024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f044c102220 0x7f044c1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0451b02700 1 -- 172.21.15.2:0/2688751024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0444009d00 con 0x7f044c102bd0 2024-03-15T12:57:21.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0452b04700 1 --2- 172.21.15.2:0/2688751024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f044c107300 0x7f044c19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:21.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.982+0000 7f0452303700 1 --2- 172.21.15.2:0/2688751024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f044c102220 0x7f044c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:21.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.983+0000 7f0451b02700 1 --2- 172.21.15.2:0/2688751024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f044c102bd0 0x7f044c198b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f043c00e9c0 tx=0x7f043c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:21.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.983+0000 7f04437fe700 1 -- 172.21.15.2:0/2688751024 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c00c9c0 con 0x7f044c102bd0 2024-03-15T12:57:21.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.983+0000 7f04437fe700 1 -- 172.21.15.2:0/2688751024 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f043c010430 con 0x7f044c102bd0 2024-03-15T12:57:21.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.983+0000 7f04437fe700 1 -- 172.21.15.2:0/2688751024 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c018680 con 0x7f044c102bd0 2024-03-15T12:57:21.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.983+0000 7f0454567700 1 -- 172.21.15.2:0/2688751024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044c19dad0 con 0x7f044c102bd0 2024-03-15T12:57:21.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.983+0000 7f0454567700 1 -- 172.21.15.2:0/2688751024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044c1a5330 con 0x7f044c102bd0 2024-03-15T12:57:21.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.984+0000 7f0454567700 1 -- 172.21.15.2:0/2688751024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f044c112320 con 0x7f044c102bd0 2024-03-15T12:57:21.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.985+0000 7f04437fe700 1 -- 172.21.15.2:0/2688751024 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f043c0105a0 con 0x7f044c102bd0 2024-03-15T12:57:21.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.986+0000 7f04437fe700 1 --2- 172.21.15.2:0/2688751024 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f04380a1bd0 0x7f04380a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:21.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.986+0000 7f04437fe700 1 -- 172.21.15.2:0/2688751024 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f043c014070 con 0x7f044c102bd0 2024-03-15T12:57:21.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.986+0000 7f0452303700 1 --2- 172.21.15.2:0/2688751024 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f04380a1bd0 0x7f04380a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:21.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.986+0000 7f04437fe700 1 --2- 172.21.15.2:0/2688751024 >> [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579] conn(0x7f04380accb0 0x7f04380af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:21.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.986+0000 7f04437fe700 1 -- 172.21.15.2:0/2688751024 --> [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f04380af820 con 0x7f04380accb0 2024-03-15T12:57:21.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.986+0000 7f04437fe700 1 -- 172.21.15.2:0/2688751024 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f043c0ce300 con 0x7f044c102bd0 2024-03-15T12:57:21.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.986+0000 7f0452b04700 1 --2- 172.21.15.2:0/2688751024 >> [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579] conn(0x7f04380accb0 0x7f04380af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:21.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.986+0000 7f0452303700 1 --2- 172.21.15.2:0/2688751024 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f04380a1bd0 0x7f04380a4090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0448011550 tx=0x7f0448009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:21.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.987+0000 7f0452b04700 1 --2- 172.21.15.2:0/2688751024 >> [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579] conn(0x7f04380accb0 0x7f04380af150 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:21.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:21.988+0000 7f04437fe700 1 -- 172.21.15.2:0/2688751024 <== osd.8 v2:172.21.15.184:6816/2462722579 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f04380af820 con 0x7f04380accb0 2024-03-15T12:57:21.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:22.006+0000 7f0454567700 1 -- 172.21.15.2:0/2688751024 --> [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f044c0626f0 con 0x7f04380accb0 2024-03-15T12:57:21.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:22.006+0000 7f04437fe700 1 -- 172.21.15.2:0/2688751024 <== osd.8 v2:172.21.15.184:6816/2462722579 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f044c0626f0 con 0x7f04380accb0 2024-03-15T12:57:21.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:22.007+0000 7f0454567700 1 -- 172.21.15.2:0/2688751024 >> [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579] conn(0x7f04380accb0 msgr2=0x7f04380af150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:21.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:22.007+0000 7f0454567700 1 --2- 172.21.15.2:0/2688751024 >> [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579] conn(0x7f04380accb0 0x7f04380af150 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:22.007+0000 7f0454567700 1 -- 172.21.15.2:0/2688751024 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f04380a1bd0 msgr2=0x7f04380a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:21.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:22.007+0000 7f0454567700 1 --2- 172.21.15.2:0/2688751024 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f04380a1bd0 0x7f04380a4090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0448011550 tx=0x7f0448009260 comp rx=0 tx=0).stop 2024-03-15T12:57:21.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:22.007+0000 7f0454567700 1 -- 172.21.15.2:0/2688751024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f044c102bd0 msgr2=0x7f044c198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:21.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:22.007+0000 7f0454567700 1 --2- 172.21.15.2:0/2688751024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f044c102bd0 0x7f044c198b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f043c00e9c0 tx=0x7f043c00ecd0 comp rx=0 tx=0).stop 2024-03-15T12:57:21.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:22.007+0000 7f0452b04700 1 -- 172.21.15.2:0/2688751024 reap_dead start 2024-03-15T12:57:21.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:22.007+0000 7f0454567700 1 -- 172.21.15.2:0/2688751024 shutdown_connections 2024-03-15T12:57:21.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:22.007+0000 7f0454567700 1 -- 172.21.15.2:0/2688751024 >> 172.21.15.2:0/2688751024 conn(0x7f044c0fda90 msgr2=0x7f044c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:21.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:22.007+0000 7f0454567700 1 -- 172.21.15.2:0/2688751024 shutdown_connections 2024-03-15T12:57:21.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:22.007+0000 7f0454567700 1 -- 172.21.15.2:0/2688751024 wait complete. 2024-03-15T12:57:22.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:21 smithi002 ceph-mon[32060]: pgmap v322: 129 pgs: 2 activating, 2 peering, 125 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 13 B/s, 0 objects/s recovering 2024-03-15T12:57:22.006 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:21 smithi002 ceph-mon[32060]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-15T12:57:22.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:21 smithi149 ceph-mon[35927]: pgmap v322: 129 pgs: 2 activating, 2 peering, 125 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 13 B/s, 0 objects/s recovering 2024-03-15T12:57:22.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:21 smithi149 ceph-mon[35927]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-15T12:57:22.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:21 smithi184 ceph-mon[35993]: pgmap v322: 129 pgs: 2 activating, 2 peering, 125 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 13 B/s, 0 objects/s recovering 2024-03-15T12:57:22.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:21 smithi184 ceph-mon[35993]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-15T12:57:22.619 INFO:teuthology.orchestra.run.smithi002.stdout:253403070504 2024-03-15T12:57:22.619 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-15T12:57:23.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.051+0000 7f2655d38700 1 -- 172.21.15.2:0/3306076903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2650102030 msgr2=0x7f26501024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:23.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.051+0000 7f2655d38700 1 --2- 172.21.15.2:0/3306076903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2650102030 0x7f26501024b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f26440099c0 tx=0x7f2644009cd0 comp rx=0 tx=0).stop 2024-03-15T12:57:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.051+0000 7f2655d38700 1 -- 172.21.15.2:0/3306076903 shutdown_connections 2024-03-15T12:57:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.051+0000 7f2655d38700 1 --2- 172.21.15.2:0/3306076903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26501029f0 0x7f265010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.051+0000 7f2655d38700 1 --2- 172.21.15.2:0/3306076903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2650108010 0x7f26501083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.051+0000 7f2655d38700 1 --2- 172.21.15.2:0/3306076903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2650102030 0x7f26501024b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.051+0000 7f2655d38700 1 -- 172.21.15.2:0/3306076903 >> 172.21.15.2:0/3306076903 conn(0x7f2650079110 msgr2=0x7f2650079520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.051+0000 7f2655d38700 1 -- 172.21.15.2:0/3306076903 shutdown_connections 2024-03-15T12:57:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.051+0000 7f2655d38700 1 -- 172.21.15.2:0/3306076903 wait complete. 2024-03-15T12:57:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.052+0000 7f2655d38700 1 Processor -- start 2024-03-15T12:57:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.052+0000 7f2655d38700 1 -- start start 2024-03-15T12:57:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.052+0000 7f2655d38700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2650102030 0x7f26501984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.052+0000 7f2655d38700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26501029f0 0x7f26501989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.052+0000 7f2655d38700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2650108010 0x7f265019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.052+0000 7f2655d38700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265019d380 con 0x7f2650108010 2024-03-15T12:57:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.052+0000 7f2655d38700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265019d4f0 con 0x7f26501029f0 2024-03-15T12:57:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.052+0000 7f2655d38700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265019d6b0 con 0x7f2650102030 2024-03-15T12:57:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2650108010 0x7f265019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26501029f0 0x7f26501989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2650102030 0x7f26501984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26501029f0 0x7f26501989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:53620/0 (socket says 172.21.15.2:53620) 2024-03-15T12:57:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2650108010 0x7f265019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50370/0 (socket says 172.21.15.2:50370) 2024-03-15T12:57:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264effd700 1 -- 172.21.15.2:0/2488488557 learned_addr learned my addr 172.21.15.2:0/2488488557 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264effd700 1 -- 172.21.15.2:0/2488488557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2650102030 msgr2=0x7f26501984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264effd700 1 --2- 172.21.15.2:0/2488488557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2650102030 0x7f26501984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264effd700 1 -- 172.21.15.2:0/2488488557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2650108010 msgr2=0x7f265019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264effd700 1 --2- 172.21.15.2:0/2488488557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2650108010 0x7f265019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264effd700 1 -- 172.21.15.2:0/2488488557 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26440095f0 con 0x7f26501029f0 2024-03-15T12:57:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264f7fe700 1 --2- 172.21.15.2:0/2488488557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2650102030 0x7f26501984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264ffff700 1 --2- 172.21.15.2:0/2488488557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2650108010 0x7f265019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264effd700 1 --2- 172.21.15.2:0/2488488557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26501029f0 0x7f26501989e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2644005910 tx=0x7f2644005260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264cff9700 1 -- 172.21.15.2:0/2488488557 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264401d070 con 0x7f26501029f0 2024-03-15T12:57:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264cff9700 1 -- 172.21.15.2:0/2488488557 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2644003680 con 0x7f26501029f0 2024-03-15T12:57:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f2655d38700 1 -- 172.21.15.2:0/2488488557 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265019d930 con 0x7f26501029f0 2024-03-15T12:57:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f264cff9700 1 -- 172.21.15.2:0/2488488557 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264400f9a0 con 0x7f26501029f0 2024-03-15T12:57:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.053+0000 7f2655d38700 1 -- 172.21.15.2:0/2488488557 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26501a51f0 con 0x7f26501029f0 2024-03-15T12:57:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.054+0000 7f2655d38700 1 -- 172.21.15.2:0/2488488557 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f26501036c0 con 0x7f26501029f0 2024-03-15T12:57:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.055+0000 7f264cff9700 1 -- 172.21.15.2:0/2488488557 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f264400b790 con 0x7f26501029f0 2024-03-15T12:57:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.056+0000 7f264cff9700 1 --2- 172.21.15.2:0/2488488557 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f26380a1bd0 0x7f26380a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.056+0000 7f264f7fe700 1 --2- 172.21.15.2:0/2488488557 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f26380a1bd0 0x7f26380a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.056+0000 7f264cff9700 1 -- 172.21.15.2:0/2488488557 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f26440ce5a0 con 0x7f26501029f0 2024-03-15T12:57:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.056+0000 7f264cff9700 1 --2- 172.21.15.2:0/2488488557 >> [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987] conn(0x7f26380accb0 0x7f26380af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.056+0000 7f264cff9700 1 -- 172.21.15.2:0/2488488557 --> [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26380af820 con 0x7f26380accb0 2024-03-15T12:57:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.056+0000 7f264cff9700 1 -- 172.21.15.2:0/2488488557 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26440ce910 con 0x7f26501029f0 2024-03-15T12:57:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.056+0000 7f264f7fe700 1 --2- 172.21.15.2:0/2488488557 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f26380a1bd0 0x7f26380a4090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2640011490 tx=0x7f2640009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.056+0000 7f264ffff700 1 --2- 172.21.15.2:0/2488488557 >> [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987] conn(0x7f26380accb0 0x7f26380af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.057+0000 7f264ffff700 1 --2- 172.21.15.2:0/2488488557 >> [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987] conn(0x7f26380accb0 0x7f26380af150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.057+0000 7f264cff9700 1 -- 172.21.15.2:0/2488488557 <== osd.1 v2:172.21.15.149:6800/3295500987 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f26380af820 con 0x7f26380accb0 2024-03-15T12:57:23.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.084+0000 7f2655d38700 1 -- 172.21.15.2:0/2488488557 --> [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2650112360 con 0x7f26380accb0 2024-03-15T12:57:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.085+0000 7f264cff9700 1 -- 172.21.15.2:0/2488488557 <== osd.1 v2:172.21.15.149:6800/3295500987 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2650112360 con 0x7f26380accb0 2024-03-15T12:57:23.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.086+0000 7f2655d38700 1 -- 172.21.15.2:0/2488488557 >> [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987] conn(0x7f26380accb0 msgr2=0x7f26380af150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:23.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.086+0000 7f2655d38700 1 --2- 172.21.15.2:0/2488488557 >> [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987] conn(0x7f26380accb0 0x7f26380af150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.086+0000 7f2655d38700 1 -- 172.21.15.2:0/2488488557 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f26380a1bd0 msgr2=0x7f26380a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.086+0000 7f2655d38700 1 --2- 172.21.15.2:0/2488488557 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f26380a1bd0 0x7f26380a4090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2640011490 tx=0x7f2640009260 comp rx=0 tx=0).stop 2024-03-15T12:57:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.086+0000 7f2655d38700 1 -- 172.21.15.2:0/2488488557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26501029f0 msgr2=0x7f26501989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.086+0000 7f2655d38700 1 --2- 172.21.15.2:0/2488488557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26501029f0 0x7f26501989e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2644005910 tx=0x7f2644005260 comp rx=0 tx=0).stop 2024-03-15T12:57:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.086+0000 7f264ffff700 1 -- 172.21.15.2:0/2488488557 reap_dead start 2024-03-15T12:57:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.087+0000 7f2655d38700 1 -- 172.21.15.2:0/2488488557 shutdown_connections 2024-03-15T12:57:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.087+0000 7f2655d38700 1 -- 172.21.15.2:0/2488488557 >> 172.21.15.2:0/2488488557 conn(0x7f2650079110 msgr2=0x7f2650100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.087+0000 7f2655d38700 1 -- 172.21.15.2:0/2488488557 shutdown_connections 2024-03-15T12:57:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:23.087+0000 7f2655d38700 1 -- 172.21.15.2:0/2488488557 wait complete. 2024-03-15T12:57:23.592 INFO:teuthology.orchestra.run.smithi002.stdout:107374182465 2024-03-15T12:57:23.593 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-15T12:57:23.944 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:57:23.944 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:57:23.944 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:57:23.945 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:57:23.945 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:57:23.946 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:57:23.950 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:57:23.950 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:57:23.962 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:57:24.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:23 smithi149 ceph-mon[35927]: pgmap v323: 129 pgs: 2 activating, 1 peering, 126 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 13 B/s, 0 objects/s recovering 2024-03-15T12:57:24.110 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:23 smithi002 ceph-mon[32060]: pgmap v323: 129 pgs: 2 activating, 1 peering, 126 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 13 B/s, 0 objects/s recovering 2024-03-15T12:57:24.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:23 smithi184 ceph-mon[35993]: pgmap v323: 129 pgs: 2 activating, 1 peering, 126 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 13 B/s, 0 objects/s recovering 2024-03-15T12:57:24.423 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:57:24.433 INFO:teuthology.orchestra.run.smithi002.stdout:55834574924 2024-03-15T12:57:24.433 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-15T12:57:26.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:25 smithi149 ceph-mon[35927]: pgmap v324: 129 pgs: 2 activating, 127 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 13 B/s, 0 objects/s recovering 2024-03-15T12:57:26.175 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:25 smithi002 ceph-mon[32060]: pgmap v324: 129 pgs: 2 activating, 127 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 13 B/s, 0 objects/s recovering 2024-03-15T12:57:26.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:25 smithi184 ceph-mon[35993]: pgmap v324: 129 pgs: 2 activating, 127 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 13 B/s, 0 objects/s recovering 2024-03-15T12:57:27.023 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:26 smithi002 ceph-mon[32060]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive) 2024-03-15T12:57:27.024 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:26 smithi002 ceph-mon[32060]: Cluster is now healthy 2024-03-15T12:57:27.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:26 smithi184 ceph-mon[35993]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive) 2024-03-15T12:57:27.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:26 smithi184 ceph-mon[35993]: Cluster is now healthy 2024-03-15T12:57:27.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:26 smithi149 ceph-mon[35927]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive) 2024-03-15T12:57:27.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:26 smithi149 ceph-mon[35927]: Cluster is now healthy 2024-03-15T12:57:28.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:27 smithi002 ceph-mon[32060]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 13 B/s, 0 objects/s recovering 2024-03-15T12:57:28.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:27 smithi184 ceph-mon[35993]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 13 B/s, 0 objects/s recovering 2024-03-15T12:57:28.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:27 smithi149 ceph-mon[35927]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 13 B/s, 0 objects/s recovering 2024-03-15T12:57:30.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:29 smithi002 ceph-mon[32060]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 11 B/s, 0 objects/s recovering 2024-03-15T12:57:30.162 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:29 smithi149 ceph-mon[35927]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 11 B/s, 0 objects/s recovering 2024-03-15T12:57:30.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:29 smithi184 ceph-mon[35993]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 11 B/s, 0 objects/s recovering 2024-03-15T12:57:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.199+0000 7f677f90c700 1 -- 172.21.15.2:0/305880428 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6778102a20 msgr2=0x7f677810eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.199+0000 7f677f90c700 1 --2- 172.21.15.2:0/305880428 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6778102a20 0x7f677810eeb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f677000b0a0 tx=0x7f677000b3b0 comp rx=0 tx=0).stop 2024-03-15T12:57:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.199+0000 7f677f90c700 1 -- 172.21.15.2:0/305880428 shutdown_connections 2024-03-15T12:57:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.199+0000 7f677f90c700 1 --2- 172.21.15.2:0/305880428 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6778102a20 0x7f677810eeb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.199+0000 7f677f90c700 1 --2- 172.21.15.2:0/305880428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67781080b0 0x7f6778108490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.199+0000 7f677f90c700 1 --2- 172.21.15.2:0/305880428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6778102060 0x7f67781024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.199+0000 7f677f90c700 1 -- 172.21.15.2:0/305880428 >> 172.21.15.2:0/305880428 conn(0x7f6778079130 msgr2=0x7f6778079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.199+0000 7f677f90c700 1 -- 172.21.15.2:0/305880428 shutdown_connections 2024-03-15T12:57:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.199+0000 7f677f90c700 1 -- 172.21.15.2:0/305880428 wait complete. 2024-03-15T12:57:31.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677f90c700 1 Processor -- start 2024-03-15T12:57:31.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677f90c700 1 -- start start 2024-03-15T12:57:31.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677f90c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6778102060 0x7f67781a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:31.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677f90c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6778102a20 0x7f67781a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:31.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677f90c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67781080b0 0x7f677819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:31.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677f90c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677819ac40 con 0x7f6778102060 2024-03-15T12:57:31.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677f90c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677819ad80 con 0x7f67781080b0 2024-03-15T12:57:31.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677f90c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677819af40 con 0x7f6778102a20 2024-03-15T12:57:31.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677d6a8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6778102060 0x7f67781a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:31.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677d6a8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6778102060 0x7f67781a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53194/0 (socket says 172.21.15.2:53194) 2024-03-15T12:57:31.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677d6a8700 1 -- 172.21.15.2:0/3380285404 learned_addr learned my addr 172.21.15.2:0/3380285404 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:31.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677cea7700 1 --2- 172.21.15.2:0/3380285404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6778102a20 0x7f67781a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:31.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677dea9700 1 --2- 172.21.15.2:0/3380285404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67781080b0 0x7f677819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:31.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677d6a8700 1 -- 172.21.15.2:0/3380285404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6778102a20 msgr2=0x7f67781a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:31.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677d6a8700 1 --2- 172.21.15.2:0/3380285404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6778102a20 0x7f67781a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:31.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677d6a8700 1 -- 172.21.15.2:0/3380285404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67781080b0 msgr2=0x7f677819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:31.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677d6a8700 1 --2- 172.21.15.2:0/3380285404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67781080b0 0x7f677819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:31.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.200+0000 7f677d6a8700 1 -- 172.21.15.2:0/3380285404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6770009d00 con 0x7f6778102060 2024-03-15T12:57:31.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.201+0000 7f677d6a8700 1 --2- 172.21.15.2:0/3380285404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6778102060 0x7f67781a0420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f676c00e9a0 tx=0x7f676c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:31.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.201+0000 7f676a7fc700 1 -- 172.21.15.2:0/3380285404 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c009790 con 0x7f6778102060 2024-03-15T12:57:31.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.201+0000 7f676a7fc700 1 -- 172.21.15.2:0/3380285404 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f676c010430 con 0x7f6778102060 2024-03-15T12:57:31.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.201+0000 7f676a7fc700 1 -- 172.21.15.2:0/3380285404 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c018660 con 0x7f6778102060 2024-03-15T12:57:31.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.201+0000 7f677f90c700 1 -- 172.21.15.2:0/3380285404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677819b220 con 0x7f6778102060 2024-03-15T12:57:31.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.201+0000 7f677f90c700 1 -- 172.21.15.2:0/3380285404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67781a74c0 con 0x7f6778102060 2024-03-15T12:57:31.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.202+0000 7f677f90c700 1 -- 172.21.15.2:0/3380285404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6778068ca0 con 0x7f6778102060 2024-03-15T12:57:31.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.203+0000 7f676a7fc700 1 -- 172.21.15.2:0/3380285404 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f676c0105a0 con 0x7f6778102060 2024-03-15T12:57:31.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.203+0000 7f676a7fc700 1 --2- 172.21.15.2:0/3380285404 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f67640a1bd0 0x7f67640a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:31.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.204+0000 7f677cea7700 1 --2- 172.21.15.2:0/3380285404 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f67640a1bd0 0x7f67640a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:31.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.204+0000 7f676a7fc700 1 -- 172.21.15.2:0/3380285404 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f676c014070 con 0x7f6778102060 2024-03-15T12:57:31.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.204+0000 7f677cea7700 1 --2- 172.21.15.2:0/3380285404 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f67640a1bd0 0x7f67640a4090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f67780799e0 tx=0x7f6774009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:31.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.208+0000 7f676a7fc700 1 -- 172.21.15.2:0/3380285404 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f676c053280 con 0x7f6778102060 2024-03-15T12:57:31.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.416+0000 7f677f90c700 1 -- 172.21.15.2:0/3380285404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f677819bb10 con 0x7f6778102060 2024-03-15T12:57:31.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.416+0000 7f676a7fc700 1 -- 172.21.15.2:0/3380285404 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f676c096720 con 0x7f6778102060 2024-03-15T12:57:31.403 INFO:teuthology.orchestra.run.smithi002.stdout:287762808870 2024-03-15T12:57:31.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.418+0000 7f677f90c700 1 -- 172.21.15.2:0/3380285404 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f67640a1bd0 msgr2=0x7f67640a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:31.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.418+0000 7f677f90c700 1 --2- 172.21.15.2:0/3380285404 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f67640a1bd0 0x7f67640a4090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f67780799e0 tx=0x7f6774009260 comp rx=0 tx=0).stop 2024-03-15T12:57:31.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.418+0000 7f677f90c700 1 -- 172.21.15.2:0/3380285404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6778102060 msgr2=0x7f67781a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:31.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.418+0000 7f677f90c700 1 --2- 172.21.15.2:0/3380285404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6778102060 0x7f67781a0420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f676c00e9a0 tx=0x7f676c00ecb0 comp rx=0 tx=0).stop 2024-03-15T12:57:31.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.419+0000 7f677f90c700 1 -- 172.21.15.2:0/3380285404 shutdown_connections 2024-03-15T12:57:31.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.419+0000 7f677f90c700 1 --2- 172.21.15.2:0/3380285404 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f67640a1bd0 0x7f67640a4090 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:31.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.419+0000 7f677f90c700 1 --2- 172.21.15.2:0/3380285404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6778102060 0x7f67781a0420 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:31.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.419+0000 7f677f90c700 1 --2- 172.21.15.2:0/3380285404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67781080b0 0x7f677819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:31.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.419+0000 7f677f90c700 1 --2- 172.21.15.2:0/3380285404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6778102a20 0x7f67781a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:31.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.419+0000 7f677f90c700 1 -- 172.21.15.2:0/3380285404 >> 172.21.15.2:0/3380285404 conn(0x7f6778079130 msgr2=0x7f677810d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:31.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.419+0000 7f677f90c700 1 -- 172.21.15.2:0/3380285404 shutdown_connections 2024-03-15T12:57:31.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:31.419+0000 7f677f90c700 1 -- 172.21.15.2:0/3380285404 wait complete. 2024-03-15T12:57:32.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:31 smithi149 ceph-mon[35927]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s; 10 B/s, 0 objects/s recovering 2024-03-15T12:57:32.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:31 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3380285404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-15T12:57:32.291 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:31 smithi002 ceph-mon[32060]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s; 10 B/s, 0 objects/s recovering 2024-03-15T12:57:32.291 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:31 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3380285404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-15T12:57:32.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:31 smithi184 ceph-mon[35993]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s; 10 B/s, 0 objects/s recovering 2024-03-15T12:57:32.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:31 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3380285404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-15T12:57:33.879 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808866 got 287762808870 for osd.9 2024-03-15T12:57:33.880 DEBUG:teuthology.parallel:result is None 2024-03-15T12:57:34.152 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:33 smithi002 ceph-mon[32060]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:57:34.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:33 smithi184 ceph-mon[35993]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:57:34.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:33 smithi149 ceph-mon[35927]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:57:35.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.759+0000 7f52fd8fd700 1 -- 172.21.15.2:0/2252239909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52f8108010 msgr2=0x7f52f81083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.759+0000 7f52fd8fd700 1 --2- 172.21.15.2:0/2252239909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52f8108010 0x7f52f81083f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f52e00098f0 tx=0x7f52e0009c00 comp rx=0 tx=0).stop 2024-03-15T12:57:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.759+0000 7f52fd8fd700 1 -- 172.21.15.2:0/2252239909 shutdown_connections 2024-03-15T12:57:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.759+0000 7f52fd8fd700 1 --2- 172.21.15.2:0/2252239909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52f81029f0 0x7f52f810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.759+0000 7f52fd8fd700 1 --2- 172.21.15.2:0/2252239909 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52f8108010 0x7f52f81083f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.759+0000 7f52fd8fd700 1 --2- 172.21.15.2:0/2252239909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52f8102030 0x7f52f81024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.759+0000 7f52fd8fd700 1 -- 172.21.15.2:0/2252239909 >> 172.21.15.2:0/2252239909 conn(0x7f52f8079110 msgr2=0x7f52f8079520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.759+0000 7f52fd8fd700 1 -- 172.21.15.2:0/2252239909 shutdown_connections 2024-03-15T12:57:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.759+0000 7f52fd8fd700 1 -- 172.21.15.2:0/2252239909 wait complete. 2024-03-15T12:57:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.759+0000 7f52fd8fd700 1 Processor -- start 2024-03-15T12:57:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52fd8fd700 1 -- start start 2024-03-15T12:57:35.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52fd8fd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52f8102030 0x7f52f81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:35.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52fd8fd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52f81029f0 0x7f52f81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:35.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52fd8fd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52f8108010 0x7f52f819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:35.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52fd8fd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f819d390 con 0x7f52f8102030 2024-03-15T12:57:35.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52fd8fd700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f819d500 con 0x7f52f8108010 2024-03-15T12:57:35.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52fd8fd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f819d6c0 con 0x7f52f81029f0 2024-03-15T12:57:35.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52f6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52f8102030 0x7f52f81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52f67fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52f81029f0 0x7f52f81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52f6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52f8102030 0x7f52f81984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53218/0 (socket says 172.21.15.2:53218) 2024-03-15T12:57:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52f6ffd700 1 -- 172.21.15.2:0/1217248242 learned_addr learned my addr 172.21.15.2:0/1217248242 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52f77fe700 1 --2- 172.21.15.2:0/1217248242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52f8108010 0x7f52f819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52f67fc700 1 -- 172.21.15.2:0/1217248242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52f8108010 msgr2=0x7f52f819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52f67fc700 1 --2- 172.21.15.2:0/1217248242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52f8108010 0x7f52f819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52f67fc700 1 -- 172.21.15.2:0/1217248242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52f8102030 msgr2=0x7f52f81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52f67fc700 1 --2- 172.21.15.2:0/1217248242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52f8102030 0x7f52f81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52f67fc700 1 -- 172.21.15.2:0/1217248242 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f0009d00 con 0x7f52f81029f0 2024-03-15T12:57:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52f6ffd700 1 --2- 172.21.15.2:0/1217248242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52f8102030 0x7f52f81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.760+0000 7f52f67fc700 1 --2- 172.21.15.2:0/1217248242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52f81029f0 0x7f52f81989f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f52e800e9f0 tx=0x7f52e800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.761+0000 7f52fc8fb700 1 -- 172.21.15.2:0/1217248242 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e800c9c0 con 0x7f52f81029f0 2024-03-15T12:57:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.761+0000 7f52fc8fb700 1 -- 172.21.15.2:0/1217248242 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f52e8010430 con 0x7f52f81029f0 2024-03-15T12:57:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.761+0000 7f52fc8fb700 1 -- 172.21.15.2:0/1217248242 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8018640 con 0x7f52f81029f0 2024-03-15T12:57:35.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.761+0000 7f52fd8fd700 1 -- 172.21.15.2:0/1217248242 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e0009520 con 0x7f52f81029f0 2024-03-15T12:57:35.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.761+0000 7f52fd8fd700 1 -- 172.21.15.2:0/1217248242 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f81a5030 con 0x7f52f81029f0 2024-03-15T12:57:35.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.762+0000 7f52fd8fd700 1 -- 172.21.15.2:0/1217248242 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f810a4f0 con 0x7f52f81029f0 2024-03-15T12:57:35.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.764+0000 7f52fc8fb700 1 -- 172.21.15.2:0/1217248242 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f52e80105a0 con 0x7f52f81029f0 2024-03-15T12:57:35.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.764+0000 7f52fc8fb700 1 --2- 172.21.15.2:0/1217248242 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f52e40a1bd0 0x7f52e40a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:35.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.764+0000 7f52f6ffd700 1 --2- 172.21.15.2:0/1217248242 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f52e40a1bd0 0x7f52e40a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:35.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.764+0000 7f52fc8fb700 1 -- 172.21.15.2:0/1217248242 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f52e8014070 con 0x7f52f81029f0 2024-03-15T12:57:35.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.765+0000 7f52f6ffd700 1 --2- 172.21.15.2:0/1217248242 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f52e40a1bd0 0x7f52e40a4090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f52e00098c0 tx=0x7f52e0019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:35.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.767+0000 7f52fc8fb700 1 -- 172.21.15.2:0/1217248242 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f52e8096650 con 0x7f52f81029f0 2024-03-15T12:57:35.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.978+0000 7f52fd8fd700 1 -- 172.21.15.2:0/1217248242 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f52f8068ca0 con 0x7f52f81029f0 2024-03-15T12:57:35.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.979+0000 7f52fc8fb700 1 -- 172.21.15.2:0/1217248242 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f52e8099d10 con 0x7f52f81029f0 2024-03-15T12:57:35.965 INFO:teuthology.orchestra.run.smithi002.stdout:77309411401 2024-03-15T12:57:35.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.981+0000 7f52fd8fd700 1 -- 172.21.15.2:0/1217248242 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f52e40a1bd0 msgr2=0x7f52e40a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:35.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.981+0000 7f52fd8fd700 1 --2- 172.21.15.2:0/1217248242 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f52e40a1bd0 0x7f52e40a4090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f52e00098c0 tx=0x7f52e0019040 comp rx=0 tx=0).stop 2024-03-15T12:57:35.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.981+0000 7f52fd8fd700 1 -- 172.21.15.2:0/1217248242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52f81029f0 msgr2=0x7f52f81989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:35.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.981+0000 7f52fd8fd700 1 --2- 172.21.15.2:0/1217248242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52f81029f0 0x7f52f81989f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f52e800e9f0 tx=0x7f52e800ed00 comp rx=0 tx=0).stop 2024-03-15T12:57:35.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.981+0000 7f52fd8fd700 1 -- 172.21.15.2:0/1217248242 shutdown_connections 2024-03-15T12:57:35.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.981+0000 7f52fd8fd700 1 --2- 172.21.15.2:0/1217248242 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f52e40a1bd0 0x7f52e40a4090 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:35.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.981+0000 7f52fd8fd700 1 --2- 172.21.15.2:0/1217248242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52f8102030 0x7f52f81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:35.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.981+0000 7f52fd8fd700 1 --2- 172.21.15.2:0/1217248242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52f8108010 0x7f52f819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:35.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.981+0000 7f52fd8fd700 1 --2- 172.21.15.2:0/1217248242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52f81029f0 0x7f52f81989f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:35.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.981+0000 7f52fd8fd700 1 -- 172.21.15.2:0/1217248242 >> 172.21.15.2:0/1217248242 conn(0x7f52f8079110 msgr2=0x7f52f8100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:35.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.981+0000 7f52fd8fd700 1 -- 172.21.15.2:0/1217248242 shutdown_connections 2024-03-15T12:57:35.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.981+0000 7f52fd8fd700 1 -- 172.21.15.2:0/1217248242 wait complete. 2024-03-15T12:57:36.128 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:35 smithi002 ceph-mon[32060]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-15T12:57:36.128 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:35 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/4151595714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-15T12:57:36.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:35 smithi184 ceph-mon[35993]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-15T12:57:36.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:35 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/4151595714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-15T12:57:36.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:35 smithi149 ceph-mon[35927]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-15T12:57:36.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:35 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/4151595714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-15T12:57:36.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.607+0000 7f112516c700 1 -- 172.21.15.2:0/79073851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1120106700 msgr2=0x7f1120106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:36.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.607+0000 7f112516c700 1 --2- 172.21.15.2:0/79073851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1120106700 0x7f1120106ae0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f11100099c0 tx=0x7f1110009cd0 comp rx=0 tx=0).stop 2024-03-15T12:57:36.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.607+0000 7f112516c700 1 -- 172.21.15.2:0/79073851 shutdown_connections 2024-03-15T12:57:36.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.607+0000 7f112516c700 1 --2- 172.21.15.2:0/79073851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1120106700 0x7f1120106ae0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:36.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.607+0000 7f112516c700 1 --2- 172.21.15.2:0/79073851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11201012e0 0x7f112010d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:36.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.607+0000 7f112516c700 1 --2- 172.21.15.2:0/79073851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1120100920 0x7f1120100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:36.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.607+0000 7f112516c700 1 -- 172.21.15.2:0/79073851 >> 172.21.15.2:0/79073851 conn(0x7f1120076240 msgr2=0x7f1120076650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:36.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.607+0000 7f112516c700 1 -- 172.21.15.2:0/79073851 shutdown_connections 2024-03-15T12:57:36.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.607+0000 7f112516c700 1 -- 172.21.15.2:0/79073851 wait complete. 2024-03-15T12:57:36.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.608+0000 7f112516c700 1 Processor -- start 2024-03-15T12:57:36.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.608+0000 7f112516c700 1 -- start start 2024-03-15T12:57:36.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.608+0000 7f112516c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1120100920 0x7f1120198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:36.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.608+0000 7f112516c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11201012e0 0x7f1120198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:36.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f112516c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1120106700 0x7f112019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:36.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f112516c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112019d450 con 0x7f1120100920 2024-03-15T12:57:36.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f112516c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112019d5c0 con 0x7f1120106700 2024-03-15T12:57:36.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f112516c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112019d780 con 0x7f11201012e0 2024-03-15T12:57:36.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f111ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1120100920 0x7f1120198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:36.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f111e59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11201012e0 0x7f1120198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f111ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1120100920 0x7f1120198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53244/0 (socket says 172.21.15.2:53244) 2024-03-15T12:57:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f111ed9d700 1 -- 172.21.15.2:0/1946590270 learned_addr learned my addr 172.21.15.2:0/1946590270 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f111f59e700 1 --2- 172.21.15.2:0/1946590270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1120106700 0x7f112019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f111ed9d700 1 -- 172.21.15.2:0/1946590270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11201012e0 msgr2=0x7f1120198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f111ed9d700 1 --2- 172.21.15.2:0/1946590270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11201012e0 0x7f1120198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f111ed9d700 1 -- 172.21.15.2:0/1946590270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1120106700 msgr2=0x7f112019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f111ed9d700 1 --2- 172.21.15.2:0/1946590270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1120106700 0x7f112019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f111ed9d700 1 -- 172.21.15.2:0/1946590270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11100095f0 con 0x7f1120100920 2024-03-15T12:57:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f111e59c700 1 --2- 172.21.15.2:0/1946590270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11201012e0 0x7f1120198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f111f59e700 1 --2- 172.21.15.2:0/1946590270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1120106700 0x7f112019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f111ed9d700 1 --2- 172.21.15.2:0/1946590270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1120100920 0x7f1120198570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1110006010 tx=0x7f1110005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f1117fff700 1 -- 172.21.15.2:0/1946590270 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111001d070 con 0x7f1120100920 2024-03-15T12:57:36.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f112516c700 1 -- 172.21.15.2:0/1946590270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112019da00 con 0x7f1120100920 2024-03-15T12:57:36.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f1117fff700 1 -- 172.21.15.2:0/1946590270 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1110003680 con 0x7f1120100920 2024-03-15T12:57:36.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f1117fff700 1 -- 172.21.15.2:0/1946590270 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111000fab0 con 0x7f1120100920 2024-03-15T12:57:36.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.609+0000 7f112516c700 1 -- 172.21.15.2:0/1946590270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11201a52c0 con 0x7f1120100920 2024-03-15T12:57:36.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.610+0000 7f112516c700 1 -- 172.21.15.2:0/1946590270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1120109c10 con 0x7f1120100920 2024-03-15T12:57:36.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.611+0000 7f1117fff700 1 -- 172.21.15.2:0/1946590270 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f111000b730 con 0x7f1120100920 2024-03-15T12:57:36.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.612+0000 7f1117fff700 1 --2- 172.21.15.2:0/1946590270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f110c0a1b50 0x7f110c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:36.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.612+0000 7f111e59c700 1 --2- 172.21.15.2:0/1946590270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f110c0a1b50 0x7f110c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:36.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.612+0000 7f1117fff700 1 -- 172.21.15.2:0/1946590270 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f11100cebf0 con 0x7f1120100920 2024-03-15T12:57:36.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.613+0000 7f111e59c700 1 --2- 172.21.15.2:0/1946590270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f110c0a1b50 0x7f110c0a4010 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1120076960 tx=0x7f1108009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:36.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.616+0000 7f1117fff700 1 -- 172.21.15.2:0/1946590270 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f11100979f0 con 0x7f1120100920 2024-03-15T12:57:36.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.820+0000 7f112516c700 1 -- 172.21.15.2:0/1946590270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f11201998d0 con 0x7f1120100920 2024-03-15T12:57:36.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.820+0000 7f1117fff700 1 -- 172.21.15.2:0/1946590270 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f111009b0b0 con 0x7f1120100920 2024-03-15T12:57:36.807 INFO:teuthology.orchestra.run.smithi002.stdout:163208757307 2024-03-15T12:57:36.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.823+0000 7f112516c700 1 -- 172.21.15.2:0/1946590270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f110c0a1b50 msgr2=0x7f110c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:36.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.823+0000 7f112516c700 1 --2- 172.21.15.2:0/1946590270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f110c0a1b50 0x7f110c0a4010 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1120076960 tx=0x7f1108009260 comp rx=0 tx=0).stop 2024-03-15T12:57:36.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.823+0000 7f112516c700 1 -- 172.21.15.2:0/1946590270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1120100920 msgr2=0x7f1120198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:36.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.823+0000 7f112516c700 1 --2- 172.21.15.2:0/1946590270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1120100920 0x7f1120198570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1110006010 tx=0x7f1110005280 comp rx=0 tx=0).stop 2024-03-15T12:57:36.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.823+0000 7f112516c700 1 -- 172.21.15.2:0/1946590270 shutdown_connections 2024-03-15T12:57:36.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.823+0000 7f112516c700 1 --2- 172.21.15.2:0/1946590270 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f110c0a1b50 0x7f110c0a4010 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:36.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.823+0000 7f112516c700 1 --2- 172.21.15.2:0/1946590270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1120100920 0x7f1120198570 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:36.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.823+0000 7f112516c700 1 --2- 172.21.15.2:0/1946590270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1120106700 0x7f112019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:36.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.823+0000 7f112516c700 1 --2- 172.21.15.2:0/1946590270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11201012e0 0x7f1120198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:36.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.823+0000 7f112516c700 1 -- 172.21.15.2:0/1946590270 >> 172.21.15.2:0/1946590270 conn(0x7f1120076240 msgr2=0x7f112010e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:36.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.824+0000 7f112516c700 1 -- 172.21.15.2:0/1946590270 shutdown_connections 2024-03-15T12:57:36.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:36.824+0000 7f112516c700 1 -- 172.21.15.2:0/1946590270 wait complete. 2024-03-15T12:57:37.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:36 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1217248242' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-15T12:57:37.134 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:36 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1946590270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-15T12:57:37.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:36 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1217248242' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-15T12:57:37.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:36 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1946590270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-15T12:57:37.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:36 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1217248242' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-15T12:57:37.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:36 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1946590270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-15T12:57:37.769 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411401 for osd.2 2024-03-15T12:57:37.770 DEBUG:teuthology.parallel:result is None 2024-03-15T12:57:37.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:37 smithi002 ceph-mon[32060]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:57:38.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.657+0000 7f5ecfd8b700 1 -- 172.21.15.2:0/518602653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec81019e0 msgr2=0x7f5ec8101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:38.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.657+0000 7f5ecfd8b700 1 --2- 172.21.15.2:0/518602653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec81019e0 0x7f5ec8101dc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009990 tx=0x7f5ec4009ca0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.657+0000 7f5ecfd8b700 1 -- 172.21.15.2:0/518602653 shutdown_connections 2024-03-15T12:57:38.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.657+0000 7f5ecfd8b700 1 --2- 172.21.15.2:0/518602653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec81019e0 0x7f5ec8101dc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.657+0000 7f5ecfd8b700 1 --2- 172.21.15.2:0/518602653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec810ea80 0x7f5ec8110e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.657+0000 7f5ecfd8b700 1 --2- 172.21.15.2:0/518602653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec8102300 0x7f5ec810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.657+0000 7f5ecfd8b700 1 -- 172.21.15.2:0/518602653 >> 172.21.15.2:0/518602653 conn(0x7f5ec8077e50 msgr2=0x7f5ec8078260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:38.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.657+0000 7f5ecfd8b700 1 -- 172.21.15.2:0/518602653 shutdown_connections 2024-03-15T12:57:38.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.657+0000 7f5ecfd8b700 1 -- 172.21.15.2:0/518602653 wait complete. 2024-03-15T12:57:38.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.658+0000 7f5ecfd8b700 1 Processor -- start 2024-03-15T12:57:38.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.658+0000 7f5ecfd8b700 1 -- start start 2024-03-15T12:57:38.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.658+0000 7f5ecfd8b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec81019e0 0x7f5ec81a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:38.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.658+0000 7f5ecfd8b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec8102300 0x7f5ec81a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:38.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.658+0000 7f5ecfd8b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec810ea80 0x7f5ec819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:38.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.658+0000 7f5ecfd8b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec819adf0 con 0x7f5ec810ea80 2024-03-15T12:57:38.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.658+0000 7f5ecfd8b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec819af60 con 0x7f5ec8102300 2024-03-15T12:57:38.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.658+0000 7f5ecfd8b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec819b120 con 0x7f5ec81019e0 2024-03-15T12:57:38.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.658+0000 7f5ecd326700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec8102300 0x7f5ec81a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:38.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.658+0000 7f5ecd326700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec8102300 0x7f5ec81a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:43832/0 (socket says 172.21.15.2:43832) 2024-03-15T12:57:38.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.658+0000 7f5ecd326700 1 -- 172.21.15.2:0/4151595714 learned_addr learned my addr 172.21.15.2:0/4151595714 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:38.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.659+0000 7f5ecdb27700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec81019e0 0x7f5ec81a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:38.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.659+0000 7f5ece328700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec810ea80 0x7f5ec819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:38.078 INFO:teuthology.orchestra.run.smithi002.stdout:343597383708 2024-03-15T12:57:38.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.659+0000 7f5ecd326700 1 -- 172.21.15.2:0/4151595714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec81019e0 msgr2=0x7f5ec81a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:38.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.659+0000 7f5ecd326700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec81019e0 0x7f5ec81a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.659+0000 7f5ecd326700 1 -- 172.21.15.2:0/4151595714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec810ea80 msgr2=0x7f5ec819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:38.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.659+0000 7f5ecd326700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec810ea80 0x7f5ec819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.659+0000 7f5ecd326700 1 -- 172.21.15.2:0/4151595714 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec40095f0 con 0x7f5ec8102300 2024-03-15T12:57:38.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.659+0000 7f5ecdb27700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec81019e0 0x7f5ec81a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:38.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.659+0000 7f5ece328700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec810ea80 0x7f5ec819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:57:38.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.659+0000 7f5ecd326700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec8102300 0x7f5ec81a0ab0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800b570 tx=0x7f5eb800b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:38.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.660+0000 7f5ebeffd700 1 -- 172.21.15.2:0/4151595714 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8019070 con 0x7f5ec8102300 2024-03-15T12:57:38.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.660+0000 7f5ecfd8b700 1 -- 172.21.15.2:0/4151595714 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec819b3a0 con 0x7f5ec8102300 2024-03-15T12:57:38.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.660+0000 7f5ebeffd700 1 -- 172.21.15.2:0/4151595714 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5eb8014e40 con 0x7f5ec8102300 2024-03-15T12:57:38.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.660+0000 7f5ebeffd700 1 -- 172.21.15.2:0/4151595714 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb801d9d0 con 0x7f5ec8102300 2024-03-15T12:57:38.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.660+0000 7f5ecfd8b700 1 -- 172.21.15.2:0/4151595714 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec81032b0 con 0x7f5ec8102300 2024-03-15T12:57:38.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.660+0000 7f5ecfd8b700 1 -- 172.21.15.2:0/4151595714 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec8109ca0 con 0x7f5ec8102300 2024-03-15T12:57:38.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.662+0000 7f5ebeffd700 1 -- 172.21.15.2:0/4151595714 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f5eb801db30 con 0x7f5ec8102300 2024-03-15T12:57:38.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.663+0000 7f5ebeffd700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5eb40a1b50 0x7f5eb40a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:38.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.663+0000 7f5ecdb27700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5eb40a1b50 0x7f5eb40a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:38.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.663+0000 7f5ebeffd700 1 -- 172.21.15.2:0/4151595714 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f5eb80d5b30 con 0x7f5ec8102300 2024-03-15T12:57:38.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.663+0000 7f5ecdb27700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5eb40a1b50 0x7f5eb40a4010 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40098e0 tx=0x7f5ec4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:38.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.671+0000 7f5ebeffd700 1 -- 172.21.15.2:0/4151595714 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5eb809e840 con 0x7f5ec8102300 2024-03-15T12:57:38.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.880+0000 7f5ecfd8b700 1 -- 172.21.15.2:0/4151595714 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f5ec81014f0 con 0x7f5ec8102300 2024-03-15T12:57:38.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.880+0000 7f5ebeffd700 1 -- 172.21.15.2:0/4151595714 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f5eb80a1f00 con 0x7f5ec8102300 2024-03-15T12:57:38.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.883+0000 7f5ecfd8b700 1 -- 172.21.15.2:0/4151595714 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5eb40a1b50 msgr2=0x7f5eb40a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:38.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.883+0000 7f5ecfd8b700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5eb40a1b50 0x7f5eb40a4010 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40098e0 tx=0x7f5ec4005960 comp rx=0 tx=0).stop 2024-03-15T12:57:38.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.883+0000 7f5ecfd8b700 1 -- 172.21.15.2:0/4151595714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec8102300 msgr2=0x7f5ec81a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:38.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.883+0000 7f5ecfd8b700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec8102300 0x7f5ec81a0ab0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800b570 tx=0x7f5eb800b880 comp rx=0 tx=0).stop 2024-03-15T12:57:38.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.883+0000 7f5ecfd8b700 1 -- 172.21.15.2:0/4151595714 shutdown_connections 2024-03-15T12:57:38.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.883+0000 7f5ecfd8b700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5eb40a1b50 0x7f5eb40a4010 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.883+0000 7f5ecfd8b700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec810ea80 0x7f5ec819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.883+0000 7f5ecfd8b700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec81019e0 0x7f5ec81a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.883+0000 7f5ecfd8b700 1 --2- 172.21.15.2:0/4151595714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec8102300 0x7f5ec81a0ab0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.883+0000 7f5ecfd8b700 1 -- 172.21.15.2:0/4151595714 >> 172.21.15.2:0/4151595714 conn(0x7f5ec8077e50 msgr2=0x7f5ec810ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:38.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.883+0000 7f5ecfd8b700 1 -- 172.21.15.2:0/4151595714 shutdown_connections 2024-03-15T12:57:38.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:35.883+0000 7f5ecfd8b700 1 -- 172.21.15.2:0/4151595714 wait complete. 2024-03-15T12:57:38.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.277+0000 7f72ddf2d700 1 -- 172.21.15.2:0/1412025452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72d81011a0 msgr2=0x7f72d8101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:38.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.277+0000 7f72ddf2d700 1 --2- 172.21.15.2:0/1412025452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72d81011a0 0x7f72d8101580 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f72c8009990 tx=0x7f72c8009ca0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.277+0000 7f72ddf2d700 1 -- 172.21.15.2:0/1412025452 shutdown_connections 2024-03-15T12:57:38.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.277+0000 7f72ddf2d700 1 --2- 172.21.15.2:0/1412025452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72d81011a0 0x7f72d8101580 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.277+0000 7f72ddf2d700 1 --2- 172.21.15.2:0/1412025452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d81061f0 0x7f72d810ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.277+0000 7f72ddf2d700 1 --2- 172.21.15.2:0/1412025452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72d8101b50 0x7f72d8105b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.277+0000 7f72ddf2d700 1 -- 172.21.15.2:0/1412025452 >> 172.21.15.2:0/1412025452 conn(0x7f72d8076f30 msgr2=0x7f72d8077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:38.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.277+0000 7f72ddf2d700 1 -- 172.21.15.2:0/1412025452 shutdown_connections 2024-03-15T12:57:38.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.277+0000 7f72ddf2d700 1 -- 172.21.15.2:0/1412025452 wait complete. 2024-03-15T12:57:38.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.278+0000 7f72ddf2d700 1 Processor -- start 2024-03-15T12:57:38.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.278+0000 7f72ddf2d700 1 -- start start 2024-03-15T12:57:38.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.278+0000 7f72ddf2d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d81011a0 0x7f72d8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:38.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.278+0000 7f72ddf2d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72d8101b50 0x7f72d8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:38.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.278+0000 7f72ddf2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72d81061f0 0x7f72d819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:38.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.278+0000 7f72ddf2d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d819d400 con 0x7f72d81061f0 2024-03-15T12:57:38.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.278+0000 7f72ddf2d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d819d570 con 0x7f72d8101b50 2024-03-15T12:57:38.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.278+0000 7f72ddf2d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d819d730 con 0x7f72d81011a0 2024-03-15T12:57:38.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.278+0000 7f72d7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72d81061f0 0x7f72d819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:38.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.278+0000 7f72d7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72d81061f0 0x7f72d819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43138/0 (socket says 172.21.15.2:43138) 2024-03-15T12:57:38.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.278+0000 7f72d7fff700 1 -- 172.21.15.2:0/1911340292 learned_addr learned my addr 172.21.15.2:0/1911340292 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:38.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.278+0000 7f72d6ffd700 1 --2- 172.21.15.2:0/1911340292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72d8101b50 0x7f72d8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:38.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.278+0000 7f72d77fe700 1 --2- 172.21.15.2:0/1911340292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d81011a0 0x7f72d8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:38.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.279+0000 7f72d7fff700 1 -- 172.21.15.2:0/1911340292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d81011a0 msgr2=0x7f72d8198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:38.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.279+0000 7f72d7fff700 1 --2- 172.21.15.2:0/1911340292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d81011a0 0x7f72d8198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.279+0000 7f72d7fff700 1 -- 172.21.15.2:0/1911340292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72d8101b50 msgr2=0x7f72d8198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:38.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.279+0000 7f72d7fff700 1 --2- 172.21.15.2:0/1911340292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72d8101b50 0x7f72d8198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.279+0000 7f72d7fff700 1 -- 172.21.15.2:0/1911340292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c80095f0 con 0x7f72d81061f0 2024-03-15T12:57:38.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.279+0000 7f72d77fe700 1 --2- 172.21.15.2:0/1911340292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d81011a0 0x7f72d8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:38.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.279+0000 7f72d7fff700 1 --2- 172.21.15.2:0/1911340292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72d81061f0 0x7f72d819cdf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f72d000ef30 tx=0x7f72d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:38.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.279+0000 7f72d4ff9700 1 -- 172.21.15.2:0/1911340292 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0018070 con 0x7f72d81061f0 2024-03-15T12:57:38.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.279+0000 7f72d4ff9700 1 -- 172.21.15.2:0/1911340292 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f72d000f040 con 0x7f72d81061f0 2024-03-15T12:57:38.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.279+0000 7f72ddf2d700 1 -- 172.21.15.2:0/1911340292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d819d9b0 con 0x7f72d81061f0 2024-03-15T12:57:38.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.279+0000 7f72d4ff9700 1 -- 172.21.15.2:0/1911340292 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000cd40 con 0x7f72d81061f0 2024-03-15T12:57:38.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.279+0000 7f72ddf2d700 1 -- 172.21.15.2:0/1911340292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d810e7d0 con 0x7f72d81061f0 2024-03-15T12:57:38.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.280+0000 7f72ddf2d700 1 -- 172.21.15.2:0/1911340292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d81094b0 con 0x7f72d81061f0 2024-03-15T12:57:38.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.281+0000 7f72d4ff9700 1 -- 172.21.15.2:0/1911340292 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f72d0004ad0 con 0x7f72d81061f0 2024-03-15T12:57:38.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.282+0000 7f72d4ff9700 1 --2- 172.21.15.2:0/1911340292 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f72c00a1b50 0x7f72c00a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:38.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.282+0000 7f72d77fe700 1 --2- 172.21.15.2:0/1911340292 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f72c00a1b50 0x7f72c00a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:38.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.282+0000 7f72d4ff9700 1 -- 172.21.15.2:0/1911340292 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f72d00cdd80 con 0x7f72d81061f0 2024-03-15T12:57:38.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.283+0000 7f72d77fe700 1 --2- 172.21.15.2:0/1911340292 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f72c00a1b50 0x7f72c00a4010 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f72c80098e0 tx=0x7f72c8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:38.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.286+0000 7f72d4ff9700 1 -- 172.21.15.2:0/1911340292 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f72d0096b80 con 0x7f72d81061f0 2024-03-15T12:57:38.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:37 smithi184 ceph-mon[35993]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:57:38.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:37 smithi149 ceph-mon[35927]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:57:38.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.494+0000 7f72ddf2d700 1 -- 172.21.15.2:0/1911340292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f72d819dba0 con 0x7f72d81061f0 2024-03-15T12:57:38.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.494+0000 7f72d4ff9700 1 -- 172.21.15.2:0/1911340292 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f72d000cea0 con 0x7f72d81061f0 2024-03-15T12:57:38.481 INFO:teuthology.orchestra.run.smithi002.stdout:141733920832 2024-03-15T12:57:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.496+0000 7f72ddf2d700 1 -- 172.21.15.2:0/1911340292 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f72c00a1b50 msgr2=0x7f72c00a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.496+0000 7f72ddf2d700 1 --2- 172.21.15.2:0/1911340292 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f72c00a1b50 0x7f72c00a4010 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f72c80098e0 tx=0x7f72c8005960 comp rx=0 tx=0).stop 2024-03-15T12:57:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.496+0000 7f72ddf2d700 1 -- 172.21.15.2:0/1911340292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72d81061f0 msgr2=0x7f72d819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.496+0000 7f72ddf2d700 1 --2- 172.21.15.2:0/1911340292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72d81061f0 0x7f72d819cdf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f72d000ef30 tx=0x7f72d000c3a0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.496+0000 7f72ddf2d700 1 -- 172.21.15.2:0/1911340292 shutdown_connections 2024-03-15T12:57:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.496+0000 7f72ddf2d700 1 --2- 172.21.15.2:0/1911340292 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f72c00a1b50 0x7f72c00a4010 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.496+0000 7f72ddf2d700 1 --2- 172.21.15.2:0/1911340292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72d81061f0 0x7f72d819cdf0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.496+0000 7f72ddf2d700 1 --2- 172.21.15.2:0/1911340292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72d81011a0 0x7f72d8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.496+0000 7f72ddf2d700 1 --2- 172.21.15.2:0/1911340292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72d8101b50 0x7f72d8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.496+0000 7f72ddf2d700 1 -- 172.21.15.2:0/1911340292 >> 172.21.15.2:0/1911340292 conn(0x7f72d8076f30 msgr2=0x7f72d81006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.497+0000 7f72ddf2d700 1 -- 172.21.15.2:0/1911340292 shutdown_connections 2024-03-15T12:57:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:38.497+0000 7f72ddf2d700 1 -- 172.21.15.2:0/1911340292 wait complete. 2024-03-15T12:57:39.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:38 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1911340292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-15T12:57:39.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:38 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1911340292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-15T12:57:39.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:38 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1911340292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-15T12:57:39.438 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383705 got 343597383708 for osd.11 2024-03-15T12:57:39.438 DEBUG:teuthology.parallel:result is None 2024-03-15T12:57:40.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:39 smithi002 ceph-mon[32060]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:57:40.121 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:57:40.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:39 smithi184 ceph-mon[35993]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:57:40.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:39 smithi149 ceph-mon[35927]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:57:41.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.026+0000 7f00675cd700 1 -- 172.21.15.2:0/147805963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0060102030 msgr2=0x7f00601024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:41.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.026+0000 7f00675cd700 1 --2- 172.21.15.2:0/147805963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0060102030 0x7f00601024b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f00500099c0 tx=0x7f0050009cd0 comp rx=0 tx=0).stop 2024-03-15T12:57:41.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.026+0000 7f00675cd700 1 -- 172.21.15.2:0/147805963 shutdown_connections 2024-03-15T12:57:41.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.026+0000 7f00675cd700 1 --2- 172.21.15.2:0/147805963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00601029f0 0x7f006010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:41.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.026+0000 7f00675cd700 1 --2- 172.21.15.2:0/147805963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0060108010 0x7f00601083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:41.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.026+0000 7f00675cd700 1 --2- 172.21.15.2:0/147805963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0060102030 0x7f00601024b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:41.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.026+0000 7f00675cd700 1 -- 172.21.15.2:0/147805963 >> 172.21.15.2:0/147805963 conn(0x7f0060079110 msgr2=0x7f0060079520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:41.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.026+0000 7f00675cd700 1 -- 172.21.15.2:0/147805963 shutdown_connections 2024-03-15T12:57:41.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.026+0000 7f00675cd700 1 -- 172.21.15.2:0/147805963 wait complete. 2024-03-15T12:57:41.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.027+0000 7f00675cd700 1 Processor -- start 2024-03-15T12:57:41.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.027+0000 7f00675cd700 1 -- start start 2024-03-15T12:57:41.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.027+0000 7f00675cd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0060102030 0x7f00600742e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:41.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.027+0000 7f00675cd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00601029f0 0x7f0060074820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:41.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.027+0000 7f00675cd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0060108010 0x7f0060074d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:41.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.027+0000 7f00675cd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00601a6870 con 0x7f0060108010 2024-03-15T12:57:41.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.027+0000 7f00675cd700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00601a69e0 con 0x7f0060102030 2024-03-15T12:57:41.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.027+0000 7f00675cd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00601a6ba0 con 0x7f00601029f0 2024-03-15T12:57:41.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.027+0000 7f0065b6a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0060108010 0x7f0060074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:41.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.027+0000 7f0065b6a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0060108010 0x7f0060074d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43152/0 (socket says 172.21.15.2:43152) 2024-03-15T12:57:41.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.027+0000 7f0065b6a700 1 -- 172.21.15.2:0/1687170892 learned_addr learned my addr 172.21.15.2:0/1687170892 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:41.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.027+0000 7f0064b68700 1 --2- 172.21.15.2:0/1687170892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00601029f0 0x7f0060074820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:41.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.027+0000 7f0065369700 1 --2- 172.21.15.2:0/1687170892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0060102030 0x7f00600742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:41.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.028+0000 7f0065b6a700 1 -- 172.21.15.2:0/1687170892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00601029f0 msgr2=0x7f0060074820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:41.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.028+0000 7f0065b6a700 1 --2- 172.21.15.2:0/1687170892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00601029f0 0x7f0060074820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:41.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.028+0000 7f0065b6a700 1 -- 172.21.15.2:0/1687170892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0060102030 msgr2=0x7f00600742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:41.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.028+0000 7f0065b6a700 1 --2- 172.21.15.2:0/1687170892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0060102030 0x7f00600742e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:41.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.028+0000 7f0065b6a700 1 -- 172.21.15.2:0/1687170892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00500095f0 con 0x7f0060108010 2024-03-15T12:57:41.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.028+0000 7f0065b6a700 1 --2- 172.21.15.2:0/1687170892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0060108010 0x7f0060074d60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0058007ee0 tx=0x7f005800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:41.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.028+0000 7f00567fc700 1 -- 172.21.15.2:0/1687170892 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0058017070 con 0x7f0060108010 2024-03-15T12:57:41.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.028+0000 7f00567fc700 1 -- 172.21.15.2:0/1687170892 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f005800f040 con 0x7f0060108010 2024-03-15T12:57:41.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.028+0000 7f00675cd700 1 -- 172.21.15.2:0/1687170892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00601a6e80 con 0x7f0060108010 2024-03-15T12:57:41.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.028+0000 7f00567fc700 1 -- 172.21.15.2:0/1687170892 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0058005680 con 0x7f0060108010 2024-03-15T12:57:41.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.028+0000 7f00675cd700 1 -- 172.21.15.2:0/1687170892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00601a72a0 con 0x7f0060108010 2024-03-15T12:57:41.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.029+0000 7f00675cd700 1 -- 172.21.15.2:0/1687170892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006010a4f0 con 0x7f0060108010 2024-03-15T12:57:41.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.030+0000 7f00567fc700 1 -- 172.21.15.2:0/1687170892 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f00580057e0 con 0x7f0060108010 2024-03-15T12:57:41.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.031+0000 7f00567fc700 1 --2- 172.21.15.2:0/1687170892 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f004c0a1b50 0x7f004c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:41.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.031+0000 7f0065369700 1 --2- 172.21.15.2:0/1687170892 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f004c0a1b50 0x7f004c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:41.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.031+0000 7f00567fc700 1 -- 172.21.15.2:0/1687170892 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f00580cd510 con 0x7f0060108010 2024-03-15T12:57:41.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.031+0000 7f0065369700 1 --2- 172.21.15.2:0/1687170892 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f004c0a1b50 0x7f004c0a4010 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f005c009be0 tx=0x7f005c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:41.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.035+0000 7f00567fc700 1 -- 172.21.15.2:0/1687170892 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0058096280 con 0x7f0060108010 2024-03-15T12:57:41.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.247+0000 7f00675cd700 1 -- 172.21.15.2:0/1687170892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f00601120a0 con 0x7f0060108010 2024-03-15T12:57:41.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.248+0000 7f00567fc700 1 -- 172.21.15.2:0/1687170892 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0058099940 con 0x7f0060108010 2024-03-15T12:57:41.234 INFO:teuthology.orchestra.run.smithi002.stdout:197568495671 2024-03-15T12:57:41.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.250+0000 7f00675cd700 1 -- 172.21.15.2:0/1687170892 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f004c0a1b50 msgr2=0x7f004c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:41.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.250+0000 7f00675cd700 1 --2- 172.21.15.2:0/1687170892 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f004c0a1b50 0x7f004c0a4010 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f005c009be0 tx=0x7f005c009260 comp rx=0 tx=0).stop 2024-03-15T12:57:41.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.250+0000 7f00675cd700 1 -- 172.21.15.2:0/1687170892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0060108010 msgr2=0x7f0060074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:41.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.250+0000 7f00675cd700 1 --2- 172.21.15.2:0/1687170892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0060108010 0x7f0060074d60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0058007ee0 tx=0x7f005800d3b0 comp rx=0 tx=0).stop 2024-03-15T12:57:41.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.250+0000 7f00675cd700 1 -- 172.21.15.2:0/1687170892 shutdown_connections 2024-03-15T12:57:41.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.250+0000 7f00675cd700 1 --2- 172.21.15.2:0/1687170892 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f004c0a1b50 0x7f004c0a4010 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:41.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.250+0000 7f00675cd700 1 --2- 172.21.15.2:0/1687170892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0060108010 0x7f0060074d60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:41.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.250+0000 7f00675cd700 1 --2- 172.21.15.2:0/1687170892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0060102030 0x7f00600742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:41.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.250+0000 7f00675cd700 1 --2- 172.21.15.2:0/1687170892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00601029f0 0x7f0060074820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:41.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.250+0000 7f00675cd700 1 -- 172.21.15.2:0/1687170892 >> 172.21.15.2:0/1687170892 conn(0x7f0060079110 msgr2=0x7f0060100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:41.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.250+0000 7f00675cd700 1 -- 172.21.15.2:0/1687170892 shutdown_connections 2024-03-15T12:57:41.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.250+0000 7f00675cd700 1 -- 172.21.15.2:0/1687170892 wait complete. 2024-03-15T12:57:42.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:41 smithi002 ceph-mon[32060]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-15T12:57:42.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:41 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1687170892' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-15T12:57:42.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:41 smithi149 ceph-mon[35927]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-15T12:57:42.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:41 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1687170892' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-15T12:57:42.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.906+0000 7f702dc5b700 1 -- 172.21.15.2:0/4021502026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7028108fb0 msgr2=0x7f70281154c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:42.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.906+0000 7f702dc5b700 1 --2- 172.21.15.2:0/4021502026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7028108fb0 0x7f70281154c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7020009990 tx=0x7f7020009ca0 comp rx=0 tx=0).stop 2024-03-15T12:57:42.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.906+0000 7f702dc5b700 1 -- 172.21.15.2:0/4021502026 shutdown_connections 2024-03-15T12:57:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.906+0000 7f702dc5b700 1 --2- 172.21.15.2:0/4021502026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f702810e680 0x7f702810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.906+0000 7f702dc5b700 1 --2- 172.21.15.2:0/4021502026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7028108fb0 0x7f70281154c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.906+0000 7f702dc5b700 1 --2- 172.21.15.2:0/4021502026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70281085f0 0x7f7028108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.906+0000 7f702dc5b700 1 -- 172.21.15.2:0/4021502026 >> 172.21.15.2:0/4021502026 conn(0x7f70280757b0 msgr2=0x7f7028077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.906+0000 7f702dc5b700 1 -- 172.21.15.2:0/4021502026 shutdown_connections 2024-03-15T12:57:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.907+0000 7f702dc5b700 1 -- 172.21.15.2:0/4021502026 wait complete. 2024-03-15T12:57:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.907+0000 7f702dc5b700 1 Processor -- start 2024-03-15T12:57:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f702dc5b700 1 -- start start 2024-03-15T12:57:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f702dc5b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70281085f0 0x7f70281a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f702dc5b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7028108fb0 0x7f70281a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f702dc5b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f702810e680 0x7f702819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f702dc5b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702819ef00 con 0x7f7028108fb0 2024-03-15T12:57:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f702dc5b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702819f070 con 0x7f70281085f0 2024-03-15T12:57:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f702dc5b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702819f230 con 0x7f702810e680 2024-03-15T12:57:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f7026ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7028108fb0 0x7f70281a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f7026ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7028108fb0 0x7f70281a4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43162/0 (socket says 172.21.15.2:43162) 2024-03-15T12:57:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f70277fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70281085f0 0x7f70281a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f7026ffd700 1 -- 172.21.15.2:0/4287333053 learned_addr learned my addr 172.21.15.2:0/4287333053 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f7027fff700 1 --2- 172.21.15.2:0/4287333053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f702810e680 0x7f702819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f7026ffd700 1 -- 172.21.15.2:0/4287333053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f702810e680 msgr2=0x7f702819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f7026ffd700 1 --2- 172.21.15.2:0/4287333053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f702810e680 0x7f702819e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f7026ffd700 1 -- 172.21.15.2:0/4287333053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70281085f0 msgr2=0x7f70281a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f7026ffd700 1 --2- 172.21.15.2:0/4287333053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70281085f0 0x7f70281a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f7026ffd700 1 -- 172.21.15.2:0/4287333053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7018009c40 con 0x7f7028108fb0 2024-03-15T12:57:42.212 INFO:teuthology.orchestra.run.smithi002.stdout:253403070508 2024-03-15T12:57:42.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.908+0000 7f70277fe700 1 --2- 172.21.15.2:0/4287333053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70281085f0 0x7f70281a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:42.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.909+0000 7f7026ffd700 1 --2- 172.21.15.2:0/4287333053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7028108fb0 0x7f70281a4bc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f701000e9a0 tx=0x7f701000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:42.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.909+0000 7f7024ff9700 1 -- 172.21.15.2:0/4287333053 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010009790 con 0x7f7028108fb0 2024-03-15T12:57:42.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.909+0000 7f702dc5b700 1 -- 172.21.15.2:0/4287333053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70200095f0 con 0x7f7028108fb0 2024-03-15T12:57:42.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.909+0000 7f702dc5b700 1 -- 172.21.15.2:0/4287333053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702819f700 con 0x7f7028108fb0 2024-03-15T12:57:42.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.909+0000 7f7024ff9700 1 -- 172.21.15.2:0/4287333053 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7010010430 con 0x7f7028108fb0 2024-03-15T12:57:42.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.909+0000 7f7024ff9700 1 -- 172.21.15.2:0/4287333053 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010018660 con 0x7f7028108fb0 2024-03-15T12:57:42.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.910+0000 7f702dc5b700 1 -- 172.21.15.2:0/4287333053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7008005330 con 0x7f7028108fb0 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.911+0000 7f7024ff9700 1 -- 172.21.15.2:0/4287333053 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f701000ca10 con 0x7f7028108fb0 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.912+0000 7f7024ff9700 1 --2- 172.21.15.2:0/4287333053 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f70140a1bd0 0x7f70140a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.912+0000 7f7024ff9700 1 -- 172.21.15.2:0/4287333053 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f7010014070 con 0x7f7028108fb0 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.912+0000 7f70277fe700 1 --2- 172.21.15.2:0/4287333053 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f70140a1bd0 0x7f70140a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.912+0000 7f70277fe700 1 --2- 172.21.15.2:0/4287333053 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f70140a1bd0 0x7f70140a4090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7028109730 tx=0x7f701800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:41.916+0000 7f7024ff9700 1 -- 172.21.15.2:0/4287333053 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7010096b10 con 0x7f7028108fb0 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:42.127+0000 7f702dc5b700 1 -- 172.21.15.2:0/4287333053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f70080058b0 con 0x7f7028108fb0 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:42.127+0000 7f7024ff9700 1 -- 172.21.15.2:0/4287333053 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f701009a1d0 con 0x7f7028108fb0 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:42.129+0000 7f702dc5b700 1 -- 172.21.15.2:0/4287333053 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f70140a1bd0 msgr2=0x7f70140a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:42.129+0000 7f702dc5b700 1 --2- 172.21.15.2:0/4287333053 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f70140a1bd0 0x7f70140a4090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7028109730 tx=0x7f701800b040 comp rx=0 tx=0).stop 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:42.129+0000 7f702dc5b700 1 -- 172.21.15.2:0/4287333053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7028108fb0 msgr2=0x7f70281a4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:42.129+0000 7f702dc5b700 1 --2- 172.21.15.2:0/4287333053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7028108fb0 0x7f70281a4bc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f701000e9a0 tx=0x7f701000ecb0 comp rx=0 tx=0).stop 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:42.130+0000 7f702dc5b700 1 -- 172.21.15.2:0/4287333053 shutdown_connections 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:42.130+0000 7f702dc5b700 1 --2- 172.21.15.2:0/4287333053 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f70140a1bd0 0x7f70140a4090 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:42.130+0000 7f702dc5b700 1 --2- 172.21.15.2:0/4287333053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f702810e680 0x7f702819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:42.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:42.130+0000 7f702dc5b700 1 --2- 172.21.15.2:0/4287333053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70281085f0 0x7f70281a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:42.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:42.130+0000 7f702dc5b700 1 --2- 172.21.15.2:0/4287333053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7028108fb0 0x7f70281a4bc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:42.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:42.130+0000 7f702dc5b700 1 -- 172.21.15.2:0/4287333053 >> 172.21.15.2:0/4287333053 conn(0x7f70280757b0 msgr2=0x7f7028076c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:42.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:42.130+0000 7f702dc5b700 1 -- 172.21.15.2:0/4287333053 shutdown_connections 2024-03-15T12:57:42.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:42.130+0000 7f702dc5b700 1 -- 172.21.15.2:0/4287333053 wait complete. 2024-03-15T12:57:42.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:41 smithi184 ceph-mon[35993]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-15T12:57:42.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:41 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1687170892' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-15T12:57:43.166 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:42 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/4287333053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-15T12:57:43.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:42 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/4287333053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-15T12:57:43.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:42 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/4287333053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-15T12:57:43.666 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495666 got 197568495671 for osd.6 2024-03-15T12:57:43.666 DEBUG:teuthology.parallel:result is None 2024-03-15T12:57:44.144 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:43 smithi002 ceph-mon[32060]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:57:44.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:43 smithi184 ceph-mon[35993]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:57:44.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.338+0000 7f9286f2a700 1 -- 172.21.15.2:0/4270658510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9280101170 msgr2=0x7f9280101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:44.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.338+0000 7f9286f2a700 1 --2- 172.21.15.2:0/4270658510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9280101170 0x7f9280101550 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9274009990 tx=0x7f9274009ca0 comp rx=0 tx=0).stop 2024-03-15T12:57:44.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.338+0000 7f9286f2a700 1 -- 172.21.15.2:0/4270658510 shutdown_connections 2024-03-15T12:57:44.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.338+0000 7f9286f2a700 1 --2- 172.21.15.2:0/4270658510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9280101170 0x7f9280101550 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:44.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.338+0000 7f9286f2a700 1 --2- 172.21.15.2:0/4270658510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92801061c0 0x7f928010dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:44.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.338+0000 7f9286f2a700 1 --2- 172.21.15.2:0/4270658510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9280101b20 0x7f9280105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:44.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.338+0000 7f9286f2a700 1 -- 172.21.15.2:0/4270658510 >> 172.21.15.2:0/4270658510 conn(0x7f9280076f30 msgr2=0x7f9280077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:44.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.338+0000 7f9286f2a700 1 -- 172.21.15.2:0/4270658510 shutdown_connections 2024-03-15T12:57:44.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.338+0000 7f9286f2a700 1 -- 172.21.15.2:0/4270658510 wait complete. 2024-03-15T12:57:44.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.339+0000 7f9286f2a700 1 Processor -- start 2024-03-15T12:57:44.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.339+0000 7f9286f2a700 1 -- start start 2024-03-15T12:57:44.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.339+0000 7f9286f2a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9280101170 0x7f92801984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:44.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.339+0000 7f9286f2a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9280101b20 0x7f9280198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:44.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.339+0000 7f9286f2a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92801061c0 0x7f928019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:44.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.339+0000 7f9286f2a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928019d3d0 con 0x7f9280101170 2024-03-15T12:57:44.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.339+0000 7f9286f2a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928019d540 con 0x7f92801061c0 2024-03-15T12:57:44.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.339+0000 7f9286f2a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928019d700 con 0x7f9280101b20 2024-03-15T12:57:44.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.340+0000 7f9284cc6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9280101170 0x7f92801984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:44.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.340+0000 7f9284cc6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9280101170 0x7f92801984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43172/0 (socket says 172.21.15.2:43172) 2024-03-15T12:57:44.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.340+0000 7f9284cc6700 1 -- 172.21.15.2:0/309824506 learned_addr learned my addr 172.21.15.2:0/309824506 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:44.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.340+0000 7f927ffff700 1 --2- 172.21.15.2:0/309824506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9280101b20 0x7f9280198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:44.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.340+0000 7f92854c7700 1 --2- 172.21.15.2:0/309824506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92801061c0 0x7f928019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:44.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.340+0000 7f9284cc6700 1 -- 172.21.15.2:0/309824506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9280101b20 msgr2=0x7f9280198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:44.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.340+0000 7f9284cc6700 1 --2- 172.21.15.2:0/309824506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9280101b20 0x7f9280198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:44.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.340+0000 7f9284cc6700 1 -- 172.21.15.2:0/309824506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92801061c0 msgr2=0x7f928019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:44.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.340+0000 7f9284cc6700 1 --2- 172.21.15.2:0/309824506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92801061c0 0x7f928019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:44.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.340+0000 7f9284cc6700 1 -- 172.21.15.2:0/309824506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92740095f0 con 0x7f9280101170 2024-03-15T12:57:44.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.340+0000 7f9284cc6700 1 --2- 172.21.15.2:0/309824506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9280101170 0x7f92801984f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f927400b2e0 tx=0x7f9274004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:44.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.340+0000 7f927dffb700 1 -- 172.21.15.2:0/309824506 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927401d070 con 0x7f9280101170 2024-03-15T12:57:44.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.340+0000 7f9286f2a700 1 -- 172.21.15.2:0/309824506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928019d980 con 0x7f9280101170 2024-03-15T12:57:44.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.340+0000 7f927dffb700 1 -- 172.21.15.2:0/309824506 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f927400b610 con 0x7f9280101170 2024-03-15T12:57:44.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.340+0000 7f927dffb700 1 -- 172.21.15.2:0/309824506 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274004a90 con 0x7f9280101170 2024-03-15T12:57:44.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.341+0000 7f9286f2a700 1 -- 172.21.15.2:0/309824506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92801a5070 con 0x7f9280101170 2024-03-15T12:57:44.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.341+0000 7f9286f2a700 1 -- 172.21.15.2:0/309824506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9280109480 con 0x7f9280101170 2024-03-15T12:57:44.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.343+0000 7f927dffb700 1 -- 172.21.15.2:0/309824506 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f9274005330 con 0x7f9280101170 2024-03-15T12:57:44.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.343+0000 7f927dffb700 1 --2- 172.21.15.2:0/309824506 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f92700a1b50 0x7f92700a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:44.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.344+0000 7f927ffff700 1 --2- 172.21.15.2:0/309824506 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f92700a1b50 0x7f92700a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:44.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.344+0000 7f927dffb700 1 -- 172.21.15.2:0/309824506 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f92740cedb0 con 0x7f9280101170 2024-03-15T12:57:44.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.344+0000 7f927ffff700 1 --2- 172.21.15.2:0/309824506 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f92700a1b50 0x7f92700a4010 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9280105c40 tx=0x7f926c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:44.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.351+0000 7f927dffb700 1 -- 172.21.15.2:0/309824506 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9274097b50 con 0x7f9280101170 2024-03-15T12:57:44.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:43 smithi149 ceph-mon[35927]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:57:44.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.556+0000 7f9286f2a700 1 -- 172.21.15.2:0/309824506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f9280199850 con 0x7f9280101170 2024-03-15T12:57:44.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.556+0000 7f927dffb700 1 -- 172.21.15.2:0/309824506 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f927409b210 con 0x7f9280101170 2024-03-15T12:57:44.543 INFO:teuthology.orchestra.run.smithi002.stdout:309237645346 2024-03-15T12:57:44.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.558+0000 7f9286f2a700 1 -- 172.21.15.2:0/309824506 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f92700a1b50 msgr2=0x7f92700a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.558+0000 7f9286f2a700 1 --2- 172.21.15.2:0/309824506 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f92700a1b50 0x7f92700a4010 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9280105c40 tx=0x7f926c009260 comp rx=0 tx=0).stop 2024-03-15T12:57:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.558+0000 7f9286f2a700 1 -- 172.21.15.2:0/309824506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9280101170 msgr2=0x7f92801984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.558+0000 7f9286f2a700 1 --2- 172.21.15.2:0/309824506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9280101170 0x7f92801984f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f927400b2e0 tx=0x7f9274004730 comp rx=0 tx=0).stop 2024-03-15T12:57:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.558+0000 7f9286f2a700 1 -- 172.21.15.2:0/309824506 shutdown_connections 2024-03-15T12:57:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.558+0000 7f9286f2a700 1 --2- 172.21.15.2:0/309824506 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f92700a1b50 0x7f92700a4010 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.558+0000 7f9286f2a700 1 --2- 172.21.15.2:0/309824506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9280101170 0x7f92801984f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.558+0000 7f9286f2a700 1 --2- 172.21.15.2:0/309824506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92801061c0 0x7f928019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.558+0000 7f9286f2a700 1 --2- 172.21.15.2:0/309824506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9280101b20 0x7f9280198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.558+0000 7f9286f2a700 1 -- 172.21.15.2:0/309824506 >> 172.21.15.2:0/309824506 conn(0x7f9280076f30 msgr2=0x7f92801006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.558+0000 7f9286f2a700 1 -- 172.21.15.2:0/309824506 shutdown_connections 2024-03-15T12:57:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:44.559+0000 7f9286f2a700 1 -- 172.21.15.2:0/309824506 wait complete. 2024-03-15T12:57:44.716 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070504 got 253403070508 for osd.8 2024-03-15T12:57:44.716 DEBUG:teuthology.parallel:result is None 2024-03-15T12:57:44.952 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:44 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/309824506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-15T12:57:45.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:44 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/309824506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-15T12:57:45.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:44 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/309824506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-15T12:57:45.365 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757302 got 163208757307 for osd.5 2024-03-15T12:57:45.365 DEBUG:teuthology.parallel:result is None 2024-03-15T12:57:45.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.589+0000 7f9c741e6700 1 -- 172.21.15.2:0/3019600936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c6c1005d0 msgr2=0x7f9c6c100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:45.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.589+0000 7f9c741e6700 1 --2- 172.21.15.2:0/3019600936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c6c1005d0 0x7f9c6c100a50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009990 tx=0x7f9c5c009ca0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.589+0000 7f9c741e6700 1 -- 172.21.15.2:0/3019600936 shutdown_connections 2024-03-15T12:57:45.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.589+0000 7f9c741e6700 1 --2- 172.21.15.2:0/3019600936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c6c100f90 0x7f9c6c10eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.589+0000 7f9c741e6700 1 --2- 172.21.15.2:0/3019600936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c6c107050 0x7f9c6c107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.589+0000 7f9c741e6700 1 --2- 172.21.15.2:0/3019600936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c6c1005d0 0x7f9c6c100a50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.589+0000 7f9c741e6700 1 -- 172.21.15.2:0/3019600936 >> 172.21.15.2:0/3019600936 conn(0x7f9c6c0774e0 msgr2=0x7f9c6c0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:45.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.590+0000 7f9c741e6700 1 -- 172.21.15.2:0/3019600936 shutdown_connections 2024-03-15T12:57:45.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.590+0000 7f9c741e6700 1 -- 172.21.15.2:0/3019600936 wait complete. 2024-03-15T12:57:45.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.590+0000 7f9c741e6700 1 Processor -- start 2024-03-15T12:57:45.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.590+0000 7f9c741e6700 1 -- start start 2024-03-15T12:57:45.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c741e6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c6c1005d0 0x7f9c6c1033a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:45.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c741e6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c6c100f90 0x7f9c6c1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:45.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c741e6700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c6c107050 0x7f9c6c101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:45.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c741e6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c1025e0 con 0x7f9c6c100f90 2024-03-15T12:57:45.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c741e6700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c1ab000 con 0x7f9c6c107050 2024-03-15T12:57:45.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c741e6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c1ab170 con 0x7f9c6c1005d0 2024-03-15T12:57:45.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c71f82700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c6c1005d0 0x7f9c6c1033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:45.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c72783700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c6c107050 0x7f9c6c101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:45.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c71781700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c6c100f90 0x7f9c6c1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:45.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c71f82700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c6c1005d0 0x7f9c6c1033a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:60994/0 (socket says 172.21.15.2:60994) 2024-03-15T12:57:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c72783700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c6c107050 0x7f9c6c101e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:40684/0 (socket says 172.21.15.2:40684) 2024-03-15T12:57:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c71f82700 1 -- 172.21.15.2:0/3415896342 learned_addr learned my addr 172.21.15.2:0/3415896342 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c71f82700 1 -- 172.21.15.2:0/3415896342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c6c107050 msgr2=0x7f9c6c101e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c71f82700 1 --2- 172.21.15.2:0/3415896342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c6c107050 0x7f9c6c101e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c71f82700 1 -- 172.21.15.2:0/3415896342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c6c100f90 msgr2=0x7f9c6c1017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c71f82700 1 --2- 172.21.15.2:0/3415896342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c6c100f90 0x7f9c6c1017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c71f82700 1 -- 172.21.15.2:0/3415896342 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c0095f0 con 0x7f9c6c1005d0 2024-03-15T12:57:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.591+0000 7f9c71781700 1 --2- 172.21.15.2:0/3415896342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c6c100f90 0x7f9c6c1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.592+0000 7f9c71f82700 1 --2- 172.21.15.2:0/3415896342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c6c1005d0 0x7f9c6c1033a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800e9a0 tx=0x7f9c6800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.592+0000 7f9c62ffd700 1 -- 172.21.15.2:0/3415896342 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c68009790 con 0x7f9c6c1005d0 2024-03-15T12:57:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.592+0000 7f9c62ffd700 1 -- 172.21.15.2:0/3415896342 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9c68010430 con 0x7f9c6c1005d0 2024-03-15T12:57:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.592+0000 7f9c741e6700 1 -- 172.21.15.2:0/3415896342 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6c1ab450 con 0x7f9c6c1005d0 2024-03-15T12:57:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.592+0000 7f9c62ffd700 1 -- 172.21.15.2:0/3415896342 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c68018660 con 0x7f9c6c1005d0 2024-03-15T12:57:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.592+0000 7f9c741e6700 1 -- 172.21.15.2:0/3415896342 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6c1ab970 con 0x7f9c6c1005d0 2024-03-15T12:57:45.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.593+0000 7f9c741e6700 1 -- 172.21.15.2:0/3415896342 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c6c10a5d0 con 0x7f9c6c1005d0 2024-03-15T12:57:45.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.594+0000 7f9c62ffd700 1 -- 172.21.15.2:0/3415896342 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f9c680105a0 con 0x7f9c6c1005d0 2024-03-15T12:57:45.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.594+0000 7f9c62ffd700 1 --2- 172.21.15.2:0/3415896342 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9c580a1bd0 0x7f9c580a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:45.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.595+0000 7f9c62ffd700 1 -- 172.21.15.2:0/3415896342 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f9c68014070 con 0x7f9c6c1005d0 2024-03-15T12:57:45.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.595+0000 7f9c71781700 1 --2- 172.21.15.2:0/3415896342 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9c580a1bd0 0x7f9c580a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:45.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.595+0000 7f9c71781700 1 --2- 172.21.15.2:0/3415896342 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9c580a1bd0 0x7f9c580a4090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c19c790 tx=0x7f9c5c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:45.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.598+0000 7f9c62ffd700 1 -- 172.21.15.2:0/3415896342 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9c680966c0 con 0x7f9c6c1005d0 2024-03-15T12:57:45.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.789+0000 7f4a3333a700 1 -- 172.21.15.2:0/1457675668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a2c1061c0 msgr2=0x7f4a2c10dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:45.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.789+0000 7f4a3333a700 1 --2- 172.21.15.2:0/1457675668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a2c1061c0 0x7f4a2c10dd90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400b0a0 tx=0x7f4a2400b3b0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.789+0000 7f4a3333a700 1 -- 172.21.15.2:0/1457675668 shutdown_connections 2024-03-15T12:57:45.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.789+0000 7f4a3333a700 1 --2- 172.21.15.2:0/1457675668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a2c101170 0x7f4a2c101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.789+0000 7f4a3333a700 1 --2- 172.21.15.2:0/1457675668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a2c1061c0 0x7f4a2c10dd90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.789+0000 7f4a3333a700 1 --2- 172.21.15.2:0/1457675668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a2c101b20 0x7f4a2c105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.789+0000 7f4a3333a700 1 -- 172.21.15.2:0/1457675668 >> 172.21.15.2:0/1457675668 conn(0x7f4a2c076f30 msgr2=0x7f4a2c077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:45.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.789+0000 7f4a3333a700 1 -- 172.21.15.2:0/1457675668 shutdown_connections 2024-03-15T12:57:45.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.789+0000 7f4a3333a700 1 -- 172.21.15.2:0/1457675668 wait complete. 2024-03-15T12:57:45.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.790+0000 7f4a3333a700 1 Processor -- start 2024-03-15T12:57:45.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.790+0000 7f4a3333a700 1 -- start start 2024-03-15T12:57:45.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.790+0000 7f4a3333a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a2c101170 0x7f4a2c197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:45.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.790+0000 7f4a3333a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a2c101b20 0x7f4a2c197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:45.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a3333a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a2c1061c0 0x7f4a2c19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:45.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a3333a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c19c3d0 con 0x7f4a2c101b20 2024-03-15T12:57:45.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a3333a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c19c540 con 0x7f4a2c101170 2024-03-15T12:57:45.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a3333a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c19c700 con 0x7f4a2c1061c0 2024-03-15T12:57:45.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a310d6700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a2c101170 0x7f4a2c197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:45.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a308d5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a2c101b20 0x7f4a2c197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:45.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a310d6700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a2c101170 0x7f4a2c197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:40710/0 (socket says 172.21.15.2:40710) 2024-03-15T12:57:45.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a310d6700 1 -- 172.21.15.2:0/2189134016 learned_addr learned my addr 172.21.15.2:0/2189134016 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:45.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a318d7700 1 --2- 172.21.15.2:0/2189134016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a2c1061c0 0x7f4a2c19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:45.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a310d6700 1 -- 172.21.15.2:0/2189134016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a2c1061c0 msgr2=0x7f4a2c19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:45.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a310d6700 1 --2- 172.21.15.2:0/2189134016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a2c1061c0 0x7f4a2c19bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a310d6700 1 -- 172.21.15.2:0/2189134016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a2c101b20 msgr2=0x7f4a2c197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:45.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a310d6700 1 --2- 172.21.15.2:0/2189134016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a2c101b20 0x7f4a2c197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a310d6700 1 -- 172.21.15.2:0/2189134016 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a20009520 con 0x7f4a2c101170 2024-03-15T12:57:45.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a310d6700 1 --2- 172.21.15.2:0/2189134016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a2c101170 0x7f4a2c197540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800e9a0 tx=0x7f4a2800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:45.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a1e7fc700 1 -- 172.21.15.2:0/2189134016 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28009790 con 0x7f4a2c101170 2024-03-15T12:57:45.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a1e7fc700 1 -- 172.21.15.2:0/2189134016 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4a28010430 con 0x7f4a2c101170 2024-03-15T12:57:45.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a3333a700 1 -- 172.21.15.2:0/2189134016 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a24009d00 con 0x7f4a2c101170 2024-03-15T12:57:45.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a1e7fc700 1 -- 172.21.15.2:0/2189134016 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28018660 con 0x7f4a2c101170 2024-03-15T12:57:45.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.791+0000 7f4a3333a700 1 -- 172.21.15.2:0/2189134016 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a2c1a4070 con 0x7f4a2c101170 2024-03-15T12:57:45.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.792+0000 7f4a3333a700 1 -- 172.21.15.2:0/2189134016 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a2c1094e0 con 0x7f4a2c101170 2024-03-15T12:57:45.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.793+0000 7f4a1e7fc700 1 -- 172.21.15.2:0/2189134016 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f4a280105a0 con 0x7f4a2c101170 2024-03-15T12:57:45.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.794+0000 7f4a1e7fc700 1 --2- 172.21.15.2:0/2189134016 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4a180a1b50 0x7f4a180a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:45.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.794+0000 7f4a1e7fc700 1 -- 172.21.15.2:0/2189134016 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f4a28014070 con 0x7f4a2c101170 2024-03-15T12:57:45.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.794+0000 7f4a308d5700 1 --2- 172.21.15.2:0/2189134016 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4a180a1b50 0x7f4a180a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:45.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.795+0000 7f4a308d5700 1 --2- 172.21.15.2:0/2189134016 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4a180a1b50 0x7f4a180a4010 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c107690 tx=0x7f4a20009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:45.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.798+0000 7f4a1e7fc700 1 -- 172.21.15.2:0/2189134016 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4a28096790 con 0x7f4a2c101170 2024-03-15T12:57:45.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.808+0000 7f9c741e6700 1 -- 172.21.15.2:0/3415896342 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9c6c102c90 con 0x7f9c6c1005d0 2024-03-15T12:57:45.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.809+0000 7f9c62ffd700 1 -- 172.21.15.2:0/3415896342 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c68099d80 con 0x7f9c6c1005d0 2024-03-15T12:57:45.795 INFO:teuthology.orchestra.run.smithi002.stdout:107374182470 2024-03-15T12:57:45.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.811+0000 7f9c741e6700 1 -- 172.21.15.2:0/3415896342 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9c580a1bd0 msgr2=0x7f9c580a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:45.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.811+0000 7f9c741e6700 1 --2- 172.21.15.2:0/3415896342 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9c580a1bd0 0x7f9c580a4090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c19c790 tx=0x7f9c5c005960 comp rx=0 tx=0).stop 2024-03-15T12:57:45.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.811+0000 7f9c741e6700 1 -- 172.21.15.2:0/3415896342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c6c1005d0 msgr2=0x7f9c6c1033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:45.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.811+0000 7f9c741e6700 1 --2- 172.21.15.2:0/3415896342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c6c1005d0 0x7f9c6c1033a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800e9a0 tx=0x7f9c6800ecb0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.811+0000 7f9c741e6700 1 -- 172.21.15.2:0/3415896342 shutdown_connections 2024-03-15T12:57:45.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.811+0000 7f9c741e6700 1 --2- 172.21.15.2:0/3415896342 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9c580a1bd0 0x7f9c580a4090 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.811+0000 7f9c741e6700 1 --2- 172.21.15.2:0/3415896342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c6c107050 0x7f9c6c101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.811+0000 7f9c741e6700 1 --2- 172.21.15.2:0/3415896342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c6c1005d0 0x7f9c6c1033a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.811+0000 7f9c741e6700 1 --2- 172.21.15.2:0/3415896342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c6c100f90 0x7f9c6c1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.811+0000 7f9c741e6700 1 -- 172.21.15.2:0/3415896342 >> 172.21.15.2:0/3415896342 conn(0x7f9c6c0774e0 msgr2=0x7f9c6c105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:45.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.811+0000 7f9c741e6700 1 -- 172.21.15.2:0/3415896342 shutdown_connections 2024-03-15T12:57:45.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:45.811+0000 7f9c741e6700 1 -- 172.21.15.2:0/3415896342 wait complete. 2024-03-15T12:57:45.987 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:45 smithi002 ceph-mon[32060]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-15T12:57:45.987 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:45 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3415896342' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-15T12:57:45.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.004+0000 7f4a3333a700 1 -- 172.21.15.2:0/2189134016 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4a2c198850 con 0x7f4a2c101170 2024-03-15T12:57:45.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.004+0000 7f4a1e7fc700 1 -- 172.21.15.2:0/2189134016 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a28099e50 con 0x7f4a2c101170 2024-03-15T12:57:45.990 INFO:teuthology.orchestra.run.smithi002.stdout:219043332146 2024-03-15T12:57:45.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.006+0000 7f4a3333a700 1 -- 172.21.15.2:0/2189134016 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4a180a1b50 msgr2=0x7f4a180a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:45.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.006+0000 7f4a3333a700 1 --2- 172.21.15.2:0/2189134016 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4a180a1b50 0x7f4a180a4010 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c107690 tx=0x7f4a20009260 comp rx=0 tx=0).stop 2024-03-15T12:57:45.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.006+0000 7f4a3333a700 1 -- 172.21.15.2:0/2189134016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a2c101170 msgr2=0x7f4a2c197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:45.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.006+0000 7f4a3333a700 1 --2- 172.21.15.2:0/2189134016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a2c101170 0x7f4a2c197540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800e9a0 tx=0x7f4a2800ecb0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.006+0000 7f4a3333a700 1 -- 172.21.15.2:0/2189134016 shutdown_connections 2024-03-15T12:57:45.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.006+0000 7f4a3333a700 1 --2- 172.21.15.2:0/2189134016 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4a180a1b50 0x7f4a180a4010 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.006+0000 7f4a3333a700 1 --2- 172.21.15.2:0/2189134016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a2c1061c0 0x7f4a2c19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.006+0000 7f4a3333a700 1 --2- 172.21.15.2:0/2189134016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a2c101170 0x7f4a2c197540 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.006+0000 7f4a3333a700 1 --2- 172.21.15.2:0/2189134016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a2c101b20 0x7f4a2c197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:45.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.006+0000 7f4a3333a700 1 -- 172.21.15.2:0/2189134016 >> 172.21.15.2:0/2189134016 conn(0x7f4a2c076f30 msgr2=0x7f4a2c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:45.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.006+0000 7f4a3333a700 1 -- 172.21.15.2:0/2189134016 shutdown_connections 2024-03-15T12:57:45.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.006+0000 7f4a3333a700 1 -- 172.21.15.2:0/2189134016 wait complete. 2024-03-15T12:57:46.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:45 smithi184 ceph-mon[35993]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-15T12:57:46.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:45 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3415896342' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-15T12:57:46.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:45 smithi149 ceph-mon[35927]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-15T12:57:46.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:45 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3415896342' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-15T12:57:46.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.410+0000 7f4a84e75700 1 -- 172.21.15.2:0/564772307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a80101a30 msgr2=0x7f4a80101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:46.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.410+0000 7f4a84e75700 1 --2- 172.21.15.2:0/564772307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a80101a30 0x7f4a80101eb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009990 tx=0x7f4a70009ca0 comp rx=0 tx=0).stop 2024-03-15T12:57:46.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.410+0000 7f4a84e75700 1 -- 172.21.15.2:0/564772307 shutdown_connections 2024-03-15T12:57:46.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.410+0000 7f4a84e75700 1 --2- 172.21.15.2:0/564772307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a801023f0 0x7f4a8010ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:46.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.410+0000 7f4a84e75700 1 --2- 172.21.15.2:0/564772307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a80105f20 0x7f4a80106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:46.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.410+0000 7f4a84e75700 1 --2- 172.21.15.2:0/564772307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a80101a30 0x7f4a80101eb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:46.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.410+0000 7f4a84e75700 1 -- 172.21.15.2:0/564772307 >> 172.21.15.2:0/564772307 conn(0x7f4a80076240 msgr2=0x7f4a80076650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:46.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.410+0000 7f4a84e75700 1 -- 172.21.15.2:0/564772307 shutdown_connections 2024-03-15T12:57:46.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.410+0000 7f4a84e75700 1 -- 172.21.15.2:0/564772307 wait complete. 2024-03-15T12:57:46.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.411+0000 7f4a84e75700 1 Processor -- start 2024-03-15T12:57:46.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.411+0000 7f4a84e75700 1 -- start start 2024-03-15T12:57:46.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.411+0000 7f4a84e75700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a80101a30 0x7f4a80198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:46.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.411+0000 7f4a84e75700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a801023f0 0x7f4a80198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:46.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.411+0000 7f4a84e75700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a80105f20 0x7f4a8019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:46.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.411+0000 7f4a84e75700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8019d410 con 0x7f4a80105f20 2024-03-15T12:57:46.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.411+0000 7f4a84e75700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8019d580 con 0x7f4a801023f0 2024-03-15T12:57:46.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.411+0000 7f4a84e75700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8019d740 con 0x7f4a80101a30 2024-03-15T12:57:46.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.411+0000 7f4a7ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a80105f20 0x7f4a8019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:46.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.411+0000 7f4a7ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a80105f20 0x7f4a8019ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43224/0 (socket says 172.21.15.2:43224) 2024-03-15T12:57:46.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.411+0000 7f4a7dd9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a801023f0 0x7f4a80198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:46.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.411+0000 7f4a7ed9d700 1 -- 172.21.15.2:0/996293443 learned_addr learned my addr 172.21.15.2:0/996293443 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:46.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.411+0000 7f4a7e59c700 1 --2- 172.21.15.2:0/996293443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a80101a30 0x7f4a80198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:46.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.412+0000 7f4a7ed9d700 1 -- 172.21.15.2:0/996293443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a80101a30 msgr2=0x7f4a80198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:46.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.412+0000 7f4a7ed9d700 1 --2- 172.21.15.2:0/996293443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a80101a30 0x7f4a80198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:46.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.412+0000 7f4a7ed9d700 1 -- 172.21.15.2:0/996293443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a801023f0 msgr2=0x7f4a80198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:46.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.412+0000 7f4a7ed9d700 1 --2- 172.21.15.2:0/996293443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a801023f0 0x7f4a80198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:46.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.412+0000 7f4a7ed9d700 1 -- 172.21.15.2:0/996293443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a700095f0 con 0x7f4a80105f20 2024-03-15T12:57:46.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.412+0000 7f4a7e59c700 1 --2- 172.21.15.2:0/996293443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a80101a30 0x7f4a80198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:46.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.412+0000 7f4a7dd9b700 1 --2- 172.21.15.2:0/996293443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a801023f0 0x7f4a80198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:46.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.412+0000 7f4a7ed9d700 1 --2- 172.21.15.2:0/996293443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a80105f20 0x7f4a8019ce00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800ef30 tx=0x7f4a7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:46.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.412+0000 7f4a6f7fe700 1 -- 172.21.15.2:0/996293443 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78018070 con 0x7f4a80105f20 2024-03-15T12:57:46.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.412+0000 7f4a6f7fe700 1 -- 172.21.15.2:0/996293443 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4a7800f040 con 0x7f4a80105f20 2024-03-15T12:57:46.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.412+0000 7f4a84e75700 1 -- 172.21.15.2:0/996293443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8019d9c0 con 0x7f4a80105f20 2024-03-15T12:57:46.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.412+0000 7f4a6f7fe700 1 -- 172.21.15.2:0/996293443 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7800cd40 con 0x7f4a80105f20 2024-03-15T12:57:46.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.412+0000 7f4a84e75700 1 -- 172.21.15.2:0/996293443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a801a5280 con 0x7f4a80105f20 2024-03-15T12:57:46.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.413+0000 7f4a84e75700 1 -- 172.21.15.2:0/996293443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8010a520 con 0x7f4a80105f20 2024-03-15T12:57:46.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.414+0000 7f4a6f7fe700 1 -- 172.21.15.2:0/996293443 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f4a78004ad0 con 0x7f4a80105f20 2024-03-15T12:57:46.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.415+0000 7f4a6f7fe700 1 --2- 172.21.15.2:0/996293443 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4a680a1b50 0x7f4a680a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:46.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.415+0000 7f4a7e59c700 1 --2- 172.21.15.2:0/996293443 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4a680a1b50 0x7f4a680a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:46.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.415+0000 7f4a6f7fe700 1 -- 172.21.15.2:0/996293443 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f4a780cd440 con 0x7f4a80105f20 2024-03-15T12:57:46.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.415+0000 7f4a7e59c700 1 --2- 172.21.15.2:0/996293443 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4a680a1b50 0x7f4a680a4010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4a74011490 tx=0x7f4a74009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:46.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.419+0000 7f4a6f7fe700 1 -- 172.21.15.2:0/996293443 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4a780961e0 con 0x7f4a80105f20 2024-03-15T12:57:46.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.631+0000 7f4a84e75700 1 -- 172.21.15.2:0/996293443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4a80199890 con 0x7f4a80105f20 2024-03-15T12:57:46.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.631+0000 7f4a6f7fe700 1 -- 172.21.15.2:0/996293443 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4a7800cea0 con 0x7f4a80105f20 2024-03-15T12:57:46.618 INFO:teuthology.orchestra.run.smithi002.stdout:34359738453 2024-03-15T12:57:46.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.633+0000 7f4a84e75700 1 -- 172.21.15.2:0/996293443 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4a680a1b50 msgr2=0x7f4a680a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:46.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.633+0000 7f4a84e75700 1 --2- 172.21.15.2:0/996293443 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4a680a1b50 0x7f4a680a4010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4a74011490 tx=0x7f4a74009260 comp rx=0 tx=0).stop 2024-03-15T12:57:46.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.633+0000 7f4a84e75700 1 -- 172.21.15.2:0/996293443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a80105f20 msgr2=0x7f4a8019ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:46.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.633+0000 7f4a84e75700 1 --2- 172.21.15.2:0/996293443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a80105f20 0x7f4a8019ce00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800ef30 tx=0x7f4a7800c3a0 comp rx=0 tx=0).stop 2024-03-15T12:57:46.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.634+0000 7f4a84e75700 1 -- 172.21.15.2:0/996293443 shutdown_connections 2024-03-15T12:57:46.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.634+0000 7f4a84e75700 1 --2- 172.21.15.2:0/996293443 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4a680a1b50 0x7f4a680a4010 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:46.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.634+0000 7f4a84e75700 1 --2- 172.21.15.2:0/996293443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a80105f20 0x7f4a8019ce00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:46.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.634+0000 7f4a84e75700 1 --2- 172.21.15.2:0/996293443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a80101a30 0x7f4a80198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:46.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.634+0000 7f4a84e75700 1 --2- 172.21.15.2:0/996293443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a801023f0 0x7f4a80198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:46.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.634+0000 7f4a84e75700 1 -- 172.21.15.2:0/996293443 >> 172.21.15.2:0/996293443 conn(0x7f4a80076240 msgr2=0x7f4a80100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:46.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.634+0000 7f4a84e75700 1 -- 172.21.15.2:0/996293443 shutdown_connections 2024-03-15T12:57:46.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:46.634+0000 7f4a84e75700 1 -- 172.21.15.2:0/996293443 wait complete. 2024-03-15T12:57:46.899 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645339 got 309237645346 for osd.10 2024-03-15T12:57:46.900 DEBUG:teuthology.parallel:result is None 2024-03-15T12:57:47.160 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:46 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2189134016' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-15T12:57:47.160 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:46 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/996293443' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-15T12:57:47.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:46 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2189134016' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-15T12:57:47.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:46 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/996293443' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-15T12:57:47.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:46 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2189134016' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-15T12:57:47.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:46 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/996293443' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-15T12:57:47.750 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:57:47.765 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332140 got 219043332146 for osd.7 2024-03-15T12:57:47.765 DEBUG:teuthology.parallel:result is None 2024-03-15T12:57:48.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:47 smithi002 ceph-mon[32060]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:57:48.234 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920828 got 141733920832 for osd.4 2024-03-15T12:57:48.235 DEBUG:teuthology.parallel:result is None 2024-03-15T12:57:48.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:47 smithi184 ceph-mon[35993]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:57:48.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:47 smithi149 ceph-mon[35927]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:57:49.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:49.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:49 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:49.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:49.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:49 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:49.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:49.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:49 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:49.953 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182465 got 107374182470 for osd.3 2024-03-15T12:57:49.953 DEBUG:teuthology.parallel:result is None 2024-03-15T12:57:50.334 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738447 got 34359738453 for osd.0 2024-03-15T12:57:50.335 DEBUG:teuthology.parallel:result is None 2024-03-15T12:57:50.636 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:50 smithi002 ceph-mon[32060]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:57:50.636 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:50 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:57:50.637 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:50 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:57:50.637 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:50 smithi002 ceph-mon[32060]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:50.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:50 smithi184 ceph-mon[35993]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:57:50.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:50 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:57:50.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:50 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:57:50.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:50 smithi184 ceph-mon[35993]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:50.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.856+0000 7f3034072700 1 -- 172.21.15.2:0/2089227341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f302c102a50 msgr2=0x7f302c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:50.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.856+0000 7f3034072700 1 --2- 172.21.15.2:0/2089227341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f302c102a50 0x7f302c10ee40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f302400b0a0 tx=0x7f302400b3b0 comp rx=0 tx=0).stop 2024-03-15T12:57:50.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.856+0000 7f3034072700 1 -- 172.21.15.2:0/2089227341 shutdown_connections 2024-03-15T12:57:50.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.856+0000 7f3034072700 1 --2- 172.21.15.2:0/2089227341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f302c102a50 0x7f302c10ee40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:50.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.856+0000 7f3034072700 1 --2- 172.21.15.2:0/2089227341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302c108100 0x7f302c1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:50.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.856+0000 7f3034072700 1 --2- 172.21.15.2:0/2089227341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f302c102090 0x7f302c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:50.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.856+0000 7f3034072700 1 -- 172.21.15.2:0/2089227341 >> 172.21.15.2:0/2089227341 conn(0x7f302c0790b0 msgr2=0x7f302c0794c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:50.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.856+0000 7f3034072700 1 -- 172.21.15.2:0/2089227341 shutdown_connections 2024-03-15T12:57:50.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.856+0000 7f3034072700 1 -- 172.21.15.2:0/2089227341 wait complete. 2024-03-15T12:57:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.857+0000 7f3034072700 1 Processor -- start 2024-03-15T12:57:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.857+0000 7f3034072700 1 -- start start 2024-03-15T12:57:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.857+0000 7f3034072700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f302c102090 0x7f302c0742e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.857+0000 7f3034072700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302c102a50 0x7f302c074820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.857+0000 7f3034072700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f302c108100 0x7f302c074d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.857+0000 7f3034072700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c1a6b40 con 0x7f302c108100 2024-03-15T12:57:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.857+0000 7f3034072700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c1a6cb0 con 0x7f302c102090 2024-03-15T12:57:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.857+0000 7f3034072700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c1a6e70 con 0x7f302c102a50 2024-03-15T12:57:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.857+0000 7f303260f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f302c108100 0x7f302c074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.857+0000 7f3031e0e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f302c102090 0x7f302c0742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.857+0000 7f303260f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f302c108100 0x7f302c074d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36364/0 (socket says 172.21.15.2:36364) 2024-03-15T12:57:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.857+0000 7f303260f700 1 -- 172.21.15.2:0/1339529237 learned_addr learned my addr 172.21.15.2:0/1339529237 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.858+0000 7f303160d700 1 --2- 172.21.15.2:0/1339529237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302c102a50 0x7f302c074820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.858+0000 7f303260f700 1 -- 172.21.15.2:0/1339529237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302c102a50 msgr2=0x7f302c074820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.858+0000 7f303260f700 1 --2- 172.21.15.2:0/1339529237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302c102a50 0x7f302c074820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.858+0000 7f303260f700 1 -- 172.21.15.2:0/1339529237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f302c102090 msgr2=0x7f302c0742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.858+0000 7f303260f700 1 --2- 172.21.15.2:0/1339529237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f302c102090 0x7f302c0742e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.858+0000 7f303260f700 1 -- 172.21.15.2:0/1339529237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3024009d00 con 0x7f302c108100 2024-03-15T12:57:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.858+0000 7f303260f700 1 --2- 172.21.15.2:0/1339529237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f302c108100 0x7f302c074d60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f302400b780 tx=0x7f3024007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.858+0000 7f3022ffd700 1 -- 172.21.15.2:0/1339529237 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302400e040 con 0x7f302c108100 2024-03-15T12:57:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.858+0000 7f3022ffd700 1 -- 172.21.15.2:0/1339529237 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3024003c10 con 0x7f302c108100 2024-03-15T12:57:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.858+0000 7f3034072700 1 -- 172.21.15.2:0/1339529237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302c1a70f0 con 0x7f302c108100 2024-03-15T12:57:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.858+0000 7f3022ffd700 1 -- 172.21.15.2:0/1339529237 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30240052a0 con 0x7f302c108100 2024-03-15T12:57:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.858+0000 7f3034072700 1 -- 172.21.15.2:0/1339529237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302c1a74b0 con 0x7f302c108100 2024-03-15T12:57:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.859+0000 7f3034072700 1 -- 172.21.15.2:0/1339529237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f302c10a520 con 0x7f302c108100 2024-03-15T12:57:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.860+0000 7f3022ffd700 1 -- 172.21.15.2:0/1339529237 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f302401d480 con 0x7f302c108100 2024-03-15T12:57:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.861+0000 7f3022ffd700 1 --2- 172.21.15.2:0/1339529237 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f30180a1ac0 0x7f30180a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.861+0000 7f3031e0e700 1 --2- 172.21.15.2:0/1339529237 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f30180a1ac0 0x7f30180a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.861+0000 7f3022ffd700 1 -- 172.21.15.2:0/1339529237 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f30240ce9d0 con 0x7f302c108100 2024-03-15T12:57:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.861+0000 7f3031e0e700 1 --2- 172.21.15.2:0/1339529237 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f30180a1ac0 0x7f30180a3f80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3028007b60 tx=0x7f30280073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:50.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:50.865+0000 7f3022ffd700 1 -- 172.21.15.2:0/1339529237 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f30240978d0 con 0x7f302c108100 2024-03-15T12:57:50.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:50 smithi149 ceph-mon[35927]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:57:50.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:50 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:57:50.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:50 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:57:50.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:50 smithi149 ceph-mon[35927]: from='mgr.14150 172.21.15.2:0/3451013547' entity='mgr.x' 2024-03-15T12:57:51.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:51.069+0000 7f3034072700 1 -- 172.21.15.2:0/1339529237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f302c075820 con 0x7f302c108100 2024-03-15T12:57:51.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:51.069+0000 7f3022ffd700 1 -- 172.21.15.2:0/1339529237 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f302409af90 con 0x7f302c108100 2024-03-15T12:57:51.058 INFO:teuthology.orchestra.run.smithi002.stdout:55834574929 2024-03-15T12:57:51.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:51.071+0000 7f3034072700 1 -- 172.21.15.2:0/1339529237 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f30180a1ac0 msgr2=0x7f30180a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:51.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:51.071+0000 7f3034072700 1 --2- 172.21.15.2:0/1339529237 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f30180a1ac0 0x7f30180a3f80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3028007b60 tx=0x7f30280073d0 comp rx=0 tx=0).stop 2024-03-15T12:57:51.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:51.071+0000 7f3034072700 1 -- 172.21.15.2:0/1339529237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f302c108100 msgr2=0x7f302c074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:51.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:51.071+0000 7f3034072700 1 --2- 172.21.15.2:0/1339529237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f302c108100 0x7f302c074d60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f302400b780 tx=0x7f3024007870 comp rx=0 tx=0).stop 2024-03-15T12:57:51.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:51.071+0000 7f3034072700 1 -- 172.21.15.2:0/1339529237 shutdown_connections 2024-03-15T12:57:51.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:51.072+0000 7f3034072700 1 --2- 172.21.15.2:0/1339529237 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f30180a1ac0 0x7f30180a3f80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:51.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:51.072+0000 7f3034072700 1 --2- 172.21.15.2:0/1339529237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f302c108100 0x7f302c074d60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:51.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:51.072+0000 7f3034072700 1 --2- 172.21.15.2:0/1339529237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f302c102090 0x7f302c0742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:51.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:51.072+0000 7f3034072700 1 --2- 172.21.15.2:0/1339529237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f302c102a50 0x7f302c074820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:51.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:51.072+0000 7f3034072700 1 -- 172.21.15.2:0/1339529237 >> 172.21.15.2:0/1339529237 conn(0x7f302c0790b0 msgr2=0x7f302c100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:51.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:51.072+0000 7f3034072700 1 -- 172.21.15.2:0/1339529237 shutdown_connections 2024-03-15T12:57:51.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:51.072+0000 7f3034072700 1 -- 172.21.15.2:0/1339529237 wait complete. 2024-03-15T12:57:51.784 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:51 smithi002 ceph-mon[32060]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-15T12:57:51.784 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:51 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1339529237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-15T12:57:51.786 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574924 got 55834574929 for osd.1 2024-03-15T12:57:51.786 DEBUG:teuthology.parallel:result is None 2024-03-15T12:57:51.786 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-15T12:57:51.786 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-15T12:57:51.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:51 smithi149 ceph-mon[35927]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-15T12:57:51.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:51 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1339529237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-15T12:57:52.032 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:57:52.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:51 smithi184 ceph-mon[35993]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-15T12:57:52.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:51 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1339529237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-15T12:57:52.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:52 smithi002 ceph-mon[32060]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-15T12:57:52.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:52 smithi149 ceph-mon[35927]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-15T12:57:53.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:52 smithi184 ceph-mon[35993]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-15T12:57:53.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.738+0000 7f6db7ed4700 1 -- 172.21.15.2:0/603420011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db01080b0 msgr2=0x7f6db0108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.738+0000 7f6db7ed4700 1 --2- 172.21.15.2:0/603420011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db01080b0 0x7f6db0108490 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009f90 tx=0x7f6da000b330 comp rx=0 tx=0).stop 2024-03-15T12:57:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.738+0000 7f6db7ed4700 1 -- 172.21.15.2:0/603420011 shutdown_connections 2024-03-15T12:57:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.738+0000 7f6db7ed4700 1 --2- 172.21.15.2:0/603420011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db01080b0 0x7f6db0108490 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.738+0000 7f6db7ed4700 1 --2- 172.21.15.2:0/603420011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6db0102a20 0x7f6db010eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.738+0000 7f6db7ed4700 1 --2- 172.21.15.2:0/603420011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db0102060 0x7f6db01024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.738+0000 7f6db7ed4700 1 -- 172.21.15.2:0/603420011 >> 172.21.15.2:0/603420011 conn(0x7f6db0079130 msgr2=0x7f6db0079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.738+0000 7f6db7ed4700 1 -- 172.21.15.2:0/603420011 shutdown_connections 2024-03-15T12:57:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.738+0000 7f6db7ed4700 1 -- 172.21.15.2:0/603420011 wait complete. 2024-03-15T12:57:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db7ed4700 1 Processor -- start 2024-03-15T12:57:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db7ed4700 1 -- start start 2024-03-15T12:57:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db7ed4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6db0102060 0x7f6db01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db7ed4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db0102a20 0x7f6db0198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db7ed4700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db01080b0 0x7f6db019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db7ed4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db019d590 con 0x7f6db0102a20 2024-03-15T12:57:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db7ed4700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db019d700 con 0x7f6db01080b0 2024-03-15T12:57:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db7ed4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db019d8c0 con 0x7f6db0102060 2024-03-15T12:57:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db546f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db0102a20 0x7f6db0198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db546f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db0102a20 0x7f6db0198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36380/0 (socket says 172.21.15.2:36380) 2024-03-15T12:57:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db546f700 1 -- 172.21.15.2:0/2164054293 learned_addr learned my addr 172.21.15.2:0/2164054293 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db5c70700 1 --2- 172.21.15.2:0/2164054293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6db0102060 0x7f6db01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db6471700 1 --2- 172.21.15.2:0/2164054293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db01080b0 0x7f6db019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db5c70700 1 -- 172.21.15.2:0/2164054293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db01080b0 msgr2=0x7f6db019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db5c70700 1 --2- 172.21.15.2:0/2164054293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db01080b0 0x7f6db019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db5c70700 1 -- 172.21.15.2:0/2164054293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db0102a20 msgr2=0x7f6db0198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.739+0000 7f6db5c70700 1 --2- 172.21.15.2:0/2164054293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db0102a20 0x7f6db0198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.740+0000 7f6db5c70700 1 -- 172.21.15.2:0/2164054293 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da0009c40 con 0x7f6db0102060 2024-03-15T12:57:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.740+0000 7f6db546f700 1 --2- 172.21.15.2:0/2164054293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db0102a20 0x7f6db0198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.740+0000 7f6db5c70700 1 --2- 172.21.15.2:0/2164054293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6db0102060 0x7f6db01986b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6da000b6c0 tx=0x7f6da00093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.740+0000 7f6daeffd700 1 -- 172.21.15.2:0/2164054293 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da000e040 con 0x7f6db0102060 2024-03-15T12:57:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.740+0000 7f6daeffd700 1 -- 172.21.15.2:0/2164054293 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6da001d470 con 0x7f6db0102060 2024-03-15T12:57:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.740+0000 7f6db7ed4700 1 -- 172.21.15.2:0/2164054293 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db019db40 con 0x7f6db0102060 2024-03-15T12:57:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.740+0000 7f6daeffd700 1 -- 172.21.15.2:0/2164054293 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da00052a0 con 0x7f6db0102060 2024-03-15T12:57:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.740+0000 7f6db7ed4700 1 -- 172.21.15.2:0/2164054293 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db01a5400 con 0x7f6db0102060 2024-03-15T12:57:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.741+0000 7f6db7ed4700 1 -- 172.21.15.2:0/2164054293 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db010a570 con 0x7f6db0102060 2024-03-15T12:57:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.742+0000 7f6daeffd700 1 -- 172.21.15.2:0/2164054293 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f6da001da00 con 0x7f6db0102060 2024-03-15T12:57:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.742+0000 7f6daeffd700 1 --2- 172.21.15.2:0/2164054293 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6d980a1ac0 0x7f6d980a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.743+0000 7f6db546f700 1 --2- 172.21.15.2:0/2164054293 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6d980a1ac0 0x7f6d980a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.743+0000 7f6daeffd700 1 -- 172.21.15.2:0/2164054293 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f6da00ce920 con 0x7f6db0102060 2024-03-15T12:57:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.743+0000 7f6db546f700 1 --2- 172.21.15.2:0/2164054293 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6d980a1ac0 0x7f6d980a3f80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6db00797f0 tx=0x7f6da4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:53.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.748+0000 7f6daeffd700 1 -- 172.21.15.2:0/2164054293 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6da0097880 con 0x7f6db0102060 2024-03-15T12:57:53.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.948+0000 7f6db7ed4700 1 -- 172.21.15.2:0/2164054293 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6db0068d10 con 0x7f6d980a1ac0 2024-03-15T12:57:53.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.957+0000 7f6daeffd700 1 -- 172.21.15.2:0/2164054293 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340561 (secure 0 0 0) 0x7f6db0068d10 con 0x7f6d980a1ac0 2024-03-15T12:57:53.949 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:57:53.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.961+0000 7f6db7ed4700 1 -- 172.21.15.2:0/2164054293 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6d980a1ac0 msgr2=0x7f6d980a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:53.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.961+0000 7f6db7ed4700 1 --2- 172.21.15.2:0/2164054293 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6d980a1ac0 0x7f6d980a3f80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6db00797f0 tx=0x7f6da4009310 comp rx=0 tx=0).stop 2024-03-15T12:57:53.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.961+0000 7f6db7ed4700 1 -- 172.21.15.2:0/2164054293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6db0102060 msgr2=0x7f6db01986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:53.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.961+0000 7f6db7ed4700 1 --2- 172.21.15.2:0/2164054293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6db0102060 0x7f6db01986b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6da000b6c0 tx=0x7f6da00093e0 comp rx=0 tx=0).stop 2024-03-15T12:57:53.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.961+0000 7f6db7ed4700 1 -- 172.21.15.2:0/2164054293 shutdown_connections 2024-03-15T12:57:53.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.961+0000 7f6db7ed4700 1 --2- 172.21.15.2:0/2164054293 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6d980a1ac0 0x7f6d980a3f80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:53.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.961+0000 7f6db7ed4700 1 --2- 172.21.15.2:0/2164054293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6db01080b0 0x7f6db019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:53.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.961+0000 7f6db7ed4700 1 --2- 172.21.15.2:0/2164054293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6db0102060 0x7f6db01986b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:53.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.961+0000 7f6db7ed4700 1 --2- 172.21.15.2:0/2164054293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6db0102a20 0x7f6db0198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:53.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.961+0000 7f6db7ed4700 1 -- 172.21.15.2:0/2164054293 >> 172.21.15.2:0/2164054293 conn(0x7f6db0079130 msgr2=0x7f6db0100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:53.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.961+0000 7f6db7ed4700 1 -- 172.21.15.2:0/2164054293 shutdown_connections 2024-03-15T12:57:53.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:53.961+0000 7f6db7ed4700 1 -- 172.21.15.2:0/2164054293 wait complete. 2024-03-15T12:57:53.952 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-03-15T12:57:54.630 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-03-15T12:57:52.763948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659439,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":852,"num_read_kb":555,"num_write":428,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":678,"ondisk_log_size":678,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400860,"kb_used_data":14228,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540964,"statfs":{"total":1151940427776,"available":1151529947136,"internally_reserved":0,"allocated":14569472,"data_stored":7958396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":51,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003589"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.787857+0000","last_change":"2024-03-15T12:56:21.210343+0000","last_active":"2024-03-15T12:57:48.787857+0000","last_peered":"2024-03-15T12:57:48.787857+0000","last_clean":"2024-03-15T12:57:48.787857+0000","last_became_active":"2024-03-15T12:56:21.209787+0000","last_became_peered":"2024-03-15T12:56:21.209787+0000","last_unstale":"2024-03-15T12:57:48.787857+0000","last_undegraded":"2024-03-15T12:57:48.787857+0000","last_fullsized":"2024-03-15T12:57:48.787857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:32:21.204944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836466+0000","last_change":"2024-03-15T12:56:21.214153+0000","last_active":"2024-03-15T12:57:19.836466+0000","last_peered":"2024-03-15T12:57:19.836466+0000","last_clean":"2024-03-15T12:57:19.836466+0000","last_became_active":"2024-03-15T12:56:21.213819+0000","last_became_peered":"2024-03-15T12:56:21.213819+0000","last_unstale":"2024-03-15T12:57:19.836466+0000","last_undegraded":"2024-03-15T12:57:19.836466+0000","last_fullsized":"2024-03-15T12:57:19.836466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:31:43.264698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836357+0000","last_change":"2024-03-15T12:56:21.221678+0000","last_active":"2024-03-15T12:57:19.836357+0000","last_peered":"2024-03-15T12:57:19.836357+0000","last_clean":"2024-03-15T12:57:19.836357+0000","last_became_active":"2024-03-15T12:56:21.221272+0000","last_became_peered":"2024-03-15T12:56:21.221272+0000","last_unstale":"2024-03-15T12:57:19.836357+0000","last_undegraded":"2024-03-15T12:57:19.836357+0000","last_fullsized":"2024-03-15T12:57:19.836357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:01:50.778707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837183+0000","last_change":"2024-03-15T12:56:21.212939+0000","last_active":"2024-03-15T12:57:19.837183+0000","last_peered":"2024-03-15T12:57:19.837183+0000","last_clean":"2024-03-15T12:57:19.837183+0000","last_became_active":"2024-03-15T12:56:21.212437+0000","last_became_peered":"2024-03-15T12:56:21.212437+0000","last_unstale":"2024-03-15T12:57:19.837183+0000","last_undegraded":"2024-03-15T12:57:19.837183+0000","last_fullsized":"2024-03-15T12:57:19.837183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:34:11.377020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.771047+0000","last_change":"2024-03-15T12:56:21.221433+0000","last_active":"2024-03-15T12:57:48.771047+0000","last_peered":"2024-03-15T12:57:48.771047+0000","last_clean":"2024-03-15T12:57:48.771047+0000","last_became_active":"2024-03-15T12:56:21.220605+0000","last_became_peered":"2024-03-15T12:56:21.220605+0000","last_unstale":"2024-03-15T12:57:48.771047+0000","last_undegraded":"2024-03-15T12:57:48.771047+0000","last_fullsized":"2024-03-15T12:57:48.771047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:20:25.169387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784751+0000","last_change":"2024-03-15T12:56:21.216539+0000","last_active":"2024-03-15T12:57:48.784751+0000","last_peered":"2024-03-15T12:57:48.784751+0000","last_clean":"2024-03-15T12:57:48.784751+0000","last_became_active":"2024-03-15T12:56:21.216197+0000","last_became_peered":"2024-03-15T12:56:21.216197+0000","last_unstale":"2024-03-15T12:57:48.784751+0000","last_undegraded":"2024-03-15T12:57:48.784751+0000","last_fullsized":"2024-03-15T12:57:48.784751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:00:23.186467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104292+0000","last_change":"2024-03-15T12:56:21.213103+0000","last_active":"2024-03-15T12:57:20.104292+0000","last_peered":"2024-03-15T12:57:20.104292+0000","last_clean":"2024-03-15T12:57:20.104292+0000","last_became_active":"2024-03-15T12:56:21.212789+0000","last_became_peered":"2024-03-15T12:56:21.212789+0000","last_unstale":"2024-03-15T12:57:20.104292+0000","last_undegraded":"2024-03-15T12:57:20.104292+0000","last_fullsized":"2024-03-15T12:57:20.104292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:41:50.140915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104063+0000","last_change":"2024-03-15T12:56:21.214767+0000","last_active":"2024-03-15T12:57:20.104063+0000","last_peered":"2024-03-15T12:57:20.104063+0000","last_clean":"2024-03-15T12:57:20.104063+0000","last_became_active":"2024-03-15T12:56:21.213896+0000","last_became_peered":"2024-03-15T12:56:21.213896+0000","last_unstale":"2024-03-15T12:57:20.104063+0000","last_undegraded":"2024-03-15T12:57:20.104063+0000","last_fullsized":"2024-03-15T12:57:20.104063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:33:53.819906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.796148+0000","last_change":"2024-03-15T12:56:21.218566+0000","last_active":"2024-03-15T12:57:48.796148+0000","last_peered":"2024-03-15T12:57:48.796148+0000","last_clean":"2024-03-15T12:57:48.796148+0000","last_became_active":"2024-03-15T12:56:21.218234+0000","last_became_peered":"2024-03-15T12:56:21.218234+0000","last_unstale":"2024-03-15T12:57:48.796148+0000","last_undegraded":"2024-03-15T12:57:48.796148+0000","last_fullsized":"2024-03-15T12:57:48.796148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:56:46.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784768+0000","last_change":"2024-03-15T12:56:21.219367+0000","last_active":"2024-03-15T12:57:48.784768+0000","last_peered":"2024-03-15T12:57:48.784768+0000","last_clean":"2024-03-15T12:57:48.784768+0000","last_became_active":"2024-03-15T12:56:21.217541+0000","last_became_peered":"2024-03-15T12:56:21.217541+0000","last_unstale":"2024-03-15T12:57:48.784768+0000","last_undegraded":"2024-03-15T12:57:48.784768+0000","last_fullsized":"2024-03-15T12:57:48.784768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:19:12.939394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104625+0000","last_change":"2024-03-15T12:56:21.222246+0000","last_active":"2024-03-15T12:57:20.104625+0000","last_peered":"2024-03-15T12:57:20.104625+0000","last_clean":"2024-03-15T12:57:20.104625+0000","last_became_active":"2024-03-15T12:56:21.221883+0000","last_became_peered":"2024-03-15T12:56:21.221883+0000","last_unstale":"2024-03-15T12:57:20.104625+0000","last_undegraded":"2024-03-15T12:57:20.104625+0000","last_fullsized":"2024-03-15T12:57:20.104625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:42:06.892581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.771534+0000","last_change":"2024-03-15T12:57:19.841386+0000","last_active":"2024-03-15T12:57:48.771534+0000","last_peered":"2024-03-15T12:57:48.771534+0000","last_clean":"2024-03-15T12:57:48.771534+0000","last_became_active":"2024-03-15T12:57:19.840981+0000","last_became_peered":"2024-03-15T12:57:19.840981+0000","last_unstale":"2024-03-15T12:57:48.771534+0000","last_undegraded":"2024-03-15T12:57:48.771534+0000","last_fullsized":"2024-03-15T12:57:48.771534+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:57:28.443452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.776665+0000","last_change":"2024-03-15T12:56:21.199631+0000","last_active":"2024-03-15T12:57:48.776665+0000","last_peered":"2024-03-15T12:57:48.776665+0000","last_clean":"2024-03-15T12:57:48.776665+0000","last_became_active":"2024-03-15T12:56:21.199230+0000","last_became_peered":"2024-03-15T12:56:21.199230+0000","last_unstale":"2024-03-15T12:57:48.776665+0000","last_undegraded":"2024-03-15T12:57:48.776665+0000","last_fullsized":"2024-03-15T12:57:48.776665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:45:51.356803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836190+0000","last_change":"2024-03-15T12:56:21.211449+0000","last_active":"2024-03-15T12:57:19.836190+0000","last_peered":"2024-03-15T12:57:19.836190+0000","last_clean":"2024-03-15T12:57:19.836190+0000","last_became_active":"2024-03-15T12:56:21.211001+0000","last_became_peered":"2024-03-15T12:56:21.211001+0000","last_unstale":"2024-03-15T12:57:19.836190+0000","last_undegraded":"2024-03-15T12:57:19.836190+0000","last_fullsized":"2024-03-15T12:57:19.836190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:32:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836223+0000","last_change":"2024-03-15T12:56:21.212001+0000","last_active":"2024-03-15T12:57:19.836223+0000","last_peered":"2024-03-15T12:57:19.836223+0000","last_clean":"2024-03-15T12:57:19.836223+0000","last_became_active":"2024-03-15T12:56:21.211497+0000","last_became_peered":"2024-03-15T12:56:21.211497+0000","last_unstale":"2024-03-15T12:57:19.836223+0000","last_undegraded":"2024-03-15T12:57:19.836223+0000","last_fullsized":"2024-03-15T12:57:19.836223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:27:16.029202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.799865+0000","last_change":"2024-03-15T12:56:21.222368+0000","last_active":"2024-03-15T12:57:48.799865+0000","last_peered":"2024-03-15T12:57:48.799865+0000","last_clean":"2024-03-15T12:57:48.799865+0000","last_became_active":"2024-03-15T12:56:21.221356+0000","last_became_peered":"2024-03-15T12:56:21.221356+0000","last_unstale":"2024-03-15T12:57:48.799865+0000","last_undegraded":"2024-03-15T12:57:48.799865+0000","last_fullsized":"2024-03-15T12:57:48.799865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:15:04.125220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.789152+0000","last_change":"2024-03-15T12:56:21.211323+0000","last_active":"2024-03-15T12:57:48.789152+0000","last_peered":"2024-03-15T12:57:48.789152+0000","last_clean":"2024-03-15T12:57:48.789152+0000","last_became_active":"2024-03-15T12:56:21.210983+0000","last_became_peered":"2024-03-15T12:56:21.210983+0000","last_unstale":"2024-03-15T12:57:48.789152+0000","last_undegraded":"2024-03-15T12:57:48.789152+0000","last_fullsized":"2024-03-15T12:57:48.789152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:10:51.487985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.775434+0000","last_change":"2024-03-15T12:56:21.209435+0000","last_active":"2024-03-15T12:57:48.775434+0000","last_peered":"2024-03-15T12:57:48.775434+0000","last_clean":"2024-03-15T12:57:48.775434+0000","last_became_active":"2024-03-15T12:56:21.208981+0000","last_became_peered":"2024-03-15T12:56:21.208981+0000","last_unstale":"2024-03-15T12:57:48.775434+0000","last_undegraded":"2024-03-15T12:57:48.775434+0000","last_fullsized":"2024-03-15T12:57:48.775434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:47:59.142040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.768813+0000","last_change":"2024-03-15T12:56:21.222014+0000","last_active":"2024-03-15T12:57:48.768813+0000","last_peered":"2024-03-15T12:57:48.768813+0000","last_clean":"2024-03-15T12:57:48.768813+0000","last_became_active":"2024-03-15T12:56:21.221548+0000","last_became_peered":"2024-03-15T12:56:21.221548+0000","last_unstale":"2024-03-15T12:57:48.768813+0000","last_undegraded":"2024-03-15T12:57:48.768813+0000","last_fullsized":"2024-03-15T12:57:48.768813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:12:48.943874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834595+0000","last_change":"2024-03-15T12:56:25.447200+0000","last_active":"2024-03-15T12:57:19.834595+0000","last_peered":"2024-03-15T12:57:19.834595+0000","last_clean":"2024-03-15T12:57:19.834595+0000","last_became_active":"2024-03-15T12:56:25.446516+0000","last_became_peered":"2024-03-15T12:56:25.446516+0000","last_unstale":"2024-03-15T12:57:19.834595+0000","last_undegraded":"2024-03-15T12:57:19.834595+0000","last_fullsized":"2024-03-15T12:57:19.834595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:06:46.221772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":108,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.518774+0000","last_change":"2024-03-15T12:56:20.210826+0000","last_active":"2024-03-15T12:57:48.518774+0000","last_peered":"2024-03-15T12:57:48.518774+0000","last_clean":"2024-03-15T12:57:48.518774+0000","last_became_active":"2024-03-15T12:56:20.210472+0000","last_became_peered":"2024-03-15T12:56:20.210472+0000","last_unstale":"2024-03-15T12:57:48.518774+0000","last_undegraded":"2024-03-15T12:57:48.518774+0000","last_fullsized":"2024-03-15T12:57:48.518774+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:33:22.896058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104377+0000","last_change":"2024-03-15T12:56:21.224665+0000","last_active":"2024-03-15T12:57:20.104377+0000","last_peered":"2024-03-15T12:57:20.104377+0000","last_clean":"2024-03-15T12:57:20.104377+0000","last_became_active":"2024-03-15T12:56:21.223826+0000","last_became_peered":"2024-03-15T12:56:21.223826+0000","last_unstale":"2024-03-15T12:57:20.104377+0000","last_undegraded":"2024-03-15T12:57:20.104377+0000","last_fullsized":"2024-03-15T12:57:20.104377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:11:57.576701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837559+0000","last_change":"2024-03-15T12:56:25.444665+0000","last_active":"2024-03-15T12:57:19.837559+0000","last_peered":"2024-03-15T12:57:19.837559+0000","last_clean":"2024-03-15T12:57:19.837559+0000","last_became_active":"2024-03-15T12:56:25.444275+0000","last_became_peered":"2024-03-15T12:56:25.444275+0000","last_unstale":"2024-03-15T12:57:19.837559+0000","last_undegraded":"2024-03-15T12:57:19.837559+0000","last_fullsized":"2024-03-15T12:57:19.837559+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:29:46.470405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104578+0000","last_change":"2024-03-15T12:56:20.215873+0000","last_active":"2024-03-15T12:57:20.104578+0000","last_peered":"2024-03-15T12:57:20.104578+0000","last_clean":"2024-03-15T12:57:20.104578+0000","last_became_active":"2024-03-15T12:56:20.215505+0000","last_became_peered":"2024-03-15T12:56:20.215505+0000","last_unstale":"2024-03-15T12:57:20.104578+0000","last_undegraded":"2024-03-15T12:57:20.104578+0000","last_fullsized":"2024-03-15T12:57:20.104578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:01:57.317382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836539+0000","last_change":"2024-03-15T12:56:21.218200+0000","last_active":"2024-03-15T12:57:19.836539+0000","last_peered":"2024-03-15T12:57:19.836539+0000","last_clean":"2024-03-15T12:57:19.836539+0000","last_became_active":"2024-03-15T12:56:21.217768+0000","last_became_peered":"2024-03-15T12:56:21.217768+0000","last_unstale":"2024-03-15T12:57:19.836539+0000","last_undegraded":"2024-03-15T12:57:19.836539+0000","last_fullsized":"2024-03-15T12:57:19.836539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:39:42.443542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837005+0000","last_change":"2024-03-15T12:56:25.444220+0000","last_active":"2024-03-15T12:57:19.837005+0000","last_peered":"2024-03-15T12:57:19.837005+0000","last_clean":"2024-03-15T12:57:19.837005+0000","last_became_active":"2024-03-15T12:56:25.443361+0000","last_became_peered":"2024-03-15T12:56:25.443361+0000","last_unstale":"2024-03-15T12:57:19.837005+0000","last_undegraded":"2024-03-15T12:57:19.837005+0000","last_fullsized":"2024-03-15T12:57:19.837005+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:21:11.593526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837757+0000","last_change":"2024-03-15T12:56:20.218241+0000","last_active":"2024-03-15T12:57:19.837757+0000","last_peered":"2024-03-15T12:57:19.837757+0000","last_clean":"2024-03-15T12:57:19.837757+0000","last_became_active":"2024-03-15T12:56:20.217339+0000","last_became_peered":"2024-03-15T12:56:20.217339+0000","last_unstale":"2024-03-15T12:57:19.837757+0000","last_undegraded":"2024-03-15T12:57:19.837757+0000","last_fullsized":"2024-03-15T12:57:19.837757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:01:22.156502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798260+0000","last_change":"2024-03-15T12:56:21.217224+0000","last_active":"2024-03-15T12:57:48.798260+0000","last_peered":"2024-03-15T12:57:48.798260+0000","last_clean":"2024-03-15T12:57:48.798260+0000","last_became_active":"2024-03-15T12:56:21.216868+0000","last_became_peered":"2024-03-15T12:56:21.216868+0000","last_unstale":"2024-03-15T12:57:48.798260+0000","last_undegraded":"2024-03-15T12:57:48.798260+0000","last_fullsized":"2024-03-15T12:57:48.798260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:03:32.668463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836741+0000","last_change":"2024-03-15T12:56:25.443861+0000","last_active":"2024-03-15T12:57:19.836741+0000","last_peered":"2024-03-15T12:57:19.836741+0000","last_clean":"2024-03-15T12:57:19.836741+0000","last_became_active":"2024-03-15T12:56:25.443386+0000","last_became_peered":"2024-03-15T12:56:25.443386+0000","last_unstale":"2024-03-15T12:57:19.836741+0000","last_undegraded":"2024-03-15T12:57:19.836741+0000","last_fullsized":"2024-03-15T12:57:19.836741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:00:23.063425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837030+0000","last_change":"2024-03-15T12:56:20.212451+0000","last_active":"2024-03-15T12:57:19.837030+0000","last_peered":"2024-03-15T12:57:19.837030+0000","last_clean":"2024-03-15T12:57:19.837030+0000","last_became_active":"2024-03-15T12:56:20.212118+0000","last_became_peered":"2024-03-15T12:56:20.212118+0000","last_unstale":"2024-03-15T12:57:19.837030+0000","last_undegraded":"2024-03-15T12:57:19.837030+0000","last_fullsized":"2024-03-15T12:57:19.837030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:24:33.667652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.766138+0000","last_change":"2024-03-15T12:57:20.107434+0000","last_active":"2024-03-15T12:57:48.766138+0000","last_peered":"2024-03-15T12:57:48.766138+0000","last_clean":"2024-03-15T12:57:48.766138+0000","last_became_active":"2024-03-15T12:57:20.107075+0000","last_became_peered":"2024-03-15T12:57:20.107075+0000","last_unstale":"2024-03-15T12:57:48.766138+0000","last_undegraded":"2024-03-15T12:57:48.766138+0000","last_fullsized":"2024-03-15T12:57:48.766138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:24:44.944629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834590+0000","last_change":"2024-03-15T12:56:25.438194+0000","last_active":"2024-03-15T12:57:19.834590+0000","last_peered":"2024-03-15T12:57:19.834590+0000","last_clean":"2024-03-15T12:57:19.834590+0000","last_became_active":"2024-03-15T12:56:25.437769+0000","last_became_peered":"2024-03-15T12:56:25.437769+0000","last_unstale":"2024-03-15T12:57:19.834590+0000","last_undegraded":"2024-03-15T12:57:19.834590+0000","last_fullsized":"2024-03-15T12:57:19.834590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:29:44.398731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":103,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:47.475653+0000","last_change":"2024-03-15T12:56:20.210552+0000","last_active":"2024-03-15T12:57:47.475653+0000","last_peered":"2024-03-15T12:57:47.475653+0000","last_clean":"2024-03-15T12:57:47.475653+0000","last_became_active":"2024-03-15T12:56:20.210191+0000","last_became_peered":"2024-03-15T12:56:20.210191+0000","last_unstale":"2024-03-15T12:57:47.475653+0000","last_undegraded":"2024-03-15T12:57:47.475653+0000","last_fullsized":"2024-03-15T12:57:47.475653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:25:48.225266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.754896+0000","last_change":"2024-03-15T12:56:21.220582+0000","last_active":"2024-03-15T12:57:48.754896+0000","last_peered":"2024-03-15T12:57:48.754896+0000","last_clean":"2024-03-15T12:57:48.754896+0000","last_became_active":"2024-03-15T12:56:21.220212+0000","last_became_peered":"2024-03-15T12:56:21.220212+0000","last_unstale":"2024-03-15T12:57:48.754896+0000","last_undegraded":"2024-03-15T12:57:48.754896+0000","last_fullsized":"2024-03-15T12:57:48.754896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:00:54.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834490+0000","last_change":"2024-03-15T12:56:25.446410+0000","last_active":"2024-03-15T12:57:19.834490+0000","last_peered":"2024-03-15T12:57:19.834490+0000","last_clean":"2024-03-15T12:57:19.834490+0000","last_became_active":"2024-03-15T12:56:25.445554+0000","last_became_peered":"2024-03-15T12:56:25.445554+0000","last_unstale":"2024-03-15T12:57:19.834490+0000","last_undegraded":"2024-03-15T12:57:19.834490+0000","last_fullsized":"2024-03-15T12:57:19.834490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:35:08.049374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.839292+0000","last_change":"2024-03-15T12:56:20.212599+0000","last_active":"2024-03-15T12:57:19.839292+0000","last_peered":"2024-03-15T12:57:19.839292+0000","last_clean":"2024-03-15T12:57:19.839292+0000","last_became_active":"2024-03-15T12:56:20.212289+0000","last_became_peered":"2024-03-15T12:56:20.212289+0000","last_unstale":"2024-03-15T12:57:19.839292+0000","last_undegraded":"2024-03-15T12:57:19.839292+0000","last_fullsized":"2024-03-15T12:57:19.839292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:12:51.036882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.764372+0000","last_change":"2024-03-15T12:56:21.219407+0000","last_active":"2024-03-15T12:57:48.764372+0000","last_peered":"2024-03-15T12:57:48.764372+0000","last_clean":"2024-03-15T12:57:48.764372+0000","last_became_active":"2024-03-15T12:56:21.219035+0000","last_became_peered":"2024-03-15T12:56:21.219035+0000","last_unstale":"2024-03-15T12:57:48.764372+0000","last_undegraded":"2024-03-15T12:57:48.764372+0000","last_fullsized":"2024-03-15T12:57:48.764372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:35:43.129767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.841693+0000","last_change":"2024-03-15T12:57:19.841693+0000","last_active":"2024-03-15T12:57:19.841693+0000","last_peered":"2024-03-15T12:57:19.841693+0000","last_clean":"2024-03-15T12:57:19.841693+0000","last_became_active":"2024-03-15T12:57:19.841275+0000","last_became_peered":"2024-03-15T12:57:19.841275+0000","last_unstale":"2024-03-15T12:57:19.841693+0000","last_undegraded":"2024-03-15T12:57:19.841693+0000","last_fullsized":"2024-03-15T12:57:19.841693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:41:18.147235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837559+0000","last_change":"2024-03-15T12:56:20.209933+0000","last_active":"2024-03-15T12:57:19.837559+0000","last_peered":"2024-03-15T12:57:19.837559+0000","last_clean":"2024-03-15T12:57:19.837559+0000","last_became_active":"2024-03-15T12:56:20.209047+0000","last_became_peered":"2024-03-15T12:56:20.209047+0000","last_unstale":"2024-03-15T12:57:19.837559+0000","last_undegraded":"2024-03-15T12:57:19.837559+0000","last_fullsized":"2024-03-15T12:57:19.837559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:24:05.551745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103877+0000","last_change":"2024-03-15T12:56:21.215198+0000","last_active":"2024-03-15T12:57:20.103877+0000","last_peered":"2024-03-15T12:57:20.103877+0000","last_clean":"2024-03-15T12:57:20.103877+0000","last_became_active":"2024-03-15T12:56:21.214844+0000","last_became_peered":"2024-03-15T12:56:21.214844+0000","last_unstale":"2024-03-15T12:57:20.103877+0000","last_undegraded":"2024-03-15T12:57:20.103877+0000","last_fullsized":"2024-03-15T12:57:20.103877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:37:08.398317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836360+0000","last_change":"2024-03-15T12:56:25.437047+0000","last_active":"2024-03-15T12:57:19.836360+0000","last_peered":"2024-03-15T12:57:19.836360+0000","last_clean":"2024-03-15T12:57:19.836360+0000","last_became_active":"2024-03-15T12:56:25.436595+0000","last_became_peered":"2024-03-15T12:56:25.436595+0000","last_unstale":"2024-03-15T12:57:19.836360+0000","last_undegraded":"2024-03-15T12:57:19.836360+0000","last_fullsized":"2024-03-15T12:57:19.836360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:59:55.147044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103923+0000","last_change":"2024-03-15T12:56:20.206348+0000","last_active":"2024-03-15T12:57:20.103923+0000","last_peered":"2024-03-15T12:57:20.103923+0000","last_clean":"2024-03-15T12:57:20.103923+0000","last_became_active":"2024-03-15T12:56:20.206013+0000","last_became_peered":"2024-03-15T12:56:20.206013+0000","last_unstale":"2024-03-15T12:57:20.103923+0000","last_undegraded":"2024-03-15T12:57:20.103923+0000","last_fullsized":"2024-03-15T12:57:20.103923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:32:26.625982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782151+0000","last_change":"2024-03-15T12:56:21.216517+0000","last_active":"2024-03-15T12:57:48.782151+0000","last_peered":"2024-03-15T12:57:48.782151+0000","last_clean":"2024-03-15T12:57:48.782151+0000","last_became_active":"2024-03-15T12:56:21.216076+0000","last_became_peered":"2024-03-15T12:56:21.216076+0000","last_unstale":"2024-03-15T12:57:48.782151+0000","last_undegraded":"2024-03-15T12:57:48.782151+0000","last_fullsized":"2024-03-15T12:57:48.782151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:19:35.547273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104273+0000","last_change":"2024-03-15T12:56:25.437367+0000","last_active":"2024-03-15T12:57:20.104273+0000","last_peered":"2024-03-15T12:57:20.104273+0000","last_clean":"2024-03-15T12:57:20.104273+0000","last_became_active":"2024-03-15T12:56:25.436930+0000","last_became_peered":"2024-03-15T12:56:25.436930+0000","last_unstale":"2024-03-15T12:57:20.104273+0000","last_undegraded":"2024-03-15T12:57:20.104273+0000","last_fullsized":"2024-03-15T12:57:20.104273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:53:56.586786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":168,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:49.477250+0000","last_change":"2024-03-15T12:56:20.214067+0000","last_active":"2024-03-15T12:57:49.477250+0000","last_peered":"2024-03-15T12:57:49.477250+0000","last_clean":"2024-03-15T12:57:49.477250+0000","last_became_active":"2024-03-15T12:56:20.213696+0000","last_became_peered":"2024-03-15T12:56:20.213696+0000","last_unstale":"2024-03-15T12:57:49.477250+0000","last_undegraded":"2024-03-15T12:57:49.477250+0000","last_fullsized":"2024-03-15T12:57:49.477250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:35:15.343660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":137,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.797642+0000","last_change":"2024-03-15T12:56:21.211106+0000","last_active":"2024-03-15T12:57:48.797642+0000","last_peered":"2024-03-15T12:57:48.797642+0000","last_clean":"2024-03-15T12:57:48.797642+0000","last_became_active":"2024-03-15T12:56:21.210756+0000","last_became_peered":"2024-03-15T12:56:21.210756+0000","last_unstale":"2024-03-15T12:57:48.797642+0000","last_undegraded":"2024-03-15T12:57:48.797642+0000","last_fullsized":"2024-03-15T12:57:48.797642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:13:25.232077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.833697+0000","last_change":"2024-03-15T12:56:25.441364+0000","last_active":"2024-03-15T12:57:19.833697+0000","last_peered":"2024-03-15T12:57:19.833697+0000","last_clean":"2024-03-15T12:57:19.833697+0000","last_became_active":"2024-03-15T12:56:25.440951+0000","last_became_peered":"2024-03-15T12:56:25.440951+0000","last_unstale":"2024-03-15T12:57:19.833697+0000","last_undegraded":"2024-03-15T12:57:19.833697+0000","last_fullsized":"2024-03-15T12:57:19.833697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:32:28.651856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104583+0000","last_change":"2024-03-15T12:56:20.208633+0000","last_active":"2024-03-15T12:57:20.104583+0000","last_peered":"2024-03-15T12:57:20.104583+0000","last_clean":"2024-03-15T12:57:20.104583+0000","last_became_active":"2024-03-15T12:56:20.208280+0000","last_became_peered":"2024-03-15T12:56:20.208280+0000","last_unstale":"2024-03-15T12:57:20.104583+0000","last_undegraded":"2024-03-15T12:57:20.104583+0000","last_fullsized":"2024-03-15T12:57:20.104583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:00:15.274090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836549+0000","last_change":"2024-03-15T12:56:21.218998+0000","last_active":"2024-03-15T12:57:19.836549+0000","last_peered":"2024-03-15T12:57:19.836549+0000","last_clean":"2024-03-15T12:57:19.836549+0000","last_became_active":"2024-03-15T12:56:21.218617+0000","last_became_peered":"2024-03-15T12:56:21.218617+0000","last_unstale":"2024-03-15T12:57:19.836549+0000","last_undegraded":"2024-03-15T12:57:19.836549+0000","last_fullsized":"2024-03-15T12:57:19.836549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:20:25.496204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837212+0000","last_change":"2024-03-15T12:56:25.438969+0000","last_active":"2024-03-15T12:57:19.837212+0000","last_peered":"2024-03-15T12:57:19.837212+0000","last_clean":"2024-03-15T12:57:19.837212+0000","last_became_active":"2024-03-15T12:56:25.438603+0000","last_became_peered":"2024-03-15T12:56:25.438603+0000","last_unstale":"2024-03-15T12:57:19.837212+0000","last_undegraded":"2024-03-15T12:57:19.837212+0000","last_fullsized":"2024-03-15T12:57:19.837212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:54:23.992253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104585+0000","last_change":"2024-03-15T12:56:20.215152+0000","last_active":"2024-03-15T12:57:20.104585+0000","last_peered":"2024-03-15T12:57:20.104585+0000","last_clean":"2024-03-15T12:57:20.104585+0000","last_became_active":"2024-03-15T12:56:20.214762+0000","last_became_peered":"2024-03-15T12:56:20.214762+0000","last_unstale":"2024-03-15T12:57:20.104585+0000","last_undegraded":"2024-03-15T12:57:20.104585+0000","last_fullsized":"2024-03-15T12:57:20.104585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:38:45.414984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.790090+0000","last_change":"2024-03-15T12:56:21.214005+0000","last_active":"2024-03-15T12:57:48.790090+0000","last_peered":"2024-03-15T12:57:48.790090+0000","last_clean":"2024-03-15T12:57:48.790090+0000","last_became_active":"2024-03-15T12:56:21.213672+0000","last_became_peered":"2024-03-15T12:56:21.213672+0000","last_unstale":"2024-03-15T12:57:48.790090+0000","last_undegraded":"2024-03-15T12:57:48.790090+0000","last_fullsized":"2024-03-15T12:57:48.790090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:13:09.384245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.833759+0000","last_change":"2024-03-15T12:56:25.443145+0000","last_active":"2024-03-15T12:57:19.833759+0000","last_peered":"2024-03-15T12:57:19.833759+0000","last_clean":"2024-03-15T12:57:19.833759+0000","last_became_active":"2024-03-15T12:56:25.442755+0000","last_became_peered":"2024-03-15T12:56:25.442755+0000","last_unstale":"2024-03-15T12:57:19.833759+0000","last_undegraded":"2024-03-15T12:57:19.833759+0000","last_fullsized":"2024-03-15T12:57:19.833759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:14:47.201919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":211,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:49.482481+0000","last_change":"2024-03-15T12:56:20.215063+0000","last_active":"2024-03-15T12:57:49.482481+0000","last_peered":"2024-03-15T12:57:49.482481+0000","last_clean":"2024-03-15T12:57:49.482481+0000","last_became_active":"2024-03-15T12:56:20.214742+0000","last_became_peered":"2024-03-15T12:56:20.214742+0000","last_unstale":"2024-03-15T12:57:49.482481+0000","last_undegraded":"2024-03-15T12:57:49.482481+0000","last_fullsized":"2024-03-15T12:57:49.482481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:38:12.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62024,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":148,"num_read_kb":135,"num_write":38,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.773641+0000","last_change":"2024-03-15T12:56:21.213519+0000","last_active":"2024-03-15T12:57:48.773641+0000","last_peered":"2024-03-15T12:57:48.773641+0000","last_clean":"2024-03-15T12:57:48.773641+0000","last_became_active":"2024-03-15T12:56:21.213198+0000","last_became_peered":"2024-03-15T12:56:21.213198+0000","last_unstale":"2024-03-15T12:57:48.773641+0000","last_undegraded":"2024-03-15T12:57:48.773641+0000","last_fullsized":"2024-03-15T12:57:48.773641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:22:13.037068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106198+0000","last_change":"2024-03-15T12:57:20.106198+0000","last_active":"2024-03-15T12:57:20.106198+0000","last_peered":"2024-03-15T12:57:20.106198+0000","last_clean":"2024-03-15T12:57:20.106198+0000","last_became_active":"2024-03-15T12:57:20.105649+0000","last_became_peered":"2024-03-15T12:57:20.105649+0000","last_unstale":"2024-03-15T12:57:20.106198+0000","last_undegraded":"2024-03-15T12:57:20.106198+0000","last_fullsized":"2024-03-15T12:57:20.106198+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:44:46.615091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834552+0000","last_change":"2024-03-15T12:56:20.216124+0000","last_active":"2024-03-15T12:57:19.834552+0000","last_peered":"2024-03-15T12:57:19.834552+0000","last_clean":"2024-03-15T12:57:19.834552+0000","last_became_active":"2024-03-15T12:56:20.215754+0000","last_became_peered":"2024-03-15T12:56:20.215754+0000","last_unstale":"2024-03-15T12:57:19.834552+0000","last_undegraded":"2024-03-15T12:57:19.834552+0000","last_fullsized":"2024-03-15T12:57:19.834552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:11:44.403503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.781601+0000","last_change":"2024-03-15T12:56:21.214775+0000","last_active":"2024-03-15T12:57:48.781601+0000","last_peered":"2024-03-15T12:57:48.781601+0000","last_clean":"2024-03-15T12:57:48.781601+0000","last_became_active":"2024-03-15T12:56:21.214445+0000","last_became_peered":"2024-03-15T12:56:21.214445+0000","last_unstale":"2024-03-15T12:57:48.781601+0000","last_undegraded":"2024-03-15T12:57:48.781601+0000","last_fullsized":"2024-03-15T12:57:48.781601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:23:24.045670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.107668+0000","last_change":"2024-03-15T12:57:20.107668+0000","last_active":"2024-03-15T12:57:20.107668+0000","last_peered":"2024-03-15T12:57:20.107668+0000","last_clean":"2024-03-15T12:57:20.107668+0000","last_became_active":"2024-03-15T12:57:20.107259+0000","last_became_peered":"2024-03-15T12:57:20.107259+0000","last_unstale":"2024-03-15T12:57:20.107668+0000","last_undegraded":"2024-03-15T12:57:20.107668+0000","last_fullsized":"2024-03-15T12:57:20.107668+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:07:37.244869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104267+0000","last_change":"2024-03-15T12:56:20.217191+0000","last_active":"2024-03-15T12:57:20.104267+0000","last_peered":"2024-03-15T12:57:20.104267+0000","last_clean":"2024-03-15T12:57:20.104267+0000","last_became_active":"2024-03-15T12:56:20.216632+0000","last_became_peered":"2024-03-15T12:56:20.216632+0000","last_unstale":"2024-03-15T12:57:20.104267+0000","last_undegraded":"2024-03-15T12:57:20.104267+0000","last_fullsized":"2024-03-15T12:57:20.104267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:16:51.187920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104510+0000","last_change":"2024-03-15T12:56:21.218650+0000","last_active":"2024-03-15T12:57:20.104510+0000","last_peered":"2024-03-15T12:57:20.104510+0000","last_clean":"2024-03-15T12:57:20.104510+0000","last_became_active":"2024-03-15T12:56:21.218329+0000","last_became_peered":"2024-03-15T12:56:21.218329+0000","last_unstale":"2024-03-15T12:57:20.104510+0000","last_undegraded":"2024-03-15T12:57:20.104510+0000","last_fullsized":"2024-03-15T12:57:20.104510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:10:45.632248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834415+0000","last_change":"2024-03-15T12:56:25.447982+0000","last_active":"2024-03-15T12:57:19.834415+0000","last_peered":"2024-03-15T12:57:19.834415+0000","last_clean":"2024-03-15T12:57:19.834415+0000","last_became_active":"2024-03-15T12:56:25.447514+0000","last_became_peered":"2024-03-15T12:56:25.447514+0000","last_unstale":"2024-03-15T12:57:19.834415+0000","last_undegraded":"2024-03-15T12:57:19.834415+0000","last_fullsized":"2024-03-15T12:57:19.834415+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:16:52.567619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":101,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:47.518178+0000","last_change":"2024-03-15T12:56:20.209778+0000","last_active":"2024-03-15T12:57:47.518178+0000","last_peered":"2024-03-15T12:57:47.518178+0000","last_clean":"2024-03-15T12:57:47.518178+0000","last_became_active":"2024-03-15T12:56:20.209311+0000","last_became_peered":"2024-03-15T12:56:20.209311+0000","last_unstale":"2024-03-15T12:57:47.518178+0000","last_undegraded":"2024-03-15T12:57:47.518178+0000","last_fullsized":"2024-03-15T12:57:47.518178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:05:21.930210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104595+0000","last_change":"2024-03-15T12:56:21.215576+0000","last_active":"2024-03-15T12:57:20.104595+0000","last_peered":"2024-03-15T12:57:20.104595+0000","last_clean":"2024-03-15T12:57:20.104595+0000","last_became_active":"2024-03-15T12:56:21.215261+0000","last_became_peered":"2024-03-15T12:56:21.215261+0000","last_unstale":"2024-03-15T12:57:20.104595+0000","last_undegraded":"2024-03-15T12:57:20.104595+0000","last_fullsized":"2024-03-15T12:57:20.104595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:35:15.461554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837297+0000","last_change":"2024-03-15T12:56:25.445559+0000","last_active":"2024-03-15T12:57:19.837297+0000","last_peered":"2024-03-15T12:57:19.837297+0000","last_clean":"2024-03-15T12:57:19.837297+0000","last_became_active":"2024-03-15T12:56:25.445092+0000","last_became_peered":"2024-03-15T12:56:25.445092+0000","last_unstale":"2024-03-15T12:57:19.837297+0000","last_undegraded":"2024-03-15T12:57:19.837297+0000","last_fullsized":"2024-03-15T12:57:19.837297+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:48:30.558243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836745+0000","last_change":"2024-03-15T12:56:20.214853+0000","last_active":"2024-03-15T12:57:19.836745+0000","last_peered":"2024-03-15T12:57:19.836745+0000","last_clean":"2024-03-15T12:57:19.836745+0000","last_became_active":"2024-03-15T12:56:20.214510+0000","last_became_peered":"2024-03-15T12:56:20.214510+0000","last_unstale":"2024-03-15T12:57:19.836745+0000","last_undegraded":"2024-03-15T12:57:19.836745+0000","last_fullsized":"2024-03-15T12:57:19.836745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:25:39.316428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798162+0000","last_change":"2024-03-15T12:56:21.216511+0000","last_active":"2024-03-15T12:57:48.798162+0000","last_peered":"2024-03-15T12:57:48.798162+0000","last_clean":"2024-03-15T12:57:48.798162+0000","last_became_active":"2024-03-15T12:56:21.216175+0000","last_became_peered":"2024-03-15T12:56:21.216175+0000","last_unstale":"2024-03-15T12:57:48.798162+0000","last_undegraded":"2024-03-15T12:57:48.798162+0000","last_fullsized":"2024-03-15T12:57:48.798162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:16:56.665841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103994+0000","last_change":"2024-03-15T12:56:25.446009+0000","last_active":"2024-03-15T12:57:20.103994+0000","last_peered":"2024-03-15T12:57:20.103994+0000","last_clean":"2024-03-15T12:57:20.103994+0000","last_became_active":"2024-03-15T12:56:25.445474+0000","last_became_peered":"2024-03-15T12:56:25.445474+0000","last_unstale":"2024-03-15T12:57:20.103994+0000","last_undegraded":"2024-03-15T12:57:20.103994+0000","last_fullsized":"2024-03-15T12:57:20.103994+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:53:46.856905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837300+0000","last_change":"2024-03-15T12:56:20.210146+0000","last_active":"2024-03-15T12:57:19.837300+0000","last_peered":"2024-03-15T12:57:19.837300+0000","last_clean":"2024-03-15T12:57:19.837300+0000","last_became_active":"2024-03-15T12:56:20.209769+0000","last_became_peered":"2024-03-15T12:56:20.209769+0000","last_unstale":"2024-03-15T12:57:19.837300+0000","last_undegraded":"2024-03-15T12:57:19.837300+0000","last_fullsized":"2024-03-15T12:57:19.837300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:47:32.505257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.770447+0000","last_change":"2024-03-15T12:56:21.214826+0000","last_active":"2024-03-15T12:57:48.770447+0000","last_peered":"2024-03-15T12:57:48.770447+0000","last_clean":"2024-03-15T12:57:48.770447+0000","last_became_active":"2024-03-15T12:56:21.214160+0000","last_became_peered":"2024-03-15T12:56:21.214160+0000","last_unstale":"2024-03-15T12:57:48.770447+0000","last_undegraded":"2024-03-15T12:57:48.770447+0000","last_fullsized":"2024-03-15T12:57:48.770447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:23:17.800522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837038+0000","last_change":"2024-03-15T12:56:25.443496+0000","last_active":"2024-03-15T12:57:19.837038+0000","last_peered":"2024-03-15T12:57:19.837038+0000","last_clean":"2024-03-15T12:57:19.837038+0000","last_became_active":"2024-03-15T12:56:25.442796+0000","last_became_peered":"2024-03-15T12:56:25.442796+0000","last_unstale":"2024-03-15T12:57:19.837038+0000","last_undegraded":"2024-03-15T12:57:19.837038+0000","last_fullsized":"2024-03-15T12:57:19.837038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:35:23.892693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.840452+0000","last_change":"2024-03-15T12:57:19.840452+0000","last_active":"2024-03-15T12:57:19.840452+0000","last_peered":"2024-03-15T12:57:19.840452+0000","last_clean":"2024-03-15T12:57:19.840452+0000","last_became_active":"2024-03-15T12:57:19.839977+0000","last_became_peered":"2024-03-15T12:57:19.839977+0000","last_unstale":"2024-03-15T12:57:19.840452+0000","last_undegraded":"2024-03-15T12:57:19.840452+0000","last_fullsized":"2024-03-15T12:57:19.840452+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:59:25.026455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.777760+0000","last_change":"2024-03-15T12:56:21.216393+0000","last_active":"2024-03-15T12:57:48.777760+0000","last_peered":"2024-03-15T12:57:48.777760+0000","last_clean":"2024-03-15T12:57:48.777760+0000","last_became_active":"2024-03-15T12:56:21.216043+0000","last_became_peered":"2024-03-15T12:56:21.216043+0000","last_unstale":"2024-03-15T12:57:48.777760+0000","last_undegraded":"2024-03-15T12:57:48.777760+0000","last_fullsized":"2024-03-15T12:57:48.777760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:05:49.843726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837337+0000","last_change":"2024-03-15T12:56:25.443854+0000","last_active":"2024-03-15T12:57:19.837337+0000","last_peered":"2024-03-15T12:57:19.837337+0000","last_clean":"2024-03-15T12:57:19.837337+0000","last_became_active":"2024-03-15T12:56:25.443422+0000","last_became_peered":"2024-03-15T12:56:25.443422+0000","last_unstale":"2024-03-15T12:57:19.837337+0000","last_undegraded":"2024-03-15T12:57:19.837337+0000","last_fullsized":"2024-03-15T12:57:19.837337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:52:18.999161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":98,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:46.517003+0000","last_change":"2024-03-15T12:56:20.211493+0000","last_active":"2024-03-15T12:57:46.517003+0000","last_peered":"2024-03-15T12:57:46.517003+0000","last_clean":"2024-03-15T12:57:46.517003+0000","last_became_active":"2024-03-15T12:56:20.211154+0000","last_became_peered":"2024-03-15T12:56:20.211154+0000","last_unstale":"2024-03-15T12:57:46.517003+0000","last_undegraded":"2024-03-15T12:57:46.517003+0000","last_fullsized":"2024-03-15T12:57:46.517003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:26:28.026899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.783016+0000","last_change":"2024-03-15T12:56:21.215999+0000","last_active":"2024-03-15T12:57:48.783016+0000","last_peered":"2024-03-15T12:57:48.783016+0000","last_clean":"2024-03-15T12:57:48.783016+0000","last_became_active":"2024-03-15T12:56:21.215657+0000","last_became_peered":"2024-03-15T12:56:21.215657+0000","last_unstale":"2024-03-15T12:57:48.783016+0000","last_undegraded":"2024-03-15T12:57:48.783016+0000","last_fullsized":"2024-03-15T12:57:48.783016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:47:59.901886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837243+0000","last_change":"2024-03-15T12:56:25.442246+0000","last_active":"2024-03-15T12:57:19.837243+0000","last_peered":"2024-03-15T12:57:19.837243+0000","last_clean":"2024-03-15T12:57:19.837243+0000","last_became_active":"2024-03-15T12:56:25.441755+0000","last_became_peered":"2024-03-15T12:56:25.441755+0000","last_unstale":"2024-03-15T12:57:19.837243+0000","last_undegraded":"2024-03-15T12:57:19.837243+0000","last_fullsized":"2024-03-15T12:57:19.837243+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:47:36.037864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.753331+0000","last_change":"2024-03-15T12:56:21.215746+0000","last_active":"2024-03-15T12:57:48.753331+0000","last_peered":"2024-03-15T12:57:48.753331+0000","last_clean":"2024-03-15T12:57:48.753331+0000","last_became_active":"2024-03-15T12:56:21.215424+0000","last_became_peered":"2024-03-15T12:56:21.215424+0000","last_unstale":"2024-03-15T12:57:48.753331+0000","last_undegraded":"2024-03-15T12:57:48.753331+0000","last_fullsized":"2024-03-15T12:57:48.753331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:01:15.498119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":101,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.733391+0000","last_change":"2024-03-15T12:56:20.219705+0000","last_active":"2024-03-15T12:57:48.733391+0000","last_peered":"2024-03-15T12:57:48.733391+0000","last_clean":"2024-03-15T12:57:48.733391+0000","last_became_active":"2024-03-15T12:56:20.219328+0000","last_became_peered":"2024-03-15T12:56:20.219328+0000","last_unstale":"2024-03-15T12:57:48.733391+0000","last_undegraded":"2024-03-15T12:57:48.733391+0000","last_fullsized":"2024-03-15T12:57:48.733391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:47:52.171116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104385+0000","last_change":"2024-03-15T12:56:25.443094+0000","last_active":"2024-03-15T12:57:20.104385+0000","last_peered":"2024-03-15T12:57:20.104385+0000","last_clean":"2024-03-15T12:57:20.104385+0000","last_became_active":"2024-03-15T12:56:25.442708+0000","last_became_peered":"2024-03-15T12:56:25.442708+0000","last_unstale":"2024-03-15T12:57:20.104385+0000","last_undegraded":"2024-03-15T12:57:20.104385+0000","last_fullsized":"2024-03-15T12:57:20.104385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:03:07.769836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104103+0000","last_change":"2024-03-15T12:56:21.223937+0000","last_active":"2024-03-15T12:57:20.104103+0000","last_peered":"2024-03-15T12:57:20.104103+0000","last_clean":"2024-03-15T12:57:20.104103+0000","last_became_active":"2024-03-15T12:56:21.223583+0000","last_became_peered":"2024-03-15T12:56:21.223583+0000","last_unstale":"2024-03-15T12:57:20.104103+0000","last_undegraded":"2024-03-15T12:57:20.104103+0000","last_fullsized":"2024-03-15T12:57:20.104103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:15:34.287445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837101+0000","last_change":"2024-03-15T12:56:20.210812+0000","last_active":"2024-03-15T12:57:19.837101+0000","last_peered":"2024-03-15T12:57:19.837101+0000","last_clean":"2024-03-15T12:57:19.837101+0000","last_became_active":"2024-03-15T12:56:20.210457+0000","last_became_peered":"2024-03-15T12:56:20.210457+0000","last_unstale":"2024-03-15T12:57:19.837101+0000","last_undegraded":"2024-03-15T12:57:19.837101+0000","last_fullsized":"2024-03-15T12:57:19.837101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:04:37.371637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":357,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104295+0000","last_change":"2024-03-15T12:55:08.392548+0000","last_active":"2024-03-15T12:57:20.104295+0000","last_peered":"2024-03-15T12:57:20.104295+0000","last_clean":"2024-03-15T12:57:20.104295+0000","last_became_active":"2024-03-15T12:55:08.391972+0000","last_became_peered":"2024-03-15T12:55:08.391972+0000","last_unstale":"2024-03-15T12:57:20.104295+0000","last_undegraded":"2024-03-15T12:57:20.104295+0000","last_fullsized":"2024-03-15T12:57:20.104295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:51:47.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:51:47.093558+0000","last_clean_scrub_stamp":"2024-03-15T12:51:47.093558+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:30:57.264886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104859+0000","last_change":"2024-03-15T12:56:25.443212+0000","last_active":"2024-03-15T12:57:20.104859+0000","last_peered":"2024-03-15T12:57:20.104859+0000","last_clean":"2024-03-15T12:57:20.104859+0000","last_became_active":"2024-03-15T12:56:25.442787+0000","last_became_peered":"2024-03-15T12:56:25.442787+0000","last_unstale":"2024-03-15T12:57:20.104859+0000","last_undegraded":"2024-03-15T12:57:20.104859+0000","last_fullsized":"2024-03-15T12:57:20.104859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:59:32.244953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.775617+0000","last_change":"2024-03-15T12:57:19.839935+0000","last_active":"2024-03-15T12:57:48.775617+0000","last_peered":"2024-03-15T12:57:48.775617+0000","last_clean":"2024-03-15T12:57:48.775617+0000","last_became_active":"2024-03-15T12:57:19.839546+0000","last_became_peered":"2024-03-15T12:57:19.839546+0000","last_unstale":"2024-03-15T12:57:48.775617+0000","last_undegraded":"2024-03-15T12:57:48.775617+0000","last_fullsized":"2024-03-15T12:57:48.775617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:05:15.544159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104492+0000","last_change":"2024-03-15T12:56:20.210466+0000","last_active":"2024-03-15T12:57:20.104492+0000","last_peered":"2024-03-15T12:57:20.104492+0000","last_clean":"2024-03-15T12:57:20.104492+0000","last_became_active":"2024-03-15T12:56:20.210157+0000","last_became_peered":"2024-03-15T12:56:20.210157+0000","last_unstale":"2024-03-15T12:57:20.104492+0000","last_undegraded":"2024-03-15T12:57:20.104492+0000","last_fullsized":"2024-03-15T12:57:20.104492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:58:03.335826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.838275+0000","last_change":"2024-03-15T12:56:25.432641+0000","last_active":"2024-03-15T12:57:19.838275+0000","last_peered":"2024-03-15T12:57:19.838275+0000","last_clean":"2024-03-15T12:57:19.838275+0000","last_became_active":"2024-03-15T12:56:25.432212+0000","last_became_peered":"2024-03-15T12:56:25.432212+0000","last_unstale":"2024-03-15T12:57:19.838275+0000","last_undegraded":"2024-03-15T12:57:19.838275+0000","last_fullsized":"2024-03-15T12:57:19.838275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:40:20.447886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.846534+0000","last_change":"2024-03-15T12:57:19.846534+0000","last_active":"2024-03-15T12:57:19.846534+0000","last_peered":"2024-03-15T12:57:19.846534+0000","last_clean":"2024-03-15T12:57:19.846534+0000","last_became_active":"2024-03-15T12:57:19.841008+0000","last_became_peered":"2024-03-15T12:57:19.841008+0000","last_unstale":"2024-03-15T12:57:19.846534+0000","last_undegraded":"2024-03-15T12:57:19.846534+0000","last_fullsized":"2024-03-15T12:57:19.846534+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:46:17.129901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.777942+0000","last_change":"2024-03-15T12:56:21.207577+0000","last_active":"2024-03-15T12:57:48.777942+0000","last_peered":"2024-03-15T12:57:48.777942+0000","last_clean":"2024-03-15T12:57:48.777942+0000","last_became_active":"2024-03-15T12:56:21.207220+0000","last_became_peered":"2024-03-15T12:56:21.207220+0000","last_unstale":"2024-03-15T12:57:48.777942+0000","last_undegraded":"2024-03-15T12:57:48.777942+0000","last_fullsized":"2024-03-15T12:57:48.777942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:42:12.104220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103876+0000","last_change":"2024-03-15T12:56:25.439762+0000","last_active":"2024-03-15T12:57:20.103876+0000","last_peered":"2024-03-15T12:57:20.103876+0000","last_clean":"2024-03-15T12:57:20.103876+0000","last_became_active":"2024-03-15T12:56:25.439366+0000","last_became_peered":"2024-03-15T12:56:25.439366+0000","last_unstale":"2024-03-15T12:57:20.103876+0000","last_undegraded":"2024-03-15T12:57:20.103876+0000","last_fullsized":"2024-03-15T12:57:20.103876+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:36:29.638449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'13","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834416+0000","last_change":"2024-03-15T12:56:20.218438+0000","last_active":"2024-03-15T12:57:19.834416+0000","last_peered":"2024-03-15T12:57:19.834416+0000","last_clean":"2024-03-15T12:57:19.834416+0000","last_became_active":"2024-03-15T12:56:20.218071+0000","last_became_peered":"2024-03-15T12:56:20.218071+0000","last_unstale":"2024-03-15T12:57:19.834416+0000","last_undegraded":"2024-03-15T12:57:19.834416+0000","last_fullsized":"2024-03-15T12:57:19.834416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:24:16.295781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104537+0000","last_change":"2024-03-15T12:56:21.218345+0000","last_active":"2024-03-15T12:57:20.104537+0000","last_peered":"2024-03-15T12:57:20.104537+0000","last_clean":"2024-03-15T12:57:20.104537+0000","last_became_active":"2024-03-15T12:56:21.216534+0000","last_became_peered":"2024-03-15T12:56:21.216534+0000","last_unstale":"2024-03-15T12:57:20.104537+0000","last_undegraded":"2024-03-15T12:57:20.104537+0000","last_fullsized":"2024-03-15T12:57:20.104537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:56:18.977509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834253+0000","last_change":"2024-03-15T12:56:25.435714+0000","last_active":"2024-03-15T12:57:19.834253+0000","last_peered":"2024-03-15T12:57:19.834253+0000","last_clean":"2024-03-15T12:57:19.834253+0000","last_became_active":"2024-03-15T12:56:25.435315+0000","last_became_peered":"2024-03-15T12:56:25.435315+0000","last_unstale":"2024-03-15T12:57:19.834253+0000","last_undegraded":"2024-03-15T12:57:19.834253+0000","last_fullsized":"2024-03-15T12:57:19.834253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:10:03.509078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837180+0000","last_change":"2024-03-15T12:56:20.209356+0000","last_active":"2024-03-15T12:57:19.837180+0000","last_peered":"2024-03-15T12:57:19.837180+0000","last_clean":"2024-03-15T12:57:19.837180+0000","last_became_active":"2024-03-15T12:56:20.208995+0000","last_became_peered":"2024-03-15T12:56:20.208995+0000","last_unstale":"2024-03-15T12:57:19.837180+0000","last_undegraded":"2024-03-15T12:57:19.837180+0000","last_fullsized":"2024-03-15T12:57:19.837180+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:14:08.554567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836397+0000","last_change":"2024-03-15T12:56:21.216315+0000","last_active":"2024-03-15T12:57:19.836397+0000","last_peered":"2024-03-15T12:57:19.836397+0000","last_clean":"2024-03-15T12:57:19.836397+0000","last_became_active":"2024-03-15T12:56:21.215943+0000","last_became_peered":"2024-03-15T12:56:21.215943+0000","last_unstale":"2024-03-15T12:57:19.836397+0000","last_undegraded":"2024-03-15T12:57:19.836397+0000","last_fullsized":"2024-03-15T12:57:19.836397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:47:29.913048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836875+0000","last_change":"2024-03-15T12:56:25.437200+0000","last_active":"2024-03-15T12:57:19.836875+0000","last_peered":"2024-03-15T12:57:19.836875+0000","last_clean":"2024-03-15T12:57:19.836875+0000","last_became_active":"2024-03-15T12:56:25.436786+0000","last_became_peered":"2024-03-15T12:56:25.436786+0000","last_unstale":"2024-03-15T12:57:19.836875+0000","last_undegraded":"2024-03-15T12:57:19.836875+0000","last_fullsized":"2024-03-15T12:57:19.836875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:12:00.286749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106318+0000","last_change":"2024-03-15T12:56:20.202240+0000","last_active":"2024-03-15T12:57:20.106318+0000","last_peered":"2024-03-15T12:57:20.106318+0000","last_clean":"2024-03-15T12:57:20.106318+0000","last_became_active":"2024-03-15T12:56:20.201136+0000","last_became_peered":"2024-03-15T12:56:20.201136+0000","last_unstale":"2024-03-15T12:57:20.106318+0000","last_undegraded":"2024-03-15T12:57:20.106318+0000","last_fullsized":"2024-03-15T12:57:20.106318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:30:35.901237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.737482+0000","last_change":"2024-03-15T12:56:21.212095+0000","last_active":"2024-03-15T12:57:48.737482+0000","last_peered":"2024-03-15T12:57:48.737482+0000","last_clean":"2024-03-15T12:57:48.737482+0000","last_became_active":"2024-03-15T12:56:21.211744+0000","last_became_peered":"2024-03-15T12:56:21.211744+0000","last_unstale":"2024-03-15T12:57:48.737482+0000","last_undegraded":"2024-03-15T12:57:48.737482+0000","last_fullsized":"2024-03-15T12:57:48.737482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:27:58.412513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106031+0000","last_change":"2024-03-15T12:57:20.106031+0000","last_active":"2024-03-15T12:57:20.106031+0000","last_peered":"2024-03-15T12:57:20.106031+0000","last_clean":"2024-03-15T12:57:20.106031+0000","last_became_active":"2024-03-15T12:57:20.105499+0000","last_became_peered":"2024-03-15T12:57:20.105499+0000","last_unstale":"2024-03-15T12:57:20.106031+0000","last_undegraded":"2024-03-15T12:57:20.106031+0000","last_fullsized":"2024-03-15T12:57:20.106031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:17:08.138164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104082+0000","last_change":"2024-03-15T12:56:20.217319+0000","last_active":"2024-03-15T12:57:20.104082+0000","last_peered":"2024-03-15T12:57:20.104082+0000","last_clean":"2024-03-15T12:57:20.104082+0000","last_became_active":"2024-03-15T12:56:20.215450+0000","last_became_peered":"2024-03-15T12:56:20.215450+0000","last_unstale":"2024-03-15T12:57:20.104082+0000","last_undegraded":"2024-03-15T12:57:20.104082+0000","last_fullsized":"2024-03-15T12:57:20.104082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:07:09.085860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780127+0000","last_change":"2024-03-15T12:56:21.209360+0000","last_active":"2024-03-15T12:57:48.780127+0000","last_peered":"2024-03-15T12:57:48.780127+0000","last_clean":"2024-03-15T12:57:48.780127+0000","last_became_active":"2024-03-15T12:56:21.208987+0000","last_became_peered":"2024-03-15T12:56:21.208987+0000","last_unstale":"2024-03-15T12:57:48.780127+0000","last_undegraded":"2024-03-15T12:57:48.780127+0000","last_fullsized":"2024-03-15T12:57:48.780127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:19:13.353400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836774+0000","last_change":"2024-03-15T12:56:25.439173+0000","last_active":"2024-03-15T12:57:19.836774+0000","last_peered":"2024-03-15T12:57:19.836774+0000","last_clean":"2024-03-15T12:57:19.836774+0000","last_became_active":"2024-03-15T12:56:25.438812+0000","last_became_peered":"2024-03-15T12:56:25.438812+0000","last_unstale":"2024-03-15T12:57:19.836774+0000","last_undegraded":"2024-03-15T12:57:19.836774+0000","last_fullsized":"2024-03-15T12:57:19.836774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:10:39.107190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834324+0000","last_change":"2024-03-15T12:56:20.211861+0000","last_active":"2024-03-15T12:57:19.834324+0000","last_peered":"2024-03-15T12:57:19.834324+0000","last_clean":"2024-03-15T12:57:19.834324+0000","last_became_active":"2024-03-15T12:56:20.211481+0000","last_became_peered":"2024-03-15T12:56:20.211481+0000","last_unstale":"2024-03-15T12:57:19.834324+0000","last_undegraded":"2024-03-15T12:57:19.834324+0000","last_fullsized":"2024-03-15T12:57:19.834324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:12:45.558423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104537+0000","last_change":"2024-03-15T12:56:21.216058+0000","last_active":"2024-03-15T12:57:20.104537+0000","last_peered":"2024-03-15T12:57:20.104537+0000","last_clean":"2024-03-15T12:57:20.104537+0000","last_became_active":"2024-03-15T12:56:21.215706+0000","last_became_peered":"2024-03-15T12:56:21.215706+0000","last_unstale":"2024-03-15T12:57:20.104537+0000","last_undegraded":"2024-03-15T12:57:20.104537+0000","last_fullsized":"2024-03-15T12:57:20.104537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:55:02.132976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836966+0000","last_change":"2024-03-15T12:56:25.444134+0000","last_active":"2024-03-15T12:57:19.836966+0000","last_peered":"2024-03-15T12:57:19.836966+0000","last_clean":"2024-03-15T12:57:19.836966+0000","last_became_active":"2024-03-15T12:56:25.443452+0000","last_became_peered":"2024-03-15T12:56:25.443452+0000","last_unstale":"2024-03-15T12:57:19.836966+0000","last_undegraded":"2024-03-15T12:57:19.836966+0000","last_fullsized":"2024-03-15T12:57:19.836966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:06:39.660016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836726+0000","last_change":"2024-03-15T12:56:20.214700+0000","last_active":"2024-03-15T12:57:19.836726+0000","last_peered":"2024-03-15T12:57:19.836726+0000","last_clean":"2024-03-15T12:57:19.836726+0000","last_became_active":"2024-03-15T12:56:20.214348+0000","last_became_peered":"2024-03-15T12:56:20.214348+0000","last_unstale":"2024-03-15T12:57:19.836726+0000","last_undegraded":"2024-03-15T12:57:19.836726+0000","last_fullsized":"2024-03-15T12:57:19.836726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:24:37.551182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104551+0000","last_change":"2024-03-15T12:56:21.220746+0000","last_active":"2024-03-15T12:57:20.104551+0000","last_peered":"2024-03-15T12:57:20.104551+0000","last_clean":"2024-03-15T12:57:20.104551+0000","last_became_active":"2024-03-15T12:56:21.220445+0000","last_became_peered":"2024-03-15T12:56:21.220445+0000","last_unstale":"2024-03-15T12:57:20.104551+0000","last_undegraded":"2024-03-15T12:57:20.104551+0000","last_fullsized":"2024-03-15T12:57:20.104551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:58:44.119476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104700+0000","last_change":"2024-03-15T12:56:25.437611+0000","last_active":"2024-03-15T12:57:20.104700+0000","last_peered":"2024-03-15T12:57:20.104700+0000","last_clean":"2024-03-15T12:57:20.104700+0000","last_became_active":"2024-03-15T12:56:25.437195+0000","last_became_peered":"2024-03-15T12:56:25.437195+0000","last_unstale":"2024-03-15T12:57:20.104700+0000","last_undegraded":"2024-03-15T12:57:20.104700+0000","last_fullsized":"2024-03-15T12:57:20.104700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:26:43.558477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":101,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.535748+0000","last_change":"2024-03-15T12:56:20.208584+0000","last_active":"2024-03-15T12:57:48.535748+0000","last_peered":"2024-03-15T12:57:48.535748+0000","last_clean":"2024-03-15T12:57:48.535748+0000","last_became_active":"2024-03-15T12:56:20.208155+0000","last_became_peered":"2024-03-15T12:56:20.208155+0000","last_unstale":"2024-03-15T12:57:48.535748+0000","last_undegraded":"2024-03-15T12:57:48.535748+0000","last_fullsized":"2024-03-15T12:57:48.535748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:26:13.148273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104718+0000","last_change":"2024-03-15T12:56:21.220421+0000","last_active":"2024-03-15T12:57:20.104718+0000","last_peered":"2024-03-15T12:57:20.104718+0000","last_clean":"2024-03-15T12:57:20.104718+0000","last_became_active":"2024-03-15T12:56:21.220095+0000","last_became_peered":"2024-03-15T12:56:21.220095+0000","last_unstale":"2024-03-15T12:57:20.104718+0000","last_undegraded":"2024-03-15T12:57:20.104718+0000","last_fullsized":"2024-03-15T12:57:20.104718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:35:33.846506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836547+0000","last_change":"2024-03-15T12:56:25.440408+0000","last_active":"2024-03-15T12:57:19.836547+0000","last_peered":"2024-03-15T12:57:19.836547+0000","last_clean":"2024-03-15T12:57:19.836547+0000","last_became_active":"2024-03-15T12:56:25.439965+0000","last_became_peered":"2024-03-15T12:56:25.439965+0000","last_unstale":"2024-03-15T12:57:19.836547+0000","last_undegraded":"2024-03-15T12:57:19.836547+0000","last_fullsized":"2024-03-15T12:57:19.836547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:12:57.278202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.105004+0000","last_change":"2024-03-15T12:56:20.216320+0000","last_active":"2024-03-15T12:57:20.105004+0000","last_peered":"2024-03-15T12:57:20.105004+0000","last_clean":"2024-03-15T12:57:20.105004+0000","last_became_active":"2024-03-15T12:56:20.215915+0000","last_became_peered":"2024-03-15T12:56:20.215915+0000","last_unstale":"2024-03-15T12:57:20.105004+0000","last_undegraded":"2024-03-15T12:57:20.105004+0000","last_fullsized":"2024-03-15T12:57:20.105004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:25:45.236269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106349+0000","last_change":"2024-03-15T12:56:21.215822+0000","last_active":"2024-03-15T12:57:20.106349+0000","last_peered":"2024-03-15T12:57:20.106349+0000","last_clean":"2024-03-15T12:57:20.106349+0000","last_became_active":"2024-03-15T12:56:21.215508+0000","last_became_peered":"2024-03-15T12:56:21.215508+0000","last_unstale":"2024-03-15T12:57:20.106349+0000","last_undegraded":"2024-03-15T12:57:20.106349+0000","last_fullsized":"2024-03-15T12:57:20.106349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:56:55.919198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103800+0000","last_change":"2024-03-15T12:56:25.436186+0000","last_active":"2024-03-15T12:57:20.103800+0000","last_peered":"2024-03-15T12:57:20.103800+0000","last_clean":"2024-03-15T12:57:20.103800+0000","last_became_active":"2024-03-15T12:56:25.435586+0000","last_became_peered":"2024-03-15T12:56:25.435586+0000","last_unstale":"2024-03-15T12:57:20.103800+0000","last_undegraded":"2024-03-15T12:57:20.103800+0000","last_fullsized":"2024-03-15T12:57:20.103800+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:02:26.850487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837444+0000","last_change":"2024-03-15T12:56:20.217277+0000","last_active":"2024-03-15T12:57:19.837444+0000","last_peered":"2024-03-15T12:57:19.837444+0000","last_clean":"2024-03-15T12:57:19.837444+0000","last_became_active":"2024-03-15T12:56:20.216930+0000","last_became_peered":"2024-03-15T12:56:20.216930+0000","last_unstale":"2024-03-15T12:57:19.837444+0000","last_undegraded":"2024-03-15T12:57:19.837444+0000","last_fullsized":"2024-03-15T12:57:19.837444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:47:51.075545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104792+0000","last_change":"2024-03-15T12:56:21.212066+0000","last_active":"2024-03-15T12:57:20.104792+0000","last_peered":"2024-03-15T12:57:20.104792+0000","last_clean":"2024-03-15T12:57:20.104792+0000","last_became_active":"2024-03-15T12:56:21.211650+0000","last_became_peered":"2024-03-15T12:56:21.211650+0000","last_unstale":"2024-03-15T12:57:20.104792+0000","last_undegraded":"2024-03-15T12:57:20.104792+0000","last_fullsized":"2024-03-15T12:57:20.104792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:40:35.322210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.772568+0000","last_change":"2024-03-15T12:56:21.212794+0000","last_active":"2024-03-15T12:57:48.772568+0000","last_peered":"2024-03-15T12:57:48.772568+0000","last_clean":"2024-03-15T12:57:48.772568+0000","last_became_active":"2024-03-15T12:56:21.212470+0000","last_became_peered":"2024-03-15T12:56:21.212470+0000","last_unstale":"2024-03-15T12:57:48.772568+0000","last_undegraded":"2024-03-15T12:57:48.772568+0000","last_fullsized":"2024-03-15T12:57:48.772568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:10:04.614223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.756066+0000","last_change":"2024-03-15T12:56:21.211746+0000","last_active":"2024-03-15T12:57:48.756066+0000","last_peered":"2024-03-15T12:57:48.756066+0000","last_clean":"2024-03-15T12:57:48.756066+0000","last_became_active":"2024-03-15T12:56:21.211431+0000","last_became_peered":"2024-03-15T12:56:21.211431+0000","last_unstale":"2024-03-15T12:57:48.756066+0000","last_undegraded":"2024-03-15T12:57:48.756066+0000","last_fullsized":"2024-03-15T12:57:48.756066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:46:31.188408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.777903+0000","last_change":"2024-03-15T12:56:21.220997+0000","last_active":"2024-03-15T12:57:48.777903+0000","last_peered":"2024-03-15T12:57:48.777903+0000","last_clean":"2024-03-15T12:57:48.777903+0000","last_became_active":"2024-03-15T12:56:21.220644+0000","last_became_peered":"2024-03-15T12:56:21.220644+0000","last_unstale":"2024-03-15T12:57:48.777903+0000","last_undegraded":"2024-03-15T12:57:48.777903+0000","last_fullsized":"2024-03-15T12:57:48.777903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:47:55.852845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836675+0000","last_change":"2024-03-15T12:56:21.211634+0000","last_active":"2024-03-15T12:57:19.836675+0000","last_peered":"2024-03-15T12:57:19.836675+0000","last_clean":"2024-03-15T12:57:19.836675+0000","last_became_active":"2024-03-15T12:56:21.211303+0000","last_became_peered":"2024-03-15T12:56:21.211303+0000","last_unstale":"2024-03-15T12:57:19.836675+0000","last_undegraded":"2024-03-15T12:57:19.836675+0000","last_fullsized":"2024-03-15T12:57:19.836675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:22:32.358270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780639+0000","last_change":"2024-03-15T12:56:21.211673+0000","last_active":"2024-03-15T12:57:48.780639+0000","last_peered":"2024-03-15T12:57:48.780639+0000","last_clean":"2024-03-15T12:57:48.780639+0000","last_became_active":"2024-03-15T12:56:21.211296+0000","last_became_peered":"2024-03-15T12:56:21.211296+0000","last_unstale":"2024-03-15T12:57:48.780639+0000","last_undegraded":"2024-03-15T12:57:48.780639+0000","last_fullsized":"2024-03-15T12:57:48.780639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:40:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.769773+0000","last_change":"2024-03-15T12:56:21.208681+0000","last_active":"2024-03-15T12:57:48.769773+0000","last_peered":"2024-03-15T12:57:48.769773+0000","last_clean":"2024-03-15T12:57:48.769773+0000","last_became_active":"2024-03-15T12:56:21.208130+0000","last_became_peered":"2024-03-15T12:56:21.208130+0000","last_unstale":"2024-03-15T12:57:48.769773+0000","last_undegraded":"2024-03-15T12:57:48.769773+0000","last_fullsized":"2024-03-15T12:57:48.769773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:34:52.971485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.793487+0000","last_change":"2024-03-15T12:56:21.216979+0000","last_active":"2024-03-15T12:57:48.793487+0000","last_peered":"2024-03-15T12:57:48.793487+0000","last_clean":"2024-03-15T12:57:48.793487+0000","last_became_active":"2024-03-15T12:56:21.216632+0000","last_became_peered":"2024-03-15T12:56:21.216632+0000","last_unstale":"2024-03-15T12:57:48.793487+0000","last_undegraded":"2024-03-15T12:57:48.793487+0000","last_fullsized":"2024-03-15T12:57:48.793487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:21:57.213512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104660+0000","last_change":"2024-03-15T12:56:21.223064+0000","last_active":"2024-03-15T12:57:20.104660+0000","last_peered":"2024-03-15T12:57:20.104660+0000","last_clean":"2024-03-15T12:57:20.104660+0000","last_became_active":"2024-03-15T12:56:21.220882+0000","last_became_peered":"2024-03-15T12:56:21.220882+0000","last_unstale":"2024-03-15T12:57:20.104660+0000","last_undegraded":"2024-03-15T12:57:20.104660+0000","last_fullsized":"2024-03-15T12:57:20.104660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:45:02.930154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.772144+0000","last_change":"2024-03-15T12:56:21.218614+0000","last_active":"2024-03-15T12:57:48.772144+0000","last_peered":"2024-03-15T12:57:48.772144+0000","last_clean":"2024-03-15T12:57:48.772144+0000","last_became_active":"2024-03-15T12:56:21.218224+0000","last_became_peered":"2024-03-15T12:56:21.218224+0000","last_unstale":"2024-03-15T12:57:48.772144+0000","last_undegraded":"2024-03-15T12:57:48.772144+0000","last_fullsized":"2024-03-15T12:57:48.772144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:41:29.790358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103965+0000","last_change":"2024-03-15T12:56:21.219019+0000","last_active":"2024-03-15T12:57:20.103965+0000","last_peered":"2024-03-15T12:57:20.103965+0000","last_clean":"2024-03-15T12:57:20.103965+0000","last_became_active":"2024-03-15T12:56:21.218673+0000","last_became_peered":"2024-03-15T12:56:21.218673+0000","last_unstale":"2024-03-15T12:57:20.103965+0000","last_undegraded":"2024-03-15T12:57:20.103965+0000","last_fullsized":"2024-03-15T12:57:20.103965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:42:11.788598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.799863+0000","last_change":"2024-03-15T12:56:21.220355+0000","last_active":"2024-03-15T12:57:48.799863+0000","last_peered":"2024-03-15T12:57:48.799863+0000","last_clean":"2024-03-15T12:57:48.799863+0000","last_became_active":"2024-03-15T12:56:21.220021+0000","last_became_peered":"2024-03-15T12:56:21.220021+0000","last_unstale":"2024-03-15T12:57:48.799863+0000","last_undegraded":"2024-03-15T12:57:48.799863+0000","last_fullsized":"2024-03-15T12:57:48.799863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:40:00.947351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.781147+0000","last_change":"2024-03-15T12:56:21.213050+0000","last_active":"2024-03-15T12:57:48.781147+0000","last_peered":"2024-03-15T12:57:48.781147+0000","last_clean":"2024-03-15T12:57:48.781147+0000","last_became_active":"2024-03-15T12:56:21.212730+0000","last_became_peered":"2024-03-15T12:56:21.212730+0000","last_unstale":"2024-03-15T12:57:48.781147+0000","last_undegraded":"2024-03-15T12:57:48.781147+0000","last_fullsized":"2024-03-15T12:57:48.781147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:56:36.396285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.770617+0000","last_change":"2024-03-15T12:56:21.213942+0000","last_active":"2024-03-15T12:57:48.770617+0000","last_peered":"2024-03-15T12:57:48.770617+0000","last_clean":"2024-03-15T12:57:48.770617+0000","last_became_active":"2024-03-15T12:56:21.213611+0000","last_became_peered":"2024-03-15T12:56:21.213611+0000","last_unstale":"2024-03-15T12:57:48.770617+0000","last_undegraded":"2024-03-15T12:57:48.770617+0000","last_fullsized":"2024-03-15T12:57:48.770617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:23:44.002970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68837,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":716,"num_read_kb":436,"num_write":245,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":206652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383712,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":501056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645348,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37020,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708132,"statfs":{"total":95995035648,"available":95957127168,"internally_reserved":0,"allocated":1724416,"data_stored":1150316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808874,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":504970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070510,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":501140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1650688,"data_stored":1097749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495673,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":496378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":559432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":497834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":559652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":496086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920835,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":500843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757310,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1658880,"data_stored":1092940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-15T12:57:54.636 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-15T12:57:54.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:54 smithi149 ceph-mon[35927]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-15T12:57:54.881 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:57:55.003 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:54 smithi002 ceph-mon[32060]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-15T12:57:55.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:54 smithi184 ceph-mon[35993]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-15T12:57:55.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:55 smithi149 ceph-mon[35927]: from='client.24934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:57:56.016 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:55 smithi002 ceph-mon[32060]: from='client.24934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:57:56.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:55 smithi184 ceph-mon[35993]: from='client.24934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:57:56.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.580+0000 7fb660b77700 1 -- 172.21.15.2:0/4004412451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65c1010e0 msgr2=0x7fb65c110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:56.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.580+0000 7fb660b77700 1 --2- 172.21.15.2:0/4004412451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65c1010e0 0x7fb65c110cc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb65400b0a0 tx=0x7fb65400b3b0 comp rx=0 tx=0).stop 2024-03-15T12:57:56.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.580+0000 7fb660b77700 1 -- 172.21.15.2:0/4004412451 shutdown_connections 2024-03-15T12:57:56.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.580+0000 7fb660b77700 1 --2- 172.21.15.2:0/4004412451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65c1010e0 0x7fb65c110cc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:56.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.580+0000 7fb660b77700 1 --2- 172.21.15.2:0/4004412451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb65c102ea0 0x7fb65c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:56.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.580+0000 7fb660b77700 1 --2- 172.21.15.2:0/4004412451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb65c100530 0x7fb65c1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:56.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.580+0000 7fb660b77700 1 -- 172.21.15.2:0/4004412451 >> 172.21.15.2:0/4004412451 conn(0x7fb65c0774b0 msgr2=0x7fb65c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:56.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.580+0000 7fb660b77700 1 -- 172.21.15.2:0/4004412451 shutdown_connections 2024-03-15T12:57:56.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.580+0000 7fb660b77700 1 -- 172.21.15.2:0/4004412451 wait complete. 2024-03-15T12:57:56.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb660b77700 1 Processor -- start 2024-03-15T12:57:56.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb660b77700 1 -- start start 2024-03-15T12:57:56.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb660b77700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb65c100530 0x7fb65c1a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:56.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb660b77700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb65c1010e0 0x7fb65c1a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:56.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb660b77700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65c102ea0 0x7fb65c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:56.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb660b77700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c19ae30 con 0x7fb65c102ea0 2024-03-15T12:57:56.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb660b77700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c19afa0 con 0x7fb65c100530 2024-03-15T12:57:56.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb660b77700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c19b160 con 0x7fb65c1010e0 2024-03-15T12:57:56.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb65a59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb65c100530 0x7fb65c1a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:56.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb65a59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb65c100530 0x7fb65c1a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:52676/0 (socket says 172.21.15.2:52676) 2024-03-15T12:57:56.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb65a59c700 1 -- 172.21.15.2:0/418362306 learned_addr learned my addr 172.21.15.2:0/418362306 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:56.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb659d9b700 1 --2- 172.21.15.2:0/418362306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb65c1010e0 0x7fb65c1a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:56.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb65ad9d700 1 --2- 172.21.15.2:0/418362306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65c102ea0 0x7fb65c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:56.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb659d9b700 1 -- 172.21.15.2:0/418362306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb65c100530 msgr2=0x7fb65c1a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:56.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb659d9b700 1 --2- 172.21.15.2:0/418362306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb65c100530 0x7fb65c1a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:56.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb659d9b700 1 -- 172.21.15.2:0/418362306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65c102ea0 msgr2=0x7fb65c19a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:56.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb659d9b700 1 --2- 172.21.15.2:0/418362306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65c102ea0 0x7fb65c19a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:56.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb659d9b700 1 -- 172.21.15.2:0/418362306 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb654009d00 con 0x7fb65c1010e0 2024-03-15T12:57:56.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.581+0000 7fb65a59c700 1 --2- 172.21.15.2:0/418362306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb65c100530 0x7fb65c1a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:56.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.582+0000 7fb659d9b700 1 --2- 172.21.15.2:0/418362306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb65c1010e0 0x7fb65c1a0aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb65000e9f0 tx=0x7fb65000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:56.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.582+0000 7fb64b7fe700 1 -- 172.21.15.2:0/418362306 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65000c9c0 con 0x7fb65c1010e0 2024-03-15T12:57:56.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.582+0000 7fb64b7fe700 1 -- 172.21.15.2:0/418362306 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb650010430 con 0x7fb65c1010e0 2024-03-15T12:57:56.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.582+0000 7fb660b77700 1 -- 172.21.15.2:0/418362306 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65c19b440 con 0x7fb65c1010e0 2024-03-15T12:57:56.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.582+0000 7fb64b7fe700 1 -- 172.21.15.2:0/418362306 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650018640 con 0x7fb65c1010e0 2024-03-15T12:57:56.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.582+0000 7fb660b77700 1 -- 172.21.15.2:0/418362306 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65c1a74e0 con 0x7fb65c1010e0 2024-03-15T12:57:56.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.583+0000 7fb660b77700 1 -- 172.21.15.2:0/418362306 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65c10c420 con 0x7fb65c1010e0 2024-03-15T12:57:56.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.584+0000 7fb64b7fe700 1 -- 172.21.15.2:0/418362306 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fb6500105a0 con 0x7fb65c1010e0 2024-03-15T12:57:56.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.584+0000 7fb64b7fe700 1 --2- 172.21.15.2:0/418362306 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb6440a1b40 0x7fb6440a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:56.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.585+0000 7fb65a59c700 1 --2- 172.21.15.2:0/418362306 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb6440a1b40 0x7fb6440a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:56.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.585+0000 7fb64b7fe700 1 -- 172.21.15.2:0/418362306 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fb650014070 con 0x7fb65c1010e0 2024-03-15T12:57:56.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.585+0000 7fb65a59c700 1 --2- 172.21.15.2:0/418362306 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb6440a1b40 0x7fb6440a4000 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb64c011550 tx=0x7fb64c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:56.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.588+0000 7fb64b7fe700 1 -- 172.21.15.2:0/418362306 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb6500966c0 con 0x7fb65c1010e0 2024-03-15T12:57:56.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.790+0000 7fb660b77700 1 -- 172.21.15.2:0/418362306 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb65c19bc00 con 0x7fb6440a1b40 2024-03-15T12:57:56.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.799+0000 7fb64b7fe700 1 -- 172.21.15.2:0/418362306 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340579 (secure 0 0 0) 0x7fb65c19bc00 con 0x7fb6440a1b40 2024-03-15T12:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:57:56.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.803+0000 7fb660b77700 1 -- 172.21.15.2:0/418362306 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb6440a1b40 msgr2=0x7fb6440a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:56.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.803+0000 7fb660b77700 1 --2- 172.21.15.2:0/418362306 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb6440a1b40 0x7fb6440a4000 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb64c011550 tx=0x7fb64c009260 comp rx=0 tx=0).stop 2024-03-15T12:57:56.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.803+0000 7fb660b77700 1 -- 172.21.15.2:0/418362306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb65c1010e0 msgr2=0x7fb65c1a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:56.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.803+0000 7fb660b77700 1 --2- 172.21.15.2:0/418362306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb65c1010e0 0x7fb65c1a0aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb65000e9f0 tx=0x7fb65000ed00 comp rx=0 tx=0).stop 2024-03-15T12:57:56.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.803+0000 7fb660b77700 1 -- 172.21.15.2:0/418362306 shutdown_connections 2024-03-15T12:57:56.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.803+0000 7fb660b77700 1 --2- 172.21.15.2:0/418362306 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fb6440a1b40 0x7fb6440a4000 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:56.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.803+0000 7fb660b77700 1 --2- 172.21.15.2:0/418362306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65c102ea0 0x7fb65c19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:56.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.803+0000 7fb660b77700 1 --2- 172.21.15.2:0/418362306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb65c100530 0x7fb65c1a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:56.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.803+0000 7fb660b77700 1 --2- 172.21.15.2:0/418362306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb65c1010e0 0x7fb65c1a0aa0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:56.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.803+0000 7fb660b77700 1 -- 172.21.15.2:0/418362306 >> 172.21.15.2:0/418362306 conn(0x7fb65c0774b0 msgr2=0x7fb65c0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:56.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.803+0000 7fb660b77700 1 -- 172.21.15.2:0/418362306 shutdown_connections 2024-03-15T12:57:56.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:56.803+0000 7fb660b77700 1 -- 172.21.15.2:0/418362306 wait complete. 2024-03-15T12:57:56.795 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-03-15T12:57:56.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:56 smithi149 ceph-mon[35927]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-15T12:57:57.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:56 smithi002 ceph-mon[32060]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-15T12:57:57.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:56 smithi184 ceph-mon[35993]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-15T12:57:57.566 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-03-15T12:57:56.764912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659580,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":903,"num_read_kb":581,"num_write":429,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":853,"ondisk_log_size":853,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400820,"kb_used_data":14228,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124541004,"statfs":{"total":1151940427776,"available":1151529988096,"internally_reserved":0,"allocated":14569472,"data_stored":7958678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":72,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003302"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.787857+0000","last_change":"2024-03-15T12:56:21.210343+0000","last_active":"2024-03-15T12:57:48.787857+0000","last_peered":"2024-03-15T12:57:48.787857+0000","last_clean":"2024-03-15T12:57:48.787857+0000","last_became_active":"2024-03-15T12:56:21.209787+0000","last_became_peered":"2024-03-15T12:56:21.209787+0000","last_unstale":"2024-03-15T12:57:48.787857+0000","last_undegraded":"2024-03-15T12:57:48.787857+0000","last_fullsized":"2024-03-15T12:57:48.787857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:32:21.204944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.777621+0000","last_change":"2024-03-15T12:56:21.214153+0000","last_active":"2024-03-15T12:57:48.777621+0000","last_peered":"2024-03-15T12:57:48.777621+0000","last_clean":"2024-03-15T12:57:48.777621+0000","last_became_active":"2024-03-15T12:56:21.213819+0000","last_became_peered":"2024-03-15T12:56:21.213819+0000","last_unstale":"2024-03-15T12:57:48.777621+0000","last_undegraded":"2024-03-15T12:57:48.777621+0000","last_fullsized":"2024-03-15T12:57:48.777621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:31:43.264698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.792624+0000","last_change":"2024-03-15T12:56:21.221678+0000","last_active":"2024-03-15T12:57:48.792624+0000","last_peered":"2024-03-15T12:57:48.792624+0000","last_clean":"2024-03-15T12:57:48.792624+0000","last_became_active":"2024-03-15T12:56:21.221272+0000","last_became_peered":"2024-03-15T12:56:21.221272+0000","last_unstale":"2024-03-15T12:57:48.792624+0000","last_undegraded":"2024-03-15T12:57:48.792624+0000","last_fullsized":"2024-03-15T12:57:48.792624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:01:50.778707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.754462+0000","last_change":"2024-03-15T12:56:21.212939+0000","last_active":"2024-03-15T12:57:48.754462+0000","last_peered":"2024-03-15T12:57:48.754462+0000","last_clean":"2024-03-15T12:57:48.754462+0000","last_became_active":"2024-03-15T12:56:21.212437+0000","last_became_peered":"2024-03-15T12:56:21.212437+0000","last_unstale":"2024-03-15T12:57:48.754462+0000","last_undegraded":"2024-03-15T12:57:48.754462+0000","last_fullsized":"2024-03-15T12:57:48.754462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:34:11.377020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.771047+0000","last_change":"2024-03-15T12:56:21.221433+0000","last_active":"2024-03-15T12:57:48.771047+0000","last_peered":"2024-03-15T12:57:48.771047+0000","last_clean":"2024-03-15T12:57:48.771047+0000","last_became_active":"2024-03-15T12:56:21.220605+0000","last_became_peered":"2024-03-15T12:56:21.220605+0000","last_unstale":"2024-03-15T12:57:48.771047+0000","last_undegraded":"2024-03-15T12:57:48.771047+0000","last_fullsized":"2024-03-15T12:57:48.771047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:20:25.169387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784751+0000","last_change":"2024-03-15T12:56:21.216539+0000","last_active":"2024-03-15T12:57:48.784751+0000","last_peered":"2024-03-15T12:57:48.784751+0000","last_clean":"2024-03-15T12:57:48.784751+0000","last_became_active":"2024-03-15T12:56:21.216197+0000","last_became_peered":"2024-03-15T12:56:21.216197+0000","last_unstale":"2024-03-15T12:57:48.784751+0000","last_undegraded":"2024-03-15T12:57:48.784751+0000","last_fullsized":"2024-03-15T12:57:48.784751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:00:23.186467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.779848+0000","last_change":"2024-03-15T12:56:21.213103+0000","last_active":"2024-03-15T12:57:48.779848+0000","last_peered":"2024-03-15T12:57:48.779848+0000","last_clean":"2024-03-15T12:57:48.779848+0000","last_became_active":"2024-03-15T12:56:21.212789+0000","last_became_peered":"2024-03-15T12:56:21.212789+0000","last_unstale":"2024-03-15T12:57:48.779848+0000","last_undegraded":"2024-03-15T12:57:48.779848+0000","last_fullsized":"2024-03-15T12:57:48.779848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:41:50.140915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.785673+0000","last_change":"2024-03-15T12:56:21.214767+0000","last_active":"2024-03-15T12:57:48.785673+0000","last_peered":"2024-03-15T12:57:48.785673+0000","last_clean":"2024-03-15T12:57:48.785673+0000","last_became_active":"2024-03-15T12:56:21.213896+0000","last_became_peered":"2024-03-15T12:56:21.213896+0000","last_unstale":"2024-03-15T12:57:48.785673+0000","last_undegraded":"2024-03-15T12:57:48.785673+0000","last_fullsized":"2024-03-15T12:57:48.785673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:33:53.819906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.796148+0000","last_change":"2024-03-15T12:56:21.218566+0000","last_active":"2024-03-15T12:57:48.796148+0000","last_peered":"2024-03-15T12:57:48.796148+0000","last_clean":"2024-03-15T12:57:48.796148+0000","last_became_active":"2024-03-15T12:56:21.218234+0000","last_became_peered":"2024-03-15T12:56:21.218234+0000","last_unstale":"2024-03-15T12:57:48.796148+0000","last_undegraded":"2024-03-15T12:57:48.796148+0000","last_fullsized":"2024-03-15T12:57:48.796148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:56:46.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784768+0000","last_change":"2024-03-15T12:56:21.219367+0000","last_active":"2024-03-15T12:57:48.784768+0000","last_peered":"2024-03-15T12:57:48.784768+0000","last_clean":"2024-03-15T12:57:48.784768+0000","last_became_active":"2024-03-15T12:56:21.217541+0000","last_became_peered":"2024-03-15T12:56:21.217541+0000","last_unstale":"2024-03-15T12:57:48.784768+0000","last_undegraded":"2024-03-15T12:57:48.784768+0000","last_fullsized":"2024-03-15T12:57:48.784768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:19:12.939394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798017+0000","last_change":"2024-03-15T12:56:21.222246+0000","last_active":"2024-03-15T12:57:48.798017+0000","last_peered":"2024-03-15T12:57:48.798017+0000","last_clean":"2024-03-15T12:57:48.798017+0000","last_became_active":"2024-03-15T12:56:21.221883+0000","last_became_peered":"2024-03-15T12:56:21.221883+0000","last_unstale":"2024-03-15T12:57:48.798017+0000","last_undegraded":"2024-03-15T12:57:48.798017+0000","last_fullsized":"2024-03-15T12:57:48.798017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:42:06.892581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.771534+0000","last_change":"2024-03-15T12:57:19.841386+0000","last_active":"2024-03-15T12:57:48.771534+0000","last_peered":"2024-03-15T12:57:48.771534+0000","last_clean":"2024-03-15T12:57:48.771534+0000","last_became_active":"2024-03-15T12:57:19.840981+0000","last_became_peered":"2024-03-15T12:57:19.840981+0000","last_unstale":"2024-03-15T12:57:48.771534+0000","last_undegraded":"2024-03-15T12:57:48.771534+0000","last_fullsized":"2024-03-15T12:57:48.771534+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:57:28.443452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.776665+0000","last_change":"2024-03-15T12:56:21.199631+0000","last_active":"2024-03-15T12:57:48.776665+0000","last_peered":"2024-03-15T12:57:48.776665+0000","last_clean":"2024-03-15T12:57:48.776665+0000","last_became_active":"2024-03-15T12:56:21.199230+0000","last_became_peered":"2024-03-15T12:56:21.199230+0000","last_unstale":"2024-03-15T12:57:48.776665+0000","last_undegraded":"2024-03-15T12:57:48.776665+0000","last_fullsized":"2024-03-15T12:57:48.776665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:45:51.356803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.775081+0000","last_change":"2024-03-15T12:56:21.211449+0000","last_active":"2024-03-15T12:57:48.775081+0000","last_peered":"2024-03-15T12:57:48.775081+0000","last_clean":"2024-03-15T12:57:48.775081+0000","last_became_active":"2024-03-15T12:56:21.211001+0000","last_became_peered":"2024-03-15T12:56:21.211001+0000","last_unstale":"2024-03-15T12:57:48.775081+0000","last_undegraded":"2024-03-15T12:57:48.775081+0000","last_fullsized":"2024-03-15T12:57:48.775081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:32:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.779936+0000","last_change":"2024-03-15T12:56:21.212001+0000","last_active":"2024-03-15T12:57:48.779936+0000","last_peered":"2024-03-15T12:57:48.779936+0000","last_clean":"2024-03-15T12:57:48.779936+0000","last_became_active":"2024-03-15T12:56:21.211497+0000","last_became_peered":"2024-03-15T12:56:21.211497+0000","last_unstale":"2024-03-15T12:57:48.779936+0000","last_undegraded":"2024-03-15T12:57:48.779936+0000","last_fullsized":"2024-03-15T12:57:48.779936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:27:16.029202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.799865+0000","last_change":"2024-03-15T12:56:21.222368+0000","last_active":"2024-03-15T12:57:48.799865+0000","last_peered":"2024-03-15T12:57:48.799865+0000","last_clean":"2024-03-15T12:57:48.799865+0000","last_became_active":"2024-03-15T12:56:21.221356+0000","last_became_peered":"2024-03-15T12:56:21.221356+0000","last_unstale":"2024-03-15T12:57:48.799865+0000","last_undegraded":"2024-03-15T12:57:48.799865+0000","last_fullsized":"2024-03-15T12:57:48.799865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:15:04.125220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.789152+0000","last_change":"2024-03-15T12:56:21.211323+0000","last_active":"2024-03-15T12:57:48.789152+0000","last_peered":"2024-03-15T12:57:48.789152+0000","last_clean":"2024-03-15T12:57:48.789152+0000","last_became_active":"2024-03-15T12:56:21.210983+0000","last_became_peered":"2024-03-15T12:56:21.210983+0000","last_unstale":"2024-03-15T12:57:48.789152+0000","last_undegraded":"2024-03-15T12:57:48.789152+0000","last_fullsized":"2024-03-15T12:57:48.789152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:10:51.487985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.775434+0000","last_change":"2024-03-15T12:56:21.209435+0000","last_active":"2024-03-15T12:57:48.775434+0000","last_peered":"2024-03-15T12:57:48.775434+0000","last_clean":"2024-03-15T12:57:48.775434+0000","last_became_active":"2024-03-15T12:56:21.208981+0000","last_became_peered":"2024-03-15T12:56:21.208981+0000","last_unstale":"2024-03-15T12:57:48.775434+0000","last_undegraded":"2024-03-15T12:57:48.775434+0000","last_fullsized":"2024-03-15T12:57:48.775434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:47:59.142040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.768813+0000","last_change":"2024-03-15T12:56:21.222014+0000","last_active":"2024-03-15T12:57:48.768813+0000","last_peered":"2024-03-15T12:57:48.768813+0000","last_clean":"2024-03-15T12:57:48.768813+0000","last_became_active":"2024-03-15T12:56:21.221548+0000","last_became_peered":"2024-03-15T12:56:21.221548+0000","last_unstale":"2024-03-15T12:57:48.768813+0000","last_undegraded":"2024-03-15T12:57:48.768813+0000","last_fullsized":"2024-03-15T12:57:48.768813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:12:48.943874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834595+0000","last_change":"2024-03-15T12:56:25.447200+0000","last_active":"2024-03-15T12:57:19.834595+0000","last_peered":"2024-03-15T12:57:19.834595+0000","last_clean":"2024-03-15T12:57:19.834595+0000","last_became_active":"2024-03-15T12:56:25.446516+0000","last_became_peered":"2024-03-15T12:56:25.446516+0000","last_unstale":"2024-03-15T12:57:19.834595+0000","last_undegraded":"2024-03-15T12:57:19.834595+0000","last_fullsized":"2024-03-15T12:57:19.834595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:06:46.221772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":113,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:53.524415+0000","last_change":"2024-03-15T12:56:20.210826+0000","last_active":"2024-03-15T12:57:53.524415+0000","last_peered":"2024-03-15T12:57:53.524415+0000","last_clean":"2024-03-15T12:57:53.524415+0000","last_became_active":"2024-03-15T12:56:20.210472+0000","last_became_peered":"2024-03-15T12:56:20.210472+0000","last_unstale":"2024-03-15T12:57:53.524415+0000","last_undegraded":"2024-03-15T12:57:53.524415+0000","last_fullsized":"2024-03-15T12:57:53.524415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:33:22.896058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.772480+0000","last_change":"2024-03-15T12:56:21.224665+0000","last_active":"2024-03-15T12:57:48.772480+0000","last_peered":"2024-03-15T12:57:48.772480+0000","last_clean":"2024-03-15T12:57:48.772480+0000","last_became_active":"2024-03-15T12:56:21.223826+0000","last_became_peered":"2024-03-15T12:56:21.223826+0000","last_unstale":"2024-03-15T12:57:48.772480+0000","last_undegraded":"2024-03-15T12:57:48.772480+0000","last_fullsized":"2024-03-15T12:57:48.772480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:11:57.576701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837559+0000","last_change":"2024-03-15T12:56:25.444665+0000","last_active":"2024-03-15T12:57:19.837559+0000","last_peered":"2024-03-15T12:57:19.837559+0000","last_clean":"2024-03-15T12:57:19.837559+0000","last_became_active":"2024-03-15T12:56:25.444275+0000","last_became_peered":"2024-03-15T12:56:25.444275+0000","last_unstale":"2024-03-15T12:57:19.837559+0000","last_undegraded":"2024-03-15T12:57:19.837559+0000","last_fullsized":"2024-03-15T12:57:19.837559+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:29:46.470405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104578+0000","last_change":"2024-03-15T12:56:20.215873+0000","last_active":"2024-03-15T12:57:20.104578+0000","last_peered":"2024-03-15T12:57:20.104578+0000","last_clean":"2024-03-15T12:57:20.104578+0000","last_became_active":"2024-03-15T12:56:20.215505+0000","last_became_peered":"2024-03-15T12:56:20.215505+0000","last_unstale":"2024-03-15T12:57:20.104578+0000","last_undegraded":"2024-03-15T12:57:20.104578+0000","last_fullsized":"2024-03-15T12:57:20.104578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:01:57.317382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.792666+0000","last_change":"2024-03-15T12:56:21.218200+0000","last_active":"2024-03-15T12:57:48.792666+0000","last_peered":"2024-03-15T12:57:48.792666+0000","last_clean":"2024-03-15T12:57:48.792666+0000","last_became_active":"2024-03-15T12:56:21.217768+0000","last_became_peered":"2024-03-15T12:56:21.217768+0000","last_unstale":"2024-03-15T12:57:48.792666+0000","last_undegraded":"2024-03-15T12:57:48.792666+0000","last_fullsized":"2024-03-15T12:57:48.792666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:39:42.443542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837005+0000","last_change":"2024-03-15T12:56:25.444220+0000","last_active":"2024-03-15T12:57:19.837005+0000","last_peered":"2024-03-15T12:57:19.837005+0000","last_clean":"2024-03-15T12:57:19.837005+0000","last_became_active":"2024-03-15T12:56:25.443361+0000","last_became_peered":"2024-03-15T12:56:25.443361+0000","last_unstale":"2024-03-15T12:57:19.837005+0000","last_undegraded":"2024-03-15T12:57:19.837005+0000","last_fullsized":"2024-03-15T12:57:19.837005+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:21:11.593526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837757+0000","last_change":"2024-03-15T12:56:20.218241+0000","last_active":"2024-03-15T12:57:19.837757+0000","last_peered":"2024-03-15T12:57:19.837757+0000","last_clean":"2024-03-15T12:57:19.837757+0000","last_became_active":"2024-03-15T12:56:20.217339+0000","last_became_peered":"2024-03-15T12:56:20.217339+0000","last_unstale":"2024-03-15T12:57:19.837757+0000","last_undegraded":"2024-03-15T12:57:19.837757+0000","last_fullsized":"2024-03-15T12:57:19.837757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:01:22.156502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798260+0000","last_change":"2024-03-15T12:56:21.217224+0000","last_active":"2024-03-15T12:57:48.798260+0000","last_peered":"2024-03-15T12:57:48.798260+0000","last_clean":"2024-03-15T12:57:48.798260+0000","last_became_active":"2024-03-15T12:56:21.216868+0000","last_became_peered":"2024-03-15T12:56:21.216868+0000","last_unstale":"2024-03-15T12:57:48.798260+0000","last_undegraded":"2024-03-15T12:57:48.798260+0000","last_fullsized":"2024-03-15T12:57:48.798260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:03:32.668463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836741+0000","last_change":"2024-03-15T12:56:25.443861+0000","last_active":"2024-03-15T12:57:19.836741+0000","last_peered":"2024-03-15T12:57:19.836741+0000","last_clean":"2024-03-15T12:57:19.836741+0000","last_became_active":"2024-03-15T12:56:25.443386+0000","last_became_peered":"2024-03-15T12:56:25.443386+0000","last_unstale":"2024-03-15T12:57:19.836741+0000","last_undegraded":"2024-03-15T12:57:19.836741+0000","last_fullsized":"2024-03-15T12:57:19.836741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:00:23.063425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837030+0000","last_change":"2024-03-15T12:56:20.212451+0000","last_active":"2024-03-15T12:57:19.837030+0000","last_peered":"2024-03-15T12:57:19.837030+0000","last_clean":"2024-03-15T12:57:19.837030+0000","last_became_active":"2024-03-15T12:56:20.212118+0000","last_became_peered":"2024-03-15T12:56:20.212118+0000","last_unstale":"2024-03-15T12:57:19.837030+0000","last_undegraded":"2024-03-15T12:57:19.837030+0000","last_fullsized":"2024-03-15T12:57:19.837030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:24:33.667652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.766138+0000","last_change":"2024-03-15T12:57:20.107434+0000","last_active":"2024-03-15T12:57:48.766138+0000","last_peered":"2024-03-15T12:57:48.766138+0000","last_clean":"2024-03-15T12:57:48.766138+0000","last_became_active":"2024-03-15T12:57:20.107075+0000","last_became_peered":"2024-03-15T12:57:20.107075+0000","last_unstale":"2024-03-15T12:57:48.766138+0000","last_undegraded":"2024-03-15T12:57:48.766138+0000","last_fullsized":"2024-03-15T12:57:48.766138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:24:44.944629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834590+0000","last_change":"2024-03-15T12:56:25.438194+0000","last_active":"2024-03-15T12:57:19.834590+0000","last_peered":"2024-03-15T12:57:19.834590+0000","last_clean":"2024-03-15T12:57:19.834590+0000","last_became_active":"2024-03-15T12:56:25.437769+0000","last_became_peered":"2024-03-15T12:56:25.437769+0000","last_unstale":"2024-03-15T12:57:19.834590+0000","last_undegraded":"2024-03-15T12:57:19.834590+0000","last_fullsized":"2024-03-15T12:57:19.834590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:29:44.398731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":108,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:52.480828+0000","last_change":"2024-03-15T12:56:20.210552+0000","last_active":"2024-03-15T12:57:52.480828+0000","last_peered":"2024-03-15T12:57:52.480828+0000","last_clean":"2024-03-15T12:57:52.480828+0000","last_became_active":"2024-03-15T12:56:20.210191+0000","last_became_peered":"2024-03-15T12:56:20.210191+0000","last_unstale":"2024-03-15T12:57:52.480828+0000","last_undegraded":"2024-03-15T12:57:52.480828+0000","last_fullsized":"2024-03-15T12:57:52.480828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:25:48.225266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.754896+0000","last_change":"2024-03-15T12:56:21.220582+0000","last_active":"2024-03-15T12:57:48.754896+0000","last_peered":"2024-03-15T12:57:48.754896+0000","last_clean":"2024-03-15T12:57:48.754896+0000","last_became_active":"2024-03-15T12:56:21.220212+0000","last_became_peered":"2024-03-15T12:56:21.220212+0000","last_unstale":"2024-03-15T12:57:48.754896+0000","last_undegraded":"2024-03-15T12:57:48.754896+0000","last_fullsized":"2024-03-15T12:57:48.754896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:00:54.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834490+0000","last_change":"2024-03-15T12:56:25.446410+0000","last_active":"2024-03-15T12:57:19.834490+0000","last_peered":"2024-03-15T12:57:19.834490+0000","last_clean":"2024-03-15T12:57:19.834490+0000","last_became_active":"2024-03-15T12:56:25.445554+0000","last_became_peered":"2024-03-15T12:56:25.445554+0000","last_unstale":"2024-03-15T12:57:19.834490+0000","last_undegraded":"2024-03-15T12:57:19.834490+0000","last_fullsized":"2024-03-15T12:57:19.834490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:35:08.049374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.839292+0000","last_change":"2024-03-15T12:56:20.212599+0000","last_active":"2024-03-15T12:57:19.839292+0000","last_peered":"2024-03-15T12:57:19.839292+0000","last_clean":"2024-03-15T12:57:19.839292+0000","last_became_active":"2024-03-15T12:56:20.212289+0000","last_became_peered":"2024-03-15T12:56:20.212289+0000","last_unstale":"2024-03-15T12:57:19.839292+0000","last_undegraded":"2024-03-15T12:57:19.839292+0000","last_fullsized":"2024-03-15T12:57:19.839292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:12:51.036882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.764372+0000","last_change":"2024-03-15T12:56:21.219407+0000","last_active":"2024-03-15T12:57:48.764372+0000","last_peered":"2024-03-15T12:57:48.764372+0000","last_clean":"2024-03-15T12:57:48.764372+0000","last_became_active":"2024-03-15T12:56:21.219035+0000","last_became_peered":"2024-03-15T12:56:21.219035+0000","last_unstale":"2024-03-15T12:57:48.764372+0000","last_undegraded":"2024-03-15T12:57:48.764372+0000","last_fullsized":"2024-03-15T12:57:48.764372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:35:43.129767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.841693+0000","last_change":"2024-03-15T12:57:19.841693+0000","last_active":"2024-03-15T12:57:19.841693+0000","last_peered":"2024-03-15T12:57:19.841693+0000","last_clean":"2024-03-15T12:57:19.841693+0000","last_became_active":"2024-03-15T12:57:19.841275+0000","last_became_peered":"2024-03-15T12:57:19.841275+0000","last_unstale":"2024-03-15T12:57:19.841693+0000","last_undegraded":"2024-03-15T12:57:19.841693+0000","last_fullsized":"2024-03-15T12:57:19.841693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:41:18.147235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837559+0000","last_change":"2024-03-15T12:56:20.209933+0000","last_active":"2024-03-15T12:57:19.837559+0000","last_peered":"2024-03-15T12:57:19.837559+0000","last_clean":"2024-03-15T12:57:19.837559+0000","last_became_active":"2024-03-15T12:56:20.209047+0000","last_became_peered":"2024-03-15T12:56:20.209047+0000","last_unstale":"2024-03-15T12:57:19.837559+0000","last_undegraded":"2024-03-15T12:57:19.837559+0000","last_fullsized":"2024-03-15T12:57:19.837559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:24:05.551745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782760+0000","last_change":"2024-03-15T12:56:21.215198+0000","last_active":"2024-03-15T12:57:48.782760+0000","last_peered":"2024-03-15T12:57:48.782760+0000","last_clean":"2024-03-15T12:57:48.782760+0000","last_became_active":"2024-03-15T12:56:21.214844+0000","last_became_peered":"2024-03-15T12:56:21.214844+0000","last_unstale":"2024-03-15T12:57:48.782760+0000","last_undegraded":"2024-03-15T12:57:48.782760+0000","last_fullsized":"2024-03-15T12:57:48.782760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:37:08.398317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836360+0000","last_change":"2024-03-15T12:56:25.437047+0000","last_active":"2024-03-15T12:57:19.836360+0000","last_peered":"2024-03-15T12:57:19.836360+0000","last_clean":"2024-03-15T12:57:19.836360+0000","last_became_active":"2024-03-15T12:56:25.436595+0000","last_became_peered":"2024-03-15T12:56:25.436595+0000","last_unstale":"2024-03-15T12:57:19.836360+0000","last_undegraded":"2024-03-15T12:57:19.836360+0000","last_fullsized":"2024-03-15T12:57:19.836360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:59:55.147044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103923+0000","last_change":"2024-03-15T12:56:20.206348+0000","last_active":"2024-03-15T12:57:20.103923+0000","last_peered":"2024-03-15T12:57:20.103923+0000","last_clean":"2024-03-15T12:57:20.103923+0000","last_became_active":"2024-03-15T12:56:20.206013+0000","last_became_peered":"2024-03-15T12:56:20.206013+0000","last_unstale":"2024-03-15T12:57:20.103923+0000","last_undegraded":"2024-03-15T12:57:20.103923+0000","last_fullsized":"2024-03-15T12:57:20.103923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:32:26.625982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782151+0000","last_change":"2024-03-15T12:56:21.216517+0000","last_active":"2024-03-15T12:57:48.782151+0000","last_peered":"2024-03-15T12:57:48.782151+0000","last_clean":"2024-03-15T12:57:48.782151+0000","last_became_active":"2024-03-15T12:56:21.216076+0000","last_became_peered":"2024-03-15T12:56:21.216076+0000","last_unstale":"2024-03-15T12:57:48.782151+0000","last_undegraded":"2024-03-15T12:57:48.782151+0000","last_fullsized":"2024-03-15T12:57:48.782151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:19:35.547273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104273+0000","last_change":"2024-03-15T12:56:25.437367+0000","last_active":"2024-03-15T12:57:20.104273+0000","last_peered":"2024-03-15T12:57:20.104273+0000","last_clean":"2024-03-15T12:57:20.104273+0000","last_became_active":"2024-03-15T12:56:25.436930+0000","last_became_peered":"2024-03-15T12:56:25.436930+0000","last_unstale":"2024-03-15T12:57:20.104273+0000","last_undegraded":"2024-03-15T12:57:20.104273+0000","last_fullsized":"2024-03-15T12:57:20.104273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:53:56.586786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":178,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:54.482340+0000","last_change":"2024-03-15T12:56:20.214067+0000","last_active":"2024-03-15T12:57:54.482340+0000","last_peered":"2024-03-15T12:57:54.482340+0000","last_clean":"2024-03-15T12:57:54.482340+0000","last_became_active":"2024-03-15T12:56:20.213696+0000","last_became_peered":"2024-03-15T12:56:20.213696+0000","last_unstale":"2024-03-15T12:57:54.482340+0000","last_undegraded":"2024-03-15T12:57:54.482340+0000","last_fullsized":"2024-03-15T12:57:54.482340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:35:15.343660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.797642+0000","last_change":"2024-03-15T12:56:21.211106+0000","last_active":"2024-03-15T12:57:48.797642+0000","last_peered":"2024-03-15T12:57:48.797642+0000","last_clean":"2024-03-15T12:57:48.797642+0000","last_became_active":"2024-03-15T12:56:21.210756+0000","last_became_peered":"2024-03-15T12:56:21.210756+0000","last_unstale":"2024-03-15T12:57:48.797642+0000","last_undegraded":"2024-03-15T12:57:48.797642+0000","last_fullsized":"2024-03-15T12:57:48.797642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:13:25.232077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.833697+0000","last_change":"2024-03-15T12:56:25.441364+0000","last_active":"2024-03-15T12:57:19.833697+0000","last_peered":"2024-03-15T12:57:19.833697+0000","last_clean":"2024-03-15T12:57:19.833697+0000","last_became_active":"2024-03-15T12:56:25.440951+0000","last_became_peered":"2024-03-15T12:56:25.440951+0000","last_unstale":"2024-03-15T12:57:19.833697+0000","last_undegraded":"2024-03-15T12:57:19.833697+0000","last_fullsized":"2024-03-15T12:57:19.833697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:32:28.651856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104583+0000","last_change":"2024-03-15T12:56:20.208633+0000","last_active":"2024-03-15T12:57:20.104583+0000","last_peered":"2024-03-15T12:57:20.104583+0000","last_clean":"2024-03-15T12:57:20.104583+0000","last_became_active":"2024-03-15T12:56:20.208280+0000","last_became_peered":"2024-03-15T12:56:20.208280+0000","last_unstale":"2024-03-15T12:57:20.104583+0000","last_undegraded":"2024-03-15T12:57:20.104583+0000","last_fullsized":"2024-03-15T12:57:20.104583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:00:15.274090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.785458+0000","last_change":"2024-03-15T12:56:21.218998+0000","last_active":"2024-03-15T12:57:48.785458+0000","last_peered":"2024-03-15T12:57:48.785458+0000","last_clean":"2024-03-15T12:57:48.785458+0000","last_became_active":"2024-03-15T12:56:21.218617+0000","last_became_peered":"2024-03-15T12:56:21.218617+0000","last_unstale":"2024-03-15T12:57:48.785458+0000","last_undegraded":"2024-03-15T12:57:48.785458+0000","last_fullsized":"2024-03-15T12:57:48.785458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:20:25.496204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837212+0000","last_change":"2024-03-15T12:56:25.438969+0000","last_active":"2024-03-15T12:57:19.837212+0000","last_peered":"2024-03-15T12:57:19.837212+0000","last_clean":"2024-03-15T12:57:19.837212+0000","last_became_active":"2024-03-15T12:56:25.438603+0000","last_became_peered":"2024-03-15T12:56:25.438603+0000","last_unstale":"2024-03-15T12:57:19.837212+0000","last_undegraded":"2024-03-15T12:57:19.837212+0000","last_fullsized":"2024-03-15T12:57:19.837212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:54:23.992253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104585+0000","last_change":"2024-03-15T12:56:20.215152+0000","last_active":"2024-03-15T12:57:20.104585+0000","last_peered":"2024-03-15T12:57:20.104585+0000","last_clean":"2024-03-15T12:57:20.104585+0000","last_became_active":"2024-03-15T12:56:20.214762+0000","last_became_peered":"2024-03-15T12:56:20.214762+0000","last_unstale":"2024-03-15T12:57:20.104585+0000","last_undegraded":"2024-03-15T12:57:20.104585+0000","last_fullsized":"2024-03-15T12:57:20.104585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:38:45.414984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.790090+0000","last_change":"2024-03-15T12:56:21.214005+0000","last_active":"2024-03-15T12:57:48.790090+0000","last_peered":"2024-03-15T12:57:48.790090+0000","last_clean":"2024-03-15T12:57:48.790090+0000","last_became_active":"2024-03-15T12:56:21.213672+0000","last_became_peered":"2024-03-15T12:56:21.213672+0000","last_unstale":"2024-03-15T12:57:48.790090+0000","last_undegraded":"2024-03-15T12:57:48.790090+0000","last_fullsized":"2024-03-15T12:57:48.790090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:13:09.384245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.833759+0000","last_change":"2024-03-15T12:56:25.443145+0000","last_active":"2024-03-15T12:57:19.833759+0000","last_peered":"2024-03-15T12:57:19.833759+0000","last_clean":"2024-03-15T12:57:19.833759+0000","last_became_active":"2024-03-15T12:56:25.442755+0000","last_became_peered":"2024-03-15T12:56:25.442755+0000","last_unstale":"2024-03-15T12:57:19.833759+0000","last_undegraded":"2024-03-15T12:57:19.833759+0000","last_fullsized":"2024-03-15T12:57:19.833759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:14:47.201919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":221,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:54.487190+0000","last_change":"2024-03-15T12:56:20.215063+0000","last_active":"2024-03-15T12:57:54.487190+0000","last_peered":"2024-03-15T12:57:54.487190+0000","last_clean":"2024-03-15T12:57:54.487190+0000","last_became_active":"2024-03-15T12:56:20.214742+0000","last_became_peered":"2024-03-15T12:56:20.214742+0000","last_unstale":"2024-03-15T12:57:54.487190+0000","last_undegraded":"2024-03-15T12:57:54.487190+0000","last_fullsized":"2024-03-15T12:57:54.487190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:38:12.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62024,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":158,"num_read_kb":140,"num_write":38,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.773641+0000","last_change":"2024-03-15T12:56:21.213519+0000","last_active":"2024-03-15T12:57:48.773641+0000","last_peered":"2024-03-15T12:57:48.773641+0000","last_clean":"2024-03-15T12:57:48.773641+0000","last_became_active":"2024-03-15T12:56:21.213198+0000","last_became_peered":"2024-03-15T12:56:21.213198+0000","last_unstale":"2024-03-15T12:57:48.773641+0000","last_undegraded":"2024-03-15T12:57:48.773641+0000","last_fullsized":"2024-03-15T12:57:48.773641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:22:13.037068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106198+0000","last_change":"2024-03-15T12:57:20.106198+0000","last_active":"2024-03-15T12:57:20.106198+0000","last_peered":"2024-03-15T12:57:20.106198+0000","last_clean":"2024-03-15T12:57:20.106198+0000","last_became_active":"2024-03-15T12:57:20.105649+0000","last_became_peered":"2024-03-15T12:57:20.105649+0000","last_unstale":"2024-03-15T12:57:20.106198+0000","last_undegraded":"2024-03-15T12:57:20.106198+0000","last_fullsized":"2024-03-15T12:57:20.106198+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:44:46.615091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834552+0000","last_change":"2024-03-15T12:56:20.216124+0000","last_active":"2024-03-15T12:57:19.834552+0000","last_peered":"2024-03-15T12:57:19.834552+0000","last_clean":"2024-03-15T12:57:19.834552+0000","last_became_active":"2024-03-15T12:56:20.215754+0000","last_became_peered":"2024-03-15T12:56:20.215754+0000","last_unstale":"2024-03-15T12:57:19.834552+0000","last_undegraded":"2024-03-15T12:57:19.834552+0000","last_fullsized":"2024-03-15T12:57:19.834552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:11:44.403503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.781601+0000","last_change":"2024-03-15T12:56:21.214775+0000","last_active":"2024-03-15T12:57:48.781601+0000","last_peered":"2024-03-15T12:57:48.781601+0000","last_clean":"2024-03-15T12:57:48.781601+0000","last_became_active":"2024-03-15T12:56:21.214445+0000","last_became_peered":"2024-03-15T12:56:21.214445+0000","last_unstale":"2024-03-15T12:57:48.781601+0000","last_undegraded":"2024-03-15T12:57:48.781601+0000","last_fullsized":"2024-03-15T12:57:48.781601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:23:24.045670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.107668+0000","last_change":"2024-03-15T12:57:20.107668+0000","last_active":"2024-03-15T12:57:20.107668+0000","last_peered":"2024-03-15T12:57:20.107668+0000","last_clean":"2024-03-15T12:57:20.107668+0000","last_became_active":"2024-03-15T12:57:20.107259+0000","last_became_peered":"2024-03-15T12:57:20.107259+0000","last_unstale":"2024-03-15T12:57:20.107668+0000","last_undegraded":"2024-03-15T12:57:20.107668+0000","last_fullsized":"2024-03-15T12:57:20.107668+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:07:37.244869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104267+0000","last_change":"2024-03-15T12:56:20.217191+0000","last_active":"2024-03-15T12:57:20.104267+0000","last_peered":"2024-03-15T12:57:20.104267+0000","last_clean":"2024-03-15T12:57:20.104267+0000","last_became_active":"2024-03-15T12:56:20.216632+0000","last_became_peered":"2024-03-15T12:56:20.216632+0000","last_unstale":"2024-03-15T12:57:20.104267+0000","last_undegraded":"2024-03-15T12:57:20.104267+0000","last_fullsized":"2024-03-15T12:57:20.104267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:16:51.187920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784344+0000","last_change":"2024-03-15T12:56:21.218650+0000","last_active":"2024-03-15T12:57:48.784344+0000","last_peered":"2024-03-15T12:57:48.784344+0000","last_clean":"2024-03-15T12:57:48.784344+0000","last_became_active":"2024-03-15T12:56:21.218329+0000","last_became_peered":"2024-03-15T12:56:21.218329+0000","last_unstale":"2024-03-15T12:57:48.784344+0000","last_undegraded":"2024-03-15T12:57:48.784344+0000","last_fullsized":"2024-03-15T12:57:48.784344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:10:45.632248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834415+0000","last_change":"2024-03-15T12:56:25.447982+0000","last_active":"2024-03-15T12:57:19.834415+0000","last_peered":"2024-03-15T12:57:19.834415+0000","last_clean":"2024-03-15T12:57:19.834415+0000","last_became_active":"2024-03-15T12:56:25.447514+0000","last_became_peered":"2024-03-15T12:56:25.447514+0000","last_unstale":"2024-03-15T12:57:19.834415+0000","last_undegraded":"2024-03-15T12:57:19.834415+0000","last_fullsized":"2024-03-15T12:57:19.834415+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:16:52.567619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":108,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:52.524593+0000","last_change":"2024-03-15T12:56:20.209778+0000","last_active":"2024-03-15T12:57:52.524593+0000","last_peered":"2024-03-15T12:57:52.524593+0000","last_clean":"2024-03-15T12:57:52.524593+0000","last_became_active":"2024-03-15T12:56:20.209311+0000","last_became_peered":"2024-03-15T12:56:20.209311+0000","last_unstale":"2024-03-15T12:57:52.524593+0000","last_undegraded":"2024-03-15T12:57:52.524593+0000","last_fullsized":"2024-03-15T12:57:52.524593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:05:21.930210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.778891+0000","last_change":"2024-03-15T12:56:21.215576+0000","last_active":"2024-03-15T12:57:48.778891+0000","last_peered":"2024-03-15T12:57:48.778891+0000","last_clean":"2024-03-15T12:57:48.778891+0000","last_became_active":"2024-03-15T12:56:21.215261+0000","last_became_peered":"2024-03-15T12:56:21.215261+0000","last_unstale":"2024-03-15T12:57:48.778891+0000","last_undegraded":"2024-03-15T12:57:48.778891+0000","last_fullsized":"2024-03-15T12:57:48.778891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:35:15.461554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837297+0000","last_change":"2024-03-15T12:56:25.445559+0000","last_active":"2024-03-15T12:57:19.837297+0000","last_peered":"2024-03-15T12:57:19.837297+0000","last_clean":"2024-03-15T12:57:19.837297+0000","last_became_active":"2024-03-15T12:56:25.445092+0000","last_became_peered":"2024-03-15T12:56:25.445092+0000","last_unstale":"2024-03-15T12:57:19.837297+0000","last_undegraded":"2024-03-15T12:57:19.837297+0000","last_fullsized":"2024-03-15T12:57:19.837297+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:48:30.558243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836745+0000","last_change":"2024-03-15T12:56:20.214853+0000","last_active":"2024-03-15T12:57:19.836745+0000","last_peered":"2024-03-15T12:57:19.836745+0000","last_clean":"2024-03-15T12:57:19.836745+0000","last_became_active":"2024-03-15T12:56:20.214510+0000","last_became_peered":"2024-03-15T12:56:20.214510+0000","last_unstale":"2024-03-15T12:57:19.836745+0000","last_undegraded":"2024-03-15T12:57:19.836745+0000","last_fullsized":"2024-03-15T12:57:19.836745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:25:39.316428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798162+0000","last_change":"2024-03-15T12:56:21.216511+0000","last_active":"2024-03-15T12:57:48.798162+0000","last_peered":"2024-03-15T12:57:48.798162+0000","last_clean":"2024-03-15T12:57:48.798162+0000","last_became_active":"2024-03-15T12:56:21.216175+0000","last_became_peered":"2024-03-15T12:56:21.216175+0000","last_unstale":"2024-03-15T12:57:48.798162+0000","last_undegraded":"2024-03-15T12:57:48.798162+0000","last_fullsized":"2024-03-15T12:57:48.798162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:16:56.665841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103994+0000","last_change":"2024-03-15T12:56:25.446009+0000","last_active":"2024-03-15T12:57:20.103994+0000","last_peered":"2024-03-15T12:57:20.103994+0000","last_clean":"2024-03-15T12:57:20.103994+0000","last_became_active":"2024-03-15T12:56:25.445474+0000","last_became_peered":"2024-03-15T12:56:25.445474+0000","last_unstale":"2024-03-15T12:57:20.103994+0000","last_undegraded":"2024-03-15T12:57:20.103994+0000","last_fullsized":"2024-03-15T12:57:20.103994+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:53:46.856905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837300+0000","last_change":"2024-03-15T12:56:20.210146+0000","last_active":"2024-03-15T12:57:19.837300+0000","last_peered":"2024-03-15T12:57:19.837300+0000","last_clean":"2024-03-15T12:57:19.837300+0000","last_became_active":"2024-03-15T12:56:20.209769+0000","last_became_peered":"2024-03-15T12:56:20.209769+0000","last_unstale":"2024-03-15T12:57:19.837300+0000","last_undegraded":"2024-03-15T12:57:19.837300+0000","last_fullsized":"2024-03-15T12:57:19.837300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:47:32.505257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.770447+0000","last_change":"2024-03-15T12:56:21.214826+0000","last_active":"2024-03-15T12:57:48.770447+0000","last_peered":"2024-03-15T12:57:48.770447+0000","last_clean":"2024-03-15T12:57:48.770447+0000","last_became_active":"2024-03-15T12:56:21.214160+0000","last_became_peered":"2024-03-15T12:56:21.214160+0000","last_unstale":"2024-03-15T12:57:48.770447+0000","last_undegraded":"2024-03-15T12:57:48.770447+0000","last_fullsized":"2024-03-15T12:57:48.770447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:23:17.800522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837038+0000","last_change":"2024-03-15T12:56:25.443496+0000","last_active":"2024-03-15T12:57:19.837038+0000","last_peered":"2024-03-15T12:57:19.837038+0000","last_clean":"2024-03-15T12:57:19.837038+0000","last_became_active":"2024-03-15T12:56:25.442796+0000","last_became_peered":"2024-03-15T12:56:25.442796+0000","last_unstale":"2024-03-15T12:57:19.837038+0000","last_undegraded":"2024-03-15T12:57:19.837038+0000","last_fullsized":"2024-03-15T12:57:19.837038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:35:23.892693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.840452+0000","last_change":"2024-03-15T12:57:19.840452+0000","last_active":"2024-03-15T12:57:19.840452+0000","last_peered":"2024-03-15T12:57:19.840452+0000","last_clean":"2024-03-15T12:57:19.840452+0000","last_became_active":"2024-03-15T12:57:19.839977+0000","last_became_peered":"2024-03-15T12:57:19.839977+0000","last_unstale":"2024-03-15T12:57:19.840452+0000","last_undegraded":"2024-03-15T12:57:19.840452+0000","last_fullsized":"2024-03-15T12:57:19.840452+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:59:25.026455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.777760+0000","last_change":"2024-03-15T12:56:21.216393+0000","last_active":"2024-03-15T12:57:48.777760+0000","last_peered":"2024-03-15T12:57:48.777760+0000","last_clean":"2024-03-15T12:57:48.777760+0000","last_became_active":"2024-03-15T12:56:21.216043+0000","last_became_peered":"2024-03-15T12:56:21.216043+0000","last_unstale":"2024-03-15T12:57:48.777760+0000","last_undegraded":"2024-03-15T12:57:48.777760+0000","last_fullsized":"2024-03-15T12:57:48.777760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:05:49.843726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837337+0000","last_change":"2024-03-15T12:56:25.443854+0000","last_active":"2024-03-15T12:57:19.837337+0000","last_peered":"2024-03-15T12:57:19.837337+0000","last_clean":"2024-03-15T12:57:19.837337+0000","last_became_active":"2024-03-15T12:56:25.443422+0000","last_became_peered":"2024-03-15T12:56:25.443422+0000","last_unstale":"2024-03-15T12:57:19.837337+0000","last_undegraded":"2024-03-15T12:57:19.837337+0000","last_fullsized":"2024-03-15T12:57:19.837337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:52:18.999161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":103,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:51.522705+0000","last_change":"2024-03-15T12:56:20.211493+0000","last_active":"2024-03-15T12:57:51.522705+0000","last_peered":"2024-03-15T12:57:51.522705+0000","last_clean":"2024-03-15T12:57:51.522705+0000","last_became_active":"2024-03-15T12:56:20.211154+0000","last_became_peered":"2024-03-15T12:56:20.211154+0000","last_unstale":"2024-03-15T12:57:51.522705+0000","last_undegraded":"2024-03-15T12:57:51.522705+0000","last_fullsized":"2024-03-15T12:57:51.522705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:26:28.026899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.783016+0000","last_change":"2024-03-15T12:56:21.215999+0000","last_active":"2024-03-15T12:57:48.783016+0000","last_peered":"2024-03-15T12:57:48.783016+0000","last_clean":"2024-03-15T12:57:48.783016+0000","last_became_active":"2024-03-15T12:56:21.215657+0000","last_became_peered":"2024-03-15T12:56:21.215657+0000","last_unstale":"2024-03-15T12:57:48.783016+0000","last_undegraded":"2024-03-15T12:57:48.783016+0000","last_fullsized":"2024-03-15T12:57:48.783016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:47:59.901886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837243+0000","last_change":"2024-03-15T12:56:25.442246+0000","last_active":"2024-03-15T12:57:19.837243+0000","last_peered":"2024-03-15T12:57:19.837243+0000","last_clean":"2024-03-15T12:57:19.837243+0000","last_became_active":"2024-03-15T12:56:25.441755+0000","last_became_peered":"2024-03-15T12:56:25.441755+0000","last_unstale":"2024-03-15T12:57:19.837243+0000","last_undegraded":"2024-03-15T12:57:19.837243+0000","last_fullsized":"2024-03-15T12:57:19.837243+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:47:36.037864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.753331+0000","last_change":"2024-03-15T12:56:21.215746+0000","last_active":"2024-03-15T12:57:48.753331+0000","last_peered":"2024-03-15T12:57:48.753331+0000","last_clean":"2024-03-15T12:57:48.753331+0000","last_became_active":"2024-03-15T12:56:21.215424+0000","last_became_peered":"2024-03-15T12:56:21.215424+0000","last_unstale":"2024-03-15T12:57:48.753331+0000","last_undegraded":"2024-03-15T12:57:48.753331+0000","last_fullsized":"2024-03-15T12:57:48.753331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:01:15.498119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":106,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:53.525177+0000","last_change":"2024-03-15T12:56:20.219705+0000","last_active":"2024-03-15T12:57:53.525177+0000","last_peered":"2024-03-15T12:57:53.525177+0000","last_clean":"2024-03-15T12:57:53.525177+0000","last_became_active":"2024-03-15T12:56:20.219328+0000","last_became_peered":"2024-03-15T12:56:20.219328+0000","last_unstale":"2024-03-15T12:57:53.525177+0000","last_undegraded":"2024-03-15T12:57:53.525177+0000","last_fullsized":"2024-03-15T12:57:53.525177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:47:52.171116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104385+0000","last_change":"2024-03-15T12:56:25.443094+0000","last_active":"2024-03-15T12:57:20.104385+0000","last_peered":"2024-03-15T12:57:20.104385+0000","last_clean":"2024-03-15T12:57:20.104385+0000","last_became_active":"2024-03-15T12:56:25.442708+0000","last_became_peered":"2024-03-15T12:56:25.442708+0000","last_unstale":"2024-03-15T12:57:20.104385+0000","last_undegraded":"2024-03-15T12:57:20.104385+0000","last_fullsized":"2024-03-15T12:57:20.104385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:03:07.769836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.788346+0000","last_change":"2024-03-15T12:56:21.223937+0000","last_active":"2024-03-15T12:57:48.788346+0000","last_peered":"2024-03-15T12:57:48.788346+0000","last_clean":"2024-03-15T12:57:48.788346+0000","last_became_active":"2024-03-15T12:56:21.223583+0000","last_became_peered":"2024-03-15T12:56:21.223583+0000","last_unstale":"2024-03-15T12:57:48.788346+0000","last_undegraded":"2024-03-15T12:57:48.788346+0000","last_fullsized":"2024-03-15T12:57:48.788346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:15:34.287445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837101+0000","last_change":"2024-03-15T12:56:20.210812+0000","last_active":"2024-03-15T12:57:19.837101+0000","last_peered":"2024-03-15T12:57:19.837101+0000","last_clean":"2024-03-15T12:57:19.837101+0000","last_became_active":"2024-03-15T12:56:20.210457+0000","last_became_peered":"2024-03-15T12:56:20.210457+0000","last_unstale":"2024-03-15T12:57:19.837101+0000","last_undegraded":"2024-03-15T12:57:19.837101+0000","last_fullsized":"2024-03-15T12:57:19.837101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:04:37.371637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":358,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104758+0000","last_change":"2024-03-15T12:55:08.392548+0000","last_active":"2024-03-15T12:57:20.104758+0000","last_peered":"2024-03-15T12:57:20.104758+0000","last_clean":"2024-03-15T12:57:20.104758+0000","last_became_active":"2024-03-15T12:55:08.391972+0000","last_became_peered":"2024-03-15T12:55:08.391972+0000","last_unstale":"2024-03-15T12:57:20.104758+0000","last_undegraded":"2024-03-15T12:57:20.104758+0000","last_fullsized":"2024-03-15T12:57:20.104758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:51:47.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:51:47.093558+0000","last_clean_scrub_stamp":"2024-03-15T12:51:47.093558+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:30:57.264886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104859+0000","last_change":"2024-03-15T12:56:25.443212+0000","last_active":"2024-03-15T12:57:20.104859+0000","last_peered":"2024-03-15T12:57:20.104859+0000","last_clean":"2024-03-15T12:57:20.104859+0000","last_became_active":"2024-03-15T12:56:25.442787+0000","last_became_peered":"2024-03-15T12:56:25.442787+0000","last_unstale":"2024-03-15T12:57:20.104859+0000","last_undegraded":"2024-03-15T12:57:20.104859+0000","last_fullsized":"2024-03-15T12:57:20.104859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:59:32.244953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.775617+0000","last_change":"2024-03-15T12:57:19.839935+0000","last_active":"2024-03-15T12:57:48.775617+0000","last_peered":"2024-03-15T12:57:48.775617+0000","last_clean":"2024-03-15T12:57:48.775617+0000","last_became_active":"2024-03-15T12:57:19.839546+0000","last_became_peered":"2024-03-15T12:57:19.839546+0000","last_unstale":"2024-03-15T12:57:48.775617+0000","last_undegraded":"2024-03-15T12:57:48.775617+0000","last_fullsized":"2024-03-15T12:57:48.775617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:05:15.544159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104492+0000","last_change":"2024-03-15T12:56:20.210466+0000","last_active":"2024-03-15T12:57:20.104492+0000","last_peered":"2024-03-15T12:57:20.104492+0000","last_clean":"2024-03-15T12:57:20.104492+0000","last_became_active":"2024-03-15T12:56:20.210157+0000","last_became_peered":"2024-03-15T12:56:20.210157+0000","last_unstale":"2024-03-15T12:57:20.104492+0000","last_undegraded":"2024-03-15T12:57:20.104492+0000","last_fullsized":"2024-03-15T12:57:20.104492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:58:03.335826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.838275+0000","last_change":"2024-03-15T12:56:25.432641+0000","last_active":"2024-03-15T12:57:19.838275+0000","last_peered":"2024-03-15T12:57:19.838275+0000","last_clean":"2024-03-15T12:57:19.838275+0000","last_became_active":"2024-03-15T12:56:25.432212+0000","last_became_peered":"2024-03-15T12:56:25.432212+0000","last_unstale":"2024-03-15T12:57:19.838275+0000","last_undegraded":"2024-03-15T12:57:19.838275+0000","last_fullsized":"2024-03-15T12:57:19.838275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:40:20.447886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.846534+0000","last_change":"2024-03-15T12:57:19.846534+0000","last_active":"2024-03-15T12:57:19.846534+0000","last_peered":"2024-03-15T12:57:19.846534+0000","last_clean":"2024-03-15T12:57:19.846534+0000","last_became_active":"2024-03-15T12:57:19.841008+0000","last_became_peered":"2024-03-15T12:57:19.841008+0000","last_unstale":"2024-03-15T12:57:19.846534+0000","last_undegraded":"2024-03-15T12:57:19.846534+0000","last_fullsized":"2024-03-15T12:57:19.846534+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:46:17.129901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.777942+0000","last_change":"2024-03-15T12:56:21.207577+0000","last_active":"2024-03-15T12:57:48.777942+0000","last_peered":"2024-03-15T12:57:48.777942+0000","last_clean":"2024-03-15T12:57:48.777942+0000","last_became_active":"2024-03-15T12:56:21.207220+0000","last_became_peered":"2024-03-15T12:56:21.207220+0000","last_unstale":"2024-03-15T12:57:48.777942+0000","last_undegraded":"2024-03-15T12:57:48.777942+0000","last_fullsized":"2024-03-15T12:57:48.777942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:42:12.104220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103876+0000","last_change":"2024-03-15T12:56:25.439762+0000","last_active":"2024-03-15T12:57:20.103876+0000","last_peered":"2024-03-15T12:57:20.103876+0000","last_clean":"2024-03-15T12:57:20.103876+0000","last_became_active":"2024-03-15T12:56:25.439366+0000","last_became_peered":"2024-03-15T12:56:25.439366+0000","last_unstale":"2024-03-15T12:57:20.103876+0000","last_undegraded":"2024-03-15T12:57:20.103876+0000","last_fullsized":"2024-03-15T12:57:20.103876+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:36:29.638449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'13","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834416+0000","last_change":"2024-03-15T12:56:20.218438+0000","last_active":"2024-03-15T12:57:19.834416+0000","last_peered":"2024-03-15T12:57:19.834416+0000","last_clean":"2024-03-15T12:57:19.834416+0000","last_became_active":"2024-03-15T12:56:20.218071+0000","last_became_peered":"2024-03-15T12:56:20.218071+0000","last_unstale":"2024-03-15T12:57:19.834416+0000","last_undegraded":"2024-03-15T12:57:19.834416+0000","last_fullsized":"2024-03-15T12:57:19.834416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:24:16.295781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.783498+0000","last_change":"2024-03-15T12:56:21.218345+0000","last_active":"2024-03-15T12:57:48.783498+0000","last_peered":"2024-03-15T12:57:48.783498+0000","last_clean":"2024-03-15T12:57:48.783498+0000","last_became_active":"2024-03-15T12:56:21.216534+0000","last_became_peered":"2024-03-15T12:56:21.216534+0000","last_unstale":"2024-03-15T12:57:48.783498+0000","last_undegraded":"2024-03-15T12:57:48.783498+0000","last_fullsized":"2024-03-15T12:57:48.783498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:56:18.977509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834253+0000","last_change":"2024-03-15T12:56:25.435714+0000","last_active":"2024-03-15T12:57:19.834253+0000","last_peered":"2024-03-15T12:57:19.834253+0000","last_clean":"2024-03-15T12:57:19.834253+0000","last_became_active":"2024-03-15T12:56:25.435315+0000","last_became_peered":"2024-03-15T12:56:25.435315+0000","last_unstale":"2024-03-15T12:57:19.834253+0000","last_undegraded":"2024-03-15T12:57:19.834253+0000","last_fullsized":"2024-03-15T12:57:19.834253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:10:03.509078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837180+0000","last_change":"2024-03-15T12:56:20.209356+0000","last_active":"2024-03-15T12:57:19.837180+0000","last_peered":"2024-03-15T12:57:19.837180+0000","last_clean":"2024-03-15T12:57:19.837180+0000","last_became_active":"2024-03-15T12:56:20.208995+0000","last_became_peered":"2024-03-15T12:56:20.208995+0000","last_unstale":"2024-03-15T12:57:19.837180+0000","last_undegraded":"2024-03-15T12:57:19.837180+0000","last_fullsized":"2024-03-15T12:57:19.837180+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:14:08.554567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782844+0000","last_change":"2024-03-15T12:56:21.216315+0000","last_active":"2024-03-15T12:57:48.782844+0000","last_peered":"2024-03-15T12:57:48.782844+0000","last_clean":"2024-03-15T12:57:48.782844+0000","last_became_active":"2024-03-15T12:56:21.215943+0000","last_became_peered":"2024-03-15T12:56:21.215943+0000","last_unstale":"2024-03-15T12:57:48.782844+0000","last_undegraded":"2024-03-15T12:57:48.782844+0000","last_fullsized":"2024-03-15T12:57:48.782844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:47:29.913048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836875+0000","last_change":"2024-03-15T12:56:25.437200+0000","last_active":"2024-03-15T12:57:19.836875+0000","last_peered":"2024-03-15T12:57:19.836875+0000","last_clean":"2024-03-15T12:57:19.836875+0000","last_became_active":"2024-03-15T12:56:25.436786+0000","last_became_peered":"2024-03-15T12:56:25.436786+0000","last_unstale":"2024-03-15T12:57:19.836875+0000","last_undegraded":"2024-03-15T12:57:19.836875+0000","last_fullsized":"2024-03-15T12:57:19.836875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:12:00.286749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106318+0000","last_change":"2024-03-15T12:56:20.202240+0000","last_active":"2024-03-15T12:57:20.106318+0000","last_peered":"2024-03-15T12:57:20.106318+0000","last_clean":"2024-03-15T12:57:20.106318+0000","last_became_active":"2024-03-15T12:56:20.201136+0000","last_became_peered":"2024-03-15T12:56:20.201136+0000","last_unstale":"2024-03-15T12:57:20.106318+0000","last_undegraded":"2024-03-15T12:57:20.106318+0000","last_fullsized":"2024-03-15T12:57:20.106318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:30:35.901237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.737482+0000","last_change":"2024-03-15T12:56:21.212095+0000","last_active":"2024-03-15T12:57:48.737482+0000","last_peered":"2024-03-15T12:57:48.737482+0000","last_clean":"2024-03-15T12:57:48.737482+0000","last_became_active":"2024-03-15T12:56:21.211744+0000","last_became_peered":"2024-03-15T12:56:21.211744+0000","last_unstale":"2024-03-15T12:57:48.737482+0000","last_undegraded":"2024-03-15T12:57:48.737482+0000","last_fullsized":"2024-03-15T12:57:48.737482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:27:58.412513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106031+0000","last_change":"2024-03-15T12:57:20.106031+0000","last_active":"2024-03-15T12:57:20.106031+0000","last_peered":"2024-03-15T12:57:20.106031+0000","last_clean":"2024-03-15T12:57:20.106031+0000","last_became_active":"2024-03-15T12:57:20.105499+0000","last_became_peered":"2024-03-15T12:57:20.105499+0000","last_unstale":"2024-03-15T12:57:20.106031+0000","last_undegraded":"2024-03-15T12:57:20.106031+0000","last_fullsized":"2024-03-15T12:57:20.106031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:17:08.138164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104082+0000","last_change":"2024-03-15T12:56:20.217319+0000","last_active":"2024-03-15T12:57:20.104082+0000","last_peered":"2024-03-15T12:57:20.104082+0000","last_clean":"2024-03-15T12:57:20.104082+0000","last_became_active":"2024-03-15T12:56:20.215450+0000","last_became_peered":"2024-03-15T12:56:20.215450+0000","last_unstale":"2024-03-15T12:57:20.104082+0000","last_undegraded":"2024-03-15T12:57:20.104082+0000","last_fullsized":"2024-03-15T12:57:20.104082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:07:09.085860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780127+0000","last_change":"2024-03-15T12:56:21.209360+0000","last_active":"2024-03-15T12:57:48.780127+0000","last_peered":"2024-03-15T12:57:48.780127+0000","last_clean":"2024-03-15T12:57:48.780127+0000","last_became_active":"2024-03-15T12:56:21.208987+0000","last_became_peered":"2024-03-15T12:56:21.208987+0000","last_unstale":"2024-03-15T12:57:48.780127+0000","last_undegraded":"2024-03-15T12:57:48.780127+0000","last_fullsized":"2024-03-15T12:57:48.780127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:19:13.353400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836774+0000","last_change":"2024-03-15T12:56:25.439173+0000","last_active":"2024-03-15T12:57:19.836774+0000","last_peered":"2024-03-15T12:57:19.836774+0000","last_clean":"2024-03-15T12:57:19.836774+0000","last_became_active":"2024-03-15T12:56:25.438812+0000","last_became_peered":"2024-03-15T12:56:25.438812+0000","last_unstale":"2024-03-15T12:57:19.836774+0000","last_undegraded":"2024-03-15T12:57:19.836774+0000","last_fullsized":"2024-03-15T12:57:19.836774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:10:39.107190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834324+0000","last_change":"2024-03-15T12:56:20.211861+0000","last_active":"2024-03-15T12:57:19.834324+0000","last_peered":"2024-03-15T12:57:19.834324+0000","last_clean":"2024-03-15T12:57:19.834324+0000","last_became_active":"2024-03-15T12:56:20.211481+0000","last_became_peered":"2024-03-15T12:56:20.211481+0000","last_unstale":"2024-03-15T12:57:19.834324+0000","last_undegraded":"2024-03-15T12:57:19.834324+0000","last_fullsized":"2024-03-15T12:57:19.834324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:12:45.558423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.783578+0000","last_change":"2024-03-15T12:56:21.216058+0000","last_active":"2024-03-15T12:57:48.783578+0000","last_peered":"2024-03-15T12:57:48.783578+0000","last_clean":"2024-03-15T12:57:48.783578+0000","last_became_active":"2024-03-15T12:56:21.215706+0000","last_became_peered":"2024-03-15T12:56:21.215706+0000","last_unstale":"2024-03-15T12:57:48.783578+0000","last_undegraded":"2024-03-15T12:57:48.783578+0000","last_fullsized":"2024-03-15T12:57:48.783578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:55:02.132976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836966+0000","last_change":"2024-03-15T12:56:25.444134+0000","last_active":"2024-03-15T12:57:19.836966+0000","last_peered":"2024-03-15T12:57:19.836966+0000","last_clean":"2024-03-15T12:57:19.836966+0000","last_became_active":"2024-03-15T12:56:25.443452+0000","last_became_peered":"2024-03-15T12:56:25.443452+0000","last_unstale":"2024-03-15T12:57:19.836966+0000","last_undegraded":"2024-03-15T12:57:19.836966+0000","last_fullsized":"2024-03-15T12:57:19.836966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:06:39.660016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836726+0000","last_change":"2024-03-15T12:56:20.214700+0000","last_active":"2024-03-15T12:57:19.836726+0000","last_peered":"2024-03-15T12:57:19.836726+0000","last_clean":"2024-03-15T12:57:19.836726+0000","last_became_active":"2024-03-15T12:56:20.214348+0000","last_became_peered":"2024-03-15T12:56:20.214348+0000","last_unstale":"2024-03-15T12:57:19.836726+0000","last_undegraded":"2024-03-15T12:57:19.836726+0000","last_fullsized":"2024-03-15T12:57:19.836726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:24:37.551182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798759+0000","last_change":"2024-03-15T12:56:21.220746+0000","last_active":"2024-03-15T12:57:48.798759+0000","last_peered":"2024-03-15T12:57:48.798759+0000","last_clean":"2024-03-15T12:57:48.798759+0000","last_became_active":"2024-03-15T12:56:21.220445+0000","last_became_peered":"2024-03-15T12:56:21.220445+0000","last_unstale":"2024-03-15T12:57:48.798759+0000","last_undegraded":"2024-03-15T12:57:48.798759+0000","last_fullsized":"2024-03-15T12:57:48.798759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:58:44.119476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104700+0000","last_change":"2024-03-15T12:56:25.437611+0000","last_active":"2024-03-15T12:57:20.104700+0000","last_peered":"2024-03-15T12:57:20.104700+0000","last_clean":"2024-03-15T12:57:20.104700+0000","last_became_active":"2024-03-15T12:56:25.437195+0000","last_became_peered":"2024-03-15T12:56:25.437195+0000","last_unstale":"2024-03-15T12:57:20.104700+0000","last_undegraded":"2024-03-15T12:57:20.104700+0000","last_fullsized":"2024-03-15T12:57:20.104700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:26:43.558477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":106,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:53.541938+0000","last_change":"2024-03-15T12:56:20.208584+0000","last_active":"2024-03-15T12:57:53.541938+0000","last_peered":"2024-03-15T12:57:53.541938+0000","last_clean":"2024-03-15T12:57:53.541938+0000","last_became_active":"2024-03-15T12:56:20.208155+0000","last_became_peered":"2024-03-15T12:56:20.208155+0000","last_unstale":"2024-03-15T12:57:53.541938+0000","last_undegraded":"2024-03-15T12:57:53.541938+0000","last_fullsized":"2024-03-15T12:57:53.541938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:26:13.148273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798239+0000","last_change":"2024-03-15T12:56:21.220421+0000","last_active":"2024-03-15T12:57:48.798239+0000","last_peered":"2024-03-15T12:57:48.798239+0000","last_clean":"2024-03-15T12:57:48.798239+0000","last_became_active":"2024-03-15T12:56:21.220095+0000","last_became_peered":"2024-03-15T12:56:21.220095+0000","last_unstale":"2024-03-15T12:57:48.798239+0000","last_undegraded":"2024-03-15T12:57:48.798239+0000","last_fullsized":"2024-03-15T12:57:48.798239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:35:33.846506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836547+0000","last_change":"2024-03-15T12:56:25.440408+0000","last_active":"2024-03-15T12:57:19.836547+0000","last_peered":"2024-03-15T12:57:19.836547+0000","last_clean":"2024-03-15T12:57:19.836547+0000","last_became_active":"2024-03-15T12:56:25.439965+0000","last_became_peered":"2024-03-15T12:56:25.439965+0000","last_unstale":"2024-03-15T12:57:19.836547+0000","last_undegraded":"2024-03-15T12:57:19.836547+0000","last_fullsized":"2024-03-15T12:57:19.836547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:12:57.278202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.105004+0000","last_change":"2024-03-15T12:56:20.216320+0000","last_active":"2024-03-15T12:57:20.105004+0000","last_peered":"2024-03-15T12:57:20.105004+0000","last_clean":"2024-03-15T12:57:20.105004+0000","last_became_active":"2024-03-15T12:56:20.215915+0000","last_became_peered":"2024-03-15T12:56:20.215915+0000","last_unstale":"2024-03-15T12:57:20.105004+0000","last_undegraded":"2024-03-15T12:57:20.105004+0000","last_fullsized":"2024-03-15T12:57:20.105004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:25:45.236269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782661+0000","last_change":"2024-03-15T12:56:21.215822+0000","last_active":"2024-03-15T12:57:48.782661+0000","last_peered":"2024-03-15T12:57:48.782661+0000","last_clean":"2024-03-15T12:57:48.782661+0000","last_became_active":"2024-03-15T12:56:21.215508+0000","last_became_peered":"2024-03-15T12:56:21.215508+0000","last_unstale":"2024-03-15T12:57:48.782661+0000","last_undegraded":"2024-03-15T12:57:48.782661+0000","last_fullsized":"2024-03-15T12:57:48.782661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:56:55.919198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103800+0000","last_change":"2024-03-15T12:56:25.436186+0000","last_active":"2024-03-15T12:57:20.103800+0000","last_peered":"2024-03-15T12:57:20.103800+0000","last_clean":"2024-03-15T12:57:20.103800+0000","last_became_active":"2024-03-15T12:56:25.435586+0000","last_became_peered":"2024-03-15T12:56:25.435586+0000","last_unstale":"2024-03-15T12:57:20.103800+0000","last_undegraded":"2024-03-15T12:57:20.103800+0000","last_fullsized":"2024-03-15T12:57:20.103800+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:02:26.850487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837444+0000","last_change":"2024-03-15T12:56:20.217277+0000","last_active":"2024-03-15T12:57:19.837444+0000","last_peered":"2024-03-15T12:57:19.837444+0000","last_clean":"2024-03-15T12:57:19.837444+0000","last_became_active":"2024-03-15T12:56:20.216930+0000","last_became_peered":"2024-03-15T12:56:20.216930+0000","last_unstale":"2024-03-15T12:57:19.837444+0000","last_undegraded":"2024-03-15T12:57:19.837444+0000","last_fullsized":"2024-03-15T12:57:19.837444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:47:51.075545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.779215+0000","last_change":"2024-03-15T12:56:21.212066+0000","last_active":"2024-03-15T12:57:48.779215+0000","last_peered":"2024-03-15T12:57:48.779215+0000","last_clean":"2024-03-15T12:57:48.779215+0000","last_became_active":"2024-03-15T12:56:21.211650+0000","last_became_peered":"2024-03-15T12:56:21.211650+0000","last_unstale":"2024-03-15T12:57:48.779215+0000","last_undegraded":"2024-03-15T12:57:48.779215+0000","last_fullsized":"2024-03-15T12:57:48.779215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:40:35.322210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.772568+0000","last_change":"2024-03-15T12:56:21.212794+0000","last_active":"2024-03-15T12:57:48.772568+0000","last_peered":"2024-03-15T12:57:48.772568+0000","last_clean":"2024-03-15T12:57:48.772568+0000","last_became_active":"2024-03-15T12:56:21.212470+0000","last_became_peered":"2024-03-15T12:56:21.212470+0000","last_unstale":"2024-03-15T12:57:48.772568+0000","last_undegraded":"2024-03-15T12:57:48.772568+0000","last_fullsized":"2024-03-15T12:57:48.772568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:10:04.614223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.756066+0000","last_change":"2024-03-15T12:56:21.211746+0000","last_active":"2024-03-15T12:57:48.756066+0000","last_peered":"2024-03-15T12:57:48.756066+0000","last_clean":"2024-03-15T12:57:48.756066+0000","last_became_active":"2024-03-15T12:56:21.211431+0000","last_became_peered":"2024-03-15T12:56:21.211431+0000","last_unstale":"2024-03-15T12:57:48.756066+0000","last_undegraded":"2024-03-15T12:57:48.756066+0000","last_fullsized":"2024-03-15T12:57:48.756066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:46:31.188408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.777903+0000","last_change":"2024-03-15T12:56:21.220997+0000","last_active":"2024-03-15T12:57:48.777903+0000","last_peered":"2024-03-15T12:57:48.777903+0000","last_clean":"2024-03-15T12:57:48.777903+0000","last_became_active":"2024-03-15T12:56:21.220644+0000","last_became_peered":"2024-03-15T12:56:21.220644+0000","last_unstale":"2024-03-15T12:57:48.777903+0000","last_undegraded":"2024-03-15T12:57:48.777903+0000","last_fullsized":"2024-03-15T12:57:48.777903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:47:55.852845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.779887+0000","last_change":"2024-03-15T12:56:21.211634+0000","last_active":"2024-03-15T12:57:48.779887+0000","last_peered":"2024-03-15T12:57:48.779887+0000","last_clean":"2024-03-15T12:57:48.779887+0000","last_became_active":"2024-03-15T12:56:21.211303+0000","last_became_peered":"2024-03-15T12:56:21.211303+0000","last_unstale":"2024-03-15T12:57:48.779887+0000","last_undegraded":"2024-03-15T12:57:48.779887+0000","last_fullsized":"2024-03-15T12:57:48.779887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:22:32.358270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780639+0000","last_change":"2024-03-15T12:56:21.211673+0000","last_active":"2024-03-15T12:57:48.780639+0000","last_peered":"2024-03-15T12:57:48.780639+0000","last_clean":"2024-03-15T12:57:48.780639+0000","last_became_active":"2024-03-15T12:56:21.211296+0000","last_became_peered":"2024-03-15T12:56:21.211296+0000","last_unstale":"2024-03-15T12:57:48.780639+0000","last_undegraded":"2024-03-15T12:57:48.780639+0000","last_fullsized":"2024-03-15T12:57:48.780639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:40:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.769773+0000","last_change":"2024-03-15T12:56:21.208681+0000","last_active":"2024-03-15T12:57:48.769773+0000","last_peered":"2024-03-15T12:57:48.769773+0000","last_clean":"2024-03-15T12:57:48.769773+0000","last_became_active":"2024-03-15T12:56:21.208130+0000","last_became_peered":"2024-03-15T12:56:21.208130+0000","last_unstale":"2024-03-15T12:57:48.769773+0000","last_undegraded":"2024-03-15T12:57:48.769773+0000","last_fullsized":"2024-03-15T12:57:48.769773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:34:52.971485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.793487+0000","last_change":"2024-03-15T12:56:21.216979+0000","last_active":"2024-03-15T12:57:48.793487+0000","last_peered":"2024-03-15T12:57:48.793487+0000","last_clean":"2024-03-15T12:57:48.793487+0000","last_became_active":"2024-03-15T12:56:21.216632+0000","last_became_peered":"2024-03-15T12:56:21.216632+0000","last_unstale":"2024-03-15T12:57:48.793487+0000","last_undegraded":"2024-03-15T12:57:48.793487+0000","last_fullsized":"2024-03-15T12:57:48.793487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:21:57.213512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.772318+0000","last_change":"2024-03-15T12:56:21.223064+0000","last_active":"2024-03-15T12:57:48.772318+0000","last_peered":"2024-03-15T12:57:48.772318+0000","last_clean":"2024-03-15T12:57:48.772318+0000","last_became_active":"2024-03-15T12:56:21.220882+0000","last_became_peered":"2024-03-15T12:56:21.220882+0000","last_unstale":"2024-03-15T12:57:48.772318+0000","last_undegraded":"2024-03-15T12:57:48.772318+0000","last_fullsized":"2024-03-15T12:57:48.772318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:45:02.930154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.772144+0000","last_change":"2024-03-15T12:56:21.218614+0000","last_active":"2024-03-15T12:57:48.772144+0000","last_peered":"2024-03-15T12:57:48.772144+0000","last_clean":"2024-03-15T12:57:48.772144+0000","last_became_active":"2024-03-15T12:56:21.218224+0000","last_became_peered":"2024-03-15T12:56:21.218224+0000","last_unstale":"2024-03-15T12:57:48.772144+0000","last_undegraded":"2024-03-15T12:57:48.772144+0000","last_fullsized":"2024-03-15T12:57:48.772144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:41:29.790358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782684+0000","last_change":"2024-03-15T12:56:21.219019+0000","last_active":"2024-03-15T12:57:48.782684+0000","last_peered":"2024-03-15T12:57:48.782684+0000","last_clean":"2024-03-15T12:57:48.782684+0000","last_became_active":"2024-03-15T12:56:21.218673+0000","last_became_peered":"2024-03-15T12:56:21.218673+0000","last_unstale":"2024-03-15T12:57:48.782684+0000","last_undegraded":"2024-03-15T12:57:48.782684+0000","last_fullsized":"2024-03-15T12:57:48.782684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:42:11.788598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.799863+0000","last_change":"2024-03-15T12:56:21.220355+0000","last_active":"2024-03-15T12:57:48.799863+0000","last_peered":"2024-03-15T12:57:48.799863+0000","last_clean":"2024-03-15T12:57:48.799863+0000","last_became_active":"2024-03-15T12:56:21.220021+0000","last_became_peered":"2024-03-15T12:56:21.220021+0000","last_unstale":"2024-03-15T12:57:48.799863+0000","last_undegraded":"2024-03-15T12:57:48.799863+0000","last_fullsized":"2024-03-15T12:57:48.799863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:40:00.947351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.781147+0000","last_change":"2024-03-15T12:56:21.213050+0000","last_active":"2024-03-15T12:57:48.781147+0000","last_peered":"2024-03-15T12:57:48.781147+0000","last_clean":"2024-03-15T12:57:48.781147+0000","last_became_active":"2024-03-15T12:56:21.212730+0000","last_became_peered":"2024-03-15T12:56:21.212730+0000","last_unstale":"2024-03-15T12:57:48.781147+0000","last_undegraded":"2024-03-15T12:57:48.781147+0000","last_fullsized":"2024-03-15T12:57:48.781147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:56:36.396285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.770617+0000","last_change":"2024-03-15T12:56:21.213942+0000","last_active":"2024-03-15T12:57:48.770617+0000","last_peered":"2024-03-15T12:57:48.770617+0000","last_clean":"2024-03-15T12:57:48.770617+0000","last_became_active":"2024-03-15T12:56:21.213611+0000","last_became_peered":"2024-03-15T12:56:21.213611+0000","last_unstale":"2024-03-15T12:57:48.770617+0000","last_undegraded":"2024-03-15T12:57:48.770617+0000","last_fullsized":"2024-03-15T12:57:48.770617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:23:44.002970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68978,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":767,"num_read_kb":462,"num_write":246,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":206934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":501056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37020,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708132,"statfs":{"total":95995035648,"available":95957127168,"internally_reserved":0,"allocated":1724416,"data_stored":1150316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":504970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070511,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":501140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332148,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36948,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708204,"statfs":{"total":95995035648,"available":95957200896,"internally_reserved":0,"allocated":1650688,"data_stored":1097890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":496378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":559432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":497834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":559652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":496086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":500984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1092940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-15T12:57:57.571 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-15T12:57:57.572 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-15T12:57:57.572 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-15T12:57:57.572 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-15T12:57:57.823 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:57:58.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:58 smithi149 ceph-mon[35927]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-15T12:57:58.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:57:58 smithi149 ceph-mon[35927]: from='client.24940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:57:58.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:58 smithi002 ceph-mon[32060]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-15T12:57:58.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:57:58 smithi002 ceph-mon[32060]: from='client.24940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:57:59.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:58 smithi184 ceph-mon[35993]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-15T12:57:59.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:57:58 smithi184 ceph-mon[35993]: from='client.24940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:57:59.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.529+0000 7f3462d9b700 1 -- 172.21.15.2:0/3604393009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c1072f0 msgr2=0x7f345c10eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:59.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.529+0000 7f3462d9b700 1 --2- 172.21.15.2:0/3604393009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c1072f0 0x7f345c10eec0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f344c009990 tx=0x7f344c009ca0 comp rx=0 tx=0).stop 2024-03-15T12:57:59.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.529+0000 7f3462d9b700 1 -- 172.21.15.2:0/3604393009 shutdown_connections 2024-03-15T12:57:59.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.529+0000 7f3462d9b700 1 --2- 172.21.15.2:0/3604393009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c1072f0 0x7f345c10eec0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:59.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.529+0000 7f3462d9b700 1 --2- 172.21.15.2:0/3604393009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f345c1022c0 0x7f345c1026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:59.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.529+0000 7f3462d9b700 1 --2- 172.21.15.2:0/3604393009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f345c102c70 0x7f345c106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:59.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.529+0000 7f3462d9b700 1 -- 172.21.15.2:0/3604393009 >> 172.21.15.2:0/3604393009 conn(0x7f345c079070 msgr2=0x7f345c079480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:59.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.529+0000 7f3462d9b700 1 -- 172.21.15.2:0/3604393009 shutdown_connections 2024-03-15T12:57:59.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.529+0000 7f3462d9b700 1 -- 172.21.15.2:0/3604393009 wait complete. 2024-03-15T12:57:59.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.530+0000 7f3462d9b700 1 Processor -- start 2024-03-15T12:57:59.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.530+0000 7f3462d9b700 1 -- start start 2024-03-15T12:57:59.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.530+0000 7f3462d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f345c1022c0 0x7f345c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:59.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.530+0000 7f3462d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c102c70 0x7f345c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:59.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.530+0000 7f3462d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f345c1072f0 0x7f345c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:59.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.530+0000 7f3462d9b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c19d380 con 0x7f345c102c70 2024-03-15T12:57:59.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.530+0000 7f3462d9b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c19d4f0 con 0x7f345c1072f0 2024-03-15T12:57:59.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.530+0000 7f3462d9b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c19d6b0 con 0x7f345c1022c0 2024-03-15T12:57:59.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.530+0000 7f345bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c102c70 0x7f345c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:59.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.530+0000 7f3460b37700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f345c1022c0 0x7f345c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:59.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.530+0000 7f3460b37700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f345c1022c0 0x7f345c1984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:44504/0 (socket says 172.21.15.2:44504) 2024-03-15T12:57:59.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.530+0000 7f3460b37700 1 -- 172.21.15.2:0/702049089 learned_addr learned my addr 172.21.15.2:0/702049089 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:57:59.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.530+0000 7f3461338700 1 --2- 172.21.15.2:0/702049089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f345c1072f0 0x7f345c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:59.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.531+0000 7f345bfff700 1 -- 172.21.15.2:0/702049089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f345c1022c0 msgr2=0x7f345c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:59.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.531+0000 7f345bfff700 1 --2- 172.21.15.2:0/702049089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f345c1022c0 0x7f345c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:59.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.531+0000 7f345bfff700 1 -- 172.21.15.2:0/702049089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f345c1072f0 msgr2=0x7f345c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:59.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.531+0000 7f345bfff700 1 --2- 172.21.15.2:0/702049089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f345c1072f0 0x7f345c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:59.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.531+0000 7f345bfff700 1 -- 172.21.15.2:0/702049089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c0095f0 con 0x7f345c102c70 2024-03-15T12:57:59.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.531+0000 7f3460b37700 1 --2- 172.21.15.2:0/702049089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f345c1022c0 0x7f345c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:57:59.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.531+0000 7f345bfff700 1 --2- 172.21.15.2:0/702049089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c102c70 0x7f345c198a30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f345000e9c0 tx=0x7f345000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:59.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.531+0000 7f3459ffb700 1 -- 172.21.15.2:0/702049089 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345000c9c0 con 0x7f345c102c70 2024-03-15T12:57:59.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.531+0000 7f3459ffb700 1 -- 172.21.15.2:0/702049089 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3450010430 con 0x7f345c102c70 2024-03-15T12:57:59.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.531+0000 7f3459ffb700 1 -- 172.21.15.2:0/702049089 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450018680 con 0x7f345c102c70 2024-03-15T12:57:59.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.531+0000 7f3462d9b700 1 -- 172.21.15.2:0/702049089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345c19d990 con 0x7f345c102c70 2024-03-15T12:57:59.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.531+0000 7f3462d9b700 1 -- 172.21.15.2:0/702049089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f345c0743e0 con 0x7f345c102c70 2024-03-15T12:57:59.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.532+0000 7f3462d9b700 1 -- 172.21.15.2:0/702049089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345c10a5a0 con 0x7f345c102c70 2024-03-15T12:57:59.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.534+0000 7f3459ffb700 1 -- 172.21.15.2:0/702049089 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f34500105a0 con 0x7f345c102c70 2024-03-15T12:57:59.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.534+0000 7f3459ffb700 1 --2- 172.21.15.2:0/702049089 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f34440a1ac0 0x7f34440a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:57:59.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.534+0000 7f3460b37700 1 --2- 172.21.15.2:0/702049089 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f34440a1ac0 0x7f34440a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:57:59.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.534+0000 7f3459ffb700 1 -- 172.21.15.2:0/702049089 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f3450014070 con 0x7f345c102c70 2024-03-15T12:57:59.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.535+0000 7f3460b37700 1 --2- 172.21.15.2:0/702049089 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f34440a1ac0 0x7f34440a3f80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3454009c10 tx=0x7f3454012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:57:59.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.538+0000 7f3459ffb700 1 -- 172.21.15.2:0/702049089 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3450096af0 con 0x7f345c102c70 2024-03-15T12:57:59.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.791+0000 7f3462d9b700 1 -- 172.21.15.2:0/702049089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f345c068ca0 con 0x7f345c102c70 2024-03-15T12:57:59.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.792+0000 7f3459ffb700 1 -- 172.21.15.2:0/702049089 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f345009a1b0 con 0x7f345c102c70 2024-03-15T12:57:59.778 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:57:59.778 INFO:teuthology.orchestra.run.smithi002.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-15T12:57:59.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.794+0000 7f3462d9b700 1 -- 172.21.15.2:0/702049089 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f34440a1ac0 msgr2=0x7f34440a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:59.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.794+0000 7f3462d9b700 1 --2- 172.21.15.2:0/702049089 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f34440a1ac0 0x7f34440a3f80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3454009c10 tx=0x7f3454012040 comp rx=0 tx=0).stop 2024-03-15T12:57:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.794+0000 7f3462d9b700 1 -- 172.21.15.2:0/702049089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c102c70 msgr2=0x7f345c198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:57:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.794+0000 7f3462d9b700 1 --2- 172.21.15.2:0/702049089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c102c70 0x7f345c198a30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f345000e9c0 tx=0x7f345000ecd0 comp rx=0 tx=0).stop 2024-03-15T12:57:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.794+0000 7f3462d9b700 1 -- 172.21.15.2:0/702049089 shutdown_connections 2024-03-15T12:57:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.794+0000 7f3462d9b700 1 --2- 172.21.15.2:0/702049089 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f34440a1ac0 0x7f34440a3f80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.794+0000 7f3462d9b700 1 --2- 172.21.15.2:0/702049089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f345c1072f0 0x7f345c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.794+0000 7f3462d9b700 1 --2- 172.21.15.2:0/702049089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f345c1022c0 0x7f345c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.794+0000 7f3462d9b700 1 --2- 172.21.15.2:0/702049089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f345c102c70 0x7f345c198a30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:57:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.794+0000 7f3462d9b700 1 -- 172.21.15.2:0/702049089 >> 172.21.15.2:0/702049089 conn(0x7f345c079070 msgr2=0x7f345c1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:57:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.794+0000 7f3462d9b700 1 -- 172.21.15.2:0/702049089 shutdown_connections 2024-03-15T12:57:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:57:59.794+0000 7f3462d9b700 1 -- 172.21.15.2:0/702049089 wait complete. 2024-03-15T12:58:00.623 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-15T12:58:00.623 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:58:00.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.749+0000 7f6f0091e640 1 -- 172.21.15.2:0/157453837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6efc102040 msgr2=0x7f6efc1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.749+0000 7f6f0091e640 1 --2- 172.21.15.2:0/157453837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6efc102040 0x7f6efc1088d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009ca0 tx=0x7f6ef402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.749+0000 7f6f0091e640 1 -- 172.21.15.2:0/157453837 shutdown_connections 2024-03-15T12:58:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.749+0000 7f6f0091e640 1 --2- 172.21.15.2:0/157453837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6efc102040 0x7f6efc1088d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.749+0000 7f6f0091e640 1 --2- 172.21.15.2:0/157453837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6efc101680 0x7f6efc101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.749+0000 7f6f0091e640 1 --2- 172.21.15.2:0/157453837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6efc100490 0x7f6efc1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.749+0000 7f6f0091e640 1 -- 172.21.15.2:0/157453837 >> 172.21.15.2:0/157453837 conn(0x7f6efc0fba20 msgr2=0x7f6efc0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.749+0000 7f6f0091e640 1 -- 172.21.15.2:0/157453837 shutdown_connections 2024-03-15T12:58:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.749+0000 7f6f0091e640 1 -- 172.21.15.2:0/157453837 wait complete. 2024-03-15T12:58:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.750+0000 7f6f0091e640 1 Processor -- start 2024-03-15T12:58:00.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.750+0000 7f6f0091e640 1 -- start start 2024-03-15T12:58:00.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.750+0000 7f6f0091e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6efc100490 0x7f6efc06a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:00.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.750+0000 7f6f0091e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6efc101680 0x7f6efc19c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:00.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.750+0000 7f6f0091e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6efc102040 0x7f6efc19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:00.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.750+0000 7f6f0091e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc19f070 con 0x7f6efc102040 2024-03-15T12:58:00.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.750+0000 7f6f0091e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc19f1e0 con 0x7f6efc100490 2024-03-15T12:58:00.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.750+0000 7f6f0091e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc19f3a0 con 0x7f6efc101680 2024-03-15T12:58:00.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.750+0000 7f6efad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6efc102040 0x7f6efc19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:00.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.750+0000 7f6ef3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6efc101680 0x7f6efc19c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:00.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.750+0000 7f6efa59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6efc100490 0x7f6efc06a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:00.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.750+0000 7f6efad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6efc102040 0x7f6efc19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47570/0 (socket says 172.21.15.2:47570) 2024-03-15T12:58:00.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.750+0000 7f6efad9d640 1 -- 172.21.15.2:0/1559113072 learned_addr learned my addr 172.21.15.2:0/1559113072 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:00.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.751+0000 7f6efad9d640 1 -- 172.21.15.2:0/1559113072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6efc101680 msgr2=0x7f6efc19c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:00.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.751+0000 7f6efad9d640 1 --2- 172.21.15.2:0/1559113072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6efc101680 0x7f6efc19c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:00.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.751+0000 7f6efad9d640 1 -- 172.21.15.2:0/1559113072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6efc100490 msgr2=0x7f6efc06a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:00.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.751+0000 7f6efad9d640 1 --2- 172.21.15.2:0/1559113072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6efc100490 0x7f6efc06a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:00.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.751+0000 7f6efad9d640 1 -- 172.21.15.2:0/1559113072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee4009490 con 0x7f6efc102040 2024-03-15T12:58:00.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.751+0000 7f6efa59c640 1 --2- 172.21.15.2:0/1559113072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6efc100490 0x7f6efc06a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:00.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.751+0000 7f6efad9d640 1 --2- 172.21.15.2:0/1559113072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6efc102040 0x7f6efc19e890 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4005cc0 tx=0x7f6ef4032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:00.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.751+0000 7f6ef37fe640 1 -- 172.21.15.2:0/1559113072 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4034c90 con 0x7f6efc102040 2024-03-15T12:58:00.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.751+0000 7f6ef37fe640 1 -- 172.21.15.2:0/1559113072 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6ef4003e40 con 0x7f6efc102040 2024-03-15T12:58:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.751+0000 7f6f0091e640 1 -- 172.21.15.2:0/1559113072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef4009900 con 0x7f6efc102040 2024-03-15T12:58:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.751+0000 7f6ef37fe640 1 -- 172.21.15.2:0/1559113072 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40048a0 con 0x7f6efc102040 2024-03-15T12:58:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.751+0000 7f6f0091e640 1 -- 172.21.15.2:0/1559113072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6efc19f8e0 con 0x7f6efc102040 2024-03-15T12:58:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.752+0000 7f6f0091e640 1 -- 172.21.15.2:0/1559113072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6efc068760 con 0x7f6efc102040 2024-03-15T12:58:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.753+0000 7f6ef37fe640 1 -- 172.21.15.2:0/1559113072 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f6ef400a040 con 0x7f6efc102040 2024-03-15T12:58:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.754+0000 7f6ef37fe640 1 --2- 172.21.15.2:0/1559113072 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6ee80a1a30 0x7f6ee80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.754+0000 7f6efa59c640 1 --2- 172.21.15.2:0/1559113072 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6ee80a1a30 0x7f6ee80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.754+0000 7f6ef37fe640 1 -- 172.21.15.2:0/1559113072 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f6ef40f04c0 con 0x7f6efc102040 2024-03-15T12:58:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.754+0000 7f6efa59c640 1 --2- 172.21.15.2:0/1559113072 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6ee80a1a30 0x7f6ee80a3ef0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6efc1014e0 tx=0x7f6eec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:00.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.758+0000 7f6ef37fe640 1 -- 172.21.15.2:0/1559113072 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6ef40b9390 con 0x7f6efc102040 2024-03-15T12:58:00.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:00 smithi149 ceph-mon[35927]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-15T12:58:00.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:00 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/702049089' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-15T12:58:00.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.942+0000 7f6f0091e640 1 -- 172.21.15.2:0/1559113072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6efc107830 con 0x7f6efc102040 2024-03-15T12:58:00.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.944+0000 7f6ef37fe640 1 -- 172.21.15.2:0/1559113072 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24526 (secure 0 0 0) 0x7f6ef40bca50 con 0x7f6efc102040 2024-03-15T12:58:00.930 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:00.930 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":91,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:57:19.826817+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:58:00.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.946+0000 7f6f0091e640 1 -- 172.21.15.2:0/1559113072 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6ee80a1a30 msgr2=0x7f6ee80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:00.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.946+0000 7f6f0091e640 1 --2- 172.21.15.2:0/1559113072 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6ee80a1a30 0x7f6ee80a3ef0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6efc1014e0 tx=0x7f6eec005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:00.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.946+0000 7f6f0091e640 1 -- 172.21.15.2:0/1559113072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6efc102040 msgr2=0x7f6efc19e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:00.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.946+0000 7f6f0091e640 1 --2- 172.21.15.2:0/1559113072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6efc102040 0x7f6efc19e890 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4005cc0 tx=0x7f6ef4032a10 comp rx=0 tx=0).stop 2024-03-15T12:58:00.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.946+0000 7f6f0091e640 1 -- 172.21.15.2:0/1559113072 shutdown_connections 2024-03-15T12:58:00.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.946+0000 7f6f0091e640 1 --2- 172.21.15.2:0/1559113072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6efc102040 0x7f6efc19e890 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:00.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.946+0000 7f6f0091e640 1 --2- 172.21.15.2:0/1559113072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6efc101680 0x7f6efc19c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.946+0000 7f6f0091e640 1 --2- 172.21.15.2:0/1559113072 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6ee80a1a30 0x7f6ee80a3ef0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.946+0000 7f6f0091e640 1 --2- 172.21.15.2:0/1559113072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6efc100490 0x7f6efc06a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.946+0000 7f6f0091e640 1 -- 172.21.15.2:0/1559113072 >> 172.21.15.2:0/1559113072 conn(0x7f6efc0fba20 msgr2=0x7f6efc104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.946+0000 7f6f0091e640 1 -- 172.21.15.2:0/1559113072 shutdown_connections 2024-03-15T12:58:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:00.946+0000 7f6f0091e640 1 -- 172.21.15.2:0/1559113072 wait complete. 2024-03-15T12:58:00.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:00 smithi002 ceph-mon[32060]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-15T12:58:00.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:00 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/702049089' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-15T12:58:00.943 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-15T12:51:47.004843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-15T12:56:18.559882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-15T12:56:19.751767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-15T12:56:23.664879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-15T12:58:00.944 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-15T12:58:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.065+0000 7fc9b2747640 1 -- 172.21.15.2:0/1241674355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9ac1071f0 msgr2=0x7fc9ac109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.065+0000 7fc9b2747640 1 --2- 172.21.15.2:0/1241674355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9ac1071f0 0x7fc9ac109640 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009ca0 tx=0x7fc9a402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.065+0000 7fc9b2747640 1 -- 172.21.15.2:0/1241674355 shutdown_connections 2024-03-15T12:58:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.065+0000 7fc9b2747640 1 --2- 172.21.15.2:0/1241674355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9ac1071f0 0x7fc9ac109640 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.065+0000 7fc9b2747640 1 --2- 172.21.15.2:0/1241674355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9ac1048c0 0x7fc9ac106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.065+0000 7fc9b2747640 1 --2- 172.21.15.2:0/1241674355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac101f90 0x7fc9ac104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.065+0000 7fc9b2747640 1 -- 172.21.15.2:0/1241674355 >> 172.21.15.2:0/1241674355 conn(0x7fc9ac0fb9a0 msgr2=0x7fc9ac0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.065+0000 7fc9b2747640 1 -- 172.21.15.2:0/1241674355 shutdown_connections 2024-03-15T12:58:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.065+0000 7fc9b2747640 1 -- 172.21.15.2:0/1241674355 wait complete. 2024-03-15T12:58:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.065+0000 7fc9b2747640 1 Processor -- start 2024-03-15T12:58:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.065+0000 7fc9b2747640 1 -- start start 2024-03-15T12:58:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9b2747640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9ac101f90 0x7fc9ac197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9b2747640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9ac1048c0 0x7fc9ac197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9b2747640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac1071f0 0x7fc9ac19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9b2747640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac19f040 con 0x7fc9ac101f90 2024-03-15T12:58:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9b2747640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac19f1b0 con 0x7fc9ac1048c0 2024-03-15T12:58:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9b2747640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac19f370 con 0x7fc9ac1071f0 2024-03-15T12:58:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9ac101f90 0x7fc9ac197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9ac101f90 0x7fc9ac197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47584/0 (socket says 172.21.15.2:47584) 2024-03-15T12:58:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9abfff640 1 -- 172.21.15.2:0/1202321167 learned_addr learned my addr 172.21.15.2:0/1202321167 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9ab7fe640 1 --2- 172.21.15.2:0/1202321167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9ac1048c0 0x7fc9ac197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9b0ce4640 1 --2- 172.21.15.2:0/1202321167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac1071f0 0x7fc9ac19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9abfff640 1 -- 172.21.15.2:0/1202321167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac1071f0 msgr2=0x7fc9ac19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9abfff640 1 --2- 172.21.15.2:0/1202321167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac1071f0 0x7fc9ac19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9abfff640 1 -- 172.21.15.2:0/1202321167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9ac1048c0 msgr2=0x7fc9ac197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9abfff640 1 --2- 172.21.15.2:0/1202321167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9ac1048c0 0x7fc9ac197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9abfff640 1 -- 172.21.15.2:0/1202321167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a4009900 con 0x7fc9ac101f90 2024-03-15T12:58:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9abfff640 1 --2- 172.21.15.2:0/1202321167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9ac101f90 0x7fc9ac197550 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00e9f0 tx=0x7fc99c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9a97fa640 1 -- 172.21.15.2:0/1202321167 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c009700 con 0x7fc9ac101f90 2024-03-15T12:58:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9b2747640 1 -- 172.21.15.2:0/1202321167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ac19f650 con 0x7fc9ac101f90 2024-03-15T12:58:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9a97fa640 1 -- 172.21.15.2:0/1202321167 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc99c00cd80 con 0x7fc9ac101f90 2024-03-15T12:58:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.066+0000 7fc9a97fa640 1 -- 172.21.15.2:0/1202321167 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c018920 con 0x7fc9ac101f90 2024-03-15T12:58:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.067+0000 7fc9b2747640 1 -- 172.21.15.2:0/1202321167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ac19fb60 con 0x7fc9ac101f90 2024-03-15T12:58:01.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.068+0000 7fc9b2747640 1 -- 172.21.15.2:0/1202321167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ac0fd490 con 0x7fc9ac101f90 2024-03-15T12:58:01.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.069+0000 7fc9a97fa640 1 -- 172.21.15.2:0/1202321167 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fc99c018a80 con 0x7fc9ac101f90 2024-03-15T12:58:01.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.069+0000 7fc9a97fa640 1 --2- 172.21.15.2:0/1202321167 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fc9940a1a30 0x7fc9940a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.069+0000 7fc9ab7fe640 1 --2- 172.21.15.2:0/1202321167 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fc9940a1a30 0x7fc9940a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.069+0000 7fc9a97fa640 1 -- 172.21.15.2:0/1202321167 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fc99c014070 con 0x7fc9ac101f90 2024-03-15T12:58:01.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.070+0000 7fc9ab7fe640 1 --2- 172.21.15.2:0/1202321167 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fc9940a1a30 0x7fc9940a3ef0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac198a60 tx=0x7fc9a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:01.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:00 smithi184 ceph-mon[35993]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-15T12:58:01.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:00 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/702049089' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-15T12:58:01.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.074+0000 7fc9a97fa640 1 -- 172.21.15.2:0/1202321167 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc99c053280 con 0x7fc9ac101f90 2024-03-15T12:58:01.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.259+0000 7fc9b2747640 1 -- 172.21.15.2:0/1202321167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9ac02d000 con 0x7fc9ac101f90 2024-03-15T12:58:01.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.259+0000 7fc9a97fa640 1 -- 172.21.15.2:0/1202321167 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc99c0969d0 con 0x7fc9ac101f90 2024-03-15T12:58:01.246 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-15T12:58:01.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.262+0000 7fc9b2747640 1 -- 172.21.15.2:0/1202321167 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fc9940a1a30 msgr2=0x7fc9940a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.262+0000 7fc9b2747640 1 --2- 172.21.15.2:0/1202321167 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fc9940a1a30 0x7fc9940a3ef0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac198a60 tx=0x7fc9a0005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:01.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.262+0000 7fc9b2747640 1 -- 172.21.15.2:0/1202321167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9ac101f90 msgr2=0x7fc9ac197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.262+0000 7fc9b2747640 1 --2- 172.21.15.2:0/1202321167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9ac101f90 0x7fc9ac197550 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00e9f0 tx=0x7fc99c00eeb0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.262+0000 7fc9b2747640 1 -- 172.21.15.2:0/1202321167 shutdown_connections 2024-03-15T12:58:01.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.262+0000 7fc9b2747640 1 --2- 172.21.15.2:0/1202321167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac1071f0 0x7fc9ac19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.262+0000 7fc9b2747640 1 --2- 172.21.15.2:0/1202321167 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fc9940a1a30 0x7fc9940a3ef0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.262+0000 7fc9b2747640 1 --2- 172.21.15.2:0/1202321167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9ac1048c0 0x7fc9ac197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.262+0000 7fc9b2747640 1 --2- 172.21.15.2:0/1202321167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9ac101f90 0x7fc9ac197550 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.262+0000 7fc9b2747640 1 -- 172.21.15.2:0/1202321167 >> 172.21.15.2:0/1202321167 conn(0x7fc9ac0fb9a0 msgr2=0x7fc9ac102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:01.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.262+0000 7fc9b2747640 1 -- 172.21.15.2:0/1202321167 shutdown_connections 2024-03-15T12:58:01.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.262+0000 7fc9b2747640 1 -- 172.21.15.2:0/1202321167 wait complete. 2024-03-15T12:58:01.259 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-15T12:58:01.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.379+0000 7f18bdd93640 1 -- 172.21.15.2:0/1328415922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18b8101fa0 msgr2=0x7f18b8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.379+0000 7f18bdd93640 1 --2- 172.21.15.2:0/1328415922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18b8101fa0 0x7f18b8108830 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009ca0 tx=0x7f18b002c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.380+0000 7f18bdd93640 1 -- 172.21.15.2:0/1328415922 shutdown_connections 2024-03-15T12:58:01.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.380+0000 7f18bdd93640 1 --2- 172.21.15.2:0/1328415922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18b8101fa0 0x7f18b8108830 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.380+0000 7f18bdd93640 1 --2- 172.21.15.2:0/1328415922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b81015e0 0x7f18b8101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.380+0000 7f18bdd93640 1 --2- 172.21.15.2:0/1328415922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b81003f0 0x7f18b8100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.380+0000 7f18bdd93640 1 -- 172.21.15.2:0/1328415922 >> 172.21.15.2:0/1328415922 conn(0x7f18b80fb980 msgr2=0x7f18b80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:01.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.380+0000 7f18bdd93640 1 -- 172.21.15.2:0/1328415922 shutdown_connections 2024-03-15T12:58:01.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.380+0000 7f18bdd93640 1 -- 172.21.15.2:0/1328415922 wait complete. 2024-03-15T12:58:01.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.380+0000 7f18bdd93640 1 Processor -- start 2024-03-15T12:58:01.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.380+0000 7f18bdd93640 1 -- start start 2024-03-15T12:58:01.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.380+0000 7f18bdd93640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18b81003f0 0x7f18b8197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18bdd93640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b81015e0 0x7f18b8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18bdd93640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b8101fa0 0x7f18b819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18bdd93640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b819f060 con 0x7f18b81003f0 2024-03-15T12:58:01.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18bdd93640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b819f1d0 con 0x7f18b8101fa0 2024-03-15T12:58:01.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18bdd93640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b819f390 con 0x7f18b81015e0 2024-03-15T12:58:01.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18b77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18b81003f0 0x7f18b8197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18b6ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b81015e0 0x7f18b8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18b77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18b81003f0 0x7f18b8197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47608/0 (socket says 172.21.15.2:47608) 2024-03-15T12:58:01.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18b77fe640 1 -- 172.21.15.2:0/2510212992 learned_addr learned my addr 172.21.15.2:0/2510212992 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:01.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18b7fff640 1 --2- 172.21.15.2:0/2510212992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b8101fa0 0x7f18b819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18b6ffd640 1 -- 172.21.15.2:0/2510212992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b8101fa0 msgr2=0x7f18b819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18b6ffd640 1 --2- 172.21.15.2:0/2510212992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b8101fa0 0x7f18b819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18b6ffd640 1 -- 172.21.15.2:0/2510212992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18b81003f0 msgr2=0x7f18b8197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18b6ffd640 1 --2- 172.21.15.2:0/2510212992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18b81003f0 0x7f18b8197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18b6ffd640 1 -- 172.21.15.2:0/2510212992 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b0009900 con 0x7f18b81015e0 2024-03-15T12:58:01.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18b77fe640 1 --2- 172.21.15.2:0/2510212992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18b81003f0 0x7f18b8197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:01.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18b6ffd640 1 --2- 172.21.15.2:0/2510212992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b81015e0 0x7f18b8197ab0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f18a000e9f0 tx=0x7f18a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:01.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18b4ff9640 1 -- 172.21.15.2:0/2510212992 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a000ccc0 con 0x7f18b81015e0 2024-03-15T12:58:01.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18bdd93640 1 -- 172.21.15.2:0/2510212992 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b819f670 con 0x7f18b81015e0 2024-03-15T12:58:01.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18b4ff9640 1 -- 172.21.15.2:0/2510212992 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f18a000ce20 con 0x7f18b81015e0 2024-03-15T12:58:01.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18b4ff9640 1 -- 172.21.15.2:0/2510212992 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0018880 con 0x7f18b81015e0 2024-03-15T12:58:01.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.381+0000 7f18bdd93640 1 -- 172.21.15.2:0/2510212992 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b819fbd0 con 0x7f18b81015e0 2024-03-15T12:58:01.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.382+0000 7f18bdd93640 1 -- 172.21.15.2:0/2510212992 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b810bdd0 con 0x7f18b81015e0 2024-03-15T12:58:01.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.384+0000 7f18b4ff9640 1 -- 172.21.15.2:0/2510212992 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f18a00189e0 con 0x7f18b81015e0 2024-03-15T12:58:01.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.384+0000 7f18b4ff9640 1 --2- 172.21.15.2:0/2510212992 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f18a40a1a30 0x7f18a40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.384+0000 7f18b77fe640 1 --2- 172.21.15.2:0/2510212992 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f18a40a1a30 0x7f18a40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.384+0000 7f18b4ff9640 1 -- 172.21.15.2:0/2510212992 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f18a0014070 con 0x7f18b81015e0 2024-03-15T12:58:01.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.385+0000 7f18b77fe640 1 --2- 172.21.15.2:0/2510212992 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f18a40a1a30 0x7f18a40a3ef0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f18a80058c0 tx=0x7f18a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:01.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.388+0000 7f18b4ff9640 1 -- 172.21.15.2:0/2510212992 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f18a0096960 con 0x7f18b81015e0 2024-03-15T12:58:01.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.572+0000 7f18bdd93640 1 -- 172.21.15.2:0/2510212992 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f18b810bf60 con 0x7f18b81015e0 2024-03-15T12:58:01.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.572+0000 7f18b4ff9640 1 -- 172.21.15.2:0/2510212992 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f18a009a020 con 0x7f18b81015e0 2024-03-15T12:58:01.558 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:58:01.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.574+0000 7f18bdd93640 1 -- 172.21.15.2:0/2510212992 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f18a40a1a30 msgr2=0x7f18a40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.574+0000 7f18bdd93640 1 --2- 172.21.15.2:0/2510212992 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f18a40a1a30 0x7f18a40a3ef0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f18a80058c0 tx=0x7f18a8005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:01.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.574+0000 7f18bdd93640 1 -- 172.21.15.2:0/2510212992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b81015e0 msgr2=0x7f18b8197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.574+0000 7f18bdd93640 1 --2- 172.21.15.2:0/2510212992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b81015e0 0x7f18b8197ab0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f18a000e9f0 tx=0x7f18a000eeb0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.574+0000 7f18bdd93640 1 -- 172.21.15.2:0/2510212992 shutdown_connections 2024-03-15T12:58:01.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.574+0000 7f18bdd93640 1 --2- 172.21.15.2:0/2510212992 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f18a40a1a30 0x7f18a40a3ef0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.574+0000 7f18bdd93640 1 --2- 172.21.15.2:0/2510212992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18b8101fa0 0x7f18b819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.574+0000 7f18bdd93640 1 --2- 172.21.15.2:0/2510212992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b81015e0 0x7f18b8197ab0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.574+0000 7f18bdd93640 1 --2- 172.21.15.2:0/2510212992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18b81003f0 0x7f18b8197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.574+0000 7f18bdd93640 1 -- 172.21.15.2:0/2510212992 >> 172.21.15.2:0/2510212992 conn(0x7f18b80fb980 msgr2=0x7f18b80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:01.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.574+0000 7f18bdd93640 1 -- 172.21.15.2:0/2510212992 shutdown_connections 2024-03-15T12:58:01.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.574+0000 7f18bdd93640 1 -- 172.21.15.2:0/2510212992 wait complete. 2024-03-15T12:58:01.571 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-15T12:58:01.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.692+0000 7f475a3e4640 1 -- 172.21.15.2:0/4087354284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4754101fb0 msgr2=0x7f4754108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.692+0000 7f475a3e4640 1 --2- 172.21.15.2:0/4087354284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4754101fb0 0x7f4754108840 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f474c009ca0 tx=0x7f474c02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.692+0000 7f475a3e4640 1 -- 172.21.15.2:0/4087354284 shutdown_connections 2024-03-15T12:58:01.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.692+0000 7f475a3e4640 1 --2- 172.21.15.2:0/4087354284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4754101fb0 0x7f4754108840 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.692+0000 7f475a3e4640 1 --2- 172.21.15.2:0/4087354284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47541015f0 0x7f4754101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.692+0000 7f475a3e4640 1 --2- 172.21.15.2:0/4087354284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4754100400 0x7f4754100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.692+0000 7f475a3e4640 1 -- 172.21.15.2:0/4087354284 >> 172.21.15.2:0/4087354284 conn(0x7f47540fb990 msgr2=0x7f47540fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:01.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.692+0000 7f475a3e4640 1 -- 172.21.15.2:0/4087354284 shutdown_connections 2024-03-15T12:58:01.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.692+0000 7f475a3e4640 1 -- 172.21.15.2:0/4087354284 wait complete. 2024-03-15T12:58:01.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f475a3e4640 1 Processor -- start 2024-03-15T12:58:01.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f475a3e4640 1 -- start start 2024-03-15T12:58:01.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f475a3e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4754100400 0x7f4754197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f475a3e4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47541015f0 0x7f4754197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f475a3e4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4754101fb0 0x7f475419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f475a3e4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475419f020 con 0x7f47541015f0 2024-03-15T12:58:01.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f475a3e4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475419f190 con 0x7f4754101fb0 2024-03-15T12:58:01.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f475a3e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475419f350 con 0x7f4754100400 2024-03-15T12:58:01.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f47537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47541015f0 0x7f4754197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f47537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47541015f0 0x7f4754197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47630/0 (socket says 172.21.15.2:47630) 2024-03-15T12:58:01.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f4753fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4754100400 0x7f4754197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f47537fe640 1 -- 172.21.15.2:0/3904700997 learned_addr learned my addr 172.21.15.2:0/3904700997 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:01.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f4758981640 1 --2- 172.21.15.2:0/3904700997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4754101fb0 0x7f475419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f47537fe640 1 -- 172.21.15.2:0/3904700997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4754100400 msgr2=0x7f4754197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f47537fe640 1 --2- 172.21.15.2:0/3904700997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4754100400 0x7f4754197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f47537fe640 1 -- 172.21.15.2:0/3904700997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4754101fb0 msgr2=0x7f475419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f47537fe640 1 --2- 172.21.15.2:0/3904700997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4754101fb0 0x7f475419eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f47537fe640 1 -- 172.21.15.2:0/3904700997 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4748009490 con 0x7f47541015f0 2024-03-15T12:58:01.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f4753fff640 1 --2- 172.21.15.2:0/3904700997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4754100400 0x7f4754197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:01.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.693+0000 7f47537fe640 1 --2- 172.21.15.2:0/3904700997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47541015f0 0x7f4754197a70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f474800ea60 tx=0x7f474800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:01.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.694+0000 7f47517fa640 1 -- 172.21.15.2:0/3904700997 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474800cde0 con 0x7f47541015f0 2024-03-15T12:58:01.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.694+0000 7f475a3e4640 1 -- 172.21.15.2:0/3904700997 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f474c009900 con 0x7f47541015f0 2024-03-15T12:58:01.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.694+0000 7f47517fa640 1 -- 172.21.15.2:0/3904700997 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f47480035f0 con 0x7f47541015f0 2024-03-15T12:58:01.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.694+0000 7f47517fa640 1 -- 172.21.15.2:0/3904700997 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748010690 con 0x7f47541015f0 2024-03-15T12:58:01.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.694+0000 7f475a3e4640 1 -- 172.21.15.2:0/3904700997 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475419f9b0 con 0x7f47541015f0 2024-03-15T12:58:01.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.695+0000 7f475a3e4640 1 -- 172.21.15.2:0/3904700997 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475410bde0 con 0x7f47541015f0 2024-03-15T12:58:01.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.696+0000 7f47517fa640 1 -- 172.21.15.2:0/3904700997 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f4748003760 con 0x7f47541015f0 2024-03-15T12:58:01.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.696+0000 7f47517fa640 1 --2- 172.21.15.2:0/3904700997 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f473c0a1a30 0x7f473c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.696+0000 7f47517fa640 1 -- 172.21.15.2:0/3904700997 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f4748014070 con 0x7f47541015f0 2024-03-15T12:58:01.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.696+0000 7f4753fff640 1 --2- 172.21.15.2:0/3904700997 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f473c0a1a30 0x7f473c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.697+0000 7f4753fff640 1 --2- 172.21.15.2:0/3904700997 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f473c0a1a30 0x7f473c0a3ef0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4754101450 tx=0x7f4744005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:01.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.700+0000 7f47517fa640 1 -- 172.21.15.2:0/3904700997 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4748096c20 con 0x7f47541015f0 2024-03-15T12:58:01.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:01 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1559113072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:01.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:01 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1202321167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:01.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:01 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2510212992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:01.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.882+0000 7f475a3e4640 1 -- 172.21.15.2:0/3904700997 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f475419fed0 con 0x7f47541015f0 2024-03-15T12:58:01.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.883+0000 7f47517fa640 1 -- 172.21.15.2:0/3904700997 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f474809a2e0 con 0x7f47541015f0 2024-03-15T12:58:01.869 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-03-15T12:58:01.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.885+0000 7f475a3e4640 1 -- 172.21.15.2:0/3904700997 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f473c0a1a30 msgr2=0x7f473c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.885+0000 7f475a3e4640 1 --2- 172.21.15.2:0/3904700997 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f473c0a1a30 0x7f473c0a3ef0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4754101450 tx=0x7f4744005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:01.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.885+0000 7f475a3e4640 1 -- 172.21.15.2:0/3904700997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47541015f0 msgr2=0x7f4754197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.885+0000 7f475a3e4640 1 --2- 172.21.15.2:0/3904700997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47541015f0 0x7f4754197a70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f474800ea60 tx=0x7f474800ef20 comp rx=0 tx=0).stop 2024-03-15T12:58:01.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.885+0000 7f475a3e4640 1 -- 172.21.15.2:0/3904700997 shutdown_connections 2024-03-15T12:58:01.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.885+0000 7f475a3e4640 1 --2- 172.21.15.2:0/3904700997 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f473c0a1a30 0x7f473c0a3ef0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.885+0000 7f475a3e4640 1 --2- 172.21.15.2:0/3904700997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4754101fb0 0x7f475419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.885+0000 7f475a3e4640 1 --2- 172.21.15.2:0/3904700997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47541015f0 0x7f4754197a70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.885+0000 7f475a3e4640 1 --2- 172.21.15.2:0/3904700997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4754100400 0x7f4754197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.885+0000 7f475a3e4640 1 -- 172.21.15.2:0/3904700997 >> 172.21.15.2:0/3904700997 conn(0x7f47540fb990 msgr2=0x7f47540fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:01.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.885+0000 7f475a3e4640 1 -- 172.21.15.2:0/3904700997 shutdown_connections 2024-03-15T12:58:01.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:01.885+0000 7f475a3e4640 1 -- 172.21.15.2:0/3904700997 wait complete. 2024-03-15T12:58:01.881 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:01 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1559113072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:01.881 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:01 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1202321167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:01.881 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:01 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2510212992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:01.882 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-15T12:58:01.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.002+0000 7f9133655640 1 -- 172.21.15.2:0/447147894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f912c101fa0 msgr2=0x7f912c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.002+0000 7f9133655640 1 --2- 172.21.15.2:0/447147894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f912c101fa0 0x7f912c108830 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f911c009ca0 tx=0x7f911c02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.002+0000 7f9133655640 1 -- 172.21.15.2:0/447147894 shutdown_connections 2024-03-15T12:58:01.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.002+0000 7f9133655640 1 --2- 172.21.15.2:0/447147894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f912c101fa0 0x7f912c108830 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.002+0000 7f9133655640 1 --2- 172.21.15.2:0/447147894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f912c1015e0 0x7f912c101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.002+0000 7f9133655640 1 --2- 172.21.15.2:0/447147894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912c1003f0 0x7f912c100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.002+0000 7f9133655640 1 -- 172.21.15.2:0/447147894 >> 172.21.15.2:0/447147894 conn(0x7f912c0fb980 msgr2=0x7f912c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.002+0000 7f9133655640 1 -- 172.21.15.2:0/447147894 shutdown_connections 2024-03-15T12:58:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.002+0000 7f9133655640 1 -- 172.21.15.2:0/447147894 wait complete. 2024-03-15T12:58:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.002+0000 7f9133655640 1 Processor -- start 2024-03-15T12:58:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.002+0000 7f9133655640 1 -- start start 2024-03-15T12:58:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f9133655640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f912c1003f0 0x7f912c197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f9133655640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912c1015e0 0x7f912c197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f9133655640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f912c101fa0 0x7f912c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f9133655640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c19f030 con 0x7f912c1003f0 2024-03-15T12:58:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f9133655640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c19f1a0 con 0x7f912c101fa0 2024-03-15T12:58:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f9133655640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c19f360 con 0x7f912c1015e0 2024-03-15T12:58:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f91313f1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f912c1003f0 0x7f912c197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f91313f1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f912c1003f0 0x7f912c197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47656/0 (socket says 172.21.15.2:47656) 2024-03-15T12:58:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f91313f1640 1 -- 172.21.15.2:0/3633180225 learned_addr learned my addr 172.21.15.2:0/3633180225 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f9131bf2640 1 --2- 172.21.15.2:0/3633180225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f912c101fa0 0x7f912c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f9130bf0640 1 --2- 172.21.15.2:0/3633180225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912c1015e0 0x7f912c197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f91313f1640 1 -- 172.21.15.2:0/3633180225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912c1015e0 msgr2=0x7f912c197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f91313f1640 1 --2- 172.21.15.2:0/3633180225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912c1015e0 0x7f912c197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f91313f1640 1 -- 172.21.15.2:0/3633180225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f912c101fa0 msgr2=0x7f912c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:01.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f91313f1640 1 --2- 172.21.15.2:0/3633180225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f912c101fa0 0x7f912c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:01.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f91313f1640 1 -- 172.21.15.2:0/3633180225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9120009490 con 0x7f912c1003f0 2024-03-15T12:58:01.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f91313f1640 1 --2- 172.21.15.2:0/3633180225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f912c1003f0 0x7f912c197540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f912400e9c0 tx=0x7f912400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:01.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f912a7fc640 1 -- 172.21.15.2:0/3633180225 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9124009700 con 0x7f912c1003f0 2024-03-15T12:58:01.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f9133655640 1 -- 172.21.15.2:0/3633180225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911c009900 con 0x7f912c1003f0 2024-03-15T12:58:01.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f912a7fc640 1 -- 172.21.15.2:0/3633180225 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f91240044f0 con 0x7f912c1003f0 2024-03-15T12:58:01.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f912a7fc640 1 -- 172.21.15.2:0/3633180225 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9124010430 con 0x7f912c1003f0 2024-03-15T12:58:01.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.003+0000 7f9133655640 1 -- 172.21.15.2:0/3633180225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c19f9c0 con 0x7f912c1003f0 2024-03-15T12:58:01.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.004+0000 7f9133655640 1 -- 172.21.15.2:0/3633180225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912c10bdd0 con 0x7f912c1003f0 2024-03-15T12:58:01.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.005+0000 7f912a7fc640 1 -- 172.21.15.2:0/3633180225 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f9124003fe0 con 0x7f912c1003f0 2024-03-15T12:58:01.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.006+0000 7f912a7fc640 1 --2- 172.21.15.2:0/3633180225 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f91140a1a30 0x7f91140a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:01.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.006+0000 7f9130bf0640 1 --2- 172.21.15.2:0/3633180225 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f91140a1a30 0x7f91140a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:01.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.006+0000 7f912a7fc640 1 -- 172.21.15.2:0/3633180225 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f9124014070 con 0x7f912c1003f0 2024-03-15T12:58:01.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.006+0000 7f9130bf0640 1 --2- 172.21.15.2:0/3633180225 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f91140a1a30 0x7f91140a3ef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f912c198a50 tx=0x7f9120005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:01.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.010+0000 7f912a7fc640 1 -- 172.21.15.2:0/3633180225 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f91240969f0 con 0x7f912c1003f0 2024-03-15T12:58:02.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:01 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1559113072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:02.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:01 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1202321167' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:02.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:01 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2510212992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:02.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.193+0000 7f9133655640 1 -- 172.21.15.2:0/3633180225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f912c19fc40 con 0x7f912c1003f0 2024-03-15T12:58:02.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.194+0000 7f912a7fc640 1 -- 172.21.15.2:0/3633180225 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f912409a0b0 con 0x7f912c1003f0 2024-03-15T12:58:02.180 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:58:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.196+0000 7f9133655640 1 -- 172.21.15.2:0/3633180225 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f91140a1a30 msgr2=0x7f91140a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.196+0000 7f9133655640 1 --2- 172.21.15.2:0/3633180225 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f91140a1a30 0x7f91140a3ef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f912c198a50 tx=0x7f9120005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.196+0000 7f9133655640 1 -- 172.21.15.2:0/3633180225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f912c1003f0 msgr2=0x7f912c197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.196+0000 7f9133655640 1 --2- 172.21.15.2:0/3633180225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f912c1003f0 0x7f912c197540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f912400e9c0 tx=0x7f912400ee80 comp rx=0 tx=0).stop 2024-03-15T12:58:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.196+0000 7f9133655640 1 -- 172.21.15.2:0/3633180225 shutdown_connections 2024-03-15T12:58:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.196+0000 7f9133655640 1 --2- 172.21.15.2:0/3633180225 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f91140a1a30 0x7f91140a3ef0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.196+0000 7f9133655640 1 --2- 172.21.15.2:0/3633180225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f912c101fa0 0x7f912c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.196+0000 7f9133655640 1 --2- 172.21.15.2:0/3633180225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f912c1015e0 0x7f912c197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.196+0000 7f9133655640 1 --2- 172.21.15.2:0/3633180225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f912c1003f0 0x7f912c197540 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.196+0000 7f9133655640 1 -- 172.21.15.2:0/3633180225 >> 172.21.15.2:0/3633180225 conn(0x7f912c0fb980 msgr2=0x7f912c0fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:02.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.196+0000 7f9133655640 1 -- 172.21.15.2:0/3633180225 shutdown_connections 2024-03-15T12:58:02.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.196+0000 7f9133655640 1 -- 172.21.15.2:0/3633180225 wait complete. 2024-03-15T12:58:02.193 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:58:02.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.310+0000 7f3d1212c640 1 -- 172.21.15.2:0/2925896237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d0c102010 msgr2=0x7f3d0c104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:02.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.310+0000 7f3d1212c640 1 --2- 172.21.15.2:0/2925896237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d0c102010 0x7f3d0c104400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009900 tx=0x7f3cfc02bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:02.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.310+0000 7f3d1212c640 1 -- 172.21.15.2:0/2925896237 shutdown_connections 2024-03-15T12:58:02.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.310+0000 7f3d1212c640 1 --2- 172.21.15.2:0/2925896237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d0c107270 0x7f3d0c1096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.310+0000 7f3d1212c640 1 --2- 172.21.15.2:0/2925896237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d0c104940 0x7f3d0c106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.310+0000 7f3d1212c640 1 --2- 172.21.15.2:0/2925896237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d0c102010 0x7f3d0c104400 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.310+0000 7f3d1212c640 1 -- 172.21.15.2:0/2925896237 >> 172.21.15.2:0/2925896237 conn(0x7f3d0c0fba20 msgr2=0x7f3d0c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:02.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.310+0000 7f3d1212c640 1 -- 172.21.15.2:0/2925896237 shutdown_connections 2024-03-15T12:58:02.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.310+0000 7f3d1212c640 1 -- 172.21.15.2:0/2925896237 wait complete. 2024-03-15T12:58:02.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.311+0000 7f3d1212c640 1 Processor -- start 2024-03-15T12:58:02.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.311+0000 7f3d1212c640 1 -- start start 2024-03-15T12:58:02.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.311+0000 7f3d1212c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d0c102010 0x7f3d0c1975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:02.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.311+0000 7f3d1212c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d0c104940 0x7f3d0c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:02.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.311+0000 7f3d1212c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d0c107270 0x7f3d0c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:02.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.311+0000 7f3d1212c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c19f0c0 con 0x7f3d0c107270 2024-03-15T12:58:02.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.311+0000 7f3d1212c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c19f230 con 0x7f3d0c102010 2024-03-15T12:58:02.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.311+0000 7f3d1212c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c19f3f0 con 0x7f3d0c104940 2024-03-15T12:58:02.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.311+0000 7f3d0bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d0c107270 0x7f3d0c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:02.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.311+0000 7f3d0b7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d0c102010 0x7f3d0c1975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:02.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.311+0000 7f3d0bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d0c107270 0x7f3d0c19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47686/0 (socket says 172.21.15.2:47686) 2024-03-15T12:58:02.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.311+0000 7f3d0bfff640 1 -- 172.21.15.2:0/3962534716 learned_addr learned my addr 172.21.15.2:0/3962534716 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:02.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.311+0000 7f3d0affd640 1 --2- 172.21.15.2:0/3962534716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d0c104940 0x7f3d0c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:02.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.312+0000 7f3d0bfff640 1 -- 172.21.15.2:0/3962534716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d0c104940 msgr2=0x7f3d0c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:02.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.312+0000 7f3d0bfff640 1 --2- 172.21.15.2:0/3962534716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d0c104940 0x7f3d0c197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.312+0000 7f3d0bfff640 1 -- 172.21.15.2:0/3962534716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d0c102010 msgr2=0x7f3d0c1975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:02.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.312+0000 7f3d0bfff640 1 --2- 172.21.15.2:0/3962534716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d0c102010 0x7f3d0c1975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.312+0000 7f3d0bfff640 1 -- 172.21.15.2:0/3962534716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc009560 con 0x7f3d0c107270 2024-03-15T12:58:02.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.312+0000 7f3d0affd640 1 --2- 172.21.15.2:0/3962534716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d0c104940 0x7f3d0c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:02.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.312+0000 7f3d0bfff640 1 --2- 172.21.15.2:0/3962534716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d0c107270 0x7f3d0c19eb80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400eb10 tx=0x7f3d0400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:02.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.312+0000 7f3d08ff9640 1 -- 172.21.15.2:0/3962534716 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d04004b20 con 0x7f3d0c107270 2024-03-15T12:58:02.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.312+0000 7f3d08ff9640 1 -- 172.21.15.2:0/3962534716 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3d04004c80 con 0x7f3d0c107270 2024-03-15T12:58:02.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.312+0000 7f3d1212c640 1 -- 172.21.15.2:0/3962534716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0c19f6d0 con 0x7f3d0c107270 2024-03-15T12:58:02.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.312+0000 7f3d08ff9640 1 -- 172.21.15.2:0/3962534716 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d04010690 con 0x7f3d0c107270 2024-03-15T12:58:02.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.312+0000 7f3d1212c640 1 -- 172.21.15.2:0/3962534716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0c19fbe0 con 0x7f3d0c107270 2024-03-15T12:58:02.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.313+0000 7f3d1212c640 1 -- 172.21.15.2:0/3962534716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0c0fd510 con 0x7f3d0c107270 2024-03-15T12:58:02.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.314+0000 7f3d08ff9640 1 -- 172.21.15.2:0/3962534716 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f3d04004df0 con 0x7f3d0c107270 2024-03-15T12:58:02.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.315+0000 7f3d08ff9640 1 --2- 172.21.15.2:0/3962534716 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3cf40a1ab0 0x7f3cf40a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:02.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.315+0000 7f3d0b7fe640 1 --2- 172.21.15.2:0/3962534716 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3cf40a1ab0 0x7f3cf40a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:02.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.315+0000 7f3d08ff9640 1 -- 172.21.15.2:0/3962534716 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f3d04014070 con 0x7f3d0c107270 2024-03-15T12:58:02.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.315+0000 7f3d0b7fe640 1 --2- 172.21.15.2:0/3962534716 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3cf40a1ab0 0x7f3cf40a3f70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.319+0000 7f3d08ff9640 1 -- 172.21.15.2:0/3962534716 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3d040cfe60 con 0x7f3d0c107270 2024-03-15T12:58:02.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.542+0000 7f3d1212c640 1 -- 172.21.15.2:0/3962534716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d0c02d000 con 0x7f3d0c107270 2024-03-15T12:58:02.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.543+0000 7f3d08ff9640 1 -- 172.21.15.2:0/3962534716 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11453 (secure 0 0 0) 0x7f3d04096b30 con 0x7f3d0c107270 2024-03-15T12:58:02.529 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:02.530 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T12:56:43.433836+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24464,24500,24460,24539,24434]},"id":1}]} 2024-03-15T12:58:02.530 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 28 2024-03-15T12:58:02.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.545+0000 7f3d1212c640 1 -- 172.21.15.2:0/3962534716 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3cf40a1ab0 msgr2=0x7f3cf40a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:02.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.545+0000 7f3d1212c640 1 --2- 172.21.15.2:0/3962534716 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3cf40a1ab0 0x7f3cf40a3f70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc005fd0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.545+0000 7f3d1212c640 1 -- 172.21.15.2:0/3962534716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d0c107270 msgr2=0x7f3d0c19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:02.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.545+0000 7f3d1212c640 1 --2- 172.21.15.2:0/3962534716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d0c107270 0x7f3d0c19eb80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400eb10 tx=0x7f3d0400c2d0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.545+0000 7f3d1212c640 1 -- 172.21.15.2:0/3962534716 shutdown_connections 2024-03-15T12:58:02.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.545+0000 7f3d1212c640 1 --2- 172.21.15.2:0/3962534716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d0c107270 0x7f3d0c19eb80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.545+0000 7f3d1212c640 1 --2- 172.21.15.2:0/3962534716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d0c104940 0x7f3d0c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.545+0000 7f3d1212c640 1 --2- 172.21.15.2:0/3962534716 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3cf40a1ab0 0x7f3cf40a3f70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.545+0000 7f3d1212c640 1 --2- 172.21.15.2:0/3962534716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d0c102010 0x7f3d0c1975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.545+0000 7f3d1212c640 1 -- 172.21.15.2:0/3962534716 >> 172.21.15.2:0/3962534716 conn(0x7f3d0c0fba20 msgr2=0x7f3d0c102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:02.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.545+0000 7f3d1212c640 1 -- 172.21.15.2:0/3962534716 shutdown_connections 2024-03-15T12:58:02.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.545+0000 7f3d1212c640 1 -- 172.21.15.2:0/3962534716 wait complete. 2024-03-15T12:58:02.544 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:58:02.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.662+0000 7f45359f6640 1 -- 172.21.15.2:0/1960370845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530102040 msgr2=0x7f45301088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:02.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.662+0000 7f45359f6640 1 --2- 172.21.15.2:0/1960370845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530102040 0x7f45301088d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4528009ca0 tx=0x7f452802c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.662+0000 7f45359f6640 1 -- 172.21.15.2:0/1960370845 shutdown_connections 2024-03-15T12:58:02.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.662+0000 7f45359f6640 1 --2- 172.21.15.2:0/1960370845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530102040 0x7f45301088d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.662+0000 7f45359f6640 1 --2- 172.21.15.2:0/1960370845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4530101680 0x7f4530101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.662+0000 7f45359f6640 1 --2- 172.21.15.2:0/1960370845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4530100490 0x7f45301008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.662+0000 7f45359f6640 1 -- 172.21.15.2:0/1960370845 >> 172.21.15.2:0/1960370845 conn(0x7f45300fba20 msgr2=0x7f45300fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:02.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.662+0000 7f45359f6640 1 -- 172.21.15.2:0/1960370845 shutdown_connections 2024-03-15T12:58:02.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.662+0000 7f45359f6640 1 -- 172.21.15.2:0/1960370845 wait complete. 2024-03-15T12:58:02.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.662+0000 7f45359f6640 1 Processor -- start 2024-03-15T12:58:02.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.662+0000 7f45359f6640 1 -- start start 2024-03-15T12:58:02.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f45359f6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4530100490 0x7f45301975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:02.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f45359f6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4530101680 0x7f4530197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:02.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f45359f6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530102040 0x7f453019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:02.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f45359f6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453019f060 con 0x7f4530102040 2024-03-15T12:58:02.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f45359f6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453019f1d0 con 0x7f4530100490 2024-03-15T12:58:02.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f45359f6640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453019f390 con 0x7f4530101680 2024-03-15T12:58:02.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f452f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530102040 0x7f453019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:02.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f452f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530102040 0x7f453019eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47704/0 (socket says 172.21.15.2:47704) 2024-03-15T12:58:02.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f452f7fe640 1 -- 172.21.15.2:0/2962453771 learned_addr learned my addr 172.21.15.2:0/2962453771 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:02.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f452effd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4530100490 0x7f45301975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:02.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f4527fff640 1 --2- 172.21.15.2:0/2962453771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4530101680 0x7f4530197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:02.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f452f7fe640 1 -- 172.21.15.2:0/2962453771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4530101680 msgr2=0x7f4530197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:02.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f452f7fe640 1 --2- 172.21.15.2:0/2962453771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4530101680 0x7f4530197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f452f7fe640 1 -- 172.21.15.2:0/2962453771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4530100490 msgr2=0x7f45301975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:02.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f452f7fe640 1 --2- 172.21.15.2:0/2962453771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4530100490 0x7f45301975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f452f7fe640 1 -- 172.21.15.2:0/2962453771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528009900 con 0x7f4530102040 2024-03-15T12:58:02.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f452f7fe640 1 --2- 172.21.15.2:0/2962453771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530102040 0x7f453019eb20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4528005cc0 tx=0x7f4528032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:02.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f45349f4640 1 -- 172.21.15.2:0/2962453771 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4528034c90 con 0x7f4530102040 2024-03-15T12:58:02.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f45349f4640 1 -- 172.21.15.2:0/2962453771 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4528003e40 con 0x7f4530102040 2024-03-15T12:58:02.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f45349f4640 1 -- 172.21.15.2:0/2962453771 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45280048a0 con 0x7f4530102040 2024-03-15T12:58:02.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.663+0000 7f45359f6640 1 -- 172.21.15.2:0/2962453771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453019f610 con 0x7f4530102040 2024-03-15T12:58:02.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.664+0000 7f45359f6640 1 -- 172.21.15.2:0/2962453771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453019fa70 con 0x7f4530102040 2024-03-15T12:58:02.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.664+0000 7f45359f6640 1 -- 172.21.15.2:0/2962453771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453010be00 con 0x7f4530102040 2024-03-15T12:58:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.666+0000 7f45349f4640 1 -- 172.21.15.2:0/2962453771 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f452800a040 con 0x7f4530102040 2024-03-15T12:58:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.667+0000 7f45349f4640 1 --2- 172.21.15.2:0/2962453771 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f451c0a1a30 0x7f451c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.667+0000 7f452effd640 1 --2- 172.21.15.2:0/2962453771 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f451c0a1a30 0x7f451c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:02.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.667+0000 7f45349f4640 1 -- 172.21.15.2:0/2962453771 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f45280f08e0 con 0x7f4530102040 2024-03-15T12:58:02.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.667+0000 7f452effd640 1 --2- 172.21.15.2:0/2962453771 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f451c0a1a30 0x7f451c0a3ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f45301014e0 tx=0x7f4520005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:02.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.670+0000 7f45349f4640 1 -- 172.21.15.2:0/2962453771 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f45280b9750 con 0x7f4530102040 2024-03-15T12:58:02.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:02 smithi002 ceph-mon[32060]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-15T12:58:02.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:02 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3904700997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:02.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:02 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3633180225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:02.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:02 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3962534716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:02.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.854+0000 7f45359f6640 1 -- 172.21.15.2:0/2962453771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f453019ff50 con 0x7f4530102040 2024-03-15T12:58:02.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.856+0000 7f45349f4640 1 -- 172.21.15.2:0/2962453771 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24526 (secure 0 0 0) 0x7f45280bce10 con 0x7f4530102040 2024-03-15T12:58:02.842 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:02.842 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":91,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:57:19.826817+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:58:02.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.858+0000 7f45359f6640 1 -- 172.21.15.2:0/2962453771 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f451c0a1a30 msgr2=0x7f451c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:02.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.858+0000 7f45359f6640 1 --2- 172.21.15.2:0/2962453771 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f451c0a1a30 0x7f451c0a3ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f45301014e0 tx=0x7f4520005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:02.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.858+0000 7f45359f6640 1 -- 172.21.15.2:0/2962453771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530102040 msgr2=0x7f453019eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:02.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.858+0000 7f45359f6640 1 --2- 172.21.15.2:0/2962453771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530102040 0x7f453019eb20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4528005cc0 tx=0x7f4528032a10 comp rx=0 tx=0).stop 2024-03-15T12:58:02.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.858+0000 7f45359f6640 1 -- 172.21.15.2:0/2962453771 shutdown_connections 2024-03-15T12:58:02.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.858+0000 7f45359f6640 1 --2- 172.21.15.2:0/2962453771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4530102040 0x7f453019eb20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.858+0000 7f45359f6640 1 --2- 172.21.15.2:0/2962453771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4530101680 0x7f4530197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.858+0000 7f45359f6640 1 --2- 172.21.15.2:0/2962453771 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f451c0a1a30 0x7f451c0a3ef0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.858+0000 7f45359f6640 1 --2- 172.21.15.2:0/2962453771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4530100490 0x7f45301975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.858+0000 7f45359f6640 1 -- 172.21.15.2:0/2962453771 >> 172.21.15.2:0/2962453771 conn(0x7f45300fba20 msgr2=0x7f45300fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:02.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.858+0000 7f45359f6640 1 -- 172.21.15.2:0/2962453771 shutdown_connections 2024-03-15T12:58:02.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.858+0000 7f45359f6640 1 -- 172.21.15.2:0/2962453771 wait complete. 2024-03-15T12:58:02.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-15T12:51:47.004843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-15T12:56:18.559882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-15T12:56:19.751767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-15T12:56:23.664879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-15T12:58:02.856 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-15T12:58:02.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:02 smithi149 ceph-mon[35927]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-15T12:58:02.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:02 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3904700997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:02.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:02 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3633180225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:02.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:02 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3962534716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:02.960 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:02 smithi184 ceph-mon[35993]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-15T12:58:02.960 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:02 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3904700997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:02.960 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:02 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3633180225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:02.960 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:02 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3962534716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:02.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.975+0000 7f09d33a2640 1 -- 172.21.15.2:0/1233045452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09cc1002e0 msgr2=0x7f09cc100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:02.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.975+0000 7f09d33a2640 1 --2- 172.21.15.2:0/1233045452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09cc1002e0 0x7f09cc100700 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009900 tx=0x7f09c802bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:02.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.975+0000 7f09d33a2640 1 -- 172.21.15.2:0/1233045452 shutdown_connections 2024-03-15T12:58:02.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.975+0000 7f09d33a2640 1 --2- 172.21.15.2:0/1233045452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09cc101e90 0x7f09cc108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.975+0000 7f09d33a2640 1 --2- 172.21.15.2:0/1233045452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09cc1014d0 0x7f09cc101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.975+0000 7f09d33a2640 1 --2- 172.21.15.2:0/1233045452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09cc1002e0 0x7f09cc100700 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.975+0000 7f09d33a2640 1 -- 172.21.15.2:0/1233045452 >> 172.21.15.2:0/1233045452 conn(0x7f09cc0fb870 msgr2=0x7f09cc0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:02.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.975+0000 7f09d33a2640 1 -- 172.21.15.2:0/1233045452 shutdown_connections 2024-03-15T12:58:02.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.975+0000 7f09d33a2640 1 -- 172.21.15.2:0/1233045452 wait complete. 2024-03-15T12:58:02.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d33a2640 1 Processor -- start 2024-03-15T12:58:02.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d33a2640 1 -- start start 2024-03-15T12:58:02.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d33a2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09cc1002e0 0x7f09cc1974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:02.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d33a2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09cc1014d0 0x7f09cc1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:02.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d33a2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09cc101e90 0x7f09cc19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:02.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d33a2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc19efa0 con 0x7f09cc1002e0 2024-03-15T12:58:02.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d33a2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc19f110 con 0x7f09cc101e90 2024-03-15T12:58:02.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d33a2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc19f2d0 con 0x7f09cc1014d0 2024-03-15T12:58:02.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d23a0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09cc1002e0 0x7f09cc1974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:02.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d23a0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09cc1002e0 0x7f09cc1974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47726/0 (socket says 172.21.15.2:47726) 2024-03-15T12:58:02.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d23a0640 1 -- 172.21.15.2:0/2147795103 learned_addr learned my addr 172.21.15.2:0/2147795103 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:02.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d1b9f640 1 --2- 172.21.15.2:0/2147795103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09cc1014d0 0x7f09cc1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:02.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d2ba1640 1 --2- 172.21.15.2:0/2147795103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09cc101e90 0x7f09cc19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:02.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d23a0640 1 -- 172.21.15.2:0/2147795103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09cc1014d0 msgr2=0x7f09cc1979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:02.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d23a0640 1 --2- 172.21.15.2:0/2147795103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09cc1014d0 0x7f09cc1979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d23a0640 1 -- 172.21.15.2:0/2147795103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09cc101e90 msgr2=0x7f09cc19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:02.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d23a0640 1 --2- 172.21.15.2:0/2147795103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09cc101e90 0x7f09cc19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:02.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d23a0640 1 -- 172.21.15.2:0/2147795103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c8009560 con 0x7f09cc1002e0 2024-03-15T12:58:02.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.976+0000 7f09d2ba1640 1 --2- 172.21.15.2:0/2147795103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09cc101e90 0x7f09cc19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:02.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.977+0000 7f09d23a0640 1 --2- 172.21.15.2:0/2147795103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09cc1002e0 0x7f09cc1974b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f09c8000c00 tx=0x7f09c80049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:02.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.977+0000 7f09c37fe640 1 -- 172.21.15.2:0/2147795103 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8004250 con 0x7f09cc1002e0 2024-03-15T12:58:02.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.977+0000 7f09d33a2640 1 -- 172.21.15.2:0/2147795103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09cc19f550 con 0x7f09cc1002e0 2024-03-15T12:58:02.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.977+0000 7f09c37fe640 1 -- 172.21.15.2:0/2147795103 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f09c80043b0 con 0x7f09cc1002e0 2024-03-15T12:58:02.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.977+0000 7f09c37fe640 1 -- 172.21.15.2:0/2147795103 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8003d50 con 0x7f09cc1002e0 2024-03-15T12:58:02.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.977+0000 7f09d33a2640 1 -- 172.21.15.2:0/2147795103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc19fa60 con 0x7f09cc1002e0 2024-03-15T12:58:02.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.978+0000 7f09d33a2640 1 -- 172.21.15.2:0/2147795103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09cc10bcc0 con 0x7f09cc1002e0 2024-03-15T12:58:02.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.979+0000 7f09c37fe640 1 -- 172.21.15.2:0/2147795103 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f09c8005000 con 0x7f09cc1002e0 2024-03-15T12:58:02.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.979+0000 7f09c37fe640 1 --2- 172.21.15.2:0/2147795103 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f09b80a1a30 0x7f09b80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:02.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.979+0000 7f09d1b9f640 1 --2- 172.21.15.2:0/2147795103 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f09b80a1a30 0x7f09b80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:02.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.979+0000 7f09c37fe640 1 -- 172.21.15.2:0/2147795103 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f09c80ef930 con 0x7f09cc1002e0 2024-03-15T12:58:02.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.980+0000 7f09d1b9f640 1 --2- 172.21.15.2:0/2147795103 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f09b80a1a30 0x7f09b80a3ef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f09cc1989c0 tx=0x7f09bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:02.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:02.984+0000 7f09c37fe640 1 -- 172.21.15.2:0/2147795103 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f09c80b8800 con 0x7f09cc1002e0 2024-03-15T12:58:03.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.164+0000 7f09d33a2640 1 -- 172.21.15.2:0/2147795103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09cc19fe40 con 0x7f09cc1002e0 2024-03-15T12:58:03.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.165+0000 7f09c37fe640 1 -- 172.21.15.2:0/2147795103 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09c80bbec0 con 0x7f09cc1002e0 2024-03-15T12:58:03.151 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-15T12:58:03.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.167+0000 7f09d33a2640 1 -- 172.21.15.2:0/2147795103 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f09b80a1a30 msgr2=0x7f09b80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:03.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.167+0000 7f09d33a2640 1 --2- 172.21.15.2:0/2147795103 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f09b80a1a30 0x7f09b80a3ef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f09cc1989c0 tx=0x7f09bc005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:03.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.167+0000 7f09d33a2640 1 -- 172.21.15.2:0/2147795103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09cc1002e0 msgr2=0x7f09cc1974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:03.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.167+0000 7f09d33a2640 1 --2- 172.21.15.2:0/2147795103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09cc1002e0 0x7f09cc1974b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f09c8000c00 tx=0x7f09c80049e0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.167+0000 7f09d33a2640 1 -- 172.21.15.2:0/2147795103 shutdown_connections 2024-03-15T12:58:03.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.167+0000 7f09d33a2640 1 --2- 172.21.15.2:0/2147795103 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f09b80a1a30 0x7f09b80a3ef0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.167+0000 7f09d33a2640 1 --2- 172.21.15.2:0/2147795103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09cc101e90 0x7f09cc19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.167+0000 7f09d33a2640 1 --2- 172.21.15.2:0/2147795103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09cc1014d0 0x7f09cc1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.167+0000 7f09d33a2640 1 --2- 172.21.15.2:0/2147795103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09cc1002e0 0x7f09cc1974b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.167+0000 7f09d33a2640 1 -- 172.21.15.2:0/2147795103 >> 172.21.15.2:0/2147795103 conn(0x7f09cc0fb870 msgr2=0x7f09cc0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:03.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.167+0000 7f09d33a2640 1 -- 172.21.15.2:0/2147795103 shutdown_connections 2024-03-15T12:58:03.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.167+0000 7f09d33a2640 1 -- 172.21.15.2:0/2147795103 wait complete. 2024-03-15T12:58:03.164 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-15T12:58:03.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.283+0000 7f5a01222640 1 -- 172.21.15.2:0/1988694159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59fc1071d0 msgr2=0x7f59fc109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:03.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.283+0000 7f5a01222640 1 --2- 172.21.15.2:0/1988694159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59fc1071d0 0x7f59fc109620 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009ca0 tx=0x7f59f402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.283+0000 7f5a01222640 1 -- 172.21.15.2:0/1988694159 shutdown_connections 2024-03-15T12:58:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.283+0000 7f5a01222640 1 --2- 172.21.15.2:0/1988694159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59fc1071d0 0x7f59fc109620 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.283+0000 7f5a01222640 1 --2- 172.21.15.2:0/1988694159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59fc1048a0 0x7f59fc106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.283+0000 7f5a01222640 1 --2- 172.21.15.2:0/1988694159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59fc101f70 0x7f59fc104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.283+0000 7f5a01222640 1 -- 172.21.15.2:0/1988694159 >> 172.21.15.2:0/1988694159 conn(0x7f59fc0fb980 msgr2=0x7f59fc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.283+0000 7f5a01222640 1 -- 172.21.15.2:0/1988694159 shutdown_connections 2024-03-15T12:58:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.283+0000 7f5a01222640 1 -- 172.21.15.2:0/1988694159 wait complete. 2024-03-15T12:58:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.284+0000 7f5a01222640 1 Processor -- start 2024-03-15T12:58:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.284+0000 7f5a01222640 1 -- start start 2024-03-15T12:58:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.284+0000 7f5a01222640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59fc101f70 0x7f59fc197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.284+0000 7f5a01222640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59fc1048a0 0x7f59fc197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.284+0000 7f5a01222640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59fc1071d0 0x7f59fc19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:03.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.284+0000 7f5a01222640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc19f040 con 0x7f59fc101f70 2024-03-15T12:58:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.284+0000 7f5a01222640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc19f1b0 con 0x7f59fc1048a0 2024-03-15T12:58:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.284+0000 7f5a01222640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc19f370 con 0x7f59fc1071d0 2024-03-15T12:58:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.284+0000 7f59fad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59fc101f70 0x7f59fc197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.284+0000 7f59fad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59fc101f70 0x7f59fc197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47734/0 (socket says 172.21.15.2:47734) 2024-03-15T12:58:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.284+0000 7f59fa59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59fc1048a0 0x7f59fc197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.284+0000 7f59fad9d640 1 -- 172.21.15.2:0/434913578 learned_addr learned my addr 172.21.15.2:0/434913578 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.284+0000 7f59fb59e640 1 --2- 172.21.15.2:0/434913578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59fc1071d0 0x7f59fc19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.285+0000 7f59fad9d640 1 -- 172.21.15.2:0/434913578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59fc1071d0 msgr2=0x7f59fc19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.285+0000 7f59fad9d640 1 --2- 172.21.15.2:0/434913578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59fc1071d0 0x7f59fc19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.285+0000 7f59fad9d640 1 -- 172.21.15.2:0/434913578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59fc1048a0 msgr2=0x7f59fc197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.285+0000 7f59fad9d640 1 --2- 172.21.15.2:0/434913578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59fc1048a0 0x7f59fc197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.285+0000 7f59fad9d640 1 -- 172.21.15.2:0/434913578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f4009900 con 0x7f59fc101f70 2024-03-15T12:58:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.285+0000 7f59fa59c640 1 --2- 172.21.15.2:0/434913578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59fc1048a0 0x7f59fc197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:03.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.285+0000 7f59fad9d640 1 --2- 172.21.15.2:0/434913578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59fc101f70 0x7f59fc197550 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00e9f0 tx=0x7f59ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.285+0000 7f59e3fff640 1 -- 172.21.15.2:0/434913578 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec009700 con 0x7f59fc101f70 2024-03-15T12:58:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.285+0000 7f59e3fff640 1 -- 172.21.15.2:0/434913578 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f59ec00cd80 con 0x7f59fc101f70 2024-03-15T12:58:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.285+0000 7f5a01222640 1 -- 172.21.15.2:0/434913578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59fc19f650 con 0x7f59fc101f70 2024-03-15T12:58:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.285+0000 7f59e3fff640 1 -- 172.21.15.2:0/434913578 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec018920 con 0x7f59fc101f70 2024-03-15T12:58:03.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.285+0000 7f5a01222640 1 -- 172.21.15.2:0/434913578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59fc19fb60 con 0x7f59fc101f70 2024-03-15T12:58:03.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.286+0000 7f5a01222640 1 -- 172.21.15.2:0/434913578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59fc0fd470 con 0x7f59fc101f70 2024-03-15T12:58:03.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.287+0000 7f59e3fff640 1 -- 172.21.15.2:0/434913578 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f59ec018a80 con 0x7f59fc101f70 2024-03-15T12:58:03.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.288+0000 7f59e3fff640 1 --2- 172.21.15.2:0/434913578 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f59e40a1a30 0x7f59e40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:03.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.288+0000 7f59fa59c640 1 --2- 172.21.15.2:0/434913578 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f59e40a1a30 0x7f59e40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:03.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.288+0000 7f59e3fff640 1 -- 172.21.15.2:0/434913578 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f59ec014070 con 0x7f59fc101f70 2024-03-15T12:58:03.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.288+0000 7f59fa59c640 1 --2- 172.21.15.2:0/434913578 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f59e40a1a30 0x7f59e40a3ef0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f59fc198a60 tx=0x7f59f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:03.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.292+0000 7f59e3fff640 1 -- 172.21.15.2:0/434913578 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f59ec053280 con 0x7f59fc101f70 2024-03-15T12:58:03.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.474+0000 7f5a01222640 1 -- 172.21.15.2:0/434913578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f59fc02d000 con 0x7f59fc101f70 2024-03-15T12:58:03.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.474+0000 7f59e3fff640 1 -- 172.21.15.2:0/434913578 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f59ec0969d0 con 0x7f59fc101f70 2024-03-15T12:58:03.460 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:58:03.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.476+0000 7f5a01222640 1 -- 172.21.15.2:0/434913578 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f59e40a1a30 msgr2=0x7f59e40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:03.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.476+0000 7f5a01222640 1 --2- 172.21.15.2:0/434913578 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f59e40a1a30 0x7f59e40a3ef0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f59fc198a60 tx=0x7f59f0005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:03.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.476+0000 7f5a01222640 1 -- 172.21.15.2:0/434913578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59fc101f70 msgr2=0x7f59fc197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:03.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.476+0000 7f5a01222640 1 --2- 172.21.15.2:0/434913578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59fc101f70 0x7f59fc197550 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00e9f0 tx=0x7f59ec00eeb0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.476+0000 7f5a01222640 1 -- 172.21.15.2:0/434913578 shutdown_connections 2024-03-15T12:58:03.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.476+0000 7f5a01222640 1 --2- 172.21.15.2:0/434913578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59fc1071d0 0x7f59fc19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.476+0000 7f5a01222640 1 --2- 172.21.15.2:0/434913578 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f59e40a1a30 0x7f59e40a3ef0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.476+0000 7f5a01222640 1 --2- 172.21.15.2:0/434913578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59fc1048a0 0x7f59fc197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.476+0000 7f5a01222640 1 --2- 172.21.15.2:0/434913578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59fc101f70 0x7f59fc197550 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.476+0000 7f5a01222640 1 -- 172.21.15.2:0/434913578 >> 172.21.15.2:0/434913578 conn(0x7f59fc0fb980 msgr2=0x7f59fc1028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.476+0000 7f5a01222640 1 -- 172.21.15.2:0/434913578 shutdown_connections 2024-03-15T12:58:03.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.476+0000 7f5a01222640 1 -- 172.21.15.2:0/434913578 wait complete. 2024-03-15T12:58:03.474 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-15T12:58:03.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.593+0000 7fd6c533a640 1 -- 172.21.15.2:0/1256437599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6c0100410 msgr2=0x7fd6c0100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:03.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.593+0000 7fd6c533a640 1 --2- 172.21.15.2:0/1256437599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6c0100410 0x7fd6c0100830 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8009900 tx=0x7fd6b802bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:03.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.593+0000 7fd6c533a640 1 -- 172.21.15.2:0/1256437599 shutdown_connections 2024-03-15T12:58:03.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.593+0000 7fd6c533a640 1 --2- 172.21.15.2:0/1256437599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6c0101fc0 0x7fd6c0108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.593+0000 7fd6c533a640 1 --2- 172.21.15.2:0/1256437599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6c0101600 0x7fd6c0101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.594+0000 7fd6c533a640 1 --2- 172.21.15.2:0/1256437599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6c0100410 0x7fd6c0100830 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.594+0000 7fd6c533a640 1 -- 172.21.15.2:0/1256437599 >> 172.21.15.2:0/1256437599 conn(0x7fd6c00fb9a0 msgr2=0x7fd6c00fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:03.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.594+0000 7fd6c533a640 1 -- 172.21.15.2:0/1256437599 shutdown_connections 2024-03-15T12:58:03.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.594+0000 7fd6c533a640 1 -- 172.21.15.2:0/1256437599 wait complete. 2024-03-15T12:58:03.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.594+0000 7fd6c533a640 1 Processor -- start 2024-03-15T12:58:03.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.594+0000 7fd6c533a640 1 -- start start 2024-03-15T12:58:03.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.594+0000 7fd6c533a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6c0100410 0x7fd6c01043a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:03.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.594+0000 7fd6c533a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6c0101600 0x7fd6c01029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:03.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6c533a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6c0101fc0 0x7fd6c0102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:03.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6c533a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c0103680 con 0x7fd6c0101fc0 2024-03-15T12:58:03.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6c533a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c01037f0 con 0x7fd6c0100410 2024-03-15T12:58:03.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6c533a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c01a1660 con 0x7fd6c0101600 2024-03-15T12:58:03.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6bf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6c0101fc0 0x7fd6c0102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:03.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6beffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6c0100410 0x7fd6c01043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:03.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6bf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6c0101fc0 0x7fd6c0102f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47744/0 (socket says 172.21.15.2:47744) 2024-03-15T12:58:03.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6bf7fe640 1 -- 172.21.15.2:0/2819845978 learned_addr learned my addr 172.21.15.2:0/2819845978 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:03.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6beffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6c0100410 0x7fd6c01043a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55642/0 (socket says 172.21.15.2:55642) 2024-03-15T12:58:03.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6be7fc640 1 --2- 172.21.15.2:0/2819845978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6c0101600 0x7fd6c01029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:03.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6bf7fe640 1 -- 172.21.15.2:0/2819845978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6c0101600 msgr2=0x7fd6c01029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:03.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6bf7fe640 1 --2- 172.21.15.2:0/2819845978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6c0101600 0x7fd6c01029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6bf7fe640 1 -- 172.21.15.2:0/2819845978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6c0100410 msgr2=0x7fd6c01043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:03.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6bf7fe640 1 --2- 172.21.15.2:0/2819845978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6c0100410 0x7fd6c01043a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6bf7fe640 1 -- 172.21.15.2:0/2819845978 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b8009560 con 0x7fd6c0101fc0 2024-03-15T12:58:03.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6bf7fe640 1 --2- 172.21.15.2:0/2819845978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6c0101fc0 0x7fd6c0102f20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000eb40 tx=0x7fd6b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:03.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6a7fff640 1 -- 172.21.15.2:0/2819845978 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0004b20 con 0x7fd6c0101fc0 2024-03-15T12:58:03.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6c533a640 1 -- 172.21.15.2:0/2819845978 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c01a1910 con 0x7fd6c0101fc0 2024-03-15T12:58:03.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6a7fff640 1 -- 172.21.15.2:0/2819845978 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd6b0004c80 con 0x7fd6c0101fc0 2024-03-15T12:58:03.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6a7fff640 1 -- 172.21.15.2:0/2819845978 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0010690 con 0x7fd6c0101fc0 2024-03-15T12:58:03.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.595+0000 7fd6c533a640 1 -- 172.21.15.2:0/2819845978 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c01a1e20 con 0x7fd6c0101fc0 2024-03-15T12:58:03.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.596+0000 7fd6c533a640 1 -- 172.21.15.2:0/2819845978 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c010bdf0 con 0x7fd6c0101fc0 2024-03-15T12:58:03.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.597+0000 7fd6a7fff640 1 -- 172.21.15.2:0/2819845978 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fd6b0004df0 con 0x7fd6c0101fc0 2024-03-15T12:58:03.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.598+0000 7fd6a7fff640 1 --2- 172.21.15.2:0/2819845978 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd6a80a1a30 0x7fd6a80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:03.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.598+0000 7fd6beffd640 1 --2- 172.21.15.2:0/2819845978 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd6a80a1a30 0x7fd6a80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:03.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.598+0000 7fd6a7fff640 1 -- 172.21.15.2:0/2819845978 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fd6b0014070 con 0x7fd6c0101fc0 2024-03-15T12:58:03.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.598+0000 7fd6beffd640 1 --2- 172.21.15.2:0/2819845978 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd6a80a1a30 0x7fd6a80a3ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd6b802c740 tx=0x7fd6b8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:03.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.602+0000 7fd6a7fff640 1 -- 172.21.15.2:0/2819845978 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd6b00d3020 con 0x7fd6c0101fc0 2024-03-15T12:58:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.787+0000 7fd6c533a640 1 -- 172.21.15.2:0/2819845978 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd6c01a2340 con 0x7fd6c0101fc0 2024-03-15T12:58:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.788+0000 7fd6a7fff640 1 -- 172.21.15.2:0/2819845978 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd6b0096ce0 con 0x7fd6c0101fc0 2024-03-15T12:58:03.774 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-03-15T12:58:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.790+0000 7fd6c533a640 1 -- 172.21.15.2:0/2819845978 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd6a80a1a30 msgr2=0x7fd6a80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.790+0000 7fd6c533a640 1 --2- 172.21.15.2:0/2819845978 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd6a80a1a30 0x7fd6a80a3ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd6b802c740 tx=0x7fd6b8005fd0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.790+0000 7fd6c533a640 1 -- 172.21.15.2:0/2819845978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6c0101fc0 msgr2=0x7fd6c0102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:03.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.790+0000 7fd6c533a640 1 --2- 172.21.15.2:0/2819845978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6c0101fc0 0x7fd6c0102f20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000eb40 tx=0x7fd6b000c2d0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.790+0000 7fd6c533a640 1 -- 172.21.15.2:0/2819845978 shutdown_connections 2024-03-15T12:58:03.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.790+0000 7fd6c533a640 1 --2- 172.21.15.2:0/2819845978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6c0101fc0 0x7fd6c0102f20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.790+0000 7fd6c533a640 1 --2- 172.21.15.2:0/2819845978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6c0101600 0x7fd6c01029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.790+0000 7fd6c533a640 1 --2- 172.21.15.2:0/2819845978 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd6a80a1a30 0x7fd6a80a3ef0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.790+0000 7fd6c533a640 1 --2- 172.21.15.2:0/2819845978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6c0100410 0x7fd6c01043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.790+0000 7fd6c533a640 1 -- 172.21.15.2:0/2819845978 >> 172.21.15.2:0/2819845978 conn(0x7fd6c00fb9a0 msgr2=0x7fd6c00fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:03.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.790+0000 7fd6c533a640 1 -- 172.21.15.2:0/2819845978 shutdown_connections 2024-03-15T12:58:03.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.790+0000 7fd6c533a640 1 -- 172.21.15.2:0/2819845978 wait complete. 2024-03-15T12:58:03.786 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:03 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2962453771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:03.786 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:03 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2147795103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:03.786 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:03 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/434913578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:03.787 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-15T12:58:03.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:03 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2962453771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:03.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:03 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2147795103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:03.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:03 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/434913578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:03.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.907+0000 7f1062de2640 1 -- 172.21.15.2:0/2065058036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105c1003f0 msgr2=0x7f105c100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:03.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.907+0000 7f1062de2640 1 --2- 172.21.15.2:0/2065058036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105c1003f0 0x7f105c100810 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1050009900 tx=0x7f105002bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:03.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.907+0000 7f1062de2640 1 -- 172.21.15.2:0/2065058036 shutdown_connections 2024-03-15T12:58:03.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.907+0000 7f1062de2640 1 --2- 172.21.15.2:0/2065058036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f105c101fa0 0x7f105c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f1062de2640 1 --2- 172.21.15.2:0/2065058036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f105c1015e0 0x7f105c101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f1062de2640 1 --2- 172.21.15.2:0/2065058036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105c1003f0 0x7f105c100810 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f1062de2640 1 -- 172.21.15.2:0/2065058036 >> 172.21.15.2:0/2065058036 conn(0x7f105c0fb980 msgr2=0x7f105c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:03.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f1062de2640 1 -- 172.21.15.2:0/2065058036 shutdown_connections 2024-03-15T12:58:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f1062de2640 1 -- 172.21.15.2:0/2065058036 wait complete. 2024-03-15T12:58:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f1062de2640 1 Processor -- start 2024-03-15T12:58:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f1062de2640 1 -- start start 2024-03-15T12:58:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f1062de2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f105c1003f0 0x7f105c197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f1062de2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105c1015e0 0x7f105c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f1062de2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f105c101fa0 0x7f105c19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f1062de2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c19f140 con 0x7f105c1015e0 2024-03-15T12:58:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f1062de2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c19f2b0 con 0x7f105c1003f0 2024-03-15T12:58:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f1062de2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c19f470 con 0x7f105c101fa0 2024-03-15T12:58:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f105bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105c1015e0 0x7f105c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f105bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105c1015e0 0x7f105c197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47760/0 (socket says 172.21.15.2:47760) 2024-03-15T12:58:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f105bfff640 1 -- 172.21.15.2:0/1586811039 learned_addr learned my addr 172.21.15.2:0/1586811039 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f1060b7e640 1 --2- 172.21.15.2:0/1586811039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f105c1003f0 0x7f105c197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.908+0000 7f106137f640 1 --2- 172.21.15.2:0/1586811039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f105c101fa0 0x7f105c19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.909+0000 7f105bfff640 1 -- 172.21.15.2:0/1586811039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f105c101fa0 msgr2=0x7f105c19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.909+0000 7f105bfff640 1 --2- 172.21.15.2:0/1586811039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f105c101fa0 0x7f105c19ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.909+0000 7f105bfff640 1 -- 172.21.15.2:0/1586811039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f105c1003f0 msgr2=0x7f105c197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.909+0000 7f105bfff640 1 --2- 172.21.15.2:0/1586811039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f105c1003f0 0x7f105c197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.909+0000 7f105bfff640 1 -- 172.21.15.2:0/1586811039 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1050009560 con 0x7f105c1015e0 2024-03-15T12:58:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.909+0000 7f106137f640 1 --2- 172.21.15.2:0/1586811039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f105c101fa0 0x7f105c19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:58:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.909+0000 7f105bfff640 1 --2- 172.21.15.2:0/1586811039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105c1015e0 0x7f105c197b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f104800e9f0 tx=0x7f104800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.909+0000 7f1059ffb640 1 -- 172.21.15.2:0/1586811039 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048009720 con 0x7f105c1015e0 2024-03-15T12:58:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.909+0000 7f1059ffb640 1 -- 172.21.15.2:0/1586811039 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f104800cd80 con 0x7f105c1015e0 2024-03-15T12:58:03.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.909+0000 7f1062de2640 1 -- 172.21.15.2:0/1586811039 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c19f750 con 0x7f105c1015e0 2024-03-15T12:58:03.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.909+0000 7f1059ffb640 1 -- 172.21.15.2:0/1586811039 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048010690 con 0x7f105c1015e0 2024-03-15T12:58:03.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.909+0000 7f1062de2640 1 -- 172.21.15.2:0/1586811039 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c19fc10 con 0x7f105c1015e0 2024-03-15T12:58:03.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.910+0000 7f1062de2640 1 -- 172.21.15.2:0/1586811039 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105c10bdd0 con 0x7f105c1015e0 2024-03-15T12:58:03.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.911+0000 7f1059ffb640 1 -- 172.21.15.2:0/1586811039 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f1048004e90 con 0x7f105c1015e0 2024-03-15T12:58:03.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.911+0000 7f1059ffb640 1 --2- 172.21.15.2:0/1586811039 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f104c0a1a30 0x7f104c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:03.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.912+0000 7f1060b7e640 1 --2- 172.21.15.2:0/1586811039 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f104c0a1a30 0x7f104c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:03.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.912+0000 7f1059ffb640 1 -- 172.21.15.2:0/1586811039 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f1048014070 con 0x7f105c1015e0 2024-03-15T12:58:03.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.912+0000 7f1060b7e640 1 --2- 172.21.15.2:0/1586811039 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f104c0a1a30 0x7f104c0a3ef0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f105002c740 tx=0x7f1050005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:03.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:03.916+0000 7f1059ffb640 1 -- 172.21.15.2:0/1586811039 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1048096a00 con 0x7f105c1015e0 2024-03-15T12:58:04.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:03 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2962453771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:04.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:03 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2147795103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:04.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:03 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/434913578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:04.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.100+0000 7f1062de2640 1 -- 172.21.15.2:0/1586811039 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f105c1a0030 con 0x7f105c1015e0 2024-03-15T12:58:04.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.101+0000 7f1059ffb640 1 -- 172.21.15.2:0/1586811039 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f104809a0c0 con 0x7f105c1015e0 2024-03-15T12:58:04.087 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:58:04.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.103+0000 7f1062de2640 1 -- 172.21.15.2:0/1586811039 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f104c0a1a30 msgr2=0x7f104c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:04.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.103+0000 7f1062de2640 1 --2- 172.21.15.2:0/1586811039 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f104c0a1a30 0x7f104c0a3ef0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f105002c740 tx=0x7f1050005fd0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.103+0000 7f1062de2640 1 -- 172.21.15.2:0/1586811039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105c1015e0 msgr2=0x7f105c197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:04.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.103+0000 7f1062de2640 1 --2- 172.21.15.2:0/1586811039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105c1015e0 0x7f105c197b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f104800e9f0 tx=0x7f104800edf0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.103+0000 7f1062de2640 1 -- 172.21.15.2:0/1586811039 shutdown_connections 2024-03-15T12:58:04.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.103+0000 7f1062de2640 1 --2- 172.21.15.2:0/1586811039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f105c101fa0 0x7f105c19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.103+0000 7f1062de2640 1 --2- 172.21.15.2:0/1586811039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105c1015e0 0x7f105c197b90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.103+0000 7f1062de2640 1 --2- 172.21.15.2:0/1586811039 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f104c0a1a30 0x7f104c0a3ef0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.103+0000 7f1062de2640 1 --2- 172.21.15.2:0/1586811039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f105c1003f0 0x7f105c197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.103+0000 7f1062de2640 1 -- 172.21.15.2:0/1586811039 >> 172.21.15.2:0/1586811039 conn(0x7f105c0fb980 msgr2=0x7f105c0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:04.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.103+0000 7f1062de2640 1 -- 172.21.15.2:0/1586811039 shutdown_connections 2024-03-15T12:58:04.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.103+0000 7f1062de2640 1 -- 172.21.15.2:0/1586811039 wait complete. 2024-03-15T12:58:04.101 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:58:04.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.218+0000 7f75f47ae640 1 -- 172.21.15.2:0/833108580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75ec101fc0 msgr2=0x7f75ec108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:04.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.218+0000 7f75f47ae640 1 --2- 172.21.15.2:0/833108580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75ec101fc0 0x7f75ec108850 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009ca0 tx=0x7f75e402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.218+0000 7f75f47ae640 1 -- 172.21.15.2:0/833108580 shutdown_connections 2024-03-15T12:58:04.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.218+0000 7f75f47ae640 1 --2- 172.21.15.2:0/833108580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75ec101fc0 0x7f75ec108850 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.218+0000 7f75f47ae640 1 --2- 172.21.15.2:0/833108580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75ec101600 0x7f75ec101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.218+0000 7f75f47ae640 1 --2- 172.21.15.2:0/833108580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75ec100410 0x7f75ec100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.218+0000 7f75f47ae640 1 -- 172.21.15.2:0/833108580 >> 172.21.15.2:0/833108580 conn(0x7f75ec0fb980 msgr2=0x7f75ec0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.218+0000 7f75f47ae640 1 -- 172.21.15.2:0/833108580 shutdown_connections 2024-03-15T12:58:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.218+0000 7f75f47ae640 1 -- 172.21.15.2:0/833108580 wait complete. 2024-03-15T12:58:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f47ae640 1 Processor -- start 2024-03-15T12:58:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f47ae640 1 -- start start 2024-03-15T12:58:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f47ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75ec100410 0x7f75ec10af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f47ae640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75ec101600 0x7f75ec109560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f47ae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75ec101fc0 0x7f75ec109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f47ae640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec10a200 con 0x7f75ec101fc0 2024-03-15T12:58:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f47ae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec10a370 con 0x7f75ec100410 2024-03-15T12:58:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f47ae640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1a15a0 con 0x7f75ec101600 2024-03-15T12:58:04.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f2d4b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75ec101fc0 0x7f75ec109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:04.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f2d4b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75ec101fc0 0x7f75ec109aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47772/0 (socket says 172.21.15.2:47772) 2024-03-15T12:58:04.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f2d4b640 1 -- 172.21.15.2:0/936028225 learned_addr learned my addr 172.21.15.2:0/936028225 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:04.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f254a640 1 --2- 172.21.15.2:0/936028225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75ec100410 0x7f75ec10af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:04.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f1d49640 1 --2- 172.21.15.2:0/936028225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75ec101600 0x7f75ec109560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:04.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f2d4b640 1 -- 172.21.15.2:0/936028225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75ec101600 msgr2=0x7f75ec109560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:04.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f2d4b640 1 --2- 172.21.15.2:0/936028225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75ec101600 0x7f75ec109560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.219+0000 7f75f2d4b640 1 -- 172.21.15.2:0/936028225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75ec100410 msgr2=0x7f75ec10af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:04.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.220+0000 7f75f2d4b640 1 --2- 172.21.15.2:0/936028225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75ec100410 0x7f75ec10af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.220+0000 7f75f2d4b640 1 -- 172.21.15.2:0/936028225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e8009490 con 0x7f75ec101fc0 2024-03-15T12:58:04.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.220+0000 7f75f2d4b640 1 --2- 172.21.15.2:0/936028225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75ec101fc0 0x7f75ec109aa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f75e402c630 tx=0x7f75e4032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:04.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.220+0000 7f75e37fe640 1 -- 172.21.15.2:0/936028225 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4034c90 con 0x7f75ec101fc0 2024-03-15T12:58:04.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.220+0000 7f75e37fe640 1 -- 172.21.15.2:0/936028225 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f75e4003e40 con 0x7f75ec101fc0 2024-03-15T12:58:04.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.220+0000 7f75f47ae640 1 -- 172.21.15.2:0/936028225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e4009900 con 0x7f75ec101fc0 2024-03-15T12:58:04.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.220+0000 7f75f47ae640 1 -- 172.21.15.2:0/936028225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec1a1b10 con 0x7f75ec101fc0 2024-03-15T12:58:04.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.220+0000 7f75e37fe640 1 -- 172.21.15.2:0/936028225 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e40048a0 con 0x7f75ec101fc0 2024-03-15T12:58:04.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.221+0000 7f75f47ae640 1 -- 172.21.15.2:0/936028225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d00052c0 con 0x7f75ec101fc0 2024-03-15T12:58:04.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.222+0000 7f75e37fe640 1 -- 172.21.15.2:0/936028225 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f75e400a040 con 0x7f75ec101fc0 2024-03-15T12:58:04.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.222+0000 7f75e37fe640 1 --2- 172.21.15.2:0/936028225 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f75d80a1a30 0x7f75d80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:04.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.223+0000 7f75f254a640 1 --2- 172.21.15.2:0/936028225 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f75d80a1a30 0x7f75d80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:04.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.223+0000 7f75e37fe640 1 -- 172.21.15.2:0/936028225 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f75e40f0360 con 0x7f75ec101fc0 2024-03-15T12:58:04.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.223+0000 7f75f254a640 1 --2- 172.21.15.2:0/936028225 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f75d80a1a30 0x7f75d80a3ef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f75ec101460 tx=0x7f75e8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:04.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.227+0000 7f75e37fe640 1 -- 172.21.15.2:0/936028225 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f75e40b9230 con 0x7f75ec101fc0 2024-03-15T12:58:04.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.446+0000 7f75f47ae640 1 -- 172.21.15.2:0/936028225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75d0005af0 con 0x7f75ec101fc0 2024-03-15T12:58:04.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.447+0000 7f75e37fe640 1 -- 172.21.15.2:0/936028225 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11453 (secure 0 0 0) 0x7f75e40bc8f0 con 0x7f75ec101fc0 2024-03-15T12:58:04.433 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:04.433 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T12:56:43.433836+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24464,24500,24460,24539,24434]},"id":1}]} 2024-03-15T12:58:04.433 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 28 2024-03-15T12:58:04.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.448+0000 7f75f47ae640 1 -- 172.21.15.2:0/936028225 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f75d80a1a30 msgr2=0x7f75d80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:04.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.448+0000 7f75f47ae640 1 --2- 172.21.15.2:0/936028225 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f75d80a1a30 0x7f75d80a3ef0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f75ec101460 tx=0x7f75e8005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:04.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.448+0000 7f75f47ae640 1 -- 172.21.15.2:0/936028225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75ec101fc0 msgr2=0x7f75ec109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:04.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.448+0000 7f75f47ae640 1 --2- 172.21.15.2:0/936028225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75ec101fc0 0x7f75ec109aa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f75e402c630 tx=0x7f75e4032a10 comp rx=0 tx=0).stop 2024-03-15T12:58:04.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.449+0000 7f75f47ae640 1 -- 172.21.15.2:0/936028225 shutdown_connections 2024-03-15T12:58:04.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.449+0000 7f75f47ae640 1 --2- 172.21.15.2:0/936028225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75ec101fc0 0x7f75ec109aa0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.449+0000 7f75f47ae640 1 --2- 172.21.15.2:0/936028225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75ec101600 0x7f75ec109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.449+0000 7f75f47ae640 1 --2- 172.21.15.2:0/936028225 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f75d80a1a30 0x7f75d80a3ef0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.449+0000 7f75f47ae640 1 --2- 172.21.15.2:0/936028225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75ec100410 0x7f75ec10af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.449+0000 7f75f47ae640 1 -- 172.21.15.2:0/936028225 >> 172.21.15.2:0/936028225 conn(0x7f75ec0fb980 msgr2=0x7f75ec1048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:04.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.449+0000 7f75f47ae640 1 -- 172.21.15.2:0/936028225 shutdown_connections 2024-03-15T12:58:04.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.449+0000 7f75f47ae640 1 -- 172.21.15.2:0/936028225 wait complete. 2024-03-15T12:58:04.446 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:58:04.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.564+0000 7fa292f81640 1 -- 172.21.15.2:0/3559340552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa28c101fc0 msgr2=0x7fa28c108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:04.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.564+0000 7fa292f81640 1 --2- 172.21.15.2:0/3559340552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa28c101fc0 0x7fa28c108850 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa284009ca0 tx=0x7fa28402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.564+0000 7fa292f81640 1 -- 172.21.15.2:0/3559340552 shutdown_connections 2024-03-15T12:58:04.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.564+0000 7fa292f81640 1 --2- 172.21.15.2:0/3559340552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa28c101fc0 0x7fa28c108850 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa292f81640 1 --2- 172.21.15.2:0/3559340552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa28c101600 0x7fa28c101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa292f81640 1 --2- 172.21.15.2:0/3559340552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c100410 0x7fa28c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa292f81640 1 -- 172.21.15.2:0/3559340552 >> 172.21.15.2:0/3559340552 conn(0x7fa28c0fb980 msgr2=0x7fa28c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:04.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa292f81640 1 -- 172.21.15.2:0/3559340552 shutdown_connections 2024-03-15T12:58:04.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa292f81640 1 -- 172.21.15.2:0/3559340552 wait complete. 2024-03-15T12:58:04.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa292f81640 1 Processor -- start 2024-03-15T12:58:04.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa292f81640 1 -- start start 2024-03-15T12:58:04.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa292f81640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c100410 0x7fa28c197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:04.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa292f81640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa28c101600 0x7fa28c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:04.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa292f81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa28c101fc0 0x7fa28c19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:04.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa292f81640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c19f070 con 0x7fa28c101fc0 2024-03-15T12:58:04.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa292f81640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c19f1e0 con 0x7fa28c100410 2024-03-15T12:58:04.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa292f81640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c19f3a0 con 0x7fa28c101600 2024-03-15T12:58:04.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa29151e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa28c101fc0 0x7fa28c19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:04.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa29151e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa28c101fc0 0x7fa28c19eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47790/0 (socket says 172.21.15.2:47790) 2024-03-15T12:58:04.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa29151e640 1 -- 172.21.15.2:0/2329153843 learned_addr learned my addr 172.21.15.2:0/2329153843 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:04.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa290d1d640 1 --2- 172.21.15.2:0/2329153843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c100410 0x7fa28c197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:04.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.565+0000 7fa28bfff640 1 --2- 172.21.15.2:0/2329153843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa28c101600 0x7fa28c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:04.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.566+0000 7fa29151e640 1 -- 172.21.15.2:0/2329153843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa28c101600 msgr2=0x7fa28c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:04.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.566+0000 7fa29151e640 1 --2- 172.21.15.2:0/2329153843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa28c101600 0x7fa28c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.566+0000 7fa29151e640 1 -- 172.21.15.2:0/2329153843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c100410 msgr2=0x7fa28c197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:04.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.566+0000 7fa29151e640 1 --2- 172.21.15.2:0/2329153843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c100410 0x7fa28c197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.566+0000 7fa29151e640 1 -- 172.21.15.2:0/2329153843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa284009900 con 0x7fa28c101fc0 2024-03-15T12:58:04.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.566+0000 7fa29151e640 1 --2- 172.21.15.2:0/2329153843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa28c101fc0 0x7fa28c19eb30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa284005cc0 tx=0x7fa284032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:04.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.566+0000 7fa289ffb640 1 -- 172.21.15.2:0/2329153843 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284034c90 con 0x7fa28c101fc0 2024-03-15T12:58:04.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.566+0000 7fa289ffb640 1 -- 172.21.15.2:0/2329153843 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa284003e40 con 0x7fa28c101fc0 2024-03-15T12:58:04.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.566+0000 7fa292f81640 1 -- 172.21.15.2:0/2329153843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c19f620 con 0x7fa28c101fc0 2024-03-15T12:58:04.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.566+0000 7fa289ffb640 1 -- 172.21.15.2:0/2329153843 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840048a0 con 0x7fa28c101fc0 2024-03-15T12:58:04.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.566+0000 7fa292f81640 1 -- 172.21.15.2:0/2329153843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c19fb20 con 0x7fa28c101fc0 2024-03-15T12:58:04.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.567+0000 7fa292f81640 1 -- 172.21.15.2:0/2329153843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c10bdf0 con 0x7fa28c101fc0 2024-03-15T12:58:04.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.568+0000 7fa289ffb640 1 -- 172.21.15.2:0/2329153843 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fa28400a040 con 0x7fa28c101fc0 2024-03-15T12:58:04.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.569+0000 7fa289ffb640 1 --2- 172.21.15.2:0/2329153843 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa27c0a1ab0 0x7fa27c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:04.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.569+0000 7fa290d1d640 1 --2- 172.21.15.2:0/2329153843 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa27c0a1ab0 0x7fa27c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:04.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.569+0000 7fa289ffb640 1 -- 172.21.15.2:0/2329153843 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fa2840f0830 con 0x7fa28c101fc0 2024-03-15T12:58:04.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.570+0000 7fa290d1d640 1 --2- 172.21.15.2:0/2329153843 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa27c0a1ab0 0x7fa27c0a3f70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa28c101460 tx=0x7fa280005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:04.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.573+0000 7fa289ffb640 1 -- 172.21.15.2:0/2329153843 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa2840b9700 con 0x7fa28c101fc0 2024-03-15T12:58:04.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.755+0000 7fa292f81640 1 -- 172.21.15.2:0/2329153843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa28c19ffc0 con 0x7fa28c101fc0 2024-03-15T12:58:04.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.757+0000 7fa289ffb640 1 -- 172.21.15.2:0/2329153843 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24526 (secure 0 0 0) 0x7fa2840bcdc0 con 0x7fa28c101fc0 2024-03-15T12:58:04.743 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:04.743 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":91,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:57:19.826817+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:58:04.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.759+0000 7fa292f81640 1 -- 172.21.15.2:0/2329153843 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa27c0a1ab0 msgr2=0x7fa27c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:04.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.759+0000 7fa292f81640 1 --2- 172.21.15.2:0/2329153843 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa27c0a1ab0 0x7fa27c0a3f70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa28c101460 tx=0x7fa280005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:04.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.759+0000 7fa292f81640 1 -- 172.21.15.2:0/2329153843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa28c101fc0 msgr2=0x7fa28c19eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:04.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.759+0000 7fa292f81640 1 --2- 172.21.15.2:0/2329153843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa28c101fc0 0x7fa28c19eb30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa284005cc0 tx=0x7fa284032a10 comp rx=0 tx=0).stop 2024-03-15T12:58:04.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.759+0000 7fa292f81640 1 -- 172.21.15.2:0/2329153843 shutdown_connections 2024-03-15T12:58:04.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.759+0000 7fa292f81640 1 --2- 172.21.15.2:0/2329153843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa28c101fc0 0x7fa28c19eb30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.759+0000 7fa292f81640 1 --2- 172.21.15.2:0/2329153843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa28c101600 0x7fa28c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.759+0000 7fa292f81640 1 --2- 172.21.15.2:0/2329153843 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa27c0a1ab0 0x7fa27c0a3f70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.759+0000 7fa292f81640 1 --2- 172.21.15.2:0/2329153843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c100410 0x7fa28c197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.759+0000 7fa292f81640 1 -- 172.21.15.2:0/2329153843 >> 172.21.15.2:0/2329153843 conn(0x7fa28c0fb980 msgr2=0x7fa28c0fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:04.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.759+0000 7fa292f81640 1 -- 172.21.15.2:0/2329153843 shutdown_connections 2024-03-15T12:58:04.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.759+0000 7fa292f81640 1 -- 172.21.15.2:0/2329153843 wait complete. 2024-03-15T12:58:04.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:04 smithi002 ceph-mon[32060]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-15T12:58:04.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:04 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2819845978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:04.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:04 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1586811039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:04.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:04 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/936028225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:04.758 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:58:04.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:04 smithi149 ceph-mon[35927]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-15T12:58:04.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:04 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2819845978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:04.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:04 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1586811039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:04.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:04 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/936028225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:04.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.874+0000 7fd96b3d0640 1 -- 172.21.15.2:0/4054709832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd964102710 msgr2=0x7fd964102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:04.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.874+0000 7fd96b3d0640 1 --2- 172.21.15.2:0/4054709832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd964102710 0x7fd964102b90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd954009900 tx=0x7fd95402bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:04.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.874+0000 7fd96b3d0640 1 -- 172.21.15.2:0/4054709832 shutdown_connections 2024-03-15T12:58:04.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.874+0000 7fd96b3d0640 1 --2- 172.21.15.2:0/4054709832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9641030d0 0x7fd964109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.874+0000 7fd96b3d0640 1 --2- 172.21.15.2:0/4054709832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd964102710 0x7fd964102b90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.874+0000 7fd96b3d0640 1 --2- 172.21.15.2:0/4054709832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd964101520 0x7fd964101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.874+0000 7fd96b3d0640 1 -- 172.21.15.2:0/4054709832 >> 172.21.15.2:0/4054709832 conn(0x7fd9640fcad0 msgr2=0x7fd9640feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:04.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.874+0000 7fd96b3d0640 1 -- 172.21.15.2:0/4054709832 shutdown_connections 2024-03-15T12:58:04.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.874+0000 7fd96b3d0640 1 -- 172.21.15.2:0/4054709832 wait complete. 2024-03-15T12:58:04.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.874+0000 7fd96b3d0640 1 Processor -- start 2024-03-15T12:58:04.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.874+0000 7fd96b3d0640 1 -- start start 2024-03-15T12:58:04.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96b3d0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd964101520 0x7fd9641975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:04.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96b3d0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd964102710 0x7fd964197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:04.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96b3d0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9641030d0 0x7fd96419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:04.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96b3d0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96419f090 con 0x7fd964102710 2024-03-15T12:58:04.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96b3d0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96419f200 con 0x7fd9641030d0 2024-03-15T12:58:04.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96b3d0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96419f3c0 con 0x7fd964101520 2024-03-15T12:58:04.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96896b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd964102710 0x7fd964197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:04.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96896b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd964102710 0x7fd964197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47800/0 (socket says 172.21.15.2:47800) 2024-03-15T12:58:04.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96896b640 1 -- 172.21.15.2:0/3182779329 learned_addr learned my addr 172.21.15.2:0/3182779329 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:04.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96916c640 1 --2- 172.21.15.2:0/3182779329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd964101520 0x7fd9641975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:04.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96996d640 1 --2- 172.21.15.2:0/3182779329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9641030d0 0x7fd96419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:04.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96896b640 1 -- 172.21.15.2:0/3182779329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd964101520 msgr2=0x7fd9641975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:04.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96896b640 1 --2- 172.21.15.2:0/3182779329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd964101520 0x7fd9641975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96896b640 1 -- 172.21.15.2:0/3182779329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9641030d0 msgr2=0x7fd96419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:04.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96896b640 1 --2- 172.21.15.2:0/3182779329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9641030d0 0x7fd96419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:04.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96896b640 1 -- 172.21.15.2:0/3182779329 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c009840 con 0x7fd964102710 2024-03-15T12:58:04.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96916c640 1 --2- 172.21.15.2:0/3182779329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd964101520 0x7fd9641975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:04.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96896b640 1 --2- 172.21.15.2:0/3182779329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd964102710 0x7fd964197ae0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd954005ab0 tx=0x7fd954004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:04.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd95a7fc640 1 -- 172.21.15.2:0/3182779329 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954004250 con 0x7fd964102710 2024-03-15T12:58:04.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96b3d0640 1 -- 172.21.15.2:0/3182779329 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd954009560 con 0x7fd964102710 2024-03-15T12:58:04.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd95a7fc640 1 -- 172.21.15.2:0/3182779329 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd9540043b0 con 0x7fd964102710 2024-03-15T12:58:04.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd95a7fc640 1 -- 172.21.15.2:0/3182779329 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954003ca0 con 0x7fd964102710 2024-03-15T12:58:04.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.875+0000 7fd96b3d0640 1 -- 172.21.15.2:0/3182779329 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96419f9c0 con 0x7fd964102710 2024-03-15T12:58:04.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.877+0000 7fd96b3d0640 1 -- 172.21.15.2:0/3182779329 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9641088c0 con 0x7fd964102710 2024-03-15T12:58:04.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.878+0000 7fd95a7fc640 1 -- 172.21.15.2:0/3182779329 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fd954005000 con 0x7fd964102710 2024-03-15T12:58:04.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.878+0000 7fd95a7fc640 1 --2- 172.21.15.2:0/3182779329 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd9500a1a30 0x7fd9500a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:04.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.878+0000 7fd96916c640 1 --2- 172.21.15.2:0/3182779329 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd9500a1a30 0x7fd9500a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:04.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.878+0000 7fd95a7fc640 1 -- 172.21.15.2:0/3182779329 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fd9540ef940 con 0x7fd964102710 2024-03-15T12:58:04.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.879+0000 7fd96916c640 1 --2- 172.21.15.2:0/3182779329 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd9500a1a30 0x7fd9500a3ef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd964102570 tx=0x7fd960005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:04.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:04.882+0000 7fd95a7fc640 1 -- 172.21.15.2:0/3182779329 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd9540b8810 con 0x7fd964102710 2024-03-15T12:58:05.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:04 smithi184 ceph-mon[35993]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-15T12:58:05.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:04 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2819845978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:05.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:04 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1586811039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:05.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:04 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/936028225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:05.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.107+0000 7fd96b3d0640 1 -- 172.21.15.2:0/3182779329 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd96402cf90 con 0x7fd964102710 2024-03-15T12:58:05.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.108+0000 7fd95a7fc640 1 -- 172.21.15.2:0/3182779329 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11453 (secure 0 0 0) 0x7fd9540bbed0 con 0x7fd964102710 2024-03-15T12:58:05.094 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:05.094 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T12:56:43.433836+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24464,24500,24460,24539,24434]},"id":1}]} 2024-03-15T12:58:05.094 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 28 2024-03-15T12:58:05.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.110+0000 7fd96b3d0640 1 -- 172.21.15.2:0/3182779329 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd9500a1a30 msgr2=0x7fd9500a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.110+0000 7fd96b3d0640 1 --2- 172.21.15.2:0/3182779329 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd9500a1a30 0x7fd9500a3ef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd964102570 tx=0x7fd960005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:05.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.110+0000 7fd96b3d0640 1 -- 172.21.15.2:0/3182779329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd964102710 msgr2=0x7fd964197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.110+0000 7fd96b3d0640 1 --2- 172.21.15.2:0/3182779329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd964102710 0x7fd964197ae0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd954005ab0 tx=0x7fd954004a30 comp rx=0 tx=0).stop 2024-03-15T12:58:05.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.110+0000 7fd96b3d0640 1 -- 172.21.15.2:0/3182779329 shutdown_connections 2024-03-15T12:58:05.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.110+0000 7fd96b3d0640 1 --2- 172.21.15.2:0/3182779329 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd9500a1a30 0x7fd9500a3ef0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.110+0000 7fd96b3d0640 1 --2- 172.21.15.2:0/3182779329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9641030d0 0x7fd96419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.110+0000 7fd96b3d0640 1 --2- 172.21.15.2:0/3182779329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd964102710 0x7fd964197ae0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.110+0000 7fd96b3d0640 1 --2- 172.21.15.2:0/3182779329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd964101520 0x7fd9641975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.110+0000 7fd96b3d0640 1 -- 172.21.15.2:0/3182779329 >> 172.21.15.2:0/3182779329 conn(0x7fd9640fcad0 msgr2=0x7fd964105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.111+0000 7fd96b3d0640 1 -- 172.21.15.2:0/3182779329 shutdown_connections 2024-03-15T12:58:05.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.111+0000 7fd96b3d0640 1 -- 172.21.15.2:0/3182779329 wait complete. 2024-03-15T12:58:05.108 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-15T12:56:20.559317+0000', 'modified': '2024-03-15T12:56:43.433836+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24539, 'mds_1': 24500, 'mds_2': 24464, 'mds_3': 24460, 'mds_4': 24434}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14640': {'gid': 14640, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.2:6839/2513297233', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6838', 'nonce': 2513297233}, {'type': 'v1', 'addr': '172.21.15.2:6839', 'nonce': 2513297233}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14652': {'gid': 14652, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.2:6841/3747539896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6840', 'nonce': 3747539896}, {'type': 'v1', 'addr': '172.21.15.2:6841', 'nonce': 3747539896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24406': {'gid': 24406, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6833/730881833', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6832', 'nonce': 730881833}, {'type': 'v1', 'addr': '172.21.15.184:6833', 'nonce': 730881833}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24434': {'gid': 24434, 'name': 'k', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.149:6833/2587208478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6832', 'nonce': 2587208478}, {'type': 'v1', 'addr': '172.21.15.149:6833', 'nonce': 2587208478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.184:6835/1255256953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 1255256953}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 1255256953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.149:6835/694548319', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 694548319}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 694548319}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6837/3698954880', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6836', 'nonce': 3698954880}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 3698954880}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6839/1170793322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6838', 'nonce': 1170793322}, {'type': 'v1', 'addr': '172.21.15.184:6839', 'nonce': 1170793322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.149:6837/1813851428', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 1813851428}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 1813851428}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.149:6839/1715135128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 1715135128}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 1715135128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24464, 24500, 24460, 24539, 24434]} 2024-03-15T12:58:05.108 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-15T12:58:05.108 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-03-15T12:58:05.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.228+0000 7f3adadb5640 1 -- 172.21.15.2:0/2955262632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad41030b0 msgr2=0x7f3ad4109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.228+0000 7f3adadb5640 1 --2- 172.21.15.2:0/2955262632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad41030b0 0x7f3ad4109940 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009ca0 tx=0x7f3acc02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.228+0000 7f3adadb5640 1 -- 172.21.15.2:0/2955262632 shutdown_connections 2024-03-15T12:58:05.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.228+0000 7f3adadb5640 1 --2- 172.21.15.2:0/2955262632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad41030b0 0x7f3ad4109940 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.228+0000 7f3adadb5640 1 --2- 172.21.15.2:0/2955262632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ad41026f0 0x7f3ad4102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.228+0000 7f3adadb5640 1 --2- 172.21.15.2:0/2955262632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ad4101500 0x7f3ad4101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.228+0000 7f3adadb5640 1 -- 172.21.15.2:0/2955262632 >> 172.21.15.2:0/2955262632 conn(0x7f3ad40fca70 msgr2=0x7f3ad40feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.228+0000 7f3adadb5640 1 -- 172.21.15.2:0/2955262632 shutdown_connections 2024-03-15T12:58:05.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.228+0000 7f3adadb5640 1 -- 172.21.15.2:0/2955262632 wait complete. 2024-03-15T12:58:05.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.229+0000 7f3adadb5640 1 Processor -- start 2024-03-15T12:58:05.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.229+0000 7f3adadb5640 1 -- start start 2024-03-15T12:58:05.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.229+0000 7f3adadb5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ad4101500 0x7f3ad4197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.229+0000 7f3adadb5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ad41026f0 0x7f3ad4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.229+0000 7f3adadb5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad41030b0 0x7f3ad419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.229+0000 7f3adadb5640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad419f050 con 0x7f3ad41030b0 2024-03-15T12:58:05.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.229+0000 7f3adadb5640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad419f1c0 con 0x7f3ad41026f0 2024-03-15T12:58:05.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.229+0000 7f3adadb5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad419f380 con 0x7f3ad4101500 2024-03-15T12:58:05.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.229+0000 7f3ad9352640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad41030b0 0x7f3ad419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.229+0000 7f3ad9352640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad41030b0 0x7f3ad419eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47828/0 (socket says 172.21.15.2:47828) 2024-03-15T12:58:05.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.229+0000 7f3ad3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ad41026f0 0x7f3ad4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.229+0000 7f3ad9352640 1 -- 172.21.15.2:0/1481055012 learned_addr learned my addr 172.21.15.2:0/1481055012 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.229+0000 7f3ad8b51640 1 --2- 172.21.15.2:0/1481055012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ad4101500 0x7f3ad4197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.230+0000 7f3ad9352640 1 -- 172.21.15.2:0/1481055012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ad4101500 msgr2=0x7f3ad4197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.230+0000 7f3ad9352640 1 --2- 172.21.15.2:0/1481055012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ad4101500 0x7f3ad4197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.230+0000 7f3ad9352640 1 -- 172.21.15.2:0/1481055012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ad41026f0 msgr2=0x7f3ad4197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.230+0000 7f3ad9352640 1 --2- 172.21.15.2:0/1481055012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ad41026f0 0x7f3ad4197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.230+0000 7f3ad9352640 1 -- 172.21.15.2:0/1481055012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc009900 con 0x7f3ad41030b0 2024-03-15T12:58:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.230+0000 7f3ad3fff640 1 --2- 172.21.15.2:0/1481055012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ad41026f0 0x7f3ad4197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.230+0000 7f3ad9352640 1 --2- 172.21.15.2:0/1481055012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad41030b0 0x7f3ad419eb10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3acc005cc0 tx=0x7f3acc032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.230+0000 7f3ad1ffb640 1 -- 172.21.15.2:0/1481055012 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc034c90 con 0x7f3ad41030b0 2024-03-15T12:58:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.230+0000 7f3adadb5640 1 -- 172.21.15.2:0/1481055012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad419f600 con 0x7f3ad41030b0 2024-03-15T12:58:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.230+0000 7f3ad1ffb640 1 -- 172.21.15.2:0/1481055012 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3acc003e40 con 0x7f3ad41030b0 2024-03-15T12:58:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.230+0000 7f3ad1ffb640 1 -- 172.21.15.2:0/1481055012 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0048a0 con 0x7f3ad41030b0 2024-03-15T12:58:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.230+0000 7f3adadb5640 1 -- 172.21.15.2:0/1481055012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad419fab0 con 0x7f3ad41030b0 2024-03-15T12:58:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.232+0000 7f3ad1ffb640 1 -- 172.21.15.2:0/1481055012 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f3acc00a040 con 0x7f3ad41030b0 2024-03-15T12:58:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.233+0000 7f3add98b740 1 Processor -- start 2024-03-15T12:58:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.233+0000 7f3add98b740 1 -- start start 2024-03-15T12:58:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.233+0000 7f3add98b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e083015f80 0x55e0830163a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.233+0000 7f3add98b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e083016970 0x55e08301b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.233+0000 7f3add98b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e08301bae0 0x55e08301df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.233+0000 7f3add98b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e082fd1cd0 con 0x55e08301bae0 2024-03-15T12:58:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.233+0000 7f3add98b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e082f84f20 con 0x55e083015f80 2024-03-15T12:58:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.233+0000 7f3add98b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e082f82450 con 0x55e083016970 2024-03-15T12:58:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.233+0000 7f3ad1ffb640 1 --2- 172.21.15.2:0/1481055012 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3ac00a1a30 0x7f3ac00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.233+0000 7f3ad3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e08301bae0 0x55e08301df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.233+0000 7f3ad3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e08301bae0 0x55e08301df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47840/0 (socket says 172.21.15.2:47840) 2024-03-15T12:58:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.233+0000 7f3ad3fff640 1 -- 172.21.15.2:0/1991943540 learned_addr learned my addr 172.21.15.2:0/1991943540 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.233+0000 7f3ad8b51640 1 --2- 172.21.15.2:0/1991943540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e083016970 0x55e08301b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.233+0000 7f3ad3fff640 1 --2- 172.21.15.2:0/1991943540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e083015f80 0x55e0830163a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.233+0000 7f3ad9352640 1 --2- 172.21.15.2:0/1481055012 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3ac00a1a30 0x7f3ac00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.234+0000 7f3ad1ffb640 1 -- 172.21.15.2:0/1481055012 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f3acc0f0520 con 0x7f3ad41030b0 2024-03-15T12:58:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.234+0000 7f3ad8b51640 1 -- 172.21.15.2:0/1991943540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e083015f80 msgr2=0x55e0830163a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.234+0000 7f3ad8b51640 1 --2- 172.21.15.2:0/1991943540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e083015f80 0x55e0830163a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.234+0000 7f3ad8b51640 1 -- 172.21.15.2:0/1991943540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e08301bae0 msgr2=0x55e08301df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.234+0000 7f3ad8b51640 1 --2- 172.21.15.2:0/1991943540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e08301bae0 0x55e08301df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.234+0000 7f3ad8b51640 1 -- 172.21.15.2:0/1991943540 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e082f32f20 con 0x55e083016970 2024-03-15T12:58:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.234+0000 7f3ad9352640 1 --2- 172.21.15.2:0/1481055012 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3ac00a1a30 0x7f3ac00a3ef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0b9ac0 tx=0x7f3acc02c0f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.234+0000 7f3ad8b51640 1 --2- 172.21.15.2:0/1991943540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e083016970 0x55e08301b470 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009bf0 tx=0x7f3abc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f5d1a4b555ff6cd server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.234+0000 7f3acaffd640 1 -- 172.21.15.2:0/1991943540 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc004030 con 0x55e083016970 2024-03-15T12:58:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.234+0000 7f3acaffd640 1 -- 172.21.15.2:0/1991943540 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3abc005020 con 0x55e083016970 2024-03-15T12:58:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.234+0000 7f3acaffd640 1 -- 172.21.15.2:0/1991943540 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc004d10 con 0x55e083016970 2024-03-15T12:58:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.235+0000 7f3add98b740 1 -- 172.21.15.2:0/1991943540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e083016970 msgr2=0x55e08301b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.235+0000 7f3add98b740 1 --2- 172.21.15.2:0/1991943540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e083016970 0x55e08301b470 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009bf0 tx=0x7f3abc011810 comp rx=0 tx=0).stop 2024-03-15T12:58:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.235+0000 7f3add98b740 1 -- 172.21.15.2:0/1991943540 shutdown_connections 2024-03-15T12:58:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.235+0000 7f3add98b740 1 --2- 172.21.15.2:0/1991943540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e08301bae0 0x55e08301df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.235+0000 7f3add98b740 1 --2- 172.21.15.2:0/1991943540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e083016970 0x55e08301b470 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.235+0000 7f3add98b740 1 --2- 172.21.15.2:0/1991943540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e083015f80 0x55e0830163a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.235+0000 7f3add98b740 1 -- 172.21.15.2:0/1991943540 >> 172.21.15.2:0/1991943540 conn(0x55e0830114f0 msgr2=0x55e0830118f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.235+0000 7f3add98b740 1 -- 172.21.15.2:0/1991943540 shutdown_connections 2024-03-15T12:58:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.235+0000 7f3add98b740 1 -- 172.21.15.2:0/1991943540 wait complete. 2024-03-15T12:58:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.235+0000 7f3add98b740 1 -- start start 2024-03-15T12:58:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.235+0000 7f3add98b740 1 Processor -- start 2024-03-15T12:58:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3add98b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e083015f80 0x55e083020170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3add98b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083016970 0x55e0830206b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3add98b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e08301bae0 0x55e0830276f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3add98b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e082f7dc30 con 0x55e083016970 2024-03-15T12:58:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3add98b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e082f22d30 con 0x55e083015f80 2024-03-15T12:58:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3add98b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e082c51310 con 0x55e08301bae0 2024-03-15T12:58:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3add98b740 10 client.0 _ll_register_callbacks cb 0x55e082f2ad40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3ad8b51640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083016970 0x55e0830206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3ad8b51640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083016970 0x55e0830206b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47854/0 (socket says 172.21.15.2:47854) 2024-03-15T12:58:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3ad8b51640 1 -- 172.21.15.2:0/3918737645 learned_addr learned my addr 172.21.15.2:0/3918737645 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3ad3fff640 1 --2- 172.21.15.2:0/3918737645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e083015f80 0x55e083020170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3ad3fff640 1 --2- 172.21.15.2:0/3918737645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e08301bae0 0x55e0830276f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3ad8b51640 1 -- 172.21.15.2:0/3918737645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e08301bae0 msgr2=0x55e0830276f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3ad8b51640 1 --2- 172.21.15.2:0/3918737645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e08301bae0 0x55e0830276f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3ad8b51640 1 -- 172.21.15.2:0/3918737645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e083015f80 msgr2=0x55e083020170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3ad8b51640 1 --2- 172.21.15.2:0/3918737645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e083015f80 0x55e083020170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3ad8b51640 1 -- 172.21.15.2:0/3918737645 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3abc009850 con 0x55e083016970 2024-03-15T12:58:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3ad8b51640 1 --2- 172.21.15.2:0/3918737645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083016970 0x55e0830206b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3abc004780 tx=0x7f3abc009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47594d64d26c1c55 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3ab3fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3ab3fff640 1 -- 172.21.15.2:0/3918737645 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc004c50 con 0x55e083016970 2024-03-15T12:58:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3ab3fff640 1 -- 172.21.15.2:0/3918737645 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f3abc004db0 con 0x55e083016970 2024-03-15T12:58:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3add98b740 1 -- 172.21.15.2:0/3918737645 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e082c7d6c0 con 0x55e083016970 2024-03-15T12:58:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3ab3fff640 1 -- 172.21.15.2:0/3918737645 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3abc004040 con 0x55e083016970 2024-03-15T12:58:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3add98b740 10 client.15240 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.236+0000 7f3add98b740 1 -- 172.21.15.2:0/3918737645 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e082f7c2b0 con 0x55e083016970 2024-03-15T12:58:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.237+0000 7f3ab3fff640 1 -- 172.21.15.2:0/3918737645 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e082c7d6c0 con 0x55e083016970 2024-03-15T12:58:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.237+0000 7f3ab3fff640 1 -- 172.21.15.2:0/3918737645 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3abc019430 con 0x55e083016970 2024-03-15T12:58:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.237+0000 7f3add98b740 10 client.15240 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.237+0000 7f3add98b740 10 client.15240 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-15T12:58:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.237+0000 7f3add98b740 20 client.15240 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.237+0000 7f3add98b740 1 --2- 172.21.15.2:0/3918737645 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55e08302c3e0 0x55e08302bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.237+0000 7f3add98b740 4 client.15240 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:58:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.237+0000 7f3add98b740 1 -- 172.21.15.2:0/3918737645 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e082cce270 con 0x55e08302c3e0 2024-03-15T12:58:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.237+0000 7f3ad3fff640 1 --2- 172.21.15.2:0/3918737645 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55e08302c3e0 0x55e08302bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.238+0000 7f3ab3fff640 10 client.15240 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T12:58:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.238+0000 7f3ad3fff640 1 --2- 172.21.15.2:0/3918737645 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55e08302c3e0 0x55e08302bc90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=52589e0d267fb1d2 server_cookie=fd8d890138093527 in_seq=0 out_seq=0 2024-03-15T12:58:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.238+0000 7f3ab3fff640 1 -- 172.21.15.2:0/3918737645 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e082cce270 con 0x55e08302c3e0 2024-03-15T12:58:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.238+0000 7f3ab3fff640 10 client.15240 handle_command_reply: tid=0 2024-03-15T12:58:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.238+0000 7f3add98b740 1 client.15240 shutdown 2024-03-15T12:58:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.238+0000 7f3add98b740 1 -- 172.21.15.2:0/3918737645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083016970 msgr2=0x55e0830206b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.238+0000 7f3add98b740 1 --2- 172.21.15.2:0/3918737645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083016970 0x55e0830206b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3abc004780 tx=0x7f3abc009fd0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.238+0000 7f3add98b740 1 -- 172.21.15.2:0/3918737645 shutdown_connections 2024-03-15T12:58:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.238+0000 7f3add98b740 1 --2- 172.21.15.2:0/3918737645 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55e08302c3e0 0x55e08302bc90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.238+0000 7f3add98b740 1 --2- 172.21.15.2:0/3918737645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e08301bae0 0x55e0830276f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.238+0000 7f3add98b740 1 --2- 172.21.15.2:0/3918737645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083016970 0x55e0830206b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.238+0000 7f3ab3fff640 0 client.15240 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T12:58:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.238+0000 7f3add98b740 1 --2- 172.21.15.2:0/3918737645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e083015f80 0x55e083020170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.238+0000 7f3add98b740 1 -- 172.21.15.2:0/3918737645 >> 172.21.15.2:0/3918737645 conn(0x55e0830114f0 msgr2=0x55e0830107d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.239+0000 7f3add98b740 1 -- 172.21.15.2:0/3918737645 shutdown_connections 2024-03-15T12:58:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.239+0000 7f3add98b740 1 -- 172.21.15.2:0/3918737645 wait complete. 2024-03-15T12:58:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.239+0000 7f3add98b740 20 client.15240 trim_cache size 0 max 16384 2024-03-15T12:58:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.255+0000 7f3add98b740 1 Processor -- start 2024-03-15T12:58:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.255+0000 7f3add98b740 1 -- start start 2024-03-15T12:58:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.255+0000 7f3add98b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e083015f80 0x55e08301ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.255+0000 7f3add98b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083016970 0x55e0830276b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.255+0000 7f3add98b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e08301bae0 0x55e0830118d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.255+0000 7f3add98b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e082f03490 con 0x55e083016970 2024-03-15T12:58:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.255+0000 7f3add98b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083028a70 con 0x55e083015f80 2024-03-15T12:58:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.255+0000 7f3add98b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083020d80 con 0x55e08301bae0 2024-03-15T12:58:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.256+0000 7f3ad8b51640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083016970 0x55e0830276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.256+0000 7f3ad8b51640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083016970 0x55e0830276b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47856/0 (socket says 172.21.15.2:47856) 2024-03-15T12:58:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.256+0000 7f3ad8b51640 1 -- 172.21.15.2:0/2774500597 learned_addr learned my addr 172.21.15.2:0/2774500597 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.256+0000 7f3ad3fff640 1 --2- 172.21.15.2:0/2774500597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e083015f80 0x55e08301ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.256+0000 7f3ad3fff640 1 --2- 172.21.15.2:0/2774500597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e08301bae0 0x55e0830118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.256+0000 7f3ad8b51640 1 -- 172.21.15.2:0/2774500597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e08301bae0 msgr2=0x55e0830118d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.256+0000 7f3ad8b51640 1 --2- 172.21.15.2:0/2774500597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e08301bae0 0x55e0830118d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.256+0000 7f3ad8b51640 1 -- 172.21.15.2:0/2774500597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e083015f80 msgr2=0x55e08301ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.256+0000 7f3ad8b51640 1 --2- 172.21.15.2:0/2774500597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e083015f80 0x55e08301ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.256+0000 7f3ad8b51640 1 -- 172.21.15.2:0/2774500597 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e082f18a20 con 0x55e083016970 2024-03-15T12:58:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.256+0000 7f3ad8b51640 1 --2- 172.21.15.2:0/2774500597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083016970 0x55e0830276b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3abc000c00 tx=0x7f3abc009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb41c24d47773de server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.256+0000 7f3aca7fc640 1 -- 172.21.15.2:0/2774500597 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0055c0 con 0x55e083016970 2024-03-15T12:58:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.256+0000 7f3aca7fc640 1 -- 172.21.15.2:0/2774500597 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3abc01ae90 con 0x55e083016970 2024-03-15T12:58:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.256+0000 7f3aca7fc640 1 -- 172.21.15.2:0/2774500597 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc004aa0 con 0x55e083016970 2024-03-15T12:58:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.257+0000 7f3add98b740 1 -- 172.21.15.2:0/2774500597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083016970 msgr2=0x55e0830276b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.257+0000 7f3add98b740 1 --2- 172.21.15.2:0/2774500597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083016970 0x55e0830276b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3abc000c00 tx=0x7f3abc009a70 comp rx=0 tx=0).stop 2024-03-15T12:58:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.257+0000 7f3add98b740 1 -- 172.21.15.2:0/2774500597 shutdown_connections 2024-03-15T12:58:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.257+0000 7f3add98b740 1 --2- 172.21.15.2:0/2774500597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e08301bae0 0x55e0830118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.257+0000 7f3add98b740 1 --2- 172.21.15.2:0/2774500597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083016970 0x55e0830276b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.257+0000 7f3add98b740 1 --2- 172.21.15.2:0/2774500597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e083015f80 0x55e08301ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.257+0000 7f3add98b740 1 -- 172.21.15.2:0/2774500597 >> 172.21.15.2:0/2774500597 conn(0x55e0830114f0 msgr2=0x55e0830173e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.257+0000 7f3add98b740 1 -- 172.21.15.2:0/2774500597 shutdown_connections 2024-03-15T12:58:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.257+0000 7f3add98b740 1 -- 172.21.15.2:0/2774500597 wait complete. 2024-03-15T12:58:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.257+0000 7f3add98b740 1 -- start start 2024-03-15T12:58:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.257+0000 7f3add98b740 1 Processor -- start 2024-03-15T12:58:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3add98b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083015f80 0x55e083027180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3add98b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e083016970 0x55e0830276c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3add98b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e08301bae0 0x55e083023cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3add98b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0830243f0 con 0x55e083015f80 2024-03-15T12:58:05.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3add98b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083024560 con 0x55e08301bae0 2024-03-15T12:58:05.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3add98b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e083024720 con 0x55e083016970 2024-03-15T12:58:05.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3add98b740 10 client.0 _ll_register_callbacks cb 0x7f3ac4009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:05.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3ad3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083015f80 0x55e083027180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3ad3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083015f80 0x55e083027180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47860/0 (socket says 172.21.15.2:47860) 2024-03-15T12:58:05.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3ad3fff640 1 -- 172.21.15.2:0/4279645241 learned_addr learned my addr 172.21.15.2:0/4279645241 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3ad8b51640 1 --2- 172.21.15.2:0/4279645241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e083016970 0x55e0830276c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3ad3fff640 1 --2- 172.21.15.2:0/4279645241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e08301bae0 0x55e083023cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3ad3fff640 1 -- 172.21.15.2:0/4279645241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e083016970 msgr2=0x55e0830276c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3ad3fff640 1 --2- 172.21.15.2:0/4279645241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e083016970 0x55e0830276c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3ad3fff640 1 -- 172.21.15.2:0/4279645241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e08301bae0 msgr2=0x55e083023cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3ad3fff640 1 --2- 172.21.15.2:0/4279645241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e08301bae0 0x55e083023cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3ad3fff640 1 -- 172.21.15.2:0/4279645241 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e082f7c2b0 con 0x55e083015f80 2024-03-15T12:58:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3ad3fff640 1 --2- 172.21.15.2:0/4279645241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083015f80 0x55e083027180 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4005e00 tx=0x7f3ac4028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f937438d5449cf server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3ab3fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3ab3fff640 1 -- 172.21.15.2:0/4279645241 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4016040 con 0x55e083015f80 2024-03-15T12:58:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3ab3fff640 1 -- 172.21.15.2:0/4279645241 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f3ac400fb60 con 0x55e083015f80 2024-03-15T12:58:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3add98b740 1 -- 172.21.15.2:0/4279645241 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e08302da50 con 0x55e083015f80 2024-03-15T12:58:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3ab3fff640 1 -- 172.21.15.2:0/4279645241 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ac400fd20 con 0x55e083015f80 2024-03-15T12:58:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3add98b740 10 client.15246 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.258+0000 7f3add98b740 1 -- 172.21.15.2:0/4279645241 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e082f3e780 con 0x55e083015f80 2024-03-15T12:58:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.259+0000 7f3ab3fff640 1 -- 172.21.15.2:0/4279645241 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e08302da50 con 0x55e083015f80 2024-03-15T12:58:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.259+0000 7f3ab3fff640 1 -- 172.21.15.2:0/4279645241 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3ac4021430 con 0x55e083015f80 2024-03-15T12:58:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.259+0000 7f3add98b740 10 client.15246 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.259+0000 7f3add98b740 10 client.15246 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-15T12:58:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.259+0000 7f3add98b740 20 client.15246 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.259+0000 7f3add98b740 1 --2- 172.21.15.2:0/4279645241 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55e08308cab0 0x55e083088ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.259+0000 7f3add98b740 4 client.15246 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-15T12:58:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.259+0000 7f3add98b740 1 -- 172.21.15.2:0/4279645241 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f3abc005020 con 0x55e08308cab0 2024-03-15T12:58:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.259+0000 7f3ad8b51640 1 --2- 172.21.15.2:0/4279645241 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55e08308cab0 0x55e083088ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.259+0000 7f3ad8b51640 1 --2- 172.21.15.2:0/4279645241 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55e08308cab0 0x55e083088ef0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2e800514739ba86c server_cookie=98a0d86eed41c530 in_seq=0 out_seq=0 2024-03-15T12:58:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.259+0000 7f3ab3fff640 10 client.15246 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T12:58:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3ab3fff640 1 -- 172.21.15.2:0/4279645241 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f3abc005020 con 0x55e08308cab0 2024-03-15T12:58:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3ab3fff640 10 client.15246 handle_command_reply: tid=0 2024-03-15T12:58:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3add98b740 1 client.15246 shutdown 2024-03-15T12:58:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3add98b740 1 -- 172.21.15.2:0/4279645241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083015f80 msgr2=0x55e083027180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3add98b740 1 --2- 172.21.15.2:0/4279645241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083015f80 0x55e083027180 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4005e00 tx=0x7f3ac4028040 comp rx=0 tx=0).stop 2024-03-15T12:58:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3add98b740 1 -- 172.21.15.2:0/4279645241 shutdown_connections 2024-03-15T12:58:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3add98b740 1 --2- 172.21.15.2:0/4279645241 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55e08308cab0 0x55e083088ef0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3add98b740 1 --2- 172.21.15.2:0/4279645241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e08301bae0 0x55e083023cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3ab3fff640 0 client.15246 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T12:58:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3add98b740 1 --2- 172.21.15.2:0/4279645241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e083016970 0x55e0830276c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3add98b740 1 --2- 172.21.15.2:0/4279645241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e083015f80 0x55e083027180 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3add98b740 1 -- 172.21.15.2:0/4279645241 >> 172.21.15.2:0/4279645241 conn(0x55e0830114f0 msgr2=0x55e083017200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3add98b740 1 -- 172.21.15.2:0/4279645241 shutdown_connections 2024-03-15T12:58:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3add98b740 1 -- 172.21.15.2:0/4279645241 wait complete. 2024-03-15T12:58:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3add98b740 20 client.15246 trim_cache size 0 max 16384 2024-03-15T12:58:05.253 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:58:05.253 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "2767718a-e2ca-11ee-95c9-87774f69a715", 2024-03-15T12:58:05.253 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 4, 2024-03-15T12:58:05.254 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24434, 2024-03-15T12:58:05.254 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-03-15T12:58:05.254 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-03-15T12:58:05.254 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-03-15T12:58:05.254 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 82.841937231000003, 2024-03-15T12:58:05.254 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 28, 2024-03-15T12:58:05.254 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 91, 2024-03-15T12:58:05.254 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 89, 2024-03-15T12:58:05.254 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 139.51108198899999 2024-03-15T12:58:05.254 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:58:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3adadb5640 1 -- 172.21.15.2:0/1481055012 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3ac00a1a30 msgr2=0x7f3ac00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3adadb5640 1 --2- 172.21.15.2:0/1481055012 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3ac00a1a30 0x7f3ac00a3ef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0b9ac0 tx=0x7f3acc02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3adadb5640 1 -- 172.21.15.2:0/1481055012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad41030b0 msgr2=0x7f3ad419eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3adadb5640 1 --2- 172.21.15.2:0/1481055012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad41030b0 0x7f3ad419eb10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3acc005cc0 tx=0x7f3acc032a10 comp rx=0 tx=0).stop 2024-03-15T12:58:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3adadb5640 1 -- 172.21.15.2:0/1481055012 shutdown_connections 2024-03-15T12:58:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3adadb5640 1 --2- 172.21.15.2:0/1481055012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ad41030b0 0x7f3ad419eb10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3adadb5640 1 --2- 172.21.15.2:0/1481055012 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3ac00a1a30 0x7f3ac00a3ef0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3adadb5640 1 --2- 172.21.15.2:0/1481055012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ad41026f0 0x7f3ad4197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.260+0000 7f3adadb5640 1 --2- 172.21.15.2:0/1481055012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ad4101500 0x7f3ad4197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.261+0000 7f3adadb5640 1 -- 172.21.15.2:0/1481055012 >> 172.21.15.2:0/1481055012 conn(0x7f3ad40fca70 msgr2=0x7f3ad4105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.261+0000 7f3adadb5640 1 -- 172.21.15.2:0/1481055012 shutdown_connections 2024-03-15T12:58:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.261+0000 7f3adadb5640 1 -- 172.21.15.2:0/1481055012 wait complete. 2024-03-15T12:58:05.257 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-15T12:58:05.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.375+0000 7f4b26239640 1 -- 172.21.15.2:0/861260828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20101fc0 msgr2=0x7f4b20108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.375+0000 7f4b26239640 1 --2- 172.21.15.2:0/861260828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20101fc0 0x7f4b20108850 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009ca0 tx=0x7f4b1802c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.375+0000 7f4b26239640 1 -- 172.21.15.2:0/861260828 shutdown_connections 2024-03-15T12:58:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.375+0000 7f4b26239640 1 --2- 172.21.15.2:0/861260828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20101fc0 0x7f4b20108850 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.375+0000 7f4b26239640 1 --2- 172.21.15.2:0/861260828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b20101600 0x7f4b20101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.375+0000 7f4b26239640 1 --2- 172.21.15.2:0/861260828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b20100410 0x7f4b20100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.375+0000 7f4b26239640 1 -- 172.21.15.2:0/861260828 >> 172.21.15.2:0/861260828 conn(0x7f4b200fb9a0 msgr2=0x7f4b200fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.375+0000 7f4b26239640 1 -- 172.21.15.2:0/861260828 shutdown_connections 2024-03-15T12:58:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.375+0000 7f4b26239640 1 -- 172.21.15.2:0/861260828 wait complete. 2024-03-15T12:58:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b26239640 1 Processor -- start 2024-03-15T12:58:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b26239640 1 -- start start 2024-03-15T12:58:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b26239640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20100410 0x7f4b201043a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b26239640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b20101600 0x7f4b201029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b26239640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b20101fc0 0x7f4b20102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b26239640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b20103680 con 0x7f4b20100410 2024-03-15T12:58:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b26239640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201037f0 con 0x7f4b20101600 2024-03-15T12:58:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b26239640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201a1580 con 0x7f4b20101fc0 2024-03-15T12:58:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b1f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20100410 0x7f4b201043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b1f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20100410 0x7f4b201043a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47874/0 (socket says 172.21.15.2:47874) 2024-03-15T12:58:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b1f7fe640 1 -- 172.21.15.2:0/3509961519 learned_addr learned my addr 172.21.15.2:0/3509961519 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b1effd640 1 --2- 172.21.15.2:0/3509961519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b20101600 0x7f4b201029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b1ffff640 1 --2- 172.21.15.2:0/3509961519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b20101fc0 0x7f4b20102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b1f7fe640 1 -- 172.21.15.2:0/3509961519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b20101fc0 msgr2=0x7f4b20102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b1f7fe640 1 --2- 172.21.15.2:0/3509961519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b20101fc0 0x7f4b20102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b1f7fe640 1 -- 172.21.15.2:0/3509961519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b20101600 msgr2=0x7f4b201029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b1f7fe640 1 --2- 172.21.15.2:0/3509961519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b20101600 0x7f4b201029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.376+0000 7f4b1f7fe640 1 -- 172.21.15.2:0/3509961519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b18009900 con 0x7f4b20100410 2024-03-15T12:58:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.377+0000 7f4b1f7fe640 1 --2- 172.21.15.2:0/3509961519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20100410 0x7f4b201043a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000e9f0 tx=0x7f4b1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.377+0000 7f4b1cff9640 1 -- 172.21.15.2:0/3509961519 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10009700 con 0x7f4b20100410 2024-03-15T12:58:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.377+0000 7f4b26239640 1 -- 172.21.15.2:0/3509961519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b201a1830 con 0x7f4b20100410 2024-03-15T12:58:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.377+0000 7f4b1cff9640 1 -- 172.21.15.2:0/3509961519 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4b100044f0 con 0x7f4b20100410 2024-03-15T12:58:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.377+0000 7f4b1cff9640 1 -- 172.21.15.2:0/3509961519 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10010430 con 0x7f4b20100410 2024-03-15T12:58:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.377+0000 7f4b26239640 1 -- 172.21.15.2:0/3509961519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b201a1d40 con 0x7f4b20100410 2024-03-15T12:58:05.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.379+0000 7f4b1cff9640 1 -- 172.21.15.2:0/3509961519 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f4b1000cd80 con 0x7f4b20100410 2024-03-15T12:58:05.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b28e18740 1 Processor -- start 2024-03-15T12:58:05.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b28e18740 1 -- start start 2024-03-15T12:58:05.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b28e18740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68312f80 0x55ea683133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b28e18740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68313970 0x55ea68318470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b28e18740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea6831af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b28e18740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea682cecd0 con 0x55ea68318ae0 2024-03-15T12:58:05.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b28e18740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea68281f20 con 0x55ea68312f80 2024-03-15T12:58:05.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b28e18740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea6827f450 con 0x55ea68313970 2024-03-15T12:58:05.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b1effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea6831af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b1ffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68312f80 0x55ea683133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b1effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea6831af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47878/0 (socket says 172.21.15.2:47878) 2024-03-15T12:58:05.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b1effd640 1 -- 172.21.15.2:0/2338646770 learned_addr learned my addr 172.21.15.2:0/2338646770 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b1cff9640 1 --2- 172.21.15.2:0/3509961519 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4b080a1a30 0x7f4b080a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b1f7fe640 1 --2- 172.21.15.2:0/2338646770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68313970 0x55ea68318470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b1ffff640 1 --2- 172.21.15.2:0/3509961519 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4b080a1a30 0x7f4b080a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b1effd640 1 -- 172.21.15.2:0/2338646770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68313970 msgr2=0x55ea68318470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b1effd640 1 --2- 172.21.15.2:0/2338646770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68313970 0x55ea68318470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b1effd640 1 -- 172.21.15.2:0/2338646770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68312f80 msgr2=0x55ea683133a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b1effd640 1 --2- 172.21.15.2:0/2338646770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68312f80 0x55ea683133a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b1effd640 1 -- 172.21.15.2:0/2338646770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea6822ff20 con 0x55ea68318ae0 2024-03-15T12:58:05.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b1ffff640 1 --2- 172.21.15.2:0/2338646770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68312f80 0x55ea683133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:05.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.380+0000 7f4b1cff9640 1 -- 172.21.15.2:0/3509961519 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f4b10014070 con 0x7f4b20100410 2024-03-15T12:58:05.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.381+0000 7f4b1effd640 1 --2- 172.21.15.2:0/2338646770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea6831af70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009ca0 tx=0x7f4b14011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a18e677895dc0c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.381+0000 7f4b1ffff640 1 --2- 172.21.15.2:0/3509961519 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4b080a1a30 0x7f4b080a3ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4b18005cc0 tx=0x7f4b1802c0f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.381+0000 7f4b05ffb640 1 -- 172.21.15.2:0/2338646770 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14019630 con 0x55ea68318ae0 2024-03-15T12:58:05.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.381+0000 7f4b05ffb640 1 -- 172.21.15.2:0/2338646770 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4b14019c60 con 0x55ea68318ae0 2024-03-15T12:58:05.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.381+0000 7f4b05ffb640 1 -- 172.21.15.2:0/2338646770 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1400eb40 con 0x55ea68318ae0 2024-03-15T12:58:05.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.381+0000 7f4b28e18740 1 -- 172.21.15.2:0/2338646770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 msgr2=0x55ea6831af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.381+0000 7f4b28e18740 1 --2- 172.21.15.2:0/2338646770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea6831af70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009ca0 tx=0x7f4b14011810 comp rx=0 tx=0).stop 2024-03-15T12:58:05.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 1 -- 172.21.15.2:0/2338646770 shutdown_connections 2024-03-15T12:58:05.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 1 --2- 172.21.15.2:0/2338646770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea6831af70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 1 --2- 172.21.15.2:0/2338646770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68313970 0x55ea68318470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 1 --2- 172.21.15.2:0/2338646770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68312f80 0x55ea683133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 1 -- 172.21.15.2:0/2338646770 >> 172.21.15.2:0/2338646770 conn(0x55ea6830e4f0 msgr2=0x55ea6830e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 1 -- 172.21.15.2:0/2338646770 shutdown_connections 2024-03-15T12:58:05.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 1 -- 172.21.15.2:0/2338646770 wait complete. 2024-03-15T12:58:05.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 1 -- start start 2024-03-15T12:58:05.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 1 Processor -- start 2024-03-15T12:58:05.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68312f80 0x55ea6831d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68313970 0x55ea6831d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea683246f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea6827ac30 con 0x55ea68318ae0 2024-03-15T12:58:05.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea6821fd30 con 0x55ea68313970 2024-03-15T12:58:05.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea67f4e310 con 0x55ea68312f80 2024-03-15T12:58:05.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b1ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea683246f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b28e18740 10 client.0 _ll_register_callbacks cb 0x55ea68227d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:05.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b1effd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68313970 0x55ea6831d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b1ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea683246f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47884/0 (socket says 172.21.15.2:47884) 2024-03-15T12:58:05.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b1f7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68312f80 0x55ea6831d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.382+0000 7f4b1ffff640 1 -- 172.21.15.2:0/3939698822 learned_addr learned my addr 172.21.15.2:0/3939698822 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.383+0000 7f4b1effd640 1 -- 172.21.15.2:0/3939698822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68312f80 msgr2=0x55ea6831d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.383+0000 7f4b1effd640 1 --2- 172.21.15.2:0/3939698822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68312f80 0x55ea6831d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.383+0000 7f4b1effd640 1 -- 172.21.15.2:0/3939698822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 msgr2=0x55ea683246f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.383+0000 7f4b1effd640 1 --2- 172.21.15.2:0/3939698822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea683246f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.383+0000 7f4b1effd640 1 -- 172.21.15.2:0/3939698822 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b140098d0 con 0x55ea68313970 2024-03-15T12:58:05.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.383+0000 7f4b1ffff640 1 --2- 172.21.15.2:0/3939698822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea683246f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:05.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.383+0000 7f4b1effd640 1 --2- 172.21.15.2:0/3939698822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68313970 0x55ea6831d6b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4b1401c040 tx=0x7f4b1400ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de59ef4a4b7b224 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.383+0000 7f4afeffd640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T12:58:05.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.383+0000 7f4afeffd640 1 -- 172.21.15.2:0/3939698822 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14005050 con 0x55ea68313970 2024-03-15T12:58:05.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.383+0000 7f4b28e18740 1 -- 172.21.15.2:0/3939698822 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea67f7a6c0 con 0x55ea68313970 2024-03-15T12:58:05.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.383+0000 7f4afeffd640 1 -- 172.21.15.2:0/3939698822 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f4b140051b0 con 0x55ea68313970 2024-03-15T12:58:05.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.383+0000 7f4afeffd640 1 -- 172.21.15.2:0/3939698822 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b14005370 con 0x55ea68313970 2024-03-15T12:58:05.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.383+0000 7f4b28e18740 10 client.25130 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:05.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.383+0000 7f4b28e18740 1 -- 172.21.15.2:0/3939698822 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ea682792b0 con 0x55ea68313970 2024-03-15T12:58:05.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.384+0000 7f4afeffd640 1 -- 172.21.15.2:0/3939698822 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55ea67f7a6c0 con 0x55ea68313970 2024-03-15T12:58:05.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.384+0000 7f4afeffd640 1 -- 172.21.15.2:0/3939698822 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4b140048f0 con 0x55ea68313970 2024-03-15T12:58:05.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.384+0000 7f4b28e18740 10 client.25130 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:05.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.384+0000 7f4b28e18740 10 client.25130 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-15T12:58:05.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.384+0000 7f4b28e18740 20 client.25130 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:05.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.384+0000 7f4b28e18740 1 --2- 172.21.15.2:0/3939698822 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ea68328c90 0x55ea6832ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.384+0000 7f4b28e18740 4 client.25130 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:58:05.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.384+0000 7f4b28e18740 1 -- 172.21.15.2:0/3939698822 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea67fcb270 con 0x55ea68328c90 2024-03-15T12:58:05.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.384+0000 7f4b1f7fe640 1 --2- 172.21.15.2:0/3939698822 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ea68328c90 0x55ea6832ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.384+0000 7f4afeffd640 10 client.25130 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T12:58:05.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.384+0000 7f4b1f7fe640 1 --2- 172.21.15.2:0/3939698822 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ea68328c90 0x55ea6832ddc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4b408d89c9179a08 server_cookie=d21dfddf9598291f in_seq=0 out_seq=0 2024-03-15T12:58:05.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.385+0000 7f4afeffd640 1 -- 172.21.15.2:0/3939698822 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ea67fcb270 con 0x55ea68328c90 2024-03-15T12:58:05.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.385+0000 7f4afeffd640 10 client.25130 handle_command_reply: tid=0 2024-03-15T12:58:05.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.385+0000 7f4b28e18740 1 client.25130 shutdown 2024-03-15T12:58:05.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.385+0000 7f4b28e18740 1 -- 172.21.15.2:0/3939698822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68313970 msgr2=0x55ea6831d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.385+0000 7f4b28e18740 1 --2- 172.21.15.2:0/3939698822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68313970 0x55ea6831d6b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4b1401c040 tx=0x7f4b1400ef80 comp rx=0 tx=0).stop 2024-03-15T12:58:05.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.385+0000 7f4b28e18740 1 -- 172.21.15.2:0/3939698822 shutdown_connections 2024-03-15T12:58:05.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.385+0000 7f4b28e18740 1 --2- 172.21.15.2:0/3939698822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea683246f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.385+0000 7f4b28e18740 1 --2- 172.21.15.2:0/3939698822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68313970 0x55ea6831d6b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.385+0000 7f4b28e18740 1 --2- 172.21.15.2:0/3939698822 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ea68328c90 0x55ea6832ddc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.386+0000 7f4b28e18740 1 --2- 172.21.15.2:0/3939698822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68312f80 0x55ea6831d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.386+0000 7f4afeffd640 0 client.25130 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T12:58:05.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.386+0000 7f4b28e18740 1 -- 172.21.15.2:0/3939698822 >> 172.21.15.2:0/3939698822 conn(0x55ea6830e4f0 msgr2=0x55ea6830d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.386+0000 7f4b28e18740 1 -- 172.21.15.2:0/3939698822 shutdown_connections 2024-03-15T12:58:05.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.386+0000 7f4b28e18740 1 -- 172.21.15.2:0/3939698822 wait complete. 2024-03-15T12:58:05.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.386+0000 7f4b28e18740 20 client.25130 trim_cache size 0 max 16384 2024-03-15T12:58:05.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.402+0000 7f4b28e18740 1 Processor -- start 2024-03-15T12:58:05.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.402+0000 7f4b28e18740 1 -- start start 2024-03-15T12:58:05.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.402+0000 7f4b28e18740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68312f80 0x55ea6831aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.402+0000 7f4b28e18740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68313970 0x55ea683246b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.402+0000 7f4b28e18740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea68311fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.402+0000 7f4b28e18740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea68200490 con 0x55ea68318ae0 2024-03-15T12:58:05.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.402+0000 7f4b28e18740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea68325a70 con 0x55ea68313970 2024-03-15T12:58:05.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.402+0000 7f4b28e18740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea6830f4e0 con 0x55ea68312f80 2024-03-15T12:58:05.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.402+0000 7f4b1ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea68311fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.402+0000 7f4b1ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea68311fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47898/0 (socket says 172.21.15.2:47898) 2024-03-15T12:58:05.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.402+0000 7f4b1ffff640 1 -- 172.21.15.2:0/986694885 learned_addr learned my addr 172.21.15.2:0/986694885 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.403+0000 7f4b1f7fe640 1 --2- 172.21.15.2:0/986694885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68312f80 0x55ea6831aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.403+0000 7f4b1effd640 1 --2- 172.21.15.2:0/986694885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68313970 0x55ea683246b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.403+0000 7f4b1ffff640 1 -- 172.21.15.2:0/986694885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68312f80 msgr2=0x55ea6831aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.403+0000 7f4b1ffff640 1 --2- 172.21.15.2:0/986694885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68312f80 0x55ea6831aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.403+0000 7f4b1ffff640 1 -- 172.21.15.2:0/986694885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68313970 msgr2=0x55ea683246b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.403+0000 7f4b1ffff640 1 --2- 172.21.15.2:0/986694885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68313970 0x55ea683246b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.403+0000 7f4b1ffff640 1 -- 172.21.15.2:0/986694885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea68215a20 con 0x55ea68318ae0 2024-03-15T12:58:05.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.403+0000 7f4b1f7fe640 1 --2- 172.21.15.2:0/986694885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68312f80 0x55ea6831aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:05.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.403+0000 7f4b1ffff640 1 --2- 172.21.15.2:0/986694885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea68311fe0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4b18032860 tx=0x7f4b18046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e9e40dc75188b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.403+0000 7f4b057fa640 1 -- 172.21.15.2:0/986694885 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18003b70 con 0x55ea68318ae0 2024-03-15T12:58:05.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.403+0000 7f4b057fa640 1 -- 172.21.15.2:0/986694885 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4b18032ad0 con 0x55ea68318ae0 2024-03-15T12:58:05.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.403+0000 7f4b057fa640 1 -- 172.21.15.2:0/986694885 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180446a0 con 0x55ea68318ae0 2024-03-15T12:58:05.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 -- 172.21.15.2:0/986694885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 msgr2=0x55ea68311fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 --2- 172.21.15.2:0/986694885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea68311fe0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4b18032860 tx=0x7f4b18046000 comp rx=0 tx=0).stop 2024-03-15T12:58:05.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 -- 172.21.15.2:0/986694885 shutdown_connections 2024-03-15T12:58:05.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 --2- 172.21.15.2:0/986694885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea68311fe0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 --2- 172.21.15.2:0/986694885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68313970 0x55ea683246b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 --2- 172.21.15.2:0/986694885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68312f80 0x55ea6831aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 -- 172.21.15.2:0/986694885 >> 172.21.15.2:0/986694885 conn(0x55ea6830e4f0 msgr2=0x55ea6831bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 -- 172.21.15.2:0/986694885 shutdown_connections 2024-03-15T12:58:05.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 -- 172.21.15.2:0/986694885 wait complete. 2024-03-15T12:58:05.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 -- start start 2024-03-15T12:58:05.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 Processor -- start 2024-03-15T12:58:05.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68312f80 0x55ea683208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68313970 0x55ea68320e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea68323ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea68324010 con 0x55ea68318ae0 2024-03-15T12:58:05.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea68324180 con 0x55ea68312f80 2024-03-15T12:58:05.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea68324340 con 0x55ea68313970 2024-03-15T12:58:05.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.404+0000 7f4b28e18740 10 client.0 _ll_register_callbacks cb 0x7f4b1401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:05.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4b1ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea68323ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4b1ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea68323ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47910/0 (socket says 172.21.15.2:47910) 2024-03-15T12:58:05.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4b1ffff640 1 -- 172.21.15.2:0/3417562788 learned_addr learned my addr 172.21.15.2:0/3417562788 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4b1f7fe640 1 --2- 172.21.15.2:0/3417562788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68312f80 0x55ea683208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4b1effd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68313970 0x55ea68320e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4b1ffff640 1 -- 172.21.15.2:0/3417562788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68313970 msgr2=0x55ea68320e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4b1ffff640 1 --2- 172.21.15.2:0/3417562788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68313970 0x55ea68320e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4b1ffff640 1 -- 172.21.15.2:0/3417562788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68312f80 msgr2=0x55ea683208e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4b1ffff640 1 --2- 172.21.15.2:0/3417562788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68312f80 0x55ea683208e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4b1ffff640 1 -- 172.21.15.2:0/3417562788 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b201a1d40 con 0x55ea68318ae0 2024-03-15T12:58:05.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4b1effd640 1 --2- 172.21.15.2:0/3417562788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68313970 0x55ea68320e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:05.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4b1ffff640 1 --2- 172.21.15.2:0/3417562788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea68323ad0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4b180325b0 tx=0x7f4b180447b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5905f7f8c99f7ae4 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4afeffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:05.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4afeffd640 1 -- 172.21.15.2:0/3417562788 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18044db0 con 0x55ea68318ae0 2024-03-15T12:58:05.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4afeffd640 1 -- 172.21.15.2:0/3417562788 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f4b18043460 con 0x55ea68318ae0 2024-03-15T12:58:05.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4b28e18740 1 -- 172.21.15.2:0/3417562788 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea68329650 con 0x55ea68318ae0 2024-03-15T12:58:05.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4afeffd640 1 -- 172.21.15.2:0/3417562788 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b18004860 con 0x55ea68318ae0 2024-03-15T12:58:05.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4b28e18740 10 client.15264 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:05.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4b28e18740 1 -- 172.21.15.2:0/3417562788 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f4b18030070 con 0x55ea68318ae0 2024-03-15T12:58:05.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.405+0000 7f4afeffd640 1 -- 172.21.15.2:0/3417562788 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55ea68329650 con 0x55ea68318ae0 2024-03-15T12:58:05.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.406+0000 7f4afeffd640 1 -- 172.21.15.2:0/3417562788 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4b1803e040 con 0x55ea68318ae0 2024-03-15T12:58:05.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.406+0000 7f4b28e18740 10 client.15264 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:05.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.406+0000 7f4b28e18740 10 client.15264 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-15T12:58:05.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.406+0000 7f4b28e18740 20 client.15264 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:05.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.406+0000 7f4b28e18740 1 --2- 172.21.15.2:0/3417562788 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ea683895f0 0x55ea68385ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.406+0000 7f4b28e18740 4 client.15264 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-15T12:58:05.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.406+0000 7f4b28e18740 1 -- 172.21.15.2:0/3417562788 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ea6832a310 con 0x55ea683895f0 2024-03-15T12:58:05.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.406+0000 7f4b1f7fe640 1 --2- 172.21.15.2:0/3417562788 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ea683895f0 0x55ea68385ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.406+0000 7f4b1f7fe640 1 --2- 172.21.15.2:0/3417562788 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ea683895f0 0x55ea68385ad0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b76a7f6d040065d6 server_cookie=1e675121a8623aab in_seq=0 out_seq=0 2024-03-15T12:58:05.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.406+0000 7f4afeffd640 10 client.15264 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T12:58:05.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.406+0000 7f4afeffd640 1 -- 172.21.15.2:0/3417562788 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ea6832a310 con 0x55ea683895f0 2024-03-15T12:58:05.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.406+0000 7f4afeffd640 10 client.15264 handle_command_reply: tid=0 2024-03-15T12:58:05.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.406+0000 7f4b28e18740 1 client.15264 shutdown 2024-03-15T12:58:05.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b28e18740 1 -- 172.21.15.2:0/3417562788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 msgr2=0x55ea68323ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b28e18740 1 --2- 172.21.15.2:0/3417562788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea68323ad0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4b180325b0 tx=0x7f4b180447b0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b28e18740 1 -- 172.21.15.2:0/3417562788 shutdown_connections 2024-03-15T12:58:05.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b28e18740 1 --2- 172.21.15.2:0/3417562788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea68318ae0 0x55ea68323ad0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b28e18740 1 --2- 172.21.15.2:0/3417562788 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ea683895f0 0x55ea68385ad0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b28e18740 1 --2- 172.21.15.2:0/3417562788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ea68313970 0x55ea68320e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b28e18740 1 --2- 172.21.15.2:0/3417562788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea68312f80 0x55ea683208e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4afeffd640 0 client.15264 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T12:58:05.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b28e18740 1 -- 172.21.15.2:0/3417562788 >> 172.21.15.2:0/3417562788 conn(0x55ea6830e4f0 msgr2=0x55ea68311fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b28e18740 1 -- 172.21.15.2:0/3417562788 shutdown_connections 2024-03-15T12:58:05.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b28e18740 1 -- 172.21.15.2:0/3417562788 wait complete. 2024-03-15T12:58:05.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b28e18740 20 client.15264 trim_cache size 0 max 16384 2024-03-15T12:58:05.412 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:58:05.412 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "2767718a-e2ca-11ee-95c9-87774f69a715", 2024-03-15T12:58:05.412 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 3, 2024-03-15T12:58:05.412 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24460, 2024-03-15T12:58:05.412 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-03-15T12:58:05.412 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-03-15T12:58:05.413 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-03-15T12:58:05.413 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 83.993131013999999, 2024-03-15T12:58:05.413 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 28, 2024-03-15T12:58:05.413 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 91, 2024-03-15T12:58:05.413 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 89, 2024-03-15T12:58:05.413 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 128.27936575000001 2024-03-15T12:58:05.413 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:58:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b26239640 1 -- 172.21.15.2:0/3509961519 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4b080a1a30 msgr2=0x7f4b080a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b26239640 1 --2- 172.21.15.2:0/3509961519 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4b080a1a30 0x7f4b080a3ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4b18005cc0 tx=0x7f4b1802c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b26239640 1 -- 172.21.15.2:0/3509961519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20100410 msgr2=0x7f4b201043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b26239640 1 --2- 172.21.15.2:0/3509961519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20100410 0x7f4b201043a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000e9f0 tx=0x7f4b1000eeb0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b26239640 1 -- 172.21.15.2:0/3509961519 shutdown_connections 2024-03-15T12:58:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b26239640 1 --2- 172.21.15.2:0/3509961519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b20101fc0 0x7f4b20102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b26239640 1 --2- 172.21.15.2:0/3509961519 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f4b080a1a30 0x7f4b080a3ef0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b26239640 1 --2- 172.21.15.2:0/3509961519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b20101600 0x7f4b201029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b26239640 1 --2- 172.21.15.2:0/3509961519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20100410 0x7f4b201043a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b26239640 1 -- 172.21.15.2:0/3509961519 >> 172.21.15.2:0/3509961519 conn(0x7f4b200fb9a0 msgr2=0x7f4b200fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b26239640 1 -- 172.21.15.2:0/3509961519 shutdown_connections 2024-03-15T12:58:05.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.407+0000 7f4b26239640 1 -- 172.21.15.2:0/3509961519 wait complete. 2024-03-15T12:58:05.416 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-15T12:58:05.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.534+0000 7f614bd81640 1 -- 172.21.15.2:0/2982037677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61441015e0 msgr2=0x7f6144101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.534+0000 7f614bd81640 1 --2- 172.21.15.2:0/2982037677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61441015e0 0x7f6144101a60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6140009900 tx=0x7f614002bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.534+0000 7f614bd81640 1 -- 172.21.15.2:0/2982037677 shutdown_connections 2024-03-15T12:58:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.534+0000 7f614bd81640 1 --2- 172.21.15.2:0/2982037677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6144101fa0 0x7f6144108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.534+0000 7f614bd81640 1 --2- 172.21.15.2:0/2982037677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61441015e0 0x7f6144101a60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.534+0000 7f614bd81640 1 --2- 172.21.15.2:0/2982037677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61441003f0 0x7f6144100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.534+0000 7f614bd81640 1 -- 172.21.15.2:0/2982037677 >> 172.21.15.2:0/2982037677 conn(0x7f61440fb980 msgr2=0x7f61440fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.534+0000 7f614bd81640 1 -- 172.21.15.2:0/2982037677 shutdown_connections 2024-03-15T12:58:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.534+0000 7f614bd81640 1 -- 172.21.15.2:0/2982037677 wait complete. 2024-03-15T12:58:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.534+0000 7f614bd81640 1 Processor -- start 2024-03-15T12:58:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.534+0000 7f614bd81640 1 -- start start 2024-03-15T12:58:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614bd81640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61441003f0 0x7f61441975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614bd81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61441015e0 0x7f6144197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614bd81640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6144101fa0 0x7f614419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614bd81640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614419f0d0 con 0x7f61441015e0 2024-03-15T12:58:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614bd81640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614419f240 con 0x7f6144101fa0 2024-03-15T12:58:05.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614bd81640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614419f400 con 0x7f61441003f0 2024-03-15T12:58:05.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614931c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61441015e0 0x7f6144197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614931c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61441015e0 0x7f6144197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47930/0 (socket says 172.21.15.2:47930) 2024-03-15T12:58:05.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614931c640 1 -- 172.21.15.2:0/367778497 learned_addr learned my addr 172.21.15.2:0/367778497 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614a31e640 1 --2- 172.21.15.2:0/367778497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6144101fa0 0x7f614419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f6149b1d640 1 --2- 172.21.15.2:0/367778497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61441003f0 0x7f61441975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614931c640 1 -- 172.21.15.2:0/367778497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61441003f0 msgr2=0x7f61441975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614931c640 1 --2- 172.21.15.2:0/367778497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61441003f0 0x7f61441975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614931c640 1 -- 172.21.15.2:0/367778497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6144101fa0 msgr2=0x7f614419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614931c640 1 --2- 172.21.15.2:0/367778497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6144101fa0 0x7f614419eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614931c640 1 -- 172.21.15.2:0/367778497 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6140009560 con 0x7f61441015e0 2024-03-15T12:58:05.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614931c640 1 --2- 172.21.15.2:0/367778497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61441015e0 0x7f6144197b20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6140005ab0 tx=0x7f6140004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f6136ffd640 1 -- 172.21.15.2:0/367778497 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6140004250 con 0x7f61441015e0 2024-03-15T12:58:05.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614bd81640 1 -- 172.21.15.2:0/367778497 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614419f680 con 0x7f61441015e0 2024-03-15T12:58:05.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f6136ffd640 1 -- 172.21.15.2:0/367778497 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f61400043b0 con 0x7f61441015e0 2024-03-15T12:58:05.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f6136ffd640 1 -- 172.21.15.2:0/367778497 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6140003d20 con 0x7f61441015e0 2024-03-15T12:58:05.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.535+0000 7f614bd81640 1 -- 172.21.15.2:0/367778497 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614419fb90 con 0x7f61441015e0 2024-03-15T12:58:05.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.538+0000 7f6136ffd640 1 -- 172.21.15.2:0/367778497 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f6140005000 con 0x7f61441015e0 2024-03-15T12:58:05.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.538+0000 7f614e965740 1 Processor -- start 2024-03-15T12:58:05.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.538+0000 7f614e965740 1 -- start start 2024-03-15T12:58:05.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.538+0000 7f614e965740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6f6f80 0x5632bc6f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.538+0000 7f614e965740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 0x5632bc6fc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.538+0000 7f6136ffd640 1 --2- 172.21.15.2:0/367778497 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f61300a1a30 0x7f61300a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.538+0000 7f614e965740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6fcae0 0x5632bc6fef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.538+0000 7f614e965740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632bc6b2cd0 con 0x5632bc6f6f80 2024-03-15T12:58:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.538+0000 7f614e965740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632bc665f20 con 0x5632bc6fcae0 2024-03-15T12:58:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.538+0000 7f614e965740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632bc663450 con 0x5632bc6f7970 2024-03-15T12:58:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f614a31e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6f6f80 0x5632bc6f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f6149b1d640 1 --2- 172.21.15.2:0/367778497 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f61300a1a30 0x7f61300a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f614931c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 0x5632bc6fc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f614a31e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6f6f80 0x5632bc6f73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47934/0 (socket says 172.21.15.2:47934) 2024-03-15T12:58:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f614a31e640 1 -- 172.21.15.2:0/1425498805 learned_addr learned my addr 172.21.15.2:0/1425498805 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f6136ffd640 1 -- 172.21.15.2:0/367778497 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f61400ef940 con 0x7f61441015e0 2024-03-15T12:58:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f614a31e640 1 --2- 172.21.15.2:0/1425498805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6fcae0 0x5632bc6fef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f614931c640 1 -- 172.21.15.2:0/1425498805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6fcae0 msgr2=0x5632bc6fef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f614931c640 1 --2- 172.21.15.2:0/1425498805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6fcae0 0x5632bc6fef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f614931c640 1 -- 172.21.15.2:0/1425498805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6f6f80 msgr2=0x5632bc6f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f614931c640 1 --2- 172.21.15.2:0/1425498805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6f6f80 0x5632bc6f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f614931c640 1 -- 172.21.15.2:0/1425498805 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632bc613f20 con 0x5632bc6f7970 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f614931c640 1 --2- 172.21.15.2:0/1425498805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 0x5632bc6fc470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f61400ed150 tx=0x7f614002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=561c191044f40f32 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f6149b1d640 1 --2- 172.21.15.2:0/367778497 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f61300a1a30 0x7f61300a3ef0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6144101440 tx=0x7f6138005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f612ffff640 1 -- 172.21.15.2:0/1425498805 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61400bc340 con 0x5632bc6f7970 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f612ffff640 1 -- 172.21.15.2:0/1425498805 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f61400bc0b0 con 0x5632bc6f7970 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.539+0000 7f612ffff640 1 -- 172.21.15.2:0/1425498805 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61400dc0b0 con 0x5632bc6f7970 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.540+0000 7f614e965740 1 -- 172.21.15.2:0/1425498805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 msgr2=0x5632bc6fc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.540+0000 7f614e965740 1 --2- 172.21.15.2:0/1425498805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 0x5632bc6fc470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f61400ed150 tx=0x7f614002bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.540+0000 7f614e965740 1 -- 172.21.15.2:0/1425498805 shutdown_connections 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.540+0000 7f614e965740 1 --2- 172.21.15.2:0/1425498805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6fcae0 0x5632bc6fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.540+0000 7f614e965740 1 --2- 172.21.15.2:0/1425498805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 0x5632bc6fc470 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.540+0000 7f614e965740 1 --2- 172.21.15.2:0/1425498805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6f6f80 0x5632bc6f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.540+0000 7f614e965740 1 -- 172.21.15.2:0/1425498805 >> 172.21.15.2:0/1425498805 conn(0x5632bc6f24f0 msgr2=0x5632bc6f28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.540+0000 7f614e965740 1 -- 172.21.15.2:0/1425498805 shutdown_connections 2024-03-15T12:58:05.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.540+0000 7f614e965740 1 -- 172.21.15.2:0/1425498805 wait complete. 2024-03-15T12:58:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.540+0000 7f614e965740 1 -- start start 2024-03-15T12:58:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614e965740 1 Processor -- start 2024-03-15T12:58:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614e965740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6f6f80 0x5632bc700ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614e965740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 0x5632bc701410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614e965740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6fcae0 0x5632bc708450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614e965740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632bc65ec30 con 0x5632bc6fcae0 2024-03-15T12:58:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614e965740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632bc603d30 con 0x5632bc6f6f80 2024-03-15T12:58:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614e965740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632bc332310 con 0x5632bc6f7970 2024-03-15T12:58:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614e965740 10 client.0 _ll_register_callbacks cb 0x5632bc60bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614a31e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6fcae0 0x5632bc708450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f6149b1d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6f6f80 0x5632bc700ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614a31e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6fcae0 0x5632bc708450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47942/0 (socket says 172.21.15.2:47942) 2024-03-15T12:58:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614a31e640 1 -- 172.21.15.2:0/2758671713 learned_addr learned my addr 172.21.15.2:0/2758671713 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f6149b1d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6f6f80 0x5632bc700ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55886/0 (socket says 172.21.15.2:55886) 2024-03-15T12:58:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614931c640 1 --2- 172.21.15.2:0/2758671713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 0x5632bc701410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614a31e640 1 -- 172.21.15.2:0/2758671713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 msgr2=0x5632bc701410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614a31e640 1 --2- 172.21.15.2:0/2758671713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 0x5632bc701410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614a31e640 1 -- 172.21.15.2:0/2758671713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6f6f80 msgr2=0x5632bc700ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614a31e640 1 --2- 172.21.15.2:0/2758671713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6f6f80 0x5632bc700ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614a31e640 1 -- 172.21.15.2:0/2758671713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614419fb90 con 0x5632bc6fcae0 2024-03-15T12:58:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f614a31e640 1 --2- 172.21.15.2:0/2758671713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6fcae0 0x5632bc708450 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f613c013bb0 tx=0x7f613c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320aae69ee575f45 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.541+0000 7f612d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.542+0000 7f612d7fa640 1 -- 172.21.15.2:0/2758671713 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c005b50 con 0x5632bc6fcae0 2024-03-15T12:58:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.542+0000 7f612d7fa640 1 -- 172.21.15.2:0/2758671713 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f613c01f410 con 0x5632bc6fcae0 2024-03-15T12:58:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.542+0000 7f614e965740 1 -- 172.21.15.2:0/2758671713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632bc35e6c0 con 0x5632bc6fcae0 2024-03-15T12:58:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.542+0000 7f612d7fa640 1 -- 172.21.15.2:0/2758671713 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f613c01f5d0 con 0x5632bc6fcae0 2024-03-15T12:58:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.542+0000 7f614e965740 10 client.15276 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.542+0000 7f614e965740 1 -- 172.21.15.2:0/2758671713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5632bc65d2b0 con 0x5632bc6fcae0 2024-03-15T12:58:05.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.542+0000 7f612d7fa640 1 -- 172.21.15.2:0/2758671713 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5632bc35e6c0 con 0x5632bc6fcae0 2024-03-15T12:58:05.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.542+0000 7f612d7fa640 1 -- 172.21.15.2:0/2758671713 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f613c010880 con 0x5632bc6fcae0 2024-03-15T12:58:05.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.542+0000 7f614e965740 10 client.15276 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:05.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.542+0000 7f614e965740 10 client.15276 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-15T12:58:05.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.542+0000 7f614e965740 20 client.15276 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:05.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.542+0000 7f614e965740 1 --2- 172.21.15.2:0/2758671713 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5632bc70d820 0x5632bc711ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.542+0000 7f614e965740 4 client.15276 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:58:05.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.542+0000 7f614e965740 1 -- 172.21.15.2:0/2758671713 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632bc3af270 con 0x5632bc70d820 2024-03-15T12:58:05.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.542+0000 7f6149b1d640 1 --2- 172.21.15.2:0/2758671713 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5632bc70d820 0x5632bc711ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.543+0000 7f6149b1d640 1 --2- 172.21.15.2:0/2758671713 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5632bc70d820 0x5632bc711ca0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3db50fe7fd7b13a8 server_cookie=865583e0973655bb in_seq=0 out_seq=0 2024-03-15T12:58:05.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.543+0000 7f612d7fa640 10 client.15276 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T12:58:05.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.544+0000 7f612d7fa640 1 -- 172.21.15.2:0/2758671713 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5632bc3af270 con 0x5632bc70d820 2024-03-15T12:58:05.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.544+0000 7f612d7fa640 10 client.15276 handle_command_reply: tid=0 2024-03-15T12:58:05.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.544+0000 7f614e965740 1 client.15276 shutdown 2024-03-15T12:58:05.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.544+0000 7f614e965740 1 -- 172.21.15.2:0/2758671713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6fcae0 msgr2=0x5632bc708450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.544+0000 7f614e965740 1 --2- 172.21.15.2:0/2758671713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6fcae0 0x5632bc708450 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f613c013bb0 tx=0x7f613c004410 comp rx=0 tx=0).stop 2024-03-15T12:58:05.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.544+0000 7f614e965740 1 -- 172.21.15.2:0/2758671713 shutdown_connections 2024-03-15T12:58:05.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.544+0000 7f614e965740 1 --2- 172.21.15.2:0/2758671713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6fcae0 0x5632bc708450 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.544+0000 7f614e965740 1 --2- 172.21.15.2:0/2758671713 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5632bc70d820 0x5632bc711ca0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.544+0000 7f614e965740 1 --2- 172.21.15.2:0/2758671713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 0x5632bc701410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.544+0000 7f614e965740 1 --2- 172.21.15.2:0/2758671713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6f6f80 0x5632bc700ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.544+0000 7f612d7fa640 0 client.15276 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T12:58:05.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.544+0000 7f614e965740 1 -- 172.21.15.2:0/2758671713 >> 172.21.15.2:0/2758671713 conn(0x5632bc6f24f0 msgr2=0x5632bc6f17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.545+0000 7f614e965740 1 -- 172.21.15.2:0/2758671713 shutdown_connections 2024-03-15T12:58:05.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.545+0000 7f614e965740 1 -- 172.21.15.2:0/2758671713 wait complete. 2024-03-15T12:58:05.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.545+0000 7f614e965740 20 client.15276 trim_cache size 0 max 16384 2024-03-15T12:58:05.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.561+0000 7f614e965740 1 Processor -- start 2024-03-15T12:58:05.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.561+0000 7f614e965740 1 -- start start 2024-03-15T12:58:05.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.561+0000 7f614e965740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6f6f80 0x5632bc6f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.561+0000 7f614e965740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 0x5632bc6f9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f614e965740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6fcae0 0x5632bc6f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f614e965740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632bc5e4490 con 0x5632bc6fcae0 2024-03-15T12:58:05.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f614e965740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632bc3af270 con 0x5632bc6f6f80 2024-03-15T12:58:05.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f614e965740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632bc35e6c0 con 0x5632bc6f7970 2024-03-15T12:58:05.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f614a31e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6fcae0 0x5632bc6f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f614931c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 0x5632bc6f9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f614a31e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6fcae0 0x5632bc6f0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47950/0 (socket says 172.21.15.2:47950) 2024-03-15T12:58:05.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f614a31e640 1 -- 172.21.15.2:0/2716294662 learned_addr learned my addr 172.21.15.2:0/2716294662 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f6149b1d640 1 --2- 172.21.15.2:0/2716294662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6f6f80 0x5632bc6f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f614a31e640 1 -- 172.21.15.2:0/2716294662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 msgr2=0x5632bc6f9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f614a31e640 1 --2- 172.21.15.2:0/2716294662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 0x5632bc6f9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f614a31e640 1 -- 172.21.15.2:0/2716294662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6f6f80 msgr2=0x5632bc6f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f614a31e640 1 --2- 172.21.15.2:0/2716294662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6f6f80 0x5632bc6f8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f614a31e640 1 -- 172.21.15.2:0/2716294662 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632bc5f9a20 con 0x5632bc6fcae0 2024-03-15T12:58:05.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f614a31e640 1 --2- 172.21.15.2:0/2716294662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6fcae0 0x5632bc6f0f40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f613c010be0 tx=0x7f613c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606a895b6316f072 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f612f7fe640 1 -- 172.21.15.2:0/2716294662 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c010e50 con 0x5632bc6fcae0 2024-03-15T12:58:05.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f612f7fe640 1 -- 172.21.15.2:0/2716294662 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f613c004de0 con 0x5632bc6fcae0 2024-03-15T12:58:05.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.562+0000 7f612f7fe640 1 -- 172.21.15.2:0/2716294662 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c020760 con 0x5632bc6fcae0 2024-03-15T12:58:05.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.563+0000 7f614e965740 1 -- 172.21.15.2:0/2716294662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6fcae0 msgr2=0x5632bc6f0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.563+0000 7f614e965740 1 --2- 172.21.15.2:0/2716294662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6fcae0 0x5632bc6f0f40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f613c010be0 tx=0x7f613c009da0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.563+0000 7f614e965740 1 -- 172.21.15.2:0/2716294662 shutdown_connections 2024-03-15T12:58:05.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.563+0000 7f614e965740 1 --2- 172.21.15.2:0/2716294662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6fcae0 0x5632bc6f0f40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.563+0000 7f614e965740 1 --2- 172.21.15.2:0/2716294662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f7970 0x5632bc6f9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.563+0000 7f614e965740 1 --2- 172.21.15.2:0/2716294662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6f6f80 0x5632bc6f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.563+0000 7f614e965740 1 -- 172.21.15.2:0/2716294662 >> 172.21.15.2:0/2716294662 conn(0x5632bc6f24f0 msgr2=0x5632bc6f28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.563+0000 7f614e965740 1 -- 172.21.15.2:0/2716294662 shutdown_connections 2024-03-15T12:58:05.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.563+0000 7f614e965740 1 -- 172.21.15.2:0/2716294662 wait complete. 2024-03-15T12:58:05.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.563+0000 7f614e965740 1 -- start start 2024-03-15T12:58:05.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.563+0000 7f614e965740 1 Processor -- start 2024-03-15T12:58:05.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614e965740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f6f80 0x5632bc703c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614e965740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6f7970 0x5632bc704160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614e965740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6fcae0 0x5632bc707180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614e965740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632bc7076c0 con 0x5632bc6f7970 2024-03-15T12:58:05.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614e965740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632bc707830 con 0x5632bc6fcae0 2024-03-15T12:58:05.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614e965740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632bc7079f0 con 0x5632bc6f6f80 2024-03-15T12:58:05.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614e965740 10 client.0 _ll_register_callbacks cb 0x7f613c003c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:05.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614931c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6f7970 0x5632bc704160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614931c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6f7970 0x5632bc704160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47956/0 (socket says 172.21.15.2:47956) 2024-03-15T12:58:05.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614931c640 1 -- 172.21.15.2:0/4226079518 learned_addr learned my addr 172.21.15.2:0/4226079518 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f6149b1d640 1 --2- 172.21.15.2:0/4226079518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f6f80 0x5632bc703c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614a31e640 1 --2- 172.21.15.2:0/4226079518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6fcae0 0x5632bc707180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614931c640 1 -- 172.21.15.2:0/4226079518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f6f80 msgr2=0x5632bc703c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614931c640 1 --2- 172.21.15.2:0/4226079518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f6f80 0x5632bc703c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614931c640 1 -- 172.21.15.2:0/4226079518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6fcae0 msgr2=0x5632bc707180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614931c640 1 --2- 172.21.15.2:0/4226079518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6fcae0 0x5632bc707180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614931c640 1 -- 172.21.15.2:0/4226079518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632bc65d2b0 con 0x5632bc6f7970 2024-03-15T12:58:05.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614931c640 1 --2- 172.21.15.2:0/4226079518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6f7970 0x5632bc704160 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f61400bc500 tx=0x7f61400b88e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af091239bcf571da server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f612d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:05.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f612d7fa640 1 -- 172.21.15.2:0/4226079518 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61400c06c0 con 0x5632bc6f7970 2024-03-15T12:58:05.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f612d7fa640 1 -- 172.21.15.2:0/4226079518 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f61400bcc70 con 0x5632bc6f7970 2024-03-15T12:58:05.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.564+0000 7f614e965740 1 -- 172.21.15.2:0/4226079518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632bc70d530 con 0x5632bc6f7970 2024-03-15T12:58:05.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.565+0000 7f612d7fa640 1 -- 172.21.15.2:0/4226079518 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61400bc340 con 0x5632bc6f7970 2024-03-15T12:58:05.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.565+0000 7f614e965740 10 client.15282 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:05.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.565+0000 7f614e965740 1 -- 172.21.15.2:0/4226079518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5632bc61f780 con 0x5632bc6f7970 2024-03-15T12:58:05.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.565+0000 7f612d7fa640 1 -- 172.21.15.2:0/4226079518 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5632bc70d530 con 0x5632bc6f7970 2024-03-15T12:58:05.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.565+0000 7f612d7fa640 1 -- 172.21.15.2:0/4226079518 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f614003c070 con 0x5632bc6f7970 2024-03-15T12:58:05.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.565+0000 7f614e965740 10 client.15282 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:05.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.565+0000 7f614e965740 10 client.15282 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-15T12:58:05.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.565+0000 7f614e965740 20 client.15282 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:05.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.565+0000 7f614e965740 1 --2- 172.21.15.2:0/4226079518 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5632bc7718b0 0x5632bc76dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.565+0000 7f614e965740 4 client.15282 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-15T12:58:05.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.565+0000 7f614e965740 1 -- 172.21.15.2:0/4226079518 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f61400bc0b0 con 0x5632bc7718b0 2024-03-15T12:58:05.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.565+0000 7f6149b1d640 1 --2- 172.21.15.2:0/4226079518 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5632bc7718b0 0x5632bc76dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.565+0000 7f612d7fa640 10 client.15282 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T12:58:05.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.565+0000 7f6149b1d640 1 --2- 172.21.15.2:0/4226079518 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5632bc7718b0 0x5632bc76dd70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e64d42f81ba571e server_cookie=c096e2d2e1b29e62 in_seq=0 out_seq=0 2024-03-15T12:58:05.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f612d7fa640 1 -- 172.21.15.2:0/4226079518 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f61400bc0b0 con 0x5632bc7718b0 2024-03-15T12:58:05.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f612d7fa640 10 client.15282 handle_command_reply: tid=0 2024-03-15T12:58:05.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614e965740 1 client.15282 shutdown 2024-03-15T12:58:05.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614e965740 1 -- 172.21.15.2:0/4226079518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6f7970 msgr2=0x5632bc704160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614e965740 1 --2- 172.21.15.2:0/4226079518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6f7970 0x5632bc704160 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f61400bc500 tx=0x7f61400b88e0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614e965740 1 -- 172.21.15.2:0/4226079518 shutdown_connections 2024-03-15T12:58:05.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614e965740 1 --2- 172.21.15.2:0/4226079518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632bc6fcae0 0x5632bc707180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614e965740 1 --2- 172.21.15.2:0/4226079518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5632bc6f7970 0x5632bc704160 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614e965740 1 --2- 172.21.15.2:0/4226079518 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5632bc7718b0 0x5632bc76dd70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614e965740 1 --2- 172.21.15.2:0/4226079518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5632bc6f6f80 0x5632bc703c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f612d7fa640 0 client.15282 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T12:58:05.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614e965740 1 -- 172.21.15.2:0/4226079518 >> 172.21.15.2:0/4226079518 conn(0x5632bc6f24f0 msgr2=0x5632bc6f8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614e965740 1 -- 172.21.15.2:0/4226079518 shutdown_connections 2024-03-15T12:58:05.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614e965740 1 -- 172.21.15.2:0/4226079518 wait complete. 2024-03-15T12:58:05.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614e965740 20 client.15282 trim_cache size 0 max 16384 2024-03-15T12:58:05.562 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:58:05.562 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "2767718a-e2ca-11ee-95c9-87774f69a715", 2024-03-15T12:58:05.562 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 2, 2024-03-15T12:58:05.562 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24464, 2024-03-15T12:58:05.562 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-03-15T12:58:05.562 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-03-15T12:58:05.562 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-03-15T12:58:05.562 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 85.166033787000003, 2024-03-15T12:58:05.562 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 28, 2024-03-15T12:58:05.563 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 91, 2024-03-15T12:58:05.563 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 89, 2024-03-15T12:58:05.563 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 131.26266681999999 2024-03-15T12:58:05.563 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:58:05.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614bd81640 1 -- 172.21.15.2:0/367778497 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f61300a1a30 msgr2=0x7f61300a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614bd81640 1 --2- 172.21.15.2:0/367778497 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f61300a1a30 0x7f61300a3ef0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6144101440 tx=0x7f6138005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614bd81640 1 -- 172.21.15.2:0/367778497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61441015e0 msgr2=0x7f6144197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.566+0000 7f614bd81640 1 --2- 172.21.15.2:0/367778497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61441015e0 0x7f6144197b20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6140005ab0 tx=0x7f6140004a30 comp rx=0 tx=0).stop 2024-03-15T12:58:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.567+0000 7f614bd81640 1 -- 172.21.15.2:0/367778497 shutdown_connections 2024-03-15T12:58:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.567+0000 7f614bd81640 1 --2- 172.21.15.2:0/367778497 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f61300a1a30 0x7f61300a3ef0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.567+0000 7f614bd81640 1 --2- 172.21.15.2:0/367778497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6144101fa0 0x7f614419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.567+0000 7f614bd81640 1 --2- 172.21.15.2:0/367778497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61441015e0 0x7f6144197b20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.567+0000 7f614bd81640 1 --2- 172.21.15.2:0/367778497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61441003f0 0x7f61441975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.567+0000 7f614bd81640 1 -- 172.21.15.2:0/367778497 >> 172.21.15.2:0/367778497 conn(0x7f61440fb980 msgr2=0x7f61440fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.567+0000 7f614bd81640 1 -- 172.21.15.2:0/367778497 shutdown_connections 2024-03-15T12:58:05.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.567+0000 7f614bd81640 1 -- 172.21.15.2:0/367778497 wait complete. 2024-03-15T12:58:05.566 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-15T12:58:05.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.683+0000 7fa62311e640 1 -- 172.21.15.2:0/3907914672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa61c102780 msgr2=0x7fa61c102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.683+0000 7fa62311e640 1 --2- 172.21.15.2:0/3907914672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa61c102780 0x7fa61c102c00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa608009900 tx=0x7fa60802bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:05.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.683+0000 7fa62311e640 1 -- 172.21.15.2:0/3907914672 shutdown_connections 2024-03-15T12:58:05.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.683+0000 7fa62311e640 1 --2- 172.21.15.2:0/3907914672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c103140 0x7fa61c1099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.683+0000 7fa62311e640 1 --2- 172.21.15.2:0/3907914672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa61c102780 0x7fa61c102c00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.683+0000 7fa62311e640 1 --2- 172.21.15.2:0/3907914672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa61c0ff7b0 0x7fa61c0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.683+0000 7fa62311e640 1 -- 172.21.15.2:0/3907914672 >> 172.21.15.2:0/3907914672 conn(0x7fa61c0fb0e0 msgr2=0x7fa61c0fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.683+0000 7fa62311e640 1 -- 172.21.15.2:0/3907914672 shutdown_connections 2024-03-15T12:58:05.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.683+0000 7fa62311e640 1 -- 172.21.15.2:0/3907914672 wait complete. 2024-03-15T12:58:05.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa62311e640 1 Processor -- start 2024-03-15T12:58:05.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa62311e640 1 -- start start 2024-03-15T12:58:05.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa62311e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa61c0ff7b0 0x7fa61c1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa62311e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa61c102780 0x7fa61c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa62311e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c103140 0x7fa61c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa62311e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c19f0b0 con 0x7fa61c102780 2024-03-15T12:58:05.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa62311e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c19f220 con 0x7fa61c0ff7b0 2024-03-15T12:58:05.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa62311e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c19f3e0 con 0x7fa61c103140 2024-03-15T12:58:05.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa61bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa61c102780 0x7fa61c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa620eba640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa61c0ff7b0 0x7fa61c1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa61bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa61c102780 0x7fa61c197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47978/0 (socket says 172.21.15.2:47978) 2024-03-15T12:58:05.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa6216bb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c103140 0x7fa61c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa61bfff640 1 -- 172.21.15.2:0/1150190207 learned_addr learned my addr 172.21.15.2:0/1150190207 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa61bfff640 1 -- 172.21.15.2:0/1150190207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c103140 msgr2=0x7fa61c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa61bfff640 1 --2- 172.21.15.2:0/1150190207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c103140 0x7fa61c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa61bfff640 1 -- 172.21.15.2:0/1150190207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa61c0ff7b0 msgr2=0x7fa61c1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa61bfff640 1 --2- 172.21.15.2:0/1150190207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa61c0ff7b0 0x7fa61c1975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.684+0000 7fa61bfff640 1 -- 172.21.15.2:0/1150190207 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa608009560 con 0x7fa61c102780 2024-03-15T12:58:05.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.685+0000 7fa61bfff640 1 --2- 172.21.15.2:0/1150190207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa61c102780 0x7fa61c197b00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa608000c00 tx=0x7fa608004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.685+0000 7fa619ffb640 1 -- 172.21.15.2:0/1150190207 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6080042d0 con 0x7fa61c102780 2024-03-15T12:58:05.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.685+0000 7fa619ffb640 1 -- 172.21.15.2:0/1150190207 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa608004430 con 0x7fa61c102780 2024-03-15T12:58:05.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.685+0000 7fa62311e640 1 -- 172.21.15.2:0/1150190207 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61c19f660 con 0x7fa61c102780 2024-03-15T12:58:05.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.685+0000 7fa619ffb640 1 -- 172.21.15.2:0/1150190207 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608003e50 con 0x7fa61c102780 2024-03-15T12:58:05.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.685+0000 7fa62311e640 1 -- 172.21.15.2:0/1150190207 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c19fbc0 con 0x7fa61c102780 2024-03-15T12:58:05.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.687+0000 7fa619ffb640 1 -- 172.21.15.2:0/1150190207 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fa6080035f0 con 0x7fa61c102780 2024-03-15T12:58:05.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa625dd5740 1 Processor -- start 2024-03-15T12:58:05.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa625dd5740 1 -- start start 2024-03-15T12:58:05.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa625dd5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fdef80 0x555d30fdf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa625dd5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdf970 0x555d30fe4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa625dd5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fe4ae0 0x555d30fe6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa625dd5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d30f9acd0 con 0x555d30fdf970 2024-03-15T12:58:05.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa625dd5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d30f4df20 con 0x555d30fdef80 2024-03-15T12:58:05.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa625dd5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d30f4b450 con 0x555d30fe4ae0 2024-03-15T12:58:05.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa619ffb640 1 --2- 172.21.15.2:0/1150190207 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa60c0a1ab0 0x7fa60c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa620eba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdf970 0x555d30fe4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa6216bb640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fdef80 0x555d30fdf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa61bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fe4ae0 0x555d30fe6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa620eba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdf970 0x555d30fe4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47984/0 (socket says 172.21.15.2:47984) 2024-03-15T12:58:05.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa620eba640 1 -- 172.21.15.2:0/1079625414 learned_addr learned my addr 172.21.15.2:0/1079625414 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa6216bb640 1 --2- 172.21.15.2:0/1150190207 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa60c0a1ab0 0x7fa60c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa619ffb640 1 -- 172.21.15.2:0/1150190207 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fa6080ef830 con 0x7fa61c102780 2024-03-15T12:58:05.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa620eba640 1 -- 172.21.15.2:0/1079625414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fe4ae0 msgr2=0x555d30fe6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa620eba640 1 --2- 172.21.15.2:0/1079625414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fe4ae0 0x555d30fe6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa620eba640 1 -- 172.21.15.2:0/1079625414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fdef80 msgr2=0x555d30fdf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa620eba640 1 --2- 172.21.15.2:0/1079625414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fdef80 0x555d30fdf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa620eba640 1 -- 172.21.15.2:0/1079625414 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d30efbf20 con 0x555d30fdf970 2024-03-15T12:58:05.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.688+0000 7fa6216bb640 1 --2- 172.21.15.2:0/1079625414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fdef80 0x555d30fdf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:05.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.689+0000 7fa620eba640 1 --2- 172.21.15.2:0/1079625414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdf970 0x555d30fe4470 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa610009ca0 tx=0x7fa610011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb103ad4e0cfd43c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.689+0000 7fa6216bb640 1 --2- 172.21.15.2:0/1150190207 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa60c0a1ab0 0x7fa60c0a3f70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x555d30fe01d0 tx=0x7fa614009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.689+0000 7fa606ffd640 1 -- 172.21.15.2:0/1079625414 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610004220 con 0x555d30fdf970 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.689+0000 7fa606ffd640 1 -- 172.21.15.2:0/1079625414 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa610004380 con 0x555d30fdf970 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.689+0000 7fa606ffd640 1 -- 172.21.15.2:0/1079625414 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610019810 con 0x555d30fdf970 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.689+0000 7fa625dd5740 1 -- 172.21.15.2:0/1079625414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdf970 msgr2=0x555d30fe4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.689+0000 7fa625dd5740 1 --2- 172.21.15.2:0/1079625414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdf970 0x555d30fe4470 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa610009ca0 tx=0x7fa610011810 comp rx=0 tx=0).stop 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.689+0000 7fa625dd5740 1 -- 172.21.15.2:0/1079625414 shutdown_connections 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.689+0000 7fa625dd5740 1 --2- 172.21.15.2:0/1079625414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fe4ae0 0x555d30fe6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.689+0000 7fa625dd5740 1 --2- 172.21.15.2:0/1079625414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdf970 0x555d30fe4470 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.689+0000 7fa625dd5740 1 --2- 172.21.15.2:0/1079625414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fdef80 0x555d30fdf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.689+0000 7fa625dd5740 1 -- 172.21.15.2:0/1079625414 >> 172.21.15.2:0/1079625414 conn(0x555d30fda4f0 msgr2=0x555d30fda8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.689+0000 7fa625dd5740 1 -- 172.21.15.2:0/1079625414 shutdown_connections 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.690+0000 7fa625dd5740 1 -- 172.21.15.2:0/1079625414 wait complete. 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.690+0000 7fa625dd5740 1 -- start start 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.690+0000 7fa625dd5740 1 Processor -- start 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.690+0000 7fa625dd5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fdef80 0x555d30fe91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.690+0000 7fa625dd5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdf970 0x555d30fe9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.690+0000 7fa625dd5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fe4ae0 0x555d30ff0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.690+0000 7fa625dd5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d30f46c30 con 0x555d30fdf970 2024-03-15T12:58:05.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.690+0000 7fa625dd5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d30eebd30 con 0x555d30fe4ae0 2024-03-15T12:58:05.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.690+0000 7fa625dd5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d30c1a310 con 0x555d30fdef80 2024-03-15T12:58:05.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.690+0000 7fa625dd5740 10 client.0 _ll_register_callbacks cb 0x555d30ef3d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:05.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.690+0000 7fa61bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdf970 0x555d30fe9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.690+0000 7fa61bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdf970 0x555d30fe9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47986/0 (socket says 172.21.15.2:47986) 2024-03-15T12:58:05.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.690+0000 7fa61bfff640 1 -- 172.21.15.2:0/3282643884 learned_addr learned my addr 172.21.15.2:0/3282643884 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.690+0000 7fa6216bb640 1 --2- 172.21.15.2:0/3282643884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fe4ae0 0x555d30ff0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.690+0000 7fa620eba640 1 --2- 172.21.15.2:0/3282643884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fdef80 0x555d30fe91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa61bfff640 1 -- 172.21.15.2:0/3282643884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fdef80 msgr2=0x555d30fe91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa61bfff640 1 --2- 172.21.15.2:0/3282643884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fdef80 0x555d30fe91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa61bfff640 1 -- 172.21.15.2:0/3282643884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fe4ae0 msgr2=0x555d30ff0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa61bfff640 1 --2- 172.21.15.2:0/3282643884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fe4ae0 0x555d30ff0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa61bfff640 1 -- 172.21.15.2:0/3282643884 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6100098d0 con 0x555d30fdf970 2024-03-15T12:58:05.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa61bfff640 1 --2- 172.21.15.2:0/3282643884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdf970 0x555d30fe9710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa6080b8e30 tx=0x7fa60808c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335fb5f9d3ca4043 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa5fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:05.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa5fbfff640 1 -- 172.21.15.2:0/3282643884 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6080bc8c0 con 0x555d30fdf970 2024-03-15T12:58:05.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa5fbfff640 1 -- 172.21.15.2:0/3282643884 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fa6080bcb60 con 0x555d30fdf970 2024-03-15T12:58:05.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa625dd5740 1 -- 172.21.15.2:0/3282643884 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d30c466c0 con 0x555d30fdf970 2024-03-15T12:58:05.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa5fbfff640 1 -- 172.21.15.2:0/3282643884 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6080bbdc0 con 0x555d30fdf970 2024-03-15T12:58:05.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa625dd5740 10 client.15294 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:05.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa625dd5740 1 -- 172.21.15.2:0/3282643884 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555d30f452b0 con 0x555d30fdf970 2024-03-15T12:58:05.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa5fbfff640 1 -- 172.21.15.2:0/3282643884 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa60808b080 con 0x555d30fdf970 2024-03-15T12:58:05.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa5fbfff640 1 -- 172.21.15.2:0/3282643884 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa60803c070 con 0x555d30fdf970 2024-03-15T12:58:05.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.691+0000 7fa625dd5740 10 client.15294 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:05.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.692+0000 7fa625dd5740 10 client.15294 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-15T12:58:05.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.692+0000 7fa625dd5740 20 client.15294 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:05.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.692+0000 7fa625dd5740 1 --2- 172.21.15.2:0/3282643884 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x555d30ff5960 0x555d30ff9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.692+0000 7fa625dd5740 4 client.15294 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:58:05.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.692+0000 7fa625dd5740 1 -- 172.21.15.2:0/3282643884 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d30c97270 con 0x555d30ff5960 2024-03-15T12:58:05.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.692+0000 7fa620eba640 1 --2- 172.21.15.2:0/3282643884 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x555d30ff5960 0x555d30ff9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.692+0000 7fa620eba640 1 --2- 172.21.15.2:0/3282643884 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x555d30ff5960 0x555d30ff9de0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4e40a6f7b45e010 server_cookie=9683a1a1ddc504c3 in_seq=0 out_seq=0 2024-03-15T12:58:05.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.692+0000 7fa5fbfff640 10 client.15294 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T12:58:05.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.693+0000 7fa5fbfff640 1 -- 172.21.15.2:0/3282643884 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555d30c97270 con 0x555d30ff5960 2024-03-15T12:58:05.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.693+0000 7fa5fbfff640 10 client.15294 handle_command_reply: tid=0 2024-03-15T12:58:05.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.693+0000 7fa625dd5740 1 client.15294 shutdown 2024-03-15T12:58:05.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.693+0000 7fa625dd5740 1 -- 172.21.15.2:0/3282643884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdf970 msgr2=0x555d30fe9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.693+0000 7fa625dd5740 1 --2- 172.21.15.2:0/3282643884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdf970 0x555d30fe9710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa6080b8e30 tx=0x7fa60808c550 comp rx=0 tx=0).stop 2024-03-15T12:58:05.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.693+0000 7fa625dd5740 1 -- 172.21.15.2:0/3282643884 shutdown_connections 2024-03-15T12:58:05.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.693+0000 7fa625dd5740 1 --2- 172.21.15.2:0/3282643884 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x555d30ff5960 0x555d30ff9de0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.693+0000 7fa625dd5740 1 --2- 172.21.15.2:0/3282643884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fe4ae0 0x555d30ff0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.693+0000 7fa625dd5740 1 --2- 172.21.15.2:0/3282643884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdf970 0x555d30fe9710 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.693+0000 7fa625dd5740 1 --2- 172.21.15.2:0/3282643884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fdef80 0x555d30fe91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.693+0000 7fa5fbfff640 0 client.15294 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T12:58:05.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.693+0000 7fa625dd5740 1 -- 172.21.15.2:0/3282643884 >> 172.21.15.2:0/3282643884 conn(0x555d30fda4f0 msgr2=0x555d30fd97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.693+0000 7fa625dd5740 1 -- 172.21.15.2:0/3282643884 shutdown_connections 2024-03-15T12:58:05.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.693+0000 7fa625dd5740 1 -- 172.21.15.2:0/3282643884 wait complete. 2024-03-15T12:58:05.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.693+0000 7fa625dd5740 20 client.15294 trim_cache size 0 max 16384 2024-03-15T12:58:05.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa625dd5740 1 Processor -- start 2024-03-15T12:58:05.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa625dd5740 1 -- start start 2024-03-15T12:58:05.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa625dd5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdef80 0x555d30fd8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa625dd5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fdf970 0x555d30fd9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa625dd5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fe4ae0 0x555d30fd99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa625dd5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d30ecc490 con 0x555d30fdef80 2024-03-15T12:58:05.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa625dd5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d30fd5660 con 0x555d30fe4ae0 2024-03-15T12:58:05.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa625dd5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d30fdaf40 con 0x555d30fdf970 2024-03-15T12:58:05.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa620eba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdef80 0x555d30fd8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa620eba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdef80 0x555d30fd8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47996/0 (socket says 172.21.15.2:47996) 2024-03-15T12:58:05.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa620eba640 1 -- 172.21.15.2:0/1141795858 learned_addr learned my addr 172.21.15.2:0/1141795858 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa6216bb640 1 --2- 172.21.15.2:0/1141795858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fe4ae0 0x555d30fd99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa61bfff640 1 --2- 172.21.15.2:0/1141795858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fdf970 0x555d30fd9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa620eba640 1 -- 172.21.15.2:0/1141795858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fdf970 msgr2=0x555d30fd9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa620eba640 1 --2- 172.21.15.2:0/1141795858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fdf970 0x555d30fd9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa620eba640 1 -- 172.21.15.2:0/1141795858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fe4ae0 msgr2=0x555d30fd99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa620eba640 1 --2- 172.21.15.2:0/1141795858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fe4ae0 0x555d30fd99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.710+0000 7fa620eba640 1 -- 172.21.15.2:0/1141795858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d30ee1a20 con 0x555d30fdef80 2024-03-15T12:58:05.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.711+0000 7fa620eba640 1 --2- 172.21.15.2:0/1141795858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdef80 0x555d30fd8f40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa610009730 tx=0x7fa610012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d88e731f7e3c72d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.711+0000 7fa6067fc640 1 -- 172.21.15.2:0/1141795858 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610003e50 con 0x555d30fdef80 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.711+0000 7fa6067fc640 1 -- 172.21.15.2:0/1141795858 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa61002c2f0 con 0x555d30fdef80 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.711+0000 7fa6067fc640 1 -- 172.21.15.2:0/1141795858 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6100052a0 con 0x555d30fdef80 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.711+0000 7fa625dd5740 1 -- 172.21.15.2:0/1141795858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdef80 msgr2=0x555d30fd8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.711+0000 7fa625dd5740 1 --2- 172.21.15.2:0/1141795858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdef80 0x555d30fd8f40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa610009730 tx=0x7fa610012f90 comp rx=0 tx=0).stop 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.711+0000 7fa625dd5740 1 -- 172.21.15.2:0/1141795858 shutdown_connections 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.711+0000 7fa625dd5740 1 --2- 172.21.15.2:0/1141795858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fe4ae0 0x555d30fd99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.711+0000 7fa625dd5740 1 --2- 172.21.15.2:0/1141795858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fdf970 0x555d30fd9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.711+0000 7fa625dd5740 1 --2- 172.21.15.2:0/1141795858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdef80 0x555d30fd8f40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.711+0000 7fa625dd5740 1 -- 172.21.15.2:0/1141795858 >> 172.21.15.2:0/1141795858 conn(0x555d30fda4f0 msgr2=0x555d30fe91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.711+0000 7fa625dd5740 1 -- 172.21.15.2:0/1141795858 shutdown_connections 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.711+0000 7fa625dd5740 1 -- 172.21.15.2:0/1141795858 wait complete. 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa625dd5740 1 -- start start 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa625dd5740 1 Processor -- start 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa625dd5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdef80 0x555d3104db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa625dd5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fdf970 0x555d3104e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa625dd5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fe4ae0 0x555d310550a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa625dd5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d30fe7100 con 0x555d30fdef80 2024-03-15T12:58:05.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa625dd5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d30ff13d0 con 0x555d30fdf970 2024-03-15T12:58:05.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa625dd5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d30fe46c0 con 0x555d30fe4ae0 2024-03-15T12:58:05.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa625dd5740 10 client.0 _ll_register_callbacks cb 0x7fa6080038a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:05.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa620eba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdef80 0x555d3104db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa620eba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdef80 0x555d3104db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48006/0 (socket says 172.21.15.2:48006) 2024-03-15T12:58:05.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa620eba640 1 -- 172.21.15.2:0/2882299340 learned_addr learned my addr 172.21.15.2:0/2882299340 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa61bfff640 1 --2- 172.21.15.2:0/2882299340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fdf970 0x555d3104e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa6216bb640 1 --2- 172.21.15.2:0/2882299340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fe4ae0 0x555d310550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa620eba640 1 -- 172.21.15.2:0/2882299340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fe4ae0 msgr2=0x555d310550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa620eba640 1 --2- 172.21.15.2:0/2882299340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fe4ae0 0x555d310550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa620eba640 1 -- 172.21.15.2:0/2882299340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fdf970 msgr2=0x555d3104e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa620eba640 1 --2- 172.21.15.2:0/2882299340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fdf970 0x555d3104e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.712+0000 7fa620eba640 1 -- 172.21.15.2:0/2882299340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d30efbf20 con 0x555d30fdef80 2024-03-15T12:58:05.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa620eba640 1 --2- 172.21.15.2:0/2882299340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdef80 0x555d3104db20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa610009bc0 tx=0x7fa610005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a713454ca80ae62e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa5fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:05.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa5fbfff640 1 -- 172.21.15.2:0/2882299340 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610005630 con 0x555d30fdef80 2024-03-15T12:58:05.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa5fbfff640 1 -- 172.21.15.2:0/2882299340 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fa6100035f0 con 0x555d30fdef80 2024-03-15T12:58:05.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa625dd5740 1 -- 172.21.15.2:0/2882299340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d30ff5010 con 0x555d30fdef80 2024-03-15T12:58:05.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa5fbfff640 1 -- 172.21.15.2:0/2882299340 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa610011d10 con 0x555d30fdef80 2024-03-15T12:58:05.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa625dd5740 10 client.15300 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:05.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa625dd5740 1 -- 172.21.15.2:0/2882299340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555d30f07780 con 0x555d30fdef80 2024-03-15T12:58:05.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa5fbfff640 1 -- 172.21.15.2:0/2882299340 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x555d30ff5010 con 0x555d30fdef80 2024-03-15T12:58:05.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa5fbfff640 1 -- 172.21.15.2:0/2882299340 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa610004c60 con 0x555d30fdef80 2024-03-15T12:58:05.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa625dd5740 10 client.15300 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:05.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa625dd5740 10 client.15300 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-15T12:58:05.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa625dd5740 20 client.15300 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:05.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa625dd5740 1 --2- 172.21.15.2:0/2882299340 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x555d3105a550 0x555d310569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa625dd5740 4 client.15300 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-15T12:58:05.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.713+0000 7fa625dd5740 1 -- 172.21.15.2:0/2882299340 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fa610004380 con 0x555d3105a550 2024-03-15T12:58:05.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa61bfff640 1 --2- 172.21.15.2:0/2882299340 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x555d3105a550 0x555d310569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa5fbfff640 10 client.15300 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T12:58:05.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa61bfff640 1 --2- 172.21.15.2:0/2882299340 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x555d3105a550 0x555d310569d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=769c9e9dee727595 server_cookie=55386bf5018ef1d5 in_seq=0 out_seq=0 2024-03-15T12:58:05.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa5fbfff640 1 -- 172.21.15.2:0/2882299340 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7fa610004380 con 0x555d3105a550 2024-03-15T12:58:05.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa5fbfff640 10 client.15300 handle_command_reply: tid=0 2024-03-15T12:58:05.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa625dd5740 1 client.15300 shutdown 2024-03-15T12:58:05.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa625dd5740 1 -- 172.21.15.2:0/2882299340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdef80 msgr2=0x555d3104db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa625dd5740 1 --2- 172.21.15.2:0/2882299340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdef80 0x555d3104db20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa610009bc0 tx=0x7fa610005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:05.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa625dd5740 1 -- 172.21.15.2:0/2882299340 shutdown_connections 2024-03-15T12:58:05.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa625dd5740 1 --2- 172.21.15.2:0/2882299340 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x555d3105a550 0x555d310569d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa625dd5740 1 --2- 172.21.15.2:0/2882299340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555d30fe4ae0 0x555d310550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa625dd5740 1 --2- 172.21.15.2:0/2882299340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d30fdf970 0x555d3104e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa5fbfff640 0 client.15300 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T12:58:05.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa625dd5740 1 --2- 172.21.15.2:0/2882299340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555d30fdef80 0x555d3104db20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa625dd5740 1 -- 172.21.15.2:0/2882299340 >> 172.21.15.2:0/2882299340 conn(0x555d30fda4f0 msgr2=0x555d30fdd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa625dd5740 1 -- 172.21.15.2:0/2882299340 shutdown_connections 2024-03-15T12:58:05.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa625dd5740 1 -- 172.21.15.2:0/2882299340 wait complete. 2024-03-15T12:58:05.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.714+0000 7fa625dd5740 20 client.15300 trim_cache size 0 max 16384 2024-03-15T12:58:05.707 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:58:05.707 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "2767718a-e2ca-11ee-95c9-87774f69a715", 2024-03-15T12:58:05.707 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 1, 2024-03-15T12:58:05.707 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24500, 2024-03-15T12:58:05.708 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-03-15T12:58:05.708 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-03-15T12:58:05.708 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-03-15T12:58:05.708 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 85.889927076000006, 2024-03-15T12:58:05.708 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 28, 2024-03-15T12:58:05.708 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 91, 2024-03-15T12:58:05.708 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 89, 2024-03-15T12:58:05.708 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 122.904981015 2024-03-15T12:58:05.708 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:58:05.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.715+0000 7fa62311e640 1 -- 172.21.15.2:0/1150190207 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa60c0a1ab0 msgr2=0x7fa60c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.715+0000 7fa62311e640 1 --2- 172.21.15.2:0/1150190207 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa60c0a1ab0 0x7fa60c0a3f70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x555d30fe01d0 tx=0x7fa614009500 comp rx=0 tx=0).stop 2024-03-15T12:58:05.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.715+0000 7fa62311e640 1 -- 172.21.15.2:0/1150190207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa61c102780 msgr2=0x7fa61c197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.715+0000 7fa62311e640 1 --2- 172.21.15.2:0/1150190207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa61c102780 0x7fa61c197b00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa608000c00 tx=0x7fa608004e50 comp rx=0 tx=0).stop 2024-03-15T12:58:05.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.715+0000 7fa62311e640 1 -- 172.21.15.2:0/1150190207 shutdown_connections 2024-03-15T12:58:05.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.715+0000 7fa62311e640 1 --2- 172.21.15.2:0/1150190207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa61c103140 0x7fa61c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.715+0000 7fa62311e640 1 --2- 172.21.15.2:0/1150190207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa61c102780 0x7fa61c197b00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.715+0000 7fa62311e640 1 --2- 172.21.15.2:0/1150190207 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa60c0a1ab0 0x7fa60c0a3f70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.715+0000 7fa62311e640 1 --2- 172.21.15.2:0/1150190207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa61c0ff7b0 0x7fa61c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.715+0000 7fa62311e640 1 -- 172.21.15.2:0/1150190207 >> 172.21.15.2:0/1150190207 conn(0x7fa61c0fb0e0 msgr2=0x7fa61c105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.715+0000 7fa62311e640 1 -- 172.21.15.2:0/1150190207 shutdown_connections 2024-03-15T12:58:05.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.715+0000 7fa62311e640 1 -- 172.21.15.2:0/1150190207 wait complete. 2024-03-15T12:58:05.711 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:05 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2329153843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:05.711 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:05 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3182779329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:05.712 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-15T12:58:05.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.829+0000 7f380918c640 1 -- 172.21.15.2:0/4150854605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3804101fc0 msgr2=0x7f3804108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.829+0000 7f380918c640 1 --2- 172.21.15.2:0/4150854605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3804101fc0 0x7f3804108850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009ca0 tx=0x7f37fc02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.829+0000 7f380918c640 1 -- 172.21.15.2:0/4150854605 shutdown_connections 2024-03-15T12:58:05.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.829+0000 7f380918c640 1 --2- 172.21.15.2:0/4150854605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3804101fc0 0x7f3804108850 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.829+0000 7f380918c640 1 --2- 172.21.15.2:0/4150854605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3804101600 0x7f3804101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.829+0000 7f380918c640 1 --2- 172.21.15.2:0/4150854605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3804100410 0x7f3804100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.829+0000 7f380918c640 1 -- 172.21.15.2:0/4150854605 >> 172.21.15.2:0/4150854605 conn(0x7f38040fb9a0 msgr2=0x7f38040fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.830+0000 7f380918c640 1 -- 172.21.15.2:0/4150854605 shutdown_connections 2024-03-15T12:58:05.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.830+0000 7f380918c640 1 -- 172.21.15.2:0/4150854605 wait complete. 2024-03-15T12:58:05.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.830+0000 7f380918c640 1 Processor -- start 2024-03-15T12:58:05.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.830+0000 7f380918c640 1 -- start start 2024-03-15T12:58:05.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.830+0000 7f380918c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3804100410 0x7f3804197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.830+0000 7f380918c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3804101600 0x7f3804197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.830+0000 7f380918c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3804101fc0 0x7f380419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f380918c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380419f040 con 0x7f3804100410 2024-03-15T12:58:05.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f380918c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380419f1b0 con 0x7f3804101fc0 2024-03-15T12:58:05.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f380918c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380419f370 con 0x7f3804101600 2024-03-15T12:58:05.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f3802d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3804100410 0x7f3804197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f3802d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3804100410 0x7f3804197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48030/0 (socket says 172.21.15.2:48030) 2024-03-15T12:58:05.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f3802d9d640 1 -- 172.21.15.2:0/1898598530 learned_addr learned my addr 172.21.15.2:0/1898598530 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f380359e640 1 --2- 172.21.15.2:0/1898598530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3804101fc0 0x7f380419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f380259c640 1 --2- 172.21.15.2:0/1898598530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3804101600 0x7f3804197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f3802d9d640 1 -- 172.21.15.2:0/1898598530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3804101600 msgr2=0x7f3804197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f3802d9d640 1 --2- 172.21.15.2:0/1898598530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3804101600 0x7f3804197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f3802d9d640 1 -- 172.21.15.2:0/1898598530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3804101fc0 msgr2=0x7f380419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f3802d9d640 1 --2- 172.21.15.2:0/1898598530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3804101fc0 0x7f380419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f3802d9d640 1 -- 172.21.15.2:0/1898598530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f4009490 con 0x7f3804100410 2024-03-15T12:58:05.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f3802d9d640 1 --2- 172.21.15.2:0/1898598530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3804100410 0x7f3804197550 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f37f400e9c0 tx=0x7f37f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f37ebfff640 1 -- 172.21.15.2:0/1898598530 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f400cde0 con 0x7f3804100410 2024-03-15T12:58:05.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f380918c640 1 -- 172.21.15.2:0/1898598530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37fc009900 con 0x7f3804100410 2024-03-15T12:58:05.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f37ebfff640 1 -- 172.21.15.2:0/1898598530 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f37f4010900 con 0x7f3804100410 2024-03-15T12:58:05.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f380918c640 1 -- 172.21.15.2:0/1898598530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380419fa20 con 0x7f3804100410 2024-03-15T12:58:05.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.831+0000 7f37ebfff640 1 -- 172.21.15.2:0/1898598530 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f4018930 con 0x7f3804100410 2024-03-15T12:58:05.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.833+0000 7f37ebfff640 1 -- 172.21.15.2:0/1898598530 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f37f4010a70 con 0x7f3804100410 2024-03-15T12:58:05.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f380bd6e740 1 Processor -- start 2024-03-15T12:58:05.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f380bd6e740 1 -- start start 2024-03-15T12:58:05.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f37ebfff640 1 --2- 172.21.15.2:0/1898598530 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f37ec0a1a30 0x7f37ec0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f380bd6e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4f7390 0x5626ff4f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f380bd6e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7d80 0x5626ff4fc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f380bd6e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4fcef0 0x5626ff4ff380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f380bd6e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ff4aecd0 con 0x5626ff4f7390 2024-03-15T12:58:05.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f380bd6e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ff461f20 con 0x5626ff4fcef0 2024-03-15T12:58:05.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f380bd6e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ff45f450 con 0x5626ff4f7d80 2024-03-15T12:58:05.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f3802d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4f7390 0x5626ff4f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f3802d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4f7390 0x5626ff4f77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48040/0 (socket says 172.21.15.2:48040) 2024-03-15T12:58:05.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f3802d9d640 1 -- 172.21.15.2:0/2261321890 learned_addr learned my addr 172.21.15.2:0/2261321890 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f380359e640 1 --2- 172.21.15.2:0/1898598530 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f37ec0a1a30 0x7f37ec0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f380259c640 1 --2- 172.21.15.2:0/2261321890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7d80 0x5626ff4fc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f380359e640 1 --2- 172.21.15.2:0/2261321890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4fcef0 0x5626ff4ff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f37ebfff640 1 -- 172.21.15.2:0/1898598530 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f37f4014070 con 0x7f3804100410 2024-03-15T12:58:05.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.834+0000 7f3802d9d640 1 -- 172.21.15.2:0/2261321890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7d80 msgr2=0x5626ff4fc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.835+0000 7f3802d9d640 1 --2- 172.21.15.2:0/2261321890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7d80 0x5626ff4fc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.835+0000 7f3802d9d640 1 -- 172.21.15.2:0/2261321890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4fcef0 msgr2=0x5626ff4ff380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.835+0000 7f3802d9d640 1 --2- 172.21.15.2:0/2261321890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4fcef0 0x5626ff4ff380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.835+0000 7f3802d9d640 1 -- 172.21.15.2:0/2261321890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626ff40ff20 con 0x5626ff4f7390 2024-03-15T12:58:05.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.835+0000 7f380259c640 1 --2- 172.21.15.2:0/2261321890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7d80 0x5626ff4fc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:05.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.835+0000 7f3802d9d640 1 --2- 172.21.15.2:0/2261321890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4f7390 0x5626ff4f77b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f37f4004770 tx=0x7f37f4018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f79f1ca9ffc3ea0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.835+0000 7f380359e640 1 --2- 172.21.15.2:0/1898598530 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f37ec0a1a30 0x7f37ec0a3ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f37fc005cc0 tx=0x7f37fc0323c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.835+0000 7f37e97fa640 1 -- 172.21.15.2:0/2261321890 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f409fff0 con 0x5626ff4f7390 2024-03-15T12:58:05.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.835+0000 7f37e97fa640 1 -- 172.21.15.2:0/2261321890 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f37f409f9f0 con 0x5626ff4f7390 2024-03-15T12:58:05.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.835+0000 7f37e97fa640 1 -- 172.21.15.2:0/2261321890 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f409fd70 con 0x5626ff4f7390 2024-03-15T12:58:05.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.835+0000 7f380bd6e740 1 -- 172.21.15.2:0/2261321890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4f7390 msgr2=0x5626ff4f77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.835+0000 7f380bd6e740 1 --2- 172.21.15.2:0/2261321890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4f7390 0x5626ff4f77b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f37f4004770 tx=0x7f37f4018420 comp rx=0 tx=0).stop 2024-03-15T12:58:05.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 1 -- 172.21.15.2:0/2261321890 shutdown_connections 2024-03-15T12:58:05.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 1 --2- 172.21.15.2:0/2261321890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4fcef0 0x5626ff4ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 1 --2- 172.21.15.2:0/2261321890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7d80 0x5626ff4fc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 1 --2- 172.21.15.2:0/2261321890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4f7390 0x5626ff4f77b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 1 -- 172.21.15.2:0/2261321890 >> 172.21.15.2:0/2261321890 conn(0x5626ff4ee4f0 msgr2=0x5626ff4ee8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 1 -- 172.21.15.2:0/2261321890 shutdown_connections 2024-03-15T12:58:05.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 1 -- 172.21.15.2:0/2261321890 wait complete. 2024-03-15T12:58:05.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 1 -- start start 2024-03-15T12:58:05.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 1 Processor -- start 2024-03-15T12:58:05.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7390 0x5626ff501580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4f7d80 0x5626ff501ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4fcef0 0x5626ff508b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ff45ac30 con 0x5626ff4fcef0 2024-03-15T12:58:05.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ff3ffd30 con 0x5626ff4f7d80 2024-03-15T12:58:05.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ff12e310 con 0x5626ff4f7390 2024-03-15T12:58:05.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380bd6e740 10 client.0 _ll_register_callbacks cb 0x5626ff407d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:05.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f3802d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7390 0x5626ff501580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4fcef0 0x5626ff508b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f3802d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7390 0x5626ff501580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:45010/0 (socket says 172.21.15.2:45010) 2024-03-15T12:58:05.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f3802d9d640 1 -- 172.21.15.2:0/2079779815 learned_addr learned my addr 172.21.15.2:0/2079779815 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.836+0000 7f380259c640 1 --2- 172.21.15.2:0/2079779815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4f7d80 0x5626ff501ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f3802d9d640 1 -- 172.21.15.2:0/2079779815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4f7d80 msgr2=0x5626ff501ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f3802d9d640 1 --2- 172.21.15.2:0/2079779815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4f7d80 0x5626ff501ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f3802d9d640 1 -- 172.21.15.2:0/2079779815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4fcef0 msgr2=0x5626ff508b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f3802d9d640 1 --2- 172.21.15.2:0/2079779815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4fcef0 0x5626ff508b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f3802d9d640 1 -- 172.21.15.2:0/2079779815 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380419fa20 con 0x5626ff4f7390 2024-03-15T12:58:05.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f380359e640 1 --2- 172.21.15.2:0/2079779815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4fcef0 0x5626ff508b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:05.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f3802d9d640 1 --2- 172.21.15.2:0/2079779815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7390 0x5626ff501580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f37f4096de0 tx=0x7f37f40a0070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5c686c1225c086 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f37e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T12:58:05.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f37e6ff5640 1 -- 172.21.15.2:0/2079779815 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f40a3120 con 0x5626ff4f7390 2024-03-15T12:58:05.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f37e6ff5640 1 -- 172.21.15.2:0/2079779815 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f37f4098a20 con 0x5626ff4f7390 2024-03-15T12:58:05.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f380bd6e740 1 -- 172.21.15.2:0/2079779815 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626ff15a6c0 con 0x5626ff4f7390 2024-03-15T12:58:05.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f37e6ff5640 1 -- 172.21.15.2:0/2079779815 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37f4098c80 con 0x5626ff4f7390 2024-03-15T12:58:05.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f380bd6e740 10 client.25114 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:05.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f380bd6e740 1 -- 172.21.15.2:0/2079779815 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5626ff4592b0 con 0x5626ff4f7390 2024-03-15T12:58:05.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f37e6ff5640 1 -- 172.21.15.2:0/2079779815 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5626ff15a6c0 con 0x5626ff4f7390 2024-03-15T12:58:05.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f37e6ff5640 1 -- 172.21.15.2:0/2079779815 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f37f4010430 con 0x5626ff4f7390 2024-03-15T12:58:05.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f380bd6e740 10 client.25114 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:05.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.837+0000 7f380bd6e740 10 client.25114 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-15T12:58:05.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.838+0000 7f380bd6e740 20 client.25114 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:05.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.838+0000 7f380bd6e740 1 --2- 172.21.15.2:0/2079779815 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5626ff50cff0 0x5626ff512030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.838+0000 7f380bd6e740 4 client.25114 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:58:05.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.838+0000 7f380bd6e740 1 -- 172.21.15.2:0/2079779815 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626ff1ab270 con 0x5626ff50cff0 2024-03-15T12:58:05.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.838+0000 7f380259c640 1 --2- 172.21.15.2:0/2079779815 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5626ff50cff0 0x5626ff512030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.838+0000 7f380259c640 1 --2- 172.21.15.2:0/2079779815 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5626ff50cff0 0x5626ff512030 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae45e64aa50f70fc server_cookie=1b89f0e50c252f18 in_seq=0 out_seq=0 2024-03-15T12:58:05.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.838+0000 7f37e6ff5640 10 client.25114 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:05.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.839+0000 7f37e6ff5640 1 -- 172.21.15.2:0/2079779815 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5626ff1ab270 con 0x5626ff50cff0 2024-03-15T12:58:05.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.839+0000 7f37e6ff5640 10 client.25114 handle_command_reply: tid=0 2024-03-15T12:58:05.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.839+0000 7f380bd6e740 1 client.25114 shutdown 2024-03-15T12:58:05.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.839+0000 7f380bd6e740 1 -- 172.21.15.2:0/2079779815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7390 msgr2=0x5626ff501580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.839+0000 7f380bd6e740 1 --2- 172.21.15.2:0/2079779815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7390 0x5626ff501580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f37f4096de0 tx=0x7f37f40a0070 comp rx=0 tx=0).stop 2024-03-15T12:58:05.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.839+0000 7f380bd6e740 1 -- 172.21.15.2:0/2079779815 shutdown_connections 2024-03-15T12:58:05.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.839+0000 7f380bd6e740 1 --2- 172.21.15.2:0/2079779815 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5626ff50cff0 0x5626ff512030 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.839+0000 7f380bd6e740 1 --2- 172.21.15.2:0/2079779815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4fcef0 0x5626ff508b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.839+0000 7f380bd6e740 1 --2- 172.21.15.2:0/2079779815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4f7d80 0x5626ff501ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.839+0000 7f380bd6e740 1 --2- 172.21.15.2:0/2079779815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7390 0x5626ff501580 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.839+0000 7f37e6ff5640 0 client.25114 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:05.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.839+0000 7f380bd6e740 1 -- 172.21.15.2:0/2079779815 >> 172.21.15.2:0/2079779815 conn(0x5626ff4ee4f0 msgr2=0x5626ff4ed7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.839+0000 7f380bd6e740 1 -- 172.21.15.2:0/2079779815 shutdown_connections 2024-03-15T12:58:05.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.839+0000 7f380bd6e740 1 -- 172.21.15.2:0/2079779815 wait complete. 2024-03-15T12:58:05.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.839+0000 7f380bd6e740 20 client.25114 trim_cache size 0 max 16384 2024-03-15T12:58:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.856+0000 7f380bd6e740 1 Processor -- start 2024-03-15T12:58:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.856+0000 7f380bd6e740 1 -- start start 2024-03-15T12:58:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.856+0000 7f380bd6e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4f7390 0x5626ff501a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.856+0000 7f380bd6e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4f7d80 0x5626ff508a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.856+0000 7f380bd6e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4fcef0 0x5626ff4f8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.856+0000 7f380bd6e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ff3e0490 con 0x5626ff4f7d80 2024-03-15T12:58:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.856+0000 7f380bd6e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ff50eaa0 con 0x5626ff4f7390 2024-03-15T12:58:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.856+0000 7f380bd6e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ff500dd0 con 0x5626ff4fcef0 2024-03-15T12:58:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.857+0000 7f380259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4f7d80 0x5626ff508a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.857+0000 7f380259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4f7d80 0x5626ff508a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48062/0 (socket says 172.21.15.2:48062) 2024-03-15T12:58:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.857+0000 7f380259c640 1 -- 172.21.15.2:0/3547102637 learned_addr learned my addr 172.21.15.2:0/3547102637 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.857+0000 7f3802d9d640 1 --2- 172.21.15.2:0/3547102637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4f7390 0x5626ff501a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.857+0000 7f380359e640 1 --2- 172.21.15.2:0/3547102637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4fcef0 0x5626ff4f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.857+0000 7f380259c640 1 -- 172.21.15.2:0/3547102637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4fcef0 msgr2=0x5626ff4f8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.857+0000 7f380259c640 1 --2- 172.21.15.2:0/3547102637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4fcef0 0x5626ff4f8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.857+0000 7f380259c640 1 -- 172.21.15.2:0/3547102637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4f7390 msgr2=0x5626ff501a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.857+0000 7f380259c640 1 --2- 172.21.15.2:0/3547102637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4f7390 0x5626ff501a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.857+0000 7f380259c640 1 -- 172.21.15.2:0/3547102637 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626ff3f5a20 con 0x5626ff4f7d80 2024-03-15T12:58:05.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.857+0000 7f380259c640 1 --2- 172.21.15.2:0/3547102637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4f7d80 0x5626ff508a90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f37f8004700 tx=0x7f37f8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba65db2400d310 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.857+0000 7f37e8ff9640 1 -- 172.21.15.2:0/3547102637 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8017040 con 0x5626ff4f7d80 2024-03-15T12:58:05.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.857+0000 7f37e8ff9640 1 -- 172.21.15.2:0/3547102637 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f37f8018040 con 0x5626ff4f7d80 2024-03-15T12:58:05.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.857+0000 7f37e8ff9640 1 -- 172.21.15.2:0/3547102637 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8005020 con 0x5626ff4f7d80 2024-03-15T12:58:05.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 -- 172.21.15.2:0/3547102637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4f7d80 msgr2=0x5626ff508a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 --2- 172.21.15.2:0/3547102637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4f7d80 0x5626ff508a90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f37f8004700 tx=0x7f37f8005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:05.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 -- 172.21.15.2:0/3547102637 shutdown_connections 2024-03-15T12:58:05.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 --2- 172.21.15.2:0/3547102637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4fcef0 0x5626ff4f8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 --2- 172.21.15.2:0/3547102637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4f7d80 0x5626ff508a90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 --2- 172.21.15.2:0/3547102637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4f7390 0x5626ff501a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 -- 172.21.15.2:0/3547102637 >> 172.21.15.2:0/3547102637 conn(0x5626ff4ee4f0 msgr2=0x5626ff4ee8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 -- 172.21.15.2:0/3547102637 shutdown_connections 2024-03-15T12:58:05.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 -- 172.21.15.2:0/3547102637 wait complete. 2024-03-15T12:58:05.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 -- start start 2024-03-15T12:58:05.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 Processor -- start 2024-03-15T12:58:05.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7390 0x5626ff504c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4f7d80 0x5626ff505190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4fcef0 0x5626ff5080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ff509780 con 0x5626ff4fcef0 2024-03-15T12:58:05.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ff4fcb10 con 0x5626ff4f7d80 2024-03-15T12:58:05.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ff508660 con 0x5626ff4f7390 2024-03-15T12:58:05.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380bd6e740 10 client.0 _ll_register_callbacks cb 0x7f37f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:05.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4fcef0 0x5626ff5080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4fcef0 0x5626ff5080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48070/0 (socket says 172.21.15.2:48070) 2024-03-15T12:58:05.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380359e640 1 -- 172.21.15.2:0/3960806839 learned_addr learned my addr 172.21.15.2:0/3960806839 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:05.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f380259c640 1 --2- 172.21.15.2:0/3960806839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4f7d80 0x5626ff505190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.858+0000 7f3802d9d640 1 --2- 172.21.15.2:0/3960806839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7390 0x5626ff504c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f380359e640 1 -- 172.21.15.2:0/3960806839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7390 msgr2=0x5626ff504c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f380359e640 1 --2- 172.21.15.2:0/3960806839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7390 0x5626ff504c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f380359e640 1 -- 172.21.15.2:0/3960806839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4f7d80 msgr2=0x5626ff505190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:05.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f380359e640 1 --2- 172.21.15.2:0/3960806839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4f7d80 0x5626ff505190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f380359e640 1 -- 172.21.15.2:0/3960806839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f8020770 con 0x5626ff4fcef0 2024-03-15T12:58:05.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f380359e640 1 --2- 172.21.15.2:0/3960806839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4fcef0 0x5626ff5080a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009ca0 tx=0x7f37fc048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12eb615a6adbc8b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:05.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f37e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:05.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f37e6ff5640 1 -- 172.21.15.2:0/3960806839 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc048070 con 0x5626ff4fcef0 2024-03-15T12:58:05.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f37e6ff5640 1 -- 172.21.15.2:0/3960806839 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f37fc034c00 con 0x5626ff4fcef0 2024-03-15T12:58:05.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f380bd6e740 1 -- 172.21.15.2:0/3960806839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626ff50ebe0 con 0x5626ff4fcef0 2024-03-15T12:58:05.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f37e6ff5640 1 -- 172.21.15.2:0/3960806839 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37fc034dc0 con 0x5626ff4fcef0 2024-03-15T12:58:05.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f380bd6e740 10 client.15318 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:05.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f380bd6e740 1 -- 172.21.15.2:0/3960806839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5626ff41b780 con 0x5626ff4fcef0 2024-03-15T12:58:05.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f37e6ff5640 1 -- 172.21.15.2:0/3960806839 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f37fc060020 con 0x5626ff4fcef0 2024-03-15T12:58:05.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f37e6ff5640 1 -- 172.21.15.2:0/3960806839 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f37fc004610 con 0x5626ff4fcef0 2024-03-15T12:58:05.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.859+0000 7f380bd6e740 10 client.15318 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:05.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f380bd6e740 10 client.15318 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-15T12:58:05.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f380bd6e740 20 client.15318 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:05.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f380bd6e740 1 --2- 172.21.15.2:0/3960806839 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5626ff56dc20 0x5626ff56a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:05.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f380bd6e740 4 client.15318 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-15T12:58:05.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f380bd6e740 1 -- 172.21.15.2:0/3960806839 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f37f409f9f0 con 0x5626ff56dc20 2024-03-15T12:58:05.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f3802d9d640 1 --2- 172.21.15.2:0/3960806839 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5626ff56dc20 0x5626ff56a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:05.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f37e6ff5640 10 client.15318 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:05.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f3802d9d640 1 --2- 172.21.15.2:0/3960806839 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5626ff56dc20 0x5626ff56a020 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bbc05983b3705af server_cookie=dea4dd95390f0295 in_seq=0 out_seq=0 2024-03-15T12:58:05.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f37e6ff5640 1 -- 172.21.15.2:0/3960806839 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f37f409f9f0 con 0x5626ff56dc20 2024-03-15T12:58:05.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f37e6ff5640 10 client.15318 handle_command_reply: tid=0 2024-03-15T12:58:05.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f380bd6e740 1 client.15318 shutdown 2024-03-15T12:58:05.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f380bd6e740 1 -- 172.21.15.2:0/3960806839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4fcef0 msgr2=0x5626ff5080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f380bd6e740 1 --2- 172.21.15.2:0/3960806839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4fcef0 0x5626ff5080a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009ca0 tx=0x7f37fc048000 comp rx=0 tx=0).stop 2024-03-15T12:58:05.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f380bd6e740 1 -- 172.21.15.2:0/3960806839 shutdown_connections 2024-03-15T12:58:05.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f380bd6e740 1 --2- 172.21.15.2:0/3960806839 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5626ff56dc20 0x5626ff56a020 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f380bd6e740 1 --2- 172.21.15.2:0/3960806839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626ff4fcef0 0x5626ff5080a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f380bd6e740 1 --2- 172.21.15.2:0/3960806839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626ff4f7d80 0x5626ff505190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f37e6ff5640 0 client.15318 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:05.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f380bd6e740 1 --2- 172.21.15.2:0/3960806839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626ff4f7390 0x5626ff504c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.860+0000 7f380bd6e740 1 -- 172.21.15.2:0/3960806839 >> 172.21.15.2:0/3960806839 conn(0x5626ff4ee4f0 msgr2=0x5626ff4f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.861+0000 7f380bd6e740 1 -- 172.21.15.2:0/3960806839 shutdown_connections 2024-03-15T12:58:05.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.861+0000 7f380bd6e740 1 -- 172.21.15.2:0/3960806839 wait complete. 2024-03-15T12:58:05.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.861+0000 7f380bd6e740 20 client.15318 trim_cache size 0 max 16384 2024-03-15T12:58:05.854 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:58:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "2767718a-e2ca-11ee-95c9-87774f69a715", 2024-03-15T12:58:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2024-03-15T12:58:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24539, 2024-03-15T12:58:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-03-15T12:58:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-03-15T12:58:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-03-15T12:58:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 104.457412752, 2024-03-15T12:58:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 28, 2024-03-15T12:58:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 91, 2024-03-15T12:58:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 89, 2024-03-15T12:58:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 111.76380034899999 2024-03-15T12:58:05.855 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:58:05.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.861+0000 7f380918c640 1 -- 172.21.15.2:0/1898598530 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f37ec0a1a30 msgr2=0x7f37ec0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.861+0000 7f380918c640 1 --2- 172.21.15.2:0/1898598530 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f37ec0a1a30 0x7f37ec0a3ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f37fc005cc0 tx=0x7f37fc0323c0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.861+0000 7f380918c640 1 -- 172.21.15.2:0/1898598530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3804100410 msgr2=0x7f3804197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:05.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.861+0000 7f380918c640 1 --2- 172.21.15.2:0/1898598530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3804100410 0x7f3804197550 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f37f400e9c0 tx=0x7f37f400ee80 comp rx=0 tx=0).stop 2024-03-15T12:58:05.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.861+0000 7f380918c640 1 -- 172.21.15.2:0/1898598530 shutdown_connections 2024-03-15T12:58:05.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.861+0000 7f380918c640 1 --2- 172.21.15.2:0/1898598530 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f37ec0a1a30 0x7f37ec0a3ef0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.861+0000 7f380918c640 1 --2- 172.21.15.2:0/1898598530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3804101fc0 0x7f380419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.861+0000 7f380918c640 1 --2- 172.21.15.2:0/1898598530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3804101600 0x7f3804197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.861+0000 7f380918c640 1 --2- 172.21.15.2:0/1898598530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3804100410 0x7f3804197550 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:05.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.861+0000 7f380918c640 1 -- 172.21.15.2:0/1898598530 >> 172.21.15.2:0/1898598530 conn(0x7f38040fb9a0 msgr2=0x7f38040fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:05.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.861+0000 7f380918c640 1 -- 172.21.15.2:0/1898598530 shutdown_connections 2024-03-15T12:58:05.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:05.861+0000 7f380918c640 1 -- 172.21.15.2:0/1898598530 wait complete. 2024-03-15T12:58:05.858 INFO:tasks.cephadm:Setup complete, yielding 2024-03-15T12:58:05.858 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-15T12:58:05.868 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-15T12:58:05.869 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- bash -c 'ceph orch status' 2024-03-15T12:58:05.870 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:05 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2329153843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:05.870 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:05 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3182779329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:06.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:05 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2329153843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:06.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:05 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3182779329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:06.110 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:58:07.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:06 smithi002 ceph-mon[32060]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-15T12:58:07.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:06 smithi184 ceph-mon[35993]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-15T12:58:07.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:06 smithi149 ceph-mon[35927]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-15T12:58:07.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.870+0000 7f3e2950c700 1 -- 172.21.15.2:0/2135185967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e24103bd0 msgr2=0x7f3e2410a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:07.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.870+0000 7f3e2950c700 1 --2- 172.21.15.2:0/2135185967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e24103bd0 0x7f3e2410a480 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c0099c0 tx=0x7f3e1c009cd0 comp rx=0 tx=0).stop 2024-03-15T12:58:07.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.870+0000 7f3e2950c700 1 -- 172.21.15.2:0/2135185967 shutdown_connections 2024-03-15T12:58:07.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.870+0000 7f3e2950c700 1 --2- 172.21.15.2:0/2135185967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e24103bd0 0x7f3e2410a480 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:07.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.870+0000 7f3e2950c700 1 --2- 172.21.15.2:0/2135185967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e24102030 0x7f3e24102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:07.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.870+0000 7f3e2950c700 1 --2- 172.21.15.2:0/2135185967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e24103230 0x7f3e24103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:07.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.870+0000 7f3e2950c700 1 -- 172.21.15.2:0/2135185967 >> 172.21.15.2:0/2135185967 conn(0x7f3e240fd5f0 msgr2=0x7f3e240ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:07.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.870+0000 7f3e2950c700 1 -- 172.21.15.2:0/2135185967 shutdown_connections 2024-03-15T12:58:07.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.870+0000 7f3e2950c700 1 -- 172.21.15.2:0/2135185967 wait complete. 2024-03-15T12:58:07.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.870+0000 7f3e2950c700 1 Processor -- start 2024-03-15T12:58:07.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e2950c700 1 -- start start 2024-03-15T12:58:07.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e2950c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e24102030 0x7f3e24197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:07.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e2950c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e24103230 0x7f3e24198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:07.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e2950c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e24103bd0 0x7f3e2419f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:07.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e2950c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2419fab0 con 0x7f3e24103bd0 2024-03-15T12:58:07.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e2950c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2419fbf0 con 0x7f3e24103230 2024-03-15T12:58:07.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e2950c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2419fd80 con 0x7f3e24102030 2024-03-15T12:58:07.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e237fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e24103bd0 0x7f3e2419f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:07.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e237fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e24103bd0 0x7f3e2419f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48094/0 (socket says 172.21.15.2:48094) 2024-03-15T12:58:07.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e22ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e24102030 0x7f3e24197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:07.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e227fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e24103230 0x7f3e24198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:07.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e237fe700 1 -- 172.21.15.2:0/2592589903 learned_addr learned my addr 172.21.15.2:0/2592589903 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:07.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e237fe700 1 -- 172.21.15.2:0/2592589903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e24102030 msgr2=0x7f3e24197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:07.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e237fe700 1 --2- 172.21.15.2:0/2592589903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e24102030 0x7f3e24197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:07.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e237fe700 1 -- 172.21.15.2:0/2592589903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e24103230 msgr2=0x7f3e24198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:07.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e237fe700 1 --2- 172.21.15.2:0/2592589903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e24103230 0x7f3e24198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:07.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e237fe700 1 -- 172.21.15.2:0/2592589903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e14009c40 con 0x7f3e24103bd0 2024-03-15T12:58:07.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e227fc700 1 --2- 172.21.15.2:0/2592589903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e24103230 0x7f3e24198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:07.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.871+0000 7f3e237fe700 1 --2- 172.21.15.2:0/2592589903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e24103bd0 0x7f3e2419f4e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00b580 tx=0x7f3e1c0055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:07.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.872+0000 7f3e1bfff700 1 -- 172.21.15.2:0/2592589903 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c01d070 con 0x7f3e24103bd0 2024-03-15T12:58:07.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.872+0000 7f3e2950c700 1 -- 172.21.15.2:0/2592589903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0c009520 con 0x7f3e24103bd0 2024-03-15T12:58:07.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.872+0000 7f3e1bfff700 1 -- 172.21.15.2:0/2592589903 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3e1c00be70 con 0x7f3e24103bd0 2024-03-15T12:58:07.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.872+0000 7f3e2950c700 1 -- 172.21.15.2:0/2592589903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1c0095f0 con 0x7f3e24103bd0 2024-03-15T12:58:07.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.872+0000 7f3e1bfff700 1 -- 172.21.15.2:0/2592589903 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c00fb00 con 0x7f3e24103bd0 2024-03-15T12:58:07.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.873+0000 7f3e2950c700 1 -- 172.21.15.2:0/2592589903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e04005330 con 0x7f3e24103bd0 2024-03-15T12:58:07.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.874+0000 7f3e1bfff700 1 -- 172.21.15.2:0/2592589903 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f3e1c00b9a0 con 0x7f3e24103bd0 2024-03-15T12:58:07.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.875+0000 7f3e1bfff700 1 --2- 172.21.15.2:0/2592589903 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e100a1b40 0x7f3e100a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:07.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.875+0000 7f3e22ffd700 1 --2- 172.21.15.2:0/2592589903 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e100a1b40 0x7f3e100a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:07.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.875+0000 7f3e1bfff700 1 -- 172.21.15.2:0/2592589903 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f3e1c0ce720 con 0x7f3e24103bd0 2024-03-15T12:58:07.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.875+0000 7f3e22ffd700 1 --2- 172.21.15.2:0/2592589903 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e100a1b40 0x7f3e100a4000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3e24103090 tx=0x7f3e1400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:07.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:07.879+0000 7f3e1bfff700 1 -- 172.21.15.2:0/2592589903 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3e1c097620 con 0x7f3e24103bd0 2024-03-15T12:58:08.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:08.103+0000 7f3e2950c700 1 -- 172.21.15.2:0/2592589903 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e04000bc0 con 0x7f3e100a1b40 2024-03-15T12:58:08.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:08.104+0000 7f3e1bfff700 1 -- 172.21.15.2:0/2592589903 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f3e04000bc0 con 0x7f3e100a1b40 2024-03-15T12:58:08.090 INFO:teuthology.orchestra.run.smithi002.stdout:Backend: cephadm 2024-03-15T12:58:08.090 INFO:teuthology.orchestra.run.smithi002.stdout:Available: Yes 2024-03-15T12:58:08.090 INFO:teuthology.orchestra.run.smithi002.stdout:Paused: No 2024-03-15T12:58:08.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:08.106+0000 7f3e2950c700 1 -- 172.21.15.2:0/2592589903 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e100a1b40 msgr2=0x7f3e100a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:08.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:08.106+0000 7f3e2950c700 1 --2- 172.21.15.2:0/2592589903 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e100a1b40 0x7f3e100a4000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3e24103090 tx=0x7f3e1400b040 comp rx=0 tx=0).stop 2024-03-15T12:58:08.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:08.106+0000 7f3e2950c700 1 -- 172.21.15.2:0/2592589903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e24103bd0 msgr2=0x7f3e2419f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:08.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:08.106+0000 7f3e2950c700 1 --2- 172.21.15.2:0/2592589903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e24103bd0 0x7f3e2419f4e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00b580 tx=0x7f3e1c0055f0 comp rx=0 tx=0).stop 2024-03-15T12:58:08.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:08.106+0000 7f3e2950c700 1 -- 172.21.15.2:0/2592589903 shutdown_connections 2024-03-15T12:58:08.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:08.106+0000 7f3e2950c700 1 --2- 172.21.15.2:0/2592589903 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e100a1b40 0x7f3e100a4000 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:08.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:08.106+0000 7f3e2950c700 1 --2- 172.21.15.2:0/2592589903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e24103bd0 0x7f3e2419f4e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:08.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:08.106+0000 7f3e2950c700 1 --2- 172.21.15.2:0/2592589903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e24102030 0x7f3e24197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:08.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:08.106+0000 7f3e2950c700 1 --2- 172.21.15.2:0/2592589903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e24103230 0x7f3e24198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:08.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:08.106+0000 7f3e2950c700 1 -- 172.21.15.2:0/2592589903 >> 172.21.15.2:0/2592589903 conn(0x7f3e240fd5f0 msgr2=0x7f3e241064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:08.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:08.106+0000 7f3e2950c700 1 -- 172.21.15.2:0/2592589903 shutdown_connections 2024-03-15T12:58:08.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:08.106+0000 7f3e2950c700 1 -- 172.21.15.2:0/2592589903 wait complete. 2024-03-15T12:58:08.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:08 smithi002 ceph-mon[32060]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:58:08.886 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-15T12:58:09.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:08 smithi184 ceph-mon[35993]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:58:09.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:08 smithi149 ceph-mon[35927]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:58:09.134 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:58:09.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:09 smithi002 ceph-mon[32060]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:10.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:09 smithi184 ceph-mon[35993]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:10.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:09 smithi149 ceph-mon[35927]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:10.710 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:10 smithi002 ceph-mon[32060]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:58:10.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.955+0000 7fbe0e5fe700 1 -- 172.21.15.2:0/4237245282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe08102a20 msgr2=0x7fbe08109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:10.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.955+0000 7fbe0e5fe700 1 --2- 172.21.15.2:0/4237245282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe08102a20 0x7fbe08109340 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000b0a0 tx=0x7fbe0000b3b0 comp rx=0 tx=0).stop 2024-03-15T12:58:10.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.955+0000 7fbe0e5fe700 1 -- 172.21.15.2:0/4237245282 shutdown_connections 2024-03-15T12:58:10.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.955+0000 7fbe0e5fe700 1 --2- 172.21.15.2:0/4237245282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe08102a20 0x7fbe08109340 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:10.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.955+0000 7fbe0e5fe700 1 --2- 172.21.15.2:0/4237245282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe08100f00 0x7fbe08101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:10.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.955+0000 7fbe0e5fe700 1 --2- 172.21.15.2:0/4237245282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe08102060 0x7fbe081024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:10.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.955+0000 7fbe0e5fe700 1 -- 172.21.15.2:0/4237245282 >> 172.21.15.2:0/4237245282 conn(0x7fbe080fc4a0 msgr2=0x7fbe080fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:10.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.955+0000 7fbe0e5fe700 1 -- 172.21.15.2:0/4237245282 shutdown_connections 2024-03-15T12:58:10.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.955+0000 7fbe0e5fe700 1 -- 172.21.15.2:0/4237245282 wait complete. 2024-03-15T12:58:10.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.956+0000 7fbe0e5fe700 1 Processor -- start 2024-03-15T12:58:10.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.956+0000 7fbe0e5fe700 1 -- start start 2024-03-15T12:58:10.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.956+0000 7fbe0e5fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe08100f00 0x7fbe0806b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:10.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.956+0000 7fbe0e5fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe08102060 0x7fbe0819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.956+0000 7fbe0e5fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe08102a20 0x7fbe081a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.956+0000 7fbe0e5fe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe081a1da0 con 0x7fbe08100f00 2024-03-15T12:58:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.956+0000 7fbe0e5fe700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe081a1f10 con 0x7fbe08102060 2024-03-15T12:58:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.956+0000 7fbe0e5fe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe081a20d0 con 0x7fbe08102a20 2024-03-15T12:58:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.956+0000 7fbe07fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe08100f00 0x7fbe0806b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.956+0000 7fbdfffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe08102060 0x7fbe0819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.956+0000 7fbe0cb9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe08102a20 0x7fbe081a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.956+0000 7fbe07fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe08100f00 0x7fbe0806b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46414/0 (socket says 172.21.15.2:46414) 2024-03-15T12:58:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.956+0000 7fbdfffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe08102060 0x7fbe0819f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:51692/0 (socket says 172.21.15.2:51692) 2024-03-15T12:58:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.956+0000 7fbdfffff700 1 -- 172.21.15.2:0/1031118252 learned_addr learned my addr 172.21.15.2:0/1031118252 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.957+0000 7fbe0cb9b700 1 -- 172.21.15.2:0/1031118252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe08102060 msgr2=0x7fbe0819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.957+0000 7fbe0cb9b700 1 --2- 172.21.15.2:0/1031118252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe08102060 0x7fbe0819f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.957+0000 7fbe0cb9b700 1 -- 172.21.15.2:0/1031118252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe08100f00 msgr2=0x7fbe0806b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.957+0000 7fbe0cb9b700 1 --2- 172.21.15.2:0/1031118252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe08100f00 0x7fbe0806b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.957+0000 7fbe0cb9b700 1 -- 172.21.15.2:0/1031118252 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe00009d00 con 0x7fbe08102a20 2024-03-15T12:58:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.957+0000 7fbe07fff700 1 --2- 172.21.15.2:0/1031118252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe08100f00 0x7fbe0806b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.957+0000 7fbdfffff700 1 --2- 172.21.15.2:0/1031118252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe08102060 0x7fbe0819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.957+0000 7fbe0cb9b700 1 --2- 172.21.15.2:0/1031118252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe08102a20 0x7fbe081a16f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbe00007970 tx=0x7fbe00008f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.957+0000 7fbe05ffb700 1 -- 172.21.15.2:0/1031118252 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000e040 con 0x7fbe08102a20 2024-03-15T12:58:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.957+0000 7fbe05ffb700 1 -- 172.21.15.2:0/1031118252 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbe0001d470 con 0x7fbe08102a20 2024-03-15T12:58:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.957+0000 7fbe0e5fe700 1 -- 172.21.15.2:0/1031118252 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe081a2350 con 0x7fbe08102a20 2024-03-15T12:58:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.957+0000 7fbe05ffb700 1 -- 172.21.15.2:0/1031118252 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000052a0 con 0x7fbe08102a20 2024-03-15T12:58:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.957+0000 7fbe0e5fe700 1 -- 172.21.15.2:0/1031118252 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe081a2810 con 0x7fbe08102a20 2024-03-15T12:58:10.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.958+0000 7fbe0e5fe700 1 -- 172.21.15.2:0/1031118252 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe08108250 con 0x7fbe08102a20 2024-03-15T12:58:10.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.959+0000 7fbe05ffb700 1 -- 172.21.15.2:0/1031118252 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fbe0001d5e0 con 0x7fbe08102a20 2024-03-15T12:58:10.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.959+0000 7fbe05ffb700 1 --2- 172.21.15.2:0/1031118252 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbdf40a1b40 0x7fbdf40a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:10.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.960+0000 7fbe07fff700 1 --2- 172.21.15.2:0/1031118252 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbdf40a1b40 0x7fbdf40a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:10.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.960+0000 7fbe05ffb700 1 -- 172.21.15.2:0/1031118252 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fbe000cea80 con 0x7fbe08102a20 2024-03-15T12:58:10.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.960+0000 7fbe07fff700 1 --2- 172.21.15.2:0/1031118252 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbdf40a1b40 0x7fbdf40a4000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8011490 tx=0x7fbdf8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:10.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:10.963+0000 7fbe05ffb700 1 -- 172.21.15.2:0/1031118252 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbe00097980 con 0x7fbe08102a20 2024-03-15T12:58:11.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:10 smithi184 ceph-mon[35993]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:58:11.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:10 smithi149 ceph-mon[35927]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:58:11.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:11.178+0000 7fbe0e5fe700 1 -- 172.21.15.2:0/1031118252 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe0802da30 con 0x7fbdf40a1b40 2024-03-15T12:58:11.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:11.192+0000 7fbe05ffb700 1 -- 172.21.15.2:0/1031118252 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fbe0802da30 con 0x7fbdf40a1b40 2024-03-15T12:58:11.178 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-15T12:58:11.178 INFO:teuthology.orchestra.run.smithi002.stdout:mds.a smithi002 running (2m) 22s ago 119s 17.5M - 19.0.0-1681-gaba3c1d2 80ec0d3daaba 8b4995460d1b 2024-03-15T12:58:11.178 INFO:teuthology.orchestra.run.smithi002.stdout:mds.b smithi149 running (117s) 51s ago 117s 28.3M - 19.0.0-1681-gaba3c1d2 80ec0d3daaba bfb0b9148888 2024-03-15T12:58:11.178 INFO:teuthology.orchestra.run.smithi002.stdout:mds.c smithi184 running (2m) 51s ago 2m 17.5M - 19.0.0-1681-gaba3c1d2 80ec0d3daaba eadf048c5df1 2024-03-15T12:58:11.178 INFO:teuthology.orchestra.run.smithi002.stdout:mds.d smithi002 running (2m) 22s ago 2m 13.7M - 19.0.0-1681-gaba3c1d2 80ec0d3daaba 2ad74e4d29bb 2024-03-15T12:58:11.178 INFO:teuthology.orchestra.run.smithi002.stdout:mds.e smithi149 running (2m) 51s ago 2m 18.7M - 19.0.0-1681-gaba3c1d2 80ec0d3daaba 9b4a7928af3c 2024-03-15T12:58:11.178 INFO:teuthology.orchestra.run.smithi002.stdout:mds.f smithi184 running (2m) 51s ago 2m 17.0M - 19.0.0-1681-gaba3c1d2 80ec0d3daaba eb3f5a7c69a9 2024-03-15T12:58:11.178 INFO:teuthology.orchestra.run.smithi002.stdout:mds.g smithi002 running (2m) 22s ago 2m 16.2M - 19.0.0-1681-gaba3c1d2 80ec0d3daaba 8b4c087eff7f 2024-03-15T12:58:11.179 INFO:teuthology.orchestra.run.smithi002.stdout:mds.h smithi149 running (2m) 51s ago 2m 18.7M - 19.0.0-1681-gaba3c1d2 80ec0d3daaba 0e291e77314f 2024-03-15T12:58:11.179 INFO:teuthology.orchestra.run.smithi002.stdout:mds.i smithi184 running (2m) 51s ago 2m 17.2M - 19.0.0-1681-gaba3c1d2 80ec0d3daaba b78052221147 2024-03-15T12:58:11.179 INFO:teuthology.orchestra.run.smithi002.stdout:mds.j smithi002 running (2m) 22s ago 2m 13.8M - 19.0.0-1681-gaba3c1d2 80ec0d3daaba 8302642d79b0 2024-03-15T12:58:11.179 INFO:teuthology.orchestra.run.smithi002.stdout:mds.k smithi149 running (2m) 51s ago 2m 16.9M - 19.0.0-1681-gaba3c1d2 80ec0d3daaba 0103c35aed93 2024-03-15T12:58:11.179 INFO:teuthology.orchestra.run.smithi002.stdout:mds.l smithi184 running (2m) 51s ago 2m 16.2M - 19.0.0-1681-gaba3c1d2 80ec0d3daaba a8956d8f1c6f 2024-03-15T12:58:11.179 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.x smithi002 *:9283,8765 running (9m) 22s ago 9m 515M - 19.0.0-1681-gaba3c1d2 80ec0d3daaba 52c2b5b203c7 2024-03-15T12:58:11.179 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.y smithi149 *:8443,8765 running (7m) 51s ago 7m 453M - 19.0.0-1681-gaba3c1d2 80ec0d3daaba 1004fc3320cc 2024-03-15T12:58:11.179 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.z smithi184 *:8443,8765 running (7m) 51s ago 7m 452M - 19.0.0-1681-gaba3c1d2 80ec0d3daaba de6c34942d94 2024-03-15T12:58:11.179 INFO:teuthology.orchestra.run.smithi002.stdout:mon.a smithi002 running (9m) 22s ago 9m 51.3M 2048M 19.0.0-1681-gaba3c1d2 80ec0d3daaba 8198c386c33e 2024-03-15T12:58:11.179 INFO:teuthology.orchestra.run.smithi002.stdout:mon.b smithi149 running (7m) 51s ago 7m 39.0M 2048M 19.0.0-1681-gaba3c1d2 80ec0d3daaba 1f242cf58ee4 2024-03-15T12:58:11.179 INFO:teuthology.orchestra.run.smithi002.stdout:mon.c smithi184 running (8m) 51s ago 8m 40.5M 2048M 19.0.0-1681-gaba3c1d2 80ec0d3daaba 535014bd6255 2024-03-15T12:58:11.179 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (7m) 22s ago 7m 53.8M 4273M 19.0.0-1681-gaba3c1d2 80ec0d3daaba 604d60c59f2c 2024-03-15T12:58:11.180 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi149 running (6m) 51s ago 6m 53.6M 1454M 19.0.0-1681-gaba3c1d2 80ec0d3daaba b85f013e2c63 2024-03-15T12:58:11.180 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi184 running (6m) 51s ago 6m 54.3M 4272M 19.0.0-1681-gaba3c1d2 80ec0d3daaba 672a332c1998 2024-03-15T12:58:11.180 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi002 running (6m) 22s ago 6m 47.4M 4273M 19.0.0-1681-gaba3c1d2 80ec0d3daaba 412edb00b7da 2024-03-15T12:58:11.180 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi149 running (5m) 51s ago 5m 45.5M 1454M 19.0.0-1681-gaba3c1d2 80ec0d3daaba 1e1e54d8bd58 2024-03-15T12:58:11.180 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi184 running (5m) 51s ago 5m 43.9M 4272M 19.0.0-1681-gaba3c1d2 80ec0d3daaba 4cb29c7f652f 2024-03-15T12:58:11.180 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6 smithi002 running (4m) 22s ago 4m 50.1M 4273M 19.0.0-1681-gaba3c1d2 80ec0d3daaba 04addd96f0fa 2024-03-15T12:58:11.180 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 smithi149 running (4m) 51s ago 4m 52.4M 1454M 19.0.0-1681-gaba3c1d2 80ec0d3daaba e0d2e9aa7bcb 2024-03-15T12:58:11.180 INFO:teuthology.orchestra.run.smithi002.stdout:osd.8 smithi184 running (4m) 51s ago 4m 44.1M 4272M 19.0.0-1681-gaba3c1d2 80ec0d3daaba 9fbfb93e3645 2024-03-15T12:58:11.180 INFO:teuthology.orchestra.run.smithi002.stdout:osd.9 smithi002 running (3m) 22s ago 3m 49.5M 4273M 19.0.0-1681-gaba3c1d2 80ec0d3daaba fcd0926988c4 2024-03-15T12:58:11.180 INFO:teuthology.orchestra.run.smithi002.stdout:osd.10 smithi149 running (3m) 51s ago 3m 52.7M 1454M 19.0.0-1681-gaba3c1d2 80ec0d3daaba d525ac5997a0 2024-03-15T12:58:11.180 INFO:teuthology.orchestra.run.smithi002.stdout:osd.11 smithi184 running (2m) 51s ago 2m 48.7M 4272M 19.0.0-1681-gaba3c1d2 80ec0d3daaba 7ffc6d74865b 2024-03-15T12:58:11.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:11.194+0000 7fbe0e5fe700 1 -- 172.21.15.2:0/1031118252 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbdf40a1b40 msgr2=0x7fbdf40a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:11.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:11.194+0000 7fbe0e5fe700 1 --2- 172.21.15.2:0/1031118252 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbdf40a1b40 0x7fbdf40a4000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8011490 tx=0x7fbdf8009260 comp rx=0 tx=0).stop 2024-03-15T12:58:11.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:11.194+0000 7fbe0e5fe700 1 -- 172.21.15.2:0/1031118252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe08102a20 msgr2=0x7fbe081a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:11.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:11.194+0000 7fbe0e5fe700 1 --2- 172.21.15.2:0/1031118252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe08102a20 0x7fbe081a16f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbe00007970 tx=0x7fbe00008f50 comp rx=0 tx=0).stop 2024-03-15T12:58:11.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:11.194+0000 7fbe0e5fe700 1 -- 172.21.15.2:0/1031118252 shutdown_connections 2024-03-15T12:58:11.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:11.194+0000 7fbe0e5fe700 1 --2- 172.21.15.2:0/1031118252 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbdf40a1b40 0x7fbdf40a4000 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:11.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:11.194+0000 7fbe0e5fe700 1 --2- 172.21.15.2:0/1031118252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe08100f00 0x7fbe0806b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:11.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:11.194+0000 7fbe0e5fe700 1 --2- 172.21.15.2:0/1031118252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe08102a20 0x7fbe081a16f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:11.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:11.194+0000 7fbe0e5fe700 1 --2- 172.21.15.2:0/1031118252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe08102060 0x7fbe0819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:11.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:11.194+0000 7fbe0e5fe700 1 -- 172.21.15.2:0/1031118252 >> 172.21.15.2:0/1031118252 conn(0x7fbe080fc4a0 msgr2=0x7fbe08105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:11.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:11.194+0000 7fbe0e5fe700 1 -- 172.21.15.2:0/1031118252 shutdown_connections 2024-03-15T12:58:11.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:11.194+0000 7fbe0e5fe700 1 -- 172.21.15.2:0/1031118252 wait complete. 2024-03-15T12:58:11.950 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-15T12:58:12.195 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:58:13.041 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:12 smithi002 ceph-mon[32060]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-15T12:58:13.041 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:12 smithi002 ceph-mon[32060]: from='client.25132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:13.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:12 smithi184 ceph-mon[35993]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-15T12:58:13.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:12 smithi184 ceph-mon[35993]: from='client.25132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:13.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:12 smithi149 ceph-mon[35927]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-15T12:58:13.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:12 smithi149 ceph-mon[35927]: from='client.25132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:14.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.043+0000 7f2164878700 1 -- 172.21.15.2:0/2975008301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160101430 msgr2=0x7f2160101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:14.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.043+0000 7f2164878700 1 --2- 172.21.15.2:0/2975008301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160101430 0x7f2160101850 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2158009990 tx=0x7f2158009ca0 comp rx=0 tx=0).stop 2024-03-15T12:58:14.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.043+0000 7f2164878700 1 -- 172.21.15.2:0/2975008301 shutdown_connections 2024-03-15T12:58:14.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.043+0000 7f2164878700 1 --2- 172.21.15.2:0/2975008301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160101430 0x7f2160101850 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:14.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.043+0000 7f2164878700 1 --2- 172.21.15.2:0/2975008301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2160102b10 0x7f21601093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:14.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.043+0000 7f2164878700 1 --2- 172.21.15.2:0/2975008301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2160102150 0x7f21601025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:14.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.043+0000 7f2164878700 1 -- 172.21.15.2:0/2975008301 >> 172.21.15.2:0/2975008301 conn(0x7f21600fbc50 msgr2=0x7f21600fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:14.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.043+0000 7f2164878700 1 -- 172.21.15.2:0/2975008301 shutdown_connections 2024-03-15T12:58:14.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.043+0000 7f2164878700 1 -- 172.21.15.2:0/2975008301 wait complete. 2024-03-15T12:58:14.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.044+0000 7f2164878700 1 Processor -- start 2024-03-15T12:58:14.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.044+0000 7f2164878700 1 -- start start 2024-03-15T12:58:14.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.044+0000 7f2164878700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160101430 0x7f2160196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:14.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.044+0000 7f2164878700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2160102150 0x7f21601973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:14.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.044+0000 7f2164878700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2160102b10 0x7f216019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:14.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.044+0000 7f2164878700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216019ea30 con 0x7f2160101430 2024-03-15T12:58:14.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.044+0000 7f2164878700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216019eb70 con 0x7f2160102b10 2024-03-15T12:58:14.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.044+0000 7f2164878700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216019ed00 con 0x7f2160102150 2024-03-15T12:58:14.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.044+0000 7f215e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160101430 0x7f2160196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:14.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.044+0000 7f215e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160101430 0x7f2160196ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46428/0 (socket says 172.21.15.2:46428) 2024-03-15T12:58:14.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.044+0000 7f215e59c700 1 -- 172.21.15.2:0/1166486127 learned_addr learned my addr 172.21.15.2:0/1166486127 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:14.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.044+0000 7f215ed9d700 1 --2- 172.21.15.2:0/1166486127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2160102b10 0x7f216019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:14.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.044+0000 7f215dd9b700 1 --2- 172.21.15.2:0/1166486127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2160102150 0x7f21601973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:14.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.045+0000 7f215e59c700 1 -- 172.21.15.2:0/1166486127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2160102150 msgr2=0x7f21601973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:14.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.045+0000 7f215e59c700 1 --2- 172.21.15.2:0/1166486127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2160102150 0x7f21601973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:14.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.045+0000 7f215e59c700 1 -- 172.21.15.2:0/1166486127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2160102b10 msgr2=0x7f216019e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:14.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.045+0000 7f215e59c700 1 --2- 172.21.15.2:0/1166486127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2160102b10 0x7f216019e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:14.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.045+0000 7f215e59c700 1 -- 172.21.15.2:0/1166486127 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21580095f0 con 0x7f2160101430 2024-03-15T12:58:14.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.045+0000 7f215dd9b700 1 --2- 172.21.15.2:0/1166486127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2160102150 0x7f21601973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:14.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.045+0000 7f215e59c700 1 --2- 172.21.15.2:0/1166486127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160101430 0x7f2160196ea0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2158005770 tx=0x7f21580057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:14.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.045+0000 7f21577fe700 1 -- 172.21.15.2:0/1166486127 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215801d070 con 0x7f2160101430 2024-03-15T12:58:14.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.045+0000 7f2164878700 1 -- 172.21.15.2:0/1166486127 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216019ef80 con 0x7f2160101430 2024-03-15T12:58:14.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.045+0000 7f21577fe700 1 -- 172.21.15.2:0/1166486127 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2158021410 con 0x7f2160101430 2024-03-15T12:58:14.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.045+0000 7f21577fe700 1 -- 172.21.15.2:0/1166486127 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158017650 con 0x7f2160101430 2024-03-15T12:58:14.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.045+0000 7f2164878700 1 -- 172.21.15.2:0/1166486127 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216019f330 con 0x7f2160101430 2024-03-15T12:58:14.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.046+0000 7f2164878700 1 -- 172.21.15.2:0/1166486127 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21601083e0 con 0x7f2160101430 2024-03-15T12:58:14.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.047+0000 7f21577fe700 1 -- 172.21.15.2:0/1166486127 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f2158021580 con 0x7f2160101430 2024-03-15T12:58:14.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.048+0000 7f21577fe700 1 --2- 172.21.15.2:0/1166486127 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f214c0a1ac0 0x7f214c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:14.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.048+0000 7f21577fe700 1 -- 172.21.15.2:0/1166486127 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f21580ceaa0 con 0x7f2160101430 2024-03-15T12:58:14.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.048+0000 7f215dd9b700 1 --2- 172.21.15.2:0/1166486127 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f214c0a1ac0 0x7f214c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:14.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.049+0000 7f215dd9b700 1 --2- 172.21.15.2:0/1166486127 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f214c0a1ac0 0x7f214c0a3f80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2160101160 tx=0x7f2148009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:14.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.052+0000 7f21577fe700 1 -- 172.21.15.2:0/1166486127 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f21580979a0 con 0x7f2160101430 2024-03-15T12:58:14.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.266+0000 7f2164878700 1 -- 172.21.15.2:0/1166486127 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f216002da30 con 0x7f214c0a1ac0 2024-03-15T12:58:14.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.272+0000 7f21577fe700 1 -- 172.21.15.2:0/1166486127 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f216002da30 con 0x7f214c0a1ac0 2024-03-15T12:58:14.258 INFO:teuthology.orchestra.run.smithi002.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-15T12:58:14.258 INFO:teuthology.orchestra.run.smithi002.stdout:mds.all 12/12 54s ago 2m smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-03-15T12:58:14.258 INFO:teuthology.orchestra.run.smithi002.stdout:mgr 3/3 54s ago 7m smithi002=x;smithi149=y;smithi184=z;count:3 2024-03-15T12:58:14.258 INFO:teuthology.orchestra.run.smithi002.stdout:mon 3/3 54s ago 8m smithi002:172.21.15.2=a;smithi149:172.21.15.149=b;smithi184:172.21.15.184=c;count:3 2024-03-15T12:58:14.258 INFO:teuthology.orchestra.run.smithi002.stdout:osd 12 54s ago - 2024-03-15T12:58:14.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.274+0000 7f2164878700 1 -- 172.21.15.2:0/1166486127 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f214c0a1ac0 msgr2=0x7f214c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:14.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.274+0000 7f2164878700 1 --2- 172.21.15.2:0/1166486127 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f214c0a1ac0 0x7f214c0a3f80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2160101160 tx=0x7f2148009260 comp rx=0 tx=0).stop 2024-03-15T12:58:14.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.274+0000 7f2164878700 1 -- 172.21.15.2:0/1166486127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160101430 msgr2=0x7f2160196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:14.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.274+0000 7f2164878700 1 --2- 172.21.15.2:0/1166486127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160101430 0x7f2160196ea0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2158005770 tx=0x7f21580057a0 comp rx=0 tx=0).stop 2024-03-15T12:58:14.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.274+0000 7f2164878700 1 -- 172.21.15.2:0/1166486127 shutdown_connections 2024-03-15T12:58:14.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.274+0000 7f2164878700 1 --2- 172.21.15.2:0/1166486127 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f214c0a1ac0 0x7f214c0a3f80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:14.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.274+0000 7f2164878700 1 --2- 172.21.15.2:0/1166486127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160101430 0x7f2160196ea0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:14.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.274+0000 7f2164878700 1 --2- 172.21.15.2:0/1166486127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2160102b10 0x7f216019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:14.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.274+0000 7f2164878700 1 --2- 172.21.15.2:0/1166486127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2160102150 0x7f21601973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:14.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.274+0000 7f2164878700 1 -- 172.21.15.2:0/1166486127 >> 172.21.15.2:0/1166486127 conn(0x7f21600fbc50 msgr2=0x7f2160105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:14.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.274+0000 7f2164878700 1 -- 172.21.15.2:0/1166486127 shutdown_connections 2024-03-15T12:58:14.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:14.274+0000 7f2164878700 1 -- 172.21.15.2:0/1166486127 wait complete. 2024-03-15T12:58:14.876 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:14 smithi002 ceph-mon[32060]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:58:14.906 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-15T12:58:15.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:14 smithi184 ceph-mon[35993]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:58:15.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:14 smithi149 ceph-mon[35927]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:58:15.154 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:58:15.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:15 smithi002 ceph-mon[32060]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:16.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:15 smithi184 ceph-mon[35993]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:16.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:15 smithi149 ceph-mon[35927]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:16.695 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:16 smithi002 ceph-mon[32060]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-15T12:58:16.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.935+0000 7f3e0946f700 1 -- 172.21.15.2:0/3605046089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e04102aa0 msgr2=0x7f3e04104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:16.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.935+0000 7f3e0946f700 1 --2- 172.21.15.2:0/3605046089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e04102aa0 0x7f3e04104e90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009990 tx=0x7f3dfc009ca0 comp rx=0 tx=0).stop 2024-03-15T12:58:16.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.935+0000 7f3e0946f700 1 -- 172.21.15.2:0/3605046089 shutdown_connections 2024-03-15T12:58:16.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.935+0000 7f3e0946f700 1 --2- 172.21.15.2:0/3605046089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e04102aa0 0x7f3e04104e90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:16.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.935+0000 7f3e0946f700 1 --2- 172.21.15.2:0/3605046089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e04107d00 0x7f3e0410a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:16.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.935+0000 7f3e0946f700 1 --2- 172.21.15.2:0/3605046089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e041053d0 0x7f3e041077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:16.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.935+0000 7f3e0946f700 1 -- 172.21.15.2:0/3605046089 >> 172.21.15.2:0/3605046089 conn(0x7f3e040fc4b0 msgr2=0x7f3e040fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:16.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.935+0000 7f3e0946f700 1 -- 172.21.15.2:0/3605046089 shutdown_connections 2024-03-15T12:58:16.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.935+0000 7f3e0946f700 1 -- 172.21.15.2:0/3605046089 wait complete. 2024-03-15T12:58:16.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.936+0000 7f3e0946f700 1 Processor -- start 2024-03-15T12:58:16.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.936+0000 7f3e0946f700 1 -- start start 2024-03-15T12:58:16.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.936+0000 7f3e0946f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e04102aa0 0x7f3e04198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:16.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.936+0000 7f3e0946f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e041053d0 0x7f3e04198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:16.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.936+0000 7f3e0946f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e04107d00 0x7f3e0419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:16.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.936+0000 7f3e0946f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0419fb90 con 0x7f3e04107d00 2024-03-15T12:58:16.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.936+0000 7f3e0946f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0419fcd0 con 0x7f3e041053d0 2024-03-15T12:58:16.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.936+0000 7f3e0946f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0419fe60 con 0x7f3e04102aa0 2024-03-15T12:58:16.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.936+0000 7f3e037fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e04107d00 0x7f3e0419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:16.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.936+0000 7f3e037fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e04107d00 0x7f3e0419f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46452/0 (socket says 172.21.15.2:46452) 2024-03-15T12:58:16.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.936+0000 7f3e037fe700 1 -- 172.21.15.2:0/585639404 learned_addr learned my addr 172.21.15.2:0/585639404 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:16.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.936+0000 7f3e02ffd700 1 --2- 172.21.15.2:0/585639404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e04102aa0 0x7f3e04198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:16.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.936+0000 7f3e027fc700 1 --2- 172.21.15.2:0/585639404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e041053d0 0x7f3e04198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:16.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.937+0000 7f3e037fe700 1 -- 172.21.15.2:0/585639404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e04102aa0 msgr2=0x7f3e04198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:16.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.937+0000 7f3e037fe700 1 --2- 172.21.15.2:0/585639404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e04102aa0 0x7f3e04198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:16.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.937+0000 7f3e037fe700 1 -- 172.21.15.2:0/585639404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e041053d0 msgr2=0x7f3e04198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:16.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.937+0000 7f3e037fe700 1 --2- 172.21.15.2:0/585639404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e041053d0 0x7f3e04198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:16.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.937+0000 7f3e037fe700 1 -- 172.21.15.2:0/585639404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc0095f0 con 0x7f3e04107d00 2024-03-15T12:58:16.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.937+0000 7f3e027fc700 1 --2- 172.21.15.2:0/585639404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e041053d0 0x7f3e04198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:16.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.937+0000 7f3e037fe700 1 --2- 172.21.15.2:0/585639404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e04107d00 0x7f3e0419f5c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3df400efc0 tx=0x7f3df400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:16.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.937+0000 7f3dfbfff700 1 -- 172.21.15.2:0/585639404 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df4018070 con 0x7f3e04107d00 2024-03-15T12:58:16.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.937+0000 7f3e0946f700 1 -- 172.21.15.2:0/585639404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e041a00e0 con 0x7f3e04107d00 2024-03-15T12:58:16.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.937+0000 7f3dfbfff700 1 -- 172.21.15.2:0/585639404 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3df400f040 con 0x7f3e04107d00 2024-03-15T12:58:16.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.937+0000 7f3dfbfff700 1 -- 172.21.15.2:0/585639404 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df400ce40 con 0x7f3e04107d00 2024-03-15T12:58:16.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.937+0000 7f3e0946f700 1 -- 172.21.15.2:0/585639404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e041a05e0 con 0x7f3e04107d00 2024-03-15T12:58:16.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.938+0000 7f3e0946f700 1 -- 172.21.15.2:0/585639404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e040fe030 con 0x7f3e04107d00 2024-03-15T12:58:16.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.939+0000 7f3dfbfff700 1 -- 172.21.15.2:0/585639404 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f3df4003a00 con 0x7f3e04107d00 2024-03-15T12:58:16.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.940+0000 7f3dfbfff700 1 --2- 172.21.15.2:0/585639404 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3df00a1ac0 0x7f3df00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:16.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.940+0000 7f3dfbfff700 1 -- 172.21.15.2:0/585639404 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f3df40cd930 con 0x7f3e04107d00 2024-03-15T12:58:16.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.940+0000 7f3e02ffd700 1 --2- 172.21.15.2:0/585639404 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3df00a1ac0 0x7f3df00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:16.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.940+0000 7f3e02ffd700 1 --2- 172.21.15.2:0/585639404 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3df00a1ac0 0x7f3df00a3f80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00b2e0 tx=0x7f3dfc00b850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:16.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:16.943+0000 7f3dfbfff700 1 -- 172.21.15.2:0/585639404 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3df4096830 con 0x7f3e04107d00 2024-03-15T12:58:17.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:16 smithi184 ceph-mon[35993]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-15T12:58:17.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:16 smithi149 ceph-mon[35927]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-15T12:58:17.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:17.154+0000 7f3e0946f700 1 -- 172.21.15.2:0/585639404 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e0402d990 con 0x7f3df00a1ac0 2024-03-15T12:58:17.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:17.156+0000 7f3dfbfff700 1 -- 172.21.15.2:0/585639404 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f3e0402d990 con 0x7f3df00a1ac0 2024-03-15T12:58:17.142 INFO:teuthology.orchestra.run.smithi002.stdout:HOST ADDR LABELS STATUS 2024-03-15T12:58:17.142 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002 172.21.15.2 2024-03-15T12:58:17.142 INFO:teuthology.orchestra.run.smithi002.stdout:smithi149 172.21.15.149 2024-03-15T12:58:17.142 INFO:teuthology.orchestra.run.smithi002.stdout:smithi184 172.21.15.184 2024-03-15T12:58:17.142 INFO:teuthology.orchestra.run.smithi002.stdout:3 hosts in cluster 2024-03-15T12:58:17.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:17.158+0000 7f3e0946f700 1 -- 172.21.15.2:0/585639404 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3df00a1ac0 msgr2=0x7f3df00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:17.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:17.158+0000 7f3e0946f700 1 --2- 172.21.15.2:0/585639404 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3df00a1ac0 0x7f3df00a3f80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00b2e0 tx=0x7f3dfc00b850 comp rx=0 tx=0).stop 2024-03-15T12:58:17.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:17.158+0000 7f3e0946f700 1 -- 172.21.15.2:0/585639404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e04107d00 msgr2=0x7f3e0419f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:17.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:17.158+0000 7f3e0946f700 1 --2- 172.21.15.2:0/585639404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e04107d00 0x7f3e0419f5c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3df400efc0 tx=0x7f3df400c3a0 comp rx=0 tx=0).stop 2024-03-15T12:58:17.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:17.158+0000 7f3e0946f700 1 -- 172.21.15.2:0/585639404 shutdown_connections 2024-03-15T12:58:17.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:17.158+0000 7f3e0946f700 1 --2- 172.21.15.2:0/585639404 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3df00a1ac0 0x7f3df00a3f80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:17.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:17.158+0000 7f3e0946f700 1 --2- 172.21.15.2:0/585639404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e04107d00 0x7f3e0419f5c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:17.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:17.158+0000 7f3e0946f700 1 --2- 172.21.15.2:0/585639404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e04102aa0 0x7f3e04198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:17.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:17.158+0000 7f3e0946f700 1 --2- 172.21.15.2:0/585639404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e041053d0 0x7f3e04198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:17.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:17.158+0000 7f3e0946f700 1 -- 172.21.15.2:0/585639404 >> 172.21.15.2:0/585639404 conn(0x7f3e040fc4b0 msgr2=0x7f3e04104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:17.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:17.158+0000 7f3e0946f700 1 -- 172.21.15.2:0/585639404 shutdown_connections 2024-03-15T12:58:17.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:17.158+0000 7f3e0946f700 1 -- 172.21.15.2:0/585639404 wait complete. 2024-03-15T12:58:17.884 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-15T12:58:18.132 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:58:18.912 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:18 smithi002 ceph-mon[32060]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:58:18.912 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:18 smithi002 ceph-mon[32060]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:19.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:18 smithi184 ceph-mon[35993]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:58:19.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:18 smithi184 ceph-mon[35993]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:19.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:18 smithi149 ceph-mon[35927]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:58:19.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:18 smithi149 ceph-mon[35927]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:19.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.795+0000 7f823bdaf700 1 -- 172.21.15.2:0/1161422208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82341018b0 msgr2=0x7f8234101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:19.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.795+0000 7f823bdaf700 1 --2- 172.21.15.2:0/1161422208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82341018b0 0x7f8234101d10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f82240099c0 tx=0x7f8224009cd0 comp rx=0 tx=0).stop 2024-03-15T12:58:19.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.795+0000 7f823bdaf700 1 -- 172.21.15.2:0/1161422208 shutdown_connections 2024-03-15T12:58:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.795+0000 7f823bdaf700 1 --2- 172.21.15.2:0/1161422208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8234102250 0x7f8234108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.795+0000 7f823bdaf700 1 --2- 172.21.15.2:0/1161422208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82341006b0 0x7f8234100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.796+0000 7f823bdaf700 1 --2- 172.21.15.2:0/1161422208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82341018b0 0x7f8234101d10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.796+0000 7f823bdaf700 1 -- 172.21.15.2:0/1161422208 >> 172.21.15.2:0/1161422208 conn(0x7f82340fbc30 msgr2=0x7f82340fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.796+0000 7f823bdaf700 1 -- 172.21.15.2:0/1161422208 shutdown_connections 2024-03-15T12:58:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.796+0000 7f823bdaf700 1 -- 172.21.15.2:0/1161422208 wait complete. 2024-03-15T12:58:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.796+0000 7f823bdaf700 1 Processor -- start 2024-03-15T12:58:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.796+0000 7f823bdaf700 1 -- start start 2024-03-15T12:58:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.796+0000 7f823bdaf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82341006b0 0x7f8234197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.796+0000 7f823bdaf700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82341018b0 0x7f8234198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.796+0000 7f823bdaf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8234102250 0x7f823419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.796+0000 7f823bdaf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823419fb80 con 0x7f8234102250 2024-03-15T12:58:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.796+0000 7f823bdaf700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823419fcc0 con 0x7f82341018b0 2024-03-15T12:58:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.796+0000 7f823bdaf700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823419fe50 con 0x7f82341006b0 2024-03-15T12:58:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f823a34c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8234102250 0x7f823419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f8239b4b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82341006b0 0x7f8234197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f823934a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82341018b0 0x7f8234198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f8239b4b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82341006b0 0x7f8234197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:50522/0 (socket says 172.21.15.2:50522) 2024-03-15T12:58:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f8239b4b700 1 -- 172.21.15.2:0/3793659703 learned_addr learned my addr 172.21.15.2:0/3793659703 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f823a34c700 1 -- 172.21.15.2:0/3793659703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82341006b0 msgr2=0x7f8234197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f823a34c700 1 --2- 172.21.15.2:0/3793659703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82341006b0 0x7f8234197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f823a34c700 1 -- 172.21.15.2:0/3793659703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82341018b0 msgr2=0x7f8234198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f823a34c700 1 --2- 172.21.15.2:0/3793659703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82341018b0 0x7f8234198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f823a34c700 1 -- 172.21.15.2:0/3793659703 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82240095f0 con 0x7f8234102250 2024-03-15T12:58:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f8239b4b700 1 --2- 172.21.15.2:0/3793659703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82341006b0 0x7f8234197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f823a34c700 1 --2- 172.21.15.2:0/3793659703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8234102250 0x7f823419f5b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f822c009f60 tx=0x7f822c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:19.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f822affd700 1 -- 172.21.15.2:0/3793659703 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c018070 con 0x7f8234102250 2024-03-15T12:58:19.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f822affd700 1 -- 172.21.15.2:0/3793659703 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f822c00f040 con 0x7f8234102250 2024-03-15T12:58:19.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f823bdaf700 1 -- 172.21.15.2:0/3793659703 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82341a00d0 con 0x7f8234102250 2024-03-15T12:58:19.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f822affd700 1 -- 172.21.15.2:0/3793659703 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00cd30 con 0x7f8234102250 2024-03-15T12:58:19.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.797+0000 7f823bdaf700 1 -- 172.21.15.2:0/3793659703 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82341a05d0 con 0x7f8234102250 2024-03-15T12:58:19.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.798+0000 7f823bdaf700 1 -- 172.21.15.2:0/3793659703 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8234068ca0 con 0x7f8234102250 2024-03-15T12:58:19.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.799+0000 7f822affd700 1 -- 172.21.15.2:0/3793659703 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f822c004ad0 con 0x7f8234102250 2024-03-15T12:58:19.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.800+0000 7f822affd700 1 --2- 172.21.15.2:0/3793659703 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f82200a1b40 0x7f82200a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:19.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.800+0000 7f8239b4b700 1 --2- 172.21.15.2:0/3793659703 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f82200a1b40 0x7f82200a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:19.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.800+0000 7f822affd700 1 -- 172.21.15.2:0/3793659703 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f822c0cd180 con 0x7f8234102250 2024-03-15T12:58:19.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.801+0000 7f8239b4b700 1 --2- 172.21.15.2:0/3793659703 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f82200a1b40 0x7f82200a4000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8230005950 tx=0x7f8230010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:19.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:19.805+0000 7f822affd700 1 -- 172.21.15.2:0/3793659703 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f822c0960e0 con 0x7f8234102250 2024-03-15T12:58:20.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:20.016+0000 7f823bdaf700 1 -- 172.21.15.2:0/3793659703 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f82341084e0 con 0x7f82200a1b40 2024-03-15T12:58:20.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:20.018+0000 7f822affd700 1 -- 172.21.15.2:0/3793659703 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f82341084e0 con 0x7f82200a1b40 2024-03-15T12:58:20.004 INFO:teuthology.orchestra.run.smithi002.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-15T12:58:20.004 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EV480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-03-15T12:58:20.004 INFO:teuthology.orchestra.run.smithi002.stdout:smithi149 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-15T12:58:20.004 INFO:teuthology.orchestra.run.smithi002.stdout:smithi184 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT626500CE400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-15T12:58:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:20.020+0000 7f823bdaf700 1 -- 172.21.15.2:0/3793659703 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f82200a1b40 msgr2=0x7f82200a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:20.020+0000 7f823bdaf700 1 --2- 172.21.15.2:0/3793659703 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f82200a1b40 0x7f82200a4000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8230005950 tx=0x7f8230010040 comp rx=0 tx=0).stop 2024-03-15T12:58:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:20.020+0000 7f823bdaf700 1 -- 172.21.15.2:0/3793659703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8234102250 msgr2=0x7f823419f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:20.020+0000 7f823bdaf700 1 --2- 172.21.15.2:0/3793659703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8234102250 0x7f823419f5b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f822c009f60 tx=0x7f822c009f90 comp rx=0 tx=0).stop 2024-03-15T12:58:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:20.020+0000 7f823bdaf700 1 -- 172.21.15.2:0/3793659703 shutdown_connections 2024-03-15T12:58:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:20.020+0000 7f823bdaf700 1 --2- 172.21.15.2:0/3793659703 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f82200a1b40 0x7f82200a4000 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:20.020+0000 7f823bdaf700 1 --2- 172.21.15.2:0/3793659703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8234102250 0x7f823419f5b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:20.020+0000 7f823bdaf700 1 --2- 172.21.15.2:0/3793659703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82341006b0 0x7f8234197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:20.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:20.020+0000 7f823bdaf700 1 --2- 172.21.15.2:0/3793659703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82341018b0 0x7f8234198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:20.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:20.020+0000 7f823bdaf700 1 -- 172.21.15.2:0/3793659703 >> 172.21.15.2:0/3793659703 conn(0x7f82340fbc30 msgr2=0x7f8234106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:20.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:20.020+0000 7f823bdaf700 1 -- 172.21.15.2:0/3793659703 shutdown_connections 2024-03-15T12:58:20.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:20.020+0000 7f823bdaf700 1 -- 172.21.15.2:0/3793659703 wait complete. 2024-03-15T12:58:20.752 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-15T12:58:20.765 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-15T12:58:20.766 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-15T12:58:21.011 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:58:21.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:20 smithi002 ceph-mon[32060]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:58:21.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:20 smithi184 ceph-mon[35993]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:58:21.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:20 smithi149 ceph-mon[35927]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:58:21.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:21 smithi002 ceph-mon[32060]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:22.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:21 smithi184 ceph-mon[35993]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:22.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:21 smithi149 ceph-mon[35927]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:22.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.814+0000 7ff6da893700 1 -- 172.21.15.2:0/805007321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6d4102a30 msgr2=0x7ff6d4109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:22.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.814+0000 7ff6da893700 1 --2- 172.21.15.2:0/805007321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6d4102a30 0x7ff6d4109350 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00b0a0 tx=0x7ff6cc00b3b0 comp rx=0 tx=0).stop 2024-03-15T12:58:22.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.814+0000 7ff6da893700 1 -- 172.21.15.2:0/805007321 shutdown_connections 2024-03-15T12:58:22.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.814+0000 7ff6da893700 1 --2- 172.21.15.2:0/805007321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6d4102a30 0x7ff6d4109350 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:22.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.814+0000 7ff6da893700 1 --2- 172.21.15.2:0/805007321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6d4100f10 0x7ff6d4101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:22.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.814+0000 7ff6da893700 1 --2- 172.21.15.2:0/805007321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d4102070 0x7ff6d41024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:22.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.814+0000 7ff6da893700 1 -- 172.21.15.2:0/805007321 >> 172.21.15.2:0/805007321 conn(0x7ff6d40fc4b0 msgr2=0x7ff6d40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:22.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.814+0000 7ff6da893700 1 -- 172.21.15.2:0/805007321 shutdown_connections 2024-03-15T12:58:22.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.814+0000 7ff6da893700 1 -- 172.21.15.2:0/805007321 wait complete. 2024-03-15T12:58:22.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.814+0000 7ff6da893700 1 Processor -- start 2024-03-15T12:58:22.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6da893700 1 -- start start 2024-03-15T12:58:22.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6da893700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6d4100f10 0x7ff6d419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:22.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6da893700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d4102070 0x7ff6d419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:22.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6da893700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6d4102a30 0x7ff6d41a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:22.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6da893700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d41a1e50 con 0x7ff6d4100f10 2024-03-15T12:58:22.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6da893700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d41a1f90 con 0x7ff6d4102a30 2024-03-15T12:58:22.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6da893700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d41a2120 con 0x7ff6d4102070 2024-03-15T12:58:22.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6d3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6d4100f10 0x7ff6d419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:22.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6d3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6d4100f10 0x7ff6d419a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33340/0 (socket says 172.21.15.2:33340) 2024-03-15T12:58:22.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6d3fff700 1 -- 172.21.15.2:0/1856130865 learned_addr learned my addr 172.21.15.2:0/1856130865 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:22.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6d8e30700 1 --2- 172.21.15.2:0/1856130865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6d4102a30 0x7ff6d41a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:22.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6d37fe700 1 --2- 172.21.15.2:0/1856130865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d4102070 0x7ff6d419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:22.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6d3fff700 1 -- 172.21.15.2:0/1856130865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d4102070 msgr2=0x7ff6d419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:22.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6d3fff700 1 --2- 172.21.15.2:0/1856130865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d4102070 0x7ff6d419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:22.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6d3fff700 1 -- 172.21.15.2:0/1856130865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6d4102a30 msgr2=0x7ff6d41a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:22.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6d3fff700 1 --2- 172.21.15.2:0/1856130865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6d4102a30 0x7ff6d41a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:22.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6d3fff700 1 -- 172.21.15.2:0/1856130865 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc009d00 con 0x7ff6d4100f10 2024-03-15T12:58:22.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.815+0000 7ff6d3fff700 1 --2- 172.21.15.2:0/1856130865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6d4100f10 0x7ff6d419a2c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400e9d0 tx=0x7ff6c400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:22.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.816+0000 7ff6d17fa700 1 -- 172.21.15.2:0/1856130865 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4009790 con 0x7ff6d4100f10 2024-03-15T12:58:22.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.816+0000 7ff6da893700 1 -- 172.21.15.2:0/1856130865 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d41a2400 con 0x7ff6d4100f10 2024-03-15T12:58:22.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.816+0000 7ff6d17fa700 1 -- 172.21.15.2:0/1856130865 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff6c400ee70 con 0x7ff6d4100f10 2024-03-15T12:58:22.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.816+0000 7ff6d17fa700 1 -- 172.21.15.2:0/1856130865 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4018680 con 0x7ff6d4100f10 2024-03-15T12:58:22.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.816+0000 7ff6da893700 1 -- 172.21.15.2:0/1856130865 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d41a2950 con 0x7ff6d4100f10 2024-03-15T12:58:22.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.817+0000 7ff6da893700 1 -- 172.21.15.2:0/1856130865 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d4068ca0 con 0x7ff6d4100f10 2024-03-15T12:58:22.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.818+0000 7ff6d17fa700 1 -- 172.21.15.2:0/1856130865 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7ff6c400cbd0 con 0x7ff6d4100f10 2024-03-15T12:58:22.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.818+0000 7ff6d17fa700 1 --2- 172.21.15.2:0/1856130865 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff6bc0a1ac0 0x7ff6bc0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:22.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.819+0000 7ff6d37fe700 1 --2- 172.21.15.2:0/1856130865 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff6bc0a1ac0 0x7ff6bc0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:22.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.819+0000 7ff6d17fa700 1 -- 172.21.15.2:0/1856130865 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7ff6c4014070 con 0x7ff6d4100f10 2024-03-15T12:58:22.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.819+0000 7ff6d37fe700 1 --2- 172.21.15.2:0/1856130865 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff6bc0a1ac0 0x7ff6bc0a3f80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009c60 tx=0x7ff6c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:22.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:22.822+0000 7ff6d17fa700 1 -- 172.21.15.2:0/1856130865 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff6c4096ac0 con 0x7ff6d4100f10 2024-03-15T12:58:23.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:22 smithi002 ceph-mon[32060]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-15T12:58:23.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:22 smithi184 ceph-mon[35993]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-15T12:58:23.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:23.073+0000 7ff6da893700 1 -- 172.21.15.2:0/1856130865 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff6d410c860 con 0x7ff6d4100f10 2024-03-15T12:58:23.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:23.074+0000 7ff6d17fa700 1 -- 172.21.15.2:0/1856130865 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+3034 (secure 0 0 0) 0x7ff6c4026bf0 con 0x7ff6d4100f10 2024-03-15T12:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout:e28 2024-03-15T12:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-15T12:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-15T12:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-15T12:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-15T12:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-15T12:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 28 2024-03-15T12:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-15T12:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-15T12:56:20.559317+0000 2024-03-15T12:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-15T12:56:43.433836+0000 2024-03-15T12:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-15T12:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-15T12:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 300 2024-03-15T12:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-15T12:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-15T12:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-15T12:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-15T12:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-15T12:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 0 2024-03-15T12:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-15T12:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 5 2024-03-15T12:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:in 0,1,2,3,4 2024-03-15T12:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24539,1=24500,2=24464,3=24460,4=24434} 2024-03-15T12:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-15T12:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-15T12:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-15T12:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3,4] 2024-03-15T12:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-15T12:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data disabled 2024-03-15T12:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-15T12:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-15T12:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 0 2024-03-15T12:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 24539 members: 24464,24500,24460,24539,24434 2024-03-15T12:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.b{0:24539} state up:active seq 3 addr [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] compat {c=[1],r=[1],i=[fff]}] 2024-03-15T12:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.i{0:24499} state up:standby-replay seq 1 addr [v2:172.21.15.184:6838/1170793322,v1:172.21.15.184:6839/1170793322] compat {c=[1],r=[1],i=[fff]}] 2024-03-15T12:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.h{1:24500} state up:active seq 11 addr [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] compat {c=[1],r=[1],i=[fff]}] 2024-03-15T12:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.c{1:24472} state up:standby-replay seq 1 addr [v2:172.21.15.184:6836/3698954880,v1:172.21.15.184:6837/3698954880] compat {c=[1],r=[1],i=[fff]}] 2024-03-15T12:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.e{2:24464} state up:active seq 13 addr [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] compat {c=[1],r=[1],i=[fff]}] 2024-03-15T12:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.l{2:24406} state up:standby-replay seq 1 addr [v2:172.21.15.184:6832/730881833,v1:172.21.15.184:6833/730881833] compat {c=[1],r=[1],i=[fff]}] 2024-03-15T12:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.f{3:24460} state up:active seq 13 addr [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] compat {c=[1],r=[1],i=[fff]}] 2024-03-15T12:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.a{3:14652} state up:standby-replay seq 1 addr [v2:172.21.15.2:6840/3747539896,v1:172.21.15.2:6841/3747539896] compat {c=[1],r=[1],i=[fff]}] 2024-03-15T12:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.k{4:24434} state up:active seq 16 addr [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] compat {c=[1],r=[1],i=[fff]}] 2024-03-15T12:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.g{4:14640} state up:standby-replay seq 1 addr [v2:172.21.15.2:6838/2513297233,v1:172.21.15.2:6839/2513297233] compat {c=[1],r=[1],i=[fff]}] 2024-03-15T12:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-15T12:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.j{-1:14562} state up:standby seq 1 addr [v2:172.21.15.2:6834/245353007,v1:172.21.15.2:6835/245353007] compat {c=[1],r=[1],i=[fff]}] 2024-03-15T12:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.d{-1:14574} state up:standby seq 1 addr [v2:172.21.15.2:6836/1406254318,v1:172.21.15.2:6837/1406254318] compat {c=[1],r=[1],i=[fff]}] 2024-03-15T12:58:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:23.076+0000 7ff6da893700 1 -- 172.21.15.2:0/1856130865 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff6bc0a1ac0 msgr2=0x7ff6bc0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:23.076+0000 7ff6da893700 1 --2- 172.21.15.2:0/1856130865 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff6bc0a1ac0 0x7ff6bc0a3f80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009c60 tx=0x7ff6c8009260 comp rx=0 tx=0).stop 2024-03-15T12:58:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:23.076+0000 7ff6da893700 1 -- 172.21.15.2:0/1856130865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6d4100f10 msgr2=0x7ff6d419a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:23.076+0000 7ff6da893700 1 --2- 172.21.15.2:0/1856130865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6d4100f10 0x7ff6d419a2c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400e9d0 tx=0x7ff6c400ece0 comp rx=0 tx=0).stop 2024-03-15T12:58:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:23.076+0000 7ff6da893700 1 -- 172.21.15.2:0/1856130865 shutdown_connections 2024-03-15T12:58:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:23.076+0000 7ff6da893700 1 --2- 172.21.15.2:0/1856130865 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff6bc0a1ac0 0x7ff6bc0a3f80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:23.076+0000 7ff6da893700 1 --2- 172.21.15.2:0/1856130865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6d4100f10 0x7ff6d419a2c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:23.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:23.076+0000 7ff6da893700 1 --2- 172.21.15.2:0/1856130865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6d4102a30 0x7ff6d41a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:23.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:23.076+0000 7ff6da893700 1 --2- 172.21.15.2:0/1856130865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d4102070 0x7ff6d419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:23.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:23.076+0000 7ff6da893700 1 -- 172.21.15.2:0/1856130865 >> 172.21.15.2:0/1856130865 conn(0x7ff6d40fc4b0 msgr2=0x7ff6d40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:23.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:23.076+0000 7ff6da893700 1 -- 172.21.15.2:0/1856130865 shutdown_connections 2024-03-15T12:58:23.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:23.076+0000 7ff6da893700 1 -- 172.21.15.2:0/1856130865 wait complete. 2024-03-15T12:58:23.065 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 28 2024-03-15T12:58:23.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:22 smithi149 ceph-mon[35927]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-15T12:58:23.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:23 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1856130865' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-15T12:58:23.890 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-15T12:58:24.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:23 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1856130865' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-15T12:58:24.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:23 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1856130865' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-15T12:58:24.137 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:58:24.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:24 smithi002 ceph-mon[32060]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:58:25.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:24 smithi184 ceph-mon[35993]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:58:25.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:24 smithi149 ceph-mon[35927]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:58:25.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.873+0000 7f0a7016c700 1 -- 172.21.15.2:0/1899131000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a681024e0 msgr2=0x7f0a68102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:25.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.873+0000 7f0a7016c700 1 --2- 172.21.15.2:0/1899131000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a681024e0 0x7f0a68102900 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800b0a0 tx=0x7f0a5800b3b0 comp rx=0 tx=0).stop 2024-03-15T12:58:25.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.873+0000 7f0a7016c700 1 -- 172.21.15.2:0/1899131000 shutdown_connections 2024-03-15T12:58:25.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.873+0000 7f0a7016c700 1 --2- 172.21.15.2:0/1899131000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a681024e0 0x7f0a68102900 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:25.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.873+0000 7f0a7016c700 1 --2- 172.21.15.2:0/1899131000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a68101b00 0x7f0a681093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:25.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.873+0000 7f0a7016c700 1 --2- 172.21.15.2:0/1899131000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a68101140 0x7f0a681015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:25.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.873+0000 7f0a7016c700 1 -- 172.21.15.2:0/1899131000 >> 172.21.15.2:0/1899131000 conn(0x7f0a680fbc50 msgr2=0x7f0a680fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:25.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.873+0000 7f0a7016c700 1 -- 172.21.15.2:0/1899131000 shutdown_connections 2024-03-15T12:58:25.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.873+0000 7f0a7016c700 1 -- 172.21.15.2:0/1899131000 wait complete. 2024-03-15T12:58:25.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.874+0000 7f0a7016c700 1 Processor -- start 2024-03-15T12:58:25.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.874+0000 7f0a7016c700 1 -- start start 2024-03-15T12:58:25.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.874+0000 7f0a7016c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a68101140 0x7f0a68198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:25.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.874+0000 7f0a7016c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a68101b00 0x7f0a68198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:25.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.874+0000 7f0a7016c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a681024e0 0x7f0a6819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:25.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.874+0000 7f0a7016c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6819fca0 con 0x7f0a681024e0 2024-03-15T12:58:25.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.874+0000 7f0a7016c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6819fde0 con 0x7f0a68101b00 2024-03-15T12:58:25.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.874+0000 7f0a7016c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6819ff70 con 0x7f0a68101140 2024-03-15T12:58:25.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.874+0000 7f0a6d707700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a68101b00 0x7f0a68198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.874+0000 7f0a6e709700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a681024e0 0x7f0a6819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.874+0000 7f0a6d707700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a68101b00 0x7f0a68198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50922/0 (socket says 172.21.15.2:50922) 2024-03-15T12:58:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.874+0000 7f0a6df08700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a68101140 0x7f0a68198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.874+0000 7f0a6d707700 1 -- 172.21.15.2:0/3413473431 learned_addr learned my addr 172.21.15.2:0/3413473431 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.875+0000 7f0a6d707700 1 -- 172.21.15.2:0/3413473431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a68101140 msgr2=0x7f0a68198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.875+0000 7f0a6d707700 1 --2- 172.21.15.2:0/3413473431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a68101140 0x7f0a68198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.875+0000 7f0a6d707700 1 -- 172.21.15.2:0/3413473431 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a681024e0 msgr2=0x7f0a6819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.875+0000 7f0a6d707700 1 --2- 172.21.15.2:0/3413473431 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a681024e0 0x7f0a6819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.875+0000 7f0a6d707700 1 -- 172.21.15.2:0/3413473431 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a58009d00 con 0x7f0a68101b00 2024-03-15T12:58:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.875+0000 7f0a6df08700 1 --2- 172.21.15.2:0/3413473431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a68101140 0x7f0a68198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.875+0000 7f0a6e709700 1 --2- 172.21.15.2:0/3413473431 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a681024e0 0x7f0a6819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.875+0000 7f0a6d707700 1 --2- 172.21.15.2:0/3413473431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a68101b00 0x7f0a68198650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00e970 tx=0x7f0a5c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:25.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.875+0000 7f0a66ffd700 1 -- 172.21.15.2:0/3413473431 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c009790 con 0x7f0a68101b00 2024-03-15T12:58:25.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.875+0000 7f0a66ffd700 1 -- 172.21.15.2:0/3413473431 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0a5c00c7e0 con 0x7f0a68101b00 2024-03-15T12:58:25.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.875+0000 7f0a7016c700 1 -- 172.21.15.2:0/3413473431 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a681a0250 con 0x7f0a68101b00 2024-03-15T12:58:25.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.875+0000 7f0a66ffd700 1 -- 172.21.15.2:0/3413473431 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c003cc0 con 0x7f0a68101b00 2024-03-15T12:58:25.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.875+0000 7f0a7016c700 1 -- 172.21.15.2:0/3413473431 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a681a0750 con 0x7f0a68101b00 2024-03-15T12:58:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.876+0000 7f0a7016c700 1 -- 172.21.15.2:0/3413473431 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a68068ca0 con 0x7f0a68101b00 2024-03-15T12:58:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.877+0000 7f0a66ffd700 1 -- 172.21.15.2:0/3413473431 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f0a5c00c950 con 0x7f0a68101b00 2024-03-15T12:58:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.878+0000 7f0a66ffd700 1 --2- 172.21.15.2:0/3413473431 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0a500a1ac0 0x7f0a500a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.878+0000 7f0a6df08700 1 --2- 172.21.15.2:0/3413473431 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0a500a1ac0 0x7f0a500a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.878+0000 7f0a66ffd700 1 -- 172.21.15.2:0/3413473431 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f0a5c014070 con 0x7f0a68101b00 2024-03-15T12:58:25.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.878+0000 7f0a6df08700 1 --2- 172.21.15.2:0/3413473431 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0a500a1ac0 0x7f0a500a3f80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0a58007a10 tx=0x7f0a5800b700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:25.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:25.886+0000 7f0a66ffd700 1 -- 172.21.15.2:0/3413473431 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0a5c096920 con 0x7f0a68101b00 2024-03-15T12:58:26.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.090+0000 7f0a7016c700 1 -- 172.21.15.2:0/3413473431 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f0a6810c860 con 0x7f0a68101b00 2024-03-15T12:58:26.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.092+0000 7f0a66ffd700 1 -- 172.21.15.2:0/3413473431 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) v1 ==== 56+0+5542 (secure 0 0 0) 0x7f0a5c099fe0 con 0x7f0a68101b00 2024-03-15T12:58:26.078 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 91 2024-03-15T12:58:26.078 INFO:teuthology.orchestra.run.smithi002.stdout:fsid 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T12:58:26.078 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-15T12:48:20.095922+0000 2024-03-15T12:58:26.078 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-15T12:57:19.826817+0000 2024-03-15T12:58:26.078 INFO:teuthology.orchestra.run.smithi002.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-15T12:58:26.078 INFO:teuthology.orchestra.run.smithi002.stdout:crush_version 27 2024-03-15T12:58:26.078 INFO:teuthology.orchestra.run.smithi002.stdout:full_ratio 0.95 2024-03-15T12:58:26.078 INFO:teuthology.orchestra.run.smithi002.stdout:backfillfull_ratio 0.9 2024-03-15T12:58:26.078 INFO:teuthology.orchestra.run.smithi002.stdout:nearfull_ratio 0.85 2024-03-15T12:58:26.078 INFO:teuthology.orchestra.run.smithi002.stdout:require_min_compat_client luminous 2024-03-15T12:58:26.079 INFO:teuthology.orchestra.run.smithi002.stdout:min_compat_client luminous 2024-03-15T12:58:26.079 INFO:teuthology.orchestra.run.smithi002.stdout:require_osd_release squid 2024-03-15T12:58:26.079 INFO:teuthology.orchestra.run.smithi002.stdout:stretch_mode_enabled false 2024-03-15T12:58:26.079 INFO:teuthology.orchestra.run.smithi002.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-15T12:58:26.079 INFO:teuthology.orchestra.run.smithi002.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-15T12:58:26.079 INFO:teuthology.orchestra.run.smithi002.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-15T12:58:26.079 INFO:teuthology.orchestra.run.smithi002.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-15T12:58:26.079 INFO:teuthology.orchestra.run.smithi002.stdout:max_osd 12 2024-03-15T12:58:26.079 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6802/2123099355,v1:172.21.15.2:6803/2123099355] [v2:172.21.15.2:6804/2123099355,v1:172.21.15.2:6805/2123099355] exists,up f9c5bb62-482c-45e1-b63b-b487ca609054 2024-03-15T12:58:26.079 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6800/3295500987,v1:172.21.15.149:6801/3295500987] [v2:172.21.15.149:6802/3295500987,v1:172.21.15.149:6803/3295500987] exists,up 5182e55d-f2cf-40be-acff-0534c2bce7ac 2024-03-15T12:58:26.079 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6800/1645410344,v1:172.21.15.184:6801/1645410344] [v2:172.21.15.184:6802/1645410344,v1:172.21.15.184:6803/1645410344] exists,up 8db2a1c3-c04d-4b78-ad1c-e587fad5ce88 2024-03-15T12:58:26.079 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6810/12565302,v1:172.21.15.2:6811/12565302] [v2:172.21.15.2:6812/12565302,v1:172.21.15.2:6813/12565302] exists,up b2438d20-597b-4510-8cb7-3f0ce89067a6 2024-03-15T12:58:26.079 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 up in weight 1 up_from 33 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6808/1790191535,v1:172.21.15.149:6809/1790191535] [v2:172.21.15.149:6810/1790191535,v1:172.21.15.149:6811/1790191535] exists,up 65f6e3fc-1a8e-4180-9f3f-7a81b98de439 2024-03-15T12:58:26.080 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6808/1659458624,v1:172.21.15.184:6809/1659458624] [v2:172.21.15.184:6810/1659458624,v1:172.21.15.184:6811/1659458624] exists,up e43c485c-25d6-41d5-8b9b-e22e4cc46727 2024-03-15T12:58:26.080 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6 up in weight 1 up_from 46 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6818/4103461638,v1:172.21.15.2:6819/4103461638] [v2:172.21.15.2:6820/4103461638,v1:172.21.15.2:6821/4103461638] exists,up 0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27 2024-03-15T12:58:26.080 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 up in weight 1 up_from 51 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6816/4241660371,v1:172.21.15.149:6817/4241660371] [v2:172.21.15.149:6818/4241660371,v1:172.21.15.149:6819/4241660371] exists,up 997ac305-e8f7-4ae6-b9ea-a491dfad807a 2024-03-15T12:58:26.080 INFO:teuthology.orchestra.run.smithi002.stdout:osd.8 up in weight 1 up_from 59 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6816/2462722579,v1:172.21.15.184:6817/2462722579] [v2:172.21.15.184:6818/2462722579,v1:172.21.15.184:6819/2462722579] exists,up 99a26e37-2f65-4775-80d0-e8388462928f 2024-03-15T12:58:26.080 INFO:teuthology.orchestra.run.smithi002.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6826/49148413,v1:172.21.15.2:6827/49148413] [v2:172.21.15.2:6828/49148413,v1:172.21.15.2:6829/49148413] exists,up 8b52d0d0-43fa-41ac-a775-722ff5480e45 2024-03-15T12:58:26.080 INFO:teuthology.orchestra.run.smithi002.stdout:osd.10 up in weight 1 up_from 72 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6824/432128202,v1:172.21.15.149:6825/432128202] [v2:172.21.15.149:6826/432128202,v1:172.21.15.149:6827/432128202] exists,up 792088a5-75ff-44b2-b1b0-09a7a0b84554 2024-03-15T12:58:26.080 INFO:teuthology.orchestra.run.smithi002.stdout:osd.11 up in weight 1 up_from 80 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6824/1184158252,v1:172.21.15.184:6825/1184158252] [v2:172.21.15.184:6826/1184158252,v1:172.21.15.184:6827/1184158252] exists,up 5a9260e1-292e-4694-863b-b6710b18feb0 2024-03-15T12:58:26.080 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.4 [2,11] 2024-03-15T12:58:26.080 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.5 [8,11] 2024-03-15T12:58:26.080 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.1 [4,9] 2024-03-15T12:58:26.080 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.1d [5,9] 2024-03-15T12:58:26.080 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.34 [4,9] 2024-03-15T12:58:26.080 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.f [9,8] 2024-03-15T12:58:26.080 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.14 [9,8] 2024-03-15T12:58:26.080 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.15 [5,10] 2024-03-15T12:58:26.081 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 4.1f [9,10] 2024-03-15T12:58:26.081 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/3349070582 expires 2024-03-16T12:49:18.658825+0000 2024-03-15T12:58:26.081 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6801/3269509343 expires 2024-03-16T12:49:18.658825+0000 2024-03-15T12:58:26.081 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6800/3269509343 expires 2024-03-16T12:49:18.658825+0000 2024-03-15T12:58:26.081 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/656434628 expires 2024-03-16T12:49:18.658825+0000 2024-03-15T12:58:26.081 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/84615744 expires 2024-03-16T12:48:51.791072+0000 2024-03-15T12:58:26.081 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/1464336909 expires 2024-03-16T12:48:51.791072+0000 2024-03-15T12:58:26.081 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/3546094985 expires 2024-03-16T12:48:51.791072+0000 2024-03-15T12:58:26.081 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6800/3111414481 expires 2024-03-16T12:48:51.791072+0000 2024-03-15T12:58:26.081 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6801/3111414481 expires 2024-03-16T12:48:51.791072+0000 2024-03-15T12:58:26.081 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/3673716509 expires 2024-03-16T12:49:18.658825+0000 2024-03-15T12:58:26.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.094+0000 7f0a7016c700 1 -- 172.21.15.2:0/3413473431 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0a500a1ac0 msgr2=0x7f0a500a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:26.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.094+0000 7f0a7016c700 1 --2- 172.21.15.2:0/3413473431 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0a500a1ac0 0x7f0a500a3f80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0a58007a10 tx=0x7f0a5800b700 comp rx=0 tx=0).stop 2024-03-15T12:58:26.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.094+0000 7f0a7016c700 1 -- 172.21.15.2:0/3413473431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a68101b00 msgr2=0x7f0a68198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:26.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.094+0000 7f0a7016c700 1 --2- 172.21.15.2:0/3413473431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a68101b00 0x7f0a68198650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00e970 tx=0x7f0a5c00ec80 comp rx=0 tx=0).stop 2024-03-15T12:58:26.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.094+0000 7f0a7016c700 1 -- 172.21.15.2:0/3413473431 shutdown_connections 2024-03-15T12:58:26.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.094+0000 7f0a7016c700 1 --2- 172.21.15.2:0/3413473431 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f0a500a1ac0 0x7f0a500a3f80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:26.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.094+0000 7f0a7016c700 1 --2- 172.21.15.2:0/3413473431 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a681024e0 0x7f0a6819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:26.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.094+0000 7f0a7016c700 1 --2- 172.21.15.2:0/3413473431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a68101140 0x7f0a68198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:26.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.094+0000 7f0a7016c700 1 --2- 172.21.15.2:0/3413473431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a68101b00 0x7f0a68198650 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:26.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.094+0000 7f0a7016c700 1 -- 172.21.15.2:0/3413473431 >> 172.21.15.2:0/3413473431 conn(0x7f0a680fbc50 msgr2=0x7f0a680fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:26.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.094+0000 7f0a7016c700 1 -- 172.21.15.2:0/3413473431 shutdown_connections 2024-03-15T12:58:26.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.094+0000 7f0a7016c700 1 -- 172.21.15.2:0/3413473431 wait complete. 2024-03-15T12:58:26.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:26 smithi002 ceph-mon[32060]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-15T12:58:26.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:26 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3413473431' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-15T12:58:26.781 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-15T12:58:26.793 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:58:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 -- 172.21.15.2:0/3718387041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2758101ff0 msgr2=0x7f27581043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 --2- 172.21.15.2:0/3718387041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2758101ff0 0x7f27581043e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2750009900 tx=0x7f275002bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 -- 172.21.15.2:0/3718387041 shutdown_connections 2024-03-15T12:58:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 --2- 172.21.15.2:0/3718387041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2758107250 0x7f2758109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 --2- 172.21.15.2:0/3718387041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2758104920 0x7f2758106d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 --2- 172.21.15.2:0/3718387041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2758101ff0 0x7f27581043e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 -- 172.21.15.2:0/3718387041 >> 172.21.15.2:0/3718387041 conn(0x7f27580fb980 msgr2=0x7f27580fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 -- 172.21.15.2:0/3718387041 shutdown_connections 2024-03-15T12:58:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 -- 172.21.15.2:0/3718387041 wait complete. 2024-03-15T12:58:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 Processor -- start 2024-03-15T12:58:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 -- start start 2024-03-15T12:58:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2758101ff0 0x7f27581975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2758104920 0x7f2758197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2758107250 0x7f275819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275819f0e0 con 0x7f2758107250 2024-03-15T12:58:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275819f250 con 0x7f2758101ff0 2024-03-15T12:58:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.916+0000 7f275ef7a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275819f410 con 0x7f2758104920 2024-03-15T12:58:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f275d517640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2758107250 0x7f275819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f275d517640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2758107250 0x7f275819eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33354/0 (socket says 172.21.15.2:33354) 2024-03-15T12:58:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f275cd16640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2758101ff0 0x7f27581975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f275d517640 1 -- 172.21.15.2:0/200526363 learned_addr learned my addr 172.21.15.2:0/200526363 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f2757fff640 1 --2- 172.21.15.2:0/200526363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2758104920 0x7f2758197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f275d517640 1 -- 172.21.15.2:0/200526363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2758104920 msgr2=0x7f2758197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f275d517640 1 --2- 172.21.15.2:0/200526363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2758104920 0x7f2758197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f275d517640 1 -- 172.21.15.2:0/200526363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2758101ff0 msgr2=0x7f27581975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f275d517640 1 --2- 172.21.15.2:0/200526363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2758101ff0 0x7f27581975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f275d517640 1 -- 172.21.15.2:0/200526363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2750009560 con 0x7f2758107250 2024-03-15T12:58:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f275d517640 1 --2- 172.21.15.2:0/200526363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2758107250 0x7f275819eba0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f274000e970 tx=0x7f274000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f2755ffb640 1 -- 172.21.15.2:0/200526363 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740009720 con 0x7f2758107250 2024-03-15T12:58:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f275ef7a640 1 -- 172.21.15.2:0/200526363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275819f6f0 con 0x7f2758107250 2024-03-15T12:58:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f2755ffb640 1 -- 172.21.15.2:0/200526363 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f27400054f0 con 0x7f2758107250 2024-03-15T12:58:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f2755ffb640 1 -- 172.21.15.2:0/200526363 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740010430 con 0x7f2758107250 2024-03-15T12:58:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.917+0000 7f275ef7a640 1 -- 172.21.15.2:0/200526363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275819fc00 con 0x7f2758107250 2024-03-15T12:58:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.918+0000 7f275ef7a640 1 -- 172.21.15.2:0/200526363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27580fd470 con 0x7f2758107250 2024-03-15T12:58:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.919+0000 7f2755ffb640 1 -- 172.21.15.2:0/200526363 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f2740004fd0 con 0x7f2758107250 2024-03-15T12:58:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.920+0000 7f2755ffb640 1 --2- 172.21.15.2:0/200526363 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f27440a1a30 0x7f27440a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.920+0000 7f275cd16640 1 --2- 172.21.15.2:0/200526363 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f27440a1a30 0x7f27440a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.920+0000 7f2755ffb640 1 -- 172.21.15.2:0/200526363 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f2740014070 con 0x7f2758107250 2024-03-15T12:58:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.920+0000 7f275cd16640 1 --2- 172.21.15.2:0/200526363 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f27440a1a30 0x7f27440a3ef0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2750000c00 tx=0x7f2750005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:26.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:26.924+0000 7f2755ffb640 1 -- 172.21.15.2:0/200526363 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2740096a50 con 0x7f2758107250 2024-03-15T12:58:27.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:26 smithi184 ceph-mon[35993]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-15T12:58:27.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:26 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3413473431' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-15T12:58:27.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.107+0000 7f275ef7a640 1 -- 172.21.15.2:0/200526363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f275802cf60 con 0x7f2758107250 2024-03-15T12:58:27.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.109+0000 7f2755ffb640 1 -- 172.21.15.2:0/200526363 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24526 (secure 0 0 0) 0x7f274009a110 con 0x7f2758107250 2024-03-15T12:58:27.095 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:27.095 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":91,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:57:19.826817+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:58:27.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.111+0000 7f275ef7a640 1 -- 172.21.15.2:0/200526363 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f27440a1a30 msgr2=0x7f27440a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.111+0000 7f275ef7a640 1 --2- 172.21.15.2:0/200526363 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f27440a1a30 0x7f27440a3ef0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2750000c00 tx=0x7f2750005fd0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.111+0000 7f275ef7a640 1 -- 172.21.15.2:0/200526363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2758107250 msgr2=0x7f275819eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.111+0000 7f275ef7a640 1 --2- 172.21.15.2:0/200526363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2758107250 0x7f275819eba0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f274000e970 tx=0x7f274000ed70 comp rx=0 tx=0).stop 2024-03-15T12:58:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.111+0000 7f275ef7a640 1 -- 172.21.15.2:0/200526363 shutdown_connections 2024-03-15T12:58:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.111+0000 7f275ef7a640 1 --2- 172.21.15.2:0/200526363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2758107250 0x7f275819eba0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.111+0000 7f275ef7a640 1 --2- 172.21.15.2:0/200526363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2758104920 0x7f2758197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.111+0000 7f275ef7a640 1 --2- 172.21.15.2:0/200526363 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f27440a1a30 0x7f27440a3ef0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.111+0000 7f275ef7a640 1 --2- 172.21.15.2:0/200526363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2758101ff0 0x7f27581975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.111+0000 7f275ef7a640 1 -- 172.21.15.2:0/200526363 >> 172.21.15.2:0/200526363 conn(0x7f27580fb980 msgr2=0x7f2758102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.111+0000 7f275ef7a640 1 -- 172.21.15.2:0/200526363 shutdown_connections 2024-03-15T12:58:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.111+0000 7f275ef7a640 1 -- 172.21.15.2:0/200526363 wait complete. 2024-03-15T12:58:27.109 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-15T12:51:47.004843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-15T12:56:18.559882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-15T12:56:19.751767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-15T12:56:23.664879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-15T12:58:27.109 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-15T12:58:27.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:26 smithi149 ceph-mon[35927]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-15T12:58:27.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:26 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3413473431' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-15T12:58:27.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.229+0000 7f84b3b56640 1 -- 172.21.15.2:0/3061425994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac1003f0 msgr2=0x7f84ac100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:27.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.229+0000 7f84b3b56640 1 --2- 172.21.15.2:0/3061425994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac1003f0 0x7f84ac100810 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009ca0 tx=0x7f84a402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.229+0000 7f84b3b56640 1 -- 172.21.15.2:0/3061425994 shutdown_connections 2024-03-15T12:58:27.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.229+0000 7f84b3b56640 1 --2- 172.21.15.2:0/3061425994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84ac101fa0 0x7f84ac108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b3b56640 1 --2- 172.21.15.2:0/3061425994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84ac1015e0 0x7f84ac101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b3b56640 1 --2- 172.21.15.2:0/3061425994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac1003f0 0x7f84ac100810 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b3b56640 1 -- 172.21.15.2:0/3061425994 >> 172.21.15.2:0/3061425994 conn(0x7f84ac0fb980 msgr2=0x7f84ac0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:27.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b3b56640 1 -- 172.21.15.2:0/3061425994 shutdown_connections 2024-03-15T12:58:27.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b3b56640 1 -- 172.21.15.2:0/3061425994 wait complete. 2024-03-15T12:58:27.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b3b56640 1 Processor -- start 2024-03-15T12:58:27.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b3b56640 1 -- start start 2024-03-15T12:58:27.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b3b56640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac1003f0 0x7f84ac1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:27.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b3b56640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84ac1015e0 0x7f84ac197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:27.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b3b56640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84ac101fa0 0x7f84ac19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:27.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b3b56640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac19f0e0 con 0x7f84ac1003f0 2024-03-15T12:58:27.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b3b56640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac19f250 con 0x7f84ac1015e0 2024-03-15T12:58:27.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b3b56640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac19f410 con 0x7f84ac101fa0 2024-03-15T12:58:27.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b18f2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac1003f0 0x7f84ac1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:27.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b18f2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac1003f0 0x7f84ac1975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33378/0 (socket says 172.21.15.2:33378) 2024-03-15T12:58:27.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b18f2640 1 -- 172.21.15.2:0/560410171 learned_addr learned my addr 172.21.15.2:0/560410171 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:27.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.230+0000 7f84b10f1640 1 --2- 172.21.15.2:0/560410171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84ac1015e0 0x7f84ac197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:27.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.231+0000 7f84b20f3640 1 --2- 172.21.15.2:0/560410171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84ac101fa0 0x7f84ac19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:27.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.231+0000 7f84b18f2640 1 -- 172.21.15.2:0/560410171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84ac101fa0 msgr2=0x7f84ac19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:27.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.231+0000 7f84b18f2640 1 --2- 172.21.15.2:0/560410171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84ac101fa0 0x7f84ac19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.231+0000 7f84b18f2640 1 -- 172.21.15.2:0/560410171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84ac1015e0 msgr2=0x7f84ac197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:27.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.231+0000 7f84b18f2640 1 --2- 172.21.15.2:0/560410171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84ac1015e0 0x7f84ac197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.231+0000 7f84b18f2640 1 -- 172.21.15.2:0/560410171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a4009900 con 0x7f84ac1003f0 2024-03-15T12:58:27.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.231+0000 7f84b18f2640 1 --2- 172.21.15.2:0/560410171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac1003f0 0x7f84ac1975f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f84a4005e30 tx=0x7f84a4032990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:27.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.231+0000 7f84aaffd640 1 -- 172.21.15.2:0/560410171 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4003970 con 0x7f84ac1003f0 2024-03-15T12:58:27.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.231+0000 7f84aaffd640 1 -- 172.21.15.2:0/560410171 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f84a4004040 con 0x7f84ac1003f0 2024-03-15T12:58:27.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.231+0000 7f84b3b56640 1 -- 172.21.15.2:0/560410171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84ac19f690 con 0x7f84ac1003f0 2024-03-15T12:58:27.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.231+0000 7f84aaffd640 1 -- 172.21.15.2:0/560410171 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a40406b0 con 0x7f84ac1003f0 2024-03-15T12:58:27.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.231+0000 7f84b3b56640 1 -- 172.21.15.2:0/560410171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ac19fb40 con 0x7f84ac1003f0 2024-03-15T12:58:27.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.232+0000 7f84b3b56640 1 -- 172.21.15.2:0/560410171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84ac10bdd0 con 0x7f84ac1003f0 2024-03-15T12:58:27.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.233+0000 7f84aaffd640 1 -- 172.21.15.2:0/560410171 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f84a400a040 con 0x7f84ac1003f0 2024-03-15T12:58:27.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.234+0000 7f84aaffd640 1 --2- 172.21.15.2:0/560410171 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f84940a1a30 0x7f84940a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:27.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.234+0000 7f84b10f1640 1 --2- 172.21.15.2:0/560410171 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f84940a1a30 0x7f84940a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:27.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.234+0000 7f84aaffd640 1 -- 172.21.15.2:0/560410171 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f84a40f0e00 con 0x7f84ac1003f0 2024-03-15T12:58:27.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.234+0000 7f84b10f1640 1 --2- 172.21.15.2:0/560410171 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f84940a1a30 0x7f84940a3ef0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f84ac198b00 tx=0x7f84a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:27.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.237+0000 7f84aaffd640 1 -- 172.21.15.2:0/560410171 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f84a40b9c70 con 0x7f84ac1003f0 2024-03-15T12:58:27.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.417+0000 7f84b3b56640 1 -- 172.21.15.2:0/560410171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84ac19ffe0 con 0x7f84ac1003f0 2024-03-15T12:58:27.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.418+0000 7f84aaffd640 1 -- 172.21.15.2:0/560410171 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f84a40bd330 con 0x7f84ac1003f0 2024-03-15T12:58:27.404 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-15T12:58:27.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.419+0000 7f84b3b56640 1 -- 172.21.15.2:0/560410171 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f84940a1a30 msgr2=0x7f84940a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:27.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.420+0000 7f84b3b56640 1 --2- 172.21.15.2:0/560410171 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f84940a1a30 0x7f84940a3ef0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f84ac198b00 tx=0x7f84a0005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:27.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.420+0000 7f84b3b56640 1 -- 172.21.15.2:0/560410171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac1003f0 msgr2=0x7f84ac1975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:27.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.420+0000 7f84b3b56640 1 --2- 172.21.15.2:0/560410171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac1003f0 0x7f84ac1975f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f84a4005e30 tx=0x7f84a4032990 comp rx=0 tx=0).stop 2024-03-15T12:58:27.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.420+0000 7f84b3b56640 1 -- 172.21.15.2:0/560410171 shutdown_connections 2024-03-15T12:58:27.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.420+0000 7f84b3b56640 1 --2- 172.21.15.2:0/560410171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84ac101fa0 0x7f84ac19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.420+0000 7f84b3b56640 1 --2- 172.21.15.2:0/560410171 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f84940a1a30 0x7f84940a3ef0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.420+0000 7f84b3b56640 1 --2- 172.21.15.2:0/560410171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84ac1015e0 0x7f84ac197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.420+0000 7f84b3b56640 1 --2- 172.21.15.2:0/560410171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac1003f0 0x7f84ac1975f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.420+0000 7f84b3b56640 1 -- 172.21.15.2:0/560410171 >> 172.21.15.2:0/560410171 conn(0x7f84ac0fb980 msgr2=0x7f84ac068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:27.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.420+0000 7f84b3b56640 1 -- 172.21.15.2:0/560410171 shutdown_connections 2024-03-15T12:58:27.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.420+0000 7f84b3b56640 1 -- 172.21.15.2:0/560410171 wait complete. 2024-03-15T12:58:27.416 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-15T12:58:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.536+0000 7f2018b81640 1 -- 172.21.15.2:0/2270233031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20140ffb90 msgr2=0x7f20140fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.536+0000 7f2018b81640 1 --2- 172.21.15.2:0/2270233031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20140ffb90 0x7f20140fffb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2004009900 tx=0x7f200402bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.536+0000 7f2018b81640 1 -- 172.21.15.2:0/2270233031 shutdown_connections 2024-03-15T12:58:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.536+0000 7f2018b81640 1 --2- 172.21.15.2:0/2270233031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2014101740 0x7f2014107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.536+0000 7f2018b81640 1 --2- 172.21.15.2:0/2270233031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2014100d80 0x7f2014101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.536+0000 7f2018b81640 1 --2- 172.21.15.2:0/2270233031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20140ffb90 0x7f20140fffb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.536+0000 7f2018b81640 1 -- 172.21.15.2:0/2270233031 >> 172.21.15.2:0/2270233031 conn(0x7f20140fb100 msgr2=0x7f20140fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.536+0000 7f2018b81640 1 -- 172.21.15.2:0/2270233031 shutdown_connections 2024-03-15T12:58:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.536+0000 7f2018b81640 1 -- 172.21.15.2:0/2270233031 wait complete. 2024-03-15T12:58:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2018b81640 1 Processor -- start 2024-03-15T12:58:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2018b81640 1 -- start start 2024-03-15T12:58:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2018b81640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20140ffb90 0x7f2014197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2018b81640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2014100d80 0x7f2014197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2018b81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2014101740 0x7f201419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2018b81640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201419f0f0 con 0x7f2014101740 2024-03-15T12:58:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2018b81640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201419f260 con 0x7f2014100d80 2024-03-15T12:58:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2018b81640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201419f420 con 0x7f20140ffb90 2024-03-15T12:58:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2012d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2014101740 0x7f201419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2012d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2014101740 0x7f201419ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33400/0 (socket says 172.21.15.2:33400) 2024-03-15T12:58:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f201259c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20140ffb90 0x7f2014197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2012d9d640 1 -- 172.21.15.2:0/720161253 learned_addr learned my addr 172.21.15.2:0/720161253 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f200bfff640 1 --2- 172.21.15.2:0/720161253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2014100d80 0x7f2014197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2012d9d640 1 -- 172.21.15.2:0/720161253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20140ffb90 msgr2=0x7f2014197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2012d9d640 1 --2- 172.21.15.2:0/720161253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20140ffb90 0x7f2014197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2012d9d640 1 -- 172.21.15.2:0/720161253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2014100d80 msgr2=0x7f2014197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2012d9d640 1 --2- 172.21.15.2:0/720161253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2014100d80 0x7f2014197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2012d9d640 1 -- 172.21.15.2:0/720161253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2004009560 con 0x7f2014101740 2024-03-15T12:58:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f201259c640 1 --2- 172.21.15.2:0/720161253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20140ffb90 0x7f2014197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f2012d9d640 1 --2- 172.21.15.2:0/720161253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2014101740 0x7f201419ebb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f200c00eb10 tx=0x7f200c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.537+0000 7f200b7fe640 1 -- 172.21.15.2:0/720161253 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200c004b20 con 0x7f2014101740 2024-03-15T12:58:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.538+0000 7f200b7fe640 1 -- 172.21.15.2:0/720161253 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f200c004c80 con 0x7f2014101740 2024-03-15T12:58:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.538+0000 7f2018b81640 1 -- 172.21.15.2:0/720161253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f201419f700 con 0x7f2014101740 2024-03-15T12:58:27.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.538+0000 7f200b7fe640 1 -- 172.21.15.2:0/720161253 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200c010690 con 0x7f2014101740 2024-03-15T12:58:27.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.538+0000 7f2018b81640 1 -- 172.21.15.2:0/720161253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f201419fbc0 con 0x7f2014101740 2024-03-15T12:58:27.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.539+0000 7f2018b81640 1 -- 172.21.15.2:0/720161253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f201410be60 con 0x7f2014101740 2024-03-15T12:58:27.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.540+0000 7f200b7fe640 1 -- 172.21.15.2:0/720161253 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f200c004df0 con 0x7f2014101740 2024-03-15T12:58:27.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.540+0000 7f200b7fe640 1 --2- 172.21.15.2:0/720161253 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20000a1a30 0x7f20000a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:27.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.540+0000 7f200b7fe640 1 -- 172.21.15.2:0/720161253 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f200c014070 con 0x7f2014101740 2024-03-15T12:58:27.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.540+0000 7f201259c640 1 --2- 172.21.15.2:0/720161253 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20000a1a30 0x7f20000a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:27.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.541+0000 7f201259c640 1 --2- 172.21.15.2:0/720161253 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20000a1a30 0x7f20000a3ef0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2004009230 tx=0x7f2004000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:27.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.545+0000 7f200b7fe640 1 -- 172.21.15.2:0/720161253 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f200c096bd0 con 0x7f2014101740 2024-03-15T12:58:27.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.729+0000 7f2018b81640 1 -- 172.21.15.2:0/720161253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20141a00c0 con 0x7f2014101740 2024-03-15T12:58:27.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.729+0000 7f200b7fe640 1 -- 172.21.15.2:0/720161253 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f200c09a290 con 0x7f2014101740 2024-03-15T12:58:27.715 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:58:27.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.731+0000 7f2018b81640 1 -- 172.21.15.2:0/720161253 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20000a1a30 msgr2=0x7f20000a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:27.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.731+0000 7f2018b81640 1 --2- 172.21.15.2:0/720161253 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20000a1a30 0x7f20000a3ef0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2004009230 tx=0x7f2004000bc0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.731+0000 7f2018b81640 1 -- 172.21.15.2:0/720161253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2014101740 msgr2=0x7f201419ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:27.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.731+0000 7f2018b81640 1 --2- 172.21.15.2:0/720161253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2014101740 0x7f201419ebb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f200c00eb10 tx=0x7f200c00c2d0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.731+0000 7f2018b81640 1 -- 172.21.15.2:0/720161253 shutdown_connections 2024-03-15T12:58:27.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.731+0000 7f2018b81640 1 --2- 172.21.15.2:0/720161253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2014101740 0x7f201419ebb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.731+0000 7f2018b81640 1 --2- 172.21.15.2:0/720161253 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f20000a1a30 0x7f20000a3ef0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.731+0000 7f2018b81640 1 --2- 172.21.15.2:0/720161253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2014100d80 0x7f2014197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.731+0000 7f2018b81640 1 --2- 172.21.15.2:0/720161253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20140ffb90 0x7f2014197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.731+0000 7f2018b81640 1 -- 172.21.15.2:0/720161253 >> 172.21.15.2:0/720161253 conn(0x7f20140fb100 msgr2=0x7f2014108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:27.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.732+0000 7f2018b81640 1 -- 172.21.15.2:0/720161253 shutdown_connections 2024-03-15T12:58:27.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.732+0000 7f2018b81640 1 -- 172.21.15.2:0/720161253 wait complete. 2024-03-15T12:58:27.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:27 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/200526363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:27.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:27 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/560410171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:27.729 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-15T12:58:27.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.848+0000 7f078f10d640 1 -- 172.21.15.2:0/1967309653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0788101fa0 msgr2=0x7f0788108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:27.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.848+0000 7f078f10d640 1 --2- 172.21.15.2:0/1967309653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0788101fa0 0x7f0788108830 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0778009900 tx=0x7f077802bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:27.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.848+0000 7f078f10d640 1 -- 172.21.15.2:0/1967309653 shutdown_connections 2024-03-15T12:58:27.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.848+0000 7f078f10d640 1 --2- 172.21.15.2:0/1967309653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0788101fa0 0x7f0788108830 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.848+0000 7f078f10d640 1 --2- 172.21.15.2:0/1967309653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07881015e0 0x7f0788101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.848+0000 7f078f10d640 1 --2- 172.21.15.2:0/1967309653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07881003f0 0x7f0788100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.848+0000 7f078f10d640 1 -- 172.21.15.2:0/1967309653 >> 172.21.15.2:0/1967309653 conn(0x7f07880fb980 msgr2=0x7f07880fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.848+0000 7f078f10d640 1 -- 172.21.15.2:0/1967309653 shutdown_connections 2024-03-15T12:58:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.848+0000 7f078f10d640 1 -- 172.21.15.2:0/1967309653 wait complete. 2024-03-15T12:58:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078f10d640 1 Processor -- start 2024-03-15T12:58:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078f10d640 1 -- start start 2024-03-15T12:58:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078f10d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07881003f0 0x7f0788104310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078f10d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07881015e0 0x7f07881029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078f10d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0788101fa0 0x7f0788102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078f10d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07881035f0 con 0x7f07881003f0 2024-03-15T12:58:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078f10d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0788103760 con 0x7f07881015e0 2024-03-15T12:58:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078f10d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07881a1680 con 0x7f0788101fa0 2024-03-15T12:58:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078cea9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07881003f0 0x7f0788104310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078cea9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07881003f0 0x7f0788104310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33418/0 (socket says 172.21.15.2:33418) 2024-03-15T12:58:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078cea9640 1 -- 172.21.15.2:0/2790154771 learned_addr learned my addr 172.21.15.2:0/2790154771 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:27.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f0787fff640 1 --2- 172.21.15.2:0/2790154771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07881015e0 0x7f07881029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:27.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078d6aa640 1 --2- 172.21.15.2:0/2790154771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0788101fa0 0x7f0788102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:27.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078cea9640 1 -- 172.21.15.2:0/2790154771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0788101fa0 msgr2=0x7f0788102f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:27.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078cea9640 1 --2- 172.21.15.2:0/2790154771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0788101fa0 0x7f0788102f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078cea9640 1 -- 172.21.15.2:0/2790154771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07881015e0 msgr2=0x7f07881029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:27.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078cea9640 1 --2- 172.21.15.2:0/2790154771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07881015e0 0x7f07881029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:27.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.849+0000 7f078cea9640 1 -- 172.21.15.2:0/2790154771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0778009560 con 0x7f07881003f0 2024-03-15T12:58:27.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.850+0000 7f078cea9640 1 --2- 172.21.15.2:0/2790154771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07881003f0 0x7f0788104310 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f078000ebc0 tx=0x7f078000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:27.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.850+0000 7f0785ffb640 1 -- 172.21.15.2:0/2790154771 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780003b70 con 0x7f07881003f0 2024-03-15T12:58:27.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.850+0000 7f078f10d640 1 -- 172.21.15.2:0/2790154771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07881a18e0 con 0x7f07881003f0 2024-03-15T12:58:27.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.850+0000 7f0785ffb640 1 -- 172.21.15.2:0/2790154771 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f078000eec0 con 0x7f07881003f0 2024-03-15T12:58:27.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.850+0000 7f0785ffb640 1 -- 172.21.15.2:0/2790154771 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780019840 con 0x7f07881003f0 2024-03-15T12:58:27.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.850+0000 7f078f10d640 1 -- 172.21.15.2:0/2790154771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07881a1dc0 con 0x7f07881003f0 2024-03-15T12:58:27.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.851+0000 7f078f10d640 1 -- 172.21.15.2:0/2790154771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078810bdd0 con 0x7f07881003f0 2024-03-15T12:58:27.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.852+0000 7f0785ffb640 1 -- 172.21.15.2:0/2790154771 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f0780004030 con 0x7f07881003f0 2024-03-15T12:58:27.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.852+0000 7f0785ffb640 1 --2- 172.21.15.2:0/2790154771 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f07700a1a30 0x7f07700a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:27.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.853+0000 7f0787fff640 1 --2- 172.21.15.2:0/2790154771 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f07700a1a30 0x7f07700a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:27.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.853+0000 7f0785ffb640 1 -- 172.21.15.2:0/2790154771 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f0780014070 con 0x7f07881003f0 2024-03-15T12:58:27.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.853+0000 7f0787fff640 1 --2- 172.21.15.2:0/2790154771 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f07700a1a30 0x7f07700a3ef0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0788104080 tx=0x7f077c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:27.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:27.857+0000 7f0785ffb640 1 -- 172.21.15.2:0/2790154771 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0780096e10 con 0x7f07881003f0 2024-03-15T12:58:28.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.041+0000 7f078f10d640 1 -- 172.21.15.2:0/2790154771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f078810bf60 con 0x7f07881003f0 2024-03-15T12:58:28.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.041+0000 7f0785ffb640 1 -- 172.21.15.2:0/2790154771 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f078009a4d0 con 0x7f07881003f0 2024-03-15T12:58:28.027 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-03-15T12:58:28.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.044+0000 7f078f10d640 1 -- 172.21.15.2:0/2790154771 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f07700a1a30 msgr2=0x7f07700a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:28.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.044+0000 7f078f10d640 1 --2- 172.21.15.2:0/2790154771 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f07700a1a30 0x7f07700a3ef0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0788104080 tx=0x7f077c005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:28.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.044+0000 7f078f10d640 1 -- 172.21.15.2:0/2790154771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07881003f0 msgr2=0x7f0788104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:28.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.044+0000 7f078f10d640 1 --2- 172.21.15.2:0/2790154771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07881003f0 0x7f0788104310 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f078000ebc0 tx=0x7f078000c6a0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.044+0000 7f078f10d640 1 -- 172.21.15.2:0/2790154771 shutdown_connections 2024-03-15T12:58:28.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.044+0000 7f078f10d640 1 --2- 172.21.15.2:0/2790154771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0788101fa0 0x7f0788102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.044+0000 7f078f10d640 1 --2- 172.21.15.2:0/2790154771 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f07700a1a30 0x7f07700a3ef0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.044+0000 7f078f10d640 1 --2- 172.21.15.2:0/2790154771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07881015e0 0x7f07881029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.044+0000 7f078f10d640 1 --2- 172.21.15.2:0/2790154771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07881003f0 0x7f0788104310 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.044+0000 7f078f10d640 1 -- 172.21.15.2:0/2790154771 >> 172.21.15.2:0/2790154771 conn(0x7f07880fb980 msgr2=0x7f07880fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:28.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.044+0000 7f078f10d640 1 -- 172.21.15.2:0/2790154771 shutdown_connections 2024-03-15T12:58:28.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.044+0000 7f078f10d640 1 -- 172.21.15.2:0/2790154771 wait complete. 2024-03-15T12:58:28.041 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-15T12:58:28.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:27 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/200526363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:28.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:27 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/560410171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:28.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:27 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/200526363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:28.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:27 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/560410171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:28.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.161+0000 7f5c0c815640 1 -- 172.21.15.2:0/570639447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c081006d0 msgr2=0x7f5c08100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:28.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.161+0000 7f5c0c815640 1 --2- 172.21.15.2:0/570639447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c081006d0 0x7f5c08100b50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009930 tx=0x7f5bf002bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:28.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.161+0000 7f5c0c815640 1 -- 172.21.15.2:0/570639447 shutdown_connections 2024-03-15T12:58:28.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.161+0000 7f5c0c815640 1 --2- 172.21.15.2:0/570639447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c08101090 0x7f5c08108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.161+0000 7f5c0c815640 1 --2- 172.21.15.2:0/570639447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c081006d0 0x7f5c08100b50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.161+0000 7f5c0c815640 1 --2- 172.21.15.2:0/570639447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c081019a0 0x7f5c08101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.161+0000 7f5c0c815640 1 -- 172.21.15.2:0/570639447 >> 172.21.15.2:0/570639447 conn(0x7f5c080fb120 msgr2=0x7f5c080fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:28.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.161+0000 7f5c0c815640 1 -- 172.21.15.2:0/570639447 shutdown_connections 2024-03-15T12:58:28.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.161+0000 7f5c0c815640 1 -- 172.21.15.2:0/570639447 wait complete. 2024-03-15T12:58:28.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.161+0000 7f5c0c815640 1 Processor -- start 2024-03-15T12:58:28.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0c815640 1 -- start start 2024-03-15T12:58:28.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0c815640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c081006d0 0x7f5c08197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:28.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0c815640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08101090 0x7f5c08197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:28.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0c815640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c081019a0 0x7f5c0819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:28.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0c815640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0819f100 con 0x7f5c08101090 2024-03-15T12:58:28.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0c815640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0819f270 con 0x7f5c081006d0 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0c815640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0819f430 con 0x7f5c081019a0 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08101090 0x7f5c08197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08101090 0x7f5c08197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50090/0 (socket says 172.21.15.2:50090) 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0659c640 1 -- 172.21.15.2:0/922567526 learned_addr learned my addr 172.21.15.2:0/922567526 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5bfe59c640 1 --2- 172.21.15.2:0/922567526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c081006d0 0x7f5c08197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c06d9d640 1 --2- 172.21.15.2:0/922567526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c081019a0 0x7f5c0819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0659c640 1 -- 172.21.15.2:0/922567526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c081019a0 msgr2=0x7f5c0819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0659c640 1 --2- 172.21.15.2:0/922567526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c081019a0 0x7f5c0819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0659c640 1 -- 172.21.15.2:0/922567526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c081006d0 msgr2=0x7f5c08197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0659c640 1 --2- 172.21.15.2:0/922567526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c081006d0 0x7f5c08197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0659c640 1 -- 172.21.15.2:0/922567526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf0009560 con 0x7f5c08101090 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5bfe59c640 1 --2- 172.21.15.2:0/922567526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c081006d0 0x7f5c08197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.162+0000 7f5c0659c640 1 --2- 172.21.15.2:0/922567526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08101090 0x7f5c08197b50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009900 tx=0x7f5bf00049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.163+0000 7f5bfffff640 1 -- 172.21.15.2:0/922567526 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0004250 con 0x7f5c08101090 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.163+0000 7f5c0c815640 1 -- 172.21.15.2:0/922567526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0819f6b0 con 0x7f5c08101090 2024-03-15T12:58:28.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.163+0000 7f5bfffff640 1 -- 172.21.15.2:0/922567526 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5bf00043b0 con 0x7f5c08101090 2024-03-15T12:58:28.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.163+0000 7f5bfffff640 1 -- 172.21.15.2:0/922567526 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0003d20 con 0x7f5c08101090 2024-03-15T12:58:28.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.163+0000 7f5c0c815640 1 -- 172.21.15.2:0/922567526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0819fbc0 con 0x7f5c08101090 2024-03-15T12:58:28.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.164+0000 7f5c0c815640 1 -- 172.21.15.2:0/922567526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0810be10 con 0x7f5c08101090 2024-03-15T12:58:28.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.165+0000 7f5bfffff640 1 -- 172.21.15.2:0/922567526 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f5bf0005000 con 0x7f5c08101090 2024-03-15T12:58:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.165+0000 7f5bfffff640 1 --2- 172.21.15.2:0/922567526 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5bf40a1a30 0x7f5bf40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.166+0000 7f5bfe59c640 1 --2- 172.21.15.2:0/922567526 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5bf40a1a30 0x7f5bf40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.166+0000 7f5bfffff640 1 -- 172.21.15.2:0/922567526 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f5bf00ef920 con 0x7f5c08101090 2024-03-15T12:58:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.166+0000 7f5bfe59c640 1 --2- 172.21.15.2:0/922567526 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5bf40a1a30 0x7f5bf40a3ef0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5c080698c0 tx=0x7f5bf8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:28.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.170+0000 7f5bfffff640 1 -- 172.21.15.2:0/922567526 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5bf00b87f0 con 0x7f5c08101090 2024-03-15T12:58:28.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.353+0000 7f5c0c815640 1 -- 172.21.15.2:0/922567526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5c081a0050 con 0x7f5c08101090 2024-03-15T12:58:28.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.353+0000 7f5bfffff640 1 -- 172.21.15.2:0/922567526 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5bf00bbeb0 con 0x7f5c08101090 2024-03-15T12:58:28.339 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:58:28.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.355+0000 7f5c0c815640 1 -- 172.21.15.2:0/922567526 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5bf40a1a30 msgr2=0x7f5bf40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:28.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.355+0000 7f5c0c815640 1 --2- 172.21.15.2:0/922567526 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5bf40a1a30 0x7f5bf40a3ef0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5c080698c0 tx=0x7f5bf8005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:28.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.355+0000 7f5c0c815640 1 -- 172.21.15.2:0/922567526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08101090 msgr2=0x7f5c08197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:28.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.355+0000 7f5c0c815640 1 --2- 172.21.15.2:0/922567526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08101090 0x7f5c08197b50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009900 tx=0x7f5bf00049e0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.355+0000 7f5c0c815640 1 -- 172.21.15.2:0/922567526 shutdown_connections 2024-03-15T12:58:28.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.355+0000 7f5c0c815640 1 --2- 172.21.15.2:0/922567526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c081019a0 0x7f5c0819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.355+0000 7f5c0c815640 1 --2- 172.21.15.2:0/922567526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08101090 0x7f5c08197b50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.355+0000 7f5c0c815640 1 --2- 172.21.15.2:0/922567526 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5bf40a1a30 0x7f5bf40a3ef0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.355+0000 7f5c0c815640 1 --2- 172.21.15.2:0/922567526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c081006d0 0x7f5c08197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.355+0000 7f5c0c815640 1 -- 172.21.15.2:0/922567526 >> 172.21.15.2:0/922567526 conn(0x7f5c080fb120 msgr2=0x7f5c080ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:28.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.355+0000 7f5c0c815640 1 -- 172.21.15.2:0/922567526 shutdown_connections 2024-03-15T12:58:28.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.355+0000 7f5c0c815640 1 -- 172.21.15.2:0/922567526 wait complete. 2024-03-15T12:58:28.352 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:58:28.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.469+0000 7f2cd2ced640 1 -- 172.21.15.2:0/181913254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ccc101510 msgr2=0x7f2ccc101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:28.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.469+0000 7f2cd2ced640 1 --2- 172.21.15.2:0/181913254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ccc101510 0x7f2ccc101930 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009ca0 tx=0x7f2cc402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.469+0000 7f2cd2ced640 1 -- 172.21.15.2:0/181913254 shutdown_connections 2024-03-15T12:58:28.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.469+0000 7f2cd2ced640 1 --2- 172.21.15.2:0/181913254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ccc1030c0 0x7f2ccc109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.469+0000 7f2cd2ced640 1 --2- 172.21.15.2:0/181913254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ccc102700 0x7f2ccc102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.469+0000 7f2cd2ced640 1 --2- 172.21.15.2:0/181913254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ccc101510 0x7f2ccc101930 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.469+0000 7f2cd2ced640 1 -- 172.21.15.2:0/181913254 >> 172.21.15.2:0/181913254 conn(0x7f2ccc0fcaa0 msgr2=0x7f2ccc0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:28.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.469+0000 7f2cd2ced640 1 -- 172.21.15.2:0/181913254 shutdown_connections 2024-03-15T12:58:28.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.469+0000 7f2cd2ced640 1 -- 172.21.15.2:0/181913254 wait complete. 2024-03-15T12:58:28.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.469+0000 7f2cd2ced640 1 Processor -- start 2024-03-15T12:58:28.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2cd2ced640 1 -- start start 2024-03-15T12:58:28.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2cd2ced640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ccc101510 0x7f2ccc1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:28.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2cd2ced640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ccc102700 0x7f2ccc197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:28.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2cd2ced640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ccc1030c0 0x7f2ccc19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:28.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2cd2ced640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc19f090 con 0x7f2ccc102700 2024-03-15T12:58:28.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2cd2ced640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc19f200 con 0x7f2ccc1030c0 2024-03-15T12:58:28.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2cd2ced640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc19f3c0 con 0x7f2ccc101510 2024-03-15T12:58:28.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ccc102700 0x7f2ccc197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:28.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ccc102700 0x7f2ccc197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50108/0 (socket says 172.21.15.2:50108) 2024-03-15T12:58:28.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2ccbfff640 1 -- 172.21.15.2:0/739085910 learned_addr learned my addr 172.21.15.2:0/739085910 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:28.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2cd0a89640 1 --2- 172.21.15.2:0/739085910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ccc101510 0x7f2ccc1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:28.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2cd128a640 1 --2- 172.21.15.2:0/739085910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ccc1030c0 0x7f2ccc19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:28.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2ccbfff640 1 -- 172.21.15.2:0/739085910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ccc101510 msgr2=0x7f2ccc1975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:28.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2ccbfff640 1 --2- 172.21.15.2:0/739085910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ccc101510 0x7f2ccc1975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2ccbfff640 1 -- 172.21.15.2:0/739085910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ccc1030c0 msgr2=0x7f2ccc19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:28.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2ccbfff640 1 --2- 172.21.15.2:0/739085910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ccc1030c0 0x7f2ccc19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2ccbfff640 1 -- 172.21.15.2:0/739085910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb4009490 con 0x7f2ccc102700 2024-03-15T12:58:28.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2ccbfff640 1 --2- 172.21.15.2:0/739085910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ccc102700 0x7f2ccc197ae0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400ea60 tx=0x7f2cb400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:28.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2cc9ffb640 1 -- 172.21.15.2:0/739085910 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb400cde0 con 0x7f2ccc102700 2024-03-15T12:58:28.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2cc9ffb640 1 -- 172.21.15.2:0/739085910 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2cb40035f0 con 0x7f2ccc102700 2024-03-15T12:58:28.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2cd2ced640 1 -- 172.21.15.2:0/739085910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc4009900 con 0x7f2ccc102700 2024-03-15T12:58:28.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.470+0000 7f2cc9ffb640 1 -- 172.21.15.2:0/739085910 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4010690 con 0x7f2ccc102700 2024-03-15T12:58:28.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.471+0000 7f2cd2ced640 1 -- 172.21.15.2:0/739085910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ccc19fa20 con 0x7f2ccc102700 2024-03-15T12:58:28.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.472+0000 7f2cd2ced640 1 -- 172.21.15.2:0/739085910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ccc108830 con 0x7f2ccc102700 2024-03-15T12:58:28.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.473+0000 7f2cc9ffb640 1 -- 172.21.15.2:0/739085910 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f2cb4003760 con 0x7f2ccc102700 2024-03-15T12:58:28.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.473+0000 7f2cc9ffb640 1 --2- 172.21.15.2:0/739085910 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2cb80a1a30 0x7f2cb80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:28.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.473+0000 7f2cd0a89640 1 --2- 172.21.15.2:0/739085910 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2cb80a1a30 0x7f2cb80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:28.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.473+0000 7f2cc9ffb640 1 -- 172.21.15.2:0/739085910 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f2cb4014070 con 0x7f2ccc102700 2024-03-15T12:58:28.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.474+0000 7f2cd0a89640 1 --2- 172.21.15.2:0/739085910 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2cb80a1a30 0x7f2cb80a3ef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2cc402c700 tx=0x7f2cc4000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:28.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.478+0000 7f2cc9ffb640 1 -- 172.21.15.2:0/739085910 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2cb4096a40 con 0x7f2ccc102700 2024-03-15T12:58:28.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.701+0000 7f2cd2ced640 1 -- 172.21.15.2:0/739085910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ccc10cef0 con 0x7f2ccc102700 2024-03-15T12:58:28.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.702+0000 7f2cc9ffb640 1 -- 172.21.15.2:0/739085910 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11453 (secure 0 0 0) 0x7f2cb409a100 con 0x7f2ccc102700 2024-03-15T12:58:28.688 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:28.688 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T12:56:43.433836+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24464,24500,24460,24539,24434]},"id":1}]} 2024-03-15T12:58:28.688 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 28 2024-03-15T12:58:28.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.704+0000 7f2cd2ced640 1 -- 172.21.15.2:0/739085910 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2cb80a1a30 msgr2=0x7f2cb80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:28.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.704+0000 7f2cd2ced640 1 --2- 172.21.15.2:0/739085910 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2cb80a1a30 0x7f2cb80a3ef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2cc402c700 tx=0x7f2cc4000f40 comp rx=0 tx=0).stop 2024-03-15T12:58:28.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.704+0000 7f2cd2ced640 1 -- 172.21.15.2:0/739085910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ccc102700 msgr2=0x7f2ccc197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:28.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.704+0000 7f2cd2ced640 1 --2- 172.21.15.2:0/739085910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ccc102700 0x7f2ccc197ae0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400ea60 tx=0x7f2cb400ef20 comp rx=0 tx=0).stop 2024-03-15T12:58:28.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.704+0000 7f2cd2ced640 1 -- 172.21.15.2:0/739085910 shutdown_connections 2024-03-15T12:58:28.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.704+0000 7f2cd2ced640 1 --2- 172.21.15.2:0/739085910 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2cb80a1a30 0x7f2cb80a3ef0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.704+0000 7f2cd2ced640 1 --2- 172.21.15.2:0/739085910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ccc1030c0 0x7f2ccc19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.704+0000 7f2cd2ced640 1 --2- 172.21.15.2:0/739085910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ccc102700 0x7f2ccc197ae0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.704+0000 7f2cd2ced640 1 --2- 172.21.15.2:0/739085910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ccc101510 0x7f2ccc1975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.704+0000 7f2cd2ced640 1 -- 172.21.15.2:0/739085910 >> 172.21.15.2:0/739085910 conn(0x7f2ccc0fcaa0 msgr2=0x7f2ccc105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:28.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.704+0000 7f2cd2ced640 1 -- 172.21.15.2:0/739085910 shutdown_connections 2024-03-15T12:58:28.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.704+0000 7f2cd2ced640 1 -- 172.21.15.2:0/739085910 wait complete. 2024-03-15T12:58:28.701 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:58:28.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.818+0000 7f35bec83640 1 -- 172.21.15.2:0/3491511686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8100410 msgr2=0x7f35b8100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:28.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.818+0000 7f35bec83640 1 --2- 172.21.15.2:0/3491511686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8100410 0x7f35b8100830 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009ca0 tx=0x7f35b002c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.818+0000 7f35bec83640 1 -- 172.21.15.2:0/3491511686 shutdown_connections 2024-03-15T12:58:28.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.818+0000 7f35bec83640 1 --2- 172.21.15.2:0/3491511686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35b8101fc0 0x7f35b8108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.818+0000 7f35bec83640 1 --2- 172.21.15.2:0/3491511686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35b8101600 0x7f35b8101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.818+0000 7f35bec83640 1 --2- 172.21.15.2:0/3491511686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8100410 0x7f35b8100830 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.818+0000 7f35bec83640 1 -- 172.21.15.2:0/3491511686 >> 172.21.15.2:0/3491511686 conn(0x7f35b80fb980 msgr2=0x7f35b80fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:28.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.818+0000 7f35bec83640 1 -- 172.21.15.2:0/3491511686 shutdown_connections 2024-03-15T12:58:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.818+0000 7f35bec83640 1 -- 172.21.15.2:0/3491511686 wait complete. 2024-03-15T12:58:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.819+0000 7f35bec83640 1 Processor -- start 2024-03-15T12:58:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.819+0000 7f35bec83640 1 -- start start 2024-03-15T12:58:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.819+0000 7f35bec83640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8100410 0x7f35b81975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.819+0000 7f35bec83640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35b8101600 0x7f35b8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.819+0000 7f35bec83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35b8101fc0 0x7f35b819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.819+0000 7f35bec83640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b819f0b0 con 0x7f35b8100410 2024-03-15T12:58:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.819+0000 7f35bec83640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b819f220 con 0x7f35b8101600 2024-03-15T12:58:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.819+0000 7f35bec83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b819f3e0 con 0x7f35b8101fc0 2024-03-15T12:58:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.819+0000 7f35bca1f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8100410 0x7f35b81975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:28.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.819+0000 7f35bca1f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8100410 0x7f35b81975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50130/0 (socket says 172.21.15.2:50130) 2024-03-15T12:58:28.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.819+0000 7f35bca1f640 1 -- 172.21.15.2:0/2011210877 learned_addr learned my addr 172.21.15.2:0/2011210877 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:28.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.819+0000 7f35b7fff640 1 --2- 172.21.15.2:0/2011210877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35b8101600 0x7f35b8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:28.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.819+0000 7f35bd220640 1 --2- 172.21.15.2:0/2011210877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35b8101fc0 0x7f35b819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:28.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.820+0000 7f35bca1f640 1 -- 172.21.15.2:0/2011210877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35b8101fc0 msgr2=0x7f35b819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:28.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.820+0000 7f35bca1f640 1 --2- 172.21.15.2:0/2011210877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35b8101fc0 0x7f35b819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.820+0000 7f35bca1f640 1 -- 172.21.15.2:0/2011210877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35b8101600 msgr2=0x7f35b8197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:28.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.820+0000 7f35bca1f640 1 --2- 172.21.15.2:0/2011210877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35b8101600 0x7f35b8197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:28.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.820+0000 7f35bca1f640 1 -- 172.21.15.2:0/2011210877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b0009900 con 0x7f35b8100410 2024-03-15T12:58:28.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.820+0000 7f35bca1f640 1 --2- 172.21.15.2:0/2011210877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8100410 0x7f35b81975c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f35b0005e30 tx=0x7f35b0032990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:28.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.820+0000 7f35b5ffb640 1 -- 172.21.15.2:0/2011210877 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0003970 con 0x7f35b8100410 2024-03-15T12:58:28.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.820+0000 7f35b5ffb640 1 -- 172.21.15.2:0/2011210877 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f35b0004040 con 0x7f35b8100410 2024-03-15T12:58:28.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.820+0000 7f35bec83640 1 -- 172.21.15.2:0/2011210877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b819f660 con 0x7f35b8100410 2024-03-15T12:58:28.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.820+0000 7f35b5ffb640 1 -- 172.21.15.2:0/2011210877 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b00406b0 con 0x7f35b8100410 2024-03-15T12:58:28.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.820+0000 7f35bec83640 1 -- 172.21.15.2:0/2011210877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b819fb10 con 0x7f35b8100410 2024-03-15T12:58:28.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.821+0000 7f35bec83640 1 -- 172.21.15.2:0/2011210877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b810bdf0 con 0x7f35b8100410 2024-03-15T12:58:28.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.822+0000 7f35b5ffb640 1 -- 172.21.15.2:0/2011210877 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f35b000a040 con 0x7f35b8100410 2024-03-15T12:58:28.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.823+0000 7f35b5ffb640 1 --2- 172.21.15.2:0/2011210877 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f35a00a1a30 0x7f35a00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:28.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.823+0000 7f35b7fff640 1 --2- 172.21.15.2:0/2011210877 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f35a00a1a30 0x7f35a00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:28.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.823+0000 7f35b5ffb640 1 -- 172.21.15.2:0/2011210877 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f35b00f0670 con 0x7f35b8100410 2024-03-15T12:58:28.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.823+0000 7f35b7fff640 1 --2- 172.21.15.2:0/2011210877 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f35a00a1a30 0x7f35a00a3ef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f35b8198ad0 tx=0x7f35ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:28.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:28.827+0000 7f35b5ffb640 1 -- 172.21.15.2:0/2011210877 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f35b00b9540 con 0x7f35b8100410 2024-03-15T12:58:28.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.011+0000 7f35bec83640 1 -- 172.21.15.2:0/2011210877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35b810c000 con 0x7f35b8100410 2024-03-15T12:58:28.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.012+0000 7f35b5ffb640 1 -- 172.21.15.2:0/2011210877 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24526 (secure 0 0 0) 0x7f35b00bcc00 con 0x7f35b8100410 2024-03-15T12:58:28.998 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:28.999 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":91,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:57:19.826817+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:58:29.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.014+0000 7f35bec83640 1 -- 172.21.15.2:0/2011210877 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f35a00a1a30 msgr2=0x7f35a00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.014+0000 7f35bec83640 1 --2- 172.21.15.2:0/2011210877 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f35a00a1a30 0x7f35a00a3ef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f35b8198ad0 tx=0x7f35ac005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:29.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.014+0000 7f35bec83640 1 -- 172.21.15.2:0/2011210877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8100410 msgr2=0x7f35b81975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.014+0000 7f35bec83640 1 --2- 172.21.15.2:0/2011210877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8100410 0x7f35b81975c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f35b0005e30 tx=0x7f35b0032990 comp rx=0 tx=0).stop 2024-03-15T12:58:29.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.015+0000 7f35bec83640 1 -- 172.21.15.2:0/2011210877 shutdown_connections 2024-03-15T12:58:29.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.015+0000 7f35bec83640 1 --2- 172.21.15.2:0/2011210877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f35b8101fc0 0x7f35b819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.015+0000 7f35bec83640 1 --2- 172.21.15.2:0/2011210877 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f35a00a1a30 0x7f35a00a3ef0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.015+0000 7f35bec83640 1 --2- 172.21.15.2:0/2011210877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35b8101600 0x7f35b8197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.015+0000 7f35bec83640 1 --2- 172.21.15.2:0/2011210877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35b8100410 0x7f35b81975c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.015+0000 7f35bec83640 1 -- 172.21.15.2:0/2011210877 >> 172.21.15.2:0/2011210877 conn(0x7f35b80fb980 msgr2=0x7f35b80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:29.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.015+0000 7f35bec83640 1 -- 172.21.15.2:0/2011210877 shutdown_connections 2024-03-15T12:58:29.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.015+0000 7f35bec83640 1 -- 172.21.15.2:0/2011210877 wait complete. 2024-03-15T12:58:29.010 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:28 smithi002 ceph-mon[32060]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:58:29.011 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:28 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/720161253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:29.011 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:28 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2790154771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:29.011 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:28 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/922567526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:29.012 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-15T12:51:47.004843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-15T12:56:18.559882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-15T12:56:19.751767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-15T12:56:23.664879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-15T12:58:29.013 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-15T12:58:29.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:28 smithi184 ceph-mon[35993]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:58:29.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:28 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/720161253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:29.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:28 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2790154771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:29.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:28 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/922567526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:29.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:28 smithi149 ceph-mon[35927]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:58:29.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:28 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/720161253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:29.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:28 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2790154771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:29.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:28 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/922567526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:29.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.133+0000 7f827c0ed640 1 -- 172.21.15.2:0/793697109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82741027b0 msgr2=0x7f8274102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.133+0000 7f827c0ed640 1 --2- 172.21.15.2:0/793697109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82741027b0 0x7f8274102c30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8270009900 tx=0x7f827002bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:29.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.133+0000 7f827c0ed640 1 -- 172.21.15.2:0/793697109 shutdown_connections 2024-03-15T12:58:29.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.133+0000 7f827c0ed640 1 --2- 172.21.15.2:0/793697109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8274103170 0x7f8274109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.133+0000 7f827c0ed640 1 --2- 172.21.15.2:0/793697109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82741027b0 0x7f8274102c30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.133+0000 7f827c0ed640 1 --2- 172.21.15.2:0/793697109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82740ff7e0 0x7f82740ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.133+0000 7f827c0ed640 1 -- 172.21.15.2:0/793697109 >> 172.21.15.2:0/793697109 conn(0x7f82740fb0f0 msgr2=0x7f82740fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:29.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.133+0000 7f827c0ed640 1 -- 172.21.15.2:0/793697109 shutdown_connections 2024-03-15T12:58:29.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.133+0000 7f827c0ed640 1 -- 172.21.15.2:0/793697109 wait complete. 2024-03-15T12:58:29.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827c0ed640 1 Processor -- start 2024-03-15T12:58:29.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827c0ed640 1 -- start start 2024-03-15T12:58:29.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827c0ed640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82740ff7e0 0x7f82741975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:29.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827c0ed640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82741027b0 0x7f8274197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:29.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827c0ed640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8274103170 0x7f827419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:29.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827c0ed640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827419f090 con 0x7f8274103170 2024-03-15T12:58:29.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827c0ed640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827419f200 con 0x7f82740ff7e0 2024-03-15T12:58:29.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827c0ed640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827419f3c0 con 0x7f82741027b0 2024-03-15T12:58:29.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827a68a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8274103170 0x7f827419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:29.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827a68a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8274103170 0x7f827419eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50146/0 (socket says 172.21.15.2:50146) 2024-03-15T12:58:29.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827a68a640 1 -- 172.21.15.2:0/4280331770 learned_addr learned my addr 172.21.15.2:0/4280331770 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:29.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f8279e89640 1 --2- 172.21.15.2:0/4280331770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82740ff7e0 0x7f82741975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:29.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f8279688640 1 --2- 172.21.15.2:0/4280331770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82741027b0 0x7f8274197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:29.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827a68a640 1 -- 172.21.15.2:0/4280331770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82741027b0 msgr2=0x7f8274197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827a68a640 1 --2- 172.21.15.2:0/4280331770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82741027b0 0x7f8274197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827a68a640 1 -- 172.21.15.2:0/4280331770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82740ff7e0 msgr2=0x7f82741975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827a68a640 1 --2- 172.21.15.2:0/4280331770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82740ff7e0 0x7f82741975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827a68a640 1 -- 172.21.15.2:0/4280331770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8270009560 con 0x7f8274103170 2024-03-15T12:58:29.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.134+0000 7f827a68a640 1 --2- 172.21.15.2:0/4280331770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8274103170 0x7f827419eb50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f826c00eb10 tx=0x7f826c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:29.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.135+0000 7f826affd640 1 -- 172.21.15.2:0/4280331770 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c004b20 con 0x7f8274103170 2024-03-15T12:58:29.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.135+0000 7f827c0ed640 1 -- 172.21.15.2:0/4280331770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827419f6a0 con 0x7f8274103170 2024-03-15T12:58:29.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.135+0000 7f826affd640 1 -- 172.21.15.2:0/4280331770 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f826c004c80 con 0x7f8274103170 2024-03-15T12:58:29.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.135+0000 7f826affd640 1 -- 172.21.15.2:0/4280331770 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c010690 con 0x7f8274103170 2024-03-15T12:58:29.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.135+0000 7f827c0ed640 1 -- 172.21.15.2:0/4280331770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827419fb80 con 0x7f8274103170 2024-03-15T12:58:29.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.136+0000 7f827c0ed640 1 -- 172.21.15.2:0/4280331770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82741088e0 con 0x7f8274103170 2024-03-15T12:58:29.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.137+0000 7f826affd640 1 -- 172.21.15.2:0/4280331770 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f826c004df0 con 0x7f8274103170 2024-03-15T12:58:29.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.137+0000 7f826affd640 1 --2- 172.21.15.2:0/4280331770 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f82600a1a30 0x7f82600a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:29.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.137+0000 7f8279e89640 1 --2- 172.21.15.2:0/4280331770 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f82600a1a30 0x7f82600a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:29.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.137+0000 7f826affd640 1 -- 172.21.15.2:0/4280331770 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f826c014070 con 0x7f8274103170 2024-03-15T12:58:29.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.138+0000 7f8279e89640 1 --2- 172.21.15.2:0/4280331770 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f82600a1a30 0x7f82600a3ef0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8274069900 tx=0x7f8264005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:29.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.142+0000 7f826affd640 1 -- 172.21.15.2:0/4280331770 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f826c0d3020 con 0x7f8274103170 2024-03-15T12:58:29.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.325+0000 7f827c0ed640 1 -- 172.21.15.2:0/4280331770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f827402d000 con 0x7f8274103170 2024-03-15T12:58:29.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.326+0000 7f826affd640 1 -- 172.21.15.2:0/4280331770 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f826c096ce0 con 0x7f8274103170 2024-03-15T12:58:29.312 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-15T12:58:29.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.328+0000 7f827c0ed640 1 -- 172.21.15.2:0/4280331770 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f82600a1a30 msgr2=0x7f82600a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.328+0000 7f827c0ed640 1 --2- 172.21.15.2:0/4280331770 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f82600a1a30 0x7f82600a3ef0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8274069900 tx=0x7f8264005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:29.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.328+0000 7f827c0ed640 1 -- 172.21.15.2:0/4280331770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8274103170 msgr2=0x7f827419eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.328+0000 7f827c0ed640 1 --2- 172.21.15.2:0/4280331770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8274103170 0x7f827419eb50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f826c00eb10 tx=0x7f826c00c2d0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.328+0000 7f827c0ed640 1 -- 172.21.15.2:0/4280331770 shutdown_connections 2024-03-15T12:58:29.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.328+0000 7f827c0ed640 1 --2- 172.21.15.2:0/4280331770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8274103170 0x7f827419eb50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.328+0000 7f827c0ed640 1 --2- 172.21.15.2:0/4280331770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82741027b0 0x7f8274197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.328+0000 7f827c0ed640 1 --2- 172.21.15.2:0/4280331770 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f82600a1a30 0x7f82600a3ef0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.328+0000 7f827c0ed640 1 --2- 172.21.15.2:0/4280331770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82740ff7e0 0x7f82741975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.328+0000 7f827c0ed640 1 -- 172.21.15.2:0/4280331770 >> 172.21.15.2:0/4280331770 conn(0x7f82740fb0f0 msgr2=0x7f8274105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:29.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.328+0000 7f827c0ed640 1 -- 172.21.15.2:0/4280331770 shutdown_connections 2024-03-15T12:58:29.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.328+0000 7f827c0ed640 1 -- 172.21.15.2:0/4280331770 wait complete. 2024-03-15T12:58:29.325 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-15T12:58:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.445+0000 7fe7cae40640 1 -- 172.21.15.2:0/894846053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7c41030b0 msgr2=0x7fe7c4109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.445+0000 7fe7cae40640 1 --2- 172.21.15.2:0/894846053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7c41030b0 0x7fe7c4109940 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009ca0 tx=0x7fe7bc02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.445+0000 7fe7cae40640 1 -- 172.21.15.2:0/894846053 shutdown_connections 2024-03-15T12:58:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.445+0000 7fe7cae40640 1 --2- 172.21.15.2:0/894846053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7c41030b0 0x7fe7c4109940 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.445+0000 7fe7cae40640 1 --2- 172.21.15.2:0/894846053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7c41026f0 0x7fe7c4102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.445+0000 7fe7cae40640 1 --2- 172.21.15.2:0/894846053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7c4101500 0x7fe7c4101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.445+0000 7fe7cae40640 1 -- 172.21.15.2:0/894846053 >> 172.21.15.2:0/894846053 conn(0x7fe7c40fca90 msgr2=0x7fe7c40feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.445+0000 7fe7cae40640 1 -- 172.21.15.2:0/894846053 shutdown_connections 2024-03-15T12:58:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.445+0000 7fe7cae40640 1 -- 172.21.15.2:0/894846053 wait complete. 2024-03-15T12:58:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7cae40640 1 Processor -- start 2024-03-15T12:58:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7cae40640 1 -- start start 2024-03-15T12:58:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7cae40640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7c4101500 0x7fe7c4197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7cae40640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7c41026f0 0x7fe7c4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7cae40640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7c41030b0 0x7fe7c419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7cae40640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c419f030 con 0x7fe7c41026f0 2024-03-15T12:58:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7cae40640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c419f1a0 con 0x7fe7c41030b0 2024-03-15T12:58:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7cae40640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c419f360 con 0x7fe7c4101500 2024-03-15T12:58:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7c41026f0 0x7fe7c4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7c41026f0 0x7fe7c4197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50174/0 (socket says 172.21.15.2:50174) 2024-03-15T12:58:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7c3fff640 1 -- 172.21.15.2:0/3803383444 learned_addr learned my addr 172.21.15.2:0/3803383444 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7c8bdc640 1 --2- 172.21.15.2:0/3803383444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7c4101500 0x7fe7c4197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:29.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7c93dd640 1 --2- 172.21.15.2:0/3803383444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7c41030b0 0x7fe7c419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:29.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7c3fff640 1 -- 172.21.15.2:0/3803383444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7c4101500 msgr2=0x7fe7c4197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7c3fff640 1 --2- 172.21.15.2:0/3803383444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7c4101500 0x7fe7c4197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7c3fff640 1 -- 172.21.15.2:0/3803383444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7c41030b0 msgr2=0x7fe7c419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7c3fff640 1 --2- 172.21.15.2:0/3803383444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7c41030b0 0x7fe7c419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7c3fff640 1 -- 172.21.15.2:0/3803383444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc009900 con 0x7fe7c41026f0 2024-03-15T12:58:29.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7c8bdc640 1 --2- 172.21.15.2:0/3803383444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7c4101500 0x7fe7c4197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:29.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.446+0000 7fe7c3fff640 1 --2- 172.21.15.2:0/3803383444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7c41026f0 0x7fe7c4197a80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000e9f0 tx=0x7fe7b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:29.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.447+0000 7fe7c1ffb640 1 -- 172.21.15.2:0/3803383444 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0009700 con 0x7fe7c41026f0 2024-03-15T12:58:29.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.447+0000 7fe7cae40640 1 -- 172.21.15.2:0/3803383444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c419f640 con 0x7fe7c41026f0 2024-03-15T12:58:29.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.447+0000 7fe7c1ffb640 1 -- 172.21.15.2:0/3803383444 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe7b000cd80 con 0x7fe7c41026f0 2024-03-15T12:58:29.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.447+0000 7fe7c1ffb640 1 -- 172.21.15.2:0/3803383444 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0018920 con 0x7fe7c41026f0 2024-03-15T12:58:29.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.447+0000 7fe7cae40640 1 -- 172.21.15.2:0/3803383444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c419fb50 con 0x7fe7c41026f0 2024-03-15T12:58:29.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.448+0000 7fe7cae40640 1 -- 172.21.15.2:0/3803383444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c4108820 con 0x7fe7c41026f0 2024-03-15T12:58:29.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.449+0000 7fe7c1ffb640 1 -- 172.21.15.2:0/3803383444 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fe7b0018a80 con 0x7fe7c41026f0 2024-03-15T12:58:29.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.449+0000 7fe7c1ffb640 1 --2- 172.21.15.2:0/3803383444 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe7b40a1a30 0x7fe7b40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:29.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.450+0000 7fe7c8bdc640 1 --2- 172.21.15.2:0/3803383444 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe7b40a1a30 0x7fe7b40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:29.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.450+0000 7fe7c1ffb640 1 -- 172.21.15.2:0/3803383444 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fe7b0014070 con 0x7fe7c41026f0 2024-03-15T12:58:29.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.450+0000 7fe7c8bdc640 1 --2- 172.21.15.2:0/3803383444 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe7b40a1a30 0x7fe7b40a3ef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4102550 tx=0x7fe7b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:29.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.454+0000 7fe7c1ffb640 1 -- 172.21.15.2:0/3803383444 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe7b0053280 con 0x7fe7c41026f0 2024-03-15T12:58:29.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.638+0000 7fe7cae40640 1 -- 172.21.15.2:0/3803383444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe7c402d000 con 0x7fe7c41026f0 2024-03-15T12:58:29.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.638+0000 7fe7c1ffb640 1 -- 172.21.15.2:0/3803383444 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe7b00969d0 con 0x7fe7c41026f0 2024-03-15T12:58:29.624 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:58:29.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.640+0000 7fe7cae40640 1 -- 172.21.15.2:0/3803383444 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe7b40a1a30 msgr2=0x7fe7b40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.640+0000 7fe7cae40640 1 --2- 172.21.15.2:0/3803383444 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe7b40a1a30 0x7fe7b40a3ef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4102550 tx=0x7fe7b8005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:29.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.640+0000 7fe7cae40640 1 -- 172.21.15.2:0/3803383444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7c41026f0 msgr2=0x7fe7c4197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.640+0000 7fe7cae40640 1 --2- 172.21.15.2:0/3803383444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7c41026f0 0x7fe7c4197a80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000e9f0 tx=0x7fe7b000eeb0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.640+0000 7fe7cae40640 1 -- 172.21.15.2:0/3803383444 shutdown_connections 2024-03-15T12:58:29.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.640+0000 7fe7cae40640 1 --2- 172.21.15.2:0/3803383444 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fe7b40a1a30 0x7fe7b40a3ef0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.640+0000 7fe7cae40640 1 --2- 172.21.15.2:0/3803383444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7c41030b0 0x7fe7c419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.640+0000 7fe7cae40640 1 --2- 172.21.15.2:0/3803383444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7c41026f0 0x7fe7c4197a80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.640+0000 7fe7cae40640 1 --2- 172.21.15.2:0/3803383444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7c4101500 0x7fe7c4197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.640+0000 7fe7cae40640 1 -- 172.21.15.2:0/3803383444 >> 172.21.15.2:0/3803383444 conn(0x7fe7c40fca90 msgr2=0x7fe7c41059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:29.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.640+0000 7fe7cae40640 1 -- 172.21.15.2:0/3803383444 shutdown_connections 2024-03-15T12:58:29.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.640+0000 7fe7cae40640 1 -- 172.21.15.2:0/3803383444 wait complete. 2024-03-15T12:58:29.637 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-15T12:58:29.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 -- 172.21.15.2:0/1629096041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce8101fa0 msgr2=0x7f6ce8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 --2- 172.21.15.2:0/1629096041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce8101fa0 0x7f6ce8108830 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009ca0 tx=0x7f6ce002c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 -- 172.21.15.2:0/1629096041 shutdown_connections 2024-03-15T12:58:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 --2- 172.21.15.2:0/1629096041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce8101fa0 0x7f6ce8108830 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 --2- 172.21.15.2:0/1629096041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ce81015e0 0x7f6ce8101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 --2- 172.21.15.2:0/1629096041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ce81003f0 0x7f6ce8100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 -- 172.21.15.2:0/1629096041 >> 172.21.15.2:0/1629096041 conn(0x7f6ce80fb980 msgr2=0x7f6ce80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 -- 172.21.15.2:0/1629096041 shutdown_connections 2024-03-15T12:58:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 -- 172.21.15.2:0/1629096041 wait complete. 2024-03-15T12:58:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 Processor -- start 2024-03-15T12:58:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 -- start start 2024-03-15T12:58:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce81003f0 0x7f6ce8104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ce81015e0 0x7f6ce81029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ce8101fa0 0x7f6ce8102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce8103660 con 0x7f6ce81003f0 2024-03-15T12:58:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce81037d0 con 0x7f6ce81015e0 2024-03-15T12:58:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.758+0000 7f6cef148640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce81a15f0 con 0x7f6ce8101fa0 2024-03-15T12:58:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6cecee4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce81003f0 0x7f6ce8104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6cecee4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce81003f0 0x7f6ce8104380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50196/0 (socket says 172.21.15.2:50196) 2024-03-15T12:58:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6cecee4640 1 -- 172.21.15.2:0/2557336793 learned_addr learned my addr 172.21.15.2:0/2557336793 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6ce7fff640 1 --2- 172.21.15.2:0/2557336793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ce81015e0 0x7f6ce81029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6ced6e5640 1 --2- 172.21.15.2:0/2557336793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ce8101fa0 0x7f6ce8102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6cecee4640 1 -- 172.21.15.2:0/2557336793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ce8101fa0 msgr2=0x7f6ce8102f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6cecee4640 1 --2- 172.21.15.2:0/2557336793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ce8101fa0 0x7f6ce8102f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6cecee4640 1 -- 172.21.15.2:0/2557336793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ce81015e0 msgr2=0x7f6ce81029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6cecee4640 1 --2- 172.21.15.2:0/2557336793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ce81015e0 0x7f6ce81029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6cecee4640 1 -- 172.21.15.2:0/2557336793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc009490 con 0x7f6ce81003f0 2024-03-15T12:58:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6cecee4640 1 --2- 172.21.15.2:0/2557336793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce81003f0 0x7f6ce8104380 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00e9f0 tx=0x7f6cdc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:29.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6ce5ffb640 1 -- 172.21.15.2:0/2557336793 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00cde0 con 0x7f6ce81003f0 2024-03-15T12:58:29.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6ce5ffb640 1 -- 172.21.15.2:0/2557336793 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6cdc010900 con 0x7f6ce81003f0 2024-03-15T12:58:29.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6cef148640 1 -- 172.21.15.2:0/2557336793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce0009900 con 0x7f6ce81003f0 2024-03-15T12:58:29.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6ce5ffb640 1 -- 172.21.15.2:0/2557336793 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0189e0 con 0x7f6ce81003f0 2024-03-15T12:58:29.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.759+0000 7f6cef148640 1 -- 172.21.15.2:0/2557336793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce81a1c70 con 0x7f6ce81003f0 2024-03-15T12:58:29.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.760+0000 7f6cef148640 1 -- 172.21.15.2:0/2557336793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce810bdd0 con 0x7f6ce81003f0 2024-03-15T12:58:29.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.761+0000 7f6ce5ffb640 1 -- 172.21.15.2:0/2557336793 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f6cdc010a70 con 0x7f6ce81003f0 2024-03-15T12:58:29.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.762+0000 7f6ce5ffb640 1 --2- 172.21.15.2:0/2557336793 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6cd80a1ab0 0x7f6cd80a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:29.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.762+0000 7f6ce7fff640 1 --2- 172.21.15.2:0/2557336793 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6cd80a1ab0 0x7f6cd80a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:29.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.762+0000 7f6ce5ffb640 1 -- 172.21.15.2:0/2557336793 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f6cdc014070 con 0x7f6ce81003f0 2024-03-15T12:58:29.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.762+0000 7f6ce7fff640 1 --2- 172.21.15.2:0/2557336793 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6cd80a1ab0 0x7f6cd80a3f70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6ce81040f0 tx=0x7f6cd4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:29.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.766+0000 7f6ce5ffb640 1 -- 172.21.15.2:0/2557336793 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6cdc096b80 con 0x7f6ce81003f0 2024-03-15T12:58:29.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.949+0000 7f6cef148640 1 -- 172.21.15.2:0/2557336793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6ce81a2130 con 0x7f6ce81003f0 2024-03-15T12:58:29.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.949+0000 7f6ce5ffb640 1 -- 172.21.15.2:0/2557336793 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6cdc09a240 con 0x7f6ce81003f0 2024-03-15T12:58:29.935 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-03-15T12:58:29.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.951+0000 7f6cef148640 1 -- 172.21.15.2:0/2557336793 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6cd80a1ab0 msgr2=0x7f6cd80a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.951+0000 7f6cef148640 1 --2- 172.21.15.2:0/2557336793 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6cd80a1ab0 0x7f6cd80a3f70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6ce81040f0 tx=0x7f6cd4005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:29.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.951+0000 7f6cef148640 1 -- 172.21.15.2:0/2557336793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce81003f0 msgr2=0x7f6ce8104380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:29.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.951+0000 7f6cef148640 1 --2- 172.21.15.2:0/2557336793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce81003f0 0x7f6ce8104380 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00e9f0 tx=0x7f6cdc00eeb0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.951+0000 7f6cef148640 1 -- 172.21.15.2:0/2557336793 shutdown_connections 2024-03-15T12:58:29.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.951+0000 7f6cef148640 1 --2- 172.21.15.2:0/2557336793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ce8101fa0 0x7f6ce8102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.951+0000 7f6cef148640 1 --2- 172.21.15.2:0/2557336793 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f6cd80a1ab0 0x7f6cd80a3f70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.951+0000 7f6cef148640 1 --2- 172.21.15.2:0/2557336793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ce81015e0 0x7f6ce81029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.951+0000 7f6cef148640 1 --2- 172.21.15.2:0/2557336793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ce81003f0 0x7f6ce8104380 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:29.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.951+0000 7f6cef148640 1 -- 172.21.15.2:0/2557336793 >> 172.21.15.2:0/2557336793 conn(0x7f6ce80fb980 msgr2=0x7f6ce80fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:29.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.951+0000 7f6cef148640 1 -- 172.21.15.2:0/2557336793 shutdown_connections 2024-03-15T12:58:29.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:29.951+0000 7f6cef148640 1 -- 172.21.15.2:0/2557336793 wait complete. 2024-03-15T12:58:29.947 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:29 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/739085910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:29.947 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:29 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2011210877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:29.947 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:29 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/4280331770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:29.947 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:29 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3803383444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:29.948 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-15T12:58:30.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:29 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/739085910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:30.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:29 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2011210877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:30.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:29 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/4280331770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:30.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:29 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3803383444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:30.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.067+0000 7fedb026b640 1 -- 172.21.15.2:0/4170069453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8100740 msgr2=0x7feda8100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.067+0000 7fedb026b640 1 --2- 172.21.15.2:0/4170069453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8100740 0x7feda8100ba0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fed98009900 tx=0x7fed9802bcd0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.067+0000 7fedb026b640 1 -- 172.21.15.2:0/4170069453 shutdown_connections 2024-03-15T12:58:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedb026b640 1 --2- 172.21.15.2:0/4170069453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feda81010e0 0x7feda8101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedb026b640 1 --2- 172.21.15.2:0/4170069453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8100740 0x7feda8100ba0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedb026b640 1 --2- 172.21.15.2:0/4170069453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda80ffde0 0x7feda8100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedb026b640 1 -- 172.21.15.2:0/4170069453 >> 172.21.15.2:0/4170069453 conn(0x7feda80fb980 msgr2=0x7feda80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedb026b640 1 -- 172.21.15.2:0/4170069453 shutdown_connections 2024-03-15T12:58:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedb026b640 1 -- 172.21.15.2:0/4170069453 wait complete. 2024-03-15T12:58:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedb026b640 1 Processor -- start 2024-03-15T12:58:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedb026b640 1 -- start start 2024-03-15T12:58:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedb026b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feda80ffde0 0x7feda81997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedb026b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8100740 0x7feda8199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedb026b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda81010e0 0x7feda81a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedb026b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda81a12c0 con 0x7feda8100740 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedb026b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda81a1430 con 0x7feda81010e0 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedb026b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda81a15f0 con 0x7feda80ffde0 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedad806640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8100740 0x7feda8199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedad806640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8100740 0x7feda8199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50218/0 (socket says 172.21.15.2:50218) 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedad806640 1 -- 172.21.15.2:0/3264273531 learned_addr learned my addr 172.21.15.2:0/3264273531 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedae007640 1 --2- 172.21.15.2:0/3264273531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feda80ffde0 0x7feda81997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.068+0000 7fedae808640 1 --2- 172.21.15.2:0/3264273531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda81010e0 0x7feda81a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.069+0000 7fedad806640 1 -- 172.21.15.2:0/3264273531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feda80ffde0 msgr2=0x7feda81997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.069+0000 7fedad806640 1 --2- 172.21.15.2:0/3264273531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feda80ffde0 0x7feda81997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.069+0000 7fedad806640 1 -- 172.21.15.2:0/3264273531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda81010e0 msgr2=0x7feda81a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.069+0000 7fedad806640 1 --2- 172.21.15.2:0/3264273531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda81010e0 0x7feda81a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.069+0000 7fedad806640 1 -- 172.21.15.2:0/3264273531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed98009560 con 0x7feda8100740 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.069+0000 7fedae808640 1 --2- 172.21.15.2:0/3264273531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda81010e0 0x7feda81a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.069+0000 7fedad806640 1 --2- 172.21.15.2:0/3264273531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8100740 0x7feda8199d10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fed98005ab0 tx=0x7fed98004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.069+0000 7fed9f7fe640 1 -- 172.21.15.2:0/3264273531 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed98004250 con 0x7feda8100740 2024-03-15T12:58:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.069+0000 7fedb026b640 1 -- 172.21.15.2:0/3264273531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda81a1870 con 0x7feda8100740 2024-03-15T12:58:30.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.069+0000 7fed9f7fe640 1 -- 172.21.15.2:0/3264273531 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fed980043b0 con 0x7feda8100740 2024-03-15T12:58:30.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.069+0000 7fed9f7fe640 1 -- 172.21.15.2:0/3264273531 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed98003d20 con 0x7feda8100740 2024-03-15T12:58:30.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.069+0000 7fedb026b640 1 -- 172.21.15.2:0/3264273531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda81a1dd0 con 0x7feda8100740 2024-03-15T12:58:30.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.070+0000 7fedb026b640 1 -- 172.21.15.2:0/3264273531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda810ad40 con 0x7feda8100740 2024-03-15T12:58:30.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.071+0000 7fed9f7fe640 1 -- 172.21.15.2:0/3264273531 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fed98005000 con 0x7feda8100740 2024-03-15T12:58:30.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.072+0000 7fed9f7fe640 1 --2- 172.21.15.2:0/3264273531 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fed940a1a30 0x7fed940a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:30.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.072+0000 7fedae007640 1 --2- 172.21.15.2:0/3264273531 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fed940a1a30 0x7fed940a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:30.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.072+0000 7fed9f7fe640 1 -- 172.21.15.2:0/3264273531 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fed980effa0 con 0x7feda8100740 2024-03-15T12:58:30.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.072+0000 7fedae007640 1 --2- 172.21.15.2:0/3264273531 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fed940a1a30 0x7fed940a3ef0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feda0009bd0 tx=0x7feda0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:30.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:29 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/739085910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:30.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:29 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2011210877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:30.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:29 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/4280331770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:30.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:29 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3803383444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:30.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.076+0000 7fed9f7fe640 1 -- 172.21.15.2:0/3264273531 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fed980b8e70 con 0x7feda8100740 2024-03-15T12:58:30.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.260+0000 7fedb026b640 1 -- 172.21.15.2:0/3264273531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7feda810a7b0 con 0x7feda8100740 2024-03-15T12:58:30.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.260+0000 7fed9f7fe640 1 -- 172.21.15.2:0/3264273531 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fed98043020 con 0x7feda8100740 2024-03-15T12:58:30.246 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:58:30.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.262+0000 7fedb026b640 1 -- 172.21.15.2:0/3264273531 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fed940a1a30 msgr2=0x7fed940a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:30.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.262+0000 7fedb026b640 1 --2- 172.21.15.2:0/3264273531 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fed940a1a30 0x7fed940a3ef0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feda0009bd0 tx=0x7feda0005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:30.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.262+0000 7fedb026b640 1 -- 172.21.15.2:0/3264273531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8100740 msgr2=0x7feda8199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:30.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.262+0000 7fedb026b640 1 --2- 172.21.15.2:0/3264273531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8100740 0x7feda8199d10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fed98005ab0 tx=0x7fed98004a30 comp rx=0 tx=0).stop 2024-03-15T12:58:30.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.262+0000 7fedb026b640 1 -- 172.21.15.2:0/3264273531 shutdown_connections 2024-03-15T12:58:30.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.262+0000 7fedb026b640 1 --2- 172.21.15.2:0/3264273531 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fed940a1a30 0x7fed940a3ef0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.262+0000 7fedb026b640 1 --2- 172.21.15.2:0/3264273531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda81010e0 0x7feda81a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.262+0000 7fedb026b640 1 --2- 172.21.15.2:0/3264273531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feda8100740 0x7feda8199d10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.262+0000 7fedb026b640 1 --2- 172.21.15.2:0/3264273531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feda80ffde0 0x7feda81997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.262+0000 7fedb026b640 1 -- 172.21.15.2:0/3264273531 >> 172.21.15.2:0/3264273531 conn(0x7feda80fb980 msgr2=0x7feda80fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:30.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.262+0000 7fedb026b640 1 -- 172.21.15.2:0/3264273531 shutdown_connections 2024-03-15T12:58:30.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.262+0000 7fedb026b640 1 -- 172.21.15.2:0/3264273531 wait complete. 2024-03-15T12:58:30.259 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:58:30.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.377+0000 7f2e89b47640 1 -- 172.21.15.2:0/2487679685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e84104980 msgr2=0x7f2e84106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:30.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.377+0000 7f2e89b47640 1 --2- 172.21.15.2:0/2487679685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e84104980 0x7f2e84106d70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009860 tx=0x7f2e7402bb70 comp rx=0 tx=0).stop 2024-03-15T12:58:30.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.377+0000 7f2e89b47640 1 -- 172.21.15.2:0/2487679685 shutdown_connections 2024-03-15T12:58:30.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.377+0000 7f2e89b47640 1 --2- 172.21.15.2:0/2487679685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e841072b0 0x7f2e841096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.377+0000 7f2e89b47640 1 --2- 172.21.15.2:0/2487679685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e84104980 0x7f2e84106d70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.377+0000 7f2e89b47640 1 --2- 172.21.15.2:0/2487679685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e84102050 0x7f2e84104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.377+0000 7f2e89b47640 1 -- 172.21.15.2:0/2487679685 >> 172.21.15.2:0/2487679685 conn(0x7f2e840fb980 msgr2=0x7f2e840fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:30.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.377+0000 7f2e89b47640 1 -- 172.21.15.2:0/2487679685 shutdown_connections 2024-03-15T12:58:30.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.377+0000 7f2e89b47640 1 -- 172.21.15.2:0/2487679685 wait complete. 2024-03-15T12:58:30.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.377+0000 7f2e89b47640 1 Processor -- start 2024-03-15T12:58:30.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e89b47640 1 -- start start 2024-03-15T12:58:30.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e89b47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e84102050 0x7f2e841975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:30.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e89b47640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e84104980 0x7f2e84197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:30.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e89b47640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e841072b0 0x7f2e8419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:30.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e89b47640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8419f0e0 con 0x7f2e84104980 2024-03-15T12:58:30.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e89b47640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8419f250 con 0x7f2e841072b0 2024-03-15T12:58:30.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e89b47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8419f410 con 0x7f2e84102050 2024-03-15T12:58:30.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e82ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e84104980 0x7f2e84197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:30.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e82ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e84104980 0x7f2e84197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50242/0 (socket says 172.21.15.2:50242) 2024-03-15T12:58:30.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e837fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e84102050 0x7f2e841975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:30.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e82ffd640 1 -- 172.21.15.2:0/1220595214 learned_addr learned my addr 172.21.15.2:0/1220595214 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:30.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e83fff640 1 --2- 172.21.15.2:0/1220595214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e841072b0 0x7f2e8419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:30.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e82ffd640 1 -- 172.21.15.2:0/1220595214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e84102050 msgr2=0x7f2e841975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:30.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e82ffd640 1 --2- 172.21.15.2:0/1220595214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e84102050 0x7f2e841975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e82ffd640 1 -- 172.21.15.2:0/1220595214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e841072b0 msgr2=0x7f2e8419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:30.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e82ffd640 1 --2- 172.21.15.2:0/1220595214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e841072b0 0x7f2e8419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e82ffd640 1 -- 172.21.15.2:0/1220595214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c009900 con 0x7f2e84104980 2024-03-15T12:58:30.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e837fe640 1 --2- 172.21.15.2:0/1220595214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e84102050 0x7f2e841975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:30.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.378+0000 7f2e82ffd640 1 --2- 172.21.15.2:0/1220595214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e84104980 0x7f2e84197b30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2e7402c0b0 tx=0x7f2e74032460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:30.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.379+0000 7f2e80ff9640 1 -- 172.21.15.2:0/1220595214 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7402c510 con 0x7f2e84104980 2024-03-15T12:58:30.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.379+0000 7f2e89b47640 1 -- 172.21.15.2:0/1220595214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e74009490 con 0x7f2e84104980 2024-03-15T12:58:30.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.379+0000 7f2e80ff9640 1 -- 172.21.15.2:0/1220595214 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2e7402cb40 con 0x7f2e84104980 2024-03-15T12:58:30.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.379+0000 7f2e80ff9640 1 -- 172.21.15.2:0/1220595214 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e74032600 con 0x7f2e84104980 2024-03-15T12:58:30.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.379+0000 7f2e89b47640 1 -- 172.21.15.2:0/1220595214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8419f9b0 con 0x7f2e84104980 2024-03-15T12:58:30.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.380+0000 7f2e89b47640 1 -- 172.21.15.2:0/1220595214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e840fd470 con 0x7f2e84104980 2024-03-15T12:58:30.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.381+0000 7f2e80ff9640 1 -- 172.21.15.2:0/1220595214 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f2e740037f0 con 0x7f2e84104980 2024-03-15T12:58:30.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.381+0000 7f2e80ff9640 1 --2- 172.21.15.2:0/1220595214 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2e6c0a1a30 0x7f2e6c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:30.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.381+0000 7f2e80ff9640 1 -- 172.21.15.2:0/1220595214 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f2e740ef860 con 0x7f2e84104980 2024-03-15T12:58:30.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.381+0000 7f2e837fe640 1 --2- 172.21.15.2:0/1220595214 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2e6c0a1a30 0x7f2e6c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:30.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.382+0000 7f2e837fe640 1 --2- 172.21.15.2:0/1220595214 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2e6c0a1a30 0x7f2e6c0a3ef0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009fc0 tx=0x7f2e78005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:30.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.386+0000 7f2e80ff9640 1 -- 172.21.15.2:0/1220595214 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2e740b87c0 con 0x7f2e84104980 2024-03-15T12:58:30.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.607+0000 7f2e89b47640 1 -- 172.21.15.2:0/1220595214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e8410cee0 con 0x7f2e84104980 2024-03-15T12:58:30.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.608+0000 7f2e80ff9640 1 -- 172.21.15.2:0/1220595214 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11453 (secure 0 0 0) 0x7f2e740bbe80 con 0x7f2e84104980 2024-03-15T12:58:30.594 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:30.594 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T12:56:43.433836+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24464,24500,24460,24539,24434]},"id":1}]} 2024-03-15T12:58:30.595 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 28 2024-03-15T12:58:30.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.610+0000 7f2e89b47640 1 -- 172.21.15.2:0/1220595214 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2e6c0a1a30 msgr2=0x7f2e6c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:30.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.610+0000 7f2e89b47640 1 --2- 172.21.15.2:0/1220595214 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2e6c0a1a30 0x7f2e6c0a3ef0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009fc0 tx=0x7f2e78005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:30.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.610+0000 7f2e89b47640 1 -- 172.21.15.2:0/1220595214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e84104980 msgr2=0x7f2e84197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:30.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.610+0000 7f2e89b47640 1 --2- 172.21.15.2:0/1220595214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e84104980 0x7f2e84197b30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2e7402c0b0 tx=0x7f2e74032460 comp rx=0 tx=0).stop 2024-03-15T12:58:30.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.610+0000 7f2e89b47640 1 -- 172.21.15.2:0/1220595214 shutdown_connections 2024-03-15T12:58:30.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.610+0000 7f2e89b47640 1 --2- 172.21.15.2:0/1220595214 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f2e6c0a1a30 0x7f2e6c0a3ef0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.610+0000 7f2e89b47640 1 --2- 172.21.15.2:0/1220595214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e841072b0 0x7f2e8419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.610+0000 7f2e89b47640 1 --2- 172.21.15.2:0/1220595214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e84104980 0x7f2e84197b30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.610+0000 7f2e89b47640 1 --2- 172.21.15.2:0/1220595214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e84102050 0x7f2e841975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.610+0000 7f2e89b47640 1 -- 172.21.15.2:0/1220595214 >> 172.21.15.2:0/1220595214 conn(0x7f2e840fb980 msgr2=0x7f2e841028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:30.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.610+0000 7f2e89b47640 1 -- 172.21.15.2:0/1220595214 shutdown_connections 2024-03-15T12:58:30.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.610+0000 7f2e89b47640 1 -- 172.21.15.2:0/1220595214 wait complete. 2024-03-15T12:58:30.607 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:58:30.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.724+0000 7ff1ce15c640 1 -- 172.21.15.2:0/3258230545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1c8102010 msgr2=0x7ff1c8104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:30.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.724+0000 7ff1ce15c640 1 --2- 172.21.15.2:0/3258230545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1c8102010 0x7ff1c8104400 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009900 tx=0x7ff1b802bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:30.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.724+0000 7ff1ce15c640 1 -- 172.21.15.2:0/3258230545 shutdown_connections 2024-03-15T12:58:30.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.724+0000 7ff1ce15c640 1 --2- 172.21.15.2:0/3258230545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1c8107270 0x7ff1c81096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.724+0000 7ff1ce15c640 1 --2- 172.21.15.2:0/3258230545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1c8104940 0x7ff1c8106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.724+0000 7ff1ce15c640 1 --2- 172.21.15.2:0/3258230545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1c8102010 0x7ff1c8104400 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.724+0000 7ff1ce15c640 1 -- 172.21.15.2:0/3258230545 >> 172.21.15.2:0/3258230545 conn(0x7ff1c80fba20 msgr2=0x7ff1c80fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:30.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.724+0000 7ff1ce15c640 1 -- 172.21.15.2:0/3258230545 shutdown_connections 2024-03-15T12:58:30.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.724+0000 7ff1ce15c640 1 -- 172.21.15.2:0/3258230545 wait complete. 2024-03-15T12:58:30.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1ce15c640 1 Processor -- start 2024-03-15T12:58:30.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1ce15c640 1 -- start start 2024-03-15T12:58:30.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1ce15c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1c8102010 0x7ff1c81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:30.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1ce15c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1c8104940 0x7ff1c8199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:30.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1ce15c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1c8107270 0x7ff1c81a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:30.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1ce15c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c81a12b0 con 0x7ff1c8102010 2024-03-15T12:58:30.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1ce15c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c81a1420 con 0x7ff1c8104940 2024-03-15T12:58:30.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1ce15c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c81a15e0 con 0x7ff1c8107270 2024-03-15T12:58:30.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1c77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1c8102010 0x7ff1c81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:30.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1c77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1c8102010 0x7ff1c81997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50258/0 (socket says 172.21.15.2:50258) 2024-03-15T12:58:30.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1c77fe640 1 -- 172.21.15.2:0/920766191 learned_addr learned my addr 172.21.15.2:0/920766191 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:30.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1c6ffd640 1 --2- 172.21.15.2:0/920766191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1c8104940 0x7ff1c8199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:30.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1c7fff640 1 --2- 172.21.15.2:0/920766191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1c8107270 0x7ff1c81a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:30.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1c77fe640 1 -- 172.21.15.2:0/920766191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1c8107270 msgr2=0x7ff1c81a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:30.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1c77fe640 1 --2- 172.21.15.2:0/920766191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1c8107270 0x7ff1c81a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1c77fe640 1 -- 172.21.15.2:0/920766191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1c8104940 msgr2=0x7ff1c8199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:30.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1c77fe640 1 --2- 172.21.15.2:0/920766191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1c8104940 0x7ff1c8199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1c77fe640 1 -- 172.21.15.2:0/920766191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8009560 con 0x7ff1c8102010 2024-03-15T12:58:30.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.725+0000 7ff1c77fe640 1 --2- 172.21.15.2:0/920766191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1c8102010 0x7ff1c81997c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8005ab0 tx=0x7ff1b8004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:30.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.726+0000 7ff1c4ff9640 1 -- 172.21.15.2:0/920766191 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8004250 con 0x7ff1c8102010 2024-03-15T12:58:30.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.726+0000 7ff1ce15c640 1 -- 172.21.15.2:0/920766191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c81a1860 con 0x7ff1c8102010 2024-03-15T12:58:30.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.726+0000 7ff1c4ff9640 1 -- 172.21.15.2:0/920766191 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff1b80043b0 con 0x7ff1c8102010 2024-03-15T12:58:30.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.726+0000 7ff1c4ff9640 1 -- 172.21.15.2:0/920766191 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8003d20 con 0x7ff1c8102010 2024-03-15T12:58:30.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.726+0000 7ff1ce15c640 1 -- 172.21.15.2:0/920766191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c81a1d70 con 0x7ff1c8102010 2024-03-15T12:58:30.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.727+0000 7ff1ce15c640 1 -- 172.21.15.2:0/920766191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c8108dc0 con 0x7ff1c8102010 2024-03-15T12:58:30.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.728+0000 7ff1c4ff9640 1 -- 172.21.15.2:0/920766191 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7ff1b8005000 con 0x7ff1c8102010 2024-03-15T12:58:30.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.728+0000 7ff1c4ff9640 1 --2- 172.21.15.2:0/920766191 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff1b00a1a30 0x7ff1b00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:30.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.728+0000 7ff1c6ffd640 1 --2- 172.21.15.2:0/920766191 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff1b00a1a30 0x7ff1b00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:30.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.728+0000 7ff1c4ff9640 1 -- 172.21.15.2:0/920766191 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7ff1b80ef940 con 0x7ff1c8102010 2024-03-15T12:58:30.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.729+0000 7ff1c6ffd640 1 --2- 172.21.15.2:0/920766191 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff1b00a1a30 0x7ff1b00a3ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff1c819acd0 tx=0x7ff1bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:30.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.733+0000 7ff1c4ff9640 1 -- 172.21.15.2:0/920766191 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff1b80b8810 con 0x7ff1c8102010 2024-03-15T12:58:30.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.918+0000 7ff1ce15c640 1 -- 172.21.15.2:0/920766191 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1c8108830 con 0x7ff1c8102010 2024-03-15T12:58:30.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.919+0000 7ff1c4ff9640 1 -- 172.21.15.2:0/920766191 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24526 (secure 0 0 0) 0x7ff1b80bbed0 con 0x7ff1c8102010 2024-03-15T12:58:30.906 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:30.906 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":91,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:57:19.826817+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:58:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.922+0000 7ff1ce15c640 1 -- 172.21.15.2:0/920766191 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff1b00a1a30 msgr2=0x7ff1b00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.922+0000 7ff1ce15c640 1 --2- 172.21.15.2:0/920766191 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff1b00a1a30 0x7ff1b00a3ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff1c819acd0 tx=0x7ff1bc005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.922+0000 7ff1ce15c640 1 -- 172.21.15.2:0/920766191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1c8102010 msgr2=0x7ff1c81997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.922+0000 7ff1ce15c640 1 --2- 172.21.15.2:0/920766191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1c8102010 0x7ff1c81997c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8005ab0 tx=0x7ff1b8004a30 comp rx=0 tx=0).stop 2024-03-15T12:58:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.922+0000 7ff1ce15c640 1 -- 172.21.15.2:0/920766191 shutdown_connections 2024-03-15T12:58:30.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.922+0000 7ff1ce15c640 1 --2- 172.21.15.2:0/920766191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1c8107270 0x7ff1c81a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.922+0000 7ff1ce15c640 1 --2- 172.21.15.2:0/920766191 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff1b00a1a30 0x7ff1b00a3ef0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.922+0000 7ff1ce15c640 1 --2- 172.21.15.2:0/920766191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1c8104940 0x7ff1c8199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.922+0000 7ff1ce15c640 1 --2- 172.21.15.2:0/920766191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1c8102010 0x7ff1c81997c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:30.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.922+0000 7ff1ce15c640 1 -- 172.21.15.2:0/920766191 >> 172.21.15.2:0/920766191 conn(0x7ff1c80fba20 msgr2=0x7ff1c8101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:30.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.922+0000 7ff1ce15c640 1 -- 172.21.15.2:0/920766191 shutdown_connections 2024-03-15T12:58:30.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:30.922+0000 7ff1ce15c640 1 -- 172.21.15.2:0/920766191 wait complete. 2024-03-15T12:58:30.918 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:30 smithi002 ceph-mon[32060]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:58:30.918 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:30 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2557336793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:30.918 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:30 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3264273531' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:30.918 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:30 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1220595214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:30.920 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-15T12:56:20.559317+0000', 'modified': '2024-03-15T12:56:43.433836+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24539, 'mds_1': 24500, 'mds_2': 24464, 'mds_3': 24460, 'mds_4': 24434}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14640': {'gid': 14640, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.2:6839/2513297233', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6838', 'nonce': 2513297233}, {'type': 'v1', 'addr': '172.21.15.2:6839', 'nonce': 2513297233}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14652': {'gid': 14652, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.2:6841/3747539896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6840', 'nonce': 3747539896}, {'type': 'v1', 'addr': '172.21.15.2:6841', 'nonce': 3747539896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24406': {'gid': 24406, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6833/730881833', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6832', 'nonce': 730881833}, {'type': 'v1', 'addr': '172.21.15.184:6833', 'nonce': 730881833}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24434': {'gid': 24434, 'name': 'k', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.149:6833/2587208478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6832', 'nonce': 2587208478}, {'type': 'v1', 'addr': '172.21.15.149:6833', 'nonce': 2587208478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.184:6835/1255256953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 1255256953}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 1255256953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.149:6835/694548319', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 694548319}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 694548319}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6837/3698954880', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6836', 'nonce': 3698954880}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 3698954880}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6839/1170793322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6838', 'nonce': 1170793322}, {'type': 'v1', 'addr': '172.21.15.184:6839', 'nonce': 1170793322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.149:6837/1813851428', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 1813851428}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 1813851428}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24539': {'gid': 24539, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.149:6839/1715135128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 1715135128}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 1715135128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24539, 'qdb_cluster': [24464, 24500, 24460, 24539, 24434]} 2024-03-15T12:58:30.920 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-15T12:58:30.920 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-03-15T12:58:31.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.037+0000 7f21966e1640 1 -- 172.21.15.2:0/2773583216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21901030b0 msgr2=0x7f2190109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.037+0000 7f21966e1640 1 --2- 172.21.15.2:0/2773583216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21901030b0 0x7f2190109940 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2188009ca0 tx=0x7f218802c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.037+0000 7f21966e1640 1 -- 172.21.15.2:0/2773583216 shutdown_connections 2024-03-15T12:58:31.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.037+0000 7f21966e1640 1 --2- 172.21.15.2:0/2773583216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21901030b0 0x7f2190109940 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.037+0000 7f21966e1640 1 --2- 172.21.15.2:0/2773583216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21901026f0 0x7f2190102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.037+0000 7f21966e1640 1 --2- 172.21.15.2:0/2773583216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2190101500 0x7f2190101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.037+0000 7f21966e1640 1 -- 172.21.15.2:0/2773583216 >> 172.21.15.2:0/2773583216 conn(0x7f21900fca90 msgr2=0x7f21900feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.037+0000 7f21966e1640 1 -- 172.21.15.2:0/2773583216 shutdown_connections 2024-03-15T12:58:31.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.037+0000 7f21966e1640 1 -- 172.21.15.2:0/2773583216 wait complete. 2024-03-15T12:58:31.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.037+0000 7f21966e1640 1 Processor -- start 2024-03-15T12:58:31.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f21966e1640 1 -- start start 2024-03-15T12:58:31.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f21966e1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2190101500 0x7f21900752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f21966e1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21901026f0 0x7f21900738e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f21966e1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21901030b0 0x7f2190073e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f21966e1640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2190074580 con 0x7f21901030b0 2024-03-15T12:58:31.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f21966e1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21900746f0 con 0x7f2190101500 2024-03-15T12:58:31.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f21966e1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21901a15e0 con 0x7f21901026f0 2024-03-15T12:58:31.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f2194c7e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21901030b0 0x7f2190073e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f218f7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21901026f0 0x7f21900738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f2194c7e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21901030b0 0x7f2190073e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50274/0 (socket says 172.21.15.2:50274) 2024-03-15T12:58:31.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f2194c7e640 1 -- 172.21.15.2:0/2633537815 learned_addr learned my addr 172.21.15.2:0/2633537815 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f218ffff640 1 --2- 172.21.15.2:0/2633537815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2190101500 0x7f21900752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f2194c7e640 1 -- 172.21.15.2:0/2633537815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21901026f0 msgr2=0x7f21900738e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f2194c7e640 1 --2- 172.21.15.2:0/2633537815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21901026f0 0x7f21900738e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f2194c7e640 1 -- 172.21.15.2:0/2633537815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2190101500 msgr2=0x7f21900752a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f2194c7e640 1 --2- 172.21.15.2:0/2633537815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2190101500 0x7f21900752a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f2194c7e640 1 -- 172.21.15.2:0/2633537815 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2184009490 con 0x7f21901030b0 2024-03-15T12:58:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f218f7fe640 1 --2- 172.21.15.2:0/2633537815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21901026f0 0x7f21900738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.038+0000 7f2194c7e640 1 --2- 172.21.15.2:0/2633537815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21901030b0 0x7f2190073e20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f218802c630 tx=0x7f2188032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.039+0000 7f218d7fa640 1 -- 172.21.15.2:0/2633537815 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188034c90 con 0x7f21901030b0 2024-03-15T12:58:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.039+0000 7f218d7fa640 1 -- 172.21.15.2:0/2633537815 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2188003e40 con 0x7f21901030b0 2024-03-15T12:58:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.039+0000 7f21966e1640 1 -- 172.21.15.2:0/2633537815 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2188009900 con 0x7f21901030b0 2024-03-15T12:58:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.039+0000 7f218d7fa640 1 -- 172.21.15.2:0/2633537815 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21880048a0 con 0x7f21901030b0 2024-03-15T12:58:31.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.039+0000 7f21966e1640 1 -- 172.21.15.2:0/2633537815 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21901a1b50 con 0x7f21901030b0 2024-03-15T12:58:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.041+0000 7f218d7fa640 1 -- 172.21.15.2:0/2633537815 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f218800a040 con 0x7f21901030b0 2024-03-15T12:58:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.041+0000 7f2199395740 1 Processor -- start 2024-03-15T12:58:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.041+0000 7f2199395740 1 -- start start 2024-03-15T12:58:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.041+0000 7f218d7fa640 1 --2- 172.21.15.2:0/2633537815 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f21780a1a30 0x7f21780a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.041+0000 7f2199395740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa097f80 0x5634aa0983a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.041+0000 7f2199395740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa098970 0x5634aa09d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.041+0000 7f2199395740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 0x5634aa09ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.041+0000 7f2199395740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634aa053cd0 con 0x5634aa098970 2024-03-15T12:58:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.041+0000 7f2199395740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634aa006f20 con 0x5634aa097f80 2024-03-15T12:58:31.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.041+0000 7f2199395740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634aa004450 con 0x5634aa09dae0 2024-03-15T12:58:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.041+0000 7f218d7fa640 1 -- 172.21.15.2:0/2633537815 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f21880f0480 con 0x7f21901030b0 2024-03-15T12:58:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.041+0000 7f218f7fe640 1 --2- 172.21.15.2:0/2633537815 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f21780a1a30 0x7f21780a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f218ffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa097f80 0x5634aa0983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f218ffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa097f80 0x5634aa0983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55834/0 (socket says 172.21.15.2:55834) 2024-03-15T12:58:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f218f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa098970 0x5634aa09d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f218ffff640 1 -- 172.21.15.2:0/4156967089 learned_addr learned my addr 172.21.15.2:0/4156967089 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f2194c7e640 1 --2- 172.21.15.2:0/4156967089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 0x5634aa09ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f218ffff640 1 -- 172.21.15.2:0/4156967089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 msgr2=0x5634aa09ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f218ffff640 1 --2- 172.21.15.2:0/4156967089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 0x5634aa09ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f218ffff640 1 -- 172.21.15.2:0/4156967089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa098970 msgr2=0x5634aa09d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f218ffff640 1 --2- 172.21.15.2:0/4156967089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa098970 0x5634aa09d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f218ffff640 1 -- 172.21.15.2:0/4156967089 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634a9fb4f20 con 0x5634aa097f80 2024-03-15T12:58:31.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f2194c7e640 1 --2- 172.21.15.2:0/4156967089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 0x5634aa09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:58:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f218f7fe640 1 --2- 172.21.15.2:0/2633537815 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f21780a1a30 0x7f21780a3ef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2190075010 tx=0x7f2184011510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f218ffff640 1 --2- 172.21.15.2:0/4156967089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa097f80 0x5634aa0983a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2180009ca0 tx=0x7f2180011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e15fc1cf25deac26 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f21767fc640 1 -- 172.21.15.2:0/4156967089 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180009e50 con 0x5634aa097f80 2024-03-15T12:58:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f21767fc640 1 -- 172.21.15.2:0/4156967089 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f21800035f0 con 0x5634aa097f80 2024-03-15T12:58:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.042+0000 7f21767fc640 1 -- 172.21.15.2:0/4156967089 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180019670 con 0x5634aa097f80 2024-03-15T12:58:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.043+0000 7f2199395740 1 -- 172.21.15.2:0/4156967089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa097f80 msgr2=0x5634aa0983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.043+0000 7f2199395740 1 --2- 172.21.15.2:0/4156967089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa097f80 0x5634aa0983a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2180009ca0 tx=0x7f2180011810 comp rx=0 tx=0).stop 2024-03-15T12:58:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.043+0000 7f2199395740 1 -- 172.21.15.2:0/4156967089 shutdown_connections 2024-03-15T12:58:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.043+0000 7f2199395740 1 --2- 172.21.15.2:0/4156967089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 0x5634aa09ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.043+0000 7f2199395740 1 --2- 172.21.15.2:0/4156967089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa098970 0x5634aa09d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.043+0000 7f2199395740 1 --2- 172.21.15.2:0/4156967089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa097f80 0x5634aa0983a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.043+0000 7f2199395740 1 -- 172.21.15.2:0/4156967089 >> 172.21.15.2:0/4156967089 conn(0x5634aa0934f0 msgr2=0x5634aa0938f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.043+0000 7f2199395740 1 -- 172.21.15.2:0/4156967089 shutdown_connections 2024-03-15T12:58:31.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.043+0000 7f2199395740 1 -- 172.21.15.2:0/4156967089 wait complete. 2024-03-15T12:58:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.043+0000 7f2199395740 1 -- start start 2024-03-15T12:58:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f2199395740 1 Processor -- start 2024-03-15T12:58:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f2199395740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa097f80 0x5634aa0a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f2199395740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa098970 0x5634aa0a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f2199395740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa09dae0 0x5634aa0a9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f2199395740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a9fffc30 con 0x5634aa097f80 2024-03-15T12:58:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f2199395740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a9fa4d30 con 0x5634aa09dae0 2024-03-15T12:58:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f2199395740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a9cd3310 con 0x5634aa098970 2024-03-15T12:58:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f2199395740 10 client.0 _ll_register_callbacks cb 0x5634a9facd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f218ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa097f80 0x5634aa0a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f218ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa097f80 0x5634aa0a21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50292/0 (socket says 172.21.15.2:50292) 2024-03-15T12:58:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f218ffff640 1 -- 172.21.15.2:0/2896433852 learned_addr learned my addr 172.21.15.2:0/2896433852 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f2194c7e640 1 --2- 172.21.15.2:0/2896433852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa09dae0 0x5634aa0a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f218f7fe640 1 --2- 172.21.15.2:0/2896433852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa098970 0x5634aa0a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f218ffff640 1 -- 172.21.15.2:0/2896433852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa098970 msgr2=0x5634aa0a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f218ffff640 1 --2- 172.21.15.2:0/2896433852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa098970 0x5634aa0a26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f218ffff640 1 -- 172.21.15.2:0/2896433852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa09dae0 msgr2=0x5634aa0a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f218ffff640 1 --2- 172.21.15.2:0/2896433852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa09dae0 0x5634aa0a9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f218ffff640 1 -- 172.21.15.2:0/2896433852 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21800098d0 con 0x5634aa097f80 2024-03-15T12:58:31.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f218ffff640 1 --2- 172.21.15.2:0/2896433852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa097f80 0x5634aa0a21b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2180009710 tx=0x7f21800042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83267184956c4d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f216f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:31.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f216f7fe640 1 -- 172.21.15.2:0/2896433852 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180009e50 con 0x5634aa097f80 2024-03-15T12:58:31.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f216f7fe640 1 -- 172.21.15.2:0/2896433852 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f21800044c0 con 0x5634aa097f80 2024-03-15T12:58:31.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.044+0000 7f2199395740 1 -- 172.21.15.2:0/2896433852 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634a9cff6c0 con 0x5634aa097f80 2024-03-15T12:58:31.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.045+0000 7f216f7fe640 1 -- 172.21.15.2:0/2896433852 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2180005250 con 0x5634aa097f80 2024-03-15T12:58:31.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.045+0000 7f2199395740 10 client.15450 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:31.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.045+0000 7f2199395740 1 -- 172.21.15.2:0/2896433852 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5634a9ffe2b0 con 0x5634aa097f80 2024-03-15T12:58:31.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.045+0000 7f216f7fe640 1 -- 172.21.15.2:0/2896433852 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5634a9cff6c0 con 0x5634aa097f80 2024-03-15T12:58:31.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.045+0000 7f216f7fe640 1 -- 172.21.15.2:0/2896433852 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2180021070 con 0x5634aa097f80 2024-03-15T12:58:31.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.045+0000 7f2199395740 10 client.15450 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:31.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.045+0000 7f2199395740 10 client.15450 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-15T12:58:31.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.045+0000 7f2199395740 20 client.15450 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:31.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.045+0000 7f2199395740 1 --2- 172.21.15.2:0/2896433852 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5634aa0aea20 0x5634aa0adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.045+0000 7f2199395740 4 client.15450 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:58:31.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.045+0000 7f2199395740 1 -- 172.21.15.2:0/2896433852 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634a9d50270 con 0x5634aa0aea20 2024-03-15T12:58:31.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.045+0000 7f218f7fe640 1 --2- 172.21.15.2:0/2896433852 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5634aa0aea20 0x5634aa0adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.046+0000 7f216f7fe640 10 client.15450 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T12:58:31.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.046+0000 7f218f7fe640 1 --2- 172.21.15.2:0/2896433852 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5634aa0aea20 0x5634aa0adca0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cd5c39b70e34b709 server_cookie=ef18d74ed3613cd in_seq=0 out_seq=0 2024-03-15T12:58:31.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.047+0000 7f216f7fe640 1 -- 172.21.15.2:0/2896433852 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5634a9d50270 con 0x5634aa0aea20 2024-03-15T12:58:31.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.047+0000 7f216f7fe640 10 client.15450 handle_command_reply: tid=0 2024-03-15T12:58:31.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.047+0000 7f2199395740 1 client.15450 shutdown 2024-03-15T12:58:31.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.047+0000 7f2199395740 1 -- 172.21.15.2:0/2896433852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa097f80 msgr2=0x5634aa0a21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.047+0000 7f2199395740 1 --2- 172.21.15.2:0/2896433852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa097f80 0x5634aa0a21b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2180009710 tx=0x7f21800042d0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.047+0000 7f2199395740 1 -- 172.21.15.2:0/2896433852 shutdown_connections 2024-03-15T12:58:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.047+0000 7f2199395740 1 --2- 172.21.15.2:0/2896433852 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5634aa0aea20 0x5634aa0adca0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.047+0000 7f2199395740 1 --2- 172.21.15.2:0/2896433852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa09dae0 0x5634aa0a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.047+0000 7f2199395740 1 --2- 172.21.15.2:0/2896433852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa098970 0x5634aa0a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.047+0000 7f2199395740 1 --2- 172.21.15.2:0/2896433852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa097f80 0x5634aa0a21b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.047+0000 7f2199395740 1 -- 172.21.15.2:0/2896433852 >> 172.21.15.2:0/2896433852 conn(0x5634aa0934f0 msgr2=0x5634aa0927d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.047+0000 7f216f7fe640 0 client.15450 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T12:58:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.047+0000 7f2199395740 1 -- 172.21.15.2:0/2896433852 shutdown_connections 2024-03-15T12:58:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.047+0000 7f2199395740 1 -- 172.21.15.2:0/2896433852 wait complete. 2024-03-15T12:58:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.047+0000 7f2199395740 20 client.15450 trim_cache size 0 max 16384 2024-03-15T12:58:31.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f2199395740 1 Processor -- start 2024-03-15T12:58:31.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f2199395740 1 -- start start 2024-03-15T12:58:31.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f2199395740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa097f80 0x5634aa099410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f2199395740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa098970 0x5634aa0a9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f2199395740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 0x5634aa0938d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f2199395740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a9f85490 con 0x5634aa097f80 2024-03-15T12:58:31.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f2199395740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a9fb8f40 con 0x5634aa098970 2024-03-15T12:58:31.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f2199395740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a9ffc890 con 0x5634aa09dae0 2024-03-15T12:58:31.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f218ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa097f80 0x5634aa099410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f218ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa097f80 0x5634aa099410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50298/0 (socket says 172.21.15.2:50298) 2024-03-15T12:58:31.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f218ffff640 1 -- 172.21.15.2:0/2475730843 learned_addr learned my addr 172.21.15.2:0/2475730843 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f218f7fe640 1 --2- 172.21.15.2:0/2475730843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa098970 0x5634aa0a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f2194c7e640 1 --2- 172.21.15.2:0/2475730843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 0x5634aa0938d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f218ffff640 1 -- 172.21.15.2:0/2475730843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 msgr2=0x5634aa0938d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f218ffff640 1 --2- 172.21.15.2:0/2475730843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 0x5634aa0938d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f218ffff640 1 -- 172.21.15.2:0/2475730843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa098970 msgr2=0x5634aa0a9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f218ffff640 1 --2- 172.21.15.2:0/2475730843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa098970 0x5634aa0a9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f218ffff640 1 -- 172.21.15.2:0/2475730843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634a9f9aa20 con 0x5634aa097f80 2024-03-15T12:58:31.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f218ffff640 1 --2- 172.21.15.2:0/2475730843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa097f80 0x5634aa099410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f21800059f0 tx=0x7f2180009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6737487c201419f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.064+0000 7f2175ffb640 1 -- 172.21.15.2:0/2475730843 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180009af0 con 0x5634aa097f80 2024-03-15T12:58:31.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.065+0000 7f2175ffb640 1 -- 172.21.15.2:0/2475730843 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2180023ec0 con 0x5634aa097f80 2024-03-15T12:58:31.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.065+0000 7f2175ffb640 1 -- 172.21.15.2:0/2475730843 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180005540 con 0x5634aa097f80 2024-03-15T12:58:31.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.065+0000 7f2199395740 1 -- 172.21.15.2:0/2475730843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa097f80 msgr2=0x5634aa099410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.065+0000 7f2199395740 1 --2- 172.21.15.2:0/2475730843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa097f80 0x5634aa099410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f21800059f0 tx=0x7f2180009860 comp rx=0 tx=0).stop 2024-03-15T12:58:31.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.065+0000 7f2199395740 1 -- 172.21.15.2:0/2475730843 shutdown_connections 2024-03-15T12:58:31.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.065+0000 7f2199395740 1 --2- 172.21.15.2:0/2475730843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 0x5634aa0938d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.065+0000 7f2199395740 1 --2- 172.21.15.2:0/2475730843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa098970 0x5634aa0a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.065+0000 7f2199395740 1 --2- 172.21.15.2:0/2475730843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa097f80 0x5634aa099410 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.065+0000 7f2199395740 1 -- 172.21.15.2:0/2475730843 >> 172.21.15.2:0/2475730843 conn(0x5634aa0934f0 msgr2=0x5634aa0a0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.065+0000 7f2199395740 1 -- 172.21.15.2:0/2475730843 shutdown_connections 2024-03-15T12:58:31.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.065+0000 7f2199395740 1 -- 172.21.15.2:0/2475730843 wait complete. 2024-03-15T12:58:31.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.065+0000 7f2199395740 1 -- start start 2024-03-15T12:58:31.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f2199395740 1 Processor -- start 2024-03-15T12:58:31.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f2199395740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa097f80 0x5634aa0a7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f2199395740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa098970 0x5634aa0a7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f2199395740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 0x5634aa0a8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f2199395740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634aa0a8b50 con 0x5634aa098970 2024-03-15T12:58:31.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f2199395740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634aa0a8cc0 con 0x5634aa097f80 2024-03-15T12:58:31.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f2199395740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634aa0a8e80 con 0x5634aa09dae0 2024-03-15T12:58:31.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f2199395740 10 client.0 _ll_register_callbacks cb 0x7f21840119e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:31.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f218f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa098970 0x5634aa0a7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f218f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa098970 0x5634aa0a7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50310/0 (socket says 172.21.15.2:50310) 2024-03-15T12:58:31.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f218f7fe640 1 -- 172.21.15.2:0/8501822 learned_addr learned my addr 172.21.15.2:0/8501822 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f218ffff640 1 --2- 172.21.15.2:0/8501822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa097f80 0x5634aa0a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f2194c7e640 1 --2- 172.21.15.2:0/8501822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 0x5634aa0a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f2194c7e640 1 -- 172.21.15.2:0/8501822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa097f80 msgr2=0x5634aa0a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f2194c7e640 1 --2- 172.21.15.2:0/8501822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa097f80 0x5634aa0a7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f2194c7e640 1 -- 172.21.15.2:0/8501822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa098970 msgr2=0x5634aa0a7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f2194c7e640 1 --2- 172.21.15.2:0/8501822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa098970 0x5634aa0a7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f2194c7e640 1 -- 172.21.15.2:0/8501822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634a9ffe2b0 con 0x5634aa09dae0 2024-03-15T12:58:31.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f218f7fe640 1 --2- 172.21.15.2:0/8501822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa098970 0x5634aa0a7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:31.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f2194c7e640 1 --2- 172.21.15.2:0/8501822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 0x5634aa0a8450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2188033000 tx=0x7f21880bd870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fa646e3bc599936 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.066+0000 7f216f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T12:58:31.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.067+0000 7f216f7fe640 1 -- 172.21.15.2:0/8501822 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21880c5850 con 0x5634aa09dae0 2024-03-15T12:58:31.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.067+0000 7f2199395740 1 -- 172.21.15.2:0/8501822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634aa0ae730 con 0x5634aa09dae0 2024-03-15T12:58:31.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.067+0000 7f216f7fe640 1 -- 172.21.15.2:0/8501822 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f21880f01e0 con 0x5634aa09dae0 2024-03-15T12:58:31.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.067+0000 7f216f7fe640 1 -- 172.21.15.2:0/8501822 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2188004040 con 0x5634aa09dae0 2024-03-15T12:58:31.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.067+0000 7f2199395740 10 client.25258 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:31.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.067+0000 7f2199395740 1 -- 172.21.15.2:0/8501822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5634a9fc0780 con 0x5634aa09dae0 2024-03-15T12:58:31.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.067+0000 7f216f7fe640 1 -- 172.21.15.2:0/8501822 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5634aa0ae730 con 0x5634aa09dae0 2024-03-15T12:58:31.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.067+0000 7f216f7fe640 1 -- 172.21.15.2:0/8501822 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f21880c54e0 con 0x5634aa09dae0 2024-03-15T12:58:31.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.067+0000 7f2199395740 10 client.25258 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:31.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.067+0000 7f2199395740 10 client.25258 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-15T12:58:31.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.067+0000 7f2199395740 20 client.25258 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:31.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.067+0000 7f2199395740 1 --2- 172.21.15.2:0/8501822 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5634aa10dbd0 0x5634aa10a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.067+0000 7f2199395740 4 client.25258 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-15T12:58:31.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.067+0000 7f2199395740 1 -- 172.21.15.2:0/8501822 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f21800035f0 con 0x5634aa10dbd0 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.067+0000 7f218ffff640 1 --2- 172.21.15.2:0/8501822 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5634aa10dbd0 0x5634aa10a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f216f7fe640 10 client.25258 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f218ffff640 1 --2- 172.21.15.2:0/8501822 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5634aa10dbd0 0x5634aa10a050 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=967cfaadf469e634 server_cookie=b014f632a00dc3e7 in_seq=0 out_seq=0 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f216f7fe640 1 -- 172.21.15.2:0/8501822 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7f21800035f0 con 0x5634aa10dbd0 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f216f7fe640 10 client.25258 handle_command_reply: tid=0 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f2199395740 1 client.25258 shutdown 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f2199395740 1 -- 172.21.15.2:0/8501822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 msgr2=0x5634aa0a8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f2199395740 1 --2- 172.21.15.2:0/8501822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 0x5634aa0a8450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2188033000 tx=0x7f21880bd870 comp rx=0 tx=0).stop 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f2199395740 1 -- 172.21.15.2:0/8501822 shutdown_connections 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f2199395740 1 --2- 172.21.15.2:0/8501822 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5634aa10dbd0 0x5634aa10a050 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f2199395740 1 --2- 172.21.15.2:0/8501822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634aa09dae0 0x5634aa0a8450 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f2199395740 1 --2- 172.21.15.2:0/8501822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5634aa098970 0x5634aa0a7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f2199395740 1 --2- 172.21.15.2:0/8501822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634aa097f80 0x5634aa0a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f216f7fe640 0 client.25258 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f2199395740 1 -- 172.21.15.2:0/8501822 >> 172.21.15.2:0/8501822 conn(0x5634aa0934f0 msgr2=0x5634aa0a0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f2199395740 1 -- 172.21.15.2:0/8501822 shutdown_connections 2024-03-15T12:58:31.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f2199395740 1 -- 172.21.15.2:0/8501822 wait complete. 2024-03-15T12:58:31.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.068+0000 7f2199395740 20 client.25258 trim_cache size 0 max 16384 2024-03-15T12:58:31.061 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:58:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "2767718a-e2ca-11ee-95c9-87774f69a715", 2024-03-15T12:58:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 4, 2024-03-15T12:58:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24434, 2024-03-15T12:58:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-03-15T12:58:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-03-15T12:58:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-03-15T12:58:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 108.65021206199999, 2024-03-15T12:58:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 28, 2024-03-15T12:58:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 91, 2024-03-15T12:58:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 89, 2024-03-15T12:58:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 165.31935727999999 2024-03-15T12:58:31.062 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:58:31.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.069+0000 7f21966e1640 1 -- 172.21.15.2:0/2633537815 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f21780a1a30 msgr2=0x7f21780a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.069+0000 7f21966e1640 1 --2- 172.21.15.2:0/2633537815 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f21780a1a30 0x7f21780a3ef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2190075010 tx=0x7f2184011510 comp rx=0 tx=0).stop 2024-03-15T12:58:31.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.069+0000 7f21966e1640 1 -- 172.21.15.2:0/2633537815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21901030b0 msgr2=0x7f2190073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.069+0000 7f21966e1640 1 --2- 172.21.15.2:0/2633537815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21901030b0 0x7f2190073e20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f218802c630 tx=0x7f2188032a10 comp rx=0 tx=0).stop 2024-03-15T12:58:31.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.069+0000 7f21966e1640 1 -- 172.21.15.2:0/2633537815 shutdown_connections 2024-03-15T12:58:31.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.069+0000 7f21966e1640 1 --2- 172.21.15.2:0/2633537815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21901030b0 0x7f2190073e20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.069+0000 7f21966e1640 1 --2- 172.21.15.2:0/2633537815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21901026f0 0x7f21900738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.069+0000 7f21966e1640 1 --2- 172.21.15.2:0/2633537815 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f21780a1a30 0x7f21780a3ef0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.069+0000 7f21966e1640 1 --2- 172.21.15.2:0/2633537815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2190101500 0x7f21900752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.069+0000 7f21966e1640 1 -- 172.21.15.2:0/2633537815 >> 172.21.15.2:0/2633537815 conn(0x7f21900fca90 msgr2=0x7f21901059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.069+0000 7f21966e1640 1 -- 172.21.15.2:0/2633537815 shutdown_connections 2024-03-15T12:58:31.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.069+0000 7f21966e1640 1 -- 172.21.15.2:0/2633537815 wait complete. 2024-03-15T12:58:31.064 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:30 smithi184 ceph-mon[35993]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:58:31.064 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:30 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2557336793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:31.064 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:30 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3264273531' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:31.064 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:30 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1220595214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:31.065 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-15T12:58:31.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:30 smithi149 ceph-mon[35927]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-15T12:58:31.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:30 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2557336793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:31.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:30 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3264273531' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:31.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:30 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1220595214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:31.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.183+0000 7faf7a2f4640 1 -- 172.21.15.2:0/3107303932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf74107250 msgr2=0x7faf74109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.183+0000 7faf7a2f4640 1 --2- 172.21.15.2:0/3107303932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf74107250 0x7faf74109640 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009ca0 tx=0x7faf6c02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.183+0000 7faf7a2f4640 1 -- 172.21.15.2:0/3107303932 shutdown_connections 2024-03-15T12:58:31.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.183+0000 7faf7a2f4640 1 --2- 172.21.15.2:0/3107303932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf74107250 0x7faf74109640 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.183+0000 7faf7a2f4640 1 --2- 172.21.15.2:0/3107303932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf74104920 0x7faf74106d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.183+0000 7faf7a2f4640 1 --2- 172.21.15.2:0/3107303932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf74101ff0 0x7faf741043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.183+0000 7faf7a2f4640 1 -- 172.21.15.2:0/3107303932 >> 172.21.15.2:0/3107303932 conn(0x7faf740fb9a0 msgr2=0x7faf740fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.183+0000 7faf7a2f4640 1 -- 172.21.15.2:0/3107303932 shutdown_connections 2024-03-15T12:58:31.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.183+0000 7faf7a2f4640 1 -- 172.21.15.2:0/3107303932 wait complete. 2024-03-15T12:58:31.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.184+0000 7faf7a2f4640 1 Processor -- start 2024-03-15T12:58:31.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.184+0000 7faf7a2f4640 1 -- start start 2024-03-15T12:58:31.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.184+0000 7faf7a2f4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf74101ff0 0x7faf74101960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.184+0000 7faf7a2f4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf74104920 0x7faf740fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.184+0000 7faf7a2f4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf74107250 0x7faf741004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.184+0000 7faf7a2f4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf74100c40 con 0x7faf74104920 2024-03-15T12:58:31.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.184+0000 7faf7a2f4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf74100db0 con 0x7faf74107250 2024-03-15T12:58:31.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.184+0000 7faf7a2f4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf741a15d0 con 0x7faf74101ff0 2024-03-15T12:58:31.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.184+0000 7faf737fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf74104920 0x7faf740fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.184+0000 7faf737fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf74104920 0x7faf740fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50334/0 (socket says 172.21.15.2:50334) 2024-03-15T12:58:31.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.184+0000 7faf737fe640 1 -- 172.21.15.2:0/1448117662 learned_addr learned my addr 172.21.15.2:0/1448117662 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.184+0000 7faf73fff640 1 --2- 172.21.15.2:0/1448117662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf74101ff0 0x7faf74101960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.184+0000 7faf78891640 1 --2- 172.21.15.2:0/1448117662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf74107250 0x7faf741004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.185+0000 7faf737fe640 1 -- 172.21.15.2:0/1448117662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf74101ff0 msgr2=0x7faf74101960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.185+0000 7faf737fe640 1 --2- 172.21.15.2:0/1448117662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf74101ff0 0x7faf74101960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.185+0000 7faf737fe640 1 -- 172.21.15.2:0/1448117662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf74107250 msgr2=0x7faf741004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.185+0000 7faf737fe640 1 --2- 172.21.15.2:0/1448117662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf74107250 0x7faf741004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.185+0000 7faf737fe640 1 -- 172.21.15.2:0/1448117662 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf64009490 con 0x7faf74104920 2024-03-15T12:58:31.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.185+0000 7faf737fe640 1 --2- 172.21.15.2:0/1448117662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf74104920 0x7faf740fffa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faf6400e9d0 tx=0x7faf6400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.185+0000 7faf717fa640 1 -- 172.21.15.2:0/1448117662 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6400cde0 con 0x7faf74104920 2024-03-15T12:58:31.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.185+0000 7faf717fa640 1 -- 172.21.15.2:0/1448117662 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7faf640035f0 con 0x7faf74104920 2024-03-15T12:58:31.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.185+0000 7faf7a2f4640 1 -- 172.21.15.2:0/1448117662 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6c009900 con 0x7faf74104920 2024-03-15T12:58:31.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.185+0000 7faf717fa640 1 -- 172.21.15.2:0/1448117662 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64010690 con 0x7faf74104920 2024-03-15T12:58:31.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.185+0000 7faf7a2f4640 1 -- 172.21.15.2:0/1448117662 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf741a1c00 con 0x7faf74104920 2024-03-15T12:58:31.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.187+0000 7faf717fa640 1 -- 172.21.15.2:0/1448117662 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7faf64003b40 con 0x7faf74104920 2024-03-15T12:58:31.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.187+0000 7faf7cfaf740 1 Processor -- start 2024-03-15T12:58:31.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.187+0000 7faf7cfaf740 1 -- start start 2024-03-15T12:58:31.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf717fa640 1 --2- 172.21.15.2:0/1448117662 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7faf5c0a1a30 0x7faf5c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf7cfaf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321bf80 0x56445321c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf7cfaf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321c970 0x564453221470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf7cfaf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564453221ae0 0x564453223f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf7cfaf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644531d7cd0 con 0x56445321c970 2024-03-15T12:58:31.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf7cfaf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445318af20 con 0x56445321bf80 2024-03-15T12:58:31.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf7cfaf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564453188450 con 0x564453221ae0 2024-03-15T12:58:31.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf78891640 1 --2- 172.21.15.2:0/1448117662 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7faf5c0a1a30 0x7faf5c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf717fa640 1 -- 172.21.15.2:0/1448117662 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7faf64014070 con 0x7faf74104920 2024-03-15T12:58:31.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf737fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321c970 0x564453221470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf73fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321bf80 0x56445321c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf737fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321c970 0x564453221470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50340/0 (socket says 172.21.15.2:50340) 2024-03-15T12:58:31.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf73fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321bf80 0x56445321c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55878/0 (socket says 172.21.15.2:55878) 2024-03-15T12:58:31.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf737fe640 1 -- 172.21.15.2:0/1651326840 learned_addr learned my addr 172.21.15.2:0/1651326840 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf78891640 1 --2- 172.21.15.2:0/1651326840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564453221ae0 0x564453223f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf737fe640 1 -- 172.21.15.2:0/1651326840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564453221ae0 msgr2=0x564453223f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf737fe640 1 --2- 172.21.15.2:0/1651326840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564453221ae0 0x564453223f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf737fe640 1 -- 172.21.15.2:0/1651326840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321bf80 msgr2=0x56445321c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf737fe640 1 --2- 172.21.15.2:0/1651326840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321bf80 0x56445321c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf737fe640 1 -- 172.21.15.2:0/1651326840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564453138f20 con 0x56445321c970 2024-03-15T12:58:31.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf73fff640 1 --2- 172.21.15.2:0/1651326840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321bf80 0x56445321c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:31.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf78891640 1 --2- 172.21.15.2:0/1448117662 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7faf5c0a1a30 0x7faf5c0a3ef0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faf6c005cc0 tx=0x7faf6c032fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf737fe640 1 --2- 172.21.15.2:0/1651326840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321c970 0x564453221470 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7faf6409a770 tx=0x7faf64003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b2240ea1f38a4d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.188+0000 7faf5a7fc640 1 -- 172.21.15.2:0/1651326840 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6409e930 con 0x56445321c970 2024-03-15T12:58:31.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.189+0000 7faf5a7fc640 1 -- 172.21.15.2:0/1651326840 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7faf640a2f80 con 0x56445321c970 2024-03-15T12:58:31.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.189+0000 7faf5a7fc640 1 -- 172.21.15.2:0/1651326840 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf640038f0 con 0x56445321c970 2024-03-15T12:58:31.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.189+0000 7faf7cfaf740 1 -- 172.21.15.2:0/1651326840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321c970 msgr2=0x564453221470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.189+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/1651326840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321c970 0x564453221470 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7faf6409a770 tx=0x7faf64003e40 comp rx=0 tx=0).stop 2024-03-15T12:58:31.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.189+0000 7faf7cfaf740 1 -- 172.21.15.2:0/1651326840 shutdown_connections 2024-03-15T12:58:31.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.189+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/1651326840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564453221ae0 0x564453223f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.189+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/1651326840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321c970 0x564453221470 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.189+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/1651326840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321bf80 0x56445321c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.189+0000 7faf7cfaf740 1 -- 172.21.15.2:0/1651326840 >> 172.21.15.2:0/1651326840 conn(0x5644532174f0 msgr2=0x5644532178f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.189+0000 7faf7cfaf740 1 -- 172.21.15.2:0/1651326840 shutdown_connections 2024-03-15T12:58:31.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.189+0000 7faf7cfaf740 1 -- 172.21.15.2:0/1651326840 wait complete. 2024-03-15T12:58:31.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf7cfaf740 1 -- start start 2024-03-15T12:58:31.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf7cfaf740 1 Processor -- start 2024-03-15T12:58:31.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf7cfaf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321bf80 0x564453226200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf7cfaf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56445321c970 0x564453226740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf7cfaf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564453221ae0 0x56445322d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf7cfaf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564453183c30 con 0x564453221ae0 2024-03-15T12:58:31.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf7cfaf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564453128d30 con 0x56445321bf80 2024-03-15T12:58:31.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf7cfaf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564452e57310 con 0x56445321c970 2024-03-15T12:58:31.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf7cfaf740 10 client.0 _ll_register_callbacks cb 0x564453130d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:31.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf78891640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564453221ae0 0x56445322d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf78891640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564453221ae0 0x56445322d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50342/0 (socket says 172.21.15.2:50342) 2024-03-15T12:58:31.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf73fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321bf80 0x564453226200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf78891640 1 -- 172.21.15.2:0/974687315 learned_addr learned my addr 172.21.15.2:0/974687315 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf737fe640 1 --2- 172.21.15.2:0/974687315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56445321c970 0x564453226740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf78891640 1 -- 172.21.15.2:0/974687315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56445321c970 msgr2=0x564453226740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf78891640 1 --2- 172.21.15.2:0/974687315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56445321c970 0x564453226740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf78891640 1 -- 172.21.15.2:0/974687315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321bf80 msgr2=0x564453226200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf78891640 1 --2- 172.21.15.2:0/974687315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321bf80 0x564453226200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.190+0000 7faf78891640 1 -- 172.21.15.2:0/974687315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf741a1c00 con 0x564453221ae0 2024-03-15T12:58:31.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf78891640 1 --2- 172.21.15.2:0/974687315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564453221ae0 0x56445322d780 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009ca0 tx=0x7faf6c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a88bfa9a5a5c25 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf537fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:31.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf537fe640 1 -- 172.21.15.2:0/974687315 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c003d80 con 0x564453221ae0 2024-03-15T12:58:31.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf537fe640 1 -- 172.21.15.2:0/974687315 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7faf6c004040 con 0x564453221ae0 2024-03-15T12:58:31.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf7cfaf740 1 -- 172.21.15.2:0/974687315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564452e836c0 con 0x564453221ae0 2024-03-15T12:58:31.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf537fe640 1 -- 172.21.15.2:0/974687315 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf6c004200 con 0x564453221ae0 2024-03-15T12:58:31.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf7cfaf740 10 client.15468 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:31.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf7cfaf740 1 -- 172.21.15.2:0/974687315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5644531822b0 con 0x564453221ae0 2024-03-15T12:58:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf537fe640 1 -- 172.21.15.2:0/974687315 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x564452e836c0 con 0x564453221ae0 2024-03-15T12:58:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf537fe640 1 -- 172.21.15.2:0/974687315 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7faf6c04c070 con 0x564453221ae0 2024-03-15T12:58:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf7cfaf740 10 client.15468 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf7cfaf740 10 client.15468 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-15T12:58:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf7cfaf740 20 client.15468 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/974687315 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x564453232b80 0x564453231e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf7cfaf740 4 client.15468 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:58:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.191+0000 7faf7cfaf740 1 -- 172.21.15.2:0/974687315 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564452ed4270 con 0x564453232b80 2024-03-15T12:58:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.192+0000 7faf73fff640 1 --2- 172.21.15.2:0/974687315 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x564453232b80 0x564453231e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.192+0000 7faf73fff640 1 --2- 172.21.15.2:0/974687315 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x564453232b80 0x564453231e00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=80ac2cddccbca00e server_cookie=36b991db47483762 in_seq=0 out_seq=0 2024-03-15T12:58:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.192+0000 7faf537fe640 10 client.15468 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T12:58:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.193+0000 7faf537fe640 1 -- 172.21.15.2:0/974687315 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564452ed4270 con 0x564453232b80 2024-03-15T12:58:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.193+0000 7faf537fe640 10 client.15468 handle_command_reply: tid=0 2024-03-15T12:58:31.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.193+0000 7faf7cfaf740 1 client.15468 shutdown 2024-03-15T12:58:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.193+0000 7faf7cfaf740 1 -- 172.21.15.2:0/974687315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564453221ae0 msgr2=0x56445322d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.193+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/974687315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564453221ae0 0x56445322d780 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009ca0 tx=0x7faf6c02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.193+0000 7faf7cfaf740 1 -- 172.21.15.2:0/974687315 shutdown_connections 2024-03-15T12:58:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.193+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/974687315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564453221ae0 0x56445322d780 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.193+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/974687315 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x564453232b80 0x564453231e00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.193+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/974687315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56445321c970 0x564453226740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.193+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/974687315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321bf80 0x564453226200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.193+0000 7faf537fe640 0 client.15468 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T12:58:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.193+0000 7faf7cfaf740 1 -- 172.21.15.2:0/974687315 >> 172.21.15.2:0/974687315 conn(0x5644532174f0 msgr2=0x5644532167d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.193+0000 7faf7cfaf740 1 -- 172.21.15.2:0/974687315 shutdown_connections 2024-03-15T12:58:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.193+0000 7faf7cfaf740 1 -- 172.21.15.2:0/974687315 wait complete. 2024-03-15T12:58:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.193+0000 7faf7cfaf740 20 client.15468 trim_cache size 0 max 16384 2024-03-15T12:58:31.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf7cfaf740 1 Processor -- start 2024-03-15T12:58:31.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf7cfaf740 1 -- start start 2024-03-15T12:58:31.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf7cfaf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321bf80 0x5644532178d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf7cfaf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56445321c970 0x56445322d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf7cfaf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564453221ae0 0x564453215f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf7cfaf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564453109490 con 0x56445321bf80 2024-03-15T12:58:31.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf7cfaf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445321bb40 con 0x564453221ae0 2024-03-15T12:58:31.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf7cfaf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564453230a00 con 0x56445321c970 2024-03-15T12:58:31.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf73fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321bf80 0x5644532178d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf73fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321bf80 0x5644532178d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50352/0 (socket says 172.21.15.2:50352) 2024-03-15T12:58:31.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf73fff640 1 -- 172.21.15.2:0/2006569038 learned_addr learned my addr 172.21.15.2:0/2006569038 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf737fe640 1 --2- 172.21.15.2:0/2006569038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56445321c970 0x56445322d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf78891640 1 --2- 172.21.15.2:0/2006569038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564453221ae0 0x564453215f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf73fff640 1 -- 172.21.15.2:0/2006569038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56445321c970 msgr2=0x56445322d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf73fff640 1 --2- 172.21.15.2:0/2006569038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56445321c970 0x56445322d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf73fff640 1 -- 172.21.15.2:0/2006569038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564453221ae0 msgr2=0x564453215f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf73fff640 1 --2- 172.21.15.2:0/2006569038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564453221ae0 0x564453215f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf73fff640 1 -- 172.21.15.2:0/2006569038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56445311ea20 con 0x56445321bf80 2024-03-15T12:58:31.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf73fff640 1 --2- 172.21.15.2:0/2006569038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321bf80 0x5644532178d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faf68009d10 tx=0x7faf68005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2d3798d485ca24 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.210+0000 7faf59ffb640 1 -- 172.21.15.2:0/2006569038 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf68017040 con 0x56445321bf80 2024-03-15T12:58:31.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.211+0000 7faf59ffb640 1 -- 172.21.15.2:0/2006569038 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7faf68018040 con 0x56445321bf80 2024-03-15T12:58:31.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.211+0000 7faf59ffb640 1 -- 172.21.15.2:0/2006569038 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf68005020 con 0x56445321bf80 2024-03-15T12:58:31.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.211+0000 7faf7cfaf740 1 -- 172.21.15.2:0/2006569038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321bf80 msgr2=0x5644532178d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.211+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/2006569038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321bf80 0x5644532178d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faf68009d10 tx=0x7faf68005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:31.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.211+0000 7faf7cfaf740 1 -- 172.21.15.2:0/2006569038 shutdown_connections 2024-03-15T12:58:31.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.211+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/2006569038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564453221ae0 0x564453215f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.211+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/2006569038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56445321c970 0x56445322d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.211+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/2006569038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321bf80 0x5644532178d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.211+0000 7faf7cfaf740 1 -- 172.21.15.2:0/2006569038 >> 172.21.15.2:0/2006569038 conn(0x5644532174f0 msgr2=0x564453224ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.211+0000 7faf7cfaf740 1 -- 172.21.15.2:0/2006569038 shutdown_connections 2024-03-15T12:58:31.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.211+0000 7faf7cfaf740 1 -- 172.21.15.2:0/2006569038 wait complete. 2024-03-15T12:58:31.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.211+0000 7faf7cfaf740 1 -- start start 2024-03-15T12:58:31.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf7cfaf740 1 Processor -- start 2024-03-15T12:58:31.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf7cfaf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321bf80 0x56445322a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf7cfaf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321c970 0x56445322a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf7cfaf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564453221ae0 0x56445328dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf7cfaf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445322d8b0 con 0x56445321bf80 2024-03-15T12:58:31.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf7cfaf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445322da20 con 0x56445321c970 2024-03-15T12:58:31.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf7cfaf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445322dbe0 con 0x564453221ae0 2024-03-15T12:58:31.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf7cfaf740 10 client.0 _ll_register_callbacks cb 0x7faf64005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:31.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf73fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321bf80 0x56445322a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf73fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321bf80 0x56445322a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50356/0 (socket says 172.21.15.2:50356) 2024-03-15T12:58:31.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf73fff640 1 -- 172.21.15.2:0/2572152631 learned_addr learned my addr 172.21.15.2:0/2572152631 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf737fe640 1 --2- 172.21.15.2:0/2572152631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321c970 0x56445322a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf78891640 1 --2- 172.21.15.2:0/2572152631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564453221ae0 0x56445328dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf73fff640 1 -- 172.21.15.2:0/2572152631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564453221ae0 msgr2=0x56445328dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf73fff640 1 --2- 172.21.15.2:0/2572152631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564453221ae0 0x56445328dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf73fff640 1 -- 172.21.15.2:0/2572152631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321c970 msgr2=0x56445322a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf73fff640 1 --2- 172.21.15.2:0/2572152631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321c970 0x56445322a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf73fff640 1 -- 172.21.15.2:0/2572152631 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf68020770 con 0x56445321bf80 2024-03-15T12:58:31.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf73fff640 1 --2- 172.21.15.2:0/2572152631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321bf80 0x56445322a410 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faf68004620 tx=0x7faf68012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c2e953afbfd78c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.212+0000 7faf537fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:31.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.213+0000 7faf537fe640 1 -- 172.21.15.2:0/2572152631 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf68020350 con 0x56445321bf80 2024-03-15T12:58:31.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.213+0000 7faf537fe640 1 -- 172.21.15.2:0/2572152631 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7faf68005040 con 0x56445321bf80 2024-03-15T12:58:31.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.213+0000 7faf7cfaf740 1 -- 172.21.15.2:0/2572152631 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564453232890 con 0x56445321bf80 2024-03-15T12:58:31.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.213+0000 7faf537fe640 1 -- 172.21.15.2:0/2572152631 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf68011a20 con 0x56445321bf80 2024-03-15T12:58:31.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.213+0000 7faf7cfaf740 10 client.15474 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:31.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.213+0000 7faf7cfaf740 1 -- 172.21.15.2:0/2572152631 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564453144780 con 0x56445321bf80 2024-03-15T12:58:31.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.213+0000 7faf537fe640 1 -- 172.21.15.2:0/2572152631 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x564453232890 con 0x56445321bf80 2024-03-15T12:58:31.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.213+0000 7faf537fe640 1 -- 172.21.15.2:0/2572152631 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7faf6802a440 con 0x56445321bf80 2024-03-15T12:58:31.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.213+0000 7faf7cfaf740 10 client.15474 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:31.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.213+0000 7faf7cfaf740 10 client.15474 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-15T12:58:31.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.213+0000 7faf7cfaf740 20 client.15474 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:31.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.213+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/2572152631 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x564453293230 0x56445328f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.213+0000 7faf7cfaf740 4 client.15474 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-15T12:58:31.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.213+0000 7faf7cfaf740 1 -- 172.21.15.2:0/2572152631 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7faf640a2f80 con 0x564453293230 2024-03-15T12:58:31.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.213+0000 7faf737fe640 1 --2- 172.21.15.2:0/2572152631 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x564453293230 0x56445328f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf537fe640 10 client.15474 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T12:58:31.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf737fe640 1 --2- 172.21.15.2:0/2572152631 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x564453293230 0x56445328f6b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f85118b3472dd67 server_cookie=6f45fe7f7b7889bd in_seq=0 out_seq=0 2024-03-15T12:58:31.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf537fe640 1 -- 172.21.15.2:0/2572152631 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7faf640a2f80 con 0x564453293230 2024-03-15T12:58:31.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf537fe640 10 client.15474 handle_command_reply: tid=0 2024-03-15T12:58:31.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf7cfaf740 1 client.15474 shutdown 2024-03-15T12:58:31.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf7cfaf740 1 -- 172.21.15.2:0/2572152631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321bf80 msgr2=0x56445322a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/2572152631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321bf80 0x56445322a410 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faf68004620 tx=0x7faf68012ea0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf7cfaf740 1 -- 172.21.15.2:0/2572152631 shutdown_connections 2024-03-15T12:58:31.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/2572152631 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x564453293230 0x56445328f6b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/2572152631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564453221ae0 0x56445328dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/2572152631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56445321c970 0x56445322a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf537fe640 0 client.15474 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T12:58:31.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf7cfaf740 1 --2- 172.21.15.2:0/2572152631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56445321bf80 0x56445322a410 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf7cfaf740 1 -- 172.21.15.2:0/2572152631 >> 172.21.15.2:0/2572152631 conn(0x5644532174f0 msgr2=0x5644532178d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf7cfaf740 1 -- 172.21.15.2:0/2572152631 shutdown_connections 2024-03-15T12:58:31.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf7cfaf740 1 -- 172.21.15.2:0/2572152631 wait complete. 2024-03-15T12:58:31.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.214+0000 7faf7cfaf740 20 client.15474 trim_cache size 0 max 16384 2024-03-15T12:58:31.207 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:58:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "2767718a-e2ca-11ee-95c9-87774f69a715", 2024-03-15T12:58:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 3, 2024-03-15T12:58:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24460, 2024-03-15T12:58:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-03-15T12:58:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-03-15T12:58:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-03-15T12:58:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 109.800710302, 2024-03-15T12:58:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 28, 2024-03-15T12:58:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 91, 2024-03-15T12:58:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 89, 2024-03-15T12:58:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 154.08694071799999 2024-03-15T12:58:31.208 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:58:31.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.215+0000 7faf7a2f4640 1 -- 172.21.15.2:0/1448117662 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7faf5c0a1a30 msgr2=0x7faf5c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.215+0000 7faf7a2f4640 1 --2- 172.21.15.2:0/1448117662 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7faf5c0a1a30 0x7faf5c0a3ef0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faf6c005cc0 tx=0x7faf6c032fb0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.215+0000 7faf7a2f4640 1 -- 172.21.15.2:0/1448117662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf74104920 msgr2=0x7faf740fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.215+0000 7faf7a2f4640 1 --2- 172.21.15.2:0/1448117662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf74104920 0x7faf740fffa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faf6400e9d0 tx=0x7faf6400ee90 comp rx=0 tx=0).stop 2024-03-15T12:58:31.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.215+0000 7faf7a2f4640 1 -- 172.21.15.2:0/1448117662 shutdown_connections 2024-03-15T12:58:31.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.215+0000 7faf7a2f4640 1 --2- 172.21.15.2:0/1448117662 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7faf5c0a1a30 0x7faf5c0a3ef0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.215+0000 7faf7a2f4640 1 --2- 172.21.15.2:0/1448117662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf74107250 0x7faf741004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.215+0000 7faf7a2f4640 1 --2- 172.21.15.2:0/1448117662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf74104920 0x7faf740fffa0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.215+0000 7faf7a2f4640 1 --2- 172.21.15.2:0/1448117662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf74101ff0 0x7faf74101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.215+0000 7faf7a2f4640 1 -- 172.21.15.2:0/1448117662 >> 172.21.15.2:0/1448117662 conn(0x7faf740fb9a0 msgr2=0x7faf74102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.215+0000 7faf7a2f4640 1 -- 172.21.15.2:0/1448117662 shutdown_connections 2024-03-15T12:58:31.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.215+0000 7faf7a2f4640 1 -- 172.21.15.2:0/1448117662 wait complete. 2024-03-15T12:58:31.211 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-15T12:58:31.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.329+0000 7ff28c2c3640 1 -- 172.21.15.2:0/765127714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2841003f0 msgr2=0x7ff284100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.329+0000 7ff28c2c3640 1 --2- 172.21.15.2:0/765127714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2841003f0 0x7ff284100810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009ca0 tx=0x7ff27c02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28c2c3640 1 -- 172.21.15.2:0/765127714 shutdown_connections 2024-03-15T12:58:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28c2c3640 1 --2- 172.21.15.2:0/765127714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff284101fa0 0x7ff284108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28c2c3640 1 --2- 172.21.15.2:0/765127714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2841015e0 0x7ff284101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28c2c3640 1 --2- 172.21.15.2:0/765127714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2841003f0 0x7ff284100810 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28c2c3640 1 -- 172.21.15.2:0/765127714 >> 172.21.15.2:0/765127714 conn(0x7ff2840fb980 msgr2=0x7ff2840fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28c2c3640 1 -- 172.21.15.2:0/765127714 shutdown_connections 2024-03-15T12:58:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28c2c3640 1 -- 172.21.15.2:0/765127714 wait complete. 2024-03-15T12:58:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28c2c3640 1 Processor -- start 2024-03-15T12:58:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28c2c3640 1 -- start start 2024-03-15T12:58:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28c2c3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2841003f0 0x7ff2840691e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28c2c3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2841015e0 0x7ff28419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28c2c3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff284101fa0 0x7ff28419e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28c2c3640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28419f150 con 0x7ff2841015e0 2024-03-15T12:58:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28c2c3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28419f2c0 con 0x7ff284101fa0 2024-03-15T12:58:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28c2c3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28419f480 con 0x7ff2841003f0 2024-03-15T12:58:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28985e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2841015e0 0x7ff28419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28985e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2841015e0 0x7ff28419c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50368/0 (socket says 172.21.15.2:50368) 2024-03-15T12:58:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28a05f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2841003f0 0x7ff2840691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28985e640 1 -- 172.21.15.2:0/3847071203 learned_addr learned my addr 172.21.15.2:0/3847071203 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.330+0000 7ff28a860640 1 --2- 172.21.15.2:0/3847071203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff284101fa0 0x7ff28419e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.331+0000 7ff28a05f640 1 -- 172.21.15.2:0/3847071203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff284101fa0 msgr2=0x7ff28419e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.331+0000 7ff28a05f640 1 --2- 172.21.15.2:0/3847071203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff284101fa0 0x7ff28419e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.331+0000 7ff28a05f640 1 -- 172.21.15.2:0/3847071203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2841015e0 msgr2=0x7ff28419c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.331+0000 7ff28a05f640 1 --2- 172.21.15.2:0/3847071203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2841015e0 0x7ff28419c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.331+0000 7ff28a05f640 1 -- 172.21.15.2:0/3847071203 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff274009490 con 0x7ff2841003f0 2024-03-15T12:58:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.331+0000 7ff28985e640 1 --2- 172.21.15.2:0/3847071203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2841015e0 0x7ff28419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.331+0000 7ff28a05f640 1 --2- 172.21.15.2:0/3847071203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2841003f0 0x7ff2840691e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff27c037040 tx=0x7ff27c032990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.331+0000 7ff2837fe640 1 -- 172.21.15.2:0/3847071203 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c003970 con 0x7ff2841003f0 2024-03-15T12:58:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.331+0000 7ff28c2c3640 1 -- 172.21.15.2:0/3847071203 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27c009900 con 0x7ff2841003f0 2024-03-15T12:58:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.331+0000 7ff2837fe640 1 -- 172.21.15.2:0/3847071203 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff27c004040 con 0x7ff2841003f0 2024-03-15T12:58:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.331+0000 7ff2837fe640 1 -- 172.21.15.2:0/3847071203 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c0406b0 con 0x7ff2841003f0 2024-03-15T12:58:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.331+0000 7ff28c2c3640 1 -- 172.21.15.2:0/3847071203 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28419fa40 con 0x7ff2841003f0 2024-03-15T12:58:31.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.333+0000 7ff2837fe640 1 -- 172.21.15.2:0/3847071203 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7ff27c00a040 con 0x7ff2841003f0 2024-03-15T12:58:31.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff28ef80740 1 Processor -- start 2024-03-15T12:58:31.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff28ef80740 1 -- start start 2024-03-15T12:58:31.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff28ef80740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 0x5597abaad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff28ef80740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597abab2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff28ef80740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abab2ae0 0x5597abab4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff28ef80740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597aba68cd0 con 0x5597abaad970 2024-03-15T12:58:31.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff28ef80740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597aba1bf20 con 0x5597abaacf80 2024-03-15T12:58:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff28ef80740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597aba19450 con 0x5597abab2ae0 2024-03-15T12:58:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff28a05f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597abab2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff28a860640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 0x5597abaad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff28a860640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 0x5597abaad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55942/0 (socket says 172.21.15.2:55942) 2024-03-15T12:58:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff28a05f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597abab2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50384/0 (socket says 172.21.15.2:50384) 2024-03-15T12:58:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff28a860640 1 -- 172.21.15.2:0/3254818958 learned_addr learned my addr 172.21.15.2:0/3254818958 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff2837fe640 1 --2- 172.21.15.2:0/3847071203 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff26c0a1a30 0x7ff26c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff28985e640 1 --2- 172.21.15.2:0/3254818958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abab2ae0 0x5597abab4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.334+0000 7ff28a860640 1 --2- 172.21.15.2:0/3847071203 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff26c0a1a30 0x7ff26c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.335+0000 7ff2837fe640 1 -- 172.21.15.2:0/3847071203 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7ff27c0f0590 con 0x7ff2841003f0 2024-03-15T12:58:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.335+0000 7ff28a860640 1 -- 172.21.15.2:0/3254818958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abab2ae0 msgr2=0x5597abab4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.335+0000 7ff28a860640 1 --2- 172.21.15.2:0/3254818958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abab2ae0 0x5597abab4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.335+0000 7ff28a860640 1 -- 172.21.15.2:0/3254818958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 msgr2=0x5597abab2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.335+0000 7ff28a860640 1 --2- 172.21.15.2:0/3254818958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597abab2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.335+0000 7ff28a860640 1 -- 172.21.15.2:0/3254818958 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597ab9c9f20 con 0x5597abaacf80 2024-03-15T12:58:31.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.335+0000 7ff28a05f640 1 --2- 172.21.15.2:0/3254818958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597abab2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:31.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.335+0000 7ff28985e640 1 --2- 172.21.15.2:0/3254818958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abab2ae0 0x5597abab4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:31.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.335+0000 7ff28a860640 1 --2- 172.21.15.2:0/3254818958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 0x5597abaad3a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff274013970 tx=0x7ff274013e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a018ee1d0925640 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.335+0000 7ff28a860640 1 --2- 172.21.15.2:0/3847071203 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff26c0a1a30 0x7ff26c0a3ef0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff27401d000 tx=0x7ff27401f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.335+0000 7ff280bfe640 1 -- 172.21.15.2:0/3254818958 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2740122d0 con 0x5597abaacf80 2024-03-15T12:58:31.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.335+0000 7ff280bfe640 1 -- 172.21.15.2:0/3254818958 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff2740044e0 con 0x5597abaacf80 2024-03-15T12:58:31.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.335+0000 7ff280bfe640 1 -- 172.21.15.2:0/3254818958 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2740108f0 con 0x5597abaacf80 2024-03-15T12:58:31.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 -- 172.21.15.2:0/3254818958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 msgr2=0x5597abaad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 --2- 172.21.15.2:0/3254818958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 0x5597abaad3a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff274013970 tx=0x7ff274013e30 comp rx=0 tx=0).stop 2024-03-15T12:58:31.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 -- 172.21.15.2:0/3254818958 shutdown_connections 2024-03-15T12:58:31.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 --2- 172.21.15.2:0/3254818958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abab2ae0 0x5597abab4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 --2- 172.21.15.2:0/3254818958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597abab2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 --2- 172.21.15.2:0/3254818958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 0x5597abaad3a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 -- 172.21.15.2:0/3254818958 >> 172.21.15.2:0/3254818958 conn(0x5597abaa84f0 msgr2=0x5597abaa88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 -- 172.21.15.2:0/3254818958 shutdown_connections 2024-03-15T12:58:31.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 -- 172.21.15.2:0/3254818958 wait complete. 2024-03-15T12:58:31.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 -- start start 2024-03-15T12:58:31.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 Processor -- start 2024-03-15T12:58:31.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 0x5597abab7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597abab76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abab2ae0 0x5597ababe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597aba14c30 con 0x5597abaad970 2024-03-15T12:58:31.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ab9b9d30 con 0x5597abaacf80 2024-03-15T12:58:31.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ab6e8310 con 0x5597abab2ae0 2024-03-15T12:58:31.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff28985e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597abab76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.336+0000 7ff28ef80740 10 client.0 _ll_register_callbacks cb 0x5597ab9c1d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:31.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff28a05f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 0x5597abab7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff28985e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597abab76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50390/0 (socket says 172.21.15.2:50390) 2024-03-15T12:58:31.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff28985e640 1 -- 172.21.15.2:0/2388028637 learned_addr learned my addr 172.21.15.2:0/2388028637 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff28a860640 1 --2- 172.21.15.2:0/2388028637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abab2ae0 0x5597ababe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff28985e640 1 -- 172.21.15.2:0/2388028637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abab2ae0 msgr2=0x5597ababe710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff28985e640 1 --2- 172.21.15.2:0/2388028637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abab2ae0 0x5597ababe710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff28985e640 1 -- 172.21.15.2:0/2388028637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 msgr2=0x5597abab7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff28985e640 1 --2- 172.21.15.2:0/2388028637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 0x5597abab7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff28985e640 1 -- 172.21.15.2:0/2388028637 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff274013520 con 0x5597abaad970 2024-03-15T12:58:31.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff28985e640 1 --2- 172.21.15.2:0/2388028637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597abab76d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff278012d20 tx=0x7ff27800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1a61e329b8783d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff269ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:31.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff269ffb640 1 -- 172.21.15.2:0/2388028637 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278017070 con 0x5597abaad970 2024-03-15T12:58:31.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff269ffb640 1 -- 172.21.15.2:0/2388028637 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7ff2780120d0 con 0x5597abaad970 2024-03-15T12:58:31.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff28ef80740 1 -- 172.21.15.2:0/2388028637 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597ab7146c0 con 0x5597abaad970 2024-03-15T12:58:31.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff269ffb640 1 -- 172.21.15.2:0/2388028637 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff278012290 con 0x5597abaad970 2024-03-15T12:58:31.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff28ef80740 10 client.15486 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:31.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff28ef80740 1 -- 172.21.15.2:0/2388028637 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5597aba132b0 con 0x5597abaad970 2024-03-15T12:58:31.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.337+0000 7ff269ffb640 1 -- 172.21.15.2:0/2388028637 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5597ab7146c0 con 0x5597abaad970 2024-03-15T12:58:31.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.338+0000 7ff269ffb640 1 -- 172.21.15.2:0/2388028637 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff278010430 con 0x5597abaad970 2024-03-15T12:58:31.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.338+0000 7ff28ef80740 10 client.15486 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:31.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.338+0000 7ff28ef80740 10 client.15486 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-15T12:58:31.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.338+0000 7ff28ef80740 20 client.15486 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:31.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.338+0000 7ff28ef80740 1 --2- 172.21.15.2:0/2388028637 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5597abac3540 0x5597abac2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.338+0000 7ff28ef80740 4 client.15486 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:58:31.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.338+0000 7ff28ef80740 1 -- 172.21.15.2:0/2388028637 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597ab765270 con 0x5597abac3540 2024-03-15T12:58:31.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.338+0000 7ff28a05f640 1 --2- 172.21.15.2:0/2388028637 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5597abac3540 0x5597abac2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.338+0000 7ff269ffb640 10 client.15486 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T12:58:31.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.338+0000 7ff28a05f640 1 --2- 172.21.15.2:0/2388028637 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5597abac3540 0x5597abac2e40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20c562ab7a8cab57 server_cookie=e9e11890caa6cd4e in_seq=0 out_seq=0 2024-03-15T12:58:31.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.339+0000 7ff269ffb640 1 -- 172.21.15.2:0/2388028637 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5597ab765270 con 0x5597abac3540 2024-03-15T12:58:31.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.339+0000 7ff269ffb640 10 client.15486 handle_command_reply: tid=0 2024-03-15T12:58:31.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.339+0000 7ff28ef80740 1 client.15486 shutdown 2024-03-15T12:58:31.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.339+0000 7ff28ef80740 1 -- 172.21.15.2:0/2388028637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 msgr2=0x5597abab76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.339+0000 7ff28ef80740 1 --2- 172.21.15.2:0/2388028637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597abab76d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff278012d20 tx=0x7ff27800f790 comp rx=0 tx=0).stop 2024-03-15T12:58:31.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.339+0000 7ff28ef80740 1 -- 172.21.15.2:0/2388028637 shutdown_connections 2024-03-15T12:58:31.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.339+0000 7ff28ef80740 1 --2- 172.21.15.2:0/2388028637 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5597abac3540 0x5597abac2e40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.339+0000 7ff28ef80740 1 --2- 172.21.15.2:0/2388028637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abab2ae0 0x5597ababe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.339+0000 7ff269ffb640 0 client.15486 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T12:58:31.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.339+0000 7ff28ef80740 1 --2- 172.21.15.2:0/2388028637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597abab76d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.339+0000 7ff28ef80740 1 --2- 172.21.15.2:0/2388028637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 0x5597abab7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.339+0000 7ff28ef80740 1 -- 172.21.15.2:0/2388028637 >> 172.21.15.2:0/2388028637 conn(0x5597abaa84f0 msgr2=0x5597abaa77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.339+0000 7ff28ef80740 1 -- 172.21.15.2:0/2388028637 shutdown_connections 2024-03-15T12:58:31.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.339+0000 7ff28ef80740 1 -- 172.21.15.2:0/2388028637 wait complete. 2024-03-15T12:58:31.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.339+0000 7ff28ef80740 20 client.15486 trim_cache size 0 max 16384 2024-03-15T12:58:31.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28ef80740 1 Processor -- start 2024-03-15T12:58:31.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28ef80740 1 -- start start 2024-03-15T12:58:31.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28ef80740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abaacf80 0x5597abab7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28ef80740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597ababe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28ef80740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abab2ae0 0x5597abaa88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28ef80740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ab99a490 con 0x5597abaad970 2024-03-15T12:58:31.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28ef80740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ababf390 con 0x5597abab2ae0 2024-03-15T12:58:31.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28ef80740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597abaaea80 con 0x5597abaacf80 2024-03-15T12:58:31.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28985e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597ababe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28985e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597ababe6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50402/0 (socket says 172.21.15.2:50402) 2024-03-15T12:58:31.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28985e640 1 -- 172.21.15.2:0/868732469 learned_addr learned my addr 172.21.15.2:0/868732469 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28a05f640 1 --2- 172.21.15.2:0/868732469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abaacf80 0x5597abab7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28a860640 1 --2- 172.21.15.2:0/868732469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abab2ae0 0x5597abaa88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28985e640 1 -- 172.21.15.2:0/868732469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abaacf80 msgr2=0x5597abab7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28985e640 1 --2- 172.21.15.2:0/868732469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abaacf80 0x5597abab7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28985e640 1 -- 172.21.15.2:0/868732469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abab2ae0 msgr2=0x5597abaa88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28985e640 1 --2- 172.21.15.2:0/868732469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abab2ae0 0x5597abaa88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28985e640 1 -- 172.21.15.2:0/868732469 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597ab9afa20 con 0x5597abaad970 2024-03-15T12:58:31.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28a860640 1 --2- 172.21.15.2:0/868732469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abab2ae0 0x5597abaa88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:58:31.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.356+0000 7ff28985e640 1 --2- 172.21.15.2:0/868732469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597ababe6d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff278012cf0 tx=0x7ff278012990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab51a94c4f70cd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.357+0000 7ff26bfff640 1 -- 172.21.15.2:0/868732469 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278017040 con 0x5597abaad970 2024-03-15T12:58:31.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.357+0000 7ff26bfff640 1 -- 172.21.15.2:0/868732469 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff278005690 con 0x5597abaad970 2024-03-15T12:58:31.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.357+0000 7ff26bfff640 1 -- 172.21.15.2:0/868732469 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278003d10 con 0x5597abaad970 2024-03-15T12:58:31.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.357+0000 7ff28ef80740 1 -- 172.21.15.2:0/868732469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 msgr2=0x5597ababe6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.357+0000 7ff28ef80740 1 --2- 172.21.15.2:0/868732469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597ababe6d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff278012cf0 tx=0x7ff278012990 comp rx=0 tx=0).stop 2024-03-15T12:58:31.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.357+0000 7ff28ef80740 1 -- 172.21.15.2:0/868732469 shutdown_connections 2024-03-15T12:58:31.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.357+0000 7ff28ef80740 1 --2- 172.21.15.2:0/868732469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abab2ae0 0x5597abaa88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.357+0000 7ff28ef80740 1 --2- 172.21.15.2:0/868732469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abaad970 0x5597ababe6d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.357+0000 7ff28ef80740 1 --2- 172.21.15.2:0/868732469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abaacf80 0x5597abab7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.357+0000 7ff28ef80740 1 -- 172.21.15.2:0/868732469 >> 172.21.15.2:0/868732469 conn(0x5597abaa84f0 msgr2=0x5597abaae370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.357+0000 7ff28ef80740 1 -- 172.21.15.2:0/868732469 shutdown_connections 2024-03-15T12:58:31.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.357+0000 7ff28ef80740 1 -- 172.21.15.2:0/868732469 wait complete. 2024-03-15T12:58:31.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.357+0000 7ff28ef80740 1 -- start start 2024-03-15T12:58:31.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.357+0000 7ff28ef80740 1 Processor -- start 2024-03-15T12:58:31.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28ef80740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 0x5597ababb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28ef80740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abaad970 0x5597ababb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28ef80740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abab2ae0 0x5597abb1ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28ef80740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597abab69e0 con 0x5597abab2ae0 2024-03-15T12:58:31.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28ef80740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597abab8180 con 0x5597abaacf80 2024-03-15T12:58:31.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28ef80740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597ababe910 con 0x5597abaad970 2024-03-15T12:58:31.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28ef80740 10 client.0 _ll_register_callbacks cb 0x7ff278009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:31.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28a860640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abab2ae0 0x5597abb1ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28a860640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abab2ae0 0x5597abb1ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50406/0 (socket says 172.21.15.2:50406) 2024-03-15T12:58:31.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28a860640 1 -- 172.21.15.2:0/1350382968 learned_addr learned my addr 172.21.15.2:0/1350382968 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28985e640 1 --2- 172.21.15.2:0/1350382968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abaad970 0x5597ababb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28a05f640 1 --2- 172.21.15.2:0/1350382968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 0x5597ababb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28a860640 1 -- 172.21.15.2:0/1350382968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abaad970 msgr2=0x5597ababb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28a860640 1 --2- 172.21.15.2:0/1350382968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abaad970 0x5597ababb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28a860640 1 -- 172.21.15.2:0/1350382968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 msgr2=0x5597ababb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28a860640 1 --2- 172.21.15.2:0/1350382968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 0x5597ababb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28a860640 1 -- 172.21.15.2:0/1350382968 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597aba132b0 con 0x5597abab2ae0 2024-03-15T12:58:31.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28a860640 1 --2- 172.21.15.2:0/1350382968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abab2ae0 0x5597abb1ebe0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff274009720 tx=0x7ff274005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f344dde28f76d6b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff269ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:31.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff269ffb640 1 -- 172.21.15.2:0/1350382968 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27402c070 con 0x5597abab2ae0 2024-03-15T12:58:31.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff28ef80740 1 -- 172.21.15.2:0/1350382968 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597abac4550 con 0x5597abab2ae0 2024-03-15T12:58:31.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.358+0000 7ff269ffb640 1 -- 172.21.15.2:0/1350382968 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7ff274004880 con 0x5597abab2ae0 2024-03-15T12:58:31.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.359+0000 7ff269ffb640 1 -- 172.21.15.2:0/1350382968 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff274005670 con 0x5597abab2ae0 2024-03-15T12:58:31.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.359+0000 7ff28ef80740 10 client.15492 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:31.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.359+0000 7ff28ef80740 1 -- 172.21.15.2:0/1350382968 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5597ab9d5780 con 0x5597abab2ae0 2024-03-15T12:58:31.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.359+0000 7ff269ffb640 1 -- 172.21.15.2:0/1350382968 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5597abac4550 con 0x5597abab2ae0 2024-03-15T12:58:31.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.359+0000 7ff269ffb640 1 -- 172.21.15.2:0/1350382968 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff274003ad0 con 0x5597abab2ae0 2024-03-15T12:58:31.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.359+0000 7ff28ef80740 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:31.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.359+0000 7ff28ef80740 10 client.15492 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-15T12:58:31.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.359+0000 7ff28ef80740 20 client.15492 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:31.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.359+0000 7ff28ef80740 1 --2- 172.21.15.2:0/1350382968 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5597abb240c0 0x5597abb204f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.359+0000 7ff28ef80740 4 client.15492 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-15T12:58:31.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.359+0000 7ff28ef80740 1 -- 172.21.15.2:0/1350382968 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5597abab2890 con 0x5597abb240c0 2024-03-15T12:58:31.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.359+0000 7ff28a05f640 1 --2- 172.21.15.2:0/1350382968 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5597abb240c0 0x5597abb204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.359+0000 7ff269ffb640 10 client.15492 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T12:58:31.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.359+0000 7ff28a05f640 1 --2- 172.21.15.2:0/1350382968 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5597abb240c0 0x5597abb204f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=584d482012409a6c server_cookie=7b7ea77f02542ea1 in_seq=0 out_seq=0 2024-03-15T12:58:31.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff269ffb640 1 -- 172.21.15.2:0/1350382968 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x5597abab2890 con 0x5597abb240c0 2024-03-15T12:58:31.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff269ffb640 10 client.15492 handle_command_reply: tid=0 2024-03-15T12:58:31.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28ef80740 1 client.15492 shutdown 2024-03-15T12:58:31.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28ef80740 1 -- 172.21.15.2:0/1350382968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abab2ae0 msgr2=0x5597abb1ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28ef80740 1 --2- 172.21.15.2:0/1350382968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abab2ae0 0x5597abb1ebe0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff274009720 tx=0x7ff274005270 comp rx=0 tx=0).stop 2024-03-15T12:58:31.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28ef80740 1 -- 172.21.15.2:0/1350382968 shutdown_connections 2024-03-15T12:58:31.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28ef80740 1 --2- 172.21.15.2:0/1350382968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597abab2ae0 0x5597abb1ebe0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28ef80740 1 --2- 172.21.15.2:0/1350382968 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5597abb240c0 0x5597abb204f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28ef80740 1 --2- 172.21.15.2:0/1350382968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597abaad970 0x5597ababb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28ef80740 1 --2- 172.21.15.2:0/1350382968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597abaacf80 0x5597ababb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff269ffb640 0 client.15492 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T12:58:31.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28ef80740 1 -- 172.21.15.2:0/1350382968 >> 172.21.15.2:0/1350382968 conn(0x5597abaa84f0 msgr2=0x5597abaa88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28ef80740 1 -- 172.21.15.2:0/1350382968 shutdown_connections 2024-03-15T12:58:31.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28ef80740 1 -- 172.21.15.2:0/1350382968 wait complete. 2024-03-15T12:58:31.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28ef80740 20 client.15492 trim_cache size 0 max 16384 2024-03-15T12:58:31.353 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:58:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "2767718a-e2ca-11ee-95c9-87774f69a715", 2024-03-15T12:58:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 2, 2024-03-15T12:58:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24464, 2024-03-15T12:58:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-03-15T12:58:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-03-15T12:58:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-03-15T12:58:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 110.960053601, 2024-03-15T12:58:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 28, 2024-03-15T12:58:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 91, 2024-03-15T12:58:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 89, 2024-03-15T12:58:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 157.056690272 2024-03-15T12:58:31.354 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:58:31.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28c2c3640 1 -- 172.21.15.2:0/3847071203 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff26c0a1a30 msgr2=0x7ff26c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28c2c3640 1 --2- 172.21.15.2:0/3847071203 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff26c0a1a30 0x7ff26c0a3ef0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff27401d000 tx=0x7ff27401f000 comp rx=0 tx=0).stop 2024-03-15T12:58:31.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28c2c3640 1 -- 172.21.15.2:0/3847071203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2841003f0 msgr2=0x7ff2840691e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28c2c3640 1 --2- 172.21.15.2:0/3847071203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2841003f0 0x7ff2840691e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff27c037040 tx=0x7ff27c032990 comp rx=0 tx=0).stop 2024-03-15T12:58:31.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28c2c3640 1 -- 172.21.15.2:0/3847071203 shutdown_connections 2024-03-15T12:58:31.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28c2c3640 1 --2- 172.21.15.2:0/3847071203 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff26c0a1a30 0x7ff26c0a3ef0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28c2c3640 1 --2- 172.21.15.2:0/3847071203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff284101fa0 0x7ff28419e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28c2c3640 1 --2- 172.21.15.2:0/3847071203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2841015e0 0x7ff28419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28c2c3640 1 --2- 172.21.15.2:0/3847071203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2841003f0 0x7ff2840691e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.360+0000 7ff28c2c3640 1 -- 172.21.15.2:0/3847071203 >> 172.21.15.2:0/3847071203 conn(0x7ff2840fb980 msgr2=0x7ff284104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.361+0000 7ff28c2c3640 1 -- 172.21.15.2:0/3847071203 shutdown_connections 2024-03-15T12:58:31.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.361+0000 7ff28c2c3640 1 -- 172.21.15.2:0/3847071203 wait complete. 2024-03-15T12:58:31.357 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-15T12:58:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.474+0000 7f76ad956640 1 -- 172.21.15.2:0/1713088789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a81015e0 msgr2=0x7f76a8101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.474+0000 7f76ad956640 1 --2- 172.21.15.2:0/1713088789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a81015e0 0x7f76a8101a60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7690009900 tx=0x7f769002bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.475+0000 7f76ad956640 1 -- 172.21.15.2:0/1713088789 shutdown_connections 2024-03-15T12:58:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.475+0000 7f76ad956640 1 --2- 172.21.15.2:0/1713088789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a8101fa0 0x7f76a8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.475+0000 7f76ad956640 1 --2- 172.21.15.2:0/1713088789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a81015e0 0x7f76a8101a60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.475+0000 7f76ad956640 1 --2- 172.21.15.2:0/1713088789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76a81003f0 0x7f76a8100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.475+0000 7f76ad956640 1 -- 172.21.15.2:0/1713088789 >> 172.21.15.2:0/1713088789 conn(0x7f76a80fb980 msgr2=0x7f76a80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.475+0000 7f76ad956640 1 -- 172.21.15.2:0/1713088789 shutdown_connections 2024-03-15T12:58:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.475+0000 7f76ad956640 1 -- 172.21.15.2:0/1713088789 wait complete. 2024-03-15T12:58:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.475+0000 7f76ad956640 1 Processor -- start 2024-03-15T12:58:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.475+0000 7f76ad956640 1 -- start start 2024-03-15T12:58:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.475+0000 7f76ad956640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76a81003f0 0x7f76a8197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.475+0000 7f76ad956640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a81015e0 0x7f76a8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.475+0000 7f76ad956640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a8101fa0 0x7f76a819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.475+0000 7f76ad956640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a819f0f0 con 0x7f76a81015e0 2024-03-15T12:58:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.475+0000 7f76ad956640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a819f260 con 0x7f76a81003f0 2024-03-15T12:58:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.475+0000 7f76ad956640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a819f420 con 0x7f76a8101fa0 2024-03-15T12:58:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76a67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a81015e0 0x7f76a8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76a6ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76a81003f0 0x7f76a8197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76a6ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76a81003f0 0x7f76a8197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55982/0 (socket says 172.21.15.2:55982) 2024-03-15T12:58:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76a6ffd640 1 -- 172.21.15.2:0/202353803 learned_addr learned my addr 172.21.15.2:0/202353803 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76a67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a81015e0 0x7f76a8197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50432/0 (socket says 172.21.15.2:50432) 2024-03-15T12:58:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76a77fe640 1 --2- 172.21.15.2:0/202353803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a8101fa0 0x7f76a819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76a67fc640 1 -- 172.21.15.2:0/202353803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a8101fa0 msgr2=0x7f76a819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76a67fc640 1 --2- 172.21.15.2:0/202353803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a8101fa0 0x7f76a819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76a67fc640 1 -- 172.21.15.2:0/202353803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76a81003f0 msgr2=0x7f76a8197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76a67fc640 1 --2- 172.21.15.2:0/202353803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76a81003f0 0x7f76a8197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76a67fc640 1 -- 172.21.15.2:0/202353803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690009560 con 0x7f76a81015e0 2024-03-15T12:58:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76a6ffd640 1 --2- 172.21.15.2:0/202353803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76a81003f0 0x7f76a8197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76a67fc640 1 --2- 172.21.15.2:0/202353803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a81015e0 0x7f76a8197b40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f769002c170 tx=0x7f76900049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76ac954640 1 -- 172.21.15.2:0/202353803 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690004250 con 0x7f76a81015e0 2024-03-15T12:58:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76ac954640 1 -- 172.21.15.2:0/202353803 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f76900043b0 con 0x7f76a81015e0 2024-03-15T12:58:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76ad956640 1 -- 172.21.15.2:0/202353803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a819f6a0 con 0x7f76a81015e0 2024-03-15T12:58:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76ac954640 1 -- 172.21.15.2:0/202353803 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690003cf0 con 0x7f76a81015e0 2024-03-15T12:58:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.476+0000 7f76ad956640 1 -- 172.21.15.2:0/202353803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a819fb80 con 0x7f76a81015e0 2024-03-15T12:58:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.478+0000 7f76ac954640 1 -- 172.21.15.2:0/202353803 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f7690005000 con 0x7f76a81015e0 2024-03-15T12:58:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76b0611740 1 Processor -- start 2024-03-15T12:58:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76b0611740 1 -- start start 2024-03-15T12:58:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76b0611740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56115040cf80 0x56115040d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76b0611740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56115040d970 0x561150412470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76b0611740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561150412ae0 0x561150414f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76ac954640 1 --2- 172.21.15.2:0/202353803 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f76940a1a30 0x7f76940a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76b0611740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611503c8cd0 con 0x561150412ae0 2024-03-15T12:58:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76b0611740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115037bf20 con 0x56115040cf80 2024-03-15T12:58:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76b0611740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561150379450 con 0x56115040d970 2024-03-15T12:58:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76a67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561150412ae0 0x561150414f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76a77fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56115040cf80 0x56115040d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76a67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561150412ae0 0x561150414f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50448/0 (socket says 172.21.15.2:50448) 2024-03-15T12:58:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76a67fc640 1 -- 172.21.15.2:0/1515254733 learned_addr learned my addr 172.21.15.2:0/1515254733 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76a6ffd640 1 --2- 172.21.15.2:0/1515254733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56115040d970 0x561150412470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76a77fe640 1 --2- 172.21.15.2:0/202353803 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f76940a1a30 0x7f76940a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76ac954640 1 -- 172.21.15.2:0/202353803 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f76900ef970 con 0x7f76a81015e0 2024-03-15T12:58:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76a67fc640 1 -- 172.21.15.2:0/1515254733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56115040d970 msgr2=0x561150412470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76a67fc640 1 --2- 172.21.15.2:0/1515254733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56115040d970 0x561150412470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76a67fc640 1 -- 172.21.15.2:0/1515254733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56115040cf80 msgr2=0x56115040d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76a67fc640 1 --2- 172.21.15.2:0/1515254733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56115040cf80 0x56115040d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76a67fc640 1 -- 172.21.15.2:0/1515254733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561150329f20 con 0x561150412ae0 2024-03-15T12:58:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.479+0000 7f76a67fc640 1 --2- 172.21.15.2:0/1515254733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561150412ae0 0x561150414f70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f76900057a0 tx=0x7f769008e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a0230ab03c4149 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.480+0000 7f76a77fe640 1 --2- 172.21.15.2:0/202353803 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f76940a1a30 0x7f76940a3ef0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x56115040e1d0 tx=0x7f76a0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.480+0000 7f769dffb640 1 -- 172.21.15.2:0/1515254733 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900bc080 con 0x561150412ae0 2024-03-15T12:58:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.480+0000 7f769dffb640 1 -- 172.21.15.2:0/1515254733 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f76900bbea0 con 0x561150412ae0 2024-03-15T12:58:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.480+0000 7f769dffb640 1 -- 172.21.15.2:0/1515254733 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900ba840 con 0x561150412ae0 2024-03-15T12:58:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.480+0000 7f76b0611740 1 -- 172.21.15.2:0/1515254733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561150412ae0 msgr2=0x561150414f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.480+0000 7f76b0611740 1 --2- 172.21.15.2:0/1515254733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561150412ae0 0x561150414f70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f76900057a0 tx=0x7f769008e790 comp rx=0 tx=0).stop 2024-03-15T12:58:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.480+0000 7f76b0611740 1 -- 172.21.15.2:0/1515254733 shutdown_connections 2024-03-15T12:58:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.480+0000 7f76b0611740 1 --2- 172.21.15.2:0/1515254733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561150412ae0 0x561150414f70 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.480+0000 7f76b0611740 1 --2- 172.21.15.2:0/1515254733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56115040d970 0x561150412470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.480+0000 7f76b0611740 1 --2- 172.21.15.2:0/1515254733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56115040cf80 0x56115040d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.480+0000 7f76b0611740 1 -- 172.21.15.2:0/1515254733 >> 172.21.15.2:0/1515254733 conn(0x5611504084f0 msgr2=0x5611504088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.480+0000 7f76b0611740 1 -- 172.21.15.2:0/1515254733 shutdown_connections 2024-03-15T12:58:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.480+0000 7f76b0611740 1 -- 172.21.15.2:0/1515254733 wait complete. 2024-03-15T12:58:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76b0611740 1 -- start start 2024-03-15T12:58:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76b0611740 1 Processor -- start 2024-03-15T12:58:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76b0611740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56115040cf80 0x561150416f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76b0611740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56115040d970 0x561150417490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76b0611740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561150412ae0 0x56115041e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76b0611740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561150374c30 con 0x56115040d970 2024-03-15T12:58:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76b0611740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561150319d30 con 0x561150412ae0 2024-03-15T12:58:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76b0611740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561150048310 con 0x56115040cf80 2024-03-15T12:58:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76b0611740 10 client.0 _ll_register_callbacks cb 0x561150321d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76a67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56115040d970 0x561150417490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76a67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56115040d970 0x561150417490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50454/0 (socket says 172.21.15.2:50454) 2024-03-15T12:58:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76a67fc640 1 -- 172.21.15.2:0/1325819381 learned_addr learned my addr 172.21.15.2:0/1325819381 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76a77fe640 1 --2- 172.21.15.2:0/1325819381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561150412ae0 0x56115041e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76a6ffd640 1 --2- 172.21.15.2:0/1325819381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56115040cf80 0x561150416f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76a67fc640 1 -- 172.21.15.2:0/1325819381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56115040cf80 msgr2=0x561150416f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.481+0000 7f76a67fc640 1 --2- 172.21.15.2:0/1325819381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56115040cf80 0x561150416f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f76a67fc640 1 -- 172.21.15.2:0/1325819381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561150412ae0 msgr2=0x56115041e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f76a67fc640 1 --2- 172.21.15.2:0/1325819381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561150412ae0 0x56115041e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f76a67fc640 1 -- 172.21.15.2:0/1325819381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a819fb80 con 0x56115040d970 2024-03-15T12:58:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f76a67fc640 1 --2- 172.21.15.2:0/1325819381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56115040d970 0x561150417490 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f76900bc4d0 tx=0x7f76900ecbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7865c164df6259f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f7686ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f7686ffd640 1 -- 172.21.15.2:0/1325819381 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900bc080 con 0x56115040d970 2024-03-15T12:58:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f7686ffd640 1 -- 172.21.15.2:0/1325819381 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f76900bc540 con 0x56115040d970 2024-03-15T12:58:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f76b0611740 1 -- 172.21.15.2:0/1325819381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611500746c0 con 0x56115040d970 2024-03-15T12:58:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f7686ffd640 1 -- 172.21.15.2:0/1325819381 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76900ecf30 con 0x56115040d970 2024-03-15T12:58:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f76b0611740 10 client.15504 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f76b0611740 1 -- 172.21.15.2:0/1325819381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5611503732b0 con 0x56115040d970 2024-03-15T12:58:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f7686ffd640 1 -- 172.21.15.2:0/1325819381 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5611500746c0 con 0x56115040d970 2024-03-15T12:58:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f7686ffd640 1 -- 172.21.15.2:0/1325819381 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f769003c070 con 0x56115040d970 2024-03-15T12:58:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f76b0611740 10 client.15504 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f76b0611740 10 client.15504 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-15T12:58:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f76b0611740 20 client.15504 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f76b0611740 1 --2- 172.21.15.2:0/1325819381 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561150422de0 0x561150427860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f76b0611740 4 client.15504 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:58:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.482+0000 7f76b0611740 1 -- 172.21.15.2:0/1325819381 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611500c5270 con 0x561150422de0 2024-03-15T12:58:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.483+0000 7f76a6ffd640 1 --2- 172.21.15.2:0/1325819381 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561150422de0 0x561150427860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.483+0000 7f7686ffd640 10 client.15504 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T12:58:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.483+0000 7f76a6ffd640 1 --2- 172.21.15.2:0/1325819381 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561150422de0 0x561150427860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99c84d5ef38063bf server_cookie=4f82778dbee54f8e in_seq=0 out_seq=0 2024-03-15T12:58:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.484+0000 7f7686ffd640 1 -- 172.21.15.2:0/1325819381 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5611500c5270 con 0x561150422de0 2024-03-15T12:58:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.484+0000 7f7686ffd640 10 client.15504 handle_command_reply: tid=0 2024-03-15T12:58:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.484+0000 7f76b0611740 1 client.15504 shutdown 2024-03-15T12:58:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.484+0000 7f76b0611740 1 -- 172.21.15.2:0/1325819381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56115040d970 msgr2=0x561150417490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.484+0000 7f76b0611740 1 --2- 172.21.15.2:0/1325819381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56115040d970 0x561150417490 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f76900bc4d0 tx=0x7f76900ecbc0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.484+0000 7f76b0611740 1 -- 172.21.15.2:0/1325819381 shutdown_connections 2024-03-15T12:58:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.484+0000 7f76b0611740 1 --2- 172.21.15.2:0/1325819381 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561150422de0 0x561150427860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.484+0000 7f76b0611740 1 --2- 172.21.15.2:0/1325819381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561150412ae0 0x56115041e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.484+0000 7f76b0611740 1 --2- 172.21.15.2:0/1325819381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56115040d970 0x561150417490 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.484+0000 7f7686ffd640 0 client.15504 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T12:58:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.484+0000 7f76b0611740 1 --2- 172.21.15.2:0/1325819381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56115040cf80 0x561150416f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.484+0000 7f76b0611740 1 -- 172.21.15.2:0/1325819381 >> 172.21.15.2:0/1325819381 conn(0x5611504084f0 msgr2=0x5611504077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.485+0000 7f76b0611740 1 -- 172.21.15.2:0/1325819381 shutdown_connections 2024-03-15T12:58:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.485+0000 7f76b0611740 1 -- 172.21.15.2:0/1325819381 wait complete. 2024-03-15T12:58:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.485+0000 7f76b0611740 20 client.15504 trim_cache size 0 max 16384 2024-03-15T12:58:31.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.501+0000 7f76b0611740 1 Processor -- start 2024-03-15T12:58:31.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.501+0000 7f76b0611740 1 -- start start 2024-03-15T12:58:31.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.501+0000 7f76b0611740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56115040cf80 0x561150415a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.501+0000 7f76b0611740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56115040d970 0x56115041e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.501+0000 7f76b0611740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561150412ae0 0x561150416f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.501+0000 7f76b0611740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611502fa490 con 0x56115040d970 2024-03-15T12:58:31.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.501+0000 7f76b0611740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115041f150 con 0x56115040cf80 2024-03-15T12:58:31.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.501+0000 7f76b0611740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561150408f40 con 0x561150412ae0 2024-03-15T12:58:31.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.502+0000 7f76a67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56115040d970 0x56115041e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.502+0000 7f76a67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56115040d970 0x56115041e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50470/0 (socket says 172.21.15.2:50470) 2024-03-15T12:58:31.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.502+0000 7f76a6ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56115040cf80 0x561150415a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.502+0000 7f76a67fc640 1 -- 172.21.15.2:0/3859713578 learned_addr learned my addr 172.21.15.2:0/3859713578 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.502+0000 7f76a77fe640 1 --2- 172.21.15.2:0/3859713578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561150412ae0 0x561150416f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.502+0000 7f76a67fc640 1 -- 172.21.15.2:0/3859713578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561150412ae0 msgr2=0x561150416f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.502+0000 7f76a67fc640 1 --2- 172.21.15.2:0/3859713578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561150412ae0 0x561150416f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.502+0000 7f76a67fc640 1 -- 172.21.15.2:0/3859713578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56115040cf80 msgr2=0x561150415a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.502+0000 7f76a67fc640 1 --2- 172.21.15.2:0/3859713578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56115040cf80 0x561150415a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.502+0000 7f76a67fc640 1 -- 172.21.15.2:0/3859713578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56115030fa20 con 0x56115040d970 2024-03-15T12:58:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.502+0000 7f76a67fc640 1 --2- 172.21.15.2:0/3859713578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56115040d970 0x56115041e490 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f76900b8ec0 tx=0x7f76900b8120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafaf0b3d5b86c9b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.502+0000 7f769d7fa640 1 -- 172.21.15.2:0/3859713578 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900bc7a0 con 0x56115040d970 2024-03-15T12:58:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.502+0000 7f769d7fa640 1 -- 172.21.15.2:0/3859713578 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f76900bcc40 con 0x56115040d970 2024-03-15T12:58:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.502+0000 7f769d7fa640 1 -- 172.21.15.2:0/3859713578 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900c4ce0 con 0x56115040d970 2024-03-15T12:58:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.503+0000 7f76b0611740 1 -- 172.21.15.2:0/3859713578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56115040d970 msgr2=0x56115041e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.503+0000 7f76b0611740 1 --2- 172.21.15.2:0/3859713578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56115040d970 0x56115041e490 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f76900b8ec0 tx=0x7f76900b8120 comp rx=0 tx=0).stop 2024-03-15T12:58:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.503+0000 7f76b0611740 1 -- 172.21.15.2:0/3859713578 shutdown_connections 2024-03-15T12:58:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.503+0000 7f76b0611740 1 --2- 172.21.15.2:0/3859713578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561150412ae0 0x561150416f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.503+0000 7f76b0611740 1 --2- 172.21.15.2:0/3859713578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56115040d970 0x56115041e490 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.503+0000 7f76b0611740 1 --2- 172.21.15.2:0/3859713578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56115040cf80 0x561150415a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.503+0000 7f76b0611740 1 -- 172.21.15.2:0/3859713578 >> 172.21.15.2:0/3859713578 conn(0x5611504084f0 msgr2=0x56115040e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.503+0000 7f76b0611740 1 -- 172.21.15.2:0/3859713578 shutdown_connections 2024-03-15T12:58:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.503+0000 7f76b0611740 1 -- 172.21.15.2:0/3859713578 wait complete. 2024-03-15T12:58:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.503+0000 7f76b0611740 1 -- start start 2024-03-15T12:58:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.503+0000 7f76b0611740 1 Processor -- start 2024-03-15T12:58:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.503+0000 7f76b0611740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56115040cf80 0x56115041a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.503+0000 7f76b0611740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56115040d970 0x56115041ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76b0611740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561150412ae0 0x56115041d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76b0611740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611504167c0 con 0x561150412ae0 2024-03-15T12:58:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76b0611740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115041dab0 con 0x56115040d970 2024-03-15T12:58:31.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76b0611740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56115041dc70 con 0x56115040cf80 2024-03-15T12:58:31.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76b0611740 10 client.0 _ll_register_callbacks cb 0x7f76a0009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:31.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76a77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561150412ae0 0x56115041d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76a77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561150412ae0 0x56115041d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50476/0 (socket says 172.21.15.2:50476) 2024-03-15T12:58:31.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76a77fe640 1 -- 172.21.15.2:0/1423380558 learned_addr learned my addr 172.21.15.2:0/1423380558 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76a6ffd640 1 --2- 172.21.15.2:0/1423380558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56115040cf80 0x56115041a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76a67fc640 1 --2- 172.21.15.2:0/1423380558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56115040d970 0x56115041ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76a77fe640 1 -- 172.21.15.2:0/1423380558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56115040cf80 msgr2=0x56115041a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76a77fe640 1 --2- 172.21.15.2:0/1423380558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56115040cf80 0x56115041a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76a77fe640 1 -- 172.21.15.2:0/1423380558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56115040d970 msgr2=0x56115041ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76a77fe640 1 --2- 172.21.15.2:0/1423380558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56115040d970 0x56115041ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76a77fe640 1 -- 172.21.15.2:0/1423380558 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611503732b0 con 0x561150412ae0 2024-03-15T12:58:31.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76a77fe640 1 --2- 172.21.15.2:0/1423380558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561150412ae0 0x56115041d570 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f76a001f940 tx=0x7f76a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7418ef03d185e27d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f7686ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f7686ffd640 1 -- 172.21.15.2:0/1423380558 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a002e070 con 0x561150412ae0 2024-03-15T12:58:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f7686ffd640 1 -- 172.21.15.2:0/1423380558 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f76a001e620 con 0x561150412ae0 2024-03-15T12:58:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76b0611740 1 -- 172.21.15.2:0/1423380558 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561150423750 con 0x561150412ae0 2024-03-15T12:58:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f7686ffd640 1 -- 172.21.15.2:0/1423380558 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76a0025040 con 0x561150412ae0 2024-03-15T12:58:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76b0611740 10 client.15510 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.504+0000 7f76b0611740 1 -- 172.21.15.2:0/1423380558 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561150335780 con 0x561150412ae0 2024-03-15T12:58:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.505+0000 7f7686ffd640 1 -- 172.21.15.2:0/1423380558 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x561150423750 con 0x561150412ae0 2024-03-15T12:58:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.505+0000 7f7686ffd640 1 -- 172.21.15.2:0/1423380558 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f76a0031020 con 0x561150412ae0 2024-03-15T12:58:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.505+0000 7f76b0611740 10 client.15510 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.505+0000 7f76b0611740 10 client.15510 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-15T12:58:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.505+0000 7f76b0611740 20 client.15510 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.505+0000 7f76b0611740 1 --2- 172.21.15.2:0/1423380558 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561150482f00 0x56115047f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.505+0000 7f76b0611740 4 client.15510 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-15T12:58:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.505+0000 7f76b0611740 1 -- 172.21.15.2:0/1423380558 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f76900bbea0 con 0x561150482f00 2024-03-15T12:58:31.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.505+0000 7f76a6ffd640 1 --2- 172.21.15.2:0/1423380558 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561150482f00 0x56115047f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.505+0000 7f7686ffd640 10 client.15510 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T12:58:31.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.505+0000 7f76a6ffd640 1 --2- 172.21.15.2:0/1423380558 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561150482f00 0x56115047f360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7997954db739d784 server_cookie=9194342feb8e06bf in_seq=0 out_seq=0 2024-03-15T12:58:31.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f7686ffd640 1 -- 172.21.15.2:0/1423380558 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f76900bbea0 con 0x561150482f00 2024-03-15T12:58:31.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f7686ffd640 10 client.15510 handle_command_reply: tid=0 2024-03-15T12:58:31.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76b0611740 1 client.15510 shutdown 2024-03-15T12:58:31.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76b0611740 1 -- 172.21.15.2:0/1423380558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561150412ae0 msgr2=0x56115041d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76b0611740 1 --2- 172.21.15.2:0/1423380558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561150412ae0 0x56115041d570 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f76a001f940 tx=0x7f76a0004ab0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76b0611740 1 -- 172.21.15.2:0/1423380558 shutdown_connections 2024-03-15T12:58:31.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76b0611740 1 --2- 172.21.15.2:0/1423380558 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561150482f00 0x56115047f360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76b0611740 1 --2- 172.21.15.2:0/1423380558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561150412ae0 0x56115041d570 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76b0611740 1 --2- 172.21.15.2:0/1423380558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56115040d970 0x56115041ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f7686ffd640 0 client.15510 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T12:58:31.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76b0611740 1 --2- 172.21.15.2:0/1423380558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56115040cf80 0x56115041a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76b0611740 1 -- 172.21.15.2:0/1423380558 >> 172.21.15.2:0/1423380558 conn(0x5611504084f0 msgr2=0x561150415a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76b0611740 1 -- 172.21.15.2:0/1423380558 shutdown_connections 2024-03-15T12:58:31.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76b0611740 1 -- 172.21.15.2:0/1423380558 wait complete. 2024-03-15T12:58:31.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76b0611740 20 client.15510 trim_cache size 0 max 16384 2024-03-15T12:58:31.499 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:58:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "2767718a-e2ca-11ee-95c9-87774f69a715", 2024-03-15T12:58:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 1, 2024-03-15T12:58:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24500, 2024-03-15T12:58:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-03-15T12:58:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-03-15T12:58:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-03-15T12:58:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 111.681407385, 2024-03-15T12:58:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 28, 2024-03-15T12:58:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 91, 2024-03-15T12:58:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 89, 2024-03-15T12:58:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 148.69646059499999 2024-03-15T12:58:31.500 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:58:31.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76ad956640 1 -- 172.21.15.2:0/202353803 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f76940a1a30 msgr2=0x7f76940a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76ad956640 1 --2- 172.21.15.2:0/202353803 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f76940a1a30 0x7f76940a3ef0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x56115040e1d0 tx=0x7f76a0009500 comp rx=0 tx=0).stop 2024-03-15T12:58:31.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76ad956640 1 -- 172.21.15.2:0/202353803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a81015e0 msgr2=0x7f76a8197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76ad956640 1 --2- 172.21.15.2:0/202353803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a81015e0 0x7f76a8197b40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f769002c170 tx=0x7f76900049e0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76ad956640 1 -- 172.21.15.2:0/202353803 shutdown_connections 2024-03-15T12:58:31.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76ad956640 1 --2- 172.21.15.2:0/202353803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a8101fa0 0x7f76a819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76ad956640 1 --2- 172.21.15.2:0/202353803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a81015e0 0x7f76a8197b40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76ad956640 1 --2- 172.21.15.2:0/202353803 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f76940a1a30 0x7f76940a3ef0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76ad956640 1 --2- 172.21.15.2:0/202353803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76a81003f0 0x7f76a8197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.506+0000 7f76ad956640 1 -- 172.21.15.2:0/202353803 >> 172.21.15.2:0/202353803 conn(0x7f76a80fb980 msgr2=0x7f76a80fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.507+0000 7f76ad956640 1 -- 172.21.15.2:0/202353803 shutdown_connections 2024-03-15T12:58:31.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.507+0000 7f76ad956640 1 -- 172.21.15.2:0/202353803 wait complete. 2024-03-15T12:58:31.503 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-15T12:58:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.621+0000 7f8fcd747640 1 -- 172.21.15.2:0/143188347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fc81026f0 msgr2=0x7f8fc8102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.621+0000 7f8fcd747640 1 --2- 172.21.15.2:0/143188347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fc81026f0 0x7f8fc8102b70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009860 tx=0x7f8fbc02bb70 comp rx=0 tx=0).stop 2024-03-15T12:58:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.622+0000 7f8fcd747640 1 -- 172.21.15.2:0/143188347 shutdown_connections 2024-03-15T12:58:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.622+0000 7f8fcd747640 1 --2- 172.21.15.2:0/143188347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fc81030b0 0x7f8fc8109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.622+0000 7f8fcd747640 1 --2- 172.21.15.2:0/143188347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fc81026f0 0x7f8fc8102b70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.622+0000 7f8fcd747640 1 --2- 172.21.15.2:0/143188347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fc8101500 0x7f8fc8101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.622+0000 7f8fcd747640 1 -- 172.21.15.2:0/143188347 >> 172.21.15.2:0/143188347 conn(0x7f8fc80fca90 msgr2=0x7f8fc80feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.622+0000 7f8fcd747640 1 -- 172.21.15.2:0/143188347 shutdown_connections 2024-03-15T12:58:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.622+0000 7f8fcd747640 1 -- 172.21.15.2:0/143188347 wait complete. 2024-03-15T12:58:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.622+0000 7f8fcd747640 1 Processor -- start 2024-03-15T12:58:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.622+0000 7f8fcd747640 1 -- start start 2024-03-15T12:58:31.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.622+0000 7f8fcd747640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fc8101500 0x7f8fc8197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.622+0000 7f8fcd747640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fc81026f0 0x7f8fc8197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.622+0000 7f8fcd747640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fc81030b0 0x7f8fc819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.622+0000 7f8fcd747640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc819f080 con 0x7f8fc81030b0 2024-03-15T12:58:31.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.622+0000 7f8fcd747640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc819f1f0 con 0x7f8fc8101500 2024-03-15T12:58:31.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.622+0000 7f8fcd747640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc819f3b0 con 0x7f8fc81026f0 2024-03-15T12:58:31.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fc77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fc81030b0 0x7f8fc819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fc6ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fc8101500 0x7f8fc8197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fc67fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fc81026f0 0x7f8fc8197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fc77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fc81030b0 0x7f8fc819eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50496/0 (socket says 172.21.15.2:50496) 2024-03-15T12:58:31.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fc77fe640 1 -- 172.21.15.2:0/674103491 learned_addr learned my addr 172.21.15.2:0/674103491 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fc6ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fc8101500 0x7f8fc8197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:56032/0 (socket says 172.21.15.2:56032) 2024-03-15T12:58:31.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fc67fc640 1 -- 172.21.15.2:0/674103491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fc8101500 msgr2=0x7f8fc8197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fc67fc640 1 --2- 172.21.15.2:0/674103491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fc8101500 0x7f8fc8197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fc67fc640 1 -- 172.21.15.2:0/674103491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fc81030b0 msgr2=0x7f8fc819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fc67fc640 1 --2- 172.21.15.2:0/674103491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fc81030b0 0x7f8fc819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fc67fc640 1 -- 172.21.15.2:0/674103491 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb8009560 con 0x7f8fc81026f0 2024-03-15T12:58:31.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fc77fe640 1 --2- 172.21.15.2:0/674103491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fc81030b0 0x7f8fc819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:31.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fc6ffd640 1 --2- 172.21.15.2:0/674103491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fc8101500 0x7f8fc8197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:31.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fc67fc640 1 --2- 172.21.15.2:0/674103491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fc81026f0 0x7f8fc8197ad0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc02c0b0 tx=0x7f8fbc032460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8faffff640 1 -- 172.21.15.2:0/674103491 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc02c510 con 0x7f8fc81026f0 2024-03-15T12:58:31.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fcd747640 1 -- 172.21.15.2:0/674103491 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fbc009490 con 0x7f8fc81026f0 2024-03-15T12:58:31.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8faffff640 1 -- 172.21.15.2:0/674103491 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8fbc02cb40 con 0x7f8fc81026f0 2024-03-15T12:58:31.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8faffff640 1 -- 172.21.15.2:0/674103491 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc032600 con 0x7f8fc81026f0 2024-03-15T12:58:31.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.623+0000 7f8fcd747640 1 -- 172.21.15.2:0/674103491 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc819f9a0 con 0x7f8fc81026f0 2024-03-15T12:58:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.625+0000 7f8faffff640 1 -- 172.21.15.2:0/674103491 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f8fbc0037f0 con 0x7f8fc81026f0 2024-03-15T12:58:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fd03f3740 1 Processor -- start 2024-03-15T12:58:31.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fd03f3740 1 -- start start 2024-03-15T12:58:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fd03f3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed875f80 0x55c5ed8763a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fd03f3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed876970 0x55c5ed87b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fd03f3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed87bae0 0x55c5ed87df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fd03f3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ed831cd0 con 0x55c5ed875f80 2024-03-15T12:58:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fd03f3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ed7e4f20 con 0x55c5ed87bae0 2024-03-15T12:58:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fd03f3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ed7e2450 con 0x55c5ed876970 2024-03-15T12:58:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8faffff640 1 --2- 172.21.15.2:0/674103491 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8fb00a1a30 0x7f8fb00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fc77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed875f80 0x55c5ed8763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fc77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed875f80 0x55c5ed8763a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50502/0 (socket says 172.21.15.2:50502) 2024-03-15T12:58:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fc77fe640 1 -- 172.21.15.2:0/1244854093 learned_addr learned my addr 172.21.15.2:0/1244854093 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fc6ffd640 1 --2- 172.21.15.2:0/1244854093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed876970 0x55c5ed87b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fc67fc640 1 --2- 172.21.15.2:0/1244854093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed87bae0 0x55c5ed87df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fc77fe640 1 --2- 172.21.15.2:0/674103491 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8fb00a1a30 0x7f8fb00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8faffff640 1 -- 172.21.15.2:0/674103491 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f8fbc0efa90 con 0x7f8fc81026f0 2024-03-15T12:58:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fc77fe640 1 -- 172.21.15.2:0/1244854093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed876970 msgr2=0x55c5ed87b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fc77fe640 1 --2- 172.21.15.2:0/1244854093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed876970 0x55c5ed87b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fc77fe640 1 -- 172.21.15.2:0/1244854093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed87bae0 msgr2=0x55c5ed87df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fc77fe640 1 --2- 172.21.15.2:0/1244854093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed87bae0 0x55c5ed87df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.626+0000 7f8fc77fe640 1 -- 172.21.15.2:0/1244854093 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5ed792f20 con 0x55c5ed875f80 2024-03-15T12:58:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.627+0000 7f8fc77fe640 1 --2- 172.21.15.2:0/1244854093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed875f80 0x55c5ed8763a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000ff20 tx=0x7f8fc00138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad754205a4b0a534 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.627+0000 7f8fad7fa640 1 -- 172.21.15.2:0/1244854093 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc001f680 con 0x55c5ed875f80 2024-03-15T12:58:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.627+0000 7f8fad7fa640 1 -- 172.21.15.2:0/1244854093 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8fc000ee50 con 0x55c5ed875f80 2024-03-15T12:58:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.627+0000 7f8fad7fa640 1 -- 172.21.15.2:0/1244854093 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0010790 con 0x55c5ed875f80 2024-03-15T12:58:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.627+0000 7f8fd03f3740 1 -- 172.21.15.2:0/1244854093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed875f80 msgr2=0x55c5ed8763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.627+0000 7f8fc77fe640 1 --2- 172.21.15.2:0/674103491 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8fb00a1a30 0x7f8fb00a3ef0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0024000 tx=0x7f8fc0022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.627+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/1244854093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed875f80 0x55c5ed8763a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000ff20 tx=0x7f8fc00138f0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.627+0000 7f8fd03f3740 1 -- 172.21.15.2:0/1244854093 shutdown_connections 2024-03-15T12:58:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.627+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/1244854093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed87bae0 0x55c5ed87df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.627+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/1244854093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed876970 0x55c5ed87b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.627+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/1244854093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed875f80 0x55c5ed8763a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fd03f3740 1 -- 172.21.15.2:0/1244854093 >> 172.21.15.2:0/1244854093 conn(0x55c5ed8714f0 msgr2=0x55c5ed8718f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fd03f3740 1 -- 172.21.15.2:0/1244854093 shutdown_connections 2024-03-15T12:58:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fd03f3740 1 -- 172.21.15.2:0/1244854093 wait complete. 2024-03-15T12:58:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fd03f3740 1 -- start start 2024-03-15T12:58:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fd03f3740 1 Processor -- start 2024-03-15T12:58:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fd03f3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed875f80 0x55c5ed880220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fd03f3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed876970 0x55c5ed880760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fd03f3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed8877a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fd03f3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ed7ddc30 con 0x55c5ed87bae0 2024-03-15T12:58:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fd03f3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ed782d30 con 0x55c5ed876970 2024-03-15T12:58:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fd03f3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ed4b1310 con 0x55c5ed875f80 2024-03-15T12:58:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fd03f3740 10 client.0 _ll_register_callbacks cb 0x55c5ed78ad40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fc77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed8877a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fc77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed8877a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50512/0 (socket says 172.21.15.2:50512) 2024-03-15T12:58:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fc77fe640 1 -- 172.21.15.2:0/3158986318 learned_addr learned my addr 172.21.15.2:0/3158986318 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fc67fc640 1 --2- 172.21.15.2:0/3158986318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed876970 0x55c5ed880760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fc6ffd640 1 --2- 172.21.15.2:0/3158986318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed875f80 0x55c5ed880220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fc77fe640 1 -- 172.21.15.2:0/3158986318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed875f80 msgr2=0x55c5ed880220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fc77fe640 1 --2- 172.21.15.2:0/3158986318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed875f80 0x55c5ed880220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fc77fe640 1 -- 172.21.15.2:0/3158986318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed876970 msgr2=0x55c5ed880760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fc77fe640 1 --2- 172.21.15.2:0/3158986318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed876970 0x55c5ed880760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.628+0000 7f8fc77fe640 1 -- 172.21.15.2:0/3158986318 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc000fb00 con 0x55c5ed87bae0 2024-03-15T12:58:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8fc77fe640 1 --2- 172.21.15.2:0/3158986318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed8877a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000e5c0 tx=0x7f8fc000f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ea849739a7a59a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8faaff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8faaff5640 1 -- 172.21.15.2:0/3158986318 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0010c20 con 0x55c5ed87bae0 2024-03-15T12:58:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8faaff5640 1 -- 172.21.15.2:0/3158986318 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f8fc0010d80 con 0x55c5ed87bae0 2024-03-15T12:58:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8fd03f3740 1 -- 172.21.15.2:0/3158986318 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5ed4dd6c0 con 0x55c5ed87bae0 2024-03-15T12:58:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8faaff5640 1 -- 172.21.15.2:0/3158986318 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fc00049d0 con 0x55c5ed87bae0 2024-03-15T12:58:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8fd03f3740 10 client.15522 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8fd03f3740 1 -- 172.21.15.2:0/3158986318 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c5ed7dc2b0 con 0x55c5ed87bae0 2024-03-15T12:58:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8faaff5640 1 -- 172.21.15.2:0/3158986318 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55c5ed4dd6c0 con 0x55c5ed87bae0 2024-03-15T12:58:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8faaff5640 1 -- 172.21.15.2:0/3158986318 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8fc0028420 con 0x55c5ed87bae0 2024-03-15T12:58:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8fd03f3740 10 client.15522 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8fd03f3740 10 client.15522 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-15T12:58:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8fd03f3740 20 client.15522 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/3158986318 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c5ed88caf0 0x55c5ed88bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8fd03f3740 4 client.15522 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:58:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.629+0000 7f8fd03f3740 1 -- 172.21.15.2:0/3158986318 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5ed52e270 con 0x55c5ed88caf0 2024-03-15T12:58:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.630+0000 7f8fc6ffd640 1 --2- 172.21.15.2:0/3158986318 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c5ed88caf0 0x55c5ed88bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.630+0000 7f8fc6ffd640 1 --2- 172.21.15.2:0/3158986318 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c5ed88caf0 0x55c5ed88bd50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4074b694db2c0d23 server_cookie=555289fbe67934f6 in_seq=0 out_seq=0 2024-03-15T12:58:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.630+0000 7f8faaff5640 10 client.15522 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.631+0000 7f8faaff5640 1 -- 172.21.15.2:0/3158986318 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c5ed52e270 con 0x55c5ed88caf0 2024-03-15T12:58:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.631+0000 7f8faaff5640 10 client.15522 handle_command_reply: tid=0 2024-03-15T12:58:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.631+0000 7f8fd03f3740 1 client.15522 shutdown 2024-03-15T12:58:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.631+0000 7f8fd03f3740 1 -- 172.21.15.2:0/3158986318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 msgr2=0x55c5ed8877a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.631+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/3158986318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed8877a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000e5c0 tx=0x7f8fc000f5b0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.631+0000 7f8fd03f3740 1 -- 172.21.15.2:0/3158986318 shutdown_connections 2024-03-15T12:58:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.631+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/3158986318 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c5ed88caf0 0x55c5ed88bd50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.631+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/3158986318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed8877a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.631+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/3158986318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed876970 0x55c5ed880760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.631+0000 7f8faaff5640 0 client.15522 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.631+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/3158986318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed875f80 0x55c5ed880220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.631+0000 7f8fd03f3740 1 -- 172.21.15.2:0/3158986318 >> 172.21.15.2:0/3158986318 conn(0x55c5ed8714f0 msgr2=0x55c5ed8707d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.631+0000 7f8fd03f3740 1 -- 172.21.15.2:0/3158986318 shutdown_connections 2024-03-15T12:58:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.631+0000 7f8fd03f3740 1 -- 172.21.15.2:0/3158986318 wait complete. 2024-03-15T12:58:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.631+0000 7f8fd03f3740 20 client.15522 trim_cache size 0 max 16384 2024-03-15T12:58:31.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fd03f3740 1 Processor -- start 2024-03-15T12:58:31.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fd03f3740 1 -- start start 2024-03-15T12:58:31.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fd03f3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed875f80 0x55c5ed877e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fd03f3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed876970 0x55c5ed8783b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fd03f3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed887760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fd03f3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ed763490 con 0x55c5ed87bae0 2024-03-15T12:58:31.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fd03f3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ed88bf80 con 0x55c5ed876970 2024-03-15T12:58:31.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fd03f3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ed88ead0 con 0x55c5ed875f80 2024-03-15T12:58:31.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fc77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed887760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fc77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed887760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50522/0 (socket says 172.21.15.2:50522) 2024-03-15T12:58:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fc77fe640 1 -- 172.21.15.2:0/2385598891 learned_addr learned my addr 172.21.15.2:0/2385598891 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fc67fc640 1 --2- 172.21.15.2:0/2385598891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed876970 0x55c5ed8783b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fc6ffd640 1 --2- 172.21.15.2:0/2385598891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed875f80 0x55c5ed877e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fc77fe640 1 -- 172.21.15.2:0/2385598891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed875f80 msgr2=0x55c5ed877e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fc77fe640 1 --2- 172.21.15.2:0/2385598891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed875f80 0x55c5ed877e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fc77fe640 1 -- 172.21.15.2:0/2385598891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed876970 msgr2=0x55c5ed8783b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fc77fe640 1 --2- 172.21.15.2:0/2385598891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed876970 0x55c5ed8783b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.648+0000 7f8fc77fe640 1 -- 172.21.15.2:0/2385598891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5ed778a20 con 0x55c5ed87bae0 2024-03-15T12:58:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.649+0000 7f8fc77fe640 1 --2- 172.21.15.2:0/2385598891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed887760 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00108c0 tx=0x7f8fc000fd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79039bd853cc9a75 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.649+0000 7f8facff9640 1 -- 172.21.15.2:0/2385598891 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc000f880 con 0x55c5ed87bae0 2024-03-15T12:58:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.649+0000 7f8facff9640 1 -- 172.21.15.2:0/2385598891 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8fc0013df0 con 0x55c5ed87bae0 2024-03-15T12:58:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.649+0000 7f8facff9640 1 -- 172.21.15.2:0/2385598891 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc00289c0 con 0x55c5ed87bae0 2024-03-15T12:58:31.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.649+0000 7f8fd03f3740 1 -- 172.21.15.2:0/2385598891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 msgr2=0x55c5ed887760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.649+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/2385598891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed887760 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00108c0 tx=0x7f8fc000fd20 comp rx=0 tx=0).stop 2024-03-15T12:58:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.649+0000 7f8fd03f3740 1 -- 172.21.15.2:0/2385598891 shutdown_connections 2024-03-15T12:58:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.649+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/2385598891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed887760 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.649+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/2385598891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed876970 0x55c5ed8783b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.649+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/2385598891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed875f80 0x55c5ed877e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.649+0000 7f8fd03f3740 1 -- 172.21.15.2:0/2385598891 >> 172.21.15.2:0/2385598891 conn(0x55c5ed8714f0 msgr2=0x55c5ed87ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.649+0000 7f8fd03f3740 1 -- 172.21.15.2:0/2385598891 shutdown_connections 2024-03-15T12:58:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.649+0000 7f8fd03f3740 1 -- 172.21.15.2:0/2385598891 wait complete. 2024-03-15T12:58:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fd03f3740 1 -- start start 2024-03-15T12:58:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fd03f3740 1 Processor -- start 2024-03-15T12:58:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fd03f3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed875f80 0x55c5ed887040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fd03f3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed876970 0x55c5ed887580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fd03f3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed883ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fd03f3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ed884020 con 0x55c5ed87bae0 2024-03-15T12:58:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fd03f3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ed884190 con 0x55c5ed876970 2024-03-15T12:58:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fd03f3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ed884350 con 0x55c5ed875f80 2024-03-15T12:58:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fd03f3740 10 client.0 _ll_register_callbacks cb 0x7f8fbc03cfc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fc77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed883ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fc77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed883ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50530/0 (socket says 172.21.15.2:50530) 2024-03-15T12:58:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fc77fe640 1 -- 172.21.15.2:0/4797352 learned_addr learned my addr 172.21.15.2:0/4797352 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fc6ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed875f80 0x55c5ed887040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fc67fc640 1 --2- 172.21.15.2:0/4797352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed876970 0x55c5ed887580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fc77fe640 1 -- 172.21.15.2:0/4797352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed875f80 msgr2=0x55c5ed887040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fc77fe640 1 --2- 172.21.15.2:0/4797352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed875f80 0x55c5ed887040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fc77fe640 1 -- 172.21.15.2:0/4797352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed876970 msgr2=0x55c5ed887580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fc77fe640 1 --2- 172.21.15.2:0/4797352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed876970 0x55c5ed887580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fc77fe640 1 -- 172.21.15.2:0/4797352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5ed7dc2b0 con 0x55c5ed87bae0 2024-03-15T12:58:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8fc77fe640 1 --2- 172.21.15.2:0/4797352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed883ae0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0015040 tx=0x7f8fc000f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132b8631d040b5c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.650+0000 7f8faaff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.651+0000 7f8faaff5640 1 -- 172.21.15.2:0/4797352 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0010c20 con 0x55c5ed87bae0 2024-03-15T12:58:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.651+0000 7f8faaff5640 1 -- 172.21.15.2:0/4797352 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f8fc0010d80 con 0x55c5ed87bae0 2024-03-15T12:58:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.651+0000 7f8fd03f3740 1 -- 172.21.15.2:0/4797352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5ed88c800 con 0x55c5ed87bae0 2024-03-15T12:58:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.651+0000 7f8faaff5640 1 -- 172.21.15.2:0/4797352 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fc0028970 con 0x55c5ed87bae0 2024-03-15T12:58:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.651+0000 7f8fd03f3740 10 client.15528 fetch_fsmap learned FSMap version 28 2024-03-15T12:58:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.651+0000 7f8fd03f3740 1 -- 172.21.15.2:0/4797352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c5ed79e780 con 0x55c5ed87bae0 2024-03-15T12:58:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.651+0000 7f8faaff5640 1 -- 172.21.15.2:0/4797352 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55c5ed88c800 con 0x55c5ed87bae0 2024-03-15T12:58:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.651+0000 7f8faaff5640 1 -- 172.21.15.2:0/4797352 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8fc0046030 con 0x55c5ed87bae0 2024-03-15T12:58:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.651+0000 7f8fd03f3740 10 client.15528 fetch_fsmap finished waiting for FSMap version 28 2024-03-15T12:58:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.651+0000 7f8fd03f3740 10 client.15528 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-15T12:58:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.651+0000 7f8fd03f3740 20 client.15528 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.651+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/4797352 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c5ed8ec4d0 0x55c5ed8e8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:31.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.651+0000 7f8fd03f3740 4 client.15528 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-15T12:58:31.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.651+0000 7f8fd03f3740 1 -- 172.21.15.2:0/4797352 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f8fc000ee50 con 0x55c5ed8ec4d0 2024-03-15T12:58:31.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.651+0000 7f8fc6ffd640 1 --2- 172.21.15.2:0/4797352 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c5ed8ec4d0 0x55c5ed8e8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:31.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8faaff5640 10 client.15528 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:31.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8fc6ffd640 1 --2- 172.21.15.2:0/4797352 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c5ed8ec4d0 0x55c5ed8e8950 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fe674a2064efbfa server_cookie=2cb7f704909565f8 in_seq=0 out_seq=0 2024-03-15T12:58:31.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8faaff5640 1 -- 172.21.15.2:0/4797352 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f8fc000ee50 con 0x55c5ed8ec4d0 2024-03-15T12:58:31.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8faaff5640 10 client.15528 handle_command_reply: tid=0 2024-03-15T12:58:31.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8fd03f3740 1 client.15528 shutdown 2024-03-15T12:58:31.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8fd03f3740 1 -- 172.21.15.2:0/4797352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 msgr2=0x55c5ed883ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/4797352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed883ae0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0015040 tx=0x7f8fc000f880 comp rx=0 tx=0).stop 2024-03-15T12:58:31.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8fd03f3740 1 -- 172.21.15.2:0/4797352 shutdown_connections 2024-03-15T12:58:31.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/4797352 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c5ed8ec4d0 0x55c5ed8e8950 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/4797352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c5ed87bae0 0x55c5ed883ae0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8faaff5640 0 client.15528 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:31.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/4797352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ed876970 0x55c5ed887580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8fd03f3740 1 --2- 172.21.15.2:0/4797352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5ed875f80 0x55c5ed887040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8fd03f3740 1 -- 172.21.15.2:0/4797352 >> 172.21.15.2:0/4797352 conn(0x55c5ed8714f0 msgr2=0x55c5ed880220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8fd03f3740 1 -- 172.21.15.2:0/4797352 shutdown_connections 2024-03-15T12:58:31.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8fd03f3740 1 -- 172.21.15.2:0/4797352 wait complete. 2024-03-15T12:58:31.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.652+0000 7f8fd03f3740 20 client.15528 trim_cache size 0 max 16384 2024-03-15T12:58:31.645 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:58:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "2767718a-e2ca-11ee-95c9-87774f69a715", 2024-03-15T12:58:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2024-03-15T12:58:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24539, 2024-03-15T12:58:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-03-15T12:58:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-03-15T12:58:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-03-15T12:58:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 130.248986688, 2024-03-15T12:58:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 28, 2024-03-15T12:58:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 91, 2024-03-15T12:58:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 89, 2024-03-15T12:58:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 137.55537130799999 2024-03-15T12:58:31.646 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:58:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.653+0000 7f8fcd747640 1 -- 172.21.15.2:0/674103491 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8fb00a1a30 msgr2=0x7f8fb00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.653+0000 7f8fcd747640 1 --2- 172.21.15.2:0/674103491 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8fb00a1a30 0x7f8fb00a3ef0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0024000 tx=0x7f8fc0022000 comp rx=0 tx=0).stop 2024-03-15T12:58:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.653+0000 7f8fcd747640 1 -- 172.21.15.2:0/674103491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fc81026f0 msgr2=0x7f8fc8197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.653+0000 7f8fcd747640 1 --2- 172.21.15.2:0/674103491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fc81026f0 0x7f8fc8197ad0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc02c0b0 tx=0x7f8fbc032460 comp rx=0 tx=0).stop 2024-03-15T12:58:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.653+0000 7f8fcd747640 1 -- 172.21.15.2:0/674103491 shutdown_connections 2024-03-15T12:58:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.653+0000 7f8fcd747640 1 --2- 172.21.15.2:0/674103491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fc81030b0 0x7f8fc819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.653+0000 7f8fcd747640 1 --2- 172.21.15.2:0/674103491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fc81026f0 0x7f8fc8197ad0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.653+0000 7f8fcd747640 1 --2- 172.21.15.2:0/674103491 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8fb00a1a30 0x7f8fb00a3ef0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.653+0000 7f8fcd747640 1 --2- 172.21.15.2:0/674103491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8fc8101500 0x7f8fc8197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.653+0000 7f8fcd747640 1 -- 172.21.15.2:0/674103491 >> 172.21.15.2:0/674103491 conn(0x7f8fc80fca90 msgr2=0x7f8fc8105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.653+0000 7f8fcd747640 1 -- 172.21.15.2:0/674103491 shutdown_connections 2024-03-15T12:58:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:31.653+0000 7f8fcd747640 1 -- 172.21.15.2:0/674103491 wait complete. 2024-03-15T12:58:31.649 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-15T12:58:31.661 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-15T12:58:31.661 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-15T12:58:31.661 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-03-15T12:58:31.661 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-15T12:58:31.662 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-15T12:58:31.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:31.662 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-03-15T12:58:31.676 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-15T12:58:31.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:31.677 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-15T12:58:31.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:31.744 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2024-03-15T12:58:31.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:31 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/920766191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:31.827 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-15T12:58:31.828 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-15T12:58:32.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:31 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/920766191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:32.073 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T12:58:32.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:31 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/920766191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:32.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:32 smithi002 ceph-mon[32060]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-15T12:58:32.959 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:32 smithi184 ceph-mon[35993]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-15T12:58:33.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:32 smithi149 ceph-mon[35927]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-15T12:58:33.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.790+0000 7fea24b67700 1 -- 172.21.15.2:0/2262336772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea2006a630 msgr2=0x7fea2010ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:33.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.790+0000 7fea24b67700 1 --2- 172.21.15.2:0/2262336772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea2006a630 0x7fea2010ee70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fea1800b0d0 tx=0x7fea1800b3e0 comp rx=0 tx=0).stop 2024-03-15T12:58:33.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.790+0000 7fea24b67700 1 -- 172.21.15.2:0/2262336772 shutdown_connections 2024-03-15T12:58:33.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.790+0000 7fea24b67700 1 --2- 172.21.15.2:0/2262336772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea2006a630 0x7fea2010ee70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.790+0000 7fea24b67700 1 --2- 172.21.15.2:0/2262336772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea20108030 0x7fea20108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.790+0000 7fea24b67700 1 --2- 172.21.15.2:0/2262336772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea20069c90 0x7fea2006a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.790+0000 7fea24b67700 1 -- 172.21.15.2:0/2262336772 >> 172.21.15.2:0/2262336772 conn(0x7fea20077e10 msgr2=0x7fea20078220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.790+0000 7fea24b67700 1 -- 172.21.15.2:0/2262336772 shutdown_connections 2024-03-15T12:58:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.790+0000 7fea24b67700 1 -- 172.21.15.2:0/2262336772 wait complete. 2024-03-15T12:58:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.791+0000 7fea24b67700 1 Processor -- start 2024-03-15T12:58:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.791+0000 7fea24b67700 1 -- start start 2024-03-15T12:58:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.791+0000 7fea24b67700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea20069c90 0x7fea201a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.791+0000 7fea24b67700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea2006a630 0x7fea201a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.791+0000 7fea24b67700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea20108030 0x7fea2019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.791+0000 7fea24b67700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2019ae80 con 0x7fea20108030 2024-03-15T12:58:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.791+0000 7fea24b67700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2019aff0 con 0x7fea2006a630 2024-03-15T12:58:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.791+0000 7fea24b67700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2019b1b0 con 0x7fea20069c90 2024-03-15T12:58:33.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.791+0000 7fea1ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea20108030 0x7fea2019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:33.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.791+0000 7fea1ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea20108030 0x7fea2019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50538/0 (socket says 172.21.15.2:50538) 2024-03-15T12:58:33.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.791+0000 7fea1ed9d700 1 -- 172.21.15.2:0/4170964979 learned_addr learned my addr 172.21.15.2:0/4170964979 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:33.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.792+0000 7fea1dd9b700 1 --2- 172.21.15.2:0/4170964979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea2006a630 0x7fea201a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:33.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.792+0000 7fea1e59c700 1 --2- 172.21.15.2:0/4170964979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea20069c90 0x7fea201a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:33.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.792+0000 7fea1ed9d700 1 -- 172.21.15.2:0/4170964979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea20069c90 msgr2=0x7fea201a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:33.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.792+0000 7fea1ed9d700 1 --2- 172.21.15.2:0/4170964979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea20069c90 0x7fea201a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:33.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.792+0000 7fea1ed9d700 1 -- 172.21.15.2:0/4170964979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea2006a630 msgr2=0x7fea201a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:33.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.792+0000 7fea1ed9d700 1 --2- 172.21.15.2:0/4170964979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea2006a630 0x7fea201a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:33.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.792+0000 7fea1ed9d700 1 -- 172.21.15.2:0/4170964979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea18009d00 con 0x7fea20108030 2024-03-15T12:58:33.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.792+0000 7fea1ed9d700 1 --2- 172.21.15.2:0/4170964979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea20108030 0x7fea2019a6d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fea1800bd30 tx=0x7fea1800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:33.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.792+0000 7fea0f7fe700 1 -- 172.21.15.2:0/4170964979 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800e040 con 0x7fea20108030 2024-03-15T12:58:33.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.792+0000 7fea0f7fe700 1 -- 172.21.15.2:0/4170964979 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fea180079d0 con 0x7fea20108030 2024-03-15T12:58:33.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.792+0000 7fea24b67700 1 -- 172.21.15.2:0/4170964979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2019b430 con 0x7fea20108030 2024-03-15T12:58:33.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.792+0000 7fea0f7fe700 1 -- 172.21.15.2:0/4170964979 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1801b9b0 con 0x7fea20108030 2024-03-15T12:58:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.792+0000 7fea24b67700 1 -- 172.21.15.2:0/4170964979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea201a7580 con 0x7fea20108030 2024-03-15T12:58:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.793+0000 7fea24b67700 1 -- 172.21.15.2:0/4170964979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea201036e0 con 0x7fea20108030 2024-03-15T12:58:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.794+0000 7fea0f7fe700 1 -- 172.21.15.2:0/4170964979 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fea1801bb10 con 0x7fea20108030 2024-03-15T12:58:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.795+0000 7fea0f7fe700 1 --2- 172.21.15.2:0/4170964979 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fea080a1ac0 0x7fea080a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.795+0000 7fea1e59c700 1 --2- 172.21.15.2:0/4170964979 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fea080a1ac0 0x7fea080a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.795+0000 7fea0f7fe700 1 -- 172.21.15.2:0/4170964979 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fea180cf060 con 0x7fea20108030 2024-03-15T12:58:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.795+0000 7fea1e59c700 1 --2- 172.21.15.2:0/4170964979 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fea080a1ac0 0x7fea080a3f80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fea10009be0 tx=0x7fea10009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:33.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:33.798+0000 7fea0f7fe700 1 -- 172.21.15.2:0/4170964979 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fea18097f60 con 0x7fea20108030 2024-03-15T12:58:34.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:34.051+0000 7fea24b67700 1 -- 172.21.15.2:0/4170964979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fea20111ff0 con 0x7fea20108030 2024-03-15T12:58:34.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:34.051+0000 7fea0f7fe700 1 -- 172.21.15.2:0/4170964979 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7fea1809b620 con 0x7fea20108030 2024-03-15T12:58:34.037 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-15T12:58:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:34.054+0000 7fea24b67700 1 -- 172.21.15.2:0/4170964979 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fea080a1ac0 msgr2=0x7fea080a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:34.054+0000 7fea24b67700 1 --2- 172.21.15.2:0/4170964979 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fea080a1ac0 0x7fea080a3f80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fea10009be0 tx=0x7fea10009260 comp rx=0 tx=0).stop 2024-03-15T12:58:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:34.054+0000 7fea24b67700 1 -- 172.21.15.2:0/4170964979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea20108030 msgr2=0x7fea2019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:34.054+0000 7fea24b67700 1 --2- 172.21.15.2:0/4170964979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea20108030 0x7fea2019a6d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fea1800bd30 tx=0x7fea1800bd60 comp rx=0 tx=0).stop 2024-03-15T12:58:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:34.054+0000 7fea24b67700 1 -- 172.21.15.2:0/4170964979 shutdown_connections 2024-03-15T12:58:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:34.054+0000 7fea24b67700 1 --2- 172.21.15.2:0/4170964979 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fea080a1ac0 0x7fea080a3f80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:34.054+0000 7fea24b67700 1 --2- 172.21.15.2:0/4170964979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea20108030 0x7fea2019a6d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:34.054+0000 7fea24b67700 1 --2- 172.21.15.2:0/4170964979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea20069c90 0x7fea201a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:34.054+0000 7fea24b67700 1 --2- 172.21.15.2:0/4170964979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea2006a630 0x7fea201a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:34.054+0000 7fea24b67700 1 -- 172.21.15.2:0/4170964979 >> 172.21.15.2:0/4170964979 conn(0x7fea20077e10 msgr2=0x7fea2010d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:34.054+0000 7fea24b67700 1 -- 172.21.15.2:0/4170964979 shutdown_connections 2024-03-15T12:58:34.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:34.054+0000 7fea24b67700 1 -- 172.21.15.2:0/4170964979 wait complete. 2024-03-15T12:58:34.687 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-15T12:58:34.687 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-15T12:58:34.687 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi002.front.sepia.ceph.com 2024-03-15T12:58:34.687 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-15T12:58:34.687 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-15T12:58:34.688 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-15T12:58:34.688 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-15T12:58:34.688 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-15T12:58:34.688 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-15T12:58:34.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:34.688 DEBUG:teuthology.orchestra.run.smithi002:> ip addr 2024-03-15T12:58:34.705 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-15T12:58:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-15T12:58:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2024-03-15T12:58:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-03-15T12:58:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2024-03-15T12:58:34.705 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-03-15T12:58:34.706 INFO:teuthology.orchestra.run.smithi002.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-15T12:58:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:02 brd ff:ff:ff:ff:ff:ff 2024-03-15T12:58:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f0 2024-03-15T12:58:34.706 INFO:teuthology.orchestra.run.smithi002.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-15T12:58:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:03 brd ff:ff:ff:ff:ff:ff 2024-03-15T12:58:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f1 2024-03-15T12:58:34.706 INFO:teuthology.orchestra.run.smithi002.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-15T12:58:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:20 brd ff:ff:ff:ff:ff:ff 2024-03-15T12:58:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f0 2024-03-15T12:58:34.706 INFO:teuthology.orchestra.run.smithi002.stdout: inet 172.21.15.2/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-15T12:58:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft 42110sec preferred_lft 42110sec 2024-03-15T12:58:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::ec4:7aff:febd:1920/64 scope link noprefixroute 2024-03-15T12:58:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-03-15T12:58:34.707 INFO:teuthology.orchestra.run.smithi002.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-15T12:58:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:21 brd ff:ff:ff:ff:ff:ff 2024-03-15T12:58:34.707 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f1 2024-03-15T12:58:34.707 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-15T12:58:34.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:34.708 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2024-03-15T12:58:34.708 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-03-15T12:58:34.708 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add name ceph-brx type bridge 2024-03-15T12:58:34.708 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr flush dev ceph-brx 2024-03-15T12:58:34.708 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set ceph-brx up 2024-03-15T12:58:34.708 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-15T12:58:34.708 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-03-15T12:58:34.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:34.819 DEBUG:teuthology.orchestra.run.smithi002:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-15T12:58:34.846 INFO:teuthology.orchestra.run.smithi002.stdout:1 2024-03-15T12:58:34.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:34.848 DEBUG:teuthology.orchestra.run.smithi002:> route 2024-03-15T12:58:34.903 INFO:teuthology.orchestra.run.smithi002.stdout:Kernel IP routing table 2024-03-15T12:58:34.903 INFO:teuthology.orchestra.run.smithi002.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-15T12:58:34.903 INFO:teuthology.orchestra.run.smithi002.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-15T12:58:34.903 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-15T12:58:34.903 INFO:teuthology.orchestra.run.smithi002.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-15T12:58:34.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:34.904 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2024-03-15T12:58:34.904 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-03-15T12:58:34.904 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-15T12:58:34.904 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-15T12:58:34.904 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-15T12:58:34.904 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-03-15T12:58:34.968 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:34 smithi002 ceph-mon[32060]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:58:34.968 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:34 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/4170964979' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-15T12:58:35.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:35.000 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-03-15T12:58:35.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:35.055 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list-id 2024-03-15T12:58:35.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:34 smithi184 ceph-mon[35993]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:58:35.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:34 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/4170964979' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-15T12:58:35.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:35.109 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2024-03-15T12:58:35.109 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-03-15T12:58:35.109 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-15T12:58:35.109 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-15T12:58:35.109 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-03-15T12:58:35.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:34 smithi149 ceph-mon[35927]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:58:35.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:34 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/4170964979' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-15T12:58:35.191 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-15T12:58:35.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:35.191 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2024-03-15T12:58:35.191 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-03-15T12:58:35.192 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-15T12:58:35.192 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-15T12:58:35.192 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-15T12:58:35.192 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-15T12:58:35.192 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-15T12:58:35.192 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-03-15T12:58:35.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:35.318 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2024-03-15T12:58:35.319 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-03-15T12:58:35.319 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set brx.0 up 2024-03-15T12:58:35.319 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set dev brx.0 master ceph-brx 2024-03-15T12:58:35.319 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-03-15T12:58:35.360 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-03-15T12:58:35.360 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-15T12:58:35.360 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-15T12:58:35.414 INFO:teuthology.orchestra.run.smithi002.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-15T12:58:35.415 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-15T12:58:35.415 DEBUG:teuthology.orchestra.run.smithi002:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-15T12:58:35.469 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2024-03-15T12:58:35.537 DEBUG:teuthology.orchestra.run.smithi002:> cat /proc/self/mounts | awk '{print $2}' 2024-03-15T12:58:35.593 INFO:teuthology.orchestra.run.smithi002.stdout:/proc 2024-03-15T12:58:35.593 INFO:teuthology.orchestra.run.smithi002.stdout:/sys 2024-03-15T12:58:35.593 INFO:teuthology.orchestra.run.smithi002.stdout:/dev 2024-03-15T12:58:35.593 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/security 2024-03-15T12:58:35.593 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/shm 2024-03-15T12:58:35.593 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/pts 2024-03-15T12:58:35.593 INFO:teuthology.orchestra.run.smithi002.stdout:/run 2024-03-15T12:58:35.593 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup 2024-03-15T12:58:35.593 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/pstore 2024-03-15T12:58:35.594 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/bpf 2024-03-15T12:58:35.594 INFO:teuthology.orchestra.run.smithi002.stdout:/ 2024-03-15T12:58:35.594 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/selinux 2024-03-15T12:58:35.594 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-03-15T12:58:35.594 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/hugepages 2024-03-15T12:58:35.594 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/mqueue 2024-03-15T12:58:35.594 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/debug 2024-03-15T12:58:35.594 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/tracing 2024-03-15T12:58:35.594 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/fuse/connections 2024-03-15T12:58:35.594 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/config 2024-03-15T12:58:35.594 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-sysctl.service 2024-03-15T12:58:35.594 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-15T12:58:35.594 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-15T12:58:35.594 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/nfs/rpc_pipefs 2024-03-15T12:58:35.595 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/1000 2024-03-15T12:58:35.595 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/ceph 2024-03-15T12:58:35.595 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay 2024-03-15T12:58:35.595 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/ddc5a2df6c5f126ea7bcf954c77b16982897aafddcc8a8833a369632c9da48f6/merged 2024-03-15T12:58:35.595 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/c8cab69b1194350f1ab37f22fb9ecd4d572d553dadfa155815b6e3304aafd679/merged 2024-03-15T12:58:35.595 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/0 2024-03-15T12:58:35.595 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-03-15T12:58:35.595 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/de0edcba7fb75af99612c7e9340d1c85cbc048ba4ec53b132164902fc55e8f0d/merged 2024-03-15T12:58:35.595 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/b556dbd452654a7280471de853f40a675f63f2735c33d3f32328c9d6e269b0a4/merged 2024-03-15T12:58:35.595 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/0a20668dbd54c385f33152d7dfc70e158dec0f7a59dac3f9d737104ca8a54b0f/merged 2024-03-15T12:58:35.595 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f822ccf9d7b8f0ed2283a52b0123cd8ca69491f62a68b9dc18cab311c2a61eae/merged 2024-03-15T12:58:35.595 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/e0f808252d75766f7d109b8e9ac8e23500976feed51562045ffe3848dab1aa02/merged 2024-03-15T12:58:35.595 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/209819ff0d0fda5f82c3194ba05b1a73df3bd98ee861e678f17c1c4a8999cfc1/merged 2024-03-15T12:58:35.595 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/1246d9dc4a387cb1fec4afdef821c97b1407c045d315e204017bb0df0323f531/merged 2024-03-15T12:58:35.596 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/1b73366dc835c10d01b7f9eb61231037762c713aaa2c489e3875a3e0e8c64e94/merged 2024-03-15T12:58:35.596 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns 2024-03-15T12:58:35.596 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-15T12:58:35.596 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-15T12:58:35.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:35.596 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2024-03-15T12:58:35.648 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-15T12:58:35.648 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/6cf3dd6a-a2de-45bf-928c-1914473d1145 --client_fs=cephfs) 2024-03-15T12:58:35.691 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2024-03-15T12:58:35.717 DEBUG:teuthology.orchestra.run.smithi002:> cat /proc/self/mounts | awk '{print $2}' 2024-03-15T12:58:35.773 INFO:teuthology.orchestra.run.smithi002.stdout:/proc 2024-03-15T12:58:35.773 INFO:teuthology.orchestra.run.smithi002.stdout:/sys 2024-03-15T12:58:35.773 INFO:teuthology.orchestra.run.smithi002.stdout:/dev 2024-03-15T12:58:35.773 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/security 2024-03-15T12:58:35.774 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/shm 2024-03-15T12:58:35.774 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/pts 2024-03-15T12:58:35.774 INFO:teuthology.orchestra.run.smithi002.stdout:/run 2024-03-15T12:58:35.774 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup 2024-03-15T12:58:35.774 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/pstore 2024-03-15T12:58:35.774 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/bpf 2024-03-15T12:58:35.774 INFO:teuthology.orchestra.run.smithi002.stdout:/ 2024-03-15T12:58:35.774 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/selinux 2024-03-15T12:58:35.774 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-03-15T12:58:35.774 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/hugepages 2024-03-15T12:58:35.774 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/mqueue 2024-03-15T12:58:35.774 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/debug 2024-03-15T12:58:35.775 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/tracing 2024-03-15T12:58:35.775 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/fuse/connections 2024-03-15T12:58:35.775 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/config 2024-03-15T12:58:35.775 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-sysctl.service 2024-03-15T12:58:35.775 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-15T12:58:35.775 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-15T12:58:35.775 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/nfs/rpc_pipefs 2024-03-15T12:58:35.775 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/1000 2024-03-15T12:58:35.775 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/ceph 2024-03-15T12:58:35.775 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay 2024-03-15T12:58:35.775 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/ddc5a2df6c5f126ea7bcf954c77b16982897aafddcc8a8833a369632c9da48f6/merged 2024-03-15T12:58:35.775 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/c8cab69b1194350f1ab37f22fb9ecd4d572d553dadfa155815b6e3304aafd679/merged 2024-03-15T12:58:35.775 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/0 2024-03-15T12:58:35.775 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-03-15T12:58:35.776 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/de0edcba7fb75af99612c7e9340d1c85cbc048ba4ec53b132164902fc55e8f0d/merged 2024-03-15T12:58:35.776 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/b556dbd452654a7280471de853f40a675f63f2735c33d3f32328c9d6e269b0a4/merged 2024-03-15T12:58:35.776 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/0a20668dbd54c385f33152d7dfc70e158dec0f7a59dac3f9d737104ca8a54b0f/merged 2024-03-15T12:58:35.776 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f822ccf9d7b8f0ed2283a52b0123cd8ca69491f62a68b9dc18cab311c2a61eae/merged 2024-03-15T12:58:35.776 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/e0f808252d75766f7d109b8e9ac8e23500976feed51562045ffe3848dab1aa02/merged 2024-03-15T12:58:35.776 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/209819ff0d0fda5f82c3194ba05b1a73df3bd98ee861e678f17c1c4a8999cfc1/merged 2024-03-15T12:58:35.776 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/1246d9dc4a387cb1fec4afdef821c97b1407c045d315e204017bb0df0323f531/merged 2024-03-15T12:58:35.776 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/1b73366dc835c10d01b7f9eb61231037762c713aaa2c489e3875a3e0e8c64e94/merged 2024-03-15T12:58:35.776 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns 2024-03-15T12:58:35.776 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-15T12:58:35.776 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-15T12:58:35.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:35.777 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2024-03-15T12:58:35.778 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi002.stderr:2024-03-15T12:58:35.791+0000 7f63186f8480 -1 init, newargv = 0x55b7401cd540 newargc=15 2024-03-15T12:58:35.778 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi002.stderr:ceph-fuse[86457]: starting ceph client 2024-03-15T12:58:35.798 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi002.stderr:ceph-fuse[86457]: starting fuse 2024-03-15T12:58:36.893 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2024-03-15T12:58:36.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:36 smithi002 ceph-mon[32060]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-15T12:58:36.927 DEBUG:teuthology.orchestra.run.smithi002:> cat /proc/self/mounts | awk '{print $2}' 2024-03-15T12:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout:/proc 2024-03-15T12:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout:/sys 2024-03-15T12:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout:/dev 2024-03-15T12:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/security 2024-03-15T12:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/shm 2024-03-15T12:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/pts 2024-03-15T12:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout:/run 2024-03-15T12:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup 2024-03-15T12:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/pstore 2024-03-15T12:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/bpf 2024-03-15T12:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout:/ 2024-03-15T12:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/selinux 2024-03-15T12:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-03-15T12:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/hugepages 2024-03-15T12:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/mqueue 2024-03-15T12:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/debug 2024-03-15T12:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/tracing 2024-03-15T12:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/fuse/connections 2024-03-15T12:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/config 2024-03-15T12:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-sysctl.service 2024-03-15T12:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-15T12:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-15T12:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/nfs/rpc_pipefs 2024-03-15T12:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/1000 2024-03-15T12:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/ceph 2024-03-15T12:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay 2024-03-15T12:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/ddc5a2df6c5f126ea7bcf954c77b16982897aafddcc8a8833a369632c9da48f6/merged 2024-03-15T12:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/c8cab69b1194350f1ab37f22fb9ecd4d572d553dadfa155815b6e3304aafd679/merged 2024-03-15T12:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/0 2024-03-15T12:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-03-15T12:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/de0edcba7fb75af99612c7e9340d1c85cbc048ba4ec53b132164902fc55e8f0d/merged 2024-03-15T12:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/b556dbd452654a7280471de853f40a675f63f2735c33d3f32328c9d6e269b0a4/merged 2024-03-15T12:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/0a20668dbd54c385f33152d7dfc70e158dec0f7a59dac3f9d737104ca8a54b0f/merged 2024-03-15T12:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f822ccf9d7b8f0ed2283a52b0123cd8ca69491f62a68b9dc18cab311c2a61eae/merged 2024-03-15T12:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/e0f808252d75766f7d109b8e9ac8e23500976feed51562045ffe3848dab1aa02/merged 2024-03-15T12:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/209819ff0d0fda5f82c3194ba05b1a73df3bd98ee861e678f17c1c4a8999cfc1/merged 2024-03-15T12:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/1246d9dc4a387cb1fec4afdef821c97b1407c045d315e204017bb0df0323f531/merged 2024-03-15T12:58:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/1b73366dc835c10d01b7f9eb61231037762c713aaa2c489e3875a3e0e8c64e94/merged 2024-03-15T12:58:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns 2024-03-15T12:58:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-15T12:58:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-15T12:58:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-15T12:58:36.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:36.988 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2024-03-15T12:58:37.045 INFO:teuthology.orchestra.run.smithi002.stdout:119 2024-03-15T12:58:37.045 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-03-15T12:58:37.045 DEBUG:teuthology.orchestra.run.smithi002:> sudo stdin-killer -- python3 -c ' 2024-03-15T12:58:37.045 DEBUG:teuthology.orchestra.run.smithi002:> import glob 2024-03-15T12:58:37.045 DEBUG:teuthology.orchestra.run.smithi002:> import re 2024-03-15T12:58:37.045 DEBUG:teuthology.orchestra.run.smithi002:> import os 2024-03-15T12:58:37.045 DEBUG:teuthology.orchestra.run.smithi002:> import subprocess 2024-03-15T12:58:37.045 DEBUG:teuthology.orchestra.run.smithi002:> 2024-03-15T12:58:37.045 DEBUG:teuthology.orchestra.run.smithi002:> def _find_admin_socket(client_name): 2024-03-15T12:58:37.045 DEBUG:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-15T12:58:37.046 DEBUG:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2024-03-15T12:58:37.046 DEBUG:teuthology.orchestra.run.smithi002:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-15T12:58:37.046 DEBUG:teuthology.orchestra.run.smithi002:> 2024-03-15T12:58:37.046 DEBUG:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2024-03-15T12:58:37.046 DEBUG:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2024-03-15T12:58:37.046 DEBUG:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2024-03-15T12:58:37.046 DEBUG:teuthology.orchestra.run.smithi002:> return files[0] 2024-03-15T12:58:37.046 DEBUG:teuthology.orchestra.run.smithi002:> 2024-03-15T12:58:37.046 DEBUG:teuthology.orchestra.run.smithi002:> for f in files: 2024-03-15T12:58:37.046 DEBUG:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-15T12:58:37.046 DEBUG:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-15T12:58:37.046 DEBUG:teuthology.orchestra.run.smithi002:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-15T12:58:37.046 DEBUG:teuthology.orchestra.run.smithi002:> contents = proc_f.read() 2024-03-15T12:58:37.046 DEBUG:teuthology.orchestra.run.smithi002:> if mountpoint in contents: 2024-03-15T12:58:37.047 DEBUG:teuthology.orchestra.run.smithi002:> return f 2024-03-15T12:58:37.047 DEBUG:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-15T12:58:37.047 DEBUG:teuthology.orchestra.run.smithi002:> 2024-03-15T12:58:37.047 DEBUG:teuthology.orchestra.run.smithi002:> print(_find_admin_socket("client.0")) 2024-03-15T12:58:37.047 DEBUG:teuthology.orchestra.run.smithi002:> ' 2024-03-15T12:58:37.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:36 smithi184 ceph-mon[35993]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-15T12:58:37.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:36 smithi149 ceph-mon[35927]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-15T12:58:37.176 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.0.86457.asok 2024-03-15T12:58:37.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-15T12:58:37.188 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.86457.asok 2024-03-15T12:58:37.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:37.188 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.86457.asok status 2024-03-15T12:58:37.327 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:58:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2024-03-15T12:58:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b", 2024-03-15T12:58:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 19.0.0-1681-gaba3c1d2 (aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b) squid (dev)", 2024-03-15T12:58:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "0", 2024-03-15T12:58:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2024-03-15T12:58:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-15T12:58:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "86457", 2024-03-15T12:58:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/volumes/qa/sv_0/6cf3dd6a-a2de-45bf-928c-1914473d1145" 2024-03-15T12:58:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T12:58:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2024-03-15T12:58:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2024-03-15T12:58:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 15540, 2024-03-15T12:58:37.328 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 15540 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "192.168.0.1:0", 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1117658858 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "192.168.0.1:0", 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1117658858 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T12:58:37.329 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.15540 192.168.0.1:0/1117658858", 2024-03-15T12:58:37.330 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "192.168.0.1:0/1117658858", 2024-03-15T12:58:37.330 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 5, 2024-03-15T12:58:37.330 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 28, 2024-03-15T12:58:37.330 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 91, 2024-03-15T12:58:37.330 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2024-03-15T12:58:37.330 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklisted": false, 2024-03-15T12:58:37.330 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs" 2024-03-15T12:58:37.330 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:58:37.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:37.336 DEBUG:teuthology.orchestra.run.smithi002:> stat --file-system '--printf=%T 2024-03-15T12:58:37.336 DEBUG:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-15T12:58:37.351 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2024-03-15T12:58:37.351 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-15T12:58:37.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:37.351 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-15T12:58:37.421 INFO:teuthology.run_tasks:Running task exec... 2024-03-15T12:58:37.439 INFO:teuthology.task.exec:Executing custom commands... 2024-03-15T12:58:37.439 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-15T12:58:37.439 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-15T12:58:37.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.578+0000 7fcb57283640 1 -- 172.21.15.2:0/1546502220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb50104910 msgr2=0x7fcb50106d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:37.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.578+0000 7fcb57283640 1 --2- 172.21.15.2:0/1546502220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb50104910 0x7fcb50106d00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009900 tx=0x7fcb4402bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:37.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.578+0000 7fcb57283640 1 -- 172.21.15.2:0/1546502220 shutdown_connections 2024-03-15T12:58:37.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.578+0000 7fcb57283640 1 --2- 172.21.15.2:0/1546502220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb50107240 0x7fcb50109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:37.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.578+0000 7fcb57283640 1 --2- 172.21.15.2:0/1546502220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb50104910 0x7fcb50106d00 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:37.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.578+0000 7fcb57283640 1 --2- 172.21.15.2:0/1546502220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb50101fe0 0x7fcb501043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:37.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.578+0000 7fcb57283640 1 -- 172.21.15.2:0/1546502220 >> 172.21.15.2:0/1546502220 conn(0x7fcb500fb9d0 msgr2=0x7fcb500fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:37.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.578+0000 7fcb57283640 1 -- 172.21.15.2:0/1546502220 shutdown_connections 2024-03-15T12:58:37.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.578+0000 7fcb57283640 1 -- 172.21.15.2:0/1546502220 wait complete. 2024-03-15T12:58:37.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.579+0000 7fcb57283640 1 Processor -- start 2024-03-15T12:58:37.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.579+0000 7fcb57283640 1 -- start start 2024-03-15T12:58:37.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.579+0000 7fcb57283640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb50101fe0 0x7fcb50101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:37.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.579+0000 7fcb57283640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb50104910 0x7fcb500fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:37.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.579+0000 7fcb57283640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb50107240 0x7fcb501004d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.579+0000 7fcb57283640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb50100c30 con 0x7fcb50107240 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.579+0000 7fcb57283640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb50100da0 con 0x7fcb50104910 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.579+0000 7fcb57283640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb501a1580 con 0x7fcb50101fe0 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.579+0000 7fcb55820640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb50107240 0x7fcb501004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.579+0000 7fcb5501f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb50101fe0 0x7fcb50101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.579+0000 7fcb5501f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb50101fe0 0x7fcb50101950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:40566/0 (socket says 172.21.15.2:40566) 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.579+0000 7fcb5481e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb50104910 0x7fcb500fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.579+0000 7fcb5501f640 1 -- 172.21.15.2:0/2043584512 learned_addr learned my addr 172.21.15.2:0/2043584512 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.580+0000 7fcb55820640 1 -- 172.21.15.2:0/2043584512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb50101fe0 msgr2=0x7fcb50101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.580+0000 7fcb55820640 1 --2- 172.21.15.2:0/2043584512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb50101fe0 0x7fcb50101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.580+0000 7fcb55820640 1 -- 172.21.15.2:0/2043584512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb50104910 msgr2=0x7fcb500fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.580+0000 7fcb55820640 1 --2- 172.21.15.2:0/2043584512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb50104910 0x7fcb500fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.580+0000 7fcb55820640 1 -- 172.21.15.2:0/2043584512 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb44009560 con 0x7fcb50107240 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.580+0000 7fcb5501f640 1 --2- 172.21.15.2:0/2043584512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb50101fe0 0x7fcb50101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.580+0000 7fcb55820640 1 --2- 172.21.15.2:0/2043584512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb50107240 0x7fcb501004d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000ea80 tx=0x7fcb4000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.580+0000 7fcb4e7fc640 1 -- 172.21.15.2:0/2043584512 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4000cea0 con 0x7fcb50107240 2024-03-15T12:58:37.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.580+0000 7fcb4e7fc640 1 -- 172.21.15.2:0/2043584512 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcb40005020 con 0x7fcb50107240 2024-03-15T12:58:37.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.580+0000 7fcb57283640 1 -- 172.21.15.2:0/2043584512 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb501a1830 con 0x7fcb50107240 2024-03-15T12:58:37.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.580+0000 7fcb4e7fc640 1 -- 172.21.15.2:0/2043584512 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40010690 con 0x7fcb50107240 2024-03-15T12:58:37.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.580+0000 7fcb57283640 1 -- 172.21.15.2:0/2043584512 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb501a1cc0 con 0x7fcb50107240 2024-03-15T12:58:37.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.581+0000 7fcb57283640 1 -- 172.21.15.2:0/2043584512 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb500fd4c0 con 0x7fcb50107240 2024-03-15T12:58:37.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.582+0000 7fcb4e7fc640 1 -- 172.21.15.2:0/2043584512 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fcb40005190 con 0x7fcb50107240 2024-03-15T12:58:37.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.582+0000 7fcb4e7fc640 1 --2- 172.21.15.2:0/2043584512 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcb380a1ad0 0x7fcb380a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:37.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.583+0000 7fcb4e7fc640 1 -- 172.21.15.2:0/2043584512 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fcb40014070 con 0x7fcb50107240 2024-03-15T12:58:37.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.583+0000 7fcb5501f640 1 --2- 172.21.15.2:0/2043584512 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcb380a1ad0 0x7fcb380a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:37.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.583+0000 7fcb5501f640 1 --2- 172.21.15.2:0/2043584512 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcb380a1ad0 0x7fcb380a3f90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcb48011500 tx=0x7fcb48005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:37.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.587+0000 7fcb4e7fc640 1 -- 172.21.15.2:0/2043584512 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcb40096cb0 con 0x7fcb50107240 2024-03-15T12:58:37.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:37.812+0000 7fcb57283640 1 -- 172.21.15.2:0/2043584512 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7fcb5002d000 con 0x7fcb50107240 2024-03-15T12:58:38.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.685+0000 7fcb4e7fc640 1 -- 172.21.15.2:0/2043584512 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcb4009a370 con 0x7fcb50107240 2024-03-15T12:58:38.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.688+0000 7fcb57283640 1 -- 172.21.15.2:0/2043584512 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcb380a1ad0 msgr2=0x7fcb380a3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:38.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.688+0000 7fcb57283640 1 --2- 172.21.15.2:0/2043584512 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcb380a1ad0 0x7fcb380a3f90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcb48011500 tx=0x7fcb48005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:38.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.688+0000 7fcb57283640 1 -- 172.21.15.2:0/2043584512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb50107240 msgr2=0x7fcb501004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:38.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.688+0000 7fcb57283640 1 --2- 172.21.15.2:0/2043584512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb50107240 0x7fcb501004d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000ea80 tx=0x7fcb4000ef40 comp rx=0 tx=0).stop 2024-03-15T12:58:38.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.688+0000 7fcb57283640 1 -- 172.21.15.2:0/2043584512 shutdown_connections 2024-03-15T12:58:38.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.688+0000 7fcb57283640 1 --2- 172.21.15.2:0/2043584512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb50107240 0x7fcb501004d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:38.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.688+0000 7fcb57283640 1 --2- 172.21.15.2:0/2043584512 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fcb380a1ad0 0x7fcb380a3f90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:38.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.688+0000 7fcb57283640 1 --2- 172.21.15.2:0/2043584512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb50104910 0x7fcb500fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:38.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.688+0000 7fcb57283640 1 --2- 172.21.15.2:0/2043584512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb50101fe0 0x7fcb50101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:38.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.688+0000 7fcb57283640 1 -- 172.21.15.2:0/2043584512 >> 172.21.15.2:0/2043584512 conn(0x7fcb500fb9d0 msgr2=0x7fcb50102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:38.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.689+0000 7fcb57283640 1 -- 172.21.15.2:0/2043584512 shutdown_connections 2024-03-15T12:58:38.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.689+0000 7fcb57283640 1 -- 172.21.15.2:0/2043584512 wait complete. 2024-03-15T12:58:38.711 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-15T12:58:38.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.839+0000 7f5fa309a640 1 -- 172.21.15.2:0/244845816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f9c101fe0 msgr2=0x7f5f9c108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:38.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.839+0000 7f5fa309a640 1 --2- 172.21.15.2:0/244845816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f9c101fe0 0x7f5f9c108870 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009ca0 tx=0x7f5f9402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:38.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.839+0000 7f5fa309a640 1 -- 172.21.15.2:0/244845816 shutdown_connections 2024-03-15T12:58:38.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.839+0000 7f5fa309a640 1 --2- 172.21.15.2:0/244845816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f9c101fe0 0x7f5f9c108870 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:38.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.839+0000 7f5fa309a640 1 --2- 172.21.15.2:0/244845816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f9c101620 0x7f5f9c101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:38.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.839+0000 7f5fa309a640 1 --2- 172.21.15.2:0/244845816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f9c100430 0x7f5f9c100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:38.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.839+0000 7f5fa309a640 1 -- 172.21.15.2:0/244845816 >> 172.21.15.2:0/244845816 conn(0x7f5f9c0fb9a0 msgr2=0x7f5f9c0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:38.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.839+0000 7f5fa309a640 1 -- 172.21.15.2:0/244845816 shutdown_connections 2024-03-15T12:58:38.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.840+0000 7f5fa309a640 1 -- 172.21.15.2:0/244845816 wait complete. 2024-03-15T12:58:38.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.840+0000 7f5fa309a640 1 Processor -- start 2024-03-15T12:58:38.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.840+0000 7f5fa309a640 1 -- start start 2024-03-15T12:58:38.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.840+0000 7f5fa309a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f9c100430 0x7f5f9c196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:38.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.840+0000 7f5fa309a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f9c101620 0x7f5f9c1968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:38.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.840+0000 7f5fa309a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f9c101fe0 0x7f5f9c19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:38.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.840+0000 7f5fa309a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c19de50 con 0x7f5f9c100430 2024-03-15T12:58:38.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.840+0000 7f5fa309a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c19dfc0 con 0x7f5f9c101620 2024-03-15T12:58:38.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.840+0000 7f5fa309a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c19e180 con 0x7f5f9c101fe0 2024-03-15T12:58:38.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.840+0000 7f5fa0e36640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f9c100430 0x7f5f9c196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:38.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.840+0000 7f5fa0e36640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f9c100430 0x7f5f9c196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43434/0 (socket says 172.21.15.2:43434) 2024-03-15T12:58:38.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.840+0000 7f5fa0e36640 1 -- 172.21.15.2:0/328895473 learned_addr learned my addr 172.21.15.2:0/328895473 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:38.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.840+0000 7f5f9bfff640 1 --2- 172.21.15.2:0/328895473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f9c101620 0x7f5f9c1968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:38.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.840+0000 7f5fa1637640 1 --2- 172.21.15.2:0/328895473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f9c101fe0 0x7f5f9c19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:38.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.841+0000 7f5fa0e36640 1 -- 172.21.15.2:0/328895473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f9c101fe0 msgr2=0x7f5f9c19d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:38.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.841+0000 7f5fa0e36640 1 --2- 172.21.15.2:0/328895473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f9c101fe0 0x7f5f9c19d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:38.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.841+0000 7f5fa0e36640 1 -- 172.21.15.2:0/328895473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f9c101620 msgr2=0x7f5f9c1968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:38.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.841+0000 7f5fa0e36640 1 --2- 172.21.15.2:0/328895473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f9c101620 0x7f5f9c1968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:38.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.841+0000 7f5fa0e36640 1 -- 172.21.15.2:0/328895473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f94009900 con 0x7f5f9c100430 2024-03-15T12:58:38.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.841+0000 7f5fa0e36640 1 --2- 172.21.15.2:0/328895473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f9c100430 0x7f5f9c196360 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00e9c0 tx=0x7f5f8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:38.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.841+0000 7f5f99ffb640 1 -- 172.21.15.2:0/328895473 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c009700 con 0x7f5f9c100430 2024-03-15T12:58:38.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.841+0000 7f5fa309a640 1 -- 172.21.15.2:0/328895473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9c19e460 con 0x7f5f9c100430 2024-03-15T12:58:38.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.841+0000 7f5f99ffb640 1 -- 172.21.15.2:0/328895473 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5f8c0044f0 con 0x7f5f9c100430 2024-03-15T12:58:38.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.841+0000 7f5f99ffb640 1 -- 172.21.15.2:0/328895473 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c010430 con 0x7f5f9c100430 2024-03-15T12:58:38.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.841+0000 7f5fa309a640 1 -- 172.21.15.2:0/328895473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9c19e970 con 0x7f5f9c100430 2024-03-15T12:58:38.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.842+0000 7f5fa309a640 1 -- 172.21.15.2:0/328895473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9c107680 con 0x7f5f9c100430 2024-03-15T12:58:38.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.843+0000 7f5f99ffb640 1 -- 172.21.15.2:0/328895473 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f5f8c003fe0 con 0x7f5f9c100430 2024-03-15T12:58:38.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.844+0000 7f5f99ffb640 1 --2- 172.21.15.2:0/328895473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5f840a1ad0 0x7f5f840a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:38.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.844+0000 7f5f9bfff640 1 --2- 172.21.15.2:0/328895473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5f840a1ad0 0x7f5f840a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:38.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.844+0000 7f5f99ffb640 1 -- 172.21.15.2:0/328895473 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f5f8c014070 con 0x7f5f9c100430 2024-03-15T12:58:38.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.844+0000 7f5f9bfff640 1 --2- 172.21.15.2:0/328895473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5f840a1ad0 0x7f5f840a3f90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c197870 tx=0x7f5f90005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:38.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:38.848+0000 7f5f99ffb640 1 -- 172.21.15.2:0/328895473 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5f8c0969f0 con 0x7f5f9c100430 2024-03-15T12:58:39.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.058+0000 7f5fa309a640 1 -- 172.21.15.2:0/328895473 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f9c02d070 con 0x7f5f840a1ad0 2024-03-15T12:58:39.049 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:38 smithi002 ceph-mon[32060]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:58:39.049 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:38 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2043584512' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-15T12:58:39.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:38 smithi184 ceph-mon[35993]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:58:39.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:38 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2043584512' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-15T12:58:39.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.080+0000 7f5f99ffb640 1 -- 172.21.15.2:0/328895473 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f5f9c02d070 con 0x7f5f840a1ad0 2024-03-15T12:58:39.066 INFO:teuthology.orchestra.run.smithi002.stdout:{} 2024-03-15T12:58:39.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.082+0000 7f5fa309a640 1 -- 172.21.15.2:0/328895473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5f840a1ad0 msgr2=0x7f5f840a3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:39.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.082+0000 7f5fa309a640 1 --2- 172.21.15.2:0/328895473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5f840a1ad0 0x7f5f840a3f90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c197870 tx=0x7f5f90005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:39.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.082+0000 7f5fa309a640 1 -- 172.21.15.2:0/328895473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f9c100430 msgr2=0x7f5f9c196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:39.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.082+0000 7f5fa309a640 1 --2- 172.21.15.2:0/328895473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f9c100430 0x7f5f9c196360 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00e9c0 tx=0x7f5f8c00ee80 comp rx=0 tx=0).stop 2024-03-15T12:58:39.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.082+0000 7f5fa309a640 1 -- 172.21.15.2:0/328895473 shutdown_connections 2024-03-15T12:58:39.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.082+0000 7f5fa309a640 1 --2- 172.21.15.2:0/328895473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f9c101fe0 0x7f5f9c19d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.082+0000 7f5fa309a640 1 --2- 172.21.15.2:0/328895473 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f5f840a1ad0 0x7f5f840a3f90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.082+0000 7f5fa309a640 1 --2- 172.21.15.2:0/328895473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f9c101620 0x7f5f9c1968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.082+0000 7f5fa309a640 1 --2- 172.21.15.2:0/328895473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f9c100430 0x7f5f9c196360 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.082+0000 7f5fa309a640 1 -- 172.21.15.2:0/328895473 >> 172.21.15.2:0/328895473 conn(0x7f5f9c0fb9a0 msgr2=0x7f5f9c104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:39.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.082+0000 7f5fa309a640 1 -- 172.21.15.2:0/328895473 shutdown_connections 2024-03-15T12:58:39.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.082+0000 7f5fa309a640 1 -- 172.21.15.2:0/328895473 wait complete. 2024-03-15T12:58:39.079 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-15T12:58:39.090 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-15T12:58:39.102 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:58:39.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:38 smithi149 ceph-mon[35927]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-15T12:58:39.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:38 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2043584512' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-15T12:58:39.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.218+0000 7f48d51c7640 1 -- 172.21.15.2:0/1055267313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48d00ffba0 msgr2=0x7f48d00fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:39.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.218+0000 7f48d51c7640 1 --2- 172.21.15.2:0/1055267313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48d00ffba0 0x7f48d00fffc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009900 tx=0x7f48c002bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:39.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.218+0000 7f48d51c7640 1 -- 172.21.15.2:0/1055267313 shutdown_connections 2024-03-15T12:58:39.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.218+0000 7f48d51c7640 1 --2- 172.21.15.2:0/1055267313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d0101750 0x7f48d0107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.218+0000 7f48d51c7640 1 --2- 172.21.15.2:0/1055267313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48d0100d90 0x7f48d0101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.218+0000 7f48d51c7640 1 --2- 172.21.15.2:0/1055267313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48d00ffba0 0x7f48d00fffc0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.218+0000 7f48d51c7640 1 -- 172.21.15.2:0/1055267313 >> 172.21.15.2:0/1055267313 conn(0x7f48d00fb110 msgr2=0x7f48d00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:39.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.218+0000 7f48d51c7640 1 -- 172.21.15.2:0/1055267313 shutdown_connections 2024-03-15T12:58:39.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.218+0000 7f48d51c7640 1 -- 172.21.15.2:0/1055267313 wait complete. 2024-03-15T12:58:39.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48d51c7640 1 Processor -- start 2024-03-15T12:58:39.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48d51c7640 1 -- start start 2024-03-15T12:58:39.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48d51c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48d00ffba0 0x7f48d0109470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:39.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48d51c7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48d0100d90 0x7f48d019c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:39.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48d51c7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d0101750 0x7f48d019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:39.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48d51c7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d019f0f0 con 0x7f48d00ffba0 2024-03-15T12:58:39.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48d51c7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d019f260 con 0x7f48d0101750 2024-03-15T12:58:39.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48d51c7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d019f420 con 0x7f48d0100d90 2024-03-15T12:58:39.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48ced9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48d00ffba0 0x7f48d0109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:39.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48ced9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48d00ffba0 0x7f48d0109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43460/0 (socket says 172.21.15.2:43460) 2024-03-15T12:58:39.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48ced9d640 1 -- 172.21.15.2:0/1090105745 learned_addr learned my addr 172.21.15.2:0/1090105745 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:39.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48c7fff640 1 --2- 172.21.15.2:0/1090105745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48d0100d90 0x7f48d019c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:39.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48cf59e640 1 --2- 172.21.15.2:0/1090105745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d0101750 0x7f48d019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:39.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48ced9d640 1 -- 172.21.15.2:0/1090105745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48d0100d90 msgr2=0x7f48d019c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:39.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48ced9d640 1 --2- 172.21.15.2:0/1090105745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48d0100d90 0x7f48d019c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48ced9d640 1 -- 172.21.15.2:0/1090105745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d0101750 msgr2=0x7f48d019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:39.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48ced9d640 1 --2- 172.21.15.2:0/1090105745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d0101750 0x7f48d019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.219+0000 7f48ced9d640 1 -- 172.21.15.2:0/1090105745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c0009560 con 0x7f48d00ffba0 2024-03-15T12:58:39.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.220+0000 7f48ced9d640 1 --2- 172.21.15.2:0/1090105745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48d00ffba0 0x7f48d0109470 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f48c0000c00 tx=0x7f48c00049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:39.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.220+0000 7f48ccd99640 1 -- 172.21.15.2:0/1090105745 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c0004250 con 0x7f48d00ffba0 2024-03-15T12:58:39.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.220+0000 7f48ccd99640 1 -- 172.21.15.2:0/1090105745 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f48c00043b0 con 0x7f48d00ffba0 2024-03-15T12:58:39.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.220+0000 7f48ccd99640 1 -- 172.21.15.2:0/1090105745 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c0003d50 con 0x7f48d00ffba0 2024-03-15T12:58:39.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.220+0000 7f48d51c7640 1 -- 172.21.15.2:0/1090105745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d019f6a0 con 0x7f48d00ffba0 2024-03-15T12:58:39.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.220+0000 7f48d51c7640 1 -- 172.21.15.2:0/1090105745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d019fbb0 con 0x7f48d00ffba0 2024-03-15T12:58:39.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.221+0000 7f48d51c7640 1 -- 172.21.15.2:0/1090105745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d0106f90 con 0x7f48d00ffba0 2024-03-15T12:58:39.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.222+0000 7f48ccd99640 1 -- 172.21.15.2:0/1090105745 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f48c0005000 con 0x7f48d00ffba0 2024-03-15T12:58:39.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.222+0000 7f48ccd99640 1 --2- 172.21.15.2:0/1090105745 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f48bc0a1a30 0x7f48bc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:39.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.222+0000 7f48c7fff640 1 --2- 172.21.15.2:0/1090105745 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f48bc0a1a30 0x7f48bc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:39.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.222+0000 7f48ccd99640 1 -- 172.21.15.2:0/1090105745 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f48c00ef960 con 0x7f48d00ffba0 2024-03-15T12:58:39.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.223+0000 7f48c7fff640 1 --2- 172.21.15.2:0/1090105745 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f48bc0a1a30 0x7f48bc0a3ef0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f48d01091e0 tx=0x7f48b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:39.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.227+0000 7f48ccd99640 1 -- 172.21.15.2:0/1090105745 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f48c00b8830 con 0x7f48d00ffba0 2024-03-15T12:58:39.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:39 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:39.062+0000 7fa426449700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:58:39.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:39 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:39.062+0000 7fa426449700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:58:39.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:39 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:39.062+0000 7fa426449700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:58:39.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:39 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:39.062+0000 7fa426449700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:58:39.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:39 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:39.062+0000 7fa426449700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:58:39.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.410+0000 7f48d51c7640 1 -- 172.21.15.2:0/1090105745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48d010bdf0 con 0x7f48d00ffba0 2024-03-15T12:58:39.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.412+0000 7f48ccd99640 1 -- 172.21.15.2:0/1090105745 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24526 (secure 0 0 0) 0x7f48c00bbef0 con 0x7f48d00ffba0 2024-03-15T12:58:39.398 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:39.398 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":91,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:57:19.826817+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:58:39.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.413+0000 7f48d51c7640 1 -- 172.21.15.2:0/1090105745 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f48bc0a1a30 msgr2=0x7f48bc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:39.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.413+0000 7f48d51c7640 1 --2- 172.21.15.2:0/1090105745 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f48bc0a1a30 0x7f48bc0a3ef0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f48d01091e0 tx=0x7f48b8005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:39.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.414+0000 7f48d51c7640 1 -- 172.21.15.2:0/1090105745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48d00ffba0 msgr2=0x7f48d0109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:39.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.414+0000 7f48d51c7640 1 --2- 172.21.15.2:0/1090105745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48d00ffba0 0x7f48d0109470 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f48c0000c00 tx=0x7f48c00049e0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.414+0000 7f48d51c7640 1 -- 172.21.15.2:0/1090105745 shutdown_connections 2024-03-15T12:58:39.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.414+0000 7f48d51c7640 1 --2- 172.21.15.2:0/1090105745 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f48bc0a1a30 0x7f48bc0a3ef0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.414+0000 7f48d51c7640 1 --2- 172.21.15.2:0/1090105745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d0101750 0x7f48d019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.414+0000 7f48d51c7640 1 --2- 172.21.15.2:0/1090105745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48d0100d90 0x7f48d019c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.414+0000 7f48d51c7640 1 --2- 172.21.15.2:0/1090105745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48d00ffba0 0x7f48d0109470 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.414+0000 7f48d51c7640 1 -- 172.21.15.2:0/1090105745 >> 172.21.15.2:0/1090105745 conn(0x7f48d00fb110 msgr2=0x7f48d0104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:39.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.414+0000 7f48d51c7640 1 -- 172.21.15.2:0/1090105745 shutdown_connections 2024-03-15T12:58:39.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.414+0000 7f48d51c7640 1 -- 172.21.15.2:0/1090105745 wait complete. 2024-03-15T12:58:39.411 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-15T12:51:47.004843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-15T12:56:18.559882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-15T12:56:19.751767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-15T12:56:23.664879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-15T12:58:39.412 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-15T12:58:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.532+0000 7f1562a23640 1 -- 172.21.15.2:0/4183952950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c101fc0 msgr2=0x7f155c108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.532+0000 7f1562a23640 1 --2- 172.21.15.2:0/4183952950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c101fc0 0x7f155c108850 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1554009be0 tx=0x7f155402bef0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.532+0000 7f1562a23640 1 -- 172.21.15.2:0/4183952950 shutdown_connections 2024-03-15T12:58:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.532+0000 7f1562a23640 1 --2- 172.21.15.2:0/4183952950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c101fc0 0x7f155c108850 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.532+0000 7f1562a23640 1 --2- 172.21.15.2:0/4183952950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f155c101600 0x7f155c101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.532+0000 7f1562a23640 1 --2- 172.21.15.2:0/4183952950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f155c100410 0x7f155c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.532+0000 7f1562a23640 1 -- 172.21.15.2:0/4183952950 >> 172.21.15.2:0/4183952950 conn(0x7f155c0fb9a0 msgr2=0x7f155c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.532+0000 7f1562a23640 1 -- 172.21.15.2:0/4183952950 shutdown_connections 2024-03-15T12:58:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.532+0000 7f1562a23640 1 -- 172.21.15.2:0/4183952950 wait complete. 2024-03-15T12:58:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f1562a23640 1 Processor -- start 2024-03-15T12:58:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f1562a23640 1 -- start start 2024-03-15T12:58:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f1562a23640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c100410 0x7f155c1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f1562a23640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f155c101600 0x7f155c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f1562a23640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f155c101fc0 0x7f155c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f1562a23640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c19f0b0 con 0x7f155c100410 2024-03-15T12:58:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f1562a23640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c19f220 con 0x7f155c101fc0 2024-03-15T12:58:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f1562a23640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c19f3e0 con 0x7f155c101600 2024-03-15T12:58:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f155bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c100410 0x7f155c1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f155bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c100410 0x7f155c1975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43490/0 (socket says 172.21.15.2:43490) 2024-03-15T12:58:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f155bfff640 1 -- 172.21.15.2:0/1072732323 learned_addr learned my addr 172.21.15.2:0/1072732323 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f155b7fe640 1 --2- 172.21.15.2:0/1072732323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f155c101600 0x7f155c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f1560fc0640 1 --2- 172.21.15.2:0/1072732323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f155c101fc0 0x7f155c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f155bfff640 1 -- 172.21.15.2:0/1072732323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f155c101600 msgr2=0x7f155c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f155bfff640 1 --2- 172.21.15.2:0/1072732323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f155c101600 0x7f155c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f155bfff640 1 -- 172.21.15.2:0/1072732323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f155c101fc0 msgr2=0x7f155c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f155bfff640 1 --2- 172.21.15.2:0/1072732323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f155c101fc0 0x7f155c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f155bfff640 1 -- 172.21.15.2:0/1072732323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1550009560 con 0x7f155c100410 2024-03-15T12:58:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.533+0000 7f155bfff640 1 --2- 172.21.15.2:0/1072732323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c100410 0x7f155c1975c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f154c00e9c0 tx=0x7f154c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.534+0000 7f15597fa640 1 -- 172.21.15.2:0/1072732323 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c009700 con 0x7f155c100410 2024-03-15T12:58:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.534+0000 7f1562a23640 1 -- 172.21.15.2:0/1072732323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1554009840 con 0x7f155c100410 2024-03-15T12:58:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.534+0000 7f15597fa640 1 -- 172.21.15.2:0/1072732323 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f154c0044f0 con 0x7f155c100410 2024-03-15T12:58:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.534+0000 7f15597fa640 1 -- 172.21.15.2:0/1072732323 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c010430 con 0x7f155c100410 2024-03-15T12:58:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.534+0000 7f1562a23640 1 -- 172.21.15.2:0/1072732323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155c19fa40 con 0x7f155c100410 2024-03-15T12:58:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.535+0000 7f1562a23640 1 -- 172.21.15.2:0/1072732323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155c10bdf0 con 0x7f155c100410 2024-03-15T12:58:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.536+0000 7f15597fa640 1 -- 172.21.15.2:0/1072732323 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f154c003fe0 con 0x7f155c100410 2024-03-15T12:58:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.536+0000 7f15597fa640 1 --2- 172.21.15.2:0/1072732323 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f15440a1a30 0x7f15440a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.536+0000 7f155b7fe640 1 --2- 172.21.15.2:0/1072732323 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f15440a1a30 0x7f15440a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.536+0000 7f15597fa640 1 -- 172.21.15.2:0/1072732323 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f154c014070 con 0x7f155c100410 2024-03-15T12:58:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.537+0000 7f155b7fe640 1 --2- 172.21.15.2:0/1072732323 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f15440a1a30 0x7f15440a3ef0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f155c198ad0 tx=0x7f1550009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:39.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.540+0000 7f15597fa640 1 -- 172.21.15.2:0/1072732323 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f154c0969f0 con 0x7f155c100410 2024-03-15T12:58:39.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.724+0000 7f1562a23640 1 -- 172.21.15.2:0/1072732323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f155c19fcc0 con 0x7f155c100410 2024-03-15T12:58:39.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.724+0000 7f15597fa640 1 -- 172.21.15.2:0/1072732323 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f154c09a0b0 con 0x7f155c100410 2024-03-15T12:58:39.710 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-15T12:58:39.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.726+0000 7f1562a23640 1 -- 172.21.15.2:0/1072732323 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f15440a1a30 msgr2=0x7f15440a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:39.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.726+0000 7f1562a23640 1 --2- 172.21.15.2:0/1072732323 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f15440a1a30 0x7f15440a3ef0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f155c198ad0 tx=0x7f1550009260 comp rx=0 tx=0).stop 2024-03-15T12:58:39.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.726+0000 7f1562a23640 1 -- 172.21.15.2:0/1072732323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c100410 msgr2=0x7f155c1975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:39.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.726+0000 7f1562a23640 1 --2- 172.21.15.2:0/1072732323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c100410 0x7f155c1975c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f154c00e9c0 tx=0x7f154c00ee80 comp rx=0 tx=0).stop 2024-03-15T12:58:39.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.726+0000 7f1562a23640 1 -- 172.21.15.2:0/1072732323 shutdown_connections 2024-03-15T12:58:39.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.726+0000 7f1562a23640 1 --2- 172.21.15.2:0/1072732323 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f15440a1a30 0x7f15440a3ef0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.726+0000 7f1562a23640 1 --2- 172.21.15.2:0/1072732323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f155c101fc0 0x7f155c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.726+0000 7f1562a23640 1 --2- 172.21.15.2:0/1072732323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f155c101600 0x7f155c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.726+0000 7f1562a23640 1 --2- 172.21.15.2:0/1072732323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c100410 0x7f155c1975c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.726+0000 7f1562a23640 1 -- 172.21.15.2:0/1072732323 >> 172.21.15.2:0/1072732323 conn(0x7f155c0fb9a0 msgr2=0x7f155c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:39.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.727+0000 7f1562a23640 1 -- 172.21.15.2:0/1072732323 shutdown_connections 2024-03-15T12:58:39.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.727+0000 7f1562a23640 1 -- 172.21.15.2:0/1072732323 wait complete. 2024-03-15T12:58:39.724 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-15T12:58:39.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.844+0000 7ff92d0df640 1 -- 172.21.15.2:0/127469307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9281071f0 msgr2=0x7ff928109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:39.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.844+0000 7ff92d0df640 1 --2- 172.21.15.2:0/127469307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9281071f0 0x7ff928109640 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff920009ca0 tx=0x7ff92002c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.844+0000 7ff92d0df640 1 -- 172.21.15.2:0/127469307 shutdown_connections 2024-03-15T12:58:39.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.844+0000 7ff92d0df640 1 --2- 172.21.15.2:0/127469307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9281071f0 0x7ff928109640 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.844+0000 7ff92d0df640 1 --2- 172.21.15.2:0/127469307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9281048c0 0x7ff928106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.844+0000 7ff92d0df640 1 --2- 172.21.15.2:0/127469307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff928101f90 0x7ff928104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.844+0000 7ff92d0df640 1 -- 172.21.15.2:0/127469307 >> 172.21.15.2:0/127469307 conn(0x7ff9280fb9a0 msgr2=0x7ff9280fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.844+0000 7ff92d0df640 1 -- 172.21.15.2:0/127469307 shutdown_connections 2024-03-15T12:58:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.844+0000 7ff92d0df640 1 -- 172.21.15.2:0/127469307 wait complete. 2024-03-15T12:58:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.844+0000 7ff92d0df640 1 Processor -- start 2024-03-15T12:58:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.844+0000 7ff92d0df640 1 -- start start 2024-03-15T12:58:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.844+0000 7ff92d0df640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff928101f90 0x7ff928197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.844+0000 7ff92d0df640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9281048c0 0x7ff928197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff92d0df640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9281071f0 0x7ff92819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff92d0df640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92819f050 con 0x7ff9281048c0 2024-03-15T12:58:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff92d0df640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92819f1c0 con 0x7ff928101f90 2024-03-15T12:58:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff92d0df640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92819f380 con 0x7ff9281071f0 2024-03-15T12:58:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff91ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9281048c0 0x7ff928197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff91ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9281048c0 0x7ff928197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43506/0 (socket says 172.21.15.2:43506) 2024-03-15T12:58:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff91ffff640 1 -- 172.21.15.2:0/493189770 learned_addr learned my addr 172.21.15.2:0/493189770 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff926d9d640 1 --2- 172.21.15.2:0/493189770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff928101f90 0x7ff928197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff92759e640 1 --2- 172.21.15.2:0/493189770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9281071f0 0x7ff92819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff91ffff640 1 -- 172.21.15.2:0/493189770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9281071f0 msgr2=0x7ff92819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff91ffff640 1 --2- 172.21.15.2:0/493189770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9281071f0 0x7ff92819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff91ffff640 1 -- 172.21.15.2:0/493189770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff928101f90 msgr2=0x7ff928197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff91ffff640 1 --2- 172.21.15.2:0/493189770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff928101f90 0x7ff928197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff91ffff640 1 -- 172.21.15.2:0/493189770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff920009900 con 0x7ff9281048c0 2024-03-15T12:58:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff926d9d640 1 --2- 172.21.15.2:0/493189770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff928101f90 0x7ff928197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff91ffff640 1 --2- 172.21.15.2:0/493189770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9281048c0 0x7ff928197aa0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff91000e9f0 tx=0x7ff91000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff924d99640 1 -- 172.21.15.2:0/493189770 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9100096c0 con 0x7ff9281048c0 2024-03-15T12:58:39.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff924d99640 1 -- 172.21.15.2:0/493189770 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff9100054f0 con 0x7ff9281048c0 2024-03-15T12:58:39.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff92d0df640 1 -- 172.21.15.2:0/493189770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92819f660 con 0x7ff9281048c0 2024-03-15T12:58:39.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff924d99640 1 -- 172.21.15.2:0/493189770 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910010430 con 0x7ff9281048c0 2024-03-15T12:58:39.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.845+0000 7ff92d0df640 1 -- 172.21.15.2:0/493189770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92819fb20 con 0x7ff9281048c0 2024-03-15T12:58:39.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.847+0000 7ff92d0df640 1 -- 172.21.15.2:0/493189770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9280fd490 con 0x7ff9281048c0 2024-03-15T12:58:39.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.847+0000 7ff924d99640 1 -- 172.21.15.2:0/493189770 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7ff910005000 con 0x7ff9281048c0 2024-03-15T12:58:39.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.848+0000 7ff924d99640 1 --2- 172.21.15.2:0/493189770 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff9140a1a30 0x7ff9140a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:39.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.848+0000 7ff926d9d640 1 --2- 172.21.15.2:0/493189770 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff9140a1a30 0x7ff9140a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:39.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.848+0000 7ff924d99640 1 -- 172.21.15.2:0/493189770 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7ff910014070 con 0x7ff9281048c0 2024-03-15T12:58:39.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.849+0000 7ff926d9d640 1 --2- 172.21.15.2:0/493189770 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff9140a1a30 0x7ff9140a3ef0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff918009fc0 tx=0x7ff918005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:39.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:39.853+0000 7ff924d99640 1 -- 172.21.15.2:0/493189770 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff910096950 con 0x7ff9281048c0 2024-03-15T12:58:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.034+0000 7ff92d0df640 1 -- 172.21.15.2:0/493189770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff92802cf60 con 0x7ff9281048c0 2024-03-15T12:58:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.035+0000 7ff924d99640 1 -- 172.21.15.2:0/493189770 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff91009a010 con 0x7ff9281048c0 2024-03-15T12:58:40.021 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:58:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.036+0000 7ff92d0df640 1 -- 172.21.15.2:0/493189770 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff9140a1a30 msgr2=0x7ff9140a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.036+0000 7ff92d0df640 1 --2- 172.21.15.2:0/493189770 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff9140a1a30 0x7ff9140a3ef0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff918009fc0 tx=0x7ff918005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.036+0000 7ff92d0df640 1 -- 172.21.15.2:0/493189770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9281048c0 msgr2=0x7ff928197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.036+0000 7ff92d0df640 1 --2- 172.21.15.2:0/493189770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9281048c0 0x7ff928197aa0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff91000e9f0 tx=0x7ff91000eeb0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.037+0000 7ff92d0df640 1 -- 172.21.15.2:0/493189770 shutdown_connections 2024-03-15T12:58:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.037+0000 7ff92d0df640 1 --2- 172.21.15.2:0/493189770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9281071f0 0x7ff92819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.037+0000 7ff92d0df640 1 --2- 172.21.15.2:0/493189770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9281048c0 0x7ff928197aa0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.037+0000 7ff92d0df640 1 --2- 172.21.15.2:0/493189770 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7ff9140a1a30 0x7ff9140a3ef0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.037+0000 7ff92d0df640 1 --2- 172.21.15.2:0/493189770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff928101f90 0x7ff928197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.037+0000 7ff92d0df640 1 -- 172.21.15.2:0/493189770 >> 172.21.15.2:0/493189770 conn(0x7ff9280fb9a0 msgr2=0x7ff928102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.037+0000 7ff92d0df640 1 -- 172.21.15.2:0/493189770 shutdown_connections 2024-03-15T12:58:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.037+0000 7ff92d0df640 1 -- 172.21.15.2:0/493189770 wait complete. 2024-03-15T12:58:40.032 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:39 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2043584512' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-15T12:58:40.032 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:39 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T12:58:40.033 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:39 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1090105745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:40.034 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-15T12:58:40.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:39 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2043584512' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-15T12:58:40.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:39 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T12:58:40.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:39 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1090105745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:40.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:39 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2043584512' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-15T12:58:40.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:39 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T12:58:40.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:39 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1090105745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:40.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.153+0000 7f89ace3b640 1 -- 172.21.15.2:0/3626965897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a80ffb90 msgr2=0x7f89a80fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.153+0000 7f89ace3b640 1 --2- 172.21.15.2:0/3626965897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a80ffb90 0x7f89a80fffb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8998009900 tx=0x7f899802bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:40.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.153+0000 7f89ace3b640 1 -- 172.21.15.2:0/3626965897 shutdown_connections 2024-03-15T12:58:40.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.153+0000 7f89ace3b640 1 --2- 172.21.15.2:0/3626965897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a8101740 0x7f89a8107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.153+0000 7f89ace3b640 1 --2- 172.21.15.2:0/3626965897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89a8100d80 0x7f89a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.153+0000 7f89ace3b640 1 --2- 172.21.15.2:0/3626965897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a80ffb90 0x7f89a80fffb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.153+0000 7f89ace3b640 1 -- 172.21.15.2:0/3626965897 >> 172.21.15.2:0/3626965897 conn(0x7f89a80fb120 msgr2=0x7f89a80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:40.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.153+0000 7f89ace3b640 1 -- 172.21.15.2:0/3626965897 shutdown_connections 2024-03-15T12:58:40.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.153+0000 7f89ace3b640 1 -- 172.21.15.2:0/3626965897 wait complete. 2024-03-15T12:58:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.154+0000 7f89ace3b640 1 Processor -- start 2024-03-15T12:58:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.154+0000 7f89ace3b640 1 -- start start 2024-03-15T12:58:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.154+0000 7f89ace3b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a80ffb90 0x7f89a81953c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.154+0000 7f89ace3b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8100d80 0x7f89a8195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.154+0000 7f89ace3b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89a8101740 0x7f89a819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.154+0000 7f89ace3b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a819ceb0 con 0x7f89a8100d80 2024-03-15T12:58:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.154+0000 7f89ace3b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a819d020 con 0x7f89a80ffb90 2024-03-15T12:58:40.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.154+0000 7f89ace3b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a819d1e0 con 0x7f89a8101740 2024-03-15T12:58:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.154+0000 7f899ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8100d80 0x7f89a8195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.154+0000 7f899ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8100d80 0x7f89a8195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43522/0 (socket says 172.21.15.2:43522) 2024-03-15T12:58:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.154+0000 7f899ffff640 1 -- 172.21.15.2:0/4088476957 learned_addr learned my addr 172.21.15.2:0/4088476957 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.154+0000 7f89a659c640 1 --2- 172.21.15.2:0/4088476957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a80ffb90 0x7f89a81953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.154+0000 7f89a6d9d640 1 --2- 172.21.15.2:0/4088476957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89a8101740 0x7f89a819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.155+0000 7f899ffff640 1 -- 172.21.15.2:0/4088476957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89a8101740 msgr2=0x7f89a819c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.155+0000 7f899ffff640 1 --2- 172.21.15.2:0/4088476957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89a8101740 0x7f89a819c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.155+0000 7f899ffff640 1 -- 172.21.15.2:0/4088476957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a80ffb90 msgr2=0x7f89a81953c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.155+0000 7f899ffff640 1 --2- 172.21.15.2:0/4088476957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a80ffb90 0x7f89a81953c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.155+0000 7f899ffff640 1 -- 172.21.15.2:0/4088476957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8998009560 con 0x7f89a8100d80 2024-03-15T12:58:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.155+0000 7f89a659c640 1 --2- 172.21.15.2:0/4088476957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a80ffb90 0x7f89a81953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.155+0000 7f899ffff640 1 --2- 172.21.15.2:0/4088476957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8100d80 0x7f89a8195900 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f899000e9f0 tx=0x7f899000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.155+0000 7f899f7fe640 1 -- 172.21.15.2:0/4088476957 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990009720 con 0x7f89a8100d80 2024-03-15T12:58:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.155+0000 7f899f7fe640 1 -- 172.21.15.2:0/4088476957 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f899000cd80 con 0x7f89a8100d80 2024-03-15T12:58:40.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.155+0000 7f89ace3b640 1 -- 172.21.15.2:0/4088476957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a819d4c0 con 0x7f89a8100d80 2024-03-15T12:58:40.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.155+0000 7f899f7fe640 1 -- 172.21.15.2:0/4088476957 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990010690 con 0x7f89a8100d80 2024-03-15T12:58:40.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.155+0000 7f89ace3b640 1 -- 172.21.15.2:0/4088476957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a819d980 con 0x7f89a8100d80 2024-03-15T12:58:40.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.156+0000 7f89ace3b640 1 -- 172.21.15.2:0/4088476957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a8106fd0 con 0x7f89a8100d80 2024-03-15T12:58:40.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.157+0000 7f899f7fe640 1 -- 172.21.15.2:0/4088476957 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7f8990004e90 con 0x7f89a8100d80 2024-03-15T12:58:40.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.158+0000 7f899f7fe640 1 --2- 172.21.15.2:0/4088476957 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f89940a1a30 0x7f89940a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:40.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.158+0000 7f89a659c640 1 --2- 172.21.15.2:0/4088476957 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f89940a1a30 0x7f89940a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:40.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.158+0000 7f899f7fe640 1 -- 172.21.15.2:0/4088476957 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f8990014070 con 0x7f89a8100d80 2024-03-15T12:58:40.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.158+0000 7f89a659c640 1 --2- 172.21.15.2:0/4088476957 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f89940a1a30 0x7f89940a3ef0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f899802c740 tx=0x7f8998005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:40.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.162+0000 7f899f7fe640 1 -- 172.21.15.2:0/4088476957 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8990096950 con 0x7f89a8100d80 2024-03-15T12:58:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.344+0000 7f89ace3b640 1 -- 172.21.15.2:0/4088476957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89a802cf60 con 0x7f89a8100d80 2024-03-15T12:58:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.344+0000 7f899f7fe640 1 -- 172.21.15.2:0/4088476957 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f899009a010 con 0x7f89a8100d80 2024-03-15T12:58:40.330 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-03-15T12:58:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.346+0000 7f89ace3b640 1 -- 172.21.15.2:0/4088476957 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f89940a1a30 msgr2=0x7f89940a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.346+0000 7f89ace3b640 1 --2- 172.21.15.2:0/4088476957 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f89940a1a30 0x7f89940a3ef0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f899802c740 tx=0x7f8998005fd0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.346+0000 7f89ace3b640 1 -- 172.21.15.2:0/4088476957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8100d80 msgr2=0x7f89a8195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.346+0000 7f89ace3b640 1 --2- 172.21.15.2:0/4088476957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8100d80 0x7f89a8195900 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f899000e9f0 tx=0x7f899000edf0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.346+0000 7f89ace3b640 1 -- 172.21.15.2:0/4088476957 shutdown_connections 2024-03-15T12:58:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.346+0000 7f89ace3b640 1 --2- 172.21.15.2:0/4088476957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89a8101740 0x7f89a819c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.346+0000 7f89ace3b640 1 --2- 172.21.15.2:0/4088476957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89a8100d80 0x7f89a8195900 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.346+0000 7f89ace3b640 1 --2- 172.21.15.2:0/4088476957 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f89940a1a30 0x7f89940a3ef0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.346+0000 7f89ace3b640 1 --2- 172.21.15.2:0/4088476957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a80ffb90 0x7f89a81953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.346+0000 7f89ace3b640 1 -- 172.21.15.2:0/4088476957 >> 172.21.15.2:0/4088476957 conn(0x7f89a80fb120 msgr2=0x7f89a81041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.346+0000 7f89ace3b640 1 -- 172.21.15.2:0/4088476957 shutdown_connections 2024-03-15T12:58:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.346+0000 7f89ace3b640 1 -- 172.21.15.2:0/4088476957 wait complete. 2024-03-15T12:58:40.343 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-15T12:58:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.462+0000 7fda78a15640 1 -- 172.21.15.2:0/317501460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda74101f70 msgr2=0x7fda74104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.462+0000 7fda78a15640 1 --2- 172.21.15.2:0/317501460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda74101f70 0x7fda74104360 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009900 tx=0x7fda6c02bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.463+0000 7fda78a15640 1 -- 172.21.15.2:0/317501460 shutdown_connections 2024-03-15T12:58:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.463+0000 7fda78a15640 1 --2- 172.21.15.2:0/317501460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda741071d0 0x7fda74109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.463+0000 7fda78a15640 1 --2- 172.21.15.2:0/317501460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda741048a0 0x7fda74106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.463+0000 7fda78a15640 1 --2- 172.21.15.2:0/317501460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda74101f70 0x7fda74104360 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.463+0000 7fda78a15640 1 -- 172.21.15.2:0/317501460 >> 172.21.15.2:0/317501460 conn(0x7fda740fb980 msgr2=0x7fda740fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.463+0000 7fda78a15640 1 -- 172.21.15.2:0/317501460 shutdown_connections 2024-03-15T12:58:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.463+0000 7fda78a15640 1 -- 172.21.15.2:0/317501460 wait complete. 2024-03-15T12:58:40.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.463+0000 7fda78a15640 1 Processor -- start 2024-03-15T12:58:40.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.463+0000 7fda78a15640 1 -- start start 2024-03-15T12:58:40.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.463+0000 7fda78a15640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda74101f70 0x7fda741975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:40.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.463+0000 7fda78a15640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda741048a0 0x7fda74197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:40.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.463+0000 7fda78a15640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda741071d0 0x7fda7419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:40.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.463+0000 7fda78a15640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7419f0c0 con 0x7fda74101f70 2024-03-15T12:58:40.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.463+0000 7fda78a15640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7419f230 con 0x7fda741071d0 2024-03-15T12:58:40.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.463+0000 7fda78a15640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7419f3f0 con 0x7fda741048a0 2024-03-15T12:58:40.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda7259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda74101f70 0x7fda741975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:40.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda7259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda74101f70 0x7fda741975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43548/0 (socket says 172.21.15.2:43548) 2024-03-15T12:58:40.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda71d9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda741048a0 0x7fda74197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:40.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda72d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda741071d0 0x7fda7419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:40.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda7259c640 1 -- 172.21.15.2:0/2486408687 learned_addr learned my addr 172.21.15.2:0/2486408687 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:40.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda72d9d640 1 -- 172.21.15.2:0/2486408687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda741048a0 msgr2=0x7fda74197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda72d9d640 1 --2- 172.21.15.2:0/2486408687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda741048a0 0x7fda74197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda72d9d640 1 -- 172.21.15.2:0/2486408687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda74101f70 msgr2=0x7fda741975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda72d9d640 1 --2- 172.21.15.2:0/2486408687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda74101f70 0x7fda741975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda72d9d640 1 -- 172.21.15.2:0/2486408687 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c009560 con 0x7fda741071d0 2024-03-15T12:58:40.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda7259c640 1 --2- 172.21.15.2:0/2486408687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda74101f70 0x7fda741975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:40.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda72d9d640 1 --2- 172.21.15.2:0/2486408687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda741071d0 0x7fda7419eb80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda64009fd0 tx=0x7fda6400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:40.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda6b7fe640 1 -- 172.21.15.2:0/2486408687 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6400ce20 con 0x7fda741071d0 2024-03-15T12:58:40.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda6b7fe640 1 -- 172.21.15.2:0/2486408687 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fda64005020 con 0x7fda741071d0 2024-03-15T12:58:40.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda78a15640 1 -- 172.21.15.2:0/2486408687 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7419f6d0 con 0x7fda741071d0 2024-03-15T12:58:40.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda6b7fe640 1 -- 172.21.15.2:0/2486408687 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda64010690 con 0x7fda741071d0 2024-03-15T12:58:40.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.464+0000 7fda78a15640 1 -- 172.21.15.2:0/2486408687 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7419fb90 con 0x7fda741071d0 2024-03-15T12:58:40.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.465+0000 7fda78a15640 1 -- 172.21.15.2:0/2486408687 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda740fd470 con 0x7fda741071d0 2024-03-15T12:58:40.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.466+0000 7fda6b7fe640 1 -- 172.21.15.2:0/2486408687 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 136278+0+0 (secure 0 0 0) 0x7fda64005570 con 0x7fda741071d0 2024-03-15T12:58:40.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.467+0000 7fda6b7fe640 1 --2- 172.21.15.2:0/2486408687 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fda600a1a30 0x7fda600a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:40.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.467+0000 7fda7259c640 1 --2- 172.21.15.2:0/2486408687 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fda600a1a30 0x7fda600a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:40.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.467+0000 7fda6b7fe640 1 -- 172.21.15.2:0/2486408687 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fda64014070 con 0x7fda741071d0 2024-03-15T12:58:40.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.467+0000 7fda7259c640 1 --2- 172.21.15.2:0/2486408687 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fda600a1a30 0x7fda600a3ef0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fda6c02c870 tx=0x7fda6c02cd20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.475+0000 7fda6b7fe640 1 -- 172.21.15.2:0/2486408687 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fda6409a830 con 0x7fda741071d0 2024-03-15T12:58:40.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.656+0000 7fda78a15640 1 -- 172.21.15.2:0/2486408687 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fda7410ced0 con 0x7fda741071d0 2024-03-15T12:58:40.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.656+0000 7fda6b7fe640 1 -- 172.21.15.2:0/2486408687 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fda6409a3c0 con 0x7fda741071d0 2024-03-15T12:58:40.642 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:58:40.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.658+0000 7fda78a15640 1 -- 172.21.15.2:0/2486408687 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fda600a1a30 msgr2=0x7fda600a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.658+0000 7fda78a15640 1 --2- 172.21.15.2:0/2486408687 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fda600a1a30 0x7fda600a3ef0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fda6c02c870 tx=0x7fda6c02cd20 comp rx=0 tx=0).stop 2024-03-15T12:58:40.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.658+0000 7fda78a15640 1 -- 172.21.15.2:0/2486408687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda741071d0 msgr2=0x7fda7419eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.658+0000 7fda78a15640 1 --2- 172.21.15.2:0/2486408687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda741071d0 0x7fda7419eb80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda64009fd0 tx=0x7fda6400ef10 comp rx=0 tx=0).stop 2024-03-15T12:58:40.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.658+0000 7fda78a15640 1 -- 172.21.15.2:0/2486408687 shutdown_connections 2024-03-15T12:58:40.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.658+0000 7fda78a15640 1 --2- 172.21.15.2:0/2486408687 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fda600a1a30 0x7fda600a3ef0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.658+0000 7fda78a15640 1 --2- 172.21.15.2:0/2486408687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda741071d0 0x7fda7419eb80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.658+0000 7fda78a15640 1 --2- 172.21.15.2:0/2486408687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda741048a0 0x7fda74197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.658+0000 7fda78a15640 1 --2- 172.21.15.2:0/2486408687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda74101f70 0x7fda741975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.658+0000 7fda78a15640 1 -- 172.21.15.2:0/2486408687 >> 172.21.15.2:0/2486408687 conn(0x7fda740fb980 msgr2=0x7fda74102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:40.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.658+0000 7fda78a15640 1 -- 172.21.15.2:0/2486408687 shutdown_connections 2024-03-15T12:58:40.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.659+0000 7fda78a15640 1 -- 172.21.15.2:0/2486408687 wait complete. 2024-03-15T12:58:40.656 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:58:40.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.776+0000 7f3ff8a4d640 1 -- 172.21.15.2:0/295613474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ff4100400 msgr2=0x7f3ff4100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.776+0000 7f3ff8a4d640 1 --2- 172.21.15.2:0/295613474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ff4100400 0x7f3ff4100820 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009900 tx=0x7f3fe402bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:40.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.776+0000 7f3ff8a4d640 1 -- 172.21.15.2:0/295613474 shutdown_connections 2024-03-15T12:58:40.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.776+0000 7f3ff8a4d640 1 --2- 172.21.15.2:0/295613474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff4101fb0 0x7f3ff4108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.776+0000 7f3ff8a4d640 1 --2- 172.21.15.2:0/295613474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ff41015f0 0x7f3ff4101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.776+0000 7f3ff8a4d640 1 --2- 172.21.15.2:0/295613474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ff4100400 0x7f3ff4100820 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.776+0000 7f3ff8a4d640 1 -- 172.21.15.2:0/295613474 >> 172.21.15.2:0/295613474 conn(0x7f3ff40fb970 msgr2=0x7f3ff40fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:40.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.776+0000 7f3ff8a4d640 1 -- 172.21.15.2:0/295613474 shutdown_connections 2024-03-15T12:58:40.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.776+0000 7f3ff8a4d640 1 -- 172.21.15.2:0/295613474 wait complete. 2024-03-15T12:58:40.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.777+0000 7f3ff8a4d640 1 Processor -- start 2024-03-15T12:58:40.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.777+0000 7f3ff8a4d640 1 -- start start 2024-03-15T12:58:40.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.777+0000 7f3ff8a4d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ff4100400 0x7f3ff406a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:40.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.777+0000 7f3ff8a4d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ff41015f0 0x7f3ff419c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:40.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.777+0000 7f3ff8a4d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff4101fb0 0x7f3ff419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:40.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.777+0000 7f3ff8a4d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff419f140 con 0x7f3ff41015f0 2024-03-15T12:58:40.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.777+0000 7f3ff8a4d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff419f2b0 con 0x7f3ff4101fb0 2024-03-15T12:58:40.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.777+0000 7f3ff8a4d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff419f470 con 0x7f3ff4100400 2024-03-15T12:58:40.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.777+0000 7f3febfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ff41015f0 0x7f3ff419c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:40.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.777+0000 7f3ff259c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ff4100400 0x7f3ff406a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:40.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.777+0000 7f3ff2d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff4101fb0 0x7f3ff419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:40.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.777+0000 7f3febfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ff41015f0 0x7f3ff419c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43570/0 (socket says 172.21.15.2:43570) 2024-03-15T12:58:40.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.777+0000 7f3ff259c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ff4100400 0x7f3ff406a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:52740/0 (socket says 172.21.15.2:52740) 2024-03-15T12:58:40.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.777+0000 7f3febfff640 1 -- 172.21.15.2:0/3101168024 learned_addr learned my addr 172.21.15.2:0/3101168024 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:40.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.777+0000 7f3ff2d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff4101fb0 0x7f3ff419e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:56114/0 (socket says 172.21.15.2:56114) 2024-03-15T12:58:40.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.778+0000 7f3febfff640 1 -- 172.21.15.2:0/3101168024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ff4100400 msgr2=0x7f3ff406a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.778+0000 7f3febfff640 1 --2- 172.21.15.2:0/3101168024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ff4100400 0x7f3ff406a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.778+0000 7f3febfff640 1 -- 172.21.15.2:0/3101168024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff4101fb0 msgr2=0x7f3ff419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.778+0000 7f3febfff640 1 --2- 172.21.15.2:0/3101168024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff4101fb0 0x7f3ff419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.778+0000 7f3febfff640 1 -- 172.21.15.2:0/3101168024 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe4009560 con 0x7f3ff41015f0 2024-03-15T12:58:40.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.778+0000 7f3febfff640 1 --2- 172.21.15.2:0/3101168024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ff41015f0 0x7f3ff419c340 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00e9f0 tx=0x7f3fdc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:40.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.778+0000 7f3feb7fe640 1 -- 172.21.15.2:0/3101168024 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc009700 con 0x7f3ff41015f0 2024-03-15T12:58:40.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.778+0000 7f3feb7fe640 1 -- 172.21.15.2:0/3101168024 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3fdc00cd80 con 0x7f3ff41015f0 2024-03-15T12:58:40.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.778+0000 7f3ff8a4d640 1 -- 172.21.15.2:0/3101168024 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff419f6f0 con 0x7f3ff41015f0 2024-03-15T12:58:40.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.778+0000 7f3feb7fe640 1 -- 172.21.15.2:0/3101168024 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc018920 con 0x7f3ff41015f0 2024-03-15T12:58:40.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.778+0000 7f3ff8a4d640 1 -- 172.21.15.2:0/3101168024 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff419fc00 con 0x7f3ff41015f0 2024-03-15T12:58:40.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.779+0000 7f3ff8a4d640 1 -- 172.21.15.2:0/3101168024 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff41077f0 con 0x7f3ff41015f0 2024-03-15T12:58:40.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.780+0000 7f3feb7fe640 1 -- 172.21.15.2:0/3101168024 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f3fdc018a80 con 0x7f3ff41015f0 2024-03-15T12:58:40.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.781+0000 7f3feb7fe640 1 --2- 172.21.15.2:0/3101168024 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3fe00a1ac0 0x7f3fe00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:40.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.781+0000 7f3ff259c640 1 --2- 172.21.15.2:0/3101168024 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3fe00a1ac0 0x7f3fe00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:40.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.781+0000 7f3feb7fe640 1 -- 172.21.15.2:0/3101168024 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f3fdc014070 con 0x7f3ff41015f0 2024-03-15T12:58:40.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.781+0000 7f3ff259c640 1 --2- 172.21.15.2:0/3101168024 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3fe00a1ac0 0x7f3fe00a3f80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009230 tx=0x7f3fe4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:40.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.785+0000 7f3feb7fe640 1 -- 172.21.15.2:0/3101168024 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3fdc096eb0 con 0x7f3ff41015f0 2024-03-15T12:58:40.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.968+0000 7f3ff8a4d640 1 -- 172.21.15.2:0/3101168024 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ff410c070 con 0x7f3ff41015f0 2024-03-15T12:58:40.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.970+0000 7f3feb7fe640 1 -- 172.21.15.2:0/3101168024 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24526 (secure 0 0 0) 0x7f3fdc09a570 con 0x7f3ff41015f0 2024-03-15T12:58:40.956 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:40.956 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":91,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:57:19.826817+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:58:40.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.972+0000 7f3ff8a4d640 1 -- 172.21.15.2:0/3101168024 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3fe00a1ac0 msgr2=0x7f3fe00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.972+0000 7f3ff8a4d640 1 --2- 172.21.15.2:0/3101168024 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3fe00a1ac0 0x7f3fe00a3f80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009230 tx=0x7f3fe4000bc0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.972+0000 7f3ff8a4d640 1 -- 172.21.15.2:0/3101168024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ff41015f0 msgr2=0x7f3ff419c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:40.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.972+0000 7f3ff8a4d640 1 --2- 172.21.15.2:0/3101168024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ff41015f0 0x7f3ff419c340 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00e9f0 tx=0x7f3fdc00eeb0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.972+0000 7f3ff8a4d640 1 -- 172.21.15.2:0/3101168024 shutdown_connections 2024-03-15T12:58:40.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.972+0000 7f3ff8a4d640 1 --2- 172.21.15.2:0/3101168024 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3fe00a1ac0 0x7f3fe00a3f80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.972+0000 7f3ff8a4d640 1 --2- 172.21.15.2:0/3101168024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ff4101fb0 0x7f3ff419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.972+0000 7f3ff8a4d640 1 --2- 172.21.15.2:0/3101168024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ff41015f0 0x7f3ff419c340 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.972+0000 7f3ff8a4d640 1 --2- 172.21.15.2:0/3101168024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ff4100400 0x7f3ff406a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:40.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.972+0000 7f3ff8a4d640 1 -- 172.21.15.2:0/3101168024 >> 172.21.15.2:0/3101168024 conn(0x7f3ff40fb970 msgr2=0x7f3ff41049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:40.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.972+0000 7f3ff8a4d640 1 -- 172.21.15.2:0/3101168024 shutdown_connections 2024-03-15T12:58:40.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:40.972+0000 7f3ff8a4d640 1 -- 172.21.15.2:0/3101168024 wait complete. 2024-03-15T12:58:40.967 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:40 smithi002 ceph-mon[32060]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-15T12:58:40.968 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:40 smithi002 ceph-mon[32060]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:40.968 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:40 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1072732323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:40.968 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:40 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/493189770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:40.968 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:40 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/4088476957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:40.968 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:40 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2486408687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:40.969 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-15T12:51:47.004843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-15T12:56:18.559882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-15T12:56:19.751767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-15T12:56:23.664879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-15T12:58:40.970 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-15T12:58:41.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:40 smithi184 ceph-mon[35993]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-15T12:58:41.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:40 smithi184 ceph-mon[35993]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:41.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:40 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1072732323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:41.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:40 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/493189770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:41.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:40 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/4088476957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:41.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:40 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2486408687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:41.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.089+0000 7f10fbb0c640 1 -- 172.21.15.2:0/1644671308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f41015e0 msgr2=0x7f10f4101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:41.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.089+0000 7f10fbb0c640 1 --2- 172.21.15.2:0/1644671308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f41015e0 0x7f10f4101a60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009900 tx=0x7f10e402bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:41.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10fbb0c640 1 -- 172.21.15.2:0/1644671308 shutdown_connections 2024-03-15T12:58:41.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10fbb0c640 1 --2- 172.21.15.2:0/1644671308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10f4101fa0 0x7f10f4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10fbb0c640 1 --2- 172.21.15.2:0/1644671308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f41015e0 0x7f10f4101a60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10fbb0c640 1 --2- 172.21.15.2:0/1644671308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10f41003f0 0x7f10f4100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10fbb0c640 1 -- 172.21.15.2:0/1644671308 >> 172.21.15.2:0/1644671308 conn(0x7f10f40fb980 msgr2=0x7f10f40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:41.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10fbb0c640 1 -- 172.21.15.2:0/1644671308 shutdown_connections 2024-03-15T12:58:41.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10fbb0c640 1 -- 172.21.15.2:0/1644671308 wait complete. 2024-03-15T12:58:41.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10fbb0c640 1 Processor -- start 2024-03-15T12:58:41.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10fbb0c640 1 -- start start 2024-03-15T12:58:41.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10fbb0c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10f41003f0 0x7f10f4197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:41.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10fbb0c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f41015e0 0x7f10f4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:41.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10fbb0c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10f4101fa0 0x7f10f419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:41.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10fbb0c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f419f080 con 0x7f10f41015e0 2024-03-15T12:58:41.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10fbb0c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f419f1f0 con 0x7f10f4101fa0 2024-03-15T12:58:41.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10fbb0c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f419f3b0 con 0x7f10f41003f0 2024-03-15T12:58:41.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10f90a7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f41015e0 0x7f10f4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:41.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10f90a7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f41015e0 0x7f10f4197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43596/0 (socket says 172.21.15.2:43596) 2024-03-15T12:58:41.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.090+0000 7f10f90a7640 1 -- 172.21.15.2:0/1783831801 learned_addr learned my addr 172.21.15.2:0/1783831801 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:41.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.091+0000 7f10fa0a9640 1 --2- 172.21.15.2:0/1783831801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10f4101fa0 0x7f10f419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:41.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.091+0000 7f10f98a8640 1 --2- 172.21.15.2:0/1783831801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10f41003f0 0x7f10f4197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:41.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.091+0000 7f10f90a7640 1 -- 172.21.15.2:0/1783831801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10f41003f0 msgr2=0x7f10f4197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:41.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.091+0000 7f10f90a7640 1 --2- 172.21.15.2:0/1783831801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10f41003f0 0x7f10f4197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.091+0000 7f10f90a7640 1 -- 172.21.15.2:0/1783831801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10f4101fa0 msgr2=0x7f10f419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:41.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.091+0000 7f10f90a7640 1 --2- 172.21.15.2:0/1783831801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10f4101fa0 0x7f10f419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.091+0000 7f10f90a7640 1 -- 172.21.15.2:0/1783831801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4009560 con 0x7f10f41015e0 2024-03-15T12:58:41.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.091+0000 7f10f90a7640 1 --2- 172.21.15.2:0/1783831801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f41015e0 0x7f10f4197ad0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f10e4005ab0 tx=0x7f10e4004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:41.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.091+0000 7f10eaffd640 1 -- 172.21.15.2:0/1783831801 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4004250 con 0x7f10f41015e0 2024-03-15T12:58:41.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.091+0000 7f10fbb0c640 1 -- 172.21.15.2:0/1783831801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f419f630 con 0x7f10f41015e0 2024-03-15T12:58:41.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.091+0000 7f10eaffd640 1 -- 172.21.15.2:0/1783831801 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f10e40043b0 con 0x7f10f41015e0 2024-03-15T12:58:41.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.091+0000 7f10eaffd640 1 -- 172.21.15.2:0/1783831801 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4003d20 con 0x7f10f41015e0 2024-03-15T12:58:41.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.091+0000 7f10fbb0c640 1 -- 172.21.15.2:0/1783831801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f419fb90 con 0x7f10f41015e0 2024-03-15T12:58:41.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.092+0000 7f10fbb0c640 1 -- 172.21.15.2:0/1783831801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f4107830 con 0x7f10f41015e0 2024-03-15T12:58:41.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.093+0000 7f10eaffd640 1 -- 172.21.15.2:0/1783831801 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f10e4005000 con 0x7f10f41015e0 2024-03-15T12:58:41.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.094+0000 7f10eaffd640 1 --2- 172.21.15.2:0/1783831801 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f10e00a1ac0 0x7f10e00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:41.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.094+0000 7f10f98a8640 1 --2- 172.21.15.2:0/1783831801 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f10e00a1ac0 0x7f10e00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:41.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.094+0000 7f10eaffd640 1 -- 172.21.15.2:0/1783831801 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f10e40ef970 con 0x7f10f41015e0 2024-03-15T12:58:41.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.094+0000 7f10f98a8640 1 --2- 172.21.15.2:0/1783831801 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f10e00a1ac0 0x7f10e00a3f80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f10f4101440 tx=0x7f10ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:41.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.098+0000 7f10eaffd640 1 -- 172.21.15.2:0/1783831801 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f10e40b87f0 con 0x7f10f41015e0 2024-03-15T12:58:41.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:40 smithi149 ceph-mon[35927]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-15T12:58:41.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:40 smithi149 ceph-mon[35927]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:41.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:40 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1072732323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:41.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:40 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/493189770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:41.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:40 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/4088476957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:41.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:40 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2486408687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:41.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.279+0000 7f10fbb0c640 1 -- 172.21.15.2:0/1783831801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f10f419f7c0 con 0x7f10f41015e0 2024-03-15T12:58:41.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.279+0000 7f10eaffd640 1 -- 172.21.15.2:0/1783831801 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f10f419f7c0 con 0x7f10f41015e0 2024-03-15T12:58:41.265 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-15T12:58:41.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.281+0000 7f10fbb0c640 1 -- 172.21.15.2:0/1783831801 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f10e00a1ac0 msgr2=0x7f10e00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:41.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.281+0000 7f10fbb0c640 1 --2- 172.21.15.2:0/1783831801 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f10e00a1ac0 0x7f10e00a3f80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f10f4101440 tx=0x7f10ec005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:41.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.281+0000 7f10fbb0c640 1 -- 172.21.15.2:0/1783831801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f41015e0 msgr2=0x7f10f4197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:41.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.281+0000 7f10fbb0c640 1 --2- 172.21.15.2:0/1783831801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f41015e0 0x7f10f4197ad0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f10e4005ab0 tx=0x7f10e4004a30 comp rx=0 tx=0).stop 2024-03-15T12:58:41.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.281+0000 7f10fbb0c640 1 -- 172.21.15.2:0/1783831801 shutdown_connections 2024-03-15T12:58:41.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.281+0000 7f10fbb0c640 1 --2- 172.21.15.2:0/1783831801 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f10e00a1ac0 0x7f10e00a3f80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.281+0000 7f10fbb0c640 1 --2- 172.21.15.2:0/1783831801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10f4101fa0 0x7f10f419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.281+0000 7f10fbb0c640 1 --2- 172.21.15.2:0/1783831801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10f41015e0 0x7f10f4197ad0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.281+0000 7f10fbb0c640 1 --2- 172.21.15.2:0/1783831801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10f41003f0 0x7f10f4197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.281+0000 7f10fbb0c640 1 -- 172.21.15.2:0/1783831801 >> 172.21.15.2:0/1783831801 conn(0x7f10f40fb980 msgr2=0x7f10f4104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:41.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.281+0000 7f10fbb0c640 1 -- 172.21.15.2:0/1783831801 shutdown_connections 2024-03-15T12:58:41.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.281+0000 7f10fbb0c640 1 -- 172.21.15.2:0/1783831801 wait complete. 2024-03-15T12:58:41.278 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-15T12:58:41.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.397+0000 7f9f6429a640 1 -- 172.21.15.2:0/221176251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f5c101600 msgr2=0x7f9f5c101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:41.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.397+0000 7f9f6429a640 1 --2- 172.21.15.2:0/221176251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f5c101600 0x7f9f5c101a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009860 tx=0x7f9f4c02bb70 comp rx=0 tx=0).stop 2024-03-15T12:58:41.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.397+0000 7f9f6429a640 1 -- 172.21.15.2:0/221176251 shutdown_connections 2024-03-15T12:58:41.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.397+0000 7f9f6429a640 1 --2- 172.21.15.2:0/221176251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f5c101fc0 0x7f9f5c108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.397+0000 7f9f6429a640 1 --2- 172.21.15.2:0/221176251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f5c101600 0x7f9f5c101a80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.397+0000 7f9f6429a640 1 --2- 172.21.15.2:0/221176251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c100410 0x7f9f5c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f6429a640 1 -- 172.21.15.2:0/221176251 >> 172.21.15.2:0/221176251 conn(0x7f9f5c0fb9a0 msgr2=0x7f9f5c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:41.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f6429a640 1 -- 172.21.15.2:0/221176251 shutdown_connections 2024-03-15T12:58:41.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f6429a640 1 -- 172.21.15.2:0/221176251 wait complete. 2024-03-15T12:58:41.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f6429a640 1 Processor -- start 2024-03-15T12:58:41.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f6429a640 1 -- start start 2024-03-15T12:58:41.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f6429a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c100410 0x7f9f5c1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:41.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f6429a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f5c101600 0x7f9f5c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:41.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f6429a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f5c101fc0 0x7f9f5c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:41.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f6429a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c19f0b0 con 0x7f9f5c101fc0 2024-03-15T12:58:41.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f6429a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c19f220 con 0x7f9f5c101600 2024-03-15T12:58:41.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f6429a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c19f3e0 con 0x7f9f5c100410 2024-03-15T12:58:41.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f62837640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f5c101fc0 0x7f9f5c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:41.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f61835640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f5c101600 0x7f9f5c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:41.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f62837640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f5c101fc0 0x7f9f5c19eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43624/0 (socket says 172.21.15.2:43624) 2024-03-15T12:58:41.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f62837640 1 -- 172.21.15.2:0/926736680 learned_addr learned my addr 172.21.15.2:0/926736680 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:41.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.398+0000 7f9f62036640 1 --2- 172.21.15.2:0/926736680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c100410 0x7f9f5c1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:41.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.399+0000 7f9f61835640 1 -- 172.21.15.2:0/926736680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c100410 msgr2=0x7f9f5c1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:41.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.399+0000 7f9f61835640 1 --2- 172.21.15.2:0/926736680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c100410 0x7f9f5c1975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.399+0000 7f9f61835640 1 -- 172.21.15.2:0/926736680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f5c101fc0 msgr2=0x7f9f5c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:41.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.399+0000 7f9f61835640 1 --2- 172.21.15.2:0/926736680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f5c101fc0 0x7f9f5c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.399+0000 7f9f61835640 1 -- 172.21.15.2:0/926736680 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f54009900 con 0x7f9f5c101600 2024-03-15T12:58:41.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.399+0000 7f9f62837640 1 --2- 172.21.15.2:0/926736680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f5c101fc0 0x7f9f5c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:41.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.399+0000 7f9f61835640 1 --2- 172.21.15.2:0/926736680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f5c101600 0x7f9f5c197b00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c02c0b0 tx=0x7f9f4c032460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:41.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.399+0000 7f9f537fe640 1 -- 172.21.15.2:0/926736680 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c02c510 con 0x7f9f5c101600 2024-03-15T12:58:41.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.399+0000 7f9f6429a640 1 -- 172.21.15.2:0/926736680 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4c009490 con 0x7f9f5c101600 2024-03-15T12:58:41.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.399+0000 7f9f537fe640 1 -- 172.21.15.2:0/926736680 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9f4c02cb40 con 0x7f9f5c101600 2024-03-15T12:58:41.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.399+0000 7f9f537fe640 1 -- 172.21.15.2:0/926736680 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c032600 con 0x7f9f5c101600 2024-03-15T12:58:41.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.399+0000 7f9f6429a640 1 -- 172.21.15.2:0/926736680 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5c19f9d0 con 0x7f9f5c101600 2024-03-15T12:58:41.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.400+0000 7f9f6429a640 1 -- 172.21.15.2:0/926736680 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5c10bdf0 con 0x7f9f5c101600 2024-03-15T12:58:41.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.401+0000 7f9f537fe640 1 -- 172.21.15.2:0/926736680 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f9f4c0037f0 con 0x7f9f5c101600 2024-03-15T12:58:41.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.402+0000 7f9f537fe640 1 --2- 172.21.15.2:0/926736680 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9f480a1ac0 0x7f9f480a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:41.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.402+0000 7f9f62036640 1 --2- 172.21.15.2:0/926736680 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9f480a1ac0 0x7f9f480a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:41.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.402+0000 7f9f537fe640 1 -- 172.21.15.2:0/926736680 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f9f4c0efef0 con 0x7f9f5c101600 2024-03-15T12:58:41.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.402+0000 7f9f62036640 1 --2- 172.21.15.2:0/926736680 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9f480a1ac0 0x7f9f480a3f80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c101460 tx=0x7f9f58005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:41.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.410+0000 7f9f537fe640 1 -- 172.21.15.2:0/926736680 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9f4c0b8e00 con 0x7f9f5c101600 2024-03-15T12:58:41.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.593+0000 7f9f6429a640 1 -- 172.21.15.2:0/926736680 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f5c10bfb0 con 0x7f9f5c101600 2024-03-15T12:58:41.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.594+0000 7f9f537fe640 1 -- 172.21.15.2:0/926736680 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f4c0bc4c0 con 0x7f9f5c101600 2024-03-15T12:58:41.580 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:58:41.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.596+0000 7f9f6429a640 1 -- 172.21.15.2:0/926736680 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9f480a1ac0 msgr2=0x7f9f480a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:41.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.596+0000 7f9f6429a640 1 --2- 172.21.15.2:0/926736680 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9f480a1ac0 0x7f9f480a3f80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c101460 tx=0x7f9f58005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:41.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.596+0000 7f9f6429a640 1 -- 172.21.15.2:0/926736680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f5c101600 msgr2=0x7f9f5c197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:41.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.596+0000 7f9f6429a640 1 --2- 172.21.15.2:0/926736680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f5c101600 0x7f9f5c197b00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c02c0b0 tx=0x7f9f4c032460 comp rx=0 tx=0).stop 2024-03-15T12:58:41.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.596+0000 7f9f6429a640 1 -- 172.21.15.2:0/926736680 shutdown_connections 2024-03-15T12:58:41.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.596+0000 7f9f6429a640 1 --2- 172.21.15.2:0/926736680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f5c101fc0 0x7f9f5c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.596+0000 7f9f6429a640 1 --2- 172.21.15.2:0/926736680 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f9f480a1ac0 0x7f9f480a3f80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.596+0000 7f9f6429a640 1 --2- 172.21.15.2:0/926736680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f5c101600 0x7f9f5c197b00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.596+0000 7f9f6429a640 1 --2- 172.21.15.2:0/926736680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f5c100410 0x7f9f5c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.596+0000 7f9f6429a640 1 -- 172.21.15.2:0/926736680 >> 172.21.15.2:0/926736680 conn(0x7f9f5c0fb9a0 msgr2=0x7f9f5c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:41.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.596+0000 7f9f6429a640 1 -- 172.21.15.2:0/926736680 shutdown_connections 2024-03-15T12:58:41.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.596+0000 7f9f6429a640 1 -- 172.21.15.2:0/926736680 wait complete. 2024-03-15T12:58:41.593 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-15T12:58:41.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.712+0000 7f7b6c0c3640 1 -- 172.21.15.2:0/2951077319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b64101f70 msgr2=0x7f7b64104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:41.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.712+0000 7f7b6c0c3640 1 --2- 172.21.15.2:0/2951077319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b64101f70 0x7f7b64104360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009860 tx=0x7f7b5402bb70 comp rx=0 tx=0).stop 2024-03-15T12:58:41.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6c0c3640 1 -- 172.21.15.2:0/2951077319 shutdown_connections 2024-03-15T12:58:41.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6c0c3640 1 --2- 172.21.15.2:0/2951077319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b641071d0 0x7f7b64109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6c0c3640 1 --2- 172.21.15.2:0/2951077319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b641048a0 0x7f7b64106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6c0c3640 1 --2- 172.21.15.2:0/2951077319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b64101f70 0x7f7b64104360 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6c0c3640 1 -- 172.21.15.2:0/2951077319 >> 172.21.15.2:0/2951077319 conn(0x7f7b640fb980 msgr2=0x7f7b640fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:41.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6c0c3640 1 -- 172.21.15.2:0/2951077319 shutdown_connections 2024-03-15T12:58:41.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6c0c3640 1 -- 172.21.15.2:0/2951077319 wait complete. 2024-03-15T12:58:41.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6c0c3640 1 Processor -- start 2024-03-15T12:58:41.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6c0c3640 1 -- start start 2024-03-15T12:58:41.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6c0c3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b64101f70 0x7f7b641018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:41.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6c0c3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b641048a0 0x7f7b640fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:41.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6c0c3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b641071d0 0x7f7b64100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:41.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6c0c3640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b64100bc0 con 0x7f7b641071d0 2024-03-15T12:58:41.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6c0c3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b64100d30 con 0x7f7b64101f70 2024-03-15T12:58:41.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6c0c3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b641a1650 con 0x7f7b641048a0 2024-03-15T12:58:41.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6a660640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b641071d0 0x7f7b64100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:41.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6a660640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b641071d0 0x7f7b64100460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43646/0 (socket says 172.21.15.2:43646) 2024-03-15T12:58:41.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b6a660640 1 -- 172.21.15.2:0/3807276406 learned_addr learned my addr 172.21.15.2:0/3807276406 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:41.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.713+0000 7f7b69e5f640 1 --2- 172.21.15.2:0/3807276406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b64101f70 0x7f7b641018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:41.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.714+0000 7f7b6965e640 1 --2- 172.21.15.2:0/3807276406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b641048a0 0x7f7b640fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:41.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.714+0000 7f7b6a660640 1 -- 172.21.15.2:0/3807276406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b641048a0 msgr2=0x7f7b640fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:41.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.714+0000 7f7b6a660640 1 --2- 172.21.15.2:0/3807276406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b641048a0 0x7f7b640fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.714+0000 7f7b6a660640 1 -- 172.21.15.2:0/3807276406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b64101f70 msgr2=0x7f7b641018e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:41.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.714+0000 7f7b6a660640 1 --2- 172.21.15.2:0/3807276406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b64101f70 0x7f7b641018e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.714+0000 7f7b6a660640 1 -- 172.21.15.2:0/3807276406 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c009900 con 0x7f7b641071d0 2024-03-15T12:58:41.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.714+0000 7f7b6a660640 1 --2- 172.21.15.2:0/3807276406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b641071d0 0x7f7b64100460 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c370 tx=0x7f7b5c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:41.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.714+0000 7f7b5affd640 1 -- 172.21.15.2:0/3807276406 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c003970 con 0x7f7b641071d0 2024-03-15T12:58:41.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.714+0000 7f7b5affd640 1 -- 172.21.15.2:0/3807276406 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7b5c004020 con 0x7f7b641071d0 2024-03-15T12:58:41.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.714+0000 7f7b6c0c3640 1 -- 172.21.15.2:0/3807276406 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b54009490 con 0x7f7b641071d0 2024-03-15T12:58:41.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.714+0000 7f7b5affd640 1 -- 172.21.15.2:0/3807276406 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c00b9f0 con 0x7f7b641071d0 2024-03-15T12:58:41.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.714+0000 7f7b6c0c3640 1 -- 172.21.15.2:0/3807276406 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b641a1c30 con 0x7f7b641071d0 2024-03-15T12:58:41.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.715+0000 7f7b6c0c3640 1 -- 172.21.15.2:0/3807276406 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b640fd470 con 0x7f7b641071d0 2024-03-15T12:58:41.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.716+0000 7f7b5affd640 1 -- 172.21.15.2:0/3807276406 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f7b5c00a040 con 0x7f7b641071d0 2024-03-15T12:58:41.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.717+0000 7f7b5affd640 1 --2- 172.21.15.2:0/3807276406 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7b500a1ac0 0x7f7b500a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:41.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.717+0000 7f7b69e5f640 1 --2- 172.21.15.2:0/3807276406 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7b500a1ac0 0x7f7b500a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:41.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.717+0000 7f7b5affd640 1 -- 172.21.15.2:0/3807276406 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f7b5c0cd870 con 0x7f7b641071d0 2024-03-15T12:58:41.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.718+0000 7f7b69e5f640 1 --2- 172.21.15.2:0/3807276406 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7b500a1ac0 0x7f7b500a3f80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7b5402c180 tx=0x7f7b54034b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:41.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.722+0000 7f7b5affd640 1 -- 172.21.15.2:0/3807276406 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7b5c0965e0 con 0x7f7b641071d0 2024-03-15T12:58:41.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.909+0000 7f7b6c0c3640 1 -- 172.21.15.2:0/3807276406 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b6402d000 con 0x7f7b641071d0 2024-03-15T12:58:41.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.909+0000 7f7b5affd640 1 -- 172.21.15.2:0/3807276406 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b5c099ca0 con 0x7f7b641071d0 2024-03-15T12:58:41.895 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-03-15T12:58:41.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.911+0000 7f7b6c0c3640 1 -- 172.21.15.2:0/3807276406 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7b500a1ac0 msgr2=0x7f7b500a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:41.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.911+0000 7f7b6c0c3640 1 --2- 172.21.15.2:0/3807276406 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7b500a1ac0 0x7f7b500a3f80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7b5402c180 tx=0x7f7b54034b40 comp rx=0 tx=0).stop 2024-03-15T12:58:41.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.911+0000 7f7b6c0c3640 1 -- 172.21.15.2:0/3807276406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b641071d0 msgr2=0x7f7b64100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:41.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.911+0000 7f7b6c0c3640 1 --2- 172.21.15.2:0/3807276406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b641071d0 0x7f7b64100460 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c370 tx=0x7f7b5c00c830 comp rx=0 tx=0).stop 2024-03-15T12:58:41.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.911+0000 7f7b6c0c3640 1 -- 172.21.15.2:0/3807276406 shutdown_connections 2024-03-15T12:58:41.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.911+0000 7f7b6c0c3640 1 --2- 172.21.15.2:0/3807276406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b641071d0 0x7f7b64100460 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.911+0000 7f7b6c0c3640 1 --2- 172.21.15.2:0/3807276406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b641048a0 0x7f7b640fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.911+0000 7f7b6c0c3640 1 --2- 172.21.15.2:0/3807276406 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f7b500a1ac0 0x7f7b500a3f80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.911+0000 7f7b6c0c3640 1 --2- 172.21.15.2:0/3807276406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b64101f70 0x7f7b641018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:41.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.911+0000 7f7b6c0c3640 1 -- 172.21.15.2:0/3807276406 >> 172.21.15.2:0/3807276406 conn(0x7f7b640fb980 msgr2=0x7f7b641028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:41.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.911+0000 7f7b6c0c3640 1 -- 172.21.15.2:0/3807276406 shutdown_connections 2024-03-15T12:58:41.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:41.911+0000 7f7b6c0c3640 1 -- 172.21.15.2:0/3807276406 wait complete. 2024-03-15T12:58:41.907 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:41 smithi002 ceph-mon[32060]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-15T12:58:41.907 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:41 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3101168024' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:41.907 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:41 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1783831801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:41.907 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:41 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/926736680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:41.908 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-15T12:58:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.028+0000 7fcceb8bb640 1 -- 172.21.15.2:0/3947353922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4101600 msgr2=0x7fcce4101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.028+0000 7fcceb8bb640 1 --2- 172.21.15.2:0/3947353922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4101600 0x7fcce4101a80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009900 tx=0x7fccd402bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.028+0000 7fcceb8bb640 1 -- 172.21.15.2:0/3947353922 shutdown_connections 2024-03-15T12:58:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.028+0000 7fcceb8bb640 1 --2- 172.21.15.2:0/3947353922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce4101fc0 0x7fcce4108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.028+0000 7fcceb8bb640 1 --2- 172.21.15.2:0/3947353922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4101600 0x7fcce4101a80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.028+0000 7fcceb8bb640 1 --2- 172.21.15.2:0/3947353922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcce4100410 0x7fcce4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.028+0000 7fcceb8bb640 1 -- 172.21.15.2:0/3947353922 >> 172.21.15.2:0/3947353922 conn(0x7fcce40fb9a0 msgr2=0x7fcce40fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.028+0000 7fcceb8bb640 1 -- 172.21.15.2:0/3947353922 shutdown_connections 2024-03-15T12:58:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.028+0000 7fcceb8bb640 1 -- 172.21.15.2:0/3947353922 wait complete. 2024-03-15T12:58:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.028+0000 7fcceb8bb640 1 Processor -- start 2024-03-15T12:58:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.028+0000 7fcceb8bb640 1 -- start start 2024-03-15T12:58:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcceb8bb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcce4100410 0x7fcce4197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcceb8bb640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce4101600 0x7fcce4197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcceb8bb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4101fc0 0x7fcce419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcceb8bb640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce419f170 con 0x7fcce4101fc0 2024-03-15T12:58:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcceb8bb640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce419f2e0 con 0x7fcce4101600 2024-03-15T12:58:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcceb8bb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce419f4a0 con 0x7fcce4100410 2024-03-15T12:58:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcce9e58640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4101fc0 0x7fcce419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcce9e58640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4101fc0 0x7fcce419ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43672/0 (socket says 172.21.15.2:43672) 2024-03-15T12:58:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcce9e58640 1 -- 172.21.15.2:0/1683245139 learned_addr learned my addr 172.21.15.2:0/1683245139 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcce9657640 1 --2- 172.21.15.2:0/1683245139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcce4100410 0x7fcce4197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcce8e56640 1 --2- 172.21.15.2:0/1683245139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce4101600 0x7fcce4197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcce9e58640 1 -- 172.21.15.2:0/1683245139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcce4100410 msgr2=0x7fcce4197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcce9e58640 1 --2- 172.21.15.2:0/1683245139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcce4100410 0x7fcce4197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcce9e58640 1 -- 172.21.15.2:0/1683245139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce4101600 msgr2=0x7fcce4197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcce9e58640 1 --2- 172.21.15.2:0/1683245139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce4101600 0x7fcce4197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcce9e58640 1 -- 172.21.15.2:0/1683245139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd4009560 con 0x7fcce4101fc0 2024-03-15T12:58:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcce9e58640 1 --2- 172.21.15.2:0/1683245139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4101fc0 0x7fcce419ec30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00eb10 tx=0x7fccdc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fccda7fc640 1 -- 172.21.15.2:0/1683245139 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc004b20 con 0x7fcce4101fc0 2024-03-15T12:58:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fccda7fc640 1 -- 172.21.15.2:0/1683245139 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fccdc004c80 con 0x7fcce4101fc0 2024-03-15T12:58:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcceb8bb640 1 -- 172.21.15.2:0/1683245139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce419f780 con 0x7fcce4101fc0 2024-03-15T12:58:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fccda7fc640 1 -- 172.21.15.2:0/1683245139 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc010690 con 0x7fcce4101fc0 2024-03-15T12:58:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.029+0000 7fcceb8bb640 1 -- 172.21.15.2:0/1683245139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce419fc60 con 0x7fcce4101fc0 2024-03-15T12:58:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.030+0000 7fcceb8bb640 1 -- 172.21.15.2:0/1683245139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce410bdf0 con 0x7fcce4101fc0 2024-03-15T12:58:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.031+0000 7fccda7fc640 1 -- 172.21.15.2:0/1683245139 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7fccdc004df0 con 0x7fcce4101fc0 2024-03-15T12:58:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.032+0000 7fccda7fc640 1 --2- 172.21.15.2:0/1683245139 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fccd00a1b40 0x7fccd00a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.032+0000 7fcce9657640 1 --2- 172.21.15.2:0/1683245139 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fccd00a1b40 0x7fccd00a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:42.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.032+0000 7fccda7fc640 1 -- 172.21.15.2:0/1683245139 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fccdc014070 con 0x7fcce4101fc0 2024-03-15T12:58:42.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.033+0000 7fcce9657640 1 --2- 172.21.15.2:0/1683245139 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fccd00a1b40 0x7fccd00a4000 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcce4101460 tx=0x7fcce0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:42.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.037+0000 7fccda7fc640 1 -- 172.21.15.2:0/1683245139 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fccdc096b60 con 0x7fcce4101fc0 2024-03-15T12:58:42.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:41 smithi184 ceph-mon[35993]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-15T12:58:42.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:41 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3101168024' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:42.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:41 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1783831801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:42.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:41 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/926736680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:42.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:41 smithi149 ceph-mon[35927]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-15T12:58:42.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:41 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3101168024' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:42.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:41 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1783831801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:42.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:41 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/926736680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:42.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.218+0000 7fcceb8bb640 1 -- 172.21.15.2:0/1683245139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcce419fee0 con 0x7fcce4101fc0 2024-03-15T12:58:42.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.219+0000 7fccda7fc640 1 -- 172.21.15.2:0/1683245139 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcce419fee0 con 0x7fcce4101fc0 2024-03-15T12:58:42.205 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:58:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.221+0000 7fcceb8bb640 1 -- 172.21.15.2:0/1683245139 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fccd00a1b40 msgr2=0x7fccd00a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.221+0000 7fcceb8bb640 1 --2- 172.21.15.2:0/1683245139 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fccd00a1b40 0x7fccd00a4000 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcce4101460 tx=0x7fcce0005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.221+0000 7fcceb8bb640 1 -- 172.21.15.2:0/1683245139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4101fc0 msgr2=0x7fcce419ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.221+0000 7fcceb8bb640 1 --2- 172.21.15.2:0/1683245139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4101fc0 0x7fcce419ec30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00eb10 tx=0x7fccdc00c2d0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.221+0000 7fcceb8bb640 1 -- 172.21.15.2:0/1683245139 shutdown_connections 2024-03-15T12:58:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.221+0000 7fcceb8bb640 1 --2- 172.21.15.2:0/1683245139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4101fc0 0x7fcce419ec30 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.221+0000 7fcceb8bb640 1 --2- 172.21.15.2:0/1683245139 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fccd00a1b40 0x7fccd00a4000 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.221+0000 7fcceb8bb640 1 --2- 172.21.15.2:0/1683245139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce4101600 0x7fcce4197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.221+0000 7fcceb8bb640 1 --2- 172.21.15.2:0/1683245139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcce4100410 0x7fcce4197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.221+0000 7fcceb8bb640 1 -- 172.21.15.2:0/1683245139 >> 172.21.15.2:0/1683245139 conn(0x7fcce40fb9a0 msgr2=0x7fcce40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.221+0000 7fcceb8bb640 1 -- 172.21.15.2:0/1683245139 shutdown_connections 2024-03-15T12:58:42.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.221+0000 7fcceb8bb640 1 -- 172.21.15.2:0/1683245139 wait complete. 2024-03-15T12:58:42.218 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-15T12:58:42.218 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:58:42.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.335+0000 7fa2d58ff640 1 -- 172.21.15.2:0/2328054118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d0101620 msgr2=0x7fa2d0101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:42.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.335+0000 7fa2d58ff640 1 --2- 172.21.15.2:0/2328054118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d0101620 0x7fa2d0101aa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009860 tx=0x7fa2c402bb70 comp rx=0 tx=0).stop 2024-03-15T12:58:42.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.335+0000 7fa2d58ff640 1 -- 172.21.15.2:0/2328054118 shutdown_connections 2024-03-15T12:58:42.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.335+0000 7fa2d58ff640 1 --2- 172.21.15.2:0/2328054118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2d0101fe0 0x7fa2d0108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.335+0000 7fa2d58ff640 1 --2- 172.21.15.2:0/2328054118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d0101620 0x7fa2d0101aa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.335+0000 7fa2d58ff640 1 --2- 172.21.15.2:0/2328054118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2d0100430 0x7fa2d0100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.335+0000 7fa2d58ff640 1 -- 172.21.15.2:0/2328054118 >> 172.21.15.2:0/2328054118 conn(0x7fa2d00fb9a0 msgr2=0x7fa2d00fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:42.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.335+0000 7fa2d58ff640 1 -- 172.21.15.2:0/2328054118 shutdown_connections 2024-03-15T12:58:42.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.335+0000 7fa2d58ff640 1 -- 172.21.15.2:0/2328054118 wait complete. 2024-03-15T12:58:42.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.335+0000 7fa2d58ff640 1 Processor -- start 2024-03-15T12:58:42.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.335+0000 7fa2d58ff640 1 -- start start 2024-03-15T12:58:42.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2d58ff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d0100430 0x7fa2d006a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:42.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2d58ff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2d0101620 0x7fa2d019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:42.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2d58ff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2d0101fe0 0x7fa2d019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:42.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2d58ff640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d019f110 con 0x7fa2d0101fe0 2024-03-15T12:58:42.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2d58ff640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d019f280 con 0x7fa2d0101620 2024-03-15T12:58:42.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2d58ff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d019f440 con 0x7fa2d0100430 2024-03-15T12:58:42.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2cf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2d0101fe0 0x7fa2d019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:42.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2ceffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d0100430 0x7fa2d006a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:42.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2ceffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d0100430 0x7fa2d006a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:52814/0 (socket says 172.21.15.2:52814) 2024-03-15T12:58:42.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2ce7fc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2d0101620 0x7fa2d019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:42.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2ceffd640 1 -- 172.21.15.2:0/2631948023 learned_addr learned my addr 172.21.15.2:0/2631948023 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:42.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2cf7fe640 1 -- 172.21.15.2:0/2631948023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d0100430 msgr2=0x7fa2d006a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:42.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2cf7fe640 1 --2- 172.21.15.2:0/2631948023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d0100430 0x7fa2d006a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2cf7fe640 1 -- 172.21.15.2:0/2631948023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2d0101620 msgr2=0x7fa2d019c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:42.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2cf7fe640 1 --2- 172.21.15.2:0/2631948023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2d0101620 0x7fa2d019c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2cf7fe640 1 -- 172.21.15.2:0/2631948023 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c4009490 con 0x7fa2d0101fe0 2024-03-15T12:58:42.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2ceffd640 1 --2- 172.21.15.2:0/2631948023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d0100430 0x7fa2d006a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:42.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2ce7fc640 1 --2- 172.21.15.2:0/2631948023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2d0101620 0x7fa2d019c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:42.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2cf7fe640 1 --2- 172.21.15.2:0/2631948023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2d0101fe0 0x7fa2d019e930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009fd0 tx=0x7fa2c800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:42.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2d48fd640 1 -- 172.21.15.2:0/2631948023 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800cd60 con 0x7fa2d0101fe0 2024-03-15T12:58:42.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2d58ff640 1 -- 172.21.15.2:0/2631948023 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d019f6c0 con 0x7fa2d0101fe0 2024-03-15T12:58:42.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2d48fd640 1 -- 172.21.15.2:0/2631948023 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa2c8005020 con 0x7fa2d0101fe0 2024-03-15T12:58:42.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.336+0000 7fa2d48fd640 1 -- 172.21.15.2:0/2631948023 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8010690 con 0x7fa2d0101fe0 2024-03-15T12:58:42.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.337+0000 7fa2d58ff640 1 -- 172.21.15.2:0/2631948023 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d019fba0 con 0x7fa2d0101fe0 2024-03-15T12:58:42.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.337+0000 7fa2d58ff640 1 -- 172.21.15.2:0/2631948023 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d0107750 con 0x7fa2d0101fe0 2024-03-15T12:58:42.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.338+0000 7fa2d48fd640 1 -- 172.21.15.2:0/2631948023 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7fa2c8005570 con 0x7fa2d0101fe0 2024-03-15T12:58:42.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.339+0000 7fa2d48fd640 1 --2- 172.21.15.2:0/2631948023 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1ac0 0x7fa2b80a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:42.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.339+0000 7fa2ceffd640 1 --2- 172.21.15.2:0/2631948023 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1ac0 0x7fa2b80a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:42.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.339+0000 7fa2d48fd640 1 -- 172.21.15.2:0/2631948023 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fa2c8014070 con 0x7fa2d0101fe0 2024-03-15T12:58:42.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.339+0000 7fa2ceffd640 1 --2- 172.21.15.2:0/2631948023 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1ac0 0x7fa2b80a3f80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0011550 tx=0x7fa2c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:42.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.343+0000 7fa2d48fd640 1 -- 172.21.15.2:0/2631948023 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa2c8096d00 con 0x7fa2d0101fe0 2024-03-15T12:58:42.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.565+0000 7fa2d58ff640 1 -- 172.21.15.2:0/2631948023 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2d01071c0 con 0x7fa2d0101fe0 2024-03-15T12:58:42.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.566+0000 7fa2d48fd640 1 -- 172.21.15.2:0/2631948023 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11453 (secure 0 0 0) 0x7fa2c809a3c0 con 0x7fa2d0101fe0 2024-03-15T12:58:42.552 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:42.552 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T12:58:37.814324+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24434,24539,24460,24464,24500]},"id":1}]} 2024-03-15T12:58:42.552 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 29 2024-03-15T12:58:42.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.568+0000 7fa2d58ff640 1 -- 172.21.15.2:0/2631948023 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1ac0 msgr2=0x7fa2b80a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:42.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.568+0000 7fa2d58ff640 1 --2- 172.21.15.2:0/2631948023 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1ac0 0x7fa2b80a3f80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0011550 tx=0x7fa2c0009260 comp rx=0 tx=0).stop 2024-03-15T12:58:42.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.568+0000 7fa2d58ff640 1 -- 172.21.15.2:0/2631948023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2d0101fe0 msgr2=0x7fa2d019e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:42.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.568+0000 7fa2d58ff640 1 --2- 172.21.15.2:0/2631948023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2d0101fe0 0x7fa2d019e930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009fd0 tx=0x7fa2c800ee50 comp rx=0 tx=0).stop 2024-03-15T12:58:42.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.568+0000 7fa2d58ff640 1 -- 172.21.15.2:0/2631948023 shutdown_connections 2024-03-15T12:58:42.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.568+0000 7fa2d58ff640 1 --2- 172.21.15.2:0/2631948023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2d0101fe0 0x7fa2d019e930 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.568+0000 7fa2d58ff640 1 --2- 172.21.15.2:0/2631948023 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1ac0 0x7fa2b80a3f80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.568+0000 7fa2d58ff640 1 --2- 172.21.15.2:0/2631948023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2d0101620 0x7fa2d019c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.568+0000 7fa2d58ff640 1 --2- 172.21.15.2:0/2631948023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d0100430 0x7fa2d006a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.568+0000 7fa2d58ff640 1 -- 172.21.15.2:0/2631948023 >> 172.21.15.2:0/2631948023 conn(0x7fa2d00fb9a0 msgr2=0x7fa2d0104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:42.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.568+0000 7fa2d58ff640 1 -- 172.21.15.2:0/2631948023 shutdown_connections 2024-03-15T12:58:42.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.568+0000 7fa2d58ff640 1 -- 172.21.15.2:0/2631948023 wait complete. 2024-03-15T12:58:42.565 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-15T12:58:42.565 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-15T12:58:42.566 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-15T12:58:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.682+0000 7f0566847640 1 -- 172.21.15.2:0/720595415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560102040 msgr2=0x7f05601088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.682+0000 7f0566847640 1 --2- 172.21.15.2:0/720595415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560102040 0x7f05601088d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0558009ca0 tx=0x7f055802c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.682+0000 7f0566847640 1 -- 172.21.15.2:0/720595415 shutdown_connections 2024-03-15T12:58:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.682+0000 7f0566847640 1 --2- 172.21.15.2:0/720595415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560102040 0x7f05601088d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.682+0000 7f0566847640 1 --2- 172.21.15.2:0/720595415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0560101680 0x7f0560101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.682+0000 7f0566847640 1 --2- 172.21.15.2:0/720595415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0560100490 0x7f05601008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.682+0000 7f0566847640 1 -- 172.21.15.2:0/720595415 >> 172.21.15.2:0/720595415 conn(0x7f05600fba20 msgr2=0x7f05600fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.682+0000 7f0566847640 1 -- 172.21.15.2:0/720595415 shutdown_connections 2024-03-15T12:58:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.682+0000 7f0566847640 1 -- 172.21.15.2:0/720595415 wait complete. 2024-03-15T12:58:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.683+0000 7f0566847640 1 Processor -- start 2024-03-15T12:58:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.683+0000 7f0566847640 1 -- start start 2024-03-15T12:58:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.683+0000 7f0566847640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0560100490 0x7f0560197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.683+0000 7f0566847640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560101680 0x7f0560197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.683+0000 7f0566847640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0560102040 0x7f056019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.683+0000 7f0566847640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056019f070 con 0x7f0560101680 2024-03-15T12:58:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.683+0000 7f0566847640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056019f1e0 con 0x7f0560100490 2024-03-15T12:58:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.683+0000 7f0566847640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056019f3a0 con 0x7f0560102040 2024-03-15T12:58:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.683+0000 7f055f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560101680 0x7f0560197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.683+0000 7f055f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560101680 0x7f0560197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43696/0 (socket says 172.21.15.2:43696) 2024-03-15T12:58:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.683+0000 7f055f7fe640 1 -- 172.21.15.2:0/3169452078 learned_addr learned my addr 172.21.15.2:0/3169452078 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.683+0000 7f055ffff640 1 --2- 172.21.15.2:0/3169452078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0560100490 0x7f0560197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.683+0000 7f0564de4640 1 --2- 172.21.15.2:0/3169452078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0560102040 0x7f056019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.684+0000 7f055f7fe640 1 -- 172.21.15.2:0/3169452078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0560102040 msgr2=0x7f056019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.684+0000 7f055f7fe640 1 --2- 172.21.15.2:0/3169452078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0560102040 0x7f056019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.684+0000 7f055f7fe640 1 -- 172.21.15.2:0/3169452078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0560100490 msgr2=0x7f0560197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.684+0000 7f055f7fe640 1 --2- 172.21.15.2:0/3169452078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0560100490 0x7f0560197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.684+0000 7f055f7fe640 1 -- 172.21.15.2:0/3169452078 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0550009490 con 0x7f0560101680 2024-03-15T12:58:42.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.684+0000 7f055ffff640 1 --2- 172.21.15.2:0/3169452078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0560100490 0x7f0560197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:42.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.684+0000 7f055f7fe640 1 --2- 172.21.15.2:0/3169452078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560101680 0x7f0560197ac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f055000ea60 tx=0x7f055000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:42.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.684+0000 7f055d7fa640 1 -- 172.21.15.2:0/3169452078 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055000cde0 con 0x7f0560101680 2024-03-15T12:58:42.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.684+0000 7f0566847640 1 -- 172.21.15.2:0/3169452078 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0558009900 con 0x7f0560101680 2024-03-15T12:58:42.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.684+0000 7f055d7fa640 1 -- 172.21.15.2:0/3169452078 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f05500035f0 con 0x7f0560101680 2024-03-15T12:58:42.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.684+0000 7f055d7fa640 1 -- 172.21.15.2:0/3169452078 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550010690 con 0x7f0560101680 2024-03-15T12:58:42.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.684+0000 7f0566847640 1 -- 172.21.15.2:0/3169452078 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056019fa50 con 0x7f0560101680 2024-03-15T12:58:42.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.685+0000 7f0566847640 1 -- 172.21.15.2:0/3169452078 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056010bdf0 con 0x7f0560101680 2024-03-15T12:58:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.686+0000 7f055d7fa640 1 -- 172.21.15.2:0/3169452078 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f0550003760 con 0x7f0560101680 2024-03-15T12:58:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.686+0000 7f055d7fa640 1 --2- 172.21.15.2:0/3169452078 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f05480a1ac0 0x7f05480a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.687+0000 7f055ffff640 1 --2- 172.21.15.2:0/3169452078 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f05480a1ac0 0x7f05480a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.687+0000 7f055d7fa640 1 -- 172.21.15.2:0/3169452078 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f0550014070 con 0x7f0560101680 2024-03-15T12:58:42.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.687+0000 7f055ffff640 1 --2- 172.21.15.2:0/3169452078 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f05480a1ac0 0x7f05480a3f80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f05601014e0 tx=0x7f0554005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:42.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.690+0000 7f055d7fa640 1 -- 172.21.15.2:0/3169452078 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0550096c50 con 0x7f0560101680 2024-03-15T12:58:42.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.869+0000 7f0566847640 1 -- 172.21.15.2:0/3169452078 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05600fdb60 con 0x7f05480a1ac0 2024-03-15T12:58:42.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.878+0000 7f055d7fa640 1 -- 172.21.15.2:0/3169452078 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340536 (secure 0 0 0) 0x7f05600fdb60 con 0x7f05480a1ac0 2024-03-15T12:58:42.869 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-03-15T12:58:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.881+0000 7f0566847640 1 -- 172.21.15.2:0/3169452078 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f05480a1ac0 msgr2=0x7f05480a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.881+0000 7f0566847640 1 --2- 172.21.15.2:0/3169452078 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f05480a1ac0 0x7f05480a3f80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f05601014e0 tx=0x7f0554005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.881+0000 7f0566847640 1 -- 172.21.15.2:0/3169452078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560101680 msgr2=0x7f0560197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.881+0000 7f0566847640 1 --2- 172.21.15.2:0/3169452078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560101680 0x7f0560197ac0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f055000ea60 tx=0x7f055000ef20 comp rx=0 tx=0).stop 2024-03-15T12:58:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.881+0000 7f0566847640 1 -- 172.21.15.2:0/3169452078 shutdown_connections 2024-03-15T12:58:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.881+0000 7f0566847640 1 --2- 172.21.15.2:0/3169452078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0560102040 0x7f056019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.881+0000 7f0566847640 1 --2- 172.21.15.2:0/3169452078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560101680 0x7f0560197ac0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.882+0000 7f0566847640 1 --2- 172.21.15.2:0/3169452078 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f05480a1ac0 0x7f05480a3f80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.882+0000 7f0566847640 1 --2- 172.21.15.2:0/3169452078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0560100490 0x7f0560197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.882+0000 7f0566847640 1 -- 172.21.15.2:0/3169452078 >> 172.21.15.2:0/3169452078 conn(0x7f05600fba20 msgr2=0x7f05600fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.882+0000 7f0566847640 1 -- 172.21.15.2:0/3169452078 shutdown_connections 2024-03-15T12:58:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:42.882+0000 7f0566847640 1 -- 172.21.15.2:0/3169452078 wait complete. 2024-03-15T12:58:42.877 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:42 smithi002 ceph-mon[32060]: pgmap v362: 129 pgs: 129 active+clean; 654 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-03-15T12:58:42.878 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:42 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3807276406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:42.878 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:42 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1683245139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:42.878 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:42 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2631948023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:42.879 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-03-15T12:58:42.776711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669661,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1360,"num_read_kb":820,"num_write":454,"num_write_kb":3107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400840,"kb_used_data":14248,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540984,"statfs":{"total":1151940427776,"available":1151529967616,"internally_reserved":0,"allocated":14589952,"data_stored":7981696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10081,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":64,"num_write":25,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003507"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.789229+0000","last_change":"2024-03-15T12:56:21.210343+0000","last_active":"2024-03-15T12:57:48.789229+0000","last_peered":"2024-03-15T12:57:48.789229+0000","last_clean":"2024-03-15T12:57:48.789229+0000","last_became_active":"2024-03-15T12:56:21.209787+0000","last_became_peered":"2024-03-15T12:56:21.209787+0000","last_unstale":"2024-03-15T12:57:48.789229+0000","last_undegraded":"2024-03-15T12:57:48.789229+0000","last_fullsized":"2024-03-15T12:57:48.789229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:32:21.204944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.778728+0000","last_change":"2024-03-15T12:56:21.214153+0000","last_active":"2024-03-15T12:57:48.778728+0000","last_peered":"2024-03-15T12:57:48.778728+0000","last_clean":"2024-03-15T12:57:48.778728+0000","last_became_active":"2024-03-15T12:56:21.213819+0000","last_became_peered":"2024-03-15T12:56:21.213819+0000","last_unstale":"2024-03-15T12:57:48.778728+0000","last_undegraded":"2024-03-15T12:57:48.778728+0000","last_fullsized":"2024-03-15T12:57:48.778728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:31:43.264698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.792740+0000","last_change":"2024-03-15T12:56:21.221678+0000","last_active":"2024-03-15T12:57:48.792740+0000","last_peered":"2024-03-15T12:57:48.792740+0000","last_clean":"2024-03-15T12:57:48.792740+0000","last_became_active":"2024-03-15T12:56:21.221272+0000","last_became_peered":"2024-03-15T12:56:21.221272+0000","last_unstale":"2024-03-15T12:57:48.792740+0000","last_undegraded":"2024-03-15T12:57:48.792740+0000","last_fullsized":"2024-03-15T12:57:48.792740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:01:50.778707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.754638+0000","last_change":"2024-03-15T12:56:21.212939+0000","last_active":"2024-03-15T12:57:48.754638+0000","last_peered":"2024-03-15T12:57:48.754638+0000","last_clean":"2024-03-15T12:57:48.754638+0000","last_became_active":"2024-03-15T12:56:21.212437+0000","last_became_peered":"2024-03-15T12:56:21.212437+0000","last_unstale":"2024-03-15T12:57:48.754638+0000","last_undegraded":"2024-03-15T12:57:48.754638+0000","last_fullsized":"2024-03-15T12:57:48.754638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:34:11.377020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.771047+0000","last_change":"2024-03-15T12:56:21.221433+0000","last_active":"2024-03-15T12:57:48.771047+0000","last_peered":"2024-03-15T12:57:48.771047+0000","last_clean":"2024-03-15T12:57:48.771047+0000","last_became_active":"2024-03-15T12:56:21.220605+0000","last_became_peered":"2024-03-15T12:56:21.220605+0000","last_unstale":"2024-03-15T12:57:48.771047+0000","last_undegraded":"2024-03-15T12:57:48.771047+0000","last_fullsized":"2024-03-15T12:57:48.771047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:20:25.169387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784751+0000","last_change":"2024-03-15T12:56:21.216539+0000","last_active":"2024-03-15T12:57:48.784751+0000","last_peered":"2024-03-15T12:57:48.784751+0000","last_clean":"2024-03-15T12:57:48.784751+0000","last_became_active":"2024-03-15T12:56:21.216197+0000","last_became_peered":"2024-03-15T12:56:21.216197+0000","last_unstale":"2024-03-15T12:57:48.784751+0000","last_undegraded":"2024-03-15T12:57:48.784751+0000","last_fullsized":"2024-03-15T12:57:48.784751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:00:23.186467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.779848+0000","last_change":"2024-03-15T12:56:21.213103+0000","last_active":"2024-03-15T12:57:48.779848+0000","last_peered":"2024-03-15T12:57:48.779848+0000","last_clean":"2024-03-15T12:57:48.779848+0000","last_became_active":"2024-03-15T12:56:21.212789+0000","last_became_peered":"2024-03-15T12:56:21.212789+0000","last_unstale":"2024-03-15T12:57:48.779848+0000","last_undegraded":"2024-03-15T12:57:48.779848+0000","last_fullsized":"2024-03-15T12:57:48.779848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:41:50.140915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.785673+0000","last_change":"2024-03-15T12:56:21.214767+0000","last_active":"2024-03-15T12:57:48.785673+0000","last_peered":"2024-03-15T12:57:48.785673+0000","last_clean":"2024-03-15T12:57:48.785673+0000","last_became_active":"2024-03-15T12:56:21.213896+0000","last_became_peered":"2024-03-15T12:56:21.213896+0000","last_unstale":"2024-03-15T12:57:48.785673+0000","last_undegraded":"2024-03-15T12:57:48.785673+0000","last_fullsized":"2024-03-15T12:57:48.785673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:33:53.819906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.796148+0000","last_change":"2024-03-15T12:56:21.218566+0000","last_active":"2024-03-15T12:57:48.796148+0000","last_peered":"2024-03-15T12:57:48.796148+0000","last_clean":"2024-03-15T12:57:48.796148+0000","last_became_active":"2024-03-15T12:56:21.218234+0000","last_became_peered":"2024-03-15T12:56:21.218234+0000","last_unstale":"2024-03-15T12:57:48.796148+0000","last_undegraded":"2024-03-15T12:57:48.796148+0000","last_fullsized":"2024-03-15T12:57:48.796148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:56:46.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784768+0000","last_change":"2024-03-15T12:56:21.219367+0000","last_active":"2024-03-15T12:57:48.784768+0000","last_peered":"2024-03-15T12:57:48.784768+0000","last_clean":"2024-03-15T12:57:48.784768+0000","last_became_active":"2024-03-15T12:56:21.217541+0000","last_became_peered":"2024-03-15T12:56:21.217541+0000","last_unstale":"2024-03-15T12:57:48.784768+0000","last_undegraded":"2024-03-15T12:57:48.784768+0000","last_fullsized":"2024-03-15T12:57:48.784768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:19:12.939394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798017+0000","last_change":"2024-03-15T12:56:21.222246+0000","last_active":"2024-03-15T12:57:48.798017+0000","last_peered":"2024-03-15T12:57:48.798017+0000","last_clean":"2024-03-15T12:57:48.798017+0000","last_became_active":"2024-03-15T12:56:21.221883+0000","last_became_peered":"2024-03-15T12:56:21.221883+0000","last_unstale":"2024-03-15T12:57:48.798017+0000","last_undegraded":"2024-03-15T12:57:48.798017+0000","last_fullsized":"2024-03-15T12:57:48.798017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:42:06.892581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.771534+0000","last_change":"2024-03-15T12:57:19.841386+0000","last_active":"2024-03-15T12:57:48.771534+0000","last_peered":"2024-03-15T12:57:48.771534+0000","last_clean":"2024-03-15T12:57:48.771534+0000","last_became_active":"2024-03-15T12:57:19.840981+0000","last_became_peered":"2024-03-15T12:57:19.840981+0000","last_unstale":"2024-03-15T12:57:48.771534+0000","last_undegraded":"2024-03-15T12:57:48.771534+0000","last_fullsized":"2024-03-15T12:57:48.771534+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:57:28.443452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.776665+0000","last_change":"2024-03-15T12:56:21.199631+0000","last_active":"2024-03-15T12:57:48.776665+0000","last_peered":"2024-03-15T12:57:48.776665+0000","last_clean":"2024-03-15T12:57:48.776665+0000","last_became_active":"2024-03-15T12:56:21.199230+0000","last_became_peered":"2024-03-15T12:56:21.199230+0000","last_unstale":"2024-03-15T12:57:48.776665+0000","last_undegraded":"2024-03-15T12:57:48.776665+0000","last_fullsized":"2024-03-15T12:57:48.776665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:45:51.356803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.775081+0000","last_change":"2024-03-15T12:56:21.211449+0000","last_active":"2024-03-15T12:57:48.775081+0000","last_peered":"2024-03-15T12:57:48.775081+0000","last_clean":"2024-03-15T12:57:48.775081+0000","last_became_active":"2024-03-15T12:56:21.211001+0000","last_became_peered":"2024-03-15T12:56:21.211001+0000","last_unstale":"2024-03-15T12:57:48.775081+0000","last_undegraded":"2024-03-15T12:57:48.775081+0000","last_fullsized":"2024-03-15T12:57:48.775081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:32:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.779936+0000","last_change":"2024-03-15T12:56:21.212001+0000","last_active":"2024-03-15T12:57:48.779936+0000","last_peered":"2024-03-15T12:57:48.779936+0000","last_clean":"2024-03-15T12:57:48.779936+0000","last_became_active":"2024-03-15T12:56:21.211497+0000","last_became_peered":"2024-03-15T12:56:21.211497+0000","last_unstale":"2024-03-15T12:57:48.779936+0000","last_undegraded":"2024-03-15T12:57:48.779936+0000","last_fullsized":"2024-03-15T12:57:48.779936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:27:16.029202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.799977+0000","last_change":"2024-03-15T12:56:21.222368+0000","last_active":"2024-03-15T12:57:48.799977+0000","last_peered":"2024-03-15T12:57:48.799977+0000","last_clean":"2024-03-15T12:57:48.799977+0000","last_became_active":"2024-03-15T12:56:21.221356+0000","last_became_peered":"2024-03-15T12:56:21.221356+0000","last_unstale":"2024-03-15T12:57:48.799977+0000","last_undegraded":"2024-03-15T12:57:48.799977+0000","last_fullsized":"2024-03-15T12:57:48.799977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:15:04.125220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.789234+0000","last_change":"2024-03-15T12:56:21.211323+0000","last_active":"2024-03-15T12:57:48.789234+0000","last_peered":"2024-03-15T12:57:48.789234+0000","last_clean":"2024-03-15T12:57:48.789234+0000","last_became_active":"2024-03-15T12:56:21.210983+0000","last_became_peered":"2024-03-15T12:56:21.210983+0000","last_unstale":"2024-03-15T12:57:48.789234+0000","last_undegraded":"2024-03-15T12:57:48.789234+0000","last_fullsized":"2024-03-15T12:57:48.789234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:10:51.487985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.775434+0000","last_change":"2024-03-15T12:56:21.209435+0000","last_active":"2024-03-15T12:57:48.775434+0000","last_peered":"2024-03-15T12:57:48.775434+0000","last_clean":"2024-03-15T12:57:48.775434+0000","last_became_active":"2024-03-15T12:56:21.208981+0000","last_became_peered":"2024-03-15T12:56:21.208981+0000","last_unstale":"2024-03-15T12:57:48.775434+0000","last_undegraded":"2024-03-15T12:57:48.775434+0000","last_fullsized":"2024-03-15T12:57:48.775434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:47:59.142040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.768999+0000","last_change":"2024-03-15T12:56:21.222014+0000","last_active":"2024-03-15T12:57:48.768999+0000","last_peered":"2024-03-15T12:57:48.768999+0000","last_clean":"2024-03-15T12:57:48.768999+0000","last_became_active":"2024-03-15T12:56:21.221548+0000","last_became_peered":"2024-03-15T12:56:21.221548+0000","last_unstale":"2024-03-15T12:57:48.768999+0000","last_undegraded":"2024-03-15T12:57:48.768999+0000","last_fullsized":"2024-03-15T12:57:48.768999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:12:48.943874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834595+0000","last_change":"2024-03-15T12:56:25.447200+0000","last_active":"2024-03-15T12:57:19.834595+0000","last_peered":"2024-03-15T12:57:19.834595+0000","last_clean":"2024-03-15T12:57:19.834595+0000","last_became_active":"2024-03-15T12:56:25.446516+0000","last_became_peered":"2024-03-15T12:56:25.446516+0000","last_unstale":"2024-03-15T12:57:19.834595+0000","last_undegraded":"2024-03-15T12:57:19.834595+0000","last_fullsized":"2024-03-15T12:57:19.834595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:06:46.221772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":160,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:38.577105+0000","last_change":"2024-03-15T12:56:20.210826+0000","last_active":"2024-03-15T12:58:38.577105+0000","last_peered":"2024-03-15T12:58:38.577105+0000","last_clean":"2024-03-15T12:58:38.577105+0000","last_became_active":"2024-03-15T12:56:20.210472+0000","last_became_peered":"2024-03-15T12:56:20.210472+0000","last_unstale":"2024-03-15T12:58:38.577105+0000","last_undegraded":"2024-03-15T12:58:38.577105+0000","last_fullsized":"2024-03-15T12:58:38.577105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:33:22.896058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780590+0000","last_change":"2024-03-15T12:56:21.224665+0000","last_active":"2024-03-15T12:57:48.780590+0000","last_peered":"2024-03-15T12:57:48.780590+0000","last_clean":"2024-03-15T12:57:48.780590+0000","last_became_active":"2024-03-15T12:56:21.223826+0000","last_became_peered":"2024-03-15T12:56:21.223826+0000","last_unstale":"2024-03-15T12:57:48.780590+0000","last_undegraded":"2024-03-15T12:57:48.780590+0000","last_fullsized":"2024-03-15T12:57:48.780590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:11:57.576701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837559+0000","last_change":"2024-03-15T12:56:25.444665+0000","last_active":"2024-03-15T12:57:19.837559+0000","last_peered":"2024-03-15T12:57:19.837559+0000","last_clean":"2024-03-15T12:57:19.837559+0000","last_became_active":"2024-03-15T12:56:25.444275+0000","last_became_peered":"2024-03-15T12:56:25.444275+0000","last_unstale":"2024-03-15T12:57:19.837559+0000","last_undegraded":"2024-03-15T12:57:19.837559+0000","last_fullsized":"2024-03-15T12:57:19.837559+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:29:46.470405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104609+0000","last_change":"2024-03-15T12:56:20.215873+0000","last_active":"2024-03-15T12:57:20.104609+0000","last_peered":"2024-03-15T12:57:20.104609+0000","last_clean":"2024-03-15T12:57:20.104609+0000","last_became_active":"2024-03-15T12:56:20.215505+0000","last_became_peered":"2024-03-15T12:56:20.215505+0000","last_unstale":"2024-03-15T12:57:20.104609+0000","last_undegraded":"2024-03-15T12:57:20.104609+0000","last_fullsized":"2024-03-15T12:57:20.104609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:01:57.317382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.792776+0000","last_change":"2024-03-15T12:56:21.218200+0000","last_active":"2024-03-15T12:57:48.792776+0000","last_peered":"2024-03-15T12:57:48.792776+0000","last_clean":"2024-03-15T12:57:48.792776+0000","last_became_active":"2024-03-15T12:56:21.217768+0000","last_became_peered":"2024-03-15T12:56:21.217768+0000","last_unstale":"2024-03-15T12:57:48.792776+0000","last_undegraded":"2024-03-15T12:57:48.792776+0000","last_fullsized":"2024-03-15T12:57:48.792776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:39:42.443542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837005+0000","last_change":"2024-03-15T12:56:25.444220+0000","last_active":"2024-03-15T12:57:19.837005+0000","last_peered":"2024-03-15T12:57:19.837005+0000","last_clean":"2024-03-15T12:57:19.837005+0000","last_became_active":"2024-03-15T12:56:25.443361+0000","last_became_peered":"2024-03-15T12:56:25.443361+0000","last_unstale":"2024-03-15T12:57:19.837005+0000","last_undegraded":"2024-03-15T12:57:19.837005+0000","last_fullsized":"2024-03-15T12:57:19.837005+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:21:11.593526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837757+0000","last_change":"2024-03-15T12:56:20.218241+0000","last_active":"2024-03-15T12:57:19.837757+0000","last_peered":"2024-03-15T12:57:19.837757+0000","last_clean":"2024-03-15T12:57:19.837757+0000","last_became_active":"2024-03-15T12:56:20.217339+0000","last_became_peered":"2024-03-15T12:56:20.217339+0000","last_unstale":"2024-03-15T12:57:19.837757+0000","last_undegraded":"2024-03-15T12:57:19.837757+0000","last_fullsized":"2024-03-15T12:57:19.837757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:01:22.156502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798260+0000","last_change":"2024-03-15T12:56:21.217224+0000","last_active":"2024-03-15T12:57:48.798260+0000","last_peered":"2024-03-15T12:57:48.798260+0000","last_clean":"2024-03-15T12:57:48.798260+0000","last_became_active":"2024-03-15T12:56:21.216868+0000","last_became_peered":"2024-03-15T12:56:21.216868+0000","last_unstale":"2024-03-15T12:57:48.798260+0000","last_undegraded":"2024-03-15T12:57:48.798260+0000","last_fullsized":"2024-03-15T12:57:48.798260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:03:32.668463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836741+0000","last_change":"2024-03-15T12:56:25.443861+0000","last_active":"2024-03-15T12:57:19.836741+0000","last_peered":"2024-03-15T12:57:19.836741+0000","last_clean":"2024-03-15T12:57:19.836741+0000","last_became_active":"2024-03-15T12:56:25.443386+0000","last_became_peered":"2024-03-15T12:56:25.443386+0000","last_unstale":"2024-03-15T12:57:19.836741+0000","last_undegraded":"2024-03-15T12:57:19.836741+0000","last_fullsized":"2024-03-15T12:57:19.836741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:00:23.063425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837030+0000","last_change":"2024-03-15T12:56:20.212451+0000","last_active":"2024-03-15T12:57:19.837030+0000","last_peered":"2024-03-15T12:57:19.837030+0000","last_clean":"2024-03-15T12:57:19.837030+0000","last_became_active":"2024-03-15T12:56:20.212118+0000","last_became_peered":"2024-03-15T12:56:20.212118+0000","last_unstale":"2024-03-15T12:57:19.837030+0000","last_undegraded":"2024-03-15T12:57:19.837030+0000","last_fullsized":"2024-03-15T12:57:19.837030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:24:33.667652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.766138+0000","last_change":"2024-03-15T12:57:20.107434+0000","last_active":"2024-03-15T12:57:48.766138+0000","last_peered":"2024-03-15T12:57:48.766138+0000","last_clean":"2024-03-15T12:57:48.766138+0000","last_became_active":"2024-03-15T12:57:20.107075+0000","last_became_peered":"2024-03-15T12:57:20.107075+0000","last_unstale":"2024-03-15T12:57:48.766138+0000","last_undegraded":"2024-03-15T12:57:48.766138+0000","last_fullsized":"2024-03-15T12:57:48.766138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:24:44.944629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834590+0000","last_change":"2024-03-15T12:56:25.438194+0000","last_active":"2024-03-15T12:57:19.834590+0000","last_peered":"2024-03-15T12:57:19.834590+0000","last_clean":"2024-03-15T12:57:19.834590+0000","last_became_active":"2024-03-15T12:56:25.437769+0000","last_became_peered":"2024-03-15T12:56:25.437769+0000","last_unstale":"2024-03-15T12:57:19.834590+0000","last_undegraded":"2024-03-15T12:57:19.834590+0000","last_fullsized":"2024-03-15T12:57:19.834590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:29:44.398731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:37.528626+0000","last_change":"2024-03-15T12:56:20.210552+0000","last_active":"2024-03-15T12:58:37.528626+0000","last_peered":"2024-03-15T12:58:37.528626+0000","last_clean":"2024-03-15T12:58:37.528626+0000","last_became_active":"2024-03-15T12:56:20.210191+0000","last_became_peered":"2024-03-15T12:56:20.210191+0000","last_unstale":"2024-03-15T12:58:37.528626+0000","last_undegraded":"2024-03-15T12:58:37.528626+0000","last_fullsized":"2024-03-15T12:58:37.528626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:25:48.225266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.754896+0000","last_change":"2024-03-15T12:56:21.220582+0000","last_active":"2024-03-15T12:57:48.754896+0000","last_peered":"2024-03-15T12:57:48.754896+0000","last_clean":"2024-03-15T12:57:48.754896+0000","last_became_active":"2024-03-15T12:56:21.220212+0000","last_became_peered":"2024-03-15T12:56:21.220212+0000","last_unstale":"2024-03-15T12:57:48.754896+0000","last_undegraded":"2024-03-15T12:57:48.754896+0000","last_fullsized":"2024-03-15T12:57:48.754896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:00:54.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834490+0000","last_change":"2024-03-15T12:56:25.446410+0000","last_active":"2024-03-15T12:57:19.834490+0000","last_peered":"2024-03-15T12:57:19.834490+0000","last_clean":"2024-03-15T12:57:19.834490+0000","last_became_active":"2024-03-15T12:56:25.445554+0000","last_became_peered":"2024-03-15T12:56:25.445554+0000","last_unstale":"2024-03-15T12:57:19.834490+0000","last_undegraded":"2024-03-15T12:57:19.834490+0000","last_fullsized":"2024-03-15T12:57:19.834490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:35:08.049374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.839292+0000","last_change":"2024-03-15T12:56:20.212599+0000","last_active":"2024-03-15T12:57:19.839292+0000","last_peered":"2024-03-15T12:57:19.839292+0000","last_clean":"2024-03-15T12:57:19.839292+0000","last_became_active":"2024-03-15T12:56:20.212289+0000","last_became_peered":"2024-03-15T12:56:20.212289+0000","last_unstale":"2024-03-15T12:57:19.839292+0000","last_undegraded":"2024-03-15T12:57:19.839292+0000","last_fullsized":"2024-03-15T12:57:19.839292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:12:51.036882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.764372+0000","last_change":"2024-03-15T12:56:21.219407+0000","last_active":"2024-03-15T12:57:48.764372+0000","last_peered":"2024-03-15T12:57:48.764372+0000","last_clean":"2024-03-15T12:57:48.764372+0000","last_became_active":"2024-03-15T12:56:21.219035+0000","last_became_peered":"2024-03-15T12:56:21.219035+0000","last_unstale":"2024-03-15T12:57:48.764372+0000","last_undegraded":"2024-03-15T12:57:48.764372+0000","last_fullsized":"2024-03-15T12:57:48.764372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:35:43.129767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.842180+0000","last_change":"2024-03-15T12:57:19.841693+0000","last_active":"2024-03-15T12:57:19.842180+0000","last_peered":"2024-03-15T12:57:19.842180+0000","last_clean":"2024-03-15T12:57:19.842180+0000","last_became_active":"2024-03-15T12:57:19.841275+0000","last_became_peered":"2024-03-15T12:57:19.841275+0000","last_unstale":"2024-03-15T12:57:19.842180+0000","last_undegraded":"2024-03-15T12:57:19.842180+0000","last_fullsized":"2024-03-15T12:57:19.842180+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:41:18.147235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837559+0000","last_change":"2024-03-15T12:56:20.209933+0000","last_active":"2024-03-15T12:57:19.837559+0000","last_peered":"2024-03-15T12:57:19.837559+0000","last_clean":"2024-03-15T12:57:19.837559+0000","last_became_active":"2024-03-15T12:56:20.209047+0000","last_became_peered":"2024-03-15T12:56:20.209047+0000","last_unstale":"2024-03-15T12:57:19.837559+0000","last_undegraded":"2024-03-15T12:57:19.837559+0000","last_fullsized":"2024-03-15T12:57:19.837559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:24:05.551745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782760+0000","last_change":"2024-03-15T12:56:21.215198+0000","last_active":"2024-03-15T12:57:48.782760+0000","last_peered":"2024-03-15T12:57:48.782760+0000","last_clean":"2024-03-15T12:57:48.782760+0000","last_became_active":"2024-03-15T12:56:21.214844+0000","last_became_peered":"2024-03-15T12:56:21.214844+0000","last_unstale":"2024-03-15T12:57:48.782760+0000","last_undegraded":"2024-03-15T12:57:48.782760+0000","last_fullsized":"2024-03-15T12:57:48.782760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:37:08.398317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836412+0000","last_change":"2024-03-15T12:56:25.437047+0000","last_active":"2024-03-15T12:57:19.836412+0000","last_peered":"2024-03-15T12:57:19.836412+0000","last_clean":"2024-03-15T12:57:19.836412+0000","last_became_active":"2024-03-15T12:56:25.436595+0000","last_became_peered":"2024-03-15T12:56:25.436595+0000","last_unstale":"2024-03-15T12:57:19.836412+0000","last_undegraded":"2024-03-15T12:57:19.836412+0000","last_fullsized":"2024-03-15T12:57:19.836412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:59:55.147044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103923+0000","last_change":"2024-03-15T12:56:20.206348+0000","last_active":"2024-03-15T12:57:20.103923+0000","last_peered":"2024-03-15T12:57:20.103923+0000","last_clean":"2024-03-15T12:57:20.103923+0000","last_became_active":"2024-03-15T12:56:20.206013+0000","last_became_peered":"2024-03-15T12:56:20.206013+0000","last_unstale":"2024-03-15T12:57:20.103923+0000","last_undegraded":"2024-03-15T12:57:20.103923+0000","last_fullsized":"2024-03-15T12:57:20.103923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:32:26.625982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782151+0000","last_change":"2024-03-15T12:56:21.216517+0000","last_active":"2024-03-15T12:57:48.782151+0000","last_peered":"2024-03-15T12:57:48.782151+0000","last_clean":"2024-03-15T12:57:48.782151+0000","last_became_active":"2024-03-15T12:56:21.216076+0000","last_became_peered":"2024-03-15T12:56:21.216076+0000","last_unstale":"2024-03-15T12:57:48.782151+0000","last_undegraded":"2024-03-15T12:57:48.782151+0000","last_fullsized":"2024-03-15T12:57:48.782151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:19:35.547273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104273+0000","last_change":"2024-03-15T12:56:25.437367+0000","last_active":"2024-03-15T12:57:20.104273+0000","last_peered":"2024-03-15T12:57:20.104273+0000","last_clean":"2024-03-15T12:57:20.104273+0000","last_became_active":"2024-03-15T12:56:25.436930+0000","last_became_peered":"2024-03-15T12:56:25.436930+0000","last_unstale":"2024-03-15T12:57:20.104273+0000","last_undegraded":"2024-03-15T12:57:20.104273+0000","last_fullsized":"2024-03-15T12:57:20.104273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:53:56.586786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":270,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:39.530740+0000","last_change":"2024-03-15T12:56:20.214067+0000","last_active":"2024-03-15T12:58:39.530740+0000","last_peered":"2024-03-15T12:58:39.530740+0000","last_clean":"2024-03-15T12:58:39.530740+0000","last_became_active":"2024-03-15T12:56:20.213696+0000","last_became_peered":"2024-03-15T12:56:20.213696+0000","last_unstale":"2024-03-15T12:58:39.530740+0000","last_undegraded":"2024-03-15T12:58:39.530740+0000","last_fullsized":"2024-03-15T12:58:39.530740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:35:15.343660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":237,"num_read_kb":237,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.797642+0000","last_change":"2024-03-15T12:56:21.211106+0000","last_active":"2024-03-15T12:57:48.797642+0000","last_peered":"2024-03-15T12:57:48.797642+0000","last_clean":"2024-03-15T12:57:48.797642+0000","last_became_active":"2024-03-15T12:56:21.210756+0000","last_became_peered":"2024-03-15T12:56:21.210756+0000","last_unstale":"2024-03-15T12:57:48.797642+0000","last_undegraded":"2024-03-15T12:57:48.797642+0000","last_fullsized":"2024-03-15T12:57:48.797642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:13:25.232077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.833697+0000","last_change":"2024-03-15T12:56:25.441364+0000","last_active":"2024-03-15T12:57:19.833697+0000","last_peered":"2024-03-15T12:57:19.833697+0000","last_clean":"2024-03-15T12:57:19.833697+0000","last_became_active":"2024-03-15T12:56:25.440951+0000","last_became_peered":"2024-03-15T12:56:25.440951+0000","last_unstale":"2024-03-15T12:57:19.833697+0000","last_undegraded":"2024-03-15T12:57:19.833697+0000","last_fullsized":"2024-03-15T12:57:19.833697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:32:28.651856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104583+0000","last_change":"2024-03-15T12:56:20.208633+0000","last_active":"2024-03-15T12:57:20.104583+0000","last_peered":"2024-03-15T12:57:20.104583+0000","last_clean":"2024-03-15T12:57:20.104583+0000","last_became_active":"2024-03-15T12:56:20.208280+0000","last_became_peered":"2024-03-15T12:56:20.208280+0000","last_unstale":"2024-03-15T12:57:20.104583+0000","last_undegraded":"2024-03-15T12:57:20.104583+0000","last_fullsized":"2024-03-15T12:57:20.104583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:00:15.274090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.785458+0000","last_change":"2024-03-15T12:56:21.218998+0000","last_active":"2024-03-15T12:57:48.785458+0000","last_peered":"2024-03-15T12:57:48.785458+0000","last_clean":"2024-03-15T12:57:48.785458+0000","last_became_active":"2024-03-15T12:56:21.218617+0000","last_became_peered":"2024-03-15T12:56:21.218617+0000","last_unstale":"2024-03-15T12:57:48.785458+0000","last_undegraded":"2024-03-15T12:57:48.785458+0000","last_fullsized":"2024-03-15T12:57:48.785458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:20:25.496204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837212+0000","last_change":"2024-03-15T12:56:25.438969+0000","last_active":"2024-03-15T12:57:19.837212+0000","last_peered":"2024-03-15T12:57:19.837212+0000","last_clean":"2024-03-15T12:57:19.837212+0000","last_became_active":"2024-03-15T12:56:25.438603+0000","last_became_peered":"2024-03-15T12:56:25.438603+0000","last_unstale":"2024-03-15T12:57:19.837212+0000","last_undegraded":"2024-03-15T12:57:19.837212+0000","last_fullsized":"2024-03-15T12:57:19.837212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:54:23.992253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104585+0000","last_change":"2024-03-15T12:56:20.215152+0000","last_active":"2024-03-15T12:57:20.104585+0000","last_peered":"2024-03-15T12:57:20.104585+0000","last_clean":"2024-03-15T12:57:20.104585+0000","last_became_active":"2024-03-15T12:56:20.214762+0000","last_became_peered":"2024-03-15T12:56:20.214762+0000","last_unstale":"2024-03-15T12:57:20.104585+0000","last_undegraded":"2024-03-15T12:57:20.104585+0000","last_fullsized":"2024-03-15T12:57:20.104585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:38:45.414984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.790199+0000","last_change":"2024-03-15T12:56:21.214005+0000","last_active":"2024-03-15T12:57:48.790199+0000","last_peered":"2024-03-15T12:57:48.790199+0000","last_clean":"2024-03-15T12:57:48.790199+0000","last_became_active":"2024-03-15T12:56:21.213672+0000","last_became_peered":"2024-03-15T12:56:21.213672+0000","last_unstale":"2024-03-15T12:57:48.790199+0000","last_undegraded":"2024-03-15T12:57:48.790199+0000","last_fullsized":"2024-03-15T12:57:48.790199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:13:09.384245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.833759+0000","last_change":"2024-03-15T12:56:25.443145+0000","last_active":"2024-03-15T12:57:19.833759+0000","last_peered":"2024-03-15T12:57:19.833759+0000","last_clean":"2024-03-15T12:57:19.833759+0000","last_became_active":"2024-03-15T12:56:25.442755+0000","last_became_peered":"2024-03-15T12:56:25.442755+0000","last_unstale":"2024-03-15T12:57:19.833759+0000","last_undegraded":"2024-03-15T12:57:19.833759+0000","last_fullsized":"2024-03-15T12:57:19.833759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:14:47.201919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'40","reported_seq":318,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:39.531730+0000","last_change":"2024-03-15T12:56:20.215063+0000","last_active":"2024-03-15T12:58:39.531730+0000","last_peered":"2024-03-15T12:58:39.531730+0000","last_clean":"2024-03-15T12:58:39.531730+0000","last_became_active":"2024-03-15T12:56:20.214742+0000","last_became_peered":"2024-03-15T12:56:20.214742+0000","last_unstale":"2024-03-15T12:58:39.531730+0000","last_undegraded":"2024-03-15T12:58:39.531730+0000","last_fullsized":"2024-03-15T12:58:39.531730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:38:12.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69742,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":194,"num_write":42,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.773641+0000","last_change":"2024-03-15T12:56:21.213519+0000","last_active":"2024-03-15T12:57:48.773641+0000","last_peered":"2024-03-15T12:57:48.773641+0000","last_clean":"2024-03-15T12:57:48.773641+0000","last_became_active":"2024-03-15T12:56:21.213198+0000","last_became_peered":"2024-03-15T12:56:21.213198+0000","last_unstale":"2024-03-15T12:57:48.773641+0000","last_undegraded":"2024-03-15T12:57:48.773641+0000","last_fullsized":"2024-03-15T12:57:48.773641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:22:13.037068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106198+0000","last_change":"2024-03-15T12:57:20.106198+0000","last_active":"2024-03-15T12:57:20.106198+0000","last_peered":"2024-03-15T12:57:20.106198+0000","last_clean":"2024-03-15T12:57:20.106198+0000","last_became_active":"2024-03-15T12:57:20.105649+0000","last_became_peered":"2024-03-15T12:57:20.105649+0000","last_unstale":"2024-03-15T12:57:20.106198+0000","last_undegraded":"2024-03-15T12:57:20.106198+0000","last_fullsized":"2024-03-15T12:57:20.106198+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:44:46.615091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834609+0000","last_change":"2024-03-15T12:56:20.216124+0000","last_active":"2024-03-15T12:57:19.834609+0000","last_peered":"2024-03-15T12:57:19.834609+0000","last_clean":"2024-03-15T12:57:19.834609+0000","last_became_active":"2024-03-15T12:56:20.215754+0000","last_became_peered":"2024-03-15T12:56:20.215754+0000","last_unstale":"2024-03-15T12:57:19.834609+0000","last_undegraded":"2024-03-15T12:57:19.834609+0000","last_fullsized":"2024-03-15T12:57:19.834609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:11:44.403503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.781601+0000","last_change":"2024-03-15T12:56:21.214775+0000","last_active":"2024-03-15T12:57:48.781601+0000","last_peered":"2024-03-15T12:57:48.781601+0000","last_clean":"2024-03-15T12:57:48.781601+0000","last_became_active":"2024-03-15T12:56:21.214445+0000","last_became_peered":"2024-03-15T12:56:21.214445+0000","last_unstale":"2024-03-15T12:57:48.781601+0000","last_undegraded":"2024-03-15T12:57:48.781601+0000","last_fullsized":"2024-03-15T12:57:48.781601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:23:24.045670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.107668+0000","last_change":"2024-03-15T12:57:20.107668+0000","last_active":"2024-03-15T12:57:20.107668+0000","last_peered":"2024-03-15T12:57:20.107668+0000","last_clean":"2024-03-15T12:57:20.107668+0000","last_became_active":"2024-03-15T12:57:20.107259+0000","last_became_peered":"2024-03-15T12:57:20.107259+0000","last_unstale":"2024-03-15T12:57:20.107668+0000","last_undegraded":"2024-03-15T12:57:20.107668+0000","last_fullsized":"2024-03-15T12:57:20.107668+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:07:37.244869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104267+0000","last_change":"2024-03-15T12:56:20.217191+0000","last_active":"2024-03-15T12:57:20.104267+0000","last_peered":"2024-03-15T12:57:20.104267+0000","last_clean":"2024-03-15T12:57:20.104267+0000","last_became_active":"2024-03-15T12:56:20.216632+0000","last_became_peered":"2024-03-15T12:56:20.216632+0000","last_unstale":"2024-03-15T12:57:20.104267+0000","last_undegraded":"2024-03-15T12:57:20.104267+0000","last_fullsized":"2024-03-15T12:57:20.104267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:16:51.187920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784344+0000","last_change":"2024-03-15T12:56:21.218650+0000","last_active":"2024-03-15T12:57:48.784344+0000","last_peered":"2024-03-15T12:57:48.784344+0000","last_clean":"2024-03-15T12:57:48.784344+0000","last_became_active":"2024-03-15T12:56:21.218329+0000","last_became_peered":"2024-03-15T12:56:21.218329+0000","last_unstale":"2024-03-15T12:57:48.784344+0000","last_undegraded":"2024-03-15T12:57:48.784344+0000","last_fullsized":"2024-03-15T12:57:48.784344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:10:45.632248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834415+0000","last_change":"2024-03-15T12:56:25.447982+0000","last_active":"2024-03-15T12:57:19.834415+0000","last_peered":"2024-03-15T12:57:19.834415+0000","last_clean":"2024-03-15T12:57:19.834415+0000","last_became_active":"2024-03-15T12:56:25.447514+0000","last_became_peered":"2024-03-15T12:56:25.447514+0000","last_unstale":"2024-03-15T12:57:19.834415+0000","last_undegraded":"2024-03-15T12:57:19.834415+0000","last_fullsized":"2024-03-15T12:57:19.834415+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:16:52.567619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'8","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:37.576101+0000","last_change":"2024-03-15T12:56:20.209778+0000","last_active":"2024-03-15T12:58:37.576101+0000","last_peered":"2024-03-15T12:58:37.576101+0000","last_clean":"2024-03-15T12:58:37.576101+0000","last_became_active":"2024-03-15T12:56:20.209311+0000","last_became_peered":"2024-03-15T12:56:20.209311+0000","last_unstale":"2024-03-15T12:58:37.576101+0000","last_undegraded":"2024-03-15T12:58:37.576101+0000","last_fullsized":"2024-03-15T12:58:37.576101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:05:21.930210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.778891+0000","last_change":"2024-03-15T12:56:21.215576+0000","last_active":"2024-03-15T12:57:48.778891+0000","last_peered":"2024-03-15T12:57:48.778891+0000","last_clean":"2024-03-15T12:57:48.778891+0000","last_became_active":"2024-03-15T12:56:21.215261+0000","last_became_peered":"2024-03-15T12:56:21.215261+0000","last_unstale":"2024-03-15T12:57:48.778891+0000","last_undegraded":"2024-03-15T12:57:48.778891+0000","last_fullsized":"2024-03-15T12:57:48.778891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:35:15.461554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837297+0000","last_change":"2024-03-15T12:56:25.445559+0000","last_active":"2024-03-15T12:57:19.837297+0000","last_peered":"2024-03-15T12:57:19.837297+0000","last_clean":"2024-03-15T12:57:19.837297+0000","last_became_active":"2024-03-15T12:56:25.445092+0000","last_became_peered":"2024-03-15T12:56:25.445092+0000","last_unstale":"2024-03-15T12:57:19.837297+0000","last_undegraded":"2024-03-15T12:57:19.837297+0000","last_fullsized":"2024-03-15T12:57:19.837297+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:48:30.558243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836745+0000","last_change":"2024-03-15T12:56:20.214853+0000","last_active":"2024-03-15T12:57:19.836745+0000","last_peered":"2024-03-15T12:57:19.836745+0000","last_clean":"2024-03-15T12:57:19.836745+0000","last_became_active":"2024-03-15T12:56:20.214510+0000","last_became_peered":"2024-03-15T12:56:20.214510+0000","last_unstale":"2024-03-15T12:57:19.836745+0000","last_undegraded":"2024-03-15T12:57:19.836745+0000","last_fullsized":"2024-03-15T12:57:19.836745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:25:39.316428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798162+0000","last_change":"2024-03-15T12:56:21.216511+0000","last_active":"2024-03-15T12:57:48.798162+0000","last_peered":"2024-03-15T12:57:48.798162+0000","last_clean":"2024-03-15T12:57:48.798162+0000","last_became_active":"2024-03-15T12:56:21.216175+0000","last_became_peered":"2024-03-15T12:56:21.216175+0000","last_unstale":"2024-03-15T12:57:48.798162+0000","last_undegraded":"2024-03-15T12:57:48.798162+0000","last_fullsized":"2024-03-15T12:57:48.798162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:16:56.665841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104042+0000","last_change":"2024-03-15T12:56:25.446009+0000","last_active":"2024-03-15T12:57:20.104042+0000","last_peered":"2024-03-15T12:57:20.104042+0000","last_clean":"2024-03-15T12:57:20.104042+0000","last_became_active":"2024-03-15T12:56:25.445474+0000","last_became_peered":"2024-03-15T12:56:25.445474+0000","last_unstale":"2024-03-15T12:57:20.104042+0000","last_undegraded":"2024-03-15T12:57:20.104042+0000","last_fullsized":"2024-03-15T12:57:20.104042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:53:46.856905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:36.411188+0000","last_change":"2024-03-15T12:56:20.210146+0000","last_active":"2024-03-15T12:58:36.411188+0000","last_peered":"2024-03-15T12:58:36.411188+0000","last_clean":"2024-03-15T12:58:36.411188+0000","last_became_active":"2024-03-15T12:56:20.209769+0000","last_became_peered":"2024-03-15T12:56:20.209769+0000","last_unstale":"2024-03-15T12:58:36.411188+0000","last_undegraded":"2024-03-15T12:58:36.411188+0000","last_fullsized":"2024-03-15T12:58:36.411188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:47:32.505257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.770795+0000","last_change":"2024-03-15T12:56:21.214826+0000","last_active":"2024-03-15T12:57:48.770795+0000","last_peered":"2024-03-15T12:57:48.770795+0000","last_clean":"2024-03-15T12:57:48.770795+0000","last_became_active":"2024-03-15T12:56:21.214160+0000","last_became_peered":"2024-03-15T12:56:21.214160+0000","last_unstale":"2024-03-15T12:57:48.770795+0000","last_undegraded":"2024-03-15T12:57:48.770795+0000","last_fullsized":"2024-03-15T12:57:48.770795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:23:17.800522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837088+0000","last_change":"2024-03-15T12:56:25.443496+0000","last_active":"2024-03-15T12:57:19.837088+0000","last_peered":"2024-03-15T12:57:19.837088+0000","last_clean":"2024-03-15T12:57:19.837088+0000","last_became_active":"2024-03-15T12:56:25.442796+0000","last_became_peered":"2024-03-15T12:56:25.442796+0000","last_unstale":"2024-03-15T12:57:19.837088+0000","last_undegraded":"2024-03-15T12:57:19.837088+0000","last_fullsized":"2024-03-15T12:57:19.837088+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:35:23.892693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.841026+0000","last_change":"2024-03-15T12:57:19.840452+0000","last_active":"2024-03-15T12:57:19.841026+0000","last_peered":"2024-03-15T12:57:19.841026+0000","last_clean":"2024-03-15T12:57:19.841026+0000","last_became_active":"2024-03-15T12:57:19.839977+0000","last_became_peered":"2024-03-15T12:57:19.839977+0000","last_unstale":"2024-03-15T12:57:19.841026+0000","last_undegraded":"2024-03-15T12:57:19.841026+0000","last_fullsized":"2024-03-15T12:57:19.841026+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:59:25.026455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.777893+0000","last_change":"2024-03-15T12:56:21.216393+0000","last_active":"2024-03-15T12:57:48.777893+0000","last_peered":"2024-03-15T12:57:48.777893+0000","last_clean":"2024-03-15T12:57:48.777893+0000","last_became_active":"2024-03-15T12:56:21.216043+0000","last_became_peered":"2024-03-15T12:56:21.216043+0000","last_unstale":"2024-03-15T12:57:48.777893+0000","last_undegraded":"2024-03-15T12:57:48.777893+0000","last_fullsized":"2024-03-15T12:57:48.777893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:05:49.843726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837389+0000","last_change":"2024-03-15T12:56:25.443854+0000","last_active":"2024-03-15T12:57:19.837389+0000","last_peered":"2024-03-15T12:57:19.837389+0000","last_clean":"2024-03-15T12:57:19.837389+0000","last_became_active":"2024-03-15T12:56:25.443422+0000","last_became_peered":"2024-03-15T12:56:25.443422+0000","last_unstale":"2024-03-15T12:57:19.837389+0000","last_undegraded":"2024-03-15T12:57:19.837389+0000","last_fullsized":"2024-03-15T12:57:19.837389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:52:18.999161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:36.575217+0000","last_change":"2024-03-15T12:56:20.211493+0000","last_active":"2024-03-15T12:58:36.575217+0000","last_peered":"2024-03-15T12:58:36.575217+0000","last_clean":"2024-03-15T12:58:36.575217+0000","last_became_active":"2024-03-15T12:56:20.211154+0000","last_became_peered":"2024-03-15T12:56:20.211154+0000","last_unstale":"2024-03-15T12:58:36.575217+0000","last_undegraded":"2024-03-15T12:58:36.575217+0000","last_fullsized":"2024-03-15T12:58:36.575217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:26:28.026899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784429+0000","last_change":"2024-03-15T12:56:21.215999+0000","last_active":"2024-03-15T12:57:48.784429+0000","last_peered":"2024-03-15T12:57:48.784429+0000","last_clean":"2024-03-15T12:57:48.784429+0000","last_became_active":"2024-03-15T12:56:21.215657+0000","last_became_peered":"2024-03-15T12:56:21.215657+0000","last_unstale":"2024-03-15T12:57:48.784429+0000","last_undegraded":"2024-03-15T12:57:48.784429+0000","last_fullsized":"2024-03-15T12:57:48.784429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:47:59.901886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837294+0000","last_change":"2024-03-15T12:56:25.442246+0000","last_active":"2024-03-15T12:57:19.837294+0000","last_peered":"2024-03-15T12:57:19.837294+0000","last_clean":"2024-03-15T12:57:19.837294+0000","last_became_active":"2024-03-15T12:56:25.441755+0000","last_became_peered":"2024-03-15T12:56:25.441755+0000","last_unstale":"2024-03-15T12:57:19.837294+0000","last_undegraded":"2024-03-15T12:57:19.837294+0000","last_fullsized":"2024-03-15T12:57:19.837294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:47:36.037864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.753452+0000","last_change":"2024-03-15T12:56:21.215746+0000","last_active":"2024-03-15T12:57:48.753452+0000","last_peered":"2024-03-15T12:57:48.753452+0000","last_clean":"2024-03-15T12:57:48.753452+0000","last_became_active":"2024-03-15T12:56:21.215424+0000","last_became_peered":"2024-03-15T12:56:21.215424+0000","last_unstale":"2024-03-15T12:57:48.753452+0000","last_undegraded":"2024-03-15T12:57:48.753452+0000","last_fullsized":"2024-03-15T12:57:48.753452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:01:15.498119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'7","reported_seq":153,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:38.576866+0000","last_change":"2024-03-15T12:56:20.219705+0000","last_active":"2024-03-15T12:58:38.576866+0000","last_peered":"2024-03-15T12:58:38.576866+0000","last_clean":"2024-03-15T12:58:38.576866+0000","last_became_active":"2024-03-15T12:56:20.219328+0000","last_became_peered":"2024-03-15T12:56:20.219328+0000","last_unstale":"2024-03-15T12:58:38.576866+0000","last_undegraded":"2024-03-15T12:58:38.576866+0000","last_fullsized":"2024-03-15T12:58:38.576866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:47:52.171116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104879+0000","last_change":"2024-03-15T12:56:25.443094+0000","last_active":"2024-03-15T12:57:20.104879+0000","last_peered":"2024-03-15T12:57:20.104879+0000","last_clean":"2024-03-15T12:57:20.104879+0000","last_became_active":"2024-03-15T12:56:25.442708+0000","last_became_peered":"2024-03-15T12:56:25.442708+0000","last_unstale":"2024-03-15T12:57:20.104879+0000","last_undegraded":"2024-03-15T12:57:20.104879+0000","last_fullsized":"2024-03-15T12:57:20.104879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:03:07.769836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.788423+0000","last_change":"2024-03-15T12:56:21.223937+0000","last_active":"2024-03-15T12:57:48.788423+0000","last_peered":"2024-03-15T12:57:48.788423+0000","last_clean":"2024-03-15T12:57:48.788423+0000","last_became_active":"2024-03-15T12:56:21.223583+0000","last_became_peered":"2024-03-15T12:56:21.223583+0000","last_unstale":"2024-03-15T12:57:48.788423+0000","last_undegraded":"2024-03-15T12:57:48.788423+0000","last_fullsized":"2024-03-15T12:57:48.788423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:15:34.287445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837150+0000","last_change":"2024-03-15T12:56:20.210812+0000","last_active":"2024-03-15T12:57:19.837150+0000","last_peered":"2024-03-15T12:57:19.837150+0000","last_clean":"2024-03-15T12:57:19.837150+0000","last_became_active":"2024-03-15T12:56:20.210457+0000","last_became_peered":"2024-03-15T12:56:20.210457+0000","last_unstale":"2024-03-15T12:57:19.837150+0000","last_undegraded":"2024-03-15T12:57:19.837150+0000","last_fullsized":"2024-03-15T12:57:19.837150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:04:37.371637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":358,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104758+0000","last_change":"2024-03-15T12:55:08.392548+0000","last_active":"2024-03-15T12:57:20.104758+0000","last_peered":"2024-03-15T12:57:20.104758+0000","last_clean":"2024-03-15T12:57:20.104758+0000","last_became_active":"2024-03-15T12:55:08.391972+0000","last_became_peered":"2024-03-15T12:55:08.391972+0000","last_unstale":"2024-03-15T12:57:20.104758+0000","last_undegraded":"2024-03-15T12:57:20.104758+0000","last_fullsized":"2024-03-15T12:57:20.104758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:51:47.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:51:47.093558+0000","last_clean_scrub_stamp":"2024-03-15T12:51:47.093558+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:30:57.264886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104929+0000","last_change":"2024-03-15T12:56:25.443212+0000","last_active":"2024-03-15T12:57:20.104929+0000","last_peered":"2024-03-15T12:57:20.104929+0000","last_clean":"2024-03-15T12:57:20.104929+0000","last_became_active":"2024-03-15T12:56:25.442787+0000","last_became_peered":"2024-03-15T12:56:25.442787+0000","last_unstale":"2024-03-15T12:57:20.104929+0000","last_undegraded":"2024-03-15T12:57:20.104929+0000","last_fullsized":"2024-03-15T12:57:20.104929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:59:32.244953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.775951+0000","last_change":"2024-03-15T12:57:19.839935+0000","last_active":"2024-03-15T12:57:48.775951+0000","last_peered":"2024-03-15T12:57:48.775951+0000","last_clean":"2024-03-15T12:57:48.775951+0000","last_became_active":"2024-03-15T12:57:19.839546+0000","last_became_peered":"2024-03-15T12:57:19.839546+0000","last_unstale":"2024-03-15T12:57:48.775951+0000","last_undegraded":"2024-03-15T12:57:48.775951+0000","last_fullsized":"2024-03-15T12:57:48.775951+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:05:15.544159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104546+0000","last_change":"2024-03-15T12:56:20.210466+0000","last_active":"2024-03-15T12:57:20.104546+0000","last_peered":"2024-03-15T12:57:20.104546+0000","last_clean":"2024-03-15T12:57:20.104546+0000","last_became_active":"2024-03-15T12:56:20.210157+0000","last_became_peered":"2024-03-15T12:56:20.210157+0000","last_unstale":"2024-03-15T12:57:20.104546+0000","last_undegraded":"2024-03-15T12:57:20.104546+0000","last_fullsized":"2024-03-15T12:57:20.104546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:58:03.335826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.838324+0000","last_change":"2024-03-15T12:56:25.432641+0000","last_active":"2024-03-15T12:57:19.838324+0000","last_peered":"2024-03-15T12:57:19.838324+0000","last_clean":"2024-03-15T12:57:19.838324+0000","last_became_active":"2024-03-15T12:56:25.432212+0000","last_became_peered":"2024-03-15T12:56:25.432212+0000","last_unstale":"2024-03-15T12:57:19.838324+0000","last_undegraded":"2024-03-15T12:57:19.838324+0000","last_fullsized":"2024-03-15T12:57:19.838324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:40:20.447886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.846534+0000","last_change":"2024-03-15T12:57:19.846534+0000","last_active":"2024-03-15T12:57:19.846534+0000","last_peered":"2024-03-15T12:57:19.846534+0000","last_clean":"2024-03-15T12:57:19.846534+0000","last_became_active":"2024-03-15T12:57:19.841008+0000","last_became_peered":"2024-03-15T12:57:19.841008+0000","last_unstale":"2024-03-15T12:57:19.846534+0000","last_undegraded":"2024-03-15T12:57:19.846534+0000","last_fullsized":"2024-03-15T12:57:19.846534+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:46:17.129901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.778482+0000","last_change":"2024-03-15T12:56:21.207577+0000","last_active":"2024-03-15T12:57:48.778482+0000","last_peered":"2024-03-15T12:57:48.778482+0000","last_clean":"2024-03-15T12:57:48.778482+0000","last_became_active":"2024-03-15T12:56:21.207220+0000","last_became_peered":"2024-03-15T12:56:21.207220+0000","last_unstale":"2024-03-15T12:57:48.778482+0000","last_undegraded":"2024-03-15T12:57:48.778482+0000","last_fullsized":"2024-03-15T12:57:48.778482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:42:12.104220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104420+0000","last_change":"2024-03-15T12:56:25.439762+0000","last_active":"2024-03-15T12:57:20.104420+0000","last_peered":"2024-03-15T12:57:20.104420+0000","last_clean":"2024-03-15T12:57:20.104420+0000","last_became_active":"2024-03-15T12:56:25.439366+0000","last_became_peered":"2024-03-15T12:56:25.439366+0000","last_unstale":"2024-03-15T12:57:20.104420+0000","last_undegraded":"2024-03-15T12:57:20.104420+0000","last_fullsized":"2024-03-15T12:57:20.104420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:36:29.638449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'16","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:38.417721+0000","last_change":"2024-03-15T12:56:20.218438+0000","last_active":"2024-03-15T12:58:38.417721+0000","last_peered":"2024-03-15T12:58:38.417721+0000","last_clean":"2024-03-15T12:58:38.417721+0000","last_became_active":"2024-03-15T12:56:20.218071+0000","last_became_peered":"2024-03-15T12:56:20.218071+0000","last_unstale":"2024-03-15T12:58:38.417721+0000","last_undegraded":"2024-03-15T12:58:38.417721+0000","last_fullsized":"2024-03-15T12:58:38.417721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:24:16.295781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.783607+0000","last_change":"2024-03-15T12:56:21.218345+0000","last_active":"2024-03-15T12:57:48.783607+0000","last_peered":"2024-03-15T12:57:48.783607+0000","last_clean":"2024-03-15T12:57:48.783607+0000","last_became_active":"2024-03-15T12:56:21.216534+0000","last_became_peered":"2024-03-15T12:56:21.216534+0000","last_unstale":"2024-03-15T12:57:48.783607+0000","last_undegraded":"2024-03-15T12:57:48.783607+0000","last_fullsized":"2024-03-15T12:57:48.783607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:56:18.977509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834310+0000","last_change":"2024-03-15T12:56:25.435714+0000","last_active":"2024-03-15T12:57:19.834310+0000","last_peered":"2024-03-15T12:57:19.834310+0000","last_clean":"2024-03-15T12:57:19.834310+0000","last_became_active":"2024-03-15T12:56:25.435315+0000","last_became_peered":"2024-03-15T12:56:25.435315+0000","last_unstale":"2024-03-15T12:57:19.834310+0000","last_undegraded":"2024-03-15T12:57:19.834310+0000","last_fullsized":"2024-03-15T12:57:19.834310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:10:03.509078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837248+0000","last_change":"2024-03-15T12:56:20.209356+0000","last_active":"2024-03-15T12:57:19.837248+0000","last_peered":"2024-03-15T12:57:19.837248+0000","last_clean":"2024-03-15T12:57:19.837248+0000","last_became_active":"2024-03-15T12:56:20.208995+0000","last_became_peered":"2024-03-15T12:56:20.208995+0000","last_unstale":"2024-03-15T12:57:19.837248+0000","last_undegraded":"2024-03-15T12:57:19.837248+0000","last_fullsized":"2024-03-15T12:57:19.837248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:14:08.554567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.783714+0000","last_change":"2024-03-15T12:56:21.216315+0000","last_active":"2024-03-15T12:57:48.783714+0000","last_peered":"2024-03-15T12:57:48.783714+0000","last_clean":"2024-03-15T12:57:48.783714+0000","last_became_active":"2024-03-15T12:56:21.215943+0000","last_became_peered":"2024-03-15T12:56:21.215943+0000","last_unstale":"2024-03-15T12:57:48.783714+0000","last_undegraded":"2024-03-15T12:57:48.783714+0000","last_fullsized":"2024-03-15T12:57:48.783714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:47:29.913048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836875+0000","last_change":"2024-03-15T12:56:25.437200+0000","last_active":"2024-03-15T12:57:19.836875+0000","last_peered":"2024-03-15T12:57:19.836875+0000","last_clean":"2024-03-15T12:57:19.836875+0000","last_became_active":"2024-03-15T12:56:25.436786+0000","last_became_peered":"2024-03-15T12:56:25.436786+0000","last_unstale":"2024-03-15T12:57:19.836875+0000","last_undegraded":"2024-03-15T12:57:19.836875+0000","last_fullsized":"2024-03-15T12:57:19.836875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:12:00.286749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106367+0000","last_change":"2024-03-15T12:56:20.202240+0000","last_active":"2024-03-15T12:57:20.106367+0000","last_peered":"2024-03-15T12:57:20.106367+0000","last_clean":"2024-03-15T12:57:20.106367+0000","last_became_active":"2024-03-15T12:56:20.201136+0000","last_became_peered":"2024-03-15T12:56:20.201136+0000","last_unstale":"2024-03-15T12:57:20.106367+0000","last_undegraded":"2024-03-15T12:57:20.106367+0000","last_fullsized":"2024-03-15T12:57:20.106367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:30:35.901237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.759652+0000","last_change":"2024-03-15T12:56:21.212095+0000","last_active":"2024-03-15T12:57:48.759652+0000","last_peered":"2024-03-15T12:57:48.759652+0000","last_clean":"2024-03-15T12:57:48.759652+0000","last_became_active":"2024-03-15T12:56:21.211744+0000","last_became_peered":"2024-03-15T12:56:21.211744+0000","last_unstale":"2024-03-15T12:57:48.759652+0000","last_undegraded":"2024-03-15T12:57:48.759652+0000","last_fullsized":"2024-03-15T12:57:48.759652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:27:58.412513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106031+0000","last_change":"2024-03-15T12:57:20.106031+0000","last_active":"2024-03-15T12:57:20.106031+0000","last_peered":"2024-03-15T12:57:20.106031+0000","last_clean":"2024-03-15T12:57:20.106031+0000","last_became_active":"2024-03-15T12:57:20.105499+0000","last_became_peered":"2024-03-15T12:57:20.105499+0000","last_unstale":"2024-03-15T12:57:20.106031+0000","last_undegraded":"2024-03-15T12:57:20.106031+0000","last_fullsized":"2024-03-15T12:57:20.106031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:17:08.138164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104141+0000","last_change":"2024-03-15T12:56:20.217319+0000","last_active":"2024-03-15T12:57:20.104141+0000","last_peered":"2024-03-15T12:57:20.104141+0000","last_clean":"2024-03-15T12:57:20.104141+0000","last_became_active":"2024-03-15T12:56:20.215450+0000","last_became_peered":"2024-03-15T12:56:20.215450+0000","last_unstale":"2024-03-15T12:57:20.104141+0000","last_undegraded":"2024-03-15T12:57:20.104141+0000","last_fullsized":"2024-03-15T12:57:20.104141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:07:09.085860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780239+0000","last_change":"2024-03-15T12:56:21.209360+0000","last_active":"2024-03-15T12:57:48.780239+0000","last_peered":"2024-03-15T12:57:48.780239+0000","last_clean":"2024-03-15T12:57:48.780239+0000","last_became_active":"2024-03-15T12:56:21.208987+0000","last_became_peered":"2024-03-15T12:56:21.208987+0000","last_unstale":"2024-03-15T12:57:48.780239+0000","last_undegraded":"2024-03-15T12:57:48.780239+0000","last_fullsized":"2024-03-15T12:57:48.780239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:19:13.353400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836774+0000","last_change":"2024-03-15T12:56:25.439173+0000","last_active":"2024-03-15T12:57:19.836774+0000","last_peered":"2024-03-15T12:57:19.836774+0000","last_clean":"2024-03-15T12:57:19.836774+0000","last_became_active":"2024-03-15T12:56:25.438812+0000","last_became_peered":"2024-03-15T12:56:25.438812+0000","last_unstale":"2024-03-15T12:57:19.836774+0000","last_undegraded":"2024-03-15T12:57:19.836774+0000","last_fullsized":"2024-03-15T12:57:19.836774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:10:39.107190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:35.836940+0000","last_change":"2024-03-15T12:56:20.211861+0000","last_active":"2024-03-15T12:58:35.836940+0000","last_peered":"2024-03-15T12:58:35.836940+0000","last_clean":"2024-03-15T12:58:35.836940+0000","last_became_active":"2024-03-15T12:56:20.211481+0000","last_became_peered":"2024-03-15T12:56:20.211481+0000","last_unstale":"2024-03-15T12:58:35.836940+0000","last_undegraded":"2024-03-15T12:58:35.836940+0000","last_fullsized":"2024-03-15T12:58:35.836940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:12:45.558423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.783692+0000","last_change":"2024-03-15T12:56:21.216058+0000","last_active":"2024-03-15T12:57:48.783692+0000","last_peered":"2024-03-15T12:57:48.783692+0000","last_clean":"2024-03-15T12:57:48.783692+0000","last_became_active":"2024-03-15T12:56:21.215706+0000","last_became_peered":"2024-03-15T12:56:21.215706+0000","last_unstale":"2024-03-15T12:57:48.783692+0000","last_undegraded":"2024-03-15T12:57:48.783692+0000","last_fullsized":"2024-03-15T12:57:48.783692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:55:02.132976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836966+0000","last_change":"2024-03-15T12:56:25.444134+0000","last_active":"2024-03-15T12:57:19.836966+0000","last_peered":"2024-03-15T12:57:19.836966+0000","last_clean":"2024-03-15T12:57:19.836966+0000","last_became_active":"2024-03-15T12:56:25.443452+0000","last_became_peered":"2024-03-15T12:56:25.443452+0000","last_unstale":"2024-03-15T12:57:19.836966+0000","last_undegraded":"2024-03-15T12:57:19.836966+0000","last_fullsized":"2024-03-15T12:57:19.836966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:06:39.660016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836785+0000","last_change":"2024-03-15T12:56:20.214700+0000","last_active":"2024-03-15T12:57:19.836785+0000","last_peered":"2024-03-15T12:57:19.836785+0000","last_clean":"2024-03-15T12:57:19.836785+0000","last_became_active":"2024-03-15T12:56:20.214348+0000","last_became_peered":"2024-03-15T12:56:20.214348+0000","last_unstale":"2024-03-15T12:57:19.836785+0000","last_undegraded":"2024-03-15T12:57:19.836785+0000","last_fullsized":"2024-03-15T12:57:19.836785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:24:37.551182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798936+0000","last_change":"2024-03-15T12:56:21.220746+0000","last_active":"2024-03-15T12:57:48.798936+0000","last_peered":"2024-03-15T12:57:48.798936+0000","last_clean":"2024-03-15T12:57:48.798936+0000","last_became_active":"2024-03-15T12:56:21.220445+0000","last_became_peered":"2024-03-15T12:56:21.220445+0000","last_unstale":"2024-03-15T12:57:48.798936+0000","last_undegraded":"2024-03-15T12:57:48.798936+0000","last_fullsized":"2024-03-15T12:57:48.798936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:58:44.119476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104747+0000","last_change":"2024-03-15T12:56:25.437611+0000","last_active":"2024-03-15T12:57:20.104747+0000","last_peered":"2024-03-15T12:57:20.104747+0000","last_clean":"2024-03-15T12:57:20.104747+0000","last_became_active":"2024-03-15T12:56:25.437195+0000","last_became_peered":"2024-03-15T12:56:25.437195+0000","last_unstale":"2024-03-15T12:57:20.104747+0000","last_undegraded":"2024-03-15T12:57:20.104747+0000","last_fullsized":"2024-03-15T12:57:20.104747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:26:43.558477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":156,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:39.603492+0000","last_change":"2024-03-15T12:56:20.208584+0000","last_active":"2024-03-15T12:58:39.603492+0000","last_peered":"2024-03-15T12:58:39.603492+0000","last_clean":"2024-03-15T12:58:39.603492+0000","last_became_active":"2024-03-15T12:56:20.208155+0000","last_became_peered":"2024-03-15T12:56:20.208155+0000","last_unstale":"2024-03-15T12:58:39.603492+0000","last_undegraded":"2024-03-15T12:58:39.603492+0000","last_fullsized":"2024-03-15T12:58:39.603492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:26:13.148273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2054,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798776+0000","last_change":"2024-03-15T12:56:21.220421+0000","last_active":"2024-03-15T12:57:48.798776+0000","last_peered":"2024-03-15T12:57:48.798776+0000","last_clean":"2024-03-15T12:57:48.798776+0000","last_became_active":"2024-03-15T12:56:21.220095+0000","last_became_peered":"2024-03-15T12:56:21.220095+0000","last_unstale":"2024-03-15T12:57:48.798776+0000","last_undegraded":"2024-03-15T12:57:48.798776+0000","last_fullsized":"2024-03-15T12:57:48.798776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:35:33.846506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836547+0000","last_change":"2024-03-15T12:56:25.440408+0000","last_active":"2024-03-15T12:57:19.836547+0000","last_peered":"2024-03-15T12:57:19.836547+0000","last_clean":"2024-03-15T12:57:19.836547+0000","last_became_active":"2024-03-15T12:56:25.439965+0000","last_became_peered":"2024-03-15T12:56:25.439965+0000","last_unstale":"2024-03-15T12:57:19.836547+0000","last_undegraded":"2024-03-15T12:57:19.836547+0000","last_fullsized":"2024-03-15T12:57:19.836547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:12:57.278202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.105052+0000","last_change":"2024-03-15T12:56:20.216320+0000","last_active":"2024-03-15T12:57:20.105052+0000","last_peered":"2024-03-15T12:57:20.105052+0000","last_clean":"2024-03-15T12:57:20.105052+0000","last_became_active":"2024-03-15T12:56:20.215915+0000","last_became_peered":"2024-03-15T12:56:20.215915+0000","last_unstale":"2024-03-15T12:57:20.105052+0000","last_undegraded":"2024-03-15T12:57:20.105052+0000","last_fullsized":"2024-03-15T12:57:20.105052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:25:45.236269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782769+0000","last_change":"2024-03-15T12:56:21.215822+0000","last_active":"2024-03-15T12:57:48.782769+0000","last_peered":"2024-03-15T12:57:48.782769+0000","last_clean":"2024-03-15T12:57:48.782769+0000","last_became_active":"2024-03-15T12:56:21.215508+0000","last_became_peered":"2024-03-15T12:56:21.215508+0000","last_unstale":"2024-03-15T12:57:48.782769+0000","last_undegraded":"2024-03-15T12:57:48.782769+0000","last_fullsized":"2024-03-15T12:57:48.782769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:56:55.919198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103859+0000","last_change":"2024-03-15T12:56:25.436186+0000","last_active":"2024-03-15T12:57:20.103859+0000","last_peered":"2024-03-15T12:57:20.103859+0000","last_clean":"2024-03-15T12:57:20.103859+0000","last_became_active":"2024-03-15T12:56:25.435586+0000","last_became_peered":"2024-03-15T12:56:25.435586+0000","last_unstale":"2024-03-15T12:57:20.103859+0000","last_undegraded":"2024-03-15T12:57:20.103859+0000","last_fullsized":"2024-03-15T12:57:20.103859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:02:26.850487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837495+0000","last_change":"2024-03-15T12:56:20.217277+0000","last_active":"2024-03-15T12:57:19.837495+0000","last_peered":"2024-03-15T12:57:19.837495+0000","last_clean":"2024-03-15T12:57:19.837495+0000","last_became_active":"2024-03-15T12:56:20.216930+0000","last_became_peered":"2024-03-15T12:56:20.216930+0000","last_unstale":"2024-03-15T12:57:19.837495+0000","last_undegraded":"2024-03-15T12:57:19.837495+0000","last_fullsized":"2024-03-15T12:57:19.837495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:47:51.075545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780182+0000","last_change":"2024-03-15T12:56:21.212066+0000","last_active":"2024-03-15T12:57:48.780182+0000","last_peered":"2024-03-15T12:57:48.780182+0000","last_clean":"2024-03-15T12:57:48.780182+0000","last_became_active":"2024-03-15T12:56:21.211650+0000","last_became_peered":"2024-03-15T12:56:21.211650+0000","last_unstale":"2024-03-15T12:57:48.780182+0000","last_undegraded":"2024-03-15T12:57:48.780182+0000","last_fullsized":"2024-03-15T12:57:48.780182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:40:35.322210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.772659+0000","last_change":"2024-03-15T12:56:21.212794+0000","last_active":"2024-03-15T12:57:48.772659+0000","last_peered":"2024-03-15T12:57:48.772659+0000","last_clean":"2024-03-15T12:57:48.772659+0000","last_became_active":"2024-03-15T12:56:21.212470+0000","last_became_peered":"2024-03-15T12:56:21.212470+0000","last_unstale":"2024-03-15T12:57:48.772659+0000","last_undegraded":"2024-03-15T12:57:48.772659+0000","last_fullsized":"2024-03-15T12:57:48.772659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:10:04.614223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.756171+0000","last_change":"2024-03-15T12:56:21.211746+0000","last_active":"2024-03-15T12:57:48.756171+0000","last_peered":"2024-03-15T12:57:48.756171+0000","last_clean":"2024-03-15T12:57:48.756171+0000","last_became_active":"2024-03-15T12:56:21.211431+0000","last_became_peered":"2024-03-15T12:56:21.211431+0000","last_unstale":"2024-03-15T12:57:48.756171+0000","last_undegraded":"2024-03-15T12:57:48.756171+0000","last_fullsized":"2024-03-15T12:57:48.756171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:46:31.188408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.778038+0000","last_change":"2024-03-15T12:56:21.220997+0000","last_active":"2024-03-15T12:57:48.778038+0000","last_peered":"2024-03-15T12:57:48.778038+0000","last_clean":"2024-03-15T12:57:48.778038+0000","last_became_active":"2024-03-15T12:56:21.220644+0000","last_became_peered":"2024-03-15T12:56:21.220644+0000","last_unstale":"2024-03-15T12:57:48.778038+0000","last_undegraded":"2024-03-15T12:57:48.778038+0000","last_fullsized":"2024-03-15T12:57:48.778038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:47:55.852845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780052+0000","last_change":"2024-03-15T12:56:21.211634+0000","last_active":"2024-03-15T12:57:48.780052+0000","last_peered":"2024-03-15T12:57:48.780052+0000","last_clean":"2024-03-15T12:57:48.780052+0000","last_became_active":"2024-03-15T12:56:21.211303+0000","last_became_peered":"2024-03-15T12:56:21.211303+0000","last_unstale":"2024-03-15T12:57:48.780052+0000","last_undegraded":"2024-03-15T12:57:48.780052+0000","last_fullsized":"2024-03-15T12:57:48.780052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:22:32.358270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780806+0000","last_change":"2024-03-15T12:56:21.211673+0000","last_active":"2024-03-15T12:57:48.780806+0000","last_peered":"2024-03-15T12:57:48.780806+0000","last_clean":"2024-03-15T12:57:48.780806+0000","last_became_active":"2024-03-15T12:56:21.211296+0000","last_became_peered":"2024-03-15T12:56:21.211296+0000","last_unstale":"2024-03-15T12:57:48.780806+0000","last_undegraded":"2024-03-15T12:57:48.780806+0000","last_fullsized":"2024-03-15T12:57:48.780806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:40:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.769894+0000","last_change":"2024-03-15T12:56:21.208681+0000","last_active":"2024-03-15T12:57:48.769894+0000","last_peered":"2024-03-15T12:57:48.769894+0000","last_clean":"2024-03-15T12:57:48.769894+0000","last_became_active":"2024-03-15T12:56:21.208130+0000","last_became_peered":"2024-03-15T12:56:21.208130+0000","last_unstale":"2024-03-15T12:57:48.769894+0000","last_undegraded":"2024-03-15T12:57:48.769894+0000","last_fullsized":"2024-03-15T12:57:48.769894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:34:52.971485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.793618+0000","last_change":"2024-03-15T12:56:21.216979+0000","last_active":"2024-03-15T12:57:48.793618+0000","last_peered":"2024-03-15T12:57:48.793618+0000","last_clean":"2024-03-15T12:57:48.793618+0000","last_became_active":"2024-03-15T12:56:21.216632+0000","last_became_peered":"2024-03-15T12:56:21.216632+0000","last_unstale":"2024-03-15T12:57:48.793618+0000","last_undegraded":"2024-03-15T12:57:48.793618+0000","last_fullsized":"2024-03-15T12:57:48.793618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:21:57.213512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.785754+0000","last_change":"2024-03-15T12:56:21.223064+0000","last_active":"2024-03-15T12:57:48.785754+0000","last_peered":"2024-03-15T12:57:48.785754+0000","last_clean":"2024-03-15T12:57:48.785754+0000","last_became_active":"2024-03-15T12:56:21.220882+0000","last_became_peered":"2024-03-15T12:56:21.220882+0000","last_unstale":"2024-03-15T12:57:48.785754+0000","last_undegraded":"2024-03-15T12:57:48.785754+0000","last_fullsized":"2024-03-15T12:57:48.785754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:45:02.930154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782047+0000","last_change":"2024-03-15T12:56:21.218614+0000","last_active":"2024-03-15T12:57:48.782047+0000","last_peered":"2024-03-15T12:57:48.782047+0000","last_clean":"2024-03-15T12:57:48.782047+0000","last_became_active":"2024-03-15T12:56:21.218224+0000","last_became_peered":"2024-03-15T12:56:21.218224+0000","last_unstale":"2024-03-15T12:57:48.782047+0000","last_undegraded":"2024-03-15T12:57:48.782047+0000","last_fullsized":"2024-03-15T12:57:48.782047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:41:29.790358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782852+0000","last_change":"2024-03-15T12:56:21.219019+0000","last_active":"2024-03-15T12:57:48.782852+0000","last_peered":"2024-03-15T12:57:48.782852+0000","last_clean":"2024-03-15T12:57:48.782852+0000","last_became_active":"2024-03-15T12:56:21.218673+0000","last_became_peered":"2024-03-15T12:56:21.218673+0000","last_unstale":"2024-03-15T12:57:48.782852+0000","last_undegraded":"2024-03-15T12:57:48.782852+0000","last_fullsized":"2024-03-15T12:57:48.782852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:42:11.788598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.799975+0000","last_change":"2024-03-15T12:56:21.220355+0000","last_active":"2024-03-15T12:57:48.799975+0000","last_peered":"2024-03-15T12:57:48.799975+0000","last_clean":"2024-03-15T12:57:48.799975+0000","last_became_active":"2024-03-15T12:56:21.220021+0000","last_became_peered":"2024-03-15T12:56:21.220021+0000","last_unstale":"2024-03-15T12:57:48.799975+0000","last_undegraded":"2024-03-15T12:57:48.799975+0000","last_fullsized":"2024-03-15T12:57:48.799975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:40:00.947351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.781265+0000","last_change":"2024-03-15T12:56:21.213050+0000","last_active":"2024-03-15T12:57:48.781265+0000","last_peered":"2024-03-15T12:57:48.781265+0000","last_clean":"2024-03-15T12:57:48.781265+0000","last_became_active":"2024-03-15T12:56:21.212730+0000","last_became_peered":"2024-03-15T12:56:21.212730+0000","last_unstale":"2024-03-15T12:57:48.781265+0000","last_undegraded":"2024-03-15T12:57:48.781265+0000","last_fullsized":"2024-03-15T12:57:48.781265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:56:36.396285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.770729+0000","last_change":"2024-03-15T12:56:21.213942+0000","last_active":"2024-03-15T12:57:48.770729+0000","last_peered":"2024-03-15T12:57:48.770729+0000","last_clean":"2024-03-15T12:57:48.770729+0000","last_became_active":"2024-03-15T12:56:21.213611+0000","last_became_peered":"2024-03-15T12:56:21.213611+0000","last_unstale":"2024-03-15T12:57:48.770729+0000","last_undegraded":"2024-03-15T12:57:48.770729+0000","last_fullsized":"2024-03-15T12:57:48.770729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:23:44.002970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79059,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1224,"num_read_kb":701,"num_write":271,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":237959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":501940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1728512,"data_stored":1151302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":504970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":501140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36948,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708204,"statfs":{"total":95995035648,"available":95957200896,"internally_reserved":0,"allocated":1650688,"data_stored":1099369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495683,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":496871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":568034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":498327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":568747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":496086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":501477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1093433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":72894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-15T12:58:42.885 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-15T12:58:43.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 -- 172.21.15.2:0/646549831 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e60101680 msgr2=0x7f3e60101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:43.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 --2- 172.21.15.2:0/646549831 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e60101680 0x7f3e60101b00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009900 tx=0x7f3e5002bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:43.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 -- 172.21.15.2:0/646549831 shutdown_connections 2024-03-15T12:58:43.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 --2- 172.21.15.2:0/646549831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e60102040 0x7f3e601088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 --2- 172.21.15.2:0/646549831 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e60101680 0x7f3e60101b00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 --2- 172.21.15.2:0/646549831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e60100490 0x7f3e601008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 -- 172.21.15.2:0/646549831 >> 172.21.15.2:0/646549831 conn(0x7f3e600fba20 msgr2=0x7f3e600fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:43.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 -- 172.21.15.2:0/646549831 shutdown_connections 2024-03-15T12:58:43.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 -- 172.21.15.2:0/646549831 wait complete. 2024-03-15T12:58:43.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 Processor -- start 2024-03-15T12:58:43.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 -- start start 2024-03-15T12:58:43.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e60100490 0x7f3e60197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:43.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e60101680 0x7f3e60197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:43.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e60102040 0x7f3e6019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:43.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6019f140 con 0x7f3e60101680 2024-03-15T12:58:43.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6019f2b0 con 0x7f3e60100490 2024-03-15T12:58:43.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.036+0000 7f3e68556640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6019f470 con 0x7f3e60102040 2024-03-15T12:58:43.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e65af1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e60101680 0x7f3e60197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:43.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e65af1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e60101680 0x7f3e60197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43706/0 (socket says 172.21.15.2:43706) 2024-03-15T12:58:43.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e65af1640 1 -- 172.21.15.2:0/1170443214 learned_addr learned my addr 172.21.15.2:0/1170443214 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:43.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e662f2640 1 --2- 172.21.15.2:0/1170443214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e60100490 0x7f3e60197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:43.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e66af3640 1 --2- 172.21.15.2:0/1170443214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e60102040 0x7f3e6019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:43.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e65af1640 1 -- 172.21.15.2:0/1170443214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e60102040 msgr2=0x7f3e6019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:43.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e65af1640 1 --2- 172.21.15.2:0/1170443214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e60102040 0x7f3e6019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e65af1640 1 -- 172.21.15.2:0/1170443214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e60100490 msgr2=0x7f3e60197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:43.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e65af1640 1 --2- 172.21.15.2:0/1170443214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e60100490 0x7f3e60197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e65af1640 1 -- 172.21.15.2:0/1170443214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e50009560 con 0x7f3e60101680 2024-03-15T12:58:43.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e662f2640 1 --2- 172.21.15.2:0/1170443214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e60100490 0x7f3e60197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:43.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e65af1640 1 --2- 172.21.15.2:0/1170443214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e60101680 0x7f3e60197b90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009230 tx=0x7f3e50004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:43.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e577fe640 1 -- 172.21.15.2:0/1170443214 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e50004250 con 0x7f3e60101680 2024-03-15T12:58:43.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e577fe640 1 -- 172.21.15.2:0/1170443214 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3e500043b0 con 0x7f3e60101680 2024-03-15T12:58:43.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e68556640 1 -- 172.21.15.2:0/1170443214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6019f6f0 con 0x7f3e60101680 2024-03-15T12:58:43.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e577fe640 1 -- 172.21.15.2:0/1170443214 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e50003cd0 con 0x7f3e60101680 2024-03-15T12:58:43.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.037+0000 7f3e68556640 1 -- 172.21.15.2:0/1170443214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6019fc50 con 0x7f3e60101680 2024-03-15T12:58:43.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.038+0000 7f3e68556640 1 -- 172.21.15.2:0/1170443214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6010be00 con 0x7f3e60101680 2024-03-15T12:58:43.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.039+0000 7f3e577fe640 1 -- 172.21.15.2:0/1170443214 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f3e50005000 con 0x7f3e60101680 2024-03-15T12:58:43.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.040+0000 7f3e577fe640 1 --2- 172.21.15.2:0/1170443214 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e4c0a1b40 0x7f3e4c0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:43.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.040+0000 7f3e577fe640 1 -- 172.21.15.2:0/1170443214 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f3e500efa20 con 0x7f3e60101680 2024-03-15T12:58:43.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.040+0000 7f3e662f2640 1 --2- 172.21.15.2:0/1170443214 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e4c0a1b40 0x7f3e4c0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:43.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.040+0000 7f3e662f2640 1 --2- 172.21.15.2:0/1170443214 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e4c0a1b40 0x7f3e4c0a4000 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3e601014e0 tx=0x7f3e58005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:43.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.044+0000 7f3e577fe640 1 -- 172.21.15.2:0/1170443214 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3e500b88a0 con 0x7f3e60101680 2024-03-15T12:58:43.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:42 smithi184 ceph-mon[35993]: pgmap v362: 129 pgs: 129 active+clean; 654 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-03-15T12:58:43.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:42 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3807276406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:43.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:42 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1683245139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:43.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:42 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2631948023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:43.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:42 smithi149 ceph-mon[35927]: pgmap v362: 129 pgs: 129 active+clean; 654 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-03-15T12:58:43.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:42 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3807276406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:43.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:42 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1683245139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:43.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:42 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2631948023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:43.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.222+0000 7f3e68556640 1 -- 172.21.15.2:0/1170443214 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e600fdbd0 con 0x7f3e4c0a1b40 2024-03-15T12:58:43.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.231+0000 7f3e577fe640 1 -- 172.21.15.2:0/1170443214 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340536 (secure 0 0 0) 0x7f3e600fdbd0 con 0x7f3e4c0a1b40 2024-03-15T12:58:43.221 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:43.223 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-03-15T12:58:43.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.234+0000 7f3e68556640 1 -- 172.21.15.2:0/1170443214 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e4c0a1b40 msgr2=0x7f3e4c0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:43.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.234+0000 7f3e68556640 1 --2- 172.21.15.2:0/1170443214 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e4c0a1b40 0x7f3e4c0a4000 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3e601014e0 tx=0x7f3e58005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:43.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.234+0000 7f3e68556640 1 -- 172.21.15.2:0/1170443214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e60101680 msgr2=0x7f3e60197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:43.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.234+0000 7f3e68556640 1 --2- 172.21.15.2:0/1170443214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e60101680 0x7f3e60197b90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009230 tx=0x7f3e50004a10 comp rx=0 tx=0).stop 2024-03-15T12:58:43.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.234+0000 7f3e68556640 1 -- 172.21.15.2:0/1170443214 shutdown_connections 2024-03-15T12:58:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.234+0000 7f3e68556640 1 --2- 172.21.15.2:0/1170443214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e60102040 0x7f3e6019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.234+0000 7f3e68556640 1 --2- 172.21.15.2:0/1170443214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e60101680 0x7f3e60197b90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.234+0000 7f3e68556640 1 --2- 172.21.15.2:0/1170443214 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e4c0a1b40 0x7f3e4c0a4000 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.234+0000 7f3e68556640 1 --2- 172.21.15.2:0/1170443214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e60100490 0x7f3e60197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.234+0000 7f3e68556640 1 -- 172.21.15.2:0/1170443214 >> 172.21.15.2:0/1170443214 conn(0x7f3e600fba20 msgr2=0x7f3e600fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.234+0000 7f3e68556640 1 -- 172.21.15.2:0/1170443214 shutdown_connections 2024-03-15T12:58:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.234+0000 7f3e68556640 1 -- 172.21.15.2:0/1170443214 wait complete. 2024-03-15T12:58:43.235 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-03-15T12:58:42.776711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669661,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1360,"num_read_kb":820,"num_write":454,"num_write_kb":3107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400840,"kb_used_data":14248,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540984,"statfs":{"total":1151940427776,"available":1151529967616,"internally_reserved":0,"allocated":14589952,"data_stored":7981696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10081,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":64,"num_write":25,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003507"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.789229+0000","last_change":"2024-03-15T12:56:21.210343+0000","last_active":"2024-03-15T12:57:48.789229+0000","last_peered":"2024-03-15T12:57:48.789229+0000","last_clean":"2024-03-15T12:57:48.789229+0000","last_became_active":"2024-03-15T12:56:21.209787+0000","last_became_peered":"2024-03-15T12:56:21.209787+0000","last_unstale":"2024-03-15T12:57:48.789229+0000","last_undegraded":"2024-03-15T12:57:48.789229+0000","last_fullsized":"2024-03-15T12:57:48.789229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:32:21.204944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.778728+0000","last_change":"2024-03-15T12:56:21.214153+0000","last_active":"2024-03-15T12:57:48.778728+0000","last_peered":"2024-03-15T12:57:48.778728+0000","last_clean":"2024-03-15T12:57:48.778728+0000","last_became_active":"2024-03-15T12:56:21.213819+0000","last_became_peered":"2024-03-15T12:56:21.213819+0000","last_unstale":"2024-03-15T12:57:48.778728+0000","last_undegraded":"2024-03-15T12:57:48.778728+0000","last_fullsized":"2024-03-15T12:57:48.778728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:31:43.264698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.792740+0000","last_change":"2024-03-15T12:56:21.221678+0000","last_active":"2024-03-15T12:57:48.792740+0000","last_peered":"2024-03-15T12:57:48.792740+0000","last_clean":"2024-03-15T12:57:48.792740+0000","last_became_active":"2024-03-15T12:56:21.221272+0000","last_became_peered":"2024-03-15T12:56:21.221272+0000","last_unstale":"2024-03-15T12:57:48.792740+0000","last_undegraded":"2024-03-15T12:57:48.792740+0000","last_fullsized":"2024-03-15T12:57:48.792740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:01:50.778707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.754638+0000","last_change":"2024-03-15T12:56:21.212939+0000","last_active":"2024-03-15T12:57:48.754638+0000","last_peered":"2024-03-15T12:57:48.754638+0000","last_clean":"2024-03-15T12:57:48.754638+0000","last_became_active":"2024-03-15T12:56:21.212437+0000","last_became_peered":"2024-03-15T12:56:21.212437+0000","last_unstale":"2024-03-15T12:57:48.754638+0000","last_undegraded":"2024-03-15T12:57:48.754638+0000","last_fullsized":"2024-03-15T12:57:48.754638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:34:11.377020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.771047+0000","last_change":"2024-03-15T12:56:21.221433+0000","last_active":"2024-03-15T12:57:48.771047+0000","last_peered":"2024-03-15T12:57:48.771047+0000","last_clean":"2024-03-15T12:57:48.771047+0000","last_became_active":"2024-03-15T12:56:21.220605+0000","last_became_peered":"2024-03-15T12:56:21.220605+0000","last_unstale":"2024-03-15T12:57:48.771047+0000","last_undegraded":"2024-03-15T12:57:48.771047+0000","last_fullsized":"2024-03-15T12:57:48.771047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:20:25.169387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784751+0000","last_change":"2024-03-15T12:56:21.216539+0000","last_active":"2024-03-15T12:57:48.784751+0000","last_peered":"2024-03-15T12:57:48.784751+0000","last_clean":"2024-03-15T12:57:48.784751+0000","last_became_active":"2024-03-15T12:56:21.216197+0000","last_became_peered":"2024-03-15T12:56:21.216197+0000","last_unstale":"2024-03-15T12:57:48.784751+0000","last_undegraded":"2024-03-15T12:57:48.784751+0000","last_fullsized":"2024-03-15T12:57:48.784751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:00:23.186467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.779848+0000","last_change":"2024-03-15T12:56:21.213103+0000","last_active":"2024-03-15T12:57:48.779848+0000","last_peered":"2024-03-15T12:57:48.779848+0000","last_clean":"2024-03-15T12:57:48.779848+0000","last_became_active":"2024-03-15T12:56:21.212789+0000","last_became_peered":"2024-03-15T12:56:21.212789+0000","last_unstale":"2024-03-15T12:57:48.779848+0000","last_undegraded":"2024-03-15T12:57:48.779848+0000","last_fullsized":"2024-03-15T12:57:48.779848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:41:50.140915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.785673+0000","last_change":"2024-03-15T12:56:21.214767+0000","last_active":"2024-03-15T12:57:48.785673+0000","last_peered":"2024-03-15T12:57:48.785673+0000","last_clean":"2024-03-15T12:57:48.785673+0000","last_became_active":"2024-03-15T12:56:21.213896+0000","last_became_peered":"2024-03-15T12:56:21.213896+0000","last_unstale":"2024-03-15T12:57:48.785673+0000","last_undegraded":"2024-03-15T12:57:48.785673+0000","last_fullsized":"2024-03-15T12:57:48.785673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:33:53.819906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.796148+0000","last_change":"2024-03-15T12:56:21.218566+0000","last_active":"2024-03-15T12:57:48.796148+0000","last_peered":"2024-03-15T12:57:48.796148+0000","last_clean":"2024-03-15T12:57:48.796148+0000","last_became_active":"2024-03-15T12:56:21.218234+0000","last_became_peered":"2024-03-15T12:56:21.218234+0000","last_unstale":"2024-03-15T12:57:48.796148+0000","last_undegraded":"2024-03-15T12:57:48.796148+0000","last_fullsized":"2024-03-15T12:57:48.796148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:56:46.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784768+0000","last_change":"2024-03-15T12:56:21.219367+0000","last_active":"2024-03-15T12:57:48.784768+0000","last_peered":"2024-03-15T12:57:48.784768+0000","last_clean":"2024-03-15T12:57:48.784768+0000","last_became_active":"2024-03-15T12:56:21.217541+0000","last_became_peered":"2024-03-15T12:56:21.217541+0000","last_unstale":"2024-03-15T12:57:48.784768+0000","last_undegraded":"2024-03-15T12:57:48.784768+0000","last_fullsized":"2024-03-15T12:57:48.784768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:19:12.939394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798017+0000","last_change":"2024-03-15T12:56:21.222246+0000","last_active":"2024-03-15T12:57:48.798017+0000","last_peered":"2024-03-15T12:57:48.798017+0000","last_clean":"2024-03-15T12:57:48.798017+0000","last_became_active":"2024-03-15T12:56:21.221883+0000","last_became_peered":"2024-03-15T12:56:21.221883+0000","last_unstale":"2024-03-15T12:57:48.798017+0000","last_undegraded":"2024-03-15T12:57:48.798017+0000","last_fullsized":"2024-03-15T12:57:48.798017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:42:06.892581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.771534+0000","last_change":"2024-03-15T12:57:19.841386+0000","last_active":"2024-03-15T12:57:48.771534+0000","last_peered":"2024-03-15T12:57:48.771534+0000","last_clean":"2024-03-15T12:57:48.771534+0000","last_became_active":"2024-03-15T12:57:19.840981+0000","last_became_peered":"2024-03-15T12:57:19.840981+0000","last_unstale":"2024-03-15T12:57:48.771534+0000","last_undegraded":"2024-03-15T12:57:48.771534+0000","last_fullsized":"2024-03-15T12:57:48.771534+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:57:28.443452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.776665+0000","last_change":"2024-03-15T12:56:21.199631+0000","last_active":"2024-03-15T12:57:48.776665+0000","last_peered":"2024-03-15T12:57:48.776665+0000","last_clean":"2024-03-15T12:57:48.776665+0000","last_became_active":"2024-03-15T12:56:21.199230+0000","last_became_peered":"2024-03-15T12:56:21.199230+0000","last_unstale":"2024-03-15T12:57:48.776665+0000","last_undegraded":"2024-03-15T12:57:48.776665+0000","last_fullsized":"2024-03-15T12:57:48.776665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:45:51.356803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.775081+0000","last_change":"2024-03-15T12:56:21.211449+0000","last_active":"2024-03-15T12:57:48.775081+0000","last_peered":"2024-03-15T12:57:48.775081+0000","last_clean":"2024-03-15T12:57:48.775081+0000","last_became_active":"2024-03-15T12:56:21.211001+0000","last_became_peered":"2024-03-15T12:56:21.211001+0000","last_unstale":"2024-03-15T12:57:48.775081+0000","last_undegraded":"2024-03-15T12:57:48.775081+0000","last_fullsized":"2024-03-15T12:57:48.775081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:32:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.779936+0000","last_change":"2024-03-15T12:56:21.212001+0000","last_active":"2024-03-15T12:57:48.779936+0000","last_peered":"2024-03-15T12:57:48.779936+0000","last_clean":"2024-03-15T12:57:48.779936+0000","last_became_active":"2024-03-15T12:56:21.211497+0000","last_became_peered":"2024-03-15T12:56:21.211497+0000","last_unstale":"2024-03-15T12:57:48.779936+0000","last_undegraded":"2024-03-15T12:57:48.779936+0000","last_fullsized":"2024-03-15T12:57:48.779936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:27:16.029202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.799977+0000","last_change":"2024-03-15T12:56:21.222368+0000","last_active":"2024-03-15T12:57:48.799977+0000","last_peered":"2024-03-15T12:57:48.799977+0000","last_clean":"2024-03-15T12:57:48.799977+0000","last_became_active":"2024-03-15T12:56:21.221356+0000","last_became_peered":"2024-03-15T12:56:21.221356+0000","last_unstale":"2024-03-15T12:57:48.799977+0000","last_undegraded":"2024-03-15T12:57:48.799977+0000","last_fullsized":"2024-03-15T12:57:48.799977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:15:04.125220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.789234+0000","last_change":"2024-03-15T12:56:21.211323+0000","last_active":"2024-03-15T12:57:48.789234+0000","last_peered":"2024-03-15T12:57:48.789234+0000","last_clean":"2024-03-15T12:57:48.789234+0000","last_became_active":"2024-03-15T12:56:21.210983+0000","last_became_peered":"2024-03-15T12:56:21.210983+0000","last_unstale":"2024-03-15T12:57:48.789234+0000","last_undegraded":"2024-03-15T12:57:48.789234+0000","last_fullsized":"2024-03-15T12:57:48.789234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:10:51.487985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.775434+0000","last_change":"2024-03-15T12:56:21.209435+0000","last_active":"2024-03-15T12:57:48.775434+0000","last_peered":"2024-03-15T12:57:48.775434+0000","last_clean":"2024-03-15T12:57:48.775434+0000","last_became_active":"2024-03-15T12:56:21.208981+0000","last_became_peered":"2024-03-15T12:56:21.208981+0000","last_unstale":"2024-03-15T12:57:48.775434+0000","last_undegraded":"2024-03-15T12:57:48.775434+0000","last_fullsized":"2024-03-15T12:57:48.775434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:47:59.142040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.768999+0000","last_change":"2024-03-15T12:56:21.222014+0000","last_active":"2024-03-15T12:57:48.768999+0000","last_peered":"2024-03-15T12:57:48.768999+0000","last_clean":"2024-03-15T12:57:48.768999+0000","last_became_active":"2024-03-15T12:56:21.221548+0000","last_became_peered":"2024-03-15T12:56:21.221548+0000","last_unstale":"2024-03-15T12:57:48.768999+0000","last_undegraded":"2024-03-15T12:57:48.768999+0000","last_fullsized":"2024-03-15T12:57:48.768999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:12:48.943874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834595+0000","last_change":"2024-03-15T12:56:25.447200+0000","last_active":"2024-03-15T12:57:19.834595+0000","last_peered":"2024-03-15T12:57:19.834595+0000","last_clean":"2024-03-15T12:57:19.834595+0000","last_became_active":"2024-03-15T12:56:25.446516+0000","last_became_peered":"2024-03-15T12:56:25.446516+0000","last_unstale":"2024-03-15T12:57:19.834595+0000","last_undegraded":"2024-03-15T12:57:19.834595+0000","last_fullsized":"2024-03-15T12:57:19.834595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:06:46.221772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":160,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:38.577105+0000","last_change":"2024-03-15T12:56:20.210826+0000","last_active":"2024-03-15T12:58:38.577105+0000","last_peered":"2024-03-15T12:58:38.577105+0000","last_clean":"2024-03-15T12:58:38.577105+0000","last_became_active":"2024-03-15T12:56:20.210472+0000","last_became_peered":"2024-03-15T12:56:20.210472+0000","last_unstale":"2024-03-15T12:58:38.577105+0000","last_undegraded":"2024-03-15T12:58:38.577105+0000","last_fullsized":"2024-03-15T12:58:38.577105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:33:22.896058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780590+0000","last_change":"2024-03-15T12:56:21.224665+0000","last_active":"2024-03-15T12:57:48.780590+0000","last_peered":"2024-03-15T12:57:48.780590+0000","last_clean":"2024-03-15T12:57:48.780590+0000","last_became_active":"2024-03-15T12:56:21.223826+0000","last_became_peered":"2024-03-15T12:56:21.223826+0000","last_unstale":"2024-03-15T12:57:48.780590+0000","last_undegraded":"2024-03-15T12:57:48.780590+0000","last_fullsized":"2024-03-15T12:57:48.780590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:11:57.576701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837559+0000","last_change":"2024-03-15T12:56:25.444665+0000","last_active":"2024-03-15T12:57:19.837559+0000","last_peered":"2024-03-15T12:57:19.837559+0000","last_clean":"2024-03-15T12:57:19.837559+0000","last_became_active":"2024-03-15T12:56:25.444275+0000","last_became_peered":"2024-03-15T12:56:25.444275+0000","last_unstale":"2024-03-15T12:57:19.837559+0000","last_undegraded":"2024-03-15T12:57:19.837559+0000","last_fullsized":"2024-03-15T12:57:19.837559+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:29:46.470405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104609+0000","last_change":"2024-03-15T12:56:20.215873+0000","last_active":"2024-03-15T12:57:20.104609+0000","last_peered":"2024-03-15T12:57:20.104609+0000","last_clean":"2024-03-15T12:57:20.104609+0000","last_became_active":"2024-03-15T12:56:20.215505+0000","last_became_peered":"2024-03-15T12:56:20.215505+0000","last_unstale":"2024-03-15T12:57:20.104609+0000","last_undegraded":"2024-03-15T12:57:20.104609+0000","last_fullsized":"2024-03-15T12:57:20.104609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:01:57.317382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.792776+0000","last_change":"2024-03-15T12:56:21.218200+0000","last_active":"2024-03-15T12:57:48.792776+0000","last_peered":"2024-03-15T12:57:48.792776+0000","last_clean":"2024-03-15T12:57:48.792776+0000","last_became_active":"2024-03-15T12:56:21.217768+0000","last_became_peered":"2024-03-15T12:56:21.217768+0000","last_unstale":"2024-03-15T12:57:48.792776+0000","last_undegraded":"2024-03-15T12:57:48.792776+0000","last_fullsized":"2024-03-15T12:57:48.792776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:39:42.443542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837005+0000","last_change":"2024-03-15T12:56:25.444220+0000","last_active":"2024-03-15T12:57:19.837005+0000","last_peered":"2024-03-15T12:57:19.837005+0000","last_clean":"2024-03-15T12:57:19.837005+0000","last_became_active":"2024-03-15T12:56:25.443361+0000","last_became_peered":"2024-03-15T12:56:25.443361+0000","last_unstale":"2024-03-15T12:57:19.837005+0000","last_undegraded":"2024-03-15T12:57:19.837005+0000","last_fullsized":"2024-03-15T12:57:19.837005+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:21:11.593526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837757+0000","last_change":"2024-03-15T12:56:20.218241+0000","last_active":"2024-03-15T12:57:19.837757+0000","last_peered":"2024-03-15T12:57:19.837757+0000","last_clean":"2024-03-15T12:57:19.837757+0000","last_became_active":"2024-03-15T12:56:20.217339+0000","last_became_peered":"2024-03-15T12:56:20.217339+0000","last_unstale":"2024-03-15T12:57:19.837757+0000","last_undegraded":"2024-03-15T12:57:19.837757+0000","last_fullsized":"2024-03-15T12:57:19.837757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:01:22.156502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798260+0000","last_change":"2024-03-15T12:56:21.217224+0000","last_active":"2024-03-15T12:57:48.798260+0000","last_peered":"2024-03-15T12:57:48.798260+0000","last_clean":"2024-03-15T12:57:48.798260+0000","last_became_active":"2024-03-15T12:56:21.216868+0000","last_became_peered":"2024-03-15T12:56:21.216868+0000","last_unstale":"2024-03-15T12:57:48.798260+0000","last_undegraded":"2024-03-15T12:57:48.798260+0000","last_fullsized":"2024-03-15T12:57:48.798260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:03:32.668463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836741+0000","last_change":"2024-03-15T12:56:25.443861+0000","last_active":"2024-03-15T12:57:19.836741+0000","last_peered":"2024-03-15T12:57:19.836741+0000","last_clean":"2024-03-15T12:57:19.836741+0000","last_became_active":"2024-03-15T12:56:25.443386+0000","last_became_peered":"2024-03-15T12:56:25.443386+0000","last_unstale":"2024-03-15T12:57:19.836741+0000","last_undegraded":"2024-03-15T12:57:19.836741+0000","last_fullsized":"2024-03-15T12:57:19.836741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:00:23.063425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837030+0000","last_change":"2024-03-15T12:56:20.212451+0000","last_active":"2024-03-15T12:57:19.837030+0000","last_peered":"2024-03-15T12:57:19.837030+0000","last_clean":"2024-03-15T12:57:19.837030+0000","last_became_active":"2024-03-15T12:56:20.212118+0000","last_became_peered":"2024-03-15T12:56:20.212118+0000","last_unstale":"2024-03-15T12:57:19.837030+0000","last_undegraded":"2024-03-15T12:57:19.837030+0000","last_fullsized":"2024-03-15T12:57:19.837030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:24:33.667652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.766138+0000","last_change":"2024-03-15T12:57:20.107434+0000","last_active":"2024-03-15T12:57:48.766138+0000","last_peered":"2024-03-15T12:57:48.766138+0000","last_clean":"2024-03-15T12:57:48.766138+0000","last_became_active":"2024-03-15T12:57:20.107075+0000","last_became_peered":"2024-03-15T12:57:20.107075+0000","last_unstale":"2024-03-15T12:57:48.766138+0000","last_undegraded":"2024-03-15T12:57:48.766138+0000","last_fullsized":"2024-03-15T12:57:48.766138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:24:44.944629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834590+0000","last_change":"2024-03-15T12:56:25.438194+0000","last_active":"2024-03-15T12:57:19.834590+0000","last_peered":"2024-03-15T12:57:19.834590+0000","last_clean":"2024-03-15T12:57:19.834590+0000","last_became_active":"2024-03-15T12:56:25.437769+0000","last_became_peered":"2024-03-15T12:56:25.437769+0000","last_unstale":"2024-03-15T12:57:19.834590+0000","last_undegraded":"2024-03-15T12:57:19.834590+0000","last_fullsized":"2024-03-15T12:57:19.834590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:29:44.398731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:37.528626+0000","last_change":"2024-03-15T12:56:20.210552+0000","last_active":"2024-03-15T12:58:37.528626+0000","last_peered":"2024-03-15T12:58:37.528626+0000","last_clean":"2024-03-15T12:58:37.528626+0000","last_became_active":"2024-03-15T12:56:20.210191+0000","last_became_peered":"2024-03-15T12:56:20.210191+0000","last_unstale":"2024-03-15T12:58:37.528626+0000","last_undegraded":"2024-03-15T12:58:37.528626+0000","last_fullsized":"2024-03-15T12:58:37.528626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:25:48.225266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.754896+0000","last_change":"2024-03-15T12:56:21.220582+0000","last_active":"2024-03-15T12:57:48.754896+0000","last_peered":"2024-03-15T12:57:48.754896+0000","last_clean":"2024-03-15T12:57:48.754896+0000","last_became_active":"2024-03-15T12:56:21.220212+0000","last_became_peered":"2024-03-15T12:56:21.220212+0000","last_unstale":"2024-03-15T12:57:48.754896+0000","last_undegraded":"2024-03-15T12:57:48.754896+0000","last_fullsized":"2024-03-15T12:57:48.754896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:00:54.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834490+0000","last_change":"2024-03-15T12:56:25.446410+0000","last_active":"2024-03-15T12:57:19.834490+0000","last_peered":"2024-03-15T12:57:19.834490+0000","last_clean":"2024-03-15T12:57:19.834490+0000","last_became_active":"2024-03-15T12:56:25.445554+0000","last_became_peered":"2024-03-15T12:56:25.445554+0000","last_unstale":"2024-03-15T12:57:19.834490+0000","last_undegraded":"2024-03-15T12:57:19.834490+0000","last_fullsized":"2024-03-15T12:57:19.834490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:35:08.049374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.839292+0000","last_change":"2024-03-15T12:56:20.212599+0000","last_active":"2024-03-15T12:57:19.839292+0000","last_peered":"2024-03-15T12:57:19.839292+0000","last_clean":"2024-03-15T12:57:19.839292+0000","last_became_active":"2024-03-15T12:56:20.212289+0000","last_became_peered":"2024-03-15T12:56:20.212289+0000","last_unstale":"2024-03-15T12:57:19.839292+0000","last_undegraded":"2024-03-15T12:57:19.839292+0000","last_fullsized":"2024-03-15T12:57:19.839292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:12:51.036882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.764372+0000","last_change":"2024-03-15T12:56:21.219407+0000","last_active":"2024-03-15T12:57:48.764372+0000","last_peered":"2024-03-15T12:57:48.764372+0000","last_clean":"2024-03-15T12:57:48.764372+0000","last_became_active":"2024-03-15T12:56:21.219035+0000","last_became_peered":"2024-03-15T12:56:21.219035+0000","last_unstale":"2024-03-15T12:57:48.764372+0000","last_undegraded":"2024-03-15T12:57:48.764372+0000","last_fullsized":"2024-03-15T12:57:48.764372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:35:43.129767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.842180+0000","last_change":"2024-03-15T12:57:19.841693+0000","last_active":"2024-03-15T12:57:19.842180+0000","last_peered":"2024-03-15T12:57:19.842180+0000","last_clean":"2024-03-15T12:57:19.842180+0000","last_became_active":"2024-03-15T12:57:19.841275+0000","last_became_peered":"2024-03-15T12:57:19.841275+0000","last_unstale":"2024-03-15T12:57:19.842180+0000","last_undegraded":"2024-03-15T12:57:19.842180+0000","last_fullsized":"2024-03-15T12:57:19.842180+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:41:18.147235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837559+0000","last_change":"2024-03-15T12:56:20.209933+0000","last_active":"2024-03-15T12:57:19.837559+0000","last_peered":"2024-03-15T12:57:19.837559+0000","last_clean":"2024-03-15T12:57:19.837559+0000","last_became_active":"2024-03-15T12:56:20.209047+0000","last_became_peered":"2024-03-15T12:56:20.209047+0000","last_unstale":"2024-03-15T12:57:19.837559+0000","last_undegraded":"2024-03-15T12:57:19.837559+0000","last_fullsized":"2024-03-15T12:57:19.837559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:24:05.551745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782760+0000","last_change":"2024-03-15T12:56:21.215198+0000","last_active":"2024-03-15T12:57:48.782760+0000","last_peered":"2024-03-15T12:57:48.782760+0000","last_clean":"2024-03-15T12:57:48.782760+0000","last_became_active":"2024-03-15T12:56:21.214844+0000","last_became_peered":"2024-03-15T12:56:21.214844+0000","last_unstale":"2024-03-15T12:57:48.782760+0000","last_undegraded":"2024-03-15T12:57:48.782760+0000","last_fullsized":"2024-03-15T12:57:48.782760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:37:08.398317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836412+0000","last_change":"2024-03-15T12:56:25.437047+0000","last_active":"2024-03-15T12:57:19.836412+0000","last_peered":"2024-03-15T12:57:19.836412+0000","last_clean":"2024-03-15T12:57:19.836412+0000","last_became_active":"2024-03-15T12:56:25.436595+0000","last_became_peered":"2024-03-15T12:56:25.436595+0000","last_unstale":"2024-03-15T12:57:19.836412+0000","last_undegraded":"2024-03-15T12:57:19.836412+0000","last_fullsized":"2024-03-15T12:57:19.836412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:59:55.147044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103923+0000","last_change":"2024-03-15T12:56:20.206348+0000","last_active":"2024-03-15T12:57:20.103923+0000","last_peered":"2024-03-15T12:57:20.103923+0000","last_clean":"2024-03-15T12:57:20.103923+0000","last_became_active":"2024-03-15T12:56:20.206013+0000","last_became_peered":"2024-03-15T12:56:20.206013+0000","last_unstale":"2024-03-15T12:57:20.103923+0000","last_undegraded":"2024-03-15T12:57:20.103923+0000","last_fullsized":"2024-03-15T12:57:20.103923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:32:26.625982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782151+0000","last_change":"2024-03-15T12:56:21.216517+0000","last_active":"2024-03-15T12:57:48.782151+0000","last_peered":"2024-03-15T12:57:48.782151+0000","last_clean":"2024-03-15T12:57:48.782151+0000","last_became_active":"2024-03-15T12:56:21.216076+0000","last_became_peered":"2024-03-15T12:56:21.216076+0000","last_unstale":"2024-03-15T12:57:48.782151+0000","last_undegraded":"2024-03-15T12:57:48.782151+0000","last_fullsized":"2024-03-15T12:57:48.782151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:19:35.547273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104273+0000","last_change":"2024-03-15T12:56:25.437367+0000","last_active":"2024-03-15T12:57:20.104273+0000","last_peered":"2024-03-15T12:57:20.104273+0000","last_clean":"2024-03-15T12:57:20.104273+0000","last_became_active":"2024-03-15T12:56:25.436930+0000","last_became_peered":"2024-03-15T12:56:25.436930+0000","last_unstale":"2024-03-15T12:57:20.104273+0000","last_undegraded":"2024-03-15T12:57:20.104273+0000","last_fullsized":"2024-03-15T12:57:20.104273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:53:56.586786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":270,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:39.530740+0000","last_change":"2024-03-15T12:56:20.214067+0000","last_active":"2024-03-15T12:58:39.530740+0000","last_peered":"2024-03-15T12:58:39.530740+0000","last_clean":"2024-03-15T12:58:39.530740+0000","last_became_active":"2024-03-15T12:56:20.213696+0000","last_became_peered":"2024-03-15T12:56:20.213696+0000","last_unstale":"2024-03-15T12:58:39.530740+0000","last_undegraded":"2024-03-15T12:58:39.530740+0000","last_fullsized":"2024-03-15T12:58:39.530740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:35:15.343660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":237,"num_read_kb":237,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.797642+0000","last_change":"2024-03-15T12:56:21.211106+0000","last_active":"2024-03-15T12:57:48.797642+0000","last_peered":"2024-03-15T12:57:48.797642+0000","last_clean":"2024-03-15T12:57:48.797642+0000","last_became_active":"2024-03-15T12:56:21.210756+0000","last_became_peered":"2024-03-15T12:56:21.210756+0000","last_unstale":"2024-03-15T12:57:48.797642+0000","last_undegraded":"2024-03-15T12:57:48.797642+0000","last_fullsized":"2024-03-15T12:57:48.797642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:13:25.232077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.833697+0000","last_change":"2024-03-15T12:56:25.441364+0000","last_active":"2024-03-15T12:57:19.833697+0000","last_peered":"2024-03-15T12:57:19.833697+0000","last_clean":"2024-03-15T12:57:19.833697+0000","last_became_active":"2024-03-15T12:56:25.440951+0000","last_became_peered":"2024-03-15T12:56:25.440951+0000","last_unstale":"2024-03-15T12:57:19.833697+0000","last_undegraded":"2024-03-15T12:57:19.833697+0000","last_fullsized":"2024-03-15T12:57:19.833697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:32:28.651856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104583+0000","last_change":"2024-03-15T12:56:20.208633+0000","last_active":"2024-03-15T12:57:20.104583+0000","last_peered":"2024-03-15T12:57:20.104583+0000","last_clean":"2024-03-15T12:57:20.104583+0000","last_became_active":"2024-03-15T12:56:20.208280+0000","last_became_peered":"2024-03-15T12:56:20.208280+0000","last_unstale":"2024-03-15T12:57:20.104583+0000","last_undegraded":"2024-03-15T12:57:20.104583+0000","last_fullsized":"2024-03-15T12:57:20.104583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:00:15.274090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.785458+0000","last_change":"2024-03-15T12:56:21.218998+0000","last_active":"2024-03-15T12:57:48.785458+0000","last_peered":"2024-03-15T12:57:48.785458+0000","last_clean":"2024-03-15T12:57:48.785458+0000","last_became_active":"2024-03-15T12:56:21.218617+0000","last_became_peered":"2024-03-15T12:56:21.218617+0000","last_unstale":"2024-03-15T12:57:48.785458+0000","last_undegraded":"2024-03-15T12:57:48.785458+0000","last_fullsized":"2024-03-15T12:57:48.785458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:20:25.496204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837212+0000","last_change":"2024-03-15T12:56:25.438969+0000","last_active":"2024-03-15T12:57:19.837212+0000","last_peered":"2024-03-15T12:57:19.837212+0000","last_clean":"2024-03-15T12:57:19.837212+0000","last_became_active":"2024-03-15T12:56:25.438603+0000","last_became_peered":"2024-03-15T12:56:25.438603+0000","last_unstale":"2024-03-15T12:57:19.837212+0000","last_undegraded":"2024-03-15T12:57:19.837212+0000","last_fullsized":"2024-03-15T12:57:19.837212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:54:23.992253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104585+0000","last_change":"2024-03-15T12:56:20.215152+0000","last_active":"2024-03-15T12:57:20.104585+0000","last_peered":"2024-03-15T12:57:20.104585+0000","last_clean":"2024-03-15T12:57:20.104585+0000","last_became_active":"2024-03-15T12:56:20.214762+0000","last_became_peered":"2024-03-15T12:56:20.214762+0000","last_unstale":"2024-03-15T12:57:20.104585+0000","last_undegraded":"2024-03-15T12:57:20.104585+0000","last_fullsized":"2024-03-15T12:57:20.104585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:38:45.414984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.790199+0000","last_change":"2024-03-15T12:56:21.214005+0000","last_active":"2024-03-15T12:57:48.790199+0000","last_peered":"2024-03-15T12:57:48.790199+0000","last_clean":"2024-03-15T12:57:48.790199+0000","last_became_active":"2024-03-15T12:56:21.213672+0000","last_became_peered":"2024-03-15T12:56:21.213672+0000","last_unstale":"2024-03-15T12:57:48.790199+0000","last_undegraded":"2024-03-15T12:57:48.790199+0000","last_fullsized":"2024-03-15T12:57:48.790199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:13:09.384245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.833759+0000","last_change":"2024-03-15T12:56:25.443145+0000","last_active":"2024-03-15T12:57:19.833759+0000","last_peered":"2024-03-15T12:57:19.833759+0000","last_clean":"2024-03-15T12:57:19.833759+0000","last_became_active":"2024-03-15T12:56:25.442755+0000","last_became_peered":"2024-03-15T12:56:25.442755+0000","last_unstale":"2024-03-15T12:57:19.833759+0000","last_undegraded":"2024-03-15T12:57:19.833759+0000","last_fullsized":"2024-03-15T12:57:19.833759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:14:47.201919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'40","reported_seq":318,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:39.531730+0000","last_change":"2024-03-15T12:56:20.215063+0000","last_active":"2024-03-15T12:58:39.531730+0000","last_peered":"2024-03-15T12:58:39.531730+0000","last_clean":"2024-03-15T12:58:39.531730+0000","last_became_active":"2024-03-15T12:56:20.214742+0000","last_became_peered":"2024-03-15T12:56:20.214742+0000","last_unstale":"2024-03-15T12:58:39.531730+0000","last_undegraded":"2024-03-15T12:58:39.531730+0000","last_fullsized":"2024-03-15T12:58:39.531730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:38:12.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69742,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":194,"num_write":42,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.773641+0000","last_change":"2024-03-15T12:56:21.213519+0000","last_active":"2024-03-15T12:57:48.773641+0000","last_peered":"2024-03-15T12:57:48.773641+0000","last_clean":"2024-03-15T12:57:48.773641+0000","last_became_active":"2024-03-15T12:56:21.213198+0000","last_became_peered":"2024-03-15T12:56:21.213198+0000","last_unstale":"2024-03-15T12:57:48.773641+0000","last_undegraded":"2024-03-15T12:57:48.773641+0000","last_fullsized":"2024-03-15T12:57:48.773641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:22:13.037068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106198+0000","last_change":"2024-03-15T12:57:20.106198+0000","last_active":"2024-03-15T12:57:20.106198+0000","last_peered":"2024-03-15T12:57:20.106198+0000","last_clean":"2024-03-15T12:57:20.106198+0000","last_became_active":"2024-03-15T12:57:20.105649+0000","last_became_peered":"2024-03-15T12:57:20.105649+0000","last_unstale":"2024-03-15T12:57:20.106198+0000","last_undegraded":"2024-03-15T12:57:20.106198+0000","last_fullsized":"2024-03-15T12:57:20.106198+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:44:46.615091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834609+0000","last_change":"2024-03-15T12:56:20.216124+0000","last_active":"2024-03-15T12:57:19.834609+0000","last_peered":"2024-03-15T12:57:19.834609+0000","last_clean":"2024-03-15T12:57:19.834609+0000","last_became_active":"2024-03-15T12:56:20.215754+0000","last_became_peered":"2024-03-15T12:56:20.215754+0000","last_unstale":"2024-03-15T12:57:19.834609+0000","last_undegraded":"2024-03-15T12:57:19.834609+0000","last_fullsized":"2024-03-15T12:57:19.834609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:11:44.403503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.781601+0000","last_change":"2024-03-15T12:56:21.214775+0000","last_active":"2024-03-15T12:57:48.781601+0000","last_peered":"2024-03-15T12:57:48.781601+0000","last_clean":"2024-03-15T12:57:48.781601+0000","last_became_active":"2024-03-15T12:56:21.214445+0000","last_became_peered":"2024-03-15T12:56:21.214445+0000","last_unstale":"2024-03-15T12:57:48.781601+0000","last_undegraded":"2024-03-15T12:57:48.781601+0000","last_fullsized":"2024-03-15T12:57:48.781601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:23:24.045670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.107668+0000","last_change":"2024-03-15T12:57:20.107668+0000","last_active":"2024-03-15T12:57:20.107668+0000","last_peered":"2024-03-15T12:57:20.107668+0000","last_clean":"2024-03-15T12:57:20.107668+0000","last_became_active":"2024-03-15T12:57:20.107259+0000","last_became_peered":"2024-03-15T12:57:20.107259+0000","last_unstale":"2024-03-15T12:57:20.107668+0000","last_undegraded":"2024-03-15T12:57:20.107668+0000","last_fullsized":"2024-03-15T12:57:20.107668+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:07:37.244869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104267+0000","last_change":"2024-03-15T12:56:20.217191+0000","last_active":"2024-03-15T12:57:20.104267+0000","last_peered":"2024-03-15T12:57:20.104267+0000","last_clean":"2024-03-15T12:57:20.104267+0000","last_became_active":"2024-03-15T12:56:20.216632+0000","last_became_peered":"2024-03-15T12:56:20.216632+0000","last_unstale":"2024-03-15T12:57:20.104267+0000","last_undegraded":"2024-03-15T12:57:20.104267+0000","last_fullsized":"2024-03-15T12:57:20.104267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:16:51.187920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784344+0000","last_change":"2024-03-15T12:56:21.218650+0000","last_active":"2024-03-15T12:57:48.784344+0000","last_peered":"2024-03-15T12:57:48.784344+0000","last_clean":"2024-03-15T12:57:48.784344+0000","last_became_active":"2024-03-15T12:56:21.218329+0000","last_became_peered":"2024-03-15T12:56:21.218329+0000","last_unstale":"2024-03-15T12:57:48.784344+0000","last_undegraded":"2024-03-15T12:57:48.784344+0000","last_fullsized":"2024-03-15T12:57:48.784344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:10:45.632248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834415+0000","last_change":"2024-03-15T12:56:25.447982+0000","last_active":"2024-03-15T12:57:19.834415+0000","last_peered":"2024-03-15T12:57:19.834415+0000","last_clean":"2024-03-15T12:57:19.834415+0000","last_became_active":"2024-03-15T12:56:25.447514+0000","last_became_peered":"2024-03-15T12:56:25.447514+0000","last_unstale":"2024-03-15T12:57:19.834415+0000","last_undegraded":"2024-03-15T12:57:19.834415+0000","last_fullsized":"2024-03-15T12:57:19.834415+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:16:52.567619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'8","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:37.576101+0000","last_change":"2024-03-15T12:56:20.209778+0000","last_active":"2024-03-15T12:58:37.576101+0000","last_peered":"2024-03-15T12:58:37.576101+0000","last_clean":"2024-03-15T12:58:37.576101+0000","last_became_active":"2024-03-15T12:56:20.209311+0000","last_became_peered":"2024-03-15T12:56:20.209311+0000","last_unstale":"2024-03-15T12:58:37.576101+0000","last_undegraded":"2024-03-15T12:58:37.576101+0000","last_fullsized":"2024-03-15T12:58:37.576101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:05:21.930210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.778891+0000","last_change":"2024-03-15T12:56:21.215576+0000","last_active":"2024-03-15T12:57:48.778891+0000","last_peered":"2024-03-15T12:57:48.778891+0000","last_clean":"2024-03-15T12:57:48.778891+0000","last_became_active":"2024-03-15T12:56:21.215261+0000","last_became_peered":"2024-03-15T12:56:21.215261+0000","last_unstale":"2024-03-15T12:57:48.778891+0000","last_undegraded":"2024-03-15T12:57:48.778891+0000","last_fullsized":"2024-03-15T12:57:48.778891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:35:15.461554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837297+0000","last_change":"2024-03-15T12:56:25.445559+0000","last_active":"2024-03-15T12:57:19.837297+0000","last_peered":"2024-03-15T12:57:19.837297+0000","last_clean":"2024-03-15T12:57:19.837297+0000","last_became_active":"2024-03-15T12:56:25.445092+0000","last_became_peered":"2024-03-15T12:56:25.445092+0000","last_unstale":"2024-03-15T12:57:19.837297+0000","last_undegraded":"2024-03-15T12:57:19.837297+0000","last_fullsized":"2024-03-15T12:57:19.837297+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:48:30.558243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836745+0000","last_change":"2024-03-15T12:56:20.214853+0000","last_active":"2024-03-15T12:57:19.836745+0000","last_peered":"2024-03-15T12:57:19.836745+0000","last_clean":"2024-03-15T12:57:19.836745+0000","last_became_active":"2024-03-15T12:56:20.214510+0000","last_became_peered":"2024-03-15T12:56:20.214510+0000","last_unstale":"2024-03-15T12:57:19.836745+0000","last_undegraded":"2024-03-15T12:57:19.836745+0000","last_fullsized":"2024-03-15T12:57:19.836745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:25:39.316428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798162+0000","last_change":"2024-03-15T12:56:21.216511+0000","last_active":"2024-03-15T12:57:48.798162+0000","last_peered":"2024-03-15T12:57:48.798162+0000","last_clean":"2024-03-15T12:57:48.798162+0000","last_became_active":"2024-03-15T12:56:21.216175+0000","last_became_peered":"2024-03-15T12:56:21.216175+0000","last_unstale":"2024-03-15T12:57:48.798162+0000","last_undegraded":"2024-03-15T12:57:48.798162+0000","last_fullsized":"2024-03-15T12:57:48.798162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:16:56.665841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104042+0000","last_change":"2024-03-15T12:56:25.446009+0000","last_active":"2024-03-15T12:57:20.104042+0000","last_peered":"2024-03-15T12:57:20.104042+0000","last_clean":"2024-03-15T12:57:20.104042+0000","last_became_active":"2024-03-15T12:56:25.445474+0000","last_became_peered":"2024-03-15T12:56:25.445474+0000","last_unstale":"2024-03-15T12:57:20.104042+0000","last_undegraded":"2024-03-15T12:57:20.104042+0000","last_fullsized":"2024-03-15T12:57:20.104042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:53:46.856905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:36.411188+0000","last_change":"2024-03-15T12:56:20.210146+0000","last_active":"2024-03-15T12:58:36.411188+0000","last_peered":"2024-03-15T12:58:36.411188+0000","last_clean":"2024-03-15T12:58:36.411188+0000","last_became_active":"2024-03-15T12:56:20.209769+0000","last_became_peered":"2024-03-15T12:56:20.209769+0000","last_unstale":"2024-03-15T12:58:36.411188+0000","last_undegraded":"2024-03-15T12:58:36.411188+0000","last_fullsized":"2024-03-15T12:58:36.411188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:47:32.505257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.770795+0000","last_change":"2024-03-15T12:56:21.214826+0000","last_active":"2024-03-15T12:57:48.770795+0000","last_peered":"2024-03-15T12:57:48.770795+0000","last_clean":"2024-03-15T12:57:48.770795+0000","last_became_active":"2024-03-15T12:56:21.214160+0000","last_became_peered":"2024-03-15T12:56:21.214160+0000","last_unstale":"2024-03-15T12:57:48.770795+0000","last_undegraded":"2024-03-15T12:57:48.770795+0000","last_fullsized":"2024-03-15T12:57:48.770795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:23:17.800522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837088+0000","last_change":"2024-03-15T12:56:25.443496+0000","last_active":"2024-03-15T12:57:19.837088+0000","last_peered":"2024-03-15T12:57:19.837088+0000","last_clean":"2024-03-15T12:57:19.837088+0000","last_became_active":"2024-03-15T12:56:25.442796+0000","last_became_peered":"2024-03-15T12:56:25.442796+0000","last_unstale":"2024-03-15T12:57:19.837088+0000","last_undegraded":"2024-03-15T12:57:19.837088+0000","last_fullsized":"2024-03-15T12:57:19.837088+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:35:23.892693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.841026+0000","last_change":"2024-03-15T12:57:19.840452+0000","last_active":"2024-03-15T12:57:19.841026+0000","last_peered":"2024-03-15T12:57:19.841026+0000","last_clean":"2024-03-15T12:57:19.841026+0000","last_became_active":"2024-03-15T12:57:19.839977+0000","last_became_peered":"2024-03-15T12:57:19.839977+0000","last_unstale":"2024-03-15T12:57:19.841026+0000","last_undegraded":"2024-03-15T12:57:19.841026+0000","last_fullsized":"2024-03-15T12:57:19.841026+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:59:25.026455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.777893+0000","last_change":"2024-03-15T12:56:21.216393+0000","last_active":"2024-03-15T12:57:48.777893+0000","last_peered":"2024-03-15T12:57:48.777893+0000","last_clean":"2024-03-15T12:57:48.777893+0000","last_became_active":"2024-03-15T12:56:21.216043+0000","last_became_peered":"2024-03-15T12:56:21.216043+0000","last_unstale":"2024-03-15T12:57:48.777893+0000","last_undegraded":"2024-03-15T12:57:48.777893+0000","last_fullsized":"2024-03-15T12:57:48.777893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:05:49.843726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837389+0000","last_change":"2024-03-15T12:56:25.443854+0000","last_active":"2024-03-15T12:57:19.837389+0000","last_peered":"2024-03-15T12:57:19.837389+0000","last_clean":"2024-03-15T12:57:19.837389+0000","last_became_active":"2024-03-15T12:56:25.443422+0000","last_became_peered":"2024-03-15T12:56:25.443422+0000","last_unstale":"2024-03-15T12:57:19.837389+0000","last_undegraded":"2024-03-15T12:57:19.837389+0000","last_fullsized":"2024-03-15T12:57:19.837389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:52:18.999161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:36.575217+0000","last_change":"2024-03-15T12:56:20.211493+0000","last_active":"2024-03-15T12:58:36.575217+0000","last_peered":"2024-03-15T12:58:36.575217+0000","last_clean":"2024-03-15T12:58:36.575217+0000","last_became_active":"2024-03-15T12:56:20.211154+0000","last_became_peered":"2024-03-15T12:56:20.211154+0000","last_unstale":"2024-03-15T12:58:36.575217+0000","last_undegraded":"2024-03-15T12:58:36.575217+0000","last_fullsized":"2024-03-15T12:58:36.575217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:26:28.026899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784429+0000","last_change":"2024-03-15T12:56:21.215999+0000","last_active":"2024-03-15T12:57:48.784429+0000","last_peered":"2024-03-15T12:57:48.784429+0000","last_clean":"2024-03-15T12:57:48.784429+0000","last_became_active":"2024-03-15T12:56:21.215657+0000","last_became_peered":"2024-03-15T12:56:21.215657+0000","last_unstale":"2024-03-15T12:57:48.784429+0000","last_undegraded":"2024-03-15T12:57:48.784429+0000","last_fullsized":"2024-03-15T12:57:48.784429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:47:59.901886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837294+0000","last_change":"2024-03-15T12:56:25.442246+0000","last_active":"2024-03-15T12:57:19.837294+0000","last_peered":"2024-03-15T12:57:19.837294+0000","last_clean":"2024-03-15T12:57:19.837294+0000","last_became_active":"2024-03-15T12:56:25.441755+0000","last_became_peered":"2024-03-15T12:56:25.441755+0000","last_unstale":"2024-03-15T12:57:19.837294+0000","last_undegraded":"2024-03-15T12:57:19.837294+0000","last_fullsized":"2024-03-15T12:57:19.837294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:47:36.037864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.753452+0000","last_change":"2024-03-15T12:56:21.215746+0000","last_active":"2024-03-15T12:57:48.753452+0000","last_peered":"2024-03-15T12:57:48.753452+0000","last_clean":"2024-03-15T12:57:48.753452+0000","last_became_active":"2024-03-15T12:56:21.215424+0000","last_became_peered":"2024-03-15T12:56:21.215424+0000","last_unstale":"2024-03-15T12:57:48.753452+0000","last_undegraded":"2024-03-15T12:57:48.753452+0000","last_fullsized":"2024-03-15T12:57:48.753452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:01:15.498119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'7","reported_seq":153,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:38.576866+0000","last_change":"2024-03-15T12:56:20.219705+0000","last_active":"2024-03-15T12:58:38.576866+0000","last_peered":"2024-03-15T12:58:38.576866+0000","last_clean":"2024-03-15T12:58:38.576866+0000","last_became_active":"2024-03-15T12:56:20.219328+0000","last_became_peered":"2024-03-15T12:56:20.219328+0000","last_unstale":"2024-03-15T12:58:38.576866+0000","last_undegraded":"2024-03-15T12:58:38.576866+0000","last_fullsized":"2024-03-15T12:58:38.576866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:47:52.171116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104879+0000","last_change":"2024-03-15T12:56:25.443094+0000","last_active":"2024-03-15T12:57:20.104879+0000","last_peered":"2024-03-15T12:57:20.104879+0000","last_clean":"2024-03-15T12:57:20.104879+0000","last_became_active":"2024-03-15T12:56:25.442708+0000","last_became_peered":"2024-03-15T12:56:25.442708+0000","last_unstale":"2024-03-15T12:57:20.104879+0000","last_undegraded":"2024-03-15T12:57:20.104879+0000","last_fullsized":"2024-03-15T12:57:20.104879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:03:07.769836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.788423+0000","last_change":"2024-03-15T12:56:21.223937+0000","last_active":"2024-03-15T12:57:48.788423+0000","last_peered":"2024-03-15T12:57:48.788423+0000","last_clean":"2024-03-15T12:57:48.788423+0000","last_became_active":"2024-03-15T12:56:21.223583+0000","last_became_peered":"2024-03-15T12:56:21.223583+0000","last_unstale":"2024-03-15T12:57:48.788423+0000","last_undegraded":"2024-03-15T12:57:48.788423+0000","last_fullsized":"2024-03-15T12:57:48.788423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:15:34.287445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837150+0000","last_change":"2024-03-15T12:56:20.210812+0000","last_active":"2024-03-15T12:57:19.837150+0000","last_peered":"2024-03-15T12:57:19.837150+0000","last_clean":"2024-03-15T12:57:19.837150+0000","last_became_active":"2024-03-15T12:56:20.210457+0000","last_became_peered":"2024-03-15T12:56:20.210457+0000","last_unstale":"2024-03-15T12:57:19.837150+0000","last_undegraded":"2024-03-15T12:57:19.837150+0000","last_fullsized":"2024-03-15T12:57:19.837150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:04:37.371637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":358,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104758+0000","last_change":"2024-03-15T12:55:08.392548+0000","last_active":"2024-03-15T12:57:20.104758+0000","last_peered":"2024-03-15T12:57:20.104758+0000","last_clean":"2024-03-15T12:57:20.104758+0000","last_became_active":"2024-03-15T12:55:08.391972+0000","last_became_peered":"2024-03-15T12:55:08.391972+0000","last_unstale":"2024-03-15T12:57:20.104758+0000","last_undegraded":"2024-03-15T12:57:20.104758+0000","last_fullsized":"2024-03-15T12:57:20.104758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:51:47.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:51:47.093558+0000","last_clean_scrub_stamp":"2024-03-15T12:51:47.093558+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:30:57.264886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104929+0000","last_change":"2024-03-15T12:56:25.443212+0000","last_active":"2024-03-15T12:57:20.104929+0000","last_peered":"2024-03-15T12:57:20.104929+0000","last_clean":"2024-03-15T12:57:20.104929+0000","last_became_active":"2024-03-15T12:56:25.442787+0000","last_became_peered":"2024-03-15T12:56:25.442787+0000","last_unstale":"2024-03-15T12:57:20.104929+0000","last_undegraded":"2024-03-15T12:57:20.104929+0000","last_fullsized":"2024-03-15T12:57:20.104929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:59:32.244953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.775951+0000","last_change":"2024-03-15T12:57:19.839935+0000","last_active":"2024-03-15T12:57:48.775951+0000","last_peered":"2024-03-15T12:57:48.775951+0000","last_clean":"2024-03-15T12:57:48.775951+0000","last_became_active":"2024-03-15T12:57:19.839546+0000","last_became_peered":"2024-03-15T12:57:19.839546+0000","last_unstale":"2024-03-15T12:57:48.775951+0000","last_undegraded":"2024-03-15T12:57:48.775951+0000","last_fullsized":"2024-03-15T12:57:48.775951+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:05:15.544159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104546+0000","last_change":"2024-03-15T12:56:20.210466+0000","last_active":"2024-03-15T12:57:20.104546+0000","last_peered":"2024-03-15T12:57:20.104546+0000","last_clean":"2024-03-15T12:57:20.104546+0000","last_became_active":"2024-03-15T12:56:20.210157+0000","last_became_peered":"2024-03-15T12:56:20.210157+0000","last_unstale":"2024-03-15T12:57:20.104546+0000","last_undegraded":"2024-03-15T12:57:20.104546+0000","last_fullsized":"2024-03-15T12:57:20.104546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:58:03.335826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.838324+0000","last_change":"2024-03-15T12:56:25.432641+0000","last_active":"2024-03-15T12:57:19.838324+0000","last_peered":"2024-03-15T12:57:19.838324+0000","last_clean":"2024-03-15T12:57:19.838324+0000","last_became_active":"2024-03-15T12:56:25.432212+0000","last_became_peered":"2024-03-15T12:56:25.432212+0000","last_unstale":"2024-03-15T12:57:19.838324+0000","last_undegraded":"2024-03-15T12:57:19.838324+0000","last_fullsized":"2024-03-15T12:57:19.838324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:40:20.447886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.846534+0000","last_change":"2024-03-15T12:57:19.846534+0000","last_active":"2024-03-15T12:57:19.846534+0000","last_peered":"2024-03-15T12:57:19.846534+0000","last_clean":"2024-03-15T12:57:19.846534+0000","last_became_active":"2024-03-15T12:57:19.841008+0000","last_became_peered":"2024-03-15T12:57:19.841008+0000","last_unstale":"2024-03-15T12:57:19.846534+0000","last_undegraded":"2024-03-15T12:57:19.846534+0000","last_fullsized":"2024-03-15T12:57:19.846534+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:46:17.129901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.778482+0000","last_change":"2024-03-15T12:56:21.207577+0000","last_active":"2024-03-15T12:57:48.778482+0000","last_peered":"2024-03-15T12:57:48.778482+0000","last_clean":"2024-03-15T12:57:48.778482+0000","last_became_active":"2024-03-15T12:56:21.207220+0000","last_became_peered":"2024-03-15T12:56:21.207220+0000","last_unstale":"2024-03-15T12:57:48.778482+0000","last_undegraded":"2024-03-15T12:57:48.778482+0000","last_fullsized":"2024-03-15T12:57:48.778482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:42:12.104220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104420+0000","last_change":"2024-03-15T12:56:25.439762+0000","last_active":"2024-03-15T12:57:20.104420+0000","last_peered":"2024-03-15T12:57:20.104420+0000","last_clean":"2024-03-15T12:57:20.104420+0000","last_became_active":"2024-03-15T12:56:25.439366+0000","last_became_peered":"2024-03-15T12:56:25.439366+0000","last_unstale":"2024-03-15T12:57:20.104420+0000","last_undegraded":"2024-03-15T12:57:20.104420+0000","last_fullsized":"2024-03-15T12:57:20.104420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:36:29.638449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'16","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:38.417721+0000","last_change":"2024-03-15T12:56:20.218438+0000","last_active":"2024-03-15T12:58:38.417721+0000","last_peered":"2024-03-15T12:58:38.417721+0000","last_clean":"2024-03-15T12:58:38.417721+0000","last_became_active":"2024-03-15T12:56:20.218071+0000","last_became_peered":"2024-03-15T12:56:20.218071+0000","last_unstale":"2024-03-15T12:58:38.417721+0000","last_undegraded":"2024-03-15T12:58:38.417721+0000","last_fullsized":"2024-03-15T12:58:38.417721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:24:16.295781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.783607+0000","last_change":"2024-03-15T12:56:21.218345+0000","last_active":"2024-03-15T12:57:48.783607+0000","last_peered":"2024-03-15T12:57:48.783607+0000","last_clean":"2024-03-15T12:57:48.783607+0000","last_became_active":"2024-03-15T12:56:21.216534+0000","last_became_peered":"2024-03-15T12:56:21.216534+0000","last_unstale":"2024-03-15T12:57:48.783607+0000","last_undegraded":"2024-03-15T12:57:48.783607+0000","last_fullsized":"2024-03-15T12:57:48.783607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:56:18.977509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834310+0000","last_change":"2024-03-15T12:56:25.435714+0000","last_active":"2024-03-15T12:57:19.834310+0000","last_peered":"2024-03-15T12:57:19.834310+0000","last_clean":"2024-03-15T12:57:19.834310+0000","last_became_active":"2024-03-15T12:56:25.435315+0000","last_became_peered":"2024-03-15T12:56:25.435315+0000","last_unstale":"2024-03-15T12:57:19.834310+0000","last_undegraded":"2024-03-15T12:57:19.834310+0000","last_fullsized":"2024-03-15T12:57:19.834310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:10:03.509078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837248+0000","last_change":"2024-03-15T12:56:20.209356+0000","last_active":"2024-03-15T12:57:19.837248+0000","last_peered":"2024-03-15T12:57:19.837248+0000","last_clean":"2024-03-15T12:57:19.837248+0000","last_became_active":"2024-03-15T12:56:20.208995+0000","last_became_peered":"2024-03-15T12:56:20.208995+0000","last_unstale":"2024-03-15T12:57:19.837248+0000","last_undegraded":"2024-03-15T12:57:19.837248+0000","last_fullsized":"2024-03-15T12:57:19.837248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:14:08.554567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.783714+0000","last_change":"2024-03-15T12:56:21.216315+0000","last_active":"2024-03-15T12:57:48.783714+0000","last_peered":"2024-03-15T12:57:48.783714+0000","last_clean":"2024-03-15T12:57:48.783714+0000","last_became_active":"2024-03-15T12:56:21.215943+0000","last_became_peered":"2024-03-15T12:56:21.215943+0000","last_unstale":"2024-03-15T12:57:48.783714+0000","last_undegraded":"2024-03-15T12:57:48.783714+0000","last_fullsized":"2024-03-15T12:57:48.783714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:47:29.913048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836875+0000","last_change":"2024-03-15T12:56:25.437200+0000","last_active":"2024-03-15T12:57:19.836875+0000","last_peered":"2024-03-15T12:57:19.836875+0000","last_clean":"2024-03-15T12:57:19.836875+0000","last_became_active":"2024-03-15T12:56:25.436786+0000","last_became_peered":"2024-03-15T12:56:25.436786+0000","last_unstale":"2024-03-15T12:57:19.836875+0000","last_undegraded":"2024-03-15T12:57:19.836875+0000","last_fullsized":"2024-03-15T12:57:19.836875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:12:00.286749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106367+0000","last_change":"2024-03-15T12:56:20.202240+0000","last_active":"2024-03-15T12:57:20.106367+0000","last_peered":"2024-03-15T12:57:20.106367+0000","last_clean":"2024-03-15T12:57:20.106367+0000","last_became_active":"2024-03-15T12:56:20.201136+0000","last_became_peered":"2024-03-15T12:56:20.201136+0000","last_unstale":"2024-03-15T12:57:20.106367+0000","last_undegraded":"2024-03-15T12:57:20.106367+0000","last_fullsized":"2024-03-15T12:57:20.106367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:30:35.901237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.759652+0000","last_change":"2024-03-15T12:56:21.212095+0000","last_active":"2024-03-15T12:57:48.759652+0000","last_peered":"2024-03-15T12:57:48.759652+0000","last_clean":"2024-03-15T12:57:48.759652+0000","last_became_active":"2024-03-15T12:56:21.211744+0000","last_became_peered":"2024-03-15T12:56:21.211744+0000","last_unstale":"2024-03-15T12:57:48.759652+0000","last_undegraded":"2024-03-15T12:57:48.759652+0000","last_fullsized":"2024-03-15T12:57:48.759652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:27:58.412513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106031+0000","last_change":"2024-03-15T12:57:20.106031+0000","last_active":"2024-03-15T12:57:20.106031+0000","last_peered":"2024-03-15T12:57:20.106031+0000","last_clean":"2024-03-15T12:57:20.106031+0000","last_became_active":"2024-03-15T12:57:20.105499+0000","last_became_peered":"2024-03-15T12:57:20.105499+0000","last_unstale":"2024-03-15T12:57:20.106031+0000","last_undegraded":"2024-03-15T12:57:20.106031+0000","last_fullsized":"2024-03-15T12:57:20.106031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:17:08.138164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104141+0000","last_change":"2024-03-15T12:56:20.217319+0000","last_active":"2024-03-15T12:57:20.104141+0000","last_peered":"2024-03-15T12:57:20.104141+0000","last_clean":"2024-03-15T12:57:20.104141+0000","last_became_active":"2024-03-15T12:56:20.215450+0000","last_became_peered":"2024-03-15T12:56:20.215450+0000","last_unstale":"2024-03-15T12:57:20.104141+0000","last_undegraded":"2024-03-15T12:57:20.104141+0000","last_fullsized":"2024-03-15T12:57:20.104141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:07:09.085860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780239+0000","last_change":"2024-03-15T12:56:21.209360+0000","last_active":"2024-03-15T12:57:48.780239+0000","last_peered":"2024-03-15T12:57:48.780239+0000","last_clean":"2024-03-15T12:57:48.780239+0000","last_became_active":"2024-03-15T12:56:21.208987+0000","last_became_peered":"2024-03-15T12:56:21.208987+0000","last_unstale":"2024-03-15T12:57:48.780239+0000","last_undegraded":"2024-03-15T12:57:48.780239+0000","last_fullsized":"2024-03-15T12:57:48.780239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:19:13.353400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836774+0000","last_change":"2024-03-15T12:56:25.439173+0000","last_active":"2024-03-15T12:57:19.836774+0000","last_peered":"2024-03-15T12:57:19.836774+0000","last_clean":"2024-03-15T12:57:19.836774+0000","last_became_active":"2024-03-15T12:56:25.438812+0000","last_became_peered":"2024-03-15T12:56:25.438812+0000","last_unstale":"2024-03-15T12:57:19.836774+0000","last_undegraded":"2024-03-15T12:57:19.836774+0000","last_fullsized":"2024-03-15T12:57:19.836774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:10:39.107190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:35.836940+0000","last_change":"2024-03-15T12:56:20.211861+0000","last_active":"2024-03-15T12:58:35.836940+0000","last_peered":"2024-03-15T12:58:35.836940+0000","last_clean":"2024-03-15T12:58:35.836940+0000","last_became_active":"2024-03-15T12:56:20.211481+0000","last_became_peered":"2024-03-15T12:56:20.211481+0000","last_unstale":"2024-03-15T12:58:35.836940+0000","last_undegraded":"2024-03-15T12:58:35.836940+0000","last_fullsized":"2024-03-15T12:58:35.836940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:12:45.558423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.783692+0000","last_change":"2024-03-15T12:56:21.216058+0000","last_active":"2024-03-15T12:57:48.783692+0000","last_peered":"2024-03-15T12:57:48.783692+0000","last_clean":"2024-03-15T12:57:48.783692+0000","last_became_active":"2024-03-15T12:56:21.215706+0000","last_became_peered":"2024-03-15T12:56:21.215706+0000","last_unstale":"2024-03-15T12:57:48.783692+0000","last_undegraded":"2024-03-15T12:57:48.783692+0000","last_fullsized":"2024-03-15T12:57:48.783692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:55:02.132976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836966+0000","last_change":"2024-03-15T12:56:25.444134+0000","last_active":"2024-03-15T12:57:19.836966+0000","last_peered":"2024-03-15T12:57:19.836966+0000","last_clean":"2024-03-15T12:57:19.836966+0000","last_became_active":"2024-03-15T12:56:25.443452+0000","last_became_peered":"2024-03-15T12:56:25.443452+0000","last_unstale":"2024-03-15T12:57:19.836966+0000","last_undegraded":"2024-03-15T12:57:19.836966+0000","last_fullsized":"2024-03-15T12:57:19.836966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:06:39.660016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836785+0000","last_change":"2024-03-15T12:56:20.214700+0000","last_active":"2024-03-15T12:57:19.836785+0000","last_peered":"2024-03-15T12:57:19.836785+0000","last_clean":"2024-03-15T12:57:19.836785+0000","last_became_active":"2024-03-15T12:56:20.214348+0000","last_became_peered":"2024-03-15T12:56:20.214348+0000","last_unstale":"2024-03-15T12:57:19.836785+0000","last_undegraded":"2024-03-15T12:57:19.836785+0000","last_fullsized":"2024-03-15T12:57:19.836785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:24:37.551182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798936+0000","last_change":"2024-03-15T12:56:21.220746+0000","last_active":"2024-03-15T12:57:48.798936+0000","last_peered":"2024-03-15T12:57:48.798936+0000","last_clean":"2024-03-15T12:57:48.798936+0000","last_became_active":"2024-03-15T12:56:21.220445+0000","last_became_peered":"2024-03-15T12:56:21.220445+0000","last_unstale":"2024-03-15T12:57:48.798936+0000","last_undegraded":"2024-03-15T12:57:48.798936+0000","last_fullsized":"2024-03-15T12:57:48.798936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:58:44.119476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104747+0000","last_change":"2024-03-15T12:56:25.437611+0000","last_active":"2024-03-15T12:57:20.104747+0000","last_peered":"2024-03-15T12:57:20.104747+0000","last_clean":"2024-03-15T12:57:20.104747+0000","last_became_active":"2024-03-15T12:56:25.437195+0000","last_became_peered":"2024-03-15T12:56:25.437195+0000","last_unstale":"2024-03-15T12:57:20.104747+0000","last_undegraded":"2024-03-15T12:57:20.104747+0000","last_fullsized":"2024-03-15T12:57:20.104747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:26:43.558477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":156,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:39.603492+0000","last_change":"2024-03-15T12:56:20.208584+0000","last_active":"2024-03-15T12:58:39.603492+0000","last_peered":"2024-03-15T12:58:39.603492+0000","last_clean":"2024-03-15T12:58:39.603492+0000","last_became_active":"2024-03-15T12:56:20.208155+0000","last_became_peered":"2024-03-15T12:56:20.208155+0000","last_unstale":"2024-03-15T12:58:39.603492+0000","last_undegraded":"2024-03-15T12:58:39.603492+0000","last_fullsized":"2024-03-15T12:58:39.603492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:26:13.148273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2054,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798776+0000","last_change":"2024-03-15T12:56:21.220421+0000","last_active":"2024-03-15T12:57:48.798776+0000","last_peered":"2024-03-15T12:57:48.798776+0000","last_clean":"2024-03-15T12:57:48.798776+0000","last_became_active":"2024-03-15T12:56:21.220095+0000","last_became_peered":"2024-03-15T12:56:21.220095+0000","last_unstale":"2024-03-15T12:57:48.798776+0000","last_undegraded":"2024-03-15T12:57:48.798776+0000","last_fullsized":"2024-03-15T12:57:48.798776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:35:33.846506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836547+0000","last_change":"2024-03-15T12:56:25.440408+0000","last_active":"2024-03-15T12:57:19.836547+0000","last_peered":"2024-03-15T12:57:19.836547+0000","last_clean":"2024-03-15T12:57:19.836547+0000","last_became_active":"2024-03-15T12:56:25.439965+0000","last_became_peered":"2024-03-15T12:56:25.439965+0000","last_unstale":"2024-03-15T12:57:19.836547+0000","last_undegraded":"2024-03-15T12:57:19.836547+0000","last_fullsized":"2024-03-15T12:57:19.836547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:12:57.278202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.105052+0000","last_change":"2024-03-15T12:56:20.216320+0000","last_active":"2024-03-15T12:57:20.105052+0000","last_peered":"2024-03-15T12:57:20.105052+0000","last_clean":"2024-03-15T12:57:20.105052+0000","last_became_active":"2024-03-15T12:56:20.215915+0000","last_became_peered":"2024-03-15T12:56:20.215915+0000","last_unstale":"2024-03-15T12:57:20.105052+0000","last_undegraded":"2024-03-15T12:57:20.105052+0000","last_fullsized":"2024-03-15T12:57:20.105052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:25:45.236269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782769+0000","last_change":"2024-03-15T12:56:21.215822+0000","last_active":"2024-03-15T12:57:48.782769+0000","last_peered":"2024-03-15T12:57:48.782769+0000","last_clean":"2024-03-15T12:57:48.782769+0000","last_became_active":"2024-03-15T12:56:21.215508+0000","last_became_peered":"2024-03-15T12:56:21.215508+0000","last_unstale":"2024-03-15T12:57:48.782769+0000","last_undegraded":"2024-03-15T12:57:48.782769+0000","last_fullsized":"2024-03-15T12:57:48.782769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:56:55.919198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103859+0000","last_change":"2024-03-15T12:56:25.436186+0000","last_active":"2024-03-15T12:57:20.103859+0000","last_peered":"2024-03-15T12:57:20.103859+0000","last_clean":"2024-03-15T12:57:20.103859+0000","last_became_active":"2024-03-15T12:56:25.435586+0000","last_became_peered":"2024-03-15T12:56:25.435586+0000","last_unstale":"2024-03-15T12:57:20.103859+0000","last_undegraded":"2024-03-15T12:57:20.103859+0000","last_fullsized":"2024-03-15T12:57:20.103859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:02:26.850487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837495+0000","last_change":"2024-03-15T12:56:20.217277+0000","last_active":"2024-03-15T12:57:19.837495+0000","last_peered":"2024-03-15T12:57:19.837495+0000","last_clean":"2024-03-15T12:57:19.837495+0000","last_became_active":"2024-03-15T12:56:20.216930+0000","last_became_peered":"2024-03-15T12:56:20.216930+0000","last_unstale":"2024-03-15T12:57:19.837495+0000","last_undegraded":"2024-03-15T12:57:19.837495+0000","last_fullsized":"2024-03-15T12:57:19.837495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:47:51.075545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780182+0000","last_change":"2024-03-15T12:56:21.212066+0000","last_active":"2024-03-15T12:57:48.780182+0000","last_peered":"2024-03-15T12:57:48.780182+0000","last_clean":"2024-03-15T12:57:48.780182+0000","last_became_active":"2024-03-15T12:56:21.211650+0000","last_became_peered":"2024-03-15T12:56:21.211650+0000","last_unstale":"2024-03-15T12:57:48.780182+0000","last_undegraded":"2024-03-15T12:57:48.780182+0000","last_fullsized":"2024-03-15T12:57:48.780182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:40:35.322210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.772659+0000","last_change":"2024-03-15T12:56:21.212794+0000","last_active":"2024-03-15T12:57:48.772659+0000","last_peered":"2024-03-15T12:57:48.772659+0000","last_clean":"2024-03-15T12:57:48.772659+0000","last_became_active":"2024-03-15T12:56:21.212470+0000","last_became_peered":"2024-03-15T12:56:21.212470+0000","last_unstale":"2024-03-15T12:57:48.772659+0000","last_undegraded":"2024-03-15T12:57:48.772659+0000","last_fullsized":"2024-03-15T12:57:48.772659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:10:04.614223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.756171+0000","last_change":"2024-03-15T12:56:21.211746+0000","last_active":"2024-03-15T12:57:48.756171+0000","last_peered":"2024-03-15T12:57:48.756171+0000","last_clean":"2024-03-15T12:57:48.756171+0000","last_became_active":"2024-03-15T12:56:21.211431+0000","last_became_peered":"2024-03-15T12:56:21.211431+0000","last_unstale":"2024-03-15T12:57:48.756171+0000","last_undegraded":"2024-03-15T12:57:48.756171+0000","last_fullsized":"2024-03-15T12:57:48.756171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:46:31.188408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.778038+0000","last_change":"2024-03-15T12:56:21.220997+0000","last_active":"2024-03-15T12:57:48.778038+0000","last_peered":"2024-03-15T12:57:48.778038+0000","last_clean":"2024-03-15T12:57:48.778038+0000","last_became_active":"2024-03-15T12:56:21.220644+0000","last_became_peered":"2024-03-15T12:56:21.220644+0000","last_unstale":"2024-03-15T12:57:48.778038+0000","last_undegraded":"2024-03-15T12:57:48.778038+0000","last_fullsized":"2024-03-15T12:57:48.778038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:47:55.852845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780052+0000","last_change":"2024-03-15T12:56:21.211634+0000","last_active":"2024-03-15T12:57:48.780052+0000","last_peered":"2024-03-15T12:57:48.780052+0000","last_clean":"2024-03-15T12:57:48.780052+0000","last_became_active":"2024-03-15T12:56:21.211303+0000","last_became_peered":"2024-03-15T12:56:21.211303+0000","last_unstale":"2024-03-15T12:57:48.780052+0000","last_undegraded":"2024-03-15T12:57:48.780052+0000","last_fullsized":"2024-03-15T12:57:48.780052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:22:32.358270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780806+0000","last_change":"2024-03-15T12:56:21.211673+0000","last_active":"2024-03-15T12:57:48.780806+0000","last_peered":"2024-03-15T12:57:48.780806+0000","last_clean":"2024-03-15T12:57:48.780806+0000","last_became_active":"2024-03-15T12:56:21.211296+0000","last_became_peered":"2024-03-15T12:56:21.211296+0000","last_unstale":"2024-03-15T12:57:48.780806+0000","last_undegraded":"2024-03-15T12:57:48.780806+0000","last_fullsized":"2024-03-15T12:57:48.780806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:40:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.769894+0000","last_change":"2024-03-15T12:56:21.208681+0000","last_active":"2024-03-15T12:57:48.769894+0000","last_peered":"2024-03-15T12:57:48.769894+0000","last_clean":"2024-03-15T12:57:48.769894+0000","last_became_active":"2024-03-15T12:56:21.208130+0000","last_became_peered":"2024-03-15T12:56:21.208130+0000","last_unstale":"2024-03-15T12:57:48.769894+0000","last_undegraded":"2024-03-15T12:57:48.769894+0000","last_fullsized":"2024-03-15T12:57:48.769894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:34:52.971485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.793618+0000","last_change":"2024-03-15T12:56:21.216979+0000","last_active":"2024-03-15T12:57:48.793618+0000","last_peered":"2024-03-15T12:57:48.793618+0000","last_clean":"2024-03-15T12:57:48.793618+0000","last_became_active":"2024-03-15T12:56:21.216632+0000","last_became_peered":"2024-03-15T12:56:21.216632+0000","last_unstale":"2024-03-15T12:57:48.793618+0000","last_undegraded":"2024-03-15T12:57:48.793618+0000","last_fullsized":"2024-03-15T12:57:48.793618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:21:57.213512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.785754+0000","last_change":"2024-03-15T12:56:21.223064+0000","last_active":"2024-03-15T12:57:48.785754+0000","last_peered":"2024-03-15T12:57:48.785754+0000","last_clean":"2024-03-15T12:57:48.785754+0000","last_became_active":"2024-03-15T12:56:21.220882+0000","last_became_peered":"2024-03-15T12:56:21.220882+0000","last_unstale":"2024-03-15T12:57:48.785754+0000","last_undegraded":"2024-03-15T12:57:48.785754+0000","last_fullsized":"2024-03-15T12:57:48.785754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:45:02.930154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782047+0000","last_change":"2024-03-15T12:56:21.218614+0000","last_active":"2024-03-15T12:57:48.782047+0000","last_peered":"2024-03-15T12:57:48.782047+0000","last_clean":"2024-03-15T12:57:48.782047+0000","last_became_active":"2024-03-15T12:56:21.218224+0000","last_became_peered":"2024-03-15T12:56:21.218224+0000","last_unstale":"2024-03-15T12:57:48.782047+0000","last_undegraded":"2024-03-15T12:57:48.782047+0000","last_fullsized":"2024-03-15T12:57:48.782047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:41:29.790358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782852+0000","last_change":"2024-03-15T12:56:21.219019+0000","last_active":"2024-03-15T12:57:48.782852+0000","last_peered":"2024-03-15T12:57:48.782852+0000","last_clean":"2024-03-15T12:57:48.782852+0000","last_became_active":"2024-03-15T12:56:21.218673+0000","last_became_peered":"2024-03-15T12:56:21.218673+0000","last_unstale":"2024-03-15T12:57:48.782852+0000","last_undegraded":"2024-03-15T12:57:48.782852+0000","last_fullsized":"2024-03-15T12:57:48.782852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:42:11.788598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.799975+0000","last_change":"2024-03-15T12:56:21.220355+0000","last_active":"2024-03-15T12:57:48.799975+0000","last_peered":"2024-03-15T12:57:48.799975+0000","last_clean":"2024-03-15T12:57:48.799975+0000","last_became_active":"2024-03-15T12:56:21.220021+0000","last_became_peered":"2024-03-15T12:56:21.220021+0000","last_unstale":"2024-03-15T12:57:48.799975+0000","last_undegraded":"2024-03-15T12:57:48.799975+0000","last_fullsized":"2024-03-15T12:57:48.799975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:40:00.947351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.781265+0000","last_change":"2024-03-15T12:56:21.213050+0000","last_active":"2024-03-15T12:57:48.781265+0000","last_peered":"2024-03-15T12:57:48.781265+0000","last_clean":"2024-03-15T12:57:48.781265+0000","last_became_active":"2024-03-15T12:56:21.212730+0000","last_became_peered":"2024-03-15T12:56:21.212730+0000","last_unstale":"2024-03-15T12:57:48.781265+0000","last_undegraded":"2024-03-15T12:57:48.781265+0000","last_fullsized":"2024-03-15T12:57:48.781265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:56:36.396285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.770729+0000","last_change":"2024-03-15T12:56:21.213942+0000","last_active":"2024-03-15T12:57:48.770729+0000","last_peered":"2024-03-15T12:57:48.770729+0000","last_clean":"2024-03-15T12:57:48.770729+0000","last_became_active":"2024-03-15T12:56:21.213611+0000","last_became_peered":"2024-03-15T12:56:21.213611+0000","last_unstale":"2024-03-15T12:57:48.770729+0000","last_undegraded":"2024-03-15T12:57:48.770729+0000","last_fullsized":"2024-03-15T12:57:48.770729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:23:44.002970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79059,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1224,"num_read_kb":701,"num_write":271,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":237959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":501940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1728512,"data_stored":1151302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":504970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":501140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36948,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708204,"statfs":{"total":95995035648,"available":95957200896,"internally_reserved":0,"allocated":1650688,"data_stored":1099369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495683,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":496871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":568034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":498327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":568747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":496086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":501477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1093433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":72894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-15T12:58:43.241 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-15T12:58:43.242 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-15T12:58:43.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.396+0000 7fd8986ce640 1 -- 172.21.15.2:0/3286197231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890101f90 msgr2=0x7fd890104380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.396+0000 7fd8986ce640 1 --2- 172.21.15.2:0/3286197231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890101f90 0x7fd890104380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009900 tx=0x7fd88c02bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.396+0000 7fd8986ce640 1 -- 172.21.15.2:0/3286197231 shutdown_connections 2024-03-15T12:58:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.396+0000 7fd8986ce640 1 --2- 172.21.15.2:0/3286197231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8901071f0 0x7fd890109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.396+0000 7fd8986ce640 1 --2- 172.21.15.2:0/3286197231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8901048c0 0x7fd890106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.396+0000 7fd8986ce640 1 --2- 172.21.15.2:0/3286197231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890101f90 0x7fd890104380 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.396+0000 7fd8986ce640 1 -- 172.21.15.2:0/3286197231 >> 172.21.15.2:0/3286197231 conn(0x7fd8900fb980 msgr2=0x7fd8900fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.396+0000 7fd8986ce640 1 -- 172.21.15.2:0/3286197231 shutdown_connections 2024-03-15T12:58:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.396+0000 7fd8986ce640 1 -- 172.21.15.2:0/3286197231 wait complete. 2024-03-15T12:58:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.396+0000 7fd8986ce640 1 Processor -- start 2024-03-15T12:58:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.396+0000 7fd8986ce640 1 -- start start 2024-03-15T12:58:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd8986ce640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890101f90 0x7fd8901975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd8986ce640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8901048c0 0x7fd890197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:43.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd8986ce640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8901071f0 0x7fd89019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:43.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd8986ce640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89019f0e0 con 0x7fd8901071f0 2024-03-15T12:58:43.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd8986ce640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89019f250 con 0x7fd890101f90 2024-03-15T12:58:43.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd8986ce640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89019f410 con 0x7fd8901048c0 2024-03-15T12:58:43.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd896c6b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8901071f0 0x7fd89019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:43.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd89646a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890101f90 0x7fd8901975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:43.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd89646a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890101f90 0x7fd8901975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:56224/0 (socket says 172.21.15.2:56224) 2024-03-15T12:58:43.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd89646a640 1 -- 172.21.15.2:0/2656316550 learned_addr learned my addr 172.21.15.2:0/2656316550 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:43.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd896c6b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8901071f0 0x7fd89019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43736/0 (socket says 172.21.15.2:43736) 2024-03-15T12:58:43.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd895c69640 1 --2- 172.21.15.2:0/2656316550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8901048c0 0x7fd890197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:43.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd89646a640 1 -- 172.21.15.2:0/2656316550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8901048c0 msgr2=0x7fd890197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:43.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd89646a640 1 --2- 172.21.15.2:0/2656316550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8901048c0 0x7fd890197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd89646a640 1 -- 172.21.15.2:0/2656316550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8901071f0 msgr2=0x7fd89019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:43.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd89646a640 1 --2- 172.21.15.2:0/2656316550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8901071f0 0x7fd89019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd89646a640 1 -- 172.21.15.2:0/2656316550 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd888009840 con 0x7fd890101f90 2024-03-15T12:58:43.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd895c69640 1 --2- 172.21.15.2:0/2656316550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8901048c0 0x7fd890197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:43.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd89646a640 1 --2- 172.21.15.2:0/2656316550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890101f90 0x7fd8901975f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd88c005ab0 tx=0x7fd88c004e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:43.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.397+0000 7fd8877fe640 1 -- 172.21.15.2:0/2656316550 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c0042d0 con 0x7fd890101f90 2024-03-15T12:58:43.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.398+0000 7fd8986ce640 1 -- 172.21.15.2:0/2656316550 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c009560 con 0x7fd890101f90 2024-03-15T12:58:43.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.398+0000 7fd8877fe640 1 -- 172.21.15.2:0/2656316550 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd88c004430 con 0x7fd890101f90 2024-03-15T12:58:43.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.398+0000 7fd8877fe640 1 -- 172.21.15.2:0/2656316550 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c003db0 con 0x7fd890101f90 2024-03-15T12:58:43.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.398+0000 7fd8986ce640 1 -- 172.21.15.2:0/2656316550 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89019fa10 con 0x7fd890101f90 2024-03-15T12:58:43.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.398+0000 7fd8986ce640 1 -- 172.21.15.2:0/2656316550 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8900fd470 con 0x7fd890101f90 2024-03-15T12:58:43.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.400+0000 7fd8877fe640 1 -- 172.21.15.2:0/2656316550 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7fd88c0035f0 con 0x7fd890101f90 2024-03-15T12:58:43.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.400+0000 7fd8877fe640 1 --2- 172.21.15.2:0/2656316550 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd87c0a1ac0 0x7fd87c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:43.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.400+0000 7fd895c69640 1 --2- 172.21.15.2:0/2656316550 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd87c0a1ac0 0x7fd87c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:43.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.400+0000 7fd8877fe640 1 -- 172.21.15.2:0/2656316550 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fd88c0f0100 con 0x7fd890101f90 2024-03-15T12:58:43.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.401+0000 7fd895c69640 1 --2- 172.21.15.2:0/2656316550 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd87c0a1ac0 0x7fd87c0a3f80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd890198b00 tx=0x7fd88000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:43.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.404+0000 7fd8877fe640 1 -- 172.21.15.2:0/2656316550 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd88c0b8f80 con 0x7fd890101f90 2024-03-15T12:58:43.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.586+0000 7fd8986ce640 1 -- 172.21.15.2:0/2656316550 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd89002d000 con 0x7fd87c0a1ac0 2024-03-15T12:58:43.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.596+0000 7fd8877fe640 1 -- 172.21.15.2:0/2656316550 <== mgr.14150 v2:172.21.15.2:6800/343748862 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340536 (secure 0 0 0) 0x7fd89002d000 con 0x7fd87c0a1ac0 2024-03-15T12:58:43.586 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:43.588 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-03-15T12:58:43.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.600+0000 7fd8986ce640 1 -- 172.21.15.2:0/2656316550 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd87c0a1ac0 msgr2=0x7fd87c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:43.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.600+0000 7fd8986ce640 1 --2- 172.21.15.2:0/2656316550 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd87c0a1ac0 0x7fd87c0a3f80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd890198b00 tx=0x7fd88000b390 comp rx=0 tx=0).stop 2024-03-15T12:58:43.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.600+0000 7fd8986ce640 1 -- 172.21.15.2:0/2656316550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890101f90 msgr2=0x7fd8901975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:43.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.600+0000 7fd8986ce640 1 --2- 172.21.15.2:0/2656316550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890101f90 0x7fd8901975f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd88c005ab0 tx=0x7fd88c004e50 comp rx=0 tx=0).stop 2024-03-15T12:58:43.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.600+0000 7fd8986ce640 1 -- 172.21.15.2:0/2656316550 shutdown_connections 2024-03-15T12:58:43.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.600+0000 7fd8986ce640 1 --2- 172.21.15.2:0/2656316550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8901071f0 0x7fd89019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.600+0000 7fd8986ce640 1 --2- 172.21.15.2:0/2656316550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8901048c0 0x7fd890197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.600+0000 7fd8986ce640 1 --2- 172.21.15.2:0/2656316550 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd87c0a1ac0 0x7fd87c0a3f80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.600+0000 7fd8986ce640 1 --2- 172.21.15.2:0/2656316550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890101f90 0x7fd8901975f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.600+0000 7fd8986ce640 1 -- 172.21.15.2:0/2656316550 >> 172.21.15.2:0/2656316550 conn(0x7fd8900fb980 msgr2=0x7fd8901028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:43.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.600+0000 7fd8986ce640 1 -- 172.21.15.2:0/2656316550 shutdown_connections 2024-03-15T12:58:43.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.600+0000 7fd8986ce640 1 -- 172.21.15.2:0/2656316550 wait complete. 2024-03-15T12:58:43.598 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-03-15T12:58:42.776711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669661,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1360,"num_read_kb":820,"num_write":454,"num_write_kb":3107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400840,"kb_used_data":14248,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540984,"statfs":{"total":1151940427776,"available":1151529967616,"internally_reserved":0,"allocated":14589952,"data_stored":7981696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10081,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":64,"num_write":25,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003507"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.789229+0000","last_change":"2024-03-15T12:56:21.210343+0000","last_active":"2024-03-15T12:57:48.789229+0000","last_peered":"2024-03-15T12:57:48.789229+0000","last_clean":"2024-03-15T12:57:48.789229+0000","last_became_active":"2024-03-15T12:56:21.209787+0000","last_became_peered":"2024-03-15T12:56:21.209787+0000","last_unstale":"2024-03-15T12:57:48.789229+0000","last_undegraded":"2024-03-15T12:57:48.789229+0000","last_fullsized":"2024-03-15T12:57:48.789229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:32:21.204944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.778728+0000","last_change":"2024-03-15T12:56:21.214153+0000","last_active":"2024-03-15T12:57:48.778728+0000","last_peered":"2024-03-15T12:57:48.778728+0000","last_clean":"2024-03-15T12:57:48.778728+0000","last_became_active":"2024-03-15T12:56:21.213819+0000","last_became_peered":"2024-03-15T12:56:21.213819+0000","last_unstale":"2024-03-15T12:57:48.778728+0000","last_undegraded":"2024-03-15T12:57:48.778728+0000","last_fullsized":"2024-03-15T12:57:48.778728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:31:43.264698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.792740+0000","last_change":"2024-03-15T12:56:21.221678+0000","last_active":"2024-03-15T12:57:48.792740+0000","last_peered":"2024-03-15T12:57:48.792740+0000","last_clean":"2024-03-15T12:57:48.792740+0000","last_became_active":"2024-03-15T12:56:21.221272+0000","last_became_peered":"2024-03-15T12:56:21.221272+0000","last_unstale":"2024-03-15T12:57:48.792740+0000","last_undegraded":"2024-03-15T12:57:48.792740+0000","last_fullsized":"2024-03-15T12:57:48.792740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:01:50.778707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.754638+0000","last_change":"2024-03-15T12:56:21.212939+0000","last_active":"2024-03-15T12:57:48.754638+0000","last_peered":"2024-03-15T12:57:48.754638+0000","last_clean":"2024-03-15T12:57:48.754638+0000","last_became_active":"2024-03-15T12:56:21.212437+0000","last_became_peered":"2024-03-15T12:56:21.212437+0000","last_unstale":"2024-03-15T12:57:48.754638+0000","last_undegraded":"2024-03-15T12:57:48.754638+0000","last_fullsized":"2024-03-15T12:57:48.754638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:34:11.377020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.771047+0000","last_change":"2024-03-15T12:56:21.221433+0000","last_active":"2024-03-15T12:57:48.771047+0000","last_peered":"2024-03-15T12:57:48.771047+0000","last_clean":"2024-03-15T12:57:48.771047+0000","last_became_active":"2024-03-15T12:56:21.220605+0000","last_became_peered":"2024-03-15T12:56:21.220605+0000","last_unstale":"2024-03-15T12:57:48.771047+0000","last_undegraded":"2024-03-15T12:57:48.771047+0000","last_fullsized":"2024-03-15T12:57:48.771047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:20:25.169387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784751+0000","last_change":"2024-03-15T12:56:21.216539+0000","last_active":"2024-03-15T12:57:48.784751+0000","last_peered":"2024-03-15T12:57:48.784751+0000","last_clean":"2024-03-15T12:57:48.784751+0000","last_became_active":"2024-03-15T12:56:21.216197+0000","last_became_peered":"2024-03-15T12:56:21.216197+0000","last_unstale":"2024-03-15T12:57:48.784751+0000","last_undegraded":"2024-03-15T12:57:48.784751+0000","last_fullsized":"2024-03-15T12:57:48.784751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:00:23.186467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.779848+0000","last_change":"2024-03-15T12:56:21.213103+0000","last_active":"2024-03-15T12:57:48.779848+0000","last_peered":"2024-03-15T12:57:48.779848+0000","last_clean":"2024-03-15T12:57:48.779848+0000","last_became_active":"2024-03-15T12:56:21.212789+0000","last_became_peered":"2024-03-15T12:56:21.212789+0000","last_unstale":"2024-03-15T12:57:48.779848+0000","last_undegraded":"2024-03-15T12:57:48.779848+0000","last_fullsized":"2024-03-15T12:57:48.779848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:41:50.140915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.785673+0000","last_change":"2024-03-15T12:56:21.214767+0000","last_active":"2024-03-15T12:57:48.785673+0000","last_peered":"2024-03-15T12:57:48.785673+0000","last_clean":"2024-03-15T12:57:48.785673+0000","last_became_active":"2024-03-15T12:56:21.213896+0000","last_became_peered":"2024-03-15T12:56:21.213896+0000","last_unstale":"2024-03-15T12:57:48.785673+0000","last_undegraded":"2024-03-15T12:57:48.785673+0000","last_fullsized":"2024-03-15T12:57:48.785673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:33:53.819906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.796148+0000","last_change":"2024-03-15T12:56:21.218566+0000","last_active":"2024-03-15T12:57:48.796148+0000","last_peered":"2024-03-15T12:57:48.796148+0000","last_clean":"2024-03-15T12:57:48.796148+0000","last_became_active":"2024-03-15T12:56:21.218234+0000","last_became_peered":"2024-03-15T12:56:21.218234+0000","last_unstale":"2024-03-15T12:57:48.796148+0000","last_undegraded":"2024-03-15T12:57:48.796148+0000","last_fullsized":"2024-03-15T12:57:48.796148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:56:46.291174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784768+0000","last_change":"2024-03-15T12:56:21.219367+0000","last_active":"2024-03-15T12:57:48.784768+0000","last_peered":"2024-03-15T12:57:48.784768+0000","last_clean":"2024-03-15T12:57:48.784768+0000","last_became_active":"2024-03-15T12:56:21.217541+0000","last_became_peered":"2024-03-15T12:56:21.217541+0000","last_unstale":"2024-03-15T12:57:48.784768+0000","last_undegraded":"2024-03-15T12:57:48.784768+0000","last_fullsized":"2024-03-15T12:57:48.784768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:19:12.939394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798017+0000","last_change":"2024-03-15T12:56:21.222246+0000","last_active":"2024-03-15T12:57:48.798017+0000","last_peered":"2024-03-15T12:57:48.798017+0000","last_clean":"2024-03-15T12:57:48.798017+0000","last_became_active":"2024-03-15T12:56:21.221883+0000","last_became_peered":"2024-03-15T12:56:21.221883+0000","last_unstale":"2024-03-15T12:57:48.798017+0000","last_undegraded":"2024-03-15T12:57:48.798017+0000","last_fullsized":"2024-03-15T12:57:48.798017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:42:06.892581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.771534+0000","last_change":"2024-03-15T12:57:19.841386+0000","last_active":"2024-03-15T12:57:48.771534+0000","last_peered":"2024-03-15T12:57:48.771534+0000","last_clean":"2024-03-15T12:57:48.771534+0000","last_became_active":"2024-03-15T12:57:19.840981+0000","last_became_peered":"2024-03-15T12:57:19.840981+0000","last_unstale":"2024-03-15T12:57:48.771534+0000","last_undegraded":"2024-03-15T12:57:48.771534+0000","last_fullsized":"2024-03-15T12:57:48.771534+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:57:28.443452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.776665+0000","last_change":"2024-03-15T12:56:21.199631+0000","last_active":"2024-03-15T12:57:48.776665+0000","last_peered":"2024-03-15T12:57:48.776665+0000","last_clean":"2024-03-15T12:57:48.776665+0000","last_became_active":"2024-03-15T12:56:21.199230+0000","last_became_peered":"2024-03-15T12:56:21.199230+0000","last_unstale":"2024-03-15T12:57:48.776665+0000","last_undegraded":"2024-03-15T12:57:48.776665+0000","last_fullsized":"2024-03-15T12:57:48.776665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:45:51.356803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.775081+0000","last_change":"2024-03-15T12:56:21.211449+0000","last_active":"2024-03-15T12:57:48.775081+0000","last_peered":"2024-03-15T12:57:48.775081+0000","last_clean":"2024-03-15T12:57:48.775081+0000","last_became_active":"2024-03-15T12:56:21.211001+0000","last_became_peered":"2024-03-15T12:56:21.211001+0000","last_unstale":"2024-03-15T12:57:48.775081+0000","last_undegraded":"2024-03-15T12:57:48.775081+0000","last_fullsized":"2024-03-15T12:57:48.775081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:32:29.076020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.779936+0000","last_change":"2024-03-15T12:56:21.212001+0000","last_active":"2024-03-15T12:57:48.779936+0000","last_peered":"2024-03-15T12:57:48.779936+0000","last_clean":"2024-03-15T12:57:48.779936+0000","last_became_active":"2024-03-15T12:56:21.211497+0000","last_became_peered":"2024-03-15T12:56:21.211497+0000","last_unstale":"2024-03-15T12:57:48.779936+0000","last_undegraded":"2024-03-15T12:57:48.779936+0000","last_fullsized":"2024-03-15T12:57:48.779936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:27:16.029202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.799977+0000","last_change":"2024-03-15T12:56:21.222368+0000","last_active":"2024-03-15T12:57:48.799977+0000","last_peered":"2024-03-15T12:57:48.799977+0000","last_clean":"2024-03-15T12:57:48.799977+0000","last_became_active":"2024-03-15T12:56:21.221356+0000","last_became_peered":"2024-03-15T12:56:21.221356+0000","last_unstale":"2024-03-15T12:57:48.799977+0000","last_undegraded":"2024-03-15T12:57:48.799977+0000","last_fullsized":"2024-03-15T12:57:48.799977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:15:04.125220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.789234+0000","last_change":"2024-03-15T12:56:21.211323+0000","last_active":"2024-03-15T12:57:48.789234+0000","last_peered":"2024-03-15T12:57:48.789234+0000","last_clean":"2024-03-15T12:57:48.789234+0000","last_became_active":"2024-03-15T12:56:21.210983+0000","last_became_peered":"2024-03-15T12:56:21.210983+0000","last_unstale":"2024-03-15T12:57:48.789234+0000","last_undegraded":"2024-03-15T12:57:48.789234+0000","last_fullsized":"2024-03-15T12:57:48.789234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:10:51.487985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.775434+0000","last_change":"2024-03-15T12:56:21.209435+0000","last_active":"2024-03-15T12:57:48.775434+0000","last_peered":"2024-03-15T12:57:48.775434+0000","last_clean":"2024-03-15T12:57:48.775434+0000","last_became_active":"2024-03-15T12:56:21.208981+0000","last_became_peered":"2024-03-15T12:56:21.208981+0000","last_unstale":"2024-03-15T12:57:48.775434+0000","last_undegraded":"2024-03-15T12:57:48.775434+0000","last_fullsized":"2024-03-15T12:57:48.775434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:47:59.142040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.768999+0000","last_change":"2024-03-15T12:56:21.222014+0000","last_active":"2024-03-15T12:57:48.768999+0000","last_peered":"2024-03-15T12:57:48.768999+0000","last_clean":"2024-03-15T12:57:48.768999+0000","last_became_active":"2024-03-15T12:56:21.221548+0000","last_became_peered":"2024-03-15T12:56:21.221548+0000","last_unstale":"2024-03-15T12:57:48.768999+0000","last_undegraded":"2024-03-15T12:57:48.768999+0000","last_fullsized":"2024-03-15T12:57:48.768999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:12:48.943874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834595+0000","last_change":"2024-03-15T12:56:25.447200+0000","last_active":"2024-03-15T12:57:19.834595+0000","last_peered":"2024-03-15T12:57:19.834595+0000","last_clean":"2024-03-15T12:57:19.834595+0000","last_became_active":"2024-03-15T12:56:25.446516+0000","last_became_peered":"2024-03-15T12:56:25.446516+0000","last_unstale":"2024-03-15T12:57:19.834595+0000","last_undegraded":"2024-03-15T12:57:19.834595+0000","last_fullsized":"2024-03-15T12:57:19.834595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:06:46.221772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":160,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:38.577105+0000","last_change":"2024-03-15T12:56:20.210826+0000","last_active":"2024-03-15T12:58:38.577105+0000","last_peered":"2024-03-15T12:58:38.577105+0000","last_clean":"2024-03-15T12:58:38.577105+0000","last_became_active":"2024-03-15T12:56:20.210472+0000","last_became_peered":"2024-03-15T12:56:20.210472+0000","last_unstale":"2024-03-15T12:58:38.577105+0000","last_undegraded":"2024-03-15T12:58:38.577105+0000","last_fullsized":"2024-03-15T12:58:38.577105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:33:22.896058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780590+0000","last_change":"2024-03-15T12:56:21.224665+0000","last_active":"2024-03-15T12:57:48.780590+0000","last_peered":"2024-03-15T12:57:48.780590+0000","last_clean":"2024-03-15T12:57:48.780590+0000","last_became_active":"2024-03-15T12:56:21.223826+0000","last_became_peered":"2024-03-15T12:56:21.223826+0000","last_unstale":"2024-03-15T12:57:48.780590+0000","last_undegraded":"2024-03-15T12:57:48.780590+0000","last_fullsized":"2024-03-15T12:57:48.780590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:11:57.576701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837559+0000","last_change":"2024-03-15T12:56:25.444665+0000","last_active":"2024-03-15T12:57:19.837559+0000","last_peered":"2024-03-15T12:57:19.837559+0000","last_clean":"2024-03-15T12:57:19.837559+0000","last_became_active":"2024-03-15T12:56:25.444275+0000","last_became_peered":"2024-03-15T12:56:25.444275+0000","last_unstale":"2024-03-15T12:57:19.837559+0000","last_undegraded":"2024-03-15T12:57:19.837559+0000","last_fullsized":"2024-03-15T12:57:19.837559+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:29:46.470405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104609+0000","last_change":"2024-03-15T12:56:20.215873+0000","last_active":"2024-03-15T12:57:20.104609+0000","last_peered":"2024-03-15T12:57:20.104609+0000","last_clean":"2024-03-15T12:57:20.104609+0000","last_became_active":"2024-03-15T12:56:20.215505+0000","last_became_peered":"2024-03-15T12:56:20.215505+0000","last_unstale":"2024-03-15T12:57:20.104609+0000","last_undegraded":"2024-03-15T12:57:20.104609+0000","last_fullsized":"2024-03-15T12:57:20.104609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:01:57.317382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.792776+0000","last_change":"2024-03-15T12:56:21.218200+0000","last_active":"2024-03-15T12:57:48.792776+0000","last_peered":"2024-03-15T12:57:48.792776+0000","last_clean":"2024-03-15T12:57:48.792776+0000","last_became_active":"2024-03-15T12:56:21.217768+0000","last_became_peered":"2024-03-15T12:56:21.217768+0000","last_unstale":"2024-03-15T12:57:48.792776+0000","last_undegraded":"2024-03-15T12:57:48.792776+0000","last_fullsized":"2024-03-15T12:57:48.792776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:39:42.443542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837005+0000","last_change":"2024-03-15T12:56:25.444220+0000","last_active":"2024-03-15T12:57:19.837005+0000","last_peered":"2024-03-15T12:57:19.837005+0000","last_clean":"2024-03-15T12:57:19.837005+0000","last_became_active":"2024-03-15T12:56:25.443361+0000","last_became_peered":"2024-03-15T12:56:25.443361+0000","last_unstale":"2024-03-15T12:57:19.837005+0000","last_undegraded":"2024-03-15T12:57:19.837005+0000","last_fullsized":"2024-03-15T12:57:19.837005+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:21:11.593526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837757+0000","last_change":"2024-03-15T12:56:20.218241+0000","last_active":"2024-03-15T12:57:19.837757+0000","last_peered":"2024-03-15T12:57:19.837757+0000","last_clean":"2024-03-15T12:57:19.837757+0000","last_became_active":"2024-03-15T12:56:20.217339+0000","last_became_peered":"2024-03-15T12:56:20.217339+0000","last_unstale":"2024-03-15T12:57:19.837757+0000","last_undegraded":"2024-03-15T12:57:19.837757+0000","last_fullsized":"2024-03-15T12:57:19.837757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:01:22.156502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798260+0000","last_change":"2024-03-15T12:56:21.217224+0000","last_active":"2024-03-15T12:57:48.798260+0000","last_peered":"2024-03-15T12:57:48.798260+0000","last_clean":"2024-03-15T12:57:48.798260+0000","last_became_active":"2024-03-15T12:56:21.216868+0000","last_became_peered":"2024-03-15T12:56:21.216868+0000","last_unstale":"2024-03-15T12:57:48.798260+0000","last_undegraded":"2024-03-15T12:57:48.798260+0000","last_fullsized":"2024-03-15T12:57:48.798260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:03:32.668463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836741+0000","last_change":"2024-03-15T12:56:25.443861+0000","last_active":"2024-03-15T12:57:19.836741+0000","last_peered":"2024-03-15T12:57:19.836741+0000","last_clean":"2024-03-15T12:57:19.836741+0000","last_became_active":"2024-03-15T12:56:25.443386+0000","last_became_peered":"2024-03-15T12:56:25.443386+0000","last_unstale":"2024-03-15T12:57:19.836741+0000","last_undegraded":"2024-03-15T12:57:19.836741+0000","last_fullsized":"2024-03-15T12:57:19.836741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:00:23.063425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837030+0000","last_change":"2024-03-15T12:56:20.212451+0000","last_active":"2024-03-15T12:57:19.837030+0000","last_peered":"2024-03-15T12:57:19.837030+0000","last_clean":"2024-03-15T12:57:19.837030+0000","last_became_active":"2024-03-15T12:56:20.212118+0000","last_became_peered":"2024-03-15T12:56:20.212118+0000","last_unstale":"2024-03-15T12:57:19.837030+0000","last_undegraded":"2024-03-15T12:57:19.837030+0000","last_fullsized":"2024-03-15T12:57:19.837030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:24:33.667652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.766138+0000","last_change":"2024-03-15T12:57:20.107434+0000","last_active":"2024-03-15T12:57:48.766138+0000","last_peered":"2024-03-15T12:57:48.766138+0000","last_clean":"2024-03-15T12:57:48.766138+0000","last_became_active":"2024-03-15T12:57:20.107075+0000","last_became_peered":"2024-03-15T12:57:20.107075+0000","last_unstale":"2024-03-15T12:57:48.766138+0000","last_undegraded":"2024-03-15T12:57:48.766138+0000","last_fullsized":"2024-03-15T12:57:48.766138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:24:44.944629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834590+0000","last_change":"2024-03-15T12:56:25.438194+0000","last_active":"2024-03-15T12:57:19.834590+0000","last_peered":"2024-03-15T12:57:19.834590+0000","last_clean":"2024-03-15T12:57:19.834590+0000","last_became_active":"2024-03-15T12:56:25.437769+0000","last_became_peered":"2024-03-15T12:56:25.437769+0000","last_unstale":"2024-03-15T12:57:19.834590+0000","last_undegraded":"2024-03-15T12:57:19.834590+0000","last_fullsized":"2024-03-15T12:57:19.834590+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:29:44.398731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:37.528626+0000","last_change":"2024-03-15T12:56:20.210552+0000","last_active":"2024-03-15T12:58:37.528626+0000","last_peered":"2024-03-15T12:58:37.528626+0000","last_clean":"2024-03-15T12:58:37.528626+0000","last_became_active":"2024-03-15T12:56:20.210191+0000","last_became_peered":"2024-03-15T12:56:20.210191+0000","last_unstale":"2024-03-15T12:58:37.528626+0000","last_undegraded":"2024-03-15T12:58:37.528626+0000","last_fullsized":"2024-03-15T12:58:37.528626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:25:48.225266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.754896+0000","last_change":"2024-03-15T12:56:21.220582+0000","last_active":"2024-03-15T12:57:48.754896+0000","last_peered":"2024-03-15T12:57:48.754896+0000","last_clean":"2024-03-15T12:57:48.754896+0000","last_became_active":"2024-03-15T12:56:21.220212+0000","last_became_peered":"2024-03-15T12:56:21.220212+0000","last_unstale":"2024-03-15T12:57:48.754896+0000","last_undegraded":"2024-03-15T12:57:48.754896+0000","last_fullsized":"2024-03-15T12:57:48.754896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:00:54.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834490+0000","last_change":"2024-03-15T12:56:25.446410+0000","last_active":"2024-03-15T12:57:19.834490+0000","last_peered":"2024-03-15T12:57:19.834490+0000","last_clean":"2024-03-15T12:57:19.834490+0000","last_became_active":"2024-03-15T12:56:25.445554+0000","last_became_peered":"2024-03-15T12:56:25.445554+0000","last_unstale":"2024-03-15T12:57:19.834490+0000","last_undegraded":"2024-03-15T12:57:19.834490+0000","last_fullsized":"2024-03-15T12:57:19.834490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:35:08.049374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.839292+0000","last_change":"2024-03-15T12:56:20.212599+0000","last_active":"2024-03-15T12:57:19.839292+0000","last_peered":"2024-03-15T12:57:19.839292+0000","last_clean":"2024-03-15T12:57:19.839292+0000","last_became_active":"2024-03-15T12:56:20.212289+0000","last_became_peered":"2024-03-15T12:56:20.212289+0000","last_unstale":"2024-03-15T12:57:19.839292+0000","last_undegraded":"2024-03-15T12:57:19.839292+0000","last_fullsized":"2024-03-15T12:57:19.839292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:12:51.036882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.764372+0000","last_change":"2024-03-15T12:56:21.219407+0000","last_active":"2024-03-15T12:57:48.764372+0000","last_peered":"2024-03-15T12:57:48.764372+0000","last_clean":"2024-03-15T12:57:48.764372+0000","last_became_active":"2024-03-15T12:56:21.219035+0000","last_became_peered":"2024-03-15T12:56:21.219035+0000","last_unstale":"2024-03-15T12:57:48.764372+0000","last_undegraded":"2024-03-15T12:57:48.764372+0000","last_fullsized":"2024-03-15T12:57:48.764372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:35:43.129767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.842180+0000","last_change":"2024-03-15T12:57:19.841693+0000","last_active":"2024-03-15T12:57:19.842180+0000","last_peered":"2024-03-15T12:57:19.842180+0000","last_clean":"2024-03-15T12:57:19.842180+0000","last_became_active":"2024-03-15T12:57:19.841275+0000","last_became_peered":"2024-03-15T12:57:19.841275+0000","last_unstale":"2024-03-15T12:57:19.842180+0000","last_undegraded":"2024-03-15T12:57:19.842180+0000","last_fullsized":"2024-03-15T12:57:19.842180+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:41:18.147235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837559+0000","last_change":"2024-03-15T12:56:20.209933+0000","last_active":"2024-03-15T12:57:19.837559+0000","last_peered":"2024-03-15T12:57:19.837559+0000","last_clean":"2024-03-15T12:57:19.837559+0000","last_became_active":"2024-03-15T12:56:20.209047+0000","last_became_peered":"2024-03-15T12:56:20.209047+0000","last_unstale":"2024-03-15T12:57:19.837559+0000","last_undegraded":"2024-03-15T12:57:19.837559+0000","last_fullsized":"2024-03-15T12:57:19.837559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:24:05.551745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782760+0000","last_change":"2024-03-15T12:56:21.215198+0000","last_active":"2024-03-15T12:57:48.782760+0000","last_peered":"2024-03-15T12:57:48.782760+0000","last_clean":"2024-03-15T12:57:48.782760+0000","last_became_active":"2024-03-15T12:56:21.214844+0000","last_became_peered":"2024-03-15T12:56:21.214844+0000","last_unstale":"2024-03-15T12:57:48.782760+0000","last_undegraded":"2024-03-15T12:57:48.782760+0000","last_fullsized":"2024-03-15T12:57:48.782760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:37:08.398317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836412+0000","last_change":"2024-03-15T12:56:25.437047+0000","last_active":"2024-03-15T12:57:19.836412+0000","last_peered":"2024-03-15T12:57:19.836412+0000","last_clean":"2024-03-15T12:57:19.836412+0000","last_became_active":"2024-03-15T12:56:25.436595+0000","last_became_peered":"2024-03-15T12:56:25.436595+0000","last_unstale":"2024-03-15T12:57:19.836412+0000","last_undegraded":"2024-03-15T12:57:19.836412+0000","last_fullsized":"2024-03-15T12:57:19.836412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:59:55.147044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103923+0000","last_change":"2024-03-15T12:56:20.206348+0000","last_active":"2024-03-15T12:57:20.103923+0000","last_peered":"2024-03-15T12:57:20.103923+0000","last_clean":"2024-03-15T12:57:20.103923+0000","last_became_active":"2024-03-15T12:56:20.206013+0000","last_became_peered":"2024-03-15T12:56:20.206013+0000","last_unstale":"2024-03-15T12:57:20.103923+0000","last_undegraded":"2024-03-15T12:57:20.103923+0000","last_fullsized":"2024-03-15T12:57:20.103923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:32:26.625982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782151+0000","last_change":"2024-03-15T12:56:21.216517+0000","last_active":"2024-03-15T12:57:48.782151+0000","last_peered":"2024-03-15T12:57:48.782151+0000","last_clean":"2024-03-15T12:57:48.782151+0000","last_became_active":"2024-03-15T12:56:21.216076+0000","last_became_peered":"2024-03-15T12:56:21.216076+0000","last_unstale":"2024-03-15T12:57:48.782151+0000","last_undegraded":"2024-03-15T12:57:48.782151+0000","last_fullsized":"2024-03-15T12:57:48.782151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:19:35.547273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104273+0000","last_change":"2024-03-15T12:56:25.437367+0000","last_active":"2024-03-15T12:57:20.104273+0000","last_peered":"2024-03-15T12:57:20.104273+0000","last_clean":"2024-03-15T12:57:20.104273+0000","last_became_active":"2024-03-15T12:56:25.436930+0000","last_became_peered":"2024-03-15T12:56:25.436930+0000","last_unstale":"2024-03-15T12:57:20.104273+0000","last_undegraded":"2024-03-15T12:57:20.104273+0000","last_fullsized":"2024-03-15T12:57:20.104273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:53:56.586786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":270,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:39.530740+0000","last_change":"2024-03-15T12:56:20.214067+0000","last_active":"2024-03-15T12:58:39.530740+0000","last_peered":"2024-03-15T12:58:39.530740+0000","last_clean":"2024-03-15T12:58:39.530740+0000","last_became_active":"2024-03-15T12:56:20.213696+0000","last_became_peered":"2024-03-15T12:56:20.213696+0000","last_unstale":"2024-03-15T12:58:39.530740+0000","last_undegraded":"2024-03-15T12:58:39.530740+0000","last_fullsized":"2024-03-15T12:58:39.530740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:35:15.343660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":237,"num_read_kb":237,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.797642+0000","last_change":"2024-03-15T12:56:21.211106+0000","last_active":"2024-03-15T12:57:48.797642+0000","last_peered":"2024-03-15T12:57:48.797642+0000","last_clean":"2024-03-15T12:57:48.797642+0000","last_became_active":"2024-03-15T12:56:21.210756+0000","last_became_peered":"2024-03-15T12:56:21.210756+0000","last_unstale":"2024-03-15T12:57:48.797642+0000","last_undegraded":"2024-03-15T12:57:48.797642+0000","last_fullsized":"2024-03-15T12:57:48.797642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:13:25.232077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.833697+0000","last_change":"2024-03-15T12:56:25.441364+0000","last_active":"2024-03-15T12:57:19.833697+0000","last_peered":"2024-03-15T12:57:19.833697+0000","last_clean":"2024-03-15T12:57:19.833697+0000","last_became_active":"2024-03-15T12:56:25.440951+0000","last_became_peered":"2024-03-15T12:56:25.440951+0000","last_unstale":"2024-03-15T12:57:19.833697+0000","last_undegraded":"2024-03-15T12:57:19.833697+0000","last_fullsized":"2024-03-15T12:57:19.833697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:32:28.651856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104583+0000","last_change":"2024-03-15T12:56:20.208633+0000","last_active":"2024-03-15T12:57:20.104583+0000","last_peered":"2024-03-15T12:57:20.104583+0000","last_clean":"2024-03-15T12:57:20.104583+0000","last_became_active":"2024-03-15T12:56:20.208280+0000","last_became_peered":"2024-03-15T12:56:20.208280+0000","last_unstale":"2024-03-15T12:57:20.104583+0000","last_undegraded":"2024-03-15T12:57:20.104583+0000","last_fullsized":"2024-03-15T12:57:20.104583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:00:15.274090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.785458+0000","last_change":"2024-03-15T12:56:21.218998+0000","last_active":"2024-03-15T12:57:48.785458+0000","last_peered":"2024-03-15T12:57:48.785458+0000","last_clean":"2024-03-15T12:57:48.785458+0000","last_became_active":"2024-03-15T12:56:21.218617+0000","last_became_peered":"2024-03-15T12:56:21.218617+0000","last_unstale":"2024-03-15T12:57:48.785458+0000","last_undegraded":"2024-03-15T12:57:48.785458+0000","last_fullsized":"2024-03-15T12:57:48.785458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:20:25.496204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837212+0000","last_change":"2024-03-15T12:56:25.438969+0000","last_active":"2024-03-15T12:57:19.837212+0000","last_peered":"2024-03-15T12:57:19.837212+0000","last_clean":"2024-03-15T12:57:19.837212+0000","last_became_active":"2024-03-15T12:56:25.438603+0000","last_became_peered":"2024-03-15T12:56:25.438603+0000","last_unstale":"2024-03-15T12:57:19.837212+0000","last_undegraded":"2024-03-15T12:57:19.837212+0000","last_fullsized":"2024-03-15T12:57:19.837212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:54:23.992253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104585+0000","last_change":"2024-03-15T12:56:20.215152+0000","last_active":"2024-03-15T12:57:20.104585+0000","last_peered":"2024-03-15T12:57:20.104585+0000","last_clean":"2024-03-15T12:57:20.104585+0000","last_became_active":"2024-03-15T12:56:20.214762+0000","last_became_peered":"2024-03-15T12:56:20.214762+0000","last_unstale":"2024-03-15T12:57:20.104585+0000","last_undegraded":"2024-03-15T12:57:20.104585+0000","last_fullsized":"2024-03-15T12:57:20.104585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:38:45.414984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.790199+0000","last_change":"2024-03-15T12:56:21.214005+0000","last_active":"2024-03-15T12:57:48.790199+0000","last_peered":"2024-03-15T12:57:48.790199+0000","last_clean":"2024-03-15T12:57:48.790199+0000","last_became_active":"2024-03-15T12:56:21.213672+0000","last_became_peered":"2024-03-15T12:56:21.213672+0000","last_unstale":"2024-03-15T12:57:48.790199+0000","last_undegraded":"2024-03-15T12:57:48.790199+0000","last_fullsized":"2024-03-15T12:57:48.790199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:13:09.384245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.833759+0000","last_change":"2024-03-15T12:56:25.443145+0000","last_active":"2024-03-15T12:57:19.833759+0000","last_peered":"2024-03-15T12:57:19.833759+0000","last_clean":"2024-03-15T12:57:19.833759+0000","last_became_active":"2024-03-15T12:56:25.442755+0000","last_became_peered":"2024-03-15T12:56:25.442755+0000","last_unstale":"2024-03-15T12:57:19.833759+0000","last_undegraded":"2024-03-15T12:57:19.833759+0000","last_fullsized":"2024-03-15T12:57:19.833759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:14:47.201919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'40","reported_seq":318,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:39.531730+0000","last_change":"2024-03-15T12:56:20.215063+0000","last_active":"2024-03-15T12:58:39.531730+0000","last_peered":"2024-03-15T12:58:39.531730+0000","last_clean":"2024-03-15T12:58:39.531730+0000","last_became_active":"2024-03-15T12:56:20.214742+0000","last_became_peered":"2024-03-15T12:56:20.214742+0000","last_unstale":"2024-03-15T12:58:39.531730+0000","last_undegraded":"2024-03-15T12:58:39.531730+0000","last_fullsized":"2024-03-15T12:58:39.531730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:38:12.975035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69742,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":194,"num_write":42,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.773641+0000","last_change":"2024-03-15T12:56:21.213519+0000","last_active":"2024-03-15T12:57:48.773641+0000","last_peered":"2024-03-15T12:57:48.773641+0000","last_clean":"2024-03-15T12:57:48.773641+0000","last_became_active":"2024-03-15T12:56:21.213198+0000","last_became_peered":"2024-03-15T12:56:21.213198+0000","last_unstale":"2024-03-15T12:57:48.773641+0000","last_undegraded":"2024-03-15T12:57:48.773641+0000","last_fullsized":"2024-03-15T12:57:48.773641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:22:13.037068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106198+0000","last_change":"2024-03-15T12:57:20.106198+0000","last_active":"2024-03-15T12:57:20.106198+0000","last_peered":"2024-03-15T12:57:20.106198+0000","last_clean":"2024-03-15T12:57:20.106198+0000","last_became_active":"2024-03-15T12:57:20.105649+0000","last_became_peered":"2024-03-15T12:57:20.105649+0000","last_unstale":"2024-03-15T12:57:20.106198+0000","last_undegraded":"2024-03-15T12:57:20.106198+0000","last_fullsized":"2024-03-15T12:57:20.106198+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:44:46.615091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834609+0000","last_change":"2024-03-15T12:56:20.216124+0000","last_active":"2024-03-15T12:57:19.834609+0000","last_peered":"2024-03-15T12:57:19.834609+0000","last_clean":"2024-03-15T12:57:19.834609+0000","last_became_active":"2024-03-15T12:56:20.215754+0000","last_became_peered":"2024-03-15T12:56:20.215754+0000","last_unstale":"2024-03-15T12:57:19.834609+0000","last_undegraded":"2024-03-15T12:57:19.834609+0000","last_fullsized":"2024-03-15T12:57:19.834609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:11:44.403503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.781601+0000","last_change":"2024-03-15T12:56:21.214775+0000","last_active":"2024-03-15T12:57:48.781601+0000","last_peered":"2024-03-15T12:57:48.781601+0000","last_clean":"2024-03-15T12:57:48.781601+0000","last_became_active":"2024-03-15T12:56:21.214445+0000","last_became_peered":"2024-03-15T12:56:21.214445+0000","last_unstale":"2024-03-15T12:57:48.781601+0000","last_undegraded":"2024-03-15T12:57:48.781601+0000","last_fullsized":"2024-03-15T12:57:48.781601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:23:24.045670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.107668+0000","last_change":"2024-03-15T12:57:20.107668+0000","last_active":"2024-03-15T12:57:20.107668+0000","last_peered":"2024-03-15T12:57:20.107668+0000","last_clean":"2024-03-15T12:57:20.107668+0000","last_became_active":"2024-03-15T12:57:20.107259+0000","last_became_peered":"2024-03-15T12:57:20.107259+0000","last_unstale":"2024-03-15T12:57:20.107668+0000","last_undegraded":"2024-03-15T12:57:20.107668+0000","last_fullsized":"2024-03-15T12:57:20.107668+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:07:37.244869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104267+0000","last_change":"2024-03-15T12:56:20.217191+0000","last_active":"2024-03-15T12:57:20.104267+0000","last_peered":"2024-03-15T12:57:20.104267+0000","last_clean":"2024-03-15T12:57:20.104267+0000","last_became_active":"2024-03-15T12:56:20.216632+0000","last_became_peered":"2024-03-15T12:56:20.216632+0000","last_unstale":"2024-03-15T12:57:20.104267+0000","last_undegraded":"2024-03-15T12:57:20.104267+0000","last_fullsized":"2024-03-15T12:57:20.104267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:16:51.187920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784344+0000","last_change":"2024-03-15T12:56:21.218650+0000","last_active":"2024-03-15T12:57:48.784344+0000","last_peered":"2024-03-15T12:57:48.784344+0000","last_clean":"2024-03-15T12:57:48.784344+0000","last_became_active":"2024-03-15T12:56:21.218329+0000","last_became_peered":"2024-03-15T12:56:21.218329+0000","last_unstale":"2024-03-15T12:57:48.784344+0000","last_undegraded":"2024-03-15T12:57:48.784344+0000","last_fullsized":"2024-03-15T12:57:48.784344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:10:45.632248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834415+0000","last_change":"2024-03-15T12:56:25.447982+0000","last_active":"2024-03-15T12:57:19.834415+0000","last_peered":"2024-03-15T12:57:19.834415+0000","last_clean":"2024-03-15T12:57:19.834415+0000","last_became_active":"2024-03-15T12:56:25.447514+0000","last_became_peered":"2024-03-15T12:56:25.447514+0000","last_unstale":"2024-03-15T12:57:19.834415+0000","last_undegraded":"2024-03-15T12:57:19.834415+0000","last_fullsized":"2024-03-15T12:57:19.834415+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:16:52.567619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'8","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:37.576101+0000","last_change":"2024-03-15T12:56:20.209778+0000","last_active":"2024-03-15T12:58:37.576101+0000","last_peered":"2024-03-15T12:58:37.576101+0000","last_clean":"2024-03-15T12:58:37.576101+0000","last_became_active":"2024-03-15T12:56:20.209311+0000","last_became_peered":"2024-03-15T12:56:20.209311+0000","last_unstale":"2024-03-15T12:58:37.576101+0000","last_undegraded":"2024-03-15T12:58:37.576101+0000","last_fullsized":"2024-03-15T12:58:37.576101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:05:21.930210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.778891+0000","last_change":"2024-03-15T12:56:21.215576+0000","last_active":"2024-03-15T12:57:48.778891+0000","last_peered":"2024-03-15T12:57:48.778891+0000","last_clean":"2024-03-15T12:57:48.778891+0000","last_became_active":"2024-03-15T12:56:21.215261+0000","last_became_peered":"2024-03-15T12:56:21.215261+0000","last_unstale":"2024-03-15T12:57:48.778891+0000","last_undegraded":"2024-03-15T12:57:48.778891+0000","last_fullsized":"2024-03-15T12:57:48.778891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:35:15.461554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837297+0000","last_change":"2024-03-15T12:56:25.445559+0000","last_active":"2024-03-15T12:57:19.837297+0000","last_peered":"2024-03-15T12:57:19.837297+0000","last_clean":"2024-03-15T12:57:19.837297+0000","last_became_active":"2024-03-15T12:56:25.445092+0000","last_became_peered":"2024-03-15T12:56:25.445092+0000","last_unstale":"2024-03-15T12:57:19.837297+0000","last_undegraded":"2024-03-15T12:57:19.837297+0000","last_fullsized":"2024-03-15T12:57:19.837297+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:48:30.558243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836745+0000","last_change":"2024-03-15T12:56:20.214853+0000","last_active":"2024-03-15T12:57:19.836745+0000","last_peered":"2024-03-15T12:57:19.836745+0000","last_clean":"2024-03-15T12:57:19.836745+0000","last_became_active":"2024-03-15T12:56:20.214510+0000","last_became_peered":"2024-03-15T12:56:20.214510+0000","last_unstale":"2024-03-15T12:57:19.836745+0000","last_undegraded":"2024-03-15T12:57:19.836745+0000","last_fullsized":"2024-03-15T12:57:19.836745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:25:39.316428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798162+0000","last_change":"2024-03-15T12:56:21.216511+0000","last_active":"2024-03-15T12:57:48.798162+0000","last_peered":"2024-03-15T12:57:48.798162+0000","last_clean":"2024-03-15T12:57:48.798162+0000","last_became_active":"2024-03-15T12:56:21.216175+0000","last_became_peered":"2024-03-15T12:56:21.216175+0000","last_unstale":"2024-03-15T12:57:48.798162+0000","last_undegraded":"2024-03-15T12:57:48.798162+0000","last_fullsized":"2024-03-15T12:57:48.798162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:16:56.665841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104042+0000","last_change":"2024-03-15T12:56:25.446009+0000","last_active":"2024-03-15T12:57:20.104042+0000","last_peered":"2024-03-15T12:57:20.104042+0000","last_clean":"2024-03-15T12:57:20.104042+0000","last_became_active":"2024-03-15T12:56:25.445474+0000","last_became_peered":"2024-03-15T12:56:25.445474+0000","last_unstale":"2024-03-15T12:57:20.104042+0000","last_undegraded":"2024-03-15T12:57:20.104042+0000","last_fullsized":"2024-03-15T12:57:20.104042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:53:46.856905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:36.411188+0000","last_change":"2024-03-15T12:56:20.210146+0000","last_active":"2024-03-15T12:58:36.411188+0000","last_peered":"2024-03-15T12:58:36.411188+0000","last_clean":"2024-03-15T12:58:36.411188+0000","last_became_active":"2024-03-15T12:56:20.209769+0000","last_became_peered":"2024-03-15T12:56:20.209769+0000","last_unstale":"2024-03-15T12:58:36.411188+0000","last_undegraded":"2024-03-15T12:58:36.411188+0000","last_fullsized":"2024-03-15T12:58:36.411188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:47:32.505257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.770795+0000","last_change":"2024-03-15T12:56:21.214826+0000","last_active":"2024-03-15T12:57:48.770795+0000","last_peered":"2024-03-15T12:57:48.770795+0000","last_clean":"2024-03-15T12:57:48.770795+0000","last_became_active":"2024-03-15T12:56:21.214160+0000","last_became_peered":"2024-03-15T12:56:21.214160+0000","last_unstale":"2024-03-15T12:57:48.770795+0000","last_undegraded":"2024-03-15T12:57:48.770795+0000","last_fullsized":"2024-03-15T12:57:48.770795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:23:17.800522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837088+0000","last_change":"2024-03-15T12:56:25.443496+0000","last_active":"2024-03-15T12:57:19.837088+0000","last_peered":"2024-03-15T12:57:19.837088+0000","last_clean":"2024-03-15T12:57:19.837088+0000","last_became_active":"2024-03-15T12:56:25.442796+0000","last_became_peered":"2024-03-15T12:56:25.442796+0000","last_unstale":"2024-03-15T12:57:19.837088+0000","last_undegraded":"2024-03-15T12:57:19.837088+0000","last_fullsized":"2024-03-15T12:57:19.837088+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:35:23.892693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.841026+0000","last_change":"2024-03-15T12:57:19.840452+0000","last_active":"2024-03-15T12:57:19.841026+0000","last_peered":"2024-03-15T12:57:19.841026+0000","last_clean":"2024-03-15T12:57:19.841026+0000","last_became_active":"2024-03-15T12:57:19.839977+0000","last_became_peered":"2024-03-15T12:57:19.839977+0000","last_unstale":"2024-03-15T12:57:19.841026+0000","last_undegraded":"2024-03-15T12:57:19.841026+0000","last_fullsized":"2024-03-15T12:57:19.841026+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:59:25.026455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.777893+0000","last_change":"2024-03-15T12:56:21.216393+0000","last_active":"2024-03-15T12:57:48.777893+0000","last_peered":"2024-03-15T12:57:48.777893+0000","last_clean":"2024-03-15T12:57:48.777893+0000","last_became_active":"2024-03-15T12:56:21.216043+0000","last_became_peered":"2024-03-15T12:56:21.216043+0000","last_unstale":"2024-03-15T12:57:48.777893+0000","last_undegraded":"2024-03-15T12:57:48.777893+0000","last_fullsized":"2024-03-15T12:57:48.777893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:05:49.843726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837389+0000","last_change":"2024-03-15T12:56:25.443854+0000","last_active":"2024-03-15T12:57:19.837389+0000","last_peered":"2024-03-15T12:57:19.837389+0000","last_clean":"2024-03-15T12:57:19.837389+0000","last_became_active":"2024-03-15T12:56:25.443422+0000","last_became_peered":"2024-03-15T12:56:25.443422+0000","last_unstale":"2024-03-15T12:57:19.837389+0000","last_undegraded":"2024-03-15T12:57:19.837389+0000","last_fullsized":"2024-03-15T12:57:19.837389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:52:18.999161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:36.575217+0000","last_change":"2024-03-15T12:56:20.211493+0000","last_active":"2024-03-15T12:58:36.575217+0000","last_peered":"2024-03-15T12:58:36.575217+0000","last_clean":"2024-03-15T12:58:36.575217+0000","last_became_active":"2024-03-15T12:56:20.211154+0000","last_became_peered":"2024-03-15T12:56:20.211154+0000","last_unstale":"2024-03-15T12:58:36.575217+0000","last_undegraded":"2024-03-15T12:58:36.575217+0000","last_fullsized":"2024-03-15T12:58:36.575217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:26:28.026899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.784429+0000","last_change":"2024-03-15T12:56:21.215999+0000","last_active":"2024-03-15T12:57:48.784429+0000","last_peered":"2024-03-15T12:57:48.784429+0000","last_clean":"2024-03-15T12:57:48.784429+0000","last_became_active":"2024-03-15T12:56:21.215657+0000","last_became_peered":"2024-03-15T12:56:21.215657+0000","last_unstale":"2024-03-15T12:57:48.784429+0000","last_undegraded":"2024-03-15T12:57:48.784429+0000","last_fullsized":"2024-03-15T12:57:48.784429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:47:59.901886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837294+0000","last_change":"2024-03-15T12:56:25.442246+0000","last_active":"2024-03-15T12:57:19.837294+0000","last_peered":"2024-03-15T12:57:19.837294+0000","last_clean":"2024-03-15T12:57:19.837294+0000","last_became_active":"2024-03-15T12:56:25.441755+0000","last_became_peered":"2024-03-15T12:56:25.441755+0000","last_unstale":"2024-03-15T12:57:19.837294+0000","last_undegraded":"2024-03-15T12:57:19.837294+0000","last_fullsized":"2024-03-15T12:57:19.837294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:47:36.037864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.753452+0000","last_change":"2024-03-15T12:56:21.215746+0000","last_active":"2024-03-15T12:57:48.753452+0000","last_peered":"2024-03-15T12:57:48.753452+0000","last_clean":"2024-03-15T12:57:48.753452+0000","last_became_active":"2024-03-15T12:56:21.215424+0000","last_became_peered":"2024-03-15T12:56:21.215424+0000","last_unstale":"2024-03-15T12:57:48.753452+0000","last_undegraded":"2024-03-15T12:57:48.753452+0000","last_fullsized":"2024-03-15T12:57:48.753452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:01:15.498119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'7","reported_seq":153,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:38.576866+0000","last_change":"2024-03-15T12:56:20.219705+0000","last_active":"2024-03-15T12:58:38.576866+0000","last_peered":"2024-03-15T12:58:38.576866+0000","last_clean":"2024-03-15T12:58:38.576866+0000","last_became_active":"2024-03-15T12:56:20.219328+0000","last_became_peered":"2024-03-15T12:56:20.219328+0000","last_unstale":"2024-03-15T12:58:38.576866+0000","last_undegraded":"2024-03-15T12:58:38.576866+0000","last_fullsized":"2024-03-15T12:58:38.576866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:47:52.171116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104879+0000","last_change":"2024-03-15T12:56:25.443094+0000","last_active":"2024-03-15T12:57:20.104879+0000","last_peered":"2024-03-15T12:57:20.104879+0000","last_clean":"2024-03-15T12:57:20.104879+0000","last_became_active":"2024-03-15T12:56:25.442708+0000","last_became_peered":"2024-03-15T12:56:25.442708+0000","last_unstale":"2024-03-15T12:57:20.104879+0000","last_undegraded":"2024-03-15T12:57:20.104879+0000","last_fullsized":"2024-03-15T12:57:20.104879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:03:07.769836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.788423+0000","last_change":"2024-03-15T12:56:21.223937+0000","last_active":"2024-03-15T12:57:48.788423+0000","last_peered":"2024-03-15T12:57:48.788423+0000","last_clean":"2024-03-15T12:57:48.788423+0000","last_became_active":"2024-03-15T12:56:21.223583+0000","last_became_peered":"2024-03-15T12:56:21.223583+0000","last_unstale":"2024-03-15T12:57:48.788423+0000","last_undegraded":"2024-03-15T12:57:48.788423+0000","last_fullsized":"2024-03-15T12:57:48.788423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:15:34.287445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837150+0000","last_change":"2024-03-15T12:56:20.210812+0000","last_active":"2024-03-15T12:57:19.837150+0000","last_peered":"2024-03-15T12:57:19.837150+0000","last_clean":"2024-03-15T12:57:19.837150+0000","last_became_active":"2024-03-15T12:56:20.210457+0000","last_became_peered":"2024-03-15T12:56:20.210457+0000","last_unstale":"2024-03-15T12:57:19.837150+0000","last_undegraded":"2024-03-15T12:57:19.837150+0000","last_fullsized":"2024-03-15T12:57:19.837150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:04:37.371637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":358,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104758+0000","last_change":"2024-03-15T12:55:08.392548+0000","last_active":"2024-03-15T12:57:20.104758+0000","last_peered":"2024-03-15T12:57:20.104758+0000","last_clean":"2024-03-15T12:57:20.104758+0000","last_became_active":"2024-03-15T12:55:08.391972+0000","last_became_peered":"2024-03-15T12:55:08.391972+0000","last_unstale":"2024-03-15T12:57:20.104758+0000","last_undegraded":"2024-03-15T12:57:20.104758+0000","last_fullsized":"2024-03-15T12:57:20.104758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:51:47.093558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:51:47.093558+0000","last_clean_scrub_stamp":"2024-03-15T12:51:47.093558+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:30:57.264886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104929+0000","last_change":"2024-03-15T12:56:25.443212+0000","last_active":"2024-03-15T12:57:20.104929+0000","last_peered":"2024-03-15T12:57:20.104929+0000","last_clean":"2024-03-15T12:57:20.104929+0000","last_became_active":"2024-03-15T12:56:25.442787+0000","last_became_peered":"2024-03-15T12:56:25.442787+0000","last_unstale":"2024-03-15T12:57:20.104929+0000","last_undegraded":"2024-03-15T12:57:20.104929+0000","last_fullsized":"2024-03-15T12:57:20.104929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:59:32.244953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.775951+0000","last_change":"2024-03-15T12:57:19.839935+0000","last_active":"2024-03-15T12:57:48.775951+0000","last_peered":"2024-03-15T12:57:48.775951+0000","last_clean":"2024-03-15T12:57:48.775951+0000","last_became_active":"2024-03-15T12:57:19.839546+0000","last_became_peered":"2024-03-15T12:57:19.839546+0000","last_unstale":"2024-03-15T12:57:48.775951+0000","last_undegraded":"2024-03-15T12:57:48.775951+0000","last_fullsized":"2024-03-15T12:57:48.775951+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:05:15.544159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104546+0000","last_change":"2024-03-15T12:56:20.210466+0000","last_active":"2024-03-15T12:57:20.104546+0000","last_peered":"2024-03-15T12:57:20.104546+0000","last_clean":"2024-03-15T12:57:20.104546+0000","last_became_active":"2024-03-15T12:56:20.210157+0000","last_became_peered":"2024-03-15T12:56:20.210157+0000","last_unstale":"2024-03-15T12:57:20.104546+0000","last_undegraded":"2024-03-15T12:57:20.104546+0000","last_fullsized":"2024-03-15T12:57:20.104546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:58:03.335826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.838324+0000","last_change":"2024-03-15T12:56:25.432641+0000","last_active":"2024-03-15T12:57:19.838324+0000","last_peered":"2024-03-15T12:57:19.838324+0000","last_clean":"2024-03-15T12:57:19.838324+0000","last_became_active":"2024-03-15T12:56:25.432212+0000","last_became_peered":"2024-03-15T12:56:25.432212+0000","last_unstale":"2024-03-15T12:57:19.838324+0000","last_undegraded":"2024-03-15T12:57:19.838324+0000","last_fullsized":"2024-03-15T12:57:19.838324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:40:20.447886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.846534+0000","last_change":"2024-03-15T12:57:19.846534+0000","last_active":"2024-03-15T12:57:19.846534+0000","last_peered":"2024-03-15T12:57:19.846534+0000","last_clean":"2024-03-15T12:57:19.846534+0000","last_became_active":"2024-03-15T12:57:19.841008+0000","last_became_peered":"2024-03-15T12:57:19.841008+0000","last_unstale":"2024-03-15T12:57:19.846534+0000","last_undegraded":"2024-03-15T12:57:19.846534+0000","last_fullsized":"2024-03-15T12:57:19.846534+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:46:17.129901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.778482+0000","last_change":"2024-03-15T12:56:21.207577+0000","last_active":"2024-03-15T12:57:48.778482+0000","last_peered":"2024-03-15T12:57:48.778482+0000","last_clean":"2024-03-15T12:57:48.778482+0000","last_became_active":"2024-03-15T12:56:21.207220+0000","last_became_peered":"2024-03-15T12:56:21.207220+0000","last_unstale":"2024-03-15T12:57:48.778482+0000","last_undegraded":"2024-03-15T12:57:48.778482+0000","last_fullsized":"2024-03-15T12:57:48.778482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:42:12.104220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104420+0000","last_change":"2024-03-15T12:56:25.439762+0000","last_active":"2024-03-15T12:57:20.104420+0000","last_peered":"2024-03-15T12:57:20.104420+0000","last_clean":"2024-03-15T12:57:20.104420+0000","last_became_active":"2024-03-15T12:56:25.439366+0000","last_became_peered":"2024-03-15T12:56:25.439366+0000","last_unstale":"2024-03-15T12:57:20.104420+0000","last_undegraded":"2024-03-15T12:57:20.104420+0000","last_fullsized":"2024-03-15T12:57:20.104420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:36:29.638449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'16","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:38.417721+0000","last_change":"2024-03-15T12:56:20.218438+0000","last_active":"2024-03-15T12:58:38.417721+0000","last_peered":"2024-03-15T12:58:38.417721+0000","last_clean":"2024-03-15T12:58:38.417721+0000","last_became_active":"2024-03-15T12:56:20.218071+0000","last_became_peered":"2024-03-15T12:56:20.218071+0000","last_unstale":"2024-03-15T12:58:38.417721+0000","last_undegraded":"2024-03-15T12:58:38.417721+0000","last_fullsized":"2024-03-15T12:58:38.417721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T17:24:16.295781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.783607+0000","last_change":"2024-03-15T12:56:21.218345+0000","last_active":"2024-03-15T12:57:48.783607+0000","last_peered":"2024-03-15T12:57:48.783607+0000","last_clean":"2024-03-15T12:57:48.783607+0000","last_became_active":"2024-03-15T12:56:21.216534+0000","last_became_peered":"2024-03-15T12:56:21.216534+0000","last_unstale":"2024-03-15T12:57:48.783607+0000","last_undegraded":"2024-03-15T12:57:48.783607+0000","last_fullsized":"2024-03-15T12:57:48.783607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:56:18.977509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.834310+0000","last_change":"2024-03-15T12:56:25.435714+0000","last_active":"2024-03-15T12:57:19.834310+0000","last_peered":"2024-03-15T12:57:19.834310+0000","last_clean":"2024-03-15T12:57:19.834310+0000","last_became_active":"2024-03-15T12:56:25.435315+0000","last_became_peered":"2024-03-15T12:56:25.435315+0000","last_unstale":"2024-03-15T12:57:19.834310+0000","last_undegraded":"2024-03-15T12:57:19.834310+0000","last_fullsized":"2024-03-15T12:57:19.834310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:10:03.509078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837248+0000","last_change":"2024-03-15T12:56:20.209356+0000","last_active":"2024-03-15T12:57:19.837248+0000","last_peered":"2024-03-15T12:57:19.837248+0000","last_clean":"2024-03-15T12:57:19.837248+0000","last_became_active":"2024-03-15T12:56:20.208995+0000","last_became_peered":"2024-03-15T12:56:20.208995+0000","last_unstale":"2024-03-15T12:57:19.837248+0000","last_undegraded":"2024-03-15T12:57:19.837248+0000","last_fullsized":"2024-03-15T12:57:19.837248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:14:08.554567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.783714+0000","last_change":"2024-03-15T12:56:21.216315+0000","last_active":"2024-03-15T12:57:48.783714+0000","last_peered":"2024-03-15T12:57:48.783714+0000","last_clean":"2024-03-15T12:57:48.783714+0000","last_became_active":"2024-03-15T12:56:21.215943+0000","last_became_peered":"2024-03-15T12:56:21.215943+0000","last_unstale":"2024-03-15T12:57:48.783714+0000","last_undegraded":"2024-03-15T12:57:48.783714+0000","last_fullsized":"2024-03-15T12:57:48.783714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:47:29.913048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836875+0000","last_change":"2024-03-15T12:56:25.437200+0000","last_active":"2024-03-15T12:57:19.836875+0000","last_peered":"2024-03-15T12:57:19.836875+0000","last_clean":"2024-03-15T12:57:19.836875+0000","last_became_active":"2024-03-15T12:56:25.436786+0000","last_became_peered":"2024-03-15T12:56:25.436786+0000","last_unstale":"2024-03-15T12:57:19.836875+0000","last_undegraded":"2024-03-15T12:57:19.836875+0000","last_fullsized":"2024-03-15T12:57:19.836875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:12:00.286749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106367+0000","last_change":"2024-03-15T12:56:20.202240+0000","last_active":"2024-03-15T12:57:20.106367+0000","last_peered":"2024-03-15T12:57:20.106367+0000","last_clean":"2024-03-15T12:57:20.106367+0000","last_became_active":"2024-03-15T12:56:20.201136+0000","last_became_peered":"2024-03-15T12:56:20.201136+0000","last_unstale":"2024-03-15T12:57:20.106367+0000","last_undegraded":"2024-03-15T12:57:20.106367+0000","last_fullsized":"2024-03-15T12:57:20.106367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:30:35.901237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.759652+0000","last_change":"2024-03-15T12:56:21.212095+0000","last_active":"2024-03-15T12:57:48.759652+0000","last_peered":"2024-03-15T12:57:48.759652+0000","last_clean":"2024-03-15T12:57:48.759652+0000","last_became_active":"2024-03-15T12:56:21.211744+0000","last_became_peered":"2024-03-15T12:56:21.211744+0000","last_unstale":"2024-03-15T12:57:48.759652+0000","last_undegraded":"2024-03-15T12:57:48.759652+0000","last_fullsized":"2024-03-15T12:57:48.759652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:27:58.412513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.106031+0000","last_change":"2024-03-15T12:57:20.106031+0000","last_active":"2024-03-15T12:57:20.106031+0000","last_peered":"2024-03-15T12:57:20.106031+0000","last_clean":"2024-03-15T12:57:20.106031+0000","last_became_active":"2024-03-15T12:57:20.105499+0000","last_became_peered":"2024-03-15T12:57:20.105499+0000","last_unstale":"2024-03-15T12:57:20.106031+0000","last_undegraded":"2024-03-15T12:57:20.106031+0000","last_fullsized":"2024-03-15T12:57:20.106031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:17:08.138164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104141+0000","last_change":"2024-03-15T12:56:20.217319+0000","last_active":"2024-03-15T12:57:20.104141+0000","last_peered":"2024-03-15T12:57:20.104141+0000","last_clean":"2024-03-15T12:57:20.104141+0000","last_became_active":"2024-03-15T12:56:20.215450+0000","last_became_peered":"2024-03-15T12:56:20.215450+0000","last_unstale":"2024-03-15T12:57:20.104141+0000","last_undegraded":"2024-03-15T12:57:20.104141+0000","last_fullsized":"2024-03-15T12:57:20.104141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:07:09.085860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780239+0000","last_change":"2024-03-15T12:56:21.209360+0000","last_active":"2024-03-15T12:57:48.780239+0000","last_peered":"2024-03-15T12:57:48.780239+0000","last_clean":"2024-03-15T12:57:48.780239+0000","last_became_active":"2024-03-15T12:56:21.208987+0000","last_became_peered":"2024-03-15T12:56:21.208987+0000","last_unstale":"2024-03-15T12:57:48.780239+0000","last_undegraded":"2024-03-15T12:57:48.780239+0000","last_fullsized":"2024-03-15T12:57:48.780239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:19:13.353400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836774+0000","last_change":"2024-03-15T12:56:25.439173+0000","last_active":"2024-03-15T12:57:19.836774+0000","last_peered":"2024-03-15T12:57:19.836774+0000","last_clean":"2024-03-15T12:57:19.836774+0000","last_became_active":"2024-03-15T12:56:25.438812+0000","last_became_peered":"2024-03-15T12:56:25.438812+0000","last_unstale":"2024-03-15T12:57:19.836774+0000","last_undegraded":"2024-03-15T12:57:19.836774+0000","last_fullsized":"2024-03-15T12:57:19.836774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:10:39.107190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:35.836940+0000","last_change":"2024-03-15T12:56:20.211861+0000","last_active":"2024-03-15T12:58:35.836940+0000","last_peered":"2024-03-15T12:58:35.836940+0000","last_clean":"2024-03-15T12:58:35.836940+0000","last_became_active":"2024-03-15T12:56:20.211481+0000","last_became_peered":"2024-03-15T12:56:20.211481+0000","last_unstale":"2024-03-15T12:58:35.836940+0000","last_undegraded":"2024-03-15T12:58:35.836940+0000","last_fullsized":"2024-03-15T12:58:35.836940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:12:45.558423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.783692+0000","last_change":"2024-03-15T12:56:21.216058+0000","last_active":"2024-03-15T12:57:48.783692+0000","last_peered":"2024-03-15T12:57:48.783692+0000","last_clean":"2024-03-15T12:57:48.783692+0000","last_became_active":"2024-03-15T12:56:21.215706+0000","last_became_peered":"2024-03-15T12:56:21.215706+0000","last_unstale":"2024-03-15T12:57:48.783692+0000","last_undegraded":"2024-03-15T12:57:48.783692+0000","last_fullsized":"2024-03-15T12:57:48.783692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:55:02.132976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836966+0000","last_change":"2024-03-15T12:56:25.444134+0000","last_active":"2024-03-15T12:57:19.836966+0000","last_peered":"2024-03-15T12:57:19.836966+0000","last_clean":"2024-03-15T12:57:19.836966+0000","last_became_active":"2024-03-15T12:56:25.443452+0000","last_became_peered":"2024-03-15T12:56:25.443452+0000","last_unstale":"2024-03-15T12:57:19.836966+0000","last_undegraded":"2024-03-15T12:57:19.836966+0000","last_fullsized":"2024-03-15T12:57:19.836966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:06:39.660016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836785+0000","last_change":"2024-03-15T12:56:20.214700+0000","last_active":"2024-03-15T12:57:19.836785+0000","last_peered":"2024-03-15T12:57:19.836785+0000","last_clean":"2024-03-15T12:57:19.836785+0000","last_became_active":"2024-03-15T12:56:20.214348+0000","last_became_peered":"2024-03-15T12:56:20.214348+0000","last_unstale":"2024-03-15T12:57:19.836785+0000","last_undegraded":"2024-03-15T12:57:19.836785+0000","last_fullsized":"2024-03-15T12:57:19.836785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:24:37.551182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798936+0000","last_change":"2024-03-15T12:56:21.220746+0000","last_active":"2024-03-15T12:57:48.798936+0000","last_peered":"2024-03-15T12:57:48.798936+0000","last_clean":"2024-03-15T12:57:48.798936+0000","last_became_active":"2024-03-15T12:56:21.220445+0000","last_became_peered":"2024-03-15T12:56:21.220445+0000","last_unstale":"2024-03-15T12:57:48.798936+0000","last_undegraded":"2024-03-15T12:57:48.798936+0000","last_fullsized":"2024-03-15T12:57:48.798936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:58:44.119476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.104747+0000","last_change":"2024-03-15T12:56:25.437611+0000","last_active":"2024-03-15T12:57:20.104747+0000","last_peered":"2024-03-15T12:57:20.104747+0000","last_clean":"2024-03-15T12:57:20.104747+0000","last_became_active":"2024-03-15T12:56:25.437195+0000","last_became_peered":"2024-03-15T12:56:25.437195+0000","last_unstale":"2024-03-15T12:57:20.104747+0000","last_undegraded":"2024-03-15T12:57:20.104747+0000","last_fullsized":"2024-03-15T12:57:20.104747+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:26:43.558477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":156,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:58:39.603492+0000","last_change":"2024-03-15T12:56:20.208584+0000","last_active":"2024-03-15T12:58:39.603492+0000","last_peered":"2024-03-15T12:58:39.603492+0000","last_clean":"2024-03-15T12:58:39.603492+0000","last_became_active":"2024-03-15T12:56:20.208155+0000","last_became_peered":"2024-03-15T12:56:20.208155+0000","last_unstale":"2024-03-15T12:58:39.603492+0000","last_undegraded":"2024-03-15T12:58:39.603492+0000","last_fullsized":"2024-03-15T12:58:39.603492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:26:13.148273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2054,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.798776+0000","last_change":"2024-03-15T12:56:21.220421+0000","last_active":"2024-03-15T12:57:48.798776+0000","last_peered":"2024-03-15T12:57:48.798776+0000","last_clean":"2024-03-15T12:57:48.798776+0000","last_became_active":"2024-03-15T12:56:21.220095+0000","last_became_peered":"2024-03-15T12:56:21.220095+0000","last_unstale":"2024-03-15T12:57:48.798776+0000","last_undegraded":"2024-03-15T12:57:48.798776+0000","last_fullsized":"2024-03-15T12:57:48.798776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:35:33.846506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.836547+0000","last_change":"2024-03-15T12:56:25.440408+0000","last_active":"2024-03-15T12:57:19.836547+0000","last_peered":"2024-03-15T12:57:19.836547+0000","last_clean":"2024-03-15T12:57:19.836547+0000","last_became_active":"2024-03-15T12:56:25.439965+0000","last_became_peered":"2024-03-15T12:56:25.439965+0000","last_unstale":"2024-03-15T12:57:19.836547+0000","last_undegraded":"2024-03-15T12:57:19.836547+0000","last_fullsized":"2024-03-15T12:57:19.836547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T22:12:57.278202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.105052+0000","last_change":"2024-03-15T12:56:20.216320+0000","last_active":"2024-03-15T12:57:20.105052+0000","last_peered":"2024-03-15T12:57:20.105052+0000","last_clean":"2024-03-15T12:57:20.105052+0000","last_became_active":"2024-03-15T12:56:20.215915+0000","last_became_peered":"2024-03-15T12:56:20.215915+0000","last_unstale":"2024-03-15T12:57:20.105052+0000","last_undegraded":"2024-03-15T12:57:20.105052+0000","last_fullsized":"2024-03-15T12:57:20.105052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:25:45.236269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782769+0000","last_change":"2024-03-15T12:56:21.215822+0000","last_active":"2024-03-15T12:57:48.782769+0000","last_peered":"2024-03-15T12:57:48.782769+0000","last_clean":"2024-03-15T12:57:48.782769+0000","last_became_active":"2024-03-15T12:56:21.215508+0000","last_became_peered":"2024-03-15T12:56:21.215508+0000","last_unstale":"2024-03-15T12:57:48.782769+0000","last_undegraded":"2024-03-15T12:57:48.782769+0000","last_fullsized":"2024-03-15T12:57:48.782769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T12:56:55.919198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:20.103859+0000","last_change":"2024-03-15T12:56:25.436186+0000","last_active":"2024-03-15T12:57:20.103859+0000","last_peered":"2024-03-15T12:57:20.103859+0000","last_clean":"2024-03-15T12:57:20.103859+0000","last_became_active":"2024-03-15T12:56:25.435586+0000","last_became_peered":"2024-03-15T12:56:25.435586+0000","last_unstale":"2024-03-15T12:57:20.103859+0000","last_undegraded":"2024-03-15T12:57:20.103859+0000","last_fullsized":"2024-03-15T12:57:20.103859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:24.411313+0000","last_clean_scrub_stamp":"2024-03-15T12:56:24.411313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:02:26.850487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:19.837495+0000","last_change":"2024-03-15T12:56:20.217277+0000","last_active":"2024-03-15T12:57:19.837495+0000","last_peered":"2024-03-15T12:57:19.837495+0000","last_clean":"2024-03-15T12:57:19.837495+0000","last_became_active":"2024-03-15T12:56:20.216930+0000","last_became_peered":"2024-03-15T12:56:20.216930+0000","last_unstale":"2024-03-15T12:57:19.837495+0000","last_undegraded":"2024-03-15T12:57:19.837495+0000","last_fullsized":"2024-03-15T12:57:19.837495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:19.174930+0000","last_clean_scrub_stamp":"2024-03-15T12:56:19.174930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T23:47:51.075545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780182+0000","last_change":"2024-03-15T12:56:21.212066+0000","last_active":"2024-03-15T12:57:48.780182+0000","last_peered":"2024-03-15T12:57:48.780182+0000","last_clean":"2024-03-15T12:57:48.780182+0000","last_became_active":"2024-03-15T12:56:21.211650+0000","last_became_peered":"2024-03-15T12:56:21.211650+0000","last_unstale":"2024-03-15T12:57:48.780182+0000","last_undegraded":"2024-03-15T12:57:48.780182+0000","last_fullsized":"2024-03-15T12:57:48.780182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T20:40:35.322210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.772659+0000","last_change":"2024-03-15T12:56:21.212794+0000","last_active":"2024-03-15T12:57:48.772659+0000","last_peered":"2024-03-15T12:57:48.772659+0000","last_clean":"2024-03-15T12:57:48.772659+0000","last_became_active":"2024-03-15T12:56:21.212470+0000","last_became_peered":"2024-03-15T12:56:21.212470+0000","last_unstale":"2024-03-15T12:57:48.772659+0000","last_undegraded":"2024-03-15T12:57:48.772659+0000","last_fullsized":"2024-03-15T12:57:48.772659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:10:04.614223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.756171+0000","last_change":"2024-03-15T12:56:21.211746+0000","last_active":"2024-03-15T12:57:48.756171+0000","last_peered":"2024-03-15T12:57:48.756171+0000","last_clean":"2024-03-15T12:57:48.756171+0000","last_became_active":"2024-03-15T12:56:21.211431+0000","last_became_peered":"2024-03-15T12:56:21.211431+0000","last_unstale":"2024-03-15T12:57:48.756171+0000","last_undegraded":"2024-03-15T12:57:48.756171+0000","last_fullsized":"2024-03-15T12:57:48.756171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:46:31.188408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.778038+0000","last_change":"2024-03-15T12:56:21.220997+0000","last_active":"2024-03-15T12:57:48.778038+0000","last_peered":"2024-03-15T12:57:48.778038+0000","last_clean":"2024-03-15T12:57:48.778038+0000","last_became_active":"2024-03-15T12:56:21.220644+0000","last_became_peered":"2024-03-15T12:56:21.220644+0000","last_unstale":"2024-03-15T12:57:48.778038+0000","last_undegraded":"2024-03-15T12:57:48.778038+0000","last_fullsized":"2024-03-15T12:57:48.778038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T00:47:55.852845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780052+0000","last_change":"2024-03-15T12:56:21.211634+0000","last_active":"2024-03-15T12:57:48.780052+0000","last_peered":"2024-03-15T12:57:48.780052+0000","last_clean":"2024-03-15T12:57:48.780052+0000","last_became_active":"2024-03-15T12:56:21.211303+0000","last_became_peered":"2024-03-15T12:56:21.211303+0000","last_unstale":"2024-03-15T12:57:48.780052+0000","last_undegraded":"2024-03-15T12:57:48.780052+0000","last_fullsized":"2024-03-15T12:57:48.780052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:22:32.358270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.780806+0000","last_change":"2024-03-15T12:56:21.211673+0000","last_active":"2024-03-15T12:57:48.780806+0000","last_peered":"2024-03-15T12:57:48.780806+0000","last_clean":"2024-03-15T12:57:48.780806+0000","last_became_active":"2024-03-15T12:56:21.211296+0000","last_became_peered":"2024-03-15T12:56:21.211296+0000","last_unstale":"2024-03-15T12:57:48.780806+0000","last_undegraded":"2024-03-15T12:57:48.780806+0000","last_fullsized":"2024-03-15T12:57:48.780806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T19:40:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.769894+0000","last_change":"2024-03-15T12:56:21.208681+0000","last_active":"2024-03-15T12:57:48.769894+0000","last_peered":"2024-03-15T12:57:48.769894+0000","last_clean":"2024-03-15T12:57:48.769894+0000","last_became_active":"2024-03-15T12:56:21.208130+0000","last_became_peered":"2024-03-15T12:56:21.208130+0000","last_unstale":"2024-03-15T12:57:48.769894+0000","last_undegraded":"2024-03-15T12:57:48.769894+0000","last_fullsized":"2024-03-15T12:57:48.769894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:34:52.971485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.793618+0000","last_change":"2024-03-15T12:56:21.216979+0000","last_active":"2024-03-15T12:57:48.793618+0000","last_peered":"2024-03-15T12:57:48.793618+0000","last_clean":"2024-03-15T12:57:48.793618+0000","last_became_active":"2024-03-15T12:56:21.216632+0000","last_became_peered":"2024-03-15T12:56:21.216632+0000","last_unstale":"2024-03-15T12:57:48.793618+0000","last_undegraded":"2024-03-15T12:57:48.793618+0000","last_fullsized":"2024-03-15T12:57:48.793618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:21:57.213512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.785754+0000","last_change":"2024-03-15T12:56:21.223064+0000","last_active":"2024-03-15T12:57:48.785754+0000","last_peered":"2024-03-15T12:57:48.785754+0000","last_clean":"2024-03-15T12:57:48.785754+0000","last_became_active":"2024-03-15T12:56:21.220882+0000","last_became_peered":"2024-03-15T12:56:21.220882+0000","last_unstale":"2024-03-15T12:57:48.785754+0000","last_undegraded":"2024-03-15T12:57:48.785754+0000","last_fullsized":"2024-03-15T12:57:48.785754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T13:45:02.930154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782047+0000","last_change":"2024-03-15T12:56:21.218614+0000","last_active":"2024-03-15T12:57:48.782047+0000","last_peered":"2024-03-15T12:57:48.782047+0000","last_clean":"2024-03-15T12:57:48.782047+0000","last_became_active":"2024-03-15T12:56:21.218224+0000","last_became_peered":"2024-03-15T12:56:21.218224+0000","last_unstale":"2024-03-15T12:57:48.782047+0000","last_undegraded":"2024-03-15T12:57:48.782047+0000","last_fullsized":"2024-03-15T12:57:48.782047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T16:41:29.790358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.782852+0000","last_change":"2024-03-15T12:56:21.219019+0000","last_active":"2024-03-15T12:57:48.782852+0000","last_peered":"2024-03-15T12:57:48.782852+0000","last_clean":"2024-03-15T12:57:48.782852+0000","last_became_active":"2024-03-15T12:56:21.218673+0000","last_became_peered":"2024-03-15T12:56:21.218673+0000","last_unstale":"2024-03-15T12:57:48.782852+0000","last_undegraded":"2024-03-15T12:57:48.782852+0000","last_fullsized":"2024-03-15T12:57:48.782852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T18:42:11.788598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.799975+0000","last_change":"2024-03-15T12:56:21.220355+0000","last_active":"2024-03-15T12:57:48.799975+0000","last_peered":"2024-03-15T12:57:48.799975+0000","last_clean":"2024-03-15T12:57:48.799975+0000","last_became_active":"2024-03-15T12:56:21.220021+0000","last_became_peered":"2024-03-15T12:56:21.220021+0000","last_unstale":"2024-03-15T12:57:48.799975+0000","last_undegraded":"2024-03-15T12:57:48.799975+0000","last_fullsized":"2024-03-15T12:57:48.799975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T14:40:00.947351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.781265+0000","last_change":"2024-03-15T12:56:21.213050+0000","last_active":"2024-03-15T12:57:48.781265+0000","last_peered":"2024-03-15T12:57:48.781265+0000","last_clean":"2024-03-15T12:57:48.781265+0000","last_became_active":"2024-03-15T12:56:21.212730+0000","last_became_peered":"2024-03-15T12:56:21.212730+0000","last_unstale":"2024-03-15T12:57:48.781265+0000","last_undegraded":"2024-03-15T12:57:48.781265+0000","last_fullsized":"2024-03-15T12:57:48.781265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T21:56:36.396285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-15T12:57:48.770729+0000","last_change":"2024-03-15T12:56:21.213942+0000","last_active":"2024-03-15T12:57:48.770729+0000","last_peered":"2024-03-15T12:57:48.770729+0000","last_clean":"2024-03-15T12:57:48.770729+0000","last_became_active":"2024-03-15T12:56:21.213611+0000","last_became_peered":"2024-03-15T12:56:21.213611+0000","last_unstale":"2024-03-15T12:57:48.770729+0000","last_undegraded":"2024-03-15T12:57:48.770729+0000","last_fullsized":"2024-03-15T12:57:48.770729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-15T12:56:20.184349+0000","last_clean_scrub_stamp":"2024-03-15T12:56:20.184349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-16T15:23:44.002970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79059,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1224,"num_read_kb":701,"num_write":271,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":237959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":501940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1728512,"data_stored":1151302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":504970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":501140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36948,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708204,"statfs":{"total":95995035648,"available":95957200896,"internally_reserved":0,"allocated":1650688,"data_stored":1099369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495683,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":496871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":568034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":498327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":568747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":496086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":501477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1093433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":72894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-15T12:58:43.604 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:58:43.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.762+0000 7f8d64eed640 1 -- 172.21.15.2:0/890597884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d601048c0 msgr2=0x7f8d60106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:43.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.762+0000 7f8d64eed640 1 --2- 172.21.15.2:0/890597884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d601048c0 0x7f8d60106cb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009860 tx=0x7f8d5402bb70 comp rx=0 tx=0).stop 2024-03-15T12:58:43.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.763+0000 7f8d64eed640 1 -- 172.21.15.2:0/890597884 shutdown_connections 2024-03-15T12:58:43.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.763+0000 7f8d64eed640 1 --2- 172.21.15.2:0/890597884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d601071f0 0x7f8d60109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.763+0000 7f8d64eed640 1 --2- 172.21.15.2:0/890597884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d601048c0 0x7f8d60106cb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.763+0000 7f8d64eed640 1 --2- 172.21.15.2:0/890597884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d60101f90 0x7f8d60104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.763+0000 7f8d64eed640 1 -- 172.21.15.2:0/890597884 >> 172.21.15.2:0/890597884 conn(0x7f8d600fb980 msgr2=0x7f8d600fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:43.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.763+0000 7f8d64eed640 1 -- 172.21.15.2:0/890597884 shutdown_connections 2024-03-15T12:58:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.763+0000 7f8d64eed640 1 -- 172.21.15.2:0/890597884 wait complete. 2024-03-15T12:58:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.763+0000 7f8d64eed640 1 Processor -- start 2024-03-15T12:58:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.763+0000 7f8d64eed640 1 -- start start 2024-03-15T12:58:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d64eed640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d60101f90 0x7f8d601975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d64eed640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d601048c0 0x7f8d60197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d64eed640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d601071f0 0x7f8d6019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d64eed640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6019f0c0 con 0x7f8d601048c0 2024-03-15T12:58:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d64eed640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6019f230 con 0x7f8d60101f90 2024-03-15T12:58:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d64eed640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6019f3f0 con 0x7f8d601071f0 2024-03-15T12:58:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d5dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d601048c0 0x7f8d60197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d5ed9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d601071f0 0x7f8d6019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d5e59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d60101f90 0x7f8d601975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d5dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d601048c0 0x7f8d60197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43762/0 (socket says 172.21.15.2:43762) 2024-03-15T12:58:43.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d5ed9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d601071f0 0x7f8d6019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:52860/0 (socket says 172.21.15.2:52860) 2024-03-15T12:58:43.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d5dd9b640 1 -- 172.21.15.2:0/2712627556 learned_addr learned my addr 172.21.15.2:0/2712627556 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:43.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d5e59c640 1 -- 172.21.15.2:0/2712627556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d601071f0 msgr2=0x7f8d6019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:43.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d5e59c640 1 --2- 172.21.15.2:0/2712627556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d601071f0 0x7f8d6019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d5e59c640 1 -- 172.21.15.2:0/2712627556 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d601048c0 msgr2=0x7f8d60197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:43.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d5e59c640 1 --2- 172.21.15.2:0/2712627556 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d601048c0 0x7f8d60197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d5e59c640 1 -- 172.21.15.2:0/2712627556 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d50009560 con 0x7f8d60101f90 2024-03-15T12:58:43.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d5ed9d640 1 --2- 172.21.15.2:0/2712627556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d601071f0 0x7f8d6019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:43.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d5e59c640 1 --2- 172.21.15.2:0/2712627556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d60101f90 0x7f8d601975d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8d50005fd0 tx=0x7f8d5000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:43.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.764+0000 7f8d4f7fe640 1 -- 172.21.15.2:0/2712627556 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5000cd00 con 0x7f8d60101f90 2024-03-15T12:58:43.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.765+0000 7f8d64eed640 1 -- 172.21.15.2:0/2712627556 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d54009490 con 0x7f8d60101f90 2024-03-15T12:58:43.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.765+0000 7f8d4f7fe640 1 -- 172.21.15.2:0/2712627556 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8d500044f0 con 0x7f8d60101f90 2024-03-15T12:58:43.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.765+0000 7f8d4f7fe640 1 -- 172.21.15.2:0/2712627556 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50010430 con 0x7f8d60101f90 2024-03-15T12:58:43.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.765+0000 7f8d64eed640 1 -- 172.21.15.2:0/2712627556 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6019faa0 con 0x7f8d60101f90 2024-03-15T12:58:43.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.765+0000 7f8d64eed640 1 -- 172.21.15.2:0/2712627556 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d600fd470 con 0x7f8d60101f90 2024-03-15T12:58:43.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.766+0000 7f8d4f7fe640 1 -- 172.21.15.2:0/2712627556 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f8d50004000 con 0x7f8d60101f90 2024-03-15T12:58:43.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.767+0000 7f8d4f7fe640 1 --2- 172.21.15.2:0/2712627556 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8d480a1ac0 0x7f8d480a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:43.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.767+0000 7f8d4f7fe640 1 -- 172.21.15.2:0/2712627556 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f8d50014070 con 0x7f8d60101f90 2024-03-15T12:58:43.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.767+0000 7f8d5dd9b640 1 --2- 172.21.15.2:0/2712627556 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8d480a1ac0 0x7f8d480a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:43.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.768+0000 7f8d5dd9b640 1 --2- 172.21.15.2:0/2712627556 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8d480a1ac0 0x7f8d480a3f80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8d60198ae0 tx=0x7f8d54034b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:43.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.776+0000 7f8d4f7fe640 1 -- 172.21.15.2:0/2712627556 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8d50096990 con 0x7f8d60101f90 2024-03-15T12:58:43.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.960+0000 7f8d64eed640 1 -- 172.21.15.2:0/2712627556 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d6002d000 con 0x7f8d60101f90 2024-03-15T12:58:43.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.961+0000 7f8d4f7fe640 1 -- 172.21.15.2:0/2712627556 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24526 (secure 0 0 0) 0x7f8d500b8e60 con 0x7f8d60101f90 2024-03-15T12:58:43.948 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:43.948 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":91,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:57:19.826817+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:58:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.964+0000 7f8d64eed640 1 -- 172.21.15.2:0/2712627556 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8d480a1ac0 msgr2=0x7f8d480a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.964+0000 7f8d64eed640 1 --2- 172.21.15.2:0/2712627556 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8d480a1ac0 0x7f8d480a3f80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8d60198ae0 tx=0x7f8d54034b60 comp rx=0 tx=0).stop 2024-03-15T12:58:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.964+0000 7f8d64eed640 1 -- 172.21.15.2:0/2712627556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d60101f90 msgr2=0x7f8d601975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.964+0000 7f8d64eed640 1 --2- 172.21.15.2:0/2712627556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d60101f90 0x7f8d601975d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8d50005fd0 tx=0x7f8d5000ee50 comp rx=0 tx=0).stop 2024-03-15T12:58:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.964+0000 7f8d64eed640 1 -- 172.21.15.2:0/2712627556 shutdown_connections 2024-03-15T12:58:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.964+0000 7f8d64eed640 1 --2- 172.21.15.2:0/2712627556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d601071f0 0x7f8d6019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.964+0000 7f8d64eed640 1 --2- 172.21.15.2:0/2712627556 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d601048c0 0x7f8d60197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.964+0000 7f8d64eed640 1 --2- 172.21.15.2:0/2712627556 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f8d480a1ac0 0x7f8d480a3f80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.964+0000 7f8d64eed640 1 --2- 172.21.15.2:0/2712627556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d60101f90 0x7f8d601975d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.964+0000 7f8d64eed640 1 -- 172.21.15.2:0/2712627556 >> 172.21.15.2:0/2712627556 conn(0x7f8d600fb980 msgr2=0x7f8d60102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.964+0000 7f8d64eed640 1 -- 172.21.15.2:0/2712627556 shutdown_connections 2024-03-15T12:58:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:43.964+0000 7f8d64eed640 1 -- 172.21.15.2:0/2712627556 wait complete. 2024-03-15T12:58:43.962 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-15T12:51:47.004843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-15T12:56:18.559882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-15T12:56:19.751767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-15T12:56:23.664879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-15T12:58:43.962 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-15T12:58:44.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.084+0000 7f84a462a640 1 -- 172.21.15.2:0/1117850573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f849c1015e0 msgr2=0x7f849c101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.084+0000 7f84a462a640 1 --2- 172.21.15.2:0/1117850573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f849c1015e0 0x7f849c101a60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f848c009900 tx=0x7f848c02bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.084+0000 7f84a462a640 1 -- 172.21.15.2:0/1117850573 shutdown_connections 2024-03-15T12:58:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.084+0000 7f84a462a640 1 --2- 172.21.15.2:0/1117850573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f849c101fa0 0x7f849c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.084+0000 7f84a462a640 1 --2- 172.21.15.2:0/1117850573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f849c1015e0 0x7f849c101a60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.084+0000 7f84a462a640 1 --2- 172.21.15.2:0/1117850573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f849c1003f0 0x7f849c100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.084+0000 7f84a462a640 1 -- 172.21.15.2:0/1117850573 >> 172.21.15.2:0/1117850573 conn(0x7f849c0fb980 msgr2=0x7f849c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.084+0000 7f84a462a640 1 -- 172.21.15.2:0/1117850573 shutdown_connections 2024-03-15T12:58:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.084+0000 7f84a462a640 1 -- 172.21.15.2:0/1117850573 wait complete. 2024-03-15T12:58:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a462a640 1 Processor -- start 2024-03-15T12:58:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a462a640 1 -- start start 2024-03-15T12:58:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a462a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f849c1003f0 0x7f849c069220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a462a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f849c1015e0 0x7f849c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a462a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f849c101fa0 0x7f849c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a462a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c19f0b0 con 0x7f849c1015e0 2024-03-15T12:58:44.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a462a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c19f220 con 0x7f849c1003f0 2024-03-15T12:58:44.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a462a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c19f3e0 con 0x7f849c101fa0 2024-03-15T12:58:44.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a1bc5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f849c1015e0 0x7f849c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:44.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a1bc5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f849c1015e0 0x7f849c19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43768/0 (socket says 172.21.15.2:43768) 2024-03-15T12:58:44.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a1bc5640 1 -- 172.21.15.2:0/4007970857 learned_addr learned my addr 172.21.15.2:0/4007970857 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:44.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a23c6640 1 --2- 172.21.15.2:0/4007970857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f849c1003f0 0x7f849c069220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:44.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a2bc7640 1 --2- 172.21.15.2:0/4007970857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f849c101fa0 0x7f849c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a1bc5640 1 -- 172.21.15.2:0/4007970857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f849c101fa0 msgr2=0x7f849c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a1bc5640 1 --2- 172.21.15.2:0/4007970857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f849c101fa0 0x7f849c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a1bc5640 1 -- 172.21.15.2:0/4007970857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f849c1003f0 msgr2=0x7f849c069220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a1bc5640 1 --2- 172.21.15.2:0/4007970857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f849c1003f0 0x7f849c069220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a1bc5640 1 -- 172.21.15.2:0/4007970857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8494009840 con 0x7f849c1015e0 2024-03-15T12:58:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a2bc7640 1 --2- 172.21.15.2:0/4007970857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f849c101fa0 0x7f849c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:58:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a1bc5640 1 --2- 172.21.15.2:0/4007970857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f849c1015e0 0x7f849c19c2b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f848c005ab0 tx=0x7f848c004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84937fe640 1 -- 172.21.15.2:0/4007970857 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c004250 con 0x7f849c1015e0 2024-03-15T12:58:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84a462a640 1 -- 172.21.15.2:0/4007970857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848c009560 con 0x7f849c1015e0 2024-03-15T12:58:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.085+0000 7f84937fe640 1 -- 172.21.15.2:0/4007970857 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f848c0043b0 con 0x7f849c1015e0 2024-03-15T12:58:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.086+0000 7f84937fe640 1 -- 172.21.15.2:0/4007970857 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c003d20 con 0x7f849c1015e0 2024-03-15T12:58:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.086+0000 7f84a462a640 1 -- 172.21.15.2:0/4007970857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849c19f9e0 con 0x7f849c1015e0 2024-03-15T12:58:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.087+0000 7f84a462a640 1 -- 172.21.15.2:0/4007970857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849c1077d0 con 0x7f849c1015e0 2024-03-15T12:58:44.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.088+0000 7f84937fe640 1 -- 172.21.15.2:0/4007970857 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f848c005000 con 0x7f849c1015e0 2024-03-15T12:58:44.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.088+0000 7f84937fe640 1 --2- 172.21.15.2:0/4007970857 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f84880a1ac0 0x7f84880a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:44.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.088+0000 7f84937fe640 1 -- 172.21.15.2:0/4007970857 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f848c0f01f0 con 0x7f849c1015e0 2024-03-15T12:58:44.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.088+0000 7f84a23c6640 1 --2- 172.21.15.2:0/4007970857 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f84880a1ac0 0x7f84880a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:44.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.089+0000 7f84a23c6640 1 --2- 172.21.15.2:0/4007970857 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f84880a1ac0 0x7f84880a3f80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f849c101440 tx=0x7f8498005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:44.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.092+0000 7f84937fe640 1 -- 172.21.15.2:0/4007970857 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f848c0b9070 con 0x7f849c1015e0 2024-03-15T12:58:44.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.274+0000 7f84a462a640 1 -- 172.21.15.2:0/4007970857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f849c10c020 con 0x7f849c1015e0 2024-03-15T12:58:44.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.275+0000 7f84937fe640 1 -- 172.21.15.2:0/4007970857 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f848c0bc730 con 0x7f849c1015e0 2024-03-15T12:58:44.261 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-15T12:58:44.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.277+0000 7f84a462a640 1 -- 172.21.15.2:0/4007970857 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f84880a1ac0 msgr2=0x7f84880a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:44.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.277+0000 7f84a462a640 1 --2- 172.21.15.2:0/4007970857 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f84880a1ac0 0x7f84880a3f80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f849c101440 tx=0x7f8498005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.277+0000 7f84a462a640 1 -- 172.21.15.2:0/4007970857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f849c1015e0 msgr2=0x7f849c19c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.277+0000 7f84a462a640 1 --2- 172.21.15.2:0/4007970857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f849c1015e0 0x7f849c19c2b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f848c005ab0 tx=0x7f848c004a30 comp rx=0 tx=0).stop 2024-03-15T12:58:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.277+0000 7f84a462a640 1 -- 172.21.15.2:0/4007970857 shutdown_connections 2024-03-15T12:58:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.277+0000 7f84a462a640 1 --2- 172.21.15.2:0/4007970857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f849c101fa0 0x7f849c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.277+0000 7f84a462a640 1 --2- 172.21.15.2:0/4007970857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f849c1015e0 0x7f849c19c2b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.277+0000 7f84a462a640 1 --2- 172.21.15.2:0/4007970857 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f84880a1ac0 0x7f84880a3f80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.277+0000 7f84a462a640 1 --2- 172.21.15.2:0/4007970857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f849c1003f0 0x7f849c069220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.277+0000 7f84a462a640 1 -- 172.21.15.2:0/4007970857 >> 172.21.15.2:0/4007970857 conn(0x7f849c0fb980 msgr2=0x7f849c1049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.277+0000 7f84a462a640 1 -- 172.21.15.2:0/4007970857 shutdown_connections 2024-03-15T12:58:44.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.277+0000 7f84a462a640 1 -- 172.21.15.2:0/4007970857 wait complete. 2024-03-15T12:58:44.274 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-15T12:58:44.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.394+0000 7fdf985d7640 1 -- 172.21.15.2:0/649857582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf901030b0 msgr2=0x7fdf90109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.394+0000 7fdf985d7640 1 --2- 172.21.15.2:0/649857582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf901030b0 0x7fdf90109940 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009be0 tx=0x7fdf8802bef0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.395+0000 7fdf985d7640 1 -- 172.21.15.2:0/649857582 shutdown_connections 2024-03-15T12:58:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.395+0000 7fdf985d7640 1 --2- 172.21.15.2:0/649857582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf901030b0 0x7fdf90109940 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.395+0000 7fdf985d7640 1 --2- 172.21.15.2:0/649857582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf901026f0 0x7fdf90102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.395+0000 7fdf985d7640 1 --2- 172.21.15.2:0/649857582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf90101500 0x7fdf90101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.395+0000 7fdf985d7640 1 -- 172.21.15.2:0/649857582 >> 172.21.15.2:0/649857582 conn(0x7fdf900fca70 msgr2=0x7fdf900feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.395+0000 7fdf985d7640 1 -- 172.21.15.2:0/649857582 shutdown_connections 2024-03-15T12:58:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.395+0000 7fdf985d7640 1 -- 172.21.15.2:0/649857582 wait complete. 2024-03-15T12:58:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.395+0000 7fdf985d7640 1 Processor -- start 2024-03-15T12:58:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.395+0000 7fdf985d7640 1 -- start start 2024-03-15T12:58:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.395+0000 7fdf985d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf90101500 0x7fdf901975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.395+0000 7fdf985d7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf901026f0 0x7fdf90197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.395+0000 7fdf985d7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf901030b0 0x7fdf9019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.395+0000 7fdf985d7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9019f090 con 0x7fdf90101500 2024-03-15T12:58:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.395+0000 7fdf985d7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9019f200 con 0x7fdf901026f0 2024-03-15T12:58:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.395+0000 7fdf985d7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9019f3c0 con 0x7fdf901030b0 2024-03-15T12:58:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf96373640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf90101500 0x7fdf901975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf96373640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf90101500 0x7fdf901975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43788/0 (socket says 172.21.15.2:43788) 2024-03-15T12:58:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf96373640 1 -- 172.21.15.2:0/302204628 learned_addr learned my addr 172.21.15.2:0/302204628 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf96b74640 1 --2- 172.21.15.2:0/302204628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf901030b0 0x7fdf9019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf95b72640 1 --2- 172.21.15.2:0/302204628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf901026f0 0x7fdf90197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf96373640 1 -- 172.21.15.2:0/302204628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf901030b0 msgr2=0x7fdf9019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf96373640 1 --2- 172.21.15.2:0/302204628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf901030b0 0x7fdf9019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf96373640 1 -- 172.21.15.2:0/302204628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf901026f0 msgr2=0x7fdf90197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:44.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf96373640 1 --2- 172.21.15.2:0/302204628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf901026f0 0x7fdf90197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf96373640 1 -- 172.21.15.2:0/302204628 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf88009840 con 0x7fdf90101500 2024-03-15T12:58:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf96373640 1 --2- 172.21.15.2:0/302204628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf90101500 0x7fdf901975a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00e9f0 tx=0x7fdf8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf877fe640 1 -- 172.21.15.2:0/302204628 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00ccc0 con 0x7fdf90101500 2024-03-15T12:58:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf877fe640 1 -- 172.21.15.2:0/302204628 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdf8c00ce20 con 0x7fdf90101500 2024-03-15T12:58:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf985d7640 1 -- 172.21.15.2:0/302204628 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9019f6a0 con 0x7fdf90101500 2024-03-15T12:58:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf877fe640 1 -- 172.21.15.2:0/302204628 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c010430 con 0x7fdf90101500 2024-03-15T12:58:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.396+0000 7fdf985d7640 1 -- 172.21.15.2:0/302204628 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9019fbb0 con 0x7fdf90101500 2024-03-15T12:58:44.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.397+0000 7fdf985d7640 1 -- 172.21.15.2:0/302204628 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf901088f0 con 0x7fdf90101500 2024-03-15T12:58:44.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.398+0000 7fdf877fe640 1 -- 172.21.15.2:0/302204628 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7fdf8c003780 con 0x7fdf90101500 2024-03-15T12:58:44.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.399+0000 7fdf877fe640 1 --2- 172.21.15.2:0/302204628 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fdf7c0a1ac0 0x7fdf7c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:44.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.399+0000 7fdf95b72640 1 --2- 172.21.15.2:0/302204628 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fdf7c0a1ac0 0x7fdf7c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:44.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.399+0000 7fdf877fe640 1 -- 172.21.15.2:0/302204628 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fdf8c014070 con 0x7fdf90101500 2024-03-15T12:58:44.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.399+0000 7fdf95b72640 1 --2- 172.21.15.2:0/302204628 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fdf7c0a1ac0 0x7fdf7c0a3f80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdf90198ab0 tx=0x7fdf80005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:44.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.403+0000 7fdf877fe640 1 -- 172.21.15.2:0/302204628 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdf8c0968f0 con 0x7fdf90101500 2024-03-15T12:58:44.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.587+0000 7fdf985d7640 1 -- 172.21.15.2:0/302204628 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf9010ced0 con 0x7fdf90101500 2024-03-15T12:58:44.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.587+0000 7fdf877fe640 1 -- 172.21.15.2:0/302204628 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf8c099fb0 con 0x7fdf90101500 2024-03-15T12:58:44.573 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:58:44.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.589+0000 7fdf985d7640 1 -- 172.21.15.2:0/302204628 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fdf7c0a1ac0 msgr2=0x7fdf7c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:44.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.589+0000 7fdf985d7640 1 --2- 172.21.15.2:0/302204628 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fdf7c0a1ac0 0x7fdf7c0a3f80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdf90198ab0 tx=0x7fdf80005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:44.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.589+0000 7fdf985d7640 1 -- 172.21.15.2:0/302204628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf90101500 msgr2=0x7fdf901975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:44.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.589+0000 7fdf985d7640 1 --2- 172.21.15.2:0/302204628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf90101500 0x7fdf901975a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00e9f0 tx=0x7fdf8c00eeb0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.589+0000 7fdf985d7640 1 -- 172.21.15.2:0/302204628 shutdown_connections 2024-03-15T12:58:44.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.589+0000 7fdf985d7640 1 --2- 172.21.15.2:0/302204628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf901030b0 0x7fdf9019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.589+0000 7fdf985d7640 1 --2- 172.21.15.2:0/302204628 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fdf7c0a1ac0 0x7fdf7c0a3f80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.589+0000 7fdf985d7640 1 --2- 172.21.15.2:0/302204628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf901026f0 0x7fdf90197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.589+0000 7fdf985d7640 1 --2- 172.21.15.2:0/302204628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf90101500 0x7fdf901975a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.589+0000 7fdf985d7640 1 -- 172.21.15.2:0/302204628 >> 172.21.15.2:0/302204628 conn(0x7fdf900fca70 msgr2=0x7fdf90105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:44.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.589+0000 7fdf985d7640 1 -- 172.21.15.2:0/302204628 shutdown_connections 2024-03-15T12:58:44.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.589+0000 7fdf985d7640 1 -- 172.21.15.2:0/302204628 wait complete. 2024-03-15T12:58:44.586 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-15T12:58:44.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.706+0000 7f3e94e76640 1 -- 172.21.15.2:0/2205616255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e90101f90 msgr2=0x7f3e90104380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:44.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.706+0000 7f3e94e76640 1 --2- 172.21.15.2:0/2205616255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e90101f90 0x7f3e90104380 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009900 tx=0x7f3e8002bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:44.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.706+0000 7f3e94e76640 1 -- 172.21.15.2:0/2205616255 shutdown_connections 2024-03-15T12:58:44.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.706+0000 7f3e94e76640 1 --2- 172.21.15.2:0/2205616255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e901071f0 0x7f3e90109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.706+0000 7f3e94e76640 1 --2- 172.21.15.2:0/2205616255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e901048c0 0x7f3e90106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.706+0000 7f3e94e76640 1 --2- 172.21.15.2:0/2205616255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e90101f90 0x7f3e90104380 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.706+0000 7f3e94e76640 1 -- 172.21.15.2:0/2205616255 >> 172.21.15.2:0/2205616255 conn(0x7f3e900fb9a0 msgr2=0x7f3e900fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:44.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.706+0000 7f3e94e76640 1 -- 172.21.15.2:0/2205616255 shutdown_connections 2024-03-15T12:58:44.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.706+0000 7f3e94e76640 1 -- 172.21.15.2:0/2205616255 wait complete. 2024-03-15T12:58:44.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.707+0000 7f3e94e76640 1 Processor -- start 2024-03-15T12:58:44.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.707+0000 7f3e94e76640 1 -- start start 2024-03-15T12:58:44.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.707+0000 7f3e94e76640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e90101f90 0x7f3e90197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:44.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.707+0000 7f3e94e76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e901048c0 0x7f3e90197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:44.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.707+0000 7f3e94e76640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e901071f0 0x7f3e9019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:44.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.707+0000 7f3e94e76640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9019f0f0 con 0x7f3e901048c0 2024-03-15T12:58:44.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.707+0000 7f3e94e76640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9019f260 con 0x7f3e90101f90 2024-03-15T12:58:44.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.707+0000 7f3e94e76640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9019f420 con 0x7f3e901071f0 2024-03-15T12:58:44.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.707+0000 7f3e85d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e901048c0 0x7f3e90197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:44.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.707+0000 7f3e85d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e901048c0 0x7f3e90197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43802/0 (socket says 172.21.15.2:43802) 2024-03-15T12:58:44.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.707+0000 7f3e85d9b640 1 -- 172.21.15.2:0/3048144860 learned_addr learned my addr 172.21.15.2:0/3048144860 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:44.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.707+0000 7f3e8e59c640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e90101f90 0x7f3e90197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:44.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.707+0000 7f3e8ed9d640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e901071f0 0x7f3e9019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:44.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.708+0000 7f3e85d9b640 1 -- 172.21.15.2:0/3048144860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e901071f0 msgr2=0x7f3e9019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:44.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.708+0000 7f3e85d9b640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e901071f0 0x7f3e9019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.708+0000 7f3e85d9b640 1 -- 172.21.15.2:0/3048144860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e90101f90 msgr2=0x7f3e90197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:44.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.708+0000 7f3e85d9b640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e90101f90 0x7f3e90197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.708+0000 7f3e85d9b640 1 -- 172.21.15.2:0/3048144860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e80009560 con 0x7f3e901048c0 2024-03-15T12:58:44.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.708+0000 7f3e8ed9d640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e901071f0 0x7f3e9019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:44.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.708+0000 7f3e8e59c640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e90101f90 0x7f3e90197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:44.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.708+0000 7f3e85d9b640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e901048c0 0x7f3e90197b40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e9c0 tx=0x7f3e7800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:44.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.711+0000 7f3e87fff640 1 -- 172.21.15.2:0/3048144860 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78009700 con 0x7f3e901048c0 2024-03-15T12:58:44.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.711+0000 7f3e87fff640 1 -- 172.21.15.2:0/3048144860 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3e7800cd80 con 0x7f3e901048c0 2024-03-15T12:58:44.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.711+0000 7f3e94e76640 1 -- 172.21.15.2:0/3048144860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9019f700 con 0x7f3e901048c0 2024-03-15T12:58:44.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.711+0000 7f3e87fff640 1 -- 172.21.15.2:0/3048144860 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78010430 con 0x7f3e901048c0 2024-03-15T12:58:44.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.711+0000 7f3e94e76640 1 -- 172.21.15.2:0/3048144860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9019fc10 con 0x7f3e901048c0 2024-03-15T12:58:44.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.712+0000 7f3e94e76640 1 -- 172.21.15.2:0/3048144860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e900fd490 con 0x7f3e901048c0 2024-03-15T12:58:44.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.713+0000 7f3e87fff640 1 -- 172.21.15.2:0/3048144860 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f3e78004ff0 con 0x7f3e901048c0 2024-03-15T12:58:44.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.714+0000 7f3e87fff640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e7c0a1ac0 0x7f3e7c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:44.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.714+0000 7f3e87fff640 1 -- 172.21.15.2:0/3048144860 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f3e78014070 con 0x7f3e901048c0 2024-03-15T12:58:44.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.714+0000 7f3e8e59c640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e7c0a1ac0 0x7f3e7c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:44.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.714+0000 7f3e8e59c640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e7c0a1ac0 0x7f3e7c0a3f80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3e80000c00 tx=0x7f3e80005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:44.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.719+0000 7f3e87fff640 1 -- 172.21.15.2:0/3048144860 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3e780969a0 con 0x7f3e901048c0 2024-03-15T12:58:44.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.901+0000 7f3e94e76640 1 -- 172.21.15.2:0/3048144860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3e9010d140 con 0x7f3e901048c0 2024-03-15T12:58:44.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.901+0000 7f3e87fff640 1 -- 172.21.15.2:0/3048144860 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3e7809a060 con 0x7f3e901048c0 2024-03-15T12:58:44.887 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-03-15T12:58:44.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.903+0000 7f3e94e76640 1 -- 172.21.15.2:0/3048144860 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e7c0a1ac0 msgr2=0x7f3e7c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:44.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.903+0000 7f3e94e76640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e7c0a1ac0 0x7f3e7c0a3f80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3e80000c00 tx=0x7f3e80005fd0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.903+0000 7f3e94e76640 1 -- 172.21.15.2:0/3048144860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e901048c0 msgr2=0x7f3e90197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:44.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.903+0000 7f3e94e76640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e901048c0 0x7f3e90197b40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e9c0 tx=0x7f3e7800ee80 comp rx=0 tx=0).stop 2024-03-15T12:58:44.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.903+0000 7f3e94e76640 1 -- 172.21.15.2:0/3048144860 shutdown_connections 2024-03-15T12:58:44.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.903+0000 7f3e94e76640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e901071f0 0x7f3e9019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.903+0000 7f3e94e76640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e901048c0 0x7f3e90197b40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.903+0000 7f3e94e76640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f3e7c0a1ac0 0x7f3e7c0a3f80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.903+0000 7f3e94e76640 1 --2- 172.21.15.2:0/3048144860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e90101f90 0x7f3e90197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:44.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.903+0000 7f3e94e76640 1 -- 172.21.15.2:0/3048144860 >> 172.21.15.2:0/3048144860 conn(0x7f3e900fb9a0 msgr2=0x7f3e901028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:44.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.903+0000 7f3e94e76640 1 -- 172.21.15.2:0/3048144860 shutdown_connections 2024-03-15T12:58:44.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:44.903+0000 7f3e94e76640 1 -- 172.21.15.2:0/3048144860 wait complete. 2024-03-15T12:58:44.899 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:44 smithi002 ceph-mon[32060]: pgmap v363: 129 pgs: 129 active+clean; 654 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-03-15T12:58:44.900 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:44 smithi002 ceph-mon[32060]: from='client.15630 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:58:44.900 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:44 smithi002 ceph-mon[32060]: from='client.15636 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:58:44.900 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:44 smithi002 ceph-mon[32060]: from='client.25514 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:58:44.900 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:44 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2712627556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:44.900 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:44 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/4007970857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:44.900 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:44 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/302204628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:44.901 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-15T12:58:45.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 -- 172.21.15.2:0/1347986269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2281027a0 msgr2=0x7fd228102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 --2- 172.21.15.2:0/1347986269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2281027a0 0x7fd228102c20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd214009900 tx=0x7fd21402bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:45.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 -- 172.21.15.2:0/1347986269 shutdown_connections 2024-03-15T12:58:45.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 --2- 172.21.15.2:0/1347986269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd228103160 0x7fd2281099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 --2- 172.21.15.2:0/1347986269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2281027a0 0x7fd228102c20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 --2- 172.21.15.2:0/1347986269 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2280ff7d0 0x7fd2280ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 -- 172.21.15.2:0/1347986269 >> 172.21.15.2:0/1347986269 conn(0x7fd2280fb120 msgr2=0x7fd2280fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:45.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 -- 172.21.15.2:0/1347986269 shutdown_connections 2024-03-15T12:58:45.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 -- 172.21.15.2:0/1347986269 wait complete. 2024-03-15T12:58:45.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 Processor -- start 2024-03-15T12:58:45.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 -- start start 2024-03-15T12:58:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2280ff7d0 0x7fd2281975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2281027a0 0x7fd228197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd228103160 0x7fd22819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22819f0a0 con 0x7fd2280ff7d0 2024-03-15T12:58:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22819f210 con 0x7fd2281027a0 2024-03-15T12:58:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.021+0000 7fd22ecff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22819f3d0 con 0x7fd228103160 2024-03-15T12:58:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd22ca9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2280ff7d0 0x7fd2281975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd22ca9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2280ff7d0 0x7fd2281975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43816/0 (socket says 172.21.15.2:43816) 2024-03-15T12:58:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd22ca9b640 1 -- 172.21.15.2:0/3017717817 learned_addr learned my addr 172.21.15.2:0/3017717817 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd227fff640 1 --2- 172.21.15.2:0/3017717817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2281027a0 0x7fd228197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd22d29c640 1 --2- 172.21.15.2:0/3017717817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd228103160 0x7fd22819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd22ca9b640 1 -- 172.21.15.2:0/3017717817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd228103160 msgr2=0x7fd22819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd22ca9b640 1 --2- 172.21.15.2:0/3017717817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd228103160 0x7fd22819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd22ca9b640 1 -- 172.21.15.2:0/3017717817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2281027a0 msgr2=0x7fd228197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd22ca9b640 1 --2- 172.21.15.2:0/3017717817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2281027a0 0x7fd228197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd22ca9b640 1 -- 172.21.15.2:0/3017717817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd220009840 con 0x7fd2280ff7d0 2024-03-15T12:58:45.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd22ca9b640 1 --2- 172.21.15.2:0/3017717817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2280ff7d0 0x7fd2281975b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00e9c0 tx=0x7fd21c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:45.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd225ffb640 1 -- 172.21.15.2:0/3017717817 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c009700 con 0x7fd2280ff7d0 2024-03-15T12:58:45.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd225ffb640 1 -- 172.21.15.2:0/3017717817 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd21c0044f0 con 0x7fd2280ff7d0 2024-03-15T12:58:45.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd225ffb640 1 -- 172.21.15.2:0/3017717817 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c010430 con 0x7fd2280ff7d0 2024-03-15T12:58:45.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd22ecff640 1 -- 172.21.15.2:0/3017717817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd214009560 con 0x7fd2280ff7d0 2024-03-15T12:58:45.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.022+0000 7fd22ecff640 1 -- 172.21.15.2:0/3017717817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22819f9b0 con 0x7fd2280ff7d0 2024-03-15T12:58:45.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.023+0000 7fd22ecff640 1 -- 172.21.15.2:0/3017717817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2281089a0 con 0x7fd2280ff7d0 2024-03-15T12:58:45.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.024+0000 7fd225ffb640 1 -- 172.21.15.2:0/3017717817 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7fd21c00ccc0 con 0x7fd2280ff7d0 2024-03-15T12:58:45.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.025+0000 7fd225ffb640 1 --2- 172.21.15.2:0/3017717817 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd2180a1ac0 0x7fd2180a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.025+0000 7fd225ffb640 1 -- 172.21.15.2:0/3017717817 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fd21c014070 con 0x7fd2280ff7d0 2024-03-15T12:58:45.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.025+0000 7fd227fff640 1 --2- 172.21.15.2:0/3017717817 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd2180a1ac0 0x7fd2180a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.025+0000 7fd227fff640 1 --2- 172.21.15.2:0/3017717817 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd2180a1ac0 0x7fd2180a3f80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd214000c00 tx=0x7fd214005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:45.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.029+0000 7fd225ffb640 1 -- 172.21.15.2:0/3017717817 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd21c0969c0 con 0x7fd2280ff7d0 2024-03-15T12:58:45.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:44 smithi184 ceph-mon[35993]: pgmap v363: 129 pgs: 129 active+clean; 654 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-03-15T12:58:45.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:44 smithi184 ceph-mon[35993]: from='client.15630 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:58:45.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:44 smithi184 ceph-mon[35993]: from='client.15636 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:58:45.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:44 smithi184 ceph-mon[35993]: from='client.25514 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:58:45.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:44 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2712627556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:45.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:44 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/4007970857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:45.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:44 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/302204628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:45.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:44 smithi149 ceph-mon[35927]: pgmap v363: 129 pgs: 129 active+clean; 654 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-03-15T12:58:45.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:44 smithi149 ceph-mon[35927]: from='client.15630 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:58:45.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:44 smithi149 ceph-mon[35927]: from='client.15636 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:58:45.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:44 smithi149 ceph-mon[35927]: from='client.25514 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-15T12:58:45.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:44 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2712627556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:45.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:44 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/4007970857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T12:58:45.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:44 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/302204628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T12:58:45.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.212+0000 7fd22ecff640 1 -- 172.21.15.2:0/3017717817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd22802cf60 con 0x7fd2280ff7d0 2024-03-15T12:58:45.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.213+0000 7fd225ffb640 1 -- 172.21.15.2:0/3017717817 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd21c09a080 con 0x7fd2280ff7d0 2024-03-15T12:58:45.199 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T12:58:45.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.215+0000 7fd22ecff640 1 -- 172.21.15.2:0/3017717817 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd2180a1ac0 msgr2=0x7fd2180a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.215+0000 7fd22ecff640 1 --2- 172.21.15.2:0/3017717817 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd2180a1ac0 0x7fd2180a3f80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd214000c00 tx=0x7fd214005fd0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.215+0000 7fd22ecff640 1 -- 172.21.15.2:0/3017717817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2280ff7d0 msgr2=0x7fd2281975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.215+0000 7fd22ecff640 1 --2- 172.21.15.2:0/3017717817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2280ff7d0 0x7fd2281975b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00e9c0 tx=0x7fd21c00edc0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.215+0000 7fd22ecff640 1 -- 172.21.15.2:0/3017717817 shutdown_connections 2024-03-15T12:58:45.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.215+0000 7fd22ecff640 1 --2- 172.21.15.2:0/3017717817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd228103160 0x7fd22819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.215+0000 7fd22ecff640 1 --2- 172.21.15.2:0/3017717817 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fd2180a1ac0 0x7fd2180a3f80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.215+0000 7fd22ecff640 1 --2- 172.21.15.2:0/3017717817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2281027a0 0x7fd228197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.215+0000 7fd22ecff640 1 --2- 172.21.15.2:0/3017717817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2280ff7d0 0x7fd2281975b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.215+0000 7fd22ecff640 1 -- 172.21.15.2:0/3017717817 >> 172.21.15.2:0/3017717817 conn(0x7fd2280fb120 msgr2=0x7fd228105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:45.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.215+0000 7fd22ecff640 1 -- 172.21.15.2:0/3017717817 shutdown_connections 2024-03-15T12:58:45.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.215+0000 7fd22ecff640 1 -- 172.21.15.2:0/3017717817 wait complete. 2024-03-15T12:58:45.214 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:58:45.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.331+0000 7f11d64ff640 1 -- 172.21.15.2:0/4289637513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11d00ff7e0 msgr2=0x7f11d00ffc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.331+0000 7f11d64ff640 1 --2- 172.21.15.2:0/4289637513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11d00ff7e0 0x7f11d00ffc00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009900 tx=0x7f11c002bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:45.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.331+0000 7f11d64ff640 1 -- 172.21.15.2:0/4289637513 shutdown_connections 2024-03-15T12:58:45.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.331+0000 7f11d64ff640 1 --2- 172.21.15.2:0/4289637513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d0103170 0x7f11d0109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.331+0000 7f11d64ff640 1 --2- 172.21.15.2:0/4289637513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11d01027b0 0x7f11d0102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.331+0000 7f11d64ff640 1 --2- 172.21.15.2:0/4289637513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11d00ff7e0 0x7f11d00ffc00 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.331+0000 7f11d64ff640 1 -- 172.21.15.2:0/4289637513 >> 172.21.15.2:0/4289637513 conn(0x7f11d00fb110 msgr2=0x7f11d00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:45.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.331+0000 7f11d64ff640 1 -- 172.21.15.2:0/4289637513 shutdown_connections 2024-03-15T12:58:45.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.331+0000 7f11d64ff640 1 -- 172.21.15.2:0/4289637513 wait complete. 2024-03-15T12:58:45.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.331+0000 7f11d64ff640 1 Processor -- start 2024-03-15T12:58:45.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.331+0000 7f11d64ff640 1 -- start start 2024-03-15T12:58:45.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11d64ff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d00ff7e0 0x7f11d01975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11d64ff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11d01027b0 0x7f11d0197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11d64ff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11d0103170 0x7f11d019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11d64ff640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d019f0d0 con 0x7f11d01027b0 2024-03-15T12:58:45.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11d64ff640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d019f240 con 0x7f11d0103170 2024-03-15T12:58:45.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11d64ff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d019f400 con 0x7f11d00ff7e0 2024-03-15T12:58:45.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11cf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11d01027b0 0x7f11d0197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11cf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11d01027b0 0x7f11d0197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43828/0 (socket says 172.21.15.2:43828) 2024-03-15T12:58:45.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11cf7fe640 1 -- 172.21.15.2:0/1811295235 learned_addr learned my addr 172.21.15.2:0/1811295235 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:45.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11cffff640 1 --2- 172.21.15.2:0/1811295235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d00ff7e0 0x7f11d01975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11d4a9c640 1 --2- 172.21.15.2:0/1811295235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11d0103170 0x7f11d019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11cf7fe640 1 -- 172.21.15.2:0/1811295235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d00ff7e0 msgr2=0x7f11d01975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11cf7fe640 1 --2- 172.21.15.2:0/1811295235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d00ff7e0 0x7f11d01975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11cf7fe640 1 -- 172.21.15.2:0/1811295235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11d0103170 msgr2=0x7f11d019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11cf7fe640 1 --2- 172.21.15.2:0/1811295235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11d0103170 0x7f11d019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11cf7fe640 1 -- 172.21.15.2:0/1811295235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c0009560 con 0x7f11d01027b0 2024-03-15T12:58:45.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11cf7fe640 1 --2- 172.21.15.2:0/1811295235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11d01027b0 0x7f11d0197b20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f11c400e9f0 tx=0x7f11c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:45.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.332+0000 7f11cd7fa640 1 -- 172.21.15.2:0/1811295235 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4009700 con 0x7f11d01027b0 2024-03-15T12:58:45.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.333+0000 7f11d64ff640 1 -- 172.21.15.2:0/1811295235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d019f6e0 con 0x7f11d01027b0 2024-03-15T12:58:45.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.333+0000 7f11cd7fa640 1 -- 172.21.15.2:0/1811295235 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f11c400cd80 con 0x7f11d01027b0 2024-03-15T12:58:45.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.333+0000 7f11cd7fa640 1 -- 172.21.15.2:0/1811295235 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4018920 con 0x7f11d01027b0 2024-03-15T12:58:45.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.333+0000 7f11d64ff640 1 -- 172.21.15.2:0/1811295235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d019fc40 con 0x7f11d01027b0 2024-03-15T12:58:45.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.333+0000 7f11d64ff640 1 -- 172.21.15.2:0/1811295235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d01088e0 con 0x7f11d01027b0 2024-03-15T12:58:45.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.334+0000 7f11cd7fa640 1 -- 172.21.15.2:0/1811295235 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f11c4018a80 con 0x7f11d01027b0 2024-03-15T12:58:45.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.335+0000 7f11cd7fa640 1 --2- 172.21.15.2:0/1811295235 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f11b80a1ac0 0x7f11b80a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.335+0000 7f11cffff640 1 --2- 172.21.15.2:0/1811295235 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f11b80a1ac0 0x7f11b80a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.335+0000 7f11cd7fa640 1 -- 172.21.15.2:0/1811295235 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f11c4014070 con 0x7f11d01027b0 2024-03-15T12:58:45.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.336+0000 7f11cffff640 1 --2- 172.21.15.2:0/1811295235 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f11b80a1ac0 0x7f11b80a3f80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f11c002c740 tx=0x7f11c0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:45.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.339+0000 7f11cd7fa640 1 -- 172.21.15.2:0/1811295235 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f11c4096eb0 con 0x7f11d01027b0 2024-03-15T12:58:45.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.558+0000 7f11d64ff640 1 -- 172.21.15.2:0/1811295235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11d010cf10 con 0x7f11d01027b0 2024-03-15T12:58:45.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.559+0000 7f11cd7fa640 1 -- 172.21.15.2:0/1811295235 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11453 (secure 0 0 0) 0x7f11c409a570 con 0x7f11d01027b0 2024-03-15T12:58:45.545 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:45.545 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T12:58:37.814324+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24434,24539,24460,24464,24500]},"id":1}]} 2024-03-15T12:58:45.545 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 29 2024-03-15T12:58:45.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.561+0000 7f11d64ff640 1 -- 172.21.15.2:0/1811295235 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f11b80a1ac0 msgr2=0x7f11b80a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.561+0000 7f11d64ff640 1 --2- 172.21.15.2:0/1811295235 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f11b80a1ac0 0x7f11b80a3f80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f11c002c740 tx=0x7f11c0005fd0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.561+0000 7f11d64ff640 1 -- 172.21.15.2:0/1811295235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11d01027b0 msgr2=0x7f11d0197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.561+0000 7f11d64ff640 1 --2- 172.21.15.2:0/1811295235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11d01027b0 0x7f11d0197b20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f11c400e9f0 tx=0x7f11c400eeb0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.561+0000 7f11d64ff640 1 -- 172.21.15.2:0/1811295235 shutdown_connections 2024-03-15T12:58:45.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.561+0000 7f11d64ff640 1 --2- 172.21.15.2:0/1811295235 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f11b80a1ac0 0x7f11b80a3f80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.561+0000 7f11d64ff640 1 --2- 172.21.15.2:0/1811295235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11d0103170 0x7f11d019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.561+0000 7f11d64ff640 1 --2- 172.21.15.2:0/1811295235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f11d01027b0 0x7f11d0197b20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.561+0000 7f11d64ff640 1 --2- 172.21.15.2:0/1811295235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11d00ff7e0 0x7f11d01975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.561+0000 7f11d64ff640 1 -- 172.21.15.2:0/1811295235 >> 172.21.15.2:0/1811295235 conn(0x7f11d00fb110 msgr2=0x7f11d0105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:45.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.561+0000 7f11d64ff640 1 -- 172.21.15.2:0/1811295235 shutdown_connections 2024-03-15T12:58:45.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.561+0000 7f11d64ff640 1 -- 172.21.15.2:0/1811295235 wait complete. 2024-03-15T12:58:45.558 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T12:58:45.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.675+0000 7f773b3c7640 1 -- 172.21.15.2:0/507237265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7734100410 msgr2=0x7f7734100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.675+0000 7f773b3c7640 1 --2- 172.21.15.2:0/507237265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7734100410 0x7f7734100830 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7730009860 tx=0x7f773002bb70 comp rx=0 tx=0).stop 2024-03-15T12:58:45.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.675+0000 7f773b3c7640 1 -- 172.21.15.2:0/507237265 shutdown_connections 2024-03-15T12:58:45.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.675+0000 7f773b3c7640 1 --2- 172.21.15.2:0/507237265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7734101fc0 0x7f7734108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.675+0000 7f773b3c7640 1 --2- 172.21.15.2:0/507237265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7734101600 0x7f7734101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.675+0000 7f773b3c7640 1 --2- 172.21.15.2:0/507237265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7734100410 0x7f7734100830 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.675+0000 7f773b3c7640 1 -- 172.21.15.2:0/507237265 >> 172.21.15.2:0/507237265 conn(0x7f77340fb980 msgr2=0x7f77340fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:45.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.675+0000 7f773b3c7640 1 -- 172.21.15.2:0/507237265 shutdown_connections 2024-03-15T12:58:45.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.675+0000 7f773b3c7640 1 -- 172.21.15.2:0/507237265 wait complete. 2024-03-15T12:58:45.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f773b3c7640 1 Processor -- start 2024-03-15T12:58:45.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f773b3c7640 1 -- start start 2024-03-15T12:58:45.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f773b3c7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7734100410 0x7f7734196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f773b3c7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7734101600 0x7f7734196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f773b3c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7734101fc0 0x7f773419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f773b3c7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773419e000 con 0x7f7734101fc0 2024-03-15T12:58:45.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f773b3c7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773419e170 con 0x7f7734101600 2024-03-15T12:58:45.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f773b3c7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773419e330 con 0x7f7734100410 2024-03-15T12:58:45.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f7739964640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7734101fc0 0x7f773419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f7739163640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7734100410 0x7f7734196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f7739964640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7734101fc0 0x7f773419dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43838/0 (socket says 172.21.15.2:43838) 2024-03-15T12:58:45.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f7738962640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7734101600 0x7f7734196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f7739964640 1 -- 172.21.15.2:0/4139947053 learned_addr learned my addr 172.21.15.2:0/4139947053 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:45.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f7739964640 1 -- 172.21.15.2:0/4139947053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7734100410 msgr2=0x7f7734196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f7739964640 1 --2- 172.21.15.2:0/4139947053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7734100410 0x7f7734196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f7739964640 1 -- 172.21.15.2:0/4139947053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7734101600 msgr2=0x7f7734196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f7739964640 1 --2- 172.21.15.2:0/4139947053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7734101600 0x7f7734196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f7739964640 1 -- 172.21.15.2:0/4139947053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7724009490 con 0x7f7734101fc0 2024-03-15T12:58:45.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f7738962640 1 --2- 172.21.15.2:0/4139947053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7734101600 0x7f7734196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:45.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.676+0000 7f7739163640 1 --2- 172.21.15.2:0/4139947053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7734100410 0x7f7734196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:45.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.677+0000 7f7739964640 1 --2- 172.21.15.2:0/4139947053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7734101fc0 0x7f773419dac0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f772c009fd0 tx=0x7f772c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:45.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.677+0000 7f772a7fc640 1 -- 172.21.15.2:0/4139947053 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c009b50 con 0x7f7734101fc0 2024-03-15T12:58:45.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.677+0000 7f772a7fc640 1 -- 172.21.15.2:0/4139947053 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f772c003970 con 0x7f7734101fc0 2024-03-15T12:58:45.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.677+0000 7f773b3c7640 1 -- 172.21.15.2:0/4139947053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7730009490 con 0x7f7734101fc0 2024-03-15T12:58:45.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.677+0000 7f772a7fc640 1 -- 172.21.15.2:0/4139947053 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c0047d0 con 0x7f7734101fc0 2024-03-15T12:58:45.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.677+0000 7f773b3c7640 1 -- 172.21.15.2:0/4139947053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773419e940 con 0x7f7734101fc0 2024-03-15T12:58:45.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.678+0000 7f773b3c7640 1 -- 172.21.15.2:0/4139947053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7734107800 con 0x7f7734101fc0 2024-03-15T12:58:45.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.679+0000 7f772a7fc640 1 -- 172.21.15.2:0/4139947053 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f772c00a040 con 0x7f7734101fc0 2024-03-15T12:58:45.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.679+0000 7f772a7fc640 1 --2- 172.21.15.2:0/4139947053 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f77200a1ac0 0x7f77200a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.680+0000 7f7739163640 1 --2- 172.21.15.2:0/4139947053 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f77200a1ac0 0x7f77200a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.680+0000 7f772a7fc640 1 -- 172.21.15.2:0/4139947053 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f772c0cd850 con 0x7f7734101fc0 2024-03-15T12:58:45.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.680+0000 7f7739163640 1 --2- 172.21.15.2:0/4139947053 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f77200a1ac0 0x7f77200a3f80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f773002c180 tx=0x7f7730034b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:45.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.684+0000 7f772a7fc640 1 -- 172.21.15.2:0/4139947053 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f772c0965c0 con 0x7f7734101fc0 2024-03-15T12:58:45.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.868+0000 7f773b3c7640 1 -- 172.21.15.2:0/4139947053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f773402d000 con 0x7f7734101fc0 2024-03-15T12:58:45.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.869+0000 7f772a7fc640 1 -- 172.21.15.2:0/4139947053 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24526 (secure 0 0 0) 0x7f772c099c80 con 0x7f7734101fc0 2024-03-15T12:58:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:58:45.856 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":91,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T12:57:19.826817+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T12:58:45.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.872+0000 7f773b3c7640 1 -- 172.21.15.2:0/4139947053 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f77200a1ac0 msgr2=0x7f77200a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.872+0000 7f773b3c7640 1 --2- 172.21.15.2:0/4139947053 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f77200a1ac0 0x7f77200a3f80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f773002c180 tx=0x7f7730034b60 comp rx=0 tx=0).stop 2024-03-15T12:58:45.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.872+0000 7f773b3c7640 1 -- 172.21.15.2:0/4139947053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7734101fc0 msgr2=0x7f773419dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.872+0000 7f773b3c7640 1 --2- 172.21.15.2:0/4139947053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7734101fc0 0x7f773419dac0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f772c009fd0 tx=0x7f772c00c800 comp rx=0 tx=0).stop 2024-03-15T12:58:45.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.872+0000 7f773b3c7640 1 -- 172.21.15.2:0/4139947053 shutdown_connections 2024-03-15T12:58:45.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.872+0000 7f773b3c7640 1 --2- 172.21.15.2:0/4139947053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7734101fc0 0x7f773419dac0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.872+0000 7f773b3c7640 1 --2- 172.21.15.2:0/4139947053 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f77200a1ac0 0x7f77200a3f80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.872+0000 7f773b3c7640 1 --2- 172.21.15.2:0/4139947053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7734101600 0x7f7734196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.872+0000 7f773b3c7640 1 --2- 172.21.15.2:0/4139947053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7734100410 0x7f7734196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.872+0000 7f773b3c7640 1 -- 172.21.15.2:0/4139947053 >> 172.21.15.2:0/4139947053 conn(0x7f77340fb980 msgr2=0x7f77341049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:45.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.872+0000 7f773b3c7640 1 -- 172.21.15.2:0/4139947053 shutdown_connections 2024-03-15T12:58:45.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:45.872+0000 7f773b3c7640 1 -- 172.21.15.2:0/4139947053 wait complete. 2024-03-15T12:58:45.867 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:45 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3048144860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:45.868 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:45 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3017717817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:45.868 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:45 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1811295235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:45.869 DEBUG:tasks.fwd_scrub:Yielding 2024-03-15T12:58:45.869 INFO:teuthology.run_tasks:Running task exec... 2024-03-15T12:58:45.881 INFO:teuthology.task.exec:Executing custom commands... 2024-03-15T12:58:45.881 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-15T12:58:45.881 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-15T12:58:45.881 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-15T12:58:45.881 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T12:58:45.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:58:45.882 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T12:58:45.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.000+0000 7f35263d4640 1 -- 172.21.15.2:0/1159310033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3520100410 msgr2=0x7f3520100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.000+0000 7f35263d4640 1 --2- 172.21.15.2:0/1159310033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3520100410 0x7f3520100830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3510009900 tx=0x7f351002bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:45.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.000+0000 7f35263d4640 1 -- 172.21.15.2:0/1159310033 shutdown_connections 2024-03-15T12:58:45.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.000+0000 7f35263d4640 1 --2- 172.21.15.2:0/1159310033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3520101fc0 0x7f3520108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.000+0000 7f35263d4640 1 --2- 172.21.15.2:0/1159310033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3520101600 0x7f3520101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.000+0000 7f35263d4640 1 --2- 172.21.15.2:0/1159310033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3520100410 0x7f3520100830 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.000+0000 7f35263d4640 1 -- 172.21.15.2:0/1159310033 >> 172.21.15.2:0/1159310033 conn(0x7f35200fb980 msgr2=0x7f35200fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:45.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.000+0000 7f35263d4640 1 -- 172.21.15.2:0/1159310033 shutdown_connections 2024-03-15T12:58:45.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.000+0000 7f35263d4640 1 -- 172.21.15.2:0/1159310033 wait complete. 2024-03-15T12:58:45.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.000+0000 7f35263d4640 1 Processor -- start 2024-03-15T12:58:45.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.000+0000 7f35263d4640 1 -- start start 2024-03-15T12:58:45.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f35263d4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3520100410 0x7f3520069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f35263d4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3520101600 0x7f352019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f35263d4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3520101fc0 0x7f352019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f35263d4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352019f040 con 0x7f3520101600 2024-03-15T12:58:45.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f35263d4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352019f1b0 con 0x7f3520100410 2024-03-15T12:58:45.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f35263d4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352019f370 con 0x7f3520101fc0 2024-03-15T12:58:45.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f351ffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3520100410 0x7f3520069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f3524971640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3520101fc0 0x7f352019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f3524971640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3520101fc0 0x7f352019e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:52990/0 (socket says 172.21.15.2:52990) 2024-03-15T12:58:45.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f351f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3520101600 0x7f352019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f3524971640 1 -- 172.21.15.2:0/1020629562 learned_addr learned my addr 172.21.15.2:0/1020629562 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:45.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f3524971640 1 -- 172.21.15.2:0/1020629562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3520100410 msgr2=0x7f3520069690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f3524971640 1 --2- 172.21.15.2:0/1020629562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3520100410 0x7f3520069690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f3524971640 1 -- 172.21.15.2:0/1020629562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3520101600 msgr2=0x7f352019c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f3524971640 1 --2- 172.21.15.2:0/1020629562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3520101600 0x7f352019c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f3524971640 1 -- 172.21.15.2:0/1020629562 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3514009490 con 0x7f3520101fc0 2024-03-15T12:58:45.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.001+0000 7f3524971640 1 --2- 172.21.15.2:0/1020629562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3520101fc0 0x7f352019e8b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f351800efd0 tx=0x7f351800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:45.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.002+0000 7f351d7fa640 1 -- 172.21.15.2:0/1020629562 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518009b40 con 0x7f3520101fc0 2024-03-15T12:58:45.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.002+0000 7f351d7fa640 1 -- 172.21.15.2:0/1020629562 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3518003b70 con 0x7f3520101fc0 2024-03-15T12:58:45.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.002+0000 7f351d7fa640 1 -- 172.21.15.2:0/1020629562 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35180108f0 con 0x7f3520101fc0 2024-03-15T12:58:45.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.002+0000 7f35263d4640 1 -- 172.21.15.2:0/1020629562 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3518009840 con 0x7f3520101fc0 2024-03-15T12:58:45.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.002+0000 7f35263d4640 1 -- 172.21.15.2:0/1020629562 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3510009560 con 0x7f3520101fc0 2024-03-15T12:58:45.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.003+0000 7f35263d4640 1 -- 172.21.15.2:0/1020629562 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3520107750 con 0x7f3520101fc0 2024-03-15T12:58:45.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.003+0000 7f41851d7640 1 -- 172.21.15.2:0/2187056963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41801015f0 msgr2=0x7f4180101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.003+0000 7f41851d7640 1 --2- 172.21.15.2:0/2187056963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41801015f0 0x7f4180101a70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4168009860 tx=0x7f416802bb70 comp rx=0 tx=0).stop 2024-03-15T12:58:45.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.003+0000 7f41851d7640 1 -- 172.21.15.2:0/2187056963 shutdown_connections 2024-03-15T12:58:45.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.003+0000 7f41851d7640 1 --2- 172.21.15.2:0/2187056963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4180101fb0 0x7f4180108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.003+0000 7f41851d7640 1 --2- 172.21.15.2:0/2187056963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41801015f0 0x7f4180101a70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.003+0000 7f41851d7640 1 --2- 172.21.15.2:0/2187056963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4180100400 0x7f4180100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.003+0000 7f41851d7640 1 -- 172.21.15.2:0/2187056963 >> 172.21.15.2:0/2187056963 conn(0x7f41800fb990 msgr2=0x7f41800fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:45.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.003+0000 7f41851d7640 1 -- 172.21.15.2:0/2187056963 shutdown_connections 2024-03-15T12:58:45.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.003+0000 7f41851d7640 1 -- 172.21.15.2:0/2187056963 wait complete. 2024-03-15T12:58:45.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f41851d7640 1 Processor -- start 2024-03-15T12:58:45.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f41851d7640 1 -- start start 2024-03-15T12:58:45.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f351d7fa640 1 -- 172.21.15.2:0/1020629562 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f3518003ce0 con 0x7f3520101fc0 2024-03-15T12:58:45.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f41851d7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4180100400 0x7f4180199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f41851d7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41801015f0 0x7f4180199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f41851d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4180101fb0 0x7f41801a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f41851d7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41801a1280 con 0x7f4180101fb0 2024-03-15T12:58:45.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f41851d7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41801a13f0 con 0x7f41801015f0 2024-03-15T12:58:45.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f41851d7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41801a15b0 con 0x7f4180100400 2024-03-15T12:58:45.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f417f59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4180101fb0 0x7f41801a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f417ed9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4180100400 0x7f4180199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f4177fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41801015f0 0x7f4180199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f417f59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4180101fb0 0x7f41801a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43886/0 (socket says 172.21.15.2:43886) 2024-03-15T12:58:45.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f417f59e640 1 -- 172.21.15.2:0/4074353321 learned_addr learned my addr 172.21.15.2:0/4074353321 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:45.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f417f59e640 1 -- 172.21.15.2:0/4074353321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4180100400 msgr2=0x7f4180199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f417f59e640 1 --2- 172.21.15.2:0/4074353321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4180100400 0x7f4180199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f417f59e640 1 -- 172.21.15.2:0/4074353321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41801015f0 msgr2=0x7f4180199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:45.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f417f59e640 1 --2- 172.21.15.2:0/4074353321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41801015f0 0x7f4180199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:45.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f351d7fa640 1 --2- 172.21.15.2:0/1020629562 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f35080a1bf0 0x7f35080a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:45.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f417f59e640 1 -- 172.21.15.2:0/4074353321 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4168009490 con 0x7f4180101fb0 2024-03-15T12:58:45.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.004+0000 7f351ffff640 1 --2- 172.21.15.2:0/1020629562 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f35080a1bf0 0x7f35080a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:45.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.005+0000 7f351d7fa640 1 -- 172.21.15.2:0/1020629562 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f3518014070 con 0x7f3520101fc0 2024-03-15T12:58:45.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.005+0000 7f417f59e640 1 --2- 172.21.15.2:0/4074353321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4180101fb0 0x7f41801a0d40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f417800eb10 tx=0x7f417800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:45.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.005+0000 7f417cd99640 1 -- 172.21.15.2:0/4074353321 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417800ce20 con 0x7f4180101fb0 2024-03-15T12:58:45.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.005+0000 7f417cd99640 1 -- 172.21.15.2:0/4074353321 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4178005020 con 0x7f4180101fb0 2024-03-15T12:58:45.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.005+0000 7f41851d7640 1 -- 172.21.15.2:0/4074353321 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41801a1890 con 0x7f4180101fb0 2024-03-15T12:58:45.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.005+0000 7f417cd99640 1 -- 172.21.15.2:0/4074353321 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4178010670 con 0x7f4180101fb0 2024-03-15T12:58:45.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.005+0000 7f41851d7640 1 -- 172.21.15.2:0/4074353321 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41801a1d20 con 0x7f4180101fb0 2024-03-15T12:58:45.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.005+0000 7f351ffff640 1 --2- 172.21.15.2:0/1020629562 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f35080a1bf0 0x7f35080a40b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f351002c740 tx=0x7f3510005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:45.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.007+0000 7f417cd99640 1 -- 172.21.15.2:0/4074353321 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f4178005190 con 0x7f4180101fb0 2024-03-15T12:58:45.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.007+0000 7f4187db2740 1 Processor -- start 2024-03-15T12:58:46.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.007+0000 7f4187db2740 1 -- start start 2024-03-15T12:58:46.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f4187db2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c09534c0 0x55d3c09538e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f4187db2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0954690 0x55d3c0954b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f4187db2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c0955050 0x55d3c095b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f4187db2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c08c2f90 con 0x55d3c09534c0 2024-03-15T12:58:46.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f4187db2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c08c04c0 con 0x55d3c0955050 2024-03-15T12:58:46.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f4187db2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c08bbca0 con 0x55d3c0954690 2024-03-15T12:58:46.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f417cd99640 1 --2- 172.21.15.2:0/4074353321 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f416c0a1ac0 0x7f416c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f4177fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c09534c0 0x55d3c09538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f4177fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c09534c0 0x55d3c09538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43898/0 (socket says 172.21.15.2:43898) 2024-03-15T12:58:46.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f4177fff640 1 -- 172.21.15.2:0/3525331109 learned_addr learned my addr 172.21.15.2:0/3525331109 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:46.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f417ed9d640 1 --2- 172.21.15.2:0/3525331109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0954690 0x55d3c0954b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f417f59e640 1 --2- 172.21.15.2:0/4074353321 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f416c0a1ac0 0x7f416c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f351d7fa640 1 -- 172.21.15.2:0/1020629562 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3518096b70 con 0x7f3520101fc0 2024-03-15T12:58:46.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f417cd99640 1 -- 172.21.15.2:0/4074353321 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f4178014070 con 0x7f4180101fb0 2024-03-15T12:58:46.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f4177fff640 1 --2- 172.21.15.2:0/3525331109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c0955050 0x55d3c095b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f417ed9d640 1 -- 172.21.15.2:0/3525331109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c0955050 msgr2=0x55d3c095b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f417ed9d640 1 --2- 172.21.15.2:0/3525331109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c0955050 0x55d3c095b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f417ed9d640 1 -- 172.21.15.2:0/3525331109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c09534c0 msgr2=0x55d3c09538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f417ed9d640 1 --2- 172.21.15.2:0/3525331109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c09534c0 0x55d3c09538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f417ed9d640 1 -- 172.21.15.2:0/3525331109 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3c087e0b0 con 0x55d3c0954690 2024-03-15T12:58:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f4177fff640 1 --2- 172.21.15.2:0/3525331109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c09534c0 0x55d3c09538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f417f59e640 1 --2- 172.21.15.2:0/4074353321 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f416c0a1ac0 0x7f416c0a3f80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4178096ea0 tx=0x7f417806c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.008+0000 7f417ed9d640 1 --2- 172.21.15.2:0/3525331109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0954690 0x55d3c0954b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4170005fd0 tx=0x7f4170011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36b1eb4edad82d3d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.009+0000 7f41757fa640 1 -- 172.21.15.2:0/3525331109 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170009db0 con 0x55d3c0954690 2024-03-15T12:58:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.009+0000 7f41757fa640 1 -- 172.21.15.2:0/3525331109 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4170004500 con 0x55d3c0954690 2024-03-15T12:58:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.009+0000 7f41757fa640 1 -- 172.21.15.2:0/3525331109 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41700196b0 con 0x55d3c0954690 2024-03-15T12:58:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.009+0000 7f4187db2740 1 -- 172.21.15.2:0/3525331109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0954690 msgr2=0x55d3c0954b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.009+0000 7f4187db2740 1 --2- 172.21.15.2:0/3525331109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0954690 0x55d3c0954b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4170005fd0 tx=0x7f4170011810 comp rx=0 tx=0).stop 2024-03-15T12:58:46.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 1 -- 172.21.15.2:0/3525331109 shutdown_connections 2024-03-15T12:58:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 1 --2- 172.21.15.2:0/3525331109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c0955050 0x55d3c095b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 1 --2- 172.21.15.2:0/3525331109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0954690 0x55d3c0954b10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 1 --2- 172.21.15.2:0/3525331109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c09534c0 0x55d3c09538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 1 -- 172.21.15.2:0/3525331109 >> 172.21.15.2:0/3525331109 conn(0x55d3c094e6a0 msgr2=0x55d3c094eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 1 -- 172.21.15.2:0/3525331109 shutdown_connections 2024-03-15T12:58:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 1 -- 172.21.15.2:0/3525331109 wait complete. 2024-03-15T12:58:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 1 -- start start 2024-03-15T12:58:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 1 Processor -- start 2024-03-15T12:58:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c09534c0 0x55d3c095d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c095dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c0955050 0x55d3c0964c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c0860f40 con 0x55d3c0954690 2024-03-15T12:58:46.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c058f4b0 con 0x55d3c0955050 2024-03-15T12:58:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c0841670 con 0x55d3c09534c0 2024-03-15T12:58:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4187db2740 10 client.0 _ll_register_callbacks cb 0x55d3c08aef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4177fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c09534c0 0x55d3c095d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4177fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c09534c0 0x55d3c095d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:53026/0 (socket says 172.21.15.2:53026) 2024-03-15T12:58:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f417ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c095dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4177fff640 1 -- 172.21.15.2:0/270666628 learned_addr learned my addr 172.21.15.2:0/270666628 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.010+0000 7f4177fff640 1 --2- 172.21.15.2:0/270666628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c0955050 0x55d3c0964c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f4177fff640 1 -- 172.21.15.2:0/270666628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c0955050 msgr2=0x55d3c0964c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f4177fff640 1 --2- 172.21.15.2:0/270666628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c0955050 0x55d3c0964c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f4177fff640 1 -- 172.21.15.2:0/270666628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 msgr2=0x55d3c095dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f4177fff640 1 --2- 172.21.15.2:0/270666628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c095dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f4177fff640 1 -- 172.21.15.2:0/270666628 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4170009890 con 0x55d3c09534c0 2024-03-15T12:58:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f4177fff640 1 --2- 172.21.15.2:0/270666628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c09534c0 0x55d3c095d700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f416802c0b0 tx=0x7f4168005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b407c06e01dd9d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f41667fc640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T12:58:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f41667fc640 1 -- 172.21.15.2:0/270666628 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416802cb30 con 0x55d3c09534c0 2024-03-15T12:58:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f4187db2740 1 -- 172.21.15.2:0/270666628 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3c0895f40 con 0x55d3c09534c0 2024-03-15T12:58:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f41667fc640 1 -- 172.21.15.2:0/270666628 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f4168032600 con 0x55d3c09534c0 2024-03-15T12:58:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f41667fc640 1 -- 172.21.15.2:0/270666628 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41680327c0 con 0x55d3c09534c0 2024-03-15T12:58:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f4187db2740 10 client.25504 fetch_fsmap learned FSMap version 29 2024-03-15T12:58:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f4187db2740 1 -- 172.21.15.2:0/270666628 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d3c08ba320 con 0x55d3c09534c0 2024-03-15T12:58:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f41667fc640 1 -- 172.21.15.2:0/270666628 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55d3c0895f40 con 0x55d3c09534c0 2024-03-15T12:58:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f41667fc640 1 -- 172.21.15.2:0/270666628 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4168003b20 con 0x55d3c09534c0 2024-03-15T12:58:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f4187db2740 10 client.25504 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:58:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.011+0000 7f4187db2740 10 client.25504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:58:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.012+0000 7f4187db2740 20 client.25504 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.012+0000 7f4187db2740 1 --2- 172.21.15.2:0/270666628 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d3c09698a0 0x55d3c096bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.012+0000 7f4187db2740 4 client.25504 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:58:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.012+0000 7f4187db2740 1 -- 172.21.15.2:0/270666628 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3c08b2310 con 0x55d3c09698a0 2024-03-15T12:58:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.012+0000 7f417ed9d640 1 --2- 172.21.15.2:0/270666628 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d3c09698a0 0x55d3c096bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.012+0000 7f41667fc640 10 client.25504 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.012+0000 7f417ed9d640 1 --2- 172.21.15.2:0/270666628 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d3c09698a0 0x55d3c096bcc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d3bcfc86ea005df server_cookie=de4c89986c8c4418 in_seq=0 out_seq=0 2024-03-15T12:58:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.013+0000 7f41667fc640 1 -- 172.21.15.2:0/270666628 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d3c08b2310 con 0x55d3c09698a0 2024-03-15T12:58:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.013+0000 7f41667fc640 10 client.25504 handle_command_reply: tid=0 2024-03-15T12:58:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.013+0000 7f4187db2740 1 client.25504 shutdown 2024-03-15T12:58:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.013+0000 7f4187db2740 1 -- 172.21.15.2:0/270666628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c09534c0 msgr2=0x55d3c095d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.013+0000 7f4187db2740 1 --2- 172.21.15.2:0/270666628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c09534c0 0x55d3c095d700 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f416802c0b0 tx=0x7f4168005f90 comp rx=0 tx=0).stop 2024-03-15T12:58:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.013+0000 7f4187db2740 1 -- 172.21.15.2:0/270666628 shutdown_connections 2024-03-15T12:58:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.013+0000 7f4187db2740 1 --2- 172.21.15.2:0/270666628 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d3c09698a0 0x55d3c096bcc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.013+0000 7f4187db2740 1 --2- 172.21.15.2:0/270666628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c0955050 0x55d3c0964c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.013+0000 7f4187db2740 1 --2- 172.21.15.2:0/270666628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c095dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.013+0000 7f41667fc640 0 client.25504 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.013+0000 7f4187db2740 1 --2- 172.21.15.2:0/270666628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c09534c0 0x55d3c095d700 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.013+0000 7f4187db2740 1 -- 172.21.15.2:0/270666628 >> 172.21.15.2:0/270666628 conn(0x55d3c094e6a0 msgr2=0x55d3c094d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:46.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.013+0000 7f4187db2740 1 -- 172.21.15.2:0/270666628 shutdown_connections 2024-03-15T12:58:46.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.013+0000 7f4187db2740 1 -- 172.21.15.2:0/270666628 wait complete. 2024-03-15T12:58:46.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.013+0000 7f4187db2740 20 client.25504 trim_cache size 0 max 16384 2024-03-15T12:58:46.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.044+0000 7f4187db2740 1 Processor -- start 2024-03-15T12:58:46.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.044+0000 7f4187db2740 1 -- start start 2024-03-15T12:58:46.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.044+0000 7f4187db2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c09534c0 0x55d3c095dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.044+0000 7f4187db2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c0964c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.044+0000 7f4187db2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0955050 0x55d3c0952010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.044+0000 7f4187db2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c0957960 con 0x55d3c0954690 2024-03-15T12:58:46.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.044+0000 7f4187db2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c095cf70 con 0x55d3c09534c0 2024-03-15T12:58:46.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.044+0000 7f4187db2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c094eee0 con 0x55d3c0955050 2024-03-15T12:58:46.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.044+0000 7f417ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c0964c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.044+0000 7f417ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c0964c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43912/0 (socket says 172.21.15.2:43912) 2024-03-15T12:58:46.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.044+0000 7f417ed9d640 1 -- 172.21.15.2:0/1888403905 learned_addr learned my addr 172.21.15.2:0/1888403905 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:46.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.044+0000 7f4177fff640 1 --2- 172.21.15.2:0/1888403905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c09534c0 0x55d3c095dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.045+0000 7f4177fff640 1 --2- 172.21.15.2:0/1888403905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0955050 0x55d3c0952010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.045+0000 7f417ed9d640 1 -- 172.21.15.2:0/1888403905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0955050 msgr2=0x55d3c0952010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.045+0000 7f417ed9d640 1 --2- 172.21.15.2:0/1888403905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0955050 0x55d3c0952010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.045+0000 7f417ed9d640 1 -- 172.21.15.2:0/1888403905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c09534c0 msgr2=0x55d3c095dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.045+0000 7f417ed9d640 1 --2- 172.21.15.2:0/1888403905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c09534c0 0x55d3c095dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.045+0000 7f417ed9d640 1 -- 172.21.15.2:0/1888403905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3c0856c00 con 0x55d3c0954690 2024-03-15T12:58:46.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.045+0000 7f417ed9d640 1 --2- 172.21.15.2:0/1888403905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c0964c40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f417001c040 tx=0x7f4170009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ef44eaf56f96f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:46.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.045+0000 7f4174ff9640 1 -- 172.21.15.2:0/1888403905 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170003810 con 0x55d3c0954690 2024-03-15T12:58:46.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.045+0000 7f4174ff9640 1 -- 172.21.15.2:0/1888403905 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4170005510 con 0x55d3c0954690 2024-03-15T12:58:46.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.045+0000 7f4174ff9640 1 -- 172.21.15.2:0/1888403905 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41700049f0 con 0x55d3c0954690 2024-03-15T12:58:46.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.046+0000 7f4187db2740 1 -- 172.21.15.2:0/1888403905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 msgr2=0x55d3c0964c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.046+0000 7f4187db2740 1 --2- 172.21.15.2:0/1888403905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c0964c40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f417001c040 tx=0x7f4170009ed0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.046+0000 7f4187db2740 1 -- 172.21.15.2:0/1888403905 shutdown_connections 2024-03-15T12:58:46.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.046+0000 7f4187db2740 1 --2- 172.21.15.2:0/1888403905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0955050 0x55d3c0952010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.046+0000 7f4187db2740 1 --2- 172.21.15.2:0/1888403905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c0964c40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.046+0000 7f4187db2740 1 --2- 172.21.15.2:0/1888403905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c09534c0 0x55d3c095dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.046+0000 7f4187db2740 1 -- 172.21.15.2:0/1888403905 >> 172.21.15.2:0/1888403905 conn(0x55d3c094e6a0 msgr2=0x55d3c0951900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:46.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.046+0000 7f4187db2740 1 -- 172.21.15.2:0/1888403905 shutdown_connections 2024-03-15T12:58:46.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.046+0000 7f4187db2740 1 -- 172.21.15.2:0/1888403905 wait complete. 2024-03-15T12:58:46.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.046+0000 7f4187db2740 1 -- start start 2024-03-15T12:58:46.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.046+0000 7f4187db2740 1 Processor -- start 2024-03-15T12:58:46.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f4187db2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c09534c0 0x55d3c09649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f4187db2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c0961460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f4187db2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0955050 0x55d3c09619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f4187db2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c0961ee0 con 0x55d3c0954690 2024-03-15T12:58:46.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f4187db2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c0962050 con 0x55d3c09534c0 2024-03-15T12:58:46.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f4187db2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c09c5190 con 0x55d3c0955050 2024-03-15T12:58:46.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f4187db2740 10 client.0 _ll_register_callbacks cb 0x7f41780093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:46.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f417ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c0961460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f417ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c0961460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43924/0 (socket says 172.21.15.2:43924) 2024-03-15T12:58:46.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f417ed9d640 1 -- 172.21.15.2:0/3303370989 learned_addr learned my addr 172.21.15.2:0/3303370989 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:46.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f4177fff640 1 --2- 172.21.15.2:0/3303370989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c09534c0 0x55d3c09649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f4177fff640 1 --2- 172.21.15.2:0/3303370989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0955050 0x55d3c09619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f417ed9d640 1 -- 172.21.15.2:0/3303370989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0955050 msgr2=0x55d3c09619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f417ed9d640 1 --2- 172.21.15.2:0/3303370989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0955050 0x55d3c09619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f417ed9d640 1 -- 172.21.15.2:0/3303370989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c09534c0 msgr2=0x55d3c09649c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f417ed9d640 1 --2- 172.21.15.2:0/3303370989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c09534c0 0x55d3c09649c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f417ed9d640 1 -- 172.21.15.2:0/3303370989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3c087e0b0 con 0x55d3c0954690 2024-03-15T12:58:46.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f417ed9d640 1 --2- 172.21.15.2:0/3303370989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c0961460 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4170013040 tx=0x7f4170005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a67331d70ff48f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:46.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f41667fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:46.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f41667fc640 1 -- 172.21.15.2:0/3303370989 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170005680 con 0x55d3c0954690 2024-03-15T12:58:46.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f41667fc640 1 -- 172.21.15.2:0/3303370989 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f4170003810 con 0x55d3c0954690 2024-03-15T12:58:46.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f4187db2740 1 -- 172.21.15.2:0/3303370989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3c08e1900 con 0x55d3c0954690 2024-03-15T12:58:46.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.047+0000 7f41667fc640 1 -- 172.21.15.2:0/3303370989 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4170003c80 con 0x55d3c0954690 2024-03-15T12:58:46.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.048+0000 7f4187db2740 10 client.15708 fetch_fsmap learned FSMap version 29 2024-03-15T12:58:46.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.048+0000 7f4187db2740 1 -- 172.21.15.2:0/3303370989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d3c087c910 con 0x55d3c0954690 2024-03-15T12:58:46.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.048+0000 7f41667fc640 1 -- 172.21.15.2:0/3303370989 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55d3c08e1900 con 0x55d3c0954690 2024-03-15T12:58:46.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.048+0000 7f41667fc640 1 -- 172.21.15.2:0/3303370989 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f417002eb10 con 0x55d3c0954690 2024-03-15T12:58:46.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.048+0000 7f4187db2740 10 client.15708 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:58:46.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.048+0000 7f4187db2740 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:58:46.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.048+0000 7f4187db2740 20 client.15708 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:46.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.048+0000 7f4187db2740 1 --2- 172.21.15.2:0/3303370989 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d3c09c68a0 0x55d3c09c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.048+0000 7f4187db2740 4 client.15708 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T12:58:46.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.048+0000 7f4187db2740 1 -- 172.21.15.2:0/3303370989 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d3c05bb860 con 0x55d3c09c68a0 2024-03-15T12:58:46.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.048+0000 7f4177fff640 1 --2- 172.21.15.2:0/3303370989 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d3c09c68a0 0x55d3c09c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.048+0000 7f4177fff640 1 --2- 172.21.15.2:0/3303370989 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d3c09c68a0 0x55d3c09c8cc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c9ff7eda21dda68 server_cookie=b2e55d50150fadd1 in_seq=0 out_seq=0 2024-03-15T12:58:46.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.048+0000 7f41667fc640 10 client.15708 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:46.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.050+0000 7f41667fc640 1 -- 172.21.15.2:0/3303370989 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d3c05bb860 con 0x55d3c09c68a0 2024-03-15T12:58:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.050+0000 7f41667fc640 10 client.15708 handle_command_reply: tid=0 2024-03-15T12:58:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.050+0000 7f4187db2740 1 client.15708 shutdown 2024-03-15T12:58:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.050+0000 7f4187db2740 1 -- 172.21.15.2:0/3303370989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 msgr2=0x55d3c0961460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.050+0000 7f4187db2740 1 --2- 172.21.15.2:0/3303370989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c0961460 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4170013040 tx=0x7f4170005df0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.050+0000 7f4187db2740 1 -- 172.21.15.2:0/3303370989 shutdown_connections 2024-03-15T12:58:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.050+0000 7f4187db2740 1 --2- 172.21.15.2:0/3303370989 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d3c09c68a0 0x55d3c09c8cc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.050+0000 7f4187db2740 1 --2- 172.21.15.2:0/3303370989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3c0955050 0x55d3c09619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.050+0000 7f4187db2740 1 --2- 172.21.15.2:0/3303370989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3c0954690 0x55d3c0961460 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.050+0000 7f41667fc640 0 client.15708 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.050+0000 7f4187db2740 1 --2- 172.21.15.2:0/3303370989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c09534c0 0x55d3c09649c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.050+0000 7f4187db2740 1 -- 172.21.15.2:0/3303370989 >> 172.21.15.2:0/3303370989 conn(0x55d3c094e6a0 msgr2=0x55d3c0952190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:46.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.050+0000 7f4187db2740 1 -- 172.21.15.2:0/3303370989 shutdown_connections 2024-03-15T12:58:46.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.050+0000 7f4187db2740 1 -- 172.21.15.2:0/3303370989 wait complete. 2024-03-15T12:58:46.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.050+0000 7f4187db2740 20 client.15708 trim_cache size 0 max 16384 2024-03-15T12:58:46.043 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:58:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T12:58:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "1ee30b19-9800-479e-9b5c-70c0e97d9193", 2024-03-15T12:58:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T12:58:46.043 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:58:46.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.051+0000 7f41851d7640 1 -- 172.21.15.2:0/4074353321 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f416c0a1ac0 msgr2=0x7f416c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.051+0000 7f41851d7640 1 --2- 172.21.15.2:0/4074353321 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f416c0a1ac0 0x7f416c0a3f80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4178096ea0 tx=0x7f417806c040 comp rx=0 tx=0).stop 2024-03-15T12:58:46.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.051+0000 7f41851d7640 1 -- 172.21.15.2:0/4074353321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4180101fb0 msgr2=0x7f41801a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.051+0000 7f41851d7640 1 --2- 172.21.15.2:0/4074353321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4180101fb0 0x7f41801a0d40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f417800eb10 tx=0x7f417800ef10 comp rx=0 tx=0).stop 2024-03-15T12:58:46.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.051+0000 7f41851d7640 1 -- 172.21.15.2:0/4074353321 shutdown_connections 2024-03-15T12:58:46.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.051+0000 7f41851d7640 1 --2- 172.21.15.2:0/4074353321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4180101fb0 0x7f41801a0d40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.051+0000 7f41851d7640 1 --2- 172.21.15.2:0/4074353321 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f416c0a1ac0 0x7f416c0a3f80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.051+0000 7f41851d7640 1 --2- 172.21.15.2:0/4074353321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41801015f0 0x7f4180199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.051+0000 7f41851d7640 1 --2- 172.21.15.2:0/4074353321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4180100400 0x7f4180199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.051+0000 7f41851d7640 1 -- 172.21.15.2:0/4074353321 >> 172.21.15.2:0/4074353321 conn(0x7f41800fb990 msgr2=0x7f41800fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:46.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.051+0000 7f41851d7640 1 -- 172.21.15.2:0/4074353321 shutdown_connections 2024-03-15T12:58:46.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.051+0000 7f41851d7640 1 -- 172.21.15.2:0/4074353321 wait complete. 2024-03-15T12:58:46.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T12:58:46.048 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T12:58:46.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:45 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3048144860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:46.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:45 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3017717817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:46.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:45 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1811295235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:46.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:45 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3048144860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T12:58:46.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:45 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3017717817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T12:58:46.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:45 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1811295235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:58:46.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.167+0000 7f86e4264640 1 -- 172.21.15.2:0/2304129146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86dc102710 msgr2=0x7f86dc102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.167+0000 7f86e4264640 1 --2- 172.21.15.2:0/2304129146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86dc102710 0x7f86dc102b90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009900 tx=0x7f86cc02bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:46.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.167+0000 7f86e4264640 1 -- 172.21.15.2:0/2304129146 shutdown_connections 2024-03-15T12:58:46.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.167+0000 7f86e4264640 1 --2- 172.21.15.2:0/2304129146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86dc1030d0 0x7f86dc109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.167+0000 7f86e4264640 1 --2- 172.21.15.2:0/2304129146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86dc102710 0x7f86dc102b90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.167+0000 7f86e4264640 1 --2- 172.21.15.2:0/2304129146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86dc101520 0x7f86dc101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.167+0000 7f86e4264640 1 -- 172.21.15.2:0/2304129146 >> 172.21.15.2:0/2304129146 conn(0x7f86dc0fcab0 msgr2=0x7f86dc0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:46.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.167+0000 7f86e4264640 1 -- 172.21.15.2:0/2304129146 shutdown_connections 2024-03-15T12:58:46.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.167+0000 7f86e4264640 1 -- 172.21.15.2:0/2304129146 wait complete. 2024-03-15T12:58:46.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e4264640 1 Processor -- start 2024-03-15T12:58:46.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e4264640 1 -- start start 2024-03-15T12:58:46.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e4264640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86dc101520 0x7f86dc0752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e4264640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86dc102710 0x7f86dc0738e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e4264640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86dc1030d0 0x7f86dc073e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e4264640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc074580 con 0x7f86dc1030d0 2024-03-15T12:58:46.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e4264640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc0746f0 con 0x7f86dc102710 2024-03-15T12:58:46.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e4264640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1a15b0 con 0x7f86dc101520 2024-03-15T12:58:46.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e2801640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86dc1030d0 0x7f86dc073e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e2801640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86dc1030d0 0x7f86dc073e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43944/0 (socket says 172.21.15.2:43944) 2024-03-15T12:58:46.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e2000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86dc101520 0x7f86dc0752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e2801640 1 -- 172.21.15.2:0/3168570877 learned_addr learned my addr 172.21.15.2:0/3168570877 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:46.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e17ff640 1 --2- 172.21.15.2:0/3168570877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86dc102710 0x7f86dc0738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e2801640 1 -- 172.21.15.2:0/3168570877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86dc101520 msgr2=0x7f86dc0752a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e2801640 1 --2- 172.21.15.2:0/3168570877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86dc101520 0x7f86dc0752a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e2801640 1 -- 172.21.15.2:0/3168570877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86dc102710 msgr2=0x7f86dc0738e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e2801640 1 --2- 172.21.15.2:0/3168570877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86dc102710 0x7f86dc0738e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e2801640 1 -- 172.21.15.2:0/3168570877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d8009490 con 0x7f86dc1030d0 2024-03-15T12:58:46.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.168+0000 7f86e2000640 1 --2- 172.21.15.2:0/3168570877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86dc101520 0x7f86dc0752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:46.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.169+0000 7f86e2801640 1 --2- 172.21.15.2:0/3168570877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86dc1030d0 0x7f86dc073e20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f86d400db20 tx=0x7f86d400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:46.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.169+0000 7f86d2ffd640 1 -- 172.21.15.2:0/3168570877 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d400bea0 con 0x7f86dc1030d0 2024-03-15T12:58:46.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.169+0000 7f86e4264640 1 -- 172.21.15.2:0/3168570877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86cc009560 con 0x7f86dc1030d0 2024-03-15T12:58:46.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.169+0000 7f86d2ffd640 1 -- 172.21.15.2:0/3168570877 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f86d4009940 con 0x7f86dc1030d0 2024-03-15T12:58:46.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.169+0000 7f86d2ffd640 1 -- 172.21.15.2:0/3168570877 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d40106c0 con 0x7f86dc1030d0 2024-03-15T12:58:46.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.169+0000 7f86e4264640 1 -- 172.21.15.2:0/3168570877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc1a1bb0 con 0x7f86dc1030d0 2024-03-15T12:58:46.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.171+0000 7f86d2ffd640 1 -- 172.21.15.2:0/3168570877 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 136329+0+0 (secure 0 0 0) 0x7f86d4010820 con 0x7f86dc1030d0 2024-03-15T12:58:46.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.171+0000 7f86e6e40740 1 Processor -- start 2024-03-15T12:58:46.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e6e40740 1 -- start start 2024-03-15T12:58:46.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e6e40740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db742d0 0x55a01db746f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e6e40740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db797c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e6e40740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db79e30 0x55a01db7c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86d2ffd640 1 --2- 172.21.15.2:0/3168570877 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f86c80a1ac0 0x7f86c80a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e6e40740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01dae4280 con 0x55a01db74cc0 2024-03-15T12:58:46.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e6e40740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01dae17b0 con 0x55a01db79e30 2024-03-15T12:58:46.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e6e40740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01dadcf90 con 0x55a01db742d0 2024-03-15T12:58:46.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e17ff640 1 --2- 172.21.15.2:0/3168570877 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f86c80a1ac0 0x7f86c80a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e2000640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db797c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43956/0 (socket says 172.21.15.2:43956) 2024-03-15T12:58:46.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e2000640 1 -- 172.21.15.2:0/2718977030 learned_addr learned my addr 172.21.15.2:0/2718977030 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:46.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e2801640 1 --2- 172.21.15.2:0/2718977030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db79e30 0x55a01db7c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e17ff640 1 --2- 172.21.15.2:0/2718977030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db742d0 0x55a01db746f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86d2ffd640 1 -- 172.21.15.2:0/3168570877 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f86d40cd5d0 con 0x7f86dc1030d0 2024-03-15T12:58:46.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e2801640 1 -- 172.21.15.2:0/2718977030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db742d0 msgr2=0x55a01db746f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e2801640 1 --2- 172.21.15.2:0/2718977030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db742d0 0x55a01db746f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e2801640 1 -- 172.21.15.2:0/2718977030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 msgr2=0x55a01db797c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e2801640 1 --2- 172.21.15.2:0/2718977030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db797c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e2801640 1 -- 172.21.15.2:0/2718977030 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a01da9b570 con 0x55a01db79e30 2024-03-15T12:58:46.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.172+0000 7f86e2000640 1 --2- 172.21.15.2:0/2718977030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db797c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:46.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.173+0000 7f86e17ff640 1 --2- 172.21.15.2:0/3168570877 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f86c80a1ac0 0x7f86c80a3f80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f86cc000c00 tx=0x7f86cc02bc30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:46.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.173+0000 7f86e2801640 1 --2- 172.21.15.2:0/2718977030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db79e30 0x55a01db7c2c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f86d40cabe0 tx=0x7f86d401e7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4523a35b38e616d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:46.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.173+0000 7f86c7fff640 1 -- 172.21.15.2:0/2718977030 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d40cb030 con 0x55a01db79e30 2024-03-15T12:58:46.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.173+0000 7f86c7fff640 1 -- 172.21.15.2:0/2718977030 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f86d4099c30 con 0x55a01db79e30 2024-03-15T12:58:46.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.173+0000 7f86c7fff640 1 -- 172.21.15.2:0/2718977030 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d405add0 con 0x55a01db79e30 2024-03-15T12:58:46.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.173+0000 7f86e6e40740 1 -- 172.21.15.2:0/2718977030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db79e30 msgr2=0x55a01db7c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.173+0000 7f86e6e40740 1 --2- 172.21.15.2:0/2718977030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db79e30 0x55a01db7c2c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f86d40cabe0 tx=0x7f86d401e7b0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.173+0000 7f86e6e40740 1 -- 172.21.15.2:0/2718977030 shutdown_connections 2024-03-15T12:58:46.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.173+0000 7f86e6e40740 1 --2- 172.21.15.2:0/2718977030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db79e30 0x55a01db7c2c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.173+0000 7f86e6e40740 1 --2- 172.21.15.2:0/2718977030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db797c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.173+0000 7f86e6e40740 1 --2- 172.21.15.2:0/2718977030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db742d0 0x55a01db746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.173+0000 7f86e6e40740 1 -- 172.21.15.2:0/2718977030 >> 172.21.15.2:0/2718977030 conn(0x55a01db6f7f0 msgr2=0x55a01db6fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:46.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e6e40740 1 -- 172.21.15.2:0/2718977030 shutdown_connections 2024-03-15T12:58:46.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e6e40740 1 -- 172.21.15.2:0/2718977030 wait complete. 2024-03-15T12:58:46.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e6e40740 1 -- start start 2024-03-15T12:58:46.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e6e40740 1 Processor -- start 2024-03-15T12:58:46.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e6e40740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db742d0 0x55a01db7e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e6e40740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db7e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e6e40740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db79e30 0x55a01db857a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e6e40740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01d7b0430 con 0x55a01db74cc0 2024-03-15T12:58:46.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e6e40740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01d76ce00 con 0x55a01db742d0 2024-03-15T12:58:46.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e6e40740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01da61fd0 con 0x55a01db79e30 2024-03-15T12:58:46.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e6e40740 10 client.0 _ll_register_callbacks cb 0x55a01dab7fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:46.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e2000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db742d0 0x55a01db7e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e17ff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db7e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e2000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db742d0 0x55a01db7e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:56492/0 (socket says 172.21.15.2:56492) 2024-03-15T12:58:46.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e2000640 1 -- 172.21.15.2:0/607381142 learned_addr learned my addr 172.21.15.2:0/607381142 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:46.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.174+0000 7f86e2801640 1 --2- 172.21.15.2:0/607381142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db79e30 0x55a01db857a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.175+0000 7f86e2000640 1 -- 172.21.15.2:0/607381142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db79e30 msgr2=0x55a01db857a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.175+0000 7f86e2000640 1 --2- 172.21.15.2:0/607381142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db79e30 0x55a01db857a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.175+0000 7f86e2000640 1 -- 172.21.15.2:0/607381142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 msgr2=0x55a01db7e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.175+0000 7f86e2000640 1 --2- 172.21.15.2:0/607381142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db7e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.175+0000 7f86e2000640 1 -- 172.21.15.2:0/607381142 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc1a1bb0 con 0x55a01db742d0 2024-03-15T12:58:46.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.175+0000 7f86e17ff640 1 --2- 172.21.15.2:0/607381142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db7e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:46.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.175+0000 7f86e2000640 1 --2- 172.21.15.2:0/607381142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db742d0 0x55a01db7e220 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f86d8012c90 tx=0x7f86d800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91ad1a73b09520db server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:46.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.175+0000 7f86c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T12:58:46.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.175+0000 7f86c57fa640 1 -- 172.21.15.2:0/607381142 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8017070 con 0x55a01db742d0 2024-03-15T12:58:46.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.175+0000 7f86c57fa640 1 -- 172.21.15.2:0/607381142 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f86d8005020 con 0x55a01db742d0 2024-03-15T12:58:46.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.175+0000 7f86e6e40740 1 -- 172.21.15.2:0/607381142 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a01d7dc7e0 con 0x55a01db742d0 2024-03-15T12:58:46.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.175+0000 7f86c57fa640 1 -- 172.21.15.2:0/607381142 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86d80122e0 con 0x55a01db742d0 2024-03-15T12:58:46.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.175+0000 7f86e6e40740 10 client.25592 fetch_fsmap learned FSMap version 29 2024-03-15T12:58:46.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.175+0000 7f86e6e40740 1 -- 172.21.15.2:0/607381142 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a01dadb610 con 0x55a01db742d0 2024-03-15T12:58:46.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.175+0000 7f86c57fa640 1 -- 172.21.15.2:0/607381142 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55a01d7dc7e0 con 0x55a01db742d0 2024-03-15T12:58:46.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.176+0000 7f86c57fa640 1 -- 172.21.15.2:0/607381142 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f86d801c470 con 0x55a01db742d0 2024-03-15T12:58:46.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.176+0000 7f86e6e40740 10 client.25592 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:58:46.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.176+0000 7f86e6e40740 10 client.25592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:58:46.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.176+0000 7f86e6e40740 20 client.25592 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:46.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.176+0000 7f86e6e40740 1 --2- 172.21.15.2:0/607381142 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a01db8a460 0x55a01db8c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.176+0000 7f86e6e40740 4 client.25592 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:58:46.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.176+0000 7f86e6e40740 1 -- 172.21.15.2:0/607381142 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a01d819720 con 0x55a01db8a460 2024-03-15T12:58:46.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.176+0000 7f86e17ff640 1 --2- 172.21.15.2:0/607381142 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a01db8a460 0x55a01db8c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.176+0000 7f86c57fa640 10 client.25592 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:46.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.176+0000 7f86e17ff640 1 --2- 172.21.15.2:0/607381142 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a01db8a460 0x55a01db8c8a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990f36570736a2b1 server_cookie=da93c67dcd4d4b1e in_seq=0 out_seq=0 2024-03-15T12:58:46.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.177+0000 7f86c57fa640 1 -- 172.21.15.2:0/607381142 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a01d819720 con 0x55a01db8a460 2024-03-15T12:58:46.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.177+0000 7f86c57fa640 10 client.25592 handle_command_reply: tid=0 2024-03-15T12:58:46.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.177+0000 7f86e6e40740 1 client.25592 shutdown 2024-03-15T12:58:46.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.177+0000 7f86e6e40740 1 -- 172.21.15.2:0/607381142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db742d0 msgr2=0x55a01db7e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.177+0000 7f86e6e40740 1 --2- 172.21.15.2:0/607381142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db742d0 0x55a01db7e220 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f86d8012c90 tx=0x7f86d800f790 comp rx=0 tx=0).stop 2024-03-15T12:58:46.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.178+0000 7f86e6e40740 1 -- 172.21.15.2:0/607381142 shutdown_connections 2024-03-15T12:58:46.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.178+0000 7f86e6e40740 1 --2- 172.21.15.2:0/607381142 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a01db8a460 0x55a01db8c8a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.178+0000 7f86e6e40740 1 --2- 172.21.15.2:0/607381142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db79e30 0x55a01db857a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.178+0000 7f86e6e40740 1 --2- 172.21.15.2:0/607381142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db7e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.178+0000 7f86e6e40740 1 --2- 172.21.15.2:0/607381142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db742d0 0x55a01db7e220 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.178+0000 7f86c57fa640 0 client.25592 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:46.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.178+0000 7f86e6e40740 1 -- 172.21.15.2:0/607381142 >> 172.21.15.2:0/607381142 conn(0x55a01db6f7f0 msgr2=0x55a01db6ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:46.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.178+0000 7f86e6e40740 1 -- 172.21.15.2:0/607381142 shutdown_connections 2024-03-15T12:58:46.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.178+0000 7f86e6e40740 1 -- 172.21.15.2:0/607381142 wait complete. 2024-03-15T12:58:46.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.178+0000 7f86e6e40740 20 client.25592 trim_cache size 0 max 16384 2024-03-15T12:58:46.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e6e40740 1 Processor -- start 2024-03-15T12:58:46.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e6e40740 1 -- start start 2024-03-15T12:58:46.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e6e40740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db742d0 0x55a01db731b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e6e40740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db7d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e6e40740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db79e30 0x55a01db7dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e6e40740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01db750f0 con 0x55a01db74cc0 2024-03-15T12:58:46.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e6e40740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01db791d0 con 0x55a01db742d0 2024-03-15T12:58:46.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e6e40740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01db76770 con 0x55a01db79e30 2024-03-15T12:58:46.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e17ff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db7d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e17ff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db7d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43978/0 (socket says 172.21.15.2:43978) 2024-03-15T12:58:46.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e17ff640 1 -- 172.21.15.2:0/2758836455 learned_addr learned my addr 172.21.15.2:0/2758836455 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:46.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e2000640 1 --2- 172.21.15.2:0/2758836455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db742d0 0x55a01db731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e2801640 1 --2- 172.21.15.2:0/2758836455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db79e30 0x55a01db7dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e17ff640 1 -- 172.21.15.2:0/2758836455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db79e30 msgr2=0x55a01db7dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e17ff640 1 --2- 172.21.15.2:0/2758836455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db79e30 0x55a01db7dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e17ff640 1 -- 172.21.15.2:0/2758836455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db742d0 msgr2=0x55a01db731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e17ff640 1 --2- 172.21.15.2:0/2758836455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db742d0 0x55a01db731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e17ff640 1 -- 172.21.15.2:0/2758836455 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a01da77ac0 con 0x55a01db74cc0 2024-03-15T12:58:46.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.194+0000 7f86e17ff640 1 --2- 172.21.15.2:0/2758836455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db7d840 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009230 tx=0x7f86cc043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ddb5f9b235de4c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:46.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.195+0000 7f86c77fe640 1 -- 172.21.15.2:0/2758836455 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc048040 con 0x55a01db74cc0 2024-03-15T12:58:46.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.195+0000 7f86c77fe640 1 -- 172.21.15.2:0/2758836455 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f86cc049040 con 0x55a01db74cc0 2024-03-15T12:58:46.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.195+0000 7f86c77fe640 1 -- 172.21.15.2:0/2758836455 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0052b0 con 0x55a01db74cc0 2024-03-15T12:58:46.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.195+0000 7f86e6e40740 1 -- 172.21.15.2:0/2758836455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 msgr2=0x55a01db7d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.195+0000 7f86e6e40740 1 --2- 172.21.15.2:0/2758836455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db7d840 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009230 tx=0x7f86cc043000 comp rx=0 tx=0).stop 2024-03-15T12:58:46.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.195+0000 7f86e6e40740 1 -- 172.21.15.2:0/2758836455 shutdown_connections 2024-03-15T12:58:46.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.195+0000 7f86e6e40740 1 --2- 172.21.15.2:0/2758836455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db79e30 0x55a01db7dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.195+0000 7f86e6e40740 1 --2- 172.21.15.2:0/2758836455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db7d840 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.195+0000 7f86e6e40740 1 --2- 172.21.15.2:0/2758836455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db742d0 0x55a01db731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.195+0000 7f86e6e40740 1 -- 172.21.15.2:0/2758836455 >> 172.21.15.2:0/2758836455 conn(0x55a01db6f7f0 msgr2=0x55a01db72aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:46.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.195+0000 7f86e6e40740 1 -- 172.21.15.2:0/2758836455 shutdown_connections 2024-03-15T12:58:46.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.195+0000 7f86e6e40740 1 -- 172.21.15.2:0/2758836455 wait complete. 2024-03-15T12:58:46.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.196+0000 7f86e6e40740 1 -- start start 2024-03-15T12:58:46.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.196+0000 7f86e6e40740 1 Processor -- start 2024-03-15T12:58:46.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.196+0000 7f86e6e40740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db742d0 0x55a01db84a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.196+0000 7f86e6e40740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db83090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.196+0000 7f86e6e40740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db79e30 0x55a01db835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.196+0000 7f86e6e40740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01db85270 con 0x55a01db74cc0 2024-03-15T12:58:46.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.196+0000 7f86e6e40740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01db83b10 con 0x55a01db79e30 2024-03-15T12:58:46.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.196+0000 7f86e6e40740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01db83cd0 con 0x55a01db742d0 2024-03-15T12:58:46.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.196+0000 7f86e6e40740 10 client.0 _ll_register_callbacks cb 0x7f86d80128a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:58:46.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.196+0000 7f86e17ff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db83090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.196+0000 7f86e2801640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db79e30 0x55a01db835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.196+0000 7f86e17ff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db83090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43984/0 (socket says 172.21.15.2:43984) 2024-03-15T12:58:46.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.196+0000 7f86e17ff640 1 -- 172.21.15.2:0/4048444289 learned_addr learned my addr 172.21.15.2:0/4048444289 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:46.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.196+0000 7f86e2000640 1 --2- 172.21.15.2:0/4048444289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db742d0 0x55a01db84a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86e17ff640 1 -- 172.21.15.2:0/4048444289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db742d0 msgr2=0x55a01db84a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86e17ff640 1 --2- 172.21.15.2:0/4048444289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db742d0 0x55a01db84a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86e17ff640 1 -- 172.21.15.2:0/4048444289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db79e30 msgr2=0x55a01db835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:58:46.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86e17ff640 1 --2- 172.21.15.2:0/4048444289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db79e30 0x55a01db835d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86e17ff640 1 -- 172.21.15.2:0/4048444289 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc10a260 con 0x55a01db74cc0 2024-03-15T12:58:46.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86e2801640 1 --2- 172.21.15.2:0/4048444289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db79e30 0x55a01db835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:46.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86e17ff640 1 --2- 172.21.15.2:0/4048444289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db83090 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f86cc036040 tx=0x7f86cc02c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef15f7c35a62722 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:46.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:58:46.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86c57fa640 1 -- 172.21.15.2:0/4048444289 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc048040 con 0x55a01db74cc0 2024-03-15T12:58:46.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86c57fa640 1 -- 172.21.15.2:0/4048444289 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f86cc004de0 con 0x55a01db74cc0 2024-03-15T12:58:46.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86e6e40740 1 -- 172.21.15.2:0/4048444289 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86d4099c30 con 0x55a01db74cc0 2024-03-15T12:58:46.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86c57fa640 1 -- 172.21.15.2:0/4048444289 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86cc004fa0 con 0x55a01db74cc0 2024-03-15T12:58:46.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86e6e40740 10 client.15726 fetch_fsmap learned FSMap version 29 2024-03-15T12:58:46.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86e6e40740 1 -- 172.21.15.2:0/4048444289 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a01da9d6a0 con 0x55a01db74cc0 2024-03-15T12:58:46.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86c57fa640 1 -- 172.21.15.2:0/4048444289 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f86d4099c30 con 0x55a01db74cc0 2024-03-15T12:58:46.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.197+0000 7f86c57fa640 1 -- 172.21.15.2:0/4048444289 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f86cc003ac0 con 0x55a01db74cc0 2024-03-15T12:58:46.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.198+0000 7f86e6e40740 10 client.15726 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:58:46.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.198+0000 7f86e6e40740 10 client.15726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:58:46.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.198+0000 7f86e6e40740 20 client.15726 populate_metadata read hostname 'smithi002' 2024-03-15T12:58:46.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.198+0000 7f86e6e40740 1 --2- 172.21.15.2:0/4048444289 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a01dbe71e0 0x55a01dbe9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.198+0000 7f86e6e40740 4 client.15726 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T12:58:46.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.198+0000 7f86e6e40740 1 -- 172.21.15.2:0/4048444289 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a01d868890 con 0x55a01dbe71e0 2024-03-15T12:58:46.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.198+0000 7f86e2000640 1 --2- 172.21.15.2:0/4048444289 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a01dbe71e0 0x55a01dbe9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.198+0000 7f86c57fa640 10 client.15726 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:46.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.198+0000 7f86e2000640 1 --2- 172.21.15.2:0/4048444289 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a01dbe71e0 0x55a01dbe9600 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b965a32c99f3e3a server_cookie=21a39387edc7a178 in_seq=0 out_seq=0 2024-03-15T12:58:46.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.198+0000 7f86c57fa640 1 -- 172.21.15.2:0/4048444289 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a01d868890 con 0x55a01dbe71e0 2024-03-15T12:58:46.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.198+0000 7f86c57fa640 10 client.15726 handle_command_reply: tid=0 2024-03-15T12:58:46.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.198+0000 7f86e6e40740 1 client.15726 shutdown 2024-03-15T12:58:46.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.198+0000 7f86e6e40740 1 -- 172.21.15.2:0/4048444289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 msgr2=0x55a01db83090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e6e40740 1 --2- 172.21.15.2:0/4048444289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db83090 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f86cc036040 tx=0x7f86cc02c3c0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e6e40740 1 -- 172.21.15.2:0/4048444289 shutdown_connections 2024-03-15T12:58:46.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e6e40740 1 --2- 172.21.15.2:0/4048444289 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a01dbe71e0 0x55a01dbe9600 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e6e40740 1 --2- 172.21.15.2:0/4048444289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a01db79e30 0x55a01db835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e6e40740 1 --2- 172.21.15.2:0/4048444289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a01db74cc0 0x55a01db83090 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e6e40740 1 --2- 172.21.15.2:0/4048444289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a01db742d0 0x55a01db84a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86c57fa640 0 client.15726 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:58:46.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e6e40740 1 -- 172.21.15.2:0/4048444289 >> 172.21.15.2:0/4048444289 conn(0x55a01db6f7f0 msgr2=0x55a01db73330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:46.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e6e40740 1 -- 172.21.15.2:0/4048444289 shutdown_connections 2024-03-15T12:58:46.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e6e40740 1 -- 172.21.15.2:0/4048444289 wait complete. 2024-03-15T12:58:46.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e6e40740 20 client.15726 trim_cache size 0 max 16384 2024-03-15T12:58:46.193 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:58:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T12:58:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T12:58:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "1ee30b19-9800-479e-9b5c-70c0e97d9193": { 2024-03-15T12:58:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T12:58:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "1ee30b19-9800-479e-9b5c-70c0e97d9193", 2024-03-15T12:58:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T12:58:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T12:58:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T12:58:46.194 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:58:46.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e4264640 1 -- 172.21.15.2:0/3168570877 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f86c80a1ac0 msgr2=0x7f86c80a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e4264640 1 --2- 172.21.15.2:0/3168570877 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f86c80a1ac0 0x7f86c80a3f80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f86cc000c00 tx=0x7f86cc02bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:46.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e4264640 1 -- 172.21.15.2:0/3168570877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86dc1030d0 msgr2=0x7f86dc073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e4264640 1 --2- 172.21.15.2:0/3168570877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86dc1030d0 0x7f86dc073e20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f86d400db20 tx=0x7f86d400b260 comp rx=0 tx=0).stop 2024-03-15T12:58:46.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e4264640 1 -- 172.21.15.2:0/3168570877 shutdown_connections 2024-03-15T12:58:46.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e4264640 1 --2- 172.21.15.2:0/3168570877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86dc1030d0 0x7f86dc073e20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e4264640 1 --2- 172.21.15.2:0/3168570877 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f86c80a1ac0 0x7f86c80a3f80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e4264640 1 --2- 172.21.15.2:0/3168570877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86dc102710 0x7f86dc0738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e4264640 1 --2- 172.21.15.2:0/3168570877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86dc101520 0x7f86dc0752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e4264640 1 -- 172.21.15.2:0/3168570877 >> 172.21.15.2:0/3168570877 conn(0x7f86dc0fcab0 msgr2=0x7f86dc105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:46.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e4264640 1 -- 172.21.15.2:0/3168570877 shutdown_connections 2024-03-15T12:58:46.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.199+0000 7f86e4264640 1 -- 172.21.15.2:0/3168570877 wait complete. 2024-03-15T12:58:46.197 INFO:tasks.cephfs.filesystem:scrub status for tag:1ee30b19-9800-479e-9b5c-70c0e97d9193 - {'path': '/', 'tag': '1ee30b19-9800-479e-9b5c-70c0e97d9193', 'options': 'recursive,force'} 2024-03-15T12:58:46.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.263+0000 7f35263d4640 1 -- 172.21.15.2:0/1020629562 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f352002cf90 con 0x7f3520101fc0 2024-03-15T12:58:46.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.728+0000 7f351d7fa640 1 -- 172.21.15.2:0/1020629562 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mgrmap(e 20) v1 ==== 136346+0+0 (secure 0 0 0) 0x7f3518096f40 con 0x7f3520101fc0 2024-03-15T12:58:46.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.728+0000 7f351d7fa640 1 -- 172.21.15.2:0/1020629562 <== mon.1 v2:172.21.15.184:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v20) v1 ==== 92+0+0 (secure 0 0 0) 0x7f35180ce810 con 0x7f3520101fc0 2024-03-15T12:58:46.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.731+0000 7f35263d4640 1 -- 172.21.15.2:0/1020629562 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f35080a1bf0 msgr2=0x7f35080a40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.731+0000 7f35263d4640 1 --2- 172.21.15.2:0/1020629562 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f35080a1bf0 0x7f35080a40b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f351002c740 tx=0x7f3510005fd0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.731+0000 7f35263d4640 1 -- 172.21.15.2:0/1020629562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3520101fc0 msgr2=0x7f352019e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.731+0000 7f35263d4640 1 --2- 172.21.15.2:0/1020629562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3520101fc0 0x7f352019e8b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f351800efd0 tx=0x7f351800c6a0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.732+0000 7f35263d4640 1 -- 172.21.15.2:0/1020629562 shutdown_connections 2024-03-15T12:58:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.732+0000 7f35263d4640 1 --2- 172.21.15.2:0/1020629562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3520101fc0 0x7f352019e8b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.732+0000 7f35263d4640 1 --2- 172.21.15.2:0/1020629562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3520101600 0x7f352019c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.732+0000 7f35263d4640 1 --2- 172.21.15.2:0/1020629562 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f35080a1bf0 0x7f35080a40b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.732+0000 7f35263d4640 1 --2- 172.21.15.2:0/1020629562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3520100410 0x7f3520069690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.732+0000 7f35263d4640 1 -- 172.21.15.2:0/1020629562 >> 172.21.15.2:0/1020629562 conn(0x7f35200fb980 msgr2=0x7f3520104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.732+0000 7f35263d4640 1 -- 172.21.15.2:0/1020629562 shutdown_connections 2024-03-15T12:58:46.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.732+0000 7f35263d4640 1 -- 172.21.15.2:0/1020629562 wait complete. 2024-03-15T12:58:46.730 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-15T12:58:46.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.853+0000 7fbdf0f5c640 1 -- 172.21.15.2:0/1384512516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdec1030e0 msgr2=0x7fbdec109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.853+0000 7fbdf0f5c640 1 --2- 172.21.15.2:0/1384512516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdec1030e0 0x7fbdec109970 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009ca0 tx=0x7fbde402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.853+0000 7fbdf0f5c640 1 -- 172.21.15.2:0/1384512516 shutdown_connections 2024-03-15T12:58:46.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.853+0000 7fbdf0f5c640 1 --2- 172.21.15.2:0/1384512516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdec1030e0 0x7fbdec109970 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.853+0000 7fbdf0f5c640 1 --2- 172.21.15.2:0/1384512516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdec102720 0x7fbdec102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.853+0000 7fbdf0f5c640 1 --2- 172.21.15.2:0/1384512516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdec101530 0x7fbdec101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.853+0000 7fbdf0f5c640 1 -- 172.21.15.2:0/1384512516 >> 172.21.15.2:0/1384512516 conn(0x7fbdec0fca80 msgr2=0x7fbdec0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:46.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.853+0000 7fbdf0f5c640 1 -- 172.21.15.2:0/1384512516 shutdown_connections 2024-03-15T12:58:46.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.853+0000 7fbdf0f5c640 1 -- 172.21.15.2:0/1384512516 wait complete. 2024-03-15T12:58:46.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.854+0000 7fbdf0f5c640 1 Processor -- start 2024-03-15T12:58:46.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.854+0000 7fbdf0f5c640 1 -- start start 2024-03-15T12:58:46.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.854+0000 7fbdf0f5c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdec101530 0x7fbdec197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.854+0000 7fbdf0f5c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdec102720 0x7fbdec1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.854+0000 7fbdf0f5c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdec1030e0 0x7fbdec19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.854+0000 7fbdf0f5c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec19ef70 con 0x7fbdec1030e0 2024-03-15T12:58:46.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.854+0000 7fbdf0f5c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec19f0e0 con 0x7fbdec101530 2024-03-15T12:58:46.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.854+0000 7fbdf0f5c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec19f2a0 con 0x7fbdec102720 2024-03-15T12:58:46.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.854+0000 7fbdead9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdec1030e0 0x7fbdec19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.854+0000 7fbdead9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdec1030e0 0x7fbdec19ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44002/0 (socket says 172.21.15.2:44002) 2024-03-15T12:58:46.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.854+0000 7fbdead9d640 1 -- 172.21.15.2:0/3153408229 learned_addr learned my addr 172.21.15.2:0/3153408229 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:46.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.855+0000 7fbde3fff640 1 --2- 172.21.15.2:0/3153408229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdec102720 0x7fbdec1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.855+0000 7fbdea59c640 1 --2- 172.21.15.2:0/3153408229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdec101530 0x7fbdec197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:46.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.855+0000 7fbdead9d640 1 -- 172.21.15.2:0/3153408229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdec102720 msgr2=0x7fbdec1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.855+0000 7fbdead9d640 1 --2- 172.21.15.2:0/3153408229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdec102720 0x7fbdec1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.855+0000 7fbdead9d640 1 -- 172.21.15.2:0/3153408229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdec101530 msgr2=0x7fbdec197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:46.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.855+0000 7fbdead9d640 1 --2- 172.21.15.2:0/3153408229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdec101530 0x7fbdec197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:46.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.855+0000 7fbdead9d640 1 -- 172.21.15.2:0/3153408229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde4009900 con 0x7fbdec1030e0 2024-03-15T12:58:46.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.855+0000 7fbdea59c640 1 --2- 172.21.15.2:0/3153408229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdec101530 0x7fbdec197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:46.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.855+0000 7fbdead9d640 1 --2- 172.21.15.2:0/3153408229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdec1030e0 0x7fbdec19ea30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbde4005cc0 tx=0x7fbde4032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:46.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.855+0000 7fbde37fe640 1 -- 172.21.15.2:0/3153408229 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4034c90 con 0x7fbdec1030e0 2024-03-15T12:58:46.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.855+0000 7fbde37fe640 1 -- 172.21.15.2:0/3153408229 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbde4003e40 con 0x7fbdec1030e0 2024-03-15T12:58:46.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.855+0000 7fbdf0f5c640 1 -- 172.21.15.2:0/3153408229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdec19f520 con 0x7fbdec1030e0 2024-03-15T12:58:46.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.855+0000 7fbde37fe640 1 -- 172.21.15.2:0/3153408229 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde40048a0 con 0x7fbdec1030e0 2024-03-15T12:58:46.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.855+0000 7fbdf0f5c640 1 -- 172.21.15.2:0/3153408229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdec19f9d0 con 0x7fbdec1030e0 2024-03-15T12:58:46.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.857+0000 7fbdf0f5c640 1 -- 172.21.15.2:0/3153408229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdec1087f0 con 0x7fbdec1030e0 2024-03-15T12:58:46.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.857+0000 7fbde37fe640 1 -- 172.21.15.2:0/3153408229 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) v1 ==== 136346+0+0 (secure 0 0 0) 0x7fbde400a040 con 0x7fbdec1030e0 2024-03-15T12:58:46.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.858+0000 7fbde37fe640 1 --2- 172.21.15.2:0/3153408229 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbdd80a1bc0 0x7fbdd80a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:46.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.858+0000 7fbdea59c640 1 -- 172.21.15.2:0/3153408229 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbdd80a1bc0 msgr2=0x7fbdd80a4080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/343748862 2024-03-15T12:58:46.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.858+0000 7fbde37fe640 1 -- 172.21.15.2:0/3153408229 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fbde40f0600 con 0x7fbdec1030e0 2024-03-15T12:58:46.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.858+0000 7fbdea59c640 1 --2- 172.21.15.2:0/3153408229 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbdd80a1bc0 0x7fbdd80a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-15T12:58:46.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:46.862+0000 7fbde37fe640 1 -- 172.21.15.2:0/3153408229 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbde40b9420 con 0x7fbdec1030e0 2024-03-15T12:58:47.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.048+0000 7fbdf0f5c640 1 -- 172.21.15.2:0/3153408229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fbdec02cf60 con 0x7fbdec1030e0 2024-03-15T12:58:47.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.051+0000 7fbde37fe640 1 -- 172.21.15.2:0/3153408229 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbde40bcae0 con 0x7fbdec1030e0 2024-03-15T12:58:47.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.054+0000 7fbdf0f5c640 1 -- 172.21.15.2:0/3153408229 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbdd80a1bc0 msgr2=0x7fbdd80a4080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-15T12:58:47.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.054+0000 7fbdf0f5c640 1 --2- 172.21.15.2:0/3153408229 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbdd80a1bc0 0x7fbdd80a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.054+0000 7fbdf0f5c640 1 -- 172.21.15.2:0/3153408229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdec1030e0 msgr2=0x7fbdec19ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:47.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.054+0000 7fbdf0f5c640 1 --2- 172.21.15.2:0/3153408229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdec1030e0 0x7fbdec19ea30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbde4005cc0 tx=0x7fbde4032a10 comp rx=0 tx=0).stop 2024-03-15T12:58:47.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.054+0000 7fbdf0f5c640 1 -- 172.21.15.2:0/3153408229 shutdown_connections 2024-03-15T12:58:47.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.054+0000 7fbdf0f5c640 1 --2- 172.21.15.2:0/3153408229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdec1030e0 0x7fbdec19ea30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.054+0000 7fbdf0f5c640 1 --2- 172.21.15.2:0/3153408229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdec102720 0x7fbdec1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.054+0000 7fbdf0f5c640 1 --2- 172.21.15.2:0/3153408229 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fbdd80a1bc0 0x7fbdd80a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.054+0000 7fbdf0f5c640 1 --2- 172.21.15.2:0/3153408229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdec101530 0x7fbdec197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.054+0000 7fbdf0f5c640 1 -- 172.21.15.2:0/3153408229 >> 172.21.15.2:0/3153408229 conn(0x7fbdec0fca80 msgr2=0x7fbdec1059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:47.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.054+0000 7fbdf0f5c640 1 -- 172.21.15.2:0/3153408229 shutdown_connections 2024-03-15T12:58:47.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.054+0000 7fbdf0f5c640 1 -- 172.21.15.2:0/3153408229 wait complete. 2024-03-15T12:58:47.053 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:46 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: ignoring --setuser ceph since I am not root 2024-03-15T12:58:47.053 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:46 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: ignoring --setgroup ceph since I am not root 2024-03-15T12:58:47.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:46 smithi002 ceph-mon[32060]: pgmap v364: 129 pgs: 129 active+clean; 655 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-03-15T12:58:47.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:46 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/4139947053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:47.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:46 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T12:58:47.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:46 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T12:58:47.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:46 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T12:58:47.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:46 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T12:58:47.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:46 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1020629562' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-15T12:58:47.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:46 smithi002 ceph-mon[32060]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-15T12:58:47.055 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-15T12:58:47.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:46 smithi184 ceph-mon[35993]: pgmap v364: 129 pgs: 129 active+clean; 655 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-03-15T12:58:47.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:46 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/4139947053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:47.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:46 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T12:58:47.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:46 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T12:58:47.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:46 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T12:58:47.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:46 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T12:58:47.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:46 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1020629562' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-15T12:58:47.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:46 smithi184 ceph-mon[35993]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-15T12:58:47.059 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:46 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: ignoring --setuser ceph since I am not root 2024-03-15T12:58:47.060 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:46 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: ignoring --setgroup ceph since I am not root 2024-03-15T12:58:47.060 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:46 smithi184 ceph-mgr[37830]: -- 172.21.15.184:0/1286769379 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5594737ce1a0 con 0x5594737a4800 2024-03-15T12:58:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:46 smithi149 ceph-mon[35927]: pgmap v364: 129 pgs: 129 active+clean; 655 KiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-03-15T12:58:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:46 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/4139947053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T12:58:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:46 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T12:58:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:46 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T12:58:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:46 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T12:58:47.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:46 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T12:58:47.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:46 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1020629562' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-15T12:58:47.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:46 smithi149 ceph-mon[35927]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-15T12:58:47.111 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:46 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: ignoring --setuser ceph since I am not root 2024-03-15T12:58:47.111 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:46 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: ignoring --setgroup ceph since I am not root 2024-03-15T12:58:47.111 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:46 smithi149 ceph-mgr[37353]: -- 172.21.15.149:0/3851781922 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ad3e62a1a0 con 0x55ad3e600800 2024-03-15T12:58:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.175+0000 7f010b9d4640 1 -- 172.21.15.2:0/1860140337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0104102030 msgr2=0x7f0104104420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.175+0000 7f010b9d4640 1 --2- 172.21.15.2:0/1860140337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0104102030 0x7f0104104420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009900 tx=0x7f00f802bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.175+0000 7f010b9d4640 1 -- 172.21.15.2:0/1860140337 shutdown_connections 2024-03-15T12:58:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.175+0000 7f010b9d4640 1 --2- 172.21.15.2:0/1860140337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0104107290 0x7f0104109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.175+0000 7f010b9d4640 1 --2- 172.21.15.2:0/1860140337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0104104960 0x7f0104106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.175+0000 7f010b9d4640 1 --2- 172.21.15.2:0/1860140337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0104102030 0x7f0104104420 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.175+0000 7f010b9d4640 1 -- 172.21.15.2:0/1860140337 >> 172.21.15.2:0/1860140337 conn(0x7f01040fb980 msgr2=0x7f01040fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.175+0000 7f010b9d4640 1 -- 172.21.15.2:0/1860140337 shutdown_connections 2024-03-15T12:58:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.175+0000 7f010b9d4640 1 -- 172.21.15.2:0/1860140337 wait complete. 2024-03-15T12:58:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.175+0000 7f010b9d4640 1 Processor -- start 2024-03-15T12:58:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.175+0000 7f010b9d4640 1 -- start start 2024-03-15T12:58:47.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f010b9d4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0104102030 0x7f0104197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:47.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f010b9d4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0104104960 0x7f0104197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:47.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f010b9d4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0104107290 0x7f010419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:47.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f010b9d4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010419f030 con 0x7f0104107290 2024-03-15T12:58:47.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f010b9d4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010419f1a0 con 0x7f0104104960 2024-03-15T12:58:47.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f010b9d4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010419f360 con 0x7f0104102030 2024-03-15T12:58:47.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f0109f71640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0104107290 0x7f010419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:47.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f0108f6f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0104104960 0x7f0104197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:47.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f0109f71640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0104107290 0x7f010419eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44024/0 (socket says 172.21.15.2:44024) 2024-03-15T12:58:47.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f0108f6f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0104104960 0x7f0104197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:56542/0 (socket says 172.21.15.2:56542) 2024-03-15T12:58:47.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f0109f71640 1 -- 172.21.15.2:0/237961528 learned_addr learned my addr 172.21.15.2:0/237961528 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:47.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f0109770640 1 --2- 172.21.15.2:0/237961528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0104102030 0x7f0104197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:47.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f0109f71640 1 -- 172.21.15.2:0/237961528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0104102030 msgr2=0x7f0104197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:47.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f0109f71640 1 --2- 172.21.15.2:0/237961528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0104102030 0x7f0104197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f0109f71640 1 -- 172.21.15.2:0/237961528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0104104960 msgr2=0x7f0104197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:47.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f0109f71640 1 --2- 172.21.15.2:0/237961528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0104104960 0x7f0104197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f0109f71640 1 -- 172.21.15.2:0/237961528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f8009560 con 0x7f0104107290 2024-03-15T12:58:47.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f0108f6f640 1 --2- 172.21.15.2:0/237961528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0104104960 0x7f0104197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:47.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f0109770640 1 --2- 172.21.15.2:0/237961528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0104102030 0x7f0104197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:47.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f0109f71640 1 --2- 172.21.15.2:0/237961528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0104107290 0x7f010419eaf0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00eb10 tx=0x7f00fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:47.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f00f67fc640 1 -- 172.21.15.2:0/237961528 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc004b20 con 0x7f0104107290 2024-03-15T12:58:47.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f00f67fc640 1 -- 172.21.15.2:0/237961528 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f00fc004c80 con 0x7f0104107290 2024-03-15T12:58:47.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f010b9d4640 1 -- 172.21.15.2:0/237961528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010419f640 con 0x7f0104107290 2024-03-15T12:58:47.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.176+0000 7f00f67fc640 1 -- 172.21.15.2:0/237961528 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc010690 con 0x7f0104107290 2024-03-15T12:58:47.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.177+0000 7f010b9d4640 1 -- 172.21.15.2:0/237961528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010419fb50 con 0x7f0104107290 2024-03-15T12:58:47.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.178+0000 7f010b9d4640 1 -- 172.21.15.2:0/237961528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01040fd470 con 0x7f0104107290 2024-03-15T12:58:47.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.179+0000 7f00f67fc640 1 -- 172.21.15.2:0/237961528 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) v1 ==== 136346+0+0 (secure 0 0 0) 0x7f00fc004df0 con 0x7f0104107290 2024-03-15T12:58:47.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.179+0000 7f00f67fc640 1 --2- 172.21.15.2:0/237961528 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f00f00a1c40 0x7f00f00a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:47.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.179+0000 7f0109770640 1 -- 172.21.15.2:0/237961528 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f00f00a1c40 msgr2=0x7f00f00a4100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/343748862 2024-03-15T12:58:47.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.179+0000 7f00f67fc640 1 -- 172.21.15.2:0/237961528 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7f00fc014070 con 0x7f0104107290 2024-03-15T12:58:47.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.179+0000 7f0109770640 1 --2- 172.21.15.2:0/237961528 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f00f00a1c40 0x7f00f00a4100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-15T12:58:47.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.184+0000 7f00f67fc640 1 -- 172.21.15.2:0/237961528 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f00fc0d3020 con 0x7f0104107290 2024-03-15T12:58:47.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.371+0000 7f010b9d4640 1 -- 172.21.15.2:0/237961528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f010402d070 con 0x7f0104107290 2024-03-15T12:58:47.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.376+0000 7f00f67fc640 1 -- 172.21.15.2:0/237961528 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) v1 ==== 133+0+0 (secure 0 0 0) 0x7f00fc096bc0 con 0x7f0104107290 2024-03-15T12:58:47.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.379+0000 7f010b9d4640 1 -- 172.21.15.2:0/237961528 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f00f00a1c40 msgr2=0x7f00f00a4100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-15T12:58:47.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.379+0000 7f010b9d4640 1 --2- 172.21.15.2:0/237961528 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f00f00a1c40 0x7f00f00a4100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.379+0000 7f010b9d4640 1 -- 172.21.15.2:0/237961528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0104107290 msgr2=0x7f010419eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:47.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.379+0000 7f010b9d4640 1 --2- 172.21.15.2:0/237961528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0104107290 0x7f010419eaf0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00eb10 tx=0x7f00fc00c2d0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.379+0000 7f010b9d4640 1 -- 172.21.15.2:0/237961528 shutdown_connections 2024-03-15T12:58:47.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.379+0000 7f010b9d4640 1 --2- 172.21.15.2:0/237961528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0104107290 0x7f010419eaf0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.379+0000 7f010b9d4640 1 --2- 172.21.15.2:0/237961528 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7f00f00a1c40 0x7f00f00a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.379+0000 7f010b9d4640 1 --2- 172.21.15.2:0/237961528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0104104960 0x7f0104197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.379+0000 7f010b9d4640 1 --2- 172.21.15.2:0/237961528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0104102030 0x7f0104197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.379+0000 7f010b9d4640 1 -- 172.21.15.2:0/237961528 >> 172.21.15.2:0/237961528 conn(0x7f01040fb980 msgr2=0x7f0104102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:47.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.379+0000 7f010b9d4640 1 -- 172.21.15.2:0/237961528 shutdown_connections 2024-03-15T12:58:47.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.379+0000 7f010b9d4640 1 -- 172.21.15.2:0/237961528 wait complete. 2024-03-15T12:58:47.376 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-15T12:58:47.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.496+0000 7fa2d37d0640 1 -- 172.21.15.2:0/3076649755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2cc101ff0 msgr2=0x7fa2cc1043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:47.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.496+0000 7fa2d37d0640 1 --2- 172.21.15.2:0/3076649755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2cc101ff0 0x7fa2cc1043e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009900 tx=0x7fa2c802bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:47.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.496+0000 7fa2d37d0640 1 -- 172.21.15.2:0/3076649755 shutdown_connections 2024-03-15T12:58:47.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.496+0000 7fa2d37d0640 1 --2- 172.21.15.2:0/3076649755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2cc107250 0x7fa2cc109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.496+0000 7fa2d37d0640 1 --2- 172.21.15.2:0/3076649755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2cc104920 0x7fa2cc106d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.496+0000 7fa2d37d0640 1 --2- 172.21.15.2:0/3076649755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2cc101ff0 0x7fa2cc1043e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.496+0000 7fa2d37d0640 1 -- 172.21.15.2:0/3076649755 >> 172.21.15.2:0/3076649755 conn(0x7fa2cc0fb980 msgr2=0x7fa2cc0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:47.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.496+0000 7fa2d37d0640 1 -- 172.21.15.2:0/3076649755 shutdown_connections 2024-03-15T12:58:47.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.496+0000 7fa2d37d0640 1 -- 172.21.15.2:0/3076649755 wait complete. 2024-03-15T12:58:47.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d37d0640 1 Processor -- start 2024-03-15T12:58:47.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d37d0640 1 -- start start 2024-03-15T12:58:47.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d37d0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2cc101ff0 0x7fa2cc101960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:47.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d37d0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2cc104920 0x7fa2cc0fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:47.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d37d0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2cc107250 0x7fa2cc1004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:47.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d37d0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc100c40 con 0x7fa2cc107250 2024-03-15T12:58:47.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d37d0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc100db0 con 0x7fa2cc104920 2024-03-15T12:58:47.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d37d0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc1a1620 con 0x7fa2cc101ff0 2024-03-15T12:58:47.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d1d6d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2cc107250 0x7fa2cc1004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:47.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d1d6d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2cc107250 0x7fa2cc1004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44048/0 (socket says 172.21.15.2:44048) 2024-03-15T12:58:47.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d1d6d640 1 -- 172.21.15.2:0/255137777 learned_addr learned my addr 172.21.15.2:0/255137777 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:47.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d156c640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2cc101ff0 0x7fa2cc101960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:47.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d0d6b640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2cc104920 0x7fa2cc0fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:47.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d1d6d640 1 -- 172.21.15.2:0/255137777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2cc101ff0 msgr2=0x7fa2cc101960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:47.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d1d6d640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2cc101ff0 0x7fa2cc101960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d1d6d640 1 -- 172.21.15.2:0/255137777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2cc104920 msgr2=0x7fa2cc0fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:47.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d1d6d640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2cc104920 0x7fa2cc0fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:47.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d1d6d640 1 -- 172.21.15.2:0/255137777 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c8009560 con 0x7fa2cc107250 2024-03-15T12:58:47.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2d1d6d640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2cc107250 0x7fa2cc1004e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400db20 tx=0x7fa2c400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:47.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.497+0000 7fa2c27fc640 1 -- 172.21.15.2:0/255137777 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c400bea0 con 0x7fa2cc107250 2024-03-15T12:58:47.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.498+0000 7fa2d37d0640 1 -- 172.21.15.2:0/255137777 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2cc1a18d0 con 0x7fa2cc107250 2024-03-15T12:58:47.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.498+0000 7fa2c27fc640 1 -- 172.21.15.2:0/255137777 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa2c4009940 con 0x7fa2cc107250 2024-03-15T12:58:47.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.498+0000 7fa2c27fc640 1 -- 172.21.15.2:0/255137777 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c40106c0 con 0x7fa2cc107250 2024-03-15T12:58:47.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.498+0000 7fa2d37d0640 1 -- 172.21.15.2:0/255137777 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2cc1a1d90 con 0x7fa2cc107250 2024-03-15T12:58:47.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.499+0000 7fa2d37d0640 1 -- 172.21.15.2:0/255137777 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2cc0fd470 con 0x7fa2cc107250 2024-03-15T12:58:47.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.500+0000 7fa2c27fc640 1 -- 172.21.15.2:0/255137777 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 20) v1 ==== 136346+0+0 (secure 0 0 0) 0x7fa2c4010820 con 0x7fa2cc107250 2024-03-15T12:58:47.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.500+0000 7fa2c27fc640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1bc0 0x7fa2b80a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:47.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.500+0000 7fa2d156c640 1 -- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1bc0 msgr2=0x7fa2b80a4080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/343748862 2024-03-15T12:58:47.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.500+0000 7fa2c27fc640 1 -- 172.21.15.2:0/255137777 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9063+0+0 (secure 0 0 0) 0x7fa2c40cd5d0 con 0x7fa2cc107250 2024-03-15T12:58:47.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.500+0000 7fa2d156c640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1bc0 0x7fa2b80a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-15T12:58:47.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.505+0000 7fa2c27fc640 1 -- 172.21.15.2:0/255137777 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa2c40963f0 con 0x7fa2cc107250 2024-03-15T12:58:47.559 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:47 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:47.288+0000 7f3077fff200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-15T12:58:47.559 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:47 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:47.383+0000 7f3077fff200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-15T12:58:47.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.701+0000 7fa2d156c640 1 -- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1bc0 msgr2=0x7fa2b80a4080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/343748862 2024-03-15T12:58:47.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.701+0000 7fa2d156c640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1bc0 0x7fa2b80a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-15T12:58:47.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:47.712+0000 7fa2d37d0640 1 -- 172.21.15.2:0/255137777 --> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2cc02d070 con 0x7fa2b80a1bc0 2024-03-15T12:58:47.706 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:47 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:47.322+0000 7fa9aadf6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-15T12:58:47.706 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:47 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:47.423+0000 7fa9aadf6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-15T12:58:47.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:47 smithi149 ceph-mon[35927]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-15T12:58:47.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:47 smithi149 ceph-mon[35927]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-03-15T12:58:47.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:47 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3153408229' entity='client.admin' 2024-03-15T12:58:47.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:47 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/237961528' entity='client.admin' 2024-03-15T12:58:47.860 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:47 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:47.542+0000 7f64a6dfa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-15T12:58:47.860 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:47 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:47.693+0000 7f64a6dfa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-15T12:58:48.038 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:47 smithi184 ceph-mon[35993]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-15T12:58:48.038 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:47 smithi184 ceph-mon[35993]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-03-15T12:58:48.038 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:47 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3153408229' entity='client.admin' 2024-03-15T12:58:48.039 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:47 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/237961528' entity='client.admin' 2024-03-15T12:58:48.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:47 smithi002 ceph-mon[32060]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-15T12:58:48.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:47 smithi002 ceph-mon[32060]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-03-15T12:58:48.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:47 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3153408229' entity='client.admin' 2024-03-15T12:58:48.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:47 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/237961528' entity='client.admin' 2024-03-15T12:58:48.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:48.102+0000 7fa2d156c640 1 -- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1bc0 msgr2=0x7fa2b80a4080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/343748862 2024-03-15T12:58:48.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:48.102+0000 7fa2d156c640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1bc0 0x7fa2b80a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-15T12:58:48.294 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:48.052+0000 7f3077fff200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-15T12:58:48.294 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:48.147+0000 7f3077fff200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-15T12:58:48.294 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:48.308+0000 7f3077fff200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-15T12:58:48.412 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:48.089+0000 7fa9aadf6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-15T12:58:48.412 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:48.184+0000 7fa9aadf6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-15T12:58:48.412 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:48.327+0000 7fa9aadf6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-15T12:58:48.412 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:48.426+0000 7fa9aadf6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-15T12:58:48.559 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:48.404+0000 7f3077fff200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-15T12:58:48.559 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:48.488+0000 7f3077fff200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-15T12:58:48.767 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:48.512+0000 7fa9aadf6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-15T12:58:48.860 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:48 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:48.609+0000 7f64a6dfa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-15T12:58:48.860 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:48 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:48.744+0000 7f64a6dfa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-15T12:58:48.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:48.903+0000 7fa2d156c640 1 -- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1bc0 msgr2=0x7fa2b80a4080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/343748862 2024-03-15T12:58:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:48.903+0000 7fa2d156c640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1bc0 0x7fa2b80a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-15T12:58:49.049 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:48.768+0000 7f3077fff200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-15T12:58:49.050 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:48.880+0000 7f3077fff200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-15T12:58:49.050 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:48.967+0000 7f3077fff200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-15T12:58:49.052 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:48.781+0000 7fa9aadf6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-15T12:58:49.052 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:48.893+0000 7fa9aadf6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-15T12:58:49.053 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:48.982+0000 7fa9aadf6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-15T12:58:49.308 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:49 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:49.063+0000 7f3077fff200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-15T12:58:49.309 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:49 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:49.244+0000 7f3077fff200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-15T12:58:49.359 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:48 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:48.930+0000 7f64a6dfa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-15T12:58:49.360 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:49 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:49.051+0000 7f64a6dfa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-15T12:58:49.360 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:49 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:49.161+0000 7f64a6dfa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-15T12:58:49.553 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:49 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:49.079+0000 7fa9aadf6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-15T12:58:49.553 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:49 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:49.257+0000 7fa9aadf6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-15T12:58:49.802 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:49 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:49.533+0000 7f64a6dfa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-15T12:58:49.802 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:49 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:49.684+0000 7f64a6dfa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-15T12:58:50.110 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:49 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:49.816+0000 7f64a6dfa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-15T12:58:50.110 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:49 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:49.966+0000 7f64a6dfa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-15T12:58:50.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:50.504+0000 7fa2d156c640 1 -- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1bc0 msgr2=0x7fa2b80a4080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/343748862 2024-03-15T12:58:50.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:50.504+0000 7fa2d156c640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1bc0 0x7fa2b80a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-15T12:58:50.552 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:50 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:50.246+0000 7fa9aadf6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-15T12:58:50.553 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:50 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:50.380+0000 7fa9aadf6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-15T12:58:50.609 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:50 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:50.210+0000 7f64a6dfa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-15T12:58:50.809 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:50 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:50.371+0000 7f3077fff200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-15T12:58:50.809 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:50 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:50.504+0000 7f3077fff200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-15T12:58:51.003 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:50 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:50.721+0000 7fa9aadf6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-15T12:58:51.003 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:50 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:50.814+0000 7fa9aadf6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-15T12:58:51.196 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:50 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:50.921+0000 7f3077fff200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-15T12:58:51.196 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:51 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:51.012+0000 7f3077fff200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-15T12:58:51.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:51 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:51.017+0000 7fa9aadf6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:58:51.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:51 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:51.209+0000 7fa9aadf6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-15T12:58:51.559 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:51 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:51.210+0000 7f3077fff200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:58:51.559 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:51 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:51.395+0000 7f3077fff200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-15T12:58:51.754 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:51 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:51.505+0000 7fa9aadf6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:58:51.859 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:51 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:51.493+0000 7f64a6dfa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-15T12:58:51.860 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:51 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:51.690+0000 7f64a6dfa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-15T12:58:51.934 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:51 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:51.681+0000 7f3077fff200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:58:52.053 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:51 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:51.768+0000 7fa9aadf6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-15T12:58:52.053 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:52 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:52.015+0000 7fa9aadf6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-15T12:58:52.308 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:51 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:51.948+0000 7f3077fff200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-15T12:58:52.309 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:52 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:52.185+0000 7f3077fff200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-15T12:58:52.569 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:52 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:52.160+0000 7f64a6dfa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-15T12:58:52.569 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:52 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:52.276+0000 7f64a6dfa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-15T12:58:52.859 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:52 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:52.583+0000 7f64a6dfa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:58:52.860 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:52 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:52.822+0000 7f64a6dfa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-15T12:58:53.302 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:52 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:52.852+0000 7fa9aadf6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-15T12:58:53.512 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:53.207+0000 7f64a6dfa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-15T12:58:53.558 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:53.119+0000 7f3077fff200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-15T12:58:53.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:53.708+0000 7fa2d156c640 1 -- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1bc0 msgr2=0x7fa2b80a4080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/343748862 2024-03-15T12:58:53.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:53.708+0000 7fa2d156c640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1bc0 0x7fa2b80a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-15T12:58:53.753 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:53.439+0000 7fa9aadf6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-15T12:58:53.753 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:53.540+0000 7fa9aadf6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-15T12:58:53.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:53.792+0000 7fa2c27fc640 1 -- 172.21.15.2:0/255137777 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 21) v1 ==== 135964+0+0 (secure 0 0 0) 0x7fa2c40967c0 con 0x7fa2cc107250 2024-03-15T12:58:53.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:53.793+0000 7fa2c27fc640 1 -- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1bc0 msgr2=0x7fa2b80a4080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-15T12:58:53.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:53.793+0000 7fa2c27fc640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/343748862,v1:172.21.15.2:6801/343748862] conn(0x7fa2b80a1bc0 0x7fa2b80a4080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:53.820 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:53.527+0000 7f64a6dfa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-15T12:58:54.020 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:53.725+0000 7f3077fff200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-15T12:58:54.020 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:53.828+0000 7f3077fff200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-15T12:58:54.021 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: Active manager daemon x restarted 2024-03-15T12:58:54.021 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: Activating manager daemon x 2024-03-15T12:58:54.021 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: osdmap e92: 12 total, 12 up, 12 in 2024-03-15T12:58:54.021 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: mgrmap e21: x(active, starting, since 0.0119006s), standbys: z, y 2024-03-15T12:58:54.021 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:58:54.021 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:58:54.021 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:58:54.021 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-15T12:58:54.022 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-15T12:58:54.022 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-15T12:58:54.022 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-15T12:58:54.022 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-15T12:58:54.022 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-15T12:58:54.022 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-15T12:58:54.022 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-15T12:58:54.022 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-15T12:58:54.022 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-15T12:58:54.022 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-15T12:58:54.022 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-15T12:58:54.022 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-15T12:58:54.022 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-15T12:58:54.022 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-15T12:58:54.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:58:54.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:58:54.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:58:54.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:58:54.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:58:54.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:58:54.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:58:54.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:58:54.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:58:54.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:58:54.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:58:54.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:58:54.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-15T12:58:54.023 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-15T12:58:54.024 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-15T12:58:54.024 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:53 smithi184 ceph-mon[35993]: Manager daemon x is now available 2024-03-15T12:58:54.024 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:53.767+0000 7fa9aadf6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-15T12:58:54.024 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: Active manager daemon x restarted 2024-03-15T12:58:54.024 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: Activating manager daemon x 2024-03-15T12:58:54.025 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: osdmap e92: 12 total, 12 up, 12 in 2024-03-15T12:58:54.025 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: mgrmap e21: x(active, starting, since 0.0119006s), standbys: z, y 2024-03-15T12:58:54.025 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:58:54.025 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:58:54.025 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:58:54.025 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-15T12:58:54.025 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-15T12:58:54.025 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-15T12:58:54.025 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-15T12:58:54.025 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-15T12:58:54.025 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-15T12:58:54.025 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-15T12:58:54.025 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-15T12:58:54.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-15T12:58:54.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-15T12:58:54.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-15T12:58:54.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-15T12:58:54.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-15T12:58:54.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-15T12:58:54.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-15T12:58:54.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:58:54.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:58:54.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:58:54.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:58:54.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:58:54.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:58:54.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:58:54.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:58:54.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:58:54.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:58:54.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:58:54.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:58:54.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-15T12:58:54.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-15T12:58:54.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-15T12:58:54.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:53 smithi002 ceph-mon[32060]: Manager daemon x is now available 2024-03-15T12:58:54.109 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:53.848+0000 7f64a6dfa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-15T12:58:54.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: Active manager daemon x restarted 2024-03-15T12:58:54.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: Activating manager daemon x 2024-03-15T12:58:54.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: osdmap e92: 12 total, 12 up, 12 in 2024-03-15T12:58:54.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: mgrmap e21: x(active, starting, since 0.0119006s), standbys: z, y 2024-03-15T12:58:54.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-15T12:58:54.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-15T12:58:54.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-15T12:58:54.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-15T12:58:54.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-15T12:58:54.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-15T12:58:54.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-15T12:58:54.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-15T12:58:54.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-15T12:58:54.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-15T12:58:54.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-15T12:58:54.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-15T12:58:54.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-15T12:58:54.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-15T12:58:54.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-15T12:58:54.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-15T12:58:54.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-15T12:58:54.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-15T12:58:54.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:53 smithi149 ceph-mon[35927]: Manager daemon x is now available 2024-03-15T12:58:54.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:54.035+0000 7fa90a854700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:58:54.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:54.035+0000 7fa90a854700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:58:54.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:54.035+0000 7fa90a854700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:58:54.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:54.035+0000 7fa90a854700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:58:54.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:54.035+0000 7fa90a854700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:58:54.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:54.037+0000 7fa90d85a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:58:54.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:54.037+0000 7fa90d85a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:58:54.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:54.037+0000 7fa90d85a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:58:54.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:54.037+0000 7fa90d85a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:58:54.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:58:54.037+0000 7fa90d85a700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:58:54.308 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z[37789]: 2024-03-15T12:58:54.110+0000 7f3077fff200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-15T12:58:54.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.797+0000 7fa2c27fc640 1 -- 172.21.15.2:0/255137777 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mgrmap(e 22) v1 ==== 136091+0+0 (secure 0 0 0) 0x7fa2c4001250 con 0x7fa2cc107250 2024-03-15T12:58:54.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.798+0000 7fa2c27fc640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2b80508d0 0x7fa2b8050cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:54.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.798+0000 7fa2c27fc640 1 -- 172.21.15.2:0/255137777 --> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2cc02d070 con 0x7fa2b80508d0 2024-03-15T12:58:54.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.800+0000 7fa2d156c640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2b80508d0 0x7fa2b8050cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:54.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.802+0000 7fa2d156c640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2b80508d0 0x7fa2b8050cb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8000c00 tx=0x7fa2c8005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:54.857 INFO:teuthology.orchestra.run.smithi002.stdout:Schedule set for path / 2024-03-15T12:58:54.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.871+0000 7fa2c27fc640 1 -- 172.21.15.2:0/255137777 <== mgr.15741 v2:172.21.15.2:6800/1550147071 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa2cc02d070 con 0x7fa2b80508d0 2024-03-15T12:58:54.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.873+0000 7fa2d37d0640 1 -- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2b80508d0 msgr2=0x7fa2b8050cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:54.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.873+0000 7fa2d37d0640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2b80508d0 0x7fa2b8050cb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8000c00 tx=0x7fa2c8005fd0 comp rx=0 tx=0).stop 2024-03-15T12:58:54.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.873+0000 7fa2d37d0640 1 -- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2cc107250 msgr2=0x7fa2cc1004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:54.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.873+0000 7fa2d37d0640 1 --2- 172.21.15.2:0/255137777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2cc107250 0x7fa2cc1004e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400db20 tx=0x7fa2c400b260 comp rx=0 tx=0).stop 2024-03-15T12:58:54.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.874+0000 7fa2d1d6d640 1 -- 172.21.15.2:0/255137777 reap_dead start 2024-03-15T12:58:54.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.874+0000 7fa2d37d0640 1 -- 172.21.15.2:0/255137777 shutdown_connections 2024-03-15T12:58:54.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.874+0000 7fa2d37d0640 1 -- 172.21.15.2:0/255137777 >> 172.21.15.2:0/255137777 conn(0x7fa2cc0fb980 msgr2=0x7fa2cc102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:54.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.874+0000 7fa2d37d0640 1 -- 172.21.15.2:0/255137777 shutdown_connections 2024-03-15T12:58:54.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.874+0000 7fa2d37d0640 1 -- 172.21.15.2:0/255137777 wait complete. 2024-03-15T12:58:54.871 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-15T12:58:54.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.992+0000 7fd0bacfb640 1 -- 172.21.15.2:0/2396788664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b40ff830 msgr2=0x7fd0b40ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:54.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.992+0000 7fd0bacfb640 1 --2- 172.21.15.2:0/2396788664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b40ff830 0x7fd0b40ffc50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8009900 tx=0x7fd0a802bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:54.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.992+0000 7fd0bacfb640 1 -- 172.21.15.2:0/2396788664 shutdown_connections 2024-03-15T12:58:54.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.992+0000 7fd0bacfb640 1 --2- 172.21.15.2:0/2396788664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0b4103260 0x7fd0b4109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:54.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.992+0000 7fd0bacfb640 1 --2- 172.21.15.2:0/2396788664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0b41028a0 0x7fd0b4102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:54.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.992+0000 7fd0bacfb640 1 --2- 172.21.15.2:0/2396788664 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b40ff830 0x7fd0b40ffc50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:54.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.992+0000 7fd0bacfb640 1 -- 172.21.15.2:0/2396788664 >> 172.21.15.2:0/2396788664 conn(0x7fd0b40fb120 msgr2=0x7fd0b40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:54.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.992+0000 7fd0bacfb640 1 -- 172.21.15.2:0/2396788664 shutdown_connections 2024-03-15T12:58:54.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.992+0000 7fd0bacfb640 1 -- 172.21.15.2:0/2396788664 wait complete. 2024-03-15T12:58:54.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.993+0000 7fd0bacfb640 1 Processor -- start 2024-03-15T12:58:54.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.993+0000 7fd0bacfb640 1 -- start start 2024-03-15T12:58:54.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.993+0000 7fd0bacfb640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0b40ff830 0x7fd0b4102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:54.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.993+0000 7fd0bacfb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b41028a0 0x7fd0b4100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:54.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.993+0000 7fd0bacfb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0b4103260 0x7fd0b4100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:54.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.993+0000 7fd0bacfb640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b41014f0 con 0x7fd0b41028a0 2024-03-15T12:58:54.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.993+0000 7fd0bacfb640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b4101660 con 0x7fd0b40ff830 2024-03-15T12:58:54.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.993+0000 7fd0bacfb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b41a15a0 con 0x7fd0b4103260 2024-03-15T12:58:54.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.993+0000 7fd0b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b41028a0 0x7fd0b4100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:54.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.993+0000 7fd0b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b41028a0 0x7fd0b4100850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48722/0 (socket says 172.21.15.2:48722) 2024-03-15T12:58:54.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.993+0000 7fd0b3fff640 1 -- 172.21.15.2:0/245992992 learned_addr learned my addr 172.21.15.2:0/245992992 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:54.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.993+0000 7fd0b8a97640 1 --2- 172.21.15.2:0/245992992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0b40ff830 0x7fd0b4102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:54.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.993+0000 7fd0b9298640 1 --2- 172.21.15.2:0/245992992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0b4103260 0x7fd0b4100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:54.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.994+0000 7fd0b3fff640 1 -- 172.21.15.2:0/245992992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0b4103260 msgr2=0x7fd0b4100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:54.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.994+0000 7fd0b3fff640 1 --2- 172.21.15.2:0/245992992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0b4103260 0x7fd0b4100d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:54.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.994+0000 7fd0b3fff640 1 -- 172.21.15.2:0/245992992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0b40ff830 msgr2=0x7fd0b4102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:54.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.994+0000 7fd0b3fff640 1 --2- 172.21.15.2:0/245992992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0b40ff830 0x7fd0b4102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:54.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.994+0000 7fd0b3fff640 1 -- 172.21.15.2:0/245992992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a8009560 con 0x7fd0b41028a0 2024-03-15T12:58:54.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.994+0000 7fd0b3fff640 1 --2- 172.21.15.2:0/245992992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b41028a0 0x7fd0b4100850 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000ea20 tx=0x7fd0a000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:54.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.994+0000 7fd0b1ffb640 1 -- 172.21.15.2:0/245992992 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0009700 con 0x7fd0b41028a0 2024-03-15T12:58:54.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.994+0000 7fd0b1ffb640 1 -- 172.21.15.2:0/245992992 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd0a000cd80 con 0x7fd0b41028a0 2024-03-15T12:58:54.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.994+0000 7fd0bacfb640 1 -- 172.21.15.2:0/245992992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b41a1850 con 0x7fd0b41028a0 2024-03-15T12:58:54.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.994+0000 7fd0b1ffb640 1 -- 172.21.15.2:0/245992992 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00187e0 con 0x7fd0b41028a0 2024-03-15T12:58:54.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.994+0000 7fd0bacfb640 1 -- 172.21.15.2:0/245992992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b41a1d60 con 0x7fd0b41028a0 2024-03-15T12:58:54.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.995+0000 7fd0bacfb640 1 -- 172.21.15.2:0/245992992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b4108970 con 0x7fd0b41028a0 2024-03-15T12:58:54.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.996+0000 7fd0b1ffb640 1 -- 172.21.15.2:0/245992992 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 22) v1 ==== 136091+0+0 (secure 0 0 0) 0x7fd0a0018980 con 0x7fd0b41028a0 2024-03-15T12:58:54.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.997+0000 7fd0b1ffb640 1 --2- 172.21.15.2:0/245992992 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd0a40a18e0 0x7fd0a40a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:54.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.997+0000 7fd0b1ffb640 1 -- 172.21.15.2:0/245992992 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fd0a0014070 con 0x7fd0b41028a0 2024-03-15T12:58:54.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.997+0000 7fd0b8a97640 1 --2- 172.21.15.2:0/245992992 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd0a40a18e0 0x7fd0a40a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:54.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:54.997+0000 7fd0b8a97640 1 --2- 172.21.15.2:0/245992992 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd0a40a18e0 0x7fd0a40a3da0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd0a802c740 tx=0x7fd0a8005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:54.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.001+0000 7fd0b1ffb640 1 -- 172.21.15.2:0/245992992 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd0a0053280 con 0x7fd0b41028a0 2024-03-15T12:58:55.110 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:55 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:55.042+0000 7f64a6dfa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-15T12:58:55.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:54 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:58:55.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:54 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:58:55.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:54 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-15T12:58:55.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:54 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-15T12:58:55.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:54 smithi149 ceph-mon[35927]: Standby manager daemon z restarted 2024-03-15T12:58:55.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:54 smithi149 ceph-mon[35927]: Standby manager daemon z started 2024-03-15T12:58:55.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:54 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.184:0/3503669635' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-15T12:58:55.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:54 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.184:0/3503669635' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-15T12:58:55.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:54 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.184:0/3503669635' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-15T12:58:55.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:54 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.184:0/3503669635' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-15T12:58:55.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:54 smithi149 ceph-mon[35927]: [15/Mar/2024:12:58:54] ENGINE Bus STARTING 2024-03-15T12:58:55.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:54 smithi149 ceph-mon[35927]: [15/Mar/2024:12:58:54] ENGINE Serving on http://172.21.15.2:8765 2024-03-15T12:58:55.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:54 smithi149 ceph-mon[35927]: [15/Mar/2024:12:58:54] ENGINE Serving on https://172.21.15.2:7150 2024-03-15T12:58:55.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:54 smithi149 ceph-mon[35927]: [15/Mar/2024:12:58:54] ENGINE Bus STARTED 2024-03-15T12:58:55.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:54 smithi149 ceph-mon[35927]: mgrmap e22: x(active, since 1.01948s), standbys: y, z 2024-03-15T12:58:55.150 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:58:55.150 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:58:55.150 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-15T12:58:55.150 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-15T12:58:55.150 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-mon[35993]: Standby manager daemon z restarted 2024-03-15T12:58:55.150 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-mon[35993]: Standby manager daemon z started 2024-03-15T12:58:55.151 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.184:0/3503669635' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-15T12:58:55.151 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.184:0/3503669635' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-15T12:58:55.151 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.184:0/3503669635' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-15T12:58:55.151 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.184:0/3503669635' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-15T12:58:55.151 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-mon[35993]: [15/Mar/2024:12:58:54] ENGINE Bus STARTING 2024-03-15T12:58:55.151 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-mon[35993]: [15/Mar/2024:12:58:54] ENGINE Serving on http://172.21.15.2:8765 2024-03-15T12:58:55.151 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-mon[35993]: [15/Mar/2024:12:58:54] ENGINE Serving on https://172.21.15.2:7150 2024-03-15T12:58:55.151 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-mon[35993]: [15/Mar/2024:12:58:54] ENGINE Bus STARTED 2024-03-15T12:58:55.151 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:54 smithi184 ceph-mon[35993]: mgrmap e22: x(active, since 1.01948s), standbys: y, z 2024-03-15T12:58:55.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.214+0000 7fd0bacfb640 1 -- 172.21.15.2:0/245992992 --> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0b402d070 con 0x7fd0a40a18e0 2024-03-15T12:58:55.213 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:58:55.213 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:58:55.213 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-15T12:58:55.213 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-15T12:58:55.213 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-mon[32060]: Standby manager daemon z restarted 2024-03-15T12:58:55.213 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-mon[32060]: Standby manager daemon z started 2024-03-15T12:58:55.213 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.184:0/3503669635' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-15T12:58:55.213 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.184:0/3503669635' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-15T12:58:55.213 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.184:0/3503669635' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-15T12:58:55.214 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.184:0/3503669635' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-15T12:58:55.214 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-mon[32060]: [15/Mar/2024:12:58:54] ENGINE Bus STARTING 2024-03-15T12:58:55.214 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-mon[32060]: [15/Mar/2024:12:58:54] ENGINE Serving on http://172.21.15.2:8765 2024-03-15T12:58:55.214 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-mon[32060]: [15/Mar/2024:12:58:54] ENGINE Serving on https://172.21.15.2:7150 2024-03-15T12:58:55.214 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-mon[32060]: [15/Mar/2024:12:58:54] ENGINE Bus STARTED 2024-03-15T12:58:55.214 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:54 smithi002 ceph-mon[32060]: mgrmap e22: x(active, since 1.01948s), standbys: y, z 2024-03-15T12:58:55.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.269+0000 7fd0b1ffb640 1 -- 172.21.15.2:0/245992992 <== mgr.15741 v2:172.21.15.2:6800/1550147071 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fd0b402d070 con 0x7fd0a40a18e0 2024-03-15T12:58:55.255 INFO:teuthology.orchestra.run.smithi002.stdout:Retention added to path / 2024-03-15T12:58:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.271+0000 7fd0bacfb640 1 -- 172.21.15.2:0/245992992 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd0a40a18e0 msgr2=0x7fd0a40a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.271+0000 7fd0bacfb640 1 --2- 172.21.15.2:0/245992992 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd0a40a18e0 0x7fd0a40a3da0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd0a802c740 tx=0x7fd0a8005fd0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.271+0000 7fd0bacfb640 1 -- 172.21.15.2:0/245992992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b41028a0 msgr2=0x7fd0b4100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.271+0000 7fd0bacfb640 1 --2- 172.21.15.2:0/245992992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b41028a0 0x7fd0b4100850 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000ea20 tx=0x7fd0a000eee0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.271+0000 7fd0bacfb640 1 -- 172.21.15.2:0/245992992 shutdown_connections 2024-03-15T12:58:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.271+0000 7fd0bacfb640 1 --2- 172.21.15.2:0/245992992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0b4103260 0x7fd0b4100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.271+0000 7fd0bacfb640 1 --2- 172.21.15.2:0/245992992 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd0a40a18e0 0x7fd0a40a3da0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.271+0000 7fd0bacfb640 1 --2- 172.21.15.2:0/245992992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b41028a0 0x7fd0b4100850 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.271+0000 7fd0bacfb640 1 --2- 172.21.15.2:0/245992992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0b40ff830 0x7fd0b4102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.271+0000 7fd0bacfb640 1 -- 172.21.15.2:0/245992992 >> 172.21.15.2:0/245992992 conn(0x7fd0b40fb120 msgr2=0x7fd0b4105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:55.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.271+0000 7fd0bacfb640 1 -- 172.21.15.2:0/245992992 shutdown_connections 2024-03-15T12:58:55.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.271+0000 7fd0bacfb640 1 -- 172.21.15.2:0/245992992 wait complete. 2024-03-15T12:58:55.268 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-15T12:58:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.388+0000 7f895a8d4640 1 -- 172.21.15.2:0/451241292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8954101520 msgr2=0x7f8954101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.388+0000 7f895a8d4640 1 --2- 172.21.15.2:0/451241292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8954101520 0x7f8954101940 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8944009900 tx=0x7f894402bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.388+0000 7f895a8d4640 1 -- 172.21.15.2:0/451241292 shutdown_connections 2024-03-15T12:58:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.388+0000 7f895a8d4640 1 --2- 172.21.15.2:0/451241292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89541030d0 0x7f8954109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.388+0000 7f895a8d4640 1 --2- 172.21.15.2:0/451241292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8954102710 0x7f8954102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.388+0000 7f895a8d4640 1 --2- 172.21.15.2:0/451241292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8954101520 0x7f8954101940 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.388+0000 7f895a8d4640 1 -- 172.21.15.2:0/451241292 >> 172.21.15.2:0/451241292 conn(0x7f89540fcab0 msgr2=0x7f89540feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.388+0000 7f895a8d4640 1 -- 172.21.15.2:0/451241292 shutdown_connections 2024-03-15T12:58:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.388+0000 7f895a8d4640 1 -- 172.21.15.2:0/451241292 wait complete. 2024-03-15T12:58:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.388+0000 7f895a8d4640 1 Processor -- start 2024-03-15T12:58:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.388+0000 7f895a8d4640 1 -- start start 2024-03-15T12:58:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f895a8d4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8954101520 0x7f89541974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f895a8d4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8954102710 0x7f8954197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f895a8d4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89541030d0 0x7f895419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f895a8d4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895419efc0 con 0x7f8954101520 2024-03-15T12:58:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f895a8d4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895419f130 con 0x7f8954102710 2024-03-15T12:58:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f895a8d4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895419f2f0 con 0x7f89541030d0 2024-03-15T12:58:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f8953fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8954101520 0x7f89541974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f8953fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8954101520 0x7f89541974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48738/0 (socket says 172.21.15.2:48738) 2024-03-15T12:58:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f8953fff640 1 -- 172.21.15.2:0/3355544366 learned_addr learned my addr 172.21.15.2:0/3355544366 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f89537fe640 1 --2- 172.21.15.2:0/3355544366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8954102710 0x7f8954197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f8958e71640 1 --2- 172.21.15.2:0/3355544366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89541030d0 0x7f895419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f8953fff640 1 -- 172.21.15.2:0/3355544366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89541030d0 msgr2=0x7f895419ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f8953fff640 1 --2- 172.21.15.2:0/3355544366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89541030d0 0x7f895419ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f8953fff640 1 -- 172.21.15.2:0/3355544366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8954102710 msgr2=0x7f8954197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f8953fff640 1 --2- 172.21.15.2:0/3355544366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8954102710 0x7f8954197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f8953fff640 1 -- 172.21.15.2:0/3355544366 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8944009560 con 0x7f8954101520 2024-03-15T12:58:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.389+0000 7f8953fff640 1 --2- 172.21.15.2:0/3355544366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8954101520 0x7f89541974d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8944000c00 tx=0x7f8944004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.390+0000 7f89517fa640 1 -- 172.21.15.2:0/3355544366 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89440042d0 con 0x7f8954101520 2024-03-15T12:58:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.390+0000 7f895a8d4640 1 -- 172.21.15.2:0/3355544366 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895419f570 con 0x7f8954101520 2024-03-15T12:58:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.390+0000 7f89517fa640 1 -- 172.21.15.2:0/3355544366 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8944004430 con 0x7f8954101520 2024-03-15T12:58:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.390+0000 7f89517fa640 1 -- 172.21.15.2:0/3355544366 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944003e20 con 0x7f8954101520 2024-03-15T12:58:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.390+0000 7f895a8d4640 1 -- 172.21.15.2:0/3355544366 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895419fad0 con 0x7f8954101520 2024-03-15T12:58:55.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.391+0000 7f895a8d4640 1 -- 172.21.15.2:0/3355544366 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89541087e0 con 0x7f8954101520 2024-03-15T12:58:55.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.392+0000 7f89517fa640 1 -- 172.21.15.2:0/3355544366 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 22) v1 ==== 136091+0+0 (secure 0 0 0) 0x7f89440035f0 con 0x7f8954101520 2024-03-15T12:58:55.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.392+0000 7f89517fa640 1 --2- 172.21.15.2:0/3355544366 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f893c0a18e0 0x7f893c0a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:55.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.392+0000 7f89537fe640 1 --2- 172.21.15.2:0/3355544366 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f893c0a18e0 0x7f893c0a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:55.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.392+0000 7f89517fa640 1 -- 172.21.15.2:0/3355544366 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f89440ef630 con 0x7f8954101520 2024-03-15T12:58:55.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.393+0000 7f89537fe640 1 --2- 172.21.15.2:0/3355544366 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f893c0a18e0 0x7f893c0a3da0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f89541989e0 tx=0x7f894800b410 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:55.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.397+0000 7f89517fa640 1 -- 172.21.15.2:0/3355544366 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f89440b86c0 con 0x7f8954101520 2024-03-15T12:58:55.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.603+0000 7f895a8d4640 1 -- 172.21.15.2:0/3355544366 --> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f8954061930 con 0x7f893c0a18e0 2024-03-15T12:58:55.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.612+0000 7f89517fa640 1 -- 172.21.15.2:0/3355544366 <== mgr.15741 v2:172.21.15.2:6800/1550147071 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f8954061930 con 0x7f893c0a18e0 2024-03-15T12:58:55.598 INFO:teuthology.orchestra.run.smithi002.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-15T00:00:00", "created": "2024-03-15T12:58:54", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-15T12:58:55.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.614+0000 7f895a8d4640 1 -- 172.21.15.2:0/3355544366 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f893c0a18e0 msgr2=0x7f893c0a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:55.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.614+0000 7f895a8d4640 1 --2- 172.21.15.2:0/3355544366 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f893c0a18e0 0x7f893c0a3da0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f89541989e0 tx=0x7f894800b410 comp rx=0 tx=0).stop 2024-03-15T12:58:55.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.614+0000 7f895a8d4640 1 -- 172.21.15.2:0/3355544366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8954101520 msgr2=0x7f89541974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:55.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.614+0000 7f895a8d4640 1 --2- 172.21.15.2:0/3355544366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8954101520 0x7f89541974d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8944000c00 tx=0x7f8944004ab0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.614+0000 7f895a8d4640 1 -- 172.21.15.2:0/3355544366 shutdown_connections 2024-03-15T12:58:55.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.614+0000 7f895a8d4640 1 --2- 172.21.15.2:0/3355544366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89541030d0 0x7f895419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.614+0000 7f895a8d4640 1 --2- 172.21.15.2:0/3355544366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8954102710 0x7f8954197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.614+0000 7f895a8d4640 1 --2- 172.21.15.2:0/3355544366 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f893c0a18e0 0x7f893c0a3da0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.614+0000 7f895a8d4640 1 --2- 172.21.15.2:0/3355544366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8954101520 0x7f89541974d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.614+0000 7f895a8d4640 1 -- 172.21.15.2:0/3355544366 >> 172.21.15.2:0/3355544366 conn(0x7f89540fcab0 msgr2=0x7f89541059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:55.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.614+0000 7f895a8d4640 1 -- 172.21.15.2:0/3355544366 shutdown_connections 2024-03-15T12:58:55.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.614+0000 7f895a8d4640 1 -- 172.21.15.2:0/3355544366 wait complete. 2024-03-15T12:58:55.612 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-15T12:58:55.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.740+0000 7f8a18615640 1 -- 172.21.15.2:0/2732150478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a10101540 msgr2=0x7f8a10101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.740+0000 7f8a18615640 1 --2- 172.21.15.2:0/2732150478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a10101540 0x7f8a10101960 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009900 tx=0x7f8a0002bc30 comp rx=0 tx=0).stop 2024-03-15T12:58:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.740+0000 7f8a18615640 1 -- 172.21.15.2:0/2732150478 shutdown_connections 2024-03-15T12:58:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.740+0000 7f8a18615640 1 --2- 172.21.15.2:0/2732150478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a101030f0 0x7f8a10109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.740+0000 7f8a18615640 1 --2- 172.21.15.2:0/2732150478 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a10102730 0x7f8a10102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.740+0000 7f8a18615640 1 --2- 172.21.15.2:0/2732150478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a10101540 0x7f8a10101960 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.740+0000 7f8a18615640 1 -- 172.21.15.2:0/2732150478 >> 172.21.15.2:0/2732150478 conn(0x7f8a100fcab0 msgr2=0x7f8a100feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.740+0000 7f8a18615640 1 -- 172.21.15.2:0/2732150478 shutdown_connections 2024-03-15T12:58:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.740+0000 7f8a18615640 1 -- 172.21.15.2:0/2732150478 wait complete. 2024-03-15T12:58:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.740+0000 7f8a18615640 1 Processor -- start 2024-03-15T12:58:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.740+0000 7f8a18615640 1 -- start start 2024-03-15T12:58:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.740+0000 7f8a18615640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a10101540 0x7f8a10199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.740+0000 7f8a18615640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a10102730 0x7f8a10199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.740+0000 7f8a18615640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a101030f0 0x7f8a101a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.740+0000 7f8a18615640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a101a1250 con 0x7f8a101030f0 2024-03-15T12:58:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a18615640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a101a13c0 con 0x7f8a10101540 2024-03-15T12:58:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a18615640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a101a1580 con 0x7f8a10102730 2024-03-15T12:58:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a16bb2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a101030f0 0x7f8a101a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a15bb0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a10102730 0x7f8a10199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a15bb0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a10102730 0x7f8a10199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:33354/0 (socket says 172.21.15.2:33354) 2024-03-15T12:58:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a15bb0640 1 -- 172.21.15.2:0/4182628301 learned_addr learned my addr 172.21.15.2:0/4182628301 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:58:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a163b1640 1 --2- 172.21.15.2:0/4182628301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a10101540 0x7f8a10199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a15bb0640 1 -- 172.21.15.2:0/4182628301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a10101540 msgr2=0x7f8a10199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a15bb0640 1 --2- 172.21.15.2:0/4182628301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a10101540 0x7f8a10199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a15bb0640 1 -- 172.21.15.2:0/4182628301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a101030f0 msgr2=0x7f8a101a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a15bb0640 1 --2- 172.21.15.2:0/4182628301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a101030f0 0x7f8a101a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a15bb0640 1 -- 172.21.15.2:0/4182628301 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a00009560 con 0x7f8a10102730 2024-03-15T12:58:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a16bb2640 1 --2- 172.21.15.2:0/4182628301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a101030f0 0x7f8a101a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:58:55.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a15bb0640 1 --2- 172.21.15.2:0/4182628301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a10102730 0x7f8a10199ca0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00e9f0 tx=0x7f8a0c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:55.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a077fe640 1 -- 172.21.15.2:0/4182628301 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c009720 con 0x7f8a10102730 2024-03-15T12:58:55.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a18615640 1 -- 172.21.15.2:0/4182628301 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a101a1860 con 0x7f8a10102730 2024-03-15T12:58:55.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a077fe640 1 -- 172.21.15.2:0/4182628301 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a0c00cd80 con 0x7f8a10102730 2024-03-15T12:58:55.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a077fe640 1 -- 172.21.15.2:0/4182628301 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c010690 con 0x7f8a10102730 2024-03-15T12:58:55.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.741+0000 7f8a18615640 1 -- 172.21.15.2:0/4182628301 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a101a1d20 con 0x7f8a10102730 2024-03-15T12:58:55.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.742+0000 7f8a18615640 1 -- 172.21.15.2:0/4182628301 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1010cf20 con 0x7f8a10102730 2024-03-15T12:58:55.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.744+0000 7f8a077fe640 1 -- 172.21.15.2:0/4182628301 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 22) v1 ==== 136091+0+0 (secure 0 0 0) 0x7f8a0c0107f0 con 0x7f8a10102730 2024-03-15T12:58:55.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.744+0000 7f8a077fe640 1 --2- 172.21.15.2:0/4182628301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f89fc0a18e0 0x7f89fc0a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:58:55.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.744+0000 7f8a077fe640 1 -- 172.21.15.2:0/4182628301 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f8a0c014070 con 0x7f8a10102730 2024-03-15T12:58:55.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.744+0000 7f8a163b1640 1 --2- 172.21.15.2:0/4182628301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f89fc0a18e0 0x7f89fc0a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:58:55.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.745+0000 7f8a163b1640 1 --2- 172.21.15.2:0/4182628301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f89fc0a18e0 0x7f89fc0a3da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8a0002c740 tx=0x7f8a00005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:58:55.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.750+0000 7f8a077fe640 1 -- 172.21.15.2:0/4182628301 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8a0c096870 con 0x7f8a10102730 2024-03-15T12:58:55.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.979+0000 7f8a18615640 1 -- 172.21.15.2:0/4182628301 --> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8a10074e40 con 0x7f89fc0a18e0 2024-03-15T12:58:55.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.986+0000 7f8a077fe640 1 -- 172.21.15.2:0/4182628301 <== mgr.15741 v2:172.21.15.2:6800/1550147071 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f8a10074e40 con 0x7f89fc0a18e0 2024-03-15T12:58:55.972 INFO:teuthology.orchestra.run.smithi002.stdout:/ 1m 6m3h 2024-03-15T12:58:55.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.988+0000 7f8a18615640 1 -- 172.21.15.2:0/4182628301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f89fc0a18e0 msgr2=0x7f89fc0a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:55.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.988+0000 7f8a18615640 1 --2- 172.21.15.2:0/4182628301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f89fc0a18e0 0x7f89fc0a3da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8a0002c740 tx=0x7f8a00005fd0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.988+0000 7f8a18615640 1 -- 172.21.15.2:0/4182628301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a10102730 msgr2=0x7f8a10199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:58:55.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.988+0000 7f8a18615640 1 --2- 172.21.15.2:0/4182628301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a10102730 0x7f8a10199ca0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00e9f0 tx=0x7f8a0c00edf0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.988+0000 7f8a18615640 1 -- 172.21.15.2:0/4182628301 shutdown_connections 2024-03-15T12:58:55.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.988+0000 7f8a18615640 1 --2- 172.21.15.2:0/4182628301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f89fc0a18e0 0x7f89fc0a3da0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.988+0000 7f8a18615640 1 --2- 172.21.15.2:0/4182628301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a101030f0 0x7f8a101a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.988+0000 7f8a18615640 1 --2- 172.21.15.2:0/4182628301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a10102730 0x7f8a10199ca0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.988+0000 7f8a18615640 1 --2- 172.21.15.2:0/4182628301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a10101540 0x7f8a10199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:58:55.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.988+0000 7f8a18615640 1 -- 172.21.15.2:0/4182628301 >> 172.21.15.2:0/4182628301 conn(0x7f8a100fcab0 msgr2=0x7f8a10073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:58:55.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.988+0000 7f8a18615640 1 -- 172.21.15.2:0/4182628301 shutdown_connections 2024-03-15T12:58:55.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:58:55.988+0000 7f8a18615640 1 -- 172.21.15.2:0/4182628301 wait complete. 2024-03-15T12:58:55.985 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:55 smithi002 ceph-mon[32060]: from='client.15762 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:55.985 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:55 smithi002 ceph-mon[32060]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:58:55.985 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:55 smithi002 ceph-mon[32060]: from='client.15804 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:55.985 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:55 smithi002 ceph-mon[32060]: from='client.15810 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:55.986 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-15T12:58:56.014 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-15T12:58:56.041 INFO:teuthology.run_tasks:Running task workunit... 2024-03-15T12:58:56.054 INFO:tasks.workunit:Pulling workunits from ref aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-15T12:58:56.054 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-15T12:58:56.054 INFO:tasks.workunit:timeout=3h 2024-03-15T12:58:56.055 INFO:tasks.workunit:cleanup=True 2024-03-15T12:58:56.055 DEBUG:teuthology.orchestra.run.smithi002:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-15T12:58:56.069 INFO:teuthology.orchestra.run.smithi002.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-15T12:58:56.069 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-15T12:58:56.069 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 77h/119d Inode: 1 Links: 2 2024-03-15T12:58:56.070 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-15T12:58:56.070 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-15T12:58:56.070 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-03-15 12:56:42.115068238 +0000 2024-03-15T12:58:56.070 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-03-15 12:56:42.115068238 +0000 2024-03-15T12:58:56.070 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-03-15 12:58:37.432302867 +0000 2024-03-15T12:58:56.070 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2024-03-15T12:58:56.070 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-15T12:58:56.070 DEBUG:teuthology.orchestra.run.smithi002:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-15T12:58:56.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:55 smithi149 ceph-mon[35927]: from='client.15762 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:56.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:55 smithi149 ceph-mon[35927]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:58:56.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:55 smithi149 ceph-mon[35927]: from='client.15804 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:56.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:55 smithi149 ceph-mon[35927]: from='client.15810 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:56.110 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:55 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:55.784+0000 7f64a6dfa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-15T12:58:56.110 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:55 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:55.939+0000 7f64a6dfa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-15T12:58:56.144 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-15T12:58:56.199 INFO:tasks.workunit.client.0.smithi002.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-15T12:58:56.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:55 smithi184 ceph-mon[35993]: from='client.15762 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:56.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:55 smithi184 ceph-mon[35993]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:58:56.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:55 smithi184 ceph-mon[35993]: from='client.15804 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:56.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:55 smithi184 ceph-mon[35993]: from='client.15810 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:56.609 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y[37349]: 2024-03-15T12:58:56.280+0000 7f64a6dfa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-15T12:58:57.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:56 smithi002 ceph-mon[32060]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:58:57.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:56 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:56 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:56 smithi002 ceph-mon[32060]: from='client.25612 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:57.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:56 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:56 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:56 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:56 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:56 smithi002 ceph-mon[32060]: Standby manager daemon y restarted 2024-03-15T12:58:57.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:56 smithi002 ceph-mon[32060]: Standby manager daemon y started 2024-03-15T12:58:57.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:56 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.149:0/4050488745' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-15T12:58:57.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:56 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.149:0/4050488745' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-15T12:58:57.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:56 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.149:0/4050488745' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-15T12:58:57.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:56 smithi002 ceph-mon[32060]: from='mgr.? 172.21.15.149:0/4050488745' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-15T12:58:57.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:56 smithi002 ceph-mon[32060]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-03-15T12:58:57.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:56 smithi184 ceph-mon[35993]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:58:57.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:56 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:56 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:56 smithi184 ceph-mon[35993]: from='client.25612 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:57.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:56 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:56 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:56 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:56 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:56 smithi184 ceph-mon[35993]: Standby manager daemon y restarted 2024-03-15T12:58:57.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:56 smithi184 ceph-mon[35993]: Standby manager daemon y started 2024-03-15T12:58:57.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:56 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.149:0/4050488745' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-15T12:58:57.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:56 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.149:0/4050488745' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-15T12:58:57.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:56 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.149:0/4050488745' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-15T12:58:57.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:56 smithi184 ceph-mon[35993]: from='mgr.? 172.21.15.149:0/4050488745' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-15T12:58:57.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:56 smithi184 ceph-mon[35993]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-03-15T12:58:57.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-mon[35927]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:58:57.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-mon[35927]: from='client.25612 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T12:58:57.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:57.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-mon[35927]: Standby manager daemon y restarted 2024-03-15T12:58:57.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-mon[35927]: Standby manager daemon y started 2024-03-15T12:58:57.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.149:0/4050488745' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-15T12:58:57.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.149:0/4050488745' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-15T12:58:57.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.149:0/4050488745' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-15T12:58:57.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-mon[35927]: from='mgr.? 172.21.15.149:0/4050488745' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-15T12:58:57.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:56 smithi149 ceph-mon[35927]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-03-15T12:58:58.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi184 to 176.2M 2024-03-15T12:58:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: Unable to set osd_memory_target on smithi184 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-03-15T12:58:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-03-15T12:58:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: Unable to set osd_memory_target on smithi002 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-15T12:58:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-03-15T12:58:58.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:58:58.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:58:58.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-15T12:58:58.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-15T12:58:58.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:58 smithi149 ceph-mon[35927]: Updating smithi184:/etc/ceph/ceph.conf 2024-03-15T12:58:58.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi184 to 176.2M 2024-03-15T12:58:58.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: Unable to set osd_memory_target on smithi184 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-03-15T12:58:58.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-03-15T12:58:58.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: Unable to set osd_memory_target on smithi002 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-15T12:58:58.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-03-15T12:58:58.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:58:58.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:58:58.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-15T12:58:58.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-15T12:58:58.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:58 smithi002 ceph-mon[32060]: Updating smithi184:/etc/ceph/ceph.conf 2024-03-15T12:58:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi184 to 176.2M 2024-03-15T12:58:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: Unable to set osd_memory_target on smithi184 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-03-15T12:58:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.559 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-03-15T12:58:58.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: Unable to set osd_memory_target on smithi002 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-15T12:58:58.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:58.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-15T12:58:58.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-03-15T12:58:58.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:58:58.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:58:58.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-15T12:58:58.560 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-15T12:58:58.561 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:58 smithi184 ceph-mon[35993]: Updating smithi184:/etc/ceph/ceph.conf 2024-03-15T12:58:59.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:58:59.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: Updating smithi002:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:58:59.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: Updating smithi184:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:58:59.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: Updating smithi149:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:58:59.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:58:59.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:58:59.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:58:59.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: Updating smithi002:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-15T12:58:59.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: Updating smithi184:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-15T12:58:59.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: Updating smithi149:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-15T12:58:59.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:59.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:59.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-03-15T12:58:59.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:59.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:59.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:59.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:59.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:59.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:58:59.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:58:59.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:58:59.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:58:59.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:58:59 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T12:59:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:59:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: Updating smithi002:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:59:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: Updating smithi184:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:59:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: Updating smithi149:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:59:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:59:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:59:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:59:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: Updating smithi002:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-15T12:59:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: Updating smithi184:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-15T12:59:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: Updating smithi149:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-15T12:59:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-03-15T12:59:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:59:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:59:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:59:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:58:59 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T12:59:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-15T12:59:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: Updating smithi002:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:59:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: Updating smithi184:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:59:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: Updating smithi149:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-15T12:59:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:59:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:59:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-15T12:59:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: Updating smithi002:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-15T12:59:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: Updating smithi184:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-15T12:59:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: Updating smithi149:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-15T12:59:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-03-15T12:59:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T12:59:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T12:59:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T12:59:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T12:59:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:58:59 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T12:59:00.553 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:59:00 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:59:00.279+0000 7fa8eb416700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:59:00.553 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:59:00 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:59:00.279+0000 7fa8eb416700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:59:00.553 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:59:00 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:59:00.279+0000 7fa8eb416700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:59:00.553 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:59:00 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:59:00.279+0000 7fa8eb416700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:59:00.553 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 12:59:00 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T12:59:00.279+0000 7fa8eb416700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-15T12:59:01.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:01 smithi149 ceph-mon[35927]: pgmap v6: 129 pgs: 129 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 183 KiB/s wr, 78 op/s 2024-03-15T12:59:02.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:01 smithi002 ceph-mon[32060]: pgmap v6: 129 pgs: 129 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 183 KiB/s wr, 78 op/s 2024-03-15T12:59:02.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:01 smithi184 ceph-mon[35993]: pgmap v6: 129 pgs: 129 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 183 KiB/s wr, 78 op/s 2024-03-15T12:59:03.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:03 smithi149 ceph-mon[35927]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 79 op/s 2024-03-15T12:59:03.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:03 smithi149 ceph-mon[35927]: mgrmap e25: x(active, since 8s), standbys: z, y 2024-03-15T12:59:03.860 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:03 smithi184 ceph-mon[35993]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 79 op/s 2024-03-15T12:59:03.861 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:03 smithi184 ceph-mon[35993]: mgrmap e25: x(active, since 8s), standbys: z, y 2024-03-15T12:59:04.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:03 smithi002 ceph-mon[32060]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 79 op/s 2024-03-15T12:59:04.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:03 smithi002 ceph-mon[32060]: mgrmap e25: x(active, since 8s), standbys: z, y 2024-03-15T12:59:05.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:05 smithi149 ceph-mon[35927]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-03-15T12:59:06.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:05 smithi002 ceph-mon[32060]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-03-15T12:59:06.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:05 smithi184 ceph-mon[35993]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-03-15T12:59:07.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:07 smithi149 ceph-mon[35927]: pgmap v9: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 110 KiB/s wr, 71 op/s 2024-03-15T12:59:08.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:07 smithi002 ceph-mon[32060]: pgmap v9: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 110 KiB/s wr, 71 op/s 2024-03-15T12:59:08.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:07 smithi184 ceph-mon[35993]: pgmap v9: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 110 KiB/s wr, 71 op/s 2024-03-15T12:59:09.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:09 smithi149 ceph-mon[35927]: pgmap v10: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 100 KiB/s wr, 65 op/s 2024-03-15T12:59:10.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:09 smithi002 ceph-mon[32060]: pgmap v10: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 100 KiB/s wr, 65 op/s 2024-03-15T12:59:10.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:09 smithi184 ceph-mon[35993]: pgmap v10: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 100 KiB/s wr, 65 op/s 2024-03-15T12:59:11.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:11 smithi149 ceph-mon[35927]: pgmap v11: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 100 KiB/s wr, 66 op/s 2024-03-15T12:59:12.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:11 smithi002 ceph-mon[32060]: pgmap v11: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 100 KiB/s wr, 66 op/s 2024-03-15T12:59:12.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:11 smithi184 ceph-mon[35993]: pgmap v11: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 100 KiB/s wr, 66 op/s 2024-03-15T12:59:13.855 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:13 smithi002 ceph-mon[32060]: pgmap v12: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 KiB/s wr, 37 op/s 2024-03-15T12:59:13.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:13 smithi149 ceph-mon[35927]: pgmap v12: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 KiB/s wr, 37 op/s 2024-03-15T12:59:14.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:13 smithi184 ceph-mon[35993]: pgmap v12: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 KiB/s wr, 37 op/s 2024-03-15T12:59:15.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:15 smithi149 ceph-mon[35927]: pgmap v13: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-03-15T12:59:16.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:15 smithi002 ceph-mon[32060]: pgmap v13: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-03-15T12:59:16.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:15 smithi184 ceph-mon[35993]: pgmap v13: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-03-15T12:59:16.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T12:59:16.198 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T12:59:16.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.321+0000 7f84b2a47640 1 -- 172.21.15.2:0/2974753630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac069b20 msgr2=0x7f84ac069fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.321+0000 7f84b2a47640 1 --2- 172.21.15.2:0/2974753630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac069b20 0x7f84ac069fa0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009930 tx=0x7f84a002bcd0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.321+0000 7f84b2a47640 1 -- 172.21.15.2:0/2974753630 shutdown_connections 2024-03-15T12:59:16.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.321+0000 7f84b2a47640 1 --2- 172.21.15.2:0/2974753630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84ac06a4e0 0x7f84ac10e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.321+0000 7f84b2a47640 1 --2- 172.21.15.2:0/2974753630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac069b20 0x7f84ac069fa0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.321+0000 7f84b2a47640 1 --2- 172.21.15.2:0/2974753630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84ac107600 0x7f84ac1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.321+0000 7f84b2a47640 1 -- 172.21.15.2:0/2974753630 >> 172.21.15.2:0/2974753630 conn(0x7f84ac077190 msgr2=0x7f84ac0775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.321+0000 7f84b2a47640 1 -- 172.21.15.2:0/2974753630 shutdown_connections 2024-03-15T12:59:16.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.321+0000 7f84b2a47640 1 -- 172.21.15.2:0/2974753630 wait complete. 2024-03-15T12:59:16.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.321+0000 7f84b2a47640 1 Processor -- start 2024-03-15T12:59:16.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84b2a47640 1 -- start start 2024-03-15T12:59:16.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84b2a47640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac069b20 0x7f84ac19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84b2a47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84ac06a4e0 0x7f84ac19dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84b2a47640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84ac107600 0x7f84ac197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84b2a47640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac197f60 con 0x7f84ac069b20 2024-03-15T12:59:16.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84b2a47640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac1980d0 con 0x7f84ac107600 2024-03-15T12:59:16.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84b2a47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac198290 con 0x7f84ac06a4e0 2024-03-15T12:59:16.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac069b20 0x7f84ac19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac069b20 0x7f84ac19d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52654/0 (socket says 172.21.15.2:52654) 2024-03-15T12:59:16.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84abfff640 1 -- 172.21.15.2:0/1530972947 learned_addr learned my addr 172.21.15.2:0/1530972947 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84b0fe4640 1 --2- 172.21.15.2:0/1530972947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84ac107600 0x7f84ac197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84ab7fe640 1 --2- 172.21.15.2:0/1530972947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84ac06a4e0 0x7f84ac19dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84abfff640 1 -- 172.21.15.2:0/1530972947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84ac06a4e0 msgr2=0x7f84ac19dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84abfff640 1 --2- 172.21.15.2:0/1530972947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84ac06a4e0 0x7f84ac19dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84abfff640 1 -- 172.21.15.2:0/1530972947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84ac107600 msgr2=0x7f84ac197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84abfff640 1 --2- 172.21.15.2:0/1530972947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84ac107600 0x7f84ac197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84abfff640 1 -- 172.21.15.2:0/1530972947 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a0009560 con 0x7f84ac069b20 2024-03-15T12:59:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.322+0000 7f84abfff640 1 --2- 172.21.15.2:0/1530972947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac069b20 0x7f84ac19d710 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f849c00e9f0 tx=0x7f849c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.323+0000 7f84a97fa640 1 -- 172.21.15.2:0/1530972947 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c009700 con 0x7f84ac069b20 2024-03-15T12:59:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.323+0000 7f84a97fa640 1 -- 172.21.15.2:0/1530972947 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f849c00cd80 con 0x7f84ac069b20 2024-03-15T12:59:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.323+0000 7f84b2a47640 1 -- 172.21.15.2:0/1530972947 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84ac198570 con 0x7f84ac069b20 2024-03-15T12:59:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.323+0000 7f84a97fa640 1 -- 172.21.15.2:0/1530972947 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c018920 con 0x7f84ac069b20 2024-03-15T12:59:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.323+0000 7f84b2a47640 1 -- 172.21.15.2:0/1530972947 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ac1a47b0 con 0x7f84ac069b20 2024-03-15T12:59:16.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84a97fa640 1 -- 172.21.15.2:0/1530972947 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7f849c018a80 con 0x7f84ac069b20 2024-03-15T12:59:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84b5624740 1 Processor -- start 2024-03-15T12:59:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84b5624740 1 -- start start 2024-03-15T12:59:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84a97fa640 1 --2- 172.21.15.2:0/1530972947 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f84940a1cc0 0x7f84940a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84b5624740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e6d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84b5624740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4e76270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84b5624740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e767b0 0x55a7a4e78ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84b5624740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a4ddc280 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84b5624740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a4dd97b0 con 0x55a7a4e767b0 2024-03-15T12:59:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84b5624740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a4dd4f90 con 0x55a7a4e6dbd0 2024-03-15T12:59:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84b0fe4640 1 --2- 172.21.15.2:0/1530972947 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f84940a1cc0 0x7f84940a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84a97fa640 1 -- 172.21.15.2:0/1530972947 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f849c014070 con 0x7f84ac069b20 2024-03-15T12:59:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e6d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e6d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52664/0 (socket says 172.21.15.2:52664) 2024-03-15T12:59:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84abfff640 1 -- 172.21.15.2:0/3525850168 learned_addr learned my addr 172.21.15.2:0/3525850168 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84ab7fe640 1 --2- 172.21.15.2:0/3525850168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4e76270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.325+0000 7f84b0fe4640 1 --2- 172.21.15.2:0/3525850168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e767b0 0x55a7a4e78ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.326+0000 7f84abfff640 1 -- 172.21.15.2:0/3525850168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e6dbd0 msgr2=0x55a7a4e76270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.326+0000 7f84abfff640 1 --2- 172.21.15.2:0/3525850168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4e76270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.326+0000 7f84abfff640 1 -- 172.21.15.2:0/3525850168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e767b0 msgr2=0x55a7a4e78ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.326+0000 7f84abfff640 1 --2- 172.21.15.2:0/3525850168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e767b0 0x55a7a4e78ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.326+0000 7f84abfff640 1 -- 172.21.15.2:0/3525850168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7a4d93570 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.326+0000 7f84abfff640 1 --2- 172.21.15.2:0/3525850168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e6d690 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f849c00c6a0 tx=0x7f849c09a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db05fa6d5f08891 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.326+0000 7f84b0fe4640 1 --2- 172.21.15.2:0/1530972947 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f84940a1cc0 0x7f84940a4180 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84a4005fd0 tx=0x7f84a4009b80 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.326+0000 7f84927fc640 1 -- 172.21.15.2:0/3525850168 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c09a860 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.326+0000 7f84927fc640 1 -- 172.21.15.2:0/3525850168 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f849c09e900 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.326+0000 7f84927fc640 1 -- 172.21.15.2:0/3525850168 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c07e960 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.327+0000 7f84b5624740 1 -- 172.21.15.2:0/3525850168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 msgr2=0x55a7a4e6d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.327+0000 7f84b5624740 1 --2- 172.21.15.2:0/3525850168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e6d690 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f849c00c6a0 tx=0x7f849c09a760 comp rx=0 tx=0).stop 2024-03-15T12:59:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.327+0000 7f84b5624740 1 -- 172.21.15.2:0/3525850168 shutdown_connections 2024-03-15T12:59:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.327+0000 7f84b5624740 1 --2- 172.21.15.2:0/3525850168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e767b0 0x55a7a4e78ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.327+0000 7f84b5624740 1 --2- 172.21.15.2:0/3525850168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4e76270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.327+0000 7f84b5624740 1 --2- 172.21.15.2:0/3525850168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e6d690 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.327+0000 7f84b5624740 1 -- 172.21.15.2:0/3525850168 >> 172.21.15.2:0/3525850168 conn(0x55a7a4e67990 msgr2=0x55a7a4e67d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.327+0000 7f84b5624740 1 -- 172.21.15.2:0/3525850168 shutdown_connections 2024-03-15T12:59:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.327+0000 7f84b5624740 1 -- 172.21.15.2:0/3525850168 wait complete. 2024-03-15T12:59:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.327+0000 7f84b5624740 1 -- start start 2024-03-15T12:59:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.327+0000 7f84b5624740 1 Processor -- start 2024-03-15T12:59:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b5624740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e73640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b5624740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4e73b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b5624740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e767b0 0x55a7a4e740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b5624740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a4aa8430 con 0x55a7a4e767b0 2024-03-15T12:59:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b5624740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a4a64e00 con 0x55a7a4e6dbd0 2024-03-15T12:59:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b5624740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a4d59fd0 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b5624740 10 client.0 _ll_register_callbacks cb 0x55a7a4daffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b0fe4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e767b0 0x55a7a4e740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b0fe4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e767b0 0x55a7a4e740c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52674/0 (socket says 172.21.15.2:52674) 2024-03-15T12:59:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84ab7fe640 1 --2- 172.21.15.2:0/339752232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4e73b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b0fe4640 1 -- 172.21.15.2:0/339752232 learned_addr learned my addr 172.21.15.2:0/339752232 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84abfff640 1 --2- 172.21.15.2:0/339752232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e73640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b0fe4640 1 -- 172.21.15.2:0/339752232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e6d2b0 msgr2=0x55a7a4e73640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b0fe4640 1 --2- 172.21.15.2:0/339752232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e73640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b0fe4640 1 -- 172.21.15.2:0/339752232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e6dbd0 msgr2=0x55a7a4e73b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b0fe4640 1 --2- 172.21.15.2:0/339752232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4e73b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b0fe4640 1 -- 172.21.15.2:0/339752232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ac1a47b0 con 0x55a7a4e767b0 2024-03-15T12:59:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b0fe4640 1 --2- 172.21.15.2:0/339752232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e767b0 0x55a7a4e740c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f84a4013fd0 tx=0x7f84a4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f342aa386e63e97e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f848b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f848b7fe640 1 -- 172.21.15.2:0/339752232 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4010d80 con 0x55a7a4e767b0 2024-03-15T12:59:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f848b7fe640 1 -- 172.21.15.2:0/339752232 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f84a402a470 con 0x55a7a4e767b0 2024-03-15T12:59:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.328+0000 7f84b5624740 1 -- 172.21.15.2:0/339752232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7a4ad47e0 con 0x55a7a4e767b0 2024-03-15T12:59:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.329+0000 7f848b7fe640 1 -- 172.21.15.2:0/339752232 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84a402a630 con 0x55a7a4e767b0 2024-03-15T12:59:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.329+0000 7f84b5624740 10 client.15840 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.329+0000 7f84b5624740 1 -- 172.21.15.2:0/339752232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a7a4dd3610 con 0x55a7a4e767b0 2024-03-15T12:59:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.329+0000 7f848b7fe640 1 -- 172.21.15.2:0/339752232 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55a7a4ad47e0 con 0x55a7a4e767b0 2024-03-15T12:59:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.329+0000 7f848b7fe640 1 -- 172.21.15.2:0/339752232 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f84a4021410 con 0x55a7a4e767b0 2024-03-15T12:59:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.329+0000 7f84b5624740 10 client.15840 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.329+0000 7f84b5624740 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.329+0000 7f84b5624740 20 client.15840 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.329+0000 7f84b5624740 1 --2- 172.21.15.2:0/339752232 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a7a4e86a70 0x55a7a4e88e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.329+0000 7f84b5624740 4 client.15840 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.329+0000 7f84b5624740 1 -- 172.21.15.2:0/339752232 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7a4b11720 con 0x55a7a4e86a70 2024-03-15T12:59:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.329+0000 7f84abfff640 1 --2- 172.21.15.2:0/339752232 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a7a4e86a70 0x55a7a4e88e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.329+0000 7f84abfff640 1 --2- 172.21.15.2:0/339752232 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a7a4e86a70 0x55a7a4e88e90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69c7f4e96621fcbf server_cookie=750cd90168c43f3e in_seq=0 out_seq=0 2024-03-15T12:59:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.329+0000 7f848b7fe640 10 client.15840 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.330+0000 7f848b7fe640 1 -- 172.21.15.2:0/339752232 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a7a4b11720 con 0x55a7a4e86a70 2024-03-15T12:59:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.330+0000 7f848b7fe640 10 client.15840 handle_command_reply: tid=0 2024-03-15T12:59:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.330+0000 7f84b5624740 1 client.15840 shutdown 2024-03-15T12:59:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.330+0000 7f84b5624740 1 -- 172.21.15.2:0/339752232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e767b0 msgr2=0x55a7a4e740c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.330+0000 7f84b5624740 1 --2- 172.21.15.2:0/339752232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e767b0 0x55a7a4e740c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f84a4013fd0 tx=0x7f84a4004ab0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.330+0000 7f84b5624740 1 -- 172.21.15.2:0/339752232 shutdown_connections 2024-03-15T12:59:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.330+0000 7f84b5624740 1 --2- 172.21.15.2:0/339752232 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a7a4e86a70 0x55a7a4e88e90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.330+0000 7f84b5624740 1 --2- 172.21.15.2:0/339752232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e767b0 0x55a7a4e740c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.330+0000 7f84b5624740 1 --2- 172.21.15.2:0/339752232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4e73b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.330+0000 7f848b7fe640 0 client.15840 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.330+0000 7f84b5624740 1 --2- 172.21.15.2:0/339752232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e73640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.330+0000 7f84b5624740 1 -- 172.21.15.2:0/339752232 >> 172.21.15.2:0/339752232 conn(0x55a7a4e67990 msgr2=0x55a7a4e69730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.330+0000 7f84b5624740 1 -- 172.21.15.2:0/339752232 shutdown_connections 2024-03-15T12:59:16.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.330+0000 7f84b5624740 1 -- 172.21.15.2:0/339752232 wait complete. 2024-03-15T12:59:16.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.331+0000 7f84b5624740 20 client.15840 trim_cache size 0 max 16384 2024-03-15T12:59:16.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.346+0000 7f84b5624740 1 Processor -- start 2024-03-15T12:59:16.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.346+0000 7f84b5624740 1 -- start start 2024-03-15T12:59:16.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84b5624740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e75d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84b5624740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4e73ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84b5624740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e767b0 0x55a7a4e668d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84b5624740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a4e69cf0 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84b5624740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a4e671f0 con 0x55a7a4e6dbd0 2024-03-15T12:59:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84b5624740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a4e86820 con 0x55a7a4e767b0 2024-03-15T12:59:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e75d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e75d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52680/0 (socket says 172.21.15.2:52680) 2024-03-15T12:59:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84abfff640 1 -- 172.21.15.2:0/318616076 learned_addr learned my addr 172.21.15.2:0/318616076 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84ab7fe640 1 --2- 172.21.15.2:0/318616076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4e73ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84b0fe4640 1 --2- 172.21.15.2:0/318616076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e767b0 0x55a7a4e668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84abfff640 1 -- 172.21.15.2:0/318616076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e767b0 msgr2=0x55a7a4e668d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84abfff640 1 --2- 172.21.15.2:0/318616076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e767b0 0x55a7a4e668d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84abfff640 1 -- 172.21.15.2:0/318616076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e6dbd0 msgr2=0x55a7a4e73ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84abfff640 1 --2- 172.21.15.2:0/318616076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4e73ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84abfff640 1 -- 172.21.15.2:0/318616076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7a4d6fac0 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f84abfff640 1 --2- 172.21.15.2:0/318616076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e75d70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f849c000c00 tx=0x7f849c085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf79b81a42682c78 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f8491ffb640 1 -- 172.21.15.2:0/318616076 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c010be0 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f8491ffb640 1 -- 172.21.15.2:0/318616076 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f849c096ed0 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.347+0000 7f8491ffb640 1 -- 172.21.15.2:0/318616076 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c0ba370 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.348+0000 7f84b5624740 1 -- 172.21.15.2:0/318616076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 msgr2=0x55a7a4e75d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.348+0000 7f84b5624740 1 --2- 172.21.15.2:0/318616076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e75d70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f849c000c00 tx=0x7f849c085040 comp rx=0 tx=0).stop 2024-03-15T12:59:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.348+0000 7f84b5624740 1 -- 172.21.15.2:0/318616076 shutdown_connections 2024-03-15T12:59:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.348+0000 7f84b5624740 1 --2- 172.21.15.2:0/318616076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e767b0 0x55a7a4e668d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.348+0000 7f84b5624740 1 --2- 172.21.15.2:0/318616076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4e73ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.348+0000 7f84b5624740 1 --2- 172.21.15.2:0/318616076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4e75d70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.348+0000 7f84b5624740 1 -- 172.21.15.2:0/318616076 >> 172.21.15.2:0/318616076 conn(0x55a7a4e67990 msgr2=0x55a7a4e661c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.348+0000 7f84b5624740 1 -- 172.21.15.2:0/318616076 shutdown_connections 2024-03-15T12:59:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.348+0000 7f84b5624740 1 -- 172.21.15.2:0/318616076 wait complete. 2024-03-15T12:59:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.348+0000 7f84b5624740 1 -- start start 2024-03-15T12:59:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.348+0000 7f84b5624740 1 Processor -- start 2024-03-15T12:59:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84b5624740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4edb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84b5624740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4edbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84b5624740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e767b0 0x55a7a4e7a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84b5624740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a4e7a810 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84b5624740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a4e7a980 con 0x55a7a4e767b0 2024-03-15T12:59:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84b5624740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a4e7ab40 con 0x55a7a4e6dbd0 2024-03-15T12:59:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84b5624740 10 client.0 _ll_register_callbacks cb 0x7f84a4009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4edb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4edb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52694/0 (socket says 172.21.15.2:52694) 2024-03-15T12:59:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84abfff640 1 -- 172.21.15.2:0/2590818714 learned_addr learned my addr 172.21.15.2:0/2590818714 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84b0fe4640 1 --2- 172.21.15.2:0/2590818714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e767b0 0x55a7a4e7a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84ab7fe640 1 --2- 172.21.15.2:0/2590818714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4edbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84abfff640 1 -- 172.21.15.2:0/2590818714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e6dbd0 msgr2=0x55a7a4edbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84abfff640 1 --2- 172.21.15.2:0/2590818714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4edbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84abfff640 1 -- 172.21.15.2:0/2590818714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e767b0 msgr2=0x55a7a4e7a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84abfff640 1 --2- 172.21.15.2:0/2590818714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e767b0 0x55a7a4e7a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84abfff640 1 -- 172.21.15.2:0/2590818714 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7a4d93570 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84abfff640 1 --2- 172.21.15.2:0/2590818714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4edb6a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f849c0cb4b0 tx=0x7f849c0ce130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0d68302c347431 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f848b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f848b7fe640 1 -- 172.21.15.2:0/2590818714 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c0cb930 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f848b7fe640 1 -- 172.21.15.2:0/2590818714 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f849c097040 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.349+0000 7f84b5624740 1 -- 172.21.15.2:0/2590818714 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f849c09e900 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.350+0000 7f848b7fe640 1 -- 172.21.15.2:0/2590818714 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f849c0b3170 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.350+0000 7f84b5624740 10 client.15846 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.350+0000 7f84b5624740 1 -- 172.21.15.2:0/2590818714 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a7a4d956a0 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.350+0000 7f848b7fe640 1 -- 172.21.15.2:0/2590818714 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f849c09e900 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.350+0000 7f848b7fe640 1 -- 172.21.15.2:0/2590818714 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f849c09bb70 con 0x55a7a4e6d2b0 2024-03-15T12:59:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.350+0000 7f84b5624740 10 client.15846 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.350+0000 7f84b5624740 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.350+0000 7f84b5624740 20 client.15846 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.350+0000 7f84b5624740 1 --2- 172.21.15.2:0/2590818714 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a7a4ee3070 0x55a7a4ee5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.350+0000 7f84b5624740 4 client.15846 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T12:59:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.350+0000 7f84b5624740 1 -- 172.21.15.2:0/2590818714 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a7a4b60890 con 0x55a7a4ee3070 2024-03-15T12:59:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.350+0000 7f84ab7fe640 1 --2- 172.21.15.2:0/2590818714 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a7a4ee3070 0x55a7a4ee5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.350+0000 7f848b7fe640 10 client.15846 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.350+0000 7f84ab7fe640 1 --2- 172.21.15.2:0/2590818714 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a7a4ee3070 0x55a7a4ee5460 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=526e1d69ace2d825 server_cookie=54003d92e7956241 in_seq=0 out_seq=0 2024-03-15T12:59:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f848b7fe640 1 -- 172.21.15.2:0/2590818714 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a7a4b60890 con 0x55a7a4ee3070 2024-03-15T12:59:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f848b7fe640 10 client.15846 handle_command_reply: tid=0 2024-03-15T12:59:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b5624740 1 client.15846 shutdown 2024-03-15T12:59:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b5624740 1 -- 172.21.15.2:0/2590818714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 msgr2=0x55a7a4edb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b5624740 1 --2- 172.21.15.2:0/2590818714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4edb6a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f849c0cb4b0 tx=0x7f849c0ce130 comp rx=0 tx=0).stop 2024-03-15T12:59:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b5624740 1 -- 172.21.15.2:0/2590818714 shutdown_connections 2024-03-15T12:59:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b5624740 1 --2- 172.21.15.2:0/2590818714 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a7a4ee3070 0x55a7a4ee5460 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b5624740 1 --2- 172.21.15.2:0/2590818714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7a4e767b0 0x55a7a4e7a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b5624740 1 --2- 172.21.15.2:0/2590818714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7a4e6dbd0 0x55a7a4edbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f848b7fe640 0 client.15846 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:16.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b5624740 1 --2- 172.21.15.2:0/2590818714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7a4e6d2b0 0x55a7a4edb6a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b5624740 1 -- 172.21.15.2:0/2590818714 >> 172.21.15.2:0/2590818714 conn(0x55a7a4e67990 msgr2=0x55a7a4e66a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b5624740 1 -- 172.21.15.2:0/2590818714 shutdown_connections 2024-03-15T12:59:16.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b5624740 1 -- 172.21.15.2:0/2590818714 wait complete. 2024-03-15T12:59:16.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b5624740 20 client.15846 trim_cache size 0 max 16384 2024-03-15T12:59:16.343 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:59:16.343 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T12:59:16.343 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T12:59:16.344 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:59:16.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b2a47640 1 -- 172.21.15.2:0/1530972947 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f84940a1cc0 msgr2=0x7f84940a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b2a47640 1 --2- 172.21.15.2:0/1530972947 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f84940a1cc0 0x7f84940a4180 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84a4005fd0 tx=0x7f84a4009b80 comp rx=0 tx=0).stop 2024-03-15T12:59:16.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b2a47640 1 -- 172.21.15.2:0/1530972947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac069b20 msgr2=0x7f84ac19d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.351+0000 7f84b2a47640 1 --2- 172.21.15.2:0/1530972947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac069b20 0x7f84ac19d710 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f849c00e9f0 tx=0x7f849c00eeb0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.352+0000 7f84b2a47640 1 -- 172.21.15.2:0/1530972947 shutdown_connections 2024-03-15T12:59:16.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.352+0000 7f84b2a47640 1 --2- 172.21.15.2:0/1530972947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84ac107600 0x7f84ac197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.352+0000 7f84b2a47640 1 --2- 172.21.15.2:0/1530972947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84ac06a4e0 0x7f84ac19dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.352+0000 7f84b2a47640 1 --2- 172.21.15.2:0/1530972947 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f84940a1cc0 0x7f84940a4180 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.352+0000 7f84b2a47640 1 --2- 172.21.15.2:0/1530972947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84ac069b20 0x7f84ac19d710 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.352+0000 7f84b2a47640 1 -- 172.21.15.2:0/1530972947 >> 172.21.15.2:0/1530972947 conn(0x7f84ac077190 msgr2=0x7f84ac0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.352+0000 7f84b2a47640 1 -- 172.21.15.2:0/1530972947 shutdown_connections 2024-03-15T12:59:16.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.352+0000 7f84b2a47640 1 -- 172.21.15.2:0/1530972947 wait complete. 2024-03-15T12:59:16.348 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T12:59:16.348 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:59:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.465+0000 7f42cba23640 1 -- 172.21.15.2:0/2249183004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42c41053d0 msgr2=0x7f42c41057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.465+0000 7f42cba23640 1 --2- 172.21.15.2:0/2249183004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42c41053d0 0x7f42c41057b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009cd0 tx=0x7f42bc02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.465+0000 7f42cba23640 1 -- 172.21.15.2:0/2249183004 shutdown_connections 2024-03-15T12:59:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.465+0000 7f42cba23640 1 --2- 172.21.15.2:0/2249183004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42c4101540 0x7f42c410d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.465+0000 7f42cba23640 1 --2- 172.21.15.2:0/2249183004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c4100b80 0x7f42c4101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.465+0000 7f42cba23640 1 --2- 172.21.15.2:0/2249183004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42c41053d0 0x7f42c41057b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.465+0000 7f42cba23640 1 -- 172.21.15.2:0/2249183004 >> 172.21.15.2:0/2249183004 conn(0x7f42c4076980 msgr2=0x7f42c40fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.465+0000 7f42cba23640 1 -- 172.21.15.2:0/2249183004 shutdown_connections 2024-03-15T12:59:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.465+0000 7f42cba23640 1 -- 172.21.15.2:0/2249183004 wait complete. 2024-03-15T12:59:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42cba23640 1 Processor -- start 2024-03-15T12:59:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42cba23640 1 -- start start 2024-03-15T12:59:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42cba23640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42c4100b80 0x7f42c419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42cba23640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42c4101540 0x7f42c419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42cba23640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c41053d0 0x7f42c41978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42cba23640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c4197f00 con 0x7f42c41053d0 2024-03-15T12:59:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42cba23640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c4198070 con 0x7f42c4100b80 2024-03-15T12:59:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42cba23640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c4198230 con 0x7f42c4101540 2024-03-15T12:59:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42c9fc0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c41053d0 0x7f42c41978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42c9fc0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c41053d0 0x7f42c41978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52716/0 (socket says 172.21.15.2:52716) 2024-03-15T12:59:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42c9fc0640 1 -- 172.21.15.2:0/169507063 learned_addr learned my addr 172.21.15.2:0/169507063 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42c97bf640 1 --2- 172.21.15.2:0/169507063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42c4100b80 0x7f42c419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42c8fbe640 1 --2- 172.21.15.2:0/169507063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42c4101540 0x7f42c419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42c9fc0640 1 -- 172.21.15.2:0/169507063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42c4101540 msgr2=0x7f42c419dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42c9fc0640 1 --2- 172.21.15.2:0/169507063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42c4101540 0x7f42c419dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42c9fc0640 1 -- 172.21.15.2:0/169507063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42c4100b80 msgr2=0x7f42c419d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42c9fc0640 1 --2- 172.21.15.2:0/169507063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42c4100b80 0x7f42c419d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42c9fc0640 1 -- 172.21.15.2:0/169507063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc009900 con 0x7f42c41053d0 2024-03-15T12:59:16.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.466+0000 7f42c9fc0640 1 --2- 172.21.15.2:0/169507063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c41053d0 0x7f42c41978b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f42b400e970 tx=0x7f42b400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.467+0000 7f42c27fc640 1 -- 172.21.15.2:0/169507063 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4009720 con 0x7f42c41053d0 2024-03-15T12:59:16.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.467+0000 7f42cba23640 1 -- 172.21.15.2:0/169507063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c4198510 con 0x7f42c41053d0 2024-03-15T12:59:16.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.467+0000 7f42c27fc640 1 -- 172.21.15.2:0/169507063 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f42b40054f0 con 0x7f42c41053d0 2024-03-15T12:59:16.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.467+0000 7f42c27fc640 1 -- 172.21.15.2:0/169507063 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4010430 con 0x7f42c41053d0 2024-03-15T12:59:16.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.467+0000 7f42cba23640 1 -- 172.21.15.2:0/169507063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c41a47a0 con 0x7f42c41053d0 2024-03-15T12:59:16.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.468+0000 7f42cba23640 1 -- 172.21.15.2:0/169507063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c4068730 con 0x7f42c41053d0 2024-03-15T12:59:16.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.469+0000 7f42c27fc640 1 -- 172.21.15.2:0/169507063 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7f42b4004f90 con 0x7f42c41053d0 2024-03-15T12:59:16.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.469+0000 7f42c27fc640 1 --2- 172.21.15.2:0/169507063 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f42ac0a1cc0 0x7f42ac0a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.469+0000 7f42c97bf640 1 --2- 172.21.15.2:0/169507063 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f42ac0a1cc0 0x7f42ac0a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.469+0000 7f42c27fc640 1 -- 172.21.15.2:0/169507063 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f42b4014070 con 0x7f42c41053d0 2024-03-15T12:59:16.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.470+0000 7f42c97bf640 1 --2- 172.21.15.2:0/169507063 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f42ac0a1cc0 0x7f42ac0a4180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f42bc005cc0 tx=0x7f42bc005c50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.474+0000 7f42c27fc640 1 -- 172.21.15.2:0/169507063 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f42b4096b10 con 0x7f42c41053d0 2024-03-15T12:59:16.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.695+0000 7f42cba23640 1 -- 172.21.15.2:0/169507063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42c4109930 con 0x7f42c41053d0 2024-03-15T12:59:16.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.696+0000 7f42c27fc640 1 -- 172.21.15.2:0/169507063 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11453 (secure 0 0 0) 0x7f42b409a1d0 con 0x7f42c41053d0 2024-03-15T12:59:16.681 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:59:16.682 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T12:58:37.814324+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24434,24539,24460,24464,24500]},"id":1}]} 2024-03-15T12:59:16.682 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 29 2024-03-15T12:59:16.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.698+0000 7f42cba23640 1 -- 172.21.15.2:0/169507063 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f42ac0a1cc0 msgr2=0x7f42ac0a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.698+0000 7f42cba23640 1 --2- 172.21.15.2:0/169507063 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f42ac0a1cc0 0x7f42ac0a4180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f42bc005cc0 tx=0x7f42bc005c50 comp rx=0 tx=0).stop 2024-03-15T12:59:16.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.698+0000 7f42cba23640 1 -- 172.21.15.2:0/169507063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c41053d0 msgr2=0x7f42c41978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.698+0000 7f42cba23640 1 --2- 172.21.15.2:0/169507063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c41053d0 0x7f42c41978b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f42b400e970 tx=0x7f42b400ed70 comp rx=0 tx=0).stop 2024-03-15T12:59:16.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.698+0000 7f42cba23640 1 -- 172.21.15.2:0/169507063 shutdown_connections 2024-03-15T12:59:16.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.698+0000 7f42cba23640 1 --2- 172.21.15.2:0/169507063 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f42ac0a1cc0 0x7f42ac0a4180 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.698+0000 7f42cba23640 1 --2- 172.21.15.2:0/169507063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c41053d0 0x7f42c41978b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.698+0000 7f42cba23640 1 --2- 172.21.15.2:0/169507063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42c4101540 0x7f42c419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.698+0000 7f42cba23640 1 --2- 172.21.15.2:0/169507063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42c4100b80 0x7f42c419d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.698+0000 7f42cba23640 1 -- 172.21.15.2:0/169507063 >> 172.21.15.2:0/169507063 conn(0x7f42c4076980 msgr2=0x7f42c40ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.698+0000 7f42cba23640 1 -- 172.21.15.2:0/169507063 shutdown_connections 2024-03-15T12:59:16.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.698+0000 7f42cba23640 1 -- 172.21.15.2:0/169507063 wait complete. 2024-03-15T12:59:16.694 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T12:59:16.695 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T12:59:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.813+0000 7f8befea3640 1 -- 172.21.15.2:0/1661216252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be8101e60 msgr2=0x7f8be810e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.813+0000 7f8befea3640 1 --2- 172.21.15.2:0/1661216252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be8101e60 0x7f8be810e320 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009ca0 tx=0x7f8be002c0f0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.813+0000 7f8befea3640 1 -- 172.21.15.2:0/1661216252 shutdown_connections 2024-03-15T12:59:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.813+0000 7f8befea3640 1 --2- 172.21.15.2:0/1661216252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be8101e60 0x7f8be810e320 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.813+0000 7f8befea3640 1 --2- 172.21.15.2:0/1661216252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8be81014a0 0x7f8be8101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.813+0000 7f8befea3640 1 --2- 172.21.15.2:0/1661216252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8be8107560 0x7f8be8107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.813+0000 7f8befea3640 1 -- 172.21.15.2:0/1661216252 >> 172.21.15.2:0/1661216252 conn(0x7f8be80fcf90 msgr2=0x7f8be80ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.813+0000 7f8befea3640 1 -- 172.21.15.2:0/1661216252 shutdown_connections 2024-03-15T12:59:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.813+0000 7f8befea3640 1 -- 172.21.15.2:0/1661216252 wait complete. 2024-03-15T12:59:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8befea3640 1 Processor -- start 2024-03-15T12:59:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8befea3640 1 -- start start 2024-03-15T12:59:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8befea3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be81014a0 0x7f8be80737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8befea3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8be8101e60 0x7f8be8073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8befea3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8be8107560 0x7f8be8074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8befea3640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be81a5e80 con 0x7f8be81014a0 2024-03-15T12:59:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8befea3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be81a5ff0 con 0x7f8be8107560 2024-03-15T12:59:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8befea3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be81a61b0 con 0x7f8be8101e60 2024-03-15T12:59:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8bedc3f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be81014a0 0x7f8be80737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8bedc3f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be81014a0 0x7f8be80737f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52730/0 (socket says 172.21.15.2:52730) 2024-03-15T12:59:16.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8bedc3f640 1 -- 172.21.15.2:0/83426003 learned_addr learned my addr 172.21.15.2:0/83426003 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8bee440640 1 --2- 172.21.15.2:0/83426003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8be8107560 0x7f8be8074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8bed43e640 1 --2- 172.21.15.2:0/83426003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8be8101e60 0x7f8be8073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8bedc3f640 1 -- 172.21.15.2:0/83426003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8be8101e60 msgr2=0x7f8be8073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8bedc3f640 1 --2- 172.21.15.2:0/83426003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8be8101e60 0x7f8be8073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8bedc3f640 1 -- 172.21.15.2:0/83426003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8be8107560 msgr2=0x7f8be8074270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8bedc3f640 1 --2- 172.21.15.2:0/83426003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8be8107560 0x7f8be8074270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.814+0000 7f8bedc3f640 1 -- 172.21.15.2:0/83426003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be0009900 con 0x7f8be81014a0 2024-03-15T12:59:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.815+0000 7f8bedc3f640 1 --2- 172.21.15.2:0/83426003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be81014a0 0x7f8be80737f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00e9f0 tx=0x7f8bdc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.815+0000 7f8bdaffd640 1 -- 172.21.15.2:0/83426003 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc009700 con 0x7f8be81014a0 2024-03-15T12:59:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.815+0000 7f8befea3640 1 -- 172.21.15.2:0/83426003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be81a6490 con 0x7f8be81014a0 2024-03-15T12:59:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.815+0000 7f8bdaffd640 1 -- 172.21.15.2:0/83426003 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8bdc00cd80 con 0x7f8be81014a0 2024-03-15T12:59:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.815+0000 7f8bdaffd640 1 -- 172.21.15.2:0/83426003 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc018920 con 0x7f8be81014a0 2024-03-15T12:59:16.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.815+0000 7f8befea3640 1 -- 172.21.15.2:0/83426003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be81a69f0 con 0x7f8be81014a0 2024-03-15T12:59:16.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.817+0000 7f8bdaffd640 1 -- 172.21.15.2:0/83426003 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7f8bdc018a80 con 0x7f8be81014a0 2024-03-15T12:59:16.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.817+0000 7f8bf2a7d740 1 Processor -- start 2024-03-15T12:59:16.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.817+0000 7f8bf2a7d740 1 -- start start 2024-03-15T12:59:16.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.817+0000 7f8bf2a7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 0x55f33d6be630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bf2a7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bf2a7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 0x55f33d6c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bf2a7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33d679960 con 0x55f33d6beb70 2024-03-15T12:59:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bf2a7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33d62d310 con 0x55f33d6be250 2024-03-15T12:59:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bf2a7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33d62a840 con 0x55f33d6c7750 2024-03-15T12:59:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bedc3f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bedc3f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52738/0 (socket says 172.21.15.2:52738) 2024-03-15T12:59:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bdaffd640 1 --2- 172.21.15.2:0/83426003 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8bd40a1cc0 0x7f8bd40a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bedc3f640 1 -- 172.21.15.2:0/1255555232 learned_addr learned my addr 172.21.15.2:0/1255555232 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bee440640 1 --2- 172.21.15.2:0/1255555232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 0x55f33d6be630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bed43e640 1 --2- 172.21.15.2:0/1255555232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 0x55f33d6c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bee440640 1 --2- 172.21.15.2:0/83426003 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8bd40a1cc0 0x7f8bd40a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bdaffd640 1 -- 172.21.15.2:0/83426003 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f8bdc014070 con 0x7f8be81014a0 2024-03-15T12:59:16.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bed43e640 1 -- 172.21.15.2:0/1255555232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 msgr2=0x55f33d6be630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bed43e640 1 --2- 172.21.15.2:0/1255555232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 0x55f33d6be630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bed43e640 1 -- 172.21.15.2:0/1255555232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 msgr2=0x55f33d6c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bed43e640 1 --2- 172.21.15.2:0/1255555232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bed43e640 1 -- 172.21.15.2:0/1255555232 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f33d5e8060 con 0x55f33d6c7750 2024-03-15T12:59:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bedc3f640 1 --2- 172.21.15.2:0/1255555232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bed43e640 1 --2- 172.21.15.2:0/1255555232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 0x55f33d6c9b40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009bf0 tx=0x7f8be4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7bb53ae17fb97a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bee440640 1 --2- 172.21.15.2:0/83426003 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8bd40a1cc0 0x7f8bd40a4180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8be0004af0 tx=0x7f8be002c0f0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.818+0000 7f8bd3fff640 1 -- 172.21.15.2:0/1255555232 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4004050 con 0x55f33d6c7750 2024-03-15T12:59:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.819+0000 7f8bd3fff640 1 -- 172.21.15.2:0/1255555232 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8be4005020 con 0x55f33d6c7750 2024-03-15T12:59:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.819+0000 7f8bd3fff640 1 -- 172.21.15.2:0/1255555232 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4004cd0 con 0x55f33d6c7750 2024-03-15T12:59:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.819+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/1255555232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 msgr2=0x55f33d6c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.819+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/1255555232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 0x55f33d6c9b40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009bf0 tx=0x7f8be4011810 comp rx=0 tx=0).stop 2024-03-15T12:59:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.819+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/1255555232 shutdown_connections 2024-03-15T12:59:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.819+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/1255555232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 0x55f33d6c9b40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.819+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/1255555232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.819+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/1255555232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 0x55f33d6be630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.819+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/1255555232 >> 172.21.15.2:0/1255555232 conn(0x55f33d6b8980 msgr2=0x55f33d6b8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.819+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/1255555232 shutdown_connections 2024-03-15T12:59:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.819+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/1255555232 wait complete. 2024-03-15T12:59:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bf2a7d740 1 -- start start 2024-03-15T12:59:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bf2a7d740 1 Processor -- start 2024-03-15T12:59:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bf2a7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 0x55f33d6c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bf2a7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6c4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bf2a7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 0x55f33d6c4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bf2a7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33d626020 con 0x55f33d6beb70 2024-03-15T12:59:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bf2a7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33d2f9430 con 0x55f33d6be250 2024-03-15T12:59:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bf2a7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33d2b5e00 con 0x55f33d6c7750 2024-03-15T12:59:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bf2a7d740 10 client.0 _ll_register_callbacks cb 0x55f33d5cdcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:16.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bed43e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bed43e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6c4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52754/0 (socket says 172.21.15.2:52754) 2024-03-15T12:59:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bed43e640 1 -- 172.21.15.2:0/2834211213 learned_addr learned my addr 172.21.15.2:0/2834211213 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bee440640 1 --2- 172.21.15.2:0/2834211213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 0x55f33d6c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bedc3f640 1 --2- 172.21.15.2:0/2834211213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 0x55f33d6c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bed43e640 1 -- 172.21.15.2:0/2834211213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 msgr2=0x55f33d6c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bed43e640 1 --2- 172.21.15.2:0/2834211213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 0x55f33d6c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bed43e640 1 -- 172.21.15.2:0/2834211213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 msgr2=0x55f33d6c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.820+0000 7f8bed43e640 1 --2- 172.21.15.2:0/2834211213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 0x55f33d6c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bed43e640 1 -- 172.21.15.2:0/2834211213 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be4009850 con 0x55f33d6beb70 2024-03-15T12:59:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bed43e640 1 --2- 172.21.15.2:0/2834211213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6c4430 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009350 tx=0x7f8be4009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a849791e3e3d2a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bd17fa640 1 -- 172.21.15.2:0/2834211213 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4004e60 con 0x55f33d6beb70 2024-03-15T12:59:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bd17fa640 1 -- 172.21.15.2:0/2834211213 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f8be4004050 con 0x55f33d6beb70 2024-03-15T12:59:16.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/2834211213 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f33d3257e0 con 0x55f33d6beb70 2024-03-15T12:59:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bd17fa640 1 -- 172.21.15.2:0/2834211213 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8be4004210 con 0x55f33d6beb70 2024-03-15T12:59:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bf2a7d740 10 client.15864 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/2834211213 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f33d6246a0 con 0x55f33d6beb70 2024-03-15T12:59:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bd17fa640 1 -- 172.21.15.2:0/2834211213 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55f33d3257e0 con 0x55f33d6beb70 2024-03-15T12:59:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bd17fa640 1 -- 172.21.15.2:0/2834211213 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8be4021040 con 0x55f33d6beb70 2024-03-15T12:59:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bf2a7d740 10 client.15864 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bf2a7d740 10 client.15864 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T12:59:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bf2a7d740 20 client.15864 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/2834211213 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55f33d6d77d0 0x55f33d6d9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bf2a7d740 4 client.15864 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.821+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/2834211213 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f33d362720 con 0x55f33d6d77d0 2024-03-15T12:59:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.822+0000 7f8bedc3f640 1 --2- 172.21.15.2:0/2834211213 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55f33d6d77d0 0x55f33d6d9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.822+0000 7f8bd17fa640 10 client.15864 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T12:59:16.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.822+0000 7f8bedc3f640 1 --2- 172.21.15.2:0/2834211213 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55f33d6d77d0 0x55f33d6d9bf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ef21f86201efb9c1 server_cookie=19e45f71d8db492a in_seq=0 out_seq=0 2024-03-15T12:59:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.823+0000 7f8bd17fa640 1 -- 172.21.15.2:0/2834211213 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f33d362720 con 0x55f33d6d77d0 2024-03-15T12:59:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.823+0000 7f8bd17fa640 10 client.15864 handle_command_reply: tid=0 2024-03-15T12:59:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.823+0000 7f8bf2a7d740 1 client.15864 shutdown 2024-03-15T12:59:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.823+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/2834211213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 msgr2=0x55f33d6c4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.823+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/2834211213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6c4430 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009350 tx=0x7f8be4009fd0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.823+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/2834211213 shutdown_connections 2024-03-15T12:59:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.823+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/2834211213 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55f33d6d77d0 0x55f33d6d9bf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.823+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/2834211213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 0x55f33d6c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.823+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/2834211213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6c4430 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.823+0000 7f8bd17fa640 0 client.15864 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T12:59:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.823+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/2834211213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 0x55f33d6c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.823+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/2834211213 >> 172.21.15.2:0/2834211213 conn(0x55f33d6b8980 msgr2=0x55f33d6ba6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.823+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/2834211213 shutdown_connections 2024-03-15T12:59:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.823+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/2834211213 wait complete. 2024-03-15T12:59:16.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.823+0000 7f8bf2a7d740 20 client.15864 trim_cache size 0 max 16384 2024-03-15T12:59:16.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bf2a7d740 1 Processor -- start 2024-03-15T12:59:16.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bf2a7d740 1 -- start start 2024-03-15T12:59:16.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bf2a7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6be250 0x55f33d6ba440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bf2a7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6beb70 0x55f33d6c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bf2a7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6c7750 0x55f33d6c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bf2a7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33d5ab130 con 0x55f33d6c7750 2024-03-15T12:59:16.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bf2a7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33d6ca110 con 0x55f33d6beb70 2024-03-15T12:59:16.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bf2a7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33d6d4330 con 0x55f33d6be250 2024-03-15T12:59:16.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bee440640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6c7750 0x55f33d6c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bee440640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6c7750 0x55f33d6c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52760/0 (socket says 172.21.15.2:52760) 2024-03-15T12:59:16.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bee440640 1 -- 172.21.15.2:0/979214765 learned_addr learned my addr 172.21.15.2:0/979214765 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bedc3f640 1 --2- 172.21.15.2:0/979214765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6be250 0x55f33d6ba440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bed43e640 1 --2- 172.21.15.2:0/979214765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6beb70 0x55f33d6c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bee440640 1 -- 172.21.15.2:0/979214765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6be250 msgr2=0x55f33d6ba440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bee440640 1 --2- 172.21.15.2:0/979214765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6be250 0x55f33d6ba440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bee440640 1 -- 172.21.15.2:0/979214765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6beb70 msgr2=0x55f33d6c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bee440640 1 --2- 172.21.15.2:0/979214765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6beb70 0x55f33d6c6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bee440640 1 -- 172.21.15.2:0/979214765 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f33d5c0c20 con 0x55f33d6c7750 2024-03-15T12:59:16.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bed43e640 1 --2- 172.21.15.2:0/979214765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6beb70 0x55f33d6c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:59:16.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bee440640 1 --2- 172.21.15.2:0/979214765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6c7750 0x55f33d6c70a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009ca0 tx=0x7f8be0009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c0cc8e8a6ad036 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.836+0000 7f8bd37fe640 1 -- 172.21.15.2:0/979214765 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be00326b0 con 0x55f33d6c7750 2024-03-15T12:59:16.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.837+0000 7f8bd37fe640 1 -- 172.21.15.2:0/979214765 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8be0003e40 con 0x55f33d6c7750 2024-03-15T12:59:16.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.837+0000 7f8bd37fe640 1 -- 172.21.15.2:0/979214765 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0004670 con 0x55f33d6c7750 2024-03-15T12:59:16.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.837+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/979214765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6c7750 msgr2=0x55f33d6c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.837+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/979214765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6c7750 0x55f33d6c70a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009ca0 tx=0x7f8be0009360 comp rx=0 tx=0).stop 2024-03-15T12:59:16.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.837+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/979214765 shutdown_connections 2024-03-15T12:59:16.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.837+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/979214765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6c7750 0x55f33d6c70a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.837+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/979214765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6beb70 0x55f33d6c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.837+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/979214765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6be250 0x55f33d6ba440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.837+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/979214765 >> 172.21.15.2:0/979214765 conn(0x55f33d6b8980 msgr2=0x55f33d6c4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.837+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/979214765 shutdown_connections 2024-03-15T12:59:16.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.837+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/979214765 wait complete. 2024-03-15T12:59:16.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.837+0000 7f8bf2a7d740 1 -- start start 2024-03-15T12:59:16.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.837+0000 7f8bf2a7d740 1 Processor -- start 2024-03-15T12:59:16.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bf2a7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 0x55f33d6cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bf2a7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bf2a7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 0x55f33d7330b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bf2a7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33d6cd430 con 0x55f33d6beb70 2024-03-15T12:59:16.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bf2a7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33d7335f0 con 0x55f33d6be250 2024-03-15T12:59:16.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bf2a7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33d733760 con 0x55f33d6c7750 2024-03-15T12:59:16.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bf2a7d740 10 client.0 _ll_register_callbacks cb 0x7f8bdc09a640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:16.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bed43e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bed43e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6ccd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52768/0 (socket says 172.21.15.2:52768) 2024-03-15T12:59:16.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bed43e640 1 -- 172.21.15.2:0/1442848980 learned_addr learned my addr 172.21.15.2:0/1442848980 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bedc3f640 1 --2- 172.21.15.2:0/1442848980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 0x55f33d6cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bee440640 1 --2- 172.21.15.2:0/1442848980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 0x55f33d7330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bed43e640 1 -- 172.21.15.2:0/1442848980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 msgr2=0x55f33d7330b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bed43e640 1 --2- 172.21.15.2:0/1442848980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 0x55f33d7330b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bed43e640 1 -- 172.21.15.2:0/1442848980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 msgr2=0x55f33d6cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bed43e640 1 --2- 172.21.15.2:0/1442848980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 0x55f33d6cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bed43e640 1 -- 172.21.15.2:0/1442848980 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be810ec20 con 0x55f33d6beb70 2024-03-15T12:59:16.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bedc3f640 1 --2- 172.21.15.2:0/1442848980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 0x55f33d6cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:16.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bed43e640 1 --2- 172.21.15.2:0/1442848980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6ccd10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8be40234d0 tx=0x7f8be4011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7665bcc9c905b215 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.838+0000 7f8bd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:16.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.839+0000 7f8bd17fa640 1 -- 172.21.15.2:0/1442848980 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4023cf0 con 0x55f33d6beb70 2024-03-15T12:59:16.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.839+0000 7f8bd17fa640 1 -- 172.21.15.2:0/1442848980 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f8be4000e40 con 0x55f33d6beb70 2024-03-15T12:59:16.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.839+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/1442848980 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8be4005020 con 0x55f33d6beb70 2024-03-15T12:59:16.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.839+0000 7f8bd17fa640 1 -- 172.21.15.2:0/1442848980 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8be4023e50 con 0x55f33d6beb70 2024-03-15T12:59:16.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.839+0000 7f8bf2a7d740 10 client.15870 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:16.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.839+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/1442848980 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f33d5e68c0 con 0x55f33d6beb70 2024-03-15T12:59:16.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.839+0000 7f8bd17fa640 1 -- 172.21.15.2:0/1442848980 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8be4005020 con 0x55f33d6beb70 2024-03-15T12:59:16.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.839+0000 7f8bd17fa640 1 -- 172.21.15.2:0/1442848980 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8be402eba0 con 0x55f33d6beb70 2024-03-15T12:59:16.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.839+0000 7f8bf2a7d740 10 client.15870 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:16.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.839+0000 7f8bf2a7d740 10 client.15870 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T12:59:16.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.839+0000 7f8bf2a7d740 20 client.15870 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:16.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.839+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/1442848980 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55f33d734b60 0x55f33d736f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.839+0000 7f8bf2a7d740 4 client.15870 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T12:59:16.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.839+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/1442848980 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f33d3b1890 con 0x55f33d734b60 2024-03-15T12:59:16.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.839+0000 7f8bedc3f640 1 --2- 172.21.15.2:0/1442848980 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55f33d734b60 0x55f33d736f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bd17fa640 10 client.15870 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T12:59:16.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bedc3f640 1 --2- 172.21.15.2:0/1442848980 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55f33d734b60 0x55f33d736f50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=442ea944161c0631 server_cookie=611fc518d50c2e73 in_seq=0 out_seq=0 2024-03-15T12:59:16.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bd17fa640 1 -- 172.21.15.2:0/1442848980 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f33d3b1890 con 0x55f33d734b60 2024-03-15T12:59:16.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bd17fa640 10 client.15870 handle_command_reply: tid=0 2024-03-15T12:59:16.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bf2a7d740 1 client.15870 shutdown 2024-03-15T12:59:16.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/1442848980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 msgr2=0x55f33d6ccd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/1442848980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6ccd10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8be40234d0 tx=0x7f8be4011e40 comp rx=0 tx=0).stop 2024-03-15T12:59:16.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/1442848980 shutdown_connections 2024-03-15T12:59:16.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/1442848980 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55f33d734b60 0x55f33d736f50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/1442848980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f33d6c7750 0x55f33d7330b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/1442848980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f33d6beb70 0x55f33d6ccd10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bd17fa640 0 client.15870 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T12:59:16.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bf2a7d740 1 --2- 172.21.15.2:0/1442848980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f33d6be250 0x55f33d6cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/1442848980 >> 172.21.15.2:0/1442848980 conn(0x55f33d6b8980 msgr2=0x55f33d6bf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/1442848980 shutdown_connections 2024-03-15T12:59:16.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bf2a7d740 1 -- 172.21.15.2:0/1442848980 wait complete. 2024-03-15T12:59:16.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.840+0000 7f8bf2a7d740 20 client.15870 trim_cache size 0 max 16384 2024-03-15T12:59:16.833 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T12:59:16.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.841+0000 7f8befea3640 1 -- 172.21.15.2:0/83426003 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8bd40a1cc0 msgr2=0x7f8bd40a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.841+0000 7f8befea3640 1 --2- 172.21.15.2:0/83426003 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8bd40a1cc0 0x7f8bd40a4180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8be0004af0 tx=0x7f8be002c0f0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.841+0000 7f8befea3640 1 -- 172.21.15.2:0/83426003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be81014a0 msgr2=0x7f8be80737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.841+0000 7f8befea3640 1 --2- 172.21.15.2:0/83426003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be81014a0 0x7f8be80737f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00e9f0 tx=0x7f8bdc00eeb0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.841+0000 7f8befea3640 1 -- 172.21.15.2:0/83426003 shutdown_connections 2024-03-15T12:59:16.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.841+0000 7f8befea3640 1 --2- 172.21.15.2:0/83426003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8be8107560 0x7f8be8074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.841+0000 7f8befea3640 1 --2- 172.21.15.2:0/83426003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8be8101e60 0x7f8be8073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.841+0000 7f8befea3640 1 --2- 172.21.15.2:0/83426003 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8bd40a1cc0 0x7f8bd40a4180 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.841+0000 7f8befea3640 1 --2- 172.21.15.2:0/83426003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be81014a0 0x7f8be80737f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.841+0000 7f8befea3640 1 -- 172.21.15.2:0/83426003 >> 172.21.15.2:0/83426003 conn(0x7f8be80fcf90 msgr2=0x7f8be80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.841+0000 7f8befea3640 1 -- 172.21.15.2:0/83426003 shutdown_connections 2024-03-15T12:59:16.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.841+0000 7f8befea3640 1 -- 172.21.15.2:0/83426003 wait complete. 2024-03-15T12:59:16.837 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T12:59:16.837 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T12:59:16.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.957+0000 7f8479f25640 1 -- 172.21.15.2:0/2557675139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8474100410 msgr2=0x7f8474100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.957+0000 7f8479f25640 1 --2- 172.21.15.2:0/2557675139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8474100410 0x7f8474100890 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8468009930 tx=0x7f846802bc30 comp rx=0 tx=0).stop 2024-03-15T12:59:16.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.957+0000 7f8479f25640 1 -- 172.21.15.2:0/2557675139 shutdown_connections 2024-03-15T12:59:16.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.957+0000 7f8479f25640 1 --2- 172.21.15.2:0/2557675139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8474100dd0 0x7f847410d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.957+0000 7f8479f25640 1 --2- 172.21.15.2:0/2557675139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8474100410 0x7f8474100890 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.957+0000 7f8479f25640 1 --2- 172.21.15.2:0/2557675139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84741064b0 0x7f8474106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.957+0000 7f8479f25640 1 -- 172.21.15.2:0/2557675139 >> 172.21.15.2:0/2557675139 conn(0x7f8474077550 msgr2=0x7f8474077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.957+0000 7f8479f25640 1 -- 172.21.15.2:0/2557675139 shutdown_connections 2024-03-15T12:59:16.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.957+0000 7f8479f25640 1 -- 172.21.15.2:0/2557675139 wait complete. 2024-03-15T12:59:16.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.957+0000 7f8479f25640 1 Processor -- start 2024-03-15T12:59:16.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8479f25640 1 -- start start 2024-03-15T12:59:16.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8479f25640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8474100410 0x7f847419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8479f25640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8474100dd0 0x7f847419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8479f25640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84741064b0 0x7f8474197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8479f25640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8474197fd0 con 0x7f8474100dd0 2024-03-15T12:59:16.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8479f25640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8474198140 con 0x7f8474100410 2024-03-15T12:59:16.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8479f25640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8474198300 con 0x7f84741064b0 2024-03-15T12:59:16.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8472ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8474100dd0 0x7f847419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8472ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8474100dd0 0x7f847419dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52784/0 (socket says 172.21.15.2:52784) 2024-03-15T12:59:16.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f84737fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8474100410 0x7f847419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8472ffd640 1 -- 172.21.15.2:0/2020115249 learned_addr learned my addr 172.21.15.2:0/2020115249 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8473fff640 1 --2- 172.21.15.2:0/2020115249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84741064b0 0x7f8474197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8472ffd640 1 -- 172.21.15.2:0/2020115249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84741064b0 msgr2=0x7f8474197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8472ffd640 1 --2- 172.21.15.2:0/2020115249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84741064b0 0x7f8474197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8472ffd640 1 -- 172.21.15.2:0/2020115249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8474100410 msgr2=0x7f847419d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8472ffd640 1 --2- 172.21.15.2:0/2020115249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8474100410 0x7f847419d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8472ffd640 1 -- 172.21.15.2:0/2020115249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8464009490 con 0x7f8474100dd0 2024-03-15T12:59:16.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.958+0000 7f8472ffd640 1 --2- 172.21.15.2:0/2020115249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8474100dd0 0x7f847419dd10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8468009900 tx=0x7f846802ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.959+0000 7f8470ff9640 1 -- 172.21.15.2:0/2020115249 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846802cc10 con 0x7f8474100dd0 2024-03-15T12:59:16.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.959+0000 7f8470ff9640 1 -- 172.21.15.2:0/2020115249 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f84680054d0 con 0x7f8474100dd0 2024-03-15T12:59:16.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.959+0000 7f8479f25640 1 -- 172.21.15.2:0/2020115249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8468009560 con 0x7f8474100dd0 2024-03-15T12:59:16.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.959+0000 7f8470ff9640 1 -- 172.21.15.2:0/2020115249 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8468003c90 con 0x7f8474100dd0 2024-03-15T12:59:16.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.959+0000 7f8479f25640 1 -- 172.21.15.2:0/2020115249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84741a4690 con 0x7f8474100dd0 2024-03-15T12:59:16.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.961+0000 7f8470ff9640 1 -- 172.21.15.2:0/2020115249 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7f8468005000 con 0x7f8474100dd0 2024-03-15T12:59:16.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.961+0000 7f847cbde740 1 Processor -- start 2024-03-15T12:59:16.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.961+0000 7f847cbde740 1 -- start start 2024-03-15T12:59:16.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f8470ff9640 1 --2- 172.21.15.2:0/2020115249 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f845c0a1cc0 0x7f845c0a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f847cbde740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0e250 0x557c10c0e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f847cbde740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0eb70 0x557c10c17210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f847cbde740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c17750 0x557c10c19b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f847cbde740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c10bc9960 con 0x557c10c0eb70 2024-03-15T12:59:16.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f847cbde740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c10b7d310 con 0x557c10c0e250 2024-03-15T12:59:16.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f847cbde740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c10b7a840 con 0x557c10c17750 2024-03-15T12:59:16.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f8472ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0eb70 0x557c10c17210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f8473fff640 1 --2- 172.21.15.2:0/2020115249 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f845c0a1cc0 0x7f845c0a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f84737fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0e250 0x557c10c0e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f8472ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0eb70 0x557c10c17210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52792/0 (socket says 172.21.15.2:52792) 2024-03-15T12:59:16.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f8472ffd640 1 -- 172.21.15.2:0/199980609 learned_addr learned my addr 172.21.15.2:0/199980609 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f84737fe640 1 --2- 172.21.15.2:0/199980609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0e250 0x557c10c0e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:51068/0 (socket says 172.21.15.2:51068) 2024-03-15T12:59:16.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f8473fff640 1 --2- 172.21.15.2:0/199980609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c17750 0x557c10c19b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f8470ff9640 1 -- 172.21.15.2:0/2020115249 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f84680efcd0 con 0x7f8474100dd0 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f8472ffd640 1 -- 172.21.15.2:0/199980609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c17750 msgr2=0x557c10c19b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f8472ffd640 1 --2- 172.21.15.2:0/199980609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c17750 0x557c10c19b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f8472ffd640 1 -- 172.21.15.2:0/199980609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0e250 msgr2=0x557c10c0e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f8472ffd640 1 --2- 172.21.15.2:0/199980609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0e250 0x557c10c0e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f8472ffd640 1 -- 172.21.15.2:0/199980609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c10b38060 con 0x557c10c0eb70 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f84737fe640 1 --2- 172.21.15.2:0/199980609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0e250 0x557c10c0e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f8473fff640 1 --2- 172.21.15.2:0/2020115249 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f845c0a1cc0 0x7f845c0a4180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f846c005fd0 tx=0x7f846c009e60 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f8472ffd640 1 --2- 172.21.15.2:0/199980609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0eb70 0x557c10c17210 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f84680bc4d0 tx=0x7f8468009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e688654aef466749 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.962+0000 7f8459ffb640 1 -- 172.21.15.2:0/199980609 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84680dc430 con 0x557c10c0eb70 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.963+0000 7f8459ffb640 1 -- 172.21.15.2:0/199980609 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f84680035f0 con 0x557c10c0eb70 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.963+0000 7f8459ffb640 1 -- 172.21.15.2:0/199980609 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84680052b0 con 0x557c10c0eb70 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.963+0000 7f847cbde740 1 -- 172.21.15.2:0/199980609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0eb70 msgr2=0x557c10c17210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.963+0000 7f847cbde740 1 --2- 172.21.15.2:0/199980609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0eb70 0x557c10c17210 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f84680bc4d0 tx=0x7f8468009260 comp rx=0 tx=0).stop 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.963+0000 7f847cbde740 1 -- 172.21.15.2:0/199980609 shutdown_connections 2024-03-15T12:59:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.963+0000 7f847cbde740 1 --2- 172.21.15.2:0/199980609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c17750 0x557c10c19b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.963+0000 7f847cbde740 1 --2- 172.21.15.2:0/199980609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0eb70 0x557c10c17210 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.963+0000 7f847cbde740 1 --2- 172.21.15.2:0/199980609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0e250 0x557c10c0e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.963+0000 7f847cbde740 1 -- 172.21.15.2:0/199980609 >> 172.21.15.2:0/199980609 conn(0x557c10c08980 msgr2=0x557c10c08d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.963+0000 7f847cbde740 1 -- 172.21.15.2:0/199980609 shutdown_connections 2024-03-15T12:59:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.963+0000 7f847cbde740 1 -- 172.21.15.2:0/199980609 wait complete. 2024-03-15T12:59:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f847cbde740 1 -- start start 2024-03-15T12:59:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f847cbde740 1 Processor -- start 2024-03-15T12:59:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f847cbde740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c0e250 0x557c10c170a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f847cbde740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0eb70 0x557c10c14430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f847cbde740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c17750 0x557c10c14990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f847cbde740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c10b76020 con 0x557c10c0eb70 2024-03-15T12:59:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f847cbde740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c10849430 con 0x557c10c17750 2024-03-15T12:59:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f847cbde740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c10805e00 con 0x557c10c0e250 2024-03-15T12:59:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f847cbde740 10 client.0 _ll_register_callbacks cb 0x557c10b1dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f8472ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0eb70 0x557c10c14430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f8472ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0eb70 0x557c10c14430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52794/0 (socket says 172.21.15.2:52794) 2024-03-15T12:59:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f8472ffd640 1 -- 172.21.15.2:0/3641608424 learned_addr learned my addr 172.21.15.2:0/3641608424 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f84737fe640 1 --2- 172.21.15.2:0/3641608424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c0e250 0x557c10c170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f8473fff640 1 --2- 172.21.15.2:0/3641608424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c17750 0x557c10c14990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f8472ffd640 1 -- 172.21.15.2:0/3641608424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c0e250 msgr2=0x557c10c170a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f8472ffd640 1 --2- 172.21.15.2:0/3641608424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c0e250 0x557c10c170a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f8472ffd640 1 -- 172.21.15.2:0/3641608424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c17750 msgr2=0x557c10c14990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f8472ffd640 1 --2- 172.21.15.2:0/3641608424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c17750 0x557c10c14990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.964+0000 7f8472ffd640 1 -- 172.21.15.2:0/3641608424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84741a4690 con 0x557c10c0eb70 2024-03-15T12:59:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f8472ffd640 1 --2- 172.21.15.2:0/3641608424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0eb70 0x557c10c14430 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f84680ed170 tx=0x7f84680ed0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6132e278229a8aa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f8452ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f8452ffd640 1 -- 172.21.15.2:0/3641608424 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84680c0690 con 0x557c10c0eb70 2024-03-15T12:59:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f8452ffd640 1 -- 172.21.15.2:0/3641608424 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f84680bcc40 con 0x557c10c0eb70 2024-03-15T12:59:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f847cbde740 1 -- 172.21.15.2:0/3641608424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c108757e0 con 0x557c10c0eb70 2024-03-15T12:59:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f8452ffd640 1 -- 172.21.15.2:0/3641608424 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84680dc430 con 0x557c10c0eb70 2024-03-15T12:59:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f847cbde740 10 client.15882 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f847cbde740 1 -- 172.21.15.2:0/3641608424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557c10b746a0 con 0x557c10c0eb70 2024-03-15T12:59:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f8452ffd640 1 -- 172.21.15.2:0/3641608424 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x557c108757e0 con 0x557c10c0eb70 2024-03-15T12:59:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f8452ffd640 1 -- 172.21.15.2:0/3641608424 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f846803c070 con 0x557c10c0eb70 2024-03-15T12:59:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f847cbde740 10 client.15882 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f847cbde740 10 client.15882 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T12:59:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f847cbde740 20 client.15882 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f847cbde740 1 --2- 172.21.15.2:0/3641608424 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557c10c27200 0x557c10c295f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f847cbde740 4 client.15882 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.965+0000 7f847cbde740 1 -- 172.21.15.2:0/3641608424 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c108b2720 con 0x557c10c27200 2024-03-15T12:59:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.966+0000 7f84737fe640 1 --2- 172.21.15.2:0/3641608424 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557c10c27200 0x557c10c295f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.966+0000 7f84737fe640 1 --2- 172.21.15.2:0/3641608424 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557c10c27200 0x557c10c295f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fe0133c209d95468 server_cookie=10d2ceb4bcb6b23b in_seq=0 out_seq=0 2024-03-15T12:59:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.966+0000 7f8452ffd640 10 client.15882 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T12:59:16.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.967+0000 7f8452ffd640 1 -- 172.21.15.2:0/3641608424 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557c108b2720 con 0x557c10c27200 2024-03-15T12:59:16.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.967+0000 7f8452ffd640 10 client.15882 handle_command_reply: tid=0 2024-03-15T12:59:16.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.967+0000 7f847cbde740 1 client.15882 shutdown 2024-03-15T12:59:16.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.967+0000 7f847cbde740 1 -- 172.21.15.2:0/3641608424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0eb70 msgr2=0x557c10c14430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.967+0000 7f847cbde740 1 --2- 172.21.15.2:0/3641608424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0eb70 0x557c10c14430 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f84680ed170 tx=0x7f84680ed0b0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.967+0000 7f847cbde740 1 -- 172.21.15.2:0/3641608424 shutdown_connections 2024-03-15T12:59:16.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.967+0000 7f847cbde740 1 --2- 172.21.15.2:0/3641608424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c17750 0x557c10c14990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.967+0000 7f847cbde740 1 --2- 172.21.15.2:0/3641608424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0eb70 0x557c10c14430 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.967+0000 7f847cbde740 1 --2- 172.21.15.2:0/3641608424 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557c10c27200 0x557c10c295f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.967+0000 7f847cbde740 1 --2- 172.21.15.2:0/3641608424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c0e250 0x557c10c170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.967+0000 7f8452ffd640 0 client.15882 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T12:59:16.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.967+0000 7f847cbde740 1 -- 172.21.15.2:0/3641608424 >> 172.21.15.2:0/3641608424 conn(0x557c10c08980 msgr2=0x557c10c0a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.967+0000 7f847cbde740 1 -- 172.21.15.2:0/3641608424 shutdown_connections 2024-03-15T12:59:16.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.967+0000 7f847cbde740 1 -- 172.21.15.2:0/3641608424 wait complete. 2024-03-15T12:59:16.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.967+0000 7f847cbde740 20 client.15882 trim_cache size 0 max 16384 2024-03-15T12:59:16.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.979+0000 7f847cbde740 1 Processor -- start 2024-03-15T12:59:16.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.979+0000 7f847cbde740 1 -- start start 2024-03-15T12:59:16.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.979+0000 7f847cbde740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c0e250 0x557c10c16b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.979+0000 7f847cbde740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0eb70 0x557c10c170a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.979+0000 7f847cbde740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c17750 0x557c10c22890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.979+0000 7f847cbde740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c10afb130 con 0x557c10c17750 2024-03-15T12:59:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.979+0000 7f847cbde740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c108b2720 con 0x557c10c0eb70 2024-03-15T12:59:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.979+0000 7f847cbde740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c10c29c50 con 0x557c10c0e250 2024-03-15T12:59:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.980+0000 7f8473fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c17750 0x557c10c22890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.980+0000 7f8473fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c17750 0x557c10c22890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52806/0 (socket says 172.21.15.2:52806) 2024-03-15T12:59:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.980+0000 7f8473fff640 1 -- 172.21.15.2:0/3129749776 learned_addr learned my addr 172.21.15.2:0/3129749776 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.980+0000 7f84737fe640 1 --2- 172.21.15.2:0/3129749776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c0e250 0x557c10c16b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.980+0000 7f8472ffd640 1 --2- 172.21.15.2:0/3129749776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0eb70 0x557c10c170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.980+0000 7f8473fff640 1 -- 172.21.15.2:0/3129749776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c0e250 msgr2=0x557c10c16b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.980+0000 7f8473fff640 1 --2- 172.21.15.2:0/3129749776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c0e250 0x557c10c16b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.980+0000 7f8473fff640 1 -- 172.21.15.2:0/3129749776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0eb70 msgr2=0x557c10c170a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.980+0000 7f8473fff640 1 --2- 172.21.15.2:0/3129749776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0eb70 0x557c10c170a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.980+0000 7f8473fff640 1 -- 172.21.15.2:0/3129749776 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c10b10c20 con 0x557c10c17750 2024-03-15T12:59:16.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.980+0000 7f8473fff640 1 --2- 172.21.15.2:0/3129749776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c17750 0x557c10c22890 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f846c011c40 tx=0x7f846c016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3426a32658a8956c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.980+0000 7f84597fa640 1 -- 172.21.15.2:0/3129749776 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c012a10 con 0x557c10c17750 2024-03-15T12:59:16.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.980+0000 7f84597fa640 1 -- 172.21.15.2:0/3129749776 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f846c003e40 con 0x557c10c17750 2024-03-15T12:59:16.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.980+0000 7f84597fa640 1 -- 172.21.15.2:0/3129749776 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c004420 con 0x557c10c17750 2024-03-15T12:59:16.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 -- 172.21.15.2:0/3129749776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c17750 msgr2=0x557c10c22890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 --2- 172.21.15.2:0/3129749776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c17750 0x557c10c22890 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f846c011c40 tx=0x7f846c016040 comp rx=0 tx=0).stop 2024-03-15T12:59:16.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 -- 172.21.15.2:0/3129749776 shutdown_connections 2024-03-15T12:59:16.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 --2- 172.21.15.2:0/3129749776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c17750 0x557c10c22890 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 --2- 172.21.15.2:0/3129749776 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0eb70 0x557c10c170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 --2- 172.21.15.2:0/3129749776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c0e250 0x557c10c16b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 -- 172.21.15.2:0/3129749776 >> 172.21.15.2:0/3129749776 conn(0x557c10c08980 msgr2=0x557c10c07b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 -- 172.21.15.2:0/3129749776 shutdown_connections 2024-03-15T12:59:16.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 -- 172.21.15.2:0/3129749776 wait complete. 2024-03-15T12:59:16.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 -- start start 2024-03-15T12:59:16.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 Processor -- start 2024-03-15T12:59:16.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0e250 0x557c10c12070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0eb70 0x557c10c1c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c17750 0x557c10c1c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c10c1cec0 con 0x557c10c0e250 2024-03-15T12:59:16.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c10c1d000 con 0x557c10c0eb70 2024-03-15T12:59:16.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c10c1d1c0 con 0x557c10c17750 2024-03-15T12:59:16.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.981+0000 7f847cbde740 10 client.0 _ll_register_callbacks cb 0x7f8464009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:16.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f84737fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0e250 0x557c10c12070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f84737fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0e250 0x557c10c12070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52814/0 (socket says 172.21.15.2:52814) 2024-03-15T12:59:16.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f84737fe640 1 -- 172.21.15.2:0/3902083456 learned_addr learned my addr 172.21.15.2:0/3902083456 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:16.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f8472ffd640 1 --2- 172.21.15.2:0/3902083456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0eb70 0x557c10c1c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f8473fff640 1 --2- 172.21.15.2:0/3902083456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c17750 0x557c10c1c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f84737fe640 1 -- 172.21.15.2:0/3902083456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c17750 msgr2=0x557c10c1c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f84737fe640 1 --2- 172.21.15.2:0/3902083456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c17750 0x557c10c1c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f84737fe640 1 -- 172.21.15.2:0/3902083456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0eb70 msgr2=0x557c10c1c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f84737fe640 1 --2- 172.21.15.2:0/3902083456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0eb70 0x557c10c1c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f84737fe640 1 -- 172.21.15.2:0/3902083456 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846c01b040 con 0x557c10c0e250 2024-03-15T12:59:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f84737fe640 1 --2- 172.21.15.2:0/3902083456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0e250 0x557c10c12070 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x557c10c29de0 tx=0x7f84640035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310611ef46ef7351 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f8452ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f8452ffd640 1 -- 172.21.15.2:0/3902083456 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464026070 con 0x557c10c0e250 2024-03-15T12:59:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f8452ffd640 1 -- 172.21.15.2:0/3902083456 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f8464012110 con 0x557c10c0e250 2024-03-15T12:59:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f847cbde740 1 -- 172.21.15.2:0/3902083456 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84680035f0 con 0x557c10c0e250 2024-03-15T12:59:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f8452ffd640 1 -- 172.21.15.2:0/3902083456 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8464017030 con 0x557c10c0e250 2024-03-15T12:59:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f847cbde740 10 client.15888 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.982+0000 7f847cbde740 1 -- 172.21.15.2:0/3902083456 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557c10b368c0 con 0x557c10c0e250 2024-03-15T12:59:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.983+0000 7f8452ffd640 1 -- 172.21.15.2:0/3902083456 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f84680035f0 con 0x557c10c0e250 2024-03-15T12:59:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.983+0000 7f8452ffd640 1 -- 172.21.15.2:0/3902083456 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8464029020 con 0x557c10c0e250 2024-03-15T12:59:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.983+0000 7f847cbde740 10 client.15888 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.983+0000 7f847cbde740 10 client.15888 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T12:59:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.983+0000 7f847cbde740 20 client.15888 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.983+0000 7f847cbde740 1 --2- 172.21.15.2:0/3902083456 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557c10c84130 0x557c10c86550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.983+0000 7f847cbde740 4 client.15888 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T12:59:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.983+0000 7f847cbde740 1 -- 172.21.15.2:0/3902083456 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557c10901890 con 0x557c10c84130 2024-03-15T12:59:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.983+0000 7f8472ffd640 1 --2- 172.21.15.2:0/3902083456 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557c10c84130 0x557c10c86550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.983+0000 7f8472ffd640 1 --2- 172.21.15.2:0/3902083456 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557c10c84130 0x557c10c86550 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fafcd8066fa598f7 server_cookie=cc84bd468b3d2c27 in_seq=0 out_seq=0 2024-03-15T12:59:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.983+0000 7f8452ffd640 10 client.15888 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T12:59:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.983+0000 7f8452ffd640 1 -- 172.21.15.2:0/3902083456 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557c10901890 con 0x557c10c84130 2024-03-15T12:59:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.983+0000 7f8452ffd640 10 client.15888 handle_command_reply: tid=0 2024-03-15T12:59:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.983+0000 7f847cbde740 1 client.15888 shutdown 2024-03-15T12:59:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f847cbde740 1 -- 172.21.15.2:0/3902083456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0e250 msgr2=0x557c10c12070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f847cbde740 1 --2- 172.21.15.2:0/3902083456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0e250 0x557c10c12070 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x557c10c29de0 tx=0x7f84640035f0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f847cbde740 1 -- 172.21.15.2:0/3902083456 shutdown_connections 2024-03-15T12:59:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f847cbde740 1 --2- 172.21.15.2:0/3902083456 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557c10c84130 0x557c10c86550 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f847cbde740 1 --2- 172.21.15.2:0/3902083456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c10c17750 0x557c10c1c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f847cbde740 1 --2- 172.21.15.2:0/3902083456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c10c0eb70 0x557c10c1c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f847cbde740 1 --2- 172.21.15.2:0/3902083456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c10c0e250 0x557c10c12070 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f8452ffd640 0 client.15888 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T12:59:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f847cbde740 1 -- 172.21.15.2:0/3902083456 >> 172.21.15.2:0/3902083456 conn(0x557c10c08980 msgr2=0x557c10c072f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f847cbde740 1 -- 172.21.15.2:0/3902083456 shutdown_connections 2024-03-15T12:59:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f847cbde740 1 -- 172.21.15.2:0/3902083456 wait complete. 2024-03-15T12:59:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f847cbde740 20 client.15888 trim_cache size 0 max 16384 2024-03-15T12:59:16.976 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T12:59:16.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f8479f25640 1 -- 172.21.15.2:0/2020115249 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f845c0a1cc0 msgr2=0x7f845c0a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f8479f25640 1 --2- 172.21.15.2:0/2020115249 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f845c0a1cc0 0x7f845c0a4180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f846c005fd0 tx=0x7f846c009e60 comp rx=0 tx=0).stop 2024-03-15T12:59:16.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f8479f25640 1 -- 172.21.15.2:0/2020115249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8474100dd0 msgr2=0x7f847419dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:16.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f8479f25640 1 --2- 172.21.15.2:0/2020115249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8474100dd0 0x7f847419dd10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8468009900 tx=0x7f846802ca20 comp rx=0 tx=0).stop 2024-03-15T12:59:16.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f8479f25640 1 -- 172.21.15.2:0/2020115249 shutdown_connections 2024-03-15T12:59:16.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f8479f25640 1 --2- 172.21.15.2:0/2020115249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84741064b0 0x7f8474197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f8479f25640 1 --2- 172.21.15.2:0/2020115249 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f845c0a1cc0 0x7f845c0a4180 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f8479f25640 1 --2- 172.21.15.2:0/2020115249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8474100dd0 0x7f847419dd10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f8479f25640 1 --2- 172.21.15.2:0/2020115249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8474100410 0x7f847419d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:16.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f8479f25640 1 -- 172.21.15.2:0/2020115249 >> 172.21.15.2:0/2020115249 conn(0x7f8474077550 msgr2=0x7f8474103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:16.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f8479f25640 1 -- 172.21.15.2:0/2020115249 shutdown_connections 2024-03-15T12:59:16.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:16.984+0000 7f8479f25640 1 -- 172.21.15.2:0/2020115249 wait complete. 2024-03-15T12:59:16.981 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T12:59:16.981 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T12:59:17.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.099+0000 7fc82b716640 1 -- 172.21.15.2:0/921750403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc824100410 msgr2=0x7fc824100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.099+0000 7fc82b716640 1 --2- 172.21.15.2:0/921750403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc824100410 0x7fc824100890 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc818009900 tx=0x7fc81802bc30 comp rx=0 tx=0).stop 2024-03-15T12:59:17.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.099+0000 7fc82b716640 1 -- 172.21.15.2:0/921750403 shutdown_connections 2024-03-15T12:59:17.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.099+0000 7fc82b716640 1 --2- 172.21.15.2:0/921750403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc824100dd0 0x7fc82410d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.099+0000 7fc82b716640 1 --2- 172.21.15.2:0/921750403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc824100410 0x7fc824100890 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.100+0000 7fc82b716640 1 --2- 172.21.15.2:0/921750403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8241064b0 0x7fc824106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.100+0000 7fc82b716640 1 -- 172.21.15.2:0/921750403 >> 172.21.15.2:0/921750403 conn(0x7fc824077550 msgr2=0x7fc824077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.100+0000 7fc82b716640 1 -- 172.21.15.2:0/921750403 shutdown_connections 2024-03-15T12:59:17.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.100+0000 7fc82b716640 1 -- 172.21.15.2:0/921750403 wait complete. 2024-03-15T12:59:17.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.100+0000 7fc82b716640 1 Processor -- start 2024-03-15T12:59:17.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.100+0000 7fc82b716640 1 -- start start 2024-03-15T12:59:17.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.100+0000 7fc82b716640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc824100410 0x7fc82419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.100+0000 7fc82b716640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc824100dd0 0x7fc82419dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.100+0000 7fc82b716640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8241064b0 0x7fc824197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.100+0000 7fc82b716640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824197fa0 con 0x7fc824100410 2024-03-15T12:59:17.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.100+0000 7fc82b716640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824198110 con 0x7fc824100dd0 2024-03-15T12:59:17.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.100+0000 7fc82b716640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8241982d0 con 0x7fc8241064b0 2024-03-15T12:59:17.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.100+0000 7fc8294b2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc824100410 0x7fc82419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc828cb1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc824100dd0 0x7fc82419dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc8294b2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc824100410 0x7fc82419d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52824/0 (socket says 172.21.15.2:52824) 2024-03-15T12:59:17.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc8294b2640 1 -- 172.21.15.2:0/1521674253 learned_addr learned my addr 172.21.15.2:0/1521674253 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:17.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc829cb3640 1 --2- 172.21.15.2:0/1521674253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8241064b0 0x7fc824197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc8294b2640 1 -- 172.21.15.2:0/1521674253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8241064b0 msgr2=0x7fc824197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc8294b2640 1 --2- 172.21.15.2:0/1521674253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8241064b0 0x7fc824197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc8294b2640 1 -- 172.21.15.2:0/1521674253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc824100dd0 msgr2=0x7fc82419dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc8294b2640 1 --2- 172.21.15.2:0/1521674253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc824100dd0 0x7fc82419dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc8294b2640 1 -- 172.21.15.2:0/1521674253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818009560 con 0x7fc824100410 2024-03-15T12:59:17.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc828cb1640 1 --2- 172.21.15.2:0/1521674253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc824100dd0 0x7fc82419dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:17.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc8294b2640 1 --2- 172.21.15.2:0/1521674253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc824100410 0x7fc82419d7a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00ebc0 tx=0x7fc81c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc8227fc640 1 -- 172.21.15.2:0/1521674253 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0099b0 con 0x7fc824100410 2024-03-15T12:59:17.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc82b716640 1 -- 172.21.15.2:0/1521674253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8241985b0 con 0x7fc824100410 2024-03-15T12:59:17.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc8227fc640 1 -- 172.21.15.2:0/1521674253 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc81c00eec0 con 0x7fc824100410 2024-03-15T12:59:17.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc8227fc640 1 -- 172.21.15.2:0/1521674253 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c018b10 con 0x7fc824100410 2024-03-15T12:59:17.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.101+0000 7fc82b716640 1 -- 172.21.15.2:0/1521674253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8241a4840 con 0x7fc824100410 2024-03-15T12:59:17.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.103+0000 7fc8227fc640 1 -- 172.21.15.2:0/1521674253 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7fc81c018c70 con 0x7fc824100410 2024-03-15T12:59:17.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc82e3c5740 1 Processor -- start 2024-03-15T12:59:17.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc82e3c5740 1 -- start start 2024-03-15T12:59:17.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc82e3c5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 0x557fe7674630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc8227fc640 1 --2- 172.21.15.2:0/1521674253 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc80c0a1cc0 0x7fc80c0a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc82e3c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674b70 0x557fe767d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc82e3c5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe767d750 0x557fe767fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc82e3c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe762f960 con 0x557fe7674b70 2024-03-15T12:59:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc82e3c5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe75e3310 con 0x557fe767d750 2024-03-15T12:59:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc82e3c5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe75e0840 con 0x557fe7674250 2024-03-15T12:59:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc828cb1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674b70 0x557fe767d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc8294b2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 0x557fe7674630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc829cb3640 1 --2- 172.21.15.2:0/1521674253 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc80c0a1cc0 0x7fc80c0a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc828cb1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674b70 0x557fe767d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52828/0 (socket says 172.21.15.2:52828) 2024-03-15T12:59:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc828cb1640 1 -- 172.21.15.2:0/840005610 learned_addr learned my addr 172.21.15.2:0/840005610 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc8227fc640 1 -- 172.21.15.2:0/1521674253 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc81c014070 con 0x7fc824100410 2024-03-15T12:59:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc829cb3640 1 --2- 172.21.15.2:0/840005610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe767d750 0x557fe767fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc8294b2640 1 -- 172.21.15.2:0/840005610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe767d750 msgr2=0x557fe767fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.104+0000 7fc8294b2640 1 --2- 172.21.15.2:0/840005610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe767d750 0x557fe767fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.105+0000 7fc8294b2640 1 -- 172.21.15.2:0/840005610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674b70 msgr2=0x557fe767d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.105+0000 7fc8294b2640 1 --2- 172.21.15.2:0/840005610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674b70 0x557fe767d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.105+0000 7fc8294b2640 1 -- 172.21.15.2:0/840005610 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fe759e060 con 0x557fe7674250 2024-03-15T12:59:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.105+0000 7fc828cb1640 1 --2- 172.21.15.2:0/840005610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674b70 0x557fe767d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.105+0000 7fc829cb3640 1 --2- 172.21.15.2:0/1521674253 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc80c0a1cc0 0x7fc80c0a4180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc814009fd0 tx=0x7fc814009b80 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.105+0000 7fc8294b2640 1 --2- 172.21.15.2:0/840005610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 0x557fe7674630 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0cb610 tx=0x7fc81c0147e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c749c8795ced3ae2 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.105+0000 7fc80b7fe640 1 -- 172.21.15.2:0/840005610 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c09a610 con 0x557fe7674250 2024-03-15T12:59:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.105+0000 7fc80b7fe640 1 -- 172.21.15.2:0/840005610 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc81c09a430 con 0x557fe7674250 2024-03-15T12:59:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.105+0000 7fc80b7fe640 1 -- 172.21.15.2:0/840005610 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c098dd0 con 0x557fe7674250 2024-03-15T12:59:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 -- 172.21.15.2:0/840005610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 msgr2=0x557fe7674630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/840005610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 0x557fe7674630 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0cb610 tx=0x7fc81c0147e0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 -- 172.21.15.2:0/840005610 shutdown_connections 2024-03-15T12:59:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/840005610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe767d750 0x557fe767fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/840005610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674b70 0x557fe767d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/840005610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 0x557fe7674630 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 -- 172.21.15.2:0/840005610 >> 172.21.15.2:0/840005610 conn(0x557fe766e980 msgr2=0x557fe766ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 -- 172.21.15.2:0/840005610 shutdown_connections 2024-03-15T12:59:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 -- 172.21.15.2:0/840005610 wait complete. 2024-03-15T12:59:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 -- start start 2024-03-15T12:59:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 Processor -- start 2024-03-15T12:59:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 0x557fe767d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674b70 0x557fe767a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe767d750 0x557fe767aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe75dc020 con 0x557fe7674b70 2024-03-15T12:59:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe72af430 con 0x557fe767d750 2024-03-15T12:59:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe726be00 con 0x557fe7674250 2024-03-15T12:59:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc828cb1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674b70 0x557fe767a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.106+0000 7fc82e3c5740 10 client.0 _ll_register_callbacks cb 0x557fe7583cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:17.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc8294b2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 0x557fe767d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc829cb3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe767d750 0x557fe767aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc828cb1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674b70 0x557fe767a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52842/0 (socket says 172.21.15.2:52842) 2024-03-15T12:59:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc828cb1640 1 -- 172.21.15.2:0/2267857614 learned_addr learned my addr 172.21.15.2:0/2267857614 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc8294b2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 0x557fe767d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:36838/0 (socket says 172.21.15.2:36838) 2024-03-15T12:59:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc828cb1640 1 -- 172.21.15.2:0/2267857614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 msgr2=0x557fe767d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc828cb1640 1 --2- 172.21.15.2:0/2267857614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 0x557fe767d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc828cb1640 1 -- 172.21.15.2:0/2267857614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe767d750 msgr2=0x557fe767aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc828cb1640 1 --2- 172.21.15.2:0/2267857614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe767d750 0x557fe767aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc828cb1640 1 -- 172.21.15.2:0/2267857614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8241a4840 con 0x557fe7674b70 2024-03-15T12:59:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc828cb1640 1 --2- 172.21.15.2:0/2267857614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674b70 0x557fe767a4d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc818009230 tx=0x7fc818004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5ec4bcbe130957 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc808ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc808ff9640 1 -- 172.21.15.2:0/2267857614 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818004390 con 0x557fe7674b70 2024-03-15T12:59:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc808ff9640 1 -- 172.21.15.2:0/2267857614 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc8180035f0 con 0x557fe7674b70 2024-03-15T12:59:17.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc82e3c5740 1 -- 172.21.15.2:0/2267857614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fe72db7e0 con 0x557fe7674b70 2024-03-15T12:59:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc808ff9640 1 -- 172.21.15.2:0/2267857614 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8180044f0 con 0x557fe7674b70 2024-03-15T12:59:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc82e3c5740 10 client.15900 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.107+0000 7fc82e3c5740 1 -- 172.21.15.2:0/2267857614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557fe75da6a0 con 0x557fe7674b70 2024-03-15T12:59:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.108+0000 7fc808ff9640 1 -- 172.21.15.2:0/2267857614 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x557fe72db7e0 con 0x557fe7674b70 2024-03-15T12:59:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.108+0000 7fc808ff9640 1 -- 172.21.15.2:0/2267857614 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc81803c040 con 0x557fe7674b70 2024-03-15T12:59:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.108+0000 7fc82e3c5740 10 client.15900 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.108+0000 7fc82e3c5740 10 client.15900 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T12:59:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.108+0000 7fc82e3c5740 20 client.15900 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.108+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/2267857614 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557fe768d890 0x557fe768fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.108+0000 7fc82e3c5740 4 client.15900 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.108+0000 7fc82e3c5740 1 -- 172.21.15.2:0/2267857614 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fe7318720 con 0x557fe768d890 2024-03-15T12:59:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.108+0000 7fc8294b2640 1 --2- 172.21.15.2:0/2267857614 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557fe768d890 0x557fe768fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.108+0000 7fc808ff9640 10 client.15900 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T12:59:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.108+0000 7fc8294b2640 1 --2- 172.21.15.2:0/2267857614 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557fe768d890 0x557fe768fcb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=95513b82baf1a193 server_cookie=273b52c3328060e8 in_seq=0 out_seq=0 2024-03-15T12:59:17.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.109+0000 7fc808ff9640 1 -- 172.21.15.2:0/2267857614 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557fe7318720 con 0x557fe768d890 2024-03-15T12:59:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.109+0000 7fc808ff9640 10 client.15900 handle_command_reply: tid=0 2024-03-15T12:59:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.109+0000 7fc82e3c5740 1 client.15900 shutdown 2024-03-15T12:59:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.109+0000 7fc82e3c5740 1 -- 172.21.15.2:0/2267857614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674b70 msgr2=0x557fe767a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.109+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/2267857614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674b70 0x557fe767a4d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc818009230 tx=0x7fc818004b80 comp rx=0 tx=0).stop 2024-03-15T12:59:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.109+0000 7fc82e3c5740 1 -- 172.21.15.2:0/2267857614 shutdown_connections 2024-03-15T12:59:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.109+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/2267857614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe767d750 0x557fe767aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.109+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/2267857614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674b70 0x557fe767a4d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.109+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/2267857614 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557fe768d890 0x557fe768fcb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.109+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/2267857614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 0x557fe767d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.109+0000 7fc82e3c5740 1 -- 172.21.15.2:0/2267857614 >> 172.21.15.2:0/2267857614 conn(0x557fe766e980 msgr2=0x557fe76706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.109+0000 7fc808ff9640 0 client.15900 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T12:59:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.109+0000 7fc82e3c5740 1 -- 172.21.15.2:0/2267857614 shutdown_connections 2024-03-15T12:59:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.109+0000 7fc82e3c5740 1 -- 172.21.15.2:0/2267857614 wait complete. 2024-03-15T12:59:17.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.110+0000 7fc82e3c5740 20 client.15900 trim_cache size 0 max 16384 2024-03-15T12:59:17.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc82e3c5740 1 Processor -- start 2024-03-15T12:59:17.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc82e3c5740 1 -- start start 2024-03-15T12:59:17.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc82e3c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674250 0x557fe767cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc82e3c5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674b70 0x557fe767d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc82e3c5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe767d750 0x557fe7688a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc82e3c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe7561130 con 0x557fe7674250 2024-03-15T12:59:17.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc82e3c5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe766a5c0 con 0x557fe767d750 2024-03-15T12:59:17.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc82e3c5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe767feb0 con 0x557fe7674b70 2024-03-15T12:59:17.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc8294b2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674250 0x557fe767cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc8294b2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674250 0x557fe767cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52856/0 (socket says 172.21.15.2:52856) 2024-03-15T12:59:17.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc8294b2640 1 -- 172.21.15.2:0/3356250821 learned_addr learned my addr 172.21.15.2:0/3356250821 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:17.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc829cb3640 1 --2- 172.21.15.2:0/3356250821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe767d750 0x557fe7688a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc828cb1640 1 --2- 172.21.15.2:0/3356250821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674b70 0x557fe767d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc8294b2640 1 -- 172.21.15.2:0/3356250821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674b70 msgr2=0x557fe767d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc8294b2640 1 --2- 172.21.15.2:0/3356250821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674b70 0x557fe767d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc8294b2640 1 -- 172.21.15.2:0/3356250821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe767d750 msgr2=0x557fe7688a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc8294b2640 1 --2- 172.21.15.2:0/3356250821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe767d750 0x557fe7688a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc8294b2640 1 -- 172.21.15.2:0/3356250821 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fe7576c20 con 0x557fe7674250 2024-03-15T12:59:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.122+0000 7fc8294b2640 1 --2- 172.21.15.2:0/3356250821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674250 0x557fe767cc00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc81c09a7d0 tx=0x7fc81c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955f0ce9d117de67 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.123+0000 7fc80affd640 1 -- 172.21.15.2:0/3356250821 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0a31c0 con 0x557fe7674250 2024-03-15T12:59:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.123+0000 7fc80affd640 1 -- 172.21.15.2:0/3356250821 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc81c0cb550 con 0x557fe7674250 2024-03-15T12:59:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.123+0000 7fc80affd640 1 -- 172.21.15.2:0/3356250821 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c01cd30 con 0x557fe7674250 2024-03-15T12:59:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.123+0000 7fc82e3c5740 1 -- 172.21.15.2:0/3356250821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674250 msgr2=0x557fe767cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.123+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/3356250821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674250 0x557fe767cc00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc81c09a7d0 tx=0x7fc81c01ccc0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.123+0000 7fc82e3c5740 1 -- 172.21.15.2:0/3356250821 shutdown_connections 2024-03-15T12:59:17.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.123+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/3356250821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe767d750 0x557fe7688a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.123+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/3356250821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674b70 0x557fe767d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.123+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/3356250821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe7674250 0x557fe767cc00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.123+0000 7fc82e3c5740 1 -- 172.21.15.2:0/3356250821 >> 172.21.15.2:0/3356250821 conn(0x557fe766e980 msgr2=0x557fe766db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.123+0000 7fc82e3c5740 1 -- 172.21.15.2:0/3356250821 shutdown_connections 2024-03-15T12:59:17.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.123+0000 7fc82e3c5740 1 -- 172.21.15.2:0/3356250821 wait complete. 2024-03-15T12:59:17.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.123+0000 7fc82e3c5740 1 -- start start 2024-03-15T12:59:17.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc82e3c5740 1 Processor -- start 2024-03-15T12:59:17.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc82e3c5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 0x557fe7685100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc82e3c5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe7674b70 0x557fe7677680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc82e3c5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe767d750 0x557fe7677bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc82e3c5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe7678100 con 0x557fe767d750 2024-03-15T12:59:17.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc82e3c5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe7678270 con 0x557fe7674b70 2024-03-15T12:59:17.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc82e3c5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe76783e0 con 0x557fe7674250 2024-03-15T12:59:17.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc82e3c5740 10 client.0 _ll_register_callbacks cb 0x7fc818009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:17.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc829cb3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe767d750 0x557fe7677bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc829cb3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe767d750 0x557fe7677bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52862/0 (socket says 172.21.15.2:52862) 2024-03-15T12:59:17.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc829cb3640 1 -- 172.21.15.2:0/509220347 learned_addr learned my addr 172.21.15.2:0/509220347 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:17.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc828cb1640 1 --2- 172.21.15.2:0/509220347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe7674b70 0x557fe7677680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc8294b2640 1 --2- 172.21.15.2:0/509220347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 0x557fe7685100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc829cb3640 1 -- 172.21.15.2:0/509220347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 msgr2=0x557fe7685100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc829cb3640 1 --2- 172.21.15.2:0/509220347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 0x557fe7685100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc829cb3640 1 -- 172.21.15.2:0/509220347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe7674b70 msgr2=0x557fe7677680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc829cb3640 1 --2- 172.21.15.2:0/509220347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe7674b70 0x557fe7677680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.124+0000 7fc829cb3640 1 -- 172.21.15.2:0/509220347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fe759e060 con 0x557fe767d750 2024-03-15T12:59:17.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc829cb3640 1 --2- 172.21.15.2:0/509220347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe767d750 0x557fe7677bc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc814004480 tx=0x7fc814005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ab3ec3329ea296 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc808ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:17.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc808ff9640 1 -- 172.21.15.2:0/509220347 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814016040 con 0x557fe767d750 2024-03-15T12:59:17.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc808ff9640 1 -- 172.21.15.2:0/509220347 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc81400fe20 con 0x557fe767d750 2024-03-15T12:59:17.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc82e3c5740 1 -- 172.21.15.2:0/509220347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc81c09a430 con 0x557fe767d750 2024-03-15T12:59:17.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc808ff9640 1 -- 172.21.15.2:0/509220347 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc814009780 con 0x557fe767d750 2024-03-15T12:59:17.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc82e3c5740 10 client.15906 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:17.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc82e3c5740 1 -- 172.21.15.2:0/509220347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557fe759c8c0 con 0x557fe767d750 2024-03-15T12:59:17.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc808ff9640 1 -- 172.21.15.2:0/509220347 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc81c09a430 con 0x557fe767d750 2024-03-15T12:59:17.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc808ff9640 1 -- 172.21.15.2:0/509220347 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fc814004810 con 0x557fe767d750 2024-03-15T12:59:17.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc82e3c5740 10 client.15906 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:17.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc82e3c5740 10 client.15906 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T12:59:17.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc82e3c5740 20 client.15906 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:17.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/509220347 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557fe76ea730 0x557fe76ecb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc82e3c5740 4 client.15906 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T12:59:17.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc82e3c5740 1 -- 172.21.15.2:0/509220347 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557fe7367890 con 0x557fe76ea730 2024-03-15T12:59:17.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.125+0000 7fc8294b2640 1 --2- 172.21.15.2:0/509220347 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557fe76ea730 0x557fe76ecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc808ff9640 10 client.15906 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T12:59:17.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc8294b2640 1 --2- 172.21.15.2:0/509220347 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557fe76ea730 0x557fe76ecb50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb8a231d85e9a04c server_cookie=64a5a1fed9c7be8c in_seq=0 out_seq=0 2024-03-15T12:59:17.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc808ff9640 1 -- 172.21.15.2:0/509220347 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557fe7367890 con 0x557fe76ea730 2024-03-15T12:59:17.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc808ff9640 10 client.15906 handle_command_reply: tid=0 2024-03-15T12:59:17.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc82e3c5740 1 client.15906 shutdown 2024-03-15T12:59:17.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc82e3c5740 1 -- 172.21.15.2:0/509220347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe767d750 msgr2=0x557fe7677bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/509220347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe767d750 0x557fe7677bc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc814004480 tx=0x7fc814005f90 comp rx=0 tx=0).stop 2024-03-15T12:59:17.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc82e3c5740 1 -- 172.21.15.2:0/509220347 shutdown_connections 2024-03-15T12:59:17.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/509220347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fe767d750 0x557fe7677bc0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/509220347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fe7674b70 0x557fe7677680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/509220347 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557fe76ea730 0x557fe76ecb50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc82e3c5740 1 --2- 172.21.15.2:0/509220347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fe7674250 0x557fe7685100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc808ff9640 0 client.15906 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T12:59:17.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc82e3c5740 1 -- 172.21.15.2:0/509220347 >> 172.21.15.2:0/509220347 conn(0x557fe766e980 msgr2=0x557fe7675090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc82e3c5740 1 -- 172.21.15.2:0/509220347 shutdown_connections 2024-03-15T12:59:17.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc82e3c5740 1 -- 172.21.15.2:0/509220347 wait complete. 2024-03-15T12:59:17.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.126+0000 7fc82e3c5740 20 client.15906 trim_cache size 0 max 16384 2024-03-15T12:59:17.119 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T12:59:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.127+0000 7fc82b716640 1 -- 172.21.15.2:0/1521674253 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc80c0a1cc0 msgr2=0x7fc80c0a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.127+0000 7fc82b716640 1 --2- 172.21.15.2:0/1521674253 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc80c0a1cc0 0x7fc80c0a4180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc814009fd0 tx=0x7fc814009b80 comp rx=0 tx=0).stop 2024-03-15T12:59:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.127+0000 7fc82b716640 1 -- 172.21.15.2:0/1521674253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc824100410 msgr2=0x7fc82419d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.127+0000 7fc82b716640 1 --2- 172.21.15.2:0/1521674253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc824100410 0x7fc82419d7a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00ebc0 tx=0x7fc81c00c6a0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.127+0000 7fc82b716640 1 -- 172.21.15.2:0/1521674253 shutdown_connections 2024-03-15T12:59:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.127+0000 7fc82b716640 1 --2- 172.21.15.2:0/1521674253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8241064b0 0x7fc824197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.127+0000 7fc82b716640 1 --2- 172.21.15.2:0/1521674253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc824100dd0 0x7fc82419dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.127+0000 7fc82b716640 1 --2- 172.21.15.2:0/1521674253 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc80c0a1cc0 0x7fc80c0a4180 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.127+0000 7fc82b716640 1 --2- 172.21.15.2:0/1521674253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc824100410 0x7fc82419d7a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.127+0000 7fc82b716640 1 -- 172.21.15.2:0/1521674253 >> 172.21.15.2:0/1521674253 conn(0x7fc824077550 msgr2=0x7fc824103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.127+0000 7fc82b716640 1 -- 172.21.15.2:0/1521674253 shutdown_connections 2024-03-15T12:59:17.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.127+0000 7fc82b716640 1 -- 172.21.15.2:0/1521674253 wait complete. 2024-03-15T12:59:17.123 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T12:59:17.123 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T12:59:17.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.244+0000 7fa0924ae640 1 -- 172.21.15.2:0/2135837331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa08c101e60 msgr2=0x7fa08c10e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.244+0000 7fa0924ae640 1 --2- 172.21.15.2:0/2135837331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa08c101e60 0x7fa08c10e320 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa084009ca0 tx=0x7fa08402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.244+0000 7fa0924ae640 1 -- 172.21.15.2:0/2135837331 shutdown_connections 2024-03-15T12:59:17.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.244+0000 7fa0924ae640 1 --2- 172.21.15.2:0/2135837331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa08c101e60 0x7fa08c10e320 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.244+0000 7fa0924ae640 1 --2- 172.21.15.2:0/2135837331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa08c1014a0 0x7fa08c101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.244+0000 7fa0924ae640 1 --2- 172.21.15.2:0/2135837331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa08c107560 0x7fa08c107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.245+0000 7fa0924ae640 1 -- 172.21.15.2:0/2135837331 >> 172.21.15.2:0/2135837331 conn(0x7fa08c0fcf90 msgr2=0x7fa08c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.245+0000 7fa0924ae640 1 -- 172.21.15.2:0/2135837331 shutdown_connections 2024-03-15T12:59:17.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.245+0000 7fa0924ae640 1 -- 172.21.15.2:0/2135837331 wait complete. 2024-03-15T12:59:17.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.245+0000 7fa0924ae640 1 Processor -- start 2024-03-15T12:59:17.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.245+0000 7fa0924ae640 1 -- start start 2024-03-15T12:59:17.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.245+0000 7fa0924ae640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa08c1014a0 0x7fa08c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.245+0000 7fa0924ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa08c101e60 0x7fa08c19dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.245+0000 7fa0924ae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa08c107560 0x7fa08c197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.245+0000 7fa0924ae640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c197ef0 con 0x7fa08c107560 2024-03-15T12:59:17.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.245+0000 7fa0924ae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c198060 con 0x7fa08c101e60 2024-03-15T12:59:17.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.245+0000 7fa0924ae640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c198220 con 0x7fa08c1014a0 2024-03-15T12:59:17.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa090a4b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa08c107560 0x7fa08c197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa08bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa08c1014a0 0x7fa08c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa08b7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa08c101e60 0x7fa08c19dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa090a4b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa08c107560 0x7fa08c197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52874/0 (socket says 172.21.15.2:52874) 2024-03-15T12:59:17.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa08bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa08c1014a0 0x7fa08c19d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:36864/0 (socket says 172.21.15.2:36864) 2024-03-15T12:59:17.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa08b7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa08c101e60 0x7fa08c19dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:51134/0 (socket says 172.21.15.2:51134) 2024-03-15T12:59:17.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa090a4b640 1 -- 172.21.15.2:0/670777253 learned_addr learned my addr 172.21.15.2:0/670777253 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:17.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa090a4b640 1 -- 172.21.15.2:0/670777253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa08c1014a0 msgr2=0x7fa08c19d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa090a4b640 1 --2- 172.21.15.2:0/670777253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa08c1014a0 0x7fa08c19d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa090a4b640 1 -- 172.21.15.2:0/670777253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa08c101e60 msgr2=0x7fa08c19dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa090a4b640 1 --2- 172.21.15.2:0/670777253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa08c101e60 0x7fa08c19dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa090a4b640 1 -- 172.21.15.2:0/670777253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa084009900 con 0x7fa08c107560 2024-03-15T12:59:17.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa090a4b640 1 --2- 172.21.15.2:0/670777253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa08c107560 0x7fa08c197850 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa084005cc0 tx=0x7fa0840325f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa0897fa640 1 -- 172.21.15.2:0/670777253 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0840328b0 con 0x7fa08c107560 2024-03-15T12:59:17.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa0897fa640 1 -- 172.21.15.2:0/670777253 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa084003e40 con 0x7fa08c107560 2024-03-15T12:59:17.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa0924ae640 1 -- 172.21.15.2:0/670777253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08c1984a0 con 0x7fa08c107560 2024-03-15T12:59:17.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa0897fa640 1 -- 172.21.15.2:0/670777253 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084040450 con 0x7fa08c107560 2024-03-15T12:59:17.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.246+0000 7fa0924ae640 1 -- 172.21.15.2:0/670777253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08c1a4710 con 0x7fa08c107560 2024-03-15T12:59:17.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.248+0000 7fa0897fa640 1 -- 172.21.15.2:0/670777253 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7fa08400a040 con 0x7fa08c107560 2024-03-15T12:59:17.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa09516c740 1 Processor -- start 2024-03-15T12:59:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa0897fa640 1 --2- 172.21.15.2:0/670777253 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa0740a1cc0 0x7fa0740a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa09516c740 1 -- start start 2024-03-15T12:59:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa0897fa640 1 -- 172.21.15.2:0/670777253 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa0840f0b20 con 0x7fa08c107560 2024-03-15T12:59:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa08bfff640 1 --2- 172.21.15.2:0/670777253 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa0740a1cc0 0x7fa0740a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa09516c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fca1250 0x55955fca1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa09516c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fcaa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa09516c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fcaa750 0x55955fcacb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa09516c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955fc5c960 con 0x55955fca1b70 2024-03-15T12:59:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa09516c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955fc10310 con 0x55955fca1250 2024-03-15T12:59:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa09516c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955fc0d840 con 0x55955fcaa750 2024-03-15T12:59:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa08b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fcaa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa08b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fcaa210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52884/0 (socket says 172.21.15.2:52884) 2024-03-15T12:59:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa08b7fe640 1 -- 172.21.15.2:0/2779090557 learned_addr learned my addr 172.21.15.2:0/2779090557 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa090a4b640 1 --2- 172.21.15.2:0/2779090557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fcaa750 0x55955fcacb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa08bfff640 1 --2- 172.21.15.2:0/2779090557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fca1250 0x55955fca1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa08b7fe640 1 -- 172.21.15.2:0/2779090557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fcaa750 msgr2=0x55955fcacb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa08b7fe640 1 --2- 172.21.15.2:0/2779090557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fcaa750 0x55955fcacb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa08b7fe640 1 -- 172.21.15.2:0/2779090557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fca1250 msgr2=0x55955fca1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa08b7fe640 1 --2- 172.21.15.2:0/2779090557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fca1250 0x55955fca1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa08b7fe640 1 -- 172.21.15.2:0/2779090557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55955fbcb060 con 0x55955fca1b70 2024-03-15T12:59:17.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.249+0000 7fa08bfff640 1 --2- 172.21.15.2:0/670777253 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa0740a1cc0 0x7fa0740a4180 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009be0 tx=0x7fa07c011440 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.250+0000 7fa08b7fe640 1 --2- 172.21.15.2:0/2779090557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fcaa210 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa080009c60 tx=0x7fa080011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4094095ceb965313 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.250+0000 7fa0727fc640 1 -- 172.21.15.2:0/2779090557 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080004230 con 0x55955fca1b70 2024-03-15T12:59:17.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.250+0000 7fa0727fc640 1 -- 172.21.15.2:0/2779090557 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa080004390 con 0x55955fca1b70 2024-03-15T12:59:17.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.250+0000 7fa0727fc640 1 -- 172.21.15.2:0/2779090557 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080019710 con 0x55955fca1b70 2024-03-15T12:59:17.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.250+0000 7fa09516c740 1 -- 172.21.15.2:0/2779090557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 msgr2=0x55955fcaa210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.250+0000 7fa09516c740 1 --2- 172.21.15.2:0/2779090557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fcaa210 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa080009c60 tx=0x7fa080011810 comp rx=0 tx=0).stop 2024-03-15T12:59:17.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.250+0000 7fa09516c740 1 -- 172.21.15.2:0/2779090557 shutdown_connections 2024-03-15T12:59:17.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.250+0000 7fa09516c740 1 --2- 172.21.15.2:0/2779090557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fcaa750 0x55955fcacb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.250+0000 7fa09516c740 1 --2- 172.21.15.2:0/2779090557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fcaa210 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.250+0000 7fa09516c740 1 --2- 172.21.15.2:0/2779090557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fca1250 0x55955fca1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.250+0000 7fa09516c740 1 -- 172.21.15.2:0/2779090557 >> 172.21.15.2:0/2779090557 conn(0x55955fc9b980 msgr2=0x55955fc9bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.251+0000 7fa09516c740 1 -- 172.21.15.2:0/2779090557 shutdown_connections 2024-03-15T12:59:17.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.251+0000 7fa09516c740 1 -- 172.21.15.2:0/2779090557 wait complete. 2024-03-15T12:59:17.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.251+0000 7fa09516c740 1 -- start start 2024-03-15T12:59:17.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.251+0000 7fa09516c740 1 Processor -- start 2024-03-15T12:59:17.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.251+0000 7fa09516c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fca1250 0x55955fcaa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.251+0000 7fa09516c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fca74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.251+0000 7fa09516c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fcaa750 0x55955fca7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.251+0000 7fa09516c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955fc09020 con 0x55955fca1b70 2024-03-15T12:59:17.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.251+0000 7fa09516c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955f8dc430 con 0x55955fcaa750 2024-03-15T12:59:17.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.251+0000 7fa09516c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955f898e00 con 0x55955fca1250 2024-03-15T12:59:17.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.251+0000 7fa09516c740 10 client.0 _ll_register_callbacks cb 0x55955fbb0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:17.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.251+0000 7fa08b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fca74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.251+0000 7fa08b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fca74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52896/0 (socket says 172.21.15.2:52896) 2024-03-15T12:59:17.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa08b7fe640 1 -- 172.21.15.2:0/2475371853 learned_addr learned my addr 172.21.15.2:0/2475371853 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:17.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa08bfff640 1 --2- 172.21.15.2:0/2475371853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fca1250 0x55955fcaa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa090a4b640 1 --2- 172.21.15.2:0/2475371853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fcaa750 0x55955fca7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa08b7fe640 1 -- 172.21.15.2:0/2475371853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fca1250 msgr2=0x55955fcaa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa08b7fe640 1 --2- 172.21.15.2:0/2475371853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fca1250 0x55955fcaa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa08b7fe640 1 -- 172.21.15.2:0/2475371853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fcaa750 msgr2=0x55955fca7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa08b7fe640 1 --2- 172.21.15.2:0/2475371853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fcaa750 0x55955fca7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa08b7fe640 1 -- 172.21.15.2:0/2475371853 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa080009890 con 0x55955fca1b70 2024-03-15T12:59:17.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa08b7fe640 1 --2- 172.21.15.2:0/2475371853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fca74c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa0800096e0 tx=0x7fa0800045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffca22d0606994e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa06b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:17.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa06b7fe640 1 -- 172.21.15.2:0/2475371853 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080004a40 con 0x55955fca1b70 2024-03-15T12:59:17.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa06b7fe640 1 -- 172.21.15.2:0/2475371853 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa080004ba0 con 0x55955fca1b70 2024-03-15T12:59:17.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa09516c740 1 -- 172.21.15.2:0/2475371853 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55955f9087e0 con 0x55955fca1b70 2024-03-15T12:59:17.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa06b7fe640 1 -- 172.21.15.2:0/2475371853 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa080004d60 con 0x55955fca1b70 2024-03-15T12:59:17.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa09516c740 10 client.15918 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:17.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa09516c740 1 -- 172.21.15.2:0/2475371853 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55955fc076a0 con 0x55955fca1b70 2024-03-15T12:59:17.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.252+0000 7fa06b7fe640 1 -- 172.21.15.2:0/2475371853 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55955f9087e0 con 0x55955fca1b70 2024-03-15T12:59:17.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.253+0000 7fa06b7fe640 1 -- 172.21.15.2:0/2475371853 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa08001a4d0 con 0x55955fca1b70 2024-03-15T12:59:17.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.253+0000 7fa09516c740 10 client.15918 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:17.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.253+0000 7fa09516c740 10 client.15918 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T12:59:17.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.253+0000 7fa09516c740 20 client.15918 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:17.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.253+0000 7fa09516c740 1 --2- 172.21.15.2:0/2475371853 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55955fcba950 0x55955fcbcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.253+0000 7fa09516c740 4 client.15918 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:17.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.253+0000 7fa09516c740 1 -- 172.21.15.2:0/2475371853 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55955f945720 con 0x55955fcba950 2024-03-15T12:59:17.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.253+0000 7fa08bfff640 1 --2- 172.21.15.2:0/2475371853 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55955fcba950 0x55955fcbcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.253+0000 7fa06b7fe640 10 client.15918 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T12:59:17.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.253+0000 7fa08bfff640 1 --2- 172.21.15.2:0/2475371853 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55955fcba950 0x55955fcbcd70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c284f810457c5e3 server_cookie=43a84e32aed278 in_seq=0 out_seq=0 2024-03-15T12:59:17.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.254+0000 7fa06b7fe640 1 -- 172.21.15.2:0/2475371853 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55955f945720 con 0x55955fcba950 2024-03-15T12:59:17.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.254+0000 7fa06b7fe640 10 client.15918 handle_command_reply: tid=0 2024-03-15T12:59:17.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.254+0000 7fa09516c740 1 client.15918 shutdown 2024-03-15T12:59:17.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.254+0000 7fa09516c740 1 -- 172.21.15.2:0/2475371853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 msgr2=0x55955fca74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.254+0000 7fa09516c740 1 --2- 172.21.15.2:0/2475371853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fca74c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa0800096e0 tx=0x7fa0800045a0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.254+0000 7fa09516c740 1 -- 172.21.15.2:0/2475371853 shutdown_connections 2024-03-15T12:59:17.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.254+0000 7fa09516c740 1 --2- 172.21.15.2:0/2475371853 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55955fcba950 0x55955fcbcd70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.254+0000 7fa09516c740 1 --2- 172.21.15.2:0/2475371853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fcaa750 0x55955fca7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.254+0000 7fa09516c740 1 --2- 172.21.15.2:0/2475371853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fca74c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.254+0000 7fa09516c740 1 --2- 172.21.15.2:0/2475371853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fca1250 0x55955fcaa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.254+0000 7fa06b7fe640 0 client.15918 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T12:59:17.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.254+0000 7fa09516c740 1 -- 172.21.15.2:0/2475371853 >> 172.21.15.2:0/2475371853 conn(0x55955fc9b980 msgr2=0x55955fc9d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.255+0000 7fa09516c740 1 -- 172.21.15.2:0/2475371853 shutdown_connections 2024-03-15T12:59:17.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.255+0000 7fa09516c740 1 -- 172.21.15.2:0/2475371853 wait complete. 2024-03-15T12:59:17.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.255+0000 7fa09516c740 20 client.15918 trim_cache size 0 max 16384 2024-03-15T12:59:17.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa09516c740 1 Processor -- start 2024-03-15T12:59:17.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa09516c740 1 -- start start 2024-03-15T12:59:17.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa09516c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fca1250 0x55955fca9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa09516c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fcaa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa09516c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fcaa750 0x55955fcb5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa09516c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955fcb6db0 con 0x55955fca1b70 2024-03-15T12:59:17.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa09516c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955fb8e130 con 0x55955fca1250 2024-03-15T12:59:17.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa09516c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955fc972e0 con 0x55955fcaa750 2024-03-15T12:59:17.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa08b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fcaa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa08b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fcaa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52898/0 (socket says 172.21.15.2:52898) 2024-03-15T12:59:17.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa08b7fe640 1 -- 172.21.15.2:0/58778322 learned_addr learned my addr 172.21.15.2:0/58778322 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:17.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa08bfff640 1 --2- 172.21.15.2:0/58778322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fca1250 0x55955fca9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa090a4b640 1 --2- 172.21.15.2:0/58778322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fcaa750 0x55955fcb5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa08b7fe640 1 -- 172.21.15.2:0/58778322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fcaa750 msgr2=0x55955fcb5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa08b7fe640 1 --2- 172.21.15.2:0/58778322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fcaa750 0x55955fcb5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa08b7fe640 1 -- 172.21.15.2:0/58778322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fca1250 msgr2=0x55955fca9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa08b7fe640 1 --2- 172.21.15.2:0/58778322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fca1250 0x55955fca9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.267+0000 7fa08b7fe640 1 -- 172.21.15.2:0/58778322 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55955fba3c20 con 0x55955fca1b70 2024-03-15T12:59:17.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.268+0000 7fa08b7fe640 1 --2- 172.21.15.2:0/58778322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fcaa130 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa0800057c0 tx=0x7fa080009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55736c94c3ef0052 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.268+0000 7fa071ffb640 1 -- 172.21.15.2:0/58778322 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080005350 con 0x55955fca1b70 2024-03-15T12:59:17.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.268+0000 7fa071ffb640 1 -- 172.21.15.2:0/58778322 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa08001a440 con 0x55955fca1b70 2024-03-15T12:59:17.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.268+0000 7fa071ffb640 1 -- 172.21.15.2:0/58778322 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0800193e0 con 0x55955fca1b70 2024-03-15T12:59:17.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.268+0000 7fa09516c740 1 -- 172.21.15.2:0/58778322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 msgr2=0x55955fcaa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.268+0000 7fa09516c740 1 --2- 172.21.15.2:0/58778322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fcaa130 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa0800057c0 tx=0x7fa080009ab0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.268+0000 7fa09516c740 1 -- 172.21.15.2:0/58778322 shutdown_connections 2024-03-15T12:59:17.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.268+0000 7fa09516c740 1 --2- 172.21.15.2:0/58778322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fcaa750 0x55955fcb5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.268+0000 7fa09516c740 1 --2- 172.21.15.2:0/58778322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1b70 0x55955fcaa130 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.268+0000 7fa09516c740 1 --2- 172.21.15.2:0/58778322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fca1250 0x55955fca9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.268+0000 7fa09516c740 1 -- 172.21.15.2:0/58778322 >> 172.21.15.2:0/58778322 conn(0x55955fc9b980 msgr2=0x55955fc9ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.268+0000 7fa09516c740 1 -- 172.21.15.2:0/58778322 shutdown_connections 2024-03-15T12:59:17.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.268+0000 7fa09516c740 1 -- 172.21.15.2:0/58778322 wait complete. 2024-03-15T12:59:17.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa09516c740 1 -- start start 2024-03-15T12:59:17.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa09516c740 1 Processor -- start 2024-03-15T12:59:17.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa09516c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1250 0x55955fca3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa09516c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fca1b70 0x55955fca4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa09516c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fcaa750 0x55955fca4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa09516c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955fca5530 con 0x55955fca1250 2024-03-15T12:59:17.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa09516c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955fd15200 con 0x55955fcaa750 2024-03-15T12:59:17.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa09516c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955fd15340 con 0x55955fca1b70 2024-03-15T12:59:17.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa09516c740 10 client.0 _ll_register_callbacks cb 0x7fa084005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:17.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa08bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1250 0x55955fca3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa08bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1250 0x55955fca3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52908/0 (socket says 172.21.15.2:52908) 2024-03-15T12:59:17.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa08bfff640 1 -- 172.21.15.2:0/3807320159 learned_addr learned my addr 172.21.15.2:0/3807320159 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:17.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa08b7fe640 1 --2- 172.21.15.2:0/3807320159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fca1b70 0x55955fca4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa090a4b640 1 --2- 172.21.15.2:0/3807320159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fcaa750 0x55955fca4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa08bfff640 1 -- 172.21.15.2:0/3807320159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fca1b70 msgr2=0x55955fca4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa08bfff640 1 --2- 172.21.15.2:0/3807320159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fca1b70 0x55955fca4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa08bfff640 1 -- 172.21.15.2:0/3807320159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fcaa750 msgr2=0x55955fca4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa08bfff640 1 --2- 172.21.15.2:0/3807320159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fcaa750 0x55955fca4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.269+0000 7fa08bfff640 1 -- 172.21.15.2:0/3807320159 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55955fc076a0 con 0x55955fca1250 2024-03-15T12:59:17.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa08bfff640 1 --2- 172.21.15.2:0/3807320159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1250 0x55955fca3b00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55955fcbd5b0 tx=0x7fa07c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7893e40788718498 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa06b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:17.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa06b7fe640 1 -- 172.21.15.2:0/3807320159 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c023040 con 0x55955fca1250 2024-03-15T12:59:17.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa09516c740 1 -- 172.21.15.2:0/3807320159 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa080004390 con 0x55955fca1250 2024-03-15T12:59:17.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa06b7fe640 1 -- 172.21.15.2:0/3807320159 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa07c005020 con 0x55955fca1250 2024-03-15T12:59:17.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa06b7fe640 1 -- 172.21.15.2:0/3807320159 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa07c0051e0 con 0x55955fca1250 2024-03-15T12:59:17.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa09516c740 10 client.15924 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:17.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa09516c740 1 -- 172.21.15.2:0/3807320159 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55955fbc98c0 con 0x55955fca1250 2024-03-15T12:59:17.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa06b7fe640 1 -- 172.21.15.2:0/3807320159 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa080004390 con 0x55955fca1250 2024-03-15T12:59:17.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa06b7fe640 1 -- 172.21.15.2:0/3807320159 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa07c029740 con 0x55955fca1250 2024-03-15T12:59:17.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa09516c740 10 client.15924 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:17.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa09516c740 10 client.15924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T12:59:17.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa09516c740 20 client.15924 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:17.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa09516c740 1 --2- 172.21.15.2:0/3807320159 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55955fd16ba0 0x55955fd18fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa09516c740 4 client.15924 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T12:59:17.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.270+0000 7fa09516c740 1 -- 172.21.15.2:0/3807320159 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55955f994890 con 0x55955fd16ba0 2024-03-15T12:59:17.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa08b7fe640 1 --2- 172.21.15.2:0/3807320159 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55955fd16ba0 0x55955fd18fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa08b7fe640 1 --2- 172.21.15.2:0/3807320159 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55955fd16ba0 0x55955fd18fc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e6c80f2715a1084 server_cookie=eebf07d4443d864f in_seq=0 out_seq=0 2024-03-15T12:59:17.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa06b7fe640 10 client.15924 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T12:59:17.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa06b7fe640 1 -- 172.21.15.2:0/3807320159 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55955f994890 con 0x55955fd16ba0 2024-03-15T12:59:17.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa06b7fe640 10 client.15924 handle_command_reply: tid=0 2024-03-15T12:59:17.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa09516c740 1 client.15924 shutdown 2024-03-15T12:59:17.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa09516c740 1 -- 172.21.15.2:0/3807320159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1250 msgr2=0x55955fca3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa09516c740 1 --2- 172.21.15.2:0/3807320159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1250 0x55955fca3b00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55955fcbd5b0 tx=0x7fa07c004730 comp rx=0 tx=0).stop 2024-03-15T12:59:17.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa09516c740 1 -- 172.21.15.2:0/3807320159 shutdown_connections 2024-03-15T12:59:17.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa09516c740 1 --2- 172.21.15.2:0/3807320159 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55955fd16ba0 0x55955fd18fc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa09516c740 1 --2- 172.21.15.2:0/3807320159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55955fcaa750 0x55955fca4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa09516c740 1 --2- 172.21.15.2:0/3807320159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55955fca1b70 0x55955fca4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa06b7fe640 0 client.15924 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T12:59:17.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa09516c740 1 --2- 172.21.15.2:0/3807320159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955fca1250 0x55955fca3b00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa09516c740 1 -- 172.21.15.2:0/3807320159 >> 172.21.15.2:0/3807320159 conn(0x55955fc9b980 msgr2=0x55955fca2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa09516c740 1 -- 172.21.15.2:0/3807320159 shutdown_connections 2024-03-15T12:59:17.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa09516c740 1 -- 172.21.15.2:0/3807320159 wait complete. 2024-03-15T12:59:17.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.271+0000 7fa09516c740 20 client.15924 trim_cache size 0 max 16384 2024-03-15T12:59:17.264 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T12:59:17.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.272+0000 7fa0924ae640 1 -- 172.21.15.2:0/670777253 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa0740a1cc0 msgr2=0x7fa0740a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.272+0000 7fa0924ae640 1 --2- 172.21.15.2:0/670777253 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa0740a1cc0 0x7fa0740a4180 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009be0 tx=0x7fa07c011440 comp rx=0 tx=0).stop 2024-03-15T12:59:17.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.272+0000 7fa0924ae640 1 -- 172.21.15.2:0/670777253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa08c107560 msgr2=0x7fa08c197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.272+0000 7fa0924ae640 1 --2- 172.21.15.2:0/670777253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa08c107560 0x7fa08c197850 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa084005cc0 tx=0x7fa0840325f0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.272+0000 7fa0924ae640 1 -- 172.21.15.2:0/670777253 shutdown_connections 2024-03-15T12:59:17.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.272+0000 7fa0924ae640 1 --2- 172.21.15.2:0/670777253 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa0740a1cc0 0x7fa0740a4180 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.272+0000 7fa0924ae640 1 --2- 172.21.15.2:0/670777253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa08c107560 0x7fa08c197850 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.272+0000 7fa0924ae640 1 --2- 172.21.15.2:0/670777253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa08c101e60 0x7fa08c19dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.272+0000 7fa0924ae640 1 --2- 172.21.15.2:0/670777253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa08c1014a0 0x7fa08c19d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.272+0000 7fa0924ae640 1 -- 172.21.15.2:0/670777253 >> 172.21.15.2:0/670777253 conn(0x7fa08c0fcf90 msgr2=0x7fa08c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.272+0000 7fa0924ae640 1 -- 172.21.15.2:0/670777253 shutdown_connections 2024-03-15T12:59:17.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.272+0000 7fa0924ae640 1 -- 172.21.15.2:0/670777253 wait complete. 2024-03-15T12:59:17.268 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T12:59:17.268 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T12:59:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.387+0000 7fac86581640 1 -- 172.21.15.2:0/84594441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac801014e0 msgr2=0x7fac8010d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.387+0000 7fac86581640 1 --2- 172.21.15.2:0/84594441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac801014e0 0x7fac8010d210 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fac78009ca0 tx=0x7fac7802c140 comp rx=0 tx=0).stop 2024-03-15T12:59:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.387+0000 7fac86581640 1 -- 172.21.15.2:0/84594441 shutdown_connections 2024-03-15T12:59:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.387+0000 7fac86581640 1 --2- 172.21.15.2:0/84594441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac801014e0 0x7fac8010d210 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.387+0000 7fac86581640 1 --2- 172.21.15.2:0/84594441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac80100b40 0x7fac80100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.387+0000 7fac86581640 1 --2- 172.21.15.2:0/84594441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac801053a0 0x7fac80105780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.387+0000 7fac86581640 1 -- 172.21.15.2:0/84594441 >> 172.21.15.2:0/84594441 conn(0x7fac800769b0 msgr2=0x7fac800fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.387+0000 7fac86581640 1 -- 172.21.15.2:0/84594441 shutdown_connections 2024-03-15T12:59:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.387+0000 7fac86581640 1 -- 172.21.15.2:0/84594441 wait complete. 2024-03-15T12:59:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.387+0000 7fac86581640 1 Processor -- start 2024-03-15T12:59:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.387+0000 7fac86581640 1 -- start start 2024-03-15T12:59:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac86581640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac80100b40 0x7fac8019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac86581640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac801014e0 0x7fac8019dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac86581640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac801053a0 0x7fac80197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac86581640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac80197fb0 con 0x7fac80100b40 2024-03-15T12:59:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac86581640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac80198120 con 0x7fac801053a0 2024-03-15T12:59:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac86581640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac801982e0 con 0x7fac801014e0 2024-03-15T12:59:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac80100b40 0x7fac8019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac80100b40 0x7fac8019d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52912/0 (socket says 172.21.15.2:52912) 2024-03-15T12:59:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac7ffff640 1 -- 172.21.15.2:0/2297135603 learned_addr learned my addr 172.21.15.2:0/2297135603 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac7f7fe640 1 --2- 172.21.15.2:0/2297135603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac801014e0 0x7fac8019dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac84b1e640 1 --2- 172.21.15.2:0/2297135603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac801053a0 0x7fac80197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac7ffff640 1 -- 172.21.15.2:0/2297135603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac801014e0 msgr2=0x7fac8019dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac7ffff640 1 --2- 172.21.15.2:0/2297135603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac801014e0 0x7fac8019dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac7ffff640 1 -- 172.21.15.2:0/2297135603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac801053a0 msgr2=0x7fac80197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac7ffff640 1 --2- 172.21.15.2:0/2297135603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac801053a0 0x7fac80197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac7ffff640 1 -- 172.21.15.2:0/2297135603 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac78009900 con 0x7fac80100b40 2024-03-15T12:59:17.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.388+0000 7fac7ffff640 1 --2- 172.21.15.2:0/2297135603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac80100b40 0x7fac8019d7b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fac7000e9f0 tx=0x7fac7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.389+0000 7fac7d7fa640 1 -- 172.21.15.2:0/2297135603 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac70009700 con 0x7fac80100b40 2024-03-15T12:59:17.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.389+0000 7fac7d7fa640 1 -- 172.21.15.2:0/2297135603 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fac7000cd80 con 0x7fac80100b40 2024-03-15T12:59:17.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.389+0000 7fac86581640 1 -- 172.21.15.2:0/2297135603 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac801985c0 con 0x7fac80100b40 2024-03-15T12:59:17.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.389+0000 7fac7d7fa640 1 -- 172.21.15.2:0/2297135603 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac70018920 con 0x7fac80100b40 2024-03-15T12:59:17.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.389+0000 7fac86581640 1 -- 172.21.15.2:0/2297135603 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac801a4850 con 0x7fac80100b40 2024-03-15T12:59:17.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.391+0000 7fac7d7fa640 1 -- 172.21.15.2:0/2297135603 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7fac70018a80 con 0x7fac80100b40 2024-03-15T12:59:17.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.391+0000 7fac88dfd740 1 Processor -- start 2024-03-15T12:59:17.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.391+0000 7fac88dfd740 1 -- start start 2024-03-15T12:59:17.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.391+0000 7fac88dfd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae4a250 0x5581dae4a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.391+0000 7fac88dfd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4ab70 0x5581dae53210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.391+0000 7fac88dfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae53750 0x5581dae55b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.391+0000 7fac7d7fa640 1 --2- 172.21.15.2:0/2297135603 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fac680a1cc0 0x7fac680a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.391+0000 7fac88dfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dae05960 con 0x5581dae53750 2024-03-15T12:59:17.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.391+0000 7fac88dfd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dadb9310 con 0x5581dae4ab70 2024-03-15T12:59:17.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.391+0000 7fac88dfd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dadb6840 con 0x5581dae4a250 2024-03-15T12:59:17.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac7f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae53750 0x5581dae55b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac84b1e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae4a250 0x5581dae4a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4ab70 0x5581dae53210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac7f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae53750 0x5581dae55b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52922/0 (socket says 172.21.15.2:52922) 2024-03-15T12:59:17.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac84b1e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae4a250 0x5581dae4a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:36918/0 (socket says 172.21.15.2:36918) 2024-03-15T12:59:17.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac7f7fe640 1 -- 172.21.15.2:0/3110565814 learned_addr learned my addr 172.21.15.2:0/3110565814 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:17.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac84b1e640 1 --2- 172.21.15.2:0/2297135603 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fac680a1cc0 0x7fac680a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac7d7fa640 1 -- 172.21.15.2:0/2297135603 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fac70014070 con 0x7fac80100b40 2024-03-15T12:59:17.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac7f7fe640 1 -- 172.21.15.2:0/3110565814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae4a250 msgr2=0x5581dae4a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac7f7fe640 1 --2- 172.21.15.2:0/3110565814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae4a250 0x5581dae4a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac7f7fe640 1 -- 172.21.15.2:0/3110565814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4ab70 msgr2=0x5581dae53210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac7f7fe640 1 --2- 172.21.15.2:0/3110565814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4ab70 0x5581dae53210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac7f7fe640 1 -- 172.21.15.2:0/3110565814 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581dad74060 con 0x5581dae53750 2024-03-15T12:59:17.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac7ffff640 1 --2- 172.21.15.2:0/3110565814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4ab70 0x5581dae53210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:17.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac7f7fe640 1 --2- 172.21.15.2:0/3110565814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae53750 0x5581dae55b40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fac74009c60 tx=0x7fac74011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1fdd0643540e2f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac84b1e640 1 --2- 172.21.15.2:0/2297135603 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fac680a1cc0 0x7fac680a4180 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fac78004af0 tx=0x7fac78005c50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac667fc640 1 -- 172.21.15.2:0/3110565814 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac74004230 con 0x5581dae53750 2024-03-15T12:59:17.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac667fc640 1 -- 172.21.15.2:0/3110565814 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fac74004390 con 0x5581dae53750 2024-03-15T12:59:17.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.392+0000 7fac667fc640 1 -- 172.21.15.2:0/3110565814 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac740196d0 con 0x5581dae53750 2024-03-15T12:59:17.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.393+0000 7fac88dfd740 1 -- 172.21.15.2:0/3110565814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae53750 msgr2=0x5581dae55b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.393+0000 7fac88dfd740 1 --2- 172.21.15.2:0/3110565814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae53750 0x5581dae55b40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fac74009c60 tx=0x7fac74011810 comp rx=0 tx=0).stop 2024-03-15T12:59:17.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.393+0000 7fac88dfd740 1 -- 172.21.15.2:0/3110565814 shutdown_connections 2024-03-15T12:59:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.393+0000 7fac88dfd740 1 --2- 172.21.15.2:0/3110565814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae53750 0x5581dae55b40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.393+0000 7fac88dfd740 1 --2- 172.21.15.2:0/3110565814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4ab70 0x5581dae53210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.393+0000 7fac88dfd740 1 --2- 172.21.15.2:0/3110565814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae4a250 0x5581dae4a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.393+0000 7fac88dfd740 1 -- 172.21.15.2:0/3110565814 >> 172.21.15.2:0/3110565814 conn(0x5581dae44980 msgr2=0x5581dae44d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.393+0000 7fac88dfd740 1 -- 172.21.15.2:0/3110565814 shutdown_connections 2024-03-15T12:59:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.393+0000 7fac88dfd740 1 -- 172.21.15.2:0/3110565814 wait complete. 2024-03-15T12:59:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.393+0000 7fac88dfd740 1 -- start start 2024-03-15T12:59:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac88dfd740 1 Processor -- start 2024-03-15T12:59:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac88dfd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4a250 0x5581dae53070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac88dfd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae4ab70 0x5581dae50400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac88dfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae53750 0x5581dae50940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac88dfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dadb2020 con 0x5581dae53750 2024-03-15T12:59:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac88dfd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581daa85430 con 0x5581dae4a250 2024-03-15T12:59:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac88dfd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581daa41e00 con 0x5581dae4ab70 2024-03-15T12:59:17.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac88dfd740 10 client.0 _ll_register_callbacks cb 0x5581dad59cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:17.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac84b1e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae53750 0x5581dae50940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac7f7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae4ab70 0x5581dae50400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac84b1e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae53750 0x5581dae50940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52934/0 (socket says 172.21.15.2:52934) 2024-03-15T12:59:17.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac84b1e640 1 -- 172.21.15.2:0/1208375877 learned_addr learned my addr 172.21.15.2:0/1208375877 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:17.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac7f7fe640 1 --2- 172.21.15.2:0/1208375877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae4ab70 0x5581dae50400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:36932/0 (socket says 172.21.15.2:36932) 2024-03-15T12:59:17.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac7ffff640 1 --2- 172.21.15.2:0/1208375877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4a250 0x5581dae53070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac84b1e640 1 -- 172.21.15.2:0/1208375877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae4ab70 msgr2=0x5581dae50400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac84b1e640 1 --2- 172.21.15.2:0/1208375877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae4ab70 0x5581dae50400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac84b1e640 1 -- 172.21.15.2:0/1208375877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4a250 msgr2=0x5581dae53070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac84b1e640 1 --2- 172.21.15.2:0/1208375877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4a250 0x5581dae53070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac84b1e640 1 -- 172.21.15.2:0/1208375877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac74009890 con 0x5581dae53750 2024-03-15T12:59:17.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac84b1e640 1 --2- 172.21.15.2:0/1208375877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae53750 0x5581dae50940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fac78009ca0 tx=0x7fac7802c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d21ee54c8c5c9b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.394+0000 7fac5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:17.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.395+0000 7fac5f7fe640 1 -- 172.21.15.2:0/1208375877 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac78032b30 con 0x5581dae53750 2024-03-15T12:59:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.395+0000 7fac5f7fe640 1 -- 172.21.15.2:0/1208375877 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fac78003970 con 0x5581dae53750 2024-03-15T12:59:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.395+0000 7fac88dfd740 1 -- 172.21.15.2:0/1208375877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581daab17e0 con 0x5581dae53750 2024-03-15T12:59:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.395+0000 7fac5f7fe640 1 -- 172.21.15.2:0/1208375877 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac78003b30 con 0x5581dae53750 2024-03-15T12:59:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.395+0000 7fac88dfd740 10 client.15936 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.395+0000 7fac88dfd740 1 -- 172.21.15.2:0/1208375877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5581dadb06a0 con 0x5581dae53750 2024-03-15T12:59:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.395+0000 7fac5f7fe640 1 -- 172.21.15.2:0/1208375877 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5581daab17e0 con 0x5581dae53750 2024-03-15T12:59:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.395+0000 7fac5f7fe640 1 -- 172.21.15.2:0/1208375877 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fac7804c070 con 0x5581dae53750 2024-03-15T12:59:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.395+0000 7fac88dfd740 10 client.15936 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.395+0000 7fac88dfd740 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.395+0000 7fac88dfd740 20 client.15936 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.395+0000 7fac88dfd740 1 --2- 172.21.15.2:0/1208375877 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5581dae63890 0x5581dae65cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.395+0000 7fac88dfd740 4 client.15936 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.395+0000 7fac88dfd740 1 -- 172.21.15.2:0/1208375877 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581daaee720 con 0x5581dae63890 2024-03-15T12:59:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.395+0000 7fac7ffff640 1 --2- 172.21.15.2:0/1208375877 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5581dae63890 0x5581dae65cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.396+0000 7fac7ffff640 1 --2- 172.21.15.2:0/1208375877 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5581dae63890 0x5581dae65cb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4252c86188a36b5 server_cookie=b83944560e888d04 in_seq=0 out_seq=0 2024-03-15T12:59:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.396+0000 7fac5f7fe640 10 client.15936 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.396+0000 7fac5f7fe640 1 -- 172.21.15.2:0/1208375877 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5581daaee720 con 0x5581dae63890 2024-03-15T12:59:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.396+0000 7fac5f7fe640 10 client.15936 handle_command_reply: tid=0 2024-03-15T12:59:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.396+0000 7fac88dfd740 1 client.15936 shutdown 2024-03-15T12:59:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.397+0000 7fac88dfd740 1 -- 172.21.15.2:0/1208375877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae53750 msgr2=0x5581dae50940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.397+0000 7fac88dfd740 1 --2- 172.21.15.2:0/1208375877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae53750 0x5581dae50940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fac78009ca0 tx=0x7fac7802c140 comp rx=0 tx=0).stop 2024-03-15T12:59:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.397+0000 7fac88dfd740 1 -- 172.21.15.2:0/1208375877 shutdown_connections 2024-03-15T12:59:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.397+0000 7fac88dfd740 1 --2- 172.21.15.2:0/1208375877 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5581dae63890 0x5581dae65cb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.397+0000 7fac88dfd740 1 --2- 172.21.15.2:0/1208375877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae53750 0x5581dae50940 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.397+0000 7fac5f7fe640 0 client.15936 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.397+0000 7fac88dfd740 1 --2- 172.21.15.2:0/1208375877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae4ab70 0x5581dae50400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.397+0000 7fac88dfd740 1 --2- 172.21.15.2:0/1208375877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4a250 0x5581dae53070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.397+0000 7fac88dfd740 1 -- 172.21.15.2:0/1208375877 >> 172.21.15.2:0/1208375877 conn(0x5581dae44980 msgr2=0x5581dae466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.397+0000 7fac88dfd740 1 -- 172.21.15.2:0/1208375877 shutdown_connections 2024-03-15T12:59:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.397+0000 7fac88dfd740 1 -- 172.21.15.2:0/1208375877 wait complete. 2024-03-15T12:59:17.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.397+0000 7fac88dfd740 20 client.15936 trim_cache size 0 max 16384 2024-03-15T12:59:17.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.409+0000 7fac88dfd740 1 Processor -- start 2024-03-15T12:59:17.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.409+0000 7fac88dfd740 1 -- start start 2024-03-15T12:59:17.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.409+0000 7fac88dfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae4a250 0x5581dae52b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.409+0000 7fac88dfd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4ab70 0x5581dae53070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.409+0000 7fac88dfd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae53750 0x5581dae5ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.409+0000 7fac88dfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dad37130 con 0x5581dae4a250 2024-03-15T12:59:17.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.409+0000 7fac88dfd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dae5fc70 con 0x5581dae4ab70 2024-03-15T12:59:17.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.409+0000 7fac88dfd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dae49540 con 0x5581dae53750 2024-03-15T12:59:17.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.409+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae4a250 0x5581dae52b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.409+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae4a250 0x5581dae52b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52950/0 (socket says 172.21.15.2:52950) 2024-03-15T12:59:17.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.409+0000 7fac7ffff640 1 -- 172.21.15.2:0/2349433865 learned_addr learned my addr 172.21.15.2:0/2349433865 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:17.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.409+0000 7fac7f7fe640 1 --2- 172.21.15.2:0/2349433865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4ab70 0x5581dae53070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.409+0000 7fac84b1e640 1 --2- 172.21.15.2:0/2349433865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae53750 0x5581dae5ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.410+0000 7fac7ffff640 1 -- 172.21.15.2:0/2349433865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae53750 msgr2=0x5581dae5ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.410+0000 7fac7ffff640 1 --2- 172.21.15.2:0/2349433865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae53750 0x5581dae5ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.410+0000 7fac7ffff640 1 -- 172.21.15.2:0/2349433865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4ab70 msgr2=0x5581dae53070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.410+0000 7fac7ffff640 1 --2- 172.21.15.2:0/2349433865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4ab70 0x5581dae53070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.410+0000 7fac7ffff640 1 -- 172.21.15.2:0/2349433865 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581dad4cc20 con 0x5581dae4a250 2024-03-15T12:59:17.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.410+0000 7fac7ffff640 1 --2- 172.21.15.2:0/2349433865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae4a250 0x5581dae52b30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fac70096c80 tx=0x7fac70085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39305d69c7c8fc9a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.410+0000 7fac65ffb640 1 -- 172.21.15.2:0/2349433865 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7009e8c0 con 0x5581dae4a250 2024-03-15T12:59:17.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.410+0000 7fac65ffb640 1 -- 172.21.15.2:0/2349433865 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fac700a2f10 con 0x5581dae4a250 2024-03-15T12:59:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.410+0000 7fac65ffb640 1 -- 172.21.15.2:0/2349433865 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac700aa110 con 0x5581dae4a250 2024-03-15T12:59:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.410+0000 7fac88dfd740 1 -- 172.21.15.2:0/2349433865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae4a250 msgr2=0x5581dae52b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.410+0000 7fac88dfd740 1 --2- 172.21.15.2:0/2349433865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae4a250 0x5581dae52b30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fac70096c80 tx=0x7fac70085040 comp rx=0 tx=0).stop 2024-03-15T12:59:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.410+0000 7fac88dfd740 1 -- 172.21.15.2:0/2349433865 shutdown_connections 2024-03-15T12:59:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.410+0000 7fac88dfd740 1 --2- 172.21.15.2:0/2349433865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae53750 0x5581dae5ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.410+0000 7fac88dfd740 1 --2- 172.21.15.2:0/2349433865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4ab70 0x5581dae53070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.410+0000 7fac88dfd740 1 --2- 172.21.15.2:0/2349433865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae4a250 0x5581dae52b30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac88dfd740 1 -- 172.21.15.2:0/2349433865 >> 172.21.15.2:0/2349433865 conn(0x5581dae44980 msgr2=0x5581dae4b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac88dfd740 1 -- 172.21.15.2:0/2349433865 shutdown_connections 2024-03-15T12:59:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac88dfd740 1 -- 172.21.15.2:0/2349433865 wait complete. 2024-03-15T12:59:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac88dfd740 1 -- start start 2024-03-15T12:59:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac88dfd740 1 Processor -- start 2024-03-15T12:59:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac88dfd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4a250 0x5581daeb7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac88dfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae4ab70 0x5581daeb7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac88dfd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae53750 0x5581dae4d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac88dfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dae4dad0 con 0x5581dae4ab70 2024-03-15T12:59:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac88dfd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dae4dc40 con 0x5581dae4a250 2024-03-15T12:59:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac88dfd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581dae4de00 con 0x5581dae53750 2024-03-15T12:59:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac88dfd740 10 client.0 _ll_register_callbacks cb 0x7fac74005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac7f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae4ab70 0x5581daeb7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac7f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae4ab70 0x5581daeb7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52964/0 (socket says 172.21.15.2:52964) 2024-03-15T12:59:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac7f7fe640 1 -- 172.21.15.2:0/536892276 learned_addr learned my addr 172.21.15.2:0/536892276 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac7ffff640 1 --2- 172.21.15.2:0/536892276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4a250 0x5581daeb7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.411+0000 7fac84b1e640 1 --2- 172.21.15.2:0/536892276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae53750 0x5581dae4d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.412+0000 7fac7f7fe640 1 -- 172.21.15.2:0/536892276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae53750 msgr2=0x5581dae4d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.412+0000 7fac7f7fe640 1 --2- 172.21.15.2:0/536892276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae53750 0x5581dae4d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.412+0000 7fac7f7fe640 1 -- 172.21.15.2:0/536892276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4a250 msgr2=0x5581daeb7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.412+0000 7fac7f7fe640 1 --2- 172.21.15.2:0/536892276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4a250 0x5581daeb7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.412+0000 7fac7f7fe640 1 -- 172.21.15.2:0/536892276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac700a94b0 con 0x5581dae4ab70 2024-03-15T12:59:17.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.412+0000 7fac7f7fe640 1 --2- 172.21.15.2:0/536892276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae4ab70 0x5581daeb7a50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5581dae45b10 tx=0x7fac740055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541386eb4bb0df6a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:17.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.412+0000 7fac5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:17.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.412+0000 7fac5f7fe640 1 -- 172.21.15.2:0/536892276 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac74005690 con 0x5581dae4ab70 2024-03-15T12:59:17.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.412+0000 7fac5f7fe640 1 -- 172.21.15.2:0/536892276 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fac740035f0 con 0x5581dae4ab70 2024-03-15T12:59:17.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.412+0000 7fac88dfd740 1 -- 172.21.15.2:0/536892276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac74004390 con 0x5581dae4ab70 2024-03-15T12:59:17.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.412+0000 7fac5f7fe640 1 -- 172.21.15.2:0/536892276 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac740037b0 con 0x5581dae4ab70 2024-03-15T12:59:17.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.412+0000 7fac88dfd740 10 client.15942 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:17.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.412+0000 7fac88dfd740 1 -- 172.21.15.2:0/536892276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5581dad728c0 con 0x5581dae4ab70 2024-03-15T12:59:17.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.412+0000 7fac5f7fe640 1 -- 172.21.15.2:0/536892276 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fac74004390 con 0x5581dae4ab70 2024-03-15T12:59:17.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.413+0000 7fac5f7fe640 1 -- 172.21.15.2:0/536892276 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fac7402c020 con 0x5581dae4ab70 2024-03-15T12:59:17.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.413+0000 7fac88dfd740 10 client.15942 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:17.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.413+0000 7fac88dfd740 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:17.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.413+0000 7fac88dfd740 20 client.15942 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:17.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.413+0000 7fac88dfd740 1 --2- 172.21.15.2:0/536892276 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5581daec0450 0x5581daec2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:17.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.413+0000 7fac88dfd740 4 client.15942 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T12:59:17.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.413+0000 7fac88dfd740 1 -- 172.21.15.2:0/536892276 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5581dab3d890 con 0x5581daec0450 2024-03-15T12:59:17.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.413+0000 7fac7ffff640 1 --2- 172.21.15.2:0/536892276 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5581daec0450 0x5581daec2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:17.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.413+0000 7fac5f7fe640 10 client.15942 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:17.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.413+0000 7fac7ffff640 1 --2- 172.21.15.2:0/536892276 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5581daec0450 0x5581daec2870 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5dbe0b1f58e091 server_cookie=a00b63341710fa7 in_seq=0 out_seq=0 2024-03-15T12:59:17.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.413+0000 7fac5f7fe640 1 -- 172.21.15.2:0/536892276 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5581dab3d890 con 0x5581daec0450 2024-03-15T12:59:17.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.413+0000 7fac5f7fe640 10 client.15942 handle_command_reply: tid=0 2024-03-15T12:59:17.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.413+0000 7fac88dfd740 1 client.15942 shutdown 2024-03-15T12:59:17.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac88dfd740 1 -- 172.21.15.2:0/536892276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae4ab70 msgr2=0x5581daeb7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac88dfd740 1 --2- 172.21.15.2:0/536892276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae4ab70 0x5581daeb7a50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5581dae45b10 tx=0x7fac740055a0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac88dfd740 1 -- 172.21.15.2:0/536892276 shutdown_connections 2024-03-15T12:59:17.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac88dfd740 1 --2- 172.21.15.2:0/536892276 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5581daec0450 0x5581daec2870 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac88dfd740 1 --2- 172.21.15.2:0/536892276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5581dae53750 0x5581dae4d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac88dfd740 1 --2- 172.21.15.2:0/536892276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5581dae4ab70 0x5581daeb7a50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac88dfd740 1 --2- 172.21.15.2:0/536892276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5581dae4a250 0x5581daeb7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac5f7fe640 0 client.15942 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:17.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac88dfd740 1 -- 172.21.15.2:0/536892276 >> 172.21.15.2:0/536892276 conn(0x5581dae44980 msgr2=0x5581dae4b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac88dfd740 1 -- 172.21.15.2:0/536892276 shutdown_connections 2024-03-15T12:59:17.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac88dfd740 1 -- 172.21.15.2:0/536892276 wait complete. 2024-03-15T12:59:17.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac88dfd740 20 client.15942 trim_cache size 0 max 16384 2024-03-15T12:59:17.406 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T12:59:17.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac86581640 1 -- 172.21.15.2:0/2297135603 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fac680a1cc0 msgr2=0x7fac680a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac86581640 1 --2- 172.21.15.2:0/2297135603 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fac680a1cc0 0x7fac680a4180 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fac78004af0 tx=0x7fac78005c50 comp rx=0 tx=0).stop 2024-03-15T12:59:17.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac86581640 1 -- 172.21.15.2:0/2297135603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac80100b40 msgr2=0x7fac8019d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:17.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac86581640 1 --2- 172.21.15.2:0/2297135603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac80100b40 0x7fac8019d7b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fac7000e9f0 tx=0x7fac7000eeb0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac86581640 1 -- 172.21.15.2:0/2297135603 shutdown_connections 2024-03-15T12:59:17.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac86581640 1 --2- 172.21.15.2:0/2297135603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac801053a0 0x7fac80197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac86581640 1 --2- 172.21.15.2:0/2297135603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac801014e0 0x7fac8019dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac86581640 1 --2- 172.21.15.2:0/2297135603 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fac680a1cc0 0x7fac680a4180 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac86581640 1 --2- 172.21.15.2:0/2297135603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac80100b40 0x7fac8019d7b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:17.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac86581640 1 -- 172.21.15.2:0/2297135603 >> 172.21.15.2:0/2297135603 conn(0x7fac800769b0 msgr2=0x7fac800ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:17.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac86581640 1 -- 172.21.15.2:0/2297135603 shutdown_connections 2024-03-15T12:59:17.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:17.414+0000 7fac86581640 1 -- 172.21.15.2:0/2297135603 wait complete. 2024-03-15T12:59:17.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:17 smithi149 ceph-mon[35927]: pgmap v14: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-03-15T12:59:17.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:17 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/169507063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:59:18.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:17 smithi002 ceph-mon[32060]: pgmap v14: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-03-15T12:59:18.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:17 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/169507063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:59:18.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:17 smithi184 ceph-mon[35993]: pgmap v14: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-03-15T12:59:18.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:17 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/169507063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:59:18.411 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T12:59:18.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:59:18.412 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T12:59:18.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.544+0000 7f5ef383b640 1 -- 172.21.15.2:0/3538966600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5eec100f80 msgr2=0x7f5eec104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.544+0000 7f5ef383b640 1 --2- 172.21.15.2:0/3538966600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5eec100f80 0x7f5eec104fc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009930 tx=0x7f5edc02bc30 comp rx=0 tx=0).stop 2024-03-15T12:59:18.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.545+0000 7f5ef383b640 1 -- 172.21.15.2:0/3538966600 shutdown_connections 2024-03-15T12:59:18.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.545+0000 7f5ef383b640 1 --2- 172.21.15.2:0/3538966600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eec1056a0 0x7f5eec10d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.545+0000 7f5ef383b640 1 --2- 172.21.15.2:0/3538966600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5eec100f80 0x7f5eec104fc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.545+0000 7f5ef383b640 1 --2- 172.21.15.2:0/3538966600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5eec1005d0 0x7f5eec1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.545+0000 7f5ef383b640 1 -- 172.21.15.2:0/3538966600 >> 172.21.15.2:0/3538966600 conn(0x7f5eec0774f0 msgr2=0x7f5eec077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:18.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.545+0000 7f5ef383b640 1 -- 172.21.15.2:0/3538966600 shutdown_connections 2024-03-15T12:59:18.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.545+0000 7f5ef383b640 1 -- 172.21.15.2:0/3538966600 wait complete. 2024-03-15T12:59:18.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.545+0000 7f5ef383b640 1 Processor -- start 2024-03-15T12:59:18.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.545+0000 7f5ef383b640 1 -- start start 2024-03-15T12:59:18.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef383b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eec1005d0 0x7f5eec19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef383b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5eec1056a0 0x7f5eec19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef383b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5eec1979b0 0x7f5eec197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef383b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec1983a0 con 0x7f5eec1979b0 2024-03-15T12:59:18.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef383b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec198510 con 0x7f5eec1005d0 2024-03-15T12:59:18.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef383b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec1986d0 con 0x7f5eec1056a0 2024-03-15T12:59:18.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef1dd8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5eec1979b0 0x7f5eec197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef1dd8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5eec1979b0 0x7f5eec197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41072/0 (socket says 172.21.15.2:41072) 2024-03-15T12:59:18.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef1dd8640 1 -- 172.21.15.2:0/3739250119 learned_addr learned my addr 172.21.15.2:0/3739250119 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:18.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef15d7640 1 --2- 172.21.15.2:0/3739250119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eec1005d0 0x7f5eec19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef0dd6640 1 --2- 172.21.15.2:0/3739250119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5eec1056a0 0x7f5eec19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef1dd8640 1 -- 172.21.15.2:0/3739250119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5eec1056a0 msgr2=0x7f5eec19dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef1dd8640 1 --2- 172.21.15.2:0/3739250119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5eec1056a0 0x7f5eec19dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef1dd8640 1 -- 172.21.15.2:0/3739250119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eec1005d0 msgr2=0x7f5eec19d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef1dd8640 1 --2- 172.21.15.2:0/3739250119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eec1005d0 0x7f5eec19d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef1dd8640 1 -- 172.21.15.2:0/3739250119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5edc009560 con 0x7f5eec1979b0 2024-03-15T12:59:18.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef15d7640 1 --2- 172.21.15.2:0/3739250119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eec1005d0 0x7f5eec19d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:18.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ef1dd8640 1 --2- 172.21.15.2:0/3739250119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5eec1979b0 0x7f5eec197e60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800eb10 tx=0x7f5ee800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:18.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.546+0000 7f5ee27fc640 1 -- 172.21.15.2:0/3739250119 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8004b20 con 0x7f5eec1979b0 2024-03-15T12:59:18.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.547+0000 7f5ee27fc640 1 -- 172.21.15.2:0/3739250119 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5ee8004c80 con 0x7f5eec1979b0 2024-03-15T12:59:18.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.547+0000 7f5ee27fc640 1 -- 172.21.15.2:0/3739250119 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8010690 con 0x7f5eec1979b0 2024-03-15T12:59:18.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.547+0000 7f5ef383b640 1 -- 172.21.15.2:0/3739250119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec1a46c0 con 0x7f5eec1979b0 2024-03-15T12:59:18.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.547+0000 7f5ef383b640 1 -- 172.21.15.2:0/3739250119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec1a4ba0 con 0x7f5eec1979b0 2024-03-15T12:59:18.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.549+0000 7f5ee27fc640 1 -- 172.21.15.2:0/3739250119 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7f5ee8004df0 con 0x7f5eec1979b0 2024-03-15T12:59:18.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.549+0000 7f5ef6417740 1 Processor -- start 2024-03-15T12:59:18.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.549+0000 7f5ef6417740 1 -- start start 2024-03-15T12:59:18.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.549+0000 7f5ef6417740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e75760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.549+0000 7f5ef6417740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 0x5604d4e7e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.549+0000 7f5ee27fc640 1 --2- 172.21.15.2:0/3739250119 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ed80a1d40 0x7f5ed80a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.549+0000 7f5ef6417740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e7e8b0 0x5604d4e80d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.549+0000 7f5ef6417740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d4de3f90 con 0x5604d4e75380 2024-03-15T12:59:18.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.549+0000 7f5ef6417740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d4de14c0 con 0x5604d4e75d30 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.549+0000 7f5ef6417740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d4ddcca0 con 0x5604d4e7e8b0 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ef0dd6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e75760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ef0dd6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e75760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41074/0 (socket says 172.21.15.2:41074) 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ef0dd6640 1 -- 172.21.15.2:0/154945499 learned_addr learned my addr 172.21.15.2:0/154945499 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ef15d7640 1 --2- 172.21.15.2:0/154945499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 0x5604d4e7e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ef1dd8640 1 --2- 172.21.15.2:0/154945499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e7e8b0 0x5604d4e80d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ef0dd6640 1 --2- 172.21.15.2:0/3739250119 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ed80a1d40 0x7f5ed80a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ee27fc640 1 -- 172.21.15.2:0/3739250119 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5ee8014070 con 0x7f5eec1979b0 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ef15d7640 1 -- 172.21.15.2:0/154945499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e7e8b0 msgr2=0x5604d4e80d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ef15d7640 1 --2- 172.21.15.2:0/154945499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e7e8b0 0x5604d4e80d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ef15d7640 1 -- 172.21.15.2:0/154945499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 msgr2=0x5604d4e75760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ef15d7640 1 --2- 172.21.15.2:0/154945499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e75760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ef15d7640 1 -- 172.21.15.2:0/154945499 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604d4d9f0b0 con 0x5604d4e75d30 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ef0dd6640 1 --2- 172.21.15.2:0/3739250119 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ed80a1d40 0x7f5ed80a4200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5edc005ec0 tx=0x7f5edc000bc0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ef15d7640 1 --2- 172.21.15.2:0/154945499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 0x5604d4e7e2e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009fa0 tx=0x7f5ee40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85df9f27a1178cc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:18.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ed77fe640 1 -- 172.21.15.2:0/154945499 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40040f0 con 0x5604d4e75d30 2024-03-15T12:59:18.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ed77fe640 1 -- 172.21.15.2:0/154945499 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5ee40054f0 con 0x5604d4e75d30 2024-03-15T12:59:18.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.550+0000 7f5ed77fe640 1 -- 172.21.15.2:0/154945499 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4004d40 con 0x5604d4e75d30 2024-03-15T12:59:18.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.551+0000 7f5ef6417740 1 -- 172.21.15.2:0/154945499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 msgr2=0x5604d4e7e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.551+0000 7f5ef6417740 1 --2- 172.21.15.2:0/154945499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 0x5604d4e7e2e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009fa0 tx=0x7f5ee40118d0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.551+0000 7f5ef6417740 1 -- 172.21.15.2:0/154945499 shutdown_connections 2024-03-15T12:59:18.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.551+0000 7f5ef6417740 1 --2- 172.21.15.2:0/154945499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e7e8b0 0x5604d4e80d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.551+0000 7f5ef6417740 1 --2- 172.21.15.2:0/154945499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 0x5604d4e7e2e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.551+0000 7f5ef6417740 1 --2- 172.21.15.2:0/154945499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e75760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.551+0000 7f5ef6417740 1 -- 172.21.15.2:0/154945499 >> 172.21.15.2:0/154945499 conn(0x5604d4e6f840 msgr2=0x5604d4e6fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:18.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.551+0000 7f5ef6417740 1 -- 172.21.15.2:0/154945499 shutdown_connections 2024-03-15T12:59:18.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.551+0000 7f5ef6417740 1 -- 172.21.15.2:0/154945499 wait complete. 2024-03-15T12:59:18.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef6417740 1 -- start start 2024-03-15T12:59:18.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef6417740 1 Processor -- start 2024-03-15T12:59:18.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef6417740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e7b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef6417740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 0x5604d4e7bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef6417740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e7e8b0 0x5604d4e7c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef6417740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d4d81f40 con 0x5604d4e75380 2024-03-15T12:59:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef6417740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d4ab04b0 con 0x5604d4e75d30 2024-03-15T12:59:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef6417740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d4d62670 con 0x5604d4e7e8b0 2024-03-15T12:59:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef6417740 10 client.0 _ll_register_callbacks cb 0x5604d4dcff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef15d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e7b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef15d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e7b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41078/0 (socket says 172.21.15.2:41078) 2024-03-15T12:59:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef15d7640 1 -- 172.21.15.2:0/2396786508 learned_addr learned my addr 172.21.15.2:0/2396786508 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef0dd6640 1 --2- 172.21.15.2:0/2396786508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 0x5604d4e7bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef1dd8640 1 --2- 172.21.15.2:0/2396786508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e7e8b0 0x5604d4e7c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef15d7640 1 -- 172.21.15.2:0/2396786508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e7e8b0 msgr2=0x5604d4e7c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef15d7640 1 --2- 172.21.15.2:0/2396786508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e7e8b0 0x5604d4e7c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef15d7640 1 -- 172.21.15.2:0/2396786508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 msgr2=0x5604d4e7bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef15d7640 1 --2- 172.21.15.2:0/2396786508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 0x5604d4e7bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.552+0000 7f5ef15d7640 1 -- 172.21.15.2:0/2396786508 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee4009c00 con 0x5604d4e75380 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.553+0000 7f5ef15d7640 1 --2- 172.21.15.2:0/2396786508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e7b870 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5ee401b040 tx=0x7f5ee40045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10910db683f2ace6 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.553+0000 7f5ed4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.553+0000 7f5ed4ff9640 1 -- 172.21.15.2:0/2396786508 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4003980 con 0x5604d4e75380 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.553+0000 7f5ed4ff9640 1 -- 172.21.15.2:0/2396786508 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5ee4003ae0 con 0x5604d4e75380 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.553+0000 7f5ef6417740 1 -- 172.21.15.2:0/2396786508 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604d4db6f40 con 0x5604d4e75380 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.553+0000 7f5ed4ff9640 1 -- 172.21.15.2:0/2396786508 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ee4003ca0 con 0x5604d4e75380 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.553+0000 7f5ef6417740 10 client.15954 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.553+0000 7f5ef6417740 1 -- 172.21.15.2:0/2396786508 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5604d4ddb320 con 0x5604d4e75380 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.553+0000 7f5ed4ff9640 1 -- 172.21.15.2:0/2396786508 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5604d4db6f40 con 0x5604d4e75380 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.553+0000 7f5ed4ff9640 1 -- 172.21.15.2:0/2396786508 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5ee4024510 con 0x5604d4e75380 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.554+0000 7f5ef6417740 10 client.15954 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.554+0000 7f5ef6417740 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.554+0000 7f5ef6417740 20 client.15954 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.554+0000 7f5ef6417740 1 --2- 172.21.15.2:0/2396786508 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5604d4e8ee40 0x5604d4e91250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.554+0000 7f5ef6417740 4 client.15954 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.554+0000 7f5ef6417740 1 -- 172.21.15.2:0/2396786508 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604d4dd3310 con 0x5604d4e8ee40 2024-03-15T12:59:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.554+0000 7f5ef0dd6640 1 --2- 172.21.15.2:0/2396786508 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5604d4e8ee40 0x5604d4e91250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.554+0000 7f5ed4ff9640 10 client.15954 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.554+0000 7f5ef0dd6640 1 --2- 172.21.15.2:0/2396786508 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5604d4e8ee40 0x5604d4e91250 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28212cc2137ee28a server_cookie=8b646ed02fbed2a6 in_seq=0 out_seq=0 2024-03-15T12:59:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.555+0000 7f5ed4ff9640 1 -- 172.21.15.2:0/2396786508 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5604d4dd3310 con 0x5604d4e8ee40 2024-03-15T12:59:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.555+0000 7f5ed4ff9640 10 client.15954 handle_command_reply: tid=0 2024-03-15T12:59:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.555+0000 7f5ef6417740 1 client.15954 shutdown 2024-03-15T12:59:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.555+0000 7f5ef6417740 1 -- 172.21.15.2:0/2396786508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 msgr2=0x5604d4e7b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.555+0000 7f5ef6417740 1 --2- 172.21.15.2:0/2396786508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e7b870 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5ee401b040 tx=0x7f5ee40045a0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.556+0000 7f5ef6417740 1 -- 172.21.15.2:0/2396786508 shutdown_connections 2024-03-15T12:59:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.556+0000 7f5ef6417740 1 --2- 172.21.15.2:0/2396786508 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5604d4e8ee40 0x5604d4e91250 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.556+0000 7f5ef6417740 1 --2- 172.21.15.2:0/2396786508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e7e8b0 0x5604d4e7c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.556+0000 7f5ed4ff9640 0 client.15954 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.556+0000 7f5ef6417740 1 --2- 172.21.15.2:0/2396786508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 0x5604d4e7bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.556+0000 7f5ef6417740 1 --2- 172.21.15.2:0/2396786508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e7b870 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.556+0000 7f5ef6417740 1 -- 172.21.15.2:0/2396786508 >> 172.21.15.2:0/2396786508 conn(0x5604d4e6f840 msgr2=0x5604d4e717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.556+0000 7f5ef6417740 1 -- 172.21.15.2:0/2396786508 shutdown_connections 2024-03-15T12:59:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.556+0000 7f5ef6417740 1 -- 172.21.15.2:0/2396786508 wait complete. 2024-03-15T12:59:18.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.556+0000 7f5ef6417740 20 client.15954 trim_cache size 0 max 16384 2024-03-15T12:59:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.572+0000 7f5ef6417740 1 Processor -- start 2024-03-15T12:59:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.572+0000 7f5ef6417740 1 -- start start 2024-03-15T12:59:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.572+0000 7f5ef6417740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75380 0x5604d4e7dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.572+0000 7f5ef6417740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e75d30 0x5604d4e89db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.572+0000 7f5ef6417740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e7e8b0 0x5604d4e8a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.572+0000 7f5ef6417740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d4e7a810 con 0x5604d4e7e8b0 2024-03-15T12:59:18.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.572+0000 7f5ef6417740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d4e76ba0 con 0x5604d4e75380 2024-03-15T12:59:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.572+0000 7f5ef6417740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d4e76d60 con 0x5604d4e75d30 2024-03-15T12:59:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.572+0000 7f5ef1dd8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e7e8b0 0x5604d4e8a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.572+0000 7f5ef15d7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75380 0x5604d4e7dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.572+0000 7f5ef1dd8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e7e8b0 0x5604d4e8a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41088/0 (socket says 172.21.15.2:41088) 2024-03-15T12:59:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.572+0000 7f5ef1dd8640 1 -- 172.21.15.2:0/1727756597 learned_addr learned my addr 172.21.15.2:0/1727756597 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.573+0000 7f5ef0dd6640 1 --2- 172.21.15.2:0/1727756597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e75d30 0x5604d4e89db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.573+0000 7f5ef1dd8640 1 -- 172.21.15.2:0/1727756597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e75d30 msgr2=0x5604d4e89db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.573+0000 7f5ef1dd8640 1 --2- 172.21.15.2:0/1727756597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e75d30 0x5604d4e89db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.573+0000 7f5ef1dd8640 1 -- 172.21.15.2:0/1727756597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75380 msgr2=0x5604d4e7dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.573+0000 7f5ef1dd8640 1 --2- 172.21.15.2:0/1727756597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75380 0x5604d4e7dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.573+0000 7f5ef1dd8640 1 -- 172.21.15.2:0/1727756597 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604d4d77c00 con 0x5604d4e7e8b0 2024-03-15T12:59:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.573+0000 7f5ef15d7640 1 --2- 172.21.15.2:0/1727756597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75380 0x5604d4e7dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.573+0000 7f5ef1dd8640 1 --2- 172.21.15.2:0/1727756597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e7e8b0 0x5604d4e8a2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80cbcb0 tx=0x7f5ee809a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e297a7411a7742e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.573+0000 7f5ed6ffd640 1 -- 172.21.15.2:0/1727756597 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee80acfb0 con 0x5604d4e7e8b0 2024-03-15T12:59:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.573+0000 7f5ed6ffd640 1 -- 172.21.15.2:0/1727756597 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5ee809b040 con 0x5604d4e7e8b0 2024-03-15T12:59:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.573+0000 7f5ed6ffd640 1 -- 172.21.15.2:0/1727756597 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8098ab0 con 0x5604d4e7e8b0 2024-03-15T12:59:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 -- 172.21.15.2:0/1727756597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e7e8b0 msgr2=0x5604d4e8a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 --2- 172.21.15.2:0/1727756597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e7e8b0 0x5604d4e8a2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80cbcb0 tx=0x7f5ee809a7d0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 -- 172.21.15.2:0/1727756597 shutdown_connections 2024-03-15T12:59:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 --2- 172.21.15.2:0/1727756597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e7e8b0 0x5604d4e8a2f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 --2- 172.21.15.2:0/1727756597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e75d30 0x5604d4e89db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 --2- 172.21.15.2:0/1727756597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75380 0x5604d4e7dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 -- 172.21.15.2:0/1727756597 >> 172.21.15.2:0/1727756597 conn(0x5604d4e6f840 msgr2=0x5604d4e6e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 -- 172.21.15.2:0/1727756597 shutdown_connections 2024-03-15T12:59:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 -- 172.21.15.2:0/1727756597 wait complete. 2024-03-15T12:59:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 -- start start 2024-03-15T12:59:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 Processor -- start 2024-03-15T12:59:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e838c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 0x5604d4e83e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e7e8b0 0x5604d4eea700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d4e846d0 con 0x5604d4e75380 2024-03-15T12:59:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d4eeac40 con 0x5604d4e75d30 2024-03-15T12:59:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604d4eead80 con 0x5604d4e7e8b0 2024-03-15T12:59:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.574+0000 7f5ef6417740 10 client.0 _ll_register_callbacks cb 0x7f5ee4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ef15d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e838c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ef15d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e838c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41104/0 (socket says 172.21.15.2:41104) 2024-03-15T12:59:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ef15d7640 1 -- 172.21.15.2:0/1227444409 learned_addr learned my addr 172.21.15.2:0/1227444409 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ef0dd6640 1 --2- 172.21.15.2:0/1227444409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 0x5604d4e83e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ef1dd8640 1 --2- 172.21.15.2:0/1227444409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e7e8b0 0x5604d4eea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ef15d7640 1 -- 172.21.15.2:0/1227444409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e7e8b0 msgr2=0x5604d4eea700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ef15d7640 1 --2- 172.21.15.2:0/1227444409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e7e8b0 0x5604d4eea700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ef15d7640 1 -- 172.21.15.2:0/1227444409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 msgr2=0x5604d4e83e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ef15d7640 1 --2- 172.21.15.2:0/1227444409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 0x5604d4e83e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ef15d7640 1 -- 172.21.15.2:0/1227444409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604d4ddb320 con 0x5604d4e75380 2024-03-15T12:59:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ef15d7640 1 --2- 172.21.15.2:0/1227444409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e838c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4004910 tx=0x7f5ee4004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41711e97cd5ba804 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ed4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ed4ff9640 1 -- 172.21.15.2:0/1227444409 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4003b80 con 0x5604d4e75380 2024-03-15T12:59:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ed4ff9640 1 -- 172.21.15.2:0/1227444409 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5ee4000e40 con 0x5604d4e75380 2024-03-15T12:59:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ef6417740 1 -- 172.21.15.2:0/1227444409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604d4e02900 con 0x5604d4e75380 2024-03-15T12:59:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ed4ff9640 1 -- 172.21.15.2:0/1227444409 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ee40035f0 con 0x5604d4e75380 2024-03-15T12:59:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ef6417740 10 client.15960 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ef6417740 1 -- 172.21.15.2:0/1227444409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f5eec1a4ba0 con 0x5604d4e75380 2024-03-15T12:59:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.575+0000 7f5ed4ff9640 1 -- 172.21.15.2:0/1227444409 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5604d4e02900 con 0x5604d4e75380 2024-03-15T12:59:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.576+0000 7f5ed4ff9640 1 -- 172.21.15.2:0/1227444409 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5ee400f9b0 con 0x5604d4e75380 2024-03-15T12:59:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.576+0000 7f5ef6417740 10 client.15960 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.576+0000 7f5ef6417740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.576+0000 7f5ef6417740 20 client.15960 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:18.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.576+0000 7f5ef6417740 1 --2- 172.21.15.2:0/1227444409 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5604d4eec260 0x5604d4eee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.576+0000 7f5ef6417740 4 client.15960 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T12:59:18.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.576+0000 7f5ef6417740 1 -- 172.21.15.2:0/1227444409 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5604d4adc860 con 0x5604d4eec260 2024-03-15T12:59:18.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.576+0000 7f5ef0dd6640 1 --2- 172.21.15.2:0/1227444409 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5604d4eec260 0x5604d4eee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.576+0000 7f5ed4ff9640 10 client.15960 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:18.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.576+0000 7f5ef0dd6640 1 --2- 172.21.15.2:0/1227444409 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5604d4eec260 0x5604d4eee680 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2420be2d32f5ca8 server_cookie=d51be431dfd8f78a in_seq=0 out_seq=0 2024-03-15T12:59:18.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.578+0000 7f5ed4ff9640 1 -- 172.21.15.2:0/1227444409 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5604d4adc860 con 0x5604d4eec260 2024-03-15T12:59:18.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.578+0000 7f5ed4ff9640 10 client.15960 handle_command_reply: tid=0 2024-03-15T12:59:18.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.578+0000 7f5ef6417740 1 client.15960 shutdown 2024-03-15T12:59:18.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.578+0000 7f5ef6417740 1 -- 172.21.15.2:0/1227444409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 msgr2=0x5604d4e838c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.578+0000 7f5ef6417740 1 --2- 172.21.15.2:0/1227444409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e838c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4004910 tx=0x7f5ee4004400 comp rx=0 tx=0).stop 2024-03-15T12:59:18.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.578+0000 7f5ef6417740 1 -- 172.21.15.2:0/1227444409 shutdown_connections 2024-03-15T12:59:18.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.578+0000 7f5ef6417740 1 --2- 172.21.15.2:0/1227444409 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5604d4eec260 0x5604d4eee680 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.578+0000 7f5ef6417740 1 --2- 172.21.15.2:0/1227444409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604d4e7e8b0 0x5604d4eea700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.578+0000 7f5ef6417740 1 --2- 172.21.15.2:0/1227444409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604d4e75d30 0x5604d4e83e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.578+0000 7f5ef6417740 1 --2- 172.21.15.2:0/1227444409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604d4e75380 0x5604d4e838c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.578+0000 7f5ef6417740 1 -- 172.21.15.2:0/1227444409 >> 172.21.15.2:0/1227444409 conn(0x5604d4e6f840 msgr2=0x5604d4e70a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.578+0000 7f5ed4ff9640 0 client.15960 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.578+0000 7f5ef6417740 1 -- 172.21.15.2:0/1227444409 shutdown_connections 2024-03-15T12:59:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.578+0000 7f5ef6417740 1 -- 172.21.15.2:0/1227444409 wait complete. 2024-03-15T12:59:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.578+0000 7f5ef6417740 20 client.15960 trim_cache size 0 max 16384 2024-03-15T12:59:18.569 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:59:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T12:59:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "e1876b4f-2020-42e1-9b9e-ed23aa67b7dc", 2024-03-15T12:59:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T12:59:18.570 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:59:18.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.579+0000 7f5ef383b640 1 -- 172.21.15.2:0/3739250119 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ed80a1d40 msgr2=0x7f5ed80a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.579+0000 7f5ef383b640 1 --2- 172.21.15.2:0/3739250119 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ed80a1d40 0x7f5ed80a4200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5edc005ec0 tx=0x7f5edc000bc0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.579+0000 7f5ef383b640 1 -- 172.21.15.2:0/3739250119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5eec1979b0 msgr2=0x7f5eec197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.579+0000 7f5ef383b640 1 --2- 172.21.15.2:0/3739250119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5eec1979b0 0x7f5eec197e60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800eb10 tx=0x7f5ee800c2d0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.579+0000 7f5ef383b640 1 -- 172.21.15.2:0/3739250119 shutdown_connections 2024-03-15T12:59:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.579+0000 7f5ef383b640 1 --2- 172.21.15.2:0/3739250119 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ed80a1d40 0x7f5ed80a4200 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.579+0000 7f5ef383b640 1 --2- 172.21.15.2:0/3739250119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5eec1979b0 0x7f5eec197e60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.579+0000 7f5ef383b640 1 --2- 172.21.15.2:0/3739250119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5eec1056a0 0x7f5eec19dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.579+0000 7f5ef383b640 1 --2- 172.21.15.2:0/3739250119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eec1005d0 0x7f5eec19d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.579+0000 7f5ef383b640 1 -- 172.21.15.2:0/3739250119 >> 172.21.15.2:0/3739250119 conn(0x7f5eec0774f0 msgr2=0x7f5eec0689b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.579+0000 7f5ef383b640 1 -- 172.21.15.2:0/3739250119 shutdown_connections 2024-03-15T12:59:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.579+0000 7f5ef383b640 1 -- 172.21.15.2:0/3739250119 wait complete. 2024-03-15T12:59:18.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T12:59:18.575 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T12:59:18.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.694+0000 7fd713ad4640 1 -- 172.21.15.2:0/1907105491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd70c1003b0 msgr2=0x7fd70c10e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.694+0000 7fd713ad4640 1 --2- 172.21.15.2:0/1907105491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd70c1003b0 0x7fd70c10e320 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd704009cd0 tx=0x7fd70402c0f0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.694+0000 7fd713ad4640 1 -- 172.21.15.2:0/1907105491 shutdown_connections 2024-03-15T12:59:18.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.694+0000 7fd713ad4640 1 --2- 172.21.15.2:0/1907105491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd70c1003b0 0x7fd70c10e320 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.694+0000 7fd713ad4640 1 --2- 172.21.15.2:0/1907105491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd70c0ff9f0 0x7fd70c0ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.694+0000 7fd713ad4640 1 --2- 172.21.15.2:0/1907105491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd70c107510 0x7fd70c1078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.694+0000 7fd713ad4640 1 -- 172.21.15.2:0/1907105491 >> 172.21.15.2:0/1907105491 conn(0x7fd70c076920 msgr2=0x7fd70c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:18.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.694+0000 7fd713ad4640 1 -- 172.21.15.2:0/1907105491 shutdown_connections 2024-03-15T12:59:18.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.694+0000 7fd713ad4640 1 -- 172.21.15.2:0/1907105491 wait complete. 2024-03-15T12:59:18.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.695+0000 7fd713ad4640 1 Processor -- start 2024-03-15T12:59:18.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.695+0000 7fd713ad4640 1 -- start start 2024-03-15T12:59:18.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.695+0000 7fd713ad4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd70c0ff9f0 0x7fd70c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.695+0000 7fd713ad4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd70c1003b0 0x7fd70c19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.695+0000 7fd713ad4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd70c107510 0x7fd70c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.695+0000 7fd713ad4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c197ea0 con 0x7fd70c107510 2024-03-15T12:59:18.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.695+0000 7fd713ad4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c198010 con 0x7fd70c1003b0 2024-03-15T12:59:18.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.695+0000 7fd713ad4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c1981d0 con 0x7fd70c0ff9f0 2024-03-15T12:59:18.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.695+0000 7fd712071640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd70c107510 0x7fd70c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.695+0000 7fd712071640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd70c107510 0x7fd70c197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41124/0 (socket says 172.21.15.2:41124) 2024-03-15T12:59:18.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.695+0000 7fd712071640 1 -- 172.21.15.2:0/877241779 learned_addr learned my addr 172.21.15.2:0/877241779 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:18.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.695+0000 7fd71106f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd70c1003b0 0x7fd70c19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.695+0000 7fd711870640 1 --2- 172.21.15.2:0/877241779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd70c0ff9f0 0x7fd70c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.696+0000 7fd712071640 1 -- 172.21.15.2:0/877241779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd70c0ff9f0 msgr2=0x7fd70c19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.696+0000 7fd712071640 1 --2- 172.21.15.2:0/877241779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd70c0ff9f0 0x7fd70c19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.696+0000 7fd712071640 1 -- 172.21.15.2:0/877241779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd70c1003b0 msgr2=0x7fd70c19dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.696+0000 7fd712071640 1 --2- 172.21.15.2:0/877241779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd70c1003b0 0x7fd70c19dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.696+0000 7fd712071640 1 -- 172.21.15.2:0/877241779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd704009900 con 0x7fd70c107510 2024-03-15T12:59:18.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.696+0000 7fd71106f640 1 --2- 172.21.15.2:0/877241779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd70c1003b0 0x7fd70c19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:18.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.696+0000 7fd712071640 1 --2- 172.21.15.2:0/877241779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd70c107510 0x7fd70c197800 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd704005cc0 tx=0x7fd704034a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:18.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.696+0000 7fd702ffd640 1 -- 172.21.15.2:0/877241779 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704032880 con 0x7fd70c107510 2024-03-15T12:59:18.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.696+0000 7fd713ad4640 1 -- 172.21.15.2:0/877241779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70c198450 con 0x7fd70c107510 2024-03-15T12:59:18.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.696+0000 7fd702ffd640 1 -- 172.21.15.2:0/877241779 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd704034e80 con 0x7fd70c107510 2024-03-15T12:59:18.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.696+0000 7fd702ffd640 1 -- 172.21.15.2:0/877241779 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7040047c0 con 0x7fd70c107510 2024-03-15T12:59:18.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.696+0000 7fd713ad4640 1 -- 172.21.15.2:0/877241779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70c1a46c0 con 0x7fd70c107510 2024-03-15T12:59:18.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.698+0000 7fd702ffd640 1 -- 172.21.15.2:0/877241779 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7fd70400a040 con 0x7fd70c107510 2024-03-15T12:59:18.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd716790740 1 Processor -- start 2024-03-15T12:59:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd702ffd640 1 --2- 172.21.15.2:0/877241779 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd6f80a1cc0 0x7fd6f80a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd702ffd640 1 -- 172.21.15.2:0/877241779 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fd7040f08f0 con 0x7fd70c107510 2024-03-15T12:59:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd71106f640 1 --2- 172.21.15.2:0/877241779 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd6f80a1cc0 0x7fd6f80a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd716790740 1 -- start start 2024-03-15T12:59:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd716790740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f0e2b0 0x559979f0e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd716790740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0ebd0 0x559979f17270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd716790740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f177b0 0x559979f19ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd716790740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979e7d280 con 0x559979f0ebd0 2024-03-15T12:59:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd716790740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979e7a7b0 con 0x559979f177b0 2024-03-15T12:59:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd716790740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979e75f90 con 0x559979f0e2b0 2024-03-15T12:59:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd71106f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0ebd0 0x559979f17270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd71106f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0ebd0 0x559979f17270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41128/0 (socket says 172.21.15.2:41128) 2024-03-15T12:59:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd71106f640 1 -- 172.21.15.2:0/2103636627 learned_addr learned my addr 172.21.15.2:0/2103636627 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:18.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd711870640 1 --2- 172.21.15.2:0/2103636627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f0e2b0 0x559979f0e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd712071640 1 --2- 172.21.15.2:0/2103636627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f177b0 0x559979f19ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd71106f640 1 --2- 172.21.15.2:0/877241779 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd6f80a1cc0 0x7fd6f80a4180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd70c106b90 tx=0x7fd6fc011860 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd71106f640 1 -- 172.21.15.2:0/2103636627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f0e2b0 msgr2=0x559979f0e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd71106f640 1 --2- 172.21.15.2:0/2103636627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f0e2b0 0x559979f0e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd71106f640 1 -- 172.21.15.2:0/2103636627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f177b0 msgr2=0x559979f19ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd71106f640 1 --2- 172.21.15.2:0/2103636627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f177b0 0x559979f19ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.699+0000 7fd71106f640 1 -- 172.21.15.2:0/2103636627 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559979e34570 con 0x559979f0ebd0 2024-03-15T12:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.700+0000 7fd71106f640 1 --2- 172.21.15.2:0/2103636627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0ebd0 0x559979f17270 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0043e0 tx=0x7fd6fc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783c31468e9f1574 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.700+0000 7fd6f7fff640 1 -- 172.21.15.2:0/2103636627 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc003d70 con 0x559979f0ebd0 2024-03-15T12:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.700+0000 7fd6f7fff640 1 -- 172.21.15.2:0/2103636627 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd6fc003ed0 con 0x559979f0ebd0 2024-03-15T12:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.700+0000 7fd6f7fff640 1 -- 172.21.15.2:0/2103636627 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc01d840 con 0x559979f0ebd0 2024-03-15T12:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.700+0000 7fd716790740 1 -- 172.21.15.2:0/2103636627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0ebd0 msgr2=0x559979f17270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.700+0000 7fd716790740 1 --2- 172.21.15.2:0/2103636627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0ebd0 0x559979f17270 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0043e0 tx=0x7fd6fc01c000 comp rx=0 tx=0).stop 2024-03-15T12:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 1 -- 172.21.15.2:0/2103636627 shutdown_connections 2024-03-15T12:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 1 --2- 172.21.15.2:0/2103636627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f177b0 0x559979f19ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 1 --2- 172.21.15.2:0/2103636627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0ebd0 0x559979f17270 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 1 --2- 172.21.15.2:0/2103636627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f0e2b0 0x559979f0e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 1 -- 172.21.15.2:0/2103636627 >> 172.21.15.2:0/2103636627 conn(0x559979f08990 msgr2=0x559979f08d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:18.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 1 -- 172.21.15.2:0/2103636627 shutdown_connections 2024-03-15T12:59:18.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 1 -- 172.21.15.2:0/2103636627 wait complete. 2024-03-15T12:59:18.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 1 -- start start 2024-03-15T12:59:18.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 1 Processor -- start 2024-03-15T12:59:18.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f0e2b0 0x559979f17260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f0ebd0 0x559979f145f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f177b0 0x559979f14d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979b49430 con 0x559979f177b0 2024-03-15T12:59:18.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979b05e00 con 0x559979f0ebd0 2024-03-15T12:59:18.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979dfafd0 con 0x559979f0e2b0 2024-03-15T12:59:18.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.701+0000 7fd716790740 10 client.0 _ll_register_callbacks cb 0x559979e50fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:18.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd711870640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f0e2b0 0x559979f17260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd712071640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f177b0 0x559979f14d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd711870640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f0e2b0 0x559979f17260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:33980/0 (socket says 172.21.15.2:33980) 2024-03-15T12:59:18.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd71106f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f0ebd0 0x559979f145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd711870640 1 -- 172.21.15.2:0/2129290394 learned_addr learned my addr 172.21.15.2:0/2129290394 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:18.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd712071640 1 -- 172.21.15.2:0/2129290394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f0e2b0 msgr2=0x559979f17260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd712071640 1 --2- 172.21.15.2:0/2129290394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f0e2b0 0x559979f17260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd712071640 1 -- 172.21.15.2:0/2129290394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f0ebd0 msgr2=0x559979f145f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd712071640 1 --2- 172.21.15.2:0/2129290394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f0ebd0 0x559979f145f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd712071640 1 -- 172.21.15.2:0/2129290394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6fc0041a0 con 0x559979f177b0 2024-03-15T12:59:18.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd712071640 1 --2- 172.21.15.2:0/2129290394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f177b0 0x559979f14d30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd7040edc80 tx=0x7fd7040bcab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6dd1929dfd3ec5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:18.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd6f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:18.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd6f57fa640 1 -- 172.21.15.2:0/2129290394 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7040bd4f0 con 0x559979f177b0 2024-03-15T12:59:18.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd6f57fa640 1 -- 172.21.15.2:0/2129290394 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fd7040c5830 con 0x559979f177b0 2024-03-15T12:59:18.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd716790740 1 -- 172.21.15.2:0/2129290394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559979b757e0 con 0x559979f177b0 2024-03-15T12:59:18.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd6f57fa640 1 -- 172.21.15.2:0/2129290394 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7040db3c0 con 0x559979f177b0 2024-03-15T12:59:18.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd716790740 10 client.15972 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:18.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd716790740 1 -- 172.21.15.2:0/2129290394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559979e74610 con 0x559979f177b0 2024-03-15T12:59:18.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.702+0000 7fd6f57fa640 1 -- 172.21.15.2:0/2129290394 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x559979b757e0 con 0x559979f177b0 2024-03-15T12:59:18.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.703+0000 7fd6f57fa640 1 -- 172.21.15.2:0/2129290394 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd7040ede50 con 0x559979f177b0 2024-03-15T12:59:18.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.703+0000 7fd716790740 10 client.15972 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:18.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.703+0000 7fd716790740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:18.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.703+0000 7fd716790740 20 client.15972 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:18.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.703+0000 7fd716790740 1 --2- 172.21.15.2:0/2129290394 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559979f277d0 0x559979f29bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.703+0000 7fd716790740 4 client.15972 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:18.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.703+0000 7fd716790740 1 -- 172.21.15.2:0/2129290394 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559979bb2720 con 0x559979f277d0 2024-03-15T12:59:18.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.703+0000 7fd711870640 1 --2- 172.21.15.2:0/2129290394 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559979f277d0 0x559979f29bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.703+0000 7fd711870640 1 --2- 172.21.15.2:0/2129290394 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559979f277d0 0x559979f29bf0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e6b12a4958040bd server_cookie=6511f705689ae81 in_seq=0 out_seq=0 2024-03-15T12:59:18.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.703+0000 7fd6f57fa640 10 client.15972 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:18.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.704+0000 7fd6f57fa640 1 -- 172.21.15.2:0/2129290394 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559979bb2720 con 0x559979f277d0 2024-03-15T12:59:18.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.704+0000 7fd6f57fa640 10 client.15972 handle_command_reply: tid=0 2024-03-15T12:59:18.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.704+0000 7fd716790740 1 client.15972 shutdown 2024-03-15T12:59:18.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.704+0000 7fd716790740 1 -- 172.21.15.2:0/2129290394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f177b0 msgr2=0x559979f14d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.704+0000 7fd716790740 1 --2- 172.21.15.2:0/2129290394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f177b0 0x559979f14d30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd7040edc80 tx=0x7fd7040bcab0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.704+0000 7fd716790740 1 -- 172.21.15.2:0/2129290394 shutdown_connections 2024-03-15T12:59:18.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.705+0000 7fd716790740 1 --2- 172.21.15.2:0/2129290394 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559979f277d0 0x559979f29bf0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.705+0000 7fd716790740 1 --2- 172.21.15.2:0/2129290394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f177b0 0x559979f14d30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.705+0000 7fd716790740 1 --2- 172.21.15.2:0/2129290394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f0ebd0 0x559979f145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.705+0000 7fd716790740 1 --2- 172.21.15.2:0/2129290394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f0e2b0 0x559979f17260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.705+0000 7fd6f57fa640 0 client.15972 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:18.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.705+0000 7fd716790740 1 -- 172.21.15.2:0/2129290394 >> 172.21.15.2:0/2129290394 conn(0x559979f08990 msgr2=0x559979f0a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:18.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.705+0000 7fd716790740 1 -- 172.21.15.2:0/2129290394 shutdown_connections 2024-03-15T12:59:18.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.705+0000 7fd716790740 1 -- 172.21.15.2:0/2129290394 wait complete. 2024-03-15T12:59:18.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.705+0000 7fd716790740 20 client.15972 trim_cache size 0 max 16384 2024-03-15T12:59:18.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd716790740 1 Processor -- start 2024-03-15T12:59:18.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd716790740 1 -- start start 2024-03-15T12:59:18.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd716790740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f0e2b0 0x559979f16d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd716790740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0ebd0 0x559979f17260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd716790740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f177b0 0x559979f22b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd716790740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979bb2720 con 0x559979f0ebd0 2024-03-15T12:59:18.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd716790740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979b757e0 con 0x559979f0e2b0 2024-03-15T12:59:18.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd716790740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979f040f0 con 0x559979f177b0 2024-03-15T12:59:18.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd71106f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0ebd0 0x559979f17260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd71106f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0ebd0 0x559979f17260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41144/0 (socket says 172.21.15.2:41144) 2024-03-15T12:59:18.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd71106f640 1 -- 172.21.15.2:0/1918778146 learned_addr learned my addr 172.21.15.2:0/1918778146 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:18.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd712071640 1 --2- 172.21.15.2:0/1918778146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f177b0 0x559979f22b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd711870640 1 --2- 172.21.15.2:0/1918778146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f0e2b0 0x559979f16d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd71106f640 1 -- 172.21.15.2:0/1918778146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f177b0 msgr2=0x559979f22b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd71106f640 1 --2- 172.21.15.2:0/1918778146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f177b0 0x559979f22b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd71106f640 1 -- 172.21.15.2:0/1918778146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f0e2b0 msgr2=0x559979f16d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd71106f640 1 --2- 172.21.15.2:0/1918778146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f0e2b0 0x559979f16d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd71106f640 1 -- 172.21.15.2:0/1918778146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559979e10ac0 con 0x559979f0ebd0 2024-03-15T12:59:18.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.721+0000 7fd71106f640 1 --2- 172.21.15.2:0/1918778146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0ebd0 0x559979f17260 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0058c0 tx=0x7fd6fc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c52ce006f7485c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:18.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.722+0000 7fd6f77fe640 1 -- 172.21.15.2:0/1918778146 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc02c030 con 0x559979f0ebd0 2024-03-15T12:59:18.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.722+0000 7fd6f77fe640 1 -- 172.21.15.2:0/1918778146 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd6fc02d040 con 0x559979f0ebd0 2024-03-15T12:59:18.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.722+0000 7fd6f77fe640 1 -- 172.21.15.2:0/1918778146 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc004af0 con 0x559979f0ebd0 2024-03-15T12:59:18.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.722+0000 7fd716790740 1 -- 172.21.15.2:0/1918778146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0ebd0 msgr2=0x559979f17260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.722+0000 7fd716790740 1 --2- 172.21.15.2:0/1918778146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0ebd0 0x559979f17260 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0058c0 tx=0x7fd6fc005850 comp rx=0 tx=0).stop 2024-03-15T12:59:18.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.722+0000 7fd716790740 1 -- 172.21.15.2:0/1918778146 shutdown_connections 2024-03-15T12:59:18.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.722+0000 7fd716790740 1 --2- 172.21.15.2:0/1918778146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f177b0 0x559979f22b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.722+0000 7fd716790740 1 --2- 172.21.15.2:0/1918778146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0ebd0 0x559979f17260 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.722+0000 7fd716790740 1 --2- 172.21.15.2:0/1918778146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f0e2b0 0x559979f16d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.722+0000 7fd716790740 1 -- 172.21.15.2:0/1918778146 >> 172.21.15.2:0/1918778146 conn(0x559979f08990 msgr2=0x559979f0f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:18.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.722+0000 7fd716790740 1 -- 172.21.15.2:0/1918778146 shutdown_connections 2024-03-15T12:59:18.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.722+0000 7fd716790740 1 -- 172.21.15.2:0/1918778146 wait complete. 2024-03-15T12:59:18.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd716790740 1 -- start start 2024-03-15T12:59:18.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd716790740 1 Processor -- start 2024-03-15T12:59:18.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd716790740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0e2b0 0x559979f12270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd716790740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f0ebd0 0x559979f1c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd716790740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f177b0 0x559979f1cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd716790740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979f1d190 con 0x559979f0e2b0 2024-03-15T12:59:18.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd716790740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979f1d300 con 0x559979f0ebd0 2024-03-15T12:59:18.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd716790740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559979f1d4c0 con 0x559979f177b0 2024-03-15T12:59:18.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd716790740 10 client.0 _ll_register_callbacks cb 0x7fd7040b9330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:18.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd711870640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0e2b0 0x559979f12270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd711870640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0e2b0 0x559979f12270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41154/0 (socket says 172.21.15.2:41154) 2024-03-15T12:59:18.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd711870640 1 -- 172.21.15.2:0/2974745886 learned_addr learned my addr 172.21.15.2:0/2974745886 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:18.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd712071640 1 --2- 172.21.15.2:0/2974745886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f177b0 0x559979f1cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd71106f640 1 --2- 172.21.15.2:0/2974745886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f0ebd0 0x559979f1c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd711870640 1 -- 172.21.15.2:0/2974745886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f177b0 msgr2=0x559979f1cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd711870640 1 --2- 172.21.15.2:0/2974745886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f177b0 0x559979f1cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd711870640 1 -- 172.21.15.2:0/2974745886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f0ebd0 msgr2=0x559979f1c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:18.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd711870640 1 --2- 172.21.15.2:0/2974745886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f0ebd0 0x559979f1c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.723+0000 7fd711870640 1 -- 172.21.15.2:0/2974745886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559979e34570 con 0x559979f0e2b0 2024-03-15T12:59:18.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd711870640 1 --2- 172.21.15.2:0/2974745886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0e2b0 0x559979f12270 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x559979f2a430 tx=0x7fd708020c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db013be07002a187 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:18.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd6f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:18.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd6f57fa640 1 -- 172.21.15.2:0/2974745886 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708010960 con 0x559979f0e2b0 2024-03-15T12:59:18.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd6f57fa640 1 -- 172.21.15.2:0/2974745886 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fd708010ac0 con 0x559979f0e2b0 2024-03-15T12:59:18.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd716790740 1 -- 172.21.15.2:0/2974745886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559979f14420 con 0x559979f0e2b0 2024-03-15T12:59:18.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd6f57fa640 1 -- 172.21.15.2:0/2974745886 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd708010c80 con 0x559979f0e2b0 2024-03-15T12:59:18.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd716790740 10 client.15978 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:18.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd716790740 1 -- 172.21.15.2:0/2974745886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559979e366a0 con 0x559979f0e2b0 2024-03-15T12:59:18.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd6f57fa640 1 -- 172.21.15.2:0/2974745886 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x559979f14420 con 0x559979f0e2b0 2024-03-15T12:59:18.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd6f57fa640 1 -- 172.21.15.2:0/2974745886 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd70801f400 con 0x559979f0e2b0 2024-03-15T12:59:18.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd716790740 10 client.15978 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:18.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd716790740 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:18.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd716790740 20 client.15978 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:18.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd716790740 1 --2- 172.21.15.2:0/2974745886 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559979f84740 0x559979f86b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:18.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd716790740 4 client.15978 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T12:59:18.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.724+0000 7fd716790740 1 -- 172.21.15.2:0/2974745886 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fd6fc003ed0 con 0x559979f84740 2024-03-15T12:59:18.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd71106f640 1 --2- 172.21.15.2:0/2974745886 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559979f84740 0x559979f86b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:18.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd6f57fa640 10 client.15978 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:18.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd71106f640 1 --2- 172.21.15.2:0/2974745886 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559979f84740 0x559979f86b60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ba3793e844d5bc0 server_cookie=8a14a46efa1eae44 in_seq=0 out_seq=0 2024-03-15T12:59:18.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd6f57fa640 1 -- 172.21.15.2:0/2974745886 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7fd6fc003ed0 con 0x559979f84740 2024-03-15T12:59:18.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd6f57fa640 10 client.15978 handle_command_reply: tid=0 2024-03-15T12:59:18.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd716790740 1 client.15978 shutdown 2024-03-15T12:59:18.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd716790740 1 -- 172.21.15.2:0/2974745886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0e2b0 msgr2=0x559979f12270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd716790740 1 --2- 172.21.15.2:0/2974745886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0e2b0 0x559979f12270 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x559979f2a430 tx=0x7fd708020c00 comp rx=0 tx=0).stop 2024-03-15T12:59:18.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd716790740 1 -- 172.21.15.2:0/2974745886 shutdown_connections 2024-03-15T12:59:18.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd716790740 1 --2- 172.21.15.2:0/2974745886 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559979f84740 0x559979f86b60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd716790740 1 --2- 172.21.15.2:0/2974745886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559979f177b0 0x559979f1cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd6f57fa640 0 client.15978 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:18.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd716790740 1 --2- 172.21.15.2:0/2974745886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559979f0ebd0 0x559979f1c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd716790740 1 --2- 172.21.15.2:0/2974745886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559979f0e2b0 0x559979f12270 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd716790740 1 -- 172.21.15.2:0/2974745886 >> 172.21.15.2:0/2974745886 conn(0x559979f08990 msgr2=0x559979f071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:18.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd716790740 1 -- 172.21.15.2:0/2974745886 shutdown_connections 2024-03-15T12:59:18.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd716790740 1 -- 172.21.15.2:0/2974745886 wait complete. 2024-03-15T12:59:18.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.725+0000 7fd716790740 20 client.15978 trim_cache size 0 max 16384 2024-03-15T12:59:18.718 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:59:18.718 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T12:59:18.718 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T12:59:18.718 INFO:teuthology.orchestra.run.smithi002.stdout: "e1876b4f-2020-42e1-9b9e-ed23aa67b7dc": { 2024-03-15T12:59:18.718 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T12:59:18.718 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "e1876b4f-2020-42e1-9b9e-ed23aa67b7dc", 2024-03-15T12:59:18.718 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T12:59:18.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T12:59:18.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T12:59:18.718 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:59:18.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.726+0000 7fd713ad4640 1 -- 172.21.15.2:0/877241779 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd6f80a1cc0 msgr2=0x7fd6f80a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.726+0000 7fd713ad4640 1 --2- 172.21.15.2:0/877241779 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd6f80a1cc0 0x7fd6f80a4180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd70c106b90 tx=0x7fd6fc011860 comp rx=0 tx=0).stop 2024-03-15T12:59:18.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.726+0000 7fd713ad4640 1 -- 172.21.15.2:0/877241779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd70c107510 msgr2=0x7fd70c197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:18.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.726+0000 7fd713ad4640 1 --2- 172.21.15.2:0/877241779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd70c107510 0x7fd70c197800 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd704005cc0 tx=0x7fd704034a80 comp rx=0 tx=0).stop 2024-03-15T12:59:18.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.726+0000 7fd713ad4640 1 -- 172.21.15.2:0/877241779 shutdown_connections 2024-03-15T12:59:18.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.726+0000 7fd713ad4640 1 --2- 172.21.15.2:0/877241779 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd6f80a1cc0 0x7fd6f80a4180 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.726+0000 7fd713ad4640 1 --2- 172.21.15.2:0/877241779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd70c107510 0x7fd70c197800 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.726+0000 7fd713ad4640 1 --2- 172.21.15.2:0/877241779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd70c1003b0 0x7fd70c19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.726+0000 7fd713ad4640 1 --2- 172.21.15.2:0/877241779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd70c0ff9f0 0x7fd70c19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:18.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.726+0000 7fd713ad4640 1 -- 172.21.15.2:0/877241779 >> 172.21.15.2:0/877241779 conn(0x7fd70c076920 msgr2=0x7fd70c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:18.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.726+0000 7fd713ad4640 1 -- 172.21.15.2:0/877241779 shutdown_connections 2024-03-15T12:59:18.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:18.726+0000 7fd713ad4640 1 -- 172.21.15.2:0/877241779 wait complete. 2024-03-15T12:59:18.722 INFO:tasks.cephfs.filesystem:scrub status for tag:e1876b4f-2020-42e1-9b9e-ed23aa67b7dc - {'path': '/', 'tag': 'e1876b4f-2020-42e1-9b9e-ed23aa67b7dc', 'options': 'recursive,force'} 2024-03-15T12:59:19.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:19 smithi149 ceph-mon[35927]: pgmap v15: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-15T12:59:19.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:19 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T12:59:19.860 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:19 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T12:59:19.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:19 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T12:59:19.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:19 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T12:59:19.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:19 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T12:59:20.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:19 smithi002 ceph-mon[32060]: pgmap v15: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-15T12:59:20.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:19 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T12:59:20.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:19 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T12:59:20.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:19 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T12:59:20.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:19 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T12:59:20.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:19 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T12:59:20.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:19 smithi184 ceph-mon[35993]: pgmap v15: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-15T12:59:20.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:19 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T12:59:20.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:19 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T12:59:20.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:19 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T12:59:20.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:19 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T12:59:20.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:19 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T12:59:22.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:21 smithi002 ceph-mon[32060]: pgmap v16: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-15T12:59:22.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:21 smithi184 ceph-mon[35993]: pgmap v16: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-15T12:59:22.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:21 smithi149 ceph-mon[35927]: pgmap v16: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-15T12:59:24.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:23 smithi002 ceph-mon[32060]: pgmap v17: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-15T12:59:24.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:23 smithi184 ceph-mon[35993]: pgmap v17: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-15T12:59:24.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:23 smithi149 ceph-mon[35927]: pgmap v17: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 12 op/s 2024-03-15T12:59:26.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:25 smithi002 ceph-mon[32060]: pgmap v18: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T12:59:26.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:25 smithi184 ceph-mon[35993]: pgmap v18: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T12:59:26.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:25 smithi149 ceph-mon[35927]: pgmap v18: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T12:59:28.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:27 smithi002 ceph-mon[32060]: pgmap v19: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T12:59:28.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:27 smithi184 ceph-mon[35993]: pgmap v19: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T12:59:28.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:27 smithi149 ceph-mon[35927]: pgmap v19: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T12:59:30.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:29 smithi002 ceph-mon[32060]: pgmap v20: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T12:59:30.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:29 smithi184 ceph-mon[35993]: pgmap v20: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T12:59:30.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:29 smithi149 ceph-mon[35927]: pgmap v20: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T12:59:31.911 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:31 smithi149 ceph-mon[35927]: pgmap v21: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T12:59:32.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:31 smithi002 ceph-mon[32060]: pgmap v21: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T12:59:32.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:31 smithi184 ceph-mon[35993]: pgmap v21: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T12:59:34.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:33 smithi002 ceph-mon[32060]: pgmap v22: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-03-15T12:59:34.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:33 smithi184 ceph-mon[35993]: pgmap v22: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-03-15T12:59:34.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:33 smithi149 ceph-mon[35927]: pgmap v22: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-03-15T12:59:36.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:35 smithi002 ceph-mon[32060]: pgmap v23: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:36.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:35 smithi184 ceph-mon[35993]: pgmap v23: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:36.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:35 smithi149 ceph-mon[35927]: pgmap v23: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:38.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:37 smithi002 ceph-mon[32060]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T12:59:38.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:37 smithi184 ceph-mon[35993]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T12:59:38.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:37 smithi149 ceph-mon[35927]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T12:59:40.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:39 smithi002 ceph-mon[32060]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:40.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:39 smithi184 ceph-mon[35993]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:40.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:39 smithi149 ceph-mon[35927]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:42.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:41 smithi002 ceph-mon[32060]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T12:59:42.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:41 smithi184 ceph-mon[35993]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T12:59:42.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:41 smithi149 ceph-mon[35927]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T12:59:44.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:43 smithi002 ceph-mon[32060]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:44.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:43 smithi184 ceph-mon[35993]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:44.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:43 smithi149 ceph-mon[35927]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:46.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:45 smithi002 ceph-mon[32060]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:46.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:45 smithi184 ceph-mon[35993]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:46.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:45 smithi149 ceph-mon[35927]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:48.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:47 smithi002 ceph-mon[32060]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T12:59:48.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:47 smithi184 ceph-mon[35993]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T12:59:48.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:47 smithi149 ceph-mon[35927]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T12:59:48.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T12:59:48.724 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T12:59:48.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.871+0000 7f52e4d4b640 1 -- 172.21.15.2:0/3855684322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e0102710 msgr2=0x7f52e0102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:48.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.871+0000 7f52e4d4b640 1 --2- 172.21.15.2:0/3855684322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e0102710 0x7f52e0102af0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009860 tx=0x7f52d002bc00 comp rx=0 tx=0).stop 2024-03-15T12:59:48.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.872+0000 7f52e4d4b640 1 -- 172.21.15.2:0/3855684322 shutdown_connections 2024-03-15T12:59:48.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.872+0000 7f52e4d4b640 1 --2- 172.21.15.2:0/3855684322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52e010ba50 0x7f52e010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.872+0000 7f52e4d4b640 1 --2- 172.21.15.2:0/3855684322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e0103030 0x7f52e010b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.872+0000 7f52e4d4b640 1 --2- 172.21.15.2:0/3855684322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e0102710 0x7f52e0102af0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.872+0000 7f52e4d4b640 1 -- 172.21.15.2:0/3855684322 >> 172.21.15.2:0/3855684322 conn(0x7f52e00775a0 msgr2=0x7f52e00779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:48.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.872+0000 7f52e4d4b640 1 -- 172.21.15.2:0/3855684322 shutdown_connections 2024-03-15T12:59:48.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.872+0000 7f52e4d4b640 1 -- 172.21.15.2:0/3855684322 wait complete. 2024-03-15T12:59:48.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52e4d4b640 1 Processor -- start 2024-03-15T12:59:48.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52e4d4b640 1 -- start start 2024-03-15T12:59:48.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52e4d4b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e0102710 0x7f52e019d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52e4d4b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e0103030 0x7f52e019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52e4d4b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52e010ba50 0x7f52e0197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52e4d4b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e0197f90 con 0x7f52e010ba50 2024-03-15T12:59:48.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52e4d4b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e0198100 con 0x7f52e0102710 2024-03-15T12:59:48.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52e4d4b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e01982c0 con 0x7f52e0103030 2024-03-15T12:59:48.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52ded9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52e010ba50 0x7f52e0197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52de59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e0102710 0x7f52e019d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52ded9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52e010ba50 0x7f52e0197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42076/0 (socket says 172.21.15.2:42076) 2024-03-15T12:59:48.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52de59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e0102710 0x7f52e019d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39216/0 (socket says 172.21.15.2:39216) 2024-03-15T12:59:48.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52ded9d640 1 -- 172.21.15.2:0/2352546218 learned_addr learned my addr 172.21.15.2:0/2352546218 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:48.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52ddd9b640 1 --2- 172.21.15.2:0/2352546218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e0103030 0x7f52e019dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52de59c640 1 -- 172.21.15.2:0/2352546218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e0103030 msgr2=0x7f52e019dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:48.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52de59c640 1 --2- 172.21.15.2:0/2352546218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e0103030 0x7f52e019dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52de59c640 1 -- 172.21.15.2:0/2352546218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52e010ba50 msgr2=0x7f52e0197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:48.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52de59c640 1 --2- 172.21.15.2:0/2352546218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52e010ba50 0x7f52e0197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52de59c640 1 -- 172.21.15.2:0/2352546218 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d8009900 con 0x7f52e0102710 2024-03-15T12:59:48.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52ded9d640 1 --2- 172.21.15.2:0/2352546218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52e010ba50 0x7f52e0197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:48.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.873+0000 7f52ddd9b640 1 --2- 172.21.15.2:0/2352546218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e0103030 0x7f52e019dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:48.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.874+0000 7f52de59c640 1 --2- 172.21.15.2:0/2352546218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e0102710 0x7f52e019d790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f52d0037040 tx=0x7f52d00346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:48.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.874+0000 7f52cf7fe640 1 -- 172.21.15.2:0/2352546218 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d002c7a0 con 0x7f52e0102710 2024-03-15T12:59:48.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.874+0000 7f52e4d4b640 1 -- 172.21.15.2:0/2352546218 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d0009490 con 0x7f52e0102710 2024-03-15T12:59:48.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.874+0000 7f52cf7fe640 1 -- 172.21.15.2:0/2352546218 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f52d00035f0 con 0x7f52e0102710 2024-03-15T12:59:48.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.874+0000 7f52cf7fe640 1 -- 172.21.15.2:0/2352546218 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00406b0 con 0x7f52e0102710 2024-03-15T12:59:48.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.874+0000 7f52e4d4b640 1 -- 172.21.15.2:0/2352546218 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e01a4650 con 0x7f52e0102710 2024-03-15T12:59:48.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.876+0000 7f52cf7fe640 1 -- 172.21.15.2:0/2352546218 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7f52d0003760 con 0x7f52e0102710 2024-03-15T12:59:48.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.877+0000 7f52cf7fe640 1 --2- 172.21.15.2:0/2352546218 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f52c80a1cc0 0x7f52c80a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.877+0000 7f52ddd9b640 1 --2- 172.21.15.2:0/2352546218 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f52c80a1cc0 0x7f52c80a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.877+0000 7f52cf7fe640 1 -- 172.21.15.2:0/2352546218 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f52d00efb70 con 0x7f52e0102710 2024-03-15T12:59:48.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.877+0000 7f52e79fd740 1 Processor -- start 2024-03-15T12:59:48.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.877+0000 7f52e79fd740 1 -- start start 2024-03-15T12:59:48.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52ddd9b640 1 --2- 172.21.15.2:0/2352546218 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f52c80a1cc0 0x7f52c80a4180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f52e010d980 tx=0x7f52d4005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:48.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52e79fd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d56c0 0x55e04f8d5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52e79fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8d5fe0 0x55e04f8de680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52e79fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8debc0 0x55e04f8e0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52e79fd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f840280 con 0x55e04f8d56c0 2024-03-15T12:59:48.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52e79fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f83d7b0 con 0x55e04f8d5fe0 2024-03-15T12:59:48.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52e79fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f838f90 con 0x55e04f8debc0 2024-03-15T12:59:48.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52de59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d56c0 0x55e04f8d5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52de59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d56c0 0x55e04f8d5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42082/0 (socket says 172.21.15.2:42082) 2024-03-15T12:59:48.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52de59c640 1 -- 172.21.15.2:0/911275232 learned_addr learned my addr 172.21.15.2:0/911275232 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:48.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52ddd9b640 1 --2- 172.21.15.2:0/911275232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8d5fe0 0x55e04f8de680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52ded9d640 1 --2- 172.21.15.2:0/911275232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8debc0 0x55e04f8e0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52de59c640 1 -- 172.21.15.2:0/911275232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8debc0 msgr2=0x55e04f8e0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:48.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52de59c640 1 --2- 172.21.15.2:0/911275232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8debc0 0x55e04f8e0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52de59c640 1 -- 172.21.15.2:0/911275232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8d5fe0 msgr2=0x55e04f8de680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:48.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52de59c640 1 --2- 172.21.15.2:0/911275232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8d5fe0 0x55e04f8de680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52de59c640 1 -- 172.21.15.2:0/911275232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e04f7f7570 con 0x55e04f8d56c0 2024-03-15T12:59:48.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.878+0000 7f52de59c640 1 --2- 172.21.15.2:0/911275232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d56c0 0x55e04f8d5aa0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f52d0000c00 tx=0x7f52d00bbfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139fac8861720ebd server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:48.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.879+0000 7f52ccff9640 1 -- 172.21.15.2:0/911275232 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00c4ab0 con 0x55e04f8d56c0 2024-03-15T12:59:48.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.879+0000 7f52ccff9640 1 -- 172.21.15.2:0/911275232 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f52d0003b40 con 0x55e04f8d56c0 2024-03-15T12:59:48.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.879+0000 7f52ccff9640 1 -- 172.21.15.2:0/911275232 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00c4620 con 0x55e04f8d56c0 2024-03-15T12:59:48.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.879+0000 7f52e79fd740 1 -- 172.21.15.2:0/911275232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d56c0 msgr2=0x55e04f8d5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:48.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.879+0000 7f52e79fd740 1 --2- 172.21.15.2:0/911275232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d56c0 0x55e04f8d5aa0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f52d0000c00 tx=0x7f52d00bbfc0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.879+0000 7f52e79fd740 1 -- 172.21.15.2:0/911275232 shutdown_connections 2024-03-15T12:59:48.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.879+0000 7f52e79fd740 1 --2- 172.21.15.2:0/911275232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8debc0 0x55e04f8e0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.879+0000 7f52e79fd740 1 --2- 172.21.15.2:0/911275232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8d5fe0 0x55e04f8de680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.879+0000 7f52e79fd740 1 --2- 172.21.15.2:0/911275232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d56c0 0x55e04f8d5aa0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.879+0000 7f52e79fd740 1 -- 172.21.15.2:0/911275232 >> 172.21.15.2:0/911275232 conn(0x55e04f8cfda0 msgr2=0x55e04f8d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:48.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52e79fd740 1 -- 172.21.15.2:0/911275232 shutdown_connections 2024-03-15T12:59:48.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52e79fd740 1 -- 172.21.15.2:0/911275232 wait complete. 2024-03-15T12:59:48.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52e79fd740 1 -- start start 2024-03-15T12:59:48.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52e79fd740 1 Processor -- start 2024-03-15T12:59:48.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52e79fd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d56c0 0x55e04f8dba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52e79fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8d5fe0 0x55e04f8dbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52e79fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8debc0 0x55e04f8dc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52e79fd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f50c430 con 0x55e04f8d56c0 2024-03-15T12:59:48.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52e79fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f4c8e00 con 0x55e04f8d5fe0 2024-03-15T12:59:48.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52e79fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f7bdfd0 con 0x55e04f8debc0 2024-03-15T12:59:48.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52de59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d56c0 0x55e04f8dba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52de59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d56c0 0x55e04f8dba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42088/0 (socket says 172.21.15.2:42088) 2024-03-15T12:59:48.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52de59c640 1 -- 172.21.15.2:0/3661973177 learned_addr learned my addr 172.21.15.2:0/3661973177 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:48.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52e79fd740 10 client.0 _ll_register_callbacks cb 0x55e04f813fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:48.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52ddd9b640 1 --2- 172.21.15.2:0/3661973177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8d5fe0 0x55e04f8dbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.880+0000 7f52ded9d640 1 --2- 172.21.15.2:0/3661973177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8debc0 0x55e04f8dc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.881+0000 7f52ded9d640 1 -- 172.21.15.2:0/3661973177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8d5fe0 msgr2=0x55e04f8dbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:48.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.881+0000 7f52ded9d640 1 --2- 172.21.15.2:0/3661973177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8d5fe0 0x55e04f8dbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.881+0000 7f52ded9d640 1 -- 172.21.15.2:0/3661973177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d56c0 msgr2=0x55e04f8dba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:48.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.881+0000 7f52ded9d640 1 --2- 172.21.15.2:0/3661973177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d56c0 0x55e04f8dba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.881+0000 7f52ded9d640 1 -- 172.21.15.2:0/3661973177 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e01a4650 con 0x55e04f8debc0 2024-03-15T12:59:48.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.881+0000 7f52de59c640 1 --2- 172.21.15.2:0/3661973177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d56c0 0x55e04f8dba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:48.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.881+0000 7f52ded9d640 1 --2- 172.21.15.2:0/3661973177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8debc0 0x55e04f8dc4a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f52d8005c40 tx=0x7f52d800ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=306e534a7483e803 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:48.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.881+0000 7f52c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T12:59:48.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.881+0000 7f52c5ffb640 1 -- 172.21.15.2:0/3661973177 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d801f070 con 0x55e04f8debc0 2024-03-15T12:59:48.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.881+0000 7f52e79fd740 1 -- 172.21.15.2:0/3661973177 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e04f5387e0 con 0x55e04f8debc0 2024-03-15T12:59:48.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.881+0000 7f52c5ffb640 1 -- 172.21.15.2:0/3661973177 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f52d8003970 con 0x55e04f8debc0 2024-03-15T12:59:48.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.881+0000 7f52c5ffb640 1 -- 172.21.15.2:0/3661973177 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52d8003b30 con 0x55e04f8debc0 2024-03-15T12:59:48.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.881+0000 7f52e79fd740 10 client.25786 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:48.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.881+0000 7f52e79fd740 1 -- 172.21.15.2:0/3661973177 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e04f837610 con 0x55e04f8debc0 2024-03-15T12:59:48.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.882+0000 7f52c5ffb640 1 -- 172.21.15.2:0/3661973177 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e04f5387e0 con 0x55e04f8debc0 2024-03-15T12:59:48.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.882+0000 7f52c5ffb640 1 -- 172.21.15.2:0/3661973177 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f52d8022020 con 0x55e04f8debc0 2024-03-15T12:59:48.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.882+0000 7f52e79fd740 10 client.25786 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:48.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.882+0000 7f52e79fd740 10 client.25786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:48.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.882+0000 7f52e79fd740 20 client.25786 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:48.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.882+0000 7f52e79fd740 1 --2- 172.21.15.2:0/3661973177 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e04f8eeca0 0x55e04f8f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.882+0000 7f52e79fd740 4 client.25786 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:48.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.882+0000 7f52e79fd740 1 -- 172.21.15.2:0/3661973177 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e04f575720 con 0x55e04f8eeca0 2024-03-15T12:59:48.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.882+0000 7f52de59c640 1 --2- 172.21.15.2:0/3661973177 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e04f8eeca0 0x55e04f8f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.882+0000 7f52c5ffb640 10 client.25786 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:48.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.882+0000 7f52de59c640 1 --2- 172.21.15.2:0/3661973177 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e04f8eeca0 0x55e04f8f10c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ed4a0d6d7c63d14 server_cookie=902b7259681539cc in_seq=0 out_seq=0 2024-03-15T12:59:48.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.883+0000 7f52c5ffb640 1 -- 172.21.15.2:0/3661973177 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e04f575720 con 0x55e04f8eeca0 2024-03-15T12:59:48.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.883+0000 7f52c5ffb640 10 client.25786 handle_command_reply: tid=0 2024-03-15T12:59:48.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.883+0000 7f52e79fd740 1 client.25786 shutdown 2024-03-15T12:59:48.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.883+0000 7f52e79fd740 1 -- 172.21.15.2:0/3661973177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8debc0 msgr2=0x55e04f8dc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:48.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.883+0000 7f52e79fd740 1 --2- 172.21.15.2:0/3661973177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8debc0 0x55e04f8dc4a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f52d8005c40 tx=0x7f52d800ba50 comp rx=0 tx=0).stop 2024-03-15T12:59:48.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.883+0000 7f52e79fd740 1 -- 172.21.15.2:0/3661973177 shutdown_connections 2024-03-15T12:59:48.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.883+0000 7f52e79fd740 1 --2- 172.21.15.2:0/3661973177 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e04f8eeca0 0x55e04f8f10c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.883+0000 7f52e79fd740 1 --2- 172.21.15.2:0/3661973177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8debc0 0x55e04f8dc4a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.883+0000 7f52c5ffb640 0 client.25786 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:48.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.883+0000 7f52e79fd740 1 --2- 172.21.15.2:0/3661973177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8d5fe0 0x55e04f8dbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.883+0000 7f52e79fd740 1 --2- 172.21.15.2:0/3661973177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d56c0 0x55e04f8dba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.883+0000 7f52e79fd740 1 -- 172.21.15.2:0/3661973177 >> 172.21.15.2:0/3661973177 conn(0x55e04f8cfda0 msgr2=0x55e04f8d1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:48.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.883+0000 7f52e79fd740 1 -- 172.21.15.2:0/3661973177 shutdown_connections 2024-03-15T12:59:48.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.884+0000 7f52e79fd740 1 -- 172.21.15.2:0/3661973177 wait complete. 2024-03-15T12:59:48.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.884+0000 7f52e79fd740 20 client.25786 trim_cache size 0 max 16384 2024-03-15T12:59:48.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.915+0000 7f52e79fd740 1 Processor -- start 2024-03-15T12:59:48.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.915+0000 7f52e79fd740 1 -- start start 2024-03-15T12:59:48.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.915+0000 7f52e79fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8d56c0 0x55e04f8de150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.915+0000 7f52e79fd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d5fe0 0x55e04f8dab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.915+0000 7f52e79fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8debc0 0x55e04f8dc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.915+0000 7f52e79fd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f8d0ba0 con 0x55e04f8d5fe0 2024-03-15T12:59:48.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.915+0000 7f52e79fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f8db0c0 con 0x55e04f8debc0 2024-03-15T12:59:48.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.915+0000 7f52e79fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f8d9f70 con 0x55e04f8d56c0 2024-03-15T12:59:48.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.915+0000 7f52ddd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d5fe0 0x55e04f8dab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.915+0000 7f52de59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8d56c0 0x55e04f8de150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.915+0000 7f52ddd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d5fe0 0x55e04f8dab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42094/0 (socket says 172.21.15.2:42094) 2024-03-15T12:59:48.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.915+0000 7f52ddd9b640 1 -- 172.21.15.2:0/545961538 learned_addr learned my addr 172.21.15.2:0/545961538 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:48.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.915+0000 7f52ded9d640 1 --2- 172.21.15.2:0/545961538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8debc0 0x55e04f8dc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.915+0000 7f52de59c640 1 -- 172.21.15.2:0/545961538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8debc0 msgr2=0x55e04f8dc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:48.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.916+0000 7f52de59c640 1 --2- 172.21.15.2:0/545961538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8debc0 0x55e04f8dc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.916+0000 7f52de59c640 1 -- 172.21.15.2:0/545961538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d5fe0 msgr2=0x55e04f8dab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:48.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.916+0000 7f52de59c640 1 --2- 172.21.15.2:0/545961538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d5fe0 0x55e04f8dab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.916+0000 7f52de59c640 1 -- 172.21.15.2:0/545961538 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e04f7d3ac0 con 0x55e04f8d56c0 2024-03-15T12:59:48.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.916+0000 7f52de59c640 1 --2- 172.21.15.2:0/545961538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8d56c0 0x55e04f8de150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f52d00bc350 tx=0x7f52d00a7040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e745380345bfa12 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:48.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.916+0000 7f52c7fff640 1 -- 172.21.15.2:0/545961538 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00ece40 con 0x55e04f8d56c0 2024-03-15T12:59:48.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.916+0000 7f52c7fff640 1 -- 172.21.15.2:0/545961538 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f52d00c4ab0 con 0x55e04f8d56c0 2024-03-15T12:59:48.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.916+0000 7f52c7fff640 1 -- 172.21.15.2:0/545961538 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00ed0d0 con 0x55e04f8d56c0 2024-03-15T12:59:48.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.917+0000 7f52e79fd740 1 -- 172.21.15.2:0/545961538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8d56c0 msgr2=0x55e04f8de150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:48.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.917+0000 7f52e79fd740 1 --2- 172.21.15.2:0/545961538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8d56c0 0x55e04f8de150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f52d00bc350 tx=0x7f52d00a7040 comp rx=0 tx=0).stop 2024-03-15T12:59:48.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.917+0000 7f52e79fd740 1 -- 172.21.15.2:0/545961538 shutdown_connections 2024-03-15T12:59:48.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.917+0000 7f52e79fd740 1 --2- 172.21.15.2:0/545961538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8debc0 0x55e04f8dc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.917+0000 7f52e79fd740 1 --2- 172.21.15.2:0/545961538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d5fe0 0x55e04f8dab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.917+0000 7f52e79fd740 1 --2- 172.21.15.2:0/545961538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8d56c0 0x55e04f8de150 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.917+0000 7f52e79fd740 1 -- 172.21.15.2:0/545961538 >> 172.21.15.2:0/545961538 conn(0x55e04f8cfda0 msgr2=0x55e04f8d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.917+0000 7f52e79fd740 1 -- 172.21.15.2:0/545961538 shutdown_connections 2024-03-15T12:59:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.917+0000 7f52e79fd740 1 -- 172.21.15.2:0/545961538 wait complete. 2024-03-15T12:59:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.917+0000 7f52e79fd740 1 -- start start 2024-03-15T12:59:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52e79fd740 1 Processor -- start 2024-03-15T12:59:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52e79fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8d56c0 0x55e04f8d96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52e79fd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d5fe0 0x55e04f8d7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52e79fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8debc0 0x55e04f8d82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52e79fd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f8d8960 con 0x55e04f8d5fe0 2024-03-15T12:59:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52e79fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f8d8ad0 con 0x55e04f8debc0 2024-03-15T12:59:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52e79fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f949530 con 0x55e04f8d56c0 2024-03-15T12:59:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52ddd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d5fe0 0x55e04f8d7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52e79fd740 10 client.0 _ll_register_callbacks cb 0x7f52d8009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52ddd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d5fe0 0x55e04f8d7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42098/0 (socket says 172.21.15.2:42098) 2024-03-15T12:59:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52ddd9b640 1 -- 172.21.15.2:0/677263155 learned_addr learned my addr 172.21.15.2:0/677263155 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52de59c640 1 --2- 172.21.15.2:0/677263155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8d56c0 0x55e04f8d96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52ded9d640 1 --2- 172.21.15.2:0/677263155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8debc0 0x55e04f8d82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52ddd9b640 1 -- 172.21.15.2:0/677263155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8d56c0 msgr2=0x55e04f8d96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52ddd9b640 1 --2- 172.21.15.2:0/677263155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8d56c0 0x55e04f8d96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52ddd9b640 1 -- 172.21.15.2:0/677263155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8debc0 msgr2=0x55e04f8d82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52ddd9b640 1 --2- 172.21.15.2:0/677263155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8debc0 0x55e04f8d82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52ddd9b640 1 -- 172.21.15.2:0/677263155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e04f7f7570 con 0x55e04f8d5fe0 2024-03-15T12:59:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.918+0000 7f52ded9d640 1 --2- 172.21.15.2:0/677263155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8debc0 0x55e04f8d82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.919+0000 7f52ddd9b640 1 --2- 172.21.15.2:0/677263155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d5fe0 0x55e04f8d7d70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f52e0199130 tx=0x7f52d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab2481a52707f53 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.919+0000 7f52c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.919+0000 7f52c5ffb640 1 -- 172.21.15.2:0/677263155 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40293f0 con 0x55e04f8d5fe0 2024-03-15T12:59:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.919+0000 7f52c5ffb640 1 -- 172.21.15.2:0/677263155 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f52d4029550 con 0x55e04f8d5fe0 2024-03-15T12:59:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.919+0000 7f52e79fd740 1 -- 172.21.15.2:0/677263155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52d0003b40 con 0x55e04f8d5fe0 2024-03-15T12:59:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.919+0000 7f52c5ffb640 1 -- 172.21.15.2:0/677263155 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52d4029710 con 0x55e04f8d5fe0 2024-03-15T12:59:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.919+0000 7f52e79fd740 10 client.15996 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.919+0000 7f52e79fd740 1 -- 172.21.15.2:0/677263155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e04f7f96a0 con 0x55e04f8d5fe0 2024-03-15T12:59:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.919+0000 7f52c5ffb640 1 -- 172.21.15.2:0/677263155 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f52d0003b40 con 0x55e04f8d5fe0 2024-03-15T12:59:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.919+0000 7f52c5ffb640 1 -- 172.21.15.2:0/677263155 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f52d401e960 con 0x55e04f8d5fe0 2024-03-15T12:59:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.920+0000 7f52e79fd740 10 client.15996 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.920+0000 7f52e79fd740 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.920+0000 7f52e79fd740 20 client.15996 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.920+0000 7f52e79fd740 1 --2- 172.21.15.2:0/677263155 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e04f8eeca0 0x55e04f94cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.920+0000 7f52e79fd740 4 client.15996 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T12:59:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.920+0000 7f52e79fd740 1 -- 172.21.15.2:0/677263155 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e04f5c4890 con 0x55e04f8eeca0 2024-03-15T12:59:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.920+0000 7f52de59c640 1 --2- 172.21.15.2:0/677263155 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e04f8eeca0 0x55e04f94cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.920+0000 7f52c5ffb640 10 client.15996 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.920+0000 7f52de59c640 1 --2- 172.21.15.2:0/677263155 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e04f8eeca0 0x55e04f94cd60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d837975a72887def server_cookie=7314e3647467e6d7 in_seq=0 out_seq=0 2024-03-15T12:59:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.920+0000 7f52c5ffb640 1 -- 172.21.15.2:0/677263155 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e04f5c4890 con 0x55e04f8eeca0 2024-03-15T12:59:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.920+0000 7f52c5ffb640 10 client.15996 handle_command_reply: tid=0 2024-03-15T12:59:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.920+0000 7f52e79fd740 1 client.15996 shutdown 2024-03-15T12:59:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e79fd740 1 -- 172.21.15.2:0/677263155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d5fe0 msgr2=0x55e04f8d7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e79fd740 1 --2- 172.21.15.2:0/677263155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d5fe0 0x55e04f8d7d70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f52e0199130 tx=0x7f52d4004730 comp rx=0 tx=0).stop 2024-03-15T12:59:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e79fd740 1 -- 172.21.15.2:0/677263155 shutdown_connections 2024-03-15T12:59:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e79fd740 1 --2- 172.21.15.2:0/677263155 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e04f8eeca0 0x55e04f94cd60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e79fd740 1 --2- 172.21.15.2:0/677263155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e04f8debc0 0x55e04f8d82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e79fd740 1 --2- 172.21.15.2:0/677263155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e04f8d5fe0 0x55e04f8d7d70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52c5ffb640 0 client.15996 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e79fd740 1 --2- 172.21.15.2:0/677263155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e04f8d56c0 0x55e04f8d96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e79fd740 1 -- 172.21.15.2:0/677263155 >> 172.21.15.2:0/677263155 conn(0x55e04f8cfda0 msgr2=0x55e04f8ca1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e79fd740 1 -- 172.21.15.2:0/677263155 shutdown_connections 2024-03-15T12:59:48.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e79fd740 1 -- 172.21.15.2:0/677263155 wait complete. 2024-03-15T12:59:48.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e79fd740 20 client.15996 trim_cache size 0 max 16384 2024-03-15T12:59:48.912 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:59:48.912 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T12:59:48.912 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T12:59:48.912 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:59:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e4d4b640 1 -- 172.21.15.2:0/2352546218 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f52c80a1cc0 msgr2=0x7f52c80a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e4d4b640 1 --2- 172.21.15.2:0/2352546218 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f52c80a1cc0 0x7f52c80a4180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f52e010d980 tx=0x7f52d4005f90 comp rx=0 tx=0).stop 2024-03-15T12:59:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e4d4b640 1 -- 172.21.15.2:0/2352546218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e0102710 msgr2=0x7f52e019d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e4d4b640 1 --2- 172.21.15.2:0/2352546218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e0102710 0x7f52e019d790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f52d0037040 tx=0x7f52d00346f0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e4d4b640 1 -- 172.21.15.2:0/2352546218 shutdown_connections 2024-03-15T12:59:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e4d4b640 1 --2- 172.21.15.2:0/2352546218 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f52c80a1cc0 0x7f52c80a4180 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e4d4b640 1 --2- 172.21.15.2:0/2352546218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52e010ba50 0x7f52e0197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e4d4b640 1 --2- 172.21.15.2:0/2352546218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52e0103030 0x7f52e019dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e4d4b640 1 --2- 172.21.15.2:0/2352546218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52e0102710 0x7f52e019d790 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:48.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.921+0000 7f52e4d4b640 1 -- 172.21.15.2:0/2352546218 >> 172.21.15.2:0/2352546218 conn(0x7f52e00775a0 msgr2=0x7f52e00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:48.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.922+0000 7f52e4d4b640 1 -- 172.21.15.2:0/2352546218 shutdown_connections 2024-03-15T12:59:48.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:48.922+0000 7f52e4d4b640 1 -- 172.21.15.2:0/2352546218 wait complete. 2024-03-15T12:59:48.918 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T12:59:48.918 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T12:59:49.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.037+0000 7f4624d6c640 1 -- 172.21.15.2:0/1790201627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4620102720 msgr2=0x7f4620102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.037+0000 7f4624d6c640 1 --2- 172.21.15.2:0/1790201627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4620102720 0x7f4620102b00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4610009900 tx=0x7f461002bcd0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.037+0000 7f4624d6c640 1 -- 172.21.15.2:0/1790201627 shutdown_connections 2024-03-15T12:59:49.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.037+0000 7f4624d6c640 1 --2- 172.21.15.2:0/1790201627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f462010baf0 0x7f462010df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.037+0000 7f4624d6c640 1 --2- 172.21.15.2:0/1790201627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4620103040 0x7f462010b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.037+0000 7f4624d6c640 1 --2- 172.21.15.2:0/1790201627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4620102720 0x7f4620102b00 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.037+0000 7f4624d6c640 1 -- 172.21.15.2:0/1790201627 >> 172.21.15.2:0/1790201627 conn(0x7f4620076770 msgr2=0x7f4620076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.037+0000 7f4624d6c640 1 -- 172.21.15.2:0/1790201627 shutdown_connections 2024-03-15T12:59:49.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.037+0000 7f4624d6c640 1 -- 172.21.15.2:0/1790201627 wait complete. 2024-03-15T12:59:49.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f4624d6c640 1 Processor -- start 2024-03-15T12:59:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f4624d6c640 1 -- start start 2024-03-15T12:59:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f4624d6c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4620102720 0x7f462019d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f4624d6c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4620103040 0x7f462019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f4624d6c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f462010baf0 0x7f4620197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f4624d6c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4620197fe0 con 0x7f4620102720 2024-03-15T12:59:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f4624d6c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4620198150 con 0x7f4620103040 2024-03-15T12:59:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f4624d6c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4620198310 con 0x7f462010baf0 2024-03-15T12:59:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f461e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4620102720 0x7f462019d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f461e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4620102720 0x7f462019d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42106/0 (socket says 172.21.15.2:42106) 2024-03-15T12:59:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f461e59c640 1 -- 172.21.15.2:0/1400582507 learned_addr learned my addr 172.21.15.2:0/1400582507 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f461dd9b640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4620103040 0x7f462019dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f461ed9d640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f462010baf0 0x7f4620197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f461dd9b640 1 -- 172.21.15.2:0/1400582507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f462010baf0 msgr2=0x7f4620197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f461dd9b640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f462010baf0 0x7f4620197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f461dd9b640 1 -- 172.21.15.2:0/1400582507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4620102720 msgr2=0x7f462019d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f461dd9b640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4620102720 0x7f462019d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f461dd9b640 1 -- 172.21.15.2:0/1400582507 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4614009490 con 0x7f4620103040 2024-03-15T12:59:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f461e59c640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4620102720 0x7f462019d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.038+0000 7f461ed9d640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f462010baf0 0x7f4620197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:59:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.039+0000 7f461dd9b640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4620103040 0x7f462019dcd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f461400e9f0 tx=0x7f461400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.039+0000 7f460f7fe640 1 -- 172.21.15.2:0/1400582507 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461400cd80 con 0x7f4620103040 2024-03-15T12:59:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.039+0000 7f4624d6c640 1 -- 172.21.15.2:0/1400582507 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4610009560 con 0x7f4620103040 2024-03-15T12:59:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.039+0000 7f460f7fe640 1 -- 172.21.15.2:0/1400582507 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f46140035f0 con 0x7f4620103040 2024-03-15T12:59:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.039+0000 7f460f7fe640 1 -- 172.21.15.2:0/1400582507 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614010690 con 0x7f4620103040 2024-03-15T12:59:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.039+0000 7f4624d6c640 1 -- 172.21.15.2:0/1400582507 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46201987e0 con 0x7f4620103040 2024-03-15T12:59:49.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.040+0000 7f4624d6c640 1 -- 172.21.15.2:0/1400582507 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462006a340 con 0x7f4620103040 2024-03-15T12:59:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.041+0000 7f460f7fe640 1 -- 172.21.15.2:0/1400582507 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7f4614003760 con 0x7f4620103040 2024-03-15T12:59:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.042+0000 7f460f7fe640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f46080a1cc0 0x7f46080a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.042+0000 7f461e59c640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f46080a1cc0 0x7f46080a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.042+0000 7f460f7fe640 1 -- 172.21.15.2:0/1400582507 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f4614014070 con 0x7f4620103040 2024-03-15T12:59:49.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.042+0000 7f461e59c640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f46080a1cc0 0x7f46080a4180 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4610000c00 tx=0x7f46100058d0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.050+0000 7f460f7fe640 1 -- 172.21.15.2:0/1400582507 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4614096d50 con 0x7f4620103040 2024-03-15T12:59:49.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.356+0000 7f4624d6c640 1 -- 172.21.15.2:0/1400582507 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4620198e70 con 0x7f4620103040 2024-03-15T12:59:49.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.357+0000 7f460f7fe640 1 -- 172.21.15.2:0/1400582507 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11453 (secure 0 0 0) 0x7f461409a410 con 0x7f4620103040 2024-03-15T12:59:49.343 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T12:59:49.343 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T12:58:37.814324+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24434,24539,24460,24464,24500]},"id":1}]} 2024-03-15T12:59:49.343 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 29 2024-03-15T12:59:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.359+0000 7f4624d6c640 1 -- 172.21.15.2:0/1400582507 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f46080a1cc0 msgr2=0x7f46080a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.359+0000 7f4624d6c640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f46080a1cc0 0x7f46080a4180 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4610000c00 tx=0x7f46100058d0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.359+0000 7f4624d6c640 1 -- 172.21.15.2:0/1400582507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4620103040 msgr2=0x7f462019dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.359+0000 7f4624d6c640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4620103040 0x7f462019dcd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f461400e9f0 tx=0x7f461400eeb0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.359+0000 7f4624d6c640 1 -- 172.21.15.2:0/1400582507 shutdown_connections 2024-03-15T12:59:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.359+0000 7f4624d6c640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f462010baf0 0x7f4620197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.359+0000 7f4624d6c640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4620103040 0x7f462019dcd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.359+0000 7f4624d6c640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f46080a1cc0 0x7f46080a4180 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.359+0000 7f4624d6c640 1 --2- 172.21.15.2:0/1400582507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4620102720 0x7f462019d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.359+0000 7f4624d6c640 1 -- 172.21.15.2:0/1400582507 >> 172.21.15.2:0/1400582507 conn(0x7f4620076770 msgr2=0x7f46200fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.359+0000 7f4624d6c640 1 -- 172.21.15.2:0/1400582507 shutdown_connections 2024-03-15T12:59:49.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.359+0000 7f4624d6c640 1 -- 172.21.15.2:0/1400582507 wait complete. 2024-03-15T12:59:49.359 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T12:59:49.360 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T12:59:49.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.511+0000 7f99f740f640 1 -- 172.21.15.2:0/1134989082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f0100f80 msgr2=0x7f99f0104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.511+0000 7f99f740f640 1 --2- 172.21.15.2:0/1134989082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f0100f80 0x7f99f0104fc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009900 tx=0x7f99e002bc30 comp rx=0 tx=0).stop 2024-03-15T12:59:49.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.512+0000 7f99f740f640 1 -- 172.21.15.2:0/1134989082 shutdown_connections 2024-03-15T12:59:49.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.512+0000 7f99f740f640 1 --2- 172.21.15.2:0/1134989082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f01056a0 0x7f99f010d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.512+0000 7f99f740f640 1 --2- 172.21.15.2:0/1134989082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f0100f80 0x7f99f0104fc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.512+0000 7f99f740f640 1 --2- 172.21.15.2:0/1134989082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f01005d0 0x7f99f01009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.512+0000 7f99f740f640 1 -- 172.21.15.2:0/1134989082 >> 172.21.15.2:0/1134989082 conn(0x7f99f00774f0 msgr2=0x7f99f0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.512+0000 7f99f740f640 1 -- 172.21.15.2:0/1134989082 shutdown_connections 2024-03-15T12:59:49.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.512+0000 7f99f740f640 1 -- 172.21.15.2:0/1134989082 wait complete. 2024-03-15T12:59:49.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.512+0000 7f99f740f640 1 Processor -- start 2024-03-15T12:59:49.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.512+0000 7f99f740f640 1 -- start start 2024-03-15T12:59:49.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f740f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f01005d0 0x7f99f01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f740f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f01056a0 0x7f99f0198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f740f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f0199440 0x7f99f01a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f740f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f01a40b0 con 0x7f99f01056a0 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f740f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f01a4220 con 0x7f99f0199440 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f740f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f01a43e0 con 0x7f99f01005d0 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f49aa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f01056a0 0x7f99f0198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f49aa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f01056a0 0x7f99f0198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42116/0 (socket says 172.21.15.2:42116) 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f49aa640 1 -- 172.21.15.2:0/985315021 learned_addr learned my addr 172.21.15.2:0/985315021 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f51ab640 1 --2- 172.21.15.2:0/985315021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f01005d0 0x7f99f01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f59ac640 1 --2- 172.21.15.2:0/985315021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f0199440 0x7f99f01a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f49aa640 1 -- 172.21.15.2:0/985315021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f01005d0 msgr2=0x7f99f01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f49aa640 1 --2- 172.21.15.2:0/985315021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f01005d0 0x7f99f01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f49aa640 1 -- 172.21.15.2:0/985315021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f0199440 msgr2=0x7f99f01a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f49aa640 1 --2- 172.21.15.2:0/985315021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f0199440 0x7f99f01a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f49aa640 1 -- 172.21.15.2:0/985315021 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e0009560 con 0x7f99f01056a0 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.513+0000 7f99f49aa640 1 --2- 172.21.15.2:0/985315021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f01056a0 0x7f99f0198f00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f99e0000c00 tx=0x7f99e002cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.514+0000 7f99e67fc640 1 -- 172.21.15.2:0/985315021 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e002ccb0 con 0x7f99f01056a0 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.514+0000 7f99f740f640 1 -- 172.21.15.2:0/985315021 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f01a4660 con 0x7f99f01056a0 2024-03-15T12:59:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.514+0000 7f99f740f640 1 -- 172.21.15.2:0/985315021 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f01a4bc0 con 0x7f99f01056a0 2024-03-15T12:59:49.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.514+0000 7f99e67fc640 1 -- 172.21.15.2:0/985315021 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f99e00054d0 con 0x7f99f01056a0 2024-03-15T12:59:49.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.514+0000 7f99e67fc640 1 -- 172.21.15.2:0/985315021 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0003c30 con 0x7f99f01056a0 2024-03-15T12:59:49.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.517+0000 7f99e67fc640 1 -- 172.21.15.2:0/985315021 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7f99e0004250 con 0x7f99f01056a0 2024-03-15T12:59:49.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.517+0000 7f99f9fdf740 1 Processor -- start 2024-03-15T12:59:49.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.517+0000 7f99f9fdf740 1 -- start start 2024-03-15T12:59:49.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f9fdf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdc250 0x55ee2ecdc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f9fdf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f9fdf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ece5750 0x55ee2ece7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f9fdf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2ec97960 con 0x55ee2ecdc250 2024-03-15T12:59:49.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f9fdf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2ec4b310 con 0x55ee2ecdcb70 2024-03-15T12:59:49.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f9fdf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2ec48840 con 0x55ee2ece5750 2024-03-15T12:59:49.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99e67fc640 1 --2- 172.21.15.2:0/985315021 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f99dc0a1cf0 0x7f99dc0a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f59ac640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdc250 0x55ee2ecdc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f59ac640 1 --2- 172.21.15.2:0/985315021 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f99dc0a1cf0 0x7f99dc0a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f51ab640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f59ac640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdc250 0x55ee2ecdc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42122/0 (socket says 172.21.15.2:42122) 2024-03-15T12:59:49.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f59ac640 1 -- 172.21.15.2:0/3546529506 learned_addr learned my addr 172.21.15.2:0/3546529506 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99e67fc640 1 -- 172.21.15.2:0/985315021 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f99e00f0000 con 0x7f99f01056a0 2024-03-15T12:59:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f49aa640 1 --2- 172.21.15.2:0/3546529506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ece5750 0x55ee2ece7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f51ab640 1 -- 172.21.15.2:0/3546529506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ece5750 msgr2=0x55ee2ece7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f51ab640 1 --2- 172.21.15.2:0/3546529506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ece5750 0x55ee2ece7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f51ab640 1 -- 172.21.15.2:0/3546529506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdc250 msgr2=0x55ee2ecdc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f51ab640 1 --2- 172.21.15.2:0/3546529506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdc250 0x55ee2ecdc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f51ab640 1 -- 172.21.15.2:0/3546529506 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee2ec06060 con 0x55ee2ecdcb70 2024-03-15T12:59:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.518+0000 7f99f49aa640 1 --2- 172.21.15.2:0/3546529506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ece5750 0x55ee2ece7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:59:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.519+0000 7f99f59ac640 1 --2- 172.21.15.2:0/3546529506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdc250 0x55ee2ecdc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.519+0000 7f99f51ab640 1 --2- 172.21.15.2:0/3546529506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece5210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009c00 tx=0x7f99ec011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b5e72c05e076386 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.519+0000 7f99f59ac640 1 --2- 172.21.15.2:0/985315021 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f99dc0a1cf0 0x7f99dc0a41b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f99e80049d0 tx=0x7f99e8009500 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.519+0000 7f99db7fe640 1 -- 172.21.15.2:0/3546529506 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec004230 con 0x55ee2ecdcb70 2024-03-15T12:59:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.519+0000 7f99db7fe640 1 -- 172.21.15.2:0/3546529506 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f99ec009e70 con 0x55ee2ecdcb70 2024-03-15T12:59:49.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.519+0000 7f99db7fe640 1 -- 172.21.15.2:0/3546529506 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec004c60 con 0x55ee2ecdcb70 2024-03-15T12:59:49.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.520+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3546529506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdcb70 msgr2=0x55ee2ece5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.520+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/3546529506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece5210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009c00 tx=0x7f99ec011810 comp rx=0 tx=0).stop 2024-03-15T12:59:49.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.520+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3546529506 shutdown_connections 2024-03-15T12:59:49.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.520+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/3546529506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ece5750 0x55ee2ece7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.520+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/3546529506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece5210 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.520+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/3546529506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdc250 0x55ee2ecdc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.521+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3546529506 >> 172.21.15.2:0/3546529506 conn(0x55ee2ecd6980 msgr2=0x55ee2ecd6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.521+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3546529506 shutdown_connections 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.521+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3546529506 wait complete. 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.521+0000 7f99f9fdf740 1 -- start start 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.521+0000 7f99f9fdf740 1 Processor -- start 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f9fdf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdc250 0x55ee2ece4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f9fdf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f9fdf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ece5750 0x55ee2ece2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f9fdf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2ec44020 con 0x55ee2ecdcb70 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f9fdf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2e917430 con 0x55ee2ecdc250 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f9fdf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2e8d3e00 con 0x55ee2ece5750 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f9fdf740 10 client.0 _ll_register_callbacks cb 0x55ee2ebebcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f49aa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f51ab640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdc250 0x55ee2ece4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f51ab640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdc250 0x55ee2ece4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39312/0 (socket says 172.21.15.2:39312) 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f51ab640 1 -- 172.21.15.2:0/2121838745 learned_addr learned my addr 172.21.15.2:0/2121838745 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f59ac640 1 --2- 172.21.15.2:0/2121838745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ece5750 0x55ee2ece2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f51ab640 1 -- 172.21.15.2:0/2121838745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ece5750 msgr2=0x55ee2ece2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f51ab640 1 --2- 172.21.15.2:0/2121838745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ece5750 0x55ee2ece2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f51ab640 1 -- 172.21.15.2:0/2121838745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdcb70 msgr2=0x55ee2ece2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f51ab640 1 --2- 172.21.15.2:0/2121838745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f51ab640 1 -- 172.21.15.2:0/2121838745 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ec009830 con 0x55ee2ecdc250 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.522+0000 7f99f59ac640 1 --2- 172.21.15.2:0/2121838745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ece5750 0x55ee2ece2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.523+0000 7f99f51ab640 1 --2- 172.21.15.2:0/2121838745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdc250 0x55ee2ece4f80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009bd0 tx=0x7f99ec005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b981f637209ad47 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.523+0000 7f99d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.523+0000 7f99d8ff9640 1 -- 172.21.15.2:0/2121838745 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec004ec0 con 0x55ee2ecdc250 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.523+0000 7f99d8ff9640 1 -- 172.21.15.2:0/2121838745 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f99ec024470 con 0x55ee2ecdc250 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.523+0000 7f99f9fdf740 1 -- 172.21.15.2:0/2121838745 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee2e9437e0 con 0x55ee2ecdc250 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.523+0000 7f99d8ff9640 1 -- 172.21.15.2:0/2121838745 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99ec005640 con 0x55ee2ecdc250 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.523+0000 7f99f9fdf740 10 client.25883 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.523+0000 7f99f9fdf740 1 -- 172.21.15.2:0/2121838745 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ee2ec426a0 con 0x55ee2ecdc250 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.523+0000 7f99d8ff9640 1 -- 172.21.15.2:0/2121838745 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f99ec024b00 con 0x55ee2ecdc250 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.523+0000 7f99d8ff9640 1 -- 172.21.15.2:0/2121838745 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f99ec0198a0 con 0x55ee2ecdc250 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.523+0000 7f99f9fdf740 10 client.25883 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:49.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.523+0000 7f99f9fdf740 10 client.25883 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T12:59:49.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.523+0000 7f99f9fdf740 20 client.25883 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:49.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.523+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/2121838745 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ee2ecf55c0 0x55ee2ecf7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.523+0000 7f99f9fdf740 4 client.25883 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:49.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.524+0000 7f99f9fdf740 1 -- 172.21.15.2:0/2121838745 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee2e9437e0 con 0x55ee2ecf55c0 2024-03-15T12:59:49.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.524+0000 7f99f49aa640 1 --2- 172.21.15.2:0/2121838745 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ee2ecf55c0 0x55ee2ecf7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.524+0000 7f99f49aa640 1 --2- 172.21.15.2:0/2121838745 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ee2ecf55c0 0x55ee2ecf7a80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=75e61423d0d1ff96 server_cookie=bc0c6a4535fb16ac in_seq=0 out_seq=0 2024-03-15T12:59:49.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.524+0000 7f99d8ff9640 10 client.25883 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T12:59:49.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.525+0000 7f99d8ff9640 1 -- 172.21.15.2:0/2121838745 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ee2e9437e0 con 0x55ee2ecf55c0 2024-03-15T12:59:49.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.525+0000 7f99d8ff9640 10 client.25883 handle_command_reply: tid=0 2024-03-15T12:59:49.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.525+0000 7f99f9fdf740 1 client.25883 shutdown 2024-03-15T12:59:49.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.525+0000 7f99f9fdf740 1 -- 172.21.15.2:0/2121838745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdc250 msgr2=0x55ee2ece4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.525+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/2121838745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdc250 0x55ee2ece4f80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009bd0 tx=0x7f99ec005250 comp rx=0 tx=0).stop 2024-03-15T12:59:49.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.525+0000 7f99f9fdf740 1 -- 172.21.15.2:0/2121838745 shutdown_connections 2024-03-15T12:59:49.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.525+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/2121838745 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ee2ecf55c0 0x55ee2ecf7a80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.525+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/2121838745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ece5750 0x55ee2ece2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.525+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/2121838745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.525+0000 7f99d8ff9640 0 client.25883 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T12:59:49.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.525+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/2121838745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdc250 0x55ee2ece4f80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.525+0000 7f99f9fdf740 1 -- 172.21.15.2:0/2121838745 >> 172.21.15.2:0/2121838745 conn(0x55ee2ecd6980 msgr2=0x55ee2ecd86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.526+0000 7f99f9fdf740 1 -- 172.21.15.2:0/2121838745 shutdown_connections 2024-03-15T12:59:49.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.526+0000 7f99f9fdf740 1 -- 172.21.15.2:0/2121838745 wait complete. 2024-03-15T12:59:49.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.526+0000 7f99f9fdf740 20 client.25883 trim_cache size 0 max 16384 2024-03-15T12:59:49.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.538+0000 7f99f9fdf740 1 Processor -- start 2024-03-15T12:59:49.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.538+0000 7f99f9fdf740 1 -- start start 2024-03-15T12:59:49.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.538+0000 7f99f9fdf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdc250 0x55ee2ecd8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.538+0000 7f99f9fdf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.538+0000 7f99f9fdf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ece5750 0x55ee2ece4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.538+0000 7f99f9fdf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2ebc9130 con 0x55ee2ecdc250 2024-03-15T12:59:49.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.538+0000 7f99f9fdf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2ece2130 con 0x55ee2ece5750 2024-03-15T12:59:49.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.538+0000 7f99f9fdf740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2ece7f40 con 0x55ee2ecdcb70 2024-03-15T12:59:49.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.539+0000 7f99f59ac640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ece5750 0x55ee2ece4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.539+0000 7f99f49aa640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.539+0000 7f99f59ac640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ece5750 0x55ee2ece4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39320/0 (socket says 172.21.15.2:39320) 2024-03-15T12:59:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.539+0000 7f99f49aa640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:60134/0 (socket says 172.21.15.2:60134) 2024-03-15T12:59:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.539+0000 7f99f59ac640 1 -- 172.21.15.2:0/2780591862 learned_addr learned my addr 172.21.15.2:0/2780591862 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.539+0000 7f99f51ab640 1 --2- 172.21.15.2:0/2780591862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdc250 0x55ee2ecd8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.539+0000 7f99f49aa640 1 -- 172.21.15.2:0/2780591862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ece5750 msgr2=0x55ee2ece4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.539+0000 7f99f49aa640 1 --2- 172.21.15.2:0/2780591862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ece5750 0x55ee2ece4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.539+0000 7f99f49aa640 1 -- 172.21.15.2:0/2780591862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdc250 msgr2=0x55ee2ecd8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.539+0000 7f99f49aa640 1 --2- 172.21.15.2:0/2780591862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdc250 0x55ee2ecd8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.539+0000 7f99f49aa640 1 -- 172.21.15.2:0/2780591862 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee2ebdec20 con 0x55ee2ecdcb70 2024-03-15T12:59:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.539+0000 7f99f49aa640 1 --2- 172.21.15.2:0/2780591862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece4a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f99e00ed390 tx=0x7f99e00dc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1a02bc30eb4d6e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.539+0000 7f99daffd640 1 -- 172.21.15.2:0/2780591862 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e00bcea0 con 0x55ee2ecdcb70 2024-03-15T12:59:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.539+0000 7f99daffd640 1 -- 172.21.15.2:0/2780591862 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f99e00035f0 con 0x55ee2ecdcb70 2024-03-15T12:59:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.539+0000 7f99daffd640 1 -- 172.21.15.2:0/2780591862 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0003e20 con 0x55ee2ecdcb70 2024-03-15T12:59:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.540+0000 7f99f9fdf740 1 -- 172.21.15.2:0/2780591862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ecdcb70 msgr2=0x55ee2ece4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.540+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/2780591862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece4a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f99e00ed390 tx=0x7f99e00dc040 comp rx=0 tx=0).stop 2024-03-15T12:59:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.540+0000 7f99f9fdf740 1 -- 172.21.15.2:0/2780591862 shutdown_connections 2024-03-15T12:59:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.540+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/2780591862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ece5750 0x55ee2ece4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.540+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/2780591862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece4a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.540+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/2780591862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ecdc250 0x55ee2ecd8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.540+0000 7f99f9fdf740 1 -- 172.21.15.2:0/2780591862 >> 172.21.15.2:0/2780591862 conn(0x55ee2ecd6980 msgr2=0x55ee2ecd7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.540+0000 7f99f9fdf740 1 -- 172.21.15.2:0/2780591862 shutdown_connections 2024-03-15T12:59:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.540+0000 7f99f9fdf740 1 -- 172.21.15.2:0/2780591862 wait complete. 2024-03-15T12:59:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.541+0000 7f99f9fdf740 1 -- start start 2024-03-15T12:59:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.541+0000 7f99f9fdf740 1 Processor -- start 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.541+0000 7f99f9fdf740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ecdc250 0x55ee2eced160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.541+0000 7f99f9fdf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.541+0000 7f99f9fdf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ece5750 0x55ee2ece9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.541+0000 7f99f51ab640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ecdc250 0x55ee2eced160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.541+0000 7f99f59ac640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ece5750 0x55ee2ece9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.541+0000 7f99f59ac640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ece5750 0x55ee2ece9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42152/0 (socket says 172.21.15.2:42152) 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f49aa640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f51ab640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ecdc250 0x55ee2eced160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:60144/0 (socket says 172.21.15.2:60144) 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f59ac640 1 -- 172.21.15.2:0/3151945137 learned_addr learned my addr 172.21.15.2:0/3151945137 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f49aa640 1 --2- 172.21.15.2:0/3151945137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39324/0 (socket says 172.21.15.2:39324) 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3151945137 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2ece9f60 con 0x55ee2ece5750 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3151945137 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2ecea0d0 con 0x55ee2ecdcb70 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3151945137 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2ecea290 con 0x55ee2ecdc250 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f9fdf740 10 client.0 _ll_register_callbacks cb 0x7f99e8017810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f51ab640 1 -- 172.21.15.2:0/3151945137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdcb70 msgr2=0x55ee2ece94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f51ab640 1 --2- 172.21.15.2:0/3151945137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f51ab640 1 -- 172.21.15.2:0/3151945137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ece5750 msgr2=0x55ee2ece9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f51ab640 1 --2- 172.21.15.2:0/3151945137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ece5750 0x55ee2ece9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f51ab640 1 -- 172.21.15.2:0/3151945137 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f01a4bc0 con 0x55ee2ecdc250 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f59ac640 1 --2- 172.21.15.2:0/3151945137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ece5750 0x55ee2ece9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f51ab640 1 --2- 172.21.15.2:0/3151945137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ecdc250 0x55ee2eced160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f99ec019c80 tx=0x7f99ec004f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=439632c76567ce72 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99d8ff9640 1 -- 172.21.15.2:0/3151945137 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec011dc0 con 0x55ee2ecdc250 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99d8ff9640 1 -- 172.21.15.2:0/3151945137 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f99ec000e40 con 0x55ee2ecdc250 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.542+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3151945137 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99ec009e70 con 0x55ee2ecdc250 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.543+0000 7f99d8ff9640 1 -- 172.21.15.2:0/3151945137 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99ec004680 con 0x55ee2ecdc250 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.543+0000 7f99f9fdf740 10 client.25816 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.543+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3151945137 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ee2ec048c0 con 0x55ee2ecdc250 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.543+0000 7f99d8ff9640 1 -- 172.21.15.2:0/3151945137 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f99ec0198c0 con 0x55ee2ecdc250 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.543+0000 7f99d8ff9640 1 -- 172.21.15.2:0/3151945137 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f99ec0247e0 con 0x55ee2ecdc250 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.543+0000 7f99f9fdf740 10 client.25816 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.543+0000 7f99f9fdf740 10 client.25816 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T12:59:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.543+0000 7f99f9fdf740 20 client.25816 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.543+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/3151945137 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ee2ed520d0 0x55ee2ed544c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.543+0000 7f99f9fdf740 4 client.25816 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T12:59:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.543+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3151945137 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f99ec009e70 con 0x55ee2ed520d0 2024-03-15T12:59:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.543+0000 7f99f49aa640 1 --2- 172.21.15.2:0/3151945137 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ee2ed520d0 0x55ee2ed544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99d8ff9640 10 client.25816 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T12:59:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99f49aa640 1 --2- 172.21.15.2:0/3151945137 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ee2ed520d0 0x55ee2ed544c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bf00c79c9c61db7f server_cookie=969121b04a337482 in_seq=0 out_seq=0 2024-03-15T12:59:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99d8ff9640 1 -- 172.21.15.2:0/3151945137 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f99ec009e70 con 0x55ee2ed520d0 2024-03-15T12:59:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99d8ff9640 10 client.25816 handle_command_reply: tid=0 2024-03-15T12:59:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99f9fdf740 1 client.25816 shutdown 2024-03-15T12:59:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3151945137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ecdc250 msgr2=0x55ee2eced160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/3151945137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ecdc250 0x55ee2eced160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f99ec019c80 tx=0x7f99ec004f20 comp rx=0 tx=0).stop 2024-03-15T12:59:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3151945137 shutdown_connections 2024-03-15T12:59:49.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/3151945137 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ee2ed520d0 0x55ee2ed544c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99d8ff9640 0 client.25816 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T12:59:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/3151945137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee2ece5750 0x55ee2ece9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/3151945137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee2ecdcb70 0x55ee2ece94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99f9fdf740 1 --2- 172.21.15.2:0/3151945137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee2ecdc250 0x55ee2eced160 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3151945137 >> 172.21.15.2:0/3151945137 conn(0x55ee2ecd6980 msgr2=0x55ee2ecd5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3151945137 shutdown_connections 2024-03-15T12:59:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99f9fdf740 1 -- 172.21.15.2:0/3151945137 wait complete. 2024-03-15T12:59:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.544+0000 7f99f9fdf740 20 client.25816 trim_cache size 0 max 16384 2024-03-15T12:59:49.535 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T12:59:49.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.545+0000 7f99f740f640 1 -- 172.21.15.2:0/985315021 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f99dc0a1cf0 msgr2=0x7f99dc0a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.545+0000 7f99f740f640 1 --2- 172.21.15.2:0/985315021 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f99dc0a1cf0 0x7f99dc0a41b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f99e80049d0 tx=0x7f99e8009500 comp rx=0 tx=0).stop 2024-03-15T12:59:49.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.545+0000 7f99f740f640 1 -- 172.21.15.2:0/985315021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f01056a0 msgr2=0x7f99f0198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.545+0000 7f99f740f640 1 --2- 172.21.15.2:0/985315021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f01056a0 0x7f99f0198f00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f99e0000c00 tx=0x7f99e002cac0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.545+0000 7f99f740f640 1 -- 172.21.15.2:0/985315021 shutdown_connections 2024-03-15T12:59:49.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.545+0000 7f99f740f640 1 --2- 172.21.15.2:0/985315021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f0199440 0x7f99f01a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.545+0000 7f99f740f640 1 --2- 172.21.15.2:0/985315021 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f99dc0a1cf0 0x7f99dc0a41b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.545+0000 7f99f740f640 1 --2- 172.21.15.2:0/985315021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f01056a0 0x7f99f0198f00 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.545+0000 7f99f740f640 1 --2- 172.21.15.2:0/985315021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f01005d0 0x7f99f01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.545+0000 7f99f740f640 1 -- 172.21.15.2:0/985315021 >> 172.21.15.2:0/985315021 conn(0x7f99f00774f0 msgr2=0x7f99f00fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.545+0000 7f99f740f640 1 -- 172.21.15.2:0/985315021 shutdown_connections 2024-03-15T12:59:49.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.545+0000 7f99f740f640 1 -- 172.21.15.2:0/985315021 wait complete. 2024-03-15T12:59:49.545 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T12:59:49.545 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T12:59:49.643 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:49 smithi002 ceph-mon[32060]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:49.643 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:49 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1400582507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:59:49.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.668+0000 7f84af0ed640 1 -- 172.21.15.2:0/4064279074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84a8107560 msgr2=0x7f84a8107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.668+0000 7f84af0ed640 1 --2- 172.21.15.2:0/4064279074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84a8107560 0x7f84a8107940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f849c009930 tx=0x7f849c02bc30 comp rx=0 tx=0).stop 2024-03-15T12:59:49.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.668+0000 7f84af0ed640 1 -- 172.21.15.2:0/4064279074 shutdown_connections 2024-03-15T12:59:49.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.668+0000 7f84af0ed640 1 --2- 172.21.15.2:0/4064279074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a8101e60 0x7f84a810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.668+0000 7f84af0ed640 1 --2- 172.21.15.2:0/4064279074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84a81014a0 0x7f84a8101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.668+0000 7f84af0ed640 1 --2- 172.21.15.2:0/4064279074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84a8107560 0x7f84a8107940 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.668+0000 7f84af0ed640 1 -- 172.21.15.2:0/4064279074 >> 172.21.15.2:0/4064279074 conn(0x7f84a80788b0 msgr2=0x7f84a8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.668+0000 7f84af0ed640 1 -- 172.21.15.2:0/4064279074 shutdown_connections 2024-03-15T12:59:49.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.668+0000 7f84af0ed640 1 -- 172.21.15.2:0/4064279074 wait complete. 2024-03-15T12:59:49.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.669+0000 7f84af0ed640 1 Processor -- start 2024-03-15T12:59:49.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.669+0000 7f84af0ed640 1 -- start start 2024-03-15T12:59:49.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.669+0000 7f84af0ed640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84a81014a0 0x7f84a819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84af0ed640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84a8101e60 0x7f84a819dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84af0ed640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a8107560 0x7f84a8197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84af0ed640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a8197f60 con 0x7f84a81014a0 2024-03-15T12:59:49.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84af0ed640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a81980d0 con 0x7f84a8101e60 2024-03-15T12:59:49.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84af0ed640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a8198290 con 0x7f84a8107560 2024-03-15T12:59:49.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84ace89640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84a81014a0 0x7f84a819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84ace89640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84a81014a0 0x7f84a819d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42172/0 (socket says 172.21.15.2:42172) 2024-03-15T12:59:49.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84ace89640 1 -- 172.21.15.2:0/2005454297 learned_addr learned my addr 172.21.15.2:0/2005454297 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84a7fff640 1 --2- 172.21.15.2:0/2005454297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84a8101e60 0x7f84a819dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84ad68a640 1 --2- 172.21.15.2:0/2005454297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a8107560 0x7f84a8197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84ace89640 1 -- 172.21.15.2:0/2005454297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a8107560 msgr2=0x7f84a8197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84ace89640 1 --2- 172.21.15.2:0/2005454297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a8107560 0x7f84a8197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84ace89640 1 -- 172.21.15.2:0/2005454297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84a8101e60 msgr2=0x7f84a819dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84ace89640 1 --2- 172.21.15.2:0/2005454297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84a8101e60 0x7f84a819dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84ace89640 1 -- 172.21.15.2:0/2005454297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8494009490 con 0x7f84a81014a0 2024-03-15T12:59:49.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84a7fff640 1 --2- 172.21.15.2:0/2005454297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84a8101e60 0x7f84a819dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:49.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84ace89640 1 --2- 172.21.15.2:0/2005454297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84a81014a0 0x7f84a819d760 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f849c009900 tx=0x7f849c02ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.670+0000 7f84a5ffb640 1 -- 172.21.15.2:0/2005454297 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c02cc10 con 0x7f84a81014a0 2024-03-15T12:59:49.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.671+0000 7f84af0ed640 1 -- 172.21.15.2:0/2005454297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849c009560 con 0x7f84a81014a0 2024-03-15T12:59:49.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.671+0000 7f84a5ffb640 1 -- 172.21.15.2:0/2005454297 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f849c0054d0 con 0x7f84a81014a0 2024-03-15T12:59:49.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.671+0000 7f84a5ffb640 1 -- 172.21.15.2:0/2005454297 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c003c60 con 0x7f84a81014a0 2024-03-15T12:59:49.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.671+0000 7f84af0ed640 1 -- 172.21.15.2:0/2005454297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a81a4620 con 0x7f84a81014a0 2024-03-15T12:59:49.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.673+0000 7f84a5ffb640 1 -- 172.21.15.2:0/2005454297 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7f849c005000 con 0x7f84a81014a0 2024-03-15T12:59:49.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.674+0000 7f84a5ffb640 1 --2- 172.21.15.2:0/2005454297 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f84980a1cc0 0x7f84980a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.674+0000 7f84b1da4740 1 Processor -- start 2024-03-15T12:59:49.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.674+0000 7f84b1da4740 1 -- start start 2024-03-15T12:59:49.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.674+0000 7f84a5ffb640 1 -- 172.21.15.2:0/2005454297 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f849c0efd40 con 0x7f84a81014a0 2024-03-15T12:59:49.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.674+0000 7f84a7fff640 1 --2- 172.21.15.2:0/2005454297 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f84980a1cc0 0x7f84980a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84b1da4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500250 0x558a78500630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84a7fff640 1 --2- 172.21.15.2:0/2005454297 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f84980a1cc0 0x7f84980a4180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f84a80ff900 tx=0x7f8494005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84ace89640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500250 0x558a78500630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84ace89640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500250 0x558a78500630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42178/0 (socket says 172.21.15.2:42178) 2024-03-15T12:59:49.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84ace89640 1 -- 172.21.15.2:0/32417340 learned_addr learned my addr 172.21.15.2:0/32417340 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84b1da4740 1 --2- 172.21.15.2:0/32417340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500b70 0x558a78509210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84b1da4740 1 --2- 172.21.15.2:0/32417340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a7850bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84b1da4740 1 -- 172.21.15.2:0/32417340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a784bb960 con 0x558a78500250 2024-03-15T12:59:49.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84b1da4740 1 -- 172.21.15.2:0/32417340 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7846f310 con 0x558a78500b70 2024-03-15T12:59:49.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84b1da4740 1 -- 172.21.15.2:0/32417340 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7846c840 con 0x558a78509750 2024-03-15T12:59:49.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84a7fff640 1 --2- 172.21.15.2:0/32417340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500b70 0x558a78509210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84ad68a640 1 --2- 172.21.15.2:0/32417340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a7850bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84ace89640 1 -- 172.21.15.2:0/32417340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 msgr2=0x558a7850bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84ace89640 1 --2- 172.21.15.2:0/32417340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a7850bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84ace89640 1 -- 172.21.15.2:0/32417340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500b70 msgr2=0x558a78509210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84ace89640 1 --2- 172.21.15.2:0/32417340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500b70 0x558a78509210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.675+0000 7f84ace89640 1 -- 172.21.15.2:0/32417340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a7842a060 con 0x558a78500250 2024-03-15T12:59:49.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.676+0000 7f84ace89640 1 --2- 172.21.15.2:0/32417340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500250 0x558a78500630 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f849c0ed740 tx=0x7f849c0b8850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9efefbce737576 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.676+0000 7f8492ffd640 1 -- 172.21.15.2:0/32417340 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c0bc5b0 con 0x558a78500250 2024-03-15T12:59:49.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.676+0000 7f8492ffd640 1 -- 172.21.15.2:0/32417340 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f849c0c0700 con 0x558a78500250 2024-03-15T12:59:49.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.676+0000 7f8492ffd640 1 -- 172.21.15.2:0/32417340 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c003e50 con 0x558a78500250 2024-03-15T12:59:49.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.677+0000 7f84b1da4740 1 -- 172.21.15.2:0/32417340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500250 msgr2=0x558a78500630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.677+0000 7f84b1da4740 1 --2- 172.21.15.2:0/32417340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500250 0x558a78500630 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f849c0ed740 tx=0x7f849c0b8850 comp rx=0 tx=0).stop 2024-03-15T12:59:49.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.677+0000 7f84b1da4740 1 -- 172.21.15.2:0/32417340 shutdown_connections 2024-03-15T12:59:49.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.677+0000 7f84b1da4740 1 --2- 172.21.15.2:0/32417340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a7850bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.677+0000 7f84b1da4740 1 --2- 172.21.15.2:0/32417340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500b70 0x558a78509210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.677+0000 7f84b1da4740 1 --2- 172.21.15.2:0/32417340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500250 0x558a78500630 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.677+0000 7f84b1da4740 1 -- 172.21.15.2:0/32417340 >> 172.21.15.2:0/32417340 conn(0x558a784fa980 msgr2=0x558a784fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.678+0000 7f84b1da4740 1 -- 172.21.15.2:0/32417340 shutdown_connections 2024-03-15T12:59:49.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.678+0000 7f84b1da4740 1 -- 172.21.15.2:0/32417340 wait complete. 2024-03-15T12:59:49.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.678+0000 7f84b1da4740 1 -- start start 2024-03-15T12:59:49.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.678+0000 7f84b1da4740 1 Processor -- start 2024-03-15T12:59:49.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84b1da4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500250 0x558a785090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84b1da4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500b70 0x558a78506440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84b1da4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a785069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84b1da4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a78468020 con 0x558a78500b70 2024-03-15T12:59:49.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84b1da4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7813b430 con 0x558a78500250 2024-03-15T12:59:49.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84b1da4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a780f7e00 con 0x558a78509750 2024-03-15T12:59:49.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84a7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500b70 0x558a78506440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84ace89640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500250 0x558a785090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84a7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500b70 0x558a78506440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42190/0 (socket says 172.21.15.2:42190) 2024-03-15T12:59:49.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84a7fff640 1 -- 172.21.15.2:0/1744909388 learned_addr learned my addr 172.21.15.2:0/1744909388 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84b1da4740 10 client.0 _ll_register_callbacks cb 0x558a7840fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:49.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84ad68a640 1 --2- 172.21.15.2:0/1744909388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a785069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84a7fff640 1 -- 172.21.15.2:0/1744909388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 msgr2=0x558a785069a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84a7fff640 1 --2- 172.21.15.2:0/1744909388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a785069a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84a7fff640 1 -- 172.21.15.2:0/1744909388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500250 msgr2=0x558a785090b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84a7fff640 1 --2- 172.21.15.2:0/1744909388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500250 0x558a785090b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84a7fff640 1 -- 172.21.15.2:0/1744909388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a81a4620 con 0x558a78500b70 2024-03-15T12:59:49.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84ace89640 1 --2- 172.21.15.2:0/1744909388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500250 0x558a785090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:49.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84ad68a640 1 --2- 172.21.15.2:0/1744909388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a785069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:49.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f84a7fff640 1 --2- 172.21.15.2:0/1744909388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500b70 0x558a78506440 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f849400fd10 tx=0x7f8494013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b010b4e7722eb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.679+0000 7f8487fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:49.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.680+0000 7f8487fff640 1 -- 172.21.15.2:0/1744909388 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494021040 con 0x558a78500b70 2024-03-15T12:59:49.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.680+0000 7f8487fff640 1 -- 172.21.15.2:0/1744909388 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f8494027630 con 0x558a78500b70 2024-03-15T12:59:49.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.680+0000 7f84b1da4740 1 -- 172.21.15.2:0/1744909388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a781677e0 con 0x558a78500b70 2024-03-15T12:59:49.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.680+0000 7f8487fff640 1 -- 172.21.15.2:0/1744909388 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84940277f0 con 0x558a78500b70 2024-03-15T12:59:49.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.680+0000 7f84b1da4740 10 client.16032 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:49.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.680+0000 7f84b1da4740 1 -- 172.21.15.2:0/1744909388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558a784666a0 con 0x558a78500b70 2024-03-15T12:59:49.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.680+0000 7f8487fff640 1 -- 172.21.15.2:0/1744909388 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x558a781677e0 con 0x558a78500b70 2024-03-15T12:59:49.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.680+0000 7f8487fff640 1 -- 172.21.15.2:0/1744909388 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f849400e8b0 con 0x558a78500b70 2024-03-15T12:59:49.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.680+0000 7f84b1da4740 10 client.16032 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:49.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.680+0000 7f84b1da4740 10 client.16032 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T12:59:49.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.680+0000 7f84b1da4740 20 client.16032 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:49.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.680+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1744909388 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x558a785196e0 0x558a7851bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.680+0000 7f84b1da4740 4 client.16032 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:49.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.680+0000 7f84b1da4740 1 -- 172.21.15.2:0/1744909388 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a781a4720 con 0x558a785196e0 2024-03-15T12:59:49.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.681+0000 7f84ace89640 1 --2- 172.21.15.2:0/1744909388 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x558a785196e0 0x558a7851bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.681+0000 7f8487fff640 10 client.16032 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T12:59:49.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.681+0000 7f84ace89640 1 --2- 172.21.15.2:0/1744909388 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x558a785196e0 0x558a7851bb00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=da68c453bec32cce server_cookie=c734b6e68ed82ef4 in_seq=0 out_seq=0 2024-03-15T12:59:49.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.682+0000 7f8487fff640 1 -- 172.21.15.2:0/1744909388 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558a781a4720 con 0x558a785196e0 2024-03-15T12:59:49.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.682+0000 7f8487fff640 10 client.16032 handle_command_reply: tid=0 2024-03-15T12:59:49.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.682+0000 7f84b1da4740 1 client.16032 shutdown 2024-03-15T12:59:49.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.682+0000 7f84b1da4740 1 -- 172.21.15.2:0/1744909388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500b70 msgr2=0x558a78506440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.682+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1744909388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500b70 0x558a78506440 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f849400fd10 tx=0x7f8494013040 comp rx=0 tx=0).stop 2024-03-15T12:59:49.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.682+0000 7f84b1da4740 1 -- 172.21.15.2:0/1744909388 shutdown_connections 2024-03-15T12:59:49.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.682+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1744909388 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x558a785196e0 0x558a7851bb00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.682+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1744909388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a785069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.682+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1744909388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500b70 0x558a78506440 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.682+0000 7f8487fff640 0 client.16032 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T12:59:49.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.682+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1744909388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500250 0x558a785090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.682+0000 7f84b1da4740 1 -- 172.21.15.2:0/1744909388 >> 172.21.15.2:0/1744909388 conn(0x558a784fa980 msgr2=0x558a784fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.682+0000 7f84b1da4740 1 -- 172.21.15.2:0/1744909388 shutdown_connections 2024-03-15T12:59:49.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.682+0000 7f84b1da4740 1 -- 172.21.15.2:0/1744909388 wait complete. 2024-03-15T12:59:49.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.682+0000 7f84b1da4740 20 client.16032 trim_cache size 0 max 16384 2024-03-15T12:59:49.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.695+0000 7f84b1da4740 1 Processor -- start 2024-03-15T12:59:49.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.695+0000 7f84b1da4740 1 -- start start 2024-03-15T12:59:49.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.695+0000 7f84b1da4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500250 0x558a784fc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.695+0000 7f84b1da4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500b70 0x558a78508b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.695+0000 7f84b1da4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a785090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.695+0000 7f84b1da4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a783ed130 con 0x558a78500b70 2024-03-15T12:59:49.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.695+0000 7f84b1da4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a785160f0 con 0x558a78500250 2024-03-15T12:59:49.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.695+0000 7f84a7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500b70 0x558a78508b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.695+0000 7f84a7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500b70 0x558a78508b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42198/0 (socket says 172.21.15.2:42198) 2024-03-15T12:59:49.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.695+0000 7f84b1da4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a78515a30 con 0x558a78509750 2024-03-15T12:59:49.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.695+0000 7f84a7fff640 1 -- 172.21.15.2:0/1165765010 learned_addr learned my addr 172.21.15.2:0/1165765010 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.695+0000 7f84ace89640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500250 0x558a784fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.695+0000 7f84ad68a640 1 --2- 172.21.15.2:0/1165765010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a785090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.696+0000 7f84ad68a640 1 -- 172.21.15.2:0/1165765010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500250 msgr2=0x558a784fc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.696+0000 7f84ad68a640 1 --2- 172.21.15.2:0/1165765010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500250 0x558a784fc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.696+0000 7f84ad68a640 1 -- 172.21.15.2:0/1165765010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500b70 msgr2=0x558a78508b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.696+0000 7f84ad68a640 1 --2- 172.21.15.2:0/1165765010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500b70 0x558a78508b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.696+0000 7f84ad68a640 1 -- 172.21.15.2:0/1165765010 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a78402c20 con 0x558a78509750 2024-03-15T12:59:49.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.696+0000 7f84ace89640 1 --2- 172.21.15.2:0/1165765010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500250 0x558a784fc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:49.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.696+0000 7f84ad68a640 1 --2- 172.21.15.2:0/1165765010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a785090b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x558a784fbb10 tx=0x7f84a0009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd18aba6c8fbe906 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.696+0000 7f84927fc640 1 -- 172.21.15.2:0/1165765010 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a0016050 con 0x558a78509750 2024-03-15T12:59:49.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.696+0000 7f84927fc640 1 -- 172.21.15.2:0/1165765010 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f84a0017040 con 0x558a78509750 2024-03-15T12:59:49.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.696+0000 7f84927fc640 1 -- 172.21.15.2:0/1165765010 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a0005710 con 0x558a78509750 2024-03-15T12:59:49.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.697+0000 7f84b1da4740 1 -- 172.21.15.2:0/1165765010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 msgr2=0x558a785090b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.697+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1165765010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a785090b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x558a784fbb10 tx=0x7f84a0009500 comp rx=0 tx=0).stop 2024-03-15T12:59:49.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.697+0000 7f84b1da4740 1 -- 172.21.15.2:0/1165765010 shutdown_connections 2024-03-15T12:59:49.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.697+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1165765010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a785090b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.697+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1165765010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500b70 0x558a78508b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.697+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1165765010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500250 0x558a784fc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.697+0000 7f84b1da4740 1 -- 172.21.15.2:0/1165765010 >> 172.21.15.2:0/1165765010 conn(0x558a784fa980 msgr2=0x558a78506560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.697+0000 7f84b1da4740 1 -- 172.21.15.2:0/1165765010 shutdown_connections 2024-03-15T12:59:49.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.697+0000 7f84b1da4740 1 -- 172.21.15.2:0/1165765010 wait complete. 2024-03-15T12:59:49.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.698+0000 7f84b1da4740 1 -- start start 2024-03-15T12:59:49.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.698+0000 7f84b1da4740 1 Processor -- start 2024-03-15T12:59:49.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.698+0000 7f84b1da4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500250 0x558a785040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84b1da4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500b70 0x558a7850e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84ace89640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500250 0x558a785040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84ace89640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500250 0x558a785040c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42214/0 (socket says 172.21.15.2:42214) 2024-03-15T12:59:49.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84ace89640 1 -- 172.21.15.2:0/1541743303 learned_addr learned my addr 172.21.15.2:0/1541743303 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1541743303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a7850e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84b1da4740 1 -- 172.21.15.2:0/1541743303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7850eec0 con 0x558a78500250 2024-03-15T12:59:49.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84b1da4740 1 -- 172.21.15.2:0/1541743303 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7850f030 con 0x558a78500b70 2024-03-15T12:59:49.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84b1da4740 1 -- 172.21.15.2:0/1541743303 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7850f1f0 con 0x558a78509750 2024-03-15T12:59:49.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84a7fff640 1 --2- 172.21.15.2:0/1541743303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500b70 0x558a7850e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84ad68a640 1 --2- 172.21.15.2:0/1541743303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a7850e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84ace89640 1 -- 172.21.15.2:0/1541743303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 msgr2=0x558a7850e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84ace89640 1 --2- 172.21.15.2:0/1541743303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a7850e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84ace89640 1 -- 172.21.15.2:0/1541743303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500b70 msgr2=0x558a7850e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84ace89640 1 --2- 172.21.15.2:0/1541743303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500b70 0x558a7850e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84ace89640 1 -- 172.21.15.2:0/1541743303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a7842a060 con 0x558a78500250 2024-03-15T12:59:49.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84ad68a640 1 --2- 172.21.15.2:0/1541743303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a7850e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:59:49.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84b1da4740 10 client.0 _ll_register_callbacks cb 0x7f849400f660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:49.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f84ace89640 1 --2- 172.21.15.2:0/1541743303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500250 0x558a785040c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f849c004650 tx=0x7f849c0bc3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f32166325efc4ea server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.699+0000 7f8487fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:49.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.700+0000 7f8487fff640 1 -- 172.21.15.2:0/1541743303 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c0bccb0 con 0x558a78500250 2024-03-15T12:59:49.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.700+0000 7f8487fff640 1 -- 172.21.15.2:0/1541743303 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f849c0bc5b0 con 0x558a78500250 2024-03-15T12:59:49.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.700+0000 7f84b1da4740 1 -- 172.21.15.2:0/1541743303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f849c0c0700 con 0x558a78500250 2024-03-15T12:59:49.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.700+0000 7f8487fff640 1 -- 172.21.15.2:0/1541743303 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f849c0ba650 con 0x558a78500250 2024-03-15T12:59:49.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.700+0000 7f84b1da4740 10 client.16038 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:49.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.700+0000 7f84b1da4740 1 -- 172.21.15.2:0/1541743303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x7f84a0011750 con 0x558a78500250 2024-03-15T12:59:49.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.700+0000 7f8487fff640 1 -- 172.21.15.2:0/1541743303 <== mon.0 v2:172.21.15.2:3300/0 4 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f849c0dad20 con 0x558a78500250 2024-03-15T12:59:49.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.700+0000 7f84b1da4740 10 client.16038 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:49.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.701+0000 7f84b1da4740 10 client.16038 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T12:59:49.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.701+0000 7f84b1da4740 20 client.16038 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:49.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.701+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1541743303 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x558a78575ee0 0x558a78578300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.701+0000 7f84b1da4740 4 client.16038 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T12:59:49.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.701+0000 7f84b1da4740 1 -- 172.21.15.2:0/1541743303 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558a781f3890 con 0x558a78575ee0 2024-03-15T12:59:49.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.701+0000 7f84a7fff640 1 --2- 172.21.15.2:0/1541743303 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x558a78575ee0 0x558a78578300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.701+0000 7f8487fff640 10 client.16038 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T12:59:49.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.701+0000 7f84a7fff640 1 --2- 172.21.15.2:0/1541743303 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x558a78575ee0 0x558a78578300 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6505813484b6e68b server_cookie=d2bc1d042d3cd7ce in_seq=0 out_seq=0 2024-03-15T12:59:49.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.702+0000 7f8487fff640 1 -- 172.21.15.2:0/1541743303 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558a781f3890 con 0x558a78575ee0 2024-03-15T12:59:49.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.702+0000 7f8487fff640 10 client.16038 handle_command_reply: tid=0 2024-03-15T12:59:49.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.702+0000 7f84b1da4740 1 client.16038 shutdown 2024-03-15T12:59:49.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.702+0000 7f84b1da4740 1 -- 172.21.15.2:0/1541743303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500250 msgr2=0x558a785040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.702+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1541743303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500250 0x558a785040c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f849c004650 tx=0x7f849c0bc3a0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.702+0000 7f84b1da4740 1 -- 172.21.15.2:0/1541743303 shutdown_connections 2024-03-15T12:59:49.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.702+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1541743303 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x558a78575ee0 0x558a78578300 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.702+0000 7f8487fff640 0 client.16038 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T12:59:49.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.702+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1541743303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a78509750 0x558a7850e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.702+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1541743303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558a78500b70 0x558a7850e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.702+0000 7f84b1da4740 1 --2- 172.21.15.2:0/1541743303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a78500250 0x558a785040c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.702+0000 7f84b1da4740 1 -- 172.21.15.2:0/1541743303 >> 172.21.15.2:0/1541743303 conn(0x558a784fa980 msgr2=0x558a78500f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.702+0000 7f84b1da4740 1 -- 172.21.15.2:0/1541743303 shutdown_connections 2024-03-15T12:59:49.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.702+0000 7f84b1da4740 1 -- 172.21.15.2:0/1541743303 wait complete. 2024-03-15T12:59:49.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.702+0000 7f84b1da4740 20 client.16038 trim_cache size 0 max 16384 2024-03-15T12:59:49.693 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T12:59:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.703+0000 7f84af0ed640 1 -- 172.21.15.2:0/2005454297 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f84980a1cc0 msgr2=0x7f84980a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.703+0000 7f84af0ed640 1 --2- 172.21.15.2:0/2005454297 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f84980a1cc0 0x7f84980a4180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f84a80ff900 tx=0x7f8494005f90 comp rx=0 tx=0).stop 2024-03-15T12:59:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.703+0000 7f84af0ed640 1 -- 172.21.15.2:0/2005454297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84a81014a0 msgr2=0x7f84a819d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.703+0000 7f84af0ed640 1 --2- 172.21.15.2:0/2005454297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84a81014a0 0x7f84a819d760 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f849c009900 tx=0x7f849c02ca20 comp rx=0 tx=0).stop 2024-03-15T12:59:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.703+0000 7f84af0ed640 1 -- 172.21.15.2:0/2005454297 shutdown_connections 2024-03-15T12:59:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.703+0000 7f84af0ed640 1 --2- 172.21.15.2:0/2005454297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84a8107560 0x7f84a8197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.703+0000 7f84af0ed640 1 --2- 172.21.15.2:0/2005454297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84a8101e60 0x7f84a819dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.703+0000 7f84af0ed640 1 --2- 172.21.15.2:0/2005454297 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f84980a1cc0 0x7f84980a4180 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.703+0000 7f84af0ed640 1 --2- 172.21.15.2:0/2005454297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84a81014a0 0x7f84a819d760 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.703+0000 7f84af0ed640 1 -- 172.21.15.2:0/2005454297 >> 172.21.15.2:0/2005454297 conn(0x7f84a80788b0 msgr2=0x7f84a8100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.703+0000 7f84af0ed640 1 -- 172.21.15.2:0/2005454297 shutdown_connections 2024-03-15T12:59:49.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.703+0000 7f84af0ed640 1 -- 172.21.15.2:0/2005454297 wait complete. 2024-03-15T12:59:49.699 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T12:59:49.699 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T12:59:49.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.857+0000 7fe011c4d640 1 -- 172.21.15.2:0/1736854675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe00c1005d0 msgr2=0x7fe00c1009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.857+0000 7fe011c4d640 1 --2- 172.21.15.2:0/1736854675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe00c1005d0 0x7fe00c1009b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009860 tx=0x7fdffc02bb70 comp rx=0 tx=0).stop 2024-03-15T12:59:49.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.857+0000 7fe011c4d640 1 -- 172.21.15.2:0/1736854675 shutdown_connections 2024-03-15T12:59:49.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.857+0000 7fe011c4d640 1 --2- 172.21.15.2:0/1736854675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe00c1056a0 0x7fe00c10d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.857+0000 7fe011c4d640 1 --2- 172.21.15.2:0/1736854675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe00c100f80 0x7fe00c104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.857+0000 7fe011c4d640 1 --2- 172.21.15.2:0/1736854675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe00c1005d0 0x7fe00c1009b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.857+0000 7fe011c4d640 1 -- 172.21.15.2:0/1736854675 >> 172.21.15.2:0/1736854675 conn(0x7fe00c0774f0 msgr2=0x7fe00c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.857+0000 7fe011c4d640 1 -- 172.21.15.2:0/1736854675 shutdown_connections 2024-03-15T12:59:49.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.857+0000 7fe011c4d640 1 -- 172.21.15.2:0/1736854675 wait complete. 2024-03-15T12:59:49.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.858+0000 7fe011c4d640 1 Processor -- start 2024-03-15T12:59:49.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.858+0000 7fe011c4d640 1 -- start start 2024-03-15T12:59:49.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.858+0000 7fe011c4d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe00c1005d0 0x7fe00c19d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.858+0000 7fe011c4d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe00c1056a0 0x7fe00c19dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.858+0000 7fe011c4d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe00c197960 0x7fe00c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.858+0000 7fe011c4d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c198350 con 0x7fe00c197960 2024-03-15T12:59:49.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.858+0000 7fe011c4d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c1984c0 con 0x7fe00c1005d0 2024-03-15T12:59:49.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.858+0000 7fe011c4d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c198680 con 0x7fe00c1056a0 2024-03-15T12:59:49.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.858+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe00c1005d0 0x7fe00c19d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.858+0000 7fe00bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe00c197960 0x7fe00c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.858+0000 7fe00bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe00c197960 0x7fe00c197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42228/0 (socket says 172.21.15.2:42228) 2024-03-15T12:59:49.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.858+0000 7fe00bfff640 1 -- 172.21.15.2:0/2437962755 learned_addr learned my addr 172.21.15.2:0/2437962755 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.858+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe00c1005d0 0x7fe00c19d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39394/0 (socket says 172.21.15.2:39394) 2024-03-15T12:59:49.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.858+0000 7fe00affd640 1 --2- 172.21.15.2:0/2437962755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe00c1056a0 0x7fe00c19dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.859+0000 7fe00bfff640 1 -- 172.21.15.2:0/2437962755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe00c1056a0 msgr2=0x7fe00c19dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.859+0000 7fe00bfff640 1 --2- 172.21.15.2:0/2437962755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe00c1056a0 0x7fe00c19dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.859+0000 7fe00bfff640 1 -- 172.21.15.2:0/2437962755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe00c1005d0 msgr2=0x7fe00c19d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.859+0000 7fe00bfff640 1 --2- 172.21.15.2:0/2437962755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe00c1005d0 0x7fe00c19d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.859+0000 7fe00bfff640 1 -- 172.21.15.2:0/2437962755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe000009560 con 0x7fe00c197960 2024-03-15T12:59:49.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.859+0000 7fe00bfff640 1 --2- 172.21.15.2:0/2437962755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe00c197960 0x7fe00c197e10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe00400eb10 tx=0x7fe00400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.859+0000 7fe008ff9640 1 -- 172.21.15.2:0/2437962755 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004004b20 con 0x7fe00c197960 2024-03-15T12:59:49.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.859+0000 7fe011c4d640 1 -- 172.21.15.2:0/2437962755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdffc009490 con 0x7fe00c197960 2024-03-15T12:59:49.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.859+0000 7fe008ff9640 1 -- 172.21.15.2:0/2437962755 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe004004c80 con 0x7fe00c197960 2024-03-15T12:59:49.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.859+0000 7fe008ff9640 1 -- 172.21.15.2:0/2437962755 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004010690 con 0x7fe00c197960 2024-03-15T12:59:49.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.859+0000 7fe011c4d640 1 -- 172.21.15.2:0/2437962755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00c1a4940 con 0x7fe00c197960 2024-03-15T12:59:49.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.862+0000 7fe008ff9640 1 -- 172.21.15.2:0/2437962755 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7fe004004df0 con 0x7fe00c197960 2024-03-15T12:59:49.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.862+0000 7fe008ff9640 1 --2- 172.21.15.2:0/2437962755 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdff40a1cc0 0x7fdff40a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.862+0000 7fe008ff9640 1 -- 172.21.15.2:0/2437962755 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fe004014070 con 0x7fe00c197960 2024-03-15T12:59:49.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.862+0000 7fe00b7fe640 1 --2- 172.21.15.2:0/2437962755 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdff40a1cc0 0x7fdff40a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.863+0000 7fe01481f740 1 Processor -- start 2024-03-15T12:59:49.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.863+0000 7fe01481f740 1 -- start start 2024-03-15T12:59:49.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.863+0000 7fe00b7fe640 1 --2- 172.21.15.2:0/2437962755 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdff40a1cc0 0x7fdff40a4180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe00c105230 tx=0x7fdffc0058d0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.863+0000 7fe01481f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e0f250 0x55f718e0f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.863+0000 7fe01481f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e0fb70 0x55f718e18210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.863+0000 7fe01481f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e18750 0x55f718e1ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.863+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e0f250 0x55f718e0f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.863+0000 7fe01481f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f718dca960 con 0x55f718e0f250 2024-03-15T12:59:49.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.863+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e0f250 0x55f718e0f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42232/0 (socket says 172.21.15.2:42232) 2024-03-15T12:59:49.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.863+0000 7fe00b7fe640 1 -- 172.21.15.2:0/572023375 learned_addr learned my addr 172.21.15.2:0/572023375 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.863+0000 7fe00affd640 1 --2- 172.21.15.2:0/572023375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e0fb70 0x55f718e18210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.863+0000 7fe01481f740 1 -- 172.21.15.2:0/572023375 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f718d7e310 con 0x55f718e0fb70 2024-03-15T12:59:49.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.863+0000 7fe01481f740 1 -- 172.21.15.2:0/572023375 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f718d7b840 con 0x55f718e18750 2024-03-15T12:59:49.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.863+0000 7fe00bfff640 1 --2- 172.21.15.2:0/572023375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e18750 0x55f718e1ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.864+0000 7fe00bfff640 1 -- 172.21.15.2:0/572023375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e0fb70 msgr2=0x55f718e18210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.864+0000 7fe00bfff640 1 --2- 172.21.15.2:0/572023375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e0fb70 0x55f718e18210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.864+0000 7fe00bfff640 1 -- 172.21.15.2:0/572023375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e0f250 msgr2=0x55f718e0f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.864+0000 7fe00bfff640 1 --2- 172.21.15.2:0/572023375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e0f250 0x55f718e0f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.864+0000 7fe00bfff640 1 -- 172.21.15.2:0/572023375 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f718d39060 con 0x55f718e18750 2024-03-15T12:59:49.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.864+0000 7fe00b7fe640 1 --2- 172.21.15.2:0/572023375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e0f250 0x55f718e0f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:49.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.864+0000 7fe00bfff640 1 --2- 172.21.15.2:0/572023375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e18750 0x55f718e1ab40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe0040cb510 tx=0x7fe004009a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e373758b5489d69f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.864+0000 7fdff1ffb640 1 -- 172.21.15.2:0/572023375 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00409a920 con 0x55f718e18750 2024-03-15T12:59:49.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.864+0000 7fdff1ffb640 1 -- 172.21.15.2:0/572023375 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe00409ea70 con 0x55f718e18750 2024-03-15T12:59:49.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.864+0000 7fdff1ffb640 1 -- 172.21.15.2:0/572023375 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0040cb7f0 con 0x55f718e18750 2024-03-15T12:59:49.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.865+0000 7fe01481f740 1 -- 172.21.15.2:0/572023375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e18750 msgr2=0x55f718e1ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.865+0000 7fe01481f740 1 --2- 172.21.15.2:0/572023375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e18750 0x55f718e1ab40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe0040cb510 tx=0x7fe004009a70 comp rx=0 tx=0).stop 2024-03-15T12:59:49.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.865+0000 7fe01481f740 1 -- 172.21.15.2:0/572023375 shutdown_connections 2024-03-15T12:59:49.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.865+0000 7fe01481f740 1 --2- 172.21.15.2:0/572023375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e18750 0x55f718e1ab40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.865+0000 7fe01481f740 1 --2- 172.21.15.2:0/572023375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e0fb70 0x55f718e18210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.865+0000 7fe01481f740 1 --2- 172.21.15.2:0/572023375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e0f250 0x55f718e0f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.865+0000 7fe01481f740 1 -- 172.21.15.2:0/572023375 >> 172.21.15.2:0/572023375 conn(0x55f718e09980 msgr2=0x55f718e09d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.865+0000 7fe01481f740 1 -- 172.21.15.2:0/572023375 shutdown_connections 2024-03-15T12:59:49.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.865+0000 7fe01481f740 1 -- 172.21.15.2:0/572023375 wait complete. 2024-03-15T12:59:49.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.865+0000 7fe01481f740 1 -- start start 2024-03-15T12:59:49.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.865+0000 7fe01481f740 1 Processor -- start 2024-03-15T12:59:49.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe01481f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e0f250 0x55f718e22370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe01481f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e0fb70 0x55f718e228b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe01481f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e18750 0x55f718e1c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e0f250 0x55f718e22370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe01481f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f718d77020 con 0x55f718e0f250 2024-03-15T12:59:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e0f250 0x55f718e22370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42246/0 (socket says 172.21.15.2:42246) 2024-03-15T12:59:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe00b7fe640 1 -- 172.21.15.2:0/2225508996 learned_addr learned my addr 172.21.15.2:0/2225508996 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe01481f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f718a4a430 con 0x55f718e0fb70 2024-03-15T12:59:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe01481f740 1 -- 172.21.15.2:0/2225508996 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f718a06e00 con 0x55f718e18750 2024-03-15T12:59:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe00bfff640 1 --2- 172.21.15.2:0/2225508996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e18750 0x55f718e1c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe00affd640 1 --2- 172.21.15.2:0/2225508996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e0fb70 0x55f718e228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe01481f740 10 client.0 _ll_register_callbacks cb 0x55f718d1ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe00b7fe640 1 -- 172.21.15.2:0/2225508996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e18750 msgr2=0x55f718e1c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe00b7fe640 1 --2- 172.21.15.2:0/2225508996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e18750 0x55f718e1c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe00b7fe640 1 -- 172.21.15.2:0/2225508996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e0fb70 msgr2=0x55f718e228b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe00b7fe640 1 --2- 172.21.15.2:0/2225508996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e0fb70 0x55f718e228b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe00b7fe640 1 -- 172.21.15.2:0/2225508996 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00c1a4940 con 0x55f718e0f250 2024-03-15T12:59:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe01481f740 1 -- 172.21.15.2:0/2225508996 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f718a767e0 con 0x55f718e0f250 2024-03-15T12:59:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fe00b7fe640 1 --2- 172.21.15.2:0/2225508996 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e0f250 0x55f718e22370 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009230 tx=0x7fdffc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44889f9ab92b7f0e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.866+0000 7fdfeaffd640 10 client.16050 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:49.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.867+0000 7fdfeaffd640 1 -- 172.21.15.2:0/2225508996 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc02caa0 con 0x55f718e0f250 2024-03-15T12:59:49.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.867+0000 7fdfeaffd640 1 -- 172.21.15.2:0/2225508996 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fdffc0035f0 con 0x55f718e0f250 2024-03-15T12:59:49.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.867+0000 7fdfeaffd640 1 -- 172.21.15.2:0/2225508996 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdffc0037d0 con 0x55f718e0f250 2024-03-15T12:59:49.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.867+0000 7fe01481f740 10 client.16050 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:49.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.867+0000 7fe01481f740 1 -- 172.21.15.2:0/2225508996 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55f718d756a0 con 0x55f718e0f250 2024-03-15T12:59:49.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.867+0000 7fdfeaffd640 1 -- 172.21.15.2:0/2225508996 <== mon.0 v2:172.21.15.2:3300/0 4 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdffc003930 con 0x55f718e0f250 2024-03-15T12:59:49.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.867+0000 7fe01481f740 10 client.16050 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:49.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.867+0000 7fe01481f740 10 client.16050 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T12:59:49.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.867+0000 7fe01481f740 20 client.16050 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:49.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.867+0000 7fe01481f740 1 --2- 172.21.15.2:0/2225508996 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55f718e2ca30 0x55f718e2ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.867+0000 7fe01481f740 4 client.16050 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:49.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.867+0000 7fe01481f740 1 -- 172.21.15.2:0/2225508996 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f718ab3720 con 0x55f718e2ca30 2024-03-15T12:59:49.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.867+0000 7fe00affd640 1 --2- 172.21.15.2:0/2225508996 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55f718e2ca30 0x55f718e2ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.868+0000 7fe00affd640 1 --2- 172.21.15.2:0/2225508996 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55f718e2ca30 0x55f718e2ee40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f071db39c4420b3 server_cookie=5402a50ea33b7e76 in_seq=0 out_seq=0 2024-03-15T12:59:49.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.868+0000 7fdfeaffd640 10 client.16050 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T12:59:49.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.869+0000 7fdfeaffd640 1 -- 172.21.15.2:0/2225508996 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f718ab3720 con 0x55f718e2ca30 2024-03-15T12:59:49.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.869+0000 7fdfeaffd640 10 client.16050 handle_command_reply: tid=0 2024-03-15T12:59:49.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.869+0000 7fe01481f740 1 client.16050 shutdown 2024-03-15T12:59:49.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.869+0000 7fe01481f740 1 -- 172.21.15.2:0/2225508996 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e0f250 msgr2=0x55f718e22370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.869+0000 7fe01481f740 1 --2- 172.21.15.2:0/2225508996 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e0f250 0x55f718e22370 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009230 tx=0x7fdffc005f90 comp rx=0 tx=0).stop 2024-03-15T12:59:49.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.869+0000 7fe01481f740 1 -- 172.21.15.2:0/2225508996 shutdown_connections 2024-03-15T12:59:49.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.869+0000 7fe01481f740 1 --2- 172.21.15.2:0/2225508996 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55f718e2ca30 0x55f718e2ee40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.869+0000 7fe01481f740 1 --2- 172.21.15.2:0/2225508996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e18750 0x55f718e1c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.869+0000 7fe01481f740 1 --2- 172.21.15.2:0/2225508996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e0fb70 0x55f718e228b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.869+0000 7fe01481f740 1 --2- 172.21.15.2:0/2225508996 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e0f250 0x55f718e22370 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.869+0000 7fe01481f740 1 -- 172.21.15.2:0/2225508996 >> 172.21.15.2:0/2225508996 conn(0x55f718e09980 msgr2=0x55f718e0b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.869+0000 7fdfeaffd640 0 client.16050 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T12:59:49.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.869+0000 7fe01481f740 1 -- 172.21.15.2:0/2225508996 shutdown_connections 2024-03-15T12:59:49.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.869+0000 7fe01481f740 1 -- 172.21.15.2:0/2225508996 wait complete. 2024-03-15T12:59:49.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.869+0000 7fe01481f740 20 client.16050 trim_cache size 0 max 16384 2024-03-15T12:59:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe01481f740 1 Processor -- start 2024-03-15T12:59:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe01481f740 1 -- start start 2024-03-15T12:59:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe01481f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e0fb70 0x55f718e22750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe01481f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e18750 0x55f718e16b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe01481f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e34750 0x55f718e17040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe01481f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f718cfc130 con 0x55f718e18750 2024-03-15T12:59:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe01481f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f718e052b0 con 0x55f718e34750 2024-03-15T12:59:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe01481f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f718e185b0 con 0x55f718e0fb70 2024-03-15T12:59:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe00affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e18750 0x55f718e16b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe00affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e18750 0x55f718e16b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42248/0 (socket says 172.21.15.2:42248) 2024-03-15T12:59:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e0fb70 0x55f718e22750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe00affd640 1 -- 172.21.15.2:0/3085704186 learned_addr learned my addr 172.21.15.2:0/3085704186 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe00bfff640 1 --2- 172.21.15.2:0/3085704186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e34750 0x55f718e17040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe00b7fe640 1 -- 172.21.15.2:0/3085704186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e34750 msgr2=0x55f718e17040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe00b7fe640 1 --2- 172.21.15.2:0/3085704186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e34750 0x55f718e17040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe00b7fe640 1 -- 172.21.15.2:0/3085704186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e18750 msgr2=0x55f718e16b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.882+0000 7fe00b7fe640 1 --2- 172.21.15.2:0/3085704186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e18750 0x55f718e16b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.883+0000 7fe00b7fe640 1 -- 172.21.15.2:0/3085704186 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f718d11c20 con 0x55f718e0fb70 2024-03-15T12:59:49.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.883+0000 7fe00affd640 1 --2- 172.21.15.2:0/3085704186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e18750 0x55f718e16b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:49.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.883+0000 7fe00b7fe640 1 --2- 172.21.15.2:0/3085704186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e0fb70 0x55f718e22750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0057c0 tx=0x7fdffc04f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5afcac4b996d7a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.883+0000 7fdff17fa640 1 -- 172.21.15.2:0/3085704186 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc003da0 con 0x55f718e0fb70 2024-03-15T12:59:49.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.883+0000 7fdff17fa640 1 -- 172.21.15.2:0/3085704186 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55f718a767e0 con 0x55f718e0fb70 2024-03-15T12:59:49.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.883+0000 7fdff17fa640 1 -- 172.21.15.2:0/3085704186 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc005420 con 0x55f718e0fb70 2024-03-15T12:59:49.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.884+0000 7fe01481f740 1 -- 172.21.15.2:0/3085704186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e0fb70 msgr2=0x55f718e22750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.884+0000 7fe01481f740 1 --2- 172.21.15.2:0/3085704186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e0fb70 0x55f718e22750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0057c0 tx=0x7fdffc04f000 comp rx=0 tx=0).stop 2024-03-15T12:59:49.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.884+0000 7fe01481f740 1 -- 172.21.15.2:0/3085704186 shutdown_connections 2024-03-15T12:59:49.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.884+0000 7fe01481f740 1 --2- 172.21.15.2:0/3085704186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e34750 0x55f718e17040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.884+0000 7fe01481f740 1 --2- 172.21.15.2:0/3085704186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e18750 0x55f718e16b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.884+0000 7fe01481f740 1 --2- 172.21.15.2:0/3085704186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e0fb70 0x55f718e22750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.884+0000 7fe01481f740 1 -- 172.21.15.2:0/3085704186 >> 172.21.15.2:0/3085704186 conn(0x55f718e09980 msgr2=0x55f718e09d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.884+0000 7fe01481f740 1 -- 172.21.15.2:0/3085704186 shutdown_connections 2024-03-15T12:59:49.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.884+0000 7fe01481f740 1 -- 172.21.15.2:0/3085704186 wait complete. 2024-03-15T12:59:49.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.884+0000 7fe01481f740 1 -- start start 2024-03-15T12:59:49.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.884+0000 7fe01481f740 1 Processor -- start 2024-03-15T12:59:49.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe01481f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e0fb70 0x55f718e28a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe01481f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e18750 0x55f718e26e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe01481f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e34750 0x55f718e27370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe01481f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f718e29310 con 0x55f718e34750 2024-03-15T12:59:49.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe01481f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f718e278b0 con 0x55f718e18750 2024-03-15T12:59:49.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe01481f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f718e27a40 con 0x55f718e0fb70 2024-03-15T12:59:49.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe01481f740 10 client.0 _ll_register_callbacks cb 0x7fe0040093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:49.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe00bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e34750 0x55f718e27370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe00affd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e18750 0x55f718e26e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e0fb70 0x55f718e28a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe00bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e34750 0x55f718e27370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42252/0 (socket says 172.21.15.2:42252) 2024-03-15T12:59:49.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe00bfff640 1 -- 172.21.15.2:0/1323387143 learned_addr learned my addr 172.21.15.2:0/1323387143 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe00b7fe640 1 -- 172.21.15.2:0/1323387143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e18750 msgr2=0x55f718e26e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe00b7fe640 1 --2- 172.21.15.2:0/1323387143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e18750 0x55f718e26e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe00b7fe640 1 -- 172.21.15.2:0/1323387143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e34750 msgr2=0x55f718e27370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:49.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe00b7fe640 1 --2- 172.21.15.2:0/1323387143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e34750 0x55f718e27370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe00b7fe640 1 -- 172.21.15.2:0/1323387143 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe000009d80 con 0x55f718e0fb70 2024-03-15T12:59:49.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe00bfff640 1 --2- 172.21.15.2:0/1323387143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e34750 0x55f718e27370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:49.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe00b7fe640 1 --2- 172.21.15.2:0/1323387143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e0fb70 0x55f718e28a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdffc02c0b0 tx=0x7fdffc003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64ec7dba30b6012 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:49.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fdfeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T12:59:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fdfeaffd640 1 -- 172.21.15.2:0/1323387143 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc003da0 con 0x55f718e0fb70 2024-03-15T12:59:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.885+0000 7fe01481f740 1 -- 172.21.15.2:0/1323387143 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f718e050f0 con 0x55f718e0fb70 2024-03-15T12:59:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.886+0000 7fdfeaffd640 1 -- 172.21.15.2:0/1323387143 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fdffc02c8a0 con 0x55f718e0fb70 2024-03-15T12:59:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.886+0000 7fdfeaffd640 1 -- 172.21.15.2:0/1323387143 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdffc005040 con 0x55f718e0fb70 2024-03-15T12:59:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.886+0000 7fe01481f740 10 client.25852 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.886+0000 7fe01481f740 1 -- 172.21.15.2:0/1323387143 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f718d756a0 con 0x55f718e0fb70 2024-03-15T12:59:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.886+0000 7fdfeaffd640 1 -- 172.21.15.2:0/1323387143 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55f718e050f0 con 0x55f718e0fb70 2024-03-15T12:59:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.886+0000 7fdfeaffd640 1 -- 172.21.15.2:0/1323387143 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdffc032600 con 0x55f718e0fb70 2024-03-15T12:59:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.886+0000 7fe01481f740 10 client.25852 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.886+0000 7fe01481f740 10 client.25852 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T12:59:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.886+0000 7fe01481f740 20 client.25852 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.886+0000 7fe01481f740 1 --2- 172.21.15.2:0/1323387143 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55f718e891f0 0x55f718e8b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.886+0000 7fe01481f740 4 client.25852 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T12:59:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.886+0000 7fe01481f740 1 -- 172.21.15.2:0/1323387143 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fe00409ea70 con 0x55f718e891f0 2024-03-15T12:59:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.886+0000 7fe00affd640 1 --2- 172.21.15.2:0/1323387143 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55f718e891f0 0x55f718e8b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fdfeaffd640 10 client.25852 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T12:59:49.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fe00affd640 1 --2- 172.21.15.2:0/1323387143 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55f718e891f0 0x55f718e8b5e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9020651ebd1e35b2 server_cookie=34412704f1cd8f58 in_seq=0 out_seq=0 2024-03-15T12:59:49.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fdfeaffd640 1 -- 172.21.15.2:0/1323387143 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fe00409ea70 con 0x55f718e891f0 2024-03-15T12:59:49.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fdfeaffd640 10 client.25852 handle_command_reply: tid=0 2024-03-15T12:59:49.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fe01481f740 1 client.25852 shutdown 2024-03-15T12:59:49.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fe01481f740 1 -- 172.21.15.2:0/1323387143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e0fb70 msgr2=0x55f718e28a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fe01481f740 1 --2- 172.21.15.2:0/1323387143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e0fb70 0x55f718e28a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdffc02c0b0 tx=0x7fdffc003b90 comp rx=0 tx=0).stop 2024-03-15T12:59:49.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fe01481f740 1 -- 172.21.15.2:0/1323387143 shutdown_connections 2024-03-15T12:59:49.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fe01481f740 1 --2- 172.21.15.2:0/1323387143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f718e34750 0x55f718e27370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fe01481f740 1 --2- 172.21.15.2:0/1323387143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f718e18750 0x55f718e26e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fe01481f740 1 --2- 172.21.15.2:0/1323387143 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55f718e891f0 0x55f718e8b5e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fe01481f740 1 --2- 172.21.15.2:0/1323387143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f718e0fb70 0x55f718e28a40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fdfeaffd640 0 client.25852 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T12:59:49.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fe01481f740 1 -- 172.21.15.2:0/1323387143 >> 172.21.15.2:0/1323387143 conn(0x55f718e09980 msgr2=0x55f718e0adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fe01481f740 1 -- 172.21.15.2:0/1323387143 shutdown_connections 2024-03-15T12:59:49.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fe01481f740 1 -- 172.21.15.2:0/1323387143 wait complete. 2024-03-15T12:59:49.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.887+0000 7fe01481f740 20 client.25852 trim_cache size 0 max 16384 2024-03-15T12:59:49.879 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T12:59:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.888+0000 7fe011c4d640 1 -- 172.21.15.2:0/2437962755 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdff40a1cc0 msgr2=0x7fdff40a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.888+0000 7fe011c4d640 1 --2- 172.21.15.2:0/2437962755 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdff40a1cc0 0x7fdff40a4180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe00c105230 tx=0x7fdffc0058d0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.888+0000 7fe011c4d640 1 -- 172.21.15.2:0/2437962755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe00c197960 msgr2=0x7fe00c197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.888+0000 7fe011c4d640 1 --2- 172.21.15.2:0/2437962755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe00c197960 0x7fe00c197e10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe00400eb10 tx=0x7fe00400c2d0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.888+0000 7fe011c4d640 1 -- 172.21.15.2:0/2437962755 shutdown_connections 2024-03-15T12:59:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.888+0000 7fe011c4d640 1 --2- 172.21.15.2:0/2437962755 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdff40a1cc0 0x7fdff40a4180 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.888+0000 7fe011c4d640 1 --2- 172.21.15.2:0/2437962755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe00c197960 0x7fe00c197e10 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.888+0000 7fe011c4d640 1 --2- 172.21.15.2:0/2437962755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe00c1056a0 0x7fe00c19dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.888+0000 7fe011c4d640 1 --2- 172.21.15.2:0/2437962755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe00c1005d0 0x7fe00c19d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.888+0000 7fe011c4d640 1 -- 172.21.15.2:0/2437962755 >> 172.21.15.2:0/2437962755 conn(0x7fe00c0774f0 msgr2=0x7fe00c104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.888+0000 7fe011c4d640 1 -- 172.21.15.2:0/2437962755 shutdown_connections 2024-03-15T12:59:49.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:49.888+0000 7fe011c4d640 1 -- 172.21.15.2:0/2437962755 wait complete. 2024-03-15T12:59:49.885 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T12:59:49.885 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T12:59:49.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.009+0000 7fe84293a640 1 -- 172.21.15.2:0/2923301626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe83c10dd30 msgr2=0x7fe83c110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:49.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.009+0000 7fe84293a640 1 --2- 172.21.15.2:0/2923301626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe83c10dd30 0x7fe83c110140 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe834009ca0 tx=0x7fe83402c180 comp rx=0 tx=0).stop 2024-03-15T12:59:49.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.010+0000 7fe84293a640 1 -- 172.21.15.2:0/2923301626 shutdown_connections 2024-03-15T12:59:49.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.010+0000 7fe84293a640 1 --2- 172.21.15.2:0/2923301626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe83c10dd30 0x7fe83c110140 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.010+0000 7fe84293a640 1 --2- 172.21.15.2:0/2923301626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe83c0741e0 0x7fe83c10d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.010+0000 7fe84293a640 1 --2- 172.21.15.2:0/2923301626 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c0738c0 0x7fe83c073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:49.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.010+0000 7fe84293a640 1 -- 172.21.15.2:0/2923301626 >> 172.21.15.2:0/2923301626 conn(0x7fe83c0fcf20 msgr2=0x7fe83c0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:49.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.010+0000 7fe84293a640 1 -- 172.21.15.2:0/2923301626 shutdown_connections 2024-03-15T12:59:49.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.010+0000 7fe84293a640 1 -- 172.21.15.2:0/2923301626 wait complete. 2024-03-15T12:59:49.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.010+0000 7fe84293a640 1 Processor -- start 2024-03-15T12:59:49.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.010+0000 7fe84293a640 1 -- start start 2024-03-15T12:59:49.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.011+0000 7fe84293a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c0738c0 0x7fe83c19fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.011+0000 7fe84293a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe83c0741e0 0x7fe83c19ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.011+0000 7fe84293a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe83c10dd30 0x7fe83c199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:49.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.011+0000 7fe84293a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c19a2b0 con 0x7fe83c0741e0 2024-03-15T12:59:49.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.011+0000 7fe84293a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c19a420 con 0x7fe83c10dd30 2024-03-15T12:59:49.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.011+0000 7fe84293a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c19a5e0 con 0x7fe83c0738c0 2024-03-15T12:59:49.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.011+0000 7fe83b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe83c0741e0 0x7fe83c19ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.011+0000 7fe83bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c0738c0 0x7fe83c19fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.011+0000 7fe83bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c0738c0 0x7fe83c19fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:60232/0 (socket says 172.21.15.2:60232) 2024-03-15T12:59:49.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.011+0000 7fe83bfff640 1 -- 172.21.15.2:0/1926957195 learned_addr learned my addr 172.21.15.2:0/1926957195 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:49.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.011+0000 7fe840ed7640 1 --2- 172.21.15.2:0/1926957195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe83c10dd30 0x7fe83c199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:49.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.011+0000 7fe83bfff640 1 -- 172.21.15.2:0/1926957195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe83c10dd30 msgr2=0x7fe83c199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.011+0000 7fe83bfff640 1 --2- 172.21.15.2:0/1926957195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe83c10dd30 0x7fe83c199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.011+0000 7fe83bfff640 1 -- 172.21.15.2:0/1926957195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe83c0741e0 msgr2=0x7fe83c19ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.011+0000 7fe83bfff640 1 --2- 172.21.15.2:0/1926957195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe83c0741e0 0x7fe83c19ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.012+0000 7fe83bfff640 1 -- 172.21.15.2:0/1926957195 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe834009900 con 0x7fe83c0738c0 2024-03-15T12:59:50.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.012+0000 7fe83bfff640 1 --2- 172.21.15.2:0/1926957195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c0738c0 0x7fe83c19fa60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00e9f0 tx=0x7fe82c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:50.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.012+0000 7fe8397fa640 1 -- 172.21.15.2:0/1926957195 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c009700 con 0x7fe83c0738c0 2024-03-15T12:59:50.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.012+0000 7fe8397fa640 1 -- 172.21.15.2:0/1926957195 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe82c00cd80 con 0x7fe83c0738c0 2024-03-15T12:59:50.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.012+0000 7fe84293a640 1 -- 172.21.15.2:0/1926957195 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c19a8c0 con 0x7fe83c0738c0 2024-03-15T12:59:50.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.012+0000 7fe8397fa640 1 -- 172.21.15.2:0/1926957195 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c018920 con 0x7fe83c0738c0 2024-03-15T12:59:50.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.012+0000 7fe84293a640 1 -- 172.21.15.2:0/1926957195 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c1a69e0 con 0x7fe83c0738c0 2024-03-15T12:59:50.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.015+0000 7fe8397fa640 1 -- 172.21.15.2:0/1926957195 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7fe82c018a80 con 0x7fe83c0738c0 2024-03-15T12:59:50.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.015+0000 7fe8455ec740 1 Processor -- start 2024-03-15T12:59:50.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.015+0000 7fe8455ec740 1 -- start start 2024-03-15T12:59:50.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe8455ec740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51250 0x561eeec51630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe8397fa640 1 --2- 172.21.15.2:0/1926957195 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe8240a1cc0 0x7fe8240a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe8455ec740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51b70 0x561eeec5a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe8455ec740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 0x561eeec5cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe8455ec740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eeec0c960 con 0x561eeec5a750 2024-03-15T12:59:50.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe8455ec740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eeebc0310 con 0x561eeec51250 2024-03-15T12:59:50.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe8455ec740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eeebbd840 con 0x561eeec51b70 2024-03-15T12:59:50.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe840ed7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51250 0x561eeec51630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe840ed7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51250 0x561eeec51630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39480/0 (socket says 172.21.15.2:39480) 2024-03-15T12:59:50.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe840ed7640 1 -- 172.21.15.2:0/2337205933 learned_addr learned my addr 172.21.15.2:0/2337205933 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:50.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe83bfff640 1 --2- 172.21.15.2:0/1926957195 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe8240a1cc0 0x7fe8240a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe840ed7640 1 --2- 172.21.15.2:0/2337205933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 0x561eeec5cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe83b7fe640 1 --2- 172.21.15.2:0/2337205933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51b70 0x561eeec5a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe8397fa640 1 -- 172.21.15.2:0/1926957195 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fe82c014070 con 0x7fe83c0738c0 2024-03-15T12:59:50.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe840ed7640 1 -- 172.21.15.2:0/2337205933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51b70 msgr2=0x561eeec5a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:50.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe840ed7640 1 --2- 172.21.15.2:0/2337205933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51b70 0x561eeec5a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe840ed7640 1 -- 172.21.15.2:0/2337205933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 msgr2=0x561eeec5cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:50.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe840ed7640 1 --2- 172.21.15.2:0/2337205933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 0x561eeec5cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.016+0000 7fe840ed7640 1 -- 172.21.15.2:0/2337205933 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561eeeb7b060 con 0x561eeec51250 2024-03-15T12:59:50.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.017+0000 7fe840ed7640 1 --2- 172.21.15.2:0/2337205933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51250 0x561eeec51630 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe834005cc0 tx=0x7fe83402c180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=126b8767595f6ef7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:50.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.017+0000 7fe83bfff640 1 --2- 172.21.15.2:0/1926957195 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe8240a1cc0 0x7fe8240a4180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0cb410 tx=0x7fe82c00cf60 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:50.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.017+0000 7fe8227fc640 1 -- 172.21.15.2:0/2337205933 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834034640 con 0x561eeec51250 2024-03-15T12:59:50.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.017+0000 7fe8227fc640 1 -- 172.21.15.2:0/2337205933 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe834003e40 con 0x561eeec51250 2024-03-15T12:59:50.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.017+0000 7fe8227fc640 1 -- 172.21.15.2:0/2337205933 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834032e10 con 0x561eeec51250 2024-03-15T12:59:50.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.018+0000 7fe8455ec740 1 -- 172.21.15.2:0/2337205933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51250 msgr2=0x561eeec51630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.018+0000 7fe8455ec740 1 --2- 172.21.15.2:0/2337205933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51250 0x561eeec51630 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe834005cc0 tx=0x7fe83402c180 comp rx=0 tx=0).stop 2024-03-15T12:59:50.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.018+0000 7fe8455ec740 1 -- 172.21.15.2:0/2337205933 shutdown_connections 2024-03-15T12:59:50.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.018+0000 7fe8455ec740 1 --2- 172.21.15.2:0/2337205933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 0x561eeec5cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.018+0000 7fe8455ec740 1 --2- 172.21.15.2:0/2337205933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51b70 0x561eeec5a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.018+0000 7fe8455ec740 1 --2- 172.21.15.2:0/2337205933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51250 0x561eeec51630 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.018+0000 7fe8455ec740 1 -- 172.21.15.2:0/2337205933 >> 172.21.15.2:0/2337205933 conn(0x561eeec4b980 msgr2=0x561eeec4bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:50.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.018+0000 7fe8455ec740 1 -- 172.21.15.2:0/2337205933 shutdown_connections 2024-03-15T12:59:50.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.018+0000 7fe8455ec740 1 -- 172.21.15.2:0/2337205933 wait complete. 2024-03-15T12:59:50.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.019+0000 7fe8455ec740 1 -- start start 2024-03-15T12:59:50.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.019+0000 7fe8455ec740 1 Processor -- start 2024-03-15T12:59:50.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.019+0000 7fe8455ec740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51250 0x561eeec5a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.019+0000 7fe8455ec740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51b70 0x561eeec574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.019+0000 7fe8455ec740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 0x561eeec57ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.019+0000 7fe8455ec740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eeebb9020 con 0x561eeec5a750 2024-03-15T12:59:50.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe8455ec740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eee88c430 con 0x561eeec51b70 2024-03-15T12:59:50.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe8455ec740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eee848e00 con 0x561eeec51250 2024-03-15T12:59:50.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe83b7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51b70 0x561eeec574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe83b7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51b70 0x561eeec574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39494/0 (socket says 172.21.15.2:39494) 2024-03-15T12:59:50.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe83b7fe640 1 -- 172.21.15.2:0/3664741858 learned_addr learned my addr 172.21.15.2:0/3664741858 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:50.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe8455ec740 10 client.0 _ll_register_callbacks cb 0x561eeeb60cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:50.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe840ed7640 1 --2- 172.21.15.2:0/3664741858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 0x561eeec57ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe83b7fe640 1 -- 172.21.15.2:0/3664741858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51250 msgr2=0x561eeec5a130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-15T12:59:50.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe83b7fe640 1 --2- 172.21.15.2:0/3664741858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51250 0x561eeec5a130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe83b7fe640 1 -- 172.21.15.2:0/3664741858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 msgr2=0x561eeec57ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:50.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe83b7fe640 1 --2- 172.21.15.2:0/3664741858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 0x561eeec57ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe83b7fe640 1 -- 172.21.15.2:0/3664741858 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c110e10 con 0x561eeec51b70 2024-03-15T12:59:50.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe840ed7640 1 --2- 172.21.15.2:0/3664741858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 0x561eeec57ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:59:50.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe83b7fe640 1 --2- 172.21.15.2:0/3664741858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51b70 0x561eeec574c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe830012700 tx=0x7fe830012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=237577505c0314cb server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:50.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe81b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T12:59:50.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe81b7fe640 1 -- 172.21.15.2:0/3664741858 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830017070 con 0x561eeec51b70 2024-03-15T12:59:50.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe8455ec740 1 -- 172.21.15.2:0/3664741858 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561eee8b87e0 con 0x561eeec51b70 2024-03-15T12:59:50.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.020+0000 7fe81b7fe640 1 -- 172.21.15.2:0/3664741858 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fe830004050 con 0x561eeec51b70 2024-03-15T12:59:50.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.021+0000 7fe81b7fe640 1 -- 172.21.15.2:0/3664741858 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe830004210 con 0x561eeec51b70 2024-03-15T12:59:50.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.021+0000 7fe8455ec740 10 client.25937 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:50.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.021+0000 7fe8455ec740 1 -- 172.21.15.2:0/3664741858 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561eeebb76a0 con 0x561eeec51b70 2024-03-15T12:59:50.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.021+0000 7fe81b7fe640 1 -- 172.21.15.2:0/3664741858 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x561eee8b87e0 con 0x561eeec51b70 2024-03-15T12:59:50.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.021+0000 7fe81b7fe640 1 -- 172.21.15.2:0/3664741858 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe8300039c0 con 0x561eeec51b70 2024-03-15T12:59:50.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.021+0000 7fe8455ec740 10 client.25937 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:50.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.021+0000 7fe8455ec740 10 client.25937 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T12:59:50.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.021+0000 7fe8455ec740 20 client.25937 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:50.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.021+0000 7fe8455ec740 1 --2- 172.21.15.2:0/3664741858 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561eeec6a920 0x561eeec6cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.021+0000 7fe8455ec740 4 client.25937 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:50.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.021+0000 7fe8455ec740 1 -- 172.21.15.2:0/3664741858 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561eee8f5720 con 0x561eeec6a920 2024-03-15T12:59:50.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.021+0000 7fe840ed7640 1 --2- 172.21.15.2:0/3664741858 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561eeec6a920 0x561eeec6cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.022+0000 7fe840ed7640 1 --2- 172.21.15.2:0/3664741858 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561eeec6a920 0x561eeec6cd60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f562bb49f464cf3b server_cookie=5a1426c1d9a9dd13 in_seq=0 out_seq=0 2024-03-15T12:59:50.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.022+0000 7fe81b7fe640 10 client.25937 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T12:59:50.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.023+0000 7fe81b7fe640 1 -- 172.21.15.2:0/3664741858 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561eee8f5720 con 0x561eeec6a920 2024-03-15T12:59:50.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.023+0000 7fe81b7fe640 10 client.25937 handle_command_reply: tid=0 2024-03-15T12:59:50.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.023+0000 7fe8455ec740 1 client.25937 shutdown 2024-03-15T12:59:50.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.023+0000 7fe8455ec740 1 -- 172.21.15.2:0/3664741858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51b70 msgr2=0x561eeec574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.023+0000 7fe8455ec740 1 --2- 172.21.15.2:0/3664741858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51b70 0x561eeec574c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe830012700 tx=0x7fe830012bc0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.023+0000 7fe8455ec740 1 -- 172.21.15.2:0/3664741858 shutdown_connections 2024-03-15T12:59:50.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.023+0000 7fe8455ec740 1 --2- 172.21.15.2:0/3664741858 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561eeec6a920 0x561eeec6cd60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.023+0000 7fe81b7fe640 0 client.25937 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T12:59:50.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.023+0000 7fe8455ec740 1 --2- 172.21.15.2:0/3664741858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 0x561eeec57ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.023+0000 7fe8455ec740 1 --2- 172.21.15.2:0/3664741858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51b70 0x561eeec574c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.023+0000 7fe8455ec740 1 --2- 172.21.15.2:0/3664741858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51250 0x561eeec5a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.023+0000 7fe8455ec740 1 -- 172.21.15.2:0/3664741858 >> 172.21.15.2:0/3664741858 conn(0x561eeec4b980 msgr2=0x561eeec4d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:50.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.023+0000 7fe8455ec740 1 -- 172.21.15.2:0/3664741858 shutdown_connections 2024-03-15T12:59:50.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.023+0000 7fe8455ec740 1 -- 172.21.15.2:0/3664741858 wait complete. 2024-03-15T12:59:50.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.023+0000 7fe8455ec740 20 client.25937 trim_cache size 0 max 16384 2024-03-15T12:59:50.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.048+0000 7fe8455ec740 1 Processor -- start 2024-03-15T12:59:50.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.048+0000 7fe8455ec740 1 -- start start 2024-03-15T12:59:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.048+0000 7fe8455ec740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51250 0x561eeec528c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.048+0000 7fe8455ec740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51b70 0x561eeec59bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.048+0000 7fe8455ec740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 0x561eeec5a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.048+0000 7fe8455ec740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eeec66d50 con 0x561eeec5a750 2024-03-15T12:59:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.048+0000 7fe8455ec740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eeeb3e130 con 0x561eeec51250 2024-03-15T12:59:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.048+0000 7fe8455ec740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eeec50c90 con 0x561eeec51b70 2024-03-15T12:59:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.049+0000 7fe840ed7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51250 0x561eeec528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.049+0000 7fe83b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51b70 0x561eeec59bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.049+0000 7fe840ed7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51250 0x561eeec528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39508/0 (socket says 172.21.15.2:39508) 2024-03-15T12:59:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.049+0000 7fe840ed7640 1 -- 172.21.15.2:0/185708923 learned_addr learned my addr 172.21.15.2:0/185708923 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.049+0000 7fe840ed7640 1 --2- 172.21.15.2:0/185708923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 0x561eeec5a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.049+0000 7fe83b7fe640 1 -- 172.21.15.2:0/185708923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51250 msgr2=0x561eeec528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.049+0000 7fe83b7fe640 1 --2- 172.21.15.2:0/185708923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51250 0x561eeec528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.049+0000 7fe83b7fe640 1 -- 172.21.15.2:0/185708923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 msgr2=0x561eeec5a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:50.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.049+0000 7fe83b7fe640 1 --2- 172.21.15.2:0/185708923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 0x561eeec5a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.049+0000 7fe83b7fe640 1 -- 172.21.15.2:0/185708923 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561eeeb53c20 con 0x561eeec51b70 2024-03-15T12:59:50.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.049+0000 7fe840ed7640 1 --2- 172.21.15.2:0/185708923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51250 0x561eeec528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:50.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.049+0000 7fe83b7fe640 1 --2- 172.21.15.2:0/185708923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51b70 0x561eeec59bf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe8300126d0 tx=0x7fe830005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb09c97772f02645 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:50.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.049+0000 7fe821ffb640 1 -- 172.21.15.2:0/185708923 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830017040 con 0x561eeec51b70 2024-03-15T12:59:50.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.049+0000 7fe821ffb640 1 -- 172.21.15.2:0/185708923 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe830003ac0 con 0x561eeec51b70 2024-03-15T12:59:50.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.049+0000 7fe821ffb640 1 -- 172.21.15.2:0/185708923 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830010a10 con 0x561eeec51b70 2024-03-15T12:59:50.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.050+0000 7fe8455ec740 1 -- 172.21.15.2:0/185708923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51b70 msgr2=0x561eeec59bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.050+0000 7fe8455ec740 1 --2- 172.21.15.2:0/185708923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51b70 0x561eeec59bf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe8300126d0 tx=0x7fe830005e30 comp rx=0 tx=0).stop 2024-03-15T12:59:50.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.050+0000 7fe8455ec740 1 -- 172.21.15.2:0/185708923 shutdown_connections 2024-03-15T12:59:50.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.050+0000 7fe8455ec740 1 --2- 172.21.15.2:0/185708923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec5a750 0x561eeec5a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.050+0000 7fe8455ec740 1 --2- 172.21.15.2:0/185708923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51b70 0x561eeec59bf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.050+0000 7fe8455ec740 1 --2- 172.21.15.2:0/185708923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec51250 0x561eeec528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.050+0000 7fe8455ec740 1 -- 172.21.15.2:0/185708923 >> 172.21.15.2:0/185708923 conn(0x561eeec4b980 msgr2=0x561eeec521b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:50.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.050+0000 7fe8455ec740 1 -- 172.21.15.2:0/185708923 shutdown_connections 2024-03-15T12:59:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.051+0000 7fe8455ec740 1 -- 172.21.15.2:0/185708923 wait complete. 2024-03-15T12:59:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.051+0000 7fe8455ec740 1 -- start start 2024-03-15T12:59:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.051+0000 7fe8455ec740 1 Processor -- start 2024-03-15T12:59:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe8455ec740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51250 0x561eeec55130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe8455ec740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec51b70 0x561eeec5f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe840ed7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51250 0x561eeec55130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe840ed7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51250 0x561eeec55130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:60264/0 (socket says 172.21.15.2:60264) 2024-03-15T12:59:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe83b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec51b70 0x561eeec5f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe83b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec51b70 0x561eeec5f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42302/0 (socket says 172.21.15.2:42302) 2024-03-15T12:59:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe840ed7640 1 -- 172.21.15.2:0/1712531714 learned_addr learned my addr 172.21.15.2:0/1712531714 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe8455ec740 1 --2- 172.21.15.2:0/1712531714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec5a750 0x561eeec5fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe8455ec740 1 -- 172.21.15.2:0/1712531714 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eeec600a0 con 0x561eeec51b70 2024-03-15T12:59:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe8455ec740 1 -- 172.21.15.2:0/1712531714 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eeec60210 con 0x561eeec5a750 2024-03-15T12:59:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe8455ec740 1 -- 172.21.15.2:0/1712531714 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eeec603d0 con 0x561eeec51250 2024-03-15T12:59:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe8455ec740 10 client.0 _ll_register_callbacks cb 0x7fe830009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe840ed7640 1 --2- 172.21.15.2:0/1712531714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec5a750 0x561eeec5fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe840ed7640 1 -- 172.21.15.2:0/1712531714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec5a750 msgr2=0x561eeec5fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe840ed7640 1 --2- 172.21.15.2:0/1712531714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec5a750 0x561eeec5fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe840ed7640 1 -- 172.21.15.2:0/1712531714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec51b70 msgr2=0x561eeec5f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe840ed7640 1 --2- 172.21.15.2:0/1712531714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec51b70 0x561eeec5f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.052+0000 7fe840ed7640 1 -- 172.21.15.2:0/1712531714 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561eeeb7b060 con 0x561eeec51250 2024-03-15T12:59:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.053+0000 7fe840ed7640 1 --2- 172.21.15.2:0/1712531714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51250 0x561eeec55130 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe83402c790 tx=0x7fe83402c180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16a5e749714fd0a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.053+0000 7fe81b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T12:59:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.053+0000 7fe81b7fe640 1 -- 172.21.15.2:0/1712531714 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834004d40 con 0x561eeec51250 2024-03-15T12:59:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.053+0000 7fe81b7fe640 1 -- 172.21.15.2:0/1712531714 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fe834034440 con 0x561eeec51250 2024-03-15T12:59:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.053+0000 7fe8455ec740 1 -- 172.21.15.2:0/1712531714 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe834003e40 con 0x561eeec51250 2024-03-15T12:59:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.053+0000 7fe81b7fe640 1 -- 172.21.15.2:0/1712531714 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe834034600 con 0x561eeec51250 2024-03-15T12:59:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.053+0000 7fe8455ec740 10 client.25867 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.053+0000 7fe8455ec740 1 -- 172.21.15.2:0/1712531714 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561eeebb76a0 con 0x561eeec51250 2024-03-15T12:59:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.053+0000 7fe81b7fe640 1 -- 172.21.15.2:0/1712531714 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe834003e40 con 0x561eeec51250 2024-03-15T12:59:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.054+0000 7fe81b7fe640 1 -- 172.21.15.2:0/1712531714 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe83404b040 con 0x561eeec51250 2024-03-15T12:59:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.054+0000 7fe8455ec740 10 client.25867 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.054+0000 7fe8455ec740 10 client.25867 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T12:59:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.054+0000 7fe8455ec740 20 client.25867 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.054+0000 7fe8455ec740 1 --2- 172.21.15.2:0/1712531714 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561eeec6a920 0x561eeecc93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.054+0000 7fe8455ec740 4 client.25867 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T12:59:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.054+0000 7fe8455ec740 1 -- 172.21.15.2:0/1712531714 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561eee944890 con 0x561eeec6a920 2024-03-15T12:59:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.054+0000 7fe83b7fe640 1 --2- 172.21.15.2:0/1712531714 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561eeec6a920 0x561eeecc93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.054+0000 7fe81b7fe640 10 client.25867 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T12:59:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.054+0000 7fe83b7fe640 1 --2- 172.21.15.2:0/1712531714 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561eeec6a920 0x561eeecc93c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d9e95e8968fab84 server_cookie=5f4ceba7e8d47eea in_seq=0 out_seq=0 2024-03-15T12:59:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.054+0000 7fe81b7fe640 1 -- 172.21.15.2:0/1712531714 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561eee944890 con 0x561eeec6a920 2024-03-15T12:59:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.055+0000 7fe81b7fe640 10 client.25867 handle_command_reply: tid=0 2024-03-15T12:59:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.055+0000 7fe8455ec740 1 client.25867 shutdown 2024-03-15T12:59:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.055+0000 7fe8455ec740 1 -- 172.21.15.2:0/1712531714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51250 msgr2=0x561eeec55130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.055+0000 7fe8455ec740 1 --2- 172.21.15.2:0/1712531714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51250 0x561eeec55130 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe83402c790 tx=0x7fe83402c180 comp rx=0 tx=0).stop 2024-03-15T12:59:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.055+0000 7fe8455ec740 1 -- 172.21.15.2:0/1712531714 shutdown_connections 2024-03-15T12:59:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.055+0000 7fe8455ec740 1 --2- 172.21.15.2:0/1712531714 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x561eeec6a920 0x561eeecc93c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.055+0000 7fe81b7fe640 0 client.25867 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T12:59:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.055+0000 7fe8455ec740 1 --2- 172.21.15.2:0/1712531714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561eeec5a750 0x561eeec5fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.055+0000 7fe8455ec740 1 --2- 172.21.15.2:0/1712531714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561eeec51b70 0x561eeec5f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.055+0000 7fe8455ec740 1 --2- 172.21.15.2:0/1712531714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561eeec51250 0x561eeec55130 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.055+0000 7fe8455ec740 1 -- 172.21.15.2:0/1712531714 >> 172.21.15.2:0/1712531714 conn(0x561eeec4b980 msgr2=0x561eeec52a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.055+0000 7fe8455ec740 1 -- 172.21.15.2:0/1712531714 shutdown_connections 2024-03-15T12:59:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.055+0000 7fe8455ec740 1 -- 172.21.15.2:0/1712531714 wait complete. 2024-03-15T12:59:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.055+0000 7fe8455ec740 20 client.25867 trim_cache size 0 max 16384 2024-03-15T12:59:50.046 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T12:59:50.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.056+0000 7fe84293a640 1 -- 172.21.15.2:0/1926957195 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe8240a1cc0 msgr2=0x7fe8240a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.056+0000 7fe84293a640 1 --2- 172.21.15.2:0/1926957195 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe8240a1cc0 0x7fe8240a4180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0cb410 tx=0x7fe82c00cf60 comp rx=0 tx=0).stop 2024-03-15T12:59:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.056+0000 7fe84293a640 1 -- 172.21.15.2:0/1926957195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c0738c0 msgr2=0x7fe83c19fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.056+0000 7fe84293a640 1 --2- 172.21.15.2:0/1926957195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c0738c0 0x7fe83c19fa60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00e9f0 tx=0x7fe82c00eeb0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.056+0000 7fe84293a640 1 -- 172.21.15.2:0/1926957195 shutdown_connections 2024-03-15T12:59:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.056+0000 7fe84293a640 1 --2- 172.21.15.2:0/1926957195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe83c10dd30 0x7fe83c199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.056+0000 7fe84293a640 1 --2- 172.21.15.2:0/1926957195 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe8240a1cc0 0x7fe8240a4180 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.056+0000 7fe84293a640 1 --2- 172.21.15.2:0/1926957195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe83c0741e0 0x7fe83c19ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.056+0000 7fe84293a640 1 --2- 172.21.15.2:0/1926957195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c0738c0 0x7fe83c19fa60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.056+0000 7fe84293a640 1 -- 172.21.15.2:0/1926957195 >> 172.21.15.2:0/1926957195 conn(0x7fe83c0fcf20 msgr2=0x7fe83c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.056+0000 7fe84293a640 1 -- 172.21.15.2:0/1926957195 shutdown_connections 2024-03-15T12:59:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.056+0000 7fe84293a640 1 -- 172.21.15.2:0/1926957195 wait complete. 2024-03-15T12:59:50.057 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T12:59:50.057 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T12:59:50.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:49 smithi184 ceph-mon[35993]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:50.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:49 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1400582507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:59:50.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:49 smithi149 ceph-mon[35927]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:50.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:49 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1400582507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T12:59:50.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.182+0000 7f3de037a640 1 -- 172.21.15.2:0/2203465003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dd81005d0 msgr2=0x7f3dd81009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.182+0000 7f3de037a640 1 --2- 172.21.15.2:0/2203465003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dd81005d0 0x7f3dd81009b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009930 tx=0x7f3dd402bc30 comp rx=0 tx=0).stop 2024-03-15T12:59:50.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.182+0000 7f3de037a640 1 -- 172.21.15.2:0/2203465003 shutdown_connections 2024-03-15T12:59:50.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.182+0000 7f3de037a640 1 --2- 172.21.15.2:0/2203465003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dd81056a0 0x7f3dd810d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.182+0000 7f3de037a640 1 --2- 172.21.15.2:0/2203465003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd8100f80 0x7f3dd8104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.182+0000 7f3de037a640 1 --2- 172.21.15.2:0/2203465003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dd81005d0 0x7f3dd81009b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.182+0000 7f3de037a640 1 -- 172.21.15.2:0/2203465003 >> 172.21.15.2:0/2203465003 conn(0x7f3dd80774f0 msgr2=0x7f3dd8077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:50.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.182+0000 7f3de037a640 1 -- 172.21.15.2:0/2203465003 shutdown_connections 2024-03-15T12:59:50.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.182+0000 7f3de037a640 1 -- 172.21.15.2:0/2203465003 wait complete. 2024-03-15T12:59:50.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.183+0000 7f3de037a640 1 Processor -- start 2024-03-15T12:59:50.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.183+0000 7f3de037a640 1 -- start start 2024-03-15T12:59:50.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.183+0000 7f3de037a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dd81005d0 0x7f3dd8199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.183+0000 7f3de037a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd81056a0 0x7f3dd819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.183+0000 7f3de037a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dd819e610 0x7f3dd819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.183+0000 7f3de037a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd819f000 con 0x7f3dd81056a0 2024-03-15T12:59:50.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.183+0000 7f3de037a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd819f170 con 0x7f3dd81005d0 2024-03-15T12:59:50.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.183+0000 7f3de037a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd819f2e0 con 0x7f3dd819e610 2024-03-15T12:59:50.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.183+0000 7f3ddd915640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd81056a0 0x7f3dd819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.183+0000 7f3ddd915640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd81056a0 0x7f3dd819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42316/0 (socket says 172.21.15.2:42316) 2024-03-15T12:59:50.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.183+0000 7f3ddd915640 1 -- 172.21.15.2:0/674518802 learned_addr learned my addr 172.21.15.2:0/674518802 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:50.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.183+0000 7f3dde917640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dd819e610 0x7f3dd819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.184+0000 7f3dde116640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dd81005d0 0x7f3dd8199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.184+0000 7f3dde917640 1 -- 172.21.15.2:0/674518802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dd81005d0 msgr2=0x7f3dd8199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.184+0000 7f3dde917640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dd81005d0 0x7f3dd8199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.184+0000 7f3dde917640 1 -- 172.21.15.2:0/674518802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd81056a0 msgr2=0x7f3dd819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.184+0000 7f3dde917640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd81056a0 0x7f3dd819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.184+0000 7f3dde917640 1 -- 172.21.15.2:0/674518802 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4009560 con 0x7f3dd819e610 2024-03-15T12:59:50.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.184+0000 7f3ddd915640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd81056a0 0x7f3dd819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:50.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.184+0000 7f3dde116640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dd81005d0 0x7f3dd8199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T12:59:50.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.184+0000 7f3dde917640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dd819e610 0x7f3dd819eac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009fd0 tx=0x7f3dd000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:50.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.184+0000 7f3dcf7fe640 1 -- 172.21.15.2:0/674518802 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd000ce20 con 0x7f3dd819e610 2024-03-15T12:59:50.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.184+0000 7f3dcf7fe640 1 -- 172.21.15.2:0/674518802 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3dd0005020 con 0x7f3dd819e610 2024-03-15T12:59:50.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.184+0000 7f3dcf7fe640 1 -- 172.21.15.2:0/674518802 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0010690 con 0x7f3dd819e610 2024-03-15T12:59:50.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.184+0000 7f3de037a640 1 -- 172.21.15.2:0/674518802 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd81a68c0 con 0x7f3dd819e610 2024-03-15T12:59:50.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.184+0000 7f3de037a640 1 -- 172.21.15.2:0/674518802 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd81a6d20 con 0x7f3dd819e610 2024-03-15T12:59:50.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.187+0000 7f3dcf7fe640 1 -- 172.21.15.2:0/674518802 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7f3dd0005570 con 0x7f3dd819e610 2024-03-15T12:59:50.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.188+0000 7f3de3032740 1 Processor -- start 2024-03-15T12:59:50.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.188+0000 7f3de3032740 1 -- start start 2024-03-15T12:59:50.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.188+0000 7f3de3032740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1250 0x55cf857f1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.188+0000 7f3de3032740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1b70 0x55cf857fa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.188+0000 7f3de3032740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857fa750 0x55cf857fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.188+0000 7f3dcf7fe640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3dc40a1cc0 0x7f3dc40a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.188+0000 7f3de3032740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf857ac960 con 0x55cf857f1b70 2024-03-15T12:59:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.188+0000 7f3de3032740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf85760310 con 0x55cf857f1250 2024-03-15T12:59:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.188+0000 7f3de3032740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8575d840 con 0x55cf857fa750 2024-03-15T12:59:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3ddd915640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1250 0x55cf857f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3dde917640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3dc40a1cc0 0x7f3dc40a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3ddd915640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1250 0x55cf857f1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39540/0 (socket says 172.21.15.2:39540) 2024-03-15T12:59:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3ddd915640 1 -- 172.21.15.2:0/2295555373 learned_addr learned my addr 172.21.15.2:0/2295555373 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3ddd915640 1 --2- 172.21.15.2:0/2295555373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857fa750 0x55cf857fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3dcf7fe640 1 -- 172.21.15.2:0/674518802 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f3dd0014070 con 0x7f3dd819e610 2024-03-15T12:59:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3ddd915640 1 -- 172.21.15.2:0/2295555373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857fa750 msgr2=0x55cf857fcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3ddd915640 1 --2- 172.21.15.2:0/2295555373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857fa750 0x55cf857fcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3ddd915640 1 -- 172.21.15.2:0/2295555373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1b70 msgr2=0x55cf857fa210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-15T12:59:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3dde116640 1 --2- 172.21.15.2:0/2295555373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1b70 0x55cf857fa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3ddd915640 1 --2- 172.21.15.2:0/2295555373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1b70 0x55cf857fa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3ddd915640 1 -- 172.21.15.2:0/2295555373 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf8571b060 con 0x55cf857f1250 2024-03-15T12:59:50.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3dde917640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3dc40a1cc0 0x7f3dc40a4180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00cb870 tx=0x7f3dd0096ec0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:50.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3dde116640 1 --2- 172.21.15.2:0/2295555373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1b70 0x55cf857fa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T12:59:50.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3ddd915640 1 --2- 172.21.15.2:0/2295555373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1250 0x55cf857f1630 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8011c70 tx=0x7f3dc8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3e56801648b9aa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:50.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.189+0000 7f3dccbfe640 1 -- 172.21.15.2:0/2295555373 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc800fc50 con 0x55cf857f1250 2024-03-15T12:59:50.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.190+0000 7f3dccbfe640 1 -- 172.21.15.2:0/2295555373 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3dc800fdb0 con 0x55cf857f1250 2024-03-15T12:59:50.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.190+0000 7f3dccbfe640 1 -- 172.21.15.2:0/2295555373 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc801e3f0 con 0x55cf857f1250 2024-03-15T12:59:50.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.190+0000 7f3de3032740 1 -- 172.21.15.2:0/2295555373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1250 msgr2=0x55cf857f1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.190+0000 7f3de3032740 1 --2- 172.21.15.2:0/2295555373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1250 0x55cf857f1630 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8011c70 tx=0x7f3dc8004730 comp rx=0 tx=0).stop 2024-03-15T12:59:50.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.190+0000 7f3de3032740 1 -- 172.21.15.2:0/2295555373 shutdown_connections 2024-03-15T12:59:50.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.190+0000 7f3de3032740 1 --2- 172.21.15.2:0/2295555373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857fa750 0x55cf857fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.190+0000 7f3de3032740 1 --2- 172.21.15.2:0/2295555373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1b70 0x55cf857fa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.190+0000 7f3de3032740 1 --2- 172.21.15.2:0/2295555373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1250 0x55cf857f1630 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.190+0000 7f3de3032740 1 -- 172.21.15.2:0/2295555373 >> 172.21.15.2:0/2295555373 conn(0x55cf857eb980 msgr2=0x55cf857ebd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:50.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.190+0000 7f3de3032740 1 -- 172.21.15.2:0/2295555373 shutdown_connections 2024-03-15T12:59:50.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.190+0000 7f3de3032740 1 -- 172.21.15.2:0/2295555373 wait complete. 2024-03-15T12:59:50.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.191+0000 7f3de3032740 1 -- start start 2024-03-15T12:59:50.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.191+0000 7f3de3032740 1 Processor -- start 2024-03-15T12:59:50.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.191+0000 7f3de3032740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf857fa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.191+0000 7f3de3032740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1b70 0x55cf857f7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.191+0000 7f3de3032740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857fa750 0x55cf857f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.191+0000 7f3ddd915640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf857fa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.191+0000 7f3de3032740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf85759020 con 0x55cf857f1250 2024-03-15T12:59:50.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.191+0000 7f3de3032740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8542c430 con 0x55cf857f1b70 2024-03-15T12:59:50.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.191+0000 7f3ddd915640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf857fa0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42334/0 (socket says 172.21.15.2:42334) 2024-03-15T12:59:50.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.191+0000 7f3de3032740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf853e8e00 con 0x55cf857fa750 2024-03-15T12:59:50.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.191+0000 7f3ddd915640 1 -- 172.21.15.2:0/2025007131 learned_addr learned my addr 172.21.15.2:0/2025007131 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:50.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.191+0000 7f3dde116640 1 --2- 172.21.15.2:0/2025007131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1b70 0x55cf857f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.191+0000 7f3de3032740 10 client.0 _ll_register_callbacks cb 0x55cf85700cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:50.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.191+0000 7f3ddd915640 1 --2- 172.21.15.2:0/2025007131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857fa750 0x55cf857f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.192+0000 7f3ddd915640 1 -- 172.21.15.2:0/2025007131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857fa750 msgr2=0x55cf857f7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:50.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.192+0000 7f3ddd915640 1 --2- 172.21.15.2:0/2025007131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857fa750 0x55cf857f7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.192+0000 7f3ddd915640 1 -- 172.21.15.2:0/2025007131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1b70 msgr2=0x55cf857f7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:50.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.192+0000 7f3ddd915640 1 --2- 172.21.15.2:0/2025007131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1b70 0x55cf857f7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.192+0000 7f3ddd915640 1 -- 172.21.15.2:0/2025007131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc8019040 con 0x55cf857f1250 2024-03-15T12:59:50.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.192+0000 7f3ddd915640 1 --2- 172.21.15.2:0/2025007131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf857fa0f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8004700 tx=0x7f3dc8004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9418edef9132adc server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:50.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.192+0000 7f3dc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:50.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.192+0000 7f3dc1ffb640 1 -- 172.21.15.2:0/2025007131 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8003800 con 0x55cf857f1250 2024-03-15T12:59:50.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.192+0000 7f3de3032740 1 -- 172.21.15.2:0/2025007131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf854587e0 con 0x55cf857f1250 2024-03-15T12:59:50.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.192+0000 7f3dc1ffb640 1 -- 172.21.15.2:0/2025007131 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f3dc8003960 con 0x55cf857f1250 2024-03-15T12:59:50.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.192+0000 7f3dc1ffb640 1 -- 172.21.15.2:0/2025007131 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dc8003b20 con 0x55cf857f1250 2024-03-15T12:59:50.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.192+0000 7f3de3032740 10 client.16083 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:50.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.192+0000 7f3de3032740 1 -- 172.21.15.2:0/2025007131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cf857576a0 con 0x55cf857f1250 2024-03-15T12:59:50.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.192+0000 7f3dc1ffb640 1 -- 172.21.15.2:0/2025007131 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55cf854587e0 con 0x55cf857f1250 2024-03-15T12:59:50.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.193+0000 7f3dc1ffb640 1 -- 172.21.15.2:0/2025007131 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3dc8005460 con 0x55cf857f1250 2024-03-15T12:59:50.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.193+0000 7f3de3032740 10 client.16083 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:50.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.193+0000 7f3de3032740 10 client.16083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:50.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.193+0000 7f3de3032740 20 client.16083 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:50.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.193+0000 7f3de3032740 1 --2- 172.21.15.2:0/2025007131 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cf85812e70 0x55cf85815260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.193+0000 7f3de3032740 4 client.16083 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:50.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.193+0000 7f3de3032740 1 -- 172.21.15.2:0/2025007131 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf85495720 con 0x55cf85812e70 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.193+0000 7f3dde116640 1 --2- 172.21.15.2:0/2025007131 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cf85812e70 0x55cf85815260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.193+0000 7f3dde116640 1 --2- 172.21.15.2:0/2025007131 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cf85812e70 0x55cf85815260 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5db3cc8717a3dc62 server_cookie=db13c41a7113b05e in_seq=0 out_seq=0 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.193+0000 7f3dc1ffb640 10 client.16083 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.194+0000 7f3dc1ffb640 1 -- 172.21.15.2:0/2025007131 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cf85495720 con 0x55cf85812e70 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.194+0000 7f3dc1ffb640 10 client.16083 handle_command_reply: tid=0 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.194+0000 7f3de3032740 1 client.16083 shutdown 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.194+0000 7f3de3032740 1 -- 172.21.15.2:0/2025007131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 msgr2=0x55cf857fa0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.194+0000 7f3de3032740 1 --2- 172.21.15.2:0/2025007131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf857fa0f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8004700 tx=0x7f3dc8004b00 comp rx=0 tx=0).stop 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.195+0000 7f3de3032740 1 -- 172.21.15.2:0/2025007131 shutdown_connections 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.195+0000 7f3de3032740 1 --2- 172.21.15.2:0/2025007131 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cf85812e70 0x55cf85815260 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.195+0000 7f3de3032740 1 --2- 172.21.15.2:0/2025007131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857fa750 0x55cf857f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.195+0000 7f3de3032740 1 --2- 172.21.15.2:0/2025007131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1b70 0x55cf857f7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.195+0000 7f3de3032740 1 --2- 172.21.15.2:0/2025007131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf857fa0f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.195+0000 7f3de3032740 1 -- 172.21.15.2:0/2025007131 >> 172.21.15.2:0/2025007131 conn(0x55cf857eb980 msgr2=0x55cf857ed6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.195+0000 7f3dc1ffb640 0 client.16083 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.195+0000 7f3de3032740 1 -- 172.21.15.2:0/2025007131 shutdown_connections 2024-03-15T12:59:50.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.195+0000 7f3de3032740 1 -- 172.21.15.2:0/2025007131 wait complete. 2024-03-15T12:59:50.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.195+0000 7f3de3032740 20 client.16083 trim_cache size 0 max 16384 2024-03-15T12:59:50.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.220+0000 7f3de3032740 1 Processor -- start 2024-03-15T12:59:50.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.220+0000 7f3de3032740 1 -- start start 2024-03-15T12:59:50.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.220+0000 7f3de3032740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf857e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.220+0000 7f3de3032740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1b70 0x55cf857eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.220+0000 7f3de3032740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857fa750 0x55cf857f9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.220+0000 7f3de3032740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf856de130 con 0x55cf857f1250 2024-03-15T12:59:50.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.220+0000 7f3de3032740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf857e6f50 con 0x55cf857f1b70 2024-03-15T12:59:50.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.220+0000 7f3de3032740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8580fb90 con 0x55cf857fa750 2024-03-15T12:59:50.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.220+0000 7f3dde116640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1b70 0x55cf857eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.220+0000 7f3ddd915640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf857e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.220+0000 7f3dde116640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1b70 0x55cf857eaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39550/0 (socket says 172.21.15.2:39550) 2024-03-15T12:59:50.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.220+0000 7f3dde116640 1 -- 172.21.15.2:0/237171608 learned_addr learned my addr 172.21.15.2:0/237171608 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:50.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.220+0000 7f3ddd915640 1 --2- 172.21.15.2:0/237171608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf857e6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42336/0 (socket says 172.21.15.2:42336) 2024-03-15T12:59:50.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.220+0000 7f3ddd915640 1 --2- 172.21.15.2:0/237171608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857fa750 0x55cf857f9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.221+0000 7f3dde116640 1 -- 172.21.15.2:0/237171608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857fa750 msgr2=0x55cf857f9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:50.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.221+0000 7f3dde116640 1 --2- 172.21.15.2:0/237171608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857fa750 0x55cf857f9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.221+0000 7f3dde116640 1 -- 172.21.15.2:0/237171608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 msgr2=0x55cf857e6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:50.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.221+0000 7f3dde116640 1 --2- 172.21.15.2:0/237171608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf857e6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.221+0000 7f3dde116640 1 -- 172.21.15.2:0/237171608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf856f3c20 con 0x55cf857f1b70 2024-03-15T12:59:50.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.221+0000 7f3dde116640 1 --2- 172.21.15.2:0/237171608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1b70 0x55cf857eaa00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3dd402e010 tx=0x7f3dd4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3231be54f6c73a5b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:50.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.221+0000 7f3dc3fff640 1 -- 172.21.15.2:0/237171608 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4045040 con 0x55cf857f1b70 2024-03-15T12:59:50.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.221+0000 7f3dc3fff640 1 -- 172.21.15.2:0/237171608 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3dd4048340 con 0x55cf857f1b70 2024-03-15T12:59:50.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.221+0000 7f3dc3fff640 1 -- 172.21.15.2:0/237171608 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4004de0 con 0x55cf857f1b70 2024-03-15T12:59:50.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.222+0000 7f3de3032740 1 -- 172.21.15.2:0/237171608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1b70 msgr2=0x55cf857eaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.222+0000 7f3de3032740 1 --2- 172.21.15.2:0/237171608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1b70 0x55cf857eaa00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3dd402e010 tx=0x7f3dd4009260 comp rx=0 tx=0).stop 2024-03-15T12:59:50.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.222+0000 7f3de3032740 1 -- 172.21.15.2:0/237171608 shutdown_connections 2024-03-15T12:59:50.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.222+0000 7f3de3032740 1 --2- 172.21.15.2:0/237171608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857fa750 0x55cf857f9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.222+0000 7f3de3032740 1 --2- 172.21.15.2:0/237171608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857f1b70 0x55cf857eaa00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.222+0000 7f3de3032740 1 --2- 172.21.15.2:0/237171608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf857e6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.222+0000 7f3de3032740 1 -- 172.21.15.2:0/237171608 >> 172.21.15.2:0/237171608 conn(0x55cf857eb980 msgr2=0x55cf857ea2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:50.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.222+0000 7f3de3032740 1 -- 172.21.15.2:0/237171608 shutdown_connections 2024-03-15T12:59:50.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.222+0000 7f3de3032740 1 -- 172.21.15.2:0/237171608 wait complete. 2024-03-15T12:59:50.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.223+0000 7f3de3032740 1 -- start start 2024-03-15T12:59:50.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.223+0000 7f3de3032740 1 Processor -- start 2024-03-15T12:59:50.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.223+0000 7f3de3032740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf85806640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.223+0000 7f3de3032740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857f1b70 0x55cf85806b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.223+0000 7f3de3032740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857fa750 0x55cf8586d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.223+0000 7f3ddd915640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf85806640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.223+0000 7f3dde116640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857f1b70 0x55cf85806b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.223+0000 7f3dde116640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857f1b70 0x55cf85806b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:60302/0 (socket says 172.21.15.2:60302) 2024-03-15T12:59:50.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.223+0000 7f3ddd915640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf85806640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42344/0 (socket says 172.21.15.2:42344) 2024-03-15T12:59:50.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.223+0000 7f3dde116640 1 -- 172.21.15.2:0/3270078158 learned_addr learned my addr 172.21.15.2:0/3270078158 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:50.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.223+0000 7f3de3032740 1 -- 172.21.15.2:0/3270078158 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf85807450 con 0x55cf857f1250 2024-03-15T12:59:50.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.223+0000 7f3de3032740 1 -- 172.21.15.2:0/3270078158 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8586dc40 con 0x55cf857fa750 2024-03-15T12:59:50.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.223+0000 7f3de3032740 1 -- 172.21.15.2:0/3270078158 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8586dd80 con 0x55cf857f1b70 2024-03-15T12:59:50.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.224+0000 7f3de3032740 10 client.0 _ll_register_callbacks cb 0x7f3dc8011740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:50.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.224+0000 7f3dde116640 1 -- 172.21.15.2:0/3270078158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857fa750 msgr2=0x55cf8586d700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-15T12:59:50.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.224+0000 7f3dde116640 1 --2- 172.21.15.2:0/3270078158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857fa750 0x55cf8586d700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.224+0000 7f3dde116640 1 -- 172.21.15.2:0/3270078158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 msgr2=0x55cf85806640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:50.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.224+0000 7f3dde116640 1 --2- 172.21.15.2:0/3270078158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf85806640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.224+0000 7f3dde116640 1 -- 172.21.15.2:0/3270078158 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd8068650 con 0x55cf857f1b70 2024-03-15T12:59:50.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.224+0000 7f3ddd915640 1 --2- 172.21.15.2:0/3270078158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf85806640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:50.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.224+0000 7f3de3032740 1 -- 172.21.15.2:0/3270078158 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf857e70f0 con 0x55cf857f1b70 2024-03-15T12:59:50.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.224+0000 7f3dde116640 1 --2- 172.21.15.2:0/3270078158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857f1b70 0x55cf85806b80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3dd402c660 tx=0x7f3dd4004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbf380eb6f5f9b26 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:50.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.224+0000 7f3dc1ffb640 10 client.25885 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T12:59:50.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.224+0000 7f3dc1ffb640 1 -- 172.21.15.2:0/3270078158 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4045040 con 0x55cf857f1b70 2024-03-15T12:59:50.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.224+0000 7f3dc1ffb640 1 -- 172.21.15.2:0/3270078158 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f3dd402c810 con 0x55cf857f1b70 2024-03-15T12:59:50.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.224+0000 7f3dc1ffb640 1 -- 172.21.15.2:0/3270078158 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dd402ca50 con 0x55cf857f1b70 2024-03-15T12:59:50.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.224+0000 7f3de3032740 10 client.25885 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:50.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.224+0000 7f3de3032740 1 -- 172.21.15.2:0/3270078158 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55cf857198c0 con 0x55cf857f1b70 2024-03-15T12:59:50.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.225+0000 7f3dc1ffb640 1 -- 172.21.15.2:0/3270078158 <== mon.1 v2:172.21.15.184:3300/0 4 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3dd4004de0 con 0x55cf857f1b70 2024-03-15T12:59:50.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.225+0000 7f3de3032740 10 client.25885 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:50.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.225+0000 7f3de3032740 10 client.25885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:50.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.225+0000 7f3de3032740 20 client.25885 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:50.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.225+0000 7f3de3032740 1 --2- 172.21.15.2:0/3270078158 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cf8586ee30 0x55cf85871250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:50.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.225+0000 7f3de3032740 4 client.25885 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T12:59:50.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.225+0000 7f3de3032740 1 -- 172.21.15.2:0/3270078158 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cf857e70f0 con 0x55cf8586ee30 2024-03-15T12:59:50.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.225+0000 7f3ddd915640 1 --2- 172.21.15.2:0/3270078158 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cf8586ee30 0x55cf85871250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:50.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.225+0000 7f3dc1ffb640 10 client.25885 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:50.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.225+0000 7f3ddd915640 1 --2- 172.21.15.2:0/3270078158 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cf8586ee30 0x55cf85871250 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2918330043186b73 server_cookie=7dfb3452f6656276 in_seq=0 out_seq=0 2024-03-15T12:59:50.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.226+0000 7f3dc1ffb640 1 -- 172.21.15.2:0/3270078158 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cf857e70f0 con 0x55cf8586ee30 2024-03-15T12:59:50.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.226+0000 7f3dc1ffb640 10 client.25885 handle_command_reply: tid=0 2024-03-15T12:59:50.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.226+0000 7f3de3032740 1 client.25885 shutdown 2024-03-15T12:59:50.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.226+0000 7f3de3032740 1 -- 172.21.15.2:0/3270078158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857f1b70 msgr2=0x55cf85806b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.226+0000 7f3de3032740 1 --2- 172.21.15.2:0/3270078158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857f1b70 0x55cf85806b80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3dd402c660 tx=0x7f3dd4004170 comp rx=0 tx=0).stop 2024-03-15T12:59:50.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.226+0000 7f3de3032740 1 -- 172.21.15.2:0/3270078158 shutdown_connections 2024-03-15T12:59:50.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.226+0000 7f3de3032740 1 --2- 172.21.15.2:0/3270078158 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cf8586ee30 0x55cf85871250 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.226+0000 7f3de3032740 1 --2- 172.21.15.2:0/3270078158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf857fa750 0x55cf8586d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.226+0000 7f3de3032740 1 --2- 172.21.15.2:0/3270078158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cf857f1b70 0x55cf85806b80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.226+0000 7f3dc1ffb640 0 client.25885 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:50.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.226+0000 7f3de3032740 1 --2- 172.21.15.2:0/3270078158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cf857f1250 0x55cf85806640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.226+0000 7f3de3032740 1 -- 172.21.15.2:0/3270078158 >> 172.21.15.2:0/3270078158 conn(0x55cf857eb980 msgr2=0x55cf857eab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:50.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.226+0000 7f3de3032740 1 -- 172.21.15.2:0/3270078158 shutdown_connections 2024-03-15T12:59:50.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.226+0000 7f3de3032740 1 -- 172.21.15.2:0/3270078158 wait complete. 2024-03-15T12:59:50.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.226+0000 7f3de3032740 20 client.25885 trim_cache size 0 max 16384 2024-03-15T12:59:50.217 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T12:59:50.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.227+0000 7f3de037a640 1 -- 172.21.15.2:0/674518802 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3dc40a1cc0 msgr2=0x7f3dc40a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.227+0000 7f3de037a640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3dc40a1cc0 0x7f3dc40a4180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00cb870 tx=0x7f3dd0096ec0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.227+0000 7f3de037a640 1 -- 172.21.15.2:0/674518802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dd819e610 msgr2=0x7f3dd819eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:50.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.227+0000 7f3de037a640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dd819e610 0x7f3dd819eac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009fd0 tx=0x7f3dd000ef10 comp rx=0 tx=0).stop 2024-03-15T12:59:50.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.227+0000 7f3de037a640 1 -- 172.21.15.2:0/674518802 shutdown_connections 2024-03-15T12:59:50.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.227+0000 7f3de037a640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3dd819e610 0x7f3dd819eac0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.227+0000 7f3de037a640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3dc40a1cc0 0x7f3dc40a4180 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.227+0000 7f3de037a640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd81056a0 0x7f3dd819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.227+0000 7f3de037a640 1 --2- 172.21.15.2:0/674518802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dd81005d0 0x7f3dd8199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:50.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.227+0000 7f3de037a640 1 -- 172.21.15.2:0/674518802 >> 172.21.15.2:0/674518802 conn(0x7f3dd80774f0 msgr2=0x7f3dd80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:50.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.227+0000 7f3de037a640 1 -- 172.21.15.2:0/674518802 shutdown_connections 2024-03-15T12:59:50.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:50.227+0000 7f3de037a640 1 -- 172.21.15.2:0/674518802 wait complete. 2024-03-15T12:59:51.228 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T12:59:51.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T12:59:51.228 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T12:59:51.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.410+0000 7f2894181640 1 -- 172.21.15.2:0/546198056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288c100f80 msgr2=0x7f288c104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.410+0000 7f2894181640 1 --2- 172.21.15.2:0/546198056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288c100f80 0x7f288c104fc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f287c009930 tx=0x7f287c030040 comp rx=0 tx=0).stop 2024-03-15T12:59:51.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.410+0000 7f2894181640 1 -- 172.21.15.2:0/546198056 shutdown_connections 2024-03-15T12:59:51.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.410+0000 7f2894181640 1 --2- 172.21.15.2:0/546198056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f288c1056a0 0x7f288c10d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.410+0000 7f2894181640 1 --2- 172.21.15.2:0/546198056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288c100f80 0x7f288c104fc0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.410+0000 7f2894181640 1 --2- 172.21.15.2:0/546198056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f288c1005d0 0x7f288c1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.410+0000 7f2894181640 1 -- 172.21.15.2:0/546198056 >> 172.21.15.2:0/546198056 conn(0x7f288c0774f0 msgr2=0x7f288c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:51.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.411+0000 7f2894181640 1 -- 172.21.15.2:0/546198056 shutdown_connections 2024-03-15T12:59:51.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.411+0000 7f2894181640 1 -- 172.21.15.2:0/546198056 wait complete. 2024-03-15T12:59:51.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.411+0000 7f2894181640 1 Processor -- start 2024-03-15T12:59:51.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.411+0000 7f2894181640 1 -- start start 2024-03-15T12:59:51.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.412+0000 7f2894181640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f288c1005d0 0x7f288c19b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.412+0000 7f2894181640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f288c1056a0 0x7f288c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.412+0000 7f2894181640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288c195770 0x7f288c195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.412+0000 7f2891f1d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f288c1005d0 0x7f288c19b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.412+0000 7f2891f1d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f288c1005d0 0x7f288c19b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39566/0 (socket says 172.21.15.2:39566) 2024-03-15T12:59:51.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.412+0000 7f2891f1d640 1 -- 172.21.15.2:0/1234739016 learned_addr learned my addr 172.21.15.2:0/1234739016 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:51.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.412+0000 7f2894181640 1 -- 172.21.15.2:0/1234739016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c196240 con 0x7f288c195770 2024-03-15T12:59:51.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.412+0000 7f2894181640 1 -- 172.21.15.2:0/1234739016 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c1963b0 con 0x7f288c1005d0 2024-03-15T12:59:51.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.412+0000 7f2894181640 1 -- 172.21.15.2:0/1234739016 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c196570 con 0x7f288c1056a0 2024-03-15T12:59:51.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.412+0000 7f289171c640 1 --2- 172.21.15.2:0/1234739016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f288c1056a0 0x7f288c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.412+0000 7f289271e640 1 --2- 172.21.15.2:0/1234739016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288c195770 0x7f288c195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.413+0000 7f289171c640 1 -- 172.21.15.2:0/1234739016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f288c1005d0 msgr2=0x7f288c19b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.413+0000 7f289171c640 1 --2- 172.21.15.2:0/1234739016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f288c1005d0 0x7f288c19b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.413+0000 7f289171c640 1 -- 172.21.15.2:0/1234739016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288c195770 msgr2=0x7f288c195bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.413+0000 7f289171c640 1 --2- 172.21.15.2:0/1234739016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288c195770 0x7f288c195bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.413+0000 7f289171c640 1 -- 172.21.15.2:0/1234739016 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c009560 con 0x7f288c1056a0 2024-03-15T12:59:51.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.413+0000 7f289271e640 1 --2- 172.21.15.2:0/1234739016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288c195770 0x7f288c195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:51.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.413+0000 7f289171c640 1 --2- 172.21.15.2:0/1234739016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f288c1056a0 0x7f288c19bbf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f287c03a040 tx=0x7f287c004960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:51.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.413+0000 7f2882ffd640 1 -- 172.21.15.2:0/1234739016 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c004250 con 0x7f288c1056a0 2024-03-15T12:59:51.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.413+0000 7f2882ffd640 1 -- 172.21.15.2:0/1234739016 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f287c0043b0 con 0x7f288c1056a0 2024-03-15T12:59:51.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.413+0000 7f2894181640 1 -- 172.21.15.2:0/1234739016 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f288c1a24b0 con 0x7f288c1056a0 2024-03-15T12:59:51.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.413+0000 7f2882ffd640 1 -- 172.21.15.2:0/1234739016 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c038e00 con 0x7f288c1056a0 2024-03-15T12:59:51.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.414+0000 7f2894181640 1 -- 172.21.15.2:0/1234739016 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288c1a29e0 con 0x7f288c1056a0 2024-03-15T12:59:51.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.416+0000 7f2882ffd640 1 -- 172.21.15.2:0/1234739016 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7f287c005000 con 0x7f288c1056a0 2024-03-15T12:59:51.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.417+0000 7f2896e3a740 1 Processor -- start 2024-03-15T12:59:51.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.417+0000 7f2896e3a740 1 -- start start 2024-03-15T12:59:51.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.417+0000 7f2882ffd640 1 --2- 172.21.15.2:0/1234739016 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f28780a1d40 0x7f28780a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.417+0000 7f2896e3a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568eb90 0x55a3a568ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.417+0000 7f2896e3a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568f540 0x55a3a5697af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.417+0000 7f2896e3a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 0x55a3a569a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.417+0000 7f2896e3a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a55f4f90 con 0x55a3a56980c0 2024-03-15T12:59:51.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.417+0000 7f2896e3a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a55f24c0 con 0x55a3a568f540 2024-03-15T12:59:51.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.417+0000 7f2896e3a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a55edca0 con 0x55a3a568eb90 2024-03-15T12:59:51.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.418+0000 7f289271e640 1 --2- 172.21.15.2:0/1234739016 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f28780a1d40 0x7f28780a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.418+0000 7f289171c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568f540 0x55a3a5697af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.418+0000 7f2891f1d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568eb90 0x55a3a568ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.418+0000 7f289171c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568f540 0x55a3a5697af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39568/0 (socket says 172.21.15.2:39568) 2024-03-15T12:59:51.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.418+0000 7f2891f1d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568eb90 0x55a3a568ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:60332/0 (socket says 172.21.15.2:60332) 2024-03-15T12:59:51.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.418+0000 7f289171c640 1 -- 172.21.15.2:0/159537429 learned_addr learned my addr 172.21.15.2:0/159537429 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:51.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.418+0000 7f289271e640 1 --2- 172.21.15.2:0/159537429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 0x55a3a569a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.418+0000 7f289171c640 1 -- 172.21.15.2:0/159537429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568eb90 msgr2=0x55a3a568ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.418+0000 7f289171c640 1 --2- 172.21.15.2:0/159537429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568eb90 0x55a3a568ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.418+0000 7f289171c640 1 -- 172.21.15.2:0/159537429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 msgr2=0x55a3a569a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.418+0000 7f289171c640 1 --2- 172.21.15.2:0/159537429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 0x55a3a569a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.418+0000 7f289171c640 1 -- 172.21.15.2:0/159537429 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3a55b00b0 con 0x55a3a568f540 2024-03-15T12:59:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.418+0000 7f289171c640 1 --2- 172.21.15.2:0/159537429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568f540 0x55a3a5697af0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f287c0f1b90 tx=0x7f287c090f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd1a0761fb511e15 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.418+0000 7f2882ffd640 1 -- 172.21.15.2:0/1234739016 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f287c0f46f0 con 0x7f288c1056a0 2024-03-15T12:59:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.419+0000 7f289271e640 1 --2- 172.21.15.2:0/1234739016 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f28780a1d40 0x7f28780a4200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f28840099e0 tx=0x7f2884009500 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.419+0000 7f2877fff640 1 -- 172.21.15.2:0/159537429 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c0c1590 con 0x55a3a568f540 2024-03-15T12:59:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.419+0000 7f2877fff640 1 -- 172.21.15.2:0/159537429 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f287c0c0e90 con 0x55a3a568f540 2024-03-15T12:59:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.419+0000 7f2877fff640 1 -- 172.21.15.2:0/159537429 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c0c66e0 con 0x55a3a568f540 2024-03-15T12:59:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.420+0000 7f2896e3a740 1 -- 172.21.15.2:0/159537429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568f540 msgr2=0x55a3a5697af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.420+0000 7f2896e3a740 1 --2- 172.21.15.2:0/159537429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568f540 0x55a3a5697af0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f287c0f1b90 tx=0x7f287c090f30 comp rx=0 tx=0).stop 2024-03-15T12:59:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.420+0000 7f2896e3a740 1 -- 172.21.15.2:0/159537429 shutdown_connections 2024-03-15T12:59:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.420+0000 7f2896e3a740 1 --2- 172.21.15.2:0/159537429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 0x55a3a569a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.420+0000 7f2896e3a740 1 --2- 172.21.15.2:0/159537429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568f540 0x55a3a5697af0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.420+0000 7f2896e3a740 1 --2- 172.21.15.2:0/159537429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568eb90 0x55a3a568ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.420+0000 7f2896e3a740 1 -- 172.21.15.2:0/159537429 >> 172.21.15.2:0/159537429 conn(0x55a3a5689050 msgr2=0x55a3a5689430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.420+0000 7f2896e3a740 1 -- 172.21.15.2:0/159537429 shutdown_connections 2024-03-15T12:59:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.420+0000 7f2896e3a740 1 -- 172.21.15.2:0/159537429 wait complete. 2024-03-15T12:59:51.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.420+0000 7f2896e3a740 1 -- start start 2024-03-15T12:59:51.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.420+0000 7f2896e3a740 1 Processor -- start 2024-03-15T12:59:51.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.421+0000 7f2896e3a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568eb90 0x55a3a5694fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.421+0000 7f2896e3a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568f540 0x55a3a56954e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.421+0000 7f2896e3a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 0x55a3a5695a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.421+0000 7f2891f1d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568eb90 0x55a3a5694fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.421+0000 7f2891f1d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568eb90 0x55a3a5694fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39582/0 (socket says 172.21.15.2:39582) 2024-03-15T12:59:51.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.421+0000 7f289171c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568f540 0x55a3a56954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.421+0000 7f289171c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568f540 0x55a3a56954e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:60342/0 (socket says 172.21.15.2:60342) 2024-03-15T12:59:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.421+0000 7f289271e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 0x55a3a5695a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.421+0000 7f289271e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 0x55a3a5695a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42390/0 (socket says 172.21.15.2:42390) 2024-03-15T12:59:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.421+0000 7f2891f1d640 1 -- 172.21.15.2:0/3524588329 learned_addr learned my addr 172.21.15.2:0/3524588329 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.421+0000 7f2896e3a740 1 -- 172.21.15.2:0/3524588329 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a5592f40 con 0x55a3a56980c0 2024-03-15T12:59:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.421+0000 7f2896e3a740 1 -- 172.21.15.2:0/3524588329 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a52c14b0 con 0x55a3a568eb90 2024-03-15T12:59:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.421+0000 7f2896e3a740 1 -- 172.21.15.2:0/3524588329 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a5573670 con 0x55a3a568f540 2024-03-15T12:59:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.422+0000 7f2891f1d640 1 -- 172.21.15.2:0/3524588329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568f540 msgr2=0x55a3a56954e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.422+0000 7f2891f1d640 1 --2- 172.21.15.2:0/3524588329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568f540 0x55a3a56954e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.422+0000 7f2891f1d640 1 -- 172.21.15.2:0/3524588329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 msgr2=0x55a3a5695a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.422+0000 7f2891f1d640 1 --2- 172.21.15.2:0/3524588329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 0x55a3a5695a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.422+0000 7f2891f1d640 1 -- 172.21.15.2:0/3524588329 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288c1a29e0 con 0x55a3a568eb90 2024-03-15T12:59:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.422+0000 7f289271e640 1 --2- 172.21.15.2:0/3524588329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 0x55a3a5695a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.422+0000 7f2896e3a740 10 client.0 _ll_register_callbacks cb 0x55a3a55e0f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.422+0000 7f2896e3a740 1 -- 172.21.15.2:0/3524588329 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3a55c7f40 con 0x55a3a568eb90 2024-03-15T12:59:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.422+0000 7f2891f1d640 1 --2- 172.21.15.2:0/3524588329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568eb90 0x55a3a5694fa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2888014760 tx=0x7f2888014c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc72cb77775069a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.422+0000 7f28757fa640 10 client.25967 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T12:59:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.422+0000 7f28757fa640 1 -- 172.21.15.2:0/3524588329 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888010040 con 0x55a3a568eb90 2024-03-15T12:59:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.422+0000 7f28757fa640 1 -- 172.21.15.2:0/3524588329 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f2888014e70 con 0x55a3a568eb90 2024-03-15T12:59:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.422+0000 7f28757fa640 1 -- 172.21.15.2:0/3524588329 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2888009b90 con 0x55a3a568eb90 2024-03-15T12:59:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.422+0000 7f2896e3a740 10 client.25967 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.423+0000 7f2896e3a740 1 -- 172.21.15.2:0/3524588329 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55a3a55ec320 con 0x55a3a568eb90 2024-03-15T12:59:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.423+0000 7f28757fa640 1 -- 172.21.15.2:0/3524588329 <== mon.2 v2:172.21.15.149:3300/0 4 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f288801f020 con 0x55a3a568eb90 2024-03-15T12:59:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.423+0000 7f2896e3a740 10 client.25967 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.423+0000 7f2896e3a740 10 client.25967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.423+0000 7f2896e3a740 20 client.25967 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.423+0000 7f2896e3a740 1 --2- 172.21.15.2:0/3524588329 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a3a56a80b0 0x55a3a56aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.423+0000 7f2896e3a740 4 client.25967 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.423+0000 7f2896e3a740 1 -- 172.21.15.2:0/3524588329 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3a55e4310 con 0x55a3a56a80b0 2024-03-15T12:59:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.423+0000 7f289171c640 1 --2- 172.21.15.2:0/3524588329 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a3a56a80b0 0x55a3a56aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.424+0000 7f28757fa640 10 client.25967 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.424+0000 7f289171c640 1 --2- 172.21.15.2:0/3524588329 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a3a56a80b0 0x55a3a56aa570 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a7ddd7a90ba1eec server_cookie=e9687cf6f02a45f4 in_seq=0 out_seq=0 2024-03-15T12:59:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.425+0000 7f28757fa640 1 -- 172.21.15.2:0/3524588329 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a3a55e4310 con 0x55a3a56a80b0 2024-03-15T12:59:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.425+0000 7f28757fa640 10 client.25967 handle_command_reply: tid=0 2024-03-15T12:59:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.425+0000 7f2896e3a740 1 client.25967 shutdown 2024-03-15T12:59:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.425+0000 7f2896e3a740 1 -- 172.21.15.2:0/3524588329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568eb90 msgr2=0x55a3a5694fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.425+0000 7f2896e3a740 1 --2- 172.21.15.2:0/3524588329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568eb90 0x55a3a5694fa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2888014760 tx=0x7f2888014c20 comp rx=0 tx=0).stop 2024-03-15T12:59:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.425+0000 7f2896e3a740 1 -- 172.21.15.2:0/3524588329 shutdown_connections 2024-03-15T12:59:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.425+0000 7f2896e3a740 1 --2- 172.21.15.2:0/3524588329 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a3a56a80b0 0x55a3a56aa570 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.425+0000 7f2896e3a740 1 --2- 172.21.15.2:0/3524588329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 0x55a3a5695a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.425+0000 7f2896e3a740 1 --2- 172.21.15.2:0/3524588329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568f540 0x55a3a56954e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.425+0000 7f28757fa640 0 client.25967 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:51.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.425+0000 7f2896e3a740 1 --2- 172.21.15.2:0/3524588329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568eb90 0x55a3a5694fa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.425+0000 7f2896e3a740 1 -- 172.21.15.2:0/3524588329 >> 172.21.15.2:0/3524588329 conn(0x55a3a5689050 msgr2=0x55a3a568afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:51.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.425+0000 7f2896e3a740 1 -- 172.21.15.2:0/3524588329 shutdown_connections 2024-03-15T12:59:51.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.425+0000 7f2896e3a740 1 -- 172.21.15.2:0/3524588329 wait complete. 2024-03-15T12:59:51.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.425+0000 7f2896e3a740 20 client.25967 trim_cache size 0 max 16384 2024-03-15T12:59:51.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.458+0000 7f2896e3a740 1 Processor -- start 2024-03-15T12:59:51.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.458+0000 7f2896e3a740 1 -- start start 2024-03-15T12:59:51.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.458+0000 7f2896e3a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568eb90 0x55a3a56976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.458+0000 7f2896e3a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568f540 0x55a3a56a3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.458+0000 7f2896e3a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 0x55a3a56a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.458+0000 7f2896e3a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a55e4330 con 0x55a3a56980c0 2024-03-15T12:59:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.458+0000 7f2896e3a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a56a4610 con 0x55a3a568f540 2024-03-15T12:59:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.458+0000 7f2896e3a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a5690b70 con 0x55a3a568eb90 2024-03-15T12:59:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.458+0000 7f289171c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568f540 0x55a3a56a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.458+0000 7f289171c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568f540 0x55a3a56a3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39592/0 (socket says 172.21.15.2:39592) 2024-03-15T12:59:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.458+0000 7f289171c640 1 -- 172.21.15.2:0/1443286841 learned_addr learned my addr 172.21.15.2:0/1443286841 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.458+0000 7f2891f1d640 1 --2- 172.21.15.2:0/1443286841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568eb90 0x55a3a56976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.458+0000 7f289271e640 1 --2- 172.21.15.2:0/1443286841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 0x55a3a56a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.459+0000 7f289171c640 1 -- 172.21.15.2:0/1443286841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568eb90 msgr2=0x55a3a56976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.459+0000 7f289171c640 1 --2- 172.21.15.2:0/1443286841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568eb90 0x55a3a56976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.459+0000 7f289171c640 1 -- 172.21.15.2:0/1443286841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 msgr2=0x55a3a56a37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.459+0000 7f289171c640 1 --2- 172.21.15.2:0/1443286841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 0x55a3a56a37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.459+0000 7f289171c640 1 -- 172.21.15.2:0/1443286841 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3a5588c00 con 0x55a3a568f540 2024-03-15T12:59:51.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.459+0000 7f289271e640 1 --2- 172.21.15.2:0/1443286841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 0x55a3a56a37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:51.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.459+0000 7f289171c640 1 --2- 172.21.15.2:0/1443286841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568f540 0x55a3a56a3290 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f287c0bd860 tx=0x7f287c0bd260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbba24d484aa30ad server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:51.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.459+0000 7f28777fe640 1 -- 172.21.15.2:0/1443286841 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c0c0e90 con 0x55a3a568f540 2024-03-15T12:59:51.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.460+0000 7f28777fe640 1 -- 172.21.15.2:0/1443286841 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f287c0c1590 con 0x55a3a568f540 2024-03-15T12:59:51.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.460+0000 7f28777fe640 1 -- 172.21.15.2:0/1443286841 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c038700 con 0x55a3a568f540 2024-03-15T12:59:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.460+0000 7f2896e3a740 1 -- 172.21.15.2:0/1443286841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568f540 msgr2=0x55a3a56a3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.460+0000 7f2896e3a740 1 --2- 172.21.15.2:0/1443286841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568f540 0x55a3a56a3290 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f287c0bd860 tx=0x7f287c0bd260 comp rx=0 tx=0).stop 2024-03-15T12:59:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.461+0000 7f2896e3a740 1 -- 172.21.15.2:0/1443286841 shutdown_connections 2024-03-15T12:59:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.461+0000 7f2896e3a740 1 --2- 172.21.15.2:0/1443286841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a56980c0 0x55a3a56a37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.461+0000 7f2896e3a740 1 --2- 172.21.15.2:0/1443286841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568f540 0x55a3a56a3290 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.461+0000 7f2896e3a740 1 --2- 172.21.15.2:0/1443286841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a568eb90 0x55a3a56976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.461+0000 7f2896e3a740 1 -- 172.21.15.2:0/1443286841 >> 172.21.15.2:0/1443286841 conn(0x55a3a5689050 msgr2=0x55a3a5693870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.461+0000 7f2896e3a740 1 -- 172.21.15.2:0/1443286841 shutdown_connections 2024-03-15T12:59:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.461+0000 7f2896e3a740 1 -- 172.21.15.2:0/1443286841 wait complete. 2024-03-15T12:59:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.461+0000 7f2896e3a740 1 -- start start 2024-03-15T12:59:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.461+0000 7f2896e3a740 1 Processor -- start 2024-03-15T12:59:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.462+0000 7f2896e3a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568eb90 0x55a3a569b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.462+0000 7f2896e3a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a568f540 0x55a3a569be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.462+0000 7f2891f1d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568eb90 0x55a3a569b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.462+0000 7f289171c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a568f540 0x55a3a569be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.462+0000 7f2891f1d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568eb90 0x55a3a569b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39608/0 (socket says 172.21.15.2:39608) 2024-03-15T12:59:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.462+0000 7f2891f1d640 1 -- 172.21.15.2:0/5252858 learned_addr learned my addr 172.21.15.2:0/5252858 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.462+0000 7f2896e3a740 1 --2- 172.21.15.2:0/5252858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a56980c0 0x55a3a5702a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.462+0000 7f2896e3a740 1 -- 172.21.15.2:0/5252858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a5702f80 con 0x55a3a568f540 2024-03-15T12:59:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.462+0000 7f2896e3a740 1 -- 172.21.15.2:0/5252858 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a57030c0 con 0x55a3a568eb90 2024-03-15T12:59:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.462+0000 7f2896e3a740 1 -- 172.21.15.2:0/5252858 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a5703200 con 0x55a3a56980c0 2024-03-15T12:59:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.462+0000 7f289271e640 1 --2- 172.21.15.2:0/5252858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a56980c0 0x55a3a5702a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.463+0000 7f2891f1d640 1 -- 172.21.15.2:0/5252858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a56980c0 msgr2=0x55a3a5702a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.463+0000 7f2891f1d640 1 --2- 172.21.15.2:0/5252858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a56980c0 0x55a3a5702a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.463+0000 7f2891f1d640 1 -- 172.21.15.2:0/5252858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a568f540 msgr2=0x55a3a569be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.463+0000 7f2891f1d640 1 --2- 172.21.15.2:0/5252858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a568f540 0x55a3a569be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.463+0000 7f2891f1d640 1 -- 172.21.15.2:0/5252858 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2884013e20 con 0x55a3a568eb90 2024-03-15T12:59:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.463+0000 7f289171c640 1 --2- 172.21.15.2:0/5252858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a568f540 0x55a3a569be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.463+0000 7f2891f1d640 1 --2- 172.21.15.2:0/5252858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568eb90 0x55a3a569b920 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f288800d010 tx=0x7f28880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2b89a562059635f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.463+0000 7f28757fa640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T12:59:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.463+0000 7f2896e3a740 10 client.0 _ll_register_callbacks cb 0x7f288800ba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.463+0000 7f2896e3a740 1 -- 172.21.15.2:0/5252858 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3a5613900 con 0x55a3a568eb90 2024-03-15T12:59:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.463+0000 7f28757fa640 1 -- 172.21.15.2:0/5252858 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888010070 con 0x55a3a568eb90 2024-03-15T12:59:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.463+0000 7f28757fa640 1 -- 172.21.15.2:0/5252858 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f2888000e40 con 0x55a3a568eb90 2024-03-15T12:59:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.463+0000 7f28757fa640 1 -- 172.21.15.2:0/5252858 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2888005c50 con 0x55a3a568eb90 2024-03-15T12:59:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.464+0000 7f2896e3a740 10 client.25973 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.464+0000 7f2896e3a740 1 -- 172.21.15.2:0/5252858 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55a3a55b00b0 con 0x55a3a568eb90 2024-03-15T12:59:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.464+0000 7f28757fa640 1 -- 172.21.15.2:0/5252858 <== mon.2 v2:172.21.15.149:3300/0 4 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f288801f040 con 0x55a3a568eb90 2024-03-15T12:59:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.464+0000 7f2896e3a740 10 client.25973 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.464+0000 7f2896e3a740 10 client.25973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.464+0000 7f2896e3a740 20 client.25973 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.464+0000 7f2896e3a740 1 --2- 172.21.15.2:0/5252858 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a3a56a80b0 0x55a3a57065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.464+0000 7f2896e3a740 4 client.25973 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T12:59:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.464+0000 7f2896e3a740 1 -- 172.21.15.2:0/5252858 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a3a52ed860 con 0x55a3a56a80b0 2024-03-15T12:59:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.464+0000 7f289171c640 1 --2- 172.21.15.2:0/5252858 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a3a56a80b0 0x55a3a57065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.465+0000 7f289171c640 1 --2- 172.21.15.2:0/5252858 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a3a56a80b0 0x55a3a57065d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60faebc8ef316d2d server_cookie=6686459c5871698 in_seq=0 out_seq=0 2024-03-15T12:59:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.465+0000 7f28757fa640 10 client.25973 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.466+0000 7f28757fa640 1 -- 172.21.15.2:0/5252858 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a3a52ed860 con 0x55a3a56a80b0 2024-03-15T12:59:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.466+0000 7f28757fa640 10 client.25973 handle_command_reply: tid=0 2024-03-15T12:59:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.466+0000 7f2896e3a740 1 client.25973 shutdown 2024-03-15T12:59:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.466+0000 7f2896e3a740 1 -- 172.21.15.2:0/5252858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568eb90 msgr2=0x55a3a569b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.466+0000 7f2896e3a740 1 --2- 172.21.15.2:0/5252858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568eb90 0x55a3a569b920 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f288800d010 tx=0x7f28880046c0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.467+0000 7f2896e3a740 1 -- 172.21.15.2:0/5252858 shutdown_connections 2024-03-15T12:59:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.467+0000 7f2896e3a740 1 --2- 172.21.15.2:0/5252858 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a3a56a80b0 0x55a3a57065d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.467+0000 7f2896e3a740 1 --2- 172.21.15.2:0/5252858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3a56980c0 0x55a3a5702a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.467+0000 7f28757fa640 0 client.25973 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.467+0000 7f2896e3a740 1 --2- 172.21.15.2:0/5252858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a3a568f540 0x55a3a569be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.467+0000 7f2896e3a740 1 --2- 172.21.15.2:0/5252858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3a568eb90 0x55a3a569b920 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.467+0000 7f2896e3a740 1 -- 172.21.15.2:0/5252858 >> 172.21.15.2:0/5252858 conn(0x55a3a5689050 msgr2=0x55a3a56aee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.467+0000 7f2896e3a740 1 -- 172.21.15.2:0/5252858 shutdown_connections 2024-03-15T12:59:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.467+0000 7f2896e3a740 1 -- 172.21.15.2:0/5252858 wait complete. 2024-03-15T12:59:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.467+0000 7f2896e3a740 20 client.25973 trim_cache size 0 max 16384 2024-03-15T12:59:51.458 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:59:51.458 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T12:59:51.458 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "387fc714-99e7-4bdf-a0c8-9379b44dafef", 2024-03-15T12:59:51.458 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T12:59:51.458 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:59:51.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.468+0000 7f2894181640 1 -- 172.21.15.2:0/1234739016 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f28780a1d40 msgr2=0x7f28780a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.468+0000 7f2894181640 1 --2- 172.21.15.2:0/1234739016 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f28780a1d40 0x7f28780a4200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f28840099e0 tx=0x7f2884009500 comp rx=0 tx=0).stop 2024-03-15T12:59:51.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.468+0000 7f2894181640 1 -- 172.21.15.2:0/1234739016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f288c1056a0 msgr2=0x7f288c19bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.468+0000 7f2894181640 1 --2- 172.21.15.2:0/1234739016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f288c1056a0 0x7f288c19bbf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f287c03a040 tx=0x7f287c004960 comp rx=0 tx=0).stop 2024-03-15T12:59:51.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.468+0000 7f2894181640 1 -- 172.21.15.2:0/1234739016 shutdown_connections 2024-03-15T12:59:51.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.468+0000 7f2894181640 1 --2- 172.21.15.2:0/1234739016 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f28780a1d40 0x7f28780a4200 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.468+0000 7f2894181640 1 --2- 172.21.15.2:0/1234739016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288c195770 0x7f288c195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.468+0000 7f2894181640 1 --2- 172.21.15.2:0/1234739016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f288c1056a0 0x7f288c19bbf0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.468+0000 7f2894181640 1 --2- 172.21.15.2:0/1234739016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f288c1005d0 0x7f288c19b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.468+0000 7f2894181640 1 -- 172.21.15.2:0/1234739016 >> 172.21.15.2:0/1234739016 conn(0x7f288c0774f0 msgr2=0x7f288c0fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:51.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.468+0000 7f2894181640 1 -- 172.21.15.2:0/1234739016 shutdown_connections 2024-03-15T12:59:51.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.468+0000 7f2894181640 1 -- 172.21.15.2:0/1234739016 wait complete. 2024-03-15T12:59:51.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T12:59:51.469 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T12:59:51.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.639+0000 7fea6b181640 1 -- 172.21.15.2:0/178029025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea641056b0 msgr2=0x7fea6410d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.639+0000 7fea6b181640 1 --2- 172.21.15.2:0/178029025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea641056b0 0x7fea6410d280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009cd0 tx=0x7fea5c02c0f0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.639+0000 7fea6b181640 1 -- 172.21.15.2:0/178029025 shutdown_connections 2024-03-15T12:59:51.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.639+0000 7fea6b181640 1 --2- 172.21.15.2:0/178029025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea641056b0 0x7fea6410d280 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.639+0000 7fea6b181640 1 --2- 172.21.15.2:0/178029025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea64100f90 0x7fea64104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.639+0000 7fea6b181640 1 --2- 172.21.15.2:0/178029025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea641005e0 0x7fea641009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.639+0000 7fea6b181640 1 -- 172.21.15.2:0/178029025 >> 172.21.15.2:0/178029025 conn(0x7fea640775a0 msgr2=0x7fea640779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:51.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.639+0000 7fea6b181640 1 -- 172.21.15.2:0/178029025 shutdown_connections 2024-03-15T12:59:51.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.639+0000 7fea6b181640 1 -- 172.21.15.2:0/178029025 wait complete. 2024-03-15T12:59:51.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea6b181640 1 Processor -- start 2024-03-15T12:59:51.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea6b181640 1 -- start start 2024-03-15T12:59:51.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea6b181640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea641005e0 0x7fea6419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea6b181640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea641056b0 0x7fea6419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea6b181640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea64196800 0x7fea64196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea6b181640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea641972b0 con 0x7fea641005e0 2024-03-15T12:59:51.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea6b181640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea64197420 con 0x7fea64196800 2024-03-15T12:59:51.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea68f1d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea641005e0 0x7fea6419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea6b181640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea64197590 con 0x7fea641056b0 2024-03-15T12:59:51.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea68f1d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea641005e0 0x7fea6419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42436/0 (socket says 172.21.15.2:42436) 2024-03-15T12:59:51.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea68f1d640 1 -- 172.21.15.2:0/85013092 learned_addr learned my addr 172.21.15.2:0/85013092 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:51.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea6971e640 1 --2- 172.21.15.2:0/85013092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea64196800 0x7fea64196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea63fff640 1 --2- 172.21.15.2:0/85013092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea641056b0 0x7fea6419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea68f1d640 1 -- 172.21.15.2:0/85013092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea641056b0 msgr2=0x7fea6419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea68f1d640 1 --2- 172.21.15.2:0/85013092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea641056b0 0x7fea6419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea68f1d640 1 -- 172.21.15.2:0/85013092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea64196800 msgr2=0x7fea64196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea68f1d640 1 --2- 172.21.15.2:0/85013092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea64196800 0x7fea64196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea68f1d640 1 -- 172.21.15.2:0/85013092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea50009490 con 0x7fea641005e0 2024-03-15T12:59:51.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.640+0000 7fea63fff640 1 --2- 172.21.15.2:0/85013092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea641056b0 0x7fea6419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:59:51.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.641+0000 7fea68f1d640 1 --2- 172.21.15.2:0/85013092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea641005e0 0x7fea6419c640 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fea5800e9f0 tx=0x7fea5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:51.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.641+0000 7fea61ffb640 1 -- 172.21.15.2:0/85013092 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58009700 con 0x7fea641005e0 2024-03-15T12:59:51.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.641+0000 7fea61ffb640 1 -- 172.21.15.2:0/85013092 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fea5800cd80 con 0x7fea641005e0 2024-03-15T12:59:51.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.641+0000 7fea6b181640 1 -- 172.21.15.2:0/85013092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5c009900 con 0x7fea641005e0 2024-03-15T12:59:51.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.641+0000 7fea61ffb640 1 -- 172.21.15.2:0/85013092 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58018920 con 0x7fea641005e0 2024-03-15T12:59:51.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.641+0000 7fea6b181640 1 -- 172.21.15.2:0/85013092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea640688f0 con 0x7fea641005e0 2024-03-15T12:59:51.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.643+0000 7fea61ffb640 1 -- 172.21.15.2:0/85013092 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7fea58018a80 con 0x7fea641005e0 2024-03-15T12:59:51.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.644+0000 7fea61ffb640 1 --2- 172.21.15.2:0/85013092 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fea540a1cc0 0x7fea540a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.644+0000 7fea61ffb640 1 -- 172.21.15.2:0/85013092 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fea58014070 con 0x7fea641005e0 2024-03-15T12:59:51.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.644+0000 7fea63fff640 1 --2- 172.21.15.2:0/85013092 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fea540a1cc0 0x7fea540a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.645+0000 7fea63fff640 1 --2- 172.21.15.2:0/85013092 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fea540a1cc0 0x7fea540a4180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fea640ffe20 tx=0x7fea50005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:51.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.645+0000 7fea6de3a740 1 Processor -- start 2024-03-15T12:59:51.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.645+0000 7fea6de3a740 1 -- start start 2024-03-15T12:59:51.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.645+0000 7fea6de3a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec4aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.645+0000 7fea6de3a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49b440 0x55cdec49b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.645+0000 7fea6de3a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec49be00 0x55cdec4b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.645+0000 7fea6de3a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec410280 con 0x55cdec49b440 2024-03-15T12:59:51.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.645+0000 7fea6de3a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec40d7b0 con 0x55cdec4aea20 2024-03-15T12:59:51.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.645+0000 7fea6de3a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec408f90 con 0x55cdec49be00 2024-03-15T12:59:51.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.645+0000 7fea63fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49b440 0x55cdec49b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.645+0000 7fea63fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49b440 0x55cdec49b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42452/0 (socket says 172.21.15.2:42452) 2024-03-15T12:59:51.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.645+0000 7fea63fff640 1 -- 172.21.15.2:0/1705648798 learned_addr learned my addr 172.21.15.2:0/1705648798 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:51.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.645+0000 7fea68f1d640 1 --2- 172.21.15.2:0/1705648798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec4aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.645+0000 7fea6971e640 1 --2- 172.21.15.2:0/1705648798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec49be00 0x55cdec4b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.646+0000 7fea63fff640 1 -- 172.21.15.2:0/1705648798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec49be00 msgr2=0x55cdec4b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.646+0000 7fea63fff640 1 --2- 172.21.15.2:0/1705648798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec49be00 0x55cdec4b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.646+0000 7fea63fff640 1 -- 172.21.15.2:0/1705648798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 msgr2=0x55cdec4aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.646+0000 7fea63fff640 1 --2- 172.21.15.2:0/1705648798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec4aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.646+0000 7fea63fff640 1 -- 172.21.15.2:0/1705648798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdec3c7570 con 0x55cdec49b440 2024-03-15T12:59:51.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.646+0000 7fea6971e640 1 --2- 172.21.15.2:0/1705648798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec49be00 0x55cdec4b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:51.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.646+0000 7fea68f1d640 1 --2- 172.21.15.2:0/1705648798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec4aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:51.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.646+0000 7fea63fff640 1 --2- 172.21.15.2:0/1705648798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49b440 0x55cdec49b8c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fea50012580 tx=0x7fea50014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccffb80a1d1c370 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:51.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.646+0000 7fea4effd640 1 -- 172.21.15.2:0/1705648798 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5001f6e0 con 0x55cdec49b440 2024-03-15T12:59:51.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.646+0000 7fea4effd640 1 -- 172.21.15.2:0/1705648798 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fea50016070 con 0x55cdec49b440 2024-03-15T12:59:51.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.646+0000 7fea4effd640 1 -- 172.21.15.2:0/1705648798 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea50027ba0 con 0x55cdec49b440 2024-03-15T12:59:51.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.647+0000 7fea6de3a740 1 -- 172.21.15.2:0/1705648798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49b440 msgr2=0x55cdec49b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.647+0000 7fea6de3a740 1 --2- 172.21.15.2:0/1705648798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49b440 0x55cdec49b8c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fea50012580 tx=0x7fea50014000 comp rx=0 tx=0).stop 2024-03-15T12:59:51.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.647+0000 7fea6de3a740 1 -- 172.21.15.2:0/1705648798 shutdown_connections 2024-03-15T12:59:51.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.647+0000 7fea6de3a740 1 --2- 172.21.15.2:0/1705648798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec49be00 0x55cdec4b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.647+0000 7fea6de3a740 1 --2- 172.21.15.2:0/1705648798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49b440 0x55cdec49b8c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.647+0000 7fea6de3a740 1 --2- 172.21.15.2:0/1705648798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec4aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.647+0000 7fea6de3a740 1 -- 172.21.15.2:0/1705648798 >> 172.21.15.2:0/1705648798 conn(0x55cdec49dfb0 msgr2=0x55cdec49e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:51.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.647+0000 7fea6de3a740 1 -- 172.21.15.2:0/1705648798 shutdown_connections 2024-03-15T12:59:51.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.647+0000 7fea6de3a740 1 -- 172.21.15.2:0/1705648798 wait complete. 2024-03-15T12:59:51.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.648+0000 7fea6de3a740 1 -- start start 2024-03-15T12:59:51.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.648+0000 7fea6de3a740 1 Processor -- start 2024-03-15T12:59:51.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.648+0000 7fea6de3a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49be00 0x55cdec4b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.648+0000 7fea6de3a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec4b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.648+0000 7fea6de3a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4b3640 0x55cdec4befd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.648+0000 7fea68f1d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49be00 0x55cdec4b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.648+0000 7fea6de3a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec0dc430 con 0x55cdec49be00 2024-03-15T12:59:51.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.648+0000 7fea6de3a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec098e00 con 0x55cdec4aea20 2024-03-15T12:59:51.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.648+0000 7fea6de3a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec38dfd0 con 0x55cdec4b3640 2024-03-15T12:59:51.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea68f1d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49be00 0x55cdec4b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42468/0 (socket says 172.21.15.2:42468) 2024-03-15T12:59:51.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea68f1d640 1 -- 172.21.15.2:0/2642009003 learned_addr learned my addr 172.21.15.2:0/2642009003 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:51.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea63fff640 1 --2- 172.21.15.2:0/2642009003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec4b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea6971e640 1 --2- 172.21.15.2:0/2642009003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4b3640 0x55cdec4befd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea6de3a740 10 client.0 _ll_register_callbacks cb 0x55cdec3e3fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:51.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea68f1d640 1 -- 172.21.15.2:0/2642009003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4b3640 msgr2=0x55cdec4befd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea68f1d640 1 --2- 172.21.15.2:0/2642009003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4b3640 0x55cdec4befd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea68f1d640 1 -- 172.21.15.2:0/2642009003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 msgr2=0x55cdec4b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea68f1d640 1 --2- 172.21.15.2:0/2642009003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec4b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea68f1d640 1 -- 172.21.15.2:0/2642009003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea50009d70 con 0x55cdec49be00 2024-03-15T12:59:51.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea68f1d640 1 --2- 172.21.15.2:0/2642009003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49be00 0x55cdec4b2bc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fea5800e9c0 tx=0x7fea5809b0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1897dff2d5e513 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:51.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea43fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:51.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea43fff640 1 -- 172.21.15.2:0/2642009003 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58010ca0 con 0x55cdec49be00 2024-03-15T12:59:51.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea43fff640 1 -- 172.21.15.2:0/2642009003 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fea580cb760 con 0x55cdec49be00 2024-03-15T12:59:51.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea6de3a740 1 -- 172.21.15.2:0/2642009003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdec1087e0 con 0x55cdec49be00 2024-03-15T12:59:51.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea43fff640 1 -- 172.21.15.2:0/2642009003 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea580cb920 con 0x55cdec49be00 2024-03-15T12:59:51.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea6de3a740 10 client.16119 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:51.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.649+0000 7fea6de3a740 1 -- 172.21.15.2:0/2642009003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cdec407610 con 0x55cdec49be00 2024-03-15T12:59:51.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.650+0000 7fea43fff640 1 -- 172.21.15.2:0/2642009003 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55cdec1087e0 con 0x55cdec49be00 2024-03-15T12:59:51.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.650+0000 7fea43fff640 1 -- 172.21.15.2:0/2642009003 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fea58099750 con 0x55cdec49be00 2024-03-15T12:59:51.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.650+0000 7fea6de3a740 10 client.16119 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:51.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.650+0000 7fea6de3a740 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:51.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.650+0000 7fea6de3a740 20 client.16119 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:51.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.650+0000 7fea6de3a740 1 --2- 172.21.15.2:0/2642009003 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cdec4c36a0 0x55cdec4c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.650+0000 7fea6de3a740 4 client.16119 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T12:59:51.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.650+0000 7fea6de3a740 1 -- 172.21.15.2:0/2642009003 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdec145720 con 0x55cdec4c36a0 2024-03-15T12:59:51.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.650+0000 7fea63fff640 1 --2- 172.21.15.2:0/2642009003 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cdec4c36a0 0x55cdec4c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.651+0000 7fea63fff640 1 --2- 172.21.15.2:0/2642009003 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cdec4c36a0 0x55cdec4c5a90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4abd0bca5a3868 server_cookie=d317056d4c72032b in_seq=0 out_seq=0 2024-03-15T12:59:51.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.651+0000 7fea43fff640 10 client.16119 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:51.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.651+0000 7fea43fff640 1 -- 172.21.15.2:0/2642009003 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cdec145720 con 0x55cdec4c36a0 2024-03-15T12:59:51.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.651+0000 7fea43fff640 10 client.16119 handle_command_reply: tid=0 2024-03-15T12:59:51.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.652+0000 7fea6de3a740 1 client.16119 shutdown 2024-03-15T12:59:51.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.652+0000 7fea6de3a740 1 -- 172.21.15.2:0/2642009003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49be00 msgr2=0x55cdec4b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.652+0000 7fea6de3a740 1 --2- 172.21.15.2:0/2642009003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49be00 0x55cdec4b2bc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fea5800e9c0 tx=0x7fea5809b0f0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.652+0000 7fea6de3a740 1 -- 172.21.15.2:0/2642009003 shutdown_connections 2024-03-15T12:59:51.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.652+0000 7fea6de3a740 1 --2- 172.21.15.2:0/2642009003 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cdec4c36a0 0x55cdec4c5a90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.652+0000 7fea43fff640 0 client.16119 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:51.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.652+0000 7fea6de3a740 1 --2- 172.21.15.2:0/2642009003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4b3640 0x55cdec4befd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.652+0000 7fea6de3a740 1 --2- 172.21.15.2:0/2642009003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec4b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.652+0000 7fea6de3a740 1 --2- 172.21.15.2:0/2642009003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49be00 0x55cdec4b2bc0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.652+0000 7fea6de3a740 1 -- 172.21.15.2:0/2642009003 >> 172.21.15.2:0/2642009003 conn(0x55cdec49dfb0 msgr2=0x55cdec49d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:51.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.653+0000 7fea6de3a740 1 -- 172.21.15.2:0/2642009003 shutdown_connections 2024-03-15T12:59:51.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.653+0000 7fea6de3a740 1 -- 172.21.15.2:0/2642009003 wait complete. 2024-03-15T12:59:51.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.653+0000 7fea6de3a740 20 client.16119 trim_cache size 0 max 16384 2024-03-15T12:59:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea6de3a740 1 Processor -- start 2024-03-15T12:59:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea6de3a740 1 -- start start 2024-03-15T12:59:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea6de3a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49be00 0x55cdec4b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea6de3a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec49ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea6de3a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4b3640 0x55cdec49d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea6de3a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec4b3b60 con 0x55cdec49be00 2024-03-15T12:59:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea6de3a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec49c960 con 0x55cdec4aea20 2024-03-15T12:59:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea6de3a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec4c3450 con 0x55cdec4b3640 2024-03-15T12:59:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea68f1d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49be00 0x55cdec4b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea68f1d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49be00 0x55cdec4b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42470/0 (socket says 172.21.15.2:42470) 2024-03-15T12:59:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea68f1d640 1 -- 172.21.15.2:0/2465499770 learned_addr learned my addr 172.21.15.2:0/2465499770 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea63fff640 1 --2- 172.21.15.2:0/2465499770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec49ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea6971e640 1 --2- 172.21.15.2:0/2465499770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4b3640 0x55cdec49d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea68f1d640 1 -- 172.21.15.2:0/2465499770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4b3640 msgr2=0x55cdec49d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea68f1d640 1 --2- 172.21.15.2:0/2465499770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4b3640 0x55cdec49d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.685+0000 7fea68f1d640 1 -- 172.21.15.2:0/2465499770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 msgr2=0x55cdec49ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.686+0000 7fea68f1d640 1 --2- 172.21.15.2:0/2465499770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec49ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.686+0000 7fea68f1d640 1 -- 172.21.15.2:0/2465499770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdec3a3ac0 con 0x55cdec49be00 2024-03-15T12:59:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.686+0000 7fea63fff640 1 --2- 172.21.15.2:0/2465499770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec49ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.686+0000 7fea68f1d640 1 --2- 172.21.15.2:0/2465499770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49be00 0x55cdec4b07c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fea580cbc00 tx=0x7fea5809eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d386fb23378aeaa server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.686+0000 7fea4e7fc640 1 -- 172.21.15.2:0/2465499770 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58010ca0 con 0x55cdec49be00 2024-03-15T12:59:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.686+0000 7fea4e7fc640 1 -- 172.21.15.2:0/2465499770 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fea580b3200 con 0x55cdec49be00 2024-03-15T12:59:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.686+0000 7fea4e7fc640 1 -- 172.21.15.2:0/2465499770 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea580960c0 con 0x55cdec49be00 2024-03-15T12:59:51.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.687+0000 7fea6de3a740 1 -- 172.21.15.2:0/2465499770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49be00 msgr2=0x55cdec4b07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.687+0000 7fea6de3a740 1 --2- 172.21.15.2:0/2465499770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49be00 0x55cdec4b07c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fea580cbc00 tx=0x7fea5809eb60 comp rx=0 tx=0).stop 2024-03-15T12:59:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.687+0000 7fea6de3a740 1 -- 172.21.15.2:0/2465499770 shutdown_connections 2024-03-15T12:59:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.687+0000 7fea6de3a740 1 --2- 172.21.15.2:0/2465499770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4b3640 0x55cdec49d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.687+0000 7fea6de3a740 1 --2- 172.21.15.2:0/2465499770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec49ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.687+0000 7fea6de3a740 1 --2- 172.21.15.2:0/2465499770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec49be00 0x55cdec4b07c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.687+0000 7fea6de3a740 1 -- 172.21.15.2:0/2465499770 >> 172.21.15.2:0/2465499770 conn(0x55cdec49dfb0 msgr2=0x55cdec4b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.687+0000 7fea6de3a740 1 -- 172.21.15.2:0/2465499770 shutdown_connections 2024-03-15T12:59:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.687+0000 7fea6de3a740 1 -- 172.21.15.2:0/2465499770 wait complete. 2024-03-15T12:59:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.687+0000 7fea6de3a740 1 -- start start 2024-03-15T12:59:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.688+0000 7fea6de3a740 1 Processor -- start 2024-03-15T12:59:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.688+0000 7fea6de3a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec4baab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.688+0000 7fea6de3a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec4b3640 0x55cdec4b8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.688+0000 7fea6de3a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4c36a0 0x55cdec4b9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.688+0000 7fea63fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec4b3640 0x55cdec4b8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.688+0000 7fea6de3a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec4b9880 con 0x55cdec4b3640 2024-03-15T12:59:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.688+0000 7fea63fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec4b3640 0x55cdec4b8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42478/0 (socket says 172.21.15.2:42478) 2024-03-15T12:59:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.688+0000 7fea68f1d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec4baab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.688+0000 7fea63fff640 1 -- 172.21.15.2:0/3754983586 learned_addr learned my addr 172.21.15.2:0/3754983586 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T12:59:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.688+0000 7fea6de3a740 1 -- 172.21.15.2:0/3754983586 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec4b99f0 con 0x55cdec4aea20 2024-03-15T12:59:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.688+0000 7fea6de3a740 1 -- 172.21.15.2:0/3754983586 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec4b9b60 con 0x55cdec4c36a0 2024-03-15T12:59:51.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.688+0000 7fea6971e640 1 --2- 172.21.15.2:0/3754983586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4c36a0 0x55cdec4b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.689+0000 7fea6de3a740 10 client.0 _ll_register_callbacks cb 0x7fea58010eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T12:59:51.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.689+0000 7fea63fff640 1 -- 172.21.15.2:0/3754983586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4c36a0 msgr2=0x55cdec4b9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.689+0000 7fea63fff640 1 --2- 172.21.15.2:0/3754983586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4c36a0 0x55cdec4b9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.689+0000 7fea63fff640 1 -- 172.21.15.2:0/3754983586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 msgr2=0x55cdec4baab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T12:59:51.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.689+0000 7fea63fff640 1 --2- 172.21.15.2:0/3754983586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec4baab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.689+0000 7fea63fff640 1 -- 172.21.15.2:0/3754983586 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cdec3c7570 con 0x55cdec4b3640 2024-03-15T12:59:51.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.689+0000 7fea68f1d640 1 --2- 172.21.15.2:0/3754983586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec4baab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T12:59:51.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.689+0000 7fea6971e640 1 --2- 172.21.15.2:0/3754983586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4c36a0 0x55cdec4b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T12:59:51.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.689+0000 7fea63fff640 1 --2- 172.21.15.2:0/3754983586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec4b3640 0x55cdec4b8e00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fea50012580 tx=0x7fea50009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c29b36a3adf23e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T12:59:51.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.689+0000 7fea43fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T12:59:51.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.689+0000 7fea43fff640 1 -- 172.21.15.2:0/3754983586 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5002b040 con 0x55cdec4b3640 2024-03-15T12:59:51.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.689+0000 7fea43fff640 1 -- 172.21.15.2:0/3754983586 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fea50018070 con 0x55cdec4b3640 2024-03-15T12:59:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.689+0000 7fea6de3a740 1 -- 172.21.15.2:0/3754983586 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea50016070 con 0x55cdec4b3640 2024-03-15T12:59:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.689+0000 7fea43fff640 1 -- 172.21.15.2:0/3754983586 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea500362b0 con 0x55cdec4b3640 2024-03-15T12:59:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.690+0000 7fea6de3a740 10 client.16125 fetch_fsmap learned FSMap version 29 2024-03-15T12:59:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.690+0000 7fea6de3a740 1 -- 172.21.15.2:0/3754983586 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cdec407610 con 0x55cdec4b3640 2024-03-15T12:59:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.690+0000 7fea43fff640 1 -- 172.21.15.2:0/3754983586 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fea5002c040 con 0x55cdec4b3640 2024-03-15T12:59:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.690+0000 7fea43fff640 1 -- 172.21.15.2:0/3754983586 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fea50025020 con 0x55cdec4b3640 2024-03-15T12:59:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.690+0000 7fea6de3a740 10 client.16125 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T12:59:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.690+0000 7fea6de3a740 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T12:59:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.690+0000 7fea6de3a740 20 client.16125 populate_metadata read hostname 'smithi002' 2024-03-15T12:59:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.690+0000 7fea6de3a740 1 --2- 172.21.15.2:0/3754983586 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cdec520390 0x55cdec5227b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T12:59:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.690+0000 7fea6de3a740 4 client.16125 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T12:59:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.690+0000 7fea6de3a740 1 -- 172.21.15.2:0/3754983586 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fea50016070 con 0x55cdec520390 2024-03-15T12:59:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.690+0000 7fea68f1d640 1 --2- 172.21.15.2:0/3754983586 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cdec520390 0x55cdec5227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T12:59:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea68f1d640 1 --2- 172.21.15.2:0/3754983586 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cdec520390 0x55cdec5227b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3279689cb7a14b75 server_cookie=1a86381b5982a74f in_seq=0 out_seq=0 2024-03-15T12:59:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea43fff640 10 client.16125 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea43fff640 1 -- 172.21.15.2:0/3754983586 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7fea50016070 con 0x55cdec520390 2024-03-15T12:59:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea43fff640 10 client.16125 handle_command_reply: tid=0 2024-03-15T12:59:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea6de3a740 1 client.16125 shutdown 2024-03-15T12:59:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea6de3a740 1 -- 172.21.15.2:0/3754983586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec4b3640 msgr2=0x55cdec4b8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea6de3a740 1 --2- 172.21.15.2:0/3754983586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec4b3640 0x55cdec4b8e00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fea50012580 tx=0x7fea50009980 comp rx=0 tx=0).stop 2024-03-15T12:59:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea6de3a740 1 -- 172.21.15.2:0/3754983586 shutdown_connections 2024-03-15T12:59:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea6de3a740 1 --2- 172.21.15.2:0/3754983586 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cdec520390 0x55cdec5227b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea43fff640 0 client.16125 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T12:59:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea6de3a740 1 --2- 172.21.15.2:0/3754983586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cdec4c36a0 0x55cdec4b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea6de3a740 1 --2- 172.21.15.2:0/3754983586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cdec4b3640 0x55cdec4b8e00 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea6de3a740 1 --2- 172.21.15.2:0/3754983586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdec4aea20 0x55cdec4baab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea6de3a740 1 -- 172.21.15.2:0/3754983586 >> 172.21.15.2:0/3754983586 conn(0x55cdec49dfb0 msgr2=0x55cdec49ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:51.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea6de3a740 1 -- 172.21.15.2:0/3754983586 shutdown_connections 2024-03-15T12:59:51.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea6de3a740 1 -- 172.21.15.2:0/3754983586 wait complete. 2024-03-15T12:59:51.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.691+0000 7fea6de3a740 20 client.16125 trim_cache size 0 max 16384 2024-03-15T12:59:51.683 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T12:59:51.683 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T12:59:51.683 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T12:59:51.683 INFO:teuthology.orchestra.run.smithi002.stdout: "387fc714-99e7-4bdf-a0c8-9379b44dafef": { 2024-03-15T12:59:51.683 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T12:59:51.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "387fc714-99e7-4bdf-a0c8-9379b44dafef", 2024-03-15T12:59:51.684 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T12:59:51.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T12:59:51.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T12:59:51.684 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T12:59:51.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.692+0000 7fea6b181640 1 -- 172.21.15.2:0/85013092 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fea540a1cc0 msgr2=0x7fea540a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.692+0000 7fea6b181640 1 --2- 172.21.15.2:0/85013092 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fea540a1cc0 0x7fea540a4180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fea640ffe20 tx=0x7fea50005f90 comp rx=0 tx=0).stop 2024-03-15T12:59:51.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.692+0000 7fea6b181640 1 -- 172.21.15.2:0/85013092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea641005e0 msgr2=0x7fea6419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T12:59:51.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.692+0000 7fea6b181640 1 --2- 172.21.15.2:0/85013092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea641005e0 0x7fea6419c640 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fea5800e9f0 tx=0x7fea5800eeb0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.692+0000 7fea6b181640 1 -- 172.21.15.2:0/85013092 shutdown_connections 2024-03-15T12:59:51.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.692+0000 7fea6b181640 1 --2- 172.21.15.2:0/85013092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea64196800 0x7fea64196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.692+0000 7fea6b181640 1 --2- 172.21.15.2:0/85013092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea641056b0 0x7fea6419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.692+0000 7fea6b181640 1 --2- 172.21.15.2:0/85013092 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fea540a1cc0 0x7fea540a4180 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.692+0000 7fea6b181640 1 --2- 172.21.15.2:0/85013092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea641005e0 0x7fea6419c640 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T12:59:51.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.692+0000 7fea6b181640 1 -- 172.21.15.2:0/85013092 >> 172.21.15.2:0/85013092 conn(0x7fea640775a0 msgr2=0x7fea640fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T12:59:51.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.692+0000 7fea6b181640 1 -- 172.21.15.2:0/85013092 shutdown_connections 2024-03-15T12:59:51.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T12:59:51.692+0000 7fea6b181640 1 -- 172.21.15.2:0/85013092 wait complete. 2024-03-15T12:59:51.692 INFO:tasks.cephfs.filesystem:scrub status for tag:387fc714-99e7-4bdf-a0c8-9379b44dafef - {'path': '/', 'tag': '387fc714-99e7-4bdf-a0c8-9379b44dafef', 'options': 'recursive,force'} 2024-03-15T12:59:52.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:51 smithi002 ceph-mon[32060]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T12:59:52.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:51 smithi184 ceph-mon[35993]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T12:59:52.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:51 smithi149 ceph-mon[35927]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T12:59:53.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:52 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T12:59:53.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:52 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T12:59:53.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:52 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T12:59:53.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:52 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T12:59:53.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:52 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T12:59:53.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:52 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T12:59:53.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:52 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T12:59:53.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:52 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T12:59:53.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:52 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T12:59:53.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:52 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T12:59:53.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:52 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T12:59:53.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:52 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T12:59:54.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:53 smithi002 ceph-mon[32060]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:54.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:53 smithi184 ceph-mon[35993]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:54.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:53 smithi149 ceph-mon[35927]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:55.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:54 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T12:59:55.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:54 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T12:59:55.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:54 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T12:59:56.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:55 smithi002 ceph-mon[32060]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:56.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:55 smithi184 ceph-mon[35993]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:56.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:55 smithi149 ceph-mon[35927]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T12:59:58.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:57 smithi002 ceph-mon[32060]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T12:59:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:57 smithi184 ceph-mon[35993]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T12:59:58.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:57 smithi149 ceph-mon[35927]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:00:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:59 smithi002 ceph-mon[32060]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:00:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:00:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 12:59:59 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:00.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:59 smithi184 ceph-mon[35993]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:00:00.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:00:00.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:00.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 12:59:59 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:00.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:59 smithi149 ceph-mon[35927]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:00:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:00:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 12:59:59 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:01.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:00 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:01.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:00 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:01.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:00 smithi149 ceph-mon[35927]: overall HEALTH_OK 2024-03-15T13:00:01.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:00 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:00:01.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:00 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:00:01.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:00 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:01.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:00 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:01.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:00 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:01.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:00 smithi002 ceph-mon[32060]: overall HEALTH_OK 2024-03-15T13:00:01.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:00 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:00:01.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:00 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:00:01.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:00 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:01.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:00 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:01.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:00 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:01.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:00 smithi184 ceph-mon[35993]: overall HEALTH_OK 2024-03-15T13:00:01.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:00 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:00:01.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:00 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:00:01.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:00 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:00:02.023 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:01 smithi149 ceph-mon[35927]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:00:02.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:01 smithi002 ceph-mon[32060]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:00:02.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:01 smithi184 ceph-mon[35993]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:00:03.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:03 smithi149 ceph-mon[35927]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:00:04.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:03 smithi002 ceph-mon[32060]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:00:04.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:03 smithi184 ceph-mon[35993]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:00:06.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:05 smithi002 ceph-mon[32060]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:00:06.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:05 smithi184 ceph-mon[35993]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:00:06.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:05 smithi149 ceph-mon[35927]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:00:08.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:07 smithi002 ceph-mon[32060]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:00:08.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:07 smithi184 ceph-mon[35993]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:00:08.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:07 smithi149 ceph-mon[35927]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:00:10.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:09 smithi002 ceph-mon[32060]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:00:10.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:09 smithi184 ceph-mon[35993]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:00:10.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:09 smithi149 ceph-mon[35927]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:00:12.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:11 smithi002 ceph-mon[32060]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:00:12.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:11 smithi184 ceph-mon[35993]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:00:12.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:11 smithi149 ceph-mon[35927]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:00:13.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:13 smithi149 ceph-mon[35927]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:00:13.991 INFO:tasks.workunit.client.0.smithi002.stderr:Updating files: 94% (11559/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-15T13:00:14.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:13 smithi002 ceph-mon[32060]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:00:14.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:13 smithi184 ceph-mon[35993]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:00:14.118 INFO:tasks.workunit.client.0.smithi002.stderr:Note: switching to 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b'. 2024-03-15T13:00:14.118 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-03-15T13:00:14.118 INFO:tasks.workunit.client.0.smithi002.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-15T13:00:14.118 INFO:tasks.workunit.client.0.smithi002.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-15T13:00:14.118 INFO:tasks.workunit.client.0.smithi002.stderr:state without impacting any branches by switching back to a branch. 2024-03-15T13:00:14.118 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-03-15T13:00:14.118 INFO:tasks.workunit.client.0.smithi002.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-15T13:00:14.118 INFO:tasks.workunit.client.0.smithi002.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-15T13:00:14.118 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-03-15T13:00:14.118 INFO:tasks.workunit.client.0.smithi002.stderr: git switch -c 2024-03-15T13:00:14.118 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-03-15T13:00:14.119 INFO:tasks.workunit.client.0.smithi002.stderr:Or undo this operation with: 2024-03-15T13:00:14.119 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-03-15T13:00:14.119 INFO:tasks.workunit.client.0.smithi002.stderr: git switch - 2024-03-15T13:00:14.119 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-03-15T13:00:14.119 INFO:tasks.workunit.client.0.smithi002.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-15T13:00:14.119 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-03-15T13:00:14.119 INFO:tasks.workunit.client.0.smithi002.stderr:HEAD is now at aba3c1d240c mds/quiesce-db: keep the db thread alive until shutdown 2024-03-15T13:00:14.121 DEBUG:teuthology.orchestra.run.smithi002:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-15T13:00:14.156 INFO:tasks.workunit.client.0.smithi002.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-15T13:00:14.158 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-15T13:00:14.158 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-15T13:00:14.925 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-15T13:00:14.988 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-15T13:00:15.028 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-15T13:00:15.030 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-15T13:00:15.030 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-15T13:00:15.068 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-15T13:00:15.071 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T13:00:15.071 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-15T13:00:15.127 INFO:tasks.workunit:Running workunits matching fs/norstats on client.0... 2024-03-15T13:00:15.128 INFO:tasks.workunit:Running workunit fs/norstats/kernel_untar_tar.sh... 2024-03-15T13:00:15.128 DEBUG:teuthology.orchestra.run.smithi002:workunit test fs/norstats/kernel_untar_tar.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh 2024-03-15T13:00:15.191 INFO:tasks.workunit.client.0.smithi002.stderr:+ KERNEL=linux-4.0.5 2024-03-15T13:00:15.191 INFO:tasks.workunit.client.0.smithi002.stderr:+ wget -q http://download.ceph.com/qa/linux-4.0.5.tar.xz 2024-03-15T13:00:16.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:15 smithi002 ceph-mon[32060]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:00:16.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:15 smithi184 ceph-mon[35993]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:00:16.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:15 smithi149 ceph-mon[35927]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:00:18.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:17 smithi002 ceph-mon[32060]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:00:18.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:17 smithi184 ceph-mon[35993]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:00:18.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:17 smithi149 ceph-mon[35927]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:00:18.723 INFO:tasks.workunit.client.0.smithi002.stderr:+ mkdir untar_tar 2024-03-15T13:00:18.728 INFO:tasks.workunit.client.0.smithi002.stderr:+ cd untar_tar 2024-03-15T13:00:18.731 INFO:tasks.workunit.client.0.smithi002.stderr:+ tar Jxvf ../linux-4.0.5.tar.xz linux-4.0.5/Documentation/ 2024-03-15T13:00:18.757 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ 2024-03-15T13:00:18.760 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/00-INDEX 2024-03-15T13:00:18.763 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/ 2024-03-15T13:00:18.765 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/README 2024-03-15T13:00:18.767 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/obsolete/ 2024-03-15T13:00:18.769 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/obsolete/proc-sys-vm-nr_pdflush_threads 2024-03-15T13:00:18.771 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-bus-usb 2024-03-15T13:00:18.773 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-class-rfkill 2024-03-15T13:00:18.776 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-koneplus 2024-03-15T13:00:18.778 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-kovaplus 2024-03-15T13:00:18.780 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra 2024-03-15T13:00:18.783 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/removed/ 2024-03-15T13:00:18.785 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/removed/devfs 2024-03-15T13:00:18.788 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/removed/dv1394 2024-03-15T13:00:18.791 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/removed/ip_queue 2024-03-15T13:00:18.793 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/removed/net_dma 2024-03-15T13:00:18.797 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/removed/o2cb 2024-03-15T13:00:18.798 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/removed/raw1394 2024-03-15T13:00:18.800 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/removed/video1394 2024-03-15T13:00:18.802 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/ 2024-03-15T13:00:18.805 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/firewire-cdev 2024-03-15T13:00:18.807 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/o2cb 2024-03-15T13:00:18.809 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/syscalls 2024-03-15T13:00:18.811 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-acpi-pmprofile 2024-03-15T13:00:18.813 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-firewire 2024-03-15T13:00:18.816 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-usb 2024-03-15T13:00:18.818 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-xen-backend 2024-03-15T13:00:18.820 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-backlight 2024-03-15T13:00:18.822 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-rfkill 2024-03-15T13:00:18.825 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-tpm 2024-03-15T13:00:18.827 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-ubi 2024-03-15T13:00:18.829 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-udc 2024-03-15T13:00:18.832 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-node 2024-03-15T13:00:18.834 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-cpu 2024-03-15T13:00:18.836 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-xen_memory 2024-03-15T13:00:18.839 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-ib_srp 2024-03-15T13:00:18.842 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-qla2xxx 2024-03-15T13:00:18.844 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-usb-usbtmc 2024-03-15T13:00:18.846 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-w1_ds28e04 2024-03-15T13:00:18.848 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-efi-vars 2024-03-15T13:00:18.851 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-dump 2024-03-15T13:00:18.853 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-elog 2024-03-15T13:00:18.856 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-module 2024-03-15T13:00:18.858 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-transport-srp 2024-03-15T13:00:18.860 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/thermal-notification 2024-03-15T13:00:18.862 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/stable/vdso 2024-03-15T13:00:18.865 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/ 2024-03-15T13:00:18.867 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-spear-pcie-gadget 2024-03-15T13:00:18.870 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget 2024-03-15T13:00:18.872 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-acm 2024-03-15T13:00:18.874 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ecm 2024-03-15T13:00:18.877 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-eem 2024-03-15T13:00:18.879 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ffs 2024-03-15T13:00:18.881 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-hid 2024-03-15T13:00:18.883 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-loopback 2024-03-15T13:00:18.885 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-mass-storage 2024-03-15T13:00:18.888 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-midi 2024-03-15T13:00:18.890 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ncm 2024-03-15T13:00:18.892 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-obex 2024-03-15T13:00:18.894 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-phonet 2024-03-15T13:00:18.896 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-rndis 2024-03-15T13:00:18.899 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-serial 2024-03-15T13:00:18.901 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-sourcesink 2024-03-15T13:00:18.903 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-subset 2024-03-15T13:00:18.905 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac1 2024-03-15T13:00:18.907 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac2 2024-03-15T13:00:18.909 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uvc 2024-03-15T13:00:18.912 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-driver-genwqe 2024-03-15T13:00:18.914 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ec 2024-03-15T13:00:18.916 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ideapad 2024-03-15T13:00:18.918 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-olpc 2024-03-15T13:00:18.920 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pfo-nx-crypto 2024-03-15T13:00:18.923 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pktcdvd 2024-03-15T13:00:18.925 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/dev-kmsg 2024-03-15T13:00:18.927 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/evm 2024-03-15T13:00:18.929 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/ima_policy 2024-03-15T13:00:18.932 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/procfs-diskstats 2024-03-15T13:00:18.934 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/pstore 2024-03-15T13:00:18.936 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ata 2024-03-15T13:00:18.939 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block 2024-03-15T13:00:18.942 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-bcache 2024-03-15T13:00:18.944 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-dm 2024-03-15T13:00:18.947 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-rssd 2024-03-15T13:00:18.949 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-zram 2024-03-15T13:00:18.951 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-acpi 2024-03-15T13:00:18.954 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-amba 2024-03-15T13:00:18.956 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-bcma 2024-03-15T13:00:18.958 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etb10 2024-03-15T13:00:18.961 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etm3x 2024-03-15T13:00:18.963 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-funnel 2024-03-15T13:00:18.966 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc 2024-03-15T13:00:18.968 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-css 2024-03-15T13:00:18.970 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-events 2024-03-15T13:00:18.973 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-format 2024-03-15T13:00:18.975 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7 2024-03-15T13:00:18.977 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_gpci 2024-03-15T13:00:18.979 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-fcoe 2024-03-15T13:00:18.982 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-hsi 2024-03-15T13:00:18.984 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-fsa9480 2024-03-15T13:00:18.986 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352 2024-03-15T13:00:18.988 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-lm3533 2024-03-15T13:00:18.991 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio 2024-03-15T13:00:18.994 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-accel-bmc150 2024-03-15T13:00:18.996 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-ad9523 2024-03-15T13:00:18.998 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-adf4350 2024-03-15T13:00:19.000 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-gyro-bmg160 2024-03-15T13:00:19.003 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-light-lm3533-als 2024-03-15T13:00:19.005 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-mpu6050 2024-03-15T13:00:19.007 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935 2024-03-15T13:00:19.010 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-trigger-sysfs 2024-03-15T13:00:19.012 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mdio 2024-03-15T13:00:19.014 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-media 2024-03-15T13:00:19.017 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mei 2024-03-15T13:00:19.019 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci 2024-03-15T13:00:19.022 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss 2024-03-15T13:00:19.024 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-drivers-ehci_hcd 2024-03-15T13:00:19.025 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-platform 2024-03-15T13:00:19.028 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rbd 2024-03-15T13:00:19.030 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rpmsg 2024-03-15T13:00:19.033 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-umc 2024-03-15T13:00:19.035 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb 2024-03-15T13:00:19.037 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg 2024-03-15T13:00:19.039 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-lvstest 2024-03-15T13:00:19.041 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-c2port 2024-03-15T13:00:19.044 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-cfq-target-latency 2024-03-15T13:00:19.046 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class 2024-03-15T13:00:19.049 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-adp8870 2024-03-15T13:00:19.051 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533 2024-03-15T13:00:19.053 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-bdi 2024-03-15T13:00:19.056 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-cxl 2024-03-15T13:00:19.058 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-devfreq 2024-03-15T13:00:19.061 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-extcon 2024-03-15T13:00:19.063 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu 2024-03-15T13:00:19.065 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-amd-iommu 2024-03-15T13:00:19.067 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-intel-iommu 2024-03-15T13:00:19.070 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-lcd 2024-03-15T13:00:19.072 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led 2024-03-15T13:00:19.074 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led-driver-lm3533 2024-03-15T13:00:19.076 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-leds-gt683r 2024-03-15T13:00:19.079 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mei 2024-03-15T13:00:19.081 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mic.txt 2024-03-15T13:00:19.083 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mtd 2024-03-15T13:00:19.086 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net 2024-03-15T13:00:19.089 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-batman-adv 2024-03-15T13:00:19.091 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-cdc_ncm 2024-03-15T13:00:19.094 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-grcan 2024-03-15T13:00:19.097 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-mesh 2024-03-15T13:00:19.099 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-queues 2024-03-15T13:00:19.102 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-statistics 2024-03-15T13:00:19.104 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pktcdvd 2024-03-15T13:00:19.107 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-power 2024-03-15T13:00:19.110 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-powercap 2024-03-15T13:00:19.113 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pwm 2024-03-15T13:00:19.117 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rc 2024-03-15T13:00:19.118 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-regulator 2024-03-15T13:00:19.121 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rtc-rtc0-device-rtc_calibration 2024-03-15T13:00:19.124 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-scsi_host 2024-03-15T13:00:19.127 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc 2024-03-15T13:00:19.129 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc-wusbhc 2024-03-15T13:00:19.132 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-dev 2024-03-15T13:00:19.136 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices 2024-03-15T13:00:19.138 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-edac 2024-03-15T13:00:19.141 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-firmware_node 2024-03-15T13:00:19.144 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-lpss_ltr 2024-03-15T13:00:19.146 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-memory 2024-03-15T13:00:19.149 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-mmc 2024-03-15T13:00:19.152 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-online 2024-03-15T13:00:19.154 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-_UDC_-gadget 2024-03-15T13:00:19.157 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-docg3 2024-03-15T13:00:19.160 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-sh_mobile_lcdc_fb 2024-03-15T13:00:19.163 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power 2024-03-15T13:00:19.166 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D0 2024-03-15T13:00:19.169 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D1 2024-03-15T13:00:19.171 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D2 2024-03-15T13:00:19.174 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D3hot 2024-03-15T13:00:19.177 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_wakeup 2024-03-15T13:00:19.180 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_state 2024-03-15T13:00:19.182 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-real_power_state 2024-03-15T13:00:19.185 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-resource_in_use 2024-03-15T13:00:19.187 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-soc 2024-03-15T13:00:19.190 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-sun 2024-03-15T13:00:19.192 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-cpu 2024-03-15T13:00:19.194 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-ibm-rtl 2024-03-15T13:00:19.197 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-xen_cpu 2024-03-15T13:00:19.199 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-genwqe 2024-03-15T13:00:19.202 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid 2024-03-15T13:00:19.204 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-lenovo 2024-03-15T13:00:19.206 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-logitech-lg4ff 2024-03-15T13:00:19.209 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-multitouch 2024-03-15T13:00:19.211 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-picolcd 2024-03-15T13:00:19.214 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-prodikeys 2024-03-15T13:00:19.216 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-arvo 2024-03-15T13:00:19.218 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-isku 2024-03-15T13:00:19.221 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone 2024-03-15T13:00:19.223 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-koneplus 2024-03-15T13:00:19.226 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-konepure 2024-03-15T13:00:19.228 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kovaplus 2024-03-15T13:00:19.230 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-lua 2024-03-15T13:00:19.233 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-pyra 2024-03-15T13:00:19.273 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-ryos 2024-03-15T13:00:19.275 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-savu 2024-03-15T13:00:19.278 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-srws1 2024-03-15T13:00:19.280 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-wiimote 2024-03-15T13:00:19.282 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-input-axp-pek 2024-03-15T13:00:19.284 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-intel-rapid-start 2024-03-15T13:00:19.286 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-pciback 2024-03-15T13:00:19.289 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-ppi 2024-03-15T13:00:19.291 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-samsung-laptop 2024-03-15T13:00:19.294 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-sunxi-sid 2024-03-15T13:00:19.296 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-tegra-fuse 2024-03-15T13:00:19.298 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-toshiba_acpi 2024-03-15T13:00:19.301 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-wacom 2024-03-15T13:00:19.303 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkback 2024-03-15T13:00:19.305 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkfront 2024-03-15T13:00:19.307 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-acpi 2024-03-15T13:00:19.310 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-dmi 2024-03-15T13:00:19.313 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi 2024-03-15T13:00:19.315 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi-runtime-map 2024-03-15T13:00:19.317 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-gsmi 2024-03-15T13:00:19.320 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-log 2024-03-15T13:00:19.322 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-memmap 2024-03-15T13:00:19.324 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-ofw 2024-03-15T13:00:19.326 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sfi 2024-03-15T13:00:19.329 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sgi_uv 2024-03-15T13:00:19.331 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-ext4 2024-03-15T13:00:19.333 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-f2fs 2024-03-15T13:00:19.336 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-nilfs2 2024-03-15T13:00:19.338 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-xfs 2024-03-15T13:00:19.340 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-gpio 2024-03-15T13:00:19.343 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-i2c-bmp085 2024-03-15T13:00:19.345 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ibft 2024-03-15T13:00:19.347 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-boot_params 2024-03-15T13:00:19.349 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-fscaps 2024-03-15T13:00:19.352 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-iommu_groups 2024-03-15T13:00:19.354 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-livepatch 2024-03-15T13:00:19.357 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm 2024-03-15T13:00:19.359 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-hugepages 2024-03-15T13:00:19.362 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-ksm 2024-03-15T13:00:19.364 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-slab 2024-03-15T13:00:19.367 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-uids 2024-03-15T13:00:19.369 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo 2024-03-15T13:00:19.371 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-memory-page-offline 2024-03-15T13:00:19.373 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-module 2024-03-15T13:00:19.376 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ocfs2 2024-03-15T13:00:19.378 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-laptop 2024-03-15T13:00:19.380 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-wmi 2024-03-15T13:00:19.383 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-at91 2024-03-15T13:00:19.385 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-brcmstb-gisb-arb 2024-03-15T13:00:19.387 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-chipidea-usb-otg 2024-03-15T13:00:19.390 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-eeepc-laptop 2024-03-15T13:00:19.392 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ideapad-laptop 2024-03-15T13:00:19.394 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-kim 2024-03-15T13:00:19.397 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-msi-laptop 2024-03-15T13:00:19.399 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-tahvo-usb 2024-03-15T13:00:19.401 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ts5500 2024-03-15T13:00:19.403 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-power 2024-03-15T13:00:19.406 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-pps 2024-03-15T13:00:19.409 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-profiling 2024-03-15T13:00:19.410 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ptp 2024-03-15T13:00:19.413 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-tty 2024-03-15T13:00:19.415 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-wusb_cbaf 2024-03-15T13:00:19.418 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/BUG-HUNTING 2024-03-15T13:00:19.421 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/Changes 2024-03-15T13:00:19.423 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/CodeOfConflict 2024-03-15T13:00:19.425 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/CodingStyle 2024-03-15T13:00:19.428 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DMA-API-HOWTO.txt 2024-03-15T13:00:19.431 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DMA-API.txt 2024-03-15T13:00:19.434 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DMA-ISA-LPC.txt 2024-03-15T13:00:19.436 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DMA-attributes.txt 2024-03-15T13:00:19.438 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/ 2024-03-15T13:00:19.441 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/.gitignore 2024-03-15T13:00:19.443 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/80211.tmpl 2024-03-15T13:00:19.446 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/Makefile 2024-03-15T13:00:19.448 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/alsa-driver-api.tmpl 2024-03-15T13:00:19.451 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/crypto-API.tmpl 2024-03-15T13:00:19.454 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/debugobjects.tmpl 2024-03-15T13:00:19.456 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/device-drivers.tmpl 2024-03-15T13:00:19.459 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/deviceiobook.tmpl 2024-03-15T13:00:19.461 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/drm.tmpl 2024-03-15T13:00:19.468 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/filesystems.tmpl 2024-03-15T13:00:19.470 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/gadget.tmpl 2024-03-15T13:00:19.473 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/genericirq.tmpl 2024-03-15T13:00:19.476 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/kernel-api.tmpl 2024-03-15T13:00:19.478 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/kernel-hacking.tmpl 2024-03-15T13:00:19.481 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/kernel-locking.tmpl 2024-03-15T13:00:19.484 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/kgdb.tmpl 2024-03-15T13:00:19.488 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/libata.tmpl 2024-03-15T13:00:19.491 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/librs.tmpl 2024-03-15T13:00:19.494 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/lsm.tmpl 2024-03-15T13:00:19.496 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/ 2024-03-15T13:00:19.498 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/Makefile 2024-03-15T13:00:19.500 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/bayer.png.b64 2024-03-15T13:00:19.502 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/constraints.png.b64 2024-03-15T13:00:19.505 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/crop.gif.b64 2024-03-15T13:00:19.507 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ 2024-03-15T13:00:19.509 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/.gitignore 2024-03-15T13:00:19.512 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/audio.xml 2024-03-15T13:00:19.515 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ca.xml 2024-03-15T13:00:19.518 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/demux.xml 2024-03-15T13:00:19.521 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbapi.xml 2024-03-15T13:00:19.523 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbproperty.xml 2024-03-15T13:00:19.527 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbstb.pdf 2024-03-15T13:00:19.529 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/examples.xml 2024-03-15T13:00:19.532 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/frontend.xml 2024-03-15T13:00:19.535 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/intro.xml 2024-03-15T13:00:19.537 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/kdapi.xml 2024-03-15T13:00:19.541 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/net.xml 2024-03-15T13:00:19.543 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/video.xml 2024-03-15T13:00:19.548 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/dvbstb.png.b64 2024-03-15T13:00:19.552 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_bt.gif.b64 2024-03-15T13:00:19.554 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_tb.gif.b64 2024-03-15T13:00:19.557 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt.gif.b64 2024-03-15T13:00:19.560 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt_example.gif.b64 2024-03-15T13:00:19.562 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/pipeline.png.b64 2024-03-15T13:00:19.565 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/selection.png.b64 2024-03-15T13:00:19.567 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/ 2024-03-15T13:00:19.570 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/.gitignore 2024-03-15T13:00:19.572 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/biblio.xml 2024-03-15T13:00:19.574 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/capture.c.xml 2024-03-15T13:00:19.577 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/common.xml 2024-03-15T13:00:19.580 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/compat.xml 2024-03-15T13:00:19.585 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/controls.xml 2024-03-15T13:00:19.592 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/crop.pdf 2024-03-15T13:00:19.595 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-capture.xml 2024-03-15T13:00:19.597 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-codec.xml 2024-03-15T13:00:19.600 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-effect.xml 2024-03-15T13:00:19.602 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-event.xml 2024-03-15T13:00:19.604 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-osd.xml 2024-03-15T13:00:19.607 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-output.xml 2024-03-15T13:00:19.609 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-overlay.xml 2024-03-15T13:00:19.612 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-radio.xml 2024-03-15T13:00:19.614 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-raw-vbi.xml 2024-03-15T13:00:19.617 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-rds.xml 2024-03-15T13:00:19.619 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sdr.xml 2024-03-15T13:00:19.621 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sliced-vbi.xml 2024-03-15T13:00:19.625 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-subdev.xml 2024-03-15T13:00:19.627 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-teletext.xml 2024-03-15T13:00:19.629 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/driver.xml 2024-03-15T13:00:19.632 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fdl-appendix.xml 2024-03-15T13:00:19.635 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_bt.pdf 2024-03-15T13:00:19.637 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_tb.pdf 2024-03-15T13:00:19.640 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-close.xml 2024-03-15T13:00:19.642 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-ioctl.xml 2024-03-15T13:00:19.644 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-mmap.xml 2024-03-15T13:00:19.647 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-munmap.xml 2024-03-15T13:00:19.649 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-open.xml 2024-03-15T13:00:19.652 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-poll.xml 2024-03-15T13:00:19.654 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-read.xml 2024-03-15T13:00:19.657 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-select.xml 2024-03-15T13:00:19.659 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-write.xml 2024-03-15T13:00:19.662 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/gen-errors.xml 2024-03-15T13:00:19.664 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/io.xml 2024-03-15T13:00:19.668 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/keytable.c.xml 2024-03-15T13:00:19.670 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/libv4l.xml 2024-03-15T13:00:19.673 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/lirc_device_interface.xml 2024-03-15T13:00:19.675 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-controller.xml 2024-03-15T13:00:19.680 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-close.xml 2024-03-15T13:00:19.682 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-ioctl.xml 2024-03-15T13:00:19.685 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-open.xml 2024-03-15T13:00:19.687 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-device-info.xml 2024-03-15T13:00:19.690 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-entities.xml 2024-03-15T13:00:19.692 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-links.xml 2024-03-15T13:00:19.695 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-setup-link.xml 2024-03-15T13:00:19.697 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pipeline.pdf 2024-03-15T13:00:19.700 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-grey.xml 2024-03-15T13:00:19.702 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-m420.xml 2024-03-15T13:00:19.705 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12.xml 2024-03-15T13:00:19.707 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12m.xml 2024-03-15T13:00:19.709 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12mt.xml 2024-03-15T13:00:19.712 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16.xml 2024-03-15T13:00:19.714 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16m.xml 2024-03-15T13:00:19.717 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv24.xml 2024-03-15T13:00:19.719 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-rgb.xml 2024-03-15T13:00:19.723 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-yuv.xml 2024-03-15T13:00:19.725 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr16.xml 2024-03-15T13:00:19.727 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr8.xml 2024-03-15T13:00:19.729 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs08.xml 2024-03-15T13:00:19.732 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs14le.xml 2024-03-15T13:00:19.734 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu08.xml 2024-03-15T13:00:19.736 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu16le.xml 2024-03-15T13:00:19.739 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-ru12le.xml 2024-03-15T13:00:19.741 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgbrg8.xml 2024-03-15T13:00:19.743 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgrbg8.xml 2024-03-15T13:00:19.746 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10.xml 2024-03-15T13:00:19.748 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10alaw8.xml 2024-03-15T13:00:19.750 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10dpcm8.xml 2024-03-15T13:00:19.753 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10p.xml 2024-03-15T13:00:19.755 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb12.xml 2024-03-15T13:00:19.757 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb8.xml 2024-03-15T13:00:19.759 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uv8.xml 2024-03-15T13:00:19.762 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uyvy.xml 2024-03-15T13:00:19.764 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-vyuy.xml 2024-03-15T13:00:19.766 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10.xml 2024-03-15T13:00:19.768 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10b.xml 2024-03-15T13:00:19.771 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y12.xml 2024-03-15T13:00:19.773 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y16.xml 2024-03-15T13:00:19.775 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y41p.xml 2024-03-15T13:00:19.778 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv410.xml 2024-03-15T13:00:19.780 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv411p.xml 2024-03-15T13:00:19.782 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420.xml 2024-03-15T13:00:19.785 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420m.xml 2024-03-15T13:00:19.787 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv422p.xml 2024-03-15T13:00:19.790 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuyv.xml 2024-03-15T13:00:19.792 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvu420m.xml 2024-03-15T13:00:19.795 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvyu.xml 2024-03-15T13:00:19.800 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt.xml 2024-03-15T13:00:19.804 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/planar-apis.xml 2024-03-15T13:00:19.806 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/remote_controllers.xml 2024-03-15T13:00:19.809 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selection-api.xml 2024-03-15T13:00:19.811 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selections-common.xml 2024-03-15T13:00:19.814 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-formats.xml 2024-03-15T13:00:19.819 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.dia 2024-03-15T13:00:19.822 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.svg 2024-03-15T13:00:19.824 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.dia 2024-03-15T13:00:19.827 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.svg 2024-03-15T13:00:19.830 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.dia 2024-03-15T13:00:19.833 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.svg 2024-03-15T13:00:19.836 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2.xml 2024-03-15T13:00:19.838 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2grab.c.xml 2024-03-15T13:00:19.841 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_525.pdf 2024-03-15T13:00:19.843 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_625.pdf 2024-03-15T13:00:19.845 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_hsync.pdf 2024-03-15T13:00:19.847 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml 2024-03-15T13:00:19.850 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-cropcap.xml 2024-03-15T13:00:19.852 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-chip-info.xml 2024-03-15T13:00:19.855 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-register.xml 2024-03-15T13:00:19.857 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-decoder-cmd.xml 2024-03-15T13:00:19.860 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dqevent.xml 2024-03-15T13:00:19.863 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dv-timings-cap.xml 2024-03-15T13:00:19.865 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-encoder-cmd.xml 2024-03-15T13:00:19.868 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-dv-timings.xml 2024-03-15T13:00:19.870 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-fmt.xml 2024-03-15T13:00:19.872 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-frameintervals.xml 2024-03-15T13:00:19.874 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-framesizes.xml 2024-03-15T13:00:19.877 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-freq-bands.xml 2024-03-15T13:00:19.879 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudio.xml 2024-03-15T13:00:19.881 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudioout.xml 2024-03-15T13:00:19.884 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enuminput.xml 2024-03-15T13:00:19.886 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumoutput.xml 2024-03-15T13:00:19.888 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumstd.xml 2024-03-15T13:00:19.892 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-expbuf.xml 2024-03-15T13:00:19.894 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audio.xml 2024-03-15T13:00:19.896 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audioout.xml 2024-03-15T13:00:19.899 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-crop.xml 2024-03-15T13:00:19.901 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ctrl.xml 2024-03-15T13:00:19.904 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-dv-timings.xml 2024-03-15T13:00:19.906 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-edid.xml 2024-03-15T13:00:19.909 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-enc-index.xml 2024-03-15T13:00:19.911 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ext-ctrls.xml 2024-03-15T13:00:19.914 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fbuf.xml 2024-03-15T13:00:19.916 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fmt.xml 2024-03-15T13:00:19.919 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-frequency.xml 2024-03-15T13:00:19.921 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-input.xml 2024-03-15T13:00:19.923 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-jpegcomp.xml 2024-03-15T13:00:19.926 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-modulator.xml 2024-03-15T13:00:19.929 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-output.xml 2024-03-15T13:00:19.932 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-parm.xml 2024-03-15T13:00:19.934 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-priority.xml 2024-03-15T13:00:19.936 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-selection.xml 2024-03-15T13:00:19.939 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-sliced-vbi-cap.xml 2024-03-15T13:00:19.942 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-std.xml 2024-03-15T13:00:19.944 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-tuner.xml 2024-03-15T13:00:19.947 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-log-status.xml 2024-03-15T13:00:19.949 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-overlay.xml 2024-03-15T13:00:19.951 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-prepare-buf.xml 2024-03-15T13:00:19.954 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-qbuf.xml 2024-03-15T13:00:19.956 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-query-dv-timings.xml 2024-03-15T13:00:19.958 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querybuf.xml 2024-03-15T13:00:20.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:19 smithi002 ceph-mon[32060]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-15T13:00:20.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:19 smithi184 ceph-mon[35993]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-15T13:00:20.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:19 smithi149 ceph-mon[35927]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-15T13:00:21.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:00:21.694 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:00:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.828+0000 7f506a9e4640 1 -- 172.21.15.2:0/2856669756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5064106890 msgr2=0x7f506410e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.828+0000 7f506a9e4640 1 --2- 172.21.15.2:0/2856669756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5064106890 0x7f506410e410 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f505c009ca0 tx=0x7f505c02c0f0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.828+0000 7f506a9e4640 1 -- 172.21.15.2:0/2856669756 shutdown_connections 2024-03-15T13:00:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.828+0000 7f506a9e4640 1 --2- 172.21.15.2:0/2856669756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5064106890 0x7f506410e410 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.828+0000 7f506a9e4640 1 --2- 172.21.15.2:0/2856669756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5064100610 0x7f5064106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.828+0000 7f506a9e4640 1 --2- 172.21.15.2:0/2856669756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50640ffc60 0x7f5064100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.828+0000 7f506a9e4640 1 -- 172.21.15.2:0/2856669756 >> 172.21.15.2:0/2856669756 conn(0x7f5064076910 msgr2=0x7f50640fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.828+0000 7f506a9e4640 1 -- 172.21.15.2:0/2856669756 shutdown_connections 2024-03-15T13:00:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.828+0000 7f506a9e4640 1 -- 172.21.15.2:0/2856669756 wait complete. 2024-03-15T13:00:21.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.829+0000 7f506a9e4640 1 Processor -- start 2024-03-15T13:00:21.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.829+0000 7f506a9e4640 1 -- start start 2024-03-15T13:00:21.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.829+0000 7f506a9e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50640ffc60 0x7f506419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.829+0000 7f506a9e4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5064106890 0x7f506419dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.829+0000 7f506a9e4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50641978d0 0x7f5064197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.829+0000 7f506a9e4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50641982c0 con 0x7f50641978d0 2024-03-15T13:00:21.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.829+0000 7f506a9e4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5064198430 con 0x7f5064106890 2024-03-15T13:00:21.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.829+0000 7f506a9e4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50641985f0 con 0x7f50640ffc60 2024-03-15T13:00:21.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.829+0000 7f5068f81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50641978d0 0x7f5064197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.830+0000 7f5068f81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50641978d0 0x7f5064197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53692/0 (socket says 172.21.15.2:53692) 2024-03-15T13:00:21.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.830+0000 7f5063fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50640ffc60 0x7f506419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.830+0000 7f5068f81640 1 -- 172.21.15.2:0/2309679685 learned_addr learned my addr 172.21.15.2:0/2309679685 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:21.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.830+0000 7f50637fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5064106890 0x7f506419dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.830+0000 7f5063fff640 1 -- 172.21.15.2:0/2309679685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5064106890 msgr2=0x7f506419dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.830+0000 7f5063fff640 1 --2- 172.21.15.2:0/2309679685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5064106890 0x7f506419dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.830+0000 7f5063fff640 1 -- 172.21.15.2:0/2309679685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50641978d0 msgr2=0x7f5064197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.830+0000 7f5063fff640 1 --2- 172.21.15.2:0/2309679685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50641978d0 0x7f5064197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.830+0000 7f5063fff640 1 -- 172.21.15.2:0/2309679685 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c009900 con 0x7f50640ffc60 2024-03-15T13:00:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.830+0000 7f5063fff640 1 --2- 172.21.15.2:0/2309679685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50640ffc60 0x7f506419d6d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f504c00e9f0 tx=0x7f504c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.830+0000 7f50617fa640 1 -- 172.21.15.2:0/2309679685 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c00ccc0 con 0x7f50640ffc60 2024-03-15T13:00:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.830+0000 7f50617fa640 1 -- 172.21.15.2:0/2309679685 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f504c00ce20 con 0x7f50640ffc60 2024-03-15T13:00:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.830+0000 7f506a9e4640 1 -- 172.21.15.2:0/2309679685 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50641a4590 con 0x7f50640ffc60 2024-03-15T13:00:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.830+0000 7f50617fa640 1 -- 172.21.15.2:0/2309679685 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c004a00 con 0x7f50640ffc60 2024-03-15T13:00:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.830+0000 7f506a9e4640 1 -- 172.21.15.2:0/2309679685 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50641a4a50 con 0x7f50640ffc60 2024-03-15T13:00:21.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.832+0000 7f50617fa640 1 -- 172.21.15.2:0/2309679685 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7f504c004b60 con 0x7f50640ffc60 2024-03-15T13:00:21.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f506d5b6740 1 Processor -- start 2024-03-15T13:00:21.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f506d5b6740 1 -- start start 2024-03-15T13:00:21.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f50617fa640 1 --2- 172.21.15.2:0/2309679685 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f50500a1cc0 0x7f50500a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f5068f81640 1 --2- 172.21.15.2:0/2309679685 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f50500a1cc0 0x7f50500a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f506d5b6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c63448690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f506d5b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c63448bd0 0x564c63451270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f506d5b6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c634517b0 0x564c63453ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f50617fa640 1 -- 172.21.15.2:0/2309679685 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f504c014070 con 0x7f50640ffc60 2024-03-15T13:00:21.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f506d5b6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c633b7280 con 0x564c634517b0 2024-03-15T13:00:21.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f506d5b6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c633b47b0 con 0x564c634482b0 2024-03-15T13:00:21.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f506d5b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c633aff90 con 0x564c63448bd0 2024-03-15T13:00:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f5068f81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c634517b0 0x564c63453ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f5063fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c63448690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f50637fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c63448bd0 0x564c63451270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f5068f81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c634517b0 0x564c63453ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53704/0 (socket says 172.21.15.2:53704) 2024-03-15T13:00:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f5063fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c63448690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39758/0 (socket says 172.21.15.2:39758) 2024-03-15T13:00:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.833+0000 7f5068f81640 1 -- 172.21.15.2:0/3187245741 learned_addr learned my addr 172.21.15.2:0/3187245741 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.834+0000 7f50637fe640 1 -- 172.21.15.2:0/3187245741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 msgr2=0x564c63448690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.834+0000 7f5068f81640 1 --2- 172.21.15.2:0/2309679685 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f50500a1cc0 0x7f50500a4180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f505c037040 tx=0x7f505c034440 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.834+0000 7f50637fe640 1 --2- 172.21.15.2:0/3187245741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c63448690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.834+0000 7f50637fe640 1 -- 172.21.15.2:0/3187245741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c634517b0 msgr2=0x564c63453ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.834+0000 7f50637fe640 1 --2- 172.21.15.2:0/3187245741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c634517b0 0x564c63453ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.834+0000 7f50637fe640 1 -- 172.21.15.2:0/3187245741 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c6336e570 con 0x564c63448bd0 2024-03-15T13:00:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.834+0000 7f5068f81640 1 --2- 172.21.15.2:0/3187245741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c634517b0 0x564c63453ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.834+0000 7f50637fe640 1 --2- 172.21.15.2:0/3187245741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c63448bd0 0x564c63451270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5064199070 tx=0x7f5054011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc4448959ce52fca server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.834+0000 7f505a7fc640 1 -- 172.21.15.2:0/3187245741 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054004050 con 0x564c63448bd0 2024-03-15T13:00:21.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.834+0000 7f505a7fc640 1 -- 172.21.15.2:0/3187245741 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5054005020 con 0x564c63448bd0 2024-03-15T13:00:21.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.834+0000 7f505a7fc640 1 -- 172.21.15.2:0/3187245741 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054004b60 con 0x564c63448bd0 2024-03-15T13:00:21.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.835+0000 7f506d5b6740 1 -- 172.21.15.2:0/3187245741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c63448bd0 msgr2=0x564c63451270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:21.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.835+0000 7f506d5b6740 1 --2- 172.21.15.2:0/3187245741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c63448bd0 0x564c63451270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5064199070 tx=0x7f5054011810 comp rx=0 tx=0).stop 2024-03-15T13:00:21.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.835+0000 7f506d5b6740 1 -- 172.21.15.2:0/3187245741 shutdown_connections 2024-03-15T13:00:21.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.835+0000 7f506d5b6740 1 --2- 172.21.15.2:0/3187245741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c634517b0 0x564c63453ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.835+0000 7f506d5b6740 1 --2- 172.21.15.2:0/3187245741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c63448bd0 0x564c63451270 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.835+0000 7f506d5b6740 1 --2- 172.21.15.2:0/3187245741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c63448690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.835+0000 7f506d5b6740 1 -- 172.21.15.2:0/3187245741 >> 172.21.15.2:0/3187245741 conn(0x564c63442990 msgr2=0x564c63442d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:21.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.835+0000 7f506d5b6740 1 -- 172.21.15.2:0/3187245741 shutdown_connections 2024-03-15T13:00:21.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.835+0000 7f506d5b6740 1 -- 172.21.15.2:0/3187245741 wait complete. 2024-03-15T13:00:21.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.835+0000 7f506d5b6740 1 -- start start 2024-03-15T13:00:21.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.835+0000 7f506d5b6740 1 Processor -- start 2024-03-15T13:00:21.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f506d5b6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c6344e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f506d5b6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c63448bd0 0x564c6344eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f506d5b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c634517b0 0x564c6344f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f506d5b6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c63083430 con 0x564c63448bd0 2024-03-15T13:00:21.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f506d5b6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6303fe00 con 0x564c634482b0 2024-03-15T13:00:21.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f506d5b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c63334fd0 con 0x564c634517b0 2024-03-15T13:00:21.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f506d5b6740 10 client.0 _ll_register_callbacks cb 0x564c6338afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:21.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f50637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c63448bd0 0x564c6344eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f50637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c63448bd0 0x564c6344eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53714/0 (socket says 172.21.15.2:53714) 2024-03-15T13:00:21.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f5063fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c6344e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f50637fe640 1 -- 172.21.15.2:0/3669990697 learned_addr learned my addr 172.21.15.2:0/3669990697 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:21.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f5068f81640 1 --2- 172.21.15.2:0/3669990697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c634517b0 0x564c6344f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f50637fe640 1 -- 172.21.15.2:0/3669990697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c634517b0 msgr2=0x564c6344f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:21.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f50637fe640 1 --2- 172.21.15.2:0/3669990697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c634517b0 0x564c6344f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f50637fe640 1 -- 172.21.15.2:0/3669990697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 msgr2=0x564c6344e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f50637fe640 1 --2- 172.21.15.2:0/3669990697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c6344e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f50637fe640 1 -- 172.21.15.2:0/3669990697 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5054009850 con 0x564c63448bd0 2024-03-15T13:00:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f5063fff640 1 --2- 172.21.15.2:0/3669990697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c6344e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f50637fe640 1 --2- 172.21.15.2:0/3669990697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c63448bd0 0x564c6344eb80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f505401b040 tx=0x7f5054005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30972a3566836b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f50437fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.836+0000 7f50437fe640 1 -- 172.21.15.2:0/3669990697 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054004250 con 0x564c63448bd0 2024-03-15T13:00:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.837+0000 7f50437fe640 1 -- 172.21.15.2:0/3669990697 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f50540043b0 con 0x564c63448bd0 2024-03-15T13:00:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.837+0000 7f506d5b6740 1 -- 172.21.15.2:0/3669990697 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c630af7e0 con 0x564c63448bd0 2024-03-15T13:00:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.837+0000 7f50437fe640 1 -- 172.21.15.2:0/3669990697 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5054004940 con 0x564c63448bd0 2024-03-15T13:00:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.837+0000 7f506d5b6740 10 client.16137 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.837+0000 7f506d5b6740 1 -- 172.21.15.2:0/3669990697 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564c633ae610 con 0x564c63448bd0 2024-03-15T13:00:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.837+0000 7f50437fe640 1 -- 172.21.15.2:0/3669990697 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x564c630af7e0 con 0x564c63448bd0 2024-03-15T13:00:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.837+0000 7f50437fe640 1 -- 172.21.15.2:0/3669990697 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5054023440 con 0x564c63448bd0 2024-03-15T13:00:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.837+0000 7f506d5b6740 10 client.16137 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.837+0000 7f506d5b6740 10 client.16137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:21.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.837+0000 7f506d5b6740 20 client.16137 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:21.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.837+0000 7f506d5b6740 1 --2- 172.21.15.2:0/3669990697 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564c634618d0 0x564c63463cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.837+0000 7f506d5b6740 4 client.16137 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:21.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.837+0000 7f506d5b6740 1 -- 172.21.15.2:0/3669990697 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c630ec720 con 0x564c634618d0 2024-03-15T13:00:21.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.837+0000 7f5063fff640 1 --2- 172.21.15.2:0/3669990697 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564c634618d0 0x564c63463cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.838+0000 7f50437fe640 10 client.16137 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:21.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.838+0000 7f5063fff640 1 --2- 172.21.15.2:0/3669990697 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564c634618d0 0x564c63463cf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4973091618e8344c server_cookie=19871986031d4e92 in_seq=0 out_seq=0 2024-03-15T13:00:21.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.838+0000 7f50437fe640 1 -- 172.21.15.2:0/3669990697 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564c630ec720 con 0x564c634618d0 2024-03-15T13:00:21.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.838+0000 7f50437fe640 10 client.16137 handle_command_reply: tid=0 2024-03-15T13:00:21.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.838+0000 7f506d5b6740 1 client.16137 shutdown 2024-03-15T13:00:21.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.839+0000 7f506d5b6740 1 -- 172.21.15.2:0/3669990697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c63448bd0 msgr2=0x564c6344eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:21.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.839+0000 7f506d5b6740 1 --2- 172.21.15.2:0/3669990697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c63448bd0 0x564c6344eb80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f505401b040 tx=0x7f5054005680 comp rx=0 tx=0).stop 2024-03-15T13:00:21.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.839+0000 7f506d5b6740 1 -- 172.21.15.2:0/3669990697 shutdown_connections 2024-03-15T13:00:21.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.839+0000 7f506d5b6740 1 --2- 172.21.15.2:0/3669990697 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564c634618d0 0x564c63463cf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.839+0000 7f506d5b6740 1 --2- 172.21.15.2:0/3669990697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c634517b0 0x564c6344f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.839+0000 7f50437fe640 0 client.16137 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:21.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.839+0000 7f506d5b6740 1 --2- 172.21.15.2:0/3669990697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c63448bd0 0x564c6344eb80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.839+0000 7f506d5b6740 1 --2- 172.21.15.2:0/3669990697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c6344e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.839+0000 7f506d5b6740 1 -- 172.21.15.2:0/3669990697 >> 172.21.15.2:0/3669990697 conn(0x564c63442990 msgr2=0x564c63444730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:21.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.839+0000 7f506d5b6740 1 -- 172.21.15.2:0/3669990697 shutdown_connections 2024-03-15T13:00:21.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.839+0000 7f506d5b6740 1 -- 172.21.15.2:0/3669990697 wait complete. 2024-03-15T13:00:21.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.839+0000 7f506d5b6740 20 client.16137 trim_cache size 0 max 16384 2024-03-15T13:00:21.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f506d5b6740 1 Processor -- start 2024-03-15T13:00:21.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f506d5b6740 1 -- start start 2024-03-15T13:00:21.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f506d5b6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c63450d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f506d5b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c63448bd0 0x564c6344eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f506d5b6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c634517b0 0x564c6345cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f506d5b6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c63453fa0 con 0x564c634517b0 2024-03-15T13:00:21.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f506d5b6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c63443a10 con 0x564c634482b0 2024-03-15T13:00:21.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f506d5b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c63442160 con 0x564c63448bd0 2024-03-15T13:00:21.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f5068f81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c634517b0 0x564c6345cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f5068f81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c634517b0 0x564c6345cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53720/0 (socket says 172.21.15.2:53720) 2024-03-15T13:00:21.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f5068f81640 1 -- 172.21.15.2:0/1702189167 learned_addr learned my addr 172.21.15.2:0/1702189167 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:21.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f5063fff640 1 --2- 172.21.15.2:0/1702189167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c63450d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f50637fe640 1 --2- 172.21.15.2:0/1702189167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c63448bd0 0x564c6344eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f5068f81640 1 -- 172.21.15.2:0/1702189167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c63448bd0 msgr2=0x564c6344eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:21.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f5068f81640 1 --2- 172.21.15.2:0/1702189167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c63448bd0 0x564c6344eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f5068f81640 1 -- 172.21.15.2:0/1702189167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 msgr2=0x564c63450d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:21.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f5068f81640 1 --2- 172.21.15.2:0/1702189167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c63450d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.855+0000 7f5068f81640 1 -- 172.21.15.2:0/1702189167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c6334aac0 con 0x564c634517b0 2024-03-15T13:00:21.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.856+0000 7f5068f81640 1 --2- 172.21.15.2:0/1702189167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c634517b0 0x564c6345cb30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f505c005cc0 tx=0x7f505c046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832e9c00454f6a1a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:21.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.856+0000 7f5059ffb640 1 -- 172.21.15.2:0/1702189167 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c032880 con 0x564c634517b0 2024-03-15T13:00:21.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.856+0000 7f5059ffb640 1 -- 172.21.15.2:0/1702189167 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f505c053070 con 0x564c634517b0 2024-03-15T13:00:21.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.856+0000 7f5059ffb640 1 -- 172.21.15.2:0/1702189167 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c004560 con 0x564c634517b0 2024-03-15T13:00:21.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.856+0000 7f506d5b6740 1 -- 172.21.15.2:0/1702189167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c634517b0 msgr2=0x564c6345cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:21.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.856+0000 7f506d5b6740 1 --2- 172.21.15.2:0/1702189167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c634517b0 0x564c6345cb30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f505c005cc0 tx=0x7f505c046000 comp rx=0 tx=0).stop 2024-03-15T13:00:21.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.856+0000 7f506d5b6740 1 -- 172.21.15.2:0/1702189167 shutdown_connections 2024-03-15T13:00:21.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.856+0000 7f506d5b6740 1 --2- 172.21.15.2:0/1702189167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c634517b0 0x564c6345cb30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.856+0000 7f506d5b6740 1 --2- 172.21.15.2:0/1702189167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c63448bd0 0x564c6344eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.856+0000 7f506d5b6740 1 --2- 172.21.15.2:0/1702189167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c63450d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.856+0000 7f506d5b6740 1 -- 172.21.15.2:0/1702189167 >> 172.21.15.2:0/1702189167 conn(0x564c63442990 msgr2=0x564c63441a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:21.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.856+0000 7f506d5b6740 1 -- 172.21.15.2:0/1702189167 shutdown_connections 2024-03-15T13:00:21.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.856+0000 7f506d5b6740 1 -- 172.21.15.2:0/1702189167 wait complete. 2024-03-15T13:00:21.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f506d5b6740 1 -- start start 2024-03-15T13:00:21.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f506d5b6740 1 Processor -- start 2024-03-15T13:00:21.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f506d5b6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c63458ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f506d5b6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c63448bd0 0x564c634591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f506d5b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c634517b0 0x564c63455170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f506d5b6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c63455780 con 0x564c63448bd0 2024-03-15T13:00:21.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f506d5b6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c634558f0 con 0x564c634482b0 2024-03-15T13:00:21.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f506d5b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c63455ab0 con 0x564c634517b0 2024-03-15T13:00:21.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f506d5b6740 10 client.0 _ll_register_callbacks cb 0x7f5054005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:21.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f50637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c63448bd0 0x564c634591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f50637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c63448bd0 0x564c634591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53736/0 (socket says 172.21.15.2:53736) 2024-03-15T13:00:21.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f50637fe640 1 -- 172.21.15.2:0/407814910 learned_addr learned my addr 172.21.15.2:0/407814910 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:21.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f5068f81640 1 --2- 172.21.15.2:0/407814910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c634517b0 0x564c63455170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f5063fff640 1 --2- 172.21.15.2:0/407814910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c63458ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f50637fe640 1 -- 172.21.15.2:0/407814910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c634517b0 msgr2=0x564c63455170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:21.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.857+0000 7f50637fe640 1 --2- 172.21.15.2:0/407814910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c634517b0 0x564c63455170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f50637fe640 1 -- 172.21.15.2:0/407814910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 msgr2=0x564c63458ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:21.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f50637fe640 1 --2- 172.21.15.2:0/407814910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c63458ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f50637fe640 1 -- 172.21.15.2:0/407814910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c030070 con 0x564c63448bd0 2024-03-15T13:00:21.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f50637fe640 1 --2- 172.21.15.2:0/407814910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c63448bd0 0x564c634591e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x564c6344dea0 tx=0x7f50540054c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d599ff200c148ad7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:21.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f50437fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f50437fe640 1 -- 172.21.15.2:0/407814910 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054003a00 con 0x564c63448bd0 2024-03-15T13:00:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f50437fe640 1 -- 172.21.15.2:0/407814910 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5054003b60 con 0x564c63448bd0 2024-03-15T13:00:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f506d5b6740 1 -- 172.21.15.2:0/407814910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5054005020 con 0x564c63448bd0 2024-03-15T13:00:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f50437fe640 1 -- 172.21.15.2:0/407814910 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5054009a70 con 0x564c63448bd0 2024-03-15T13:00:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f506d5b6740 10 client.16143 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f506d5b6740 1 -- 172.21.15.2:0/407814910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564c633706a0 con 0x564c63448bd0 2024-03-15T13:00:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f50437fe640 1 -- 172.21.15.2:0/407814910 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5054005020 con 0x564c63448bd0 2024-03-15T13:00:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f50437fe640 1 -- 172.21.15.2:0/407814910 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f505402da10 con 0x564c63448bd0 2024-03-15T13:00:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f506d5b6740 10 client.16143 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f506d5b6740 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f506d5b6740 20 client.16143 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f506d5b6740 1 --2- 172.21.15.2:0/407814910 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564c634bdf00 0x564c634c0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f506d5b6740 4 client.16143 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:00:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.858+0000 7f506d5b6740 1 -- 172.21.15.2:0/407814910 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564c6313b890 con 0x564c634bdf00 2024-03-15T13:00:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f5063fff640 1 --2- 172.21.15.2:0/407814910 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564c634bdf00 0x564c634c0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f5063fff640 1 --2- 172.21.15.2:0/407814910 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564c634bdf00 0x564c634c0320 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b739ee2d9bbdd7e server_cookie=ab1a75c17feacb22 in_seq=0 out_seq=0 2024-03-15T13:00:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f50437fe640 10 client.16143 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f50437fe640 1 -- 172.21.15.2:0/407814910 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564c6313b890 con 0x564c634bdf00 2024-03-15T13:00:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f50437fe640 10 client.16143 handle_command_reply: tid=0 2024-03-15T13:00:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f506d5b6740 1 client.16143 shutdown 2024-03-15T13:00:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f506d5b6740 1 -- 172.21.15.2:0/407814910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c63448bd0 msgr2=0x564c634591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f506d5b6740 1 --2- 172.21.15.2:0/407814910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c63448bd0 0x564c634591e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x564c6344dea0 tx=0x7f50540054c0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f506d5b6740 1 -- 172.21.15.2:0/407814910 shutdown_connections 2024-03-15T13:00:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f506d5b6740 1 --2- 172.21.15.2:0/407814910 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564c634bdf00 0x564c634c0320 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f50437fe640 0 client.16143 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f506d5b6740 1 --2- 172.21.15.2:0/407814910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c634517b0 0x564c63455170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f506d5b6740 1 --2- 172.21.15.2:0/407814910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c63448bd0 0x564c634591e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f506d5b6740 1 --2- 172.21.15.2:0/407814910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c634482b0 0x564c63458ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f506d5b6740 1 -- 172.21.15.2:0/407814910 >> 172.21.15.2:0/407814910 conn(0x564c63442990 msgr2=0x564c63443b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f506d5b6740 1 -- 172.21.15.2:0/407814910 shutdown_connections 2024-03-15T13:00:21.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.859+0000 7f506d5b6740 1 -- 172.21.15.2:0/407814910 wait complete. 2024-03-15T13:00:21.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.860+0000 7f506d5b6740 20 client.16143 trim_cache size 0 max 16384 2024-03-15T13:00:21.852 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:00:21.852 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:00:21.852 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:00:21.852 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:00:21.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.860+0000 7f506a9e4640 1 -- 172.21.15.2:0/2309679685 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f50500a1cc0 msgr2=0x7f50500a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:21.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.860+0000 7f506a9e4640 1 --2- 172.21.15.2:0/2309679685 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f50500a1cc0 0x7f50500a4180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f505c037040 tx=0x7f505c034440 comp rx=0 tx=0).stop 2024-03-15T13:00:21.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.860+0000 7f506a9e4640 1 -- 172.21.15.2:0/2309679685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50640ffc60 msgr2=0x7f506419d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:21.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.860+0000 7f506a9e4640 1 --2- 172.21.15.2:0/2309679685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50640ffc60 0x7f506419d6d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f504c00e9f0 tx=0x7f504c00eeb0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.860+0000 7f506a9e4640 1 -- 172.21.15.2:0/2309679685 shutdown_connections 2024-03-15T13:00:21.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.860+0000 7f506a9e4640 1 --2- 172.21.15.2:0/2309679685 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f50500a1cc0 0x7f50500a4180 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.860+0000 7f506a9e4640 1 --2- 172.21.15.2:0/2309679685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50641978d0 0x7f5064197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.860+0000 7f506a9e4640 1 --2- 172.21.15.2:0/2309679685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5064106890 0x7f506419dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.860+0000 7f506a9e4640 1 --2- 172.21.15.2:0/2309679685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50640ffc60 0x7f506419d6d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.860+0000 7f506a9e4640 1 -- 172.21.15.2:0/2309679685 >> 172.21.15.2:0/2309679685 conn(0x7f5064076910 msgr2=0x7f50641049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:21.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.860+0000 7f506a9e4640 1 -- 172.21.15.2:0/2309679685 shutdown_connections 2024-03-15T13:00:21.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.860+0000 7f506a9e4640 1 -- 172.21.15.2:0/2309679685 wait complete. 2024-03-15T13:00:21.857 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:00:21.857 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:00:21.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.974+0000 7fbaff1f4640 1 -- 172.21.15.2:0/1542540565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf8102720 msgr2=0x7fbaf8102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:21.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.974+0000 7fbaff1f4640 1 --2- 172.21.15.2:0/1542540565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf8102720 0x7fbaf8102b00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009900 tx=0x7fbaec02bcd0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.974+0000 7fbaff1f4640 1 -- 172.21.15.2:0/1542540565 shutdown_connections 2024-03-15T13:00:21.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.974+0000 7fbaff1f4640 1 --2- 172.21.15.2:0/1542540565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaf810baf0 0x7fbaf810df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.974+0000 7fbaff1f4640 1 --2- 172.21.15.2:0/1542540565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaf8103040 0x7fbaf810b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.974+0000 7fbaff1f4640 1 --2- 172.21.15.2:0/1542540565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf8102720 0x7fbaf8102b00 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.974+0000 7fbaff1f4640 1 -- 172.21.15.2:0/1542540565 >> 172.21.15.2:0/1542540565 conn(0x7fbaf8076770 msgr2=0x7fbaf8076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:21.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.974+0000 7fbaff1f4640 1 -- 172.21.15.2:0/1542540565 shutdown_connections 2024-03-15T13:00:21.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.974+0000 7fbaff1f4640 1 -- 172.21.15.2:0/1542540565 wait complete. 2024-03-15T13:00:21.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.974+0000 7fbaff1f4640 1 Processor -- start 2024-03-15T13:00:21.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.974+0000 7fbaff1f4640 1 -- start start 2024-03-15T13:00:21.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbaff1f4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf8102720 0x7fbaf819d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbaff1f4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaf8103040 0x7fbaf819dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbaff1f4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaf810baf0 0x7fbaf8197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbaff1f4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf8197f80 con 0x7fbaf8102720 2024-03-15T13:00:21.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbaff1f4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf81980f0 con 0x7fbaf8103040 2024-03-15T13:00:21.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbaff1f4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf81982b0 con 0x7fbaf810baf0 2024-03-15T13:00:21.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbafcf90640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf8102720 0x7fbaf819d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbafd791640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaf810baf0 0x7fbaf8197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbafcf90640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf8102720 0x7fbaf819d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53760/0 (socket says 172.21.15.2:53760) 2024-03-15T13:00:21.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbafcf90640 1 -- 172.21.15.2:0/224461327 learned_addr learned my addr 172.21.15.2:0/224461327 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:21.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbaf7fff640 1 --2- 172.21.15.2:0/224461327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaf8103040 0x7fbaf819dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbafcf90640 1 -- 172.21.15.2:0/224461327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaf810baf0 msgr2=0x7fbaf8197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:21.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbafcf90640 1 --2- 172.21.15.2:0/224461327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaf810baf0 0x7fbaf8197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbafcf90640 1 -- 172.21.15.2:0/224461327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaf8103040 msgr2=0x7fbaf819dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:21.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbafcf90640 1 --2- 172.21.15.2:0/224461327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaf8103040 0x7fbaf819dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:21.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbafcf90640 1 -- 172.21.15.2:0/224461327 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaec009560 con 0x7fbaf8102720 2024-03-15T13:00:21.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbafd791640 1 --2- 172.21.15.2:0/224461327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaf810baf0 0x7fbaf8197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:21.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbaf7fff640 1 --2- 172.21.15.2:0/224461327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaf8103040 0x7fbaf819dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:21.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbafcf90640 1 --2- 172.21.15.2:0/224461327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf8102720 0x7fbaf819d780 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009230 tx=0x7fbaec02cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:21.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbaf5ffb640 1 -- 172.21.15.2:0/224461327 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec004250 con 0x7fbaf8102720 2024-03-15T13:00:21.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbaff1f4640 1 -- 172.21.15.2:0/224461327 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf8198530 con 0x7fbaf8102720 2024-03-15T13:00:21.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbaf5ffb640 1 -- 172.21.15.2:0/224461327 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbaec0043b0 con 0x7fbaf8102720 2024-03-15T13:00:21.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.975+0000 7fbaf5ffb640 1 -- 172.21.15.2:0/224461327 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec003bf0 con 0x7fbaf8102720 2024-03-15T13:00:21.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.976+0000 7fbaff1f4640 1 -- 172.21.15.2:0/224461327 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf81a4820 con 0x7fbaf8102720 2024-03-15T13:00:21.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.977+0000 7fbaff1f4640 1 -- 172.21.15.2:0/224461327 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf806a340 con 0x7fbaf8102720 2024-03-15T13:00:21.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.978+0000 7fbaf5ffb640 1 -- 172.21.15.2:0/224461327 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7fbaec003d50 con 0x7fbaf8102720 2024-03-15T13:00:21.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.978+0000 7fbaf5ffb640 1 --2- 172.21.15.2:0/224461327 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbae80a1cc0 0x7fbae80a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:21.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.979+0000 7fbaf7fff640 1 --2- 172.21.15.2:0/224461327 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbae80a1cc0 0x7fbae80a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:21.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.979+0000 7fbaf5ffb640 1 -- 172.21.15.2:0/224461327 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fbaec0f03b0 con 0x7fbaf8102720 2024-03-15T13:00:21.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.979+0000 7fbaf7fff640 1 --2- 172.21.15.2:0/224461327 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbae80a1cc0 0x7fbae80a4180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbaf810d870 tx=0x7fbae4005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:21.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:21.983+0000 7fbaf5ffb640 1 -- 172.21.15.2:0/224461327 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbaec0b8eb0 con 0x7fbaf8102720 2024-03-15T13:00:22.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:21 smithi002 ceph-mon[32060]: pgmap v46: 129 pgs: 129 active+clean; 83 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.8 MiB/s wr, 50 op/s 2024-03-15T13:00:22.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:21 smithi184 ceph-mon[35993]: pgmap v46: 129 pgs: 129 active+clean; 83 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.8 MiB/s wr, 50 op/s 2024-03-15T13:00:22.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:21 smithi149 ceph-mon[35927]: pgmap v46: 129 pgs: 129 active+clean; 83 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.8 MiB/s wr, 50 op/s 2024-03-15T13:00:22.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.204+0000 7fbaff1f4640 1 -- 172.21.15.2:0/224461327 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbaf8068730 con 0x7fbaf8102720 2024-03-15T13:00:22.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.205+0000 7fbaf5ffb640 1 -- 172.21.15.2:0/224461327 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11453 (secure 0 0 0) 0x7fbaec043090 con 0x7fbaf8102720 2024-03-15T13:00:22.190 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:00:22.191 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T12:58:37.814324+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24434,24539,24460,24464,24500]},"id":1}]} 2024-03-15T13:00:22.191 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 29 2024-03-15T13:00:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.207+0000 7fbaff1f4640 1 -- 172.21.15.2:0/224461327 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbae80a1cc0 msgr2=0x7fbae80a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.207+0000 7fbaff1f4640 1 --2- 172.21.15.2:0/224461327 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbae80a1cc0 0x7fbae80a4180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbaf810d870 tx=0x7fbae4005f90 comp rx=0 tx=0).stop 2024-03-15T13:00:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.207+0000 7fbaff1f4640 1 -- 172.21.15.2:0/224461327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf8102720 msgr2=0x7fbaf819d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.207+0000 7fbaff1f4640 1 --2- 172.21.15.2:0/224461327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf8102720 0x7fbaf819d780 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009230 tx=0x7fbaec02cb60 comp rx=0 tx=0).stop 2024-03-15T13:00:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.207+0000 7fbaff1f4640 1 -- 172.21.15.2:0/224461327 shutdown_connections 2024-03-15T13:00:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.207+0000 7fbaff1f4640 1 --2- 172.21.15.2:0/224461327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbaf810baf0 0x7fbaf8197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.207+0000 7fbaff1f4640 1 --2- 172.21.15.2:0/224461327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaf8103040 0x7fbaf819dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.207+0000 7fbaff1f4640 1 --2- 172.21.15.2:0/224461327 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbae80a1cc0 0x7fbae80a4180 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.207+0000 7fbaff1f4640 1 --2- 172.21.15.2:0/224461327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf8102720 0x7fbaf819d780 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.207+0000 7fbaff1f4640 1 -- 172.21.15.2:0/224461327 >> 172.21.15.2:0/224461327 conn(0x7fbaf8076770 msgr2=0x7fbaf80fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.207+0000 7fbaff1f4640 1 -- 172.21.15.2:0/224461327 shutdown_connections 2024-03-15T13:00:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.207+0000 7fbaff1f4640 1 -- 172.21.15.2:0/224461327 wait complete. 2024-03-15T13:00:22.203 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:00:22.203 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:00:22.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.323+0000 7fed73581640 1 -- 172.21.15.2:0/4012541294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed6c0ffa60 msgr2=0x7fed6c0ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.323+0000 7fed73581640 1 --2- 172.21.15.2:0/4012541294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed6c0ffa60 0x7fed6c0ffee0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009900 tx=0x7fed5c02bc30 comp rx=0 tx=0).stop 2024-03-15T13:00:22.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.323+0000 7fed73581640 1 -- 172.21.15.2:0/4012541294 shutdown_connections 2024-03-15T13:00:22.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.323+0000 7fed73581640 1 --2- 172.21.15.2:0/4012541294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed6c100420 0x7fed6c10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.323+0000 7fed73581640 1 --2- 172.21.15.2:0/4012541294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed6c0ffa60 0x7fed6c0ffee0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.323+0000 7fed73581640 1 --2- 172.21.15.2:0/4012541294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed6c1075f0 0x7fed6c1079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.323+0000 7fed73581640 1 -- 172.21.15.2:0/4012541294 >> 172.21.15.2:0/4012541294 conn(0x7fed6c076a20 msgr2=0x7fed6c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.323+0000 7fed73581640 1 -- 172.21.15.2:0/4012541294 shutdown_connections 2024-03-15T13:00:22.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.323+0000 7fed73581640 1 -- 172.21.15.2:0/4012541294 wait complete. 2024-03-15T13:00:22.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.323+0000 7fed73581640 1 Processor -- start 2024-03-15T13:00:22.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed73581640 1 -- start start 2024-03-15T13:00:22.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed73581640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed6c0ffa60 0x7fed6c100c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed73581640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed6c100420 0x7fed6c101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed73581640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed6c1075f0 0x7fed6c1016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed73581640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c1a5eb0 con 0x7fed6c0ffa60 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed73581640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c1a6020 con 0x7fed6c1075f0 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed73581640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c1a61e0 con 0x7fed6c100420 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed7131d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed6c0ffa60 0x7fed6c100c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed7131d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed6c0ffa60 0x7fed6c100c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53782/0 (socket says 172.21.15.2:53782) 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed7131d640 1 -- 172.21.15.2:0/395266681 learned_addr learned my addr 172.21.15.2:0/395266681 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed71b1e640 1 --2- 172.21.15.2:0/395266681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed6c1075f0 0x7fed6c1016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed70b1c640 1 --2- 172.21.15.2:0/395266681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed6c100420 0x7fed6c101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed7131d640 1 -- 172.21.15.2:0/395266681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed6c100420 msgr2=0x7fed6c101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed7131d640 1 --2- 172.21.15.2:0/395266681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed6c100420 0x7fed6c101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed7131d640 1 -- 172.21.15.2:0/395266681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed6c1075f0 msgr2=0x7fed6c1016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed7131d640 1 --2- 172.21.15.2:0/395266681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed6c1075f0 0x7fed6c1016c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed7131d640 1 -- 172.21.15.2:0/395266681 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c009560 con 0x7fed6c0ffa60 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed7131d640 1 --2- 172.21.15.2:0/395266681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed6c0ffa60 0x7fed6c100c40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fed6800e9f0 tx=0x7fed6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed627fc640 1 -- 172.21.15.2:0/395266681 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68009700 con 0x7fed6c0ffa60 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.324+0000 7fed73581640 1 -- 172.21.15.2:0/395266681 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6c1a64c0 con 0x7fed6c0ffa60 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.325+0000 7fed627fc640 1 -- 172.21.15.2:0/395266681 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fed6800cd80 con 0x7fed6c0ffa60 2024-03-15T13:00:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.325+0000 7fed627fc640 1 -- 172.21.15.2:0/395266681 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68018920 con 0x7fed6c0ffa60 2024-03-15T13:00:22.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.325+0000 7fed73581640 1 -- 172.21.15.2:0/395266681 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c1a69f0 con 0x7fed6c0ffa60 2024-03-15T13:00:22.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.327+0000 7fed627fc640 1 -- 172.21.15.2:0/395266681 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7fed68018a80 con 0x7fed6c0ffa60 2024-03-15T13:00:22.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.327+0000 7fed75dfd740 1 Processor -- start 2024-03-15T13:00:22.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.327+0000 7fed75dfd740 1 -- start start 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.327+0000 7fed75dfd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05250 0x55dd2be05630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.327+0000 7fed75dfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be0e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.327+0000 7fed75dfd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be0e750 0x55dd2be10b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.327+0000 7fed75dfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2bdc0960 con 0x55dd2be05b70 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.327+0000 7fed75dfd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2bd74310 con 0x55dd2be05250 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.327+0000 7fed75dfd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2bd71840 con 0x55dd2be0e750 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed627fc640 1 --2- 172.21.15.2:0/395266681 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fed580a1cf0 0x7fed580a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed7131d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be0e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed71b1e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05250 0x55dd2be05630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed7131d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be0e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53792/0 (socket says 172.21.15.2:53792) 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed7131d640 1 -- 172.21.15.2:0/1803724206 learned_addr learned my addr 172.21.15.2:0/1803724206 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed70b1c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be0e750 0x55dd2be10b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed71b1e640 1 --2- 172.21.15.2:0/395266681 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fed580a1cf0 0x7fed580a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed627fc640 1 -- 172.21.15.2:0/395266681 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fed68014070 con 0x7fed6c0ffa60 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed7131d640 1 -- 172.21.15.2:0/1803724206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be0e750 msgr2=0x55dd2be10b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed7131d640 1 --2- 172.21.15.2:0/1803724206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be0e750 0x55dd2be10b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed7131d640 1 -- 172.21.15.2:0/1803724206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05250 msgr2=0x55dd2be05630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed7131d640 1 --2- 172.21.15.2:0/1803724206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05250 0x55dd2be05630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed7131d640 1 -- 172.21.15.2:0/1803724206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd2bd2f060 con 0x55dd2be05b70 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed71b1e640 1 --2- 172.21.15.2:0/1803724206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05250 0x55dd2be05630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed7131d640 1 --2- 172.21.15.2:0/1803724206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be0e210 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fed6809a4c0 tx=0x7fed6809a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a244c15896d2e0b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed71b1e640 1 --2- 172.21.15.2:0/395266681 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fed580a1cf0 0x7fed580a41b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fed64005c40 tx=0x7fed64009500 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed577fe640 1 -- 172.21.15.2:0/1803724206 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6809a7c0 con 0x55dd2be05b70 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed577fe640 1 -- 172.21.15.2:0/1803724206 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fed6809e860 con 0x55dd2be05b70 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.328+0000 7fed577fe640 1 -- 172.21.15.2:0/1803724206 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed680ba330 con 0x55dd2be05b70 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.329+0000 7fed75dfd740 1 -- 172.21.15.2:0/1803724206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 msgr2=0x55dd2be0e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.329+0000 7fed75dfd740 1 --2- 172.21.15.2:0/1803724206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be0e210 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fed6809a4c0 tx=0x7fed6809a300 comp rx=0 tx=0).stop 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.329+0000 7fed75dfd740 1 -- 172.21.15.2:0/1803724206 shutdown_connections 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.329+0000 7fed75dfd740 1 --2- 172.21.15.2:0/1803724206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be0e750 0x55dd2be10b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.329+0000 7fed75dfd740 1 --2- 172.21.15.2:0/1803724206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be0e210 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.329+0000 7fed75dfd740 1 --2- 172.21.15.2:0/1803724206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05250 0x55dd2be05630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.329+0000 7fed75dfd740 1 -- 172.21.15.2:0/1803724206 >> 172.21.15.2:0/1803724206 conn(0x55dd2bdff980 msgr2=0x55dd2bdffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.329+0000 7fed75dfd740 1 -- 172.21.15.2:0/1803724206 shutdown_connections 2024-03-15T13:00:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.329+0000 7fed75dfd740 1 -- 172.21.15.2:0/1803724206 wait complete. 2024-03-15T13:00:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.329+0000 7fed75dfd740 1 -- start start 2024-03-15T13:00:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed75dfd740 1 Processor -- start 2024-03-15T13:00:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed75dfd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05250 0x55dd2be0e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed75dfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be0b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed75dfd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be0e750 0x55dd2be0baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed75dfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2bd6d020 con 0x55dd2be05b70 2024-03-15T13:00:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed75dfd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2ba40430 con 0x55dd2be05250 2024-03-15T13:00:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed75dfd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2b9fce00 con 0x55dd2be0e750 2024-03-15T13:00:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed75dfd740 10 client.0 _ll_register_callbacks cb 0x55dd2bd14cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed70b1c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be0b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed7131d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05250 0x55dd2be0e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed70b1c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be0b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53804/0 (socket says 172.21.15.2:53804) 2024-03-15T13:00:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed70b1c640 1 -- 172.21.15.2:0/675168518 learned_addr learned my addr 172.21.15.2:0/675168518 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed71b1e640 1 --2- 172.21.15.2:0/675168518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be0e750 0x55dd2be0baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed70b1c640 1 -- 172.21.15.2:0/675168518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be0e750 msgr2=0x55dd2be0baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed70b1c640 1 --2- 172.21.15.2:0/675168518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be0e750 0x55dd2be0baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed70b1c640 1 -- 172.21.15.2:0/675168518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05250 msgr2=0x55dd2be0e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed70b1c640 1 --2- 172.21.15.2:0/675168518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05250 0x55dd2be0e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.330+0000 7fed70b1c640 1 -- 172.21.15.2:0/675168518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c1a69f0 con 0x55dd2be05b70 2024-03-15T13:00:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed70b1c640 1 --2- 172.21.15.2:0/675168518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be0b4e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fed5c036040 tx=0x7fed5c02c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ab0229598d4a35 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed54ff9640 1 -- 172.21.15.2:0/675168518 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c004040 con 0x55dd2be05b70 2024-03-15T13:00:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed54ff9640 1 -- 172.21.15.2:0/675168518 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fed5c0041a0 con 0x55dd2be05b70 2024-03-15T13:00:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed75dfd740 1 -- 172.21.15.2:0/675168518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd2ba6c7e0 con 0x55dd2be05b70 2024-03-15T13:00:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed54ff9640 1 -- 172.21.15.2:0/675168518 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed5c004360 con 0x55dd2be05b70 2024-03-15T13:00:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed75dfd740 10 client.16161 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed75dfd740 1 -- 172.21.15.2:0/675168518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dd2bd6b6a0 con 0x55dd2be05b70 2024-03-15T13:00:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed54ff9640 1 -- 172.21.15.2:0/675168518 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55dd2ba6c7e0 con 0x55dd2be05b70 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed54ff9640 1 -- 172.21.15.2:0/675168518 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fed5c03c040 con 0x55dd2be05b70 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed75dfd740 10 client.16161 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed75dfd740 10 client.16161 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed75dfd740 20 client.16161 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed75dfd740 1 --2- 172.21.15.2:0/675168518 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55dd2be1e8f0 0x55dd2be20d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed75dfd740 4 client.16161 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed75dfd740 1 -- 172.21.15.2:0/675168518 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd2baa9720 con 0x55dd2be1e8f0 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.331+0000 7fed7131d640 1 --2- 172.21.15.2:0/675168518 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55dd2be1e8f0 0x55dd2be20d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.332+0000 7fed54ff9640 10 client.16161 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.332+0000 7fed7131d640 1 --2- 172.21.15.2:0/675168518 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55dd2be1e8f0 0x55dd2be20d10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=23eb27ddb76086ab server_cookie=ad1b69a710ffbe07 in_seq=0 out_seq=0 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.332+0000 7fed54ff9640 1 -- 172.21.15.2:0/675168518 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dd2baa9720 con 0x55dd2be1e8f0 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.332+0000 7fed54ff9640 10 client.16161 handle_command_reply: tid=0 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.332+0000 7fed75dfd740 1 client.16161 shutdown 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.333+0000 7fed75dfd740 1 -- 172.21.15.2:0/675168518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 msgr2=0x55dd2be0b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.333+0000 7fed75dfd740 1 --2- 172.21.15.2:0/675168518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be0b4e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fed5c036040 tx=0x7fed5c02c770 comp rx=0 tx=0).stop 2024-03-15T13:00:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.333+0000 7fed75dfd740 1 -- 172.21.15.2:0/675168518 shutdown_connections 2024-03-15T13:00:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.333+0000 7fed75dfd740 1 --2- 172.21.15.2:0/675168518 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55dd2be1e8f0 0x55dd2be20d10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.333+0000 7fed75dfd740 1 --2- 172.21.15.2:0/675168518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be0e750 0x55dd2be0baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.333+0000 7fed75dfd740 1 --2- 172.21.15.2:0/675168518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be0b4e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.333+0000 7fed75dfd740 1 --2- 172.21.15.2:0/675168518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05250 0x55dd2be0e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.333+0000 7fed54ff9640 0 client.16161 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:00:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.333+0000 7fed75dfd740 1 -- 172.21.15.2:0/675168518 >> 172.21.15.2:0/675168518 conn(0x55dd2bdff980 msgr2=0x55dd2be016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.333+0000 7fed75dfd740 1 -- 172.21.15.2:0/675168518 shutdown_connections 2024-03-15T13:00:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.333+0000 7fed75dfd740 1 -- 172.21.15.2:0/675168518 wait complete. 2024-03-15T13:00:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.333+0000 7fed75dfd740 20 client.16161 trim_cache size 0 max 16384 2024-03-15T13:00:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.345+0000 7fed75dfd740 1 Processor -- start 2024-03-15T13:00:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.345+0000 7fed75dfd740 1 -- start start 2024-03-15T13:00:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.345+0000 7fed75dfd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be05250 0x55dd2be0dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.345+0000 7fed75dfd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05b70 0x55dd2be0e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.345+0000 7fed75dfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be0e750 0x55dd2be19ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.345+0000 7fed75dfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2bcf2130 con 0x55dd2be0e750 2024-03-15T13:00:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.345+0000 7fed75dfd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2be00bb0 con 0x55dd2be05b70 2024-03-15T13:00:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.345+0000 7fed75dfd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2bdfb5c0 con 0x55dd2be05250 2024-03-15T13:00:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.345+0000 7fed71b1e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be0e750 0x55dd2be19ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.346+0000 7fed71b1e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be0e750 0x55dd2be19ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53810/0 (socket says 172.21.15.2:53810) 2024-03-15T13:00:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.346+0000 7fed71b1e640 1 -- 172.21.15.2:0/2293775490 learned_addr learned my addr 172.21.15.2:0/2293775490 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.346+0000 7fed70b1c640 1 --2- 172.21.15.2:0/2293775490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05b70 0x55dd2be0e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.346+0000 7fed7131d640 1 --2- 172.21.15.2:0/2293775490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be05250 0x55dd2be0dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.346+0000 7fed71b1e640 1 -- 172.21.15.2:0/2293775490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be05250 msgr2=0x55dd2be0dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.346+0000 7fed71b1e640 1 --2- 172.21.15.2:0/2293775490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be05250 0x55dd2be0dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.346+0000 7fed71b1e640 1 -- 172.21.15.2:0/2293775490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05b70 msgr2=0x55dd2be0e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.346+0000 7fed71b1e640 1 --2- 172.21.15.2:0/2293775490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05b70 0x55dd2be0e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.346+0000 7fed71b1e640 1 -- 172.21.15.2:0/2293775490 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd2bd07c20 con 0x55dd2be0e750 2024-03-15T13:00:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.346+0000 7fed71b1e640 1 --2- 172.21.15.2:0/2293775490 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be0e750 0x55dd2be19ad0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fed64011f10 tx=0x7fed64008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abe683baa633f9e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.346+0000 7fed56ffd640 1 -- 172.21.15.2:0/2293775490 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64025040 con 0x55dd2be0e750 2024-03-15T13:00:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.346+0000 7fed56ffd640 1 -- 172.21.15.2:0/2293775490 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fed6400fcc0 con 0x55dd2be0e750 2024-03-15T13:00:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.346+0000 7fed56ffd640 1 -- 172.21.15.2:0/2293775490 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64005630 con 0x55dd2be0e750 2024-03-15T13:00:22.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 -- 172.21.15.2:0/2293775490 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be0e750 msgr2=0x55dd2be19ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 --2- 172.21.15.2:0/2293775490 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be0e750 0x55dd2be19ad0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fed64011f10 tx=0x7fed64008000 comp rx=0 tx=0).stop 2024-03-15T13:00:22.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 -- 172.21.15.2:0/2293775490 shutdown_connections 2024-03-15T13:00:22.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 --2- 172.21.15.2:0/2293775490 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be0e750 0x55dd2be19ad0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 --2- 172.21.15.2:0/2293775490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be05b70 0x55dd2be0e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 --2- 172.21.15.2:0/2293775490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be05250 0x55dd2be0dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 -- 172.21.15.2:0/2293775490 >> 172.21.15.2:0/2293775490 conn(0x55dd2bdff980 msgr2=0x55dd2be01640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 -- 172.21.15.2:0/2293775490 shutdown_connections 2024-03-15T13:00:22.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 -- 172.21.15.2:0/2293775490 wait complete. 2024-03-15T13:00:22.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 -- start start 2024-03-15T13:00:22.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 Processor -- start 2024-03-15T13:00:22.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be05250 0x55dd2be16050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be0e750 0x55dd2be12910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2be16920 con 0x55dd2be05b70 2024-03-15T13:00:22.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2be12e50 con 0x55dd2be0e750 2024-03-15T13:00:22.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2be13010 con 0x55dd2be05250 2024-03-15T13:00:22.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.347+0000 7fed75dfd740 10 client.0 _ll_register_callbacks cb 0x7fed5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:22.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed70b1c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed70b1c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be123d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53822/0 (socket says 172.21.15.2:53822) 2024-03-15T13:00:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed70b1c640 1 -- 172.21.15.2:0/1927805203 learned_addr learned my addr 172.21.15.2:0/1927805203 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed71b1e640 1 --2- 172.21.15.2:0/1927805203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be0e750 0x55dd2be12910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed7131d640 1 --2- 172.21.15.2:0/1927805203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be05250 0x55dd2be16050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed70b1c640 1 -- 172.21.15.2:0/1927805203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be05250 msgr2=0x55dd2be16050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed70b1c640 1 --2- 172.21.15.2:0/1927805203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be05250 0x55dd2be16050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed70b1c640 1 -- 172.21.15.2:0/1927805203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be0e750 msgr2=0x55dd2be12910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed70b1c640 1 --2- 172.21.15.2:0/1927805203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be0e750 0x55dd2be12910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed70b1c640 1 -- 172.21.15.2:0/1927805203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed64019040 con 0x55dd2be05b70 2024-03-15T13:00:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed70b1c640 1 --2- 172.21.15.2:0/1927805203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be123d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x55dd2be0ad40 tx=0x7fed5c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c765b5deb357e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed54ff9640 1 -- 172.21.15.2:0/1927805203 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c004450 con 0x55dd2be05b70 2024-03-15T13:00:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed75dfd740 1 -- 172.21.15.2:0/1927805203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed6809e860 con 0x55dd2be05b70 2024-03-15T13:00:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed54ff9640 1 -- 172.21.15.2:0/1927805203 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fed5c004de0 con 0x55dd2be05b70 2024-03-15T13:00:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed54ff9640 1 -- 172.21.15.2:0/1927805203 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed5c005090 con 0x55dd2be05b70 2024-03-15T13:00:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed75dfd740 10 client.16167 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed75dfd740 1 -- 172.21.15.2:0/1927805203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dd2bd2d8c0 con 0x55dd2be05b70 2024-03-15T13:00:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.348+0000 7fed54ff9640 1 -- 172.21.15.2:0/1927805203 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fed6809e860 con 0x55dd2be05b70 2024-03-15T13:00:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.349+0000 7fed54ff9640 1 -- 172.21.15.2:0/1927805203 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fed5c03c040 con 0x55dd2be05b70 2024-03-15T13:00:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.349+0000 7fed75dfd740 10 client.16167 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.349+0000 7fed75dfd740 10 client.16167 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:00:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.349+0000 7fed75dfd740 20 client.16167 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.349+0000 7fed75dfd740 1 --2- 172.21.15.2:0/1927805203 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55dd2be7b410 0x55dd2be7d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.349+0000 7fed75dfd740 4 client.16167 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:00:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.349+0000 7fed75dfd740 1 -- 172.21.15.2:0/1927805203 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dd2baf8890 con 0x55dd2be7b410 2024-03-15T13:00:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.349+0000 7fed7131d640 1 --2- 172.21.15.2:0/1927805203 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55dd2be7b410 0x55dd2be7d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.349+0000 7fed54ff9640 10 client.16167 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:00:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.349+0000 7fed7131d640 1 --2- 172.21.15.2:0/1927805203 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55dd2be7b410 0x55dd2be7d830 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c277052f9e122631 server_cookie=13456c2718698db1 in_seq=0 out_seq=0 2024-03-15T13:00:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.349+0000 7fed54ff9640 1 -- 172.21.15.2:0/1927805203 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dd2baf8890 con 0x55dd2be7b410 2024-03-15T13:00:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.349+0000 7fed54ff9640 10 client.16167 handle_command_reply: tid=0 2024-03-15T13:00:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.349+0000 7fed75dfd740 1 client.16167 shutdown 2024-03-15T13:00:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.349+0000 7fed75dfd740 1 -- 172.21.15.2:0/1927805203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 msgr2=0x55dd2be123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.349+0000 7fed75dfd740 1 --2- 172.21.15.2:0/1927805203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be123d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x55dd2be0ad40 tx=0x7fed5c005720 comp rx=0 tx=0).stop 2024-03-15T13:00:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed75dfd740 1 -- 172.21.15.2:0/1927805203 shutdown_connections 2024-03-15T13:00:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed75dfd740 1 --2- 172.21.15.2:0/1927805203 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55dd2be7b410 0x55dd2be7d830 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed75dfd740 1 --2- 172.21.15.2:0/1927805203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd2be0e750 0x55dd2be12910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed75dfd740 1 --2- 172.21.15.2:0/1927805203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dd2be05b70 0x55dd2be123d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed54ff9640 0 client.16167 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:00:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed75dfd740 1 --2- 172.21.15.2:0/1927805203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dd2be05250 0x55dd2be16050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed75dfd740 1 -- 172.21.15.2:0/1927805203 >> 172.21.15.2:0/1927805203 conn(0x55dd2bdff980 msgr2=0x55dd2bdfe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed75dfd740 1 -- 172.21.15.2:0/1927805203 shutdown_connections 2024-03-15T13:00:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed75dfd740 1 -- 172.21.15.2:0/1927805203 wait complete. 2024-03-15T13:00:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed75dfd740 20 client.16167 trim_cache size 0 max 16384 2024-03-15T13:00:22.342 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:00:22.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed73581640 1 -- 172.21.15.2:0/395266681 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fed580a1cf0 msgr2=0x7fed580a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed73581640 1 --2- 172.21.15.2:0/395266681 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fed580a1cf0 0x7fed580a41b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fed64005c40 tx=0x7fed64009500 comp rx=0 tx=0).stop 2024-03-15T13:00:22.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed73581640 1 -- 172.21.15.2:0/395266681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed6c0ffa60 msgr2=0x7fed6c100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed73581640 1 --2- 172.21.15.2:0/395266681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed6c0ffa60 0x7fed6c100c40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fed6800e9f0 tx=0x7fed6800eeb0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed73581640 1 -- 172.21.15.2:0/395266681 shutdown_connections 2024-03-15T13:00:22.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed73581640 1 --2- 172.21.15.2:0/395266681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed6c1075f0 0x7fed6c1016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed73581640 1 --2- 172.21.15.2:0/395266681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed6c100420 0x7fed6c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed73581640 1 --2- 172.21.15.2:0/395266681 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fed580a1cf0 0x7fed580a41b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed73581640 1 --2- 172.21.15.2:0/395266681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed6c0ffa60 0x7fed6c100c40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed73581640 1 -- 172.21.15.2:0/395266681 >> 172.21.15.2:0/395266681 conn(0x7fed6c076a20 msgr2=0x7fed6c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed73581640 1 -- 172.21.15.2:0/395266681 shutdown_connections 2024-03-15T13:00:22.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.350+0000 7fed73581640 1 -- 172.21.15.2:0/395266681 wait complete. 2024-03-15T13:00:22.346 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:00:22.346 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:00:22.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.465+0000 7fa8dbf55640 1 -- 172.21.15.2:0/1687651193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d41056a0 msgr2=0x7fa8d410d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.465+0000 7fa8dbf55640 1 --2- 172.21.15.2:0/1687651193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d41056a0 0x7fa8d410d270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009be0 tx=0x7fa8cc02bef0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.465+0000 7fa8dbf55640 1 -- 172.21.15.2:0/1687651193 shutdown_connections 2024-03-15T13:00:22.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.465+0000 7fa8dbf55640 1 --2- 172.21.15.2:0/1687651193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d41056a0 0x7fa8d410d270 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.465+0000 7fa8dbf55640 1 --2- 172.21.15.2:0/1687651193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8d4100f80 0x7fa8d4104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.465+0000 7fa8dbf55640 1 --2- 172.21.15.2:0/1687651193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8d41005d0 0x7fa8d41009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.465+0000 7fa8dbf55640 1 -- 172.21.15.2:0/1687651193 >> 172.21.15.2:0/1687651193 conn(0x7fa8d40774f0 msgr2=0x7fa8d4077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.465+0000 7fa8dbf55640 1 -- 172.21.15.2:0/1687651193 shutdown_connections 2024-03-15T13:00:22.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.465+0000 7fa8dbf55640 1 -- 172.21.15.2:0/1687651193 wait complete. 2024-03-15T13:00:22.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.465+0000 7fa8dbf55640 1 Processor -- start 2024-03-15T13:00:22.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.465+0000 7fa8dbf55640 1 -- start start 2024-03-15T13:00:22.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8dbf55640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8d41005d0 0x7fa8d41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8dbf55640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d41056a0 0x7fa8d4198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8dbf55640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8d41993b0 0x7fa8d41a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8dbf55640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d41a4030 con 0x7fa8d41005d0 2024-03-15T13:00:22.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8dbf55640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d41a41a0 con 0x7fa8d41993b0 2024-03-15T13:00:22.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8dbf55640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d41a4360 con 0x7fa8d41056a0 2024-03-15T13:00:22.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8d9cf1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8d41005d0 0x7fa8d41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8d9cf1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8d41005d0 0x7fa8d41988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53846/0 (socket says 172.21.15.2:53846) 2024-03-15T13:00:22.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8d9cf1640 1 -- 172.21.15.2:0/1945545615 learned_addr learned my addr 172.21.15.2:0/1945545615 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8d94f0640 1 --2- 172.21.15.2:0/1945545615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d41056a0 0x7fa8d4198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8da4f2640 1 --2- 172.21.15.2:0/1945545615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8d41993b0 0x7fa8d41a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8d9cf1640 1 -- 172.21.15.2:0/1945545615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d41056a0 msgr2=0x7fa8d4198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8d9cf1640 1 --2- 172.21.15.2:0/1945545615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d41056a0 0x7fa8d4198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8d9cf1640 1 -- 172.21.15.2:0/1945545615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8d41993b0 msgr2=0x7fa8d41a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8d9cf1640 1 --2- 172.21.15.2:0/1945545615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8d41993b0 0x7fa8d41a3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8d9cf1640 1 -- 172.21.15.2:0/1945545615 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d0009560 con 0x7fa8d41005d0 2024-03-15T13:00:22.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8d9cf1640 1 --2- 172.21.15.2:0/1945545615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8d41005d0 0x7fa8d41988e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000e9c0 tx=0x7fa8d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8caffd640 1 -- 172.21.15.2:0/1945545615 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d000cde0 con 0x7fa8d41005d0 2024-03-15T13:00:22.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8dbf55640 1 -- 172.21.15.2:0/1945545615 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8cc009840 con 0x7fa8d41005d0 2024-03-15T13:00:22.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8caffd640 1 -- 172.21.15.2:0/1945545615 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa8d00035f0 con 0x7fa8d41005d0 2024-03-15T13:00:22.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8caffd640 1 -- 172.21.15.2:0/1945545615 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0010670 con 0x7fa8d41005d0 2024-03-15T13:00:22.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.466+0000 7fa8dbf55640 1 -- 172.21.15.2:0/1945545615 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d41a4a10 con 0x7fa8d41005d0 2024-03-15T13:00:22.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.468+0000 7fa8caffd640 1 -- 172.21.15.2:0/1945545615 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7fa8d0003760 con 0x7fa8d41005d0 2024-03-15T13:00:22.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8dec05740 1 Processor -- start 2024-03-15T13:00:22.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8dec05740 1 -- start start 2024-03-15T13:00:22.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8caffd640 1 --2- 172.21.15.2:0/1945545615 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa8c00a1cc0 0x7fa8c00a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8dec05740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c491250 0x557e6c491630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8dec05740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491b70 0x557e6c49a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8dec05740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c49a750 0x557e6c49cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8dec05740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c44c960 con 0x557e6c49a750 2024-03-15T13:00:22.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8dec05740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c400310 con 0x557e6c491250 2024-03-15T13:00:22.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8dec05740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c3fd840 con 0x557e6c491b70 2024-03-15T13:00:22.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8caffd640 1 -- 172.21.15.2:0/1945545615 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa8d0014070 con 0x7fa8d41005d0 2024-03-15T13:00:22.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8da4f2640 1 --2- 172.21.15.2:0/1945545615 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa8c00a1cc0 0x7fa8c00a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8d9cf1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c491250 0x557e6c491630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8d94f0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491b70 0x557e6c49a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8d9cf1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c491250 0x557e6c491630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39886/0 (socket says 172.21.15.2:39886) 2024-03-15T13:00:22.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8d9cf1640 1 -- 172.21.15.2:0/2552935772 learned_addr learned my addr 172.21.15.2:0/2552935772 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.469+0000 7fa8da4f2640 1 --2- 172.21.15.2:0/2552935772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c49a750 0x557e6c49cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.470+0000 7fa8d94f0640 1 -- 172.21.15.2:0/2552935772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c491250 msgr2=0x557e6c491630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.470+0000 7fa8d94f0640 1 --2- 172.21.15.2:0/2552935772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c491250 0x557e6c491630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.470+0000 7fa8d94f0640 1 -- 172.21.15.2:0/2552935772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c49a750 msgr2=0x557e6c49cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.470+0000 7fa8d94f0640 1 --2- 172.21.15.2:0/2552935772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c49a750 0x557e6c49cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.470+0000 7fa8d94f0640 1 -- 172.21.15.2:0/2552935772 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6c3bb060 con 0x557e6c491b70 2024-03-15T13:00:22.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.470+0000 7fa8d9cf1640 1 --2- 172.21.15.2:0/2552935772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c491250 0x557e6c491630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:22.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.470+0000 7fa8da4f2640 1 --2- 172.21.15.2:0/1945545615 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa8c00a1cc0 0x7fa8c00a4180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009370 tx=0x7fa8cc02ca40 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.470+0000 7fa8d94f0640 1 --2- 172.21.15.2:0/2552935772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491b70 0x557e6c49a210 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa8d406ac50 tx=0x7fa8c4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eca33bb7eec2300e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.470+0000 7fa8bffff640 1 -- 172.21.15.2:0/2552935772 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4004050 con 0x557e6c491b70 2024-03-15T13:00:22.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.470+0000 7fa8bffff640 1 -- 172.21.15.2:0/2552935772 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa8c4005020 con 0x557e6c491b70 2024-03-15T13:00:22.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.470+0000 7fa8bffff640 1 -- 172.21.15.2:0/2552935772 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4004b10 con 0x557e6c491b70 2024-03-15T13:00:22.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.471+0000 7fa8dec05740 1 -- 172.21.15.2:0/2552935772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491b70 msgr2=0x557e6c49a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.471+0000 7fa8dec05740 1 --2- 172.21.15.2:0/2552935772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491b70 0x557e6c49a210 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa8d406ac50 tx=0x7fa8c4011810 comp rx=0 tx=0).stop 2024-03-15T13:00:22.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.471+0000 7fa8dec05740 1 -- 172.21.15.2:0/2552935772 shutdown_connections 2024-03-15T13:00:22.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.471+0000 7fa8dec05740 1 --2- 172.21.15.2:0/2552935772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c49a750 0x557e6c49cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.471+0000 7fa8dec05740 1 --2- 172.21.15.2:0/2552935772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491b70 0x557e6c49a210 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.471+0000 7fa8dec05740 1 --2- 172.21.15.2:0/2552935772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c491250 0x557e6c491630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.471+0000 7fa8dec05740 1 -- 172.21.15.2:0/2552935772 >> 172.21.15.2:0/2552935772 conn(0x557e6c48b980 msgr2=0x557e6c48bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.471+0000 7fa8dec05740 1 -- 172.21.15.2:0/2552935772 shutdown_connections 2024-03-15T13:00:22.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.471+0000 7fa8dec05740 1 -- 172.21.15.2:0/2552935772 wait complete. 2024-03-15T13:00:22.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.471+0000 7fa8dec05740 1 -- start start 2024-03-15T13:00:22.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.471+0000 7fa8dec05740 1 Processor -- start 2024-03-15T13:00:22.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8dec05740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 0x557e6c49a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8dec05740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c491b70 0x557e6c4974a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8dec05740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c49a750 0x557e6c497ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8dec05740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c3f9020 con 0x557e6c491b70 2024-03-15T13:00:22.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8dec05740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c0cc430 con 0x557e6c49a750 2024-03-15T13:00:22.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8dec05740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c088e00 con 0x557e6c491250 2024-03-15T13:00:22.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8dec05740 10 client.0 _ll_register_callbacks cb 0x557e6c3a0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:22.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8d94f0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c491b70 0x557e6c4974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8d9cf1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 0x557e6c49a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8d94f0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c491b70 0x557e6c4974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53864/0 (socket says 172.21.15.2:53864) 2024-03-15T13:00:22.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8da4f2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c49a750 0x557e6c497ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8d94f0640 1 -- 172.21.15.2:0/2697343902 learned_addr learned my addr 172.21.15.2:0/2697343902 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8d94f0640 1 -- 172.21.15.2:0/2697343902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 msgr2=0x557e6c49a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8d94f0640 1 --2- 172.21.15.2:0/2697343902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 0x557e6c49a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8d94f0640 1 -- 172.21.15.2:0/2697343902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c49a750 msgr2=0x557e6c497ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8d94f0640 1 --2- 172.21.15.2:0/2697343902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c49a750 0x557e6c497ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8d94f0640 1 -- 172.21.15.2:0/2697343902 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c4009850 con 0x557e6c491b70 2024-03-15T13:00:22.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8d9cf1640 1 --2- 172.21.15.2:0/2697343902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 0x557e6c49a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:22.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8da4f2640 1 --2- 172.21.15.2:0/2697343902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c49a750 0x557e6c497ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8d94f0640 1 --2- 172.21.15.2:0/2697343902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c491b70 0x557e6c4974a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40057c0 tx=0x7fa8c4009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a76c36e4c83365 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8bd7fa640 1 -- 172.21.15.2:0/2697343902 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4004460 con 0x557e6c491b70 2024-03-15T13:00:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8bd7fa640 1 -- 172.21.15.2:0/2697343902 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa8c40048b0 con 0x557e6c491b70 2024-03-15T13:00:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.472+0000 7fa8dec05740 1 -- 172.21.15.2:0/2697343902 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e6c0f87e0 con 0x557e6c491b70 2024-03-15T13:00:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.473+0000 7fa8bd7fa640 1 -- 172.21.15.2:0/2697343902 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8c4004ab0 con 0x557e6c491b70 2024-03-15T13:00:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.473+0000 7fa8dec05740 10 client.16179 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.473+0000 7fa8dec05740 1 -- 172.21.15.2:0/2697343902 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557e6c3f76a0 con 0x557e6c491b70 2024-03-15T13:00:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.473+0000 7fa8bd7fa640 1 -- 172.21.15.2:0/2697343902 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x557e6c0f87e0 con 0x557e6c491b70 2024-03-15T13:00:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.473+0000 7fa8bd7fa640 1 -- 172.21.15.2:0/2697343902 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa8c4021040 con 0x557e6c491b70 2024-03-15T13:00:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.473+0000 7fa8dec05740 10 client.16179 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.473+0000 7fa8dec05740 10 client.16179 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:00:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.473+0000 7fa8dec05740 20 client.16179 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.473+0000 7fa8dec05740 1 --2- 172.21.15.2:0/2697343902 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557e6c4aa790 0x557e6c4acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.473+0000 7fa8dec05740 4 client.16179 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:22.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.473+0000 7fa8dec05740 1 -- 172.21.15.2:0/2697343902 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e6c135720 con 0x557e6c4aa790 2024-03-15T13:00:22.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.473+0000 7fa8d9cf1640 1 --2- 172.21.15.2:0/2697343902 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557e6c4aa790 0x557e6c4acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.474+0000 7fa8d9cf1640 1 --2- 172.21.15.2:0/2697343902 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557e6c4aa790 0x557e6c4acbb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f53924e220681d00 server_cookie=c5a123e615f66db1 in_seq=0 out_seq=0 2024-03-15T13:00:22.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.474+0000 7fa8bd7fa640 10 client.16179 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:00:22.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.474+0000 7fa8bd7fa640 1 -- 172.21.15.2:0/2697343902 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557e6c135720 con 0x557e6c4aa790 2024-03-15T13:00:22.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.474+0000 7fa8bd7fa640 10 client.16179 handle_command_reply: tid=0 2024-03-15T13:00:22.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.474+0000 7fa8dec05740 1 client.16179 shutdown 2024-03-15T13:00:22.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.474+0000 7fa8dec05740 1 -- 172.21.15.2:0/2697343902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c491b70 msgr2=0x557e6c4974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.474+0000 7fa8dec05740 1 --2- 172.21.15.2:0/2697343902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c491b70 0x557e6c4974a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40057c0 tx=0x7fa8c4009fa0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.474+0000 7fa8dec05740 1 -- 172.21.15.2:0/2697343902 shutdown_connections 2024-03-15T13:00:22.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.474+0000 7fa8dec05740 1 --2- 172.21.15.2:0/2697343902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c49a750 0x557e6c497ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.474+0000 7fa8dec05740 1 --2- 172.21.15.2:0/2697343902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c491b70 0x557e6c4974a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.474+0000 7fa8dec05740 1 --2- 172.21.15.2:0/2697343902 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557e6c4aa790 0x557e6c4acbb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.474+0000 7fa8dec05740 1 --2- 172.21.15.2:0/2697343902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 0x557e6c49a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.474+0000 7fa8bd7fa640 0 client.16179 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:00:22.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.474+0000 7fa8dec05740 1 -- 172.21.15.2:0/2697343902 >> 172.21.15.2:0/2697343902 conn(0x557e6c48b980 msgr2=0x557e6c48d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.475+0000 7fa8dec05740 1 -- 172.21.15.2:0/2697343902 shutdown_connections 2024-03-15T13:00:22.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.475+0000 7fa8dec05740 1 -- 172.21.15.2:0/2697343902 wait complete. 2024-03-15T13:00:22.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.475+0000 7fa8dec05740 20 client.16179 trim_cache size 0 max 16384 2024-03-15T13:00:22.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8dec05740 1 Processor -- start 2024-03-15T13:00:22.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8dec05740 1 -- start start 2024-03-15T13:00:22.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8dec05740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 0x557e6c48aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8dec05740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c491b70 0x557e6c499bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8dec05740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c49a750 0x557e6c49a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8dec05740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c37e130 con 0x557e6c49a750 2024-03-15T13:00:22.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8dec05740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c490eb0 con 0x557e6c491b70 2024-03-15T13:00:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8dec05740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c490340 con 0x557e6c491250 2024-03-15T13:00:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8da4f2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c49a750 0x557e6c49a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8da4f2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c49a750 0x557e6c49a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53866/0 (socket says 172.21.15.2:53866) 2024-03-15T13:00:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8d9cf1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 0x557e6c48aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8da4f2640 1 -- 172.21.15.2:0/2769674619 learned_addr learned my addr 172.21.15.2:0/2769674619 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8d94f0640 1 --2- 172.21.15.2:0/2769674619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c491b70 0x557e6c499bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8da4f2640 1 -- 172.21.15.2:0/2769674619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 msgr2=0x557e6c48aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8da4f2640 1 --2- 172.21.15.2:0/2769674619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 0x557e6c48aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8da4f2640 1 -- 172.21.15.2:0/2769674619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c491b70 msgr2=0x557e6c499bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8da4f2640 1 --2- 172.21.15.2:0/2769674619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c491b70 0x557e6c499bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.487+0000 7fa8da4f2640 1 -- 172.21.15.2:0/2769674619 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6c393c20 con 0x557e6c49a750 2024-03-15T13:00:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.488+0000 7fa8d9cf1640 1 --2- 172.21.15.2:0/2769674619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 0x557e6c48aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.488+0000 7fa8da4f2640 1 --2- 172.21.15.2:0/2769674619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c49a750 0x557e6c49a110 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009be0 tx=0x7fa8cc04b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5f6364ad728bb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.488+0000 7fa8bf7fe640 1 -- 172.21.15.2:0/2769674619 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc048070 con 0x557e6c49a750 2024-03-15T13:00:22.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.488+0000 7fa8bf7fe640 1 -- 172.21.15.2:0/2769674619 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa8cc0045d0 con 0x557e6c49a750 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.488+0000 7fa8bf7fe640 1 -- 172.21.15.2:0/2769674619 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc0349a0 con 0x557e6c49a750 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.488+0000 7fa8dec05740 1 -- 172.21.15.2:0/2769674619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c49a750 msgr2=0x557e6c49a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.488+0000 7fa8dec05740 1 --2- 172.21.15.2:0/2769674619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c49a750 0x557e6c49a110 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009be0 tx=0x7fa8cc04b000 comp rx=0 tx=0).stop 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.488+0000 7fa8dec05740 1 -- 172.21.15.2:0/2769674619 shutdown_connections 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.488+0000 7fa8dec05740 1 --2- 172.21.15.2:0/2769674619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c49a750 0x557e6c49a110 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.488+0000 7fa8dec05740 1 --2- 172.21.15.2:0/2769674619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c491b70 0x557e6c499bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.488+0000 7fa8dec05740 1 --2- 172.21.15.2:0/2769674619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 0x557e6c48aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.488+0000 7fa8dec05740 1 -- 172.21.15.2:0/2769674619 >> 172.21.15.2:0/2769674619 conn(0x557e6c48b980 msgr2=0x557e6c48a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8dec05740 1 -- 172.21.15.2:0/2769674619 shutdown_connections 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8dec05740 1 -- 172.21.15.2:0/2769674619 wait complete. 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8dec05740 1 -- start start 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8dec05740 1 Processor -- start 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8dec05740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 0x557e6c493d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8dec05740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c491b70 0x557e6c494250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8dec05740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c49a750 0x557e6c494790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8dec05740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c4955c0 con 0x557e6c491b70 2024-03-15T13:00:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8dec05740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c505040 con 0x557e6c49a750 2024-03-15T13:00:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8dec05740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6c5051d0 con 0x557e6c491250 2024-03-15T13:00:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8dec05740 10 client.0 _ll_register_callbacks cb 0x7fa8c4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8d94f0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c491b70 0x557e6c494250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8d94f0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c491b70 0x557e6c494250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53876/0 (socket says 172.21.15.2:53876) 2024-03-15T13:00:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8d94f0640 1 -- 172.21.15.2:0/1236228259 learned_addr learned my addr 172.21.15.2:0/1236228259 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8d9cf1640 1 --2- 172.21.15.2:0/1236228259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 0x557e6c493d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.489+0000 7fa8da4f2640 1 --2- 172.21.15.2:0/1236228259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c49a750 0x557e6c494790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8d94f0640 1 -- 172.21.15.2:0/1236228259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 msgr2=0x557e6c493d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8d94f0640 1 --2- 172.21.15.2:0/1236228259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 0x557e6c493d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8d94f0640 1 -- 172.21.15.2:0/1236228259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c49a750 msgr2=0x557e6c494790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8d94f0640 1 --2- 172.21.15.2:0/1236228259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c49a750 0x557e6c494790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8d94f0640 1 -- 172.21.15.2:0/1236228259 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d00b7040 con 0x557e6c491b70 2024-03-15T13:00:22.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8d94f0640 1 --2- 172.21.15.2:0/1236228259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c491b70 0x557e6c494250 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x557e6c496d00 tx=0x7fa8c4005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89054696cea7c318 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8bd7fa640 1 -- 172.21.15.2:0/1236228259 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4009a70 con 0x557e6c491b70 2024-03-15T13:00:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8dec05740 1 -- 172.21.15.2:0/1236228259 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8c4005020 con 0x557e6c491b70 2024-03-15T13:00:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8bd7fa640 1 -- 172.21.15.2:0/1236228259 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa8c4000e40 con 0x557e6c491b70 2024-03-15T13:00:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8bd7fa640 1 -- 172.21.15.2:0/1236228259 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8c4011d40 con 0x557e6c491b70 2024-03-15T13:00:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8dec05740 10 client.16185 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8dec05740 1 -- 172.21.15.2:0/1236228259 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fa8d410da80 con 0x557e6c491b70 2024-03-15T13:00:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8bd7fa640 1 -- 172.21.15.2:0/1236228259 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa8c4005020 con 0x557e6c491b70 2024-03-15T13:00:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8bd7fa640 1 -- 172.21.15.2:0/1236228259 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa8c402dcb0 con 0x557e6c491b70 2024-03-15T13:00:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8dec05740 10 client.16185 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8dec05740 10 client.16185 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:00:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.490+0000 7fa8dec05740 20 client.16185 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8dec05740 1 --2- 172.21.15.2:0/1236228259 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557e6c506720 0x557e6c508b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8dec05740 4 client.16185 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:00:22.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8dec05740 1 -- 172.21.15.2:0/1236228259 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557e6c184890 con 0x557e6c506720 2024-03-15T13:00:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8d9cf1640 1 --2- 172.21.15.2:0/1236228259 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557e6c506720 0x557e6c508b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8bd7fa640 10 client.16185 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:00:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8d9cf1640 1 --2- 172.21.15.2:0/1236228259 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557e6c506720 0x557e6c508b40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=214787bdb923dab6 server_cookie=ac0ac8603fa9c969 in_seq=0 out_seq=0 2024-03-15T13:00:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8bd7fa640 1 -- 172.21.15.2:0/1236228259 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557e6c184890 con 0x557e6c506720 2024-03-15T13:00:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8bd7fa640 10 client.16185 handle_command_reply: tid=0 2024-03-15T13:00:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8dec05740 1 client.16185 shutdown 2024-03-15T13:00:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8dec05740 1 -- 172.21.15.2:0/1236228259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c491b70 msgr2=0x557e6c494250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8dec05740 1 --2- 172.21.15.2:0/1236228259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c491b70 0x557e6c494250 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x557e6c496d00 tx=0x7fa8c4005f80 comp rx=0 tx=0).stop 2024-03-15T13:00:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8dec05740 1 -- 172.21.15.2:0/1236228259 shutdown_connections 2024-03-15T13:00:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8dec05740 1 --2- 172.21.15.2:0/1236228259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e6c49a750 0x557e6c494790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8dec05740 1 --2- 172.21.15.2:0/1236228259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557e6c491b70 0x557e6c494250 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8dec05740 1 --2- 172.21.15.2:0/1236228259 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x557e6c506720 0x557e6c508b40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8dec05740 1 --2- 172.21.15.2:0/1236228259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6c491250 0x557e6c493d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8bd7fa640 0 client.16185 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:00:22.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.491+0000 7fa8dec05740 1 -- 172.21.15.2:0/1236228259 >> 172.21.15.2:0/1236228259 conn(0x557e6c48b980 msgr2=0x557e6c48ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.492+0000 7fa8dec05740 1 -- 172.21.15.2:0/1236228259 shutdown_connections 2024-03-15T13:00:22.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.492+0000 7fa8dec05740 1 -- 172.21.15.2:0/1236228259 wait complete. 2024-03-15T13:00:22.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.492+0000 7fa8dec05740 20 client.16185 trim_cache size 0 max 16384 2024-03-15T13:00:22.484 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:00:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.492+0000 7fa8dbf55640 1 -- 172.21.15.2:0/1945545615 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa8c00a1cc0 msgr2=0x7fa8c00a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.492+0000 7fa8dbf55640 1 --2- 172.21.15.2:0/1945545615 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa8c00a1cc0 0x7fa8c00a4180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009370 tx=0x7fa8cc02ca40 comp rx=0 tx=0).stop 2024-03-15T13:00:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.492+0000 7fa8dbf55640 1 -- 172.21.15.2:0/1945545615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8d41005d0 msgr2=0x7fa8d41988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.492+0000 7fa8dbf55640 1 --2- 172.21.15.2:0/1945545615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8d41005d0 0x7fa8d41988e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000e9c0 tx=0x7fa8d000ee80 comp rx=0 tx=0).stop 2024-03-15T13:00:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.492+0000 7fa8dbf55640 1 -- 172.21.15.2:0/1945545615 shutdown_connections 2024-03-15T13:00:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.492+0000 7fa8dbf55640 1 --2- 172.21.15.2:0/1945545615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8d41993b0 0x7fa8d41a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.492+0000 7fa8dbf55640 1 --2- 172.21.15.2:0/1945545615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8d41056a0 0x7fa8d4198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.492+0000 7fa8dbf55640 1 --2- 172.21.15.2:0/1945545615 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa8c00a1cc0 0x7fa8c00a4180 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.492+0000 7fa8dbf55640 1 --2- 172.21.15.2:0/1945545615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8d41005d0 0x7fa8d41988e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.492+0000 7fa8dbf55640 1 -- 172.21.15.2:0/1945545615 >> 172.21.15.2:0/1945545615 conn(0x7fa8d40774f0 msgr2=0x7fa8d40fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.492+0000 7fa8dbf55640 1 -- 172.21.15.2:0/1945545615 shutdown_connections 2024-03-15T13:00:22.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.492+0000 7fa8dbf55640 1 -- 172.21.15.2:0/1945545615 wait complete. 2024-03-15T13:00:22.488 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:00:22.488 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:00:22.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.607+0000 7f963920e640 1 -- 172.21.15.2:0/594775602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9634100610 msgr2=0x7f96341009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.607+0000 7f963920e640 1 --2- 172.21.15.2:0/594775602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9634100610 0x7f96341009f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9624009930 tx=0x7f962402bc30 comp rx=0 tx=0).stop 2024-03-15T13:00:22.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.607+0000 7f963920e640 1 -- 172.21.15.2:0/594775602 shutdown_connections 2024-03-15T13:00:22.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.607+0000 7f963920e640 1 --2- 172.21.15.2:0/594775602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96341056e0 0x7f963410d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.607+0000 7f963920e640 1 --2- 172.21.15.2:0/594775602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9634100fc0 0x7f9634105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.607+0000 7f963920e640 1 --2- 172.21.15.2:0/594775602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9634100610 0x7f96341009f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.607+0000 7f963920e640 1 -- 172.21.15.2:0/594775602 >> 172.21.15.2:0/594775602 conn(0x7f9634076770 msgr2=0x7f9634076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.607+0000 7f963920e640 1 -- 172.21.15.2:0/594775602 shutdown_connections 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.607+0000 7f963920e640 1 -- 172.21.15.2:0/594775602 wait complete. 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f963920e640 1 Processor -- start 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f963920e640 1 -- start start 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f963920e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9634100610 0x7f963410db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f963920e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96341056e0 0x7f963410e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f963920e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f963410e610 0x7f96341a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f963920e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96341a62d0 con 0x7f96341056e0 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f963920e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96341a6440 con 0x7f963410e610 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f963920e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96341a6600 con 0x7f9634100610 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f962bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96341056e0 0x7f963410e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f9632d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9634100610 0x7f963410db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f9632d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9634100610 0x7f963410db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:36932/0 (socket says 172.21.15.2:36932) 2024-03-15T13:00:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f963359e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f963410e610 0x7f96341a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f9632d9d640 1 -- 172.21.15.2:0/153107650 learned_addr learned my addr 172.21.15.2:0/153107650 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f962bfff640 1 -- 172.21.15.2:0/153107650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9634100610 msgr2=0x7f963410db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f962bfff640 1 --2- 172.21.15.2:0/153107650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9634100610 0x7f963410db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f962bfff640 1 -- 172.21.15.2:0/153107650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f963410e610 msgr2=0x7f96341a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.608+0000 7f962bfff640 1 --2- 172.21.15.2:0/153107650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f963410e610 0x7f96341a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.609+0000 7f962bfff640 1 -- 172.21.15.2:0/153107650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c009840 con 0x7f96341056e0 2024-03-15T13:00:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.609+0000 7f962bfff640 1 --2- 172.21.15.2:0/153107650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96341056e0 0x7f963410e0d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f961c00e9c0 tx=0x7f961c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.609+0000 7f9630d99640 1 -- 172.21.15.2:0/153107650 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c009640 con 0x7f96341056e0 2024-03-15T13:00:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.609+0000 7f9630d99640 1 -- 172.21.15.2:0/153107650 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f961c00ccc0 con 0x7f96341056e0 2024-03-15T13:00:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.609+0000 7f963920e640 1 -- 172.21.15.2:0/153107650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9624009560 con 0x7f96341056e0 2024-03-15T13:00:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.609+0000 7f9630d99640 1 -- 172.21.15.2:0/153107650 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c018750 con 0x7f96341056e0 2024-03-15T13:00:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.609+0000 7f963920e640 1 -- 172.21.15.2:0/153107650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96341a6c60 con 0x7f96341056e0 2024-03-15T13:00:22.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.611+0000 7f9630d99640 1 -- 172.21.15.2:0/153107650 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7f961c010a20 con 0x7f96341056e0 2024-03-15T13:00:22.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.611+0000 7f963bded740 1 Processor -- start 2024-03-15T13:00:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.611+0000 7f9630d99640 1 --2- 172.21.15.2:0/153107650 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f96200a1cf0 0x7f96200a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.611+0000 7f963bded740 1 -- start start 2024-03-15T13:00:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f963359e640 1 --2- 172.21.15.2:0/153107650 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f96200a1cf0 0x7f96200a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f9630d99640 1 -- 172.21.15.2:0/153107650 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f961c014070 con 0x7f96341056e0 2024-03-15T13:00:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f963bded740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b34250 0x55e246b34630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f963bded740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34b70 0x55e246b3d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f963bded740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b3d750 0x55e246b3fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f963bded740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e246aef960 con 0x55e246b3d750 2024-03-15T13:00:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f963bded740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e246aa3310 con 0x55e246b34250 2024-03-15T13:00:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f963bded740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e246aa0840 con 0x55e246b34b70 2024-03-15T13:00:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f9632d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b34250 0x55e246b34630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f9632d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b34250 0x55e246b34630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39930/0 (socket says 172.21.15.2:39930) 2024-03-15T13:00:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f9632d9d640 1 -- 172.21.15.2:0/1527309079 learned_addr learned my addr 172.21.15.2:0/1527309079 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f962bfff640 1 --2- 172.21.15.2:0/1527309079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34b70 0x55e246b3d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f963359e640 1 --2- 172.21.15.2:0/1527309079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b3d750 0x55e246b3fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f963359e640 1 --2- 172.21.15.2:0/153107650 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f96200a1cf0 0x7f96200a41b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f962c005fd0 tx=0x7f962c011920 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f9632d9d640 1 -- 172.21.15.2:0/1527309079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34b70 msgr2=0x55e246b3d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f9632d9d640 1 --2- 172.21.15.2:0/1527309079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34b70 0x55e246b3d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f9632d9d640 1 -- 172.21.15.2:0/1527309079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b3d750 msgr2=0x55e246b3fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f9632d9d640 1 --2- 172.21.15.2:0/1527309079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b3d750 0x55e246b3fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f9632d9d640 1 -- 172.21.15.2:0/1527309079 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e246a5e060 con 0x55e246b34250 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f962bfff640 1 --2- 172.21.15.2:0/1527309079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34b70 0x55e246b3d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.612+0000 7f9632d9d640 1 --2- 172.21.15.2:0/1527309079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b34250 0x55e246b34630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f962402c710 tx=0x7f962402c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=513c68b7650d8578 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.613+0000 7f96297fa640 1 -- 172.21.15.2:0/1527309079 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96240047c0 con 0x55e246b34250 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.613+0000 7f96297fa640 1 -- 172.21.15.2:0/1527309079 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9624004920 con 0x55e246b34250 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.613+0000 7f96297fa640 1 -- 172.21.15.2:0/1527309079 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624003960 con 0x55e246b34250 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.613+0000 7f963bded740 1 -- 172.21.15.2:0/1527309079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b34250 msgr2=0x55e246b34630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.613+0000 7f963bded740 1 --2- 172.21.15.2:0/1527309079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b34250 0x55e246b34630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f962402c710 tx=0x7f962402c740 comp rx=0 tx=0).stop 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.613+0000 7f963bded740 1 -- 172.21.15.2:0/1527309079 shutdown_connections 2024-03-15T13:00:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.613+0000 7f963bded740 1 --2- 172.21.15.2:0/1527309079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b3d750 0x55e246b3fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.613+0000 7f963bded740 1 --2- 172.21.15.2:0/1527309079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34b70 0x55e246b3d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.613+0000 7f963bded740 1 --2- 172.21.15.2:0/1527309079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b34250 0x55e246b34630 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.613+0000 7f963bded740 1 -- 172.21.15.2:0/1527309079 >> 172.21.15.2:0/1527309079 conn(0x55e246b2e980 msgr2=0x55e246b2ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.613+0000 7f963bded740 1 -- 172.21.15.2:0/1527309079 shutdown_connections 2024-03-15T13:00:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f963bded740 1 -- 172.21.15.2:0/1527309079 wait complete. 2024-03-15T13:00:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f963bded740 1 -- start start 2024-03-15T13:00:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f963bded740 1 Processor -- start 2024-03-15T13:00:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f963bded740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b34250 0x55e246b3d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f963bded740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34b70 0x55e246b3a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f963bded740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b3d750 0x55e246b3a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f963bded740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e246a9c020 con 0x55e246b3d750 2024-03-15T13:00:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f963bded740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24676f430 con 0x55e246b34250 2024-03-15T13:00:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f963bded740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24672be00 con 0x55e246b34b70 2024-03-15T13:00:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f963bded740 10 client.0 _ll_register_callbacks cb 0x55e246a43cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f963359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b3d750 0x55e246b3a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f963359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b3d750 0x55e246b3a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53920/0 (socket says 172.21.15.2:53920) 2024-03-15T13:00:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f9632d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b34250 0x55e246b3d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f963359e640 1 -- 172.21.15.2:0/1946318694 learned_addr learned my addr 172.21.15.2:0/1946318694 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f962bfff640 1 --2- 172.21.15.2:0/1946318694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34b70 0x55e246b3a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f963359e640 1 -- 172.21.15.2:0/1946318694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34b70 msgr2=0x55e246b3a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.614+0000 7f963359e640 1 --2- 172.21.15.2:0/1946318694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34b70 0x55e246b3a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f963359e640 1 -- 172.21.15.2:0/1946318694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b34250 msgr2=0x55e246b3d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f963359e640 1 --2- 172.21.15.2:0/1946318694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b34250 0x55e246b3d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f963359e640 1 -- 172.21.15.2:0/1946318694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9634069ac0 con 0x55e246b3d750 2024-03-15T13:00:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f963359e640 1 --2- 172.21.15.2:0/1946318694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b3d750 0x55e246b3a910 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f962c004550 tx=0x7f962c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428645f2f09c9a6e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f961a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f961a7fc640 1 -- 172.21.15.2:0/1946318694 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c023070 con 0x55e246b3d750 2024-03-15T13:00:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f961a7fc640 1 -- 172.21.15.2:0/1946318694 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f962c00fc10 con 0x55e246b3d750 2024-03-15T13:00:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f963bded740 1 -- 172.21.15.2:0/1946318694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e24679b7e0 con 0x55e246b3d750 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f961a7fc640 1 -- 172.21.15.2:0/1946318694 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f962c00fdd0 con 0x55e246b3d750 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f963bded740 10 client.16197 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f963bded740 1 -- 172.21.15.2:0/1946318694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e246a9a6a0 con 0x55e246b3d750 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f961a7fc640 1 -- 172.21.15.2:0/1946318694 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e24679b7e0 con 0x55e246b3d750 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f961a7fc640 1 -- 172.21.15.2:0/1946318694 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f962c00e440 con 0x55e246b3d750 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f963bded740 10 client.16197 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f963bded740 10 client.16197 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f963bded740 20 client.16197 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f963bded740 1 --2- 172.21.15.2:0/1946318694 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e246b4d860 0x55e246b4fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f963bded740 4 client.16197 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.615+0000 7f963bded740 1 -- 172.21.15.2:0/1946318694 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2467d8720 con 0x55e246b4d860 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.616+0000 7f9632d9d640 1 --2- 172.21.15.2:0/1946318694 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e246b4d860 0x55e246b4fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.616+0000 7f9632d9d640 1 --2- 172.21.15.2:0/1946318694 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e246b4d860 0x55e246b4fc80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f882587c08f8d8e3 server_cookie=c7f8039c378d3cb3 in_seq=0 out_seq=0 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.616+0000 7f961a7fc640 10 client.16197 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.617+0000 7f961a7fc640 1 -- 172.21.15.2:0/1946318694 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e2467d8720 con 0x55e246b4d860 2024-03-15T13:00:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.617+0000 7f961a7fc640 10 client.16197 handle_command_reply: tid=0 2024-03-15T13:00:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.617+0000 7f963bded740 1 client.16197 shutdown 2024-03-15T13:00:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.617+0000 7f963bded740 1 -- 172.21.15.2:0/1946318694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b3d750 msgr2=0x55e246b3a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.617+0000 7f963bded740 1 --2- 172.21.15.2:0/1946318694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b3d750 0x55e246b3a910 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f962c004550 tx=0x7f962c004ab0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.617+0000 7f963bded740 1 -- 172.21.15.2:0/1946318694 shutdown_connections 2024-03-15T13:00:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.617+0000 7f963bded740 1 --2- 172.21.15.2:0/1946318694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b3d750 0x55e246b3a910 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.617+0000 7f963bded740 1 --2- 172.21.15.2:0/1946318694 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e246b4d860 0x55e246b4fc80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.617+0000 7f963bded740 1 --2- 172.21.15.2:0/1946318694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34b70 0x55e246b3a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.617+0000 7f963bded740 1 --2- 172.21.15.2:0/1946318694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b34250 0x55e246b3d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.617+0000 7f961a7fc640 0 client.16197 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:00:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.617+0000 7f963bded740 1 -- 172.21.15.2:0/1946318694 >> 172.21.15.2:0/1946318694 conn(0x55e246b2e980 msgr2=0x55e246b306d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.617+0000 7f963bded740 1 -- 172.21.15.2:0/1946318694 shutdown_connections 2024-03-15T13:00:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.617+0000 7f963bded740 1 -- 172.21.15.2:0/1946318694 wait complete. 2024-03-15T13:00:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.617+0000 7f963bded740 20 client.16197 trim_cache size 0 max 16384 2024-03-15T13:00:22.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.629+0000 7f963bded740 1 Processor -- start 2024-03-15T13:00:22.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.629+0000 7f963bded740 1 -- start start 2024-03-15T13:00:22.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f963bded740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34250 0x55e246b3cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f963bded740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b34b70 0x55e246b3d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f963bded740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b3d750 0x55e246b48a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f963bded740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e246a21130 con 0x55e246b34b70 2024-03-15T13:00:22.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f963bded740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e246b33eb0 con 0x55e246b3d750 2024-03-15T13:00:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f963bded740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e246b2a0a0 con 0x55e246b34250 2024-03-15T13:00:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f9632d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34250 0x55e246b3cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f962bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b34b70 0x55e246b3d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f963359e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b3d750 0x55e246b48a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f9632d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34250 0x55e246b3cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:36962/0 (socket says 172.21.15.2:36962) 2024-03-15T13:00:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f9632d9d640 1 -- 172.21.15.2:0/779269505 learned_addr learned my addr 172.21.15.2:0/779269505 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f9632d9d640 1 -- 172.21.15.2:0/779269505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b3d750 msgr2=0x55e246b48a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f9632d9d640 1 --2- 172.21.15.2:0/779269505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b3d750 0x55e246b48a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f9632d9d640 1 -- 172.21.15.2:0/779269505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b34b70 msgr2=0x55e246b3d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f9632d9d640 1 --2- 172.21.15.2:0/779269505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b34b70 0x55e246b3d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f9632d9d640 1 -- 172.21.15.2:0/779269505 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e246a36c20 con 0x55e246b34250 2024-03-15T13:00:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f963359e640 1 --2- 172.21.15.2:0/779269505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b3d750 0x55e246b48a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f962bfff640 1 --2- 172.21.15.2:0/779269505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b34b70 0x55e246b3d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f9632d9d640 1 --2- 172.21.15.2:0/779269505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34250 0x55e246b3cb00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9624036040 tx=0x7f9624041040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bfc1e0c5a41d5f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.630+0000 7f9628ff9640 1 -- 172.21.15.2:0/779269505 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962402ca90 con 0x55e246b34250 2024-03-15T13:00:22.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.631+0000 7f9628ff9640 1 -- 172.21.15.2:0/779269505 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f962402cbf0 con 0x55e246b34250 2024-03-15T13:00:22.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.631+0000 7f9628ff9640 1 -- 172.21.15.2:0/779269505 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96240050d0 con 0x55e246b34250 2024-03-15T13:00:22.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.631+0000 7f963bded740 1 -- 172.21.15.2:0/779269505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34250 msgr2=0x55e246b3cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.631+0000 7f963bded740 1 --2- 172.21.15.2:0/779269505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34250 0x55e246b3cb00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9624036040 tx=0x7f9624041040 comp rx=0 tx=0).stop 2024-03-15T13:00:22.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.631+0000 7f963bded740 1 -- 172.21.15.2:0/779269505 shutdown_connections 2024-03-15T13:00:22.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.631+0000 7f963bded740 1 --2- 172.21.15.2:0/779269505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b3d750 0x55e246b48a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.631+0000 7f963bded740 1 --2- 172.21.15.2:0/779269505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b34b70 0x55e246b3d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.631+0000 7f963bded740 1 --2- 172.21.15.2:0/779269505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34250 0x55e246b3cb00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.631+0000 7f963bded740 1 -- 172.21.15.2:0/779269505 >> 172.21.15.2:0/779269505 conn(0x55e246b2e980 msgr2=0x55e246b35920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.631+0000 7f963bded740 1 -- 172.21.15.2:0/779269505 shutdown_connections 2024-03-15T13:00:22.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.631+0000 7f963bded740 1 -- 172.21.15.2:0/779269505 wait complete. 2024-03-15T13:00:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.631+0000 7f963bded740 1 -- start start 2024-03-15T13:00:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f963bded740 1 Processor -- start 2024-03-15T13:00:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f963bded740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34250 0x55e246b40cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f963bded740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b34b70 0x55e246b41200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f963bded740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b3d750 0x55e246ba8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f963bded740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e246b419a0 con 0x55e246b34b70 2024-03-15T13:00:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f963bded740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e246b41b10 con 0x55e246b3d750 2024-03-15T13:00:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f963bded740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e246ba8650 con 0x55e246b34250 2024-03-15T13:00:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f963bded740 10 client.0 _ll_register_callbacks cb 0x7f962c003c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f962bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b34b70 0x55e246b41200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f962bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b34b70 0x55e246b41200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53932/0 (socket says 172.21.15.2:53932) 2024-03-15T13:00:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f962bfff640 1 -- 172.21.15.2:0/2438409903 learned_addr learned my addr 172.21.15.2:0/2438409903 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f9632d9d640 1 --2- 172.21.15.2:0/2438409903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34250 0x55e246b40cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f963359e640 1 --2- 172.21.15.2:0/2438409903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b3d750 0x55e246ba8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f962bfff640 1 -- 172.21.15.2:0/2438409903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34250 msgr2=0x55e246b40cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f962bfff640 1 --2- 172.21.15.2:0/2438409903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34250 0x55e246b40cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f962bfff640 1 -- 172.21.15.2:0/2438409903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b3d750 msgr2=0x55e246ba8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f962bfff640 1 --2- 172.21.15.2:0/2438409903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b3d750 0x55e246ba8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f962bfff640 1 -- 172.21.15.2:0/2438409903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9624009560 con 0x55e246b34b70 2024-03-15T13:00:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f962bfff640 1 --2- 172.21.15.2:0/2438409903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b34b70 0x55e246b41200 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f961c0ce120 tx=0x7f961c0ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52245feb5b9a976e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f961a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f961a7fc640 1 -- 172.21.15.2:0/2438409903 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c09e930 con 0x55e246b34b70 2024-03-15T13:00:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f961a7fc640 1 -- 172.21.15.2:0/2438409903 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f961c0a2f80 con 0x55e246b34b70 2024-03-15T13:00:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.632+0000 7f963bded740 1 -- 172.21.15.2:0/2438409903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9624004920 con 0x55e246b34b70 2024-03-15T13:00:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.633+0000 7f961a7fc640 1 -- 172.21.15.2:0/2438409903 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f961c0b3220 con 0x55e246b34b70 2024-03-15T13:00:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.633+0000 7f963bded740 10 client.16203 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.633+0000 7f963bded740 1 -- 172.21.15.2:0/2438409903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e246a9a6a0 con 0x55e246b34b70 2024-03-15T13:00:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.633+0000 7f961a7fc640 1 -- 172.21.15.2:0/2438409903 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9624004920 con 0x55e246b34b70 2024-03-15T13:00:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.633+0000 7f961a7fc640 1 -- 172.21.15.2:0/2438409903 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f961c0850b0 con 0x55e246b34b70 2024-03-15T13:00:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.633+0000 7f963bded740 10 client.16203 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:22.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.633+0000 7f963bded740 10 client.16203 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:00:22.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.633+0000 7f963bded740 20 client.16203 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:22.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.633+0000 7f963bded740 1 --2- 172.21.15.2:0/2438409903 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e246ba9f00 0x55e246bac320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.633+0000 7f963bded740 4 client.16203 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:00:22.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.633+0000 7f963bded740 1 -- 172.21.15.2:0/2438409903 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e246827890 con 0x55e246ba9f00 2024-03-15T13:00:22.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.633+0000 7f9632d9d640 1 --2- 172.21.15.2:0/2438409903 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e246ba9f00 0x55e246bac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f9632d9d640 1 --2- 172.21.15.2:0/2438409903 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e246ba9f00 0x55e246bac320 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a02567316ec2f59 server_cookie=b2db34cadf63e855 in_seq=0 out_seq=0 2024-03-15T13:00:22.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f961a7fc640 10 client.16203 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:00:22.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f961a7fc640 1 -- 172.21.15.2:0/2438409903 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e246827890 con 0x55e246ba9f00 2024-03-15T13:00:22.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f961a7fc640 10 client.16203 handle_command_reply: tid=0 2024-03-15T13:00:22.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963bded740 1 client.16203 shutdown 2024-03-15T13:00:22.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963bded740 1 -- 172.21.15.2:0/2438409903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b34b70 msgr2=0x55e246b41200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963bded740 1 --2- 172.21.15.2:0/2438409903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b34b70 0x55e246b41200 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f961c0ce120 tx=0x7f961c0ba040 comp rx=0 tx=0).stop 2024-03-15T13:00:22.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963bded740 1 -- 172.21.15.2:0/2438409903 shutdown_connections 2024-03-15T13:00:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963bded740 1 --2- 172.21.15.2:0/2438409903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e246b3d750 0x55e246ba8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963bded740 1 --2- 172.21.15.2:0/2438409903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e246b34b70 0x55e246b41200 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963bded740 1 --2- 172.21.15.2:0/2438409903 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e246ba9f00 0x55e246bac320 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963bded740 1 --2- 172.21.15.2:0/2438409903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e246b34250 0x55e246b40cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963bded740 1 -- 172.21.15.2:0/2438409903 >> 172.21.15.2:0/2438409903 conn(0x55e246b2e980 msgr2=0x55e246b39c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f961a7fc640 0 client.16203 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:00:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963bded740 1 -- 172.21.15.2:0/2438409903 shutdown_connections 2024-03-15T13:00:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963bded740 1 -- 172.21.15.2:0/2438409903 wait complete. 2024-03-15T13:00:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963bded740 20 client.16203 trim_cache size 0 max 16384 2024-03-15T13:00:22.626 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:00:22.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963920e640 1 -- 172.21.15.2:0/153107650 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f96200a1cf0 msgr2=0x7f96200a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963920e640 1 --2- 172.21.15.2:0/153107650 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f96200a1cf0 0x7f96200a41b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f962c005fd0 tx=0x7f962c011920 comp rx=0 tx=0).stop 2024-03-15T13:00:22.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963920e640 1 -- 172.21.15.2:0/153107650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96341056e0 msgr2=0x7f963410e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.634+0000 7f963920e640 1 --2- 172.21.15.2:0/153107650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96341056e0 0x7f963410e0d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f961c00e9c0 tx=0x7f961c00edc0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.635+0000 7f963920e640 1 -- 172.21.15.2:0/153107650 shutdown_connections 2024-03-15T13:00:22.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.635+0000 7f963920e640 1 --2- 172.21.15.2:0/153107650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f963410e610 0x7f96341a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.635+0000 7f963920e640 1 --2- 172.21.15.2:0/153107650 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f96200a1cf0 0x7f96200a41b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.635+0000 7f963920e640 1 --2- 172.21.15.2:0/153107650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96341056e0 0x7f963410e0d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.635+0000 7f963920e640 1 --2- 172.21.15.2:0/153107650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9634100610 0x7f963410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.635+0000 7f963920e640 1 -- 172.21.15.2:0/153107650 >> 172.21.15.2:0/153107650 conn(0x7f9634076770 msgr2=0x7f96340fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.635+0000 7f963920e640 1 -- 172.21.15.2:0/153107650 shutdown_connections 2024-03-15T13:00:22.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.635+0000 7f963920e640 1 -- 172.21.15.2:0/153107650 wait complete. 2024-03-15T13:00:22.631 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:00:22.631 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:00:22.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.751+0000 7f2494727640 1 -- 172.21.15.2:0/1979901951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f248c1056a0 msgr2=0x7f248c10d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.751+0000 7f2494727640 1 --2- 172.21.15.2:0/1979901951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f248c1056a0 0x7f248c10d270 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f247c009cd0 tx=0x7f247c02c0f0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.751+0000 7f2494727640 1 -- 172.21.15.2:0/1979901951 shutdown_connections 2024-03-15T13:00:22.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.751+0000 7f2494727640 1 --2- 172.21.15.2:0/1979901951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f248c1056a0 0x7f248c10d270 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.751+0000 7f2494727640 1 --2- 172.21.15.2:0/1979901951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f248c100f80 0x7f248c104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.751+0000 7f2494727640 1 --2- 172.21.15.2:0/1979901951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c1005d0 0x7f248c1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.751+0000 7f2494727640 1 -- 172.21.15.2:0/1979901951 >> 172.21.15.2:0/1979901951 conn(0x7f248c0774f0 msgr2=0x7f248c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.751+0000 7f2494727640 1 -- 172.21.15.2:0/1979901951 shutdown_connections 2024-03-15T13:00:22.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.751+0000 7f2494727640 1 -- 172.21.15.2:0/1979901951 wait complete. 2024-03-15T13:00:22.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.751+0000 7f2494727640 1 Processor -- start 2024-03-15T13:00:22.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.751+0000 7f2494727640 1 -- start start 2024-03-15T13:00:22.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2494727640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c1005d0 0x7f248c19d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2494727640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f248c1056a0 0x7f248c19dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2494727640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f248c1979f0 0x7f248c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2494727640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c1983e0 con 0x7f248c1056a0 2024-03-15T13:00:22.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2494727640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c198550 con 0x7f248c1979f0 2024-03-15T13:00:22.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2494727640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c198710 con 0x7f248c1005d0 2024-03-15T13:00:22.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2491cc2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f248c1056a0 0x7f248c19dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2491cc2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f248c1056a0 0x7f248c19dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53942/0 (socket says 172.21.15.2:53942) 2024-03-15T13:00:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2491cc2640 1 -- 172.21.15.2:0/2241708082 learned_addr learned my addr 172.21.15.2:0/2241708082 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2492cc4640 1 --2- 172.21.15.2:0/2241708082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f248c1979f0 0x7f248c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f24924c3640 1 --2- 172.21.15.2:0/2241708082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c1005d0 0x7f248c19d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2491cc2640 1 -- 172.21.15.2:0/2241708082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c1005d0 msgr2=0x7f248c19d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2491cc2640 1 --2- 172.21.15.2:0/2241708082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c1005d0 0x7f248c19d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2491cc2640 1 -- 172.21.15.2:0/2241708082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f248c1979f0 msgr2=0x7f248c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2491cc2640 1 --2- 172.21.15.2:0/2241708082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f248c1979f0 0x7f248c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2491cc2640 1 -- 172.21.15.2:0/2241708082 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c009900 con 0x7f248c1056a0 2024-03-15T13:00:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2491cc2640 1 --2- 172.21.15.2:0/2241708082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f248c1056a0 0x7f248c19dd80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f248000e9c0 tx=0x7f248000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f248b7fe640 1 -- 172.21.15.2:0/2241708082 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480009700 con 0x7f248c1056a0 2024-03-15T13:00:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2494727640 1 -- 172.21.15.2:0/2241708082 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248c1a4700 con 0x7f248c1056a0 2024-03-15T13:00:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f248b7fe640 1 -- 172.21.15.2:0/2241708082 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f248000cd80 con 0x7f248c1056a0 2024-03-15T13:00:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f248b7fe640 1 -- 172.21.15.2:0/2241708082 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480010430 con 0x7f248c1056a0 2024-03-15T13:00:22.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.752+0000 7f2494727640 1 -- 172.21.15.2:0/2241708082 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248c1a4c60 con 0x7f248c1056a0 2024-03-15T13:00:22.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.754+0000 7f248b7fe640 1 -- 172.21.15.2:0/2241708082 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7f2480004ff0 con 0x7f248c1056a0 2024-03-15T13:00:22.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.755+0000 7f24973d6740 1 Processor -- start 2024-03-15T13:00:22.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.755+0000 7f24973d6740 1 -- start start 2024-03-15T13:00:22.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.755+0000 7f24973d6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 0x5653a5654630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.755+0000 7f24973d6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a5654b70 0x5653a565d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.755+0000 7f24973d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a565d750 0x5653a565fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.755+0000 7f24973d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a560f960 con 0x5653a565d750 2024-03-15T13:00:22.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.755+0000 7f248b7fe640 1 --2- 172.21.15.2:0/2241708082 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f24740a1cc0 0x7f24740a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.755+0000 7f24973d6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a55c3310 con 0x5653a5654250 2024-03-15T13:00:22.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.755+0000 7f24973d6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a55c0840 con 0x5653a5654b70 2024-03-15T13:00:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f2492cc4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 0x5653a5654630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f2491cc2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a565d750 0x5653a565fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f2492cc4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 0x5653a5654630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39982/0 (socket says 172.21.15.2:39982) 2024-03-15T13:00:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f24924c3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a5654b70 0x5653a565d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f2492cc4640 1 -- 172.21.15.2:0/3925068797 learned_addr learned my addr 172.21.15.2:0/3925068797 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f2492cc4640 1 --2- 172.21.15.2:0/2241708082 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f24740a1cc0 0x7f24740a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f248b7fe640 1 -- 172.21.15.2:0/2241708082 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f2480014070 con 0x7f248c1056a0 2024-03-15T13:00:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f24924c3640 1 -- 172.21.15.2:0/3925068797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 msgr2=0x5653a5654630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f24924c3640 1 --2- 172.21.15.2:0/3925068797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 0x5653a5654630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f24924c3640 1 -- 172.21.15.2:0/3925068797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a565d750 msgr2=0x5653a565fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f24924c3640 1 --2- 172.21.15.2:0/3925068797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a565d750 0x5653a565fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f24924c3640 1 -- 172.21.15.2:0/3925068797 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653a557e060 con 0x5653a5654b70 2024-03-15T13:00:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f2492cc4640 1 --2- 172.21.15.2:0/2241708082 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f24740a1cc0 0x7f24740a4180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f247c033000 tx=0x7f247c005c50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f24924c3640 1 --2- 172.21.15.2:0/3925068797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a5654b70 0x5653a565d210 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2484009bd0 tx=0x7f2484011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d725de5f91c09348 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f2488ff9640 1 -- 172.21.15.2:0/3925068797 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484004230 con 0x5653a5654b70 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f2488ff9640 1 -- 172.21.15.2:0/3925068797 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2484009e40 con 0x5653a5654b70 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.756+0000 7f2488ff9640 1 -- 172.21.15.2:0/3925068797 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484004ca0 con 0x5653a5654b70 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.757+0000 7f24973d6740 1 -- 172.21.15.2:0/3925068797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a5654b70 msgr2=0x5653a565d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.757+0000 7f24973d6740 1 --2- 172.21.15.2:0/3925068797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a5654b70 0x5653a565d210 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2484009bd0 tx=0x7f2484011810 comp rx=0 tx=0).stop 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.757+0000 7f24973d6740 1 -- 172.21.15.2:0/3925068797 shutdown_connections 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.757+0000 7f24973d6740 1 --2- 172.21.15.2:0/3925068797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a565d750 0x5653a565fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.757+0000 7f24973d6740 1 --2- 172.21.15.2:0/3925068797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a5654b70 0x5653a565d210 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.757+0000 7f24973d6740 1 --2- 172.21.15.2:0/3925068797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 0x5653a5654630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.757+0000 7f24973d6740 1 -- 172.21.15.2:0/3925068797 >> 172.21.15.2:0/3925068797 conn(0x5653a564e980 msgr2=0x5653a564ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.757+0000 7f24973d6740 1 -- 172.21.15.2:0/3925068797 shutdown_connections 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.757+0000 7f24973d6740 1 -- 172.21.15.2:0/3925068797 wait complete. 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.757+0000 7f24973d6740 1 -- start start 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24973d6740 1 Processor -- start 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24973d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a5654250 0x5653a565cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24973d6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654b70 0x5653a565a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24973d6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a565d750 0x5653a565a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24973d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a55bc020 con 0x5653a5654250 2024-03-15T13:00:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24973d6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a528f430 con 0x5653a5654b70 2024-03-15T13:00:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24973d6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a524be00 con 0x5653a565d750 2024-03-15T13:00:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24973d6740 10 client.0 _ll_register_callbacks cb 0x5653a5563cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24924c3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a5654250 0x5653a565cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24924c3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a5654250 0x5653a565cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53968/0 (socket says 172.21.15.2:53968) 2024-03-15T13:00:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24924c3640 1 -- 172.21.15.2:0/1943840607 learned_addr learned my addr 172.21.15.2:0/1943840607 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f2492cc4640 1 --2- 172.21.15.2:0/1943840607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a565d750 0x5653a565a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f2491cc2640 1 --2- 172.21.15.2:0/1943840607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654b70 0x5653a565a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24924c3640 1 -- 172.21.15.2:0/1943840607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a565d750 msgr2=0x5653a565a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24924c3640 1 --2- 172.21.15.2:0/1943840607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a565d750 0x5653a565a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24924c3640 1 -- 172.21.15.2:0/1943840607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654b70 msgr2=0x5653a565a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24924c3640 1 --2- 172.21.15.2:0/1943840607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654b70 0x5653a565a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24924c3640 1 -- 172.21.15.2:0/1943840607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2484009830 con 0x5653a5654250 2024-03-15T13:00:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f24924c3640 1 --2- 172.21.15.2:0/1943840607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a5654250 0x5653a565cfe0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2484004c90 tx=0x7f2484004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386e21abab117ce7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f2471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.758+0000 7f2471ffb640 1 -- 172.21.15.2:0/1943840607 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484024670 con 0x5653a5654250 2024-03-15T13:00:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.759+0000 7f2471ffb640 1 -- 172.21.15.2:0/1943840607 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f24840247d0 con 0x5653a5654250 2024-03-15T13:00:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.759+0000 7f24973d6740 1 -- 172.21.15.2:0/1943840607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653a52bb7e0 con 0x5653a5654250 2024-03-15T13:00:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.759+0000 7f2471ffb640 1 -- 172.21.15.2:0/1943840607 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2484024990 con 0x5653a5654250 2024-03-15T13:00:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.759+0000 7f24973d6740 10 client.16215 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.759+0000 7f24973d6740 1 -- 172.21.15.2:0/1943840607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5653a55ba6a0 con 0x5653a5654250 2024-03-15T13:00:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.759+0000 7f2471ffb640 1 -- 172.21.15.2:0/1943840607 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5653a52bb7e0 con 0x5653a5654250 2024-03-15T13:00:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.759+0000 7f2471ffb640 1 -- 172.21.15.2:0/1943840607 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f24840196d0 con 0x5653a5654250 2024-03-15T13:00:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.759+0000 7f24973d6740 10 client.16215 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.759+0000 7f24973d6740 10 client.16215 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:00:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.759+0000 7f24973d6740 20 client.16215 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.759+0000 7f24973d6740 1 --2- 172.21.15.2:0/1943840607 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5653a566d730 0x5653a566fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.759+0000 7f24973d6740 4 client.16215 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:22.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.759+0000 7f24973d6740 1 -- 172.21.15.2:0/1943840607 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653a52f8720 con 0x5653a566d730 2024-03-15T13:00:22.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.759+0000 7f2491cc2640 1 --2- 172.21.15.2:0/1943840607 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5653a566d730 0x5653a566fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.760+0000 7f2471ffb640 10 client.16215 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:00:22.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.760+0000 7f2491cc2640 1 --2- 172.21.15.2:0/1943840607 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5653a566d730 0x5653a566fb70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21f7901200498158 server_cookie=75126c8044dadefa in_seq=0 out_seq=0 2024-03-15T13:00:22.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.761+0000 7f2471ffb640 1 -- 172.21.15.2:0/1943840607 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5653a52f8720 con 0x5653a566d730 2024-03-15T13:00:22.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.761+0000 7f2471ffb640 10 client.16215 handle_command_reply: tid=0 2024-03-15T13:00:22.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.761+0000 7f24973d6740 1 client.16215 shutdown 2024-03-15T13:00:22.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.761+0000 7f24973d6740 1 -- 172.21.15.2:0/1943840607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a5654250 msgr2=0x5653a565cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.761+0000 7f24973d6740 1 --2- 172.21.15.2:0/1943840607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a5654250 0x5653a565cfe0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2484004c90 tx=0x7f2484004cc0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.761+0000 7f24973d6740 1 -- 172.21.15.2:0/1943840607 shutdown_connections 2024-03-15T13:00:22.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.761+0000 7f24973d6740 1 --2- 172.21.15.2:0/1943840607 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5653a566d730 0x5653a566fb70 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.761+0000 7f24973d6740 1 --2- 172.21.15.2:0/1943840607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a565d750 0x5653a565a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.761+0000 7f24973d6740 1 --2- 172.21.15.2:0/1943840607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654b70 0x5653a565a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.761+0000 7f2471ffb640 0 client.16215 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:00:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.761+0000 7f24973d6740 1 --2- 172.21.15.2:0/1943840607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a5654250 0x5653a565cfe0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.761+0000 7f24973d6740 1 -- 172.21.15.2:0/1943840607 >> 172.21.15.2:0/1943840607 conn(0x5653a564e980 msgr2=0x5653a56506d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.761+0000 7f24973d6740 1 -- 172.21.15.2:0/1943840607 shutdown_connections 2024-03-15T13:00:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.761+0000 7f24973d6740 1 -- 172.21.15.2:0/1943840607 wait complete. 2024-03-15T13:00:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.761+0000 7f24973d6740 20 client.16215 trim_cache size 0 max 16384 2024-03-15T13:00:22.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.773+0000 7f24973d6740 1 Processor -- start 2024-03-15T13:00:22.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.773+0000 7f24973d6740 1 -- start start 2024-03-15T13:00:22.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.773+0000 7f24973d6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 0x5653a5655920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.773+0000 7f24973d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a5654b70 0x5653a564da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.773+0000 7f24973d6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a565d750 0x5653a565caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.773+0000 7f24973d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a5541130 con 0x5653a5654b70 2024-03-15T13:00:22.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.773+0000 7f24973d6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a566a240 con 0x5653a5654250 2024-03-15T13:00:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.773+0000 7f24973d6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a5655e90 con 0x5653a565d750 2024-03-15T13:00:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.773+0000 7f2491cc2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a5654b70 0x5653a564da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.774+0000 7f24924c3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 0x5653a5655920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.774+0000 7f2491cc2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a5654b70 0x5653a564da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53976/0 (socket says 172.21.15.2:53976) 2024-03-15T13:00:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.774+0000 7f24924c3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 0x5653a5655920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:39994/0 (socket says 172.21.15.2:39994) 2024-03-15T13:00:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.774+0000 7f2491cc2640 1 -- 172.21.15.2:0/1223098141 learned_addr learned my addr 172.21.15.2:0/1223098141 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.774+0000 7f2492cc4640 1 --2- 172.21.15.2:0/1223098141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a565d750 0x5653a565caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.774+0000 7f2491cc2640 1 -- 172.21.15.2:0/1223098141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a565d750 msgr2=0x5653a565caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.774+0000 7f2491cc2640 1 --2- 172.21.15.2:0/1223098141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a565d750 0x5653a565caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.774+0000 7f2491cc2640 1 -- 172.21.15.2:0/1223098141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 msgr2=0x5653a5655920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.774+0000 7f2491cc2640 1 --2- 172.21.15.2:0/1223098141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 0x5653a5655920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.774+0000 7f2491cc2640 1 -- 172.21.15.2:0/1223098141 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653a5556c20 con 0x5653a5654b70 2024-03-15T13:00:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.774+0000 7f24924c3640 1 --2- 172.21.15.2:0/1223098141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 0x5653a5655920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.774+0000 7f2491cc2640 1 --2- 172.21.15.2:0/1223098141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a5654b70 0x5653a564da00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f248009a4c0 tx=0x7f2480085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a24ffb19d9ef04 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.774+0000 7f2473fff640 1 -- 172.21.15.2:0/1223098141 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248009aec0 con 0x5653a5654b70 2024-03-15T13:00:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.774+0000 7f2473fff640 1 -- 172.21.15.2:0/1223098141 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f248009a7c0 con 0x5653a5654b70 2024-03-15T13:00:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.774+0000 7f2473fff640 1 -- 172.21.15.2:0/1223098141 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24800047e0 con 0x5653a5654b70 2024-03-15T13:00:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 -- 172.21.15.2:0/1223098141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a5654b70 msgr2=0x5653a564da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 --2- 172.21.15.2:0/1223098141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a5654b70 0x5653a564da00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f248009a4c0 tx=0x7f2480085040 comp rx=0 tx=0).stop 2024-03-15T13:00:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 -- 172.21.15.2:0/1223098141 shutdown_connections 2024-03-15T13:00:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 --2- 172.21.15.2:0/1223098141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a565d750 0x5653a565caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 --2- 172.21.15.2:0/1223098141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a5654b70 0x5653a564da00 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 --2- 172.21.15.2:0/1223098141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 0x5653a5655920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 -- 172.21.15.2:0/1223098141 >> 172.21.15.2:0/1223098141 conn(0x5653a564e980 msgr2=0x5653a564d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 -- 172.21.15.2:0/1223098141 shutdown_connections 2024-03-15T13:00:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 -- 172.21.15.2:0/1223098141 wait complete. 2024-03-15T13:00:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 -- start start 2024-03-15T13:00:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 Processor -- start 2024-03-15T13:00:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 0x5653a5660d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a5654b70 0x5653a56612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a565d750 0x5653a56c8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a5661a30 con 0x5653a565d750 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a5661ba0 con 0x5653a5654250 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653a56c8580 con 0x5653a5654b70 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24973d6740 10 client.0 _ll_register_callbacks cb 0x7f248000cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f2492cc4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a565d750 0x5653a56c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f2492cc4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a565d750 0x5653a56c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53992/0 (socket says 172.21.15.2:53992) 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f2492cc4640 1 -- 172.21.15.2:0/3424791112 learned_addr learned my addr 172.21.15.2:0/3424791112 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f2491cc2640 1 --2- 172.21.15.2:0/3424791112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a5654b70 0x5653a56612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.775+0000 7f24924c3640 1 --2- 172.21.15.2:0/3424791112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 0x5653a5660d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.776+0000 7f2492cc4640 1 -- 172.21.15.2:0/3424791112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a5654b70 msgr2=0x5653a56612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.776+0000 7f2492cc4640 1 --2- 172.21.15.2:0/3424791112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a5654b70 0x5653a56612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.776+0000 7f2492cc4640 1 -- 172.21.15.2:0/3424791112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 msgr2=0x5653a5660d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.776+0000 7f2492cc4640 1 --2- 172.21.15.2:0/3424791112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 0x5653a5660d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.776+0000 7f2492cc4640 1 -- 172.21.15.2:0/3424791112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653a55ba6a0 con 0x5653a565d750 2024-03-15T13:00:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.776+0000 7f2492cc4640 1 --2- 172.21.15.2:0/3424791112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a565d750 0x5653a56c8040 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f247c02c630 tx=0x7f247c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8dd8feef0b6469d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.776+0000 7f2471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.776+0000 7f2471ffb640 1 -- 172.21.15.2:0/3424791112 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c003b70 con 0x5653a565d750 2024-03-15T13:00:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.776+0000 7f24973d6740 1 -- 172.21.15.2:0/3424791112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2484009e40 con 0x5653a565d750 2024-03-15T13:00:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.776+0000 7f2471ffb640 1 -- 172.21.15.2:0/3424791112 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f247c003cd0 con 0x5653a565d750 2024-03-15T13:00:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.776+0000 7f2471ffb640 1 -- 172.21.15.2:0/3424791112 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f247c003e90 con 0x5653a565d750 2024-03-15T13:00:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.776+0000 7f24973d6740 10 client.16221 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.776+0000 7f24973d6740 1 -- 172.21.15.2:0/3424791112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f248c1a4c60 con 0x5653a565d750 2024-03-15T13:00:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.776+0000 7f2471ffb640 1 -- 172.21.15.2:0/3424791112 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2484009e40 con 0x5653a565d750 2024-03-15T13:00:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f2471ffb640 1 -- 172.21.15.2:0/3424791112 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f247c04e040 con 0x5653a565d750 2024-03-15T13:00:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 10 client.16221 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 10 client.16221 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:00:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 20 client.16221 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 1 --2- 172.21.15.2:0/3424791112 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5653a56c9df0 0x5653a56cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 4 client.16221 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:00:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 1 -- 172.21.15.2:0/3424791112 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5653a5347890 con 0x5653a56c9df0 2024-03-15T13:00:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24924c3640 1 --2- 172.21.15.2:0/3424791112 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5653a56c9df0 0x5653a56cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f2471ffb640 10 client.16221 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:00:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24924c3640 1 --2- 172.21.15.2:0/3424791112 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5653a56c9df0 0x5653a56cc210 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=388dc7cfe63bef6f server_cookie=67a0ced93e568844 in_seq=0 out_seq=0 2024-03-15T13:00:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f2471ffb640 1 -- 172.21.15.2:0/3424791112 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5653a5347890 con 0x5653a56c9df0 2024-03-15T13:00:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f2471ffb640 10 client.16221 handle_command_reply: tid=0 2024-03-15T13:00:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 1 client.16221 shutdown 2024-03-15T13:00:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 1 -- 172.21.15.2:0/3424791112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a565d750 msgr2=0x5653a56c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 1 --2- 172.21.15.2:0/3424791112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a565d750 0x5653a56c8040 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f247c02c630 tx=0x7f247c02c0f0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 1 -- 172.21.15.2:0/3424791112 shutdown_connections 2024-03-15T13:00:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 1 --2- 172.21.15.2:0/3424791112 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5653a56c9df0 0x5653a56cc210 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 1 --2- 172.21.15.2:0/3424791112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653a565d750 0x5653a56c8040 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 1 --2- 172.21.15.2:0/3424791112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5653a5654b70 0x5653a56612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f2471ffb640 0 client.16221 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:00:22.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 1 --2- 172.21.15.2:0/3424791112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653a5654250 0x5653a5660d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 1 -- 172.21.15.2:0/3424791112 >> 172.21.15.2:0/3424791112 conn(0x5653a564e980 msgr2=0x5653a564db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 1 -- 172.21.15.2:0/3424791112 shutdown_connections 2024-03-15T13:00:22.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.777+0000 7f24973d6740 1 -- 172.21.15.2:0/3424791112 wait complete. 2024-03-15T13:00:22.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.778+0000 7f24973d6740 20 client.16221 trim_cache size 0 max 16384 2024-03-15T13:00:22.770 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:00:22.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.778+0000 7f2494727640 1 -- 172.21.15.2:0/2241708082 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f24740a1cc0 msgr2=0x7f24740a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.778+0000 7f2494727640 1 --2- 172.21.15.2:0/2241708082 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f24740a1cc0 0x7f24740a4180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f247c033000 tx=0x7f247c005c50 comp rx=0 tx=0).stop 2024-03-15T13:00:22.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.778+0000 7f2494727640 1 -- 172.21.15.2:0/2241708082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f248c1056a0 msgr2=0x7f248c19dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.778+0000 7f2494727640 1 --2- 172.21.15.2:0/2241708082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f248c1056a0 0x7f248c19dd80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f248000e9c0 tx=0x7f248000ee80 comp rx=0 tx=0).stop 2024-03-15T13:00:22.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.778+0000 7f2494727640 1 -- 172.21.15.2:0/2241708082 shutdown_connections 2024-03-15T13:00:22.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.778+0000 7f2494727640 1 --2- 172.21.15.2:0/2241708082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f248c1979f0 0x7f248c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.778+0000 7f2494727640 1 --2- 172.21.15.2:0/2241708082 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f24740a1cc0 0x7f24740a4180 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.778+0000 7f2494727640 1 --2- 172.21.15.2:0/2241708082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f248c1056a0 0x7f248c19dd80 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.778+0000 7f2494727640 1 --2- 172.21.15.2:0/2241708082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f248c1005d0 0x7f248c19d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.778+0000 7f2494727640 1 -- 172.21.15.2:0/2241708082 >> 172.21.15.2:0/2241708082 conn(0x7f248c0774f0 msgr2=0x7f248c104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.778+0000 7f2494727640 1 -- 172.21.15.2:0/2241708082 shutdown_connections 2024-03-15T13:00:22.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.778+0000 7f2494727640 1 -- 172.21.15.2:0/2241708082 wait complete. 2024-03-15T13:00:22.773 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:22 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/224461327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:00:22.774 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:00:22.774 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:00:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.892+0000 7fa6e0cdf640 1 -- 172.21.15.2:0/2845653057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6dc107570 msgr2=0x7fa6dc107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.892+0000 7fa6e0cdf640 1 --2- 172.21.15.2:0/2845653057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6dc107570 0x7fa6dc107950 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009900 tx=0x7fa6d402bc30 comp rx=0 tx=0).stop 2024-03-15T13:00:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.893+0000 7fa6e0cdf640 1 -- 172.21.15.2:0/2845653057 shutdown_connections 2024-03-15T13:00:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.893+0000 7fa6e0cdf640 1 --2- 172.21.15.2:0/2845653057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6dc101e70 0x7fa6dc10e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.893+0000 7fa6e0cdf640 1 --2- 172.21.15.2:0/2845653057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6dc1014b0 0x7fa6dc101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.893+0000 7fa6e0cdf640 1 --2- 172.21.15.2:0/2845653057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6dc107570 0x7fa6dc107950 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.893+0000 7fa6e0cdf640 1 -- 172.21.15.2:0/2845653057 >> 172.21.15.2:0/2845653057 conn(0x7fa6dc0fcf20 msgr2=0x7fa6dc0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.893+0000 7fa6e0cdf640 1 -- 172.21.15.2:0/2845653057 shutdown_connections 2024-03-15T13:00:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.893+0000 7fa6e0cdf640 1 -- 172.21.15.2:0/2845653057 wait complete. 2024-03-15T13:00:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.893+0000 7fa6e0cdf640 1 Processor -- start 2024-03-15T13:00:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.893+0000 7fa6e0cdf640 1 -- start start 2024-03-15T13:00:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6e0cdf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6dc1014b0 0x7fa6dc19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6e0cdf640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6dc101e70 0x7fa6dc19dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6e0cdf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6dc107570 0x7fa6dc1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6e0cdf640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc197f30 con 0x7fa6dc1014b0 2024-03-15T13:00:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6e0cdf640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc1980a0 con 0x7fa6dc101e70 2024-03-15T13:00:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6e0cdf640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc198260 con 0x7fa6dc107570 2024-03-15T13:00:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6da59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6dc1014b0 0x7fa6dc19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6da59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6dc1014b0 0x7fa6dc19d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54012/0 (socket says 172.21.15.2:54012) 2024-03-15T13:00:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6da59c640 1 -- 172.21.15.2:0/3030725953 learned_addr learned my addr 172.21.15.2:0/3030725953 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6dad9d640 1 --2- 172.21.15.2:0/3030725953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6dc107570 0x7fa6dc1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6d9d9b640 1 --2- 172.21.15.2:0/3030725953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6dc101e70 0x7fa6dc19dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6da59c640 1 -- 172.21.15.2:0/3030725953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6dc107570 msgr2=0x7fa6dc1978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6da59c640 1 --2- 172.21.15.2:0/3030725953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6dc107570 0x7fa6dc1978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6da59c640 1 -- 172.21.15.2:0/3030725953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6dc101e70 msgr2=0x7fa6dc19dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6da59c640 1 --2- 172.21.15.2:0/3030725953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6dc101e70 0x7fa6dc19dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6da59c640 1 -- 172.21.15.2:0/3030725953 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6cc009840 con 0x7fa6dc1014b0 2024-03-15T13:00:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6da59c640 1 --2- 172.21.15.2:0/3030725953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6dc1014b0 0x7fa6dc19d6e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009230 tx=0x7fa6d402cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6d37fe640 1 -- 172.21.15.2:0/3030725953 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4004250 con 0x7fa6dc1014b0 2024-03-15T13:00:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6d37fe640 1 -- 172.21.15.2:0/3030725953 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa6d40043b0 con 0x7fa6dc1014b0 2024-03-15T13:00:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6d37fe640 1 -- 172.21.15.2:0/3030725953 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4003c20 con 0x7fa6dc1014b0 2024-03-15T13:00:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6e0cdf640 1 -- 172.21.15.2:0/3030725953 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d4009560 con 0x7fa6dc1014b0 2024-03-15T13:00:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.894+0000 7fa6e0cdf640 1 -- 172.21.15.2:0/3030725953 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6dc1a45a0 con 0x7fa6dc1014b0 2024-03-15T13:00:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.896+0000 7fa6d37fe640 1 -- 172.21.15.2:0/3030725953 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7fa6d4005020 con 0x7fa6dc1014b0 2024-03-15T13:00:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6e399a740 1 Processor -- start 2024-03-15T13:00:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6e399a740 1 -- start start 2024-03-15T13:00:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6d37fe640 1 --2- 172.21.15.2:0/3030725953 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa6c80a1cc0 0x7fa6c80a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6e399a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce8375250 0x55cce8375630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6e399a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375b70 0x55cce837e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6e399a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce837e750 0x55cce8380b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6e399a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce8330960 con 0x55cce837e750 2024-03-15T13:00:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6e399a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce82e4310 con 0x55cce8375250 2024-03-15T13:00:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6e399a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce82e1840 con 0x55cce8375b70 2024-03-15T13:00:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6dad9d640 1 --2- 172.21.15.2:0/3030725953 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa6c80a1cc0 0x7fa6c80a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6d37fe640 1 -- 172.21.15.2:0/3030725953 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa6d40efbf0 con 0x7fa6dc1014b0 2024-03-15T13:00:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6da59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce8375250 0x55cce8375630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6dad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce837e750 0x55cce8380b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6d9d9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375b70 0x55cce837e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6da59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce8375250 0x55cce8375630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:40022/0 (socket says 172.21.15.2:40022) 2024-03-15T13:00:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6dad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce837e750 0x55cce8380b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54022/0 (socket says 172.21.15.2:54022) 2024-03-15T13:00:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.897+0000 7fa6da59c640 1 -- 172.21.15.2:0/1749245506 learned_addr learned my addr 172.21.15.2:0/1749245506 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.898+0000 7fa6da59c640 1 -- 172.21.15.2:0/1749245506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375b70 msgr2=0x55cce837e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.898+0000 7fa6da59c640 1 --2- 172.21.15.2:0/1749245506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375b70 0x55cce837e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.898+0000 7fa6da59c640 1 -- 172.21.15.2:0/1749245506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce837e750 msgr2=0x55cce8380b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.898+0000 7fa6da59c640 1 --2- 172.21.15.2:0/1749245506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce837e750 0x55cce8380b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.898+0000 7fa6da59c640 1 -- 172.21.15.2:0/1749245506 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cce829f060 con 0x55cce8375250 2024-03-15T13:00:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.898+0000 7fa6d9d9b640 1 --2- 172.21.15.2:0/1749245506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375b70 0x55cce837e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.898+0000 7fa6dad9d640 1 --2- 172.21.15.2:0/3030725953 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa6c80a1cc0 0x7fa6c80a4180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc005fd0 tx=0x7fa6cc0114f0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.898+0000 7fa6da59c640 1 --2- 172.21.15.2:0/1749245506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce8375250 0x55cce8375630 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40b88c0 tx=0x7fa6d4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b44cefaf7cf995c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.898+0000 7fa6d0ff9640 1 -- 172.21.15.2:0/1749245506 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d40bbfa0 con 0x55cce8375250 2024-03-15T13:00:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.898+0000 7fa6d0ff9640 1 -- 172.21.15.2:0/1749245506 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa6d40bbdc0 con 0x55cce8375250 2024-03-15T13:00:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.898+0000 7fa6d0ff9640 1 -- 172.21.15.2:0/1749245506 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d40bc460 con 0x55cce8375250 2024-03-15T13:00:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.899+0000 7fa6e399a740 1 -- 172.21.15.2:0/1749245506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce8375250 msgr2=0x55cce8375630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.899+0000 7fa6e399a740 1 --2- 172.21.15.2:0/1749245506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce8375250 0x55cce8375630 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40b88c0 tx=0x7fa6d4005fd0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.899+0000 7fa6e399a740 1 -- 172.21.15.2:0/1749245506 shutdown_connections 2024-03-15T13:00:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.899+0000 7fa6e399a740 1 --2- 172.21.15.2:0/1749245506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce837e750 0x55cce8380b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.899+0000 7fa6e399a740 1 --2- 172.21.15.2:0/1749245506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375b70 0x55cce837e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.899+0000 7fa6e399a740 1 --2- 172.21.15.2:0/1749245506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce8375250 0x55cce8375630 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.899+0000 7fa6e399a740 1 -- 172.21.15.2:0/1749245506 >> 172.21.15.2:0/1749245506 conn(0x55cce836f980 msgr2=0x55cce836fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.899+0000 7fa6e399a740 1 -- 172.21.15.2:0/1749245506 shutdown_connections 2024-03-15T13:00:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.899+0000 7fa6e399a740 1 -- 172.21.15.2:0/1749245506 wait complete. 2024-03-15T13:00:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.899+0000 7fa6e399a740 1 -- start start 2024-03-15T13:00:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.899+0000 7fa6e399a740 1 Processor -- start 2024-03-15T13:00:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6e399a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375250 0x55cce837e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6e399a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375b70 0x55cce837b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6e399a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce837e750 0x55cce837b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6e399a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce82dd020 con 0x55cce8375b70 2024-03-15T13:00:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6e399a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce7fb0430 con 0x55cce837e750 2024-03-15T13:00:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6e399a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce7f6ce00 con 0x55cce8375250 2024-03-15T13:00:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6e399a740 10 client.0 _ll_register_callbacks cb 0x55cce8284cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6d9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375b70 0x55cce837b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6da59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375250 0x55cce837e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6d9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375b70 0x55cce837b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54034/0 (socket says 172.21.15.2:54034) 2024-03-15T13:00:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6d9d9b640 1 -- 172.21.15.2:0/1990357574 learned_addr learned my addr 172.21.15.2:0/1990357574 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6dad9d640 1 --2- 172.21.15.2:0/1990357574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce837e750 0x55cce837b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6d9d9b640 1 -- 172.21.15.2:0/1990357574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375250 msgr2=0x55cce837e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6d9d9b640 1 --2- 172.21.15.2:0/1990357574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375250 0x55cce837e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6d9d9b640 1 -- 172.21.15.2:0/1990357574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce837e750 msgr2=0x55cce837b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6d9d9b640 1 --2- 172.21.15.2:0/1990357574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce837e750 0x55cce837b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6d9d9b640 1 -- 172.21.15.2:0/1990357574 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6dc1a45a0 con 0x55cce8375b70 2024-03-15T13:00:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6d9d9b640 1 --2- 172.21.15.2:0/1990357574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375b70 0x55cce837b410 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40126e0 tx=0x7fa6c4012ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a052bd2caf58dc server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6c1ffb640 1 -- 172.21.15.2:0/1990357574 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4017070 con 0x55cce8375b70 2024-03-15T13:00:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6c1ffb640 1 -- 172.21.15.2:0/1990357574 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa6c400f5f0 con 0x55cce8375b70 2024-03-15T13:00:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.900+0000 7fa6e399a740 1 -- 172.21.15.2:0/1990357574 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cce7fdc7e0 con 0x55cce8375b70 2024-03-15T13:00:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.901+0000 7fa6c1ffb640 1 -- 172.21.15.2:0/1990357574 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6c400f7b0 con 0x55cce8375b70 2024-03-15T13:00:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.901+0000 7fa6e399a740 10 client.16233 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.901+0000 7fa6e399a740 1 -- 172.21.15.2:0/1990357574 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cce82db6a0 con 0x55cce8375b70 2024-03-15T13:00:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.901+0000 7fa6c1ffb640 1 -- 172.21.15.2:0/1990357574 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55cce7fdc7e0 con 0x55cce8375b70 2024-03-15T13:00:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.901+0000 7fa6c1ffb640 1 -- 172.21.15.2:0/1990357574 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa6c4010a40 con 0x55cce8375b70 2024-03-15T13:00:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.901+0000 7fa6e399a740 10 client.16233 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.901+0000 7fa6e399a740 10 client.16233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.901+0000 7fa6e399a740 20 client.16233 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.901+0000 7fa6e399a740 1 --2- 172.21.15.2:0/1990357574 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cce838e760 0x55cce8390b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.901+0000 7fa6e399a740 4 client.16233 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.901+0000 7fa6e399a740 1 -- 172.21.15.2:0/1990357574 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cce8019720 con 0x55cce838e760 2024-03-15T13:00:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.901+0000 7fa6da59c640 1 --2- 172.21.15.2:0/1990357574 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cce838e760 0x55cce8390b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6da59c640 1 --2- 172.21.15.2:0/1990357574 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cce838e760 0x55cce8390b80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc6e3cca60fc0082 server_cookie=a99600c647c44660 in_seq=0 out_seq=0 2024-03-15T13:00:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6c1ffb640 10 client.16233 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6c1ffb640 1 -- 172.21.15.2:0/1990357574 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cce8019720 con 0x55cce838e760 2024-03-15T13:00:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6c1ffb640 10 client.16233 handle_command_reply: tid=0 2024-03-15T13:00:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6e399a740 1 client.16233 shutdown 2024-03-15T13:00:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6e399a740 1 -- 172.21.15.2:0/1990357574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375b70 msgr2=0x55cce837b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6e399a740 1 --2- 172.21.15.2:0/1990357574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375b70 0x55cce837b410 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40126e0 tx=0x7fa6c4012ae0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6e399a740 1 -- 172.21.15.2:0/1990357574 shutdown_connections 2024-03-15T13:00:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6e399a740 1 --2- 172.21.15.2:0/1990357574 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cce838e760 0x55cce8390b80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6e399a740 1 --2- 172.21.15.2:0/1990357574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce837e750 0x55cce837b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6e399a740 1 --2- 172.21.15.2:0/1990357574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375b70 0x55cce837b410 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6c1ffb640 0 client.16233 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6e399a740 1 --2- 172.21.15.2:0/1990357574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375250 0x55cce837e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6e399a740 1 -- 172.21.15.2:0/1990357574 >> 172.21.15.2:0/1990357574 conn(0x55cce836f980 msgr2=0x55cce83716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6e399a740 1 -- 172.21.15.2:0/1990357574 shutdown_connections 2024-03-15T13:00:22.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6e399a740 1 -- 172.21.15.2:0/1990357574 wait complete. 2024-03-15T13:00:22.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.902+0000 7fa6e399a740 20 client.16233 trim_cache size 0 max 16384 2024-03-15T13:00:22.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6e399a740 1 Processor -- start 2024-03-15T13:00:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6e399a740 1 -- start start 2024-03-15T13:00:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6e399a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce8375250 0x55cce837db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6e399a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375b70 0x55cce837e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6e399a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce837e750 0x55cce8389890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6e399a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce8262130 con 0x55cce8375b70 2024-03-15T13:00:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6e399a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce7fdc810 con 0x55cce8375250 2024-03-15T13:00:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6e399a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce8377030 con 0x55cce837e750 2024-03-15T13:00:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6d9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375b70 0x55cce837e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6d9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375b70 0x55cce837e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54050/0 (socket says 172.21.15.2:54050) 2024-03-15T13:00:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6d9d9b640 1 -- 172.21.15.2:0/1495416212 learned_addr learned my addr 172.21.15.2:0/1495416212 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6da59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce8375250 0x55cce837db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6dad9d640 1 --2- 172.21.15.2:0/1495416212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce837e750 0x55cce8389890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6d9d9b640 1 -- 172.21.15.2:0/1495416212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce837e750 msgr2=0x55cce8389890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6d9d9b640 1 --2- 172.21.15.2:0/1495416212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce837e750 0x55cce8389890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6d9d9b640 1 -- 172.21.15.2:0/1495416212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce8375250 msgr2=0x55cce837db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6d9d9b640 1 --2- 172.21.15.2:0/1495416212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce8375250 0x55cce837db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.915+0000 7fa6d9d9b640 1 -- 172.21.15.2:0/1495416212 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cce8277c20 con 0x55cce8375b70 2024-03-15T13:00:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.916+0000 7fa6d9d9b640 1 --2- 172.21.15.2:0/1495416212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375b70 0x55cce837e080 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4010e70 tx=0x7fa6c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8dc728a1e96419 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.916+0000 7fa6c3fff640 1 -- 172.21.15.2:0/1495416212 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4017040 con 0x55cce8375b70 2024-03-15T13:00:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.916+0000 7fa6c3fff640 1 -- 172.21.15.2:0/1495416212 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa6c4010430 con 0x55cce8375b70 2024-03-15T13:00:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.916+0000 7fa6c3fff640 1 -- 172.21.15.2:0/1495416212 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4003970 con 0x55cce8375b70 2024-03-15T13:00:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.916+0000 7fa6e399a740 1 -- 172.21.15.2:0/1495416212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375b70 msgr2=0x55cce837e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.916+0000 7fa6e399a740 1 --2- 172.21.15.2:0/1495416212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375b70 0x55cce837e080 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4010e70 tx=0x7fa6c4009e30 comp rx=0 tx=0).stop 2024-03-15T13:00:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.916+0000 7fa6e399a740 1 -- 172.21.15.2:0/1495416212 shutdown_connections 2024-03-15T13:00:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.916+0000 7fa6e399a740 1 --2- 172.21.15.2:0/1495416212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce837e750 0x55cce8389890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.916+0000 7fa6e399a740 1 --2- 172.21.15.2:0/1495416212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375b70 0x55cce837e080 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.916+0000 7fa6e399a740 1 --2- 172.21.15.2:0/1495416212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce8375250 0x55cce837db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.916+0000 7fa6e399a740 1 -- 172.21.15.2:0/1495416212 >> 172.21.15.2:0/1495416212 conn(0x55cce836f980 msgr2=0x55cce8376920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.916+0000 7fa6e399a740 1 -- 172.21.15.2:0/1495416212 shutdown_connections 2024-03-15T13:00:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.916+0000 7fa6e399a740 1 -- 172.21.15.2:0/1495416212 wait complete. 2024-03-15T13:00:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6e399a740 1 -- start start 2024-03-15T13:00:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6e399a740 1 Processor -- start 2024-03-15T13:00:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6e399a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375250 0x55cce8381f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6e399a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375b70 0x55cce8382440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6e399a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce837e750 0x55cce83e9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6e399a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce8382be0 con 0x55cce8375250 2024-03-15T13:00:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6e399a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce83e9580 con 0x55cce837e750 2024-03-15T13:00:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6e399a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cce83e9740 con 0x55cce8375b70 2024-03-15T13:00:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6e399a740 10 client.0 _ll_register_callbacks cb 0x7fa6c4009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6da59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375250 0x55cce8381f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6da59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375250 0x55cce8381f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54060/0 (socket says 172.21.15.2:54060) 2024-03-15T13:00:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6da59c640 1 -- 172.21.15.2:0/799459995 learned_addr learned my addr 172.21.15.2:0/799459995 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6dad9d640 1 --2- 172.21.15.2:0/799459995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce837e750 0x55cce83e9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6d9d9b640 1 --2- 172.21.15.2:0/799459995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375b70 0x55cce8382440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6da59c640 1 -- 172.21.15.2:0/799459995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375b70 msgr2=0x55cce8382440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6da59c640 1 --2- 172.21.15.2:0/799459995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375b70 0x55cce8382440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6da59c640 1 -- 172.21.15.2:0/799459995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce837e750 msgr2=0x55cce83e9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6da59c640 1 --2- 172.21.15.2:0/799459995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce837e750 0x55cce83e9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6da59c640 1 -- 172.21.15.2:0/799459995 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cce82db6a0 con 0x55cce8375250 2024-03-15T13:00:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6da59c640 1 --2- 172.21.15.2:0/799459995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375250 0x55cce8381f00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa6d402ce10 tx=0x7fa6d40bc330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6507633e7bed5b71 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.917+0000 7fa6c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.918+0000 7fa6c1ffb640 1 -- 172.21.15.2:0/799459995 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d40bcb60 con 0x55cce8375250 2024-03-15T13:00:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.918+0000 7fa6c1ffb640 1 -- 172.21.15.2:0/799459995 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa6d40d6070 con 0x55cce8375250 2024-03-15T13:00:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.918+0000 7fa6e399a740 1 -- 172.21.15.2:0/799459995 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6d40bbdc0 con 0x55cce8375250 2024-03-15T13:00:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.918+0000 7fa6c1ffb640 1 -- 172.21.15.2:0/799459995 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6d40c05b0 con 0x55cce8375250 2024-03-15T13:00:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.918+0000 7fa6e399a740 10 client.16239 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.918+0000 7fa6e399a740 1 -- 172.21.15.2:0/799459995 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cce829d8c0 con 0x55cce8375250 2024-03-15T13:00:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.918+0000 7fa6c1ffb640 1 -- 172.21.15.2:0/799459995 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa6d40bbdc0 con 0x55cce8375250 2024-03-15T13:00:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.918+0000 7fa6c1ffb640 1 -- 172.21.15.2:0/799459995 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa6d403c070 con 0x55cce8375250 2024-03-15T13:00:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.918+0000 7fa6e399a740 10 client.16239 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.918+0000 7fa6e399a740 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.918+0000 7fa6e399a740 20 client.16239 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.918+0000 7fa6e399a740 1 --2- 172.21.15.2:0/799459995 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cce83eaa80 0x55cce83ece70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.918+0000 7fa6e399a740 4 client.16239 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:00:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.918+0000 7fa6e399a740 1 -- 172.21.15.2:0/799459995 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cce8068890 con 0x55cce83eaa80 2024-03-15T13:00:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.918+0000 7fa6d9d9b640 1 --2- 172.21.15.2:0/799459995 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cce83eaa80 0x55cce83ece70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6d9d9b640 1 --2- 172.21.15.2:0/799459995 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cce83eaa80 0x55cce83ece70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9334da9e9928afdb server_cookie=19383d48119dea55 in_seq=0 out_seq=0 2024-03-15T13:00:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6c1ffb640 10 client.16239 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6c1ffb640 1 -- 172.21.15.2:0/799459995 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cce8068890 con 0x55cce83eaa80 2024-03-15T13:00:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6c1ffb640 10 client.16239 handle_command_reply: tid=0 2024-03-15T13:00:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6e399a740 1 client.16239 shutdown 2024-03-15T13:00:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6e399a740 1 -- 172.21.15.2:0/799459995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375250 msgr2=0x55cce8381f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6e399a740 1 --2- 172.21.15.2:0/799459995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375250 0x55cce8381f00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa6d402ce10 tx=0x7fa6d40bc330 comp rx=0 tx=0).stop 2024-03-15T13:00:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6e399a740 1 -- 172.21.15.2:0/799459995 shutdown_connections 2024-03-15T13:00:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6e399a740 1 --2- 172.21.15.2:0/799459995 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cce83eaa80 0x55cce83ece70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6e399a740 1 --2- 172.21.15.2:0/799459995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce837e750 0x55cce83e9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6c1ffb640 0 client.16239 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6e399a740 1 --2- 172.21.15.2:0/799459995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cce8375b70 0x55cce8382440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6e399a740 1 --2- 172.21.15.2:0/799459995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cce8375250 0x55cce8381f00 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6e399a740 1 -- 172.21.15.2:0/799459995 >> 172.21.15.2:0/799459995 conn(0x55cce836f980 msgr2=0x55cce837ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6e399a740 1 -- 172.21.15.2:0/799459995 shutdown_connections 2024-03-15T13:00:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6e399a740 1 -- 172.21.15.2:0/799459995 wait complete. 2024-03-15T13:00:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.919+0000 7fa6e399a740 20 client.16239 trim_cache size 0 max 16384 2024-03-15T13:00:22.912 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:00:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.920+0000 7fa6e0cdf640 1 -- 172.21.15.2:0/3030725953 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa6c80a1cc0 msgr2=0x7fa6c80a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.920+0000 7fa6e0cdf640 1 --2- 172.21.15.2:0/3030725953 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa6c80a1cc0 0x7fa6c80a4180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc005fd0 tx=0x7fa6cc0114f0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.920+0000 7fa6e0cdf640 1 -- 172.21.15.2:0/3030725953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6dc1014b0 msgr2=0x7fa6dc19d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.920+0000 7fa6e0cdf640 1 --2- 172.21.15.2:0/3030725953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6dc1014b0 0x7fa6dc19d6e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009230 tx=0x7fa6d402cac0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.920+0000 7fa6e0cdf640 1 -- 172.21.15.2:0/3030725953 shutdown_connections 2024-03-15T13:00:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.920+0000 7fa6e0cdf640 1 --2- 172.21.15.2:0/3030725953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6dc107570 0x7fa6dc1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.920+0000 7fa6e0cdf640 1 --2- 172.21.15.2:0/3030725953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6dc101e70 0x7fa6dc19dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.920+0000 7fa6e0cdf640 1 --2- 172.21.15.2:0/3030725953 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa6c80a1cc0 0x7fa6c80a4180 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.920+0000 7fa6e0cdf640 1 --2- 172.21.15.2:0/3030725953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6dc1014b0 0x7fa6dc19d6e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.920+0000 7fa6e0cdf640 1 -- 172.21.15.2:0/3030725953 >> 172.21.15.2:0/3030725953 conn(0x7fa6dc0fcf20 msgr2=0x7fa6dc0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.920+0000 7fa6e0cdf640 1 -- 172.21.15.2:0/3030725953 shutdown_connections 2024-03-15T13:00:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:22.920+0000 7fa6e0cdf640 1 -- 172.21.15.2:0/3030725953 wait complete. 2024-03-15T13:00:23.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:22 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/224461327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:00:23.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:22 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/224461327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:00:23.917 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:00:23.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:00:23.918 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:00:24.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.054+0000 7fb2e41e2640 1 -- 172.21.15.2:0/1920520601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2dc10de90 msgr2=0x7fb2dc110280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.054+0000 7fb2e41e2640 1 --2- 172.21.15.2:0/1920520601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2dc10de90 0x7fb2dc110280 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009900 tx=0x7fb2d402bc30 comp rx=0 tx=0).stop 2024-03-15T13:00:24.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.054+0000 7fb2e41e2640 1 -- 172.21.15.2:0/1920520601 shutdown_connections 2024-03-15T13:00:24.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.054+0000 7fb2e41e2640 1 --2- 172.21.15.2:0/1920520601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2dc10de90 0x7fb2dc110280 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.054+0000 7fb2e41e2640 1 --2- 172.21.15.2:0/1920520601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2dc068ff0 0x7fb2dc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.054+0000 7fb2e41e2640 1 --2- 172.21.15.2:0/1920520601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2dc0686d0 0x7fb2dc068ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.054+0000 7fb2e41e2640 1 -- 172.21.15.2:0/1920520601 >> 172.21.15.2:0/1920520601 conn(0x7fb2dc0774f0 msgr2=0x7fb2dc077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.054+0000 7fb2e41e2640 1 -- 172.21.15.2:0/1920520601 shutdown_connections 2024-03-15T13:00:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.054+0000 7fb2e41e2640 1 -- 172.21.15.2:0/1920520601 wait complete. 2024-03-15T13:00:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.055+0000 7fb2e41e2640 1 Processor -- start 2024-03-15T13:00:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.055+0000 7fb2e41e2640 1 -- start start 2024-03-15T13:00:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.055+0000 7fb2e41e2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2dc0686d0 0x7fb2dc19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.055+0000 7fb2e41e2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2dc068ff0 0x7fb2dc19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.055+0000 7fb2e41e2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2dc10de90 0x7fb2dc199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.055+0000 7fb2e41e2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc19a0f0 con 0x7fb2dc10de90 2024-03-15T13:00:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.055+0000 7fb2e41e2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc19a260 con 0x7fb2dc0686d0 2024-03-15T13:00:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.055+0000 7fb2e41e2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc19a420 con 0x7fb2dc068ff0 2024-03-15T13:00:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.055+0000 7fb2e277f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2dc10de90 0x7fb2dc199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.055+0000 7fb2e1f7e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2dc0686d0 0x7fb2dc19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.055+0000 7fb2e277f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2dc10de90 0x7fb2dc199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54088/0 (socket says 172.21.15.2:54088) 2024-03-15T13:00:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.055+0000 7fb2e177d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2dc068ff0 0x7fb2dc19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.055+0000 7fb2e277f640 1 -- 172.21.15.2:0/358037112 learned_addr learned my addr 172.21.15.2:0/358037112 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.056+0000 7fb2e277f640 1 -- 172.21.15.2:0/358037112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2dc068ff0 msgr2=0x7fb2dc19fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.056+0000 7fb2e277f640 1 --2- 172.21.15.2:0/358037112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2dc068ff0 0x7fb2dc19fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.056+0000 7fb2e277f640 1 -- 172.21.15.2:0/358037112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2dc0686d0 msgr2=0x7fb2dc19f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.056+0000 7fb2e277f640 1 --2- 172.21.15.2:0/358037112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2dc0686d0 0x7fb2dc19f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.056+0000 7fb2e277f640 1 -- 172.21.15.2:0/358037112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d8009840 con 0x7fb2dc10de90 2024-03-15T13:00:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.056+0000 7fb2e277f640 1 --2- 172.21.15.2:0/358037112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2dc10de90 0x7fb2dc199aa0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4005940 tx=0x7fb2d402cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.056+0000 7fb2d2ffd640 1 -- 172.21.15.2:0/358037112 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4005d50 con 0x7fb2dc10de90 2024-03-15T13:00:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.056+0000 7fb2e41e2640 1 -- 172.21.15.2:0/358037112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d4009560 con 0x7fb2dc10de90 2024-03-15T13:00:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.056+0000 7fb2d2ffd640 1 -- 172.21.15.2:0/358037112 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb2d40054f0 con 0x7fb2dc10de90 2024-03-15T13:00:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.056+0000 7fb2d2ffd640 1 -- 172.21.15.2:0/358037112 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4003b20 con 0x7fb2dc10de90 2024-03-15T13:00:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.056+0000 7fb2e41e2640 1 -- 172.21.15.2:0/358037112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2dc1a67b0 con 0x7fb2dc10de90 2024-03-15T13:00:24.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.059+0000 7fb2d2ffd640 1 -- 172.21.15.2:0/358037112 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7fb2d4004350 con 0x7fb2dc10de90 2024-03-15T13:00:24.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.059+0000 7fb2e6dc1740 1 Processor -- start 2024-03-15T13:00:24.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.059+0000 7fb2e6dc1740 1 -- start start 2024-03-15T13:00:24.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.059+0000 7fb2e6dc1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4e380 0x563deae4e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.059+0000 7fb2e6dc1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae4ed30 0x563deae572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.059+0000 7fb2e6dc1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae578b0 0x563deae59d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.059+0000 7fb2e6dc1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deadbcf90 con 0x563deae4e380 2024-03-15T13:00:24.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.059+0000 7fb2e6dc1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deadba4c0 con 0x563deae578b0 2024-03-15T13:00:24.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.059+0000 7fb2e6dc1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deadb5ca0 con 0x563deae4ed30 2024-03-15T13:00:24.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.059+0000 7fb2d2ffd640 1 --2- 172.21.15.2:0/358037112 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb2c80a1cc0 0x7fb2c80a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.059+0000 7fb2e177d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4e380 0x563deae4e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2e1f7e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae4ed30 0x563deae572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2e177d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4e380 0x563deae4e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54100/0 (socket says 172.21.15.2:54100) 2024-03-15T13:00:24.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2e177d640 1 -- 172.21.15.2:0/1040257184 learned_addr learned my addr 172.21.15.2:0/1040257184 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:24.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2e277f640 1 --2- 172.21.15.2:0/358037112 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb2c80a1cc0 0x7fb2c80a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2d2ffd640 1 -- 172.21.15.2:0/358037112 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fb2d40f01e0 con 0x7fb2dc10de90 2024-03-15T13:00:24.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2e177d640 1 --2- 172.21.15.2:0/1040257184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae578b0 0x563deae59d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2e1f7e640 1 -- 172.21.15.2:0/1040257184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae578b0 msgr2=0x563deae59d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2e1f7e640 1 --2- 172.21.15.2:0/1040257184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae578b0 0x563deae59d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2e1f7e640 1 -- 172.21.15.2:0/1040257184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4e380 msgr2=0x563deae4e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2e1f7e640 1 --2- 172.21.15.2:0/1040257184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4e380 0x563deae4e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2e1f7e640 1 -- 172.21.15.2:0/1040257184 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563dead780b0 con 0x563deae4ed30 2024-03-15T13:00:24.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2e277f640 1 --2- 172.21.15.2:0/358037112 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb2c80a1cc0 0x7fb2c80a4180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40ed5d0 tx=0x7fb2d40ac040 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:24.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2e1f7e640 1 --2- 172.21.15.2:0/1040257184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae4ed30 0x563deae572e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8005fd0 tx=0x7fb2d8011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f60c4ac7e9d5c455 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:24.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2c7fff640 1 -- 172.21.15.2:0/1040257184 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8003c30 con 0x563deae4ed30 2024-03-15T13:00:24.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2c7fff640 1 -- 172.21.15.2:0/1040257184 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb2d8003d90 con 0x563deae4ed30 2024-03-15T13:00:24.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.060+0000 7fb2c7fff640 1 -- 172.21.15.2:0/1040257184 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d800fbc0 con 0x563deae4ed30 2024-03-15T13:00:24.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.061+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/1040257184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae4ed30 msgr2=0x563deae572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.061+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/1040257184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae4ed30 0x563deae572e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8005fd0 tx=0x7fb2d8011a00 comp rx=0 tx=0).stop 2024-03-15T13:00:24.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.061+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/1040257184 shutdown_connections 2024-03-15T13:00:24.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.061+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/1040257184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae578b0 0x563deae59d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.061+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/1040257184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae4ed30 0x563deae572e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.061+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/1040257184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4e380 0x563deae4e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.061+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/1040257184 >> 172.21.15.2:0/1040257184 conn(0x563deae48840 msgr2=0x563deae48c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:24.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.061+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/1040257184 shutdown_connections 2024-03-15T13:00:24.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.061+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/1040257184 wait complete. 2024-03-15T13:00:24.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.062+0000 7fb2e6dc1740 1 -- start start 2024-03-15T13:00:24.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.062+0000 7fb2e6dc1740 1 Processor -- start 2024-03-15T13:00:24.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.062+0000 7fb2e6dc1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4e380 0x563deae54880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.062+0000 7fb2e6dc1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4ed30 0x563deae54dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.062+0000 7fb2e6dc1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae578b0 0x563deae55300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.062+0000 7fb2e6dc1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dead5af40 con 0x563deae4e380 2024-03-15T13:00:24.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.062+0000 7fb2e6dc1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deaa894b0 con 0x563deae4ed30 2024-03-15T13:00:24.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.062+0000 7fb2e6dc1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dead3b670 con 0x563deae578b0 2024-03-15T13:00:24.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.062+0000 7fb2e6dc1740 10 client.0 _ll_register_callbacks cb 0x563deada8f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:24.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.062+0000 7fb2e177d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4e380 0x563deae54880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.062+0000 7fb2e177d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4e380 0x563deae54880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54106/0 (socket says 172.21.15.2:54106) 2024-03-15T13:00:24.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.062+0000 7fb2e177d640 1 -- 172.21.15.2:0/2259814494 learned_addr learned my addr 172.21.15.2:0/2259814494 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:24.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.062+0000 7fb2e1f7e640 1 --2- 172.21.15.2:0/2259814494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4ed30 0x563deae54dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.062+0000 7fb2e177d640 1 --2- 172.21.15.2:0/2259814494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae578b0 0x563deae55300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.062+0000 7fb2e177d640 1 -- 172.21.15.2:0/2259814494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae578b0 msgr2=0x563deae55300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.063+0000 7fb2e177d640 1 --2- 172.21.15.2:0/2259814494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae578b0 0x563deae55300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.063+0000 7fb2e177d640 1 -- 172.21.15.2:0/2259814494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4ed30 msgr2=0x563deae54dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.063+0000 7fb2e177d640 1 --2- 172.21.15.2:0/2259814494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4ed30 0x563deae54dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.063+0000 7fb2e177d640 1 -- 172.21.15.2:0/2259814494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d8009d90 con 0x563deae4e380 2024-03-15T13:00:24.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.063+0000 7fb2e1f7e640 1 --2- 172.21.15.2:0/2259814494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4ed30 0x563deae54dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:00:24.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.063+0000 7fb2e177d640 1 --2- 172.21.15.2:0/2259814494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4e380 0x563deae54880 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009f60 tx=0x7fb2cc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54eb907b1735b7b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:24.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.063+0000 7fb2c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:24.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.063+0000 7fb2c57fa640 1 -- 172.21.15.2:0/2259814494 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc012110 con 0x563deae4e380 2024-03-15T13:00:24.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.063+0000 7fb2c57fa640 1 -- 172.21.15.2:0/2259814494 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fb2cc012270 con 0x563deae4e380 2024-03-15T13:00:24.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.063+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/2259814494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563dead8ff40 con 0x563deae4e380 2024-03-15T13:00:24.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.063+0000 7fb2c57fa640 1 -- 172.21.15.2:0/2259814494 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2cc004050 con 0x563deae4e380 2024-03-15T13:00:24.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.063+0000 7fb2e6dc1740 10 client.16251 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:24.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.063+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/2259814494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563deadb4320 con 0x563deae4e380 2024-03-15T13:00:24.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.063+0000 7fb2c57fa640 1 -- 172.21.15.2:0/2259814494 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x563dead8ff40 con 0x563deae4e380 2024-03-15T13:00:24.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.064+0000 7fb2c57fa640 1 -- 172.21.15.2:0/2259814494 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb2cc010430 con 0x563deae4e380 2024-03-15T13:00:24.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.064+0000 7fb2e6dc1740 10 client.16251 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:24.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.064+0000 7fb2e6dc1740 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:24.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.064+0000 7fb2e6dc1740 20 client.16251 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:24.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.064+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/2259814494 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563deae67e10 0x563deae6a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.064+0000 7fb2e6dc1740 4 client.16251 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.064+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/2259814494 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563deadac310 con 0x563deae67e10 2024-03-15T13:00:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.064+0000 7fb2e1f7e640 1 --2- 172.21.15.2:0/2259814494 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563deae67e10 0x563deae6a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.064+0000 7fb2e1f7e640 1 --2- 172.21.15.2:0/2259814494 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563deae67e10 0x563deae6a230 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d57e441ad35cec49 server_cookie=3bba86df36f01bac in_seq=0 out_seq=0 2024-03-15T13:00:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.064+0000 7fb2c57fa640 10 client.16251 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.066+0000 7fb2c57fa640 1 -- 172.21.15.2:0/2259814494 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563deadac310 con 0x563deae67e10 2024-03-15T13:00:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.066+0000 7fb2c57fa640 10 client.16251 handle_command_reply: tid=0 2024-03-15T13:00:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.066+0000 7fb2e6dc1740 1 client.16251 shutdown 2024-03-15T13:00:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.066+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/2259814494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4e380 msgr2=0x563deae54880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.066+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/2259814494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4e380 0x563deae54880 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009f60 tx=0x7fb2cc005f90 comp rx=0 tx=0).stop 2024-03-15T13:00:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.066+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/2259814494 shutdown_connections 2024-03-15T13:00:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.066+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/2259814494 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563deae67e10 0x563deae6a230 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.066+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/2259814494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae578b0 0x563deae55300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.066+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/2259814494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4ed30 0x563deae54dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.066+0000 7fb2c57fa640 0 client.16251 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.066+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/2259814494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4e380 0x563deae54880 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.066+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/2259814494 >> 172.21.15.2:0/2259814494 conn(0x563deae48840 msgr2=0x563deae4a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.066+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/2259814494 shutdown_connections 2024-03-15T13:00:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.066+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/2259814494 wait complete. 2024-03-15T13:00:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.066+0000 7fb2e6dc1740 20 client.16251 trim_cache size 0 max 16384 2024-03-15T13:00:24.057 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:23 smithi002 ceph-mon[32060]: pgmap v47: 129 pgs: 129 active+clean; 83 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.8 MiB/s wr, 46 op/s 2024-03-15T13:00:24.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:23 smithi184 ceph-mon[35993]: pgmap v47: 129 pgs: 129 active+clean; 83 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.8 MiB/s wr, 46 op/s 2024-03-15T13:00:24.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.086+0000 7fb2e6dc1740 1 Processor -- start 2024-03-15T13:00:24.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.086+0000 7fb2e6dc1740 1 -- start start 2024-03-15T13:00:24.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.086+0000 7fb2e6dc1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4e380 0x563deae56fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.086+0000 7fb2e6dc1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae4ed30 0x563deae62db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.086+0000 7fb2e6dc1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae578b0 0x563deae632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.086+0000 7fb2e6dc1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deae64970 con 0x563deae578b0 2024-03-15T13:00:24.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.086+0000 7fb2e6dc1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deae48370 con 0x563deae4e380 2024-03-15T13:00:24.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.086+0000 7fb2e6dc1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deae42dc0 con 0x563deae4ed30 2024-03-15T13:00:24.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.086+0000 7fb2e177d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae578b0 0x563deae632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.086+0000 7fb2e1f7e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae4ed30 0x563deae62db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.086+0000 7fb2e177d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae578b0 0x563deae632f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54114/0 (socket says 172.21.15.2:54114) 2024-03-15T13:00:24.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.086+0000 7fb2e177d640 1 -- 172.21.15.2:0/705803443 learned_addr learned my addr 172.21.15.2:0/705803443 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:24.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.086+0000 7fb2e177d640 1 --2- 172.21.15.2:0/705803443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4e380 0x563deae56fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.087+0000 7fb2e177d640 1 -- 172.21.15.2:0/705803443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae4ed30 msgr2=0x563deae62db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.087+0000 7fb2e177d640 1 --2- 172.21.15.2:0/705803443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae4ed30 0x563deae62db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.087+0000 7fb2e177d640 1 -- 172.21.15.2:0/705803443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4e380 msgr2=0x563deae56fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.087+0000 7fb2e177d640 1 --2- 172.21.15.2:0/705803443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4e380 0x563deae56fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.087+0000 7fb2e177d640 1 -- 172.21.15.2:0/705803443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563dead50c00 con 0x563deae578b0 2024-03-15T13:00:24.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.087+0000 7fb2e1f7e640 1 --2- 172.21.15.2:0/705803443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae4ed30 0x563deae62db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:24.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.087+0000 7fb2e177d640 1 --2- 172.21.15.2:0/705803443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae578b0 0x563deae632f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc000c00 tx=0x7fb2cc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0fb76d3169d6ae server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:24.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.087+0000 7fb2c77fe640 1 -- 172.21.15.2:0/705803443 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc0129e0 con 0x563deae578b0 2024-03-15T13:00:24.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.087+0000 7fb2c77fe640 1 -- 172.21.15.2:0/705803443 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb2cc009ad0 con 0x563deae578b0 2024-03-15T13:00:24.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.087+0000 7fb2c77fe640 1 -- 172.21.15.2:0/705803443 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc00fc20 con 0x563deae578b0 2024-03-15T13:00:24.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.088+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/705803443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae578b0 msgr2=0x563deae632f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.088+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/705803443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae578b0 0x563deae632f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc000c00 tx=0x7fb2cc009f90 comp rx=0 tx=0).stop 2024-03-15T13:00:24.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.088+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/705803443 shutdown_connections 2024-03-15T13:00:24.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.088+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/705803443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae578b0 0x563deae632f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.088+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/705803443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae4ed30 0x563deae62db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.088+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/705803443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4e380 0x563deae56fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.088+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/705803443 >> 172.21.15.2:0/705803443 conn(0x563deae48840 msgr2=0x563deae478e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:24.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.088+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/705803443 shutdown_connections 2024-03-15T13:00:24.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.088+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/705803443 wait complete. 2024-03-15T13:00:24.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.088+0000 7fb2e6dc1740 1 -- start start 2024-03-15T13:00:24.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.088+0000 7fb2e6dc1740 1 Processor -- start 2024-03-15T13:00:24.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e6dc1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4e380 0x563deae5c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e6dc1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4ed30 0x563deae5ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e6dc1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae578b0 0x563deae5d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e6dc1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deae528b0 con 0x563deae4ed30 2024-03-15T13:00:24.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e6dc1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deaec36f0 con 0x563deae4e380 2024-03-15T13:00:24.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e6dc1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563deaec3830 con 0x563deae578b0 2024-03-15T13:00:24.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e6dc1740 10 client.0 _ll_register_callbacks cb 0x7fb2cc009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:24.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e1f7e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4ed30 0x563deae5ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e1f7e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4ed30 0x563deae5ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54126/0 (socket says 172.21.15.2:54126) 2024-03-15T13:00:24.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e1f7e640 1 -- 172.21.15.2:0/3051030599 learned_addr learned my addr 172.21.15.2:0/3051030599 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e177d640 1 --2- 172.21.15.2:0/3051030599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4e380 0x563deae5c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e177d640 1 --2- 172.21.15.2:0/3051030599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae578b0 0x563deae5d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e1f7e640 1 -- 172.21.15.2:0/3051030599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae578b0 msgr2=0x563deae5d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e1f7e640 1 --2- 172.21.15.2:0/3051030599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae578b0 0x563deae5d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e1f7e640 1 -- 172.21.15.2:0/3051030599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4e380 msgr2=0x563deae5c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e1f7e640 1 --2- 172.21.15.2:0/3051030599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4e380 0x563deae5c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e1f7e640 1 -- 172.21.15.2:0/3051030599 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563dead780b0 con 0x563deae4ed30 2024-03-15T13:00:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e177d640 1 --2- 172.21.15.2:0/3051030599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae578b0 0x563deae5d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:00:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e1f7e640 1 --2- 172.21.15.2:0/3051030599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4ed30 0x563deae5ce00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009370 tx=0x7fb2d800f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f426afd05f4917 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2c57fa640 1 -- 172.21.15.2:0/3051030599 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8029040 con 0x563deae4ed30 2024-03-15T13:00:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2c57fa640 1 -- 172.21.15.2:0/3051030599 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fb2d8009500 con 0x563deae4ed30 2024-03-15T13:00:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.089+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/3051030599 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563deaab5860 con 0x563deae4ed30 2024-03-15T13:00:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.090+0000 7fb2c57fa640 1 -- 172.21.15.2:0/3051030599 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2d802b2c0 con 0x563deae4ed30 2024-03-15T13:00:24.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.090+0000 7fb2e6dc1740 10 client.16257 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:24.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.090+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/3051030599 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563deadb4320 con 0x563deae4ed30 2024-03-15T13:00:24.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.090+0000 7fb2c57fa640 1 -- 172.21.15.2:0/3051030599 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x563deaab5860 con 0x563deae4ed30 2024-03-15T13:00:24.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.090+0000 7fb2c57fa640 1 -- 172.21.15.2:0/3051030599 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb2d8021040 con 0x563deae4ed30 2024-03-15T13:00:24.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.090+0000 7fb2e6dc1740 10 client.16257 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:24.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.090+0000 7fb2e6dc1740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:24.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.090+0000 7fb2e6dc1740 20 client.16257 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:24.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.090+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/3051030599 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563deaec4f40 0x563deaec7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.090+0000 7fb2e6dc1740 4 client.16257 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:00:24.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.090+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/3051030599 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563deab06410 con 0x563deaec4f40 2024-03-15T13:00:24.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.090+0000 7fb2e177d640 1 --2- 172.21.15.2:0/3051030599 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563deaec4f40 0x563deaec7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.091+0000 7fb2c57fa640 10 client.16257 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:24.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.091+0000 7fb2e177d640 1 --2- 172.21.15.2:0/3051030599 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563deaec4f40 0x563deaec7360 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e53cc98adeb7decf server_cookie=3b5844c852b73ba1 in_seq=0 out_seq=0 2024-03-15T13:00:24.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:23 smithi149 ceph-mon[35927]: pgmap v47: 129 pgs: 129 active+clean; 83 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.8 MiB/s wr, 46 op/s 2024-03-15T13:00:24.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.140+0000 7fb2c57fa640 1 -- 172.21.15.2:0/3051030599 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563deab06410 con 0x563deaec4f40 2024-03-15T13:00:24.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.140+0000 7fb2c57fa640 10 client.16257 handle_command_reply: tid=0 2024-03-15T13:00:24.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.140+0000 7fb2e6dc1740 1 client.16257 shutdown 2024-03-15T13:00:24.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.140+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/3051030599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4ed30 msgr2=0x563deae5ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.140+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/3051030599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4ed30 0x563deae5ce00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009370 tx=0x7fb2d800f6f0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.140+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/3051030599 shutdown_connections 2024-03-15T13:00:24.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.140+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/3051030599 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563deaec4f40 0x563deaec7360 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.140+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/3051030599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563deae578b0 0x563deae5d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.140+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/3051030599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563deae4ed30 0x563deae5ce00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.140+0000 7fb2e6dc1740 1 --2- 172.21.15.2:0/3051030599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563deae4e380 0x563deae5c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.140+0000 7fb2c57fa640 0 client.16257 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:24.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.140+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/3051030599 >> 172.21.15.2:0/3051030599 conn(0x563deae48840 msgr2=0x563deae54670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:24.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.140+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/3051030599 shutdown_connections 2024-03-15T13:00:24.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.140+0000 7fb2e6dc1740 1 -- 172.21.15.2:0/3051030599 wait complete. 2024-03-15T13:00:24.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.141+0000 7fb2e6dc1740 20 client.16257 trim_cache size 0 max 16384 2024-03-15T13:00:24.128 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:00:24.128 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:00:24.128 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "50af8c8b-51b9-47a3-8bd4-fbfc4fb70a8e", 2024-03-15T13:00:24.128 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:00:24.128 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:00:24.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.141+0000 7fb2e41e2640 1 -- 172.21.15.2:0/358037112 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb2c80a1cc0 msgr2=0x7fb2c80a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.141+0000 7fb2e41e2640 1 --2- 172.21.15.2:0/358037112 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb2c80a1cc0 0x7fb2c80a4180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40ed5d0 tx=0x7fb2d40ac040 comp rx=0 tx=0).stop 2024-03-15T13:00:24.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.141+0000 7fb2e41e2640 1 -- 172.21.15.2:0/358037112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2dc10de90 msgr2=0x7fb2dc199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.141+0000 7fb2e41e2640 1 --2- 172.21.15.2:0/358037112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2dc10de90 0x7fb2dc199aa0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4005940 tx=0x7fb2d402cac0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.141+0000 7fb2e41e2640 1 -- 172.21.15.2:0/358037112 shutdown_connections 2024-03-15T13:00:24.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.141+0000 7fb2e41e2640 1 --2- 172.21.15.2:0/358037112 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb2c80a1cc0 0x7fb2c80a4180 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.141+0000 7fb2e41e2640 1 --2- 172.21.15.2:0/358037112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2dc10de90 0x7fb2dc199aa0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.141+0000 7fb2e41e2640 1 --2- 172.21.15.2:0/358037112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2dc068ff0 0x7fb2dc19fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.141+0000 7fb2e41e2640 1 --2- 172.21.15.2:0/358037112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2dc0686d0 0x7fb2dc19f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.141+0000 7fb2e41e2640 1 -- 172.21.15.2:0/358037112 >> 172.21.15.2:0/358037112 conn(0x7fb2dc0774f0 msgr2=0x7fb2dc10f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:24.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.141+0000 7fb2e41e2640 1 -- 172.21.15.2:0/358037112 shutdown_connections 2024-03-15T13:00:24.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.141+0000 7fb2e41e2640 1 -- 172.21.15.2:0/358037112 wait complete. 2024-03-15T13:00:24.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:00:24.137 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:00:24.154 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querycap.xml 2024-03-15T13:00:24.161 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-queryctrl.xml 2024-03-15T13:00:24.165 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querystd.xml 2024-03-15T13:00:24.168 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-reqbufs.xml 2024-03-15T13:00:24.171 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-s-hw-freq-seek.xml 2024-03-15T13:00:24.175 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-streamon.xml 2024-03-15T13:00:24.178 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-interval.xml 2024-03-15T13:00:24.180 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-size.xml 2024-03-15T13:00:24.183 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-mbus-code.xml 2024-03-15T13:00:24.186 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-crop.xml 2024-03-15T13:00:24.189 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-fmt.xml 2024-03-15T13:00:24.192 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-frame-interval.xml 2024-03-15T13:00:24.195 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-selection.xml 2024-03-15T13:00:24.198 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml 2024-03-15T13:00:24.201 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_525.gif.b64 2024-03-15T13:00:24.205 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_625.gif.b64 2024-03-15T13:00:24.208 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_hsync.gif.b64 2024-03-15T13:00:24.211 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/media_api.tmpl 2024-03-15T13:00:24.214 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/mtdnand.tmpl 2024-03-15T13:00:24.218 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/networking.tmpl 2024-03-15T13:00:24.221 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/rapidio.tmpl 2024-03-15T13:00:24.224 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/regulator.tmpl 2024-03-15T13:00:24.228 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/s390-drivers.tmpl 2024-03-15T13:00:24.231 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/scsi.tmpl 2024-03-15T13:00:24.234 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/sh.tmpl 2024-03-15T13:00:24.237 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/stylesheet.xsl 2024-03-15T13:00:24.241 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/tracepoint.tmpl 2024-03-15T13:00:24.244 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/uio-howto.tmpl 2024-03-15T13:00:24.248 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/usb.tmpl 2024-03-15T13:00:24.252 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/w1.tmpl 2024-03-15T13:00:24.254 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/writing-an-alsa-driver.tmpl 2024-03-15T13:00:24.261 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/writing_musb_glue_layer.tmpl 2024-03-15T13:00:24.264 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/writing_usb_driver.tmpl 2024-03-15T13:00:24.268 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/DocBook/z8530book.tmpl 2024-03-15T13:00:24.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.283+0000 7fcf3d29a640 1 -- 172.21.15.2:0/2583466416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf381056b0 msgr2=0x7fcf3810d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.283+0000 7fcf3d29a640 1 --2- 172.21.15.2:0/2583466416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf381056b0 0x7fcf3810d280 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009cd0 tx=0x7fcf3002c0f0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.284+0000 7fcf3d29a640 1 -- 172.21.15.2:0/2583466416 shutdown_connections 2024-03-15T13:00:24.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.284+0000 7fcf3d29a640 1 --2- 172.21.15.2:0/2583466416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf381056b0 0x7fcf3810d280 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.284+0000 7fcf3d29a640 1 --2- 172.21.15.2:0/2583466416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf38100f90 0x7fcf38104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.284+0000 7fcf3d29a640 1 --2- 172.21.15.2:0/2583466416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf381005e0 0x7fcf381009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.284+0000 7fcf3d29a640 1 -- 172.21.15.2:0/2583466416 >> 172.21.15.2:0/2583466416 conn(0x7fcf380775a0 msgr2=0x7fcf380779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:24.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.284+0000 7fcf3d29a640 1 -- 172.21.15.2:0/2583466416 shutdown_connections 2024-03-15T13:00:24.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.284+0000 7fcf3d29a640 1 -- 172.21.15.2:0/2583466416 wait complete. 2024-03-15T13:00:24.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.284+0000 7fcf3d29a640 1 Processor -- start 2024-03-15T13:00:24.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.285+0000 7fcf3d29a640 1 -- start start 2024-03-15T13:00:24.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.285+0000 7fcf3d29a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf381005e0 0x7fcf38198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.285+0000 7fcf3d29a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf381056b0 0x7fcf38198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.285+0000 7fcf3d29a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf381993e0 0x7fcf381a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.285+0000 7fcf3d29a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf381a4030 con 0x7fcf381993e0 2024-03-15T13:00:24.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.285+0000 7fcf3d29a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf381a41a0 con 0x7fcf381005e0 2024-03-15T13:00:24.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.285+0000 7fcf3d29a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf381a4360 con 0x7fcf381056b0 2024-03-15T13:00:24.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.285+0000 7fcf377fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf381993e0 0x7fcf381a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.285+0000 7fcf367fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf381056b0 0x7fcf38198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.285+0000 7fcf377fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf381993e0 0x7fcf381a39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54146/0 (socket says 172.21.15.2:54146) 2024-03-15T13:00:24.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.285+0000 7fcf377fe640 1 -- 172.21.15.2:0/2379617637 learned_addr learned my addr 172.21.15.2:0/2379617637 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:24.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.285+0000 7fcf36ffd640 1 --2- 172.21.15.2:0/2379617637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf381005e0 0x7fcf38198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.286+0000 7fcf377fe640 1 -- 172.21.15.2:0/2379617637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf381056b0 msgr2=0x7fcf38198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.286+0000 7fcf377fe640 1 --2- 172.21.15.2:0/2379617637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf381056b0 0x7fcf38198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.286+0000 7fcf377fe640 1 -- 172.21.15.2:0/2379617637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf381005e0 msgr2=0x7fcf38198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.286+0000 7fcf377fe640 1 --2- 172.21.15.2:0/2379617637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf381005e0 0x7fcf38198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.286+0000 7fcf377fe640 1 -- 172.21.15.2:0/2379617637 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf30009900 con 0x7fcf381993e0 2024-03-15T13:00:24.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.286+0000 7fcf377fe640 1 --2- 172.21.15.2:0/2379617637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf381993e0 0x7fcf381a39e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcf30005cc0 tx=0x7fcf30034a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:24.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.286+0000 7fcf1ffff640 1 -- 172.21.15.2:0/2379617637 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf30032880 con 0x7fcf381993e0 2024-03-15T13:00:24.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.286+0000 7fcf3d29a640 1 -- 172.21.15.2:0/2379617637 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf381a45e0 con 0x7fcf381993e0 2024-03-15T13:00:24.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.286+0000 7fcf1ffff640 1 -- 172.21.15.2:0/2379617637 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcf30034e80 con 0x7fcf381993e0 2024-03-15T13:00:24.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.286+0000 7fcf1ffff640 1 -- 172.21.15.2:0/2379617637 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf300047c0 con 0x7fcf381993e0 2024-03-15T13:00:24.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.286+0000 7fcf3d29a640 1 -- 172.21.15.2:0/2379617637 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf381a4ae0 con 0x7fcf381993e0 2024-03-15T13:00:24.274 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/EDID/ 2024-03-15T13:00:24.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.289+0000 7fcf3fe7c740 1 Processor -- start 2024-03-15T13:00:24.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.289+0000 7fcf3fe7c740 1 -- start start 2024-03-15T13:00:24.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.289+0000 7fcf1ffff640 1 -- 172.21.15.2:0/2379617637 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 136511+0+0 (secure 0 0 0) 0x7fcf3000a040 con 0x7fcf381993e0 2024-03-15T13:00:24.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.289+0000 7fcf3fe7c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8af60 0x55f878c8b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf3fe7c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8b880 0x55f878c98680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf3fe7c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c98bc0 0x55f878c9afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf3fe7c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f878bfa280 con 0x55f878c98bc0 2024-03-15T13:00:24.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf3fe7c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f878bf77b0 con 0x55f878c8af60 2024-03-15T13:00:24.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf3fe7c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f878bf2f90 con 0x55f878c8b880 2024-03-15T13:00:24.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8b880 0x55f878c98680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8b880 0x55f878c98680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37158/0 (socket says 172.21.15.2:37158) 2024-03-15T13:00:24.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf36ffd640 1 -- 172.21.15.2:0/1290015611 learned_addr learned my addr 172.21.15.2:0/1290015611 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:24.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf367fc640 1 --2- 172.21.15.2:0/1290015611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c98bc0 0x55f878c9afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf367fc640 1 --2- 172.21.15.2:0/1290015611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8af60 0x55f878c8b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf1ffff640 1 --2- 172.21.15.2:0/2379617637 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcf200a1d40 0x7fcf200a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf36ffd640 1 -- 172.21.15.2:0/1290015611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8af60 msgr2=0x55f878c8b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf36ffd640 1 --2- 172.21.15.2:0/1290015611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8af60 0x55f878c8b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf377fe640 1 --2- 172.21.15.2:0/2379617637 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcf200a1d40 0x7fcf200a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf36ffd640 1 -- 172.21.15.2:0/1290015611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c98bc0 msgr2=0x55f878c9afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf36ffd640 1 --2- 172.21.15.2:0/1290015611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c98bc0 0x55f878c9afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf36ffd640 1 -- 172.21.15.2:0/1290015611 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f878bb1570 con 0x55f878c8b880 2024-03-15T13:00:24.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf367fc640 1 --2- 172.21.15.2:0/1290015611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8af60 0x55f878c8b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:00:24.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf1ffff640 1 -- 172.21.15.2:0/2379617637 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fcf300f0980 con 0x7fcf381993e0 2024-03-15T13:00:24.278 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/EDID/1024x768.S 2024-03-15T13:00:24.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.290+0000 7fcf36ffd640 1 --2- 172.21.15.2:0/1290015611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8b880 0x55f878c98680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009cf0 tx=0x7fcf28011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e30bf21a3e5b048b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:24.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.291+0000 7fcf1d7fa640 1 -- 172.21.15.2:0/1290015611 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28009ea0 con 0x55f878c8b880 2024-03-15T13:00:24.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.291+0000 7fcf377fe640 1 --2- 172.21.15.2:0/2379617637 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcf200a1d40 0x7fcf200a4200 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcf300edd10 tx=0x7fcf300de040 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:24.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.291+0000 7fcf1d7fa640 1 -- 172.21.15.2:0/1290015611 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcf28004230 con 0x55f878c8b880 2024-03-15T13:00:24.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.291+0000 7fcf1d7fa640 1 -- 172.21.15.2:0/1290015611 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28019840 con 0x55f878c8b880 2024-03-15T13:00:24.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.291+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/1290015611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8b880 msgr2=0x55f878c98680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.291+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/1290015611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8b880 0x55f878c98680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009cf0 tx=0x7fcf28011810 comp rx=0 tx=0).stop 2024-03-15T13:00:24.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.291+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/1290015611 shutdown_connections 2024-03-15T13:00:24.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.291+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/1290015611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c98bc0 0x55f878c9afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.291+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/1290015611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8b880 0x55f878c98680 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.291+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/1290015611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8af60 0x55f878c8b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.291+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/1290015611 >> 172.21.15.2:0/1290015611 conn(0x55f878c85990 msgr2=0x55f878c85d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:24.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/1290015611 shutdown_connections 2024-03-15T13:00:24.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/1290015611 wait complete. 2024-03-15T13:00:24.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf3fe7c740 1 -- start start 2024-03-15T13:00:24.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf3fe7c740 1 Processor -- start 2024-03-15T13:00:24.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf3fe7c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c8af60 0x55f878c985c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf3fe7c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8b880 0x55f878c95950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf3fe7c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c98bc0 0x55f878c95f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf3fe7c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8788c6430 con 0x55f878c8af60 2024-03-15T13:00:24.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf3fe7c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f878882e00 con 0x55f878c8b880 2024-03-15T13:00:24.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf3fe7c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f878b77fd0 con 0x55f878c98bc0 2024-03-15T13:00:24.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf3fe7c740 10 client.0 _ll_register_callbacks cb 0x55f878bcdfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:24.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf367fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c8af60 0x55f878c985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8b880 0x55f878c95950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8b880 0x55f878c95950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:40112/0 (socket says 172.21.15.2:40112) 2024-03-15T13:00:24.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf367fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c8af60 0x55f878c985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54172/0 (socket says 172.21.15.2:54172) 2024-03-15T13:00:24.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf36ffd640 1 -- 172.21.15.2:0/3501842172 learned_addr learned my addr 172.21.15.2:0/3501842172 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:24.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.292+0000 7fcf367fc640 1 --2- 172.21.15.2:0/3501842172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c98bc0 0x55f878c95f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.293+0000 7fcf36ffd640 1 -- 172.21.15.2:0/3501842172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c98bc0 msgr2=0x55f878c95f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.293+0000 7fcf36ffd640 1 --2- 172.21.15.2:0/3501842172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c98bc0 0x55f878c95f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.293+0000 7fcf36ffd640 1 -- 172.21.15.2:0/3501842172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c8af60 msgr2=0x55f878c985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.293+0000 7fcf36ffd640 1 --2- 172.21.15.2:0/3501842172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c8af60 0x55f878c985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.293+0000 7fcf36ffd640 1 -- 172.21.15.2:0/3501842172 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf28009950 con 0x55f878c8b880 2024-03-15T13:00:24.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.293+0000 7fcf36ffd640 1 --2- 172.21.15.2:0/3501842172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8b880 0x55f878c95950 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcf28005730 tx=0x7fcf28005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1df7c134f17c58b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:24.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.293+0000 7fcf1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:00:24.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.293+0000 7fcf1aff5640 1 -- 172.21.15.2:0/3501842172 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28009ea0 con 0x55f878c8b880 2024-03-15T13:00:24.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.293+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/3501842172 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8788f27e0 con 0x55f878c8b880 2024-03-15T13:00:24.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.293+0000 7fcf1aff5640 1 -- 172.21.15.2:0/3501842172 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fcf280035f0 con 0x55f878c8b880 2024-03-15T13:00:24.284 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/EDID/1280x1024.S 2024-03-15T13:00:24.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.293+0000 7fcf1aff5640 1 -- 172.21.15.2:0/3501842172 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf280037b0 con 0x55f878c8b880 2024-03-15T13:00:24.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.293+0000 7fcf3fe7c740 10 client.26132 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:24.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.293+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/3501842172 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f878bf1610 con 0x55f878c8b880 2024-03-15T13:00:24.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.294+0000 7fcf1aff5640 1 -- 172.21.15.2:0/3501842172 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55f8788f27e0 con 0x55f878c8b880 2024-03-15T13:00:24.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.294+0000 7fcf1aff5640 1 -- 172.21.15.2:0/3501842172 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcf28021040 con 0x55f878c8b880 2024-03-15T13:00:24.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.294+0000 7fcf3fe7c740 10 client.26132 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:24.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.294+0000 7fcf3fe7c740 10 client.26132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:24.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.294+0000 7fcf3fe7c740 20 client.26132 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:24.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.294+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/3501842172 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f878ca8e30 0x55f878cab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.294+0000 7fcf3fe7c740 4 client.26132 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:24.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.294+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/3501842172 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f87892f720 con 0x55f878ca8e30 2024-03-15T13:00:24.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.294+0000 7fcf367fc640 1 --2- 172.21.15.2:0/3501842172 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f878ca8e30 0x55f878cab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.295+0000 7fcf1aff5640 10 client.26132 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:24.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.295+0000 7fcf367fc640 1 --2- 172.21.15.2:0/3501842172 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f878ca8e30 0x55f878cab250 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df9fde3df2f36dcb server_cookie=c2e6a3cc5c7f922 in_seq=0 out_seq=0 2024-03-15T13:00:24.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.296+0000 7fcf1aff5640 1 -- 172.21.15.2:0/3501842172 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f87892f720 con 0x55f878ca8e30 2024-03-15T13:00:24.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.296+0000 7fcf1aff5640 10 client.26132 handle_command_reply: tid=0 2024-03-15T13:00:24.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.296+0000 7fcf3fe7c740 1 client.26132 shutdown 2024-03-15T13:00:24.287 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/EDID/1600x1200.S 2024-03-15T13:00:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.296+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/3501842172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8b880 msgr2=0x55f878c95950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.296+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/3501842172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8b880 0x55f878c95950 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcf28005730 tx=0x7fcf28005760 comp rx=0 tx=0).stop 2024-03-15T13:00:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.296+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/3501842172 shutdown_connections 2024-03-15T13:00:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.296+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/3501842172 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f878ca8e30 0x55f878cab250 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.296+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/3501842172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c98bc0 0x55f878c95f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.296+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/3501842172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8b880 0x55f878c95950 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.296+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/3501842172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c8af60 0x55f878c985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.296+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/3501842172 >> 172.21.15.2:0/3501842172 conn(0x55f878c85990 msgr2=0x55f878c87730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.296+0000 7fcf1aff5640 0 client.26132 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.297+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/3501842172 shutdown_connections 2024-03-15T13:00:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.297+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/3501842172 wait complete. 2024-03-15T13:00:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.297+0000 7fcf3fe7c740 20 client.26132 trim_cache size 0 max 16384 2024-03-15T13:00:24.289 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/EDID/1680x1050.S 2024-03-15T13:00:24.289 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/EDID/1920x1080.S 2024-03-15T13:00:24.289 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/EDID/800x600.S 2024-03-15T13:00:24.291 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/EDID/HOWTO.txt 2024-03-15T13:00:24.293 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/EDID/Makefile 2024-03-15T13:00:24.296 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/EDID/edid.S 2024-03-15T13:00:24.298 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/EDID/hex 2024-03-15T13:00:24.301 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/HOWTO 2024-03-15T13:00:24.305 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/IPMI.txt 2024-03-15T13:00:24.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.320+0000 7fcf3fe7c740 1 Processor -- start 2024-03-15T13:00:24.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.320+0000 7fcf3fe7c740 1 -- start start 2024-03-15T13:00:24.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.320+0000 7fcf3fe7c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8af60 0x55f878c98080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.320+0000 7fcf3fe7c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8b880 0x55f878c985c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.320+0000 7fcf3fe7c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c98bc0 0x55f878ca4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.320+0000 7fcf3fe7c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f878c93f70 con 0x55f878c98bc0 2024-03-15T13:00:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.320+0000 7fcf3fe7c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f878c8b6a0 con 0x55f878c8af60 2024-03-15T13:00:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.320+0000 7fcf3fe7c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f878c913b0 con 0x55f878c8b880 2024-03-15T13:00:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.320+0000 7fcf367fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c98bc0 0x55f878ca4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.320+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8b880 0x55f878c985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.320+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8b880 0x55f878c985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37174/0 (socket says 172.21.15.2:37174) 2024-03-15T13:00:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.320+0000 7fcf367fc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8af60 0x55f878c98080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.320+0000 7fcf36ffd640 1 -- 172.21.15.2:0/2826378678 learned_addr learned my addr 172.21.15.2:0/2826378678 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.321+0000 7fcf36ffd640 1 -- 172.21.15.2:0/2826378678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8af60 msgr2=0x55f878c98080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.321+0000 7fcf36ffd640 1 --2- 172.21.15.2:0/2826378678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8af60 0x55f878c98080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.321+0000 7fcf36ffd640 1 -- 172.21.15.2:0/2826378678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c98bc0 msgr2=0x55f878ca4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.321+0000 7fcf36ffd640 1 --2- 172.21.15.2:0/2826378678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c98bc0 0x55f878ca4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.321+0000 7fcf36ffd640 1 -- 172.21.15.2:0/2826378678 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f878b8dac0 con 0x55f878c8b880 2024-03-15T13:00:24.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.321+0000 7fcf36ffd640 1 --2- 172.21.15.2:0/2826378678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8b880 0x55f878c985c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf28011850 tx=0x7fcf28005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2352af2785887f73 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:24.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.321+0000 7fcf1cff9640 1 -- 172.21.15.2:0/2826378678 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28009ea0 con 0x55f878c8b880 2024-03-15T13:00:24.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.321+0000 7fcf1cff9640 1 -- 172.21.15.2:0/2826378678 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcf28019420 con 0x55f878c8b880 2024-03-15T13:00:24.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.321+0000 7fcf1cff9640 1 -- 172.21.15.2:0/2826378678 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28003d20 con 0x55f878c8b880 2024-03-15T13:00:24.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/2826378678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8b880 msgr2=0x55f878c985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/2826378678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8b880 0x55f878c985c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf28011850 tx=0x7fcf28005b90 comp rx=0 tx=0).stop 2024-03-15T13:00:24.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/2826378678 shutdown_connections 2024-03-15T13:00:24.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/2826378678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c98bc0 0x55f878ca4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/2826378678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8b880 0x55f878c985c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/2826378678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c8af60 0x55f878c98080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/2826378678 >> 172.21.15.2:0/2826378678 conn(0x55f878c85990 msgr2=0x55f878c90ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:24.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/2826378678 shutdown_connections 2024-03-15T13:00:24.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/2826378678 wait complete. 2024-03-15T13:00:24.310 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/IRQ-affinity.txt 2024-03-15T13:00:24.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 -- start start 2024-03-15T13:00:24.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 Processor -- start 2024-03-15T13:00:24.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8af60 0x55f878c9a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c8b880 0x55f878c98fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c98bc0 0x55f878c994e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f878c99a20 con 0x55f878c8b880 2024-03-15T13:00:24.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f878c99b90 con 0x55f878c98bc0 2024-03-15T13:00:24.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f878c99d50 con 0x55f878c8af60 2024-03-15T13:00:24.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf3fe7c740 10 client.0 _ll_register_callbacks cb 0x7fcf2c0097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:24.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c8b880 0x55f878c98fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c8b880 0x55f878c98fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54196/0 (socket says 172.21.15.2:54196) 2024-03-15T13:00:24.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf36ffd640 1 -- 172.21.15.2:0/743297394 learned_addr learned my addr 172.21.15.2:0/743297394 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:24.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.323+0000 7fcf367fc640 1 --2- 172.21.15.2:0/743297394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8af60 0x55f878c9a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.324+0000 7fcf367fc640 1 --2- 172.21.15.2:0/743297394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c98bc0 0x55f878c994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.324+0000 7fcf36ffd640 1 -- 172.21.15.2:0/743297394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8af60 msgr2=0x55f878c9a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.324+0000 7fcf36ffd640 1 --2- 172.21.15.2:0/743297394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8af60 0x55f878c9a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.324+0000 7fcf36ffd640 1 -- 172.21.15.2:0/743297394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c98bc0 msgr2=0x55f878c994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.324+0000 7fcf36ffd640 1 --2- 172.21.15.2:0/743297394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c98bc0 0x55f878c994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.324+0000 7fcf36ffd640 1 -- 172.21.15.2:0/743297394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f878bf1610 con 0x55f878c8b880 2024-03-15T13:00:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.324+0000 7fcf367fc640 1 --2- 172.21.15.2:0/743297394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8af60 0x55f878c9a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.324+0000 7fcf36ffd640 1 --2- 172.21.15.2:0/743297394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c8b880 0x55f878c98fa0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcf2802e040 tx=0x7fcf28019c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1667790cecdafd64 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.324+0000 7fcf1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.324+0000 7fcf1aff5640 1 -- 172.21.15.2:0/743297394 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28043070 con 0x55f878c8b880 2024-03-15T13:00:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.324+0000 7fcf1aff5640 1 -- 172.21.15.2:0/743297394 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fcf28000e40 con 0x55f878c8b880 2024-03-15T13:00:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.324+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/743297394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf28004230 con 0x55f878c8b880 2024-03-15T13:00:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.324+0000 7fcf1aff5640 1 -- 172.21.15.2:0/743297394 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf28009ea0 con 0x55f878c8b880 2024-03-15T13:00:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.324+0000 7fcf3fe7c740 10 client.16275 fetch_fsmap learned FSMap version 29 2024-03-15T13:00:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.324+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/743297394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f878bb36a0 con 0x55f878c8b880 2024-03-15T13:00:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.325+0000 7fcf1aff5640 1 -- 172.21.15.2:0/743297394 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcf28004230 con 0x55f878c8b880 2024-03-15T13:00:24.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.325+0000 7fcf1aff5640 1 -- 172.21.15.2:0/743297394 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fcf28046020 con 0x55f878c8b880 2024-03-15T13:00:24.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.325+0000 7fcf3fe7c740 10 client.16275 fetch_fsmap finished waiting for FSMap version 29 2024-03-15T13:00:24.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.325+0000 7fcf3fe7c740 10 client.16275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:24.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.325+0000 7fcf3fe7c740 20 client.16275 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:24.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.325+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/743297394 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f878d04740 0x55f878d06b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:24.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.325+0000 7fcf3fe7c740 4 client.16275 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:00:24.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.325+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/743297394 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f87897e890 con 0x55f878d04740 2024-03-15T13:00:24.315 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/IRQ-domain.txt 2024-03-15T13:00:24.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.325+0000 7fcf367fc640 1 --2- 172.21.15.2:0/743297394 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f878d04740 0x55f878d06b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:24.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.326+0000 7fcf367fc640 1 --2- 172.21.15.2:0/743297394 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f878d04740 0x55f878d06b60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1f606e176a6777b server_cookie=9d88542ccc7dc924 in_seq=0 out_seq=0 2024-03-15T13:00:24.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.326+0000 7fcf1aff5640 10 client.16275 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:24.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.326+0000 7fcf1aff5640 1 -- 172.21.15.2:0/743297394 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f87897e890 con 0x55f878d04740 2024-03-15T13:00:24.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.326+0000 7fcf1aff5640 10 client.16275 handle_command_reply: tid=0 2024-03-15T13:00:24.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.326+0000 7fcf3fe7c740 1 client.16275 shutdown 2024-03-15T13:00:24.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.326+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/743297394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c8b880 msgr2=0x55f878c98fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.326+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/743297394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c8b880 0x55f878c98fa0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcf2802e040 tx=0x7fcf28019c70 comp rx=0 tx=0).stop 2024-03-15T13:00:24.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.326+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/743297394 shutdown_connections 2024-03-15T13:00:24.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.326+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/743297394 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f878d04740 0x55f878d06b60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.326+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/743297394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f878c98bc0 0x55f878c994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.326+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/743297394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f878c8b880 0x55f878c98fa0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.326+0000 7fcf3fe7c740 1 --2- 172.21.15.2:0/743297394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f878c8af60 0x55f878c9a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.327+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/743297394 >> 172.21.15.2:0/743297394 conn(0x55f878c85990 msgr2=0x55f878c94150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:24.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.327+0000 7fcf1aff5640 0 client.16275 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:24.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.327+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/743297394 shutdown_connections 2024-03-15T13:00:24.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.327+0000 7fcf3fe7c740 1 -- 172.21.15.2:0/743297394 wait complete. 2024-03-15T13:00:24.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.327+0000 7fcf3fe7c740 20 client.16275 trim_cache size 0 max 16384 2024-03-15T13:00:24.319 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:00:24.319 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T13:00:24.319 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:00:24.319 INFO:teuthology.orchestra.run.smithi002.stdout: "50af8c8b-51b9-47a3-8bd4-fbfc4fb70a8e": { 2024-03-15T13:00:24.319 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:00:24.319 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "50af8c8b-51b9-47a3-8bd4-fbfc4fb70a8e", 2024-03-15T13:00:24.319 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:00:24.319 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:00:24.319 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:00:24.319 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:00:24.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.327+0000 7fcf3d29a640 1 -- 172.21.15.2:0/2379617637 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcf200a1d40 msgr2=0x7fcf200a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.327+0000 7fcf3d29a640 1 --2- 172.21.15.2:0/2379617637 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcf200a1d40 0x7fcf200a4200 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcf300edd10 tx=0x7fcf300de040 comp rx=0 tx=0).stop 2024-03-15T13:00:24.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.327+0000 7fcf3d29a640 1 -- 172.21.15.2:0/2379617637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf381993e0 msgr2=0x7fcf381a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:24.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.327+0000 7fcf3d29a640 1 --2- 172.21.15.2:0/2379617637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf381993e0 0x7fcf381a39e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcf30005cc0 tx=0x7fcf30034a80 comp rx=0 tx=0).stop 2024-03-15T13:00:24.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.328+0000 7fcf3d29a640 1 -- 172.21.15.2:0/2379617637 shutdown_connections 2024-03-15T13:00:24.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.328+0000 7fcf3d29a640 1 --2- 172.21.15.2:0/2379617637 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcf200a1d40 0x7fcf200a4200 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.328+0000 7fcf3d29a640 1 --2- 172.21.15.2:0/2379617637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf381993e0 0x7fcf381a39e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.328+0000 7fcf3d29a640 1 --2- 172.21.15.2:0/2379617637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf381056b0 0x7fcf38198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.328+0000 7fcf3d29a640 1 --2- 172.21.15.2:0/2379617637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf381005e0 0x7fcf38198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:24.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.328+0000 7fcf3d29a640 1 -- 172.21.15.2:0/2379617637 >> 172.21.15.2:0/2379617637 conn(0x7fcf380775a0 msgr2=0x7fcf380fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:24.321 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/IRQ.txt 2024-03-15T13:00:24.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.328+0000 7fcf3d29a640 1 -- 172.21.15.2:0/2379617637 shutdown_connections 2024-03-15T13:00:24.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:24.328+0000 7fcf3d29a640 1 -- 172.21.15.2:0/2379617637 wait complete. 2024-03-15T13:00:24.322 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/Intel-IOMMU.txt 2024-03-15T13:00:24.322 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/Makefile 2024-03-15T13:00:24.322 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ManagementStyle 2024-03-15T13:00:24.325 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/PCI/ 2024-03-15T13:00:24.326 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/PCI/00-INDEX 2024-03-15T13:00:24.327 INFO:tasks.cephfs.filesystem:scrub status for tag:50af8c8b-51b9-47a3-8bd4-fbfc4fb70a8e - {'path': '/', 'tag': '50af8c8b-51b9-47a3-8bd4-fbfc4fb70a8e', 'options': 'recursive,force'} 2024-03-15T13:00:24.328 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/PCI/MSI-HOWTO.txt 2024-03-15T13:00:24.331 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/PCI/PCIEBUS-HOWTO.txt 2024-03-15T13:00:24.333 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/PCI/pci-error-recovery.txt 2024-03-15T13:00:24.336 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/PCI/pci-iov-howto.txt 2024-03-15T13:00:24.338 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/PCI/pci.txt 2024-03-15T13:00:24.341 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/PCI/pcieaer-howto.txt 2024-03-15T13:00:24.343 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/ 2024-03-15T13:00:24.347 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/00-INDEX 2024-03-15T13:00:24.348 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/NMI-RCU.txt 2024-03-15T13:00:24.351 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/RTFP.txt 2024-03-15T13:00:24.355 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/UP.txt 2024-03-15T13:00:24.357 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/arrayRCU.txt 2024-03-15T13:00:24.359 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/checklist.txt 2024-03-15T13:00:24.361 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/listRCU.txt 2024-03-15T13:00:24.364 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/lockdep-splat.txt 2024-03-15T13:00:24.366 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/lockdep.txt 2024-03-15T13:00:24.368 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/rcu.txt 2024-03-15T13:00:24.371 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/rcu_dereference.txt 2024-03-15T13:00:24.374 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/rcubarrier.txt 2024-03-15T13:00:24.376 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/rculist_nulls.txt 2024-03-15T13:00:24.379 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/rcuref.txt 2024-03-15T13:00:24.381 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/stallwarn.txt 2024-03-15T13:00:24.383 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/torture.txt 2024-03-15T13:00:24.386 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/trace.txt 2024-03-15T13:00:24.389 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/RCU/whatisRCU.txt 2024-03-15T13:00:24.392 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/SAK.txt 2024-03-15T13:00:24.395 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/SM501.txt 2024-03-15T13:00:24.397 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/SecurityBugs 2024-03-15T13:00:24.400 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/SubmitChecklist 2024-03-15T13:00:24.402 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/SubmittingDrivers 2024-03-15T13:00:24.405 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/SubmittingPatches 2024-03-15T13:00:24.408 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/VGA-softcursor.txt 2024-03-15T13:00:24.411 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/accounting/ 2024-03-15T13:00:24.414 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/accounting/.gitignore 2024-03-15T13:00:24.416 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/accounting/Makefile 2024-03-15T13:00:24.418 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/accounting/cgroupstats.txt 2024-03-15T13:00:24.420 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/accounting/delay-accounting.txt 2024-03-15T13:00:24.422 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/accounting/getdelays.c 2024-03-15T13:00:24.425 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/accounting/taskstats-struct.txt 2024-03-15T13:00:24.428 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/accounting/taskstats.txt 2024-03-15T13:00:24.430 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/acpi/ 2024-03-15T13:00:24.432 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/acpi/apei/ 2024-03-15T13:00:24.434 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/acpi/apei/einj.txt 2024-03-15T13:00:24.437 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/acpi/apei/output_format.txt 2024-03-15T13:00:24.439 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/acpi/debug.txt 2024-03-15T13:00:24.442 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/acpi/dsdt-override.txt 2024-03-15T13:00:24.444 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/acpi/enumeration.txt 2024-03-15T13:00:24.447 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/acpi/gpio-properties.txt 2024-03-15T13:00:24.449 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/acpi/initrd_table_override.txt 2024-03-15T13:00:24.451 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/acpi/method-customizing.txt 2024-03-15T13:00:24.454 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/acpi/method-tracing.txt 2024-03-15T13:00:24.456 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/acpi/namespace.txt 2024-03-15T13:00:24.459 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/acpi/scan_handlers.txt 2024-03-15T13:00:24.461 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/acpi/video_extension.txt 2024-03-15T13:00:24.463 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/aoe/ 2024-03-15T13:00:24.466 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/aoe/aoe.txt 2024-03-15T13:00:24.469 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/aoe/autoload.sh 2024-03-15T13:00:24.473 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/aoe/status.sh 2024-03-15T13:00:24.475 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/aoe/todo.txt 2024-03-15T13:00:24.477 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/aoe/udev-install.sh 2024-03-15T13:00:24.479 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/aoe/udev.txt 2024-03-15T13:00:24.481 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/applying-patches.txt 2024-03-15T13:00:24.484 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/ 2024-03-15T13:00:24.486 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/00-INDEX 2024-03-15T13:00:24.488 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Atmel/ 2024-03-15T13:00:24.490 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Atmel/README 2024-03-15T13:00:24.493 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Booting 2024-03-15T13:00:24.496 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/CCN.txt 2024-03-15T13:00:24.498 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/IXP4xx 2024-03-15T13:00:24.500 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Interrupts 2024-03-15T13:00:24.503 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Makefile 2024-03-15T13:00:24.505 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Marvell/ 2024-03-15T13:00:24.507 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Marvell/README 2024-03-15T13:00:24.509 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Netwinder 2024-03-15T13:00:24.512 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/OMAP/ 2024-03-15T13:00:24.514 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/OMAP/DSS 2024-03-15T13:00:24.517 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/OMAP/omap_pm 2024-03-15T13:00:24.520 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Porting 2024-03-15T13:00:24.522 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/README 2024-03-15T13:00:24.525 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/ 2024-03-15T13:00:24.526 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/ADSBitsy 2024-03-15T13:00:24.529 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/Assabet 2024-03-15T13:00:24.533 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/Brutus 2024-03-15T13:00:24.533 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/CERF 2024-03-15T13:00:24.536 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/FreeBird 2024-03-15T13:00:24.538 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsClient 2024-03-15T13:00:24.540 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsMaster 2024-03-15T13:00:24.543 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/HUW_WEBPANEL 2024-03-15T13:00:24.545 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/Itsy 2024-03-15T13:00:24.548 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/LART 2024-03-15T13:00:24.550 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/PLEB 2024-03-15T13:00:24.552 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/Pangolin 2024-03-15T13:00:24.554 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/Tifon 2024-03-15T13:00:24.557 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/Victor 2024-03-15T13:00:24.559 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/Yopy 2024-03-15T13:00:24.561 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/empeg 2024-03-15T13:00:24.563 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/nanoEngine 2024-03-15T13:00:24.566 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SA1100/serial_UART 2024-03-15T13:00:24.568 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/ 2024-03-15T13:00:24.570 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/.gitignore 2024-03-15T13:00:24.572 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/Makefile 2024-03-15T13:00:24.574 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/vrl4.c 2024-03-15T13:00:24.576 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-mmcif.txt 2024-03-15T13:00:24.579 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-sdhi.txt 2024-03-15T13:00:24.581 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SPEAr/ 2024-03-15T13:00:24.584 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/SPEAr/overview.txt 2024-03-15T13:00:24.586 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/ 2024-03-15T13:00:24.589 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/CPUfreq.txt 2024-03-15T13:00:24.592 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/EB2410ITX.txt 2024-03-15T13:00:24.594 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/GPIO.txt 2024-03-15T13:00:24.596 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/H1940.txt 2024-03-15T13:00:24.598 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/NAND.txt 2024-03-15T13:00:24.603 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Overview.txt 2024-03-15T13:00:24.605 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2412.txt 2024-03-15T13:00:24.608 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2413.txt 2024-03-15T13:00:24.610 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/SMDK2440.txt 2024-03-15T13:00:24.612 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Suspend.txt 2024-03-15T13:00:24.614 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/USB-Host.txt 2024-03-15T13:00:24.617 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung/ 2024-03-15T13:00:24.619 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung/GPIO.txt 2024-03-15T13:00:24.622 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung/Overview.txt 2024-03-15T13:00:24.624 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Samsung/clksrc-change-registers.awk 2024-03-15T13:00:24.626 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/Setup 2024-03-15T13:00:24.629 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/VFP/ 2024-03-15T13:00:24.631 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/VFP/release-notes.txt 2024-03-15T13:00:24.634 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/cluster-pm-race-avoidance.txt 2024-03-15T13:00:24.637 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/firmware.txt 2024-03-15T13:00:24.639 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/kernel_mode_neon.txt 2024-03-15T13:00:24.641 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/kernel_user_helpers.txt 2024-03-15T13:00:24.644 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/mem_alignment 2024-03-15T13:00:24.646 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/memory.txt 2024-03-15T13:00:24.648 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/msm/ 2024-03-15T13:00:24.650 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/msm/gpiomux.txt 2024-03-15T13:00:24.653 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/nwfpe/ 2024-03-15T13:00:24.656 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/nwfpe/NOTES 2024-03-15T13:00:24.658 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/nwfpe/README 2024-03-15T13:00:24.660 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/nwfpe/README.FPE 2024-03-15T13:00:24.663 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/nwfpe/TODO 2024-03-15T13:00:24.665 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/pxa/ 2024-03-15T13:00:24.668 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/pxa/mfp.txt 2024-03-15T13:00:24.670 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/sti/ 2024-03-15T13:00:24.673 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/sti/overview.txt 2024-03-15T13:00:24.676 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/sti/stih407-overview.txt 2024-03-15T13:00:24.678 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/sti/stih415-overview.txt 2024-03-15T13:00:24.680 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/sti/stih416-overview.txt 2024-03-15T13:00:24.682 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/sti/stih418-overview.txt 2024-03-15T13:00:24.685 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/sunxi/ 2024-03-15T13:00:24.688 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/sunxi/README 2024-03-15T13:00:24.690 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/sunxi/clocks.txt 2024-03-15T13:00:24.693 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/swp_emulation 2024-03-15T13:00:24.695 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/tcm.txt 2024-03-15T13:00:24.698 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/uefi.txt 2024-03-15T13:00:24.700 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm/vlocks.txt 2024-03-15T13:00:24.702 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm64/ 2024-03-15T13:00:24.706 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm64/booting.txt 2024-03-15T13:00:24.708 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm64/legacy_instructions.txt 2024-03-15T13:00:24.710 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm64/memory.txt 2024-03-15T13:00:24.712 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/arm64/tagged-pointers.txt 2024-03-15T13:00:24.715 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/assoc_array.txt 2024-03-15T13:00:24.718 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/atomic_ops.txt 2024-03-15T13:00:24.721 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/auxdisplay/ 2024-03-15T13:00:24.723 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/auxdisplay/.gitignore 2024-03-15T13:00:24.726 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/auxdisplay/Makefile 2024-03-15T13:00:24.728 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b 2024-03-15T13:00:24.731 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b-example.c 2024-03-15T13:00:24.734 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/auxdisplay/ks0108 2024-03-15T13:00:24.738 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/backlight/ 2024-03-15T13:00:24.743 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/backlight/lp855x-driver.txt 2024-03-15T13:00:24.753 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/bad_memory.txt 2024-03-15T13:00:24.755 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/basic_profiling.txt 2024-03-15T13:00:24.758 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/bcache.txt 2024-03-15T13:00:24.761 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/binfmt_misc.txt 2024-03-15T13:00:24.763 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blackfin/ 2024-03-15T13:00:24.766 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blackfin/00-INDEX 2024-03-15T13:00:24.769 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blackfin/Makefile 2024-03-15T13:00:24.772 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blackfin/bfin-gpio-notes.txt 2024-03-15T13:00:24.775 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blackfin/bfin-spi-notes.txt 2024-03-15T13:00:24.777 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blackfin/gptimers-example.c 2024-03-15T13:00:24.780 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/ 2024-03-15T13:00:24.783 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/00-INDEX 2024-03-15T13:00:24.786 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/biodoc.txt 2024-03-15T13:00:24.790 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/biovecs.txt 2024-03-15T13:00:24.793 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/capability.txt 2024-03-15T13:00:24.796 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/cfq-iosched.txt 2024-03-15T13:00:24.799 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/cmdline-partition.txt 2024-03-15T13:00:24.814 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/data-integrity.txt 2024-03-15T13:00:25.066 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/deadline-iosched.txt 2024-03-15T13:00:25.068 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/ioprio.txt 2024-03-15T13:00:25.071 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/null_blk.txt 2024-03-15T13:00:25.074 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/queue-sysfs.txt 2024-03-15T13:00:25.076 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/request.txt 2024-03-15T13:00:25.078 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/stat.txt 2024-03-15T13:00:25.081 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/switching-sched.txt 2024-03-15T13:00:25.084 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/block/writeback_cache_control.txt 2024-03-15T13:00:25.086 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/ 2024-03-15T13:00:25.089 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/00-INDEX 2024-03-15T13:00:25.092 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/README.DAC960 2024-03-15T13:00:25.095 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/cciss.txt 2024-03-15T13:00:25.098 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/cpqarray.txt 2024-03-15T13:00:25.101 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/drbd/ 2024-03-15T13:00:25.103 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-8.3-data-packets.svg 2024-03-15T13:00:25.107 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-data-packets.svg 2024-03-15T13:00:25.110 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/drbd/README.txt 2024-03-15T13:00:25.112 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/drbd/conn-states-8.dot 2024-03-15T13:00:25.115 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/drbd/data-structure-v9.txt 2024-03-15T13:00:25.117 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/drbd/disk-states-8.dot 2024-03-15T13:00:25.120 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/drbd/drbd-connection-state-overview.dot 2024-03-15T13:00:25.123 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/drbd/node-states-8.dot 2024-03-15T13:00:25.125 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/floppy.txt 2024-03-15T13:00:25.128 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/mflash.txt 2024-03-15T13:00:25.135 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/nbd.txt 2024-03-15T13:00:25.137 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/paride.txt 2024-03-15T13:00:25.140 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/ramdisk.txt 2024-03-15T13:00:25.142 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/blockdev/zram.txt 2024-03-15T13:00:25.145 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/braille-console.txt 2024-03-15T13:00:25.148 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/bt8xxgpio.txt 2024-03-15T13:00:25.150 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/btmrvl.txt 2024-03-15T13:00:25.153 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/bus-devices/ 2024-03-15T13:00:25.156 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt 2024-03-15T13:00:25.159 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/bus-virt-phys-mapping.txt 2024-03-15T13:00:25.163 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cachetlb.txt 2024-03-15T13:00:25.166 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cdrom/ 2024-03-15T13:00:25.168 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cdrom/00-INDEX 2024-03-15T13:00:25.170 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cdrom/Makefile 2024-03-15T13:00:25.173 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cdrom/cdrom-standard.tex 2024-03-15T13:00:25.178 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cdrom/ide-cd 2024-03-15T13:00:25.202 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cdrom/packet-writing.txt 2024-03-15T13:00:25.204 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cgroups/ 2024-03-15T13:00:25.207 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cgroups/00-INDEX 2024-03-15T13:00:25.209 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cgroups/blkio-controller.txt 2024-03-15T13:00:25.212 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cgroups/cgroups.txt 2024-03-15T13:00:25.215 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cgroups/cpuacct.txt 2024-03-15T13:00:25.217 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cgroups/cpusets.txt 2024-03-15T13:00:25.222 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cgroups/devices.txt 2024-03-15T13:00:25.224 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cgroups/freezer-subsystem.txt 2024-03-15T13:00:25.226 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cgroups/hugetlb.txt 2024-03-15T13:00:25.228 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cgroups/memcg_test.txt 2024-03-15T13:00:25.231 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cgroups/memory.txt 2024-03-15T13:00:25.234 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cgroups/net_cls.txt 2024-03-15T13:00:25.236 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cgroups/net_prio.txt 2024-03-15T13:00:25.239 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cgroups/unified-hierarchy.txt 2024-03-15T13:00:25.241 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/circular-buffers.txt 2024-03-15T13:00:25.244 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/clk.txt 2024-03-15T13:00:25.246 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/coccinelle.txt 2024-03-15T13:00:25.249 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/connector/ 2024-03-15T13:00:25.251 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/connector/.gitignore 2024-03-15T13:00:25.253 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/connector/Makefile 2024-03-15T13:00:25.255 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/connector/cn_test.c 2024-03-15T13:00:25.258 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/connector/connector.txt 2024-03-15T13:00:25.260 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/connector/ucon.c 2024-03-15T13:00:25.262 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/console/ 2024-03-15T13:00:25.265 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/console/console.txt 2024-03-15T13:00:25.267 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpu-freq/ 2024-03-15T13:00:25.269 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpu-freq/amd-powernow.txt 2024-03-15T13:00:25.272 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpu-freq/boost.txt 2024-03-15T13:00:25.274 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpu-freq/core.txt 2024-03-15T13:00:25.277 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpu-freq/cpu-drivers.txt 2024-03-15T13:00:25.279 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-nforce2.txt 2024-03-15T13:00:25.281 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-stats.txt 2024-03-15T13:00:25.284 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpu-freq/governors.txt 2024-03-15T13:00:25.286 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpu-freq/index.txt 2024-03-15T13:00:25.289 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpu-freq/intel-pstate.txt 2024-03-15T13:00:25.291 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpu-freq/pcc-cpufreq.txt 2024-03-15T13:00:25.293 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpu-freq/user-guide.txt 2024-03-15T13:00:25.295 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpu-hotplug.txt 2024-03-15T13:00:25.299 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpu-load.txt 2024-03-15T13:00:25.301 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpuidle/ 2024-03-15T13:00:25.304 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpuidle/core.txt 2024-03-15T13:00:25.306 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpuidle/driver.txt 2024-03-15T13:00:25.307 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpuidle/governor.txt 2024-03-15T13:00:25.309 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cpuidle/sysfs.txt 2024-03-15T13:00:25.312 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cputopology.txt 2024-03-15T13:00:25.314 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/crc32.txt 2024-03-15T13:00:25.317 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cris/ 2024-03-15T13:00:25.319 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/cris/README 2024-03-15T13:00:25.321 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/crypto/ 2024-03-15T13:00:25.324 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/crypto/api-intro.txt 2024-03-15T13:00:25.326 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/crypto/asymmetric-keys.txt 2024-03-15T13:00:25.328 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/crypto/async-tx-api.txt 2024-03-15T13:00:25.331 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/crypto/crypto-API-userspace.txt 2024-03-15T13:00:25.333 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/crypto/descore-readme.txt 2024-03-15T13:00:25.335 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dcdbas.txt 2024-03-15T13:00:25.338 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/debugging-modules.txt 2024-03-15T13:00:25.340 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/debugging-via-ohci1394.txt 2024-03-15T13:00:25.343 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dell_rbu.txt 2024-03-15T13:00:25.345 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/development-process/ 2024-03-15T13:00:25.347 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/development-process/1.Intro 2024-03-15T13:00:25.350 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/development-process/2.Process 2024-03-15T13:00:25.353 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/development-process/3.Early-stage 2024-03-15T13:00:25.356 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/development-process/4.Coding 2024-03-15T13:00:25.358 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/development-process/5.Posting 2024-03-15T13:00:25.361 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/development-process/6.Followthrough 2024-03-15T13:00:25.363 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/development-process/7.AdvancedTopics 2024-03-15T13:00:25.365 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/development-process/8.Conclusion 2024-03-15T13:00:25.367 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/ 2024-03-15T13:00:25.370 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/cache-policies.txt 2024-03-15T13:00:25.372 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/cache.txt 2024-03-15T13:00:25.375 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/delay.txt 2024-03-15T13:00:25.377 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/dm-crypt.txt 2024-03-15T13:00:25.379 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/dm-flakey.txt 2024-03-15T13:00:25.381 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/dm-io.txt 2024-03-15T13:00:25.383 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/dm-log.txt 2024-03-15T13:00:25.385 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/dm-queue-length.txt 2024-03-15T13:00:25.388 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/dm-raid.txt 2024-03-15T13:00:25.390 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/dm-service-time.txt 2024-03-15T13:00:25.392 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/dm-uevent.txt 2024-03-15T13:00:25.394 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/era.txt 2024-03-15T13:00:25.396 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/kcopyd.txt 2024-03-15T13:00:25.398 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/linear.txt 2024-03-15T13:00:25.403 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/persistent-data.txt 2024-03-15T13:00:25.405 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/snapshot.txt 2024-03-15T13:00:25.407 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/statistics.txt 2024-03-15T13:00:25.409 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/striped.txt 2024-03-15T13:00:25.412 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/switch.txt 2024-03-15T13:00:25.415 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/thin-provisioning.txt 2024-03-15T13:00:25.417 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/verity.txt 2024-03-15T13:00:25.419 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/device-mapper/zero.txt 2024-03-15T13:00:25.422 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devices.txt 2024-03-15T13:00:25.428 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/ 2024-03-15T13:00:25.430 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/00-INDEX 2024-03-15T13:00:25.433 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ 2024-03-15T13:00:25.435 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ABI.txt 2024-03-15T13:00:25.438 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/ 2024-03-15T13:00:25.440 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/interrupts.txt 2024-03-15T13:00:25.442 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/pmu.txt 2024-03-15T13:00:25.444 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ 2024-03-15T13:00:25.447 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/adapteva.txt 2024-03-15T13:00:25.450 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/ 2024-03-15T13:00:25.452 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-clk-manager.txt 2024-03-15T13:00:25.455 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-sdram-edac.txt 2024-03-15T13:00:25.457 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-system.txt 2024-03-15T13:00:25.460 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/amlogic.txt 2024-03-15T13:00:25.462 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arch_timer.txt 2024-03-15T13:00:25.465 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arm-boards 2024-03-15T13:00:25.468 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt 2024-03-15T13:00:25.470 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp.txt 2024-03-15T13:00:25.472 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-375.txt 2024-03-15T13:00:25.475 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-380-mpcore-soc-ctrl.txt 2024-03-15T13:00:25.477 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-38x.txt 2024-03-15T13:00:25.480 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-cpu-reset.txt 2024-03-15T13:00:25.482 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armadeus.txt 2024-03-15T13:00:25.484 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-at91.txt 2024-03-15T13:00:25.486 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-pmc.txt 2024-03-15T13:00:25.489 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/axxia.txt 2024-03-15T13:00:25.491 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/ 2024-03-15T13:00:25.493 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm11351.txt 2024-03-15T13:00:25.496 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm21664.txt 2024-03-15T13:00:25.498 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm63138.txt 2024-03-15T13:00:25.501 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/brcm,bcm11351-cpu-method 2024-03-15T13:00:25.504 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/cygnus.txt 2024-03-15T13:00:25.506 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-resetmgr.txt 2024-03-15T13:00:25.508 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-timer.txt 2024-03-15T13:00:25.510 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-wdt.txt 2024-03-15T13:00:25.513 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm2835.txt 2024-03-15T13:00:25.516 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm4708.txt 2024-03-15T13:00:25.518 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/brcm-brcmstb.txt 2024-03-15T13:00:25.521 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda.txt 2024-03-15T13:00:25.523 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/ 2024-03-15T13:00:25.525 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/combophy.txt 2024-03-15T13:00:25.528 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/l2ecc.txt 2024-03-15T13:00:25.530 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/mem-ctrlr.txt 2024-03-15T13:00:25.532 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cavium-thunder.txt 2024-03-15T13:00:25.536 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cci.txt 2024-03-15T13:00:25.538 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ccn.txt 2024-03-15T13:00:25.540 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coherency-fabric.txt 2024-03-15T13:00:25.543 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coresight.txt 2024-03-15T13:00:25.545 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/ 2024-03-15T13:00:25.547 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/marvell,berlin-smp 2024-03-15T13:00:25.550 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpus.txt 2024-03-15T13:00:25.553 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci.txt 2024-03-15T13:00:25.555 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/ 2024-03-15T13:00:25.558 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/cp-intc.txt 2024-03-15T13:00:25.561 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/digicolor.txt 2024-03-15T13:00:25.563 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/ 2024-03-15T13:00:25.565 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/power_domain.txt 2024-03-15T13:00:25.568 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/smp-sysram.txt 2024-03-15T13:00:25.570 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/ 2024-03-15T13:00:25.573 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/tlm,trusted-foundations.txt 2024-03-15T13:00:25.575 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fsl.txt 2024-03-15T13:00:25.579 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fw-cfg.txt 2024-03-15T13:00:25.581 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/geniatech.txt 2024-03-15T13:00:25.583 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic-v3.txt 2024-03-15T13:00:25.586 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic.txt 2024-03-15T13:00:25.588 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/global_timer.txt 2024-03-15T13:00:25.591 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/ 2024-03-15T13:00:25.593 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/hisilicon.txt 2024-03-15T13:00:25.596 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/idle-states.txt 2024-03-15T13:00:25.599 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/insignal-boards.txt 2024-03-15T13:00:25.601 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/ 2024-03-15T13:00:25.603 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/keystone.txt 2024-03-15T13:00:25.606 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/kirkwood.txt 2024-03-15T13:00:25.610 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/l2cc.txt 2024-03-15T13:00:25.612 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx-mic.txt 2024-03-15T13:00:25.614 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx.txt 2024-03-15T13:00:25.616 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,berlin.txt 2024-03-15T13:00:25.618 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,dove.txt 2024-03-15T13:00:25.621 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,kirkwood.txt 2024-03-15T13:00:25.623 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek.txt 2024-03-15T13:00:25.625 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/ 2024-03-15T13:00:25.628 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/mediatek,sysirq.txt 2024-03-15T13:00:25.630 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/moxart.txt 2024-03-15T13:00:25.633 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/ 2024-03-15T13:00:25.636 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/feroceon.txt 2024-03-15T13:00:25.638 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/intc.txt 2024-03-15T13:00:25.640 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/mrvl.txt 2024-03-15T13:00:25.642 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/tauros2.txt 2024-03-15T13:00:25.644 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/timer.txt 2024-03-15T13:00:25.647 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ 2024-03-15T13:00:25.650 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,kpss-acc.txt 2024-03-15T13:00:25.653 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,saw2.txt 2024-03-15T13:00:25.656 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ssbi.txt 2024-03-15T13:00:25.658 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/timer.txt 2024-03-15T13:00:25.661 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mvebu-system-controller.txt 2024-03-15T13:00:25.664 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/nspire.txt 2024-03-15T13:00:25.666 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/olimex.txt 2024-03-15T13:00:25.669 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/ 2024-03-15T13:00:25.671 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/counter.txt 2024-03-15T13:00:25.674 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/crossbar.txt 2024-03-15T13:00:25.676 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dmm.txt 2024-03-15T13:00:25.678 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dsp.txt 2024-03-15T13:00:25.681 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/intc.txt 2024-03-15T13:00:25.683 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/iva.txt 2024-03-15T13:00:25.685 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/l3-noc.txt 2024-03-15T13:00:25.687 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/mpu.txt 2024-03-15T13:00:25.690 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/omap.txt 2024-03-15T13:00:25.692 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/prcm.txt 2024-03-15T13:00:25.694 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/timer.txt 2024-03-15T13:00:25.697 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/picoxcell.txt 2024-03-15T13:00:25.700 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/pmu.txt 2024-03-15T13:00:25.702 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/primecell.txt 2024-03-15T13:00:25.705 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/psci.txt 2024-03-15T13:00:25.707 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip.txt 2024-03-15T13:00:25.709 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/ 2024-03-15T13:00:25.711 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu-sram.txt 2024-03-15T13:00:25.714 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu.txt 2024-03-15T13:00:25.717 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/smp-sram.txt 2024-03-15T13:00:25.719 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rtsm-dcscb.txt 2024-03-15T13:00:25.722 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung-boards.txt 2024-03-15T13:00:25.725 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/ 2024-03-15T13:00:25.728 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-adc.txt 2024-03-15T13:00:25.731 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-chipid.txt 2024-03-15T13:00:25.733 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/interrupt-combiner.txt 2024-03-15T13:00:25.736 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/pmu.txt 2024-03-15T13:00:25.739 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/sysreg.txt 2024-03-15T13:00:25.741 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/shmobile.txt 2024-03-15T13:00:25.744 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sirf.txt 2024-03-15T13:00:25.746 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-misc.txt 2024-03-15T13:00:25.749 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-timer.txt 2024-03-15T13:00:25.751 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear.txt 2024-03-15T13:00:25.753 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/ 2024-03-15T13:00:25.756 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/shirq.txt 2024-03-15T13:00:25.758 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sprd.txt 2024-03-15T13:00:25.761 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-nomadik.txt 2024-03-15T13:00:25.763 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-u300.txt 2024-03-15T13:00:25.766 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sti.txt 2024-03-15T13:00:25.768 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sunxi.txt 2024-03-15T13:00:25.770 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra.txt 2024-03-15T13:00:25.772 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/ 2024-03-15T13:00:25.775 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-ahb.txt 2024-03-15T13:00:25.778 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-emc.txt 2024-03-15T13:00:25.780 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-flowctrl.txt 2024-03-15T13:00:25.782 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-mc.txt 2024-03-15T13:00:25.785 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-pmc.txt 2024-03-15T13:00:25.787 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra30-mc.txt 2024-03-15T13:00:25.789 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/topology.txt 2024-03-15T13:00:25.793 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/twd.txt 2024-03-15T13:00:25.795 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/ 2024-03-15T13:00:25.797 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/power_domain.txt 2024-03-15T13:00:25.800 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-fpga-irq.txt 2024-03-15T13:00:25.803 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-sysreg.txt 2024-03-15T13:00:25.806 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-scc.txt 2024-03-15T13:00:25.808 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-sysreg.txt 2024-03-15T13:00:25.811 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress.txt 2024-03-15T13:00:25.813 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vic.txt 2024-03-15T13:00:25.821 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500.txt 2024-03-15T13:00:25.823 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/ 2024-03-15T13:00:25.826 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-intc.txt 2024-03-15T13:00:25.828 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-pmc.txt 2024-03-15T13:00:25.831 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-timer.txt 2024-03-15T13:00:25.833 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xen.txt 2024-03-15T13:00:25.836 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xilinx.txt 2024-03-15T13:00:25.839 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ 2024-03-15T13:00:25.841 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-platform.txt 2024-03-15T13:00:25.844 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-st.txt 2024-03-15T13:00:25.846 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/apm-xgene.txt 2024-03-15T13:00:25.849 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/atmel-at91_cf.txt 2024-03-15T13:00:25.851 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/cavium-compact-flash.txt 2024-03-15T13:00:25.854 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/exynos-sata.txt 2024-03-15T13:00:25.856 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/fsl-sata.txt 2024-03-15T13:00:25.858 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-pata.txt 2024-03-15T13:00:25.861 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-sata.txt 2024-03-15T13:00:25.863 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/marvell.txt 2024-03-15T13:00:25.865 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/pata-arasan.txt 2024-03-15T13:00:25.868 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/qcom-sata.txt 2024-03-15T13:00:25.870 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_highbank.txt 2024-03-15T13:00:25.872 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_rcar.txt 2024-03-15T13:00:25.875 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/tegra-sata.txt 2024-03-15T13:00:25.877 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/btmrvl.txt 2024-03-15T13:00:25.880 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ 2024-03-15T13:00:25.882 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/bcma.txt 2024-03-15T13:00:25.885 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/brcm,gisb-arb.txt 2024-03-15T13:00:25.887 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/imx-weim.txt 2024-03-15T13:00:25.890 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/mvebu-mbus.txt 2024-03-15T13:00:25.892 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/omap-ocp2scp.txt 2024-03-15T13:00:25.895 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ti-gpmc.txt 2024-03-15T13:00:25.898 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/ 2024-03-15T13:00:25.900 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/clocks.txt 2024-03-15T13:00:25.903 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/dscr.txt 2024-03-15T13:00:25.905 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/emifa.txt 2024-03-15T13:00:25.908 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/interrupt.txt 2024-03-15T13:00:25.910 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/soc.txt 2024-03-15T13:00:25.912 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/timer64.txt 2024-03-15T13:00:25.915 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/chosen.txt 2024-03-15T13:00:25.917 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ 2024-03-15T13:00:25.920 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/alphascale,acc.txt 2024-03-15T13:00:25.922 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/altr_socfpga.txt 2024-03-15T13:00:25.924 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/arm-integrator.txt 2024-03-15T13:00:25.927 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/at91-clock.txt 2024-03-15T13:00:25.929 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/axi-clkgen.txt 2024-03-15T13:00:25.931 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-cygnus-clock.txt 2024-03-15T13:00:25.934 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-kona-clock.txt 2024-03-15T13:00:25.936 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/calxeda.txt 2024-03-15T13:00:25.938 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-exynos-audss.txt 2024-03-15T13:00:25.941 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-palmas-clk32kg-clocks.txt 2024-03-15T13:00:25.943 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-s5pv210-audss.txt 2024-03-15T13:00:25.945 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clock-bindings.txt 2024-03-15T13:00:25.948 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clps711x-clock.txt 2024-03-15T13:00:25.950 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/efm32-clock.txt 2024-03-15T13:00:25.953 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/emev2-clock.txt 2024-03-15T13:00:25.955 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos3250-clock.txt 2024-03-15T13:00:25.957 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4-clock.txt 2024-03-15T13:00:25.960 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4415-clock.txt 2024-03-15T13:00:25.962 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5250-clock.txt 2024-03-15T13:00:25.964 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5260-clock.txt 2024-03-15T13:00:25.967 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5410-clock.txt 2024-03-15T13:00:25.969 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5420-clock.txt 2024-03-15T13:00:25.972 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5440-clock.txt 2024-03-15T13:00:25.975 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos7-clock.txt 2024-03-15T13:00:25.977 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-clock.txt 2024-03-15T13:00:25.980 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-factor-clock.txt 2024-03-15T13:00:25.982 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/gpio-gate-clock.txt 2024-03-15T13:00:25.984 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hi3620-clock.txt 2024-03-15T13:00:25.986 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hix5hd2-clock.txt 2024-03-15T13:00:25.989 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx1-clock.txt 2024-03-15T13:00:25.991 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx21-clock.txt 2024-03-15T13:00:25.994 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx23-clock.txt 2024-03-15T13:00:25.995 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx25-clock.txt 2024-03-15T13:00:25.998 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx27-clock.txt 2024-03-15T13:00:26.002 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx28-clock.txt 2024-03-15T13:00:26.004 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx31-clock.txt 2024-03-15T13:00:26.006 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx35-clock.txt 2024-03-15T13:00:26.008 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx5-clock.txt 2024-03-15T13:00:26.010 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6q-clock.txt 2024-03-15T13:00:26.013 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sl-clock.txt 2024-03-15T13:00:26.015 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sx-clock.txt 2024-03-15T13:00:26.018 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-gate.txt 2024-03-15T13:00:26.020 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-pll.txt 2024-03-15T13:00:26.022 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/lsi,axm5516-clks.txt 2024-03-15T13:00:26.025 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,mmp2.txt 2024-03-15T13:00:26.027 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa168.txt 2024-03-15T13:00:26.029 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa910.txt 2024-03-15T13:00:26.031 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77686.txt 2024-03-15T13:00:26.033 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77802.txt 2024-03-15T13:00:26.036 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/moxa,moxart-clock.txt 2024-03-15T13:00:26.038 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-core-clock.txt 2024-03-15T13:00:26.040 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-corediv-clock.txt 2024-03-15T13:00:26.043 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-cpu-clock.txt 2024-03-15T13:00:26.045 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-gated-clock.txt 2024-03-15T13:00:26.047 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nspire-clock.txt 2024-03-15T13:00:26.050 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra114-car.txt 2024-03-15T13:00:26.052 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra124-car.txt 2024-03-15T13:00:26.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:25 smithi002 ceph-mon[32060]: pgmap v48: 129 pgs: 129 active+clean; 83 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.8 MiB/s wr, 46 op/s 2024-03-15T13:00:26.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:25 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:00:26.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:25 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:26.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:25 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:26.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:25 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:26.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:25 smithi002 ceph-mon[32060]: mgrmap e26: x(active, since 90s), standbys: z, y 2024-03-15T13:00:26.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:25 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:00:26.054 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra20-car.txt 2024-03-15T13:00:26.057 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra30-car.txt 2024-03-15T13:00:26.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:25 smithi184 ceph-mon[35993]: pgmap v48: 129 pgs: 129 active+clean; 83 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.8 MiB/s wr, 46 op/s 2024-03-15T13:00:26.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:25 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:00:26.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:25 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:26.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:25 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:26.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:25 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:26.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:25 smithi184 ceph-mon[35993]: mgrmap e26: x(active, since 90s), standbys: z, y 2024-03-15T13:00:26.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:25 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:00:26.059 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/prima2-clock.txt 2024-03-15T13:00:26.061 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/pxa-clock.txt 2024-03-15T13:00:26.064 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,gcc.txt 2024-03-15T13:00:26.066 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,lcc.txt 2024-03-15T13:00:26.068 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,mmcc.txt 2024-03-15T13:00:26.071 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qoriq-clock.txt 2024-03-15T13:00:26.073 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-div6-clocks.txt 2024-03-15T13:00:26.075 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-mstp-clocks.txt 2024-03-15T13:00:26.078 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a73a4-cpg-clocks.txt 2024-03-15T13:00:26.080 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7740-cpg-clocks.txt 2024-03-15T13:00:26.083 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7779-cpg-clocks.txt 2024-03-15T13:00:26.085 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rcar-gen2-cpg-clocks.txt 2024-03-15T13:00:26.088 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rz-cpg-clocks.txt 2024-03-15T13:00:26.090 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,sh73a0-cpg-clocks.txt 2024-03-15T13:00:26.092 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3188-cru.txt 2024-03-15T13:00:26.095 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3288-cru.txt 2024-03-15T13:00:26.097 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip.txt 2024-03-15T13:00:26.100 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2410-clock.txt 2024-03-15T13:00:26.102 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2412-clock.txt 2024-03-15T13:00:26.104 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2443-clock.txt 2024-03-15T13:00:26.107 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c64xx-clock.txt 2024-03-15T13:00:26.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:25 smithi149 ceph-mon[35927]: pgmap v48: 129 pgs: 129 active+clean; 83 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.8 MiB/s wr, 46 op/s 2024-03-15T13:00:26.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:25 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:00:26.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:25 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:26.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:25 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:26.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:25 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:26.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:25 smithi149 ceph-mon[35927]: mgrmap e26: x(active, since 90s), standbys: z, y 2024-03-15T13:00:26.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:25 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:00:26.110 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s5pv210-clock.txt 2024-03-15T13:00:26.112 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si5351.txt 2024-03-15T13:00:26.114 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si570.txt 2024-03-15T13:00:26.116 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st,nomadik.txt 2024-03-15T13:00:26.118 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/ 2024-03-15T13:00:26.120 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-divmux.txt 2024-03-15T13:00:26.123 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-mux.txt 2024-03-15T13:00:26.126 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-pll.txt 2024-03-15T13:00:26.128 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-prediv.txt 2024-03-15T13:00:26.130 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-vcc.txt 2024-03-15T13:00:26.133 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen.txt 2024-03-15T13:00:26.135 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,flexgen.txt 2024-03-15T13:00:26.138 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,quadfs.txt 2024-03-15T13:00:26.141 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ste-u300-syscon-clock.txt 2024-03-15T13:00:26.144 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/sunxi.txt 2024-03-15T13:00:26.146 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti,cdce706.txt 2024-03-15T13:00:26.148 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti-keystone-pllctrl.txt 2024-03-15T13:00:26.150 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/ 2024-03-15T13:00:26.153 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/apll.txt 2024-03-15T13:00:26.156 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/autoidle.txt 2024-03-15T13:00:26.158 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/clockdomain.txt 2024-03-15T13:00:26.160 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/composite.txt 2024-03-15T13:00:26.163 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/divider.txt 2024-03-15T13:00:26.165 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dpll.txt 2024-03-15T13:00:26.168 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dra7-atl.txt 2024-03-15T13:00:26.170 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fapll.txt 2024-03-15T13:00:26.173 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fixed-factor-clock.txt 2024-03-15T13:00:26.175 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/gate.txt 2024-03-15T13:00:26.177 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/interface.txt 2024-03-15T13:00:26.180 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/mux.txt 2024-03-15T13:00:26.182 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vf610-clock.txt 2024-03-15T13:00:26.185 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vt8500.txt 2024-03-15T13:00:26.188 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/xgene.txt 2024-03-15T13:00:26.190 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/zynq-7000.txt 2024-03-15T13:00:26.192 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/ 2024-03-15T13:00:26.195 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/arm_big_little_dt.txt 2024-03-15T13:00:26.198 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-dt.txt 2024-03-15T13:00:26.200 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-exynos5440.txt 2024-03-15T13:00:26.202 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-spear.txt 2024-03-15T13:00:26.205 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/ 2024-03-15T13:00:26.207 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/amd-ccp.txt 2024-03-15T13:00:26.210 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/atmel-crypto.txt 2024-03-15T13:00:26.212 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-dcp.txt 2024-03-15T13:00:26.214 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-imx-sahara.txt 2024-03-15T13:00:26.217 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec2.txt 2024-03-15T13:00:26.219 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec4.txt 2024-03-15T13:00:26.221 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec6.txt 2024-03-15T13:00:26.224 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/mv_cesa.txt 2024-03-15T13:00:26.226 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-aes.txt 2024-03-15T13:00:26.228 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-des.txt 2024-03-15T13:00:26.230 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-sham.txt 2024-03-15T13:00:26.233 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/picochip-spacc.txt 2024-03-15T13:00:26.235 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/qcom-qce.txt 2024-03-15T13:00:26.237 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/samsung-sss.txt 2024-03-15T13:00:26.240 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/ 2024-03-15T13:00:26.243 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/ 2024-03-15T13:00:26.245 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/exynos-ppmu.txt 2024-03-15T13:00:26.247 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ 2024-03-15T13:00:26.251 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/arm-pl330.txt 2024-03-15T13:00:26.254 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-dma.txt 2024-03-15T13:00:26.256 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-xdma.txt 2024-03-15T13:00:26.258 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/bcm2835-dma.txt 2024-03-15T13:00:26.261 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/dma.txt 2024-03-15T13:00:26.263 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-edma.txt 2024-03-15T13:00:26.265 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-dma.txt 2024-03-15T13:00:26.268 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-sdma.txt 2024-03-15T13:00:26.270 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-mxs-dma.txt 2024-03-15T13:00:26.272 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/img-mdc-dma.txt 2024-03-15T13:00:26.274 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/k3dma.txt 2024-03-15T13:00:26.276 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mmp-dma.txt 2024-03-15T13:00:26.279 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/moxa,moxart-dma.txt 2024-03-15T13:00:26.281 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mpc512x-dma.txt 2024-03-15T13:00:26.284 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mv-xor.txt 2024-03-15T13:00:26.287 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/nbpfaxi.txt 2024-03-15T13:00:26.289 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_adm.txt 2024-03-15T13:00:26.291 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_bam_dma.txt 2024-03-15T13:00:26.294 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/rcar-audmapp.txt 2024-03-15T13:00:26.296 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/renesas,rcar-dmac.txt 2024-03-15T13:00:26.299 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/shdma.txt 2024-03-15T13:00:26.301 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sirfsoc-dma.txt 2024-03-15T13:00:26.303 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/snps-dma.txt 2024-03-15T13:00:26.306 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-coh901318.txt 2024-03-15T13:00:26.308 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-dma40.txt 2024-03-15T13:00:26.311 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sun6i-dma.txt 2024-03-15T13:00:26.313 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/tegra20-apbdma.txt 2024-03-15T13:00:26.315 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ti-edma.txt 2024-03-15T13:00:26.318 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/ 2024-03-15T13:00:26.320 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt 2024-03-15T13:00:26.323 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt 2024-03-15T13:00:26.324 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/ 2024-03-15T13:00:26.327 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/ 2024-03-15T13:00:26.329 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/marvell,dove-lcd.txt 2024-03-15T13:00:26.332 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/ 2024-03-15T13:00:26.335 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/hlcdc-dc.txt 2024-03-15T13:00:26.338 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/ 2024-03-15T13:00:26.340 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/dw_hdmi.txt 2024-03-15T13:00:26.343 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/ 2024-03-15T13:00:26.346 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/tda998x.txt 2024-03-15T13:00:26.348 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ 2024-03-15T13:00:26.351 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/fsl-imx-drm.txt 2024-03-15T13:00:26.354 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/hdmi.txt 2024-03-15T13:00:26.356 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ldb.txt 2024-03-15T13:00:26.358 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/ 2024-03-15T13:00:26.361 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/gpu.txt 2024-03-15T13:00:26.364 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/hdmi.txt 2024-03-15T13:00:26.366 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/mdp.txt 2024-03-15T13:00:26.368 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/ 2024-03-15T13:00:26.371 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/panel.txt 2024-03-15T13:00:26.374 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/slave.txt 2024-03-15T13:00:26.376 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tfp410.txt 2024-03-15T13:00:26.379 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tilcdc.txt 2024-03-15T13:00:26.381 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/eeprom.txt 2024-03-15T13:00:26.384 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/ 2024-03-15T13:00:26.387 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-palmas.txt 2024-03-15T13:00:26.389 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-rt8973a.txt 2024-03-15T13:00:26.392 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-sm5502.txt 2024-03-15T13:00:26.394 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/ 2024-03-15T13:00:26.397 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/mxsfb.txt 2024-03-15T13:00:26.400 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/sm501fb.txt 2024-03-15T13:00:26.402 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/ 2024-03-15T13:00:26.405 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/altera-socfpga-fpga-mgr.txt 2024-03-15T13:00:26.408 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/ 2024-03-15T13:00:26.411 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.txt 2024-03-15T13:00:26.413 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/ 2024-03-15T13:00:26.416 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/8xxx_gpio.txt 2024-03-15T13:00:26.419 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/abilis,tb10x-gpio.txt 2024-03-15T13:00:26.421 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cavium-octeon-gpio.txt 2024-03-15T13:00:26.423 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cirrus,clps711x-mctrl-gpio.txt 2024-03-15T13:00:26.426 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fsl-imx-gpio.txt 2024-03-15T13:00:26.428 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fujitsu,mb86s70-gpio.txt 2024-03-15T13:00:26.430 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74x164.txt 2024-03-15T13:00:26.432 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74xx-mmio.txt 2024-03-15T13:00:26.434 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-adnp.txt 2024-03-15T13:00:26.437 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-bcm-kona.txt 2024-03-15T13:00:26.439 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-clps711x.txt 2024-03-15T13:00:26.441 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-davinci.txt 2024-03-15T13:00:26.443 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-dsp-keystone.txt 2024-03-15T13:00:26.446 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-fan.txt 2024-03-15T13:00:26.448 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-grgpio.txt 2024-03-15T13:00:26.450 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-lp3943.txt 2024-03-15T13:00:26.453 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-max732x.txt 2024-03-15T13:00:26.455 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mcp23s08.txt 2024-03-15T13:00:26.457 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mm-lantiq.txt 2024-03-15T13:00:26.460 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-msm.txt 2024-03-15T13:00:26.462 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mvebu.txt 2024-03-15T13:00:26.465 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mxs.txt 2024-03-15T13:00:26.467 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-nmk.txt 2024-03-15T13:00:26.469 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-omap.txt 2024-03-15T13:00:26.471 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-palmas.txt 2024-03-15T13:00:26.473 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pca953x.txt 2024-03-15T13:00:26.476 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pcf857x.txt 2024-03-15T13:00:26.479 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-poweroff.txt 2024-03-15T13:00:26.481 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-restart.txt 2024-03-15T13:00:26.483 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-samsung.txt 2024-03-15T13:00:26.486 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stericsson-coh901.txt 2024-03-15T13:00:26.488 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stmpe.txt 2024-03-15T13:00:26.490 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stp-xway.txt 2024-03-15T13:00:26.493 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-sx150x.txt 2024-03-15T13:00:26.495 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-twl4030.txt 2024-03-15T13:00:26.498 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090-pdc.txt 2024-03-15T13:00:26.500 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090.txt 2024-03-15T13:00:26.502 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-vf610.txt 2024-03-15T13:00:26.504 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene-sb.txt 2024-03-15T13:00:26.506 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene.txt 2024-03-15T13:00:26.509 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xilinx.txt 2024-03-15T13:00:26.511 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zevio.txt 2024-03-15T13:00:26.514 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zynq.txt 2024-03-15T13:00:26.516 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio.txt 2024-03-15T13:00:26.519 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_atmel.txt 2024-03-15T13:00:26.521 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_lpc32xx.txt 2024-03-15T13:00:26.524 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/moxa,moxart-gpio.txt 2024-03-15T13:00:26.526 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/mrvl-gpio.txt 2024-03-15T13:00:26.528 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/nvidia,tegra20-gpio.txt 2024-03-15T13:00:26.531 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/pl061-gpio.txt 2024-03-15T13:00:26.533 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/renesas,gpio-rcar.txt 2024-03-15T13:00:26.535 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/snps-dwapb-gpio.txt 2024-03-15T13:00:26.537 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/sodaville.txt 2024-03-15T13:00:26.540 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/spear_spics.txt 2024-03-15T13:00:26.542 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/ 2024-03-15T13:00:26.548 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,gk20a.txt 2024-03-15T13:00:26.550 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,tegra20-host1x.txt 2024-03-15T13:00:26.553 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-g2d.txt 2024-03-15T13:00:26.556 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-rotator.txt 2024-03-15T13:00:26.558 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/st,stih4xx.txt 2024-03-15T13:00:26.560 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/graph.txt 2024-03-15T13:00:26.563 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/ 2024-03-15T13:00:26.565 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/hid-over-i2c.txt 2024-03-15T13:00:26.568 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/ 2024-03-15T13:00:26.571 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/client-devices.txt 2024-03-15T13:00:26.574 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/nokia-modem.txt 2024-03-15T13:00:26.576 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/omap-ssi.txt 2024-03-15T13:00:26.578 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ 2024-03-15T13:00:26.581 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ads1015.txt 2024-03-15T13:00:26.583 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/g762.txt 2024-03-15T13:00:26.586 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ibmpowernv.txt 2024-03-15T13:00:26.588 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/lm90.txt 2024-03-15T13:00:26.590 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ltc2978.txt 2024-03-15T13:00:26.593 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ntc_thermistor.txt 2024-03-15T13:00:26.595 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/pwm-fan.txt 2024-03-15T13:00:26.597 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/vexpress.txt 2024-03-15T13:00:26.600 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/ 2024-03-15T13:00:26.603 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/atmel-trng.txt 2024-03-15T13:00:26.605 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/omap_rng.txt 2024-03-15T13:00:26.607 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/timeriomem_rng.txt 2024-03-15T13:00:26.609 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ 2024-03-15T13:00:26.612 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,bcm2835-i2c.txt 2024-03-15T13:00:26.615 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,iproc-i2c.txt 2024-03-15T13:00:26.617 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-arb-gpio-challenge.txt 2024-03-15T13:00:26.620 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-at91.txt 2024-03-15T13:00:26.622 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-axxia.txt 2024-03-15T13:00:26.625 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-bcm-kona.txt 2024-03-15T13:00:26.627 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cadence.txt 2024-03-15T13:00:26.629 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cbus-gpio.txt 2024-03-15T13:00:26.632 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cros-ec-tunnel.txt 2024-03-15T13:00:26.634 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-davinci.txt 2024-03-15T13:00:26.636 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-designware.txt 2024-03-15T13:00:26.639 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-efm32.txt 2024-03-15T13:00:26.641 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-exynos5.txt 2024-03-15T13:00:26.643 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-gpio.txt 2024-03-15T13:00:26.646 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-hix5hd2.txt 2024-03-15T13:00:26.648 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-img-scb.txt 2024-03-15T13:00:26.651 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-imx.txt 2024-03-15T13:00:26.653 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-meson.txt 2024-03-15T13:00:26.655 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mpc.txt 2024-03-15T13:00:26.658 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-gpio.txt 2024-03-15T13:00:26.660 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pca954x.txt 2024-03-15T13:00:26.662 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pinctrl.txt 2024-03-15T13:00:26.665 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux.txt 2024-03-15T13:00:26.667 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mv64xxx.txt 2024-03-15T13:00:26.669 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mxs.txt 2024-03-15T13:00:26.672 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-nomadik.txt 2024-03-15T13:00:26.674 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-ocores.txt 2024-03-15T13:00:26.677 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-octeon.txt 2024-03-15T13:00:26.679 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-omap.txt 2024-03-15T13:00:26.682 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-opal.txt 2024-03-15T13:00:26.684 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pnx.txt 2024-03-15T13:00:26.687 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa-pci-ce4100.txt 2024-03-15T13:00:26.689 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa.txt 2024-03-15T13:00:26.691 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rcar.txt 2024-03-15T13:00:26.693 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-riic.txt 2024-03-15T13:00:26.696 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rk3x.txt 2024-03-15T13:00:26.698 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-s3c2410.txt 2024-03-15T13:00:26.700 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sh_mobile.txt 2024-03-15T13:00:26.703 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sirf.txt 2024-03-15T13:00:26.705 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st-ddci2c.txt 2024-03-15T13:00:26.708 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st.txt 2024-03-15T13:00:26.710 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sunxi-p2wi.txt 2024-03-15T13:00:26.712 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-versatile.txt 2024-03-15T13:00:26.714 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-vt8500.txt 2024-03-15T13:00:26.717 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-xiic.txt 2024-03-15T13:00:26.719 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina209.txt 2024-03-15T13:00:26.726 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina2xx.txt 2024-03-15T13:00:26.726 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/max6697.txt 2024-03-15T13:00:26.727 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/nvidia,tegra20-i2c.txt 2024-03-15T13:00:26.730 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/qcom,i2c-qup.txt 2024-03-15T13:00:26.732 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ti,bq32k.txt 2024-03-15T13:00:26.734 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/trivial-devices.txt 2024-03-15T13:00:26.737 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/ 2024-03-15T13:00:26.740 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/ 2024-03-15T13:00:26.742 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/bma180.txt 2024-03-15T13:00:26.744 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/ 2024-03-15T13:00:26.747 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/at91_adc.txt 2024-03-15T13:00:26.750 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/cc10001_adc.txt 2024-03-15T13:00:26.753 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/max1027-adc.txt 2024-03-15T13:00:26.755 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/nuvoton-nau7802.txt 2024-03-15T13:00:26.757 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-iadc.txt 2024-03-15T13:00:26.760 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-vadc.txt 2024-03-15T13:00:26.762 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/rockchip-saradc.txt 2024-03-15T13:00:26.764 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/twl4030-madc.txt 2024-03-15T13:00:26.767 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/vf610-adc.txt 2024-03-15T13:00:26.769 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/xilinx-xadc.txt 2024-03-15T13:00:26.771 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ 2024-03-15T13:00:26.774 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ad7303.txt 2024-03-15T13:00:26.777 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/max5821.txt 2024-03-15T13:00:26.780 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/ 2024-03-15T13:00:26.783 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/adf4350.txt 2024-03-15T13:00:26.785 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/ 2024-03-15T13:00:26.788 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/dht11.txt 2024-03-15T13:00:26.791 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/iio-bindings.txt 2024-03-15T13:00:26.794 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/ 2024-03-15T13:00:26.796 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/apds9300.txt 2024-03-15T13:00:26.800 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/cm36651.txt 2024-03-15T13:00:26.802 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/gp2ap020a00f.txt 2024-03-15T13:00:26.804 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/tsl2563.txt 2024-03-15T13:00:26.806 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ 2024-03-15T13:00:26.809 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ak8975.txt 2024-03-15T13:00:26.812 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/hmc5843.txt 2024-03-15T13:00:26.815 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/ 2024-03-15T13:00:26.817 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/as3935.txt 2024-03-15T13:00:26.820 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/sensorhub.txt 2024-03-15T13:00:26.826 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/st-sensors.txt 2024-03-15T13:00:26.828 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ 2024-03-15T13:00:26.831 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ads7846.txt 2024-03-15T13:00:26.833 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/atmel,maxtouch.txt 2024-03-15T13:00:26.836 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cap11xx.txt 2024-03-15T13:00:26.838 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/clps711x-keypad.txt 2024-03-15T13:00:26.840 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cros-ec-keyb.txt 2024-03-15T13:00:26.843 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/e3x0-button.txt 2024-03-15T13:00:26.845 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elan_i2c.txt 2024-03-15T13:00:26.847 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elants_i2c.txt 2024-03-15T13:00:26.850 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/fsl-mma8450.txt 2024-03-15T13:00:26.852 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-beeper.txt 2024-03-15T13:00:26.854 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys-polled.txt 2024-03-15T13:00:26.856 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys.txt 2024-03-15T13:00:26.859 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-matrix-keypad.txt 2024-03-15T13:00:26.861 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/imx-keypad.txt 2024-03-15T13:00:26.863 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/input-reset.txt 2024-03-15T13:00:26.866 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/lpc32xx-key.txt 2024-03-15T13:00:26.868 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/matrix-keymap.txt 2024-03-15T13:00:26.870 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/nvidia,tegra20-kbc.txt 2024-03-15T13:00:26.872 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/omap-keypad.txt 2024-03-15T13:00:26.875 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ps2keyb-mouse-apbps2.txt 2024-03-15T13:00:26.877 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pwm-beeper.txt 2024-03-15T13:00:26.879 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pxa27x-keypad.txt 2024-03-15T13:00:26.882 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-keypad.txt 2024-03-15T13:00:26.884 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-pwrkey.txt 2024-03-15T13:00:26.886 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-vib.txt 2024-03-15T13:00:26.888 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/regulator-haptic.txt 2024-03-15T13:00:26.891 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/rotary-encoder.txt 2024-03-15T13:00:26.893 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/samsung-keypad.txt 2024-03-15T13:00:26.895 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/spear-keyboard.txt 2024-03-15T13:00:26.897 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/st-keyscan.txt 2024-03-15T13:00:26.900 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/stmpe-keypad.txt 2024-03-15T13:00:26.902 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/sun4i-lradc-keys.txt 2024-03-15T13:00:26.904 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tca8418_keypad.txt 2024-03-15T13:00:26.907 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv260x.txt 2024-03-15T13:00:26.909 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv2667.txt 2024-03-15T13:00:26.911 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,nspire-keypad.txt 2024-03-15T13:00:26.914 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,palmas-pwrbutton.txt 2024-03-15T13:00:26.916 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ 2024-03-15T13:00:26.918 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/auo_pixcir_ts.txt 2024-03-15T13:00:26.920 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/bu21013.txt 2024-03-15T13:00:26.923 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/edt-ft5x06.txt 2024-03-15T13:00:26.925 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/egalax-ts.txt 2024-03-15T13:00:26.928 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/lpc32xx-tsc.txt 2024-03-15T13:00:26.931 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/mms114.txt 2024-03-15T13:00:26.933 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/pixcir_i2c_ts.txt 2024-03-15T13:00:26.936 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sitronix-st1232.txt 2024-03-15T13:00:26.938 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/stmpe.txt 2024-03-15T13:00:26.941 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sun4i.txt 2024-03-15T13:00:26.943 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ti-tsc-adc.txt 2024-03-15T13:00:26.945 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/touchscreen.txt 2024-03-15T13:00:26.948 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2005.txt 2024-03-15T13:00:26.950 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2007.txt 2024-03-15T13:00:26.953 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/zforce_ts.txt 2024-03-15T13:00:26.955 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tps65218-pwrbutton.txt 2024-03-15T13:00:26.958 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-keypad.txt 2024-03-15T13:00:26.961 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-pwrbutton.txt 2024-03-15T13:00:26.963 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ 2024-03-15T13:00:26.966 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/abilis,tb10x-ictl.txt 2024-03-15T13:00:26.969 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun4i-ic.txt 2024-03-15T13:00:26.971 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun67i-sc-nmi.txt 2024-03-15T13:00:26.973 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/atmel,aic.txt 2024-03-15T13:00:26.975 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm2835-armctrl-ic.txt 2024-03-15T13:00:26.978 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm7120-l2-intc.txt 2024-03-15T13:00:26.980 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,l2-intc.txt 2024-03-15T13:00:26.982 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/cirrus,clps711x-intc.txt 2024-03-15T13:00:26.984 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/digicolor-ic.txt 2024-03-15T13:00:26.987 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/interrupts.txt 2024-03-15T13:00:26.989 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/lsi,zevio-intc.txt 2024-03-15T13:00:26.993 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,armada-370-xp-mpic.txt 2024-03-15T13:00:26.994 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,orion-intc.txt 2024-03-15T13:00:26.996 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/mips-gic.txt 2024-03-15T13:00:26.998 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/opencores,or1k-pic.txt 2024-03-15T13:00:27.001 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,intc-irqpin.txt 2024-03-15T13:00:27.003 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,irqc.txt 2024-03-15T13:00:27.005 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/samsung,s3c24xx-irq.txt 2024-03-15T13:00:27.008 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/snps,dw-apb-ictl.txt 2024-03-15T13:00:27.010 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,keystone-irq.txt 2024-03-15T13:00:27.012 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,omap-intc-irq.txt 2024-03-15T13:00:27.015 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ 2024-03-15T13:00:27.018 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/arm,smmu.txt 2024-03-15T13:00:27.020 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/iommu.txt 2024-03-15T13:00:27.022 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra20-gart.txt 2024-03-15T13:00:27.025 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra30-smmu.txt 2024-03-15T13:00:27.027 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/renesas,ipmmu-vmsa.txt 2024-03-15T13:00:27.030 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/rockchip,iommu.txt 2024-03-15T13:00:27.032 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt 2024-03-15T13:00:27.035 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ti,omap-iommu.txt 2024-03-15T13:00:27.037 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/ 2024-03-15T13:00:27.040 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/common.txt 2024-03-15T13:00:27.042 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-gpio.txt 2024-03-15T13:00:27.045 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp55xx.txt 2024-03-15T13:00:27.047 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp8860.txt 2024-03-15T13:00:27.304 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-ns2.txt 2024-03-15T13:00:27.306 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-pwm.txt 2024-03-15T13:00:27.309 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/pca963x.txt 2024-03-15T13:00:27.311 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/register-bit-led.txt 2024-03-15T13:00:27.314 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/tca6507.txt 2024-03-15T13:00:27.317 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/ 2024-03-15T13:00:27.320 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2-timings.txt 2024-03-15T13:00:27.322 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2.txt 2024-03-15T13:00:27.324 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/ 2024-03-15T13:00:27.327 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/altera-mailbox.txt 2024-03-15T13:00:27.330 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/mailbox.txt 2024-03-15T13:00:27.332 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/omap-mailbox.txt 2024-03-15T13:00:27.335 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/marvell.txt 2024-03-15T13:00:27.338 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ 2024-03-15T13:00:27.341 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/atmel-isi.txt 2024-03-15T13:00:27.343 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/coda.txt 2024-03-15T13:00:27.345 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-fimc-lite.txt 2024-03-15T13:00:27.348 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-jpeg-codec.txt 2024-03-15T13:00:27.350 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos4-fimc-is.txt 2024-03-15T13:00:27.352 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos5-gsc.txt 2024-03-15T13:00:27.355 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/gpio-ir-receiver.txt 2024-03-15T13:00:27.357 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/hix5hd2-ir.txt 2024-03-15T13:00:27.359 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ 2024-03-15T13:00:27.362 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7343.txt 2024-03-15T13:00:27.364 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7604.txt 2024-03-15T13:00:27.366 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9m111.txt 2024-03-15T13:00:27.369 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9p031.txt 2024-03-15T13:00:27.371 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/nokia,smia.txt 2024-03-15T13:00:27.373 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ths8200.txt 2024-03-15T13:00:27.376 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp514x.txt 2024-03-15T13:00:27.378 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp7002.txt 2024-03-15T13:00:27.380 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/img-ir-rev1.txt 2024-03-15T13:00:27.383 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/meson-ir.txt 2024-03-15T13:00:27.385 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/pxa-camera.txt 2024-03-15T13:00:27.387 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/rcar_vin.txt 2024-03-15T13:00:27.390 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/renesas,vsp1.txt 2024-03-15T13:00:27.392 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/s5p-mfc.txt 2024-03-15T13:00:27.394 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-fimc.txt 2024-03-15T13:00:27.397 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-mipi-csis.txt 2024-03-15T13:00:27.399 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5c73m3.txt 2024-03-15T13:00:27.402 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k5baf.txt 2024-03-15T13:00:27.404 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k6a3.txt 2024-03-15T13:00:27.406 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sh_mobile_ceu.txt 2024-03-15T13:00:27.408 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/si4713.txt 2024-03-15T13:00:27.411 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/st-rc.txt 2024-03-15T13:00:27.413 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sunxi-ir.txt 2024-03-15T13:00:27.416 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ti-am437x-vpfe.txt 2024-03-15T13:00:27.418 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/video-interfaces.txt 2024-03-15T13:00:27.421 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ 2024-03-15T13:00:27.424 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ 2024-03-15T13:00:27.426 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ifc.txt 2024-03-15T13:00:27.428 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-devbus.txt 2024-03-15T13:00:27.431 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-sdram-controller.txt 2024-03-15T13:00:27.433 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra-mc.txt 2024-03-15T13:00:27.435 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/renesas-memory-controllers.txt 2024-03-15T13:00:27.438 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/synopsys.txt 2024-03-15T13:00:27.440 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti-aemif.txt 2024-03-15T13:00:27.442 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/ 2024-03-15T13:00:27.445 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/emif.txt 2024-03-15T13:00:27.447 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/ 2024-03-15T13:00:27.451 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta-intc.txt 2024-03-15T13:00:27.453 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta.txt 2024-03-15T13:00:27.455 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/pdc-intc.txt 2024-03-15T13:00:27.458 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ 2024-03-15T13:00:27.461 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/88pm860x.txt 2024-03-15T13:00:27.463 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ab8500.txt 2024-03-15T13:00:27.465 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/arizona.txt 2024-03-15T13:00:27.468 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3711.txt 2024-03-15T13:00:27.470 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3722.txt 2024-03-15T13:00:27.472 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-gpbr.txt 2024-03-15T13:00:27.474 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-hlcdc.txt 2024-03-15T13:00:27.477 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-matrix.txt 2024-03-15T13:00:27.479 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-smc.txt 2024-03-15T13:00:27.482 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bcm590xx.txt 2024-03-15T13:00:27.484 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bfticu.txt 2024-03-15T13:00:27.487 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/cros-ec.txt 2024-03-15T13:00:27.489 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9052-i2c.txt 2024-03-15T13:00:27.492 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9055.txt 2024-03-15T13:00:27.494 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9063.txt 2024-03-15T13:00:27.497 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/hi6421.txt 2024-03-15T13:00:27.499 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/lp3943.txt 2024-03-15T13:00:27.502 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max14577.txt 2024-03-15T13:00:27.504 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77686.txt 2024-03-15T13:00:27.506 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77693.txt 2024-03-15T13:00:27.509 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8925.txt 2024-03-15T13:00:27.511 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8998.txt 2024-03-15T13:00:27.513 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/mc13xxx.txt 2024-03-15T13:00:27.516 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-host.txt 2024-03-15T13:00:27.519 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-tll.txt 2024-03-15T13:00:27.521 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/palmas.txt 2024-03-15T13:00:27.524 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom,spmi-pmic.txt 2024-03-15T13:00:27.526 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-pm8xxx.txt 2024-03-15T13:00:27.528 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-rpm.txt 2024-03-15T13:00:27.531 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qriox.txt 2024-03-15T13:00:27.533 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rk808.txt 2024-03-15T13:00:27.536 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rn5t618.txt 2024-03-15T13:00:27.538 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mpa01.txt 2024-03-15T13:00:27.540 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mps11.txt 2024-03-15T13:00:27.543 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/stmpe.txt 2024-03-15T13:00:27.545 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/sun6i-prcm.txt 2024-03-15T13:00:27.547 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/syscon.txt 2024-03-15T13:00:27.550 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tc3589x.txt 2024-03-15T13:00:27.552 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ti-keystone-devctrl.txt 2024-03-15T13:00:27.555 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps6507x.txt 2024-03-15T13:00:27.557 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps65910.txt 2024-03-15T13:00:27.560 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl-familly.txt 2024-03-15T13:00:27.562 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-audio.txt 2024-03-15T13:00:27.565 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-power.txt 2024-03-15T13:00:27.567 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl6040.txt 2024-03-15T13:00:27.570 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/ 2024-03-15T13:00:27.575 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/ 2024-03-15T13:00:27.577 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/mipi-dsi-bus.txt 2024-03-15T13:00:27.580 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/nvidia,tegra114-mipi.txt 2024-03-15T13:00:27.583 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ 2024-03-15T13:00:27.586 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/ 2024-03-15T13:00:27.588 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bcm3384-intc.txt 2024-03-15T13:00:27.590 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bmips.txt 2024-03-15T13:00:27.592 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/cm-dsl.txt 2024-03-15T13:00:27.595 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/usb.txt 2024-03-15T13:00:27.597 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ 2024-03-15T13:00:27.600 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/bootbus.txt 2024-03-15T13:00:27.602 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/cib.txt 2024-03-15T13:00:27.604 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu.txt 2024-03-15T13:00:27.607 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu2.txt 2024-03-15T13:00:27.609 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/dma-engine.txt 2024-03-15T13:00:27.611 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/uctl.txt 2024-03-15T13:00:27.613 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cpu_irq.txt 2024-03-15T13:00:27.617 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ralink.txt 2024-03-15T13:00:27.619 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ 2024-03-15T13:00:27.622 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/allwinner,sunxi-sid.txt 2024-03-15T13:00:27.625 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/arm-charlcd.txt 2024-03-15T13:00:27.627 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/at25.txt 2024-03-15T13:00:27.629 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/atmel-ssc.txt 2024-03-15T13:00:27.632 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/bmp085.txt 2024-03-15T13:00:27.634 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt 2024-03-15T13:00:27.636 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ifm-csi.txt 2024-03-15T13:00:27.639 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/lis302.txt 2024-03-15T13:00:27.641 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.txt 2024-03-15T13:00:27.643 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/smc.txt 2024-03-15T13:00:27.645 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/sram.txt 2024-03-15T13:00:27.648 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ti,dac7512.txt 2024-03-15T13:00:27.650 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ 2024-03-15T13:00:27.653 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/arasan,sdhci.txt 2024-03-15T13:00:27.656 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/atmel-hsmci.txt 2024-03-15T13:00:27.658 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/brcm,bcm2835-sdhci.txt 2024-03-15T13:00:27.660 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/davinci_mmc.txt 2024-03-15T13:00:27.663 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/exynos-dw-mshc.txt 2024-03-15T13:00:27.665 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-esdhc.txt 2024-03-15T13:00:27.668 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-esdhc.txt 2024-03-15T13:00:27.670 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-mmc.txt 2024-03-15T13:00:27.672 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/img-dw-mshc.txt 2024-03-15T13:00:27.675 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/k3-dw-mshc.txt 2024-03-15T13:00:27.678 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/kona-sdhci.txt 2024-03-15T13:00:27.680 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-emmc.txt 2024-03-15T13:00:27.682 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-simple.txt 2024-03-15T13:00:27.684 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-spi-slot.txt 2024-03-15T13:00:27.687 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc.txt 2024-03-15T13:00:27.689 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmci.txt 2024-03-15T13:00:27.692 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/moxa,moxart-mmc.txt 2024-03-15T13:00:27.694 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mxs-mmc.txt 2024-03-15T13:00:27.697 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.txt 2024-03-15T13:00:27.699 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/orion-sdio.txt 2024-03-15T13:00:27.702 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/pxa-mmc.txt 2024-03-15T13:00:27.704 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/renesas,mmcif.txt 2024-03-15T13:00:27.706 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.txt 2024-03-15T13:00:27.708 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/samsung-sdhci.txt 2024-03-15T13:00:27.711 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-dove.txt 2024-03-15T13:00:27.713 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-fujitsu.txt 2024-03-15T13:00:27.715 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-msm.txt 2024-03-15T13:00:27.718 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-pxa.txt 2024-03-15T13:00:27.720 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-sirf.txt 2024-03-15T13:00:27.723 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-spear.txt 2024-03-15T13:00:27.725 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-st.txt 2024-03-15T13:00:27.727 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/socfpga-dw-mshc.txt 2024-03-15T13:00:27.730 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sunxi-mmc.txt 2024-03-15T13:00:27.732 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/synopsys-dw-mshc.txt 2024-03-15T13:00:27.735 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap-hsmmc.txt 2024-03-15T13:00:27.737 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap.txt 2024-03-15T13:00:27.740 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/tmio_mmc.txt 2024-03-15T13:00:27.743 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/usdhi6rol0.txt 2024-03-15T13:00:27.746 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/vt8500-sdmmc.txt 2024-03-15T13:00:27.749 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/ 2024-03-15T13:00:27.752 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/arm-versatile.txt 2024-03-15T13:00:27.754 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-dataflash.txt 2024-03-15T13:00:27.757 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-nand.txt 2024-03-15T13:00:27.760 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/davinci-nand.txt 2024-03-15T13:00:27.763 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/denali-nand.txt 2024-03-15T13:00:27.765 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/diskonchip.txt 2024-03-15T13:00:27.768 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/elm.txt 2024-03-15T13:00:27.770 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/flctl-nand.txt 2024-03-15T13:00:27.773 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-quadspi.txt 2024-03-15T13:00:27.775 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-upm-nand.txt 2024-03-15T13:00:27.777 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsmc-nand.txt 2024-03-15T13:00:27.779 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpio-control-nand.txt 2024-03-15T13:00:27.782 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nand.txt 2024-03-15T13:00:27.785 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nor.txt 2024-03-15T13:00:27.787 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-onenand.txt 2024-03-15T13:00:27.789 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmi-nand.txt 2024-03-15T13:00:27.791 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/hisi504-nand.txt 2024-03-15T13:00:27.794 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-mlc.txt 2024-03-15T13:00:27.796 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-slc.txt 2024-03-15T13:00:27.798 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/m25p80.txt 2024-03-15T13:00:27.800 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mtd-physmap.txt 2024-03-15T13:00:27.803 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mxc-nand.txt 2024-03-15T13:00:27.806 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/nand.txt 2024-03-15T13:00:27.808 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/orion-nand.txt 2024-03-15T13:00:27.810 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/partition.txt 2024-03-15T13:00:27.813 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/pxa3xx-nand.txt 2024-03-15T13:00:27.815 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/spear_smi.txt 2024-03-15T13:00:27.817 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/st-fsm.txt 2024-03-15T13:00:27.820 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/sunxi-nand.txt 2024-03-15T13:00:27.822 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ 2024-03-15T13:00:27.836 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-emac.txt 2024-03-15T13:00:27.839 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-mdio.txt 2024-03-15T13:00:27.841 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun7i-a20-gmac.txt 2024-03-15T13:00:27.844 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/altera_tse.txt 2024-03-15T13:00:27.846 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe-phy.txt 2024-03-15T13:00:27.849 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe.txt 2024-03-15T13:00:27.851 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/apm-xgene-enet.txt 2024-03-15T13:00:27.854 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/arc_emac.txt 2024-03-15T13:00:27.856 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcm87xx.txt 2024-03-15T13:00:27.858 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcmgenet.txt 2024-03-15T13:00:27.861 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-mdio-unimac.txt 2024-03-15T13:00:27.864 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-sf2.txt 2024-03-15T13:00:27.867 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-systemport.txt 2024-03-15T13:00:27.869 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/calxeda-xgmac.txt 2024-03-15T13:00:27.872 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/ 2024-03-15T13:00:27.874 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/atmel-can.txt 2024-03-15T13:00:27.878 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/c_can.txt 2024-03-15T13:00:27.880 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/cc770.txt 2024-03-15T13:00:27.883 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/fsl-flexcan.txt 2024-03-15T13:00:27.886 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/grcan.txt 2024-03-15T13:00:27.889 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/m_can.txt 2024-03-15T13:00:27.892 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/microchip,mcp251x.txt 2024-03-15T13:00:27.894 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/mpc5xxx-mscan.txt 2024-03-15T13:00:27.897 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/rcar_can.txt 2024-03-15T13:00:27.900 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/sja1000.txt 2024-03-15T13:00:27.903 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/xilinx_can.txt 2024-03-15T13:00:27.905 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mdio.txt 2024-03-15T13:00:27.908 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mix.txt 2024-03-15T13:00:27.911 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-pip.txt 2024-03-15T13:00:27.914 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cdns-emac.txt 2024-03-15T13:00:27.917 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw-phy-sel.txt 2024-03-15T13:00:27.920 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw.txt 2024-03-15T13:00:27.923 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davicom-dm9000.txt 2024-03-15T13:00:27.927 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci-mdio.txt 2024-03-15T13:00:27.930 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci_emac.txt 2024-03-15T13:00:27.933 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/ 2024-03-15T13:00:27.936 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/dsa.txt 2024-03-15T13:00:27.939 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/emac_rockchip.txt 2024-03-15T13:00:27.942 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ethernet.txt 2024-03-15T13:00:27.945 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fixed-link.txt 2024-03-15T13:00:27.947 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-fec.txt 2024-03-15T13:00:27.950 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-tsec-phy.txt 2024-03-15T13:00:27.953 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/gpmc-eth.txt 2024-03-15T13:00:27.956 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hip04-net.txt 2024-03-15T13:00:27.959 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hix5hd2-gmac.txt 2024-03-15T13:00:27.961 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/ 2024-03-15T13:00:27.964 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/at86rf230.txt 2024-03-15T13:00:27.968 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/cc2520.txt 2024-03-15T13:00:27.971 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/keystone-netcp.txt 2024-03-15T13:00:27.974 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/lpc-eth.txt 2024-03-15T13:00:27.978 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/macb.txt 2024-03-15T13:00:27.980 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt 2024-03-15T13:00:27.983 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-mdio.txt 2024-03-15T13:00:27.986 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-net.txt 2024-03-15T13:00:27.989 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pp2.txt 2024-03-15T13:00:27.991 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pxa168.txt 2024-03-15T13:00:27.994 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-gpio.txt 2024-03-15T13:00:27.996 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-gpio.txt 2024-03-15T13:00:27.999 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-mmioreg.txt 2024-03-15T13:00:28.001 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux.txt 2024-03-15T13:00:28.004 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/meson-dwmac.txt 2024-03-15T13:00:28.006 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ks8851.txt 2024-03-15T13:00:28.009 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ksz90x1.txt 2024-03-15T13:00:28.011 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel.txt 2024-03-15T13:00:28.014 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/moxa,moxart-mac.txt 2024-03-15T13:00:28.016 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/ 2024-03-15T13:00:28.022 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/pn544.txt 2024-03-15T13:00:28.022 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfca.txt 2024-03-15T13:00:28.023 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfcb.txt 2024-03-15T13:00:28.025 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/trf7970a.txt 2024-03-15T13:00:28.028 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/opencores-ethoc.txt 2024-03-15T13:00:28.031 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/phy.txt 2024-03-15T13:00:28.033 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/qca-qca7000-spi.txt 2024-03-15T13:00:28.036 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/rockchip-dwmac.txt 2024-03-15T13:00:28.038 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/samsung-sxgbe.txt 2024-03-15T13:00:28.040 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sh_eth.txt 2024-03-15T13:00:28.043 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc-lan91c111.txt 2024-03-15T13:00:28.045 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc911x.txt 2024-03-15T13:00:28.047 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/socfpga-dwmac.txt 2024-03-15T13:00:28.050 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sti-dwmac.txt 2024-03-15T13:00:28.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:27 smithi002 ceph-mon[32060]: pgmap v49: 129 pgs: 129 active+clean; 86 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.1 MiB/s wr, 78 op/s 2024-03-15T13:00:28.052 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/stmmac.txt 2024-03-15T13:00:28.055 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-rhine.txt 2024-03-15T13:00:28.057 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-velocity.txt 2024-03-15T13:00:28.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:27 smithi184 ceph-mon[35993]: pgmap v49: 129 pgs: 129 active+clean; 86 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.1 MiB/s wr, 78 op/s 2024-03-15T13:00:28.060 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ 2024-03-15T13:00:28.062 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/brcm,bcm43xx-fmac.txt 2024-03-15T13:00:28.065 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt 2024-03-15T13:00:28.067 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ti,wl1251.txt 2024-03-15T13:00:28.070 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/ 2024-03-15T13:00:28.073 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/nios2.txt 2024-03-15T13:00:28.075 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/timer.txt 2024-03-15T13:00:28.077 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/ 2024-03-15T13:00:28.080 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/nvidia,nvec.txt 2024-03-15T13:00:28.083 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/open-pic.txt 2024-03-15T13:00:28.086 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/ 2024-03-15T13:00:28.088 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101aw03.txt 2024-03-15T13:00:28.090 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101xtn01.txt 2024-03-15T13:00:28.092 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b116xw03.txt 2024-03-15T13:00:28.095 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133htn01.txt 2024-03-15T13:00:28.097 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133xtn01.txt 2024-03-15T13:00:28.099 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/avic,tm070ddh03.txt 2024-03-15T13:00:28.102 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wa01a.txt 2024-03-15T13:00:28.104 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wb03.txt 2024-03-15T13:00:28.107 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et057090dhu.txt 2024-03-15T13:00:28.109 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et070080dh6.txt 2024-03-15T13:00:28.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:27 smithi149 ceph-mon[35927]: pgmap v49: 129 pgs: 129 active+clean; 86 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.1 MiB/s wr, 78 op/s 2024-03-15T13:00:28.111 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,etm0700g0dh6.txt 2024-03-15T13:00:28.113 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/foxlink,fl500wvr00-a0t.txt 2024-03-15T13:00:28.116 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/giantplus,gpg482739qs5.txt 2024-03-15T13:00:28.118 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hannstar,hsd070pww1.txt 2024-03-15T13:00:28.120 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hit,tx23d38vm0caa.txt 2024-03-15T13:00:28.123 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,g121i1-l01.txt 2024-03-15T13:00:28.125 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n116bge.txt 2024-03-15T13:00:28.128 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n156bge-l21.txt 2024-03-15T13:00:28.130 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,ld070wx3-sl01.txt 2024-03-15T13:00:28.133 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lh500wx1-sd03.txt 2024-03-15T13:00:28.135 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lp129qe.txt 2024-03-15T13:00:28.138 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/panasonic,vvx10f004b00.txt 2024-03-15T13:00:28.140 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ld9040.txt 2024-03-15T13:00:28.143 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ltn101nt05.txt 2024-03-15T13:00:28.145 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,s6e8aa0.txt 2024-03-15T13:00:28.148 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/sharp,lq101r1sx01.txt 2024-03-15T13:00:28.150 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/simple-panel.txt 2024-03-15T13:00:28.153 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ 2024-03-15T13:00:28.156 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/83xx-512x-pci.txt 2024-03-15T13:00:28.159 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/designware-pcie.txt 2024-03-15T13:00:28.161 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.txt 2024-03-15T13:00:28.164 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,pci.txt 2024-03-15T13:00:28.167 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/host-generic-pci.txt 2024-03-15T13:00:28.169 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/layerscape-pci.txt 2024-03-15T13:00:28.171 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/mvebu-pci.txt 2024-03-15T13:00:28.174 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/nvidia,tegra20-pcie.txt 2024-03-15T13:00:28.176 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-keystone.txt 2024-03-15T13:00:28.179 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-rcar-gen2.txt 2024-03-15T13:00:28.181 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci.txt 2024-03-15T13:00:28.184 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ralink,rt3883-pci.txt 2024-03-15T13:00:28.186 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/rcar-pci.txt 2024-03-15T13:00:28.189 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/samsung,exynos5440-pcie.txt 2024-03-15T13:00:28.191 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/spear13xx-pcie.txt 2024-03-15T13:00:28.193 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ti-pci.txt 2024-03-15T13:00:28.196 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/v3-v360epc-pci.txt 2024-03-15T13:00:28.198 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/versatile.txt 2024-03-15T13:00:28.201 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xgene-pci.txt 2024-03-15T13:00:28.203 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xilinx-pcie.txt 2024-03-15T13:00:28.205 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ 2024-03-15T13:00:28.208 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/apm-xgene-phy.txt 2024-03-15T13:00:28.211 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/bcm-phy.txt 2024-03-15T13:00:28.213 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-sata-phy.txt 2024-03-15T13:00:28.215 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-usb-phy.txt 2024-03-15T13:00:28.218 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/hix5hd2-phy.txt 2024-03-15T13:00:28.220 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-bindings.txt 2024-03-15T13:00:28.222 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy28lp.txt 2024-03-15T13:00:28.226 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy365x.txt 2024-03-15T13:00:28.227 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-mvebu.txt 2024-03-15T13:00:28.229 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih407-usb.txt 2024-03-15T13:00:28.231 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih41x-usb.txt 2024-03-15T13:00:28.234 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-apq8064-sata-phy.txt 2024-03-15T13:00:28.236 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-dwc3-usb-phy.txt 2024-03-15T13:00:28.238 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-ipq806x-sata-phy.txt 2024-03-15T13:00:28.240 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rcar-gen2-phy.txt 2024-03-15T13:00:28.243 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rockchip-usb-phy.txt 2024-03-15T13:00:28.245 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/samsung-phy.txt 2024-03-15T13:00:28.247 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/st-spear-miphy.txt 2024-03-15T13:00:28.250 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/sun4i-usb-phy.txt 2024-03-15T13:00:28.253 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ti-phy.txt 2024-03-15T13:00:28.255 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ 2024-03-15T13:00:28.258 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/abilis,tb10x-iomux.txt 2024-03-15T13:00:28.260 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/allwinner,sunxi-pinctrl.txt 2024-03-15T13:00:28.263 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/atmel,at91-pinctrl.txt 2024-03-15T13:00:28.265 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm11351-pinctrl.txt 2024-03-15T13:00:28.267 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm2835-gpio.txt 2024-03-15T13:00:28.270 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx-pinctrl.txt 2024-03-15T13:00:28.272 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx25-pinctrl.txt 2024-03-15T13:00:28.274 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx27-pinctrl.txt 2024-03-15T13:00:28.276 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx35-pinctrl.txt 2024-03-15T13:00:28.279 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx51-pinctrl.txt 2024-03-15T13:00:28.281 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx53-pinctrl.txt 2024-03-15T13:00:28.283 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6dl-pinctrl.txt 2024-03-15T13:00:28.286 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6q-pinctrl.txt 2024-03-15T13:00:28.288 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sl-pinctrl.txt 2024-03-15T13:00:28.290 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sx-pinctrl.txt 2024-03-15T13:00:28.293 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,mxs-pinctrl.txt 2024-03-15T13:00:28.296 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,vf610-pinctrl.txt 2024-03-15T13:00:28.298 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pdc-pinctrl.txt 2024-03-15T13:00:28.301 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pinctrl.txt 2024-03-15T13:00:28.303 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,falcon-pinumx.txt 2024-03-15T13:00:28.306 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,xway-pinumx.txt 2024-03-15T13:00:28.308 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-370-pinctrl.txt 2024-03-15T13:00:28.311 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-375-pinctrl.txt 2024-03-15T13:00:28.313 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-38x-pinctrl.txt 2024-03-15T13:00:28.316 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-xp-pinctrl.txt 2024-03-15T13:00:28.318 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,dove-pinctrl.txt 2024-03-15T13:00:28.320 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,kirkwood-pinctrl.txt 2024-03-15T13:00:28.323 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,mvebu-pinctrl.txt 2024-03-15T13:00:28.325 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,orion-pinctrl.txt 2024-03-15T13:00:28.328 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/meson,pinctrl.txt 2024-03-15T13:00:28.330 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra114-pinmux.txt 2024-03-15T13:00:28.332 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-pinmux.txt 2024-03-15T13:00:28.335 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-xusb-padctl.txt 2024-03-15T13:00:28.337 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra20-pinmux.txt 2024-03-15T13:00:28.340 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra30-pinmux.txt 2024-03-15T13:00:28.343 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt 2024-03-15T13:00:28.345 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt 2024-03-15T13:00:28.347 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-single.txt 2024-03-15T13:00:28.350 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-sirf.txt 2024-03-15T13:00:28.353 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-st.txt 2024-03-15T13:00:28.355 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-vt8500.txt 2024-03-15T13:00:28.357 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl_spear.txt 2024-03-15T13:00:28.360 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8064-pinctrl.txt 2024-03-15T13:00:28.363 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8084-pinctrl.txt 2024-03-15T13:00:28.365 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,ipq8064-pinctrl.txt 2024-03-15T13:00:28.368 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8916-pinctrl.txt 2024-03-15T13:00:28.370 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8960-pinctrl.txt 2024-03-15T13:00:28.373 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8974-pinctrl.txt 2024-03-15T13:00:28.376 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-gpio.txt 2024-03-15T13:00:28.378 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-mpp.txt 2024-03-15T13:00:28.381 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt 2024-03-15T13:00:28.384 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/rockchip,pinctrl.txt 2024-03-15T13:00:28.387 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/samsung-pinctrl.txt 2024-03-15T13:00:28.389 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,abx500.txt 2024-03-15T13:00:28.392 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,nomadik.txt 2024-03-15T13:00:28.394 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ti,omap-pinctrl.txt 2024-03-15T13:00:28.397 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/xlnx,zynq-pinctrl.txt 2024-03-15T13:00:28.399 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ 2024-03-15T13:00:28.402 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/bq2415x.txt 2024-03-15T13:00:28.405 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/isp1704.txt 2024-03-15T13:00:28.407 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ltc2941.txt 2024-03-15T13:00:28.410 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/opp.txt 2024-03-15T13:00:28.412 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power-controller.txt 2024-03-15T13:00:28.414 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power_domain.txt 2024-03-15T13:00:28.417 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/renesas,sysc-rmobile.txt 2024-03-15T13:00:28.420 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ 2024-03-15T13:00:28.422 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/keystone-reset.txt 2024-03-15T13:00:28.424 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ltc2952-poweroff.txt 2024-03-15T13:00:28.426 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/st-reset.txt 2024-03-15T13:00:28.430 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/syscon-reboot.txt 2024-03-15T13:00:28.431 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rockchip-io-domain.txt 2024-03-15T13:00:28.434 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rx51-battery.txt 2024-03-15T13:00:28.437 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/twl-charger.txt 2024-03-15T13:00:28.439 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ 2024-03-15T13:00:28.442 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/ 2024-03-15T13:00:28.443 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/btemp.txt 2024-03-15T13:00:28.445 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/chargalg.txt 2024-03-15T13:00:28.448 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/charger.txt 2024-03-15T13:00:28.450 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/fg.txt 2024-03-15T13:00:28.453 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/axxia-reset.txt 2024-03-15T13:00:28.457 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/charger-manager.txt 2024-03-15T13:00:28.459 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/gpio-charger.txt 2024-03-15T13:00:28.461 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/imx-snvs-poweroff.txt 2024-03-15T13:00:28.464 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/lp8727_charger.txt 2024-03-15T13:00:28.466 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max17042_battery.txt 2024-03-15T13:00:28.469 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max8925_batter.txt 2024-03-15T13:00:28.471 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/msm-poweroff.txt 2024-03-15T13:00:28.473 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/olpc_battery.txt 2024-03-15T13:00:28.475 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/power_supply.txt 2024-03-15T13:00:28.477 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/qnap-poweroff.txt 2024-03-15T13:00:28.480 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/restart-poweroff.txt 2024-03-15T13:00:28.482 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/sbs_sbs-battery.txt 2024-03-15T13:00:28.484 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ti,bq24735.txt 2024-03-15T13:00:28.487 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/tps65090.txt 2024-03-15T13:00:28.489 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/ 2024-03-15T13:00:28.492 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ 2024-03-15T13:00:28.494 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/akebono.txt 2024-03-15T13:00:28.496 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/cpm.txt 2024-03-15T13:00:28.498 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/emac.txt 2024-03-15T13:00:28.501 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/hsta.txt 2024-03-15T13:00:28.503 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ndfc.txt 2024-03-15T13:00:28.506 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ppc440spe-adma.txt 2024-03-15T13:00:28.508 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/reboot.txt 2024-03-15T13:00:28.511 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ 2024-03-15T13:00:28.513 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/board.txt 2024-03-15T13:00:28.516 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cache_sram.txt 2024-03-15T13:00:28.519 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ccf.txt 2024-03-15T13:00:28.521 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/ 2024-03-15T13:00:28.524 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm.txt 2024-03-15T13:00:28.526 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/ 2024-03-15T13:00:28.529 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/brg.txt 2024-03-15T13:00:28.531 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/i2c.txt 2024-03-15T13:00:28.533 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/pic.txt 2024-03-15T13:00:28.536 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/usb.txt 2024-03-15T13:00:28.539 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/gpio.txt 2024-03-15T13:00:28.542 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/network.txt 2024-03-15T13:00:28.544 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe.txt 2024-03-15T13:00:28.547 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ 2024-03-15T13:00:28.550 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/firmware.txt 2024-03-15T13:00:28.552 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/par_io.txt 2024-03-15T13:00:28.555 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/pincfg.txt 2024-03-15T13:00:28.557 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ucc.txt 2024-03-15T13:00:28.560 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/usb.txt 2024-03-15T13:00:28.563 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/serial.txt 2024-03-15T13:00:28.566 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpus.txt 2024-03-15T13:00:28.569 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dcsr.txt 2024-03-15T13:00:28.571 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/diu.txt 2024-03-15T13:00:28.573 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dma.txt 2024-03-15T13:00:28.576 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ecm.txt 2024-03-15T13:00:28.578 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/fman.txt 2024-03-15T13:00:28.581 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/gtm.txt 2024-03-15T13:00:28.584 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/guts.txt 2024-03-15T13:00:28.586 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/interlaken-lac.txt 2024-03-15T13:00:28.589 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/l2cache.txt 2024-03-15T13:00:28.592 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/lbc.txt 2024-03-15T13:00:28.594 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcm.txt 2024-03-15T13:00:28.597 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcu-mpc8349emitx.txt 2024-03-15T13:00:28.599 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mem-ctrlr.txt 2024-03-15T13:00:28.601 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5121-psc.txt 2024-03-15T13:00:28.604 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5200.txt 2024-03-15T13:00:28.606 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt 2024-03-15T13:00:28.608 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-timer.txt 2024-03-15T13:00:28.611 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic.txt 2024-03-15T13:00:28.614 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/msi-pic.txt 2024-03-15T13:00:28.616 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pamu.txt 2024-03-15T13:00:28.619 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pmc.txt 2024-03-15T13:00:28.621 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/raideng.txt 2024-03-15T13:00:28.624 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio-rmu.txt 2024-03-15T13:00:28.626 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio.txt 2024-03-15T13:00:28.629 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/ 2024-03-15T13:00:28.632 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/gamecube.txt 2024-03-15T13:00:28.635 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/wii.txt 2024-03-15T13:00:28.637 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/ 2024-03-15T13:00:28.642 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/pps-gpio.txt 2024-03-15T13:00:28.644 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ 2024-03-15T13:00:28.648 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-hlcdc-pwm.txt 2024-03-15T13:00:28.650 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-pwm.txt 2024-03-15T13:00:28.652 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-tcb-pwm.txt 2024-03-15T13:00:28.655 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/bcm-kona-pwm.txt 2024-03-15T13:00:28.657 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/cirrus,clps711x-pwm.txt 2024-03-15T13:00:28.660 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/img-pwm.txt 2024-03-15T13:00:28.662 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/imx-pwm.txt 2024-03-15T13:00:28.664 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/lpc32xx-pwm.txt 2024-03-15T13:00:28.667 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/mxs-pwm.txt 2024-03-15T13:00:28.669 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nvidia,tegra20-pwm.txt 2024-03-15T13:00:28.672 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nxp,pca9685-pwm.txt 2024-03-15T13:00:28.674 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-bcm2835.txt 2024-03-15T13:00:28.676 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-fsl-ftm.txt 2024-03-15T13:00:28.678 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-lp3943.txt 2024-03-15T13:00:28.681 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-rockchip.txt 2024-03-15T13:00:28.684 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-samsung.txt 2024-03-15T13:00:28.686 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-st.txt 2024-03-15T13:00:28.688 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-sun4i.txt 2024-03-15T13:00:28.691 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiecap.txt 2024-03-15T13:00:28.693 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiehrpwm.txt 2024-03-15T13:00:28.696 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tipwmss.txt 2024-03-15T13:00:28.698 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm.txt 2024-03-15T13:00:28.700 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pxa-pwm.txt 2024-03-15T13:00:28.703 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/renesas,tpu-pwm.txt 2024-03-15T13:00:28.705 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/spear-pwm.txt 2024-03-15T13:00:28.707 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwm.txt 2024-03-15T13:00:28.710 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwmled.txt 2024-03-15T13:00:28.712 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/vt8500-pwm.txt 2024-03-15T13:00:28.714 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/ 2024-03-15T13:00:28.718 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/regmap.txt 2024-03-15T13:00:28.721 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ 2024-03-15T13:00:28.723 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm800.txt 2024-03-15T13:00:28.725 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm860x.txt 2024-03-15T13:00:28.728 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/act8865-regulator.txt 2024-03-15T13:00:28.730 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/anatop-regulator.txt 2024-03-15T13:00:28.733 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/as3722-regulator.txt 2024-03-15T13:00:28.737 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9210.txt 2024-03-15T13:00:28.738 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9211.txt 2024-03-15T13:00:28.740 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fan53555.txt 2024-03-15T13:00:28.742 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fixed-regulator.txt 2024-03-15T13:00:28.745 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/gpio-regulator.txt 2024-03-15T13:00:28.747 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/isl9305.txt 2024-03-15T13:00:28.750 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/lp872x.txt 2024-03-15T13:00:28.753 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ltc3589.txt 2024-03-15T13:00:28.755 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max1586-regulator.txt 2024-03-15T13:00:28.758 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max77802.txt 2024-03-15T13:00:28.760 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8660.txt 2024-03-15T13:00:28.762 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8907.txt 2024-03-15T13:00:28.765 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8925-regulator.txt 2024-03-15T13:00:28.767 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8952.txt 2024-03-15T13:00:28.770 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8973-regulator.txt 2024-03-15T13:00:28.772 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8997-regulator.txt 2024-03-15T13:00:28.775 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/mt6397-regulator.txt 2024-03-15T13:00:28.777 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/palmas-pmic.txt 2024-03-15T13:00:28.780 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pbias-regulator.txt 2024-03-15T13:00:28.782 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pfuze100.txt 2024-03-15T13:00:28.785 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pwm-regulator.txt 2024-03-15T13:00:28.787 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/regulator.txt 2024-03-15T13:00:28.791 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/s5m8767-regulator.txt 2024-03-15T13:00:28.793 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/sky81452-regulator.txt 2024-03-15T13:00:28.796 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ti-abb-regulator.txt 2024-03-15T13:00:28.798 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps51632-regulator.txt 2024-03-15T13:00:28.800 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps62360-regulator.txt 2024-03-15T13:00:28.802 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65090.txt 2024-03-15T13:00:28.805 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65217.txt 2024-03-15T13:00:28.807 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65218.txt 2024-03-15T13:00:28.809 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps6586x.txt 2024-03-15T13:00:28.812 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/twl-regulator.txt 2024-03-15T13:00:28.814 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/vexpress.txt 2024-03-15T13:00:28.816 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/ 2024-03-15T13:00:28.819 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt 2024-03-15T13:00:28.822 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/ 2024-03-15T13:00:28.825 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/allwinner,sunxi-clock-reset.txt 2024-03-15T13:00:28.827 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/fsl,imx-src.txt 2024-03-15T13:00:28.830 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/reset.txt 2024-03-15T13:00:28.832 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/sirf,rstc.txt 2024-03-15T13:00:28.834 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/socfpga-reset.txt 2024-03-15T13:00:28.841 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-picophyreset.txt 2024-03-15T13:00:28.843 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-powerdown.txt 2024-03-15T13:00:28.846 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-softreset.txt 2024-03-15T13:00:28.848 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/resource-names.txt 2024-03-15T13:00:28.851 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/ 2024-03-15T13:00:28.854 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/apm,rng.txt 2024-03-15T13:00:28.856 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/brcm,bcm2835.txt 2024-03-15T13:00:28.859 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/qcom,prng.txt 2024-03-15T13:00:28.861 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/ 2024-03-15T13:00:28.864 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/armada-380-rtc.txt 2024-03-15T13:00:28.866 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91rm9200-rtc.txt 2024-03-15T13:00:28.869 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91sam9-rtc.txt 2024-03-15T13:00:28.871 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dallas,ds1339.txt 2024-03-15T13:00:28.873 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dw-apb.txt 2024-03-15T13:00:28.875 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/haoyu,hym8563.txt 2024-03-15T13:00:28.878 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/imxdi-rtc.txt 2024-03-15T13:00:28.880 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/isil,isl12057.txt 2024-03-15T13:00:28.882 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/lpc32xx-rtc.txt 2024-03-15T13:00:28.885 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/maxim,ds1742.txt 2024-03-15T13:00:28.887 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/moxa,moxart-rtc.txt 2024-03-15T13:00:28.890 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.txt 2024-03-15T13:00:28.892 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nxp,rtc-2123.txt 2024-03-15T13:00:28.894 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/olpc-xo1-rtc.txt 2024-03-15T13:00:28.896 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/orion-rtc.txt 2024-03-15T13:00:28.899 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/pxa-rtc.txt 2024-03-15T13:00:28.902 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-cmos.txt 2024-03-15T13:00:28.904 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-omap.txt 2024-03-15T13:00:28.906 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-opal.txt 2024-03-15T13:00:28.909 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-palmas.txt 2024-03-15T13:00:28.911 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/s3c-rtc.txt 2024-03-15T13:00:28.914 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sa1100-rtc.txt 2024-03-15T13:00:28.916 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/snvs-rtc.txt 2024-03-15T13:00:28.918 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/spear-rtc.txt 2024-03-15T13:00:28.921 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/stmp3xxx-rtc.txt 2024-03-15T13:00:28.923 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sun6i-rtc.txt 2024-03-15T13:00:28.925 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sunxi-rtc.txt 2024-03-15T13:00:28.928 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/twl-rtc.txt 2024-03-15T13:00:28.930 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/via,vt8500-rtc.txt 2024-03-15T13:00:28.932 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/xgene-rtc.txt 2024-03-15T13:00:28.934 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/ 2024-03-15T13:00:28.937 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/ 2024-03-15T13:00:28.940 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/st33zp24-i2c.txt 2024-03-15T13:00:28.943 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/ 2024-03-15T13:00:28.946 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/8250.txt 2024-03-15T13:00:28.948 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_jtaguart.txt 2024-03-15T13:00:28.951 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_uart.txt 2024-03-15T13:00:28.953 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/arc-uart.txt 2024-03-15T13:00:28.956 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/atmel-usart.txt 2024-03-15T13:00:28.958 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/axis,etraxfs-uart.txt 2024-03-15T13:00:28.960 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/bcm63xx-uart.txt 2024-03-15T13:00:28.962 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cavium-uart.txt 2024-03-15T13:00:28.965 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cdns,uart.txt 2024-03-15T13:00:28.967 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cirrus,clps711x-uart.txt 2024-03-15T13:00:28.970 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/digicolor-usart.txt 2024-03-15T13:00:28.972 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/efm32-uart.txt 2024-03-15T13:00:28.974 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-imx-uart.txt 2024-03-15T13:00:28.976 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-lpuart.txt 2024-03-15T13:00:28.978 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-mxs-auart.txt 2024-03-15T13:00:28.981 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/lantiq_asc.txt 2024-03-15T13:00:28.983 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/maxim,max310x.txt 2024-03-15T13:00:28.985 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl,pxa-ssp.txt 2024-03-15T13:00:28.987 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl-serial.txt 2024-03-15T13:00:28.990 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mtk-uart.txt 2024-03-15T13:00:28.992 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nvidia,tegra20-hsuart.txt 2024-03-15T13:00:28.994 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp,sc16is7xx.txt 2024-03-15T13:00:28.997 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp-lpc32xx-hsuart.txt 2024-03-15T13:00:28.999 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/omap_serial.txt 2024-03-15T13:00:29.001 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/pl011.txt 2024-03-15T13:00:29.004 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qca,ar9330-uart.txt 2024-03-15T13:00:29.006 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uart.txt 2024-03-15T13:00:29.009 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uartdm.txt 2024-03-15T13:00:29.011 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/renesas,sci-serial.txt 2024-03-15T13:00:29.013 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/rs485.txt 2024-03-15T13:00:29.016 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/samsung_uart.txt 2024-03-15T13:00:29.018 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sirf-uart.txt 2024-03-15T13:00:29.021 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.txt 2024-03-15T13:00:29.023 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sprd-uart.txt 2024-03-15T13:00:29.025 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/st-asc.txt 2024-03-15T13:00:29.028 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/vt8500-uart.txt 2024-03-15T13:00:29.030 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/ 2024-03-15T13:00:29.033 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/allwinner,sun4i-ps2.txt 2024-03-15T13:00:29.035 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/altera_ps2.txt 2024-03-15T13:00:29.038 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/olpc,ap-sp.txt 2024-03-15T13:00:29.041 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/snps-arc_ps2.txt 2024-03-15T13:00:29.042 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ 2024-03-15T13:00:29.045 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/ 2024-03-15T13:00:29.048 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman-portals.txt 2024-03-15T13:00:29.051 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman.txt 2024-03-15T13:00:29.053 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman-portals.txt 2024-03-15T13:00:29.055 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman.txt 2024-03-15T13:00:29.058 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/ 2024-03-15T13:00:29.061 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/qcom,gsbi.txt 2024-03-15T13:00:29.064 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/ 2024-03-15T13:00:29.067 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-dma.txt 2024-03-15T13:00:29.070 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-qmss.txt 2024-03-15T13:00:29.072 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ 2024-03-15T13:00:29.076 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,adau1701.txt 2024-03-15T13:00:29.078 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-i2s.txt 2024-03-15T13:00:29.081 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-spdif-tx.txt 2024-03-15T13:00:29.083 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,ssm2602.txt 2024-03-15T13:00:29.085 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4104.txt 2024-03-15T13:00:29.091 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4554.c 2024-03-15T13:00:29.102 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4642.txt 2024-03-15T13:00:29.105 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak5386.txt 2024-03-15T13:00:29.109 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5623.txt 2024-03-15T13:00:29.112 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5632.txt 2024-03-15T13:00:29.115 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/armada-370db-audio.txt 2024-03-15T13:00:29.117 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/arndale.txt 2024-03-15T13:00:29.120 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-at91sam9g20ek-wm8731-audio.txt 2024-03-15T13:00:29.123 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-sam9x5-wm8731-audio.txt 2024-03-15T13:00:29.125 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-wm8904.txt 2024-03-15T13:00:29.128 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel_ac97c.txt 2024-03-15T13:00:29.131 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/bcm2835-i2s.txt 2024-03-15T13:00:29.134 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cdns,xtfpga-i2s.txt 2024-03-15T13:00:29.136 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs35l32.txt 2024-03-15T13:00:29.139 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4265.txt 2024-03-15T13:00:29.141 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4270.txt 2024-03-15T13:00:29.144 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4271.txt 2024-03-15T13:00:29.147 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l52.txt 2024-03-15T13:00:29.150 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l56.txt 2024-03-15T13:00:29.152 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l73.txt 2024-03-15T13:00:29.156 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42xx8.txt 2024-03-15T13:00:29.159 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/da9055.txt 2024-03-15T13:00:29.162 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-evm-audio.txt 2024-03-15T13:00:29.164 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-mcasp-audio.txt 2024-03-15T13:00:29.167 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/designware-i2s.txt 2024-03-15T13:00:29.170 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/es8328.txt 2024-03-15T13:00:29.174 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/eukrea-tlv320.txt 2024-03-15T13:00:29.176 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,asrc.txt 2024-03-15T13:00:29.179 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,esai.txt 2024-03-15T13:00:29.182 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,spdif.txt 2024-03-15T13:00:29.185 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,ssi.txt 2024-03-15T13:00:29.187 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-asoc-card.txt 2024-03-15T13:00:29.190 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-sai.txt 2024-03-15T13:00:29.193 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/hdmi.txt 2024-03-15T13:00:29.195 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-es8328.txt 2024-03-15T13:00:29.198 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-sgtl5000.txt 2024-03-15T13:00:29.201 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-spdif.txt 2024-03-15T13:00:29.204 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-wm8962.txt 2024-03-15T13:00:29.208 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audmux.txt 2024-03-15T13:00:29.211 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ingenic,jz4740-i2s.txt 2024-03-15T13:00:29.214 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98090.txt 2024-03-15T13:00:29.216 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98095.txt 2024-03-15T13:00:29.219 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98357a.txt 2024-03-15T13:00:29.222 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa-ssp.txt 2024-03-15T13:00:29.225 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa2xx-pcm.txt 2024-03-15T13:00:29.227 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mvebu-audio.txt 2024-03-15T13:00:29.230 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-audio-sgtl5000.txt 2024-03-15T13:00:29.233 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-saif.txt 2024-03-15T13:00:29.235 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nokia,rx51.txt 2024-03-15T13:00:29.238 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-alc5632.txt 2024-03-15T13:00:29.241 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-max98090.txt 2024-03-15T13:00:29.243 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5640.txt 2024-03-15T13:00:29.246 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5677.txt 2024-03-15T13:00:29.249 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-trimslice.txt 2024-03-15T13:00:29.251 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8753.txt 2024-03-15T13:00:29.254 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8903.txt 2024-03-15T13:00:29.256 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm9712.txt 2024-03-15T13:00:29.259 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-ac97.txt 2024-03-15T13:00:29.262 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-das.txt 2024-03-15T13:00:29.264 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-i2s.txt 2024-03-15T13:00:29.267 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-ahub.txt 2024-03-15T13:00:29.270 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-hda.txt 2024-03-15T13:00:29.273 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-i2s.txt 2024-03-15T13:00:29.275 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-abe-twl6040.txt 2024-03-15T13:00:29.278 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-dmic.txt 2024-03-15T13:00:29.280 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcbsp.txt 2024-03-15T13:00:29.283 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcpdm.txt 2024-03-15T13:00:29.286 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-twl4030.txt 2024-03-15T13:00:29.289 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm1792a.txt 2024-03-15T13:00:29.291 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm512x.txt 2024-03-15T13:00:29.294 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,fsi.txt 2024-03-15T13:00:29.297 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,rsnd.txt 2024-03-15T13:00:29.299 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rockchip-i2s.txt 2024-03-15T13:00:29.302 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5631.txt 2024-03-15T13:00:29.305 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5640.txt 2024-03-15T13:00:29.308 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5677.txt 2024-03-15T13:00:29.312 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,odroidx2-max98090.txt 2024-03-15T13:00:29.314 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,smdk-wm8994.txt 2024-03-15T13:00:29.317 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung-i2s.txt 2024-03-15T13:00:29.320 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sgtl5000.txt 2024-03-15T13:00:29.322 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/simple-card.txt 2024-03-15T13:00:29.325 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-codec.txt 2024-03-15T13:00:29.327 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-port.txt 2024-03-15T13:00:29.330 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio.txt 2024-03-15T13:00:29.333 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-usp.txt 2024-03-15T13:00:29.335 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/snow.txt 2024-03-15T13:00:29.338 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/soc-ac97link.txt 2024-03-15T13:00:29.340 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-receiver.txt 2024-03-15T13:00:29.343 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-transmitter.txt 2024-03-15T13:00:29.345 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm2518.txt 2024-03-15T13:00:29.348 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm4567.txt 2024-03-15T13:00:29.351 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta32x.txt 2024-03-15T13:00:29.353 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta350.txt 2024-03-15T13:00:29.356 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tas2552.txt 2024-03-15T13:00:29.358 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tdm-slot.txt 2024-03-15T13:00:29.361 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,pcm1681.txt 2024-03-15T13:00:29.363 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,tas5086.txt 2024-03-15T13:00:29.366 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic31xx.txt 2024-03-15T13:00:29.369 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic32x4.txt 2024-03-15T13:00:29.371 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic3x.txt 2024-03-15T13:00:29.374 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tpa6130a2.txt 2024-03-15T13:00:29.377 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ts3a227e.txt 2024-03-15T13:00:29.380 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-mop500.txt 2024-03-15T13:00:29.384 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-msp.txt 2024-03-15T13:00:29.387 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/widgets.txt 2024-03-15T13:00:29.389 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8510.txt 2024-03-15T13:00:29.392 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8523.txt 2024-03-15T13:00:29.395 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8580.txt 2024-03-15T13:00:29.397 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8711.txt 2024-03-15T13:00:29.400 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8728.txt 2024-03-15T13:00:29.403 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8731.txt 2024-03-15T13:00:29.406 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8737.txt 2024-03-15T13:00:29.408 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8741.txt 2024-03-15T13:00:29.411 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8750.txt 2024-03-15T13:00:29.414 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8753.txt 2024-03-15T13:00:29.417 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8770.txt 2024-03-15T13:00:29.420 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8776.txt 2024-03-15T13:00:29.423 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8804.txt 2024-03-15T13:00:29.426 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8903.txt 2024-03-15T13:00:29.428 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8904.txt 2024-03-15T13:00:29.431 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8960.txt 2024-03-15T13:00:29.434 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8962.txt 2024-03-15T13:00:29.437 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8994.txt 2024-03-15T13:00:29.440 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ 2024-03-15T13:00:29.443 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/brcm,bcm2835-spi.txt 2024-03-15T13:00:29.448 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/efm32-spi.txt 2024-03-15T13:00:29.451 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-imx-cspi.txt 2024-03-15T13:00:29.454 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-spi.txt 2024-03-15T13:00:29.457 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/mxs-spi.txt 2024-03-15T13:00:29.460 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra114-spi.txt 2024-03-15T13:00:29.462 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-sflash.txt 2024-03-15T13:00:29.467 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-slink.txt 2024-03-15T13:00:29.469 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/omap-spi.txt 2024-03-15T13:00:29.472 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/qcom,spi-qup.txt 2024-03-15T13:00:29.475 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-hspi.txt 2024-03-15T13:00:29.478 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-msiof.txt 2024-03-15T13:00:29.480 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/snps,dw-apb-ssi.txt 2024-03-15T13:00:29.483 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-bus.txt 2024-03-15T13:00:29.485 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-cadence.txt 2024-03-15T13:00:29.488 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-davinci.txt 2024-03-15T13:00:29.491 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-dw.txt 2024-03-15T13:00:29.493 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-fsl-dspi.txt 2024-03-15T13:00:29.496 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-gpio.txt 2024-03-15T13:00:29.499 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-img-spfi.txt 2024-03-15T13:00:29.501 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-meson.txt 2024-03-15T13:00:29.505 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-octeon.txt 2024-03-15T13:00:29.507 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-orion.txt 2024-03-15T13:00:29.510 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rockchip.txt 2024-03-15T13:00:29.512 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rspi.txt 2024-03-15T13:00:29.514 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-samsung.txt 2024-03-15T13:00:29.517 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sc18is602.txt 2024-03-15T13:00:29.519 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sirf.txt 2024-03-15T13:00:29.521 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-st-ssc.txt 2024-03-15T13:00:29.524 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun4i.txt 2024-03-15T13:00:29.526 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun6i.txt 2024-03-15T13:00:29.529 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-xtensa-xtfpga.txt 2024-03-15T13:00:29.531 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_altera.txt 2024-03-15T13:00:29.533 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_atmel.txt 2024-03-15T13:00:29.536 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_oc_tiny.txt 2024-03-15T13:00:29.538 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_pl022.txt 2024-03-15T13:00:29.540 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ti_qspi.txt 2024-03-15T13:00:29.542 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/ 2024-03-15T13:00:29.545 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/qcom,spmi-pmic-arb.txt 2024-03-15T13:00:29.547 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/spmi.txt 2024-03-15T13:00:29.550 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/ 2024-03-15T13:00:29.553 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/ 2024-03-15T13:00:29.555 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/ 2024-03-15T13:00:29.556 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/lpc32xx-adc.txt 2024-03-15T13:00:29.559 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/mxs-lradc.txt 2024-03-15T13:00:29.561 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/spear-adc.txt 2024-03-15T13:00:29.564 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/submitting-patches.txt 2024-03-15T13:00:29.568 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ 2024-03-15T13:00:29.570 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/armada-thermal.txt 2024-03-15T13:00:29.572 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/db8500-thermal.txt 2024-03-15T13:00:29.575 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/dove-thermal.txt 2024-03-15T13:00:29.577 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/exynos-thermal.txt 2024-03-15T13:00:29.579 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/imx-thermal.txt 2024-03-15T13:00:29.582 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/kirkwood-thermal.txt 2024-03-15T13:00:29.584 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rcar-thermal.txt 2024-03-15T13:00:29.587 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rockchip-thermal.txt 2024-03-15T13:00:29.589 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/spear-thermal.txt 2024-03-15T13:00:29.592 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/st-thermal.txt 2024-03-15T13:00:29.594 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/tegra-soctherm.txt 2024-03-15T13:00:29.596 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/thermal.txt 2024-03-15T13:00:29.599 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ti_soc_thermal.txt 2024-03-15T13:00:29.601 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ 2024-03-15T13:00:29.604 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun4i-timer.txt 2024-03-15T13:00:29.606 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun5i-a13-hstimer.txt 2024-03-15T13:00:29.608 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/amlogic,meson6-timer.txt 2024-03-15T13:00:29.610 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/arm,sp804.txt 2024-03-15T13:00:29.613 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/brcm,bcm2835-system-timer.txt 2024-03-15T13:00:29.615 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cadence,ttc-timer.txt 2024-03-15T13:00:29.617 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cirrus,clps711x-timer.txt 2024-03-15T13:00:29.620 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/digicolor-timer.txt 2024-03-15T13:00:29.621 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/energymicro,efm32-timer.txt 2024-03-15T13:00:29.624 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,ftm-timer.txt 2024-03-15T13:00:29.626 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,imxgpt.txt 2024-03-15T13:00:29.629 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/lsi,zevio-timer.txt 2024-03-15T13:00:29.631 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,armada-370-xp-timer.txt 2024-03-15T13:00:29.633 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,orion-timer.txt 2024-03-15T13:00:29.636 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/mediatek,mtk-timer.txt 2024-03-15T13:00:29.638 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/moxa,moxart-timer.txt 2024-03-15T13:00:29.641 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra20-timer.txt 2024-03-15T13:00:29.643 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra30-timer.txt 2024-03-15T13:00:29.645 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,cmt.txt 2024-03-15T13:00:29.647 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,mtu2.txt 2024-03-15T13:00:29.650 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,tmu.txt 2024-03-15T13:00:29.653 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/rockchip,rk3288-timer.txt 2024-03-15T13:00:29.654 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/samsung,exynos4210-mct.txt 2024-03-15T13:00:29.656 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/stericsson-u300-apptimer.txt 2024-03-15T13:00:29.659 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ti,keystone-timer.txt 2024-03-15T13:00:29.662 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ 2024-03-15T13:00:29.665 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ufshcd-pltfrm.txt 2024-03-15T13:00:29.667 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/unittest.txt 2024-03-15T13:00:29.670 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ 2024-03-15T13:00:29.673 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/am33xx-usb.txt 2024-03-15T13:00:29.675 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/atmel-usb.txt 2024-03-15T13:00:29.678 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-imx.txt 2024-03-15T13:00:29.680 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-qcom.txt 2024-03-15T13:00:29.682 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-usb2.txt 2024-03-15T13:00:29.685 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-zevio.txt 2024-03-15T13:00:29.687 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc2.txt 2024-03-15T13:00:29.689 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3-st.txt 2024-03-15T13:00:29.692 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3.txt 2024-03-15T13:00:29.695 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-omap.txt 2024-03-15T13:00:29.697 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-orion.txt 2024-03-15T13:00:29.700 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-st.txt 2024-03-15T13:00:29.702 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/exynos-usb.txt 2024-03-15T13:00:29.704 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/fsl-usb.txt 2024-03-15T13:00:29.707 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/generic.txt 2024-03-15T13:00:29.709 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/gr-udc.txt 2024-03-15T13:00:29.712 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/isp1301.txt 2024-03-15T13:00:29.714 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-phy.txt 2024-03-15T13:00:29.716 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-usb.txt 2024-03-15T13:00:29.719 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/lpc32xx-udc.txt 2024-03-15T13:00:29.721 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/msm-hsusb.txt 2024-03-15T13:00:29.723 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/mxs-phy.txt 2024-03-15T13:00:29.726 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-ehci.txt 2024-03-15T13:00:29.728 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-usb-phy.txt 2024-03-15T13:00:29.730 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-nxp.txt 2024-03-15T13:00:29.733 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-omap3.txt 2024-03-15T13:00:29.736 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-st.txt 2024-03-15T13:00:29.738 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/omap-usb.txt 2024-03-15T13:00:29.741 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/pxa-usb.txt 2024-03-15T13:00:29.768 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/qcom,dwc3.txt 2024-03-15T13:00:29.806 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/renesas_usbhs.txt 2024-03-15T13:00:29.815 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-hsotg.txt 2024-03-15T13:00:29.818 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-usbphy.txt 2024-03-15T13:00:29.821 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/spear-usb.txt 2024-03-15T13:00:29.824 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/twlxxxx-usb.txt 2024-03-15T13:00:29.827 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/udc-xilinx.txt 2024-03-15T13:00:29.830 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ehci.txt 2024-03-15T13:00:29.833 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-nop-xceiv.txt 2024-03-15T13:00:29.835 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ohci.txt 2024-03-15T13:00:29.838 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-uhci.txt 2024-03-15T13:00:29.851 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-xhci.txt 2024-03-15T13:00:30.023 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb3503.txt 2024-03-15T13:00:30.026 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usbmisc-imx.txt 2024-03-15T13:00:30.029 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ux500-usb.txt 2024-03-15T13:00:30.031 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/vendor-prefixes.txt 2024-03-15T13:00:30.034 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ 2024-03-15T13:00:30.037 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7123.txt 2024-03-15T13:00:30.039 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7511.txt 2024-03-15T13:00:30.041 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/analog-tv-connector.txt 2024-03-15T13:00:30.043 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/arm,pl11x.txt 2024-03-15T13:00:30.046 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/atmel,lcdc.txt 2024-03-15T13:00:30.048 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/ 2024-03-15T13:00:30.051 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/88pm860x.txt 2024-03-15T13:00:30.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:29 smithi002 ceph-mon[32060]: pgmap v50: 129 pgs: 129 active+clean; 86 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.1 MiB/s wr, 74 op/s 2024-03-15T13:00:30.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:29 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:00:30.054 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/gpio-backlight.txt 2024-03-15T13:00:30.056 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/lp855x.txt 2024-03-15T13:00:30.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:29 smithi184 ceph-mon[35993]: pgmap v50: 129 pgs: 129 active+clean; 86 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.1 MiB/s wr, 74 op/s 2024-03-15T13:00:30.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:29 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:00:30.059 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/max8925-backlight.txt 2024-03-15T13:00:30.061 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/pwm-backlight.txt 2024-03-15T13:00:30.063 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/tps65217-backlight.txt 2024-03-15T13:00:30.066 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ 2024-03-15T13:00:30.068 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ps8622.txt 2024-03-15T13:00:30.071 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ptn3460.txt 2024-03-15T13:00:30.073 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/cirrus,clps711x-fb.txt 2024-03-15T13:00:30.076 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/display-timing.txt 2024-03-15T13:00:30.079 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dvi-connector.txt 2024-03-15T13:00:30.081 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dw_hdmi-rockchip.txt 2024-03-15T13:00:30.083 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos7-decon.txt 2024-03-15T13:00:30.086 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dp.txt 2024-03-15T13:00:30.088 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dsim.txt 2024-03-15T13:00:30.091 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmi.txt 2024-03-15T13:00:30.093 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiddc.txt 2024-03-15T13:00:30.096 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiphy.txt 2024-03-15T13:00:30.098 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_mixer.txt 2024-03-15T13:00:30.100 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/fsl,imx-fb.txt 2024-03-15T13:00:30.102 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/hdmi-connector.txt 2024-03-15T13:00:30.105 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/lgphilips,lb035q02.txt 2024-03-15T13:00:30.108 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dpi.txt 2024-03-15T13:00:30.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:29 smithi149 ceph-mon[35927]: pgmap v50: 129 pgs: 129 active+clean; 86 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.1 MiB/s wr, 74 op/s 2024-03-15T13:00:30.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:29 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:00:30.110 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dsi-cm.txt 2024-03-15T13:00:30.112 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/renesas,du.txt 2024-03-15T13:00:30.114 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-drm.txt 2024-03-15T13:00:30.117 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-vop.txt 2024-03-15T13:00:30.119 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/samsung-fimd.txt 2024-03-15T13:00:30.122 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sharp,ls037v7dw01.txt 2024-03-15T13:00:30.124 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer-sunxi.txt 2024-03-15T13:00:30.126 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer.txt 2024-03-15T13:00:30.129 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sony,acx565akm.txt 2024-03-15T13:00:30.131 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1289fb.txt 2024-03-15T13:00:30.133 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1307fb.txt 2024-03-15T13:00:30.135 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/thine,thc63lvdm83d 2024-03-15T13:00:30.138 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,dra7-dss.txt 2024-03-15T13:00:30.140 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap-dss.txt 2024-03-15T13:00:30.142 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap2-dss.txt 2024-03-15T13:00:30.145 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap3-dss.txt 2024-03-15T13:00:30.147 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap4-dss.txt 2024-03-15T13:00:30.149 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap5-dss.txt 2024-03-15T13:00:30.152 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,opa362.txt 2024-03-15T13:00:30.154 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tfp410.txt 2024-03-15T13:00:30.156 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tpd12s015.txt 2024-03-15T13:00:30.159 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/toppoly,td028ttec1.txt 2024-03-15T13:00:30.161 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/tpo,td043mtea1.txt 2024-03-15T13:00:30.164 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/vga-connector.txt 2024-03-15T13:00:30.166 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/via,vt8500-fb.txt 2024-03-15T13:00:30.169 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,prizm-ge-rops.txt 2024-03-15T13:00:30.171 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,wm8505-fb.txt 2024-03-15T13:00:30.174 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/ 2024-03-15T13:00:30.177 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/mmio.txt 2024-03-15T13:00:30.180 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/ 2024-03-15T13:00:30.183 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/fsl-imx-owire.txt 2024-03-15T13:00:30.185 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/omap-hdq.txt 2024-03-15T13:00:30.187 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/w1-gpio.txt 2024-03-15T13:00:30.190 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ 2024-03-15T13:00:30.193 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-at91rm9200-wdt.txt 2024-03-15T13:00:30.196 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-wdt.txt 2024-03-15T13:00:30.198 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/brcm,bcm2835-pm-wdog.txt 2024-03-15T13:00:30.200 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/cadence-wdt.txt 2024-03-15T13:00:30.202 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/davinci-wdt.txt 2024-03-15T13:00:30.205 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/dw_wdt.txt 2024-03-15T13:00:30.207 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/fsl-imx-wdt.txt 2024-03-15T13:00:30.209 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/gpio-wdt.txt 2024-03-15T13:00:30.211 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/imgpdc-wdt.txt 2024-03-15T13:00:30.214 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ingenic,jz4740-wdt.txt 2024-03-15T13:00:30.216 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/marvel.txt 2024-03-15T13:00:30.219 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/men-a021-wdt.txt 2024-03-15T13:00:30.221 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/meson6-wdt.txt 2024-03-15T13:00:30.223 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/moxa,moxart-watchdog.txt 2024-03-15T13:00:30.226 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/mtk-wdt.txt 2024-03-15T13:00:30.228 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/of-xilinx-wdt.txt 2024-03-15T13:00:30.231 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/omap-wdt.txt 2024-03-15T13:00:30.233 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/pnx4008-wdt.txt 2024-03-15T13:00:30.235 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qca-ar7130-wdt.txt 2024-03-15T13:00:30.238 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qcom-wdt.txt 2024-03-15T13:00:30.240 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/rt2880-wdt.txt 2024-03-15T13:00:30.243 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/samsung-wdt.txt 2024-03-15T13:00:30.245 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sirfsoc_wdt.txt 2024-03-15T13:00:30.247 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/stericsson-coh901327.txt 2024-03-15T13:00:30.249 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sunxi-wdt.txt 2024-03-15T13:00:30.252 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/twl4030-wdt.txt 2024-03-15T13:00:30.254 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ 2024-03-15T13:00:30.256 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ce4100.txt 2024-03-15T13:00:30.259 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/interrupt.txt 2024-03-15T13:00:30.261 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/timer.txt 2024-03-15T13:00:30.264 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/xilinx.txt 2024-03-15T13:00:30.267 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/ 2024-03-15T13:00:30.269 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/xillybus.txt 2024-03-15T13:00:30.272 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/booting-without-of.txt 2024-03-15T13:00:30.276 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/changesets.txt 2024-03-15T13:00:30.278 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/dynamic-resolution-notes.txt 2024-03-15T13:00:30.281 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/of_selftest.txt 2024-03-15T13:00:30.284 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/overlay-notes.txt 2024-03-15T13:00:30.286 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/todo.txt 2024-03-15T13:00:30.289 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/devicetree/usage-model.txt 2024-03-15T13:00:30.291 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/digsig.txt 2024-03-15T13:00:30.293 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dma-buf-sharing.txt 2024-03-15T13:00:30.296 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dmaengine/ 2024-03-15T13:00:30.298 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dmaengine/00-INDEX 2024-03-15T13:00:30.300 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dmaengine/client.txt 2024-03-15T13:00:30.303 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dmaengine/dmatest.txt 2024-03-15T13:00:30.305 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dmaengine/provider.txt 2024-03-15T13:00:30.307 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dontdiff 2024-03-15T13:00:30.310 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/driver-model/ 2024-03-15T13:00:30.312 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/driver-model/binding.txt 2024-03-15T13:00:30.314 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/driver-model/bus.txt 2024-03-15T13:00:30.317 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/driver-model/class.txt 2024-03-15T13:00:30.319 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/driver-model/design-patterns.txt 2024-03-15T13:00:30.321 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/driver-model/device.txt 2024-03-15T13:00:30.323 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/driver-model/devres.txt 2024-03-15T13:00:30.326 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/driver-model/driver.txt 2024-03-15T13:00:30.328 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/driver-model/overview.txt 2024-03-15T13:00:30.330 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/driver-model/platform.txt 2024-03-15T13:00:30.333 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/driver-model/porting.txt 2024-03-15T13:00:30.335 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dvb/ 2024-03-15T13:00:30.338 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dvb/README.dvb-usb 2024-03-15T13:00:30.341 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dvb/avermedia.txt 2024-03-15T13:00:30.344 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dvb/bt8xx.txt 2024-03-15T13:00:30.346 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dvb/cards.txt 2024-03-15T13:00:30.348 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dvb/ci.txt 2024-03-15T13:00:30.351 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dvb/contributors.txt 2024-03-15T13:00:30.353 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dvb/faq.txt 2024-03-15T13:00:30.355 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dvb/get_dvb_firmware 2024-03-15T13:00:30.358 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dvb/lmedm04.txt 2024-03-15T13:00:30.360 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dvb/opera-firmware.txt 2024-03-15T13:00:30.363 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dvb/readme.txt 2024-03-15T13:00:30.365 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dvb/technisat.txt 2024-03-15T13:00:30.367 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dvb/ttusb-dec.txt 2024-03-15T13:00:30.369 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dvb/udev.txt 2024-03-15T13:00:30.371 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/dynamic-debug-howto.txt 2024-03-15T13:00:30.374 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/early-userspace/ 2024-03-15T13:00:30.376 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/early-userspace/README 2024-03-15T13:00:30.379 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/early-userspace/buffer-format.txt 2024-03-15T13:00:30.382 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/edac.txt 2024-03-15T13:00:30.385 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/efi-stub.txt 2024-03-15T13:00:30.387 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/eisa.txt 2024-03-15T13:00:30.390 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/email-clients.txt 2024-03-15T13:00:30.392 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/extcon/ 2024-03-15T13:00:30.395 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/extcon/porting-android-switch-class 2024-03-15T13:00:30.397 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fault-injection/ 2024-03-15T13:00:30.400 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fault-injection/fault-injection.txt 2024-03-15T13:00:30.402 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fault-injection/notifier-error-inject.txt 2024-03-15T13:00:30.405 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fault-injection/provoke-crashes.txt 2024-03-15T13:00:30.407 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/ 2024-03-15T13:00:30.409 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/00-INDEX 2024-03-15T13:00:30.411 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/api.txt 2024-03-15T13:00:30.414 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/arkfb.txt 2024-03-15T13:00:30.416 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/aty128fb.txt 2024-03-15T13:00:30.418 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/cirrusfb.txt 2024-03-15T13:00:30.421 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/cmap_xfbdev.txt 2024-03-15T13:00:30.423 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/deferred_io.txt 2024-03-15T13:00:30.426 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/efifb.txt 2024-03-15T13:00:30.428 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/ep93xx-fb.txt 2024-03-15T13:00:30.430 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/fbcon.txt 2024-03-15T13:00:30.433 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/framebuffer.txt 2024-03-15T13:00:30.435 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/gxfb.txt 2024-03-15T13:00:30.438 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/intel810.txt 2024-03-15T13:00:30.440 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/intelfb.txt 2024-03-15T13:00:30.443 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/internals.txt 2024-03-15T13:00:30.445 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/lxfb.txt 2024-03-15T13:00:30.447 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/matroxfb.txt 2024-03-15T13:00:30.449 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/metronomefb.txt 2024-03-15T13:00:30.451 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/modedb.txt 2024-03-15T13:00:30.453 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/pvr2fb.txt 2024-03-15T13:00:30.456 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/pxafb.txt 2024-03-15T13:00:30.458 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/s3fb.txt 2024-03-15T13:00:30.460 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/sa1100fb.txt 2024-03-15T13:00:30.463 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/sh7760fb.txt 2024-03-15T13:00:30.465 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/sisfb.txt 2024-03-15T13:00:30.467 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/sm501.txt 2024-03-15T13:00:30.469 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/sstfb.txt 2024-03-15T13:00:30.472 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/tgafb.txt 2024-03-15T13:00:30.474 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/tridentfb.txt 2024-03-15T13:00:30.476 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/udlfb.txt 2024-03-15T13:00:30.478 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/uvesafb.txt 2024-03-15T13:00:30.481 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/vesafb.txt 2024-03-15T13:00:30.483 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/viafb.modes 2024-03-15T13:00:30.486 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/viafb.txt 2024-03-15T13:00:30.489 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fb/vt8623fb.txt 2024-03-15T13:00:30.491 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/ 2024-03-15T13:00:30.493 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/.gitignore 2024-03-15T13:00:30.496 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/00-INDEX 2024-03-15T13:00:30.498 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/9p.txt 2024-03-15T13:00:30.501 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/Locking 2024-03-15T13:00:30.503 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/Makefile 2024-03-15T13:00:30.505 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/adfs.txt 2024-03-15T13:00:30.507 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/affs.txt 2024-03-15T13:00:30.509 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/afs.txt 2024-03-15T13:00:30.512 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/autofs4-mount-control.txt 2024-03-15T13:00:30.515 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/autofs4.txt 2024-03-15T13:00:30.517 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/automount-support.txt 2024-03-15T13:00:30.520 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/befs.txt 2024-03-15T13:00:30.522 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/bfs.txt 2024-03-15T13:00:30.524 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/btrfs.txt 2024-03-15T13:00:30.527 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/caching/ 2024-03-15T13:00:30.529 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/caching/backend-api.txt 2024-03-15T13:00:30.532 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/caching/cachefiles.txt 2024-03-15T13:00:30.535 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/caching/fscache.txt 2024-03-15T13:00:30.538 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/caching/netfs-api.txt 2024-03-15T13:00:30.541 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/caching/object.txt 2024-03-15T13:00:30.544 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/caching/operations.txt 2024-03-15T13:00:30.546 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/ceph.txt 2024-03-15T13:00:30.549 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/cifs/ 2024-03-15T13:00:30.551 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/cifs/AUTHORS 2024-03-15T13:00:30.553 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/cifs/CHANGES 2024-03-15T13:00:30.556 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/cifs/README 2024-03-15T13:00:30.559 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/cifs/TODO 2024-03-15T13:00:30.562 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/cifs/cifs.txt 2024-03-15T13:00:30.564 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/cifs/winucase_convert.pl 2024-03-15T13:00:30.567 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/coda.txt 2024-03-15T13:00:30.570 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/configfs/ 2024-03-15T13:00:30.572 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/configfs/Makefile 2024-03-15T13:00:30.575 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs.txt 2024-03-15T13:00:30.578 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_explicit.c 2024-03-15T13:00:30.580 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_macros.c 2024-03-15T13:00:30.583 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/cramfs.txt 2024-03-15T13:00:30.585 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/dax.txt 2024-03-15T13:00:30.588 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/debugfs.txt 2024-03-15T13:00:30.590 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/devpts.txt 2024-03-15T13:00:30.592 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/directory-locking 2024-03-15T13:00:30.595 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/dlmfs.txt 2024-03-15T13:00:30.597 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/dnotify.txt 2024-03-15T13:00:30.599 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/dnotify_test.c 2024-03-15T13:00:30.602 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/ecryptfs.txt 2024-03-15T13:00:30.604 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/efivarfs.txt 2024-03-15T13:00:30.606 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/exofs.txt 2024-03-15T13:00:30.609 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/ext2.txt 2024-03-15T13:00:30.611 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/ext3.txt 2024-03-15T13:00:30.613 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/ext4.txt 2024-03-15T13:00:30.616 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/f2fs.txt 2024-03-15T13:00:30.619 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/fiemap.txt 2024-03-15T13:00:30.622 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/files.txt 2024-03-15T13:00:30.625 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/fuse.txt 2024-03-15T13:00:30.627 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/gfs2-glocks.txt 2024-03-15T13:00:30.629 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/gfs2-uevents.txt 2024-03-15T13:00:30.632 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/gfs2.txt 2024-03-15T13:00:30.634 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/hfs.txt 2024-03-15T13:00:30.636 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/hfsplus.txt 2024-03-15T13:00:30.638 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/hpfs.txt 2024-03-15T13:00:30.642 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/inotify.txt 2024-03-15T13:00:30.643 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/isofs.txt 2024-03-15T13:00:30.645 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/jfs.txt 2024-03-15T13:00:30.647 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/locks.txt 2024-03-15T13:00:30.650 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/logfs.txt 2024-03-15T13:00:30.652 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/mandatory-locking.txt 2024-03-15T13:00:30.655 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/ncpfs.txt 2024-03-15T13:00:30.657 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nfs/ 2024-03-15T13:00:30.660 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nfs/00-INDEX 2024-03-15T13:00:30.661 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nfs/Exporting 2024-03-15T13:00:30.664 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nfs/fault_injection.txt 2024-03-15T13:00:30.666 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nfs/idmapper.txt 2024-03-15T13:00:30.669 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nfs/knfsd-stats.txt 2024-03-15T13:00:30.671 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs-rdma.txt 2024-03-15T13:00:30.673 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs.txt 2024-03-15T13:00:30.676 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs41-server.txt 2024-03-15T13:00:30.678 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsd-admin-interfaces.txt 2024-03-15T13:00:30.681 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsroot.txt 2024-03-15T13:00:30.684 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs-block-server.txt 2024-03-15T13:00:30.686 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs.txt 2024-03-15T13:00:30.688 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-cache.txt 2024-03-15T13:00:30.691 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-server-gss.txt 2024-03-15T13:00:30.693 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/nilfs2.txt 2024-03-15T13:00:30.696 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/ntfs.txt 2024-03-15T13:00:30.699 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/ocfs2.txt 2024-03-15T13:00:30.701 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/omfs.txt 2024-03-15T13:00:30.704 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/overlayfs.txt 2024-03-15T13:00:30.706 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/path-lookup.txt 2024-03-15T13:00:30.709 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/ 2024-03-15T13:00:30.711 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/design_notes.txt 2024-03-15T13:00:30.714 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/info.txt 2024-03-15T13:00:30.716 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/network_protocol.txt 2024-03-15T13:00:30.719 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/porting 2024-03-15T13:00:30.722 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/proc.txt 2024-03-15T13:00:30.726 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/qnx6.txt 2024-03-15T13:00:30.728 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/quota.txt 2024-03-15T13:00:30.730 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/ramfs-rootfs-initramfs.txt 2024-03-15T13:00:30.733 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/relay.txt 2024-03-15T13:00:30.736 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/romfs.txt 2024-03-15T13:00:30.738 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/seq_file.txt 2024-03-15T13:00:30.742 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/sharedsubtree.txt 2024-03-15T13:00:30.745 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/spufs.txt 2024-03-15T13:00:30.748 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/squashfs.txt 2024-03-15T13:00:30.751 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/sysfs-pci.txt 2024-03-15T13:00:30.753 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/sysfs-tagging.txt 2024-03-15T13:00:30.756 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/sysfs.txt 2024-03-15T13:00:30.758 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/sysv-fs.txt 2024-03-15T13:00:30.761 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/tmpfs.txt 2024-03-15T13:00:30.763 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/ubifs.txt 2024-03-15T13:00:30.766 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/udf.txt 2024-03-15T13:00:30.769 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/ufs.txt 2024-03-15T13:00:30.771 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/vfat.txt 2024-03-15T13:00:30.774 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/vfs.txt 2024-03-15T13:00:30.778 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/xfs-delayed-logging-design.txt 2024-03-15T13:00:30.782 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/xfs-self-describing-metadata.txt 2024-03-15T13:00:30.784 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/filesystems/xfs.txt 2024-03-15T13:00:30.788 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/firmware_class/ 2024-03-15T13:00:30.792 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/firmware_class/README 2024-03-15T13:00:30.795 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/firmware_class/hotplug-script 2024-03-15T13:00:30.797 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/flexible-arrays.txt 2024-03-15T13:00:30.801 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fmc/ 2024-03-15T13:00:30.804 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fmc/00-INDEX 2024-03-15T13:00:30.807 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fmc/API.txt 2024-03-15T13:00:30.809 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fmc/FMC-and-SDB.txt 2024-03-15T13:00:30.812 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fmc/carrier.txt 2024-03-15T13:00:30.814 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fmc/fmc-chardev.txt 2024-03-15T13:00:30.817 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fmc/fmc-fakedev.txt 2024-03-15T13:00:30.820 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fmc/fmc-trivial.txt 2024-03-15T13:00:30.822 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fmc/fmc-write-eeprom.txt 2024-03-15T13:00:30.825 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fmc/identifiers.txt 2024-03-15T13:00:30.827 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fmc/mezzanine.txt 2024-03-15T13:00:30.830 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/fmc/parameters.txt 2024-03-15T13:00:30.833 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/frv/ 2024-03-15T13:00:30.836 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/frv/README.txt 2024-03-15T13:00:30.838 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/frv/atomic-ops.txt 2024-03-15T13:00:30.841 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/frv/booting.txt 2024-03-15T13:00:30.843 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/frv/clock.txt 2024-03-15T13:00:30.846 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/frv/configuring.txt 2024-03-15T13:00:30.849 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/frv/features.txt 2024-03-15T13:00:30.864 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/frv/gdbinit 2024-03-15T13:00:31.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:30 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:00:31.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:30 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:00:31.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:30 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:00:31.138 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/frv/gdbstub.txt 2024-03-15T13:00:31.141 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/frv/kernel-ABI.txt 2024-03-15T13:00:31.143 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/frv/mmu-layout.txt 2024-03-15T13:00:31.146 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/futex-requeue-pi.txt 2024-03-15T13:00:31.149 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/gcov.txt 2024-03-15T13:00:31.151 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/gdb-kernel-debugging.txt 2024-03-15T13:00:31.153 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/gpio/ 2024-03-15T13:00:31.156 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/gpio/00-INDEX 2024-03-15T13:00:31.158 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/gpio/board.txt 2024-03-15T13:00:31.161 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/gpio/consumer.txt 2024-03-15T13:00:31.164 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/gpio/driver.txt 2024-03-15T13:00:31.166 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/gpio/gpio-legacy.txt 2024-03-15T13:00:31.169 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/gpio/gpio.txt 2024-03-15T13:00:31.171 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/gpio/sysfs.txt 2024-03-15T13:00:31.174 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hid/ 2024-03-15T13:00:31.176 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hid/hid-sensor.txt 2024-03-15T13:00:31.179 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hid/hid-transport.txt 2024-03-15T13:00:31.181 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hid/hiddev.txt 2024-03-15T13:00:31.183 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hid/hidraw.txt 2024-03-15T13:00:31.186 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hid/uhid.txt 2024-03-15T13:00:31.190 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/highuid.txt 2024-03-15T13:00:31.192 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hsi.txt 2024-03-15T13:00:31.193 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hw_random.txt 2024-03-15T13:00:31.196 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ 2024-03-15T13:00:31.198 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ab8500 2024-03-15T13:00:31.200 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/abituguru 2024-03-15T13:00:31.203 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/abituguru-datasheet 2024-03-15T13:00:31.205 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/abituguru3 2024-03-15T13:00:31.207 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/abx500 2024-03-15T13:00:31.209 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/acpi_power_meter 2024-03-15T13:00:31.212 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ad7314 2024-03-15T13:00:31.214 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/adc128d818 2024-03-15T13:00:31.217 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/adm1021 2024-03-15T13:00:31.219 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/adm1025 2024-03-15T13:00:31.221 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/adm1026 2024-03-15T13:00:31.224 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/adm1031 2024-03-15T13:00:31.226 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/adm1275 2024-03-15T13:00:31.228 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/adm9240 2024-03-15T13:00:31.231 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ads1015 2024-03-15T13:00:31.233 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ads7828 2024-03-15T13:00:31.236 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/adt7410 2024-03-15T13:00:31.238 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/adt7411 2024-03-15T13:00:31.240 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/adt7462 2024-03-15T13:00:31.242 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/adt7470 2024-03-15T13:00:31.244 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/adt7475 2024-03-15T13:00:31.247 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/amc6821 2024-03-15T13:00:31.249 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/asb100 2024-03-15T13:00:31.252 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/asc7621 2024-03-15T13:00:31.254 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/coretemp 2024-03-15T13:00:31.256 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/da9052 2024-03-15T13:00:31.258 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/da9055 2024-03-15T13:00:31.261 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/dme1737 2024-03-15T13:00:31.265 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ds1621 2024-03-15T13:00:31.266 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ds620 2024-03-15T13:00:31.268 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/emc1403 2024-03-15T13:00:31.270 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/emc2103 2024-03-15T13:00:31.272 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/emc6w201 2024-03-15T13:00:31.275 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/f71805f 2024-03-15T13:00:31.277 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/f71882fg 2024-03-15T13:00:31.279 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/fam15h_power 2024-03-15T13:00:31.282 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/g760a 2024-03-15T13:00:31.284 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/g762 2024-03-15T13:00:31.286 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/gl518sm 2024-03-15T13:00:31.289 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/hih6130 2024-03-15T13:00:31.293 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/htu21 2024-03-15T13:00:31.294 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/hwmon-kernel-api.txt 2024-03-15T13:00:31.296 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ibmaem 2024-03-15T13:00:31.298 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ibmpowernv 2024-03-15T13:00:31.300 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ina209 2024-03-15T13:00:31.303 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ina2xx 2024-03-15T13:00:31.305 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/it87 2024-03-15T13:00:31.307 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/jc42 2024-03-15T13:00:31.310 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/k10temp 2024-03-15T13:00:31.312 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/k8temp 2024-03-15T13:00:31.314 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lineage-pem 2024-03-15T13:00:31.316 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm25066 2024-03-15T13:00:31.318 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm63 2024-03-15T13:00:31.321 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm70 2024-03-15T13:00:31.323 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm73 2024-03-15T13:00:31.325 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm75 2024-03-15T13:00:31.327 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm77 2024-03-15T13:00:31.330 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm78 2024-03-15T13:00:31.332 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm80 2024-03-15T13:00:31.335 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm83 2024-03-15T13:00:31.337 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm85 2024-03-15T13:00:31.340 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm87 2024-03-15T13:00:31.343 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm90 2024-03-15T13:00:31.345 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm92 2024-03-15T13:00:31.347 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm93 2024-03-15T13:00:31.350 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm95234 2024-03-15T13:00:31.352 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/lm95245 2024-03-15T13:00:31.354 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ltc2945 2024-03-15T13:00:31.357 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ltc2978 2024-03-15T13:00:31.359 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ltc4151 2024-03-15T13:00:31.361 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ltc4215 2024-03-15T13:00:31.364 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ltc4245 2024-03-15T13:00:31.367 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ltc4260 2024-03-15T13:00:31.369 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ltc4261 2024-03-15T13:00:31.371 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/max16064 2024-03-15T13:00:31.373 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/max16065 2024-03-15T13:00:31.376 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/max1619 2024-03-15T13:00:31.378 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/max1668 2024-03-15T13:00:31.380 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/max197 2024-03-15T13:00:31.382 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/max34440 2024-03-15T13:00:31.384 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/max6639 2024-03-15T13:00:31.386 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/max6642 2024-03-15T13:00:31.389 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/max6650 2024-03-15T13:00:31.391 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/max6697 2024-03-15T13:00:31.393 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/max8688 2024-03-15T13:00:31.396 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/mc13783-adc 2024-03-15T13:00:31.398 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/mcp3021 2024-03-15T13:00:31.400 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/menf21bmc 2024-03-15T13:00:31.403 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/nct6683 2024-03-15T13:00:31.405 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/nct6775 2024-03-15T13:00:31.408 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/nct7802 2024-03-15T13:00:31.410 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ntc_thermistor 2024-03-15T13:00:31.413 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/pc87360 2024-03-15T13:00:31.415 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/pc87427 2024-03-15T13:00:31.417 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/pcf8591 2024-03-15T13:00:31.420 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/pmbus 2024-03-15T13:00:31.423 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/pmbus-core 2024-03-15T13:00:31.425 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/powr1220 2024-03-15T13:00:31.427 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/pwm-fan 2024-03-15T13:00:31.430 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/sch5627 2024-03-15T13:00:31.432 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/sch5636 2024-03-15T13:00:31.434 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/sht15 2024-03-15T13:00:31.437 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/sht21 2024-03-15T13:00:31.439 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/shtc1 2024-03-15T13:00:31.441 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/sis5595 2024-03-15T13:00:31.443 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/smm665 2024-03-15T13:00:31.446 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/smsc47b397 2024-03-15T13:00:31.448 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/smsc47m1 2024-03-15T13:00:31.451 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/smsc47m192 2024-03-15T13:00:31.453 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/submitting-patches 2024-03-15T13:00:31.455 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/sysfs-interface 2024-03-15T13:00:31.458 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/thmc50 2024-03-15T13:00:31.460 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/tmp102 2024-03-15T13:00:31.463 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/tmp103 2024-03-15T13:00:31.465 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/tmp401 2024-03-15T13:00:31.467 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/tmp421 2024-03-15T13:00:31.469 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/tps40422 2024-03-15T13:00:31.472 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/twl4030-madc-hwmon 2024-03-15T13:00:31.474 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ucd9000 2024-03-15T13:00:31.477 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/ucd9200 2024-03-15T13:00:31.479 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/userspace-tools 2024-03-15T13:00:31.481 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/vexpress 2024-03-15T13:00:31.484 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/via686a 2024-03-15T13:00:31.486 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/vt1211 2024-03-15T13:00:31.488 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/w83627ehf 2024-03-15T13:00:31.490 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/w83627hf 2024-03-15T13:00:31.492 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/w83781d 2024-03-15T13:00:31.494 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/w83791d 2024-03-15T13:00:31.497 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/w83792d 2024-03-15T13:00:31.499 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/w83793 2024-03-15T13:00:31.502 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/w83795 2024-03-15T13:00:31.504 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/w83l785ts 2024-03-15T13:00:31.506 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/w83l786ng 2024-03-15T13:00:31.508 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/wm831x 2024-03-15T13:00:31.510 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/wm8350 2024-03-15T13:00:31.513 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwmon/zl6100 2024-03-15T13:00:31.515 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/hwspinlock.txt 2024-03-15T13:00:31.518 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/ 2024-03-15T13:00:31.521 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/ 2024-03-15T13:00:31.523 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1535 2024-03-15T13:00:31.525 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1563 2024-03-15T13:00:31.528 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali15x3 2024-03-15T13:00:31.530 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd756 2024-03-15T13:00:31.533 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd8111 2024-03-15T13:00:31.535 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-diolan-u2c 2024-03-15T13:00:31.538 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-i801 2024-03-15T13:00:31.540 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ismt 2024-03-15T13:00:31.542 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-nforce2 2024-03-15T13:00:31.544 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ocores 2024-03-15T13:00:31.547 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport 2024-03-15T13:00:31.549 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport-light 2024-03-15T13:00:31.551 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-pca-isa 2024-03-15T13:00:31.553 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-piix4 2024-03-15T13:00:31.556 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis5595 2024-03-15T13:00:31.558 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis630 2024-03-15T13:00:31.560 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis96x 2024-03-15T13:00:31.563 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-taos-evm 2024-03-15T13:00:31.565 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-via 2024-03-15T13:00:31.568 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-viapro 2024-03-15T13:00:31.570 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/busses/scx200_acb 2024-03-15T13:00:31.572 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/dev-interface 2024-03-15T13:00:31.575 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/fault-codes 2024-03-15T13:00:31.577 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/functionality 2024-03-15T13:00:31.580 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/i2c-protocol 2024-03-15T13:00:31.583 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/i2c-stub 2024-03-15T13:00:31.585 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/instantiating-devices 2024-03-15T13:00:31.587 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/muxes/ 2024-03-15T13:00:31.589 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/muxes/i2c-mux-gpio 2024-03-15T13:00:31.591 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/old-module-parameters 2024-03-15T13:00:31.594 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/smbus-protocol 2024-03-15T13:00:31.597 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/summary 2024-03-15T13:00:31.599 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/ten-bit-addresses 2024-03-15T13:00:31.601 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/upgrading-clients 2024-03-15T13:00:31.604 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2c/writing-clients 2024-03-15T13:00:31.606 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2o/ 2024-03-15T13:00:31.609 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2o/README 2024-03-15T13:00:31.611 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/i2o/ioctl 2024-03-15T13:00:31.614 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ia64/ 2024-03-15T13:00:31.616 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ia64/.gitignore 2024-03-15T13:00:31.619 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ia64/IRQ-redir.txt 2024-03-15T13:00:31.621 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ia64/Makefile 2024-03-15T13:00:31.623 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ia64/README 2024-03-15T13:00:31.625 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ia64/aliasing-test.c 2024-03-15T13:00:31.627 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ia64/aliasing.txt 2024-03-15T13:00:31.629 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ia64/efirtc.txt 2024-03-15T13:00:31.632 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ia64/err_inject.txt 2024-03-15T13:00:31.635 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ia64/fsys.txt 2024-03-15T13:00:31.637 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ia64/mca.txt 2024-03-15T13:00:31.639 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ia64/serial.txt 2024-03-15T13:00:31.642 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ia64/xen.txt 2024-03-15T13:00:31.644 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ide/ 2024-03-15T13:00:31.647 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ide/00-INDEX 2024-03-15T13:00:31.649 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-cd.1994-2004 2024-03-15T13:00:31.652 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-floppy.1996-2002 2024-03-15T13:00:31.656 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-tape.1995-2002 2024-03-15T13:00:31.659 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ide/ide-tape.txt 2024-03-15T13:00:31.661 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ide/ide.txt 2024-03-15T13:00:31.663 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ide/warm-plug-howto.txt 2024-03-15T13:00:31.666 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/infiniband/ 2024-03-15T13:00:31.669 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/infiniband/core_locking.txt 2024-03-15T13:00:31.671 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/infiniband/ipoib.txt 2024-03-15T13:00:31.673 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/infiniband/sysfs.txt 2024-03-15T13:00:31.676 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/infiniband/user_mad.txt 2024-03-15T13:00:31.678 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/infiniband/user_verbs.txt 2024-03-15T13:00:31.680 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/init.txt 2024-03-15T13:00:31.682 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/initrd.txt 2024-03-15T13:00:31.685 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/ 2024-03-15T13:00:31.688 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/alps.txt 2024-03-15T13:00:31.690 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/amijoy.txt 2024-03-15T13:00:31.692 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/appletouch.txt 2024-03-15T13:00:31.694 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/atarikbd.txt 2024-03-15T13:00:31.697 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/bcm5974.txt 2024-03-15T13:00:31.701 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/cd32.txt 2024-03-15T13:00:31.703 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/cma3000_d0x.txt 2024-03-15T13:00:31.706 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/cs461x.txt 2024-03-15T13:00:31.708 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/edt-ft5x06.txt 2024-03-15T13:00:31.711 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/elantech.txt 2024-03-15T13:00:31.714 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/event-codes.txt 2024-03-15T13:00:31.716 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/ff.txt 2024-03-15T13:00:31.719 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/gamepad.txt 2024-03-15T13:00:31.721 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/gameport-programming.txt 2024-03-15T13:00:31.724 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/gpio-tilt.txt 2024-03-15T13:00:31.726 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/iforce-protocol.txt 2024-03-15T13:00:31.728 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/input-programming.txt 2024-03-15T13:00:31.730 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/input.txt 2024-03-15T13:00:31.733 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/interactive.fig 2024-03-15T13:00:31.735 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/joystick-api.txt 2024-03-15T13:00:31.738 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/joystick-parport.txt 2024-03-15T13:00:31.741 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/joystick.txt 2024-03-15T13:00:31.744 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/multi-touch-protocol.txt 2024-03-15T13:00:31.746 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/notifier.txt 2024-03-15T13:00:31.749 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/ntrig.txt 2024-03-15T13:00:31.751 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/rotary-encoder.txt 2024-03-15T13:00:31.754 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/sentelic.txt 2024-03-15T13:00:31.757 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/shape.fig 2024-03-15T13:00:31.760 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/walkera0701.txt 2024-03-15T13:00:31.762 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/xpad.txt 2024-03-15T13:00:31.765 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/input/yealink.txt 2024-03-15T13:00:31.767 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/intel_txt.txt 2024-03-15T13:00:31.770 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/io-mapping.txt 2024-03-15T13:00:31.772 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/io_ordering.txt 2024-03-15T13:00:31.775 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ioctl/ 2024-03-15T13:00:31.777 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ioctl/00-INDEX 2024-03-15T13:00:31.779 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ioctl/botching-up-ioctls.txt 2024-03-15T13:00:31.781 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ioctl/cdrom.txt 2024-03-15T13:00:31.783 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ioctl/hdio.txt 2024-03-15T13:00:31.786 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ioctl/ioctl-decoding.txt 2024-03-15T13:00:31.788 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ioctl/ioctl-number.txt 2024-03-15T13:00:31.791 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/iostats.txt 2024-03-15T13:00:31.796 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/irqflags-tracing.txt 2024-03-15T13:00:31.798 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isapnp.txt 2024-03-15T13:00:31.800 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/ 2024-03-15T13:00:31.803 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/00-INDEX 2024-03-15T13:00:31.806 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/CREDITS 2024-03-15T13:00:31.808 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/HiSax.cert 2024-03-15T13:00:31.810 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/INTERFACE 2024-03-15T13:00:31.813 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.CAPI 2024-03-15T13:00:31.815 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.fax 2024-03-15T13:00:31.817 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README 2024-03-15T13:00:31.820 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.FAQ 2024-03-15T13:00:31.822 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.HiSax 2024-03-15T13:00:31.825 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.act2000 2024-03-15T13:00:31.827 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.audio 2024-03-15T13:00:31.830 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.avmb1 2024-03-15T13:00:31.832 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.concap 2024-03-15T13:00:31.835 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.diversion 2024-03-15T13:00:31.837 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.fax 2024-03-15T13:00:31.840 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.gigaset 2024-03-15T13:00:31.842 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.hfc-pci 2024-03-15T13:00:31.845 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.hysdn 2024-03-15T13:00:31.847 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.icn 2024-03-15T13:00:31.849 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.mISDN 2024-03-15T13:00:31.851 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.pcbit 2024-03-15T13:00:31.853 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.sc 2024-03-15T13:00:31.856 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.syncppp 2024-03-15T13:00:31.858 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/README.x25 2024-03-15T13:00:31.860 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/isdn/syncPPP.FAQ 2024-03-15T13:00:31.863 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ja_JP/ 2024-03-15T13:00:32.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:31 smithi002 ceph-mon[32060]: pgmap v51: 129 pgs: 129 active+clean; 94 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 7.9 MiB/s wr, 216 op/s 2024-03-15T13:00:32.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:31 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:00:32.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:31 smithi184 ceph-mon[35993]: pgmap v51: 129 pgs: 129 active+clean; 94 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 7.9 MiB/s wr, 216 op/s 2024-03-15T13:00:32.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:31 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:00:32.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:31 smithi149 ceph-mon[35927]: pgmap v51: 129 pgs: 129 active+clean; 94 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 7.9 MiB/s wr, 216 op/s 2024-03-15T13:00:32.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:31 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:00:32.227 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ja_JP/HOWTO 2024-03-15T13:00:32.231 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ja_JP/SubmitChecklist 2024-03-15T13:00:32.234 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ja_JP/SubmittingPatches 2024-03-15T13:00:32.237 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ja_JP/stable_api_nonsense.txt 2024-03-15T13:00:32.240 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ja_JP/stable_kernel_rules.txt 2024-03-15T13:00:32.242 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/java.txt 2024-03-15T13:00:32.245 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kasan.txt 2024-03-15T13:00:32.248 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kbuild/ 2024-03-15T13:00:32.251 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kbuild/00-INDEX 2024-03-15T13:00:32.254 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kbuild/headers_install.txt 2024-03-15T13:00:32.257 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kbuild/kbuild.txt 2024-03-15T13:00:32.260 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kbuild/kconfig-language.txt 2024-03-15T13:00:32.262 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kbuild/kconfig.txt 2024-03-15T13:00:32.265 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kbuild/makefiles.txt 2024-03-15T13:00:32.268 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kbuild/modules.txt 2024-03-15T13:00:32.271 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kdump/ 2024-03-15T13:00:32.274 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kdump/gdbmacros.txt 2024-03-15T13:00:32.278 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kdump/kdump.txt 2024-03-15T13:00:32.280 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kernel-doc-nano-HOWTO.txt 2024-03-15T13:00:32.284 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kernel-docs.txt 2024-03-15T13:00:32.287 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kernel-parameters.txt 2024-03-15T13:00:32.293 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kernel-per-CPU-kthreads.txt 2024-03-15T13:00:32.296 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kmemcheck.txt 2024-03-15T13:00:32.299 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kmemleak.txt 2024-03-15T13:00:32.302 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ko_KR/ 2024-03-15T13:00:32.304 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ko_KR/HOWTO 2024-03-15T13:00:32.308 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ko_KR/stable_api_nonsense.txt 2024-03-15T13:00:32.311 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kobject.txt 2024-03-15T13:00:32.315 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kprobes.txt 2024-03-15T13:00:32.318 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kref.txt 2024-03-15T13:00:32.321 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/kselftest.txt 2024-03-15T13:00:32.323 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/laptops/ 2024-03-15T13:00:32.326 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/laptops/.gitignore 2024-03-15T13:00:32.329 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/laptops/00-INDEX 2024-03-15T13:00:32.332 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/laptops/Makefile 2024-03-15T13:00:32.334 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/laptops/asus-laptop.txt 2024-03-15T13:00:32.336 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/laptops/disk-shock-protection.txt 2024-03-15T13:00:32.339 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/laptops/dslm.c 2024-03-15T13:00:32.342 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/laptops/freefall.c 2024-03-15T13:00:32.344 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/laptops/laptop-mode.txt 2024-03-15T13:00:32.348 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/laptops/sony-laptop.txt 2024-03-15T13:00:32.350 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/laptops/sonypi.txt 2024-03-15T13:00:32.353 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/laptops/thinkpad-acpi.txt 2024-03-15T13:00:32.356 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/laptops/toshiba_haps.txt 2024-03-15T13:00:32.359 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ldm.txt 2024-03-15T13:00:32.362 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/leds/ 2024-03-15T13:00:32.364 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/leds/00-INDEX 2024-03-15T13:00:32.367 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/leds/leds-blinkm.txt 2024-03-15T13:00:32.369 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/leds/leds-class.txt 2024-03-15T13:00:32.372 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/leds/leds-lm3556.txt 2024-03-15T13:00:32.375 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/leds/leds-lp3944.txt 2024-03-15T13:00:32.378 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/leds/leds-lp5521.txt 2024-03-15T13:00:32.381 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/leds/leds-lp5523.txt 2024-03-15T13:00:32.384 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/leds/leds-lp5562.txt 2024-03-15T13:00:32.387 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/leds/leds-lp55xx.txt 2024-03-15T13:00:32.390 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/leds/ledtrig-oneshot.txt 2024-03-15T13:00:32.392 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/leds/ledtrig-transient.txt 2024-03-15T13:00:32.395 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/local_ops.txt 2024-03-15T13:00:32.399 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/locking/ 2024-03-15T13:00:32.401 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/locking/00-INDEX 2024-03-15T13:00:32.405 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/locking/lglock.txt 2024-03-15T13:00:32.408 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/locking/lockdep-design.txt 2024-03-15T13:00:32.411 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/locking/lockstat.txt 2024-03-15T13:00:32.414 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/locking/locktorture.txt 2024-03-15T13:00:32.416 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/locking/mutex-design.txt 2024-03-15T13:00:32.419 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/locking/rt-mutex-design.txt 2024-03-15T13:00:32.423 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/locking/rt-mutex.txt 2024-03-15T13:00:32.426 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/locking/spinlocks.txt 2024-03-15T13:00:32.428 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/locking/ww-mutex-design.txt 2024-03-15T13:00:32.432 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/lockup-watchdogs.txt 2024-03-15T13:00:32.435 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/logo.gif 2024-03-15T13:00:32.438 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/logo.txt 2024-03-15T13:00:32.440 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/lzo.txt 2024-03-15T13:00:32.443 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/m68k/ 2024-03-15T13:00:32.446 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/m68k/00-INDEX 2024-03-15T13:00:32.449 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/m68k/README.buddha 2024-03-15T13:00:32.452 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/m68k/kernel-options.txt 2024-03-15T13:00:32.455 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/magic-number.txt 2024-03-15T13:00:32.458 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mailbox.txt 2024-03-15T13:00:32.461 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/md.txt 2024-03-15T13:00:32.463 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/media-framework.txt 2024-03-15T13:00:32.467 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/memory-barriers.txt 2024-03-15T13:00:32.656 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/memory-devices/ 2024-03-15T13:00:32.658 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/memory-devices/ti-emif.txt 2024-03-15T13:00:32.663 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/memory-hotplug.txt 2024-03-15T13:00:32.667 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/metag/ 2024-03-15T13:00:32.669 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/metag/00-INDEX 2024-03-15T13:00:32.672 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/metag/kernel-ABI.txt 2024-03-15T13:00:32.674 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mic/ 2024-03-15T13:00:32.676 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mic/Makefile 2024-03-15T13:00:32.679 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mic/mic_overview.txt 2024-03-15T13:00:32.681 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mic/mpssd/ 2024-03-15T13:00:32.683 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mic/mpssd/.gitignore 2024-03-15T13:00:32.686 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mic/mpssd/Makefile 2024-03-15T13:00:32.688 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mic/mpssd/micctrl 2024-03-15T13:00:32.690 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mic/mpssd/mpss 2024-03-15T13:00:32.693 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.c 2024-03-15T13:00:32.696 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.h 2024-03-15T13:00:32.699 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mic/mpssd/sysfs.c 2024-03-15T13:00:32.701 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mips/ 2024-03-15T13:00:32.704 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mips/00-INDEX 2024-03-15T13:00:32.706 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mips/AU1xxx_IDE.README 2024-03-15T13:00:32.709 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/ 2024-03-15T13:00:32.711 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/Makefile 2024-03-15T13:00:32.714 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/ad525x_dpot.txt 2024-03-15T13:00:32.716 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/apds990x.txt 2024-03-15T13:00:32.718 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/bh1770glc.txt 2024-03-15T13:00:32.721 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/c2port.txt 2024-03-15T13:00:32.725 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/eeprom 2024-03-15T13:00:32.725 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/ics932s401 2024-03-15T13:00:32.727 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/isl29003 2024-03-15T13:00:32.729 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/lis3lv02d 2024-03-15T13:00:32.731 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/max6875 2024-03-15T13:00:32.734 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/mei/ 2024-03-15T13:00:32.736 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/mei/.gitignore 2024-03-15T13:00:32.738 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/mei/Makefile 2024-03-15T13:00:32.741 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/mei/TODO 2024-03-15T13:00:32.743 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-amt-version.c 2024-03-15T13:00:32.746 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-client-bus.txt 2024-03-15T13:00:32.748 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei.txt 2024-03-15T13:00:32.751 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/misc-devices/spear-pcie-gadget.txt 2024-03-15T13:00:32.754 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mmc/ 2024-03-15T13:00:32.757 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mmc/00-INDEX 2024-03-15T13:00:32.760 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mmc/mmc-async-req.txt 2024-03-15T13:00:32.762 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-attrs.txt 2024-03-15T13:00:32.764 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-parts.txt 2024-03-15T13:00:32.766 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mn10300/ 2024-03-15T13:00:32.769 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mn10300/ABI.txt 2024-03-15T13:00:32.771 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mn10300/compartmentalisation.txt 2024-03-15T13:00:32.773 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/module-signing.txt 2024-03-15T13:00:32.776 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mono.txt 2024-03-15T13:00:32.778 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mtd/ 2024-03-15T13:00:32.780 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mtd/nand/ 2024-03-15T13:00:32.783 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt 2024-03-15T13:00:32.785 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mtd/nand_ecc.txt 2024-03-15T13:00:32.790 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/mtd/spi-nor.txt 2024-03-15T13:00:32.792 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/namespaces/ 2024-03-15T13:00:32.794 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/namespaces/compatibility-list.txt 2024-03-15T13:00:32.797 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/namespaces/resource-control.txt 2024-03-15T13:00:32.799 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/netlabel/ 2024-03-15T13:00:32.802 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/netlabel/00-INDEX 2024-03-15T13:00:32.804 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/netlabel/cipso_ipv4.txt 2024-03-15T13:00:32.807 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/netlabel/draft-ietf-cipso-ipsecurity-01.txt 2024-03-15T13:00:32.809 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/netlabel/introduction.txt 2024-03-15T13:00:32.811 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/netlabel/lsm_interface.txt 2024-03-15T13:00:32.813 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ 2024-03-15T13:00:32.816 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/00-INDEX 2024-03-15T13:00:32.818 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/3c509.txt 2024-03-15T13:00:32.821 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/6pack.txt 2024-03-15T13:00:32.823 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/LICENSE.qla3xxx 2024-03-15T13:00:32.825 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlcnic 2024-03-15T13:00:32.828 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlge 2024-03-15T13:00:32.830 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/Makefile 2024-03-15T13:00:32.832 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/PLIP.txt 2024-03-15T13:00:32.835 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/README.ipw2100 2024-03-15T13:00:32.837 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/README.ipw2200 2024-03-15T13:00:32.840 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/README.sb1000 2024-03-15T13:00:32.842 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/alias.txt 2024-03-15T13:00:32.844 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/altera_tse.txt 2024-03-15T13:00:32.847 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/arcnet-hardware.txt 2024-03-15T13:00:32.851 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/arcnet.txt 2024-03-15T13:00:32.854 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/atm.txt 2024-03-15T13:00:32.857 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ax25.txt 2024-03-15T13:00:32.859 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/batman-adv.txt 2024-03-15T13:00:32.861 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/baycom.txt 2024-03-15T13:00:32.863 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/bonding.txt 2024-03-15T13:00:32.868 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/bridge.txt 2024-03-15T13:00:32.870 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/caif/ 2024-03-15T13:00:32.873 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/caif/Linux-CAIF.txt 2024-03-15T13:00:32.875 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/caif/README 2024-03-15T13:00:32.892 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/caif/spi_porting.txt 2024-03-15T13:00:33.089 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/can.txt 2024-03-15T13:00:33.092 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/cdc_mbim.txt 2024-03-15T13:00:33.095 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/cops.txt 2024-03-15T13:00:33.097 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/cs89x0.txt 2024-03-15T13:00:33.100 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/cxacru-cf.py 2024-03-15T13:00:33.102 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/cxacru.txt 2024-03-15T13:00:33.104 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/cxgb.txt 2024-03-15T13:00:33.107 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/dccp.txt 2024-03-15T13:00:33.109 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/dctcp.txt 2024-03-15T13:00:33.112 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/de4x5.txt 2024-03-15T13:00:33.114 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/decnet.txt 2024-03-15T13:00:33.116 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/dl2k.txt 2024-03-15T13:00:33.119 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/dm9000.txt 2024-03-15T13:00:33.121 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/dmfe.txt 2024-03-15T13:00:33.124 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/dns_resolver.txt 2024-03-15T13:00:33.126 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/driver.txt 2024-03-15T13:00:33.128 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/e100.txt 2024-03-15T13:00:33.130 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/e1000.txt 2024-03-15T13:00:33.133 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/e1000e.txt 2024-03-15T13:00:33.136 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/eql.txt 2024-03-15T13:00:33.138 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/fib_trie.txt 2024-03-15T13:00:33.140 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/filter.txt 2024-03-15T13:00:33.143 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/fore200e.txt 2024-03-15T13:00:33.146 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/framerelay.txt 2024-03-15T13:00:33.148 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/gen_stats.txt 2024-03-15T13:00:33.150 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/generic-hdlc.txt 2024-03-15T13:00:33.153 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/generic_netlink.txt 2024-03-15T13:00:33.155 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/gianfar.txt 2024-03-15T13:00:33.157 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/i40e.txt 2024-03-15T13:00:33.160 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/i40evf.txt 2024-03-15T13:00:33.162 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ieee802154.txt 2024-03-15T13:00:33.164 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/igb.txt 2024-03-15T13:00:33.166 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/igbvf.txt 2024-03-15T13:00:33.168 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ip-sysctl.txt 2024-03-15T13:00:33.172 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ip_dynaddr.txt 2024-03-15T13:00:33.174 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ipddp.txt 2024-03-15T13:00:33.177 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/iphase.txt 2024-03-15T13:00:33.179 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ipsec.txt 2024-03-15T13:00:33.181 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ipv6.txt 2024-03-15T13:00:33.184 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ipvlan.txt 2024-03-15T13:00:33.185 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ipvs-sysctl.txt 2024-03-15T13:00:33.188 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/irda.txt 2024-03-15T13:00:33.190 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ixgb.txt 2024-03-15T13:00:33.193 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ixgbe.txt 2024-03-15T13:00:33.195 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ixgbevf.txt 2024-03-15T13:00:33.197 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/l2tp.txt 2024-03-15T13:00:33.200 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/lapb-module.txt 2024-03-15T13:00:33.202 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ltpc.txt 2024-03-15T13:00:33.204 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/mac80211-auth-assoc-deauth.txt 2024-03-15T13:00:33.206 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/mac80211-injection.txt 2024-03-15T13:00:33.208 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/ 2024-03-15T13:00:33.211 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/README 2024-03-15T13:00:33.213 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/hostapd.conf 2024-03-15T13:00:33.215 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/wpa_supplicant.conf 2024-03-15T13:00:33.217 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/multiqueue.txt 2024-03-15T13:00:33.220 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/netconsole.txt 2024-03-15T13:00:33.222 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/netdev-FAQ.txt 2024-03-15T13:00:33.225 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/netdev-features.txt 2024-03-15T13:00:33.227 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/netdevices.txt 2024-03-15T13:00:33.229 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/netif-msg.txt 2024-03-15T13:00:33.232 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/netlink_mmap.txt 2024-03-15T13:00:33.234 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/nf_conntrack-sysctl.txt 2024-03-15T13:00:33.236 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/nfc.txt 2024-03-15T13:00:33.239 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/openvswitch.txt 2024-03-15T13:00:33.241 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/operstates.txt 2024-03-15T13:00:33.244 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/packet_mmap.txt 2024-03-15T13:00:33.247 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/phonet.txt 2024-03-15T13:00:33.249 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/phy.txt 2024-03-15T13:00:33.252 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/pktgen.txt 2024-03-15T13:00:33.255 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/policy-routing.txt 2024-03-15T13:00:33.257 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ppp_generic.txt 2024-03-15T13:00:33.260 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/proc_net_tcp.txt 2024-03-15T13:00:33.262 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/radiotap-headers.txt 2024-03-15T13:00:33.264 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/ray_cs.txt 2024-03-15T13:00:33.266 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/rds.txt 2024-03-15T13:00:33.269 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/regulatory.txt 2024-03-15T13:00:33.271 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/rxrpc.txt 2024-03-15T13:00:33.274 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/s2io.txt 2024-03-15T13:00:33.276 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/scaling.txt 2024-03-15T13:00:33.279 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/sctp.txt 2024-03-15T13:00:33.281 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/secid.txt 2024-03-15T13:00:33.284 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/skfp.txt 2024-03-15T13:00:33.286 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/smc9.txt 2024-03-15T13:00:33.288 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/spider_net.txt 2024-03-15T13:00:33.291 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/stmmac.txt 2024-03-15T13:00:33.293 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/switchdev.txt 2024-03-15T13:00:33.296 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/tc-actions-env-rules.txt 2024-03-15T13:00:33.298 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/tcp-thin.txt 2024-03-15T13:00:33.300 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/tcp.txt 2024-03-15T13:00:33.305 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/team.txt 2024-03-15T13:00:33.307 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/timestamping.txt 2024-03-15T13:00:33.311 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/timestamping/ 2024-03-15T13:00:33.313 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/timestamping/.gitignore 2024-03-15T13:00:33.315 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/timestamping/Makefile 2024-03-15T13:00:33.318 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c 2024-03-15T13:00:33.320 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/timestamping/timestamping.c 2024-03-15T13:00:33.323 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c 2024-03-15T13:00:33.325 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/tlan.txt 2024-03-15T13:00:33.328 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/tproxy.txt 2024-03-15T13:00:33.330 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/tuntap.txt 2024-03-15T13:00:33.332 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/udplite.txt 2024-03-15T13:00:33.335 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/vortex.txt 2024-03-15T13:00:33.337 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/vxge.txt 2024-03-15T13:00:33.339 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/vxlan.txt 2024-03-15T13:00:33.341 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/x25-iface.txt 2024-03-15T13:00:33.344 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/x25.txt 2024-03-15T13:00:33.346 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/xfrm_proc.txt 2024-03-15T13:00:33.349 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/xfrm_sync.txt 2024-03-15T13:00:33.351 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/xfrm_sysctl.txt 2024-03-15T13:00:33.353 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/networking/z8530drv.txt 2024-03-15T13:00:33.355 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/nfc/ 2024-03-15T13:00:33.358 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/nfc/nfc-hci.txt 2024-03-15T13:00:33.361 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/nfc/nfc-pn544.txt 2024-03-15T13:00:33.363 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/nios2/ 2024-03-15T13:00:33.366 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/nios2/README 2024-03-15T13:00:33.368 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/nommu-mmap.txt 2024-03-15T13:00:33.371 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/numastat.txt 2024-03-15T13:00:33.374 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/oops-tracing.txt 2024-03-15T13:00:33.376 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/padata.txt 2024-03-15T13:00:33.378 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/parisc/ 2024-03-15T13:00:33.381 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/parisc/00-INDEX 2024-03-15T13:00:33.383 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/parisc/debugging 2024-03-15T13:00:33.385 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/parisc/registers 2024-03-15T13:00:33.388 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/parport-lowlevel.txt 2024-03-15T13:00:33.391 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/parport.txt 2024-03-15T13:00:33.394 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pcmcia/ 2024-03-15T13:00:33.396 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pcmcia/.gitignore 2024-03-15T13:00:33.398 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pcmcia/Makefile 2024-03-15T13:00:33.401 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pcmcia/crc32hash.c 2024-03-15T13:00:33.403 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pcmcia/devicetable.txt 2024-03-15T13:00:33.405 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pcmcia/driver-changes.txt 2024-03-15T13:00:33.407 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pcmcia/driver.txt 2024-03-15T13:00:33.410 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pcmcia/locking.txt 2024-03-15T13:00:33.412 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/percpu-rw-semaphore.txt 2024-03-15T13:00:33.415 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/phy.txt 2024-03-15T13:00:33.417 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/phy/ 2024-03-15T13:00:33.419 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/phy/samsung-usb2.txt 2024-03-15T13:00:33.422 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pi-futex.txt 2024-03-15T13:00:33.426 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pinctrl.txt 2024-03-15T13:00:33.430 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/platform/ 2024-03-15T13:00:33.432 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/platform/x86-laptop-drivers.txt 2024-03-15T13:00:33.435 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pnp.txt 2024-03-15T13:00:33.438 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/ 2024-03-15T13:00:33.441 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/00-INDEX 2024-03-15T13:00:33.443 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/apm-acpi.txt 2024-03-15T13:00:33.445 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/basic-pm-debugging.txt 2024-03-15T13:00:33.448 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/charger-manager.txt 2024-03-15T13:00:33.450 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/devices.txt 2024-03-15T13:00:33.454 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/drivers-testing.txt 2024-03-15T13:00:33.456 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/freezing-of-tasks.txt 2024-03-15T13:00:33.459 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/interface.txt 2024-03-15T13:00:33.461 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/notifiers.txt 2024-03-15T13:00:33.463 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/opp.txt 2024-03-15T13:00:33.466 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/pci.txt 2024-03-15T13:00:33.469 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/pm_qos_interface.txt 2024-03-15T13:00:33.471 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/power_supply_class.txt 2024-03-15T13:00:33.474 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/powercap/ 2024-03-15T13:00:33.476 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/powercap/powercap.txt 2024-03-15T13:00:33.479 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/regulator/ 2024-03-15T13:00:33.481 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/regulator/consumer.txt 2024-03-15T13:00:33.484 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/regulator/design.txt 2024-03-15T13:00:33.486 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/regulator/machine.txt 2024-03-15T13:00:33.489 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/regulator/overview.txt 2024-03-15T13:00:33.492 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/regulator/regulator.txt 2024-03-15T13:00:33.493 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/runtime_pm.txt 2024-03-15T13:00:33.497 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/s2ram.txt 2024-03-15T13:00:33.499 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/states.txt 2024-03-15T13:00:33.502 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/suspend-and-cpuhotplug.txt 2024-03-15T13:00:33.504 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/suspend-and-interrupts.txt 2024-03-15T13:00:33.507 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/swsusp-and-swap-files.txt 2024-03-15T13:00:33.511 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/swsusp-dmcrypt.txt 2024-03-15T13:00:33.514 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/swsusp.txt 2024-03-15T13:00:33.516 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/tricks.txt 2024-03-15T13:00:33.518 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/userland-swsusp.txt 2024-03-15T13:00:33.521 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/power/video.txt 2024-03-15T13:00:33.523 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/powerpc/ 2024-03-15T13:00:33.526 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/powerpc/00-INDEX 2024-03-15T13:00:33.528 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/powerpc/bootwrapper.txt 2024-03-15T13:00:33.531 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/powerpc/cpu_families.txt 2024-03-15T13:00:33.533 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/powerpc/cpu_features.txt 2024-03-15T13:00:33.536 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/powerpc/cxl.txt 2024-03-15T13:00:33.538 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/powerpc/eeh-pci-error-recovery.txt 2024-03-15T13:00:33.541 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/powerpc/firmware-assisted-dump.txt 2024-03-15T13:00:33.543 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/powerpc/hvcs.txt 2024-03-15T13:00:33.546 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/powerpc/mpc52xx.txt 2024-03-15T13:00:33.548 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/powerpc/pmu-ebb.txt 2024-03-15T13:00:33.551 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/powerpc/ptrace.txt 2024-03-15T13:00:33.553 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/powerpc/qe_firmware.txt 2024-03-15T13:00:33.556 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/powerpc/transactional_memory.txt 2024-03-15T13:00:33.559 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pps/ 2024-03-15T13:00:33.561 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pps/pps.txt 2024-03-15T13:00:33.563 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/prctl/ 2024-03-15T13:00:33.566 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/prctl/.gitignore 2024-03-15T13:00:33.569 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/prctl/Makefile 2024-03-15T13:00:33.571 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-ctxt-sw-stress-test.c 2024-03-15T13:00:33.573 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-on-off-stress-test.c 2024-03-15T13:00:33.575 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-test.c 2024-03-15T13:00:33.577 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/prctl/no_new_privs.txt 2024-03-15T13:00:33.579 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/prctl/seccomp_filter.txt 2024-03-15T13:00:33.582 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/preempt-locking.txt 2024-03-15T13:00:33.585 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/printk-formats.txt 2024-03-15T13:00:33.587 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pti/ 2024-03-15T13:00:33.589 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pti/pti_intel_mid.txt 2024-03-15T13:00:33.592 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ptp/ 2024-03-15T13:00:33.594 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ptp/.gitignore 2024-03-15T13:00:33.596 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ptp/Makefile 2024-03-15T13:00:33.598 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ptp/ptp.txt 2024-03-15T13:00:33.600 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ptp/testptp.c 2024-03-15T13:00:33.603 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ptp/testptp.mk 2024-03-15T13:00:33.605 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/pwm.txt 2024-03-15T13:00:33.607 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/ramoops.txt 2024-03-15T13:00:33.610 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/rapidio/ 2024-03-15T13:00:33.612 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/rapidio/rapidio.txt 2024-03-15T13:00:33.615 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/rapidio/sysfs.txt 2024-03-15T13:00:33.618 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/rapidio/tsi721.txt 2024-03-15T13:00:33.620 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/rbtree.txt 2024-03-15T13:00:33.623 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/remoteproc.txt 2024-03-15T13:00:33.626 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/rfkill.txt 2024-03-15T13:00:33.628 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/robust-futex-ABI.txt 2024-03-15T13:00:33.630 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/robust-futexes.txt 2024-03-15T13:00:33.633 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/rpmsg.txt 2024-03-15T13:00:33.636 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/rtc.txt 2024-03-15T13:00:33.639 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/s390/ 2024-03-15T13:00:33.641 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/s390/00-INDEX 2024-03-15T13:00:33.643 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/s390/3270.ChangeLog 2024-03-15T13:00:33.646 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/s390/3270.txt 2024-03-15T13:00:33.648 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/s390/CommonIO 2024-03-15T13:00:33.651 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/s390/DASD 2024-03-15T13:00:33.653 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/s390/Debugging390.txt 2024-03-15T13:00:33.657 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/s390/cds.txt 2024-03-15T13:00:33.660 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/s390/config3270.sh 2024-03-15T13:00:33.661 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/s390/driver-model.txt 2024-03-15T13:00:33.664 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/s390/kvm.txt 2024-03-15T13:00:33.667 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/s390/monreader.txt 2024-03-15T13:00:33.669 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/s390/qeth.txt 2024-03-15T13:00:33.671 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/s390/s390dbf.txt 2024-03-15T13:00:33.674 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/s390/zfcpdump.txt 2024-03-15T13:00:33.676 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scheduler/ 2024-03-15T13:00:33.679 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scheduler/00-INDEX 2024-03-15T13:00:33.682 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scheduler/completion.txt 2024-03-15T13:00:33.684 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scheduler/sched-arch.txt 2024-03-15T13:00:33.686 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scheduler/sched-bwc.txt 2024-03-15T13:00:33.688 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scheduler/sched-deadline.txt 2024-03-15T13:00:33.691 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scheduler/sched-design-CFS.txt 2024-03-15T13:00:33.694 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scheduler/sched-domains.txt 2024-03-15T13:00:33.695 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scheduler/sched-nice-design.txt 2024-03-15T13:00:33.698 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scheduler/sched-rt-group.txt 2024-03-15T13:00:33.700 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scheduler/sched-stats.txt 2024-03-15T13:00:33.703 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/ 2024-03-15T13:00:33.705 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/00-INDEX 2024-03-15T13:00:33.708 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/53c700.txt 2024-03-15T13:00:33.711 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/BusLogic.txt 2024-03-15T13:00:33.714 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.1992-1997 2024-03-15T13:00:33.717 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.arcmsr 2024-03-15T13:00:33.719 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ips 2024-03-15T13:00:33.722 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.lpfc 2024-03-15T13:00:33.726 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid 2024-03-15T13:00:33.729 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid_sas 2024-03-15T13:00:33.732 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ncr53c8xx 2024-03-15T13:00:33.734 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx 2024-03-15T13:00:33.738 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx_2 2024-03-15T13:00:33.740 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/FlashPoint.txt 2024-03-15T13:00:33.742 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/LICENSE.FlashPoint 2024-03-15T13:00:33.745 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla2xxx 2024-03-15T13:00:33.747 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla4xxx 2024-03-15T13:00:33.750 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/Mylex.txt 2024-03-15T13:00:33.752 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/NinjaSCSI.txt 2024-03-15T13:00:33.755 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/aacraid.txt 2024-03-15T13:00:33.757 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/advansys.txt 2024-03-15T13:00:33.760 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/aha152x.txt 2024-03-15T13:00:33.762 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/aic79xx.txt 2024-03-15T13:00:33.765 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/aic7xxx.txt 2024-03-15T13:00:33.767 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/arcmsr_spec.txt 2024-03-15T13:00:33.770 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/bfa.txt 2024-03-15T13:00:33.772 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/bnx2fc.txt 2024-03-15T13:00:33.774 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/cxgb3i.txt 2024-03-15T13:00:33.777 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/dc395x.txt 2024-03-15T13:00:33.779 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/dpti.txt 2024-03-15T13:00:33.781 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/dtc3x80.txt 2024-03-15T13:00:33.784 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/g_NCR5380.txt 2024-03-15T13:00:33.786 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/hpsa.txt 2024-03-15T13:00:33.789 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/hptiop.txt 2024-03-15T13:00:33.791 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/in2000.txt 2024-03-15T13:00:33.793 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/libsas.txt 2024-03-15T13:00:33.796 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/link_power_management_policy.txt 2024-03-15T13:00:33.798 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/lpfc.txt 2024-03-15T13:00:33.801 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/megaraid.txt 2024-03-15T13:00:33.803 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/ncr53c8xx.txt 2024-03-15T13:00:33.807 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/osd.txt 2024-03-15T13:00:33.809 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/osst.txt 2024-03-15T13:00:33.812 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/ppa.txt 2024-03-15T13:00:33.814 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/qlogicfas.txt 2024-03-15T13:00:33.817 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/scsi-changer.txt 2024-03-15T13:00:33.819 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/scsi-generic.txt 2024-03-15T13:00:33.821 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/scsi-parameters.txt 2024-03-15T13:00:33.823 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/scsi.txt 2024-03-15T13:00:33.825 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/scsi_eh.txt 2024-03-15T13:00:33.828 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/scsi_fc_transport.txt 2024-03-15T13:00:33.831 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/scsi_mid_low_api.txt 2024-03-15T13:00:33.835 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/ 2024-03-15T13:00:33.837 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/Makefile 2024-03-15T13:00:33.839 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/rport_state_diagram.dot 2024-03-15T13:00:33.842 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/st.txt 2024-03-15T13:00:33.845 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/sym53c500_cs.txt 2024-03-15T13:00:33.847 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/sym53c8xx_2.txt 2024-03-15T13:00:33.850 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/tmscsim.txt 2024-03-15T13:00:33.853 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/ufs.txt 2024-03-15T13:00:33.856 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/scsi/wd719x.txt 2024-03-15T13:00:33.858 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/security/ 2024-03-15T13:00:33.861 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/security/00-INDEX 2024-03-15T13:00:33.863 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/security/IMA-templates.txt 2024-03-15T13:00:33.866 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/security/LSM.txt 2024-03-15T13:00:33.869 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/security/SELinux.txt 2024-03-15T13:00:33.871 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/security/Smack.txt 2024-03-15T13:00:33.874 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/security/Yama.txt 2024-03-15T13:00:33.876 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/security/apparmor.txt 2024-03-15T13:00:33.878 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/security/credentials.txt 2024-03-15T13:00:33.881 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/security/keys-ecryptfs.txt 2024-03-15T13:00:33.884 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/security/keys-request-key.txt 2024-03-15T13:00:33.886 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/security/keys-trusted-encrypted.txt 2024-03-15T13:00:33.888 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/security/keys.txt 2024-03-15T13:00:33.916 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/security/tomoyo.txt 2024-03-15T13:00:34.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:33 smithi002 ceph-mon[32060]: pgmap v52: 129 pgs: 129 active+clean; 94 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1.1 MiB/s wr, 174 op/s 2024-03-15T13:00:34.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:33 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:00:34.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:33 smithi184 ceph-mon[35993]: pgmap v52: 129 pgs: 129 active+clean; 94 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1.1 MiB/s wr, 174 op/s 2024-03-15T13:00:34.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:33 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:00:34.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:33 smithi149 ceph-mon[35927]: pgmap v52: 129 pgs: 129 active+clean; 94 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1.1 MiB/s wr, 174 op/s 2024-03-15T13:00:34.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:33 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:00:34.165 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/serial-console.txt 2024-03-15T13:00:34.201 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/serial/ 2024-03-15T13:00:34.525 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/serial/00-INDEX 2024-03-15T13:00:34.528 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/serial/README.cycladesZ 2024-03-15T13:00:34.531 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/serial/driver 2024-03-15T13:00:34.534 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/serial/moxa-smartio 2024-03-15T13:00:34.537 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/serial/n_gsm.txt 2024-03-15T13:00:34.539 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/serial/rocket.txt 2024-03-15T13:00:34.541 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/serial/serial-rs485.txt 2024-03-15T13:00:34.544 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/serial/tty.txt 2024-03-15T13:00:34.547 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sgi-ioc4.txt 2024-03-15T13:00:34.550 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sh/ 2024-03-15T13:00:34.558 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sh/new-machine.txt 2024-03-15T13:00:34.565 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sh/register-banks.txt 2024-03-15T13:00:34.569 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/smsc_ece1099.txt 2024-03-15T13:00:34.575 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/ 2024-03-15T13:00:34.577 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/ 2024-03-15T13:00:34.581 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/ALSA-Configuration.txt 2024-03-15T13:00:34.590 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/Audigy-mixer.txt 2024-03-15T13:00:34.594 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/Audiophile-Usb.txt 2024-03-15T13:00:34.597 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/Bt87x.txt 2024-03-15T13:00:34.603 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/CMIPCI.txt 2024-03-15T13:00:34.610 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/Channel-Mapping-API.txt 2024-03-15T13:00:34.613 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/ControlNames.txt 2024-03-15T13:00:34.615 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Controls.txt 2024-03-15T13:00:34.618 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Models.txt 2024-03-15T13:00:34.620 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio.txt 2024-03-15T13:00:34.624 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/Joystick.txt 2024-03-15T13:00:34.626 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/MIXART.txt 2024-03-15T13:00:34.629 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/OSS-Emulation.txt 2024-03-15T13:00:34.631 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/Procfile.txt 2024-03-15T13:00:34.633 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/README.maya44 2024-03-15T13:00:34.637 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/SB-Live-mixer.txt 2024-03-15T13:00:34.639 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/VIA82xx-mixer.txt 2024-03-15T13:00:34.641 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/alsa-parameters.txt 2024-03-15T13:00:34.644 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/compress_offload.txt 2024-03-15T13:00:34.647 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/emu10k1-jack.txt 2024-03-15T13:00:34.649 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/hda_codec.txt 2024-03-15T13:00:34.652 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/hdspm.txt 2024-03-15T13:00:34.655 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/powersave.txt 2024-03-15T13:00:34.658 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/seq_oss.html 2024-03-15T13:00:34.660 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/serial-u16550.txt 2024-03-15T13:00:34.663 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/soc/ 2024-03-15T13:00:34.665 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt 2024-03-15T13:00:34.668 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt 2024-03-15T13:00:34.670 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt 2024-03-15T13:00:34.673 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/soc/codec.txt 2024-03-15T13:00:34.675 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt 2024-03-15T13:00:34.677 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/soc/jack.txt 2024-03-15T13:00:34.680 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/soc/machine.txt 2024-03-15T13:00:34.684 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/soc/overview.txt 2024-03-15T13:00:34.686 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/soc/platform.txt 2024-03-15T13:00:34.688 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt 2024-03-15T13:00:34.691 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/ 2024-03-15T13:00:34.694 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/ALS 2024-03-15T13:00:34.696 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16 2024-03-15T13:00:34.698 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/CMI8330 2024-03-15T13:00:34.700 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/ESS 2024-03-15T13:00:34.702 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/ESS1868 2024-03-15T13:00:34.705 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/Introduction 2024-03-15T13:00:34.708 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/MultiSound 2024-03-15T13:00:34.710 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/OPL3 2024-03-15T13:00:34.712 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/Opti 2024-03-15T13:00:34.716 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/PAS16 2024-03-15T13:00:34.718 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/PSS 2024-03-15T13:00:34.721 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/PSS-updates 2024-03-15T13:00:34.724 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/README.OSS 2024-03-15T13:00:34.728 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/README.modules 2024-03-15T13:00:34.730 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/README.ymfsb 2024-03-15T13:00:34.733 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/SoundPro 2024-03-15T13:00:34.735 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/Soundblaster 2024-03-15T13:00:34.737 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/Tropez+ 2024-03-15T13:00:34.739 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/VIBRA16 2024-03-15T13:00:34.742 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/WaveArtist 2024-03-15T13:00:34.744 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/btaudio 2024-03-15T13:00:34.746 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/mwave 2024-03-15T13:00:34.749 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/oss-parameters.txt 2024-03-15T13:00:34.752 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sound/oss/ultrasound 2024-03-15T13:00:34.754 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sparse.txt 2024-03-15T13:00:34.758 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/spi/ 2024-03-15T13:00:34.760 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/spi/.gitignore 2024-03-15T13:00:34.762 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/spi/00-INDEX 2024-03-15T13:00:34.764 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/spi/Makefile 2024-03-15T13:00:34.766 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/spi/butterfly 2024-03-15T13:00:34.769 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/spi/ep93xx_spi 2024-03-15T13:00:34.772 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/spi/pxa2xx 2024-03-15T13:00:34.774 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/spi/spi-lm70llp 2024-03-15T13:00:34.776 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/spi/spi-sc18is602 2024-03-15T13:00:34.779 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/spi/spi-summary 2024-03-15T13:00:34.782 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/spi/spidev 2024-03-15T13:00:34.785 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/spi/spidev_fdx.c 2024-03-15T13:00:34.786 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/spi/spidev_test.c 2024-03-15T13:00:34.789 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/stable_api_nonsense.txt 2024-03-15T13:00:34.791 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/stable_kernel_rules.txt 2024-03-15T13:00:34.794 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/static-keys.txt 2024-03-15T13:00:34.796 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/svga.txt 2024-03-15T13:00:34.800 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sysctl/ 2024-03-15T13:00:34.802 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sysctl/00-INDEX 2024-03-15T13:00:34.805 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sysctl/README 2024-03-15T13:00:34.807 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sysctl/abi.txt 2024-03-15T13:00:34.809 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sysctl/fs.txt 2024-03-15T13:00:34.812 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sysctl/kernel.txt 2024-03-15T13:00:34.815 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sysctl/net.txt 2024-03-15T13:00:34.818 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sysctl/sunrpc.txt 2024-03-15T13:00:34.820 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sysctl/vm.txt 2024-03-15T13:00:34.822 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sysfs-rules.txt 2024-03-15T13:00:34.825 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/sysrq.txt 2024-03-15T13:00:34.828 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/target/ 2024-03-15T13:00:34.830 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.py 2024-03-15T13:00:34.833 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.txt 2024-03-15T13:00:34.836 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/target/tcmu-design.txt 2024-03-15T13:00:34.839 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/thermal/ 2024-03-15T13:00:34.841 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/thermal/cpu-cooling-api.txt 2024-03-15T13:00:34.844 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal 2024-03-15T13:00:34.846 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal_emulation 2024-03-15T13:00:34.848 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/thermal/intel_powerclamp.txt 2024-03-15T13:00:34.850 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/thermal/nouveau_thermal 2024-03-15T13:00:34.853 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/thermal/sysfs-api.txt 2024-03-15T13:00:34.856 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/thermal/x86_pkg_temperature_thermal 2024-03-15T13:00:34.858 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/this_cpu_ops.txt 2024-03-15T13:00:34.861 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/timers/ 2024-03-15T13:00:34.863 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/timers/.gitignore 2024-03-15T13:00:34.865 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/timers/00-INDEX 2024-03-15T13:00:34.867 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/timers/Makefile 2024-03-15T13:00:34.870 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/timers/NO_HZ.txt 2024-03-15T13:00:34.872 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/timers/highres.txt 2024-03-15T13:00:34.875 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/timers/hpet.txt 2024-03-15T13:00:34.877 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/timers/hpet_example.c 2024-03-15T13:00:34.880 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/timers/hrtimers.txt 2024-03-15T13:00:34.882 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/timers/timekeeping.txt 2024-03-15T13:00:34.885 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/timers/timer_stats.txt 2024-03-15T13:00:34.888 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/timers/timers-howto.txt 2024-03-15T13:00:34.890 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/tpm/ 2024-03-15T13:00:34.892 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/tpm/xen-tpmfront.txt 2024-03-15T13:00:34.895 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/ 2024-03-15T13:00:34.898 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/coresight.txt 2024-03-15T13:00:34.900 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/events-kmem.txt 2024-03-15T13:00:34.903 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/events-nmi.txt 2024-03-15T13:00:34.905 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/events-power.txt 2024-03-15T13:00:34.908 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/events.txt 2024-03-15T13:00:34.910 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/ftrace-design.txt 2024-03-15T13:00:34.913 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/ftrace.txt 2024-03-15T13:00:34.933 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/function-graph-fold.vim 2024-03-15T13:00:35.140 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/kprobetrace.txt 2024-03-15T13:00:35.142 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/mmiotrace.txt 2024-03-15T13:00:35.145 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/postprocess/ 2024-03-15T13:00:35.147 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-pagealloc-postprocess.pl 2024-03-15T13:00:35.149 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-vmscan-postprocess.pl 2024-03-15T13:00:35.152 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/ring-buffer-design.txt 2024-03-15T13:00:35.155 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/tracepoint-analysis.txt 2024-03-15T13:00:35.158 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/tracepoints.txt 2024-03-15T13:00:35.160 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/trace/uprobetracer.txt 2024-03-15T13:00:35.163 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/unaligned-memory-access.txt 2024-03-15T13:00:35.166 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/unicode.txt 2024-03-15T13:00:35.168 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/unshare.txt 2024-03-15T13:00:35.171 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/ 2024-03-15T13:00:35.173 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/CREDITS 2024-03-15T13:00:35.176 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/URB.txt 2024-03-15T13:00:35.178 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/WUSB-Design-overview.txt 2024-03-15T13:00:35.181 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/acm.txt 2024-03-15T13:00:35.183 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/anchors.txt 2024-03-15T13:00:35.185 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/authorization.txt 2024-03-15T13:00:35.188 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/bulk-streams.txt 2024-03-15T13:00:35.190 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/callbacks.txt 2024-03-15T13:00:35.192 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/chipidea.txt 2024-03-15T13:00:35.194 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/dma.txt 2024-03-15T13:00:35.197 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/dwc3.txt 2024-03-15T13:00:35.199 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/ehci.txt 2024-03-15T13:00:35.201 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/error-codes.txt 2024-03-15T13:00:35.204 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/functionfs.txt 2024-03-15T13:00:35.206 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/gadget-testing.txt 2024-03-15T13:00:35.209 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/gadget_configfs.txt 2024-03-15T13:00:35.211 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/gadget_hid.txt 2024-03-15T13:00:35.214 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/gadget_multi.txt 2024-03-15T13:00:35.216 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/gadget_printer.txt 2024-03-15T13:00:35.218 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/gadget_serial.txt 2024-03-15T13:00:35.221 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/hotplug.txt 2024-03-15T13:00:35.223 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/iuu_phoenix.txt 2024-03-15T13:00:35.225 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/linux-cdc-acm.inf 2024-03-15T13:00:35.227 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/linux.inf 2024-03-15T13:00:35.229 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/mass-storage.txt 2024-03-15T13:00:35.232 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/misc_usbsevseg.txt 2024-03-15T13:00:35.234 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/mtouchusb.txt 2024-03-15T13:00:35.236 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/ohci.txt 2024-03-15T13:00:35.239 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/persist.txt 2024-03-15T13:00:35.241 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/power-management.txt 2024-03-15T13:00:35.244 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/proc_usb_info.txt 2024-03-15T13:00:35.246 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/rio.txt 2024-03-15T13:00:35.249 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/usb-help.txt 2024-03-15T13:00:35.251 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/usb-serial.txt 2024-03-15T13:00:35.254 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/usbmon.txt 2024-03-15T13:00:35.256 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/usb/wusb-cbaf 2024-03-15T13:00:35.259 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vDSO/ 2024-03-15T13:00:35.261 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vDSO/.gitignore 2024-03-15T13:00:35.264 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vDSO/Makefile 2024-03-15T13:00:35.266 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vDSO/parse_vdso.c 2024-03-15T13:00:35.268 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vDSO/vdso_standalone_test_x86.c 2024-03-15T13:00:35.271 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vDSO/vdso_test.c 2024-03-15T13:00:35.273 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vfio.txt 2024-03-15T13:00:35.276 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vgaarbiter.txt 2024-03-15T13:00:35.279 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video-output.txt 2024-03-15T13:00:35.281 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/ 2024-03-15T13:00:35.283 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/4CCs.txt 2024-03-15T13:00:35.286 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/API.html 2024-03-15T13:00:35.288 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.au0828 2024-03-15T13:00:35.290 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.bttv 2024-03-15T13:00:35.293 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx23885 2024-03-15T13:00:35.295 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx88 2024-03-15T13:00:35.299 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.em28xx 2024-03-15T13:00:35.302 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.ivtv 2024-03-15T13:00:35.304 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7134 2024-03-15T13:00:35.306 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7164 2024-03-15T13:00:35.308 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tm6000 2024-03-15T13:00:35.310 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tuner 2024-03-15T13:00:35.313 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.usbvision 2024-03-15T13:00:35.315 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/Makefile 2024-03-15T13:00:35.317 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/README.cpia2 2024-03-15T13:00:35.319 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/README.cx88 2024-03-15T13:00:35.322 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/README.davinci-vpbe 2024-03-15T13:00:35.324 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/README.ir 2024-03-15T13:00:35.326 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/README.ivtv 2024-03-15T13:00:35.329 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/README.pvrusb2 2024-03-15T13:00:35.331 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/README.saa7134 2024-03-15T13:00:35.334 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/Zoran 2024-03-15T13:00:35.336 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/ 2024-03-15T13:00:35.338 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/CONTRIBUTORS 2024-03-15T13:00:35.341 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/Cards 2024-03-15T13:00:35.343 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/ICs 2024-03-15T13:00:35.345 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/Insmod-options 2024-03-15T13:00:35.348 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/MAKEDEV 2024-03-15T13:00:35.350 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modprobe.conf 2024-03-15T13:00:35.353 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modules.conf 2024-03-15T13:00:35.355 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/PROBLEMS 2024-03-15T13:00:35.357 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/README 2024-03-15T13:00:35.359 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.WINVIEW 2024-03-15T13:00:35.362 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.freeze 2024-03-15T13:00:35.364 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.quirks 2024-03-15T13:00:35.366 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/Sound-FAQ 2024-03-15T13:00:35.369 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/Specs 2024-03-15T13:00:35.371 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/THANKS 2024-03-15T13:00:35.373 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/bttv/Tuners 2024-03-15T13:00:35.375 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cafe_ccic 2024-03-15T13:00:35.378 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cpia2_overview.txt 2024-03-15T13:00:35.381 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cx18.txt 2024-03-15T13:00:35.382 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/ 2024-03-15T13:00:35.384 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.hm12 2024-03-15T13:00:35.388 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.vbi 2024-03-15T13:00:35.390 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-calling.txt 2024-03-15T13:00:35.393 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-api.txt 2024-03-15T13:00:35.395 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-regs.txt 2024-03-15T13:00:35.397 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-dma.txt 2024-03-15T13:00:35.400 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-encoder-api.txt 2024-03-15T13:00:35.403 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-memory.txt 2024-03-15T13:00:35.405 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-osd-api.txt 2024-03-15T13:00:35.407 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-upload.txt 2024-03-15T13:00:35.410 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cx88/ 2024-03-15T13:00:35.412 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/cx88/hauppauge-wintv-cx88-ir.txt 2024-03-15T13:00:35.415 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/extract_xc3028.pl 2024-03-15T13:00:35.419 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/fimc.txt 2024-03-15T13:00:35.421 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/gspca.txt 2024-03-15T13:00:35.424 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/hauppauge-wintv-cx88-ir.txt 2024-03-15T13:00:35.426 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/lifeview.txt 2024-03-15T13:00:35.429 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/meye.txt 2024-03-15T13:00:35.431 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/not-in-cx2388x-datasheet.txt 2024-03-15T13:00:35.434 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/omap3isp.txt 2024-03-15T13:00:35.436 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/omap4_camera.txt 2024-03-15T13:00:35.439 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/pxa_camera.txt 2024-03-15T13:00:35.441 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/radiotrack.txt 2024-03-15T13:00:35.444 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/sh_mobile_ceu_camera.txt 2024-03-15T13:00:35.446 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/si470x.txt 2024-03-15T13:00:35.449 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/si4713.txt 2024-03-15T13:00:35.451 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/si476x.txt 2024-03-15T13:00:35.453 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/soc-camera.txt 2024-03-15T13:00:35.456 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/uvcvideo.txt 2024-03-15T13:00:35.458 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/v4l2-controls.txt 2024-03-15T13:00:35.461 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/v4l2-framework.txt 2024-03-15T13:00:35.465 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/v4l2-pci-skeleton.c 2024-03-15T13:00:35.468 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/videobuf 2024-03-15T13:00:35.470 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/vivid.txt 2024-03-15T13:00:35.473 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/video4linux/zr364xx.txt 2024-03-15T13:00:35.475 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/ 2024-03-15T13:00:35.478 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/00-INDEX 2024-03-15T13:00:35.481 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/ 2024-03-15T13:00:35.483 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/00-INDEX 2024-03-15T13:00:35.485 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/api.txt 2024-03-15T13:00:35.490 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/cpuid.txt 2024-03-15T13:00:35.492 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/ 2024-03-15T13:00:35.495 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/README 2024-03-15T13:00:35.498 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/arm-vgic.txt 2024-03-15T13:00:35.501 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/mpic.txt 2024-03-15T13:00:35.504 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/s390_flic.txt 2024-03-15T13:00:35.507 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vfio.txt 2024-03-15T13:00:35.510 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vm.txt 2024-03-15T13:00:35.512 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/xics.txt 2024-03-15T13:00:35.515 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/hypercalls.txt 2024-03-15T13:00:35.519 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/locking.txt 2024-03-15T13:00:35.522 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/mmu.txt 2024-03-15T13:00:35.525 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/msr.txt 2024-03-15T13:00:35.527 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/nested-vmx.txt 2024-03-15T13:00:35.530 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/ppc-pv.txt 2024-03-15T13:00:35.533 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/review-checklist.txt 2024-03-15T13:00:35.535 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/s390-diag.txt 2024-03-15T13:00:35.538 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/kvm/timekeeping.txt 2024-03-15T13:00:35.541 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/paravirt_ops.txt 2024-03-15T13:00:35.545 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/uml/ 2024-03-15T13:00:35.547 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/virtual/uml/UserModeLinux-HOWTO.txt 2024-03-15T13:00:35.553 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/ 2024-03-15T13:00:35.556 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/.gitignore 2024-03-15T13:00:35.559 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/00-INDEX 2024-03-15T13:00:35.562 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/active_mm.txt 2024-03-15T13:00:35.565 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/balance 2024-03-15T13:00:35.567 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/cleancache.txt 2024-03-15T13:00:35.570 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/frontswap.txt 2024-03-15T13:00:35.572 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/highmem.txt 2024-03-15T13:00:35.575 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/hugetlbpage.txt 2024-03-15T13:00:35.578 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/hwpoison.txt 2024-03-15T13:00:35.581 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/ksm.txt 2024-03-15T13:00:35.585 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/numa 2024-03-15T13:00:35.587 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/numa_memory_policy.txt 2024-03-15T13:00:35.590 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/overcommit-accounting 2024-03-15T13:00:35.592 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/page_migration 2024-03-15T13:00:35.595 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/page_owner.txt 2024-03-15T13:00:35.598 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/pagemap.txt 2024-03-15T13:00:35.601 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/remap_file_pages.txt 2024-03-15T13:00:35.603 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/slub.txt 2024-03-15T13:00:35.605 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/soft-dirty.txt 2024-03-15T13:00:35.608 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/split_page_table_lock 2024-03-15T13:00:35.610 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/transhuge.txt 2024-03-15T13:00:35.613 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/unevictable-lru.txt 2024-03-15T13:00:35.617 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vm/zswap.txt 2024-03-15T13:00:35.619 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/vme_api.txt 2024-03-15T13:00:35.622 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/volatile-considered-harmful.txt 2024-03-15T13:00:35.625 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/ 2024-03-15T13:00:35.628 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/00-INDEX 2024-03-15T13:00:35.630 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/masters/ 2024-03-15T13:00:35.632 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/masters/00-INDEX 2024-03-15T13:00:35.636 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/masters/ds2482 2024-03-15T13:00:35.639 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/masters/ds2490 2024-03-15T13:00:35.641 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/masters/mxc-w1 2024-03-15T13:00:35.644 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/masters/omap-hdq 2024-03-15T13:00:35.646 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/masters/w1-gpio 2024-03-15T13:00:35.649 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/slaves/ 2024-03-15T13:00:35.652 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/slaves/00-INDEX 2024-03-15T13:00:35.656 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2406 2024-03-15T13:00:35.658 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2423 2024-03-15T13:00:35.661 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds28e04 2024-03-15T13:00:35.664 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/slaves/w1_therm 2024-03-15T13:00:35.666 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/w1.generic 2024-03-15T13:00:35.669 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/w1/w1.netlink 2024-03-15T13:00:35.672 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/watchdog/ 2024-03-15T13:00:35.674 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/watchdog/Makefile 2024-03-15T13:00:35.677 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/watchdog/convert_drivers_to_kernel_api.txt 2024-03-15T13:00:35.680 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/watchdog/hpwdt.txt 2024-03-15T13:00:35.683 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/watchdog/pcwd-watchdog.txt 2024-03-15T13:00:35.685 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/watchdog/src/ 2024-03-15T13:00:35.687 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/watchdog/src/.gitignore 2024-03-15T13:00:35.691 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/watchdog/src/Makefile 2024-03-15T13:00:35.693 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-simple.c 2024-03-15T13:00:35.696 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-test.c 2024-03-15T13:00:35.699 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/watchdog/watchdog-api.txt 2024-03-15T13:00:35.702 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/watchdog/watchdog-kernel-api.txt 2024-03-15T13:00:35.705 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/watchdog/watchdog-parameters.txt 2024-03-15T13:00:35.708 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/watchdog/wdt.txt 2024-03-15T13:00:35.710 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/wimax/ 2024-03-15T13:00:35.713 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/wimax/README.i2400m 2024-03-15T13:00:35.717 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/wimax/README.wimax 2024-03-15T13:00:35.720 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/workqueue.txt 2024-03-15T13:00:35.723 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/ 2024-03-15T13:00:35.726 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/00-INDEX 2024-03-15T13:00:35.729 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/boot.txt 2024-03-15T13:00:35.732 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/early-microcode.txt 2024-03-15T13:00:35.735 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/earlyprintk.txt 2024-03-15T13:00:35.738 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/entry_64.txt 2024-03-15T13:00:35.740 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/exception-tables.txt 2024-03-15T13:00:35.743 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/i386/ 2024-03-15T13:00:35.745 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/i386/IO-APIC.txt 2024-03-15T13:00:35.748 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/intel_mpx.txt 2024-03-15T13:00:35.751 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/mtrr.txt 2024-03-15T13:00:35.754 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/pat.txt 2024-03-15T13:00:35.757 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/tlb.txt 2024-03-15T13:00:35.759 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/usb-legacy-support.txt 2024-03-15T13:00:35.762 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/x86_64/ 2024-03-15T13:00:35.765 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/x86_64/00-INDEX 2024-03-15T13:00:35.768 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/x86_64/boot-options.txt 2024-03-15T13:00:35.770 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/x86_64/cpu-hotplug-spec 2024-03-15T13:00:35.773 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/x86_64/fake-numa-for-cpusets 2024-03-15T13:00:35.776 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/x86_64/kernel-stacks 2024-03-15T13:00:35.779 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/x86_64/machinecheck 2024-03-15T13:00:35.782 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/x86_64/mm.txt 2024-03-15T13:00:35.784 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/x86_64/uefi.txt 2024-03-15T13:00:35.787 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/x86/zero-page.txt 2024-03-15T13:00:35.791 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/xillybus.txt 2024-03-15T13:00:35.796 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/xtensa/ 2024-03-15T13:00:35.799 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/xtensa/atomctl.txt 2024-03-15T13:00:35.802 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/xtensa/mmu.txt 2024-03-15T13:00:35.804 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/xz.txt 2024-03-15T13:00:35.807 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/ 2024-03-15T13:00:35.811 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/CodingStyle 2024-03-15T13:00:35.814 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/HOWTO 2024-03-15T13:00:35.817 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/IRQ.txt 2024-03-15T13:00:35.820 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/SecurityBugs 2024-03-15T13:00:35.823 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingDrivers 2024-03-15T13:00:35.825 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingPatches 2024-03-15T13:00:35.828 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/arm/ 2024-03-15T13:00:35.830 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/arm/Booting 2024-03-15T13:00:35.833 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/arm/kernel_user_helpers.txt 2024-03-15T13:00:35.836 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/arm64/ 2024-03-15T13:00:35.839 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/arm64/booting.txt 2024-03-15T13:00:35.843 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/arm64/memory.txt 2024-03-15T13:00:35.845 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/arm64/tagged-pointers.txt 2024-03-15T13:00:35.847 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/basic_profiling.txt 2024-03-15T13:00:35.850 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/email-clients.txt 2024-03-15T13:00:35.853 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/ 2024-03-15T13:00:35.854 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/sysfs.txt 2024-03-15T13:00:35.857 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/gpio.txt 2024-03-15T13:00:35.860 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/io_ordering.txt 2024-03-15T13:00:35.863 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/magic-number.txt 2024-03-15T13:00:35.866 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/oops-tracing.txt 2024-03-15T13:00:35.868 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/sparse.txt 2024-03-15T13:00:35.871 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/stable_api_nonsense.txt 2024-03-15T13:00:35.873 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/stable_kernel_rules.txt 2024-03-15T13:00:35.875 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/ 2024-03-15T13:00:35.877 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/omap3isp.txt 2024-03-15T13:00:35.881 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/v4l2-framework.txt 2024-03-15T13:00:35.884 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zh_CN/volatile-considered-harmful.txt 2024-03-15T13:00:35.886 INFO:tasks.workunit.client.0.smithi002.stdout:linux-4.0.5/Documentation/zorro.txt 2024-03-15T13:00:36.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:35 smithi002 ceph-mon[32060]: pgmap v53: 129 pgs: 129 active+clean; 94 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1.1 MiB/s wr, 174 op/s 2024-03-15T13:00:36.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:35 smithi184 ceph-mon[35993]: pgmap v53: 129 pgs: 129 active+clean; 94 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1.1 MiB/s wr, 174 op/s 2024-03-15T13:00:36.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:35 smithi149 ceph-mon[35927]: pgmap v53: 129 pgs: 129 active+clean; 94 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 1.1 MiB/s wr, 174 op/s 2024-03-15T13:00:38.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:37 smithi002 ceph-mon[32060]: pgmap v54: 129 pgs: 129 active+clean; 109 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.4 MiB/s wr, 291 op/s 2024-03-15T13:00:38.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:37 smithi184 ceph-mon[35993]: pgmap v54: 129 pgs: 129 active+clean; 109 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.4 MiB/s wr, 291 op/s 2024-03-15T13:00:38.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:37 smithi149 ceph-mon[35927]: pgmap v54: 129 pgs: 129 active+clean; 109 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.4 MiB/s wr, 291 op/s 2024-03-15T13:00:40.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:39 smithi002 ceph-mon[32060]: pgmap v55: 129 pgs: 129 active+clean; 109 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-03-15T13:00:40.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:39 smithi184 ceph-mon[35993]: pgmap v55: 129 pgs: 129 active+clean; 109 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-03-15T13:00:40.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:39 smithi149 ceph-mon[35927]: pgmap v55: 129 pgs: 129 active+clean; 109 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-03-15T13:00:41.497 INFO:tasks.workunit.client.0.smithi002.stderr:+ tar cf doc.tar linux-4.0.5 2024-03-15T13:00:42.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:41 smithi002 ceph-mon[32060]: pgmap v56: 129 pgs: 129 active+clean; 115 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 301 op/s 2024-03-15T13:00:42.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:41 smithi184 ceph-mon[35993]: pgmap v56: 129 pgs: 129 active+clean; 115 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 301 op/s 2024-03-15T13:00:42.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:41 smithi149 ceph-mon[35927]: pgmap v56: 129 pgs: 129 active+clean; 115 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 301 op/s 2024-03-15T13:00:44.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:43 smithi002 ceph-mon[32060]: pgmap v57: 129 pgs: 129 active+clean; 115 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.9 MiB/s wr, 159 op/s 2024-03-15T13:00:44.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:43 smithi184 ceph-mon[35993]: pgmap v57: 129 pgs: 129 active+clean; 115 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.9 MiB/s wr, 159 op/s 2024-03-15T13:00:44.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:43 smithi149 ceph-mon[35927]: pgmap v57: 129 pgs: 129 active+clean; 115 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.9 MiB/s wr, 159 op/s 2024-03-15T13:00:45.500 INFO:tasks.workunit.client.0.smithi002.stderr:+ tar xf doc.tar 2024-03-15T13:00:46.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:45 smithi002 ceph-mon[32060]: pgmap v58: 129 pgs: 129 active+clean; 115 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.9 MiB/s wr, 159 op/s 2024-03-15T13:00:46.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:45 smithi184 ceph-mon[35993]: pgmap v58: 129 pgs: 129 active+clean; 115 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.9 MiB/s wr, 159 op/s 2024-03-15T13:00:46.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:45 smithi149 ceph-mon[35927]: pgmap v58: 129 pgs: 129 active+clean; 115 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.9 MiB/s wr, 159 op/s 2024-03-15T13:00:48.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:47 smithi002 ceph-mon[32060]: pgmap v59: 129 pgs: 129 active+clean; 131 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 185 op/s 2024-03-15T13:00:48.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:47 smithi184 ceph-mon[35993]: pgmap v59: 129 pgs: 129 active+clean; 131 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 185 op/s 2024-03-15T13:00:48.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:47 smithi149 ceph-mon[35927]: pgmap v59: 129 pgs: 129 active+clean; 131 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 185 op/s 2024-03-15T13:00:50.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:49 smithi002 ceph-mon[32060]: pgmap v60: 129 pgs: 129 active+clean; 131 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.9 MiB/s wr, 68 op/s 2024-03-15T13:00:50.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:49 smithi184 ceph-mon[35993]: pgmap v60: 129 pgs: 129 active+clean; 131 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.9 MiB/s wr, 68 op/s 2024-03-15T13:00:50.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:49 smithi149 ceph-mon[35927]: pgmap v60: 129 pgs: 129 active+clean; 131 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.9 MiB/s wr, 68 op/s 2024-03-15T13:00:52.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:51 smithi002 ceph-mon[32060]: pgmap v61: 129 pgs: 129 active+clean; 144 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 3.1 MiB/s wr, 176 op/s 2024-03-15T13:00:52.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:51 smithi184 ceph-mon[35993]: pgmap v61: 129 pgs: 129 active+clean; 144 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 3.1 MiB/s wr, 176 op/s 2024-03-15T13:00:52.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:51 smithi149 ceph-mon[35927]: pgmap v61: 129 pgs: 129 active+clean; 144 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 3.1 MiB/s wr, 176 op/s 2024-03-15T13:00:54.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:53 smithi002 ceph-mon[32060]: pgmap v62: 129 pgs: 129 active+clean; 144 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 2.6 MiB/s wr, 134 op/s 2024-03-15T13:00:54.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:53 smithi184 ceph-mon[35993]: pgmap v62: 129 pgs: 129 active+clean; 144 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 2.6 MiB/s wr, 134 op/s 2024-03-15T13:00:54.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:53 smithi149 ceph-mon[35927]: pgmap v62: 129 pgs: 129 active+clean; 144 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 2.6 MiB/s wr, 134 op/s 2024-03-15T13:00:54.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:00:54.328 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:00:54.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.460+0000 7f0830cc0640 1 -- 172.21.15.2:0/1029402029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082c105c30 msgr2=0x7f082c106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:54.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.460+0000 7f0830cc0640 1 --2- 172.21.15.2:0/1029402029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082c105c30 0x7f082c106010 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f081c009900 tx=0x7f081c02bc30 comp rx=0 tx=0).stop 2024-03-15T13:00:54.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.460+0000 7f0830cc0640 1 -- 172.21.15.2:0/1029402029 shutdown_connections 2024-03-15T13:00:54.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.460+0000 7f0830cc0640 1 --2- 172.21.15.2:0/1029402029 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f082c100700 0x7f082c10c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.460+0000 7f0830cc0640 1 --2- 172.21.15.2:0/1029402029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f082c0ffd40 0x7f082c1001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.460+0000 7f0830cc0640 1 --2- 172.21.15.2:0/1029402029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082c105c30 0x7f082c106010 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.460+0000 7f0830cc0640 1 -- 172.21.15.2:0/1029402029 >> 172.21.15.2:0/1029402029 conn(0x7f082c0756f0 msgr2=0x7f082c075b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.460+0000 7f0830cc0640 1 -- 172.21.15.2:0/1029402029 shutdown_connections 2024-03-15T13:00:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.460+0000 7f0830cc0640 1 -- 172.21.15.2:0/1029402029 wait complete. 2024-03-15T13:00:54.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.461+0000 7f0830cc0640 1 Processor -- start 2024-03-15T13:00:54.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.461+0000 7f0830cc0640 1 -- start start 2024-03-15T13:00:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.461+0000 7f0830cc0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f082c0ffd40 0x7f082c19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.461+0000 7f0830cc0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082c100700 0x7f082c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.461+0000 7f0830cc0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f082c105c30 0x7f082c196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.461+0000 7f0830cc0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c196ee0 con 0x7f082c100700 2024-03-15T13:00:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.461+0000 7f0830cc0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c197050 con 0x7f082c105c30 2024-03-15T13:00:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.461+0000 7f0830cc0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c197210 con 0x7f082c0ffd40 2024-03-15T13:00:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.461+0000 7f0821d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082c100700 0x7f082c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.461+0000 7f082a59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f082c0ffd40 0x7f082c19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.461+0000 7f0821d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082c100700 0x7f082c19cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60810/0 (socket says 172.21.15.2:60810) 2024-03-15T13:00:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.461+0000 7f082ad9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f082c105c30 0x7f082c196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.461+0000 7f082a59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f082c0ffd40 0x7f082c19c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:34724/0 (socket says 172.21.15.2:34724) 2024-03-15T13:00:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.462+0000 7f0821d9b640 1 -- 172.21.15.2:0/2603376022 learned_addr learned my addr 172.21.15.2:0/2603376022 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.462+0000 7f0821d9b640 1 -- 172.21.15.2:0/2603376022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f082c0ffd40 msgr2=0x7f082c19c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.462+0000 7f0821d9b640 1 --2- 172.21.15.2:0/2603376022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f082c0ffd40 0x7f082c19c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.462+0000 7f0821d9b640 1 -- 172.21.15.2:0/2603376022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f082c105c30 msgr2=0x7f082c196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.462+0000 7f0821d9b640 1 --2- 172.21.15.2:0/2603376022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f082c105c30 0x7f082c196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.462+0000 7f0821d9b640 1 -- 172.21.15.2:0/2603376022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c009560 con 0x7f082c100700 2024-03-15T13:00:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.462+0000 7f082a59c640 1 --2- 172.21.15.2:0/2603376022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f082c0ffd40 0x7f082c19c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.462+0000 7f0821d9b640 1 --2- 172.21.15.2:0/2603376022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082c100700 0x7f082c19cbc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f081400e9f0 tx=0x7f081400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.462+0000 7f0823fff640 1 -- 172.21.15.2:0/2603376022 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814009700 con 0x7f082c100700 2024-03-15T13:00:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.462+0000 7f0830cc0640 1 -- 172.21.15.2:0/2603376022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082c1974f0 con 0x7f082c100700 2024-03-15T13:00:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.462+0000 7f0823fff640 1 -- 172.21.15.2:0/2603376022 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f081400cd80 con 0x7f082c100700 2024-03-15T13:00:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.462+0000 7f0823fff640 1 -- 172.21.15.2:0/2603376022 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814018920 con 0x7f082c100700 2024-03-15T13:00:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.462+0000 7f0830cc0640 1 -- 172.21.15.2:0/2603376022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082c10d490 con 0x7f082c100700 2024-03-15T13:00:54.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.465+0000 7f0823fff640 1 -- 172.21.15.2:0/2603376022 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f0814018a80 con 0x7f082c100700 2024-03-15T13:00:54.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.465+0000 7f0833978740 1 Processor -- start 2024-03-15T13:00:54.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f0833978740 1 -- start start 2024-03-15T13:00:54.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f0833978740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc549fe0 0x563ebc54a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f0833978740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc557680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f0833978740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc557bc0 0x563ebc559fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f0823fff640 1 --2- 172.21.15.2:0/2603376022 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f08180a1ba0 0x7f08180a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f0833978740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebc4b9280 con 0x563ebc54a900 2024-03-15T13:00:54.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f0833978740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebc4b67b0 con 0x563ebc557bc0 2024-03-15T13:00:54.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f0833978740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebc4b1f90 con 0x563ebc549fe0 2024-03-15T13:00:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f082a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc557680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f082ad9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc549fe0 0x563ebc54a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f082a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc557680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60818/0 (socket says 172.21.15.2:60818) 2024-03-15T13:00:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f082a59c640 1 -- 172.21.15.2:0/655055014 learned_addr learned my addr 172.21.15.2:0/655055014 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f082ad9d640 1 --2- 172.21.15.2:0/655055014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc557bc0 0x563ebc559fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f0823fff640 1 -- 172.21.15.2:0/2603376022 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f0814014070 con 0x7f082c100700 2024-03-15T13:00:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f0821d9b640 1 --2- 172.21.15.2:0/2603376022 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f08180a1ba0 0x7f08180a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f082a59c640 1 -- 172.21.15.2:0/655055014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc549fe0 msgr2=0x563ebc54a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.466+0000 7f082a59c640 1 --2- 172.21.15.2:0/655055014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc549fe0 0x563ebc54a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.467+0000 7f082a59c640 1 -- 172.21.15.2:0/655055014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc557bc0 msgr2=0x563ebc559fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.467+0000 7f082a59c640 1 --2- 172.21.15.2:0/655055014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc557bc0 0x563ebc559fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.467+0000 7f082a59c640 1 -- 172.21.15.2:0/655055014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ebc470570 con 0x563ebc54a900 2024-03-15T13:00:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.467+0000 7f0821d9b640 1 --2- 172.21.15.2:0/2603376022 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f08180a1ba0 0x7f08180a4060 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08140cb2d0 tx=0x7f081400cef0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.467+0000 7f082a59c640 1 --2- 172.21.15.2:0/655055014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc557680 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f081c036040 tx=0x7f081c004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c3290c066b5c85 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.467+0000 7f0820d99640 1 -- 172.21.15.2:0/655055014 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c02c810 con 0x563ebc54a900 2024-03-15T13:00:54.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.467+0000 7f0820d99640 1 -- 172.21.15.2:0/655055014 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f081c004500 con 0x563ebc54a900 2024-03-15T13:00:54.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.467+0000 7f0820d99640 1 -- 172.21.15.2:0/655055014 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c003cc0 con 0x563ebc54a900 2024-03-15T13:00:54.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.468+0000 7f0833978740 1 -- 172.21.15.2:0/655055014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 msgr2=0x563ebc557680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:54.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.468+0000 7f0833978740 1 --2- 172.21.15.2:0/655055014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc557680 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f081c036040 tx=0x7f081c004b00 comp rx=0 tx=0).stop 2024-03-15T13:00:54.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.468+0000 7f0833978740 1 -- 172.21.15.2:0/655055014 shutdown_connections 2024-03-15T13:00:54.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.468+0000 7f0833978740 1 --2- 172.21.15.2:0/655055014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc557bc0 0x563ebc559fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.468+0000 7f0833978740 1 --2- 172.21.15.2:0/655055014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc557680 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.468+0000 7f0833978740 1 --2- 172.21.15.2:0/655055014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc549fe0 0x563ebc54a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.468+0000 7f0833978740 1 -- 172.21.15.2:0/655055014 >> 172.21.15.2:0/655055014 conn(0x563ebc544990 msgr2=0x563ebc544d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:54.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.468+0000 7f0833978740 1 -- 172.21.15.2:0/655055014 shutdown_connections 2024-03-15T13:00:54.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.468+0000 7f0833978740 1 -- 172.21.15.2:0/655055014 wait complete. 2024-03-15T13:00:54.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.468+0000 7f0833978740 1 -- start start 2024-03-15T13:00:54.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f0833978740 1 Processor -- start 2024-03-15T13:00:54.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f0833978740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc549fe0 0x563ebc557560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f0833978740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc5508b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f0833978740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc557bc0 0x563ebc550e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f0833978740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebc185430 con 0x563ebc54a900 2024-03-15T13:00:54.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f0833978740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebc141e00 con 0x563ebc549fe0 2024-03-15T13:00:54.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f0833978740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebc436fd0 con 0x563ebc557bc0 2024-03-15T13:00:54.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f0833978740 10 client.0 _ll_register_callbacks cb 0x563ebc48cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:54.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f082a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc5508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f082ad9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc549fe0 0x563ebc557560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f082ad9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc549fe0 0x563ebc557560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50768/0 (socket says 172.21.15.2:50768) 2024-03-15T13:00:54.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f082ad9d640 1 -- 172.21.15.2:0/281138670 learned_addr learned my addr 172.21.15.2:0/281138670 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:54.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f082ad9d640 1 --2- 172.21.15.2:0/281138670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc557bc0 0x563ebc550e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f082ad9d640 1 -- 172.21.15.2:0/281138670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc557bc0 msgr2=0x563ebc550e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:54.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f082ad9d640 1 --2- 172.21.15.2:0/281138670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc557bc0 0x563ebc550e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f082ad9d640 1 -- 172.21.15.2:0/281138670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 msgr2=0x563ebc5508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:54.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f082ad9d640 1 --2- 172.21.15.2:0/281138670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc5508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f082ad9d640 1 -- 172.21.15.2:0/281138670 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082c069270 con 0x563ebc549fe0 2024-03-15T13:00:54.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.469+0000 7f082a59c640 1 --2- 172.21.15.2:0/281138670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc5508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:54.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f082ad9d640 1 --2- 172.21.15.2:0/281138670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc549fe0 0x563ebc557560 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0824011e00 tx=0x7f0824009df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=110a3efb2b30f8af server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:54.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f0811ffb640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:00:54.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f0811ffb640 1 -- 172.21.15.2:0/281138670 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0824025070 con 0x563ebc549fe0 2024-03-15T13:00:54.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f0833978740 1 -- 172.21.15.2:0/281138670 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ebc1ee720 con 0x563ebc549fe0 2024-03-15T13:00:54.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f0811ffb640 1 -- 172.21.15.2:0/281138670 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f0824003970 con 0x563ebc549fe0 2024-03-15T13:00:54.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f0811ffb640 1 -- 172.21.15.2:0/281138670 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f082400fee0 con 0x563ebc549fe0 2024-03-15T13:00:54.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f0833978740 10 client.26150 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:54.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f0833978740 1 -- 172.21.15.2:0/281138670 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563ebc4b0610 con 0x563ebc549fe0 2024-03-15T13:00:54.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f0811ffb640 1 -- 172.21.15.2:0/281138670 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x563ebc1ee720 con 0x563ebc549fe0 2024-03-15T13:00:54.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f0811ffb640 1 -- 172.21.15.2:0/281138670 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f082400e3b0 con 0x563ebc549fe0 2024-03-15T13:00:54.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f0833978740 10 client.26150 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:54.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f0833978740 10 client.26150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:54.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f0833978740 20 client.26150 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:54.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f0833978740 1 --2- 172.21.15.2:0/281138670 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563ebc563940 0x563ebc565d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f0833978740 4 client.26150 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:54.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.470+0000 7f0833978740 1 -- 172.21.15.2:0/281138670 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ebc23d890 con 0x563ebc563940 2024-03-15T13:00:54.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.471+0000 7f082a59c640 1 --2- 172.21.15.2:0/281138670 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563ebc563940 0x563ebc565d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.471+0000 7f082a59c640 1 --2- 172.21.15.2:0/281138670 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563ebc563940 0x563ebc565d60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6323bb34f121a89c server_cookie=f6d3bfdcb4675c34 in_seq=0 out_seq=0 2024-03-15T13:00:54.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.471+0000 7f0811ffb640 10 client.26150 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:54.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.472+0000 7f0811ffb640 1 -- 172.21.15.2:0/281138670 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563ebc23d890 con 0x563ebc563940 2024-03-15T13:00:54.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.472+0000 7f0811ffb640 10 client.26150 handle_command_reply: tid=0 2024-03-15T13:00:54.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.472+0000 7f0833978740 1 client.26150 shutdown 2024-03-15T13:00:54.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.472+0000 7f0833978740 1 -- 172.21.15.2:0/281138670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc549fe0 msgr2=0x563ebc557560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:54.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.472+0000 7f0833978740 1 --2- 172.21.15.2:0/281138670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc549fe0 0x563ebc557560 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0824011e00 tx=0x7f0824009df0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.472+0000 7f0833978740 1 -- 172.21.15.2:0/281138670 shutdown_connections 2024-03-15T13:00:54.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.472+0000 7f0833978740 1 --2- 172.21.15.2:0/281138670 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563ebc563940 0x563ebc565d60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.472+0000 7f0833978740 1 --2- 172.21.15.2:0/281138670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc557bc0 0x563ebc550e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.472+0000 7f0833978740 1 --2- 172.21.15.2:0/281138670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc5508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.472+0000 7f0833978740 1 --2- 172.21.15.2:0/281138670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc549fe0 0x563ebc557560 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.472+0000 7f0833978740 1 -- 172.21.15.2:0/281138670 >> 172.21.15.2:0/281138670 conn(0x563ebc544990 msgr2=0x563ebc546730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:54.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.472+0000 7f0811ffb640 0 client.26150 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:54.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.472+0000 7f0833978740 1 -- 172.21.15.2:0/281138670 shutdown_connections 2024-03-15T13:00:54.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.472+0000 7f0833978740 1 -- 172.21.15.2:0/281138670 wait complete. 2024-03-15T13:00:54.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.472+0000 7f0833978740 20 client.26150 trim_cache size 0 max 16384 2024-03-15T13:00:54.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.490+0000 7f0833978740 1 Processor -- start 2024-03-15T13:00:54.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.490+0000 7f0833978740 1 -- start start 2024-03-15T13:00:54.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.490+0000 7f0833978740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc549fe0 0x563ebc556eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.490+0000 7f0833978740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc54a900 0x563ebc5573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.490+0000 7f0833978740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc557bc0 0x563ebc54db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.490+0000 7f0833978740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebc557930 con 0x563ebc557bc0 2024-03-15T13:00:54.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.490+0000 7f0833978740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebc554ed0 con 0x563ebc549fe0 2024-03-15T13:00:54.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.490+0000 7f0833978740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebc5621c0 con 0x563ebc54a900 2024-03-15T13:00:54.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.490+0000 7f082ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc557bc0 0x563ebc54db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.490+0000 7f082a59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc54a900 0x563ebc5573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.490+0000 7f082ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc557bc0 0x563ebc54db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60846/0 (socket says 172.21.15.2:60846) 2024-03-15T13:00:54.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.490+0000 7f082ad9d640 1 -- 172.21.15.2:0/3802720884 learned_addr learned my addr 172.21.15.2:0/3802720884 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:54.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.490+0000 7f082ad9d640 1 --2- 172.21.15.2:0/3802720884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc549fe0 0x563ebc556eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.491+0000 7f082ad9d640 1 -- 172.21.15.2:0/3802720884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc54a900 msgr2=0x563ebc5573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:54.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.491+0000 7f082ad9d640 1 --2- 172.21.15.2:0/3802720884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc54a900 0x563ebc5573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.491+0000 7f082ad9d640 1 -- 172.21.15.2:0/3802720884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc549fe0 msgr2=0x563ebc556eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:54.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.491+0000 7f082ad9d640 1 --2- 172.21.15.2:0/3802720884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc549fe0 0x563ebc556eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.491+0000 7f082ad9d640 1 -- 172.21.15.2:0/3802720884 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ebc44cac0 con 0x563ebc557bc0 2024-03-15T13:00:54.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.491+0000 7f082ad9d640 1 --2- 172.21.15.2:0/3802720884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc557bc0 0x563ebc54db30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f082400fca0 tx=0x7f0824009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8720a77f4528bddc server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:54.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.491+0000 7f0813fff640 1 -- 172.21.15.2:0/3802720884 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0824025070 con 0x563ebc557bc0 2024-03-15T13:00:54.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.491+0000 7f0813fff640 1 -- 172.21.15.2:0/3802720884 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0824026040 con 0x563ebc557bc0 2024-03-15T13:00:54.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.491+0000 7f0813fff640 1 -- 172.21.15.2:0/3802720884 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082400ece0 con 0x563ebc557bc0 2024-03-15T13:00:54.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 -- 172.21.15.2:0/3802720884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc557bc0 msgr2=0x563ebc54db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:54.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 --2- 172.21.15.2:0/3802720884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc557bc0 0x563ebc54db30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f082400fca0 tx=0x7f0824009e60 comp rx=0 tx=0).stop 2024-03-15T13:00:54.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 -- 172.21.15.2:0/3802720884 shutdown_connections 2024-03-15T13:00:54.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 --2- 172.21.15.2:0/3802720884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc557bc0 0x563ebc54db30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 --2- 172.21.15.2:0/3802720884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc54a900 0x563ebc5573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 --2- 172.21.15.2:0/3802720884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc549fe0 0x563ebc556eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 -- 172.21.15.2:0/3802720884 >> 172.21.15.2:0/3802720884 conn(0x563ebc544990 msgr2=0x563ebc550680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:54.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 -- 172.21.15.2:0/3802720884 shutdown_connections 2024-03-15T13:00:54.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 -- 172.21.15.2:0/3802720884 wait complete. 2024-03-15T13:00:54.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 -- start start 2024-03-15T13:00:54.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 Processor -- start 2024-03-15T13:00:54.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc549fe0 0x563ebc55a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc55abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc557bc0 0x563ebc55b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebc55f810 con 0x563ebc54a900 2024-03-15T13:00:54.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebc55f980 con 0x563ebc557bc0 2024-03-15T13:00:54.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.492+0000 7f0833978740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ebc55fb40 con 0x563ebc549fe0 2024-03-15T13:00:54.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f0833978740 10 client.0 _ll_register_callbacks cb 0x7f0824011a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:54.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f082a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc55abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f082a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc55abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60860/0 (socket says 172.21.15.2:60860) 2024-03-15T13:00:54.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f082a59c640 1 -- 172.21.15.2:0/513183403 learned_addr learned my addr 172.21.15.2:0/513183403 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:54.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f082ad9d640 1 --2- 172.21.15.2:0/513183403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc549fe0 0x563ebc55a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f082ad9d640 1 --2- 172.21.15.2:0/513183403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc557bc0 0x563ebc55b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f082a59c640 1 -- 172.21.15.2:0/513183403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc549fe0 msgr2=0x563ebc55a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:54.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f082a59c640 1 --2- 172.21.15.2:0/513183403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc549fe0 0x563ebc55a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f082a59c640 1 -- 172.21.15.2:0/513183403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc557bc0 msgr2=0x563ebc55b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:54.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f082a59c640 1 --2- 172.21.15.2:0/513183403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc557bc0 0x563ebc55b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f082a59c640 1 -- 172.21.15.2:0/513183403 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ebc4b0610 con 0x563ebc54a900 2024-03-15T13:00:54.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f082a59c640 1 --2- 172.21.15.2:0/513183403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc55abf0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f081c009900 tx=0x7f081c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7446749baa0b06f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:54.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f0811ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:54.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f0811ffb640 1 -- 172.21.15.2:0/513183403 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c02cb20 con 0x563ebc54a900 2024-03-15T13:00:54.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f0811ffb640 1 -- 172.21.15.2:0/513183403 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f081c02cc80 con 0x563ebc54a900 2024-03-15T13:00:54.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f0833978740 1 -- 172.21.15.2:0/513183403 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ebc1b17e0 con 0x563ebc54a900 2024-03-15T13:00:54.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f0811ffb640 1 -- 172.21.15.2:0/513183403 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f081c056890 con 0x563ebc54a900 2024-03-15T13:00:54.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f0833978740 10 client.16293 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:54.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.493+0000 7f0833978740 1 -- 172.21.15.2:0/513183403 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563ebc4726a0 con 0x563ebc54a900 2024-03-15T13:00:54.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.494+0000 7f0811ffb640 1 -- 172.21.15.2:0/513183403 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x563ebc1b17e0 con 0x563ebc54a900 2024-03-15T13:00:54.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.494+0000 7f0811ffb640 1 -- 172.21.15.2:0/513183403 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f081c05d080 con 0x563ebc54a900 2024-03-15T13:00:54.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.494+0000 7f0833978740 10 client.16293 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:54.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.494+0000 7f0833978740 10 client.16293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:54.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.494+0000 7f0833978740 20 client.16293 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:54.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.494+0000 7f0833978740 1 --2- 172.21.15.2:0/513183403 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563ebc5bfc30 0x563ebc5c2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.494+0000 7f0833978740 4 client.16293 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:00:54.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.494+0000 7f0833978740 1 -- 172.21.15.2:0/513183403 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563ebc1a04e0 con 0x563ebc5bfc30 2024-03-15T13:00:54.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.494+0000 7f082ad9d640 1 --2- 172.21.15.2:0/513183403 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563ebc5bfc30 0x563ebc5c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.494+0000 7f082ad9d640 1 --2- 172.21.15.2:0/513183403 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563ebc5bfc30 0x563ebc5c2050 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ed7ee5ae964df30 server_cookie=13d454f462da022b in_seq=0 out_seq=0 2024-03-15T13:00:54.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.494+0000 7f0811ffb640 10 client.16293 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:54.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.495+0000 7f0811ffb640 1 -- 172.21.15.2:0/513183403 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563ebc1a04e0 con 0x563ebc5bfc30 2024-03-15T13:00:54.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.495+0000 7f0811ffb640 10 client.16293 handle_command_reply: tid=0 2024-03-15T13:00:54.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.495+0000 7f0833978740 1 client.16293 shutdown 2024-03-15T13:00:54.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.495+0000 7f0833978740 1 -- 172.21.15.2:0/513183403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 msgr2=0x563ebc55abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:54.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.495+0000 7f0833978740 1 --2- 172.21.15.2:0/513183403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc55abf0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f081c009900 tx=0x7f081c005e50 comp rx=0 tx=0).stop 2024-03-15T13:00:54.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.495+0000 7f0833978740 1 -- 172.21.15.2:0/513183403 shutdown_connections 2024-03-15T13:00:54.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.495+0000 7f0833978740 1 --2- 172.21.15.2:0/513183403 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563ebc5bfc30 0x563ebc5c2050 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.495+0000 7f0833978740 1 --2- 172.21.15.2:0/513183403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ebc557bc0 0x563ebc55b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.495+0000 7f0833978740 1 --2- 172.21.15.2:0/513183403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ebc54a900 0x563ebc55abf0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.495+0000 7f0811ffb640 0 client.16293 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:54.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.495+0000 7f0833978740 1 --2- 172.21.15.2:0/513183403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ebc549fe0 0x563ebc55a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.495+0000 7f0833978740 1 -- 172.21.15.2:0/513183403 >> 172.21.15.2:0/513183403 conn(0x563ebc544990 msgr2=0x563ebc553ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:54.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.496+0000 7f0833978740 1 -- 172.21.15.2:0/513183403 shutdown_connections 2024-03-15T13:00:54.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.496+0000 7f0833978740 1 -- 172.21.15.2:0/513183403 wait complete. 2024-03-15T13:00:54.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.496+0000 7f0833978740 20 client.16293 trim_cache size 0 max 16384 2024-03-15T13:00:54.487 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:00:54.487 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:00:54.487 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:00:54.487 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:00:54.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.496+0000 7f0830cc0640 1 -- 172.21.15.2:0/2603376022 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f08180a1ba0 msgr2=0x7f08180a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:54.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.496+0000 7f0830cc0640 1 --2- 172.21.15.2:0/2603376022 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f08180a1ba0 0x7f08180a4060 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08140cb2d0 tx=0x7f081400cef0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.496+0000 7f0830cc0640 1 -- 172.21.15.2:0/2603376022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082c100700 msgr2=0x7f082c19cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:54.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.496+0000 7f0830cc0640 1 --2- 172.21.15.2:0/2603376022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082c100700 0x7f082c19cbc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f081400e9f0 tx=0x7f081400eeb0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.496+0000 7f0830cc0640 1 -- 172.21.15.2:0/2603376022 shutdown_connections 2024-03-15T13:00:54.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.496+0000 7f0830cc0640 1 --2- 172.21.15.2:0/2603376022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f082c105c30 0x7f082c196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.496+0000 7f0830cc0640 1 --2- 172.21.15.2:0/2603376022 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f08180a1ba0 0x7f08180a4060 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.496+0000 7f0830cc0640 1 --2- 172.21.15.2:0/2603376022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f082c100700 0x7f082c19cbc0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.496+0000 7f0830cc0640 1 --2- 172.21.15.2:0/2603376022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f082c0ffd40 0x7f082c19c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.496+0000 7f0830cc0640 1 -- 172.21.15.2:0/2603376022 >> 172.21.15.2:0/2603376022 conn(0x7f082c0756f0 msgr2=0x7f082c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:54.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.496+0000 7f0830cc0640 1 -- 172.21.15.2:0/2603376022 shutdown_connections 2024-03-15T13:00:54.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.496+0000 7f0830cc0640 1 -- 172.21.15.2:0/2603376022 wait complete. 2024-03-15T13:00:54.494 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:00:54.494 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:00:54.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.643+0000 7fdbded0d640 1 -- 172.21.15.2:0/3898796985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbd8100db0 msgr2=0x7fdbd810d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:54.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.643+0000 7fdbded0d640 1 --2- 172.21.15.2:0/3898796985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbd8100db0 0x7fdbd810d2c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009cd0 tx=0x7fdbd002c0f0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.644+0000 7fdbded0d640 1 -- 172.21.15.2:0/3898796985 shutdown_connections 2024-03-15T13:00:54.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.644+0000 7fdbded0d640 1 --2- 172.21.15.2:0/3898796985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbd8100db0 0x7fdbd810d2c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.644+0000 7fdbded0d640 1 --2- 172.21.15.2:0/3898796985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbd81003f0 0x7fdbd8100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.644+0000 7fdbded0d640 1 --2- 172.21.15.2:0/3898796985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbd8106490 0x7fdbd8106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.644+0000 7fdbded0d640 1 -- 172.21.15.2:0/3898796985 >> 172.21.15.2:0/3898796985 conn(0x7fdbd80774f0 msgr2=0x7fdbd8077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:54.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.644+0000 7fdbded0d640 1 -- 172.21.15.2:0/3898796985 shutdown_connections 2024-03-15T13:00:54.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.644+0000 7fdbded0d640 1 -- 172.21.15.2:0/3898796985 wait complete. 2024-03-15T13:00:54.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.644+0000 7fdbded0d640 1 Processor -- start 2024-03-15T13:00:54.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.644+0000 7fdbded0d640 1 -- start start 2024-03-15T13:00:54.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbded0d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbd81003f0 0x7fdbd8196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbded0d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbd8100db0 0x7fdbd8196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbded0d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbd8106490 0x7fdbd8197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbded0d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd81a5ec0 con 0x7fdbd81003f0 2024-03-15T13:00:54.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbded0d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd81a6030 con 0x7fdbd8100db0 2024-03-15T13:00:54.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbded0d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd81a61f0 con 0x7fdbd8106490 2024-03-15T13:00:54.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbdcaa9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbd81003f0 0x7fdbd8196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbdcaa9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbd81003f0 0x7fdbd8196780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60888/0 (socket says 172.21.15.2:60888) 2024-03-15T13:00:54.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbdcaa9640 1 -- 172.21.15.2:0/3773414736 learned_addr learned my addr 172.21.15.2:0/3773414736 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:54.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbd7fff640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbd8100db0 0x7fdbd8196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbdd2aa640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbd8106490 0x7fdbd8197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbdd2aa640 1 -- 172.21.15.2:0/3773414736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbd8100db0 msgr2=0x7fdbd8196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:54.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbdd2aa640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbd8100db0 0x7fdbd8196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbdd2aa640 1 -- 172.21.15.2:0/3773414736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbd81003f0 msgr2=0x7fdbd8196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:54.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbdd2aa640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbd81003f0 0x7fdbd8196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbdd2aa640 1 -- 172.21.15.2:0/3773414736 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd0009900 con 0x7fdbd8106490 2024-03-15T13:00:54.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbd7fff640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbd8100db0 0x7fdbd8196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:54.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.645+0000 7fdbdcaa9640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbd81003f0 0x7fdbd8196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:54.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.646+0000 7fdbdd2aa640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbd8106490 0x7fdbd8197200 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0037040 tx=0x7fdbd0034a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:54.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.646+0000 7fdbd5ffb640 1 -- 172.21.15.2:0/3773414736 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0032880 con 0x7fdbd8106490 2024-03-15T13:00:54.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.646+0000 7fdbded0d640 1 -- 172.21.15.2:0/3773414736 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd81a6470 con 0x7fdbd8106490 2024-03-15T13:00:54.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.646+0000 7fdbd5ffb640 1 -- 172.21.15.2:0/3773414736 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdbd0034e80 con 0x7fdbd8106490 2024-03-15T13:00:54.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.646+0000 7fdbd5ffb640 1 -- 172.21.15.2:0/3773414736 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0004790 con 0x7fdbd8106490 2024-03-15T13:00:54.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.646+0000 7fdbded0d640 1 -- 172.21.15.2:0/3773414736 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd81a68f0 con 0x7fdbd8106490 2024-03-15T13:00:54.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.647+0000 7fdbded0d640 1 -- 172.21.15.2:0/3773414736 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd80682e0 con 0x7fdbd8106490 2024-03-15T13:00:54.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.648+0000 7fdbd5ffb640 1 -- 172.21.15.2:0/3773414736 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fdbd000a040 con 0x7fdbd8106490 2024-03-15T13:00:54.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.649+0000 7fdbd5ffb640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdbc80a1ba0 0x7fdbc80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:54.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.649+0000 7fdbdcaa9640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdbc80a1ba0 0x7fdbc80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:54.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.649+0000 7fdbd5ffb640 1 -- 172.21.15.2:0/3773414736 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fdbd00f0580 con 0x7fdbd8106490 2024-03-15T13:00:54.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.649+0000 7fdbdcaa9640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdbc80a1ba0 0x7fdbc80a4060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc011440 tx=0x7fdbcc005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:54.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.653+0000 7fdbd5ffb640 1 -- 172.21.15.2:0/3773414736 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdbd00b91f0 con 0x7fdbd8106490 2024-03-15T13:00:54.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.924+0000 7fdbded0d640 1 -- 172.21.15.2:0/3773414736 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbd81089c0 con 0x7fdbd8106490 2024-03-15T13:00:54.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.925+0000 7fdbd5ffb640 1 -- 172.21.15.2:0/3773414736 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11462 (secure 0 0 0) 0x7fdbd00bc8b0 con 0x7fdbd8106490 2024-03-15T13:00:54.911 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:00:54.911 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:00:32.135225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24434,24539,24460,24464,24500]},"id":1}]} 2024-03-15T13:00:54.912 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 33 2024-03-15T13:00:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.927+0000 7fdbded0d640 1 -- 172.21.15.2:0/3773414736 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdbc80a1ba0 msgr2=0x7fdbc80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.927+0000 7fdbded0d640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdbc80a1ba0 0x7fdbc80a4060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc011440 tx=0x7fdbcc005f90 comp rx=0 tx=0).stop 2024-03-15T13:00:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.927+0000 7fdbded0d640 1 -- 172.21.15.2:0/3773414736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbd8106490 msgr2=0x7fdbd8197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.927+0000 7fdbded0d640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbd8106490 0x7fdbd8197200 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0037040 tx=0x7fdbd0034a80 comp rx=0 tx=0).stop 2024-03-15T13:00:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.927+0000 7fdbded0d640 1 -- 172.21.15.2:0/3773414736 shutdown_connections 2024-03-15T13:00:54.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.927+0000 7fdbded0d640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbd8106490 0x7fdbd8197200 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.927+0000 7fdbded0d640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbd8100db0 0x7fdbd8196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.927+0000 7fdbded0d640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdbc80a1ba0 0x7fdbc80a4060 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.927+0000 7fdbded0d640 1 --2- 172.21.15.2:0/3773414736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbd81003f0 0x7fdbd8196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.927+0000 7fdbded0d640 1 -- 172.21.15.2:0/3773414736 >> 172.21.15.2:0/3773414736 conn(0x7fdbd80774f0 msgr2=0x7fdbd8102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.927+0000 7fdbded0d640 1 -- 172.21.15.2:0/3773414736 shutdown_connections 2024-03-15T13:00:54.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:54.927+0000 7fdbded0d640 1 -- 172.21.15.2:0/3773414736 wait complete. 2024-03-15T13:00:54.925 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:00:54.925 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:00:55.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.058+0000 7fea544d4640 1 -- 172.21.15.2:0/665553332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea4c107580 msgr2=0x7fea4c107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.058+0000 7fea544d4640 1 --2- 172.21.15.2:0/665553332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea4c107580 0x7fea4c107960 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fea44009900 tx=0x7fea4402bc30 comp rx=0 tx=0).stop 2024-03-15T13:00:55.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.058+0000 7fea544d4640 1 -- 172.21.15.2:0/665553332 shutdown_connections 2024-03-15T13:00:55.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.058+0000 7fea544d4640 1 --2- 172.21.15.2:0/665553332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea4c1003b0 0x7fea4c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.058+0000 7fea544d4640 1 --2- 172.21.15.2:0/665553332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c0ff9f0 0x7fea4c0ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.058+0000 7fea544d4640 1 --2- 172.21.15.2:0/665553332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea4c107580 0x7fea4c107960 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.058+0000 7fea544d4640 1 -- 172.21.15.2:0/665553332 >> 172.21.15.2:0/665553332 conn(0x7fea4c0769b0 msgr2=0x7fea4c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.058+0000 7fea544d4640 1 -- 172.21.15.2:0/665553332 shutdown_connections 2024-03-15T13:00:55.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.058+0000 7fea544d4640 1 -- 172.21.15.2:0/665553332 wait complete. 2024-03-15T13:00:55.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.059+0000 7fea544d4640 1 Processor -- start 2024-03-15T13:00:55.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.059+0000 7fea544d4640 1 -- start start 2024-03-15T13:00:55.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.059+0000 7fea544d4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea4c0ff9f0 0x7fea4c19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.059+0000 7fea544d4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea4c1003b0 0x7fea4c19dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.059+0000 7fea544d4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c107580 0x7fea4c1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.059+0000 7fea544d4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c197f00 con 0x7fea4c0ff9f0 2024-03-15T13:00:55.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.059+0000 7fea544d4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c198070 con 0x7fea4c107580 2024-03-15T13:00:55.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.059+0000 7fea544d4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c198230 con 0x7fea4c1003b0 2024-03-15T13:00:55.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.059+0000 7fea52270640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea4c0ff9f0 0x7fea4c19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.059+0000 7fea52270640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea4c0ff9f0 0x7fea4c19d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60914/0 (socket says 172.21.15.2:60914) 2024-03-15T13:00:55.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.059+0000 7fea52270640 1 -- 172.21.15.2:0/2760296905 learned_addr learned my addr 172.21.15.2:0/2760296905 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.059+0000 7fea51a6f640 1 --2- 172.21.15.2:0/2760296905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea4c1003b0 0x7fea4c19dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.060+0000 7fea52a71640 1 --2- 172.21.15.2:0/2760296905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c107580 0x7fea4c1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.060+0000 7fea52270640 1 -- 172.21.15.2:0/2760296905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea4c1003b0 msgr2=0x7fea4c19dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.060+0000 7fea52270640 1 --2- 172.21.15.2:0/2760296905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea4c1003b0 0x7fea4c19dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.060+0000 7fea52270640 1 -- 172.21.15.2:0/2760296905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c107580 msgr2=0x7fea4c1978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.060+0000 7fea52270640 1 --2- 172.21.15.2:0/2760296905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c107580 0x7fea4c1978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.060+0000 7fea52270640 1 -- 172.21.15.2:0/2760296905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea44009560 con 0x7fea4c0ff9f0 2024-03-15T13:00:55.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.060+0000 7fea51a6f640 1 --2- 172.21.15.2:0/2760296905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea4c1003b0 0x7fea4c19dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:55.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.060+0000 7fea52270640 1 --2- 172.21.15.2:0/2760296905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea4c0ff9f0 0x7fea4c19d6b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fea44009230 tx=0x7fea4402cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.060+0000 7fea437fe640 1 -- 172.21.15.2:0/2760296905 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44004250 con 0x7fea4c0ff9f0 2024-03-15T13:00:55.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.060+0000 7fea544d4640 1 -- 172.21.15.2:0/2760296905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4c1984b0 con 0x7fea4c0ff9f0 2024-03-15T13:00:55.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.060+0000 7fea437fe640 1 -- 172.21.15.2:0/2760296905 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fea440043b0 con 0x7fea4c0ff9f0 2024-03-15T13:00:55.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.060+0000 7fea437fe640 1 -- 172.21.15.2:0/2760296905 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44003bf0 con 0x7fea4c0ff9f0 2024-03-15T13:00:55.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.060+0000 7fea544d4640 1 -- 172.21.15.2:0/2760296905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c1a4750 con 0x7fea4c0ff9f0 2024-03-15T13:00:55.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.063+0000 7fea437fe640 1 -- 172.21.15.2:0/2760296905 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fea44003d50 con 0x7fea4c0ff9f0 2024-03-15T13:00:55.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.064+0000 7fea57193740 1 Processor -- start 2024-03-15T13:00:55.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.064+0000 7fea57193740 1 -- start start 2024-03-15T13:00:55.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.064+0000 7fea437fe640 1 --2- 172.21.15.2:0/2760296905 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fea380a1ba0 0x7fea380a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.064+0000 7fea57193740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2250 0x561b150b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.064+0000 7fea57193740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2b70 0x561b150bb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.064+0000 7fea57193740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.064+0000 7fea57193740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1506d960 con 0x561b150bb750 2024-03-15T13:00:55.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.064+0000 7fea57193740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b15021310 con 0x561b150b2b70 2024-03-15T13:00:55.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.064+0000 7fea57193740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1501e840 con 0x561b150b2250 2024-03-15T13:00:55.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.064+0000 7fea437fe640 1 -- 172.21.15.2:0/2760296905 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fea440efa40 con 0x7fea4c0ff9f0 2024-03-15T13:00:55.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.064+0000 7fea52a71640 1 --2- 172.21.15.2:0/2760296905 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fea380a1ba0 0x7fea380a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.064+0000 7fea51a6f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2b70 0x561b150bb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.064+0000 7fea52270640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2250 0x561b150b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.065+0000 7fea51a6f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2b70 0x561b150bb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50836/0 (socket says 172.21.15.2:50836) 2024-03-15T13:00:55.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.065+0000 7fea51a6f640 1 -- 172.21.15.2:0/4011189079 learned_addr learned my addr 172.21.15.2:0/4011189079 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.065+0000 7fea52a71640 1 --2- 172.21.15.2:0/4011189079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.065+0000 7fea51a6f640 1 -- 172.21.15.2:0/4011189079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2250 msgr2=0x561b150b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.065+0000 7fea51a6f640 1 --2- 172.21.15.2:0/4011189079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2250 0x561b150b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.065+0000 7fea51a6f640 1 -- 172.21.15.2:0/4011189079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 msgr2=0x561b150bdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.065+0000 7fea51a6f640 1 --2- 172.21.15.2:0/4011189079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.065+0000 7fea51a6f640 1 -- 172.21.15.2:0/4011189079 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b14fdc060 con 0x561b150b2b70 2024-03-15T13:00:55.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.065+0000 7fea52a71640 1 --2- 172.21.15.2:0/4011189079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:00:55.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.065+0000 7fea51a6f640 1 --2- 172.21.15.2:0/4011189079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2b70 0x561b150bb210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0114f0 tx=0x7fea3c0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c212dfc7c48531 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.065+0000 7fea52a71640 1 --2- 172.21.15.2:0/2760296905 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fea380a1ba0 0x7fea380a4060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x561b150adb10 tx=0x7fea48009bd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.065+0000 7fea40bfe640 1 -- 172.21.15.2:0/4011189079 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c005020 con 0x561b150b2b70 2024-03-15T13:00:55.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.065+0000 7fea40bfe640 1 -- 172.21.15.2:0/4011189079 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fea3c005650 con 0x561b150b2b70 2024-03-15T13:00:55.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.065+0000 7fea40bfe640 1 -- 172.21.15.2:0/4011189079 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c00f880 con 0x561b150b2b70 2024-03-15T13:00:55.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.066+0000 7fea57193740 1 -- 172.21.15.2:0/4011189079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2b70 msgr2=0x561b150bb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.066+0000 7fea57193740 1 --2- 172.21.15.2:0/4011189079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2b70 0x561b150bb210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0114f0 tx=0x7fea3c0119b0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.066+0000 7fea57193740 1 -- 172.21.15.2:0/4011189079 shutdown_connections 2024-03-15T13:00:55.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.066+0000 7fea57193740 1 --2- 172.21.15.2:0/4011189079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.066+0000 7fea57193740 1 --2- 172.21.15.2:0/4011189079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2b70 0x561b150bb210 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.066+0000 7fea57193740 1 --2- 172.21.15.2:0/4011189079 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2250 0x561b150b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.066+0000 7fea57193740 1 -- 172.21.15.2:0/4011189079 >> 172.21.15.2:0/4011189079 conn(0x561b150ac980 msgr2=0x561b150acd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.066+0000 7fea57193740 1 -- 172.21.15.2:0/4011189079 shutdown_connections 2024-03-15T13:00:55.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.066+0000 7fea57193740 1 -- 172.21.15.2:0/4011189079 wait complete. 2024-03-15T13:00:55.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea57193740 1 -- start start 2024-03-15T13:00:55.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea57193740 1 Processor -- start 2024-03-15T13:00:55.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea57193740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150b2250 0x561b150bb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea57193740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 0x561b150b8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea57193740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150bb750 0x561b150b8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea57193740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1501a020 con 0x561b150b2250 2024-03-15T13:00:55.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea57193740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b14ced430 con 0x561b150bb750 2024-03-15T13:00:55.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea57193740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b14ca9e00 con 0x561b150b2b70 2024-03-15T13:00:55.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea57193740 10 client.0 _ll_register_callbacks cb 0x561b14fc1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:55.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea52270640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150b2250 0x561b150bb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea52270640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150b2250 0x561b150bb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60930/0 (socket says 172.21.15.2:60930) 2024-03-15T13:00:55.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea52270640 1 -- 172.21.15.2:0/2338424487 learned_addr learned my addr 172.21.15.2:0/2338424487 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea51a6f640 1 --2- 172.21.15.2:0/2338424487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 0x561b150b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea52a71640 1 --2- 172.21.15.2:0/2338424487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150bb750 0x561b150b8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea52270640 1 -- 172.21.15.2:0/2338424487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 msgr2=0x561b150b8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea52270640 1 --2- 172.21.15.2:0/2338424487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 0x561b150b8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea52270640 1 -- 172.21.15.2:0/2338424487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150bb750 msgr2=0x561b150b8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea52270640 1 --2- 172.21.15.2:0/2338424487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150bb750 0x561b150b8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea52270640 1 -- 172.21.15.2:0/2338424487 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3c009c80 con 0x561b150b2250 2024-03-15T13:00:55.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.067+0000 7fea51a6f640 1 --2- 172.21.15.2:0/2338424487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 0x561b150b8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:55.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea52270640 1 --2- 172.21.15.2:0/2338424487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150b2250 0x561b150bb080 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fea440ecfb0 tx=0x7fea440dc180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83ecd17651e2011 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:55.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea35ffb640 1 -- 172.21.15.2:0/2338424487 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea440bbf70 con 0x561b150b2250 2024-03-15T13:00:55.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea57193740 1 -- 172.21.15.2:0/2338424487 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b14d197e0 con 0x561b150b2250 2024-03-15T13:00:55.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea35ffb640 1 -- 172.21.15.2:0/2338424487 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fea440bcb30 con 0x561b150b2250 2024-03-15T13:00:55.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea35ffb640 1 -- 172.21.15.2:0/2338424487 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea440bbd90 con 0x561b150b2250 2024-03-15T13:00:55.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea57193740 10 client.16311 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:55.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea57193740 1 -- 172.21.15.2:0/2338424487 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561b150186a0 con 0x561b150b2250 2024-03-15T13:00:55.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea35ffb640 1 -- 172.21.15.2:0/2338424487 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x561b14d197e0 con 0x561b150b2250 2024-03-15T13:00:55.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea35ffb640 1 -- 172.21.15.2:0/2338424487 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fea4403c070 con 0x561b150b2250 2024-03-15T13:00:55.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea57193740 10 client.16311 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:55.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea57193740 10 client.16311 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:00:55.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea57193740 20 client.16311 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:55.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea57193740 1 --2- 172.21.15.2:0/2338424487 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x561b150cb500 0x561b150cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea57193740 4 client.16311 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:55.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.068+0000 7fea57193740 1 -- 172.21.15.2:0/2338424487 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b14d56720 con 0x561b150cb500 2024-03-15T13:00:55.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.069+0000 7fea51a6f640 1 --2- 172.21.15.2:0/2338424487 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x561b150cb500 0x561b150cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.069+0000 7fea51a6f640 1 --2- 172.21.15.2:0/2338424487 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x561b150cb500 0x561b150cd8f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=12dc49acad60e65c server_cookie=e2427d5a6d61043c in_seq=0 out_seq=0 2024-03-15T13:00:55.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.069+0000 7fea35ffb640 10 client.16311 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:00:55.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.070+0000 7fea35ffb640 1 -- 172.21.15.2:0/2338424487 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561b14d56720 con 0x561b150cb500 2024-03-15T13:00:55.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.070+0000 7fea35ffb640 10 client.16311 handle_command_reply: tid=0 2024-03-15T13:00:55.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.070+0000 7fea57193740 1 client.16311 shutdown 2024-03-15T13:00:55.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.070+0000 7fea57193740 1 -- 172.21.15.2:0/2338424487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150b2250 msgr2=0x561b150bb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.070+0000 7fea57193740 1 --2- 172.21.15.2:0/2338424487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150b2250 0x561b150bb080 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fea440ecfb0 tx=0x7fea440dc180 comp rx=0 tx=0).stop 2024-03-15T13:00:55.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.070+0000 7fea57193740 1 -- 172.21.15.2:0/2338424487 shutdown_connections 2024-03-15T13:00:55.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.070+0000 7fea57193740 1 --2- 172.21.15.2:0/2338424487 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x561b150cb500 0x561b150cd8f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.070+0000 7fea57193740 1 --2- 172.21.15.2:0/2338424487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150bb750 0x561b150b8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.070+0000 7fea57193740 1 --2- 172.21.15.2:0/2338424487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 0x561b150b8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.070+0000 7fea57193740 1 --2- 172.21.15.2:0/2338424487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150b2250 0x561b150bb080 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.070+0000 7fea35ffb640 0 client.16311 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:00:55.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.070+0000 7fea57193740 1 -- 172.21.15.2:0/2338424487 >> 172.21.15.2:0/2338424487 conn(0x561b150ac980 msgr2=0x561b150ae6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.071+0000 7fea57193740 1 -- 172.21.15.2:0/2338424487 shutdown_connections 2024-03-15T13:00:55.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.071+0000 7fea57193740 1 -- 172.21.15.2:0/2338424487 wait complete. 2024-03-15T13:00:55.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.071+0000 7fea57193740 20 client.16311 trim_cache size 0 max 16384 2024-03-15T13:00:55.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea57193740 1 Processor -- start 2024-03-15T13:00:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea57193740 1 -- start start 2024-03-15T13:00:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea57193740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2250 0x561b150b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea57193740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 0x561b150bab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea57193740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea57193740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b14f9f130 con 0x561b150bb750 2024-03-15T13:00:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea57193740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b14d56720 con 0x561b150b2250 2024-03-15T13:00:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea57193740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1509ea40 con 0x561b150b2b70 2024-03-15T13:00:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea52a71640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea52a71640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60936/0 (socket says 172.21.15.2:60936) 2024-03-15T13:00:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea52a71640 1 -- 172.21.15.2:0/1420720042 learned_addr learned my addr 172.21.15.2:0/1420720042 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea51a6f640 1 --2- 172.21.15.2:0/1420720042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 0x561b150bab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea52270640 1 --2- 172.21.15.2:0/1420720042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2250 0x561b150b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea52a71640 1 -- 172.21.15.2:0/1420720042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 msgr2=0x561b150bab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea52a71640 1 --2- 172.21.15.2:0/1420720042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 0x561b150bab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea52a71640 1 -- 172.21.15.2:0/1420720042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2250 msgr2=0x561b150b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea52a71640 1 --2- 172.21.15.2:0/1420720042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2250 0x561b150b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea52a71640 1 -- 172.21.15.2:0/1420720042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b14fb4c20 con 0x561b150bb750 2024-03-15T13:00:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.086+0000 7fea51a6f640 1 --2- 172.21.15.2:0/1420720042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 0x561b150bab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.087+0000 7fea52a71640 1 --2- 172.21.15.2:0/1420720042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bb080 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fea48009fd0 tx=0x7fea48015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e5b4a6d5d866d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.087+0000 7fea37fff640 1 -- 172.21.15.2:0/1420720042 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea48024070 con 0x561b150bb750 2024-03-15T13:00:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.087+0000 7fea37fff640 1 -- 172.21.15.2:0/1420720042 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fea4801fe90 con 0x561b150bb750 2024-03-15T13:00:55.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.087+0000 7fea37fff640 1 -- 172.21.15.2:0/1420720042 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4800e610 con 0x561b150bb750 2024-03-15T13:00:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 -- 172.21.15.2:0/1420720042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 msgr2=0x561b150bb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 --2- 172.21.15.2:0/1420720042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bb080 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fea48009fd0 tx=0x7fea48015040 comp rx=0 tx=0).stop 2024-03-15T13:00:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 -- 172.21.15.2:0/1420720042 shutdown_connections 2024-03-15T13:00:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 --2- 172.21.15.2:0/1420720042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bb080 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 --2- 172.21.15.2:0/1420720042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 0x561b150bab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 --2- 172.21.15.2:0/1420720042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2250 0x561b150b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 -- 172.21.15.2:0/1420720042 >> 172.21.15.2:0/1420720042 conn(0x561b150ac980 msgr2=0x561b150b8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 -- 172.21.15.2:0/1420720042 shutdown_connections 2024-03-15T13:00:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 -- 172.21.15.2:0/1420720042 wait complete. 2024-03-15T13:00:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 -- start start 2024-03-15T13:00:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 Processor -- start 2024-03-15T13:00:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2250 0x561b150c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 0x561b150bf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b150c0030 con 0x561b150bb750 2024-03-15T13:00:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b150c01a0 con 0x561b150b2250 2024-03-15T13:00:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b150c0360 con 0x561b150b2b70 2024-03-15T13:00:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.088+0000 7fea57193740 10 client.0 _ll_register_callbacks cb 0x7fea4402ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.089+0000 7fea52a71640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.089+0000 7fea51a6f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 0x561b150bf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.089+0000 7fea52a71640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60946/0 (socket says 172.21.15.2:60946) 2024-03-15T13:00:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.089+0000 7fea52a71640 1 -- 172.21.15.2:0/2681358489 learned_addr learned my addr 172.21.15.2:0/2681358489 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.089+0000 7fea52270640 1 --2- 172.21.15.2:0/2681358489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2250 0x561b150c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.089+0000 7fea51a6f640 1 -- 172.21.15.2:0/2681358489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2250 msgr2=0x561b150c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.089+0000 7fea51a6f640 1 --2- 172.21.15.2:0/2681358489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2250 0x561b150c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.089+0000 7fea51a6f640 1 -- 172.21.15.2:0/2681358489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 msgr2=0x561b150bfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.089+0000 7fea51a6f640 1 --2- 172.21.15.2:0/2681358489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.089+0000 7fea51a6f640 1 -- 172.21.15.2:0/2681358489 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b14fdc060 con 0x561b150b2b70 2024-03-15T13:00:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.089+0000 7fea51a6f640 1 --2- 172.21.15.2:0/2681358489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 0x561b150bf5b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fea3c01b040 tx=0x7fea3c0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b34e1f02ce8210d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.089+0000 7fea35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:00:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.089+0000 7fea35ffb640 1 -- 172.21.15.2:0/2681358489 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c029040 con 0x561b150b2b70 2024-03-15T13:00:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.089+0000 7fea35ffb640 1 -- 172.21.15.2:0/2681358489 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fea3c004dc0 con 0x561b150b2b70 2024-03-15T13:00:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.089+0000 7fea57193740 1 -- 172.21.15.2:0/2681358489 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea3c005650 con 0x561b150b2b70 2024-03-15T13:00:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.090+0000 7fea35ffb640 1 -- 172.21.15.2:0/2681358489 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea3c0291a0 con 0x561b150b2b70 2024-03-15T13:00:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.090+0000 7fea57193740 10 client.26113 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.090+0000 7fea57193740 1 -- 172.21.15.2:0/2681358489 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fea480119d0 con 0x561b150b2b70 2024-03-15T13:00:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.090+0000 7fea35ffb640 1 -- 172.21.15.2:0/2681358489 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fea3c005650 con 0x561b150b2b70 2024-03-15T13:00:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.090+0000 7fea35ffb640 1 -- 172.21.15.2:0/2681358489 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fea3c00f560 con 0x561b150b2b70 2024-03-15T13:00:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.090+0000 7fea57193740 10 client.26113 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:55.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.090+0000 7fea57193740 10 client.26113 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:00:55.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.090+0000 7fea57193740 20 client.26113 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:55.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.090+0000 7fea57193740 1 --2- 172.21.15.2:0/2681358489 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x561b15128320 0x561b1512a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.090+0000 7fea57193740 4 client.26113 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:00:55.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.090+0000 7fea57193740 1 -- 172.21.15.2:0/2681358489 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561b14da5890 con 0x561b15128320 2024-03-15T13:00:55.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.090+0000 7fea52270640 1 --2- 172.21.15.2:0/2681358489 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x561b15128320 0x561b1512a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea52270640 1 --2- 172.21.15.2:0/2681358489 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x561b15128320 0x561b1512a740 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9940d6457a65e955 server_cookie=c335a36960235013 in_seq=0 out_seq=0 2024-03-15T13:00:55.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea35ffb640 10 client.26113 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:00:55.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea35ffb640 1 -- 172.21.15.2:0/2681358489 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561b14da5890 con 0x561b15128320 2024-03-15T13:00:55.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea35ffb640 10 client.26113 handle_command_reply: tid=0 2024-03-15T13:00:55.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea57193740 1 client.26113 shutdown 2024-03-15T13:00:55.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea57193740 1 -- 172.21.15.2:0/2681358489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 msgr2=0x561b150bf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea57193740 1 --2- 172.21.15.2:0/2681358489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 0x561b150bf5b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fea3c01b040 tx=0x7fea3c0119b0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea57193740 1 -- 172.21.15.2:0/2681358489 shutdown_connections 2024-03-15T13:00:55.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea57193740 1 --2- 172.21.15.2:0/2681358489 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x561b15128320 0x561b1512a740 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea57193740 1 --2- 172.21.15.2:0/2681358489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b150bb750 0x561b150bfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea57193740 1 --2- 172.21.15.2:0/2681358489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561b150b2b70 0x561b150bf5b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea35ffb640 0 client.26113 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:00:55.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea57193740 1 --2- 172.21.15.2:0/2681358489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b150b2250 0x561b150c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea57193740 1 -- 172.21.15.2:0/2681358489 >> 172.21.15.2:0/2681358489 conn(0x561b150ac980 msgr2=0x561b150b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea57193740 1 -- 172.21.15.2:0/2681358489 shutdown_connections 2024-03-15T13:00:55.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea57193740 1 -- 172.21.15.2:0/2681358489 wait complete. 2024-03-15T13:00:55.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.091+0000 7fea57193740 20 client.26113 trim_cache size 0 max 16384 2024-03-15T13:00:55.083 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:00:55.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.092+0000 7fea544d4640 1 -- 172.21.15.2:0/2760296905 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fea380a1ba0 msgr2=0x7fea380a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.092+0000 7fea544d4640 1 --2- 172.21.15.2:0/2760296905 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fea380a1ba0 0x7fea380a4060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x561b150adb10 tx=0x7fea48009bd0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.092+0000 7fea544d4640 1 -- 172.21.15.2:0/2760296905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea4c0ff9f0 msgr2=0x7fea4c19d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.092+0000 7fea544d4640 1 --2- 172.21.15.2:0/2760296905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea4c0ff9f0 0x7fea4c19d6b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fea44009230 tx=0x7fea4402cac0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.092+0000 7fea544d4640 1 -- 172.21.15.2:0/2760296905 shutdown_connections 2024-03-15T13:00:55.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.092+0000 7fea544d4640 1 --2- 172.21.15.2:0/2760296905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c107580 0x7fea4c1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.092+0000 7fea544d4640 1 --2- 172.21.15.2:0/2760296905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea4c1003b0 0x7fea4c19dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.092+0000 7fea544d4640 1 --2- 172.21.15.2:0/2760296905 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fea380a1ba0 0x7fea380a4060 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.092+0000 7fea544d4640 1 --2- 172.21.15.2:0/2760296905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea4c0ff9f0 0x7fea4c19d6b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.092+0000 7fea544d4640 1 -- 172.21.15.2:0/2760296905 >> 172.21.15.2:0/2760296905 conn(0x7fea4c0769b0 msgr2=0x7fea4c0fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.092+0000 7fea544d4640 1 -- 172.21.15.2:0/2760296905 shutdown_connections 2024-03-15T13:00:55.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.092+0000 7fea544d4640 1 -- 172.21.15.2:0/2760296905 wait complete. 2024-03-15T13:00:55.091 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:00:55.091 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:00:55.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.219+0000 7fc0d670d640 1 -- 172.21.15.2:0/1138396099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0d0101e60 msgr2=0x7fc0d010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.219+0000 7fc0d670d640 1 --2- 172.21.15.2:0/1138396099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0d0101e60 0x7fc0d010e320 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009ca0 tx=0x7fc0c802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.219+0000 7fc0d670d640 1 -- 172.21.15.2:0/1138396099 shutdown_connections 2024-03-15T13:00:55.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.219+0000 7fc0d670d640 1 --2- 172.21.15.2:0/1138396099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0d0101e60 0x7fc0d010e320 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.219+0000 7fc0d670d640 1 --2- 172.21.15.2:0/1138396099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0d01014a0 0x7fc0d0101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.219+0000 7fc0d670d640 1 --2- 172.21.15.2:0/1138396099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0d0107560 0x7fc0d0107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.219+0000 7fc0d670d640 1 -- 172.21.15.2:0/1138396099 >> 172.21.15.2:0/1138396099 conn(0x7fc0d00fcf90 msgr2=0x7fc0d00ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.219+0000 7fc0d670d640 1 -- 172.21.15.2:0/1138396099 shutdown_connections 2024-03-15T13:00:55.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.219+0000 7fc0d670d640 1 -- 172.21.15.2:0/1138396099 wait complete. 2024-03-15T13:00:55.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.219+0000 7fc0d670d640 1 Processor -- start 2024-03-15T13:00:55.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.219+0000 7fc0d670d640 1 -- start start 2024-03-15T13:00:55.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0d670d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0d01014a0 0x7fc0d00737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0d670d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0d0101e60 0x7fc0d0073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0d670d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0d0107560 0x7fc0d0074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0d670d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d01a5de0 con 0x7fc0d01014a0 2024-03-15T13:00:55.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0d670d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d01a5f50 con 0x7fc0d0101e60 2024-03-15T13:00:55.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0d670d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d01a6110 con 0x7fc0d0107560 2024-03-15T13:00:55.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0cffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0d01014a0 0x7fc0d00737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0cffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0d01014a0 0x7fc0d00737f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60962/0 (socket says 172.21.15.2:60962) 2024-03-15T13:00:55.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0cffff640 1 -- 172.21.15.2:0/896247839 learned_addr learned my addr 172.21.15.2:0/896247839 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0cf7fe640 1 --2- 172.21.15.2:0/896247839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0d0101e60 0x7fc0d0073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0d4caa640 1 --2- 172.21.15.2:0/896247839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0d0107560 0x7fc0d0074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0cffff640 1 -- 172.21.15.2:0/896247839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0d0107560 msgr2=0x7fc0d0074270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0cffff640 1 --2- 172.21.15.2:0/896247839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0d0107560 0x7fc0d0074270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0cffff640 1 -- 172.21.15.2:0/896247839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0d0101e60 msgr2=0x7fc0d0073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0cffff640 1 --2- 172.21.15.2:0/896247839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0d0101e60 0x7fc0d0073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0cffff640 1 -- 172.21.15.2:0/896247839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c8009900 con 0x7fc0d01014a0 2024-03-15T13:00:55.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0cffff640 1 --2- 172.21.15.2:0/896247839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0d01014a0 0x7fc0d00737f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000e9f0 tx=0x7fc0c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0cd7fa640 1 -- 172.21.15.2:0/896247839 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0009720 con 0x7fc0d01014a0 2024-03-15T13:00:55.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0d670d640 1 -- 172.21.15.2:0/896247839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d01a63f0 con 0x7fc0d01014a0 2024-03-15T13:00:55.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0cd7fa640 1 -- 172.21.15.2:0/896247839 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc0c000cd80 con 0x7fc0d01014a0 2024-03-15T13:00:55.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.220+0000 7fc0cd7fa640 1 -- 172.21.15.2:0/896247839 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0010690 con 0x7fc0d01014a0 2024-03-15T13:00:55.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.221+0000 7fc0d670d640 1 -- 172.21.15.2:0/896247839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d01a68b0 con 0x7fc0d01014a0 2024-03-15T13:00:55.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0cd7fa640 1 -- 172.21.15.2:0/896247839 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fc0c0004e90 con 0x7fc0d01014a0 2024-03-15T13:00:55.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0d93c1740 1 Processor -- start 2024-03-15T13:00:55.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0d93c1740 1 -- start start 2024-03-15T13:00:55.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0d93c1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5250 0x55e9921c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0d93c1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0d93c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921ce750 0x55e9921d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0cd7fa640 1 --2- 172.21.15.2:0/896247839 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc0b80a1ba0 0x7fc0b80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0d93c1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e992180960 con 0x55e9921c5b70 2024-03-15T13:00:55.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0d93c1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e992134310 con 0x55e9921c5250 2024-03-15T13:00:55.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0d93c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e992131840 con 0x55e9921ce750 2024-03-15T13:00:55.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0d4caa640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5250 0x55e9921c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0cd7fa640 1 -- 172.21.15.2:0/896247839 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc0c0014070 con 0x7fc0d01014a0 2024-03-15T13:00:55.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0cffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0d4caa640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5250 0x55e9921c5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50878/0 (socket says 172.21.15.2:50878) 2024-03-15T13:00:55.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0d4caa640 1 -- 172.21.15.2:0/2682790673 learned_addr learned my addr 172.21.15.2:0/2682790673 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0cf7fe640 1 --2- 172.21.15.2:0/2682790673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921ce750 0x55e9921d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.223+0000 7fc0d4caa640 1 --2- 172.21.15.2:0/896247839 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc0b80a1ba0 0x7fc0b80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.224+0000 7fc0cffff640 1 -- 172.21.15.2:0/2682790673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921ce750 msgr2=0x55e9921d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.224+0000 7fc0cffff640 1 --2- 172.21.15.2:0/2682790673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921ce750 0x55e9921d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.224+0000 7fc0cffff640 1 -- 172.21.15.2:0/2682790673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5250 msgr2=0x55e9921c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.224+0000 7fc0cffff640 1 --2- 172.21.15.2:0/2682790673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5250 0x55e9921c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.224+0000 7fc0cffff640 1 -- 172.21.15.2:0/2682790673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9920ef060 con 0x55e9921c5b70 2024-03-15T13:00:55.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.224+0000 7fc0cf7fe640 1 --2- 172.21.15.2:0/2682790673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921ce750 0x55e9921d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:55.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.224+0000 7fc0cffff640 1 --2- 172.21.15.2:0/2682790673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921ce210 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00cb230 tx=0x7fc0c006c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3124b105fb31f371 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.224+0000 7fc0d4caa640 1 --2- 172.21.15.2:0/896247839 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc0b80a1ba0 0x7fc0b80a4060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009430 tx=0x7fc0c8005ef0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.224+0000 7fc0b67fc640 1 -- 172.21.15.2:0/2682790673 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c009ae10 con 0x55e9921c5b70 2024-03-15T13:00:55.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.224+0000 7fc0b67fc640 1 -- 172.21.15.2:0/2682790673 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc0c0083e90 con 0x55e9921c5b70 2024-03-15T13:00:55.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.224+0000 7fc0b67fc640 1 -- 172.21.15.2:0/2682790673 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0098b30 con 0x55e9921c5b70 2024-03-15T13:00:55.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.225+0000 7fc0d93c1740 1 -- 172.21.15.2:0/2682790673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 msgr2=0x55e9921ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.225+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/2682790673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921ce210 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00cb230 tx=0x7fc0c006c9d0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.225+0000 7fc0d93c1740 1 -- 172.21.15.2:0/2682790673 shutdown_connections 2024-03-15T13:00:55.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.225+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/2682790673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921ce750 0x55e9921d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.225+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/2682790673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921ce210 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.225+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/2682790673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5250 0x55e9921c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.225+0000 7fc0d93c1740 1 -- 172.21.15.2:0/2682790673 >> 172.21.15.2:0/2682790673 conn(0x55e9921bf980 msgr2=0x55e9921bfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.225+0000 7fc0d93c1740 1 -- 172.21.15.2:0/2682790673 shutdown_connections 2024-03-15T13:00:55.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.225+0000 7fc0d93c1740 1 -- 172.21.15.2:0/2682790673 wait complete. 2024-03-15T13:00:55.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.225+0000 7fc0d93c1740 1 -- start start 2024-03-15T13:00:55.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.225+0000 7fc0d93c1740 1 Processor -- start 2024-03-15T13:00:55.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.225+0000 7fc0d93c1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5250 0x55e9921ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.225+0000 7fc0d93c1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0d93c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921ce750 0x55e9921cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0d93c1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99212d020 con 0x55e9921c5b70 2024-03-15T13:00:55.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0d93c1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e991e00430 con 0x55e9921c5250 2024-03-15T13:00:55.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0d93c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e991dbce00 con 0x55e9921ce750 2024-03-15T13:00:55.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0d93c1740 10 client.0 _ll_register_callbacks cb 0x55e9920d4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:55.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0cf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0cf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60976/0 (socket says 172.21.15.2:60976) 2024-03-15T13:00:55.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0d4caa640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921ce750 0x55e9921cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0cf7fe640 1 -- 172.21.15.2:0/897295305 learned_addr learned my addr 172.21.15.2:0/897295305 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0cffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5250 0x55e9921ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0cf7fe640 1 -- 172.21.15.2:0/897295305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921ce750 msgr2=0x55e9921cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0cf7fe640 1 --2- 172.21.15.2:0/897295305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921ce750 0x55e9921cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0cf7fe640 1 -- 172.21.15.2:0/897295305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5250 msgr2=0x55e9921ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0cf7fe640 1 --2- 172.21.15.2:0/897295305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5250 0x55e9921ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0cf7fe640 1 -- 172.21.15.2:0/897295305 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d01a68b0 con 0x55e9921c5b70 2024-03-15T13:00:55.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0d4caa640 1 --2- 172.21.15.2:0/897295305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921ce750 0x55e9921cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:55.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0cf7fe640 1 --2- 172.21.15.2:0/897295305 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921cb490 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4012720 tx=0x7fc0c4012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff0cdb514e2dd93 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:55.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0af7fe640 1 -- 172.21.15.2:0/897295305 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4017070 con 0x55e9921c5b70 2024-03-15T13:00:55.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0af7fe640 1 -- 172.21.15.2:0/897295305 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc0c4004030 con 0x55e9921c5b70 2024-03-15T13:00:55.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0d93c1740 1 -- 172.21.15.2:0/897295305 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e991e2c7e0 con 0x55e9921c5b70 2024-03-15T13:00:55.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0af7fe640 1 -- 172.21.15.2:0/897295305 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0c4004230 con 0x55e9921c5b70 2024-03-15T13:00:55.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0d93c1740 10 client.16329 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:55.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.226+0000 7fc0d93c1740 1 -- 172.21.15.2:0/897295305 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e99212b6a0 con 0x55e9921c5b70 2024-03-15T13:00:55.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.227+0000 7fc0af7fe640 1 -- 172.21.15.2:0/897295305 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e991e2c7e0 con 0x55e9921c5b70 2024-03-15T13:00:55.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.227+0000 7fc0af7fe640 1 -- 172.21.15.2:0/897295305 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fc0c40039c0 con 0x55e9921c5b70 2024-03-15T13:00:55.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.227+0000 7fc0d93c1740 10 client.16329 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:55.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.227+0000 7fc0d93c1740 10 client.16329 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:00:55.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.227+0000 7fc0d93c1740 20 client.16329 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:55.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.227+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/897295305 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55e9921de960 0x55e9921e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.227+0000 7fc0d93c1740 4 client.16329 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:55.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.227+0000 7fc0d93c1740 1 -- 172.21.15.2:0/897295305 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e991e69720 con 0x55e9921de960 2024-03-15T13:00:55.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.227+0000 7fc0cffff640 1 --2- 172.21.15.2:0/897295305 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55e9921de960 0x55e9921e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.227+0000 7fc0cffff640 1 --2- 172.21.15.2:0/897295305 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55e9921de960 0x55e9921e0d80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1856ebe883d88553 server_cookie=154202cbec838a6d in_seq=0 out_seq=0 2024-03-15T13:00:55.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.227+0000 7fc0af7fe640 10 client.16329 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:00:55.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.228+0000 7fc0af7fe640 1 -- 172.21.15.2:0/897295305 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e991e69720 con 0x55e9921de960 2024-03-15T13:00:55.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.228+0000 7fc0af7fe640 10 client.16329 handle_command_reply: tid=0 2024-03-15T13:00:55.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.228+0000 7fc0d93c1740 1 client.16329 shutdown 2024-03-15T13:00:55.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.228+0000 7fc0d93c1740 1 -- 172.21.15.2:0/897295305 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 msgr2=0x55e9921cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.228+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/897295305 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921cb490 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4012720 tx=0x7fc0c4012be0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.228+0000 7fc0d93c1740 1 -- 172.21.15.2:0/897295305 shutdown_connections 2024-03-15T13:00:55.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.228+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/897295305 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55e9921de960 0x55e9921e0d80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.228+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/897295305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921ce750 0x55e9921cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.228+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/897295305 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921cb490 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.228+0000 7fc0af7fe640 0 client.16329 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:00:55.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.228+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/897295305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5250 0x55e9921ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.228+0000 7fc0d93c1740 1 -- 172.21.15.2:0/897295305 >> 172.21.15.2:0/897295305 conn(0x55e9921bf980 msgr2=0x55e9921c16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.228+0000 7fc0d93c1740 1 -- 172.21.15.2:0/897295305 shutdown_connections 2024-03-15T13:00:55.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.228+0000 7fc0d93c1740 1 -- 172.21.15.2:0/897295305 wait complete. 2024-03-15T13:00:55.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.228+0000 7fc0d93c1740 20 client.16329 trim_cache size 0 max 16384 2024-03-15T13:00:55.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d93c1740 1 Processor -- start 2024-03-15T13:00:55.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d93c1740 1 -- start start 2024-03-15T13:00:55.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d93c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921c5250 0x55e9921cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d93c1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5b70 0x55e9921ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d93c1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921ce750 0x55e9921d9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d93c1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9921dadb0 con 0x55e9921ce750 2024-03-15T13:00:55.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d93c1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9920b2130 con 0x55e9921c5b70 2024-03-15T13:00:55.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d93c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e991e69720 con 0x55e9921c5250 2024-03-15T13:00:55.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d4caa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921ce750 0x55e9921d9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d4caa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921ce750 0x55e9921d9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60982/0 (socket says 172.21.15.2:60982) 2024-03-15T13:00:55.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d4caa640 1 -- 172.21.15.2:0/3135629145 learned_addr learned my addr 172.21.15.2:0/3135629145 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0cffff640 1 --2- 172.21.15.2:0/3135629145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921c5250 0x55e9921cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0cf7fe640 1 --2- 172.21.15.2:0/3135629145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5b70 0x55e9921ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d4caa640 1 -- 172.21.15.2:0/3135629145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921c5250 msgr2=0x55e9921cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d4caa640 1 --2- 172.21.15.2:0/3135629145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921c5250 0x55e9921cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d4caa640 1 -- 172.21.15.2:0/3135629145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5b70 msgr2=0x55e9921ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d4caa640 1 --2- 172.21.15.2:0/3135629145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5b70 0x55e9921ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0d4caa640 1 -- 172.21.15.2:0/3135629145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9920c7c20 con 0x55e9921ce750 2024-03-15T13:00:55.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.241+0000 7fc0cffff640 1 --2- 172.21.15.2:0/3135629145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921c5250 0x55e9921cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:55.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.242+0000 7fc0d4caa640 1 --2- 172.21.15.2:0/3135629145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921ce750 0x55e9921d9ad0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009ca0 tx=0x7fc0c803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5265134f8a565eb server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.242+0000 7fc0b5ffb640 1 -- 172.21.15.2:0/3135629145 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8048070 con 0x55e9921ce750 2024-03-15T13:00:55.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.242+0000 7fc0b5ffb640 1 -- 172.21.15.2:0/3135629145 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc0c8032e70 con 0x55e9921ce750 2024-03-15T13:00:55.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.242+0000 7fc0b5ffb640 1 -- 172.21.15.2:0/3135629145 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c80042a0 con 0x55e9921ce750 2024-03-15T13:00:55.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.242+0000 7fc0d93c1740 1 -- 172.21.15.2:0/3135629145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921ce750 msgr2=0x55e9921d9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.242+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/3135629145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921ce750 0x55e9921d9ad0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009ca0 tx=0x7fc0c803e000 comp rx=0 tx=0).stop 2024-03-15T13:00:55.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.242+0000 7fc0d93c1740 1 -- 172.21.15.2:0/3135629145 shutdown_connections 2024-03-15T13:00:55.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.242+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/3135629145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921ce750 0x55e9921d9ad0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.242+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/3135629145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921c5b70 0x55e9921ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.242+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/3135629145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921c5250 0x55e9921cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.242+0000 7fc0d93c1740 1 -- 172.21.15.2:0/3135629145 >> 172.21.15.2:0/3135629145 conn(0x55e9921bf980 msgr2=0x55e9921beb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.242+0000 7fc0d93c1740 1 -- 172.21.15.2:0/3135629145 shutdown_connections 2024-03-15T13:00:55.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.242+0000 7fc0d93c1740 1 -- 172.21.15.2:0/3135629145 wait complete. 2024-03-15T13:00:55.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.243+0000 7fc0d93c1740 1 -- start start 2024-03-15T13:00:55.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.243+0000 7fc0d93c1740 1 Processor -- start 2024-03-15T13:00:55.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.243+0000 7fc0d93c1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921c5250 0x55e9921c8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.243+0000 7fc0d93c1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.243+0000 7fc0d93c1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921ce750 0x55e9922331e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.243+0000 7fc0d93c1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e992233720 con 0x55e9921c5b70 2024-03-15T13:00:55.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.243+0000 7fc0d93c1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e992233890 con 0x55e9921ce750 2024-03-15T13:00:55.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.243+0000 7fc0d93c1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e992233a50 con 0x55e9921c5250 2024-03-15T13:00:55.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.243+0000 7fc0d93c1740 10 client.0 _ll_register_callbacks cb 0x7fc0c4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:55.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.243+0000 7fc0cf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.243+0000 7fc0cffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921c5250 0x55e9921c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.243+0000 7fc0cffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921c5250 0x55e9921c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:34906/0 (socket says 172.21.15.2:34906) 2024-03-15T13:00:55.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.243+0000 7fc0cffff640 1 -- 172.21.15.2:0/3590573041 learned_addr learned my addr 172.21.15.2:0/3590573041 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.243+0000 7fc0d4caa640 1 --2- 172.21.15.2:0/3590573041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921ce750 0x55e9922331e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0cffff640 1 -- 172.21.15.2:0/3590573041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921ce750 msgr2=0x55e9922331e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0cffff640 1 --2- 172.21.15.2:0/3590573041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921ce750 0x55e9922331e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0cffff640 1 -- 172.21.15.2:0/3590573041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 msgr2=0x55e9921c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0cffff640 1 --2- 172.21.15.2:0/3590573041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0cffff640 1 -- 172.21.15.2:0/3590573041 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9920ef060 con 0x55e9921c5250 2024-03-15T13:00:55.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0cf7fe640 1 --2- 172.21.15.2:0/3590573041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:55.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0cffff640 1 --2- 172.21.15.2:0/3590573041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921c5250 0x55e9921c8390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00cb2e0 tx=0x7fc0c00ba040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58c3297986a48b44 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:00:55.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0af7fe640 1 -- 172.21.15.2:0/3590573041 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c009ae10 con 0x55e9921c5250 2024-03-15T13:00:55.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0af7fe640 1 -- 172.21.15.2:0/3590573041 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc0c009e860 con 0x55e9921c5250 2024-03-15T13:00:55.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0d93c1740 1 -- 172.21.15.2:0/3590573041 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0c0083e90 con 0x55e9921c5250 2024-03-15T13:00:55.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0af7fe640 1 -- 172.21.15.2:0/3590573041 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0c0098a10 con 0x55e9921c5250 2024-03-15T13:00:55.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0d93c1740 10 client.26131 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:55.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0d93c1740 1 -- 172.21.15.2:0/3590573041 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fc0d010ec20 con 0x55e9921c5250 2024-03-15T13:00:55.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0af7fe640 1 -- 172.21.15.2:0/3590573041 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc0c0083e90 con 0x55e9921c5250 2024-03-15T13:00:55.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0af7fe640 1 -- 172.21.15.2:0/3590573041 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fc0c001f060 con 0x55e9921c5250 2024-03-15T13:00:55.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0d93c1740 10 client.26131 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:55.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0d93c1740 10 client.26131 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:00:55.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.244+0000 7fc0d93c1740 20 client.26131 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:55.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/3590573041 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55e99223b010 0x55e99223d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0d93c1740 4 client.26131 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:00:55.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0d93c1740 1 -- 172.21.15.2:0/3590573041 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e991eb8890 con 0x55e99223b010 2024-03-15T13:00:55.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0cf7fe640 1 --2- 172.21.15.2:0/3590573041 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55e99223b010 0x55e99223d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0af7fe640 10 client.26131 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:00:55.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0cf7fe640 1 --2- 172.21.15.2:0/3590573041 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55e99223b010 0x55e99223d430 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=93d03b43c88750d2 server_cookie=5f154883e043ea49 in_seq=0 out_seq=0 2024-03-15T13:00:55.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0af7fe640 1 -- 172.21.15.2:0/3590573041 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e991eb8890 con 0x55e99223b010 2024-03-15T13:00:55.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0af7fe640 10 client.26131 handle_command_reply: tid=0 2024-03-15T13:00:55.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0d93c1740 1 client.26131 shutdown 2024-03-15T13:00:55.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0d93c1740 1 -- 172.21.15.2:0/3590573041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921c5250 msgr2=0x55e9921c8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/3590573041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921c5250 0x55e9921c8390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00cb2e0 tx=0x7fc0c00ba040 comp rx=0 tx=0).stop 2024-03-15T13:00:55.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0d93c1740 1 -- 172.21.15.2:0/3590573041 shutdown_connections 2024-03-15T13:00:55.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/3590573041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9921ce750 0x55e9922331e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/3590573041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e9921c5b70 0x55e9921c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/3590573041 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55e99223b010 0x55e99223d430 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0d93c1740 1 --2- 172.21.15.2:0/3590573041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9921c5250 0x55e9921c8390 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0d93c1740 1 -- 172.21.15.2:0/3590573041 >> 172.21.15.2:0/3590573041 conn(0x55e9921bf980 msgr2=0x55e9921c6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.245+0000 7fc0af7fe640 0 client.26131 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:00:55.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.246+0000 7fc0d93c1740 1 -- 172.21.15.2:0/3590573041 shutdown_connections 2024-03-15T13:00:55.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.246+0000 7fc0d93c1740 1 -- 172.21.15.2:0/3590573041 wait complete. 2024-03-15T13:00:55.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.246+0000 7fc0d93c1740 20 client.26131 trim_cache size 0 max 16384 2024-03-15T13:00:55.240 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:00:55.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.246+0000 7fc0d670d640 1 -- 172.21.15.2:0/896247839 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc0b80a1ba0 msgr2=0x7fc0b80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.246+0000 7fc0d670d640 1 --2- 172.21.15.2:0/896247839 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc0b80a1ba0 0x7fc0b80a4060 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009430 tx=0x7fc0c8005ef0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.246+0000 7fc0d670d640 1 -- 172.21.15.2:0/896247839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0d01014a0 msgr2=0x7fc0d00737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.246+0000 7fc0d670d640 1 --2- 172.21.15.2:0/896247839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0d01014a0 0x7fc0d00737f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000e9f0 tx=0x7fc0c000edf0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.246+0000 7fc0d670d640 1 -- 172.21.15.2:0/896247839 shutdown_connections 2024-03-15T13:00:55.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.246+0000 7fc0d670d640 1 --2- 172.21.15.2:0/896247839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0d0107560 0x7fc0d0074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.246+0000 7fc0d670d640 1 --2- 172.21.15.2:0/896247839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0d0101e60 0x7fc0d0073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.246+0000 7fc0d670d640 1 --2- 172.21.15.2:0/896247839 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc0b80a1ba0 0x7fc0b80a4060 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.246+0000 7fc0d670d640 1 --2- 172.21.15.2:0/896247839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0d01014a0 0x7fc0d00737f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.246+0000 7fc0d670d640 1 -- 172.21.15.2:0/896247839 >> 172.21.15.2:0/896247839 conn(0x7fc0d00fcf90 msgr2=0x7fc0d00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.246+0000 7fc0d670d640 1 -- 172.21.15.2:0/896247839 shutdown_connections 2024-03-15T13:00:55.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.246+0000 7fc0d670d640 1 -- 172.21.15.2:0/896247839 wait complete. 2024-03-15T13:00:55.243 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:00:55.243 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:00:55.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.363+0000 7f8a4a5a0640 1 -- 172.21.15.2:0/582903279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a44103050 msgr2=0x7f8a4410b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.364+0000 7f8a4a5a0640 1 --2- 172.21.15.2:0/582903279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a44103050 0x7f8a4410b530 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009860 tx=0x7f8a3802bc00 comp rx=0 tx=0).stop 2024-03-15T13:00:55.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.364+0000 7f8a4a5a0640 1 -- 172.21.15.2:0/582903279 shutdown_connections 2024-03-15T13:00:55.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.364+0000 7f8a4a5a0640 1 --2- 172.21.15.2:0/582903279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a4410bb00 0x7f8a4410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.364+0000 7f8a4a5a0640 1 --2- 172.21.15.2:0/582903279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a44103050 0x7f8a4410b530 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.364+0000 7f8a4a5a0640 1 --2- 172.21.15.2:0/582903279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a44102730 0x7f8a44102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.364+0000 7f8a4a5a0640 1 -- 172.21.15.2:0/582903279 >> 172.21.15.2:0/582903279 conn(0x7f8a44076770 msgr2=0x7f8a44076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.364+0000 7f8a4a5a0640 1 -- 172.21.15.2:0/582903279 shutdown_connections 2024-03-15T13:00:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.364+0000 7f8a4a5a0640 1 -- 172.21.15.2:0/582903279 wait complete. 2024-03-15T13:00:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.364+0000 7f8a4a5a0640 1 Processor -- start 2024-03-15T13:00:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.364+0000 7f8a4a5a0640 1 -- start start 2024-03-15T13:00:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a4a5a0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a44102730 0x7f8a4419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a4a5a0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a44103050 0x7f8a4419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a4a5a0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a4410bb00 0x7f8a441979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a4a5a0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44197ff0 con 0x7f8a44102730 2024-03-15T13:00:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a4a5a0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44198160 con 0x7f8a44103050 2024-03-15T13:00:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a4a5a0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44198320 con 0x7f8a4410bb00 2024-03-15T13:00:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a44102730 0x7f8a4419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a44102730 0x7f8a4419d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32774/0 (socket says 172.21.15.2:32774) 2024-03-15T13:00:55.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a43fff640 1 -- 172.21.15.2:0/2393300010 learned_addr learned my addr 172.21.15.2:0/2393300010 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a437fe640 1 --2- 172.21.15.2:0/2393300010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a44103050 0x7f8a4419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a48b3d640 1 --2- 172.21.15.2:0/2393300010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a4410bb00 0x7f8a441979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a43fff640 1 -- 172.21.15.2:0/2393300010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a4410bb00 msgr2=0x7f8a441979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a43fff640 1 --2- 172.21.15.2:0/2393300010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a4410bb00 0x7f8a441979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a43fff640 1 -- 172.21.15.2:0/2393300010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a44103050 msgr2=0x7f8a4419dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a43fff640 1 --2- 172.21.15.2:0/2393300010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a44103050 0x7f8a4419dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a43fff640 1 -- 172.21.15.2:0/2393300010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009560 con 0x7f8a44102730 2024-03-15T13:00:55.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a43fff640 1 --2- 172.21.15.2:0/2393300010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a44102730 0x7f8a4419d7f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400e9c0 tx=0x7f8a3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a417fa640 1 -- 172.21.15.2:0/2393300010 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400cde0 con 0x7f8a44102730 2024-03-15T13:00:55.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a4a5a0640 1 -- 172.21.15.2:0/2393300010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a38009490 con 0x7f8a44102730 2024-03-15T13:00:55.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a417fa640 1 -- 172.21.15.2:0/2393300010 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a340035f0 con 0x7f8a44102730 2024-03-15T13:00:55.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a417fa640 1 -- 172.21.15.2:0/2393300010 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34010670 con 0x7f8a44102730 2024-03-15T13:00:55.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.365+0000 7f8a4a5a0640 1 -- 172.21.15.2:0/2393300010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a441a46b0 con 0x7f8a44102730 2024-03-15T13:00:55.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.368+0000 7f8a417fa640 1 -- 172.21.15.2:0/2393300010 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f8a34003760 con 0x7f8a44102730 2024-03-15T13:00:55.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.368+0000 7f8a4d177740 1 Processor -- start 2024-03-15T13:00:55.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.368+0000 7f8a4d177740 1 -- start start 2024-03-15T13:00:55.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a4d177740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548250 0x563d77548630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a4d177740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 0x563d77551210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a4d177740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d77553b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a4d177740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d77503960 con 0x563d77551750 2024-03-15T13:00:55.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a4d177740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d774b7310 con 0x563d77548250 2024-03-15T13:00:55.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a4d177740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d774b4840 con 0x563d77548b70 2024-03-15T13:00:55.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 0x563d77551210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 0x563d77551210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:34922/0 (socket says 172.21.15.2:34922) 2024-03-15T13:00:55.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a43fff640 1 -- 172.21.15.2:0/1002836832 learned_addr learned my addr 172.21.15.2:0/1002836832 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a48b3d640 1 --2- 172.21.15.2:0/1002836832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548250 0x563d77548630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a437fe640 1 --2- 172.21.15.2:0/1002836832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d77553b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a417fa640 1 --2- 172.21.15.2:0/2393300010 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8a2c0a1ba0 0x7f8a2c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a43fff640 1 -- 172.21.15.2:0/1002836832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548250 msgr2=0x563d77548630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a43fff640 1 --2- 172.21.15.2:0/1002836832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548250 0x563d77548630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a43fff640 1 -- 172.21.15.2:0/1002836832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 msgr2=0x563d77553b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a43fff640 1 --2- 172.21.15.2:0/1002836832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d77553b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a43fff640 1 -- 172.21.15.2:0/1002836832 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d77472060 con 0x563d77548b70 2024-03-15T13:00:55.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a437fe640 1 --2- 172.21.15.2:0/1002836832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d77553b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:55.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.369+0000 7f8a48b3d640 1 --2- 172.21.15.2:0/2393300010 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8a2c0a1ba0 0x7f8a2c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.370+0000 7f8a417fa640 1 -- 172.21.15.2:0/2393300010 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f8a34014070 con 0x7f8a44102730 2024-03-15T13:00:55.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.370+0000 7f8a43fff640 1 --2- 172.21.15.2:0/1002836832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 0x563d77551210 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8a340cb2d0 tx=0x7f8a34020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f71befa8b9ba9176 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.370+0000 7f8a48b3d640 1 --2- 172.21.15.2:0/2393300010 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8a2c0a1ba0 0x7f8a2c0a4060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c005c40 tx=0x7f8a3c004a40 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.370+0000 7f8a2a7fc640 1 -- 172.21.15.2:0/1002836832 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a340d1310 con 0x563d77548b70 2024-03-15T13:00:55.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.370+0000 7f8a2a7fc640 1 -- 172.21.15.2:0/1002836832 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a3409a610 con 0x563d77548b70 2024-03-15T13:00:55.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.370+0000 7f8a2a7fc640 1 -- 172.21.15.2:0/1002836832 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34065bf0 con 0x563d77548b70 2024-03-15T13:00:55.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.371+0000 7f8a4d177740 1 -- 172.21.15.2:0/1002836832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 msgr2=0x563d77551210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.371+0000 7f8a4d177740 1 --2- 172.21.15.2:0/1002836832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 0x563d77551210 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8a340cb2d0 tx=0x7f8a34020000 comp rx=0 tx=0).stop 2024-03-15T13:00:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.371+0000 7f8a4d177740 1 -- 172.21.15.2:0/1002836832 shutdown_connections 2024-03-15T13:00:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.371+0000 7f8a4d177740 1 --2- 172.21.15.2:0/1002836832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d77553b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.371+0000 7f8a4d177740 1 --2- 172.21.15.2:0/1002836832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 0x563d77551210 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.371+0000 7f8a4d177740 1 --2- 172.21.15.2:0/1002836832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548250 0x563d77548630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.371+0000 7f8a4d177740 1 -- 172.21.15.2:0/1002836832 >> 172.21.15.2:0/1002836832 conn(0x563d77542980 msgr2=0x563d77542d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.371+0000 7f8a4d177740 1 -- 172.21.15.2:0/1002836832 shutdown_connections 2024-03-15T13:00:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.371+0000 7f8a4d177740 1 -- 172.21.15.2:0/1002836832 wait complete. 2024-03-15T13:00:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.371+0000 7f8a4d177740 1 -- start start 2024-03-15T13:00:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.371+0000 7f8a4d177740 1 Processor -- start 2024-03-15T13:00:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a4d177740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77548250 0x563d775510f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a4d177740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 0x563d7754e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a4d177740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77551750 0x563d7754ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a4d177740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d774b0020 con 0x563d77548250 2024-03-15T13:00:55.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a4d177740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d77183430 con 0x563d77551750 2024-03-15T13:00:55.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a4d177740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d7713fe00 con 0x563d77548b70 2024-03-15T13:00:55.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a4d177740 10 client.0 _ll_register_callbacks cb 0x563d77457cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:55.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77548250 0x563d775510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77548250 0x563d775510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32794/0 (socket says 172.21.15.2:32794) 2024-03-15T13:00:55.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a48b3d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77551750 0x563d7754ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a43fff640 1 -- 172.21.15.2:0/88467243 learned_addr learned my addr 172.21.15.2:0/88467243 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a437fe640 1 --2- 172.21.15.2:0/88467243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 0x563d7754e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a43fff640 1 -- 172.21.15.2:0/88467243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 msgr2=0x563d7754e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a43fff640 1 --2- 172.21.15.2:0/88467243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 0x563d7754e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a43fff640 1 -- 172.21.15.2:0/88467243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77551750 msgr2=0x563d7754ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a43fff640 1 --2- 172.21.15.2:0/88467243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77551750 0x563d7754ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a43fff640 1 -- 172.21.15.2:0/88467243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a441a46b0 con 0x563d77548250 2024-03-15T13:00:55.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a43fff640 1 --2- 172.21.15.2:0/88467243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77548250 0x563d775510f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8a340cb410 tx=0x7f8a34003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2203169169a5af7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a237fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:55.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a237fe640 1 -- 172.21.15.2:0/88467243 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a340d1310 con 0x563d77548250 2024-03-15T13:00:55.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a237fe640 1 -- 172.21.15.2:0/88467243 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f8a34003b40 con 0x563d77548250 2024-03-15T13:00:55.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.372+0000 7f8a4d177740 1 -- 172.21.15.2:0/88467243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d771af7e0 con 0x563d77548250 2024-03-15T13:00:55.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.373+0000 7f8a237fe640 1 -- 172.21.15.2:0/88467243 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a3409a380 con 0x563d77548250 2024-03-15T13:00:55.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.373+0000 7f8a4d177740 10 client.16347 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:55.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.373+0000 7f8a4d177740 1 -- 172.21.15.2:0/88467243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563d774ae6a0 con 0x563d77548250 2024-03-15T13:00:55.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.373+0000 7f8a237fe640 1 -- 172.21.15.2:0/88467243 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x563d771af7e0 con 0x563d77548250 2024-03-15T13:00:55.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.373+0000 7f8a237fe640 1 -- 172.21.15.2:0/88467243 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f8a3401f060 con 0x563d77548250 2024-03-15T13:00:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.373+0000 7f8a4d177740 10 client.16347 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.373+0000 7f8a4d177740 10 client.16347 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:00:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.373+0000 7f8a4d177740 20 client.16347 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.373+0000 7f8a4d177740 1 --2- 172.21.15.2:0/88467243 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563d77561470 0x563d77563860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.373+0000 7f8a4d177740 4 client.16347 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.373+0000 7f8a4d177740 1 -- 172.21.15.2:0/88467243 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d771ec720 con 0x563d77561470 2024-03-15T13:00:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.373+0000 7f8a437fe640 1 --2- 172.21.15.2:0/88467243 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563d77561470 0x563d77563860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.374+0000 7f8a237fe640 10 client.16347 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:00:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.374+0000 7f8a437fe640 1 --2- 172.21.15.2:0/88467243 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563d77561470 0x563d77563860 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd19ee112e5538d8 server_cookie=fcf9b67a68934489 in_seq=0 out_seq=0 2024-03-15T13:00:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.375+0000 7f8a237fe640 1 -- 172.21.15.2:0/88467243 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563d771ec720 con 0x563d77561470 2024-03-15T13:00:55.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.375+0000 7f8a237fe640 10 client.16347 handle_command_reply: tid=0 2024-03-15T13:00:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.375+0000 7f8a4d177740 1 client.16347 shutdown 2024-03-15T13:00:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.375+0000 7f8a4d177740 1 -- 172.21.15.2:0/88467243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77548250 msgr2=0x563d775510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.375+0000 7f8a4d177740 1 --2- 172.21.15.2:0/88467243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77548250 0x563d775510f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8a340cb410 tx=0x7f8a34003e40 comp rx=0 tx=0).stop 2024-03-15T13:00:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.375+0000 7f8a4d177740 1 -- 172.21.15.2:0/88467243 shutdown_connections 2024-03-15T13:00:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.375+0000 7f8a4d177740 1 --2- 172.21.15.2:0/88467243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77551750 0x563d7754ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.375+0000 7f8a4d177740 1 --2- 172.21.15.2:0/88467243 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563d77561470 0x563d77563860 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.375+0000 7f8a4d177740 1 --2- 172.21.15.2:0/88467243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 0x563d7754e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.375+0000 7f8a4d177740 1 --2- 172.21.15.2:0/88467243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77548250 0x563d775510f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.375+0000 7f8a4d177740 1 -- 172.21.15.2:0/88467243 >> 172.21.15.2:0/88467243 conn(0x563d77542980 msgr2=0x563d775446d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.375+0000 7f8a237fe640 0 client.16347 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:00:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.375+0000 7f8a4d177740 1 -- 172.21.15.2:0/88467243 shutdown_connections 2024-03-15T13:00:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.375+0000 7f8a4d177740 1 -- 172.21.15.2:0/88467243 wait complete. 2024-03-15T13:00:55.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.375+0000 7f8a4d177740 20 client.16347 trim_cache size 0 max 16384 2024-03-15T13:00:55.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a4d177740 1 Processor -- start 2024-03-15T13:00:55.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a4d177740 1 -- start start 2024-03-15T13:00:55.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a4d177740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548250 0x563d77541a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a4d177740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 0x563d77550bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a4d177740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d775510f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a4d177740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d77435130 con 0x563d77551750 2024-03-15T13:00:55.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a4d177740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d77534a40 con 0x563d77548250 2024-03-15T13:00:55.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a4d177740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d7754f8e0 con 0x563d77548b70 2024-03-15T13:00:55.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a48b3d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d775510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a43fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548250 0x563d77541a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a437fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 0x563d77550bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a48b3d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d775510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32808/0 (socket says 172.21.15.2:32808) 2024-03-15T13:00:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a48b3d640 1 -- 172.21.15.2:0/2827466021 learned_addr learned my addr 172.21.15.2:0/2827466021 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a43fff640 1 -- 172.21.15.2:0/2827466021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 msgr2=0x563d77550bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a43fff640 1 --2- 172.21.15.2:0/2827466021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 0x563d77550bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a43fff640 1 -- 172.21.15.2:0/2827466021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 msgr2=0x563d775510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a43fff640 1 --2- 172.21.15.2:0/2827466021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d775510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a43fff640 1 -- 172.21.15.2:0/2827466021 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d7744ac20 con 0x563d77548250 2024-03-15T13:00:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.388+0000 7f8a43fff640 1 --2- 172.21.15.2:0/2827466021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548250 0x563d77541a00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a340cb960 tx=0x7f8a34098d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee53ba09bc6a69d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.389+0000 7f8a29ffb640 1 -- 172.21.15.2:0/2827466021 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a340d1310 con 0x563d77548250 2024-03-15T13:00:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.389+0000 7f8a29ffb640 1 -- 172.21.15.2:0/2827466021 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8a34096790 con 0x563d77548250 2024-03-15T13:00:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.389+0000 7f8a29ffb640 1 -- 172.21.15.2:0/2827466021 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a340b3380 con 0x563d77548250 2024-03-15T13:00:55.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.389+0000 7f8a4d177740 1 -- 172.21.15.2:0/2827466021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548250 msgr2=0x563d77541a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.389+0000 7f8a4d177740 1 --2- 172.21.15.2:0/2827466021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548250 0x563d77541a00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a340cb960 tx=0x7f8a34098d50 comp rx=0 tx=0).stop 2024-03-15T13:00:55.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.389+0000 7f8a4d177740 1 -- 172.21.15.2:0/2827466021 shutdown_connections 2024-03-15T13:00:55.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.389+0000 7f8a4d177740 1 --2- 172.21.15.2:0/2827466021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d775510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.389+0000 7f8a4d177740 1 --2- 172.21.15.2:0/2827466021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548b70 0x563d77550bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.389+0000 7f8a4d177740 1 --2- 172.21.15.2:0/2827466021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548250 0x563d77541a00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.389+0000 7f8a4d177740 1 -- 172.21.15.2:0/2827466021 >> 172.21.15.2:0/2827466021 conn(0x563d77542980 msgr2=0x563d775412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.389+0000 7f8a4d177740 1 -- 172.21.15.2:0/2827466021 shutdown_connections 2024-03-15T13:00:55.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.389+0000 7f8a4d177740 1 -- 172.21.15.2:0/2827466021 wait complete. 2024-03-15T13:00:55.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.390+0000 7f8a4d177740 1 -- start start 2024-03-15T13:00:55.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.390+0000 7f8a4d177740 1 Processor -- start 2024-03-15T13:00:55.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.390+0000 7f8a4d177740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548250 0x563d77559320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.390+0000 7f8a4d177740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548b70 0x563d77554690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.390+0000 7f8a4d177740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d77554bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.390+0000 7f8a4d177740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d775552d0 con 0x563d77551750 2024-03-15T13:00:55.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.390+0000 7f8a4d177740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d77555440 con 0x563d77548b70 2024-03-15T13:00:55.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.390+0000 7f8a4d177740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d775bbcc0 con 0x563d77548250 2024-03-15T13:00:55.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.390+0000 7f8a4d177740 10 client.0 _ll_register_callbacks cb 0x7f8a38037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:55.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.390+0000 7f8a48b3d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d77554bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.390+0000 7f8a48b3d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d77554bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32820/0 (socket says 172.21.15.2:32820) 2024-03-15T13:00:55.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.390+0000 7f8a48b3d640 1 -- 172.21.15.2:0/2704931010 learned_addr learned my addr 172.21.15.2:0/2704931010 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.390+0000 7f8a43fff640 1 --2- 172.21.15.2:0/2704931010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548250 0x563d77559320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.390+0000 7f8a437fe640 1 --2- 172.21.15.2:0/2704931010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548b70 0x563d77554690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a48b3d640 1 -- 172.21.15.2:0/2704931010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548250 msgr2=0x563d77559320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a48b3d640 1 --2- 172.21.15.2:0/2704931010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548250 0x563d77559320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a48b3d640 1 -- 172.21.15.2:0/2704931010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548b70 msgr2=0x563d77554690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a48b3d640 1 --2- 172.21.15.2:0/2704931010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548b70 0x563d77554690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a48b3d640 1 -- 172.21.15.2:0/2704931010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d774ae6a0 con 0x563d77551750 2024-03-15T13:00:55.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a43fff640 1 --2- 172.21.15.2:0/2704931010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548250 0x563d77559320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:55.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a48b3d640 1 --2- 172.21.15.2:0/2704931010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d77554bd0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00fbe0 tx=0x7f8a3c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7150d5021e5f550e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a237fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:55.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a237fe640 1 -- 172.21.15.2:0/2704931010 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c022040 con 0x563d77551750 2024-03-15T13:00:55.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a237fe640 1 -- 172.21.15.2:0/2704931010 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f8a3c00fe50 con 0x563d77551750 2024-03-15T13:00:55.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a4d177740 1 -- 172.21.15.2:0/2704931010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d7723b890 con 0x563d77551750 2024-03-15T13:00:55.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a237fe640 1 -- 172.21.15.2:0/2704931010 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a3c003b70 con 0x563d77551750 2024-03-15T13:00:55.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a4d177740 10 client.16353 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:55.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a4d177740 1 -- 172.21.15.2:0/2704931010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563d774708c0 con 0x563d77551750 2024-03-15T13:00:55.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a237fe640 1 -- 172.21.15.2:0/2704931010 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x563d7723b890 con 0x563d77551750 2024-03-15T13:00:55.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a237fe640 1 -- 172.21.15.2:0/2704931010 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f8a3c0043a0 con 0x563d77551750 2024-03-15T13:00:55.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a4d177740 10 client.16353 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:55.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a4d177740 10 client.16353 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:00:55.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a4d177740 20 client.16353 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:55.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.391+0000 7f8a4d177740 1 --2- 172.21.15.2:0/2704931010 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563d775bd4f0 0x563d775bf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.392+0000 7f8a4d177740 4 client.16353 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:00:55.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.392+0000 7f8a4d177740 1 -- 172.21.15.2:0/2704931010 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563d7715b700 con 0x563d775bd4f0 2024-03-15T13:00:55.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.392+0000 7f8a43fff640 1 --2- 172.21.15.2:0/2704931010 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563d775bd4f0 0x563d775bf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.392+0000 7f8a237fe640 10 client.16353 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:00:55.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.392+0000 7f8a43fff640 1 --2- 172.21.15.2:0/2704931010 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563d775bd4f0 0x563d775bf910 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=43d2d7487f1da0a0 server_cookie=eaeffdf9b859d9be in_seq=0 out_seq=0 2024-03-15T13:00:55.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.392+0000 7f8a237fe640 1 -- 172.21.15.2:0/2704931010 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563d7715b700 con 0x563d775bd4f0 2024-03-15T13:00:55.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.392+0000 7f8a237fe640 10 client.16353 handle_command_reply: tid=0 2024-03-15T13:00:55.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.392+0000 7f8a4d177740 1 client.16353 shutdown 2024-03-15T13:00:55.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.392+0000 7f8a4d177740 1 -- 172.21.15.2:0/2704931010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 msgr2=0x563d77554bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.392+0000 7f8a4d177740 1 --2- 172.21.15.2:0/2704931010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d77554bd0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00fbe0 tx=0x7f8a3c004ab0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4d177740 1 -- 172.21.15.2:0/2704931010 shutdown_connections 2024-03-15T13:00:55.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4d177740 1 --2- 172.21.15.2:0/2704931010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d77551750 0x563d77554bd0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4d177740 1 --2- 172.21.15.2:0/2704931010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d77548b70 0x563d77554690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4d177740 1 --2- 172.21.15.2:0/2704931010 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563d775bd4f0 0x563d775bf910 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4d177740 1 --2- 172.21.15.2:0/2704931010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563d77548250 0x563d77559320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a237fe640 0 client.16353 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:00:55.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4d177740 1 -- 172.21.15.2:0/2704931010 >> 172.21.15.2:0/2704931010 conn(0x563d77542980 msgr2=0x563d77541b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4d177740 1 -- 172.21.15.2:0/2704931010 shutdown_connections 2024-03-15T13:00:55.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4d177740 1 -- 172.21.15.2:0/2704931010 wait complete. 2024-03-15T13:00:55.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4d177740 20 client.16353 trim_cache size 0 max 16384 2024-03-15T13:00:55.387 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:00:55.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4a5a0640 1 -- 172.21.15.2:0/2393300010 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8a2c0a1ba0 msgr2=0x7f8a2c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4a5a0640 1 --2- 172.21.15.2:0/2393300010 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8a2c0a1ba0 0x7f8a2c0a4060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c005c40 tx=0x7f8a3c004a40 comp rx=0 tx=0).stop 2024-03-15T13:00:55.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4a5a0640 1 -- 172.21.15.2:0/2393300010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a44102730 msgr2=0x7f8a4419d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4a5a0640 1 --2- 172.21.15.2:0/2393300010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a44102730 0x7f8a4419d7f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400e9c0 tx=0x7f8a3400ee80 comp rx=0 tx=0).stop 2024-03-15T13:00:55.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4a5a0640 1 -- 172.21.15.2:0/2393300010 shutdown_connections 2024-03-15T13:00:55.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4a5a0640 1 --2- 172.21.15.2:0/2393300010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a4410bb00 0x7f8a441979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4a5a0640 1 --2- 172.21.15.2:0/2393300010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a44103050 0x7f8a4419dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4a5a0640 1 --2- 172.21.15.2:0/2393300010 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8a2c0a1ba0 0x7f8a2c0a4060 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4a5a0640 1 --2- 172.21.15.2:0/2393300010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a44102730 0x7f8a4419d7f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4a5a0640 1 -- 172.21.15.2:0/2393300010 >> 172.21.15.2:0/2393300010 conn(0x7f8a44076770 msgr2=0x7f8a440fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4a5a0640 1 -- 172.21.15.2:0/2393300010 shutdown_connections 2024-03-15T13:00:55.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.393+0000 7f8a4a5a0640 1 -- 172.21.15.2:0/2393300010 wait complete. 2024-03-15T13:00:55.390 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:00:55.390 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:00:55.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.526+0000 7fcd944a7640 1 -- 172.21.15.2:0/3058236816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c100f80 msgr2=0x7fcd8c104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.526+0000 7fcd944a7640 1 --2- 172.21.15.2:0/3058236816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c100f80 0x7fcd8c104fc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009930 tx=0x7fcd7c02bc30 comp rx=0 tx=0).stop 2024-03-15T13:00:55.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.526+0000 7fcd944a7640 1 -- 172.21.15.2:0/3058236816 shutdown_connections 2024-03-15T13:00:55.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.526+0000 7fcd944a7640 1 --2- 172.21.15.2:0/3058236816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd8c1056a0 0x7fcd8c10d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.526+0000 7fcd944a7640 1 --2- 172.21.15.2:0/3058236816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c100f80 0x7fcd8c104fc0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.526+0000 7fcd944a7640 1 --2- 172.21.15.2:0/3058236816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd8c1005d0 0x7fcd8c1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.526+0000 7fcd944a7640 1 -- 172.21.15.2:0/3058236816 >> 172.21.15.2:0/3058236816 conn(0x7fcd8c0774f0 msgr2=0x7fcd8c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.526+0000 7fcd944a7640 1 -- 172.21.15.2:0/3058236816 shutdown_connections 2024-03-15T13:00:55.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.526+0000 7fcd944a7640 1 -- 172.21.15.2:0/3058236816 wait complete. 2024-03-15T13:00:55.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.527+0000 7fcd944a7640 1 Processor -- start 2024-03-15T13:00:55.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.527+0000 7fcd944a7640 1 -- start start 2024-03-15T13:00:55.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.527+0000 7fcd944a7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c1005d0 0x7fcd8c19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.527+0000 7fcd944a7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd8c1056a0 0x7fcd8c19dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.527+0000 7fcd944a7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd8c197950 0x7fcd8c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.527+0000 7fcd944a7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c198340 con 0x7fcd8c1005d0 2024-03-15T13:00:55.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd944a7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c1984b0 con 0x7fcd8c197950 2024-03-15T13:00:55.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd944a7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c198670 con 0x7fcd8c1056a0 2024-03-15T13:00:55.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd92243640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c1005d0 0x7fcd8c19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd92243640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c1005d0 0x7fcd8c19d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32838/0 (socket says 172.21.15.2:32838) 2024-03-15T13:00:55.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd91a42640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd8c1056a0 0x7fcd8c19dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd92243640 1 -- 172.21.15.2:0/314261323 learned_addr learned my addr 172.21.15.2:0/314261323 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd92a44640 1 --2- 172.21.15.2:0/314261323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd8c197950 0x7fcd8c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd92243640 1 -- 172.21.15.2:0/314261323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd8c1056a0 msgr2=0x7fcd8c19dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd92243640 1 --2- 172.21.15.2:0/314261323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd8c1056a0 0x7fcd8c19dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd92243640 1 -- 172.21.15.2:0/314261323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd8c197950 msgr2=0x7fcd8c197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd92243640 1 --2- 172.21.15.2:0/314261323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd8c197950 0x7fcd8c197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd92243640 1 -- 172.21.15.2:0/314261323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c009560 con 0x7fcd8c1005d0 2024-03-15T13:00:55.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd92243640 1 --2- 172.21.15.2:0/314261323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c1005d0 0x7fcd8c19d7a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800d950 tx=0x7fcd8800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd837fe640 1 -- 172.21.15.2:0/314261323 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88009a30 con 0x7fcd8c1005d0 2024-03-15T13:00:55.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd944a7640 1 -- 172.21.15.2:0/314261323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c101f90 con 0x7fcd8c1005d0 2024-03-15T13:00:55.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd837fe640 1 -- 172.21.15.2:0/314261323 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcd88009b90 con 0x7fcd8c1005d0 2024-03-15T13:00:55.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd837fe640 1 -- 172.21.15.2:0/314261323 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd880047a0 con 0x7fcd8c1005d0 2024-03-15T13:00:55.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.528+0000 7fcd944a7640 1 -- 172.21.15.2:0/314261323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c1024c0 con 0x7fcd8c1005d0 2024-03-15T13:00:55.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.531+0000 7fcd837fe640 1 -- 172.21.15.2:0/314261323 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fcd88009d00 con 0x7fcd8c1005d0 2024-03-15T13:00:55.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.531+0000 7fcd97164740 1 Processor -- start 2024-03-15T13:00:55.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.532+0000 7fcd97164740 1 -- start start 2024-03-15T13:00:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.532+0000 7fcd97164740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55994233a250 0x55994233a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.532+0000 7fcd97164740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233ab70 0x559942343210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.532+0000 7fcd97164740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942343750 0x559942345b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.532+0000 7fcd97164740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599422f5960 con 0x55994233ab70 2024-03-15T13:00:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.532+0000 7fcd97164740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599422a9310 con 0x55994233a250 2024-03-15T13:00:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.532+0000 7fcd97164740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599422a6840 con 0x559942343750 2024-03-15T13:00:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.532+0000 7fcd92a44640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55994233a250 0x55994233a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.532+0000 7fcd92a44640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55994233a250 0x55994233a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:50962/0 (socket says 172.21.15.2:50962) 2024-03-15T13:00:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.532+0000 7fcd92a44640 1 -- 172.21.15.2:0/4044735737 learned_addr learned my addr 172.21.15.2:0/4044735737 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.532+0000 7fcd92243640 1 --2- 172.21.15.2:0/4044735737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233ab70 0x559942343210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.532+0000 7fcd91a42640 1 --2- 172.21.15.2:0/4044735737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942343750 0x559942345b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.533+0000 7fcd91a42640 1 -- 172.21.15.2:0/4044735737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55994233a250 msgr2=0x55994233a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.533+0000 7fcd91a42640 1 --2- 172.21.15.2:0/4044735737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55994233a250 0x55994233a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.533+0000 7fcd837fe640 1 --2- 172.21.15.2:0/314261323 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcd780a1ba0 0x7fcd780a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.533+0000 7fcd91a42640 1 -- 172.21.15.2:0/4044735737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233ab70 msgr2=0x559942343210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.533+0000 7fcd91a42640 1 --2- 172.21.15.2:0/4044735737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233ab70 0x559942343210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.533+0000 7fcd91a42640 1 -- 172.21.15.2:0/4044735737 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559942264060 con 0x559942343750 2024-03-15T13:00:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.533+0000 7fcd837fe640 1 -- 172.21.15.2:0/314261323 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fcd880cd510 con 0x7fcd8c1005d0 2024-03-15T13:00:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.533+0000 7fcd92a44640 1 --2- 172.21.15.2:0/314261323 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcd780a1ba0 0x7fcd780a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.533+0000 7fcd91a42640 1 --2- 172.21.15.2:0/4044735737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942343750 0x559942345b40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c036040 tx=0x7fcd7c005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aead2f301d37f43d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.533+0000 7fcd80bfe640 1 -- 172.21.15.2:0/4044735737 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c004800 con 0x559942343750 2024-03-15T13:00:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.533+0000 7fcd80bfe640 1 -- 172.21.15.2:0/4044735737 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcd7c004960 con 0x559942343750 2024-03-15T13:00:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.533+0000 7fcd80bfe640 1 -- 172.21.15.2:0/4044735737 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c003830 con 0x559942343750 2024-03-15T13:00:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.534+0000 7fcd97164740 1 -- 172.21.15.2:0/4044735737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942343750 msgr2=0x559942345b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.534+0000 7fcd92a44640 1 --2- 172.21.15.2:0/314261323 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcd780a1ba0 0x7fcd780a4060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009fd0 tx=0x7fcd84009500 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.534+0000 7fcd97164740 1 --2- 172.21.15.2:0/4044735737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942343750 0x559942345b40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c036040 tx=0x7fcd7c005c00 comp rx=0 tx=0).stop 2024-03-15T13:00:55.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.534+0000 7fcd97164740 1 -- 172.21.15.2:0/4044735737 shutdown_connections 2024-03-15T13:00:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.534+0000 7fcd97164740 1 --2- 172.21.15.2:0/4044735737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942343750 0x559942345b40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.534+0000 7fcd97164740 1 --2- 172.21.15.2:0/4044735737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233ab70 0x559942343210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.534+0000 7fcd97164740 1 --2- 172.21.15.2:0/4044735737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55994233a250 0x55994233a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.534+0000 7fcd97164740 1 -- 172.21.15.2:0/4044735737 >> 172.21.15.2:0/4044735737 conn(0x559942334980 msgr2=0x559942334d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.534+0000 7fcd97164740 1 -- 172.21.15.2:0/4044735737 shutdown_connections 2024-03-15T13:00:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.534+0000 7fcd97164740 1 -- 172.21.15.2:0/4044735737 wait complete. 2024-03-15T13:00:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.534+0000 7fcd97164740 1 -- start start 2024-03-15T13:00:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.534+0000 7fcd97164740 1 Processor -- start 2024-03-15T13:00:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd97164740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55994233a250 0x5599423430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd97164740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233ab70 0x559942340430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd97164740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559942343750 0x559942340990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd97164740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599422a2020 con 0x55994233ab70 2024-03-15T13:00:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd97164740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559941f75430 con 0x559942343750 2024-03-15T13:00:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd97164740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559941f31e00 con 0x55994233a250 2024-03-15T13:00:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd97164740 10 client.0 _ll_register_callbacks cb 0x559942249cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd91a42640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233ab70 0x559942340430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd91a42640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233ab70 0x559942340430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32870/0 (socket says 172.21.15.2:32870) 2024-03-15T13:00:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd92243640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55994233a250 0x5599423430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd91a42640 1 -- 172.21.15.2:0/1513562798 learned_addr learned my addr 172.21.15.2:0/1513562798 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd92a44640 1 --2- 172.21.15.2:0/1513562798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559942343750 0x559942340990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd91a42640 1 -- 172.21.15.2:0/1513562798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55994233a250 msgr2=0x5599423430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd91a42640 1 --2- 172.21.15.2:0/1513562798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55994233a250 0x5599423430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd91a42640 1 -- 172.21.15.2:0/1513562798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559942343750 msgr2=0x559942340990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd91a42640 1 --2- 172.21.15.2:0/1513562798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559942343750 0x559942340990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd91a42640 1 -- 172.21.15.2:0/1513562798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c10da80 con 0x55994233ab70 2024-03-15T13:00:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd92243640 1 --2- 172.21.15.2:0/1513562798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55994233a250 0x5599423430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd91a42640 1 --2- 172.21.15.2:0/1513562798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233ab70 0x559942340430 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c005090 tx=0x7fcd7c0050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68d849f2fcb0bf server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.535+0000 7fcd75ffb640 1 -- 172.21.15.2:0/1513562798 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c004800 con 0x55994233ab70 2024-03-15T13:00:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.536+0000 7fcd75ffb640 1 -- 172.21.15.2:0/1513562798 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fcd7c04d070 con 0x55994233ab70 2024-03-15T13:00:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.536+0000 7fcd97164740 1 -- 172.21.15.2:0/1513562798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559941fa17e0 con 0x55994233ab70 2024-03-15T13:00:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.536+0000 7fcd75ffb640 1 -- 172.21.15.2:0/1513562798 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd7c02cc40 con 0x55994233ab70 2024-03-15T13:00:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.536+0000 7fcd97164740 10 client.16365 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.536+0000 7fcd97164740 1 -- 172.21.15.2:0/1513562798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5599422a06a0 con 0x55994233ab70 2024-03-15T13:00:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.536+0000 7fcd75ffb640 1 -- 172.21.15.2:0/1513562798 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x559941fa17e0 con 0x55994233ab70 2024-03-15T13:00:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.536+0000 7fcd75ffb640 1 -- 172.21.15.2:0/1513562798 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fcd7c03c050 con 0x55994233ab70 2024-03-15T13:00:55.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.536+0000 7fcd97164740 10 client.16365 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.536+0000 7fcd97164740 10 client.16365 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:00:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.536+0000 7fcd97164740 20 client.16365 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.536+0000 7fcd97164740 1 --2- 172.21.15.2:0/1513562798 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5599423533e0 0x559942355800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.536+0000 7fcd97164740 4 client.16365 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.536+0000 7fcd97164740 1 -- 172.21.15.2:0/1513562798 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559941fde720 con 0x5599423533e0 2024-03-15T13:00:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.536+0000 7fcd92243640 1 --2- 172.21.15.2:0/1513562798 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5599423533e0 0x559942355800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.537+0000 7fcd92243640 1 --2- 172.21.15.2:0/1513562798 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5599423533e0 0x559942355800 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb5895b45c3a6738 server_cookie=df80a1056d97d1d6 in_seq=0 out_seq=0 2024-03-15T13:00:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.537+0000 7fcd75ffb640 10 client.16365 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:00:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.538+0000 7fcd75ffb640 1 -- 172.21.15.2:0/1513562798 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559941fde720 con 0x5599423533e0 2024-03-15T13:00:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.538+0000 7fcd75ffb640 10 client.16365 handle_command_reply: tid=0 2024-03-15T13:00:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.538+0000 7fcd97164740 1 client.16365 shutdown 2024-03-15T13:00:55.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.538+0000 7fcd97164740 1 -- 172.21.15.2:0/1513562798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233ab70 msgr2=0x559942340430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.538+0000 7fcd97164740 1 --2- 172.21.15.2:0/1513562798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233ab70 0x559942340430 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c005090 tx=0x7fcd7c0050c0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.538+0000 7fcd97164740 1 -- 172.21.15.2:0/1513562798 shutdown_connections 2024-03-15T13:00:55.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.538+0000 7fcd97164740 1 --2- 172.21.15.2:0/1513562798 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5599423533e0 0x559942355800 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.538+0000 7fcd97164740 1 --2- 172.21.15.2:0/1513562798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559942343750 0x559942340990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.538+0000 7fcd97164740 1 --2- 172.21.15.2:0/1513562798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233ab70 0x559942340430 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.538+0000 7fcd97164740 1 --2- 172.21.15.2:0/1513562798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55994233a250 0x5599423430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.538+0000 7fcd97164740 1 -- 172.21.15.2:0/1513562798 >> 172.21.15.2:0/1513562798 conn(0x559942334980 msgr2=0x5599423366d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.538+0000 7fcd75ffb640 0 client.16365 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:00:55.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.538+0000 7fcd97164740 1 -- 172.21.15.2:0/1513562798 shutdown_connections 2024-03-15T13:00:55.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.538+0000 7fcd97164740 1 -- 172.21.15.2:0/1513562798 wait complete. 2024-03-15T13:00:55.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.538+0000 7fcd97164740 20 client.16365 trim_cache size 0 max 16384 2024-03-15T13:00:55.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.553+0000 7fcd97164740 1 Processor -- start 2024-03-15T13:00:55.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.553+0000 7fcd97164740 1 -- start start 2024-03-15T13:00:55.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.553+0000 7fcd97164740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233a250 0x55994233ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.553+0000 7fcd97164740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559942343750 0x559942342b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.553+0000 7fcd97164740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942336990 0x5599423430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.553+0000 7fcd97164740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559942227130 con 0x55994233a250 2024-03-15T13:00:55.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.553+0000 7fcd97164740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559941fde720 con 0x559942343750 2024-03-15T13:00:55.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.553+0000 7fcd97164740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559942339340 con 0x559942336990 2024-03-15T13:00:55.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.553+0000 7fcd92243640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233a250 0x55994233ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.553+0000 7fcd92243640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233a250 0x55994233ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32886/0 (socket says 172.21.15.2:32886) 2024-03-15T13:00:55.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.553+0000 7fcd92243640 1 -- 172.21.15.2:0/287752822 learned_addr learned my addr 172.21.15.2:0/287752822 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.553+0000 7fcd91a42640 1 --2- 172.21.15.2:0/287752822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559942343750 0x559942342b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.553+0000 7fcd92a44640 1 --2- 172.21.15.2:0/287752822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942336990 0x5599423430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.554+0000 7fcd92243640 1 -- 172.21.15.2:0/287752822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942336990 msgr2=0x5599423430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.554+0000 7fcd92243640 1 --2- 172.21.15.2:0/287752822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942336990 0x5599423430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.554+0000 7fcd92243640 1 -- 172.21.15.2:0/287752822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559942343750 msgr2=0x559942342b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.554+0000 7fcd92243640 1 --2- 172.21.15.2:0/287752822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559942343750 0x559942342b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.554+0000 7fcd92243640 1 -- 172.21.15.2:0/287752822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55994223cc20 con 0x55994233a250 2024-03-15T13:00:55.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.554+0000 7fcd92243640 1 --2- 172.21.15.2:0/287752822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233a250 0x55994233ed00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fcd88099af0 tx=0x7fcd880d30e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a995875a89a5e3c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.554+0000 7fcd77fff640 1 -- 172.21.15.2:0/287752822 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88099930 con 0x55994233a250 2024-03-15T13:00:55.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.554+0000 7fcd77fff640 1 -- 172.21.15.2:0/287752822 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcd88099750 con 0x55994233a250 2024-03-15T13:00:55.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.554+0000 7fcd77fff640 1 -- 172.21.15.2:0/287752822 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88098180 con 0x55994233a250 2024-03-15T13:00:55.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.556+0000 7fcd97164740 1 -- 172.21.15.2:0/287752822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233a250 msgr2=0x55994233ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.556+0000 7fcd97164740 1 --2- 172.21.15.2:0/287752822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233a250 0x55994233ed00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fcd88099af0 tx=0x7fcd880d30e0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.556+0000 7fcd97164740 1 -- 172.21.15.2:0/287752822 shutdown_connections 2024-03-15T13:00:55.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.556+0000 7fcd97164740 1 --2- 172.21.15.2:0/287752822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942336990 0x5599423430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.556+0000 7fcd97164740 1 --2- 172.21.15.2:0/287752822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559942343750 0x559942342b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.556+0000 7fcd97164740 1 --2- 172.21.15.2:0/287752822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233a250 0x55994233ed00 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.556+0000 7fcd97164740 1 -- 172.21.15.2:0/287752822 >> 172.21.15.2:0/287752822 conn(0x559942334980 msgr2=0x559942333b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.556+0000 7fcd97164740 1 -- 172.21.15.2:0/287752822 shutdown_connections 2024-03-15T13:00:55.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.556+0000 7fcd97164740 1 -- 172.21.15.2:0/287752822 wait complete. 2024-03-15T13:00:55.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.556+0000 7fcd97164740 1 -- start start 2024-03-15T13:00:55.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.556+0000 7fcd97164740 1 Processor -- start 2024-03-15T13:00:55.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd97164740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559942336990 0x559942348580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd97164740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233a250 0x559942348ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd97164740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942343750 0x559942349000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd97164740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599423b74d0 con 0x55994233a250 2024-03-15T13:00:55.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd97164740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599423b7640 con 0x559942336990 2024-03-15T13:00:55.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd97164740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599423b7800 con 0x559942343750 2024-03-15T13:00:55.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd97164740 10 client.0 _ll_register_callbacks cb 0x7fcd880d0ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:55.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd91a42640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233a250 0x559942348ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd92243640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559942336990 0x559942348580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd91a42640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233a250 0x559942348ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32896/0 (socket says 172.21.15.2:32896) 2024-03-15T13:00:55.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd91a42640 1 -- 172.21.15.2:0/576877260 learned_addr learned my addr 172.21.15.2:0/576877260 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd92a44640 1 --2- 172.21.15.2:0/576877260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942343750 0x559942349000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd91a42640 1 -- 172.21.15.2:0/576877260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942343750 msgr2=0x559942349000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd91a42640 1 --2- 172.21.15.2:0/576877260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942343750 0x559942349000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd91a42640 1 -- 172.21.15.2:0/576877260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559942336990 msgr2=0x559942348580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd91a42640 1 --2- 172.21.15.2:0/576877260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559942336990 0x559942348580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.557+0000 7fcd91a42640 1 -- 172.21.15.2:0/576877260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c1024c0 con 0x55994233a250 2024-03-15T13:00:55.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd91a42640 1 --2- 172.21.15.2:0/576877260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233a250 0x559942348ac0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c03ff60 tx=0x7fcd7c02c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f3cea4a73997d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:55.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd75ffb640 1 -- 172.21.15.2:0/576877260 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c02cbd0 con 0x55994233a250 2024-03-15T13:00:55.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd97164740 1 -- 172.21.15.2:0/576877260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599423300f0 con 0x55994233a250 2024-03-15T13:00:55.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd75ffb640 1 -- 172.21.15.2:0/576877260 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fcd7c000e40 con 0x55994233a250 2024-03-15T13:00:55.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd75ffb640 1 -- 172.21.15.2:0/576877260 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd7c0056f0 con 0x55994233a250 2024-03-15T13:00:55.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd97164740 10 client.16371 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:55.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd97164740 1 -- 172.21.15.2:0/576877260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5599422628c0 con 0x55994233a250 2024-03-15T13:00:55.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd75ffb640 1 -- 172.21.15.2:0/576877260 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcd7c064080 con 0x55994233a250 2024-03-15T13:00:55.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd75ffb640 1 -- 172.21.15.2:0/576877260 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fcd7c03c040 con 0x55994233a250 2024-03-15T13:00:55.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd97164740 10 client.16371 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:55.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd97164740 10 client.16371 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:00:55.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd97164740 20 client.16371 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:55.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd97164740 1 --2- 172.21.15.2:0/576877260 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5599423b8bf0 0x5599423bb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd97164740 4 client.16371 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:00:55.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.558+0000 7fcd97164740 1 -- 172.21.15.2:0/576877260 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fcd7c004960 con 0x5599423b8bf0 2024-03-15T13:00:55.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd92243640 1 --2- 172.21.15.2:0/576877260 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5599423b8bf0 0x5599423bb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd92243640 1 --2- 172.21.15.2:0/576877260 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5599423b8bf0 0x5599423bb010 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbd768aec6d94194 server_cookie=d79e405eceb2a61f in_seq=0 out_seq=0 2024-03-15T13:00:55.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd75ffb640 10 client.16371 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:00:55.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd75ffb640 1 -- 172.21.15.2:0/576877260 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fcd7c004960 con 0x5599423b8bf0 2024-03-15T13:00:55.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd75ffb640 10 client.16371 handle_command_reply: tid=0 2024-03-15T13:00:55.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd97164740 1 client.16371 shutdown 2024-03-15T13:00:55.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd97164740 1 -- 172.21.15.2:0/576877260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233a250 msgr2=0x559942348ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd97164740 1 --2- 172.21.15.2:0/576877260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233a250 0x559942348ac0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c03ff60 tx=0x7fcd7c02c3e0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd97164740 1 -- 172.21.15.2:0/576877260 shutdown_connections 2024-03-15T13:00:55.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd97164740 1 --2- 172.21.15.2:0/576877260 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5599423b8bf0 0x5599423bb010 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd97164740 1 --2- 172.21.15.2:0/576877260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559942343750 0x559942349000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd97164740 1 --2- 172.21.15.2:0/576877260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55994233a250 0x559942348ac0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd97164740 1 --2- 172.21.15.2:0/576877260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559942336990 0x559942348580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd75ffb640 0 client.16371 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:00:55.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd97164740 1 -- 172.21.15.2:0/576877260 >> 172.21.15.2:0/576877260 conn(0x559942334980 msgr2=0x559942333b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd97164740 1 -- 172.21.15.2:0/576877260 shutdown_connections 2024-03-15T13:00:55.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.559+0000 7fcd97164740 1 -- 172.21.15.2:0/576877260 wait complete. 2024-03-15T13:00:55.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.560+0000 7fcd97164740 20 client.16371 trim_cache size 0 max 16384 2024-03-15T13:00:55.551 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:00:55.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.560+0000 7fcd944a7640 1 -- 172.21.15.2:0/314261323 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcd780a1ba0 msgr2=0x7fcd780a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.560+0000 7fcd944a7640 1 --2- 172.21.15.2:0/314261323 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcd780a1ba0 0x7fcd780a4060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009fd0 tx=0x7fcd84009500 comp rx=0 tx=0).stop 2024-03-15T13:00:55.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.560+0000 7fcd944a7640 1 -- 172.21.15.2:0/314261323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c1005d0 msgr2=0x7fcd8c19d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.560+0000 7fcd944a7640 1 --2- 172.21.15.2:0/314261323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c1005d0 0x7fcd8c19d7a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800d950 tx=0x7fcd8800de10 comp rx=0 tx=0).stop 2024-03-15T13:00:55.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.560+0000 7fcd944a7640 1 -- 172.21.15.2:0/314261323 shutdown_connections 2024-03-15T13:00:55.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.560+0000 7fcd944a7640 1 --2- 172.21.15.2:0/314261323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd8c197950 0x7fcd8c197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.560+0000 7fcd944a7640 1 --2- 172.21.15.2:0/314261323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd8c1056a0 0x7fcd8c19dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.560+0000 7fcd944a7640 1 --2- 172.21.15.2:0/314261323 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcd780a1ba0 0x7fcd780a4060 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.560+0000 7fcd944a7640 1 --2- 172.21.15.2:0/314261323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c1005d0 0x7fcd8c19d7a0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.560+0000 7fcd944a7640 1 -- 172.21.15.2:0/314261323 >> 172.21.15.2:0/314261323 conn(0x7fcd8c0774f0 msgr2=0x7fcd8c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.560+0000 7fcd944a7640 1 -- 172.21.15.2:0/314261323 shutdown_connections 2024-03-15T13:00:55.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.561+0000 7fcd944a7640 1 -- 172.21.15.2:0/314261323 wait complete. 2024-03-15T13:00:55.560 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:00:55.560 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:00:55.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.724+0000 7f316f96c640 1 -- 172.21.15.2:0/3201515080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31680686d0 msgr2=0x7f3168068ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.724+0000 7f316f96c640 1 --2- 172.21.15.2:0/3201515080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31680686d0 0x7f3168068ab0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3164009900 tx=0x7f316402bc30 comp rx=0 tx=0).stop 2024-03-15T13:00:55.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.724+0000 7f316f96c640 1 -- 172.21.15.2:0/3201515080 shutdown_connections 2024-03-15T13:00:55.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.724+0000 7f316f96c640 1 --2- 172.21.15.2:0/3201515080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f316810de90 0x7f3168110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.724+0000 7f316f96c640 1 --2- 172.21.15.2:0/3201515080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3168068ff0 0x7f316810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.724+0000 7f316f96c640 1 --2- 172.21.15.2:0/3201515080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31680686d0 0x7f3168068ab0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.724+0000 7f316f96c640 1 -- 172.21.15.2:0/3201515080 >> 172.21.15.2:0/3201515080 conn(0x7f31680774f0 msgr2=0x7f3168077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.725+0000 7f316f96c640 1 -- 172.21.15.2:0/3201515080 shutdown_connections 2024-03-15T13:00:55.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.725+0000 7f316f96c640 1 -- 172.21.15.2:0/3201515080 wait complete. 2024-03-15T13:00:55.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.725+0000 7f316f96c640 1 Processor -- start 2024-03-15T13:00:55.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.725+0000 7f316f96c640 1 -- start start 2024-03-15T13:00:55.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316f96c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31680686d0 0x7f316819fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316f96c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3168068ff0 0x7f316819ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316f96c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f316810de90 0x7f3168199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316f96c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316819a2f0 con 0x7f3168068ff0 2024-03-15T13:00:55.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316f96c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316819a460 con 0x7f31680686d0 2024-03-15T13:00:55.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316f96c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316819a620 con 0x7f316810de90 2024-03-15T13:00:55.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316d708640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31680686d0 0x7f316819fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316d708640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31680686d0 0x7f316819fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:51010/0 (socket says 172.21.15.2:51010) 2024-03-15T13:00:55.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316cf07640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3168068ff0 0x7f316819ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316d708640 1 -- 172.21.15.2:0/3153351494 learned_addr learned my addr 172.21.15.2:0/3153351494 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316df09640 1 --2- 172.21.15.2:0/3153351494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f316810de90 0x7f3168199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316d708640 1 -- 172.21.15.2:0/3153351494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f316810de90 msgr2=0x7f3168199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316d708640 1 --2- 172.21.15.2:0/3153351494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f316810de90 0x7f3168199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316d708640 1 -- 172.21.15.2:0/3153351494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3168068ff0 msgr2=0x7f316819ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316d708640 1 --2- 172.21.15.2:0/3153351494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3168068ff0 0x7f316819ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316d708640 1 -- 172.21.15.2:0/3153351494 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3164009560 con 0x7f31680686d0 2024-03-15T13:00:55.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.726+0000 7f316cf07640 1 --2- 172.21.15.2:0/3153351494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3168068ff0 0x7f316819ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:55.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.727+0000 7f316d708640 1 --2- 172.21.15.2:0/3153351494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31680686d0 0x7f316819fa50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3164009230 tx=0x7f316402cac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.727+0000 7f315e7fc640 1 -- 172.21.15.2:0/3153351494 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164004250 con 0x7f31680686d0 2024-03-15T13:00:55.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.727+0000 7f316f96c640 1 -- 172.21.15.2:0/3153351494 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316819a8a0 con 0x7f31680686d0 2024-03-15T13:00:55.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.727+0000 7f315e7fc640 1 -- 172.21.15.2:0/3153351494 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f31640043b0 con 0x7f31680686d0 2024-03-15T13:00:55.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.727+0000 7f315e7fc640 1 -- 172.21.15.2:0/3153351494 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164003bf0 con 0x7f31680686d0 2024-03-15T13:00:55.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.727+0000 7f316f96c640 1 -- 172.21.15.2:0/3153351494 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31681a69d0 con 0x7f31680686d0 2024-03-15T13:00:55.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.730+0000 7f3172628740 1 Processor -- start 2024-03-15T13:00:55.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.730+0000 7f3172628740 1 -- start start 2024-03-15T13:00:55.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.730+0000 7f3172628740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56f250 0x55b1be56f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.730+0000 7f3172628740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56fb70 0x55b1be578210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.730+0000 7f3172628740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be578750 0x55b1be57ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.730+0000 7f3172628740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1be52a960 con 0x55b1be56fb70 2024-03-15T13:00:55.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.730+0000 7f3172628740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1be4de310 con 0x55b1be56f250 2024-03-15T13:00:55.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.730+0000 7f3172628740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1be4db840 con 0x55b1be578750 2024-03-15T13:00:55.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.730+0000 7f315e7fc640 1 -- 172.21.15.2:0/3153351494 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f3164003d50 con 0x7f31680686d0 2024-03-15T13:00:55.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.730+0000 7f316df09640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56f250 0x55b1be56f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.730+0000 7f316df09640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56f250 0x55b1be56f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:51016/0 (socket says 172.21.15.2:51016) 2024-03-15T13:00:55.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.730+0000 7f316df09640 1 -- 172.21.15.2:0/4222423076 learned_addr learned my addr 172.21.15.2:0/4222423076 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.730+0000 7f316cf07640 1 --2- 172.21.15.2:0/4222423076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be578750 0x55b1be57ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.730+0000 7f316d708640 1 --2- 172.21.15.2:0/4222423076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56fb70 0x55b1be578210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.731+0000 7f316df09640 1 -- 172.21.15.2:0/4222423076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be578750 msgr2=0x55b1be57ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.731+0000 7f316df09640 1 --2- 172.21.15.2:0/4222423076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be578750 0x55b1be57ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.731+0000 7f316df09640 1 -- 172.21.15.2:0/4222423076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56fb70 msgr2=0x55b1be578210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.731+0000 7f316df09640 1 --2- 172.21.15.2:0/4222423076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56fb70 0x55b1be578210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.731+0000 7f316df09640 1 -- 172.21.15.2:0/4222423076 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1be499060 con 0x55b1be56f250 2024-03-15T13:00:55.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.731+0000 7f315e7fc640 1 --2- 172.21.15.2:0/3153351494 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f31540a1ba0 0x7f31540a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.731+0000 7f316df09640 1 --2- 172.21.15.2:0/4222423076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56f250 0x55b1be56f630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3160009f30 tx=0x7f31600118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0f0119f9e6d0d00 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.731+0000 7f316d708640 1 --2- 172.21.15.2:0/3153351494 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f31540a1ba0 0x7f31540a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.731+0000 7f315e7fc640 1 -- 172.21.15.2:0/3153351494 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f31640efa60 con 0x7f31680686d0 2024-03-15T13:00:55.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.731+0000 7f31537fe640 1 -- 172.21.15.2:0/4222423076 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160004d20 con 0x55b1be56f250 2024-03-15T13:00:55.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.731+0000 7f31537fe640 1 -- 172.21.15.2:0/4222423076 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3160004e80 con 0x55b1be56f250 2024-03-15T13:00:55.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.731+0000 7f31537fe640 1 -- 172.21.15.2:0/4222423076 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160005020 con 0x55b1be56f250 2024-03-15T13:00:55.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.732+0000 7f3172628740 1 -- 172.21.15.2:0/4222423076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56f250 msgr2=0x55b1be56f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.732+0000 7f316d708640 1 --2- 172.21.15.2:0/3153351494 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f31540a1ba0 0x7f31540a4060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f31640ed020 tx=0x7f31640b8040 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.732+0000 7f3172628740 1 --2- 172.21.15.2:0/4222423076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56f250 0x55b1be56f630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3160009f30 tx=0x7f31600118d0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.732+0000 7f3172628740 1 -- 172.21.15.2:0/4222423076 shutdown_connections 2024-03-15T13:00:55.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.732+0000 7f3172628740 1 --2- 172.21.15.2:0/4222423076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be578750 0x55b1be57ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.732+0000 7f3172628740 1 --2- 172.21.15.2:0/4222423076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56fb70 0x55b1be578210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.732+0000 7f3172628740 1 --2- 172.21.15.2:0/4222423076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56f250 0x55b1be56f630 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.732+0000 7f3172628740 1 -- 172.21.15.2:0/4222423076 >> 172.21.15.2:0/4222423076 conn(0x55b1be569980 msgr2=0x55b1be569d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.733+0000 7f3172628740 1 -- 172.21.15.2:0/4222423076 shutdown_connections 2024-03-15T13:00:55.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.733+0000 7f3172628740 1 -- 172.21.15.2:0/4222423076 wait complete. 2024-03-15T13:00:55.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.733+0000 7f3172628740 1 -- start start 2024-03-15T13:00:55.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.733+0000 7f3172628740 1 Processor -- start 2024-03-15T13:00:55.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f3172628740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56f250 0x55b1be578100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f3172628740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be56fb70 0x55b1be575490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f3172628740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be578750 0x55b1be575a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f3172628740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1be4d7020 con 0x55b1be578750 2024-03-15T13:00:55.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f316cf07640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be56fb70 0x55b1be575490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f316cf07640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be56fb70 0x55b1be575490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:35024/0 (socket says 172.21.15.2:35024) 2024-03-15T13:00:55.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f316cf07640 1 -- 172.21.15.2:0/3163382415 learned_addr learned my addr 172.21.15.2:0/3163382415 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f316df09640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56f250 0x55b1be578100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f3172628740 1 -- 172.21.15.2:0/3163382415 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1be1aa430 con 0x55b1be56f250 2024-03-15T13:00:55.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f3172628740 1 -- 172.21.15.2:0/3163382415 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1be166e00 con 0x55b1be56fb70 2024-03-15T13:00:55.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f3172628740 10 client.0 _ll_register_callbacks cb 0x55b1be47ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:55.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f316df09640 1 --2- 172.21.15.2:0/3163382415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be578750 0x55b1be575a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f316df09640 1 -- 172.21.15.2:0/3163382415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be56fb70 msgr2=0x55b1be575490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f316df09640 1 --2- 172.21.15.2:0/3163382415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be56fb70 0x55b1be575490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f316df09640 1 -- 172.21.15.2:0/3163382415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be578750 msgr2=0x55b1be575a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f316df09640 1 --2- 172.21.15.2:0/3163382415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be578750 0x55b1be575a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f316df09640 1 -- 172.21.15.2:0/3163382415 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3160009be0 con 0x55b1be56f250 2024-03-15T13:00:55.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.734+0000 7f316cf07640 1 --2- 172.21.15.2:0/3163382415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be56fb70 0x55b1be575490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:55.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.735+0000 7f316df09640 1 --2- 172.21.15.2:0/3163382415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56f250 0x55b1be578100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f316001b040 tx=0x7f3160004590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7472c60a64be45a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.735+0000 7f3150ff9640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:00:55.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.735+0000 7f3150ff9640 1 -- 172.21.15.2:0/3163382415 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31600047f0 con 0x55b1be56f250 2024-03-15T13:00:55.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.735+0000 7f3150ff9640 1 -- 172.21.15.2:0/3163382415 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f3160005200 con 0x55b1be56f250 2024-03-15T13:00:55.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.735+0000 7f3172628740 1 -- 172.21.15.2:0/3163382415 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1be213720 con 0x55b1be56f250 2024-03-15T13:00:55.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.735+0000 7f3150ff9640 1 -- 172.21.15.2:0/3163382415 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31600053c0 con 0x55b1be56f250 2024-03-15T13:00:55.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.735+0000 7f3172628740 10 client.26246 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:55.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.735+0000 7f3172628740 1 -- 172.21.15.2:0/3163382415 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b1be4d56a0 con 0x55b1be56f250 2024-03-15T13:00:55.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.735+0000 7f3150ff9640 1 -- 172.21.15.2:0/3163382415 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55b1be213720 con 0x55b1be56f250 2024-03-15T13:00:55.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.736+0000 7f3150ff9640 1 -- 172.21.15.2:0/3163382415 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f3160021070 con 0x55b1be56f250 2024-03-15T13:00:55.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.736+0000 7f3172628740 10 client.26246 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:55.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.736+0000 7f3172628740 10 client.26246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:55.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.736+0000 7f3172628740 20 client.26246 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:55.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.736+0000 7f3172628740 1 --2- 172.21.15.2:0/3163382415 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b1be588820 0x55b1be58ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.736+0000 7f3172628740 4 client.26246 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:55.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.736+0000 7f3172628740 1 -- 172.21.15.2:0/3163382415 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1be262890 con 0x55b1be588820 2024-03-15T13:00:55.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.736+0000 7f316cf07640 1 --2- 172.21.15.2:0/3163382415 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b1be588820 0x55b1be58ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.736+0000 7f316cf07640 1 --2- 172.21.15.2:0/3163382415 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b1be588820 0x55b1be58ac40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b25dcc3c8ea2c323 server_cookie=a22f53b819880ebd in_seq=0 out_seq=0 2024-03-15T13:00:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.736+0000 7f3150ff9640 10 client.26246 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.737+0000 7f3150ff9640 1 -- 172.21.15.2:0/3163382415 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b1be262890 con 0x55b1be588820 2024-03-15T13:00:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.737+0000 7f3150ff9640 10 client.26246 handle_command_reply: tid=0 2024-03-15T13:00:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.737+0000 7f3172628740 1 client.26246 shutdown 2024-03-15T13:00:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.737+0000 7f3172628740 1 -- 172.21.15.2:0/3163382415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56f250 msgr2=0x55b1be578100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.737+0000 7f3172628740 1 --2- 172.21.15.2:0/3163382415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56f250 0x55b1be578100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f316001b040 tx=0x7f3160004590 comp rx=0 tx=0).stop 2024-03-15T13:00:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.737+0000 7f3172628740 1 -- 172.21.15.2:0/3163382415 shutdown_connections 2024-03-15T13:00:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.737+0000 7f3172628740 1 --2- 172.21.15.2:0/3163382415 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b1be588820 0x55b1be58ac40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.737+0000 7f3172628740 1 --2- 172.21.15.2:0/3163382415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be578750 0x55b1be575a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.737+0000 7f3172628740 1 --2- 172.21.15.2:0/3163382415 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be56fb70 0x55b1be575490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.737+0000 7f3172628740 1 --2- 172.21.15.2:0/3163382415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56f250 0x55b1be578100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.737+0000 7f3172628740 1 -- 172.21.15.2:0/3163382415 >> 172.21.15.2:0/3163382415 conn(0x55b1be569980 msgr2=0x55b1be56b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.737+0000 7f3150ff9640 0 client.26246 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.737+0000 7f3172628740 1 -- 172.21.15.2:0/3163382415 shutdown_connections 2024-03-15T13:00:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.737+0000 7f3172628740 1 -- 172.21.15.2:0/3163382415 wait complete. 2024-03-15T13:00:55.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.737+0000 7f3172628740 20 client.26246 trim_cache size 0 max 16384 2024-03-15T13:00:55.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.753+0000 7f3172628740 1 Processor -- start 2024-03-15T13:00:55.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.753+0000 7f3172628740 1 -- start start 2024-03-15T13:00:55.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.753+0000 7f3172628740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56f250 0x55b1be568a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.753+0000 7f3172628740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56fb70 0x55b1be577bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.753+0000 7f3172628740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be578750 0x55b1be578100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.753+0000 7f3172628740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1be45c130 con 0x55b1be56f250 2024-03-15T13:00:55.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.753+0000 7f3172628740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1be5853a0 con 0x55b1be56fb70 2024-03-15T13:00:55.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.753+0000 7f3172628740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1be56eeb0 con 0x55b1be578750 2024-03-15T13:00:55.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.754+0000 7f316df09640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56f250 0x55b1be568a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.754+0000 7f316df09640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56f250 0x55b1be568a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32936/0 (socket says 172.21.15.2:32936) 2024-03-15T13:00:55.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.754+0000 7f316df09640 1 -- 172.21.15.2:0/1385902937 learned_addr learned my addr 172.21.15.2:0/1385902937 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.754+0000 7f316cf07640 1 --2- 172.21.15.2:0/1385902937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56fb70 0x55b1be577bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.754+0000 7f316df09640 1 --2- 172.21.15.2:0/1385902937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be578750 0x55b1be578100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.754+0000 7f316df09640 1 -- 172.21.15.2:0/1385902937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be578750 msgr2=0x55b1be578100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.754+0000 7f316df09640 1 --2- 172.21.15.2:0/1385902937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be578750 0x55b1be578100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.754+0000 7f316df09640 1 -- 172.21.15.2:0/1385902937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56fb70 msgr2=0x55b1be577bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.754+0000 7f316df09640 1 --2- 172.21.15.2:0/1385902937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56fb70 0x55b1be577bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.754+0000 7f316df09640 1 -- 172.21.15.2:0/1385902937 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1be471c20 con 0x55b1be56f250 2024-03-15T13:00:55.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.754+0000 7f316df09640 1 --2- 172.21.15.2:0/1385902937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56f250 0x55b1be568a00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f31600044b0 tx=0x7f3160009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df22c6dd4bde4a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.755+0000 7f3152ffd640 1 -- 172.21.15.2:0/1385902937 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160004d00 con 0x55b1be56f250 2024-03-15T13:00:55.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.755+0000 7f3152ffd640 1 -- 172.21.15.2:0/1385902937 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f316002c070 con 0x55b1be56f250 2024-03-15T13:00:55.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.755+0000 7f3152ffd640 1 -- 172.21.15.2:0/1385902937 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316000f730 con 0x55b1be56f250 2024-03-15T13:00:55.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.756+0000 7f3172628740 1 -- 172.21.15.2:0/1385902937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56f250 msgr2=0x55b1be568a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.756+0000 7f3172628740 1 --2- 172.21.15.2:0/1385902937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56f250 0x55b1be568a00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f31600044b0 tx=0x7f3160009b50 comp rx=0 tx=0).stop 2024-03-15T13:00:55.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.756+0000 7f3172628740 1 -- 172.21.15.2:0/1385902937 shutdown_connections 2024-03-15T13:00:55.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.756+0000 7f3172628740 1 --2- 172.21.15.2:0/1385902937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be578750 0x55b1be578100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.756+0000 7f3172628740 1 --2- 172.21.15.2:0/1385902937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be56fb70 0x55b1be577bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.756+0000 7f3172628740 1 --2- 172.21.15.2:0/1385902937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56f250 0x55b1be568a00 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.756+0000 7f3172628740 1 -- 172.21.15.2:0/1385902937 >> 172.21.15.2:0/1385902937 conn(0x55b1be569980 msgr2=0x55b1be5682f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.756+0000 7f3172628740 1 -- 172.21.15.2:0/1385902937 shutdown_connections 2024-03-15T13:00:55.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.756+0000 7f3172628740 1 -- 172.21.15.2:0/1385902937 wait complete. 2024-03-15T13:00:55.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.756+0000 7f3172628740 1 -- start start 2024-03-15T13:00:55.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.756+0000 7f3172628740 1 Processor -- start 2024-03-15T13:00:55.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f3172628740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be56f250 0x55b1be571880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f3172628740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56fb70 0x55b1be571dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f3172628740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be578750 0x55b1be572300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f3172628740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1be573320 con 0x55b1be56fb70 2024-03-15T13:00:55.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f3172628740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1be573490 con 0x55b1be578750 2024-03-15T13:00:55.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f3172628740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1be5e3090 con 0x55b1be56f250 2024-03-15T13:00:55.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f3172628740 10 client.0 _ll_register_callbacks cb 0x7f31640f4040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:55.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f316cf07640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56fb70 0x55b1be571dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f316cf07640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56fb70 0x55b1be571dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32938/0 (socket says 172.21.15.2:32938) 2024-03-15T13:00:55.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f316cf07640 1 -- 172.21.15.2:0/4243667929 learned_addr learned my addr 172.21.15.2:0/4243667929 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:55.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f316df09640 1 --2- 172.21.15.2:0/4243667929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be56f250 0x55b1be571880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f316df09640 1 --2- 172.21.15.2:0/4243667929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be578750 0x55b1be572300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f316cf07640 1 -- 172.21.15.2:0/4243667929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be56f250 msgr2=0x55b1be571880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f316cf07640 1 --2- 172.21.15.2:0/4243667929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be56f250 0x55b1be571880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f316cf07640 1 -- 172.21.15.2:0/4243667929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be578750 msgr2=0x55b1be572300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:55.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f316cf07640 1 --2- 172.21.15.2:0/4243667929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be578750 0x55b1be572300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f316cf07640 1 -- 172.21.15.2:0/4243667929 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1be4d56a0 con 0x55b1be56fb70 2024-03-15T13:00:55.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f316cf07640 1 --2- 172.21.15.2:0/4243667929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56fb70 0x55b1be571dc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x55b1be576a60 tx=0x7f3158012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4369f73d3626fa61 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:55.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.757+0000 7f3150ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:55.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.758+0000 7f3150ff9640 1 -- 172.21.15.2:0/4243667929 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158017040 con 0x55b1be56fb70 2024-03-15T13:00:55.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.758+0000 7f3150ff9640 1 -- 172.21.15.2:0/4243667929 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f31580222b0 con 0x55b1be56fb70 2024-03-15T13:00:55.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.758+0000 7f3172628740 1 -- 172.21.15.2:0/4243667929 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1be5650f0 con 0x55b1be56fb70 2024-03-15T13:00:55.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.758+0000 7f3150ff9640 1 -- 172.21.15.2:0/4243667929 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31580122e0 con 0x55b1be56fb70 2024-03-15T13:00:55.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.758+0000 7f3172628740 10 client.16389 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:55.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.758+0000 7f3172628740 1 -- 172.21.15.2:0/4243667929 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b1be4978c0 con 0x55b1be56fb70 2024-03-15T13:00:55.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.758+0000 7f3150ff9640 1 -- 172.21.15.2:0/4243667929 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3158033030 con 0x55b1be56fb70 2024-03-15T13:00:55.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.758+0000 7f3150ff9640 1 -- 172.21.15.2:0/4243667929 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7f315802b5a0 con 0x55b1be56fb70 2024-03-15T13:00:55.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.758+0000 7f3172628740 10 client.16389 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:55.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.758+0000 7f3172628740 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:55.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.758+0000 7f3172628740 20 client.16389 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:55.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.758+0000 7f3172628740 1 --2- 172.21.15.2:0/4243667929 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b1be5e4980 0x55b1be5e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:55.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.758+0000 7f3172628740 4 client.16389 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:00:55.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.758+0000 7f3172628740 1 -- 172.21.15.2:0/4243667929 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b1be1d67e0 con 0x55b1be5e4980 2024-03-15T13:00:55.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.759+0000 7f316df09640 1 --2- 172.21.15.2:0/4243667929 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b1be5e4980 0x55b1be5e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:55.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.759+0000 7f3150ff9640 10 client.16389 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:55.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.759+0000 7f316df09640 1 --2- 172.21.15.2:0/4243667929 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b1be5e4980 0x55b1be5e6da0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdfb688131299c10 server_cookie=fe480c8f5f44e6f2 in_seq=0 out_seq=0 2024-03-15T13:00:55.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.764+0000 7f3150ff9640 1 -- 172.21.15.2:0/4243667929 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b1be1d67e0 con 0x55b1be5e4980 2024-03-15T13:00:55.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.764+0000 7f3150ff9640 10 client.16389 handle_command_reply: tid=0 2024-03-15T13:00:55.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.764+0000 7f3172628740 1 client.16389 shutdown 2024-03-15T13:00:55.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.764+0000 7f3172628740 1 -- 172.21.15.2:0/4243667929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56fb70 msgr2=0x55b1be571dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.764+0000 7f3172628740 1 --2- 172.21.15.2:0/4243667929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56fb70 0x55b1be571dc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x55b1be576a60 tx=0x7f3158012f90 comp rx=0 tx=0).stop 2024-03-15T13:00:55.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.764+0000 7f3172628740 1 -- 172.21.15.2:0/4243667929 shutdown_connections 2024-03-15T13:00:55.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.764+0000 7f3172628740 1 --2- 172.21.15.2:0/4243667929 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b1be5e4980 0x55b1be5e6da0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.764+0000 7f3172628740 1 --2- 172.21.15.2:0/4243667929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1be578750 0x55b1be572300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.764+0000 7f3172628740 1 --2- 172.21.15.2:0/4243667929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b1be56fb70 0x55b1be571dc0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.764+0000 7f3150ff9640 0 client.16389 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:55.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.764+0000 7f3172628740 1 --2- 172.21.15.2:0/4243667929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1be56f250 0x55b1be571880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.764+0000 7f3172628740 1 -- 172.21.15.2:0/4243667929 >> 172.21.15.2:0/4243667929 conn(0x55b1be569980 msgr2=0x55b1be568b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.764+0000 7f3172628740 1 -- 172.21.15.2:0/4243667929 shutdown_connections 2024-03-15T13:00:55.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.764+0000 7f3172628740 1 -- 172.21.15.2:0/4243667929 wait complete. 2024-03-15T13:00:55.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.764+0000 7f3172628740 20 client.16389 trim_cache size 0 max 16384 2024-03-15T13:00:55.751 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:00:55.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.765+0000 7f316f96c640 1 -- 172.21.15.2:0/3153351494 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f31540a1ba0 msgr2=0x7f31540a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.765+0000 7f316f96c640 1 --2- 172.21.15.2:0/3153351494 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f31540a1ba0 0x7f31540a4060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f31640ed020 tx=0x7f31640b8040 comp rx=0 tx=0).stop 2024-03-15T13:00:55.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.765+0000 7f316f96c640 1 -- 172.21.15.2:0/3153351494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31680686d0 msgr2=0x7f316819fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:55.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.765+0000 7f316f96c640 1 --2- 172.21.15.2:0/3153351494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31680686d0 0x7f316819fa50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3164009230 tx=0x7f316402cac0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.765+0000 7f316f96c640 1 -- 172.21.15.2:0/3153351494 shutdown_connections 2024-03-15T13:00:55.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.765+0000 7f316f96c640 1 --2- 172.21.15.2:0/3153351494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f316810de90 0x7f3168199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.765+0000 7f316f96c640 1 --2- 172.21.15.2:0/3153351494 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f31540a1ba0 0x7f31540a4060 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.765+0000 7f316f96c640 1 --2- 172.21.15.2:0/3153351494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3168068ff0 0x7f316819ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.765+0000 7f316f96c640 1 --2- 172.21.15.2:0/3153351494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31680686d0 0x7f316819fa50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:55.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.765+0000 7f316f96c640 1 -- 172.21.15.2:0/3153351494 >> 172.21.15.2:0/3153351494 conn(0x7f31680774f0 msgr2=0x7f316810f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:55.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.765+0000 7f316f96c640 1 -- 172.21.15.2:0/3153351494 shutdown_connections 2024-03-15T13:00:55.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:55.765+0000 7f316f96c640 1 -- 172.21.15.2:0/3153351494 wait complete. 2024-03-15T13:00:56.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:55 smithi002 ceph-mon[32060]: pgmap v63: 129 pgs: 129 active+clean; 144 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 2.6 MiB/s wr, 134 op/s 2024-03-15T13:00:56.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:55 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3773414736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:00:56.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:55 smithi184 ceph-mon[35993]: pgmap v63: 129 pgs: 129 active+clean; 144 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 2.6 MiB/s wr, 134 op/s 2024-03-15T13:00:56.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:55 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3773414736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:00:56.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:55 smithi149 ceph-mon[35927]: pgmap v63: 129 pgs: 129 active+clean; 144 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 2.6 MiB/s wr, 134 op/s 2024-03-15T13:00:56.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:55 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3773414736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:00:56.763 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:00:56.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:00:56.764 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:00:56.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.902+0000 7fd9d54c3640 1 -- 172.21.15.2:0/2088028865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9d0100750 msgr2=0x7fd9d010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:56.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.902+0000 7fd9d54c3640 1 --2- 172.21.15.2:0/2088028865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9d0100750 0x7fd9d010ca20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009ca0 tx=0x7fd9c802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.902+0000 7fd9d54c3640 1 -- 172.21.15.2:0/2088028865 shutdown_connections 2024-03-15T13:00:56.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.902+0000 7fd9d54c3640 1 --2- 172.21.15.2:0/2088028865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9d0100750 0x7fd9d010ca20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.902+0000 7fd9d54c3640 1 --2- 172.21.15.2:0/2088028865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9d00ffdb0 0x7fd9d0100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.902+0000 7fd9d54c3640 1 --2- 172.21.15.2:0/2088028865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d0104b20 0x7fd9d0104f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.902+0000 7fd9d54c3640 1 -- 172.21.15.2:0/2088028865 >> 172.21.15.2:0/2088028865 conn(0x7fd9d0075730 msgr2=0x7fd9d0075b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:56.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.902+0000 7fd9d54c3640 1 -- 172.21.15.2:0/2088028865 shutdown_connections 2024-03-15T13:00:56.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.902+0000 7fd9d54c3640 1 -- 172.21.15.2:0/2088028865 wait complete. 2024-03-15T13:00:56.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.903+0000 7fd9d54c3640 1 Processor -- start 2024-03-15T13:00:56.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.903+0000 7fd9d54c3640 1 -- start start 2024-03-15T13:00:56.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9d54c3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9d00ffdb0 0x7fd9d01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9d54c3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d0100750 0x7fd9d0198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9d54c3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9d0104b20 0x7fd9d01a3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9d54c3640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d0199670 con 0x7fd9d00ffdb0 2024-03-15T13:00:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9d54c3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d01a3e90 con 0x7fd9d0104b20 2024-03-15T13:00:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9d54c3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d01a3fd0 con 0x7fd9d0100750 2024-03-15T13:00:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9ceffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9d00ffdb0 0x7fd9d01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9ceffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9d00ffdb0 0x7fd9d01988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32948/0 (socket says 172.21.15.2:32948) 2024-03-15T13:00:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9ceffd640 1 -- 172.21.15.2:0/673336152 learned_addr learned my addr 172.21.15.2:0/673336152 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9c67fc640 1 --2- 172.21.15.2:0/673336152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d0100750 0x7fd9d0198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9cf7fe640 1 --2- 172.21.15.2:0/673336152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9d0104b20 0x7fd9d01a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9c67fc640 1 -- 172.21.15.2:0/673336152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9d0104b20 msgr2=0x7fd9d01a3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9c67fc640 1 --2- 172.21.15.2:0/673336152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9d0104b20 0x7fd9d01a3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9c67fc640 1 -- 172.21.15.2:0/673336152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9d00ffdb0 msgr2=0x7fd9d01988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9c67fc640 1 --2- 172.21.15.2:0/673336152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9d00ffdb0 0x7fd9d01988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9c67fc640 1 -- 172.21.15.2:0/673336152 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c8009900 con 0x7fd9d0100750 2024-03-15T13:00:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9ceffd640 1 --2- 172.21.15.2:0/673336152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9d00ffdb0 0x7fd9d01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.904+0000 7fd9c67fc640 1 --2- 172.21.15.2:0/673336152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d0100750 0x7fd9d0198e10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800e9f0 tx=0x7fd9b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.905+0000 7fd9ccff9640 1 -- 172.21.15.2:0/673336152 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b800ccc0 con 0x7fd9d0100750 2024-03-15T13:00:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.905+0000 7fd9ccff9640 1 -- 172.21.15.2:0/673336152 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd9b800ce20 con 0x7fd9d0100750 2024-03-15T13:00:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.905+0000 7fd9d54c3640 1 -- 172.21.15.2:0/673336152 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d01a4230 con 0x7fd9d0100750 2024-03-15T13:00:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.905+0000 7fd9ccff9640 1 -- 172.21.15.2:0/673336152 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8018880 con 0x7fd9d0100750 2024-03-15T13:00:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.905+0000 7fd9d54c3640 1 -- 172.21.15.2:0/673336152 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d01a4740 con 0x7fd9d0100750 2024-03-15T13:00:56.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.907+0000 7fd9ccff9640 1 -- 172.21.15.2:0/673336152 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fd9b80189e0 con 0x7fd9d0100750 2024-03-15T13:00:56.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.907+0000 7fd9d817e740 1 Processor -- start 2024-03-15T13:00:56.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.907+0000 7fd9ccff9640 1 --2- 172.21.15.2:0/673336152 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd9bc0a1ba0 0x7fd9bc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.907+0000 7fd9d817e740 1 -- start start 2024-03-15T13:00:56.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9cf7fe640 1 --2- 172.21.15.2:0/673336152 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd9bc0a1ba0 0x7fd9bc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9ccff9640 1 -- 172.21.15.2:0/673336152 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fd9b8014070 con 0x7fd9d0100750 2024-03-15T13:00:56.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9d817e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694380 0x56134b694760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9d817e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694d30 0x56134b69d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9d817e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b69d8b0 0x56134b69fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9d817e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134b602f90 con 0x56134b694d30 2024-03-15T13:00:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9d817e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134b6004c0 con 0x56134b69d8b0 2024-03-15T13:00:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9d817e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134b5fbca0 con 0x56134b694380 2024-03-15T13:00:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9c67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694d30 0x56134b69d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9c67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694d30 0x56134b69d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32956/0 (socket says 172.21.15.2:32956) 2024-03-15T13:00:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9c67fc640 1 -- 172.21.15.2:0/787682449 learned_addr learned my addr 172.21.15.2:0/787682449 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9ceffd640 1 --2- 172.21.15.2:0/787682449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694380 0x56134b694760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9cf7fe640 1 --2- 172.21.15.2:0/787682449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b69d8b0 0x56134b69fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9cf7fe640 1 --2- 172.21.15.2:0/673336152 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd9bc0a1ba0 0x7fd9bc0a4060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8004af0 tx=0x7fd9c8005c50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9c67fc640 1 -- 172.21.15.2:0/787682449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694380 msgr2=0x56134b694760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9c67fc640 1 --2- 172.21.15.2:0/787682449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694380 0x56134b694760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9c67fc640 1 -- 172.21.15.2:0/787682449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b69d8b0 msgr2=0x56134b69fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9c67fc640 1 --2- 172.21.15.2:0/787682449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b69d8b0 0x56134b69fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9c67fc640 1 -- 172.21.15.2:0/787682449 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56134b5be0b0 con 0x56134b694d30 2024-03-15T13:00:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9cf7fe640 1 --2- 172.21.15.2:0/787682449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b69d8b0 0x56134b69fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:00:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9c67fc640 1 --2- 172.21.15.2:0/787682449 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694d30 0x56134b69d2e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80a29e0 tx=0x7fd9b809a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e8ef5f5dd584ce server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.908+0000 7fd9c57fa640 1 -- 172.21.15.2:0/787682449 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8010b90 con 0x56134b694d30 2024-03-15T13:00:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.909+0000 7fd9c57fa640 1 -- 172.21.15.2:0/787682449 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd9b80cb510 con 0x56134b694d30 2024-03-15T13:00:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.909+0000 7fd9c57fa640 1 -- 172.21.15.2:0/787682449 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8096570 con 0x56134b694d30 2024-03-15T13:00:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.909+0000 7fd9d817e740 1 -- 172.21.15.2:0/787682449 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694d30 msgr2=0x56134b69d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.909+0000 7fd9d817e740 1 --2- 172.21.15.2:0/787682449 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694d30 0x56134b69d2e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80a29e0 tx=0x7fd9b809a820 comp rx=0 tx=0).stop 2024-03-15T13:00:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.909+0000 7fd9d817e740 1 -- 172.21.15.2:0/787682449 shutdown_connections 2024-03-15T13:00:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.909+0000 7fd9d817e740 1 --2- 172.21.15.2:0/787682449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b69d8b0 0x56134b69fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.909+0000 7fd9d817e740 1 --2- 172.21.15.2:0/787682449 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694d30 0x56134b69d2e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.909+0000 7fd9d817e740 1 --2- 172.21.15.2:0/787682449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694380 0x56134b694760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.909+0000 7fd9d817e740 1 -- 172.21.15.2:0/787682449 >> 172.21.15.2:0/787682449 conn(0x56134b68e840 msgr2=0x56134b68ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9d817e740 1 -- 172.21.15.2:0/787682449 shutdown_connections 2024-03-15T13:00:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9d817e740 1 -- 172.21.15.2:0/787682449 wait complete. 2024-03-15T13:00:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9d817e740 1 -- start start 2024-03-15T13:00:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9d817e740 1 Processor -- start 2024-03-15T13:00:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9d817e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694380 0x56134b69a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9d817e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694d30 0x56134b69ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9d817e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b69d8b0 0x56134b69b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9d817e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134b5a0f40 con 0x56134b694d30 2024-03-15T13:00:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9d817e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134b2cf4b0 con 0x56134b69d8b0 2024-03-15T13:00:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9d817e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134b581670 con 0x56134b694380 2024-03-15T13:00:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9d817e740 10 client.0 _ll_register_callbacks cb 0x56134b5eef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9c67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694d30 0x56134b69ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9c67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694d30 0x56134b69ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32964/0 (socket says 172.21.15.2:32964) 2024-03-15T13:00:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9c67fc640 1 -- 172.21.15.2:0/2856020103 learned_addr learned my addr 172.21.15.2:0/2856020103 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9ceffd640 1 --2- 172.21.15.2:0/2856020103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694380 0x56134b69a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9cf7fe640 1 --2- 172.21.15.2:0/2856020103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b69d8b0 0x56134b69b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9c67fc640 1 -- 172.21.15.2:0/2856020103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694380 msgr2=0x56134b69a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9c67fc640 1 --2- 172.21.15.2:0/2856020103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694380 0x56134b69a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9c67fc640 1 -- 172.21.15.2:0/2856020103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b69d8b0 msgr2=0x56134b69b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9c67fc640 1 --2- 172.21.15.2:0/2856020103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b69d8b0 0x56134b69b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9c67fc640 1 -- 172.21.15.2:0/2856020103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d01a4740 con 0x56134b694d30 2024-03-15T13:00:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.910+0000 7fd9cf7fe640 1 --2- 172.21.15.2:0/2856020103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b69d8b0 0x56134b69b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:00:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9c67fc640 1 --2- 172.21.15.2:0/2856020103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694d30 0x56134b69ae90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd9b809ea80 tx=0x7fd9b8010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50a95a81fa6f10b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9b67fc640 1 -- 172.21.15.2:0/2856020103 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b80b3220 con 0x56134b694d30 2024-03-15T13:00:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9b67fc640 1 -- 172.21.15.2:0/2856020103 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fd9b80ba1c0 con 0x56134b694d30 2024-03-15T13:00:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9d817e740 1 -- 172.21.15.2:0/2856020103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56134b5d5f40 con 0x56134b694d30 2024-03-15T13:00:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9b67fc640 1 -- 172.21.15.2:0/2856020103 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9b80ba380 con 0x56134b694d30 2024-03-15T13:00:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9d817e740 10 client.16401 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9d817e740 1 -- 172.21.15.2:0/2856020103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56134b5fa320 con 0x56134b694d30 2024-03-15T13:00:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9b67fc640 1 -- 172.21.15.2:0/2856020103 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x56134b5d5f40 con 0x56134b694d30 2024-03-15T13:00:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9b67fc640 1 -- 172.21.15.2:0/2856020103 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fd9b80996e0 con 0x56134b694d30 2024-03-15T13:00:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9d817e740 10 client.16401 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9d817e740 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9d817e740 20 client.16401 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9d817e740 1 --2- 172.21.15.2:0/2856020103 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56134b6adb50 0x56134b6aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9d817e740 4 client.16401 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.911+0000 7fd9d817e740 1 -- 172.21.15.2:0/2856020103 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56134b5f2310 con 0x56134b6adb50 2024-03-15T13:00:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.912+0000 7fd9ceffd640 1 --2- 172.21.15.2:0/2856020103 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56134b6adb50 0x56134b6aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.912+0000 7fd9b67fc640 10 client.16401 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.912+0000 7fd9ceffd640 1 --2- 172.21.15.2:0/2856020103 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56134b6adb50 0x56134b6aff70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ccd2eab12330856 server_cookie=d22cb219a011eff2 in_seq=0 out_seq=0 2024-03-15T13:00:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.913+0000 7fd9b67fc640 1 -- 172.21.15.2:0/2856020103 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56134b5f2310 con 0x56134b6adb50 2024-03-15T13:00:56.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.913+0000 7fd9b67fc640 10 client.16401 handle_command_reply: tid=0 2024-03-15T13:00:56.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.913+0000 7fd9d817e740 1 client.16401 shutdown 2024-03-15T13:00:56.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.913+0000 7fd9d817e740 1 -- 172.21.15.2:0/2856020103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694d30 msgr2=0x56134b69ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:56.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.913+0000 7fd9d817e740 1 --2- 172.21.15.2:0/2856020103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694d30 0x56134b69ae90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd9b809ea80 tx=0x7fd9b8010dc0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.913+0000 7fd9d817e740 1 -- 172.21.15.2:0/2856020103 shutdown_connections 2024-03-15T13:00:56.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.913+0000 7fd9d817e740 1 --2- 172.21.15.2:0/2856020103 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56134b6adb50 0x56134b6aff70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.914+0000 7fd9d817e740 1 --2- 172.21.15.2:0/2856020103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b69d8b0 0x56134b69b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.914+0000 7fd9d817e740 1 --2- 172.21.15.2:0/2856020103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694d30 0x56134b69ae90 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.914+0000 7fd9d817e740 1 --2- 172.21.15.2:0/2856020103 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694380 0x56134b69a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.914+0000 7fd9d817e740 1 -- 172.21.15.2:0/2856020103 >> 172.21.15.2:0/2856020103 conn(0x56134b68e840 msgr2=0x56134b6907a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:56.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.914+0000 7fd9b67fc640 0 client.16401 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:56.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.914+0000 7fd9d817e740 1 -- 172.21.15.2:0/2856020103 shutdown_connections 2024-03-15T13:00:56.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.914+0000 7fd9d817e740 1 -- 172.21.15.2:0/2856020103 wait complete. 2024-03-15T13:00:56.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.914+0000 7fd9d817e740 20 client.16401 trim_cache size 0 max 16384 2024-03-15T13:00:56.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.933+0000 7fd9d817e740 1 Processor -- start 2024-03-15T13:00:56.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.933+0000 7fd9d817e740 1 -- start start 2024-03-15T13:00:56.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9d817e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b694380 0x56134b69d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9d817e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694d30 0x56134b69b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9d817e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b69d8b0 0x56134b690590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9d817e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134b6997c0 con 0x56134b69d8b0 2024-03-15T13:00:56.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9d817e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134b694ac0 con 0x56134b694380 2024-03-15T13:00:56.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9d817e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134b68fc80 con 0x56134b694d30 2024-03-15T13:00:56.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9cf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b69d8b0 0x56134b690590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9cf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b69d8b0 0x56134b690590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32972/0 (socket says 172.21.15.2:32972) 2024-03-15T13:00:56.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9c67fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694d30 0x56134b69b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9cf7fe640 1 -- 172.21.15.2:0/450834276 learned_addr learned my addr 172.21.15.2:0/450834276 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:56.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9ceffd640 1 --2- 172.21.15.2:0/450834276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b694380 0x56134b69d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9cf7fe640 1 -- 172.21.15.2:0/450834276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694d30 msgr2=0x56134b69b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9cf7fe640 1 --2- 172.21.15.2:0/450834276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694d30 0x56134b69b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9cf7fe640 1 -- 172.21.15.2:0/450834276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b694380 msgr2=0x56134b69d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9cf7fe640 1 --2- 172.21.15.2:0/450834276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b694380 0x56134b69d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9cf7fe640 1 -- 172.21.15.2:0/450834276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56134b596c00 con 0x56134b69d8b0 2024-03-15T13:00:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9c67fc640 1 --2- 172.21.15.2:0/450834276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694d30 0x56134b69b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.934+0000 7fd9cf7fe640 1 --2- 172.21.15.2:0/450834276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b69d8b0 0x56134b690590 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009ca0 tx=0x7fd9c8044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7a9599cbfd7ac4 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.935+0000 7fd9c4ff9640 1 -- 172.21.15.2:0/450834276 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80326b0 con 0x56134b69d8b0 2024-03-15T13:00:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.935+0000 7fd9c4ff9640 1 -- 172.21.15.2:0/450834276 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd9c8003e40 con 0x56134b69d8b0 2024-03-15T13:00:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.935+0000 7fd9c4ff9640 1 -- 172.21.15.2:0/450834276 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80046e0 con 0x56134b69d8b0 2024-03-15T13:00:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.935+0000 7fd9d817e740 1 -- 172.21.15.2:0/450834276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b69d8b0 msgr2=0x56134b690590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.935+0000 7fd9d817e740 1 --2- 172.21.15.2:0/450834276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b69d8b0 0x56134b690590 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009ca0 tx=0x7fd9c8044000 comp rx=0 tx=0).stop 2024-03-15T13:00:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.935+0000 7fd9d817e740 1 -- 172.21.15.2:0/450834276 shutdown_connections 2024-03-15T13:00:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.935+0000 7fd9d817e740 1 --2- 172.21.15.2:0/450834276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b69d8b0 0x56134b690590 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9d817e740 1 --2- 172.21.15.2:0/450834276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b694d30 0x56134b69b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9d817e740 1 --2- 172.21.15.2:0/450834276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b694380 0x56134b69d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9d817e740 1 -- 172.21.15.2:0/450834276 >> 172.21.15.2:0/450834276 conn(0x56134b68e840 msgr2=0x56134b68fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9d817e740 1 -- 172.21.15.2:0/450834276 shutdown_connections 2024-03-15T13:00:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9d817e740 1 -- 172.21.15.2:0/450834276 wait complete. 2024-03-15T13:00:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9d817e740 1 -- start start 2024-03-15T13:00:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9d817e740 1 Processor -- start 2024-03-15T13:00:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9d817e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694380 0x56134b701770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9d817e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b694d30 0x56134b701cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9d817e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b69d8b0 0x56134b697900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9d817e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134b697fd0 con 0x56134b694380 2024-03-15T13:00:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9d817e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134b698140 con 0x56134b694d30 2024-03-15T13:00:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9d817e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134b698300 con 0x56134b69d8b0 2024-03-15T13:00:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9d817e740 10 client.0 _ll_register_callbacks cb 0x7fd9c0009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9ceffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694380 0x56134b701770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9ceffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694380 0x56134b701770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32980/0 (socket says 172.21.15.2:32980) 2024-03-15T13:00:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9ceffd640 1 -- 172.21.15.2:0/1755549237 learned_addr learned my addr 172.21.15.2:0/1755549237 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9c67fc640 1 --2- 172.21.15.2:0/1755549237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b694d30 0x56134b701cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.936+0000 7fd9cf7fe640 1 --2- 172.21.15.2:0/1755549237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b69d8b0 0x56134b697900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.937+0000 7fd9ceffd640 1 -- 172.21.15.2:0/1755549237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b69d8b0 msgr2=0x56134b697900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.937+0000 7fd9ceffd640 1 --2- 172.21.15.2:0/1755549237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b69d8b0 0x56134b697900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.937+0000 7fd9ceffd640 1 -- 172.21.15.2:0/1755549237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b694d30 msgr2=0x56134b701cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.937+0000 7fd9ceffd640 1 --2- 172.21.15.2:0/1755549237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b694d30 0x56134b701cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.937+0000 7fd9ceffd640 1 -- 172.21.15.2:0/1755549237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56134b5fa320 con 0x56134b694380 2024-03-15T13:00:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.937+0000 7fd9c67fc640 1 --2- 172.21.15.2:0/1755549237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b694d30 0x56134b701cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.937+0000 7fd9ceffd640 1 --2- 172.21.15.2:0/1755549237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694380 0x56134b701770 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x56134b6b07a0 tx=0x7fd9c0012c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16686cf969f38542 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.937+0000 7fd9b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.937+0000 7fd9b67fc640 1 -- 172.21.15.2:0/1755549237 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0017040 con 0x56134b694380 2024-03-15T13:00:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.937+0000 7fd9b67fc640 1 -- 172.21.15.2:0/1755549237 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fd9c00220b0 con 0x56134b694380 2024-03-15T13:00:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.937+0000 7fd9d817e740 1 -- 172.21.15.2:0/1755549237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56134b621900 con 0x56134b694380 2024-03-15T13:00:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.937+0000 7fd9b67fc640 1 -- 172.21.15.2:0/1755549237 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9c00122d0 con 0x56134b694380 2024-03-15T13:00:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.937+0000 7fd9d817e740 10 client.16407 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.937+0000 7fd9d817e740 1 -- 172.21.15.2:0/1755549237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fd9d010d320 con 0x56134b694380 2024-03-15T13:00:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.938+0000 7fd9b67fc640 1 -- 172.21.15.2:0/1755549237 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x56134b621900 con 0x56134b694380 2024-03-15T13:00:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.938+0000 7fd9b67fc640 1 -- 172.21.15.2:0/1755549237 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fd9c0020400 con 0x56134b694380 2024-03-15T13:00:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.938+0000 7fd9d817e740 10 client.16407 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.938+0000 7fd9d817e740 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.938+0000 7fd9d817e740 20 client.16407 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.938+0000 7fd9d817e740 1 --2- 172.21.15.2:0/1755549237 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56134b70a8a0 0x56134b70ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.938+0000 7fd9d817e740 4 client.16407 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:00:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.938+0000 7fd9d817e740 1 -- 172.21.15.2:0/1755549237 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56134b2fb860 con 0x56134b70a8a0 2024-03-15T13:00:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.938+0000 7fd9c67fc640 1 --2- 172.21.15.2:0/1755549237 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56134b70a8a0 0x56134b70ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.939+0000 7fd9c67fc640 1 --2- 172.21.15.2:0/1755549237 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56134b70a8a0 0x56134b70ccc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae9fe04a49ffb559 server_cookie=922c1724d099012d in_seq=0 out_seq=0 2024-03-15T13:00:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.939+0000 7fd9b67fc640 10 client.16407 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.940+0000 7fd9b67fc640 1 -- 172.21.15.2:0/1755549237 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56134b2fb860 con 0x56134b70a8a0 2024-03-15T13:00:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.940+0000 7fd9b67fc640 10 client.16407 handle_command_reply: tid=0 2024-03-15T13:00:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.940+0000 7fd9d817e740 1 client.16407 shutdown 2024-03-15T13:00:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.940+0000 7fd9d817e740 1 -- 172.21.15.2:0/1755549237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694380 msgr2=0x56134b701770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.940+0000 7fd9d817e740 1 --2- 172.21.15.2:0/1755549237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694380 0x56134b701770 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x56134b6b07a0 tx=0x7fd9c0012c20 comp rx=0 tx=0).stop 2024-03-15T13:00:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.940+0000 7fd9d817e740 1 -- 172.21.15.2:0/1755549237 shutdown_connections 2024-03-15T13:00:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.940+0000 7fd9d817e740 1 --2- 172.21.15.2:0/1755549237 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56134b70a8a0 0x56134b70ccc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.940+0000 7fd9d817e740 1 --2- 172.21.15.2:0/1755549237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56134b69d8b0 0x56134b697900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.940+0000 7fd9d817e740 1 --2- 172.21.15.2:0/1755549237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56134b694d30 0x56134b701cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.940+0000 7fd9b67fc640 0 client.16407 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.940+0000 7fd9d817e740 1 --2- 172.21.15.2:0/1755549237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56134b694380 0x56134b701770 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.940+0000 7fd9d817e740 1 -- 172.21.15.2:0/1755549237 >> 172.21.15.2:0/1755549237 conn(0x56134b68e840 msgr2=0x56134b690710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.940+0000 7fd9d817e740 1 -- 172.21.15.2:0/1755549237 shutdown_connections 2024-03-15T13:00:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.940+0000 7fd9d817e740 1 -- 172.21.15.2:0/1755549237 wait complete. 2024-03-15T13:00:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.940+0000 7fd9d817e740 20 client.16407 trim_cache size 0 max 16384 2024-03-15T13:00:56.940 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:00:56.940 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:00:56.940 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "f57cac92-747b-4922-a9c1-fd28695999a2", 2024-03-15T13:00:56.940 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:00:56.940 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:00:56.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.941+0000 7fd9d54c3640 1 -- 172.21.15.2:0/673336152 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd9bc0a1ba0 msgr2=0x7fd9bc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:56.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.941+0000 7fd9d54c3640 1 --2- 172.21.15.2:0/673336152 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd9bc0a1ba0 0x7fd9bc0a4060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8004af0 tx=0x7fd9c8005c50 comp rx=0 tx=0).stop 2024-03-15T13:00:56.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.941+0000 7fd9d54c3640 1 -- 172.21.15.2:0/673336152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d0100750 msgr2=0x7fd9d0198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:56.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.941+0000 7fd9d54c3640 1 --2- 172.21.15.2:0/673336152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d0100750 0x7fd9d0198e10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800e9f0 tx=0x7fd9b800eeb0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.941+0000 7fd9d54c3640 1 -- 172.21.15.2:0/673336152 shutdown_connections 2024-03-15T13:00:56.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.941+0000 7fd9d54c3640 1 --2- 172.21.15.2:0/673336152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9d0104b20 0x7fd9d01a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.941+0000 7fd9d54c3640 1 --2- 172.21.15.2:0/673336152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9d0100750 0x7fd9d0198e10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.941+0000 7fd9d54c3640 1 --2- 172.21.15.2:0/673336152 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd9bc0a1ba0 0x7fd9bc0a4060 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.941+0000 7fd9d54c3640 1 --2- 172.21.15.2:0/673336152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9d00ffdb0 0x7fd9d01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:56.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.941+0000 7fd9d54c3640 1 -- 172.21.15.2:0/673336152 >> 172.21.15.2:0/673336152 conn(0x7fd9d0075730 msgr2=0x7fd9d0103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:56.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.941+0000 7fd9d54c3640 1 -- 172.21.15.2:0/673336152 shutdown_connections 2024-03-15T13:00:56.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:56.941+0000 7fd9d54c3640 1 -- 172.21.15.2:0/673336152 wait complete. 2024-03-15T13:00:56.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:00:56.942 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:00:57.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.081+0000 7fc2f87ac640 1 -- 172.21.15.2:0/1731401893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2f0103030 msgr2=0x7fc2f010b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:57.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.081+0000 7fc2f87ac640 1 --2- 172.21.15.2:0/1731401893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2f0103030 0x7fc2f010b510 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009900 tx=0x7fc2e002bcd0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.081+0000 7fc2f87ac640 1 -- 172.21.15.2:0/1731401893 shutdown_connections 2024-03-15T13:00:57.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.081+0000 7fc2f87ac640 1 --2- 172.21.15.2:0/1731401893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2f010ba50 0x7fc2f010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.081+0000 7fc2f87ac640 1 --2- 172.21.15.2:0/1731401893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2f0103030 0x7fc2f010b510 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.081+0000 7fc2f87ac640 1 --2- 172.21.15.2:0/1731401893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2f0102710 0x7fc2f0102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.081+0000 7fc2f87ac640 1 -- 172.21.15.2:0/1731401893 >> 172.21.15.2:0/1731401893 conn(0x7fc2f00775a0 msgr2=0x7fc2f00779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:57.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.081+0000 7fc2f87ac640 1 -- 172.21.15.2:0/1731401893 shutdown_connections 2024-03-15T13:00:57.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.081+0000 7fc2f87ac640 1 -- 172.21.15.2:0/1731401893 wait complete. 2024-03-15T13:00:57.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.082+0000 7fc2f87ac640 1 Processor -- start 2024-03-15T13:00:57.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.082+0000 7fc2f87ac640 1 -- start start 2024-03-15T13:00:57.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.082+0000 7fc2f87ac640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2f0102710 0x7fc2f019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.082+0000 7fc2f87ac640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2f0103030 0x7fc2f019dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.082+0000 7fc2f87ac640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2f010ba50 0x7fc2f01978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.082+0000 7fc2f87ac640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f0197f10 con 0x7fc2f0102710 2024-03-15T13:00:57.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.082+0000 7fc2f87ac640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f0198080 con 0x7fc2f0103030 2024-03-15T13:00:57.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.082+0000 7fc2f87ac640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f0198240 con 0x7fc2f010ba50 2024-03-15T13:00:57.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.082+0000 7fc2f6548640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2f0102710 0x7fc2f019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.082+0000 7fc2f5d47640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2f0103030 0x7fc2f019dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.082+0000 7fc2f6548640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2f0102710 0x7fc2f019d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32988/0 (socket says 172.21.15.2:32988) 2024-03-15T13:00:57.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.082+0000 7fc2f6548640 1 -- 172.21.15.2:0/1499254551 learned_addr learned my addr 172.21.15.2:0/1499254551 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:57.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.082+0000 7fc2f6d49640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2f010ba50 0x7fc2f01978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.083+0000 7fc2f6548640 1 -- 172.21.15.2:0/1499254551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2f010ba50 msgr2=0x7fc2f01978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:57.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.083+0000 7fc2f6548640 1 --2- 172.21.15.2:0/1499254551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2f010ba50 0x7fc2f01978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.083+0000 7fc2f6548640 1 -- 172.21.15.2:0/1499254551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2f0103030 msgr2=0x7fc2f019dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:57.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.083+0000 7fc2f6548640 1 --2- 172.21.15.2:0/1499254551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2f0103030 0x7fc2f019dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.083+0000 7fc2f6548640 1 -- 172.21.15.2:0/1499254551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e0009560 con 0x7fc2f0102710 2024-03-15T13:00:57.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.083+0000 7fc2f6548640 1 --2- 172.21.15.2:0/1499254551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2f0102710 0x7fc2f019d710 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00d950 tx=0x7fc2ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:57.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.083+0000 7fc2e77fe640 1 -- 172.21.15.2:0/1499254551 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec009a30 con 0x7fc2f0102710 2024-03-15T13:00:57.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.083+0000 7fc2e77fe640 1 -- 172.21.15.2:0/1499254551 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc2ec009b90 con 0x7fc2f0102710 2024-03-15T13:00:57.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.083+0000 7fc2f87ac640 1 -- 172.21.15.2:0/1499254551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f0198520 con 0x7fc2f0102710 2024-03-15T13:00:57.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.083+0000 7fc2e77fe640 1 -- 172.21.15.2:0/1499254551 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0047a0 con 0x7fc2f0102710 2024-03-15T13:00:57.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.083+0000 7fc2f87ac640 1 -- 172.21.15.2:0/1499254551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f01a47b0 con 0x7fc2f0102710 2024-03-15T13:00:57.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.085+0000 7fc2e77fe640 1 -- 172.21.15.2:0/1499254551 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fc2ec009d00 con 0x7fc2f0102710 2024-03-15T13:00:57.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.086+0000 7fc2fb389740 1 Processor -- start 2024-03-15T13:00:57.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.086+0000 7fc2fb389740 1 -- start start 2024-03-15T13:00:57.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.086+0000 7fc2e77fe640 1 --2- 172.21.15.2:0/1499254551 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc2dc0a1ba0 0x7fc2dc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.086+0000 7fc2e77fe640 1 -- 172.21.15.2:0/1499254551 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc2ec0cd460 con 0x7fc2f0102710 2024-03-15T13:00:57.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.086+0000 7fc2f6d49640 1 --2- 172.21.15.2:0/1499254551 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc2dc0a1ba0 0x7fc2dc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.086+0000 7fc2fb389740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7a2b0 0x564501b7a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.086+0000 7fc2fb389740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7abd0 0x564501b83270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.086+0000 7fc2fb389740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 0x564501b85ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.086+0000 7fc2fb389740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564501ae9280 con 0x564501b837b0 2024-03-15T13:00:57.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.086+0000 7fc2fb389740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564501ae67b0 con 0x564501b7abd0 2024-03-15T13:00:57.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.086+0000 7fc2fb389740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564501ae1f90 con 0x564501b7a2b0 2024-03-15T13:00:57.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.087+0000 7fc2f6548640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7a2b0 0x564501b7a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.087+0000 7fc2f6d49640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 0x564501b85ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.087+0000 7fc2f6548640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7a2b0 0x564501b7a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:35122/0 (socket says 172.21.15.2:35122) 2024-03-15T13:00:57.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.087+0000 7fc2f6548640 1 -- 172.21.15.2:0/3179485514 learned_addr learned my addr 172.21.15.2:0/3179485514 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:57.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.087+0000 7fc2f5d47640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7abd0 0x564501b83270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.087+0000 7fc2f6548640 1 -- 172.21.15.2:0/3179485514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7abd0 msgr2=0x564501b83270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:57.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.087+0000 7fc2f6548640 1 --2- 172.21.15.2:0/3179485514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7abd0 0x564501b83270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.087+0000 7fc2f6548640 1 -- 172.21.15.2:0/3179485514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 msgr2=0x564501b85ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:57.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.087+0000 7fc2f6d49640 1 --2- 172.21.15.2:0/1499254551 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc2dc0a1ba0 0x7fc2dc0a4060 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8005fd0 tx=0x7fc2e8014b70 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:57.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.087+0000 7fc2f6548640 1 --2- 172.21.15.2:0/3179485514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 0x564501b85ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.087+0000 7fc2f6548640 1 -- 172.21.15.2:0/3179485514 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564501aa0570 con 0x564501b7a2b0 2024-03-15T13:00:57.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.087+0000 7fc2f6548640 1 --2- 172.21.15.2:0/3179485514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7a2b0 0x564501b7a690 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0967b0 tx=0x7fc2ec09a4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6441d22a2993ed39 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:57.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.087+0000 7fc2e4ff9640 1 -- 172.21.15.2:0/3179485514 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0a2580 con 0x564501b7a2b0 2024-03-15T13:00:57.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.087+0000 7fc2e4ff9640 1 -- 172.21.15.2:0/3179485514 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc2ec0cd170 con 0x564501b7a2b0 2024-03-15T13:00:57.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.087+0000 7fc2e4ff9640 1 -- 172.21.15.2:0/3179485514 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec095b30 con 0x564501b7a2b0 2024-03-15T13:00:57.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.088+0000 7fc2fb389740 1 -- 172.21.15.2:0/3179485514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7a2b0 msgr2=0x564501b7a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:57.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.088+0000 7fc2fb389740 1 --2- 172.21.15.2:0/3179485514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7a2b0 0x564501b7a690 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0967b0 tx=0x7fc2ec09a4e0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.088+0000 7fc2fb389740 1 -- 172.21.15.2:0/3179485514 shutdown_connections 2024-03-15T13:00:57.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.088+0000 7fc2fb389740 1 --2- 172.21.15.2:0/3179485514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 0x564501b85ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.088+0000 7fc2fb389740 1 --2- 172.21.15.2:0/3179485514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7abd0 0x564501b83270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.088+0000 7fc2fb389740 1 --2- 172.21.15.2:0/3179485514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7a2b0 0x564501b7a690 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.088+0000 7fc2fb389740 1 -- 172.21.15.2:0/3179485514 >> 172.21.15.2:0/3179485514 conn(0x564501b74990 msgr2=0x564501b74d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:57.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.088+0000 7fc2fb389740 1 -- 172.21.15.2:0/3179485514 shutdown_connections 2024-03-15T13:00:57.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.088+0000 7fc2fb389740 1 -- 172.21.15.2:0/3179485514 wait complete. 2024-03-15T13:00:57.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.089+0000 7fc2fb389740 1 -- start start 2024-03-15T13:00:57.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.089+0000 7fc2fb389740 1 Processor -- start 2024-03-15T13:00:57.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.089+0000 7fc2fb389740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7a2b0 0x564501b83270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.089+0000 7fc2fb389740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7abd0 0x564501b80600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.089+0000 7fc2fb389740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 0x564501b80d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.089+0000 7fc2fb389740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645017b5430 con 0x564501b837b0 2024-03-15T13:00:57.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.089+0000 7fc2fb389740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564501771e00 con 0x564501b7a2b0 2024-03-15T13:00:57.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.089+0000 7fc2fb389740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564501a66fd0 con 0x564501b7abd0 2024-03-15T13:00:57.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.090+0000 7fc2fb389740 10 client.0 _ll_register_callbacks cb 0x564501abcfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:57.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.090+0000 7fc2f6548640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7a2b0 0x564501b83270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.090+0000 7fc2f6548640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7a2b0 0x564501b83270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:51136/0 (socket says 172.21.15.2:51136) 2024-03-15T13:00:57.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.090+0000 7fc2f6548640 1 -- 172.21.15.2:0/1903381330 learned_addr learned my addr 172.21.15.2:0/1903381330 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:57.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.090+0000 7fc2f6d49640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 0x564501b80d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.090+0000 7fc2f5d47640 1 --2- 172.21.15.2:0/1903381330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7abd0 0x564501b80600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.090+0000 7fc2f6548640 1 -- 172.21.15.2:0/1903381330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7abd0 msgr2=0x564501b80600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:57.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.090+0000 7fc2f6548640 1 --2- 172.21.15.2:0/1903381330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7abd0 0x564501b80600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.090+0000 7fc2f6548640 1 -- 172.21.15.2:0/1903381330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 msgr2=0x564501b80d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.090+0000 7fc2f6548640 1 --2- 172.21.15.2:0/1903381330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 0x564501b80d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.090+0000 7fc2f6548640 1 -- 172.21.15.2:0/1903381330 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f01a47b0 con 0x564501b7a2b0 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.090+0000 7fc2f6548640 1 --2- 172.21.15.2:0/1903381330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7a2b0 0x564501b83270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0cd7b0 tx=0x7fc2ec0966f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29037728a0fe2e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.090+0000 7fc2d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.091+0000 7fc2d9ffb640 1 -- 172.21.15.2:0/1903381330 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0ca870 con 0x564501b7a2b0 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.091+0000 7fc2d9ffb640 1 -- 172.21.15.2:0/1903381330 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc2ec078e40 con 0x564501b7a2b0 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.091+0000 7fc2fb389740 1 -- 172.21.15.2:0/1903381330 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645017e17e0 con 0x564501b7a2b0 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.091+0000 7fc2d9ffb640 1 -- 172.21.15.2:0/1903381330 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2ec0a2580 con 0x564501b7a2b0 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.091+0000 7fc2fb389740 10 client.26282 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.091+0000 7fc2fb389740 1 -- 172.21.15.2:0/1903381330 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564501ae0610 con 0x564501b7a2b0 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.091+0000 7fc2d9ffb640 1 -- 172.21.15.2:0/1903381330 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5645017e17e0 con 0x564501b7a2b0 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.092+0000 7fc2d9ffb640 1 -- 172.21.15.2:0/1903381330 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fc2ec095b30 con 0x564501b7a2b0 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.092+0000 7fc2fb389740 10 client.26282 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.092+0000 7fc2fb389740 10 client.26282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:57.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.092+0000 7fc2fb389740 20 client.26282 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:57.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.092+0000 7fc2fb389740 1 --2- 172.21.15.2:0/1903381330 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564501b93570 0x564501b95960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.092+0000 7fc2fb389740 4 client.26282 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:00:57.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.092+0000 7fc2fb389740 1 -- 172.21.15.2:0/1903381330 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56450181e720 con 0x564501b93570 2024-03-15T13:00:57.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.092+0000 7fc2f5d47640 1 --2- 172.21.15.2:0/1903381330 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564501b93570 0x564501b95960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.092+0000 7fc2d9ffb640 10 client.26282 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:57.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.092+0000 7fc2f5d47640 1 --2- 172.21.15.2:0/1903381330 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564501b93570 0x564501b95960 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2ee01ccc3cf527 server_cookie=7355291c6aa93649 in_seq=0 out_seq=0 2024-03-15T13:00:57.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.094+0000 7fc2d9ffb640 1 -- 172.21.15.2:0/1903381330 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56450181e720 con 0x564501b93570 2024-03-15T13:00:57.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.094+0000 7fc2d9ffb640 10 client.26282 handle_command_reply: tid=0 2024-03-15T13:00:57.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.094+0000 7fc2fb389740 1 client.26282 shutdown 2024-03-15T13:00:57.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.094+0000 7fc2fb389740 1 -- 172.21.15.2:0/1903381330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7a2b0 msgr2=0x564501b83270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:57.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.094+0000 7fc2fb389740 1 --2- 172.21.15.2:0/1903381330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7a2b0 0x564501b83270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0cd7b0 tx=0x7fc2ec0966f0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.094+0000 7fc2fb389740 1 -- 172.21.15.2:0/1903381330 shutdown_connections 2024-03-15T13:00:57.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.094+0000 7fc2fb389740 1 --2- 172.21.15.2:0/1903381330 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564501b93570 0x564501b95960 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.094+0000 7fc2fb389740 1 --2- 172.21.15.2:0/1903381330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 0x564501b80d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.095+0000 7fc2fb389740 1 --2- 172.21.15.2:0/1903381330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7abd0 0x564501b80600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.095+0000 7fc2fb389740 1 --2- 172.21.15.2:0/1903381330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7a2b0 0x564501b83270 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.095+0000 7fc2fb389740 1 -- 172.21.15.2:0/1903381330 >> 172.21.15.2:0/1903381330 conn(0x564501b74990 msgr2=0x564501b76730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:57.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.095+0000 7fc2d9ffb640 0 client.26282 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:57.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.095+0000 7fc2fb389740 1 -- 172.21.15.2:0/1903381330 shutdown_connections 2024-03-15T13:00:57.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.095+0000 7fc2fb389740 1 -- 172.21.15.2:0/1903381330 wait complete. 2024-03-15T13:00:57.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.095+0000 7fc2fb389740 20 client.26282 trim_cache size 0 max 16384 2024-03-15T13:00:57.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2fb389740 1 Processor -- start 2024-03-15T13:00:57.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2fb389740 1 -- start start 2024-03-15T13:00:57.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2fb389740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7a2b0 0x564501b7f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2fb389740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b7abd0 0x564501b82d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2fb389740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b837b0 0x564501b83270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2fb389740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564501b79f10 con 0x564501b7abd0 2024-03-15T13:00:57.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2fb389740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564501b70320 con 0x564501b837b0 2024-03-15T13:00:57.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2fb389740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564501b759e0 con 0x564501b7a2b0 2024-03-15T13:00:57.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2f5d47640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b7abd0 0x564501b82d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2f6548640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7a2b0 0x564501b7f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2f6548640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7a2b0 0x564501b7f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:35152/0 (socket says 172.21.15.2:35152) 2024-03-15T13:00:57.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2f6548640 1 -- 172.21.15.2:0/1724434355 learned_addr learned my addr 172.21.15.2:0/1724434355 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:57.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2f6d49640 1 --2- 172.21.15.2:0/1724434355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b837b0 0x564501b83270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2f6548640 1 -- 172.21.15.2:0/1724434355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b837b0 msgr2=0x564501b83270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:57.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2f6548640 1 --2- 172.21.15.2:0/1724434355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b837b0 0x564501b83270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2f6548640 1 -- 172.21.15.2:0/1724434355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b7abd0 msgr2=0x564501b82d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:57.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2f6548640 1 --2- 172.21.15.2:0/1724434355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b7abd0 0x564501b82d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2f6548640 1 -- 172.21.15.2:0/1724434355 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564501a7cac0 con 0x564501b7a2b0 2024-03-15T13:00:57.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.115+0000 7fc2f5d47640 1 --2- 172.21.15.2:0/1724434355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b7abd0 0x564501b82d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:57.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.116+0000 7fc2f6548640 1 --2- 172.21.15.2:0/1724434355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7a2b0 0x564501b7f5e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec096240 tx=0x7fc2ec09a4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7124ef1a1b0927d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:57.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.116+0000 7fc2dbfff640 1 -- 172.21.15.2:0/1724434355 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec098dc0 con 0x564501b7a2b0 2024-03-15T13:00:57.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.116+0000 7fc2dbfff640 1 -- 172.21.15.2:0/1724434355 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc2ec096080 con 0x564501b7a2b0 2024-03-15T13:00:57.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.116+0000 7fc2dbfff640 1 -- 172.21.15.2:0/1724434355 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec095b30 con 0x564501b7a2b0 2024-03-15T13:00:57.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 -- 172.21.15.2:0/1724434355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7a2b0 msgr2=0x564501b7f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:57.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 --2- 172.21.15.2:0/1724434355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7a2b0 0x564501b7f5e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec096240 tx=0x7fc2ec09a4e0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 -- 172.21.15.2:0/1724434355 shutdown_connections 2024-03-15T13:00:57.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 --2- 172.21.15.2:0/1724434355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b837b0 0x564501b83270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 --2- 172.21.15.2:0/1724434355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b7abd0 0x564501b82d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 --2- 172.21.15.2:0/1724434355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7a2b0 0x564501b7f5e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 -- 172.21.15.2:0/1724434355 >> 172.21.15.2:0/1724434355 conn(0x564501b74990 msgr2=0x564501b7eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:57.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 -- 172.21.15.2:0/1724434355 shutdown_connections 2024-03-15T13:00:57.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 -- 172.21.15.2:0/1724434355 wait complete. 2024-03-15T13:00:57.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 -- start start 2024-03-15T13:00:57.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 Processor -- start 2024-03-15T13:00:57.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7a2b0 0x564501b84250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7abd0 0x564501b84790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 0x564501b8a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564501b84f10 con 0x564501b837b0 2024-03-15T13:00:57.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564501b85080 con 0x564501b7a2b0 2024-03-15T13:00:57.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564501b8ac20 con 0x564501b7abd0 2024-03-15T13:00:57.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.117+0000 7fc2fb389740 10 client.0 _ll_register_callbacks cb 0x7fc2e8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:00:57.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2f6d49640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 0x564501b8a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2f5d47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7abd0 0x564501b84790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2f6548640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7a2b0 0x564501b84250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2f6d49640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 0x564501b8a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33030/0 (socket says 172.21.15.2:33030) 2024-03-15T13:00:57.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2f6548640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7a2b0 0x564501b84250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:51154/0 (socket says 172.21.15.2:51154) 2024-03-15T13:00:57.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2f5d47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7abd0 0x564501b84790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:35166/0 (socket says 172.21.15.2:35166) 2024-03-15T13:00:57.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2f6d49640 1 -- 172.21.15.2:0/4037972290 learned_addr learned my addr 172.21.15.2:0/4037972290 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:00:57.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2f6d49640 1 -- 172.21.15.2:0/4037972290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7abd0 msgr2=0x564501b84790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:57.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2f6d49640 1 --2- 172.21.15.2:0/4037972290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7abd0 0x564501b84790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2f6d49640 1 -- 172.21.15.2:0/4037972290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7a2b0 msgr2=0x564501b84250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:00:57.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2f6d49640 1 --2- 172.21.15.2:0/4037972290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7a2b0 0x564501b84250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2f6d49640 1 -- 172.21.15.2:0/4037972290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f010eb70 con 0x564501b837b0 2024-03-15T13:00:57.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2f5d47640 1 --2- 172.21.15.2:0/4037972290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7abd0 0x564501b84790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:00:57.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2f6d49640 1 --2- 172.21.15.2:0/4037972290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 0x564501b8a6e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8012bd0 tx=0x7fc2e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cf3ae68f0e64e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:00:57.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:00:57.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2d9ffb640 1 -- 172.21.15.2:0/4037972290 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8027070 con 0x564501b837b0 2024-03-15T13:00:57.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2d9ffb640 1 -- 172.21.15.2:0/4037972290 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc2e8022cb0 con 0x564501b837b0 2024-03-15T13:00:57.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2fb389740 1 -- 172.21.15.2:0/4037972290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2ec0cd170 con 0x564501b837b0 2024-03-15T13:00:57.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2d9ffb640 1 -- 172.21.15.2:0/4037972290 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2e8022e70 con 0x564501b837b0 2024-03-15T13:00:57.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2fb389740 10 client.16425 fetch_fsmap learned FSMap version 33 2024-03-15T13:00:57.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.118+0000 7fc2fb389740 1 -- 172.21.15.2:0/4037972290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564501ae0610 con 0x564501b837b0 2024-03-15T13:00:57.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.119+0000 7fc2d9ffb640 1 -- 172.21.15.2:0/4037972290 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc2ec0cd170 con 0x564501b837b0 2024-03-15T13:00:57.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.119+0000 7fc2d9ffb640 1 -- 172.21.15.2:0/4037972290 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7966+0+0 (secure 0 0 0) 0x7fc2e80045b0 con 0x564501b837b0 2024-03-15T13:00:57.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.119+0000 7fc2fb389740 10 client.16425 fetch_fsmap finished waiting for FSMap version 33 2024-03-15T13:00:57.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.119+0000 7fc2fb389740 10 client.16425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:00:57.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.119+0000 7fc2fb389740 20 client.16425 populate_metadata read hostname 'smithi002' 2024-03-15T13:00:57.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.119+0000 7fc2fb389740 1 --2- 172.21.15.2:0/4037972290 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564501befc10 0x564501bf2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:00:57.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.119+0000 7fc2fb389740 4 client.16425 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:00:57.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.119+0000 7fc2fb389740 1 -- 172.21.15.2:0/4037972290 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56450186d890 con 0x564501befc10 2024-03-15T13:00:57.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.119+0000 7fc2f6548640 1 --2- 172.21.15.2:0/4037972290 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564501befc10 0x564501bf2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:00:57.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.120+0000 7fc2d9ffb640 10 client.16425 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:57.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.120+0000 7fc2f6548640 1 --2- 172.21.15.2:0/4037972290 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564501befc10 0x564501bf2030 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ea36a9f7b08d37 server_cookie=59b1e41b7527650d in_seq=0 out_seq=0 2024-03-15T13:00:57.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.123+0000 7fc2d9ffb640 1 -- 172.21.15.2:0/4037972290 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x56450186d890 con 0x564501befc10 2024-03-15T13:00:57.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.123+0000 7fc2d9ffb640 10 client.16425 handle_command_reply: tid=0 2024-03-15T13:00:57.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.123+0000 7fc2fb389740 1 client.16425 shutdown 2024-03-15T13:00:57.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.123+0000 7fc2fb389740 1 -- 172.21.15.2:0/4037972290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 msgr2=0x564501b8a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:57.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.123+0000 7fc2fb389740 1 --2- 172.21.15.2:0/4037972290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 0x564501b8a6e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8012bd0 tx=0x7fc2e8004ab0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.123+0000 7fc2fb389740 1 -- 172.21.15.2:0/4037972290 shutdown_connections 2024-03-15T13:00:57.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.123+0000 7fc2fb389740 1 --2- 172.21.15.2:0/4037972290 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564501befc10 0x564501bf2030 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.123+0000 7fc2fb389740 1 --2- 172.21.15.2:0/4037972290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564501b837b0 0x564501b8a6e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.123+0000 7fc2fb389740 1 --2- 172.21.15.2:0/4037972290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501b7abd0 0x564501b84790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.123+0000 7fc2fb389740 1 --2- 172.21.15.2:0/4037972290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564501b7a2b0 0x564501b84250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.123+0000 7fc2d9ffb640 0 client.16425 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:00:57.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.123+0000 7fc2fb389740 1 -- 172.21.15.2:0/4037972290 >> 172.21.15.2:0/4037972290 conn(0x564501b74990 msgr2=0x564501b75ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:57.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.123+0000 7fc2fb389740 1 -- 172.21.15.2:0/4037972290 shutdown_connections 2024-03-15T13:00:57.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.123+0000 7fc2fb389740 1 -- 172.21.15.2:0/4037972290 wait complete. 2024-03-15T13:00:57.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.124+0000 7fc2fb389740 20 client.16425 trim_cache size 0 max 16384 2024-03-15T13:00:57.113 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:00:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (259 inodes in the stack)", 2024-03-15T13:00:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:00:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "f57cac92-747b-4922-a9c1-fd28695999a2": { 2024-03-15T13:00:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:00:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "f57cac92-747b-4922-a9c1-fd28695999a2", 2024-03-15T13:00:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:00:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:00:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:00:57.114 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:00:57.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.124+0000 7fc2f87ac640 1 -- 172.21.15.2:0/1499254551 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc2dc0a1ba0 msgr2=0x7fc2dc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.124+0000 7fc2f87ac640 1 --2- 172.21.15.2:0/1499254551 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc2dc0a1ba0 0x7fc2dc0a4060 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8005fd0 tx=0x7fc2e8014b70 comp rx=0 tx=0).stop 2024-03-15T13:00:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.124+0000 7fc2f87ac640 1 -- 172.21.15.2:0/1499254551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2f0102710 msgr2=0x7fc2f019d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:00:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.124+0000 7fc2f87ac640 1 --2- 172.21.15.2:0/1499254551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2f0102710 0x7fc2f019d710 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00d950 tx=0x7fc2ec00de10 comp rx=0 tx=0).stop 2024-03-15T13:00:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.124+0000 7fc2f87ac640 1 -- 172.21.15.2:0/1499254551 shutdown_connections 2024-03-15T13:00:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.124+0000 7fc2f87ac640 1 --2- 172.21.15.2:0/1499254551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2f010ba50 0x7fc2f01978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.124+0000 7fc2f87ac640 1 --2- 172.21.15.2:0/1499254551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2f0103030 0x7fc2f019dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.124+0000 7fc2f87ac640 1 --2- 172.21.15.2:0/1499254551 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc2dc0a1ba0 0x7fc2dc0a4060 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.124+0000 7fc2f87ac640 1 --2- 172.21.15.2:0/1499254551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2f0102710 0x7fc2f019d710 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:00:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.124+0000 7fc2f87ac640 1 -- 172.21.15.2:0/1499254551 >> 172.21.15.2:0/1499254551 conn(0x7fc2f00775a0 msgr2=0x7fc2f00fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:00:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.124+0000 7fc2f87ac640 1 -- 172.21.15.2:0/1499254551 shutdown_connections 2024-03-15T13:00:57.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:00:57.124+0000 7fc2f87ac640 1 -- 172.21.15.2:0/1499254551 wait complete. 2024-03-15T13:00:57.120 INFO:tasks.cephfs.filesystem:scrub status for tag:f57cac92-747b-4922-a9c1-fd28695999a2 - {'path': '/', 'tag': 'f57cac92-747b-4922-a9c1-fd28695999a2', 'options': 'recursive,force'} 2024-03-15T13:00:58.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:57 smithi002 ceph-mon[32060]: pgmap v64: 129 pgs: 129 active+clean; 149 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 3.0 MiB/s wr, 255 op/s 2024-03-15T13:00:58.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:57 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:00:58.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:57 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:58.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:57 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:58.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:57 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:58.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:57 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:58.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:57 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:58.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:57 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:58.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:57 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:58.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:57 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:58.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:57 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:58.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:57 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:57 smithi184 ceph-mon[35993]: pgmap v64: 129 pgs: 129 active+clean; 149 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 3.0 MiB/s wr, 255 op/s 2024-03-15T13:00:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:57 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:00:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:57 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:57 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:57 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:57 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:57 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:57 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:57 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:57 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:57 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:57 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:58.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:57 smithi149 ceph-mon[35927]: pgmap v64: 129 pgs: 129 active+clean; 149 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 3.0 MiB/s wr, 255 op/s 2024-03-15T13:00:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:57 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:00:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:57 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:57 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:57 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:57 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:57 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:57 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:57 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:57 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:57 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:57 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:00:59.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:58 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:00:59.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:58 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:00:59.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:00:59.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:58 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: pgmap v65: 129 pgs: 129 active+clean; 149 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.7 MiB/s wr, 228 op/s 2024-03-15T13:01:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:00:59 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: pgmap v65: 129 pgs: 129 active+clean; 149 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.7 MiB/s wr, 228 op/s 2024-03-15T13:01:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:00:59 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: pgmap v65: 129 pgs: 129 active+clean; 149 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.7 MiB/s wr, 228 op/s 2024-03-15T13:01:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:00.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:00.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:00.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:00.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:00.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:00.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:00.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:00:59 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:01.038 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:00 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:01:01.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:00 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:01:01.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:00 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:01:02.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:01 smithi002 ceph-mon[32060]: pgmap v66: 129 pgs: 129 active+clean; 154 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 345 op/s 2024-03-15T13:01:02.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:01 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:01:02.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:01 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:01:02.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:01 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:01:02.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:01 smithi184 ceph-mon[35993]: pgmap v66: 129 pgs: 129 active+clean; 154 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 345 op/s 2024-03-15T13:01:02.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:01 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:01:02.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:01 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:01:02.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:01 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:01:02.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:01 smithi149 ceph-mon[35927]: pgmap v66: 129 pgs: 129 active+clean; 154 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 345 op/s 2024-03-15T13:01:02.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:01 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:01:02.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:01 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:01:02.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:01 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:01:03.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:02 smithi149 ceph-mon[35927]: pgmap v67: 129 pgs: 129 active+clean; 154 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 991 KiB/s wr, 237 op/s 2024-03-15T13:01:03.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:02 smithi002 ceph-mon[32060]: pgmap v67: 129 pgs: 129 active+clean; 154 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 991 KiB/s wr, 237 op/s 2024-03-15T13:01:03.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:02 smithi184 ceph-mon[35993]: pgmap v67: 129 pgs: 129 active+clean; 154 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 991 KiB/s wr, 237 op/s 2024-03-15T13:01:05.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:04 smithi149 ceph-mon[35927]: pgmap v68: 129 pgs: 129 active+clean; 154 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 991 KiB/s wr, 237 op/s 2024-03-15T13:01:05.110 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:04 smithi184 ceph-mon[35993]: pgmap v68: 129 pgs: 129 active+clean; 154 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 991 KiB/s wr, 237 op/s 2024-03-15T13:01:05.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:04 smithi002 ceph-mon[32060]: pgmap v68: 129 pgs: 129 active+clean; 154 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 991 KiB/s wr, 237 op/s 2024-03-15T13:01:07.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:06 smithi149 ceph-mon[35927]: pgmap v69: 129 pgs: 129 active+clean; 163 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 351 op/s 2024-03-15T13:01:07.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:06 smithi002 ceph-mon[32060]: pgmap v69: 129 pgs: 129 active+clean; 163 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 351 op/s 2024-03-15T13:01:07.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:06 smithi184 ceph-mon[35993]: pgmap v69: 129 pgs: 129 active+clean; 163 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 351 op/s 2024-03-15T13:01:09.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:08 smithi149 ceph-mon[35927]: pgmap v70: 129 pgs: 129 active+clean; 163 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.4 MiB/s wr, 230 op/s 2024-03-15T13:01:09.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:08 smithi002 ceph-mon[32060]: pgmap v70: 129 pgs: 129 active+clean; 163 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.4 MiB/s wr, 230 op/s 2024-03-15T13:01:09.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:08 smithi184 ceph-mon[35993]: pgmap v70: 129 pgs: 129 active+clean; 163 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.4 MiB/s wr, 230 op/s 2024-03-15T13:01:10.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:09 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:01:10.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:09 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:01:10.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:09 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:01:10.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:09 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:01:10.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:09 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:01:10.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:09 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:01:10.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:09 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:01:10.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:09 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:01:10.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:09 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:01:10.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:09 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:01:10.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:09 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:01:10.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:09 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:01:10.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:09 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:01:10.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:09 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:01:10.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:09 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:01:11.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:10 smithi149 ceph-mon[35927]: pgmap v71: 129 pgs: 129 active+clean; 175 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 359 op/s 2024-03-15T13:01:11.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:10 smithi002 ceph-mon[32060]: pgmap v71: 129 pgs: 129 active+clean; 175 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 359 op/s 2024-03-15T13:01:11.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:10 smithi184 ceph-mon[35993]: pgmap v71: 129 pgs: 129 active+clean; 175 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 359 op/s 2024-03-15T13:01:13.104 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:12 smithi002 ceph-mon[32060]: pgmap v72: 129 pgs: 129 active+clean; 175 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 241 op/s 2024-03-15T13:01:13.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:12 smithi149 ceph-mon[35927]: pgmap v72: 129 pgs: 129 active+clean; 175 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 241 op/s 2024-03-15T13:01:13.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:12 smithi184 ceph-mon[35993]: pgmap v72: 129 pgs: 129 active+clean; 175 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 241 op/s 2024-03-15T13:01:15.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:14 smithi149 ceph-mon[35927]: pgmap v73: 129 pgs: 129 active+clean; 175 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 241 op/s 2024-03-15T13:01:15.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:14 smithi002 ceph-mon[32060]: pgmap v73: 129 pgs: 129 active+clean; 175 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 241 op/s 2024-03-15T13:01:15.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:14 smithi184 ceph-mon[35993]: pgmap v73: 129 pgs: 129 active+clean; 175 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 241 op/s 2024-03-15T13:01:17.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:16 smithi149 ceph-mon[35927]: pgmap v74: 129 pgs: 129 active+clean; 186 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 369 op/s 2024-03-15T13:01:17.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:16 smithi002 ceph-mon[32060]: pgmap v74: 129 pgs: 129 active+clean; 186 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 369 op/s 2024-03-15T13:01:17.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:16 smithi184 ceph-mon[35993]: pgmap v74: 129 pgs: 129 active+clean; 186 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 369 op/s 2024-03-15T13:01:18.649 INFO:tasks.workunit.client.0.smithi002.stderr:+ sync 2024-03-15T13:01:19.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:18 smithi149 ceph-mon[35927]: pgmap v75: 129 pgs: 129 active+clean; 186 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.0 MiB/s wr, 255 op/s 2024-03-15T13:01:19.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:18 smithi002 ceph-mon[32060]: pgmap v75: 129 pgs: 129 active+clean; 186 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.0 MiB/s wr, 255 op/s 2024-03-15T13:01:19.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:18 smithi184 ceph-mon[35993]: pgmap v75: 129 pgs: 129 active+clean; 186 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 2.0 MiB/s wr, 255 op/s 2024-03-15T13:01:20.357 INFO:tasks.workunit.client.0.smithi002.stderr:+ tar c linux-4.0.5 2024-03-15T13:01:21.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:20 smithi002 ceph-mon[32060]: pgmap v76: 129 pgs: 129 active+clean; 194 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 371 op/s 2024-03-15T13:01:21.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:20 smithi184 ceph-mon[35993]: pgmap v76: 129 pgs: 129 active+clean; 194 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 371 op/s 2024-03-15T13:01:21.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:20 smithi149 ceph-mon[35927]: pgmap v76: 129 pgs: 129 active+clean; 194 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 371 op/s 2024-03-15T13:01:22.727 INFO:tasks.workunit.client.0.smithi002.stderr:+ rm -rf linux-4.0.5 2024-03-15T13:01:23.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:22 smithi002 ceph-mon[32060]: pgmap v77: 129 pgs: 129 active+clean; 194 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.7 MiB/s wr, 243 op/s 2024-03-15T13:01:23.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:22 smithi184 ceph-mon[35993]: pgmap v77: 129 pgs: 129 active+clean; 194 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.7 MiB/s wr, 243 op/s 2024-03-15T13:01:23.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:22 smithi149 ceph-mon[35927]: pgmap v77: 129 pgs: 129 active+clean; 194 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.7 MiB/s wr, 243 op/s 2024-03-15T13:01:25.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:24 smithi002 ceph-mon[32060]: pgmap v78: 129 pgs: 129 active+clean; 194 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.7 MiB/s wr, 243 op/s 2024-03-15T13:01:25.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:24 smithi184 ceph-mon[35993]: pgmap v78: 129 pgs: 129 active+clean; 194 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.7 MiB/s wr, 243 op/s 2024-03-15T13:01:25.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:24 smithi149 ceph-mon[35927]: pgmap v78: 129 pgs: 129 active+clean; 194 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.7 MiB/s wr, 243 op/s 2024-03-15T13:01:26.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:25 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:01:26.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:25 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:01:26.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:25 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:01:27.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:01:27.121 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:01:27.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:26 smithi002 ceph-mon[32060]: pgmap v79: 129 pgs: 129 active+clean; 198 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2024-03-15T13:01:27.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.245+0000 7fcb0f0e1640 1 -- 172.21.15.2:0/813511956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb08102710 msgr2=0x7fcb08102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.245+0000 7fcb0f0e1640 1 --2- 172.21.15.2:0/813511956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb08102710 0x7fcb08102af0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009900 tx=0x7fcafc02bcd0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.245+0000 7fcb0f0e1640 1 -- 172.21.15.2:0/813511956 shutdown_connections 2024-03-15T13:01:27.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.245+0000 7fcb0f0e1640 1 --2- 172.21.15.2:0/813511956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb0810ba50 0x7fcb0810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.245+0000 7fcb0f0e1640 1 --2- 172.21.15.2:0/813511956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb08103030 0x7fcb0810b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.245+0000 7fcb0f0e1640 1 --2- 172.21.15.2:0/813511956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb08102710 0x7fcb08102af0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.245+0000 7fcb0f0e1640 1 -- 172.21.15.2:0/813511956 >> 172.21.15.2:0/813511956 conn(0x7fcb080775a0 msgr2=0x7fcb080779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.245+0000 7fcb0f0e1640 1 -- 172.21.15.2:0/813511956 shutdown_connections 2024-03-15T13:01:27.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.245+0000 7fcb0f0e1640 1 -- 172.21.15.2:0/813511956 wait complete. 2024-03-15T13:01:27.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.245+0000 7fcb0f0e1640 1 Processor -- start 2024-03-15T13:01:27.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb0f0e1640 1 -- start start 2024-03-15T13:01:27.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb0f0e1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb08102710 0x7fcb0819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb0f0e1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb08103030 0x7fcb0819dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb0f0e1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb0810ba50 0x7fcb08197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb0f0e1640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb08197f70 con 0x7fcb08103030 2024-03-15T13:01:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb0f0e1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb081980e0 con 0x7fcb08102710 2024-03-15T13:01:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb0f0e1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb081982a0 con 0x7fcb0810ba50 2024-03-15T13:01:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb07fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb08103030 0x7fcb0819dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb07fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb08103030 0x7fcb0819dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35004/0 (socket says 172.21.15.2:35004) 2024-03-15T13:01:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb07fff640 1 -- 172.21.15.2:0/239833242 learned_addr learned my addr 172.21.15.2:0/239833242 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb0d67e640 1 --2- 172.21.15.2:0/239833242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb0810ba50 0x7fcb08197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb0ce7d640 1 --2- 172.21.15.2:0/239833242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb08102710 0x7fcb0819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb07fff640 1 -- 172.21.15.2:0/239833242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb0810ba50 msgr2=0x7fcb08197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb07fff640 1 --2- 172.21.15.2:0/239833242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb0810ba50 0x7fcb08197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb07fff640 1 -- 172.21.15.2:0/239833242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb08102710 msgr2=0x7fcb0819d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb07fff640 1 --2- 172.21.15.2:0/239833242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb08102710 0x7fcb0819d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb07fff640 1 -- 172.21.15.2:0/239833242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc009560 con 0x7fcb08103030 2024-03-15T13:01:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb07fff640 1 --2- 172.21.15.2:0/239833242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb08103030 0x7fcb0819dcb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400e9f0 tx=0x7fcaf400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb05ffb640 1 -- 172.21.15.2:0/239833242 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4009720 con 0x7fcb08103030 2024-03-15T13:01:27.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb05ffb640 1 -- 172.21.15.2:0/239833242 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcaf400cd80 con 0x7fcb08103030 2024-03-15T13:01:27.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb0f0e1640 1 -- 172.21.15.2:0/239833242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb08198580 con 0x7fcb08103030 2024-03-15T13:01:27.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.246+0000 7fcb05ffb640 1 -- 172.21.15.2:0/239833242 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4010690 con 0x7fcb08103030 2024-03-15T13:01:27.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.247+0000 7fcb0f0e1640 1 -- 172.21.15.2:0/239833242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb081a4810 con 0x7fcb08103030 2024-03-15T13:01:27.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.249+0000 7fcb05ffb640 1 -- 172.21.15.2:0/239833242 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fcaf4004e90 con 0x7fcb08103030 2024-03-15T13:01:27.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb11d95740 1 Processor -- start 2024-03-15T13:01:27.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb11d95740 1 -- start start 2024-03-15T13:01:27.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb11d95740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f5f2b0 0x5565c4f5f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb11d95740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 0x5565c4f68270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb05ffb640 1 --2- 172.21.15.2:0/239833242 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcaf80a1ba0 0x7fcaf80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb11d95740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f687b0 0x5565c4f6aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb11d95740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c4ece280 con 0x5565c4f5f2b0 2024-03-15T13:01:27.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb11d95740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c4ecb7b0 con 0x5565c4f687b0 2024-03-15T13:01:27.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb11d95740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c4ec6f90 con 0x5565c4f5fbd0 2024-03-15T13:01:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb07fff640 1 --2- 172.21.15.2:0/239833242 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcaf80a1ba0 0x7fcaf80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb0d67e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f5f2b0 0x5565c4f5f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb0ce7d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 0x5565c4f68270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb0d67e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f5f2b0 0x5565c4f5f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35018/0 (socket says 172.21.15.2:35018) 2024-03-15T13:01:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb0d67e640 1 -- 172.21.15.2:0/358540959 learned_addr learned my addr 172.21.15.2:0/358540959 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb0d67e640 1 --2- 172.21.15.2:0/358540959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f687b0 0x5565c4f6aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb05ffb640 1 -- 172.21.15.2:0/239833242 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fcaf4014070 con 0x7fcb08103030 2024-03-15T13:01:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb0ce7d640 1 -- 172.21.15.2:0/358540959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f687b0 msgr2=0x5565c4f6aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb0ce7d640 1 --2- 172.21.15.2:0/358540959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f687b0 0x5565c4f6aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb0ce7d640 1 -- 172.21.15.2:0/358540959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f5f2b0 msgr2=0x5565c4f5f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb0ce7d640 1 --2- 172.21.15.2:0/358540959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f5f2b0 0x5565c4f5f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.250+0000 7fcb0ce7d640 1 -- 172.21.15.2:0/358540959 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565c4e85570 con 0x5565c4f5fbd0 2024-03-15T13:01:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.251+0000 7fcb07fff640 1 --2- 172.21.15.2:0/239833242 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcaf80a1ba0 0x7fcaf80a4060 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4096460 tx=0x7fcaf4096aa0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.251+0000 7fcb0ce7d640 1 --2- 172.21.15.2:0/358540959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 0x5565c4f68270 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009230 tx=0x7fcafc02c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=492e5fe9627d87a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.251+0000 7fcaf2ffd640 1 -- 172.21.15.2:0/358540959 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc0046c0 con 0x5565c4f5fbd0 2024-03-15T13:01:27.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.251+0000 7fcaf2ffd640 1 -- 172.21.15.2:0/358540959 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcafc004820 con 0x5565c4f5fbd0 2024-03-15T13:01:27.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.251+0000 7fcaf2ffd640 1 -- 172.21.15.2:0/358540959 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc004f20 con 0x5565c4f5fbd0 2024-03-15T13:01:27.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 -- 172.21.15.2:0/358540959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 msgr2=0x5565c4f68270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 --2- 172.21.15.2:0/358540959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 0x5565c4f68270 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009230 tx=0x7fcafc02c790 comp rx=0 tx=0).stop 2024-03-15T13:01:27.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 -- 172.21.15.2:0/358540959 shutdown_connections 2024-03-15T13:01:27.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 --2- 172.21.15.2:0/358540959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f687b0 0x5565c4f6aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 --2- 172.21.15.2:0/358540959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 0x5565c4f68270 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 --2- 172.21.15.2:0/358540959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f5f2b0 0x5565c4f5f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 -- 172.21.15.2:0/358540959 >> 172.21.15.2:0/358540959 conn(0x5565c4f59990 msgr2=0x5565c4f59d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 -- 172.21.15.2:0/358540959 shutdown_connections 2024-03-15T13:01:27.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 -- 172.21.15.2:0/358540959 wait complete. 2024-03-15T13:01:27.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 -- start start 2024-03-15T13:01:27.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 Processor -- start 2024-03-15T13:01:27.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f5f2b0 0x5565c4f68220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f5fbd0 0x5565c4f655b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f687b0 0x5565c4f65cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c4b9a430 con 0x5565c4f5fbd0 2024-03-15T13:01:27.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c4b56e00 con 0x5565c4f5f2b0 2024-03-15T13:01:27.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.252+0000 7fcb11d95740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c4e4bfd0 con 0x5565c4f687b0 2024-03-15T13:01:27.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcb0ce7d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f5fbd0 0x5565c4f655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcb11d95740 10 client.0 _ll_register_callbacks cb 0x5565c4ea1fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:27.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcb0d67e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f5f2b0 0x5565c4f68220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcb0ce7d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f5fbd0 0x5565c4f655b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35026/0 (socket says 172.21.15.2:35026) 2024-03-15T13:01:27.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcb0ce7d640 1 -- 172.21.15.2:0/96343214 learned_addr learned my addr 172.21.15.2:0/96343214 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcb0d67e640 1 --2- 172.21.15.2:0/96343214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f687b0 0x5565c4f65cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcb0ce7d640 1 -- 172.21.15.2:0/96343214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f687b0 msgr2=0x5565c4f65cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcb0ce7d640 1 --2- 172.21.15.2:0/96343214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f687b0 0x5565c4f65cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcb0ce7d640 1 -- 172.21.15.2:0/96343214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f5f2b0 msgr2=0x5565c4f68220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcb0ce7d640 1 --2- 172.21.15.2:0/96343214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f5f2b0 0x5565c4f68220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcb0ce7d640 1 -- 172.21.15.2:0/96343214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0810eb70 con 0x5565c4f5fbd0 2024-03-15T13:01:27.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcb0ce7d640 1 --2- 172.21.15.2:0/96343214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f5fbd0 0x5565c4f655b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0092a0 tx=0x7fcafc02c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb748b33d2eea681 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcae7fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:01:27.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcae7fff640 1 -- 172.21.15.2:0/96343214 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc005120 con 0x5565c4f5fbd0 2024-03-15T13:01:27.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcae7fff640 1 -- 172.21.15.2:0/96343214 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fcafc005280 con 0x5565c4f5fbd0 2024-03-15T13:01:27.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcb11d95740 1 -- 172.21.15.2:0/96343214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565c4bc67e0 con 0x5565c4f5fbd0 2024-03-15T13:01:27.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcae7fff640 1 -- 172.21.15.2:0/96343214 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcafc005460 con 0x5565c4f5fbd0 2024-03-15T13:01:27.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcb11d95740 10 client.16437 fetch_fsmap learned FSMap version 34 2024-03-15T13:01:27.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.253+0000 7fcb11d95740 1 -- 172.21.15.2:0/96343214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5565c4ec5610 con 0x5565c4f5fbd0 2024-03-15T13:01:27.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.254+0000 7fcae7fff640 1 -- 172.21.15.2:0/96343214 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5565c4bc67e0 con 0x5565c4f5fbd0 2024-03-15T13:01:27.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.254+0000 7fcae7fff640 1 -- 172.21.15.2:0/96343214 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fcafc052070 con 0x5565c4f5fbd0 2024-03-15T13:01:27.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.254+0000 7fcb11d95740 10 client.16437 fetch_fsmap finished waiting for FSMap version 34 2024-03-15T13:01:27.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.254+0000 7fcb11d95740 10 client.16437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:01:27.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.254+0000 7fcb11d95740 20 client.16437 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:27.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.254+0000 7fcb11d95740 1 --2- 172.21.15.2:0/96343214 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5565c4f786e0 0x5565c4f7ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.254+0000 7fcb11d95740 4 client.16437 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:01:27.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.254+0000 7fcb11d95740 1 -- 172.21.15.2:0/96343214 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565c4c03720 con 0x5565c4f786e0 2024-03-15T13:01:27.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.254+0000 7fcb0d67e640 1 --2- 172.21.15.2:0/96343214 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5565c4f786e0 0x5565c4f7ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.254+0000 7fcae7fff640 10 client.16437 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:27.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.254+0000 7fcb0d67e640 1 --2- 172.21.15.2:0/96343214 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5565c4f786e0 0x5565c4f7ab00 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b555541714b4dad7 server_cookie=71eacf8816145268 in_seq=0 out_seq=0 2024-03-15T13:01:27.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.257+0000 7fcae7fff640 1 -- 172.21.15.2:0/96343214 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5565c4c03720 con 0x5565c4f786e0 2024-03-15T13:01:27.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.257+0000 7fcae7fff640 10 client.16437 handle_command_reply: tid=0 2024-03-15T13:01:27.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.257+0000 7fcb11d95740 1 client.16437 shutdown 2024-03-15T13:01:27.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.257+0000 7fcb11d95740 1 -- 172.21.15.2:0/96343214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f5fbd0 msgr2=0x5565c4f655b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.257+0000 7fcb11d95740 1 --2- 172.21.15.2:0/96343214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f5fbd0 0x5565c4f655b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0092a0 tx=0x7fcafc02c3d0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.257+0000 7fcb11d95740 1 -- 172.21.15.2:0/96343214 shutdown_connections 2024-03-15T13:01:27.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.257+0000 7fcb11d95740 1 --2- 172.21.15.2:0/96343214 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5565c4f786e0 0x5565c4f7ab00 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.257+0000 7fcb11d95740 1 --2- 172.21.15.2:0/96343214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f687b0 0x5565c4f65cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.257+0000 7fcb11d95740 1 --2- 172.21.15.2:0/96343214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f5fbd0 0x5565c4f655b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.257+0000 7fcae7fff640 0 client.16437 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:27.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.257+0000 7fcb11d95740 1 --2- 172.21.15.2:0/96343214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f5f2b0 0x5565c4f68220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.257+0000 7fcb11d95740 1 -- 172.21.15.2:0/96343214 >> 172.21.15.2:0/96343214 conn(0x5565c4f59990 msgr2=0x5565c4f5b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.257+0000 7fcb11d95740 1 -- 172.21.15.2:0/96343214 shutdown_connections 2024-03-15T13:01:27.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.257+0000 7fcb11d95740 1 -- 172.21.15.2:0/96343214 wait complete. 2024-03-15T13:01:27.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.257+0000 7fcb11d95740 20 client.16437 trim_cache size 0 max 16384 2024-03-15T13:01:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.273+0000 7fcb11d95740 1 Processor -- start 2024-03-15T13:01:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.273+0000 7fcb11d95740 1 -- start start 2024-03-15T13:01:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.273+0000 7fcb11d95740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f5f2b0 0x5565c4f65550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb11d95740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 0x5565c4f67ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb11d95740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f687b0 0x5565c4f68220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb11d95740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c4f7b160 con 0x5565c4f687b0 2024-03-15T13:01:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb11d95740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c4f74d70 con 0x5565c4f5f2b0 2024-03-15T13:01:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb11d95740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c4f5ea50 con 0x5565c4f5fbd0 2024-03-15T13:01:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb0d67e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f687b0 0x5565c4f68220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb0d67e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f687b0 0x5565c4f68220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35036/0 (socket says 172.21.15.2:35036) 2024-03-15T13:01:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb0d67e640 1 -- 172.21.15.2:0/3601975452 learned_addr learned my addr 172.21.15.2:0/3601975452 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb0ce7d640 1 --2- 172.21.15.2:0/3601975452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 0x5565c4f67ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb0d67e640 1 --2- 172.21.15.2:0/3601975452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f5f2b0 0x5565c4f65550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb0d67e640 1 -- 172.21.15.2:0/3601975452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 msgr2=0x5565c4f67ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb0d67e640 1 --2- 172.21.15.2:0/3601975452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 0x5565c4f67ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb0d67e640 1 -- 172.21.15.2:0/3601975452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f5f2b0 msgr2=0x5565c4f65550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb0d67e640 1 --2- 172.21.15.2:0/3601975452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f5f2b0 0x5565c4f65550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb0d67e640 1 -- 172.21.15.2:0/3601975452 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565c4e61ac0 con 0x5565c4f687b0 2024-03-15T13:01:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcb0d67e640 1 --2- 172.21.15.2:0/3601975452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f687b0 0x5565c4f68220 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fcb00005c10 tx=0x7fcb000096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1168af0f9ad32c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcaf27fc640 1 -- 172.21.15.2:0/3601975452 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00025e50 con 0x5565c4f687b0 2024-03-15T13:01:27.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcaf27fc640 1 -- 172.21.15.2:0/3601975452 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcb0000fa00 con 0x5565c4f687b0 2024-03-15T13:01:27.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.274+0000 7fcaf27fc640 1 -- 172.21.15.2:0/3601975452 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00012ea0 con 0x5565c4f687b0 2024-03-15T13:01:27.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.275+0000 7fcb11d95740 1 -- 172.21.15.2:0/3601975452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f687b0 msgr2=0x5565c4f68220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.275+0000 7fcb11d95740 1 --2- 172.21.15.2:0/3601975452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f687b0 0x5565c4f68220 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fcb00005c10 tx=0x7fcb000096a0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.275+0000 7fcb11d95740 1 -- 172.21.15.2:0/3601975452 shutdown_connections 2024-03-15T13:01:27.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.275+0000 7fcb11d95740 1 --2- 172.21.15.2:0/3601975452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f687b0 0x5565c4f68220 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.275+0000 7fcb11d95740 1 --2- 172.21.15.2:0/3601975452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 0x5565c4f67ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.275+0000 7fcb11d95740 1 --2- 172.21.15.2:0/3601975452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f5f2b0 0x5565c4f65550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.275+0000 7fcb11d95740 1 -- 172.21.15.2:0/3601975452 >> 172.21.15.2:0/3601975452 conn(0x5565c4f59990 msgr2=0x5565c4f64e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.275+0000 7fcb11d95740 1 -- 172.21.15.2:0/3601975452 shutdown_connections 2024-03-15T13:01:27.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.275+0000 7fcb11d95740 1 -- 172.21.15.2:0/3601975452 wait complete. 2024-03-15T13:01:27.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.275+0000 7fcb11d95740 1 -- start start 2024-03-15T13:01:27.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.275+0000 7fcb11d95740 1 Processor -- start 2024-03-15T13:01:27.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb11d95740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f5f2b0 0x5565c4f61860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb11d95740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 0x5565c4f61da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb11d95740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f687b0 0x5565c4f622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb11d95740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c4f62870 con 0x5565c4f687b0 2024-03-15T13:01:27.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb11d95740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c4f629e0 con 0x5565c4f5f2b0 2024-03-15T13:01:27.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb11d95740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c4f62ba0 con 0x5565c4f5fbd0 2024-03-15T13:01:27.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb11d95740 10 client.0 _ll_register_callbacks cb 0x7fcb00009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:27.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb0d67e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f687b0 0x5565c4f622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb0d67e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f687b0 0x5565c4f622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35052/0 (socket says 172.21.15.2:35052) 2024-03-15T13:01:27.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb0d67e640 1 -- 172.21.15.2:0/1460813899 learned_addr learned my addr 172.21.15.2:0/1460813899 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb0ce7d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 0x5565c4f61da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb0d67e640 1 --2- 172.21.15.2:0/1460813899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f5f2b0 0x5565c4f61860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb0ce7d640 1 -- 172.21.15.2:0/1460813899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f5f2b0 msgr2=0x5565c4f61860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb0ce7d640 1 --2- 172.21.15.2:0/1460813899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f5f2b0 0x5565c4f61860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb0ce7d640 1 -- 172.21.15.2:0/1460813899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f687b0 msgr2=0x5565c4f622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb0ce7d640 1 --2- 172.21.15.2:0/1460813899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f687b0 0x5565c4f622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb0ce7d640 1 -- 172.21.15.2:0/1460813899 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb00013980 con 0x5565c4f5fbd0 2024-03-15T13:01:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb0d67e640 1 --2- 172.21.15.2:0/1460813899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f687b0 0x5565c4f622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcb0ce7d640 1 --2- 172.21.15.2:0/1460813899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 0x5565c4f61da0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcafc004650 tx=0x7fcafc004ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea8e29a28c69f334 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.276+0000 7fcae7fff640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:01:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.277+0000 7fcae7fff640 1 -- 172.21.15.2:0/1460813899 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc065070 con 0x5565c4f5fbd0 2024-03-15T13:01:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.277+0000 7fcb11d95740 1 -- 172.21.15.2:0/1460813899 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcafc004820 con 0x5565c4f5fbd0 2024-03-15T13:01:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.277+0000 7fcae7fff640 1 -- 172.21.15.2:0/1460813899 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fcafc04a900 con 0x5565c4f5fbd0 2024-03-15T13:01:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.277+0000 7fcae7fff640 1 -- 172.21.15.2:0/1460813899 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcafc04aae0 con 0x5565c4f5fbd0 2024-03-15T13:01:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.277+0000 7fcb11d95740 10 client.26239 fetch_fsmap learned FSMap version 34 2024-03-15T13:01:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.277+0000 7fcb11d95740 1 -- 172.21.15.2:0/1460813899 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5565c4e876a0 con 0x5565c4f5fbd0 2024-03-15T13:01:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.277+0000 7fcae7fff640 1 -- 172.21.15.2:0/1460813899 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fcafc004820 con 0x5565c4f5fbd0 2024-03-15T13:01:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.277+0000 7fcae7fff640 1 -- 172.21.15.2:0/1460813899 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 34) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fcafc068020 con 0x5565c4f5fbd0 2024-03-15T13:01:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.277+0000 7fcb11d95740 10 client.26239 fetch_fsmap finished waiting for FSMap version 34 2024-03-15T13:01:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.277+0000 7fcb11d95740 10 client.26239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:01:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.277+0000 7fcb11d95740 20 client.26239 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.277+0000 7fcb11d95740 1 --2- 172.21.15.2:0/1460813899 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5565c4fd5880 0x5565c4fd7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.277+0000 7fcb11d95740 4 client.26239 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:01:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.277+0000 7fcb11d95740 1 -- 172.21.15.2:0/1460813899 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5565c4c52890 con 0x5565c4fd5880 2024-03-15T13:01:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.277+0000 7fcb0d67e640 1 --2- 172.21.15.2:0/1460813899 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5565c4fd5880 0x5565c4fd7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.278+0000 7fcb0d67e640 1 --2- 172.21.15.2:0/1460813899 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5565c4fd5880 0x5565c4fd7ca0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41cc059f6bdb470c server_cookie=b5900d92b298ecec in_seq=0 out_seq=0 2024-03-15T13:01:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.278+0000 7fcae7fff640 10 client.26239 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.279+0000 7fcae7fff640 1 -- 172.21.15.2:0/1460813899 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5565c4c52890 con 0x5565c4fd5880 2024-03-15T13:01:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.279+0000 7fcae7fff640 10 client.26239 handle_command_reply: tid=0 2024-03-15T13:01:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.279+0000 7fcb11d95740 1 client.26239 shutdown 2024-03-15T13:01:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.279+0000 7fcb11d95740 1 -- 172.21.15.2:0/1460813899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 msgr2=0x5565c4f61da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.279+0000 7fcb11d95740 1 --2- 172.21.15.2:0/1460813899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 0x5565c4f61da0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcafc004650 tx=0x7fcafc004ba0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.279+0000 7fcb11d95740 1 -- 172.21.15.2:0/1460813899 shutdown_connections 2024-03-15T13:01:27.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.279+0000 7fcb11d95740 1 --2- 172.21.15.2:0/1460813899 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5565c4fd5880 0x5565c4fd7ca0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.279+0000 7fcb11d95740 1 --2- 172.21.15.2:0/1460813899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5565c4f687b0 0x5565c4f622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.279+0000 7fcb11d95740 1 --2- 172.21.15.2:0/1460813899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5565c4f5fbd0 0x5565c4f61da0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.279+0000 7fcb11d95740 1 --2- 172.21.15.2:0/1460813899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5565c4f5f2b0 0x5565c4f61860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.279+0000 7fcae7fff640 0 client.26239 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:27.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.279+0000 7fcb11d95740 1 -- 172.21.15.2:0/1460813899 >> 172.21.15.2:0/1460813899 conn(0x5565c4f59990 msgr2=0x5565c4f581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.279+0000 7fcb11d95740 1 -- 172.21.15.2:0/1460813899 shutdown_connections 2024-03-15T13:01:27.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.279+0000 7fcb11d95740 1 -- 172.21.15.2:0/1460813899 wait complete. 2024-03-15T13:01:27.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.280+0000 7fcb11d95740 20 client.26239 trim_cache size 0 max 16384 2024-03-15T13:01:27.271 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:01:27.271 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:01:27.272 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:01:27.272 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:01:27.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.280+0000 7fcb0f0e1640 1 -- 172.21.15.2:0/239833242 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcaf80a1ba0 msgr2=0x7fcaf80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.280+0000 7fcb0f0e1640 1 --2- 172.21.15.2:0/239833242 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcaf80a1ba0 0x7fcaf80a4060 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4096460 tx=0x7fcaf4096aa0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.280+0000 7fcb0f0e1640 1 -- 172.21.15.2:0/239833242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb08103030 msgr2=0x7fcb0819dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.280+0000 7fcb0f0e1640 1 --2- 172.21.15.2:0/239833242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb08103030 0x7fcb0819dcb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400e9f0 tx=0x7fcaf400edf0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.280+0000 7fcb0f0e1640 1 -- 172.21.15.2:0/239833242 shutdown_connections 2024-03-15T13:01:27.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.280+0000 7fcb0f0e1640 1 --2- 172.21.15.2:0/239833242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb0810ba50 0x7fcb08197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.280+0000 7fcb0f0e1640 1 --2- 172.21.15.2:0/239833242 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fcaf80a1ba0 0x7fcaf80a4060 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.280+0000 7fcb0f0e1640 1 --2- 172.21.15.2:0/239833242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb08103030 0x7fcb0819dcb0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.280+0000 7fcb0f0e1640 1 --2- 172.21.15.2:0/239833242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb08102710 0x7fcb0819d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.280+0000 7fcb0f0e1640 1 -- 172.21.15.2:0/239833242 >> 172.21.15.2:0/239833242 conn(0x7fcb080775a0 msgr2=0x7fcb080fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.280+0000 7fcb0f0e1640 1 -- 172.21.15.2:0/239833242 shutdown_connections 2024-03-15T13:01:27.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.280+0000 7fcb0f0e1640 1 -- 172.21.15.2:0/239833242 wait complete. 2024-03-15T13:01:27.277 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:01:27.278 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:01:27.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:26 smithi184 ceph-mon[35993]: pgmap v79: 129 pgs: 129 active+clean; 198 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2024-03-15T13:01:27.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:26 smithi149 ceph-mon[35927]: pgmap v79: 129 pgs: 129 active+clean; 198 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2024-03-15T13:01:27.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.435+0000 7f0ff2495640 1 -- 172.21.15.2:0/776464455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fec106490 msgr2=0x7f0fec106870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.435+0000 7f0ff2495640 1 --2- 172.21.15.2:0/776464455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fec106490 0x7f0fec106870 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009930 tx=0x7f0fdc02bc30 comp rx=0 tx=0).stop 2024-03-15T13:01:27.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.435+0000 7f0ff2495640 1 -- 172.21.15.2:0/776464455 shutdown_connections 2024-03-15T13:01:27.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.436+0000 7f0ff2495640 1 --2- 172.21.15.2:0/776464455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fec100db0 0x7f0fec10d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.436+0000 7f0ff2495640 1 --2- 172.21.15.2:0/776464455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fec1003f0 0x7f0fec100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.436+0000 7f0ff2495640 1 --2- 172.21.15.2:0/776464455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fec106490 0x7f0fec106870 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.436+0000 7f0ff2495640 1 -- 172.21.15.2:0/776464455 >> 172.21.15.2:0/776464455 conn(0x7f0fec0774f0 msgr2=0x7f0fec077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.436+0000 7f0ff2495640 1 -- 172.21.15.2:0/776464455 shutdown_connections 2024-03-15T13:01:27.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.436+0000 7f0ff2495640 1 -- 172.21.15.2:0/776464455 wait complete. 2024-03-15T13:01:27.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.436+0000 7f0ff2495640 1 Processor -- start 2024-03-15T13:01:27.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.436+0000 7f0ff2495640 1 -- start start 2024-03-15T13:01:27.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.436+0000 7f0ff2495640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fec1003f0 0x7f0fec19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.436+0000 7f0ff2495640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fec100db0 0x7f0fec19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.436+0000 7f0ff2495640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fec106490 0x7f0fec1968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.436+0000 7f0ff2495640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec196f20 con 0x7f0fec106490 2024-03-15T13:01:27.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.436+0000 7f0ff2495640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec197090 con 0x7f0fec100db0 2024-03-15T13:01:27.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.436+0000 7f0ff2495640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec197250 con 0x7f0fec1003f0 2024-03-15T13:01:27.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0ff0a32640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fec106490 0x7f0fec1968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0febfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fec1003f0 0x7f0fec19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0feb7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fec100db0 0x7f0fec19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0febfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fec1003f0 0x7f0fec19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:49026/0 (socket says 172.21.15.2:49026) 2024-03-15T13:01:27.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0ff0a32640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fec106490 0x7f0fec1968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35068/0 (socket says 172.21.15.2:35068) 2024-03-15T13:01:27.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0febfff640 1 -- 172.21.15.2:0/2668507246 learned_addr learned my addr 172.21.15.2:0/2668507246 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0feb7fe640 1 --2- 172.21.15.2:0/2668507246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fec100db0 0x7f0fec19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:52408/0 (socket says 172.21.15.2:52408) 2024-03-15T13:01:27.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0febfff640 1 -- 172.21.15.2:0/2668507246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fec100db0 msgr2=0x7f0fec19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0febfff640 1 --2- 172.21.15.2:0/2668507246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fec100db0 0x7f0fec19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0febfff640 1 -- 172.21.15.2:0/2668507246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fec106490 msgr2=0x7f0fec1968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0febfff640 1 --2- 172.21.15.2:0/2668507246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fec106490 0x7f0fec1968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0febfff640 1 -- 172.21.15.2:0/2668507246 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc009560 con 0x7f0fec1003f0 2024-03-15T13:01:27.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0febfff640 1 --2- 172.21.15.2:0/2668507246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fec1003f0 0x7f0fec19c720 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009900 tx=0x7f0fdc02ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0fe97fa640 1 -- 172.21.15.2:0/2668507246 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc02cc10 con 0x7f0fec1003f0 2024-03-15T13:01:27.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0ff2495640 1 -- 172.21.15.2:0/2668507246 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fec1974d0 con 0x7f0fec1003f0 2024-03-15T13:01:27.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0ff2495640 1 -- 172.21.15.2:0/2668507246 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fec1a37c0 con 0x7f0fec1003f0 2024-03-15T13:01:27.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0fe97fa640 1 -- 172.21.15.2:0/2668507246 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0fdc0054d0 con 0x7f0fec1003f0 2024-03-15T13:01:27.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.437+0000 7f0fe97fa640 1 -- 172.21.15.2:0/2668507246 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc003c60 con 0x7f0fec1003f0 2024-03-15T13:01:27.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.438+0000 7f0ff2495640 1 -- 172.21.15.2:0/2668507246 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fec0682e0 con 0x7f0fec1003f0 2024-03-15T13:01:27.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.440+0000 7f0fe97fa640 1 -- 172.21.15.2:0/2668507246 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f0fdc005000 con 0x7f0fec1003f0 2024-03-15T13:01:27.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.440+0000 7f0fe97fa640 1 --2- 172.21.15.2:0/2668507246 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f0fd40a1ba0 0x7f0fd40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.440+0000 7f0feb7fe640 1 --2- 172.21.15.2:0/2668507246 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f0fd40a1ba0 0x7f0fd40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.440+0000 7f0fe97fa640 1 -- 172.21.15.2:0/2668507246 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f0fdc0f0240 con 0x7f0fec1003f0 2024-03-15T13:01:27.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.441+0000 7f0feb7fe640 1 --2- 172.21.15.2:0/2668507246 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f0fd40a1ba0 0x7f0fd40a4060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0fec104bb0 tx=0x7f0fe0005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.447+0000 7f0fe97fa640 1 -- 172.21.15.2:0/2668507246 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0fdc0b8ed0 con 0x7f0fec1003f0 2024-03-15T13:01:27.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.672+0000 7f0ff2495640 1 -- 172.21.15.2:0/2668507246 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fec197e00 con 0x7f0fec1003f0 2024-03-15T13:01:27.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.673+0000 7f0fe97fa640 1 -- 172.21.15.2:0/2668507246 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+11463 (secure 0 0 0) 0x7f0fdc043090 con 0x7f0fec1003f0 2024-03-15T13:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:01:27.659 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:01:24.310166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24500,24464,24460,24434,24539]},"id":1}]} 2024-03-15T13:01:27.659 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 34 2024-03-15T13:01:27.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.675+0000 7f0ff2495640 1 -- 172.21.15.2:0/2668507246 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f0fd40a1ba0 msgr2=0x7f0fd40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.675+0000 7f0ff2495640 1 --2- 172.21.15.2:0/2668507246 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f0fd40a1ba0 0x7f0fd40a4060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0fec104bb0 tx=0x7f0fe0005f90 comp rx=0 tx=0).stop 2024-03-15T13:01:27.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.675+0000 7f0ff2495640 1 -- 172.21.15.2:0/2668507246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fec1003f0 msgr2=0x7f0fec19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.675+0000 7f0ff2495640 1 --2- 172.21.15.2:0/2668507246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fec1003f0 0x7f0fec19c720 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009900 tx=0x7f0fdc02ca20 comp rx=0 tx=0).stop 2024-03-15T13:01:27.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.675+0000 7f0ff2495640 1 -- 172.21.15.2:0/2668507246 shutdown_connections 2024-03-15T13:01:27.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.675+0000 7f0ff2495640 1 --2- 172.21.15.2:0/2668507246 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f0fd40a1ba0 0x7f0fd40a4060 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.675+0000 7f0ff2495640 1 --2- 172.21.15.2:0/2668507246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0fec106490 0x7f0fec1968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.675+0000 7f0ff2495640 1 --2- 172.21.15.2:0/2668507246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fec100db0 0x7f0fec19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.675+0000 7f0ff2495640 1 --2- 172.21.15.2:0/2668507246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fec1003f0 0x7f0fec19c720 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.675+0000 7f0ff2495640 1 -- 172.21.15.2:0/2668507246 >> 172.21.15.2:0/2668507246 conn(0x7f0fec0774f0 msgr2=0x7f0fec102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.675+0000 7f0ff2495640 1 -- 172.21.15.2:0/2668507246 shutdown_connections 2024-03-15T13:01:27.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.675+0000 7f0ff2495640 1 -- 172.21.15.2:0/2668507246 wait complete. 2024-03-15T13:01:27.672 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:01:27.672 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:01:27.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.794+0000 7fd29af81640 1 -- 172.21.15.2:0/2028330709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294101e60 msgr2=0x7fd29410e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.794+0000 7fd29af81640 1 --2- 172.21.15.2:0/2028330709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294101e60 0x7fd29410e320 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009ca0 tx=0x7fd28c02c0f0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.794+0000 7fd29af81640 1 -- 172.21.15.2:0/2028330709 shutdown_connections 2024-03-15T13:01:27.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.794+0000 7fd29af81640 1 --2- 172.21.15.2:0/2028330709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294101e60 0x7fd29410e320 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.794+0000 7fd29af81640 1 --2- 172.21.15.2:0/2028330709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2941014a0 0x7fd294101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.794+0000 7fd29af81640 1 --2- 172.21.15.2:0/2028330709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd294107560 0x7fd294107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.794+0000 7fd29af81640 1 -- 172.21.15.2:0/2028330709 >> 172.21.15.2:0/2028330709 conn(0x7fd2940788f0 msgr2=0x7fd294078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.794+0000 7fd29af81640 1 -- 172.21.15.2:0/2028330709 shutdown_connections 2024-03-15T13:01:27.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.794+0000 7fd29af81640 1 -- 172.21.15.2:0/2028330709 wait complete. 2024-03-15T13:01:27.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.794+0000 7fd29af81640 1 Processor -- start 2024-03-15T13:01:27.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.794+0000 7fd29af81640 1 -- start start 2024-03-15T13:01:27.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd29af81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2941014a0 0x7fd29419d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd29af81640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd294101e60 0x7fd29419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd29af81640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd294107560 0x7fd294197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd29af81640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd294197ef0 con 0x7fd2941014a0 2024-03-15T13:01:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd29af81640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd294198060 con 0x7fd294107560 2024-03-15T13:01:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd29af81640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd294198220 con 0x7fd294101e60 2024-03-15T13:01:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd298d1d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2941014a0 0x7fd29419d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd298d1d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2941014a0 0x7fd29419d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35084/0 (socket says 172.21.15.2:35084) 2024-03-15T13:01:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd298d1d640 1 -- 172.21.15.2:0/3490110220 learned_addr learned my addr 172.21.15.2:0/3490110220 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd293fff640 1 --2- 172.21.15.2:0/3490110220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd294101e60 0x7fd29419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd29951e640 1 --2- 172.21.15.2:0/3490110220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd294107560 0x7fd294197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd298d1d640 1 -- 172.21.15.2:0/3490110220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd294101e60 msgr2=0x7fd29419dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd298d1d640 1 --2- 172.21.15.2:0/3490110220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd294101e60 0x7fd29419dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd298d1d640 1 -- 172.21.15.2:0/3490110220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd294107560 msgr2=0x7fd294197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd298d1d640 1 --2- 172.21.15.2:0/3490110220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd294107560 0x7fd294197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd298d1d640 1 -- 172.21.15.2:0/3490110220 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c009900 con 0x7fd2941014a0 2024-03-15T13:01:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd298d1d640 1 --2- 172.21.15.2:0/3490110220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2941014a0 0x7fd29419d6e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd28800d950 tx=0x7fd28800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd291ffb640 1 -- 172.21.15.2:0/3490110220 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288009a30 con 0x7fd2941014a0 2024-03-15T13:01:27.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd291ffb640 1 -- 172.21.15.2:0/3490110220 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd288009b90 con 0x7fd2941014a0 2024-03-15T13:01:27.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd29af81640 1 -- 172.21.15.2:0/3490110220 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd294198500 con 0x7fd2941014a0 2024-03-15T13:01:27.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.795+0000 7fd291ffb640 1 -- 172.21.15.2:0/3490110220 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2880047a0 con 0x7fd2941014a0 2024-03-15T13:01:27.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.796+0000 7fd29af81640 1 -- 172.21.15.2:0/3490110220 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2941a4710 con 0x7fd2941014a0 2024-03-15T13:01:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.798+0000 7fd291ffb640 1 -- 172.21.15.2:0/3490110220 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fd288009d00 con 0x7fd2941014a0 2024-03-15T13:01:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.798+0000 7fd29d7fd740 1 Processor -- start 2024-03-15T13:01:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.798+0000 7fd29d7fd740 1 -- start start 2024-03-15T13:01:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.798+0000 7fd291ffb640 1 --2- 172.21.15.2:0/3490110220 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd2840a1c20 0x7fd2840a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.798+0000 7fd29d7fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26a9250 0x55ddd26a9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.798+0000 7fd29d7fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9b70 0x55ddd26b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd29d7fd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26b2750 0x55ddd26b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd29d7fd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd2664960 con 0x55ddd26b2750 2024-03-15T13:01:27.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd29d7fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd2618310 con 0x55ddd26a9b70 2024-03-15T13:01:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd29d7fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd2615840 con 0x55ddd26a9250 2024-03-15T13:01:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd29951e640 1 --2- 172.21.15.2:0/3490110220 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd2840a1c20 0x7fd2840a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd291ffb640 1 -- 172.21.15.2:0/3490110220 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fd2880cd4c0 con 0x7fd2941014a0 2024-03-15T13:01:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd298d1d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26a9250 0x55ddd26a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd293fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9b70 0x55ddd26b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd298d1d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26a9250 0x55ddd26a9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:49046/0 (socket says 172.21.15.2:49046) 2024-03-15T13:01:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd29951e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26b2750 0x55ddd26b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd298d1d640 1 -- 172.21.15.2:0/3654420109 learned_addr learned my addr 172.21.15.2:0/3654420109 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd298d1d640 1 -- 172.21.15.2:0/3654420109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9b70 msgr2=0x55ddd26b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd298d1d640 1 --2- 172.21.15.2:0/3654420109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9b70 0x55ddd26b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd298d1d640 1 -- 172.21.15.2:0/3654420109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26b2750 msgr2=0x55ddd26b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd298d1d640 1 --2- 172.21.15.2:0/3654420109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26b2750 0x55ddd26b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd298d1d640 1 -- 172.21.15.2:0/3654420109 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddd25d3060 con 0x55ddd26a9250 2024-03-15T13:01:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd29951e640 1 --2- 172.21.15.2:0/3490110220 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd2840a1c20 0x7fd2840a40e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd28c004af0 tx=0x7fd28c032f40 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd298d1d640 1 --2- 172.21.15.2:0/3654420109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26a9250 0x55ddd26a9630 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd288096710 tx=0x7fd28807dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa4f99f0f6cb8e8d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd27effd640 1 -- 172.21.15.2:0/3654420109 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28809de90 con 0x55ddd26a9250 2024-03-15T13:01:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd27effd640 1 -- 172.21.15.2:0/3654420109 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd2880a24e0 con 0x55ddd26a9250 2024-03-15T13:01:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.799+0000 7fd27effd640 1 -- 172.21.15.2:0/3654420109 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28807de40 con 0x55ddd26a9250 2024-03-15T13:01:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.800+0000 7fd29d7fd740 1 -- 172.21.15.2:0/3654420109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26a9250 msgr2=0x55ddd26a9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.800+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/3654420109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26a9250 0x55ddd26a9630 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd288096710 tx=0x7fd28807dc60 comp rx=0 tx=0).stop 2024-03-15T13:01:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.800+0000 7fd29d7fd740 1 -- 172.21.15.2:0/3654420109 shutdown_connections 2024-03-15T13:01:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.800+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/3654420109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26b2750 0x55ddd26b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.800+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/3654420109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9b70 0x55ddd26b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.800+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/3654420109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26a9250 0x55ddd26a9630 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.800+0000 7fd29d7fd740 1 -- 172.21.15.2:0/3654420109 >> 172.21.15.2:0/3654420109 conn(0x55ddd26a3980 msgr2=0x55ddd26a3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.800+0000 7fd29d7fd740 1 -- 172.21.15.2:0/3654420109 shutdown_connections 2024-03-15T13:01:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.800+0000 7fd29d7fd740 1 -- 172.21.15.2:0/3654420109 wait complete. 2024-03-15T13:01:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.800+0000 7fd29d7fd740 1 -- start start 2024-03-15T13:01:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.800+0000 7fd29d7fd740 1 Processor -- start 2024-03-15T13:01:27.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd29d7fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9250 0x55ddd26b2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd29d7fd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26a9b70 0x55ddd26af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd29d7fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26b2750 0x55ddd26af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd29d7fd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd2611020 con 0x55ddd26a9b70 2024-03-15T13:01:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd29d7fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd22e4430 con 0x55ddd26a9250 2024-03-15T13:01:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd29d7fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd22a0e00 con 0x55ddd26b2750 2024-03-15T13:01:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd29d7fd740 10 client.0 _ll_register_callbacks cb 0x55ddd25b8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd293fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26a9b70 0x55ddd26af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd293fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26a9b70 0x55ddd26af3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35100/0 (socket says 172.21.15.2:35100) 2024-03-15T13:01:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd293fff640 1 -- 172.21.15.2:0/3929228453 learned_addr learned my addr 172.21.15.2:0/3929228453 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd298d1d640 1 --2- 172.21.15.2:0/3929228453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9250 0x55ddd26b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd29951e640 1 --2- 172.21.15.2:0/3929228453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26b2750 0x55ddd26af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd293fff640 1 -- 172.21.15.2:0/3929228453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26b2750 msgr2=0x55ddd26af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd293fff640 1 --2- 172.21.15.2:0/3929228453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26b2750 0x55ddd26af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd293fff640 1 -- 172.21.15.2:0/3929228453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9250 msgr2=0x55ddd26b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd293fff640 1 --2- 172.21.15.2:0/3929228453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9250 0x55ddd26b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd293fff640 1 -- 172.21.15.2:0/3929228453 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2941a4710 con 0x55ddd26a9b70 2024-03-15T13:01:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd293fff640 1 --2- 172.21.15.2:0/3929228453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26a9b70 0x55ddd26af3f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd280012700 tx=0x7fd280012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90862f5f7109efe5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.801+0000 7fd273fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:01:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.802+0000 7fd273fff640 1 -- 172.21.15.2:0/3929228453 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280017070 con 0x55ddd26a9b70 2024-03-15T13:01:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.802+0000 7fd273fff640 1 -- 172.21.15.2:0/3929228453 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fd280004020 con 0x55ddd26a9b70 2024-03-15T13:01:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.802+0000 7fd29d7fd740 1 -- 172.21.15.2:0/3929228453 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddd23107e0 con 0x55ddd26a9b70 2024-03-15T13:01:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.802+0000 7fd273fff640 1 -- 172.21.15.2:0/3929228453 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2800041e0 con 0x55ddd26a9b70 2024-03-15T13:01:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.802+0000 7fd29d7fd740 10 client.16461 fetch_fsmap learned FSMap version 34 2024-03-15T13:01:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.802+0000 7fd29d7fd740 1 -- 172.21.15.2:0/3929228453 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ddd260f6a0 con 0x55ddd26a9b70 2024-03-15T13:01:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.802+0000 7fd273fff640 1 -- 172.21.15.2:0/3929228453 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55ddd23107e0 con 0x55ddd26a9b70 2024-03-15T13:01:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.802+0000 7fd273fff640 1 -- 172.21.15.2:0/3929228453 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fd280003840 con 0x55ddd26a9b70 2024-03-15T13:01:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.802+0000 7fd29d7fd740 10 client.16461 fetch_fsmap finished waiting for FSMap version 34 2024-03-15T13:01:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.802+0000 7fd29d7fd740 10 client.16461 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:01:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.802+0000 7fd29d7fd740 20 client.16461 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.802+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/3929228453 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ddd26c28e0 0x55ddd26c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.802+0000 7fd29d7fd740 4 client.16461 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:01:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.802+0000 7fd29d7fd740 1 -- 172.21.15.2:0/3929228453 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddd234d720 con 0x55ddd26c28e0 2024-03-15T13:01:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.802+0000 7fd298d1d640 1 --2- 172.21.15.2:0/3929228453 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ddd26c28e0 0x55ddd26c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.803+0000 7fd298d1d640 1 --2- 172.21.15.2:0/3929228453 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ddd26c28e0 0x55ddd26c4d00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=812b99382604423a server_cookie=2967c91b13e9a093 in_seq=0 out_seq=0 2024-03-15T13:01:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.803+0000 7fd273fff640 10 client.16461 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:01:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.803+0000 7fd273fff640 1 -- 172.21.15.2:0/3929228453 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ddd234d720 con 0x55ddd26c28e0 2024-03-15T13:01:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.803+0000 7fd273fff640 10 client.16461 handle_command_reply: tid=0 2024-03-15T13:01:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.803+0000 7fd29d7fd740 1 client.16461 shutdown 2024-03-15T13:01:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.804+0000 7fd29d7fd740 1 -- 172.21.15.2:0/3929228453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26a9b70 msgr2=0x55ddd26af3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.804+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/3929228453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26a9b70 0x55ddd26af3f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd280012700 tx=0x7fd280012bc0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.804+0000 7fd29d7fd740 1 -- 172.21.15.2:0/3929228453 shutdown_connections 2024-03-15T13:01:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.804+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/3929228453 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ddd26c28e0 0x55ddd26c4d00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.804+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/3929228453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26b2750 0x55ddd26af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.804+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/3929228453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26a9b70 0x55ddd26af3f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.804+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/3929228453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9250 0x55ddd26b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.804+0000 7fd273fff640 0 client.16461 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:01:27.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.804+0000 7fd29d7fd740 1 -- 172.21.15.2:0/3929228453 >> 172.21.15.2:0/3929228453 conn(0x55ddd26a3980 msgr2=0x55ddd26a56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.804+0000 7fd29d7fd740 1 -- 172.21.15.2:0/3929228453 shutdown_connections 2024-03-15T13:01:27.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.804+0000 7fd29d7fd740 1 -- 172.21.15.2:0/3929228453 wait complete. 2024-03-15T13:01:27.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.804+0000 7fd29d7fd740 20 client.16461 trim_cache size 0 max 16384 2024-03-15T13:01:27.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.816+0000 7fd29d7fd740 1 Processor -- start 2024-03-15T13:01:27.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.816+0000 7fd29d7fd740 1 -- start start 2024-03-15T13:01:27.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd29d7fd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26a9250 0x55ddd26aa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd29d7fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9b70 0x55ddd26b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd29d7fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26b2750 0x55ddd26b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd29d7fd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd2596130 con 0x55ddd26a9250 2024-03-15T13:01:27.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd29d7fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd26b4f00 con 0x55ddd26a9b70 2024-03-15T13:01:27.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd29d7fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd26a4550 con 0x55ddd26b2750 2024-03-15T13:01:27.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd298d1d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26a9250 0x55ddd26aa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd298d1d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26a9250 0x55ddd26aa7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35114/0 (socket says 172.21.15.2:35114) 2024-03-15T13:01:27.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd298d1d640 1 -- 172.21.15.2:0/2018937444 learned_addr learned my addr 172.21.15.2:0/2018937444 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd293fff640 1 --2- 172.21.15.2:0/2018937444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9b70 0x55ddd26b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd29951e640 1 --2- 172.21.15.2:0/2018937444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26b2750 0x55ddd26b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd298d1d640 1 -- 172.21.15.2:0/2018937444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26b2750 msgr2=0x55ddd26b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd298d1d640 1 --2- 172.21.15.2:0/2018937444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26b2750 0x55ddd26b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd298d1d640 1 -- 172.21.15.2:0/2018937444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9b70 msgr2=0x55ddd26b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd298d1d640 1 --2- 172.21.15.2:0/2018937444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9b70 0x55ddd26b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd298d1d640 1 -- 172.21.15.2:0/2018937444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddd25abc20 con 0x55ddd26a9250 2024-03-15T13:01:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd298d1d640 1 --2- 172.21.15.2:0/2018937444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26a9250 0x55ddd26aa7e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd288099a40 tx=0x7fd2880963b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c039d5ce29238d36 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd27e7fc640 1 -- 172.21.15.2:0/2018937444 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28809a4c0 con 0x55ddd26a9250 2024-03-15T13:01:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd27e7fc640 1 -- 172.21.15.2:0/2018937444 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd288096420 con 0x55ddd26a9250 2024-03-15T13:01:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.817+0000 7fd27e7fc640 1 -- 172.21.15.2:0/2018937444 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28807d090 con 0x55ddd26a9250 2024-03-15T13:01:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.818+0000 7fd29d7fd740 1 -- 172.21.15.2:0/2018937444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26a9250 msgr2=0x55ddd26aa7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.818+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/2018937444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26a9250 0x55ddd26aa7e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd288099a40 tx=0x7fd2880963b0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.818+0000 7fd29d7fd740 1 -- 172.21.15.2:0/2018937444 shutdown_connections 2024-03-15T13:01:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.818+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/2018937444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26b2750 0x55ddd26b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.818+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/2018937444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9b70 0x55ddd26b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.818+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/2018937444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26a9250 0x55ddd26aa7e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.818+0000 7fd29d7fd740 1 -- 172.21.15.2:0/2018937444 >> 172.21.15.2:0/2018937444 conn(0x55ddd26a3980 msgr2=0x55ddd26af750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.818+0000 7fd29d7fd740 1 -- 172.21.15.2:0/2018937444 shutdown_connections 2024-03-15T13:01:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.818+0000 7fd29d7fd740 1 -- 172.21.15.2:0/2018937444 wait complete. 2024-03-15T13:01:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.818+0000 7fd29d7fd740 1 -- start start 2024-03-15T13:01:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.818+0000 7fd29d7fd740 1 Processor -- start 2024-03-15T13:01:27.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29d7fd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9250 0x55ddd26b6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29d7fd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26a9b70 0x55ddd26b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29d7fd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26b2750 0x55ddd2717160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29d7fd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd27176a0 con 0x55ddd26b2750 2024-03-15T13:01:27.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29d7fd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd2717810 con 0x55ddd26a9250 2024-03-15T13:01:27.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29d7fd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd27179d0 con 0x55ddd26a9b70 2024-03-15T13:01:27.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29d7fd740 10 client.0 _ll_register_callbacks cb 0x7fd280009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:27.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29951e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26b2750 0x55ddd2717160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd293fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26a9b70 0x55ddd26b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd298d1d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9250 0x55ddd26b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29951e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26b2750 0x55ddd2717160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35124/0 (socket says 172.21.15.2:35124) 2024-03-15T13:01:27.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd293fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26a9b70 0x55ddd26b67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:49072/0 (socket says 172.21.15.2:49072) 2024-03-15T13:01:27.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29951e640 1 -- 172.21.15.2:0/663966794 learned_addr learned my addr 172.21.15.2:0/663966794 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29951e640 1 -- 172.21.15.2:0/663966794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26a9b70 msgr2=0x55ddd26b67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29951e640 1 --2- 172.21.15.2:0/663966794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26a9b70 0x55ddd26b67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29951e640 1 -- 172.21.15.2:0/663966794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9250 msgr2=0x55ddd26b6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29951e640 1 --2- 172.21.15.2:0/663966794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9250 0x55ddd26b6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29951e640 1 -- 172.21.15.2:0/663966794 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddd25d3060 con 0x55ddd26b2750 2024-03-15T13:01:27.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd29951e640 1 --2- 172.21.15.2:0/663966794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26b2750 0x55ddd2717160 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd28c02c700 tx=0x7fd28c048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3d34b48fabc5f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd273fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:01:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.819+0000 7fd273fff640 1 -- 172.21.15.2:0/663966794 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c048070 con 0x55ddd26b2750 2024-03-15T13:01:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.820+0000 7fd29d7fd740 1 -- 172.21.15.2:0/663966794 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2880a24e0 con 0x55ddd26b2750 2024-03-15T13:01:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.820+0000 7fd273fff640 1 -- 172.21.15.2:0/663966794 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fd28c034df0 con 0x55ddd26b2750 2024-03-15T13:01:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.820+0000 7fd273fff640 1 -- 172.21.15.2:0/663966794 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd28c0326b0 con 0x55ddd26b2750 2024-03-15T13:01:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.820+0000 7fd29d7fd740 10 client.16467 fetch_fsmap learned FSMap version 34 2024-03-15T13:01:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.820+0000 7fd29d7fd740 1 -- 172.21.15.2:0/663966794 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ddd25d18c0 con 0x55ddd26b2750 2024-03-15T13:01:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.820+0000 7fd273fff640 1 -- 172.21.15.2:0/663966794 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd2880a24e0 con 0x55ddd26b2750 2024-03-15T13:01:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.820+0000 7fd273fff640 1 -- 172.21.15.2:0/663966794 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fd28c0043a0 con 0x55ddd26b2750 2024-03-15T13:01:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.820+0000 7fd29d7fd740 10 client.16467 fetch_fsmap finished waiting for FSMap version 34 2024-03-15T13:01:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.820+0000 7fd29d7fd740 10 client.16467 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:01:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.820+0000 7fd29d7fd740 20 client.16467 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.820+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/663966794 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ddd271f2a0 0x55ddd27216c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.820+0000 7fd29d7fd740 4 client.16467 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:01:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.820+0000 7fd29d7fd740 1 -- 172.21.15.2:0/663966794 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ddd239c890 con 0x55ddd271f2a0 2024-03-15T13:01:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.820+0000 7fd298d1d640 1 --2- 172.21.15.2:0/663966794 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ddd271f2a0 0x55ddd27216c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd298d1d640 1 --2- 172.21.15.2:0/663966794 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ddd271f2a0 0x55ddd27216c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6e4dace021ec5223 server_cookie=b66589d6aee5a56f in_seq=0 out_seq=0 2024-03-15T13:01:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd273fff640 10 client.16467 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:01:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd273fff640 1 -- 172.21.15.2:0/663966794 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ddd239c890 con 0x55ddd271f2a0 2024-03-15T13:01:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd273fff640 10 client.16467 handle_command_reply: tid=0 2024-03-15T13:01:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd29d7fd740 1 client.16467 shutdown 2024-03-15T13:01:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd29d7fd740 1 -- 172.21.15.2:0/663966794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26b2750 msgr2=0x55ddd2717160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/663966794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26b2750 0x55ddd2717160 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd28c02c700 tx=0x7fd28c048000 comp rx=0 tx=0).stop 2024-03-15T13:01:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd29d7fd740 1 -- 172.21.15.2:0/663966794 shutdown_connections 2024-03-15T13:01:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/663966794 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ddd271f2a0 0x55ddd27216c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/663966794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddd26b2750 0x55ddd2717160 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/663966794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddd26a9b70 0x55ddd26b67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd273fff640 0 client.16467 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:01:27.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd29d7fd740 1 --2- 172.21.15.2:0/663966794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ddd26a9250 0x55ddd26b6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd29d7fd740 1 -- 172.21.15.2:0/663966794 >> 172.21.15.2:0/663966794 conn(0x55ddd26a3980 msgr2=0x55ddd26aa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd29d7fd740 1 -- 172.21.15.2:0/663966794 shutdown_connections 2024-03-15T13:01:27.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd29d7fd740 1 -- 172.21.15.2:0/663966794 wait complete. 2024-03-15T13:01:27.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.821+0000 7fd29d7fd740 20 client.16467 trim_cache size 0 max 16384 2024-03-15T13:01:27.813 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:01:27.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.822+0000 7fd29af81640 1 -- 172.21.15.2:0/3490110220 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd2840a1c20 msgr2=0x7fd2840a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.822+0000 7fd29af81640 1 --2- 172.21.15.2:0/3490110220 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd2840a1c20 0x7fd2840a40e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd28c004af0 tx=0x7fd28c032f40 comp rx=0 tx=0).stop 2024-03-15T13:01:27.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.822+0000 7fd29af81640 1 -- 172.21.15.2:0/3490110220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2941014a0 msgr2=0x7fd29419d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.822+0000 7fd29af81640 1 --2- 172.21.15.2:0/3490110220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2941014a0 0x7fd29419d6e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd28800d950 tx=0x7fd28800de10 comp rx=0 tx=0).stop 2024-03-15T13:01:27.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.822+0000 7fd29af81640 1 -- 172.21.15.2:0/3490110220 shutdown_connections 2024-03-15T13:01:27.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.822+0000 7fd29af81640 1 --2- 172.21.15.2:0/3490110220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd294107560 0x7fd294197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.822+0000 7fd29af81640 1 --2- 172.21.15.2:0/3490110220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd294101e60 0x7fd29419dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.822+0000 7fd29af81640 1 --2- 172.21.15.2:0/3490110220 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd2840a1c20 0x7fd2840a40e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.822+0000 7fd29af81640 1 --2- 172.21.15.2:0/3490110220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2941014a0 0x7fd29419d6e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.822+0000 7fd29af81640 1 -- 172.21.15.2:0/3490110220 >> 172.21.15.2:0/3490110220 conn(0x7fd2940788f0 msgr2=0x7fd2940fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.822+0000 7fd29af81640 1 -- 172.21.15.2:0/3490110220 shutdown_connections 2024-03-15T13:01:27.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.822+0000 7fd29af81640 1 -- 172.21.15.2:0/3490110220 wait complete. 2024-03-15T13:01:27.818 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:01:27.818 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:01:27.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.941+0000 7f5b11050640 1 -- 172.21.15.2:0/1291513612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0c100f80 msgr2=0x7f5b0c104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.941+0000 7f5b11050640 1 --2- 172.21.15.2:0/1291513612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0c100f80 0x7f5b0c104fc0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009930 tx=0x7f5b0002bc30 comp rx=0 tx=0).stop 2024-03-15T13:01:27.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.941+0000 7f5b11050640 1 -- 172.21.15.2:0/1291513612 shutdown_connections 2024-03-15T13:01:27.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.941+0000 7f5b11050640 1 --2- 172.21.15.2:0/1291513612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b0c1056a0 0x7f5b0c10d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.941+0000 7f5b11050640 1 --2- 172.21.15.2:0/1291513612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0c100f80 0x7f5b0c104fc0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.941+0000 7f5b11050640 1 --2- 172.21.15.2:0/1291513612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b0c1005d0 0x7f5b0c1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.941+0000 7f5b11050640 1 -- 172.21.15.2:0/1291513612 >> 172.21.15.2:0/1291513612 conn(0x7f5b0c0774f0 msgr2=0x7f5b0c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.942+0000 7f5b11050640 1 -- 172.21.15.2:0/1291513612 shutdown_connections 2024-03-15T13:01:27.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.942+0000 7f5b11050640 1 -- 172.21.15.2:0/1291513612 wait complete. 2024-03-15T13:01:27.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.942+0000 7f5b11050640 1 Processor -- start 2024-03-15T13:01:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.942+0000 7f5b11050640 1 -- start start 2024-03-15T13:01:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b11050640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b0c1005d0 0x7f5b0c19d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b11050640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0c1056a0 0x7f5b0c19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b11050640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b0c197a30 0x7f5b0c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b11050640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c198420 con 0x7f5b0c1056a0 2024-03-15T13:01:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b11050640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c198590 con 0x7f5b0c197a30 2024-03-15T13:01:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b11050640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c198750 con 0x7f5b0c1005d0 2024-03-15T13:01:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b0a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0c1056a0 0x7f5b0c19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b0a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0c1056a0 0x7f5b0c19ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50830/0 (socket says 172.21.15.2:50830) 2024-03-15T13:01:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b0a59c640 1 -- 172.21.15.2:0/3393993333 learned_addr learned my addr 172.21.15.2:0/3393993333 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b0ad9d640 1 --2- 172.21.15.2:0/3393993333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b0c1005d0 0x7f5b0c19d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b0b59e640 1 --2- 172.21.15.2:0/3393993333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b0c197a30 0x7f5b0c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b0a59c640 1 -- 172.21.15.2:0/3393993333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b0c1005d0 msgr2=0x7f5b0c19d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b0a59c640 1 --2- 172.21.15.2:0/3393993333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b0c1005d0 0x7f5b0c19d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b0a59c640 1 -- 172.21.15.2:0/3393993333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b0c197a30 msgr2=0x7f5b0c197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b0a59c640 1 --2- 172.21.15.2:0/3393993333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b0c197a30 0x7f5b0c197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b0a59c640 1 -- 172.21.15.2:0/3393993333 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b00009560 con 0x7f5b0c1056a0 2024-03-15T13:01:27.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b0ad9d640 1 --2- 172.21.15.2:0/3393993333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b0c1005d0 0x7f5b0c19d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:27.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b0a59c640 1 --2- 172.21.15.2:0/3393993333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0c1056a0 0x7f5b0c19ddc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5b00005ab0 tx=0x7f5b0002ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5af3fff640 1 -- 172.21.15.2:0/3393993333 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0002cc30 con 0x7f5b0c1056a0 2024-03-15T13:01:27.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5b11050640 1 -- 172.21.15.2:0/3393993333 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0c1a4740 con 0x7f5b0c1056a0 2024-03-15T13:01:27.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5af3fff640 1 -- 172.21.15.2:0/3393993333 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5b00005430 con 0x7f5b0c1056a0 2024-03-15T13:01:27.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.943+0000 7f5af3fff640 1 -- 172.21.15.2:0/3393993333 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b00003b80 con 0x7f5b0c1056a0 2024-03-15T13:01:27.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.944+0000 7f5b11050640 1 -- 172.21.15.2:0/3393993333 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c1a4bf0 con 0x7f5b0c1056a0 2024-03-15T13:01:27.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.946+0000 7f5b13c2a740 1 Processor -- start 2024-03-15T13:01:27.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.946+0000 7f5af3fff640 1 -- 172.21.15.2:0/3393993333 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f5b00003ce0 con 0x7f5b0c1056a0 2024-03-15T13:01:27.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.946+0000 7f5b13c2a740 1 -- start start 2024-03-15T13:01:27.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.946+0000 7f5b13c2a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 0x561e42a6b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.946+0000 7f5b13c2a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a74210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.946+0000 7f5b13c2a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a74750 0x561e42a76b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.946+0000 7f5b13c2a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e42a26960 con 0x561e42a6bb70 2024-03-15T13:01:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.946+0000 7f5b13c2a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e429da310 con 0x561e42a6b250 2024-03-15T13:01:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.946+0000 7f5b13c2a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e429d7840 con 0x561e42a74750 2024-03-15T13:01:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.946+0000 7f5b0ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a74210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.946+0000 7f5b0ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a74210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50838/0 (socket says 172.21.15.2:50838) 2024-03-15T13:01:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.946+0000 7f5b0ad9d640 1 -- 172.21.15.2:0/3657825325 learned_addr learned my addr 172.21.15.2:0/3657825325 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.946+0000 7f5b0b59e640 1 --2- 172.21.15.2:0/3657825325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 0x561e42a6b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.946+0000 7f5b0a59c640 1 --2- 172.21.15.2:0/3657825325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a74750 0x561e42a76b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.947+0000 7f5af3fff640 1 --2- 172.21.15.2:0/3393993333 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5af40a1ba0 0x7f5af40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.947+0000 7f5b0ad9d640 1 -- 172.21.15.2:0/3657825325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a74750 msgr2=0x561e42a76b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.947+0000 7f5b0ad9d640 1 --2- 172.21.15.2:0/3657825325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a74750 0x561e42a76b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.947+0000 7f5b0ad9d640 1 -- 172.21.15.2:0/3657825325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 msgr2=0x561e42a6b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.947+0000 7f5b0ad9d640 1 --2- 172.21.15.2:0/3657825325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 0x561e42a6b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.947+0000 7f5b0ad9d640 1 -- 172.21.15.2:0/3657825325 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e42995060 con 0x561e42a6bb70 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.947+0000 7f5b0b59e640 1 --2- 172.21.15.2:0/3393993333 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5af40a1ba0 0x7f5af40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.947+0000 7f5af3fff640 1 -- 172.21.15.2:0/3393993333 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5b000ef950 con 0x7f5b0c1056a0 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.947+0000 7f5b0ad9d640 1 --2- 172.21.15.2:0/3657825325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a74210 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0114f0 tx=0x7f5afc0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdd99e91bd4701e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.947+0000 7f5af17fa640 1 -- 172.21.15.2:0/3657825325 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc004020 con 0x561e42a6bb70 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.947+0000 7f5af17fa640 1 -- 172.21.15.2:0/3657825325 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5afc004180 con 0x561e42a6bb70 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.947+0000 7f5af17fa640 1 -- 172.21.15.2:0/3657825325 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00f860 con 0x561e42a6bb70 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.948+0000 7f5b13c2a740 1 -- 172.21.15.2:0/3657825325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 msgr2=0x561e42a74210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.948+0000 7f5b0b59e640 1 --2- 172.21.15.2:0/3393993333 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5af40a1ba0 0x7f5af40a4060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5b04005c40 tx=0x7f5b04004a40 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.948+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/3657825325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a74210 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0114f0 tx=0x7f5afc0119b0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.948+0000 7f5b13c2a740 1 -- 172.21.15.2:0/3657825325 shutdown_connections 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.948+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/3657825325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a74750 0x561e42a76b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.948+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/3657825325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a74210 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.948+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/3657825325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 0x561e42a6b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.948+0000 7f5b13c2a740 1 -- 172.21.15.2:0/3657825325 >> 172.21.15.2:0/3657825325 conn(0x561e42a65980 msgr2=0x561e42a65d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.948+0000 7f5b13c2a740 1 -- 172.21.15.2:0/3657825325 shutdown_connections 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.948+0000 7f5b13c2a740 1 -- 172.21.15.2:0/3657825325 wait complete. 2024-03-15T13:01:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.948+0000 7f5b13c2a740 1 -- start start 2024-03-15T13:01:27.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.948+0000 7f5b13c2a740 1 Processor -- start 2024-03-15T13:01:27.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b13c2a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 0x561e42a740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b13c2a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a6bb70 0x561e42a71430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b13c2a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a74750 0x561e42a71990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b13c2a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e429d3020 con 0x561e42a74750 2024-03-15T13:01:27.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b13c2a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e426a6430 con 0x561e42a6b250 2024-03-15T13:01:27.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b13c2a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e42662e00 con 0x561e42a6bb70 2024-03-15T13:01:27.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b13c2a740 10 client.0 _ll_register_callbacks cb 0x561e4297acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:27.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b0b59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a74750 0x561e42a71990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b0ad9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 0x561e42a740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b0a59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a6bb70 0x561e42a71430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b0ad9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 0x561e42a740a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59562/0 (socket says 172.21.15.2:59562) 2024-03-15T13:01:27.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b0a59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a6bb70 0x561e42a71430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37410/0 (socket says 172.21.15.2:37410) 2024-03-15T13:01:27.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b0b59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a74750 0x561e42a71990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50844/0 (socket says 172.21.15.2:50844) 2024-03-15T13:01:27.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b0ad9d640 1 -- 172.21.15.2:0/4215427001 learned_addr learned my addr 172.21.15.2:0/4215427001 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b0a59c640 1 -- 172.21.15.2:0/4215427001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 msgr2=0x561e42a740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b0a59c640 1 --2- 172.21.15.2:0/4215427001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 0x561e42a740a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b0a59c640 1 -- 172.21.15.2:0/4215427001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a74750 msgr2=0x561e42a71990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b0a59c640 1 --2- 172.21.15.2:0/4215427001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a74750 0x561e42a71990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b0a59c640 1 -- 172.21.15.2:0/4215427001 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5afc009cc0 con 0x561e42a6bb70 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b0b59e640 1 --2- 172.21.15.2:0/4215427001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a74750 0x561e42a71990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b0a59c640 1 --2- 172.21.15.2:0/4215427001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a6bb70 0x561e42a71430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5b000ece60 tx=0x7f5b000058b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ddfee0ff9c3df4 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5aeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5aeeff5640 1 -- 172.21.15.2:0/4215427001 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b000bc110 con 0x561e42a6bb70 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5b13c2a740 1 -- 172.21.15.2:0/4215427001 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e426d27e0 con 0x561e42a6bb70 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.949+0000 7f5aeeff5640 1 -- 172.21.15.2:0/4215427001 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5b000bca40 con 0x561e42a6bb70 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.950+0000 7f5aeeff5640 1 -- 172.21.15.2:0/4215427001 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b000bbe80 con 0x561e42a6bb70 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.950+0000 7f5b13c2a740 10 client.26275 fetch_fsmap learned FSMap version 35 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.950+0000 7f5b13c2a740 1 -- 172.21.15.2:0/4215427001 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x561e429d16a0 con 0x561e42a6bb70 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.950+0000 7f5aeeff5640 1 -- 172.21.15.2:0/4215427001 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x561e426d27e0 con 0x561e42a6bb70 2024-03-15T13:01:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.950+0000 7f5aeeff5640 1 -- 172.21.15.2:0/4215427001 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 35) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f5b000f7030 con 0x561e42a6bb70 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.950+0000 7f5b13c2a740 10 client.26275 fetch_fsmap finished waiting for FSMap version 35 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.950+0000 7f5b13c2a740 10 client.26275 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.950+0000 7f5b13c2a740 20 client.26275 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.950+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/4215427001 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561e42a846a0 0x561e42a86ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.950+0000 7f5b13c2a740 4 client.26275 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.950+0000 7f5b13c2a740 1 -- 172.21.15.2:0/4215427001 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e4270f720 con 0x561e42a846a0 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.950+0000 7f5b0ad9d640 1 --2- 172.21.15.2:0/4215427001 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561e42a846a0 0x561e42a86ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.950+0000 7f5aeeff5640 10 client.26275 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.950+0000 7f5b0ad9d640 1 --2- 172.21.15.2:0/4215427001 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561e42a846a0 0x561e42a86ac0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bcd0d2016fd647cd server_cookie=8566e8ca38e0fcc5 in_seq=0 out_seq=0 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.951+0000 7f5aeeff5640 1 -- 172.21.15.2:0/4215427001 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561e4270f720 con 0x561e42a846a0 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.951+0000 7f5aeeff5640 10 client.26275 handle_command_reply: tid=0 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.951+0000 7f5b13c2a740 1 client.26275 shutdown 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.951+0000 7f5b13c2a740 1 -- 172.21.15.2:0/4215427001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a6bb70 msgr2=0x561e42a71430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.951+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/4215427001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a6bb70 0x561e42a71430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5b000ece60 tx=0x7f5b000058b0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.952+0000 7f5b13c2a740 1 -- 172.21.15.2:0/4215427001 shutdown_connections 2024-03-15T13:01:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.952+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/4215427001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a74750 0x561e42a71990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.952+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/4215427001 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561e42a846a0 0x561e42a86ac0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.952+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/4215427001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a6bb70 0x561e42a71430 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.952+0000 7f5aeeff5640 0 client.26275 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:01:27.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.952+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/4215427001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 0x561e42a740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.952+0000 7f5b13c2a740 1 -- 172.21.15.2:0/4215427001 >> 172.21.15.2:0/4215427001 conn(0x561e42a65980 msgr2=0x561e42a676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.952+0000 7f5b13c2a740 1 -- 172.21.15.2:0/4215427001 shutdown_connections 2024-03-15T13:01:27.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.952+0000 7f5b13c2a740 1 -- 172.21.15.2:0/4215427001 wait complete. 2024-03-15T13:01:27.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.952+0000 7f5b13c2a740 20 client.26275 trim_cache size 0 max 16384 2024-03-15T13:01:27.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.965+0000 7f5b13c2a740 1 Processor -- start 2024-03-15T13:01:27.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.965+0000 7f5b13c2a740 1 -- start start 2024-03-15T13:01:27.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.965+0000 7f5b13c2a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a6b250 0x561e42a73b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.965+0000 7f5b13c2a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.965+0000 7f5b13c2a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a74750 0x561e42a7fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.965+0000 7f5b13c2a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e42958130 con 0x561e42a6bb70 2024-03-15T13:01:27.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.965+0000 7f5b13c2a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e42a66550 con 0x561e42a74750 2024-03-15T13:01:27.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.965+0000 7f5b13c2a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e42a87120 con 0x561e42a6b250 2024-03-15T13:01:27.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.965+0000 7f5b0a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.966+0000 7f5b0a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a740a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50850/0 (socket says 172.21.15.2:50850) 2024-03-15T13:01:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.966+0000 7f5b0ad9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a6b250 0x561e42a73b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.966+0000 7f5b0a59c640 1 -- 172.21.15.2:0/2653012201 learned_addr learned my addr 172.21.15.2:0/2653012201 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.966+0000 7f5b0b59e640 1 --2- 172.21.15.2:0/2653012201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a74750 0x561e42a7fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.966+0000 7f5b0a59c640 1 -- 172.21.15.2:0/2653012201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a6b250 msgr2=0x561e42a73b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.966+0000 7f5b0a59c640 1 --2- 172.21.15.2:0/2653012201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a6b250 0x561e42a73b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.966+0000 7f5b0a59c640 1 -- 172.21.15.2:0/2653012201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a74750 msgr2=0x561e42a7fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.966+0000 7f5b0a59c640 1 --2- 172.21.15.2:0/2653012201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a74750 0x561e42a7fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.966+0000 7f5b0a59c640 1 -- 172.21.15.2:0/2653012201 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e4296dc20 con 0x561e42a6bb70 2024-03-15T13:01:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.966+0000 7f5b0a59c640 1 --2- 172.21.15.2:0/2653012201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a740a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009900 tx=0x7f5b0002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81271ecdda145fec server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.966+0000 7f5af0ff9640 1 -- 172.21.15.2:0/2653012201 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b000bbe80 con 0x561e42a6bb70 2024-03-15T13:01:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.966+0000 7f5af0ff9640 1 -- 172.21.15.2:0/2653012201 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5b000bc110 con 0x561e42a6bb70 2024-03-15T13:01:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.966+0000 7f5af0ff9640 1 -- 172.21.15.2:0/2653012201 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b000c4ae0 con 0x561e42a6bb70 2024-03-15T13:01:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 -- 172.21.15.2:0/2653012201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 msgr2=0x561e42a740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/2653012201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a740a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009900 tx=0x7f5b0002bc30 comp rx=0 tx=0).stop 2024-03-15T13:01:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 -- 172.21.15.2:0/2653012201 shutdown_connections 2024-03-15T13:01:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/2653012201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a74750 0x561e42a7fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/2653012201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a740a0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/2653012201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a6b250 0x561e42a73b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 -- 172.21.15.2:0/2653012201 >> 172.21.15.2:0/2653012201 conn(0x561e42a65980 msgr2=0x561e42a64b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 -- 172.21.15.2:0/2653012201 shutdown_connections 2024-03-15T13:01:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 -- 172.21.15.2:0/2653012201 wait complete. 2024-03-15T13:01:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 -- start start 2024-03-15T13:01:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 Processor -- start 2024-03-15T13:01:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 0x561e42a7bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a7c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a74750 0x561e42a78170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e42a787d0 con 0x561e42a6bb70 2024-03-15T13:01:27.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e42a78940 con 0x561e42a6b250 2024-03-15T13:01:27.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.967+0000 7f5b13c2a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e42a78b00 con 0x561e42a74750 2024-03-15T13:01:27.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5b13c2a740 10 client.0 _ll_register_callbacks cb 0x7f5b0002cd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:27.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5b0a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a7c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5b0a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a7c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50858/0 (socket says 172.21.15.2:50858) 2024-03-15T13:01:27.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5b0a59c640 1 -- 172.21.15.2:0/3033854538 learned_addr learned my addr 172.21.15.2:0/3033854538 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:27.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5b0ad9d640 1 --2- 172.21.15.2:0/3033854538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 0x561e42a7bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5b0b59e640 1 --2- 172.21.15.2:0/3033854538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a74750 0x561e42a78170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5b0a59c640 1 -- 172.21.15.2:0/3033854538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a74750 msgr2=0x561e42a78170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5b0a59c640 1 --2- 172.21.15.2:0/3033854538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a74750 0x561e42a78170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5b0a59c640 1 -- 172.21.15.2:0/3033854538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 msgr2=0x561e42a7bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:27.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5b0a59c640 1 --2- 172.21.15.2:0/3033854538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 0x561e42a7bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5b0a59c640 1 -- 172.21.15.2:0/3033854538 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e429d16a0 con 0x561e42a6bb70 2024-03-15T13:01:27.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5b0a59c640 1 --2- 172.21.15.2:0/3033854538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a7c230 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5b00006010 tx=0x7f5b000f55c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3884f1e05e6f433 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:27.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5aeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:01:27.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5aeeff5640 1 -- 172.21.15.2:0/3033854538 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b00005160 con 0x561e42a6bb70 2024-03-15T13:01:27.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5aeeff5640 1 -- 172.21.15.2:0/3033854538 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5b000bca40 con 0x561e42a6bb70 2024-03-15T13:01:27.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5b13c2a740 1 -- 172.21.15.2:0/3033854538 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e42a610f0 con 0x561e42a6bb70 2024-03-15T13:01:27.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5aeeff5640 1 -- 172.21.15.2:0/3033854538 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b000052c0 con 0x561e42a6bb70 2024-03-15T13:01:27.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5b13c2a740 10 client.16485 fetch_fsmap learned FSMap version 35 2024-03-15T13:01:27.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5b13c2a740 1 -- 172.21.15.2:0/3033854538 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x561e429938c0 con 0x561e42a6bb70 2024-03-15T13:01:27.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.968+0000 7f5aeeff5640 1 -- 172.21.15.2:0/3033854538 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x561e42a610f0 con 0x561e42a6bb70 2024-03-15T13:01:27.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.969+0000 7f5aeeff5640 1 -- 172.21.15.2:0/3033854538 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 35) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f5b000f7070 con 0x561e42a6bb70 2024-03-15T13:01:27.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.969+0000 7f5b13c2a740 10 client.16485 fetch_fsmap finished waiting for FSMap version 35 2024-03-15T13:01:27.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.969+0000 7f5b13c2a740 10 client.16485 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:01:27.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.969+0000 7f5b13c2a740 20 client.16485 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:27.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.969+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/3033854538 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561e42ae0ba0 0x561e42ae2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:27.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.969+0000 7f5b13c2a740 4 client.16485 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.969+0000 7f5b13c2a740 1 -- 172.21.15.2:0/3033854538 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f5afc004180 con 0x561e42ae0ba0 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.969+0000 7f5b0ad9d640 1 --2- 172.21.15.2:0/3033854538 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561e42ae0ba0 0x561e42ae2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.969+0000 7f5b0ad9d640 1 --2- 172.21.15.2:0/3033854538 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561e42ae0ba0 0x561e42ae2f90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b123ca3684ef83f server_cookie=d858840ad52b3a96 in_seq=0 out_seq=0 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.969+0000 7f5aeeff5640 10 client.16485 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.969+0000 7f5aeeff5640 1 -- 172.21.15.2:0/3033854538 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f5afc004180 con 0x561e42ae0ba0 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.969+0000 7f5aeeff5640 10 client.16485 handle_command_reply: tid=0 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.969+0000 7f5b13c2a740 1 client.16485 shutdown 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b13c2a740 1 -- 172.21.15.2:0/3033854538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 msgr2=0x561e42a7c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/3033854538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a7c230 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5b00006010 tx=0x7f5b000f55c0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b13c2a740 1 -- 172.21.15.2:0/3033854538 shutdown_connections 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/3033854538 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561e42ae0ba0 0x561e42ae2f90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/3033854538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561e42a74750 0x561e42a78170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5aeeff5640 0 client.16485 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/3033854538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561e42a6bb70 0x561e42a7c230 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b13c2a740 1 --2- 172.21.15.2:0/3033854538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e42a6b250 0x561e42a7bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b13c2a740 1 -- 172.21.15.2:0/3033854538 >> 172.21.15.2:0/3033854538 conn(0x561e42a65980 msgr2=0x561e42a6bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b13c2a740 1 -- 172.21.15.2:0/3033854538 shutdown_connections 2024-03-15T13:01:27.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b13c2a740 1 -- 172.21.15.2:0/3033854538 wait complete. 2024-03-15T13:01:27.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b13c2a740 20 client.16485 trim_cache size 0 max 16384 2024-03-15T13:01:27.962 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:01:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b11050640 1 -- 172.21.15.2:0/3393993333 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5af40a1ba0 msgr2=0x7f5af40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b11050640 1 --2- 172.21.15.2:0/3393993333 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5af40a1ba0 0x7f5af40a4060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5b04005c40 tx=0x7f5b04004a40 comp rx=0 tx=0).stop 2024-03-15T13:01:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b11050640 1 -- 172.21.15.2:0/3393993333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0c1056a0 msgr2=0x7f5b0c19ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b11050640 1 --2- 172.21.15.2:0/3393993333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0c1056a0 0x7f5b0c19ddc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5b00005ab0 tx=0x7f5b0002ca40 comp rx=0 tx=0).stop 2024-03-15T13:01:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b11050640 1 -- 172.21.15.2:0/3393993333 shutdown_connections 2024-03-15T13:01:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b11050640 1 --2- 172.21.15.2:0/3393993333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b0c197a30 0x7f5b0c197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b11050640 1 --2- 172.21.15.2:0/3393993333 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5af40a1ba0 0x7f5af40a4060 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b11050640 1 --2- 172.21.15.2:0/3393993333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0c1056a0 0x7f5b0c19ddc0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b11050640 1 --2- 172.21.15.2:0/3393993333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b0c1005d0 0x7f5b0c19d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b11050640 1 -- 172.21.15.2:0/3393993333 >> 172.21.15.2:0/3393993333 conn(0x7f5b0c0774f0 msgr2=0x7f5b0c0689d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b11050640 1 -- 172.21.15.2:0/3393993333 shutdown_connections 2024-03-15T13:01:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:27.970+0000 7f5b11050640 1 -- 172.21.15.2:0/3393993333 wait complete. 2024-03-15T13:01:27.965 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:27 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2668507246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:01:27.966 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:01:27.967 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:01:28.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.089+0000 7feaf4a1c640 1 -- 172.21.15.2:0/321128519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf01005d0 msgr2=0x7feaf01009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.089+0000 7feaf4a1c640 1 --2- 172.21.15.2:0/321128519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf01005d0 0x7feaf01009b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7feae0009930 tx=0x7feae002bc30 comp rx=0 tx=0).stop 2024-03-15T13:01:28.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.089+0000 7feaf4a1c640 1 -- 172.21.15.2:0/321128519 shutdown_connections 2024-03-15T13:01:28.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.089+0000 7feaf4a1c640 1 --2- 172.21.15.2:0/321128519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaf01056a0 0x7feaf010d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.089+0000 7feaf4a1c640 1 --2- 172.21.15.2:0/321128519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaf0100f80 0x7feaf0104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.089+0000 7feaf4a1c640 1 --2- 172.21.15.2:0/321128519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf01005d0 0x7feaf01009b0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.089+0000 7feaf4a1c640 1 -- 172.21.15.2:0/321128519 >> 172.21.15.2:0/321128519 conn(0x7feaf00774f0 msgr2=0x7feaf0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.089+0000 7feaf4a1c640 1 -- 172.21.15.2:0/321128519 shutdown_connections 2024-03-15T13:01:28.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.089+0000 7feaf4a1c640 1 -- 172.21.15.2:0/321128519 wait complete. 2024-03-15T13:01:28.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.090+0000 7feaf4a1c640 1 Processor -- start 2024-03-15T13:01:28.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.090+0000 7feaf4a1c640 1 -- start start 2024-03-15T13:01:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.090+0000 7feaf4a1c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf01005d0 0x7feaf010ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.090+0000 7feaf4a1c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaf01056a0 0x7feaf010f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.090+0000 7feaf4a1c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaf010f9d0 0x7feaf01a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.090+0000 7feaf4a1c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf01a6360 con 0x7feaf01005d0 2024-03-15T13:01:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.090+0000 7feaf4a1c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf01a64d0 con 0x7feaf01056a0 2024-03-15T13:01:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.090+0000 7feaf4a1c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf01a6690 con 0x7feaf010f9d0 2024-03-15T13:01:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.090+0000 7feaee59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf01005d0 0x7feaf010ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.090+0000 7feaee59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf01005d0 0x7feaf010ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50876/0 (socket says 172.21.15.2:50876) 2024-03-15T13:01:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.090+0000 7feaee59c640 1 -- 172.21.15.2:0/1141123252 learned_addr learned my addr 172.21.15.2:0/1141123252 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.090+0000 7feae7fff640 1 --2- 172.21.15.2:0/1141123252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaf01056a0 0x7feaf010f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.090+0000 7feaeed9d640 1 --2- 172.21.15.2:0/1141123252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaf010f9d0 0x7feaf01a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.091+0000 7feaee59c640 1 -- 172.21.15.2:0/1141123252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaf010f9d0 msgr2=0x7feaf01a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.091+0000 7feaee59c640 1 --2- 172.21.15.2:0/1141123252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaf010f9d0 0x7feaf01a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.091+0000 7feaee59c640 1 -- 172.21.15.2:0/1141123252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaf01056a0 msgr2=0x7feaf010f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.091+0000 7feaee59c640 1 --2- 172.21.15.2:0/1141123252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaf01056a0 0x7feaf010f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.091+0000 7feaee59c640 1 -- 172.21.15.2:0/1141123252 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae0009560 con 0x7feaf01005d0 2024-03-15T13:01:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.091+0000 7feaee59c640 1 --2- 172.21.15.2:0/1141123252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf01005d0 0x7feaf010ef50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7feae0009900 tx=0x7feae002ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.091+0000 7feae77fe640 1 -- 172.21.15.2:0/1141123252 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae002cc10 con 0x7feaf01005d0 2024-03-15T13:01:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.091+0000 7feaf4a1c640 1 -- 172.21.15.2:0/1141123252 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf01a6910 con 0x7feaf01005d0 2024-03-15T13:01:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.091+0000 7feae77fe640 1 -- 172.21.15.2:0/1141123252 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7feae00054d0 con 0x7feaf01005d0 2024-03-15T13:01:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.091+0000 7feae77fe640 1 -- 172.21.15.2:0/1141123252 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0003c90 con 0x7feaf01005d0 2024-03-15T13:01:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.091+0000 7feaf4a1c640 1 -- 172.21.15.2:0/1141123252 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf01a6e70 con 0x7feaf01005d0 2024-03-15T13:01:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.093+0000 7feae77fe640 1 -- 172.21.15.2:0/1141123252 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7feae0005000 con 0x7feaf01005d0 2024-03-15T13:01:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.093+0000 7feaf75f6740 1 Processor -- start 2024-03-15T13:01:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaf75f6740 1 -- start start 2024-03-15T13:01:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaf75f6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59ec250 0x5567e59ec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaf75f6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ecb70 0x5567e59f5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaf75f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59f5750 0x5567e59f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feae77fe640 1 --2- 172.21.15.2:0/1141123252 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feadc0a1bd0 0x7feadc0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaf75f6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e59a7960 con 0x5567e59ecb70 2024-03-15T13:01:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaf75f6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e595b310 con 0x5567e59ec250 2024-03-15T13:01:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaf75f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e5958840 con 0x5567e59f5750 2024-03-15T13:01:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaee59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ecb70 0x5567e59f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaeed9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59ec250 0x5567e59ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feae7fff640 1 --2- 172.21.15.2:0/1141123252 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feadc0a1bd0 0x7feadc0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaeed9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59ec250 0x5567e59ec630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59604/0 (socket says 172.21.15.2:59604) 2024-03-15T13:01:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaeed9d640 1 -- 172.21.15.2:0/1617330577 learned_addr learned my addr 172.21.15.2:0/1617330577 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaee59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ecb70 0x5567e59f5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50886/0 (socket says 172.21.15.2:50886) 2024-03-15T13:01:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaeed9d640 1 --2- 172.21.15.2:0/1617330577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59f5750 0x5567e59f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feae77fe640 1 -- 172.21.15.2:0/1141123252 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7feae00efaf0 con 0x7feaf01005d0 2024-03-15T13:01:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaeed9d640 1 -- 172.21.15.2:0/1617330577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59f5750 msgr2=0x5567e59f7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaeed9d640 1 --2- 172.21.15.2:0/1617330577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59f5750 0x5567e59f7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaeed9d640 1 -- 172.21.15.2:0/1617330577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ecb70 msgr2=0x5567e59f5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaeed9d640 1 --2- 172.21.15.2:0/1617330577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ecb70 0x5567e59f5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.094+0000 7feaeed9d640 1 -- 172.21.15.2:0/1617330577 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567e5916060 con 0x5567e59ec250 2024-03-15T13:01:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.095+0000 7feae7fff640 1 --2- 172.21.15.2:0/1141123252 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feadc0a1bd0 0x7feadc0a4090 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feaf01109e0 tx=0x7fead8005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.095+0000 7feaeed9d640 1 --2- 172.21.15.2:0/1617330577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59ec250 0x5567e59ec630 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feae8005f40 tx=0x7feae8005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff7623eb4c8b398 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.095+0000 7feae4ff9640 1 -- 172.21.15.2:0/1617330577 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8004b20 con 0x5567e59ec250 2024-03-15T13:01:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.095+0000 7feae4ff9640 1 -- 172.21.15.2:0/1617330577 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7feae8004c80 con 0x5567e59ec250 2024-03-15T13:01:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.095+0000 7feae4ff9640 1 -- 172.21.15.2:0/1617330577 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8005440 con 0x5567e59ec250 2024-03-15T13:01:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 -- 172.21.15.2:0/1617330577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59ec250 msgr2=0x5567e59ec630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 --2- 172.21.15.2:0/1617330577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59ec250 0x5567e59ec630 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feae8005f40 tx=0x7feae8005bd0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 -- 172.21.15.2:0/1617330577 shutdown_connections 2024-03-15T13:01:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 --2- 172.21.15.2:0/1617330577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59f5750 0x5567e59f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 --2- 172.21.15.2:0/1617330577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ecb70 0x5567e59f5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 --2- 172.21.15.2:0/1617330577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59ec250 0x5567e59ec630 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 -- 172.21.15.2:0/1617330577 >> 172.21.15.2:0/1617330577 conn(0x5567e59e6980 msgr2=0x5567e59e6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 -- 172.21.15.2:0/1617330577 shutdown_connections 2024-03-15T13:01:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 -- 172.21.15.2:0/1617330577 wait complete. 2024-03-15T13:01:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 -- start start 2024-03-15T13:01:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 Processor -- start 2024-03-15T13:01:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59ec250 0x5567e59f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 0x5567e59f2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59f5750 0x5567e59f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e5954020 con 0x5567e59f5750 2024-03-15T13:01:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e5627430 con 0x5567e59ec250 2024-03-15T13:01:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.096+0000 7feaf75f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e55e3e00 con 0x5567e59ecb70 2024-03-15T13:01:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feaf75f6740 10 client.0 _ll_register_callbacks cb 0x5567e58fbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feaee59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59ec250 0x5567e59f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feae7fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 0x5567e59f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feaeed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59f5750 0x5567e59f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feaee59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59ec250 0x5567e59f50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59606/0 (socket says 172.21.15.2:59606) 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feaee59c640 1 -- 172.21.15.2:0/3093937117 learned_addr learned my addr 172.21.15.2:0/3093937117 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feae7fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 0x5567e59f2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37458/0 (socket says 172.21.15.2:37458) 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feaeed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59f5750 0x5567e59f2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50898/0 (socket says 172.21.15.2:50898) 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feaeed9d640 1 -- 172.21.15.2:0/3093937117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 msgr2=0x5567e59f2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feaeed9d640 1 --2- 172.21.15.2:0/3093937117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 0x5567e59f2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feaeed9d640 1 -- 172.21.15.2:0/3093937117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59ec250 msgr2=0x5567e59f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feaeed9d640 1 --2- 172.21.15.2:0/3093937117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59ec250 0x5567e59f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feaeed9d640 1 -- 172.21.15.2:0/3093937117 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae8019040 con 0x5567e59f5750 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feaee59c640 1 --2- 172.21.15.2:0/3093937117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59ec250 0x5567e59f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feae7fff640 1 --2- 172.21.15.2:0/3093937117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 0x5567e59f2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feaeed9d640 1 --2- 172.21.15.2:0/3093937117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59f5750 0x5567e59f2a20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feae80051a0 tx=0x7feae800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e75f97ceccc41c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7fead5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:01:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7fead5ffb640 1 -- 172.21.15.2:0/3093937117 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8025070 con 0x5567e59f5750 2024-03-15T13:01:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7fead5ffb640 1 -- 172.21.15.2:0/3093937117 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7feae8004df0 con 0x5567e59f5750 2024-03-15T13:01:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feaf75f6740 1 -- 172.21.15.2:0/3093937117 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567e56537e0 con 0x5567e59f5750 2024-03-15T13:01:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7fead5ffb640 1 -- 172.21.15.2:0/3093937117 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7feae8005240 con 0x5567e59f5750 2024-03-15T13:01:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.097+0000 7feaf75f6740 10 client.16497 fetch_fsmap learned FSMap version 35 2024-03-15T13:01:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.098+0000 7feaf75f6740 1 -- 172.21.15.2:0/3093937117 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5567e59526a0 con 0x5567e59f5750 2024-03-15T13:01:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.098+0000 7fead5ffb640 1 -- 172.21.15.2:0/3093937117 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5567e56537e0 con 0x5567e59f5750 2024-03-15T13:01:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.098+0000 7fead5ffb640 1 -- 172.21.15.2:0/3093937117 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 35) v1 ==== 7978+0+0 (secure 0 0 0) 0x7feae801f6e0 con 0x5567e59f5750 2024-03-15T13:01:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.098+0000 7feaf75f6740 10 client.16497 fetch_fsmap finished waiting for FSMap version 35 2024-03-15T13:01:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.098+0000 7feaf75f6740 10 client.16497 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:01:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.098+0000 7feaf75f6740 20 client.16497 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.098+0000 7feaf75f6740 1 --2- 172.21.15.2:0/3093937117 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5567e5a05830 0x5567e5a07c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.098+0000 7feaf75f6740 4 client.16497 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:01:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.098+0000 7feaf75f6740 1 -- 172.21.15.2:0/3093937117 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567e5690720 con 0x5567e5a05830 2024-03-15T13:01:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.098+0000 7feaee59c640 1 --2- 172.21.15.2:0/3093937117 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5567e5a05830 0x5567e5a07c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.099+0000 7feaee59c640 1 --2- 172.21.15.2:0/3093937117 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5567e5a05830 0x5567e5a07c50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b24dd9fc610ffc93 server_cookie=c6e3bfff09185936 in_seq=0 out_seq=0 2024-03-15T13:01:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.099+0000 7fead5ffb640 10 client.16497 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:01:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.099+0000 7fead5ffb640 1 -- 172.21.15.2:0/3093937117 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5567e5690720 con 0x5567e5a05830 2024-03-15T13:01:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.099+0000 7fead5ffb640 10 client.16497 handle_command_reply: tid=0 2024-03-15T13:01:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.100+0000 7feaf75f6740 1 client.16497 shutdown 2024-03-15T13:01:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.100+0000 7feaf75f6740 1 -- 172.21.15.2:0/3093937117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59f5750 msgr2=0x5567e59f2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.100+0000 7feaf75f6740 1 --2- 172.21.15.2:0/3093937117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59f5750 0x5567e59f2a20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feae80051a0 tx=0x7feae800fce0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.100+0000 7feaf75f6740 1 -- 172.21.15.2:0/3093937117 shutdown_connections 2024-03-15T13:01:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.100+0000 7feaf75f6740 1 --2- 172.21.15.2:0/3093937117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59f5750 0x5567e59f2a20 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.100+0000 7feaf75f6740 1 --2- 172.21.15.2:0/3093937117 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5567e5a05830 0x5567e5a07c50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.100+0000 7feaf75f6740 1 --2- 172.21.15.2:0/3093937117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 0x5567e59f2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.100+0000 7feaf75f6740 1 --2- 172.21.15.2:0/3093937117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59ec250 0x5567e59f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.100+0000 7fead5ffb640 0 client.16497 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:01:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.100+0000 7feaf75f6740 1 -- 172.21.15.2:0/3093937117 >> 172.21.15.2:0/3093937117 conn(0x5567e59e6980 msgr2=0x5567e59e86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.100+0000 7feaf75f6740 1 -- 172.21.15.2:0/3093937117 shutdown_connections 2024-03-15T13:01:28.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.100+0000 7feaf75f6740 1 -- 172.21.15.2:0/3093937117 wait complete. 2024-03-15T13:01:28.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.100+0000 7feaf75f6740 20 client.16497 trim_cache size 0 max 16384 2024-03-15T13:01:28.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaf75f6740 1 Processor -- start 2024-03-15T13:01:28.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaf75f6740 1 -- start start 2024-03-15T13:01:28.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaf75f6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ec250 0x5567e59e52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaf75f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 0x5567e59f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaf75f6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59f5750 0x5567e59f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaf75f6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e58d9130 con 0x5567e59ec250 2024-03-15T13:01:28.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaf75f6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e59e64b0 con 0x5567e59f5750 2024-03-15T13:01:28.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaf75f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e59e5860 con 0x5567e59ecb70 2024-03-15T13:01:28.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaee59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ec250 0x5567e59e52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaee59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ec250 0x5567e59e52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50912/0 (socket says 172.21.15.2:50912) 2024-03-15T13:01:28.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaee59c640 1 -- 172.21.15.2:0/3502001604 learned_addr learned my addr 172.21.15.2:0/3502001604 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:28.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feae7fff640 1 --2- 172.21.15.2:0/3502001604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 0x5567e59f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaeed9d640 1 --2- 172.21.15.2:0/3502001604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59f5750 0x5567e59f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaee59c640 1 -- 172.21.15.2:0/3502001604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 msgr2=0x5567e59f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaee59c640 1 --2- 172.21.15.2:0/3502001604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 0x5567e59f4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaee59c640 1 -- 172.21.15.2:0/3502001604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59f5750 msgr2=0x5567e59f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaee59c640 1 --2- 172.21.15.2:0/3502001604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59f5750 0x5567e59f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaee59c640 1 -- 172.21.15.2:0/3502001604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567e58eec20 con 0x5567e59ec250 2024-03-15T13:01:28.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaeed9d640 1 --2- 172.21.15.2:0/3502001604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59f5750 0x5567e59f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:01:28.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7feaee59c640 1 --2- 172.21.15.2:0/3502001604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ec250 0x5567e59e52f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7feae0004c40 tx=0x7feae00a7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9845176b5f2f16b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.113+0000 7fead7fff640 1 -- 172.21.15.2:0/3502001604 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae00bcbe0 con 0x5567e59ec250 2024-03-15T13:01:28.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.114+0000 7fead7fff640 1 -- 172.21.15.2:0/3502001604 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7feae00bc4e0 con 0x5567e59ec250 2024-03-15T13:01:28.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.114+0000 7fead7fff640 1 -- 172.21.15.2:0/3502001604 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae00a70b0 con 0x5567e59ec250 2024-03-15T13:01:28.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.114+0000 7feaf75f6740 1 -- 172.21.15.2:0/3502001604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ec250 msgr2=0x5567e59e52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.114+0000 7feaf75f6740 1 --2- 172.21.15.2:0/3502001604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ec250 0x5567e59e52f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7feae0004c40 tx=0x7feae00a7040 comp rx=0 tx=0).stop 2024-03-15T13:01:28.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.114+0000 7feaf75f6740 1 -- 172.21.15.2:0/3502001604 shutdown_connections 2024-03-15T13:01:28.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.114+0000 7feaf75f6740 1 --2- 172.21.15.2:0/3502001604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59f5750 0x5567e59f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.114+0000 7feaf75f6740 1 --2- 172.21.15.2:0/3502001604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 0x5567e59f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.114+0000 7feaf75f6740 1 --2- 172.21.15.2:0/3502001604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ec250 0x5567e59e52f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.114+0000 7feaf75f6740 1 -- 172.21.15.2:0/3502001604 >> 172.21.15.2:0/3502001604 conn(0x5567e59e6980 msgr2=0x5567e59ed920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.114+0000 7feaf75f6740 1 -- 172.21.15.2:0/3502001604 shutdown_connections 2024-03-15T13:01:28.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.114+0000 7feaf75f6740 1 -- 172.21.15.2:0/3502001604 wait complete. 2024-03-15T13:01:28.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaf75f6740 1 -- start start 2024-03-15T13:01:28.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaf75f6740 1 Processor -- start 2024-03-15T13:01:28.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaf75f6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ec250 0x5567e59f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaf75f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 0x5567e59f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaf75f6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59f5750 0x5567e59f9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaf75f6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e59fd930 con 0x5567e59ec250 2024-03-15T13:01:28.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaf75f6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e5a600a0 con 0x5567e59f5750 2024-03-15T13:01:28.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaf75f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e5a60210 con 0x5567e59ecb70 2024-03-15T13:01:28.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaf75f6740 10 client.0 _ll_register_callbacks cb 0x7feae80115d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:28.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaee59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ec250 0x5567e59f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaee59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ec250 0x5567e59f8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50922/0 (socket says 172.21.15.2:50922) 2024-03-15T13:01:28.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaee59c640 1 -- 172.21.15.2:0/538329702 learned_addr learned my addr 172.21.15.2:0/538329702 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:28.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feae7fff640 1 --2- 172.21.15.2:0/538329702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 0x5567e59f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaeed9d640 1 --2- 172.21.15.2:0/538329702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59f5750 0x5567e59f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaee59c640 1 -- 172.21.15.2:0/538329702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 msgr2=0x5567e59f90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaee59c640 1 --2- 172.21.15.2:0/538329702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 0x5567e59f90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaee59c640 1 -- 172.21.15.2:0/538329702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59f5750 msgr2=0x5567e59f9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaee59c640 1 --2- 172.21.15.2:0/538329702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59f5750 0x5567e59f9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.115+0000 7feaee59c640 1 -- 172.21.15.2:0/538329702 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf01a6e70 con 0x5567e59ec250 2024-03-15T13:01:28.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7feaee59c640 1 --2- 172.21.15.2:0/538329702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ec250 0x5567e59f8bb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7feae002bc70 tx=0x7feae00c0740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ccb4034b4cf053 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7fead5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:01:28.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7fead5ffb640 1 -- 172.21.15.2:0/538329702 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae00052b0 con 0x5567e59ec250 2024-03-15T13:01:28.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7fead5ffb640 1 -- 172.21.15.2:0/538329702 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7feae00ba580 con 0x5567e59ec250 2024-03-15T13:01:28.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7feaf75f6740 1 -- 172.21.15.2:0/538329702 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567e59e20f0 con 0x5567e59ec250 2024-03-15T13:01:28.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7fead5ffb640 1 -- 172.21.15.2:0/538329702 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7feae00ba7e0 con 0x5567e59ec250 2024-03-15T13:01:28.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7feaf75f6740 10 client.16503 fetch_fsmap learned FSMap version 35 2024-03-15T13:01:28.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7feaf75f6740 1 -- 172.21.15.2:0/538329702 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5567e59148c0 con 0x5567e59ec250 2024-03-15T13:01:28.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7fead5ffb640 1 -- 172.21.15.2:0/538329702 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5567e59e20f0 con 0x5567e59ec250 2024-03-15T13:01:28.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7fead5ffb640 1 -- 172.21.15.2:0/538329702 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 35) v1 ==== 7978+0+0 (secure 0 0 0) 0x7feae00d7020 con 0x5567e59ec250 2024-03-15T13:01:28.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7feaf75f6740 10 client.16503 fetch_fsmap finished waiting for FSMap version 35 2024-03-15T13:01:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7feaf75f6740 10 client.16503 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:01:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7feaf75f6740 20 client.16503 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7feaf75f6740 1 --2- 172.21.15.2:0/538329702 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5567e5a61530 0x5567e5a63950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7feaf75f6740 4 client.16503 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:01:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7feaf75f6740 1 -- 172.21.15.2:0/538329702 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7feae8004c80 con 0x5567e5a61530 2024-03-15T13:01:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.116+0000 7feae7fff640 1 --2- 172.21.15.2:0/538329702 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5567e5a61530 0x5567e5a63950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7fead5ffb640 10 client.16503 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:01:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7feae7fff640 1 --2- 172.21.15.2:0/538329702 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5567e5a61530 0x5567e5a63950 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=189f6151acdf729d server_cookie=48a49c5bbb99560f in_seq=0 out_seq=0 2024-03-15T13:01:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7fead5ffb640 1 -- 172.21.15.2:0/538329702 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7feae8004c80 con 0x5567e5a61530 2024-03-15T13:01:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7fead5ffb640 10 client.16503 handle_command_reply: tid=0 2024-03-15T13:01:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7feaf75f6740 1 client.16503 shutdown 2024-03-15T13:01:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7feaf75f6740 1 -- 172.21.15.2:0/538329702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ec250 msgr2=0x5567e59f8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7feaf75f6740 1 --2- 172.21.15.2:0/538329702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ec250 0x5567e59f8bb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7feae002bc70 tx=0x7feae00c0740 comp rx=0 tx=0).stop 2024-03-15T13:01:28.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7feaf75f6740 1 -- 172.21.15.2:0/538329702 shutdown_connections 2024-03-15T13:01:28.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7feaf75f6740 1 --2- 172.21.15.2:0/538329702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567e59f5750 0x5567e59f9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7feaf75f6740 1 --2- 172.21.15.2:0/538329702 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5567e5a61530 0x5567e5a63950 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7feaf75f6740 1 --2- 172.21.15.2:0/538329702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5567e59ecb70 0x5567e59f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7feaf75f6740 1 --2- 172.21.15.2:0/538329702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567e59ec250 0x5567e59f8bb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7fead5ffb640 0 client.16503 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:01:28.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7feaf75f6740 1 -- 172.21.15.2:0/538329702 >> 172.21.15.2:0/538329702 conn(0x5567e59e6980 msgr2=0x5567e59f1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7feaf75f6740 1 -- 172.21.15.2:0/538329702 shutdown_connections 2024-03-15T13:01:28.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7feaf75f6740 1 -- 172.21.15.2:0/538329702 wait complete. 2024-03-15T13:01:28.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.117+0000 7feaf75f6740 20 client.16503 trim_cache size 0 max 16384 2024-03-15T13:01:28.110 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:01:28.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.118+0000 7feaf4a1c640 1 -- 172.21.15.2:0/1141123252 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feadc0a1bd0 msgr2=0x7feadc0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.118+0000 7feaf4a1c640 1 --2- 172.21.15.2:0/1141123252 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feadc0a1bd0 0x7feadc0a4090 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feaf01109e0 tx=0x7fead8005f90 comp rx=0 tx=0).stop 2024-03-15T13:01:28.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.118+0000 7feaf4a1c640 1 -- 172.21.15.2:0/1141123252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf01005d0 msgr2=0x7feaf010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.118+0000 7feaf4a1c640 1 --2- 172.21.15.2:0/1141123252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf01005d0 0x7feaf010ef50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7feae0009900 tx=0x7feae002ca20 comp rx=0 tx=0).stop 2024-03-15T13:01:28.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.118+0000 7feaf4a1c640 1 -- 172.21.15.2:0/1141123252 shutdown_connections 2024-03-15T13:01:28.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.118+0000 7feaf4a1c640 1 --2- 172.21.15.2:0/1141123252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaf010f9d0 0x7feaf01a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.118+0000 7feaf4a1c640 1 --2- 172.21.15.2:0/1141123252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feaf01056a0 0x7feaf010f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.118+0000 7feaf4a1c640 1 --2- 172.21.15.2:0/1141123252 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feadc0a1bd0 0x7feadc0a4090 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.118+0000 7feaf4a1c640 1 --2- 172.21.15.2:0/1141123252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf01005d0 0x7feaf010ef50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.118+0000 7feaf4a1c640 1 -- 172.21.15.2:0/1141123252 >> 172.21.15.2:0/1141123252 conn(0x7feaf00774f0 msgr2=0x7feaf00fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.118+0000 7feaf4a1c640 1 -- 172.21.15.2:0/1141123252 shutdown_connections 2024-03-15T13:01:28.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.118+0000 7feaf4a1c640 1 -- 172.21.15.2:0/1141123252 wait complete. 2024-03-15T13:01:28.114 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:01:28.114 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:01:28.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.236+0000 7f270ecaa640 1 -- 172.21.15.2:0/2477412520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27081056e0 msgr2=0x7f270810d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.236+0000 7f270ecaa640 1 --2- 172.21.15.2:0/2477412520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27081056e0 0x7f270810d2b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2700009be0 tx=0x7f270002bef0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.236+0000 7f270ecaa640 1 -- 172.21.15.2:0/2477412520 shutdown_connections 2024-03-15T13:01:28.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.236+0000 7f270ecaa640 1 --2- 172.21.15.2:0/2477412520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27081056e0 0x7f270810d2b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.236+0000 7f270ecaa640 1 --2- 172.21.15.2:0/2477412520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2708100fc0 0x7f2708105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.236+0000 7f270ecaa640 1 --2- 172.21.15.2:0/2477412520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2708100610 0x7f27081009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.236+0000 7f270ecaa640 1 -- 172.21.15.2:0/2477412520 >> 172.21.15.2:0/2477412520 conn(0x7f2708076770 msgr2=0x7f2708076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.236+0000 7f270ecaa640 1 -- 172.21.15.2:0/2477412520 shutdown_connections 2024-03-15T13:01:28.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.236+0000 7f270ecaa640 1 -- 172.21.15.2:0/2477412520 wait complete. 2024-03-15T13:01:28.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270ecaa640 1 Processor -- start 2024-03-15T13:01:28.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270ecaa640 1 -- start start 2024-03-15T13:01:28.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270ecaa640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2708100610 0x7f270819d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270ecaa640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27081056e0 0x7f270819dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270ecaa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2708197930 0x7f2708197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270ecaa640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2708198320 con 0x7f2708197930 2024-03-15T13:01:28.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270ecaa640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2708198490 con 0x7f2708100610 2024-03-15T13:01:28.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270ecaa640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2708198650 con 0x7f27081056e0 2024-03-15T13:01:28.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270d247640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2708197930 0x7f2708197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270ca46640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2708100610 0x7f270819d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270d247640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2708197930 0x7f2708197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50950/0 (socket says 172.21.15.2:50950) 2024-03-15T13:01:28.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f2707fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27081056e0 0x7f270819dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270d247640 1 -- 172.21.15.2:0/3926794073 learned_addr learned my addr 172.21.15.2:0/3926794073 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:28.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270ca46640 1 --2- 172.21.15.2:0/3926794073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2708100610 0x7f270819d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59634/0 (socket says 172.21.15.2:59634) 2024-03-15T13:01:28.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270d247640 1 -- 172.21.15.2:0/3926794073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27081056e0 msgr2=0x7f270819dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270d247640 1 --2- 172.21.15.2:0/3926794073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27081056e0 0x7f270819dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270d247640 1 -- 172.21.15.2:0/3926794073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2708100610 msgr2=0x7f270819d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270d247640 1 --2- 172.21.15.2:0/3926794073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2708100610 0x7f270819d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f270d247640 1 -- 172.21.15.2:0/3926794073 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc009560 con 0x7f2708197930 2024-03-15T13:01:28.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.237+0000 7f2707fff640 1 --2- 172.21.15.2:0/3926794073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27081056e0 0x7f270819dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:28.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.238+0000 7f270d247640 1 --2- 172.21.15.2:0/3926794073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2708197930 0x7f2708197de0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2700006010 tx=0x7f2700033680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.238+0000 7f2705ffb640 1 -- 172.21.15.2:0/3926794073 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700003b80 con 0x7f2708197930 2024-03-15T13:01:28.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.238+0000 7f270ecaa640 1 -- 172.21.15.2:0/3926794073 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2700009840 con 0x7f2708197930 2024-03-15T13:01:28.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.238+0000 7f2705ffb640 1 -- 172.21.15.2:0/3926794073 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2700003ce0 con 0x7f2708197930 2024-03-15T13:01:28.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.238+0000 7f2705ffb640 1 -- 172.21.15.2:0/3926794073 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700034810 con 0x7f2708197930 2024-03-15T13:01:28.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.238+0000 7f270ecaa640 1 -- 172.21.15.2:0/3926794073 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27081a49b0 con 0x7f2708197930 2024-03-15T13:01:28.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.241+0000 7f2711885740 1 Processor -- start 2024-03-15T13:01:28.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.241+0000 7f2705ffb640 1 -- 172.21.15.2:0/3926794073 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f2700004040 con 0x7f2708197930 2024-03-15T13:01:28.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.241+0000 7f2711885740 1 -- start start 2024-03-15T13:01:28.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.241+0000 7f2711885740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c75f80 0x557fb4c76360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.241+0000 7f2711885740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c768a0 0x557fb4c83620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.241+0000 7f2711885740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c85f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.241+0000 7f2711885740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4c31960 con 0x557fb4c83b60 2024-03-15T13:01:28.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.241+0000 7f2711885740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4be5310 con 0x557fb4c75f80 2024-03-15T13:01:28.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.241+0000 7f2711885740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4be2840 con 0x557fb4c768a0 2024-03-15T13:01:28.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.241+0000 7f2707fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c85f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.241+0000 7f2707fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c85f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50958/0 (socket says 172.21.15.2:50958) 2024-03-15T13:01:28.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.241+0000 7f2707fff640 1 -- 172.21.15.2:0/2866265585 learned_addr learned my addr 172.21.15.2:0/2866265585 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:28.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.241+0000 7f2705ffb640 1 --2- 172.21.15.2:0/3926794073 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f26f80a1ba0 0x7f26f80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.241+0000 7f270ca46640 1 --2- 172.21.15.2:0/2866265585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c768a0 0x557fb4c83620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.241+0000 7f2707fff640 1 --2- 172.21.15.2:0/2866265585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c75f80 0x557fb4c76360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.242+0000 7f270d247640 1 --2- 172.21.15.2:0/3926794073 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f26f80a1ba0 0x7f26f80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.242+0000 7f270ca46640 1 -- 172.21.15.2:0/2866265585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c75f80 msgr2=0x557fb4c76360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.242+0000 7f270ca46640 1 --2- 172.21.15.2:0/2866265585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c75f80 0x557fb4c76360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.242+0000 7f270ca46640 1 -- 172.21.15.2:0/2866265585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 msgr2=0x557fb4c85f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.242+0000 7f270ca46640 1 --2- 172.21.15.2:0/2866265585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c85f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.242+0000 7f270ca46640 1 -- 172.21.15.2:0/2866265585 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fb4ba0060 con 0x557fb4c768a0 2024-03-15T13:01:28.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.242+0000 7f2707fff640 1 --2- 172.21.15.2:0/2866265585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c85f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:28.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.242+0000 7f2705ffb640 1 -- 172.21.15.2:0/3926794073 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f27000f08b0 con 0x7f2708197930 2024-03-15T13:01:28.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.242+0000 7f270ca46640 1 --2- 172.21.15.2:0/2866265585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c768a0 0x557fb4c83620 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009530 tx=0x7f26fc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c347cd5db0db8e53 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.242+0000 7f270d247640 1 --2- 172.21.15.2:0/3926794073 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f26f80a1ba0 0x7f26f80a4060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f27000bd1d0 tx=0x7f27000904f0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.242+0000 7f26f2ffd640 1 -- 172.21.15.2:0/2866265585 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc004220 con 0x557fb4c768a0 2024-03-15T13:01:28.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.242+0000 7f26f2ffd640 1 -- 172.21.15.2:0/2866265585 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f26fc004380 con 0x557fb4c768a0 2024-03-15T13:01:28.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.242+0000 7f26f2ffd640 1 -- 172.21.15.2:0/2866265585 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0198a0 con 0x557fb4c768a0 2024-03-15T13:01:28.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.243+0000 7f2711885740 1 -- 172.21.15.2:0/2866265585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c768a0 msgr2=0x557fb4c83620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.243+0000 7f2711885740 1 --2- 172.21.15.2:0/2866265585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c768a0 0x557fb4c83620 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009530 tx=0x7f26fc011810 comp rx=0 tx=0).stop 2024-03-15T13:01:28.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.243+0000 7f2711885740 1 -- 172.21.15.2:0/2866265585 shutdown_connections 2024-03-15T13:01:28.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.243+0000 7f2711885740 1 --2- 172.21.15.2:0/2866265585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c85f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.243+0000 7f2711885740 1 --2- 172.21.15.2:0/2866265585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c768a0 0x557fb4c83620 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.243+0000 7f2711885740 1 --2- 172.21.15.2:0/2866265585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c75f80 0x557fb4c76360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.243+0000 7f2711885740 1 -- 172.21.15.2:0/2866265585 >> 172.21.15.2:0/2866265585 conn(0x557fb4c70980 msgr2=0x557fb4c70d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.243+0000 7f2711885740 1 -- 172.21.15.2:0/2866265585 shutdown_connections 2024-03-15T13:01:28.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.243+0000 7f2711885740 1 -- 172.21.15.2:0/2866265585 wait complete. 2024-03-15T13:01:28.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.243+0000 7f2711885740 1 -- start start 2024-03-15T13:01:28.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.243+0000 7f2711885740 1 Processor -- start 2024-03-15T13:01:28.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f2711885740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c75f80 0x557fb4c83550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f2711885740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c768a0 0x557fb4c808e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f2711885740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c83b60 0x557fb4c80ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f2711885740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4bde020 con 0x557fb4c768a0 2024-03-15T13:01:28.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f2711885740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb48b1430 con 0x557fb4c83b60 2024-03-15T13:01:28.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f2711885740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb486de00 con 0x557fb4c75f80 2024-03-15T13:01:28.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f2711885740 10 client.0 _ll_register_callbacks cb 0x557fb4b85cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:28.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f270ca46640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c768a0 0x557fb4c808e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f2707fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c75f80 0x557fb4c83550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f270ca46640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c768a0 0x557fb4c808e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50974/0 (socket says 172.21.15.2:50974) 2024-03-15T13:01:28.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f270ca46640 1 -- 172.21.15.2:0/1108918824 learned_addr learned my addr 172.21.15.2:0/1108918824 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:28.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f2707fff640 1 --2- 172.21.15.2:0/1108918824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c83b60 0x557fb4c80ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f270ca46640 1 -- 172.21.15.2:0/1108918824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c75f80 msgr2=0x557fb4c83550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f270ca46640 1 --2- 172.21.15.2:0/1108918824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c75f80 0x557fb4c83550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f270ca46640 1 -- 172.21.15.2:0/1108918824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c83b60 msgr2=0x557fb4c80ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f270ca46640 1 --2- 172.21.15.2:0/1108918824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c83b60 0x557fb4c80ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f270ca46640 1 -- 172.21.15.2:0/1108918824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26fc009a20 con 0x557fb4c768a0 2024-03-15T13:01:28.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f2707fff640 1 --2- 172.21.15.2:0/1108918824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c75f80 0x557fb4c83550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:28.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f270ca46640 1 --2- 172.21.15.2:0/1108918824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c768a0 0x557fb4c808e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f26fc01b040 tx=0x7f26fc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60f80018efe8cdf server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f26e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:01:28.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f26e7fff640 1 -- 172.21.15.2:0/1108918824 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc005670 con 0x557fb4c768a0 2024-03-15T13:01:28.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f26e7fff640 1 -- 172.21.15.2:0/1108918824 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f26fc003dc0 con 0x557fb4c768a0 2024-03-15T13:01:28.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.244+0000 7f2711885740 1 -- 172.21.15.2:0/1108918824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fb48dd7e0 con 0x557fb4c768a0 2024-03-15T13:01:28.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.245+0000 7f26e7fff640 1 -- 172.21.15.2:0/1108918824 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26fc0035f0 con 0x557fb4c768a0 2024-03-15T13:01:28.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.245+0000 7f2711885740 10 client.16515 fetch_fsmap learned FSMap version 35 2024-03-15T13:01:28.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.245+0000 7f2711885740 1 -- 172.21.15.2:0/1108918824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557fb4bdc6a0 con 0x557fb4c768a0 2024-03-15T13:01:28.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.245+0000 7f26e7fff640 1 -- 172.21.15.2:0/1108918824 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x557fb48dd7e0 con 0x557fb4c768a0 2024-03-15T13:01:28.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.245+0000 7f26e7fff640 1 -- 172.21.15.2:0/1108918824 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 35) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f26fc0195a0 con 0x557fb4c768a0 2024-03-15T13:01:28.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.245+0000 7f2711885740 10 client.16515 fetch_fsmap finished waiting for FSMap version 35 2024-03-15T13:01:28.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.245+0000 7f2711885740 10 client.16515 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:01:28.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.245+0000 7f2711885740 20 client.16515 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:28.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.245+0000 7f2711885740 1 --2- 172.21.15.2:0/1108918824 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557fb4c93ce0 0x557fb4c96100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.245+0000 7f2711885740 4 client.16515 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:01:28.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.245+0000 7f2711885740 1 -- 172.21.15.2:0/1108918824 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fb491a720 con 0x557fb4c93ce0 2024-03-15T13:01:28.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.245+0000 7f2707fff640 1 --2- 172.21.15.2:0/1108918824 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557fb4c93ce0 0x557fb4c96100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.246+0000 7f26e7fff640 10 client.16515 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:01:28.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.246+0000 7f2707fff640 1 --2- 172.21.15.2:0/1108918824 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557fb4c93ce0 0x557fb4c96100 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34300410a7f65719 server_cookie=708804fe21e0e58 in_seq=0 out_seq=0 2024-03-15T13:01:28.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.247+0000 7f26e7fff640 1 -- 172.21.15.2:0/1108918824 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557fb491a720 con 0x557fb4c93ce0 2024-03-15T13:01:28.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.247+0000 7f26e7fff640 10 client.16515 handle_command_reply: tid=0 2024-03-15T13:01:28.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.247+0000 7f2711885740 1 client.16515 shutdown 2024-03-15T13:01:28.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.247+0000 7f2711885740 1 -- 172.21.15.2:0/1108918824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c768a0 msgr2=0x557fb4c808e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.247+0000 7f2711885740 1 --2- 172.21.15.2:0/1108918824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c768a0 0x557fb4c808e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f26fc01b040 tx=0x7f26fc005270 comp rx=0 tx=0).stop 2024-03-15T13:01:28.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.247+0000 7f2711885740 1 -- 172.21.15.2:0/1108918824 shutdown_connections 2024-03-15T13:01:28.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.247+0000 7f2711885740 1 --2- 172.21.15.2:0/1108918824 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557fb4c93ce0 0x557fb4c96100 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.247+0000 7f2711885740 1 --2- 172.21.15.2:0/1108918824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c83b60 0x557fb4c80ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.247+0000 7f2711885740 1 --2- 172.21.15.2:0/1108918824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c768a0 0x557fb4c808e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.247+0000 7f26e7fff640 0 client.16515 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:01:28.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.247+0000 7f2711885740 1 --2- 172.21.15.2:0/1108918824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c75f80 0x557fb4c83550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.247+0000 7f2711885740 1 -- 172.21.15.2:0/1108918824 >> 172.21.15.2:0/1108918824 conn(0x557fb4c70980 msgr2=0x557fb4c726d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.247+0000 7f2711885740 1 -- 172.21.15.2:0/1108918824 shutdown_connections 2024-03-15T13:01:28.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.247+0000 7f2711885740 1 -- 172.21.15.2:0/1108918824 wait complete. 2024-03-15T13:01:28.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.247+0000 7f2711885740 20 client.16515 trim_cache size 0 max 16384 2024-03-15T13:01:28.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.260+0000 7f2711885740 1 Processor -- start 2024-03-15T13:01:28.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2711885740 1 -- start start 2024-03-15T13:01:28.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2711885740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c75f80 0x557fb4c83010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2711885740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c768a0 0x557fb4c83550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2711885740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c8eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2711885740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4b63130 con 0x557fb4c83b60 2024-03-15T13:01:28.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2711885740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4c77010 con 0x557fb4c768a0 2024-03-15T13:01:28.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2711885740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4c707e0 con 0x557fb4c75f80 2024-03-15T13:01:28.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2707fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c8eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2707fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c8eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50988/0 (socket says 172.21.15.2:50988) 2024-03-15T13:01:28.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2707fff640 1 -- 172.21.15.2:0/15934535 learned_addr learned my addr 172.21.15.2:0/15934535 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:28.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f270ca46640 1 --2- 172.21.15.2:0/15934535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c768a0 0x557fb4c83550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2707fff640 1 --2- 172.21.15.2:0/15934535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c75f80 0x557fb4c83010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2707fff640 1 -- 172.21.15.2:0/15934535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c75f80 msgr2=0x557fb4c83010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2707fff640 1 --2- 172.21.15.2:0/15934535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c75f80 0x557fb4c83010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2707fff640 1 -- 172.21.15.2:0/15934535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c768a0 msgr2=0x557fb4c83550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2707fff640 1 --2- 172.21.15.2:0/15934535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c768a0 0x557fb4c83550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2707fff640 1 -- 172.21.15.2:0/15934535 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fb4b78c20 con 0x557fb4c83b60 2024-03-15T13:01:28.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f2707fff640 1 --2- 172.21.15.2:0/15934535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c8eef0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x557fb4c96970 tx=0x7f26f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a603038587d43ba server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f26f27fc640 1 -- 172.21.15.2:0/15934535 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4016070 con 0x557fb4c83b60 2024-03-15T13:01:28.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.261+0000 7f26f27fc640 1 -- 172.21.15.2:0/15934535 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f26f4009ec0 con 0x557fb4c83b60 2024-03-15T13:01:28.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.262+0000 7f26f27fc640 1 -- 172.21.15.2:0/15934535 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4005630 con 0x557fb4c83b60 2024-03-15T13:01:28.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.262+0000 7f2711885740 1 -- 172.21.15.2:0/15934535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 msgr2=0x557fb4c8eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.262+0000 7f2711885740 1 --2- 172.21.15.2:0/15934535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c8eef0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x557fb4c96970 tx=0x7f26f4004730 comp rx=0 tx=0).stop 2024-03-15T13:01:28.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.262+0000 7f2711885740 1 -- 172.21.15.2:0/15934535 shutdown_connections 2024-03-15T13:01:28.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.262+0000 7f2711885740 1 --2- 172.21.15.2:0/15934535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c8eef0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.262+0000 7f2711885740 1 --2- 172.21.15.2:0/15934535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c768a0 0x557fb4c83550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.262+0000 7f2711885740 1 --2- 172.21.15.2:0/15934535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c75f80 0x557fb4c83010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.262+0000 7f2711885740 1 -- 172.21.15.2:0/15934535 >> 172.21.15.2:0/15934535 conn(0x557fb4c70980 msgr2=0x557fb4c6fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.262+0000 7f2711885740 1 -- 172.21.15.2:0/15934535 shutdown_connections 2024-03-15T13:01:28.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.262+0000 7f2711885740 1 -- 172.21.15.2:0/15934535 wait complete. 2024-03-15T13:01:28.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.262+0000 7f2711885740 1 -- start start 2024-03-15T13:01:28.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2711885740 1 Processor -- start 2024-03-15T13:01:28.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2711885740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c75f80 0x557fb4c8b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2711885740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c768a0 0x557fb4c7da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2711885740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c7df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2711885740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4c7e4a0 con 0x557fb4c83b60 2024-03-15T13:01:28.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2711885740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4c7e610 con 0x557fb4c75f80 2024-03-15T13:01:28.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2711885740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb4c7e7d0 con 0x557fb4c768a0 2024-03-15T13:01:28.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2711885740 10 client.0 _ll_register_callbacks cb 0x7f26fc011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:28.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2707fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c7df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2707fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c75f80 0x557fb4c8b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2707fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c7df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50992/0 (socket says 172.21.15.2:50992) 2024-03-15T13:01:28.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2707fff640 1 -- 172.21.15.2:0/1877391462 learned_addr learned my addr 172.21.15.2:0/1877391462 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:28.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f270ca46640 1 --2- 172.21.15.2:0/1877391462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c768a0 0x557fb4c7da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2707fff640 1 -- 172.21.15.2:0/1877391462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c768a0 msgr2=0x557fb4c7da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2707fff640 1 --2- 172.21.15.2:0/1877391462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c768a0 0x557fb4c7da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2707fff640 1 -- 172.21.15.2:0/1877391462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c75f80 msgr2=0x557fb4c8b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2707fff640 1 --2- 172.21.15.2:0/1877391462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c75f80 0x557fb4c8b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.263+0000 7f2707fff640 1 -- 172.21.15.2:0/1877391462 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f40260b0 con 0x557fb4c83b60 2024-03-15T13:01:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f2707fff640 1 --2- 172.21.15.2:0/1877391462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c7df60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f26f4011f80 tx=0x7f26f4005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d98fb76d6f1fc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f26e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:01:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f26e7fff640 1 -- 172.21.15.2:0/1877391462 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4016070 con 0x557fb4c83b60 2024-03-15T13:01:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f26e7fff640 1 -- 172.21.15.2:0/1877391462 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f26f40054e0 con 0x557fb4c83b60 2024-03-15T13:01:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f2711885740 1 -- 172.21.15.2:0/1877391462 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fb4969890 con 0x557fb4c83b60 2024-03-15T13:01:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f26e7fff640 1 -- 172.21.15.2:0/1877391462 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26f40056a0 con 0x557fb4c83b60 2024-03-15T13:01:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f2711885740 10 client.16521 fetch_fsmap learned FSMap version 35 2024-03-15T13:01:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f2711885740 1 -- 172.21.15.2:0/1877391462 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557fb4b9e8c0 con 0x557fb4c83b60 2024-03-15T13:01:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f26e7fff640 1 -- 172.21.15.2:0/1877391462 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x557fb4969890 con 0x557fb4c83b60 2024-03-15T13:01:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f26e7fff640 1 -- 172.21.15.2:0/1877391462 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 35) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f26f403c070 con 0x557fb4c83b60 2024-03-15T13:01:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f2711885740 10 client.16521 fetch_fsmap finished waiting for FSMap version 35 2024-03-15T13:01:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f2711885740 10 client.16521 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:01:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f2711885740 20 client.16521 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f2711885740 1 --2- 172.21.15.2:0/1877391462 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557fb4cf08e0 0x557fb4cf2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f2711885740 4 client.16521 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:01:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.264+0000 7f2711885740 1 -- 172.21.15.2:0/1877391462 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557fb4889700 con 0x557fb4cf08e0 2024-03-15T13:01:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f270ca46640 1 --2- 172.21.15.2:0/1877391462 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557fb4cf08e0 0x557fb4cf2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f270ca46640 1 --2- 172.21.15.2:0/1877391462 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557fb4cf08e0 0x557fb4cf2d00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1874529a97e54ae5 server_cookie=6aeafbafa6b94525 in_seq=0 out_seq=0 2024-03-15T13:01:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f26e7fff640 10 client.16521 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:01:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f26e7fff640 1 -- 172.21.15.2:0/1877391462 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557fb4889700 con 0x557fb4cf08e0 2024-03-15T13:01:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f26e7fff640 10 client.16521 handle_command_reply: tid=0 2024-03-15T13:01:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f2711885740 1 client.16521 shutdown 2024-03-15T13:01:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f2711885740 1 -- 172.21.15.2:0/1877391462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 msgr2=0x557fb4c7df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f2711885740 1 --2- 172.21.15.2:0/1877391462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c7df60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f26f4011f80 tx=0x7f26f4005210 comp rx=0 tx=0).stop 2024-03-15T13:01:28.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f2711885740 1 -- 172.21.15.2:0/1877391462 shutdown_connections 2024-03-15T13:01:28.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f2711885740 1 --2- 172.21.15.2:0/1877391462 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557fb4cf08e0 0x557fb4cf2d00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f2711885740 1 --2- 172.21.15.2:0/1877391462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb4c83b60 0x557fb4c7df60 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f2711885740 1 --2- 172.21.15.2:0/1877391462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557fb4c768a0 0x557fb4c7da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f26e7fff640 0 client.16521 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:01:28.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f2711885740 1 --2- 172.21.15.2:0/1877391462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557fb4c75f80 0x557fb4c8b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f2711885740 1 -- 172.21.15.2:0/1877391462 >> 172.21.15.2:0/1877391462 conn(0x557fb4c70980 msgr2=0x557fb4c6f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f2711885740 1 -- 172.21.15.2:0/1877391462 shutdown_connections 2024-03-15T13:01:28.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f2711885740 1 -- 172.21.15.2:0/1877391462 wait complete. 2024-03-15T13:01:28.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.265+0000 7f2711885740 20 client.16521 trim_cache size 0 max 16384 2024-03-15T13:01:28.257 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:01:28.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.266+0000 7f270ecaa640 1 -- 172.21.15.2:0/3926794073 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f26f80a1ba0 msgr2=0x7f26f80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.266+0000 7f270ecaa640 1 --2- 172.21.15.2:0/3926794073 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f26f80a1ba0 0x7f26f80a4060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f27000bd1d0 tx=0x7f27000904f0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.266+0000 7f270ecaa640 1 -- 172.21.15.2:0/3926794073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2708197930 msgr2=0x7f2708197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.266+0000 7f270ecaa640 1 --2- 172.21.15.2:0/3926794073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2708197930 0x7f2708197de0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2700006010 tx=0x7f2700033680 comp rx=0 tx=0).stop 2024-03-15T13:01:28.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.266+0000 7f270ecaa640 1 -- 172.21.15.2:0/3926794073 shutdown_connections 2024-03-15T13:01:28.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.266+0000 7f270ecaa640 1 --2- 172.21.15.2:0/3926794073 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f26f80a1ba0 0x7f26f80a4060 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.266+0000 7f270ecaa640 1 --2- 172.21.15.2:0/3926794073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2708197930 0x7f2708197de0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.266+0000 7f270ecaa640 1 --2- 172.21.15.2:0/3926794073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27081056e0 0x7f270819dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.266+0000 7f270ecaa640 1 --2- 172.21.15.2:0/3926794073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2708100610 0x7f270819d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.266+0000 7f270ecaa640 1 -- 172.21.15.2:0/3926794073 >> 172.21.15.2:0/3926794073 conn(0x7f2708076770 msgr2=0x7f27080fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.266+0000 7f270ecaa640 1 -- 172.21.15.2:0/3926794073 shutdown_connections 2024-03-15T13:01:28.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.266+0000 7f270ecaa640 1 -- 172.21.15.2:0/3926794073 wait complete. 2024-03-15T13:01:28.262 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:01:28.263 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:01:28.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:27 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2668507246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:01:28.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:27 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2668507246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:01:28.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.383+0000 7fb50902a640 1 -- 172.21.15.2:0/2262948982 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5041014a0 msgr2=0x7fb504101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.383+0000 7fb50902a640 1 --2- 172.21.15.2:0/2262948982 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5041014a0 0x7fb504101920 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009900 tx=0x7fb4ec02bc30 comp rx=0 tx=0).stop 2024-03-15T13:01:28.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.383+0000 7fb50902a640 1 -- 172.21.15.2:0/2262948982 shutdown_connections 2024-03-15T13:01:28.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.383+0000 7fb50902a640 1 --2- 172.21.15.2:0/2262948982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb504101e60 0x7fb50410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.383+0000 7fb50902a640 1 --2- 172.21.15.2:0/2262948982 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5041014a0 0x7fb504101920 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.383+0000 7fb50902a640 1 --2- 172.21.15.2:0/2262948982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb504107560 0x7fb504107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.383+0000 7fb50902a640 1 -- 172.21.15.2:0/2262948982 >> 172.21.15.2:0/2262948982 conn(0x7fb5040788f0 msgr2=0x7fb504078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.383+0000 7fb50902a640 1 -- 172.21.15.2:0/2262948982 shutdown_connections 2024-03-15T13:01:28.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.383+0000 7fb50902a640 1 -- 172.21.15.2:0/2262948982 wait complete. 2024-03-15T13:01:28.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.384+0000 7fb50902a640 1 Processor -- start 2024-03-15T13:01:28.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.384+0000 7fb50902a640 1 -- start start 2024-03-15T13:01:28.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.384+0000 7fb50902a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5041014a0 0x7fb5040737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.384+0000 7fb50902a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb504101e60 0x7fb504073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.384+0000 7fb50902a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb504107560 0x7fb504074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.384+0000 7fb50902a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5041a5e80 con 0x7fb504107560 2024-03-15T13:01:28.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.384+0000 7fb50902a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5041a5ff0 con 0x7fb5041014a0 2024-03-15T13:01:28.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.384+0000 7fb50902a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5041a61b0 con 0x7fb504101e60 2024-03-15T13:01:28.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.384+0000 7fb50359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb504107560 0x7fb504074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.384+0000 7fb50359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb504107560 0x7fb504074270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51006/0 (socket says 172.21.15.2:51006) 2024-03-15T13:01:28.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.384+0000 7fb50359e640 1 -- 172.21.15.2:0/2836724144 learned_addr learned my addr 172.21.15.2:0/2836724144 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:28.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.384+0000 7fb502d9d640 1 --2- 172.21.15.2:0/2836724144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5041014a0 0x7fb5040737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.385+0000 7fb50259c640 1 --2- 172.21.15.2:0/2836724144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb504101e60 0x7fb504073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.385+0000 7fb50359e640 1 -- 172.21.15.2:0/2836724144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb504101e60 msgr2=0x7fb504073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.385+0000 7fb50359e640 1 --2- 172.21.15.2:0/2836724144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb504101e60 0x7fb504073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.385+0000 7fb50359e640 1 -- 172.21.15.2:0/2836724144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5041014a0 msgr2=0x7fb5040737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.385+0000 7fb50359e640 1 --2- 172.21.15.2:0/2836724144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5041014a0 0x7fb5040737f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.385+0000 7fb50359e640 1 -- 172.21.15.2:0/2836724144 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4009840 con 0x7fb504107560 2024-03-15T13:01:28.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.385+0000 7fb50359e640 1 --2- 172.21.15.2:0/2836724144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb504107560 0x7fb504074270 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400efd0 tx=0x7fb4f400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.385+0000 7fb4fbfff640 1 -- 172.21.15.2:0/2836724144 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4009b40 con 0x7fb504107560 2024-03-15T13:01:28.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.385+0000 7fb4fbfff640 1 -- 172.21.15.2:0/2836724144 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb4f4003b70 con 0x7fb504107560 2024-03-15T13:01:28.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.385+0000 7fb50902a640 1 -- 172.21.15.2:0/2836724144 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ec009560 con 0x7fb504107560 2024-03-15T13:01:28.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.385+0000 7fb4fbfff640 1 -- 172.21.15.2:0/2836724144 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40108f0 con 0x7fb504107560 2024-03-15T13:01:28.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.385+0000 7fb50902a640 1 -- 172.21.15.2:0/2836724144 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5041a6790 con 0x7fb504107560 2024-03-15T13:01:28.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.387+0000 7fb4fbfff640 1 -- 172.21.15.2:0/2836724144 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fb4f4003ce0 con 0x7fb504107560 2024-03-15T13:01:28.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb50bc09740 1 Processor -- start 2024-03-15T13:01:28.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb50bc09740 1 -- start start 2024-03-15T13:01:28.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb4fbfff640 1 --2- 172.21.15.2:0/2836724144 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb4f00a1ba0 0x7fb4f00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb50bc09740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e8b250 0x564db9e8b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb50bc09740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8bb70 0x564db9e94210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb50bc09740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e94750 0x564db9e96b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb50bc09740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9e46960 con 0x564db9e94750 2024-03-15T13:01:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb50bc09740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9dfa310 con 0x564db9e8bb70 2024-03-15T13:01:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb50bc09740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9df7840 con 0x564db9e8b250 2024-03-15T13:01:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb50259c640 1 --2- 172.21.15.2:0/2836724144 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb4f00a1ba0 0x7fb4f00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb50359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e94750 0x564db9e96b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb50359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e94750 0x564db9e96b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51010/0 (socket says 172.21.15.2:51010) 2024-03-15T13:01:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb50359e640 1 -- 172.21.15.2:0/1314192233 learned_addr learned my addr 172.21.15.2:0/1314192233 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb502d9d640 1 --2- 172.21.15.2:0/1314192233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e8b250 0x564db9e8b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb4fbfff640 1 -- 172.21.15.2:0/2836724144 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fb4f4014070 con 0x7fb504107560 2024-03-15T13:01:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.388+0000 7fb50259c640 1 --2- 172.21.15.2:0/1314192233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8bb70 0x564db9e94210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.389+0000 7fb502d9d640 1 -- 172.21.15.2:0/1314192233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8bb70 msgr2=0x564db9e94210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.389+0000 7fb502d9d640 1 --2- 172.21.15.2:0/1314192233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8bb70 0x564db9e94210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.389+0000 7fb502d9d640 1 -- 172.21.15.2:0/1314192233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e94750 msgr2=0x564db9e96b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.389+0000 7fb502d9d640 1 --2- 172.21.15.2:0/1314192233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e94750 0x564db9e96b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.389+0000 7fb502d9d640 1 -- 172.21.15.2:0/1314192233 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db9db5060 con 0x564db9e8b250 2024-03-15T13:01:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.389+0000 7fb50359e640 1 --2- 172.21.15.2:0/1314192233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e94750 0x564db9e96b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.389+0000 7fb50259c640 1 --2- 172.21.15.2:0/2836724144 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb4f00a1ba0 0x7fb4f00a4060 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec005ab0 tx=0x7fb4ec004750 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.389+0000 7fb502d9d640 1 --2- 172.21.15.2:0/1314192233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e8b250 0x564db9e8b630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009c20 tx=0x7fb4fc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7eec69e3aac03b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.389+0000 7fb4f97fa640 1 -- 172.21.15.2:0/1314192233 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc004050 con 0x564db9e8b250 2024-03-15T13:01:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.389+0000 7fb4f97fa640 1 -- 172.21.15.2:0/1314192233 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb4fc0035f0 con 0x564db9e8b250 2024-03-15T13:01:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.389+0000 7fb4f97fa640 1 -- 172.21.15.2:0/1314192233 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc019620 con 0x564db9e8b250 2024-03-15T13:01:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.390+0000 7fb50bc09740 1 -- 172.21.15.2:0/1314192233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e8b250 msgr2=0x564db9e8b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.390+0000 7fb50bc09740 1 --2- 172.21.15.2:0/1314192233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e8b250 0x564db9e8b630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009c20 tx=0x7fb4fc011810 comp rx=0 tx=0).stop 2024-03-15T13:01:28.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.390+0000 7fb50bc09740 1 -- 172.21.15.2:0/1314192233 shutdown_connections 2024-03-15T13:01:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.390+0000 7fb50bc09740 1 --2- 172.21.15.2:0/1314192233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e94750 0x564db9e96b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.390+0000 7fb50bc09740 1 --2- 172.21.15.2:0/1314192233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8bb70 0x564db9e94210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.390+0000 7fb50bc09740 1 --2- 172.21.15.2:0/1314192233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e8b250 0x564db9e8b630 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.390+0000 7fb50bc09740 1 -- 172.21.15.2:0/1314192233 >> 172.21.15.2:0/1314192233 conn(0x564db9e85980 msgr2=0x564db9e85d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.390+0000 7fb50bc09740 1 -- 172.21.15.2:0/1314192233 shutdown_connections 2024-03-15T13:01:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.390+0000 7fb50bc09740 1 -- 172.21.15.2:0/1314192233 wait complete. 2024-03-15T13:01:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.390+0000 7fb50bc09740 1 -- start start 2024-03-15T13:01:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.390+0000 7fb50bc09740 1 Processor -- start 2024-03-15T13:01:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50bc09740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8b250 0x564db9e94020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50bc09740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e8bb70 0x564db9e913b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50bc09740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e94750 0x564db9e918f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50bc09740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9df3020 con 0x564db9e8bb70 2024-03-15T13:01:28.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50bc09740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9ac6430 con 0x564db9e8b250 2024-03-15T13:01:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50bc09740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9a82e00 con 0x564db9e94750 2024-03-15T13:01:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50bc09740 10 client.0 _ll_register_callbacks cb 0x564db9d9acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e8bb70 0x564db9e913b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e8bb70 0x564db9e913b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51016/0 (socket says 172.21.15.2:51016) 2024-03-15T13:01:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50259c640 1 -- 172.21.15.2:0/3955079221 learned_addr learned my addr 172.21.15.2:0/3955079221 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb502d9d640 1 --2- 172.21.15.2:0/3955079221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8b250 0x564db9e94020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50359e640 1 --2- 172.21.15.2:0/3955079221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e94750 0x564db9e918f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50259c640 1 -- 172.21.15.2:0/3955079221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e94750 msgr2=0x564db9e918f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50259c640 1 --2- 172.21.15.2:0/3955079221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e94750 0x564db9e918f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50259c640 1 -- 172.21.15.2:0/3955079221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8b250 msgr2=0x564db9e94020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50259c640 1 --2- 172.21.15.2:0/3955079221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8b250 0x564db9e94020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50259c640 1 -- 172.21.15.2:0/3955079221 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc009850 con 0x564db9e8bb70 2024-03-15T13:01:28.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb50259c640 1 --2- 172.21.15.2:0/3955079221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e8bb70 0x564db9e913b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec035000 tx=0x7fb4ec02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c6081c8b67a5ff server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb4ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:01:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.391+0000 7fb4ea7fc640 1 -- 172.21.15.2:0/3955079221 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec048070 con 0x564db9e8bb70 2024-03-15T13:01:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.392+0000 7fb4ea7fc640 1 -- 172.21.15.2:0/3955079221 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fb4ec0035f0 con 0x564db9e8bb70 2024-03-15T13:01:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.392+0000 7fb50bc09740 1 -- 172.21.15.2:0/3955079221 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564db9af27e0 con 0x564db9e8bb70 2024-03-15T13:01:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.392+0000 7fb4ea7fc640 1 -- 172.21.15.2:0/3955079221 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4ec0037b0 con 0x564db9e8bb70 2024-03-15T13:01:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.392+0000 7fb50bc09740 10 client.16533 fetch_fsmap learned FSMap version 35 2024-03-15T13:01:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.392+0000 7fb50bc09740 1 -- 172.21.15.2:0/3955079221 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564db9df16a0 con 0x564db9e8bb70 2024-03-15T13:01:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.392+0000 7fb4ea7fc640 1 -- 172.21.15.2:0/3955079221 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x564db9af27e0 con 0x564db9e8bb70 2024-03-15T13:01:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.392+0000 7fb4ea7fc640 1 -- 172.21.15.2:0/3955079221 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 35) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fb4ec0052f0 con 0x564db9e8bb70 2024-03-15T13:01:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.392+0000 7fb50bc09740 10 client.16533 fetch_fsmap finished waiting for FSMap version 35 2024-03-15T13:01:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.392+0000 7fb50bc09740 10 client.16533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:01:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.392+0000 7fb50bc09740 20 client.16533 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.392+0000 7fb50bc09740 1 --2- 172.21.15.2:0/3955079221 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564db9ea48b0 0x564db9ea6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.392+0000 7fb50bc09740 4 client.16533 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:01:28.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.392+0000 7fb50bc09740 1 -- 172.21.15.2:0/3955079221 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564db9b2f720 con 0x564db9ea48b0 2024-03-15T13:01:28.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.392+0000 7fb502d9d640 1 --2- 172.21.15.2:0/3955079221 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564db9ea48b0 0x564db9ea6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.393+0000 7fb502d9d640 1 --2- 172.21.15.2:0/3955079221 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564db9ea48b0 0x564db9ea6cd0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=845011ca3841d5a7 server_cookie=c37fe451f4d14eba in_seq=0 out_seq=0 2024-03-15T13:01:28.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.393+0000 7fb4ea7fc640 10 client.16533 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.395+0000 7fb4ea7fc640 1 -- 172.21.15.2:0/3955079221 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564db9b2f720 con 0x564db9ea48b0 2024-03-15T13:01:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.395+0000 7fb4ea7fc640 10 client.16533 handle_command_reply: tid=0 2024-03-15T13:01:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.395+0000 7fb50bc09740 1 client.16533 shutdown 2024-03-15T13:01:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.395+0000 7fb50bc09740 1 -- 172.21.15.2:0/3955079221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e8bb70 msgr2=0x564db9e913b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.395+0000 7fb50bc09740 1 --2- 172.21.15.2:0/3955079221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e8bb70 0x564db9e913b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec035000 tx=0x7fb4ec02bc30 comp rx=0 tx=0).stop 2024-03-15T13:01:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.395+0000 7fb50bc09740 1 -- 172.21.15.2:0/3955079221 shutdown_connections 2024-03-15T13:01:28.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.395+0000 7fb50bc09740 1 --2- 172.21.15.2:0/3955079221 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564db9ea48b0 0x564db9ea6cd0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.395+0000 7fb50bc09740 1 --2- 172.21.15.2:0/3955079221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e94750 0x564db9e918f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.395+0000 7fb50bc09740 1 --2- 172.21.15.2:0/3955079221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e8bb70 0x564db9e913b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.395+0000 7fb4ea7fc640 0 client.16533 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:28.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.395+0000 7fb50bc09740 1 --2- 172.21.15.2:0/3955079221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8b250 0x564db9e94020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.395+0000 7fb50bc09740 1 -- 172.21.15.2:0/3955079221 >> 172.21.15.2:0/3955079221 conn(0x564db9e85980 msgr2=0x564db9e876d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.395+0000 7fb50bc09740 1 -- 172.21.15.2:0/3955079221 shutdown_connections 2024-03-15T13:01:28.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.395+0000 7fb50bc09740 1 -- 172.21.15.2:0/3955079221 wait complete. 2024-03-15T13:01:28.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.395+0000 7fb50bc09740 20 client.16533 trim_cache size 0 max 16384 2024-03-15T13:01:28.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb50bc09740 1 Processor -- start 2024-03-15T13:01:28.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb50bc09740 1 -- start start 2024-03-15T13:01:28.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb50bc09740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e8b250 0x564db9e93ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb50bc09740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8bb70 0x564db9e94020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb50bc09740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e94750 0x564db9e9fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb50bc09740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9d78130 con 0x564db9e94750 2024-03-15T13:01:28.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb50bc09740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9e8aeb0 con 0x564db9e8bb70 2024-03-15T13:01:28.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb50bc09740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9e7ff90 con 0x564db9e8b250 2024-03-15T13:01:28.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb50359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e94750 0x564db9e9fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb50259c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8bb70 0x564db9e94020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb502d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e8b250 0x564db9e93ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb50259c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8bb70 0x564db9e94020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59704/0 (socket says 172.21.15.2:59704) 2024-03-15T13:01:28.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb50359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e94750 0x564db9e9fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51020/0 (socket says 172.21.15.2:51020) 2024-03-15T13:01:28.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb50259c640 1 -- 172.21.15.2:0/98233313 learned_addr learned my addr 172.21.15.2:0/98233313 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:28.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb502d9d640 1 -- 172.21.15.2:0/98233313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8bb70 msgr2=0x564db9e94020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb502d9d640 1 --2- 172.21.15.2:0/98233313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8bb70 0x564db9e94020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb502d9d640 1 -- 172.21.15.2:0/98233313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e94750 msgr2=0x564db9e9fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb502d9d640 1 --2- 172.21.15.2:0/98233313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e94750 0x564db9e9fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb502d9d640 1 -- 172.21.15.2:0/98233313 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db9d8dc20 con 0x564db9e8b250 2024-03-15T13:01:28.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.408+0000 7fb50359e640 1 --2- 172.21.15.2:0/98233313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e94750 0x564db9e9fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:28.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.409+0000 7fb502d9d640 1 --2- 172.21.15.2:0/98233313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e8b250 0x564db9e93ae0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc003f70 tx=0x7fb4fc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b62e8c60fb4cff1a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.409+0000 7fb4f8ff9640 1 -- 172.21.15.2:0/98233313 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc029070 con 0x564db9e8b250 2024-03-15T13:01:28.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.409+0000 7fb4f8ff9640 1 -- 172.21.15.2:0/98233313 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb4fc009e50 con 0x564db9e8b250 2024-03-15T13:01:28.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.409+0000 7fb4f8ff9640 1 -- 172.21.15.2:0/98233313 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc0052a0 con 0x564db9e8b250 2024-03-15T13:01:28.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.409+0000 7fb50bc09740 1 -- 172.21.15.2:0/98233313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e8b250 msgr2=0x564db9e93ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.409+0000 7fb50bc09740 1 --2- 172.21.15.2:0/98233313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e8b250 0x564db9e93ae0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc003f70 tx=0x7fb4fc014000 comp rx=0 tx=0).stop 2024-03-15T13:01:28.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.409+0000 7fb50bc09740 1 -- 172.21.15.2:0/98233313 shutdown_connections 2024-03-15T13:01:28.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.409+0000 7fb50bc09740 1 --2- 172.21.15.2:0/98233313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e94750 0x564db9e9fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.409+0000 7fb50bc09740 1 --2- 172.21.15.2:0/98233313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8bb70 0x564db9e94020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50bc09740 1 --2- 172.21.15.2:0/98233313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e8b250 0x564db9e93ae0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50bc09740 1 -- 172.21.15.2:0/98233313 >> 172.21.15.2:0/98233313 conn(0x564db9e85980 msgr2=0x564db9e914d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50bc09740 1 -- 172.21.15.2:0/98233313 shutdown_connections 2024-03-15T13:01:28.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50bc09740 1 -- 172.21.15.2:0/98233313 wait complete. 2024-03-15T13:01:28.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50bc09740 1 -- start start 2024-03-15T13:01:28.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50bc09740 1 Processor -- start 2024-03-15T13:01:28.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50bc09740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8b250 0x564db9e98070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50bc09740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e8bb70 0x564db9e985b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50bc09740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e94750 0x564db9eff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50bc09740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9e98cb0 con 0x564db9e8bb70 2024-03-15T13:01:28.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50bc09740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9eff6e0 con 0x564db9e8b250 2024-03-15T13:01:28.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50bc09740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db9eff850 con 0x564db9e94750 2024-03-15T13:01:28.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50bc09740 10 client.0 _ll_register_callbacks cb 0x7fb4ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:28.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e8bb70 0x564db9e985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e8bb70 0x564db9e985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51030/0 (socket says 172.21.15.2:51030) 2024-03-15T13:01:28.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50259c640 1 -- 172.21.15.2:0/268475714 learned_addr learned my addr 172.21.15.2:0/268475714 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:28.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50359e640 1 --2- 172.21.15.2:0/268475714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e94750 0x564db9eff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb502d9d640 1 --2- 172.21.15.2:0/268475714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8b250 0x564db9e98070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50259c640 1 -- 172.21.15.2:0/268475714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e94750 msgr2=0x564db9eff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50259c640 1 --2- 172.21.15.2:0/268475714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e94750 0x564db9eff1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50259c640 1 -- 172.21.15.2:0/268475714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8b250 msgr2=0x564db9e98070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:28.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50259c640 1 --2- 172.21.15.2:0/268475714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8b250 0x564db9e98070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb50259c640 1 -- 172.21.15.2:0/268475714 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f4085040 con 0x564db9e8bb70 2024-03-15T13:01:28.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.410+0000 7fb502d9d640 1 --2- 172.21.15.2:0/268475714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8b250 0x564db9e98070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:01:28.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb50259c640 1 --2- 172.21.15.2:0/268475714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e8bb70 0x564db9e985b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec005700 tx=0x7fb4ec042300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54757efd09f3895d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:28.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb4ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:01:28.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb4ea7fc640 1 -- 172.21.15.2:0/268475714 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec048030 con 0x564db9e8bb70 2024-03-15T13:01:28.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb4ea7fc640 1 -- 172.21.15.2:0/268475714 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fb4ec049070 con 0x564db9e8bb70 2024-03-15T13:01:28.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb50bc09740 1 -- 172.21.15.2:0/268475714 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4fc0035f0 con 0x564db9e8bb70 2024-03-15T13:01:28.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb4ea7fc640 1 -- 172.21.15.2:0/268475714 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4ec004410 con 0x564db9e8bb70 2024-03-15T13:01:28.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb50bc09740 10 client.16539 fetch_fsmap learned FSMap version 35 2024-03-15T13:01:28.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb50bc09740 1 -- 172.21.15.2:0/268475714 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x564db9db5060 con 0x564db9e8bb70 2024-03-15T13:01:28.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb4ea7fc640 1 -- 172.21.15.2:0/268475714 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb4fc0035f0 con 0x564db9e8bb70 2024-03-15T13:01:28.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb4ea7fc640 1 -- 172.21.15.2:0/268475714 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 35) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fb4ec058070 con 0x564db9e8bb70 2024-03-15T13:01:28.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb50bc09740 10 client.16539 fetch_fsmap finished waiting for FSMap version 35 2024-03-15T13:01:28.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb50bc09740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:01:28.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb50bc09740 20 client.16539 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:28.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb50bc09740 1 --2- 172.21.15.2:0/268475714 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564db9f00e70 0x564db9f03290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:28.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb50bc09740 4 client.16539 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:01:28.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.411+0000 7fb50bc09740 1 -- 172.21.15.2:0/268475714 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564db9b7e890 con 0x564db9f00e70 2024-03-15T13:01:28.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.412+0000 7fb502d9d640 1 --2- 172.21.15.2:0/268475714 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564db9f00e70 0x564db9f03290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:28.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.412+0000 7fb4ea7fc640 10 client.16539 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:28.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.412+0000 7fb502d9d640 1 --2- 172.21.15.2:0/268475714 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564db9f00e70 0x564db9f03290 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79a433762fce48c6 server_cookie=d2bc75d25ac82f00 in_seq=0 out_seq=0 2024-03-15T13:01:28.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.414+0000 7fb4ea7fc640 1 -- 172.21.15.2:0/268475714 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564db9b7e890 con 0x564db9f00e70 2024-03-15T13:01:28.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.414+0000 7fb4ea7fc640 10 client.16539 handle_command_reply: tid=0 2024-03-15T13:01:28.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.414+0000 7fb50bc09740 1 client.16539 shutdown 2024-03-15T13:01:28.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.414+0000 7fb50bc09740 1 -- 172.21.15.2:0/268475714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e8bb70 msgr2=0x564db9e985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.414+0000 7fb50bc09740 1 --2- 172.21.15.2:0/268475714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e8bb70 0x564db9e985b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec005700 tx=0x7fb4ec042300 comp rx=0 tx=0).stop 2024-03-15T13:01:28.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.414+0000 7fb50bc09740 1 -- 172.21.15.2:0/268475714 shutdown_connections 2024-03-15T13:01:28.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.414+0000 7fb50bc09740 1 --2- 172.21.15.2:0/268475714 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564db9f00e70 0x564db9f03290 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.414+0000 7fb50bc09740 1 --2- 172.21.15.2:0/268475714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db9e94750 0x564db9eff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.414+0000 7fb50bc09740 1 --2- 172.21.15.2:0/268475714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564db9e8bb70 0x564db9e985b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.414+0000 7fb4ea7fc640 0 client.16539 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:28.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.414+0000 7fb50bc09740 1 --2- 172.21.15.2:0/268475714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564db9e8b250 0x564db9e98070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.414+0000 7fb50bc09740 1 -- 172.21.15.2:0/268475714 >> 172.21.15.2:0/268475714 conn(0x564db9e85980 msgr2=0x564db9e90c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.414+0000 7fb50bc09740 1 -- 172.21.15.2:0/268475714 shutdown_connections 2024-03-15T13:01:28.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.414+0000 7fb50bc09740 1 -- 172.21.15.2:0/268475714 wait complete. 2024-03-15T13:01:28.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.415+0000 7fb50bc09740 20 client.16539 trim_cache size 0 max 16384 2024-03-15T13:01:28.405 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:01:28.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.415+0000 7fb50902a640 1 -- 172.21.15.2:0/2836724144 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb4f00a1ba0 msgr2=0x7fb4f00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.415+0000 7fb50902a640 1 --2- 172.21.15.2:0/2836724144 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb4f00a1ba0 0x7fb4f00a4060 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec005ab0 tx=0x7fb4ec004750 comp rx=0 tx=0).stop 2024-03-15T13:01:28.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.415+0000 7fb50902a640 1 -- 172.21.15.2:0/2836724144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb504107560 msgr2=0x7fb504074270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:28.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.415+0000 7fb50902a640 1 --2- 172.21.15.2:0/2836724144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb504107560 0x7fb504074270 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400efd0 tx=0x7fb4f400c6a0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.415+0000 7fb50902a640 1 -- 172.21.15.2:0/2836724144 shutdown_connections 2024-03-15T13:01:28.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.415+0000 7fb50902a640 1 --2- 172.21.15.2:0/2836724144 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb4f00a1ba0 0x7fb4f00a4060 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.415+0000 7fb50902a640 1 --2- 172.21.15.2:0/2836724144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb504107560 0x7fb504074270 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.415+0000 7fb50902a640 1 --2- 172.21.15.2:0/2836724144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb504101e60 0x7fb504073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.415+0000 7fb50902a640 1 --2- 172.21.15.2:0/2836724144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5041014a0 0x7fb5040737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:28.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.415+0000 7fb50902a640 1 -- 172.21.15.2:0/2836724144 >> 172.21.15.2:0/2836724144 conn(0x7fb5040788f0 msgr2=0x7fb504100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:28.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.415+0000 7fb50902a640 1 -- 172.21.15.2:0/2836724144 shutdown_connections 2024-03-15T13:01:28.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:28.415+0000 7fb50902a640 1 -- 172.21.15.2:0/2836724144 wait complete. 2024-03-15T13:01:29.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:28 smithi002 ceph-mon[32060]: pgmap v80: 129 pgs: 129 active+clean; 198 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-03-15T13:01:29.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:28 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:01:29.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:28 smithi184 ceph-mon[35993]: pgmap v80: 129 pgs: 129 active+clean; 198 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-03-15T13:01:29.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:28 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:01:29.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:28 smithi149 ceph-mon[35927]: pgmap v80: 129 pgs: 129 active+clean; 198 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-03-15T13:01:29.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:28 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:01:29.412 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:01:29.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:01:29.412 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:01:29.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.546+0000 7f5a6fc84640 1 -- 172.21.15.2:0/452607470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a68100f80 msgr2=0x7f5a68104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.546+0000 7f5a6fc84640 1 --2- 172.21.15.2:0/452607470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a68100f80 0x7f5a68104fc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5a580098f0 tx=0x7f5a5802dda0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.546+0000 7f5a6fc84640 1 -- 172.21.15.2:0/452607470 shutdown_connections 2024-03-15T13:01:29.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.546+0000 7f5a6fc84640 1 --2- 172.21.15.2:0/452607470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a681056a0 0x7f5a6810d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.546+0000 7f5a6fc84640 1 --2- 172.21.15.2:0/452607470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a68100f80 0x7f5a68104fc0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.546+0000 7f5a6fc84640 1 --2- 172.21.15.2:0/452607470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a681005d0 0x7f5a681009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.546+0000 7f5a6fc84640 1 -- 172.21.15.2:0/452607470 >> 172.21.15.2:0/452607470 conn(0x7f5a680774f0 msgr2=0x7f5a68077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:29.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.546+0000 7f5a6fc84640 1 -- 172.21.15.2:0/452607470 shutdown_connections 2024-03-15T13:01:29.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.547+0000 7f5a6fc84640 1 -- 172.21.15.2:0/452607470 wait complete. 2024-03-15T13:01:29.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.547+0000 7f5a6fc84640 1 Processor -- start 2024-03-15T13:01:29.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.547+0000 7f5a6fc84640 1 -- start start 2024-03-15T13:01:29.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.547+0000 7f5a6fc84640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a681005d0 0x7f5a68199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.547+0000 7f5a6fc84640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a681056a0 0x7f5a6819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.547+0000 7f5a6fc84640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a6819e600 0x7f5a6819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.547+0000 7f5a6fc84640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6819eff0 con 0x7f5a681056a0 2024-03-15T13:01:29.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.547+0000 7f5a6fc84640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6819f160 con 0x7f5a6819e600 2024-03-15T13:01:29.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.547+0000 7f5a6fc84640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6819f2d0 con 0x7f5a681005d0 2024-03-15T13:01:29.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a6d21f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a681056a0 0x7f5a6819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a6d21f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a681056a0 0x7f5a6819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51054/0 (socket says 172.21.15.2:51054) 2024-03-15T13:01:29.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a6da20640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a681005d0 0x7f5a68199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a6d21f640 1 -- 172.21.15.2:0/3524397800 learned_addr learned my addr 172.21.15.2:0/3524397800 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:29.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a6e221640 1 --2- 172.21.15.2:0/3524397800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a6819e600 0x7f5a6819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a6d21f640 1 -- 172.21.15.2:0/3524397800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a681005d0 msgr2=0x7f5a68199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a6d21f640 1 --2- 172.21.15.2:0/3524397800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a681005d0 0x7f5a68199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a6d21f640 1 -- 172.21.15.2:0/3524397800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a6819e600 msgr2=0x7f5a6819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a6d21f640 1 --2- 172.21.15.2:0/3524397800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a6819e600 0x7f5a6819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a6d21f640 1 -- 172.21.15.2:0/3524397800 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a64009560 con 0x7f5a681056a0 2024-03-15T13:01:29.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a6d21f640 1 --2- 172.21.15.2:0/3524397800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a681056a0 0x7f5a6819a280 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5a580098c0 tx=0x7f5a5802fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:29.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a5effd640 1 -- 172.21.15.2:0/3524397800 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a580369c0 con 0x7f5a681056a0 2024-03-15T13:01:29.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a5effd640 1 -- 172.21.15.2:0/3524397800 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5a58036e90 con 0x7f5a681056a0 2024-03-15T13:01:29.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a6fc84640 1 -- 172.21.15.2:0/3524397800 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a58009490 con 0x7f5a681056a0 2024-03-15T13:01:29.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a5effd640 1 -- 172.21.15.2:0/3524397800 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5803f680 con 0x7f5a681056a0 2024-03-15T13:01:29.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.548+0000 7f5a6fc84640 1 -- 172.21.15.2:0/3524397800 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a681a6b40 con 0x7f5a681056a0 2024-03-15T13:01:29.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.551+0000 7f5a72868740 1 Processor -- start 2024-03-15T13:01:29.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.551+0000 7f5a5effd640 1 -- 172.21.15.2:0/3524397800 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f5a5803d070 con 0x7f5a681056a0 2024-03-15T13:01:29.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.551+0000 7f5a72868740 1 -- start start 2024-03-15T13:01:29.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.551+0000 7f5a72868740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1a380 0x55738cf1a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.551+0000 7f5a72868740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1ad30 0x55738cf232e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.551+0000 7f5a72868740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf25d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.551+0000 7f5a72868740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738ce88f90 con 0x55738cf238b0 2024-03-15T13:01:29.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.551+0000 7f5a72868740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738ce864c0 con 0x55738cf1ad30 2024-03-15T13:01:29.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.551+0000 7f5a72868740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738ce81ca0 con 0x55738cf1a380 2024-03-15T13:01:29.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a6e221640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1a380 0x55738cf1a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a6d21f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf25d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a6e221640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1a380 0x55738cf1a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37610/0 (socket says 172.21.15.2:37610) 2024-03-15T13:01:29.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a6e221640 1 -- 172.21.15.2:0/795668301 learned_addr learned my addr 172.21.15.2:0/795668301 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:29.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a6da20640 1 --2- 172.21.15.2:0/795668301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1ad30 0x55738cf232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a6da20640 1 -- 172.21.15.2:0/795668301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1a380 msgr2=0x55738cf1a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a6da20640 1 --2- 172.21.15.2:0/795668301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1a380 0x55738cf1a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a6da20640 1 -- 172.21.15.2:0/795668301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 msgr2=0x55738cf25d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a6da20640 1 --2- 172.21.15.2:0/795668301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf25d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a6da20640 1 -- 172.21.15.2:0/795668301 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55738ce440b0 con 0x55738cf1ad30 2024-03-15T13:01:29.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a6e221640 1 --2- 172.21.15.2:0/795668301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1a380 0x55738cf1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:29.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a5effd640 1 --2- 172.21.15.2:0/3524397800 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5a540a1ba0 0x7f5a540a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a6e221640 1 --2- 172.21.15.2:0/3524397800 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5a540a1ba0 0x7f5a540a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a5effd640 1 -- 172.21.15.2:0/3524397800 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5a580efed0 con 0x7f5a681056a0 2024-03-15T13:01:29.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a6da20640 1 --2- 172.21.15.2:0/795668301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1ad30 0x55738cf232e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5a64005fd0 tx=0x7f5a64011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d3e42b0d02039d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:29.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a53fff640 1 -- 172.21.15.2:0/795668301 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64004220 con 0x55738cf1ad30 2024-03-15T13:01:29.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.552+0000 7f5a53fff640 1 -- 172.21.15.2:0/795668301 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5a64004380 con 0x55738cf1ad30 2024-03-15T13:01:29.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.553+0000 7f5a53fff640 1 -- 172.21.15.2:0/795668301 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a640198d0 con 0x55738cf1ad30 2024-03-15T13:01:29.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.553+0000 7f5a72868740 1 -- 172.21.15.2:0/795668301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1ad30 msgr2=0x55738cf232e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.553+0000 7f5a6e221640 1 --2- 172.21.15.2:0/3524397800 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5a540a1ba0 0x7f5a540a4060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5a60005f40 tx=0x7f5a60009600 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:29.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.553+0000 7f5a72868740 1 --2- 172.21.15.2:0/795668301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1ad30 0x55738cf232e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5a64005fd0 tx=0x7f5a64011810 comp rx=0 tx=0).stop 2024-03-15T13:01:29.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.553+0000 7f5a72868740 1 -- 172.21.15.2:0/795668301 shutdown_connections 2024-03-15T13:01:29.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.553+0000 7f5a72868740 1 --2- 172.21.15.2:0/795668301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf25d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.553+0000 7f5a72868740 1 --2- 172.21.15.2:0/795668301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1ad30 0x55738cf232e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.553+0000 7f5a72868740 1 --2- 172.21.15.2:0/795668301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1a380 0x55738cf1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.553+0000 7f5a72868740 1 -- 172.21.15.2:0/795668301 >> 172.21.15.2:0/795668301 conn(0x55738cf14840 msgr2=0x55738cf14c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:29.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.553+0000 7f5a72868740 1 -- 172.21.15.2:0/795668301 shutdown_connections 2024-03-15T13:01:29.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.553+0000 7f5a72868740 1 -- 172.21.15.2:0/795668301 wait complete. 2024-03-15T13:01:29.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.554+0000 7f5a72868740 1 -- start start 2024-03-15T13:01:29.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.554+0000 7f5a72868740 1 Processor -- start 2024-03-15T13:01:29.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.554+0000 7f5a72868740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1a380 0x55738cf207e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.554+0000 7f5a72868740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf1ad30 0x55738cf20d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.554+0000 7f5a72868740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf238b0 0x55738cf21260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.554+0000 7f5a72868740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738ce26f40 con 0x55738cf1ad30 2024-03-15T13:01:29.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.554+0000 7f5a72868740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738cb554b0 con 0x55738cf1a380 2024-03-15T13:01:29.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.554+0000 7f5a72868740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738ce07670 con 0x55738cf238b0 2024-03-15T13:01:29.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.554+0000 7f5a72868740 10 client.0 _ll_register_callbacks cb 0x55738ce74f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:29.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.554+0000 7f5a6e221640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf238b0 0x55738cf21260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.554+0000 7f5a6da20640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1a380 0x55738cf207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.554+0000 7f5a6e221640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf238b0 0x55738cf21260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37620/0 (socket says 172.21.15.2:37620) 2024-03-15T13:01:29.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.554+0000 7f5a6d21f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf1ad30 0x55738cf20d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.554+0000 7f5a6e221640 1 -- 172.21.15.2:0/3057159652 learned_addr learned my addr 172.21.15.2:0/3057159652 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:29.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.554+0000 7f5a6da20640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1a380 0x55738cf207e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59760/0 (socket says 172.21.15.2:59760) 2024-03-15T13:01:29.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.555+0000 7f5a6e221640 1 -- 172.21.15.2:0/3057159652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1a380 msgr2=0x55738cf207e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.555+0000 7f5a6e221640 1 --2- 172.21.15.2:0/3057159652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1a380 0x55738cf207e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.555+0000 7f5a6e221640 1 -- 172.21.15.2:0/3057159652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf1ad30 msgr2=0x55738cf20d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.555+0000 7f5a6e221640 1 --2- 172.21.15.2:0/3057159652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf1ad30 0x55738cf20d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.555+0000 7f5a6e221640 1 -- 172.21.15.2:0/3057159652 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a64009a10 con 0x55738cf238b0 2024-03-15T13:01:29.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.555+0000 7f5a6d21f640 1 --2- 172.21.15.2:0/3057159652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf1ad30 0x55738cf20d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:29.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.555+0000 7f5a6e221640 1 --2- 172.21.15.2:0/3057159652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf238b0 0x55738cf21260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5a60011f70 tx=0x7f5a60004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d99e6fcefb0ee35 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:29.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.555+0000 7f5a517fa640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:01:29.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.555+0000 7f5a517fa640 1 -- 172.21.15.2:0/3057159652 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6001ed30 con 0x55738cf238b0 2024-03-15T13:01:29.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.555+0000 7f5a517fa640 1 -- 172.21.15.2:0/3057159652 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5a60025070 con 0x55738cf238b0 2024-03-15T13:01:29.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.555+0000 7f5a72868740 1 -- 172.21.15.2:0/3057159652 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55738ce5bf40 con 0x55738cf238b0 2024-03-15T13:01:29.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.555+0000 7f5a517fa640 1 -- 172.21.15.2:0/3057159652 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a6001ee90 con 0x55738cf238b0 2024-03-15T13:01:29.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.555+0000 7f5a72868740 10 client.26347 fetch_fsmap learned FSMap version 35 2024-03-15T13:01:29.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.555+0000 7f5a72868740 1 -- 172.21.15.2:0/3057159652 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55738ce80320 con 0x55738cf238b0 2024-03-15T13:01:29.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.556+0000 7f5a517fa640 1 -- 172.21.15.2:0/3057159652 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55738ce5bf40 con 0x55738cf238b0 2024-03-15T13:01:29.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.556+0000 7f5a517fa640 1 -- 172.21.15.2:0/3057159652 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 35) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f5a6001f530 con 0x55738cf238b0 2024-03-15T13:01:29.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.556+0000 7f5a72868740 10 client.26347 fetch_fsmap finished waiting for FSMap version 35 2024-03-15T13:01:29.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.556+0000 7f5a72868740 10 client.26347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:01:29.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.556+0000 7f5a72868740 20 client.26347 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:29.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.556+0000 7f5a72868740 1 --2- 172.21.15.2:0/3057159652 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55738cf33f70 0x55738cf36390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.556+0000 7f5a72868740 4 client.26347 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:01:29.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.556+0000 7f5a72868740 1 -- 172.21.15.2:0/3057159652 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55738ce78310 con 0x55738cf33f70 2024-03-15T13:01:29.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.556+0000 7f5a6da20640 1 --2- 172.21.15.2:0/3057159652 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55738cf33f70 0x55738cf36390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.557+0000 7f5a517fa640 10 client.26347 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:29.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.557+0000 7f5a6da20640 1 --2- 172.21.15.2:0/3057159652 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55738cf33f70 0x55738cf36390 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c012116ac79533 server_cookie=ed34ec0a26191e57 in_seq=0 out_seq=0 2024-03-15T13:01:29.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.561+0000 7f5a517fa640 1 -- 172.21.15.2:0/3057159652 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55738ce78310 con 0x55738cf33f70 2024-03-15T13:01:29.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.561+0000 7f5a517fa640 10 client.26347 handle_command_reply: tid=0 2024-03-15T13:01:29.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.561+0000 7f5a72868740 1 client.26347 shutdown 2024-03-15T13:01:29.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.561+0000 7f5a72868740 1 -- 172.21.15.2:0/3057159652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf238b0 msgr2=0x55738cf21260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.561+0000 7f5a72868740 1 --2- 172.21.15.2:0/3057159652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf238b0 0x55738cf21260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5a60011f70 tx=0x7f5a60004a40 comp rx=0 tx=0).stop 2024-03-15T13:01:29.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.561+0000 7f5a72868740 1 -- 172.21.15.2:0/3057159652 shutdown_connections 2024-03-15T13:01:29.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.561+0000 7f5a72868740 1 --2- 172.21.15.2:0/3057159652 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55738cf33f70 0x55738cf36390 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.561+0000 7f5a72868740 1 --2- 172.21.15.2:0/3057159652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf238b0 0x55738cf21260 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.561+0000 7f5a72868740 1 --2- 172.21.15.2:0/3057159652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf1ad30 0x55738cf20d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.561+0000 7f5a72868740 1 --2- 172.21.15.2:0/3057159652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1a380 0x55738cf207e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.561+0000 7f5a72868740 1 -- 172.21.15.2:0/3057159652 >> 172.21.15.2:0/3057159652 conn(0x55738cf14840 msgr2=0x55738cf167a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:29.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.561+0000 7f5a517fa640 0 client.26347 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:29.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.562+0000 7f5a72868740 1 -- 172.21.15.2:0/3057159652 shutdown_connections 2024-03-15T13:01:29.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.562+0000 7f5a72868740 1 -- 172.21.15.2:0/3057159652 wait complete. 2024-03-15T13:01:29.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.562+0000 7f5a72868740 20 client.26347 trim_cache size 0 max 16384 2024-03-15T13:01:29.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a72868740 1 Processor -- start 2024-03-15T13:01:29.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a72868740 1 -- start start 2024-03-15T13:01:29.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a72868740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1a380 0x55738cf22f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a72868740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1ad30 0x55738cf2ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a72868740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf2f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a72868740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738cf202c0 con 0x55738cf238b0 2024-03-15T13:01:29.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a72868740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738cf19cd0 con 0x55738cf1ad30 2024-03-15T13:01:29.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a72868740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738cf30930 con 0x55738cf1a380 2024-03-15T13:01:29.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a6e221640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf2f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a6e221640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf2f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51072/0 (socket says 172.21.15.2:51072) 2024-03-15T13:01:29.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a6d21f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1ad30 0x55738cf2ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a6e221640 1 -- 172.21.15.2:0/3998294019 learned_addr learned my addr 172.21.15.2:0/3998294019 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:29.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a6da20640 1 --2- 172.21.15.2:0/3998294019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1a380 0x55738cf22f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a6e221640 1 -- 172.21.15.2:0/3998294019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1a380 msgr2=0x55738cf22f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a6e221640 1 --2- 172.21.15.2:0/3998294019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1a380 0x55738cf22f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.580+0000 7f5a6e221640 1 -- 172.21.15.2:0/3998294019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1ad30 msgr2=0x55738cf2ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.581+0000 7f5a6e221640 1 --2- 172.21.15.2:0/3998294019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1ad30 0x55738cf2ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.581+0000 7f5a6e221640 1 -- 172.21.15.2:0/3998294019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55738ce1cc00 con 0x55738cf238b0 2024-03-15T13:01:29.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.581+0000 7f5a6e221640 1 --2- 172.21.15.2:0/3998294019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf2f340 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5a60000f80 tx=0x7f5a6001fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae0de1876aa7fa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:29.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.581+0000 7f5a537fe640 1 -- 172.21.15.2:0/3998294019 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6001f610 con 0x55738cf238b0 2024-03-15T13:01:29.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.581+0000 7f5a537fe640 1 -- 172.21.15.2:0/3998294019 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5a60023070 con 0x55738cf238b0 2024-03-15T13:01:29.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.581+0000 7f5a537fe640 1 -- 172.21.15.2:0/3998294019 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60003cf0 con 0x55738cf238b0 2024-03-15T13:01:29.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.582+0000 7f5a72868740 1 -- 172.21.15.2:0/3998294019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 msgr2=0x55738cf2f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.582+0000 7f5a72868740 1 --2- 172.21.15.2:0/3998294019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf2f340 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5a60000f80 tx=0x7f5a6001fb00 comp rx=0 tx=0).stop 2024-03-15T13:01:29.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.582+0000 7f5a72868740 1 -- 172.21.15.2:0/3998294019 shutdown_connections 2024-03-15T13:01:29.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.582+0000 7f5a72868740 1 --2- 172.21.15.2:0/3998294019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf2f340 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.582+0000 7f5a72868740 1 --2- 172.21.15.2:0/3998294019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1ad30 0x55738cf2ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.582+0000 7f5a72868740 1 --2- 172.21.15.2:0/3998294019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1a380 0x55738cf22f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.582+0000 7f5a72868740 1 -- 172.21.15.2:0/3998294019 >> 172.21.15.2:0/3998294019 conn(0x55738cf14840 msgr2=0x55738cf1f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:29.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.582+0000 7f5a72868740 1 -- 172.21.15.2:0/3998294019 shutdown_connections 2024-03-15T13:01:29.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.582+0000 7f5a72868740 1 -- 172.21.15.2:0/3998294019 wait complete. 2024-03-15T13:01:29.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.582+0000 7f5a72868740 1 -- start start 2024-03-15T13:01:29.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.582+0000 7f5a72868740 1 Processor -- start 2024-03-15T13:01:29.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.582+0000 7f5a72868740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1a380 0x55738cf2b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.582+0000 7f5a72868740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1ad30 0x55738cf26d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a72868740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf27450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a72868740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738cf27b20 con 0x55738cf238b0 2024-03-15T13:01:29.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a72868740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738cf8e840 con 0x55738cf1a380 2024-03-15T13:01:29.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a72868740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738cf8e980 con 0x55738cf1ad30 2024-03-15T13:01:29.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a72868740 10 client.0 _ll_register_callbacks cb 0x7f5a60011b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:29.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a6e221640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf27450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a6d21f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1ad30 0x55738cf26d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a6d21f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1ad30 0x55738cf26d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37638/0 (socket says 172.21.15.2:37638) 2024-03-15T13:01:29.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a6da20640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1a380 0x55738cf2b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a6e221640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf27450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51084/0 (socket says 172.21.15.2:51084) 2024-03-15T13:01:29.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a6d21f640 1 -- 172.21.15.2:0/312052489 learned_addr learned my addr 172.21.15.2:0/312052489 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:29.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a6d21f640 1 -- 172.21.15.2:0/312052489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1a380 msgr2=0x55738cf2b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a6d21f640 1 --2- 172.21.15.2:0/312052489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1a380 0x55738cf2b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a6d21f640 1 -- 172.21.15.2:0/312052489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 msgr2=0x55738cf27450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a6d21f640 1 --2- 172.21.15.2:0/312052489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf27450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a6d21f640 1 -- 172.21.15.2:0/312052489 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55738ce80320 con 0x55738cf1ad30 2024-03-15T13:01:29.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a6da20640 1 --2- 172.21.15.2:0/312052489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1a380 0x55738cf2b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:29.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a6e221640 1 --2- 172.21.15.2:0/312052489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf27450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:29.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a6d21f640 1 --2- 172.21.15.2:0/312052489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1ad30 0x55738cf26d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5a580b8e40 tx=0x7f5a580f35b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9345f13ddcacd888 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:29.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a517fa640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:01:29.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a517fa640 1 -- 172.21.15.2:0/312052489 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a580bc340 con 0x55738cf1ad30 2024-03-15T13:01:29.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a72868740 1 -- 172.21.15.2:0/312052489 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55738cea7900 con 0x55738cf1ad30 2024-03-15T13:01:29.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.583+0000 7f5a517fa640 1 -- 172.21.15.2:0/312052489 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5a580bd0e0 con 0x55738cf1ad30 2024-03-15T13:01:29.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.584+0000 7f5a517fa640 1 -- 172.21.15.2:0/312052489 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a580bc9e0 con 0x55738cf1ad30 2024-03-15T13:01:29.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.584+0000 7f5a72868740 10 client.26353 fetch_fsmap learned FSMap version 35 2024-03-15T13:01:29.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.584+0000 7f5a72868740 1 -- 172.21.15.2:0/312052489 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55738ce42910 con 0x55738cf1ad30 2024-03-15T13:01:29.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.584+0000 7f5a517fa640 1 -- 172.21.15.2:0/312052489 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5a580d9080 con 0x55738cf1ad30 2024-03-15T13:01:29.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.584+0000 7f5a517fa640 1 -- 172.21.15.2:0/312052489 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 35) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f5a58046070 con 0x55738cf1ad30 2024-03-15T13:01:29.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.584+0000 7f5a72868740 10 client.26353 fetch_fsmap finished waiting for FSMap version 35 2024-03-15T13:01:29.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.584+0000 7f5a72868740 10 client.26353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:01:29.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.584+0000 7f5a72868740 20 client.26353 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:29.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.584+0000 7f5a72868740 1 --2- 172.21.15.2:0/312052489 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55738cf8fda0 0x55738cf92190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.584+0000 7f5a72868740 4 client.26353 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:01:29.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.584+0000 7f5a72868740 1 -- 172.21.15.2:0/312052489 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55738cb81860 con 0x55738cf8fda0 2024-03-15T13:01:29.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.584+0000 7f5a6da20640 1 --2- 172.21.15.2:0/312052489 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55738cf8fda0 0x55738cf92190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.585+0000 7f5a6da20640 1 --2- 172.21.15.2:0/312052489 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55738cf8fda0 0x55738cf92190 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f16de99908529b02 server_cookie=440e0deadd3051b0 in_seq=0 out_seq=0 2024-03-15T13:01:29.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.585+0000 7f5a517fa640 10 client.26353 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:29.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.586+0000 7f5a517fa640 1 -- 172.21.15.2:0/312052489 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55738cb81860 con 0x55738cf8fda0 2024-03-15T13:01:29.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.586+0000 7f5a517fa640 10 client.26353 handle_command_reply: tid=0 2024-03-15T13:01:29.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.586+0000 7f5a72868740 1 client.26353 shutdown 2024-03-15T13:01:29.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.586+0000 7f5a72868740 1 -- 172.21.15.2:0/312052489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1ad30 msgr2=0x55738cf26d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.586+0000 7f5a72868740 1 --2- 172.21.15.2:0/312052489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1ad30 0x55738cf26d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5a580b8e40 tx=0x7f5a580f35b0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.586+0000 7f5a72868740 1 -- 172.21.15.2:0/312052489 shutdown_connections 2024-03-15T13:01:29.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.586+0000 7f5a72868740 1 --2- 172.21.15.2:0/312052489 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55738cf8fda0 0x55738cf92190 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.586+0000 7f5a72868740 1 --2- 172.21.15.2:0/312052489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55738cf238b0 0x55738cf27450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.586+0000 7f5a72868740 1 --2- 172.21.15.2:0/312052489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55738cf1ad30 0x55738cf26d10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.586+0000 7f5a72868740 1 --2- 172.21.15.2:0/312052489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738cf1a380 0x55738cf2b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.586+0000 7f5a517fa640 0 client.26353 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:29.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.586+0000 7f5a72868740 1 -- 172.21.15.2:0/312052489 >> 172.21.15.2:0/312052489 conn(0x55738cf14840 msgr2=0x55738cf15e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:29.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.586+0000 7f5a72868740 1 -- 172.21.15.2:0/312052489 shutdown_connections 2024-03-15T13:01:29.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.586+0000 7f5a72868740 1 -- 172.21.15.2:0/312052489 wait complete. 2024-03-15T13:01:29.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.587+0000 7f5a72868740 20 client.26353 trim_cache size 0 max 16384 2024-03-15T13:01:29.577 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:01:29.577 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:01:29.577 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "341d08a0-e4ef-480d-bbd4-14e25675ae21", 2024-03-15T13:01:29.577 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:01:29.577 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:01:29.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.587+0000 7f5a6fc84640 1 -- 172.21.15.2:0/3524397800 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5a540a1ba0 msgr2=0x7f5a540a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.587+0000 7f5a6fc84640 1 --2- 172.21.15.2:0/3524397800 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5a540a1ba0 0x7f5a540a4060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5a60005f40 tx=0x7f5a60009600 comp rx=0 tx=0).stop 2024-03-15T13:01:29.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.587+0000 7f5a6fc84640 1 -- 172.21.15.2:0/3524397800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a681056a0 msgr2=0x7f5a6819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.587+0000 7f5a6fc84640 1 --2- 172.21.15.2:0/3524397800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a681056a0 0x7f5a6819a280 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5a580098c0 tx=0x7f5a5802fd70 comp rx=0 tx=0).stop 2024-03-15T13:01:29.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.587+0000 7f5a6fc84640 1 -- 172.21.15.2:0/3524397800 shutdown_connections 2024-03-15T13:01:29.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.587+0000 7f5a6fc84640 1 --2- 172.21.15.2:0/3524397800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a6819e600 0x7f5a6819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.587+0000 7f5a6fc84640 1 --2- 172.21.15.2:0/3524397800 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5a540a1ba0 0x7f5a540a4060 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.587+0000 7f5a6fc84640 1 --2- 172.21.15.2:0/3524397800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a681056a0 0x7f5a6819a280 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.587+0000 7f5a6fc84640 1 --2- 172.21.15.2:0/3524397800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a681005d0 0x7f5a68199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.587+0000 7f5a6fc84640 1 -- 172.21.15.2:0/3524397800 >> 172.21.15.2:0/3524397800 conn(0x7f5a680774f0 msgr2=0x7f5a680fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:29.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.587+0000 7f5a6fc84640 1 -- 172.21.15.2:0/3524397800 shutdown_connections 2024-03-15T13:01:29.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.587+0000 7f5a6fc84640 1 -- 172.21.15.2:0/3524397800 wait complete. 2024-03-15T13:01:29.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:01:29.584 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:01:29.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.709+0000 7fdb02047640 1 -- 172.21.15.2:0/1498411940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdafc1005e0 msgr2=0x7fdafc1009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.709+0000 7fdb02047640 1 --2- 172.21.15.2:0/1498411940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdafc1005e0 0x7fdafc1009c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009930 tx=0x7fdaec02bc30 comp rx=0 tx=0).stop 2024-03-15T13:01:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.709+0000 7fdb02047640 1 -- 172.21.15.2:0/1498411940 shutdown_connections 2024-03-15T13:01:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.709+0000 7fdb02047640 1 --2- 172.21.15.2:0/1498411940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdafc1056b0 0x7fdafc10d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.709+0000 7fdb02047640 1 --2- 172.21.15.2:0/1498411940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdafc100f90 0x7fdafc104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.709+0000 7fdb02047640 1 --2- 172.21.15.2:0/1498411940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdafc1005e0 0x7fdafc1009c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.709+0000 7fdb02047640 1 -- 172.21.15.2:0/1498411940 >> 172.21.15.2:0/1498411940 conn(0x7fdafc0775a0 msgr2=0x7fdafc0779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.709+0000 7fdb02047640 1 -- 172.21.15.2:0/1498411940 shutdown_connections 2024-03-15T13:01:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.709+0000 7fdb02047640 1 -- 172.21.15.2:0/1498411940 wait complete. 2024-03-15T13:01:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.709+0000 7fdb02047640 1 Processor -- start 2024-03-15T13:01:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.709+0000 7fdb02047640 1 -- start start 2024-03-15T13:01:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdb02047640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdafc1005e0 0x7fdafc19b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdb02047640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdafc1056b0 0x7fdafc19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdb02047640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdafc195710 0x7fdafc195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdb02047640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc1961e0 con 0x7fdafc1005e0 2024-03-15T13:01:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdb02047640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc196350 con 0x7fdafc1056b0 2024-03-15T13:01:29.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdb02047640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc196510 con 0x7fdafc195710 2024-03-15T13:01:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdafb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdafc1005e0 0x7fdafc19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdafb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdafc1005e0 0x7fdafc19b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51108/0 (socket says 172.21.15.2:51108) 2024-03-15T13:01:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdafb7fe640 1 -- 172.21.15.2:0/891952067 learned_addr learned my addr 172.21.15.2:0/891952067 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdafaffd640 1 --2- 172.21.15.2:0/891952067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdafc1056b0 0x7fdafc19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdafbfff640 1 --2- 172.21.15.2:0/891952067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdafc195710 0x7fdafc195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdafb7fe640 1 -- 172.21.15.2:0/891952067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdafc195710 msgr2=0x7fdafc195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdafb7fe640 1 --2- 172.21.15.2:0/891952067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdafc195710 0x7fdafc195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdafb7fe640 1 -- 172.21.15.2:0/891952067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdafc1056b0 msgr2=0x7fdafc19bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdafb7fe640 1 --2- 172.21.15.2:0/891952067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdafc1056b0 0x7fdafc19bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdafb7fe640 1 -- 172.21.15.2:0/891952067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec009560 con 0x7fdafc1005e0 2024-03-15T13:01:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdafb7fe640 1 --2- 172.21.15.2:0/891952067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdafc1005e0 0x7fdafc19b650 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdaec000c00 tx=0x7fdaec02cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.710+0000 7fdaf8ff9640 1 -- 172.21.15.2:0/891952067 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec004250 con 0x7fdafc1005e0 2024-03-15T13:01:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.711+0000 7fdaf8ff9640 1 -- 172.21.15.2:0/891952067 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdaec0043b0 con 0x7fdafc1005e0 2024-03-15T13:01:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.711+0000 7fdaf8ff9640 1 -- 172.21.15.2:0/891952067 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec003cd0 con 0x7fdafc1005e0 2024-03-15T13:01:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.711+0000 7fdb02047640 1 -- 172.21.15.2:0/891952067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdafc1a2450 con 0x7fdafc1005e0 2024-03-15T13:01:29.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.711+0000 7fdb02047640 1 -- 172.21.15.2:0/891952067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdafc1a2930 con 0x7fdafc1005e0 2024-03-15T13:01:29.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.713+0000 7fdaf8ff9640 1 -- 172.21.15.2:0/891952067 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fdaec005000 con 0x7fdafc1005e0 2024-03-15T13:01:29.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.713+0000 7fdb04c28740 1 Processor -- start 2024-03-15T13:01:29.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.713+0000 7fdb04c28740 1 -- start start 2024-03-15T13:01:29.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.713+0000 7fdaf8ff9640 1 --2- 172.21.15.2:0/891952067 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdae40a1ba0 0x7fdae40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.713+0000 7fdb04c28740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151f2b0 0x55871151f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.713+0000 7fdb04c28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711528270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.713+0000 7fdb04c28740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587115287b0 0x55871152aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.713+0000 7fdb04c28740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871148e280 con 0x55871151f2b0 2024-03-15T13:01:29.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.713+0000 7fdb04c28740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871148b7b0 con 0x5587115287b0 2024-03-15T13:01:29.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdb04c28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558711486f90 con 0x55871151fbd0 2024-03-15T13:01:29.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdafb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151f2b0 0x55871151f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdafaffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711528270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdafbfff640 1 --2- 172.21.15.2:0/891952067 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdae40a1ba0 0x7fdae40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdafaffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711528270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37660/0 (socket says 172.21.15.2:37660) 2024-03-15T13:01:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdafaffd640 1 -- 172.21.15.2:0/4207691653 learned_addr learned my addr 172.21.15.2:0/4207691653 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdafbfff640 1 --2- 172.21.15.2:0/4207691653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587115287b0 0x55871152aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdaf8ff9640 1 -- 172.21.15.2:0/891952067 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fdaec0efbc0 con 0x7fdafc1005e0 2024-03-15T13:01:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdafaffd640 1 -- 172.21.15.2:0/4207691653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587115287b0 msgr2=0x55871152aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdafaffd640 1 --2- 172.21.15.2:0/4207691653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587115287b0 0x55871152aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdafaffd640 1 -- 172.21.15.2:0/4207691653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151f2b0 msgr2=0x55871151f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdafaffd640 1 --2- 172.21.15.2:0/4207691653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151f2b0 0x55871151f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdafaffd640 1 -- 172.21.15.2:0/4207691653 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558711445570 con 0x55871151fbd0 2024-03-15T13:01:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdafb7fe640 1 --2- 172.21.15.2:0/4207691653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151f2b0 0x55871151f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdafbfff640 1 --2- 172.21.15.2:0/891952067 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdae40a1ba0 0x7fdae40a4060 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4005fd0 tx=0x7fdaf4009b00 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdafaffd640 1 --2- 172.21.15.2:0/4207691653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711528270 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000bf80 tx=0x7fdaf0014650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=337bf1174e336f67 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:29.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdae1ffb640 1 -- 172.21.15.2:0/4207691653 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0019640 con 0x55871151fbd0 2024-03-15T13:01:29.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdae1ffb640 1 -- 172.21.15.2:0/4207691653 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdaf0013040 con 0x55871151fbd0 2024-03-15T13:01:29.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.714+0000 7fdae1ffb640 1 -- 172.21.15.2:0/4207691653 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0018b90 con 0x55871151fbd0 2024-03-15T13:01:29.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.715+0000 7fdb04c28740 1 -- 172.21.15.2:0/4207691653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 msgr2=0x558711528270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.715+0000 7fdb04c28740 1 --2- 172.21.15.2:0/4207691653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711528270 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000bf80 tx=0x7fdaf0014650 comp rx=0 tx=0).stop 2024-03-15T13:01:29.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.715+0000 7fdb04c28740 1 -- 172.21.15.2:0/4207691653 shutdown_connections 2024-03-15T13:01:29.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.715+0000 7fdb04c28740 1 --2- 172.21.15.2:0/4207691653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587115287b0 0x55871152aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.715+0000 7fdb04c28740 1 --2- 172.21.15.2:0/4207691653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711528270 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.715+0000 7fdb04c28740 1 --2- 172.21.15.2:0/4207691653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151f2b0 0x55871151f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.715+0000 7fdb04c28740 1 -- 172.21.15.2:0/4207691653 >> 172.21.15.2:0/4207691653 conn(0x558711519990 msgr2=0x558711519d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:29.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.715+0000 7fdb04c28740 1 -- 172.21.15.2:0/4207691653 shutdown_connections 2024-03-15T13:01:29.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.715+0000 7fdb04c28740 1 -- 172.21.15.2:0/4207691653 wait complete. 2024-03-15T13:01:29.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.716+0000 7fdb04c28740 1 -- start start 2024-03-15T13:01:29.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.716+0000 7fdb04c28740 1 Processor -- start 2024-03-15T13:01:29.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.716+0000 7fdb04c28740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55871151f2b0 0x558711528200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.716+0000 7fdb04c28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711525590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.716+0000 7fdb04c28740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587115287b0 0x558711525cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.716+0000 7fdb04c28740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871115a430 con 0x5587115287b0 2024-03-15T13:01:29.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.716+0000 7fdb04c28740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558711116e00 con 0x55871151f2b0 2024-03-15T13:01:29.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.716+0000 7fdb04c28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871140bfd0 con 0x55871151fbd0 2024-03-15T13:01:29.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.716+0000 7fdb04c28740 10 client.0 _ll_register_callbacks cb 0x558711461fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:29.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.716+0000 7fdafbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587115287b0 0x558711525cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.716+0000 7fdafaffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711525590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.716+0000 7fdafbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587115287b0 0x558711525cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51136/0 (socket says 172.21.15.2:51136) 2024-03-15T13:01:29.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.716+0000 7fdafb7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55871151f2b0 0x558711528200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.716+0000 7fdafaffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711525590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37672/0 (socket says 172.21.15.2:37672) 2024-03-15T13:01:29.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.716+0000 7fdafbfff640 1 -- 172.21.15.2:0/4287534203 learned_addr learned my addr 172.21.15.2:0/4287534203 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:29.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdafbfff640 1 -- 172.21.15.2:0/4287534203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 msgr2=0x558711525590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdafbfff640 1 --2- 172.21.15.2:0/4287534203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711525590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdafbfff640 1 -- 172.21.15.2:0/4287534203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55871151f2b0 msgr2=0x558711528200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdafbfff640 1 --2- 172.21.15.2:0/4287534203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55871151f2b0 0x558711528200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdafbfff640 1 -- 172.21.15.2:0/4287534203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf000bbb0 con 0x5587115287b0 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdafb7fe640 1 --2- 172.21.15.2:0/4287534203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55871151f2b0 0x558711528200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdafaffd640 1 --2- 172.21.15.2:0/4287534203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711525590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdafbfff640 1 --2- 172.21.15.2:0/4287534203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587115287b0 0x558711525cd0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4013430 tx=0x7fdaf4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66b41659d7ba6bd server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdadaffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdadaffd640 1 -- 172.21.15.2:0/4287534203 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4018070 con 0x5587115287b0 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdadaffd640 1 -- 172.21.15.2:0/4287534203 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fdaf400fc90 con 0x5587115287b0 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdb04c28740 1 -- 172.21.15.2:0/4287534203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587111867e0 con 0x5587115287b0 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdadaffd640 1 -- 172.21.15.2:0/4287534203 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaf400fe50 con 0x5587115287b0 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdb04c28740 10 client.16569 fetch_fsmap learned FSMap version 35 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdb04c28740 1 -- 172.21.15.2:0/4287534203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558711485610 con 0x5587115287b0 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.717+0000 7fdadaffd640 1 -- 172.21.15.2:0/4287534203 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5587111867e0 con 0x5587115287b0 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.718+0000 7fdadaffd640 1 -- 172.21.15.2:0/4287534203 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 35) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fdaf4020400 con 0x5587115287b0 2024-03-15T13:01:29.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.718+0000 7fdb04c28740 10 client.16569 fetch_fsmap finished waiting for FSMap version 35 2024-03-15T13:01:29.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.718+0000 7fdb04c28740 10 client.16569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:01:29.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.718+0000 7fdb04c28740 20 client.16569 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:29.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.718+0000 7fdb04c28740 1 --2- 172.21.15.2:0/4287534203 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558711538a30 0x55871153ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.718+0000 7fdb04c28740 4 client.16569 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:01:29.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.718+0000 7fdb04c28740 1 -- 172.21.15.2:0/4287534203 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587111c3720 con 0x558711538a30 2024-03-15T13:01:29.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.718+0000 7fdafb7fe640 1 --2- 172.21.15.2:0/4287534203 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558711538a30 0x55871153ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.718+0000 7fdafb7fe640 1 --2- 172.21.15.2:0/4287534203 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558711538a30 0x55871153ae50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=230c174251662653 server_cookie=bb9d87ebd819dbdb in_seq=0 out_seq=0 2024-03-15T13:01:29.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.718+0000 7fdadaffd640 10 client.16569 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:29.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.720+0000 7fdadaffd640 1 -- 172.21.15.2:0/4287534203 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5587111c3720 con 0x558711538a30 2024-03-15T13:01:29.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.720+0000 7fdadaffd640 10 client.16569 handle_command_reply: tid=0 2024-03-15T13:01:29.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.720+0000 7fdb04c28740 1 client.16569 shutdown 2024-03-15T13:01:29.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.720+0000 7fdb04c28740 1 -- 172.21.15.2:0/4287534203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587115287b0 msgr2=0x558711525cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.720+0000 7fdb04c28740 1 --2- 172.21.15.2:0/4287534203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587115287b0 0x558711525cd0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4013430 tx=0x7fdaf4004ab0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.720+0000 7fdb04c28740 1 -- 172.21.15.2:0/4287534203 shutdown_connections 2024-03-15T13:01:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.720+0000 7fdb04c28740 1 --2- 172.21.15.2:0/4287534203 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558711538a30 0x55871153ae50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.720+0000 7fdadaffd640 0 client.16569 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.720+0000 7fdb04c28740 1 --2- 172.21.15.2:0/4287534203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587115287b0 0x558711525cd0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.720+0000 7fdb04c28740 1 --2- 172.21.15.2:0/4287534203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711525590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.720+0000 7fdb04c28740 1 --2- 172.21.15.2:0/4287534203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55871151f2b0 0x558711528200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.720+0000 7fdb04c28740 1 -- 172.21.15.2:0/4287534203 >> 172.21.15.2:0/4287534203 conn(0x558711519990 msgr2=0x55871151b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.720+0000 7fdb04c28740 1 -- 172.21.15.2:0/4287534203 shutdown_connections 2024-03-15T13:01:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.720+0000 7fdb04c28740 1 -- 172.21.15.2:0/4287534203 wait complete. 2024-03-15T13:01:29.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.720+0000 7fdb04c28740 20 client.16569 trim_cache size 0 max 16384 2024-03-15T13:01:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdb04c28740 1 Processor -- start 2024-03-15T13:01:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdb04c28740 1 -- start start 2024-03-15T13:01:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdb04c28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558711519990 0x558711527cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdb04c28740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55871151f2b0 0x558711528200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdb04c28740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151fbd0 0x558711533b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdb04c28740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871151e630 con 0x55871151fbd0 2024-03-15T13:01:29.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdb04c28740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871151ef10 con 0x55871151f2b0 2024-03-15T13:01:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdb04c28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871151a9e0 con 0x558711519990 2024-03-15T13:01:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdafbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151fbd0 0x558711533b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdafbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151fbd0 0x558711533b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51148/0 (socket says 172.21.15.2:51148) 2024-03-15T13:01:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdafbfff640 1 -- 172.21.15.2:0/1172440533 learned_addr learned my addr 172.21.15.2:0/1172440533 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdafb7fe640 1 --2- 172.21.15.2:0/1172440533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558711519990 0x558711527cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdafaffd640 1 --2- 172.21.15.2:0/1172440533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55871151f2b0 0x558711528200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdafbfff640 1 -- 172.21.15.2:0/1172440533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558711519990 msgr2=0x558711527cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdafbfff640 1 --2- 172.21.15.2:0/1172440533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558711519990 0x558711527cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdafbfff640 1 -- 172.21.15.2:0/1172440533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55871151f2b0 msgr2=0x558711528200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdafbfff640 1 --2- 172.21.15.2:0/1172440533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55871151f2b0 0x558711528200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdafbfff640 1 -- 172.21.15.2:0/1172440533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558711421ac0 con 0x55871151fbd0 2024-03-15T13:01:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdafaffd640 1 --2- 172.21.15.2:0/1172440533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55871151f2b0 0x558711528200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:01:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.737+0000 7fdafbfff640 1 --2- 172.21.15.2:0/1172440533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151fbd0 0x558711533b60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400a000 tx=0x7fdaf4009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6e21e4b130498 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:29.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.738+0000 7fdae17fa640 1 -- 172.21.15.2:0/1172440533 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4018040 con 0x55871151fbd0 2024-03-15T13:01:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.738+0000 7fdae17fa640 1 -- 172.21.15.2:0/1172440533 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdaf4020400 con 0x55871151fbd0 2024-03-15T13:01:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.738+0000 7fdae17fa640 1 -- 172.21.15.2:0/1172440533 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4004660 con 0x55871151fbd0 2024-03-15T13:01:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.738+0000 7fdb04c28740 1 -- 172.21.15.2:0/1172440533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151fbd0 msgr2=0x558711533b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.738+0000 7fdb04c28740 1 --2- 172.21.15.2:0/1172440533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151fbd0 0x558711533b60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400a000 tx=0x7fdaf4009a50 comp rx=0 tx=0).stop 2024-03-15T13:01:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.738+0000 7fdb04c28740 1 -- 172.21.15.2:0/1172440533 shutdown_connections 2024-03-15T13:01:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.738+0000 7fdb04c28740 1 --2- 172.21.15.2:0/1172440533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151fbd0 0x558711533b60 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.738+0000 7fdb04c28740 1 --2- 172.21.15.2:0/1172440533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55871151f2b0 0x558711528200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.738+0000 7fdb04c28740 1 --2- 172.21.15.2:0/1172440533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558711519990 0x558711527cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.738+0000 7fdb04c28740 1 -- 172.21.15.2:0/1172440533 >> 172.21.15.2:0/1172440533 conn(0x5587115256f0 msgr2=0x558711523e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.738+0000 7fdb04c28740 1 -- 172.21.15.2:0/1172440533 shutdown_connections 2024-03-15T13:01:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.738+0000 7fdb04c28740 1 -- 172.21.15.2:0/1172440533 wait complete. 2024-03-15T13:01:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.739+0000 7fdb04c28740 1 -- start start 2024-03-15T13:01:29.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.739+0000 7fdb04c28740 1 Processor -- start 2024-03-15T13:01:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.739+0000 7fdb04c28740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558711519990 0x558711530040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.739+0000 7fdb04c28740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151f2b0 0x558711530580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.739+0000 7fdb04c28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711529a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.739+0000 7fdb04c28740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871152a0f0 con 0x55871151f2b0 2024-03-15T13:01:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.739+0000 7fdb04c28740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871152a260 con 0x558711519990 2024-03-15T13:01:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.739+0000 7fdb04c28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871152a420 con 0x55871151fbd0 2024-03-15T13:01:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.739+0000 7fdb04c28740 10 client.0 _ll_register_callbacks cb 0x7fdaf4009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.739+0000 7fdafaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151f2b0 0x558711530580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.739+0000 7fdafb7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558711519990 0x558711530040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.739+0000 7fdafb7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558711519990 0x558711530040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:59838/0 (socket says 172.21.15.2:59838) 2024-03-15T13:01:29.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.739+0000 7fdafb7fe640 1 -- 172.21.15.2:0/2487536739 learned_addr learned my addr 172.21.15.2:0/2487536739 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.739+0000 7fdafaffd640 1 --2- 172.21.15.2:0/2487536739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151f2b0 0x558711530580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51160/0 (socket says 172.21.15.2:51160) 2024-03-15T13:01:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.739+0000 7fdafbfff640 1 --2- 172.21.15.2:0/2487536739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711529a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdafaffd640 1 -- 172.21.15.2:0/2487536739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 msgr2=0x558711529a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdafaffd640 1 --2- 172.21.15.2:0/2487536739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711529a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdafaffd640 1 -- 172.21.15.2:0/2487536739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558711519990 msgr2=0x558711530040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdafaffd640 1 --2- 172.21.15.2:0/2487536739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558711519990 0x558711530040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdafaffd640 1 -- 172.21.15.2:0/2487536739 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558711485610 con 0x55871151f2b0 2024-03-15T13:01:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdafb7fe640 1 --2- 172.21.15.2:0/2487536739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558711519990 0x558711530040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdafaffd640 1 --2- 172.21.15.2:0/2487536739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151f2b0 0x558711530580 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0005d00 tx=0x7fdaf00189d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446a258f17e8551c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdadaffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:01:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdadaffd640 1 -- 172.21.15.2:0/2487536739 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0018d50 con 0x55871151f2b0 2024-03-15T13:01:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdadaffd640 1 -- 172.21.15.2:0/2487536739 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fdaf0018eb0 con 0x55871151f2b0 2024-03-15T13:01:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdb04c28740 1 -- 172.21.15.2:0/2487536739 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdaf0013040 con 0x55871151f2b0 2024-03-15T13:01:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdadaffd640 1 -- 172.21.15.2:0/2487536739 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaf0004020 con 0x55871151f2b0 2024-03-15T13:01:29.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdb04c28740 10 client.16575 fetch_fsmap learned FSMap version 35 2024-03-15T13:01:29.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdb04c28740 1 -- 172.21.15.2:0/2487536739 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5587114476a0 con 0x55871151f2b0 2024-03-15T13:01:29.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdadaffd640 1 -- 172.21.15.2:0/2487536739 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdaf0013040 con 0x55871151f2b0 2024-03-15T13:01:29.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdadaffd640 1 -- 172.21.15.2:0/2487536739 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 35) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fdaf00035f0 con 0x55871151f2b0 2024-03-15T13:01:29.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdb04c28740 10 client.16575 fetch_fsmap finished waiting for FSMap version 35 2024-03-15T13:01:29.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdb04c28740 10 client.16575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:01:29.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdb04c28740 20 client.16575 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:29.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdb04c28740 1 --2- 172.21.15.2:0/2487536739 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5587115287b0 0x5587115976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:29.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdb04c28740 4 client.16575 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:01:29.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.740+0000 7fdb04c28740 1 -- 172.21.15.2:0/2487536739 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558711212890 con 0x5587115287b0 2024-03-15T13:01:29.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.741+0000 7fdafb7fe640 1 --2- 172.21.15.2:0/2487536739 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5587115287b0 0x5587115976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:29.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.741+0000 7fdafb7fe640 1 --2- 172.21.15.2:0/2487536739 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5587115287b0 0x5587115976c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d7190a83fffeb3 server_cookie=1a30d8f691253745 in_seq=0 out_seq=0 2024-03-15T13:01:29.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.741+0000 7fdadaffd640 10 client.16575 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.750+0000 7fdadaffd640 1 -- 172.21.15.2:0/2487536739 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x558711212890 con 0x5587115287b0 2024-03-15T13:01:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.750+0000 7fdadaffd640 10 client.16575 handle_command_reply: tid=0 2024-03-15T13:01:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.750+0000 7fdb04c28740 1 client.16575 shutdown 2024-03-15T13:01:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.750+0000 7fdb04c28740 1 -- 172.21.15.2:0/2487536739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151f2b0 msgr2=0x558711530580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.750+0000 7fdb04c28740 1 --2- 172.21.15.2:0/2487536739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151f2b0 0x558711530580 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0005d00 tx=0x7fdaf00189d0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.750+0000 7fdb04c28740 1 -- 172.21.15.2:0/2487536739 shutdown_connections 2024-03-15T13:01:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.750+0000 7fdb04c28740 1 --2- 172.21.15.2:0/2487536739 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5587115287b0 0x5587115976c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.750+0000 7fdb04c28740 1 --2- 172.21.15.2:0/2487536739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55871151fbd0 0x558711529a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.750+0000 7fdb04c28740 1 --2- 172.21.15.2:0/2487536739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55871151f2b0 0x558711530580 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.750+0000 7fdadaffd640 0 client.16575 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.750+0000 7fdb04c28740 1 --2- 172.21.15.2:0/2487536739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558711519990 0x558711530040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.750+0000 7fdb04c28740 1 -- 172.21.15.2:0/2487536739 >> 172.21.15.2:0/2487536739 conn(0x5587115256f0 msgr2=0x5587115181c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.750+0000 7fdb04c28740 1 -- 172.21.15.2:0/2487536739 shutdown_connections 2024-03-15T13:01:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.750+0000 7fdb04c28740 1 -- 172.21.15.2:0/2487536739 wait complete. 2024-03-15T13:01:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.750+0000 7fdb04c28740 20 client.16575 trim_cache size 0 max 16384 2024-03-15T13:01:29.737 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:01:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (817 inodes in the stack)", 2024-03-15T13:01:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:01:29.737 INFO:teuthology.orchestra.run.smithi002.stdout: "341d08a0-e4ef-480d-bbd4-14e25675ae21": { 2024-03-15T13:01:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:01:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "341d08a0-e4ef-480d-bbd4-14e25675ae21", 2024-03-15T13:01:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:01:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:01:29.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:01:29.738 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:01:29.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.751+0000 7fdb02047640 1 -- 172.21.15.2:0/891952067 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdae40a1ba0 msgr2=0x7fdae40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.751+0000 7fdb02047640 1 --2- 172.21.15.2:0/891952067 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdae40a1ba0 0x7fdae40a4060 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4005fd0 tx=0x7fdaf4009b00 comp rx=0 tx=0).stop 2024-03-15T13:01:29.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.751+0000 7fdb02047640 1 -- 172.21.15.2:0/891952067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdafc1005e0 msgr2=0x7fdafc19b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:29.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.751+0000 7fdb02047640 1 --2- 172.21.15.2:0/891952067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdafc1005e0 0x7fdafc19b650 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdaec000c00 tx=0x7fdaec02cab0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.751+0000 7fdb02047640 1 -- 172.21.15.2:0/891952067 shutdown_connections 2024-03-15T13:01:29.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.751+0000 7fdb02047640 1 --2- 172.21.15.2:0/891952067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdafc195710 0x7fdafc195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.751+0000 7fdb02047640 1 --2- 172.21.15.2:0/891952067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdafc1056b0 0x7fdafc19bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.751+0000 7fdb02047640 1 --2- 172.21.15.2:0/891952067 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdae40a1ba0 0x7fdae40a4060 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.751+0000 7fdb02047640 1 --2- 172.21.15.2:0/891952067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdafc1005e0 0x7fdafc19b650 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:29.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.751+0000 7fdb02047640 1 -- 172.21.15.2:0/891952067 >> 172.21.15.2:0/891952067 conn(0x7fdafc0775a0 msgr2=0x7fdafc0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:29.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.751+0000 7fdb02047640 1 -- 172.21.15.2:0/891952067 shutdown_connections 2024-03-15T13:01:29.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:29.751+0000 7fdb02047640 1 -- 172.21.15.2:0/891952067 wait complete. 2024-03-15T13:01:29.747 INFO:tasks.cephfs.filesystem:scrub status for tag:341d08a0-e4ef-480d-bbd4-14e25675ae21 - {'path': '/', 'tag': '341d08a0-e4ef-480d-bbd4-14e25675ae21', 'options': 'recursive,force'} 2024-03-15T13:01:30.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:29 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:30.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:29 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:30.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:29 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:30.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:29 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:30.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:29 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:30.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:29 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:30.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:29 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:30.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:29 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:30.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:29 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:30.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:29 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:30.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:29 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:30.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:29 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:30.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:29 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:30.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:29 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:30.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:29 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:30.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:29 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:30.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:29 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:30.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:29 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:30.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:29 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:30.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:29 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:30.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:29 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:30.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:29 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:30.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:29 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:30.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:29 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:30.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:29 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:30.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:29 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:30.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:29 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:30.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:29 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:30.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:29 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:30.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:29 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.261 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:01:31.261 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.261 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:31.261 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:31.261 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.261 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:31.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:31.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:31.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:31.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:31.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:31.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: pgmap v81: 129 pgs: 129 active+clean; 203 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 231 op/s 2024-03-15T13:01:31.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:01:31.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:31.263 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.263 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:31.263 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.263 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:31.263 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:30 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:01:31.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:31.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:31.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:31.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:31.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:31.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:31.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:31.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:31.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: pgmap v81: 129 pgs: 129 active+clean; 203 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 231 op/s 2024-03-15T13:01:31.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:01:31.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:31.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:31.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:31.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:30 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:01:31.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:31.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:31.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:31.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:31.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:31.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:31.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:31.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:31.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: pgmap v81: 129 pgs: 129 active+clean; 203 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 231 op/s 2024-03-15T13:01:31.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:01:31.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:31.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:31.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:31.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:31.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:30 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:32.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:31 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:32.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:31 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:32.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:31 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:32.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:31 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:32.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:31 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:32.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:31 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:32.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:31 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:01:32.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:31 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:32.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:31 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:32.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:31 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:32.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:31 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:32.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:31 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:32.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:31 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:32.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:31 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:32.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:31 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:01:32.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:31 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:32.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:31 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:32.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:31 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:32.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:31 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:32.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:31 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:32.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:31 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:32.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:31 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:32.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:31 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:01:32.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:31 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:33.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:32 smithi002 ceph-mon[32060]: pgmap v82: 129 pgs: 129 active+clean; 203 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 738 KiB/s wr, 115 op/s 2024-03-15T13:01:33.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:32 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:33.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:32 smithi184 ceph-mon[35993]: pgmap v82: 129 pgs: 129 active+clean; 203 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 738 KiB/s wr, 115 op/s 2024-03-15T13:01:33.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:32 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:33.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:32 smithi149 ceph-mon[35927]: pgmap v82: 129 pgs: 129 active+clean; 203 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 738 KiB/s wr, 115 op/s 2024-03-15T13:01:33.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:32 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:01:35.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:34 smithi002 ceph-mon[32060]: pgmap v83: 129 pgs: 129 active+clean; 203 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 738 KiB/s wr, 115 op/s 2024-03-15T13:01:35.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:34 smithi184 ceph-mon[35993]: pgmap v83: 129 pgs: 129 active+clean; 203 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 738 KiB/s wr, 115 op/s 2024-03-15T13:01:35.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:34 smithi149 ceph-mon[35927]: pgmap v83: 129 pgs: 129 active+clean; 203 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 738 KiB/s wr, 115 op/s 2024-03-15T13:01:37.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:36 smithi002 ceph-mon[32060]: pgmap v84: 129 pgs: 129 active+clean; 207 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 209 op/s 2024-03-15T13:01:37.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:36 smithi184 ceph-mon[35993]: pgmap v84: 129 pgs: 129 active+clean; 207 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 209 op/s 2024-03-15T13:01:37.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:36 smithi149 ceph-mon[35927]: pgmap v84: 129 pgs: 129 active+clean; 207 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 209 op/s 2024-03-15T13:01:39.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:38 smithi002 ceph-mon[32060]: pgmap v85: 129 pgs: 129 active+clean; 207 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 812 KiB/s wr, 172 op/s 2024-03-15T13:01:39.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:38 smithi184 ceph-mon[35993]: pgmap v85: 129 pgs: 129 active+clean; 207 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 812 KiB/s wr, 172 op/s 2024-03-15T13:01:39.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:38 smithi149 ceph-mon[35927]: pgmap v85: 129 pgs: 129 active+clean; 207 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 812 KiB/s wr, 172 op/s 2024-03-15T13:01:40.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:39 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:01:40.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:39 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:01:40.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:39 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:01:40.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:39 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:01:40.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:39 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:01:40.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:39 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:01:40.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:39 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:01:40.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:39 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:01:40.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:39 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:01:40.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:39 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:01:40.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:39 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:01:40.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:39 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:01:41.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:40 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:01:41.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:40 smithi002 ceph-mon[32060]: pgmap v86: 129 pgs: 129 active+clean; 209 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 254 op/s 2024-03-15T13:01:41.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:40 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:01:41.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:40 smithi184 ceph-mon[35993]: pgmap v86: 129 pgs: 129 active+clean; 209 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 254 op/s 2024-03-15T13:01:41.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:40 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:01:41.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:40 smithi149 ceph-mon[35927]: pgmap v86: 129 pgs: 129 active+clean; 209 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 254 op/s 2024-03-15T13:01:43.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:42 smithi002 ceph-mon[32060]: pgmap v87: 129 pgs: 129 active+clean; 209 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 823 KiB/s wr, 175 op/s 2024-03-15T13:01:43.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:42 smithi184 ceph-mon[35993]: pgmap v87: 129 pgs: 129 active+clean; 209 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 823 KiB/s wr, 175 op/s 2024-03-15T13:01:43.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:42 smithi149 ceph-mon[35927]: pgmap v87: 129 pgs: 129 active+clean; 209 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 823 KiB/s wr, 175 op/s 2024-03-15T13:01:44.739 INFO:tasks.workunit.client.0.smithi002.stderr:+ tar xf doc.tar 2024-03-15T13:01:45.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:44 smithi002 ceph-mon[32060]: pgmap v88: 129 pgs: 129 active+clean; 209 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 823 KiB/s wr, 175 op/s 2024-03-15T13:01:45.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:44 smithi184 ceph-mon[35993]: pgmap v88: 129 pgs: 129 active+clean; 209 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 823 KiB/s wr, 175 op/s 2024-03-15T13:01:45.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:44 smithi149 ceph-mon[35927]: pgmap v88: 129 pgs: 129 active+clean; 209 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 823 KiB/s wr, 175 op/s 2024-03-15T13:01:47.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:46 smithi002 ceph-mon[32060]: pgmap v89: 129 pgs: 129 active+clean; 205 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 310 op/s 2024-03-15T13:01:47.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:46 smithi184 ceph-mon[35993]: pgmap v89: 129 pgs: 129 active+clean; 205 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 310 op/s 2024-03-15T13:01:47.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:46 smithi149 ceph-mon[35927]: pgmap v89: 129 pgs: 129 active+clean; 205 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 310 op/s 2024-03-15T13:01:49.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:48 smithi002 ceph-mon[32060]: pgmap v90: 129 pgs: 129 active+clean; 205 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 986 KiB/s wr, 216 op/s 2024-03-15T13:01:49.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:48 smithi184 ceph-mon[35993]: pgmap v90: 129 pgs: 129 active+clean; 205 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 986 KiB/s wr, 216 op/s 2024-03-15T13:01:49.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:48 smithi149 ceph-mon[35927]: pgmap v90: 129 pgs: 129 active+clean; 205 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 986 KiB/s wr, 216 op/s 2024-03-15T13:01:51.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:50 smithi002 ceph-mon[32060]: pgmap v91: 129 pgs: 129 active+clean; 209 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 384 op/s 2024-03-15T13:01:51.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:50 smithi184 ceph-mon[35993]: pgmap v91: 129 pgs: 129 active+clean; 209 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 384 op/s 2024-03-15T13:01:51.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:50 smithi149 ceph-mon[35927]: pgmap v91: 129 pgs: 129 active+clean; 209 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 384 op/s 2024-03-15T13:01:53.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:52 smithi002 ceph-mon[32060]: pgmap v92: 129 pgs: 129 active+clean; 209 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.2 MiB/s wr, 303 op/s 2024-03-15T13:01:53.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:52 smithi184 ceph-mon[35993]: pgmap v92: 129 pgs: 129 active+clean; 209 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.2 MiB/s wr, 303 op/s 2024-03-15T13:01:53.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:52 smithi149 ceph-mon[35927]: pgmap v92: 129 pgs: 129 active+clean; 209 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.2 MiB/s wr, 303 op/s 2024-03-15T13:01:55.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:54 smithi002 ceph-mon[32060]: pgmap v93: 129 pgs: 129 active+clean; 209 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.2 MiB/s wr, 303 op/s 2024-03-15T13:01:55.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:54 smithi184 ceph-mon[35993]: pgmap v93: 129 pgs: 129 active+clean; 209 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.2 MiB/s wr, 303 op/s 2024-03-15T13:01:55.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:54 smithi149 ceph-mon[35927]: pgmap v93: 129 pgs: 129 active+clean; 209 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.2 MiB/s wr, 303 op/s 2024-03-15T13:01:57.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:56 smithi002 ceph-mon[32060]: pgmap v94: 129 pgs: 129 active+clean; 223 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 414 op/s 2024-03-15T13:01:57.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:56 smithi184 ceph-mon[35993]: pgmap v94: 129 pgs: 129 active+clean; 223 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 414 op/s 2024-03-15T13:01:57.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:56 smithi149 ceph-mon[35927]: pgmap v94: 129 pgs: 129 active+clean; 223 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 414 op/s 2024-03-15T13:01:59.307 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:01:58 smithi002 ceph-mon[32060]: pgmap v95: 129 pgs: 129 active+clean; 223 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.9 MiB/s wr, 279 op/s 2024-03-15T13:01:59.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:01:58 smithi184 ceph-mon[35993]: pgmap v95: 129 pgs: 129 active+clean; 223 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.9 MiB/s wr, 279 op/s 2024-03-15T13:01:59.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:01:58 smithi149 ceph-mon[35927]: pgmap v95: 129 pgs: 129 active+clean; 223 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 1.9 MiB/s wr, 279 op/s 2024-03-15T13:01:59.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:01:59.749 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:01:59.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.898+0000 7f80f7f6c640 1 -- 172.21.15.2:0/766122142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80f01056b0 msgr2=0x7f80f0068650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:59.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.898+0000 7f80f7f6c640 1 --2- 172.21.15.2:0/766122142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80f01056b0 0x7f80f0068650 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009900 tx=0x7f80e802bc30 comp rx=0 tx=0).stop 2024-03-15T13:01:59.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.898+0000 7f80f7f6c640 1 -- 172.21.15.2:0/766122142 shutdown_connections 2024-03-15T13:01:59.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.898+0000 7f80f7f6c640 1 --2- 172.21.15.2:0/766122142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80f01056b0 0x7f80f0068650 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.898+0000 7f80f7f6c640 1 --2- 172.21.15.2:0/766122142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80f0100f90 0x7f80f0104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.898+0000 7f80f7f6c640 1 --2- 172.21.15.2:0/766122142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f01005e0 0x7f80f01009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.898+0000 7f80f7f6c640 1 -- 172.21.15.2:0/766122142 >> 172.21.15.2:0/766122142 conn(0x7f80f00775a0 msgr2=0x7f80f00779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:59.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.898+0000 7f80f7f6c640 1 -- 172.21.15.2:0/766122142 shutdown_connections 2024-03-15T13:01:59.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.898+0000 7f80f7f6c640 1 -- 172.21.15.2:0/766122142 wait complete. 2024-03-15T13:01:59.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.899+0000 7f80f7f6c640 1 Processor -- start 2024-03-15T13:01:59.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.899+0000 7f80f7f6c640 1 -- start start 2024-03-15T13:01:59.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.899+0000 7f80f7f6c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80f01005e0 0x7f80f019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.899+0000 7f80f7f6c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80f01056b0 0x7f80f019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.899+0000 7f80f7f6c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f0199a40 0x7f80f0199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.899+0000 7f80f7f6c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f019a430 con 0x7f80f01056b0 2024-03-15T13:01:59.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.899+0000 7f80f7f6c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f019a5a0 con 0x7f80f01005e0 2024-03-15T13:01:59.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.899+0000 7f80f7f6c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f019a760 con 0x7f80f0199a40 2024-03-15T13:01:59.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.899+0000 7f80f5507640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80f01056b0 0x7f80f019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.899+0000 7f80f5507640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80f01056b0 0x7f80f019fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33862/0 (socket says 172.21.15.2:33862) 2024-03-15T13:01:59.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.899+0000 7f80f5507640 1 -- 172.21.15.2:0/2550479433 learned_addr learned my addr 172.21.15.2:0/2550479433 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:59.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.899+0000 7f80f6509640 1 --2- 172.21.15.2:0/2550479433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f0199a40 0x7f80f0199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.899+0000 7f80f5d08640 1 --2- 172.21.15.2:0/2550479433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80f01005e0 0x7f80f019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.900+0000 7f80f5507640 1 -- 172.21.15.2:0/2550479433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f0199a40 msgr2=0x7f80f0199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.900+0000 7f80f5507640 1 --2- 172.21.15.2:0/2550479433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f0199a40 0x7f80f0199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.900+0000 7f80f5507640 1 -- 172.21.15.2:0/2550479433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80f01005e0 msgr2=0x7f80f019f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.900+0000 7f80f5507640 1 --2- 172.21.15.2:0/2550479433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80f01005e0 0x7f80f019f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.900+0000 7f80f5507640 1 -- 172.21.15.2:0/2550479433 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e4009840 con 0x7f80f01056b0 2024-03-15T13:01:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.900+0000 7f80f6509640 1 --2- 172.21.15.2:0/2550479433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f0199a40 0x7f80f0199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.900+0000 7f80f5507640 1 --2- 172.21.15.2:0/2550479433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80f01056b0 0x7f80f019fdd0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f80e400efd0 tx=0x7f80e400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.900+0000 7f80eeffd640 1 -- 172.21.15.2:0/2550479433 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4009b20 con 0x7f80f01056b0 2024-03-15T13:01:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.900+0000 7f80eeffd640 1 -- 172.21.15.2:0/2550479433 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f80e4003b70 con 0x7f80f01056b0 2024-03-15T13:01:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.900+0000 7f80eeffd640 1 -- 172.21.15.2:0/2550479433 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e40189c0 con 0x7f80f01056b0 2024-03-15T13:01:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.900+0000 7f80f7f6c640 1 -- 172.21.15.2:0/2550479433 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e8009560 con 0x7f80f01056b0 2024-03-15T13:01:59.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.901+0000 7f80f7f6c640 1 -- 172.21.15.2:0/2550479433 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f01a6a80 con 0x7f80f01056b0 2024-03-15T13:01:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.903+0000 7f80eeffd640 1 -- 172.21.15.2:0/2550479433 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f80e4018b20 con 0x7f80f01056b0 2024-03-15T13:01:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.903+0000 7f80fac24740 1 Processor -- start 2024-03-15T13:01:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.903+0000 7f80fac24740 1 -- start start 2024-03-15T13:01:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80eeffd640 1 --2- 172.21.15.2:0/2550479433 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f80d80a1c20 0x7f80d80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80fac24740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945db2b0 0x55c2945db690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80fac24740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945dbbd0 0x55c2945e4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80fac24740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945e47b0 0x55c2945e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80fac24740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c29454a280 con 0x55c2945db2b0 2024-03-15T13:01:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80fac24740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2945477b0 con 0x55c2945dbbd0 2024-03-15T13:01:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80fac24740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c294542f90 con 0x55c2945e47b0 2024-03-15T13:01:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80eeffd640 1 -- 172.21.15.2:0/2550479433 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f80e4014070 con 0x7f80f01056b0 2024-03-15T13:01:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80f5d08640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945db2b0 0x55c2945db690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80f5507640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945dbbd0 0x55c2945e4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80f6509640 1 --2- 172.21.15.2:0/2550479433 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f80d80a1c20 0x7f80d80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80f5d08640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945db2b0 0x55c2945db690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33866/0 (socket says 172.21.15.2:33866) 2024-03-15T13:01:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80f5d08640 1 -- 172.21.15.2:0/782349154 learned_addr learned my addr 172.21.15.2:0/782349154 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80f6509640 1 --2- 172.21.15.2:0/782349154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945e47b0 0x55c2945e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80f5d08640 1 -- 172.21.15.2:0/782349154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945e47b0 msgr2=0x55c2945e6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:59.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80f5d08640 1 --2- 172.21.15.2:0/782349154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945e47b0 0x55c2945e6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80f5d08640 1 -- 172.21.15.2:0/782349154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945dbbd0 msgr2=0x55c2945e4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:59.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80f5d08640 1 --2- 172.21.15.2:0/782349154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945dbbd0 0x55c2945e4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80f5d08640 1 -- 172.21.15.2:0/782349154 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c294501570 con 0x55c2945db2b0 2024-03-15T13:01:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.904+0000 7f80f5507640 1 --2- 172.21.15.2:0/782349154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945dbbd0 0x55c2945e4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.905+0000 7f80f6509640 1 --2- 172.21.15.2:0/2550479433 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f80d80a1c20 0x7f80d80a40e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f80e8006010 tx=0x7f80e8004f80 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.905+0000 7f80f5d08640 1 --2- 172.21.15.2:0/782349154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945db2b0 0x55c2945db690 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009c90 tx=0x7f80e0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6024d6cba491e430 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.905+0000 7f80d7fff640 1 -- 172.21.15.2:0/782349154 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0019620 con 0x55c2945db2b0 2024-03-15T13:01:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.905+0000 7f80d7fff640 1 -- 172.21.15.2:0/782349154 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f80e0009e40 con 0x55c2945db2b0 2024-03-15T13:01:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.905+0000 7f80d7fff640 1 -- 172.21.15.2:0/782349154 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0018b60 con 0x55c2945db2b0 2024-03-15T13:01:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.906+0000 7f80fac24740 1 -- 172.21.15.2:0/782349154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945db2b0 msgr2=0x55c2945db690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.906+0000 7f80fac24740 1 --2- 172.21.15.2:0/782349154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945db2b0 0x55c2945db690 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009c90 tx=0x7f80e0011810 comp rx=0 tx=0).stop 2024-03-15T13:01:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.906+0000 7f80fac24740 1 -- 172.21.15.2:0/782349154 shutdown_connections 2024-03-15T13:01:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.906+0000 7f80fac24740 1 --2- 172.21.15.2:0/782349154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945e47b0 0x55c2945e6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.906+0000 7f80fac24740 1 --2- 172.21.15.2:0/782349154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945dbbd0 0x55c2945e4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.906+0000 7f80fac24740 1 --2- 172.21.15.2:0/782349154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945db2b0 0x55c2945db690 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.906+0000 7f80fac24740 1 -- 172.21.15.2:0/782349154 >> 172.21.15.2:0/782349154 conn(0x55c2945d5990 msgr2=0x55c2945d5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:59.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.906+0000 7f80fac24740 1 -- 172.21.15.2:0/782349154 shutdown_connections 2024-03-15T13:01:59.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.906+0000 7f80fac24740 1 -- 172.21.15.2:0/782349154 wait complete. 2024-03-15T13:01:59.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.907+0000 7f80fac24740 1 -- start start 2024-03-15T13:01:59.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.907+0000 7f80fac24740 1 Processor -- start 2024-03-15T13:01:59.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.907+0000 7f80fac24740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945db2b0 0x55c2945e41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.907+0000 7f80fac24740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945dbbd0 0x55c2945e15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.907+0000 7f80fac24740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945e47b0 0x55c2945e1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.907+0000 7f80fac24740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c294216430 con 0x55c2945dbbd0 2024-03-15T13:01:59.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.907+0000 7f80fac24740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2941d2e00 con 0x55c2945e47b0 2024-03-15T13:01:59.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.907+0000 7f80fac24740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2944c7fd0 con 0x55c2945db2b0 2024-03-15T13:01:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80fac24740 10 client.0 _ll_register_callbacks cb 0x55c29451dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80f5d08640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945db2b0 0x55c2945e41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80f5507640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945dbbd0 0x55c2945e15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80f5d08640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945db2b0 0x55c2945e41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:45808/0 (socket says 172.21.15.2:45808) 2024-03-15T13:01:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80f5d08640 1 -- 172.21.15.2:0/141671709 learned_addr learned my addr 172.21.15.2:0/141671709 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80f6509640 1 --2- 172.21.15.2:0/141671709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945e47b0 0x55c2945e1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80f5d08640 1 -- 172.21.15.2:0/141671709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945e47b0 msgr2=0x55c2945e1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80f5d08640 1 --2- 172.21.15.2:0/141671709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945e47b0 0x55c2945e1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80f5d08640 1 -- 172.21.15.2:0/141671709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945dbbd0 msgr2=0x55c2945e15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80f5d08640 1 --2- 172.21.15.2:0/141671709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945dbbd0 0x55c2945e15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80f5d08640 1 -- 172.21.15.2:0/141671709 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e00098f0 con 0x55c2945db2b0 2024-03-15T13:01:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80f5d08640 1 --2- 172.21.15.2:0/141671709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945db2b0 0x55c2945e41c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f80e001b040 tx=0x7f80e0018600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e19471f52b3d0958 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:01:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80d57fa640 1 -- 172.21.15.2:0/141671709 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0005690 con 0x55c2945db2b0 2024-03-15T13:01:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80d57fa640 1 -- 172.21.15.2:0/141671709 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f80e0018c70 con 0x55c2945db2b0 2024-03-15T13:01:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.908+0000 7f80fac24740 1 -- 172.21.15.2:0/141671709 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2942427e0 con 0x55c2945db2b0 2024-03-15T13:01:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.909+0000 7f80d57fa640 1 -- 172.21.15.2:0/141671709 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80e0018e30 con 0x55c2945db2b0 2024-03-15T13:01:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.909+0000 7f80fac24740 10 client.26383 fetch_fsmap learned FSMap version 36 2024-03-15T13:01:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.909+0000 7f80fac24740 1 -- 172.21.15.2:0/141671709 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c294541610 con 0x55c2945db2b0 2024-03-15T13:01:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.909+0000 7f80d57fa640 1 -- 172.21.15.2:0/141671709 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55c2942427e0 con 0x55c2945db2b0 2024-03-15T13:01:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.909+0000 7f80d57fa640 1 -- 172.21.15.2:0/141671709 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 36) v1 ==== 7982+0+0 (secure 0 0 0) 0x7f80e0021040 con 0x55c2945db2b0 2024-03-15T13:01:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.909+0000 7f80fac24740 10 client.26383 fetch_fsmap finished waiting for FSMap version 36 2024-03-15T13:01:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.909+0000 7f80fac24740 10 client.26383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:01:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.909+0000 7f80fac24740 20 client.26383 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.909+0000 7f80fac24740 1 --2- 172.21.15.2:0/141671709 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2945f49f0 0x55c2945f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.909+0000 7f80fac24740 4 client.26383 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:01:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.909+0000 7f80fac24740 1 -- 172.21.15.2:0/141671709 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c29427f720 con 0x55c2945f49f0 2024-03-15T13:01:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.909+0000 7f80f5507640 1 --2- 172.21.15.2:0/141671709 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2945f49f0 0x55c2945f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.910+0000 7f80f5507640 1 --2- 172.21.15.2:0/141671709 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2945f49f0 0x55c2945f6e30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3587a737e403b39 server_cookie=1189154c37865f9b in_seq=0 out_seq=0 2024-03-15T13:01:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.910+0000 7f80d57fa640 10 client.26383 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.911+0000 7f80d57fa640 1 -- 172.21.15.2:0/141671709 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c29427f720 con 0x55c2945f49f0 2024-03-15T13:01:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.911+0000 7f80d57fa640 10 client.26383 handle_command_reply: tid=0 2024-03-15T13:01:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.911+0000 7f80fac24740 1 client.26383 shutdown 2024-03-15T13:01:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.911+0000 7f80fac24740 1 -- 172.21.15.2:0/141671709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945db2b0 msgr2=0x55c2945e41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.911+0000 7f80fac24740 1 --2- 172.21.15.2:0/141671709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945db2b0 0x55c2945e41c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f80e001b040 tx=0x7f80e0018600 comp rx=0 tx=0).stop 2024-03-15T13:01:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.912+0000 7f80fac24740 1 -- 172.21.15.2:0/141671709 shutdown_connections 2024-03-15T13:01:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.912+0000 7f80fac24740 1 --2- 172.21.15.2:0/141671709 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2945f49f0 0x55c2945f6e30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.912+0000 7f80fac24740 1 --2- 172.21.15.2:0/141671709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945e47b0 0x55c2945e1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.912+0000 7f80fac24740 1 --2- 172.21.15.2:0/141671709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945dbbd0 0x55c2945e15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.912+0000 7f80fac24740 1 --2- 172.21.15.2:0/141671709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945db2b0 0x55c2945e41c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.912+0000 7f80d57fa640 0 client.26383 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.912+0000 7f80fac24740 1 -- 172.21.15.2:0/141671709 >> 172.21.15.2:0/141671709 conn(0x55c2945d5990 msgr2=0x55c2945d7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.912+0000 7f80fac24740 1 -- 172.21.15.2:0/141671709 shutdown_connections 2024-03-15T13:01:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.912+0000 7f80fac24740 1 -- 172.21.15.2:0/141671709 wait complete. 2024-03-15T13:01:59.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.912+0000 7f80fac24740 20 client.26383 trim_cache size 0 max 16384 2024-03-15T13:01:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.934+0000 7f80fac24740 1 Processor -- start 2024-03-15T13:01:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.934+0000 7f80fac24740 1 -- start start 2024-03-15T13:01:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.934+0000 7f80fac24740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945db2b0 0x55c2945e3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.934+0000 7f80fac24740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945dbbd0 0x55c2945efb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.934+0000 7f80fac24740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945e47b0 0x55c2945f0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.934+0000 7f80fac24740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2945da5d0 con 0x55c2945e47b0 2024-03-15T13:01:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.934+0000 7f80fac24740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2945dfb10 con 0x55c2945db2b0 2024-03-15T13:01:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.934+0000 7f80fac24740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2945d6560 con 0x55c2945dbbd0 2024-03-15T13:01:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.935+0000 7f80f5d08640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945db2b0 0x55c2945e3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.935+0000 7f80f5d08640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945db2b0 0x55c2945e3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44354/0 (socket says 172.21.15.2:44354) 2024-03-15T13:01:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.935+0000 7f80f5d08640 1 -- 172.21.15.2:0/1602863495 learned_addr learned my addr 172.21.15.2:0/1602863495 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.935+0000 7f80f6509640 1 --2- 172.21.15.2:0/1602863495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945e47b0 0x55c2945f0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.935+0000 7f80f5507640 1 --2- 172.21.15.2:0/1602863495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945dbbd0 0x55c2945efb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.935+0000 7f80f5d08640 1 -- 172.21.15.2:0/1602863495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945dbbd0 msgr2=0x55c2945efb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.935+0000 7f80f5d08640 1 --2- 172.21.15.2:0/1602863495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945dbbd0 0x55c2945efb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.935+0000 7f80f5d08640 1 -- 172.21.15.2:0/1602863495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945e47b0 msgr2=0x55c2945f0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.935+0000 7f80f5d08640 1 --2- 172.21.15.2:0/1602863495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945e47b0 0x55c2945f0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.935+0000 7f80f5d08640 1 -- 172.21.15.2:0/1602863495 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2944ddac0 con 0x55c2945db2b0 2024-03-15T13:01:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.935+0000 7f80f5507640 1 --2- 172.21.15.2:0/1602863495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945dbbd0 0x55c2945efb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:01:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.935+0000 7f80f6509640 1 --2- 172.21.15.2:0/1602863495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945e47b0 0x55c2945f0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:01:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.936+0000 7f80f5d08640 1 --2- 172.21.15.2:0/1602863495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945db2b0 0x55c2945e3cf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f80e0005350 tx=0x7f80e0005380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e82c6ccfd8a2845 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:59.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.936+0000 7f80d77fe640 1 -- 172.21.15.2:0/1602863495 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0005470 con 0x55c2945db2b0 2024-03-15T13:01:59.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.936+0000 7f80d77fe640 1 -- 172.21.15.2:0/1602863495 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f80e0005690 con 0x55c2945db2b0 2024-03-15T13:01:59.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.936+0000 7f80d77fe640 1 -- 172.21.15.2:0/1602863495 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0004840 con 0x55c2945db2b0 2024-03-15T13:01:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.936+0000 7f80fac24740 1 -- 172.21.15.2:0/1602863495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945db2b0 msgr2=0x55c2945e3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 --2- 172.21.15.2:0/1602863495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945db2b0 0x55c2945e3cf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f80e0005350 tx=0x7f80e0005380 comp rx=0 tx=0).stop 2024-03-15T13:01:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 -- 172.21.15.2:0/1602863495 shutdown_connections 2024-03-15T13:01:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 --2- 172.21.15.2:0/1602863495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945e47b0 0x55c2945f0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 --2- 172.21.15.2:0/1602863495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945dbbd0 0x55c2945efb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 --2- 172.21.15.2:0/1602863495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945db2b0 0x55c2945e3cf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 -- 172.21.15.2:0/1602863495 >> 172.21.15.2:0/1602863495 conn(0x55c2945d5990 msgr2=0x55c2945d5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 -- 172.21.15.2:0/1602863495 shutdown_connections 2024-03-15T13:01:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 -- 172.21.15.2:0/1602863495 wait complete. 2024-03-15T13:01:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 -- start start 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 Processor -- start 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945db2b0 0x55c2945e55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945dbbd0 0x55c2945e5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945e47b0 0x55c2945ea660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2945e6230 con 0x55c2945e47b0 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2945eaba0 con 0x55c2945db2b0 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2945ead60 con 0x55c2945dbbd0 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.937+0000 7f80fac24740 10 client.0 _ll_register_callbacks cb 0x7f80e0009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.938+0000 7f80f6509640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945e47b0 0x55c2945ea660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.938+0000 7f80f5507640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945dbbd0 0x55c2945e5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.938+0000 7f80f6509640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945e47b0 0x55c2945ea660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33888/0 (socket says 172.21.15.2:33888) 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.938+0000 7f80f6509640 1 -- 172.21.15.2:0/2282606304 learned_addr learned my addr 172.21.15.2:0/2282606304 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.938+0000 7f80f5d08640 1 --2- 172.21.15.2:0/2282606304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945db2b0 0x55c2945e55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.938+0000 7f80f6509640 1 -- 172.21.15.2:0/2282606304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945dbbd0 msgr2=0x55c2945e5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.938+0000 7f80f6509640 1 --2- 172.21.15.2:0/2282606304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945dbbd0 0x55c2945e5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.938+0000 7f80f6509640 1 -- 172.21.15.2:0/2282606304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945db2b0 msgr2=0x55c2945e55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:01:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.938+0000 7f80f6509640 1 --2- 172.21.15.2:0/2282606304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945db2b0 0x55c2945e55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.938+0000 7f80f6509640 1 -- 172.21.15.2:0/2282606304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e8049050 con 0x55c2945e47b0 2024-03-15T13:01:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.938+0000 7f80f5d08640 1 --2- 172.21.15.2:0/2282606304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945db2b0 0x55c2945e55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:01:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.938+0000 7f80f6509640 1 --2- 172.21.15.2:0/2282606304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945e47b0 0x55c2945ea660 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f80e802c170 tx=0x7f80e802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b76f49c4fc8583 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:01:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.938+0000 7f80d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:01:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.938+0000 7f80d57fa640 1 -- 172.21.15.2:0/2282606304 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e8005210 con 0x55c2945e47b0 2024-03-15T13:01:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.938+0000 7f80d57fa640 1 -- 172.21.15.2:0/2282606304 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f80e8005370 con 0x55c2945e47b0 2024-03-15T13:01:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.939+0000 7f80fac24740 1 -- 172.21.15.2:0/2282606304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80e0009e40 con 0x55c2945e47b0 2024-03-15T13:01:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.939+0000 7f80d57fa640 1 -- 172.21.15.2:0/2282606304 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80e8005530 con 0x55c2945e47b0 2024-03-15T13:01:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.939+0000 7f80fac24740 10 client.16593 fetch_fsmap learned FSMap version 36 2024-03-15T13:01:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.939+0000 7f80fac24740 1 -- 172.21.15.2:0/2282606304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c294541610 con 0x55c2945e47b0 2024-03-15T13:01:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.939+0000 7f80d57fa640 1 -- 172.21.15.2:0/2282606304 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f80e805f080 con 0x55c2945e47b0 2024-03-15T13:01:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.939+0000 7f80d57fa640 1 -- 172.21.15.2:0/2282606304 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) v1 ==== 7982+0+0 (secure 0 0 0) 0x7f80e8055020 con 0x55c2945e47b0 2024-03-15T13:01:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.939+0000 7f80fac24740 10 client.16593 fetch_fsmap finished waiting for FSMap version 36 2024-03-15T13:01:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.939+0000 7f80fac24740 10 client.16593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:01:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.939+0000 7f80fac24740 20 client.16593 populate_metadata read hostname 'smithi002' 2024-03-15T13:01:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.939+0000 7f80fac24740 1 --2- 172.21.15.2:0/2282606304 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2946511a0 0x55c2946535c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:01:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.939+0000 7f80fac24740 4 client.16593 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:01:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.939+0000 7f80fac24740 1 -- 172.21.15.2:0/2282606304 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c2942ce890 con 0x55c2946511a0 2024-03-15T13:01:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.939+0000 7f80f5d08640 1 --2- 172.21.15.2:0/2282606304 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2946511a0 0x55c2946535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:01:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.940+0000 7f80f5d08640 1 --2- 172.21.15.2:0/2282606304 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2946511a0 0x55c2946535c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9a566e62d7b3248 server_cookie=ee71ed753975498a in_seq=0 out_seq=0 2024-03-15T13:01:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.940+0000 7f80d57fa640 10 client.16593 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.940+0000 7f80d57fa640 1 -- 172.21.15.2:0/2282606304 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c2942ce890 con 0x55c2946511a0 2024-03-15T13:01:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.940+0000 7f80d57fa640 10 client.16593 handle_command_reply: tid=0 2024-03-15T13:01:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.940+0000 7f80fac24740 1 client.16593 shutdown 2024-03-15T13:01:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.941+0000 7f80fac24740 1 -- 172.21.15.2:0/2282606304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945e47b0 msgr2=0x55c2945ea660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.941+0000 7f80fac24740 1 --2- 172.21.15.2:0/2282606304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945e47b0 0x55c2945ea660 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f80e802c170 tx=0x7f80e802bc30 comp rx=0 tx=0).stop 2024-03-15T13:01:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.941+0000 7f80fac24740 1 -- 172.21.15.2:0/2282606304 shutdown_connections 2024-03-15T13:01:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.941+0000 7f80fac24740 1 --2- 172.21.15.2:0/2282606304 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2946511a0 0x55c2946535c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.941+0000 7f80fac24740 1 --2- 172.21.15.2:0/2282606304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2945e47b0 0x55c2945ea660 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.941+0000 7f80fac24740 1 --2- 172.21.15.2:0/2282606304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2945dbbd0 0x55c2945e5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.941+0000 7f80fac24740 1 --2- 172.21.15.2:0/2282606304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2945db2b0 0x55c2945e55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.941+0000 7f80d57fa640 0 client.16593 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:01:59.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.941+0000 7f80fac24740 1 -- 172.21.15.2:0/2282606304 >> 172.21.15.2:0/2282606304 conn(0x55c2945d5990 msgr2=0x55c2945dfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:59.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.941+0000 7f80fac24740 1 -- 172.21.15.2:0/2282606304 shutdown_connections 2024-03-15T13:01:59.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.941+0000 7f80fac24740 1 -- 172.21.15.2:0/2282606304 wait complete. 2024-03-15T13:01:59.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.941+0000 7f80fac24740 20 client.16593 trim_cache size 0 max 16384 2024-03-15T13:01:59.931 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:01:59.931 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:01:59.932 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:01:59.932 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:01:59.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.942+0000 7f80f7f6c640 1 -- 172.21.15.2:0/2550479433 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f80d80a1c20 msgr2=0x7f80d80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.942+0000 7f80f7f6c640 1 --2- 172.21.15.2:0/2550479433 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f80d80a1c20 0x7f80d80a40e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f80e8006010 tx=0x7f80e8004f80 comp rx=0 tx=0).stop 2024-03-15T13:01:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.942+0000 7f80f7f6c640 1 -- 172.21.15.2:0/2550479433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80f01056b0 msgr2=0x7f80f019fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:01:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.942+0000 7f80f7f6c640 1 --2- 172.21.15.2:0/2550479433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80f01056b0 0x7f80f019fdd0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f80e400efd0 tx=0x7f80e400c6a0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.942+0000 7f80f7f6c640 1 -- 172.21.15.2:0/2550479433 shutdown_connections 2024-03-15T13:01:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.942+0000 7f80f7f6c640 1 --2- 172.21.15.2:0/2550479433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80f0199a40 0x7f80f0199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.942+0000 7f80f7f6c640 1 --2- 172.21.15.2:0/2550479433 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f80d80a1c20 0x7f80d80a40e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.942+0000 7f80f7f6c640 1 --2- 172.21.15.2:0/2550479433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80f01056b0 0x7f80f019fdd0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.942+0000 7f80f7f6c640 1 --2- 172.21.15.2:0/2550479433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80f01005e0 0x7f80f019f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:01:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.942+0000 7f80f7f6c640 1 -- 172.21.15.2:0/2550479433 >> 172.21.15.2:0/2550479433 conn(0x7f80f00775a0 msgr2=0x7f80f0103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:01:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.942+0000 7f80f7f6c640 1 -- 172.21.15.2:0/2550479433 shutdown_connections 2024-03-15T13:01:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:01:59.942+0000 7f80f7f6c640 1 -- 172.21.15.2:0/2550479433 wait complete. 2024-03-15T13:01:59.939 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:01:59.939 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:02:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.080+0000 7f546ce81640 1 -- 172.21.15.2:0/1966067366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f546810bb30 msgr2=0x7f546810dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.080+0000 7f546ce81640 1 --2- 172.21.15.2:0/1966067366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f546810bb30 0x7f546810dfd0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5460009ca0 tx=0x7f546002c0f0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.080+0000 7f546ce81640 1 -- 172.21.15.2:0/1966067366 shutdown_connections 2024-03-15T13:02:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.080+0000 7f546ce81640 1 --2- 172.21.15.2:0/1966067366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f546810bb30 0x7f546810dfd0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.080+0000 7f546ce81640 1 --2- 172.21.15.2:0/1966067366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5468103080 0x7f546810b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.080+0000 7f546ce81640 1 --2- 172.21.15.2:0/1966067366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5468102760 0x7f5468102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.080+0000 7f546ce81640 1 -- 172.21.15.2:0/1966067366 >> 172.21.15.2:0/1966067366 conn(0x7f54680774f0 msgr2=0x7f5468077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.080+0000 7f546ce81640 1 -- 172.21.15.2:0/1966067366 shutdown_connections 2024-03-15T13:02:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.080+0000 7f546ce81640 1 -- 172.21.15.2:0/1966067366 wait complete. 2024-03-15T13:02:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.081+0000 7f546ce81640 1 Processor -- start 2024-03-15T13:02:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.081+0000 7f546ce81640 1 -- start start 2024-03-15T13:02:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.081+0000 7f546ce81640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5468102760 0x7f546819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.081+0000 7f546ce81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5468103080 0x7f546819dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.081+0000 7f546ce81640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f546810bb30 0x7f5468197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.081+0000 7f546ce81640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5468197fb0 con 0x7f5468103080 2024-03-15T13:02:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.081+0000 7f546ce81640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5468198120 con 0x7f5468102760 2024-03-15T13:02:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.081+0000 7f546ce81640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54681982e0 con 0x7f546810bb30 2024-03-15T13:02:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.081+0000 7f5465d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5468103080 0x7f546819dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.081+0000 7f5465d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5468103080 0x7f546819dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33902/0 (socket says 172.21.15.2:33902) 2024-03-15T13:02:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.081+0000 7f5465d9b640 1 -- 172.21.15.2:0/2216975526 learned_addr learned my addr 172.21.15.2:0/2216975526 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.081+0000 7f546659c640 1 --2- 172.21.15.2:0/2216975526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5468102760 0x7f546819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.081+0000 7f5466d9d640 1 --2- 172.21.15.2:0/2216975526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f546810bb30 0x7f5468197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.082+0000 7f5466d9d640 1 -- 172.21.15.2:0/2216975526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5468102760 msgr2=0x7f546819d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.082+0000 7f5466d9d640 1 --2- 172.21.15.2:0/2216975526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5468102760 0x7f546819d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.082+0000 7f5466d9d640 1 -- 172.21.15.2:0/2216975526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5468103080 msgr2=0x7f546819dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.082+0000 7f5466d9d640 1 --2- 172.21.15.2:0/2216975526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5468103080 0x7f546819dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.082+0000 7f5466d9d640 1 -- 172.21.15.2:0/2216975526 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5460009900 con 0x7f546810bb30 2024-03-15T13:02:00.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.082+0000 7f5465d9b640 1 --2- 172.21.15.2:0/2216975526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5468103080 0x7f546819dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:00.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.082+0000 7f5466d9d640 1 --2- 172.21.15.2:0/2216975526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f546810bb30 0x7f5468197960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5460009bf0 tx=0x7f54600325e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.082+0000 7f54577fe640 1 -- 172.21.15.2:0/2216975526 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460032880 con 0x7f546810bb30 2024-03-15T13:02:00.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.082+0000 7f546ce81640 1 -- 172.21.15.2:0/2216975526 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5468198560 con 0x7f546810bb30 2024-03-15T13:02:00.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.082+0000 7f54577fe640 1 -- 172.21.15.2:0/2216975526 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5460003e40 con 0x7f546810bb30 2024-03-15T13:02:00.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.082+0000 7f54577fe640 1 -- 172.21.15.2:0/2216975526 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460004720 con 0x7f546810bb30 2024-03-15T13:02:00.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.082+0000 7f546ce81640 1 -- 172.21.15.2:0/2216975526 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54681a4850 con 0x7f546810bb30 2024-03-15T13:02:00.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.083+0000 7f546ce81640 1 -- 172.21.15.2:0/2216975526 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54680682e0 con 0x7f546810bb30 2024-03-15T13:02:00.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.084+0000 7f54577fe640 1 -- 172.21.15.2:0/2216975526 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f546000a040 con 0x7f546810bb30 2024-03-15T13:02:00.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.085+0000 7f54577fe640 1 --2- 172.21.15.2:0/2216975526 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f54500a1ba0 0x7f54500a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.085+0000 7f546659c640 1 --2- 172.21.15.2:0/2216975526 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f54500a1ba0 0x7f54500a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.085+0000 7f54577fe640 1 -- 172.21.15.2:0/2216975526 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f54600f0550 con 0x7f546810bb30 2024-03-15T13:02:00.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.086+0000 7f546659c640 1 --2- 172.21.15.2:0/2216975526 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f54500a1ba0 0x7f54500a4060 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5468198aa0 tx=0x7f5458005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.090+0000 7f54577fe640 1 -- 172.21.15.2:0/2216975526 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f54600b91e0 con 0x7f546810bb30 2024-03-15T13:02:00.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.378+0000 7f546ce81640 1 -- 172.21.15.2:0/2216975526 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5468111680 con 0x7f546810bb30 2024-03-15T13:02:00.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.379+0000 7f54577fe640 1 -- 172.21.15.2:0/2216975526 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+11468 (secure 0 0 0) 0x7f54600bc8a0 con 0x7f546810bb30 2024-03-15T13:02:00.365 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:02:00.365 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:01:29.310243+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24464,24500,24460,24539,24434]},"id":1}]} 2024-03-15T13:02:00.365 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 36 2024-03-15T13:02:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.381+0000 7f546ce81640 1 -- 172.21.15.2:0/2216975526 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f54500a1ba0 msgr2=0x7f54500a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.381+0000 7f546ce81640 1 --2- 172.21.15.2:0/2216975526 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f54500a1ba0 0x7f54500a4060 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5468198aa0 tx=0x7f5458005f90 comp rx=0 tx=0).stop 2024-03-15T13:02:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.381+0000 7f546ce81640 1 -- 172.21.15.2:0/2216975526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f546810bb30 msgr2=0x7f5468197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.381+0000 7f546ce81640 1 --2- 172.21.15.2:0/2216975526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f546810bb30 0x7f5468197960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5460009bf0 tx=0x7f54600325e0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.381+0000 7f546ce81640 1 -- 172.21.15.2:0/2216975526 shutdown_connections 2024-03-15T13:02:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.381+0000 7f546ce81640 1 --2- 172.21.15.2:0/2216975526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f546810bb30 0x7f5468197960 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.381+0000 7f546ce81640 1 --2- 172.21.15.2:0/2216975526 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f54500a1ba0 0x7f54500a4060 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.381+0000 7f546ce81640 1 --2- 172.21.15.2:0/2216975526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5468103080 0x7f546819dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.381+0000 7f546ce81640 1 --2- 172.21.15.2:0/2216975526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5468102760 0x7f546819d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.381+0000 7f546ce81640 1 -- 172.21.15.2:0/2216975526 >> 172.21.15.2:0/2216975526 conn(0x7f54680774f0 msgr2=0x7f54680fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.381+0000 7f546ce81640 1 -- 172.21.15.2:0/2216975526 shutdown_connections 2024-03-15T13:02:00.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.382+0000 7f546ce81640 1 -- 172.21.15.2:0/2216975526 wait complete. 2024-03-15T13:02:00.379 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:02:00.379 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:02:00.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.521+0000 7ff8c9a8f640 1 -- 172.21.15.2:0/617721235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8c40686d0 msgr2=0x7ff8c4068ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.521+0000 7ff8c9a8f640 1 --2- 172.21.15.2:0/617721235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8c40686d0 0x7ff8c4068ab0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009900 tx=0x7ff8b402bc30 comp rx=0 tx=0).stop 2024-03-15T13:02:00.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.521+0000 7ff8c9a8f640 1 -- 172.21.15.2:0/617721235 shutdown_connections 2024-03-15T13:02:00.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.521+0000 7ff8c9a8f640 1 --2- 172.21.15.2:0/617721235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8c410de90 0x7ff8c4110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.521+0000 7ff8c9a8f640 1 --2- 172.21.15.2:0/617721235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c4068ff0 0x7ff8c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.521+0000 7ff8c9a8f640 1 --2- 172.21.15.2:0/617721235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8c40686d0 0x7ff8c4068ab0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.521+0000 7ff8c9a8f640 1 -- 172.21.15.2:0/617721235 >> 172.21.15.2:0/617721235 conn(0x7ff8c40774f0 msgr2=0x7ff8c4077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.521+0000 7ff8c9a8f640 1 -- 172.21.15.2:0/617721235 shutdown_connections 2024-03-15T13:02:00.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.521+0000 7ff8c9a8f640 1 -- 172.21.15.2:0/617721235 wait complete. 2024-03-15T13:02:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.522+0000 7ff8c9a8f640 1 Processor -- start 2024-03-15T13:02:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.522+0000 7ff8c9a8f640 1 -- start start 2024-03-15T13:02:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.522+0000 7ff8c9a8f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8c40686d0 0x7ff8c419fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.522+0000 7ff8c9a8f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c4068ff0 0x7ff8c419ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.522+0000 7ff8c9a8f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8c410de90 0x7ff8c4199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.522+0000 7ff8c9a8f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c419a2b0 con 0x7ff8c410de90 2024-03-15T13:02:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.522+0000 7ff8c9a8f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c419a420 con 0x7ff8c40686d0 2024-03-15T13:02:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.522+0000 7ff8c9a8f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c419a5e0 con 0x7ff8c4068ff0 2024-03-15T13:02:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.523+0000 7ff8c37fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8c40686d0 0x7ff8c419fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.523+0000 7ff8c2ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c4068ff0 0x7ff8c419ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.523+0000 7ff8c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8c410de90 0x7ff8c4199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.523+0000 7ff8c37fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8c40686d0 0x7ff8c419fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44394/0 (socket says 172.21.15.2:44394) 2024-03-15T13:02:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.523+0000 7ff8c37fe640 1 -- 172.21.15.2:0/2717118511 learned_addr learned my addr 172.21.15.2:0/2717118511 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.523+0000 7ff8c2ffd640 1 -- 172.21.15.2:0/2717118511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8c40686d0 msgr2=0x7ff8c419fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.523+0000 7ff8c2ffd640 1 --2- 172.21.15.2:0/2717118511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8c40686d0 0x7ff8c419fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.523+0000 7ff8c2ffd640 1 -- 172.21.15.2:0/2717118511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8c410de90 msgr2=0x7ff8c4199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.523+0000 7ff8c2ffd640 1 --2- 172.21.15.2:0/2717118511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8c410de90 0x7ff8c4199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.523+0000 7ff8c2ffd640 1 -- 172.21.15.2:0/2717118511 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b8009490 con 0x7ff8c4068ff0 2024-03-15T13:02:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.523+0000 7ff8c2ffd640 1 --2- 172.21.15.2:0/2717118511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c4068ff0 0x7ff8c419ffa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800e9f0 tx=0x7ff8b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.523+0000 7ff8c0ff9640 1 -- 172.21.15.2:0/2717118511 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b800cd80 con 0x7ff8c4068ff0 2024-03-15T13:02:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.523+0000 7ff8c9a8f640 1 -- 172.21.15.2:0/2717118511 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b4009560 con 0x7ff8c4068ff0 2024-03-15T13:02:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.523+0000 7ff8c0ff9640 1 -- 172.21.15.2:0/2717118511 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff8b80035f0 con 0x7ff8c4068ff0 2024-03-15T13:02:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.523+0000 7ff8c0ff9640 1 -- 172.21.15.2:0/2717118511 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8010690 con 0x7ff8c4068ff0 2024-03-15T13:02:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.524+0000 7ff8c9a8f640 1 -- 172.21.15.2:0/2717118511 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c41a69e0 con 0x7ff8c4068ff0 2024-03-15T13:02:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.526+0000 7ff8c0ff9640 1 -- 172.21.15.2:0/2717118511 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7ff8b8003760 con 0x7ff8c4068ff0 2024-03-15T13:02:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8cc673740 1 Processor -- start 2024-03-15T13:02:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8cc673740 1 -- start start 2024-03-15T13:02:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8cc673740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205250 0x55c8b6205630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8cc673740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205b70 0x55c8b620e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8cc673740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b620e750 0x55c8b6210b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8cc673740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b61c0960 con 0x55c8b6205b70 2024-03-15T13:02:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8cc673740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b6174310 con 0x55c8b6205250 2024-03-15T13:02:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8cc673740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b6171840 con 0x55c8b620e750 2024-03-15T13:02:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8c0ff9640 1 --2- 172.21.15.2:0/2717118511 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff8ac0a1ba0 0x7ff8ac0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8c37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205b70 0x55c8b620e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8c37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205b70 0x55c8b620e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33928/0 (socket says 172.21.15.2:33928) 2024-03-15T13:02:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8c37fe640 1 -- 172.21.15.2:0/200202598 learned_addr learned my addr 172.21.15.2:0/200202598 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8c2ffd640 1 --2- 172.21.15.2:0/200202598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b620e750 0x55c8b6210b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8c3fff640 1 --2- 172.21.15.2:0/200202598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205250 0x55c8b6205630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8c3fff640 1 --2- 172.21.15.2:0/2717118511 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff8ac0a1ba0 0x7ff8ac0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8c0ff9640 1 -- 172.21.15.2:0/2717118511 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7ff8b8014070 con 0x7ff8c4068ff0 2024-03-15T13:02:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8c37fe640 1 -- 172.21.15.2:0/200202598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b620e750 msgr2=0x55c8b6210b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8c37fe640 1 --2- 172.21.15.2:0/200202598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b620e750 0x55c8b6210b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8c37fe640 1 -- 172.21.15.2:0/200202598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205250 msgr2=0x55c8b6205630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8c37fe640 1 --2- 172.21.15.2:0/200202598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205250 0x55c8b6205630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8c37fe640 1 -- 172.21.15.2:0/200202598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8b612f060 con 0x55c8b6205b70 2024-03-15T13:02:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.527+0000 7ff8c2ffd640 1 --2- 172.21.15.2:0/200202598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b620e750 0x55c8b6210b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:02:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.528+0000 7ff8c37fe640 1 --2- 172.21.15.2:0/200202598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205b70 0x55c8b620e210 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009430 tx=0x7ff8b402c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41e881330781616 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.528+0000 7ff8c3fff640 1 --2- 172.21.15.2:0/2717118511 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff8ac0a1ba0 0x7ff8ac0a4060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff8c406aa20 tx=0x7ff8bc009500 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.528+0000 7ff8a9ffb640 1 -- 172.21.15.2:0/200202598 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4004800 con 0x55c8b6205b70 2024-03-15T13:02:00.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.528+0000 7ff8a9ffb640 1 -- 172.21.15.2:0/200202598 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff8b4004960 con 0x55c8b6205b70 2024-03-15T13:02:00.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.528+0000 7ff8a9ffb640 1 -- 172.21.15.2:0/200202598 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4003850 con 0x55c8b6205b70 2024-03-15T13:02:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.529+0000 7ff8cc673740 1 -- 172.21.15.2:0/200202598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205b70 msgr2=0x55c8b620e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.529+0000 7ff8cc673740 1 --2- 172.21.15.2:0/200202598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205b70 0x55c8b620e210 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009430 tx=0x7ff8b402c660 comp rx=0 tx=0).stop 2024-03-15T13:02:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.529+0000 7ff8cc673740 1 -- 172.21.15.2:0/200202598 shutdown_connections 2024-03-15T13:02:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.529+0000 7ff8cc673740 1 --2- 172.21.15.2:0/200202598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b620e750 0x55c8b6210b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.529+0000 7ff8cc673740 1 --2- 172.21.15.2:0/200202598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205b70 0x55c8b620e210 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.529+0000 7ff8cc673740 1 --2- 172.21.15.2:0/200202598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205250 0x55c8b6205630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.529+0000 7ff8cc673740 1 -- 172.21.15.2:0/200202598 >> 172.21.15.2:0/200202598 conn(0x55c8b61ff980 msgr2=0x55c8b61ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.529+0000 7ff8cc673740 1 -- 172.21.15.2:0/200202598 shutdown_connections 2024-03-15T13:02:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.529+0000 7ff8cc673740 1 -- 172.21.15.2:0/200202598 wait complete. 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.529+0000 7ff8cc673740 1 -- start start 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.530+0000 7ff8cc673740 1 Processor -- start 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.530+0000 7ff8cc673740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b6205250 0x55c8b620e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.530+0000 7ff8cc673740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205b70 0x55c8b620b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.530+0000 7ff8cc673740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b620e750 0x55c8b620b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.530+0000 7ff8cc673740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b616d020 con 0x55c8b620e750 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.530+0000 7ff8cc673740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b5e40430 con 0x55c8b6205b70 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.530+0000 7ff8cc673740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b5dfce00 con 0x55c8b6205250 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.530+0000 7ff8c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b620e750 0x55c8b620b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.530+0000 7ff8cc673740 10 client.0 _ll_register_callbacks cb 0x55c8b6114cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.530+0000 7ff8c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b620e750 0x55c8b620b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33938/0 (socket says 172.21.15.2:33938) 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.530+0000 7ff8c2ffd640 1 --2- 172.21.15.2:0/1692316135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205b70 0x55c8b620b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.530+0000 7ff8c3fff640 1 -- 172.21.15.2:0/1692316135 learned_addr learned my addr 172.21.15.2:0/1692316135 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.530+0000 7ff8c37fe640 1 --2- 172.21.15.2:0/1692316135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b6205250 0x55c8b620e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.531+0000 7ff8c3fff640 1 -- 172.21.15.2:0/1692316135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b6205250 msgr2=0x55c8b620e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.531+0000 7ff8c3fff640 1 --2- 172.21.15.2:0/1692316135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b6205250 0x55c8b620e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.531+0000 7ff8c3fff640 1 -- 172.21.15.2:0/1692316135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205b70 msgr2=0x55c8b620b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.531+0000 7ff8c3fff640 1 --2- 172.21.15.2:0/1692316135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205b70 0x55c8b620b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.531+0000 7ff8c3fff640 1 -- 172.21.15.2:0/1692316135 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c4110d70 con 0x55c8b620e750 2024-03-15T13:02:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.531+0000 7ff8c2ffd640 1 --2- 172.21.15.2:0/1692316135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205b70 0x55c8b620b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.531+0000 7ff8c3fff640 1 --2- 172.21.15.2:0/1692316135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b620e750 0x55c8b620b9e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc010d80 tx=0x7ff8bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9530106eb88a27e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.531+0000 7ff8a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.531+0000 7ff8a2ffd640 1 -- 172.21.15.2:0/1692316135 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc00ed70 con 0x55c8b620e750 2024-03-15T13:02:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.531+0000 7ff8a2ffd640 1 -- 172.21.15.2:0/1692316135 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7ff8bc01e410 con 0x55c8b620e750 2024-03-15T13:02:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.531+0000 7ff8cc673740 1 -- 172.21.15.2:0/1692316135 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8b5e6c7e0 con 0x55c8b620e750 2024-03-15T13:02:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.532+0000 7ff8a2ffd640 1 -- 172.21.15.2:0/1692316135 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8bc01e5d0 con 0x55c8b620e750 2024-03-15T13:02:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.532+0000 7ff8cc673740 10 client.16611 fetch_fsmap learned FSMap version 36 2024-03-15T13:02:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.532+0000 7ff8cc673740 1 -- 172.21.15.2:0/1692316135 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c8b616b6a0 con 0x55c8b620e750 2024-03-15T13:02:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.532+0000 7ff8a2ffd640 1 -- 172.21.15.2:0/1692316135 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55c8b5e6c7e0 con 0x55c8b620e750 2024-03-15T13:02:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.532+0000 7ff8a2ffd640 1 -- 172.21.15.2:0/1692316135 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) v1 ==== 7982+0+0 (secure 0 0 0) 0x7ff8bc028810 con 0x55c8b620e750 2024-03-15T13:02:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.532+0000 7ff8cc673740 10 client.16611 fetch_fsmap finished waiting for FSMap version 36 2024-03-15T13:02:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.532+0000 7ff8cc673740 10 client.16611 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:02:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.532+0000 7ff8cc673740 20 client.16611 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.532+0000 7ff8cc673740 1 --2- 172.21.15.2:0/1692316135 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c8b621e8b0 0x55c8b6220cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.532+0000 7ff8cc673740 4 client.16611 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:02:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.532+0000 7ff8cc673740 1 -- 172.21.15.2:0/1692316135 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8b5ea9720 con 0x55c8b621e8b0 2024-03-15T13:02:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.532+0000 7ff8c37fe640 1 --2- 172.21.15.2:0/1692316135 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c8b621e8b0 0x55c8b6220cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.533+0000 7ff8c37fe640 1 --2- 172.21.15.2:0/1692316135 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c8b621e8b0 0x55c8b6220cd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e5602ae983c5a16a server_cookie=294f6fde9735df7d in_seq=0 out_seq=0 2024-03-15T13:02:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.533+0000 7ff8a2ffd640 10 client.16611 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:02:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.534+0000 7ff8a2ffd640 1 -- 172.21.15.2:0/1692316135 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c8b5ea9720 con 0x55c8b621e8b0 2024-03-15T13:02:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.534+0000 7ff8a2ffd640 10 client.16611 handle_command_reply: tid=0 2024-03-15T13:02:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.534+0000 7ff8cc673740 1 client.16611 shutdown 2024-03-15T13:02:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.534+0000 7ff8cc673740 1 -- 172.21.15.2:0/1692316135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b620e750 msgr2=0x55c8b620b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.534+0000 7ff8cc673740 1 --2- 172.21.15.2:0/1692316135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b620e750 0x55c8b620b9e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc010d80 tx=0x7ff8bc004a40 comp rx=0 tx=0).stop 2024-03-15T13:02:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.534+0000 7ff8cc673740 1 -- 172.21.15.2:0/1692316135 shutdown_connections 2024-03-15T13:02:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.534+0000 7ff8cc673740 1 --2- 172.21.15.2:0/1692316135 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c8b621e8b0 0x55c8b6220cd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.534+0000 7ff8cc673740 1 --2- 172.21.15.2:0/1692316135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b620e750 0x55c8b620b9e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.534+0000 7ff8cc673740 1 --2- 172.21.15.2:0/1692316135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205b70 0x55c8b620b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.534+0000 7ff8cc673740 1 --2- 172.21.15.2:0/1692316135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b6205250 0x55c8b620e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.534+0000 7ff8cc673740 1 -- 172.21.15.2:0/1692316135 >> 172.21.15.2:0/1692316135 conn(0x55c8b61ff980 msgr2=0x55c8b62016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.534+0000 7ff8a2ffd640 0 client.16611 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:02:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.534+0000 7ff8cc673740 1 -- 172.21.15.2:0/1692316135 shutdown_connections 2024-03-15T13:02:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.535+0000 7ff8cc673740 1 -- 172.21.15.2:0/1692316135 wait complete. 2024-03-15T13:02:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.535+0000 7ff8cc673740 20 client.16611 trim_cache size 0 max 16384 2024-03-15T13:02:00.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.551+0000 7ff8cc673740 1 Processor -- start 2024-03-15T13:02:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.551+0000 7ff8cc673740 1 -- start start 2024-03-15T13:02:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.551+0000 7ff8cc673740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205250 0x55c8b620db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.551+0000 7ff8cc673740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205b70 0x55c8b620e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.551+0000 7ff8cc673740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b620e750 0x55c8b6219a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.551+0000 7ff8cc673740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b60f2130 con 0x55c8b6205b70 2024-03-15T13:02:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.551+0000 7ff8cc673740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b61ff4b0 con 0x55c8b6205250 2024-03-15T13:02:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.551+0000 7ff8cc673740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b61fb0e0 con 0x55c8b620e750 2024-03-15T13:02:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.551+0000 7ff8c2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205b70 0x55c8b620e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.552+0000 7ff8c2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205b70 0x55c8b620e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33952/0 (socket says 172.21.15.2:33952) 2024-03-15T13:02:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.552+0000 7ff8c2ffd640 1 -- 172.21.15.2:0/4003025139 learned_addr learned my addr 172.21.15.2:0/4003025139 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.552+0000 7ff8c37fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205250 0x55c8b620db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.552+0000 7ff8c3fff640 1 --2- 172.21.15.2:0/4003025139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b620e750 0x55c8b6219a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.552+0000 7ff8c2ffd640 1 -- 172.21.15.2:0/4003025139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b620e750 msgr2=0x55c8b6219a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.552+0000 7ff8c2ffd640 1 --2- 172.21.15.2:0/4003025139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b620e750 0x55c8b6219a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.552+0000 7ff8c2ffd640 1 -- 172.21.15.2:0/4003025139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205250 msgr2=0x55c8b620db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.552+0000 7ff8c2ffd640 1 --2- 172.21.15.2:0/4003025139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205250 0x55c8b620db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.552+0000 7ff8c2ffd640 1 -- 172.21.15.2:0/4003025139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8b6107c20 con 0x55c8b6205b70 2024-03-15T13:02:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.552+0000 7ff8c2ffd640 1 --2- 172.21.15.2:0/4003025139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205b70 0x55c8b620e0d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80cb980 tx=0x7ff8b8096b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a376a3aaf61a512 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.552+0000 7ff8a97fa640 1 -- 172.21.15.2:0/4003025139 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b809a1c0 con 0x55c8b6205b70 2024-03-15T13:02:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.552+0000 7ff8a97fa640 1 -- 172.21.15.2:0/4003025139 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff8b8003b60 con 0x55c8b6205b70 2024-03-15T13:02:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.552+0000 7ff8a97fa640 1 -- 172.21.15.2:0/4003025139 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8096ed0 con 0x55c8b6205b70 2024-03-15T13:02:00.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.553+0000 7ff8cc673740 1 -- 172.21.15.2:0/4003025139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205b70 msgr2=0x55c8b620e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.553+0000 7ff8cc673740 1 --2- 172.21.15.2:0/4003025139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205b70 0x55c8b620e0d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80cb980 tx=0x7ff8b8096b60 comp rx=0 tx=0).stop 2024-03-15T13:02:00.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.553+0000 7ff8cc673740 1 -- 172.21.15.2:0/4003025139 shutdown_connections 2024-03-15T13:02:00.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.553+0000 7ff8cc673740 1 --2- 172.21.15.2:0/4003025139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b620e750 0x55c8b6219a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.553+0000 7ff8cc673740 1 --2- 172.21.15.2:0/4003025139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205b70 0x55c8b620e0d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.553+0000 7ff8cc673740 1 --2- 172.21.15.2:0/4003025139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b6205250 0x55c8b620db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.553+0000 7ff8cc673740 1 -- 172.21.15.2:0/4003025139 >> 172.21.15.2:0/4003025139 conn(0x55c8b61ff980 msgr2=0x55c8b6201640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.553+0000 7ff8cc673740 1 -- 172.21.15.2:0/4003025139 shutdown_connections 2024-03-15T13:02:00.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.553+0000 7ff8cc673740 1 -- 172.21.15.2:0/4003025139 wait complete. 2024-03-15T13:02:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.554+0000 7ff8cc673740 1 -- start start 2024-03-15T13:02:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.554+0000 7ff8cc673740 1 Processor -- start 2024-03-15T13:02:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.554+0000 7ff8cc673740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205250 0x55c8b6207de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.554+0000 7ff8cc673740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b6205b70 0x55c8b6208320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.554+0000 7ff8cc673740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b620e750 0x55c8b6279180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.554+0000 7ff8cc673740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b6208ab0 con 0x55c8b6205250 2024-03-15T13:02:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.554+0000 7ff8cc673740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b6208bf0 con 0x55c8b620e750 2024-03-15T13:02:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.554+0000 7ff8cc673740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b62796c0 con 0x55c8b6205b70 2024-03-15T13:02:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.554+0000 7ff8cc673740 10 client.0 _ll_register_callbacks cb 0x7ff8bc010a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.555+0000 7ff8c2ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b6205b70 0x55c8b6208320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.555+0000 7ff8c3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b620e750 0x55c8b6279180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.555+0000 7ff8c2ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b6205b70 0x55c8b6208320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:45916/0 (socket says 172.21.15.2:45916) 2024-03-15T13:02:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.555+0000 7ff8c3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b620e750 0x55c8b6279180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44424/0 (socket says 172.21.15.2:44424) 2024-03-15T13:02:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.555+0000 7ff8c2ffd640 1 -- 172.21.15.2:0/4003060043 learned_addr learned my addr 172.21.15.2:0/4003060043 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.555+0000 7ff8c37fe640 1 --2- 172.21.15.2:0/4003060043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205250 0x55c8b6207de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.555+0000 7ff8c3fff640 1 -- 172.21.15.2:0/4003060043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b6205b70 msgr2=0x55c8b6208320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.555+0000 7ff8c3fff640 1 --2- 172.21.15.2:0/4003060043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b6205b70 0x55c8b6208320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.555+0000 7ff8c3fff640 1 -- 172.21.15.2:0/4003060043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205250 msgr2=0x55c8b6207de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.555+0000 7ff8c3fff640 1 --2- 172.21.15.2:0/4003060043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205250 0x55c8b6207de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.555+0000 7ff8c3fff640 1 -- 172.21.15.2:0/4003060043 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c41a69e0 con 0x55c8b620e750 2024-03-15T13:02:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.555+0000 7ff8c37fe640 1 --2- 172.21.15.2:0/4003060043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205250 0x55c8b6207de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:02:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.555+0000 7ff8c3fff640 1 --2- 172.21.15.2:0/4003060043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b620e750 0x55c8b6279180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc028b70 tx=0x7ff8bc00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66f6023c26430b33 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.555+0000 7ff8a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:02:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.555+0000 7ff8a2ffd640 1 -- 172.21.15.2:0/4003060043 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc00f4e0 con 0x55c8b620e750 2024-03-15T13:02:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.556+0000 7ff8cc673740 1 -- 172.21.15.2:0/4003060043 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8b4004960 con 0x55c8b620e750 2024-03-15T13:02:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.556+0000 7ff8a2ffd640 1 -- 172.21.15.2:0/4003060043 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7ff8bc036030 con 0x55c8b620e750 2024-03-15T13:02:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.556+0000 7ff8a2ffd640 1 -- 172.21.15.2:0/4003060043 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8bc00e7b0 con 0x55c8b620e750 2024-03-15T13:02:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.556+0000 7ff8cc673740 10 client.26480 fetch_fsmap learned FSMap version 36 2024-03-15T13:02:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.556+0000 7ff8cc673740 1 -- 172.21.15.2:0/4003060043 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c8b612d8c0 con 0x55c8b620e750 2024-03-15T13:02:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.556+0000 7ff8a2ffd640 1 -- 172.21.15.2:0/4003060043 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff8b4004960 con 0x55c8b620e750 2024-03-15T13:02:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.556+0000 7ff8a2ffd640 1 -- 172.21.15.2:0/4003060043 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 36) v1 ==== 7982+0+0 (secure 0 0 0) 0x7ff8bc032540 con 0x55c8b620e750 2024-03-15T13:02:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.556+0000 7ff8cc673740 10 client.26480 fetch_fsmap finished waiting for FSMap version 36 2024-03-15T13:02:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.556+0000 7ff8cc673740 10 client.26480 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:02:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.556+0000 7ff8cc673740 20 client.26480 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.556+0000 7ff8cc673740 1 --2- 172.21.15.2:0/4003060043 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c8b627ae50 0x55c8b627d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.556+0000 7ff8cc673740 4 client.26480 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:02:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.556+0000 7ff8cc673740 1 -- 172.21.15.2:0/4003060043 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c8b5ef8890 con 0x55c8b627ae50 2024-03-15T13:02:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.557+0000 7ff8c37fe640 1 --2- 172.21.15.2:0/4003060043 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c8b627ae50 0x55c8b627d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.557+0000 7ff8a2ffd640 10 client.26480 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:02:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.557+0000 7ff8c37fe640 1 --2- 172.21.15.2:0/4003060043 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c8b627ae50 0x55c8b627d270 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ef8462a57ac2fc29 server_cookie=8b7b59b170901c6d in_seq=0 out_seq=0 2024-03-15T13:02:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.557+0000 7ff8a2ffd640 1 -- 172.21.15.2:0/4003060043 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c8b5ef8890 con 0x55c8b627ae50 2024-03-15T13:02:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.557+0000 7ff8a2ffd640 10 client.26480 handle_command_reply: tid=0 2024-03-15T13:02:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.557+0000 7ff8cc673740 1 client.26480 shutdown 2024-03-15T13:02:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.558+0000 7ff8cc673740 1 -- 172.21.15.2:0/4003060043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b620e750 msgr2=0x55c8b6279180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.558+0000 7ff8cc673740 1 --2- 172.21.15.2:0/4003060043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b620e750 0x55c8b6279180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc028b70 tx=0x7ff8bc00ef90 comp rx=0 tx=0).stop 2024-03-15T13:02:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.558+0000 7ff8cc673740 1 -- 172.21.15.2:0/4003060043 shutdown_connections 2024-03-15T13:02:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.558+0000 7ff8cc673740 1 --2- 172.21.15.2:0/4003060043 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c8b627ae50 0x55c8b627d270 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.558+0000 7ff8cc673740 1 --2- 172.21.15.2:0/4003060043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8b620e750 0x55c8b6279180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.558+0000 7ff8cc673740 1 --2- 172.21.15.2:0/4003060043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8b6205b70 0x55c8b6208320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.558+0000 7ff8cc673740 1 --2- 172.21.15.2:0/4003060043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c8b6205250 0x55c8b6207de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.558+0000 7ff8cc673740 1 -- 172.21.15.2:0/4003060043 >> 172.21.15.2:0/4003060043 conn(0x55c8b61ff980 msgr2=0x55c8b61fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.558+0000 7ff8a2ffd640 0 client.26480 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:02:00.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.558+0000 7ff8cc673740 1 -- 172.21.15.2:0/4003060043 shutdown_connections 2024-03-15T13:02:00.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.558+0000 7ff8cc673740 1 -- 172.21.15.2:0/4003060043 wait complete. 2024-03-15T13:02:00.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.558+0000 7ff8cc673740 20 client.26480 trim_cache size 0 max 16384 2024-03-15T13:02:00.548 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:02:00.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.559+0000 7ff8c9a8f640 1 -- 172.21.15.2:0/2717118511 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff8ac0a1ba0 msgr2=0x7ff8ac0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.559+0000 7ff8c9a8f640 1 --2- 172.21.15.2:0/2717118511 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff8ac0a1ba0 0x7ff8ac0a4060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff8c406aa20 tx=0x7ff8bc009500 comp rx=0 tx=0).stop 2024-03-15T13:02:00.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.559+0000 7ff8c9a8f640 1 -- 172.21.15.2:0/2717118511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c4068ff0 msgr2=0x7ff8c419ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.559+0000 7ff8c9a8f640 1 --2- 172.21.15.2:0/2717118511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c4068ff0 0x7ff8c419ffa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800e9f0 tx=0x7ff8b800eeb0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.559+0000 7ff8c9a8f640 1 -- 172.21.15.2:0/2717118511 shutdown_connections 2024-03-15T13:02:00.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.559+0000 7ff8c9a8f640 1 --2- 172.21.15.2:0/2717118511 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff8ac0a1ba0 0x7ff8ac0a4060 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.559+0000 7ff8c9a8f640 1 --2- 172.21.15.2:0/2717118511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8c410de90 0x7ff8c4199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.559+0000 7ff8c9a8f640 1 --2- 172.21.15.2:0/2717118511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c4068ff0 0x7ff8c419ffa0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.559+0000 7ff8c9a8f640 1 --2- 172.21.15.2:0/2717118511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8c40686d0 0x7ff8c419fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.559+0000 7ff8c9a8f640 1 -- 172.21.15.2:0/2717118511 >> 172.21.15.2:0/2717118511 conn(0x7ff8c40774f0 msgr2=0x7ff8c410f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.559+0000 7ff8c9a8f640 1 -- 172.21.15.2:0/2717118511 shutdown_connections 2024-03-15T13:02:00.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.559+0000 7ff8c9a8f640 1 -- 172.21.15.2:0/2717118511 wait complete. 2024-03-15T13:02:00.557 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:02:00.557 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:02:00.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.679+0000 7f6787db5640 1 -- 172.21.15.2:0/3966438195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6780107500 msgr2=0x7f67801078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.679+0000 7f6787db5640 1 --2- 172.21.15.2:0/3966438195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6780107500 0x7f67801078e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f677c009930 tx=0x7f677c02bc30 comp rx=0 tx=0).stop 2024-03-15T13:02:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.679+0000 7f6787db5640 1 -- 172.21.15.2:0/3966438195 shutdown_connections 2024-03-15T13:02:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.679+0000 7f6787db5640 1 --2- 172.21.15.2:0/3966438195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6780069090 0x7f678010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.679+0000 7f6787db5640 1 --2- 172.21.15.2:0/3966438195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67800686d0 0x7f6780068b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.679+0000 7f6787db5640 1 --2- 172.21.15.2:0/3966438195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6780107500 0x7f67801078e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.679+0000 7f6787db5640 1 -- 172.21.15.2:0/3966438195 >> 172.21.15.2:0/3966438195 conn(0x7f67800774f0 msgr2=0x7f6780077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.679+0000 7f6787db5640 1 -- 172.21.15.2:0/3966438195 shutdown_connections 2024-03-15T13:02:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.679+0000 7f6787db5640 1 -- 172.21.15.2:0/3966438195 wait complete. 2024-03-15T13:02:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.679+0000 7f6787db5640 1 Processor -- start 2024-03-15T13:02:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.679+0000 7f6787db5640 1 -- start start 2024-03-15T13:02:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6787db5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67800686d0 0x7f678019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6787db5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6780069090 0x7f678019dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6787db5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6780107500 0x7f67801979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6787db5640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6780198000 con 0x7f6780069090 2024-03-15T13:02:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6787db5640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6780198170 con 0x7f67800686d0 2024-03-15T13:02:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6787db5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6780198330 con 0x7f6780107500 2024-03-15T13:02:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6785350640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6780069090 0x7f678019dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6785350640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6780069090 0x7f678019dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33974/0 (socket says 172.21.15.2:33974) 2024-03-15T13:02:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6785350640 1 -- 172.21.15.2:0/70534245 learned_addr learned my addr 172.21.15.2:0/70534245 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6785b51640 1 --2- 172.21.15.2:0/70534245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67800686d0 0x7f678019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6786352640 1 --2- 172.21.15.2:0/70534245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6780107500 0x7f67801979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6785350640 1 -- 172.21.15.2:0/70534245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6780107500 msgr2=0x7f67801979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6785350640 1 --2- 172.21.15.2:0/70534245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6780107500 0x7f67801979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6785350640 1 -- 172.21.15.2:0/70534245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67800686d0 msgr2=0x7f678019d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6785350640 1 --2- 172.21.15.2:0/70534245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67800686d0 0x7f678019d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6785350640 1 -- 172.21.15.2:0/70534245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c009560 con 0x7f6780069090 2024-03-15T13:02:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6786352640 1 --2- 172.21.15.2:0/70534245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6780107500 0x7f67801979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:02:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6785350640 1 --2- 172.21.15.2:0/70534245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6780069090 0x7f678019dcf0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f677000d950 tx=0x7f677000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6776ffd640 1 -- 172.21.15.2:0/70534245 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770009c30 con 0x7f6780069090 2024-03-15T13:02:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6776ffd640 1 -- 172.21.15.2:0/70534245 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6770009d90 con 0x7f6780069090 2024-03-15T13:02:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6787db5640 1 -- 172.21.15.2:0/70534245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6780198610 con 0x7f6780069090 2024-03-15T13:02:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6776ffd640 1 -- 172.21.15.2:0/70534245 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677000f740 con 0x7f6780069090 2024-03-15T13:02:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.680+0000 7f6787db5640 1 -- 172.21.15.2:0/70534245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67801a4850 con 0x7f6780069090 2024-03-15T13:02:00.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.682+0000 7f6776ffd640 1 -- 172.21.15.2:0/70534245 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f6770010b50 con 0x7f6780069090 2024-03-15T13:02:00.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.683+0000 7f678a992740 1 Processor -- start 2024-03-15T13:02:00.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.683+0000 7f678a992740 1 -- start start 2024-03-15T13:02:00.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.683+0000 7f678a992740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443b84630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.683+0000 7f678a992740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b84b70 0x561443b8d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.683+0000 7f678a992740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b8d750 0x561443b8fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.683+0000 7f678a992740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561443b3f960 con 0x561443b84250 2024-03-15T13:02:00.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.683+0000 7f6776ffd640 1 --2- 172.21.15.2:0/70534245 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f676c0a1ba0 0x7f676c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.683+0000 7f678a992740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561443af3310 con 0x561443b8d750 2024-03-15T13:02:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.683+0000 7f678a992740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561443af0840 con 0x561443b84b70 2024-03-15T13:02:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.683+0000 7f6786352640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443b84630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.683+0000 7f6785b51640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b84b70 0x561443b8d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.683+0000 7f6785350640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b8d750 0x561443b8fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.683+0000 7f6785b51640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b84b70 0x561443b8d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:45944/0 (socket says 172.21.15.2:45944) 2024-03-15T13:02:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.683+0000 7f6785b51640 1 -- 172.21.15.2:0/912052156 learned_addr learned my addr 172.21.15.2:0/912052156 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.683+0000 7f6776ffd640 1 -- 172.21.15.2:0/70534245 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f67700cd7f0 con 0x7f6780069090 2024-03-15T13:02:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.684+0000 7f6786352640 1 --2- 172.21.15.2:0/70534245 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f676c0a1ba0 0x7f676c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.684+0000 7f6785b51640 1 -- 172.21.15.2:0/912052156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b8d750 msgr2=0x561443b8fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.684+0000 7f6785b51640 1 --2- 172.21.15.2:0/912052156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b8d750 0x561443b8fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.684+0000 7f6785b51640 1 -- 172.21.15.2:0/912052156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 msgr2=0x561443b84630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.684+0000 7f6785b51640 1 --2- 172.21.15.2:0/912052156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443b84630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.684+0000 7f6785b51640 1 -- 172.21.15.2:0/912052156 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561443aae060 con 0x561443b84b70 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.684+0000 7f6785b51640 1 --2- 172.21.15.2:0/912052156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b84b70 0x561443b8d210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f677c009230 tx=0x7f677c02c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f50cc23a5d3393f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.684+0000 7f6786352640 1 --2- 172.21.15.2:0/70534245 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f676c0a1ba0 0x7f676c0a4060 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6778009fd0 tx=0x7f6778009500 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.684+0000 7f676bfff640 1 -- 172.21.15.2:0/912052156 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c0046c0 con 0x561443b84b70 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.684+0000 7f676bfff640 1 -- 172.21.15.2:0/912052156 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f677c004820 con 0x561443b84b70 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.684+0000 7f676bfff640 1 -- 172.21.15.2:0/912052156 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c004520 con 0x561443b84b70 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.685+0000 7f678a992740 1 -- 172.21.15.2:0/912052156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b84b70 msgr2=0x561443b8d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.685+0000 7f678a992740 1 --2- 172.21.15.2:0/912052156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b84b70 0x561443b8d210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f677c009230 tx=0x7f677c02c680 comp rx=0 tx=0).stop 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.685+0000 7f678a992740 1 -- 172.21.15.2:0/912052156 shutdown_connections 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.685+0000 7f678a992740 1 --2- 172.21.15.2:0/912052156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b8d750 0x561443b8fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.685+0000 7f678a992740 1 --2- 172.21.15.2:0/912052156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b84b70 0x561443b8d210 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.685+0000 7f678a992740 1 --2- 172.21.15.2:0/912052156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443b84630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.685+0000 7f678a992740 1 -- 172.21.15.2:0/912052156 >> 172.21.15.2:0/912052156 conn(0x561443b7e980 msgr2=0x561443b7ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.685+0000 7f678a992740 1 -- 172.21.15.2:0/912052156 shutdown_connections 2024-03-15T13:02:00.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.685+0000 7f678a992740 1 -- 172.21.15.2:0/912052156 wait complete. 2024-03-15T13:02:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.685+0000 7f678a992740 1 -- start start 2024-03-15T13:02:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.685+0000 7f678a992740 1 Processor -- start 2024-03-15T13:02:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f678a992740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443b8d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f678a992740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84b70 0x561443b8a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f678a992740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b8d750 0x561443b8a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f678a992740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561443aec020 con 0x561443b84250 2024-03-15T13:02:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f678a992740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614437bf430 con 0x561443b84b70 2024-03-15T13:02:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f678a992740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144377be00 con 0x561443b8d750 2024-03-15T13:02:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f678a992740 10 client.0 _ll_register_callbacks cb 0x561443a93cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f6785b51640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443b8d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f6785b51640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443b8d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33982/0 (socket says 172.21.15.2:33982) 2024-03-15T13:02:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f6785b51640 1 -- 172.21.15.2:0/1292696496 learned_addr learned my addr 172.21.15.2:0/1292696496 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f6786352640 1 --2- 172.21.15.2:0/1292696496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b8d750 0x561443b8a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f6785350640 1 --2- 172.21.15.2:0/1292696496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84b70 0x561443b8a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f6785b51640 1 -- 172.21.15.2:0/1292696496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b8d750 msgr2=0x561443b8a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f6785b51640 1 --2- 172.21.15.2:0/1292696496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b8d750 0x561443b8a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f6785b51640 1 -- 172.21.15.2:0/1292696496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84b70 msgr2=0x561443b8a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f6785b51640 1 --2- 172.21.15.2:0/1292696496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84b70 0x561443b8a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f6785b51640 1 -- 172.21.15.2:0/1292696496 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678010ec30 con 0x561443b84250 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f6785b51640 1 --2- 172.21.15.2:0/1292696496 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443b8d0d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f677c000c00 tx=0x7f677c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd368254b6764ce9 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.686+0000 7f67697fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.687+0000 7f67697fa640 1 -- 172.21.15.2:0/1292696496 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c0051a0 con 0x561443b84250 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.687+0000 7f67697fa640 1 -- 172.21.15.2:0/1292696496 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f677c005300 con 0x561443b84250 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.687+0000 7f678a992740 1 -- 172.21.15.2:0/1292696496 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614437eb7e0 con 0x561443b84250 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.687+0000 7f67697fa640 1 -- 172.21.15.2:0/1292696496 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f677c004500 con 0x561443b84250 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.687+0000 7f678a992740 10 client.16629 fetch_fsmap learned FSMap version 36 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.687+0000 7f678a992740 1 -- 172.21.15.2:0/1292696496 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561443aea6a0 con 0x561443b84250 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.687+0000 7f67697fa640 1 -- 172.21.15.2:0/1292696496 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5614437eb7e0 con 0x561443b84250 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.687+0000 7f67697fa640 1 -- 172.21.15.2:0/1292696496 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) v1 ==== 7982+0+0 (secure 0 0 0) 0x7f677c03c050 con 0x561443b84250 2024-03-15T13:02:00.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.687+0000 7f678a992740 10 client.16629 fetch_fsmap finished waiting for FSMap version 36 2024-03-15T13:02:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.687+0000 7f678a992740 10 client.16629 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:02:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.687+0000 7f678a992740 20 client.16629 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.687+0000 7f678a992740 1 --2- 172.21.15.2:0/1292696496 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561443b9d490 0x561443b9f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.687+0000 7f678a992740 4 client.16629 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:02:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.687+0000 7f678a992740 1 -- 172.21.15.2:0/1292696496 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561443828720 con 0x561443b9d490 2024-03-15T13:02:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.687+0000 7f6785350640 1 --2- 172.21.15.2:0/1292696496 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561443b9d490 0x561443b9f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.688+0000 7f67697fa640 10 client.16629 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:02:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.688+0000 7f6785350640 1 --2- 172.21.15.2:0/1292696496 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561443b9d490 0x561443b9f8b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4baa9947abc13961 server_cookie=99df58a1c4ee368 in_seq=0 out_seq=0 2024-03-15T13:02:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.688+0000 7f67697fa640 1 -- 172.21.15.2:0/1292696496 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561443828720 con 0x561443b9d490 2024-03-15T13:02:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.688+0000 7f67697fa640 10 client.16629 handle_command_reply: tid=0 2024-03-15T13:02:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.688+0000 7f678a992740 1 client.16629 shutdown 2024-03-15T13:02:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.689+0000 7f678a992740 1 -- 172.21.15.2:0/1292696496 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 msgr2=0x561443b8d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.689+0000 7f678a992740 1 --2- 172.21.15.2:0/1292696496 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443b8d0d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f677c000c00 tx=0x7f677c009260 comp rx=0 tx=0).stop 2024-03-15T13:02:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.689+0000 7f678a992740 1 -- 172.21.15.2:0/1292696496 shutdown_connections 2024-03-15T13:02:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.689+0000 7f678a992740 1 --2- 172.21.15.2:0/1292696496 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561443b9d490 0x561443b9f8b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.689+0000 7f678a992740 1 --2- 172.21.15.2:0/1292696496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b8d750 0x561443b8a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.689+0000 7f678a992740 1 --2- 172.21.15.2:0/1292696496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84b70 0x561443b8a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.689+0000 7f67697fa640 0 client.16629 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:02:00.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.689+0000 7f678a992740 1 --2- 172.21.15.2:0/1292696496 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443b8d0d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.689+0000 7f678a992740 1 -- 172.21.15.2:0/1292696496 >> 172.21.15.2:0/1292696496 conn(0x561443b7e980 msgr2=0x561443b806d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.689+0000 7f678a992740 1 -- 172.21.15.2:0/1292696496 shutdown_connections 2024-03-15T13:02:00.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.689+0000 7f678a992740 1 -- 172.21.15.2:0/1292696496 wait complete. 2024-03-15T13:02:00.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.689+0000 7f678a992740 20 client.16629 trim_cache size 0 max 16384 2024-03-15T13:02:00.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f678a992740 1 Processor -- start 2024-03-15T13:02:00.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f678a992740 1 -- start start 2024-03-15T13:02:00.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f678a992740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84250 0x561443b7da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f678a992740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84b70 0x561443b8cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f678a992740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b8d750 0x561443b8d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f678a992740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561443a71130 con 0x561443b84b70 2024-03-15T13:02:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f678a992740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561443b80d10 con 0x561443b84250 2024-03-15T13:02:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f678a992740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561443b8fbc0 con 0x561443b8d750 2024-03-15T13:02:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f6785350640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84b70 0x561443b8cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f6785350640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84b70 0x561443b8cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33986/0 (socket says 172.21.15.2:33986) 2024-03-15T13:02:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f6785350640 1 -- 172.21.15.2:0/1775622272 learned_addr learned my addr 172.21.15.2:0/1775622272 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f6785b51640 1 --2- 172.21.15.2:0/1775622272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84250 0x561443b7da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f6786352640 1 --2- 172.21.15.2:0/1775622272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b8d750 0x561443b8d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f6785350640 1 -- 172.21.15.2:0/1775622272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b8d750 msgr2=0x561443b8d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f6785350640 1 --2- 172.21.15.2:0/1775622272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b8d750 0x561443b8d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f6785350640 1 -- 172.21.15.2:0/1775622272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84250 msgr2=0x561443b7da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f6785350640 1 --2- 172.21.15.2:0/1775622272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84250 0x561443b7da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f6785350640 1 -- 172.21.15.2:0/1775622272 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561443a86c20 con 0x561443b84b70 2024-03-15T13:02:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f6785350640 1 --2- 172.21.15.2:0/1775622272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84b70 0x561443b8cb90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6770000c00 tx=0x7f6770016310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961ab7438f82e986 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.702+0000 7f676b7fe640 1 -- 172.21.15.2:0/1775622272 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677009a070 con 0x561443b84b70 2024-03-15T13:02:00.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.703+0000 7f676b7fe640 1 -- 172.21.15.2:0/1775622272 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f677009e1c0 con 0x561443b84b70 2024-03-15T13:02:00.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.703+0000 7f676b7fe640 1 -- 172.21.15.2:0/1775622272 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677009a2d0 con 0x561443b84b70 2024-03-15T13:02:00.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.703+0000 7f678a992740 1 -- 172.21.15.2:0/1775622272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84b70 msgr2=0x561443b8cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.703+0000 7f678a992740 1 --2- 172.21.15.2:0/1775622272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84b70 0x561443b8cb90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6770000c00 tx=0x7f6770016310 comp rx=0 tx=0).stop 2024-03-15T13:02:00.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.703+0000 7f678a992740 1 -- 172.21.15.2:0/1775622272 shutdown_connections 2024-03-15T13:02:00.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.703+0000 7f678a992740 1 --2- 172.21.15.2:0/1775622272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b8d750 0x561443b8d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.703+0000 7f678a992740 1 --2- 172.21.15.2:0/1775622272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84b70 0x561443b8cb90 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.703+0000 7f678a992740 1 --2- 172.21.15.2:0/1775622272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84250 0x561443b7da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.703+0000 7f678a992740 1 -- 172.21.15.2:0/1775622272 >> 172.21.15.2:0/1775622272 conn(0x561443b7e980 msgr2=0x561443b7d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.703+0000 7f678a992740 1 -- 172.21.15.2:0/1775622272 shutdown_connections 2024-03-15T13:02:00.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.703+0000 7f678a992740 1 -- 172.21.15.2:0/1775622272 wait complete. 2024-03-15T13:02:00.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.704+0000 7f678a992740 1 -- start start 2024-03-15T13:02:00.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.704+0000 7f678a992740 1 Processor -- start 2024-03-15T13:02:00.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.704+0000 7f678a992740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443bf1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.704+0000 7f678a992740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84b70 0x561443bf1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.704+0000 7f678a992740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b8d750 0x561443b873d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.704+0000 7f678a992740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561443b87ad0 con 0x561443b84250 2024-03-15T13:02:00.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.704+0000 7f678a992740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561443b87c40 con 0x561443b84b70 2024-03-15T13:02:00.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.704+0000 7f678a992740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561443b87e00 con 0x561443b8d750 2024-03-15T13:02:00.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.704+0000 7f678a992740 10 client.0 _ll_register_callbacks cb 0x7f6778009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:00.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.704+0000 7f6785b51640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443bf1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.704+0000 7f6785b51640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443bf1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33988/0 (socket says 172.21.15.2:33988) 2024-03-15T13:02:00.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.704+0000 7f6785b51640 1 -- 172.21.15.2:0/1461404930 learned_addr learned my addr 172.21.15.2:0/1461404930 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.704+0000 7f6785350640 1 --2- 172.21.15.2:0/1461404930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84b70 0x561443bf1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.704+0000 7f6786352640 1 --2- 172.21.15.2:0/1461404930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b8d750 0x561443b873d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.705+0000 7f6785b51640 1 -- 172.21.15.2:0/1461404930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b8d750 msgr2=0x561443b873d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.705+0000 7f6785b51640 1 --2- 172.21.15.2:0/1461404930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b8d750 0x561443b873d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.705+0000 7f6785b51640 1 -- 172.21.15.2:0/1461404930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84b70 msgr2=0x561443bf1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.705+0000 7f6785b51640 1 --2- 172.21.15.2:0/1461404930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84b70 0x561443bf1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.705+0000 7f6785b51640 1 -- 172.21.15.2:0/1461404930 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67801a4850 con 0x561443b84250 2024-03-15T13:02:00.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.705+0000 7f6785350640 1 --2- 172.21.15.2:0/1461404930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84b70 0x561443bf1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:00.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.705+0000 7f6785b51640 1 --2- 172.21.15.2:0/1461404930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443bf1050 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f677c034440 tx=0x7f677c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a6a106ae9490c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.705+0000 7f67697fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:00.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.705+0000 7f67697fa640 1 -- 172.21.15.2:0/1461404930 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c0046c0 con 0x561443b84250 2024-03-15T13:02:00.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.705+0000 7f67697fa640 1 -- 172.21.15.2:0/1461404930 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f677c000e40 con 0x561443b84250 2024-03-15T13:02:00.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.705+0000 7f678a992740 1 -- 172.21.15.2:0/1461404930 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561443b7a0f0 con 0x561443b84250 2024-03-15T13:02:00.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.705+0000 7f67697fa640 1 -- 172.21.15.2:0/1461404930 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f677c034510 con 0x561443b84250 2024-03-15T13:02:00.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.705+0000 7f678a992740 10 client.16635 fetch_fsmap learned FSMap version 36 2024-03-15T13:02:00.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.705+0000 7f678a992740 1 -- 172.21.15.2:0/1461404930 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561443aac8c0 con 0x561443b84250 2024-03-15T13:02:00.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.706+0000 7f67697fa640 1 -- 172.21.15.2:0/1461404930 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f677c068080 con 0x561443b84250 2024-03-15T13:02:00.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.706+0000 7f67697fa640 1 -- 172.21.15.2:0/1461404930 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) v1 ==== 7982+0+0 (secure 0 0 0) 0x7f677c03c040 con 0x561443b84250 2024-03-15T13:02:00.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.706+0000 7f678a992740 10 client.16635 fetch_fsmap finished waiting for FSMap version 36 2024-03-15T13:02:00.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.706+0000 7f678a992740 10 client.16635 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:02:00.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.706+0000 7f678a992740 20 client.16635 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:00.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.706+0000 7f678a992740 1 --2- 172.21.15.2:0/1461404930 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561443bf9d70 0x561443bfc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.706+0000 7f678a992740 4 client.16635 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:02:00.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.706+0000 7f678a992740 1 -- 172.21.15.2:0/1461404930 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f677c004820 con 0x561443bf9d70 2024-03-15T13:02:00.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.706+0000 7f6785350640 1 --2- 172.21.15.2:0/1461404930 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561443bf9d70 0x561443bfc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.707+0000 7f67697fa640 10 client.16635 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:02:00.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.707+0000 7f6785350640 1 --2- 172.21.15.2:0/1461404930 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561443bf9d70 0x561443bfc190 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7579875d256778a0 server_cookie=523f55fe2f51307 in_seq=0 out_seq=0 2024-03-15T13:02:00.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.707+0000 7f67697fa640 1 -- 172.21.15.2:0/1461404930 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f677c004820 con 0x561443bf9d70 2024-03-15T13:02:00.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.707+0000 7f67697fa640 10 client.16635 handle_command_reply: tid=0 2024-03-15T13:02:00.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.707+0000 7f678a992740 1 client.16635 shutdown 2024-03-15T13:02:00.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.707+0000 7f678a992740 1 -- 172.21.15.2:0/1461404930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 msgr2=0x561443bf1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.707+0000 7f678a992740 1 --2- 172.21.15.2:0/1461404930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443bf1050 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f677c034440 tx=0x7f677c004270 comp rx=0 tx=0).stop 2024-03-15T13:02:00.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.707+0000 7f678a992740 1 -- 172.21.15.2:0/1461404930 shutdown_connections 2024-03-15T13:02:00.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.707+0000 7f678a992740 1 --2- 172.21.15.2:0/1461404930 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x561443bf9d70 0x561443bfc190 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.707+0000 7f678a992740 1 --2- 172.21.15.2:0/1461404930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561443b8d750 0x561443b873d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.707+0000 7f678a992740 1 --2- 172.21.15.2:0/1461404930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561443b84b70 0x561443bf1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.707+0000 7f678a992740 1 --2- 172.21.15.2:0/1461404930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561443b84250 0x561443bf1050 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.707+0000 7f678a992740 1 -- 172.21.15.2:0/1461404930 >> 172.21.15.2:0/1461404930 conn(0x561443b7e980 msgr2=0x561443b7db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.707+0000 7f67697fa640 0 client.16635 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:02:00.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.708+0000 7f678a992740 1 -- 172.21.15.2:0/1461404930 shutdown_connections 2024-03-15T13:02:00.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.708+0000 7f678a992740 1 -- 172.21.15.2:0/1461404930 wait complete. 2024-03-15T13:02:00.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.708+0000 7f678a992740 20 client.16635 trim_cache size 0 max 16384 2024-03-15T13:02:00.699 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:02:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.708+0000 7f6787db5640 1 -- 172.21.15.2:0/70534245 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f676c0a1ba0 msgr2=0x7f676c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.708+0000 7f6787db5640 1 --2- 172.21.15.2:0/70534245 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f676c0a1ba0 0x7f676c0a4060 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6778009fd0 tx=0x7f6778009500 comp rx=0 tx=0).stop 2024-03-15T13:02:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.708+0000 7f6787db5640 1 -- 172.21.15.2:0/70534245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6780069090 msgr2=0x7f678019dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.708+0000 7f6787db5640 1 --2- 172.21.15.2:0/70534245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6780069090 0x7f678019dcf0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f677000d950 tx=0x7f677000de10 comp rx=0 tx=0).stop 2024-03-15T13:02:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.708+0000 7f6787db5640 1 -- 172.21.15.2:0/70534245 shutdown_connections 2024-03-15T13:02:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.708+0000 7f6787db5640 1 --2- 172.21.15.2:0/70534245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6780107500 0x7f67801979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.709+0000 7f6787db5640 1 --2- 172.21.15.2:0/70534245 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f676c0a1ba0 0x7f676c0a4060 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.709+0000 7f6787db5640 1 --2- 172.21.15.2:0/70534245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6780069090 0x7f678019dcf0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.709+0000 7f6787db5640 1 --2- 172.21.15.2:0/70534245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67800686d0 0x7f678019d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.709+0000 7f6787db5640 1 -- 172.21.15.2:0/70534245 >> 172.21.15.2:0/70534245 conn(0x7f67800774f0 msgr2=0x7f67800fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.709+0000 7f6787db5640 1 -- 172.21.15.2:0/70534245 shutdown_connections 2024-03-15T13:02:00.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.709+0000 7f6787db5640 1 -- 172.21.15.2:0/70534245 wait complete. 2024-03-15T13:02:00.707 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:02:00.707 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:02:00.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.829+0000 7fa3019b5640 1 -- 172.21.15.2:0/4140506176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc1056a0 msgr2=0x7fa2fc10d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.829+0000 7fa3019b5640 1 --2- 172.21.15.2:0/4140506176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc1056a0 0x7fa2fc10d270 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009930 tx=0x7fa2f402bc30 comp rx=0 tx=0).stop 2024-03-15T13:02:00.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.829+0000 7fa3019b5640 1 -- 172.21.15.2:0/4140506176 shutdown_connections 2024-03-15T13:02:00.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.829+0000 7fa3019b5640 1 --2- 172.21.15.2:0/4140506176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc1056a0 0x7fa2fc10d270 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.829+0000 7fa3019b5640 1 --2- 172.21.15.2:0/4140506176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2fc100f80 0x7fa2fc104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.829+0000 7fa3019b5640 1 --2- 172.21.15.2:0/4140506176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2fc1005d0 0x7fa2fc1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.829+0000 7fa3019b5640 1 -- 172.21.15.2:0/4140506176 >> 172.21.15.2:0/4140506176 conn(0x7fa2fc0774f0 msgr2=0x7fa2fc077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.830+0000 7fa3019b5640 1 -- 172.21.15.2:0/4140506176 shutdown_connections 2024-03-15T13:02:00.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.830+0000 7fa3019b5640 1 -- 172.21.15.2:0/4140506176 wait complete. 2024-03-15T13:02:00.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.830+0000 7fa3019b5640 1 Processor -- start 2024-03-15T13:02:00.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.830+0000 7fa3019b5640 1 -- start start 2024-03-15T13:02:00.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.830+0000 7fa3019b5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc1005d0 0x7fa2fc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.830+0000 7fa3019b5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2fc1056a0 0x7fa2fc198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.830+0000 7fa3019b5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2fc199420 0x7fa2fc1a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.830+0000 7fa3019b5640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc1a4070 con 0x7fa2fc1005d0 2024-03-15T13:02:00.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.830+0000 7fa3019b5640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc1a41e0 con 0x7fa2fc1056a0 2024-03-15T13:02:00.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.830+0000 7fa3019b5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc1a43a0 con 0x7fa2fc199420 2024-03-15T13:02:00.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa2faffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc1005d0 0x7fa2fc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa2faffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc1005d0 0x7fa2fc198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34000/0 (socket says 172.21.15.2:34000) 2024-03-15T13:02:00.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa2faffd640 1 -- 172.21.15.2:0/1836842440 learned_addr learned my addr 172.21.15.2:0/1836842440 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa2fb7fe640 1 --2- 172.21.15.2:0/1836842440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2fc199420 0x7fa2fc1a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa2fa7fc640 1 --2- 172.21.15.2:0/1836842440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2fc1056a0 0x7fa2fc198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa2faffd640 1 -- 172.21.15.2:0/1836842440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2fc199420 msgr2=0x7fa2fc1a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa2faffd640 1 --2- 172.21.15.2:0/1836842440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2fc199420 0x7fa2fc1a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa2faffd640 1 -- 172.21.15.2:0/1836842440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2fc1056a0 msgr2=0x7fa2fc198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa2faffd640 1 --2- 172.21.15.2:0/1836842440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2fc1056a0 0x7fa2fc198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa2faffd640 1 -- 172.21.15.2:0/1836842440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f4009560 con 0x7fa2fc1005d0 2024-03-15T13:02:00.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa2faffd640 1 --2- 172.21.15.2:0/1836842440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc1005d0 0x7fa2fc198950 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00ebc0 tx=0x7fa2ec00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa3009b3640 1 -- 172.21.15.2:0/1836842440 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0099b0 con 0x7fa2fc1005d0 2024-03-15T13:02:00.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa3009b3640 1 -- 172.21.15.2:0/1836842440 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa2ec00eec0 con 0x7fa2fc1005d0 2024-03-15T13:02:00.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa3019b5640 1 -- 172.21.15.2:0/1836842440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2fc1a4680 con 0x7fa2fc1005d0 2024-03-15T13:02:00.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa3009b3640 1 -- 172.21.15.2:0/1836842440 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec018b10 con 0x7fa2fc1005d0 2024-03-15T13:02:00.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.831+0000 7fa3019b5640 1 -- 172.21.15.2:0/1836842440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2fc1a4be0 con 0x7fa2fc1005d0 2024-03-15T13:02:00.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.834+0000 7fa3009b3640 1 -- 172.21.15.2:0/1836842440 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fa2ec018c70 con 0x7fa2fc1005d0 2024-03-15T13:02:00.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.834+0000 7fa30458b740 1 Processor -- start 2024-03-15T13:02:00.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.834+0000 7fa30458b740 1 -- start start 2024-03-15T13:02:00.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.834+0000 7fa30458b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536aba250 0x561536aba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.834+0000 7fa30458b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536abab70 0x561536ac3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.834+0000 7fa3009b3640 1 --2- 172.21.15.2:0/1836842440 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2e40a1c20 0x7fa2e40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.834+0000 7fa30458b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536ac3750 0x561536ac5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.834+0000 7fa30458b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561536a75960 con 0x561536aba250 2024-03-15T13:02:00.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.834+0000 7fa30458b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561536a29310 con 0x561536ac3750 2024-03-15T13:02:00.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.834+0000 7fa30458b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561536a26840 con 0x561536abab70 2024-03-15T13:02:00.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.834+0000 7fa2fb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536aba250 0x561536aba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.834+0000 7fa2fb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536aba250 0x561536aba630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34004/0 (socket says 172.21.15.2:34004) 2024-03-15T13:02:00.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.834+0000 7fa2fb7fe640 1 -- 172.21.15.2:0/110509883 learned_addr learned my addr 172.21.15.2:0/110509883 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.835+0000 7fa2fa7fc640 1 --2- 172.21.15.2:0/1836842440 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2e40a1c20 0x7fa2e40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.835+0000 7fa3009b3640 1 -- 172.21.15.2:0/1836842440 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa2ec014070 con 0x7fa2fc1005d0 2024-03-15T13:02:00.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.835+0000 7fa2faffd640 1 --2- 172.21.15.2:0/110509883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536abab70 0x561536ac3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.835+0000 7fa2fb7fe640 1 --2- 172.21.15.2:0/110509883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536ac3750 0x561536ac5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.835+0000 7fa2fb7fe640 1 -- 172.21.15.2:0/110509883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536abab70 msgr2=0x561536ac3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.835+0000 7fa2fb7fe640 1 --2- 172.21.15.2:0/110509883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536abab70 0x561536ac3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.835+0000 7fa2fb7fe640 1 -- 172.21.15.2:0/110509883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536ac3750 msgr2=0x561536ac5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.835+0000 7fa2fb7fe640 1 --2- 172.21.15.2:0/110509883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536ac3750 0x561536ac5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.835+0000 7fa2fb7fe640 1 -- 172.21.15.2:0/110509883 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615369e4060 con 0x561536aba250 2024-03-15T13:02:00.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.835+0000 7fa2fa7fc640 1 --2- 172.21.15.2:0/1836842440 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2e40a1c20 0x7fa2e40a40e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc0691a0 tx=0x7fa2f0005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.835+0000 7fa2fb7fe640 1 --2- 172.21.15.2:0/110509883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536aba250 0x561536aba630 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4006010 tx=0x7fa2f4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410cf93bf14ac705 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.835+0000 7fa2e1ffb640 1 -- 172.21.15.2:0/110509883 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4004a20 con 0x561536aba250 2024-03-15T13:02:00.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.835+0000 7fa2e1ffb640 1 -- 172.21.15.2:0/110509883 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa2f4004030 con 0x561536aba250 2024-03-15T13:02:00.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.835+0000 7fa2e1ffb640 1 -- 172.21.15.2:0/110509883 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4003750 con 0x561536aba250 2024-03-15T13:02:00.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.836+0000 7fa30458b740 1 -- 172.21.15.2:0/110509883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536aba250 msgr2=0x561536aba630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.836+0000 7fa30458b740 1 --2- 172.21.15.2:0/110509883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536aba250 0x561536aba630 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4006010 tx=0x7fa2f4005b50 comp rx=0 tx=0).stop 2024-03-15T13:02:00.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.836+0000 7fa30458b740 1 -- 172.21.15.2:0/110509883 shutdown_connections 2024-03-15T13:02:00.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.836+0000 7fa30458b740 1 --2- 172.21.15.2:0/110509883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536ac3750 0x561536ac5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.836+0000 7fa30458b740 1 --2- 172.21.15.2:0/110509883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536abab70 0x561536ac3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.836+0000 7fa30458b740 1 --2- 172.21.15.2:0/110509883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536aba250 0x561536aba630 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.836+0000 7fa30458b740 1 -- 172.21.15.2:0/110509883 >> 172.21.15.2:0/110509883 conn(0x561536ab4980 msgr2=0x561536ab4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.836+0000 7fa30458b740 1 -- 172.21.15.2:0/110509883 shutdown_connections 2024-03-15T13:02:00.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.836+0000 7fa30458b740 1 -- 172.21.15.2:0/110509883 wait complete. 2024-03-15T13:02:00.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.836+0000 7fa30458b740 1 -- start start 2024-03-15T13:02:00.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa30458b740 1 Processor -- start 2024-03-15T13:02:00.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa30458b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536aba250 0x561536ac3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa30458b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536abab70 0x561536ac04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa30458b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536ac3750 0x561536ac0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa30458b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561536a22020 con 0x561536ac3750 2024-03-15T13:02:00.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa30458b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615366f5430 con 0x561536aba250 2024-03-15T13:02:00.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa30458b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615366b1e00 con 0x561536abab70 2024-03-15T13:02:00.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa30458b740 10 client.0 _ll_register_callbacks cb 0x5615369c9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:00.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa2fb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536ac3750 0x561536ac0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa2fa7fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536abab70 0x561536ac04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa2fb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536ac3750 0x561536ac0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34014/0 (socket says 172.21.15.2:34014) 2024-03-15T13:02:00.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa2fb7fe640 1 -- 172.21.15.2:0/4234674151 learned_addr learned my addr 172.21.15.2:0/4234674151 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa2faffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536aba250 0x561536ac3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa2fb7fe640 1 -- 172.21.15.2:0/4234674151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536abab70 msgr2=0x561536ac04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa2fb7fe640 1 --2- 172.21.15.2:0/4234674151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536abab70 0x561536ac04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa2fb7fe640 1 -- 172.21.15.2:0/4234674151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536aba250 msgr2=0x561536ac3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa2fb7fe640 1 --2- 172.21.15.2:0/4234674151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536aba250 0x561536ac3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa2fb7fe640 1 -- 172.21.15.2:0/4234674151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2fc10da80 con 0x561536ac3750 2024-03-15T13:02:00.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa2fb7fe640 1 --2- 172.21.15.2:0/4234674151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536ac3750 0x561536ac0ab0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4005ab0 tx=0x7fa2f402cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cf123e02345e64 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.837+0000 7fa2daffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:00.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.838+0000 7fa2daffd640 1 -- 172.21.15.2:0/4234674151 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4004a20 con 0x561536ac3750 2024-03-15T13:02:00.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.838+0000 7fa2daffd640 1 -- 172.21.15.2:0/4234674151 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa2f40038e0 con 0x561536ac3750 2024-03-15T13:02:00.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.838+0000 7fa30458b740 1 -- 172.21.15.2:0/4234674151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615367217e0 con 0x561536ac3750 2024-03-15T13:02:00.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.838+0000 7fa2daffd640 1 -- 172.21.15.2:0/4234674151 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2f4003aa0 con 0x561536ac3750 2024-03-15T13:02:00.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.838+0000 7fa30458b740 10 client.16647 fetch_fsmap learned FSMap version 36 2024-03-15T13:02:00.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.838+0000 7fa30458b740 1 -- 172.21.15.2:0/4234674151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561536a206a0 con 0x561536ac3750 2024-03-15T13:02:00.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.838+0000 7fa2daffd640 1 -- 172.21.15.2:0/4234674151 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5615367217e0 con 0x561536ac3750 2024-03-15T13:02:00.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.838+0000 7fa2daffd640 1 -- 172.21.15.2:0/4234674151 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) v1 ==== 7982+0+0 (secure 0 0 0) 0x7fa2f403c070 con 0x561536ac3750 2024-03-15T13:02:00.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.838+0000 7fa30458b740 10 client.16647 fetch_fsmap finished waiting for FSMap version 36 2024-03-15T13:02:00.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.838+0000 7fa30458b740 10 client.16647 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:02:00.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.838+0000 7fa30458b740 20 client.16647 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:00.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.838+0000 7fa30458b740 1 --2- 172.21.15.2:0/4234674151 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x561536ad3700 0x561536ad5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.838+0000 7fa30458b740 4 client.16647 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:02:00.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.838+0000 7fa30458b740 1 -- 172.21.15.2:0/4234674151 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56153675e720 con 0x561536ad3700 2024-03-15T13:02:00.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.838+0000 7fa2faffd640 1 --2- 172.21.15.2:0/4234674151 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x561536ad3700 0x561536ad5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.839+0000 7fa2daffd640 10 client.16647 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:02:00.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.839+0000 7fa2faffd640 1 --2- 172.21.15.2:0/4234674151 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x561536ad3700 0x561536ad5b20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=346a6e819d50eeff server_cookie=3a518ff6183f7fe0 in_seq=0 out_seq=0 2024-03-15T13:02:00.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.840+0000 7fa2daffd640 1 -- 172.21.15.2:0/4234674151 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56153675e720 con 0x561536ad3700 2024-03-15T13:02:00.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.840+0000 7fa2daffd640 10 client.16647 handle_command_reply: tid=0 2024-03-15T13:02:00.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.840+0000 7fa30458b740 1 client.16647 shutdown 2024-03-15T13:02:00.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.840+0000 7fa30458b740 1 -- 172.21.15.2:0/4234674151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536ac3750 msgr2=0x561536ac0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.840+0000 7fa30458b740 1 --2- 172.21.15.2:0/4234674151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536ac3750 0x561536ac0ab0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4005ab0 tx=0x7fa2f402cb40 comp rx=0 tx=0).stop 2024-03-15T13:02:00.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.840+0000 7fa30458b740 1 -- 172.21.15.2:0/4234674151 shutdown_connections 2024-03-15T13:02:00.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.840+0000 7fa30458b740 1 --2- 172.21.15.2:0/4234674151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536ac3750 0x561536ac0ab0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.840+0000 7fa30458b740 1 --2- 172.21.15.2:0/4234674151 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x561536ad3700 0x561536ad5b20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.840+0000 7fa30458b740 1 --2- 172.21.15.2:0/4234674151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536abab70 0x561536ac04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.840+0000 7fa30458b740 1 --2- 172.21.15.2:0/4234674151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536aba250 0x561536ac3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.840+0000 7fa2daffd640 0 client.16647 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:02:00.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.840+0000 7fa30458b740 1 -- 172.21.15.2:0/4234674151 >> 172.21.15.2:0/4234674151 conn(0x561536ab4980 msgr2=0x561536ab66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.840+0000 7fa30458b740 1 -- 172.21.15.2:0/4234674151 shutdown_connections 2024-03-15T13:02:00.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.840+0000 7fa30458b740 1 -- 172.21.15.2:0/4234674151 wait complete. 2024-03-15T13:02:00.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.840+0000 7fa30458b740 20 client.16647 trim_cache size 0 max 16384 2024-03-15T13:02:00.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa30458b740 1 Processor -- start 2024-03-15T13:02:00.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa30458b740 1 -- start start 2024-03-15T13:02:00.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa30458b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536aba250 0x561536ab3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa30458b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536abab70 0x561536ac2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa30458b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536ac3750 0x561536ac3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa30458b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615369a7130 con 0x561536ac3750 2024-03-15T13:02:00.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa30458b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615367217e0 con 0x561536aba250 2024-03-15T13:02:00.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa30458b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561536ab9eb0 con 0x561536abab70 2024-03-15T13:02:00.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa2fb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536ac3750 0x561536ac3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa2fb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536ac3750 0x561536ac3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34022/0 (socket says 172.21.15.2:34022) 2024-03-15T13:02:00.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa2fb7fe640 1 -- 172.21.15.2:0/2232572266 learned_addr learned my addr 172.21.15.2:0/2232572266 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa2faffd640 1 --2- 172.21.15.2:0/2232572266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536aba250 0x561536ab3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa2fa7fc640 1 --2- 172.21.15.2:0/2232572266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536abab70 0x561536ac2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa2fb7fe640 1 -- 172.21.15.2:0/2232572266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536abab70 msgr2=0x561536ac2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa2fb7fe640 1 --2- 172.21.15.2:0/2232572266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536abab70 0x561536ac2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa2fb7fe640 1 -- 172.21.15.2:0/2232572266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536aba250 msgr2=0x561536ab3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa2fb7fe640 1 --2- 172.21.15.2:0/2232572266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536aba250 0x561536ab3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa2fb7fe640 1 -- 172.21.15.2:0/2232572266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615369bcc20 con 0x561536ac3750 2024-03-15T13:02:00.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.855+0000 7fa2fb7fe640 1 --2- 172.21.15.2:0/2232572266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536ac3750 0x561536ac3110 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4005a80 tx=0x7fa2f402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2559ebe2cfb3c095 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.856+0000 7fa2e17fa640 1 -- 172.21.15.2:0/2232572266 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4004920 con 0x561536ac3750 2024-03-15T13:02:00.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.856+0000 7fa2e17fa640 1 -- 172.21.15.2:0/2232572266 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa2f4004a80 con 0x561536ac3750 2024-03-15T13:02:00.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.856+0000 7fa2e17fa640 1 -- 172.21.15.2:0/2232572266 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4005040 con 0x561536ac3750 2024-03-15T13:02:00.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.856+0000 7fa30458b740 1 -- 172.21.15.2:0/2232572266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536ac3750 msgr2=0x561536ac3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.856+0000 7fa30458b740 1 --2- 172.21.15.2:0/2232572266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536ac3750 0x561536ac3110 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4005a80 tx=0x7fa2f402bc30 comp rx=0 tx=0).stop 2024-03-15T13:02:00.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.856+0000 7fa30458b740 1 -- 172.21.15.2:0/2232572266 shutdown_connections 2024-03-15T13:02:00.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.856+0000 7fa30458b740 1 --2- 172.21.15.2:0/2232572266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536ac3750 0x561536ac3110 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.856+0000 7fa30458b740 1 --2- 172.21.15.2:0/2232572266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536abab70 0x561536ac2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.856+0000 7fa30458b740 1 --2- 172.21.15.2:0/2232572266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536aba250 0x561536ab3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.856+0000 7fa30458b740 1 -- 172.21.15.2:0/2232572266 >> 172.21.15.2:0/2232572266 conn(0x561536ab4980 msgr2=0x561536ab32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.856+0000 7fa30458b740 1 -- 172.21.15.2:0/2232572266 shutdown_connections 2024-03-15T13:02:00.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.856+0000 7fa30458b740 1 -- 172.21.15.2:0/2232572266 wait complete. 2024-03-15T13:02:00.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.857+0000 7fa30458b740 1 -- start start 2024-03-15T13:02:00.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.857+0000 7fa30458b740 1 Processor -- start 2024-03-15T13:02:00.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.857+0000 7fa30458b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536aba250 0x561536acb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.857+0000 7fa30458b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536abab70 0x561536ac6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.857+0000 7fa30458b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536ac3750 0x561536ac6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.857+0000 7fa30458b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561536ac7020 con 0x561536aba250 2024-03-15T13:02:00.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.857+0000 7fa30458b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561536ac7190 con 0x561536abab70 2024-03-15T13:02:00.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.857+0000 7fa30458b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561536ac7350 con 0x561536ac3750 2024-03-15T13:02:00.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.857+0000 7fa30458b740 10 client.0 _ll_register_callbacks cb 0x7fa2ec018420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:00.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.857+0000 7fa2faffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536aba250 0x561536acb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.857+0000 7fa2faffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536aba250 0x561536acb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34032/0 (socket says 172.21.15.2:34032) 2024-03-15T13:02:00.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.857+0000 7fa2faffd640 1 -- 172.21.15.2:0/3857204730 learned_addr learned my addr 172.21.15.2:0/3857204730 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.857+0000 7fa2fa7fc640 1 --2- 172.21.15.2:0/3857204730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536abab70 0x561536ac6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.857+0000 7fa2fb7fe640 1 --2- 172.21.15.2:0/3857204730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536ac3750 0x561536ac6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa2faffd640 1 -- 172.21.15.2:0/3857204730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536ac3750 msgr2=0x561536ac6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa2faffd640 1 --2- 172.21.15.2:0/3857204730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536ac3750 0x561536ac6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa2faffd640 1 -- 172.21.15.2:0/3857204730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536abab70 msgr2=0x561536ac6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa2faffd640 1 --2- 172.21.15.2:0/3857204730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536abab70 0x561536ac6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa2faffd640 1 -- 172.21.15.2:0/3857204730 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561536a206a0 con 0x561536aba250 2024-03-15T13:02:00.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa2faffd640 1 --2- 172.21.15.2:0/3857204730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536aba250 0x561536acb200 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0cb550 tx=0x7fa2ec0b0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629f794c9d2ce305 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa2daffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:00.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa2daffd640 1 -- 172.21.15.2:0/3857204730 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0a3120 con 0x561536aba250 2024-03-15T13:02:00.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa2daffd640 1 -- 172.21.15.2:0/3857204730 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa2ec0970a0 con 0x561536aba250 2024-03-15T13:02:00.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa30458b740 1 -- 172.21.15.2:0/3857204730 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2f4004030 con 0x561536aba250 2024-03-15T13:02:00.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa2daffd640 1 -- 172.21.15.2:0/3857204730 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2ec097260 con 0x561536aba250 2024-03-15T13:02:00.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa30458b740 10 client.16653 fetch_fsmap learned FSMap version 36 2024-03-15T13:02:00.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa30458b740 1 -- 172.21.15.2:0/3857204730 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5615369e28c0 con 0x561536aba250 2024-03-15T13:02:00.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa2daffd640 1 -- 172.21.15.2:0/3857204730 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa2f4004030 con 0x561536aba250 2024-03-15T13:02:00.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa2daffd640 1 -- 172.21.15.2:0/3857204730 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) v1 ==== 7982+0+0 (secure 0 0 0) 0x7fa2ec0b3160 con 0x561536aba250 2024-03-15T13:02:00.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa30458b740 10 client.16653 fetch_fsmap finished waiting for FSMap version 36 2024-03-15T13:02:00.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa30458b740 10 client.16653 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:02:00.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.858+0000 7fa30458b740 20 client.16653 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:00.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.859+0000 7fa30458b740 1 --2- 172.21.15.2:0/3857204730 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x561536b2f640 0x561536b31a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.859+0000 7fa30458b740 4 client.16653 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:02:00.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.859+0000 7fa30458b740 1 -- 172.21.15.2:0/3857204730 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5615367ad890 con 0x561536b2f640 2024-03-15T13:02:00.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.859+0000 7fa2fa7fc640 1 --2- 172.21.15.2:0/3857204730 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x561536b2f640 0x561536b31a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.859+0000 7fa2daffd640 10 client.16653 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:02:00.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.859+0000 7fa2fa7fc640 1 --2- 172.21.15.2:0/3857204730 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x561536b2f640 0x561536b31a60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5bdb8203e0fe0ca server_cookie=96c6837aaaac58c2 in_seq=0 out_seq=0 2024-03-15T13:02:00.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.859+0000 7fa2daffd640 1 -- 172.21.15.2:0/3857204730 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5615367ad890 con 0x561536b2f640 2024-03-15T13:02:00.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.859+0000 7fa2daffd640 10 client.16653 handle_command_reply: tid=0 2024-03-15T13:02:00.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.859+0000 7fa30458b740 1 client.16653 shutdown 2024-03-15T13:02:00.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.859+0000 7fa30458b740 1 -- 172.21.15.2:0/3857204730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536aba250 msgr2=0x561536acb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.859+0000 7fa30458b740 1 --2- 172.21.15.2:0/3857204730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536aba250 0x561536acb200 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0cb550 tx=0x7fa2ec0b0040 comp rx=0 tx=0).stop 2024-03-15T13:02:00.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa30458b740 1 -- 172.21.15.2:0/3857204730 shutdown_connections 2024-03-15T13:02:00.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa30458b740 1 --2- 172.21.15.2:0/3857204730 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x561536b2f640 0x561536b31a60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa30458b740 1 --2- 172.21.15.2:0/3857204730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561536ac3750 0x561536ac6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa30458b740 1 --2- 172.21.15.2:0/3857204730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561536abab70 0x561536ac6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa30458b740 1 --2- 172.21.15.2:0/3857204730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561536aba250 0x561536acb200 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa30458b740 1 -- 172.21.15.2:0/3857204730 >> 172.21.15.2:0/3857204730 conn(0x561536ab4980 msgr2=0x561536ab6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa2daffd640 0 client.16653 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:02:00.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa30458b740 1 -- 172.21.15.2:0/3857204730 shutdown_connections 2024-03-15T13:02:00.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa30458b740 1 -- 172.21.15.2:0/3857204730 wait complete. 2024-03-15T13:02:00.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa30458b740 20 client.16653 trim_cache size 0 max 16384 2024-03-15T13:02:00.852 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:02:00.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa3019b5640 1 -- 172.21.15.2:0/1836842440 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2e40a1c20 msgr2=0x7fa2e40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa3019b5640 1 --2- 172.21.15.2:0/1836842440 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2e40a1c20 0x7fa2e40a40e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc0691a0 tx=0x7fa2f0005f90 comp rx=0 tx=0).stop 2024-03-15T13:02:00.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa3019b5640 1 -- 172.21.15.2:0/1836842440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc1005d0 msgr2=0x7fa2fc198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa3019b5640 1 --2- 172.21.15.2:0/1836842440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc1005d0 0x7fa2fc198950 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00ebc0 tx=0x7fa2ec00c6a0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa3019b5640 1 -- 172.21.15.2:0/1836842440 shutdown_connections 2024-03-15T13:02:00.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa3019b5640 1 --2- 172.21.15.2:0/1836842440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2fc199420 0x7fa2fc1a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa3019b5640 1 --2- 172.21.15.2:0/1836842440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2fc1056a0 0x7fa2fc198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa3019b5640 1 --2- 172.21.15.2:0/1836842440 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2e40a1c20 0x7fa2e40a40e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa3019b5640 1 --2- 172.21.15.2:0/1836842440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc1005d0 0x7fa2fc198950 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa3019b5640 1 -- 172.21.15.2:0/1836842440 >> 172.21.15.2:0/1836842440 conn(0x7fa2fc0774f0 msgr2=0x7fa2fc0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa3019b5640 1 -- 172.21.15.2:0/1836842440 shutdown_connections 2024-03-15T13:02:00.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.860+0000 7fa3019b5640 1 -- 172.21.15.2:0/1836842440 wait complete. 2024-03-15T13:02:00.856 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:02:00.856 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:02:00.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.998+0000 7f800aa6b640 1 -- 172.21.15.2:0/4057729594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8004107a90 msgr2=0x7f8004107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.998+0000 7f800aa6b640 1 --2- 172.21.15.2:0/4057729594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8004107a90 0x7f8004107f10 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009930 tx=0x7f7ff002bc30 comp rx=0 tx=0).stop 2024-03-15T13:02:00.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.998+0000 7f800aa6b640 1 -- 172.21.15.2:0/4057729594 shutdown_connections 2024-03-15T13:02:00.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.998+0000 7f800aa6b640 1 --2- 172.21.15.2:0/4057729594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8004108450 0x7f8004114960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.998+0000 7f800aa6b640 1 --2- 172.21.15.2:0/4057729594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8004107a90 0x7f8004107f10 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.999+0000 7f800aa6b640 1 --2- 172.21.15.2:0/4057729594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f800410daa0 0x7f800410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.999+0000 7f800aa6b640 1 -- 172.21.15.2:0/4057729594 >> 172.21.15.2:0/4057729594 conn(0x7f8004075a30 msgr2=0x7f8004077e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.999+0000 7f800aa6b640 1 -- 172.21.15.2:0/4057729594 shutdown_connections 2024-03-15T13:02:00.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.999+0000 7f800aa6b640 1 -- 172.21.15.2:0/4057729594 wait complete. 2024-03-15T13:02:00.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:00.999+0000 7f800aa6b640 1 Processor -- start 2024-03-15T13:02:00.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.000+0000 7f800aa6b640 1 -- start start 2024-03-15T13:02:00.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.000+0000 7f800aa6b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8004107a90 0x7f800419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.000+0000 7f800aa6b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8004108450 0x7f800419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.000+0000 7f800aa6b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f800410daa0 0x7f80041a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.000+0000 7f800aa6b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80041a3000 con 0x7f8004108450 2024-03-15T13:02:00.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.000+0000 7f800aa6b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80041a3170 con 0x7f8004107a90 2024-03-15T13:02:00.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.000+0000 7f800aa6b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80041a3330 con 0x7f800410daa0 2024-03-15T13:02:00.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.000+0000 7f8009008640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f800410daa0 0x7f80041a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.000+0000 7f8003fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8004108450 0x7f800419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.000+0000 7f8009008640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f800410daa0 0x7f80041a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:46040/0 (socket says 172.21.15.2:46040) 2024-03-15T13:02:00.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.000+0000 7f8009008640 1 -- 172.21.15.2:0/497383758 learned_addr learned my addr 172.21.15.2:0/497383758 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.000+0000 7f8008807640 1 --2- 172.21.15.2:0/497383758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8004107a90 0x7f800419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.001+0000 7f8003fff640 1 -- 172.21.15.2:0/497383758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f800410daa0 msgr2=0x7f80041a2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.001+0000 7f8003fff640 1 --2- 172.21.15.2:0/497383758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f800410daa0 0x7f80041a2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.001+0000 7f8003fff640 1 -- 172.21.15.2:0/497383758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8004107a90 msgr2=0x7f800419e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.001+0000 7f8003fff640 1 --2- 172.21.15.2:0/497383758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8004107a90 0x7f800419e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.001+0000 7f8003fff640 1 -- 172.21.15.2:0/497383758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff0009560 con 0x7f8004108450 2024-03-15T13:02:00.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.001+0000 7f8008807640 1 --2- 172.21.15.2:0/497383758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8004107a90 0x7f800419e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:02:00.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.001+0000 7f8003fff640 1 --2- 172.21.15.2:0/497383758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8004108450 0x7f800419e6d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0000c00 tx=0x7f7ff00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.001+0000 7f8001ffb640 1 -- 172.21.15.2:0/497383758 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0004ae0 con 0x7f8004108450 2024-03-15T13:02:00.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.001+0000 7f800aa6b640 1 -- 172.21.15.2:0/497383758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80041a35b0 con 0x7f8004108450 2024-03-15T13:02:00.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.001+0000 7f800aa6b640 1 -- 172.21.15.2:0/497383758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80041aae90 con 0x7f8004108450 2024-03-15T13:02:00.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.002+0000 7f8001ffb640 1 -- 172.21.15.2:0/497383758 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7ff00054d0 con 0x7f8004108450 2024-03-15T13:02:00.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.002+0000 7f8001ffb640 1 -- 172.21.15.2:0/497383758 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0003b10 con 0x7f8004108450 2024-03-15T13:02:00.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.004+0000 7f8001ffb640 1 -- 172.21.15.2:0/497383758 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f7ff0003c70 con 0x7f8004108450 2024-03-15T13:02:00.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.005+0000 7f800d646740 1 Processor -- start 2024-03-15T13:02:00.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.005+0000 7f800d646740 1 -- start start 2024-03-15T13:02:00.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.005+0000 7f8001ffb640 1 --2- 172.21.15.2:0/497383758 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7ff40a1c20 0x7f7ff40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.005+0000 7f8009008640 1 --2- 172.21.15.2:0/497383758 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7ff40a1c20 0x7f7ff40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.005+0000 7f800d646740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 0x5591e2cc0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.005+0000 7f8001ffb640 1 -- 172.21.15.2:0/497383758 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f7ff00ef9a0 con 0x7f8004108450 2024-03-15T13:02:00.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.005+0000 7f800d646740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc0b70 0x5591e2cc9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.005+0000 7f800d646740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc9750 0x5591e2ccbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.005+0000 7f800d646740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e2c7b960 con 0x5591e2cc0b70 2024-03-15T13:02:00.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.005+0000 7f800d646740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e2c2f310 con 0x5591e2cc9750 2024-03-15T13:02:00.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.005+0000 7f800d646740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e2c2c840 con 0x5591e2cc0250 2024-03-15T13:02:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.005+0000 7f8008807640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 0x5591e2cc0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.006+0000 7f8009008640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc9750 0x5591e2ccbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.006+0000 7f8009008640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc9750 0x5591e2ccbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44586/0 (socket says 172.21.15.2:44586) 2024-03-15T13:02:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.006+0000 7f8009008640 1 -- 172.21.15.2:0/3872450132 learned_addr learned my addr 172.21.15.2:0/3872450132 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.006+0000 7f8009008640 1 --2- 172.21.15.2:0/497383758 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7ff40a1c20 0x7f7ff40a40e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0049d0 tx=0x7f7ffc011920 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.006+0000 7f8003fff640 1 --2- 172.21.15.2:0/3872450132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc0b70 0x5591e2cc9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.006+0000 7f8009008640 1 -- 172.21.15.2:0/3872450132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 msgr2=0x5591e2cc0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.006+0000 7f8009008640 1 --2- 172.21.15.2:0/3872450132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 0x5591e2cc0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.006+0000 7f8009008640 1 -- 172.21.15.2:0/3872450132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc0b70 msgr2=0x5591e2cc9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.006+0000 7f8009008640 1 --2- 172.21.15.2:0/3872450132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc0b70 0x5591e2cc9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.006+0000 7f8009008640 1 -- 172.21.15.2:0/3872450132 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591e2bea060 con 0x5591e2cc9750 2024-03-15T13:02:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.006+0000 7f8008807640 1 --2- 172.21.15.2:0/3872450132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 0x5591e2cc0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T13:02:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.006+0000 7f8003fff640 1 --2- 172.21.15.2:0/3872450132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc0b70 0x5591e2cc9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:02:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.006+0000 7f8009008640 1 --2- 172.21.15.2:0/3872450132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc9750 0x5591e2ccbb40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00f7e0 tx=0x7f7ffc01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c52a595da9feda7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.006+0000 7f7feeffd640 1 -- 172.21.15.2:0/3872450132 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc026640 con 0x5591e2cc9750 2024-03-15T13:02:00.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.007+0000 7f7feeffd640 1 -- 172.21.15.2:0/3872450132 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7ffc026c70 con 0x5591e2cc9750 2024-03-15T13:02:00.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.007+0000 7f7feeffd640 1 -- 172.21.15.2:0/3872450132 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc01dd90 con 0x5591e2cc9750 2024-03-15T13:02:00.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.008+0000 7f800d646740 1 -- 172.21.15.2:0/3872450132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc9750 msgr2=0x5591e2ccbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:00.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.008+0000 7f800d646740 1 --2- 172.21.15.2:0/3872450132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc9750 0x5591e2ccbb40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00f7e0 tx=0x7f7ffc01c000 comp rx=0 tx=0).stop 2024-03-15T13:02:00.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.008+0000 7f800d646740 1 -- 172.21.15.2:0/3872450132 shutdown_connections 2024-03-15T13:02:00.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.008+0000 7f800d646740 1 --2- 172.21.15.2:0/3872450132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc9750 0x5591e2ccbb40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.008+0000 7f800d646740 1 --2- 172.21.15.2:0/3872450132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc0b70 0x5591e2cc9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.008+0000 7f800d646740 1 --2- 172.21.15.2:0/3872450132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 0x5591e2cc0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.008+0000 7f800d646740 1 -- 172.21.15.2:0/3872450132 >> 172.21.15.2:0/3872450132 conn(0x5591e2cba980 msgr2=0x5591e2cbad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:00.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.008+0000 7f800d646740 1 -- 172.21.15.2:0/3872450132 shutdown_connections 2024-03-15T13:02:00.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.008+0000 7f800d646740 1 -- 172.21.15.2:0/3872450132 wait complete. 2024-03-15T13:02:00.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.008+0000 7f800d646740 1 -- start start 2024-03-15T13:02:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.008+0000 7f800d646740 1 Processor -- start 2024-03-15T13:02:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.008+0000 7f800d646740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc0250 0x5591e2cc90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f800d646740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0b70 0x5591e2cc6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f800d646740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc9750 0x5591e2cc6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f800d646740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e2c28020 con 0x5591e2cc9750 2024-03-15T13:02:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f800d646740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e28fb430 con 0x5591e2cc0250 2024-03-15T13:02:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f800d646740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e28b7e00 con 0x5591e2cc0b70 2024-03-15T13:02:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f800d646740 10 client.0 _ll_register_callbacks cb 0x5591e2bcfcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f8009008640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc9750 0x5591e2cc6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f8008807640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc0250 0x5591e2cc90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f8009008640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc9750 0x5591e2cc6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34068/0 (socket says 172.21.15.2:34068) 2024-03-15T13:02:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f8009008640 1 -- 172.21.15.2:0/3022031298 learned_addr learned my addr 172.21.15.2:0/3022031298 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f8003fff640 1 --2- 172.21.15.2:0/3022031298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0b70 0x5591e2cc6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f8008807640 1 -- 172.21.15.2:0/3022031298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0b70 msgr2=0x5591e2cc6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f8008807640 1 --2- 172.21.15.2:0/3022031298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0b70 0x5591e2cc6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f8008807640 1 -- 172.21.15.2:0/3022031298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc9750 msgr2=0x5591e2cc6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f8008807640 1 --2- 172.21.15.2:0/3022031298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc9750 0x5591e2cc6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f8008807640 1 -- 172.21.15.2:0/3022031298 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ffc00f5a0 con 0x5591e2cc0250 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f8008807640 1 --2- 172.21.15.2:0/3022031298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc0250 0x5591e2cc90f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8010670 tx=0x7f7ff8010b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fedb90c0068e30b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.009+0000 7f7fe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.010+0000 7f7fe3fff640 1 -- 172.21.15.2:0/3022031298 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800ead0 con 0x5591e2cc0250 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.010+0000 7f800d646740 1 -- 172.21.15.2:0/3022031298 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591e29277e0 con 0x5591e2cc0250 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.010+0000 7f7fe3fff640 1 -- 172.21.15.2:0/3022031298 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f7ff8016070 con 0x5591e2cc0250 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.010+0000 7f7fe3fff640 1 -- 172.21.15.2:0/3022031298 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ff8010e70 con 0x5591e2cc0250 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.010+0000 7f800d646740 10 client.26528 fetch_fsmap learned FSMap version 37 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.010+0000 7f800d646740 1 -- 172.21.15.2:0/3022031298 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5591e2c266a0 con 0x5591e2cc0250 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.010+0000 7f7fe3fff640 1 -- 172.21.15.2:0/3022031298 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5591e29277e0 con 0x5591e2cc0250 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.010+0000 7f7fe3fff640 1 -- 172.21.15.2:0/3022031298 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 37) v1 ==== 7986+0+0 (secure 0 0 0) 0x7f7ff801a420 con 0x5591e2cc0250 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.010+0000 7f800d646740 10 client.26528 fetch_fsmap finished waiting for FSMap version 37 2024-03-15T13:02:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.011+0000 7f800d646740 10 client.26528 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:02:01.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.011+0000 7f800d646740 20 client.26528 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:01.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.011+0000 7f800d646740 1 --2- 172.21.15.2:0/3022031298 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5591e2cd9890 0x5591e2cdbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.011+0000 7f800d646740 4 client.26528 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:02:01.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.011+0000 7f800d646740 1 -- 172.21.15.2:0/3022031298 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591e2964720 con 0x5591e2cd9890 2024-03-15T13:02:01.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.011+0000 7f8003fff640 1 --2- 172.21.15.2:0/3022031298 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5591e2cd9890 0x5591e2cdbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.012+0000 7f7fe3fff640 10 client.26528 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:02:01.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.012+0000 7f8003fff640 1 --2- 172.21.15.2:0/3022031298 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5591e2cd9890 0x5591e2cdbcb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9194a98f27face0 server_cookie=33486b5661dd0268 in_seq=0 out_seq=0 2024-03-15T13:02:01.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.012+0000 7f7fe3fff640 1 -- 172.21.15.2:0/3022031298 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5591e2964720 con 0x5591e2cd9890 2024-03-15T13:02:01.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.012+0000 7f7fe3fff640 10 client.26528 handle_command_reply: tid=0 2024-03-15T13:02:01.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.012+0000 7f800d646740 1 client.26528 shutdown 2024-03-15T13:02:01.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.013+0000 7f800d646740 1 -- 172.21.15.2:0/3022031298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc0250 msgr2=0x5591e2cc90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:01.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.013+0000 7f800d646740 1 --2- 172.21.15.2:0/3022031298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc0250 0x5591e2cc90f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8010670 tx=0x7f7ff8010b30 comp rx=0 tx=0).stop 2024-03-15T13:02:01.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.013+0000 7f800d646740 1 -- 172.21.15.2:0/3022031298 shutdown_connections 2024-03-15T13:02:01.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.013+0000 7f800d646740 1 --2- 172.21.15.2:0/3022031298 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5591e2cd9890 0x5591e2cdbcb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.013+0000 7f800d646740 1 --2- 172.21.15.2:0/3022031298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc9750 0x5591e2cc6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.013+0000 7f800d646740 1 --2- 172.21.15.2:0/3022031298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0b70 0x5591e2cc6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.013+0000 7f800d646740 1 --2- 172.21.15.2:0/3022031298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc0250 0x5591e2cc90f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.013+0000 7f800d646740 1 -- 172.21.15.2:0/3022031298 >> 172.21.15.2:0/3022031298 conn(0x5591e2cba980 msgr2=0x5591e2cbc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:01.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.013+0000 7f7fe3fff640 0 client.26528 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:02:01.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.013+0000 7f800d646740 1 -- 172.21.15.2:0/3022031298 shutdown_connections 2024-03-15T13:02:01.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.013+0000 7f800d646740 1 -- 172.21.15.2:0/3022031298 wait complete. 2024-03-15T13:02:01.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.013+0000 7f800d646740 20 client.26528 trim_cache size 0 max 16384 2024-03-15T13:02:01.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f800d646740 1 Processor -- start 2024-03-15T13:02:01.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f800d646740 1 -- start start 2024-03-15T13:02:01.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f800d646740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 0x5591e2cbc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f800d646740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc0b70 0x5591e2cb9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f800d646740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc9750 0x5591e2cc8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f800d646740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e2bad130 con 0x5591e2cc9750 2024-03-15T13:02:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f800d646740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e29277e0 con 0x5591e2cc0b70 2024-03-15T13:02:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f800d646740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e2cbb550 con 0x5591e2cc0250 2024-03-15T13:02:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f8009008640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc9750 0x5591e2cc8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f8008807640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 0x5591e2cbc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f8009008640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc9750 0x5591e2cc8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34072/0 (socket says 172.21.15.2:34072) 2024-03-15T13:02:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f8009008640 1 -- 172.21.15.2:0/2881812482 learned_addr learned my addr 172.21.15.2:0/2881812482 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f8003fff640 1 --2- 172.21.15.2:0/2881812482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc0b70 0x5591e2cb9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f8009008640 1 -- 172.21.15.2:0/2881812482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 msgr2=0x5591e2cbc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f8009008640 1 --2- 172.21.15.2:0/2881812482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 0x5591e2cbc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f8009008640 1 -- 172.21.15.2:0/2881812482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc0b70 msgr2=0x5591e2cb9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f8009008640 1 --2- 172.21.15.2:0/2881812482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc0b70 0x5591e2cb9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.029+0000 7f8009008640 1 -- 172.21.15.2:0/2881812482 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591e2bc2c20 con 0x5591e2cc9750 2024-03-15T13:02:01.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.030+0000 7f8009008640 1 --2- 172.21.15.2:0/2881812482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc9750 0x5591e2cc8bb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00fca0 tx=0x7f7ffc0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4268947cb7eeb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:01.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.030+0000 7f7fee7fc640 1 -- 172.21.15.2:0/2881812482 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc02a050 con 0x5591e2cc9750 2024-03-15T13:02:01.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.030+0000 7f7fee7fc640 1 -- 172.21.15.2:0/2881812482 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7ffc02b040 con 0x5591e2cc9750 2024-03-15T13:02:01.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.030+0000 7f7fee7fc640 1 -- 172.21.15.2:0/2881812482 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc026800 con 0x5591e2cc9750 2024-03-15T13:02:01.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.031+0000 7f800d646740 1 -- 172.21.15.2:0/2881812482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc9750 msgr2=0x5591e2cc8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:01.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.031+0000 7f800d646740 1 --2- 172.21.15.2:0/2881812482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc9750 0x5591e2cc8bb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00fca0 tx=0x7f7ffc0099b0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.031+0000 7f800d646740 1 -- 172.21.15.2:0/2881812482 shutdown_connections 2024-03-15T13:02:01.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.031+0000 7f800d646740 1 --2- 172.21.15.2:0/2881812482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc9750 0x5591e2cc8bb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.031+0000 7f800d646740 1 --2- 172.21.15.2:0/2881812482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc0b70 0x5591e2cb9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.031+0000 7f800d646740 1 --2- 172.21.15.2:0/2881812482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 0x5591e2cbc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.031+0000 7f800d646740 1 -- 172.21.15.2:0/2881812482 >> 172.21.15.2:0/2881812482 conn(0x5591e2cba980 msgr2=0x5591e2cb92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:01.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.031+0000 7f800d646740 1 -- 172.21.15.2:0/2881812482 shutdown_connections 2024-03-15T13:02:01.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.031+0000 7f800d646740 1 -- 172.21.15.2:0/2881812482 wait complete. 2024-03-15T13:02:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.031+0000 7f800d646740 1 -- start start 2024-03-15T13:02:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.031+0000 7f800d646740 1 Processor -- start 2024-03-15T13:02:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f800d646740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 0x5591e2cd1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f800d646740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc0b70 0x5591e2ccd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f800d646740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc9750 0x5591e2ccdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f800d646740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e2cce6e0 con 0x5591e2cc0b70 2024-03-15T13:02:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f800d646740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e2d35150 con 0x5591e2cc9750 2024-03-15T13:02:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f800d646740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e2d35290 con 0x5591e2cc0250 2024-03-15T13:02:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f8003fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc0b70 0x5591e2ccd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f800d646740 10 client.0 _ll_register_callbacks cb 0x7f7ff800b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f8008807640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 0x5591e2cd1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f8003fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc0b70 0x5591e2ccd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34088/0 (socket says 172.21.15.2:34088) 2024-03-15T13:02:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f8003fff640 1 -- 172.21.15.2:0/3030582984 learned_addr learned my addr 172.21.15.2:0/3030582984 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f8009008640 1 --2- 172.21.15.2:0/3030582984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc9750 0x5591e2ccdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f8003fff640 1 -- 172.21.15.2:0/3030582984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 msgr2=0x5591e2cd1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:01.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f8003fff640 1 --2- 172.21.15.2:0/3030582984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 0x5591e2cd1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f8003fff640 1 -- 172.21.15.2:0/3030582984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc9750 msgr2=0x5591e2ccdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:01.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f8003fff640 1 --2- 172.21.15.2:0/3030582984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc9750 0x5591e2ccdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f8003fff640 1 -- 172.21.15.2:0/3030582984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591e2bea060 con 0x5591e2cc0b70 2024-03-15T13:02:01.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f8003fff640 1 --2- 172.21.15.2:0/3030582984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc0b70 0x5591e2ccd8a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7ff002c170 tx=0x7f7ff00dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca685ee5eae1e42 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:01.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f7fe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:01.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f7fe3fff640 1 -- 172.21.15.2:0/3030582984 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff00c04e0 con 0x5591e2cc0b70 2024-03-15T13:02:01.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f7fe3fff640 1 -- 172.21.15.2:0/3030582984 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f7ff00c4b30 con 0x5591e2cc0b70 2024-03-15T13:02:01.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.032+0000 7f800d646740 1 -- 172.21.15.2:0/3030582984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ffc026c70 con 0x5591e2cc0b70 2024-03-15T13:02:01.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.033+0000 7f7fe3fff640 1 -- 172.21.15.2:0/3030582984 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ff0004370 con 0x5591e2cc0b70 2024-03-15T13:02:01.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.033+0000 7f800d646740 10 client.16671 fetch_fsmap learned FSMap version 37 2024-03-15T13:02:01.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.033+0000 7f800d646740 1 -- 172.21.15.2:0/3030582984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5591e2be88c0 con 0x5591e2cc0b70 2024-03-15T13:02:01.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.033+0000 7f7fe3fff640 1 -- 172.21.15.2:0/3030582984 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7ffc026c70 con 0x5591e2cc0b70 2024-03-15T13:02:01.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.033+0000 7f7fe3fff640 1 -- 172.21.15.2:0/3030582984 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 37) v1 ==== 7986+0+0 (secure 0 0 0) 0x7f7ff004f060 con 0x5591e2cc0b70 2024-03-15T13:02:01.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.033+0000 7f800d646740 10 client.16671 fetch_fsmap finished waiting for FSMap version 37 2024-03-15T13:02:01.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.033+0000 7f800d646740 10 client.16671 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:02:01.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.033+0000 7f800d646740 20 client.16671 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:01.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.033+0000 7f800d646740 1 --2- 172.21.15.2:0/3030582984 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5591e2d368d0 0x5591e2d38cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.033+0000 7f800d646740 4 client.16671 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:02:01.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.033+0000 7f800d646740 1 -- 172.21.15.2:0/3030582984 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5591e29b3890 con 0x5591e2d368d0 2024-03-15T13:02:01.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.033+0000 7f8008807640 1 --2- 172.21.15.2:0/3030582984 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5591e2d368d0 0x5591e2d38cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f8008807640 1 --2- 172.21.15.2:0/3030582984 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5591e2d368d0 0x5591e2d38cf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6914908965e9d847 server_cookie=d2ce2e70716066be in_seq=0 out_seq=0 2024-03-15T13:02:01.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f7fe3fff640 10 client.16671 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:02:01.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f7fe3fff640 1 -- 172.21.15.2:0/3030582984 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5591e29b3890 con 0x5591e2d368d0 2024-03-15T13:02:01.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f7fe3fff640 10 client.16671 handle_command_reply: tid=0 2024-03-15T13:02:01.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f800d646740 1 client.16671 shutdown 2024-03-15T13:02:01.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f800d646740 1 -- 172.21.15.2:0/3030582984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc0b70 msgr2=0x5591e2ccd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:01.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f800d646740 1 --2- 172.21.15.2:0/3030582984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc0b70 0x5591e2ccd8a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7ff002c170 tx=0x7f7ff00dc040 comp rx=0 tx=0).stop 2024-03-15T13:02:01.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f800d646740 1 -- 172.21.15.2:0/3030582984 shutdown_connections 2024-03-15T13:02:01.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f800d646740 1 --2- 172.21.15.2:0/3030582984 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5591e2d368d0 0x5591e2d38cf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f800d646740 1 --2- 172.21.15.2:0/3030582984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591e2cc9750 0x5591e2ccdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f7fe3fff640 0 client.16671 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:02:01.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f800d646740 1 --2- 172.21.15.2:0/3030582984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5591e2cc0b70 0x5591e2ccd8a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f800d646740 1 --2- 172.21.15.2:0/3030582984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5591e2cc0250 0x5591e2cd1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f800d646740 1 -- 172.21.15.2:0/3030582984 >> 172.21.15.2:0/3030582984 conn(0x5591e2cba980 msgr2=0x5591e2cb9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:01.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f800d646740 1 -- 172.21.15.2:0/3030582984 shutdown_connections 2024-03-15T13:02:01.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f800d646740 1 -- 172.21.15.2:0/3030582984 wait complete. 2024-03-15T13:02:01.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.034+0000 7f800d646740 20 client.16671 trim_cache size 0 max 16384 2024-03-15T13:02:01.025 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:02:01.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.035+0000 7f800aa6b640 1 -- 172.21.15.2:0/497383758 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7ff40a1c20 msgr2=0x7f7ff40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:01.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.035+0000 7f800aa6b640 1 --2- 172.21.15.2:0/497383758 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7ff40a1c20 0x7f7ff40a40e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0049d0 tx=0x7f7ffc011920 comp rx=0 tx=0).stop 2024-03-15T13:02:01.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.035+0000 7f800aa6b640 1 -- 172.21.15.2:0/497383758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8004108450 msgr2=0x7f800419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:01.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.035+0000 7f800aa6b640 1 --2- 172.21.15.2:0/497383758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8004108450 0x7f800419e6d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0000c00 tx=0x7f7ff00048f0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.035+0000 7f800aa6b640 1 -- 172.21.15.2:0/497383758 shutdown_connections 2024-03-15T13:02:01.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.035+0000 7f800aa6b640 1 --2- 172.21.15.2:0/497383758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f800410daa0 0x7f80041a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.035+0000 7f800aa6b640 1 --2- 172.21.15.2:0/497383758 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7ff40a1c20 0x7f7ff40a40e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.035+0000 7f800aa6b640 1 --2- 172.21.15.2:0/497383758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8004108450 0x7f800419e6d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.035+0000 7f800aa6b640 1 --2- 172.21.15.2:0/497383758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8004107a90 0x7f800419e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.035+0000 7f800aa6b640 1 -- 172.21.15.2:0/497383758 >> 172.21.15.2:0/497383758 conn(0x7f8004075a30 msgr2=0x7f80040762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:01.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.035+0000 7f800aa6b640 1 -- 172.21.15.2:0/497383758 shutdown_connections 2024-03-15T13:02:01.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.035+0000 7f800aa6b640 1 -- 172.21.15.2:0/497383758 wait complete. 2024-03-15T13:02:01.031 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:00 smithi002 ceph-mon[32060]: pgmap v96: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 356 op/s 2024-03-15T13:02:01.031 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:00 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2216975526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:02:01.032 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:02:01.032 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:02:01.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.153+0000 7f280d48f640 1 -- 172.21.15.2:0/2784447759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2808100dd0 msgr2=0x7f280810d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:01.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.153+0000 7f280d48f640 1 --2- 172.21.15.2:0/2784447759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2808100dd0 0x7f280810d2e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009930 tx=0x7f27f802bc30 comp rx=0 tx=0).stop 2024-03-15T13:02:01.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.153+0000 7f280d48f640 1 -- 172.21.15.2:0/2784447759 shutdown_connections 2024-03-15T13:02:01.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.153+0000 7f280d48f640 1 --2- 172.21.15.2:0/2784447759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2808100dd0 0x7f280810d2e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.153+0000 7f280d48f640 1 --2- 172.21.15.2:0/2784447759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2808100410 0x7f2808100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.153+0000 7f280d48f640 1 --2- 172.21.15.2:0/2784447759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28081064b0 0x7f2808106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.153+0000 7f280d48f640 1 -- 172.21.15.2:0/2784447759 >> 172.21.15.2:0/2784447759 conn(0x7f2808077550 msgr2=0x7f2808077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:01.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.153+0000 7f280d48f640 1 -- 172.21.15.2:0/2784447759 shutdown_connections 2024-03-15T13:02:01.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.153+0000 7f280d48f640 1 -- 172.21.15.2:0/2784447759 wait complete. 2024-03-15T13:02:01.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.153+0000 7f280d48f640 1 Processor -- start 2024-03-15T13:02:01.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f280d48f640 1 -- start start 2024-03-15T13:02:01.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f280d48f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2808100410 0x7f28080fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f280d48f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2808100dd0 0x7f28080fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f280d48f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28081064b0 0x7f28080fde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f280d48f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28081a5ec0 con 0x7f2808100410 2024-03-15T13:02:01.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f280d48f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28081a6030 con 0x7f2808100dd0 2024-03-15T13:02:01.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f280d48f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28081a61f0 con 0x7f28081064b0 2024-03-15T13:02:01.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f2806ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2808100410 0x7f28080fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f2806ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2808100410 0x7f28080fd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34100/0 (socket says 172.21.15.2:34100) 2024-03-15T13:02:01.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f2806ffd640 1 -- 172.21.15.2:0/1734344878 learned_addr learned my addr 172.21.15.2:0/1734344878 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:01.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f28067fc640 1 --2- 172.21.15.2:0/1734344878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2808100dd0 0x7f28080fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f28077fe640 1 --2- 172.21.15.2:0/1734344878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28081064b0 0x7f28080fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f2806ffd640 1 -- 172.21.15.2:0/1734344878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28081064b0 msgr2=0x7f28080fde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:01.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f2806ffd640 1 --2- 172.21.15.2:0/1734344878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28081064b0 0x7f28080fde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f2806ffd640 1 -- 172.21.15.2:0/1734344878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2808100dd0 msgr2=0x7f28080fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:01.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f2806ffd640 1 --2- 172.21.15.2:0/1734344878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2808100dd0 0x7f28080fd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f2806ffd640 1 -- 172.21.15.2:0/1734344878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f8009560 con 0x7f2808100410 2024-03-15T13:02:01.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f28077fe640 1 --2- 172.21.15.2:0/1734344878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28081064b0 0x7f28080fde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:02:01.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f2806ffd640 1 --2- 172.21.15.2:0/1734344878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2808100410 0x7f28080fd390 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f280000eb90 tx=0x7f280000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:01.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.154+0000 7f27fffff640 1 -- 172.21.15.2:0/1734344878 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28000099b0 con 0x7f2808100410 2024-03-15T13:02:01.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.155+0000 7f27fffff640 1 -- 172.21.15.2:0/1734344878 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f280000ee90 con 0x7f2808100410 2024-03-15T13:02:01.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.155+0000 7f280d48f640 1 -- 172.21.15.2:0/1734344878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28081a64d0 con 0x7f2808100410 2024-03-15T13:02:01.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.155+0000 7f27fffff640 1 -- 172.21.15.2:0/1734344878 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800010690 con 0x7f2808100410 2024-03-15T13:02:01.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.155+0000 7f280d48f640 1 -- 172.21.15.2:0/1734344878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28081a6990 con 0x7f2808100410 2024-03-15T13:02:01.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.157+0000 7f27fffff640 1 -- 172.21.15.2:0/1734344878 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f2800004f90 con 0x7f2808100410 2024-03-15T13:02:01.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.157+0000 7f2810075740 1 Processor -- start 2024-03-15T13:02:01.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f2810075740 1 -- start start 2024-03-15T13:02:01.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f2810075740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 0x55735ea06630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f2810075740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea06b70 0x55735ea0f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f2810075740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea0f750 0x55735ea11b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f2810075740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735e9c1960 con 0x55735ea06b70 2024-03-15T13:02:01.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f2810075740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735e975310 con 0x55735ea06250 2024-03-15T13:02:01.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f2810075740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735e972840 con 0x55735ea0f750 2024-03-15T13:02:01.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f2806ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea06b70 0x55735ea0f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f28077fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 0x55735ea06630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f28067fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea0f750 0x55735ea11b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f28077fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 0x55735ea06630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44644/0 (socket says 172.21.15.2:44644) 2024-03-15T13:02:01.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f2806ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea06b70 0x55735ea0f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34114/0 (socket says 172.21.15.2:34114) 2024-03-15T13:02:01.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f28077fe640 1 -- 172.21.15.2:0/2828220120 learned_addr learned my addr 172.21.15.2:0/2828220120 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:01.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f27fffff640 1 --2- 172.21.15.2:0/1734344878 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f27f40a1ba0 0x7f27f40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f28067fc640 1 -- 172.21.15.2:0/2828220120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 msgr2=0x55735ea06630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:01.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f28067fc640 1 --2- 172.21.15.2:0/2828220120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 0x55735ea06630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f28067fc640 1 -- 172.21.15.2:0/2828220120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea06b70 msgr2=0x55735ea0f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:01.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f28067fc640 1 --2- 172.21.15.2:0/2828220120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea06b70 0x55735ea0f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f28067fc640 1 -- 172.21.15.2:0/2828220120 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55735e930060 con 0x55735ea0f750 2024-03-15T13:02:01.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f2806ffd640 1 --2- 172.21.15.2:0/2828220120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea06b70 0x55735ea0f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:01.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f28077fe640 1 --2- 172.21.15.2:0/1734344878 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f27f40a1ba0 0x7f27f40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.158+0000 7f27fffff640 1 -- 172.21.15.2:0/1734344878 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f2800014070 con 0x7f2808100410 2024-03-15T13:02:01.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.159+0000 7f28067fc640 1 --2- 172.21.15.2:0/2828220120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea0f750 0x55735ea11b40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f27f0005d00 tx=0x7f27f00107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d3078fdbdef6034 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:01.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.159+0000 7f28077fe640 1 --2- 172.21.15.2:0/1734344878 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f27f40a1ba0 0x7f27f40a4060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f27f802c7b0 tx=0x7f27f8009260 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:01.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.159+0000 7f27fd7fa640 1 -- 172.21.15.2:0/2828220120 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f001c480 con 0x55735ea0f750 2024-03-15T13:02:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.159+0000 7f27fd7fa640 1 -- 172.21.15.2:0/2828220120 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f27f0015070 con 0x55735ea0f750 2024-03-15T13:02:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.159+0000 7f27fd7fa640 1 -- 172.21.15.2:0/2828220120 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f00119b0 con 0x55735ea0f750 2024-03-15T13:02:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.159+0000 7f2810075740 1 -- 172.21.15.2:0/2828220120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea0f750 msgr2=0x55735ea11b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.159+0000 7f2810075740 1 --2- 172.21.15.2:0/2828220120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea0f750 0x55735ea11b40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f27f0005d00 tx=0x7f27f00107a0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 1 -- 172.21.15.2:0/2828220120 shutdown_connections 2024-03-15T13:02:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 1 --2- 172.21.15.2:0/2828220120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea0f750 0x55735ea11b40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 1 --2- 172.21.15.2:0/2828220120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea06b70 0x55735ea0f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 1 --2- 172.21.15.2:0/2828220120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 0x55735ea06630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 1 -- 172.21.15.2:0/2828220120 >> 172.21.15.2:0/2828220120 conn(0x55735ea00980 msgr2=0x55735ea00d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 1 -- 172.21.15.2:0/2828220120 shutdown_connections 2024-03-15T13:02:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 1 -- 172.21.15.2:0/2828220120 wait complete. 2024-03-15T13:02:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 1 -- start start 2024-03-15T13:02:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 1 Processor -- start 2024-03-15T13:02:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 0x55735ea0f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06b70 0x55735ea0c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea0f750 0x55735ea0cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735e96e020 con 0x55735ea0f750 2024-03-15T13:02:01.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735e641430 con 0x55735ea06250 2024-03-15T13:02:01.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735e5fde00 con 0x55735ea06b70 2024-03-15T13:02:01.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2810075740 10 client.0 _ll_register_callbacks cb 0x55735e915cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:01.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f28077fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea0f750 0x55735ea0cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f28077fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea0f750 0x55735ea0cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34120/0 (socket says 172.21.15.2:34120) 2024-03-15T13:02:01.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f28077fe640 1 -- 172.21.15.2:0/929003631 learned_addr learned my addr 172.21.15.2:0/929003631 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:01.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f28067fc640 1 --2- 172.21.15.2:0/929003631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06b70 0x55735ea0c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.160+0000 7f2806ffd640 1 --2- 172.21.15.2:0/929003631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 0x55735ea0f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.161+0000 7f28077fe640 1 -- 172.21.15.2:0/929003631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06b70 msgr2=0x55735ea0c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:01.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.161+0000 7f28077fe640 1 --2- 172.21.15.2:0/929003631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06b70 0x55735ea0c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.161+0000 7f28077fe640 1 -- 172.21.15.2:0/929003631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 msgr2=0x55735ea0f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:01.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.161+0000 7f28077fe640 1 --2- 172.21.15.2:0/929003631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 0x55735ea0f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.161+0000 7f28077fe640 1 -- 172.21.15.2:0/929003631 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f000b8c0 con 0x55735ea0f750 2024-03-15T13:02:01.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.161+0000 7f28067fc640 1 --2- 172.21.15.2:0/929003631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06b70 0x55735ea0c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:01.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.161+0000 7f28077fe640 1 --2- 172.21.15.2:0/929003631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea0f750 0x55735ea0cae0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009230 tx=0x7f27f80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15458a16bc04e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:01.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.161+0000 7f27ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:01.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.161+0000 7f27ee7fc640 1 -- 172.21.15.2:0/929003631 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8003a00 con 0x55735ea0f750 2024-03-15T13:02:01.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.161+0000 7f27ee7fc640 1 -- 172.21.15.2:0/929003631 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f27f8003b60 con 0x55735ea0f750 2024-03-15T13:02:01.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.161+0000 7f2810075740 1 -- 172.21.15.2:0/929003631 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55735e66d7e0 con 0x55735ea0f750 2024-03-15T13:02:01.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.161+0000 7f27ee7fc640 1 -- 172.21.15.2:0/929003631 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27f8003d20 con 0x55735ea0f750 2024-03-15T13:02:01.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.161+0000 7f2810075740 10 client.16683 fetch_fsmap learned FSMap version 37 2024-03-15T13:02:01.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.161+0000 7f2810075740 1 -- 172.21.15.2:0/929003631 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55735e96c6a0 con 0x55735ea0f750 2024-03-15T13:02:01.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.161+0000 7f27ee7fc640 1 -- 172.21.15.2:0/929003631 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55735e66d7e0 con 0x55735ea0f750 2024-03-15T13:02:01.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.162+0000 7f27ee7fc640 1 -- 172.21.15.2:0/929003631 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 37) v1 ==== 7986+0+0 (secure 0 0 0) 0x7f27f8048050 con 0x55735ea0f750 2024-03-15T13:02:01.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.162+0000 7f2810075740 10 client.16683 fetch_fsmap finished waiting for FSMap version 37 2024-03-15T13:02:01.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.162+0000 7f2810075740 10 client.16683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:02:01.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.162+0000 7f2810075740 20 client.16683 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:01.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.162+0000 7f2810075740 1 --2- 172.21.15.2:0/929003631 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55735ea1f7d0 0x55735ea21bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.162+0000 7f2810075740 4 client.16683 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:02:01.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.162+0000 7f2810075740 1 -- 172.21.15.2:0/929003631 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55735e6aa720 con 0x55735ea1f7d0 2024-03-15T13:02:01.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.162+0000 7f2806ffd640 1 --2- 172.21.15.2:0/929003631 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55735ea1f7d0 0x55735ea21bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.162+0000 7f27ee7fc640 10 client.16683 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:01.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.162+0000 7f2806ffd640 1 --2- 172.21.15.2:0/929003631 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55735ea1f7d0 0x55735ea21bf0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4aac15efa52388f server_cookie=50ddb5cfd42157ac in_seq=0 out_seq=0 2024-03-15T13:02:01.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.164+0000 7f27ee7fc640 1 -- 172.21.15.2:0/929003631 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55735e6aa720 con 0x55735ea1f7d0 2024-03-15T13:02:01.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.164+0000 7f27ee7fc640 10 client.16683 handle_command_reply: tid=0 2024-03-15T13:02:01.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.164+0000 7f2810075740 1 client.16683 shutdown 2024-03-15T13:02:01.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.164+0000 7f2810075740 1 -- 172.21.15.2:0/929003631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea0f750 msgr2=0x55735ea0cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:01.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.164+0000 7f2810075740 1 --2- 172.21.15.2:0/929003631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea0f750 0x55735ea0cae0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009230 tx=0x7f27f80058b0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.164+0000 7f2810075740 1 -- 172.21.15.2:0/929003631 shutdown_connections 2024-03-15T13:02:01.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.164+0000 7f2810075740 1 --2- 172.21.15.2:0/929003631 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55735ea1f7d0 0x55735ea21bf0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.164+0000 7f2810075740 1 --2- 172.21.15.2:0/929003631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea0f750 0x55735ea0cae0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.164+0000 7f27ee7fc640 0 client.16683 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:01.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.164+0000 7f2810075740 1 --2- 172.21.15.2:0/929003631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06b70 0x55735ea0c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.164+0000 7f2810075740 1 --2- 172.21.15.2:0/929003631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 0x55735ea0f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.164+0000 7f2810075740 1 -- 172.21.15.2:0/929003631 >> 172.21.15.2:0/929003631 conn(0x55735ea00980 msgr2=0x55735ea026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:01.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.164+0000 7f2810075740 1 -- 172.21.15.2:0/929003631 shutdown_connections 2024-03-15T13:02:01.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.164+0000 7f2810075740 1 -- 172.21.15.2:0/929003631 wait complete. 2024-03-15T13:02:01.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.164+0000 7f2810075740 20 client.16683 trim_cache size 0 max 16384 2024-03-15T13:02:01.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.176+0000 7f2810075740 1 Processor -- start 2024-03-15T13:02:01.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.176+0000 7f2810075740 1 -- start start 2024-03-15T13:02:01.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.176+0000 7f2810075740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06250 0x55735ea0ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f2810075740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea06b70 0x55735e9ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f2810075740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea0f750 0x55735ea0ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f2810075740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735ea1bdb0 con 0x55735ea06b70 2024-03-15T13:02:01.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f2810075740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735e8f3130 con 0x55735ea0f750 2024-03-15T13:02:01.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f2810075740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735e9ff830 con 0x55735ea06250 2024-03-15T13:02:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f28067fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea06b70 0x55735e9ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f28067fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea06b70 0x55735e9ff2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34124/0 (socket says 172.21.15.2:34124) 2024-03-15T13:02:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f28067fc640 1 -- 172.21.15.2:0/357567706 learned_addr learned my addr 172.21.15.2:0/357567706 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f2806ffd640 1 --2- 172.21.15.2:0/357567706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06250 0x55735ea0ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f28077fe640 1 --2- 172.21.15.2:0/357567706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea0f750 0x55735ea0ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f28067fc640 1 -- 172.21.15.2:0/357567706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06250 msgr2=0x55735ea0ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f28067fc640 1 --2- 172.21.15.2:0/357567706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06250 0x55735ea0ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f28067fc640 1 -- 172.21.15.2:0/357567706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea0f750 msgr2=0x55735ea0ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f28067fc640 1 --2- 172.21.15.2:0/357567706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea0f750 0x55735ea0ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f28067fc640 1 -- 172.21.15.2:0/357567706 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55735e908c20 con 0x55735ea06b70 2024-03-15T13:02:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f28067fc640 1 --2- 172.21.15.2:0/357567706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea06b70 0x55735e9ff2f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x55735ea01b10 tx=0x7f27f000b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521aa73f5f4b31ac server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f27fcff9640 1 -- 172.21.15.2:0/357567706 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0021050 con 0x55735ea06b70 2024-03-15T13:02:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f27fcff9640 1 -- 172.21.15.2:0/357567706 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f27f0022040 con 0x55735ea06b70 2024-03-15T13:02:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.177+0000 7f27fcff9640 1 -- 172.21.15.2:0/357567706 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0011680 con 0x55735ea06b70 2024-03-15T13:02:01.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.178+0000 7f2810075740 1 -- 172.21.15.2:0/357567706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea06b70 msgr2=0x55735e9ff2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:01.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.178+0000 7f2810075740 1 --2- 172.21.15.2:0/357567706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea06b70 0x55735e9ff2f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x55735ea01b10 tx=0x7f27f000b830 comp rx=0 tx=0).stop 2024-03-15T13:02:01.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.178+0000 7f2810075740 1 -- 172.21.15.2:0/357567706 shutdown_connections 2024-03-15T13:02:01.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.178+0000 7f2810075740 1 --2- 172.21.15.2:0/357567706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea0f750 0x55735ea0ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.178+0000 7f2810075740 1 --2- 172.21.15.2:0/357567706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea06b70 0x55735e9ff2f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.178+0000 7f2810075740 1 --2- 172.21.15.2:0/357567706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06250 0x55735ea0ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.178+0000 7f2810075740 1 -- 172.21.15.2:0/357567706 >> 172.21.15.2:0/357567706 conn(0x55735ea00980 msgr2=0x55735ea02640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:01.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.178+0000 7f2810075740 1 -- 172.21.15.2:0/357567706 shutdown_connections 2024-03-15T13:02:01.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.178+0000 7f2810075740 1 -- 172.21.15.2:0/357567706 wait complete. 2024-03-15T13:02:01.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.178+0000 7f2810075740 1 -- start start 2024-03-15T13:02:01.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.178+0000 7f2810075740 1 Processor -- start 2024-03-15T13:02:01.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f2810075740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 0x55735ea16f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f2810075740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06b70 0x55735ea174d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f2810075740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea0f750 0x55735ea133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f2810075740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735ea13ae0 con 0x55735ea0f750 2024-03-15T13:02:01.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f2810075740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735ea13c50 con 0x55735ea06250 2024-03-15T13:02:01.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f2810075740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735ea13e10 con 0x55735ea06b70 2024-03-15T13:02:01.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f2810075740 10 client.0 _ll_register_callbacks cb 0x7f27f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:01.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f28077fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea0f750 0x55735ea133e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f28077fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea0f750 0x55735ea133e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34140/0 (socket says 172.21.15.2:34140) 2024-03-15T13:02:01.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f28077fe640 1 -- 172.21.15.2:0/1977064300 learned_addr learned my addr 172.21.15.2:0/1977064300 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:01.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f2806ffd640 1 --2- 172.21.15.2:0/1977064300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 0x55735ea16f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f28067fc640 1 --2- 172.21.15.2:0/1977064300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06b70 0x55735ea174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f28077fe640 1 -- 172.21.15.2:0/1977064300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06b70 msgr2=0x55735ea174d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:01.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f28077fe640 1 --2- 172.21.15.2:0/1977064300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06b70 0x55735ea174d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f28077fe640 1 -- 172.21.15.2:0/1977064300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 msgr2=0x55735ea16f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:01.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f28077fe640 1 --2- 172.21.15.2:0/1977064300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 0x55735ea16f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f28077fe640 1 -- 172.21.15.2:0/1977064300 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55735e930060 con 0x55735ea0f750 2024-03-15T13:02:01.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f28077fe640 1 --2- 172.21.15.2:0/1977064300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea0f750 0x55735ea133e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f27f804b640 tx=0x7f27f80098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6929fb3ce1e0e7f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:01.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f27ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:01.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.179+0000 7f27ee7fc640 1 -- 172.21.15.2:0/1977064300 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f804bde0 con 0x55735ea0f750 2024-03-15T13:02:01.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.180+0000 7f27ee7fc640 1 -- 172.21.15.2:0/1977064300 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f27f8003800 con 0x55735ea0f750 2024-03-15T13:02:01.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.180+0000 7f2810075740 1 -- 172.21.15.2:0/1977064300 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55735e9fc0f0 con 0x55735ea0f750 2024-03-15T13:02:01.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.180+0000 7f27ee7fc640 1 -- 172.21.15.2:0/1977064300 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27f80039c0 con 0x55735ea0f750 2024-03-15T13:02:01.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.180+0000 7f2810075740 10 client.16689 fetch_fsmap learned FSMap version 37 2024-03-15T13:02:01.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.180+0000 7f2810075740 1 -- 172.21.15.2:0/1977064300 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55735e92e8c0 con 0x55735ea0f750 2024-03-15T13:02:01.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.180+0000 7f27ee7fc640 1 -- 172.21.15.2:0/1977064300 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55735e9fc0f0 con 0x55735ea0f750 2024-03-15T13:02:01.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.180+0000 7f27ee7fc640 1 -- 172.21.15.2:0/1977064300 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 37) v1 ==== 7986+0+0 (secure 0 0 0) 0x7f27f8048040 con 0x55735ea0f750 2024-03-15T13:02:01.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.180+0000 7f2810075740 10 client.16689 fetch_fsmap finished waiting for FSMap version 37 2024-03-15T13:02:01.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.180+0000 7f2810075740 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:02:01.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.180+0000 7f2810075740 20 client.16689 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:01.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.180+0000 7f2810075740 1 --2- 172.21.15.2:0/1977064300 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55735ea7c090 0x55735ea7e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:01.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.180+0000 7f2810075740 4 client.16689 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:02:01.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.180+0000 7f2810075740 1 -- 172.21.15.2:0/1977064300 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f27f0015070 con 0x55735ea7c090 2024-03-15T13:02:01.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.180+0000 7f2806ffd640 1 --2- 172.21.15.2:0/1977064300 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55735ea7c090 0x55735ea7e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:01.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.181+0000 7f2806ffd640 1 --2- 172.21.15.2:0/1977064300 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55735ea7c090 0x55735ea7e4b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75065ccd15557a8a server_cookie=3850898b7daae1e3 in_seq=0 out_seq=0 2024-03-15T13:02:01.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.181+0000 7f27ee7fc640 10 client.16689 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:01.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.182+0000 7f27ee7fc640 1 -- 172.21.15.2:0/1977064300 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f27f0015070 con 0x55735ea7c090 2024-03-15T13:02:01.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.182+0000 7f27ee7fc640 10 client.16689 handle_command_reply: tid=0 2024-03-15T13:02:01.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.182+0000 7f2810075740 1 client.16689 shutdown 2024-03-15T13:02:01.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f2810075740 1 -- 172.21.15.2:0/1977064300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea0f750 msgr2=0x55735ea133e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:01.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f2810075740 1 --2- 172.21.15.2:0/1977064300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea0f750 0x55735ea133e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f27f804b640 tx=0x7f27f80098c0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f2810075740 1 -- 172.21.15.2:0/1977064300 shutdown_connections 2024-03-15T13:02:01.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f2810075740 1 --2- 172.21.15.2:0/1977064300 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55735ea7c090 0x55735ea7e4b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f2810075740 1 --2- 172.21.15.2:0/1977064300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55735ea0f750 0x55735ea133e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f2810075740 1 --2- 172.21.15.2:0/1977064300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55735ea06b70 0x55735ea174d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f27ee7fc640 0 client.16689 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:01.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f2810075740 1 --2- 172.21.15.2:0/1977064300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55735ea06250 0x55735ea16f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f2810075740 1 -- 172.21.15.2:0/1977064300 >> 172.21.15.2:0/1977064300 conn(0x55735ea00980 msgr2=0x55735ea06f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:01.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f2810075740 1 -- 172.21.15.2:0/1977064300 shutdown_connections 2024-03-15T13:02:01.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f2810075740 1 -- 172.21.15.2:0/1977064300 wait complete. 2024-03-15T13:02:01.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f2810075740 20 client.16689 trim_cache size 0 max 16384 2024-03-15T13:02:01.174 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:02:01.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f280d48f640 1 -- 172.21.15.2:0/1734344878 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f27f40a1ba0 msgr2=0x7f27f40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:01.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f280d48f640 1 --2- 172.21.15.2:0/1734344878 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f27f40a1ba0 0x7f27f40a4060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f27f802c7b0 tx=0x7f27f8009260 comp rx=0 tx=0).stop 2024-03-15T13:02:01.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f280d48f640 1 -- 172.21.15.2:0/1734344878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2808100410 msgr2=0x7f28080fd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:01.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f280d48f640 1 --2- 172.21.15.2:0/1734344878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2808100410 0x7f28080fd390 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f280000eb90 tx=0x7f280000c6a0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f280d48f640 1 -- 172.21.15.2:0/1734344878 shutdown_connections 2024-03-15T13:02:01.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f280d48f640 1 --2- 172.21.15.2:0/1734344878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28081064b0 0x7f28080fde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f280d48f640 1 --2- 172.21.15.2:0/1734344878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2808100dd0 0x7f28080fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f280d48f640 1 --2- 172.21.15.2:0/1734344878 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f27f40a1ba0 0x7f27f40a4060 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f280d48f640 1 --2- 172.21.15.2:0/1734344878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2808100410 0x7f28080fd390 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:01.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.183+0000 7f280d48f640 1 -- 172.21.15.2:0/1734344878 >> 172.21.15.2:0/1734344878 conn(0x7f2808077550 msgr2=0x7f2808069dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:01.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.184+0000 7f280d48f640 1 -- 172.21.15.2:0/1734344878 shutdown_connections 2024-03-15T13:02:01.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:01.184+0000 7f280d48f640 1 -- 172.21.15.2:0/1734344878 wait complete. 2024-03-15T13:02:01.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:00 smithi184 ceph-mon[35993]: pgmap v96: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 356 op/s 2024-03-15T13:02:01.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:00 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2216975526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:02:01.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:00 smithi149 ceph-mon[35927]: pgmap v96: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 356 op/s 2024-03-15T13:02:01.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:00 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2216975526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:02:01.375 INFO:tasks.workunit.client.0.smithi002.stderr:+ sync 2024-03-15T13:02:02.181 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:02:02.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:02:02.181 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:02:02.205 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:01 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:02:02.205 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:01 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:02:02.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.305+0000 7f44b958c640 1 -- 172.21.15.2:0/1442850796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44b4069a60 msgr2=0x7f44b4069ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.305+0000 7f44b958c640 1 --2- 172.21.15.2:0/1442850796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44b4069a60 0x7f44b4069ee0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009900 tx=0x7f44a802bc30 comp rx=0 tx=0).stop 2024-03-15T13:02:02.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.305+0000 7f44b958c640 1 -- 172.21.15.2:0/1442850796 shutdown_connections 2024-03-15T13:02:02.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.305+0000 7f44b958c640 1 --2- 172.21.15.2:0/1442850796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44b406a420 0x7f44b410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.305+0000 7f44b958c640 1 --2- 172.21.15.2:0/1442850796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44b4069a60 0x7f44b4069ee0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.305+0000 7f44b958c640 1 --2- 172.21.15.2:0/1442850796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44b4107560 0x7f44b4107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.305+0000 7f44b958c640 1 -- 172.21.15.2:0/1442850796 >> 172.21.15.2:0/1442850796 conn(0x7f44b40773d0 msgr2=0x7f44b40777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:02.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.305+0000 7f44b958c640 1 -- 172.21.15.2:0/1442850796 shutdown_connections 2024-03-15T13:02:02.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.305+0000 7f44b958c640 1 -- 172.21.15.2:0/1442850796 wait complete. 2024-03-15T13:02:02.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.306+0000 7f44b958c640 1 Processor -- start 2024-03-15T13:02:02.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.306+0000 7f44b958c640 1 -- start start 2024-03-15T13:02:02.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.306+0000 7f44b958c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44b4069a60 0x7f44b419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.306+0000 7f44b958c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44b406a420 0x7f44b419dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.306+0000 7f44b958c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44b4107560 0x7f44b41978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.306+0000 7f44b958c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b4197f20 con 0x7f44b4107560 2024-03-15T13:02:02.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.306+0000 7f44b958c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b4198090 con 0x7f44b406a420 2024-03-15T13:02:02.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.306+0000 7f44b958c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b4198250 con 0x7f44b4069a60 2024-03-15T13:02:02.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.306+0000 7f44b37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44b4107560 0x7f44b41978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.306+0000 7f44b27fc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44b406a420 0x7f44b419dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.306+0000 7f44b37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44b4107560 0x7f44b41978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34156/0 (socket says 172.21.15.2:34156) 2024-03-15T13:02:02.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.306+0000 7f44b37fe640 1 -- 172.21.15.2:0/74919128 learned_addr learned my addr 172.21.15.2:0/74919128 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:02.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.307+0000 7f44b2ffd640 1 --2- 172.21.15.2:0/74919128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44b4069a60 0x7f44b419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.307+0000 7f44b27fc640 1 -- 172.21.15.2:0/74919128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44b4069a60 msgr2=0x7f44b419d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.307+0000 7f44b27fc640 1 --2- 172.21.15.2:0/74919128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44b4069a60 0x7f44b419d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.307+0000 7f44b27fc640 1 -- 172.21.15.2:0/74919128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44b4107560 msgr2=0x7f44b41978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.307+0000 7f44b27fc640 1 --2- 172.21.15.2:0/74919128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44b4107560 0x7f44b41978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.307+0000 7f44b27fc640 1 -- 172.21.15.2:0/74919128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a8009560 con 0x7f44b406a420 2024-03-15T13:02:02.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.307+0000 7f44b37fe640 1 --2- 172.21.15.2:0/74919128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44b4107560 0x7f44b41978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:02.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.307+0000 7f44b27fc640 1 --2- 172.21.15.2:0/74919128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44b406a420 0x7f44b419dc10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009230 tx=0x7f44a802ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:02.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.307+0000 7f449bfff640 1 -- 172.21.15.2:0/74919128 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a802cc30 con 0x7f44b406a420 2024-03-15T13:02:02.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.307+0000 7f44b958c640 1 -- 172.21.15.2:0/74919128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b41984d0 con 0x7f44b406a420 2024-03-15T13:02:02.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.307+0000 7f449bfff640 1 -- 172.21.15.2:0/74919128 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f44a8005430 con 0x7f44b406a420 2024-03-15T13:02:02.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.307+0000 7f449bfff640 1 -- 172.21.15.2:0/74919128 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8003b50 con 0x7f44b406a420 2024-03-15T13:02:02.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.307+0000 7f44b958c640 1 -- 172.21.15.2:0/74919128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b41a4770 con 0x7f44b406a420 2024-03-15T13:02:02.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.310+0000 7f44bc16c740 1 Processor -- start 2024-03-15T13:02:02.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.310+0000 7f44bc16c740 1 -- start start 2024-03-15T13:02:02.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f449bfff640 1 -- 172.21.15.2:0/74919128 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f44a8003cb0 con 0x7f44b406a420 2024-03-15T13:02:02.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44bc16c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44bc16c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8a6d30 0x55821b8af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44bc16c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8af8b0 0x55821b8b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44bc16c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821b814f90 con 0x55821b8a6380 2024-03-15T13:02:02.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44bc16c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821b8124c0 con 0x55821b8af8b0 2024-03-15T13:02:02.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44bc16c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821b80dca0 con 0x55821b8a6d30 2024-03-15T13:02:02.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44b37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44b27fc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8af8b0 0x55821b8b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44b37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8a6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34164/0 (socket says 172.21.15.2:34164) 2024-03-15T13:02:02.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44b37fe640 1 -- 172.21.15.2:0/612097641 learned_addr learned my addr 172.21.15.2:0/612097641 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:02.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44b2ffd640 1 --2- 172.21.15.2:0/612097641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8a6d30 0x55821b8af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44b37fe640 1 -- 172.21.15.2:0/612097641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8a6d30 msgr2=0x55821b8af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44b37fe640 1 --2- 172.21.15.2:0/612097641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8a6d30 0x55821b8af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44b37fe640 1 -- 172.21.15.2:0/612097641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8af8b0 msgr2=0x55821b8b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44b37fe640 1 --2- 172.21.15.2:0/612097641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8af8b0 0x55821b8b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44b37fe640 1 -- 172.21.15.2:0/612097641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55821b7d00b0 con 0x55821b8a6380 2024-03-15T13:02:02.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f44b37fe640 1 --2- 172.21.15.2:0/612097641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8a6760 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0118b0 tx=0x7f44ac011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53b7e34b4ca0f5a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:02.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.311+0000 7f449bfff640 1 --2- 172.21.15.2:0/74919128 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f449c0a1ba0 0x7f449c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.312+0000 7f44997fa640 1 -- 172.21.15.2:0/612097641 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac011310 con 0x55821b8a6380 2024-03-15T13:02:02.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.312+0000 7f44b2ffd640 1 --2- 172.21.15.2:0/74919128 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f449c0a1ba0 0x7f449c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.312+0000 7f44997fa640 1 -- 172.21.15.2:0/612097641 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f44ac00fe90 con 0x55821b8a6380 2024-03-15T13:02:02.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.312+0000 7f449bfff640 1 -- 172.21.15.2:0/74919128 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f44a80ef9b0 con 0x7f44b406a420 2024-03-15T13:02:02.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.312+0000 7f44997fa640 1 -- 172.21.15.2:0/612097641 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac010430 con 0x55821b8a6380 2024-03-15T13:02:02.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.313+0000 7f44b2ffd640 1 --2- 172.21.15.2:0/74919128 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f449c0a1ba0 0x7f449c0a4060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55821b8a1ac0 tx=0x7f44a4005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:02.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.313+0000 7f44bc16c740 1 -- 172.21.15.2:0/612097641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 msgr2=0x55821b8a6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.313+0000 7f44bc16c740 1 --2- 172.21.15.2:0/612097641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8a6760 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0118b0 tx=0x7f44ac011d70 comp rx=0 tx=0).stop 2024-03-15T13:02:02.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.313+0000 7f44bc16c740 1 -- 172.21.15.2:0/612097641 shutdown_connections 2024-03-15T13:02:02.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.313+0000 7f44bc16c740 1 --2- 172.21.15.2:0/612097641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8af8b0 0x55821b8b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.313+0000 7f44bc16c740 1 --2- 172.21.15.2:0/612097641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8a6d30 0x55821b8af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.313+0000 7f44bc16c740 1 --2- 172.21.15.2:0/612097641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8a6760 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.313+0000 7f44bc16c740 1 -- 172.21.15.2:0/612097641 >> 172.21.15.2:0/612097641 conn(0x55821b8a0840 msgr2=0x55821b8a0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:02.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.313+0000 7f44bc16c740 1 -- 172.21.15.2:0/612097641 shutdown_connections 2024-03-15T13:02:02.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.313+0000 7f44bc16c740 1 -- 172.21.15.2:0/612097641 wait complete. 2024-03-15T13:02:02.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.313+0000 7f44bc16c740 1 -- start start 2024-03-15T13:02:02.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.313+0000 7f44bc16c740 1 Processor -- start 2024-03-15T13:02:02.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44bc16c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44bc16c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8a6d30 0x55821b8acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44bc16c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8af8b0 0x55821b8ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44bc16c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821b7b2f40 con 0x55821b8a6380 2024-03-15T13:02:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44bc16c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821b4e14b0 con 0x55821b8a6d30 2024-03-15T13:02:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44bc16c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821b793670 con 0x55821b8af8b0 2024-03-15T13:02:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44bc16c740 10 client.0 _ll_register_callbacks cb 0x55821b800f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44b2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44b2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34174/0 (socket says 172.21.15.2:34174) 2024-03-15T13:02:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44b2ffd640 1 -- 172.21.15.2:0/3621884844 learned_addr learned my addr 172.21.15.2:0/3621884844 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44b27fc640 1 --2- 172.21.15.2:0/3621884844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8a6d30 0x55821b8acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44b37fe640 1 --2- 172.21.15.2:0/3621884844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8af8b0 0x55821b8ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44b2ffd640 1 -- 172.21.15.2:0/3621884844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8af8b0 msgr2=0x55821b8ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44b2ffd640 1 --2- 172.21.15.2:0/3621884844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8af8b0 0x55821b8ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44b2ffd640 1 -- 172.21.15.2:0/3621884844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8a6d30 msgr2=0x55821b8acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44b2ffd640 1 --2- 172.21.15.2:0/3621884844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8a6d30 0x55821b8acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44b2ffd640 1 -- 172.21.15.2:0/3621884844 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ac011560 con 0x55821b8a6380 2024-03-15T13:02:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44b27fc640 1 --2- 172.21.15.2:0/3621884844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8a6d30 0x55821b8acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:02.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f44b2ffd640 1 --2- 172.21.15.2:0/3621884844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8ac870 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f44a4004620 tx=0x7f44a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c344d028a86c3dfe server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:02.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.314+0000 7f4496ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:02.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.315+0000 7f4496ff5640 1 -- 172.21.15.2:0/3621884844 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4017070 con 0x55821b8a6380 2024-03-15T13:02:02.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.315+0000 7f4496ff5640 1 -- 172.21.15.2:0/3621884844 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f44a4005020 con 0x55821b8a6380 2024-03-15T13:02:02.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.315+0000 7f44bc16c740 1 -- 172.21.15.2:0/3621884844 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55821b7e7f40 con 0x55821b8a6380 2024-03-15T13:02:02.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.315+0000 7f4496ff5640 1 -- 172.21.15.2:0/3621884844 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44a40051e0 con 0x55821b8a6380 2024-03-15T13:02:02.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.315+0000 7f44bc16c740 10 client.16701 fetch_fsmap learned FSMap version 37 2024-03-15T13:02:02.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.315+0000 7f44bc16c740 1 -- 172.21.15.2:0/3621884844 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55821b80c320 con 0x55821b8a6380 2024-03-15T13:02:02.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.315+0000 7f4496ff5640 1 -- 172.21.15.2:0/3621884844 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55821b7e7f40 con 0x55821b8a6380 2024-03-15T13:02:02.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.315+0000 7f4496ff5640 1 -- 172.21.15.2:0/3621884844 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 37) v1 ==== 7986+0+0 (secure 0 0 0) 0x7f44a401f6d0 con 0x55821b8a6380 2024-03-15T13:02:02.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.315+0000 7f44bc16c740 10 client.16701 fetch_fsmap finished waiting for FSMap version 37 2024-03-15T13:02:02.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.315+0000 7f44bc16c740 10 client.16701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:02:02.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.315+0000 7f44bc16c740 20 client.16701 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:02.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.315+0000 7f44bc16c740 1 --2- 172.21.15.2:0/3621884844 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55821b8bff80 0x55821b8c23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.315+0000 7f44bc16c740 4 client.16701 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:02:02.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.315+0000 7f44bc16c740 1 -- 172.21.15.2:0/3621884844 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55821b804310 con 0x55821b8bff80 2024-03-15T13:02:02.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.316+0000 7f44b27fc640 1 --2- 172.21.15.2:0/3621884844 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55821b8bff80 0x55821b8c23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.316+0000 7f44b27fc640 1 --2- 172.21.15.2:0/3621884844 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55821b8bff80 0x55821b8c23a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d73c5f5e1c5d3a0 server_cookie=46abb4704af8c776 in_seq=0 out_seq=0 2024-03-15T13:02:02.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.316+0000 7f4496ff5640 10 client.16701 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:02.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.317+0000 7f4496ff5640 1 -- 172.21.15.2:0/3621884844 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55821b804310 con 0x55821b8bff80 2024-03-15T13:02:02.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.317+0000 7f4496ff5640 10 client.16701 handle_command_reply: tid=0 2024-03-15T13:02:02.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.317+0000 7f44bc16c740 1 client.16701 shutdown 2024-03-15T13:02:02.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.317+0000 7f44bc16c740 1 -- 172.21.15.2:0/3621884844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 msgr2=0x55821b8ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.317+0000 7f44bc16c740 1 --2- 172.21.15.2:0/3621884844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8ac870 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f44a4004620 tx=0x7f44a4004730 comp rx=0 tx=0).stop 2024-03-15T13:02:02.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.317+0000 7f44bc16c740 1 -- 172.21.15.2:0/3621884844 shutdown_connections 2024-03-15T13:02:02.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.317+0000 7f44bc16c740 1 --2- 172.21.15.2:0/3621884844 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55821b8bff80 0x55821b8c23a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.317+0000 7f44bc16c740 1 --2- 172.21.15.2:0/3621884844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8af8b0 0x55821b8ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.317+0000 7f44bc16c740 1 --2- 172.21.15.2:0/3621884844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8a6d30 0x55821b8acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.317+0000 7f4496ff5640 0 client.16701 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:02.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.317+0000 7f44bc16c740 1 --2- 172.21.15.2:0/3621884844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8ac870 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.317+0000 7f44bc16c740 1 -- 172.21.15.2:0/3621884844 >> 172.21.15.2:0/3621884844 conn(0x55821b8a0840 msgr2=0x55821b8a27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:02.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.317+0000 7f44bc16c740 1 -- 172.21.15.2:0/3621884844 shutdown_connections 2024-03-15T13:02:02.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.317+0000 7f44bc16c740 1 -- 172.21.15.2:0/3621884844 wait complete. 2024-03-15T13:02:02.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.317+0000 7f44bc16c740 20 client.16701 trim_cache size 0 max 16384 2024-03-15T13:02:02.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:01 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:02:02.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:01 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:02:02.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.334+0000 7f44bc16c740 1 Processor -- start 2024-03-15T13:02:02.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.334+0000 7f44bc16c740 1 -- start start 2024-03-15T13:02:02.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.334+0000 7f44bc16c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8a6380 0x55821b8aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44bc16c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8a6d30 0x55821b8ad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44bc16c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8af8b0 0x55821b8badb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44bc16c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821b8a5440 con 0x55821b8af8b0 2024-03-15T13:02:02.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44bc16c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821b8be6d0 con 0x55821b8a6380 2024-03-15T13:02:02.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44bc16c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821b8a06c0 con 0x55821b8a6d30 2024-03-15T13:02:02.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44b37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8af8b0 0x55821b8badb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44b37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8af8b0 0x55821b8badb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34184/0 (socket says 172.21.15.2:34184) 2024-03-15T13:02:02.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44b2ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8a6380 0x55821b8aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44b37fe640 1 -- 172.21.15.2:0/2656677128 learned_addr learned my addr 172.21.15.2:0/2656677128 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:02.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44b27fc640 1 --2- 172.21.15.2:0/2656677128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8a6d30 0x55821b8ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44b37fe640 1 -- 172.21.15.2:0/2656677128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8a6d30 msgr2=0x55821b8ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44b37fe640 1 --2- 172.21.15.2:0/2656677128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8a6d30 0x55821b8ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44b37fe640 1 -- 172.21.15.2:0/2656677128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8a6380 msgr2=0x55821b8aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44b37fe640 1 --2- 172.21.15.2:0/2656677128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8a6380 0x55821b8aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44b37fe640 1 -- 172.21.15.2:0/2656677128 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55821b7a8c00 con 0x55821b8af8b0 2024-03-15T13:02:02.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44b2ffd640 1 --2- 172.21.15.2:0/2656677128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8a6380 0x55821b8aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:02.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f44b37fe640 1 --2- 172.21.15.2:0/2656677128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8af8b0 0x55821b8badb0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00f7b0 tx=0x7f44ac01abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59dcd071f8a9018 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:02.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.335+0000 7f4498ff9640 1 -- 172.21.15.2:0/2656677128 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac011110 con 0x55821b8af8b0 2024-03-15T13:02:02.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.336+0000 7f4498ff9640 1 -- 172.21.15.2:0/2656677128 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f44ac011270 con 0x55821b8af8b0 2024-03-15T13:02:02.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.336+0000 7f4498ff9640 1 -- 172.21.15.2:0/2656677128 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac004e60 con 0x55821b8af8b0 2024-03-15T13:02:02.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.336+0000 7f44bc16c740 1 -- 172.21.15.2:0/2656677128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8af8b0 msgr2=0x55821b8badb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.336+0000 7f44bc16c740 1 --2- 172.21.15.2:0/2656677128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8af8b0 0x55821b8badb0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00f7b0 tx=0x7f44ac01abe0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.336+0000 7f44bc16c740 1 -- 172.21.15.2:0/2656677128 shutdown_connections 2024-03-15T13:02:02.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.336+0000 7f44bc16c740 1 --2- 172.21.15.2:0/2656677128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8af8b0 0x55821b8badb0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.336+0000 7f44bc16c740 1 --2- 172.21.15.2:0/2656677128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8a6d30 0x55821b8ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.336+0000 7f44bc16c740 1 --2- 172.21.15.2:0/2656677128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8a6380 0x55821b8aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.336+0000 7f44bc16c740 1 -- 172.21.15.2:0/2656677128 >> 172.21.15.2:0/2656677128 conn(0x55821b8a0840 msgr2=0x55821b8a2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:02.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.336+0000 7f44bc16c740 1 -- 172.21.15.2:0/2656677128 shutdown_connections 2024-03-15T13:02:02.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.336+0000 7f44bc16c740 1 -- 172.21.15.2:0/2656677128 wait complete. 2024-03-15T13:02:02.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44bc16c740 1 -- start start 2024-03-15T13:02:02.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44bc16c740 1 Processor -- start 2024-03-15T13:02:02.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44bc16c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8aa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44bc16c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8a6d30 0x55821b8b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44bc16c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8af8b0 0x55821b8b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44bc16c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821b8b5370 con 0x55821b8a6380 2024-03-15T13:02:02.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44bc16c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821b8b54e0 con 0x55821b8af8b0 2024-03-15T13:02:02.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44bc16c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821b8b56a0 con 0x55821b8a6d30 2024-03-15T13:02:02.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44bc16c740 10 client.0 _ll_register_callbacks cb 0x7f44a40044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:02.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44b2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44b2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8aa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34194/0 (socket says 172.21.15.2:34194) 2024-03-15T13:02:02.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44b2ffd640 1 -- 172.21.15.2:0/3778293227 learned_addr learned my addr 172.21.15.2:0/3778293227 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:02.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44b27fc640 1 --2- 172.21.15.2:0/3778293227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8a6d30 0x55821b8b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44b37fe640 1 --2- 172.21.15.2:0/3778293227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8af8b0 0x55821b8b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44b2ffd640 1 -- 172.21.15.2:0/3778293227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8a6d30 msgr2=0x55821b8b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44b2ffd640 1 --2- 172.21.15.2:0/3778293227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8a6d30 0x55821b8b48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44b2ffd640 1 -- 172.21.15.2:0/3778293227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8af8b0 msgr2=0x55821b8b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44b2ffd640 1 --2- 172.21.15.2:0/3778293227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8af8b0 0x55821b8b4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44b2ffd640 1 -- 172.21.15.2:0/3778293227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55821b80c320 con 0x55821b8a6380 2024-03-15T13:02:02.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f44b2ffd640 1 --2- 172.21.15.2:0/3778293227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8aa450 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f44a401ffd0 tx=0x7f44a401f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6108efd79f8cbe7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:02.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.337+0000 7f4496ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:02.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.338+0000 7f4496ff5640 1 -- 172.21.15.2:0/3778293227 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4012d00 con 0x55821b8a6380 2024-03-15T13:02:02.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.338+0000 7f4496ff5640 1 -- 172.21.15.2:0/3778293227 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f44a4005020 con 0x55821b8a6380 2024-03-15T13:02:02.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.338+0000 7f44bc16c740 1 -- 172.21.15.2:0/3778293227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55821b833900 con 0x55821b8a6380 2024-03-15T13:02:02.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.338+0000 7f4496ff5640 1 -- 172.21.15.2:0/3778293227 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44a4005220 con 0x55821b8a6380 2024-03-15T13:02:02.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.338+0000 7f44bc16c740 10 client.16707 fetch_fsmap learned FSMap version 37 2024-03-15T13:02:02.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.338+0000 7f44bc16c740 1 -- 172.21.15.2:0/3778293227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55821b7d00b0 con 0x55821b8a6380 2024-03-15T13:02:02.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.338+0000 7f4496ff5640 1 -- 172.21.15.2:0/3778293227 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55821b833900 con 0x55821b8a6380 2024-03-15T13:02:02.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.338+0000 7f4496ff5640 1 -- 172.21.15.2:0/3778293227 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 37) v1 ==== 7986+0+0 (secure 0 0 0) 0x7f44a4037670 con 0x55821b8a6380 2024-03-15T13:02:02.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.338+0000 7f44bc16c740 10 client.16707 fetch_fsmap finished waiting for FSMap version 37 2024-03-15T13:02:02.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.338+0000 7f44bc16c740 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:02:02.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.338+0000 7f44bc16c740 20 client.16707 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:02.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.338+0000 7f44bc16c740 1 --2- 172.21.15.2:0/3778293227 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55821b8bff80 0x55821b91f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.338+0000 7f44bc16c740 4 client.16707 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:02:02.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.338+0000 7f44bc16c740 1 -- 172.21.15.2:0/3778293227 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55821b50d860 con 0x55821b8bff80 2024-03-15T13:02:02.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.338+0000 7f44b27fc640 1 --2- 172.21.15.2:0/3778293227 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55821b8bff80 0x55821b91f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.339+0000 7f4496ff5640 10 client.16707 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:02.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.339+0000 7f44b27fc640 1 --2- 172.21.15.2:0/3778293227 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55821b8bff80 0x55821b91f010 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea75b2cb00875250 server_cookie=f7acbcdafc96787e in_seq=0 out_seq=0 2024-03-15T13:02:02.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.340+0000 7f4496ff5640 1 -- 172.21.15.2:0/3778293227 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55821b50d860 con 0x55821b8bff80 2024-03-15T13:02:02.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.340+0000 7f4496ff5640 10 client.16707 handle_command_reply: tid=0 2024-03-15T13:02:02.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.340+0000 7f44bc16c740 1 client.16707 shutdown 2024-03-15T13:02:02.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.340+0000 7f44bc16c740 1 -- 172.21.15.2:0/3778293227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 msgr2=0x55821b8aa450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.340+0000 7f44bc16c740 1 --2- 172.21.15.2:0/3778293227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8aa450 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f44a401ffd0 tx=0x7f44a401f870 comp rx=0 tx=0).stop 2024-03-15T13:02:02.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.340+0000 7f44bc16c740 1 -- 172.21.15.2:0/3778293227 shutdown_connections 2024-03-15T13:02:02.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.340+0000 7f44bc16c740 1 --2- 172.21.15.2:0/3778293227 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55821b8bff80 0x55821b91f010 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.340+0000 7f44bc16c740 1 --2- 172.21.15.2:0/3778293227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55821b8af8b0 0x55821b8b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.340+0000 7f44bc16c740 1 --2- 172.21.15.2:0/3778293227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55821b8a6d30 0x55821b8b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.340+0000 7f44bc16c740 1 --2- 172.21.15.2:0/3778293227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55821b8a6380 0x55821b8aa450 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.340+0000 7f44bc16c740 1 -- 172.21.15.2:0/3778293227 >> 172.21.15.2:0/3778293227 conn(0x55821b8a0840 msgr2=0x55821b89f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:02.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.340+0000 7f4496ff5640 0 client.16707 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:02.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.341+0000 7f44bc16c740 1 -- 172.21.15.2:0/3778293227 shutdown_connections 2024-03-15T13:02:02.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.341+0000 7f44bc16c740 1 -- 172.21.15.2:0/3778293227 wait complete. 2024-03-15T13:02:02.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.341+0000 7f44bc16c740 20 client.16707 trim_cache size 0 max 16384 2024-03-15T13:02:02.331 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:02:02.331 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:02:02.332 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "65486c52-18d5-447a-8c5f-8bf67a0b6a19", 2024-03-15T13:02:02.332 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:02:02.332 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:02:02.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.341+0000 7f44b958c640 1 -- 172.21.15.2:0/74919128 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f449c0a1ba0 msgr2=0x7f449c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.341+0000 7f44b958c640 1 --2- 172.21.15.2:0/74919128 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f449c0a1ba0 0x7f449c0a4060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55821b8a1ac0 tx=0x7f44a4005f90 comp rx=0 tx=0).stop 2024-03-15T13:02:02.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.341+0000 7f44b958c640 1 -- 172.21.15.2:0/74919128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44b406a420 msgr2=0x7f44b419dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.341+0000 7f44b958c640 1 --2- 172.21.15.2:0/74919128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44b406a420 0x7f44b419dc10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009230 tx=0x7f44a802ca40 comp rx=0 tx=0).stop 2024-03-15T13:02:02.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.341+0000 7f44b958c640 1 -- 172.21.15.2:0/74919128 shutdown_connections 2024-03-15T13:02:02.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.341+0000 7f44b958c640 1 --2- 172.21.15.2:0/74919128 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f449c0a1ba0 0x7f449c0a4060 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.341+0000 7f44b958c640 1 --2- 172.21.15.2:0/74919128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44b4107560 0x7f44b41978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.341+0000 7f44b958c640 1 --2- 172.21.15.2:0/74919128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44b406a420 0x7f44b419dc10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.341+0000 7f44b958c640 1 --2- 172.21.15.2:0/74919128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44b4069a60 0x7f44b419d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.341+0000 7f44b958c640 1 -- 172.21.15.2:0/74919128 >> 172.21.15.2:0/74919128 conn(0x7f44b40773d0 msgr2=0x7f44b40ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:02.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.341+0000 7f44b958c640 1 -- 172.21.15.2:0/74919128 shutdown_connections 2024-03-15T13:02:02.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.341+0000 7f44b958c640 1 -- 172.21.15.2:0/74919128 wait complete. 2024-03-15T13:02:02.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:02:02.338 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:02:02.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:01 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:02:02.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:01 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:02:02.424 INFO:tasks.workunit.client.0.smithi002.stderr:+ tar c linux-4.0.5 2024-03-15T13:02:02.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.513+0000 7fd021d8e640 1 -- 172.21.15.2:0/3168143930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd01c100290 msgr2=0x7fd01c100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.513+0000 7fd021d8e640 1 --2- 172.21.15.2:0/3168143930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd01c100290 0x7fd01c100670 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009860 tx=0x7fd00c02bb70 comp rx=0 tx=0).stop 2024-03-15T13:02:02.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.513+0000 7fd021d8e640 1 -- 172.21.15.2:0/3168143930 shutdown_connections 2024-03-15T13:02:02.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.513+0000 7fd021d8e640 1 --2- 172.21.15.2:0/3168143930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c10de40 0x7fd01c110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.513+0000 7fd021d8e640 1 --2- 172.21.15.2:0/3168143930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd01c100bb0 0x7fd01c10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.513+0000 7fd021d8e640 1 --2- 172.21.15.2:0/3168143930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd01c100290 0x7fd01c100670 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.513+0000 7fd021d8e640 1 -- 172.21.15.2:0/3168143930 >> 172.21.15.2:0/3168143930 conn(0x7fd01c0775a0 msgr2=0x7fd01c0779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:02.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.513+0000 7fd021d8e640 1 -- 172.21.15.2:0/3168143930 shutdown_connections 2024-03-15T13:02:02.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.513+0000 7fd021d8e640 1 -- 172.21.15.2:0/3168143930 wait complete. 2024-03-15T13:02:02.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.514+0000 7fd021d8e640 1 Processor -- start 2024-03-15T13:02:02.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.514+0000 7fd021d8e640 1 -- start start 2024-03-15T13:02:02.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd021d8e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c100290 0x7fd01c19f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd021d8e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd01c100bb0 0x7fd01c19fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd021d8e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd01c10de40 0x7fd01c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd021d8e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c19a130 con 0x7fd01c100290 2024-03-15T13:02:02.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd021d8e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c19a2a0 con 0x7fd01c10de40 2024-03-15T13:02:02.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd021d8e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c19a460 con 0x7fd01c100bb0 2024-03-15T13:02:02.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd01b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c100290 0x7fd01c19f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd01b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c100290 0x7fd01c19f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34218/0 (socket says 172.21.15.2:34218) 2024-03-15T13:02:02.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd01b7fe640 1 -- 172.21.15.2:0/1404044063 learned_addr learned my addr 172.21.15.2:0/1404044063 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:02.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd01affd640 1 --2- 172.21.15.2:0/1404044063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd01c100bb0 0x7fd01c19fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd01bfff640 1 --2- 172.21.15.2:0/1404044063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd01c10de40 0x7fd01c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd01b7fe640 1 -- 172.21.15.2:0/1404044063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd01c100bb0 msgr2=0x7fd01c19fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd01b7fe640 1 --2- 172.21.15.2:0/1404044063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd01c100bb0 0x7fd01c19fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd01b7fe640 1 -- 172.21.15.2:0/1404044063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd01c10de40 msgr2=0x7fd01c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd01b7fe640 1 --2- 172.21.15.2:0/1404044063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd01c10de40 0x7fd01c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd01b7fe640 1 -- 172.21.15.2:0/1404044063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c009490 con 0x7fd01c100290 2024-03-15T13:02:02.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd01affd640 1 --2- 172.21.15.2:0/1404044063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd01c100bb0 0x7fd01c19fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:02.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.515+0000 7fd01b7fe640 1 --2- 172.21.15.2:0/1404044063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c100290 0x7fd01c19f930 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd00c000c00 tx=0x7fd00c042f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:02.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.516+0000 7fd018ff9640 1 -- 172.21.15.2:0/1404044063 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c004340 con 0x7fd01c100290 2024-03-15T13:02:02.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.516+0000 7fd018ff9640 1 -- 172.21.15.2:0/1404044063 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd00c0044a0 con 0x7fd01c100290 2024-03-15T13:02:02.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.516+0000 7fd021d8e640 1 -- 172.21.15.2:0/1404044063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01c19a6e0 con 0x7fd01c100290 2024-03-15T13:02:02.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.516+0000 7fd018ff9640 1 -- 172.21.15.2:0/1404044063 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c003850 con 0x7fd01c100290 2024-03-15T13:02:02.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.516+0000 7fd021d8e640 1 -- 172.21.15.2:0/1404044063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01c102680 con 0x7fd01c100290 2024-03-15T13:02:02.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.518+0000 7fd018ff9640 1 -- 172.21.15.2:0/1404044063 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fd00c04a030 con 0x7fd01c100290 2024-03-15T13:02:02.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd024968740 1 Processor -- start 2024-03-15T13:02:02.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd024968740 1 -- start start 2024-03-15T13:02:02.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd024968740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28e2b0 0x55b17d28e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd024968740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28ebd0 0x55b17d297270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd024968740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d2977b0 0x55b17d299ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd024968740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d1fd280 con 0x55b17d2977b0 2024-03-15T13:02:02.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd024968740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d1fa7b0 con 0x55b17d28e2b0 2024-03-15T13:02:02.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd018ff9640 1 --2- 172.21.15.2:0/1404044063 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd0040a1ba0 0x7fd0040a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd024968740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d1f5f90 con 0x55b17d28ebd0 2024-03-15T13:02:02.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd01bfff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28e2b0 0x55b17d28e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd01affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d2977b0 0x55b17d299ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd018ff9640 1 -- 172.21.15.2:0/1404044063 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fd00c0f7970 con 0x7fd01c100290 2024-03-15T13:02:02.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd01bfff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28e2b0 0x55b17d28e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44764/0 (socket says 172.21.15.2:44764) 2024-03-15T13:02:02.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd01bfff640 1 -- 172.21.15.2:0/471972904 learned_addr learned my addr 172.21.15.2:0/471972904 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:02.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd01b7fe640 1 --2- 172.21.15.2:0/471972904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28ebd0 0x55b17d297270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.519+0000 7fd01bfff640 1 --2- 172.21.15.2:0/1404044063 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd0040a1ba0 0x7fd0040a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.520+0000 7fd01bfff640 1 -- 172.21.15.2:0/471972904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28ebd0 msgr2=0x55b17d297270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.520+0000 7fd01bfff640 1 --2- 172.21.15.2:0/471972904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28ebd0 0x55b17d297270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.520+0000 7fd01bfff640 1 -- 172.21.15.2:0/471972904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d2977b0 msgr2=0x55b17d299ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.520+0000 7fd01bfff640 1 --2- 172.21.15.2:0/471972904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d2977b0 0x55b17d299ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.520+0000 7fd01bfff640 1 -- 172.21.15.2:0/471972904 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b17d1b4570 con 0x55b17d28e2b0 2024-03-15T13:02:02.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.520+0000 7fd01bfff640 1 --2- 172.21.15.2:0/471972904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28e2b0 0x55b17d28e690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd01400ea70 tx=0x7fd01400ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eff83fb9668792 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:02.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.520+0000 7fd001ffb640 1 -- 172.21.15.2:0/471972904 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014013070 con 0x55b17d28e2b0 2024-03-15T13:02:02.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.520+0000 7fd01bfff640 1 --2- 172.21.15.2:0/1404044063 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd0040a1ba0 0x7fd0040a4060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd014023000 tx=0x7fd014020000 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:02.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.520+0000 7fd001ffb640 1 -- 172.21.15.2:0/471972904 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd014020070 con 0x55b17d28e2b0 2024-03-15T13:02:02.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.520+0000 7fd001ffb640 1 -- 172.21.15.2:0/471972904 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014005590 con 0x55b17d28e2b0 2024-03-15T13:02:02.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.521+0000 7fd024968740 1 -- 172.21.15.2:0/471972904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28e2b0 msgr2=0x55b17d28e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.521+0000 7fd024968740 1 --2- 172.21.15.2:0/471972904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28e2b0 0x55b17d28e690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd01400ea70 tx=0x7fd01400ee70 comp rx=0 tx=0).stop 2024-03-15T13:02:02.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.521+0000 7fd024968740 1 -- 172.21.15.2:0/471972904 shutdown_connections 2024-03-15T13:02:02.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.521+0000 7fd024968740 1 --2- 172.21.15.2:0/471972904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d2977b0 0x55b17d299ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.521+0000 7fd024968740 1 --2- 172.21.15.2:0/471972904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28ebd0 0x55b17d297270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.521+0000 7fd024968740 1 --2- 172.21.15.2:0/471972904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28e2b0 0x55b17d28e690 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.521+0000 7fd024968740 1 -- 172.21.15.2:0/471972904 >> 172.21.15.2:0/471972904 conn(0x55b17d288990 msgr2=0x55b17d288d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:02.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.521+0000 7fd024968740 1 -- 172.21.15.2:0/471972904 shutdown_connections 2024-03-15T13:02:02.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.521+0000 7fd024968740 1 -- 172.21.15.2:0/471972904 wait complete. 2024-03-15T13:02:02.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.521+0000 7fd024968740 1 -- start start 2024-03-15T13:02:02.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd024968740 1 Processor -- start 2024-03-15T13:02:02.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd024968740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28e2b0 0x55b17d294610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd024968740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28ebd0 0x55b17d294b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd024968740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d2977b0 0x55b17d295090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd024968740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17cec9430 con 0x55b17d2977b0 2024-03-15T13:02:02.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd024968740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17ce85e00 con 0x55b17d28ebd0 2024-03-15T13:02:02.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd024968740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d17afd0 con 0x55b17d28e2b0 2024-03-15T13:02:02.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd024968740 10 client.0 _ll_register_callbacks cb 0x55b17d1d0fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:02.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd01b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28e2b0 0x55b17d294610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd01b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28e2b0 0x55b17d294610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:46222/0 (socket says 172.21.15.2:46222) 2024-03-15T13:02:02.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd01b7fe640 1 -- 172.21.15.2:0/3614289804 learned_addr learned my addr 172.21.15.2:0/3614289804 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:02.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd01affd640 1 --2- 172.21.15.2:0/3614289804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28ebd0 0x55b17d294b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd01bfff640 1 --2- 172.21.15.2:0/3614289804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d2977b0 0x55b17d295090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd01b7fe640 1 -- 172.21.15.2:0/3614289804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28ebd0 msgr2=0x55b17d294b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd01b7fe640 1 --2- 172.21.15.2:0/3614289804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28ebd0 0x55b17d294b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd01b7fe640 1 -- 172.21.15.2:0/3614289804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d2977b0 msgr2=0x55b17d295090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd01b7fe640 1 --2- 172.21.15.2:0/3614289804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d2977b0 0x55b17d295090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.522+0000 7fd01b7fe640 1 -- 172.21.15.2:0/3614289804 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01400e6b0 con 0x55b17d28e2b0 2024-03-15T13:02:02.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fd01b7fe640 1 --2- 172.21.15.2:0/3614289804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28e2b0 0x55b17d294610 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd00c048020 tx=0x7fd00c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=649866f510bed140 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:02.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fcffaffd640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:02:02.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fcffaffd640 1 -- 172.21.15.2:0/3614289804 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0c3cc0 con 0x55b17d28e2b0 2024-03-15T13:02:02.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fd024968740 1 -- 172.21.15.2:0/3614289804 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b17cef57e0 con 0x55b17d28e2b0 2024-03-15T13:02:02.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fcffaffd640 1 -- 172.21.15.2:0/3614289804 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fd00c0c4a60 con 0x55b17d28e2b0 2024-03-15T13:02:02.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fcffaffd640 1 -- 172.21.15.2:0/3614289804 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd00c0c4360 con 0x55b17d28e2b0 2024-03-15T13:02:02.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fd024968740 10 client.26512 fetch_fsmap learned FSMap version 37 2024-03-15T13:02:02.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fd024968740 1 -- 172.21.15.2:0/3614289804 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55b17d1f4610 con 0x55b17d28e2b0 2024-03-15T13:02:02.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fcffaffd640 1 -- 172.21.15.2:0/3614289804 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd00c0a8020 con 0x55b17d28e2b0 2024-03-15T13:02:02.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fcffaffd640 1 -- 172.21.15.2:0/3614289804 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 37) v1 ==== 7986+0+0 (secure 0 0 0) 0x7fd00c0c09d0 con 0x55b17d28e2b0 2024-03-15T13:02:02.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fd024968740 10 client.26512 fetch_fsmap finished waiting for FSMap version 37 2024-03-15T13:02:02.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fd024968740 10 client.26512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:02:02.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fd024968740 20 client.26512 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:02.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fd024968740 1 --2- 172.21.15.2:0/3614289804 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b17d2a7680 0x55b17d2a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fd024968740 4 client.26512 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:02:02.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.523+0000 7fd024968740 1 -- 172.21.15.2:0/3614289804 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b17cf32720 con 0x55b17d2a7680 2024-03-15T13:02:02.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.524+0000 7fd01affd640 1 --2- 172.21.15.2:0/3614289804 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b17d2a7680 0x55b17d2a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.524+0000 7fcffaffd640 10 client.26512 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:02.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.524+0000 7fd01affd640 1 --2- 172.21.15.2:0/3614289804 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b17d2a7680 0x55b17d2a9aa0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4f21c0832584159 server_cookie=6ff3655ce93b9ef6 in_seq=0 out_seq=0 2024-03-15T13:02:02.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.525+0000 7fcffaffd640 1 -- 172.21.15.2:0/3614289804 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b17cf32720 con 0x55b17d2a7680 2024-03-15T13:02:02.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.525+0000 7fcffaffd640 10 client.26512 handle_command_reply: tid=0 2024-03-15T13:02:02.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.525+0000 7fd024968740 1 client.26512 shutdown 2024-03-15T13:02:02.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.525+0000 7fd024968740 1 -- 172.21.15.2:0/3614289804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28e2b0 msgr2=0x55b17d294610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.525+0000 7fd024968740 1 --2- 172.21.15.2:0/3614289804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28e2b0 0x55b17d294610 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd00c048020 tx=0x7fd00c005f90 comp rx=0 tx=0).stop 2024-03-15T13:02:02.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.525+0000 7fd024968740 1 -- 172.21.15.2:0/3614289804 shutdown_connections 2024-03-15T13:02:02.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.525+0000 7fd024968740 1 --2- 172.21.15.2:0/3614289804 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b17d2a7680 0x55b17d2a9aa0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.525+0000 7fd024968740 1 --2- 172.21.15.2:0/3614289804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d2977b0 0x55b17d295090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.525+0000 7fd024968740 1 --2- 172.21.15.2:0/3614289804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28ebd0 0x55b17d294b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.525+0000 7fd024968740 1 --2- 172.21.15.2:0/3614289804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28e2b0 0x55b17d294610 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.525+0000 7fcffaffd640 0 client.26512 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:02.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.525+0000 7fd024968740 1 -- 172.21.15.2:0/3614289804 >> 172.21.15.2:0/3614289804 conn(0x55b17d288990 msgr2=0x55b17d28a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:02.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.526+0000 7fd024968740 1 -- 172.21.15.2:0/3614289804 shutdown_connections 2024-03-15T13:02:02.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.526+0000 7fd024968740 1 -- 172.21.15.2:0/3614289804 wait complete. 2024-03-15T13:02:02.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.526+0000 7fd024968740 20 client.26512 trim_cache size 0 max 16384 2024-03-15T13:02:02.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.545+0000 7fd024968740 1 Processor -- start 2024-03-15T13:02:02.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.545+0000 7fd024968740 1 -- start start 2024-03-15T13:02:02.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd024968740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d28e2b0 0x55b17d296d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd024968740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28ebd0 0x55b17d292db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd024968740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d2977b0 0x55b17d2a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd024968740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d28dcf0 con 0x55b17d28e2b0 2024-03-15T13:02:02.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd024968740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d2932f0 con 0x55b17d28ebd0 2024-03-15T13:02:02.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd024968740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d28df10 con 0x55b17d2977b0 2024-03-15T13:02:02.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd01b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d28e2b0 0x55b17d296d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd01b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d28e2b0 0x55b17d296d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34238/0 (socket says 172.21.15.2:34238) 2024-03-15T13:02:02.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd01bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d2977b0 0x55b17d2a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd01b7fe640 1 -- 172.21.15.2:0/1268792464 learned_addr learned my addr 172.21.15.2:0/1268792464 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:02.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd01affd640 1 --2- 172.21.15.2:0/1268792464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28ebd0 0x55b17d292db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd01b7fe640 1 -- 172.21.15.2:0/1268792464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d2977b0 msgr2=0x55b17d2a2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd01b7fe640 1 --2- 172.21.15.2:0/1268792464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d2977b0 0x55b17d2a2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd01b7fe640 1 -- 172.21.15.2:0/1268792464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28ebd0 msgr2=0x55b17d292db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd01b7fe640 1 --2- 172.21.15.2:0/1268792464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28ebd0 0x55b17d292db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd01b7fe640 1 -- 172.21.15.2:0/1268792464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b17d190ac0 con 0x55b17d28e2b0 2024-03-15T13:02:02.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd01b7fe640 1 --2- 172.21.15.2:0/1268792464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d28e2b0 0x55b17d296d40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0c07c0 tx=0x7fd00c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f092edf0ba3ea1ec server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:02.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd0017fa640 1 -- 172.21.15.2:0/1268792464 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0c3cc0 con 0x55b17d28e2b0 2024-03-15T13:02:02.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd0017fa640 1 -- 172.21.15.2:0/1268792464 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd00c0a8070 con 0x55b17d28e2b0 2024-03-15T13:02:02.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.546+0000 7fd0017fa640 1 -- 172.21.15.2:0/1268792464 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0c4360 con 0x55b17d28e2b0 2024-03-15T13:02:02.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.547+0000 7fd024968740 1 -- 172.21.15.2:0/1268792464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d28e2b0 msgr2=0x55b17d296d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.547+0000 7fd024968740 1 --2- 172.21.15.2:0/1268792464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d28e2b0 0x55b17d296d40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0c07c0 tx=0x7fd00c004610 comp rx=0 tx=0).stop 2024-03-15T13:02:02.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.547+0000 7fd024968740 1 -- 172.21.15.2:0/1268792464 shutdown_connections 2024-03-15T13:02:02.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.547+0000 7fd024968740 1 --2- 172.21.15.2:0/1268792464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d2977b0 0x55b17d2a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.547+0000 7fd024968740 1 --2- 172.21.15.2:0/1268792464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d28ebd0 0x55b17d292db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.547+0000 7fd024968740 1 --2- 172.21.15.2:0/1268792464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d28e2b0 0x55b17d296d40 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.547+0000 7fd024968740 1 -- 172.21.15.2:0/1268792464 >> 172.21.15.2:0/1268792464 conn(0x55b17d288990 msgr2=0x55b17d294e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:02.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.547+0000 7fd024968740 1 -- 172.21.15.2:0/1268792464 shutdown_connections 2024-03-15T13:02:02.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.547+0000 7fd024968740 1 -- 172.21.15.2:0/1268792464 wait complete. 2024-03-15T13:02:02.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.547+0000 7fd024968740 1 -- start start 2024-03-15T13:02:02.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd024968740 1 Processor -- start 2024-03-15T13:02:02.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd024968740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28e2b0 0x55b17d29f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd024968740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d28ebd0 0x55b17d298b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd024968740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d2977b0 0x55b17d299040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd024968740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d299580 con 0x55b17d28ebd0 2024-03-15T13:02:02.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd024968740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d2996f0 con 0x55b17d2977b0 2024-03-15T13:02:02.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd024968740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17d2998b0 con 0x55b17d28e2b0 2024-03-15T13:02:02.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd024968740 10 client.0 _ll_register_callbacks cb 0x7fd00c0097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:02.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd01affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d28ebd0 0x55b17d298b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd01b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28e2b0 0x55b17d29f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd01affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d28ebd0 0x55b17d298b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34242/0 (socket says 172.21.15.2:34242) 2024-03-15T13:02:02.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd01affd640 1 -- 172.21.15.2:0/3083610102 learned_addr learned my addr 172.21.15.2:0/3083610102 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:02.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd01bfff640 1 --2- 172.21.15.2:0/3083610102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d2977b0 0x55b17d299040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd01affd640 1 -- 172.21.15.2:0/3083610102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28e2b0 msgr2=0x55b17d29f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd01affd640 1 --2- 172.21.15.2:0/3083610102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28e2b0 0x55b17d29f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd01affd640 1 -- 172.21.15.2:0/3083610102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d2977b0 msgr2=0x55b17d299040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:02.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd01affd640 1 --2- 172.21.15.2:0/3083610102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d2977b0 0x55b17d299040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd01affd640 1 -- 172.21.15.2:0/3083610102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01c102680 con 0x55b17d28ebd0 2024-03-15T13:02:02.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.548+0000 7fd01b7fe640 1 --2- 172.21.15.2:0/3083610102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28e2b0 0x55b17d29f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:02.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fd01affd640 1 --2- 172.21.15.2:0/3083610102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d28ebd0 0x55b17d298b00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd010012560 tx=0x7fd0100039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658ce4a9f3bec78d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:02.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fcffaffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:02.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fcffaffd640 1 -- 172.21.15.2:0/3083610102 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010017030 con 0x55b17d28ebd0 2024-03-15T13:02:02.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fcffaffd640 1 -- 172.21.15.2:0/3083610102 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fd0100202b0 con 0x55b17d28ebd0 2024-03-15T13:02:02.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fd024968740 1 -- 172.21.15.2:0/3083610102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd014020070 con 0x55b17d28ebd0 2024-03-15T13:02:02.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fcffaffd640 1 -- 172.21.15.2:0/3083610102 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd010020770 con 0x55b17d28ebd0 2024-03-15T13:02:02.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fd024968740 10 client.16725 fetch_fsmap learned FSMap version 37 2024-03-15T13:02:02.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fd024968740 1 -- 172.21.15.2:0/3083610102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55b17d1b66a0 con 0x55b17d28ebd0 2024-03-15T13:02:02.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fcffaffd640 1 -- 172.21.15.2:0/3083610102 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd014020070 con 0x55b17d28ebd0 2024-03-15T13:02:02.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fcffaffd640 1 -- 172.21.15.2:0/3083610102 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 37) v1 ==== 7986+0+0 (secure 0 0 0) 0x7fd010012b90 con 0x55b17d28ebd0 2024-03-15T13:02:02.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fd024968740 10 client.16725 fetch_fsmap finished waiting for FSMap version 37 2024-03-15T13:02:02.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fd024968740 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:02:02.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fd024968740 20 client.16725 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:02.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fd024968740 1 --2- 172.21.15.2:0/3083610102 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b17d3045d0 0x55b17d3069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:02.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fd024968740 4 client.16725 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:02:02.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.549+0000 7fd024968740 1 -- 172.21.15.2:0/3083610102 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b17cf81890 con 0x55b17d3045d0 2024-03-15T13:02:02.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.550+0000 7fd01b7fe640 1 --2- 172.21.15.2:0/3083610102 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b17d3045d0 0x55b17d3069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:02.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.552+0000 7fcffaffd640 10 client.16725 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:02.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.552+0000 7fd01b7fe640 1 --2- 172.21.15.2:0/3083610102 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b17d3045d0 0x55b17d3069f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41de6a515ab37893 server_cookie=4948c920edf7584f in_seq=0 out_seq=0 2024-03-15T13:02:02.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.552+0000 7fcffaffd640 1 -- 172.21.15.2:0/3083610102 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x55b17cf81890 con 0x55b17d3045d0 2024-03-15T13:02:02.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.552+0000 7fcffaffd640 10 client.16725 handle_command_reply: tid=0 2024-03-15T13:02:02.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.552+0000 7fd024968740 1 client.16725 shutdown 2024-03-15T13:02:02.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd024968740 1 -- 172.21.15.2:0/3083610102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d28ebd0 msgr2=0x55b17d298b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd024968740 1 --2- 172.21.15.2:0/3083610102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d28ebd0 0x55b17d298b00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd010012560 tx=0x7fd0100039c0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd024968740 1 -- 172.21.15.2:0/3083610102 shutdown_connections 2024-03-15T13:02:02.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd024968740 1 --2- 172.21.15.2:0/3083610102 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b17d3045d0 0x55b17d3069f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd024968740 1 --2- 172.21.15.2:0/3083610102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b17d2977b0 0x55b17d299040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd024968740 1 --2- 172.21.15.2:0/3083610102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b17d28ebd0 0x55b17d298b00 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fcffaffd640 0 client.16725 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:02.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd024968740 1 --2- 172.21.15.2:0/3083610102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b17d28e2b0 0x55b17d29f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd024968740 1 -- 172.21.15.2:0/3083610102 >> 172.21.15.2:0/3083610102 conn(0x55b17d288990 msgr2=0x55b17d289ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:02.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd024968740 1 -- 172.21.15.2:0/3083610102 shutdown_connections 2024-03-15T13:02:02.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd024968740 1 -- 172.21.15.2:0/3083610102 wait complete. 2024-03-15T13:02:02.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd024968740 20 client.16725 trim_cache size 0 max 16384 2024-03-15T13:02:02.543 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:02:02.543 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (1098 inodes in the stack)", 2024-03-15T13:02:02.543 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:02:02.543 INFO:teuthology.orchestra.run.smithi002.stdout: "65486c52-18d5-447a-8c5f-8bf67a0b6a19": { 2024-03-15T13:02:02.543 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:02:02.543 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65486c52-18d5-447a-8c5f-8bf67a0b6a19", 2024-03-15T13:02:02.543 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:02:02.544 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:02:02.544 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:02:02.544 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:02:02.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd021d8e640 1 -- 172.21.15.2:0/1404044063 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd0040a1ba0 msgr2=0x7fd0040a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd021d8e640 1 --2- 172.21.15.2:0/1404044063 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd0040a1ba0 0x7fd0040a4060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd014023000 tx=0x7fd014020000 comp rx=0 tx=0).stop 2024-03-15T13:02:02.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd021d8e640 1 -- 172.21.15.2:0/1404044063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c100290 msgr2=0x7fd01c19f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:02.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd021d8e640 1 --2- 172.21.15.2:0/1404044063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c100290 0x7fd01c19f930 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd00c000c00 tx=0x7fd00c042f20 comp rx=0 tx=0).stop 2024-03-15T13:02:02.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd021d8e640 1 -- 172.21.15.2:0/1404044063 shutdown_connections 2024-03-15T13:02:02.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd021d8e640 1 --2- 172.21.15.2:0/1404044063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd01c10de40 0x7fd01c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd021d8e640 1 --2- 172.21.15.2:0/1404044063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd01c100bb0 0x7fd01c19fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd021d8e640 1 --2- 172.21.15.2:0/1404044063 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd0040a1ba0 0x7fd0040a4060 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd021d8e640 1 --2- 172.21.15.2:0/1404044063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c100290 0x7fd01c19f930 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:02.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.553+0000 7fd021d8e640 1 -- 172.21.15.2:0/1404044063 >> 172.21.15.2:0/1404044063 conn(0x7fd01c0775a0 msgr2=0x7fd01c10f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:02.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.554+0000 7fd021d8e640 1 -- 172.21.15.2:0/1404044063 shutdown_connections 2024-03-15T13:02:02.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:02.554+0000 7fd021d8e640 1 -- 172.21.15.2:0/1404044063 wait complete. 2024-03-15T13:02:02.551 INFO:tasks.cephfs.filesystem:scrub status for tag:65486c52-18d5-447a-8c5f-8bf67a0b6a19 - {'path': '/', 'tag': '65486c52-18d5-447a-8c5f-8bf67a0b6a19', 'options': 'recursive,force'} 2024-03-15T13:02:03.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: pgmap v97: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-03-15T13:02:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:02:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:02:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:02:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:03.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:02 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: pgmap v97: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-03-15T13:02:03.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:03.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:03.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:03.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:02:03.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:02:03.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:02:03.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:03.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:03.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:03.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:02 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: pgmap v97: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-03-15T13:02:03.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:03.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:03.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:03.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:02:03.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:02:03.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:02:03.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:03.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:03.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:03.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:03.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:02 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:02:04.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:04.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:04.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:04.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:04.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:04.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:04.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:04.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:04.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:04.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:04.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:04.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:04.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:04.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:04.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:03 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:02:04.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:04.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:04.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:04.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:04.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:04.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:04.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:04.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:04.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:04.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:04.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:04.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:04.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:04.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.311 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:04.311 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:03 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:02:04.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:04.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:04.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:04.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:04.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:04.360 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:04.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:04.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:04.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:04.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:04.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:04.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:04.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:04.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:04.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:03 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:04.926 INFO:tasks.workunit.client.0.smithi002.stderr:+ echo Ok 2024-03-15T13:02:04.927 INFO:tasks.workunit.client.0.smithi002.stdout:Ok 2024-03-15T13:02:04.927 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-15T13:02:04.927 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-15T13:02:05.209 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:04 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:05.210 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:04 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:05.210 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:04 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:05.210 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:04 smithi184 ceph-mon[35993]: pgmap v98: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-03-15T13:02:05.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:04 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:05.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:04 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:05.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:04 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:05.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:04 smithi002 ceph-mon[32060]: pgmap v98: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-03-15T13:02:05.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:04 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:05.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:04 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:05.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:04 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:05.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:04 smithi149 ceph-mon[35927]: pgmap v98: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-03-15T13:02:07.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:06 smithi002 ceph-mon[32060]: pgmap v99: 129 pgs: 129 active+clean; 240 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 246 op/s 2024-03-15T13:02:07.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:06 smithi184 ceph-mon[35993]: pgmap v99: 129 pgs: 129 active+clean; 240 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 246 op/s 2024-03-15T13:02:07.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:06 smithi149 ceph-mon[35927]: pgmap v99: 129 pgs: 129 active+clean; 240 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 246 op/s 2024-03-15T13:02:09.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:08 smithi002 ceph-mon[32060]: pgmap v100: 129 pgs: 129 active+clean; 240 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-03-15T13:02:09.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:08 smithi184 ceph-mon[35993]: pgmap v100: 129 pgs: 129 active+clean; 240 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-03-15T13:02:09.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:08 smithi149 ceph-mon[35927]: pgmap v100: 129 pgs: 129 active+clean; 240 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-03-15T13:02:11.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:10 smithi002 ceph-mon[32060]: pgmap v101: 129 pgs: 129 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-03-15T13:02:11.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:10 smithi184 ceph-mon[35993]: pgmap v101: 129 pgs: 129 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-03-15T13:02:11.361 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:10 smithi149 ceph-mon[35927]: pgmap v101: 129 pgs: 129 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-03-15T13:02:13.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:12 smithi002 ceph-mon[32060]: pgmap v102: 129 pgs: 129 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-03-15T13:02:13.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:12 smithi184 ceph-mon[35993]: pgmap v102: 129 pgs: 129 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-03-15T13:02:13.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:12 smithi149 ceph-mon[35927]: pgmap v102: 129 pgs: 129 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-03-15T13:02:15.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:14 smithi002 ceph-mon[32060]: pgmap v103: 129 pgs: 129 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-03-15T13:02:15.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:14 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:02:15.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:14 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:02:15.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:14 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:02:15.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:14 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:02:15.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:14 smithi184 ceph-mon[35993]: pgmap v103: 129 pgs: 129 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-03-15T13:02:15.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:14 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:02:15.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:14 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:02:15.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:14 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:02:15.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:14 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:02:15.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:14 smithi149 ceph-mon[35927]: pgmap v103: 129 pgs: 129 active+clean; 246 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-03-15T13:02:15.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:14 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:02:15.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:14 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:02:15.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:14 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:02:15.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:14 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:02:16.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:15 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:02:16.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:15 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:02:16.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:15 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:02:17.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:16 smithi002 ceph-mon[32060]: pgmap v104: 129 pgs: 129 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-03-15T13:02:17.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:16 smithi184 ceph-mon[35993]: pgmap v104: 129 pgs: 129 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-03-15T13:02:17.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:16 smithi149 ceph-mon[35927]: pgmap v104: 129 pgs: 129 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-03-15T13:02:19.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:18 smithi002 ceph-mon[32060]: pgmap v105: 129 pgs: 129 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 932 KiB/s wr, 130 op/s 2024-03-15T13:02:19.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:18 smithi184 ceph-mon[35993]: pgmap v105: 129 pgs: 129 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 932 KiB/s wr, 130 op/s 2024-03-15T13:02:19.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:18 smithi149 ceph-mon[35927]: pgmap v105: 129 pgs: 129 active+clean; 251 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 932 KiB/s wr, 130 op/s 2024-03-15T13:02:21.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:20 smithi002 ceph-mon[32060]: pgmap v106: 129 pgs: 129 active+clean; 256 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2024-03-15T13:02:21.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:20 smithi184 ceph-mon[35993]: pgmap v106: 129 pgs: 129 active+clean; 256 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2024-03-15T13:02:21.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:20 smithi149 ceph-mon[35927]: pgmap v106: 129 pgs: 129 active+clean; 256 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 224 op/s 2024-03-15T13:02:23.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:22 smithi002 ceph-mon[32060]: pgmap v107: 129 pgs: 129 active+clean; 256 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 951 KiB/s wr, 183 op/s 2024-03-15T13:02:23.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:22 smithi184 ceph-mon[35993]: pgmap v107: 129 pgs: 129 active+clean; 256 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 951 KiB/s wr, 183 op/s 2024-03-15T13:02:23.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:22 smithi149 ceph-mon[35927]: pgmap v107: 129 pgs: 129 active+clean; 256 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 951 KiB/s wr, 183 op/s 2024-03-15T13:02:25.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:24 smithi002 ceph-mon[32060]: pgmap v108: 129 pgs: 129 active+clean; 256 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 951 KiB/s wr, 183 op/s 2024-03-15T13:02:25.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:24 smithi184 ceph-mon[35993]: pgmap v108: 129 pgs: 129 active+clean; 256 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 951 KiB/s wr, 183 op/s 2024-03-15T13:02:25.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:24 smithi149 ceph-mon[35927]: pgmap v108: 129 pgs: 129 active+clean; 256 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 951 KiB/s wr, 183 op/s 2024-03-15T13:02:27.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:26 smithi002 ceph-mon[32060]: pgmap v109: 129 pgs: 129 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 285 op/s 2024-03-15T13:02:27.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:26 smithi184 ceph-mon[35993]: pgmap v109: 129 pgs: 129 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 285 op/s 2024-03-15T13:02:27.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:26 smithi149 ceph-mon[35927]: pgmap v109: 129 pgs: 129 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 285 op/s 2024-03-15T13:02:27.447 INFO:tasks.workunit:Stopping ['fs/norstats'] on client.0... 2024-03-15T13:02:27.447 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-15T13:02:27.856 DEBUG:teuthology.parallel:result is None 2024-03-15T13:02:27.856 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-15T13:02:27.894 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-15T13:02:27.895 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-15T13:02:27.908 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-15T13:02:27.908 INFO:teuthology.task.exec:Executing custom commands... 2024-03-15T13:02:27.908 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-15T13:02:27.908 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-15T13:02:27.963 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-15T13:02:29.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:28 smithi002 ceph-mon[32060]: pgmap v110: 129 pgs: 129 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 966 KiB/s wr, 195 op/s 2024-03-15T13:02:29.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:28 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:02:29.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:28 smithi184 ceph-mon[35993]: pgmap v110: 129 pgs: 129 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 966 KiB/s wr, 195 op/s 2024-03-15T13:02:29.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:28 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:02:29.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:28 smithi149 ceph-mon[35927]: pgmap v110: 129 pgs: 129 active+clean; 262 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 966 KiB/s wr, 195 op/s 2024-03-15T13:02:29.362 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:28 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:02:31.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:30 smithi002 ceph-mon[32060]: pgmap v111: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 298 op/s 2024-03-15T13:02:31.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:30 smithi184 ceph-mon[35993]: pgmap v111: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 298 op/s 2024-03-15T13:02:31.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:30 smithi149 ceph-mon[35927]: pgmap v111: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 298 op/s 2024-03-15T13:02:32.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:02:32.553 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:02:32.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.685+0000 7f23e8610640 1 -- 172.21.15.2:0/1336402959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e0106480 msgr2=0x7f23e0106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:32.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.685+0000 7f23e8610640 1 --2- 172.21.15.2:0/1336402959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e0106480 0x7f23e0106860 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009900 tx=0x7f23dc02bc30 comp rx=0 tx=0).stop 2024-03-15T13:02:32.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.685+0000 7f23e8610640 1 -- 172.21.15.2:0/1336402959 shutdown_connections 2024-03-15T13:02:32.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.685+0000 7f23e8610640 1 --2- 172.21.15.2:0/1336402959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e0101000 0x7f23e010d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.685+0000 7f23e8610640 1 --2- 172.21.15.2:0/1336402959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e0100640 0x7f23e0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.685+0000 7f23e8610640 1 --2- 172.21.15.2:0/1336402959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e0106480 0x7f23e0106860 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.685+0000 7f23e8610640 1 -- 172.21.15.2:0/1336402959 >> 172.21.15.2:0/1336402959 conn(0x7f23e0077190 msgr2=0x7f23e00775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:32.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.685+0000 7f23e8610640 1 -- 172.21.15.2:0/1336402959 shutdown_connections 2024-03-15T13:02:32.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.685+0000 7f23e8610640 1 -- 172.21.15.2:0/1336402959 wait complete. 2024-03-15T13:02:32.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.686+0000 7f23e8610640 1 Processor -- start 2024-03-15T13:02:32.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.686+0000 7f23e8610640 1 -- start start 2024-03-15T13:02:32.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.686+0000 7f23e8610640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e0100640 0x7f23e019fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.686+0000 7f23e8610640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e0101000 0x7f23e019ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.686+0000 7f23e8610640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e0106480 0x7f23e0199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.686+0000 7f23e8610640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e019a2b0 con 0x7f23e0106480 2024-03-15T13:02:32.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.686+0000 7f23e8610640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e019a420 con 0x7f23e0100640 2024-03-15T13:02:32.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.686+0000 7f23e8610640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e019a5e0 con 0x7f23e0101000 2024-03-15T13:02:32.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.686+0000 7f23e6bad640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e0106480 0x7f23e0199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.686+0000 7f23e5bab640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e0101000 0x7f23e019ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.686+0000 7f23e63ac640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e0100640 0x7f23e019fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.686+0000 7f23e5bab640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e0101000 0x7f23e019ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:36210/0 (socket says 172.21.15.2:36210) 2024-03-15T13:02:32.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.686+0000 7f23e5bab640 1 -- 172.21.15.2:0/4045770827 learned_addr learned my addr 172.21.15.2:0/4045770827 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:32.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.686+0000 7f23e6bad640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e0106480 0x7f23e0199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56558/0 (socket says 172.21.15.2:56558) 2024-03-15T13:02:32.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.687+0000 7f23e6bad640 1 -- 172.21.15.2:0/4045770827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e0101000 msgr2=0x7f23e019ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:32.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.687+0000 7f23e6bad640 1 --2- 172.21.15.2:0/4045770827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e0101000 0x7f23e019ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.687+0000 7f23e6bad640 1 -- 172.21.15.2:0/4045770827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e0100640 msgr2=0x7f23e019fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:32.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.687+0000 7f23e6bad640 1 --2- 172.21.15.2:0/4045770827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e0100640 0x7f23e019fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.687+0000 7f23e6bad640 1 -- 172.21.15.2:0/4045770827 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc009560 con 0x7f23e0106480 2024-03-15T13:02:32.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.687+0000 7f23e5bab640 1 --2- 172.21.15.2:0/4045770827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e0101000 0x7f23e019ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:32.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.687+0000 7f23e6bad640 1 --2- 172.21.15.2:0/4045770827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e0106480 0x7f23e0199b20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009fd0 tx=0x7f23d800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:32.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.687+0000 7f23d77fe640 1 -- 172.21.15.2:0/4045770827 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800ce20 con 0x7f23e0106480 2024-03-15T13:02:32.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.687+0000 7f23d77fe640 1 -- 172.21.15.2:0/4045770827 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f23d8005020 con 0x7f23e0106480 2024-03-15T13:02:32.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.687+0000 7f23e8610640 1 -- 172.21.15.2:0/4045770827 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e019a8c0 con 0x7f23e0106480 2024-03-15T13:02:32.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.687+0000 7f23d77fe640 1 -- 172.21.15.2:0/4045770827 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8010690 con 0x7f23e0106480 2024-03-15T13:02:32.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.687+0000 7f23e8610640 1 -- 172.21.15.2:0/4045770827 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e01a69e0 con 0x7f23e0106480 2024-03-15T13:02:32.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.689+0000 7f23d77fe640 1 -- 172.21.15.2:0/4045770827 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f23d8005570 con 0x7f23e0106480 2024-03-15T13:02:32.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.690+0000 7f23eb1ea740 1 Processor -- start 2024-03-15T13:02:32.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.690+0000 7f23eb1ea740 1 -- start start 2024-03-15T13:02:32.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23eb1ea740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a22b0 0x55cff00a2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23eb1ea740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a2bd0 0x55cff00ab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23eb1ea740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00ab7b0 0x55cff00adba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23eb1ea740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff0011280 con 0x55cff00a22b0 2024-03-15T13:02:32.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23eb1ea740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff000e7b0 con 0x55cff00a2bd0 2024-03-15T13:02:32.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23eb1ea740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff0009f90 con 0x55cff00ab7b0 2024-03-15T13:02:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23d77fe640 1 --2- 172.21.15.2:0/4045770827 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f23cc0a1ba0 0x7f23cc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23e63ac640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a2bd0 0x55cff00ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23e5bab640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a22b0 0x55cff00a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23e63ac640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a2bd0 0x55cff00ab270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55262/0 (socket says 172.21.15.2:55262) 2024-03-15T13:02:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23e63ac640 1 -- 172.21.15.2:0/2485593640 learned_addr learned my addr 172.21.15.2:0/2485593640 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23e5bab640 1 --2- 172.21.15.2:0/2485593640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00ab7b0 0x55cff00adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23d77fe640 1 -- 172.21.15.2:0/4045770827 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f23d8014070 con 0x7f23e0106480 2024-03-15T13:02:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23e5bab640 1 -- 172.21.15.2:0/2485593640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00ab7b0 msgr2=0x55cff00adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23e5bab640 1 --2- 172.21.15.2:0/2485593640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00ab7b0 0x55cff00adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23e5bab640 1 -- 172.21.15.2:0/2485593640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a2bd0 msgr2=0x55cff00ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23e5bab640 1 --2- 172.21.15.2:0/2485593640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a2bd0 0x55cff00ab270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23e6bad640 1 --2- 172.21.15.2:0/4045770827 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f23cc0a1ba0 0x7f23cc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23e5bab640 1 -- 172.21.15.2:0/2485593640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfeffc8570 con 0x55cff00a22b0 2024-03-15T13:02:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23e63ac640 1 --2- 172.21.15.2:0/2485593640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a2bd0 0x55cff00ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.691+0000 7f23e5bab640 1 --2- 172.21.15.2:0/2485593640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a22b0 0x55cff00a2690 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f23e019b300 tx=0x7f23d00095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d21026f4fcfc69 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:32.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.692+0000 7f23d4bfe640 1 -- 172.21.15.2:0/2485593640 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0013ce0 con 0x55cff00a22b0 2024-03-15T13:02:32.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.692+0000 7f23e6bad640 1 --2- 172.21.15.2:0/4045770827 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f23cc0a1ba0 0x7f23cc0a4060 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f23d800c760 tx=0x7f23d8005870 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:32.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.692+0000 7f23d4bfe640 1 -- 172.21.15.2:0/2485593640 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f23d0004500 con 0x55cff00a22b0 2024-03-15T13:02:32.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.692+0000 7f23d4bfe640 1 -- 172.21.15.2:0/2485593640 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d00055d0 con 0x55cff00a22b0 2024-03-15T13:02:32.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.692+0000 7f23eb1ea740 1 -- 172.21.15.2:0/2485593640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a22b0 msgr2=0x55cff00a2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:32.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.692+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/2485593640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a22b0 0x55cff00a2690 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f23e019b300 tx=0x7f23d00095b0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.692+0000 7f23eb1ea740 1 -- 172.21.15.2:0/2485593640 shutdown_connections 2024-03-15T13:02:32.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.692+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/2485593640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00ab7b0 0x55cff00adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.692+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/2485593640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a2bd0 0x55cff00ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.692+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/2485593640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a22b0 0x55cff00a2690 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.692+0000 7f23eb1ea740 1 -- 172.21.15.2:0/2485593640 >> 172.21.15.2:0/2485593640 conn(0x55cff009c990 msgr2=0x55cff009cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:32.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23eb1ea740 1 -- 172.21.15.2:0/2485593640 shutdown_connections 2024-03-15T13:02:32.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23eb1ea740 1 -- 172.21.15.2:0/2485593640 wait complete. 2024-03-15T13:02:32.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23eb1ea740 1 -- start start 2024-03-15T13:02:32.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23eb1ea740 1 Processor -- start 2024-03-15T13:02:32.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23eb1ea740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a22b0 0x55cff00ab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23eb1ea740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a2bd0 0x55cff00a85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23eb1ea740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00ab7b0 0x55cff00a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23eb1ea740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfefcdd430 con 0x55cff00a22b0 2024-03-15T13:02:32.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23eb1ea740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfefc99e00 con 0x55cff00a2bd0 2024-03-15T13:02:32.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23eb1ea740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfeff8efd0 con 0x55cff00ab7b0 2024-03-15T13:02:32.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23eb1ea740 10 client.0 _ll_register_callbacks cb 0x55cfeffe4fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:32.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23e5bab640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a22b0 0x55cff00ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23e5bab640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a22b0 0x55cff00ab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56572/0 (socket says 172.21.15.2:56572) 2024-03-15T13:02:32.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23e5bab640 1 -- 172.21.15.2:0/834343425 learned_addr learned my addr 172.21.15.2:0/834343425 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:32.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23e63ac640 1 --2- 172.21.15.2:0/834343425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a2bd0 0x55cff00a85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.693+0000 7f23e5bab640 1 --2- 172.21.15.2:0/834343425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00ab7b0 0x55cff00a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23e5bab640 1 -- 172.21.15.2:0/834343425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00ab7b0 msgr2=0x55cff00a8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:32.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23e5bab640 1 --2- 172.21.15.2:0/834343425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00ab7b0 0x55cff00a8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23e5bab640 1 -- 172.21.15.2:0/834343425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a2bd0 msgr2=0x55cff00a85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:32.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23e5bab640 1 --2- 172.21.15.2:0/834343425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a2bd0 0x55cff00a85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23e5bab640 1 -- 172.21.15.2:0/834343425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d001b040 con 0x55cff00a22b0 2024-03-15T13:02:32.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23e5bab640 1 --2- 172.21.15.2:0/834343425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a22b0 0x55cff00ab250 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f23d0006010 tx=0x7f23d00137c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67297590095a6c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:32.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:32.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23c9ffb640 1 -- 172.21.15.2:0/834343425 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d00048d0 con 0x55cff00a22b0 2024-03-15T13:02:32.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23c9ffb640 1 -- 172.21.15.2:0/834343425 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f23d0004a30 con 0x55cff00a22b0 2024-03-15T13:02:32.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23eb1ea740 1 -- 172.21.15.2:0/834343425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfefd46720 con 0x55cff00a22b0 2024-03-15T13:02:32.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23c9ffb640 1 -- 172.21.15.2:0/834343425 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23d0004bf0 con 0x55cff00a22b0 2024-03-15T13:02:32.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23eb1ea740 10 client.16737 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:32.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23eb1ea740 1 -- 172.21.15.2:0/834343425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55cff0008610 con 0x55cff00a22b0 2024-03-15T13:02:32.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23c9ffb640 1 -- 172.21.15.2:0/834343425 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55cfefd46720 con 0x55cff00a22b0 2024-03-15T13:02:32.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23c9ffb640 1 -- 172.21.15.2:0/834343425 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f23d0020d00 con 0x55cff00a22b0 2024-03-15T13:02:32.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23eb1ea740 10 client.16737 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:32.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23eb1ea740 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:02:32.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.694+0000 7f23eb1ea740 20 client.16737 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:32.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.695+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/834343425 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cff00bb960 0x55cff00bdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.695+0000 7f23eb1ea740 4 client.16737 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:02:32.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.695+0000 7f23eb1ea740 1 -- 172.21.15.2:0/834343425 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfefd95890 con 0x55cff00bb960 2024-03-15T13:02:32.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.695+0000 7f23e63ac640 1 --2- 172.21.15.2:0/834343425 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cff00bb960 0x55cff00bdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.695+0000 7f23c9ffb640 10 client.16737 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:32.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.695+0000 7f23e63ac640 1 --2- 172.21.15.2:0/834343425 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cff00bb960 0x55cff00bdd80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a5e82f27c544e4c server_cookie=653ac28854898364 in_seq=0 out_seq=0 2024-03-15T13:02:32.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.696+0000 7f23c9ffb640 1 -- 172.21.15.2:0/834343425 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cfefd95890 con 0x55cff00bb960 2024-03-15T13:02:32.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.696+0000 7f23c9ffb640 10 client.16737 handle_command_reply: tid=0 2024-03-15T13:02:32.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.696+0000 7f23eb1ea740 1 client.16737 shutdown 2024-03-15T13:02:32.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.696+0000 7f23eb1ea740 1 -- 172.21.15.2:0/834343425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a22b0 msgr2=0x55cff00ab250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:32.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.696+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/834343425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a22b0 0x55cff00ab250 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f23d0006010 tx=0x7f23d00137c0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.696+0000 7f23eb1ea740 1 -- 172.21.15.2:0/834343425 shutdown_connections 2024-03-15T13:02:32.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.696+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/834343425 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cff00bb960 0x55cff00bdd80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.696+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/834343425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00ab7b0 0x55cff00a8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.696+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/834343425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a2bd0 0x55cff00a85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.696+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/834343425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a22b0 0x55cff00ab250 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.696+0000 7f23c9ffb640 0 client.16737 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:32.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.696+0000 7f23eb1ea740 1 -- 172.21.15.2:0/834343425 >> 172.21.15.2:0/834343425 conn(0x55cff009c990 msgr2=0x55cff009e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:32.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.696+0000 7f23eb1ea740 1 -- 172.21.15.2:0/834343425 shutdown_connections 2024-03-15T13:02:32.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.696+0000 7f23eb1ea740 1 -- 172.21.15.2:0/834343425 wait complete. 2024-03-15T13:02:32.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.696+0000 7f23eb1ea740 20 client.16737 trim_cache size 0 max 16384 2024-03-15T13:02:32.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.712+0000 7f23eb1ea740 1 Processor -- start 2024-03-15T13:02:32.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23eb1ea740 1 -- start start 2024-03-15T13:02:32.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23eb1ea740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a22b0 0x55cff00a8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23eb1ea740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a2bd0 0x55cff00aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23eb1ea740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00ab7b0 0x55cff00ab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23eb1ea740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfefd95890 con 0x55cff00a2bd0 2024-03-15T13:02:32.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23eb1ea740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfefd46720 con 0x55cff00a22b0 2024-03-15T13:02:32.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23eb1ea740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff009d9e0 con 0x55cff00ab7b0 2024-03-15T13:02:32.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23e63ac640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a2bd0 0x55cff00aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23e63ac640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a2bd0 0x55cff00aad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56586/0 (socket says 172.21.15.2:56586) 2024-03-15T13:02:32.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23e63ac640 1 -- 172.21.15.2:0/2484153248 learned_addr learned my addr 172.21.15.2:0/2484153248 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:32.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23e5bab640 1 --2- 172.21.15.2:0/2484153248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a22b0 0x55cff00a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23e5bab640 1 --2- 172.21.15.2:0/2484153248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00ab7b0 0x55cff00ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23e63ac640 1 -- 172.21.15.2:0/2484153248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00ab7b0 msgr2=0x55cff00ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:32.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23e63ac640 1 --2- 172.21.15.2:0/2484153248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00ab7b0 0x55cff00ab250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23e63ac640 1 -- 172.21.15.2:0/2484153248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a22b0 msgr2=0x55cff00a8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:32.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23e63ac640 1 --2- 172.21.15.2:0/2484153248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a22b0 0x55cff00a8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23e63ac640 1 -- 172.21.15.2:0/2484153248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfeffa4ac0 con 0x55cff00a2bd0 2024-03-15T13:02:32.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23e63ac640 1 --2- 172.21.15.2:0/2484153248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a2bd0 0x55cff00aad10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009230 tx=0x7f23dc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d151f959766ca7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:32.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.713+0000 7f23cbfff640 1 -- 172.21.15.2:0/2484153248 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc04a330 con 0x55cff00a2bd0 2024-03-15T13:02:32.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.714+0000 7f23cbfff640 1 -- 172.21.15.2:0/2484153248 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f23dc004510 con 0x55cff00a2bd0 2024-03-15T13:02:32.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.714+0000 7f23cbfff640 1 -- 172.21.15.2:0/2484153248 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc04a860 con 0x55cff00a2bd0 2024-03-15T13:02:32.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.714+0000 7f23eb1ea740 1 -- 172.21.15.2:0/2484153248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a2bd0 msgr2=0x55cff00aad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:32.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.714+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/2484153248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a2bd0 0x55cff00aad10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009230 tx=0x7f23dc009260 comp rx=0 tx=0).stop 2024-03-15T13:02:32.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.714+0000 7f23eb1ea740 1 -- 172.21.15.2:0/2484153248 shutdown_connections 2024-03-15T13:02:32.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.714+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/2484153248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00ab7b0 0x55cff00ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.714+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/2484153248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a2bd0 0x55cff00aad10 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.714+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/2484153248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00a22b0 0x55cff00a8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.714+0000 7f23eb1ea740 1 -- 172.21.15.2:0/2484153248 >> 172.21.15.2:0/2484153248 conn(0x55cff009c990 msgr2=0x55cff00a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:32.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.714+0000 7f23eb1ea740 1 -- 172.21.15.2:0/2484153248 shutdown_connections 2024-03-15T13:02:32.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.714+0000 7f23eb1ea740 1 -- 172.21.15.2:0/2484153248 wait complete. 2024-03-15T13:02:32.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.714+0000 7f23eb1ea740 1 -- start start 2024-03-15T13:02:32.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23eb1ea740 1 Processor -- start 2024-03-15T13:02:32.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23eb1ea740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00a22b0 0x55cff00b3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23eb1ea740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a2bd0 0x55cff00acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23eb1ea740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00ab7b0 0x55cff00ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23eb1ea740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff00ad5f0 con 0x55cff00a2bd0 2024-03-15T13:02:32.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23eb1ea740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff00ad760 con 0x55cff00ab7b0 2024-03-15T13:02:32.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23eb1ea740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cff00ad920 con 0x55cff00a22b0 2024-03-15T13:02:32.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23eb1ea740 10 client.0 _ll_register_callbacks cb 0x7f23d80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:32.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23e63ac640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a2bd0 0x55cff00acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23e5bab640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00a22b0 0x55cff00b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23e63ac640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a2bd0 0x55cff00acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56588/0 (socket says 172.21.15.2:56588) 2024-03-15T13:02:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23e63ac640 1 -- 172.21.15.2:0/102308326 learned_addr learned my addr 172.21.15.2:0/102308326 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23e5bab640 1 --2- 172.21.15.2:0/102308326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00ab7b0 0x55cff00ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23e63ac640 1 -- 172.21.15.2:0/102308326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00a22b0 msgr2=0x55cff00b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23e63ac640 1 --2- 172.21.15.2:0/102308326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00a22b0 0x55cff00b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23e63ac640 1 -- 172.21.15.2:0/102308326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00ab7b0 msgr2=0x55cff00ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23e63ac640 1 --2- 172.21.15.2:0/102308326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00ab7b0 0x55cff00ad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23e63ac640 1 -- 172.21.15.2:0/102308326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23dc030070 con 0x55cff00a2bd0 2024-03-15T13:02:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23e63ac640 1 --2- 172.21.15.2:0/102308326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a2bd0 0x55cff00acb70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f23dc04d000 tx=0x7f23dc04a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4978c22e88473ee7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23c9ffb640 1 -- 172.21.15.2:0/102308326 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc02cc20 con 0x55cff00a2bd0 2024-03-15T13:02:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23c9ffb640 1 -- 172.21.15.2:0/102308326 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f23dc046040 con 0x55cff00a2bd0 2024-03-15T13:02:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.715+0000 7f23eb1ea740 1 -- 172.21.15.2:0/102308326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfefd097e0 con 0x55cff00a2bd0 2024-03-15T13:02:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.716+0000 7f23c9ffb640 1 -- 172.21.15.2:0/102308326 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23dc04a600 con 0x55cff00a2bd0 2024-03-15T13:02:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.716+0000 7f23eb1ea740 10 client.16743 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.716+0000 7f23eb1ea740 1 -- 172.21.15.2:0/102308326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55cfeffca6a0 con 0x55cff00a2bd0 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.716+0000 7f23c9ffb640 1 -- 172.21.15.2:0/102308326 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55cfefd097e0 con 0x55cff00a2bd0 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.716+0000 7f23c9ffb640 1 -- 172.21.15.2:0/102308326 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f23dc04a050 con 0x55cff00a2bd0 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.716+0000 7f23eb1ea740 10 client.16743 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.716+0000 7f23eb1ea740 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.716+0000 7f23eb1ea740 20 client.16743 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.716+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/102308326 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cff01184d0 0x55cff011a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.716+0000 7f23eb1ea740 4 client.16743 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.716+0000 7f23eb1ea740 1 -- 172.21.15.2:0/102308326 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cfefcf84e0 con 0x55cff01184d0 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.716+0000 7f23e5bab640 1 --2- 172.21.15.2:0/102308326 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cff01184d0 0x55cff011a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23c9ffb640 10 client.16743 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23e5bab640 1 --2- 172.21.15.2:0/102308326 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cff01184d0 0x55cff011a8f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4607ac0dab43f2d5 server_cookie=d702c9fd142e472b in_seq=0 out_seq=0 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23c9ffb640 1 -- 172.21.15.2:0/102308326 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cfefcf84e0 con 0x55cff01184d0 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23c9ffb640 10 client.16743 handle_command_reply: tid=0 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23eb1ea740 1 client.16743 shutdown 2024-03-15T13:02:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23eb1ea740 1 -- 172.21.15.2:0/102308326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a2bd0 msgr2=0x55cff00acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/102308326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a2bd0 0x55cff00acb70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f23dc04d000 tx=0x7f23dc04a3b0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23eb1ea740 1 -- 172.21.15.2:0/102308326 shutdown_connections 2024-03-15T13:02:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/102308326 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cff01184d0 0x55cff011a8f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/102308326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff00ab7b0 0x55cff00ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23c9ffb640 0 client.16743 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/102308326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cff00a2bd0 0x55cff00acb70 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23eb1ea740 1 --2- 172.21.15.2:0/102308326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cff00a22b0 0x55cff00b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23eb1ea740 1 -- 172.21.15.2:0/102308326 >> 172.21.15.2:0/102308326 conn(0x55cff009c990 msgr2=0x55cff009b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23eb1ea740 1 -- 172.21.15.2:0/102308326 shutdown_connections 2024-03-15T13:02:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23eb1ea740 1 -- 172.21.15.2:0/102308326 wait complete. 2024-03-15T13:02:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.717+0000 7f23eb1ea740 20 client.16743 trim_cache size 0 max 16384 2024-03-15T13:02:32.710 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:02:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:02:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:02:32.710 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:02:32.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.718+0000 7f23e8610640 1 -- 172.21.15.2:0/4045770827 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f23cc0a1ba0 msgr2=0x7f23cc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:32.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.718+0000 7f23e8610640 1 --2- 172.21.15.2:0/4045770827 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f23cc0a1ba0 0x7f23cc0a4060 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f23d800c760 tx=0x7f23d8005870 comp rx=0 tx=0).stop 2024-03-15T13:02:32.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.718+0000 7f23e8610640 1 -- 172.21.15.2:0/4045770827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e0106480 msgr2=0x7f23e0199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:32.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.718+0000 7f23e8610640 1 --2- 172.21.15.2:0/4045770827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e0106480 0x7f23e0199b20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009fd0 tx=0x7f23d800ef10 comp rx=0 tx=0).stop 2024-03-15T13:02:32.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.718+0000 7f23e8610640 1 -- 172.21.15.2:0/4045770827 shutdown_connections 2024-03-15T13:02:32.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.718+0000 7f23e8610640 1 --2- 172.21.15.2:0/4045770827 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f23cc0a1ba0 0x7f23cc0a4060 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.718+0000 7f23e8610640 1 --2- 172.21.15.2:0/4045770827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23e0106480 0x7f23e0199b20 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.718+0000 7f23e8610640 1 --2- 172.21.15.2:0/4045770827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23e0101000 0x7f23e019ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.718+0000 7f23e8610640 1 --2- 172.21.15.2:0/4045770827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23e0100640 0x7f23e019fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.718+0000 7f23e8610640 1 -- 172.21.15.2:0/4045770827 >> 172.21.15.2:0/4045770827 conn(0x7f23e0077190 msgr2=0x7f23e0069b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:32.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.718+0000 7f23e8610640 1 -- 172.21.15.2:0/4045770827 shutdown_connections 2024-03-15T13:02:32.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.718+0000 7f23e8610640 1 -- 172.21.15.2:0/4045770827 wait complete. 2024-03-15T13:02:32.714 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:02:32.714 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:02:32.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.832+0000 7f4d8370d640 1 -- 172.21.15.2:0/921891047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d7c100db0 msgr2=0x7f4d7c10d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.832+0000 7f4d8370d640 1 --2- 172.21.15.2:0/921891047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d7c100db0 0x7f4d7c10d2c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009cd0 tx=0x7f4d7402c0f0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.832+0000 7f4d8370d640 1 -- 172.21.15.2:0/921891047 shutdown_connections 2024-03-15T13:02:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.832+0000 7f4d8370d640 1 --2- 172.21.15.2:0/921891047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d7c100db0 0x7f4d7c10d2c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.832+0000 7f4d8370d640 1 --2- 172.21.15.2:0/921891047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d7c1003f0 0x7f4d7c100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.832+0000 7f4d8370d640 1 --2- 172.21.15.2:0/921891047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d7c106490 0x7f4d7c106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.832+0000 7f4d8370d640 1 -- 172.21.15.2:0/921891047 >> 172.21.15.2:0/921891047 conn(0x7f4d7c0774f0 msgr2=0x7f4d7c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.832+0000 7f4d8370d640 1 -- 172.21.15.2:0/921891047 shutdown_connections 2024-03-15T13:02:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.832+0000 7f4d8370d640 1 -- 172.21.15.2:0/921891047 wait complete. 2024-03-15T13:02:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.832+0000 7f4d8370d640 1 Processor -- start 2024-03-15T13:02:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.832+0000 7f4d8370d640 1 -- start start 2024-03-15T13:02:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d8370d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d7c1003f0 0x7f4d7c19d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d8370d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d7c100db0 0x7f4d7c19dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d8370d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d7c106490 0x7f4d7c197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d8370d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c197f80 con 0x7f4d7c100db0 2024-03-15T13:02:32.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d8370d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c1980f0 con 0x7f4d7c106490 2024-03-15T13:02:32.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d8370d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c1982b0 con 0x7f4d7c1003f0 2024-03-15T13:02:32.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d80ca8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d7c100db0 0x7f4d7c19dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d80ca8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d7c100db0 0x7f4d7c19dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56614/0 (socket says 172.21.15.2:56614) 2024-03-15T13:02:32.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d80ca8640 1 -- 172.21.15.2:0/88257928 learned_addr learned my addr 172.21.15.2:0/88257928 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:32.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d81caa640 1 --2- 172.21.15.2:0/88257928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d7c106490 0x7f4d7c197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d814a9640 1 --2- 172.21.15.2:0/88257928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d7c1003f0 0x7f4d7c19d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d80ca8640 1 -- 172.21.15.2:0/88257928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d7c1003f0 msgr2=0x7f4d7c19d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:32.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d80ca8640 1 --2- 172.21.15.2:0/88257928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d7c1003f0 0x7f4d7c19d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d80ca8640 1 -- 172.21.15.2:0/88257928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d7c106490 msgr2=0x7f4d7c197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:32.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d80ca8640 1 --2- 172.21.15.2:0/88257928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d7c106490 0x7f4d7c197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:32.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d80ca8640 1 -- 172.21.15.2:0/88257928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c009490 con 0x7f4d7c100db0 2024-03-15T13:02:32.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d80ca8640 1 --2- 172.21.15.2:0/88257928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d7c100db0 0x7f4d7c19dcc0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c00ea60 tx=0x7f4d6c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:32.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d727fc640 1 -- 172.21.15.2:0/88257928 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c00cde0 con 0x7f4d7c100db0 2024-03-15T13:02:32.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d8370d640 1 -- 172.21.15.2:0/88257928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d78009490 con 0x7f4d7c100db0 2024-03-15T13:02:32.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d727fc640 1 -- 172.21.15.2:0/88257928 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4d6c0035f0 con 0x7f4d7c100db0 2024-03-15T13:02:32.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d727fc640 1 -- 172.21.15.2:0/88257928 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c010690 con 0x7f4d7c100db0 2024-03-15T13:02:32.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.833+0000 7f4d8370d640 1 -- 172.21.15.2:0/88257928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d74009900 con 0x7f4d7c100db0 2024-03-15T13:02:32.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.834+0000 7f4d8370d640 1 -- 172.21.15.2:0/88257928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7c0682e0 con 0x7f4d7c100db0 2024-03-15T13:02:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.836+0000 7f4d727fc640 1 -- 172.21.15.2:0/88257928 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f4d6c003760 con 0x7f4d7c100db0 2024-03-15T13:02:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.837+0000 7f4d727fc640 1 --2- 172.21.15.2:0/88257928 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f4d680a1ba0 0x7f4d680a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.837+0000 7f4d814a9640 1 --2- 172.21.15.2:0/88257928 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f4d680a1ba0 0x7f4d680a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.837+0000 7f4d727fc640 1 -- 172.21.15.2:0/88257928 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f4d6c014070 con 0x7f4d7c100db0 2024-03-15T13:02:32.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.837+0000 7f4d814a9640 1 --2- 172.21.15.2:0/88257928 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f4d680a1ba0 0x7f4d680a4060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c198a70 tx=0x7f4d78005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:32.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:32.840+0000 7f4d727fc640 1 -- 172.21.15.2:0/88257928 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4d6c096d00 con 0x7f4d7c100db0 2024-03-15T13:02:33.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.061+0000 7f4d8370d640 1 -- 172.21.15.2:0/88257928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d7c1089f0 con 0x7f4d7c100db0 2024-03-15T13:02:33.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.062+0000 7f4d727fc640 1 -- 172.21.15.2:0/88257928 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+11472 (secure 0 0 0) 0x7f4d7c1089f0 con 0x7f4d7c100db0 2024-03-15T13:02:33.047 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:02:33.048 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:02:27.136969+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24500,24464,24460,24434,24539]},"id":1}]} 2024-03-15T13:02:33.048 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 38 2024-03-15T13:02:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.064+0000 7f4d8370d640 1 -- 172.21.15.2:0/88257928 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f4d680a1ba0 msgr2=0x7f4d680a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.064+0000 7f4d8370d640 1 --2- 172.21.15.2:0/88257928 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f4d680a1ba0 0x7f4d680a4060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c198a70 tx=0x7f4d78005f90 comp rx=0 tx=0).stop 2024-03-15T13:02:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.064+0000 7f4d8370d640 1 -- 172.21.15.2:0/88257928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d7c100db0 msgr2=0x7f4d7c19dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.064+0000 7f4d8370d640 1 --2- 172.21.15.2:0/88257928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d7c100db0 0x7f4d7c19dcc0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c00ea60 tx=0x7f4d6c00ef20 comp rx=0 tx=0).stop 2024-03-15T13:02:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.064+0000 7f4d8370d640 1 -- 172.21.15.2:0/88257928 shutdown_connections 2024-03-15T13:02:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.064+0000 7f4d8370d640 1 --2- 172.21.15.2:0/88257928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d7c106490 0x7f4d7c197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.064+0000 7f4d8370d640 1 --2- 172.21.15.2:0/88257928 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f4d680a1ba0 0x7f4d680a4060 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.064+0000 7f4d8370d640 1 --2- 172.21.15.2:0/88257928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d7c100db0 0x7f4d7c19dcc0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.064+0000 7f4d8370d640 1 --2- 172.21.15.2:0/88257928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d7c1003f0 0x7f4d7c19d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.064+0000 7f4d8370d640 1 -- 172.21.15.2:0/88257928 >> 172.21.15.2:0/88257928 conn(0x7f4d7c0774f0 msgr2=0x7f4d7c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.064+0000 7f4d8370d640 1 -- 172.21.15.2:0/88257928 shutdown_connections 2024-03-15T13:02:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.064+0000 7f4d8370d640 1 -- 172.21.15.2:0/88257928 wait complete. 2024-03-15T13:02:33.061 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:02:33.061 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:02:33.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.180+0000 7f82a1180640 1 -- 172.21.15.2:0/979728362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f829c100610 msgr2=0x7f829c1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.180+0000 7f82a1180640 1 --2- 172.21.15.2:0/979728362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f829c100610 0x7f829c1009f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8294009cd0 tx=0x7f829402c0f0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.180+0000 7f82a1180640 1 -- 172.21.15.2:0/979728362 shutdown_connections 2024-03-15T13:02:33.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.180+0000 7f82a1180640 1 --2- 172.21.15.2:0/979728362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f829c1056e0 0x7f829c10d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.180+0000 7f82a1180640 1 --2- 172.21.15.2:0/979728362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f829c100fc0 0x7f829c105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.180+0000 7f82a1180640 1 --2- 172.21.15.2:0/979728362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f829c100610 0x7f829c1009f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.180+0000 7f82a1180640 1 -- 172.21.15.2:0/979728362 >> 172.21.15.2:0/979728362 conn(0x7f829c076770 msgr2=0x7f829c076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.180+0000 7f82a1180640 1 -- 172.21.15.2:0/979728362 shutdown_connections 2024-03-15T13:02:33.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.180+0000 7f82a1180640 1 -- 172.21.15.2:0/979728362 wait complete. 2024-03-15T13:02:33.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.181+0000 7f82a1180640 1 Processor -- start 2024-03-15T13:02:33.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.181+0000 7f82a1180640 1 -- start start 2024-03-15T13:02:33.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.181+0000 7f82a1180640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f829c100610 0x7f829c19d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.181+0000 7f82a1180640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f829c1056e0 0x7f829c19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.181+0000 7f82a1180640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f829c197990 0x7f829c197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.181+0000 7f82a1180640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c198380 con 0x7f829c197990 2024-03-15T13:02:33.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.181+0000 7f82a1180640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c1984f0 con 0x7f829c100610 2024-03-15T13:02:33.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.181+0000 7f82a1180640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c1986b0 con 0x7f829c1056e0 2024-03-15T13:02:33.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.181+0000 7f829b59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f829c197990 0x7f829c197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.181+0000 7f829ad9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f829c100610 0x7f829c19d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.181+0000 7f829ad9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f829c100610 0x7f829c19d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55312/0 (socket says 172.21.15.2:55312) 2024-03-15T13:02:33.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.181+0000 7f829ad9d640 1 -- 172.21.15.2:0/4030862943 learned_addr learned my addr 172.21.15.2:0/4030862943 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.181+0000 7f829a59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f829c1056e0 0x7f829c19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.182+0000 7f829a59c640 1 -- 172.21.15.2:0/4030862943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f829c100610 msgr2=0x7f829c19d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.182+0000 7f829a59c640 1 --2- 172.21.15.2:0/4030862943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f829c100610 0x7f829c19d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.182+0000 7f829a59c640 1 -- 172.21.15.2:0/4030862943 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f829c197990 msgr2=0x7f829c197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.182+0000 7f829a59c640 1 --2- 172.21.15.2:0/4030862943 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f829c197990 0x7f829c197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.182+0000 7f829a59c640 1 -- 172.21.15.2:0/4030862943 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8294009900 con 0x7f829c1056e0 2024-03-15T13:02:33.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.182+0000 7f829b59e640 1 --2- 172.21.15.2:0/4030862943 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f829c197990 0x7f829c197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:33.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.182+0000 7f829a59c640 1 --2- 172.21.15.2:0/4030862943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f829c1056e0 0x7f829c19dd20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f829000e9c0 tx=0x7f829000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.182+0000 7f8283fff640 1 -- 172.21.15.2:0/4030862943 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829000ccc0 con 0x7f829c1056e0 2024-03-15T13:02:33.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.182+0000 7f82a1180640 1 -- 172.21.15.2:0/4030862943 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829c1a46a0 con 0x7f829c1056e0 2024-03-15T13:02:33.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.182+0000 7f8283fff640 1 -- 172.21.15.2:0/4030862943 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f829000ce20 con 0x7f829c1056e0 2024-03-15T13:02:33.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.182+0000 7f8283fff640 1 -- 172.21.15.2:0/4030862943 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290004a00 con 0x7f829c1056e0 2024-03-15T13:02:33.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.182+0000 7f82a1180640 1 -- 172.21.15.2:0/4030862943 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829c1a4bb0 con 0x7f829c1056e0 2024-03-15T13:02:33.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.184+0000 7f8283fff640 1 -- 172.21.15.2:0/4030862943 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f8290004b60 con 0x7f829c1056e0 2024-03-15T13:02:33.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.184+0000 7f82a3e30740 1 Processor -- start 2024-03-15T13:02:33.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f82a3e30740 1 -- start start 2024-03-15T13:02:33.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f8283fff640 1 --2- 172.21.15.2:0/4030862943 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f82840a1ba0 0x7f82840a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f82a3e30740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 0x55fe257eb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f82a3e30740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f829b59e640 1 --2- 172.21.15.2:0/4030862943 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f82840a1ba0 0x7f82840a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f82a3e30740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257f8b60 0x55fe257faf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f82a3e30740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe257a6960 con 0x55fe257eb820 2024-03-15T13:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f82a3e30740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe2575a310 con 0x55fe257f8b60 2024-03-15T13:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f82a3e30740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe25757840 con 0x55fe257eaf00 2024-03-15T13:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f8283fff640 1 -- 172.21.15.2:0/4030862943 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f8290014070 con 0x7f829c1056e0 2024-03-15T13:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f829ad9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 0x55fe257eb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f829a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f829ad9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 0x55fe257eb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:36300/0 (socket says 172.21.15.2:36300) 2024-03-15T13:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f829ad9d640 1 -- 172.21.15.2:0/3513300753 learned_addr learned my addr 172.21.15.2:0/3513300753 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f829b59e640 1 --2- 172.21.15.2:0/3513300753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257f8b60 0x55fe257faf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f829a59c640 1 -- 172.21.15.2:0/3513300753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 msgr2=0x55fe257eb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f829a59c640 1 --2- 172.21.15.2:0/3513300753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 0x55fe257eb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f829a59c640 1 -- 172.21.15.2:0/3513300753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257f8b60 msgr2=0x55fe257faf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f829a59c640 1 --2- 172.21.15.2:0/3513300753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257f8b60 0x55fe257faf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f829b59e640 1 --2- 172.21.15.2:0/4030862943 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f82840a1ba0 0x7f82840a4060 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f828c0117b0 tx=0x7f828c011bb0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f829a59c640 1 -- 172.21.15.2:0/3513300753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe25715060 con 0x55fe257eb820 2024-03-15T13:02:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f829ad9d640 1 --2- 172.21.15.2:0/3513300753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 0x55fe257eb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.185+0000 7f829a59c640 1 --2- 172.21.15.2:0/3513300753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f8620 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f82900caec0 tx=0x7f82900096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc6c365bd751f69 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.186+0000 7f82817fa640 1 -- 172.21.15.2:0/3513300753 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829009a3a0 con 0x55fe257eb820 2024-03-15T13:02:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.186+0000 7f82817fa640 1 -- 172.21.15.2:0/3513300753 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f829009e4f0 con 0x55fe257eb820 2024-03-15T13:02:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.186+0000 7f82817fa640 1 -- 172.21.15.2:0/3513300753 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290096a10 con 0x55fe257eb820 2024-03-15T13:02:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.186+0000 7f82a3e30740 1 -- 172.21.15.2:0/3513300753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 msgr2=0x55fe257f8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.186+0000 7f82a3e30740 1 --2- 172.21.15.2:0/3513300753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f8620 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f82900caec0 tx=0x7f82900096e0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.186+0000 7f82a3e30740 1 -- 172.21.15.2:0/3513300753 shutdown_connections 2024-03-15T13:02:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.186+0000 7f82a3e30740 1 --2- 172.21.15.2:0/3513300753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257f8b60 0x55fe257faf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.186+0000 7f82a3e30740 1 --2- 172.21.15.2:0/3513300753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f8620 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.186+0000 7f82a3e30740 1 --2- 172.21.15.2:0/3513300753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 0x55fe257eb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.186+0000 7f82a3e30740 1 -- 172.21.15.2:0/3513300753 >> 172.21.15.2:0/3513300753 conn(0x55fe257e5980 msgr2=0x55fe257e5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.186+0000 7f82a3e30740 1 -- 172.21.15.2:0/3513300753 shutdown_connections 2024-03-15T13:02:33.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.186+0000 7f82a3e30740 1 -- 172.21.15.2:0/3513300753 wait complete. 2024-03-15T13:02:33.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f82a3e30740 1 -- start start 2024-03-15T13:02:33.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f82a3e30740 1 Processor -- start 2024-03-15T13:02:33.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f82a3e30740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 0x55fe257f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f82a3e30740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f82a3e30740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257f8b60 0x55fe257f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f82a3e30740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe25753020 con 0x55fe257eb820 2024-03-15T13:02:33.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f82a3e30740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe25426430 con 0x55fe257f8b60 2024-03-15T13:02:33.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f82a3e30740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe253e2e00 con 0x55fe257eaf00 2024-03-15T13:02:33.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f82a3e30740 10 client.0 _ll_register_callbacks cb 0x55fe256facb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:33.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f829a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f829a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56652/0 (socket says 172.21.15.2:56652) 2024-03-15T13:02:33.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f829a59c640 1 -- 172.21.15.2:0/2420126694 learned_addr learned my addr 172.21.15.2:0/2420126694 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f829ad9d640 1 --2- 172.21.15.2:0/2420126694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 0x55fe257f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f829b59e640 1 --2- 172.21.15.2:0/2420126694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257f8b60 0x55fe257f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f829a59c640 1 -- 172.21.15.2:0/2420126694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 msgr2=0x55fe257f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f829a59c640 1 --2- 172.21.15.2:0/2420126694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 0x55fe257f84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f829a59c640 1 -- 172.21.15.2:0/2420126694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257f8b60 msgr2=0x55fe257f5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f829a59c640 1 --2- 172.21.15.2:0/2420126694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257f8b60 0x55fe257f5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.187+0000 7f829a59c640 1 -- 172.21.15.2:0/2420126694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829c1a4bb0 con 0x55fe257eb820 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f829a59c640 1 --2- 172.21.15.2:0/2420126694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f5830 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f82900caf20 tx=0x7f8290004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fe75358989eeba server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f827eff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f827eff5640 1 -- 172.21.15.2:0/2420126694 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82900a2b40 con 0x55fe257eb820 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f827eff5640 1 -- 172.21.15.2:0/2420126694 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f829009a3a0 con 0x55fe257eb820 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f82a3e30740 1 -- 172.21.15.2:0/2420126694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe254527e0 con 0x55fe257eb820 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f827eff5640 1 -- 172.21.15.2:0/2420126694 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8290096a10 con 0x55fe257eb820 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f82a3e30740 10 client.16761 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f82a3e30740 1 -- 172.21.15.2:0/2420126694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55fe257516a0 con 0x55fe257eb820 2024-03-15T13:02:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f827eff5640 1 -- 172.21.15.2:0/2420126694 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55fe254527e0 con 0x55fe257eb820 2024-03-15T13:02:33.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f827eff5640 1 -- 172.21.15.2:0/2420126694 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f8290083b20 con 0x55fe257eb820 2024-03-15T13:02:33.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f82a3e30740 10 client.16761 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:33.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f82a3e30740 10 client.16761 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:02:33.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f82a3e30740 20 client.16761 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:33.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f82a3e30740 1 --2- 172.21.15.2:0/2420126694 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55fe258088f0 0x55fe2580ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f82a3e30740 4 client.16761 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:02:33.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f82a3e30740 1 -- 172.21.15.2:0/2420126694 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe2548f720 con 0x55fe258088f0 2024-03-15T13:02:33.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.188+0000 7f829ad9d640 1 --2- 172.21.15.2:0/2420126694 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55fe258088f0 0x55fe2580ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.189+0000 7f829ad9d640 1 --2- 172.21.15.2:0/2420126694 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55fe258088f0 0x55fe2580ad10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1dde37b7d8bfa37a server_cookie=859c50659c55826f in_seq=0 out_seq=0 2024-03-15T13:02:33.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.189+0000 7f827eff5640 10 client.16761 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:02:33.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.190+0000 7f827eff5640 1 -- 172.21.15.2:0/2420126694 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fe2548f720 con 0x55fe258088f0 2024-03-15T13:02:33.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.190+0000 7f827eff5640 10 client.16761 handle_command_reply: tid=0 2024-03-15T13:02:33.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.190+0000 7f82a3e30740 1 client.16761 shutdown 2024-03-15T13:02:33.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.190+0000 7f82a3e30740 1 -- 172.21.15.2:0/2420126694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 msgr2=0x55fe257f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.190+0000 7f82a3e30740 1 --2- 172.21.15.2:0/2420126694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f5830 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f82900caf20 tx=0x7f8290004e80 comp rx=0 tx=0).stop 2024-03-15T13:02:33.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.190+0000 7f82a3e30740 1 -- 172.21.15.2:0/2420126694 shutdown_connections 2024-03-15T13:02:33.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.190+0000 7f82a3e30740 1 --2- 172.21.15.2:0/2420126694 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55fe258088f0 0x55fe2580ad10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.190+0000 7f82a3e30740 1 --2- 172.21.15.2:0/2420126694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257f8b60 0x55fe257f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.190+0000 7f82a3e30740 1 --2- 172.21.15.2:0/2420126694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f5830 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.190+0000 7f82a3e30740 1 --2- 172.21.15.2:0/2420126694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 0x55fe257f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.190+0000 7f827eff5640 0 client.16761 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:02:33.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.190+0000 7f82a3e30740 1 -- 172.21.15.2:0/2420126694 >> 172.21.15.2:0/2420126694 conn(0x55fe257e5980 msgr2=0x55fe257e76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.190+0000 7f82a3e30740 1 -- 172.21.15.2:0/2420126694 shutdown_connections 2024-03-15T13:02:33.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.190+0000 7f82a3e30740 1 -- 172.21.15.2:0/2420126694 wait complete. 2024-03-15T13:02:33.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.190+0000 7f82a3e30740 20 client.16761 trim_cache size 0 max 16384 2024-03-15T13:02:33.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.202+0000 7f82a3e30740 1 Processor -- start 2024-03-15T13:02:33.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.202+0000 7f82a3e30740 1 -- start start 2024-03-15T13:02:33.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.202+0000 7f82a3e30740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 0x55fe257e6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.202+0000 7f82a3e30740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.202+0000 7f82a3e30740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257f8b60 0x55fe257f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.202+0000 7f82a3e30740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe256d8130 con 0x55fe257eb820 2024-03-15T13:02:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.202+0000 7f82a3e30740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe257e7310 con 0x55fe257f8b60 2024-03-15T13:02:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.202+0000 7f82a3e30740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe257ebe50 con 0x55fe257eaf00 2024-03-15T13:02:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.202+0000 7f829a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.203+0000 7f829a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56664/0 (socket says 172.21.15.2:56664) 2024-03-15T13:02:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.203+0000 7f829a59c640 1 -- 172.21.15.2:0/3853006742 learned_addr learned my addr 172.21.15.2:0/3853006742 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.203+0000 7f829b59e640 1 --2- 172.21.15.2:0/3853006742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257f8b60 0x55fe257f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.203+0000 7f829ad9d640 1 --2- 172.21.15.2:0/3853006742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 0x55fe257e6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.203+0000 7f829a59c640 1 -- 172.21.15.2:0/3853006742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 msgr2=0x55fe257e6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.203+0000 7f829a59c640 1 --2- 172.21.15.2:0/3853006742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 0x55fe257e6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.203+0000 7f829a59c640 1 -- 172.21.15.2:0/3853006742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257f8b60 msgr2=0x55fe257f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.203+0000 7f829a59c640 1 --2- 172.21.15.2:0/3853006742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257f8b60 0x55fe257f84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.203+0000 7f829a59c640 1 -- 172.21.15.2:0/3853006742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe256edc20 con 0x55fe257eb820 2024-03-15T13:02:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.203+0000 7f829ad9d640 1 --2- 172.21.15.2:0/3853006742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 0x55fe257e6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:02:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.203+0000 7f829a59c640 1 --2- 172.21.15.2:0/3853006742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f7f60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8290083e80 tx=0x7f829000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3622cc9174f98e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.203+0000 7f8280ff9640 1 -- 172.21.15.2:0/3853006742 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82900b8b60 con 0x55fe257eb820 2024-03-15T13:02:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.203+0000 7f8280ff9640 1 -- 172.21.15.2:0/3853006742 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f829009a3a0 con 0x55fe257eb820 2024-03-15T13:02:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.203+0000 7f8280ff9640 1 -- 172.21.15.2:0/3853006742 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82900a26b0 con 0x55fe257eb820 2024-03-15T13:02:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 -- 172.21.15.2:0/3853006742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 msgr2=0x55fe257f7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 --2- 172.21.15.2:0/3853006742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f7f60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8290083e80 tx=0x7f829000cf90 comp rx=0 tx=0).stop 2024-03-15T13:02:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 -- 172.21.15.2:0/3853006742 shutdown_connections 2024-03-15T13:02:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 --2- 172.21.15.2:0/3853006742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257f8b60 0x55fe257f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 --2- 172.21.15.2:0/3853006742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257eb820 0x55fe257f7f60 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 --2- 172.21.15.2:0/3853006742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eaf00 0x55fe257e6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 -- 172.21.15.2:0/3853006742 >> 172.21.15.2:0/3853006742 conn(0x55fe257e5980 msgr2=0x55fe257e4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 -- 172.21.15.2:0/3853006742 shutdown_connections 2024-03-15T13:02:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 -- 172.21.15.2:0/3853006742 wait complete. 2024-03-15T13:02:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 -- start start 2024-03-15T13:02:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 Processor -- start 2024-03-15T13:02:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257eaf00 0x55fe25800410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eb820 0x55fe25800950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257f8b60 0x55fe257f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe257f31c0 con 0x55fe257f8b60 2024-03-15T13:02:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe257f3330 con 0x55fe257eaf00 2024-03-15T13:02:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe257f34f0 con 0x55fe257eb820 2024-03-15T13:02:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f82a3e30740 10 client.0 _ll_register_callbacks cb 0x7f828c009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f829b59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257f8b60 0x55fe257f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f829ad9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257eaf00 0x55fe25800410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f829b59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257f8b60 0x55fe257f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56676/0 (socket says 172.21.15.2:56676) 2024-03-15T13:02:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f829ad9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257eaf00 0x55fe25800410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55338/0 (socket says 172.21.15.2:55338) 2024-03-15T13:02:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f829b59e640 1 -- 172.21.15.2:0/1425290674 learned_addr learned my addr 172.21.15.2:0/1425290674 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.204+0000 7f829a59c640 1 --2- 172.21.15.2:0/1425290674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eb820 0x55fe25800950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f829b59e640 1 -- 172.21.15.2:0/1425290674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eb820 msgr2=0x55fe25800950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f829b59e640 1 --2- 172.21.15.2:0/1425290674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eb820 0x55fe25800950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f829b59e640 1 -- 172.21.15.2:0/1425290674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257eaf00 msgr2=0x55fe25800410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f829b59e640 1 --2- 172.21.15.2:0/1425290674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257eaf00 0x55fe25800410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f829b59e640 1 -- 172.21.15.2:0/1425290674 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe257516a0 con 0x55fe257f8b60 2024-03-15T13:02:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f829b59e640 1 --2- 172.21.15.2:0/1425290674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257f8b60 0x55fe257f2af0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f828c00f790 tx=0x7f828c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8256c6ea14e18ee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f827eff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f827eff5640 1 -- 172.21.15.2:0/1425290674 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c016040 con 0x55fe257f8b60 2024-03-15T13:02:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f827eff5640 1 -- 172.21.15.2:0/1425290674 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f828c028600 con 0x55fe257f8b60 2024-03-15T13:02:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f82a3e30740 1 -- 172.21.15.2:0/1425290674 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f829009e4f0 con 0x55fe257f8b60 2024-03-15T13:02:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f827eff5640 1 -- 172.21.15.2:0/1425290674 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f828c0287c0 con 0x55fe257f8b60 2024-03-15T13:02:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f82a3e30740 10 client.16767 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f82a3e30740 1 -- 172.21.15.2:0/1425290674 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55fe257138c0 con 0x55fe257f8b60 2024-03-15T13:02:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f827eff5640 1 -- 172.21.15.2:0/1425290674 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f829009e4f0 con 0x55fe257f8b60 2024-03-15T13:02:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f827eff5640 1 -- 172.21.15.2:0/1425290674 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f828c01fa10 con 0x55fe257f8b60 2024-03-15T13:02:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f82a3e30740 10 client.16767 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.205+0000 7f82a3e30740 10 client.16767 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:02:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f82a3e30740 20 client.16767 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f82a3e30740 1 --2- 172.21.15.2:0/1425290674 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55fe25865640 0x55fe25867a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f82a3e30740 4 client.16767 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:02:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f82a3e30740 1 -- 172.21.15.2:0/1425290674 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fe254de890 con 0x55fe25865640 2024-03-15T13:02:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f829ad9d640 1 --2- 172.21.15.2:0/1425290674 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55fe25865640 0x55fe25867a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f827eff5640 10 client.16767 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:02:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f829ad9d640 1 --2- 172.21.15.2:0/1425290674 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55fe25865640 0x55fe25867a30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=13ebdacb7323b769 server_cookie=78aab37a3b32dee in_seq=0 out_seq=0 2024-03-15T13:02:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f827eff5640 1 -- 172.21.15.2:0/1425290674 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fe254de890 con 0x55fe25865640 2024-03-15T13:02:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f827eff5640 10 client.16767 handle_command_reply: tid=0 2024-03-15T13:02:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f82a3e30740 1 client.16767 shutdown 2024-03-15T13:02:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f82a3e30740 1 -- 172.21.15.2:0/1425290674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257f8b60 msgr2=0x55fe257f2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f82a3e30740 1 --2- 172.21.15.2:0/1425290674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257f8b60 0x55fe257f2af0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f828c00f790 tx=0x7f828c004730 comp rx=0 tx=0).stop 2024-03-15T13:02:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f82a3e30740 1 -- 172.21.15.2:0/1425290674 shutdown_connections 2024-03-15T13:02:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f82a3e30740 1 --2- 172.21.15.2:0/1425290674 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55fe25865640 0x55fe25867a30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f82a3e30740 1 --2- 172.21.15.2:0/1425290674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fe257f8b60 0x55fe257f2af0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f827eff5640 0 client.16767 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:02:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f82a3e30740 1 --2- 172.21.15.2:0/1425290674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55fe257eb820 0x55fe25800950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f82a3e30740 1 --2- 172.21.15.2:0/1425290674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe257eaf00 0x55fe25800410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f82a3e30740 1 -- 172.21.15.2:0/1425290674 >> 172.21.15.2:0/1425290674 conn(0x55fe257e5980 msgr2=0x55fe257f05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.206+0000 7f82a3e30740 1 -- 172.21.15.2:0/1425290674 shutdown_connections 2024-03-15T13:02:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.207+0000 7f82a3e30740 1 -- 172.21.15.2:0/1425290674 wait complete. 2024-03-15T13:02:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.207+0000 7f82a3e30740 20 client.16767 trim_cache size 0 max 16384 2024-03-15T13:02:33.199 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:02:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.207+0000 7f82a1180640 1 -- 172.21.15.2:0/4030862943 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f82840a1ba0 msgr2=0x7f82840a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.207+0000 7f82a1180640 1 --2- 172.21.15.2:0/4030862943 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f82840a1ba0 0x7f82840a4060 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f828c0117b0 tx=0x7f828c011bb0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.207+0000 7f82a1180640 1 -- 172.21.15.2:0/4030862943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f829c1056e0 msgr2=0x7f829c19dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.207+0000 7f82a1180640 1 --2- 172.21.15.2:0/4030862943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f829c1056e0 0x7f829c19dd20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f829000e9c0 tx=0x7f829000ee80 comp rx=0 tx=0).stop 2024-03-15T13:02:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.207+0000 7f82a1180640 1 -- 172.21.15.2:0/4030862943 shutdown_connections 2024-03-15T13:02:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.207+0000 7f82a1180640 1 --2- 172.21.15.2:0/4030862943 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f82840a1ba0 0x7f82840a4060 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.207+0000 7f82a1180640 1 --2- 172.21.15.2:0/4030862943 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f829c197990 0x7f829c197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.207+0000 7f82a1180640 1 --2- 172.21.15.2:0/4030862943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f829c1056e0 0x7f829c19dd20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.207+0000 7f82a1180640 1 --2- 172.21.15.2:0/4030862943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f829c100610 0x7f829c19d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.207+0000 7f82a1180640 1 -- 172.21.15.2:0/4030862943 >> 172.21.15.2:0/4030862943 conn(0x7f829c076770 msgr2=0x7f829c0fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.207+0000 7f82a1180640 1 -- 172.21.15.2:0/4030862943 shutdown_connections 2024-03-15T13:02:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.207+0000 7f82a1180640 1 -- 172.21.15.2:0/4030862943 wait complete. 2024-03-15T13:02:33.203 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:02:33.203 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:02:33.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:33 smithi002 ceph-mon[32060]: pgmap v112: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 830 KiB/s wr, 204 op/s 2024-03-15T13:02:33.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:33 smithi184 ceph-mon[35993]: pgmap v112: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 830 KiB/s wr, 204 op/s 2024-03-15T13:02:33.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.323+0000 7f1f8c7d7640 1 -- 172.21.15.2:0/1290692978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f84107580 msgr2=0x7f1f84107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.323+0000 7f1f8c7d7640 1 --2- 172.21.15.2:0/1290692978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f84107580 0x7f1f84107960 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009900 tx=0x7f1f8002bc30 comp rx=0 tx=0).stop 2024-03-15T13:02:33.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.323+0000 7f1f8c7d7640 1 -- 172.21.15.2:0/1290692978 shutdown_connections 2024-03-15T13:02:33.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.323+0000 7f1f8c7d7640 1 --2- 172.21.15.2:0/1290692978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f841003b0 0x7f1f8410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.323+0000 7f1f8c7d7640 1 --2- 172.21.15.2:0/1290692978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f840ff9f0 0x7f1f840ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.323+0000 7f1f8c7d7640 1 --2- 172.21.15.2:0/1290692978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f84107580 0x7f1f84107960 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.323+0000 7f1f8c7d7640 1 -- 172.21.15.2:0/1290692978 >> 172.21.15.2:0/1290692978 conn(0x7f1f840769b0 msgr2=0x7f1f840fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.323+0000 7f1f8c7d7640 1 -- 172.21.15.2:0/1290692978 shutdown_connections 2024-03-15T13:02:33.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.323+0000 7f1f8c7d7640 1 -- 172.21.15.2:0/1290692978 wait complete. 2024-03-15T13:02:33.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8c7d7640 1 Processor -- start 2024-03-15T13:02:33.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8c7d7640 1 -- start start 2024-03-15T13:02:33.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8c7d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f840ff9f0 0x7f1f8419d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8c7d7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f841003b0 0x7f1f8419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8c7d7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f84107580 0x7f1f84197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8c7d7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f84197ec0 con 0x7f1f840ff9f0 2024-03-15T13:02:33.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8c7d7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f84198030 con 0x7f1f841003b0 2024-03-15T13:02:33.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8c7d7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f841981f0 con 0x7f1f84107580 2024-03-15T13:02:33.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8a573640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f840ff9f0 0x7f1f8419d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8a573640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f840ff9f0 0x7f1f8419d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56680/0 (socket says 172.21.15.2:56680) 2024-03-15T13:02:33.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8a573640 1 -- 172.21.15.2:0/2496400577 learned_addr learned my addr 172.21.15.2:0/2496400577 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f89d72640 1 --2- 172.21.15.2:0/2496400577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f841003b0 0x7f1f8419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8ad74640 1 --2- 172.21.15.2:0/2496400577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f84107580 0x7f1f84197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8a573640 1 -- 172.21.15.2:0/2496400577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f84107580 msgr2=0x7f1f84197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8a573640 1 --2- 172.21.15.2:0/2496400577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f84107580 0x7f1f84197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8a573640 1 -- 172.21.15.2:0/2496400577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f841003b0 msgr2=0x7f1f8419dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8a573640 1 --2- 172.21.15.2:0/2496400577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f841003b0 0x7f1f8419dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8a573640 1 -- 172.21.15.2:0/2496400577 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f80009560 con 0x7f1f840ff9f0 2024-03-15T13:02:33.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.324+0000 7f1f8a573640 1 --2- 172.21.15.2:0/2496400577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f840ff9f0 0x7f1f8419d6c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009230 tx=0x7f1f8002cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.325+0000 7f1f7b7fe640 1 -- 172.21.15.2:0/2496400577 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80004250 con 0x7f1f840ff9f0 2024-03-15T13:02:33.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.325+0000 7f1f7b7fe640 1 -- 172.21.15.2:0/2496400577 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1f800043b0 con 0x7f1f840ff9f0 2024-03-15T13:02:33.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.325+0000 7f1f7b7fe640 1 -- 172.21.15.2:0/2496400577 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80003bf0 con 0x7f1f840ff9f0 2024-03-15T13:02:33.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.325+0000 7f1f8c7d7640 1 -- 172.21.15.2:0/2496400577 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f84198470 con 0x7f1f840ff9f0 2024-03-15T13:02:33.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.325+0000 7f1f8c7d7640 1 -- 172.21.15.2:0/2496400577 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f841a4760 con 0x7f1f840ff9f0 2024-03-15T13:02:33.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.327+0000 7f1f7b7fe640 1 -- 172.21.15.2:0/2496400577 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f1f80003d50 con 0x7f1f840ff9f0 2024-03-15T13:02:33.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.327+0000 7f1f8f3b1740 1 Processor -- start 2024-03-15T13:02:33.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.327+0000 7f1f8f3b1740 1 -- start start 2024-03-15T13:02:33.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8f3b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 0x55a830d21630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8f3b1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21b70 0x55a830d2a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f7b7fe640 1 --2- 172.21.15.2:0/2496400577 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1f700a1ba0 0x7f1f700a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8f3b1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d2cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8f3b1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a830cdc960 con 0x55a830d2a750 2024-03-15T13:02:33.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8f3b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a830c90310 con 0x55a830d21250 2024-03-15T13:02:33.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8f3b1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a830c8d840 con 0x55a830d21b70 2024-03-15T13:02:33.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8ad74640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d2cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f89d72640 1 --2- 172.21.15.2:0/2496400577 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1f700a1ba0 0x7f1f700a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8ad74640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d2cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56696/0 (socket says 172.21.15.2:56696) 2024-03-15T13:02:33.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8ad74640 1 -- 172.21.15.2:0/3828957052 learned_addr learned my addr 172.21.15.2:0/3828957052 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8a573640 1 --2- 172.21.15.2:0/3828957052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21b70 0x55a830d2a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f7b7fe640 1 -- 172.21.15.2:0/2496400577 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f1f800efaf0 con 0x7f1f840ff9f0 2024-03-15T13:02:33.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8ad74640 1 --2- 172.21.15.2:0/3828957052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 0x55a830d21630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8a573640 1 -- 172.21.15.2:0/3828957052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 msgr2=0x55a830d21630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8a573640 1 --2- 172.21.15.2:0/3828957052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 0x55a830d21630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8a573640 1 -- 172.21.15.2:0/3828957052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 msgr2=0x55a830d2cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8a573640 1 --2- 172.21.15.2:0/3828957052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d2cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8a573640 1 -- 172.21.15.2:0/3828957052 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a830c4b060 con 0x55a830d21b70 2024-03-15T13:02:33.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8ad74640 1 --2- 172.21.15.2:0/3828957052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d2cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:33.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f89d72640 1 --2- 172.21.15.2:0/2496400577 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1f700a1ba0 0x7f1f700a4060 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f84106c10 tx=0x7f1f74005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.328+0000 7f1f8a573640 1 --2- 172.21.15.2:0/3828957052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21b70 0x55a830d2a210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f800ecfa0 tx=0x7f1f800395d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62bbbd669c31e0a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.329+0000 7f1f78ff9640 1 -- 172.21.15.2:0/3828957052 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f800bbde0 con 0x55a830d21b70 2024-03-15T13:02:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.329+0000 7f1f78ff9640 1 -- 172.21.15.2:0/3828957052 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1f800dc0b0 con 0x55a830d21b70 2024-03-15T13:02:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.329+0000 7f1f78ff9640 1 -- 172.21.15.2:0/3828957052 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f800c4c20 con 0x55a830d21b70 2024-03-15T13:02:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.329+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/3828957052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21b70 msgr2=0x55a830d2a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.329+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/3828957052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21b70 0x55a830d2a210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f800ecfa0 tx=0x7f1f800395d0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.329+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/3828957052 shutdown_connections 2024-03-15T13:02:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.329+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/3828957052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d2cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.329+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/3828957052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21b70 0x55a830d2a210 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.329+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/3828957052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 0x55a830d21630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.329+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/3828957052 >> 172.21.15.2:0/3828957052 conn(0x55a830d1b980 msgr2=0x55a830d1bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.329+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/3828957052 shutdown_connections 2024-03-15T13:02:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.329+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/3828957052 wait complete. 2024-03-15T13:02:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8f3b1740 1 -- start start 2024-03-15T13:02:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8f3b1740 1 Processor -- start 2024-03-15T13:02:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8f3b1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21250 0x55a830d2a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8f3b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21b70 0x55a830d274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8f3b1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d27ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8f3b1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a830c89020 con 0x55a830d2a750 2024-03-15T13:02:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8f3b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a83095c430 con 0x55a830d21b70 2024-03-15T13:02:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8f3b1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a830918e00 con 0x55a830d21250 2024-03-15T13:02:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8f3b1740 10 client.0 _ll_register_callbacks cb 0x55a830c30cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8a573640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21250 0x55a830d2a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8a573640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21250 0x55a830d2a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:36364/0 (socket says 172.21.15.2:36364) 2024-03-15T13:02:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8ad74640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d27ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8a573640 1 -- 172.21.15.2:0/4085720823 learned_addr learned my addr 172.21.15.2:0/4085720823 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f89d72640 1 --2- 172.21.15.2:0/4085720823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21b70 0x55a830d274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8a573640 1 -- 172.21.15.2:0/4085720823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21b70 msgr2=0x55a830d274c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8a573640 1 --2- 172.21.15.2:0/4085720823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21b70 0x55a830d274c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8a573640 1 -- 172.21.15.2:0/4085720823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 msgr2=0x55a830d27ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8a573640 1 --2- 172.21.15.2:0/4085720823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d27ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8a573640 1 -- 172.21.15.2:0/4085720823 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f841a4760 con 0x55a830d21250 2024-03-15T13:02:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.330+0000 7f1f8ad74640 1 --2- 172.21.15.2:0/4085720823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d27ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f8a573640 1 --2- 172.21.15.2:0/4085720823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21250 0x55a830d2a130 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1f800c0790 tx=0x7f1f800057b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c40bbeffae254df0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:02:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f6dffb640 1 -- 172.21.15.2:0/4085720823 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f800a70b0 con 0x55a830d21250 2024-03-15T13:02:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f6dffb640 1 -- 172.21.15.2:0/4085720823 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f1f800bc480 con 0x55a830d21250 2024-03-15T13:02:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/4085720823 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8309887e0 con 0x55a830d21250 2024-03-15T13:02:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f6dffb640 1 -- 172.21.15.2:0/4085720823 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f800bbde0 con 0x55a830d21250 2024-03-15T13:02:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f8f3b1740 10 client.26572 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/4085720823 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55a830c876a0 con 0x55a830d21250 2024-03-15T13:02:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f6dffb640 1 -- 172.21.15.2:0/4085720823 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55a8309887e0 con 0x55a830d21250 2024-03-15T13:02:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f6dffb640 1 -- 172.21.15.2:0/4085720823 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f1f8003c070 con 0x55a830d21250 2024-03-15T13:02:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f8f3b1740 10 client.26572 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f8f3b1740 10 client.26572 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:02:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f8f3b1740 20 client.26572 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:33.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/4085720823 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55a830d3a300 0x55a830d3c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f8f3b1740 4 client.26572 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:02:33.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.331+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/4085720823 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8309c5720 con 0x55a830d3a300 2024-03-15T13:02:33.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.332+0000 7f1f89d72640 1 --2- 172.21.15.2:0/4085720823 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55a830d3a300 0x55a830d3c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.332+0000 7f1f89d72640 1 --2- 172.21.15.2:0/4085720823 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55a830d3a300 0x55a830d3c720 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fac93458f3f6801b server_cookie=afd9d6bc4ddd498f in_seq=0 out_seq=0 2024-03-15T13:02:33.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.332+0000 7f1f6dffb640 10 client.26572 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:02:33.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.333+0000 7f1f6dffb640 1 -- 172.21.15.2:0/4085720823 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a8309c5720 con 0x55a830d3a300 2024-03-15T13:02:33.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.333+0000 7f1f6dffb640 10 client.26572 handle_command_reply: tid=0 2024-03-15T13:02:33.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.333+0000 7f1f8f3b1740 1 client.26572 shutdown 2024-03-15T13:02:33.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.333+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/4085720823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21250 msgr2=0x55a830d2a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.333+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/4085720823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21250 0x55a830d2a130 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1f800c0790 tx=0x7f1f800057b0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.333+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/4085720823 shutdown_connections 2024-03-15T13:02:33.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.333+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/4085720823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d27ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.333+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/4085720823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21b70 0x55a830d274c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.333+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/4085720823 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55a830d3a300 0x55a830d3c720 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.333+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/4085720823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21250 0x55a830d2a130 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.333+0000 7f1f6dffb640 0 client.26572 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:02:33.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.333+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/4085720823 >> 172.21.15.2:0/4085720823 conn(0x55a830d1b980 msgr2=0x55a830d1d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.333+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/4085720823 shutdown_connections 2024-03-15T13:02:33.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.333+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/4085720823 wait complete. 2024-03-15T13:02:33.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.333+0000 7f1f8f3b1740 20 client.26572 trim_cache size 0 max 16384 2024-03-15T13:02:33.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.345+0000 7f1f8f3b1740 1 Processor -- start 2024-03-15T13:02:33.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.345+0000 7f1f8f3b1740 1 -- start start 2024-03-15T13:02:33.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.345+0000 7f1f8f3b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 0x55a830d29bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.345+0000 7f1f8f3b1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21b70 0x55a830d2a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.345+0000 7f1f8f3b1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d227a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.345+0000 7f1f8f3b1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a830c0e130 con 0x55a830d2a750 2024-03-15T13:02:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.345+0000 7f1f8f3b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8309c5720 con 0x55a830d21250 2024-03-15T13:02:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.345+0000 7f1f8f3b1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8309887e0 con 0x55a830d21b70 2024-03-15T13:02:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.346+0000 7f1f8ad74640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d227a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.346+0000 7f1f8a573640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 0x55a830d29bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.346+0000 7f1f8ad74640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d227a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56728/0 (socket says 172.21.15.2:56728) 2024-03-15T13:02:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.346+0000 7f1f8ad74640 1 -- 172.21.15.2:0/3950188883 learned_addr learned my addr 172.21.15.2:0/3950188883 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.346+0000 7f1f89d72640 1 --2- 172.21.15.2:0/3950188883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21b70 0x55a830d2a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.346+0000 7f1f8ad74640 1 -- 172.21.15.2:0/3950188883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21b70 msgr2=0x55a830d2a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.346+0000 7f1f8ad74640 1 --2- 172.21.15.2:0/3950188883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21b70 0x55a830d2a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.346+0000 7f1f8ad74640 1 -- 172.21.15.2:0/3950188883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 msgr2=0x55a830d29bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.346+0000 7f1f8ad74640 1 --2- 172.21.15.2:0/3950188883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 0x55a830d29bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.346+0000 7f1f8ad74640 1 -- 172.21.15.2:0/3950188883 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a830c23c20 con 0x55a830d2a750 2024-03-15T13:02:33.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.346+0000 7f1f8ad74640 1 --2- 172.21.15.2:0/3950188883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d227a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c003d30 tx=0x7f1f7c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b477c579f4af4992 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.346+0000 7f1f6ffff640 1 -- 172.21.15.2:0/3950188883 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c016050 con 0x55a830d2a750 2024-03-15T13:02:33.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.346+0000 7f1f6ffff640 1 -- 172.21.15.2:0/3950188883 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1f7c004540 con 0x55a830d2a750 2024-03-15T13:02:33.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.346+0000 7f1f6ffff640 1 -- 172.21.15.2:0/3950188883 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c005860 con 0x55a830d2a750 2024-03-15T13:02:33.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/3950188883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 msgr2=0x55a830d227a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/3950188883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d227a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c003d30 tx=0x7f1f7c009b80 comp rx=0 tx=0).stop 2024-03-15T13:02:33.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/3950188883 shutdown_connections 2024-03-15T13:02:33.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/3950188883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d2a750 0x55a830d227a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/3950188883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d21b70 0x55a830d2a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/3950188883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 0x55a830d29bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/3950188883 >> 172.21.15.2:0/3950188883 conn(0x55a830d1b980 msgr2=0x55a830d22090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/3950188883 shutdown_connections 2024-03-15T13:02:33.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/3950188883 wait complete. 2024-03-15T13:02:33.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 -- start start 2024-03-15T13:02:33.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 Processor -- start 2024-03-15T13:02:33.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 0x55a830d2dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d21b70 0x55a830d2e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d2a750 0x55a830d98f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a830d994c0 con 0x55a830d21b70 2024-03-15T13:02:33.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a830d99600 con 0x55a830d21250 2024-03-15T13:02:33.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a830d99740 con 0x55a830d2a750 2024-03-15T13:02:33.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8f3b1740 10 client.0 _ll_register_callbacks cb 0x7f1f7c009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:33.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f89d72640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d21b70 0x55a830d2e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.347+0000 7f1f8a573640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 0x55a830d2dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f89d72640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d21b70 0x55a830d2e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56734/0 (socket says 172.21.15.2:56734) 2024-03-15T13:02:33.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f8ad74640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d2a750 0x55a830d98f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f89d72640 1 -- 172.21.15.2:0/2345006839 learned_addr learned my addr 172.21.15.2:0/2345006839 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f8a573640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 0x55a830d2dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55394/0 (socket says 172.21.15.2:55394) 2024-03-15T13:02:33.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f89d72640 1 -- 172.21.15.2:0/2345006839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d2a750 msgr2=0x55a830d98f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f89d72640 1 --2- 172.21.15.2:0/2345006839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d2a750 0x55a830d98f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f89d72640 1 -- 172.21.15.2:0/2345006839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 msgr2=0x55a830d2dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f89d72640 1 --2- 172.21.15.2:0/2345006839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 0x55a830d2dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f89d72640 1 -- 172.21.15.2:0/2345006839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c011550 con 0x55a830d21b70 2024-03-15T13:02:33.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f8a573640 1 --2- 172.21.15.2:0/2345006839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 0x55a830d2dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:33.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f89d72640 1 --2- 172.21.15.2:0/2345006839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d21b70 0x55a830d2e4e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009710 tx=0x7f1f74013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c370c28e322a4002 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:33.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f6dffb640 1 -- 172.21.15.2:0/2345006839 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74025070 con 0x55a830d21b70 2024-03-15T13:02:33.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f6dffb640 1 -- 172.21.15.2:0/2345006839 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f1f74005050 con 0x55a830d21b70 2024-03-15T13:02:33.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/2345006839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f800dc0b0 con 0x55a830d21b70 2024-03-15T13:02:33.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f6dffb640 1 -- 172.21.15.2:0/2345006839 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f740202e0 con 0x55a830d21b70 2024-03-15T13:02:33.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f8f3b1740 10 client.16785 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:33.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/2345006839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55a830c498c0 con 0x55a830d21b70 2024-03-15T13:02:33.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.348+0000 7f1f6dffb640 1 -- 172.21.15.2:0/2345006839 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1f800dc0b0 con 0x55a830d21b70 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f6dffb640 1 -- 172.21.15.2:0/2345006839 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f1f7401a040 con 0x55a830d21b70 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8f3b1740 10 client.16785 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8f3b1740 10 client.16785 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8f3b1740 20 client.16785 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/2345006839 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55a830d9b060 0x55a830d9d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8f3b1740 4 client.16785 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/2345006839 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a830a14890 con 0x55a830d9b060 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8a573640 1 --2- 172.21.15.2:0/2345006839 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55a830d9b060 0x55a830d9d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8a573640 1 --2- 172.21.15.2:0/2345006839 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55a830d9b060 0x55a830d9d480 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5096acb265ee9c5c server_cookie=246b9724957206b5 in_seq=0 out_seq=0 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f6dffb640 10 client.16785 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f6dffb640 1 -- 172.21.15.2:0/2345006839 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a830a14890 con 0x55a830d9b060 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f6dffb640 10 client.16785 handle_command_reply: tid=0 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8f3b1740 1 client.16785 shutdown 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/2345006839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d21b70 msgr2=0x55a830d2e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/2345006839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d21b70 0x55a830d2e4e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009710 tx=0x7f1f74013040 comp rx=0 tx=0).stop 2024-03-15T13:02:33.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/2345006839 shutdown_connections 2024-03-15T13:02:33.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/2345006839 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55a830d9b060 0x55a830d9d480 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/2345006839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a830d2a750 0x55a830d98f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/2345006839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a830d21b70 0x55a830d2e4e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f6dffb640 0 client.16785 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:02:33.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8f3b1740 1 --2- 172.21.15.2:0/2345006839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a830d21250 0x55a830d2dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.349+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/2345006839 >> 172.21.15.2:0/2345006839 conn(0x55a830d1b980 msgr2=0x55a830d22920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.350+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/2345006839 shutdown_connections 2024-03-15T13:02:33.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.350+0000 7f1f8f3b1740 1 -- 172.21.15.2:0/2345006839 wait complete. 2024-03-15T13:02:33.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.350+0000 7f1f8f3b1740 20 client.16785 trim_cache size 0 max 16384 2024-03-15T13:02:33.341 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:02:33.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.350+0000 7f1f8c7d7640 1 -- 172.21.15.2:0/2496400577 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1f700a1ba0 msgr2=0x7f1f700a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.350+0000 7f1f8c7d7640 1 --2- 172.21.15.2:0/2496400577 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1f700a1ba0 0x7f1f700a4060 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f84106c10 tx=0x7f1f74005f90 comp rx=0 tx=0).stop 2024-03-15T13:02:33.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.350+0000 7f1f8c7d7640 1 -- 172.21.15.2:0/2496400577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f840ff9f0 msgr2=0x7f1f8419d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.350+0000 7f1f8c7d7640 1 --2- 172.21.15.2:0/2496400577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f840ff9f0 0x7f1f8419d6c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009230 tx=0x7f1f8002cac0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.350+0000 7f1f8c7d7640 1 -- 172.21.15.2:0/2496400577 shutdown_connections 2024-03-15T13:02:33.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.350+0000 7f1f8c7d7640 1 --2- 172.21.15.2:0/2496400577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f84107580 0x7f1f84197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.350+0000 7f1f8c7d7640 1 --2- 172.21.15.2:0/2496400577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f841003b0 0x7f1f8419dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.350+0000 7f1f8c7d7640 1 --2- 172.21.15.2:0/2496400577 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1f700a1ba0 0x7f1f700a4060 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.350+0000 7f1f8c7d7640 1 --2- 172.21.15.2:0/2496400577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f840ff9f0 0x7f1f8419d6c0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.350+0000 7f1f8c7d7640 1 -- 172.21.15.2:0/2496400577 >> 172.21.15.2:0/2496400577 conn(0x7f1f840769b0 msgr2=0x7f1f840fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.350+0000 7f1f8c7d7640 1 -- 172.21.15.2:0/2496400577 shutdown_connections 2024-03-15T13:02:33.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.350+0000 7f1f8c7d7640 1 -- 172.21.15.2:0/2496400577 wait complete. 2024-03-15T13:02:33.346 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:02:33.346 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:02:33.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:33 smithi149 ceph-mon[35927]: pgmap v112: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 830 KiB/s wr, 204 op/s 2024-03-15T13:02:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.467+0000 7ff1fe1d0640 1 -- 172.21.15.2:0/1421842998 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f8101e70 msgr2=0x7ff1f810e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.467+0000 7ff1fe1d0640 1 --2- 172.21.15.2:0/1421842998 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f8101e70 0x7ff1f810e330 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009ca0 tx=0x7ff1f002c0f0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.468+0000 7ff1fe1d0640 1 -- 172.21.15.2:0/1421842998 shutdown_connections 2024-03-15T13:02:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.468+0000 7ff1fe1d0640 1 --2- 172.21.15.2:0/1421842998 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f8101e70 0x7ff1f810e330 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.468+0000 7ff1fe1d0640 1 --2- 172.21.15.2:0/1421842998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1f81014b0 0x7ff1f8101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.468+0000 7ff1fe1d0640 1 --2- 172.21.15.2:0/1421842998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f8107570 0x7ff1f8107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.468+0000 7ff1fe1d0640 1 -- 172.21.15.2:0/1421842998 >> 172.21.15.2:0/1421842998 conn(0x7ff1f80fcf20 msgr2=0x7ff1f80ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.468+0000 7ff1fe1d0640 1 -- 172.21.15.2:0/1421842998 shutdown_connections 2024-03-15T13:02:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.468+0000 7ff1fe1d0640 1 -- 172.21.15.2:0/1421842998 wait complete. 2024-03-15T13:02:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.468+0000 7ff1fe1d0640 1 Processor -- start 2024-03-15T13:02:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.468+0000 7ff1fe1d0640 1 -- start start 2024-03-15T13:02:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1fe1d0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1f81014b0 0x7ff1f819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1fe1d0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f8101e70 0x7ff1f819dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1fe1d0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f8107570 0x7ff1f8197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1fe1d0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f8197eb0 con 0x7ff1f8101e70 2024-03-15T13:02:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1fe1d0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f8198020 con 0x7ff1f81014b0 2024-03-15T13:02:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1fe1d0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f81981e0 con 0x7ff1f8107570 2024-03-15T13:02:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1f6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f8101e70 0x7ff1f819dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1f6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f8101e70 0x7ff1f819dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56750/0 (socket says 172.21.15.2:56750) 2024-03-15T13:02:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1f6ffd640 1 -- 172.21.15.2:0/1960428262 learned_addr learned my addr 172.21.15.2:0/1960428262 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1f77fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1f81014b0 0x7ff1f819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1f7fff640 1 --2- 172.21.15.2:0/1960428262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f8107570 0x7ff1f8197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1f6ffd640 1 -- 172.21.15.2:0/1960428262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f8107570 msgr2=0x7ff1f8197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1f6ffd640 1 --2- 172.21.15.2:0/1960428262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f8107570 0x7ff1f8197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1f6ffd640 1 -- 172.21.15.2:0/1960428262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1f81014b0 msgr2=0x7ff1f819d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1f6ffd640 1 --2- 172.21.15.2:0/1960428262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1f81014b0 0x7ff1f819d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1f6ffd640 1 -- 172.21.15.2:0/1960428262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f0009900 con 0x7ff1f8101e70 2024-03-15T13:02:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1f6ffd640 1 --2- 172.21.15.2:0/1960428262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f8101e70 0x7ff1f819dbf0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00e9f0 tx=0x7ff1ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1f4ff9640 1 -- 172.21.15.2:0/1960428262 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec009700 con 0x7ff1f8101e70 2024-03-15T13:02:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1fe1d0640 1 -- 172.21.15.2:0/1960428262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f81984c0 con 0x7ff1f8101e70 2024-03-15T13:02:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1f4ff9640 1 -- 172.21.15.2:0/1960428262 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff1ec00cd80 con 0x7ff1f8101e70 2024-03-15T13:02:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1f4ff9640 1 -- 172.21.15.2:0/1960428262 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec018920 con 0x7ff1f8101e70 2024-03-15T13:02:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.469+0000 7ff1fe1d0640 1 -- 172.21.15.2:0/1960428262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f81a4750 con 0x7ff1f8101e70 2024-03-15T13:02:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.471+0000 7ff1f4ff9640 1 -- 172.21.15.2:0/1960428262 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7ff1ec018a80 con 0x7ff1f8101e70 2024-03-15T13:02:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff200daa740 1 Processor -- start 2024-03-15T13:02:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff200daa740 1 -- start start 2024-03-15T13:02:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff200daa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 0x55e229d49630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff200daa740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d49b70 0x55e229d52210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff200daa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d52750 0x55e229d54b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff200daa740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e229d04960 con 0x55e229d49b70 2024-03-15T13:02:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff200daa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e229cb8310 con 0x55e229d49250 2024-03-15T13:02:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff200daa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e229cb5840 con 0x55e229d52750 2024-03-15T13:02:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff1f4ff9640 1 --2- 172.21.15.2:0/1960428262 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff1e00a1ba0 0x7ff1e00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff1f77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d49b70 0x55e229d52210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff1f77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d49b70 0x55e229d52210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56760/0 (socket says 172.21.15.2:56760) 2024-03-15T13:02:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 0x55e229d49630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff1f6ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d52750 0x55e229d54b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff1f77fe640 1 -- 172.21.15.2:0/1729486722 learned_addr learned my addr 172.21.15.2:0/1729486722 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff1f7fff640 1 --2- 172.21.15.2:0/1960428262 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff1e00a1ba0 0x7ff1e00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff1f4ff9640 1 -- 172.21.15.2:0/1960428262 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7ff1ec014070 con 0x7ff1f8101e70 2024-03-15T13:02:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff1f77fe640 1 -- 172.21.15.2:0/1729486722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d52750 msgr2=0x55e229d54b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff1f77fe640 1 --2- 172.21.15.2:0/1729486722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d52750 0x55e229d54b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff1f77fe640 1 -- 172.21.15.2:0/1729486722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 msgr2=0x55e229d49630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.472+0000 7ff1f77fe640 1 --2- 172.21.15.2:0/1729486722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 0x55e229d49630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.473+0000 7ff1f77fe640 1 -- 172.21.15.2:0/1729486722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e229c73060 con 0x55e229d49b70 2024-03-15T13:02:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.473+0000 7ff1f7fff640 1 --2- 172.21.15.2:0/1960428262 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff1e00a1ba0 0x7ff1e00a4060 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0004af0 tx=0x7ff1f0005c50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.473+0000 7ff1f77fe640 1 --2- 172.21.15.2:0/1729486722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d49b70 0x55e229d52210 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009ca0 tx=0x7ff1e8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4501d1d1e0e62870 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.473+0000 7ff1ddffb640 1 -- 172.21.15.2:0/1729486722 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8005220 con 0x55e229d49b70 2024-03-15T13:02:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.473+0000 7ff1ddffb640 1 -- 172.21.15.2:0/1729486722 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff1e8004c70 con 0x55e229d49b70 2024-03-15T13:02:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.473+0000 7ff1ddffb640 1 -- 172.21.15.2:0/1729486722 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8019700 con 0x55e229d49b70 2024-03-15T13:02:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.473+0000 7ff200daa740 1 -- 172.21.15.2:0/1729486722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d49b70 msgr2=0x55e229d52210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.473+0000 7ff200daa740 1 --2- 172.21.15.2:0/1729486722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d49b70 0x55e229d52210 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009ca0 tx=0x7ff1e8011810 comp rx=0 tx=0).stop 2024-03-15T13:02:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 1 -- 172.21.15.2:0/1729486722 shutdown_connections 2024-03-15T13:02:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 1 --2- 172.21.15.2:0/1729486722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d52750 0x55e229d54b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 1 --2- 172.21.15.2:0/1729486722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d49b70 0x55e229d52210 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 1 --2- 172.21.15.2:0/1729486722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 0x55e229d49630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 1 -- 172.21.15.2:0/1729486722 >> 172.21.15.2:0/1729486722 conn(0x55e229d43980 msgr2=0x55e229d43d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 1 -- 172.21.15.2:0/1729486722 shutdown_connections 2024-03-15T13:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 1 -- 172.21.15.2:0/1729486722 wait complete. 2024-03-15T13:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 1 -- start start 2024-03-15T13:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 1 Processor -- start 2024-03-15T13:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d49250 0x55e229d520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49b70 0x55e229d4f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d52750 0x55e229d4fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e229cb1020 con 0x55e229d49250 2024-03-15T13:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e229984430 con 0x55e229d49b70 2024-03-15T13:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e229940e00 con 0x55e229d52750 2024-03-15T13:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff200daa740 10 client.0 _ll_register_callbacks cb 0x55e229c58cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.474+0000 7ff1f77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d49250 0x55e229d520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff1f77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d49250 0x55e229d520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56770/0 (socket says 172.21.15.2:56770) 2024-03-15T13:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff1f77fe640 1 -- 172.21.15.2:0/3311475398 learned_addr learned my addr 172.21.15.2:0/3311475398 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff1f6ffd640 1 --2- 172.21.15.2:0/3311475398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49b70 0x55e229d4f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff1f7fff640 1 --2- 172.21.15.2:0/3311475398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d52750 0x55e229d4fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff1f77fe640 1 -- 172.21.15.2:0/3311475398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d52750 msgr2=0x55e229d4fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff1f77fe640 1 --2- 172.21.15.2:0/3311475398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d52750 0x55e229d4fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff1f77fe640 1 -- 172.21.15.2:0/3311475398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49b70 msgr2=0x55e229d4f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff1f77fe640 1 --2- 172.21.15.2:0/3311475398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49b70 0x55e229d4f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff1f77fe640 1 -- 172.21.15.2:0/3311475398 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e80098d0 con 0x55e229d49250 2024-03-15T13:02:33.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff1f6ffd640 1 --2- 172.21.15.2:0/3311475398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49b70 0x55e229d4f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:02:33.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff1f77fe640 1 --2- 172.21.15.2:0/3311475398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d49250 0x55e229d520f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009380 tx=0x7ff1e8005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522b202851ed6017 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff1d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:33.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff1d6ffd640 1 -- 172.21.15.2:0/3311475398 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8005430 con 0x55e229d49250 2024-03-15T13:02:33.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff1d6ffd640 1 -- 172.21.15.2:0/3311475398 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7ff1e80194a0 con 0x55e229d49250 2024-03-15T13:02:33.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff200daa740 1 -- 172.21.15.2:0/3311475398 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2299b07e0 con 0x55e229d49250 2024-03-15T13:02:33.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff1d6ffd640 1 -- 172.21.15.2:0/3311475398 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1e8019660 con 0x55e229d49250 2024-03-15T13:02:33.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff200daa740 10 client.16797 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:33.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.475+0000 7ff200daa740 1 -- 172.21.15.2:0/3311475398 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55e229caf6a0 con 0x55e229d49250 2024-03-15T13:02:33.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.476+0000 7ff1d6ffd640 1 -- 172.21.15.2:0/3311475398 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e2299b07e0 con 0x55e229d49250 2024-03-15T13:02:33.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.476+0000 7ff1d6ffd640 1 -- 172.21.15.2:0/3311475398 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7ff1e8021040 con 0x55e229d49250 2024-03-15T13:02:33.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.476+0000 7ff200daa740 10 client.16797 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:33.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.476+0000 7ff200daa740 10 client.16797 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:02:33.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.476+0000 7ff200daa740 20 client.16797 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:33.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.476+0000 7ff200daa740 1 --2- 172.21.15.2:0/3311475398 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e229d62840 0x55e229d64c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.476+0000 7ff200daa740 4 client.16797 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:02:33.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.476+0000 7ff200daa740 1 -- 172.21.15.2:0/3311475398 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2299ed720 con 0x55e229d62840 2024-03-15T13:02:33.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.476+0000 7ff1f6ffd640 1 --2- 172.21.15.2:0/3311475398 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e229d62840 0x55e229d64c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.476+0000 7ff1d6ffd640 10 client.16797 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:02:33.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.476+0000 7ff1f6ffd640 1 --2- 172.21.15.2:0/3311475398 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e229d62840 0x55e229d64c30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b095f607c864d8e server_cookie=4ff9324222c1a5b5 in_seq=0 out_seq=0 2024-03-15T13:02:33.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.477+0000 7ff1d6ffd640 1 -- 172.21.15.2:0/3311475398 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7ff1ec0a94b0 con 0x55e229d62840 2024-03-15T13:02:33.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.477+0000 7ff1d6ffd640 10 client.16797 handle_command_reply: tid=0 2024-03-15T13:02:33.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.477+0000 7ff200daa740 1 client.16797 shutdown 2024-03-15T13:02:33.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.477+0000 7ff200daa740 1 -- 172.21.15.2:0/3311475398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d49250 msgr2=0x55e229d520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.477+0000 7ff200daa740 1 --2- 172.21.15.2:0/3311475398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d49250 0x55e229d520f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009380 tx=0x7ff1e8005db0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.478+0000 7ff200daa740 1 -- 172.21.15.2:0/3311475398 shutdown_connections 2024-03-15T13:02:33.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.478+0000 7ff200daa740 1 --2- 172.21.15.2:0/3311475398 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e229d62840 0x55e229d64c30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.478+0000 7ff200daa740 1 --2- 172.21.15.2:0/3311475398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d52750 0x55e229d4fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.478+0000 7ff200daa740 1 --2- 172.21.15.2:0/3311475398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49b70 0x55e229d4f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.478+0000 7ff1d6ffd640 0 client.16797 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:02:33.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.478+0000 7ff200daa740 1 --2- 172.21.15.2:0/3311475398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d49250 0x55e229d520f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.478+0000 7ff200daa740 1 -- 172.21.15.2:0/3311475398 >> 172.21.15.2:0/3311475398 conn(0x55e229d43980 msgr2=0x55e229d456d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.478+0000 7ff200daa740 1 -- 172.21.15.2:0/3311475398 shutdown_connections 2024-03-15T13:02:33.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.478+0000 7ff200daa740 1 -- 172.21.15.2:0/3311475398 wait complete. 2024-03-15T13:02:33.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.478+0000 7ff200daa740 20 client.16797 trim_cache size 0 max 16384 2024-03-15T13:02:33.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.490+0000 7ff200daa740 1 Processor -- start 2024-03-15T13:02:33.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.490+0000 7ff200daa740 1 -- start start 2024-03-15T13:02:33.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.490+0000 7ff200daa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 0x55e229d42a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.490+0000 7ff200daa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d49b70 0x55e229d51bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.490+0000 7ff200daa740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d52750 0x55e229d520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.490+0000 7ff200daa740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e229c36130 con 0x55e229d52750 2024-03-15T13:02:33.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.490+0000 7ff200daa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e229d4af60 con 0x55e229d49250 2024-03-15T13:02:33.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.490+0000 7ff200daa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e229d42f40 con 0x55e229d49b70 2024-03-15T13:02:33.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.490+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d52750 0x55e229d520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.491+0000 7ff1f77fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 0x55e229d42a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.491+0000 7ff1f6ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d49b70 0x55e229d51bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.491+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d52750 0x55e229d520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56778/0 (socket says 172.21.15.2:56778) 2024-03-15T13:02:33.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.491+0000 7ff1f7fff640 1 -- 172.21.15.2:0/3774156630 learned_addr learned my addr 172.21.15.2:0/3774156630 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.491+0000 7ff1f77fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 0x55e229d42a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55428/0 (socket says 172.21.15.2:55428) 2024-03-15T13:02:33.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.491+0000 7ff1f6ffd640 1 -- 172.21.15.2:0/3774156630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 msgr2=0x55e229d42a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.491+0000 7ff1f6ffd640 1 --2- 172.21.15.2:0/3774156630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 0x55e229d42a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.491+0000 7ff1f6ffd640 1 -- 172.21.15.2:0/3774156630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d52750 msgr2=0x55e229d520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.491+0000 7ff1f6ffd640 1 --2- 172.21.15.2:0/3774156630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d52750 0x55e229d520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.491+0000 7ff1f6ffd640 1 -- 172.21.15.2:0/3774156630 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e229c4bc20 con 0x55e229d49b70 2024-03-15T13:02:33.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.491+0000 7ff1f7fff640 1 --2- 172.21.15.2:0/3774156630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d52750 0x55e229d520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:33.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.491+0000 7ff1f6ffd640 1 --2- 172.21.15.2:0/3774156630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d49b70 0x55e229d51bb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0cb270 tx=0x7ff1ec06db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5572d0ce744287e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.491+0000 7ff1dd7fa640 1 -- 172.21.15.2:0/3774156630 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec09e8a0 con 0x55e229d49b70 2024-03-15T13:02:33.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.491+0000 7ff1dd7fa640 1 -- 172.21.15.2:0/3774156630 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e2299ed720 con 0x55e229d49b70 2024-03-15T13:02:33.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.491+0000 7ff1dd7fa640 1 -- 172.21.15.2:0/3774156630 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec0cb4f0 con 0x55e229d49b70 2024-03-15T13:02:33.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.492+0000 7ff200daa740 1 -- 172.21.15.2:0/3774156630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d49b70 msgr2=0x55e229d51bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.492+0000 7ff200daa740 1 --2- 172.21.15.2:0/3774156630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d49b70 0x55e229d51bb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0cb270 tx=0x7ff1ec06db70 comp rx=0 tx=0).stop 2024-03-15T13:02:33.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.492+0000 7ff200daa740 1 -- 172.21.15.2:0/3774156630 shutdown_connections 2024-03-15T13:02:33.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.492+0000 7ff200daa740 1 --2- 172.21.15.2:0/3774156630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d52750 0x55e229d520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.492+0000 7ff200daa740 1 --2- 172.21.15.2:0/3774156630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d49b70 0x55e229d51bb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.492+0000 7ff200daa740 1 --2- 172.21.15.2:0/3774156630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 0x55e229d42a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.492+0000 7ff200daa740 1 -- 172.21.15.2:0/3774156630 >> 172.21.15.2:0/3774156630 conn(0x55e229d43980 msgr2=0x55e229d422f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.492+0000 7ff200daa740 1 -- 172.21.15.2:0/3774156630 shutdown_connections 2024-03-15T13:02:33.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.492+0000 7ff200daa740 1 -- 172.21.15.2:0/3774156630 wait complete. 2024-03-15T13:02:33.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.492+0000 7ff200daa740 1 -- start start 2024-03-15T13:02:33.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.492+0000 7ff200daa740 1 Processor -- start 2024-03-15T13:02:33.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff200daa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 0x55e229d5a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff200daa740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d49b70 0x55e229d564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff200daa740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d52750 0x55e229d56a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff200daa740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e229d56f40 con 0x55e229d52750 2024-03-15T13:02:33.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff200daa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e229d570b0 con 0x55e229d49250 2024-03-15T13:02:33.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff200daa740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e229d57270 con 0x55e229d49b70 2024-03-15T13:02:33.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff200daa740 10 client.0 _ll_register_callbacks cb 0x7ff1ec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:33.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d52750 0x55e229d56a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1f6ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d49b70 0x55e229d564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1f77fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 0x55e229d5a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d52750 0x55e229d56a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56794/0 (socket says 172.21.15.2:56794) 2024-03-15T13:02:33.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1f77fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 0x55e229d5a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:55436/0 (socket says 172.21.15.2:55436) 2024-03-15T13:02:33.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1f6ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d49b70 0x55e229d564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:36436/0 (socket says 172.21.15.2:36436) 2024-03-15T13:02:33.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1f7fff640 1 -- 172.21.15.2:0/1562553768 learned_addr learned my addr 172.21.15.2:0/1562553768 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1f7fff640 1 -- 172.21.15.2:0/1562553768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d49b70 msgr2=0x55e229d564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1f7fff640 1 --2- 172.21.15.2:0/1562553768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d49b70 0x55e229d564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1f7fff640 1 -- 172.21.15.2:0/1562553768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 msgr2=0x55e229d5a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1f7fff640 1 --2- 172.21.15.2:0/1562553768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 0x55e229d5a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1f7fff640 1 -- 172.21.15.2:0/1562553768 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f810ec30 con 0x55e229d52750 2024-03-15T13:02:33.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1f7fff640 1 --2- 172.21.15.2:0/1562553768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d52750 0x55e229d56a00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0037040 tx=0x7ff1f00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408301ac2fbac87d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:33.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1d6ffd640 1 -- 172.21.15.2:0/1562553768 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0032b70 con 0x55e229d52750 2024-03-15T13:02:33.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff1d6ffd640 1 -- 172.21.15.2:0/1562553768 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7ff1f0003d80 con 0x55e229d52750 2024-03-15T13:02:33.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.493+0000 7ff200daa740 1 -- 172.21.15.2:0/1562553768 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e229d3f0f0 con 0x55e229d52750 2024-03-15T13:02:33.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.494+0000 7ff1d6ffd640 1 -- 172.21.15.2:0/1562553768 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1f0004040 con 0x55e229d52750 2024-03-15T13:02:33.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.494+0000 7ff200daa740 10 client.16803 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:33.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.494+0000 7ff200daa740 1 -- 172.21.15.2:0/1562553768 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7ff1f81a4750 con 0x55e229d52750 2024-03-15T13:02:33.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.494+0000 7ff1d6ffd640 1 -- 172.21.15.2:0/1562553768 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e229d3f0f0 con 0x55e229d52750 2024-03-15T13:02:33.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.494+0000 7ff1d6ffd640 1 -- 172.21.15.2:0/1562553768 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7ff1f004e040 con 0x55e229d52750 2024-03-15T13:02:33.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.494+0000 7ff200daa740 10 client.16803 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:33.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.494+0000 7ff200daa740 10 client.16803 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:02:33.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.494+0000 7ff200daa740 20 client.16803 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:33.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.494+0000 7ff200daa740 1 --2- 172.21.15.2:0/1562553768 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e229dbf6c0 0x55e229dc1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.494+0000 7ff200daa740 4 client.16803 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:02:33.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.494+0000 7ff200daa740 1 -- 172.21.15.2:0/1562553768 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7ff1e8004c70 con 0x55e229dbf6c0 2024-03-15T13:02:33.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.494+0000 7ff1f77fe640 1 --2- 172.21.15.2:0/1562553768 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e229dbf6c0 0x55e229dc1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.494+0000 7ff1f77fe640 1 --2- 172.21.15.2:0/1562553768 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e229dbf6c0 0x55e229dc1ae0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a5e58e786c5dec5 server_cookie=630af398bbd3216b in_seq=0 out_seq=0 2024-03-15T13:02:33.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.494+0000 7ff1d6ffd640 10 client.16803 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:02:33.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff1d6ffd640 1 -- 172.21.15.2:0/1562553768 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7ff1e8004c70 con 0x55e229dbf6c0 2024-03-15T13:02:33.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff1d6ffd640 10 client.16803 handle_command_reply: tid=0 2024-03-15T13:02:33.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff200daa740 1 client.16803 shutdown 2024-03-15T13:02:33.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff200daa740 1 -- 172.21.15.2:0/1562553768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d52750 msgr2=0x55e229d56a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff200daa740 1 --2- 172.21.15.2:0/1562553768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d52750 0x55e229d56a00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0037040 tx=0x7ff1f00095c0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff200daa740 1 -- 172.21.15.2:0/1562553768 shutdown_connections 2024-03-15T13:02:33.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff200daa740 1 --2- 172.21.15.2:0/1562553768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e229d52750 0x55e229d56a00 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff200daa740 1 --2- 172.21.15.2:0/1562553768 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55e229dbf6c0 0x55e229dc1ae0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff200daa740 1 --2- 172.21.15.2:0/1562553768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e229d49b70 0x55e229d564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff1d6ffd640 0 client.16803 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:02:33.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff200daa740 1 --2- 172.21.15.2:0/1562553768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e229d49250 0x55e229d5a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff200daa740 1 -- 172.21.15.2:0/1562553768 >> 172.21.15.2:0/1562553768 conn(0x55e229d43980 msgr2=0x55e229d49f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff200daa740 1 -- 172.21.15.2:0/1562553768 shutdown_connections 2024-03-15T13:02:33.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff200daa740 1 -- 172.21.15.2:0/1562553768 wait complete. 2024-03-15T13:02:33.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff200daa740 20 client.16803 trim_cache size 0 max 16384 2024-03-15T13:02:33.487 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:02:33.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff1fe1d0640 1 -- 172.21.15.2:0/1960428262 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff1e00a1ba0 msgr2=0x7ff1e00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff1fe1d0640 1 --2- 172.21.15.2:0/1960428262 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff1e00a1ba0 0x7ff1e00a4060 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0004af0 tx=0x7ff1f0005c50 comp rx=0 tx=0).stop 2024-03-15T13:02:33.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff1fe1d0640 1 -- 172.21.15.2:0/1960428262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f8101e70 msgr2=0x7ff1f819dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.495+0000 7ff1fe1d0640 1 --2- 172.21.15.2:0/1960428262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f8101e70 0x7ff1f819dbf0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00e9f0 tx=0x7ff1ec00eeb0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.496+0000 7ff1fe1d0640 1 -- 172.21.15.2:0/1960428262 shutdown_connections 2024-03-15T13:02:33.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.496+0000 7ff1fe1d0640 1 --2- 172.21.15.2:0/1960428262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1f8107570 0x7ff1f8197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.496+0000 7ff1fe1d0640 1 --2- 172.21.15.2:0/1960428262 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff1e00a1ba0 0x7ff1e00a4060 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.496+0000 7ff1fe1d0640 1 --2- 172.21.15.2:0/1960428262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f8101e70 0x7ff1f819dbf0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.496+0000 7ff1fe1d0640 1 --2- 172.21.15.2:0/1960428262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1f81014b0 0x7ff1f819d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.496+0000 7ff1fe1d0640 1 -- 172.21.15.2:0/1960428262 >> 172.21.15.2:0/1960428262 conn(0x7ff1f80fcf20 msgr2=0x7ff1f80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.496+0000 7ff1fe1d0640 1 -- 172.21.15.2:0/1960428262 shutdown_connections 2024-03-15T13:02:33.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.496+0000 7ff1fe1d0640 1 -- 172.21.15.2:0/1960428262 wait complete. 2024-03-15T13:02:33.492 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:02:33.492 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:02:33.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.612+0000 7fa16bf1e640 1 -- 172.21.15.2:0/1571355470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164107560 msgr2=0x7fa164107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.612+0000 7fa16bf1e640 1 --2- 172.21.15.2:0/1571355470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164107560 0x7fa164107940 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa160009900 tx=0x7fa16002bc30 comp rx=0 tx=0).stop 2024-03-15T13:02:33.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.612+0000 7fa16bf1e640 1 -- 172.21.15.2:0/1571355470 shutdown_connections 2024-03-15T13:02:33.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.612+0000 7fa16bf1e640 1 --2- 172.21.15.2:0/1571355470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa164101e60 0x7fa16410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.612+0000 7fa16bf1e640 1 --2- 172.21.15.2:0/1571355470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1641014a0 0x7fa164101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.612+0000 7fa16bf1e640 1 --2- 172.21.15.2:0/1571355470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164107560 0x7fa164107940 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.612+0000 7fa16bf1e640 1 -- 172.21.15.2:0/1571355470 >> 172.21.15.2:0/1571355470 conn(0x7fa1640788f0 msgr2=0x7fa164078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.612+0000 7fa16bf1e640 1 -- 172.21.15.2:0/1571355470 shutdown_connections 2024-03-15T13:02:33.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.612+0000 7fa16bf1e640 1 -- 172.21.15.2:0/1571355470 wait complete. 2024-03-15T13:02:33.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa16bf1e640 1 Processor -- start 2024-03-15T13:02:33.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa16bf1e640 1 -- start start 2024-03-15T13:02:33.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa16bf1e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1641014a0 0x7fa1640737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa16bf1e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164101e60 0x7fa164073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa16bf1e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa164107560 0x7fa164074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa16bf1e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641a5e20 con 0x7fa164101e60 2024-03-15T13:02:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa16bf1e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641a5f90 con 0x7fa1641014a0 2024-03-15T13:02:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa16bf1e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641a6150 con 0x7fa164107560 2024-03-15T13:02:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa1694b9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164101e60 0x7fa164073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa16a4bb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa164107560 0x7fa164074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa1694b9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164101e60 0x7fa164073d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56814/0 (socket says 172.21.15.2:56814) 2024-03-15T13:02:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa1694b9640 1 -- 172.21.15.2:0/1366282027 learned_addr learned my addr 172.21.15.2:0/1366282027 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa169cba640 1 --2- 172.21.15.2:0/1366282027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1641014a0 0x7fa1640737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa1694b9640 1 -- 172.21.15.2:0/1366282027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa164107560 msgr2=0x7fa164074270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa1694b9640 1 --2- 172.21.15.2:0/1366282027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa164107560 0x7fa164074270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa1694b9640 1 -- 172.21.15.2:0/1366282027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1641014a0 msgr2=0x7fa1640737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa1694b9640 1 --2- 172.21.15.2:0/1366282027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1641014a0 0x7fa1640737f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa1694b9640 1 -- 172.21.15.2:0/1366282027 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa160009560 con 0x7fa164101e60 2024-03-15T13:02:33.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.613+0000 7fa1694b9640 1 --2- 172.21.15.2:0/1366282027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164101e60 0x7fa164073d30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa15400e9f0 tx=0x7fa15400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.614+0000 7fa15affd640 1 -- 172.21.15.2:0/1366282027 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154009720 con 0x7fa164101e60 2024-03-15T13:02:33.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.614+0000 7fa16bf1e640 1 -- 172.21.15.2:0/1366282027 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1641a6430 con 0x7fa164101e60 2024-03-15T13:02:33.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.614+0000 7fa15affd640 1 -- 172.21.15.2:0/1366282027 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa15400cd80 con 0x7fa164101e60 2024-03-15T13:02:33.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.614+0000 7fa15affd640 1 -- 172.21.15.2:0/1366282027 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154010690 con 0x7fa164101e60 2024-03-15T13:02:33.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.614+0000 7fa16bf1e640 1 -- 172.21.15.2:0/1366282027 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1641a68f0 con 0x7fa164101e60 2024-03-15T13:02:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.616+0000 7fa15affd640 1 -- 172.21.15.2:0/1366282027 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fa154004e90 con 0x7fa164101e60 2024-03-15T13:02:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.616+0000 7fa16ebdc740 1 Processor -- start 2024-03-15T13:02:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.616+0000 7fa16ebdc740 1 -- start start 2024-03-15T13:02:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.616+0000 7fa15affd640 1 --2- 172.21.15.2:0/1366282027 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa1500a1ba0 0x7fa1500a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.616+0000 7fa16ebdc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9afc250 0x5641b9afc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.616+0000 7fa16ebdc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afcb70 0x5641b9b05210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.616+0000 7fa16ebdc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9b05750 0x5641b9b07b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.616+0000 7fa16ebdc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b9ab7960 con 0x5641b9afcb70 2024-03-15T13:02:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.616+0000 7fa16ebdc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b9a6b310 con 0x5641b9b05750 2024-03-15T13:02:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.616+0000 7fa16ebdc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b9a68840 con 0x5641b9afc250 2024-03-15T13:02:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.616+0000 7fa1694b9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afcb70 0x5641b9b05210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa16a4bb640 1 --2- 172.21.15.2:0/1366282027 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa1500a1ba0 0x7fa1500a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa1694b9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afcb70 0x5641b9b05210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56822/0 (socket says 172.21.15.2:56822) 2024-03-15T13:02:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa1694b9640 1 -- 172.21.15.2:0/4244378724 learned_addr learned my addr 172.21.15.2:0/4244378724 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa169cba640 1 --2- 172.21.15.2:0/4244378724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9afc250 0x5641b9afc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa16a4bb640 1 --2- 172.21.15.2:0/4244378724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9b05750 0x5641b9b07b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa15affd640 1 -- 172.21.15.2:0/1366282027 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa154014070 con 0x7fa164101e60 2024-03-15T13:02:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa1694b9640 1 -- 172.21.15.2:0/4244378724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9afc250 msgr2=0x5641b9afc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa1694b9640 1 --2- 172.21.15.2:0/4244378724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9afc250 0x5641b9afc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa1694b9640 1 -- 172.21.15.2:0/4244378724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9b05750 msgr2=0x5641b9b07b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa1694b9640 1 --2- 172.21.15.2:0/4244378724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9b05750 0x5641b9b07b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa1694b9640 1 -- 172.21.15.2:0/4244378724 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641b9a26060 con 0x5641b9afcb70 2024-03-15T13:02:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa16a4bb640 1 --2- 172.21.15.2:0/1366282027 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa1500a1ba0 0x7fa1500a4060 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x5641b9af7b10 tx=0x7fa15c009a90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa1694b9640 1 --2- 172.21.15.2:0/4244378724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afcb70 0x5641b9b05210 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa15401c000 tx=0x7fa15409a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12838c6eaae37815 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa14ffff640 1 -- 172.21.15.2:0/4244378724 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15401d260 con 0x5641b9afcb70 2024-03-15T13:02:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa14ffff640 1 -- 172.21.15.2:0/4244378724 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa15409ae80 con 0x5641b9afcb70 2024-03-15T13:02:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.617+0000 7fa14ffff640 1 -- 172.21.15.2:0/4244378724 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540cb520 con 0x5641b9afcb70 2024-03-15T13:02:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.618+0000 7fa16ebdc740 1 -- 172.21.15.2:0/4244378724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afcb70 msgr2=0x5641b9b05210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.618+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/4244378724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afcb70 0x5641b9b05210 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa15401c000 tx=0x7fa15409a550 comp rx=0 tx=0).stop 2024-03-15T13:02:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.618+0000 7fa16ebdc740 1 -- 172.21.15.2:0/4244378724 shutdown_connections 2024-03-15T13:02:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.618+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/4244378724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9b05750 0x5641b9b07b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.618+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/4244378724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afcb70 0x5641b9b05210 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.618+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/4244378724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9afc250 0x5641b9afc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.618+0000 7fa16ebdc740 1 -- 172.21.15.2:0/4244378724 >> 172.21.15.2:0/4244378724 conn(0x5641b9af6980 msgr2=0x5641b9af6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.618+0000 7fa16ebdc740 1 -- 172.21.15.2:0/4244378724 shutdown_connections 2024-03-15T13:02:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.618+0000 7fa16ebdc740 1 -- 172.21.15.2:0/4244378724 wait complete. 2024-03-15T13:02:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.618+0000 7fa16ebdc740 1 -- start start 2024-03-15T13:02:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.618+0000 7fa16ebdc740 1 Processor -- start 2024-03-15T13:02:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa16ebdc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afc250 0x5641b9b05110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa16ebdc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9afcb70 0x5641b9b024a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa16ebdc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9b05750 0x5641b9b02ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa16ebdc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b9a64020 con 0x5641b9afc250 2024-03-15T13:02:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa16ebdc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b9737430 con 0x5641b9afcb70 2024-03-15T13:02:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa16ebdc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b96f3e00 con 0x5641b9b05750 2024-03-15T13:02:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa16ebdc740 10 client.0 _ll_register_callbacks cb 0x5641b9a0bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa169cba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afc250 0x5641b9b05110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa169cba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afc250 0x5641b9b05110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56828/0 (socket says 172.21.15.2:56828) 2024-03-15T13:02:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa169cba640 1 -- 172.21.15.2:0/3786343117 learned_addr learned my addr 172.21.15.2:0/3786343117 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa1694b9640 1 --2- 172.21.15.2:0/3786343117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9afcb70 0x5641b9b024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa16a4bb640 1 --2- 172.21.15.2:0/3786343117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9b05750 0x5641b9b02ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa169cba640 1 -- 172.21.15.2:0/3786343117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9b05750 msgr2=0x5641b9b02ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa169cba640 1 --2- 172.21.15.2:0/3786343117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9b05750 0x5641b9b02ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa169cba640 1 -- 172.21.15.2:0/3786343117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9afcb70 msgr2=0x5641b9b024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa169cba640 1 --2- 172.21.15.2:0/3786343117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9afcb70 0x5641b9b024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa169cba640 1 -- 172.21.15.2:0/3786343117 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1641a68f0 con 0x5641b9afc250 2024-03-15T13:02:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa169cba640 1 --2- 172.21.15.2:0/3786343117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afc250 0x5641b9b05110 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa160036040 tx=0x7fa16002ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d42583708535d37 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.619+0000 7fa14d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.620+0000 7fa14d7fa640 1 -- 172.21.15.2:0/3786343117 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160004040 con 0x5641b9afc250 2024-03-15T13:02:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.620+0000 7fa14d7fa640 1 -- 172.21.15.2:0/3786343117 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa1600041a0 con 0x5641b9afc250 2024-03-15T13:02:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.620+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3786343117 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641b97637e0 con 0x5641b9afc250 2024-03-15T13:02:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.620+0000 7fa14d7fa640 1 -- 172.21.15.2:0/3786343117 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa160004360 con 0x5641b9afc250 2024-03-15T13:02:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.620+0000 7fa16ebdc740 10 client.16815 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.620+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3786343117 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5641b9a626a0 con 0x5641b9afc250 2024-03-15T13:02:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.620+0000 7fa14d7fa640 1 -- 172.21.15.2:0/3786343117 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5641b97637e0 con 0x5641b9afc250 2024-03-15T13:02:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.620+0000 7fa14d7fa640 1 -- 172.21.15.2:0/3786343117 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fa16003c040 con 0x5641b9afc250 2024-03-15T13:02:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.620+0000 7fa16ebdc740 10 client.16815 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.620+0000 7fa16ebdc740 10 client.16815 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:02:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.620+0000 7fa16ebdc740 20 client.16815 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.620+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3786343117 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5641b9b157d0 0x5641b9b17bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.620+0000 7fa16ebdc740 4 client.16815 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:02:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.620+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3786343117 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641b97a0720 con 0x5641b9b157d0 2024-03-15T13:02:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.620+0000 7fa1694b9640 1 --2- 172.21.15.2:0/3786343117 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5641b9b157d0 0x5641b9b17bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.621+0000 7fa1694b9640 1 --2- 172.21.15.2:0/3786343117 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5641b9b157d0 0x5641b9b17bf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f04013ca3cc0b1b server_cookie=7ab67ffd5c417de5 in_seq=0 out_seq=0 2024-03-15T13:02:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.621+0000 7fa14d7fa640 10 client.16815 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:02:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.622+0000 7fa14d7fa640 1 -- 172.21.15.2:0/3786343117 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5641b97a0720 con 0x5641b9b157d0 2024-03-15T13:02:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.622+0000 7fa14d7fa640 10 client.16815 handle_command_reply: tid=0 2024-03-15T13:02:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.622+0000 7fa16ebdc740 1 client.16815 shutdown 2024-03-15T13:02:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.622+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3786343117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afc250 msgr2=0x5641b9b05110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.622+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3786343117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afc250 0x5641b9b05110 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa160036040 tx=0x7fa16002ca40 comp rx=0 tx=0).stop 2024-03-15T13:02:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.622+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3786343117 shutdown_connections 2024-03-15T13:02:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.622+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3786343117 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5641b9b157d0 0x5641b9b17bf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.622+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3786343117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9b05750 0x5641b9b02ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.622+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3786343117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9afcb70 0x5641b9b024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.622+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3786343117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afc250 0x5641b9b05110 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.622+0000 7fa14d7fa640 0 client.16815 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:02:33.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.622+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3786343117 >> 172.21.15.2:0/3786343117 conn(0x5641b9af6980 msgr2=0x5641b9af86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.622+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3786343117 shutdown_connections 2024-03-15T13:02:33.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.622+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3786343117 wait complete. 2024-03-15T13:02:33.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.622+0000 7fa16ebdc740 20 client.16815 trim_cache size 0 max 16384 2024-03-15T13:02:33.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.634+0000 7fa16ebdc740 1 Processor -- start 2024-03-15T13:02:33.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.634+0000 7fa16ebdc740 1 -- start start 2024-03-15T13:02:33.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa16ebdc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9afc250 0x5641b9af5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa16ebdc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afcb70 0x5641b9b04bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa16ebdc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9b05750 0x5641b9b05110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa16ebdc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b99e9130 con 0x5641b9afcb70 2024-03-15T13:02:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa16ebdc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b9b02870 con 0x5641b9afc250 2024-03-15T13:02:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa16ebdc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b9af6500 con 0x5641b9b05750 2024-03-15T13:02:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa1694b9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afcb70 0x5641b9b04bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa1694b9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afcb70 0x5641b9b04bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56838/0 (socket says 172.21.15.2:56838) 2024-03-15T13:02:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa1694b9640 1 -- 172.21.15.2:0/3280606747 learned_addr learned my addr 172.21.15.2:0/3280606747 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa169cba640 1 --2- 172.21.15.2:0/3280606747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9afc250 0x5641b9af5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa16a4bb640 1 --2- 172.21.15.2:0/3280606747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9b05750 0x5641b9b05110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa1694b9640 1 -- 172.21.15.2:0/3280606747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9b05750 msgr2=0x5641b9b05110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa1694b9640 1 --2- 172.21.15.2:0/3280606747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9b05750 0x5641b9b05110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa1694b9640 1 -- 172.21.15.2:0/3280606747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9afc250 msgr2=0x5641b9af5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa1694b9640 1 --2- 172.21.15.2:0/3280606747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9afc250 0x5641b9af5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa1694b9640 1 -- 172.21.15.2:0/3280606747 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641b99fec20 con 0x5641b9afcb70 2024-03-15T13:02:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa1694b9640 1 --2- 172.21.15.2:0/3280606747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afcb70 0x5641b9b04bd0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa1540cb350 tx=0x7fa1540cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbcca26ea07d72f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa14f7fe640 1 -- 172.21.15.2:0/3280606747 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540b2070 con 0x5641b9afcb70 2024-03-15T13:02:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa14f7fe640 1 -- 172.21.15.2:0/3280606747 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa1540b3270 con 0x5641b9afcb70 2024-03-15T13:02:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.635+0000 7fa14f7fe640 1 -- 172.21.15.2:0/3280606747 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540cb070 con 0x5641b9afcb70 2024-03-15T13:02:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.636+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3280606747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afcb70 msgr2=0x5641b9b04bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.636+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3280606747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afcb70 0x5641b9b04bd0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa1540cb350 tx=0x7fa1540cd040 comp rx=0 tx=0).stop 2024-03-15T13:02:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.636+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3280606747 shutdown_connections 2024-03-15T13:02:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.636+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3280606747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9b05750 0x5641b9b05110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.636+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3280606747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9afcb70 0x5641b9b04bd0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.636+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3280606747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9afc250 0x5641b9af5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.636+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3280606747 >> 172.21.15.2:0/3280606747 conn(0x5641b9af6980 msgr2=0x5641b9af52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.636+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3280606747 shutdown_connections 2024-03-15T13:02:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.636+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3280606747 wait complete. 2024-03-15T13:02:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.636+0000 7fa16ebdc740 1 -- start start 2024-03-15T13:02:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.636+0000 7fa16ebdc740 1 Processor -- start 2024-03-15T13:02:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16ebdc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9afc250 0x5641b9b08af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16ebdc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9afcb70 0x5641b9b09030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16ebdc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9b05750 0x5641b9b09570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16ebdc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b9b700d0 con 0x5641b9b05750 2024-03-15T13:02:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16ebdc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b9b70240 con 0x5641b9afcb70 2024-03-15T13:02:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16ebdc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641b9b70400 con 0x5641b9afc250 2024-03-15T13:02:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16ebdc740 10 client.0 _ll_register_callbacks cb 0x7fa160009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16a4bb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9b05750 0x5641b9b09570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16a4bb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9b05750 0x5641b9b09570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56848/0 (socket says 172.21.15.2:56848) 2024-03-15T13:02:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16a4bb640 1 -- 172.21.15.2:0/3103898224 learned_addr learned my addr 172.21.15.2:0/3103898224 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa1694b9640 1 --2- 172.21.15.2:0/3103898224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9afcb70 0x5641b9b09030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa169cba640 1 --2- 172.21.15.2:0/3103898224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9afc250 0x5641b9b08af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16a4bb640 1 -- 172.21.15.2:0/3103898224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9afc250 msgr2=0x5641b9b08af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16a4bb640 1 --2- 172.21.15.2:0/3103898224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9afc250 0x5641b9b08af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16a4bb640 1 -- 172.21.15.2:0/3103898224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9afcb70 msgr2=0x5641b9b09030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16a4bb640 1 --2- 172.21.15.2:0/3103898224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9afcb70 0x5641b9b09030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16a4bb640 1 -- 172.21.15.2:0/3103898224 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641b9a26060 con 0x5641b9b05750 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa16a4bb640 1 --2- 172.21.15.2:0/3103898224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9b05750 0x5641b9b09570 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0131e0 tx=0x7fa15c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14b659a74cc8346 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.637+0000 7fa14d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.638+0000 7fa14d7fa640 1 -- 172.21.15.2:0/3103898224 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c0045d0 con 0x5641b9b05750 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.638+0000 7fa14d7fa640 1 -- 172.21.15.2:0/3103898224 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa15c004730 con 0x5641b9b05750 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.638+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3103898224 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa15409ae80 con 0x5641b9b05750 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.638+0000 7fa14d7fa640 1 -- 172.21.15.2:0/3103898224 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa15c004b20 con 0x5641b9b05750 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.638+0000 7fa16ebdc740 10 client.16821 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.638+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3103898224 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5641b9a248c0 con 0x5641b9b05750 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.638+0000 7fa14d7fa640 1 -- 172.21.15.2:0/3103898224 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa15409ae80 con 0x5641b9b05750 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.638+0000 7fa14d7fa640 1 -- 172.21.15.2:0/3103898224 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fa15c005830 con 0x5641b9b05750 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.638+0000 7fa16ebdc740 10 client.16821 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.638+0000 7fa16ebdc740 10 client.16821 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:02:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.638+0000 7fa16ebdc740 20 client.16821 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.638+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3103898224 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5641b9b71c90 0x5641b9b740b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.638+0000 7fa16ebdc740 4 client.16821 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:02:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.638+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3103898224 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5641b97ef890 con 0x5641b9b71c90 2024-03-15T13:02:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.638+0000 7fa169cba640 1 --2- 172.21.15.2:0/3103898224 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5641b9b71c90 0x5641b9b740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa14d7fa640 10 client.16821 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:02:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa169cba640 1 --2- 172.21.15.2:0/3103898224 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5641b9b71c90 0x5641b9b740b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=205c4d1d96250785 server_cookie=ff6b769952c4ad01 in_seq=0 out_seq=0 2024-03-15T13:02:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa14d7fa640 1 -- 172.21.15.2:0/3103898224 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5641b97ef890 con 0x5641b9b71c90 2024-03-15T13:02:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa14d7fa640 10 client.16821 handle_command_reply: tid=0 2024-03-15T13:02:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa16ebdc740 1 client.16821 shutdown 2024-03-15T13:02:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3103898224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9b05750 msgr2=0x5641b9b09570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3103898224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9b05750 0x5641b9b09570 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0131e0 tx=0x7fa15c009500 comp rx=0 tx=0).stop 2024-03-15T13:02:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3103898224 shutdown_connections 2024-03-15T13:02:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3103898224 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5641b9b71c90 0x5641b9b740b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3103898224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5641b9b05750 0x5641b9b09570 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3103898224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641b9afcb70 0x5641b9b09030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa16ebdc740 1 --2- 172.21.15.2:0/3103898224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5641b9afc250 0x5641b9b08af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa14d7fa640 0 client.16821 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:02:33.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3103898224 >> 172.21.15.2:0/3103898224 conn(0x5641b9af6980 msgr2=0x5641b9af5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3103898224 shutdown_connections 2024-03-15T13:02:33.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa16ebdc740 1 -- 172.21.15.2:0/3103898224 wait complete. 2024-03-15T13:02:33.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.639+0000 7fa16ebdc740 20 client.16821 trim_cache size 0 max 16384 2024-03-15T13:02:33.631 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:02:33.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.640+0000 7fa16bf1e640 1 -- 172.21.15.2:0/1366282027 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa1500a1ba0 msgr2=0x7fa1500a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.640+0000 7fa16bf1e640 1 --2- 172.21.15.2:0/1366282027 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa1500a1ba0 0x7fa1500a4060 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x5641b9af7b10 tx=0x7fa15c009a90 comp rx=0 tx=0).stop 2024-03-15T13:02:33.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.640+0000 7fa16bf1e640 1 -- 172.21.15.2:0/1366282027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164101e60 msgr2=0x7fa164073d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.640+0000 7fa16bf1e640 1 --2- 172.21.15.2:0/1366282027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164101e60 0x7fa164073d30 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa15400e9f0 tx=0x7fa15400edf0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.640+0000 7fa16bf1e640 1 -- 172.21.15.2:0/1366282027 shutdown_connections 2024-03-15T13:02:33.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.640+0000 7fa16bf1e640 1 --2- 172.21.15.2:0/1366282027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa164107560 0x7fa164074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.640+0000 7fa16bf1e640 1 --2- 172.21.15.2:0/1366282027 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa1500a1ba0 0x7fa1500a4060 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.640+0000 7fa16bf1e640 1 --2- 172.21.15.2:0/1366282027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164101e60 0x7fa164073d30 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.640+0000 7fa16bf1e640 1 --2- 172.21.15.2:0/1366282027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1641014a0 0x7fa1640737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.640+0000 7fa16bf1e640 1 -- 172.21.15.2:0/1366282027 >> 172.21.15.2:0/1366282027 conn(0x7fa1640788f0 msgr2=0x7fa164100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.640+0000 7fa16bf1e640 1 -- 172.21.15.2:0/1366282027 shutdown_connections 2024-03-15T13:02:33.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.640+0000 7fa16bf1e640 1 -- 172.21.15.2:0/1366282027 wait complete. 2024-03-15T13:02:33.636 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:02:33.636 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:02:33.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.756+0000 7fa537a37640 1 -- 172.21.15.2:0/202813159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa530100750 msgr2=0x7fa53010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.756+0000 7fa537a37640 1 --2- 172.21.15.2:0/202813159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa530100750 0x7fa53010ca20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa528009ca0 tx=0x7fa52802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.756+0000 7fa537a37640 1 -- 172.21.15.2:0/202813159 shutdown_connections 2024-03-15T13:02:33.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.756+0000 7fa537a37640 1 --2- 172.21.15.2:0/202813159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa530100750 0x7fa53010ca20 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.756+0000 7fa537a37640 1 --2- 172.21.15.2:0/202813159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5300ffdb0 0x7fa530100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.756+0000 7fa537a37640 1 --2- 172.21.15.2:0/202813159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa530104b20 0x7fa530104f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.756+0000 7fa537a37640 1 -- 172.21.15.2:0/202813159 >> 172.21.15.2:0/202813159 conn(0x7fa530075730 msgr2=0x7fa530075b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.756+0000 7fa537a37640 1 -- 172.21.15.2:0/202813159 shutdown_connections 2024-03-15T13:02:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.756+0000 7fa537a37640 1 -- 172.21.15.2:0/202813159 wait complete. 2024-03-15T13:02:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.756+0000 7fa537a37640 1 Processor -- start 2024-03-15T13:02:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.756+0000 7fa537a37640 1 -- start start 2024-03-15T13:02:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa537a37640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5300ffdb0 0x7fa53019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa537a37640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa530100750 0x7fa53019dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa537a37640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa530104b20 0x7fa5301978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa537a37640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa530197ef0 con 0x7fa530100750 2024-03-15T13:02:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa537a37640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa530198060 con 0x7fa530104b20 2024-03-15T13:02:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa537a37640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa530198220 con 0x7fa5300ffdb0 2024-03-15T13:02:33.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa534fd2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa530100750 0x7fa53019dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa534fd2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa530100750 0x7fa53019dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56862/0 (socket says 172.21.15.2:56862) 2024-03-15T13:02:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa534fd2640 1 -- 172.21.15.2:0/1223588785 learned_addr learned my addr 172.21.15.2:0/1223588785 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa5357d3640 1 --2- 172.21.15.2:0/1223588785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5300ffdb0 0x7fa53019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa535fd4640 1 --2- 172.21.15.2:0/1223588785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa530104b20 0x7fa5301978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa534fd2640 1 -- 172.21.15.2:0/1223588785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5300ffdb0 msgr2=0x7fa53019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa534fd2640 1 --2- 172.21.15.2:0/1223588785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5300ffdb0 0x7fa53019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa534fd2640 1 -- 172.21.15.2:0/1223588785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa530104b20 msgr2=0x7fa5301978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa534fd2640 1 --2- 172.21.15.2:0/1223588785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa530104b20 0x7fa5301978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa534fd2640 1 -- 172.21.15.2:0/1223588785 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa528009900 con 0x7fa530100750 2024-03-15T13:02:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa534fd2640 1 --2- 172.21.15.2:0/1223588785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa530100750 0x7fa53019dc30 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa52000e9f0 tx=0x7fa52000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa5267fc640 1 -- 172.21.15.2:0/1223588785 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520009700 con 0x7fa530100750 2024-03-15T13:02:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa537a37640 1 -- 172.21.15.2:0/1223588785 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa530198500 con 0x7fa530100750 2024-03-15T13:02:33.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa5267fc640 1 -- 172.21.15.2:0/1223588785 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa52000cd80 con 0x7fa530100750 2024-03-15T13:02:33.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa5267fc640 1 -- 172.21.15.2:0/1223588785 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520018920 con 0x7fa530100750 2024-03-15T13:02:33.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.757+0000 7fa537a37640 1 -- 172.21.15.2:0/1223588785 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5301a4790 con 0x7fa530100750 2024-03-15T13:02:33.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.760+0000 7fa5267fc640 1 -- 172.21.15.2:0/1223588785 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fa520018a80 con 0x7fa530100750 2024-03-15T13:02:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.760+0000 7fa53a60d740 1 Processor -- start 2024-03-15T13:02:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.760+0000 7fa53a60d740 1 -- start start 2024-03-15T13:02:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.760+0000 7fa53a60d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963810250 0x55e963810630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.760+0000 7fa53a60d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810b70 0x55e963819210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.760+0000 7fa53a60d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963819750 0x55e96381bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.760+0000 7fa53a60d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9637cb960 con 0x55e963810250 2024-03-15T13:02:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.760+0000 7fa53a60d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96377f310 con 0x55e963810b70 2024-03-15T13:02:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.760+0000 7fa53a60d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96377c840 con 0x55e963819750 2024-03-15T13:02:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.760+0000 7fa535fd4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963810250 0x55e963810630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.760+0000 7fa5267fc640 1 --2- 172.21.15.2:0/1223588785 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa51c0a1ba0 0x7fa51c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.760+0000 7fa535fd4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963810250 0x55e963810630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56864/0 (socket says 172.21.15.2:56864) 2024-03-15T13:02:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa535fd4640 1 -- 172.21.15.2:0/1641542936 learned_addr learned my addr 172.21.15.2:0/1641542936 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa5357d3640 1 --2- 172.21.15.2:0/1641542936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810b70 0x55e963819210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa534fd2640 1 --2- 172.21.15.2:0/1641542936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963819750 0x55e96381bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa535fd4640 1 --2- 172.21.15.2:0/1223588785 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa51c0a1ba0 0x7fa51c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa5357d3640 1 -- 172.21.15.2:0/1641542936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963819750 msgr2=0x55e96381bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa5357d3640 1 --2- 172.21.15.2:0/1641542936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963819750 0x55e96381bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa5357d3640 1 -- 172.21.15.2:0/1641542936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963810250 msgr2=0x55e963810630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa5357d3640 1 --2- 172.21.15.2:0/1641542936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963810250 0x55e963810630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa5357d3640 1 -- 172.21.15.2:0/1641542936 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e96373a060 con 0x55e963810b70 2024-03-15T13:02:33.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa5267fc640 1 -- 172.21.15.2:0/1223588785 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa520014070 con 0x7fa530100750 2024-03-15T13:02:33.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa535fd4640 1 --2- 172.21.15.2:0/1641542936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963810250 0x55e963810630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:33.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa5357d3640 1 --2- 172.21.15.2:0/1641542936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810b70 0x55e963819210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009bd0 tx=0x7fa52c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb7dcc92f2101f2f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa535fd4640 1 --2- 172.21.15.2:0/1223588785 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa51c0a1ba0 0x7fa51c0a4060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa528033000 tx=0x7fa52802c0f0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa51b7fe640 1 -- 172.21.15.2:0/1641542936 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c004230 con 0x55e963810b70 2024-03-15T13:02:33.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa51b7fe640 1 -- 172.21.15.2:0/1641542936 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa52c009e40 con 0x55e963810b70 2024-03-15T13:02:33.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.761+0000 7fa51b7fe640 1 -- 172.21.15.2:0/1641542936 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c004ce0 con 0x55e963810b70 2024-03-15T13:02:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.762+0000 7fa53a60d740 1 -- 172.21.15.2:0/1641542936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810b70 msgr2=0x55e963819210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.762+0000 7fa53a60d740 1 --2- 172.21.15.2:0/1641542936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810b70 0x55e963819210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009bd0 tx=0x7fa52c011810 comp rx=0 tx=0).stop 2024-03-15T13:02:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.762+0000 7fa53a60d740 1 -- 172.21.15.2:0/1641542936 shutdown_connections 2024-03-15T13:02:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.762+0000 7fa53a60d740 1 --2- 172.21.15.2:0/1641542936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963819750 0x55e96381bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.762+0000 7fa53a60d740 1 --2- 172.21.15.2:0/1641542936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810b70 0x55e963819210 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.762+0000 7fa53a60d740 1 --2- 172.21.15.2:0/1641542936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963810250 0x55e963810630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.762+0000 7fa53a60d740 1 -- 172.21.15.2:0/1641542936 >> 172.21.15.2:0/1641542936 conn(0x55e96380a980 msgr2=0x55e96380ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.762+0000 7fa53a60d740 1 -- 172.21.15.2:0/1641542936 shutdown_connections 2024-03-15T13:02:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.762+0000 7fa53a60d740 1 -- 172.21.15.2:0/1641542936 wait complete. 2024-03-15T13:02:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.762+0000 7fa53a60d740 1 -- start start 2024-03-15T13:02:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.762+0000 7fa53a60d740 1 Processor -- start 2024-03-15T13:02:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa53a60d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810250 0x55e9638190d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa53a60d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963810b70 0x55e963816460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa53a60d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963819750 0x55e9638169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa53a60d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e963778020 con 0x55e963810b70 2024-03-15T13:02:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa53a60d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96344b430 con 0x55e963810250 2024-03-15T13:02:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa53a60d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e963407e00 con 0x55e963819750 2024-03-15T13:02:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa53a60d740 10 client.0 _ll_register_callbacks cb 0x55e96371fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa534fd2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963810b70 0x55e963816460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa534fd2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963810b70 0x55e963816460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56876/0 (socket says 172.21.15.2:56876) 2024-03-15T13:02:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa534fd2640 1 -- 172.21.15.2:0/3263378174 learned_addr learned my addr 172.21.15.2:0/3263378174 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa5357d3640 1 --2- 172.21.15.2:0/3263378174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810250 0x55e9638190d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa535fd4640 1 --2- 172.21.15.2:0/3263378174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963819750 0x55e9638169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa534fd2640 1 -- 172.21.15.2:0/3263378174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963819750 msgr2=0x55e9638169e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa534fd2640 1 --2- 172.21.15.2:0/3263378174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963819750 0x55e9638169e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa534fd2640 1 -- 172.21.15.2:0/3263378174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810250 msgr2=0x55e9638190d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa534fd2640 1 --2- 172.21.15.2:0/3263378174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810250 0x55e9638190d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa534fd2640 1 -- 172.21.15.2:0/3263378174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52c009830 con 0x55e963810b70 2024-03-15T13:02:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa5357d3640 1 --2- 172.21.15.2:0/3263378174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810250 0x55e9638190d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:02:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa534fd2640 1 --2- 172.21.15.2:0/3263378174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963810b70 0x55e963816460 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa52009a500 tx=0x7fa52009e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceaa37e557d1fd51 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa518ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa518ff9640 1 -- 172.21.15.2:0/3263378174 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5200ba0b0 con 0x55e963810b70 2024-03-15T13:02:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.763+0000 7fa518ff9640 1 -- 172.21.15.2:0/3263378174 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa5200ba210 con 0x55e963810b70 2024-03-15T13:02:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.764+0000 7fa53a60d740 1 -- 172.21.15.2:0/3263378174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9634777e0 con 0x55e963810b70 2024-03-15T13:02:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.764+0000 7fa518ff9640 1 -- 172.21.15.2:0/3263378174 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5200b30d0 con 0x55e963810b70 2024-03-15T13:02:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.764+0000 7fa53a60d740 10 client.16833 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.764+0000 7fa53a60d740 1 -- 172.21.15.2:0/3263378174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55e9637766a0 con 0x55e963810b70 2024-03-15T13:02:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.764+0000 7fa518ff9640 1 -- 172.21.15.2:0/3263378174 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e9634777e0 con 0x55e963810b70 2024-03-15T13:02:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.764+0000 7fa518ff9640 1 -- 172.21.15.2:0/3263378174 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fa520085270 con 0x55e963810b70 2024-03-15T13:02:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.764+0000 7fa53a60d740 10 client.16833 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.764+0000 7fa53a60d740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:02:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.764+0000 7fa53a60d740 20 client.16833 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.764+0000 7fa53a60d740 1 --2- 172.21.15.2:0/3263378174 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e9638298d0 0x55e96382bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.764+0000 7fa53a60d740 4 client.16833 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:02:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.764+0000 7fa53a60d740 1 -- 172.21.15.2:0/3263378174 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9634b4720 con 0x55e9638298d0 2024-03-15T13:02:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.764+0000 7fa5357d3640 1 --2- 172.21.15.2:0/3263378174 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e9638298d0 0x55e96382bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.765+0000 7fa518ff9640 10 client.16833 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.765+0000 7fa5357d3640 1 --2- 172.21.15.2:0/3263378174 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e9638298d0 0x55e96382bcf0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=860ca88eb9ebf9bc server_cookie=aafb338477fc6e60 in_seq=0 out_seq=0 2024-03-15T13:02:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.765+0000 7fa518ff9640 1 -- 172.21.15.2:0/3263378174 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e9634b4720 con 0x55e9638298d0 2024-03-15T13:02:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.765+0000 7fa518ff9640 10 client.16833 handle_command_reply: tid=0 2024-03-15T13:02:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.765+0000 7fa53a60d740 1 client.16833 shutdown 2024-03-15T13:02:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.765+0000 7fa53a60d740 1 -- 172.21.15.2:0/3263378174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963810b70 msgr2=0x55e963816460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.765+0000 7fa53a60d740 1 --2- 172.21.15.2:0/3263378174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963810b70 0x55e963816460 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa52009a500 tx=0x7fa52009e040 comp rx=0 tx=0).stop 2024-03-15T13:02:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.765+0000 7fa53a60d740 1 -- 172.21.15.2:0/3263378174 shutdown_connections 2024-03-15T13:02:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.765+0000 7fa53a60d740 1 --2- 172.21.15.2:0/3263378174 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e9638298d0 0x55e96382bcf0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.765+0000 7fa53a60d740 1 --2- 172.21.15.2:0/3263378174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963819750 0x55e9638169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.765+0000 7fa53a60d740 1 --2- 172.21.15.2:0/3263378174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963810b70 0x55e963816460 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.765+0000 7fa518ff9640 0 client.16833 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.765+0000 7fa53a60d740 1 --2- 172.21.15.2:0/3263378174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810250 0x55e9638190d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.765+0000 7fa53a60d740 1 -- 172.21.15.2:0/3263378174 >> 172.21.15.2:0/3263378174 conn(0x55e96380a980 msgr2=0x55e96380c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.766+0000 7fa53a60d740 1 -- 172.21.15.2:0/3263378174 shutdown_connections 2024-03-15T13:02:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.766+0000 7fa53a60d740 1 -- 172.21.15.2:0/3263378174 wait complete. 2024-03-15T13:02:33.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.766+0000 7fa53a60d740 20 client.16833 trim_cache size 0 max 16384 2024-03-15T13:02:33.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.778+0000 7fa53a60d740 1 Processor -- start 2024-03-15T13:02:33.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.778+0000 7fa53a60d740 1 -- start start 2024-03-15T13:02:33.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.778+0000 7fa53a60d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963810250 0x55e9638092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.778+0000 7fa53a60d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810b70 0x55e963818b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.778+0000 7fa53a60d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963819750 0x55e9638190d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.778+0000 7fa53a60d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9636fd130 con 0x55e963819750 2024-03-15T13:02:33.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.778+0000 7fa53a60d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e963809860 con 0x55e963810b70 2024-03-15T13:02:33.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.778+0000 7fa53a60d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96380b5a0 con 0x55e963810250 2024-03-15T13:02:33.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.778+0000 7fa535fd4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963819750 0x55e9638190d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.778+0000 7fa535fd4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963819750 0x55e9638190d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56878/0 (socket says 172.21.15.2:56878) 2024-03-15T13:02:33.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.778+0000 7fa535fd4640 1 -- 172.21.15.2:0/591480078 learned_addr learned my addr 172.21.15.2:0/591480078 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.778+0000 7fa5357d3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963810250 0x55e9638092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.778+0000 7fa534fd2640 1 --2- 172.21.15.2:0/591480078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810b70 0x55e963818b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.779+0000 7fa535fd4640 1 -- 172.21.15.2:0/591480078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963810250 msgr2=0x55e9638092f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.779+0000 7fa535fd4640 1 --2- 172.21.15.2:0/591480078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963810250 0x55e9638092f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.779+0000 7fa535fd4640 1 -- 172.21.15.2:0/591480078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810b70 msgr2=0x55e963818b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.779+0000 7fa535fd4640 1 --2- 172.21.15.2:0/591480078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810b70 0x55e963818b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.779+0000 7fa535fd4640 1 -- 172.21.15.2:0/591480078 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e963712c20 con 0x55e963819750 2024-03-15T13:02:33.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.779+0000 7fa535fd4640 1 --2- 172.21.15.2:0/591480078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963819750 0x55e9638190d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa52802c700 tx=0x7fa528009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0af2ba155fa5c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.779+0000 7fa51affd640 1 -- 172.21.15.2:0/591480078 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5280328b0 con 0x55e963819750 2024-03-15T13:02:33.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.779+0000 7fa51affd640 1 -- 172.21.15.2:0/591480078 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa528034790 con 0x55e963819750 2024-03-15T13:02:33.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.779+0000 7fa51affd640 1 -- 172.21.15.2:0/591480078 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa528004670 con 0x55e963819750 2024-03-15T13:02:33.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.779+0000 7fa53a60d740 1 -- 172.21.15.2:0/591480078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963819750 msgr2=0x55e9638190d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.779+0000 7fa53a60d740 1 --2- 172.21.15.2:0/591480078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963819750 0x55e9638190d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa52802c700 tx=0x7fa528009360 comp rx=0 tx=0).stop 2024-03-15T13:02:33.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.779+0000 7fa53a60d740 1 -- 172.21.15.2:0/591480078 shutdown_connections 2024-03-15T13:02:33.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.779+0000 7fa53a60d740 1 --2- 172.21.15.2:0/591480078 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963819750 0x55e9638190d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.779+0000 7fa53a60d740 1 --2- 172.21.15.2:0/591480078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810b70 0x55e963818b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.779+0000 7fa53a60d740 1 --2- 172.21.15.2:0/591480078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963810250 0x55e9638092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa53a60d740 1 -- 172.21.15.2:0/591480078 >> 172.21.15.2:0/591480078 conn(0x55e96380a980 msgr2=0x55e96380c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa53a60d740 1 -- 172.21.15.2:0/591480078 shutdown_connections 2024-03-15T13:02:33.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa53a60d740 1 -- 172.21.15.2:0/591480078 wait complete. 2024-03-15T13:02:33.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa53a60d740 1 -- start start 2024-03-15T13:02:33.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa53a60d740 1 Processor -- start 2024-03-15T13:02:33.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa53a60d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810250 0x55e96381e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa53a60d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963810b70 0x55e96381eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa53a60d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963819750 0x55e96381f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa53a60d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9638144c0 con 0x55e963819750 2024-03-15T13:02:33.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa53a60d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9638851d0 con 0x55e963810250 2024-03-15T13:02:33.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa53a60d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e963885310 con 0x55e963810b70 2024-03-15T13:02:33.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa53a60d740 10 client.0 _ll_register_callbacks cb 0x7fa52009a410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:33.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa535fd4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963819750 0x55e96381f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa535fd4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963819750 0x55e96381f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56894/0 (socket says 172.21.15.2:56894) 2024-03-15T13:02:33.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa535fd4640 1 -- 172.21.15.2:0/806383231 learned_addr learned my addr 172.21.15.2:0/806383231 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:33.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa5357d3640 1 --2- 172.21.15.2:0/806383231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810250 0x55e96381e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.780+0000 7fa534fd2640 1 --2- 172.21.15.2:0/806383231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963810b70 0x55e96381eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.781+0000 7fa535fd4640 1 -- 172.21.15.2:0/806383231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963810b70 msgr2=0x55e96381eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.781+0000 7fa535fd4640 1 --2- 172.21.15.2:0/806383231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963810b70 0x55e96381eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.781+0000 7fa535fd4640 1 -- 172.21.15.2:0/806383231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810250 msgr2=0x55e96381e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:33.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.781+0000 7fa535fd4640 1 --2- 172.21.15.2:0/806383231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810250 0x55e96381e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.781+0000 7fa535fd4640 1 -- 172.21.15.2:0/806383231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa528030070 con 0x55e963819750 2024-03-15T13:02:33.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.781+0000 7fa534fd2640 1 --2- 172.21.15.2:0/806383231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963810b70 0x55e96381eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:33.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.781+0000 7fa535fd4640 1 --2- 172.21.15.2:0/806383231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963819750 0x55e96381f060 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa528004af0 tx=0x7fa5280048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6566cb2469abd4a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:33.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.781+0000 7fa518ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:33.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.781+0000 7fa518ff9640 1 -- 172.21.15.2:0/806383231 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa528034570 con 0x55e963819750 2024-03-15T13:02:33.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.781+0000 7fa53a60d740 1 -- 172.21.15.2:0/806383231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e963503890 con 0x55e963819750 2024-03-15T13:02:33.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.781+0000 7fa518ff9640 1 -- 172.21.15.2:0/806383231 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa528032ac0 con 0x55e963819750 2024-03-15T13:02:33.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.781+0000 7fa518ff9640 1 -- 172.21.15.2:0/806383231 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa528034790 con 0x55e963819750 2024-03-15T13:02:33.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.781+0000 7fa53a60d740 10 client.16839 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:33.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.781+0000 7fa53a60d740 1 -- 172.21.15.2:0/806383231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55e9637388c0 con 0x55e963819750 2024-03-15T13:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.781+0000 7fa518ff9640 1 -- 172.21.15.2:0/806383231 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e963503890 con 0x55e963819750 2024-03-15T13:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.782+0000 7fa518ff9640 1 -- 172.21.15.2:0/806383231 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fa52803e040 con 0x55e963819750 2024-03-15T13:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.782+0000 7fa53a60d740 10 client.16839 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.782+0000 7fa53a60d740 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.782+0000 7fa53a60d740 20 client.16839 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.782+0000 7fa53a60d740 1 --2- 172.21.15.2:0/806383231 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e963886870 0x55e963888c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.782+0000 7fa53a60d740 4 client.16839 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.782+0000 7fa53a60d740 1 -- 172.21.15.2:0/806383231 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e963423700 con 0x55e963886870 2024-03-15T13:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.782+0000 7fa5357d3640 1 --2- 172.21.15.2:0/806383231 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e963886870 0x55e963888c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.782+0000 7fa518ff9640 10 client.16839 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.782+0000 7fa5357d3640 1 --2- 172.21.15.2:0/806383231 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e963886870 0x55e963888c90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91e81afaf2e89ec3 server_cookie=7d53a55f87cf74e7 in_seq=0 out_seq=0 2024-03-15T13:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.782+0000 7fa518ff9640 1 -- 172.21.15.2:0/806383231 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e963423700 con 0x55e963886870 2024-03-15T13:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.782+0000 7fa518ff9640 10 client.16839 handle_command_reply: tid=0 2024-03-15T13:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.782+0000 7fa53a60d740 1 client.16839 shutdown 2024-03-15T13:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.782+0000 7fa53a60d740 1 -- 172.21.15.2:0/806383231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963819750 msgr2=0x55e96381f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.782+0000 7fa53a60d740 1 --2- 172.21.15.2:0/806383231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963819750 0x55e96381f060 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa528004af0 tx=0x7fa5280048a0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa53a60d740 1 -- 172.21.15.2:0/806383231 shutdown_connections 2024-03-15T13:02:33.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa53a60d740 1 --2- 172.21.15.2:0/806383231 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e963886870 0x55e963888c90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa53a60d740 1 --2- 172.21.15.2:0/806383231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e963819750 0x55e96381f060 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa53a60d740 1 --2- 172.21.15.2:0/806383231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e963810b70 0x55e96381eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa518ff9640 0 client.16839 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:33.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa53a60d740 1 --2- 172.21.15.2:0/806383231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e963810250 0x55e96381e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa53a60d740 1 -- 172.21.15.2:0/806383231 >> 172.21.15.2:0/806383231 conn(0x55e96380a980 msgr2=0x55e963811090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa53a60d740 1 -- 172.21.15.2:0/806383231 shutdown_connections 2024-03-15T13:02:33.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa53a60d740 1 -- 172.21.15.2:0/806383231 wait complete. 2024-03-15T13:02:33.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa53a60d740 20 client.16839 trim_cache size 0 max 16384 2024-03-15T13:02:33.776 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:02:33.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa537a37640 1 -- 172.21.15.2:0/1223588785 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa51c0a1ba0 msgr2=0x7fa51c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa537a37640 1 --2- 172.21.15.2:0/1223588785 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa51c0a1ba0 0x7fa51c0a4060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa528033000 tx=0x7fa52802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa537a37640 1 -- 172.21.15.2:0/1223588785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa530100750 msgr2=0x7fa53019dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:33.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa537a37640 1 --2- 172.21.15.2:0/1223588785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa530100750 0x7fa53019dc30 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa52000e9f0 tx=0x7fa52000eeb0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa537a37640 1 -- 172.21.15.2:0/1223588785 shutdown_connections 2024-03-15T13:02:33.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa537a37640 1 --2- 172.21.15.2:0/1223588785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa530104b20 0x7fa5301978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa537a37640 1 --2- 172.21.15.2:0/1223588785 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa51c0a1ba0 0x7fa51c0a4060 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa537a37640 1 --2- 172.21.15.2:0/1223588785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa530100750 0x7fa53019dc30 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa537a37640 1 --2- 172.21.15.2:0/1223588785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5300ffdb0 0x7fa53019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:33.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa537a37640 1 -- 172.21.15.2:0/1223588785 >> 172.21.15.2:0/1223588785 conn(0x7fa530075730 msgr2=0x7fa5300fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:33.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa537a37640 1 -- 172.21.15.2:0/1223588785 shutdown_connections 2024-03-15T13:02:33.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:33.783+0000 7fa537a37640 1 -- 172.21.15.2:0/1223588785 wait complete. 2024-03-15T13:02:34.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:34 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/88257928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:02:34.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:34 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/88257928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:02:34.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:34 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/88257928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:02:34.780 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:02:34.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:02:34.780 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:02:34.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.915+0000 7f3c71b81640 1 -- 172.21.15.2:0/2818940062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c107570 msgr2=0x7f3c6c107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.915+0000 7f3c71b81640 1 --2- 172.21.15.2:0/2818940062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c107570 0x7f3c6c107950 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009900 tx=0x7f3c5c02bc30 comp rx=0 tx=0).stop 2024-03-15T13:02:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.916+0000 7f3c71b81640 1 -- 172.21.15.2:0/2818940062 shutdown_connections 2024-03-15T13:02:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.916+0000 7f3c71b81640 1 --2- 172.21.15.2:0/2818940062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c6c101e70 0x7f3c6c10e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.916+0000 7f3c71b81640 1 --2- 172.21.15.2:0/2818940062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c1014b0 0x7f3c6c101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.916+0000 7f3c71b81640 1 --2- 172.21.15.2:0/2818940062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c107570 0x7f3c6c107950 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.916+0000 7f3c71b81640 1 -- 172.21.15.2:0/2818940062 >> 172.21.15.2:0/2818940062 conn(0x7f3c6c0fcf20 msgr2=0x7f3c6c0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.916+0000 7f3c71b81640 1 -- 172.21.15.2:0/2818940062 shutdown_connections 2024-03-15T13:02:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.916+0000 7f3c71b81640 1 -- 172.21.15.2:0/2818940062 wait complete. 2024-03-15T13:02:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.916+0000 7f3c71b81640 1 Processor -- start 2024-03-15T13:02:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.916+0000 7f3c71b81640 1 -- start start 2024-03-15T13:02:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c71b81640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c1014b0 0x7f3c6c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c71b81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c101e70 0x7f3c6c19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c71b81640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c6c107570 0x7f3c6c197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c71b81640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c197ed0 con 0x7f3c6c101e70 2024-03-15T13:02:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c71b81640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c198040 con 0x7f3c6c1014b0 2024-03-15T13:02:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c71b81640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c198200 con 0x7f3c6c107570 2024-03-15T13:02:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c6affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c101e70 0x7f3c6c19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c6affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c101e70 0x7f3c6c19dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56918/0 (socket says 172.21.15.2:56918) 2024-03-15T13:02:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c6affd640 1 -- 172.21.15.2:0/3187592443 learned_addr learned my addr 172.21.15.2:0/3187592443 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c6b7fe640 1 --2- 172.21.15.2:0/3187592443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c1014b0 0x7f3c6c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c6bfff640 1 --2- 172.21.15.2:0/3187592443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c6c107570 0x7f3c6c197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c6affd640 1 -- 172.21.15.2:0/3187592443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c6c107570 msgr2=0x7f3c6c197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c6affd640 1 --2- 172.21.15.2:0/3187592443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c6c107570 0x7f3c6c197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c6affd640 1 -- 172.21.15.2:0/3187592443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c1014b0 msgr2=0x7f3c6c19d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c6affd640 1 --2- 172.21.15.2:0/3187592443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c1014b0 0x7f3c6c19d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c6affd640 1 -- 172.21.15.2:0/3187592443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c009560 con 0x7f3c6c101e70 2024-03-15T13:02:34.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c6b7fe640 1 --2- 172.21.15.2:0/3187592443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c1014b0 0x7f3c6c19d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:34.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c6affd640 1 --2- 172.21.15.2:0/3187592443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c101e70 0x7f3c6c19dc10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000e9f0 tx=0x7f3c6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:34.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c68ff9640 1 -- 172.21.15.2:0/3187592443 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60009700 con 0x7f3c6c101e70 2024-03-15T13:02:34.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c68ff9640 1 -- 172.21.15.2:0/3187592443 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3c6000cd80 con 0x7f3c6c101e70 2024-03-15T13:02:34.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c71b81640 1 -- 172.21.15.2:0/3187592443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6c1984e0 con 0x7f3c6c101e70 2024-03-15T13:02:34.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.917+0000 7f3c68ff9640 1 -- 172.21.15.2:0/3187592443 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60018920 con 0x7f3c6c101e70 2024-03-15T13:02:34.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.918+0000 7f3c71b81640 1 -- 172.21.15.2:0/3187592443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6c1a4770 con 0x7f3c6c101e70 2024-03-15T13:02:34.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.920+0000 7f3c68ff9640 1 -- 172.21.15.2:0/3187592443 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f3c60018a80 con 0x7f3c6c101e70 2024-03-15T13:02:34.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.920+0000 7f3c74838740 1 Processor -- start 2024-03-15T13:02:34.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.920+0000 7f3c74838740 1 -- start start 2024-03-15T13:02:34.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.920+0000 7f3c74838740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616459631e0 0x5616459635c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.920+0000 7f3c74838740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x56164596c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.920+0000 7f3c74838740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164596ccf0 0x56164596d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.920+0000 7f3c74838740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616458d1f90 con 0x5616459631e0 2024-03-15T13:02:34.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.920+0000 7f3c74838740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616458cf4c0 con 0x561645963b90 2024-03-15T13:02:34.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.920+0000 7f3c74838740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616458caca0 con 0x56164596ccf0 2024-03-15T13:02:34.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c6bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616459631e0 0x5616459635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c6bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616459631e0 0x5616459635c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56930/0 (socket says 172.21.15.2:56930) 2024-03-15T13:02:34.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c6affd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164596ccf0 0x56164596d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c6bfff640 1 -- 172.21.15.2:0/2932701994 learned_addr learned my addr 172.21.15.2:0/2932701994 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:34.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c6b7fe640 1 --2- 172.21.15.2:0/2932701994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x56164596c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c68ff9640 1 --2- 172.21.15.2:0/3187592443 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3c540a1ba0 0x7f3c540a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c6bfff640 1 --2- 172.21.15.2:0/3187592443 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3c540a1ba0 0x7f3c540a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c6bfff640 1 -- 172.21.15.2:0/2932701994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164596ccf0 msgr2=0x56164596d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:34.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c6bfff640 1 --2- 172.21.15.2:0/2932701994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164596ccf0 0x56164596d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c6bfff640 1 -- 172.21.15.2:0/2932701994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 msgr2=0x56164596c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:34.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c6bfff640 1 --2- 172.21.15.2:0/2932701994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x56164596c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c68ff9640 1 -- 172.21.15.2:0/3187592443 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f3c60014070 con 0x7f3c6c101e70 2024-03-15T13:02:34.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c6bfff640 1 -- 172.21.15.2:0/2932701994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56164588d0b0 con 0x5616459631e0 2024-03-15T13:02:34.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c6bfff640 1 --2- 172.21.15.2:0/2932701994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616459631e0 0x5616459635c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3c64005c40 tx=0x7f3c6400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361fcd716b533220 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c51ffb640 1 -- 172.21.15.2:0/2932701994 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c64003ec0 con 0x5616459631e0 2024-03-15T13:02:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c6bfff640 1 --2- 172.21.15.2:0/3187592443 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3c540a1ba0 0x7f3c540a4060 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3c64023000 tx=0x7f3c64021000 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c51ffb640 1 -- 172.21.15.2:0/2932701994 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3c64013170 con 0x5616459631e0 2024-03-15T13:02:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.921+0000 7f3c51ffb640 1 -- 172.21.15.2:0/2932701994 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c640108d0 con 0x5616459631e0 2024-03-15T13:02:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.922+0000 7f3c74838740 1 -- 172.21.15.2:0/2932701994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616459631e0 msgr2=0x5616459635c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.922+0000 7f3c74838740 1 --2- 172.21.15.2:0/2932701994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616459631e0 0x5616459635c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3c64005c40 tx=0x7f3c6400f3c0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.922+0000 7f3c74838740 1 -- 172.21.15.2:0/2932701994 shutdown_connections 2024-03-15T13:02:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.922+0000 7f3c74838740 1 --2- 172.21.15.2:0/2932701994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164596ccf0 0x56164596d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.922+0000 7f3c74838740 1 --2- 172.21.15.2:0/2932701994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x56164596c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.922+0000 7f3c74838740 1 --2- 172.21.15.2:0/2932701994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616459631e0 0x5616459635c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.922+0000 7f3c74838740 1 -- 172.21.15.2:0/2932701994 >> 172.21.15.2:0/2932701994 conn(0x56164595d840 msgr2=0x56164595dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.922+0000 7f3c74838740 1 -- 172.21.15.2:0/2932701994 shutdown_connections 2024-03-15T13:02:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.922+0000 7f3c74838740 1 -- 172.21.15.2:0/2932701994 wait complete. 2024-03-15T13:02:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.922+0000 7f3c74838740 1 -- start start 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c74838740 1 Processor -- start 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c74838740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616459631e0 0x5616459707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c74838740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x561645970d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c74838740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164596ccf0 0x561645971260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c74838740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56164586ff40 con 0x5616459631e0 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c74838740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56164559e4b0 con 0x561645963b90 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c74838740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561645850670 con 0x56164596ccf0 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c74838740 10 client.0 _ll_register_callbacks cb 0x5616458bdf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c6b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616459631e0 0x5616459707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c6b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616459631e0 0x5616459707e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56938/0 (socket says 172.21.15.2:56938) 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c6b7fe640 1 -- 172.21.15.2:0/3221036265 learned_addr learned my addr 172.21.15.2:0/3221036265 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c6affd640 1 --2- 172.21.15.2:0/3221036265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x561645970d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c6bfff640 1 --2- 172.21.15.2:0/3221036265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164596ccf0 0x561645971260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c6b7fe640 1 -- 172.21.15.2:0/3221036265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164596ccf0 msgr2=0x561645971260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c6b7fe640 1 --2- 172.21.15.2:0/3221036265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164596ccf0 0x561645971260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c6b7fe640 1 -- 172.21.15.2:0/3221036265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 msgr2=0x561645970d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:34.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c6b7fe640 1 --2- 172.21.15.2:0/3221036265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x561645970d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.923+0000 7f3c6b7fe640 1 -- 172.21.15.2:0/3221036265 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c64013700 con 0x5616459631e0 2024-03-15T13:02:34.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c6b7fe640 1 --2- 172.21.15.2:0/3221036265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616459631e0 0x5616459707e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009900 tx=0x7f3c5c004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af03fa7ca056de33 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:34.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c4affd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:34.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c4affd640 1 -- 172.21.15.2:0/3221036265 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c004350 con 0x5616459631e0 2024-03-15T13:02:34.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c4affd640 1 -- 172.21.15.2:0/3221036265 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f3c5c0044b0 con 0x5616459631e0 2024-03-15T13:02:34.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c74838740 1 -- 172.21.15.2:0/3221036265 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616458a4f40 con 0x5616459631e0 2024-03-15T13:02:34.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c4affd640 1 -- 172.21.15.2:0/3221036265 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c5c0035f0 con 0x5616459631e0 2024-03-15T13:02:34.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c74838740 10 client.16851 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:34.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c74838740 1 -- 172.21.15.2:0/3221036265 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5616458c9320 con 0x5616459631e0 2024-03-15T13:02:34.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c4affd640 1 -- 172.21.15.2:0/3221036265 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5616458a4f40 con 0x5616459631e0 2024-03-15T13:02:34.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c4affd640 1 -- 172.21.15.2:0/3221036265 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f3c5c03c040 con 0x5616459631e0 2024-03-15T13:02:34.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c74838740 10 client.16851 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:34.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c74838740 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:02:34.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c74838740 20 client.16851 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:34.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c74838740 1 --2- 172.21.15.2:0/3221036265 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x561645981390 0x5616459837b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c74838740 4 client.16851 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:02:34.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.924+0000 7f3c74838740 1 -- 172.21.15.2:0/3221036265 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616458c1310 con 0x561645981390 2024-03-15T13:02:34.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.925+0000 7f3c6affd640 1 --2- 172.21.15.2:0/3221036265 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x561645981390 0x5616459837b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.925+0000 7f3c4affd640 10 client.16851 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:34.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.925+0000 7f3c6affd640 1 --2- 172.21.15.2:0/3221036265 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x561645981390 0x5616459837b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b767de55aacf102e server_cookie=86c59aeb13675eb7 in_seq=0 out_seq=0 2024-03-15T13:02:34.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.926+0000 7f3c4affd640 1 -- 172.21.15.2:0/3221036265 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5616458c1310 con 0x561645981390 2024-03-15T13:02:34.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.926+0000 7f3c4affd640 10 client.16851 handle_command_reply: tid=0 2024-03-15T13:02:34.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.926+0000 7f3c74838740 1 client.16851 shutdown 2024-03-15T13:02:34.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.926+0000 7f3c74838740 1 -- 172.21.15.2:0/3221036265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616459631e0 msgr2=0x5616459707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:34.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.926+0000 7f3c74838740 1 --2- 172.21.15.2:0/3221036265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616459631e0 0x5616459707e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009900 tx=0x7f3c5c004be0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.926+0000 7f3c74838740 1 -- 172.21.15.2:0/3221036265 shutdown_connections 2024-03-15T13:02:34.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.926+0000 7f3c74838740 1 --2- 172.21.15.2:0/3221036265 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x561645981390 0x5616459837b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.926+0000 7f3c74838740 1 --2- 172.21.15.2:0/3221036265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56164596ccf0 0x561645971260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.926+0000 7f3c74838740 1 --2- 172.21.15.2:0/3221036265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x561645970d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.926+0000 7f3c4affd640 0 client.16851 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:34.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.926+0000 7f3c74838740 1 --2- 172.21.15.2:0/3221036265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5616459631e0 0x5616459707e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.926+0000 7f3c74838740 1 -- 172.21.15.2:0/3221036265 >> 172.21.15.2:0/3221036265 conn(0x56164595d840 msgr2=0x56164595f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:34.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.926+0000 7f3c74838740 1 -- 172.21.15.2:0/3221036265 shutdown_connections 2024-03-15T13:02:34.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.926+0000 7f3c74838740 1 -- 172.21.15.2:0/3221036265 wait complete. 2024-03-15T13:02:34.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.926+0000 7f3c74838740 20 client.16851 trim_cache size 0 max 16384 2024-03-15T13:02:34.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.942+0000 7f3c74838740 1 Processor -- start 2024-03-15T13:02:34.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c74838740 1 -- start start 2024-03-15T13:02:34.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c74838740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616459631e0 0x561645972ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c74838740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x56164596e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c74838740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56164596ccf0 0x56164597c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c74838740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56164596bda0 con 0x56164596ccf0 2024-03-15T13:02:34.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c74838740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56164596e620 con 0x561645963b90 2024-03-15T13:02:34.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c74838740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56164596e7e0 con 0x5616459631e0 2024-03-15T13:02:34.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c6bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56164596ccf0 0x56164597c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c6affd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x56164596e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c6bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56164596ccf0 0x56164597c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56942/0 (socket says 172.21.15.2:56942) 2024-03-15T13:02:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c6bfff640 1 -- 172.21.15.2:0/2756465989 learned_addr learned my addr 172.21.15.2:0/2756465989 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c6b7fe640 1 --2- 172.21.15.2:0/2756465989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616459631e0 0x561645972ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c6affd640 1 -- 172.21.15.2:0/2756465989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616459631e0 msgr2=0x561645972ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c6affd640 1 --2- 172.21.15.2:0/2756465989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616459631e0 0x561645972ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c6affd640 1 -- 172.21.15.2:0/2756465989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56164596ccf0 msgr2=0x56164597c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c6affd640 1 --2- 172.21.15.2:0/2756465989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56164596ccf0 0x56164597c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c6affd640 1 -- 172.21.15.2:0/2756465989 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561645865c00 con 0x561645963b90 2024-03-15T13:02:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c6bfff640 1 --2- 172.21.15.2:0/2756465989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56164596ccf0 0x56164597c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c6b7fe640 1 --2- 172.21.15.2:0/2756465989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616459631e0 0x561645972ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c6affd640 1 --2- 172.21.15.2:0/2756465989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x56164596e0e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3c600cb280 tx=0x7f3c600cd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ee3bed2b87cec65 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.943+0000 7f3c517fa640 1 -- 172.21.15.2:0/2756465989 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c600cf040 con 0x561645963b90 2024-03-15T13:02:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.944+0000 7f3c517fa640 1 -- 172.21.15.2:0/2756465989 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3c60098b10 con 0x561645963b90 2024-03-15T13:02:34.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.944+0000 7f3c517fa640 1 -- 172.21.15.2:0/2756465989 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c600ba0b0 con 0x561645963b90 2024-03-15T13:02:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.944+0000 7f3c74838740 1 -- 172.21.15.2:0/2756465989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 msgr2=0x56164596e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.944+0000 7f3c74838740 1 --2- 172.21.15.2:0/2756465989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x56164596e0e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3c600cb280 tx=0x7f3c600cd040 comp rx=0 tx=0).stop 2024-03-15T13:02:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.944+0000 7f3c74838740 1 -- 172.21.15.2:0/2756465989 shutdown_connections 2024-03-15T13:02:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.944+0000 7f3c74838740 1 --2- 172.21.15.2:0/2756465989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56164596ccf0 0x56164597c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.944+0000 7f3c74838740 1 --2- 172.21.15.2:0/2756465989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x56164596e0e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.944+0000 7f3c74838740 1 --2- 172.21.15.2:0/2756465989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616459631e0 0x561645972ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.944+0000 7f3c74838740 1 -- 172.21.15.2:0/2756465989 >> 172.21.15.2:0/2756465989 conn(0x56164595d840 msgr2=0x56164596f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.944+0000 7f3c74838740 1 -- 172.21.15.2:0/2756465989 shutdown_connections 2024-03-15T13:02:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.944+0000 7f3c74838740 1 -- 172.21.15.2:0/2756465989 wait complete. 2024-03-15T13:02:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.944+0000 7f3c74838740 1 -- start start 2024-03-15T13:02:34.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c74838740 1 Processor -- start 2024-03-15T13:02:34.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c74838740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616459631e0 0x5616459d5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c74838740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x5616459d64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c74838740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56164596ccf0 0x561645970550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c74838740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561645970c20 con 0x56164596ccf0 2024-03-15T13:02:34.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c74838740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561645970d90 con 0x561645963b90 2024-03-15T13:02:34.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c74838740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561645970f50 con 0x5616459631e0 2024-03-15T13:02:34.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c74838740 10 client.0 _ll_register_callbacks cb 0x7f3c5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:34.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c6bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56164596ccf0 0x561645970550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c6bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56164596ccf0 0x561645970550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56958/0 (socket says 172.21.15.2:56958) 2024-03-15T13:02:34.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c6affd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x5616459d64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c6bfff640 1 -- 172.21.15.2:0/3014886832 learned_addr learned my addr 172.21.15.2:0/3014886832 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:34.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c6b7fe640 1 --2- 172.21.15.2:0/3014886832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616459631e0 0x5616459d5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c6bfff640 1 -- 172.21.15.2:0/3014886832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616459631e0 msgr2=0x5616459d5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:34.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c6bfff640 1 --2- 172.21.15.2:0/3014886832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616459631e0 0x5616459d5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c6bfff640 1 -- 172.21.15.2:0/3014886832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 msgr2=0x5616459d64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:34.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c6bfff640 1 --2- 172.21.15.2:0/3014886832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x5616459d64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c6bfff640 1 -- 172.21.15.2:0/3014886832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616458c9320 con 0x56164596ccf0 2024-03-15T13:02:34.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.945+0000 7f3c6affd640 1 --2- 172.21.15.2:0/3014886832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x5616459d64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:34.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c6bfff640 1 --2- 172.21.15.2:0/3014886832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56164596ccf0 0x561645970550 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400a010 tx=0x7f3c640133a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750158cc82d610e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:34.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c4affd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:34.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c4affd640 1 -- 172.21.15.2:0/3014886832 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c64009d40 con 0x56164596ccf0 2024-03-15T13:02:34.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c4affd640 1 -- 172.21.15.2:0/3014886832 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f3c640106a0 con 0x56164596ccf0 2024-03-15T13:02:34.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c74838740 1 -- 172.21.15.2:0/3014886832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616458f0900 con 0x56164596ccf0 2024-03-15T13:02:34.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c4affd640 1 -- 172.21.15.2:0/3014886832 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c6400fed0 con 0x56164596ccf0 2024-03-15T13:02:34.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c74838740 10 client.16857 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:34.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c74838740 1 -- 172.21.15.2:0/3014886832 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56164588d0b0 con 0x56164596ccf0 2024-03-15T13:02:34.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c4affd640 1 -- 172.21.15.2:0/3014886832 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3c6403b020 con 0x56164596ccf0 2024-03-15T13:02:34.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c4affd640 1 -- 172.21.15.2:0/3014886832 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7f3c64005610 con 0x56164596ccf0 2024-03-15T13:02:34.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c74838740 10 client.16857 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c74838740 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:02:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c74838740 20 client.16857 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c74838740 1 --2- 172.21.15.2:0/3014886832 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5616459ddf10 0x5616459e0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c74838740 4 client.16857 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:02:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.946+0000 7f3c74838740 1 -- 172.21.15.2:0/3014886832 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5616455ca860 con 0x5616459ddf10 2024-03-15T13:02:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.947+0000 7f3c6b7fe640 1 --2- 172.21.15.2:0/3014886832 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5616459ddf10 0x5616459e0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.947+0000 7f3c6b7fe640 1 --2- 172.21.15.2:0/3014886832 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5616459ddf10 0x5616459e0330 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e4d59bcf754a99 server_cookie=97ebaadb028fdbae in_seq=0 out_seq=0 2024-03-15T13:02:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.947+0000 7f3c4affd640 10 client.16857 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.948+0000 7f3c4affd640 1 -- 172.21.15.2:0/3014886832 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5616455ca860 con 0x5616459ddf10 2024-03-15T13:02:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.948+0000 7f3c4affd640 10 client.16857 handle_command_reply: tid=0 2024-03-15T13:02:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.948+0000 7f3c74838740 1 client.16857 shutdown 2024-03-15T13:02:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.948+0000 7f3c74838740 1 -- 172.21.15.2:0/3014886832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56164596ccf0 msgr2=0x561645970550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.948+0000 7f3c74838740 1 --2- 172.21.15.2:0/3014886832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56164596ccf0 0x561645970550 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400a010 tx=0x7f3c640133a0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.948+0000 7f3c74838740 1 -- 172.21.15.2:0/3014886832 shutdown_connections 2024-03-15T13:02:34.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.948+0000 7f3c74838740 1 --2- 172.21.15.2:0/3014886832 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5616459ddf10 0x5616459e0330 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.948+0000 7f3c4affd640 0 client.16857 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:34.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.948+0000 7f3c74838740 1 --2- 172.21.15.2:0/3014886832 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56164596ccf0 0x561645970550 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.948+0000 7f3c74838740 1 --2- 172.21.15.2:0/3014886832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561645963b90 0x5616459d64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.948+0000 7f3c74838740 1 --2- 172.21.15.2:0/3014886832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616459631e0 0x5616459d5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.948+0000 7f3c74838740 1 -- 172.21.15.2:0/3014886832 >> 172.21.15.2:0/3014886832 conn(0x56164595d840 msgr2=0x5616459880b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:34.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.948+0000 7f3c74838740 1 -- 172.21.15.2:0/3014886832 shutdown_connections 2024-03-15T13:02:34.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.948+0000 7f3c74838740 1 -- 172.21.15.2:0/3014886832 wait complete. 2024-03-15T13:02:34.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.948+0000 7f3c74838740 20 client.16857 trim_cache size 0 max 16384 2024-03-15T13:02:34.940 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:02:34.940 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:02:34.941 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "cd79f7c7-492f-4520-a29f-f1fabdf921c8", 2024-03-15T13:02:34.941 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:02:34.941 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:02:34.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.949+0000 7f3c71b81640 1 -- 172.21.15.2:0/3187592443 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3c540a1ba0 msgr2=0x7f3c540a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:34.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.949+0000 7f3c71b81640 1 --2- 172.21.15.2:0/3187592443 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3c540a1ba0 0x7f3c540a4060 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3c64023000 tx=0x7f3c64021000 comp rx=0 tx=0).stop 2024-03-15T13:02:34.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.949+0000 7f3c71b81640 1 -- 172.21.15.2:0/3187592443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c101e70 msgr2=0x7f3c6c19dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:34.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.949+0000 7f3c71b81640 1 --2- 172.21.15.2:0/3187592443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c101e70 0x7f3c6c19dc10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000e9f0 tx=0x7f3c6000eeb0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.949+0000 7f3c71b81640 1 -- 172.21.15.2:0/3187592443 shutdown_connections 2024-03-15T13:02:34.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.949+0000 7f3c71b81640 1 --2- 172.21.15.2:0/3187592443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c6c107570 0x7f3c6c197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.949+0000 7f3c71b81640 1 --2- 172.21.15.2:0/3187592443 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3c540a1ba0 0x7f3c540a4060 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.949+0000 7f3c71b81640 1 --2- 172.21.15.2:0/3187592443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c101e70 0x7f3c6c19dc10 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.949+0000 7f3c71b81640 1 --2- 172.21.15.2:0/3187592443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c1014b0 0x7f3c6c19d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:34.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.949+0000 7f3c71b81640 1 -- 172.21.15.2:0/3187592443 >> 172.21.15.2:0/3187592443 conn(0x7f3c6c0fcf20 msgr2=0x7f3c6c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:34.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.949+0000 7f3c71b81640 1 -- 172.21.15.2:0/3187592443 shutdown_connections 2024-03-15T13:02:34.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:34.949+0000 7f3c71b81640 1 -- 172.21.15.2:0/3187592443 wait complete. 2024-03-15T13:02:34.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:02:34.946 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:02:35.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 -- 172.21.15.2:0/2403091212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee28100640 msgr2=0x7fee28100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:35.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 --2- 172.21.15.2:0/2403091212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee28100640 0x7fee28100aa0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fee18009900 tx=0x7fee1802bc30 comp rx=0 tx=0).stop 2024-03-15T13:02:35.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 -- 172.21.15.2:0/2403091212 shutdown_connections 2024-03-15T13:02:35.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 --2- 172.21.15.2:0/2403091212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee28100fe0 0x7fee2810d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 --2- 172.21.15.2:0/2403091212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee28100640 0x7fee28100aa0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 --2- 172.21.15.2:0/2403091212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee28105370 0x7fee28105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 -- 172.21.15.2:0/2403091212 >> 172.21.15.2:0/2403091212 conn(0x7fee280771c0 msgr2=0x7fee280775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:35.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 -- 172.21.15.2:0/2403091212 shutdown_connections 2024-03-15T13:02:35.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 -- 172.21.15.2:0/2403091212 wait complete. 2024-03-15T13:02:35.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 Processor -- start 2024-03-15T13:02:35.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 -- start start 2024-03-15T13:02:35.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee28100640 0x7fee2819d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee28100fe0 0x7fee2819dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee28105370 0x7fee28197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee28198050 con 0x7fee28100640 2024-03-15T13:02:35.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee281981c0 con 0x7fee28105370 2024-03-15T13:02:35.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.068+0000 7fee2ef83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee28198380 con 0x7fee28100fe0 2024-03-15T13:02:35.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee2cd1f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee28100640 0x7fee2819d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee2cd1f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee28100640 0x7fee2819d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56976/0 (socket says 172.21.15.2:56976) 2024-03-15T13:02:35.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee2cd1f640 1 -- 172.21.15.2:0/4238447609 learned_addr learned my addr 172.21.15.2:0/4238447609 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:35.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee27fff640 1 --2- 172.21.15.2:0/4238447609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee28100fe0 0x7fee2819dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee2d520640 1 --2- 172.21.15.2:0/4238447609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee28105370 0x7fee28197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee2cd1f640 1 -- 172.21.15.2:0/4238447609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee28100fe0 msgr2=0x7fee2819dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:35.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee2cd1f640 1 --2- 172.21.15.2:0/4238447609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee28100fe0 0x7fee2819dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee2cd1f640 1 -- 172.21.15.2:0/4238447609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee28105370 msgr2=0x7fee28197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:35.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee2cd1f640 1 --2- 172.21.15.2:0/4238447609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee28105370 0x7fee28197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee2cd1f640 1 -- 172.21.15.2:0/4238447609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee18009560 con 0x7fee28100640 2024-03-15T13:02:35.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee2cd1f640 1 --2- 172.21.15.2:0/4238447609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee28100640 0x7fee2819d850 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fee1000e9f0 tx=0x7fee1000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:35.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee25ffb640 1 -- 172.21.15.2:0/4238447609 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10009720 con 0x7fee28100640 2024-03-15T13:02:35.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee2ef83640 1 -- 172.21.15.2:0/4238447609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee28198660 con 0x7fee28100640 2024-03-15T13:02:35.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee25ffb640 1 -- 172.21.15.2:0/4238447609 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fee1000cd80 con 0x7fee28100640 2024-03-15T13:02:35.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee25ffb640 1 -- 172.21.15.2:0/4238447609 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10010690 con 0x7fee28100640 2024-03-15T13:02:35.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.069+0000 7fee2ef83640 1 -- 172.21.15.2:0/4238447609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee281a48f0 con 0x7fee28100640 2024-03-15T13:02:35.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.072+0000 7fee25ffb640 1 -- 172.21.15.2:0/4238447609 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fee10004e90 con 0x7fee28100640 2024-03-15T13:02:35.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.072+0000 7fee31c34740 1 Processor -- start 2024-03-15T13:02:35.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.072+0000 7fee31c34740 1 -- start start 2024-03-15T13:02:35.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.072+0000 7fee25ffb640 1 --2- 172.21.15.2:0/4238447609 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fee140a1ba0 0x7fee140a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.072+0000 7fee31c34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e952b0 0x55c6a8e95690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.072+0000 7fee31c34740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e9e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.072+0000 7fee31c34740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8ea0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.072+0000 7fee31c34740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8e04280 con 0x55c6a8e9e7b0 2024-03-15T13:02:35.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.072+0000 7fee31c34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8e017b0 con 0x55c6a8e952b0 2024-03-15T13:02:35.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.072+0000 7fee31c34740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8dfcf90 con 0x55c6a8e95bd0 2024-03-15T13:02:35.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee2d520640 1 --2- 172.21.15.2:0/4238447609 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fee140a1ba0 0x7fee140a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee2d520640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8ea0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee2cd1f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e952b0 0x55c6a8e95690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee27fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e9e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee2d520640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8ea0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56980/0 (socket says 172.21.15.2:56980) 2024-03-15T13:02:35.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee2d520640 1 -- 172.21.15.2:0/1935933707 learned_addr learned my addr 172.21.15.2:0/1935933707 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:35.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee25ffb640 1 -- 172.21.15.2:0/4238447609 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fee10014070 con 0x7fee28100640 2024-03-15T13:02:35.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee2d520640 1 -- 172.21.15.2:0/1935933707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e95bd0 msgr2=0x55c6a8e9e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee2d520640 1 --2- 172.21.15.2:0/1935933707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e9e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee2d520640 1 -- 172.21.15.2:0/1935933707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e952b0 msgr2=0x55c6a8e95690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee2d520640 1 --2- 172.21.15.2:0/1935933707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e952b0 0x55c6a8e95690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee2d520640 1 -- 172.21.15.2:0/1935933707 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6a8dbb570 con 0x55c6a8e9e7b0 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee27fff640 1 --2- 172.21.15.2:0/1935933707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e9e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee2d520640 1 --2- 172.21.15.2:0/1935933707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8ea0ba0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fee20012e20 tx=0x7fee2000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee68ab5279f08408 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee2d520640 1 --2- 172.21.15.2:0/4238447609 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fee140a1ba0 0x7fee140a4060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fee2000fc30 tx=0x7fee2001d000 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee1effd640 1 -- 172.21.15.2:0/1935933707 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee20012910 con 0x55c6a8e9e7b0 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee1effd640 1 -- 172.21.15.2:0/1935933707 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fee2001cb40 con 0x55c6a8e9e7b0 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.073+0000 7fee1effd640 1 -- 172.21.15.2:0/1935933707 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2001eb10 con 0x55c6a8e9e7b0 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.074+0000 7fee31c34740 1 -- 172.21.15.2:0/1935933707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e9e7b0 msgr2=0x55c6a8ea0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.074+0000 7fee31c34740 1 --2- 172.21.15.2:0/1935933707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8ea0ba0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fee20012e20 tx=0x7fee2000f730 comp rx=0 tx=0).stop 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.074+0000 7fee31c34740 1 -- 172.21.15.2:0/1935933707 shutdown_connections 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.074+0000 7fee31c34740 1 --2- 172.21.15.2:0/1935933707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8ea0ba0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.074+0000 7fee31c34740 1 --2- 172.21.15.2:0/1935933707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e9e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.074+0000 7fee31c34740 1 --2- 172.21.15.2:0/1935933707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e952b0 0x55c6a8e95690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.074+0000 7fee31c34740 1 -- 172.21.15.2:0/1935933707 >> 172.21.15.2:0/1935933707 conn(0x55c6a8e8f990 msgr2=0x55c6a8e8fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:35.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.074+0000 7fee31c34740 1 -- 172.21.15.2:0/1935933707 shutdown_connections 2024-03-15T13:02:35.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.074+0000 7fee31c34740 1 -- 172.21.15.2:0/1935933707 wait complete. 2024-03-15T13:02:35.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.074+0000 7fee31c34740 1 -- start start 2024-03-15T13:02:35.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee31c34740 1 Processor -- start 2024-03-15T13:02:35.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee31c34740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e952b0 0x55c6a8e9e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee31c34740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e9b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee31c34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8e9bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee31c34740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8ad0430 con 0x55c6a8e95bd0 2024-03-15T13:02:35.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee31c34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8a8ce00 con 0x55c6a8e9e7b0 2024-03-15T13:02:35.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee31c34740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8d81fd0 con 0x55c6a8e952b0 2024-03-15T13:02:35.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee31c34740 10 client.0 _ll_register_callbacks cb 0x55c6a8dd7fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:35.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee27fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e9b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee27fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e9b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56986/0 (socket says 172.21.15.2:56986) 2024-03-15T13:02:35.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee27fff640 1 -- 172.21.15.2:0/293609976 learned_addr learned my addr 172.21.15.2:0/293609976 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:35.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee2cd1f640 1 --2- 172.21.15.2:0/293609976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e952b0 0x55c6a8e9e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee2d520640 1 --2- 172.21.15.2:0/293609976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8e9bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee27fff640 1 -- 172.21.15.2:0/293609976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e952b0 msgr2=0x55c6a8e9e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:35.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee27fff640 1 --2- 172.21.15.2:0/293609976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e952b0 0x55c6a8e9e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee27fff640 1 -- 172.21.15.2:0/293609976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e9e7b0 msgr2=0x55c6a8e9bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:35.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee27fff640 1 --2- 172.21.15.2:0/293609976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8e9bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.075+0000 7fee27fff640 1 -- 172.21.15.2:0/293609976 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2001a040 con 0x55c6a8e95bd0 2024-03-15T13:02:35.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee27fff640 1 --2- 172.21.15.2:0/293609976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e9b5a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fee18009900 tx=0x7fee1802ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873f2ad4eb2fb6d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:35.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee07fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:35.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee07fff640 1 -- 172.21.15.2:0/293609976 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18004040 con 0x55c6a8e95bd0 2024-03-15T13:02:35.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee07fff640 1 -- 172.21.15.2:0/293609976 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fee180041a0 con 0x55c6a8e95bd0 2024-03-15T13:02:35.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee31c34740 1 -- 172.21.15.2:0/293609976 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6a8afc7e0 con 0x55c6a8e95bd0 2024-03-15T13:02:35.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee07fff640 1 -- 172.21.15.2:0/293609976 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee18004360 con 0x55c6a8e95bd0 2024-03-15T13:02:35.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee31c34740 10 client.16869 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:35.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee31c34740 1 -- 172.21.15.2:0/293609976 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55c6a8dfb610 con 0x55c6a8e95bd0 2024-03-15T13:02:35.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee07fff640 1 -- 172.21.15.2:0/293609976 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55c6a8afc7e0 con 0x55c6a8e95bd0 2024-03-15T13:02:35.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee07fff640 1 -- 172.21.15.2:0/293609976 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fee1803c040 con 0x55c6a8e95bd0 2024-03-15T13:02:35.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee31c34740 10 client.16869 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:35.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee31c34740 10 client.16869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:02:35.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee31c34740 20 client.16869 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:35.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee31c34740 1 --2- 172.21.15.2:0/293609976 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c6a8eae9c0 0x55c6a8eb0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee31c34740 4 client.16869 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:02:35.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.076+0000 7fee31c34740 1 -- 172.21.15.2:0/293609976 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6a8b39720 con 0x55c6a8eae9c0 2024-03-15T13:02:35.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.077+0000 7fee2cd1f640 1 --2- 172.21.15.2:0/293609976 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c6a8eae9c0 0x55c6a8eb0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.077+0000 7fee07fff640 10 client.16869 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:35.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.077+0000 7fee2cd1f640 1 --2- 172.21.15.2:0/293609976 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c6a8eae9c0 0x55c6a8eb0de0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3995fb20e4dd38e8 server_cookie=7c1dbb791789252e in_seq=0 out_seq=0 2024-03-15T13:02:35.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.078+0000 7fee07fff640 1 -- 172.21.15.2:0/293609976 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c6a8b39720 con 0x55c6a8eae9c0 2024-03-15T13:02:35.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.078+0000 7fee07fff640 10 client.16869 handle_command_reply: tid=0 2024-03-15T13:02:35.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.078+0000 7fee31c34740 1 client.16869 shutdown 2024-03-15T13:02:35.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.078+0000 7fee31c34740 1 -- 172.21.15.2:0/293609976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e95bd0 msgr2=0x55c6a8e9b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:35.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.078+0000 7fee31c34740 1 --2- 172.21.15.2:0/293609976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e9b5a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fee18009900 tx=0x7fee1802ca40 comp rx=0 tx=0).stop 2024-03-15T13:02:35.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.078+0000 7fee31c34740 1 -- 172.21.15.2:0/293609976 shutdown_connections 2024-03-15T13:02:35.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.078+0000 7fee31c34740 1 --2- 172.21.15.2:0/293609976 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c6a8eae9c0 0x55c6a8eb0de0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.078+0000 7fee31c34740 1 --2- 172.21.15.2:0/293609976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8e9bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.078+0000 7fee31c34740 1 --2- 172.21.15.2:0/293609976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e9b5a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.078+0000 7fee07fff640 0 client.16869 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:35.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.078+0000 7fee31c34740 1 --2- 172.21.15.2:0/293609976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e952b0 0x55c6a8e9e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.078+0000 7fee31c34740 1 -- 172.21.15.2:0/293609976 >> 172.21.15.2:0/293609976 conn(0x55c6a8e8f990 msgr2=0x55c6a8e91730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:35.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.078+0000 7fee31c34740 1 -- 172.21.15.2:0/293609976 shutdown_connections 2024-03-15T13:02:35.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.078+0000 7fee31c34740 1 -- 172.21.15.2:0/293609976 wait complete. 2024-03-15T13:02:35.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.078+0000 7fee31c34740 20 client.16869 trim_cache size 0 max 16384 2024-03-15T13:02:35.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.094+0000 7fee31c34740 1 Processor -- start 2024-03-15T13:02:35.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.094+0000 7fee31c34740 1 -- start start 2024-03-15T13:02:35.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.094+0000 7fee31c34740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e952b0 0x55c6a8e9a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.094+0000 7fee31c34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e91520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.094+0000 7fee31c34740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8e9dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.094+0000 7fee31c34740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8e8b0f0 con 0x55c6a8e952b0 2024-03-15T13:02:35.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.094+0000 7fee31c34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8e9c590 con 0x55c6a8e95bd0 2024-03-15T13:02:35.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.094+0000 7fee31c34740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8e909e0 con 0x55c6a8e9e7b0 2024-03-15T13:02:35.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.094+0000 7fee2cd1f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e952b0 0x55c6a8e9a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.095+0000 7fee2cd1f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e952b0 0x55c6a8e9a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57002/0 (socket says 172.21.15.2:57002) 2024-03-15T13:02:35.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.095+0000 7fee2cd1f640 1 -- 172.21.15.2:0/2418790558 learned_addr learned my addr 172.21.15.2:0/2418790558 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:35.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.095+0000 7fee27fff640 1 --2- 172.21.15.2:0/2418790558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e91520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.095+0000 7fee2d520640 1 --2- 172.21.15.2:0/2418790558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8e9dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.095+0000 7fee2cd1f640 1 -- 172.21.15.2:0/2418790558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e9e7b0 msgr2=0x55c6a8e9dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:35.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.095+0000 7fee2cd1f640 1 --2- 172.21.15.2:0/2418790558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8e9dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.095+0000 7fee2cd1f640 1 -- 172.21.15.2:0/2418790558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e95bd0 msgr2=0x55c6a8e91520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:35.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.095+0000 7fee2cd1f640 1 --2- 172.21.15.2:0/2418790558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e91520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.095+0000 7fee2cd1f640 1 -- 172.21.15.2:0/2418790558 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6a8d97ac0 con 0x55c6a8e952b0 2024-03-15T13:02:35.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.095+0000 7fee27fff640 1 --2- 172.21.15.2:0/2418790558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e91520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:02:35.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.095+0000 7fee2d520640 1 --2- 172.21.15.2:0/2418790558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8e9dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:02:35.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.095+0000 7fee2cd1f640 1 --2- 172.21.15.2:0/2418790558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e952b0 0x55c6a8e9a700 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fee10096b70 tx=0x7fee1001b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6f0aee0ab46565 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:35.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.095+0000 7fee1e7fc640 1 -- 172.21.15.2:0/2418790558 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1009ae10 con 0x55c6a8e952b0 2024-03-15T13:02:35.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.095+0000 7fee1e7fc640 1 -- 172.21.15.2:0/2418790558 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fee10083e90 con 0x55c6a8e952b0 2024-03-15T13:02:35.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.095+0000 7fee1e7fc640 1 -- 172.21.15.2:0/2418790558 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10098880 con 0x55c6a8e952b0 2024-03-15T13:02:35.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 -- 172.21.15.2:0/2418790558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e952b0 msgr2=0x55c6a8e9a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:35.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 --2- 172.21.15.2:0/2418790558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e952b0 0x55c6a8e9a700 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fee10096b70 tx=0x7fee1001b4f0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 -- 172.21.15.2:0/2418790558 shutdown_connections 2024-03-15T13:02:35.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 --2- 172.21.15.2:0/2418790558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8e9dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 --2- 172.21.15.2:0/2418790558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e95bd0 0x55c6a8e91520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 --2- 172.21.15.2:0/2418790558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e952b0 0x55c6a8e9a700 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 -- 172.21.15.2:0/2418790558 >> 172.21.15.2:0/2418790558 conn(0x55c6a8e8f990 msgr2=0x55c6a8e90e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:35.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 -- 172.21.15.2:0/2418790558 shutdown_connections 2024-03-15T13:02:35.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 -- 172.21.15.2:0/2418790558 wait complete. 2024-03-15T13:02:35.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 -- start start 2024-03-15T13:02:35.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 Processor -- start 2024-03-15T13:02:35.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e952b0 0x55c6a8ea6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e95bd0 0x55c6a8ea6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8e9fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8ea01a0 con 0x55c6a8e95bd0 2024-03-15T13:02:35.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8ea0310 con 0x55c6a8e952b0 2024-03-15T13:02:35.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8ea04d0 con 0x55c6a8e9e7b0 2024-03-15T13:02:35.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee31c34740 10 client.0 _ll_register_callbacks cb 0x7fee1802c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:02:35.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.096+0000 7fee27fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e95bd0 0x55c6a8ea6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee27fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e95bd0 0x55c6a8ea6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57014/0 (socket says 172.21.15.2:57014) 2024-03-15T13:02:35.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee27fff640 1 -- 172.21.15.2:0/3935543693 learned_addr learned my addr 172.21.15.2:0/3935543693 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:02:35.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee2cd1f640 1 --2- 172.21.15.2:0/3935543693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e952b0 0x55c6a8ea6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee2d520640 1 --2- 172.21.15.2:0/3935543693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8e9fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee27fff640 1 -- 172.21.15.2:0/3935543693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e9e7b0 msgr2=0x55c6a8e9fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee27fff640 1 --2- 172.21.15.2:0/3935543693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8e9fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee27fff640 1 -- 172.21.15.2:0/3935543693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e952b0 msgr2=0x55c6a8ea6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:02:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee27fff640 1 --2- 172.21.15.2:0/3935543693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e952b0 0x55c6a8ea6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee27fff640 1 -- 172.21.15.2:0/3935543693 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee281a48f0 con 0x55c6a8e95bd0 2024-03-15T13:02:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee27fff640 1 --2- 172.21.15.2:0/3935543693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e95bd0 0x55c6a8ea6550 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x55c6a8e9ae00 tx=0x7fee18004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4085e5a22b378f9c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:02:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee07fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:02:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee07fff640 1 -- 172.21.15.2:0/3935543693 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee180035f0 con 0x55c6a8e95bd0 2024-03-15T13:02:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee07fff640 1 -- 172.21.15.2:0/3935543693 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fee18003750 con 0x55c6a8e95bd0 2024-03-15T13:02:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee31c34740 1 -- 172.21.15.2:0/3935543693 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee2001cb40 con 0x55c6a8e95bd0 2024-03-15T13:02:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee07fff640 1 -- 172.21.15.2:0/3935543693 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee18005160 con 0x55c6a8e95bd0 2024-03-15T13:02:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee31c34740 10 client.16875 fetch_fsmap learned FSMap version 38 2024-03-15T13:02:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee31c34740 1 -- 172.21.15.2:0/3935543693 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55c6a8dbd6a0 con 0x55c6a8e95bd0 2024-03-15T13:02:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.097+0000 7fee07fff640 1 -- 172.21.15.2:0/3935543693 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fee2001cb40 con 0x55c6a8e95bd0 2024-03-15T13:02:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee07fff640 1 -- 172.21.15.2:0/3935543693 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7990+0+0 (secure 0 0 0) 0x7fee1803c040 con 0x55c6a8e95bd0 2024-03-15T13:02:35.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee31c34740 10 client.16875 fetch_fsmap finished waiting for FSMap version 38 2024-03-15T13:02:35.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee31c34740 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:02:35.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee31c34740 20 client.16875 populate_metadata read hostname 'smithi002' 2024-03-15T13:02:35.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee31c34740 1 --2- 172.21.15.2:0/3935543693 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c6a8f0b530 0x55c6a8f0d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:02:35.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee31c34740 4 client.16875 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:02:35.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee31c34740 1 -- 172.21.15.2:0/3935543693 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c6a8b88890 con 0x55c6a8f0b530 2024-03-15T13:02:35.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee2cd1f640 1 --2- 172.21.15.2:0/3935543693 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c6a8f0b530 0x55c6a8f0d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:02:35.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee07fff640 10 client.16875 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:35.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee2cd1f640 1 --2- 172.21.15.2:0/3935543693 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c6a8f0b530 0x55c6a8f0d950 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc099430e19d4af2 server_cookie=9297a2d17828a6f8 in_seq=0 out_seq=0 2024-03-15T13:02:35.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee07fff640 1 -- 172.21.15.2:0/3935543693 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c6a8b88890 con 0x55c6a8f0b530 2024-03-15T13:02:35.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee07fff640 10 client.16875 handle_command_reply: tid=0 2024-03-15T13:02:35.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee31c34740 1 client.16875 shutdown 2024-03-15T13:02:35.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee31c34740 1 -- 172.21.15.2:0/3935543693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e95bd0 msgr2=0x55c6a8ea6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:35.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee31c34740 1 --2- 172.21.15.2:0/3935543693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e95bd0 0x55c6a8ea6550 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x55c6a8e9ae00 tx=0x7fee18004490 comp rx=0 tx=0).stop 2024-03-15T13:02:35.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee31c34740 1 -- 172.21.15.2:0/3935543693 shutdown_connections 2024-03-15T13:02:35.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee31c34740 1 --2- 172.21.15.2:0/3935543693 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c6a8f0b530 0x55c6a8f0d950 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee31c34740 1 --2- 172.21.15.2:0/3935543693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6a8e9e7b0 0x55c6a8e9fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee07fff640 0 client.16875 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:02:35.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee31c34740 1 --2- 172.21.15.2:0/3935543693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6a8e95bd0 0x55c6a8ea6550 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee31c34740 1 --2- 172.21.15.2:0/3935543693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6a8e952b0 0x55c6a8ea6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.098+0000 7fee31c34740 1 -- 172.21.15.2:0/3935543693 >> 172.21.15.2:0/3935543693 conn(0x55c6a8e8f990 msgr2=0x55c6a8e916a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:35.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.099+0000 7fee31c34740 1 -- 172.21.15.2:0/3935543693 shutdown_connections 2024-03-15T13:02:35.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.099+0000 7fee31c34740 1 -- 172.21.15.2:0/3935543693 wait complete. 2024-03-15T13:02:35.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.099+0000 7fee31c34740 20 client.16875 trim_cache size 0 max 16384 2024-03-15T13:02:35.093 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:02:35.093 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T13:02:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:02:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "cd79f7c7-492f-4520-a29f-f1fabdf921c8": { 2024-03-15T13:02:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:02:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "cd79f7c7-492f-4520-a29f-f1fabdf921c8", 2024-03-15T13:02:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:02:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:02:35.094 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:02:35.094 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:02:35.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.099+0000 7fee2ef83640 1 -- 172.21.15.2:0/4238447609 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fee140a1ba0 msgr2=0x7fee140a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:35.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.099+0000 7fee2ef83640 1 --2- 172.21.15.2:0/4238447609 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fee140a1ba0 0x7fee140a4060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fee2000fc30 tx=0x7fee2001d000 comp rx=0 tx=0).stop 2024-03-15T13:02:35.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.099+0000 7fee2ef83640 1 -- 172.21.15.2:0/4238447609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee28100640 msgr2=0x7fee2819d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:02:35.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.099+0000 7fee2ef83640 1 --2- 172.21.15.2:0/4238447609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee28100640 0x7fee2819d850 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fee1000e9f0 tx=0x7fee1000edf0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.099+0000 7fee2ef83640 1 -- 172.21.15.2:0/4238447609 shutdown_connections 2024-03-15T13:02:35.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.099+0000 7fee2ef83640 1 --2- 172.21.15.2:0/4238447609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee28105370 0x7fee28197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.099+0000 7fee2ef83640 1 --2- 172.21.15.2:0/4238447609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee28100fe0 0x7fee2819dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.099+0000 7fee2ef83640 1 --2- 172.21.15.2:0/4238447609 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fee140a1ba0 0x7fee140a4060 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.099+0000 7fee2ef83640 1 --2- 172.21.15.2:0/4238447609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee28100640 0x7fee2819d850 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:02:35.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.099+0000 7fee2ef83640 1 -- 172.21.15.2:0/4238447609 >> 172.21.15.2:0/4238447609 conn(0x7fee280771c0 msgr2=0x7fee28104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:02:35.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.099+0000 7fee2ef83640 1 -- 172.21.15.2:0/4238447609 shutdown_connections 2024-03-15T13:02:35.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:02:35.099+0000 7fee2ef83640 1 -- 172.21.15.2:0/4238447609 wait complete. 2024-03-15T13:02:35.096 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:35 smithi002 ceph-mon[32060]: pgmap v113: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 830 KiB/s wr, 204 op/s 2024-03-15T13:02:35.097 INFO:tasks.cephfs.filesystem:scrub status for tag:cd79f7c7-492f-4520-a29f-f1fabdf921c8 - {'path': '/', 'tag': 'cd79f7c7-492f-4520-a29f-f1fabdf921c8', 'options': 'recursive,force'} 2024-03-15T13:02:35.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:35 smithi184 ceph-mon[35993]: pgmap v113: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 830 KiB/s wr, 204 op/s 2024-03-15T13:02:35.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:35 smithi149 ceph-mon[35927]: pgmap v113: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 830 KiB/s wr, 204 op/s 2024-03-15T13:02:36.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:36 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:02:36.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:36 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:36.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:36 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:02:36.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:36 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:36.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:36 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:02:36.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:36 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:36.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:36 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:02:36.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:36 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:36.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:36 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:02:36.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:36 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:36.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:36 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:02:36.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:36 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:02:37.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:37 smithi002 ceph-mon[32060]: pgmap v114: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 845 KiB/s wr, 212 op/s 2024-03-15T13:02:37.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:37 smithi184 ceph-mon[35993]: pgmap v114: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 845 KiB/s wr, 212 op/s 2024-03-15T13:02:37.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:37 smithi149 ceph-mon[35927]: pgmap v114: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 845 KiB/s wr, 212 op/s 2024-03-15T13:02:39.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:39 smithi002 ceph-mon[32060]: pgmap v115: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 361 KiB/s wr, 110 op/s 2024-03-15T13:02:39.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:39 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:02:39.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:39 smithi184 ceph-mon[35993]: pgmap v115: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 361 KiB/s wr, 110 op/s 2024-03-15T13:02:39.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:39 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:02:39.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:39 smithi149 ceph-mon[35927]: pgmap v115: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 361 KiB/s wr, 110 op/s 2024-03-15T13:02:39.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:39 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:02:41.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:41 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:02:41.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:41 smithi002 ceph-mon[32060]: pgmap v116: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 362 KiB/s wr, 115 op/s 2024-03-15T13:02:41.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:41 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:02:41.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:41 smithi184 ceph-mon[35993]: pgmap v116: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 362 KiB/s wr, 115 op/s 2024-03-15T13:02:41.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:41 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:02:41.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:41 smithi149 ceph-mon[35927]: pgmap v116: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 362 KiB/s wr, 115 op/s 2024-03-15T13:02:43.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:43 smithi002 ceph-mon[32060]: pgmap v117: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-03-15T13:02:43.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:43 smithi184 ceph-mon[35993]: pgmap v117: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-03-15T13:02:43.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:43 smithi149 ceph-mon[35927]: pgmap v117: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-03-15T13:02:45.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:45 smithi002 ceph-mon[32060]: pgmap v118: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-03-15T13:02:45.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:45 smithi184 ceph-mon[35993]: pgmap v118: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-03-15T13:02:45.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:45 smithi149 ceph-mon[35927]: pgmap v118: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-03-15T13:02:47.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:47 smithi002 ceph-mon[32060]: pgmap v119: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-03-15T13:02:47.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:47 smithi184 ceph-mon[35993]: pgmap v119: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-03-15T13:02:47.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:47 smithi149 ceph-mon[35927]: pgmap v119: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-03-15T13:02:49.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:49 smithi002 ceph-mon[32060]: pgmap v120: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-03-15T13:02:49.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:49 smithi184 ceph-mon[35993]: pgmap v120: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-03-15T13:02:49.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:49 smithi149 ceph-mon[35927]: pgmap v120: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-03-15T13:02:51.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:51 smithi002 ceph-mon[32060]: pgmap v121: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-03-15T13:02:51.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:51 smithi184 ceph-mon[35993]: pgmap v121: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-03-15T13:02:51.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:51 smithi149 ceph-mon[35927]: pgmap v121: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-03-15T13:02:53.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:53 smithi002 ceph-mon[32060]: pgmap v122: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-15T13:02:53.310 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:53 smithi184 ceph-mon[35993]: pgmap v122: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-15T13:02:53.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:53 smithi149 ceph-mon[35927]: pgmap v122: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-15T13:02:55.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:55 smithi002 ceph-mon[32060]: pgmap v123: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-15T13:02:55.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:55 smithi184 ceph-mon[35993]: pgmap v123: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-15T13:02:55.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:55 smithi149 ceph-mon[35927]: pgmap v123: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-15T13:02:57.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:57 smithi002 ceph-mon[32060]: pgmap v124: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-15T13:02:57.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:57 smithi184 ceph-mon[35993]: pgmap v124: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-15T13:02:57.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:57 smithi149 ceph-mon[35927]: pgmap v124: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-15T13:02:59.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:02:59 smithi002 ceph-mon[32060]: pgmap v125: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-15T13:02:59.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:02:59 smithi184 ceph-mon[35993]: pgmap v125: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-15T13:02:59.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:02:59 smithi149 ceph-mon[35927]: pgmap v125: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-15T13:03:01.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:01 smithi002 ceph-mon[32060]: pgmap v126: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-15T13:03:01.309 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:01 smithi184 ceph-mon[35993]: pgmap v126: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-15T13:03:01.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:01 smithi149 ceph-mon[35927]: pgmap v126: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-15T13:03:03.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:03 smithi002 ceph-mon[32060]: pgmap v127: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:03.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:03 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:03:03.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:03 smithi184 ceph-mon[35993]: pgmap v127: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:03.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:03 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:03:03.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:03 smithi149 ceph-mon[35927]: pgmap v127: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:03.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:03 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:03:04.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:04 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:03:04.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:04 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:03:04.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:04 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:03:04.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:04 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:03:04.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:04 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:03:04.308 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:04 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:03:04.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:04 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:03:04.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:04 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:03:04.359 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:04 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:03:05.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:03:05.099 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:03:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.231+0000 7fe13fdbc640 1 -- 172.21.15.2:0/4102460906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe138100f90 msgr2=0x7fe138104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.231+0000 7fe13fdbc640 1 --2- 172.21.15.2:0/4102460906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe138100f90 0x7fe138104fd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe128009930 tx=0x7fe12802bc30 comp rx=0 tx=0).stop 2024-03-15T13:03:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.231+0000 7fe13fdbc640 1 -- 172.21.15.2:0/4102460906 shutdown_connections 2024-03-15T13:03:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.231+0000 7fe13fdbc640 1 --2- 172.21.15.2:0/4102460906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1381056b0 0x7fe13810d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.231+0000 7fe13fdbc640 1 --2- 172.21.15.2:0/4102460906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe138100f90 0x7fe138104fd0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.231+0000 7fe13fdbc640 1 --2- 172.21.15.2:0/4102460906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1381005e0 0x7fe1381009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.231+0000 7fe13fdbc640 1 -- 172.21.15.2:0/4102460906 >> 172.21.15.2:0/4102460906 conn(0x7fe1380775a0 msgr2=0x7fe1380779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.231+0000 7fe13fdbc640 1 -- 172.21.15.2:0/4102460906 shutdown_connections 2024-03-15T13:03:05.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.231+0000 7fe13fdbc640 1 -- 172.21.15.2:0/4102460906 wait complete. 2024-03-15T13:03:05.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13fdbc640 1 Processor -- start 2024-03-15T13:03:05.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13fdbc640 1 -- start start 2024-03-15T13:03:05.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13fdbc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1381005e0 0x7fe13819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13fdbc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1381056b0 0x7fe13819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13fdbc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe138196800 0x7fe138196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13fdbc640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1381972b0 con 0x7fe1381005e0 2024-03-15T13:03:05.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13fdbc640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe138197420 con 0x7fe138196800 2024-03-15T13:03:05.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13fdbc640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe138197590 con 0x7fe1381056b0 2024-03-15T13:03:05.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13db58640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1381005e0 0x7fe13819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13db58640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1381005e0 0x7fe13819c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39816/0 (socket says 172.21.15.2:39816) 2024-03-15T13:03:05.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13db58640 1 -- 172.21.15.2:0/3014727657 learned_addr learned my addr 172.21.15.2:0/3014727657 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13d357640 1 --2- 172.21.15.2:0/3014727657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1381056b0 0x7fe13819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13e359640 1 --2- 172.21.15.2:0/3014727657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe138196800 0x7fe138196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13d357640 1 -- 172.21.15.2:0/3014727657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe138196800 msgr2=0x7fe138196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13d357640 1 --2- 172.21.15.2:0/3014727657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe138196800 0x7fe138196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13d357640 1 -- 172.21.15.2:0/3014727657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1381005e0 msgr2=0x7fe13819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13d357640 1 --2- 172.21.15.2:0/3014727657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1381005e0 0x7fe13819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13d357640 1 -- 172.21.15.2:0/3014727657 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe128009560 con 0x7fe1381056b0 2024-03-15T13:03:05.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.232+0000 7fe13db58640 1 --2- 172.21.15.2:0/3014727657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1381005e0 0x7fe13819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:05.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.233+0000 7fe13d357640 1 --2- 172.21.15.2:0/3014727657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1381056b0 0x7fe13819cbd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe128036040 tx=0x7fe12802ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.233+0000 7fe12effd640 1 -- 172.21.15.2:0/3014727657 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12802cc30 con 0x7fe1381056b0 2024-03-15T13:03:05.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.233+0000 7fe13fdbc640 1 -- 172.21.15.2:0/3014727657 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1381a34e0 con 0x7fe1381056b0 2024-03-15T13:03:05.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.233+0000 7fe12effd640 1 -- 172.21.15.2:0/3014727657 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe1280054d0 con 0x7fe1381056b0 2024-03-15T13:03:05.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.233+0000 7fe12effd640 1 -- 172.21.15.2:0/3014727657 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128003ba0 con 0x7fe1381056b0 2024-03-15T13:03:05.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.233+0000 7fe13fdbc640 1 -- 172.21.15.2:0/3014727657 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1381a39c0 con 0x7fe1381056b0 2024-03-15T13:03:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.235+0000 7fe12effd640 1 -- 172.21.15.2:0/3014727657 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fe128003d00 con 0x7fe1381056b0 2024-03-15T13:03:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.235+0000 7fe142996740 1 Processor -- start 2024-03-15T13:03:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe142996740 1 -- start start 2024-03-15T13:03:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe142996740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4f4c0 0x559a6cd4f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe12effd640 1 --2- 172.21.15.2:0/3014727657 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe1240a1ba0 0x7fe1240a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe142996740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4fde0 0x559a6cd58480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe142996740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd5adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe142996740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6ccbc280 con 0x559a6cd589c0 2024-03-15T13:03:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe142996740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6ccb97b0 con 0x559a6cd4fde0 2024-03-15T13:03:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe142996740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6ccb4f90 con 0x559a6cd4f4c0 2024-03-15T13:03:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe13e359640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd5adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe13d357640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4fde0 0x559a6cd58480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe13db58640 1 --2- 172.21.15.2:0/3014727657 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe1240a1ba0 0x7fe1240a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe13e359640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd5adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39828/0 (socket says 172.21.15.2:39828) 2024-03-15T13:03:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe12effd640 1 -- 172.21.15.2:0/3014727657 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fe1280efa90 con 0x7fe1381056b0 2024-03-15T13:03:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe13e359640 1 -- 172.21.15.2:0/3524626728 learned_addr learned my addr 172.21.15.2:0/3524626728 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe13d357640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4fde0 0x559a6cd58480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47626/0 (socket says 172.21.15.2:47626) 2024-03-15T13:03:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe13e359640 1 --2- 172.21.15.2:0/3524626728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4f4c0 0x559a6cd4f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe13e359640 1 -- 172.21.15.2:0/3524626728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4f4c0 msgr2=0x559a6cd4f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe13e359640 1 --2- 172.21.15.2:0/3524626728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4f4c0 0x559a6cd4f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe13e359640 1 -- 172.21.15.2:0/3524626728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4fde0 msgr2=0x559a6cd58480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe13e359640 1 --2- 172.21.15.2:0/3524626728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4fde0 0x559a6cd58480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe13e359640 1 -- 172.21.15.2:0/3524626728 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a6cc73570 con 0x559a6cd589c0 2024-03-15T13:03:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.236+0000 7fe13d357640 1 --2- 172.21.15.2:0/3524626728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4fde0 0x559a6cd58480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.237+0000 7fe13db58640 1 --2- 172.21.15.2:0/3014727657 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe1240a1ba0 0x7fe1240a4060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe134005fd0 tx=0x7fe13400b4a0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.237+0000 7fe13e359640 1 --2- 172.21.15.2:0/3524626728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd5adb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe130005f40 tx=0x7fe130005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5369a4952befcd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.237+0000 7fe123fff640 1 -- 172.21.15.2:0/3524626728 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130004b20 con 0x559a6cd589c0 2024-03-15T13:03:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.237+0000 7fe123fff640 1 -- 172.21.15.2:0/3524626728 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe13000fd50 con 0x559a6cd589c0 2024-03-15T13:03:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.237+0000 7fe123fff640 1 -- 172.21.15.2:0/3524626728 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130005520 con 0x559a6cd589c0 2024-03-15T13:03:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.238+0000 7fe142996740 1 -- 172.21.15.2:0/3524626728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 msgr2=0x559a6cd5adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.238+0000 7fe142996740 1 --2- 172.21.15.2:0/3524626728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd5adb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe130005f40 tx=0x7fe130005bd0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.238+0000 7fe142996740 1 -- 172.21.15.2:0/3524626728 shutdown_connections 2024-03-15T13:03:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.238+0000 7fe142996740 1 --2- 172.21.15.2:0/3524626728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd5adb0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.238+0000 7fe142996740 1 --2- 172.21.15.2:0/3524626728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4fde0 0x559a6cd58480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.238+0000 7fe142996740 1 --2- 172.21.15.2:0/3524626728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4f4c0 0x559a6cd4f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.238+0000 7fe142996740 1 -- 172.21.15.2:0/3524626728 >> 172.21.15.2:0/3524626728 conn(0x559a6cd49ba0 msgr2=0x559a6cd49f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.238+0000 7fe142996740 1 -- 172.21.15.2:0/3524626728 shutdown_connections 2024-03-15T13:03:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.238+0000 7fe142996740 1 -- 172.21.15.2:0/3524626728 wait complete. 2024-03-15T13:03:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.238+0000 7fe142996740 1 -- start start 2024-03-15T13:03:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.238+0000 7fe142996740 1 Processor -- start 2024-03-15T13:03:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe142996740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4f4c0 0x559a6cd583f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe142996740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4fde0 0x559a6cd55780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe142996740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd55ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe142996740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6c988430 con 0x559a6cd589c0 2024-03-15T13:03:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe142996740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6c944e00 con 0x559a6cd4f4c0 2024-03-15T13:03:05.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe142996740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6cc39fd0 con 0x559a6cd4fde0 2024-03-15T13:03:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe142996740 10 client.0 _ll_register_callbacks cb 0x559a6cc8ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe13e359640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd55ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe13e359640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd55ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39836/0 (socket says 172.21.15.2:39836) 2024-03-15T13:03:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe13e359640 1 -- 172.21.15.2:0/2477218140 learned_addr learned my addr 172.21.15.2:0/2477218140 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe13db58640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4f4c0 0x559a6cd583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe13d357640 1 --2- 172.21.15.2:0/2477218140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4fde0 0x559a6cd55780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe13e359640 1 -- 172.21.15.2:0/2477218140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4fde0 msgr2=0x559a6cd55780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe13e359640 1 --2- 172.21.15.2:0/2477218140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4fde0 0x559a6cd55780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe13e359640 1 -- 172.21.15.2:0/2477218140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4f4c0 msgr2=0x559a6cd583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe13e359640 1 --2- 172.21.15.2:0/2477218140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4f4c0 0x559a6cd583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe13e359640 1 -- 172.21.15.2:0/2477218140 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe130019040 con 0x559a6cd589c0 2024-03-15T13:03:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe13e359640 1 --2- 172.21.15.2:0/2477218140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd55ec0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe1300051e0 tx=0x7fe13000fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518d3195d331719c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe1217fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe1217fa640 1 -- 172.21.15.2:0/2477218140 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130025070 con 0x559a6cd589c0 2024-03-15T13:03:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe1217fa640 1 -- 172.21.15.2:0/2477218140 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fe130004c50 con 0x559a6cd589c0 2024-03-15T13:03:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.239+0000 7fe142996740 1 -- 172.21.15.2:0/2477218140 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a6c9b47e0 con 0x559a6cd589c0 2024-03-15T13:03:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.240+0000 7fe1217fa640 1 -- 172.21.15.2:0/2477218140 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe130004e10 con 0x559a6cd589c0 2024-03-15T13:03:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.240+0000 7fe142996740 10 client.16887 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.240+0000 7fe142996740 1 -- 172.21.15.2:0/2477218140 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x559a6ccb3610 con 0x559a6cd589c0 2024-03-15T13:03:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.240+0000 7fe1217fa640 1 -- 172.21.15.2:0/2477218140 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x559a6c9b47e0 con 0x559a6cd589c0 2024-03-15T13:03:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.240+0000 7fe1217fa640 1 -- 172.21.15.2:0/2477218140 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7fe13001f640 con 0x559a6cd589c0 2024-03-15T13:03:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.240+0000 7fe142996740 10 client.16887 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.240+0000 7fe142996740 10 client.16887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.240+0000 7fe142996740 20 client.16887 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.240+0000 7fe142996740 1 --2- 172.21.15.2:0/2477218140 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559a6cd68b10 0x559a6cd6af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.240+0000 7fe142996740 4 client.16887 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.240+0000 7fe142996740 1 -- 172.21.15.2:0/2477218140 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a6c9f1720 con 0x559a6cd68b10 2024-03-15T13:03:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.240+0000 7fe13db58640 1 --2- 172.21.15.2:0/2477218140 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559a6cd68b10 0x559a6cd6af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.241+0000 7fe1217fa640 10 client.16887 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.241+0000 7fe13db58640 1 --2- 172.21.15.2:0/2477218140 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559a6cd68b10 0x559a6cd6af30 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3bf839af5b7215f server_cookie=98f73767b8d2a483 in_seq=0 out_seq=0 2024-03-15T13:03:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.241+0000 7fe1217fa640 1 -- 172.21.15.2:0/2477218140 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559a6c9f1720 con 0x559a6cd68b10 2024-03-15T13:03:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.241+0000 7fe1217fa640 10 client.16887 handle_command_reply: tid=0 2024-03-15T13:03:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.241+0000 7fe142996740 1 client.16887 shutdown 2024-03-15T13:03:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.241+0000 7fe142996740 1 -- 172.21.15.2:0/2477218140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 msgr2=0x559a6cd55ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.241+0000 7fe142996740 1 --2- 172.21.15.2:0/2477218140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd55ec0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe1300051e0 tx=0x7fe13000fca0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.241+0000 7fe142996740 1 -- 172.21.15.2:0/2477218140 shutdown_connections 2024-03-15T13:03:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.242+0000 7fe142996740 1 --2- 172.21.15.2:0/2477218140 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559a6cd68b10 0x559a6cd6af30 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.242+0000 7fe142996740 1 --2- 172.21.15.2:0/2477218140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd55ec0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.242+0000 7fe142996740 1 --2- 172.21.15.2:0/2477218140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4fde0 0x559a6cd55780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.242+0000 7fe142996740 1 --2- 172.21.15.2:0/2477218140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4f4c0 0x559a6cd583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.242+0000 7fe1217fa640 0 client.16887 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.242+0000 7fe142996740 1 -- 172.21.15.2:0/2477218140 >> 172.21.15.2:0/2477218140 conn(0x559a6cd49ba0 msgr2=0x559a6cd4b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.242+0000 7fe142996740 1 -- 172.21.15.2:0/2477218140 shutdown_connections 2024-03-15T13:03:05.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.242+0000 7fe142996740 1 -- 172.21.15.2:0/2477218140 wait complete. 2024-03-15T13:03:05.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.242+0000 7fe142996740 20 client.16887 trim_cache size 0 max 16384 2024-03-15T13:03:05.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.258+0000 7fe142996740 1 Processor -- start 2024-03-15T13:03:05.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.258+0000 7fe142996740 1 -- start start 2024-03-15T13:03:05.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.258+0000 7fe142996740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4f4c0 0x559a6cd46c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.258+0000 7fe142996740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4fde0 0x559a6cd63d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.258+0000 7fe142996740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd64280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.258+0000 7fe142996740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6cd6b590 con 0x559a6cd589c0 2024-03-15T13:03:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.258+0000 7fe142996740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6cd430f0 con 0x559a6cd4fde0 2024-03-15T13:03:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.258+0000 7fe142996740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6cd43320 con 0x559a6cd4f4c0 2024-03-15T13:03:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.258+0000 7fe13e359640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd64280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.258+0000 7fe13e359640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd64280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39850/0 (socket says 172.21.15.2:39850) 2024-03-15T13:03:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.258+0000 7fe13e359640 1 -- 172.21.15.2:0/2766923159 learned_addr learned my addr 172.21.15.2:0/2766923159 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.259+0000 7fe13db58640 1 --2- 172.21.15.2:0/2766923159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4f4c0 0x559a6cd46c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.259+0000 7fe13d357640 1 --2- 172.21.15.2:0/2766923159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4fde0 0x559a6cd63d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.259+0000 7fe13e359640 1 -- 172.21.15.2:0/2766923159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4f4c0 msgr2=0x559a6cd46c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.259+0000 7fe13e359640 1 --2- 172.21.15.2:0/2766923159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4f4c0 0x559a6cd46c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.259+0000 7fe13e359640 1 -- 172.21.15.2:0/2766923159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4fde0 msgr2=0x559a6cd63d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.259+0000 7fe13e359640 1 --2- 172.21.15.2:0/2766923159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4fde0 0x559a6cd63d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.259+0000 7fe13e359640 1 -- 172.21.15.2:0/2766923159 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a6cc4fac0 con 0x559a6cd589c0 2024-03-15T13:03:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.259+0000 7fe13e359640 1 --2- 172.21.15.2:0/2766923159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd64280 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe130011aa0 tx=0x7fe130011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df56f97210ee3ce server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.259+0000 7fe1237fe640 1 -- 172.21.15.2:0/2766923159 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130025070 con 0x559a6cd589c0 2024-03-15T13:03:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.259+0000 7fe1237fe640 1 -- 172.21.15.2:0/2766923159 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe130023070 con 0x559a6cd589c0 2024-03-15T13:03:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.259+0000 7fe1237fe640 1 -- 172.21.15.2:0/2766923159 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13001fd00 con 0x559a6cd589c0 2024-03-15T13:03:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 -- 172.21.15.2:0/2766923159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 msgr2=0x559a6cd64280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 --2- 172.21.15.2:0/2766923159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd64280 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe130011aa0 tx=0x7fe130011850 comp rx=0 tx=0).stop 2024-03-15T13:03:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 -- 172.21.15.2:0/2766923159 shutdown_connections 2024-03-15T13:03:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 --2- 172.21.15.2:0/2766923159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd589c0 0x559a6cd64280 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 --2- 172.21.15.2:0/2766923159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd4fde0 0x559a6cd63d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 --2- 172.21.15.2:0/2766923159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4f4c0 0x559a6cd46c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 -- 172.21.15.2:0/2766923159 >> 172.21.15.2:0/2766923159 conn(0x559a6cd49ba0 msgr2=0x559a6cd4b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 -- 172.21.15.2:0/2766923159 shutdown_connections 2024-03-15T13:03:05.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 -- 172.21.15.2:0/2766923159 wait complete. 2024-03-15T13:03:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 -- start start 2024-03-15T13:03:05.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 Processor -- start 2024-03-15T13:03:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4f4c0 0x559a6cd60080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd4fde0 0x559a6cd605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd589c0 0x559a6cd59a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6cd5a0f0 con 0x559a6cd4fde0 2024-03-15T13:03:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6cd5a260 con 0x559a6cd589c0 2024-03-15T13:03:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.260+0000 7fe142996740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6cd5a420 con 0x559a6cd4f4c0 2024-03-15T13:03:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe142996740 10 client.0 _ll_register_callbacks cb 0x7fe128005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe13d357640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd4fde0 0x559a6cd605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe13d357640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd4fde0 0x559a6cd605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39856/0 (socket says 172.21.15.2:39856) 2024-03-15T13:03:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe13d357640 1 -- 172.21.15.2:0/2099874552 learned_addr learned my addr 172.21.15.2:0/2099874552 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe13db58640 1 --2- 172.21.15.2:0/2099874552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4f4c0 0x559a6cd60080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe13e359640 1 --2- 172.21.15.2:0/2099874552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd589c0 0x559a6cd59a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe13d357640 1 -- 172.21.15.2:0/2099874552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4f4c0 msgr2=0x559a6cd60080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe13d357640 1 --2- 172.21.15.2:0/2099874552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4f4c0 0x559a6cd60080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe13d357640 1 -- 172.21.15.2:0/2099874552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd589c0 msgr2=0x559a6cd59a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe13d357640 1 --2- 172.21.15.2:0/2099874552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd589c0 0x559a6cd59a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe13d357640 1 -- 172.21.15.2:0/2099874552 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a6ccb3610 con 0x559a6cd4fde0 2024-03-15T13:03:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe13d357640 1 --2- 172.21.15.2:0/2099874552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd4fde0 0x559a6cd605c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe1280bc120 tx=0x7fe1280b6f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc91f629b342478 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe1217fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe1217fa640 1 -- 172.21.15.2:0/2099874552 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1280bcb20 con 0x559a6cd4fde0 2024-03-15T13:03:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe1217fa640 1 -- 172.21.15.2:0/2099874552 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fe128004250 con 0x559a6cd4fde0 2024-03-15T13:03:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe142996740 1 -- 172.21.15.2:0/2099874552 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe13000fd50 con 0x559a6cd4fde0 2024-03-15T13:03:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe1217fa640 1 -- 172.21.15.2:0/2099874552 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1280bc420 con 0x559a6cd4fde0 2024-03-15T13:03:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe142996740 10 client.16893 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.261+0000 7fe142996740 1 -- 172.21.15.2:0/2099874552 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x559a6cc756a0 con 0x559a6cd4fde0 2024-03-15T13:03:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.262+0000 7fe1217fa640 1 -- 172.21.15.2:0/2099874552 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe13000fd50 con 0x559a6cd4fde0 2024-03-15T13:03:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.262+0000 7fe1217fa640 1 -- 172.21.15.2:0/2099874552 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7fe12803c070 con 0x559a6cd4fde0 2024-03-15T13:03:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.262+0000 7fe142996740 10 client.16893 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.262+0000 7fe142996740 10 client.16893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.262+0000 7fe142996740 20 client.16893 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.262+0000 7fe142996740 1 --2- 172.21.15.2:0/2099874552 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559a6cdc5410 0x559a6cdc7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.262+0000 7fe142996740 4 client.16893 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:03:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.262+0000 7fe142996740 1 -- 172.21.15.2:0/2099874552 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559a6ca40890 con 0x559a6cdc5410 2024-03-15T13:03:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.262+0000 7fe13db58640 1 --2- 172.21.15.2:0/2099874552 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559a6cdc5410 0x559a6cdc7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.262+0000 7fe1217fa640 10 client.16893 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.262+0000 7fe13db58640 1 --2- 172.21.15.2:0/2099874552 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559a6cdc5410 0x559a6cdc7830 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c01d751f9b764f28 server_cookie=a4379dacedc58eff in_seq=0 out_seq=0 2024-03-15T13:03:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe1217fa640 1 -- 172.21.15.2:0/2099874552 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559a6ca40890 con 0x559a6cdc5410 2024-03-15T13:03:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe1217fa640 10 client.16893 handle_command_reply: tid=0 2024-03-15T13:03:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe142996740 1 client.16893 shutdown 2024-03-15T13:03:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe142996740 1 -- 172.21.15.2:0/2099874552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd4fde0 msgr2=0x559a6cd605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe142996740 1 --2- 172.21.15.2:0/2099874552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd4fde0 0x559a6cd605c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe1280bc120 tx=0x7fe1280b6f90 comp rx=0 tx=0).stop 2024-03-15T13:03:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe142996740 1 -- 172.21.15.2:0/2099874552 shutdown_connections 2024-03-15T13:03:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe142996740 1 --2- 172.21.15.2:0/2099874552 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x559a6cdc5410 0x559a6cdc7830 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe142996740 1 --2- 172.21.15.2:0/2099874552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a6cd589c0 0x559a6cd59a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe142996740 1 --2- 172.21.15.2:0/2099874552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6cd4fde0 0x559a6cd605c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe142996740 1 --2- 172.21.15.2:0/2099874552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a6cd4f4c0 0x559a6cd60080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe1217fa640 0 client.16893 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe142996740 1 -- 172.21.15.2:0/2099874552 >> 172.21.15.2:0/2099874552 conn(0x559a6cd49ba0 msgr2=0x559a6cd4abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe142996740 1 -- 172.21.15.2:0/2099874552 shutdown_connections 2024-03-15T13:03:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe142996740 1 -- 172.21.15.2:0/2099874552 wait complete. 2024-03-15T13:03:05.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe142996740 20 client.16893 trim_cache size 0 max 16384 2024-03-15T13:03:05.255 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:03:05.255 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:03:05.255 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:03:05.255 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:03:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe13fdbc640 1 -- 172.21.15.2:0/3014727657 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe1240a1ba0 msgr2=0x7fe1240a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe13fdbc640 1 --2- 172.21.15.2:0/3014727657 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe1240a1ba0 0x7fe1240a4060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe134005fd0 tx=0x7fe13400b4a0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe13fdbc640 1 -- 172.21.15.2:0/3014727657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1381056b0 msgr2=0x7fe13819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe13fdbc640 1 --2- 172.21.15.2:0/3014727657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1381056b0 0x7fe13819cbd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe128036040 tx=0x7fe12802ca40 comp rx=0 tx=0).stop 2024-03-15T13:03:05.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe13fdbc640 1 -- 172.21.15.2:0/3014727657 shutdown_connections 2024-03-15T13:03:05.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe13fdbc640 1 --2- 172.21.15.2:0/3014727657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe138196800 0x7fe138196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe13fdbc640 1 --2- 172.21.15.2:0/3014727657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1381056b0 0x7fe13819cbd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe13fdbc640 1 --2- 172.21.15.2:0/3014727657 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe1240a1ba0 0x7fe1240a4060 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe13fdbc640 1 --2- 172.21.15.2:0/3014727657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1381005e0 0x7fe13819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe13fdbc640 1 -- 172.21.15.2:0/3014727657 >> 172.21.15.2:0/3014727657 conn(0x7fe1380775a0 msgr2=0x7fe1380fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.263+0000 7fe13fdbc640 1 -- 172.21.15.2:0/3014727657 shutdown_connections 2024-03-15T13:03:05.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.264+0000 7fe13fdbc640 1 -- 172.21.15.2:0/3014727657 wait complete. 2024-03-15T13:03:05.260 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:03:05.260 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:03:05.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.378+0000 7ffb11379640 1 -- 172.21.15.2:0/506271230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb0c100620 msgr2=0x7ffb0c100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.378+0000 7ffb11379640 1 --2- 172.21.15.2:0/506271230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb0c100620 0x7ffb0c100a00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009930 tx=0x7ffb0402bc30 comp rx=0 tx=0).stop 2024-03-15T13:03:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.378+0000 7ffb11379640 1 -- 172.21.15.2:0/506271230 shutdown_connections 2024-03-15T13:03:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.378+0000 7ffb11379640 1 --2- 172.21.15.2:0/506271230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb0c1056f0 0x7ffb0c10d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.378+0000 7ffb11379640 1 --2- 172.21.15.2:0/506271230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0c100fd0 0x7ffb0c105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.378+0000 7ffb11379640 1 --2- 172.21.15.2:0/506271230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb0c100620 0x7ffb0c100a00 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.378+0000 7ffb11379640 1 -- 172.21.15.2:0/506271230 >> 172.21.15.2:0/506271230 conn(0x7ffb0c077490 msgr2=0x7ffb0c0778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.378+0000 7ffb11379640 1 -- 172.21.15.2:0/506271230 shutdown_connections 2024-03-15T13:03:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.378+0000 7ffb11379640 1 -- 172.21.15.2:0/506271230 wait complete. 2024-03-15T13:03:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.378+0000 7ffb11379640 1 Processor -- start 2024-03-15T13:03:05.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.378+0000 7ffb11379640 1 -- start start 2024-03-15T13:03:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb11379640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0c100620 0x7ffb0c1020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb11379640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb0c1056f0 0x7ffb0c1025f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb11379640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb0c102b30 0x7ffb0c1a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb11379640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c1a6300 con 0x7ffb0c1056f0 2024-03-15T13:03:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb11379640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c1a6470 con 0x7ffb0c100620 2024-03-15T13:03:05.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb11379640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c1a6630 con 0x7ffb0c102b30 2024-03-15T13:03:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb0a7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb0c1056f0 0x7ffb0c1025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb0affd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0c100620 0x7ffb0c1020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb0a7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb0c1056f0 0x7ffb0c1025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39880/0 (socket says 172.21.15.2:39880) 2024-03-15T13:03:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb0a7fc640 1 -- 172.21.15.2:0/1278632835 learned_addr learned my addr 172.21.15.2:0/1278632835 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb0b7fe640 1 --2- 172.21.15.2:0/1278632835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb0c102b30 0x7ffb0c1a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb0a7fc640 1 -- 172.21.15.2:0/1278632835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb0c102b30 msgr2=0x7ffb0c1a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb0a7fc640 1 --2- 172.21.15.2:0/1278632835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb0c102b30 0x7ffb0c1a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb0a7fc640 1 -- 172.21.15.2:0/1278632835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0c100620 msgr2=0x7ffb0c1020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb0a7fc640 1 --2- 172.21.15.2:0/1278632835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0c100620 0x7ffb0c1020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb0a7fc640 1 -- 172.21.15.2:0/1278632835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb04009560 con 0x7ffb0c1056f0 2024-03-15T13:03:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb0a7fc640 1 --2- 172.21.15.2:0/1278632835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb0c1056f0 0x7ffb0c1025f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000e9c0 tx=0x7ffb0000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffaf3fff640 1 -- 172.21.15.2:0/1278632835 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00009700 con 0x7ffb0c1056f0 2024-03-15T13:03:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffaf3fff640 1 -- 172.21.15.2:0/1278632835 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ffb0000cd80 con 0x7ffb0c1056f0 2024-03-15T13:03:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb11379640 1 -- 172.21.15.2:0/1278632835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c1a6910 con 0x7ffb0c1056f0 2024-03-15T13:03:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffaf3fff640 1 -- 172.21.15.2:0/1278632835 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00010430 con 0x7ffb0c1056f0 2024-03-15T13:03:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.379+0000 7ffb11379640 1 -- 172.21.15.2:0/1278632835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c1a6e20 con 0x7ffb0c1056f0 2024-03-15T13:03:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.380+0000 7ffb11379640 1 -- 172.21.15.2:0/1278632835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c068730 con 0x7ffb0c1056f0 2024-03-15T13:03:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.381+0000 7ffaf3fff640 1 -- 172.21.15.2:0/1278632835 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7ffb00004fd0 con 0x7ffb0c1056f0 2024-03-15T13:03:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.382+0000 7ffaf3fff640 1 --2- 172.21.15.2:0/1278632835 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ffaf40a1ba0 0x7ffaf40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.382+0000 7ffb0affd640 1 --2- 172.21.15.2:0/1278632835 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ffaf40a1ba0 0x7ffaf40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.382+0000 7ffaf3fff640 1 -- 172.21.15.2:0/1278632835 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7ffb00014070 con 0x7ffb0c1056f0 2024-03-15T13:03:05.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.383+0000 7ffb0affd640 1 --2- 172.21.15.2:0/1278632835 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ffaf40a1ba0 0x7ffaf40a4060 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffb04000c00 tx=0x7ffb04005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.386+0000 7ffaf3fff640 1 -- 172.21.15.2:0/1278632835 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ffb00096c60 con 0x7ffb0c1056f0 2024-03-15T13:03:05.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.609+0000 7ffb11379640 1 -- 172.21.15.2:0/1278632835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb0c1099d0 con 0x7ffb0c1056f0 2024-03-15T13:03:05.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.610+0000 7ffaf3fff640 1 -- 172.21.15.2:0/1278632835 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+11470 (secure 0 0 0) 0x7ffb0009a320 con 0x7ffb0c1056f0 2024-03-15T13:03:05.595 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:03:05.596 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:02:37.136579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24539,24434,24460,24500,24464]},"id":1}]} 2024-03-15T13:03:05.596 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 39 2024-03-15T13:03:05.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.612+0000 7ffb11379640 1 -- 172.21.15.2:0/1278632835 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ffaf40a1ba0 msgr2=0x7ffaf40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.612+0000 7ffb11379640 1 --2- 172.21.15.2:0/1278632835 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ffaf40a1ba0 0x7ffaf40a4060 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffb04000c00 tx=0x7ffb04005fd0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.612+0000 7ffb11379640 1 -- 172.21.15.2:0/1278632835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb0c1056f0 msgr2=0x7ffb0c1025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.612+0000 7ffb11379640 1 --2- 172.21.15.2:0/1278632835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb0c1056f0 0x7ffb0c1025f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000e9c0 tx=0x7ffb0000ee80 comp rx=0 tx=0).stop 2024-03-15T13:03:05.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.612+0000 7ffb11379640 1 -- 172.21.15.2:0/1278632835 shutdown_connections 2024-03-15T13:03:05.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.612+0000 7ffb11379640 1 --2- 172.21.15.2:0/1278632835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb0c102b30 0x7ffb0c1a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.612+0000 7ffb11379640 1 --2- 172.21.15.2:0/1278632835 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ffaf40a1ba0 0x7ffaf40a4060 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.612+0000 7ffb11379640 1 --2- 172.21.15.2:0/1278632835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb0c1056f0 0x7ffb0c1025f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.612+0000 7ffb11379640 1 --2- 172.21.15.2:0/1278632835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb0c100620 0x7ffb0c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.612+0000 7ffb11379640 1 -- 172.21.15.2:0/1278632835 >> 172.21.15.2:0/1278632835 conn(0x7ffb0c077490 msgr2=0x7ffb0c069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.612+0000 7ffb11379640 1 -- 172.21.15.2:0/1278632835 shutdown_connections 2024-03-15T13:03:05.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.612+0000 7ffb11379640 1 -- 172.21.15.2:0/1278632835 wait complete. 2024-03-15T13:03:05.608 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:03:05.609 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:03:05.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.728+0000 7f68aaa23640 1 -- 172.21.15.2:0/1552382435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68a4100fc0 msgr2=0x7f68a4105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.728+0000 7f68aaa23640 1 --2- 172.21.15.2:0/1552382435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68a4100fc0 0x7f68a4105000 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6898009930 tx=0x7f689802bc30 comp rx=0 tx=0).stop 2024-03-15T13:03:05.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.728+0000 7f68aaa23640 1 -- 172.21.15.2:0/1552382435 shutdown_connections 2024-03-15T13:03:05.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.728+0000 7f68aaa23640 1 --2- 172.21.15.2:0/1552382435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68a41056e0 0x7f68a410d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.728+0000 7f68aaa23640 1 --2- 172.21.15.2:0/1552382435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68a4100fc0 0x7f68a4105000 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.728+0000 7f68aaa23640 1 --2- 172.21.15.2:0/1552382435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a4100610 0x7f68a41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.728+0000 7f68aaa23640 1 -- 172.21.15.2:0/1552382435 >> 172.21.15.2:0/1552382435 conn(0x7f68a4076770 msgr2=0x7f68a4076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.728+0000 7f68aaa23640 1 -- 172.21.15.2:0/1552382435 shutdown_connections 2024-03-15T13:03:05.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.728+0000 7f68aaa23640 1 -- 172.21.15.2:0/1552382435 wait complete. 2024-03-15T13:03:05.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68aaa23640 1 Processor -- start 2024-03-15T13:03:05.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68aaa23640 1 -- start start 2024-03-15T13:03:05.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68aaa23640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68a4100610 0x7f68a419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68aaa23640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a41056e0 0x7f68a419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68aaa23640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68a4197960 0x7f68a4197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68aaa23640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a4198350 con 0x7f68a4100610 2024-03-15T13:03:05.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68aaa23640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a41984c0 con 0x7f68a4197960 2024-03-15T13:03:05.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68aaa23640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a4198680 con 0x7f68a41056e0 2024-03-15T13:03:05.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68a4100610 0x7f68a419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68a4100610 0x7f68a419d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39900/0 (socket says 172.21.15.2:39900) 2024-03-15T13:03:05.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68a3fff640 1 -- 172.21.15.2:0/2984166294 learned_addr learned my addr 172.21.15.2:0/2984166294 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68a8fc0640 1 --2- 172.21.15.2:0/2984166294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68a4197960 0x7f68a4197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68a37fe640 1 --2- 172.21.15.2:0/2984166294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a41056e0 0x7f68a419dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68a3fff640 1 -- 172.21.15.2:0/2984166294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a41056e0 msgr2=0x7f68a419dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68a3fff640 1 --2- 172.21.15.2:0/2984166294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a41056e0 0x7f68a419dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68a3fff640 1 -- 172.21.15.2:0/2984166294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68a4197960 msgr2=0x7f68a4197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68a3fff640 1 --2- 172.21.15.2:0/2984166294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68a4197960 0x7f68a4197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.729+0000 7f68a3fff640 1 -- 172.21.15.2:0/2984166294 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6898009560 con 0x7f68a4100610 2024-03-15T13:03:05.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.730+0000 7f68a3fff640 1 --2- 172.21.15.2:0/2984166294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68a4100610 0x7f68a419d7b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f689400e9f0 tx=0x7f689400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.730+0000 7f68a17fa640 1 -- 172.21.15.2:0/2984166294 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894009700 con 0x7f68a4100610 2024-03-15T13:03:05.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.730+0000 7f68a17fa640 1 -- 172.21.15.2:0/2984166294 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f689400cd80 con 0x7f68a4100610 2024-03-15T13:03:05.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.730+0000 7f68aaa23640 1 -- 172.21.15.2:0/2984166294 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a41a4670 con 0x7f68a4100610 2024-03-15T13:03:05.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.730+0000 7f68a17fa640 1 -- 172.21.15.2:0/2984166294 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894018920 con 0x7f68a4100610 2024-03-15T13:03:05.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.730+0000 7f68aaa23640 1 -- 172.21.15.2:0/2984166294 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a41a4b50 con 0x7f68a4100610 2024-03-15T13:03:05.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.732+0000 7f68a17fa640 1 -- 172.21.15.2:0/2984166294 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f6894018a80 con 0x7f68a4100610 2024-03-15T13:03:05.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.732+0000 7f68ad602740 1 Processor -- start 2024-03-15T13:03:05.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.732+0000 7f68a17fa640 1 --2- 172.21.15.2:0/2984166294 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f688c0a1ba0 0x7f688c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.732+0000 7f68ad602740 1 -- start start 2024-03-15T13:03:05.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68a8fc0640 1 --2- 172.21.15.2:0/2984166294 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f688c0a1ba0 0x7f688c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68a17fa640 1 -- 172.21.15.2:0/2984166294 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f6894014070 con 0x7f68a4100610 2024-03-15T13:03:05.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68ad602740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5250 0x5569421d5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68ad602740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421d5b70 0x5569421de210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68ad602740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421de750 0x5569421e0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68ad602740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556942190960 con 0x5569421de750 2024-03-15T13:03:05.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68ad602740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556942144310 con 0x5569421d5250 2024-03-15T13:03:05.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68ad602740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556942141840 con 0x5569421d5b70 2024-03-15T13:03:05.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68a8fc0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421de750 0x5569421e0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5250 0x5569421d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68a8fc0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421de750 0x5569421e0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39904/0 (socket says 172.21.15.2:39904) 2024-03-15T13:03:05.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68a8fc0640 1 -- 172.21.15.2:0/3488637566 learned_addr learned my addr 172.21.15.2:0/3488637566 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68a37fe640 1 --2- 172.21.15.2:0/3488637566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421d5b70 0x5569421de210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68a3fff640 1 -- 172.21.15.2:0/3488637566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421d5b70 msgr2=0x5569421de210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68a3fff640 1 --2- 172.21.15.2:0/3488637566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421d5b70 0x5569421de210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68a3fff640 1 -- 172.21.15.2:0/3488637566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421de750 msgr2=0x5569421e0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68a3fff640 1 --2- 172.21.15.2:0/3488637566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421de750 0x5569421e0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68a3fff640 1 -- 172.21.15.2:0/3488637566 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569420ff060 con 0x5569421d5250 2024-03-15T13:03:05.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68a8fc0640 1 --2- 172.21.15.2:0/3488637566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421de750 0x5569421e0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:05.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68a8fc0640 1 --2- 172.21.15.2:0/2984166294 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f688c0a1ba0 0x7f688c0a4060 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f689c009fd0 tx=0x7f689c013770 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f68a3fff640 1 --2- 172.21.15.2:0/3488637566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5250 0x5569421d5630 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f68940cb270 tx=0x7f689409a220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=161a814ef3d52629 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f688a7fc640 1 -- 172.21.15.2:0/3488637566 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68940a2ef0 con 0x5569421d5250 2024-03-15T13:03:05.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.733+0000 7f688a7fc640 1 -- 172.21.15.2:0/3488637566 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6894010be0 con 0x5569421d5250 2024-03-15T13:03:05.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.734+0000 7f688a7fc640 1 -- 172.21.15.2:0/3488637566 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68940ba2d0 con 0x5569421d5250 2024-03-15T13:03:05.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.734+0000 7f68ad602740 1 -- 172.21.15.2:0/3488637566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5250 msgr2=0x5569421d5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.734+0000 7f68ad602740 1 --2- 172.21.15.2:0/3488637566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5250 0x5569421d5630 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f68940cb270 tx=0x7f689409a220 comp rx=0 tx=0).stop 2024-03-15T13:03:05.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.734+0000 7f68ad602740 1 -- 172.21.15.2:0/3488637566 shutdown_connections 2024-03-15T13:03:05.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.734+0000 7f68ad602740 1 --2- 172.21.15.2:0/3488637566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421de750 0x5569421e0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.734+0000 7f68ad602740 1 --2- 172.21.15.2:0/3488637566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421d5b70 0x5569421de210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.734+0000 7f68ad602740 1 --2- 172.21.15.2:0/3488637566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5250 0x5569421d5630 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.734+0000 7f68ad602740 1 -- 172.21.15.2:0/3488637566 >> 172.21.15.2:0/3488637566 conn(0x5569421cf980 msgr2=0x5569421cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.734+0000 7f68ad602740 1 -- 172.21.15.2:0/3488637566 shutdown_connections 2024-03-15T13:03:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.734+0000 7f68ad602740 1 -- 172.21.15.2:0/3488637566 wait complete. 2024-03-15T13:03:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.734+0000 7f68ad602740 1 -- start start 2024-03-15T13:03:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68ad602740 1 Processor -- start 2024-03-15T13:03:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68ad602740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421d5250 0x5569421de110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68ad602740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421d5b70 0x5569421db4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68ad602740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421de750 0x5569421dbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68ad602740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55694213d020 con 0x5569421d5250 2024-03-15T13:03:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68ad602740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556941e10430 con 0x5569421de750 2024-03-15T13:03:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68ad602740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556941dcce00 con 0x5569421d5b70 2024-03-15T13:03:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68ad602740 10 client.0 _ll_register_callbacks cb 0x5569420e4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421d5250 0x5569421de110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421d5250 0x5569421de110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39906/0 (socket says 172.21.15.2:39906) 2024-03-15T13:03:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68a3fff640 1 -- 172.21.15.2:0/2520138092 learned_addr learned my addr 172.21.15.2:0/2520138092 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68a8fc0640 1 --2- 172.21.15.2:0/2520138092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421de750 0x5569421dbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68a37fe640 1 --2- 172.21.15.2:0/2520138092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421d5b70 0x5569421db4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68a3fff640 1 -- 172.21.15.2:0/2520138092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421d5b70 msgr2=0x5569421db4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68a3fff640 1 --2- 172.21.15.2:0/2520138092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421d5b70 0x5569421db4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68a3fff640 1 -- 172.21.15.2:0/2520138092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421de750 msgr2=0x5569421dbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68a3fff640 1 --2- 172.21.15.2:0/2520138092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421de750 0x5569421dbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68a3fff640 1 -- 172.21.15.2:0/2520138092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a41a4b50 con 0x5569421d5250 2024-03-15T13:03:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68a3fff640 1 --2- 172.21.15.2:0/2520138092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421d5250 0x5569421de110 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6894000c00 tx=0x7f689409a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5586b468a742fe08 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.735+0000 7f68837fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.736+0000 7f68837fe640 1 -- 172.21.15.2:0/2520138092 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894097050 con 0x5569421d5250 2024-03-15T13:03:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.736+0000 7f68837fe640 1 -- 172.21.15.2:0/2520138092 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f68940a2ef0 con 0x5569421d5250 2024-03-15T13:03:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.736+0000 7f68ad602740 1 -- 172.21.15.2:0/2520138092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556941e3c7e0 con 0x5569421d5250 2024-03-15T13:03:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.736+0000 7f68837fe640 1 -- 172.21.15.2:0/2520138092 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6894096e70 con 0x5569421d5250 2024-03-15T13:03:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.736+0000 7f68ad602740 10 client.16911 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.736+0000 7f68ad602740 1 -- 172.21.15.2:0/2520138092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55694213b6a0 con 0x5569421d5250 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.736+0000 7f68837fe640 1 -- 172.21.15.2:0/2520138092 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f68940a4020 con 0x5569421d5250 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.736+0000 7f68837fe640 1 -- 172.21.15.2:0/2520138092 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7f6894085140 con 0x5569421d5250 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.736+0000 7f68ad602740 10 client.16911 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.736+0000 7f68ad602740 10 client.16911 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.736+0000 7f68ad602740 20 client.16911 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.736+0000 7f68ad602740 1 --2- 172.21.15.2:0/2520138092 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5569421ee4a0 0x5569421f0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.736+0000 7f68ad602740 4 client.16911 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.736+0000 7f68ad602740 1 -- 172.21.15.2:0/2520138092 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556941e79720 con 0x5569421ee4a0 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.736+0000 7f68a37fe640 1 --2- 172.21.15.2:0/2520138092 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5569421ee4a0 0x5569421f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.737+0000 7f68a37fe640 1 --2- 172.21.15.2:0/2520138092 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5569421ee4a0 0x5569421f0890 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eb851e80331a14b8 server_cookie=ba4f954c946a19e9 in_seq=0 out_seq=0 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.737+0000 7f68837fe640 10 client.16911 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.737+0000 7f68837fe640 1 -- 172.21.15.2:0/2520138092 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556941e79720 con 0x5569421ee4a0 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.737+0000 7f68837fe640 10 client.16911 handle_command_reply: tid=0 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.738+0000 7f68ad602740 1 client.16911 shutdown 2024-03-15T13:03:05.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.738+0000 7f68ad602740 1 -- 172.21.15.2:0/2520138092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421d5250 msgr2=0x5569421de110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.738+0000 7f68ad602740 1 --2- 172.21.15.2:0/2520138092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421d5250 0x5569421de110 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6894000c00 tx=0x7f689409a290 comp rx=0 tx=0).stop 2024-03-15T13:03:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.738+0000 7f68ad602740 1 -- 172.21.15.2:0/2520138092 shutdown_connections 2024-03-15T13:03:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.738+0000 7f68ad602740 1 --2- 172.21.15.2:0/2520138092 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5569421ee4a0 0x5569421f0890 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.738+0000 7f68ad602740 1 --2- 172.21.15.2:0/2520138092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421de750 0x5569421dbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.738+0000 7f68ad602740 1 --2- 172.21.15.2:0/2520138092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421d5b70 0x5569421db4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.738+0000 7f68ad602740 1 --2- 172.21.15.2:0/2520138092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421d5250 0x5569421de110 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.738+0000 7f68837fe640 0 client.16911 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:03:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.738+0000 7f68ad602740 1 -- 172.21.15.2:0/2520138092 >> 172.21.15.2:0/2520138092 conn(0x5569421cf980 msgr2=0x5569421d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.738+0000 7f68ad602740 1 -- 172.21.15.2:0/2520138092 shutdown_connections 2024-03-15T13:03:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.738+0000 7f68ad602740 1 -- 172.21.15.2:0/2520138092 wait complete. 2024-03-15T13:03:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.738+0000 7f68ad602740 20 client.16911 trim_cache size 0 max 16384 2024-03-15T13:03:05.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.750+0000 7f68ad602740 1 Processor -- start 2024-03-15T13:03:05.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.750+0000 7f68ad602740 1 -- start start 2024-03-15T13:03:05.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.750+0000 7f68ad602740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421d5250 0x5569421ddbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.750+0000 7f68ad602740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5b70 0x5569421de110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.750+0000 7f68ad602740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421de750 0x5569421e9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.750+0000 7f68ad602740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569420c2130 con 0x5569421de750 2024-03-15T13:03:05.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.750+0000 7f68ad602740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569421dc440 con 0x5569421d5b70 2024-03-15T13:03:05.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.750+0000 7f68ad602740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569421cb2b0 con 0x5569421d5250 2024-03-15T13:03:05.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.750+0000 7f68a8fc0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421de750 0x5569421e9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.750+0000 7f68a8fc0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421de750 0x5569421e9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39916/0 (socket says 172.21.15.2:39916) 2024-03-15T13:03:05.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.750+0000 7f68a8fc0640 1 -- 172.21.15.2:0/913932678 learned_addr learned my addr 172.21.15.2:0/913932678 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.751+0000 7f68a3fff640 1 --2- 172.21.15.2:0/913932678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421d5250 0x5569421ddbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.751+0000 7f68a37fe640 1 --2- 172.21.15.2:0/913932678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5b70 0x5569421de110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.751+0000 7f68a8fc0640 1 -- 172.21.15.2:0/913932678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421d5250 msgr2=0x5569421ddbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.751+0000 7f68a8fc0640 1 --2- 172.21.15.2:0/913932678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421d5250 0x5569421ddbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.751+0000 7f68a8fc0640 1 -- 172.21.15.2:0/913932678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5b70 msgr2=0x5569421de110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.751+0000 7f68a8fc0640 1 --2- 172.21.15.2:0/913932678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5b70 0x5569421de110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.751+0000 7f68a8fc0640 1 -- 172.21.15.2:0/913932678 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569420d7c20 con 0x5569421de750 2024-03-15T13:03:05.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.751+0000 7f68a37fe640 1 --2- 172.21.15.2:0/913932678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5b70 0x5569421de110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:03:05.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.751+0000 7f68a8fc0640 1 --2- 172.21.15.2:0/913932678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421de750 0x5569421e9ad0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f689c012380 tx=0x7f689c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4436334073e61eb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.751+0000 7f6889ffb640 1 -- 172.21.15.2:0/913932678 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c018050 con 0x5569421de750 2024-03-15T13:03:05.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.751+0000 7f6889ffb640 1 -- 172.21.15.2:0/913932678 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f689c003e00 con 0x5569421de750 2024-03-15T13:03:05.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.751+0000 7f6889ffb640 1 -- 172.21.15.2:0/913932678 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c005490 con 0x5569421de750 2024-03-15T13:03:05.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.751+0000 7f68ad602740 1 -- 172.21.15.2:0/913932678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421de750 msgr2=0x5569421e9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 --2- 172.21.15.2:0/913932678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421de750 0x5569421e9ad0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f689c012380 tx=0x7f689c009500 comp rx=0 tx=0).stop 2024-03-15T13:03:05.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 -- 172.21.15.2:0/913932678 shutdown_connections 2024-03-15T13:03:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 --2- 172.21.15.2:0/913932678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421de750 0x5569421e9ad0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 --2- 172.21.15.2:0/913932678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5b70 0x5569421de110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 --2- 172.21.15.2:0/913932678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421d5250 0x5569421ddbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 -- 172.21.15.2:0/913932678 >> 172.21.15.2:0/913932678 conn(0x5569421cf980 msgr2=0x5569421ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 -- 172.21.15.2:0/913932678 shutdown_connections 2024-03-15T13:03:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 -- 172.21.15.2:0/913932678 wait complete. 2024-03-15T13:03:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 -- start start 2024-03-15T13:03:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 Processor -- start 2024-03-15T13:03:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421d5250 0x5569421d7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5b70 0x5569421d83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421de750 0x556942248d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569421d8b30 con 0x5569421d5250 2024-03-15T13:03:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569421d8c70 con 0x5569421d5b70 2024-03-15T13:03:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556942249260 con 0x5569421de750 2024-03-15T13:03:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68ad602740 10 client.0 _ll_register_callbacks cb 0x7f689c013640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:05.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421d5250 0x5569421d7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421d5250 0x5569421d7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39932/0 (socket says 172.21.15.2:39932) 2024-03-15T13:03:05.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68a3fff640 1 -- 172.21.15.2:0/1866494505 learned_addr learned my addr 172.21.15.2:0/1866494505 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68a37fe640 1 --2- 172.21.15.2:0/1866494505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5b70 0x5569421d83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.752+0000 7f68a8fc0640 1 --2- 172.21.15.2:0/1866494505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421de750 0x556942248d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68a3fff640 1 -- 172.21.15.2:0/1866494505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421de750 msgr2=0x556942248d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68a3fff640 1 --2- 172.21.15.2:0/1866494505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421de750 0x556942248d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68a3fff640 1 -- 172.21.15.2:0/1866494505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5b70 msgr2=0x5569421d83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68a3fff640 1 --2- 172.21.15.2:0/1866494505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5b70 0x5569421d83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68a3fff640 1 -- 172.21.15.2:0/1866494505 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689c012110 con 0x5569421d5250 2024-03-15T13:03:05.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68a3fff640 1 --2- 172.21.15.2:0/1866494505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421d5250 0x5569421d7e60 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6894096c60 tx=0x7f689409a220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239c5f4d4ff89b15 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68837fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:05.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68837fe640 1 -- 172.21.15.2:0/1866494505 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894099aa0 con 0x5569421d5250 2024-03-15T13:03:05.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68837fe640 1 -- 172.21.15.2:0/1866494505 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f68940a5030 con 0x5569421d5250 2024-03-15T13:03:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68ad602740 1 -- 172.21.15.2:0/1866494505 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6894010be0 con 0x5569421d5250 2024-03-15T13:03:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68837fe640 1 -- 172.21.15.2:0/1866494505 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6894098b00 con 0x5569421d5250 2024-03-15T13:03:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68ad602740 10 client.16917 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68ad602740 1 -- 172.21.15.2:0/1866494505 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5569420fd8c0 con 0x5569421d5250 2024-03-15T13:03:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68837fe640 1 -- 172.21.15.2:0/1866494505 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6894089080 con 0x5569421d5250 2024-03-15T13:03:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68837fe640 1 -- 172.21.15.2:0/1866494505 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7f68940ae400 con 0x5569421d5250 2024-03-15T13:03:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.753+0000 7f68ad602740 10 client.16917 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68ad602740 10 client.16917 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:03:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68ad602740 20 client.16917 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68ad602740 1 --2- 172.21.15.2:0/1866494505 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55694224a6e0 0x55694224cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68ad602740 4 client.16917 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:03:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68ad602740 1 -- 172.21.15.2:0/1866494505 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556941ec8890 con 0x55694224a6e0 2024-03-15T13:03:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68a37fe640 1 --2- 172.21.15.2:0/1866494505 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55694224a6e0 0x55694224cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68a37fe640 1 --2- 172.21.15.2:0/1866494505 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55694224a6e0 0x55694224cad0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=60b042c2a51e7f25 server_cookie=9f1a3093646ff3e1 in_seq=0 out_seq=0 2024-03-15T13:03:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68837fe640 10 client.16917 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:03:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68837fe640 1 -- 172.21.15.2:0/1866494505 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556941ec8890 con 0x55694224a6e0 2024-03-15T13:03:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68837fe640 10 client.16917 handle_command_reply: tid=0 2024-03-15T13:03:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68ad602740 1 client.16917 shutdown 2024-03-15T13:03:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68ad602740 1 -- 172.21.15.2:0/1866494505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421d5250 msgr2=0x5569421d7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68ad602740 1 --2- 172.21.15.2:0/1866494505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421d5250 0x5569421d7e60 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6894096c60 tx=0x7f689409a220 comp rx=0 tx=0).stop 2024-03-15T13:03:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68ad602740 1 -- 172.21.15.2:0/1866494505 shutdown_connections 2024-03-15T13:03:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68ad602740 1 --2- 172.21.15.2:0/1866494505 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55694224a6e0 0x55694224cad0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68ad602740 1 --2- 172.21.15.2:0/1866494505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569421de750 0x556942248d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68ad602740 1 --2- 172.21.15.2:0/1866494505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569421d5b70 0x5569421d83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68ad602740 1 --2- 172.21.15.2:0/1866494505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5569421d5250 0x5569421d7e60 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68837fe640 0 client.16917 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:03:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.754+0000 7f68ad602740 1 -- 172.21.15.2:0/1866494505 >> 172.21.15.2:0/1866494505 conn(0x5569421cf980 msgr2=0x5569421d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.755+0000 7f68ad602740 1 -- 172.21.15.2:0/1866494505 shutdown_connections 2024-03-15T13:03:05.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.755+0000 7f68ad602740 1 -- 172.21.15.2:0/1866494505 wait complete. 2024-03-15T13:03:05.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.755+0000 7f68ad602740 20 client.16917 trim_cache size 0 max 16384 2024-03-15T13:03:05.747 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:03:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.755+0000 7f68aaa23640 1 -- 172.21.15.2:0/2984166294 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f688c0a1ba0 msgr2=0x7f688c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.755+0000 7f68aaa23640 1 --2- 172.21.15.2:0/2984166294 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f688c0a1ba0 0x7f688c0a4060 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f689c009fd0 tx=0x7f689c013770 comp rx=0 tx=0).stop 2024-03-15T13:03:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.755+0000 7f68aaa23640 1 -- 172.21.15.2:0/2984166294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68a4100610 msgr2=0x7f68a419d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.755+0000 7f68aaa23640 1 --2- 172.21.15.2:0/2984166294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68a4100610 0x7f68a419d7b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f689400e9f0 tx=0x7f689400eeb0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.755+0000 7f68aaa23640 1 -- 172.21.15.2:0/2984166294 shutdown_connections 2024-03-15T13:03:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.755+0000 7f68aaa23640 1 --2- 172.21.15.2:0/2984166294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68a4197960 0x7f68a4197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.755+0000 7f68aaa23640 1 --2- 172.21.15.2:0/2984166294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a41056e0 0x7f68a419dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.755+0000 7f68aaa23640 1 --2- 172.21.15.2:0/2984166294 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f688c0a1ba0 0x7f688c0a4060 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.755+0000 7f68aaa23640 1 --2- 172.21.15.2:0/2984166294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68a4100610 0x7f68a419d7b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.755+0000 7f68aaa23640 1 -- 172.21.15.2:0/2984166294 >> 172.21.15.2:0/2984166294 conn(0x7f68a4076770 msgr2=0x7f68a4069e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.755+0000 7f68aaa23640 1 -- 172.21.15.2:0/2984166294 shutdown_connections 2024-03-15T13:03:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.755+0000 7f68aaa23640 1 -- 172.21.15.2:0/2984166294 wait complete. 2024-03-15T13:03:05.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:05 smithi002 ceph-mon[32060]: pgmap v128: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:05.751 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:03:05.751 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:03:05.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:05 smithi184 ceph-mon[35993]: pgmap v128: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:05.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.871+0000 7fb6b0919640 1 -- 172.21.15.2:0/1895319992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6ac10bb00 msgr2=0x7fb6ac10df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.871+0000 7fb6b0919640 1 --2- 172.21.15.2:0/1895319992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6ac10bb00 0x7fb6ac10df30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009ca0 tx=0x7fb6a402c180 comp rx=0 tx=0).stop 2024-03-15T13:03:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.871+0000 7fb6b0919640 1 -- 172.21.15.2:0/1895319992 shutdown_connections 2024-03-15T13:03:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.871+0000 7fb6b0919640 1 --2- 172.21.15.2:0/1895319992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6ac10bb00 0x7fb6ac10df30 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.871+0000 7fb6b0919640 1 --2- 172.21.15.2:0/1895319992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6ac103050 0x7fb6ac10b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.871+0000 7fb6b0919640 1 --2- 172.21.15.2:0/1895319992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6ac102730 0x7fb6ac102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.871+0000 7fb6b0919640 1 -- 172.21.15.2:0/1895319992 >> 172.21.15.2:0/1895319992 conn(0x7fb6ac076770 msgr2=0x7fb6ac076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.871+0000 7fb6b0919640 1 -- 172.21.15.2:0/1895319992 shutdown_connections 2024-03-15T13:03:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.871+0000 7fb6b0919640 1 -- 172.21.15.2:0/1895319992 wait complete. 2024-03-15T13:03:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.871+0000 7fb6b0919640 1 Processor -- start 2024-03-15T13:03:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.871+0000 7fb6b0919640 1 -- start start 2024-03-15T13:03:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6b0919640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6ac102730 0x7fb6ac19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6b0919640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6ac103050 0x7fb6ac19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6b0919640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6ac10bb00 0x7fb6ac197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6b0919640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac197f90 con 0x7fb6ac102730 2024-03-15T13:03:05.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6b0919640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac198100 con 0x7fb6ac10bb00 2024-03-15T13:03:05.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6b0919640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac1982c0 con 0x7fb6ac103050 2024-03-15T13:03:05.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6aa59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6ac102730 0x7fb6ac19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6aa59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6ac102730 0x7fb6ac19d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39950/0 (socket says 172.21.15.2:39950) 2024-03-15T13:03:05.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6aa59c640 1 -- 172.21.15.2:0/1421193046 learned_addr learned my addr 172.21.15.2:0/1421193046 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6a9d9b640 1 --2- 172.21.15.2:0/1421193046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6ac103050 0x7fb6ac19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6aad9d640 1 --2- 172.21.15.2:0/1421193046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6ac10bb00 0x7fb6ac197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6aa59c640 1 -- 172.21.15.2:0/1421193046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6ac103050 msgr2=0x7fb6ac19dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6aa59c640 1 --2- 172.21.15.2:0/1421193046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6ac103050 0x7fb6ac19dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6aa59c640 1 -- 172.21.15.2:0/1421193046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6ac10bb00 msgr2=0x7fb6ac197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6aa59c640 1 --2- 172.21.15.2:0/1421193046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6ac10bb00 0x7fb6ac197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6aa59c640 1 -- 172.21.15.2:0/1421193046 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a4009900 con 0x7fb6ac102730 2024-03-15T13:03:05.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6aa59c640 1 --2- 172.21.15.2:0/1421193046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6ac102730 0x7fb6ac19d740 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00e9f0 tx=0x7fb69c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb69b7fe640 1 -- 172.21.15.2:0/1421193046 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c009700 con 0x7fb6ac102730 2024-03-15T13:03:05.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb69b7fe640 1 -- 172.21.15.2:0/1421193046 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb69c00cd80 con 0x7fb6ac102730 2024-03-15T13:03:05.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6b0919640 1 -- 172.21.15.2:0/1421193046 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ac1985a0 con 0x7fb6ac102730 2024-03-15T13:03:05.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb69b7fe640 1 -- 172.21.15.2:0/1421193046 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c018920 con 0x7fb6ac102730 2024-03-15T13:03:05.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.872+0000 7fb6b0919640 1 -- 172.21.15.2:0/1421193046 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ac1a47e0 con 0x7fb6ac102730 2024-03-15T13:03:05.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:05 smithi149 ceph-mon[35927]: pgmap v128: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:05.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.875+0000 7fb69b7fe640 1 -- 172.21.15.2:0/1421193046 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fb69c018a80 con 0x7fb6ac102730 2024-03-15T13:03:05.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.875+0000 7fb6b35d0740 1 Processor -- start 2024-03-15T13:03:05.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.875+0000 7fb6b35d0740 1 -- start start 2024-03-15T13:03:05.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.875+0000 7fb69b7fe640 1 --2- 172.21.15.2:0/1421193046 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb6940a1ba0 0x7fb6940a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.875+0000 7fb6b35d0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a72876250 0x559a72876630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.875+0000 7fb6b35d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876b70 0x559a7287f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.875+0000 7fb6b35d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a7287f750 0x559a72881b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.875+0000 7fb6b35d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a72831960 con 0x559a7287f750 2024-03-15T13:03:05.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.875+0000 7fb6b35d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a727e5310 con 0x559a72876b70 2024-03-15T13:03:05.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.875+0000 7fb69b7fe640 1 -- 172.21.15.2:0/1421193046 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fb69c014070 con 0x7fb6ac102730 2024-03-15T13:03:05.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.875+0000 7fb6b35d0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a727e2840 con 0x559a72876250 2024-03-15T13:03:05.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb6aad9d640 1 --2- 172.21.15.2:0/1421193046 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb6940a1ba0 0x7fb6940a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb6a9d9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876b70 0x559a7287f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb6aad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a7287f750 0x559a72881b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb6a9d9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876b70 0x559a7287f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47758/0 (socket says 172.21.15.2:47758) 2024-03-15T13:03:05.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb6a9d9b640 1 -- 172.21.15.2:0/1589054591 learned_addr learned my addr 172.21.15.2:0/1589054591 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb6aa59c640 1 --2- 172.21.15.2:0/1589054591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a72876250 0x559a72876630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb6a9d9b640 1 -- 172.21.15.2:0/1589054591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a72876250 msgr2=0x559a72876630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb6a9d9b640 1 --2- 172.21.15.2:0/1589054591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a72876250 0x559a72876630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb6a9d9b640 1 -- 172.21.15.2:0/1589054591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a7287f750 msgr2=0x559a72881b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb6a9d9b640 1 --2- 172.21.15.2:0/1589054591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a7287f750 0x559a72881b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb6a9d9b640 1 -- 172.21.15.2:0/1589054591 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a727a0060 con 0x559a72876b70 2024-03-15T13:03:05.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb6aad9d640 1 --2- 172.21.15.2:0/1421193046 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb6940a1ba0 0x7fb6940a4060 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4005cc0 tx=0x7fb6a40324f0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb6a9d9b640 1 --2- 172.21.15.2:0/1589054591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876b70 0x559a7287f210 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009c00 tx=0x7fb6a0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29031947f0d5d42e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb698ff9640 1 -- 172.21.15.2:0/1589054591 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0004050 con 0x559a72876b70 2024-03-15T13:03:05.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb698ff9640 1 -- 172.21.15.2:0/1589054591 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb6a00035f0 con 0x559a72876b70 2024-03-15T13:03:05.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.876+0000 7fb698ff9640 1 -- 172.21.15.2:0/1589054591 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0019690 con 0x559a72876b70 2024-03-15T13:03:05.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.877+0000 7fb6b35d0740 1 -- 172.21.15.2:0/1589054591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876b70 msgr2=0x559a7287f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.877+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/1589054591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876b70 0x559a7287f210 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009c00 tx=0x7fb6a0011810 comp rx=0 tx=0).stop 2024-03-15T13:03:05.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.877+0000 7fb6b35d0740 1 -- 172.21.15.2:0/1589054591 shutdown_connections 2024-03-15T13:03:05.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.877+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/1589054591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a7287f750 0x559a72881b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.877+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/1589054591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876b70 0x559a7287f210 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.877+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/1589054591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a72876250 0x559a72876630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.877+0000 7fb6b35d0740 1 -- 172.21.15.2:0/1589054591 >> 172.21.15.2:0/1589054591 conn(0x559a72870980 msgr2=0x559a72870d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.877+0000 7fb6b35d0740 1 -- 172.21.15.2:0/1589054591 shutdown_connections 2024-03-15T13:03:05.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.877+0000 7fb6b35d0740 1 -- 172.21.15.2:0/1589054591 wait complete. 2024-03-15T13:03:05.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.877+0000 7fb6b35d0740 1 -- start start 2024-03-15T13:03:05.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6b35d0740 1 Processor -- start 2024-03-15T13:03:05.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6b35d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876250 0x559a7287f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6b35d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876b70 0x559a7287c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6b35d0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a7287f750 0x559a7287c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6b35d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a727de020 con 0x559a72876b70 2024-03-15T13:03:05.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6b35d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a724b1430 con 0x559a72876250 2024-03-15T13:03:05.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6b35d0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7246de00 con 0x559a7287f750 2024-03-15T13:03:05.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6b35d0740 10 client.0 _ll_register_callbacks cb 0x559a72785cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:05.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6a9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876b70 0x559a7287c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6a9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876b70 0x559a7287c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39972/0 (socket says 172.21.15.2:39972) 2024-03-15T13:03:05.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6a9d9b640 1 -- 172.21.15.2:0/3617264013 learned_addr learned my addr 172.21.15.2:0/3617264013 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6aa59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876250 0x559a7287f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6aad9d640 1 --2- 172.21.15.2:0/3617264013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a7287f750 0x559a7287c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6a9d9b640 1 -- 172.21.15.2:0/3617264013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a7287f750 msgr2=0x559a7287c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6a9d9b640 1 --2- 172.21.15.2:0/3617264013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a7287f750 0x559a7287c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6a9d9b640 1 -- 172.21.15.2:0/3617264013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876250 msgr2=0x559a7287f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6a9d9b640 1 --2- 172.21.15.2:0/3617264013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876250 0x559a7287f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6a9d9b640 1 -- 172.21.15.2:0/3617264013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a0009830 con 0x559a72876b70 2024-03-15T13:03:05.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb6a9d9b640 1 --2- 172.21.15.2:0/3617264013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876b70 0x559a7287c440 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009bd0 tx=0x7fb6a0003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa4cac9e76a5626 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.878+0000 7fb691ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:05.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.879+0000 7fb691ffb640 1 -- 172.21.15.2:0/3617264013 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0005610 con 0x559a72876b70 2024-03-15T13:03:05.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.879+0000 7fb691ffb640 1 -- 172.21.15.2:0/3617264013 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fb6a0019430 con 0x559a72876b70 2024-03-15T13:03:05.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.879+0000 7fb6b35d0740 1 -- 172.21.15.2:0/3617264013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a724dd7e0 con 0x559a72876b70 2024-03-15T13:03:05.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.879+0000 7fb691ffb640 1 -- 172.21.15.2:0/3617264013 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6a00195f0 con 0x559a72876b70 2024-03-15T13:03:05.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.879+0000 7fb6b35d0740 10 client.16929 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:05.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.879+0000 7fb6b35d0740 1 -- 172.21.15.2:0/3617264013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x559a727dc6a0 con 0x559a72876b70 2024-03-15T13:03:05.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.879+0000 7fb691ffb640 1 -- 172.21.15.2:0/3617264013 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x559a724dd7e0 con 0x559a72876b70 2024-03-15T13:03:05.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.879+0000 7fb691ffb640 1 -- 172.21.15.2:0/3617264013 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7fb6a000f580 con 0x559a72876b70 2024-03-15T13:03:05.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.879+0000 7fb6b35d0740 10 client.16929 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:05.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.879+0000 7fb6b35d0740 10 client.16929 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:03:05.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.879+0000 7fb6b35d0740 20 client.16929 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:05.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.879+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/3617264013 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559a7288f720 0x559a72891b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.879+0000 7fb6b35d0740 4 client.16929 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:05.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.879+0000 7fb6b35d0740 1 -- 172.21.15.2:0/3617264013 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a7251a720 con 0x559a7288f720 2024-03-15T13:03:05.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.879+0000 7fb6aa59c640 1 --2- 172.21.15.2:0/3617264013 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559a7288f720 0x559a72891b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.880+0000 7fb6aa59c640 1 --2- 172.21.15.2:0/3617264013 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559a7288f720 0x559a72891b40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=329171ad8b375bd2 server_cookie=a013c52163998fed in_seq=0 out_seq=0 2024-03-15T13:03:05.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.880+0000 7fb691ffb640 10 client.16929 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:03:05.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.881+0000 7fb691ffb640 1 -- 172.21.15.2:0/3617264013 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559a7251a720 con 0x559a7288f720 2024-03-15T13:03:05.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.881+0000 7fb691ffb640 10 client.16929 handle_command_reply: tid=0 2024-03-15T13:03:05.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.881+0000 7fb6b35d0740 1 client.16929 shutdown 2024-03-15T13:03:05.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.881+0000 7fb6b35d0740 1 -- 172.21.15.2:0/3617264013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876b70 msgr2=0x559a7287c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.881+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/3617264013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876b70 0x559a7287c440 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009bd0 tx=0x7fb6a0003ec0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.881+0000 7fb6b35d0740 1 -- 172.21.15.2:0/3617264013 shutdown_connections 2024-03-15T13:03:05.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.881+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/3617264013 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559a7288f720 0x559a72891b40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.881+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/3617264013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a7287f750 0x559a7287c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.881+0000 7fb691ffb640 0 client.16929 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:03:05.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.881+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/3617264013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876b70 0x559a7287c440 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.881+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/3617264013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876250 0x559a7287f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.881+0000 7fb6b35d0740 1 -- 172.21.15.2:0/3617264013 >> 172.21.15.2:0/3617264013 conn(0x559a72870980 msgr2=0x559a728726d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.881+0000 7fb6b35d0740 1 -- 172.21.15.2:0/3617264013 shutdown_connections 2024-03-15T13:03:05.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.881+0000 7fb6b35d0740 1 -- 172.21.15.2:0/3617264013 wait complete. 2024-03-15T13:03:05.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.881+0000 7fb6b35d0740 20 client.16929 trim_cache size 0 max 16384 2024-03-15T13:03:05.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.893+0000 7fb6b35d0740 1 Processor -- start 2024-03-15T13:03:05.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.893+0000 7fb6b35d0740 1 -- start start 2024-03-15T13:03:05.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6b35d0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a72876250 0x559a7287eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6b35d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876b70 0x559a7287f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6b35d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a7287f750 0x559a7288aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6b35d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a72763130 con 0x559a72876b70 2024-03-15T13:03:05.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6b35d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7288f520 con 0x559a7287f750 2024-03-15T13:03:05.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6b35d0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a72871980 con 0x559a72876250 2024-03-15T13:03:05.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6a9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876b70 0x559a7287f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6a9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876b70 0x559a7287f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39974/0 (socket says 172.21.15.2:39974) 2024-03-15T13:03:05.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6a9d9b640 1 -- 172.21.15.2:0/2066998720 learned_addr learned my addr 172.21.15.2:0/2066998720 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6aad9d640 1 --2- 172.21.15.2:0/2066998720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a7287f750 0x559a7288aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6aa59c640 1 --2- 172.21.15.2:0/2066998720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a72876250 0x559a7287eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6a9d9b640 1 -- 172.21.15.2:0/2066998720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a72876250 msgr2=0x559a7287eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6a9d9b640 1 --2- 172.21.15.2:0/2066998720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a72876250 0x559a7287eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6a9d9b640 1 -- 172.21.15.2:0/2066998720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a7287f750 msgr2=0x559a7288aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6a9d9b640 1 --2- 172.21.15.2:0/2066998720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a7287f750 0x559a7288aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6a9d9b640 1 -- 172.21.15.2:0/2066998720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a72778c20 con 0x559a72876b70 2024-03-15T13:03:05.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb6a9d9b640 1 --2- 172.21.15.2:0/2066998720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876b70 0x559a7287f0b0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00059f0 tx=0x7fb6a0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9de89667ad0fb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb693fff640 1 -- 172.21.15.2:0/2066998720 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a000fee0 con 0x559a72876b70 2024-03-15T13:03:05.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb693fff640 1 -- 172.21.15.2:0/2066998720 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb6a0003d20 con 0x559a72876b70 2024-03-15T13:03:05.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.894+0000 7fb693fff640 1 -- 172.21.15.2:0/2066998720 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a002e8d0 con 0x559a72876b70 2024-03-15T13:03:05.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 -- 172.21.15.2:0/2066998720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876b70 msgr2=0x559a7287f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/2066998720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876b70 0x559a7287f0b0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00059f0 tx=0x7fb6a0005dc0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 -- 172.21.15.2:0/2066998720 shutdown_connections 2024-03-15T13:03:05.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/2066998720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a7287f750 0x559a7288aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/2066998720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876b70 0x559a7287f0b0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/2066998720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a72876250 0x559a7287eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 -- 172.21.15.2:0/2066998720 >> 172.21.15.2:0/2066998720 conn(0x559a72870980 msgr2=0x559a7287c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 -- 172.21.15.2:0/2066998720 shutdown_connections 2024-03-15T13:03:05.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 -- 172.21.15.2:0/2066998720 wait complete. 2024-03-15T13:03:05.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 -- start start 2024-03-15T13:03:05.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 Processor -- start 2024-03-15T13:03:05.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876250 0x559a72882de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876b70 0x559a72883320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a7287f750 0x559a728ea010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a72883c20 con 0x559a72876250 2024-03-15T13:03:05.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a728ea550 con 0x559a72876b70 2024-03-15T13:03:05.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a728ea690 con 0x559a7287f750 2024-03-15T13:03:05.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.895+0000 7fb6b35d0740 10 client.0 _ll_register_callbacks cb 0x7fb6a001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb6aa59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876250 0x559a72882de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb6aa59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876250 0x559a72882de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39980/0 (socket says 172.21.15.2:39980) 2024-03-15T13:03:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb6aa59c640 1 -- 172.21.15.2:0/2724489616 learned_addr learned my addr 172.21.15.2:0/2724489616 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb6aad9d640 1 --2- 172.21.15.2:0/2724489616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a7287f750 0x559a728ea010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb6a9d9b640 1 --2- 172.21.15.2:0/2724489616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876b70 0x559a72883320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb6aa59c640 1 -- 172.21.15.2:0/2724489616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a7287f750 msgr2=0x559a728ea010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb6aa59c640 1 --2- 172.21.15.2:0/2724489616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a7287f750 0x559a728ea010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb6aa59c640 1 -- 172.21.15.2:0/2724489616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876b70 msgr2=0x559a72883320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb6aa59c640 1 --2- 172.21.15.2:0/2724489616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876b70 0x559a72883320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb6aa59c640 1 -- 172.21.15.2:0/2724489616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a727dc6a0 con 0x559a72876250 2024-03-15T13:03:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb6aa59c640 1 --2- 172.21.15.2:0/2724489616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876250 0x559a72882de0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb69c09a4c0 tx=0x7fb69c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a798ef6d34bcb59c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb691ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb691ffb640 1 -- 172.21.15.2:0/2724489616 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c0b3280 con 0x559a72876250 2024-03-15T13:03:05.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb691ffb640 1 -- 172.21.15.2:0/2724489616 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fb69c0cb4b0 con 0x559a72876250 2024-03-15T13:03:05.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb6b35d0740 1 -- 172.21.15.2:0/2724489616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6a00035f0 con 0x559a72876250 2024-03-15T13:03:05.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb691ffb640 1 -- 172.21.15.2:0/2724489616 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb69c096e30 con 0x559a72876250 2024-03-15T13:03:05.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb6b35d0740 10 client.16935 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:05.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb6b35d0740 1 -- 172.21.15.2:0/2724489616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x559a7279e8c0 con 0x559a72876250 2024-03-15T13:03:05.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.896+0000 7fb691ffb640 1 -- 172.21.15.2:0/2724489616 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb6a00035f0 con 0x559a72876250 2024-03-15T13:03:05.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.897+0000 7fb691ffb640 1 -- 172.21.15.2:0/2724489616 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7fb69c0ae200 con 0x559a72876250 2024-03-15T13:03:05.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.897+0000 7fb6b35d0740 10 client.16935 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:05.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.897+0000 7fb6b35d0740 10 client.16935 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:03:05.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.897+0000 7fb6b35d0740 20 client.16935 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:05.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.897+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/2724489616 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559a728ebe80 0x559a728ee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:05.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.897+0000 7fb6b35d0740 4 client.16935 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:03:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.897+0000 7fb6b35d0740 1 -- 172.21.15.2:0/2724489616 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559a72569890 con 0x559a728ebe80 2024-03-15T13:03:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.897+0000 7fb6a9d9b640 1 --2- 172.21.15.2:0/2724489616 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559a728ebe80 0x559a728ee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.897+0000 7fb691ffb640 10 client.16935 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:03:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.897+0000 7fb6a9d9b640 1 --2- 172.21.15.2:0/2724489616 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559a728ebe80 0x559a728ee2a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a6cd09c6a3481f2a server_cookie=bf9e80a1ffa6e692 in_seq=0 out_seq=0 2024-03-15T13:03:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.897+0000 7fb691ffb640 1 -- 172.21.15.2:0/2724489616 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559a72569890 con 0x559a728ebe80 2024-03-15T13:03:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.897+0000 7fb691ffb640 10 client.16935 handle_command_reply: tid=0 2024-03-15T13:03:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.897+0000 7fb6b35d0740 1 client.16935 shutdown 2024-03-15T13:03:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.897+0000 7fb6b35d0740 1 -- 172.21.15.2:0/2724489616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876250 msgr2=0x559a72882de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.897+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/2724489616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876250 0x559a72882de0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb69c09a4c0 tx=0x7fb69c01a040 comp rx=0 tx=0).stop 2024-03-15T13:03:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b35d0740 1 -- 172.21.15.2:0/2724489616 shutdown_connections 2024-03-15T13:03:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/2724489616 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559a728ebe80 0x559a728ee2a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/2724489616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a7287f750 0x559a728ea010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/2724489616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a72876b70 0x559a72883320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb691ffb640 0 client.16935 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:03:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b35d0740 1 --2- 172.21.15.2:0/2724489616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a72876250 0x559a72882de0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b35d0740 1 -- 172.21.15.2:0/2724489616 >> 172.21.15.2:0/2724489616 conn(0x559a72870980 msgr2=0x559a72871b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b35d0740 1 -- 172.21.15.2:0/2724489616 shutdown_connections 2024-03-15T13:03:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b35d0740 1 -- 172.21.15.2:0/2724489616 wait complete. 2024-03-15T13:03:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b35d0740 20 client.16935 trim_cache size 0 max 16384 2024-03-15T13:03:05.891 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:03:05.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b0919640 1 -- 172.21.15.2:0/1421193046 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb6940a1ba0 msgr2=0x7fb6940a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b0919640 1 --2- 172.21.15.2:0/1421193046 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb6940a1ba0 0x7fb6940a4060 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4005cc0 tx=0x7fb6a40324f0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b0919640 1 -- 172.21.15.2:0/1421193046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6ac102730 msgr2=0x7fb6ac19d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b0919640 1 --2- 172.21.15.2:0/1421193046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6ac102730 0x7fb6ac19d740 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00e9f0 tx=0x7fb69c00eeb0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b0919640 1 -- 172.21.15.2:0/1421193046 shutdown_connections 2024-03-15T13:03:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b0919640 1 --2- 172.21.15.2:0/1421193046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6ac10bb00 0x7fb6ac197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b0919640 1 --2- 172.21.15.2:0/1421193046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6ac103050 0x7fb6ac19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b0919640 1 --2- 172.21.15.2:0/1421193046 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb6940a1ba0 0x7fb6940a4060 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b0919640 1 --2- 172.21.15.2:0/1421193046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6ac102730 0x7fb6ac19d740 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b0919640 1 -- 172.21.15.2:0/1421193046 >> 172.21.15.2:0/1421193046 conn(0x7fb6ac076770 msgr2=0x7fb6ac0fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b0919640 1 -- 172.21.15.2:0/1421193046 shutdown_connections 2024-03-15T13:03:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:05.898+0000 7fb6b0919640 1 -- 172.21.15.2:0/1421193046 wait complete. 2024-03-15T13:03:05.894 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:03:05.895 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:03:06.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.014+0000 7fce6db3e640 1 -- 172.21.15.2:0/2976306400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce681014a0 msgr2=0x7fce68101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.014+0000 7fce6db3e640 1 --2- 172.21.15.2:0/2976306400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce681014a0 0x7fce68101920 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009900 tx=0x7fce5c02bc30 comp rx=0 tx=0).stop 2024-03-15T13:03:06.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.014+0000 7fce6db3e640 1 -- 172.21.15.2:0/2976306400 shutdown_connections 2024-03-15T13:03:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.014+0000 7fce6db3e640 1 --2- 172.21.15.2:0/2976306400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce68101e60 0x7fce6810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.014+0000 7fce6db3e640 1 --2- 172.21.15.2:0/2976306400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce681014a0 0x7fce68101920 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.014+0000 7fce6db3e640 1 --2- 172.21.15.2:0/2976306400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce68107560 0x7fce68107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.014+0000 7fce6db3e640 1 -- 172.21.15.2:0/2976306400 >> 172.21.15.2:0/2976306400 conn(0x7fce680788f0 msgr2=0x7fce68078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.014+0000 7fce6db3e640 1 -- 172.21.15.2:0/2976306400 shutdown_connections 2024-03-15T13:03:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.014+0000 7fce6db3e640 1 -- 172.21.15.2:0/2976306400 wait complete. 2024-03-15T13:03:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.014+0000 7fce6db3e640 1 Processor -- start 2024-03-15T13:03:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce6db3e640 1 -- start start 2024-03-15T13:03:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce6db3e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce681014a0 0x7fce6819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce6db3e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce68101e60 0x7fce6819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce6db3e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce68107560 0x7fce68197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce6db3e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce68197ec0 con 0x7fce68101e60 2024-03-15T13:03:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce6db3e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce68198030 con 0x7fce68107560 2024-03-15T13:03:06.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce6db3e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce681981f0 con 0x7fce681014a0 2024-03-15T13:03:06.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce66ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce68101e60 0x7fce6819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce66ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce68101e60 0x7fce6819dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39990/0 (socket says 172.21.15.2:39990) 2024-03-15T13:03:06.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce66ffd640 1 -- 172.21.15.2:0/2123989340 learned_addr learned my addr 172.21.15.2:0/2123989340 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:06.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce67fff640 1 --2- 172.21.15.2:0/2123989340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce68107560 0x7fce68197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce677fe640 1 --2- 172.21.15.2:0/2123989340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce681014a0 0x7fce6819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce66ffd640 1 -- 172.21.15.2:0/2123989340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce681014a0 msgr2=0x7fce6819d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce66ffd640 1 --2- 172.21.15.2:0/2123989340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce681014a0 0x7fce6819d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce66ffd640 1 -- 172.21.15.2:0/2123989340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce68107560 msgr2=0x7fce68197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce66ffd640 1 --2- 172.21.15.2:0/2123989340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce68107560 0x7fce68197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce66ffd640 1 -- 172.21.15.2:0/2123989340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c009560 con 0x7fce68101e60 2024-03-15T13:03:06.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.015+0000 7fce66ffd640 1 --2- 172.21.15.2:0/2123989340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce68101e60 0x7fce6819dc00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009230 tx=0x7fce5c02cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.016+0000 7fce64ff9640 1 -- 172.21.15.2:0/2123989340 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c004250 con 0x7fce68101e60 2024-03-15T13:03:06.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.016+0000 7fce64ff9640 1 -- 172.21.15.2:0/2123989340 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fce5c0043b0 con 0x7fce68101e60 2024-03-15T13:03:06.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.016+0000 7fce6db3e640 1 -- 172.21.15.2:0/2123989340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce68198470 con 0x7fce68101e60 2024-03-15T13:03:06.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.016+0000 7fce64ff9640 1 -- 172.21.15.2:0/2123989340 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c003bf0 con 0x7fce68101e60 2024-03-15T13:03:06.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.016+0000 7fce6db3e640 1 -- 172.21.15.2:0/2123989340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce681a4760 con 0x7fce68101e60 2024-03-15T13:03:06.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.018+0000 7fce64ff9640 1 -- 172.21.15.2:0/2123989340 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fce5c003d50 con 0x7fce68101e60 2024-03-15T13:03:06.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.018+0000 7fce707f8740 1 Processor -- start 2024-03-15T13:03:06.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.018+0000 7fce707f8740 1 -- start start 2024-03-15T13:03:06.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.018+0000 7fce64ff9640 1 --2- 172.21.15.2:0/2123989340 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fce500a1c20 0x7fce500a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.018+0000 7fce707f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557456892250 0x557456892630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.018+0000 7fce707f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557456892b70 0x55745689b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.018+0000 7fce707f8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55745689b750 0x55745689db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.018+0000 7fce707f8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745684d960 con 0x55745689b750 2024-03-15T13:03:06.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.018+0000 7fce707f8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557456801310 con 0x557456892b70 2024-03-15T13:03:06.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.018+0000 7fce707f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574567fe840 con 0x557456892250 2024-03-15T13:03:06.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.018+0000 7fce67fff640 1 --2- 172.21.15.2:0/2123989340 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fce500a1c20 0x7fce500a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.018+0000 7fce677fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557456892250 0x557456892630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.019+0000 7fce66ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557456892b70 0x55745689b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.019+0000 7fce677fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557456892250 0x557456892630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:43906/0 (socket says 172.21.15.2:43906) 2024-03-15T13:03:06.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.019+0000 7fce677fe640 1 -- 172.21.15.2:0/3728023870 learned_addr learned my addr 172.21.15.2:0/3728023870 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:06.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.019+0000 7fce67fff640 1 --2- 172.21.15.2:0/3728023870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55745689b750 0x55745689db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.019+0000 7fce64ff9640 1 -- 172.21.15.2:0/2123989340 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fce5c0efa10 con 0x7fce68101e60 2024-03-15T13:03:06.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.019+0000 7fce677fe640 1 -- 172.21.15.2:0/3728023870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557456892b70 msgr2=0x55745689b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.019+0000 7fce677fe640 1 --2- 172.21.15.2:0/3728023870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557456892b70 0x55745689b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.019+0000 7fce677fe640 1 -- 172.21.15.2:0/3728023870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55745689b750 msgr2=0x55745689db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.019+0000 7fce677fe640 1 --2- 172.21.15.2:0/3728023870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55745689b750 0x55745689db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.019+0000 7fce677fe640 1 -- 172.21.15.2:0/3728023870 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574567bc060 con 0x557456892250 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.019+0000 7fce67fff640 1 --2- 172.21.15.2:0/2123989340 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fce500a1c20 0x7fce500a40e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fce60005fd0 tx=0x7fce60009f10 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.019+0000 7fce677fe640 1 --2- 172.21.15.2:0/3728023870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557456892250 0x557456892630 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fce58009c90 tx=0x7fce58011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2e522d0371172e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.019+0000 7fce4dffb640 1 -- 172.21.15.2:0/3728023870 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58005220 con 0x557456892250 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.019+0000 7fce4dffb640 1 -- 172.21.15.2:0/3728023870 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fce58004c70 con 0x557456892250 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.019+0000 7fce4dffb640 1 -- 172.21.15.2:0/3728023870 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58019690 con 0x557456892250 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.020+0000 7fce707f8740 1 -- 172.21.15.2:0/3728023870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557456892250 msgr2=0x557456892630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.020+0000 7fce707f8740 1 --2- 172.21.15.2:0/3728023870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557456892250 0x557456892630 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fce58009c90 tx=0x7fce58011810 comp rx=0 tx=0).stop 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.020+0000 7fce707f8740 1 -- 172.21.15.2:0/3728023870 shutdown_connections 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.020+0000 7fce707f8740 1 --2- 172.21.15.2:0/3728023870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55745689b750 0x55745689db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.020+0000 7fce707f8740 1 --2- 172.21.15.2:0/3728023870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557456892b70 0x55745689b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.020+0000 7fce707f8740 1 --2- 172.21.15.2:0/3728023870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557456892250 0x557456892630 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.020+0000 7fce707f8740 1 -- 172.21.15.2:0/3728023870 >> 172.21.15.2:0/3728023870 conn(0x55745688c980 msgr2=0x55745688cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.020+0000 7fce707f8740 1 -- 172.21.15.2:0/3728023870 shutdown_connections 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.020+0000 7fce707f8740 1 -- 172.21.15.2:0/3728023870 wait complete. 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.020+0000 7fce707f8740 1 -- start start 2024-03-15T13:03:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.020+0000 7fce707f8740 1 Processor -- start 2024-03-15T13:03:06.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce707f8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892250 0x55745689b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce707f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557456892b70 0x557456898420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce707f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745689b750 0x557456898960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce707f8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574567fa020 con 0x557456892250 2024-03-15T13:03:06.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce707f8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574564cd430 con 0x557456892b70 2024-03-15T13:03:06.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce707f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557456489e00 con 0x55745689b750 2024-03-15T13:03:06.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce707f8740 10 client.0 _ll_register_callbacks cb 0x5574567a1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:06.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce677fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892250 0x55745689b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce677fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892250 0x55745689b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40012/0 (socket says 172.21.15.2:40012) 2024-03-15T13:03:06.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce677fe640 1 -- 172.21.15.2:0/2652175214 learned_addr learned my addr 172.21.15.2:0/2652175214 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:06.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce66ffd640 1 --2- 172.21.15.2:0/2652175214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557456892b70 0x557456898420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce67fff640 1 --2- 172.21.15.2:0/2652175214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745689b750 0x557456898960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce677fe640 1 -- 172.21.15.2:0/2652175214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745689b750 msgr2=0x557456898960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce677fe640 1 --2- 172.21.15.2:0/2652175214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745689b750 0x557456898960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce677fe640 1 -- 172.21.15.2:0/2652175214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557456892b70 msgr2=0x557456898420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce677fe640 1 --2- 172.21.15.2:0/2652175214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557456892b70 0x557456898420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce677fe640 1 -- 172.21.15.2:0/2652175214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce580098f0 con 0x557456892250 2024-03-15T13:03:06.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce677fe640 1 --2- 172.21.15.2:0/2652175214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892250 0x55745689b090 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fce58000c00 tx=0x7fce580096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81ec93459de7eab server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:06.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.021+0000 7fce46ffd640 1 -- 172.21.15.2:0/2652175214 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58005430 con 0x557456892250 2024-03-15T13:03:06.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.022+0000 7fce46ffd640 1 -- 172.21.15.2:0/2652175214 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fce58004030 con 0x557456892250 2024-03-15T13:03:06.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.022+0000 7fce707f8740 1 -- 172.21.15.2:0/2652175214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574564f97e0 con 0x557456892250 2024-03-15T13:03:06.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.022+0000 7fce46ffd640 1 -- 172.21.15.2:0/2652175214 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce58004b00 con 0x557456892250 2024-03-15T13:03:06.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.022+0000 7fce707f8740 10 client.16947 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:06.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.022+0000 7fce707f8740 1 -- 172.21.15.2:0/2652175214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5574567f86a0 con 0x557456892250 2024-03-15T13:03:06.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.022+0000 7fce46ffd640 1 -- 172.21.15.2:0/2652175214 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5574564f97e0 con 0x557456892250 2024-03-15T13:03:06.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.022+0000 7fce46ffd640 1 -- 172.21.15.2:0/2652175214 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7fce58021040 con 0x557456892250 2024-03-15T13:03:06.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.022+0000 7fce707f8740 10 client.16947 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:06.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.022+0000 7fce707f8740 10 client.16947 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:03:06.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.022+0000 7fce707f8740 20 client.16947 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:06.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.022+0000 7fce707f8740 1 --2- 172.21.15.2:0/2652175214 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5574568ab8b0 0x5574568adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.022+0000 7fce707f8740 4 client.16947 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:06.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.022+0000 7fce707f8740 1 -- 172.21.15.2:0/2652175214 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557456536720 con 0x5574568ab8b0 2024-03-15T13:03:06.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.022+0000 7fce66ffd640 1 --2- 172.21.15.2:0/2652175214 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5574568ab8b0 0x5574568adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.023+0000 7fce46ffd640 10 client.16947 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:03:06.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.023+0000 7fce66ffd640 1 --2- 172.21.15.2:0/2652175214 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5574568ab8b0 0x5574568adcd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70468f590fc1b623 server_cookie=5e97f87b9c16af79 in_seq=0 out_seq=0 2024-03-15T13:03:06.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.023+0000 7fce46ffd640 1 -- 172.21.15.2:0/2652175214 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557456536720 con 0x5574568ab8b0 2024-03-15T13:03:06.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.023+0000 7fce46ffd640 10 client.16947 handle_command_reply: tid=0 2024-03-15T13:03:06.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.023+0000 7fce707f8740 1 client.16947 shutdown 2024-03-15T13:03:06.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.023+0000 7fce707f8740 1 -- 172.21.15.2:0/2652175214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892250 msgr2=0x55745689b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.023+0000 7fce707f8740 1 --2- 172.21.15.2:0/2652175214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892250 0x55745689b090 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fce58000c00 tx=0x7fce580096e0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.024+0000 7fce707f8740 1 -- 172.21.15.2:0/2652175214 shutdown_connections 2024-03-15T13:03:06.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.024+0000 7fce707f8740 1 --2- 172.21.15.2:0/2652175214 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5574568ab8b0 0x5574568adcd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.024+0000 7fce707f8740 1 --2- 172.21.15.2:0/2652175214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745689b750 0x557456898960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.024+0000 7fce707f8740 1 --2- 172.21.15.2:0/2652175214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557456892b70 0x557456898420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.024+0000 7fce46ffd640 0 client.16947 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:03:06.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.024+0000 7fce707f8740 1 --2- 172.21.15.2:0/2652175214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892250 0x55745689b090 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.024+0000 7fce707f8740 1 -- 172.21.15.2:0/2652175214 >> 172.21.15.2:0/2652175214 conn(0x55745688c980 msgr2=0x55745688e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.024+0000 7fce707f8740 1 -- 172.21.15.2:0/2652175214 shutdown_connections 2024-03-15T13:03:06.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.024+0000 7fce707f8740 1 -- 172.21.15.2:0/2652175214 wait complete. 2024-03-15T13:03:06.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.024+0000 7fce707f8740 20 client.16947 trim_cache size 0 max 16384 2024-03-15T13:03:06.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.036+0000 7fce707f8740 1 Processor -- start 2024-03-15T13:03:06.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.036+0000 7fce707f8740 1 -- start start 2024-03-15T13:03:06.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.036+0000 7fce707f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557456892250 0x55745688b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.036+0000 7fce707f8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892b70 0x55745689ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.036+0000 7fce707f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55745689b750 0x55745689b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.036+0000 7fce707f8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745677f130 con 0x557456892b70 2024-03-15T13:03:06.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.036+0000 7fce707f8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574568aa050 con 0x55745689b750 2024-03-15T13:03:06.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.036+0000 7fce707f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574568a7c70 con 0x557456892250 2024-03-15T13:03:06.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.036+0000 7fce66ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892b70 0x55745689ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.036+0000 7fce677fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557456892250 0x55745688b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.036+0000 7fce66ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892b70 0x55745689ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40028/0 (socket says 172.21.15.2:40028) 2024-03-15T13:03:06.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.036+0000 7fce67fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55745689b750 0x55745689b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.036+0000 7fce66ffd640 1 -- 172.21.15.2:0/3871728214 learned_addr learned my addr 172.21.15.2:0/3871728214 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:06.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.037+0000 7fce677fe640 1 -- 172.21.15.2:0/3871728214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55745689b750 msgr2=0x55745689b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.037+0000 7fce677fe640 1 --2- 172.21.15.2:0/3871728214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55745689b750 0x55745689b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.037+0000 7fce677fe640 1 -- 172.21.15.2:0/3871728214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892b70 msgr2=0x55745689ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.037+0000 7fce677fe640 1 --2- 172.21.15.2:0/3871728214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892b70 0x55745689ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.037+0000 7fce677fe640 1 -- 172.21.15.2:0/3871728214 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557456794c20 con 0x557456892250 2024-03-15T13:03:06.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.037+0000 7fce66ffd640 1 --2- 172.21.15.2:0/3871728214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892b70 0x55745689ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:06.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.037+0000 7fce677fe640 1 --2- 172.21.15.2:0/3871728214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557456892250 0x55745688b2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fce58023530 tx=0x7fce58011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b86a08cb914a8c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.037+0000 7fce4d7fa640 1 -- 172.21.15.2:0/3871728214 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce580049d0 con 0x557456892250 2024-03-15T13:03:06.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.037+0000 7fce4d7fa640 1 -- 172.21.15.2:0/3871728214 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fce58023720 con 0x557456892250 2024-03-15T13:03:06.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.037+0000 7fce4d7fa640 1 -- 172.21.15.2:0/3871728214 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58019690 con 0x557456892250 2024-03-15T13:03:06.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.037+0000 7fce707f8740 1 -- 172.21.15.2:0/3871728214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557456892250 msgr2=0x55745688b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.037+0000 7fce707f8740 1 --2- 172.21.15.2:0/3871728214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557456892250 0x55745688b2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fce58023530 tx=0x7fce58011810 comp rx=0 tx=0).stop 2024-03-15T13:03:06.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 1 -- 172.21.15.2:0/3871728214 shutdown_connections 2024-03-15T13:03:06.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 1 --2- 172.21.15.2:0/3871728214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55745689b750 0x55745689b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 1 --2- 172.21.15.2:0/3871728214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892b70 0x55745689ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 1 --2- 172.21.15.2:0/3871728214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557456892250 0x55745688b2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 1 -- 172.21.15.2:0/3871728214 >> 172.21.15.2:0/3871728214 conn(0x55745688c980 msgr2=0x557456898540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 1 -- 172.21.15.2:0/3871728214 shutdown_connections 2024-03-15T13:03:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 1 -- 172.21.15.2:0/3871728214 wait complete. 2024-03-15T13:03:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 1 -- start start 2024-03-15T13:03:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 1 Processor -- start 2024-03-15T13:03:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892250 0x557456894c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557456892b70 0x5574568951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745689b750 0x557456895710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557456906270 con 0x557456892250 2024-03-15T13:03:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574569063e0 con 0x557456892b70 2024-03-15T13:03:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574569065a0 con 0x55745689b750 2024-03-15T13:03:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce707f8740 10 client.0 _ll_register_callbacks cb 0x7fce600117a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce677fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892250 0x557456894c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce677fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892250 0x557456894c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40040/0 (socket says 172.21.15.2:40040) 2024-03-15T13:03:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce677fe640 1 -- 172.21.15.2:0/2843815084 learned_addr learned my addr 172.21.15.2:0/2843815084 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce67fff640 1 --2- 172.21.15.2:0/2843815084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745689b750 0x557456895710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.038+0000 7fce66ffd640 1 --2- 172.21.15.2:0/2843815084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557456892b70 0x5574568951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce677fe640 1 -- 172.21.15.2:0/2843815084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745689b750 msgr2=0x557456895710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce677fe640 1 --2- 172.21.15.2:0/2843815084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745689b750 0x557456895710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce677fe640 1 -- 172.21.15.2:0/2843815084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557456892b70 msgr2=0x5574568951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce677fe640 1 --2- 172.21.15.2:0/2843815084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557456892b70 0x5574568951d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce677fe640 1 -- 172.21.15.2:0/2843815084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce681a4760 con 0x557456892250 2024-03-15T13:03:06.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce677fe640 1 --2- 172.21.15.2:0/2843815084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892250 0x557456894c90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fce58005460 tx=0x7fce58004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d4fc6640bb7e5b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:06.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce46ffd640 1 -- 172.21.15.2:0/2843815084 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58005760 con 0x557456892250 2024-03-15T13:03:06.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce46ffd640 1 -- 172.21.15.2:0/2843815084 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fce58000e40 con 0x557456892250 2024-03-15T13:03:06.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce707f8740 1 -- 172.21.15.2:0/2843815084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574568880f0 con 0x557456892250 2024-03-15T13:03:06.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce46ffd640 1 -- 172.21.15.2:0/2843815084 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce58005500 con 0x557456892250 2024-03-15T13:03:06.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce707f8740 10 client.16953 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:06.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce707f8740 1 -- 172.21.15.2:0/2843815084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5574567f86a0 con 0x557456892250 2024-03-15T13:03:06.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce46ffd640 1 -- 172.21.15.2:0/2843815084 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5574568880f0 con 0x557456892250 2024-03-15T13:03:06.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce46ffd640 1 -- 172.21.15.2:0/2843815084 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7fce58039a60 con 0x557456892250 2024-03-15T13:03:06.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce707f8740 10 client.16953 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:06.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce707f8740 10 client.16953 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:03:06.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce707f8740 20 client.16953 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:06.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.039+0000 7fce707f8740 1 --2- 172.21.15.2:0/2843815084 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557456907930 0x557456909d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce707f8740 4 client.16953 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:03:06.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce707f8740 1 -- 172.21.15.2:0/2843815084 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fce58004c70 con 0x557456907930 2024-03-15T13:03:06.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce66ffd640 1 --2- 172.21.15.2:0/2843815084 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557456907930 0x557456909d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce46ffd640 10 client.16953 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:03:06.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce66ffd640 1 --2- 172.21.15.2:0/2843815084 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557456907930 0x557456909d50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f3e88925ff69e34f server_cookie=b35202294cd2a1a0 in_seq=0 out_seq=0 2024-03-15T13:03:06.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce46ffd640 1 -- 172.21.15.2:0/2843815084 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fce58004c70 con 0x557456907930 2024-03-15T13:03:06.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce46ffd640 10 client.16953 handle_command_reply: tid=0 2024-03-15T13:03:06.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce707f8740 1 client.16953 shutdown 2024-03-15T13:03:06.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce707f8740 1 -- 172.21.15.2:0/2843815084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892250 msgr2=0x557456894c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce707f8740 1 --2- 172.21.15.2:0/2843815084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892250 0x557456894c90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fce58005460 tx=0x7fce58004c00 comp rx=0 tx=0).stop 2024-03-15T13:03:06.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce707f8740 1 -- 172.21.15.2:0/2843815084 shutdown_connections 2024-03-15T13:03:06.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce707f8740 1 --2- 172.21.15.2:0/2843815084 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557456907930 0x557456909d50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce707f8740 1 --2- 172.21.15.2:0/2843815084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55745689b750 0x557456895710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce707f8740 1 --2- 172.21.15.2:0/2843815084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557456892b70 0x5574568951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce707f8740 1 --2- 172.21.15.2:0/2843815084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557456892250 0x557456894c90 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce46ffd640 0 client.16953 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:03:06.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce707f8740 1 -- 172.21.15.2:0/2843815084 >> 172.21.15.2:0/2843815084 conn(0x55745688c980 msgr2=0x557456892f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.040+0000 7fce707f8740 1 -- 172.21.15.2:0/2843815084 shutdown_connections 2024-03-15T13:03:06.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.041+0000 7fce707f8740 1 -- 172.21.15.2:0/2843815084 wait complete. 2024-03-15T13:03:06.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.041+0000 7fce707f8740 20 client.16953 trim_cache size 0 max 16384 2024-03-15T13:03:06.033 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:03:06.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.041+0000 7fce6db3e640 1 -- 172.21.15.2:0/2123989340 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fce500a1c20 msgr2=0x7fce500a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.041+0000 7fce6db3e640 1 --2- 172.21.15.2:0/2123989340 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fce500a1c20 0x7fce500a40e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fce60005fd0 tx=0x7fce60009f10 comp rx=0 tx=0).stop 2024-03-15T13:03:06.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.041+0000 7fce6db3e640 1 -- 172.21.15.2:0/2123989340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce68101e60 msgr2=0x7fce6819dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.041+0000 7fce6db3e640 1 --2- 172.21.15.2:0/2123989340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce68101e60 0x7fce6819dc00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009230 tx=0x7fce5c02cac0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.041+0000 7fce6db3e640 1 -- 172.21.15.2:0/2123989340 shutdown_connections 2024-03-15T13:03:06.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.041+0000 7fce6db3e640 1 --2- 172.21.15.2:0/2123989340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce68107560 0x7fce68197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.041+0000 7fce6db3e640 1 --2- 172.21.15.2:0/2123989340 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fce500a1c20 0x7fce500a40e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.041+0000 7fce6db3e640 1 --2- 172.21.15.2:0/2123989340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce68101e60 0x7fce6819dc00 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.041+0000 7fce6db3e640 1 --2- 172.21.15.2:0/2123989340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce681014a0 0x7fce6819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.041+0000 7fce6db3e640 1 -- 172.21.15.2:0/2123989340 >> 172.21.15.2:0/2123989340 conn(0x7fce680788f0 msgr2=0x7fce68100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.041+0000 7fce6db3e640 1 -- 172.21.15.2:0/2123989340 shutdown_connections 2024-03-15T13:03:06.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.041+0000 7fce6db3e640 1 -- 172.21.15.2:0/2123989340 wait complete. 2024-03-15T13:03:06.037 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:03:06.037 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:03:06.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.157+0000 7f55c8e91640 1 -- 172.21.15.2:0/671874347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55c41064b0 msgr2=0x7f55c4106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.157+0000 7f55c8e91640 1 --2- 172.21.15.2:0/671874347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55c41064b0 0x7f55c4106890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009c10 tx=0x7f55b402bef0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.157+0000 7f55c8e91640 1 -- 172.21.15.2:0/671874347 shutdown_connections 2024-03-15T13:03:06.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.157+0000 7f55c8e91640 1 --2- 172.21.15.2:0/671874347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c4100dd0 0x7f55c410d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.157+0000 7f55c8e91640 1 --2- 172.21.15.2:0/671874347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55c4100410 0x7f55c4100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.157+0000 7f55c8e91640 1 --2- 172.21.15.2:0/671874347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55c41064b0 0x7f55c4106890 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.157+0000 7f55c8e91640 1 -- 172.21.15.2:0/671874347 >> 172.21.15.2:0/671874347 conn(0x7f55c4077550 msgr2=0x7f55c4077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.157+0000 7f55c8e91640 1 -- 172.21.15.2:0/671874347 shutdown_connections 2024-03-15T13:03:06.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.157+0000 7f55c8e91640 1 -- 172.21.15.2:0/671874347 wait complete. 2024-03-15T13:03:06.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c8e91640 1 Processor -- start 2024-03-15T13:03:06.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c8e91640 1 -- start start 2024-03-15T13:03:06.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c8e91640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55c4100410 0x7f55c41967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c8e91640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55c4100dd0 0x7f55c4196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c8e91640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c41064b0 0x7f55c4197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c8e91640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c41a5f10 con 0x7f55c41064b0 2024-03-15T13:03:06.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c8e91640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c41a6080 con 0x7f55c4100dd0 2024-03-15T13:03:06.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c8e91640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c41a6240 con 0x7f55c4100410 2024-03-15T13:03:06.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c2d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c41064b0 0x7f55c4197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c2d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c41064b0 0x7f55c4197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40070/0 (socket says 172.21.15.2:40070) 2024-03-15T13:03:06.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c2d9d640 1 -- 172.21.15.2:0/3996548773 learned_addr learned my addr 172.21.15.2:0/3996548773 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:06.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c259c640 1 --2- 172.21.15.2:0/3996548773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55c4100410 0x7f55c41967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c1d9b640 1 --2- 172.21.15.2:0/3996548773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55c4100dd0 0x7f55c4196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c2d9d640 1 -- 172.21.15.2:0/3996548773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55c4100410 msgr2=0x7f55c41967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c2d9d640 1 --2- 172.21.15.2:0/3996548773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55c4100410 0x7f55c41967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c2d9d640 1 -- 172.21.15.2:0/3996548773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55c4100dd0 msgr2=0x7f55c4196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c2d9d640 1 --2- 172.21.15.2:0/3996548773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55c4100dd0 0x7f55c4196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c2d9d640 1 -- 172.21.15.2:0/3996548773 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc009560 con 0x7f55c41064b0 2024-03-15T13:03:06.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.158+0000 7f55c259c640 1 --2- 172.21.15.2:0/3996548773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55c4100410 0x7f55c41967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:06.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.159+0000 7f55c2d9d640 1 --2- 172.21.15.2:0/3996548773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c41064b0 0x7f55c4197250 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00e940 tx=0x7f55bc00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.159+0000 7f55bb7fe640 1 -- 172.21.15.2:0/3996548773 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00cd80 con 0x7f55c41064b0 2024-03-15T13:03:06.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.159+0000 7f55c8e91640 1 -- 172.21.15.2:0/3996548773 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b4009840 con 0x7f55c41064b0 2024-03-15T13:03:06.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.159+0000 7f55bb7fe640 1 -- 172.21.15.2:0/3996548773 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f55bc0035f0 con 0x7f55c41064b0 2024-03-15T13:03:06.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.159+0000 7f55bb7fe640 1 -- 172.21.15.2:0/3996548773 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc004500 con 0x7f55c41064b0 2024-03-15T13:03:06.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.159+0000 7f55c8e91640 1 -- 172.21.15.2:0/3996548773 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c41a68a0 con 0x7f55c41064b0 2024-03-15T13:03:06.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.161+0000 7f55bb7fe640 1 -- 172.21.15.2:0/3996548773 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f55bc003760 con 0x7f55c41064b0 2024-03-15T13:03:06.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.161+0000 7f55cba60740 1 Processor -- start 2024-03-15T13:03:06.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55cba60740 1 -- start start 2024-03-15T13:03:06.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55cba60740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287ae250 0x557a287ae630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55cba60740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287aeb70 0x557a287b7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55cba60740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287b7750 0x557a287b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55cba60740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a28769960 con 0x557a287b7750 2024-03-15T13:03:06.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55cba60740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2871d310 con 0x557a287aeb70 2024-03-15T13:03:06.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55cba60740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2871a840 con 0x557a287ae250 2024-03-15T13:03:06.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55bb7fe640 1 --2- 172.21.15.2:0/3996548773 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f55b00a1ba0 0x7f55b00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55c259c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287aeb70 0x557a287b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55c1d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287b7750 0x557a287b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55c259c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287aeb70 0x557a287b7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47852/0 (socket says 172.21.15.2:47852) 2024-03-15T13:03:06.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55c259c640 1 -- 172.21.15.2:0/363301671 learned_addr learned my addr 172.21.15.2:0/363301671 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:06.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55c1d9b640 1 --2- 172.21.15.2:0/363301671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287ae250 0x557a287ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55c2d9d640 1 --2- 172.21.15.2:0/3996548773 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f55b00a1ba0 0x7f55b00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55bb7fe640 1 -- 172.21.15.2:0/3996548773 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f55bc014070 con 0x7f55c41064b0 2024-03-15T13:03:06.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55c259c640 1 -- 172.21.15.2:0/363301671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287ae250 msgr2=0x557a287ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55c259c640 1 --2- 172.21.15.2:0/363301671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287ae250 0x557a287ae630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55c259c640 1 -- 172.21.15.2:0/363301671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287b7750 msgr2=0x557a287b9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55c259c640 1 --2- 172.21.15.2:0/363301671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287b7750 0x557a287b9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55c259c640 1 -- 172.21.15.2:0/363301671 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a286d8060 con 0x557a287aeb70 2024-03-15T13:03:06.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.162+0000 7f55c1d9b640 1 --2- 172.21.15.2:0/363301671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287b7750 0x557a287b9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:06.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.163+0000 7f55c259c640 1 --2- 172.21.15.2:0/363301671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287aeb70 0x557a287b7210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009be0 tx=0x7f55b4033e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2282c73ce5c233af server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.163+0000 7f55c2d9d640 1 --2- 172.21.15.2:0/3996548773 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f55b00a1ba0 0x7f55b00a4060 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0cb9c0 tx=0x7f55bc06db50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.163+0000 7f55b8ff9640 1 -- 172.21.15.2:0/363301671 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b402c910 con 0x557a287aeb70 2024-03-15T13:03:06.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.163+0000 7f55b8ff9640 1 -- 172.21.15.2:0/363301671 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f55b402ca70 con 0x557a287aeb70 2024-03-15T13:03:06.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.163+0000 7f55b8ff9640 1 -- 172.21.15.2:0/363301671 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4004370 con 0x557a287aeb70 2024-03-15T13:03:06.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.163+0000 7f55cba60740 1 -- 172.21.15.2:0/363301671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287aeb70 msgr2=0x557a287b7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.163+0000 7f55cba60740 1 --2- 172.21.15.2:0/363301671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287aeb70 0x557a287b7210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009be0 tx=0x7f55b4033e50 comp rx=0 tx=0).stop 2024-03-15T13:03:06.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.163+0000 7f55cba60740 1 -- 172.21.15.2:0/363301671 shutdown_connections 2024-03-15T13:03:06.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.163+0000 7f55cba60740 1 --2- 172.21.15.2:0/363301671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287b7750 0x557a287b9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.163+0000 7f55cba60740 1 --2- 172.21.15.2:0/363301671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287aeb70 0x557a287b7210 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55cba60740 1 --2- 172.21.15.2:0/363301671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287ae250 0x557a287ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55cba60740 1 -- 172.21.15.2:0/363301671 >> 172.21.15.2:0/363301671 conn(0x557a287a8980 msgr2=0x557a287a8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55cba60740 1 -- 172.21.15.2:0/363301671 shutdown_connections 2024-03-15T13:03:06.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55cba60740 1 -- 172.21.15.2:0/363301671 wait complete. 2024-03-15T13:03:06.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55cba60740 1 -- start start 2024-03-15T13:03:06.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55cba60740 1 Processor -- start 2024-03-15T13:03:06.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55cba60740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287ae250 0x557a287b7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55cba60740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287aeb70 0x557a287b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55cba60740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287b7750 0x557a287b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55cba60740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a28716020 con 0x557a287ae250 2024-03-15T13:03:06.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55cba60740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a283e9430 con 0x557a287aeb70 2024-03-15T13:03:06.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55cba60740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a283a5e00 con 0x557a287b7750 2024-03-15T13:03:06.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55cba60740 10 client.0 _ll_register_callbacks cb 0x557a286bdcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:06.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55c1d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287ae250 0x557a287b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55c1d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287ae250 0x557a287b7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40096/0 (socket says 172.21.15.2:40096) 2024-03-15T13:03:06.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55c1d9b640 1 -- 172.21.15.2:0/793497484 learned_addr learned my addr 172.21.15.2:0/793497484 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:06.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55c259c640 1 --2- 172.21.15.2:0/793497484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287aeb70 0x557a287b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.164+0000 7f55c1d9b640 1 --2- 172.21.15.2:0/793497484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287b7750 0x557a287b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.165+0000 7f55c1d9b640 1 -- 172.21.15.2:0/793497484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287b7750 msgr2=0x557a287b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.165+0000 7f55c1d9b640 1 --2- 172.21.15.2:0/793497484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287b7750 0x557a287b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.165+0000 7f55c1d9b640 1 -- 172.21.15.2:0/793497484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287aeb70 msgr2=0x557a287b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.165+0000 7f55c1d9b640 1 --2- 172.21.15.2:0/793497484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287aeb70 0x557a287b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.165+0000 7f55c1d9b640 1 -- 172.21.15.2:0/793497484 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c410dbe0 con 0x557a287ae250 2024-03-15T13:03:06.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.165+0000 7f55c1d9b640 1 --2- 172.21.15.2:0/793497484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287ae250 0x557a287b7060 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f55ac004650 tx=0x7f55ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bc53ebc9646847 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.165+0000 7f55a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:06.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.165+0000 7f55a9ffb640 1 -- 172.21.15.2:0/793497484 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac012b60 con 0x557a287ae250 2024-03-15T13:03:06.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.165+0000 7f55a9ffb640 1 -- 172.21.15.2:0/793497484 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f55ac012cc0 con 0x557a287ae250 2024-03-15T13:03:06.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.165+0000 7f55cba60740 1 -- 172.21.15.2:0/793497484 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a28452720 con 0x557a287ae250 2024-03-15T13:03:06.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.165+0000 7f55a9ffb640 1 -- 172.21.15.2:0/793497484 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55ac012e80 con 0x557a287ae250 2024-03-15T13:03:06.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.165+0000 7f55cba60740 10 client.16965 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:06.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.165+0000 7f55cba60740 1 -- 172.21.15.2:0/793497484 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x557a287146a0 con 0x557a287ae250 2024-03-15T13:03:06.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.165+0000 7f55a9ffb640 1 -- 172.21.15.2:0/793497484 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x557a28452720 con 0x557a287ae250 2024-03-15T13:03:06.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.166+0000 7f55a9ffb640 1 -- 172.21.15.2:0/793497484 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7f55ac004bd0 con 0x557a287ae250 2024-03-15T13:03:06.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.166+0000 7f55cba60740 10 client.16965 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:06.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.166+0000 7f55cba60740 10 client.16965 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:03:06.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.166+0000 7f55cba60740 20 client.16965 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:06.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.166+0000 7f55cba60740 1 --2- 172.21.15.2:0/793497484 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557a287c77c0 0x557a287c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.166+0000 7f55cba60740 4 client.16965 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:06.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.166+0000 7f55cba60740 1 -- 172.21.15.2:0/793497484 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a284a1890 con 0x557a287c77c0 2024-03-15T13:03:06.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.166+0000 7f55c259c640 1 --2- 172.21.15.2:0/793497484 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557a287c77c0 0x557a287c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.166+0000 7f55a9ffb640 10 client.16965 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:03:06.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.166+0000 7f55c259c640 1 --2- 172.21.15.2:0/793497484 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557a287c77c0 0x557a287c9be0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40d57bbd7c08c4bb server_cookie=22727e4b8bb9caf5 in_seq=0 out_seq=0 2024-03-15T13:03:06.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.167+0000 7f55a9ffb640 1 -- 172.21.15.2:0/793497484 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557a284a1890 con 0x557a287c77c0 2024-03-15T13:03:06.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.167+0000 7f55a9ffb640 10 client.16965 handle_command_reply: tid=0 2024-03-15T13:03:06.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.167+0000 7f55cba60740 1 client.16965 shutdown 2024-03-15T13:03:06.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.167+0000 7f55cba60740 1 -- 172.21.15.2:0/793497484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287ae250 msgr2=0x557a287b7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.167+0000 7f55cba60740 1 --2- 172.21.15.2:0/793497484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287ae250 0x557a287b7060 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f55ac004650 tx=0x7f55ac004730 comp rx=0 tx=0).stop 2024-03-15T13:03:06.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.167+0000 7f55cba60740 1 -- 172.21.15.2:0/793497484 shutdown_connections 2024-03-15T13:03:06.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.167+0000 7f55cba60740 1 --2- 172.21.15.2:0/793497484 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557a287c77c0 0x557a287c9be0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.167+0000 7f55cba60740 1 --2- 172.21.15.2:0/793497484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287b7750 0x557a287b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.167+0000 7f55cba60740 1 --2- 172.21.15.2:0/793497484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287aeb70 0x557a287b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.167+0000 7f55cba60740 1 --2- 172.21.15.2:0/793497484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287ae250 0x557a287b7060 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.167+0000 7f55cba60740 1 -- 172.21.15.2:0/793497484 >> 172.21.15.2:0/793497484 conn(0x557a287a8980 msgr2=0x557a287aa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.167+0000 7f55a9ffb640 0 client.16965 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:03:06.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.167+0000 7f55cba60740 1 -- 172.21.15.2:0/793497484 shutdown_connections 2024-03-15T13:03:06.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.167+0000 7f55cba60740 1 -- 172.21.15.2:0/793497484 wait complete. 2024-03-15T13:03:06.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.167+0000 7f55cba60740 20 client.16965 trim_cache size 0 max 16384 2024-03-15T13:03:06.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55cba60740 1 Processor -- start 2024-03-15T13:03:06.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55cba60740 1 -- start start 2024-03-15T13:03:06.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55cba60740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287a8980 0x557a287b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55cba60740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287ae250 0x557a287b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55cba60740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287aeb70 0x557a287b7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55cba60740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2869b130 con 0x557a287ae250 2024-03-15T13:03:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55cba60740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a287b5560 con 0x557a287a8980 2024-03-15T13:03:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55cba60740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a28452720 con 0x557a287aeb70 2024-03-15T13:03:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55c259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287ae250 0x557a287b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55c259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287ae250 0x557a287b6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40100/0 (socket says 172.21.15.2:40100) 2024-03-15T13:03:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55c259c640 1 -- 172.21.15.2:0/2798521026 learned_addr learned my addr 172.21.15.2:0/2798521026 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55c1d9b640 1 --2- 172.21.15.2:0/2798521026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287a8980 0x557a287b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55c1d9b640 1 --2- 172.21.15.2:0/2798521026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287aeb70 0x557a287b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55c259c640 1 -- 172.21.15.2:0/2798521026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287aeb70 msgr2=0x557a287b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55c259c640 1 --2- 172.21.15.2:0/2798521026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287aeb70 0x557a287b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55c259c640 1 -- 172.21.15.2:0/2798521026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287a8980 msgr2=0x557a287b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55c259c640 1 --2- 172.21.15.2:0/2798521026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287a8980 0x557a287b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55c259c640 1 -- 172.21.15.2:0/2798521026 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a286b0c20 con 0x557a287ae250 2024-03-15T13:03:06.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.180+0000 7f55c259c640 1 --2- 172.21.15.2:0/2798521026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287ae250 0x557a287b6b20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f55b402c4d0 tx=0x7f55b4041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad57f19e1c25ee0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.181+0000 7f55abfff640 1 -- 172.21.15.2:0/2798521026 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4003d90 con 0x557a287ae250 2024-03-15T13:03:06.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.181+0000 7f55abfff640 1 -- 172.21.15.2:0/2798521026 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f55b4004040 con 0x557a287ae250 2024-03-15T13:03:06.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.181+0000 7f55abfff640 1 -- 172.21.15.2:0/2798521026 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4034690 con 0x557a287ae250 2024-03-15T13:03:06.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.181+0000 7f55cba60740 1 -- 172.21.15.2:0/2798521026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287ae250 msgr2=0x557a287b6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.181+0000 7f55cba60740 1 --2- 172.21.15.2:0/2798521026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287ae250 0x557a287b6b20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f55b402c4d0 tx=0x7f55b4041040 comp rx=0 tx=0).stop 2024-03-15T13:03:06.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.181+0000 7f55cba60740 1 -- 172.21.15.2:0/2798521026 shutdown_connections 2024-03-15T13:03:06.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.181+0000 7f55cba60740 1 --2- 172.21.15.2:0/2798521026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287aeb70 0x557a287b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.181+0000 7f55cba60740 1 --2- 172.21.15.2:0/2798521026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287ae250 0x557a287b6b20 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.181+0000 7f55cba60740 1 --2- 172.21.15.2:0/2798521026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287a8980 0x557a287b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.181+0000 7f55cba60740 1 -- 172.21.15.2:0/2798521026 >> 172.21.15.2:0/2798521026 conn(0x557a287b4a90 msgr2=0x557a287a7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.181+0000 7f55cba60740 1 -- 172.21.15.2:0/2798521026 shutdown_connections 2024-03-15T13:03:06.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.181+0000 7f55cba60740 1 -- 172.21.15.2:0/2798521026 wait complete. 2024-03-15T13:03:06.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.181+0000 7f55cba60740 1 -- start start 2024-03-15T13:03:06.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.181+0000 7f55cba60740 1 Processor -- start 2024-03-15T13:03:06.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55cba60740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287a8980 0x557a287b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55cba60740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287ae250 0x557a287b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55cba60740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287aeb70 0x557a28822070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55cba60740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a287b1ab0 con 0x557a287a8980 2024-03-15T13:03:06.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55cba60740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a287b1c20 con 0x557a287ae250 2024-03-15T13:03:06.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55cba60740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a288225b0 con 0x557a287aeb70 2024-03-15T13:03:06.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55cba60740 10 client.0 _ll_register_callbacks cb 0x7f55ac004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:06.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55c1d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287a8980 0x557a287b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55c1d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287a8980 0x557a287b0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40116/0 (socket says 172.21.15.2:40116) 2024-03-15T13:03:06.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55c1d9b640 1 -- 172.21.15.2:0/2643697764 learned_addr learned my addr 172.21.15.2:0/2643697764 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:06.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55c259c640 1 --2- 172.21.15.2:0/2643697764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287ae250 0x557a287b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55c1d9b640 1 --2- 172.21.15.2:0/2643697764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287aeb70 0x557a28822070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55c1d9b640 1 -- 172.21.15.2:0/2643697764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287aeb70 msgr2=0x557a28822070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55c1d9b640 1 --2- 172.21.15.2:0/2643697764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287aeb70 0x557a28822070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55c1d9b640 1 -- 172.21.15.2:0/2643697764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287ae250 msgr2=0x557a287b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55c1d9b640 1 --2- 172.21.15.2:0/2643697764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287ae250 0x557a287b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55c1d9b640 1 -- 172.21.15.2:0/2643697764 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a286d8060 con 0x557a287a8980 2024-03-15T13:03:06.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55c1d9b640 1 --2- 172.21.15.2:0/2643697764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287a8980 0x557a287b0dd0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f55ac001eb0 tx=0x7f55ac032040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb6c043ef39317f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:06.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55a9ffb640 1 -- 172.21.15.2:0/2643697764 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac012230 con 0x557a287a8980 2024-03-15T13:03:06.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55a9ffb640 1 -- 172.21.15.2:0/2643697764 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f55ac012930 con 0x557a287a8980 2024-03-15T13:03:06.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.182+0000 7f55cba60740 1 -- 172.21.15.2:0/2643697764 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a284157e0 con 0x557a287a8980 2024-03-15T13:03:06.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.183+0000 7f55a9ffb640 1 -- 172.21.15.2:0/2643697764 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55ac012390 con 0x557a287a8980 2024-03-15T13:03:06.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.183+0000 7f55cba60740 10 client.16971 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:06.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.183+0000 7f55cba60740 1 -- 172.21.15.2:0/2643697764 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x557a286d68c0 con 0x557a287a8980 2024-03-15T13:03:06.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.183+0000 7f55a9ffb640 1 -- 172.21.15.2:0/2643697764 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x557a284157e0 con 0x557a287a8980 2024-03-15T13:03:06.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.183+0000 7f55a9ffb640 1 -- 172.21.15.2:0/2643697764 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7f55ac03c070 con 0x557a287a8980 2024-03-15T13:03:06.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.183+0000 7f55cba60740 10 client.16971 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:06.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.183+0000 7f55cba60740 10 client.16971 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:03:06.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.183+0000 7f55cba60740 20 client.16971 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:06.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.183+0000 7f55cba60740 1 --2- 172.21.15.2:0/2643697764 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557a287b7750 0x557a28825d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.183+0000 7f55cba60740 4 client.16971 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:03:06.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.183+0000 7f55cba60740 1 -- 172.21.15.2:0/2643697764 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557a283c1700 con 0x557a287b7750 2024-03-15T13:03:06.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.183+0000 7f55c259c640 1 --2- 172.21.15.2:0/2643697764 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557a287b7750 0x557a28825d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.183+0000 7f55a9ffb640 10 client.16971 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:03:06.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.183+0000 7f55c259c640 1 --2- 172.21.15.2:0/2643697764 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557a287b7750 0x557a28825d00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2013db97ef1ca47e server_cookie=16c65c61169cb00d in_seq=0 out_seq=0 2024-03-15T13:03:06.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55a9ffb640 1 -- 172.21.15.2:0/2643697764 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557a283c1700 con 0x557a287b7750 2024-03-15T13:03:06.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55a9ffb640 10 client.16971 handle_command_reply: tid=0 2024-03-15T13:03:06.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55cba60740 1 client.16971 shutdown 2024-03-15T13:03:06.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55cba60740 1 -- 172.21.15.2:0/2643697764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287a8980 msgr2=0x557a287b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55cba60740 1 --2- 172.21.15.2:0/2643697764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287a8980 0x557a287b0dd0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f55ac001eb0 tx=0x7f55ac032040 comp rx=0 tx=0).stop 2024-03-15T13:03:06.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55cba60740 1 -- 172.21.15.2:0/2643697764 shutdown_connections 2024-03-15T13:03:06.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55cba60740 1 --2- 172.21.15.2:0/2643697764 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557a287b7750 0x557a28825d00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55cba60740 1 --2- 172.21.15.2:0/2643697764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a287aeb70 0x557a28822070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55cba60740 1 --2- 172.21.15.2:0/2643697764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a287ae250 0x557a287b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55a9ffb640 0 client.16971 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:03:06.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55cba60740 1 --2- 172.21.15.2:0/2643697764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557a287a8980 0x557a287b0dd0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55cba60740 1 -- 172.21.15.2:0/2643697764 >> 172.21.15.2:0/2643697764 conn(0x557a287b4a90 msgr2=0x557a287a7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55cba60740 1 -- 172.21.15.2:0/2643697764 shutdown_connections 2024-03-15T13:03:06.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55cba60740 1 -- 172.21.15.2:0/2643697764 wait complete. 2024-03-15T13:03:06.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55cba60740 20 client.16971 trim_cache size 0 max 16384 2024-03-15T13:03:06.176 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:03:06.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55c8e91640 1 -- 172.21.15.2:0/3996548773 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f55b00a1ba0 msgr2=0x7f55b00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55c8e91640 1 --2- 172.21.15.2:0/3996548773 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f55b00a1ba0 0x7f55b00a4060 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0cb9c0 tx=0x7f55bc06db50 comp rx=0 tx=0).stop 2024-03-15T13:03:06.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55c8e91640 1 -- 172.21.15.2:0/3996548773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c41064b0 msgr2=0x7f55c4197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55c8e91640 1 --2- 172.21.15.2:0/3996548773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c41064b0 0x7f55c4197250 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00e940 tx=0x7f55bc00ee00 comp rx=0 tx=0).stop 2024-03-15T13:03:06.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55c8e91640 1 -- 172.21.15.2:0/3996548773 shutdown_connections 2024-03-15T13:03:06.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55c8e91640 1 --2- 172.21.15.2:0/3996548773 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f55b00a1ba0 0x7f55b00a4060 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55c8e91640 1 --2- 172.21.15.2:0/3996548773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c41064b0 0x7f55c4197250 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55c8e91640 1 --2- 172.21.15.2:0/3996548773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55c4100dd0 0x7f55c4196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55c8e91640 1 --2- 172.21.15.2:0/3996548773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55c4100410 0x7f55c41967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.184+0000 7f55c8e91640 1 -- 172.21.15.2:0/3996548773 >> 172.21.15.2:0/3996548773 conn(0x7f55c4077550 msgr2=0x7f55c4103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.185+0000 7f55c8e91640 1 -- 172.21.15.2:0/3996548773 shutdown_connections 2024-03-15T13:03:06.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.185+0000 7f55c8e91640 1 -- 172.21.15.2:0/3996548773 wait complete. 2024-03-15T13:03:06.180 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:03:06.180 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:03:06.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.299+0000 7f2e9d22e640 1 -- 172.21.15.2:0/3743793581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e981074f0 msgr2=0x7f2e981078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.299+0000 7f2e9d22e640 1 --2- 172.21.15.2:0/3743793581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e981074f0 0x7f2e981078d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009890 tx=0x7f2e9002bb70 comp rx=0 tx=0).stop 2024-03-15T13:03:06.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.299+0000 7f2e9d22e640 1 -- 172.21.15.2:0/3743793581 shutdown_connections 2024-03-15T13:03:06.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.299+0000 7f2e9d22e640 1 --2- 172.21.15.2:0/3743793581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e980690b0 0x7f2e9810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.299+0000 7f2e9d22e640 1 --2- 172.21.15.2:0/3743793581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e980686f0 0x7f2e98068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.299+0000 7f2e9d22e640 1 --2- 172.21.15.2:0/3743793581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e981074f0 0x7f2e981078d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.299+0000 7f2e9d22e640 1 -- 172.21.15.2:0/3743793581 >> 172.21.15.2:0/3743793581 conn(0x7f2e980774f0 msgr2=0x7f2e98077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.299+0000 7f2e9d22e640 1 -- 172.21.15.2:0/3743793581 shutdown_connections 2024-03-15T13:03:06.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.299+0000 7f2e9d22e640 1 -- 172.21.15.2:0/3743793581 wait complete. 2024-03-15T13:03:06.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.300+0000 7f2e9d22e640 1 Processor -- start 2024-03-15T13:03:06.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.300+0000 7f2e9d22e640 1 -- start start 2024-03-15T13:03:06.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.300+0000 7f2e9d22e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e980686f0 0x7f2e9819fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.300+0000 7f2e9d22e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e980690b0 0x7f2e9819ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.300+0000 7f2e9d22e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e981074f0 0x7f2e98199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.300+0000 7f2e9d22e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9819a2c0 con 0x7f2e980690b0 2024-03-15T13:03:06.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.300+0000 7f2e9d22e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9819a430 con 0x7f2e980686f0 2024-03-15T13:03:06.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.300+0000 7f2e9d22e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9819a5f0 con 0x7f2e981074f0 2024-03-15T13:03:06.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e9659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e980690b0 0x7f2e9819ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e96d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e980686f0 0x7f2e9819fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e9659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e980690b0 0x7f2e9819ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40136/0 (socket says 172.21.15.2:40136) 2024-03-15T13:03:06.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e9659c640 1 -- 172.21.15.2:0/1286930309 learned_addr learned my addr 172.21.15.2:0/1286930309 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:06.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e9759e640 1 --2- 172.21.15.2:0/1286930309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e981074f0 0x7f2e98199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e96d9d640 1 -- 172.21.15.2:0/1286930309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e981074f0 msgr2=0x7f2e98199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e96d9d640 1 --2- 172.21.15.2:0/1286930309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e981074f0 0x7f2e98199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e96d9d640 1 -- 172.21.15.2:0/1286930309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e980690b0 msgr2=0x7f2e9819ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e96d9d640 1 --2- 172.21.15.2:0/1286930309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e980690b0 0x7f2e9819ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e96d9d640 1 -- 172.21.15.2:0/1286930309 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e88009900 con 0x7f2e980686f0 2024-03-15T13:03:06.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e9659c640 1 --2- 172.21.15.2:0/1286930309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e980690b0 0x7f2e9819ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:06.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e96d9d640 1 --2- 172.21.15.2:0/1286930309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e980686f0 0x7f2e9819fa70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2e9002c0b0 tx=0x7f2e900045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e8ffff640 1 -- 172.21.15.2:0/1286930309 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e900348d0 con 0x7f2e980686f0 2024-03-15T13:03:06.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e8ffff640 1 -- 172.21.15.2:0/1286930309 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2e9002c9e0 con 0x7f2e980686f0 2024-03-15T13:03:06.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e8ffff640 1 -- 172.21.15.2:0/1286930309 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90032600 con 0x7f2e980686f0 2024-03-15T13:03:06.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e9d22e640 1 -- 172.21.15.2:0/1286930309 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e90009490 con 0x7f2e980686f0 2024-03-15T13:03:06.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.301+0000 7f2e9d22e640 1 -- 172.21.15.2:0/1286930309 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9819a9d0 con 0x7f2e980686f0 2024-03-15T13:03:06.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.304+0000 7f2e8ffff640 1 -- 172.21.15.2:0/1286930309 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f2e90003680 con 0x7f2e980686f0 2024-03-15T13:03:06.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.304+0000 7f2e9fe11740 1 Processor -- start 2024-03-15T13:03:06.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.304+0000 7f2e9fe11740 1 -- start start 2024-03-15T13:03:06.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.304+0000 7f2e9fe11740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270250 0x558c21270630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.304+0000 7f2e9fe11740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21270b70 0x558c21279210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.304+0000 7f2e9fe11740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21279750 0x558c2127bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.304+0000 7f2e9fe11740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c2122b960 con 0x558c21270b70 2024-03-15T13:03:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.304+0000 7f2e9fe11740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c211df310 con 0x558c21279750 2024-03-15T13:03:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.304+0000 7f2e9fe11740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c211dc840 con 0x558c21270250 2024-03-15T13:03:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.304+0000 7f2e96d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21270b70 0x558c21279210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.304+0000 7f2e8ffff640 1 --2- 172.21.15.2:0/1286930309 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f2e840a1c20 0x7f2e840a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.304+0000 7f2e96d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21270b70 0x558c21279210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40138/0 (socket says 172.21.15.2:40138) 2024-03-15T13:03:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.304+0000 7f2e9659c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21279750 0x558c2127bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.304+0000 7f2e96d9d640 1 -- 172.21.15.2:0/1036864000 learned_addr learned my addr 172.21.15.2:0/1036864000 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.305+0000 7f2e9759e640 1 --2- 172.21.15.2:0/1036864000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270250 0x558c21270630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.305+0000 7f2e9759e640 1 --2- 172.21.15.2:0/1286930309 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f2e840a1c20 0x7f2e840a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.305+0000 7f2e8ffff640 1 -- 172.21.15.2:0/1286930309 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f2e900ef9c0 con 0x7f2e980686f0 2024-03-15T13:03:06.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.305+0000 7f2e9659c640 1 -- 172.21.15.2:0/1036864000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270250 msgr2=0x558c21270630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.305+0000 7f2e9659c640 1 --2- 172.21.15.2:0/1036864000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270250 0x558c21270630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.305+0000 7f2e9659c640 1 -- 172.21.15.2:0/1036864000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21270b70 msgr2=0x558c21279210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.305+0000 7f2e9659c640 1 --2- 172.21.15.2:0/1036864000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21270b70 0x558c21279210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.305+0000 7f2e9659c640 1 -- 172.21.15.2:0/1036864000 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c2119a060 con 0x558c21279750 2024-03-15T13:03:06.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.305+0000 7f2e9759e640 1 --2- 172.21.15.2:0/1286930309 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f2e840a1c20 0x7f2e840a40e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e88005fd0 tx=0x7f2e880095c0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.305+0000 7f2e9659c640 1 --2- 172.21.15.2:0/1036864000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21279750 0x558c2127bb40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e9819b310 tx=0x7f2e800119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61a35b47d0efa590 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.305+0000 7f2e8d7fa640 1 -- 172.21.15.2:0/1036864000 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000f5c0 con 0x558c21279750 2024-03-15T13:03:06.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.305+0000 7f2e8d7fa640 1 -- 172.21.15.2:0/1036864000 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2e8000fbf0 con 0x558c21279750 2024-03-15T13:03:06.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.305+0000 7f2e8d7fa640 1 -- 172.21.15.2:0/1036864000 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80019c40 con 0x558c21279750 2024-03-15T13:03:06.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.306+0000 7f2e9fe11740 1 -- 172.21.15.2:0/1036864000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21279750 msgr2=0x558c2127bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.306+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/1036864000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21279750 0x558c2127bb40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e9819b310 tx=0x7f2e800119a0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.306+0000 7f2e9fe11740 1 -- 172.21.15.2:0/1036864000 shutdown_connections 2024-03-15T13:03:06.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.306+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/1036864000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21279750 0x558c2127bb40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.306+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/1036864000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21270b70 0x558c21279210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.306+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/1036864000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270250 0x558c21270630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.306+0000 7f2e9fe11740 1 -- 172.21.15.2:0/1036864000 >> 172.21.15.2:0/1036864000 conn(0x558c2126a980 msgr2=0x558c2126ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.306+0000 7f2e9fe11740 1 -- 172.21.15.2:0/1036864000 shutdown_connections 2024-03-15T13:03:06.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.306+0000 7f2e9fe11740 1 -- 172.21.15.2:0/1036864000 wait complete. 2024-03-15T13:03:06.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.306+0000 7f2e9fe11740 1 -- start start 2024-03-15T13:03:06.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.306+0000 7f2e9fe11740 1 Processor -- start 2024-03-15T13:03:06.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9fe11740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21270250 0x558c21279110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9fe11740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270b70 0x558c212764a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9fe11740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21279750 0x558c21276ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9fe11740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c211d8020 con 0x558c21279750 2024-03-15T13:03:06.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9fe11740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c20eab430 con 0x558c21270250 2024-03-15T13:03:06.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9fe11740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c20e67e00 con 0x558c21270b70 2024-03-15T13:03:06.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9fe11740 10 client.0 _ll_register_callbacks cb 0x558c2117fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:06.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9759e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21279750 0x558c21276ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9759e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21279750 0x558c21276ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40154/0 (socket says 172.21.15.2:40154) 2024-03-15T13:03:06.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9759e640 1 -- 172.21.15.2:0/555766620 learned_addr learned my addr 172.21.15.2:0/555766620 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:06.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e96d9d640 1 --2- 172.21.15.2:0/555766620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21270250 0x558c21279110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9659c640 1 --2- 172.21.15.2:0/555766620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270b70 0x558c212764a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9759e640 1 -- 172.21.15.2:0/555766620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270b70 msgr2=0x558c212764a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9759e640 1 --2- 172.21.15.2:0/555766620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270b70 0x558c212764a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9759e640 1 -- 172.21.15.2:0/555766620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21270250 msgr2=0x558c21279110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9759e640 1 --2- 172.21.15.2:0/555766620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21270250 0x558c21279110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9759e640 1 -- 172.21.15.2:0/555766620 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e80009d10 con 0x558c21279750 2024-03-15T13:03:06.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9759e640 1 --2- 172.21.15.2:0/555766620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21279750 0x558c21276ab0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2e8801e6b0 tx=0x7f2e88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd66bd301285f51d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:06.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e7e7fc640 1 -- 172.21.15.2:0/555766620 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88023070 con 0x558c21279750 2024-03-15T13:03:06.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e7e7fc640 1 -- 172.21.15.2:0/555766620 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f2e88004b20 con 0x558c21279750 2024-03-15T13:03:06.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.307+0000 7f2e9fe11740 1 -- 172.21.15.2:0/555766620 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c20ed77e0 con 0x558c21279750 2024-03-15T13:03:06.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.308+0000 7f2e7e7fc640 1 -- 172.21.15.2:0/555766620 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e88013070 con 0x558c21279750 2024-03-15T13:03:06.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.308+0000 7f2e9fe11740 10 client.16983 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:06.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.308+0000 7f2e9fe11740 1 -- 172.21.15.2:0/555766620 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x558c211d66a0 con 0x558c21279750 2024-03-15T13:03:06.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.308+0000 7f2e7e7fc640 1 -- 172.21.15.2:0/555766620 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x558c20ed77e0 con 0x558c21279750 2024-03-15T13:03:06.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.308+0000 7f2e7e7fc640 1 -- 172.21.15.2:0/555766620 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7f2e88004390 con 0x558c21279750 2024-03-15T13:03:06.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.308+0000 7f2e9fe11740 10 client.16983 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:06.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.308+0000 7f2e9fe11740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:06.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.308+0000 7f2e9fe11740 20 client.16983 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:06.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.308+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/555766620 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558c21289920 0x558c2128bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.308+0000 7f2e9fe11740 4 client.16983 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:06.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.308+0000 7f2e9fe11740 1 -- 172.21.15.2:0/555766620 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c20f14720 con 0x558c21289920 2024-03-15T13:03:06.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.308+0000 7f2e96d9d640 1 --2- 172.21.15.2:0/555766620 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558c21289920 0x558c2128bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e96d9d640 1 --2- 172.21.15.2:0/555766620 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558c21289920 0x558c2128bd40 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4626973e068dc07a server_cookie=b6581c20cbdfccc5 in_seq=0 out_seq=0 2024-03-15T13:03:06.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e7e7fc640 10 client.16983 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:06.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e7e7fc640 1 -- 172.21.15.2:0/555766620 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558c20f14720 con 0x558c21289920 2024-03-15T13:03:06.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e7e7fc640 10 client.16983 handle_command_reply: tid=0 2024-03-15T13:03:06.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e9fe11740 1 client.16983 shutdown 2024-03-15T13:03:06.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e9fe11740 1 -- 172.21.15.2:0/555766620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21279750 msgr2=0x558c21276ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/555766620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21279750 0x558c21276ab0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2e8801e6b0 tx=0x7f2e88004a40 comp rx=0 tx=0).stop 2024-03-15T13:03:06.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e9fe11740 1 -- 172.21.15.2:0/555766620 shutdown_connections 2024-03-15T13:03:06.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/555766620 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558c21289920 0x558c2128bd40 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/555766620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21279750 0x558c21276ab0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e7e7fc640 0 client.16983 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:06.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/555766620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270b70 0x558c212764a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/555766620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21270250 0x558c21279110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e9fe11740 1 -- 172.21.15.2:0/555766620 >> 172.21.15.2:0/555766620 conn(0x558c2126a980 msgr2=0x558c2126c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e9fe11740 1 -- 172.21.15.2:0/555766620 shutdown_connections 2024-03-15T13:03:06.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.309+0000 7f2e9fe11740 1 -- 172.21.15.2:0/555766620 wait complete. 2024-03-15T13:03:06.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.310+0000 7f2e9fe11740 20 client.16983 trim_cache size 0 max 16384 2024-03-15T13:03:06.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.322+0000 7f2e9fe11740 1 Processor -- start 2024-03-15T13:03:06.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.322+0000 7f2e9fe11740 1 -- start start 2024-03-15T13:03:06.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.322+0000 7f2e9fe11740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21270250 0x558c212692f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.322+0000 7f2e9fe11740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270b70 0x558c21278bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.322+0000 7f2e9fe11740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21279750 0x558c21279110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.322+0000 7f2e9fe11740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c2115d130 con 0x558c21279750 2024-03-15T13:03:06.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.322+0000 7f2e9fe11740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c2126b980 con 0x558c21270250 2024-03-15T13:03:06.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.322+0000 7f2e9fe11740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c21269830 con 0x558c21270b70 2024-03-15T13:03:06.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.322+0000 7f2e9759e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21279750 0x558c21279110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.322+0000 7f2e9759e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21279750 0x558c21279110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40166/0 (socket says 172.21.15.2:40166) 2024-03-15T13:03:06.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.322+0000 7f2e9759e640 1 -- 172.21.15.2:0/3688870879 learned_addr learned my addr 172.21.15.2:0/3688870879 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:06.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.322+0000 7f2e96d9d640 1 --2- 172.21.15.2:0/3688870879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21270250 0x558c212692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.322+0000 7f2e9659c640 1 --2- 172.21.15.2:0/3688870879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270b70 0x558c21278bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.323+0000 7f2e9759e640 1 -- 172.21.15.2:0/3688870879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270b70 msgr2=0x558c21278bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.323+0000 7f2e9759e640 1 --2- 172.21.15.2:0/3688870879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270b70 0x558c21278bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.323+0000 7f2e9759e640 1 -- 172.21.15.2:0/3688870879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21270250 msgr2=0x558c212692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.323+0000 7f2e9759e640 1 --2- 172.21.15.2:0/3688870879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21270250 0x558c212692f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.323+0000 7f2e9759e640 1 -- 172.21.15.2:0/3688870879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c21172c20 con 0x558c21279750 2024-03-15T13:03:06.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.323+0000 7f2e9759e640 1 --2- 172.21.15.2:0/3688870879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21279750 0x558c21279110 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800cc70 tx=0x7f2e88009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1eb56c55b7aa7f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.323+0000 7f2e8cff9640 1 -- 172.21.15.2:0/3688870879 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88013070 con 0x558c21279750 2024-03-15T13:03:06.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.323+0000 7f2e8cff9640 1 -- 172.21.15.2:0/3688870879 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2e88014040 con 0x558c21279750 2024-03-15T13:03:06.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.323+0000 7f2e8cff9640 1 -- 172.21.15.2:0/3688870879 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88023040 con 0x558c21279750 2024-03-15T13:03:06.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 -- 172.21.15.2:0/3688870879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21279750 msgr2=0x558c21279110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/3688870879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21279750 0x558c21279110 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800cc70 tx=0x7f2e88009cc0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 -- 172.21.15.2:0/3688870879 shutdown_connections 2024-03-15T13:03:06.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/3688870879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21279750 0x558c21279110 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/3688870879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270b70 0x558c21278bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/3688870879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21270250 0x558c212692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 -- 172.21.15.2:0/3688870879 >> 172.21.15.2:0/3688870879 conn(0x558c2126a980 msgr2=0x558c212765c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 -- 172.21.15.2:0/3688870879 shutdown_connections 2024-03-15T13:03:06.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 -- 172.21.15.2:0/3688870879 wait complete. 2024-03-15T13:03:06.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 -- start start 2024-03-15T13:03:06.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 Processor -- start 2024-03-15T13:03:06.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270250 0x558c2127d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21270b70 0x558c2127d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21279750 0x558c212de250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c212de790 con 0x558c21270b70 2024-03-15T13:03:06.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c212de900 con 0x558c21279750 2024-03-15T13:03:06.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c212deac0 con 0x558c21270250 2024-03-15T13:03:06.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.324+0000 7f2e9fe11740 10 client.0 _ll_register_callbacks cb 0x7f2e8800e260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:06.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e9659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21270b70 0x558c2127d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e96d9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270250 0x558c2127d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e9659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21270b70 0x558c2127d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40176/0 (socket says 172.21.15.2:40176) 2024-03-15T13:03:06.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e9659c640 1 -- 172.21.15.2:0/3808418758 learned_addr learned my addr 172.21.15.2:0/3808418758 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:06.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e9759e640 1 --2- 172.21.15.2:0/3808418758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21279750 0x558c212de250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e9659c640 1 -- 172.21.15.2:0/3808418758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270250 msgr2=0x558c2127d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e9659c640 1 --2- 172.21.15.2:0/3808418758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270250 0x558c2127d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e9659c640 1 -- 172.21.15.2:0/3808418758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21279750 msgr2=0x558c212de250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:06.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e9659c640 1 --2- 172.21.15.2:0/3808418758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21279750 0x558c212de250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e9659c640 1 -- 172.21.15.2:0/3808418758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c211d66a0 con 0x558c21270b70 2024-03-15T13:03:06.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e9659c640 1 --2- 172.21.15.2:0/3808418758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21270b70 0x558c2127d7c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2e800198d0 tx=0x7f2e8000ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9114bbdc0c2241f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:06.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:06.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e7e7fc640 1 -- 172.21.15.2:0/3808418758 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000f7d0 con 0x558c21270b70 2024-03-15T13:03:06.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e7e7fc640 1 -- 172.21.15.2:0/3808418758 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f2e8000f930 con 0x558c21270b70 2024-03-15T13:03:06.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e9fe11740 1 -- 172.21.15.2:0/3808418758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c212660f0 con 0x558c21270b70 2024-03-15T13:03:06.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e7e7fc640 1 -- 172.21.15.2:0/3808418758 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e80019970 con 0x558c21270b70 2024-03-15T13:03:06.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e9fe11740 10 client.16989 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:06.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e9fe11740 1 -- 172.21.15.2:0/3808418758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x558c211988c0 con 0x558c21270b70 2024-03-15T13:03:06.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.325+0000 7f2e7e7fc640 1 -- 172.21.15.2:0/3808418758 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x558c212660f0 con 0x558c21270b70 2024-03-15T13:03:06.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.326+0000 7f2e7e7fc640 1 -- 172.21.15.2:0/3808418758 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7f2e8002c020 con 0x558c21270b70 2024-03-15T13:03:06.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.326+0000 7f2e9fe11740 10 client.16989 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:06.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.326+0000 7f2e9fe11740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:06.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.326+0000 7f2e9fe11740 20 client.16989 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:06.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.326+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/3808418758 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558c212e63c0 0x558c212e87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:06.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.326+0000 7f2e9fe11740 4 client.16989 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:03:06.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.326+0000 7f2e9fe11740 1 -- 172.21.15.2:0/3808418758 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f2e8000fbf0 con 0x558c212e63c0 2024-03-15T13:03:06.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.326+0000 7f2e96d9d640 1 --2- 172.21.15.2:0/3808418758 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558c212e63c0 0x558c212e87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:06.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.326+0000 7f2e96d9d640 1 --2- 172.21.15.2:0/3808418758 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558c212e63c0 0x558c212e87e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45cedcca08c87e98 server_cookie=4bd374ab1fd48ece in_seq=0 out_seq=0 2024-03-15T13:03:06.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.326+0000 7f2e7e7fc640 10 client.16989 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:06.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.326+0000 7f2e7e7fc640 1 -- 172.21.15.2:0/3808418758 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f2e8000fbf0 con 0x558c212e63c0 2024-03-15T13:03:06.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.326+0000 7f2e7e7fc640 10 client.16989 handle_command_reply: tid=0 2024-03-15T13:03:06.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.326+0000 7f2e9fe11740 1 client.16989 shutdown 2024-03-15T13:03:06.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.326+0000 7f2e9fe11740 1 -- 172.21.15.2:0/3808418758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21270b70 msgr2=0x558c2127d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.326+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/3808418758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21270b70 0x558c2127d7c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2e800198d0 tx=0x7f2e8000ff90 comp rx=0 tx=0).stop 2024-03-15T13:03:06.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9fe11740 1 -- 172.21.15.2:0/3808418758 shutdown_connections 2024-03-15T13:03:06.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/3808418758 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558c212e63c0 0x558c212e87e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/3808418758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558c21279750 0x558c212de250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e7e7fc640 0 client.16989 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:06.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/3808418758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558c21270b70 0x558c2127d7c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9fe11740 1 --2- 172.21.15.2:0/3808418758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558c21270250 0x558c2127d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9fe11740 1 -- 172.21.15.2:0/3808418758 >> 172.21.15.2:0/3808418758 conn(0x558c2126a980 msgr2=0x558c21270f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9fe11740 1 -- 172.21.15.2:0/3808418758 shutdown_connections 2024-03-15T13:03:06.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9fe11740 1 -- 172.21.15.2:0/3808418758 wait complete. 2024-03-15T13:03:06.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9fe11740 20 client.16989 trim_cache size 0 max 16384 2024-03-15T13:03:06.319 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:03:06.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9d22e640 1 -- 172.21.15.2:0/1286930309 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f2e840a1c20 msgr2=0x7f2e840a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9d22e640 1 --2- 172.21.15.2:0/1286930309 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f2e840a1c20 0x7f2e840a40e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e88005fd0 tx=0x7f2e880095c0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9d22e640 1 -- 172.21.15.2:0/1286930309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e980686f0 msgr2=0x7f2e9819fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:06.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9d22e640 1 --2- 172.21.15.2:0/1286930309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e980686f0 0x7f2e9819fa70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2e9002c0b0 tx=0x7f2e900045b0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9d22e640 1 -- 172.21.15.2:0/1286930309 shutdown_connections 2024-03-15T13:03:06.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9d22e640 1 --2- 172.21.15.2:0/1286930309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e981074f0 0x7f2e98199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9d22e640 1 --2- 172.21.15.2:0/1286930309 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f2e840a1c20 0x7f2e840a40e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9d22e640 1 --2- 172.21.15.2:0/1286930309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e980690b0 0x7f2e9819ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9d22e640 1 --2- 172.21.15.2:0/1286930309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e980686f0 0x7f2e9819fa70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:06.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9d22e640 1 -- 172.21.15.2:0/1286930309 >> 172.21.15.2:0/1286930309 conn(0x7f2e980774f0 msgr2=0x7f2e9810c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:06.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9d22e640 1 -- 172.21.15.2:0/1286930309 shutdown_connections 2024-03-15T13:03:06.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:06.327+0000 7f2e9d22e640 1 -- 172.21.15.2:0/1286930309 wait complete. 2024-03-15T13:03:06.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:06 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1278632835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:03:06.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:06 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1278632835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:03:06.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:06 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1278632835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:03:07.324 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:03:07.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:03:07.324 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:03:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.458+0000 7f60b473e640 1 -- 172.21.15.2:0/1484838054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac100dd0 msgr2=0x7f60ac10d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.458+0000 7f60b473e640 1 --2- 172.21.15.2:0/1484838054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac100dd0 0x7f60ac10d2e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009be0 tx=0x7f60a402bef0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.458+0000 7f60b473e640 1 -- 172.21.15.2:0/1484838054 shutdown_connections 2024-03-15T13:03:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.458+0000 7f60b473e640 1 --2- 172.21.15.2:0/1484838054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac100dd0 0x7f60ac10d2e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.458+0000 7f60b473e640 1 --2- 172.21.15.2:0/1484838054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ac100410 0x7f60ac100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.458+0000 7f60b473e640 1 --2- 172.21.15.2:0/1484838054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60ac1064b0 0x7f60ac106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.458+0000 7f60b473e640 1 -- 172.21.15.2:0/1484838054 >> 172.21.15.2:0/1484838054 conn(0x7f60ac077550 msgr2=0x7f60ac077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.458+0000 7f60b473e640 1 -- 172.21.15.2:0/1484838054 shutdown_connections 2024-03-15T13:03:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.458+0000 7f60b473e640 1 -- 172.21.15.2:0/1484838054 wait complete. 2024-03-15T13:03:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.459+0000 7f60b473e640 1 Processor -- start 2024-03-15T13:03:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.459+0000 7f60b473e640 1 -- start start 2024-03-15T13:03:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.459+0000 7f60b473e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ac100410 0x7f60ac199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.459+0000 7f60b473e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac100dd0 0x7f60ac19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.459+0000 7f60b473e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60ac1064b0 0x7f60ac19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.459+0000 7f60b473e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac19ec00 con 0x7f60ac1064b0 2024-03-15T13:03:07.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.459+0000 7f60b473e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac19ed70 con 0x7f60ac100dd0 2024-03-15T13:03:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.459+0000 7f60b473e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac19ef30 con 0x7f60ac100410 2024-03-15T13:03:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.459+0000 7f60b24da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ac100410 0x7f60ac199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.459+0000 7f60b1cd9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac100dd0 0x7f60ac19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.459+0000 7f60b2cdb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60ac1064b0 0x7f60ac19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.459+0000 7f60b1cd9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac100dd0 0x7f60ac19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47948/0 (socket says 172.21.15.2:47948) 2024-03-15T13:03:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.459+0000 7f60b24da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ac100410 0x7f60ac199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:44040/0 (socket says 172.21.15.2:44040) 2024-03-15T13:03:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.459+0000 7f60b1cd9640 1 -- 172.21.15.2:0/1859798821 learned_addr learned my addr 172.21.15.2:0/1859798821 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.460+0000 7f60b1cd9640 1 -- 172.21.15.2:0/1859798821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ac100410 msgr2=0x7f60ac199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.460+0000 7f60b1cd9640 1 --2- 172.21.15.2:0/1859798821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ac100410 0x7f60ac199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.460+0000 7f60b1cd9640 1 -- 172.21.15.2:0/1859798821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60ac1064b0 msgr2=0x7f60ac19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.460+0000 7f60b1cd9640 1 --2- 172.21.15.2:0/1859798821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60ac1064b0 0x7f60ac19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.460+0000 7f60b1cd9640 1 -- 172.21.15.2:0/1859798821 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a4009840 con 0x7f60ac100dd0 2024-03-15T13:03:07.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.460+0000 7f60b24da640 1 --2- 172.21.15.2:0/1859798821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ac100410 0x7f60ac199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:07.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.460+0000 7f60b1cd9640 1 --2- 172.21.15.2:0/1859798821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac100dd0 0x7f60ac19a280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f609c00e9f0 tx=0x7f609c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.464+0000 7f60a37fe640 1 -- 172.21.15.2:0/1859798821 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c009700 con 0x7f60ac100dd0 2024-03-15T13:03:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.465+0000 7f60a37fe640 1 -- 172.21.15.2:0/1859798821 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f609c00ccc0 con 0x7f60ac100dd0 2024-03-15T13:03:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.465+0000 7f60a37fe640 1 -- 172.21.15.2:0/1859798821 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c018820 con 0x7f60ac100dd0 2024-03-15T13:03:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.465+0000 7f60b473e640 1 -- 172.21.15.2:0/1859798821 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ac19f210 con 0x7f60ac100dd0 2024-03-15T13:03:07.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.465+0000 7f60b473e640 1 -- 172.21.15.2:0/1859798821 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac1a6a90 con 0x7f60ac100dd0 2024-03-15T13:03:07.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.467+0000 7f60a37fe640 1 -- 172.21.15.2:0/1859798821 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f609c010430 con 0x7f60ac100dd0 2024-03-15T13:03:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.467+0000 7f60b73f2740 1 Processor -- start 2024-03-15T13:03:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.467+0000 7f60b73f2740 1 -- start start 2024-03-15T13:03:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.467+0000 7f60a37fe640 1 --2- 172.21.15.2:0/1859798821 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f60980a1c20 0x7f60980a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.467+0000 7f60b73f2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382380 0x5578e2382760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.467+0000 7f60b73f2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e2382d30 0x5578e238b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b73f2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e238b8b0 0x5578e238dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b73f2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e22f0f90 con 0x5578e238b8b0 2024-03-15T13:03:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b73f2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e22ee4c0 con 0x5578e2382380 2024-03-15T13:03:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b73f2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e22e9ca0 con 0x5578e2382d30 2024-03-15T13:03:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b24da640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382380 0x5578e2382760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b2cdb640 1 --2- 172.21.15.2:0/1859798821 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f60980a1c20 0x7f60980a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b24da640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382380 0x5578e2382760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:47954/0 (socket says 172.21.15.2:47954) 2024-03-15T13:03:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b1cd9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e2382d30 0x5578e238b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b24da640 1 -- 172.21.15.2:0/1328603129 learned_addr learned my addr 172.21.15.2:0/1328603129 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60a37fe640 1 -- 172.21.15.2:0/1859798821 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f609c014070 con 0x7f60ac100dd0 2024-03-15T13:03:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b2cdb640 1 --2- 172.21.15.2:0/1328603129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e238b8b0 0x5578e238dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b1cd9640 1 -- 172.21.15.2:0/1328603129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382380 msgr2=0x5578e2382760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b1cd9640 1 --2- 172.21.15.2:0/1328603129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382380 0x5578e2382760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b1cd9640 1 -- 172.21.15.2:0/1328603129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e238b8b0 msgr2=0x5578e238dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b1cd9640 1 --2- 172.21.15.2:0/1328603129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e238b8b0 0x5578e238dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b1cd9640 1 -- 172.21.15.2:0/1328603129 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578e22ac0b0 con 0x5578e2382d30 2024-03-15T13:03:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b24da640 1 --2- 172.21.15.2:0/1328603129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382380 0x5578e2382760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b2cdb640 1 --2- 172.21.15.2:0/1859798821 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f60980a1c20 0x7f60980a40e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f60a4004af0 tx=0x7f60a402c960 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60b1cd9640 1 --2- 172.21.15.2:0/1328603129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e2382d30 0x5578e238b2e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f609c0cb310 tx=0x7f609c09a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a71f57f55934d43 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.468+0000 7f60a0ff9640 1 -- 172.21.15.2:0/1328603129 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c09aef0 con 0x5578e2382d30 2024-03-15T13:03:07.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.469+0000 7f60a0ff9640 1 -- 172.21.15.2:0/1328603129 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f609c0acc30 con 0x5578e2382d30 2024-03-15T13:03:07.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.469+0000 7f60a0ff9640 1 -- 172.21.15.2:0/1328603129 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c096e60 con 0x5578e2382d30 2024-03-15T13:03:07.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.469+0000 7f60b73f2740 1 -- 172.21.15.2:0/1328603129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e2382d30 msgr2=0x5578e238b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.469+0000 7f60b73f2740 1 --2- 172.21.15.2:0/1328603129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e2382d30 0x5578e238b2e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f609c0cb310 tx=0x7f609c09a3f0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.469+0000 7f60b73f2740 1 -- 172.21.15.2:0/1328603129 shutdown_connections 2024-03-15T13:03:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.469+0000 7f60b73f2740 1 --2- 172.21.15.2:0/1328603129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e238b8b0 0x5578e238dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.469+0000 7f60b73f2740 1 --2- 172.21.15.2:0/1328603129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e2382d30 0x5578e238b2e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.469+0000 7f60b73f2740 1 --2- 172.21.15.2:0/1328603129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382380 0x5578e2382760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.469+0000 7f60b73f2740 1 -- 172.21.15.2:0/1328603129 >> 172.21.15.2:0/1328603129 conn(0x5578e237c840 msgr2=0x5578e237cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.469+0000 7f60b73f2740 1 -- 172.21.15.2:0/1328603129 shutdown_connections 2024-03-15T13:03:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.469+0000 7f60b73f2740 1 -- 172.21.15.2:0/1328603129 wait complete. 2024-03-15T13:03:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b73f2740 1 -- start start 2024-03-15T13:03:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b73f2740 1 Processor -- start 2024-03-15T13:03:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b73f2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382380 0x5578e2388860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b73f2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e2382d30 0x5578e2388da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b73f2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e238b8b0 0x5578e23892e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b73f2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e228ef40 con 0x5578e2382380 2024-03-15T13:03:07.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b73f2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e1fbd4b0 con 0x5578e238b8b0 2024-03-15T13:03:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b73f2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e226f670 con 0x5578e2382d30 2024-03-15T13:03:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b73f2740 10 client.0 _ll_register_callbacks cb 0x5578e22dcf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b24da640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382380 0x5578e2388860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b24da640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382380 0x5578e2388860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40212/0 (socket says 172.21.15.2:40212) 2024-03-15T13:03:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b24da640 1 -- 172.21.15.2:0/1691529117 learned_addr learned my addr 172.21.15.2:0/1691529117 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b1cd9640 1 --2- 172.21.15.2:0/1691529117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e2382d30 0x5578e2388da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b2cdb640 1 --2- 172.21.15.2:0/1691529117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e238b8b0 0x5578e23892e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b24da640 1 -- 172.21.15.2:0/1691529117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e2382d30 msgr2=0x5578e2388da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b24da640 1 --2- 172.21.15.2:0/1691529117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e2382d30 0x5578e2388da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b24da640 1 -- 172.21.15.2:0/1691529117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e238b8b0 msgr2=0x5578e23892e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b24da640 1 --2- 172.21.15.2:0/1691529117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e238b8b0 0x5578e23892e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.470+0000 7f60b24da640 1 -- 172.21.15.2:0/1691529117 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac1a6a90 con 0x5578e2382380 2024-03-15T13:03:07.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f60b24da640 1 --2- 172.21.15.2:0/1691529117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382380 0x5578e2388860 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f60a8011d40 tx=0x7f60a800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933540e0cd9a4695 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f6095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f6095ffb640 1 -- 172.21.15.2:0/1691529117 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8016070 con 0x5578e2382380 2024-03-15T13:03:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f6095ffb640 1 -- 172.21.15.2:0/1691529117 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f60a801a3f0 con 0x5578e2382380 2024-03-15T13:03:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f60b73f2740 1 -- 172.21.15.2:0/1691529117 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578e22c3f40 con 0x5578e2382380 2024-03-15T13:03:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f6095ffb640 1 -- 172.21.15.2:0/1691529117 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60a801a5b0 con 0x5578e2382380 2024-03-15T13:03:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f60b73f2740 10 client.17001 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f60b73f2740 1 -- 172.21.15.2:0/1691529117 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5578e22e8320 con 0x5578e2382380 2024-03-15T13:03:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f6095ffb640 1 -- 172.21.15.2:0/1691529117 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5578e22c3f40 con 0x5578e2382380 2024-03-15T13:03:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f6095ffb640 1 -- 172.21.15.2:0/1691529117 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7f60a8019720 con 0x5578e2382380 2024-03-15T13:03:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f60b73f2740 10 client.17001 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f60b73f2740 10 client.17001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f60b73f2740 20 client.17001 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f60b73f2740 1 --2- 172.21.15.2:0/1691529117 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5578e239c0b0 0x5578e239e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f60b73f2740 4 client.17001 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.471+0000 7f60b73f2740 1 -- 172.21.15.2:0/1691529117 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578e22e0310 con 0x5578e239c0b0 2024-03-15T13:03:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.472+0000 7f60b1cd9640 1 --2- 172.21.15.2:0/1691529117 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5578e239c0b0 0x5578e239e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.472+0000 7f6095ffb640 10 client.17001 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.472+0000 7f60b1cd9640 1 --2- 172.21.15.2:0/1691529117 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5578e239c0b0 0x5578e239e4d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f16bf99809f2b7b6 server_cookie=d99888d5f4207f8c in_seq=0 out_seq=0 2024-03-15T13:03:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.473+0000 7f6095ffb640 1 -- 172.21.15.2:0/1691529117 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5578e22e0310 con 0x5578e239c0b0 2024-03-15T13:03:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.473+0000 7f6095ffb640 10 client.17001 handle_command_reply: tid=0 2024-03-15T13:03:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.473+0000 7f60b73f2740 1 client.17001 shutdown 2024-03-15T13:03:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.473+0000 7f60b73f2740 1 -- 172.21.15.2:0/1691529117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382380 msgr2=0x5578e2388860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.473+0000 7f60b73f2740 1 --2- 172.21.15.2:0/1691529117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382380 0x5578e2388860 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f60a8011d40 tx=0x7f60a800f790 comp rx=0 tx=0).stop 2024-03-15T13:03:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.473+0000 7f60b73f2740 1 -- 172.21.15.2:0/1691529117 shutdown_connections 2024-03-15T13:03:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.473+0000 7f60b73f2740 1 --2- 172.21.15.2:0/1691529117 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5578e239c0b0 0x5578e239e4d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.473+0000 7f60b73f2740 1 --2- 172.21.15.2:0/1691529117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e238b8b0 0x5578e23892e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.473+0000 7f60b73f2740 1 --2- 172.21.15.2:0/1691529117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e2382d30 0x5578e2388da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.473+0000 7f6095ffb640 0 client.17001 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:07.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.473+0000 7f60b73f2740 1 --2- 172.21.15.2:0/1691529117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382380 0x5578e2388860 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.473+0000 7f60b73f2740 1 -- 172.21.15.2:0/1691529117 >> 172.21.15.2:0/1691529117 conn(0x5578e237c840 msgr2=0x5578e237e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:07.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.473+0000 7f60b73f2740 1 -- 172.21.15.2:0/1691529117 shutdown_connections 2024-03-15T13:03:07.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.473+0000 7f60b73f2740 1 -- 172.21.15.2:0/1691529117 wait complete. 2024-03-15T13:03:07.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.473+0000 7f60b73f2740 20 client.17001 trim_cache size 0 max 16384 2024-03-15T13:03:07.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b73f2740 1 Processor -- start 2024-03-15T13:03:07.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b73f2740 1 -- start start 2024-03-15T13:03:07.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b73f2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382380 0x5578e238af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b73f2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382d30 0x5578e2389100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b73f2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e238b8b0 0x5578e2396e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b73f2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e23989c0 con 0x5578e2382d30 2024-03-15T13:03:07.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b73f2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e239a700 con 0x5578e2382380 2024-03-15T13:03:07.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b73f2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e2383670 con 0x5578e238b8b0 2024-03-15T13:03:07.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b1cd9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382d30 0x5578e2389100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b1cd9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382d30 0x5578e2389100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40218/0 (socket says 172.21.15.2:40218) 2024-03-15T13:03:07.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b1cd9640 1 -- 172.21.15.2:0/1008742959 learned_addr learned my addr 172.21.15.2:0/1008742959 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:07.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b24da640 1 --2- 172.21.15.2:0/1008742959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382380 0x5578e238af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b2cdb640 1 --2- 172.21.15.2:0/1008742959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e238b8b0 0x5578e2396e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b1cd9640 1 -- 172.21.15.2:0/1008742959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e238b8b0 msgr2=0x5578e2396e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b1cd9640 1 --2- 172.21.15.2:0/1008742959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e238b8b0 0x5578e2396e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b1cd9640 1 -- 172.21.15.2:0/1008742959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382380 msgr2=0x5578e238af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b1cd9640 1 --2- 172.21.15.2:0/1008742959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382380 0x5578e238af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b1cd9640 1 -- 172.21.15.2:0/1008742959 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578e2284c00 con 0x5578e2382d30 2024-03-15T13:03:07.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b2cdb640 1 --2- 172.21.15.2:0/1008742959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e238b8b0 0x5578e2396e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:03:07.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f60b1cd9640 1 --2- 172.21.15.2:0/1008742959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382d30 0x5578e2389100 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f609c00e9c0 tx=0x7f609c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9778ae570a1b5110 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:07.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.490+0000 7f6097fff640 1 -- 172.21.15.2:0/1008742959 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c09e940 con 0x5578e2382d30 2024-03-15T13:03:07.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.491+0000 7f6097fff640 1 -- 172.21.15.2:0/1008742959 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f609c09a7f0 con 0x5578e2382d30 2024-03-15T13:03:07.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.491+0000 7f6097fff640 1 -- 172.21.15.2:0/1008742959 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c0cb590 con 0x5578e2382d30 2024-03-15T13:03:07.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.491+0000 7f60b73f2740 1 -- 172.21.15.2:0/1008742959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382d30 msgr2=0x5578e2389100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.491+0000 7f60b73f2740 1 --2- 172.21.15.2:0/1008742959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382d30 0x5578e2389100 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f609c00e9c0 tx=0x7f609c00ce30 comp rx=0 tx=0).stop 2024-03-15T13:03:07.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.491+0000 7f60b73f2740 1 -- 172.21.15.2:0/1008742959 shutdown_connections 2024-03-15T13:03:07.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.491+0000 7f60b73f2740 1 --2- 172.21.15.2:0/1008742959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e238b8b0 0x5578e2396e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.491+0000 7f60b73f2740 1 --2- 172.21.15.2:0/1008742959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382d30 0x5578e2389100 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.491+0000 7f60b73f2740 1 --2- 172.21.15.2:0/1008742959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382380 0x5578e238af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.491+0000 7f60b73f2740 1 -- 172.21.15.2:0/1008742959 >> 172.21.15.2:0/1008742959 conn(0x5578e237c840 msgr2=0x5578e237b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:07.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.491+0000 7f60b73f2740 1 -- 172.21.15.2:0/1008742959 shutdown_connections 2024-03-15T13:03:07.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.491+0000 7f60b73f2740 1 -- 172.21.15.2:0/1008742959 wait complete. 2024-03-15T13:03:07.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.491+0000 7f60b73f2740 1 -- start start 2024-03-15T13:03:07.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.491+0000 7f60b73f2740 1 Processor -- start 2024-03-15T13:03:07.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b73f2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382380 0x5578e2385520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b73f2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382d30 0x5578e2385a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b73f2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e238b8b0 0x5578e23f6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b73f2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e2386180 con 0x5578e2382380 2024-03-15T13:03:07.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b73f2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e23f6e60 con 0x5578e2382d30 2024-03-15T13:03:07.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b73f2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e23f6fa0 con 0x5578e238b8b0 2024-03-15T13:03:07.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b73f2740 10 client.0 _ll_register_callbacks cb 0x7f60a8009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:07.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b24da640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382380 0x5578e2385520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b24da640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382380 0x5578e2385520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40220/0 (socket says 172.21.15.2:40220) 2024-03-15T13:03:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b24da640 1 -- 172.21.15.2:0/29064611 learned_addr learned my addr 172.21.15.2:0/29064611 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b1cd9640 1 --2- 172.21.15.2:0/29064611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382d30 0x5578e2385a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b2cdb640 1 --2- 172.21.15.2:0/29064611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e238b8b0 0x5578e23f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b24da640 1 -- 172.21.15.2:0/29064611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e238b8b0 msgr2=0x5578e23f6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b24da640 1 --2- 172.21.15.2:0/29064611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e238b8b0 0x5578e23f6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b24da640 1 -- 172.21.15.2:0/29064611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382d30 msgr2=0x5578e2385a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b24da640 1 --2- 172.21.15.2:0/29064611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382d30 0x5578e2385a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b24da640 1 -- 172.21.15.2:0/29064611 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578e22ac0b0 con 0x5578e2382380 2024-03-15T13:03:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b24da640 1 --2- 172.21.15.2:0/29064611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382380 0x5578e2385520 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f60a8011a10 tx=0x7f60a8009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebd9322451a9407 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f6095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f6095ffb640 1 -- 172.21.15.2:0/29064611 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8016040 con 0x5578e2382380 2024-03-15T13:03:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f6095ffb640 1 -- 172.21.15.2:0/29064611 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f60a8000e40 con 0x5578e2382380 2024-03-15T13:03:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.492+0000 7f60b73f2740 1 -- 172.21.15.2:0/29064611 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578e230f900 con 0x5578e2382380 2024-03-15T13:03:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.493+0000 7f6095ffb640 1 -- 172.21.15.2:0/29064611 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60a8019590 con 0x5578e2382380 2024-03-15T13:03:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.493+0000 7f60b73f2740 10 client.17007 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.493+0000 7f60b73f2740 1 -- 172.21.15.2:0/29064611 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5578e22aa910 con 0x5578e2382380 2024-03-15T13:03:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.493+0000 7f6095ffb640 1 -- 172.21.15.2:0/29064611 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5578e230f900 con 0x5578e2382380 2024-03-15T13:03:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.493+0000 7f6095ffb640 1 -- 172.21.15.2:0/29064611 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7f60a8032060 con 0x5578e2382380 2024-03-15T13:03:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.493+0000 7f60b73f2740 10 client.17007 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.493+0000 7f60b73f2740 10 client.17007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.493+0000 7f60b73f2740 20 client.17007 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.493+0000 7f60b73f2740 1 --2- 172.21.15.2:0/29064611 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5578e23f88e0 0x5578e23fad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.493+0000 7f60b73f2740 4 client.17007 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:03:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.493+0000 7f60b73f2740 1 -- 172.21.15.2:0/29064611 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5578e1fe9860 con 0x5578e23f88e0 2024-03-15T13:03:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.493+0000 7f60b1cd9640 1 --2- 172.21.15.2:0/29064611 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5578e23f88e0 0x5578e23fad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.493+0000 7f6095ffb640 10 client.17007 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:07.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.493+0000 7f60b1cd9640 1 --2- 172.21.15.2:0/29064611 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5578e23f88e0 0x5578e23fad00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f80aca5eb8b0ed07 server_cookie=8736ca2d9f4e0fe4 in_seq=0 out_seq=0 2024-03-15T13:03:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.495+0000 7f6095ffb640 1 -- 172.21.15.2:0/29064611 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5578e1fe9860 con 0x5578e23f88e0 2024-03-15T13:03:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.495+0000 7f6095ffb640 10 client.17007 handle_command_reply: tid=0 2024-03-15T13:03:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.495+0000 7f60b73f2740 1 client.17007 shutdown 2024-03-15T13:03:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.495+0000 7f60b73f2740 1 -- 172.21.15.2:0/29064611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382380 msgr2=0x5578e2385520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.495+0000 7f60b73f2740 1 --2- 172.21.15.2:0/29064611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382380 0x5578e2385520 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f60a8011a10 tx=0x7f60a8009cb0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.495+0000 7f60b73f2740 1 -- 172.21.15.2:0/29064611 shutdown_connections 2024-03-15T13:03:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.495+0000 7f60b73f2740 1 --2- 172.21.15.2:0/29064611 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5578e23f88e0 0x5578e23fad00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.495+0000 7f60b73f2740 1 --2- 172.21.15.2:0/29064611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5578e238b8b0 0x5578e23f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.495+0000 7f60b73f2740 1 --2- 172.21.15.2:0/29064611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5578e2382d30 0x5578e2385a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.495+0000 7f60b73f2740 1 --2- 172.21.15.2:0/29064611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578e2382380 0x5578e2385520 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.495+0000 7f60b73f2740 1 -- 172.21.15.2:0/29064611 >> 172.21.15.2:0/29064611 conn(0x5578e237c840 msgr2=0x5578e23a2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.495+0000 7f6095ffb640 0 client.17007 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.495+0000 7f60b73f2740 1 -- 172.21.15.2:0/29064611 shutdown_connections 2024-03-15T13:03:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.495+0000 7f60b73f2740 1 -- 172.21.15.2:0/29064611 wait complete. 2024-03-15T13:03:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.495+0000 7f60b73f2740 20 client.17007 trim_cache size 0 max 16384 2024-03-15T13:03:07.487 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:03:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:03:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "ed15a12a-b224-4119-b1c7-4a7578d2de14", 2024-03-15T13:03:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:03:07.488 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:03:07.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.496+0000 7f60b473e640 1 -- 172.21.15.2:0/1859798821 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f60980a1c20 msgr2=0x7f60980a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.496+0000 7f60b473e640 1 --2- 172.21.15.2:0/1859798821 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f60980a1c20 0x7f60980a40e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f60a4004af0 tx=0x7f60a402c960 comp rx=0 tx=0).stop 2024-03-15T13:03:07.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.496+0000 7f60b473e640 1 -- 172.21.15.2:0/1859798821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac100dd0 msgr2=0x7f60ac19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.496+0000 7f60b473e640 1 --2- 172.21.15.2:0/1859798821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac100dd0 0x7f60ac19a280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f609c00e9f0 tx=0x7f609c00edf0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.496+0000 7f60b473e640 1 -- 172.21.15.2:0/1859798821 shutdown_connections 2024-03-15T13:03:07.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.496+0000 7f60b473e640 1 --2- 172.21.15.2:0/1859798821 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f60980a1c20 0x7f60980a40e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.496+0000 7f60b473e640 1 --2- 172.21.15.2:0/1859798821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60ac1064b0 0x7f60ac19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.496+0000 7f60b473e640 1 --2- 172.21.15.2:0/1859798821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60ac100dd0 0x7f60ac19a280 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.496+0000 7f60b473e640 1 --2- 172.21.15.2:0/1859798821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60ac100410 0x7f60ac199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.496+0000 7f60b473e640 1 -- 172.21.15.2:0/1859798821 >> 172.21.15.2:0/1859798821 conn(0x7f60ac077550 msgr2=0x7f60ac10b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:07.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.496+0000 7f60b473e640 1 -- 172.21.15.2:0/1859798821 shutdown_connections 2024-03-15T13:03:07.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.496+0000 7f60b473e640 1 -- 172.21.15.2:0/1859798821 wait complete. 2024-03-15T13:03:07.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:03:07.492 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:03:07.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.612+0000 7fecea95e640 1 -- 172.21.15.2:0/3061265570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fece4107500 msgr2=0x7fece41078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.612+0000 7fecea95e640 1 --2- 172.21.15.2:0/3061265570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fece4107500 0x7fece41078e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009930 tx=0x7fecd402bc30 comp rx=0 tx=0).stop 2024-03-15T13:03:07.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.612+0000 7fecea95e640 1 -- 172.21.15.2:0/3061265570 shutdown_connections 2024-03-15T13:03:07.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.612+0000 7fecea95e640 1 --2- 172.21.15.2:0/3061265570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fece4101e20 0x7fece410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.612+0000 7fecea95e640 1 --2- 172.21.15.2:0/3061265570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece4101460 0x7fece41018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.612+0000 7fecea95e640 1 --2- 172.21.15.2:0/3061265570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fece4107500 0x7fece41078e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.612+0000 7fecea95e640 1 -- 172.21.15.2:0/3061265570 >> 172.21.15.2:0/3061265570 conn(0x7fece40fcf90 msgr2=0x7fece40ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:07.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.612+0000 7fecea95e640 1 -- 172.21.15.2:0/3061265570 shutdown_connections 2024-03-15T13:03:07.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.612+0000 7fecea95e640 1 -- 172.21.15.2:0/3061265570 wait complete. 2024-03-15T13:03:07.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.612+0000 7fecea95e640 1 Processor -- start 2024-03-15T13:03:07.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.612+0000 7fecea95e640 1 -- start start 2024-03-15T13:03:07.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fecea95e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece4101460 0x7fece419d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fecea95e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fece4101e20 0x7fece419dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fecea95e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fece4107500 0x7fece4197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fecea95e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece4197fc0 con 0x7fece4101e20 2024-03-15T13:03:07.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fecea95e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece4198130 con 0x7fece4107500 2024-03-15T13:03:07.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fecea95e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece41982f0 con 0x7fece4101460 2024-03-15T13:03:07.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fece37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fece4101e20 0x7fece419dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fece37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fece4101e20 0x7fece419dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40238/0 (socket says 172.21.15.2:40238) 2024-03-15T13:03:07.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fece37fe640 1 -- 172.21.15.2:0/2265970604 learned_addr learned my addr 172.21.15.2:0/2265970604 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:07.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fece3fff640 1 --2- 172.21.15.2:0/2265970604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece4101460 0x7fece419d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fece8efb640 1 --2- 172.21.15.2:0/2265970604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fece4107500 0x7fece4197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fece37fe640 1 -- 172.21.15.2:0/2265970604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece4101460 msgr2=0x7fece419d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fece37fe640 1 --2- 172.21.15.2:0/2265970604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece4101460 0x7fece419d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fece37fe640 1 -- 172.21.15.2:0/2265970604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fece4107500 msgr2=0x7fece4197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fece37fe640 1 --2- 172.21.15.2:0/2265970604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fece4107500 0x7fece4197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fece37fe640 1 -- 172.21.15.2:0/2265970604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd4009560 con 0x7fece4101e20 2024-03-15T13:03:07.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fece37fe640 1 --2- 172.21.15.2:0/2265970604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fece4101e20 0x7fece419dcb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fecd800e9f0 tx=0x7fecd800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:07.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fece17fa640 1 -- 172.21.15.2:0/2265970604 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8009720 con 0x7fece4101e20 2024-03-15T13:03:07.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fece17fa640 1 -- 172.21.15.2:0/2265970604 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fecd800cd80 con 0x7fece4101e20 2024-03-15T13:03:07.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fece17fa640 1 -- 172.21.15.2:0/2265970604 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8010690 con 0x7fece4101e20 2024-03-15T13:03:07.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fecea95e640 1 -- 172.21.15.2:0/2265970604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece41985d0 con 0x7fece4101e20 2024-03-15T13:03:07.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.613+0000 7fecea95e640 1 -- 172.21.15.2:0/2265970604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece41a4810 con 0x7fece4101e20 2024-03-15T13:03:07.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.615+0000 7fece17fa640 1 -- 172.21.15.2:0/2265970604 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fecd8004e90 con 0x7fece4101e20 2024-03-15T13:03:07.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7feced61b740 1 Processor -- start 2024-03-15T13:03:07.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7feced61b740 1 -- start start 2024-03-15T13:03:07.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7fece17fa640 1 --2- 172.21.15.2:0/2265970604 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feccc0a1c20 0x7feccc0a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7feced61b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f52b0 0x55eb733f5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7feced61b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f5bd0 0x55eb733fe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7feced61b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733fe7b0 0x55eb73400ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7feced61b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb73364280 con 0x55eb733fe7b0 2024-03-15T13:03:07.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7feced61b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb733617b0 con 0x55eb733f5bd0 2024-03-15T13:03:07.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7feced61b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7335cf90 con 0x55eb733f52b0 2024-03-15T13:03:07.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7fece8efb640 1 --2- 172.21.15.2:0/2265970604 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feccc0a1c20 0x7feccc0a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7fece17fa640 1 -- 172.21.15.2:0/2265970604 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fecd8014070 con 0x7fece4101e20 2024-03-15T13:03:07.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7fece3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f52b0 0x55eb733f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7fece37fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f5bd0 0x55eb733fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7fece8efb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733fe7b0 0x55eb73400ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7fece3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f52b0 0x55eb733f5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:44108/0 (socket says 172.21.15.2:44108) 2024-03-15T13:03:07.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7fece37fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f5bd0 0x55eb733fe270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48004/0 (socket says 172.21.15.2:48004) 2024-03-15T13:03:07.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7fece3fff640 1 -- 172.21.15.2:0/3804240273 learned_addr learned my addr 172.21.15.2:0/3804240273 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:07.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7fece3fff640 1 -- 172.21.15.2:0/3804240273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f5bd0 msgr2=0x55eb733fe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7fece3fff640 1 --2- 172.21.15.2:0/3804240273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f5bd0 0x55eb733fe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7fece3fff640 1 -- 172.21.15.2:0/3804240273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733fe7b0 msgr2=0x55eb73400ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.616+0000 7fece3fff640 1 --2- 172.21.15.2:0/3804240273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733fe7b0 0x55eb73400ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.617+0000 7fece3fff640 1 -- 172.21.15.2:0/3804240273 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb7331b570 con 0x55eb733f52b0 2024-03-15T13:03:07.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.617+0000 7fece8efb640 1 --2- 172.21.15.2:0/2265970604 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feccc0a1c20 0x7feccc0a40e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009fd0 tx=0x7fecdc009d40 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:07.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.617+0000 7fece3fff640 1 --2- 172.21.15.2:0/3804240273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f52b0 0x55eb733f5690 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009230 tx=0x7fecd402c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3e330a30192e1f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:07.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.617+0000 7fecca7fc640 1 -- 172.21.15.2:0/3804240273 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4004ea0 con 0x55eb733f52b0 2024-03-15T13:03:07.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.617+0000 7fecca7fc640 1 -- 172.21.15.2:0/3804240273 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fecd402c810 con 0x55eb733f52b0 2024-03-15T13:03:07.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.617+0000 7fecca7fc640 1 -- 172.21.15.2:0/3804240273 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4005020 con 0x55eb733f52b0 2024-03-15T13:03:07.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.617+0000 7feced61b740 1 -- 172.21.15.2:0/3804240273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f52b0 msgr2=0x55eb733f5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.617+0000 7feced61b740 1 --2- 172.21.15.2:0/3804240273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f52b0 0x55eb733f5690 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009230 tx=0x7fecd402c5d0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 1 -- 172.21.15.2:0/3804240273 shutdown_connections 2024-03-15T13:03:07.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 1 --2- 172.21.15.2:0/3804240273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733fe7b0 0x55eb73400ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 1 --2- 172.21.15.2:0/3804240273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f5bd0 0x55eb733fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 1 --2- 172.21.15.2:0/3804240273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f52b0 0x55eb733f5690 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 1 -- 172.21.15.2:0/3804240273 >> 172.21.15.2:0/3804240273 conn(0x55eb733ef990 msgr2=0x55eb733efd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:07.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 1 -- 172.21.15.2:0/3804240273 shutdown_connections 2024-03-15T13:03:07.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 1 -- 172.21.15.2:0/3804240273 wait complete. 2024-03-15T13:03:07.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 1 -- start start 2024-03-15T13:03:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 1 Processor -- start 2024-03-15T13:03:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f52b0 0x55eb733fe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f5bd0 0x55eb733fb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733fe7b0 0x55eb733fba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb73030430 con 0x55eb733fe7b0 2024-03-15T13:03:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb72fece00 con 0x55eb733f52b0 2024-03-15T13:03:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb732e1fd0 con 0x55eb733f5bd0 2024-03-15T13:03:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.618+0000 7feced61b740 10 client.0 _ll_register_callbacks cb 0x55eb73337fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fece8efb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733fe7b0 0x55eb733fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fece37fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f5bd0 0x55eb733fb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fece8efb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733fe7b0 0x55eb733fba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40252/0 (socket says 172.21.15.2:40252) 2024-03-15T13:03:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fece37fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f5bd0 0x55eb733fb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:44120/0 (socket says 172.21.15.2:44120) 2024-03-15T13:03:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fece8efb640 1 -- 172.21.15.2:0/3980159789 learned_addr learned my addr 172.21.15.2:0/3980159789 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fece3fff640 1 --2- 172.21.15.2:0/3980159789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f52b0 0x55eb733fe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fece8efb640 1 -- 172.21.15.2:0/3980159789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f5bd0 msgr2=0x55eb733fb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fece8efb640 1 --2- 172.21.15.2:0/3980159789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f5bd0 0x55eb733fb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fece8efb640 1 -- 172.21.15.2:0/3980159789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f52b0 msgr2=0x55eb733fe150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fece8efb640 1 --2- 172.21.15.2:0/3980159789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f52b0 0x55eb733fe150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fece8efb640 1 -- 172.21.15.2:0/3980159789 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece410ec30 con 0x55eb733fe7b0 2024-03-15T13:03:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fece37fe640 1 --2- 172.21.15.2:0/3980159789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f5bd0 0x55eb733fb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fece8efb640 1 --2- 172.21.15.2:0/3980159789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733fe7b0 0x55eb733fba80 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fecdc010d50 tx=0x7fecdc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268be5143aaa1e9e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fecc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fecc37fe640 1 -- 172.21.15.2:0/3980159789 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc010d80 con 0x55eb733fe7b0 2024-03-15T13:03:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fecc37fe640 1 -- 172.21.15.2:0/3980159789 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fecdc0044c0 con 0x55eb733fe7b0 2024-03-15T13:03:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7feced61b740 1 -- 172.21.15.2:0/3980159789 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb7305c7e0 con 0x55eb733fe7b0 2024-03-15T13:03:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fecc37fe640 1 -- 172.21.15.2:0/3980159789 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecdc004680 con 0x55eb733fe7b0 2024-03-15T13:03:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7feced61b740 10 client.17019 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7feced61b740 1 -- 172.21.15.2:0/3980159789 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55eb7335b610 con 0x55eb733fe7b0 2024-03-15T13:03:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.619+0000 7fecc37fe640 1 -- 172.21.15.2:0/3980159789 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55eb7305c7e0 con 0x55eb733fe7b0 2024-03-15T13:03:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.620+0000 7fecc37fe640 1 -- 172.21.15.2:0/3980159789 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7fecdc020410 con 0x55eb733fe7b0 2024-03-15T13:03:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.620+0000 7feced61b740 10 client.17019 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.620+0000 7feced61b740 10 client.17019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.620+0000 7feced61b740 20 client.17019 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.620+0000 7feced61b740 1 --2- 172.21.15.2:0/3980159789 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55eb7340e950 0x55eb73410d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.620+0000 7feced61b740 4 client.17019 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.620+0000 7feced61b740 1 -- 172.21.15.2:0/3980159789 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb73099720 con 0x55eb7340e950 2024-03-15T13:03:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.620+0000 7fece3fff640 1 --2- 172.21.15.2:0/3980159789 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55eb7340e950 0x55eb73410d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.620+0000 7fece3fff640 1 --2- 172.21.15.2:0/3980159789 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55eb7340e950 0x55eb73410d70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91c4fbf5efe2182d server_cookie=bc7c6e14a84f5f9a in_seq=0 out_seq=0 2024-03-15T13:03:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.620+0000 7fecc37fe640 10 client.17019 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:07.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.621+0000 7fecc37fe640 1 -- 172.21.15.2:0/3980159789 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55eb73099720 con 0x55eb7340e950 2024-03-15T13:03:07.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.621+0000 7fecc37fe640 10 client.17019 handle_command_reply: tid=0 2024-03-15T13:03:07.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.621+0000 7feced61b740 1 client.17019 shutdown 2024-03-15T13:03:07.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.621+0000 7feced61b740 1 -- 172.21.15.2:0/3980159789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733fe7b0 msgr2=0x55eb733fba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.621+0000 7feced61b740 1 --2- 172.21.15.2:0/3980159789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733fe7b0 0x55eb733fba80 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fecdc010d50 tx=0x7fecdc009500 comp rx=0 tx=0).stop 2024-03-15T13:03:07.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.622+0000 7feced61b740 1 -- 172.21.15.2:0/3980159789 shutdown_connections 2024-03-15T13:03:07.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.622+0000 7feced61b740 1 --2- 172.21.15.2:0/3980159789 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55eb7340e950 0x55eb73410d70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.622+0000 7feced61b740 1 --2- 172.21.15.2:0/3980159789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733fe7b0 0x55eb733fba80 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.622+0000 7feced61b740 1 --2- 172.21.15.2:0/3980159789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f5bd0 0x55eb733fb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.622+0000 7fecc37fe640 0 client.17019 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:07.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.622+0000 7feced61b740 1 --2- 172.21.15.2:0/3980159789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f52b0 0x55eb733fe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.622+0000 7feced61b740 1 -- 172.21.15.2:0/3980159789 >> 172.21.15.2:0/3980159789 conn(0x55eb733ef990 msgr2=0x55eb733f1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:07.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.622+0000 7feced61b740 1 -- 172.21.15.2:0/3980159789 shutdown_connections 2024-03-15T13:03:07.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.622+0000 7feced61b740 1 -- 172.21.15.2:0/3980159789 wait complete. 2024-03-15T13:03:07.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.622+0000 7feced61b740 20 client.17019 trim_cache size 0 max 16384 2024-03-15T13:03:07.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7feced61b740 1 Processor -- start 2024-03-15T13:03:07.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7feced61b740 1 -- start start 2024-03-15T13:03:07.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7feced61b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733f52b0 0x55eb733fba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7feced61b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f5bd0 0x55eb733fdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7feced61b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733fe7b0 0x55eb733fe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7feced61b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb733f4f10 con 0x55eb733f52b0 2024-03-15T13:03:07.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7feced61b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb733eb2b0 con 0x55eb733f5bd0 2024-03-15T13:03:07.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7feced61b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb733f09e0 con 0x55eb733fe7b0 2024-03-15T13:03:07.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7fece3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733f52b0 0x55eb733fba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7fece3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733f52b0 0x55eb733fba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40256/0 (socket says 172.21.15.2:40256) 2024-03-15T13:03:07.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7fece3fff640 1 -- 172.21.15.2:0/3195980740 learned_addr learned my addr 172.21.15.2:0/3195980740 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:07.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7fece37fe640 1 --2- 172.21.15.2:0/3195980740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f5bd0 0x55eb733fdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7fece8efb640 1 --2- 172.21.15.2:0/3195980740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733fe7b0 0x55eb733fe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7fece3fff640 1 -- 172.21.15.2:0/3195980740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733fe7b0 msgr2=0x55eb733fe150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7fece3fff640 1 --2- 172.21.15.2:0/3195980740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733fe7b0 0x55eb733fe150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7fece3fff640 1 -- 172.21.15.2:0/3195980740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f5bd0 msgr2=0x55eb733fdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7fece3fff640 1 --2- 172.21.15.2:0/3195980740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f5bd0 0x55eb733fdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7fece3fff640 1 -- 172.21.15.2:0/3195980740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb732f7ac0 con 0x55eb733f52b0 2024-03-15T13:03:07.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7fece8efb640 1 --2- 172.21.15.2:0/3195980740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733fe7b0 0x55eb733fe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:03:07.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7fece37fe640 1 --2- 172.21.15.2:0/3195980740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f5bd0 0x55eb733fdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:03:07.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7fece3fff640 1 --2- 172.21.15.2:0/3195980740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733f52b0 0x55eb733fba30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fecd402c7b0 tx=0x7fecd4041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23b3768527b41b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:07.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.638+0000 7fecc9ffb640 1 -- 172.21.15.2:0/3195980740 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd402cc80 con 0x55eb733f52b0 2024-03-15T13:03:07.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.639+0000 7fecc9ffb640 1 -- 172.21.15.2:0/3195980740 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fecd4053670 con 0x55eb733f52b0 2024-03-15T13:03:07.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.639+0000 7fecc9ffb640 1 -- 172.21.15.2:0/3195980740 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4005040 con 0x55eb733f52b0 2024-03-15T13:03:07.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.639+0000 7feced61b740 1 -- 172.21.15.2:0/3195980740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733f52b0 msgr2=0x55eb733fba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.639+0000 7feced61b740 1 --2- 172.21.15.2:0/3195980740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733f52b0 0x55eb733fba30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fecd402c7b0 tx=0x7fecd4041040 comp rx=0 tx=0).stop 2024-03-15T13:03:07.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.639+0000 7feced61b740 1 -- 172.21.15.2:0/3195980740 shutdown_connections 2024-03-15T13:03:07.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.639+0000 7feced61b740 1 --2- 172.21.15.2:0/3195980740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733fe7b0 0x55eb733fe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.639+0000 7feced61b740 1 --2- 172.21.15.2:0/3195980740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733f5bd0 0x55eb733fdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.639+0000 7feced61b740 1 --2- 172.21.15.2:0/3195980740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733f52b0 0x55eb733fba30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.639+0000 7feced61b740 1 -- 172.21.15.2:0/3195980740 >> 172.21.15.2:0/3195980740 conn(0x55eb733ef990 msgr2=0x55eb733eea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:07.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.639+0000 7feced61b740 1 -- 172.21.15.2:0/3195980740 shutdown_connections 2024-03-15T13:03:07.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.639+0000 7feced61b740 1 -- 172.21.15.2:0/3195980740 wait complete. 2024-03-15T13:03:07.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.639+0000 7feced61b740 1 -- start start 2024-03-15T13:03:07.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7feced61b740 1 Processor -- start 2024-03-15T13:03:07.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7feced61b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f52b0 0x55eb733f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7feced61b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733f5bd0 0x55eb733f8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7feced61b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733fe7b0 0x55eb733f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7feced61b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb733feb90 con 0x55eb733f5bd0 2024-03-15T13:03:07.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7feced61b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb733fecd0 con 0x55eb733fe7b0 2024-03-15T13:03:07.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7feced61b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb733fee90 con 0x55eb733f52b0 2024-03-15T13:03:07.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7feced61b740 10 client.0 _ll_register_callbacks cb 0x7fecdc009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:07.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7fece37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733f5bd0 0x55eb733f8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7fece37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733f5bd0 0x55eb733f8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40266/0 (socket says 172.21.15.2:40266) 2024-03-15T13:03:07.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7fece37fe640 1 -- 172.21.15.2:0/425945250 learned_addr learned my addr 172.21.15.2:0/425945250 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:07.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7fece3fff640 1 --2- 172.21.15.2:0/425945250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f52b0 0x55eb733f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7fece8efb640 1 --2- 172.21.15.2:0/425945250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733fe7b0 0x55eb733f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7fece37fe640 1 -- 172.21.15.2:0/425945250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f52b0 msgr2=0x55eb733f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7fece37fe640 1 --2- 172.21.15.2:0/425945250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f52b0 0x55eb733f8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7fece37fe640 1 -- 172.21.15.2:0/425945250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733fe7b0 msgr2=0x55eb733f91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:07.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7fece37fe640 1 --2- 172.21.15.2:0/425945250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733fe7b0 0x55eb733f91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7fece37fe640 1 -- 172.21.15.2:0/425945250 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb7331b570 con 0x55eb733f5bd0 2024-03-15T13:03:07.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7fece37fe640 1 --2- 172.21.15.2:0/425945250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733f5bd0 0x55eb733f8ca0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fecd809a4b0 tx=0x7fecd800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5fbe3246023980 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:07.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.640+0000 7fecc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:07.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.641+0000 7fecc37fe640 1 -- 172.21.15.2:0/425945250 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd80b9e10 con 0x55eb733f5bd0 2024-03-15T13:03:07.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.641+0000 7fecc37fe640 1 -- 172.21.15.2:0/425945250 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fecd809a7b0 con 0x55eb733f5bd0 2024-03-15T13:03:07.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.641+0000 7feced61b740 1 -- 172.21.15.2:0/425945250 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecd402c810 con 0x55eb733f5bd0 2024-03-15T13:03:07.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.641+0000 7fecc37fe640 1 -- 172.21.15.2:0/425945250 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecd809e900 con 0x55eb733f5bd0 2024-03-15T13:03:07.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.641+0000 7feced61b740 10 client.17025 fetch_fsmap learned FSMap version 39 2024-03-15T13:03:07.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.641+0000 7feced61b740 1 -- 172.21.15.2:0/425945250 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55eb7331d6a0 con 0x55eb733f5bd0 2024-03-15T13:03:07.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.641+0000 7fecc37fe640 1 -- 172.21.15.2:0/425945250 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fecd402c810 con 0x55eb733f5bd0 2024-03-15T13:03:07.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.641+0000 7fecc37fe640 1 -- 172.21.15.2:0/425945250 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7986+0+0 (secure 0 0 0) 0x7fecd801f060 con 0x55eb733f5bd0 2024-03-15T13:03:07.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.641+0000 7feced61b740 10 client.17025 fetch_fsmap finished waiting for FSMap version 39 2024-03-15T13:03:07.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.641+0000 7feced61b740 10 client.17025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:07.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.641+0000 7feced61b740 20 client.17025 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:07.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.641+0000 7feced61b740 1 --2- 172.21.15.2:0/425945250 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55eb7340e950 0x55eb73471270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:07.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.641+0000 7feced61b740 4 client.17025 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:03:07.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.641+0000 7feced61b740 1 -- 172.21.15.2:0/425945250 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eb730e8890 con 0x55eb7340e950 2024-03-15T13:03:07.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.641+0000 7fece3fff640 1 --2- 172.21.15.2:0/425945250 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55eb7340e950 0x55eb73471270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:07.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7fecc37fe640 10 client.17025 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:07.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7fece3fff640 1 --2- 172.21.15.2:0/425945250 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55eb7340e950 0x55eb73471270 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd073148524f278b server_cookie=41bcea2eac72c6f8 in_seq=0 out_seq=0 2024-03-15T13:03:07.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7fecc37fe640 1 -- 172.21.15.2:0/425945250 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55eb730e8890 con 0x55eb7340e950 2024-03-15T13:03:07.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7fecc37fe640 10 client.17025 handle_command_reply: tid=0 2024-03-15T13:03:07.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7feced61b740 1 client.17025 shutdown 2024-03-15T13:03:07.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7feced61b740 1 -- 172.21.15.2:0/425945250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733f5bd0 msgr2=0x55eb733f8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7feced61b740 1 --2- 172.21.15.2:0/425945250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733f5bd0 0x55eb733f8ca0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fecd809a4b0 tx=0x7fecd800cef0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7feced61b740 1 -- 172.21.15.2:0/425945250 shutdown_connections 2024-03-15T13:03:07.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7feced61b740 1 --2- 172.21.15.2:0/425945250 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55eb7340e950 0x55eb73471270 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7feced61b740 1 --2- 172.21.15.2:0/425945250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb733fe7b0 0x55eb733f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7feced61b740 1 --2- 172.21.15.2:0/425945250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eb733f5bd0 0x55eb733f8ca0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7fecc37fe640 0 client.17025 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:07.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7feced61b740 1 --2- 172.21.15.2:0/425945250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb733f52b0 0x55eb733f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7feced61b740 1 -- 172.21.15.2:0/425945250 >> 172.21.15.2:0/425945250 conn(0x55eb733ef990 msgr2=0x55eb733ee1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:07.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7feced61b740 1 -- 172.21.15.2:0/425945250 shutdown_connections 2024-03-15T13:03:07.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7feced61b740 1 -- 172.21.15.2:0/425945250 wait complete. 2024-03-15T13:03:07.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.642+0000 7feced61b740 20 client.17025 trim_cache size 0 max 16384 2024-03-15T13:03:07.636 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:03:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T13:03:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:03:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "ed15a12a-b224-4119-b1c7-4a7578d2de14": { 2024-03-15T13:03:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:03:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "ed15a12a-b224-4119-b1c7-4a7578d2de14", 2024-03-15T13:03:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:03:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:03:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:03:07.637 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:03:07.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.643+0000 7fecea95e640 1 -- 172.21.15.2:0/2265970604 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feccc0a1c20 msgr2=0x7feccc0a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.643+0000 7fecea95e640 1 --2- 172.21.15.2:0/2265970604 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feccc0a1c20 0x7feccc0a40e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009fd0 tx=0x7fecdc009d40 comp rx=0 tx=0).stop 2024-03-15T13:03:07.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.643+0000 7fecea95e640 1 -- 172.21.15.2:0/2265970604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fece4101e20 msgr2=0x7fece419dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:07.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.643+0000 7fecea95e640 1 --2- 172.21.15.2:0/2265970604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fece4101e20 0x7fece419dcb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fecd800e9f0 tx=0x7fecd800edf0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.643+0000 7fecea95e640 1 -- 172.21.15.2:0/2265970604 shutdown_connections 2024-03-15T13:03:07.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.643+0000 7fecea95e640 1 --2- 172.21.15.2:0/2265970604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fece4107500 0x7fece4197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.643+0000 7fecea95e640 1 --2- 172.21.15.2:0/2265970604 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feccc0a1c20 0x7feccc0a40e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.643+0000 7fecea95e640 1 --2- 172.21.15.2:0/2265970604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fece4101e20 0x7fece419dcb0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.643+0000 7fecea95e640 1 --2- 172.21.15.2:0/2265970604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fece4101460 0x7fece419d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:07.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.643+0000 7fecea95e640 1 -- 172.21.15.2:0/2265970604 >> 172.21.15.2:0/2265970604 conn(0x7fece40fcf90 msgr2=0x7fece40fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:07.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.643+0000 7fecea95e640 1 -- 172.21.15.2:0/2265970604 shutdown_connections 2024-03-15T13:03:07.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:07.643+0000 7fecea95e640 1 -- 172.21.15.2:0/2265970604 wait complete. 2024-03-15T13:03:07.639 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:07 smithi002 ceph-mon[32060]: pgmap v129: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:03:07.639 INFO:tasks.cephfs.filesystem:scrub status for tag:ed15a12a-b224-4119-b1c7-4a7578d2de14 - {'path': '/', 'tag': 'ed15a12a-b224-4119-b1c7-4a7578d2de14', 'options': 'recursive,force'} 2024-03-15T13:03:07.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:07 smithi184 ceph-mon[35993]: pgmap v129: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:03:07.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:07 smithi149 ceph-mon[35927]: pgmap v129: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:03:08.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:08 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:03:08.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:08 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:03:08.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:08 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:03:08.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:08 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:03:08.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:08 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:03:08.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:08 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:03:08.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:08 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:03:08.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:08 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:03:08.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:08 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:03:08.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:08 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:03:08.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:08 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:03:08.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:08 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:03:09.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:09 smithi002 ceph-mon[32060]: pgmap v130: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:03:09.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:09 smithi184 ceph-mon[35993]: pgmap v130: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:03:09.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:09 smithi149 ceph-mon[35927]: pgmap v130: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:03:10.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:10 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:03:10.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:10 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:03:10.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:10 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:03:11.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:11 smithi002 ceph-mon[32060]: pgmap v131: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:03:11.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:11 smithi184 ceph-mon[35993]: pgmap v131: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:03:11.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:11 smithi149 ceph-mon[35927]: pgmap v131: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:03:13.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:13 smithi002 ceph-mon[32060]: pgmap v132: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:03:13.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:13 smithi184 ceph-mon[35993]: pgmap v132: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:03:13.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:13 smithi149 ceph-mon[35927]: pgmap v132: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:03:15.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:15 smithi002 ceph-mon[32060]: pgmap v133: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:03:15.809 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:15 smithi184 ceph-mon[35993]: pgmap v133: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:03:15.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:15 smithi149 ceph-mon[35927]: pgmap v133: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:03:17.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:17 smithi002 ceph-mon[32060]: pgmap v134: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:03:17.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:17 smithi184 ceph-mon[35993]: pgmap v134: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:03:17.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:17 smithi149 ceph-mon[35927]: pgmap v134: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:03:19.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:19 smithi002 ceph-mon[32060]: pgmap v135: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-15T13:03:19.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:19 smithi184 ceph-mon[35993]: pgmap v135: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-15T13:03:19.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:19 smithi149 ceph-mon[35927]: pgmap v135: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-15T13:03:21.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:21 smithi002 ceph-mon[32060]: pgmap v136: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-15T13:03:21.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:21 smithi184 ceph-mon[35993]: pgmap v136: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-15T13:03:21.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:21 smithi149 ceph-mon[35927]: pgmap v136: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-15T13:03:23.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:23 smithi002 ceph-mon[32060]: pgmap v137: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:23.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:23 smithi184 ceph-mon[35993]: pgmap v137: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:23.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:23 smithi149 ceph-mon[35927]: pgmap v137: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:25.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:25 smithi002 ceph-mon[32060]: pgmap v138: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:25.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:25 smithi184 ceph-mon[35993]: pgmap v138: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:25.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:25 smithi149 ceph-mon[35927]: pgmap v138: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:27.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:27 smithi002 ceph-mon[32060]: pgmap v139: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:03:27.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:27 smithi184 ceph-mon[35993]: pgmap v139: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:03:27.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:27 smithi149 ceph-mon[35927]: pgmap v139: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:03:29.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:29 smithi002 ceph-mon[32060]: pgmap v140: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:29.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:29 smithi184 ceph-mon[35993]: pgmap v140: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:29.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:29 smithi149 ceph-mon[35927]: pgmap v140: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:30.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:30 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:03:30.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:30 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:03:30.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:30 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:03:31.760 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:31 smithi149 ceph-mon[35927]: pgmap v141: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:03:31.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:31 smithi002 ceph-mon[32060]: pgmap v141: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:03:31.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:31 smithi184 ceph-mon[35993]: pgmap v141: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:03:33.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:33 smithi002 ceph-mon[32060]: pgmap v142: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:33.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:33 smithi184 ceph-mon[35993]: pgmap v142: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:33.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:33 smithi149 ceph-mon[35927]: pgmap v142: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:35.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:35 smithi002 ceph-mon[32060]: pgmap v143: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:35.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:35 smithi184 ceph-mon[35993]: pgmap v143: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:35.861 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:35 smithi149 ceph-mon[35927]: pgmap v143: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:37.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:03:37.641 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:03:37.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.773+0000 7efe8093e640 1 -- 172.21.15.2:0/3803831056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c101460 msgr2=0x7efe7c1018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.773+0000 7efe8093e640 1 --2- 172.21.15.2:0/3803831056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c101460 0x7efe7c1018e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7efe70009930 tx=0x7efe7002bc30 comp rx=0 tx=0).stop 2024-03-15T13:03:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.773+0000 7efe8093e640 1 -- 172.21.15.2:0/3803831056 shutdown_connections 2024-03-15T13:03:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.773+0000 7efe8093e640 1 --2- 172.21.15.2:0/3803831056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe7c101e20 0x7efe7c10e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.773+0000 7efe8093e640 1 --2- 172.21.15.2:0/3803831056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c101460 0x7efe7c1018e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.773+0000 7efe8093e640 1 --2- 172.21.15.2:0/3803831056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe7c107500 0x7efe7c1078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.773+0000 7efe8093e640 1 -- 172.21.15.2:0/3803831056 >> 172.21.15.2:0/3803831056 conn(0x7efe7c0fcf90 msgr2=0x7efe7c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.773+0000 7efe8093e640 1 -- 172.21.15.2:0/3803831056 shutdown_connections 2024-03-15T13:03:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.773+0000 7efe8093e640 1 -- 172.21.15.2:0/3803831056 wait complete. 2024-03-15T13:03:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.774+0000 7efe8093e640 1 Processor -- start 2024-03-15T13:03:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.774+0000 7efe8093e640 1 -- start start 2024-03-15T13:03:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.774+0000 7efe8093e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c101460 0x7efe7c19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.774+0000 7efe8093e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe7c101e20 0x7efe7c19dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.774+0000 7efe8093e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe7c107500 0x7efe7c197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.774+0000 7efe8093e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c197fa0 con 0x7efe7c101460 2024-03-15T13:03:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.774+0000 7efe8093e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c198110 con 0x7efe7c101e20 2024-03-15T13:03:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.774+0000 7efe8093e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c1982d0 con 0x7efe7c107500 2024-03-15T13:03:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.774+0000 7efe7a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c101460 0x7efe7c19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.774+0000 7efe7a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c101460 0x7efe7c19d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35108/0 (socket says 172.21.15.2:35108) 2024-03-15T13:03:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.774+0000 7efe7a59c640 1 -- 172.21.15.2:0/2783623699 learned_addr learned my addr 172.21.15.2:0/2783623699 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.774+0000 7efe79d9b640 1 --2- 172.21.15.2:0/2783623699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe7c101e20 0x7efe7c19dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.774+0000 7efe7ad9d640 1 --2- 172.21.15.2:0/2783623699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe7c107500 0x7efe7c197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.775+0000 7efe7a59c640 1 -- 172.21.15.2:0/2783623699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe7c107500 msgr2=0x7efe7c197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.775+0000 7efe7a59c640 1 --2- 172.21.15.2:0/2783623699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe7c107500 0x7efe7c197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.775+0000 7efe7a59c640 1 -- 172.21.15.2:0/2783623699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe7c101e20 msgr2=0x7efe7c19dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.775+0000 7efe7a59c640 1 --2- 172.21.15.2:0/2783623699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe7c101e20 0x7efe7c19dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.775+0000 7efe7a59c640 1 -- 172.21.15.2:0/2783623699 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe70009560 con 0x7efe7c101460 2024-03-15T13:03:37.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.775+0000 7efe7a59c640 1 --2- 172.21.15.2:0/2783623699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c101460 0x7efe7c19d7a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7efe7400e9f0 tx=0x7efe7400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:37.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.775+0000 7efe6b7fe640 1 -- 172.21.15.2:0/2783623699 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe74009720 con 0x7efe7c101460 2024-03-15T13:03:37.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.775+0000 7efe6b7fe640 1 -- 172.21.15.2:0/2783623699 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efe7400cd80 con 0x7efe7c101460 2024-03-15T13:03:37.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.775+0000 7efe8093e640 1 -- 172.21.15.2:0/2783623699 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7c1985b0 con 0x7efe7c101460 2024-03-15T13:03:37.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.775+0000 7efe6b7fe640 1 -- 172.21.15.2:0/2783623699 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe74010690 con 0x7efe7c101460 2024-03-15T13:03:37.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.775+0000 7efe8093e640 1 -- 172.21.15.2:0/2783623699 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7c1a4840 con 0x7efe7c101460 2024-03-15T13:03:37.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.777+0000 7efe6b7fe640 1 -- 172.21.15.2:0/2783623699 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7efe74004e90 con 0x7efe7c101460 2024-03-15T13:03:37.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.778+0000 7efe6b7fe640 1 --2- 172.21.15.2:0/2783623699 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efe640a1ba0 0x7efe640a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.778+0000 7efe835f6740 1 Processor -- start 2024-03-15T13:03:37.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.778+0000 7efe79d9b640 1 --2- 172.21.15.2:0/2783623699 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efe640a1ba0 0x7efe640a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.778+0000 7efe6b7fe640 1 -- 172.21.15.2:0/2783623699 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7efe74014070 con 0x7efe7c101460 2024-03-15T13:03:37.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.778+0000 7efe835f6740 1 -- start start 2024-03-15T13:03:37.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.778+0000 7efe835f6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d8f2b0 0x565339d8f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.778+0000 7efe835f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d98270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.778+0000 7efe835f6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d987b0 0x565339d9aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.778+0000 7efe835f6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x565339cfe280 con 0x565339d987b0 2024-03-15T13:03:37.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.778+0000 7efe835f6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x565339cfb7b0 con 0x565339d8f2b0 2024-03-15T13:03:37.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.778+0000 7efe835f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x565339cf6f90 con 0x565339d8fbd0 2024-03-15T13:03:37.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.778+0000 7efe7a59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d8f2b0 0x565339d8f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.778+0000 7efe7ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d987b0 0x565339d9aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.778+0000 7efe7a59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d8f2b0 0x565339d8f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:41830/0 (socket says 172.21.15.2:41830) 2024-03-15T13:03:37.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.778+0000 7efe7a59c640 1 -- 172.21.15.2:0/4274241988 learned_addr learned my addr 172.21.15.2:0/4274241988 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:37.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.779+0000 7efe79d9b640 1 --2- 172.21.15.2:0/2783623699 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efe640a1ba0 0x7efe640a4060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efe70000c00 tx=0x7efe70005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:37.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.779+0000 7efe79d9b640 1 --2- 172.21.15.2:0/4274241988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d98270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.779+0000 7efe7a59c640 1 -- 172.21.15.2:0/4274241988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 msgr2=0x565339d98270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:37.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.779+0000 7efe7a59c640 1 --2- 172.21.15.2:0/4274241988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d98270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.779+0000 7efe7a59c640 1 -- 172.21.15.2:0/4274241988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d987b0 msgr2=0x565339d9aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:37.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.779+0000 7efe7a59c640 1 --2- 172.21.15.2:0/4274241988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d987b0 0x565339d9aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.779+0000 7efe7a59c640 1 -- 172.21.15.2:0/4274241988 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565339cb5570 con 0x565339d8f2b0 2024-03-15T13:03:37.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.779+0000 7efe7a59c640 1 --2- 172.21.15.2:0/4274241988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d8f2b0 0x565339d8f690 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efe740970e0 tx=0x7efe7409a070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90c851fa5bdd41f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:37.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.779+0000 7efe68ff9640 1 -- 172.21.15.2:0/4274241988 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe740b3210 con 0x565339d8f2b0 2024-03-15T13:03:37.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.779+0000 7efe68ff9640 1 -- 172.21.15.2:0/4274241988 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efe74083e90 con 0x565339d8f2b0 2024-03-15T13:03:37.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.779+0000 7efe68ff9640 1 -- 172.21.15.2:0/4274241988 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe74096d80 con 0x565339d8f2b0 2024-03-15T13:03:37.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.780+0000 7efe835f6740 1 -- 172.21.15.2:0/4274241988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d8f2b0 msgr2=0x565339d8f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:37.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.780+0000 7efe835f6740 1 --2- 172.21.15.2:0/4274241988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d8f2b0 0x565339d8f690 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efe740970e0 tx=0x7efe7409a070 comp rx=0 tx=0).stop 2024-03-15T13:03:37.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.780+0000 7efe835f6740 1 -- 172.21.15.2:0/4274241988 shutdown_connections 2024-03-15T13:03:37.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.780+0000 7efe835f6740 1 --2- 172.21.15.2:0/4274241988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d987b0 0x565339d9aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.780+0000 7efe835f6740 1 --2- 172.21.15.2:0/4274241988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d98270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.780+0000 7efe835f6740 1 --2- 172.21.15.2:0/4274241988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d8f2b0 0x565339d8f690 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.780+0000 7efe835f6740 1 -- 172.21.15.2:0/4274241988 >> 172.21.15.2:0/4274241988 conn(0x565339d89990 msgr2=0x565339d89d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:37.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.780+0000 7efe835f6740 1 -- 172.21.15.2:0/4274241988 shutdown_connections 2024-03-15T13:03:37.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.780+0000 7efe835f6740 1 -- 172.21.15.2:0/4274241988 wait complete. 2024-03-15T13:03:37.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.780+0000 7efe835f6740 1 -- start start 2024-03-15T13:03:37.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.780+0000 7efe835f6740 1 Processor -- start 2024-03-15T13:03:37.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe835f6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d8f2b0 0x565339d98270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe835f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d95600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe835f6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d987b0 0x565339d95d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe835f6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653399ca430 con 0x565339d8f2b0 2024-03-15T13:03:37.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe835f6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x565339986e00 con 0x565339d987b0 2024-03-15T13:03:37.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe835f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x565339c7bfd0 con 0x565339d8fbd0 2024-03-15T13:03:37.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe835f6740 10 client.0 _ll_register_callbacks cb 0x565339cd1fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:37.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe7a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d8f2b0 0x565339d98270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe7a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d8f2b0 0x565339d98270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35114/0 (socket says 172.21.15.2:35114) 2024-03-15T13:03:37.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe7a59c640 1 -- 172.21.15.2:0/3038722616 learned_addr learned my addr 172.21.15.2:0/3038722616 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:37.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe79d9b640 1 --2- 172.21.15.2:0/3038722616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d95600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe7ad9d640 1 --2- 172.21.15.2:0/3038722616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d987b0 0x565339d95d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe7a59c640 1 -- 172.21.15.2:0/3038722616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 msgr2=0x565339d95600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:37.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe7a59c640 1 --2- 172.21.15.2:0/3038722616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d95600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe7a59c640 1 -- 172.21.15.2:0/3038722616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d987b0 msgr2=0x565339d95d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:37.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe7a59c640 1 --2- 172.21.15.2:0/3038722616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d987b0 0x565339d95d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe7a59c640 1 -- 172.21.15.2:0/3038722616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7c1a4840 con 0x565339d8f2b0 2024-03-15T13:03:37.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe7a59c640 1 --2- 172.21.15.2:0/3038722616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d8f2b0 0x565339d98270 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7efe74009380 tx=0x7efe7409a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f9ed349ea3ca3d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:37.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:37.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.781+0000 7efe61ffb640 1 -- 172.21.15.2:0/3038722616 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe740a2eb0 con 0x565339d8f2b0 2024-03-15T13:03:37.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.782+0000 7efe61ffb640 1 -- 172.21.15.2:0/3038722616 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7efe740b3210 con 0x565339d8f2b0 2024-03-15T13:03:37.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.782+0000 7efe835f6740 1 -- 172.21.15.2:0/3038722616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653399f67e0 con 0x565339d8f2b0 2024-03-15T13:03:37.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.782+0000 7efe61ffb640 1 -- 172.21.15.2:0/3038722616 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe740ba0d0 con 0x565339d8f2b0 2024-03-15T13:03:37.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.782+0000 7efe835f6740 10 client.17037 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:37.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.782+0000 7efe835f6740 1 -- 172.21.15.2:0/3038722616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x565339cf5610 con 0x565339d8f2b0 2024-03-15T13:03:37.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.782+0000 7efe61ffb640 1 -- 172.21.15.2:0/3038722616 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5653399f67e0 con 0x565339d8f2b0 2024-03-15T13:03:37.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.782+0000 7efe61ffb640 1 -- 172.21.15.2:0/3038722616 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7efe7401f060 con 0x565339d8f2b0 2024-03-15T13:03:37.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.782+0000 7efe835f6740 10 client.17037 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:37.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.782+0000 7efe835f6740 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:37.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.782+0000 7efe835f6740 20 client.17037 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:37.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.782+0000 7efe835f6740 1 --2- 172.21.15.2:0/3038722616 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x565339da8790 0x565339daabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.782+0000 7efe835f6740 4 client.17037 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:37.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.782+0000 7efe835f6740 1 -- 172.21.15.2:0/3038722616 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565339a33720 con 0x565339da8790 2024-03-15T13:03:37.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.782+0000 7efe79d9b640 1 --2- 172.21.15.2:0/3038722616 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x565339da8790 0x565339daabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.783+0000 7efe79d9b640 1 --2- 172.21.15.2:0/3038722616 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x565339da8790 0x565339daabb0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=457010506abf3698 server_cookie=30d29d9be3891879 in_seq=0 out_seq=0 2024-03-15T13:03:37.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.783+0000 7efe61ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:37.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.784+0000 7efe61ffb640 1 -- 172.21.15.2:0/3038722616 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x565339a33720 con 0x565339da8790 2024-03-15T13:03:37.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.784+0000 7efe61ffb640 10 client.17037 handle_command_reply: tid=0 2024-03-15T13:03:37.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.784+0000 7efe835f6740 1 client.17037 shutdown 2024-03-15T13:03:37.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.784+0000 7efe835f6740 1 -- 172.21.15.2:0/3038722616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d8f2b0 msgr2=0x565339d98270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:37.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.784+0000 7efe835f6740 1 --2- 172.21.15.2:0/3038722616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d8f2b0 0x565339d98270 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7efe74009380 tx=0x7efe7409a890 comp rx=0 tx=0).stop 2024-03-15T13:03:37.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.784+0000 7efe835f6740 1 -- 172.21.15.2:0/3038722616 shutdown_connections 2024-03-15T13:03:37.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.784+0000 7efe835f6740 1 --2- 172.21.15.2:0/3038722616 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x565339da8790 0x565339daabb0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.784+0000 7efe835f6740 1 --2- 172.21.15.2:0/3038722616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d987b0 0x565339d95d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.784+0000 7efe835f6740 1 --2- 172.21.15.2:0/3038722616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d95600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.784+0000 7efe61ffb640 0 client.17037 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:37.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.784+0000 7efe835f6740 1 --2- 172.21.15.2:0/3038722616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d8f2b0 0x565339d98270 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.784+0000 7efe835f6740 1 -- 172.21.15.2:0/3038722616 >> 172.21.15.2:0/3038722616 conn(0x565339d89990 msgr2=0x565339d8b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:37.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.784+0000 7efe835f6740 1 -- 172.21.15.2:0/3038722616 shutdown_connections 2024-03-15T13:03:37.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.784+0000 7efe835f6740 1 -- 172.21.15.2:0/3038722616 wait complete. 2024-03-15T13:03:37.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.784+0000 7efe835f6740 20 client.17037 trim_cache size 0 max 16384 2024-03-15T13:03:37.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.800+0000 7efe835f6740 1 Processor -- start 2024-03-15T13:03:37.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.800+0000 7efe835f6740 1 -- start start 2024-03-15T13:03:37.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.800+0000 7efe835f6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d8f2b0 0x565339d8b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.800+0000 7efe835f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d888d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.800+0000 7efe835f6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d987b0 0x565339d88e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.800+0000 7efe835f6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x565339d9b190 con 0x565339d8f2b0 2024-03-15T13:03:37.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.800+0000 7efe835f6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x565339d8ec70 con 0x565339d987b0 2024-03-15T13:03:37.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.800+0000 7efe835f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x565339d85320 con 0x565339d8fbd0 2024-03-15T13:03:37.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.801+0000 7efe7a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d8f2b0 0x565339d8b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.801+0000 7efe79d9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d888d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.801+0000 7efe7a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d8f2b0 0x565339d8b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35118/0 (socket says 172.21.15.2:35118) 2024-03-15T13:03:37.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.801+0000 7efe7a59c640 1 -- 172.21.15.2:0/1072207320 learned_addr learned my addr 172.21.15.2:0/1072207320 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:37.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.801+0000 7efe7ad9d640 1 --2- 172.21.15.2:0/1072207320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d987b0 0x565339d88e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.801+0000 7efe7a59c640 1 -- 172.21.15.2:0/1072207320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 msgr2=0x565339d888d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:37.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.801+0000 7efe7a59c640 1 --2- 172.21.15.2:0/1072207320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d888d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.801+0000 7efe7a59c640 1 -- 172.21.15.2:0/1072207320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d987b0 msgr2=0x565339d88e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:37.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.801+0000 7efe7a59c640 1 --2- 172.21.15.2:0/1072207320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d987b0 0x565339d88e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.801+0000 7efe7a59c640 1 -- 172.21.15.2:0/1072207320 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565339c91ac0 con 0x565339d8f2b0 2024-03-15T13:03:37.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.801+0000 7efe7a59c640 1 --2- 172.21.15.2:0/1072207320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d8f2b0 0x565339d8b5f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7efe74097090 tx=0x7efe74005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f691ead1f17300 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:37.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.801+0000 7efe63fff640 1 -- 172.21.15.2:0/1072207320 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe74098cd0 con 0x565339d8f2b0 2024-03-15T13:03:37.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.801+0000 7efe63fff640 1 -- 172.21.15.2:0/1072207320 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efe7409ae10 con 0x565339d8f2b0 2024-03-15T13:03:37.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.801+0000 7efe63fff640 1 -- 172.21.15.2:0/1072207320 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe740a2eb0 con 0x565339d8f2b0 2024-03-15T13:03:37.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 -- 172.21.15.2:0/1072207320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d8f2b0 msgr2=0x565339d8b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:37.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 --2- 172.21.15.2:0/1072207320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d8f2b0 0x565339d8b5f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7efe74097090 tx=0x7efe74005e30 comp rx=0 tx=0).stop 2024-03-15T13:03:37.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 -- 172.21.15.2:0/1072207320 shutdown_connections 2024-03-15T13:03:37.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 --2- 172.21.15.2:0/1072207320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d987b0 0x565339d88e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 --2- 172.21.15.2:0/1072207320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d888d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 --2- 172.21.15.2:0/1072207320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d8f2b0 0x565339d8b5f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 -- 172.21.15.2:0/1072207320 >> 172.21.15.2:0/1072207320 conn(0x565339d89990 msgr2=0x565339d881c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:37.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 -- 172.21.15.2:0/1072207320 shutdown_connections 2024-03-15T13:03:37.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 -- 172.21.15.2:0/1072207320 wait complete. 2024-03-15T13:03:37.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 -- start start 2024-03-15T13:03:37.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 Processor -- start 2024-03-15T13:03:37.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d8f2b0 0x565339d9bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d9c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d987b0 0x565339d9c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x565339da0960 con 0x565339d987b0 2024-03-15T13:03:37.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x565339e03000 con 0x565339d8f2b0 2024-03-15T13:03:37.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x565339e03190 con 0x565339d8fbd0 2024-03-15T13:03:37.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe835f6740 10 client.0 _ll_register_callbacks cb 0x7efe740b1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:37.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe7ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d987b0 0x565339d9c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe7ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d987b0 0x565339d9c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35124/0 (socket says 172.21.15.2:35124) 2024-03-15T13:03:37.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe7ad9d640 1 -- 172.21.15.2:0/4197989429 learned_addr learned my addr 172.21.15.2:0/4197989429 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:37.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe7a59c640 1 --2- 172.21.15.2:0/4197989429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d8f2b0 0x565339d9bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.802+0000 7efe79d9b640 1 --2- 172.21.15.2:0/4197989429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d9c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.803+0000 7efe7ad9d640 1 -- 172.21.15.2:0/4197989429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 msgr2=0x565339d9c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:37.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.803+0000 7efe7ad9d640 1 --2- 172.21.15.2:0/4197989429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d9c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.803+0000 7efe7ad9d640 1 -- 172.21.15.2:0/4197989429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d8f2b0 msgr2=0x565339d9bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:37.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.803+0000 7efe7ad9d640 1 --2- 172.21.15.2:0/4197989429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d8f2b0 0x565339d9bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.803+0000 7efe7ad9d640 1 -- 172.21.15.2:0/4197989429 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565339cf5610 con 0x565339d987b0 2024-03-15T13:03:37.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.803+0000 7efe7ad9d640 1 --2- 172.21.15.2:0/4197989429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d987b0 0x565339d9c610 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efe6c004320 tx=0x7efe6c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc282f26f3d45e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:37.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.803+0000 7efe61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:37.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.803+0000 7efe61ffb640 1 -- 172.21.15.2:0/4197989429 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c016040 con 0x565339d987b0 2024-03-15T13:03:37.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.803+0000 7efe61ffb640 1 -- 172.21.15.2:0/4197989429 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7efe6c012bb0 con 0x565339d987b0 2024-03-15T13:03:37.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.803+0000 7efe835f6740 1 -- 172.21.15.2:0/4197989429 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565339a82890 con 0x565339d987b0 2024-03-15T13:03:37.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.803+0000 7efe61ffb640 1 -- 172.21.15.2:0/4197989429 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe6c0048a0 con 0x565339d987b0 2024-03-15T13:03:37.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.803+0000 7efe835f6740 10 client.17043 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:37.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.803+0000 7efe835f6740 1 -- 172.21.15.2:0/4197989429 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x565339cb76a0 con 0x565339d987b0 2024-03-15T13:03:37.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.803+0000 7efe61ffb640 1 -- 172.21.15.2:0/4197989429 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x565339a82890 con 0x565339d987b0 2024-03-15T13:03:37.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.804+0000 7efe61ffb640 1 -- 172.21.15.2:0/4197989429 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7efe6c01c3a0 con 0x565339d987b0 2024-03-15T13:03:37.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.804+0000 7efe835f6740 10 client.17043 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:37.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.804+0000 7efe835f6740 10 client.17043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:37.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.804+0000 7efe835f6740 20 client.17043 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:37.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.804+0000 7efe835f6740 1 --2- 172.21.15.2:0/4197989429 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x565339e04b70 0x565339e06f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.804+0000 7efe835f6740 4 client.17043 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:03:37.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.804+0000 7efe835f6740 1 -- 172.21.15.2:0/4197989429 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5653399e54e0 con 0x565339e04b70 2024-03-15T13:03:37.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.804+0000 7efe7a59c640 1 --2- 172.21.15.2:0/4197989429 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x565339e04b70 0x565339e06f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.804+0000 7efe7a59c640 1 --2- 172.21.15.2:0/4197989429 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x565339e04b70 0x565339e06f90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14443d32c735703 server_cookie=148c1b55a27cfdf1 in_seq=0 out_seq=0 2024-03-15T13:03:37.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.804+0000 7efe61ffb640 10 client.17043 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:37.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.804+0000 7efe61ffb640 1 -- 172.21.15.2:0/4197989429 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5653399e54e0 con 0x565339e04b70 2024-03-15T13:03:37.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.804+0000 7efe61ffb640 10 client.17043 handle_command_reply: tid=0 2024-03-15T13:03:37.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.804+0000 7efe835f6740 1 client.17043 shutdown 2024-03-15T13:03:37.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe835f6740 1 -- 172.21.15.2:0/4197989429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d987b0 msgr2=0x565339d9c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:37.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe835f6740 1 --2- 172.21.15.2:0/4197989429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d987b0 0x565339d9c610 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efe6c004320 tx=0x7efe6c003f90 comp rx=0 tx=0).stop 2024-03-15T13:03:37.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe835f6740 1 -- 172.21.15.2:0/4197989429 shutdown_connections 2024-03-15T13:03:37.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe835f6740 1 --2- 172.21.15.2:0/4197989429 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x565339e04b70 0x565339e06f90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe835f6740 1 --2- 172.21.15.2:0/4197989429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x565339d987b0 0x565339d9c610 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe835f6740 1 --2- 172.21.15.2:0/4197989429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565339d8fbd0 0x565339d9c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe835f6740 1 --2- 172.21.15.2:0/4197989429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565339d8f2b0 0x565339d9bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe61ffb640 0 client.17043 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:37.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe835f6740 1 -- 172.21.15.2:0/4197989429 >> 172.21.15.2:0/4197989429 conn(0x565339d89990 msgr2=0x565339d8b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:37.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe835f6740 1 -- 172.21.15.2:0/4197989429 shutdown_connections 2024-03-15T13:03:37.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe835f6740 1 -- 172.21.15.2:0/4197989429 wait complete. 2024-03-15T13:03:37.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe835f6740 20 client.17043 trim_cache size 0 max 16384 2024-03-15T13:03:37.797 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:03:37.797 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:03:37.797 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:03:37.797 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:03:37.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe8093e640 1 -- 172.21.15.2:0/2783623699 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efe640a1ba0 msgr2=0x7efe640a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:37.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe8093e640 1 --2- 172.21.15.2:0/2783623699 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efe640a1ba0 0x7efe640a4060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efe70000c00 tx=0x7efe70005fd0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe8093e640 1 -- 172.21.15.2:0/2783623699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c101460 msgr2=0x7efe7c19d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:37.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe8093e640 1 --2- 172.21.15.2:0/2783623699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c101460 0x7efe7c19d7a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7efe7400e9f0 tx=0x7efe7400edf0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe8093e640 1 -- 172.21.15.2:0/2783623699 shutdown_connections 2024-03-15T13:03:37.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe8093e640 1 --2- 172.21.15.2:0/2783623699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe7c107500 0x7efe7c197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe8093e640 1 --2- 172.21.15.2:0/2783623699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe7c101e20 0x7efe7c19dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe8093e640 1 --2- 172.21.15.2:0/2783623699 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efe640a1ba0 0x7efe640a4060 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe8093e640 1 --2- 172.21.15.2:0/2783623699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c101460 0x7efe7c19d7a0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe8093e640 1 -- 172.21.15.2:0/2783623699 >> 172.21.15.2:0/2783623699 conn(0x7efe7c0fcf90 msgr2=0x7efe7c0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:37.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe8093e640 1 -- 172.21.15.2:0/2783623699 shutdown_connections 2024-03-15T13:03:37.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.805+0000 7efe8093e640 1 -- 172.21.15.2:0/2783623699 wait complete. 2024-03-15T13:03:37.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:37 smithi002 ceph-mon[32060]: pgmap v144: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:03:37.801 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:03:37.802 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:03:37.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:37 smithi184 ceph-mon[35993]: pgmap v144: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:03:37.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:37 smithi149 ceph-mon[35927]: pgmap v144: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:03:37.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.919+0000 7ffa0456c640 1 -- 172.21.15.2:0/2872326240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9fc100600 msgr2=0x7ff9fc1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:37.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.919+0000 7ffa0456c640 1 --2- 172.21.15.2:0/2872326240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9fc100600 0x7ff9fc1009e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009900 tx=0x7ff9f802bc30 comp rx=0 tx=0).stop 2024-03-15T13:03:37.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.920+0000 7ffa0456c640 1 -- 172.21.15.2:0/2872326240 shutdown_connections 2024-03-15T13:03:37.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.920+0000 7ffa0456c640 1 --2- 172.21.15.2:0/2872326240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9fc1056d0 0x7ff9fc10d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.920+0000 7ffa0456c640 1 --2- 172.21.15.2:0/2872326240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9fc100fb0 0x7ff9fc104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.920+0000 7ffa0456c640 1 --2- 172.21.15.2:0/2872326240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9fc100600 0x7ff9fc1009e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.920+0000 7ffa0456c640 1 -- 172.21.15.2:0/2872326240 >> 172.21.15.2:0/2872326240 conn(0x7ff9fc076770 msgr2=0x7ff9fc076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:37.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.920+0000 7ffa0456c640 1 -- 172.21.15.2:0/2872326240 shutdown_connections 2024-03-15T13:03:37.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.920+0000 7ffa0456c640 1 -- 172.21.15.2:0/2872326240 wait complete. 2024-03-15T13:03:37.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.920+0000 7ffa0456c640 1 Processor -- start 2024-03-15T13:03:37.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.920+0000 7ffa0456c640 1 -- start start 2024-03-15T13:03:37.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.920+0000 7ffa0456c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9fc100600 0x7ff9fc10db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.920+0000 7ffa0456c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9fc1056d0 0x7ff9fc10e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.920+0000 7ffa0456c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9fc10e600 0x7ff9fc1a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.920+0000 7ffa0456c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc1a6280 con 0x7ff9fc10e600 2024-03-15T13:03:37.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.920+0000 7ffa0456c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc1a63f0 con 0x7ff9fc100600 2024-03-15T13:03:37.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.920+0000 7ffa0456c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc1a65b0 con 0x7ff9fc1056d0 2024-03-15T13:03:37.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ffa02b09640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9fc10e600 0x7ff9fc1a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ffa02308640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9fc100600 0x7ff9fc10db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ffa02308640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9fc100600 0x7ff9fc10db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:53988/0 (socket says 172.21.15.2:53988) 2024-03-15T13:03:37.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ffa01b07640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9fc1056d0 0x7ff9fc10e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ffa02308640 1 -- 172.21.15.2:0/2103687357 learned_addr learned my addr 172.21.15.2:0/2103687357 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:37.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ffa02308640 1 -- 172.21.15.2:0/2103687357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9fc1056d0 msgr2=0x7ff9fc10e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:37.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ffa02308640 1 --2- 172.21.15.2:0/2103687357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9fc1056d0 0x7ff9fc10e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ffa02308640 1 -- 172.21.15.2:0/2103687357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9fc10e600 msgr2=0x7ff9fc1a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:37.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ffa02308640 1 --2- 172.21.15.2:0/2103687357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9fc10e600 0x7ff9fc1a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:37.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ffa02308640 1 -- 172.21.15.2:0/2103687357 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f8009560 con 0x7ff9fc100600 2024-03-15T13:03:37.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ffa01b07640 1 --2- 172.21.15.2:0/2103687357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9fc1056d0 0x7ff9fc10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:37.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ffa02308640 1 --2- 172.21.15.2:0/2103687357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9fc100600 0x7ff9fc10db80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009230 tx=0x7ff9f802cac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:37.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ff9f37fe640 1 -- 172.21.15.2:0/2103687357 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8004250 con 0x7ff9fc100600 2024-03-15T13:03:37.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ffa0456c640 1 -- 172.21.15.2:0/2103687357 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc1a6830 con 0x7ff9fc100600 2024-03-15T13:03:37.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ff9f37fe640 1 -- 172.21.15.2:0/2103687357 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff9f80043b0 con 0x7ff9fc100600 2024-03-15T13:03:37.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ff9f37fe640 1 -- 172.21.15.2:0/2103687357 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8003bf0 con 0x7ff9fc100600 2024-03-15T13:03:37.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.921+0000 7ffa0456c640 1 -- 172.21.15.2:0/2103687357 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9fc1a6d40 con 0x7ff9fc100600 2024-03-15T13:03:37.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.922+0000 7ffa0456c640 1 -- 172.21.15.2:0/2103687357 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9fc101ad0 con 0x7ff9fc100600 2024-03-15T13:03:37.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.924+0000 7ff9f37fe640 1 -- 172.21.15.2:0/2103687357 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7ff9f8003d50 con 0x7ff9fc100600 2024-03-15T13:03:37.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.924+0000 7ff9f37fe640 1 --2- 172.21.15.2:0/2103687357 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff9e80a1bd0 0x7ff9e80a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:37.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.924+0000 7ffa01b07640 1 --2- 172.21.15.2:0/2103687357 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff9e80a1bd0 0x7ff9e80a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:37.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.924+0000 7ff9f37fe640 1 -- 172.21.15.2:0/2103687357 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7ff9f80f0200 con 0x7ff9fc100600 2024-03-15T13:03:37.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.925+0000 7ffa01b07640 1 --2- 172.21.15.2:0/2103687357 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff9e80a1bd0 0x7ff9e80a4090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc10f610 tx=0x7ff9ec005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:37.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:37.932+0000 7ff9f37fe640 1 -- 172.21.15.2:0/2103687357 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff9f80b8e30 con 0x7ff9fc100600 2024-03-15T13:03:38.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.155+0000 7ffa0456c640 1 -- 172.21.15.2:0/2103687357 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9fc108900 con 0x7ff9fc100600 2024-03-15T13:03:38.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.156+0000 7ff9f37fe640 1 -- 172.21.15.2:0/2103687357 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11468 (secure 0 0 0) 0x7ff9f80bc4f0 con 0x7ff9fc100600 2024-03-15T13:03:38.141 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:03:38.141 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:03:29.311079+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24464,24500,24460,24539,24434]},"id":1}]} 2024-03-15T13:03:38.142 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 40 2024-03-15T13:03:38.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.158+0000 7ffa0456c640 1 -- 172.21.15.2:0/2103687357 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff9e80a1bd0 msgr2=0x7ff9e80a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.158+0000 7ffa0456c640 1 --2- 172.21.15.2:0/2103687357 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff9e80a1bd0 0x7ff9e80a4090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc10f610 tx=0x7ff9ec005f90 comp rx=0 tx=0).stop 2024-03-15T13:03:38.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.158+0000 7ffa0456c640 1 -- 172.21.15.2:0/2103687357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9fc100600 msgr2=0x7ff9fc10db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.158+0000 7ffa0456c640 1 --2- 172.21.15.2:0/2103687357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9fc100600 0x7ff9fc10db80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009230 tx=0x7ff9f802cac0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.158+0000 7ffa0456c640 1 -- 172.21.15.2:0/2103687357 shutdown_connections 2024-03-15T13:03:38.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.158+0000 7ffa0456c640 1 --2- 172.21.15.2:0/2103687357 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff9e80a1bd0 0x7ff9e80a4090 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.158+0000 7ffa0456c640 1 --2- 172.21.15.2:0/2103687357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9fc10e600 0x7ff9fc1a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.158+0000 7ffa0456c640 1 --2- 172.21.15.2:0/2103687357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9fc1056d0 0x7ff9fc10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.158+0000 7ffa0456c640 1 --2- 172.21.15.2:0/2103687357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9fc100600 0x7ff9fc10db80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.158+0000 7ffa0456c640 1 -- 172.21.15.2:0/2103687357 >> 172.21.15.2:0/2103687357 conn(0x7ff9fc076770 msgr2=0x7ff9fc0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.158+0000 7ffa0456c640 1 -- 172.21.15.2:0/2103687357 shutdown_connections 2024-03-15T13:03:38.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.158+0000 7ffa0456c640 1 -- 172.21.15.2:0/2103687357 wait complete. 2024-03-15T13:03:38.154 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:03:38.155 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:03:38.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.274+0000 7fa4b1864640 1 -- 172.21.15.2:0/4243073196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4ac104b20 msgr2=0x7fa4ac104f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.274+0000 7fa4b1864640 1 --2- 172.21.15.2:0/4243073196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4ac104b20 0x7fa4ac104f00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009900 tx=0x7fa49c02bc30 comp rx=0 tx=0).stop 2024-03-15T13:03:38.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.274+0000 7fa4b1864640 1 -- 172.21.15.2:0/4243073196 shutdown_connections 2024-03-15T13:03:38.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.274+0000 7fa4b1864640 1 --2- 172.21.15.2:0/4243073196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4ac100750 0x7fa4ac10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.274+0000 7fa4b1864640 1 --2- 172.21.15.2:0/4243073196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4ac0ffdb0 0x7fa4ac100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.274+0000 7fa4b1864640 1 --2- 172.21.15.2:0/4243073196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4ac104b20 0x7fa4ac104f00 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.274+0000 7fa4b1864640 1 -- 172.21.15.2:0/4243073196 >> 172.21.15.2:0/4243073196 conn(0x7fa4ac075730 msgr2=0x7fa4ac075b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.274+0000 7fa4b1864640 1 -- 172.21.15.2:0/4243073196 shutdown_connections 2024-03-15T13:03:38.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.274+0000 7fa4b1864640 1 -- 172.21.15.2:0/4243073196 wait complete. 2024-03-15T13:03:38.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4b1864640 1 Processor -- start 2024-03-15T13:03:38.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4b1864640 1 -- start start 2024-03-15T13:03:38.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4b1864640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4ac0ffdb0 0x7fa4ac101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4b1864640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4ac100750 0x7fa4ac101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4b1864640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4ac104b20 0x7fa4ac102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4b1864640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac1a5e40 con 0x7fa4ac100750 2024-03-15T13:03:38.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4b1864640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac1a5fb0 con 0x7fa4ac104b20 2024-03-15T13:03:38.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4b1864640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac1a6170 con 0x7fa4ac0ffdb0 2024-03-15T13:03:38.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4a3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4ac100750 0x7fa4ac101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4a3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4ac100750 0x7fa4ac101d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51274/0 (socket says 172.21.15.2:51274) 2024-03-15T13:03:38.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4a3fff640 1 -- 172.21.15.2:0/1091792987 learned_addr learned my addr 172.21.15.2:0/1091792987 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4aaffd640 1 --2- 172.21.15.2:0/1091792987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4ac0ffdb0 0x7fa4ac101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4ab7fe640 1 --2- 172.21.15.2:0/1091792987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4ac104b20 0x7fa4ac102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4a3fff640 1 -- 172.21.15.2:0/1091792987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4ac0ffdb0 msgr2=0x7fa4ac101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4a3fff640 1 --2- 172.21.15.2:0/1091792987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4ac0ffdb0 0x7fa4ac101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4a3fff640 1 -- 172.21.15.2:0/1091792987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4ac104b20 msgr2=0x7fa4ac102290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4a3fff640 1 --2- 172.21.15.2:0/1091792987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4ac104b20 0x7fa4ac102290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.275+0000 7fa4a3fff640 1 -- 172.21.15.2:0/1091792987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c009560 con 0x7fa4ac100750 2024-03-15T13:03:38.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.276+0000 7fa4a3fff640 1 --2- 172.21.15.2:0/1091792987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4ac100750 0x7fa4ac101d50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa49400ea20 tx=0x7fa49400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.276+0000 7fa4b0862640 1 -- 172.21.15.2:0/1091792987 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49400cd40 con 0x7fa4ac100750 2024-03-15T13:03:38.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.276+0000 7fa4b0862640 1 -- 172.21.15.2:0/1091792987 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa4940168e0 con 0x7fa4ac100750 2024-03-15T13:03:38.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.276+0000 7fa4b1864640 1 -- 172.21.15.2:0/1091792987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ac1a6450 con 0x7fa4ac100750 2024-03-15T13:03:38.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.276+0000 7fa4b0862640 1 -- 172.21.15.2:0/1091792987 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49400b860 con 0x7fa4ac100750 2024-03-15T13:03:38.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.276+0000 7fa4b1864640 1 -- 172.21.15.2:0/1091792987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ac1a6960 con 0x7fa4ac100750 2024-03-15T13:03:38.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.278+0000 7fa4b0862640 1 -- 172.21.15.2:0/1091792987 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fa494016a50 con 0x7fa4ac100750 2024-03-15T13:03:38.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.278+0000 7fa4b4440740 1 Processor -- start 2024-03-15T13:03:38.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.278+0000 7fa4b4440740 1 -- start start 2024-03-15T13:03:38.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.278+0000 7fa4b4440740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6f5250 0x55ae1a6f5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.278+0000 7fa4b4440740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.278+0000 7fa4b4440740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6fe750 0x55ae1a700b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.278+0000 7fa4b4440740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1a6b0960 con 0x55ae1a6f5b70 2024-03-15T13:03:38.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.278+0000 7fa4b4440740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1a664310 con 0x55ae1a6fe750 2024-03-15T13:03:38.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.278+0000 7fa4b4440740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1a661840 con 0x55ae1a6f5250 2024-03-15T13:03:38.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.278+0000 7fa4b0862640 1 --2- 172.21.15.2:0/1091792987 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa4980a1bd0 0x7fa4980a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4aaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4a3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6fe750 0x55ae1a700b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4aaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fe210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51286/0 (socket says 172.21.15.2:51286) 2024-03-15T13:03:38.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4aaffd640 1 -- 172.21.15.2:0/256826267 learned_addr learned my addr 172.21.15.2:0/256826267 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4ab7fe640 1 --2- 172.21.15.2:0/256826267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6f5250 0x55ae1a6f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4ab7fe640 1 --2- 172.21.15.2:0/1091792987 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa4980a1bd0 0x7fa4980a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4b0862640 1 -- 172.21.15.2:0/1091792987 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa4940cd990 con 0x7fa4ac100750 2024-03-15T13:03:38.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4aaffd640 1 -- 172.21.15.2:0/256826267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6f5250 msgr2=0x55ae1a6f5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4aaffd640 1 --2- 172.21.15.2:0/256826267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6f5250 0x55ae1a6f5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4aaffd640 1 -- 172.21.15.2:0/256826267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6fe750 msgr2=0x55ae1a700b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4aaffd640 1 --2- 172.21.15.2:0/256826267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6fe750 0x55ae1a700b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4aaffd640 1 -- 172.21.15.2:0/256826267 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae1a61f060 con 0x55ae1a6f5b70 2024-03-15T13:03:38.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4a3fff640 1 --2- 172.21.15.2:0/256826267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6fe750 0x55ae1a700b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:38.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4aaffd640 1 --2- 172.21.15.2:0/256826267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fe210 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa49c036040 tx=0x7fa49c02c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff34438983bfa70 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4ab7fe640 1 --2- 172.21.15.2:0/1091792987 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa4980a1bd0 0x7fa4980a4090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4005fd0 tx=0x7fa4a4009500 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4a1ffb640 1 -- 172.21.15.2:0/256826267 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c004780 con 0x55ae1a6f5b70 2024-03-15T13:03:38.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4a1ffb640 1 -- 172.21.15.2:0/256826267 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa49c0048e0 con 0x55ae1a6f5b70 2024-03-15T13:03:38.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.279+0000 7fa4a1ffb640 1 -- 172.21.15.2:0/256826267 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c003850 con 0x55ae1a6f5b70 2024-03-15T13:03:38.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.280+0000 7fa4b4440740 1 -- 172.21.15.2:0/256826267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5b70 msgr2=0x55ae1a6fe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.280+0000 7fa4b4440740 1 --2- 172.21.15.2:0/256826267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fe210 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa49c036040 tx=0x7fa49c02c6d0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.280+0000 7fa4b4440740 1 -- 172.21.15.2:0/256826267 shutdown_connections 2024-03-15T13:03:38.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.280+0000 7fa4b4440740 1 --2- 172.21.15.2:0/256826267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6fe750 0x55ae1a700b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.280+0000 7fa4b4440740 1 --2- 172.21.15.2:0/256826267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fe210 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.280+0000 7fa4b4440740 1 --2- 172.21.15.2:0/256826267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6f5250 0x55ae1a6f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.280+0000 7fa4b4440740 1 -- 172.21.15.2:0/256826267 >> 172.21.15.2:0/256826267 conn(0x55ae1a6ef980 msgr2=0x55ae1a6efd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.280+0000 7fa4b4440740 1 -- 172.21.15.2:0/256826267 shutdown_connections 2024-03-15T13:03:38.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.280+0000 7fa4b4440740 1 -- 172.21.15.2:0/256826267 wait complete. 2024-03-15T13:03:38.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.280+0000 7fa4b4440740 1 -- start start 2024-03-15T13:03:38.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4b4440740 1 Processor -- start 2024-03-15T13:03:38.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4b4440740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6f5250 0x55ae1a6fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4b4440740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4b4440740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6fe750 0x55ae1a6fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4b4440740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1a65d020 con 0x55ae1a6f5b70 2024-03-15T13:03:38.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4b4440740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1a330430 con 0x55ae1a6f5250 2024-03-15T13:03:38.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4b4440740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1a2ece00 con 0x55ae1a6fe750 2024-03-15T13:03:38.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4b4440740 10 client.0 _ll_register_callbacks cb 0x55ae1a604cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:38.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4a3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4a3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51292/0 (socket says 172.21.15.2:51292) 2024-03-15T13:03:38.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4a3fff640 1 -- 172.21.15.2:0/3679653083 learned_addr learned my addr 172.21.15.2:0/3679653083 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4aaffd640 1 --2- 172.21.15.2:0/3679653083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6f5250 0x55ae1a6fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4ab7fe640 1 --2- 172.21.15.2:0/3679653083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6fe750 0x55ae1a6fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4a3fff640 1 -- 172.21.15.2:0/3679653083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6fe750 msgr2=0x55ae1a6fb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4a3fff640 1 --2- 172.21.15.2:0/3679653083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6fe750 0x55ae1a6fb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4a3fff640 1 -- 172.21.15.2:0/3679653083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6f5250 msgr2=0x55ae1a6fe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4a3fff640 1 --2- 172.21.15.2:0/3679653083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6f5250 0x55ae1a6fe0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4a3fff640 1 -- 172.21.15.2:0/3679653083 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ac10d320 con 0x55ae1a6f5b70 2024-03-15T13:03:38.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa4a3fff640 1 --2- 172.21.15.2:0/3679653083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fb470 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa494000c00 tx=0x7fa49409a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56260a36de3a77dd server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa48affd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:38.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa48affd640 1 -- 172.21.15.2:0/3679653083 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4940a2c60 con 0x55ae1a6f5b70 2024-03-15T13:03:38.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.281+0000 7fa48affd640 1 -- 172.21.15.2:0/3679653083 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa49409a320 con 0x55ae1a6f5b70 2024-03-15T13:03:38.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.282+0000 7fa4b4440740 1 -- 172.21.15.2:0/3679653083 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae1a35c7e0 con 0x55ae1a6f5b70 2024-03-15T13:03:38.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.282+0000 7fa48affd640 1 -- 172.21.15.2:0/3679653083 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa49409e3c0 con 0x55ae1a6f5b70 2024-03-15T13:03:38.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.282+0000 7fa4b4440740 10 client.17061 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:38.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.282+0000 7fa4b4440740 1 -- 172.21.15.2:0/3679653083 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ae1a65b6a0 con 0x55ae1a6f5b70 2024-03-15T13:03:38.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.282+0000 7fa48affd640 1 -- 172.21.15.2:0/3679653083 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55ae1a35c7e0 con 0x55ae1a6f5b70 2024-03-15T13:03:38.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.282+0000 7fa48affd640 1 -- 172.21.15.2:0/3679653083 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7fa494096070 con 0x55ae1a6f5b70 2024-03-15T13:03:38.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.282+0000 7fa4b4440740 10 client.17061 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:38.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.282+0000 7fa4b4440740 10 client.17061 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:03:38.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.282+0000 7fa4b4440740 20 client.17061 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:38.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.282+0000 7fa4b4440740 1 --2- 172.21.15.2:0/3679653083 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ae1a70e7c0 0x55ae1a710be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.282+0000 7fa4b4440740 4 client.17061 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:38.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.282+0000 7fa4b4440740 1 -- 172.21.15.2:0/3679653083 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae1a399720 con 0x55ae1a70e7c0 2024-03-15T13:03:38.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.282+0000 7fa4aaffd640 1 --2- 172.21.15.2:0/3679653083 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ae1a70e7c0 0x55ae1a710be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.283+0000 7fa48affd640 10 client.17061 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:03:38.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.283+0000 7fa4aaffd640 1 --2- 172.21.15.2:0/3679653083 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ae1a70e7c0 0x55ae1a710be0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2f079dadfde451d6 server_cookie=8a0196b734978f00 in_seq=0 out_seq=0 2024-03-15T13:03:38.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.283+0000 7fa48affd640 1 -- 172.21.15.2:0/3679653083 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ae1a399720 con 0x55ae1a70e7c0 2024-03-15T13:03:38.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.283+0000 7fa48affd640 10 client.17061 handle_command_reply: tid=0 2024-03-15T13:03:38.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.283+0000 7fa4b4440740 1 client.17061 shutdown 2024-03-15T13:03:38.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.283+0000 7fa4b4440740 1 -- 172.21.15.2:0/3679653083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5b70 msgr2=0x55ae1a6fb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.283+0000 7fa4b4440740 1 --2- 172.21.15.2:0/3679653083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fb470 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa494000c00 tx=0x7fa49409a0f0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.283+0000 7fa4b4440740 1 -- 172.21.15.2:0/3679653083 shutdown_connections 2024-03-15T13:03:38.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.283+0000 7fa4b4440740 1 --2- 172.21.15.2:0/3679653083 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ae1a70e7c0 0x55ae1a710be0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.283+0000 7fa4b4440740 1 --2- 172.21.15.2:0/3679653083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6fe750 0x55ae1a6fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.283+0000 7fa4b4440740 1 --2- 172.21.15.2:0/3679653083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fb470 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.283+0000 7fa48affd640 0 client.17061 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:03:38.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.283+0000 7fa4b4440740 1 --2- 172.21.15.2:0/3679653083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6f5250 0x55ae1a6fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.283+0000 7fa4b4440740 1 -- 172.21.15.2:0/3679653083 >> 172.21.15.2:0/3679653083 conn(0x55ae1a6ef980 msgr2=0x55ae1a6f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.284+0000 7fa4b4440740 1 -- 172.21.15.2:0/3679653083 shutdown_connections 2024-03-15T13:03:38.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.284+0000 7fa4b4440740 1 -- 172.21.15.2:0/3679653083 wait complete. 2024-03-15T13:03:38.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.284+0000 7fa4b4440740 20 client.17061 trim_cache size 0 max 16384 2024-03-15T13:03:38.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.296+0000 7fa4b4440740 1 Processor -- start 2024-03-15T13:03:38.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.296+0000 7fa4b4440740 1 -- start start 2024-03-15T13:03:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.296+0000 7fa4b4440740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5250 0x55ae1a6ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.296+0000 7fa4b4440740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.296+0000 7fa4b4440740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6fe750 0x55ae1a6fe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.296+0000 7fa4b4440740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1a5e2130 con 0x55ae1a6f5250 2024-03-15T13:03:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.296+0000 7fa4b4440740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1a70e5f0 con 0x55ae1a6f5b70 2024-03-15T13:03:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.296+0000 7fa4b4440740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1a6f0980 con 0x55ae1a6fe750 2024-03-15T13:03:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.296+0000 7fa4aaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5250 0x55ae1a6ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.296+0000 7fa4aaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5250 0x55ae1a6ee2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51302/0 (socket says 172.21.15.2:51302) 2024-03-15T13:03:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.296+0000 7fa4aaffd640 1 -- 172.21.15.2:0/25947003 learned_addr learned my addr 172.21.15.2:0/25947003 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.296+0000 7fa4a3fff640 1 --2- 172.21.15.2:0/25947003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.296+0000 7fa4ab7fe640 1 --2- 172.21.15.2:0/25947003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6fe750 0x55ae1a6fe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4aaffd640 1 -- 172.21.15.2:0/25947003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6fe750 msgr2=0x55ae1a6fe070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4aaffd640 1 --2- 172.21.15.2:0/25947003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6fe750 0x55ae1a6fe070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4aaffd640 1 -- 172.21.15.2:0/25947003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6f5b70 msgr2=0x55ae1a6fdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4aaffd640 1 --2- 172.21.15.2:0/25947003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4aaffd640 1 -- 172.21.15.2:0/25947003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae1a5f7c20 con 0x55ae1a6f5250 2024-03-15T13:03:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4aaffd640 1 --2- 172.21.15.2:0/25947003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5250 0x55ae1a6ee2f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa49c000c00 tx=0x7fa49c042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa065f146896fde server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4a17fa640 1 -- 172.21.15.2:0/25947003 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c02ca10 con 0x55ae1a6f5250 2024-03-15T13:03:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4a17fa640 1 -- 172.21.15.2:0/25947003 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa49c004510 con 0x55ae1a6f5250 2024-03-15T13:03:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4a17fa640 1 -- 172.21.15.2:0/25947003 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c003ac0 con 0x55ae1a6f5250 2024-03-15T13:03:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4b4440740 1 -- 172.21.15.2:0/25947003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5250 msgr2=0x55ae1a6ee2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4b4440740 1 --2- 172.21.15.2:0/25947003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5250 0x55ae1a6ee2f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa49c000c00 tx=0x7fa49c042040 comp rx=0 tx=0).stop 2024-03-15T13:03:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4b4440740 1 -- 172.21.15.2:0/25947003 shutdown_connections 2024-03-15T13:03:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4b4440740 1 --2- 172.21.15.2:0/25947003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6fe750 0x55ae1a6fe070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4b4440740 1 --2- 172.21.15.2:0/25947003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6f5b70 0x55ae1a6fdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4b4440740 1 --2- 172.21.15.2:0/25947003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5250 0x55ae1a6ee2f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.297+0000 7fa4b4440740 1 -- 172.21.15.2:0/25947003 >> 172.21.15.2:0/25947003 conn(0x55ae1a6ef980 msgr2=0x55ae1a6f6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.298+0000 7fa4b4440740 1 -- 172.21.15.2:0/25947003 shutdown_connections 2024-03-15T13:03:38.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.298+0000 7fa4b4440740 1 -- 172.21.15.2:0/25947003 wait complete. 2024-03-15T13:03:38.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.298+0000 7fa4b4440740 1 -- start start 2024-03-15T13:03:38.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.298+0000 7fa4b4440740 1 Processor -- start 2024-03-15T13:03:38.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.298+0000 7fa4b4440740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5250 0x55ae1a705c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.298+0000 7fa4b4440740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6f5b70 0x55ae1a706180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.298+0000 7fa4b4440740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6fe750 0x55ae1a7020e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.298+0000 7fa4b4440740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1a7027b0 con 0x55ae1a6f5250 2024-03-15T13:03:38.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.298+0000 7fa4b4440740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1a702920 con 0x55ae1a6fe750 2024-03-15T13:03:38.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.298+0000 7fa4b4440740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1a702ae0 con 0x55ae1a6f5b70 2024-03-15T13:03:38.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.298+0000 7fa4b4440740 10 client.0 _ll_register_callbacks cb 0x7fa494016d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:38.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.298+0000 7fa4aaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5250 0x55ae1a705c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.298+0000 7fa4aaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5250 0x55ae1a705c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51304/0 (socket says 172.21.15.2:51304) 2024-03-15T13:03:38.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.298+0000 7fa4aaffd640 1 -- 172.21.15.2:0/3734348018 learned_addr learned my addr 172.21.15.2:0/3734348018 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.298+0000 7fa4a3fff640 1 --2- 172.21.15.2:0/3734348018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6f5b70 0x55ae1a706180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.299+0000 7fa4ab7fe640 1 --2- 172.21.15.2:0/3734348018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6fe750 0x55ae1a7020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.299+0000 7fa4aaffd640 1 -- 172.21.15.2:0/3734348018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6f5b70 msgr2=0x55ae1a706180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.299+0000 7fa4aaffd640 1 --2- 172.21.15.2:0/3734348018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6f5b70 0x55ae1a706180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.299+0000 7fa4aaffd640 1 -- 172.21.15.2:0/3734348018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6fe750 msgr2=0x55ae1a7020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.299+0000 7fa4aaffd640 1 --2- 172.21.15.2:0/3734348018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6fe750 0x55ae1a7020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.299+0000 7fa4aaffd640 1 -- 172.21.15.2:0/3734348018 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae1a61f060 con 0x55ae1a6f5250 2024-03-15T13:03:38.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.299+0000 7fa4aaffd640 1 --2- 172.21.15.2:0/3734348018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5250 0x55ae1a705c40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa49c005ab0 tx=0x7fa49c02c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dadf1f157184d78 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.299+0000 7fa48affd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:38.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.299+0000 7fa48affd640 1 -- 172.21.15.2:0/3734348018 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c02ca10 con 0x55ae1a6f5250 2024-03-15T13:03:38.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.299+0000 7fa48affd640 1 -- 172.21.15.2:0/3734348018 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa49c02cb70 con 0x55ae1a6f5250 2024-03-15T13:03:38.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.299+0000 7fa4b4440740 1 -- 172.21.15.2:0/3734348018 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa49c0048e0 con 0x55ae1a6f5250 2024-03-15T13:03:38.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.299+0000 7fa48affd640 1 -- 172.21.15.2:0/3734348018 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa49c0035f0 con 0x55ae1a6f5250 2024-03-15T13:03:38.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.299+0000 7fa4b4440740 10 client.17067 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:38.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.299+0000 7fa4b4440740 1 -- 172.21.15.2:0/3734348018 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ae1a61d8c0 con 0x55ae1a6f5250 2024-03-15T13:03:38.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.299+0000 7fa48affd640 1 -- 172.21.15.2:0/3734348018 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa49c065020 con 0x55ae1a6f5250 2024-03-15T13:03:38.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.300+0000 7fa48affd640 1 -- 172.21.15.2:0/3734348018 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7fa49c062070 con 0x55ae1a6f5250 2024-03-15T13:03:38.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.300+0000 7fa4b4440740 10 client.17067 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:38.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.300+0000 7fa4b4440740 10 client.17067 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:03:38.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.300+0000 7fa4b4440740 20 client.17067 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:38.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.300+0000 7fa4b4440740 1 --2- 172.21.15.2:0/3734348018 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ae1a76aeb0 0x55ae1a76d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.300+0000 7fa4b4440740 4 client.17067 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:03:38.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.300+0000 7fa4b4440740 1 -- 172.21.15.2:0/3734348018 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ae1a3e8890 con 0x55ae1a76aeb0 2024-03-15T13:03:38.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.300+0000 7fa4a3fff640 1 --2- 172.21.15.2:0/3734348018 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ae1a76aeb0 0x55ae1a76d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.300+0000 7fa4a3fff640 1 --2- 172.21.15.2:0/3734348018 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ae1a76aeb0 0x55ae1a76d2d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a797aefa10653b5d server_cookie=2436cbc3821c9975 in_seq=0 out_seq=0 2024-03-15T13:03:38.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.300+0000 7fa48affd640 10 client.17067 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:03:38.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.300+0000 7fa48affd640 1 -- 172.21.15.2:0/3734348018 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ae1a3e8890 con 0x55ae1a76aeb0 2024-03-15T13:03:38.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.300+0000 7fa48affd640 10 client.17067 handle_command_reply: tid=0 2024-03-15T13:03:38.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.300+0000 7fa4b4440740 1 client.17067 shutdown 2024-03-15T13:03:38.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.300+0000 7fa4b4440740 1 -- 172.21.15.2:0/3734348018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5250 msgr2=0x55ae1a705c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.300+0000 7fa4b4440740 1 --2- 172.21.15.2:0/3734348018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5250 0x55ae1a705c40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa49c005ab0 tx=0x7fa49c02c3c0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b4440740 1 -- 172.21.15.2:0/3734348018 shutdown_connections 2024-03-15T13:03:38.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b4440740 1 --2- 172.21.15.2:0/3734348018 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55ae1a76aeb0 0x55ae1a76d2d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b4440740 1 --2- 172.21.15.2:0/3734348018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae1a6fe750 0x55ae1a7020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b4440740 1 --2- 172.21.15.2:0/3734348018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae1a6f5b70 0x55ae1a706180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa48affd640 0 client.17067 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:03:38.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b4440740 1 --2- 172.21.15.2:0/3734348018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae1a6f5250 0x55ae1a705c40 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b4440740 1 -- 172.21.15.2:0/3734348018 >> 172.21.15.2:0/3734348018 conn(0x55ae1a6ef980 msgr2=0x55ae1a6fad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b4440740 1 -- 172.21.15.2:0/3734348018 shutdown_connections 2024-03-15T13:03:38.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b4440740 1 -- 172.21.15.2:0/3734348018 wait complete. 2024-03-15T13:03:38.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b4440740 20 client.17067 trim_cache size 0 max 16384 2024-03-15T13:03:38.310 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:03:38.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b1864640 1 -- 172.21.15.2:0/1091792987 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa4980a1bd0 msgr2=0x7fa4980a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b1864640 1 --2- 172.21.15.2:0/1091792987 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa4980a1bd0 0x7fa4980a4090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4005fd0 tx=0x7fa4a4009500 comp rx=0 tx=0).stop 2024-03-15T13:03:38.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b1864640 1 -- 172.21.15.2:0/1091792987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4ac100750 msgr2=0x7fa4ac101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b1864640 1 --2- 172.21.15.2:0/1091792987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4ac100750 0x7fa4ac101d50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa49400ea20 tx=0x7fa49400eee0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b1864640 1 -- 172.21.15.2:0/1091792987 shutdown_connections 2024-03-15T13:03:38.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b1864640 1 --2- 172.21.15.2:0/1091792987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4ac104b20 0x7fa4ac102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b1864640 1 --2- 172.21.15.2:0/1091792987 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa4980a1bd0 0x7fa4980a4090 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b1864640 1 --2- 172.21.15.2:0/1091792987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4ac100750 0x7fa4ac101d50 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b1864640 1 --2- 172.21.15.2:0/1091792987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4ac0ffdb0 0x7fa4ac101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b1864640 1 -- 172.21.15.2:0/1091792987 >> 172.21.15.2:0/1091792987 conn(0x7fa4ac075730 msgr2=0x7fa4ac0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b1864640 1 -- 172.21.15.2:0/1091792987 shutdown_connections 2024-03-15T13:03:38.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.301+0000 7fa4b1864640 1 -- 172.21.15.2:0/1091792987 wait complete. 2024-03-15T13:03:38.312 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:03:38.312 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:03:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.432+0000 7f47509a0640 1 -- 172.21.15.2:0/4066470644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f474c0fff50 msgr2=0x7f474c100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.432+0000 7f47509a0640 1 --2- 172.21.15.2:0/4066470644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f474c0fff50 0x7f474c100330 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4740009930 tx=0x7f474002bc20 comp rx=0 tx=0).stop 2024-03-15T13:03:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.432+0000 7f47509a0640 1 -- 172.21.15.2:0/4066470644 shutdown_connections 2024-03-15T13:03:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.432+0000 7f47509a0640 1 --2- 172.21.15.2:0/4066470644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f474c106710 0x7f474c10e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.432+0000 7f47509a0640 1 --2- 172.21.15.2:0/4066470644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f474c100900 0x7f474c106030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.432+0000 7f47509a0640 1 --2- 172.21.15.2:0/4066470644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f474c0fff50 0x7f474c100330 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.432+0000 7f47509a0640 1 -- 172.21.15.2:0/4066470644 >> 172.21.15.2:0/4066470644 conn(0x7f474c0755d0 msgr2=0x7f474c0759e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.432+0000 7f47509a0640 1 -- 172.21.15.2:0/4066470644 shutdown_connections 2024-03-15T13:03:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.432+0000 7f47509a0640 1 -- 172.21.15.2:0/4066470644 wait complete. 2024-03-15T13:03:38.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f47509a0640 1 Processor -- start 2024-03-15T13:03:38.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f47509a0640 1 -- start start 2024-03-15T13:03:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f47509a0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f474c0fff50 0x7f474c19d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f47509a0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f474c106710 0x7f474c19dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f47509a0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f474c197940 0x7f474c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f47509a0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474c198330 con 0x7f474c0fff50 2024-03-15T13:03:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f47509a0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474c1984a0 con 0x7f474c106710 2024-03-15T13:03:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f47509a0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474c198660 con 0x7f474c197940 2024-03-15T13:03:38.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f474b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f474c0fff50 0x7f474c19d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f474b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f474c0fff50 0x7f474c19d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51314/0 (socket says 172.21.15.2:51314) 2024-03-15T13:03:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f474b7fe640 1 -- 172.21.15.2:0/64544994 learned_addr learned my addr 172.21.15.2:0/64544994 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f474affd640 1 --2- 172.21.15.2:0/64544994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f474c106710 0x7f474c19dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f474bfff640 1 --2- 172.21.15.2:0/64544994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f474c197940 0x7f474c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f474b7fe640 1 -- 172.21.15.2:0/64544994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f474c197940 msgr2=0x7f474c197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f474b7fe640 1 --2- 172.21.15.2:0/64544994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f474c197940 0x7f474c197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f474b7fe640 1 -- 172.21.15.2:0/64544994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f474c106710 msgr2=0x7f474c19dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f474b7fe640 1 --2- 172.21.15.2:0/64544994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f474c106710 0x7f474c19dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f474b7fe640 1 -- 172.21.15.2:0/64544994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4740009560 con 0x7f474c0fff50 2024-03-15T13:03:38.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.433+0000 7f474bfff640 1 --2- 172.21.15.2:0/64544994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f474c197940 0x7f474c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:03:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.434+0000 7f474b7fe640 1 --2- 172.21.15.2:0/64544994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f474c0fff50 0x7f474c19d790 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4740009900 tx=0x7f474002ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.434+0000 7f4748ff9640 1 -- 172.21.15.2:0/64544994 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474002cc00 con 0x7f474c0fff50 2024-03-15T13:03:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.434+0000 7f4748ff9640 1 -- 172.21.15.2:0/64544994 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f47400054d0 con 0x7f474c0fff50 2024-03-15T13:03:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.434+0000 7f47509a0640 1 -- 172.21.15.2:0/64544994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f474c1a4650 con 0x7f474c0fff50 2024-03-15T13:03:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.434+0000 7f4748ff9640 1 -- 172.21.15.2:0/64544994 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740003c90 con 0x7f474c0fff50 2024-03-15T13:03:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.434+0000 7f47509a0640 1 -- 172.21.15.2:0/64544994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f474c1a4b50 con 0x7f474c0fff50 2024-03-15T13:03:38.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.436+0000 7f4748ff9640 1 -- 172.21.15.2:0/64544994 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f4740005000 con 0x7f474c0fff50 2024-03-15T13:03:38.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.436+0000 7f47547dd740 1 Processor -- start 2024-03-15T13:03:38.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.436+0000 7f47547dd740 1 -- start start 2024-03-15T13:03:38.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.436+0000 7f4748ff9640 1 --2- 172.21.15.2:0/64544994 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f47340a1c20 0x7f47340a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f47547dd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b233d440 0x5609b233d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f47547dd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b233dd60 0x5609b2346400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f474bfff640 1 --2- 172.21.15.2:0/64544994 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f47340a1c20 0x7f47340a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f47547dd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b2346940 0x5609b2348d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f47547dd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b22f8960 con 0x5609b233d440 2024-03-15T13:03:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f47547dd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b22ac310 con 0x5609b233dd60 2024-03-15T13:03:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f47547dd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b22a9840 con 0x5609b2346940 2024-03-15T13:03:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f4748ff9640 1 -- 172.21.15.2:0/64544994 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f47400efb00 con 0x7f474c0fff50 2024-03-15T13:03:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f474b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b233d440 0x5609b233d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f474b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b233d440 0x5609b233d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51316/0 (socket says 172.21.15.2:51316) 2024-03-15T13:03:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f474b7fe640 1 -- 172.21.15.2:0/2927797971 learned_addr learned my addr 172.21.15.2:0/2927797971 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f474affd640 1 --2- 172.21.15.2:0/2927797971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b233dd60 0x5609b2346400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f474bfff640 1 --2- 172.21.15.2:0/2927797971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b2346940 0x5609b2348d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f474b7fe640 1 -- 172.21.15.2:0/2927797971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b2346940 msgr2=0x5609b2348d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f474b7fe640 1 --2- 172.21.15.2:0/2927797971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b2346940 0x5609b2348d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f474b7fe640 1 -- 172.21.15.2:0/2927797971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b233dd60 msgr2=0x5609b2346400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f474b7fe640 1 --2- 172.21.15.2:0/2927797971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b233dd60 0x5609b2346400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f474b7fe640 1 -- 172.21.15.2:0/2927797971 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609b2267060 con 0x5609b233d440 2024-03-15T13:03:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f474bfff640 1 --2- 172.21.15.2:0/64544994 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f47340a1c20 0x7f47340a40e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4744005c40 tx=0x7f4744012b50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f474b7fe640 1 --2- 172.21.15.2:0/2927797971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b233d440 0x5609b233d820 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f47400b8ec0 tx=0x7f47400bbe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8057d7fe0d4d60 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f4731ffb640 1 -- 172.21.15.2:0/2927797971 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47400dc110 con 0x5609b233d440 2024-03-15T13:03:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f4731ffb640 1 -- 172.21.15.2:0/2927797971 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f47400bcbf0 con 0x5609b233d440 2024-03-15T13:03:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.437+0000 7f4731ffb640 1 -- 172.21.15.2:0/2927797971 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47400a7160 con 0x5609b233d440 2024-03-15T13:03:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.438+0000 7f47547dd740 1 -- 172.21.15.2:0/2927797971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b233d440 msgr2=0x5609b233d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.438+0000 7f47547dd740 1 --2- 172.21.15.2:0/2927797971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b233d440 0x5609b233d820 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f47400b8ec0 tx=0x7f47400bbe50 comp rx=0 tx=0).stop 2024-03-15T13:03:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.438+0000 7f47547dd740 1 -- 172.21.15.2:0/2927797971 shutdown_connections 2024-03-15T13:03:38.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.438+0000 7f47547dd740 1 --2- 172.21.15.2:0/2927797971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b2346940 0x5609b2348d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.438+0000 7f47547dd740 1 --2- 172.21.15.2:0/2927797971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b233dd60 0x5609b2346400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.438+0000 7f47547dd740 1 --2- 172.21.15.2:0/2927797971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b233d440 0x5609b233d820 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.438+0000 7f47547dd740 1 -- 172.21.15.2:0/2927797971 >> 172.21.15.2:0/2927797971 conn(0x5609b2337b70 msgr2=0x5609b2337f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.438+0000 7f47547dd740 1 -- 172.21.15.2:0/2927797971 shutdown_connections 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.438+0000 7f47547dd740 1 -- 172.21.15.2:0/2927797971 wait complete. 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f47547dd740 1 -- start start 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f47547dd740 1 Processor -- start 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f47547dd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b233d440 0x5609b2346310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f47547dd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b233dd60 0x5609b23436a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f47547dd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b2346940 0x5609b2343cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f47547dd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b22a5020 con 0x5609b2346940 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f47547dd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b1f78430 con 0x5609b233d440 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f47547dd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b1f34e00 con 0x5609b233dd60 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f47547dd740 10 client.0 _ll_register_callbacks cb 0x5609b224ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f474bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b2346940 0x5609b2343cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f474bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b2346940 0x5609b2343cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51326/0 (socket says 172.21.15.2:51326) 2024-03-15T13:03:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f474bfff640 1 -- 172.21.15.2:0/1429768023 learned_addr learned my addr 172.21.15.2:0/1429768023 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f474b7fe640 1 --2- 172.21.15.2:0/1429768023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b233d440 0x5609b2346310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f474affd640 1 --2- 172.21.15.2:0/1429768023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b233dd60 0x5609b23436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f474bfff640 1 -- 172.21.15.2:0/1429768023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b233dd60 msgr2=0x5609b23436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f474bfff640 1 --2- 172.21.15.2:0/1429768023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b233dd60 0x5609b23436a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f474bfff640 1 -- 172.21.15.2:0/1429768023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b233d440 msgr2=0x5609b2346310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f474bfff640 1 --2- 172.21.15.2:0/1429768023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b233d440 0x5609b2346310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.439+0000 7f474bfff640 1 -- 172.21.15.2:0/1429768023 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f474c1a4b50 con 0x5609b2346940 2024-03-15T13:03:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f474bfff640 1 --2- 172.21.15.2:0/1429768023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b2346940 0x5609b2343cb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f474400f620 tx=0x7f4744004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14da04471735e890 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f472affd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f472affd640 1 -- 172.21.15.2:0/1429768023 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474401f810 con 0x5609b2346940 2024-03-15T13:03:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f47547dd740 1 -- 172.21.15.2:0/1429768023 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609b1fa47e0 con 0x5609b2346940 2024-03-15T13:03:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f472affd640 1 -- 172.21.15.2:0/1429768023 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f474401f970 con 0x5609b2346940 2024-03-15T13:03:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f472affd640 1 -- 172.21.15.2:0/1429768023 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f474401fb30 con 0x5609b2346940 2024-03-15T13:03:38.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f47547dd740 10 client.17079 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:38.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f47547dd740 1 -- 172.21.15.2:0/1429768023 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5609b22a36a0 con 0x5609b2346940 2024-03-15T13:03:38.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f472affd640 1 -- 172.21.15.2:0/1429768023 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5609b1fa47e0 con 0x5609b2346940 2024-03-15T13:03:38.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f472affd640 1 -- 172.21.15.2:0/1429768023 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7f474401c3a0 con 0x5609b2346940 2024-03-15T13:03:38.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f47547dd740 10 client.17079 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:38.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f47547dd740 10 client.17079 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:03:38.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f47547dd740 20 client.17079 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:38.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f47547dd740 1 --2- 172.21.15.2:0/1429768023 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5609b235ace0 0x5609b235d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f47547dd740 4 client.17079 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:38.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.440+0000 7f47547dd740 1 -- 172.21.15.2:0/1429768023 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609b1fe1720 con 0x5609b235ace0 2024-03-15T13:03:38.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.441+0000 7f474b7fe640 1 --2- 172.21.15.2:0/1429768023 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5609b235ace0 0x5609b235d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.441+0000 7f474b7fe640 1 --2- 172.21.15.2:0/1429768023 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5609b235ace0 0x5609b235d100 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a2d3b37ce60d4bb2 server_cookie=733e6c8dc199c981 in_seq=0 out_seq=0 2024-03-15T13:03:38.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.441+0000 7f472affd640 10 client.17079 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:03:38.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.442+0000 7f472affd640 1 -- 172.21.15.2:0/1429768023 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5609b1fe1720 con 0x5609b235ace0 2024-03-15T13:03:38.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.442+0000 7f472affd640 10 client.17079 handle_command_reply: tid=0 2024-03-15T13:03:38.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.442+0000 7f47547dd740 1 client.17079 shutdown 2024-03-15T13:03:38.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.442+0000 7f47547dd740 1 -- 172.21.15.2:0/1429768023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b2346940 msgr2=0x5609b2343cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.442+0000 7f47547dd740 1 --2- 172.21.15.2:0/1429768023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b2346940 0x5609b2343cb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f474400f620 tx=0x7f4744004ab0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.442+0000 7f47547dd740 1 -- 172.21.15.2:0/1429768023 shutdown_connections 2024-03-15T13:03:38.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.442+0000 7f47547dd740 1 --2- 172.21.15.2:0/1429768023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b2346940 0x5609b2343cb0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.442+0000 7f47547dd740 1 --2- 172.21.15.2:0/1429768023 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5609b235ace0 0x5609b235d100 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.442+0000 7f47547dd740 1 --2- 172.21.15.2:0/1429768023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b233dd60 0x5609b23436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.442+0000 7f47547dd740 1 --2- 172.21.15.2:0/1429768023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b233d440 0x5609b2346310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.442+0000 7f472affd640 0 client.17079 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:03:38.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.442+0000 7f47547dd740 1 -- 172.21.15.2:0/1429768023 >> 172.21.15.2:0/1429768023 conn(0x5609b2337b70 msgr2=0x5609b23398c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.442+0000 7f47547dd740 1 -- 172.21.15.2:0/1429768023 shutdown_connections 2024-03-15T13:03:38.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.442+0000 7f47547dd740 1 -- 172.21.15.2:0/1429768023 wait complete. 2024-03-15T13:03:38.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.442+0000 7f47547dd740 20 client.17079 trim_cache size 0 max 16384 2024-03-15T13:03:38.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f47547dd740 1 Processor -- start 2024-03-15T13:03:38.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f47547dd740 1 -- start start 2024-03-15T13:03:38.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f47547dd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b233d440 0x5609b23437c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f47547dd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b233dd60 0x5609b2345dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f47547dd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b2346940 0x5609b2346310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f47547dd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b222a130 con 0x5609b2346940 2024-03-15T13:03:38.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f47547dd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b2357780 con 0x5609b233dd60 2024-03-15T13:03:38.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f47547dd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b233c530 con 0x5609b233d440 2024-03-15T13:03:38.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f474bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b2346940 0x5609b2346310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f474bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b2346940 0x5609b2346310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51328/0 (socket says 172.21.15.2:51328) 2024-03-15T13:03:38.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f474bfff640 1 -- 172.21.15.2:0/1196933081 learned_addr learned my addr 172.21.15.2:0/1196933081 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f474b7fe640 1 --2- 172.21.15.2:0/1196933081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b233d440 0x5609b23437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f474affd640 1 --2- 172.21.15.2:0/1196933081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b233dd60 0x5609b2345dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f474bfff640 1 -- 172.21.15.2:0/1196933081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b233d440 msgr2=0x5609b23437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f474bfff640 1 --2- 172.21.15.2:0/1196933081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b233d440 0x5609b23437c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f474bfff640 1 -- 172.21.15.2:0/1196933081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b233dd60 msgr2=0x5609b2345dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f474bfff640 1 --2- 172.21.15.2:0/1196933081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b233dd60 0x5609b2345dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f474bfff640 1 -- 172.21.15.2:0/1196933081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609b223fc20 con 0x5609b2346940 2024-03-15T13:03:38.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.455+0000 7f474bfff640 1 --2- 172.21.15.2:0/1196933081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b2346940 0x5609b2346310 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f474400e740 tx=0x7f4744012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223609a68a8fa405 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.456+0000 7f47317fa640 1 -- 172.21.15.2:0/1196933081 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474400e7c0 con 0x5609b2346940 2024-03-15T13:03:38.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.456+0000 7f47317fa640 1 -- 172.21.15.2:0/1196933081 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4744009c90 con 0x5609b2346940 2024-03-15T13:03:38.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.456+0000 7f47317fa640 1 -- 172.21.15.2:0/1196933081 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474401cb20 con 0x5609b2346940 2024-03-15T13:03:38.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.456+0000 7f47547dd740 1 -- 172.21.15.2:0/1196933081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b2346940 msgr2=0x5609b2346310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.456+0000 7f47547dd740 1 --2- 172.21.15.2:0/1196933081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b2346940 0x5609b2346310 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f474400e740 tx=0x7f4744012590 comp rx=0 tx=0).stop 2024-03-15T13:03:38.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.456+0000 7f47547dd740 1 -- 172.21.15.2:0/1196933081 shutdown_connections 2024-03-15T13:03:38.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.456+0000 7f47547dd740 1 --2- 172.21.15.2:0/1196933081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b2346940 0x5609b2346310 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.456+0000 7f47547dd740 1 --2- 172.21.15.2:0/1196933081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b233dd60 0x5609b2345dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.456+0000 7f47547dd740 1 --2- 172.21.15.2:0/1196933081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b233d440 0x5609b23437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.456+0000 7f47547dd740 1 -- 172.21.15.2:0/1196933081 >> 172.21.15.2:0/1196933081 conn(0x5609b2337b70 msgr2=0x5609b233ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.456+0000 7f47547dd740 1 -- 172.21.15.2:0/1196933081 shutdown_connections 2024-03-15T13:03:38.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.456+0000 7f47547dd740 1 -- 172.21.15.2:0/1196933081 wait complete. 2024-03-15T13:03:38.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f47547dd740 1 -- start start 2024-03-15T13:03:38.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f47547dd740 1 Processor -- start 2024-03-15T13:03:38.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f47547dd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b233d440 0x5609b2341380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f47547dd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b233dd60 0x5609b234f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f47547dd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b2346940 0x5609b234fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f47547dd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b2350590 con 0x5609b233dd60 2024-03-15T13:03:38.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f47547dd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b2350700 con 0x5609b2346940 2024-03-15T13:03:38.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f47547dd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609b23b65a0 con 0x5609b233d440 2024-03-15T13:03:38.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f47547dd740 10 client.0 _ll_register_callbacks cb 0x7f474400ed80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:38.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f474affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b233dd60 0x5609b234f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f474affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b233dd60 0x5609b234f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51342/0 (socket says 172.21.15.2:51342) 2024-03-15T13:03:38.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f474affd640 1 -- 172.21.15.2:0/103257119 learned_addr learned my addr 172.21.15.2:0/103257119 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f474bfff640 1 --2- 172.21.15.2:0/103257119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b2346940 0x5609b234fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f474b7fe640 1 --2- 172.21.15.2:0/103257119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b233d440 0x5609b2341380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f474affd640 1 -- 172.21.15.2:0/103257119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b233d440 msgr2=0x5609b2341380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f474affd640 1 --2- 172.21.15.2:0/103257119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b233d440 0x5609b2341380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f474affd640 1 -- 172.21.15.2:0/103257119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b2346940 msgr2=0x5609b234fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f474affd640 1 --2- 172.21.15.2:0/103257119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b2346940 0x5609b234fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.457+0000 7f474affd640 1 -- 172.21.15.2:0/103257119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609b22a36a0 con 0x5609b233dd60 2024-03-15T13:03:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f474affd640 1 --2- 172.21.15.2:0/103257119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b233dd60 0x5609b234f9a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f474c199130 tx=0x7f473c012d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf31e434cc35e7a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f472affd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f472affd640 1 -- 172.21.15.2:0/103257119 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c017040 con 0x5609b233dd60 2024-03-15T13:03:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f472affd640 1 -- 172.21.15.2:0/103257119 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f473c00faf0 con 0x5609b233dd60 2024-03-15T13:03:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f47547dd740 1 -- 172.21.15.2:0/103257119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47400bcbf0 con 0x5609b233dd60 2024-03-15T13:03:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f472affd640 1 -- 172.21.15.2:0/103257119 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f473c00fcb0 con 0x5609b233dd60 2024-03-15T13:03:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f47547dd740 10 client.17085 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f47547dd740 1 -- 172.21.15.2:0/103257119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5609b22658c0 con 0x5609b233dd60 2024-03-15T13:03:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f472affd640 1 -- 172.21.15.2:0/103257119 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f47400bcbf0 con 0x5609b233dd60 2024-03-15T13:03:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f472affd640 1 -- 172.21.15.2:0/103257119 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7f473c010430 con 0x5609b233dd60 2024-03-15T13:03:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f47547dd740 10 client.17085 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f47547dd740 10 client.17085 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:03:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f47547dd740 20 client.17085 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f47547dd740 1 --2- 172.21.15.2:0/103257119 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5609b23b7e00 0x5609b23ba220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f47547dd740 4 client.17085 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:03:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f47547dd740 1 -- 172.21.15.2:0/103257119 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5609b2030890 con 0x5609b23b7e00 2024-03-15T13:03:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.458+0000 7f474b7fe640 1 --2- 172.21.15.2:0/103257119 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5609b23b7e00 0x5609b23ba220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f474b7fe640 1 --2- 172.21.15.2:0/103257119 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5609b23b7e00 0x5609b23ba220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f7f1a8bad5364785 server_cookie=c5cd9ab9639fc68 in_seq=0 out_seq=0 2024-03-15T13:03:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f472affd640 10 client.17085 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:03:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f472affd640 1 -- 172.21.15.2:0/103257119 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5609b2030890 con 0x5609b23b7e00 2024-03-15T13:03:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f472affd640 10 client.17085 handle_command_reply: tid=0 2024-03-15T13:03:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f47547dd740 1 client.17085 shutdown 2024-03-15T13:03:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f47547dd740 1 -- 172.21.15.2:0/103257119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b233dd60 msgr2=0x5609b234f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f47547dd740 1 --2- 172.21.15.2:0/103257119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b233dd60 0x5609b234f9a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f474c199130 tx=0x7f473c012d20 comp rx=0 tx=0).stop 2024-03-15T13:03:38.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f47547dd740 1 -- 172.21.15.2:0/103257119 shutdown_connections 2024-03-15T13:03:38.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f47547dd740 1 --2- 172.21.15.2:0/103257119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5609b2346940 0x5609b234fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f47547dd740 1 --2- 172.21.15.2:0/103257119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5609b233dd60 0x5609b234f9a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f47547dd740 1 --2- 172.21.15.2:0/103257119 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5609b23b7e00 0x5609b23ba220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f47547dd740 1 --2- 172.21.15.2:0/103257119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5609b233d440 0x5609b2341380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f47547dd740 1 -- 172.21.15.2:0/103257119 >> 172.21.15.2:0/103257119 conn(0x5609b2337b70 msgr2=0x5609b23437c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f472affd640 0 client.17085 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:03:38.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f47547dd740 1 -- 172.21.15.2:0/103257119 shutdown_connections 2024-03-15T13:03:38.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f47547dd740 1 -- 172.21.15.2:0/103257119 wait complete. 2024-03-15T13:03:38.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.459+0000 7f47547dd740 20 client.17085 trim_cache size 0 max 16384 2024-03-15T13:03:38.452 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:03:38.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.460+0000 7f47509a0640 1 -- 172.21.15.2:0/64544994 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f47340a1c20 msgr2=0x7f47340a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.460+0000 7f47509a0640 1 --2- 172.21.15.2:0/64544994 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f47340a1c20 0x7f47340a40e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4744005c40 tx=0x7f4744012b50 comp rx=0 tx=0).stop 2024-03-15T13:03:38.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.460+0000 7f47509a0640 1 -- 172.21.15.2:0/64544994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f474c0fff50 msgr2=0x7f474c19d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.460+0000 7f47509a0640 1 --2- 172.21.15.2:0/64544994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f474c0fff50 0x7f474c19d790 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4740009900 tx=0x7f474002ca10 comp rx=0 tx=0).stop 2024-03-15T13:03:38.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.460+0000 7f47509a0640 1 -- 172.21.15.2:0/64544994 shutdown_connections 2024-03-15T13:03:38.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.460+0000 7f47509a0640 1 --2- 172.21.15.2:0/64544994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f474c197940 0x7f474c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.460+0000 7f47509a0640 1 --2- 172.21.15.2:0/64544994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f474c106710 0x7f474c19dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.460+0000 7f47509a0640 1 --2- 172.21.15.2:0/64544994 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f47340a1c20 0x7f47340a40e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.460+0000 7f47509a0640 1 --2- 172.21.15.2:0/64544994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f474c0fff50 0x7f474c19d790 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.460+0000 7f47509a0640 1 -- 172.21.15.2:0/64544994 >> 172.21.15.2:0/64544994 conn(0x7f474c0755d0 msgr2=0x7f474c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.460+0000 7f47509a0640 1 -- 172.21.15.2:0/64544994 shutdown_connections 2024-03-15T13:03:38.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.460+0000 7f47509a0640 1 -- 172.21.15.2:0/64544994 wait complete. 2024-03-15T13:03:38.456 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:03:38.456 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:03:38.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.576+0000 7f319dd08640 1 -- 172.21.15.2:0/2963728470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3198102760 msgr2=0x7f3198102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.576+0000 7f319dd08640 1 --2- 172.21.15.2:0/2963728470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3198102760 0x7f3198102b40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3188009900 tx=0x7f318802bcd0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.576+0000 7f319dd08640 1 -- 172.21.15.2:0/2963728470 shutdown_connections 2024-03-15T13:03:38.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.576+0000 7f319dd08640 1 --2- 172.21.15.2:0/2963728470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f319810bb30 0x7f319810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.576+0000 7f319dd08640 1 --2- 172.21.15.2:0/2963728470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3198103080 0x7f319810b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.576+0000 7f319dd08640 1 --2- 172.21.15.2:0/2963728470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3198102760 0x7f3198102b40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.576+0000 7f319dd08640 1 -- 172.21.15.2:0/2963728470 >> 172.21.15.2:0/2963728470 conn(0x7f3198077550 msgr2=0x7f3198077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.576+0000 7f319dd08640 1 -- 172.21.15.2:0/2963728470 shutdown_connections 2024-03-15T13:03:38.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.576+0000 7f319dd08640 1 -- 172.21.15.2:0/2963728470 wait complete. 2024-03-15T13:03:38.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f319dd08640 1 Processor -- start 2024-03-15T13:03:38.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f319dd08640 1 -- start start 2024-03-15T13:03:38.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f319dd08640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3198102760 0x7f319819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f319dd08640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3198103080 0x7f319819dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f319dd08640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f319810bb30 0x7f31981979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f319dd08640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3198198000 con 0x7f319810bb30 2024-03-15T13:03:38.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f319dd08640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3198198170 con 0x7f3198103080 2024-03-15T13:03:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f319dd08640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3198198330 con 0x7f3198102760 2024-03-15T13:03:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f3197fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f319810bb30 0x7f31981979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f3196ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3198103080 0x7f319819dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f3197fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f319810bb30 0x7f31981979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51348/0 (socket says 172.21.15.2:51348) 2024-03-15T13:03:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f3197fff640 1 -- 172.21.15.2:0/823039684 learned_addr learned my addr 172.21.15.2:0/823039684 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f31977fe640 1 --2- 172.21.15.2:0/823039684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3198102760 0x7f319819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f3197fff640 1 -- 172.21.15.2:0/823039684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3198102760 msgr2=0x7f319819d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f3197fff640 1 --2- 172.21.15.2:0/823039684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3198102760 0x7f319819d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f3197fff640 1 -- 172.21.15.2:0/823039684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3198103080 msgr2=0x7f319819dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f3197fff640 1 --2- 172.21.15.2:0/823039684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3198103080 0x7f319819dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f3197fff640 1 -- 172.21.15.2:0/823039684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3188009560 con 0x7f319810bb30 2024-03-15T13:03:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.577+0000 7f3196ffd640 1 --2- 172.21.15.2:0/823039684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3198103080 0x7f319819dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.578+0000 7f3197fff640 1 --2- 172.21.15.2:0/823039684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f319810bb30 0x7f31981979b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f319000eb10 tx=0x7f319000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.578+0000 7f3194ff9640 1 -- 172.21.15.2:0/823039684 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190004b20 con 0x7f319810bb30 2024-03-15T13:03:38.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.578+0000 7f3194ff9640 1 -- 172.21.15.2:0/823039684 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3190004c80 con 0x7f319810bb30 2024-03-15T13:03:38.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.578+0000 7f319dd08640 1 -- 172.21.15.2:0/823039684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3198198610 con 0x7f319810bb30 2024-03-15T13:03:38.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.578+0000 7f3194ff9640 1 -- 172.21.15.2:0/823039684 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190010690 con 0x7f319810bb30 2024-03-15T13:03:38.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.578+0000 7f319dd08640 1 -- 172.21.15.2:0/823039684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31981a4850 con 0x7f319810bb30 2024-03-15T13:03:38.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f3194ff9640 1 -- 172.21.15.2:0/823039684 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f3190004df0 con 0x7f319810bb30 2024-03-15T13:03:38.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f31a09b6740 1 Processor -- start 2024-03-15T13:03:38.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f31a09b6740 1 -- start start 2024-03-15T13:03:38.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f31a09b6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bc5250 0x563b10bc5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f31a09b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bc5b70 0x563b10bce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f31a09b6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bce750 0x563b10bd0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f31a09b6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b10b80960 con 0x563b10bce750 2024-03-15T13:03:38.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f31a09b6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b10b34310 con 0x563b10bc5250 2024-03-15T13:03:38.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f31a09b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b10b31840 con 0x563b10bc5b70 2024-03-15T13:03:38.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f31977fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bc5b70 0x563b10bce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f3196ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bce750 0x563b10bd0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f31977fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bc5b70 0x563b10bce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:48948/0 (socket says 172.21.15.2:48948) 2024-03-15T13:03:38.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f31977fe640 1 -- 172.21.15.2:0/2199222588 learned_addr learned my addr 172.21.15.2:0/2199222588 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f3196ffd640 1 --2- 172.21.15.2:0/2199222588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bc5250 0x563b10bc5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f3194ff9640 1 --2- 172.21.15.2:0/823039684 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f31800a1ba0 0x7f31800a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f3197fff640 1 --2- 172.21.15.2:0/823039684 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f31800a1ba0 0x7f31800a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f31977fe640 1 -- 172.21.15.2:0/2199222588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bc5250 msgr2=0x563b10bc5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.581+0000 7f31977fe640 1 --2- 172.21.15.2:0/2199222588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bc5250 0x563b10bc5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.582+0000 7f31977fe640 1 -- 172.21.15.2:0/2199222588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bce750 msgr2=0x563b10bd0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.582+0000 7f31977fe640 1 --2- 172.21.15.2:0/2199222588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bce750 0x563b10bd0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.582+0000 7f31977fe640 1 -- 172.21.15.2:0/2199222588 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b10aef060 con 0x563b10bc5b70 2024-03-15T13:03:38.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.582+0000 7f3194ff9640 1 -- 172.21.15.2:0/823039684 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f3190014070 con 0x7f319810bb30 2024-03-15T13:03:38.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.582+0000 7f31977fe640 1 --2- 172.21.15.2:0/2199222588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bc5b70 0x563b10bce210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3188036040 tx=0x7f3188005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab0c6aa0cc2b7c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.582+0000 7f3197fff640 1 --2- 172.21.15.2:0/823039684 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f31800a1ba0 0x7f31800a4060 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f31900cba70 tx=0x7f319001b4f0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.582+0000 7f317dffb640 1 -- 172.21.15.2:0/2199222588 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188004ea0 con 0x563b10bc5b70 2024-03-15T13:03:38.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.582+0000 7f317dffb640 1 -- 172.21.15.2:0/2199222588 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3188045070 con 0x563b10bc5b70 2024-03-15T13:03:38.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.582+0000 7f317dffb640 1 -- 172.21.15.2:0/2199222588 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31880043c0 con 0x563b10bc5b70 2024-03-15T13:03:38.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 -- 172.21.15.2:0/2199222588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bc5b70 msgr2=0x563b10bce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 --2- 172.21.15.2:0/2199222588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bc5b70 0x563b10bce210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3188036040 tx=0x7f3188005bf0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 -- 172.21.15.2:0/2199222588 shutdown_connections 2024-03-15T13:03:38.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 --2- 172.21.15.2:0/2199222588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bce750 0x563b10bd0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 --2- 172.21.15.2:0/2199222588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bc5b70 0x563b10bce210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 --2- 172.21.15.2:0/2199222588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bc5250 0x563b10bc5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 -- 172.21.15.2:0/2199222588 >> 172.21.15.2:0/2199222588 conn(0x563b10bbf980 msgr2=0x563b10bbfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 -- 172.21.15.2:0/2199222588 shutdown_connections 2024-03-15T13:03:38.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 -- 172.21.15.2:0/2199222588 wait complete. 2024-03-15T13:03:38.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 -- start start 2024-03-15T13:03:38.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 Processor -- start 2024-03-15T13:03:38.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bc5250 0x563b10bce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5b70 0x563b10bcb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bce750 0x563b10bcbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b10b2d020 con 0x563b10bc5b70 2024-03-15T13:03:38.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b10800430 con 0x563b10bc5250 2024-03-15T13:03:38.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.583+0000 7f31a09b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b107bce00 con 0x563b10bce750 2024-03-15T13:03:38.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f31a09b6740 10 client.0 _ll_register_callbacks cb 0x563b10ad4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:38.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f31977fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5b70 0x563b10bcb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f31977fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5b70 0x563b10bcb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51360/0 (socket says 172.21.15.2:51360) 2024-03-15T13:03:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f3196ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bc5250 0x563b10bce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f31977fe640 1 -- 172.21.15.2:0/3290229002 learned_addr learned my addr 172.21.15.2:0/3290229002 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f3196ffd640 1 --2- 172.21.15.2:0/3290229002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bce750 0x563b10bcbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f31977fe640 1 -- 172.21.15.2:0/3290229002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bce750 msgr2=0x563b10bcbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f31977fe640 1 --2- 172.21.15.2:0/3290229002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bce750 0x563b10bcbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f31977fe640 1 -- 172.21.15.2:0/3290229002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bc5250 msgr2=0x563b10bce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f31977fe640 1 --2- 172.21.15.2:0/3290229002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bc5250 0x563b10bce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f31977fe640 1 -- 172.21.15.2:0/3290229002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319810ec20 con 0x563b10bc5b70 2024-03-15T13:03:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f31977fe640 1 --2- 172.21.15.2:0/3290229002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5b70 0x563b10bcb4d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f3188006010 tx=0x7f318802bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39ea43319e4552d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f3176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f3176ffd640 1 -- 172.21.15.2:0/3290229002 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188050070 con 0x563b10bc5b70 2024-03-15T13:03:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f3176ffd640 1 -- 172.21.15.2:0/3290229002 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f3188047070 con 0x563b10bc5b70 2024-03-15T13:03:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f31a09b6740 1 -- 172.21.15.2:0/3290229002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b1082c7e0 con 0x563b10bc5b70 2024-03-15T13:03:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f3176ffd640 1 -- 172.21.15.2:0/3290229002 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f318802cc00 con 0x563b10bc5b70 2024-03-15T13:03:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f31a09b6740 10 client.17097 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.584+0000 7f31a09b6740 1 -- 172.21.15.2:0/3290229002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563b10b2b6a0 con 0x563b10bc5b70 2024-03-15T13:03:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.585+0000 7f3176ffd640 1 -- 172.21.15.2:0/3290229002 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x563b1082c7e0 con 0x563b10bc5b70 2024-03-15T13:03:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.585+0000 7f3176ffd640 1 -- 172.21.15.2:0/3290229002 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7f318803c050 con 0x563b10bc5b70 2024-03-15T13:03:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.585+0000 7f31a09b6740 10 client.17097 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.585+0000 7f31a09b6740 10 client.17097 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:03:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.585+0000 7f31a09b6740 20 client.17097 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.585+0000 7f31a09b6740 1 --2- 172.21.15.2:0/3290229002 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563b10bde5e0 0x563b10be0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.585+0000 7f31a09b6740 4 client.17097 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.585+0000 7f31a09b6740 1 -- 172.21.15.2:0/3290229002 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b10869720 con 0x563b10bde5e0 2024-03-15T13:03:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.585+0000 7f3196ffd640 1 --2- 172.21.15.2:0/3290229002 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563b10bde5e0 0x563b10be0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.585+0000 7f3176ffd640 10 client.17097 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:03:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.585+0000 7f3196ffd640 1 --2- 172.21.15.2:0/3290229002 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563b10bde5e0 0x563b10be0a00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58685c3df55fc534 server_cookie=bfffbd945df29c8f in_seq=0 out_seq=0 2024-03-15T13:03:38.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.586+0000 7f3176ffd640 1 -- 172.21.15.2:0/3290229002 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563b10869720 con 0x563b10bde5e0 2024-03-15T13:03:38.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.586+0000 7f3176ffd640 10 client.17097 handle_command_reply: tid=0 2024-03-15T13:03:38.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.586+0000 7f31a09b6740 1 client.17097 shutdown 2024-03-15T13:03:38.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.586+0000 7f31a09b6740 1 -- 172.21.15.2:0/3290229002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5b70 msgr2=0x563b10bcb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.586+0000 7f31a09b6740 1 --2- 172.21.15.2:0/3290229002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5b70 0x563b10bcb4d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f3188006010 tx=0x7f318802bcd0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.586+0000 7f31a09b6740 1 -- 172.21.15.2:0/3290229002 shutdown_connections 2024-03-15T13:03:38.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.586+0000 7f31a09b6740 1 --2- 172.21.15.2:0/3290229002 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563b10bde5e0 0x563b10be0a00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.586+0000 7f31a09b6740 1 --2- 172.21.15.2:0/3290229002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bce750 0x563b10bcbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.586+0000 7f31a09b6740 1 --2- 172.21.15.2:0/3290229002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5b70 0x563b10bcb4d0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.586+0000 7f3176ffd640 0 client.17097 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:03:38.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.586+0000 7f31a09b6740 1 --2- 172.21.15.2:0/3290229002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bc5250 0x563b10bce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.586+0000 7f31a09b6740 1 -- 172.21.15.2:0/3290229002 >> 172.21.15.2:0/3290229002 conn(0x563b10bbf980 msgr2=0x563b10bc16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.586+0000 7f31a09b6740 1 -- 172.21.15.2:0/3290229002 shutdown_connections 2024-03-15T13:03:38.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.586+0000 7f31a09b6740 1 -- 172.21.15.2:0/3290229002 wait complete. 2024-03-15T13:03:38.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.586+0000 7f31a09b6740 20 client.17097 trim_cache size 0 max 16384 2024-03-15T13:03:38.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f31a09b6740 1 Processor -- start 2024-03-15T13:03:38.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f31a09b6740 1 -- start start 2024-03-15T13:03:38.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f31a09b6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5250 0x563b10bcdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f31a09b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bc5b70 0x563b10bce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f31a09b6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bce750 0x563b10bd9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f31a09b6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b10ab2130 con 0x563b10bc5250 2024-03-15T13:03:38.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f31a09b6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b10be1060 con 0x563b10bce750 2024-03-15T13:03:38.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f31a09b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b10bbb5c0 con 0x563b10bc5b70 2024-03-15T13:03:38.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f3196ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5250 0x563b10bcdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f3196ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5250 0x563b10bcdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51362/0 (socket says 172.21.15.2:51362) 2024-03-15T13:03:38.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f3196ffd640 1 -- 172.21.15.2:0/2042670261 learned_addr learned my addr 172.21.15.2:0/2042670261 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f31977fe640 1 --2- 172.21.15.2:0/2042670261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bc5b70 0x563b10bce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f3196ffd640 1 --2- 172.21.15.2:0/2042670261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bce750 0x563b10bd9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f3196ffd640 1 -- 172.21.15.2:0/2042670261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bc5b70 msgr2=0x563b10bce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f3196ffd640 1 --2- 172.21.15.2:0/2042670261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bc5b70 0x563b10bce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f3196ffd640 1 -- 172.21.15.2:0/2042670261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bce750 msgr2=0x563b10bd9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f3196ffd640 1 --2- 172.21.15.2:0/2042670261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bce750 0x563b10bd9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f3196ffd640 1 -- 172.21.15.2:0/2042670261 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b10ac7c20 con 0x563b10bc5250 2024-03-15T13:03:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f3196ffd640 1 --2- 172.21.15.2:0/2042670261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5250 0x563b10bcdc00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f318c012d00 tx=0x7f318c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf2222ae57b996d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.599+0000 7f317d7fa640 1 -- 172.21.15.2:0/2042670261 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c02c070 con 0x563b10bc5250 2024-03-15T13:03:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.600+0000 7f317d7fa640 1 -- 172.21.15.2:0/2042670261 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f318c026d70 con 0x563b10bc5250 2024-03-15T13:03:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.600+0000 7f317d7fa640 1 -- 172.21.15.2:0/2042670261 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c016040 con 0x563b10bc5250 2024-03-15T13:03:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.600+0000 7f31a09b6740 1 -- 172.21.15.2:0/2042670261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5250 msgr2=0x563b10bcdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.600+0000 7f31a09b6740 1 --2- 172.21.15.2:0/2042670261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5250 0x563b10bcdc00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f318c012d00 tx=0x7f318c0046c0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.600+0000 7f31a09b6740 1 -- 172.21.15.2:0/2042670261 shutdown_connections 2024-03-15T13:03:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.600+0000 7f31a09b6740 1 --2- 172.21.15.2:0/2042670261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bce750 0x563b10bd9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.600+0000 7f31a09b6740 1 --2- 172.21.15.2:0/2042670261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bc5b70 0x563b10bce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.600+0000 7f31a09b6740 1 --2- 172.21.15.2:0/2042670261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5250 0x563b10bcdc00 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.600+0000 7f31a09b6740 1 -- 172.21.15.2:0/2042670261 >> 172.21.15.2:0/2042670261 conn(0x563b10bbf980 msgr2=0x563b10bbeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.600+0000 7f31a09b6740 1 -- 172.21.15.2:0/2042670261 shutdown_connections 2024-03-15T13:03:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.600+0000 7f31a09b6740 1 -- 172.21.15.2:0/2042670261 wait complete. 2024-03-15T13:03:38.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.600+0000 7f31a09b6740 1 -- start start 2024-03-15T13:03:38.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31a09b6740 1 Processor -- start 2024-03-15T13:03:38.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31a09b6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bc5250 0x563b10bd2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31a09b6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5b70 0x563b10bd2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31a09b6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bce750 0x563b10c32ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31a09b6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b10c33410 con 0x563b10bc5b70 2024-03-15T13:03:38.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31a09b6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b10c33580 con 0x563b10bc5250 2024-03-15T13:03:38.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31a09b6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b10c33740 con 0x563b10bce750 2024-03-15T13:03:38.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31a09b6740 10 client.0 _ll_register_callbacks cb 0x7f318c009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:38.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31977fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5b70 0x563b10bd2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f3196ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bc5250 0x563b10bd2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31977fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5b70 0x563b10bd2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51378/0 (socket says 172.21.15.2:51378) 2024-03-15T13:03:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31977fe640 1 -- 172.21.15.2:0/359672772 learned_addr learned my addr 172.21.15.2:0/359672772 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f3196ffd640 1 --2- 172.21.15.2:0/359672772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bce750 0x563b10c32ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31977fe640 1 -- 172.21.15.2:0/359672772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bce750 msgr2=0x563b10c32ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31977fe640 1 --2- 172.21.15.2:0/359672772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bce750 0x563b10c32ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31977fe640 1 -- 172.21.15.2:0/359672772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bc5250 msgr2=0x563b10bd2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31977fe640 1 --2- 172.21.15.2:0/359672772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bc5250 0x563b10bd2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31977fe640 1 -- 172.21.15.2:0/359672772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c019040 con 0x563b10bc5b70 2024-03-15T13:03:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f31977fe640 1 --2- 172.21.15.2:0/359672772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5b70 0x563b10bd2770 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3188006010 tx=0x7f3188004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e3eb337f40202d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.601+0000 7f3176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.602+0000 7f3176ffd640 1 -- 172.21.15.2:0/359672772 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188050070 con 0x563b10bc5b70 2024-03-15T13:03:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.602+0000 7f3176ffd640 1 -- 172.21.15.2:0/359672772 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f3188047040 con 0x563b10bc5b70 2024-03-15T13:03:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.602+0000 7f31a09b6740 1 -- 172.21.15.2:0/359672772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3188045070 con 0x563b10bc5b70 2024-03-15T13:03:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.602+0000 7f3176ffd640 1 -- 172.21.15.2:0/359672772 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f318802cbc0 con 0x563b10bc5b70 2024-03-15T13:03:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.602+0000 7f31a09b6740 10 client.17103 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.602+0000 7f31a09b6740 1 -- 172.21.15.2:0/359672772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563b10aed8c0 con 0x563b10bc5b70 2024-03-15T13:03:38.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.602+0000 7f3176ffd640 1 -- 172.21.15.2:0/359672772 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3188045070 con 0x563b10bc5b70 2024-03-15T13:03:38.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.602+0000 7f3176ffd640 1 -- 172.21.15.2:0/359672772 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7f318803c040 con 0x563b10bc5b70 2024-03-15T13:03:38.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.602+0000 7f31a09b6740 10 client.17103 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:38.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.602+0000 7f31a09b6740 10 client.17103 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:03:38.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.602+0000 7f31a09b6740 20 client.17103 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:38.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.602+0000 7f31a09b6740 1 --2- 172.21.15.2:0/359672772 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563b10c3abb0 0x563b10c3cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.602+0000 7f31a09b6740 4 client.17103 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:03:38.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.602+0000 7f31a09b6740 1 -- 172.21.15.2:0/359672772 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563b108b8890 con 0x563b10c3abb0 2024-03-15T13:03:38.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.602+0000 7f3196ffd640 1 --2- 172.21.15.2:0/359672772 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563b10c3abb0 0x563b10c3cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f3176ffd640 10 client.17103 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:03:38.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f3196ffd640 1 --2- 172.21.15.2:0/359672772 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563b10c3abb0 0x563b10c3cfd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3bd777450e5189a0 server_cookie=5c40fd5b6c70742d in_seq=0 out_seq=0 2024-03-15T13:03:38.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f3176ffd640 1 -- 172.21.15.2:0/359672772 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563b108b8890 con 0x563b10c3abb0 2024-03-15T13:03:38.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f3176ffd640 10 client.17103 handle_command_reply: tid=0 2024-03-15T13:03:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f31a09b6740 1 client.17103 shutdown 2024-03-15T13:03:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f31a09b6740 1 -- 172.21.15.2:0/359672772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5b70 msgr2=0x563b10bd2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f31a09b6740 1 --2- 172.21.15.2:0/359672772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5b70 0x563b10bd2770 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3188006010 tx=0x7f3188004560 comp rx=0 tx=0).stop 2024-03-15T13:03:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f31a09b6740 1 -- 172.21.15.2:0/359672772 shutdown_connections 2024-03-15T13:03:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f31a09b6740 1 --2- 172.21.15.2:0/359672772 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563b10c3abb0 0x563b10c3cfd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f31a09b6740 1 --2- 172.21.15.2:0/359672772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b10bce750 0x563b10c32ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f3176ffd640 0 client.17103 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:03:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f31a09b6740 1 --2- 172.21.15.2:0/359672772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563b10bc5b70 0x563b10bd2770 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f31a09b6740 1 --2- 172.21.15.2:0/359672772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b10bc5250 0x563b10bd2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f31a09b6740 1 -- 172.21.15.2:0/359672772 >> 172.21.15.2:0/359672772 conn(0x563b10bbf980 msgr2=0x563b10bc0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f31a09b6740 1 -- 172.21.15.2:0/359672772 shutdown_connections 2024-03-15T13:03:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f31a09b6740 1 -- 172.21.15.2:0/359672772 wait complete. 2024-03-15T13:03:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.603+0000 7f31a09b6740 20 client.17103 trim_cache size 0 max 16384 2024-03-15T13:03:38.596 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:03:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.604+0000 7f319dd08640 1 -- 172.21.15.2:0/823039684 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f31800a1ba0 msgr2=0x7f31800a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.604+0000 7f319dd08640 1 --2- 172.21.15.2:0/823039684 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f31800a1ba0 0x7f31800a4060 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f31900cba70 tx=0x7f319001b4f0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.604+0000 7f319dd08640 1 -- 172.21.15.2:0/823039684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f319810bb30 msgr2=0x7f31981979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.604+0000 7f319dd08640 1 --2- 172.21.15.2:0/823039684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f319810bb30 0x7f31981979b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f319000eb10 tx=0x7f319000c2d0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.604+0000 7f319dd08640 1 -- 172.21.15.2:0/823039684 shutdown_connections 2024-03-15T13:03:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.604+0000 7f319dd08640 1 --2- 172.21.15.2:0/823039684 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f31800a1ba0 0x7f31800a4060 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.604+0000 7f319dd08640 1 --2- 172.21.15.2:0/823039684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f319810bb30 0x7f31981979b0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.604+0000 7f319dd08640 1 --2- 172.21.15.2:0/823039684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3198103080 0x7f319819dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.604+0000 7f319dd08640 1 --2- 172.21.15.2:0/823039684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3198102760 0x7f319819d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.604+0000 7f319dd08640 1 -- 172.21.15.2:0/823039684 >> 172.21.15.2:0/823039684 conn(0x7f3198077550 msgr2=0x7f31980fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.604+0000 7f319dd08640 1 -- 172.21.15.2:0/823039684 shutdown_connections 2024-03-15T13:03:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.604+0000 7f319dd08640 1 -- 172.21.15.2:0/823039684 wait complete. 2024-03-15T13:03:38.600 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:03:38.600 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:03:38.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.719+0000 7ff64f465640 1 -- 172.21.15.2:0/2714132095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff648100bd0 msgr2=0x7ff64810d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.719+0000 7ff64f465640 1 --2- 172.21.15.2:0/2714132095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff648100bd0 0x7ff64810d920 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff638009900 tx=0x7ff63802bc30 comp rx=0 tx=0).stop 2024-03-15T13:03:38.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.719+0000 7ff64f465640 1 -- 172.21.15.2:0/2714132095 shutdown_connections 2024-03-15T13:03:38.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.719+0000 7ff64f465640 1 --2- 172.21.15.2:0/2714132095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff64810de60 0x7ff648110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.719+0000 7ff64f465640 1 --2- 172.21.15.2:0/2714132095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff648100bd0 0x7ff64810d920 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.719+0000 7ff64f465640 1 --2- 172.21.15.2:0/2714132095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6481002b0 0x7ff648100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.719+0000 7ff64f465640 1 -- 172.21.15.2:0/2714132095 >> 172.21.15.2:0/2714132095 conn(0x7ff648076770 msgr2=0x7ff648076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.720+0000 7ff64f465640 1 -- 172.21.15.2:0/2714132095 shutdown_connections 2024-03-15T13:03:38.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.720+0000 7ff64f465640 1 -- 172.21.15.2:0/2714132095 wait complete. 2024-03-15T13:03:38.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.720+0000 7ff64f465640 1 Processor -- start 2024-03-15T13:03:38.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.720+0000 7ff64f465640 1 -- start start 2024-03-15T13:03:38.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.720+0000 7ff64f465640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6481002b0 0x7ff648199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.720+0000 7ff64f465640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff648100bd0 0x7ff64819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.720+0000 7ff64f465640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff64810de60 0x7ff64819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.720+0000 7ff64f465640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64819ec60 con 0x7ff64810de60 2024-03-15T13:03:38.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.720+0000 7ff64f465640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64819edd0 con 0x7ff648100bd0 2024-03-15T13:03:38.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.720+0000 7ff64f465640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64819ef90 con 0x7ff6481002b0 2024-03-15T13:03:38.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff64da02640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff64810de60 0x7ff64819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff64d201640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6481002b0 0x7ff648199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff64d201640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6481002b0 0x7ff648199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:48978/0 (socket says 172.21.15.2:48978) 2024-03-15T13:03:38.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff64d201640 1 -- 172.21.15.2:0/3492992787 learned_addr learned my addr 172.21.15.2:0/3492992787 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff64da02640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff64810de60 0x7ff64819e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51410/0 (socket says 172.21.15.2:51410) 2024-03-15T13:03:38.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff64ca00640 1 --2- 172.21.15.2:0/3492992787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff648100bd0 0x7ff64819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff64d201640 1 -- 172.21.15.2:0/3492992787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff648100bd0 msgr2=0x7ff64819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff64d201640 1 --2- 172.21.15.2:0/3492992787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff648100bd0 0x7ff64819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff64d201640 1 -- 172.21.15.2:0/3492992787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff64810de60 msgr2=0x7ff64819e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff64d201640 1 --2- 172.21.15.2:0/3492992787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff64810de60 0x7ff64819e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff64d201640 1 -- 172.21.15.2:0/3492992787 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff638009560 con 0x7ff6481002b0 2024-03-15T13:03:38.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff64da02640 1 --2- 172.21.15.2:0/3492992787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff64810de60 0x7ff64819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:38.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff64d201640 1 --2- 172.21.15.2:0/3492992787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6481002b0 0x7ff648199da0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff64400e9f0 tx=0x7ff64400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff63e7fc640 1 -- 172.21.15.2:0/3492992787 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644009700 con 0x7ff6481002b0 2024-03-15T13:03:38.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff64f465640 1 -- 172.21.15.2:0/3492992787 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64819f270 con 0x7ff6481002b0 2024-03-15T13:03:38.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff63e7fc640 1 -- 172.21.15.2:0/3492992787 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff64400cd80 con 0x7ff6481002b0 2024-03-15T13:03:38.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff63e7fc640 1 -- 172.21.15.2:0/3492992787 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644018920 con 0x7ff6481002b0 2024-03-15T13:03:38.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.721+0000 7ff64f465640 1 -- 172.21.15.2:0/3492992787 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6481a6af0 con 0x7ff6481002b0 2024-03-15T13:03:38.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.724+0000 7ff65203d740 1 Processor -- start 2024-03-15T13:03:38.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.724+0000 7ff63e7fc640 1 -- 172.21.15.2:0/3492992787 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7ff644018a80 con 0x7ff6481002b0 2024-03-15T13:03:38.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.724+0000 7ff65203d740 1 -- start start 2024-03-15T13:03:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff65203d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822250 0x5556a2822630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff65203d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822b70 0x5556a282b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff65203d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a282b750 0x5556a282db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff65203d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a27dd960 con 0x5556a2822250 2024-03-15T13:03:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff65203d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a2791310 con 0x5556a2822b70 2024-03-15T13:03:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff65203d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a278e840 con 0x5556a282b750 2024-03-15T13:03:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff64da02640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822250 0x5556a2822630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff64da02640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822250 0x5556a2822630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51424/0 (socket says 172.21.15.2:51424) 2024-03-15T13:03:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff64da02640 1 -- 172.21.15.2:0/3554425739 learned_addr learned my addr 172.21.15.2:0/3554425739 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff64d201640 1 --2- 172.21.15.2:0/3554425739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822b70 0x5556a282b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff64ca00640 1 --2- 172.21.15.2:0/3554425739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a282b750 0x5556a282db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff63e7fc640 1 --2- 172.21.15.2:0/3492992787 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff6340a1ba0 0x7ff6340a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff64da02640 1 -- 172.21.15.2:0/3554425739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a282b750 msgr2=0x5556a282db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff64da02640 1 --2- 172.21.15.2:0/3554425739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a282b750 0x5556a282db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff64da02640 1 -- 172.21.15.2:0/3554425739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822b70 msgr2=0x5556a282b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff64da02640 1 --2- 172.21.15.2:0/3554425739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822b70 0x5556a282b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff64da02640 1 -- 172.21.15.2:0/3554425739 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556a274c060 con 0x5556a2822250 2024-03-15T13:03:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff64d201640 1 --2- 172.21.15.2:0/3554425739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822b70 0x5556a282b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff63e7fc640 1 -- 172.21.15.2:0/3492992787 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7ff644014070 con 0x7ff6481002b0 2024-03-15T13:03:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff64da02640 1 --2- 172.21.15.2:0/3492992787 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff6340a1ba0 0x7ff6340a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff64da02640 1 --2- 172.21.15.2:0/3554425739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822250 0x5556a2822630 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff6400107a0 tx=0x7ff640010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa55b6ab196460b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.725+0000 7ff6337fe640 1 -- 172.21.15.2:0/3554425739 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64000ee10 con 0x5556a2822250 2024-03-15T13:03:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.726+0000 7ff6337fe640 1 -- 172.21.15.2:0/3554425739 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff640015070 con 0x5556a2822250 2024-03-15T13:03:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.726+0000 7ff6337fe640 1 -- 172.21.15.2:0/3554425739 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64001b5e0 con 0x5556a2822250 2024-03-15T13:03:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.726+0000 7ff65203d740 1 -- 172.21.15.2:0/3554425739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822250 msgr2=0x5556a2822630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.726+0000 7ff64da02640 1 --2- 172.21.15.2:0/3492992787 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff6340a1ba0 0x7ff6340a4060 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff640011010 tx=0x7ff640020000 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.726+0000 7ff65203d740 1 --2- 172.21.15.2:0/3554425739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822250 0x5556a2822630 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff6400107a0 tx=0x7ff640010ba0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.726+0000 7ff65203d740 1 -- 172.21.15.2:0/3554425739 shutdown_connections 2024-03-15T13:03:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.726+0000 7ff65203d740 1 --2- 172.21.15.2:0/3554425739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a282b750 0x5556a282db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.726+0000 7ff65203d740 1 --2- 172.21.15.2:0/3554425739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822b70 0x5556a282b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.726+0000 7ff65203d740 1 --2- 172.21.15.2:0/3554425739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822250 0x5556a2822630 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.726+0000 7ff65203d740 1 -- 172.21.15.2:0/3554425739 >> 172.21.15.2:0/3554425739 conn(0x5556a281c980 msgr2=0x5556a281cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.726+0000 7ff65203d740 1 -- 172.21.15.2:0/3554425739 shutdown_connections 2024-03-15T13:03:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.726+0000 7ff65203d740 1 -- 172.21.15.2:0/3554425739 wait complete. 2024-03-15T13:03:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff65203d740 1 -- start start 2024-03-15T13:03:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff65203d740 1 Processor -- start 2024-03-15T13:03:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff65203d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a2822250 0x5556a282b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff65203d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822b70 0x5556a2828390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff65203d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a282b750 0x5556a28288d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff65203d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a278a020 con 0x5556a2822b70 2024-03-15T13:03:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff65203d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a245d430 con 0x5556a282b750 2024-03-15T13:03:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff65203d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a2419e00 con 0x5556a2822250 2024-03-15T13:03:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff65203d740 10 client.0 _ll_register_callbacks cb 0x5556a2731cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff64ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822b70 0x5556a2828390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff64ca00640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822b70 0x5556a2828390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51426/0 (socket says 172.21.15.2:51426) 2024-03-15T13:03:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff64ca00640 1 -- 172.21.15.2:0/3354862239 learned_addr learned my addr 172.21.15.2:0/3354862239 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff64d201640 1 --2- 172.21.15.2:0/3354862239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a2822250 0x5556a282b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff64da02640 1 --2- 172.21.15.2:0/3354862239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a282b750 0x5556a28288d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff64ca00640 1 -- 172.21.15.2:0/3354862239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a2822250 msgr2=0x5556a282b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff64ca00640 1 --2- 172.21.15.2:0/3354862239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a2822250 0x5556a282b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff64ca00640 1 -- 172.21.15.2:0/3354862239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a282b750 msgr2=0x5556a28288d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff64ca00640 1 --2- 172.21.15.2:0/3354862239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a282b750 0x5556a28288d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.727+0000 7ff64ca00640 1 -- 172.21.15.2:0/3354862239 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6400103d0 con 0x5556a2822b70 2024-03-15T13:03:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff64ca00640 1 --2- 172.21.15.2:0/3354862239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822b70 0x5556a2828390 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff638006010 tx=0x7ff63802ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85502c858ff78361 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff630ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:38.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff630ff9640 1 -- 172.21.15.2:0/3354862239 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63802cc30 con 0x5556a2822b70 2024-03-15T13:03:38.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff630ff9640 1 -- 172.21.15.2:0/3354862239 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7ff638046070 con 0x5556a2822b70 2024-03-15T13:03:38.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff65203d740 1 -- 172.21.15.2:0/3354862239 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556a24897e0 con 0x5556a2822b70 2024-03-15T13:03:38.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff630ff9640 1 -- 172.21.15.2:0/3354862239 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff638004250 con 0x5556a2822b70 2024-03-15T13:03:38.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff65203d740 10 client.17115 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:38.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff65203d740 1 -- 172.21.15.2:0/3354862239 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5556a27886a0 con 0x5556a2822b70 2024-03-15T13:03:38.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff630ff9640 1 -- 172.21.15.2:0/3354862239 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5556a24897e0 con 0x5556a2822b70 2024-03-15T13:03:38.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff630ff9640 1 -- 172.21.15.2:0/3354862239 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7ff63803c040 con 0x5556a2822b70 2024-03-15T13:03:38.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff65203d740 10 client.17115 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:38.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff65203d740 10 client.17115 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:03:38.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff65203d740 20 client.17115 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:38.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff65203d740 1 --2- 172.21.15.2:0/3354862239 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5556a283b850 0x5556a283dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff65203d740 4 client.17115 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:38.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.728+0000 7ff65203d740 1 -- 172.21.15.2:0/3354862239 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556a24c6720 con 0x5556a283b850 2024-03-15T13:03:38.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.729+0000 7ff64d201640 1 --2- 172.21.15.2:0/3354862239 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5556a283b850 0x5556a283dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.729+0000 7ff630ff9640 10 client.17115 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:03:38.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.729+0000 7ff64d201640 1 --2- 172.21.15.2:0/3354862239 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5556a283b850 0x5556a283dc70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d82c03a9029bf814 server_cookie=5695d0bfb121f6ea in_seq=0 out_seq=0 2024-03-15T13:03:38.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.730+0000 7ff630ff9640 1 -- 172.21.15.2:0/3354862239 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7ff6440a94b0 con 0x5556a283b850 2024-03-15T13:03:38.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.730+0000 7ff630ff9640 10 client.17115 handle_command_reply: tid=0 2024-03-15T13:03:38.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.730+0000 7ff65203d740 1 client.17115 shutdown 2024-03-15T13:03:38.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.730+0000 7ff65203d740 1 -- 172.21.15.2:0/3354862239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822b70 msgr2=0x5556a2828390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.730+0000 7ff65203d740 1 --2- 172.21.15.2:0/3354862239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822b70 0x5556a2828390 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff638006010 tx=0x7ff63802ca40 comp rx=0 tx=0).stop 2024-03-15T13:03:38.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.730+0000 7ff65203d740 1 -- 172.21.15.2:0/3354862239 shutdown_connections 2024-03-15T13:03:38.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.730+0000 7ff65203d740 1 --2- 172.21.15.2:0/3354862239 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5556a283b850 0x5556a283dc70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.730+0000 7ff65203d740 1 --2- 172.21.15.2:0/3354862239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a282b750 0x5556a28288d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.730+0000 7ff65203d740 1 --2- 172.21.15.2:0/3354862239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822b70 0x5556a2828390 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.730+0000 7ff630ff9640 0 client.17115 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:03:38.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.730+0000 7ff65203d740 1 --2- 172.21.15.2:0/3354862239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a2822250 0x5556a282b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.730+0000 7ff65203d740 1 -- 172.21.15.2:0/3354862239 >> 172.21.15.2:0/3354862239 conn(0x5556a281c980 msgr2=0x5556a281e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.730+0000 7ff65203d740 1 -- 172.21.15.2:0/3354862239 shutdown_connections 2024-03-15T13:03:38.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.730+0000 7ff65203d740 1 -- 172.21.15.2:0/3354862239 wait complete. 2024-03-15T13:03:38.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.730+0000 7ff65203d740 20 client.17115 trim_cache size 0 max 16384 2024-03-15T13:03:38.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.742+0000 7ff65203d740 1 Processor -- start 2024-03-15T13:03:38.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.742+0000 7ff65203d740 1 -- start start 2024-03-15T13:03:38.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.742+0000 7ff65203d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822250 0x5556a281b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff65203d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822b70 0x5556a282aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff65203d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a282b750 0x5556a282b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff65203d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a270f130 con 0x5556a2822250 2024-03-15T13:03:38.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff65203d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a282df00 con 0x5556a2822b70 2024-03-15T13:03:38.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff65203d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a281b830 con 0x5556a282b750 2024-03-15T13:03:38.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff64d201640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822250 0x5556a281b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff64d201640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822250 0x5556a281b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51428/0 (socket says 172.21.15.2:51428) 2024-03-15T13:03:38.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff64d201640 1 -- 172.21.15.2:0/799149299 learned_addr learned my addr 172.21.15.2:0/799149299 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff64ca00640 1 --2- 172.21.15.2:0/799149299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822b70 0x5556a282aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff64da02640 1 --2- 172.21.15.2:0/799149299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a282b750 0x5556a282b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff64d201640 1 -- 172.21.15.2:0/799149299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a282b750 msgr2=0x5556a282b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff64d201640 1 --2- 172.21.15.2:0/799149299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a282b750 0x5556a282b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff64d201640 1 -- 172.21.15.2:0/799149299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822b70 msgr2=0x5556a282aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff64d201640 1 --2- 172.21.15.2:0/799149299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822b70 0x5556a282aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff64d201640 1 -- 172.21.15.2:0/799149299 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556a2724c20 con 0x5556a2822250 2024-03-15T13:03:38.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff64da02640 1 --2- 172.21.15.2:0/799149299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a282b750 0x5556a282b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:03:38.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff64d201640 1 --2- 172.21.15.2:0/799149299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822250 0x5556a281b2f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff6440cb870 tx=0x7ff6440cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaee404653c522c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff632ffd640 1 -- 172.21.15.2:0/799149299 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64409a5d0 con 0x5556a2822250 2024-03-15T13:03:38.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff632ffd640 1 -- 172.21.15.2:0/799149299 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5556a24c6720 con 0x5556a2822250 2024-03-15T13:03:38.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.743+0000 7ff632ffd640 1 -- 172.21.15.2:0/799149299 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644010be0 con 0x5556a2822250 2024-03-15T13:03:38.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.744+0000 7ff65203d740 1 -- 172.21.15.2:0/799149299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822250 msgr2=0x5556a281b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.744+0000 7ff65203d740 1 --2- 172.21.15.2:0/799149299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822250 0x5556a281b2f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff6440cb870 tx=0x7ff6440cd040 comp rx=0 tx=0).stop 2024-03-15T13:03:38.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.744+0000 7ff65203d740 1 -- 172.21.15.2:0/799149299 shutdown_connections 2024-03-15T13:03:38.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.744+0000 7ff65203d740 1 --2- 172.21.15.2:0/799149299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a282b750 0x5556a282b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.744+0000 7ff65203d740 1 --2- 172.21.15.2:0/799149299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822b70 0x5556a282aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.744+0000 7ff65203d740 1 --2- 172.21.15.2:0/799149299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a2822250 0x5556a281b2f0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.744+0000 7ff65203d740 1 -- 172.21.15.2:0/799149299 >> 172.21.15.2:0/799149299 conn(0x5556a281c980 msgr2=0x5556a2823920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.744+0000 7ff65203d740 1 -- 172.21.15.2:0/799149299 shutdown_connections 2024-03-15T13:03:38.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.744+0000 7ff65203d740 1 -- 172.21.15.2:0/799149299 wait complete. 2024-03-15T13:03:38.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.744+0000 7ff65203d740 1 -- start start 2024-03-15T13:03:38.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.744+0000 7ff65203d740 1 Processor -- start 2024-03-15T13:03:38.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff65203d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822250 0x5556a2833030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff65203d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a2822b70 0x5556a2833570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff65203d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a282b750 0x5556a2825590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff65203d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a2825c90 con 0x5556a282b750 2024-03-15T13:03:38.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff65203d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a2825e00 con 0x5556a2822250 2024-03-15T13:03:38.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff65203d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a2825fc0 con 0x5556a2822b70 2024-03-15T13:03:38.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff65203d740 10 client.0 _ll_register_callbacks cb 0x7ff638005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:38.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff64da02640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a282b750 0x5556a2825590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff64da02640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a282b750 0x5556a2825590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51432/0 (socket says 172.21.15.2:51432) 2024-03-15T13:03:38.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff64da02640 1 -- 172.21.15.2:0/4239896019 learned_addr learned my addr 172.21.15.2:0/4239896019 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff64d201640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822250 0x5556a2833030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff64ca00640 1 --2- 172.21.15.2:0/4239896019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a2822b70 0x5556a2833570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff64d201640 1 -- 172.21.15.2:0/4239896019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a2822b70 msgr2=0x5556a2833570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff64d201640 1 --2- 172.21.15.2:0/4239896019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a2822b70 0x5556a2833570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff64d201640 1 -- 172.21.15.2:0/4239896019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a282b750 msgr2=0x5556a2825590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff64d201640 1 --2- 172.21.15.2:0/4239896019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a282b750 0x5556a2825590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff64d201640 1 -- 172.21.15.2:0/4239896019 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64819f270 con 0x5556a2822250 2024-03-15T13:03:38.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff64da02640 1 --2- 172.21.15.2:0/4239896019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a282b750 0x5556a2825590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:38.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff64d201640 1 --2- 172.21.15.2:0/4239896019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822250 0x5556a2833030 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff6440cb870 tx=0x7ff6440d1740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec0a832d1c2db643 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff630ff9640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:03:38.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff630ff9640 1 -- 172.21.15.2:0/4239896019 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64409e8a0 con 0x5556a2822250 2024-03-15T13:03:38.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff65203d740 1 -- 172.21.15.2:0/4239896019 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff640015070 con 0x5556a2822250 2024-03-15T13:03:38.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.745+0000 7ff630ff9640 1 -- 172.21.15.2:0/4239896019 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7ff64409a800 con 0x5556a2822250 2024-03-15T13:03:38.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.746+0000 7ff630ff9640 1 -- 172.21.15.2:0/4239896019 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff64409a5d0 con 0x5556a2822250 2024-03-15T13:03:38.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.746+0000 7ff65203d740 10 client.26984 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:38.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.746+0000 7ff65203d740 1 -- 172.21.15.2:0/4239896019 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5556a274a8c0 con 0x5556a2822250 2024-03-15T13:03:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.746+0000 7ff630ff9640 1 -- 172.21.15.2:0/4239896019 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff640015070 con 0x5556a2822250 2024-03-15T13:03:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.746+0000 7ff630ff9640 1 -- 172.21.15.2:0/4239896019 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7ff6440a7020 con 0x5556a2822250 2024-03-15T13:03:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.746+0000 7ff65203d740 10 client.26984 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.746+0000 7ff65203d740 10 client.26984 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:03:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.746+0000 7ff65203d740 20 client.26984 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.746+0000 7ff65203d740 1 --2- 172.21.15.2:0/4239896019 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5556a28980e0 0x5556a289a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.746+0000 7ff65203d740 4 client.26984 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:03:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.746+0000 7ff65203d740 1 -- 172.21.15.2:0/4239896019 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5556a2515890 con 0x5556a28980e0 2024-03-15T13:03:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.746+0000 7ff64ca00640 1 --2- 172.21.15.2:0/4239896019 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5556a28980e0 0x5556a289a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.746+0000 7ff630ff9640 10 client.26984 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:03:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.746+0000 7ff64ca00640 1 --2- 172.21.15.2:0/4239896019 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5556a28980e0 0x5556a289a500 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41f20317e78b3588 server_cookie=895ae81c390fc487 in_seq=0 out_seq=0 2024-03-15T13:03:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff630ff9640 1 -- 172.21.15.2:0/4239896019 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5556a2515890 con 0x5556a28980e0 2024-03-15T13:03:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff630ff9640 10 client.26984 handle_command_reply: tid=0 2024-03-15T13:03:38.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff65203d740 1 client.26984 shutdown 2024-03-15T13:03:38.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff65203d740 1 -- 172.21.15.2:0/4239896019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822250 msgr2=0x5556a2833030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff65203d740 1 --2- 172.21.15.2:0/4239896019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822250 0x5556a2833030 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff6440cb870 tx=0x7ff6440d1740 comp rx=0 tx=0).stop 2024-03-15T13:03:38.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff65203d740 1 -- 172.21.15.2:0/4239896019 shutdown_connections 2024-03-15T13:03:38.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff65203d740 1 --2- 172.21.15.2:0/4239896019 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5556a28980e0 0x5556a289a500 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff65203d740 1 --2- 172.21.15.2:0/4239896019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556a282b750 0x5556a2825590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff65203d740 1 --2- 172.21.15.2:0/4239896019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a2822b70 0x5556a2833570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff630ff9640 0 client.26984 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:03:38.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff65203d740 1 --2- 172.21.15.2:0/4239896019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556a2822250 0x5556a2833030 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff65203d740 1 -- 172.21.15.2:0/4239896019 >> 172.21.15.2:0/4239896019 conn(0x5556a281c980 msgr2=0x5556a281dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff65203d740 1 -- 172.21.15.2:0/4239896019 shutdown_connections 2024-03-15T13:03:38.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff65203d740 1 -- 172.21.15.2:0/4239896019 wait complete. 2024-03-15T13:03:38.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff65203d740 20 client.26984 trim_cache size 0 max 16384 2024-03-15T13:03:38.740 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:03:38.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff64f465640 1 -- 172.21.15.2:0/3492992787 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff6340a1ba0 msgr2=0x7ff6340a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff64f465640 1 --2- 172.21.15.2:0/3492992787 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff6340a1ba0 0x7ff6340a4060 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff640011010 tx=0x7ff640020000 comp rx=0 tx=0).stop 2024-03-15T13:03:38.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff64f465640 1 -- 172.21.15.2:0/3492992787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6481002b0 msgr2=0x7ff648199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.747+0000 7ff64f465640 1 --2- 172.21.15.2:0/3492992787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6481002b0 0x7ff648199da0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff64400e9f0 tx=0x7ff64400eeb0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.748+0000 7ff64f465640 1 -- 172.21.15.2:0/3492992787 shutdown_connections 2024-03-15T13:03:38.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.748+0000 7ff64f465640 1 --2- 172.21.15.2:0/3492992787 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff6340a1ba0 0x7ff6340a4060 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.748+0000 7ff64f465640 1 --2- 172.21.15.2:0/3492992787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff64810de60 0x7ff64819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.748+0000 7ff64f465640 1 --2- 172.21.15.2:0/3492992787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff648100bd0 0x7ff64819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.748+0000 7ff64f465640 1 --2- 172.21.15.2:0/3492992787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6481002b0 0x7ff648199da0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.748+0000 7ff64f465640 1 -- 172.21.15.2:0/3492992787 >> 172.21.15.2:0/3492992787 conn(0x7ff648076770 msgr2=0x7ff64810f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.748+0000 7ff64f465640 1 -- 172.21.15.2:0/3492992787 shutdown_connections 2024-03-15T13:03:38.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.748+0000 7ff64f465640 1 -- 172.21.15.2:0/3492992787 wait complete. 2024-03-15T13:03:38.742 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:38 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2103687357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:03:38.743 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:03:38.744 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:03:38.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:38 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2103687357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:03:38.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.863+0000 7fc56242a640 1 -- 172.21.15.2:0/2099033568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc55c10bab0 msgr2=0x7fc55c10df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.863+0000 7fc56242a640 1 --2- 172.21.15.2:0/2099033568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc55c10bab0 0x7fc55c10df20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc554009ca0 tx=0x7fc55402c180 comp rx=0 tx=0).stop 2024-03-15T13:03:38.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.863+0000 7fc56242a640 1 -- 172.21.15.2:0/2099033568 shutdown_connections 2024-03-15T13:03:38.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.863+0000 7fc56242a640 1 --2- 172.21.15.2:0/2099033568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc55c10bab0 0x7fc55c10df20 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.863+0000 7fc56242a640 1 --2- 172.21.15.2:0/2099033568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc55c103090 0x7fc55c10b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.863+0000 7fc56242a640 1 --2- 172.21.15.2:0/2099033568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc55c102770 0x7fc55c102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.863+0000 7fc56242a640 1 -- 172.21.15.2:0/2099033568 >> 172.21.15.2:0/2099033568 conn(0x7fc55c077550 msgr2=0x7fc55c077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.863+0000 7fc56242a640 1 -- 172.21.15.2:0/2099033568 shutdown_connections 2024-03-15T13:03:38.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.863+0000 7fc56242a640 1 -- 172.21.15.2:0/2099033568 wait complete. 2024-03-15T13:03:38.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.863+0000 7fc56242a640 1 Processor -- start 2024-03-15T13:03:38.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.863+0000 7fc56242a640 1 -- start start 2024-03-15T13:03:38.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc56242a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc55c102770 0x7fc55c19d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc56242a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc55c103090 0x7fc55c19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc56242a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc55c10bab0 0x7fc55c1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc56242a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c197f00 con 0x7fc55c102770 2024-03-15T13:03:38.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc56242a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c198070 con 0x7fc55c10bab0 2024-03-15T13:03:38.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc56242a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c198230 con 0x7fc55c103090 2024-03-15T13:03:38.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc55bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc55c102770 0x7fc55c19d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc55bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc55c102770 0x7fc55c19d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51454/0 (socket says 172.21.15.2:51454) 2024-03-15T13:03:38.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc55bfff640 1 -- 172.21.15.2:0/4012263264 learned_addr learned my addr 172.21.15.2:0/4012263264 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc55b7fe640 1 --2- 172.21.15.2:0/4012263264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc55c103090 0x7fc55c19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc5609c7640 1 --2- 172.21.15.2:0/4012263264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc55c10bab0 0x7fc55c1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc55bfff640 1 -- 172.21.15.2:0/4012263264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc55c103090 msgr2=0x7fc55c19dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc55bfff640 1 --2- 172.21.15.2:0/4012263264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc55c103090 0x7fc55c19dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc55bfff640 1 -- 172.21.15.2:0/4012263264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc55c10bab0 msgr2=0x7fc55c1978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc55bfff640 1 --2- 172.21.15.2:0/4012263264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc55c10bab0 0x7fc55c1978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc55bfff640 1 -- 172.21.15.2:0/4012263264 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc550009490 con 0x7fc55c102770 2024-03-15T13:03:38.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.864+0000 7fc55bfff640 1 --2- 172.21.15.2:0/4012263264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc55c102770 0x7fc55c19d700 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc55000ea60 tx=0x7fc55000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.865+0000 7fc5597fa640 1 -- 172.21.15.2:0/4012263264 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55000cde0 con 0x7fc55c102770 2024-03-15T13:03:38.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.865+0000 7fc56242a640 1 -- 172.21.15.2:0/4012263264 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc554009900 con 0x7fc55c102770 2024-03-15T13:03:38.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.865+0000 7fc5597fa640 1 -- 172.21.15.2:0/4012263264 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc5500035f0 con 0x7fc55c102770 2024-03-15T13:03:38.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.865+0000 7fc5597fa640 1 -- 172.21.15.2:0/4012263264 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc550010690 con 0x7fc55c102770 2024-03-15T13:03:38.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.865+0000 7fc56242a640 1 -- 172.21.15.2:0/4012263264 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55c1a45c0 con 0x7fc55c102770 2024-03-15T13:03:38.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.867+0000 7fc5597fa640 1 -- 172.21.15.2:0/4012263264 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fc550003760 con 0x7fc55c102770 2024-03-15T13:03:38.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.867+0000 7fc565006740 1 Processor -- start 2024-03-15T13:03:38.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.867+0000 7fc565006740 1 -- start start 2024-03-15T13:03:38.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.867+0000 7fc565006740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b4ff00 0x55b461b502e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.867+0000 7fc5597fa640 1 --2- 172.21.15.2:0/4012263264 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc5440a1ba0 0x7fc5440a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.867+0000 7fc565006740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b50820 0x55b461b5d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.867+0000 7fc565006740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b5db60 0x55b461b5ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.867+0000 7fc565006740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b461b0b960 con 0x55b461b5db60 2024-03-15T13:03:38.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.867+0000 7fc565006740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b461abf310 con 0x55b461b50820 2024-03-15T13:03:38.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.867+0000 7fc565006740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b461abc840 con 0x55b461b4ff00 2024-03-15T13:03:38.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.867+0000 7fc55bfff640 1 --2- 172.21.15.2:0/4012263264 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc5440a1ba0 0x7fc5440a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.867+0000 7fc5609c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b5db60 0x55b461b5ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.867+0000 7fc55b7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b50820 0x55b461b5d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.867+0000 7fc5609c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b5db60 0x55b461b5ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51460/0 (socket says 172.21.15.2:51460) 2024-03-15T13:03:38.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.867+0000 7fc5609c7640 1 -- 172.21.15.2:0/3986389921 learned_addr learned my addr 172.21.15.2:0/3986389921 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.868+0000 7fc5609c7640 1 --2- 172.21.15.2:0/3986389921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b4ff00 0x55b461b502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.868+0000 7fc5597fa640 1 -- 172.21.15.2:0/4012263264 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc550014070 con 0x7fc55c102770 2024-03-15T13:03:38.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.868+0000 7fc55b7fe640 1 -- 172.21.15.2:0/3986389921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b4ff00 msgr2=0x55b461b502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.868+0000 7fc55b7fe640 1 --2- 172.21.15.2:0/3986389921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b4ff00 0x55b461b502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.868+0000 7fc55b7fe640 1 -- 172.21.15.2:0/3986389921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b5db60 msgr2=0x55b461b5ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.868+0000 7fc55b7fe640 1 --2- 172.21.15.2:0/3986389921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b5db60 0x55b461b5ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.868+0000 7fc55b7fe640 1 -- 172.21.15.2:0/3986389921 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b461a7a060 con 0x55b461b50820 2024-03-15T13:03:38.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.868+0000 7fc55bfff640 1 --2- 172.21.15.2:0/4012263264 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc5440a1ba0 0x7fc5440a4060 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc550099730 tx=0x7fc55008ff40 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.868+0000 7fc55b7fe640 1 --2- 172.21.15.2:0/3986389921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b50820 0x55b461b5d620 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc55c1990a0 tx=0x7fc54c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44e7aaaa7776a83 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.868+0000 7fc5427fc640 1 -- 172.21.15.2:0/3986389921 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c004050 con 0x55b461b50820 2024-03-15T13:03:38.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.868+0000 7fc5427fc640 1 -- 172.21.15.2:0/3986389921 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc54c005020 con 0x55b461b50820 2024-03-15T13:03:38.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.868+0000 7fc5427fc640 1 -- 172.21.15.2:0/3986389921 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c004b20 con 0x55b461b50820 2024-03-15T13:03:38.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.869+0000 7fc565006740 1 -- 172.21.15.2:0/3986389921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b50820 msgr2=0x55b461b5d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.869+0000 7fc565006740 1 --2- 172.21.15.2:0/3986389921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b50820 0x55b461b5d620 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc55c1990a0 tx=0x7fc54c011810 comp rx=0 tx=0).stop 2024-03-15T13:03:38.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.869+0000 7fc565006740 1 -- 172.21.15.2:0/3986389921 shutdown_connections 2024-03-15T13:03:38.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.869+0000 7fc565006740 1 --2- 172.21.15.2:0/3986389921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b5db60 0x55b461b5ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.869+0000 7fc565006740 1 --2- 172.21.15.2:0/3986389921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b50820 0x55b461b5d620 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.869+0000 7fc565006740 1 --2- 172.21.15.2:0/3986389921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b4ff00 0x55b461b502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.869+0000 7fc565006740 1 -- 172.21.15.2:0/3986389921 >> 172.21.15.2:0/3986389921 conn(0x55b461b4a980 msgr2=0x55b461b4ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.869+0000 7fc565006740 1 -- 172.21.15.2:0/3986389921 shutdown_connections 2024-03-15T13:03:38.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.869+0000 7fc565006740 1 -- 172.21.15.2:0/3986389921 wait complete. 2024-03-15T13:03:38.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.869+0000 7fc565006740 1 -- start start 2024-03-15T13:03:38.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.869+0000 7fc565006740 1 Processor -- start 2024-03-15T13:03:38.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc565006740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b4ff00 0x55b461b5d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc565006740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b5a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc565006740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b5db60 0x55b461b5ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc565006740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b461ab8020 con 0x55b461b50820 2024-03-15T13:03:38.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc565006740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46178b430 con 0x55b461b4ff00 2024-03-15T13:03:38.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc565006740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b461747e00 con 0x55b461b5db60 2024-03-15T13:03:38.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc565006740 10 client.0 _ll_register_callbacks cb 0x55b461a5fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:38.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc55b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b5a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc55b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b5a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51466/0 (socket says 172.21.15.2:51466) 2024-03-15T13:03:38.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc55b7fe640 1 -- 172.21.15.2:0/2975171286 learned_addr learned my addr 172.21.15.2:0/2975171286 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc5609c7640 1 --2- 172.21.15.2:0/2975171286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b4ff00 0x55b461b5d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc5609c7640 1 --2- 172.21.15.2:0/2975171286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b5db60 0x55b461b5ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc55b7fe640 1 -- 172.21.15.2:0/2975171286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b5db60 msgr2=0x55b461b5ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc55b7fe640 1 --2- 172.21.15.2:0/2975171286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b5db60 0x55b461b5ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc55b7fe640 1 -- 172.21.15.2:0/2975171286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b4ff00 msgr2=0x55b461b5d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc55b7fe640 1 --2- 172.21.15.2:0/2975171286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b4ff00 0x55b461b5d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc55b7fe640 1 -- 172.21.15.2:0/2975171286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c009830 con 0x55b461b50820 2024-03-15T13:03:38.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc55b7fe640 1 --2- 172.21.15.2:0/2975171286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b5a810 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc54c01b040 tx=0x7fc54c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d11745835cb560 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc53b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:38.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc53b7fe640 1 -- 172.21.15.2:0/2975171286 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c0044e0 con 0x55b461b50820 2024-03-15T13:03:38.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc53b7fe640 1 -- 172.21.15.2:0/2975171286 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc54c0048a0 con 0x55b461b50820 2024-03-15T13:03:38.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.870+0000 7fc565006740 1 -- 172.21.15.2:0/2975171286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4617b77e0 con 0x55b461b50820 2024-03-15T13:03:38.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.871+0000 7fc53b7fe640 1 -- 172.21.15.2:0/2975171286 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc54c004a60 con 0x55b461b50820 2024-03-15T13:03:38.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.871+0000 7fc565006740 10 client.17133 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:38.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.871+0000 7fc565006740 1 -- 172.21.15.2:0/2975171286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b461ab66a0 con 0x55b461b50820 2024-03-15T13:03:38.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.871+0000 7fc53b7fe640 1 -- 172.21.15.2:0/2975171286 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55b4617b77e0 con 0x55b461b50820 2024-03-15T13:03:38.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.871+0000 7fc53b7fe640 1 -- 172.21.15.2:0/2975171286 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7fc54c021040 con 0x55b461b50820 2024-03-15T13:03:38.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.871+0000 7fc565006740 10 client.17133 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:38.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.871+0000 7fc565006740 10 client.17133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:38.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.871+0000 7fc565006740 20 client.17133 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:38.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.871+0000 7fc565006740 1 --2- 172.21.15.2:0/2975171286 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b461b6dce0 0x55b461b70100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.871+0000 7fc565006740 4 client.17133 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:38.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.871+0000 7fc565006740 1 -- 172.21.15.2:0/2975171286 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4617f4720 con 0x55b461b6dce0 2024-03-15T13:03:38.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.871+0000 7fc5609c7640 1 --2- 172.21.15.2:0/2975171286 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b461b6dce0 0x55b461b70100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.872+0000 7fc53b7fe640 10 client.17133 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:38.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.872+0000 7fc5609c7640 1 --2- 172.21.15.2:0/2975171286 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b461b6dce0 0x55b461b70100 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39bf58e59bcea66 server_cookie=6721847176780f57 in_seq=0 out_seq=0 2024-03-15T13:03:38.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.872+0000 7fc53b7fe640 1 -- 172.21.15.2:0/2975171286 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b4617f4720 con 0x55b461b6dce0 2024-03-15T13:03:38.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.872+0000 7fc53b7fe640 10 client.17133 handle_command_reply: tid=0 2024-03-15T13:03:38.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.872+0000 7fc565006740 1 client.17133 shutdown 2024-03-15T13:03:38.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.872+0000 7fc565006740 1 -- 172.21.15.2:0/2975171286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 msgr2=0x55b461b5a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.872+0000 7fc565006740 1 --2- 172.21.15.2:0/2975171286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b5a810 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc54c01b040 tx=0x7fc54c004740 comp rx=0 tx=0).stop 2024-03-15T13:03:38.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.872+0000 7fc565006740 1 -- 172.21.15.2:0/2975171286 shutdown_connections 2024-03-15T13:03:38.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.872+0000 7fc565006740 1 --2- 172.21.15.2:0/2975171286 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b461b6dce0 0x55b461b70100 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.872+0000 7fc565006740 1 --2- 172.21.15.2:0/2975171286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b5db60 0x55b461b5ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.872+0000 7fc565006740 1 --2- 172.21.15.2:0/2975171286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b5a810 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.872+0000 7fc53b7fe640 0 client.17133 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:38.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.872+0000 7fc565006740 1 --2- 172.21.15.2:0/2975171286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b4ff00 0x55b461b5d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.872+0000 7fc565006740 1 -- 172.21.15.2:0/2975171286 >> 172.21.15.2:0/2975171286 conn(0x55b461b4a980 msgr2=0x55b461b4c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.873+0000 7fc565006740 1 -- 172.21.15.2:0/2975171286 shutdown_connections 2024-03-15T13:03:38.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.873+0000 7fc565006740 1 -- 172.21.15.2:0/2975171286 wait complete. 2024-03-15T13:03:38.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.873+0000 7fc565006740 20 client.17133 trim_cache size 0 max 16384 2024-03-15T13:03:38.866 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:38 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2103687357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:03:38.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc565006740 1 Processor -- start 2024-03-15T13:03:38.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc565006740 1 -- start start 2024-03-15T13:03:38.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc565006740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b4ff00 0x55b461b5cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc565006740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b5d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc565006740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b5db60 0x55b461b68fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc565006740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b461a3d130 con 0x55b461b50820 2024-03-15T13:03:38.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc565006740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b461b4fbc0 con 0x55b461b5db60 2024-03-15T13:03:38.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc565006740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b461b6a7b0 con 0x55b461b4ff00 2024-03-15T13:03:38.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc55b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b5d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc55b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b5d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51482/0 (socket says 172.21.15.2:51482) 2024-03-15T13:03:38.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc55b7fe640 1 -- 172.21.15.2:0/1728162473 learned_addr learned my addr 172.21.15.2:0/1728162473 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc5609c7640 1 --2- 172.21.15.2:0/1728162473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b4ff00 0x55b461b5cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc5609c7640 1 --2- 172.21.15.2:0/1728162473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b5db60 0x55b461b68fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc55b7fe640 1 -- 172.21.15.2:0/1728162473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b4ff00 msgr2=0x55b461b5cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc55b7fe640 1 --2- 172.21.15.2:0/1728162473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b4ff00 0x55b461b5cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc55b7fe640 1 -- 172.21.15.2:0/1728162473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b5db60 msgr2=0x55b461b68fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc55b7fe640 1 --2- 172.21.15.2:0/1728162473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b5db60 0x55b461b68fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.885+0000 7fc55b7fe640 1 -- 172.21.15.2:0/1728162473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b461a52c20 con 0x55b461b50820 2024-03-15T13:03:38.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.886+0000 7fc55b7fe640 1 --2- 172.21.15.2:0/1728162473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b5d480 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009710 tx=0x7fc54c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6484c158fe1752 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.886+0000 7fc541ffb640 1 -- 172.21.15.2:0/1728162473 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c009a50 con 0x55b461b50820 2024-03-15T13:03:38.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.886+0000 7fc541ffb640 1 -- 172.21.15.2:0/1728162473 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc54c005d30 con 0x55b461b50820 2024-03-15T13:03:38.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.886+0000 7fc541ffb640 1 -- 172.21.15.2:0/1728162473 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c0047e0 con 0x55b461b50820 2024-03-15T13:03:38.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.886+0000 7fc565006740 1 -- 172.21.15.2:0/1728162473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 msgr2=0x55b461b5d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.886+0000 7fc565006740 1 --2- 172.21.15.2:0/1728162473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b5d480 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009710 tx=0x7fc54c004060 comp rx=0 tx=0).stop 2024-03-15T13:03:38.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.886+0000 7fc565006740 1 -- 172.21.15.2:0/1728162473 shutdown_connections 2024-03-15T13:03:38.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.886+0000 7fc565006740 1 --2- 172.21.15.2:0/1728162473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b5db60 0x55b461b68fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.886+0000 7fc565006740 1 --2- 172.21.15.2:0/1728162473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b5d480 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.886+0000 7fc565006740 1 --2- 172.21.15.2:0/1728162473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b4ff00 0x55b461b5cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.886+0000 7fc565006740 1 -- 172.21.15.2:0/1728162473 >> 172.21.15.2:0/1728162473 conn(0x55b461b4a980 msgr2=0x55b461b49b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.886+0000 7fc565006740 1 -- 172.21.15.2:0/1728162473 shutdown_connections 2024-03-15T13:03:38.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.886+0000 7fc565006740 1 -- 172.21.15.2:0/1728162473 wait complete. 2024-03-15T13:03:38.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.887+0000 7fc565006740 1 -- start start 2024-03-15T13:03:38.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.887+0000 7fc565006740 1 Processor -- start 2024-03-15T13:03:38.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.887+0000 7fc565006740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b4ff00 0x55b461b65910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.887+0000 7fc565006740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b60eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.887+0000 7fc565006740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b5db60 0x55b461b614c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.887+0000 7fc565006740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b461b61b90 con 0x55b461b50820 2024-03-15T13:03:38.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.887+0000 7fc565006740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b461b61d00 con 0x55b461b5db60 2024-03-15T13:03:38.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.887+0000 7fc565006740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b461bc8590 con 0x55b461b4ff00 2024-03-15T13:03:38.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.887+0000 7fc565006740 10 client.0 _ll_register_callbacks cb 0x7fc550005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:38.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.887+0000 7fc55b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b60eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.887+0000 7fc55b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b60eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51492/0 (socket says 172.21.15.2:51492) 2024-03-15T13:03:38.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.887+0000 7fc55b7fe640 1 -- 172.21.15.2:0/4168890186 learned_addr learned my addr 172.21.15.2:0/4168890186 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:38.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.887+0000 7fc5609c7640 1 --2- 172.21.15.2:0/4168890186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b4ff00 0x55b461b65910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.887+0000 7fc5609c7640 1 --2- 172.21.15.2:0/4168890186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b5db60 0x55b461b614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc55b7fe640 1 -- 172.21.15.2:0/4168890186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b4ff00 msgr2=0x55b461b65910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc55b7fe640 1 --2- 172.21.15.2:0/4168890186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b4ff00 0x55b461b65910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc55b7fe640 1 -- 172.21.15.2:0/4168890186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b5db60 msgr2=0x55b461b614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:38.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc55b7fe640 1 --2- 172.21.15.2:0/4168890186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b5db60 0x55b461b614c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc55b7fe640 1 -- 172.21.15.2:0/4168890186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b461ab66a0 con 0x55b461b50820 2024-03-15T13:03:38.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc55b7fe640 1 --2- 172.21.15.2:0/4168890186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b60eb0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009350 tx=0x7fc54c0097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444cd1fe0daafa14 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:38.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc53b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:38.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc53b7fe640 1 -- 172.21.15.2:0/4168890186 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c023a10 con 0x55b461b50820 2024-03-15T13:03:38.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc53b7fe640 1 -- 172.21.15.2:0/4168890186 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc54c000e40 con 0x55b461b50820 2024-03-15T13:03:38.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc565006740 1 -- 172.21.15.2:0/4168890186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc54c005020 con 0x55b461b50820 2024-03-15T13:03:38.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc53b7fe640 1 -- 172.21.15.2:0/4168890186 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc54c023b70 con 0x55b461b50820 2024-03-15T13:03:38.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc565006740 10 client.17139 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:38.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc565006740 1 -- 172.21.15.2:0/4168890186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b461a788c0 con 0x55b461b50820 2024-03-15T13:03:38.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc53b7fe640 1 -- 172.21.15.2:0/4168890186 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc54c005020 con 0x55b461b50820 2024-03-15T13:03:38.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc53b7fe640 1 -- 172.21.15.2:0/4168890186 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7fc54c0044a0 con 0x55b461b50820 2024-03-15T13:03:38.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.888+0000 7fc565006740 10 client.17139 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:38.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc565006740 10 client.17139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:38.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc565006740 20 client.17139 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:38.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc565006740 1 --2- 172.21.15.2:0/4168890186 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b461bc9ab0 0x55b461bcbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:38.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc565006740 4 client.17139 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:03:38.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc565006740 1 -- 172.21.15.2:0/4168890186 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b461843890 con 0x55b461bc9ab0 2024-03-15T13:03:38.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc5609c7640 1 --2- 172.21.15.2:0/4168890186 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b461bc9ab0 0x55b461bcbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:38.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc53b7fe640 10 client.17139 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:38.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc5609c7640 1 --2- 172.21.15.2:0/4168890186 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b461bc9ab0 0x55b461bcbea0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff459a1467fadf68 server_cookie=b5da0bf025260b19 in_seq=0 out_seq=0 2024-03-15T13:03:38.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc53b7fe640 1 -- 172.21.15.2:0/4168890186 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b461843890 con 0x55b461bc9ab0 2024-03-15T13:03:38.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc53b7fe640 10 client.17139 handle_command_reply: tid=0 2024-03-15T13:03:38.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc565006740 1 client.17139 shutdown 2024-03-15T13:03:38.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc565006740 1 -- 172.21.15.2:0/4168890186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 msgr2=0x55b461b60eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc565006740 1 --2- 172.21.15.2:0/4168890186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b60eb0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009350 tx=0x7fc54c0097a0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc565006740 1 -- 172.21.15.2:0/4168890186 shutdown_connections 2024-03-15T13:03:38.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc565006740 1 --2- 172.21.15.2:0/4168890186 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55b461bc9ab0 0x55b461bcbea0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc565006740 1 --2- 172.21.15.2:0/4168890186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b461b5db60 0x55b461b614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc565006740 1 --2- 172.21.15.2:0/4168890186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b461b50820 0x55b461b60eb0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc53b7fe640 0 client.17139 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:38.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc565006740 1 --2- 172.21.15.2:0/4168890186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b461b4ff00 0x55b461b65910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.889+0000 7fc565006740 1 -- 172.21.15.2:0/4168890186 >> 172.21.15.2:0/4168890186 conn(0x55b461b4a980 msgr2=0x55b461b5a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.890+0000 7fc565006740 1 -- 172.21.15.2:0/4168890186 shutdown_connections 2024-03-15T13:03:38.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.890+0000 7fc565006740 1 -- 172.21.15.2:0/4168890186 wait complete. 2024-03-15T13:03:38.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.890+0000 7fc565006740 20 client.17139 trim_cache size 0 max 16384 2024-03-15T13:03:38.882 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:03:38.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.890+0000 7fc56242a640 1 -- 172.21.15.2:0/4012263264 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc5440a1ba0 msgr2=0x7fc5440a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.890+0000 7fc56242a640 1 --2- 172.21.15.2:0/4012263264 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc5440a1ba0 0x7fc5440a4060 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc550099730 tx=0x7fc55008ff40 comp rx=0 tx=0).stop 2024-03-15T13:03:38.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.890+0000 7fc56242a640 1 -- 172.21.15.2:0/4012263264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc55c102770 msgr2=0x7fc55c19d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:38.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.890+0000 7fc56242a640 1 --2- 172.21.15.2:0/4012263264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc55c102770 0x7fc55c19d700 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc55000ea60 tx=0x7fc55000ef20 comp rx=0 tx=0).stop 2024-03-15T13:03:38.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.890+0000 7fc56242a640 1 -- 172.21.15.2:0/4012263264 shutdown_connections 2024-03-15T13:03:38.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.890+0000 7fc56242a640 1 --2- 172.21.15.2:0/4012263264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc55c10bab0 0x7fc55c1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.890+0000 7fc56242a640 1 --2- 172.21.15.2:0/4012263264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc55c103090 0x7fc55c19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.890+0000 7fc56242a640 1 --2- 172.21.15.2:0/4012263264 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc5440a1ba0 0x7fc5440a4060 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.890+0000 7fc56242a640 1 --2- 172.21.15.2:0/4012263264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc55c102770 0x7fc55c19d700 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:38.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.890+0000 7fc56242a640 1 -- 172.21.15.2:0/4012263264 >> 172.21.15.2:0/4012263264 conn(0x7fc55c077550 msgr2=0x7fc55c0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:38.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.890+0000 7fc56242a640 1 -- 172.21.15.2:0/4012263264 shutdown_connections 2024-03-15T13:03:38.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:38.890+0000 7fc56242a640 1 -- 172.21.15.2:0/4012263264 wait complete. 2024-03-15T13:03:39.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:39 smithi002 ceph-mon[32060]: pgmap v145: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:39.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:39 smithi184 ceph-mon[35993]: pgmap v145: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:39.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:39 smithi149 ceph-mon[35927]: pgmap v145: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:39.886 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:03:39.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:03:39.887 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:03:40.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.022+0000 7f33a258c640 1 -- 172.21.15.2:0/392451899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f339c1056a0 msgr2=0x7f339c10d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.022+0000 7f33a258c640 1 --2- 172.21.15.2:0/392451899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f339c1056a0 0x7f339c10d270 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3394009cd0 tx=0x7f339402c0f0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.022+0000 7f33a258c640 1 -- 172.21.15.2:0/392451899 shutdown_connections 2024-03-15T13:03:40.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.022+0000 7f33a258c640 1 --2- 172.21.15.2:0/392451899 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f339c1056a0 0x7f339c10d270 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.022+0000 7f33a258c640 1 --2- 172.21.15.2:0/392451899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f339c100f80 0x7f339c104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.022+0000 7f33a258c640 1 --2- 172.21.15.2:0/392451899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f339c1005d0 0x7f339c1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.022+0000 7f33a258c640 1 -- 172.21.15.2:0/392451899 >> 172.21.15.2:0/392451899 conn(0x7f339c0774f0 msgr2=0x7f339c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:40.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.022+0000 7f33a258c640 1 -- 172.21.15.2:0/392451899 shutdown_connections 2024-03-15T13:03:40.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.022+0000 7f33a258c640 1 -- 172.21.15.2:0/392451899 wait complete. 2024-03-15T13:03:40.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.022+0000 7f33a258c640 1 Processor -- start 2024-03-15T13:03:40.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.022+0000 7f33a258c640 1 -- start start 2024-03-15T13:03:40.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f33a258c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f339c1005d0 0x7f339c19d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f33a258c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f339c1056a0 0x7f339c19dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f33a258c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f339c197920 0x7f339c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f33a258c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339c198310 con 0x7f339c1056a0 2024-03-15T13:03:40.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f33a258c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339c198480 con 0x7f339c1005d0 2024-03-15T13:03:40.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f33a258c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339c198640 con 0x7f339c197920 2024-03-15T13:03:40.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f339b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f339c1056a0 0x7f339c19dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f339b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f339c1056a0 0x7f339c19dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51504/0 (socket says 172.21.15.2:51504) 2024-03-15T13:03:40.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f339b7fe640 1 -- 172.21.15.2:0/3794772077 learned_addr learned my addr 172.21.15.2:0/3794772077 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:40.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f33a0b29640 1 --2- 172.21.15.2:0/3794772077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f339c197920 0x7f339c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f339bfff640 1 --2- 172.21.15.2:0/3794772077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f339c1005d0 0x7f339c19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f339b7fe640 1 -- 172.21.15.2:0/3794772077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f339c197920 msgr2=0x7f339c197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f339b7fe640 1 --2- 172.21.15.2:0/3794772077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f339c197920 0x7f339c197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f339b7fe640 1 -- 172.21.15.2:0/3794772077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f339c1005d0 msgr2=0x7f339c19d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f339b7fe640 1 --2- 172.21.15.2:0/3794772077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f339c1005d0 0x7f339c19d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f339b7fe640 1 -- 172.21.15.2:0/3794772077 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c009490 con 0x7f339c1056a0 2024-03-15T13:03:40.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.023+0000 7f339b7fe640 1 --2- 172.21.15.2:0/3794772077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f339c1056a0 0x7f339c19dcb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f339000e9f0 tx=0x7f339000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:40.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.024+0000 7f33997fa640 1 -- 172.21.15.2:0/3794772077 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33900096c0 con 0x7f339c1056a0 2024-03-15T13:03:40.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.024+0000 7f33a258c640 1 -- 172.21.15.2:0/3794772077 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3394009900 con 0x7f339c1056a0 2024-03-15T13:03:40.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.024+0000 7f33997fa640 1 -- 172.21.15.2:0/3794772077 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f33900054f0 con 0x7f339c1056a0 2024-03-15T13:03:40.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.024+0000 7f33997fa640 1 -- 172.21.15.2:0/3794772077 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3390010430 con 0x7f339c1056a0 2024-03-15T13:03:40.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.024+0000 7f33a258c640 1 -- 172.21.15.2:0/3794772077 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339c1a4960 con 0x7f339c1056a0 2024-03-15T13:03:40.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.026+0000 7f33997fa640 1 -- 172.21.15.2:0/3794772077 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f3390005000 con 0x7f339c1056a0 2024-03-15T13:03:40.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.026+0000 7f33a5169740 1 Processor -- start 2024-03-15T13:03:40.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.026+0000 7f33a5169740 1 -- start start 2024-03-15T13:03:40.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.026+0000 7f33a5169740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384260 0x564ec6384640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.026+0000 7f33a5169740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384c10 0x564ec638d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.026+0000 7f33a5169740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec638dd10 0x564ec638e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.026+0000 7f33997fa640 1 --2- 172.21.15.2:0/3794772077 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f33840a1ba0 0x7f33840a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.026+0000 7f33a5169740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec62f2f90 con 0x564ec6384260 2024-03-15T13:03:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.026+0000 7f33a5169740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec62f04c0 con 0x564ec6384c10 2024-03-15T13:03:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.026+0000 7f33a5169740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec62ebca0 con 0x564ec638dd10 2024-03-15T13:03:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.026+0000 7f33a0b29640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384260 0x564ec6384640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.026+0000 7f339b7fe640 1 --2- 172.21.15.2:0/3794772077 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f33840a1ba0 0x7f33840a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.026+0000 7f33a0b29640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384260 0x564ec6384640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51508/0 (socket says 172.21.15.2:51508) 2024-03-15T13:03:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.027+0000 7f33a0b29640 1 -- 172.21.15.2:0/1275600358 learned_addr learned my addr 172.21.15.2:0/1275600358 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.027+0000 7f339bfff640 1 --2- 172.21.15.2:0/1275600358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384c10 0x564ec638d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.027+0000 7f33997fa640 1 -- 172.21.15.2:0/3794772077 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f3390014070 con 0x7f339c1056a0 2024-03-15T13:03:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.027+0000 7f33a0b29640 1 --2- 172.21.15.2:0/1275600358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec638dd10 0x564ec638e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.027+0000 7f339bfff640 1 -- 172.21.15.2:0/1275600358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec638dd10 msgr2=0x564ec638e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.027+0000 7f339bfff640 1 --2- 172.21.15.2:0/1275600358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec638dd10 0x564ec638e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.027+0000 7f339bfff640 1 -- 172.21.15.2:0/1275600358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384260 msgr2=0x564ec6384640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.027+0000 7f339bfff640 1 --2- 172.21.15.2:0/1275600358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384260 0x564ec6384640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.027+0000 7f339bfff640 1 -- 172.21.15.2:0/1275600358 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ec62ae0b0 con 0x564ec6384c10 2024-03-15T13:03:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.027+0000 7f33a0b29640 1 --2- 172.21.15.2:0/1275600358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384260 0x564ec6384640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.027+0000 7f339bfff640 1 --2- 172.21.15.2:0/1275600358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384c10 0x564ec638d740 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f338c009d80 tx=0x7f338c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e49313375575d2b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.027+0000 7f339b7fe640 1 --2- 172.21.15.2:0/3794772077 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f33840a1ba0 0x7f33840a4060 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f33900cb2d0 tx=0x7f3390085040 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.027+0000 7f33827fc640 1 -- 172.21.15.2:0/1275600358 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c004250 con 0x564ec6384c10 2024-03-15T13:03:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.027+0000 7f33827fc640 1 -- 172.21.15.2:0/1275600358 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f338c0043b0 con 0x564ec6384c10 2024-03-15T13:03:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.027+0000 7f33827fc640 1 -- 172.21.15.2:0/1275600358 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c019980 con 0x564ec6384c10 2024-03-15T13:03:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.028+0000 7f33a5169740 1 -- 172.21.15.2:0/1275600358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384c10 msgr2=0x564ec638d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.028+0000 7f33a5169740 1 --2- 172.21.15.2:0/1275600358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384c10 0x564ec638d740 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f338c009d80 tx=0x7f338c011810 comp rx=0 tx=0).stop 2024-03-15T13:03:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.028+0000 7f33a5169740 1 -- 172.21.15.2:0/1275600358 shutdown_connections 2024-03-15T13:03:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.028+0000 7f33a5169740 1 --2- 172.21.15.2:0/1275600358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec638dd10 0x564ec638e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.028+0000 7f33a5169740 1 --2- 172.21.15.2:0/1275600358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384c10 0x564ec638d740 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.028+0000 7f33a5169740 1 --2- 172.21.15.2:0/1275600358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384260 0x564ec6384640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.028+0000 7f33a5169740 1 -- 172.21.15.2:0/1275600358 >> 172.21.15.2:0/1275600358 conn(0x564ec637e840 msgr2=0x564ec637ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.028+0000 7f33a5169740 1 -- 172.21.15.2:0/1275600358 shutdown_connections 2024-03-15T13:03:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.028+0000 7f33a5169740 1 -- 172.21.15.2:0/1275600358 wait complete. 2024-03-15T13:03:40.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.028+0000 7f33a5169740 1 -- start start 2024-03-15T13:03:40.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a5169740 1 Processor -- start 2024-03-15T13:03:40.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a5169740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384260 0x564ec63917e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a5169740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec6384c10 0x564ec6391d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a5169740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec638dd10 0x564ec6392260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a5169740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec6290f40 con 0x564ec638dd10 2024-03-15T13:03:40.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a5169740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec5fbf4b0 con 0x564ec6384260 2024-03-15T13:03:40.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a5169740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec6271670 con 0x564ec6384c10 2024-03-15T13:03:40.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a5169740 10 client.0 _ll_register_callbacks cb 0x564ec62def70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:40.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a0b29640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec638dd10 0x564ec6392260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a0b29640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec638dd10 0x564ec6392260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51512/0 (socket says 172.21.15.2:51512) 2024-03-15T13:03:40.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a0b29640 1 -- 172.21.15.2:0/3743245266 learned_addr learned my addr 172.21.15.2:0/3743245266 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:40.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a0b29640 1 --2- 172.21.15.2:0/3743245266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384260 0x564ec63917e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f339bfff640 1 --2- 172.21.15.2:0/3743245266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec6384c10 0x564ec6391d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a0b29640 1 -- 172.21.15.2:0/3743245266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec6384c10 msgr2=0x564ec6391d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a0b29640 1 --2- 172.21.15.2:0/3743245266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec6384c10 0x564ec6391d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a0b29640 1 -- 172.21.15.2:0/3743245266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384260 msgr2=0x564ec63917e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a0b29640 1 --2- 172.21.15.2:0/3743245266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384260 0x564ec63917e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.029+0000 7f33a0b29640 1 -- 172.21.15.2:0/3743245266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338c0099e0 con 0x564ec638dd10 2024-03-15T13:03:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f33a0b29640 1 --2- 172.21.15.2:0/3743245266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec638dd10 0x564ec6392260 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3394005cc0 tx=0x7f3394005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2c2b8e2b0edf31 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f337b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f337b7fe640 1 -- 172.21.15.2:0/3743245266 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394032a90 con 0x564ec638dd10 2024-03-15T13:03:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f337b7fe640 1 -- 172.21.15.2:0/3743245266 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f3394003970 con 0x564ec638dd10 2024-03-15T13:03:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f33a5169740 1 -- 172.21.15.2:0/3743245266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ec62c5f40 con 0x564ec638dd10 2024-03-15T13:03:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f337b7fe640 1 -- 172.21.15.2:0/3743245266 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3394003b30 con 0x564ec638dd10 2024-03-15T13:03:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f33a5169740 10 client.17151 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f33a5169740 1 -- 172.21.15.2:0/3743245266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564ec62ea320 con 0x564ec638dd10 2024-03-15T13:03:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f337b7fe640 1 -- 172.21.15.2:0/3743245266 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x564ec62c5f40 con 0x564ec638dd10 2024-03-15T13:03:40.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f337b7fe640 1 -- 172.21.15.2:0/3743245266 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7f3394044440 con 0x564ec638dd10 2024-03-15T13:03:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f33a5169740 10 client.17151 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f33a5169740 10 client.17151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f33a5169740 20 client.17151 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f33a5169740 1 --2- 172.21.15.2:0/3743245266 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564ec63a2300 0x564ec63a4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f33a5169740 4 client.17151 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.030+0000 7f33a5169740 1 -- 172.21.15.2:0/3743245266 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ec62e2310 con 0x564ec63a2300 2024-03-15T13:03:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.031+0000 7f339bfff640 1 --2- 172.21.15.2:0/3743245266 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564ec63a2300 0x564ec63a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.031+0000 7f337b7fe640 10 client.17151 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.031+0000 7f339bfff640 1 --2- 172.21.15.2:0/3743245266 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564ec63a2300 0x564ec63a4720 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de62df6be0ec4dd2 server_cookie=fab08e12cfc09dc6 in_seq=0 out_seq=0 2024-03-15T13:03:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.032+0000 7f337b7fe640 1 -- 172.21.15.2:0/3743245266 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564ec62e2310 con 0x564ec63a2300 2024-03-15T13:03:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.032+0000 7f337b7fe640 10 client.17151 handle_command_reply: tid=0 2024-03-15T13:03:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.032+0000 7f33a5169740 1 client.17151 shutdown 2024-03-15T13:03:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.032+0000 7f33a5169740 1 -- 172.21.15.2:0/3743245266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec638dd10 msgr2=0x564ec6392260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.032+0000 7f33a5169740 1 --2- 172.21.15.2:0/3743245266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec638dd10 0x564ec6392260 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3394005cc0 tx=0x7f3394005c50 comp rx=0 tx=0).stop 2024-03-15T13:03:40.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.032+0000 7f33a5169740 1 -- 172.21.15.2:0/3743245266 shutdown_connections 2024-03-15T13:03:40.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.032+0000 7f33a5169740 1 --2- 172.21.15.2:0/3743245266 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564ec63a2300 0x564ec63a4720 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.032+0000 7f33a5169740 1 --2- 172.21.15.2:0/3743245266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec638dd10 0x564ec6392260 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.032+0000 7f33a5169740 1 --2- 172.21.15.2:0/3743245266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec6384c10 0x564ec6391d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.032+0000 7f33a5169740 1 --2- 172.21.15.2:0/3743245266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384260 0x564ec63917e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.032+0000 7f337b7fe640 0 client.17151 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:40.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.032+0000 7f33a5169740 1 -- 172.21.15.2:0/3743245266 >> 172.21.15.2:0/3743245266 conn(0x564ec637e840 msgr2=0x564ec63807a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:40.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.032+0000 7f33a5169740 1 -- 172.21.15.2:0/3743245266 shutdown_connections 2024-03-15T13:03:40.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.032+0000 7f33a5169740 1 -- 172.21.15.2:0/3743245266 wait complete. 2024-03-15T13:03:40.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.032+0000 7f33a5169740 20 client.17151 trim_cache size 0 max 16384 2024-03-15T13:03:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.048+0000 7f33a5169740 1 Processor -- start 2024-03-15T13:03:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.048+0000 7f33a5169740 1 -- start start 2024-03-15T13:03:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.048+0000 7f33a5169740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384260 0x564ec6393f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.048+0000 7f33a5169740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384c10 0x564ec638f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f33a5169740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec638dd10 0x564ec637fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f33a5169740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec639ed60 con 0x564ec6384260 2024-03-15T13:03:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f33a5169740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec6383690 con 0x564ec6384c10 2024-03-15T13:03:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f33a5169740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec63915d0 con 0x564ec638dd10 2024-03-15T13:03:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f33a0b29640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384260 0x564ec6393f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f33a0b29640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384260 0x564ec6393f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51518/0 (socket says 172.21.15.2:51518) 2024-03-15T13:03:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f33a0b29640 1 -- 172.21.15.2:0/533738958 learned_addr learned my addr 172.21.15.2:0/533738958 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f339bfff640 1 --2- 172.21.15.2:0/533738958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384c10 0x564ec638f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f33a0b29640 1 --2- 172.21.15.2:0/533738958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec638dd10 0x564ec637fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f33a0b29640 1 -- 172.21.15.2:0/533738958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec638dd10 msgr2=0x564ec637fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f33a0b29640 1 --2- 172.21.15.2:0/533738958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec638dd10 0x564ec637fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f33a0b29640 1 -- 172.21.15.2:0/533738958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384c10 msgr2=0x564ec638f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f33a0b29640 1 --2- 172.21.15.2:0/533738958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384c10 0x564ec638f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f33a0b29640 1 -- 172.21.15.2:0/533738958 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ec638c0c0 con 0x564ec6384260 2024-03-15T13:03:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f33a0b29640 1 --2- 172.21.15.2:0/533738958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384260 0x564ec6393f10 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f339402c630 tx=0x7f339402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180ac13673fb1791 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f3381ffb640 1 -- 172.21.15.2:0/533738958 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394034ba0 con 0x564ec6384260 2024-03-15T13:03:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f3381ffb640 1 -- 172.21.15.2:0/533738958 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f339404b030 con 0x564ec6384260 2024-03-15T13:03:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.049+0000 7f3381ffb640 1 -- 172.21.15.2:0/533738958 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394032af0 con 0x564ec6384260 2024-03-15T13:03:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.050+0000 7f33a5169740 1 -- 172.21.15.2:0/533738958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384260 msgr2=0x564ec6393f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.050+0000 7f33a5169740 1 --2- 172.21.15.2:0/533738958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384260 0x564ec6393f10 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f339402c630 tx=0x7f339402c0f0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.050+0000 7f33a5169740 1 -- 172.21.15.2:0/533738958 shutdown_connections 2024-03-15T13:03:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.050+0000 7f33a5169740 1 --2- 172.21.15.2:0/533738958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec638dd10 0x564ec637fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.050+0000 7f33a5169740 1 --2- 172.21.15.2:0/533738958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec6384c10 0x564ec638f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.050+0000 7f33a5169740 1 --2- 172.21.15.2:0/533738958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384260 0x564ec6393f10 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.050+0000 7f33a5169740 1 -- 172.21.15.2:0/533738958 >> 172.21.15.2:0/533738958 conn(0x564ec637e840 msgr2=0x564ec63900b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.050+0000 7f33a5169740 1 -- 172.21.15.2:0/533738958 shutdown_connections 2024-03-15T13:03:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.050+0000 7f33a5169740 1 -- 172.21.15.2:0/533738958 wait complete. 2024-03-15T13:03:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.050+0000 7f33a5169740 1 -- start start 2024-03-15T13:03:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.050+0000 7f33a5169740 1 Processor -- start 2024-03-15T13:03:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f33a5169740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec6384260 0x564ec639caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f33a5169740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384c10 0x564ec639cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f33a5169740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec638dd10 0x564ec63fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f33a5169740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec63fd0d0 con 0x564ec6384c10 2024-03-15T13:03:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f33a5169740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec63fd210 con 0x564ec638dd10 2024-03-15T13:03:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f33a5169740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec63fd380 con 0x564ec6384260 2024-03-15T13:03:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f33a5169740 10 client.0 _ll_register_callbacks cb 0x7f3390005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f339bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384c10 0x564ec639cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f339bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384c10 0x564ec639cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51534/0 (socket says 172.21.15.2:51534) 2024-03-15T13:03:40.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f339bfff640 1 -- 172.21.15.2:0/3852584322 learned_addr learned my addr 172.21.15.2:0/3852584322 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:40.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f33a0b29640 1 --2- 172.21.15.2:0/3852584322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec6384260 0x564ec639caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f33a0b29640 1 --2- 172.21.15.2:0/3852584322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec638dd10 0x564ec63fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f339bfff640 1 -- 172.21.15.2:0/3852584322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec6384260 msgr2=0x564ec639caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f339bfff640 1 --2- 172.21.15.2:0/3852584322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec6384260 0x564ec639caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f339bfff640 1 -- 172.21.15.2:0/3852584322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec638dd10 msgr2=0x564ec63fcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f339bfff640 1 --2- 172.21.15.2:0/3852584322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec638dd10 0x564ec63fcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f339bfff640 1 -- 172.21.15.2:0/3852584322 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ec62ea320 con 0x564ec6384c10 2024-03-15T13:03:40.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f33a0b29640 1 --2- 172.21.15.2:0/3852584322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec638dd10 0x564ec63fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:03:40.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f339bfff640 1 --2- 172.21.15.2:0/3852584322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384c10 0x564ec639cfe0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x564ec6385050 tx=0x7f338c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95edeb6816785c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f337b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f337b7fe640 1 -- 172.21.15.2:0/3852584322 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c005580 con 0x564ec6384c10 2024-03-15T13:03:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f337b7fe640 1 -- 172.21.15.2:0/3852584322 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f338c009e70 con 0x564ec6384c10 2024-03-15T13:03:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.051+0000 7f33a5169740 1 -- 172.21.15.2:0/3852584322 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ec6311900 con 0x564ec6384c10 2024-03-15T13:03:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.052+0000 7f337b7fe640 1 -- 172.21.15.2:0/3852584322 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f338c0298c0 con 0x564ec6384c10 2024-03-15T13:03:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.052+0000 7f33a5169740 10 client.17157 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.052+0000 7f33a5169740 1 -- 172.21.15.2:0/3852584322 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564ec6286c00 con 0x564ec6384c10 2024-03-15T13:03:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.052+0000 7f337b7fe640 1 -- 172.21.15.2:0/3852584322 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x564ec6311900 con 0x564ec6384c10 2024-03-15T13:03:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.052+0000 7f337b7fe640 1 -- 172.21.15.2:0/3852584322 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7f338c003c40 con 0x564ec6384c10 2024-03-15T13:03:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.052+0000 7f33a5169740 10 client.17157 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.052+0000 7f33a5169740 10 client.17157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.052+0000 7f33a5169740 20 client.17157 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.052+0000 7f33a5169740 1 --2- 172.21.15.2:0/3852584322 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564ec63fec80 0x564ec64010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.052+0000 7f33a5169740 4 client.17157 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:03:40.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.052+0000 7f33a5169740 1 -- 172.21.15.2:0/3852584322 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564ec5feb860 con 0x564ec63fec80 2024-03-15T13:03:40.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.052+0000 7f33a0b29640 1 --2- 172.21.15.2:0/3852584322 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564ec63fec80 0x564ec64010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.053+0000 7f337b7fe640 10 client.17157 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:40.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.053+0000 7f33a0b29640 1 --2- 172.21.15.2:0/3852584322 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564ec63fec80 0x564ec64010a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d6cd49cf2761b server_cookie=e0108e42bd98deb1 in_seq=0 out_seq=0 2024-03-15T13:03:40.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.054+0000 7f337b7fe640 1 -- 172.21.15.2:0/3852584322 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564ec5feb860 con 0x564ec63fec80 2024-03-15T13:03:40.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.054+0000 7f337b7fe640 10 client.17157 handle_command_reply: tid=0 2024-03-15T13:03:40.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.054+0000 7f33a5169740 1 client.17157 shutdown 2024-03-15T13:03:40.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.054+0000 7f33a5169740 1 -- 172.21.15.2:0/3852584322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384c10 msgr2=0x564ec639cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.054+0000 7f33a5169740 1 --2- 172.21.15.2:0/3852584322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384c10 0x564ec639cfe0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x564ec6385050 tx=0x7f338c005d30 comp rx=0 tx=0).stop 2024-03-15T13:03:40.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.054+0000 7f33a5169740 1 -- 172.21.15.2:0/3852584322 shutdown_connections 2024-03-15T13:03:40.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.054+0000 7f33a5169740 1 --2- 172.21.15.2:0/3852584322 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564ec63fec80 0x564ec64010a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.054+0000 7f33a5169740 1 --2- 172.21.15.2:0/3852584322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564ec638dd10 0x564ec63fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.054+0000 7f33a5169740 1 --2- 172.21.15.2:0/3852584322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564ec6384c10 0x564ec639cfe0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.054+0000 7f337b7fe640 0 client.17157 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:40.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.054+0000 7f33a5169740 1 --2- 172.21.15.2:0/3852584322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564ec6384260 0x564ec639caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.054+0000 7f33a5169740 1 -- 172.21.15.2:0/3852584322 >> 172.21.15.2:0/3852584322 conn(0x564ec637e840 msgr2=0x564ec6391760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:40.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.054+0000 7f33a5169740 1 -- 172.21.15.2:0/3852584322 shutdown_connections 2024-03-15T13:03:40.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.054+0000 7f33a5169740 1 -- 172.21.15.2:0/3852584322 wait complete. 2024-03-15T13:03:40.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.054+0000 7f33a5169740 20 client.17157 trim_cache size 0 max 16384 2024-03-15T13:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "884ede83-1fc3-46dd-aa99-a911b3eaf8d8", 2024-03-15T13:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:03:40.047 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:03:40.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.055+0000 7f33a258c640 1 -- 172.21.15.2:0/3794772077 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f33840a1ba0 msgr2=0x7f33840a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.055+0000 7f33a258c640 1 --2- 172.21.15.2:0/3794772077 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f33840a1ba0 0x7f33840a4060 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f33900cb2d0 tx=0x7f3390085040 comp rx=0 tx=0).stop 2024-03-15T13:03:40.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.055+0000 7f33a258c640 1 -- 172.21.15.2:0/3794772077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f339c1056a0 msgr2=0x7f339c19dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.055+0000 7f33a258c640 1 --2- 172.21.15.2:0/3794772077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f339c1056a0 0x7f339c19dcb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f339000e9f0 tx=0x7f339000eeb0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.055+0000 7f33a258c640 1 -- 172.21.15.2:0/3794772077 shutdown_connections 2024-03-15T13:03:40.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.055+0000 7f33a258c640 1 --2- 172.21.15.2:0/3794772077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f339c197920 0x7f339c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.055+0000 7f33a258c640 1 --2- 172.21.15.2:0/3794772077 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f33840a1ba0 0x7f33840a4060 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.055+0000 7f33a258c640 1 --2- 172.21.15.2:0/3794772077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f339c1056a0 0x7f339c19dcb0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.055+0000 7f33a258c640 1 --2- 172.21.15.2:0/3794772077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f339c1005d0 0x7f339c19d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.055+0000 7f33a258c640 1 -- 172.21.15.2:0/3794772077 >> 172.21.15.2:0/3794772077 conn(0x7f339c0774f0 msgr2=0x7f339c068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:40.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.055+0000 7f33a258c640 1 -- 172.21.15.2:0/3794772077 shutdown_connections 2024-03-15T13:03:40.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.055+0000 7f33a258c640 1 -- 172.21.15.2:0/3794772077 wait complete. 2024-03-15T13:03:40.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:03:40.051 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:03:40.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.170+0000 7f64d2a84640 1 -- 172.21.15.2:0/4062404729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64cc100f90 msgr2=0x7f64cc104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.170+0000 7f64d2a84640 1 --2- 172.21.15.2:0/4062404729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64cc100f90 0x7f64cc104fd0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009930 tx=0x7f64b802bc30 comp rx=0 tx=0).stop 2024-03-15T13:03:40.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d2a84640 1 -- 172.21.15.2:0/4062404729 shutdown_connections 2024-03-15T13:03:40.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d2a84640 1 --2- 172.21.15.2:0/4062404729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64cc1056b0 0x7f64cc10d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d2a84640 1 --2- 172.21.15.2:0/4062404729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64cc100f90 0x7f64cc104fd0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d2a84640 1 --2- 172.21.15.2:0/4062404729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64cc1005e0 0x7f64cc1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d2a84640 1 -- 172.21.15.2:0/4062404729 >> 172.21.15.2:0/4062404729 conn(0x7f64cc0775a0 msgr2=0x7f64cc0779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:40.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d2a84640 1 -- 172.21.15.2:0/4062404729 shutdown_connections 2024-03-15T13:03:40.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d2a84640 1 -- 172.21.15.2:0/4062404729 wait complete. 2024-03-15T13:03:40.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d2a84640 1 Processor -- start 2024-03-15T13:03:40.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d2a84640 1 -- start start 2024-03-15T13:03:40.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d2a84640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64cc1005e0 0x7f64cc19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d2a84640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64cc1056b0 0x7f64cc19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d2a84640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64cc196860 0x7f64cc196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d2a84640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc197310 con 0x7f64cc196860 2024-03-15T13:03:40.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d2a84640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc197480 con 0x7f64cc1056b0 2024-03-15T13:03:40.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d2a84640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc1975f0 con 0x7f64cc1005e0 2024-03-15T13:03:40.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d1021640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64cc196860 0x7f64cc196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d1021640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64cc196860 0x7f64cc196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51556/0 (socket says 172.21.15.2:51556) 2024-03-15T13:03:40.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.171+0000 7f64d1021640 1 -- 172.21.15.2:0/2848111174 learned_addr learned my addr 172.21.15.2:0/2848111174 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:40.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.172+0000 7f64cbfff640 1 --2- 172.21.15.2:0/2848111174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64cc1056b0 0x7f64cc19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.172+0000 7f64d0820640 1 --2- 172.21.15.2:0/2848111174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64cc1005e0 0x7f64cc19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.172+0000 7f64d1021640 1 -- 172.21.15.2:0/2848111174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64cc1005e0 msgr2=0x7f64cc19c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.172+0000 7f64d1021640 1 --2- 172.21.15.2:0/2848111174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64cc1005e0 0x7f64cc19c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.172+0000 7f64d1021640 1 -- 172.21.15.2:0/2848111174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64cc1056b0 msgr2=0x7f64cc19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.172+0000 7f64d1021640 1 --2- 172.21.15.2:0/2848111174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64cc1056b0 0x7f64cc19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.172+0000 7f64d1021640 1 -- 172.21.15.2:0/2848111174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b8009560 con 0x7f64cc196860 2024-03-15T13:03:40.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.172+0000 7f64d1021640 1 --2- 172.21.15.2:0/2848111174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64cc196860 0x7f64cc196d10 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f64c400eb10 tx=0x7f64c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:40.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.172+0000 7f64c9ffb640 1 -- 172.21.15.2:0/2848111174 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4004b20 con 0x7f64cc196860 2024-03-15T13:03:40.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.172+0000 7f64c9ffb640 1 -- 172.21.15.2:0/2848111174 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f64c4004c80 con 0x7f64cc196860 2024-03-15T13:03:40.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.172+0000 7f64d2a84640 1 -- 172.21.15.2:0/2848111174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64cc1a3550 con 0x7f64cc196860 2024-03-15T13:03:40.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.172+0000 7f64c9ffb640 1 -- 172.21.15.2:0/2848111174 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4010690 con 0x7f64cc196860 2024-03-15T13:03:40.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.172+0000 7f64d2a84640 1 -- 172.21.15.2:0/2848111174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc1a39e0 con 0x7f64cc196860 2024-03-15T13:03:40.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.174+0000 7f64c9ffb640 1 -- 172.21.15.2:0/2848111174 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f64c4004df0 con 0x7f64cc196860 2024-03-15T13:03:40.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64d565d740 1 Processor -- start 2024-03-15T13:03:40.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64d565d740 1 -- start start 2024-03-15T13:03:40.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64c9ffb640 1 --2- 172.21.15.2:0/2848111174 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f64bc0a1ba0 0x7f64bc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64d565d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6ed2b0 0x558e6e6ed690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64c9ffb640 1 -- 172.21.15.2:0/2848111174 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f64c4014070 con 0x7f64cc196860 2024-03-15T13:03:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64d565d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6edbd0 0x558e6e6f6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64cbfff640 1 --2- 172.21.15.2:0/2848111174 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f64bc0a1ba0 0x7f64bc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64d565d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 0x558e6e6f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64d565d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6e65c280 con 0x558e6e6edbd0 2024-03-15T13:03:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64d565d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6e6597b0 con 0x558e6e6ed2b0 2024-03-15T13:03:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64d565d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6e654f90 con 0x558e6e6f67b0 2024-03-15T13:03:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64cbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6edbd0 0x558e6e6f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64cbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6edbd0 0x558e6e6f6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51566/0 (socket says 172.21.15.2:51566) 2024-03-15T13:03:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64cbfff640 1 -- 172.21.15.2:0/4060977646 learned_addr learned my addr 172.21.15.2:0/4060977646 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64d0820640 1 --2- 172.21.15.2:0/4060977646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6ed2b0 0x558e6e6ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64d1021640 1 --2- 172.21.15.2:0/4060977646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 0x558e6e6f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64cbfff640 1 -- 172.21.15.2:0/4060977646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 msgr2=0x558e6e6f8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64cbfff640 1 --2- 172.21.15.2:0/4060977646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 0x558e6e6f8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64cbfff640 1 -- 172.21.15.2:0/4060977646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6ed2b0 msgr2=0x558e6e6ed690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64cbfff640 1 --2- 172.21.15.2:0/4060977646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6ed2b0 0x558e6e6ed690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64cbfff640 1 -- 172.21.15.2:0/4060977646 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e6e613570 con 0x558e6e6edbd0 2024-03-15T13:03:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64d0820640 1 --2- 172.21.15.2:0/4060977646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6ed2b0 0x558e6e6ed690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.175+0000 7f64d1021640 1 --2- 172.21.15.2:0/4060977646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 0x558e6e6f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:03:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.176+0000 7f64cbfff640 1 --2- 172.21.15.2:0/2848111174 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f64bc0a1ba0 0x7f64bc0a4060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f64b8005ec0 tx=0x7f64b80046c0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.176+0000 7f64cbfff640 1 --2- 172.21.15.2:0/4060977646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6edbd0 0x558e6e6f6270 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f64cc06ac60 tx=0x7f64b8048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a68a54e4ed867f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.176+0000 7f64b6ffd640 1 -- 172.21.15.2:0/4060977646 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8048070 con 0x558e6e6edbd0 2024-03-15T13:03:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.176+0000 7f64b6ffd640 1 -- 172.21.15.2:0/4060977646 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f64b8004410 con 0x558e6e6edbd0 2024-03-15T13:03:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.176+0000 7f64b6ffd640 1 -- 172.21.15.2:0/4060977646 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b80035f0 con 0x558e6e6edbd0 2024-03-15T13:03:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.176+0000 7f64d565d740 1 -- 172.21.15.2:0/4060977646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6edbd0 msgr2=0x558e6e6f6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.176+0000 7f64d565d740 1 --2- 172.21.15.2:0/4060977646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6edbd0 0x558e6e6f6270 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f64cc06ac60 tx=0x7f64b8048000 comp rx=0 tx=0).stop 2024-03-15T13:03:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.176+0000 7f64d565d740 1 -- 172.21.15.2:0/4060977646 shutdown_connections 2024-03-15T13:03:40.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.176+0000 7f64d565d740 1 --2- 172.21.15.2:0/4060977646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 0x558e6e6f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.176+0000 7f64d565d740 1 --2- 172.21.15.2:0/4060977646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6edbd0 0x558e6e6f6270 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.176+0000 7f64d565d740 1 --2- 172.21.15.2:0/4060977646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6ed2b0 0x558e6e6ed690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.176+0000 7f64d565d740 1 -- 172.21.15.2:0/4060977646 >> 172.21.15.2:0/4060977646 conn(0x558e6e6e7990 msgr2=0x558e6e6e7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64d565d740 1 -- 172.21.15.2:0/4060977646 shutdown_connections 2024-03-15T13:03:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64d565d740 1 -- 172.21.15.2:0/4060977646 wait complete. 2024-03-15T13:03:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64d565d740 1 -- start start 2024-03-15T13:03:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64d565d740 1 Processor -- start 2024-03-15T13:03:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64d565d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6ed2b0 0x558e6e6f9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64d565d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6edbd0 0x558e6e6f9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64d565d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 0x558e6e6fa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64d565d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6e328430 con 0x558e6e6ed2b0 2024-03-15T13:03:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64d565d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6e2e4e00 con 0x558e6e6edbd0 2024-03-15T13:03:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64d565d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6e5d9fd0 con 0x558e6e6f67b0 2024-03-15T13:03:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64d565d740 10 client.0 _ll_register_callbacks cb 0x558e6e62ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:40.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64d0820640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6ed2b0 0x558e6e6f9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64d0820640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6ed2b0 0x558e6e6f9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51578/0 (socket says 172.21.15.2:51578) 2024-03-15T13:03:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64d0820640 1 -- 172.21.15.2:0/737489555 learned_addr learned my addr 172.21.15.2:0/737489555 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64cbfff640 1 --2- 172.21.15.2:0/737489555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6edbd0 0x558e6e6f9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.177+0000 7f64d1021640 1 --2- 172.21.15.2:0/737489555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 0x558e6e6fa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.178+0000 7f64d0820640 1 -- 172.21.15.2:0/737489555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 msgr2=0x558e6e6fa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.178+0000 7f64d0820640 1 --2- 172.21.15.2:0/737489555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 0x558e6e6fa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.178+0000 7f64d0820640 1 -- 172.21.15.2:0/737489555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6edbd0 msgr2=0x558e6e6f9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.178+0000 7f64d0820640 1 --2- 172.21.15.2:0/737489555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6edbd0 0x558e6e6f9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.178+0000 7f64d0820640 1 -- 172.21.15.2:0/737489555 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc10da90 con 0x558e6e6ed2b0 2024-03-15T13:03:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.178+0000 7f64d0820640 1 --2- 172.21.15.2:0/737489555 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6ed2b0 0x558e6e6f9650 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f64c0012810 tx=0x7f64c0012c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30412ffb4e8f0c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.178+0000 7f64abfff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.178+0000 7f64abfff640 1 -- 172.21.15.2:0/737489555 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c0017070 con 0x558e6e6ed2b0 2024-03-15T13:03:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.178+0000 7f64abfff640 1 -- 172.21.15.2:0/737489555 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f64c0004020 con 0x558e6e6ed2b0 2024-03-15T13:03:40.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.178+0000 7f64d565d740 1 -- 172.21.15.2:0/737489555 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e6e3547e0 con 0x558e6e6ed2b0 2024-03-15T13:03:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.178+0000 7f64abfff640 1 -- 172.21.15.2:0/737489555 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64c00122e0 con 0x558e6e6ed2b0 2024-03-15T13:03:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.178+0000 7f64d565d740 10 client.17169 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.178+0000 7f64d565d740 1 -- 172.21.15.2:0/737489555 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558e6e653610 con 0x558e6e6ed2b0 2024-03-15T13:03:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.178+0000 7f64abfff640 1 -- 172.21.15.2:0/737489555 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x558e6e3547e0 con 0x558e6e6ed2b0 2024-03-15T13:03:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.178+0000 7f64abfff640 1 -- 172.21.15.2:0/737489555 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7f64c00038b0 con 0x558e6e6ed2b0 2024-03-15T13:03:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.179+0000 7f64d565d740 10 client.17169 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.179+0000 7f64d565d740 10 client.17169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.179+0000 7f64d565d740 20 client.17169 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.179+0000 7f64d565d740 1 --2- 172.21.15.2:0/737489555 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558e6e708d10 0x558e6e70b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.179+0000 7f64d565d740 4 client.17169 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:03:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.179+0000 7f64d565d740 1 -- 172.21.15.2:0/737489555 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e6e391720 con 0x558e6e708d10 2024-03-15T13:03:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.179+0000 7f64cbfff640 1 --2- 172.21.15.2:0/737489555 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558e6e708d10 0x558e6e70b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.179+0000 7f64abfff640 10 client.17169 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.179+0000 7f64cbfff640 1 --2- 172.21.15.2:0/737489555 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558e6e708d10 0x558e6e70b130 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43b74ad3de4727b1 server_cookie=71458bb55499ff66 in_seq=0 out_seq=0 2024-03-15T13:03:40.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.180+0000 7f64abfff640 1 -- 172.21.15.2:0/737489555 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558e6e391720 con 0x558e6e708d10 2024-03-15T13:03:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.180+0000 7f64abfff640 10 client.17169 handle_command_reply: tid=0 2024-03-15T13:03:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.180+0000 7f64d565d740 1 client.17169 shutdown 2024-03-15T13:03:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.180+0000 7f64d565d740 1 -- 172.21.15.2:0/737489555 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6ed2b0 msgr2=0x558e6e6f9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.180+0000 7f64d565d740 1 --2- 172.21.15.2:0/737489555 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6ed2b0 0x558e6e6f9650 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f64c0012810 tx=0x7f64c0012c10 comp rx=0 tx=0).stop 2024-03-15T13:03:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.180+0000 7f64d565d740 1 -- 172.21.15.2:0/737489555 shutdown_connections 2024-03-15T13:03:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.180+0000 7f64d565d740 1 --2- 172.21.15.2:0/737489555 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558e6e708d10 0x558e6e70b130 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.180+0000 7f64d565d740 1 --2- 172.21.15.2:0/737489555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 0x558e6e6fa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.180+0000 7f64d565d740 1 --2- 172.21.15.2:0/737489555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6edbd0 0x558e6e6f9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.180+0000 7f64abfff640 0 client.17169 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.180+0000 7f64d565d740 1 --2- 172.21.15.2:0/737489555 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6ed2b0 0x558e6e6f9650 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.180+0000 7f64d565d740 1 -- 172.21.15.2:0/737489555 >> 172.21.15.2:0/737489555 conn(0x558e6e6e7990 msgr2=0x558e6e6e9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.180+0000 7f64d565d740 1 -- 172.21.15.2:0/737489555 shutdown_connections 2024-03-15T13:03:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.180+0000 7f64d565d740 1 -- 172.21.15.2:0/737489555 wait complete. 2024-03-15T13:03:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.180+0000 7f64d565d740 20 client.17169 trim_cache size 0 max 16384 2024-03-15T13:03:40.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.196+0000 7f64d565d740 1 Processor -- start 2024-03-15T13:03:40.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.196+0000 7f64d565d740 1 -- start start 2024-03-15T13:03:40.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d565d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6ed2b0 0x558e6e703ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d565d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6edbd0 0x558e6e704020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d565d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6f67b0 0x558e6e6f08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d565d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6e70b790 con 0x558e6e6f67b0 2024-03-15T13:03:40.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d565d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6e6f1bc0 con 0x558e6e6ed2b0 2024-03-15T13:03:40.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d565d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6e6ed9c0 con 0x558e6e6edbd0 2024-03-15T13:03:40.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d1021640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6f67b0 0x558e6e6f08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d0820640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6ed2b0 0x558e6e703ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d1021640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6f67b0 0x558e6e6f08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51590/0 (socket says 172.21.15.2:51590) 2024-03-15T13:03:40.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d0820640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6ed2b0 0x558e6e703ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:54372/0 (socket says 172.21.15.2:54372) 2024-03-15T13:03:40.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d1021640 1 -- 172.21.15.2:0/1142481541 learned_addr learned my addr 172.21.15.2:0/1142481541 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:40.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64cbfff640 1 --2- 172.21.15.2:0/1142481541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6edbd0 0x558e6e704020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d1021640 1 -- 172.21.15.2:0/1142481541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6edbd0 msgr2=0x558e6e704020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d1021640 1 --2- 172.21.15.2:0/1142481541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6edbd0 0x558e6e704020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d1021640 1 -- 172.21.15.2:0/1142481541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6ed2b0 msgr2=0x558e6e703ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d1021640 1 --2- 172.21.15.2:0/1142481541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6ed2b0 0x558e6e703ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d1021640 1 -- 172.21.15.2:0/1142481541 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e6e5efac0 con 0x558e6e6f67b0 2024-03-15T13:03:40.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64d1021640 1 --2- 172.21.15.2:0/1142481541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6f67b0 0x558e6e6f08d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f64c409a7a0 tx=0x7f64c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc22c0b1aaaa9e38 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:40.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64b67fc640 1 -- 172.21.15.2:0/1142481541 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c409aaa0 con 0x558e6e6f67b0 2024-03-15T13:03:40.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64b67fc640 1 -- 172.21.15.2:0/1142481541 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f64c409ebf0 con 0x558e6e6f67b0 2024-03-15T13:03:40.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.197+0000 7f64b67fc640 1 -- 172.21.15.2:0/1142481541 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4099ef0 con 0x558e6e6f67b0 2024-03-15T13:03:40.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.198+0000 7f64d565d740 1 -- 172.21.15.2:0/1142481541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6f67b0 msgr2=0x558e6e6f08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.198+0000 7f64d565d740 1 --2- 172.21.15.2:0/1142481541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6f67b0 0x558e6e6f08d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f64c409a7a0 tx=0x7f64c4004a40 comp rx=0 tx=0).stop 2024-03-15T13:03:40.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.198+0000 7f64d565d740 1 -- 172.21.15.2:0/1142481541 shutdown_connections 2024-03-15T13:03:40.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.198+0000 7f64d565d740 1 --2- 172.21.15.2:0/1142481541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6f67b0 0x558e6e6f08d0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.198+0000 7f64d565d740 1 --2- 172.21.15.2:0/1142481541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6edbd0 0x558e6e704020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.198+0000 7f64d565d740 1 --2- 172.21.15.2:0/1142481541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6ed2b0 0x558e6e703ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.198+0000 7f64d565d740 1 -- 172.21.15.2:0/1142481541 >> 172.21.15.2:0/1142481541 conn(0x558e6e6e7990 msgr2=0x558e6e6ee950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:40.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.198+0000 7f64d565d740 1 -- 172.21.15.2:0/1142481541 shutdown_connections 2024-03-15T13:03:40.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.198+0000 7f64d565d740 1 -- 172.21.15.2:0/1142481541 wait complete. 2024-03-15T13:03:40.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.198+0000 7f64d565d740 1 -- start start 2024-03-15T13:03:40.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.198+0000 7f64d565d740 1 Processor -- start 2024-03-15T13:03:40.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d565d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6ed2b0 0x558e6e6fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d565d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6edbd0 0x558e6e6fc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d565d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 0x558e6e6fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d565d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6e6fd3a0 con 0x558e6e6ed2b0 2024-03-15T13:03:40.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d565d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6e6fd510 con 0x558e6e6edbd0 2024-03-15T13:03:40.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d565d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e6e7645e0 con 0x558e6e6f67b0 2024-03-15T13:03:40.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d565d740 10 client.0 _ll_register_callbacks cb 0x7f64c0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:03:40.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d0820640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6ed2b0 0x558e6e6fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d0820640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6ed2b0 0x558e6e6fe3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51596/0 (socket says 172.21.15.2:51596) 2024-03-15T13:03:40.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d0820640 1 -- 172.21.15.2:0/2911023893 learned_addr learned my addr 172.21.15.2:0/2911023893 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:03:40.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d1021640 1 --2- 172.21.15.2:0/2911023893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 0x558e6e6fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64cbfff640 1 --2- 172.21.15.2:0/2911023893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6edbd0 0x558e6e6fc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d0820640 1 -- 172.21.15.2:0/2911023893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 msgr2=0x558e6e6fcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d0820640 1 --2- 172.21.15.2:0/2911023893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 0x558e6e6fcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d0820640 1 -- 172.21.15.2:0/2911023893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6edbd0 msgr2=0x558e6e6fc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:03:40.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d0820640 1 --2- 172.21.15.2:0/2911023893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6edbd0 0x558e6e6fc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d0820640 1 -- 172.21.15.2:0/2911023893 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc1a3550 con 0x558e6e6ed2b0 2024-03-15T13:03:40.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d0820640 1 --2- 172.21.15.2:0/2911023893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6ed2b0 0x558e6e6fe3c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f64c000f800 tx=0x7f64c000f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119d911e38cea064 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:03:40.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64abfff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:03:40.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64abfff640 1 -- 172.21.15.2:0/2911023893 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c0017040 con 0x558e6e6ed2b0 2024-03-15T13:03:40.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64abfff640 1 -- 172.21.15.2:0/2911023893 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f64c00035f0 con 0x558e6e6ed2b0 2024-03-15T13:03:40.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.199+0000 7f64d565d740 1 -- 172.21.15.2:0/2911023893 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64b8004410 con 0x558e6e6ed2b0 2024-03-15T13:03:40.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.200+0000 7f64abfff640 1 -- 172.21.15.2:0/2911023893 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64c00037b0 con 0x558e6e6ed2b0 2024-03-15T13:03:40.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.200+0000 7f64d565d740 10 client.17175 fetch_fsmap learned FSMap version 40 2024-03-15T13:03:40.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.200+0000 7f64d565d740 1 -- 172.21.15.2:0/2911023893 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558e6e6156a0 con 0x558e6e6ed2b0 2024-03-15T13:03:40.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.200+0000 7f64abfff640 1 -- 172.21.15.2:0/2911023893 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f64b8004410 con 0x558e6e6ed2b0 2024-03-15T13:03:40.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.200+0000 7f64abfff640 1 -- 172.21.15.2:0/2911023893 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7982+0+0 (secure 0 0 0) 0x7f64c0004280 con 0x558e6e6ed2b0 2024-03-15T13:03:40.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.200+0000 7f64d565d740 10 client.17175 fetch_fsmap finished waiting for FSMap version 40 2024-03-15T13:03:40.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.200+0000 7f64d565d740 10 client.17175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:03:40.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.200+0000 7f64d565d740 20 client.17175 populate_metadata read hostname 'smithi002' 2024-03-15T13:03:40.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.200+0000 7f64d565d740 1 --2- 172.21.15.2:0/2911023893 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558e6e765dc0 0x558e6e7681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:03:40.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.200+0000 7f64d565d740 4 client.17175 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:03:40.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.200+0000 7f64d565d740 1 -- 172.21.15.2:0/2911023893 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558e6e3e0890 con 0x558e6e765dc0 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.200+0000 7f64cbfff640 1 --2- 172.21.15.2:0/2911023893 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558e6e765dc0 0x558e6e7681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64cbfff640 1 --2- 172.21.15.2:0/2911023893 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558e6e765dc0 0x558e6e7681e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef14519d81a6f766 server_cookie=16c26cad91d08ba6 in_seq=0 out_seq=0 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64abfff640 10 client.17175 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64abfff640 1 -- 172.21.15.2:0/2911023893 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558e6e3e0890 con 0x558e6e765dc0 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64abfff640 10 client.17175 handle_command_reply: tid=0 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64d565d740 1 client.17175 shutdown 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64d565d740 1 -- 172.21.15.2:0/2911023893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6ed2b0 msgr2=0x558e6e6fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64d565d740 1 --2- 172.21.15.2:0/2911023893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6ed2b0 0x558e6e6fe3c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f64c000f800 tx=0x7f64c000f830 comp rx=0 tx=0).stop 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64d565d740 1 -- 172.21.15.2:0/2911023893 shutdown_connections 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64d565d740 1 --2- 172.21.15.2:0/2911023893 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x558e6e765dc0 0x558e6e7681e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64d565d740 1 --2- 172.21.15.2:0/2911023893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e6e6f67b0 0x558e6e6fcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64d565d740 1 --2- 172.21.15.2:0/2911023893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e6e6edbd0 0x558e6e6fc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64d565d740 1 --2- 172.21.15.2:0/2911023893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558e6e6ed2b0 0x558e6e6fe3c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64abfff640 0 client.17175 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64d565d740 1 -- 172.21.15.2:0/2911023893 >> 172.21.15.2:0/2911023893 conn(0x558e6e6e7990 msgr2=0x558e6e6e61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:40.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64d565d740 1 -- 172.21.15.2:0/2911023893 shutdown_connections 2024-03-15T13:03:40.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64d565d740 1 -- 172.21.15.2:0/2911023893 wait complete. 2024-03-15T13:03:40.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.201+0000 7f64d565d740 20 client.17175 trim_cache size 0 max 16384 2024-03-15T13:03:40.193 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:03:40.193 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T13:03:40.193 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:03:40.193 INFO:teuthology.orchestra.run.smithi002.stdout: "884ede83-1fc3-46dd-aa99-a911b3eaf8d8": { 2024-03-15T13:03:40.193 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:03:40.193 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "884ede83-1fc3-46dd-aa99-a911b3eaf8d8", 2024-03-15T13:03:40.193 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:03:40.193 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:03:40.193 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:03:40.194 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:03:40.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.202+0000 7f64d2a84640 1 -- 172.21.15.2:0/2848111174 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f64bc0a1ba0 msgr2=0x7f64bc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.202+0000 7f64d2a84640 1 --2- 172.21.15.2:0/2848111174 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f64bc0a1ba0 0x7f64bc0a4060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f64b8005ec0 tx=0x7f64b80046c0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.202+0000 7f64d2a84640 1 -- 172.21.15.2:0/2848111174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64cc196860 msgr2=0x7f64cc196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:03:40.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.202+0000 7f64d2a84640 1 --2- 172.21.15.2:0/2848111174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64cc196860 0x7f64cc196d10 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f64c400eb10 tx=0x7f64c400c2d0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.202+0000 7f64d2a84640 1 -- 172.21.15.2:0/2848111174 shutdown_connections 2024-03-15T13:03:40.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.202+0000 7f64d2a84640 1 --2- 172.21.15.2:0/2848111174 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f64bc0a1ba0 0x7f64bc0a4060 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.202+0000 7f64d2a84640 1 --2- 172.21.15.2:0/2848111174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64cc196860 0x7f64cc196d10 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.202+0000 7f64d2a84640 1 --2- 172.21.15.2:0/2848111174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64cc1056b0 0x7f64cc19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.202+0000 7f64d2a84640 1 --2- 172.21.15.2:0/2848111174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64cc1005e0 0x7f64cc19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:03:40.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.202+0000 7f64d2a84640 1 -- 172.21.15.2:0/2848111174 >> 172.21.15.2:0/2848111174 conn(0x7f64cc0775a0 msgr2=0x7f64cc0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:03:40.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.202+0000 7f64d2a84640 1 -- 172.21.15.2:0/2848111174 shutdown_connections 2024-03-15T13:03:40.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:03:40.202+0000 7f64d2a84640 1 -- 172.21.15.2:0/2848111174 wait complete. 2024-03-15T13:03:40.198 INFO:tasks.cephfs.filesystem:scrub status for tag:884ede83-1fc3-46dd-aa99-a911b3eaf8d8 - {'path': '/', 'tag': '884ede83-1fc3-46dd-aa99-a911b3eaf8d8', 'options': 'recursive,force'} 2024-03-15T13:03:40.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:40 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:03:40.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:40 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:03:40.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:40 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:03:40.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:40 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:03:40.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:40 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:03:40.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:40 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:03:40.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:40 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:03:40.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:40 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:03:40.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:40 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:03:40.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:40 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:03:40.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:40 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:03:40.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:40 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:03:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:41 smithi002 ceph-mon[32060]: pgmap v146: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:03:41.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:41 smithi184 ceph-mon[35993]: pgmap v146: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:03:41.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:41 smithi149 ceph-mon[35927]: pgmap v146: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:03:43.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:43 smithi002 ceph-mon[32060]: pgmap v147: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:43.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:43 smithi184 ceph-mon[35993]: pgmap v147: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:43.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:43 smithi149 ceph-mon[35927]: pgmap v147: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:45.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:45 smithi002 ceph-mon[32060]: pgmap v148: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:45.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:45 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:03:45.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:45 smithi184 ceph-mon[35993]: pgmap v148: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:45.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:45 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:03:45.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:45 smithi149 ceph-mon[35927]: pgmap v148: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:45.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:45 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:03:47.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:47 smithi002 ceph-mon[32060]: pgmap v149: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:03:47.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:47 smithi184 ceph-mon[35993]: pgmap v149: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:03:47.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:47 smithi149 ceph-mon[35927]: pgmap v149: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:03:49.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:49 smithi002 ceph-mon[32060]: pgmap v150: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:03:49.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:49 smithi184 ceph-mon[35993]: pgmap v150: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:03:49.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:49 smithi149 ceph-mon[35927]: pgmap v150: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:03:51.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:51 smithi002 ceph-mon[32060]: pgmap v151: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:03:51.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:51 smithi184 ceph-mon[35993]: pgmap v151: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:03:51.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:51 smithi149 ceph-mon[35927]: pgmap v151: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:03:53.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:53 smithi002 ceph-mon[32060]: pgmap v152: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:03:53.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:53 smithi184 ceph-mon[35993]: pgmap v152: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:03:53.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:53 smithi149 ceph-mon[35927]: pgmap v152: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:03:55.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:55 smithi002 ceph-mon[32060]: pgmap v153: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:03:55.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:55 smithi184 ceph-mon[35993]: pgmap v153: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:03:55.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:55 smithi149 ceph-mon[35927]: pgmap v153: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:03:57.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:57 smithi002 ceph-mon[32060]: pgmap v154: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:03:57.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:57 smithi184 ceph-mon[35993]: pgmap v154: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:03:57.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:57 smithi149 ceph-mon[35927]: pgmap v154: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:03:59.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:03:59 smithi002 ceph-mon[32060]: pgmap v155: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:59.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:03:59 smithi184 ceph-mon[35993]: pgmap v155: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:03:59.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:03:59 smithi149 ceph-mon[35927]: pgmap v155: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:01.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:01 smithi002 ceph-mon[32060]: pgmap v156: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:04:01.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:01 smithi184 ceph-mon[35993]: pgmap v156: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:04:01.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:01 smithi149 ceph-mon[35927]: pgmap v156: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:04:03.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:03 smithi002 ceph-mon[32060]: pgmap v157: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:03.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:03 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:04:03.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:03 smithi184 ceph-mon[35993]: pgmap v157: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:03.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:03 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:04:03.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:03 smithi149 ceph-mon[35927]: pgmap v157: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:03.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:03 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:04:04.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:04 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:04:04.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:04 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:04:04.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:04 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:04:04.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:04 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:04:04.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:04 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:04:04.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:04 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:04:04.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:04 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:04:04.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:04 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:04:04.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:04 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:04:05.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:05 smithi002 ceph-mon[32060]: pgmap v158: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:05.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:05 smithi184 ceph-mon[35993]: pgmap v158: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:05.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:05 smithi149 ceph-mon[35927]: pgmap v158: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:07.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:07 smithi184 ceph-mon[35993]: pgmap v159: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:04:07.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:07 smithi149 ceph-mon[35927]: pgmap v159: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:04:08.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:07 smithi002 ceph-mon[32060]: pgmap v159: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:04:09.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:09 smithi184 ceph-mon[35993]: pgmap v160: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:04:09.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:09 smithi149 ceph-mon[35927]: pgmap v160: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:04:10.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:09 smithi002 ceph-mon[32060]: pgmap v160: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:04:10.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:04:10.199 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:04:10.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.331+0000 7fb9133c9640 1 -- 172.21.15.2:0/3808311393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb90c100e20 msgr2=0x7fb90c1052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.331+0000 7fb9133c9640 1 --2- 172.21.15.2:0/3808311393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb90c100e20 0x7fb90c1052e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009900 tx=0x7fb8fc02bcd0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.331+0000 7fb9133c9640 1 -- 172.21.15.2:0/3808311393 shutdown_connections 2024-03-15T13:04:10.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.331+0000 7fb9133c9640 1 --2- 172.21.15.2:0/3808311393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb90c105820 0x7fb90c105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.331+0000 7fb9133c9640 1 --2- 172.21.15.2:0/3808311393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb90c100e20 0x7fb90c1052e0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.331+0000 7fb9133c9640 1 --2- 172.21.15.2:0/3808311393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb90c100500 0x7fb90c1008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.331+0000 7fb9133c9640 1 -- 172.21.15.2:0/3808311393 >> 172.21.15.2:0/3808311393 conn(0x7fb90c0775a0 msgr2=0x7fb90c0779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.331+0000 7fb9133c9640 1 -- 172.21.15.2:0/3808311393 shutdown_connections 2024-03-15T13:04:10.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.331+0000 7fb9133c9640 1 -- 172.21.15.2:0/3808311393 wait complete. 2024-03-15T13:04:10.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.331+0000 7fb9133c9640 1 Processor -- start 2024-03-15T13:04:10.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.331+0000 7fb9133c9640 1 -- start start 2024-03-15T13:04:10.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb9133c9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb90c100500 0x7fb90c19e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb9133c9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb90c100e20 0x7fb90c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb9133c9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb90c105820 0x7fb90c1a29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb9133c9640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c1a2fa0 con 0x7fb90c100e20 2024-03-15T13:04:10.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb9133c9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c1a3110 con 0x7fb90c105820 2024-03-15T13:04:10.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb9133c9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c1a32d0 con 0x7fb90c100500 2024-03-15T13:04:10.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb910964640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb90c100e20 0x7fb90c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb910964640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb90c100e20 0x7fb90c19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41604/0 (socket says 172.21.15.2:41604) 2024-03-15T13:04:10.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb910964640 1 -- 172.21.15.2:0/2275295318 learned_addr learned my addr 172.21.15.2:0/2275295318 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb911165640 1 --2- 172.21.15.2:0/2275295318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb90c100500 0x7fb90c19e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb911966640 1 --2- 172.21.15.2:0/2275295318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb90c105820 0x7fb90c1a29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb910964640 1 -- 172.21.15.2:0/2275295318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb90c100500 msgr2=0x7fb90c19e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb910964640 1 --2- 172.21.15.2:0/2275295318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb90c100500 0x7fb90c19e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb910964640 1 -- 172.21.15.2:0/2275295318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb90c105820 msgr2=0x7fb90c1a29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb910964640 1 --2- 172.21.15.2:0/2275295318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb90c105820 0x7fb90c1a29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb910964640 1 -- 172.21.15.2:0/2275295318 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb904009840 con 0x7fb90c100e20 2024-03-15T13:04:10.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb910964640 1 --2- 172.21.15.2:0/2275295318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb90c100e20 0x7fb90c19e620 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009230 tx=0x7fb8fc02cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb9027fc640 1 -- 172.21.15.2:0/2275295318 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc004250 con 0x7fb90c100e20 2024-03-15T13:04:10.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb9027fc640 1 -- 172.21.15.2:0/2275295318 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb8fc0043b0 con 0x7fb90c100e20 2024-03-15T13:04:10.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb9133c9640 1 -- 172.21.15.2:0/2275295318 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8fc009560 con 0x7fb90c100e20 2024-03-15T13:04:10.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.332+0000 7fb9027fc640 1 -- 172.21.15.2:0/2275295318 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc003c20 con 0x7fb90c100e20 2024-03-15T13:04:10.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.333+0000 7fb9133c9640 1 -- 172.21.15.2:0/2275295318 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90c1aac50 con 0x7fb90c100e20 2024-03-15T13:04:10.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.334+0000 7fb9027fc640 1 -- 172.21.15.2:0/2275295318 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fb8fc005020 con 0x7fb90c100e20 2024-03-15T13:04:10.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.335+0000 7fb915f9c740 1 Processor -- start 2024-03-15T13:04:10.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.335+0000 7fb915f9c740 1 -- start start 2024-03-15T13:04:10.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.335+0000 7fb9027fc640 1 --2- 172.21.15.2:0/2275295318 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb8f80a1c20 0x7fb8f80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.335+0000 7fb915f9c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 0x560640548690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.335+0000 7fb915f9c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560640548bd0 0x560640551270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.335+0000 7fb915f9c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606405517b0 0x560640553ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.335+0000 7fb915f9c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606404b7280 con 0x5606405517b0 2024-03-15T13:04:10.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.335+0000 7fb915f9c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606404b47b0 con 0x5606405482b0 2024-03-15T13:04:10.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.335+0000 7fb915f9c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606404aff90 con 0x560640548bd0 2024-03-15T13:04:10.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.335+0000 7fb911966640 1 --2- 172.21.15.2:0/2275295318 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb8f80a1c20 0x7fb8f80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.335+0000 7fb9027fc640 1 -- 172.21.15.2:0/2275295318 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fb8fc0f0040 con 0x7fb90c100e20 2024-03-15T13:04:10.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb910964640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560640548bd0 0x560640551270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb911165640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 0x560640548690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb910964640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560640548bd0 0x560640551270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:39690/0 (socket says 172.21.15.2:39690) 2024-03-15T13:04:10.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb911165640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 0x560640548690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60854/0 (socket says 172.21.15.2:60854) 2024-03-15T13:04:10.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb910964640 1 -- 172.21.15.2:0/274163598 learned_addr learned my addr 172.21.15.2:0/274163598 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb911966640 1 --2- 172.21.15.2:0/274163598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606405517b0 0x560640553ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb910964640 1 -- 172.21.15.2:0/274163598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 msgr2=0x560640548690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb910964640 1 --2- 172.21.15.2:0/274163598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 0x560640548690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb910964640 1 -- 172.21.15.2:0/274163598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606405517b0 msgr2=0x560640553ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb910964640 1 --2- 172.21.15.2:0/274163598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606405517b0 0x560640553ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb910964640 1 -- 172.21.15.2:0/274163598 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56064046e570 con 0x560640548bd0 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb911966640 1 --2- 172.21.15.2:0/2275295318 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb8f80a1c20 0x7fb8f80a40e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb904005fd0 tx=0x7fb904013c90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb910964640 1 --2- 172.21.15.2:0/274163598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560640548bd0 0x560640551270 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0bc9c0 tx=0x7fb8fc0b8d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26af15c6c6c189db server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb8f77fe640 1 -- 172.21.15.2:0/274163598 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc04a710 con 0x560640548bd0 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb8f77fe640 1 -- 172.21.15.2:0/274163598 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb8fc02ce90 con 0x560640548bd0 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.336+0000 7fb8f77fe640 1 -- 172.21.15.2:0/274163598 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc04a300 con 0x560640548bd0 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.337+0000 7fb915f9c740 1 -- 172.21.15.2:0/274163598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560640548bd0 msgr2=0x560640551270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.337+0000 7fb915f9c740 1 --2- 172.21.15.2:0/274163598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560640548bd0 0x560640551270 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0bc9c0 tx=0x7fb8fc0b8d90 comp rx=0 tx=0).stop 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.337+0000 7fb915f9c740 1 -- 172.21.15.2:0/274163598 shutdown_connections 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.337+0000 7fb915f9c740 1 --2- 172.21.15.2:0/274163598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5606405517b0 0x560640553ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.337+0000 7fb915f9c740 1 --2- 172.21.15.2:0/274163598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560640548bd0 0x560640551270 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.337+0000 7fb915f9c740 1 --2- 172.21.15.2:0/274163598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 0x560640548690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.337+0000 7fb915f9c740 1 -- 172.21.15.2:0/274163598 >> 172.21.15.2:0/274163598 conn(0x560640542990 msgr2=0x560640542d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.337+0000 7fb915f9c740 1 -- 172.21.15.2:0/274163598 shutdown_connections 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.337+0000 7fb915f9c740 1 -- 172.21.15.2:0/274163598 wait complete. 2024-03-15T13:04:10.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.337+0000 7fb915f9c740 1 -- start start 2024-03-15T13:04:10.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.337+0000 7fb915f9c740 1 Processor -- start 2024-03-15T13:04:10.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb915f9c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 0x56064054e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb915f9c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x56064054eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb915f9c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405517b0 0x56064054f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb915f9c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560640183430 con 0x560640548bd0 2024-03-15T13:04:10.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb915f9c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064013fe00 con 0x5606405482b0 2024-03-15T13:04:10.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb915f9c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560640434fd0 con 0x5606405517b0 2024-03-15T13:04:10.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb915f9c740 10 client.0 _ll_register_callbacks cb 0x56064048afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:10.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb910964640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x56064054eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb910964640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x56064054eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41622/0 (socket says 172.21.15.2:41622) 2024-03-15T13:04:10.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb910964640 1 -- 172.21.15.2:0/525889431 learned_addr learned my addr 172.21.15.2:0/525889431 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb911165640 1 --2- 172.21.15.2:0/525889431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 0x56064054e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb911966640 1 --2- 172.21.15.2:0/525889431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405517b0 0x56064054f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb910964640 1 -- 172.21.15.2:0/525889431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405517b0 msgr2=0x56064054f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb910964640 1 --2- 172.21.15.2:0/525889431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405517b0 0x56064054f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb910964640 1 -- 172.21.15.2:0/525889431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 msgr2=0x56064054e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb910964640 1 --2- 172.21.15.2:0/525889431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 0x56064054e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb910964640 1 -- 172.21.15.2:0/525889431 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90c1aac50 con 0x560640548bd0 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb911966640 1 --2- 172.21.15.2:0/525889431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405517b0 0x56064054f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.338+0000 7fb911165640 1 --2- 172.21.15.2:0/525889431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 0x56064054e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb910964640 1 --2- 172.21.15.2:0/525889431 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x56064054eb70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0ed600 tx=0x7fb8fc0bd190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7a45f20123c0ed server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb8f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb8f4ff9640 1 -- 172.21.15.2:0/525889431 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc04b120 con 0x560640548bd0 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb8f4ff9640 1 -- 172.21.15.2:0/525889431 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fb8fc04a6d0 con 0x560640548bd0 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb915f9c740 1 -- 172.21.15.2:0/525889431 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606401af7e0 con 0x560640548bd0 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb8f4ff9640 1 -- 172.21.15.2:0/525889431 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8fc0bca30 con 0x560640548bd0 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb915f9c740 10 client.17187 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb915f9c740 1 -- 172.21.15.2:0/525889431 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5606404ae610 con 0x560640548bd0 2024-03-15T13:04:10.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb8f4ff9640 1 -- 172.21.15.2:0/525889431 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5606401af7e0 con 0x560640548bd0 2024-03-15T13:04:10.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb8f4ff9640 1 -- 172.21.15.2:0/525889431 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fb8fc03c070 con 0x560640548bd0 2024-03-15T13:04:10.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb915f9c740 10 client.17187 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:10.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb915f9c740 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:10.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb915f9c740 20 client.17187 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:10.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb915f9c740 1 --2- 172.21.15.2:0/525889431 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x560640561310 0x560640563730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb915f9c740 4 client.17187 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:10.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.339+0000 7fb915f9c740 1 -- 172.21.15.2:0/525889431 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606401ec720 con 0x560640561310 2024-03-15T13:04:10.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.340+0000 7fb911165640 1 --2- 172.21.15.2:0/525889431 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x560640561310 0x560640563730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.340+0000 7fb911165640 1 --2- 172.21.15.2:0/525889431 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x560640561310 0x560640563730 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecfc089a79455162 server_cookie=dbd9b320739adae5 in_seq=0 out_seq=0 2024-03-15T13:04:10.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.340+0000 7fb8f4ff9640 10 client.17187 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:10.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.341+0000 7fb8f4ff9640 1 -- 172.21.15.2:0/525889431 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5606401ec720 con 0x560640561310 2024-03-15T13:04:10.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.341+0000 7fb8f4ff9640 10 client.17187 handle_command_reply: tid=0 2024-03-15T13:04:10.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.341+0000 7fb915f9c740 1 client.17187 shutdown 2024-03-15T13:04:10.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.341+0000 7fb915f9c740 1 -- 172.21.15.2:0/525889431 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 msgr2=0x56064054eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.341+0000 7fb915f9c740 1 --2- 172.21.15.2:0/525889431 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x56064054eb70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0ed600 tx=0x7fb8fc0bd190 comp rx=0 tx=0).stop 2024-03-15T13:04:10.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.341+0000 7fb915f9c740 1 -- 172.21.15.2:0/525889431 shutdown_connections 2024-03-15T13:04:10.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.341+0000 7fb915f9c740 1 --2- 172.21.15.2:0/525889431 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x560640561310 0x560640563730 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.341+0000 7fb915f9c740 1 --2- 172.21.15.2:0/525889431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405517b0 0x56064054f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.341+0000 7fb915f9c740 1 --2- 172.21.15.2:0/525889431 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x56064054eb70 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.341+0000 7fb915f9c740 1 --2- 172.21.15.2:0/525889431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 0x56064054e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.341+0000 7fb8f4ff9640 0 client.17187 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:10.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.341+0000 7fb915f9c740 1 -- 172.21.15.2:0/525889431 >> 172.21.15.2:0/525889431 conn(0x560640542990 msgr2=0x560640544730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.341+0000 7fb915f9c740 1 -- 172.21.15.2:0/525889431 shutdown_connections 2024-03-15T13:04:10.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.341+0000 7fb915f9c740 1 -- 172.21.15.2:0/525889431 wait complete. 2024-03-15T13:04:10.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.341+0000 7fb915f9c740 20 client.17187 trim_cache size 0 max 16384 2024-03-15T13:04:10.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.357+0000 7fb915f9c740 1 Processor -- start 2024-03-15T13:04:10.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.357+0000 7fb915f9c740 1 -- start start 2024-03-15T13:04:10.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.357+0000 7fb915f9c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405482b0 0x560640550d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.357+0000 7fb915f9c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x56064054eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.357+0000 7fb915f9c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405517b0 0x56064054d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.357+0000 7fb915f9c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606405427f0 con 0x560640548bd0 2024-03-15T13:04:10.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.357+0000 7fb915f9c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064053ce20 con 0x5606405517b0 2024-03-15T13:04:10.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.357+0000 7fb915f9c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064054cb80 con 0x5606405482b0 2024-03-15T13:04:10.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.357+0000 7fb910964640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x56064054eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.358+0000 7fb910964640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x56064054eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41628/0 (socket says 172.21.15.2:41628) 2024-03-15T13:04:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.358+0000 7fb910964640 1 -- 172.21.15.2:0/320554458 learned_addr learned my addr 172.21.15.2:0/320554458 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.358+0000 7fb911165640 1 --2- 172.21.15.2:0/320554458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405482b0 0x560640550d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.358+0000 7fb911966640 1 --2- 172.21.15.2:0/320554458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405517b0 0x56064054d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.358+0000 7fb910964640 1 -- 172.21.15.2:0/320554458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405482b0 msgr2=0x560640550d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.358+0000 7fb910964640 1 --2- 172.21.15.2:0/320554458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405482b0 0x560640550d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.358+0000 7fb910964640 1 -- 172.21.15.2:0/320554458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405517b0 msgr2=0x56064054d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.358+0000 7fb910964640 1 --2- 172.21.15.2:0/320554458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405517b0 0x56064054d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.358+0000 7fb910964640 1 -- 172.21.15.2:0/320554458 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56064044aac0 con 0x560640548bd0 2024-03-15T13:04:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.358+0000 7fb910964640 1 --2- 172.21.15.2:0/320554458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x56064054eed0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc004c40 tx=0x7fb8fc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f6d04ff3f1dca6 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.358+0000 7fb8f6ffd640 1 -- 172.21.15.2:0/320554458 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc0bca30 con 0x560640548bd0 2024-03-15T13:04:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.358+0000 7fb8f6ffd640 1 -- 172.21.15.2:0/320554458 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb8fc0052d0 con 0x560640548bd0 2024-03-15T13:04:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.358+0000 7fb8f6ffd640 1 -- 172.21.15.2:0/320554458 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc0bcc90 con 0x560640548bd0 2024-03-15T13:04:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.359+0000 7fb915f9c740 1 -- 172.21.15.2:0/320554458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 msgr2=0x56064054eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.359+0000 7fb915f9c740 1 --2- 172.21.15.2:0/320554458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x56064054eed0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc004c40 tx=0x7fb8fc009260 comp rx=0 tx=0).stop 2024-03-15T13:04:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.359+0000 7fb915f9c740 1 -- 172.21.15.2:0/320554458 shutdown_connections 2024-03-15T13:04:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.359+0000 7fb915f9c740 1 --2- 172.21.15.2:0/320554458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405517b0 0x56064054d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.359+0000 7fb915f9c740 1 --2- 172.21.15.2:0/320554458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x56064054eed0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.359+0000 7fb915f9c740 1 --2- 172.21.15.2:0/320554458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405482b0 0x560640550d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.359+0000 7fb915f9c740 1 -- 172.21.15.2:0/320554458 >> 172.21.15.2:0/320554458 conn(0x560640542990 msgr2=0x56064054cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.359+0000 7fb915f9c740 1 -- 172.21.15.2:0/320554458 shutdown_connections 2024-03-15T13:04:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.359+0000 7fb915f9c740 1 -- 172.21.15.2:0/320554458 wait complete. 2024-03-15T13:04:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.359+0000 7fb915f9c740 1 -- start start 2024-03-15T13:04:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.359+0000 7fb915f9c740 1 Processor -- start 2024-03-15T13:04:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb915f9c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 0x560640553130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb915f9c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x560640551b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb915f9c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405517b0 0x5606405520d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb915f9c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606405537d0 con 0x560640548bd0 2024-03-15T13:04:10.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb915f9c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560640553940 con 0x5606405482b0 2024-03-15T13:04:10.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb915f9c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560640552610 con 0x5606405517b0 2024-03-15T13:04:10.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb915f9c740 10 client.0 _ll_register_callbacks cb 0x7fb8fc0c0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:10.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb910964640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x560640551b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb910964640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x560640551b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41632/0 (socket says 172.21.15.2:41632) 2024-03-15T13:04:10.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb910964640 1 -- 172.21.15.2:0/4165409042 learned_addr learned my addr 172.21.15.2:0/4165409042 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb911165640 1 --2- 172.21.15.2:0/4165409042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 0x560640553130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb911966640 1 --2- 172.21.15.2:0/4165409042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405517b0 0x5606405520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb910964640 1 -- 172.21.15.2:0/4165409042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405517b0 msgr2=0x5606405520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb910964640 1 --2- 172.21.15.2:0/4165409042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405517b0 0x5606405520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb910964640 1 -- 172.21.15.2:0/4165409042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 msgr2=0x560640553130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb910964640 1 --2- 172.21.15.2:0/4165409042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 0x560640553130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb910964640 1 -- 172.21.15.2:0/4165409042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606404ae610 con 0x560640548bd0 2024-03-15T13:04:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb910964640 1 --2- 172.21.15.2:0/4165409042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x560640551b90 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0bace0 tx=0x7fb8fc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e7fab0a25a92ca server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb8f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb8f4ff9640 1 -- 172.21.15.2:0/4165409042 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc0bca30 con 0x560640548bd0 2024-03-15T13:04:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb8f4ff9640 1 -- 172.21.15.2:0/4165409042 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fb8fc04b120 con 0x560640548bd0 2024-03-15T13:04:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb915f9c740 1 -- 172.21.15.2:0/4165409042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560640554130 con 0x560640548bd0 2024-03-15T13:04:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb8f4ff9640 1 -- 172.21.15.2:0/4165409042 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8fc0bd130 con 0x560640548bd0 2024-03-15T13:04:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb915f9c740 10 client.17193 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.360+0000 7fb915f9c740 1 -- 172.21.15.2:0/4165409042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5606404706a0 con 0x560640548bd0 2024-03-15T13:04:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.361+0000 7fb8f4ff9640 1 -- 172.21.15.2:0/4165409042 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x560640554130 con 0x560640548bd0 2024-03-15T13:04:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.361+0000 7fb8f4ff9640 1 -- 172.21.15.2:0/4165409042 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fb8fc03c070 con 0x560640548bd0 2024-03-15T13:04:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.361+0000 7fb915f9c740 10 client.17193 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.361+0000 7fb915f9c740 10 client.17193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.361+0000 7fb915f9c740 20 client.17193 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.361+0000 7fb915f9c740 1 --2- 172.21.15.2:0/4165409042 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5606405bcbc0 0x5606405befb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.361+0000 7fb915f9c740 4 client.17193 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:04:10.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.361+0000 7fb915f9c740 1 -- 172.21.15.2:0/4165409042 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fb8fc02ce90 con 0x5606405bcbc0 2024-03-15T13:04:10.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.361+0000 7fb911165640 1 --2- 172.21.15.2:0/4165409042 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5606405bcbc0 0x5606405befb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.361+0000 7fb911165640 1 --2- 172.21.15.2:0/4165409042 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5606405bcbc0 0x5606405befb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c6735a306494f7f server_cookie=25d34d450f9f4f0b in_seq=0 out_seq=0 2024-03-15T13:04:10.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.361+0000 7fb8f4ff9640 10 client.17193 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:10.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb8f4ff9640 1 -- 172.21.15.2:0/4165409042 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fb8fc02ce90 con 0x5606405bcbc0 2024-03-15T13:04:10.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb8f4ff9640 10 client.17193 handle_command_reply: tid=0 2024-03-15T13:04:10.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb915f9c740 1 client.17193 shutdown 2024-03-15T13:04:10.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb915f9c740 1 -- 172.21.15.2:0/4165409042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 msgr2=0x560640551b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb915f9c740 1 --2- 172.21.15.2:0/4165409042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x560640551b90 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0bace0 tx=0x7fb8fc005fd0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb915f9c740 1 -- 172.21.15.2:0/4165409042 shutdown_connections 2024-03-15T13:04:10.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb915f9c740 1 --2- 172.21.15.2:0/4165409042 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5606405bcbc0 0x5606405befb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb915f9c740 1 --2- 172.21.15.2:0/4165409042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5606405517b0 0x5606405520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb915f9c740 1 --2- 172.21.15.2:0/4165409042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560640548bd0 0x560640551b90 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb8f4ff9640 0 client.17193 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:10.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb915f9c740 1 --2- 172.21.15.2:0/4165409042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606405482b0 0x560640553130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb915f9c740 1 -- 172.21.15.2:0/4165409042 >> 172.21.15.2:0/4165409042 conn(0x560640542990 msgr2=0x560640541a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb915f9c740 1 -- 172.21.15.2:0/4165409042 shutdown_connections 2024-03-15T13:04:10.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb915f9c740 1 -- 172.21.15.2:0/4165409042 wait complete. 2024-03-15T13:04:10.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb915f9c740 20 client.17193 trim_cache size 0 max 16384 2024-03-15T13:04:10.354 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:04:10.354 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:04:10.355 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:04:10.355 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:04:10.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb9133c9640 1 -- 172.21.15.2:0/2275295318 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb8f80a1c20 msgr2=0x7fb8f80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb9133c9640 1 --2- 172.21.15.2:0/2275295318 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb8f80a1c20 0x7fb8f80a40e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb904005fd0 tx=0x7fb904013c90 comp rx=0 tx=0).stop 2024-03-15T13:04:10.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb9133c9640 1 -- 172.21.15.2:0/2275295318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb90c100e20 msgr2=0x7fb90c19e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb9133c9640 1 --2- 172.21.15.2:0/2275295318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb90c100e20 0x7fb90c19e620 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009230 tx=0x7fb8fc02cb60 comp rx=0 tx=0).stop 2024-03-15T13:04:10.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb9133c9640 1 -- 172.21.15.2:0/2275295318 shutdown_connections 2024-03-15T13:04:10.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb9133c9640 1 --2- 172.21.15.2:0/2275295318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb90c105820 0x7fb90c1a29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb9133c9640 1 --2- 172.21.15.2:0/2275295318 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb8f80a1c20 0x7fb8f80a40e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb9133c9640 1 --2- 172.21.15.2:0/2275295318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb90c100e20 0x7fb90c19e620 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb9133c9640 1 --2- 172.21.15.2:0/2275295318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb90c100500 0x7fb90c19e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb9133c9640 1 -- 172.21.15.2:0/2275295318 >> 172.21.15.2:0/2275295318 conn(0x7fb90c0775a0 msgr2=0x7fb90c1032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.362+0000 7fb9133c9640 1 -- 172.21.15.2:0/2275295318 shutdown_connections 2024-03-15T13:04:10.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.363+0000 7fb9133c9640 1 -- 172.21.15.2:0/2275295318 wait complete. 2024-03-15T13:04:10.359 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:04:10.359 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:04:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.477+0000 7fa422264640 1 -- 172.21.15.2:0/3186541549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa41c100fd0 msgr2=0x7fa41c105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.477+0000 7fa422264640 1 --2- 172.21.15.2:0/3186541549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa41c100fd0 0x7fa41c105010 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa404009900 tx=0x7fa40402bc30 comp rx=0 tx=0).stop 2024-03-15T13:04:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.477+0000 7fa422264640 1 -- 172.21.15.2:0/3186541549 shutdown_connections 2024-03-15T13:04:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.477+0000 7fa422264640 1 --2- 172.21.15.2:0/3186541549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa41c1056f0 0x7fa41c10d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.477+0000 7fa422264640 1 --2- 172.21.15.2:0/3186541549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa41c100fd0 0x7fa41c105010 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.477+0000 7fa422264640 1 --2- 172.21.15.2:0/3186541549 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa41c100620 0x7fa41c100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.477+0000 7fa422264640 1 -- 172.21.15.2:0/3186541549 >> 172.21.15.2:0/3186541549 conn(0x7fa41c077490 msgr2=0x7fa41c0778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.477+0000 7fa422264640 1 -- 172.21.15.2:0/3186541549 shutdown_connections 2024-03-15T13:04:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.477+0000 7fa422264640 1 -- 172.21.15.2:0/3186541549 wait complete. 2024-03-15T13:04:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.477+0000 7fa422264640 1 Processor -- start 2024-03-15T13:04:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.477+0000 7fa422264640 1 -- start start 2024-03-15T13:04:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa422264640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa41c100620 0x7fa41c19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa422264640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa41c1056f0 0x7fa41c19dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa422264640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa41c197970 0x7fa41c197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa422264640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c198360 con 0x7fa41c197970 2024-03-15T13:04:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa422264640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c1984d0 con 0x7fa41c1056f0 2024-03-15T13:04:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa422264640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c198690 con 0x7fa41c100620 2024-03-15T13:04:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa41bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa41c100620 0x7fa41c19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa420801640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa41c197970 0x7fa41c197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa41bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa41c100620 0x7fa41c19d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:39730/0 (socket says 172.21.15.2:39730) 2024-03-15T13:04:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa41b7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa41c1056f0 0x7fa41c19dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa41bfff640 1 -- 172.21.15.2:0/3364285711 learned_addr learned my addr 172.21.15.2:0/3364285711 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa420801640 1 -- 172.21.15.2:0/3364285711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa41c100620 msgr2=0x7fa41c19d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa420801640 1 --2- 172.21.15.2:0/3364285711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa41c100620 0x7fa41c19d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa420801640 1 -- 172.21.15.2:0/3364285711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa41c1056f0 msgr2=0x7fa41c19dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa420801640 1 --2- 172.21.15.2:0/3364285711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa41c1056f0 0x7fa41c19dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa420801640 1 -- 172.21.15.2:0/3364285711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa404009560 con 0x7fa41c197970 2024-03-15T13:04:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa41b7fe640 1 --2- 172.21.15.2:0/3364285711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa41c1056f0 0x7fa41c19dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa420801640 1 --2- 172.21.15.2:0/3364285711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa41c197970 0x7fa41c197e20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa414009fd0 tx=0x7fa41400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.478+0000 7fa4197fa640 1 -- 172.21.15.2:0/3364285711 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41400ce20 con 0x7fa41c197970 2024-03-15T13:04:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.479+0000 7fa422264640 1 -- 172.21.15.2:0/3364285711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41c1a4680 con 0x7fa41c197970 2024-03-15T13:04:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.479+0000 7fa4197fa640 1 -- 172.21.15.2:0/3364285711 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa414005020 con 0x7fa41c197970 2024-03-15T13:04:10.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.479+0000 7fa4197fa640 1 -- 172.21.15.2:0/3364285711 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414010690 con 0x7fa41c197970 2024-03-15T13:04:10.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.479+0000 7fa422264640 1 -- 172.21.15.2:0/3364285711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41c1a4b60 con 0x7fa41c197970 2024-03-15T13:04:10.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.480+0000 7fa422264640 1 -- 172.21.15.2:0/3364285711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa41c068730 con 0x7fa41c197970 2024-03-15T13:04:10.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.481+0000 7fa4197fa640 1 -- 172.21.15.2:0/3364285711 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fa414005570 con 0x7fa41c197970 2024-03-15T13:04:10.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.481+0000 7fa4197fa640 1 --2- 172.21.15.2:0/3364285711 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa4080a1ba0 0x7fa4080a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.481+0000 7fa41bfff640 1 --2- 172.21.15.2:0/3364285711 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa4080a1ba0 0x7fa4080a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.481+0000 7fa4197fa640 1 -- 172.21.15.2:0/3364285711 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa414014070 con 0x7fa41c197970 2024-03-15T13:04:10.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.482+0000 7fa41bfff640 1 --2- 172.21.15.2:0/3364285711 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa4080a1ba0 0x7fa4080a4060 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa41c198ab0 tx=0x7fa40c005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.486+0000 7fa4197fa640 1 -- 172.21.15.2:0/3364285711 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa41409a900 con 0x7fa41c197970 2024-03-15T13:04:10.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.712+0000 7fa422264640 1 -- 172.21.15.2:0/3364285711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa41c100fd0 con 0x7fa41c197970 2024-03-15T13:04:10.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.712+0000 7fa4197fa640 1 -- 172.21.15.2:0/3364285711 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+11466 (secure 0 0 0) 0x7fa41409a490 con 0x7fa41c197970 2024-03-15T13:04:10.698 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:04:10.698 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:04:09.162095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24434,24539,24460,24464,24500]},"id":1}]} 2024-03-15T13:04:10.698 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 41 2024-03-15T13:04:10.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.714+0000 7fa422264640 1 -- 172.21.15.2:0/3364285711 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa4080a1ba0 msgr2=0x7fa4080a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.714+0000 7fa422264640 1 --2- 172.21.15.2:0/3364285711 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa4080a1ba0 0x7fa4080a4060 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa41c198ab0 tx=0x7fa40c005f90 comp rx=0 tx=0).stop 2024-03-15T13:04:10.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.714+0000 7fa422264640 1 -- 172.21.15.2:0/3364285711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa41c197970 msgr2=0x7fa41c197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.714+0000 7fa422264640 1 --2- 172.21.15.2:0/3364285711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa41c197970 0x7fa41c197e20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa414009fd0 tx=0x7fa41400ef10 comp rx=0 tx=0).stop 2024-03-15T13:04:10.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.714+0000 7fa422264640 1 -- 172.21.15.2:0/3364285711 shutdown_connections 2024-03-15T13:04:10.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.714+0000 7fa422264640 1 --2- 172.21.15.2:0/3364285711 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa4080a1ba0 0x7fa4080a4060 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.714+0000 7fa422264640 1 --2- 172.21.15.2:0/3364285711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa41c197970 0x7fa41c197e20 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.714+0000 7fa422264640 1 --2- 172.21.15.2:0/3364285711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa41c1056f0 0x7fa41c19dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.714+0000 7fa422264640 1 --2- 172.21.15.2:0/3364285711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa41c100620 0x7fa41c19d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.714+0000 7fa422264640 1 -- 172.21.15.2:0/3364285711 >> 172.21.15.2:0/3364285711 conn(0x7fa41c077490 msgr2=0x7fa41c104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.714+0000 7fa422264640 1 -- 172.21.15.2:0/3364285711 shutdown_connections 2024-03-15T13:04:10.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.714+0000 7fa422264640 1 -- 172.21.15.2:0/3364285711 wait complete. 2024-03-15T13:04:10.711 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:04:10.711 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:04:10.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.830+0000 7f578d647640 1 -- 172.21.15.2:0/2769420242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f578810bad0 msgr2=0x7f578810df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.830+0000 7f578d647640 1 --2- 172.21.15.2:0/2769420242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f578810bad0 0x7f578810df00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5780009900 tx=0x7f578002bcd0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.830+0000 7f578d647640 1 -- 172.21.15.2:0/2769420242 shutdown_connections 2024-03-15T13:04:10.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.830+0000 7f578d647640 1 --2- 172.21.15.2:0/2769420242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f578810bad0 0x7f578810df00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.830+0000 7f578d647640 1 --2- 172.21.15.2:0/2769420242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5788103020 0x7f578810b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.830+0000 7f578d647640 1 --2- 172.21.15.2:0/2769420242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5788102700 0x7f5788102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.830+0000 7f578d647640 1 -- 172.21.15.2:0/2769420242 >> 172.21.15.2:0/2769420242 conn(0x7f5788076770 msgr2=0x7f5788076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.830+0000 7f578d647640 1 -- 172.21.15.2:0/2769420242 shutdown_connections 2024-03-15T13:04:10.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.830+0000 7f578d647640 1 -- 172.21.15.2:0/2769420242 wait complete. 2024-03-15T13:04:10.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.831+0000 7f578d647640 1 Processor -- start 2024-03-15T13:04:10.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.831+0000 7f578d647640 1 -- start start 2024-03-15T13:04:10.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.831+0000 7f578d647640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5788102700 0x7f5788100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.831+0000 7f578d647640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5788103020 0x7f5788100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.831+0000 7f578d647640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f578810bad0 0x7f5788100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.831+0000 7f578d647640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57881a5ec0 con 0x7f5788103020 2024-03-15T13:04:10.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.831+0000 7f578d647640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57881a6030 con 0x7f5788102700 2024-03-15T13:04:10.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.831+0000 7f578d647640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57881a61f0 con 0x7f578810bad0 2024-03-15T13:04:10.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.831+0000 7f57867fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5788103020 0x7f5788100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.831+0000 7f57867fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5788103020 0x7f5788100890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41678/0 (socket says 172.21.15.2:41678) 2024-03-15T13:04:10.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.831+0000 7f57867fc640 1 -- 172.21.15.2:0/4210398622 learned_addr learned my addr 172.21.15.2:0/4210398622 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.831+0000 7f5786ffd640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5788102700 0x7f5788100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.831+0000 7f57877fe640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f578810bad0 0x7f5788100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.832+0000 7f57867fc640 1 -- 172.21.15.2:0/4210398622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f578810bad0 msgr2=0x7f5788100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.832+0000 7f57867fc640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f578810bad0 0x7f5788100dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.832+0000 7f57867fc640 1 -- 172.21.15.2:0/4210398622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5788102700 msgr2=0x7f5788100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.832+0000 7f57867fc640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5788102700 0x7f5788100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.832+0000 7f57867fc640 1 -- 172.21.15.2:0/4210398622 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5780009560 con 0x7f5788103020 2024-03-15T13:04:10.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.832+0000 7f5786ffd640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5788102700 0x7f5788100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:10.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.832+0000 7f57877fe640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f578810bad0 0x7f5788100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:10.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.832+0000 7f57867fc640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5788103020 0x7f5788100890 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f577000e9f0 tx=0x7f577000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.832+0000 7f577ffff640 1 -- 172.21.15.2:0/4210398622 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770009700 con 0x7f5788103020 2024-03-15T13:04:10.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.832+0000 7f578d647640 1 -- 172.21.15.2:0/4210398622 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57881a64d0 con 0x7f5788103020 2024-03-15T13:04:10.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.832+0000 7f577ffff640 1 -- 172.21.15.2:0/4210398622 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f577000cd80 con 0x7f5788103020 2024-03-15T13:04:10.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.832+0000 7f577ffff640 1 -- 172.21.15.2:0/4210398622 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770018920 con 0x7f5788103020 2024-03-15T13:04:10.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.832+0000 7f578d647640 1 -- 172.21.15.2:0/4210398622 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57881a69e0 con 0x7f5788103020 2024-03-15T13:04:10.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.834+0000 7f577ffff640 1 -- 172.21.15.2:0/4210398622 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f5770018a80 con 0x7f5788103020 2024-03-15T13:04:10.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.834+0000 7f5790228740 1 Processor -- start 2024-03-15T13:04:10.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.834+0000 7f5790228740 1 -- start start 2024-03-15T13:04:10.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f5790228740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2250 0x559d452e2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f5790228740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2b70 0x559d452eb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f5790228740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452eb750 0x559d452edb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f5790228740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4529d960 con 0x559d452e2250 2024-03-15T13:04:10.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f5790228740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d45251310 con 0x559d452e2b70 2024-03-15T13:04:10.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f5790228740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4524e840 con 0x559d452eb750 2024-03-15T13:04:10.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f577ffff640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f57740a1ba0 0x7f57740a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f5786ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2b70 0x559d452eb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f57877fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2250 0x559d452e2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f5786ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2b70 0x559d452eb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:60924/0 (socket says 172.21.15.2:60924) 2024-03-15T13:04:10.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f57867fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452eb750 0x559d452edb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f5786ffd640 1 -- 172.21.15.2:0/2337386603 learned_addr learned my addr 172.21.15.2:0/2337386603 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f57877fe640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f57740a1ba0 0x7f57740a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f577ffff640 1 -- 172.21.15.2:0/4210398622 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5770014070 con 0x7f5788103020 2024-03-15T13:04:10.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f5786ffd640 1 -- 172.21.15.2:0/2337386603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452eb750 msgr2=0x559d452edb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f5786ffd640 1 --2- 172.21.15.2:0/2337386603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452eb750 0x559d452edb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f5786ffd640 1 -- 172.21.15.2:0/2337386603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2250 msgr2=0x559d452e2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f5786ffd640 1 --2- 172.21.15.2:0/2337386603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2250 0x559d452e2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.835+0000 7f5786ffd640 1 -- 172.21.15.2:0/2337386603 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d4520c060 con 0x559d452e2b70 2024-03-15T13:04:10.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.836+0000 7f5786ffd640 1 --2- 172.21.15.2:0/2337386603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2b70 0x559d452eb210 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5778011860 tx=0x7f5778011c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf3243a91d2a6edf server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.836+0000 7f57877fe640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f57740a1ba0 0x7f57740a4060 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5780005940 tx=0x7f57800058d0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.836+0000 7f577d7fa640 1 -- 172.21.15.2:0/2337386603 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778011310 con 0x559d452e2b70 2024-03-15T13:04:10.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.836+0000 7f577d7fa640 1 -- 172.21.15.2:0/2337386603 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5778003e40 con 0x559d452e2b70 2024-03-15T13:04:10.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.836+0000 7f577d7fa640 1 -- 172.21.15.2:0/2337386603 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778004550 con 0x559d452e2b70 2024-03-15T13:04:10.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.836+0000 7f5790228740 1 -- 172.21.15.2:0/2337386603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2b70 msgr2=0x559d452eb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.836+0000 7f5790228740 1 --2- 172.21.15.2:0/2337386603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2b70 0x559d452eb210 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5778011860 tx=0x7f5778011c60 comp rx=0 tx=0).stop 2024-03-15T13:04:10.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 1 -- 172.21.15.2:0/2337386603 shutdown_connections 2024-03-15T13:04:10.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 1 --2- 172.21.15.2:0/2337386603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452eb750 0x559d452edb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 1 --2- 172.21.15.2:0/2337386603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2b70 0x559d452eb210 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 1 --2- 172.21.15.2:0/2337386603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2250 0x559d452e2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 1 -- 172.21.15.2:0/2337386603 >> 172.21.15.2:0/2337386603 conn(0x559d452dc980 msgr2=0x559d452dcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 1 -- 172.21.15.2:0/2337386603 shutdown_connections 2024-03-15T13:04:10.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 1 -- 172.21.15.2:0/2337386603 wait complete. 2024-03-15T13:04:10.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 1 -- start start 2024-03-15T13:04:10.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 1 Processor -- start 2024-03-15T13:04:10.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452e2250 0x559d452eb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2b70 0x559d452e84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452eb750 0x559d452e8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4524a020 con 0x559d452eb750 2024-03-15T13:04:10.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d44f1d430 con 0x559d452e2b70 2024-03-15T13:04:10.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d44ed9e00 con 0x559d452e2250 2024-03-15T13:04:10.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5790228740 10 client.0 _ll_register_callbacks cb 0x559d451f1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:10.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f57877fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452eb750 0x559d452e8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f57877fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452eb750 0x559d452e8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41694/0 (socket says 172.21.15.2:41694) 2024-03-15T13:04:10.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f57877fe640 1 -- 172.21.15.2:0/4267808612 learned_addr learned my addr 172.21.15.2:0/4267808612 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f57867fc640 1 --2- 172.21.15.2:0/4267808612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2b70 0x559d452e84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.837+0000 7f5786ffd640 1 --2- 172.21.15.2:0/4267808612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452e2250 0x559d452eb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f57877fe640 1 -- 172.21.15.2:0/4267808612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452e2250 msgr2=0x559d452eb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f57877fe640 1 --2- 172.21.15.2:0/4267808612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452e2250 0x559d452eb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f57877fe640 1 -- 172.21.15.2:0/4267808612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2b70 msgr2=0x559d452e84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f57877fe640 1 --2- 172.21.15.2:0/4267808612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2b70 0x559d452e84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f57877fe640 1 -- 172.21.15.2:0/4267808612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57780114c0 con 0x559d452eb750 2024-03-15T13:04:10.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f57867fc640 1 --2- 172.21.15.2:0/4267808612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2b70 0x559d452e84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:10.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f57877fe640 1 --2- 172.21.15.2:0/4267808612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452eb750 0x559d452e8ab0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5780035000 tx=0x7f5780009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381bef9183ab7169 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f576e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:10.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f576e7fc640 1 -- 172.21.15.2:0/4267808612 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57800042c0 con 0x559d452eb750 2024-03-15T13:04:10.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f576e7fc640 1 -- 172.21.15.2:0/4267808612 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5780049040 con 0x559d452eb750 2024-03-15T13:04:10.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f5790228740 1 -- 172.21.15.2:0/4267808612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d44f497e0 con 0x559d452eb750 2024-03-15T13:04:10.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f576e7fc640 1 -- 172.21.15.2:0/4267808612 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5780004420 con 0x559d452eb750 2024-03-15T13:04:10.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f5790228740 10 client.17211 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:10.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f5790228740 1 -- 172.21.15.2:0/4267808612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x559d452486a0 con 0x559d452eb750 2024-03-15T13:04:10.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f576e7fc640 1 -- 172.21.15.2:0/4267808612 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x559d44f497e0 con 0x559d452eb750 2024-03-15T13:04:10.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.838+0000 7f576e7fc640 1 -- 172.21.15.2:0/4267808612 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f578003c070 con 0x559d452eb750 2024-03-15T13:04:10.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.839+0000 7f5790228740 10 client.17211 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:10.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.839+0000 7f5790228740 10 client.17211 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:04:10.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.839+0000 7f5790228740 20 client.17211 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:10.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.839+0000 7f5790228740 1 --2- 172.21.15.2:0/4267808612 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x559d452fb810 0x559d452fdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.839+0000 7f5790228740 4 client.17211 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:10.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.839+0000 7f5790228740 1 -- 172.21.15.2:0/4267808612 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d44f86720 con 0x559d452fb810 2024-03-15T13:04:10.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.839+0000 7f5786ffd640 1 --2- 172.21.15.2:0/4267808612 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x559d452fb810 0x559d452fdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.839+0000 7f5786ffd640 1 --2- 172.21.15.2:0/4267808612 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x559d452fb810 0x559d452fdc30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3859233d66cd6560 server_cookie=b82f4f9cf36d2480 in_seq=0 out_seq=0 2024-03-15T13:04:10.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.839+0000 7f576e7fc640 10 client.17211 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:04:10.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.840+0000 7f576e7fc640 1 -- 172.21.15.2:0/4267808612 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559d44f86720 con 0x559d452fb810 2024-03-15T13:04:10.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.840+0000 7f576e7fc640 10 client.17211 handle_command_reply: tid=0 2024-03-15T13:04:10.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.840+0000 7f5790228740 1 client.17211 shutdown 2024-03-15T13:04:10.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.840+0000 7f5790228740 1 -- 172.21.15.2:0/4267808612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452eb750 msgr2=0x559d452e8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.840+0000 7f5790228740 1 --2- 172.21.15.2:0/4267808612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452eb750 0x559d452e8ab0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5780035000 tx=0x7f5780009260 comp rx=0 tx=0).stop 2024-03-15T13:04:10.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.840+0000 7f5790228740 1 -- 172.21.15.2:0/4267808612 shutdown_connections 2024-03-15T13:04:10.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.840+0000 7f5790228740 1 --2- 172.21.15.2:0/4267808612 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x559d452fb810 0x559d452fdc30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.840+0000 7f5790228740 1 --2- 172.21.15.2:0/4267808612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452eb750 0x559d452e8ab0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.840+0000 7f5790228740 1 --2- 172.21.15.2:0/4267808612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2b70 0x559d452e84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.840+0000 7f576e7fc640 0 client.17211 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:04:10.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.840+0000 7f5790228740 1 --2- 172.21.15.2:0/4267808612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452e2250 0x559d452eb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.840+0000 7f5790228740 1 -- 172.21.15.2:0/4267808612 >> 172.21.15.2:0/4267808612 conn(0x559d452dc980 msgr2=0x559d452de6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.840+0000 7f5790228740 1 -- 172.21.15.2:0/4267808612 shutdown_connections 2024-03-15T13:04:10.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.840+0000 7f5790228740 1 -- 172.21.15.2:0/4267808612 wait complete. 2024-03-15T13:04:10.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.840+0000 7f5790228740 20 client.17211 trim_cache size 0 max 16384 2024-03-15T13:04:10.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.852+0000 7f5790228740 1 Processor -- start 2024-03-15T13:04:10.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.852+0000 7f5790228740 1 -- start start 2024-03-15T13:04:10.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f5790228740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2250 0x559d452de880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f5790228740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452e2b70 0x559d452eabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f5790228740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452eb750 0x559d452eb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f5790228740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d451cf130 con 0x559d452e2250 2024-03-15T13:04:10.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f5790228740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d452e1eb0 con 0x559d452eb750 2024-03-15T13:04:10.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f5790228740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d452dd9b0 con 0x559d452e2b70 2024-03-15T13:04:10.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f5786ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2250 0x559d452de880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f5786ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2250 0x559d452de880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41702/0 (socket says 172.21.15.2:41702) 2024-03-15T13:04:10.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f5786ffd640 1 -- 172.21.15.2:0/2298197457 learned_addr learned my addr 172.21.15.2:0/2298197457 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f57867fc640 1 --2- 172.21.15.2:0/2298197457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452e2b70 0x559d452eabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f57877fe640 1 --2- 172.21.15.2:0/2298197457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452eb750 0x559d452eb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f5786ffd640 1 -- 172.21.15.2:0/2298197457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452e2b70 msgr2=0x559d452eabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f5786ffd640 1 --2- 172.21.15.2:0/2298197457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452e2b70 0x559d452eabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f5786ffd640 1 -- 172.21.15.2:0/2298197457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452eb750 msgr2=0x559d452eb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f5786ffd640 1 --2- 172.21.15.2:0/2298197457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452eb750 0x559d452eb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f5786ffd640 1 -- 172.21.15.2:0/2298197457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d451e4c20 con 0x559d452e2250 2024-03-15T13:04:10.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f5786ffd640 1 --2- 172.21.15.2:0/2298197457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2250 0x559d452de880 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f57780117b0 tx=0x7f577800ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237446063f71f766 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.853+0000 7f577cff9640 1 -- 172.21.15.2:0/2298197457 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778003970 con 0x559d452e2250 2024-03-15T13:04:10.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.854+0000 7f577cff9640 1 -- 172.21.15.2:0/2298197457 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f577800fde0 con 0x559d452e2250 2024-03-15T13:04:10.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.854+0000 7f577cff9640 1 -- 172.21.15.2:0/2298197457 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778028d20 con 0x559d452e2250 2024-03-15T13:04:10.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.854+0000 7f5790228740 1 -- 172.21.15.2:0/2298197457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2250 msgr2=0x559d452de880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.854+0000 7f5790228740 1 --2- 172.21.15.2:0/2298197457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2250 0x559d452de880 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f57780117b0 tx=0x7f577800ffa0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.854+0000 7f5790228740 1 -- 172.21.15.2:0/2298197457 shutdown_connections 2024-03-15T13:04:10.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.854+0000 7f5790228740 1 --2- 172.21.15.2:0/2298197457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452eb750 0x559d452eb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.854+0000 7f5790228740 1 --2- 172.21.15.2:0/2298197457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452e2b70 0x559d452eabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.854+0000 7f5790228740 1 --2- 172.21.15.2:0/2298197457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2250 0x559d452de880 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.854+0000 7f5790228740 1 -- 172.21.15.2:0/2298197457 >> 172.21.15.2:0/2298197457 conn(0x559d452dc980 msgr2=0x559d452dbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.854+0000 7f5790228740 1 -- 172.21.15.2:0/2298197457 shutdown_connections 2024-03-15T13:04:10.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.854+0000 7f5790228740 1 -- 172.21.15.2:0/2298197457 wait complete. 2024-03-15T13:04:10.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.854+0000 7f5790228740 1 -- start start 2024-03-15T13:04:10.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f5790228740 1 Processor -- start 2024-03-15T13:04:10.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f5790228740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2250 0x559d452e4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f5790228740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2b70 0x559d452e53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f5790228740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452eb750 0x559d453560e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f5790228740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d452e5b80 con 0x559d452e2b70 2024-03-15T13:04:10.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f5790228740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d452e5cf0 con 0x559d452e2250 2024-03-15T13:04:10.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f5790228740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d45356620 con 0x559d452eb750 2024-03-15T13:04:10.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f5790228740 10 client.0 _ll_register_callbacks cb 0x7f5780009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:10.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f57867fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2b70 0x559d452e53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f57867fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2b70 0x559d452e53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41704/0 (socket says 172.21.15.2:41704) 2024-03-15T13:04:10.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f57867fc640 1 -- 172.21.15.2:0/924833353 learned_addr learned my addr 172.21.15.2:0/924833353 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f5786ffd640 1 --2- 172.21.15.2:0/924833353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2250 0x559d452e4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f57877fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452eb750 0x559d453560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f57867fc640 1 -- 172.21.15.2:0/924833353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452eb750 msgr2=0x559d453560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f57867fc640 1 --2- 172.21.15.2:0/924833353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452eb750 0x559d453560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f57867fc640 1 -- 172.21.15.2:0/924833353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2250 msgr2=0x559d452e4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f57867fc640 1 --2- 172.21.15.2:0/924833353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2250 0x559d452e4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f57867fc640 1 -- 172.21.15.2:0/924833353 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d4520c060 con 0x559d452e2b70 2024-03-15T13:04:10.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f57867fc640 1 --2- 172.21.15.2:0/924833353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2b70 0x559d452e53e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f577009a490 tx=0x7f57700c4da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba575bf31150b992 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.855+0000 7f576e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:10.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.856+0000 7f576e7fc640 1 -- 172.21.15.2:0/924833353 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57700a2bf0 con 0x559d452e2b70 2024-03-15T13:04:10.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.856+0000 7f576e7fc640 1 -- 172.21.15.2:0/924833353 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f57700850b0 con 0x559d452e2b70 2024-03-15T13:04:10.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.856+0000 7f5790228740 1 -- 172.21.15.2:0/924833353 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5778003e40 con 0x559d452e2b70 2024-03-15T13:04:10.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.856+0000 7f576e7fc640 1 -- 172.21.15.2:0/924833353 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57700b3200 con 0x559d452e2b70 2024-03-15T13:04:10.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.856+0000 7f5790228740 10 client.17217 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:10.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.856+0000 7f5790228740 1 -- 172.21.15.2:0/924833353 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x559d4520a8c0 con 0x559d452e2b70 2024-03-15T13:04:10.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.856+0000 7f576e7fc640 1 -- 172.21.15.2:0/924833353 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5778003e40 con 0x559d452e2b70 2024-03-15T13:04:10.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.856+0000 7f576e7fc640 1 -- 172.21.15.2:0/924833353 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f5770099650 con 0x559d452e2b70 2024-03-15T13:04:10.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.856+0000 7f5790228740 10 client.17217 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:10.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.856+0000 7f5790228740 10 client.17217 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:04:10.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.856+0000 7f5790228740 20 client.17217 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:10.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.856+0000 7f5790228740 1 --2- 172.21.15.2:0/924833353 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x559d45357e80 0x559d4535a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.856+0000 7f5790228740 4 client.17217 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:04:10.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.856+0000 7f5790228740 1 -- 172.21.15.2:0/924833353 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559d44fd5890 con 0x559d45357e80 2024-03-15T13:04:10.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.856+0000 7f5786ffd640 1 --2- 172.21.15.2:0/924833353 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x559d45357e80 0x559d4535a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f576e7fc640 10 client.17217 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:04:10.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f5786ffd640 1 --2- 172.21.15.2:0/924833353 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x559d45357e80 0x559d4535a2a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da2849c1ad670858 server_cookie=40dd738c1225539a in_seq=0 out_seq=0 2024-03-15T13:04:10.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f576e7fc640 1 -- 172.21.15.2:0/924833353 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559d44fd5890 con 0x559d45357e80 2024-03-15T13:04:10.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f576e7fc640 10 client.17217 handle_command_reply: tid=0 2024-03-15T13:04:10.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f5790228740 1 client.17217 shutdown 2024-03-15T13:04:10.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f5790228740 1 -- 172.21.15.2:0/924833353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2b70 msgr2=0x559d452e53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f5790228740 1 --2- 172.21.15.2:0/924833353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2b70 0x559d452e53e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f577009a490 tx=0x7f57700c4da0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f5790228740 1 -- 172.21.15.2:0/924833353 shutdown_connections 2024-03-15T13:04:10.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f5790228740 1 --2- 172.21.15.2:0/924833353 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x559d45357e80 0x559d4535a2a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f5790228740 1 --2- 172.21.15.2:0/924833353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559d452eb750 0x559d453560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f5790228740 1 --2- 172.21.15.2:0/924833353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559d452e2b70 0x559d452e53e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f576e7fc640 0 client.17217 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:04:10.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f5790228740 1 --2- 172.21.15.2:0/924833353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d452e2250 0x559d452e4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f5790228740 1 -- 172.21.15.2:0/924833353 >> 172.21.15.2:0/924833353 conn(0x559d452dc980 msgr2=0x559d452e7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f5790228740 1 -- 172.21.15.2:0/924833353 shutdown_connections 2024-03-15T13:04:10.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f5790228740 1 -- 172.21.15.2:0/924833353 wait complete. 2024-03-15T13:04:10.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.857+0000 7f5790228740 20 client.17217 trim_cache size 0 max 16384 2024-03-15T13:04:10.851 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:04:10.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.858+0000 7f578d647640 1 -- 172.21.15.2:0/4210398622 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f57740a1ba0 msgr2=0x7f57740a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.858+0000 7f578d647640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f57740a1ba0 0x7f57740a4060 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5780005940 tx=0x7f57800058d0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.858+0000 7f578d647640 1 -- 172.21.15.2:0/4210398622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5788103020 msgr2=0x7f5788100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.858+0000 7f578d647640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5788103020 0x7f5788100890 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f577000e9f0 tx=0x7f577000eeb0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.858+0000 7f578d647640 1 -- 172.21.15.2:0/4210398622 shutdown_connections 2024-03-15T13:04:10.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.858+0000 7f578d647640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f578810bad0 0x7f5788100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.858+0000 7f578d647640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f57740a1ba0 0x7f57740a4060 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.858+0000 7f578d647640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5788103020 0x7f5788100890 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.858+0000 7f578d647640 1 --2- 172.21.15.2:0/4210398622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5788102700 0x7f5788100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.858+0000 7f578d647640 1 -- 172.21.15.2:0/4210398622 >> 172.21.15.2:0/4210398622 conn(0x7f5788076770 msgr2=0x7f57880fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.858+0000 7f578d647640 1 -- 172.21.15.2:0/4210398622 shutdown_connections 2024-03-15T13:04:10.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.858+0000 7f578d647640 1 -- 172.21.15.2:0/4210398622 wait complete. 2024-03-15T13:04:10.853 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:10 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:10.854 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:04:10.854 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:04:10.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:10 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:10.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 -- 172.21.15.2:0/1192917666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16c4103080 msgr2=0x7f16c410b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 --2- 172.21.15.2:0/1192917666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16c4103080 0x7f16c410b560 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009900 tx=0x7f16b002bcd0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 -- 172.21.15.2:0/1192917666 shutdown_connections 2024-03-15T13:04:10.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 --2- 172.21.15.2:0/1192917666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16c410bb30 0x7f16c410dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 --2- 172.21.15.2:0/1192917666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16c4103080 0x7f16c410b560 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 --2- 172.21.15.2:0/1192917666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16c4102760 0x7f16c4102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 -- 172.21.15.2:0/1192917666 >> 172.21.15.2:0/1192917666 conn(0x7f16c4077550 msgr2=0x7f16c4077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 -- 172.21.15.2:0/1192917666 shutdown_connections 2024-03-15T13:04:10.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 -- 172.21.15.2:0/1192917666 wait complete. 2024-03-15T13:04:10.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 Processor -- start 2024-03-15T13:04:10.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 -- start start 2024-03-15T13:04:10.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16c4102760 0x7f16c419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16c4103080 0x7f16c419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16c410bb30 0x7f16c4197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c41980c0 con 0x7f16c4103080 2024-03-15T13:04:10.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c4198230 con 0x7f16c4102760 2024-03-15T13:04:10.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.973+0000 7f16cb0d0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c41983f0 con 0x7f16c410bb30 2024-03-15T13:04:10.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16c4103080 0x7f16c419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16c4103080 0x7f16c419ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41732/0 (socket says 172.21.15.2:41732) 2024-03-15T13:04:10.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16c3fff640 1 -- 172.21.15.2:0/1748102834 learned_addr learned my addr 172.21.15.2:0/1748102834 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16c8e6c640 1 --2- 172.21.15.2:0/1748102834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16c4102760 0x7f16c419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16c966d640 1 --2- 172.21.15.2:0/1748102834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16c410bb30 0x7f16c4197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16c3fff640 1 -- 172.21.15.2:0/1748102834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16c410bb30 msgr2=0x7f16c4197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16c3fff640 1 --2- 172.21.15.2:0/1748102834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16c410bb30 0x7f16c4197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16c3fff640 1 -- 172.21.15.2:0/1748102834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16c4102760 msgr2=0x7f16c419d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16c3fff640 1 --2- 172.21.15.2:0/1748102834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16c4102760 0x7f16c419d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16c3fff640 1 -- 172.21.15.2:0/1748102834 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b0009560 con 0x7f16c4103080 2024-03-15T13:04:10.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16c3fff640 1 --2- 172.21.15.2:0/1748102834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16c4103080 0x7f16c419ddb0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009230 tx=0x7f16b002cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16c1ffb640 1 -- 172.21.15.2:0/1748102834 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0004250 con 0x7f16c4103080 2024-03-15T13:04:10.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16c1ffb640 1 -- 172.21.15.2:0/1748102834 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f16b00043b0 con 0x7f16c4103080 2024-03-15T13:04:10.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16cb0d0640 1 -- 172.21.15.2:0/1748102834 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c4198670 con 0x7f16c4103080 2024-03-15T13:04:10.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16c1ffb640 1 -- 172.21.15.2:0/1748102834 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0003bf0 con 0x7f16c4103080 2024-03-15T13:04:10.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.974+0000 7f16cb0d0640 1 -- 172.21.15.2:0/1748102834 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c41a4910 con 0x7f16c4103080 2024-03-15T13:04:10.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.976+0000 7f16c1ffb640 1 -- 172.21.15.2:0/1748102834 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f16b0003d50 con 0x7f16c4103080 2024-03-15T13:04:10.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16cdd81740 1 Processor -- start 2024-03-15T13:04:10.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16cdd81740 1 -- start start 2024-03-15T13:04:10.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16c1ffb640 1 --2- 172.21.15.2:0/1748102834 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f16b40a1ba0 0x7f16b40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16cdd81740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca4f00 0x562140ca52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16cdd81740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca5820 0x562140cb2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16cdd81740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140cb2b60 0x562140cb4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16cdd81740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562140c60960 con 0x562140ca5820 2024-03-15T13:04:10.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16cdd81740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562140c14310 con 0x562140ca4f00 2024-03-15T13:04:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16cdd81740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562140c11840 con 0x562140cb2b60 2024-03-15T13:04:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16c966d640 1 --2- 172.21.15.2:0/1748102834 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f16b40a1ba0 0x7f16b40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16c1ffb640 1 -- 172.21.15.2:0/1748102834 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f16b00f00c0 con 0x7f16c4103080 2024-03-15T13:04:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca5820 0x562140cb2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16c8e6c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca4f00 0x562140ca52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca5820 0x562140cb2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41748/0 (socket says 172.21.15.2:41748) 2024-03-15T13:04:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16c3fff640 1 -- 172.21.15.2:0/2364417422 learned_addr learned my addr 172.21.15.2:0/2364417422 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.977+0000 7f16c966d640 1 --2- 172.21.15.2:0/2364417422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140cb2b60 0x562140cb4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.978+0000 7f16c8e6c640 1 -- 172.21.15.2:0/2364417422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140cb2b60 msgr2=0x562140cb4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.978+0000 7f16c8e6c640 1 --2- 172.21.15.2:0/2364417422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140cb2b60 0x562140cb4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.978+0000 7f16c8e6c640 1 -- 172.21.15.2:0/2364417422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca5820 msgr2=0x562140cb2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.978+0000 7f16c8e6c640 1 --2- 172.21.15.2:0/2364417422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca5820 0x562140cb2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.978+0000 7f16c8e6c640 1 -- 172.21.15.2:0/2364417422 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562140bcf060 con 0x562140ca4f00 2024-03-15T13:04:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.978+0000 7f16c3fff640 1 --2- 172.21.15.2:0/2364417422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca5820 0x562140cb2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.978+0000 7f16c966d640 1 --2- 172.21.15.2:0/1748102834 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f16b40a1ba0 0x7f16b40a4060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009fd0 tx=0x7f16bc009d80 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.978+0000 7f16c8e6c640 1 --2- 172.21.15.2:0/2364417422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca4f00 0x562140ca52e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009bf0 tx=0x7f16b8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65d741e216f16209 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.978+0000 7f16aeffd640 1 -- 172.21.15.2:0/2364417422 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8004050 con 0x562140ca4f00 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.978+0000 7f16aeffd640 1 -- 172.21.15.2:0/2364417422 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f16b80035f0 con 0x562140ca4f00 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.978+0000 7f16aeffd640 1 -- 172.21.15.2:0/2364417422 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8019680 con 0x562140ca4f00 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.979+0000 7f16cdd81740 1 -- 172.21.15.2:0/2364417422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca4f00 msgr2=0x562140ca52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.979+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2364417422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca4f00 0x562140ca52e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009bf0 tx=0x7f16b8011810 comp rx=0 tx=0).stop 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.979+0000 7f16cdd81740 1 -- 172.21.15.2:0/2364417422 shutdown_connections 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.979+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2364417422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140cb2b60 0x562140cb4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.979+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2364417422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca5820 0x562140cb2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.979+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2364417422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca4f00 0x562140ca52e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.979+0000 7f16cdd81740 1 -- 172.21.15.2:0/2364417422 >> 172.21.15.2:0/2364417422 conn(0x562140c9f980 msgr2=0x562140c9fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.979+0000 7f16cdd81740 1 -- 172.21.15.2:0/2364417422 shutdown_connections 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.979+0000 7f16cdd81740 1 -- 172.21.15.2:0/2364417422 wait complete. 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.979+0000 7f16cdd81740 1 -- start start 2024-03-15T13:04:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.979+0000 7f16cdd81740 1 Processor -- start 2024-03-15T13:04:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.979+0000 7f16cdd81740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140ca4f00 0x562140cb24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16cdd81740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca5820 0x562140caf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16cdd81740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140cb2b60 0x562140cafda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16cdd81740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562140c0d020 con 0x562140cb2b60 2024-03-15T13:04:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16cdd81740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621408e0430 con 0x562140ca5820 2024-03-15T13:04:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16cdd81740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56214089ce00 con 0x562140ca4f00 2024-03-15T13:04:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16cdd81740 10 client.0 _ll_register_callbacks cb 0x562140bb4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16c966d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140cb2b60 0x562140cafda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16c8e6c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140ca4f00 0x562140cb24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16c966d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140cb2b60 0x562140cafda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41752/0 (socket says 172.21.15.2:41752) 2024-03-15T13:04:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca5820 0x562140caf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16c966d640 1 -- 172.21.15.2:0/2464674479 learned_addr learned my addr 172.21.15.2:0/2464674479 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16c8e6c640 1 -- 172.21.15.2:0/2464674479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca5820 msgr2=0x562140caf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16c8e6c640 1 --2- 172.21.15.2:0/2464674479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca5820 0x562140caf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16c8e6c640 1 -- 172.21.15.2:0/2464674479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140cb2b60 msgr2=0x562140cafda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16c8e6c640 1 --2- 172.21.15.2:0/2464674479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140cb2b60 0x562140cafda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16c8e6c640 1 -- 172.21.15.2:0/2464674479 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b8009850 con 0x562140ca4f00 2024-03-15T13:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16c966d640 1 --2- 172.21.15.2:0/2464674479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140cb2b60 0x562140cafda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16c8e6c640 1 --2- 172.21.15.2:0/2464674479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140ca4f00 0x562140cb24b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f16b801b040 tx=0x7f16b8009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdef9a11831c4d93 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16a3fff640 1 -- 172.21.15.2:0/2464674479 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b80054d0 con 0x562140ca4f00 2024-03-15T13:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16cdd81740 1 -- 172.21.15.2:0/2464674479 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56214090c7e0 con 0x562140ca4f00 2024-03-15T13:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16a3fff640 1 -- 172.21.15.2:0/2464674479 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f16b8005630 con 0x562140ca4f00 2024-03-15T13:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16a3fff640 1 -- 172.21.15.2:0/2464674479 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16b8025470 con 0x562140ca4f00 2024-03-15T13:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16cdd81740 10 client.27022 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.980+0000 7f16cdd81740 1 -- 172.21.15.2:0/2464674479 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562140c0b6a0 con 0x562140ca4f00 2024-03-15T13:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.981+0000 7f16a3fff640 1 -- 172.21.15.2:0/2464674479 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x56214090c7e0 con 0x562140ca4f00 2024-03-15T13:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.981+0000 7f16a3fff640 1 -- 172.21.15.2:0/2464674479 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f16b8021040 con 0x562140ca4f00 2024-03-15T13:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.981+0000 7f16cdd81740 10 client.27022 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:10.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.981+0000 7f16cdd81740 10 client.27022 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:04:10.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.981+0000 7f16cdd81740 20 client.27022 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:10.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.981+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2464674479 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x562140cc2e00 0x562140cc5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.981+0000 7f16cdd81740 4 client.27022 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:10.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.981+0000 7f16cdd81740 1 -- 172.21.15.2:0/2464674479 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562140949720 con 0x562140cc2e00 2024-03-15T13:04:10.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.981+0000 7f16c3fff640 1 --2- 172.21.15.2:0/2464674479 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x562140cc2e00 0x562140cc5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.981+0000 7f16c3fff640 1 --2- 172.21.15.2:0/2464674479 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x562140cc2e00 0x562140cc5220 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6dbb18b8f5bd8947 server_cookie=803f73798a70dcb7 in_seq=0 out_seq=0 2024-03-15T13:04:10.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.981+0000 7f16a3fff640 10 client.27022 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:04:10.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.982+0000 7f16a3fff640 1 -- 172.21.15.2:0/2464674479 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562140949720 con 0x562140cc2e00 2024-03-15T13:04:10.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.982+0000 7f16a3fff640 10 client.27022 handle_command_reply: tid=0 2024-03-15T13:04:10.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.982+0000 7f16cdd81740 1 client.27022 shutdown 2024-03-15T13:04:10.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.982+0000 7f16cdd81740 1 -- 172.21.15.2:0/2464674479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140ca4f00 msgr2=0x562140cb24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.982+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2464674479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140ca4f00 0x562140cb24b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f16b801b040 tx=0x7f16b8009fd0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.982+0000 7f16cdd81740 1 -- 172.21.15.2:0/2464674479 shutdown_connections 2024-03-15T13:04:10.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.982+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2464674479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140cb2b60 0x562140cafda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.982+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2464674479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca5820 0x562140caf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.982+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2464674479 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x562140cc2e00 0x562140cc5220 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.982+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2464674479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140ca4f00 0x562140cb24b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.982+0000 7f16cdd81740 1 -- 172.21.15.2:0/2464674479 >> 172.21.15.2:0/2464674479 conn(0x562140c9f980 msgr2=0x562140ca16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.982+0000 7f16a3fff640 0 client.27022 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:04:10.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.982+0000 7f16cdd81740 1 -- 172.21.15.2:0/2464674479 shutdown_connections 2024-03-15T13:04:10.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.982+0000 7f16cdd81740 1 -- 172.21.15.2:0/2464674479 wait complete. 2024-03-15T13:04:10.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.982+0000 7f16cdd81740 20 client.27022 trim_cache size 0 max 16384 2024-03-15T13:04:10.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16cdd81740 1 Processor -- start 2024-03-15T13:04:10.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16cdd81740 1 -- start start 2024-03-15T13:04:10.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16cdd81740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca4f00 0x562140cb1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16cdd81740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140ca5820 0x562140cb24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16cdd81740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140cb2b60 0x562140cbdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16cdd81740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562140b92130 con 0x562140ca4f00 2024-03-15T13:04:10.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16cdd81740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562140cc1540 con 0x562140cb2b60 2024-03-15T13:04:10.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16cdd81740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562140cbf800 con 0x562140ca5820 2024-03-15T13:04:10.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16c8e6c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca4f00 0x562140cb1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16c8e6c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca4f00 0x562140cb1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41764/0 (socket says 172.21.15.2:41764) 2024-03-15T13:04:10.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16c8e6c640 1 -- 172.21.15.2:0/2742419779 learned_addr learned my addr 172.21.15.2:0/2742419779 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16c966d640 1 --2- 172.21.15.2:0/2742419779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140cb2b60 0x562140cbdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16c3fff640 1 --2- 172.21.15.2:0/2742419779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140ca5820 0x562140cb24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16c8e6c640 1 -- 172.21.15.2:0/2742419779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140ca5820 msgr2=0x562140cb24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16c8e6c640 1 --2- 172.21.15.2:0/2742419779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140ca5820 0x562140cb24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16c8e6c640 1 -- 172.21.15.2:0/2742419779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140cb2b60 msgr2=0x562140cbdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16c8e6c640 1 --2- 172.21.15.2:0/2742419779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140cb2b60 0x562140cbdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.995+0000 7f16c8e6c640 1 -- 172.21.15.2:0/2742419779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562140ba7c20 con 0x562140ca4f00 2024-03-15T13:04:10.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.996+0000 7f16c8e6c640 1 --2- 172.21.15.2:0/2742419779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca4f00 0x562140cb1f70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009820 tx=0x7f16b8009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7e455eba00bde0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.996+0000 7f16ae7fc640 1 -- 172.21.15.2:0/2742419779 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8003b70 con 0x562140ca4f00 2024-03-15T13:04:10.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.996+0000 7f16ae7fc640 1 -- 172.21.15.2:0/2742419779 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f16b8023980 con 0x562140ca4f00 2024-03-15T13:04:10.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.996+0000 7f16ae7fc640 1 -- 172.21.15.2:0/2742419779 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b80259f0 con 0x562140ca4f00 2024-03-15T13:04:10.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.996+0000 7f16cdd81740 1 -- 172.21.15.2:0/2742419779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca4f00 msgr2=0x562140cb1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.996+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2742419779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca4f00 0x562140cb1f70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009820 tx=0x7f16b8009a70 comp rx=0 tx=0).stop 2024-03-15T13:04:10.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.996+0000 7f16cdd81740 1 -- 172.21.15.2:0/2742419779 shutdown_connections 2024-03-15T13:04:10.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.996+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2742419779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140cb2b60 0x562140cbdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.996+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2742419779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140ca5820 0x562140cb24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.996+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2742419779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca4f00 0x562140cb1f70 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.996+0000 7f16cdd81740 1 -- 172.21.15.2:0/2742419779 >> 172.21.15.2:0/2742419779 conn(0x562140c9f980 msgr2=0x562140caae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.996+0000 7f16cdd81740 1 -- 172.21.15.2:0/2742419779 shutdown_connections 2024-03-15T13:04:10.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.996+0000 7f16cdd81740 1 -- 172.21.15.2:0/2742419779 wait complete. 2024-03-15T13:04:10.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16cdd81740 1 -- start start 2024-03-15T13:04:10.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16cdd81740 1 Processor -- start 2024-03-15T13:04:10.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16cdd81740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca4f00 0x562140cb6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16cdd81740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca5820 0x562140cb6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16cdd81740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140cb2b60 0x562140d16690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16cdd81740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562140d16c20 con 0x562140ca5820 2024-03-15T13:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16cdd81740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562140d16d90 con 0x562140ca4f00 2024-03-15T13:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16cdd81740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562140d16f50 con 0x562140cb2b60 2024-03-15T13:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16cdd81740 10 client.0 _ll_register_callbacks cb 0x7f16b0004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca5820 0x562140cb6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16c8e6c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca4f00 0x562140cb6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca5820 0x562140cb6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41780/0 (socket says 172.21.15.2:41780) 2024-03-15T13:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16c3fff640 1 -- 172.21.15.2:0/2761058869 learned_addr learned my addr 172.21.15.2:0/2761058869 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16c966d640 1 --2- 172.21.15.2:0/2761058869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140cb2b60 0x562140d16690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16c3fff640 1 -- 172.21.15.2:0/2761058869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140cb2b60 msgr2=0x562140d16690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16c3fff640 1 --2- 172.21.15.2:0/2761058869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140cb2b60 0x562140d16690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16c3fff640 1 -- 172.21.15.2:0/2761058869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca4f00 msgr2=0x562140cb6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16c3fff640 1 --2- 172.21.15.2:0/2761058869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca4f00 0x562140cb6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16c3fff640 1 -- 172.21.15.2:0/2761058869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562140c0b6a0 con 0x562140ca5820 2024-03-15T13:04:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.997+0000 7f16c8e6c640 1 --2- 172.21.15.2:0/2761058869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca4f00 0x562140cb6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16c3fff640 1 --2- 172.21.15.2:0/2761058869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca5820 0x562140cb6f90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f16b00bca40 tx=0x7f16b002bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7801c179983cea server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16a3fff640 1 -- 172.21.15.2:0/2761058869 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b00c0c00 con 0x562140ca5820 2024-03-15T13:04:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16a3fff640 1 -- 172.21.15.2:0/2761058869 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f16b00c5250 con 0x562140ca5820 2024-03-15T13:04:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16cdd81740 1 -- 172.21.15.2:0/2761058869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562140cafc10 con 0x562140ca5820 2024-03-15T13:04:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16a3fff640 1 -- 172.21.15.2:0/2761058869 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16b0007d60 con 0x562140ca5820 2024-03-15T13:04:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16cdd81740 10 client.17235 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16cdd81740 1 -- 172.21.15.2:0/2761058869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562140bcd8c0 con 0x562140ca5820 2024-03-15T13:04:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16a3fff640 1 -- 172.21.15.2:0/2761058869 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x562140cafc10 con 0x562140ca5820 2024-03-15T13:04:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16a3fff640 1 -- 172.21.15.2:0/2761058869 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f16b003c070 con 0x562140ca5820 2024-03-15T13:04:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16cdd81740 10 client.17235 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16cdd81740 10 client.17235 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:04:10.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16cdd81740 20 client.17235 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2761058869 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x562140d1f770 0x562140d21b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16cdd81740 4 client.17235 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:04:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.998+0000 7f16cdd81740 1 -- 172.21.15.2:0/2761058869 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f16b80035f0 con 0x562140d1f770 2024-03-15T13:04:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16c8e6c640 1 --2- 172.21.15.2:0/2761058869 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x562140d1f770 0x562140d21b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16c8e6c640 1 --2- 172.21.15.2:0/2761058869 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x562140d1f770 0x562140d21b90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c8bd1834ca0f3b12 server_cookie=26a37f78557deb9e in_seq=0 out_seq=0 2024-03-15T13:04:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16a3fff640 10 client.17235 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:04:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16a3fff640 1 -- 172.21.15.2:0/2761058869 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f16b80035f0 con 0x562140d1f770 2024-03-15T13:04:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16a3fff640 10 client.17235 handle_command_reply: tid=0 2024-03-15T13:04:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16cdd81740 1 client.17235 shutdown 2024-03-15T13:04:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16cdd81740 1 -- 172.21.15.2:0/2761058869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca5820 msgr2=0x562140cb6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2761058869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca5820 0x562140cb6f90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f16b00bca40 tx=0x7f16b002bcd0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16cdd81740 1 -- 172.21.15.2:0/2761058869 shutdown_connections 2024-03-15T13:04:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2761058869 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x562140d1f770 0x562140d21b90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2761058869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562140cb2b60 0x562140d16690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16a3fff640 0 client.17235 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:04:10.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2761058869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562140ca5820 0x562140cb6f90 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16cdd81740 1 --2- 172.21.15.2:0/2761058869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562140ca4f00 0x562140cb6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16cdd81740 1 -- 172.21.15.2:0/2761058869 >> 172.21.15.2:0/2761058869 conn(0x562140c9f980 msgr2=0x562140caa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16cdd81740 1 -- 172.21.15.2:0/2761058869 shutdown_connections 2024-03-15T13:04:10.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16cdd81740 1 -- 172.21.15.2:0/2761058869 wait complete. 2024-03-15T13:04:10.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:10.999+0000 7f16cdd81740 20 client.17235 trim_cache size 0 max 16384 2024-03-15T13:04:10.991 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:04:10.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.000+0000 7f16cb0d0640 1 -- 172.21.15.2:0/1748102834 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f16b40a1ba0 msgr2=0x7f16b40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.000+0000 7f16cb0d0640 1 --2- 172.21.15.2:0/1748102834 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f16b40a1ba0 0x7f16b40a4060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009fd0 tx=0x7f16bc009d80 comp rx=0 tx=0).stop 2024-03-15T13:04:10.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.000+0000 7f16cb0d0640 1 -- 172.21.15.2:0/1748102834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16c4103080 msgr2=0x7f16c419ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:10.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.000+0000 7f16cb0d0640 1 --2- 172.21.15.2:0/1748102834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16c4103080 0x7f16c419ddb0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009230 tx=0x7f16b002cb60 comp rx=0 tx=0).stop 2024-03-15T13:04:10.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.000+0000 7f16cb0d0640 1 -- 172.21.15.2:0/1748102834 shutdown_connections 2024-03-15T13:04:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.000+0000 7f16cb0d0640 1 --2- 172.21.15.2:0/1748102834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16c410bb30 0x7f16c4197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.000+0000 7f16cb0d0640 1 --2- 172.21.15.2:0/1748102834 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f16b40a1ba0 0x7f16b40a4060 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.000+0000 7f16cb0d0640 1 --2- 172.21.15.2:0/1748102834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16c4103080 0x7f16c419ddb0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.000+0000 7f16cb0d0640 1 --2- 172.21.15.2:0/1748102834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16c4102760 0x7f16c419d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.000+0000 7f16cb0d0640 1 -- 172.21.15.2:0/1748102834 >> 172.21.15.2:0/1748102834 conn(0x7f16c4077550 msgr2=0x7f16c40fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.000+0000 7f16cb0d0640 1 -- 172.21.15.2:0/1748102834 shutdown_connections 2024-03-15T13:04:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.000+0000 7f16cb0d0640 1 -- 172.21.15.2:0/1748102834 wait complete. 2024-03-15T13:04:10.996 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:04:10.996 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:04:11.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:10 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:11.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.116+0000 7f5e122c3640 1 -- 172.21.15.2:0/573223778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c1005d0 msgr2=0x7f5e0c1009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.116+0000 7f5e122c3640 1 --2- 172.21.15.2:0/573223778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c1005d0 0x7f5e0c1009b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009890 tx=0x7f5dfc02bb70 comp rx=0 tx=0).stop 2024-03-15T13:04:11.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.116+0000 7f5e122c3640 1 -- 172.21.15.2:0/573223778 shutdown_connections 2024-03-15T13:04:11.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.116+0000 7f5e122c3640 1 --2- 172.21.15.2:0/573223778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e0c1056a0 0x7f5e0c10d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.116+0000 7f5e122c3640 1 --2- 172.21.15.2:0/573223778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c100f80 0x7f5e0c104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.116+0000 7f5e122c3640 1 --2- 172.21.15.2:0/573223778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c1005d0 0x7f5e0c1009b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.116+0000 7f5e122c3640 1 -- 172.21.15.2:0/573223778 >> 172.21.15.2:0/573223778 conn(0x7f5e0c0774f0 msgr2=0x7f5e0c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.116+0000 7f5e122c3640 1 -- 172.21.15.2:0/573223778 shutdown_connections 2024-03-15T13:04:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.116+0000 7f5e122c3640 1 -- 172.21.15.2:0/573223778 wait complete. 2024-03-15T13:04:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.116+0000 7f5e122c3640 1 Processor -- start 2024-03-15T13:04:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.116+0000 7f5e122c3640 1 -- start start 2024-03-15T13:04:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e122c3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c1005d0 0x7f5e0c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e122c3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c1056a0 0x7f5e0c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e122c3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e0c19e610 0x7f5e0c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e122c3640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c19f000 con 0x7f5e0c19e610 2024-03-15T13:04:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e122c3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c19f170 con 0x7f5e0c1056a0 2024-03-15T13:04:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e122c3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c19f2e0 con 0x7f5e0c1005d0 2024-03-15T13:04:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e10860640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e0c19e610 0x7f5e0c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e10860640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e0c19e610 0x7f5e0c19eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41784/0 (socket says 172.21.15.2:41784) 2024-03-15T13:04:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e10860640 1 -- 172.21.15.2:0/1904967656 learned_addr learned my addr 172.21.15.2:0/1904967656 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e0b7fe640 1 --2- 172.21.15.2:0/1904967656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c1056a0 0x7f5e0c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e0bfff640 1 --2- 172.21.15.2:0/1904967656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c1005d0 0x7f5e0c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e10860640 1 -- 172.21.15.2:0/1904967656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c1005d0 msgr2=0x7f5e0c199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e10860640 1 --2- 172.21.15.2:0/1904967656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c1005d0 0x7f5e0c199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e10860640 1 -- 172.21.15.2:0/1904967656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c1056a0 msgr2=0x7f5e0c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e10860640 1 --2- 172.21.15.2:0/1904967656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c1056a0 0x7f5e0c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e10860640 1 -- 172.21.15.2:0/1904967656 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e00009490 con 0x7f5e0c19e610 2024-03-15T13:04:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e10860640 1 --2- 172.21.15.2:0/1904967656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e0c19e610 0x7f5e0c19eac0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400efc0 tx=0x7f5e0400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e097fa640 1 -- 172.21.15.2:0/1904967656 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e04003970 con 0x7f5e0c19e610 2024-03-15T13:04:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e122c3640 1 -- 172.21.15.2:0/1904967656 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e04009900 con 0x7f5e0c19e610 2024-03-15T13:04:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e097fa640 1 -- 172.21.15.2:0/1904967656 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5e04004020 con 0x7f5e0c19e610 2024-03-15T13:04:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e097fa640 1 -- 172.21.15.2:0/1904967656 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0400b9b0 con 0x7f5e0c19e610 2024-03-15T13:04:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.117+0000 7f5e122c3640 1 -- 172.21.15.2:0/1904967656 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dfc009490 con 0x7f5e0c19e610 2024-03-15T13:04:11.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e097fa640 1 -- 172.21.15.2:0/1904967656 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f5e0400a040 con 0x7f5e0c19e610 2024-03-15T13:04:11.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e14f7d740 1 Processor -- start 2024-03-15T13:04:11.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e14f7d740 1 -- start start 2024-03-15T13:04:11.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e14f7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 0x557c21f33630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e14f7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33b70 0x557c21f3c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e14f7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 0x557c21f3eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e14f7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21eee960 con 0x557c21f3c750 2024-03-15T13:04:11.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e14f7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21ea2310 con 0x557c21f33250 2024-03-15T13:04:11.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e14f7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21e9f840 con 0x557c21f33b70 2024-03-15T13:04:11.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e097fa640 1 --2- 172.21.15.2:0/1904967656 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5df40a1ba0 0x7f5df40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 0x557c21f33630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 0x557c21f33630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:32798/0 (socket says 172.21.15.2:32798) 2024-03-15T13:04:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e0b7fe640 1 -- 172.21.15.2:0/261829551 learned_addr learned my addr 172.21.15.2:0/261829551 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e0bfff640 1 --2- 172.21.15.2:0/261829551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33b70 0x557c21f3c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e0b7fe640 1 --2- 172.21.15.2:0/261829551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 0x557c21f3eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e10860640 1 --2- 172.21.15.2:0/1904967656 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5df40a1ba0 0x7f5df40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e0b7fe640 1 -- 172.21.15.2:0/261829551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33b70 msgr2=0x557c21f3c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e097fa640 1 -- 172.21.15.2:0/1904967656 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5e040cdaa0 con 0x7f5e0c19e610 2024-03-15T13:04:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e0b7fe640 1 --2- 172.21.15.2:0/261829551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33b70 0x557c21f3c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e0b7fe640 1 -- 172.21.15.2:0/261829551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 msgr2=0x557c21f3eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e0b7fe640 1 --2- 172.21.15.2:0/261829551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 0x557c21f3eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.120+0000 7f5e0b7fe640 1 -- 172.21.15.2:0/261829551 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c21e5d060 con 0x557c21f33250 2024-03-15T13:04:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.121+0000 7f5e0bfff640 1 --2- 172.21.15.2:0/261829551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33b70 0x557c21f3c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.121+0000 7f5e0b7fe640 1 --2- 172.21.15.2:0/261829551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 0x557c21f33630 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5e000058c0 tx=0x7f5e00009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe157e2cf1cb56c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.121+0000 7f5e10860640 1 --2- 172.21.15.2:0/1904967656 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5df40a1ba0 0x7f5df40a4060 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5e040967e0 tx=0x7f5e0406d630 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.121+0000 7f5df27fc640 1 -- 172.21.15.2:0/261829551 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00011020 con 0x557c21f33250 2024-03-15T13:04:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.121+0000 7f5df27fc640 1 -- 172.21.15.2:0/261829551 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5e00011180 con 0x557c21f33250 2024-03-15T13:04:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.121+0000 7f5df27fc640 1 -- 172.21.15.2:0/261829551 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e000035f0 con 0x557c21f33250 2024-03-15T13:04:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.122+0000 7f5e14f7d740 1 -- 172.21.15.2:0/261829551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 msgr2=0x557c21f33630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.122+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/261829551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 0x557c21f33630 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5e000058c0 tx=0x7f5e00009370 comp rx=0 tx=0).stop 2024-03-15T13:04:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.122+0000 7f5e14f7d740 1 -- 172.21.15.2:0/261829551 shutdown_connections 2024-03-15T13:04:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.122+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/261829551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 0x557c21f3eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.122+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/261829551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33b70 0x557c21f3c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.122+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/261829551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 0x557c21f33630 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.122+0000 7f5e14f7d740 1 -- 172.21.15.2:0/261829551 >> 172.21.15.2:0/261829551 conn(0x557c21f2d980 msgr2=0x557c21f2dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.122+0000 7f5e14f7d740 1 -- 172.21.15.2:0/261829551 shutdown_connections 2024-03-15T13:04:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.122+0000 7f5e14f7d740 1 -- 172.21.15.2:0/261829551 wait complete. 2024-03-15T13:04:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.122+0000 7f5e14f7d740 1 -- start start 2024-03-15T13:04:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.122+0000 7f5e14f7d740 1 Processor -- start 2024-03-15T13:04:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.122+0000 7f5e14f7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33250 0x557c21f3c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.122+0000 7f5e14f7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33b70 0x557c21f39470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e14f7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 0x557c21f399f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e14f7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21e9b020 con 0x557c21f3c750 2024-03-15T13:04:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e14f7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21b6e430 con 0x557c21f33b70 2024-03-15T13:04:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e14f7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21b2ae00 con 0x557c21f33250 2024-03-15T13:04:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e14f7d740 10 client.0 _ll_register_callbacks cb 0x557c21e42cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 0x557c21f399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e0bfff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33b70 0x557c21f39470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e0bfff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33b70 0x557c21f39470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:32812/0 (socket says 172.21.15.2:32812) 2024-03-15T13:04:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33250 0x557c21f3c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e0bfff640 1 -- 172.21.15.2:0/3232356189 learned_addr learned my addr 172.21.15.2:0/3232356189 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e0b7fe640 1 -- 172.21.15.2:0/3232356189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33b70 msgr2=0x557c21f39470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e0b7fe640 1 --2- 172.21.15.2:0/3232356189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33b70 0x557c21f39470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e0b7fe640 1 -- 172.21.15.2:0/3232356189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 msgr2=0x557c21f399f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e0b7fe640 1 --2- 172.21.15.2:0/3232356189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 0x557c21f399f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e0b7fe640 1 -- 172.21.15.2:0/3232356189 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e00019040 con 0x557c21f33250 2024-03-15T13:04:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e0bfff640 1 --2- 172.21.15.2:0/3232356189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33b70 0x557c21f39470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e0b7fe640 1 --2- 172.21.15.2:0/3232356189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33250 0x557c21f3c0e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009870 tx=0x7f5e00005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f8fee49068e147 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5deb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:04:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5deb7fe640 1 -- 172.21.15.2:0/3232356189 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0000fa80 con 0x557c21f33250 2024-03-15T13:04:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5deb7fe640 1 -- 172.21.15.2:0/3232356189 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5e000035f0 con 0x557c21f33250 2024-03-15T13:04:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.123+0000 7f5e14f7d740 1 -- 172.21.15.2:0/3232356189 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c21b9a7e0 con 0x557c21f33250 2024-03-15T13:04:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.124+0000 7f5deb7fe640 1 -- 172.21.15.2:0/3232356189 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e0000fbe0 con 0x557c21f33250 2024-03-15T13:04:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.124+0000 7f5e14f7d740 10 client.27040 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.124+0000 7f5e14f7d740 1 -- 172.21.15.2:0/3232356189 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x557c21e996a0 con 0x557c21f33250 2024-03-15T13:04:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.124+0000 7f5deb7fe640 1 -- 172.21.15.2:0/3232356189 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x557c21b9a7e0 con 0x557c21f33250 2024-03-15T13:04:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.124+0000 7f5deb7fe640 1 -- 172.21.15.2:0/3232356189 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f5e0001d070 con 0x557c21f33250 2024-03-15T13:04:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.124+0000 7f5e14f7d740 10 client.27040 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.124+0000 7f5e14f7d740 10 client.27040 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:04:11.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.124+0000 7f5e14f7d740 20 client.27040 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.124+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/3232356189 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557c21f4c7f0 0x557c21f4ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.124+0000 7f5e14f7d740 4 client.27040 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.124+0000 7f5e14f7d740 1 -- 172.21.15.2:0/3232356189 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c21bd7720 con 0x557c21f4c7f0 2024-03-15T13:04:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.124+0000 7f5e0bfff640 1 --2- 172.21.15.2:0/3232356189 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557c21f4c7f0 0x557c21f4ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.124+0000 7f5e0bfff640 1 --2- 172.21.15.2:0/3232356189 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557c21f4c7f0 0x557c21f4ec10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac9bddb727857a24 server_cookie=10dd35c59275ee8a in_seq=0 out_seq=0 2024-03-15T13:04:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.124+0000 7f5deb7fe640 10 client.27040 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:04:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.125+0000 7f5deb7fe640 1 -- 172.21.15.2:0/3232356189 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557c21bd7720 con 0x557c21f4c7f0 2024-03-15T13:04:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.125+0000 7f5deb7fe640 10 client.27040 handle_command_reply: tid=0 2024-03-15T13:04:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.126+0000 7f5e14f7d740 1 client.27040 shutdown 2024-03-15T13:04:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.126+0000 7f5e14f7d740 1 -- 172.21.15.2:0/3232356189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33250 msgr2=0x557c21f3c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.126+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/3232356189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33250 0x557c21f3c0e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009870 tx=0x7f5e00005f90 comp rx=0 tx=0).stop 2024-03-15T13:04:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.126+0000 7f5e14f7d740 1 -- 172.21.15.2:0/3232356189 shutdown_connections 2024-03-15T13:04:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.126+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/3232356189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 0x557c21f399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.126+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/3232356189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33b70 0x557c21f39470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.126+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/3232356189 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557c21f4c7f0 0x557c21f4ec10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.126+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/3232356189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33250 0x557c21f3c0e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.126+0000 7f5deb7fe640 0 client.27040 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:04:11.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.126+0000 7f5e14f7d740 1 -- 172.21.15.2:0/3232356189 >> 172.21.15.2:0/3232356189 conn(0x557c21f2d980 msgr2=0x557c21f2f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.126+0000 7f5e14f7d740 1 -- 172.21.15.2:0/3232356189 shutdown_connections 2024-03-15T13:04:11.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.126+0000 7f5e14f7d740 1 -- 172.21.15.2:0/3232356189 wait complete. 2024-03-15T13:04:11.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.126+0000 7f5e14f7d740 20 client.27040 trim_cache size 0 max 16384 2024-03-15T13:04:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.138+0000 7f5e14f7d740 1 Processor -- start 2024-03-15T13:04:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.138+0000 7f5e14f7d740 1 -- start start 2024-03-15T13:04:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.138+0000 7f5e14f7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 0x557c21f2cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.138+0000 7f5e14f7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f33b70 0x557c21f3bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.138+0000 7f5e14f7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f3c750 0x557c21f3c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.138+0000 7f5e14f7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21e20130 con 0x557c21f33b70 2024-03-15T13:04:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.138+0000 7f5e14f7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21bd7720 con 0x557c21f33250 2024-03-15T13:04:11.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.138+0000 7f5e14f7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21f1fa40 con 0x557c21f3c750 2024-03-15T13:04:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.139+0000 7f5e0bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f33b70 0x557c21f3bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.139+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 0x557c21f2cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.139+0000 7f5e0bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f33b70 0x557c21f3bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41808/0 (socket says 172.21.15.2:41808) 2024-03-15T13:04:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.139+0000 7f5e0bfff640 1 -- 172.21.15.2:0/1274772365 learned_addr learned my addr 172.21.15.2:0/1274772365 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.139+0000 7f5e0b7fe640 1 --2- 172.21.15.2:0/1274772365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f3c750 0x557c21f3c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.139+0000 7f5e0bfff640 1 -- 172.21.15.2:0/1274772365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f3c750 msgr2=0x557c21f3c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.139+0000 7f5e0bfff640 1 --2- 172.21.15.2:0/1274772365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f3c750 0x557c21f3c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.139+0000 7f5e0bfff640 1 -- 172.21.15.2:0/1274772365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 msgr2=0x557c21f2cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.139+0000 7f5e0bfff640 1 --2- 172.21.15.2:0/1274772365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 0x557c21f2cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.139+0000 7f5e0bfff640 1 -- 172.21.15.2:0/1274772365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c21e35c20 con 0x557c21f33b70 2024-03-15T13:04:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.139+0000 7f5e0b7fe640 1 --2- 172.21.15.2:0/1274772365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 0x557c21f2cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.139+0000 7f5e0bfff640 1 --2- 172.21.15.2:0/1274772365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f33b70 0x557c21f3bba0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc037040 tx=0x7f5dfc0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c3ceedab7dfe0f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.139+0000 7f5df1ffb640 1 -- 172.21.15.2:0/1274772365 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc034440 con 0x557c21f33b70 2024-03-15T13:04:11.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.139+0000 7f5df1ffb640 1 -- 172.21.15.2:0/1274772365 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5dfc02c510 con 0x557c21f33b70 2024-03-15T13:04:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.139+0000 7f5df1ffb640 1 -- 172.21.15.2:0/1274772365 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc032860 con 0x557c21f33b70 2024-03-15T13:04:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.140+0000 7f5e14f7d740 1 -- 172.21.15.2:0/1274772365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f33b70 msgr2=0x557c21f3bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.140+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/1274772365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f33b70 0x557c21f3bba0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc037040 tx=0x7f5dfc0058d0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.140+0000 7f5e14f7d740 1 -- 172.21.15.2:0/1274772365 shutdown_connections 2024-03-15T13:04:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.140+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/1274772365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f3c750 0x557c21f3c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.140+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/1274772365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f33b70 0x557c21f3bba0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.140+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/1274772365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 0x557c21f2cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.140+0000 7f5e14f7d740 1 -- 172.21.15.2:0/1274772365 >> 172.21.15.2:0/1274772365 conn(0x557c21f2d980 msgr2=0x557c21f347e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.140+0000 7f5e14f7d740 1 -- 172.21.15.2:0/1274772365 shutdown_connections 2024-03-15T13:04:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.140+0000 7f5e14f7d740 1 -- 172.21.15.2:0/1274772365 wait complete. 2024-03-15T13:04:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.140+0000 7f5e14f7d740 1 -- start start 2024-03-15T13:04:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.140+0000 7f5e14f7d740 1 Processor -- start 2024-03-15T13:04:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e14f7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 0x557c21f43d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e14f7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33b70 0x557c21f44240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e14f7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 0x557c21f401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e14f7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21f40880 con 0x557c21f3c750 2024-03-15T13:04:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e14f7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21f409f0 con 0x557c21f33250 2024-03-15T13:04:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e14f7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c21f40bb0 con 0x557c21f33b70 2024-03-15T13:04:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e14f7d740 10 client.0 _ll_register_callbacks cb 0x7f5e00011480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 0x557c21f401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e0bfff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33b70 0x557c21f44240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 0x557c21f401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41810/0 (socket says 172.21.15.2:41810) 2024-03-15T13:04:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e0b7fe640 1 -- 172.21.15.2:0/675026120 learned_addr learned my addr 172.21.15.2:0/675026120 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e0b7fe640 1 --2- 172.21.15.2:0/675026120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 0x557c21f43d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e0b7fe640 1 -- 172.21.15.2:0/675026120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33b70 msgr2=0x557c21f44240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e0b7fe640 1 --2- 172.21.15.2:0/675026120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33b70 0x557c21f44240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e0b7fe640 1 -- 172.21.15.2:0/675026120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 msgr2=0x557c21f43d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e0b7fe640 1 --2- 172.21.15.2:0/675026120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 0x557c21f43d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e0b7fe640 1 -- 172.21.15.2:0/675026120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c21e996a0 con 0x557c21f3c750 2024-03-15T13:04:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e0b7fe640 1 --2- 172.21.15.2:0/675026120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 0x557c21f401d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009fd0 tx=0x7f5e000112f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab93ce2a3b63d48 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5deb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5deb7fe640 1 -- 172.21.15.2:0/675026120 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00011530 con 0x557c21f3c750 2024-03-15T13:04:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5deb7fe640 1 -- 172.21.15.2:0/675026120 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5e00043060 con 0x557c21f3c750 2024-03-15T13:04:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e14f7d740 1 -- 172.21.15.2:0/675026120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e00011180 con 0x557c21f3c750 2024-03-15T13:04:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5deb7fe640 1 -- 172.21.15.2:0/675026120 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e0001f440 con 0x557c21f3c750 2024-03-15T13:04:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e14f7d740 10 client.17253 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.141+0000 7f5e14f7d740 1 -- 172.21.15.2:0/675026120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f5dfc030070 con 0x557c21f3c750 2024-03-15T13:04:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.142+0000 7f5deb7fe640 1 -- 172.21.15.2:0/675026120 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5e00011180 con 0x557c21f3c750 2024-03-15T13:04:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.142+0000 7f5deb7fe640 1 -- 172.21.15.2:0/675026120 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f5e0001d040 con 0x557c21f3c750 2024-03-15T13:04:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.142+0000 7f5e14f7d740 10 client.17253 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.142+0000 7f5e14f7d740 10 client.17253 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:04:11.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.142+0000 7f5e14f7d740 20 client.17253 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.142+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/675026120 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557c21fa8d60 0x557c21fab150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.142+0000 7f5e14f7d740 4 client.17253 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:04:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.142+0000 7f5e14f7d740 1 -- 172.21.15.2:0/675026120 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557c21c26890 con 0x557c21fa8d60 2024-03-15T13:04:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.142+0000 7f5e0bfff640 1 --2- 172.21.15.2:0/675026120 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557c21fa8d60 0x557c21fab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.142+0000 7f5deb7fe640 10 client.17253 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:04:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.142+0000 7f5e0bfff640 1 --2- 172.21.15.2:0/675026120 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557c21fa8d60 0x557c21fab150 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8df459c21292953b server_cookie=68ebb82ff36581be in_seq=0 out_seq=0 2024-03-15T13:04:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5deb7fe640 1 -- 172.21.15.2:0/675026120 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557c21c26890 con 0x557c21fa8d60 2024-03-15T13:04:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5deb7fe640 10 client.17253 handle_command_reply: tid=0 2024-03-15T13:04:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e14f7d740 1 client.17253 shutdown 2024-03-15T13:04:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e14f7d740 1 -- 172.21.15.2:0/675026120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 msgr2=0x557c21f401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/675026120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 0x557c21f401d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009fd0 tx=0x7f5e000112f0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e14f7d740 1 -- 172.21.15.2:0/675026120 shutdown_connections 2024-03-15T13:04:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/675026120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557c21f3c750 0x557c21f401d0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/675026120 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557c21fa8d60 0x557c21fab150 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/675026120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557c21f33b70 0x557c21f44240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5deb7fe640 0 client.17253 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:04:11.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e14f7d740 1 --2- 172.21.15.2:0/675026120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c21f33250 0x557c21f43d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e14f7d740 1 -- 172.21.15.2:0/675026120 >> 172.21.15.2:0/675026120 conn(0x557c21f2d980 msgr2=0x557c21f534d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e14f7d740 1 -- 172.21.15.2:0/675026120 shutdown_connections 2024-03-15T13:04:11.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e14f7d740 1 -- 172.21.15.2:0/675026120 wait complete. 2024-03-15T13:04:11.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e14f7d740 20 client.17253 trim_cache size 0 max 16384 2024-03-15T13:04:11.135 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:04:11.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e122c3640 1 -- 172.21.15.2:0/1904967656 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5df40a1ba0 msgr2=0x7f5df40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e122c3640 1 --2- 172.21.15.2:0/1904967656 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5df40a1ba0 0x7f5df40a4060 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5e040967e0 tx=0x7f5e0406d630 comp rx=0 tx=0).stop 2024-03-15T13:04:11.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e122c3640 1 -- 172.21.15.2:0/1904967656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e0c19e610 msgr2=0x7f5e0c19eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.143+0000 7f5e122c3640 1 --2- 172.21.15.2:0/1904967656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e0c19e610 0x7f5e0c19eac0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5e0400efc0 tx=0x7f5e0400c800 comp rx=0 tx=0).stop 2024-03-15T13:04:11.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.144+0000 7f5e122c3640 1 -- 172.21.15.2:0/1904967656 shutdown_connections 2024-03-15T13:04:11.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.144+0000 7f5e122c3640 1 --2- 172.21.15.2:0/1904967656 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5df40a1ba0 0x7f5df40a4060 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.144+0000 7f5e122c3640 1 --2- 172.21.15.2:0/1904967656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e0c19e610 0x7f5e0c19eac0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.144+0000 7f5e122c3640 1 --2- 172.21.15.2:0/1904967656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c1056a0 0x7f5e0c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.144+0000 7f5e122c3640 1 --2- 172.21.15.2:0/1904967656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0c1005d0 0x7f5e0c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.144+0000 7f5e122c3640 1 -- 172.21.15.2:0/1904967656 >> 172.21.15.2:0/1904967656 conn(0x7f5e0c0774f0 msgr2=0x7f5e0c068830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.144+0000 7f5e122c3640 1 -- 172.21.15.2:0/1904967656 shutdown_connections 2024-03-15T13:04:11.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.144+0000 7f5e122c3640 1 -- 172.21.15.2:0/1904967656 wait complete. 2024-03-15T13:04:11.139 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:04:11.139 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:04:11.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.259+0000 7f204880e640 1 -- 172.21.15.2:0/3926280517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20440686f0 msgr2=0x7f2044068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.259+0000 7f204880e640 1 --2- 172.21.15.2:0/3926280517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20440686f0 0x7f2044068b70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f202c009900 tx=0x7f202c02bc30 comp rx=0 tx=0).stop 2024-03-15T13:04:11.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.259+0000 7f204880e640 1 -- 172.21.15.2:0/3926280517 shutdown_connections 2024-03-15T13:04:11.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.259+0000 7f204880e640 1 --2- 172.21.15.2:0/3926280517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20440690b0 0x7f204410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.259+0000 7f204880e640 1 --2- 172.21.15.2:0/3926280517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20440686f0 0x7f2044068b70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.260+0000 7f204880e640 1 --2- 172.21.15.2:0/3926280517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20441074f0 0x7f20441078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.260+0000 7f204880e640 1 -- 172.21.15.2:0/3926280517 >> 172.21.15.2:0/3926280517 conn(0x7f20440774f0 msgr2=0x7f2044077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.260+0000 7f204880e640 1 -- 172.21.15.2:0/3926280517 shutdown_connections 2024-03-15T13:04:11.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.260+0000 7f204880e640 1 -- 172.21.15.2:0/3926280517 wait complete. 2024-03-15T13:04:11.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.260+0000 7f204880e640 1 Processor -- start 2024-03-15T13:04:11.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.260+0000 7f204880e640 1 -- start start 2024-03-15T13:04:11.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.260+0000 7f204880e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20440686f0 0x7f204419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.260+0000 7f204880e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20440690b0 0x7f204419dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.260+0000 7f204880e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20441074f0 0x7f20441978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.260+0000 7f204880e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2044197f40 con 0x7f20440686f0 2024-03-15T13:04:11.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.260+0000 7f204880e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20441980b0 con 0x7f20440690b0 2024-03-15T13:04:11.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.260+0000 7f204880e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2044198270 con 0x7f20441074f0 2024-03-15T13:04:11.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f204259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20440686f0 0x7f204419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f204259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20440686f0 0x7f204419d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41836/0 (socket says 172.21.15.2:41836) 2024-03-15T13:04:11.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f204259c640 1 -- 172.21.15.2:0/1641124386 learned_addr learned my addr 172.21.15.2:0/1641124386 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:11.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f2041d9b640 1 --2- 172.21.15.2:0/1641124386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20440690b0 0x7f204419dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f2042d9d640 1 --2- 172.21.15.2:0/1641124386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20441074f0 0x7f20441978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f204259c640 1 -- 172.21.15.2:0/1641124386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20441074f0 msgr2=0x7f20441978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f204259c640 1 --2- 172.21.15.2:0/1641124386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20441074f0 0x7f20441978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f204259c640 1 -- 172.21.15.2:0/1641124386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20440690b0 msgr2=0x7f204419dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f204259c640 1 --2- 172.21.15.2:0/1641124386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20440690b0 0x7f204419dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f204259c640 1 -- 172.21.15.2:0/1641124386 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c009560 con 0x7f20440686f0 2024-03-15T13:04:11.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f204259c640 1 --2- 172.21.15.2:0/1641124386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20440686f0 0x7f204419d740 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f203c00e9f0 tx=0x7f203c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f203b7fe640 1 -- 172.21.15.2:0/1641124386 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c009720 con 0x7f20440686f0 2024-03-15T13:04:11.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f204880e640 1 -- 172.21.15.2:0/1641124386 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2044198550 con 0x7f20440686f0 2024-03-15T13:04:11.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f203b7fe640 1 -- 172.21.15.2:0/1641124386 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f203c00cd80 con 0x7f20440686f0 2024-03-15T13:04:11.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f203b7fe640 1 -- 172.21.15.2:0/1641124386 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c010690 con 0x7f20440686f0 2024-03-15T13:04:11.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.261+0000 7f204880e640 1 -- 172.21.15.2:0/1641124386 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20441a47e0 con 0x7f20440686f0 2024-03-15T13:04:11.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.263+0000 7f203b7fe640 1 -- 172.21.15.2:0/1641124386 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f203c004e90 con 0x7f20440686f0 2024-03-15T13:04:11.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f204b3ed740 1 Processor -- start 2024-03-15T13:04:11.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f204b3ed740 1 -- start start 2024-03-15T13:04:11.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f204b3ed740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37250 0x564d26c37630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f204b3ed740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37b70 0x564d26c40210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f204b3ed740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c42b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f203b7fe640 1 --2- 172.21.15.2:0/1641124386 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f20300a1ba0 0x7f20300a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f204b3ed740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d26bf2960 con 0x564d26c40750 2024-03-15T13:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f204b3ed740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d26ba6310 con 0x564d26c37b70 2024-03-15T13:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f204b3ed740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d26ba3840 con 0x564d26c37250 2024-03-15T13:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f2041d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c42b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f2041d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c42b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41844/0 (socket says 172.21.15.2:41844) 2024-03-15T13:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f204259c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37b70 0x564d26c40210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f2041d9b640 1 -- 172.21.15.2:0/1009571209 learned_addr learned my addr 172.21.15.2:0/1009571209 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f2042d9d640 1 --2- 172.21.15.2:0/1009571209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37250 0x564d26c37630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f2042d9d640 1 --2- 172.21.15.2:0/1641124386 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f20300a1ba0 0x7f20300a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f203b7fe640 1 -- 172.21.15.2:0/1641124386 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f203c014070 con 0x7f20440686f0 2024-03-15T13:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.264+0000 7f2041d9b640 1 -- 172.21.15.2:0/1009571209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37250 msgr2=0x564d26c37630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.265+0000 7f2041d9b640 1 --2- 172.21.15.2:0/1009571209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37250 0x564d26c37630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.265+0000 7f2041d9b640 1 -- 172.21.15.2:0/1009571209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37b70 msgr2=0x564d26c40210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.265+0000 7f2041d9b640 1 --2- 172.21.15.2:0/1009571209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37b70 0x564d26c40210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.265+0000 7f2041d9b640 1 -- 172.21.15.2:0/1009571209 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d26b61060 con 0x564d26c40750 2024-03-15T13:04:11.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.265+0000 7f204259c640 1 --2- 172.21.15.2:0/1009571209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37b70 0x564d26c40210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:11.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.265+0000 7f2042d9d640 1 --2- 172.21.15.2:0/1009571209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37250 0x564d26c37630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:11.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.265+0000 7f2041d9b640 1 --2- 172.21.15.2:0/1009571209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c42b40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f202c036040 tx=0x7f202c02c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2e22532cea16ed server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.265+0000 7f2042d9d640 1 --2- 172.21.15.2:0/1641124386 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f20300a1ba0 0x7f20300a4060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2034005fd0 tx=0x7f2034009500 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.265+0000 7f2038ff9640 1 -- 172.21.15.2:0/1009571209 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c004780 con 0x564d26c40750 2024-03-15T13:04:11.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.265+0000 7f2038ff9640 1 -- 172.21.15.2:0/1009571209 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f202c0048e0 con 0x564d26c40750 2024-03-15T13:04:11.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.265+0000 7f2038ff9640 1 -- 172.21.15.2:0/1009571209 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c003810 con 0x564d26c40750 2024-03-15T13:04:11.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 -- 172.21.15.2:0/1009571209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 msgr2=0x564d26c42b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 --2- 172.21.15.2:0/1009571209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c42b40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f202c036040 tx=0x7f202c02c6d0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 -- 172.21.15.2:0/1009571209 shutdown_connections 2024-03-15T13:04:11.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 --2- 172.21.15.2:0/1009571209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c42b40 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 --2- 172.21.15.2:0/1009571209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37b70 0x564d26c40210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 --2- 172.21.15.2:0/1009571209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37250 0x564d26c37630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 -- 172.21.15.2:0/1009571209 >> 172.21.15.2:0/1009571209 conn(0x564d26c31980 msgr2=0x564d26c31d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 -- 172.21.15.2:0/1009571209 shutdown_connections 2024-03-15T13:04:11.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 -- 172.21.15.2:0/1009571209 wait complete. 2024-03-15T13:04:11.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 -- start start 2024-03-15T13:04:11.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 Processor -- start 2024-03-15T13:04:11.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37250 0x564d26c400e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37b70 0x564d26c3d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c3d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d26b9f020 con 0x564d26c40750 2024-03-15T13:04:11.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d26872430 con 0x564d26c37b70 2024-03-15T13:04:11.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d2682ee00 con 0x564d26c37250 2024-03-15T13:04:11.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f204b3ed740 10 client.0 _ll_register_callbacks cb 0x564d26b46cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:11.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.266+0000 7f2042d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c3d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f2042d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c3d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41854/0 (socket says 172.21.15.2:41854) 2024-03-15T13:04:11.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f2042d9d640 1 -- 172.21.15.2:0/981442447 learned_addr learned my addr 172.21.15.2:0/981442447 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:11.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f204259c640 1 --2- 172.21.15.2:0/981442447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37250 0x564d26c400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f2041d9b640 1 --2- 172.21.15.2:0/981442447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37b70 0x564d26c3d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f2042d9d640 1 -- 172.21.15.2:0/981442447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37250 msgr2=0x564d26c400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f2042d9d640 1 --2- 172.21.15.2:0/981442447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37250 0x564d26c400e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f2042d9d640 1 -- 172.21.15.2:0/981442447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37b70 msgr2=0x564d26c3d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f2042d9d640 1 --2- 172.21.15.2:0/981442447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37b70 0x564d26c3d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f2042d9d640 1 -- 172.21.15.2:0/981442447 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204410ec20 con 0x564d26c40750 2024-03-15T13:04:11.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f204259c640 1 --2- 172.21.15.2:0/981442447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37250 0x564d26c400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:11.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f2042d9d640 1 --2- 172.21.15.2:0/981442447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c3d9f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f20340118c0 tx=0x7f2034004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed03e58b81a5475 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f2029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:11.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f2029ffb640 1 -- 172.21.15.2:0/981442447 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034020db0 con 0x564d26c40750 2024-03-15T13:04:11.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f2029ffb640 1 -- 172.21.15.2:0/981442447 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f203401e410 con 0x564d26c40750 2024-03-15T13:04:11.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f204b3ed740 1 -- 172.21.15.2:0/981442447 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d2689e7e0 con 0x564d26c40750 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f2029ffb640 1 -- 172.21.15.2:0/981442447 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f203401e5d0 con 0x564d26c40750 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f204b3ed740 10 client.17265 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f204b3ed740 1 -- 172.21.15.2:0/981442447 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564d26b9d6a0 con 0x564d26c40750 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.267+0000 7f2029ffb640 1 -- 172.21.15.2:0/981442447 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x564d2689e7e0 con 0x564d26c40750 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.268+0000 7f2029ffb640 1 -- 172.21.15.2:0/981442447 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f203402a640 con 0x564d26c40750 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.268+0000 7f204b3ed740 10 client.17265 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.268+0000 7f204b3ed740 10 client.17265 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.268+0000 7f204b3ed740 20 client.17265 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.268+0000 7f204b3ed740 1 --2- 172.21.15.2:0/981442447 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x564d26c508a0 0x564d26c52cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.268+0000 7f204b3ed740 4 client.17265 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.268+0000 7f204b3ed740 1 -- 172.21.15.2:0/981442447 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d268db720 con 0x564d26c508a0 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.268+0000 7f204259c640 1 --2- 172.21.15.2:0/981442447 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x564d26c508a0 0x564d26c52cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.268+0000 7f2029ffb640 10 client.17265 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.268+0000 7f204259c640 1 --2- 172.21.15.2:0/981442447 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x564d26c508a0 0x564d26c52cc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1dd3f5e8161b6ef5 server_cookie=114d89882dcaacb6 in_seq=0 out_seq=0 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.269+0000 7f2029ffb640 1 -- 172.21.15.2:0/981442447 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564d268db720 con 0x564d26c508a0 2024-03-15T13:04:11.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.269+0000 7f2029ffb640 10 client.17265 handle_command_reply: tid=0 2024-03-15T13:04:11.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.269+0000 7f204b3ed740 1 client.17265 shutdown 2024-03-15T13:04:11.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.269+0000 7f204b3ed740 1 -- 172.21.15.2:0/981442447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 msgr2=0x564d26c3d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.269+0000 7f204b3ed740 1 --2- 172.21.15.2:0/981442447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c3d9f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f20340118c0 tx=0x7f2034004ab0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.269+0000 7f204b3ed740 1 -- 172.21.15.2:0/981442447 shutdown_connections 2024-03-15T13:04:11.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.269+0000 7f204b3ed740 1 --2- 172.21.15.2:0/981442447 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x564d26c508a0 0x564d26c52cc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.269+0000 7f204b3ed740 1 --2- 172.21.15.2:0/981442447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c3d9f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.269+0000 7f2029ffb640 0 client.17265 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:04:11.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.269+0000 7f204b3ed740 1 --2- 172.21.15.2:0/981442447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37b70 0x564d26c3d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.269+0000 7f204b3ed740 1 --2- 172.21.15.2:0/981442447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37250 0x564d26c400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.269+0000 7f204b3ed740 1 -- 172.21.15.2:0/981442447 >> 172.21.15.2:0/981442447 conn(0x564d26c31980 msgr2=0x564d26c336d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.269+0000 7f204b3ed740 1 -- 172.21.15.2:0/981442447 shutdown_connections 2024-03-15T13:04:11.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.269+0000 7f204b3ed740 1 -- 172.21.15.2:0/981442447 wait complete. 2024-03-15T13:04:11.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.269+0000 7f204b3ed740 20 client.17265 trim_cache size 0 max 16384 2024-03-15T13:04:11.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f204b3ed740 1 Processor -- start 2024-03-15T13:04:11.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f204b3ed740 1 -- start start 2024-03-15T13:04:11.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f204b3ed740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37250 0x564d26c3fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f204b3ed740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37b70 0x564d26c400e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f204b3ed740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c4ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f204b3ed740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d26b24130 con 0x564d26c40750 2024-03-15T13:04:11.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f204b3ed740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d26c23a40 con 0x564d26c37250 2024-03-15T13:04:11.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f204b3ed740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d26c329d0 con 0x564d26c37b70 2024-03-15T13:04:11.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f2042d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c4ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f2042d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c4ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41866/0 (socket says 172.21.15.2:41866) 2024-03-15T13:04:11.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f2042d9d640 1 -- 172.21.15.2:0/3967873090 learned_addr learned my addr 172.21.15.2:0/3967873090 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:11.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f204259c640 1 --2- 172.21.15.2:0/3967873090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37250 0x564d26c3fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f2041d9b640 1 --2- 172.21.15.2:0/3967873090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37b70 0x564d26c400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f2042d9d640 1 -- 172.21.15.2:0/3967873090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37b70 msgr2=0x564d26c400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f2042d9d640 1 --2- 172.21.15.2:0/3967873090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37b70 0x564d26c400e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f2042d9d640 1 -- 172.21.15.2:0/3967873090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37250 msgr2=0x564d26c3fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f2042d9d640 1 --2- 172.21.15.2:0/3967873090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37250 0x564d26c3fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f2042d9d640 1 -- 172.21.15.2:0/3967873090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d26b39c20 con 0x564d26c40750 2024-03-15T13:04:11.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.282+0000 7f2042d9d640 1 --2- 172.21.15.2:0/3967873090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c4ba70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f20340115e0 tx=0x7f2034009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f556a4bc42d55d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.283+0000 7f202bfff640 1 -- 172.21.15.2:0/3967873090 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203402ac10 con 0x564d26c40750 2024-03-15T13:04:11.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.283+0000 7f202bfff640 1 -- 172.21.15.2:0/3967873090 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f203402ad70 con 0x564d26c40750 2024-03-15T13:04:11.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.283+0000 7f202bfff640 1 -- 172.21.15.2:0/3967873090 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034003970 con 0x564d26c40750 2024-03-15T13:04:11.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.283+0000 7f204b3ed740 1 -- 172.21.15.2:0/3967873090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 msgr2=0x564d26c4ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.283+0000 7f204b3ed740 1 --2- 172.21.15.2:0/3967873090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c4ba70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f20340115e0 tx=0x7f2034009b80 comp rx=0 tx=0).stop 2024-03-15T13:04:11.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.283+0000 7f204b3ed740 1 -- 172.21.15.2:0/3967873090 shutdown_connections 2024-03-15T13:04:11.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.283+0000 7f204b3ed740 1 --2- 172.21.15.2:0/3967873090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c40750 0x564d26c4ba70 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.283+0000 7f204b3ed740 1 --2- 172.21.15.2:0/3967873090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c37b70 0x564d26c400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.283+0000 7f204b3ed740 1 --2- 172.21.15.2:0/3967873090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37250 0x564d26c3fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.283+0000 7f204b3ed740 1 -- 172.21.15.2:0/3967873090 >> 172.21.15.2:0/3967873090 conn(0x564d26c31980 msgr2=0x564d26c38920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.283+0000 7f204b3ed740 1 -- 172.21.15.2:0/3967873090 shutdown_connections 2024-03-15T13:04:11.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.283+0000 7f204b3ed740 1 -- 172.21.15.2:0/3967873090 wait complete. 2024-03-15T13:04:11.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f204b3ed740 1 -- start start 2024-03-15T13:04:11.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f204b3ed740 1 Processor -- start 2024-03-15T13:04:11.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f204b3ed740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37250 0x564d26c44540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f204b3ed740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c37b70 0x564d26c44a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f204b3ed740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c40750 0x564d26ca4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f204b3ed740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d26ca4800 con 0x564d26c37b70 2024-03-15T13:04:11.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f204b3ed740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d26ca4970 con 0x564d26c37250 2024-03-15T13:04:11.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f204b3ed740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d26ca4b30 con 0x564d26c40750 2024-03-15T13:04:11.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f204b3ed740 10 client.0 _ll_register_callbacks cb 0x7f2034009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:11.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f2041d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c37b70 0x564d26c44a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f2041d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c37b70 0x564d26c44a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41874/0 (socket says 172.21.15.2:41874) 2024-03-15T13:04:11.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f2041d9b640 1 -- 172.21.15.2:0/664033772 learned_addr learned my addr 172.21.15.2:0/664033772 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:11.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f204259c640 1 --2- 172.21.15.2:0/664033772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37250 0x564d26c44540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f2042d9d640 1 --2- 172.21.15.2:0/664033772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c40750 0x564d26ca4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f2041d9b640 1 -- 172.21.15.2:0/664033772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c40750 msgr2=0x564d26ca4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f2041d9b640 1 --2- 172.21.15.2:0/664033772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c40750 0x564d26ca4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f2041d9b640 1 -- 172.21.15.2:0/664033772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37250 msgr2=0x564d26c44540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f2041d9b640 1 --2- 172.21.15.2:0/664033772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37250 0x564d26c44540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f2041d9b640 1 -- 172.21.15.2:0/664033772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d26b9d6a0 con 0x564d26c37b70 2024-03-15T13:04:11.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.284+0000 7f2041d9b640 1 --2- 172.21.15.2:0/664033772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c37b70 0x564d26c44a80 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f202c005bc0 tx=0x7f202c02ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9aebda1f795db5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f2029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:11.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f2029ffb640 1 -- 172.21.15.2:0/664033772 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c004fe0 con 0x564d26c37b70 2024-03-15T13:04:11.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f2029ffb640 1 -- 172.21.15.2:0/664033772 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f202c005140 con 0x564d26c37b70 2024-03-15T13:04:11.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f204b3ed740 1 -- 172.21.15.2:0/664033772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f202c0048e0 con 0x564d26c37b70 2024-03-15T13:04:11.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f2029ffb640 1 -- 172.21.15.2:0/664033772 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f202c005300 con 0x564d26c37b70 2024-03-15T13:04:11.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f204b3ed740 10 client.17271 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:11.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f204b3ed740 1 -- 172.21.15.2:0/664033772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564d26b5f8c0 con 0x564d26c37b70 2024-03-15T13:04:11.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f2029ffb640 1 -- 172.21.15.2:0/664033772 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f202c0048e0 con 0x564d26c37b70 2024-03-15T13:04:11.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f2029ffb640 1 -- 172.21.15.2:0/664033772 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f202c03c050 con 0x564d26c37b70 2024-03-15T13:04:11.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f204b3ed740 10 client.17271 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:11.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f204b3ed740 10 client.17271 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:04:11.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f204b3ed740 20 client.17271 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:11.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f204b3ed740 1 --2- 172.21.15.2:0/664033772 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x564d26cad0b0 0x564d26caf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f204b3ed740 4 client.17271 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:04:11.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f204b3ed740 1 -- 172.21.15.2:0/664033772 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564d2692a890 con 0x564d26cad0b0 2024-03-15T13:04:11.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.285+0000 7f204259c640 1 --2- 172.21.15.2:0/664033772 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x564d26cad0b0 0x564d26caf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f204259c640 1 --2- 172.21.15.2:0/664033772 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x564d26cad0b0 0x564d26caf4d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5fa3156119167b62 server_cookie=90c7c60e941b0bbe in_seq=0 out_seq=0 2024-03-15T13:04:11.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f2029ffb640 10 client.17271 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:04:11.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f2029ffb640 1 -- 172.21.15.2:0/664033772 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564d2692a890 con 0x564d26cad0b0 2024-03-15T13:04:11.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f2029ffb640 10 client.17271 handle_command_reply: tid=0 2024-03-15T13:04:11.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f204b3ed740 1 client.17271 shutdown 2024-03-15T13:04:11.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f204b3ed740 1 -- 172.21.15.2:0/664033772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c37b70 msgr2=0x564d26c44a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f204b3ed740 1 --2- 172.21.15.2:0/664033772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c37b70 0x564d26c44a80 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f202c005bc0 tx=0x7f202c02ccc0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f204b3ed740 1 -- 172.21.15.2:0/664033772 shutdown_connections 2024-03-15T13:04:11.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f204b3ed740 1 --2- 172.21.15.2:0/664033772 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x564d26cad0b0 0x564d26caf4d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f204b3ed740 1 --2- 172.21.15.2:0/664033772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564d26c40750 0x564d26ca4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f2029ffb640 0 client.17271 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:04:11.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f204b3ed740 1 --2- 172.21.15.2:0/664033772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d26c37b70 0x564d26c44a80 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f204b3ed740 1 --2- 172.21.15.2:0/664033772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d26c37250 0x564d26c44540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f204b3ed740 1 -- 172.21.15.2:0/664033772 >> 172.21.15.2:0/664033772 conn(0x564d26c31980 msgr2=0x564d26c3cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f204b3ed740 1 -- 172.21.15.2:0/664033772 shutdown_connections 2024-03-15T13:04:11.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f204b3ed740 1 -- 172.21.15.2:0/664033772 wait complete. 2024-03-15T13:04:11.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.286+0000 7f204b3ed740 20 client.17271 trim_cache size 0 max 16384 2024-03-15T13:04:11.279 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:04:11.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.287+0000 7f204880e640 1 -- 172.21.15.2:0/1641124386 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f20300a1ba0 msgr2=0x7f20300a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.287+0000 7f204880e640 1 --2- 172.21.15.2:0/1641124386 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f20300a1ba0 0x7f20300a4060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2034005fd0 tx=0x7f2034009500 comp rx=0 tx=0).stop 2024-03-15T13:04:11.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.287+0000 7f204880e640 1 -- 172.21.15.2:0/1641124386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20440686f0 msgr2=0x7f204419d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.287+0000 7f204880e640 1 --2- 172.21.15.2:0/1641124386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20440686f0 0x7f204419d740 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f203c00e9f0 tx=0x7f203c00edf0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.287+0000 7f204880e640 1 -- 172.21.15.2:0/1641124386 shutdown_connections 2024-03-15T13:04:11.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.287+0000 7f204880e640 1 --2- 172.21.15.2:0/1641124386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20441074f0 0x7f20441978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.287+0000 7f204880e640 1 --2- 172.21.15.2:0/1641124386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20440690b0 0x7f204419dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.287+0000 7f204880e640 1 --2- 172.21.15.2:0/1641124386 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f20300a1ba0 0x7f20300a4060 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.287+0000 7f204880e640 1 --2- 172.21.15.2:0/1641124386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20440686f0 0x7f204419d740 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.287+0000 7f204880e640 1 -- 172.21.15.2:0/1641124386 >> 172.21.15.2:0/1641124386 conn(0x7f20440774f0 msgr2=0x7f20440fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.287+0000 7f204880e640 1 -- 172.21.15.2:0/1641124386 shutdown_connections 2024-03-15T13:04:11.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.287+0000 7f204880e640 1 -- 172.21.15.2:0/1641124386 wait complete. 2024-03-15T13:04:11.283 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:04:11.283 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:04:11.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.403+0000 7fc0480a3640 1 -- 172.21.15.2:0/763532271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc040073890 msgr2=0x7fc040073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.403+0000 7fc0480a3640 1 --2- 172.21.15.2:0/763532271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc040073890 0x7fc040073c70 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc038009900 tx=0x7fc03802bcd0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.403+0000 7fc0480a3640 1 -- 172.21.15.2:0/763532271 shutdown_connections 2024-03-15T13:04:11.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.403+0000 7fc0480a3640 1 --2- 172.21.15.2:0/763532271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc040112490 0x7fc040114880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.403+0000 7fc0480a3640 1 --2- 172.21.15.2:0/763532271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0400741b0 0x7fc040111f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.403+0000 7fc0480a3640 1 --2- 172.21.15.2:0/763532271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc040073890 0x7fc040073c70 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.403+0000 7fc0480a3640 1 -- 172.21.15.2:0/763532271 >> 172.21.15.2:0/763532271 conn(0x7fc0400788f0 msgr2=0x7fc040078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.403+0000 7fc0480a3640 1 -- 172.21.15.2:0/763532271 shutdown_connections 2024-03-15T13:04:11.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.403+0000 7fc0480a3640 1 -- 172.21.15.2:0/763532271 wait complete. 2024-03-15T13:04:11.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc0480a3640 1 Processor -- start 2024-03-15T13:04:11.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc0480a3640 1 -- start start 2024-03-15T13:04:11.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc0480a3640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc040073890 0x7fc04019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc0480a3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0400741b0 0x7fc04019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc0480a3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc040112490 0x7fc0401a2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc0480a3640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0401a2f80 con 0x7fc0400741b0 2024-03-15T13:04:11.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc0480a3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0401a30f0 con 0x7fc040112490 2024-03-15T13:04:11.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc0480a3640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0401a32b0 con 0x7fc040073890 2024-03-15T13:04:11.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc04563e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0400741b0 0x7fc04019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc04563e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0400741b0 0x7fc04019e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41902/0 (socket says 172.21.15.2:41902) 2024-03-15T13:04:11.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc04563e640 1 -- 172.21.15.2:0/2365729186 learned_addr learned my addr 172.21.15.2:0/2365729186 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:11.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc046640640 1 --2- 172.21.15.2:0/2365729186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc040112490 0x7fc0401a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc045e3f640 1 --2- 172.21.15.2:0/2365729186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc040073890 0x7fc04019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc04563e640 1 -- 172.21.15.2:0/2365729186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc040073890 msgr2=0x7fc04019e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc04563e640 1 --2- 172.21.15.2:0/2365729186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc040073890 0x7fc04019e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc04563e640 1 -- 172.21.15.2:0/2365729186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc040112490 msgr2=0x7fc0401a2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc04563e640 1 --2- 172.21.15.2:0/2365729186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc040112490 0x7fc0401a2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc04563e640 1 -- 172.21.15.2:0/2365729186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc038009560 con 0x7fc0400741b0 2024-03-15T13:04:11.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.404+0000 7fc04563e640 1 --2- 172.21.15.2:0/2365729186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0400741b0 0x7fc04019e600 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc03400e9f0 tx=0x7fc03400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.405+0000 7fc03effd640 1 -- 172.21.15.2:0/2365729186 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034009700 con 0x7fc0400741b0 2024-03-15T13:04:11.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.405+0000 7fc03effd640 1 -- 172.21.15.2:0/2365729186 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc03400cd80 con 0x7fc0400741b0 2024-03-15T13:04:11.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.405+0000 7fc0480a3640 1 -- 172.21.15.2:0/2365729186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0401a3590 con 0x7fc0400741b0 2024-03-15T13:04:11.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.405+0000 7fc03effd640 1 -- 172.21.15.2:0/2365729186 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034018920 con 0x7fc0400741b0 2024-03-15T13:04:11.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.405+0000 7fc0480a3640 1 -- 172.21.15.2:0/2365729186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0401aae10 con 0x7fc0400741b0 2024-03-15T13:04:11.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.407+0000 7fc03effd640 1 -- 172.21.15.2:0/2365729186 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fc034018a80 con 0x7fc0400741b0 2024-03-15T13:04:11.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.407+0000 7fc04ac7d740 1 Processor -- start 2024-03-15T13:04:11.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.407+0000 7fc04ac7d740 1 -- start start 2024-03-15T13:04:11.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.407+0000 7fc04ac7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd4668630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.407+0000 7fc04ac7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 0x55cfd4671210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.407+0000 7fc04ac7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 0x55cfd4673b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.407+0000 7fc04ac7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd4623960 con 0x55cfd4668250 2024-03-15T13:04:11.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.407+0000 7fc04ac7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd45d7310 con 0x55cfd4668b70 2024-03-15T13:04:11.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.407+0000 7fc04ac7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd45d4840 con 0x55cfd4671750 2024-03-15T13:04:11.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc046640640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd4668630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc045e3f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 0x55cfd4671210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc046640640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd4668630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41910/0 (socket says 172.21.15.2:41910) 2024-03-15T13:04:11.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc04563e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 0x55cfd4673b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc045e3f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 0x55cfd4671210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:32924/0 (socket says 172.21.15.2:32924) 2024-03-15T13:04:11.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc046640640 1 -- 172.21.15.2:0/1497846232 learned_addr learned my addr 172.21.15.2:0/1497846232 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:11.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc03effd640 1 --2- 172.21.15.2:0/2365729186 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc0280a1ba0 0x7fc0280a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc046640640 1 --2- 172.21.15.2:0/2365729186 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc0280a1ba0 0x7fc0280a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc046640640 1 -- 172.21.15.2:0/1497846232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 msgr2=0x55cfd4673b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc03effd640 1 -- 172.21.15.2:0/2365729186 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc034014070 con 0x7fc0400741b0 2024-03-15T13:04:11.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc046640640 1 --2- 172.21.15.2:0/1497846232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 0x55cfd4673b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc046640640 1 -- 172.21.15.2:0/1497846232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 msgr2=0x55cfd4671210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc046640640 1 --2- 172.21.15.2:0/1497846232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 0x55cfd4671210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc046640640 1 -- 172.21.15.2:0/1497846232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfd4592060 con 0x55cfd4668250 2024-03-15T13:04:11.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc04563e640 1 --2- 172.21.15.2:0/1497846232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 0x55cfd4673b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:11.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc046640640 1 --2- 172.21.15.2:0/1497846232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd4668630 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc030013a60 tx=0x7fc030013f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea408b16bde0af3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc027fff640 1 -- 172.21.15.2:0/1497846232 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03000fe40 con 0x55cfd4668250 2024-03-15T13:04:11.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc027fff640 1 -- 172.21.15.2:0/1497846232 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc0300133f0 con 0x55cfd4668250 2024-03-15T13:04:11.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc027fff640 1 -- 172.21.15.2:0/1497846232 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0300108e0 con 0x55cfd4668250 2024-03-15T13:04:11.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.408+0000 7fc046640640 1 --2- 172.21.15.2:0/2365729186 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc0280a1ba0 0x7fc0280a4060 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc030023000 tx=0x7fc030020000 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.409+0000 7fc04ac7d740 1 -- 172.21.15.2:0/1497846232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 msgr2=0x55cfd4668630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.409+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/1497846232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd4668630 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc030013a60 tx=0x7fc030013f20 comp rx=0 tx=0).stop 2024-03-15T13:04:11.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.409+0000 7fc04ac7d740 1 -- 172.21.15.2:0/1497846232 shutdown_connections 2024-03-15T13:04:11.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.409+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/1497846232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 0x55cfd4673b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.409+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/1497846232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 0x55cfd4671210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.409+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/1497846232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd4668630 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.409+0000 7fc04ac7d740 1 -- 172.21.15.2:0/1497846232 >> 172.21.15.2:0/1497846232 conn(0x55cfd4662980 msgr2=0x55cfd4662d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.409+0000 7fc04ac7d740 1 -- 172.21.15.2:0/1497846232 shutdown_connections 2024-03-15T13:04:11.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.409+0000 7fc04ac7d740 1 -- 172.21.15.2:0/1497846232 wait complete. 2024-03-15T13:04:11.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.409+0000 7fc04ac7d740 1 -- start start 2024-03-15T13:04:11.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc04ac7d740 1 Processor -- start 2024-03-15T13:04:11.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc04ac7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd4671080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc04ac7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 0x55cfd466e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc04ac7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 0x55cfd466ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc04ac7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd45d0020 con 0x55cfd4668250 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc04ac7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd42a3430 con 0x55cfd4668b70 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc04ac7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd425fe00 con 0x55cfd4671750 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc04ac7d740 10 client.0 _ll_register_callbacks cb 0x55cfd4577cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc045e3f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd4671080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc045e3f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd4671080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41920/0 (socket says 172.21.15.2:41920) 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc045e3f640 1 -- 172.21.15.2:0/2042149322 learned_addr learned my addr 172.21.15.2:0/2042149322 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc04563e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 0x55cfd466e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc046640640 1 --2- 172.21.15.2:0/2042149322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 0x55cfd466ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc04563e640 1 -- 172.21.15.2:0/2042149322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 msgr2=0x55cfd466ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc04563e640 1 --2- 172.21.15.2:0/2042149322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 0x55cfd466ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc04563e640 1 -- 172.21.15.2:0/2042149322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 msgr2=0x55cfd4671080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc04563e640 1 --2- 172.21.15.2:0/2042149322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd4671080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc04563e640 1 -- 172.21.15.2:0/2042149322 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc030013640 con 0x55cfd4668b70 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.410+0000 7fc045e3f640 1 --2- 172.21.15.2:0/2042149322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd4671080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc04563e640 1 --2- 172.21.15.2:0/2042149322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 0x55cfd466e480 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc0340cb230 tx=0x7fc034009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6159b8008d191dbf server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc0257fa640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:04:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc0257fa640 1 -- 172.21.15.2:0/2042149322 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0340cb6b0 con 0x55cfd4668b70 2024-03-15T13:04:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc04ac7d740 1 -- 172.21.15.2:0/2042149322 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfd42cf7e0 con 0x55cfd4668b70 2024-03-15T13:04:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc0257fa640 1 -- 172.21.15.2:0/2042149322 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc0340a2eb0 con 0x55cfd4668b70 2024-03-15T13:04:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc0257fa640 1 -- 172.21.15.2:0/2042149322 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0340b31d0 con 0x55cfd4668b70 2024-03-15T13:04:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc04ac7d740 10 client.27146 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc04ac7d740 1 -- 172.21.15.2:0/2042149322 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55cfd45ce6a0 con 0x55cfd4668b70 2024-03-15T13:04:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc0257fa640 1 -- 172.21.15.2:0/2042149322 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55cfd42cf7e0 con 0x55cfd4668b70 2024-03-15T13:04:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc0257fa640 1 -- 172.21.15.2:0/2042149322 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fc034096510 con 0x55cfd4668b70 2024-03-15T13:04:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc04ac7d740 10 client.27146 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc04ac7d740 10 client.27146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc04ac7d740 20 client.27146 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/2042149322 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cfd4681850 0x55cfd4683c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc04ac7d740 4 client.27146 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.411+0000 7fc04ac7d740 1 -- 172.21.15.2:0/2042149322 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfd430c720 con 0x55cfd4681850 2024-03-15T13:04:11.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.412+0000 7fc045e3f640 1 --2- 172.21.15.2:0/2042149322 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cfd4681850 0x55cfd4683c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.412+0000 7fc0257fa640 10 client.27146 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:11.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.412+0000 7fc045e3f640 1 --2- 172.21.15.2:0/2042149322 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cfd4681850 0x55cfd4683c70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c59ad309d8c2386d server_cookie=f81739b0193f8d7d in_seq=0 out_seq=0 2024-03-15T13:04:11.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.413+0000 7fc0257fa640 1 -- 172.21.15.2:0/2042149322 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cfd430c720 con 0x55cfd4681850 2024-03-15T13:04:11.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.413+0000 7fc0257fa640 10 client.27146 handle_command_reply: tid=0 2024-03-15T13:04:11.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.413+0000 7fc04ac7d740 1 client.27146 shutdown 2024-03-15T13:04:11.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.413+0000 7fc04ac7d740 1 -- 172.21.15.2:0/2042149322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 msgr2=0x55cfd466e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.413+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/2042149322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 0x55cfd466e480 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc0340cb230 tx=0x7fc034009310 comp rx=0 tx=0).stop 2024-03-15T13:04:11.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.413+0000 7fc04ac7d740 1 -- 172.21.15.2:0/2042149322 shutdown_connections 2024-03-15T13:04:11.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.413+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/2042149322 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cfd4681850 0x55cfd4683c70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.413+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/2042149322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 0x55cfd466ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.413+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/2042149322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 0x55cfd466e480 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.413+0000 7fc0257fa640 0 client.27146 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:11.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.413+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/2042149322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd4671080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.413+0000 7fc04ac7d740 1 -- 172.21.15.2:0/2042149322 >> 172.21.15.2:0/2042149322 conn(0x55cfd4662980 msgr2=0x55cfd46646d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.413+0000 7fc04ac7d740 1 -- 172.21.15.2:0/2042149322 shutdown_connections 2024-03-15T13:04:11.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.413+0000 7fc04ac7d740 1 -- 172.21.15.2:0/2042149322 wait complete. 2024-03-15T13:04:11.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.413+0000 7fc04ac7d740 20 client.27146 trim_cache size 0 max 16384 2024-03-15T13:04:11.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.425+0000 7fc04ac7d740 1 Processor -- start 2024-03-15T13:04:11.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.425+0000 7fc04ac7d740 1 -- start start 2024-03-15T13:04:11.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc04ac7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd467cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc04ac7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 0x55cfd467d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc04ac7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 0x55cfd466fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc04ac7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd4555130 con 0x55cfd4668250 2024-03-15T13:04:11.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc04ac7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd46800b0 con 0x55cfd4668b70 2024-03-15T13:04:11.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc04ac7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd467e370 con 0x55cfd4671750 2024-03-15T13:04:11.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc045e3f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd467cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc045e3f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd467cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41928/0 (socket says 172.21.15.2:41928) 2024-03-15T13:04:11.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc045e3f640 1 -- 172.21.15.2:0/233045169 learned_addr learned my addr 172.21.15.2:0/233045169 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:11.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc04563e640 1 --2- 172.21.15.2:0/233045169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 0x55cfd467d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc046640640 1 --2- 172.21.15.2:0/233045169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 0x55cfd466fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc045e3f640 1 -- 172.21.15.2:0/233045169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 msgr2=0x55cfd466fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc045e3f640 1 --2- 172.21.15.2:0/233045169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 0x55cfd466fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc045e3f640 1 -- 172.21.15.2:0/233045169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 msgr2=0x55cfd467d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc045e3f640 1 --2- 172.21.15.2:0/233045169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 0x55cfd467d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc045e3f640 1 -- 172.21.15.2:0/233045169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfd456ac20 con 0x55cfd4668250 2024-03-15T13:04:11.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc045e3f640 1 --2- 172.21.15.2:0/233045169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd467cad0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc038005ab0 tx=0x7fc038004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4aaf7a85662d6c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc0277fe640 1 -- 172.21.15.2:0/233045169 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc038045040 con 0x55cfd4668250 2024-03-15T13:04:11.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc0277fe640 1 -- 172.21.15.2:0/233045169 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc038051070 con 0x55cfd4668250 2024-03-15T13:04:11.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.426+0000 7fc0277fe640 1 -- 172.21.15.2:0/233045169 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03804daf0 con 0x55cfd4668250 2024-03-15T13:04:11.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.427+0000 7fc04ac7d740 1 -- 172.21.15.2:0/233045169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 msgr2=0x55cfd467cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.427+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/233045169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd467cad0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc038005ab0 tx=0x7fc038004ba0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.427+0000 7fc04ac7d740 1 -- 172.21.15.2:0/233045169 shutdown_connections 2024-03-15T13:04:11.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.427+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/233045169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4671750 0x55cfd466fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.427+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/233045169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668b70 0x55cfd467d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.427+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/233045169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4668250 0x55cfd467cad0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.427+0000 7fc04ac7d740 1 -- 172.21.15.2:0/233045169 >> 172.21.15.2:0/233045169 conn(0x55cfd4662980 msgr2=0x55cfd466e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.427+0000 7fc04ac7d740 1 -- 172.21.15.2:0/233045169 shutdown_connections 2024-03-15T13:04:11.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.427+0000 7fc04ac7d740 1 -- 172.21.15.2:0/233045169 wait complete. 2024-03-15T13:04:11.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.427+0000 7fc04ac7d740 1 -- start start 2024-03-15T13:04:11.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.427+0000 7fc04ac7d740 1 Processor -- start 2024-03-15T13:04:11.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc04ac7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668250 0x55cfd4674d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc04ac7d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4668b70 0x55cfd4675250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc04ac7d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4671750 0x55cfd46dc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc04ac7d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd46759b0 con 0x55cfd4671750 2024-03-15T13:04:11.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc04ac7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd4675b20 con 0x55cfd4668250 2024-03-15T13:04:11.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc04ac7d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd46dc660 con 0x55cfd4668b70 2024-03-15T13:04:11.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc04ac7d740 10 client.0 _ll_register_callbacks cb 0x7fc03400cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:11.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc046640640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4671750 0x55cfd46dc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc046640640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4671750 0x55cfd46dc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41938/0 (socket says 172.21.15.2:41938) 2024-03-15T13:04:11.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc046640640 1 -- 172.21.15.2:0/3214848149 learned_addr learned my addr 172.21.15.2:0/3214848149 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:11.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc045e3f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668250 0x55cfd4674d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc04563e640 1 --2- 172.21.15.2:0/3214848149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4668b70 0x55cfd4675250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc046640640 1 -- 172.21.15.2:0/3214848149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4668b70 msgr2=0x55cfd4675250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc046640640 1 --2- 172.21.15.2:0/3214848149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4668b70 0x55cfd4675250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc046640640 1 -- 172.21.15.2:0/3214848149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668250 msgr2=0x55cfd4674d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:11.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc046640640 1 --2- 172.21.15.2:0/3214848149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668250 0x55cfd4674d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc046640640 1 -- 172.21.15.2:0/3214848149 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0401151b0 con 0x55cfd4671750 2024-03-15T13:04:11.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc046640640 1 --2- 172.21.15.2:0/3214848149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4671750 0x55cfd46dc120 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc030009240 tx=0x7fc030016050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423092a27d14d4f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:11.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc0257fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:11.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc0257fa640 1 -- 172.21.15.2:0/3214848149 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03002d050 con 0x55cfd4671750 2024-03-15T13:04:11.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc0257fa640 1 -- 172.21.15.2:0/3214848149 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc030010860 con 0x55cfd4671750 2024-03-15T13:04:11.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.428+0000 7fc04ac7d740 1 -- 172.21.15.2:0/3214848149 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0300133f0 con 0x55cfd4671750 2024-03-15T13:04:11.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.429+0000 7fc0257fa640 1 -- 172.21.15.2:0/3214848149 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc03000fe40 con 0x55cfd4671750 2024-03-15T13:04:11.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.429+0000 7fc04ac7d740 10 client.17289 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:11.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.429+0000 7fc04ac7d740 1 -- 172.21.15.2:0/3214848149 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55cfd45908c0 con 0x55cfd4671750 2024-03-15T13:04:11.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.429+0000 7fc0257fa640 1 -- 172.21.15.2:0/3214848149 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc0300133f0 con 0x55cfd4671750 2024-03-15T13:04:11.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.429+0000 7fc0257fa640 1 -- 172.21.15.2:0/3214848149 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7fc03001fd10 con 0x55cfd4671750 2024-03-15T13:04:11.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.429+0000 7fc04ac7d740 10 client.17289 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:11.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.429+0000 7fc04ac7d740 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:11.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.429+0000 7fc04ac7d740 20 client.17289 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:11.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.429+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/3214848149 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cfd46ddeb0 0x55cfd46e02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:11.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.429+0000 7fc04ac7d740 4 client.17289 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:04:11.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.429+0000 7fc04ac7d740 1 -- 172.21.15.2:0/3214848149 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cfd435b890 con 0x55cfd46ddeb0 2024-03-15T13:04:11.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.429+0000 7fc045e3f640 1 --2- 172.21.15.2:0/3214848149 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cfd46ddeb0 0x55cfd46e02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:11.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc045e3f640 1 --2- 172.21.15.2:0/3214848149 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cfd46ddeb0 0x55cfd46e02d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=302e9ef9c8d496fd server_cookie=32a9c85a069aaf48 in_seq=0 out_seq=0 2024-03-15T13:04:11.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc0257fa640 10 client.17289 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:11.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc0257fa640 1 -- 172.21.15.2:0/3214848149 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cfd435b890 con 0x55cfd46ddeb0 2024-03-15T13:04:11.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc0257fa640 10 client.17289 handle_command_reply: tid=0 2024-03-15T13:04:11.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc04ac7d740 1 client.17289 shutdown 2024-03-15T13:04:11.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc04ac7d740 1 -- 172.21.15.2:0/3214848149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4671750 msgr2=0x55cfd46dc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/3214848149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4671750 0x55cfd46dc120 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc030009240 tx=0x7fc030016050 comp rx=0 tx=0).stop 2024-03-15T13:04:11.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc04ac7d740 1 -- 172.21.15.2:0/3214848149 shutdown_connections 2024-03-15T13:04:11.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/3214848149 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cfd46ddeb0 0x55cfd46e02d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/3214848149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cfd4671750 0x55cfd46dc120 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/3214848149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfd4668b70 0x55cfd4675250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc0257fa640 0 client.17289 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:11.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc04ac7d740 1 --2- 172.21.15.2:0/3214848149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cfd4668250 0x55cfd4674d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc04ac7d740 1 -- 172.21.15.2:0/3214848149 >> 172.21.15.2:0/3214848149 conn(0x55cfd4662980 msgr2=0x55cfd466d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc04ac7d740 1 -- 172.21.15.2:0/3214848149 shutdown_connections 2024-03-15T13:04:11.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc04ac7d740 1 -- 172.21.15.2:0/3214848149 wait complete. 2024-03-15T13:04:11.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.430+0000 7fc04ac7d740 20 client.17289 trim_cache size 0 max 16384 2024-03-15T13:04:11.422 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:04:11.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.431+0000 7fc0480a3640 1 -- 172.21.15.2:0/2365729186 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc0280a1ba0 msgr2=0x7fc0280a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.431+0000 7fc0480a3640 1 --2- 172.21.15.2:0/2365729186 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc0280a1ba0 0x7fc0280a4060 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc030023000 tx=0x7fc030020000 comp rx=0 tx=0).stop 2024-03-15T13:04:11.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.431+0000 7fc0480a3640 1 -- 172.21.15.2:0/2365729186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0400741b0 msgr2=0x7fc04019e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:11.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.431+0000 7fc0480a3640 1 --2- 172.21.15.2:0/2365729186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0400741b0 0x7fc04019e600 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc03400e9f0 tx=0x7fc03400eeb0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.431+0000 7fc0480a3640 1 -- 172.21.15.2:0/2365729186 shutdown_connections 2024-03-15T13:04:11.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.431+0000 7fc0480a3640 1 --2- 172.21.15.2:0/2365729186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc040112490 0x7fc0401a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.431+0000 7fc0480a3640 1 --2- 172.21.15.2:0/2365729186 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc0280a1ba0 0x7fc0280a4060 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.431+0000 7fc0480a3640 1 --2- 172.21.15.2:0/2365729186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0400741b0 0x7fc04019e600 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.431+0000 7fc0480a3640 1 --2- 172.21.15.2:0/2365729186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc040073890 0x7fc04019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:11.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.431+0000 7fc0480a3640 1 -- 172.21.15.2:0/2365729186 >> 172.21.15.2:0/2365729186 conn(0x7fc0400788f0 msgr2=0x7fc04010df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:11.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.431+0000 7fc0480a3640 1 -- 172.21.15.2:0/2365729186 shutdown_connections 2024-03-15T13:04:11.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:11.431+0000 7fc0480a3640 1 -- 172.21.15.2:0/2365729186 wait complete. 2024-03-15T13:04:11.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:11 smithi002 ceph-mon[32060]: pgmap v161: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:04:11.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:11 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3364285711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:04:11.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:11 smithi149 ceph-mon[35927]: pgmap v161: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:04:11.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:11 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3364285711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:04:12.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:11 smithi184 ceph-mon[35993]: pgmap v161: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:04:12.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:11 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3364285711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:04:12.427 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:04:12.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:04:12.427 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:04:12.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.561+0000 7f64cab3e640 1 -- 172.21.15.2:0/1821037974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64c4100f80 msgr2=0x7f64c4104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.561+0000 7f64cab3e640 1 --2- 172.21.15.2:0/1821037974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64c4100f80 0x7f64c4104fc0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009930 tx=0x7f64b802bc30 comp rx=0 tx=0).stop 2024-03-15T13:04:12.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.561+0000 7f64cab3e640 1 -- 172.21.15.2:0/1821037974 shutdown_connections 2024-03-15T13:04:12.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.561+0000 7f64cab3e640 1 --2- 172.21.15.2:0/1821037974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64c41056a0 0x7f64c410d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.561+0000 7f64cab3e640 1 --2- 172.21.15.2:0/1821037974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64c4100f80 0x7f64c4104fc0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.561+0000 7f64cab3e640 1 --2- 172.21.15.2:0/1821037974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64c41005d0 0x7f64c41009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.561+0000 7f64cab3e640 1 -- 172.21.15.2:0/1821037974 >> 172.21.15.2:0/1821037974 conn(0x7f64c40774f0 msgr2=0x7f64c4077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:12.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.561+0000 7f64cab3e640 1 -- 172.21.15.2:0/1821037974 shutdown_connections 2024-03-15T13:04:12.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.561+0000 7f64cab3e640 1 -- 172.21.15.2:0/1821037974 wait complete. 2024-03-15T13:04:12.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.561+0000 7f64cab3e640 1 Processor -- start 2024-03-15T13:04:12.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.561+0000 7f64cab3e640 1 -- start start 2024-03-15T13:04:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.561+0000 7f64cab3e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64c41005d0 0x7f64c4102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.561+0000 7f64cab3e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64c41056a0 0x7f64c41025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64cab3e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64c4102ae0 0x7f64c41a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64cab3e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c41a62f0 con 0x7f64c41005d0 2024-03-15T13:04:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64cab3e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c41a6460 con 0x7f64c41056a0 2024-03-15T13:04:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64cab3e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c41a6620 con 0x7f64c4102ae0 2024-03-15T13:04:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64c88da640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64c41005d0 0x7f64c4102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64c88da640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64c41005d0 0x7f64c4102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41962/0 (socket says 172.21.15.2:41962) 2024-03-15T13:04:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64c88da640 1 -- 172.21.15.2:0/2423197373 learned_addr learned my addr 172.21.15.2:0/2423197373 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64c3fff640 1 --2- 172.21.15.2:0/2423197373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64c41056a0 0x7f64c41025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64c90db640 1 --2- 172.21.15.2:0/2423197373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64c4102ae0 0x7f64c41a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64c88da640 1 -- 172.21.15.2:0/2423197373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64c4102ae0 msgr2=0x7f64c41a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64c88da640 1 --2- 172.21.15.2:0/2423197373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64c4102ae0 0x7f64c41a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64c88da640 1 -- 172.21.15.2:0/2423197373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64c41056a0 msgr2=0x7f64c41025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64c88da640 1 --2- 172.21.15.2:0/2423197373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64c41056a0 0x7f64c41025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64c88da640 1 -- 172.21.15.2:0/2423197373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b8009560 con 0x7f64c41005d0 2024-03-15T13:04:12.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64c88da640 1 --2- 172.21.15.2:0/2423197373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64c41005d0 0x7f64c4102060 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f64b400e9f0 tx=0x7f64b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:12.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64c1ffb640 1 -- 172.21.15.2:0/2423197373 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4009720 con 0x7f64c41005d0 2024-03-15T13:04:12.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64c1ffb640 1 -- 172.21.15.2:0/2423197373 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f64b400cd80 con 0x7f64c41005d0 2024-03-15T13:04:12.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64cab3e640 1 -- 172.21.15.2:0/2423197373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c41a6900 con 0x7f64c41005d0 2024-03-15T13:04:12.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.562+0000 7f64c1ffb640 1 -- 172.21.15.2:0/2423197373 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4010690 con 0x7f64c41005d0 2024-03-15T13:04:12.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.563+0000 7f64cab3e640 1 -- 172.21.15.2:0/2423197373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c41a6d90 con 0x7f64c41005d0 2024-03-15T13:04:12.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.565+0000 7f64c1ffb640 1 -- 172.21.15.2:0/2423197373 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f64b4004e50 con 0x7f64c41005d0 2024-03-15T13:04:12.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.565+0000 7f64c1ffb640 1 --2- 172.21.15.2:0/2423197373 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f64ac0a1bd0 0x7f64ac0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.565+0000 7f64cd7f7740 1 Processor -- start 2024-03-15T13:04:12.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.565+0000 7f64c3fff640 1 --2- 172.21.15.2:0/2423197373 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f64ac0a1bd0 0x7f64ac0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.565+0000 7f64c1ffb640 1 -- 172.21.15.2:0/2423197373 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f64b4014070 con 0x7f64c41005d0 2024-03-15T13:04:12.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.565+0000 7f64cd7f7740 1 -- start start 2024-03-15T13:04:12.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.565+0000 7f64cd7f7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdca380 0x563edfdca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.565+0000 7f64cd7f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdcad30 0x563edfdd32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.565+0000 7f64cd7f7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdd38b0 0x563edfdd5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64cd7f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edfd38f90 con 0x563edfdcad30 2024-03-15T13:04:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64cd7f7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edfd364c0 con 0x563edfdca380 2024-03-15T13:04:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64cd7f7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edfd31ca0 con 0x563edfdd38b0 2024-03-15T13:04:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64c88da640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdca380 0x563edfdca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64c88da640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdca380 0x563edfdca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:32988/0 (socket says 172.21.15.2:32988) 2024-03-15T13:04:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64c88da640 1 -- 172.21.15.2:0/2858921665 learned_addr learned my addr 172.21.15.2:0/2858921665 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64c90db640 1 --2- 172.21.15.2:0/2858921665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdd38b0 0x563edfdd5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64c3fff640 1 --2- 172.21.15.2:0/2423197373 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f64ac0a1bd0 0x7f64ac0a4090 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009900 tx=0x7f64b8005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64c3fff640 1 --2- 172.21.15.2:0/2858921665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdcad30 0x563edfdd32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64c88da640 1 -- 172.21.15.2:0/2858921665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdd38b0 msgr2=0x563edfdd5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64c88da640 1 --2- 172.21.15.2:0/2858921665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdd38b0 0x563edfdd5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64c88da640 1 -- 172.21.15.2:0/2858921665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdcad30 msgr2=0x563edfdd32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64c88da640 1 --2- 172.21.15.2:0/2858921665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdcad30 0x563edfdd32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64c88da640 1 -- 172.21.15.2:0/2858921665 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563edfcf40b0 con 0x563edfdca380 2024-03-15T13:04:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64c3fff640 1 --2- 172.21.15.2:0/2858921665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdcad30 0x563edfdd32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:04:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64c88da640 1 --2- 172.21.15.2:0/2858921665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdca380 0x563edfdca760 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009380 tx=0x7f64b4096b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd1ef3872193dcea server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64aaffd640 1 -- 172.21.15.2:0/2858921665 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b409e890 con 0x563edfdca380 2024-03-15T13:04:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64aaffd640 1 -- 172.21.15.2:0/2858921665 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f64b40a2ee0 con 0x563edfdca380 2024-03-15T13:04:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.566+0000 7f64aaffd640 1 -- 172.21.15.2:0/2858921665 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b40a2a50 con 0x563edfdca380 2024-03-15T13:04:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.567+0000 7f64cd7f7740 1 -- 172.21.15.2:0/2858921665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdca380 msgr2=0x563edfdca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.567+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/2858921665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdca380 0x563edfdca760 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009380 tx=0x7f64b4096b10 comp rx=0 tx=0).stop 2024-03-15T13:04:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.567+0000 7f64cd7f7740 1 -- 172.21.15.2:0/2858921665 shutdown_connections 2024-03-15T13:04:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.567+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/2858921665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdd38b0 0x563edfdd5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.567+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/2858921665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdcad30 0x563edfdd32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.567+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/2858921665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdca380 0x563edfdca760 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.567+0000 7f64cd7f7740 1 -- 172.21.15.2:0/2858921665 >> 172.21.15.2:0/2858921665 conn(0x563edfdc4840 msgr2=0x563edfdc4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.567+0000 7f64cd7f7740 1 -- 172.21.15.2:0/2858921665 shutdown_connections 2024-03-15T13:04:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.567+0000 7f64cd7f7740 1 -- 172.21.15.2:0/2858921665 wait complete. 2024-03-15T13:04:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.567+0000 7f64cd7f7740 1 -- start start 2024-03-15T13:04:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.567+0000 7f64cd7f7740 1 Processor -- start 2024-03-15T13:04:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64cd7f7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdca380 0x563edfdd0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64cd7f7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdcad30 0x563edfdd0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64cd7f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfdd13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64cd7f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edfcd6f40 con 0x563edfdd38b0 2024-03-15T13:04:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64cd7f7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edfa054b0 con 0x563edfdcad30 2024-03-15T13:04:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64cd7f7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edfcb7670 con 0x563edfdca380 2024-03-15T13:04:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64cd7f7740 10 client.0 _ll_register_callbacks cb 0x563edfd24f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64c90db640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfdd13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64c3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdcad30 0x563edfdd0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64c3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdcad30 0x563edfdd0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:32990/0 (socket says 172.21.15.2:32990) 2024-03-15T13:04:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64c3fff640 1 -- 172.21.15.2:0/3954692958 learned_addr learned my addr 172.21.15.2:0/3954692958 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64c88da640 1 --2- 172.21.15.2:0/3954692958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdca380 0x563edfdd0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64c3fff640 1 -- 172.21.15.2:0/3954692958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdca380 msgr2=0x563edfdd0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64c3fff640 1 --2- 172.21.15.2:0/3954692958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdca380 0x563edfdd0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64c3fff640 1 -- 172.21.15.2:0/3954692958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 msgr2=0x563edfdd13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64c3fff640 1 --2- 172.21.15.2:0/3954692958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfdd13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64c3fff640 1 -- 172.21.15.2:0/3954692958 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c41a6d90 con 0x563edfdcad30 2024-03-15T13:04:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64c88da640 1 --2- 172.21.15.2:0/3954692958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdca380 0x563edfdd0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.568+0000 7f64c90db640 1 --2- 172.21.15.2:0/3954692958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfdd13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64c3fff640 1 --2- 172.21.15.2:0/3954692958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdcad30 0x563edfdd0e80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f64b8005ab0 tx=0x7f64b8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b928e9ad757ddec server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:04:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64a3fff640 1 -- 172.21.15.2:0/3954692958 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b80043d0 con 0x563edfdcad30 2024-03-15T13:04:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64a3fff640 1 -- 172.21.15.2:0/3954692958 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f64b8047040 con 0x563edfdcad30 2024-03-15T13:04:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3954692958 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563edfd0bf40 con 0x563edfdcad30 2024-03-15T13:04:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64a3fff640 1 -- 172.21.15.2:0/3954692958 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64b8004de0 con 0x563edfdcad30 2024-03-15T13:04:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64cd7f7740 10 client.27164 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3954692958 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563edfd30320 con 0x563edfdcad30 2024-03-15T13:04:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64a3fff640 1 -- 172.21.15.2:0/3954692958 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x563edfd0bf40 con 0x563edfdcad30 2024-03-15T13:04:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64a3fff640 1 -- 172.21.15.2:0/3954692958 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f64b80038e0 con 0x563edfdcad30 2024-03-15T13:04:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64cd7f7740 10 client.27164 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64cd7f7740 10 client.27164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64cd7f7740 20 client.27164 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3954692958 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563edfde4010 0x563edfde6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64cd7f7740 4 client.27164 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.569+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3954692958 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563edfd28310 con 0x563edfde4010 2024-03-15T13:04:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.570+0000 7f64c88da640 1 --2- 172.21.15.2:0/3954692958 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563edfde4010 0x563edfde6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.570+0000 7f64c88da640 1 --2- 172.21.15.2:0/3954692958 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563edfde4010 0x563edfde6430 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc53eb2ea1a0885 server_cookie=83dd0289b6316b0c in_seq=0 out_seq=0 2024-03-15T13:04:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.570+0000 7f64a3fff640 10 client.27164 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:12.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.571+0000 7f64a3fff640 1 -- 172.21.15.2:0/3954692958 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563edfd28310 con 0x563edfde4010 2024-03-15T13:04:12.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.571+0000 7f64a3fff640 10 client.27164 handle_command_reply: tid=0 2024-03-15T13:04:12.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.571+0000 7f64cd7f7740 1 client.27164 shutdown 2024-03-15T13:04:12.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.571+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3954692958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdcad30 msgr2=0x563edfdd0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.571+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3954692958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdcad30 0x563edfdd0e80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f64b8005ab0 tx=0x7f64b8009260 comp rx=0 tx=0).stop 2024-03-15T13:04:12.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.571+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3954692958 shutdown_connections 2024-03-15T13:04:12.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.571+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3954692958 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563edfde4010 0x563edfde6430 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.571+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3954692958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfdd13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.571+0000 7f64a3fff640 0 client.27164 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:12.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.571+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3954692958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdcad30 0x563edfdd0e80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.571+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3954692958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdca380 0x563edfdd0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.571+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3954692958 >> 172.21.15.2:0/3954692958 conn(0x563edfdc4840 msgr2=0x563edfdc67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:12.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.571+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3954692958 shutdown_connections 2024-03-15T13:04:12.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.571+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3954692958 wait complete. 2024-03-15T13:04:12.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.571+0000 7f64cd7f7740 20 client.27164 trim_cache size 0 max 16384 2024-03-15T13:04:12.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64cd7f7740 1 Processor -- start 2024-03-15T13:04:12.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64cd7f7740 1 -- start start 2024-03-15T13:04:12.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64cd7f7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdca380 0x563edfdd3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64cd7f7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdcad30 0x563edfdd11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64cd7f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfddedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64cd7f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edfdcc180 con 0x563edfdd38b0 2024-03-15T13:04:12.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64cd7f7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edfdc0140 con 0x563edfdca380 2024-03-15T13:04:12.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64cd7f7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edfdc9490 con 0x563edfdcad30 2024-03-15T13:04:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64c90db640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfddedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64c90db640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfddedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41980/0 (socket says 172.21.15.2:41980) 2024-03-15T13:04:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64c88da640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdca380 0x563edfdd3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64c90db640 1 -- 172.21.15.2:0/3270067772 learned_addr learned my addr 172.21.15.2:0/3270067772 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64c3fff640 1 --2- 172.21.15.2:0/3270067772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdcad30 0x563edfdd11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64c88da640 1 -- 172.21.15.2:0/3270067772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdcad30 msgr2=0x563edfdd11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64c88da640 1 --2- 172.21.15.2:0/3270067772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdcad30 0x563edfdd11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64c88da640 1 -- 172.21.15.2:0/3270067772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 msgr2=0x563edfddedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64c88da640 1 --2- 172.21.15.2:0/3270067772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfddedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64c88da640 1 -- 172.21.15.2:0/3270067772 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563edfcccc00 con 0x563edfdca380 2024-03-15T13:04:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64c90db640 1 --2- 172.21.15.2:0/3270067772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfddedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.588+0000 7f64c3fff640 1 --2- 172.21.15.2:0/3270067772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdcad30 0x563edfdd11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.589+0000 7f64c88da640 1 --2- 172.21.15.2:0/3270067772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdca380 0x563edfdd3070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f64b40a29b0 tx=0x7f64b40969e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1abaf70166abffe server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.589+0000 7f64aa7fc640 1 -- 172.21.15.2:0/3270067772 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b40a94b0 con 0x563edfdca380 2024-03-15T13:04:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.589+0000 7f64aa7fc640 1 -- 172.21.15.2:0/3270067772 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f64b409e890 con 0x563edfdca380 2024-03-15T13:04:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.589+0000 7f64aa7fc640 1 -- 172.21.15.2:0/3270067772 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b40a2a50 con 0x563edfdca380 2024-03-15T13:04:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3270067772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdca380 msgr2=0x563edfdd3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3270067772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdca380 0x563edfdd3070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f64b40a29b0 tx=0x7f64b40969e0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3270067772 shutdown_connections 2024-03-15T13:04:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3270067772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfddedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3270067772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdcad30 0x563edfdd11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3270067772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdca380 0x563edfdd3070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3270067772 >> 172.21.15.2:0/3270067772 conn(0x563edfdc4840 msgr2=0x563edfdc38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3270067772 shutdown_connections 2024-03-15T13:04:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3270067772 wait complete. 2024-03-15T13:04:12.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 -- start start 2024-03-15T13:04:12.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 Processor -- start 2024-03-15T13:04:12.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdca380 0x563edfdcd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdcad30 0x563edfdcdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfe3e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edfe3ee00 con 0x563edfdd38b0 2024-03-15T13:04:12.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edfe3ef40 con 0x563edfdcad30 2024-03-15T13:04:12.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edfe3f080 con 0x563edfdca380 2024-03-15T13:04:12.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.590+0000 7f64cd7f7740 10 client.0 _ll_register_callbacks cb 0x7f64b4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:12.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64c3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdcad30 0x563edfdcdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64c90db640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfe3e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64c3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdcad30 0x563edfdcdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:32998/0 (socket says 172.21.15.2:32998) 2024-03-15T13:04:12.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64c3fff640 1 -- 172.21.15.2:0/3252861031 learned_addr learned my addr 172.21.15.2:0/3252861031 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:12.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64c88da640 1 --2- 172.21.15.2:0/3252861031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdca380 0x563edfdcd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64c3fff640 1 -- 172.21.15.2:0/3252861031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdca380 msgr2=0x563edfdcd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64c3fff640 1 --2- 172.21.15.2:0/3252861031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdca380 0x563edfdcd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64c3fff640 1 -- 172.21.15.2:0/3252861031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 msgr2=0x563edfe3e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64c3fff640 1 --2- 172.21.15.2:0/3252861031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfe3e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64c3fff640 1 -- 172.21.15.2:0/3252861031 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563edfd30320 con 0x563edfdcad30 2024-03-15T13:04:12.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64c88da640 1 --2- 172.21.15.2:0/3252861031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdca380 0x563edfdcd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:12.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64c90db640 1 --2- 172.21.15.2:0/3252861031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfe3e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:12.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64c3fff640 1 --2- 172.21.15.2:0/3252861031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdcad30 0x563edfdcdca0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009230 tx=0x7f64b8004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe7320547701820 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:12.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:04:12.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64a3fff640 1 -- 172.21.15.2:0/3252861031 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b802cbf0 con 0x563edfdcad30 2024-03-15T13:04:12.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3252861031 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563edfd57900 con 0x563edfdcad30 2024-03-15T13:04:12.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64a3fff640 1 -- 172.21.15.2:0/3252861031 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f64b8003800 con 0x563edfdcad30 2024-03-15T13:04:12.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64a3fff640 1 -- 172.21.15.2:0/3252861031 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64b80039c0 con 0x563edfdcad30 2024-03-15T13:04:12.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64cd7f7740 10 client.27170 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:12.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.591+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3252861031 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f64bc003970 con 0x563edfdcad30 2024-03-15T13:04:12.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.592+0000 7f64a3fff640 1 -- 172.21.15.2:0/3252861031 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x563edfd57900 con 0x563edfdcad30 2024-03-15T13:04:12.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.592+0000 7f64a3fff640 1 -- 172.21.15.2:0/3252861031 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f64b8034560 con 0x563edfdcad30 2024-03-15T13:04:12.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.592+0000 7f64cd7f7740 10 client.27170 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:12.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.592+0000 7f64cd7f7740 10 client.27170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:12.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.592+0000 7f64cd7f7740 20 client.27170 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:12.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.592+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3252861031 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563edfe405e0 0x563edfe42a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.592+0000 7f64cd7f7740 4 client.27170 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:04:12.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.592+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3252861031 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563edfa31860 con 0x563edfe405e0 2024-03-15T13:04:12.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.592+0000 7f64c88da640 1 --2- 172.21.15.2:0/3252861031 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563edfe405e0 0x563edfe42a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.592+0000 7f64c88da640 1 --2- 172.21.15.2:0/3252861031 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563edfe405e0 0x563edfe42a00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7d1e0b988acabdd server_cookie=7ad972dcc8a41124 in_seq=0 out_seq=0 2024-03-15T13:04:12.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.592+0000 7f64a3fff640 10 client.27170 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:12.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.593+0000 7f64a3fff640 1 -- 172.21.15.2:0/3252861031 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563edfa31860 con 0x563edfe405e0 2024-03-15T13:04:12.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.593+0000 7f64a3fff640 10 client.27170 handle_command_reply: tid=0 2024-03-15T13:04:12.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.593+0000 7f64cd7f7740 1 client.27170 shutdown 2024-03-15T13:04:12.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.593+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3252861031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdcad30 msgr2=0x563edfdcdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.593+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3252861031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdcad30 0x563edfdcdca0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009230 tx=0x7f64b8004600 comp rx=0 tx=0).stop 2024-03-15T13:04:12.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3252861031 shutdown_connections 2024-03-15T13:04:12.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3252861031 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563edfe405e0 0x563edfe42a00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3252861031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563edfdd38b0 0x563edfe3e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3252861031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563edfdcad30 0x563edfdcdca0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64a3fff640 0 client.27170 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:12.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cd7f7740 1 --2- 172.21.15.2:0/3252861031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563edfdca380 0x563edfdcd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3252861031 >> 172.21.15.2:0/3252861031 conn(0x563edfdc4840 msgr2=0x563edfdc5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:12.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3252861031 shutdown_connections 2024-03-15T13:04:12.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cd7f7740 1 -- 172.21.15.2:0/3252861031 wait complete. 2024-03-15T13:04:12.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cd7f7740 20 client.27170 trim_cache size 0 max 16384 2024-03-15T13:04:12.587 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:04:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:04:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "abe43775-4f89-4322-b928-f44766dfc449", 2024-03-15T13:04:12.587 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:04:12.587 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:04:12.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cab3e640 1 -- 172.21.15.2:0/2423197373 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f64ac0a1bd0 msgr2=0x7f64ac0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cab3e640 1 --2- 172.21.15.2:0/2423197373 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f64ac0a1bd0 0x7f64ac0a4090 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009900 tx=0x7f64b8005fd0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cab3e640 1 -- 172.21.15.2:0/2423197373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64c41005d0 msgr2=0x7f64c4102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cab3e640 1 --2- 172.21.15.2:0/2423197373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64c41005d0 0x7f64c4102060 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f64b400e9f0 tx=0x7f64b400edf0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cab3e640 1 -- 172.21.15.2:0/2423197373 shutdown_connections 2024-03-15T13:04:12.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cab3e640 1 --2- 172.21.15.2:0/2423197373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64c4102ae0 0x7f64c41a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cab3e640 1 --2- 172.21.15.2:0/2423197373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64c41056a0 0x7f64c41025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cab3e640 1 --2- 172.21.15.2:0/2423197373 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f64ac0a1bd0 0x7f64ac0a4090 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cab3e640 1 --2- 172.21.15.2:0/2423197373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64c41005d0 0x7f64c4102060 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cab3e640 1 -- 172.21.15.2:0/2423197373 >> 172.21.15.2:0/2423197373 conn(0x7f64c40774f0 msgr2=0x7f64c40ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:12.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.594+0000 7f64cab3e640 1 -- 172.21.15.2:0/2423197373 shutdown_connections 2024-03-15T13:04:12.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.595+0000 7f64cab3e640 1 -- 172.21.15.2:0/2423197373 wait complete. 2024-03-15T13:04:12.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:04:12.591 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:04:12.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.710+0000 7f74ef12c640 1 -- 172.21.15.2:0/804561146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74e8107500 msgr2=0x7f74e81078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.710+0000 7f74ef12c640 1 --2- 172.21.15.2:0/804561146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74e8107500 0x7f74e81078e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009cd0 tx=0x7f74e002c0f0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.710+0000 7f74ef12c640 1 -- 172.21.15.2:0/804561146 shutdown_connections 2024-03-15T13:04:12.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.710+0000 7f74ef12c640 1 --2- 172.21.15.2:0/804561146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e8101e20 0x7f74e810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.710+0000 7f74ef12c640 1 --2- 172.21.15.2:0/804561146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e8101460 0x7f74e81018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.710+0000 7f74ef12c640 1 --2- 172.21.15.2:0/804561146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74e8107500 0x7f74e81078e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.711+0000 7f74ef12c640 1 -- 172.21.15.2:0/804561146 >> 172.21.15.2:0/804561146 conn(0x7f74e80fcf90 msgr2=0x7f74e80ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:12.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.711+0000 7f74ef12c640 1 -- 172.21.15.2:0/804561146 shutdown_connections 2024-03-15T13:04:12.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.711+0000 7f74ef12c640 1 -- 172.21.15.2:0/804561146 wait complete. 2024-03-15T13:04:12.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.711+0000 7f74ef12c640 1 Processor -- start 2024-03-15T13:04:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.711+0000 7f74ef12c640 1 -- start start 2024-03-15T13:04:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.711+0000 7f74ef12c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e8101460 0x7f74e819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.711+0000 7f74ef12c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74e8101e20 0x7f74e819dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.711+0000 7f74ef12c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e8107500 0x7f74e8197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.711+0000 7f74ef12c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e8197fb0 con 0x7f74e8101e20 2024-03-15T13:04:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.711+0000 7f74ef12c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e8198120 con 0x7f74e8107500 2024-03-15T13:04:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.711+0000 7f74ef12c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e81982e0 con 0x7f74e8101460 2024-03-15T13:04:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.711+0000 7f74e7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74e8101e20 0x7f74e819dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.711+0000 7f74e7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74e8101e20 0x7f74e819dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42004/0 (socket says 172.21.15.2:42004) 2024-03-15T13:04:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.712+0000 7f74e7fff640 1 -- 172.21.15.2:0/3087489908 learned_addr learned my addr 172.21.15.2:0/3087489908 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.712+0000 7f74ed6c9640 1 --2- 172.21.15.2:0/3087489908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e8107500 0x7f74e8197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.712+0000 7f74ecec8640 1 --2- 172.21.15.2:0/3087489908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e8101460 0x7f74e819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.712+0000 7f74e7fff640 1 -- 172.21.15.2:0/3087489908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e8101460 msgr2=0x7f74e819d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.712+0000 7f74e7fff640 1 --2- 172.21.15.2:0/3087489908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e8101460 0x7f74e819d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.712+0000 7f74e7fff640 1 -- 172.21.15.2:0/3087489908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e8107500 msgr2=0x7f74e8197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.712+0000 7f74e7fff640 1 --2- 172.21.15.2:0/3087489908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e8107500 0x7f74e8197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.712+0000 7f74e7fff640 1 -- 172.21.15.2:0/3087489908 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d0009490 con 0x7f74e8101e20 2024-03-15T13:04:12.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.712+0000 7f74e7fff640 1 --2- 172.21.15.2:0/3087489908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74e8101e20 0x7f74e819dcf0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f74d800e9f0 tx=0x7f74d800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:12.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.712+0000 7f74e5ffb640 1 -- 172.21.15.2:0/3087489908 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8009700 con 0x7f74e8101e20 2024-03-15T13:04:12.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.712+0000 7f74e5ffb640 1 -- 172.21.15.2:0/3087489908 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f74d800ccc0 con 0x7f74e8101e20 2024-03-15T13:04:12.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.712+0000 7f74ef12c640 1 -- 172.21.15.2:0/3087489908 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e0009900 con 0x7f74e8101e20 2024-03-15T13:04:12.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.712+0000 7f74e5ffb640 1 -- 172.21.15.2:0/3087489908 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8018820 con 0x7f74e8101e20 2024-03-15T13:04:12.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.712+0000 7f74ef12c640 1 -- 172.21.15.2:0/3087489908 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e81a4670 con 0x7f74e8101e20 2024-03-15T13:04:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74e5ffb640 1 -- 172.21.15.2:0/3087489908 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f74d8010430 con 0x7f74e8101e20 2024-03-15T13:04:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74f1de7740 1 Processor -- start 2024-03-15T13:04:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74f1de7740 1 -- start start 2024-03-15T13:04:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74f1de7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74f1de7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 0x55bca97bf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74f1de7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97bf7b0 0x55bca97c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74f1de7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca9725280 con 0x55bca97b62b0 2024-03-15T13:04:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74f1de7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca97227b0 con 0x55bca97bf7b0 2024-03-15T13:04:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74f1de7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca971df90 con 0x55bca97b6bd0 2024-03-15T13:04:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74e5ffb640 1 --2- 172.21.15.2:0/3087489908 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f74d40a1ba0 0x7f74d40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74ed6c9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74ed6c9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97b6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42012/0 (socket says 172.21.15.2:42012) 2024-03-15T13:04:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74ed6c9640 1 -- 172.21.15.2:0/3452244093 learned_addr learned my addr 172.21.15.2:0/3452244093 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74ecec8640 1 --2- 172.21.15.2:0/3452244093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 0x55bca97bf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74e7fff640 1 --2- 172.21.15.2:0/3452244093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97bf7b0 0x55bca97c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.715+0000 7f74ed6c9640 1 --2- 172.21.15.2:0/3087489908 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f74d40a1ba0 0x7f74d40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.716+0000 7f74e5ffb640 1 -- 172.21.15.2:0/3087489908 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f74d8014070 con 0x7f74e8101e20 2024-03-15T13:04:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.716+0000 7f74ecec8640 1 -- 172.21.15.2:0/3452244093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97bf7b0 msgr2=0x55bca97c1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.716+0000 7f74ecec8640 1 --2- 172.21.15.2:0/3452244093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97bf7b0 0x55bca97c1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.716+0000 7f74ecec8640 1 -- 172.21.15.2:0/3452244093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 msgr2=0x55bca97b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.716+0000 7f74ecec8640 1 --2- 172.21.15.2:0/3452244093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.716+0000 7f74ecec8640 1 -- 172.21.15.2:0/3452244093 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bca96dc570 con 0x55bca97b6bd0 2024-03-15T13:04:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.716+0000 7f74ed6c9640 1 --2- 172.21.15.2:0/3452244093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.716+0000 7f74ecec8640 1 --2- 172.21.15.2:0/3452244093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 0x55bca97bf270 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f74e0037040 tx=0x7f74e0032d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=804e8083c276bfaa server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.716+0000 7f74ed6c9640 1 --2- 172.21.15.2:0/3087489908 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f74d40a1ba0 0x7f74d40a4060 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f74d00097d0 tx=0x7f74d0004730 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:12.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.716+0000 7f74deffd640 1 -- 172.21.15.2:0/3452244093 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0034440 con 0x55bca97b6bd0 2024-03-15T13:04:12.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.716+0000 7f74deffd640 1 -- 172.21.15.2:0/3452244093 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f74e00345a0 con 0x55bca97b6bd0 2024-03-15T13:04:12.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.716+0000 7f74deffd640 1 -- 172.21.15.2:0/3452244093 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0003b70 con 0x55bca97b6bd0 2024-03-15T13:04:12.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.717+0000 7f74f1de7740 1 -- 172.21.15.2:0/3452244093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 msgr2=0x55bca97bf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.717+0000 7f74f1de7740 1 --2- 172.21.15.2:0/3452244093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 0x55bca97bf270 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f74e0037040 tx=0x7f74e0032d20 comp rx=0 tx=0).stop 2024-03-15T13:04:12.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.717+0000 7f74f1de7740 1 -- 172.21.15.2:0/3452244093 shutdown_connections 2024-03-15T13:04:12.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.717+0000 7f74f1de7740 1 --2- 172.21.15.2:0/3452244093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97bf7b0 0x55bca97c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.717+0000 7f74f1de7740 1 --2- 172.21.15.2:0/3452244093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 0x55bca97bf270 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.717+0000 7f74f1de7740 1 --2- 172.21.15.2:0/3452244093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.717+0000 7f74f1de7740 1 -- 172.21.15.2:0/3452244093 >> 172.21.15.2:0/3452244093 conn(0x55bca97b0990 msgr2=0x55bca97b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:12.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.717+0000 7f74f1de7740 1 -- 172.21.15.2:0/3452244093 shutdown_connections 2024-03-15T13:04:12.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.717+0000 7f74f1de7740 1 -- 172.21.15.2:0/3452244093 wait complete. 2024-03-15T13:04:12.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.717+0000 7f74f1de7740 1 -- start start 2024-03-15T13:04:12.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.717+0000 7f74f1de7740 1 Processor -- start 2024-03-15T13:04:12.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74f1de7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97bc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74f1de7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 0x55bca97bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74f1de7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97bf7b0 0x55bca97bd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74f1de7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca93f1430 con 0x55bca97b62b0 2024-03-15T13:04:12.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74f1de7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca93ade00 con 0x55bca97bf7b0 2024-03-15T13:04:12.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74f1de7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca96a2fd0 con 0x55bca97b6bd0 2024-03-15T13:04:12.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74f1de7740 10 client.0 _ll_register_callbacks cb 0x55bca96f8fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:12.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74ecec8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97bc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74e7fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 0x55bca97bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74ecec8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97bc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42024/0 (socket says 172.21.15.2:42024) 2024-03-15T13:04:12.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74ecec8640 1 -- 172.21.15.2:0/3599085121 learned_addr learned my addr 172.21.15.2:0/3599085121 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:12.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74ed6c9640 1 --2- 172.21.15.2:0/3599085121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97bf7b0 0x55bca97bd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74ecec8640 1 -- 172.21.15.2:0/3599085121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 msgr2=0x55bca97bcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74ecec8640 1 --2- 172.21.15.2:0/3599085121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 0x55bca97bcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74ecec8640 1 -- 172.21.15.2:0/3599085121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97bf7b0 msgr2=0x55bca97bd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74ecec8640 1 --2- 172.21.15.2:0/3599085121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97bf7b0 0x55bca97bd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74ecec8640 1 -- 172.21.15.2:0/3599085121 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e810ec30 con 0x55bca97b62b0 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74e7fff640 1 --2- 172.21.15.2:0/3599085121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 0x55bca97bcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74ecec8640 1 --2- 172.21.15.2:0/3599085121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97bc630 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f74e002c700 tx=0x7f74e0009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4599c1f5cd7726 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74c7fff640 1 -- 172.21.15.2:0/3599085121 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0032b10 con 0x55bca97b62b0 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74c7fff640 1 -- 172.21.15.2:0/3599085121 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f74e0032680 con 0x55bca97b62b0 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.718+0000 7f74f1de7740 1 -- 172.21.15.2:0/3599085121 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bca941d7e0 con 0x55bca97b62b0 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.719+0000 7f74c7fff640 1 -- 172.21.15.2:0/3599085121 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74e0003970 con 0x55bca97b62b0 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.719+0000 7f74f1de7740 10 client.17319 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.719+0000 7f74f1de7740 1 -- 172.21.15.2:0/3599085121 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55bca971c610 con 0x55bca97b62b0 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.719+0000 7f74c7fff640 1 -- 172.21.15.2:0/3599085121 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f74e005c020 con 0x55bca97b62b0 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.719+0000 7f74c7fff640 1 -- 172.21.15.2:0/3599085121 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f74e0051030 con 0x55bca97b62b0 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.719+0000 7f74f1de7740 10 client.17319 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.719+0000 7f74f1de7740 10 client.17319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.719+0000 7f74f1de7740 20 client.17319 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:12.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.719+0000 7f74f1de7740 1 --2- 172.21.15.2:0/3599085121 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bca97cf790 0x55bca97d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.719+0000 7f74f1de7740 4 client.17319 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:12.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.719+0000 7f74f1de7740 1 -- 172.21.15.2:0/3599085121 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bca945a720 con 0x55bca97cf790 2024-03-15T13:04:12.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.719+0000 7f74e7fff640 1 --2- 172.21.15.2:0/3599085121 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bca97cf790 0x55bca97d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.720+0000 7f74e7fff640 1 --2- 172.21.15.2:0/3599085121 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bca97cf790 0x55bca97d1bd0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14868030427a363a server_cookie=efc4d65c889b1451 in_seq=0 out_seq=0 2024-03-15T13:04:12.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.720+0000 7f74c7fff640 10 client.17319 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:12.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.720+0000 7f74c7fff640 1 -- 172.21.15.2:0/3599085121 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bca945a720 con 0x55bca97cf790 2024-03-15T13:04:12.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.720+0000 7f74c7fff640 10 client.17319 handle_command_reply: tid=0 2024-03-15T13:04:12.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.721+0000 7f74f1de7740 1 client.17319 shutdown 2024-03-15T13:04:12.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.721+0000 7f74f1de7740 1 -- 172.21.15.2:0/3599085121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 msgr2=0x55bca97bc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.721+0000 7f74f1de7740 1 --2- 172.21.15.2:0/3599085121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97bc630 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f74e002c700 tx=0x7f74e0009870 comp rx=0 tx=0).stop 2024-03-15T13:04:12.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.721+0000 7f74f1de7740 1 -- 172.21.15.2:0/3599085121 shutdown_connections 2024-03-15T13:04:12.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.721+0000 7f74f1de7740 1 --2- 172.21.15.2:0/3599085121 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bca97cf790 0x55bca97d1bd0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.721+0000 7f74f1de7740 1 --2- 172.21.15.2:0/3599085121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97bf7b0 0x55bca97bd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.721+0000 7f74f1de7740 1 --2- 172.21.15.2:0/3599085121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 0x55bca97bcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.721+0000 7f74c7fff640 0 client.17319 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:12.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.721+0000 7f74f1de7740 1 --2- 172.21.15.2:0/3599085121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97bc630 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.721+0000 7f74f1de7740 1 -- 172.21.15.2:0/3599085121 >> 172.21.15.2:0/3599085121 conn(0x55bca97b0990 msgr2=0x55bca97b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:12.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.721+0000 7f74f1de7740 1 -- 172.21.15.2:0/3599085121 shutdown_connections 2024-03-15T13:04:12.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.721+0000 7f74f1de7740 1 -- 172.21.15.2:0/3599085121 wait complete. 2024-03-15T13:04:12.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.721+0000 7f74f1de7740 20 client.17319 trim_cache size 0 max 16384 2024-03-15T13:04:12.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74f1de7740 1 Processor -- start 2024-03-15T13:04:12.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74f1de7740 1 -- start start 2024-03-15T13:04:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74f1de7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97bed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74f1de7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 0x55bca97bced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74f1de7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97bf7b0 0x55bca97caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74f1de7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca97b01d0 con 0x55bca97b62b0 2024-03-15T13:04:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74f1de7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca97ac2b0 con 0x55bca97bf7b0 2024-03-15T13:04:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74f1de7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca97bab80 con 0x55bca97b6bd0 2024-03-15T13:04:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74ecec8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74ecec8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97bed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42034/0 (socket says 172.21.15.2:42034) 2024-03-15T13:04:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74ecec8640 1 -- 172.21.15.2:0/2515386030 learned_addr learned my addr 172.21.15.2:0/2515386030 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74e7fff640 1 --2- 172.21.15.2:0/2515386030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 0x55bca97bced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74ed6c9640 1 --2- 172.21.15.2:0/2515386030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97bf7b0 0x55bca97caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74ecec8640 1 -- 172.21.15.2:0/2515386030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 msgr2=0x55bca97bced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74ecec8640 1 --2- 172.21.15.2:0/2515386030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 0x55bca97bced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74ecec8640 1 -- 172.21.15.2:0/2515386030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97bf7b0 msgr2=0x55bca97caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74ecec8640 1 --2- 172.21.15.2:0/2515386030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97bf7b0 0x55bca97caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.737+0000 7f74ecec8640 1 -- 172.21.15.2:0/2515386030 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bca96b8ac0 con 0x55bca97b62b0 2024-03-15T13:04:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.738+0000 7f74ecec8640 1 --2- 172.21.15.2:0/2515386030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97bed60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009bf0 tx=0x7f74e0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a486d11c8f47a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.738+0000 7f74de7fc640 1 -- 172.21.15.2:0/2515386030 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0053560 con 0x55bca97b62b0 2024-03-15T13:04:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.738+0000 7f74de7fc640 1 -- 172.21.15.2:0/2515386030 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f74e005c050 con 0x55bca97b62b0 2024-03-15T13:04:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.738+0000 7f74de7fc640 1 -- 172.21.15.2:0/2515386030 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0053a70 con 0x55bca97b62b0 2024-03-15T13:04:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.738+0000 7f74f1de7740 1 -- 172.21.15.2:0/2515386030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 msgr2=0x55bca97bed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.738+0000 7f74f1de7740 1 --2- 172.21.15.2:0/2515386030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97bed60 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009bf0 tx=0x7f74e0009800 comp rx=0 tx=0).stop 2024-03-15T13:04:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.738+0000 7f74f1de7740 1 -- 172.21.15.2:0/2515386030 shutdown_connections 2024-03-15T13:04:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.738+0000 7f74f1de7740 1 --2- 172.21.15.2:0/2515386030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97bf7b0 0x55bca97caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.738+0000 7f74f1de7740 1 --2- 172.21.15.2:0/2515386030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b6bd0 0x55bca97bced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.738+0000 7f74f1de7740 1 --2- 172.21.15.2:0/2515386030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97b62b0 0x55bca97bed60 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.738+0000 7f74f1de7740 1 -- 172.21.15.2:0/2515386030 >> 172.21.15.2:0/2515386030 conn(0x55bca97b0990 msgr2=0x55bca97b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.738+0000 7f74f1de7740 1 -- 172.21.15.2:0/2515386030 shutdown_connections 2024-03-15T13:04:12.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.738+0000 7f74f1de7740 1 -- 172.21.15.2:0/2515386030 wait complete. 2024-03-15T13:04:12.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.739+0000 7f74f1de7740 1 -- start start 2024-03-15T13:04:12.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.739+0000 7f74f1de7740 1 Processor -- start 2024-03-15T13:04:12.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.739+0000 7f74f1de7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b62b0 0x55bca97c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.739+0000 7f74f1de7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97b6bd0 0x55bca97c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.739+0000 7f74f1de7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97bf7b0 0x55bca982a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.739+0000 7f74f1de7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca97c3a80 con 0x55bca97bf7b0 2024-03-15T13:04:12.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.739+0000 7f74f1de7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca97c3bf0 con 0x55bca97b6bd0 2024-03-15T13:04:12.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.739+0000 7f74f1de7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca982a5d0 con 0x55bca97b62b0 2024-03-15T13:04:12.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.739+0000 7f74f1de7740 10 client.0 _ll_register_callbacks cb 0x7f74e0043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:12.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.739+0000 7f74ed6c9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97bf7b0 0x55bca982a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.739+0000 7f74ed6c9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97bf7b0 0x55bca982a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42046/0 (socket says 172.21.15.2:42046) 2024-03-15T13:04:12.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.739+0000 7f74ed6c9640 1 -- 172.21.15.2:0/186793053 learned_addr learned my addr 172.21.15.2:0/186793053 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:12.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.739+0000 7f74e7fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97b6bd0 0x55bca97c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.739+0000 7f74ecec8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b62b0 0x55bca97c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74ed6c9640 1 -- 172.21.15.2:0/186793053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b62b0 msgr2=0x55bca97c2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74ed6c9640 1 --2- 172.21.15.2:0/186793053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b62b0 0x55bca97c2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74ed6c9640 1 -- 172.21.15.2:0/186793053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97b6bd0 msgr2=0x55bca97c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74ed6c9640 1 --2- 172.21.15.2:0/186793053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97b6bd0 0x55bca97c32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74ed6c9640 1 -- 172.21.15.2:0/186793053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bca971c610 con 0x55bca97bf7b0 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74ed6c9640 1 --2- 172.21.15.2:0/186793053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97bf7b0 0x55bca982a060 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f74d0013300 tx=0x7f74d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa2239b4b174ad5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74c7fff640 1 -- 172.21.15.2:0/186793053 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d0027070 con 0x55bca97bf7b0 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74c7fff640 1 -- 172.21.15.2:0/186793053 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f74d0013330 con 0x55bca97bf7b0 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74f1de7740 1 -- 172.21.15.2:0/186793053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74e00345a0 con 0x55bca97bf7b0 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74c7fff640 1 -- 172.21.15.2:0/186793053 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74d00134f0 con 0x55bca97bf7b0 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74f1de7740 10 client.17325 fetch_fsmap learned FSMap version 41 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74f1de7740 1 -- 172.21.15.2:0/186793053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55bca96de6a0 con 0x55bca97bf7b0 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74c7fff640 1 -- 172.21.15.2:0/186793053 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f74e00345a0 con 0x55bca97bf7b0 2024-03-15T13:04:12.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74c7fff640 1 -- 172.21.15.2:0/186793053 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7978+0+0 (secure 0 0 0) 0x7f74d0022bb0 con 0x55bca97bf7b0 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74f1de7740 10 client.17325 fetch_fsmap finished waiting for FSMap version 41 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74f1de7740 10 client.17325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74f1de7740 20 client.17325 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74f1de7740 1 --2- 172.21.15.2:0/186793053 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bca982bf20 0x55bca982e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74f1de7740 4 client.17325 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.740+0000 7f74f1de7740 1 -- 172.21.15.2:0/186793053 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bca94a9890 con 0x55bca982bf20 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74ecec8640 1 --2- 172.21.15.2:0/186793053 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bca982bf20 0x55bca982e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74ecec8640 1 --2- 172.21.15.2:0/186793053 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bca982bf20 0x55bca982e340 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a65b652edb34c2c5 server_cookie=331d410cbebf2c7d in_seq=0 out_seq=0 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74c7fff640 10 client.17325 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74c7fff640 1 -- 172.21.15.2:0/186793053 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bca94a9890 con 0x55bca982bf20 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74c7fff640 10 client.17325 handle_command_reply: tid=0 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74f1de7740 1 client.17325 shutdown 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74f1de7740 1 -- 172.21.15.2:0/186793053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97bf7b0 msgr2=0x55bca982a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74f1de7740 1 --2- 172.21.15.2:0/186793053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97bf7b0 0x55bca982a060 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f74d0013300 tx=0x7f74d00046c0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74f1de7740 1 -- 172.21.15.2:0/186793053 shutdown_connections 2024-03-15T13:04:12.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74f1de7740 1 --2- 172.21.15.2:0/186793053 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bca982bf20 0x55bca982e340 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74f1de7740 1 --2- 172.21.15.2:0/186793053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bca97bf7b0 0x55bca982a060 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74f1de7740 1 --2- 172.21.15.2:0/186793053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bca97b6bd0 0x55bca97c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74f1de7740 1 --2- 172.21.15.2:0/186793053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bca97b62b0 0x55bca97c2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74c7fff640 0 client.17325 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74f1de7740 1 -- 172.21.15.2:0/186793053 >> 172.21.15.2:0/186793053 conn(0x55bca97b0990 msgr2=0x55bca97af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74f1de7740 1 -- 172.21.15.2:0/186793053 shutdown_connections 2024-03-15T13:04:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.741+0000 7f74f1de7740 1 -- 172.21.15.2:0/186793053 wait complete. 2024-03-15T13:04:12.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.742+0000 7f74f1de7740 20 client.17325 trim_cache size 0 max 16384 2024-03-15T13:04:12.733 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:04:12.733 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T13:04:12.733 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:04:12.733 INFO:teuthology.orchestra.run.smithi002.stdout: "abe43775-4f89-4322-b928-f44766dfc449": { 2024-03-15T13:04:12.734 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:04:12.734 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "abe43775-4f89-4322-b928-f44766dfc449", 2024-03-15T13:04:12.734 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:04:12.734 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:04:12.734 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:04:12.734 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:04:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.742+0000 7f74ef12c640 1 -- 172.21.15.2:0/3087489908 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f74d40a1ba0 msgr2=0x7f74d40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.742+0000 7f74ef12c640 1 --2- 172.21.15.2:0/3087489908 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f74d40a1ba0 0x7f74d40a4060 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f74d00097d0 tx=0x7f74d0004730 comp rx=0 tx=0).stop 2024-03-15T13:04:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.742+0000 7f74ef12c640 1 -- 172.21.15.2:0/3087489908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74e8101e20 msgr2=0x7f74e819dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.742+0000 7f74ef12c640 1 --2- 172.21.15.2:0/3087489908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74e8101e20 0x7f74e819dcf0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f74d800e9f0 tx=0x7f74d800edf0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.742+0000 7f74ef12c640 1 -- 172.21.15.2:0/3087489908 shutdown_connections 2024-03-15T13:04:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.742+0000 7f74ef12c640 1 --2- 172.21.15.2:0/3087489908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e8107500 0x7f74e8197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.742+0000 7f74ef12c640 1 --2- 172.21.15.2:0/3087489908 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f74d40a1ba0 0x7f74d40a4060 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.742+0000 7f74ef12c640 1 --2- 172.21.15.2:0/3087489908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74e8101e20 0x7f74e819dcf0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.742+0000 7f74ef12c640 1 --2- 172.21.15.2:0/3087489908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e8101460 0x7f74e819d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:12.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.742+0000 7f74ef12c640 1 -- 172.21.15.2:0/3087489908 >> 172.21.15.2:0/3087489908 conn(0x7f74e80fcf90 msgr2=0x7f74e80fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:12.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.742+0000 7f74ef12c640 1 -- 172.21.15.2:0/3087489908 shutdown_connections 2024-03-15T13:04:12.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:12.742+0000 7f74ef12c640 1 -- 172.21.15.2:0/3087489908 wait complete. 2024-03-15T13:04:12.738 INFO:tasks.cephfs.filesystem:scrub status for tag:abe43775-4f89-4322-b928-f44766dfc449 - {'path': '/', 'tag': 'abe43775-4f89-4322-b928-f44766dfc449', 'options': 'recursive,force'} 2024-03-15T13:04:13.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:13 smithi149 ceph-mon[35927]: pgmap v162: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:04:13.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:13 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:04:13.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:13 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:04:13.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:13 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:04:13.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:13 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:04:14.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:13 smithi002 ceph-mon[32060]: pgmap v162: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:04:14.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:13 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:04:14.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:13 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:04:14.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:13 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:04:14.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:13 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:04:14.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:13 smithi184 ceph-mon[35993]: pgmap v162: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:04:14.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:13 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:04:14.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:13 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:04:14.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:13 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:04:14.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:13 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:04:15.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:15 smithi149 ceph-mon[35927]: pgmap v163: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:04:15.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:15 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:04:16.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:15 smithi002 ceph-mon[32060]: pgmap v163: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:04:16.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:15 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:04:16.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:15 smithi184 ceph-mon[35993]: pgmap v163: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:04:16.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:15 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:04:17.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:17 smithi149 ceph-mon[35927]: pgmap v164: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:04:18.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:17 smithi002 ceph-mon[32060]: pgmap v164: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:04:18.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:17 smithi184 ceph-mon[35993]: pgmap v164: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:04:19.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:19 smithi149 ceph-mon[35927]: pgmap v165: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-15T13:04:20.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:19 smithi002 ceph-mon[32060]: pgmap v165: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-15T13:04:20.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:19 smithi184 ceph-mon[35993]: pgmap v165: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-15T13:04:21.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:21 smithi149 ceph-mon[35927]: pgmap v166: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-15T13:04:22.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:21 smithi002 ceph-mon[32060]: pgmap v166: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-15T13:04:22.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:21 smithi184 ceph-mon[35993]: pgmap v166: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-15T13:04:23.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:23 smithi149 ceph-mon[35927]: pgmap v167: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:04:24.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:23 smithi002 ceph-mon[32060]: pgmap v167: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:04:24.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:23 smithi184 ceph-mon[35993]: pgmap v167: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:04:25.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:25 smithi149 ceph-mon[35927]: pgmap v168: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:04:25.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:25 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:26.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:25 smithi002 ceph-mon[32060]: pgmap v168: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:04:26.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:25 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:26.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:25 smithi184 ceph-mon[35993]: pgmap v168: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:04:26.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:25 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:27.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:27 smithi149 ceph-mon[35927]: pgmap v169: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:04:28.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:27 smithi002 ceph-mon[32060]: pgmap v169: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:04:28.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:27 smithi184 ceph-mon[35993]: pgmap v169: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:04:29.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:29 smithi149 ceph-mon[35927]: pgmap v170: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:30.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:29 smithi002 ceph-mon[32060]: pgmap v170: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:30.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:29 smithi184 ceph-mon[35993]: pgmap v170: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:31.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:31 smithi149 ceph-mon[35927]: pgmap v171: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:04:32.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:31 smithi002 ceph-mon[32060]: pgmap v171: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:04:32.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:31 smithi184 ceph-mon[35993]: pgmap v171: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:04:33.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:33 smithi149 ceph-mon[35927]: pgmap v172: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:34.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:33 smithi002 ceph-mon[32060]: pgmap v172: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:34.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:33 smithi184 ceph-mon[35993]: pgmap v172: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:35.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:35 smithi149 ceph-mon[35927]: pgmap v173: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:35.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:35 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:35.860 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:35 smithi184 ceph-mon[35993]: pgmap v173: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:35.860 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:35 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:36.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:35 smithi002 ceph-mon[32060]: pgmap v173: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:36.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:35 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:37.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:37 smithi149 ceph-mon[35927]: pgmap v174: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:04:38.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:37 smithi002 ceph-mon[32060]: pgmap v174: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:04:38.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:37 smithi184 ceph-mon[35993]: pgmap v174: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:04:39.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:39 smithi149 ceph-mon[35927]: pgmap v175: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:40.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:39 smithi002 ceph-mon[32060]: pgmap v175: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:40.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:39 smithi184 ceph-mon[35993]: pgmap v175: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:41.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:40 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:41.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:40 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:41.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:40 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:42.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:41 smithi002 ceph-mon[32060]: pgmap v176: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:04:42.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:41 smithi184 ceph-mon[35993]: pgmap v176: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:04:42.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:41 smithi149 ceph-mon[35927]: pgmap v176: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:04:42.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:04:42.739 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:04:42.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.872+0000 7fc2f36f4640 1 -- 172.21.15.2:0/3062900913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2ec103f30 msgr2=0x7fc2ec1043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:42.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.872+0000 7fc2f36f4640 1 --2- 172.21.15.2:0/3062900913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2ec103f30 0x7fc2ec1043e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009ca0 tx=0x7fc2e402c0f0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.872+0000 7fc2f36f4640 1 -- 172.21.15.2:0/3062900913 shutdown_connections 2024-03-15T13:04:42.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.872+0000 7fc2f36f4640 1 --2- 172.21.15.2:0/3062900913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2ec103f30 0x7fc2ec1043e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.872+0000 7fc2f36f4640 1 --2- 172.21.15.2:0/3062900913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2ec06a060 0x7fc2ec06a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.872+0000 7fc2f36f4640 1 --2- 172.21.15.2:0/3062900913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2ec106430 0x7fc2ec069b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.872+0000 7fc2f36f4640 1 -- 172.21.15.2:0/3062900913 >> 172.21.15.2:0/3062900913 conn(0x7fc2ec077190 msgr2=0x7fc2ec0775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:42.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.872+0000 7fc2f36f4640 1 -- 172.21.15.2:0/3062900913 shutdown_connections 2024-03-15T13:04:42.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.872+0000 7fc2f36f4640 1 -- 172.21.15.2:0/3062900913 wait complete. 2024-03-15T13:04:42.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.872+0000 7fc2f36f4640 1 Processor -- start 2024-03-15T13:04:42.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f36f4640 1 -- start start 2024-03-15T13:04:42.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f36f4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2ec06a060 0x7fc2ec19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f36f4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2ec103f30 0x7fc2ec19e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f36f4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2ec106430 0x7fc2ec1a2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f36f4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec1a2e20 con 0x7fc2ec103f30 2024-03-15T13:04:42.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f36f4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec1a2f90 con 0x7fc2ec06a060 2024-03-15T13:04:42.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f36f4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec1a3150 con 0x7fc2ec106430 2024-03-15T13:04:42.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f0c8f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2ec103f30 0x7fc2ec19e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f0c8f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2ec103f30 0x7fc2ec19e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33054/0 (socket says 172.21.15.2:33054) 2024-03-15T13:04:42.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f1490640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2ec06a060 0x7fc2ec19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/1375145292 learned_addr learned my addr 172.21.15.2:0/1375145292 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:42.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f1c91640 1 --2- 172.21.15.2:0/1375145292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2ec106430 0x7fc2ec1a2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/1375145292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2ec106430 msgr2=0x7fc2ec1a2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:42.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f0c8f640 1 --2- 172.21.15.2:0/1375145292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2ec106430 0x7fc2ec1a2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/1375145292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2ec06a060 msgr2=0x7fc2ec19dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:42.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f0c8f640 1 --2- 172.21.15.2:0/1375145292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2ec06a060 0x7fc2ec19dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/1375145292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e4009900 con 0x7fc2ec103f30 2024-03-15T13:04:42.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.873+0000 7fc2f0c8f640 1 --2- 172.21.15.2:0/1375145292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2ec103f30 0x7fc2ec19e4f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00e9f0 tx=0x7fc2dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:42.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.874+0000 7fc2e27fc640 1 -- 172.21.15.2:0/1375145292 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc009700 con 0x7fc2ec103f30 2024-03-15T13:04:42.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.874+0000 7fc2f36f4640 1 -- 172.21.15.2:0/1375145292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ec1a3430 con 0x7fc2ec103f30 2024-03-15T13:04:42.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.874+0000 7fc2e27fc640 1 -- 172.21.15.2:0/1375145292 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc2dc00cd80 con 0x7fc2ec103f30 2024-03-15T13:04:42.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.874+0000 7fc2e27fc640 1 -- 172.21.15.2:0/1375145292 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc018920 con 0x7fc2ec103f30 2024-03-15T13:04:42.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.874+0000 7fc2f36f4640 1 -- 172.21.15.2:0/1375145292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ec1aacb0 con 0x7fc2ec103f30 2024-03-15T13:04:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.876+0000 7fc2e27fc640 1 -- 172.21.15.2:0/1375145292 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fc2dc018a80 con 0x7fc2ec103f30 2024-03-15T13:04:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.876+0000 7fc2f63ac740 1 Processor -- start 2024-03-15T13:04:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.876+0000 7fc2f63ac740 1 -- start start 2024-03-15T13:04:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.876+0000 7fc2e27fc640 1 --2- 172.21.15.2:0/1375145292 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc2d80a1ba0 0x7fc2d80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f63ac740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590602732b0 0x559060273690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f63ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559060273bd0 0x55906027c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f63ac740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55906027c7b0 0x55906027eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f63ac740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590601e2280 con 0x5590602732b0 2024-03-15T13:04:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f63ac740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590601df7b0 con 0x55906027c7b0 2024-03-15T13:04:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f63ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590601daf90 con 0x559060273bd0 2024-03-15T13:04:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f1c91640 1 --2- 172.21.15.2:0/1375145292 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc2d80a1ba0 0x7fc2d80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2e27fc640 1 -- 172.21.15.2:0/1375145292 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc2dc014070 con 0x7fc2ec103f30 2024-03-15T13:04:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f1490640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590602732b0 0x559060273690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f0c8f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559060273bd0 0x55906027c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f1490640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590602732b0 0x559060273690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33060/0 (socket says 172.21.15.2:33060) 2024-03-15T13:04:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f1490640 1 -- 172.21.15.2:0/799408523 learned_addr learned my addr 172.21.15.2:0/799408523 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f1c91640 1 --2- 172.21.15.2:0/799408523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55906027c7b0 0x55906027eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/799408523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55906027c7b0 msgr2=0x55906027eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f0c8f640 1 --2- 172.21.15.2:0/799408523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55906027c7b0 0x55906027eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/799408523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590602732b0 msgr2=0x559060273690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f0c8f640 1 --2- 172.21.15.2:0/799408523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590602732b0 0x559060273690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/799408523 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559060199570 con 0x559060273bd0 2024-03-15T13:04:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f1c91640 1 --2- 172.21.15.2:0/1375145292 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc2d80a1ba0 0x7fc2d80a4060 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4004af0 tx=0x7fc2e4005fa0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2f0c8f640 1 --2- 172.21.15.2:0/799408523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559060273bd0 0x55906027c270 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0cb270 tx=0x7fc2dc09a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac82601f14e2641a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.877+0000 7fc2d77fe640 1 -- 172.21.15.2:0/799408523 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc09a800 con 0x559060273bd0 2024-03-15T13:04:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.878+0000 7fc2d77fe640 1 -- 172.21.15.2:0/799408523 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc2dc09e8a0 con 0x559060273bd0 2024-03-15T13:04:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.878+0000 7fc2d77fe640 1 -- 172.21.15.2:0/799408523 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc010cb0 con 0x559060273bd0 2024-03-15T13:04:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.878+0000 7fc2f63ac740 1 -- 172.21.15.2:0/799408523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559060273bd0 msgr2=0x55906027c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.878+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/799408523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559060273bd0 0x55906027c270 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0cb270 tx=0x7fc2dc09a160 comp rx=0 tx=0).stop 2024-03-15T13:04:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.878+0000 7fc2f63ac740 1 -- 172.21.15.2:0/799408523 shutdown_connections 2024-03-15T13:04:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.878+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/799408523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55906027c7b0 0x55906027eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.878+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/799408523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559060273bd0 0x55906027c270 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.878+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/799408523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5590602732b0 0x559060273690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.878+0000 7fc2f63ac740 1 -- 172.21.15.2:0/799408523 >> 172.21.15.2:0/799408523 conn(0x55906026d990 msgr2=0x55906026dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.878+0000 7fc2f63ac740 1 -- 172.21.15.2:0/799408523 shutdown_connections 2024-03-15T13:04:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.878+0000 7fc2f63ac740 1 -- 172.21.15.2:0/799408523 wait complete. 2024-03-15T13:04:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.879+0000 7fc2f63ac740 1 -- start start 2024-03-15T13:04:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.879+0000 7fc2f63ac740 1 Processor -- start 2024-03-15T13:04:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.879+0000 7fc2f63ac740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 0x559060279650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.879+0000 7fc2f63ac740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x559060279b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.879+0000 7fc2f63ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 0x55906027a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.879+0000 7fc2f63ac740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55905feae430 con 0x559060273bd0 2024-03-15T13:04:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.879+0000 7fc2f63ac740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55905fe6ae00 con 0x5590602732b0 2024-03-15T13:04:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.879+0000 7fc2f63ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55906015ffd0 con 0x55906027c7b0 2024-03-15T13:04:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.879+0000 7fc2f63ac740 10 client.0 _ll_register_callbacks cb 0x5590601b5fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.879+0000 7fc2f0c8f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x559060279b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.879+0000 7fc2f0c8f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x559060279b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33072/0 (socket says 172.21.15.2:33072) 2024-03-15T13:04:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.879+0000 7fc2f1490640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 0x559060279650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.879+0000 7fc2f1c91640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 0x55906027a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.879+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/1781735623 learned_addr learned my addr 172.21.15.2:0/1781735623 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/1781735623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 msgr2=0x55906027a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2f0c8f640 1 --2- 172.21.15.2:0/1781735623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 0x55906027a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/1781735623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 msgr2=0x559060279650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2f0c8f640 1 --2- 172.21.15.2:0/1781735623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 0x559060279650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/1781735623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ec1aacb0 con 0x559060273bd0 2024-03-15T13:04:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2f1490640 1 --2- 172.21.15.2:0/1781735623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 0x559060279650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2f0c8f640 1 --2- 172.21.15.2:0/1781735623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x559060279b90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0cb320 tx=0x7fc2dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8650ab539aff95f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2d4ff9640 1 -- 172.21.15.2:0/1781735623 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc0988a0 con 0x559060273bd0 2024-03-15T13:04:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2d4ff9640 1 -- 172.21.15.2:0/1781735623 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc2dc09a800 con 0x559060273bd0 2024-03-15T13:04:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2f63ac740 1 -- 172.21.15.2:0/1781735623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55905feda7e0 con 0x559060273bd0 2024-03-15T13:04:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2d4ff9640 1 -- 172.21.15.2:0/1781735623 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2dc0b3120 con 0x559060273bd0 2024-03-15T13:04:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2f63ac740 10 client.17337 fetch_fsmap learned FSMap version 45 2024-03-15T13:04:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2f63ac740 1 -- 172.21.15.2:0/1781735623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5590601d9610 con 0x559060273bd0 2024-03-15T13:04:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2d4ff9640 1 -- 172.21.15.2:0/1781735623 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc2dc0b7020 con 0x559060273bd0 2024-03-15T13:04:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2d4ff9640 1 -- 172.21.15.2:0/1781735623 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fc2dc0965e0 con 0x559060273bd0 2024-03-15T13:04:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2f63ac740 10 client.17337 fetch_fsmap finished waiting for FSMap version 45 2024-03-15T13:04:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2f63ac740 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2f63ac740 20 client.17337 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.880+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/1781735623 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55906028c530 0x55906028e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.881+0000 7fc2f63ac740 4 client.17337 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.881+0000 7fc2f63ac740 1 -- 172.21.15.2:0/1781735623 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55905ff17720 con 0x55906028c530 2024-03-15T13:04:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.881+0000 7fc2f1490640 1 --2- 172.21.15.2:0/1781735623 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55906028c530 0x55906028e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.881+0000 7fc2d4ff9640 10 client.17337 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.881+0000 7fc2f1490640 1 --2- 172.21.15.2:0/1781735623 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55906028c530 0x55906028e950 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f1660fcd03bec9 server_cookie=2322d6807bd669fc in_seq=0 out_seq=0 2024-03-15T13:04:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.882+0000 7fc2d4ff9640 1 -- 172.21.15.2:0/1781735623 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55905ff17720 con 0x55906028c530 2024-03-15T13:04:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.882+0000 7fc2d4ff9640 10 client.17337 handle_command_reply: tid=0 2024-03-15T13:04:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.882+0000 7fc2f63ac740 1 client.17337 shutdown 2024-03-15T13:04:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.882+0000 7fc2f63ac740 1 -- 172.21.15.2:0/1781735623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 msgr2=0x559060279b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.882+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/1781735623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x559060279b90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0cb320 tx=0x7fc2dc000bc0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.882+0000 7fc2f63ac740 1 -- 172.21.15.2:0/1781735623 shutdown_connections 2024-03-15T13:04:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.882+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/1781735623 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55906028c530 0x55906028e950 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.882+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/1781735623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 0x55906027a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.882+0000 7fc2d4ff9640 0 client.17337 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.882+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/1781735623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x559060279b90 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.882+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/1781735623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 0x559060279650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.882+0000 7fc2f63ac740 1 -- 172.21.15.2:0/1781735623 >> 172.21.15.2:0/1781735623 conn(0x55906026d990 msgr2=0x55906026f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.882+0000 7fc2f63ac740 1 -- 172.21.15.2:0/1781735623 shutdown_connections 2024-03-15T13:04:42.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.882+0000 7fc2f63ac740 1 -- 172.21.15.2:0/1781735623 wait complete. 2024-03-15T13:04:42.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.882+0000 7fc2f63ac740 20 client.17337 trim_cache size 0 max 16384 2024-03-15T13:04:42.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f63ac740 1 Processor -- start 2024-03-15T13:04:42.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f63ac740 1 -- start start 2024-03-15T13:04:42.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f63ac740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 0x55906027bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f63ac740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x559060279ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f63ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 0x559060287a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f63ac740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55906028f020 con 0x559060273bd0 2024-03-15T13:04:42.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f63ac740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559060277ba0 con 0x5590602732b0 2024-03-15T13:04:42.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f63ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559060267e20 con 0x55906027c7b0 2024-03-15T13:04:42.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f0c8f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x559060279ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f0c8f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x559060279ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33084/0 (socket says 172.21.15.2:33084) 2024-03-15T13:04:42.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/2039360690 learned_addr learned my addr 172.21.15.2:0/2039360690 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f1490640 1 --2- 172.21.15.2:0/2039360690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 0x55906027bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f1c91640 1 --2- 172.21.15.2:0/2039360690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 0x559060287a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/2039360690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 msgr2=0x559060287a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f0c8f640 1 --2- 172.21.15.2:0/2039360690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 0x559060287a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/2039360690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 msgr2=0x55906027bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f0c8f640 1 --2- 172.21.15.2:0/2039360690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 0x55906027bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/2039360690 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559060175ac0 con 0x559060273bd0 2024-03-15T13:04:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f1c91640 1 --2- 172.21.15.2:0/2039360690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 0x559060287a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:04:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2f0c8f640 1 --2- 172.21.15.2:0/2039360690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x559060279ef0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00c6a0 tx=0x7fc2dc09a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db38004f065bf530 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2d6ffd640 1 -- 172.21.15.2:0/2039360690 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc0988a0 con 0x559060273bd0 2024-03-15T13:04:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2d6ffd640 1 -- 172.21.15.2:0/2039360690 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc2dc0b7030 con 0x559060273bd0 2024-03-15T13:04:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.899+0000 7fc2d6ffd640 1 -- 172.21.15.2:0/2039360690 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc0ba0e0 con 0x559060273bd0 2024-03-15T13:04:42.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.900+0000 7fc2f63ac740 1 -- 172.21.15.2:0/2039360690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 msgr2=0x559060279ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:42.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.900+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/2039360690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x559060279ef0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00c6a0 tx=0x7fc2dc09a160 comp rx=0 tx=0).stop 2024-03-15T13:04:42.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.900+0000 7fc2f63ac740 1 -- 172.21.15.2:0/2039360690 shutdown_connections 2024-03-15T13:04:42.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.900+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/2039360690 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 0x559060287a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.900+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/2039360690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x559060279ef0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.900+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/2039360690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 0x55906027bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.900+0000 7fc2f63ac740 1 -- 172.21.15.2:0/2039360690 >> 172.21.15.2:0/2039360690 conn(0x55906026d990 msgr2=0x55906026ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:42.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.900+0000 7fc2f63ac740 1 -- 172.21.15.2:0/2039360690 shutdown_connections 2024-03-15T13:04:42.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.900+0000 7fc2f63ac740 1 -- 172.21.15.2:0/2039360690 wait complete. 2024-03-15T13:04:42.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.900+0000 7fc2f63ac740 1 -- start start 2024-03-15T13:04:42.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.900+0000 7fc2f63ac740 1 Processor -- start 2024-03-15T13:04:42.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f63ac740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 0x559060275ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f63ac740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x5590602763e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f63ac740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 0x559060276920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f63ac740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590602e6e50 con 0x559060273bd0 2024-03-15T13:04:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f63ac740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590602e6fc0 con 0x5590602732b0 2024-03-15T13:04:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f63ac740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590602e7180 con 0x55906027c7b0 2024-03-15T13:04:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f63ac740 10 client.0 _ll_register_callbacks cb 0x7fc2e8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f0c8f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x5590602763e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f1490640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 0x559060275ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f0c8f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x5590602763e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33088/0 (socket says 172.21.15.2:33088) 2024-03-15T13:04:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/2040319979 learned_addr learned my addr 172.21.15.2:0/2040319979 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f1c91640 1 --2- 172.21.15.2:0/2040319979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 0x559060276920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/2040319979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 msgr2=0x559060276920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f0c8f640 1 --2- 172.21.15.2:0/2040319979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 0x559060276920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/2040319979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 msgr2=0x559060275ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:42.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f0c8f640 1 --2- 172.21.15.2:0/2040319979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 0x559060275ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f0c8f640 1 -- 172.21.15.2:0/2040319979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590601d9610 con 0x559060273bd0 2024-03-15T13:04:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f1490640 1 --2- 172.21.15.2:0/2040319979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 0x559060275ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2f0c8f640 1 --2- 172.21.15.2:0/2040319979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x5590602763e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc09a790 tx=0x7fc2dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ffc6fdbec6eb6c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.901+0000 7fc2d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.902+0000 7fc2d4ff9640 1 -- 172.21.15.2:0/2040319979 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc099e50 con 0x559060273bd0 2024-03-15T13:04:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.902+0000 7fc2d4ff9640 1 -- 172.21.15.2:0/2040319979 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc2dc0b8080 con 0x559060273bd0 2024-03-15T13:04:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.902+0000 7fc2f63ac740 1 -- 172.21.15.2:0/2040319979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2dc09e8a0 con 0x559060273bd0 2024-03-15T13:04:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.902+0000 7fc2d4ff9640 1 -- 172.21.15.2:0/2040319979 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2dc099fb0 con 0x559060273bd0 2024-03-15T13:04:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.902+0000 7fc2f63ac740 10 client.17343 fetch_fsmap learned FSMap version 45 2024-03-15T13:04:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.902+0000 7fc2f63ac740 1 -- 172.21.15.2:0/2040319979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55906019b6a0 con 0x559060273bd0 2024-03-15T13:04:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.902+0000 7fc2d4ff9640 1 -- 172.21.15.2:0/2040319979 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc2dc09e8a0 con 0x559060273bd0 2024-03-15T13:04:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.902+0000 7fc2d4ff9640 1 -- 172.21.15.2:0/2040319979 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fc2dc09f7c0 con 0x559060273bd0 2024-03-15T13:04:42.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.902+0000 7fc2f63ac740 10 client.17343 fetch_fsmap finished waiting for FSMap version 45 2024-03-15T13:04:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.902+0000 7fc2f63ac740 10 client.17343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.902+0000 7fc2f63ac740 20 client.17343 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.902+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/2040319979 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5590602e8490 0x5590602ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.902+0000 7fc2f63ac740 4 client.17343 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:04:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.902+0000 7fc2f63ac740 1 -- 172.21.15.2:0/2040319979 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55905ff66890 con 0x5590602e8490 2024-03-15T13:04:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.902+0000 7fc2f1490640 1 --2- 172.21.15.2:0/2040319979 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5590602e8490 0x5590602ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2f1490640 1 --2- 172.21.15.2:0/2040319979 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5590602e8490 0x5590602ea880 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc2ca3843c96e003 server_cookie=b1c5e1e2a5ccf683 in_seq=0 out_seq=0 2024-03-15T13:04:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2d4ff9640 10 client.17343 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2d4ff9640 1 -- 172.21.15.2:0/2040319979 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55905ff66890 con 0x5590602e8490 2024-03-15T13:04:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2d4ff9640 10 client.17343 handle_command_reply: tid=0 2024-03-15T13:04:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2f63ac740 1 client.17343 shutdown 2024-03-15T13:04:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2f63ac740 1 -- 172.21.15.2:0/2040319979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 msgr2=0x5590602763e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/2040319979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x5590602763e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc09a790 tx=0x7fc2dc000bc0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2f63ac740 1 -- 172.21.15.2:0/2040319979 shutdown_connections 2024-03-15T13:04:42.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/2040319979 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5590602e8490 0x5590602ea880 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/2040319979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55906027c7b0 0x559060276920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2d4ff9640 0 client.17343 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:42.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/2040319979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559060273bd0 0x5590602763e0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2f63ac740 1 --2- 172.21.15.2:0/2040319979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590602732b0 0x559060275ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2f63ac740 1 -- 172.21.15.2:0/2040319979 >> 172.21.15.2:0/2040319979 conn(0x55906026d990 msgr2=0x55906026eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:42.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2f63ac740 1 -- 172.21.15.2:0/2040319979 shutdown_connections 2024-03-15T13:04:42.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2f63ac740 1 -- 172.21.15.2:0/2040319979 wait complete. 2024-03-15T13:04:42.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.903+0000 7fc2f63ac740 20 client.17343 trim_cache size 0 max 16384 2024-03-15T13:04:42.897 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:04:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:04:42.897 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:04:42.897 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:04:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.904+0000 7fc2f36f4640 1 -- 172.21.15.2:0/1375145292 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc2d80a1ba0 msgr2=0x7fc2d80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.904+0000 7fc2f36f4640 1 --2- 172.21.15.2:0/1375145292 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc2d80a1ba0 0x7fc2d80a4060 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4004af0 tx=0x7fc2e4005fa0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.904+0000 7fc2f36f4640 1 -- 172.21.15.2:0/1375145292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2ec103f30 msgr2=0x7fc2ec19e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.904+0000 7fc2f36f4640 1 --2- 172.21.15.2:0/1375145292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2ec103f30 0x7fc2ec19e4f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00e9f0 tx=0x7fc2dc00eeb0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.904+0000 7fc2f36f4640 1 -- 172.21.15.2:0/1375145292 shutdown_connections 2024-03-15T13:04:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.904+0000 7fc2f36f4640 1 --2- 172.21.15.2:0/1375145292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2ec106430 0x7fc2ec1a2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.904+0000 7fc2f36f4640 1 --2- 172.21.15.2:0/1375145292 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc2d80a1ba0 0x7fc2d80a4060 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.904+0000 7fc2f36f4640 1 --2- 172.21.15.2:0/1375145292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2ec103f30 0x7fc2ec19e4f0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.904+0000 7fc2f36f4640 1 --2- 172.21.15.2:0/1375145292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2ec06a060 0x7fc2ec19dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.904+0000 7fc2f36f4640 1 -- 172.21.15.2:0/1375145292 >> 172.21.15.2:0/1375145292 conn(0x7fc2ec077190 msgr2=0x7fc2ec102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.904+0000 7fc2f36f4640 1 -- 172.21.15.2:0/1375145292 shutdown_connections 2024-03-15T13:04:42.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:42.904+0000 7fc2f36f4640 1 -- 172.21.15.2:0/1375145292 wait complete. 2024-03-15T13:04:42.899 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:42 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:42.900 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:04:42.900 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:04:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.018+0000 7f7925064640 1 -- 172.21.15.2:0/2996271243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7920106880 msgr2=0x7f792010e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.018+0000 7f7925064640 1 --2- 172.21.15.2:0/2996271243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7920106880 0x7f792010e400 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7918009ca0 tx=0x7f791802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.018+0000 7f7925064640 1 -- 172.21.15.2:0/2996271243 shutdown_connections 2024-03-15T13:04:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.018+0000 7f7925064640 1 --2- 172.21.15.2:0/2996271243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7920106880 0x7f792010e400 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.018+0000 7f7925064640 1 --2- 172.21.15.2:0/2996271243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f792006a7a0 0x7f7920106150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.018+0000 7f7925064640 1 --2- 172.21.15.2:0/2996271243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7920069df0 0x7f792006a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.018+0000 7f7925064640 1 -- 172.21.15.2:0/2996271243 >> 172.21.15.2:0/2996271243 conn(0x7f7920077490 msgr2=0x7f79200778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.018+0000 7f7925064640 1 -- 172.21.15.2:0/2996271243 shutdown_connections 2024-03-15T13:04:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.018+0000 7f7925064640 1 -- 172.21.15.2:0/2996271243 wait complete. 2024-03-15T13:04:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.018+0000 7f7925064640 1 Processor -- start 2024-03-15T13:04:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7925064640 1 -- start start 2024-03-15T13:04:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7925064640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7920069df0 0x7f792019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7925064640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7920106880 0x7f792019dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7925064640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7920197820 0x7f7920197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7925064640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79201982d0 con 0x7f7920106880 2024-03-15T13:04:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7925064640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7920198440 con 0x7f7920197820 2024-03-15T13:04:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7925064640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79201985b0 con 0x7f7920069df0 2024-03-15T13:04:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7917fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7920106880 0x7f792019dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7917fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7920106880 0x7f792019dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33098/0 (socket says 172.21.15.2:33098) 2024-03-15T13:04:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7917fff640 1 -- 172.21.15.2:0/2946891282 learned_addr learned my addr 172.21.15.2:0/2946891282 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f791ed9d640 1 --2- 172.21.15.2:0/2946891282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7920069df0 0x7f792019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f791f59e640 1 --2- 172.21.15.2:0/2946891282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7920197820 0x7f7920197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7917fff640 1 -- 172.21.15.2:0/2946891282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7920069df0 msgr2=0x7f792019d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7917fff640 1 --2- 172.21.15.2:0/2946891282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7920069df0 0x7f792019d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7917fff640 1 -- 172.21.15.2:0/2946891282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7920197820 msgr2=0x7f7920197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7917fff640 1 --2- 172.21.15.2:0/2946891282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7920197820 0x7f7920197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7917fff640 1 -- 172.21.15.2:0/2946891282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7918009900 con 0x7f7920106880 2024-03-15T13:04:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7917fff640 1 --2- 172.21.15.2:0/2946891282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7920106880 0x7f792019dbf0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f790800e9c0 tx=0x7f790800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f791cd99640 1 -- 172.21.15.2:0/2946891282 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908009700 con 0x7f7920106880 2024-03-15T13:04:43.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f791cd99640 1 -- 172.21.15.2:0/2946891282 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f790800cd80 con 0x7f7920106880 2024-03-15T13:04:43.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f7925064640 1 -- 172.21.15.2:0/2946891282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79201a4560 con 0x7f7920106880 2024-03-15T13:04:43.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.019+0000 7f791cd99640 1 -- 172.21.15.2:0/2946891282 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908010430 con 0x7f7920106880 2024-03-15T13:04:43.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.020+0000 7f7925064640 1 -- 172.21.15.2:0/2946891282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79201a4a40 con 0x7f7920106880 2024-03-15T13:04:43.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.021+0000 7f7925064640 1 -- 172.21.15.2:0/2946891282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7920102c10 con 0x7f7920106880 2024-03-15T13:04:43.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.021+0000 7f791cd99640 1 -- 172.21.15.2:0/2946891282 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f7908004ff0 con 0x7f7920106880 2024-03-15T13:04:43.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.022+0000 7f791cd99640 1 --2- 172.21.15.2:0/2946891282 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f790c0a1ba0 0x7f790c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.022+0000 7f791cd99640 1 -- 172.21.15.2:0/2946891282 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f7908014070 con 0x7f7920106880 2024-03-15T13:04:43.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.022+0000 7f791ed9d640 1 --2- 172.21.15.2:0/2946891282 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f790c0a1ba0 0x7f790c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.022+0000 7f791ed9d640 1 --2- 172.21.15.2:0/2946891282 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f790c0a1ba0 0x7f790c0a4060 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7910009bd0 tx=0x7f7910005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.027+0000 7f791cd99640 1 -- 172.21.15.2:0/2946891282 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7908096a20 con 0x7f7920106880 2024-03-15T13:04:43.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:42 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:43.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:42 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:43.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.245+0000 7f7925064640 1 -- 172.21.15.2:0/2946891282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7920109a50 con 0x7f7920106880 2024-03-15T13:04:43.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.246+0000 7f791cd99640 1 -- 172.21.15.2:0/2946891282 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+11459 (secure 0 0 0) 0x7f790809a0e0 con 0x7f7920106880 2024-03-15T13:04:43.231 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:04:43.231 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:04:40.757552+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24434,24539,24460,24464,24500]},"id":1}]} 2024-03-15T13:04:43.231 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 45 2024-03-15T13:04:43.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.248+0000 7f7925064640 1 -- 172.21.15.2:0/2946891282 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f790c0a1ba0 msgr2=0x7f790c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.248+0000 7f7925064640 1 --2- 172.21.15.2:0/2946891282 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f790c0a1ba0 0x7f790c0a4060 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7910009bd0 tx=0x7f7910005f90 comp rx=0 tx=0).stop 2024-03-15T13:04:43.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.248+0000 7f7925064640 1 -- 172.21.15.2:0/2946891282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7920106880 msgr2=0x7f792019dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.248+0000 7f7925064640 1 --2- 172.21.15.2:0/2946891282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7920106880 0x7f792019dbf0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f790800e9c0 tx=0x7f790800ee80 comp rx=0 tx=0).stop 2024-03-15T13:04:43.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.248+0000 7f7925064640 1 -- 172.21.15.2:0/2946891282 shutdown_connections 2024-03-15T13:04:43.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.248+0000 7f7925064640 1 --2- 172.21.15.2:0/2946891282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7920197820 0x7f7920197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.248+0000 7f7925064640 1 --2- 172.21.15.2:0/2946891282 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f790c0a1ba0 0x7f790c0a4060 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.248+0000 7f7925064640 1 --2- 172.21.15.2:0/2946891282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7920106880 0x7f792019dbf0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.248+0000 7f7925064640 1 --2- 172.21.15.2:0/2946891282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7920069df0 0x7f792019d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.248+0000 7f7925064640 1 -- 172.21.15.2:0/2946891282 >> 172.21.15.2:0/2946891282 conn(0x7f7920077490 msgr2=0x7f79200fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.248+0000 7f7925064640 1 -- 172.21.15.2:0/2946891282 shutdown_connections 2024-03-15T13:04:43.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.248+0000 7f7925064640 1 -- 172.21.15.2:0/2946891282 wait complete. 2024-03-15T13:04:43.244 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:04:43.244 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:04:43.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.363+0000 7f10ba62e640 1 -- 172.21.15.2:0/1901310283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10b4107500 msgr2=0x7f10b41078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.363+0000 7f10ba62e640 1 --2- 172.21.15.2:0/1901310283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10b4107500 0x7f10b41078e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009930 tx=0x7f10a402bc30 comp rx=0 tx=0).stop 2024-03-15T13:04:43.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.363+0000 7f10ba62e640 1 -- 172.21.15.2:0/1901310283 shutdown_connections 2024-03-15T13:04:43.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.363+0000 7f10ba62e640 1 --2- 172.21.15.2:0/1901310283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10b4069090 0x7f10b410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.363+0000 7f10ba62e640 1 --2- 172.21.15.2:0/1901310283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10b40686d0 0x7f10b4068b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.363+0000 7f10ba62e640 1 --2- 172.21.15.2:0/1901310283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10b4107500 0x7f10b41078e0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.363+0000 7f10ba62e640 1 -- 172.21.15.2:0/1901310283 >> 172.21.15.2:0/1901310283 conn(0x7f10b40774f0 msgr2=0x7f10b4077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.363+0000 7f10ba62e640 1 -- 172.21.15.2:0/1901310283 shutdown_connections 2024-03-15T13:04:43.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.363+0000 7f10ba62e640 1 -- 172.21.15.2:0/1901310283 wait complete. 2024-03-15T13:04:43.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.364+0000 7f10ba62e640 1 Processor -- start 2024-03-15T13:04:43.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.364+0000 7f10ba62e640 1 -- start start 2024-03-15T13:04:43.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.364+0000 7f10ba62e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10b40686d0 0x7f10b41003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.364+0000 7f10ba62e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10b4069090 0x7f10b41008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.364+0000 7f10ba62e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10b4107500 0x7f10b4100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.364+0000 7f10ba62e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b41a5ee0 con 0x7f10b4107500 2024-03-15T13:04:43.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.364+0000 7f10ba62e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b41a6050 con 0x7f10b4069090 2024-03-15T13:04:43.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.364+0000 7f10ba62e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b41a6210 con 0x7f10b40686d0 2024-03-15T13:04:43.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.364+0000 7f10b8bcb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10b4107500 0x7f10b4100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10b3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10b40686d0 0x7f10b41003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10b8bcb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10b4107500 0x7f10b4100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33120/0 (socket says 172.21.15.2:33120) 2024-03-15T13:04:43.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10b8bcb640 1 -- 172.21.15.2:0/1529285052 learned_addr learned my addr 172.21.15.2:0/1529285052 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10b37fe640 1 --2- 172.21.15.2:0/1529285052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10b4069090 0x7f10b41008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10b8bcb640 1 -- 172.21.15.2:0/1529285052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10b40686d0 msgr2=0x7f10b41003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10b8bcb640 1 --2- 172.21.15.2:0/1529285052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10b40686d0 0x7f10b41003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10b8bcb640 1 -- 172.21.15.2:0/1529285052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10b4069090 msgr2=0x7f10b41008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10b8bcb640 1 --2- 172.21.15.2:0/1529285052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10b4069090 0x7f10b41008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10b8bcb640 1 -- 172.21.15.2:0/1529285052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a4009560 con 0x7f10b4107500 2024-03-15T13:04:43.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10b3fff640 1 --2- 172.21.15.2:0/1529285052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10b40686d0 0x7f10b41003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:43.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10b8bcb640 1 --2- 172.21.15.2:0/1529285052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10b4107500 0x7f10b4100e30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00eb10 tx=0x7f10ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10b17fa640 1 -- 172.21.15.2:0/1529285052 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac004b20 con 0x7f10b4107500 2024-03-15T13:04:43.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10b17fa640 1 -- 172.21.15.2:0/1529285052 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f10ac004c80 con 0x7f10b4107500 2024-03-15T13:04:43.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10b17fa640 1 -- 172.21.15.2:0/1529285052 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac010690 con 0x7f10b4107500 2024-03-15T13:04:43.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10ba62e640 1 -- 172.21.15.2:0/1529285052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b41a64f0 con 0x7f10b4107500 2024-03-15T13:04:43.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.365+0000 7f10ba62e640 1 -- 172.21.15.2:0/1529285052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b41a6a00 con 0x7f10b4107500 2024-03-15T13:04:43.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.367+0000 7f10b17fa640 1 -- 172.21.15.2:0/1529285052 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f10ac004df0 con 0x7f10b4107500 2024-03-15T13:04:43.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10bd214740 1 Processor -- start 2024-03-15T13:04:43.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10bd214740 1 -- start start 2024-03-15T13:04:43.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10b17fa640 1 --2- 172.21.15.2:0/1529285052 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f109c0a1bd0 0x7f109c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10bd214740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c6226ff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10bd214740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 0x55c622708210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10bd214740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c622708750 0x55c62270ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10bd214740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6226ba960 con 0x55c6226ff250 2024-03-15T13:04:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10bd214740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62266e310 con 0x55c622708750 2024-03-15T13:04:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10bd214740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62266b840 con 0x55c6226ffb70 2024-03-15T13:04:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c6226ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10b37fe640 1 --2- 172.21.15.2:0/1529285052 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f109c0a1bd0 0x7f109c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c6226ff630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33124/0 (socket says 172.21.15.2:33124) 2024-03-15T13:04:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10b3fff640 1 -- 172.21.15.2:0/1870617733 learned_addr learned my addr 172.21.15.2:0/1870617733 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10b17fa640 1 -- 172.21.15.2:0/1529285052 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f10ac014070 con 0x7f10b4107500 2024-03-15T13:04:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10b8bcb640 1 --2- 172.21.15.2:0/1870617733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c622708750 0x55c62270ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10b37fe640 1 --2- 172.21.15.2:0/1870617733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 0x55c622708210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10b3fff640 1 -- 172.21.15.2:0/1870617733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 msgr2=0x55c622708210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10b3fff640 1 --2- 172.21.15.2:0/1870617733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 0x55c622708210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10b3fff640 1 -- 172.21.15.2:0/1870617733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c622708750 msgr2=0x55c62270ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10b3fff640 1 --2- 172.21.15.2:0/1870617733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c622708750 0x55c62270ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10b3fff640 1 -- 172.21.15.2:0/1870617733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c622629060 con 0x55c6226ff250 2024-03-15T13:04:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.368+0000 7f10b37fe640 1 --2- 172.21.15.2:0/1870617733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 0x55c622708210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:04:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.369+0000 7f10b3fff640 1 --2- 172.21.15.2:0/1870617733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c6226ff630 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f10a4006010 tx=0x7f10a4004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db171cf328826a1c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.369+0000 7f10b37fe640 1 --2- 172.21.15.2:0/1529285052 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f109c0a1bd0 0x7f109c0a4090 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55c6226fed20 tx=0x7f10a8009840 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.369+0000 7f109a7fc640 1 -- 172.21.15.2:0/1870617733 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a402c650 con 0x55c6226ff250 2024-03-15T13:04:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.369+0000 7f109a7fc640 1 -- 172.21.15.2:0/1870617733 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f10a4004030 con 0x55c6226ff250 2024-03-15T13:04:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.369+0000 7f109a7fc640 1 -- 172.21.15.2:0/1870617733 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4003ac0 con 0x55c6226ff250 2024-03-15T13:04:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.369+0000 7f10bd214740 1 -- 172.21.15.2:0/1870617733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 msgr2=0x55c6226ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.369+0000 7f10bd214740 1 --2- 172.21.15.2:0/1870617733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c6226ff630 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f10a4006010 tx=0x7f10a4004b00 comp rx=0 tx=0).stop 2024-03-15T13:04:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.369+0000 7f10bd214740 1 -- 172.21.15.2:0/1870617733 shutdown_connections 2024-03-15T13:04:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.369+0000 7f10bd214740 1 --2- 172.21.15.2:0/1870617733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c622708750 0x55c62270ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.369+0000 7f10bd214740 1 --2- 172.21.15.2:0/1870617733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 0x55c622708210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.369+0000 7f10bd214740 1 --2- 172.21.15.2:0/1870617733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c6226ff630 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.369+0000 7f10bd214740 1 -- 172.21.15.2:0/1870617733 >> 172.21.15.2:0/1870617733 conn(0x55c6226f9980 msgr2=0x55c6226f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.369+0000 7f10bd214740 1 -- 172.21.15.2:0/1870617733 shutdown_connections 2024-03-15T13:04:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.369+0000 7f10bd214740 1 -- 172.21.15.2:0/1870617733 wait complete. 2024-03-15T13:04:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.370+0000 7f10bd214740 1 -- start start 2024-03-15T13:04:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.370+0000 7f10bd214740 1 Processor -- start 2024-03-15T13:04:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.370+0000 7f10bd214740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c622708050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.370+0000 7f10bd214740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 0x55c6227053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.370+0000 7f10bd214740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c622708750 0x55c622705920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.370+0000 7f10bd214740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c622667020 con 0x55c6226ff250 2024-03-15T13:04:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.370+0000 7f10bd214740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62233a430 con 0x55c622708750 2024-03-15T13:04:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.370+0000 7f10bd214740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6222f6e00 con 0x55c6226ffb70 2024-03-15T13:04:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.370+0000 7f10bd214740 10 client.0 _ll_register_callbacks cb 0x55c62260ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.370+0000 7f10b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c622708050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.370+0000 7f10b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c622708050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33130/0 (socket says 172.21.15.2:33130) 2024-03-15T13:04:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.370+0000 7f10b3fff640 1 -- 172.21.15.2:0/3297566573 learned_addr learned my addr 172.21.15.2:0/3297566573 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.370+0000 7f10b8bcb640 1 --2- 172.21.15.2:0/3297566573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c622708750 0x55c622705920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.370+0000 7f10b37fe640 1 --2- 172.21.15.2:0/3297566573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 0x55c6227053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10b3fff640 1 -- 172.21.15.2:0/3297566573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 msgr2=0x55c6227053e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10b3fff640 1 --2- 172.21.15.2:0/3297566573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 0x55c6227053e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10b3fff640 1 -- 172.21.15.2:0/3297566573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c622708750 msgr2=0x55c622705920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10b3fff640 1 --2- 172.21.15.2:0/3297566573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c622708750 0x55c622705920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10b3fff640 1 -- 172.21.15.2:0/3297566573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b410ec30 con 0x55c6226ff250 2024-03-15T13:04:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10b3fff640 1 --2- 172.21.15.2:0/3297566573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c622708050 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f10a4005ec0 tx=0x7f10a402cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24860908a55e9eb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10937fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10937fe640 1 -- 172.21.15.2:0/3297566573 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4003c50 con 0x55c6226ff250 2024-03-15T13:04:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10937fe640 1 -- 172.21.15.2:0/3297566573 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f10a404d070 con 0x55c6226ff250 2024-03-15T13:04:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10bd214740 1 -- 172.21.15.2:0/3297566573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6223667e0 con 0x55c6226ff250 2024-03-15T13:04:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10937fe640 1 -- 172.21.15.2:0/3297566573 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10a40035f0 con 0x55c6226ff250 2024-03-15T13:04:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10bd214740 10 client.17361 fetch_fsmap learned FSMap version 45 2024-03-15T13:04:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10bd214740 1 -- 172.21.15.2:0/3297566573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55c6226656a0 con 0x55c6226ff250 2024-03-15T13:04:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10937fe640 1 -- 172.21.15.2:0/3297566573 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55c6223667e0 con 0x55c6226ff250 2024-03-15T13:04:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10937fe640 1 -- 172.21.15.2:0/3297566573 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f10a403c050 con 0x55c6226ff250 2024-03-15T13:04:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10bd214740 10 client.17361 fetch_fsmap finished waiting for FSMap version 45 2024-03-15T13:04:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10bd214740 10 client.17361 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:04:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10bd214740 20 client.17361 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.371+0000 7f10bd214740 1 --2- 172.21.15.2:0/3297566573 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c622718470 0x55c62271a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.372+0000 7f10bd214740 4 client.17361 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.372+0000 7f10bd214740 1 -- 172.21.15.2:0/3297566573 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6223a3720 con 0x55c622718470 2024-03-15T13:04:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.372+0000 7f10b37fe640 1 --2- 172.21.15.2:0/3297566573 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c622718470 0x55c62271a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.372+0000 7f10b37fe640 1 --2- 172.21.15.2:0/3297566573 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c622718470 0x55c62271a890 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a4911527cfea326 server_cookie=36cc0742beb21de0 in_seq=0 out_seq=0 2024-03-15T13:04:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.372+0000 7f10937fe640 10 client.17361 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:04:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.373+0000 7f10937fe640 1 -- 172.21.15.2:0/3297566573 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c6223a3720 con 0x55c622718470 2024-03-15T13:04:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.373+0000 7f10937fe640 10 client.17361 handle_command_reply: tid=0 2024-03-15T13:04:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.373+0000 7f10bd214740 1 client.17361 shutdown 2024-03-15T13:04:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.373+0000 7f10bd214740 1 -- 172.21.15.2:0/3297566573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 msgr2=0x55c622708050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.373+0000 7f10bd214740 1 --2- 172.21.15.2:0/3297566573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c622708050 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f10a4005ec0 tx=0x7f10a402cac0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.373+0000 7f10bd214740 1 -- 172.21.15.2:0/3297566573 shutdown_connections 2024-03-15T13:04:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.373+0000 7f10bd214740 1 --2- 172.21.15.2:0/3297566573 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c622718470 0x55c62271a890 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.373+0000 7f10bd214740 1 --2- 172.21.15.2:0/3297566573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c622708750 0x55c622705920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.373+0000 7f10bd214740 1 --2- 172.21.15.2:0/3297566573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 0x55c6227053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.373+0000 7f10937fe640 0 client.17361 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:04:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.373+0000 7f10bd214740 1 --2- 172.21.15.2:0/3297566573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c622708050 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.373+0000 7f10bd214740 1 -- 172.21.15.2:0/3297566573 >> 172.21.15.2:0/3297566573 conn(0x55c6226f9980 msgr2=0x55c6226fb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.374+0000 7f10bd214740 1 -- 172.21.15.2:0/3297566573 shutdown_connections 2024-03-15T13:04:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.374+0000 7f10bd214740 1 -- 172.21.15.2:0/3297566573 wait complete. 2024-03-15T13:04:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.374+0000 7f10bd214740 20 client.17361 trim_cache size 0 max 16384 2024-03-15T13:04:43.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10bd214740 1 Processor -- start 2024-03-15T13:04:43.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10bd214740 1 -- start start 2024-03-15T13:04:43.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10bd214740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c622707b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10bd214740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 0x55c622708050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10bd214740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c622708750 0x55c622713a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10bd214740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6225ec130 con 0x55c6226ff250 2024-03-15T13:04:43.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10bd214740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62270ab40 con 0x55c622708750 2024-03-15T13:04:43.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10bd214740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6226f55c0 con 0x55c6226ffb70 2024-03-15T13:04:43.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c622707b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c622707b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33138/0 (socket says 172.21.15.2:33138) 2024-03-15T13:04:43.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10b3fff640 1 -- 172.21.15.2:0/2877683735 learned_addr learned my addr 172.21.15.2:0/2877683735 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10b37fe640 1 --2- 172.21.15.2:0/2877683735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 0x55c622708050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10b8bcb640 1 --2- 172.21.15.2:0/2877683735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c622708750 0x55c622713a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10b3fff640 1 -- 172.21.15.2:0/2877683735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 msgr2=0x55c622708050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10b3fff640 1 --2- 172.21.15.2:0/2877683735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 0x55c622708050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10b3fff640 1 -- 172.21.15.2:0/2877683735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c622708750 msgr2=0x55c622713a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10b3fff640 1 --2- 172.21.15.2:0/2877683735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c622708750 0x55c622713a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.386+0000 7f10b3fff640 1 -- 172.21.15.2:0/2877683735 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c622601c20 con 0x55c6226ff250 2024-03-15T13:04:43.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.387+0000 7f10b3fff640 1 --2- 172.21.15.2:0/2877683735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c622707b10 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f10a4004530 tx=0x7f10a4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a16d14517ed27e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.387+0000 7f1099ffb640 1 -- 172.21.15.2:0/2877683735 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4004b00 con 0x55c6226ff250 2024-03-15T13:04:43.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.387+0000 7f1099ffb640 1 -- 172.21.15.2:0/2877683735 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f10a402cb30 con 0x55c6226ff250 2024-03-15T13:04:43.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.387+0000 7f1099ffb640 1 -- 172.21.15.2:0/2877683735 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4005500 con 0x55c6226ff250 2024-03-15T13:04:43.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.387+0000 7f10bd214740 1 -- 172.21.15.2:0/2877683735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 msgr2=0x55c622707b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.387+0000 7f10bd214740 1 --2- 172.21.15.2:0/2877683735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c622707b10 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f10a4004530 tx=0x7f10a4005c90 comp rx=0 tx=0).stop 2024-03-15T13:04:43.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.387+0000 7f10bd214740 1 -- 172.21.15.2:0/2877683735 shutdown_connections 2024-03-15T13:04:43.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.387+0000 7f10bd214740 1 --2- 172.21.15.2:0/2877683735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c622708750 0x55c622713a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.387+0000 7f10bd214740 1 --2- 172.21.15.2:0/2877683735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ffb70 0x55c622708050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.387+0000 7f10bd214740 1 --2- 172.21.15.2:0/2877683735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c6226ff250 0x55c622707b10 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.387+0000 7f10bd214740 1 -- 172.21.15.2:0/2877683735 >> 172.21.15.2:0/2877683735 conn(0x55c6226f9980 msgr2=0x55c622700920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.387+0000 7f10bd214740 1 -- 172.21.15.2:0/2877683735 shutdown_connections 2024-03-15T13:04:43.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.387+0000 7f10bd214740 1 -- 172.21.15.2:0/2877683735 wait complete. 2024-03-15T13:04:43.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10bd214740 1 -- start start 2024-03-15T13:04:43.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10bd214740 1 Processor -- start 2024-03-15T13:04:43.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10bd214740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ff250 0x55c62270c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10bd214740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6226ffb70 0x55c62270cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10bd214740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c622708750 0x55c62276bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10bd214740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62276c3d0 con 0x55c622708750 2024-03-15T13:04:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10bd214740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62276c540 con 0x55c6226ffb70 2024-03-15T13:04:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10bd214740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62276c700 con 0x55c6226ff250 2024-03-15T13:04:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10bd214740 10 client.0 _ll_register_callbacks cb 0x7f10a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10b8bcb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c622708750 0x55c62276bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10b3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ff250 0x55c62270c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10b8bcb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c622708750 0x55c62276bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33142/0 (socket says 172.21.15.2:33142) 2024-03-15T13:04:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10b8bcb640 1 -- 172.21.15.2:0/534446343 learned_addr learned my addr 172.21.15.2:0/534446343 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10b37fe640 1 --2- 172.21.15.2:0/534446343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6226ffb70 0x55c62270cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10b8bcb640 1 -- 172.21.15.2:0/534446343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ff250 msgr2=0x55c62270c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10b8bcb640 1 --2- 172.21.15.2:0/534446343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ff250 0x55c62270c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10b8bcb640 1 -- 172.21.15.2:0/534446343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6226ffb70 msgr2=0x55c62270cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10b8bcb640 1 --2- 172.21.15.2:0/534446343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6226ffb70 0x55c62270cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10b8bcb640 1 -- 172.21.15.2:0/534446343 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6226656a0 con 0x55c622708750 2024-03-15T13:04:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.388+0000 7f10b37fe640 1 --2- 172.21.15.2:0/534446343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6226ffb70 0x55c62270cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:43.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10b8bcb640 1 --2- 172.21.15.2:0/534446343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c622708750 0x55c62276bd00 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0049d0 tx=0x7f10ac072d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a4ad7eb262dbfc server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10937fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:43.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10937fe640 1 -- 172.21.15.2:0/534446343 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac017ed0 con 0x55c622708750 2024-03-15T13:04:43.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10937fe640 1 -- 172.21.15.2:0/534446343 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f10ac09aa50 con 0x55c622708750 2024-03-15T13:04:43.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10bd214740 1 -- 172.21.15.2:0/534446343 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10a4004030 con 0x55c622708750 2024-03-15T13:04:43.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10937fe640 1 -- 172.21.15.2:0/534446343 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10ac09eba0 con 0x55c622708750 2024-03-15T13:04:43.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10bd214740 10 client.17367 fetch_fsmap learned FSMap version 45 2024-03-15T13:04:43.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10bd214740 1 -- 172.21.15.2:0/534446343 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55c6226278c0 con 0x55c622708750 2024-03-15T13:04:43.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10937fe640 1 -- 172.21.15.2:0/534446343 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f10a4004030 con 0x55c622708750 2024-03-15T13:04:43.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10937fe640 1 -- 172.21.15.2:0/534446343 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f10ac01f060 con 0x55c622708750 2024-03-15T13:04:43.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10bd214740 10 client.17367 fetch_fsmap finished waiting for FSMap version 45 2024-03-15T13:04:43.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10bd214740 10 client.17367 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:04:43.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10bd214740 20 client.17367 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:43.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10bd214740 1 --2- 172.21.15.2:0/534446343 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c622774f40 0x55c622777360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10bd214740 4 client.17367 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:04:43.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10bd214740 1 -- 172.21.15.2:0/534446343 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c6223f2890 con 0x55c622774f40 2024-03-15T13:04:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.389+0000 7f10b3fff640 1 --2- 172.21.15.2:0/534446343 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c622774f40 0x55c622777360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10937fe640 10 client.17367 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:04:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10b3fff640 1 --2- 172.21.15.2:0/534446343 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c622774f40 0x55c622777360 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f131c9d932b22d44 server_cookie=e2d77fc1d8aead40 in_seq=0 out_seq=0 2024-03-15T13:04:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10937fe640 1 -- 172.21.15.2:0/534446343 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c6223f2890 con 0x55c622774f40 2024-03-15T13:04:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10937fe640 10 client.17367 handle_command_reply: tid=0 2024-03-15T13:04:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10bd214740 1 client.17367 shutdown 2024-03-15T13:04:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10bd214740 1 -- 172.21.15.2:0/534446343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c622708750 msgr2=0x55c62276bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10bd214740 1 --2- 172.21.15.2:0/534446343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c622708750 0x55c62276bd00 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0049d0 tx=0x7f10ac072d40 comp rx=0 tx=0).stop 2024-03-15T13:04:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10bd214740 1 -- 172.21.15.2:0/534446343 shutdown_connections 2024-03-15T13:04:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10bd214740 1 --2- 172.21.15.2:0/534446343 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c622774f40 0x55c622777360 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10bd214740 1 --2- 172.21.15.2:0/534446343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c622708750 0x55c62276bd00 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10937fe640 0 client.17367 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:04:43.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10bd214740 1 --2- 172.21.15.2:0/534446343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6226ffb70 0x55c62270cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10bd214740 1 --2- 172.21.15.2:0/534446343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c6226ff250 0x55c62270c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10bd214740 1 -- 172.21.15.2:0/534446343 >> 172.21.15.2:0/534446343 conn(0x55c6226f9980 msgr2=0x55c622704c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10bd214740 1 -- 172.21.15.2:0/534446343 shutdown_connections 2024-03-15T13:04:43.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10bd214740 1 -- 172.21.15.2:0/534446343 wait complete. 2024-03-15T13:04:43.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.390+0000 7f10bd214740 20 client.17367 trim_cache size 0 max 16384 2024-03-15T13:04:43.383 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:04:43.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.391+0000 7f10ba62e640 1 -- 172.21.15.2:0/1529285052 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f109c0a1bd0 msgr2=0x7f109c0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.391+0000 7f10ba62e640 1 --2- 172.21.15.2:0/1529285052 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f109c0a1bd0 0x7f109c0a4090 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55c6226fed20 tx=0x7f10a8009840 comp rx=0 tx=0).stop 2024-03-15T13:04:43.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.391+0000 7f10ba62e640 1 -- 172.21.15.2:0/1529285052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10b4107500 msgr2=0x7f10b4100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.391+0000 7f10ba62e640 1 --2- 172.21.15.2:0/1529285052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10b4107500 0x7f10b4100e30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00eb10 tx=0x7f10ac00c2d0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.391+0000 7f10ba62e640 1 -- 172.21.15.2:0/1529285052 shutdown_connections 2024-03-15T13:04:43.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.391+0000 7f10ba62e640 1 --2- 172.21.15.2:0/1529285052 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f109c0a1bd0 0x7f109c0a4090 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.391+0000 7f10ba62e640 1 --2- 172.21.15.2:0/1529285052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10b4107500 0x7f10b4100e30 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.391+0000 7f10ba62e640 1 --2- 172.21.15.2:0/1529285052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10b4069090 0x7f10b41008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.391+0000 7f10ba62e640 1 --2- 172.21.15.2:0/1529285052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10b40686d0 0x7f10b41003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.391+0000 7f10ba62e640 1 -- 172.21.15.2:0/1529285052 >> 172.21.15.2:0/1529285052 conn(0x7f10b40774f0 msgr2=0x7f10b40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.391+0000 7f10ba62e640 1 -- 172.21.15.2:0/1529285052 shutdown_connections 2024-03-15T13:04:43.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.391+0000 7f10ba62e640 1 -- 172.21.15.2:0/1529285052 wait complete. 2024-03-15T13:04:43.387 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:04:43.387 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:04:43.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.506+0000 7f7883b2c640 1 -- 172.21.15.2:0/1164062790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c107570 msgr2=0x7f787c107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.506+0000 7f7883b2c640 1 --2- 172.21.15.2:0/1164062790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c107570 0x7f787c107950 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7878009890 tx=0x7f787802bb70 comp rx=0 tx=0).stop 2024-03-15T13:04:43.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.506+0000 7f7883b2c640 1 -- 172.21.15.2:0/1164062790 shutdown_connections 2024-03-15T13:04:43.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.506+0000 7f7883b2c640 1 --2- 172.21.15.2:0/1164062790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f787c101e70 0x7f787c10e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.506+0000 7f7883b2c640 1 --2- 172.21.15.2:0/1164062790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c1014b0 0x7f787c101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.506+0000 7f7883b2c640 1 --2- 172.21.15.2:0/1164062790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c107570 0x7f787c107950 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.506+0000 7f7883b2c640 1 -- 172.21.15.2:0/1164062790 >> 172.21.15.2:0/1164062790 conn(0x7f787c0fcf20 msgr2=0x7f787c0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.506+0000 7f7883b2c640 1 -- 172.21.15.2:0/1164062790 shutdown_connections 2024-03-15T13:04:43.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.506+0000 7f7883b2c640 1 -- 172.21.15.2:0/1164062790 wait complete. 2024-03-15T13:04:43.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f7883b2c640 1 Processor -- start 2024-03-15T13:04:43.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f7883b2c640 1 -- start start 2024-03-15T13:04:43.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f7883b2c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f787c1014b0 0x7f787c0737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f7883b2c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c101e70 0x7f787c073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f7883b2c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c107570 0x7f787c074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f7883b2c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c1a5e70 con 0x7f787c107570 2024-03-15T13:04:43.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f7883b2c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c1a5fe0 con 0x7f787c1014b0 2024-03-15T13:04:43.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f7883b2c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c1a61a0 con 0x7f787c101e70 2024-03-15T13:04:43.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f78820c9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c107570 0x7f787c074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f78810c7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c101e70 0x7f787c073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f78820c9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c107570 0x7f787c074270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33156/0 (socket says 172.21.15.2:33156) 2024-03-15T13:04:43.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f78820c9640 1 -- 172.21.15.2:0/4256288236 learned_addr learned my addr 172.21.15.2:0/4256288236 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f78818c8640 1 --2- 172.21.15.2:0/4256288236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f787c1014b0 0x7f787c0737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f78820c9640 1 -- 172.21.15.2:0/4256288236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c101e70 msgr2=0x7f787c073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f78820c9640 1 --2- 172.21.15.2:0/4256288236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c101e70 0x7f787c073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f78820c9640 1 -- 172.21.15.2:0/4256288236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f787c1014b0 msgr2=0x7f787c0737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f78820c9640 1 --2- 172.21.15.2:0/4256288236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f787c1014b0 0x7f787c0737f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f78820c9640 1 -- 172.21.15.2:0/4256288236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c009560 con 0x7f787c107570 2024-03-15T13:04:43.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.507+0000 7f78810c7640 1 --2- 172.21.15.2:0/4256288236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c101e70 0x7f787c073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:43.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.508+0000 7f78820c9640 1 --2- 172.21.15.2:0/4256288236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c107570 0x7f787c074270 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f787400eb10 tx=0x7f787400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.508+0000 7f7872ffd640 1 -- 172.21.15.2:0/4256288236 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874004b20 con 0x7f787c107570 2024-03-15T13:04:43.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.508+0000 7f7883b2c640 1 -- 172.21.15.2:0/4256288236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7878009490 con 0x7f787c107570 2024-03-15T13:04:43.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.508+0000 7f7872ffd640 1 -- 172.21.15.2:0/4256288236 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7874004c80 con 0x7f787c107570 2024-03-15T13:04:43.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.508+0000 7f7872ffd640 1 -- 172.21.15.2:0/4256288236 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874010690 con 0x7f787c107570 2024-03-15T13:04:43.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.508+0000 7f7883b2c640 1 -- 172.21.15.2:0/4256288236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c1a67b0 con 0x7f787c107570 2024-03-15T13:04:43.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.511+0000 7f7872ffd640 1 -- 172.21.15.2:0/4256288236 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f7874004df0 con 0x7f787c107570 2024-03-15T13:04:43.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.511+0000 7f78867e5740 1 Processor -- start 2024-03-15T13:04:43.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.511+0000 7f78867e5740 1 -- start start 2024-03-15T13:04:43.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.511+0000 7f78867e5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522250 0x55ac8e522630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.511+0000 7f78867e5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522b70 0x55ac8e52b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.511+0000 7f78867e5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e52b750 0x55ac8e52db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.511+0000 7f78867e5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac8e4dd960 con 0x55ac8e522b70 2024-03-15T13:04:43.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.511+0000 7f78867e5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac8e491310 con 0x55ac8e52b750 2024-03-15T13:04:43.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.511+0000 7f78867e5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac8e48e840 con 0x55ac8e522250 2024-03-15T13:04:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.511+0000 7f78818c8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522b70 0x55ac8e52b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.511+0000 7f78818c8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522b70 0x55ac8e52b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33166/0 (socket says 172.21.15.2:33166) 2024-03-15T13:04:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.511+0000 7f78818c8640 1 -- 172.21.15.2:0/1122714376 learned_addr learned my addr 172.21.15.2:0/1122714376 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.511+0000 7f78810c7640 1 --2- 172.21.15.2:0/1122714376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522250 0x55ac8e522630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.511+0000 7f7872ffd640 1 --2- 172.21.15.2:0/4256288236 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f78680a1ba0 0x7f78680a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.511+0000 7f78810c7640 1 --2- 172.21.15.2:0/1122714376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e52b750 0x55ac8e52db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.512+0000 7f78820c9640 1 --2- 172.21.15.2:0/4256288236 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f78680a1ba0 0x7f78680a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.512+0000 7f78810c7640 1 -- 172.21.15.2:0/1122714376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e52b750 msgr2=0x55ac8e52db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.512+0000 7f78810c7640 1 --2- 172.21.15.2:0/1122714376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e52b750 0x55ac8e52db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.512+0000 7f78810c7640 1 -- 172.21.15.2:0/1122714376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522b70 msgr2=0x55ac8e52b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.512+0000 7f78810c7640 1 --2- 172.21.15.2:0/1122714376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522b70 0x55ac8e52b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.512+0000 7f78810c7640 1 -- 172.21.15.2:0/1122714376 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac8e44c060 con 0x55ac8e522250 2024-03-15T13:04:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.512+0000 7f78818c8640 1 --2- 172.21.15.2:0/1122714376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522b70 0x55ac8e52b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:43.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.512+0000 7f7872ffd640 1 -- 172.21.15.2:0/4256288236 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f7874014070 con 0x7f787c107570 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.512+0000 7f78810c7640 1 --2- 172.21.15.2:0/1122714376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522250 0x55ac8e522630 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f786c009530 tx=0x7f786c009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9090b754d7a7dd67 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.512+0000 7f78820c9640 1 --2- 172.21.15.2:0/4256288236 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f78680a1ba0 0x7f78680a4060 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f787409a670 tx=0x7f7874009a70 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.512+0000 7f7867fff640 1 -- 172.21.15.2:0/1122714376 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c00fc20 con 0x55ac8e522250 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.512+0000 7f7867fff640 1 -- 172.21.15.2:0/1122714376 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f786c00fd80 con 0x55ac8e522250 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.512+0000 7f7867fff640 1 -- 172.21.15.2:0/1122714376 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c0047a0 con 0x55ac8e522250 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.513+0000 7f78867e5740 1 -- 172.21.15.2:0/1122714376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522250 msgr2=0x55ac8e522630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.513+0000 7f78867e5740 1 --2- 172.21.15.2:0/1122714376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522250 0x55ac8e522630 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f786c009530 tx=0x7f786c009c90 comp rx=0 tx=0).stop 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.513+0000 7f78867e5740 1 -- 172.21.15.2:0/1122714376 shutdown_connections 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.513+0000 7f78867e5740 1 --2- 172.21.15.2:0/1122714376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e52b750 0x55ac8e52db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.513+0000 7f78867e5740 1 --2- 172.21.15.2:0/1122714376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522b70 0x55ac8e52b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.513+0000 7f78867e5740 1 --2- 172.21.15.2:0/1122714376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522250 0x55ac8e522630 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.513+0000 7f78867e5740 1 -- 172.21.15.2:0/1122714376 >> 172.21.15.2:0/1122714376 conn(0x55ac8e51c980 msgr2=0x55ac8e51cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.513+0000 7f78867e5740 1 -- 172.21.15.2:0/1122714376 shutdown_connections 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.513+0000 7f78867e5740 1 -- 172.21.15.2:0/1122714376 wait complete. 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.513+0000 7f78867e5740 1 -- start start 2024-03-15T13:04:43.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.513+0000 7f78867e5740 1 Processor -- start 2024-03-15T13:04:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78867e5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e522250 0x55ac8e52b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78867e5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522b70 0x55ac8e528440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78867e5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e52b750 0x55ac8e5289a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78867e5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac8e48a020 con 0x55ac8e522b70 2024-03-15T13:04:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78867e5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac8e15d430 con 0x55ac8e522250 2024-03-15T13:04:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78867e5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac8e119e00 con 0x55ac8e52b750 2024-03-15T13:04:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78867e5740 10 client.0 _ll_register_callbacks cb 0x55ac8e431cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78818c8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522b70 0x55ac8e528440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78818c8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522b70 0x55ac8e528440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33180/0 (socket says 172.21.15.2:33180) 2024-03-15T13:04:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78818c8640 1 -- 172.21.15.2:0/2217167494 learned_addr learned my addr 172.21.15.2:0/2217167494 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78810c7640 1 --2- 172.21.15.2:0/2217167494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e52b750 0x55ac8e5289a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78810c7640 1 --2- 172.21.15.2:0/2217167494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e522250 0x55ac8e52b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78818c8640 1 -- 172.21.15.2:0/2217167494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e52b750 msgr2=0x55ac8e5289a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78818c8640 1 --2- 172.21.15.2:0/2217167494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e52b750 0x55ac8e5289a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78818c8640 1 -- 172.21.15.2:0/2217167494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e522250 msgr2=0x55ac8e52b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78818c8640 1 --2- 172.21.15.2:0/2217167494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e522250 0x55ac8e52b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78818c8640 1 -- 172.21.15.2:0/2217167494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c019040 con 0x55ac8e522b70 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78818c8640 1 --2- 172.21.15.2:0/2217167494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522b70 0x55ac8e528440 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7878005940 tx=0x7f7878004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b487c84faad71ad server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.514+0000 7f78657fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.515+0000 7f78657fa640 1 -- 172.21.15.2:0/2217167494 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878034640 con 0x55ac8e522b70 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.515+0000 7f78867e5740 1 -- 172.21.15.2:0/2217167494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac8e1897e0 con 0x55ac8e522b70 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.515+0000 7f78657fa640 1 -- 172.21.15.2:0/2217167494 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f78780347a0 con 0x55ac8e522b70 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.515+0000 7f78657fa640 1 -- 172.21.15.2:0/2217167494 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7878034960 con 0x55ac8e522b70 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.515+0000 7f78867e5740 10 client.17379 fetch_fsmap learned FSMap version 45 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.515+0000 7f78867e5740 1 -- 172.21.15.2:0/2217167494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55ac8e4886a0 con 0x55ac8e522b70 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.515+0000 7f78657fa640 1 -- 172.21.15.2:0/2217167494 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7878050080 con 0x55ac8e522b70 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.515+0000 7f78657fa640 1 -- 172.21.15.2:0/2217167494 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f78780046e0 con 0x55ac8e522b70 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.515+0000 7f78867e5740 10 client.17379 fetch_fsmap finished waiting for FSMap version 45 2024-03-15T13:04:43.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.515+0000 7f78867e5740 10 client.17379 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:04:43.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.515+0000 7f78867e5740 20 client.17379 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:43.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.515+0000 7f78867e5740 1 --2- 172.21.15.2:0/2217167494 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ac8e53b800 0x55ac8e53dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.515+0000 7f78867e5740 4 client.17379 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:43.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.515+0000 7f78867e5740 1 -- 172.21.15.2:0/2217167494 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac8e1c6720 con 0x55ac8e53b800 2024-03-15T13:04:43.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.515+0000 7f78810c7640 1 --2- 172.21.15.2:0/2217167494 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ac8e53b800 0x55ac8e53dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.516+0000 7f78810c7640 1 --2- 172.21.15.2:0/2217167494 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ac8e53b800 0x55ac8e53dc20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8dbe9ef2047224e4 server_cookie=76ed6b15de6ef898 in_seq=0 out_seq=0 2024-03-15T13:04:43.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.516+0000 7f78657fa640 10 client.17379 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:04:43.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.516+0000 7f78657fa640 1 -- 172.21.15.2:0/2217167494 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ac8e1c6720 con 0x55ac8e53b800 2024-03-15T13:04:43.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.516+0000 7f78657fa640 10 client.17379 handle_command_reply: tid=0 2024-03-15T13:04:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.517+0000 7f78867e5740 1 client.17379 shutdown 2024-03-15T13:04:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.517+0000 7f78867e5740 1 -- 172.21.15.2:0/2217167494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522b70 msgr2=0x55ac8e528440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.517+0000 7f78867e5740 1 --2- 172.21.15.2:0/2217167494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522b70 0x55ac8e528440 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7878005940 tx=0x7f7878004270 comp rx=0 tx=0).stop 2024-03-15T13:04:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.517+0000 7f78867e5740 1 -- 172.21.15.2:0/2217167494 shutdown_connections 2024-03-15T13:04:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.517+0000 7f78867e5740 1 --2- 172.21.15.2:0/2217167494 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ac8e53b800 0x55ac8e53dc20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.517+0000 7f78867e5740 1 --2- 172.21.15.2:0/2217167494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e52b750 0x55ac8e5289a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.517+0000 7f78867e5740 1 --2- 172.21.15.2:0/2217167494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522b70 0x55ac8e528440 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.517+0000 7f78657fa640 0 client.17379 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:04:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.517+0000 7f78867e5740 1 --2- 172.21.15.2:0/2217167494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e522250 0x55ac8e52b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.517+0000 7f78867e5740 1 -- 172.21.15.2:0/2217167494 >> 172.21.15.2:0/2217167494 conn(0x55ac8e51c980 msgr2=0x55ac8e51e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.517+0000 7f78867e5740 1 -- 172.21.15.2:0/2217167494 shutdown_connections 2024-03-15T13:04:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.517+0000 7f78867e5740 1 -- 172.21.15.2:0/2217167494 wait complete. 2024-03-15T13:04:43.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.517+0000 7f78867e5740 20 client.17379 trim_cache size 0 max 16384 2024-03-15T13:04:43.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.529+0000 7f78867e5740 1 Processor -- start 2024-03-15T13:04:43.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.529+0000 7f78867e5740 1 -- start start 2024-03-15T13:04:43.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78867e5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522250 0x55ac8e523b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78867e5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522b70 0x55ac8e52ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78867e5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e52b750 0x55ac8e52b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78867e5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac8e40f130 con 0x55ac8e522250 2024-03-15T13:04:43.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78867e5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac8e1c6750 con 0x55ac8e52b750 2024-03-15T13:04:43.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78867e5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac8e53b5d0 con 0x55ac8e522b70 2024-03-15T13:04:43.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78810c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522250 0x55ac8e523b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78810c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522250 0x55ac8e523b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33188/0 (socket says 172.21.15.2:33188) 2024-03-15T13:04:43.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78810c7640 1 -- 172.21.15.2:0/3420743458 learned_addr learned my addr 172.21.15.2:0/3420743458 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78818c8640 1 --2- 172.21.15.2:0/3420743458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522b70 0x55ac8e52ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78810c7640 1 --2- 172.21.15.2:0/3420743458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e52b750 0x55ac8e52b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78810c7640 1 -- 172.21.15.2:0/3420743458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522b70 msgr2=0x55ac8e52ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78810c7640 1 --2- 172.21.15.2:0/3420743458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522b70 0x55ac8e52ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78810c7640 1 -- 172.21.15.2:0/3420743458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e52b750 msgr2=0x55ac8e52b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78810c7640 1 --2- 172.21.15.2:0/3420743458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e52b750 0x55ac8e52b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78810c7640 1 -- 172.21.15.2:0/3420743458 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac8e424c20 con 0x55ac8e522250 2024-03-15T13:04:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78810c7640 1 --2- 172.21.15.2:0/3420743458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522250 0x55ac8e523b60 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f786c0058c0 tx=0x7f786c0054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31388c162a3a360 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78677fe640 1 -- 172.21.15.2:0/3420743458 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c005640 con 0x55ac8e522250 2024-03-15T13:04:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78677fe640 1 -- 172.21.15.2:0/3420743458 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f786c031c90 con 0x55ac8e522250 2024-03-15T13:04:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.530+0000 7f78677fe640 1 -- 172.21.15.2:0/3420743458 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c01e930 con 0x55ac8e522250 2024-03-15T13:04:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.531+0000 7f78867e5740 1 -- 172.21.15.2:0/3420743458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522250 msgr2=0x55ac8e523b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.531+0000 7f78867e5740 1 --2- 172.21.15.2:0/3420743458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522250 0x55ac8e523b60 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f786c0058c0 tx=0x7f786c0054e0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.531+0000 7f78867e5740 1 -- 172.21.15.2:0/3420743458 shutdown_connections 2024-03-15T13:04:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.531+0000 7f78867e5740 1 --2- 172.21.15.2:0/3420743458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e52b750 0x55ac8e52b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.531+0000 7f78867e5740 1 --2- 172.21.15.2:0/3420743458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522b70 0x55ac8e52ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.531+0000 7f78867e5740 1 --2- 172.21.15.2:0/3420743458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522250 0x55ac8e523b60 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.531+0000 7f78867e5740 1 -- 172.21.15.2:0/3420743458 >> 172.21.15.2:0/3420743458 conn(0x55ac8e51c980 msgr2=0x55ac8e528560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.531+0000 7f78867e5740 1 -- 172.21.15.2:0/3420743458 shutdown_connections 2024-03-15T13:04:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.531+0000 7f78867e5740 1 -- 172.21.15.2:0/3420743458 wait complete. 2024-03-15T13:04:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.531+0000 7f78867e5740 1 -- start start 2024-03-15T13:04:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.531+0000 7f78867e5740 1 Processor -- start 2024-03-15T13:04:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78867e5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522250 0x55ac8e533340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78867e5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522b70 0x55ac8e52f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78867e5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e52b750 0x55ac8e52fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78867e5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac8e5302a0 con 0x55ac8e522250 2024-03-15T13:04:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78867e5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac8e530410 con 0x55ac8e52b750 2024-03-15T13:04:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78867e5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac8e5970c0 con 0x55ac8e522b70 2024-03-15T13:04:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78867e5740 10 client.0 _ll_register_callbacks cb 0x7f78740093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78810c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522250 0x55ac8e533340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78810c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522250 0x55ac8e533340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33196/0 (socket says 172.21.15.2:33196) 2024-03-15T13:04:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78810c7640 1 -- 172.21.15.2:0/3664081332 learned_addr learned my addr 172.21.15.2:0/3664081332 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78818c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522b70 0x55ac8e52f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78810c7640 1 --2- 172.21.15.2:0/3664081332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e52b750 0x55ac8e52fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78810c7640 1 -- 172.21.15.2:0/3664081332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522b70 msgr2=0x55ac8e52f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78810c7640 1 --2- 172.21.15.2:0/3664081332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522b70 0x55ac8e52f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78810c7640 1 -- 172.21.15.2:0/3664081332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e52b750 msgr2=0x55ac8e52fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78810c7640 1 --2- 172.21.15.2:0/3664081332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e52b750 0x55ac8e52fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78810c7640 1 -- 172.21.15.2:0/3664081332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac8e44c060 con 0x55ac8e522250 2024-03-15T13:04:43.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78810c7640 1 --2- 172.21.15.2:0/3664081332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522250 0x55ac8e533340 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f786c0115b0 tx=0x7f786c01e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebcda86066a5f90 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78657fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:43.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78657fa640 1 -- 172.21.15.2:0/3664081332 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c035070 con 0x55ac8e522250 2024-03-15T13:04:43.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78657fa640 1 -- 172.21.15.2:0/3664081332 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f786c031e40 con 0x55ac8e522250 2024-03-15T13:04:43.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.532+0000 7f78867e5740 1 -- 172.21.15.2:0/3664081332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f786c00fd80 con 0x55ac8e522250 2024-03-15T13:04:43.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.533+0000 7f78657fa640 1 -- 172.21.15.2:0/3664081332 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f786c011040 con 0x55ac8e522250 2024-03-15T13:04:43.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.533+0000 7f78867e5740 10 client.17385 fetch_fsmap learned FSMap version 45 2024-03-15T13:04:43.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.533+0000 7f78867e5740 1 -- 172.21.15.2:0/3664081332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55ac8e44a8c0 con 0x55ac8e522250 2024-03-15T13:04:43.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.533+0000 7f78657fa640 1 -- 172.21.15.2:0/3664081332 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f786c00fd80 con 0x55ac8e522250 2024-03-15T13:04:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.533+0000 7f78657fa640 1 -- 172.21.15.2:0/3664081332 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) v1 ==== 7962+0+0 (secure 0 0 0) 0x7f786c003710 con 0x55ac8e522250 2024-03-15T13:04:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.533+0000 7f78867e5740 10 client.17385 fetch_fsmap finished waiting for FSMap version 45 2024-03-15T13:04:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.533+0000 7f78867e5740 10 client.17385 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:04:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.533+0000 7f78867e5740 20 client.17385 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.533+0000 7f78867e5740 1 --2- 172.21.15.2:0/3664081332 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ac8e598420 0x55ac8e59a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.533+0000 7f78867e5740 4 client.17385 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:04:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.533+0000 7f78867e5740 1 -- 172.21.15.2:0/3664081332 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ac8e215890 con 0x55ac8e598420 2024-03-15T13:04:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.533+0000 7f78818c8640 1 --2- 172.21.15.2:0/3664081332 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ac8e598420 0x55ac8e59a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78657fa640 10 client.17385 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:04:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78818c8640 1 --2- 172.21.15.2:0/3664081332 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ac8e598420 0x55ac8e59a840 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=befc814b6b29d36b server_cookie=ab94761a115c2eb4 in_seq=0 out_seq=0 2024-03-15T13:04:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78657fa640 1 -- 172.21.15.2:0/3664081332 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ac8e215890 con 0x55ac8e598420 2024-03-15T13:04:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78657fa640 10 client.17385 handle_command_reply: tid=0 2024-03-15T13:04:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78867e5740 1 client.17385 shutdown 2024-03-15T13:04:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78867e5740 1 -- 172.21.15.2:0/3664081332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522250 msgr2=0x55ac8e533340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78867e5740 1 --2- 172.21.15.2:0/3664081332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522250 0x55ac8e533340 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f786c0115b0 tx=0x7f786c01e510 comp rx=0 tx=0).stop 2024-03-15T13:04:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78867e5740 1 -- 172.21.15.2:0/3664081332 shutdown_connections 2024-03-15T13:04:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78867e5740 1 --2- 172.21.15.2:0/3664081332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac8e52b750 0x55ac8e52fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78867e5740 1 --2- 172.21.15.2:0/3664081332 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55ac8e598420 0x55ac8e59a840 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78867e5740 1 --2- 172.21.15.2:0/3664081332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac8e522b70 0x55ac8e52f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78867e5740 1 --2- 172.21.15.2:0/3664081332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac8e522250 0x55ac8e533340 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78657fa640 0 client.17385 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:04:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78867e5740 1 -- 172.21.15.2:0/3664081332 >> 172.21.15.2:0/3664081332 conn(0x55ac8e51c980 msgr2=0x55ac8e5424e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78867e5740 1 -- 172.21.15.2:0/3664081332 shutdown_connections 2024-03-15T13:04:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78867e5740 1 -- 172.21.15.2:0/3664081332 wait complete. 2024-03-15T13:04:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f78867e5740 20 client.17385 trim_cache size 0 max 16384 2024-03-15T13:04:43.527 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:04:43.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f7883b2c640 1 -- 172.21.15.2:0/4256288236 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f78680a1ba0 msgr2=0x7f78680a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f7883b2c640 1 --2- 172.21.15.2:0/4256288236 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f78680a1ba0 0x7f78680a4060 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f787409a670 tx=0x7f7874009a70 comp rx=0 tx=0).stop 2024-03-15T13:04:43.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f7883b2c640 1 -- 172.21.15.2:0/4256288236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c107570 msgr2=0x7f787c074270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.534+0000 7f7883b2c640 1 --2- 172.21.15.2:0/4256288236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c107570 0x7f787c074270 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f787400eb10 tx=0x7f787400c2d0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.535+0000 7f7883b2c640 1 -- 172.21.15.2:0/4256288236 shutdown_connections 2024-03-15T13:04:43.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.535+0000 7f7883b2c640 1 --2- 172.21.15.2:0/4256288236 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f78680a1ba0 0x7f78680a4060 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.535+0000 7f7883b2c640 1 --2- 172.21.15.2:0/4256288236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c107570 0x7f787c074270 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.535+0000 7f7883b2c640 1 --2- 172.21.15.2:0/4256288236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f787c101e70 0x7f787c073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.535+0000 7f7883b2c640 1 --2- 172.21.15.2:0/4256288236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f787c1014b0 0x7f787c0737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.535+0000 7f7883b2c640 1 -- 172.21.15.2:0/4256288236 >> 172.21.15.2:0/4256288236 conn(0x7f787c0fcf20 msgr2=0x7f787c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.535+0000 7f7883b2c640 1 -- 172.21.15.2:0/4256288236 shutdown_connections 2024-03-15T13:04:43.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.535+0000 7f7883b2c640 1 -- 172.21.15.2:0/4256288236 wait complete. 2024-03-15T13:04:43.530 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:04:43.531 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:04:43.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.650+0000 7fa21b9c7640 1 -- 172.21.15.2:0/3315829813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa214100f80 msgr2=0x7fa214104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.650+0000 7fa21b9c7640 1 --2- 172.21.15.2:0/3315829813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa214100f80 0x7fa214104fc0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa204009930 tx=0x7fa20402bc30 comp rx=0 tx=0).stop 2024-03-15T13:04:43.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.650+0000 7fa21b9c7640 1 -- 172.21.15.2:0/3315829813 shutdown_connections 2024-03-15T13:04:43.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.650+0000 7fa21b9c7640 1 --2- 172.21.15.2:0/3315829813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2141056a0 0x7fa21410d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.650+0000 7fa21b9c7640 1 --2- 172.21.15.2:0/3315829813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa214100f80 0x7fa214104fc0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.650+0000 7fa21b9c7640 1 --2- 172.21.15.2:0/3315829813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2141005d0 0x7fa2141009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.650+0000 7fa21b9c7640 1 -- 172.21.15.2:0/3315829813 >> 172.21.15.2:0/3315829813 conn(0x7fa2140774f0 msgr2=0x7fa214077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.650+0000 7fa21b9c7640 1 -- 172.21.15.2:0/3315829813 shutdown_connections 2024-03-15T13:04:43.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.650+0000 7fa21b9c7640 1 -- 172.21.15.2:0/3315829813 wait complete. 2024-03-15T13:04:43.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.651+0000 7fa21b9c7640 1 Processor -- start 2024-03-15T13:04:43.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.651+0000 7fa21b9c7640 1 -- start start 2024-03-15T13:04:43.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.651+0000 7fa21b9c7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2141005d0 0x7fa21419d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.651+0000 7fa21b9c7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2141056a0 0x7fa21419dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.651+0000 7fa21b9c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2141979c0 0x7fa214197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.651+0000 7fa21b9c7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2141983b0 con 0x7fa2141979c0 2024-03-15T13:04:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.651+0000 7fa21b9c7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa214198520 con 0x7fa2141056a0 2024-03-15T13:04:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.651+0000 7fa21b9c7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2141986e0 con 0x7fa2141005d0 2024-03-15T13:04:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.651+0000 7fa219f64640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2141979c0 0x7fa214197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.651+0000 7fa219f64640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2141979c0 0x7fa214197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33218/0 (socket says 172.21.15.2:33218) 2024-03-15T13:04:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.651+0000 7fa219f64640 1 -- 172.21.15.2:0/2471580159 learned_addr learned my addr 172.21.15.2:0/2471580159 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.651+0000 7fa218f62640 1 --2- 172.21.15.2:0/2471580159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2141056a0 0x7fa21419dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.651+0000 7fa219763640 1 --2- 172.21.15.2:0/2471580159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2141005d0 0x7fa21419d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.651+0000 7fa219f64640 1 -- 172.21.15.2:0/2471580159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2141005d0 msgr2=0x7fa21419d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.651+0000 7fa219f64640 1 --2- 172.21.15.2:0/2471580159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2141005d0 0x7fa21419d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.652+0000 7fa219f64640 1 -- 172.21.15.2:0/2471580159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2141056a0 msgr2=0x7fa21419dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.652+0000 7fa219f64640 1 --2- 172.21.15.2:0/2471580159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2141056a0 0x7fa21419dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.652+0000 7fa219f64640 1 -- 172.21.15.2:0/2471580159 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204009560 con 0x7fa2141979c0 2024-03-15T13:04:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.652+0000 7fa219f64640 1 --2- 172.21.15.2:0/2471580159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2141979c0 0x7fa214197e70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00eb10 tx=0x7fa20c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.652+0000 7fa20a7fc640 1 -- 172.21.15.2:0/2471580159 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c004b20 con 0x7fa2141979c0 2024-03-15T13:04:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.652+0000 7fa20a7fc640 1 -- 172.21.15.2:0/2471580159 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa20c004c80 con 0x7fa2141979c0 2024-03-15T13:04:43.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.652+0000 7fa21b9c7640 1 -- 172.21.15.2:0/2471580159 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2141a46d0 con 0x7fa2141979c0 2024-03-15T13:04:43.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.652+0000 7fa20a7fc640 1 -- 172.21.15.2:0/2471580159 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c010690 con 0x7fa2141979c0 2024-03-15T13:04:43.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.652+0000 7fa21b9c7640 1 -- 172.21.15.2:0/2471580159 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2141a4c00 con 0x7fa2141979c0 2024-03-15T13:04:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.654+0000 7fa20a7fc640 1 -- 172.21.15.2:0/2471580159 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fa20c004df0 con 0x7fa2141979c0 2024-03-15T13:04:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.654+0000 7fa21e59e740 1 Processor -- start 2024-03-15T13:04:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa21e59e740 1 -- start start 2024-03-15T13:04:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa21e59e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475aaf80 0x5622475ab360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa21e59e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475ab8a0 0x5622475b8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa21e59e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475b8b60 0x5622475baf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa20a7fc640 1 --2- 172.21.15.2:0/2471580159 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2000a1c20 0x7fa2000a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa21e59e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562247566960 con 0x5622475aaf80 2024-03-15T13:04:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa21e59e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56224751a310 con 0x5622475b8b60 2024-03-15T13:04:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa21e59e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562247517840 con 0x5622475ab8a0 2024-03-15T13:04:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa218f62640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475aaf80 0x5622475ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa20a7fc640 1 -- 172.21.15.2:0/2471580159 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa20c014070 con 0x7fa2141979c0 2024-03-15T13:04:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa218f62640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475aaf80 0x5622475ab360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33224/0 (socket says 172.21.15.2:33224) 2024-03-15T13:04:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa219763640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475ab8a0 0x5622475b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa218f62640 1 -- 172.21.15.2:0/935108094 learned_addr learned my addr 172.21.15.2:0/935108094 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa219f64640 1 --2- 172.21.15.2:0/935108094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475b8b60 0x5622475baf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa218f62640 1 --2- 172.21.15.2:0/2471580159 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2000a1c20 0x7fa2000a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa219763640 1 -- 172.21.15.2:0/935108094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475b8b60 msgr2=0x5622475baf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa219763640 1 --2- 172.21.15.2:0/935108094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475b8b60 0x5622475baf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa219763640 1 -- 172.21.15.2:0/935108094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475aaf80 msgr2=0x5622475ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa219763640 1 --2- 172.21.15.2:0/935108094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475aaf80 0x5622475ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa219763640 1 -- 172.21.15.2:0/935108094 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622474d5060 con 0x5622475ab8a0 2024-03-15T13:04:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa218f62640 1 --2- 172.21.15.2:0/935108094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475aaf80 0x5622475ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa218f62640 1 --2- 172.21.15.2:0/2471580159 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2000a1c20 0x7fa2000a40e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa204000c00 tx=0x7fa204005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.655+0000 7fa219763640 1 --2- 172.21.15.2:0/935108094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475ab8a0 0x5622475b8620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa210009bf0 tx=0x7fa210011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f61e71eab91bfd86 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.656+0000 7fa1ff7fe640 1 -- 172.21.15.2:0/935108094 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210004050 con 0x5622475ab8a0 2024-03-15T13:04:43.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.656+0000 7fa1ff7fe640 1 -- 172.21.15.2:0/935108094 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa210005020 con 0x5622475ab8a0 2024-03-15T13:04:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.656+0000 7fa1ff7fe640 1 -- 172.21.15.2:0/935108094 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210004d60 con 0x5622475ab8a0 2024-03-15T13:04:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.656+0000 7fa21e59e740 1 -- 172.21.15.2:0/935108094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475ab8a0 msgr2=0x5622475b8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.656+0000 7fa21e59e740 1 --2- 172.21.15.2:0/935108094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475ab8a0 0x5622475b8620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa210009bf0 tx=0x7fa210011810 comp rx=0 tx=0).stop 2024-03-15T13:04:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.656+0000 7fa21e59e740 1 -- 172.21.15.2:0/935108094 shutdown_connections 2024-03-15T13:04:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.656+0000 7fa21e59e740 1 --2- 172.21.15.2:0/935108094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475b8b60 0x5622475baf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.656+0000 7fa21e59e740 1 --2- 172.21.15.2:0/935108094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475ab8a0 0x5622475b8620 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.656+0000 7fa21e59e740 1 --2- 172.21.15.2:0/935108094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475aaf80 0x5622475ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.656+0000 7fa21e59e740 1 -- 172.21.15.2:0/935108094 >> 172.21.15.2:0/935108094 conn(0x5622475a5980 msgr2=0x5622475a5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.656+0000 7fa21e59e740 1 -- 172.21.15.2:0/935108094 shutdown_connections 2024-03-15T13:04:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.656+0000 7fa21e59e740 1 -- 172.21.15.2:0/935108094 wait complete. 2024-03-15T13:04:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.657+0000 7fa21e59e740 1 -- start start 2024-03-15T13:04:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.657+0000 7fa21e59e740 1 Processor -- start 2024-03-15T13:04:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.657+0000 7fa21e59e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475aaf80 0x5622475b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.657+0000 7fa21e59e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475ab8a0 0x5622475b5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.657+0000 7fa21e59e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475b8b60 0x5622475b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.657+0000 7fa21e59e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562247513020 con 0x5622475ab8a0 2024-03-15T13:04:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.657+0000 7fa21e59e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622471e6430 con 0x5622475aaf80 2024-03-15T13:04:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.657+0000 7fa21e59e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622471a2e00 con 0x5622475b8b60 2024-03-15T13:04:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.657+0000 7fa21e59e740 10 client.0 _ll_register_callbacks cb 0x5622474bacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.657+0000 7fa218f62640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475ab8a0 0x5622475b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.657+0000 7fa218f62640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475ab8a0 0x5622475b5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33236/0 (socket says 172.21.15.2:33236) 2024-03-15T13:04:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.657+0000 7fa218f62640 1 -- 172.21.15.2:0/4076886242 learned_addr learned my addr 172.21.15.2:0/4076886242 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.657+0000 7fa219f64640 1 --2- 172.21.15.2:0/4076886242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475b8b60 0x5622475b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.657+0000 7fa219763640 1 --2- 172.21.15.2:0/4076886242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475aaf80 0x5622475b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.658+0000 7fa218f62640 1 -- 172.21.15.2:0/4076886242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475b8b60 msgr2=0x5622475b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.658+0000 7fa218f62640 1 --2- 172.21.15.2:0/4076886242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475b8b60 0x5622475b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.658+0000 7fa218f62640 1 -- 172.21.15.2:0/4076886242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475aaf80 msgr2=0x5622475b8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.658+0000 7fa218f62640 1 --2- 172.21.15.2:0/4076886242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475aaf80 0x5622475b8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.658+0000 7fa218f62640 1 -- 172.21.15.2:0/4076886242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa210009850 con 0x5622475ab8a0 2024-03-15T13:04:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.658+0000 7fa218f62640 1 --2- 172.21.15.2:0/4076886242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475ab8a0 0x5622475b5810 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa204036040 tx=0x7fa204009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6629430aa53d7192 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.658+0000 7fa1fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.658+0000 7fa1fcff9640 1 -- 172.21.15.2:0/4076886242 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204048050 con 0x5622475ab8a0 2024-03-15T13:04:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.658+0000 7fa1fcff9640 1 -- 172.21.15.2:0/4076886242 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa204003980 con 0x5622475ab8a0 2024-03-15T13:04:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.658+0000 7fa21e59e740 1 -- 172.21.15.2:0/4076886242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622472127e0 con 0x5622475ab8a0 2024-03-15T13:04:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.658+0000 7fa1fcff9640 1 -- 172.21.15.2:0/4076886242 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa204003b40 con 0x5622475ab8a0 2024-03-15T13:04:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.658+0000 7fa21e59e740 10 client.17397 fetch_fsmap learned FSMap version 45 2024-03-15T13:04:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.658+0000 7fa21e59e740 1 -- 172.21.15.2:0/4076886242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5622475116a0 con 0x5622475ab8a0 2024-03-15T13:04:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.658+0000 7fa1fcff9640 1 -- 172.21.15.2:0/4076886242 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5622472127e0 con 0x5622475ab8a0 2024-03-15T13:04:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.659+0000 7fa1fcff9640 1 -- 172.21.15.2:0/4076886242 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fa204043480 con 0x5622475ab8a0 2024-03-15T13:04:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.659+0000 7fa21e59e740 10 client.17397 fetch_fsmap finished waiting for FSMap version 45 2024-03-15T13:04:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.659+0000 7fa21e59e740 10 client.17397 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:04:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.659+0000 7fa21e59e740 20 client.17397 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.659+0000 7fa21e59e740 1 --2- 172.21.15.2:0/4076886242 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5622475c8c90 0x5622475cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.659+0000 7fa21e59e740 4 client.17397 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.659+0000 7fa21e59e740 1 -- 172.21.15.2:0/4076886242 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56224724f720 con 0x5622475c8c90 2024-03-15T13:04:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.659+0000 7fa219763640 1 --2- 172.21.15.2:0/4076886242 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5622475c8c90 0x5622475cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.659+0000 7fa1fcff9640 10 client.17397 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:04:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.659+0000 7fa219763640 1 --2- 172.21.15.2:0/4076886242 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5622475c8c90 0x5622475cb0b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b08db4878c5a47f5 server_cookie=baf1e99f547cf38e in_seq=0 out_seq=0 2024-03-15T13:04:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.660+0000 7fa1fcff9640 1 -- 172.21.15.2:0/4076886242 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56224724f720 con 0x5622475c8c90 2024-03-15T13:04:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.660+0000 7fa1fcff9640 10 client.17397 handle_command_reply: tid=0 2024-03-15T13:04:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.660+0000 7fa21e59e740 1 client.17397 shutdown 2024-03-15T13:04:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.660+0000 7fa21e59e740 1 -- 172.21.15.2:0/4076886242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475ab8a0 msgr2=0x5622475b5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.660+0000 7fa21e59e740 1 --2- 172.21.15.2:0/4076886242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475ab8a0 0x5622475b5810 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa204036040 tx=0x7fa204009260 comp rx=0 tx=0).stop 2024-03-15T13:04:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.660+0000 7fa21e59e740 1 -- 172.21.15.2:0/4076886242 shutdown_connections 2024-03-15T13:04:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.660+0000 7fa21e59e740 1 --2- 172.21.15.2:0/4076886242 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5622475c8c90 0x5622475cb0b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.660+0000 7fa21e59e740 1 --2- 172.21.15.2:0/4076886242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475b8b60 0x5622475b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.660+0000 7fa21e59e740 1 --2- 172.21.15.2:0/4076886242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475ab8a0 0x5622475b5810 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.660+0000 7fa1fcff9640 0 client.17397 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:04:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.660+0000 7fa21e59e740 1 --2- 172.21.15.2:0/4076886242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475aaf80 0x5622475b8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.660+0000 7fa21e59e740 1 -- 172.21.15.2:0/4076886242 >> 172.21.15.2:0/4076886242 conn(0x5622475a5980 msgr2=0x5622475a76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.660+0000 7fa21e59e740 1 -- 172.21.15.2:0/4076886242 shutdown_connections 2024-03-15T13:04:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.660+0000 7fa21e59e740 1 -- 172.21.15.2:0/4076886242 wait complete. 2024-03-15T13:04:43.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.661+0000 7fa21e59e740 20 client.17397 trim_cache size 0 max 16384 2024-03-15T13:04:43.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.673+0000 7fa21e59e740 1 Processor -- start 2024-03-15T13:04:43.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.673+0000 7fa21e59e740 1 -- start start 2024-03-15T13:04:43.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.673+0000 7fa21e59e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475aaf80 0x5622475b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.673+0000 7fa21e59e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475ab8a0 0x5622475b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.673+0000 7fa21e59e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475b8b60 0x5622475b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.673+0000 7fa21e59e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562247498130 con 0x5622475b8b60 2024-03-15T13:04:43.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.673+0000 7fa21e59e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622475aa5c0 con 0x5622475ab8a0 2024-03-15T13:04:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.673+0000 7fa21e59e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622475a15c0 con 0x5622475aaf80 2024-03-15T13:04:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.673+0000 7fa219f64640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475b8b60 0x5622475b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.673+0000 7fa219f64640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475b8b60 0x5622475b8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33244/0 (socket says 172.21.15.2:33244) 2024-03-15T13:04:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.673+0000 7fa219f64640 1 -- 172.21.15.2:0/369565274 learned_addr learned my addr 172.21.15.2:0/369565274 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.673+0000 7fa219763640 1 --2- 172.21.15.2:0/369565274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475aaf80 0x5622475b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.673+0000 7fa218f62640 1 --2- 172.21.15.2:0/369565274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475ab8a0 0x5622475b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa219f64640 1 -- 172.21.15.2:0/369565274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475aaf80 msgr2=0x5622475b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa219f64640 1 --2- 172.21.15.2:0/369565274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475aaf80 0x5622475b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa219f64640 1 -- 172.21.15.2:0/369565274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475ab8a0 msgr2=0x5622475b7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa219f64640 1 --2- 172.21.15.2:0/369565274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475ab8a0 0x5622475b7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa219f64640 1 -- 172.21.15.2:0/369565274 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622475b1470 con 0x5622475b8b60 2024-03-15T13:04:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa219763640 1 --2- 172.21.15.2:0/369565274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475aaf80 0x5622475b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa219f64640 1 --2- 172.21.15.2:0/369565274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475b8b60 0x5622475b8480 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa20c018000 tx=0x7fa20c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc723185e85db7d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa1feffd640 1 -- 172.21.15.2:0/369565274 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c09aa40 con 0x5622475b8b60 2024-03-15T13:04:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa1feffd640 1 -- 172.21.15.2:0/369565274 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa20c09eb90 con 0x5622475b8b60 2024-03-15T13:04:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa1feffd640 1 -- 172.21.15.2:0/369565274 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c096790 con 0x5622475b8b60 2024-03-15T13:04:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa21e59e740 1 -- 172.21.15.2:0/369565274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475b8b60 msgr2=0x5622475b8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa21e59e740 1 --2- 172.21.15.2:0/369565274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475b8b60 0x5622475b8480 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa20c018000 tx=0x7fa20c00c950 comp rx=0 tx=0).stop 2024-03-15T13:04:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa21e59e740 1 -- 172.21.15.2:0/369565274 shutdown_connections 2024-03-15T13:04:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa21e59e740 1 --2- 172.21.15.2:0/369565274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475b8b60 0x5622475b8480 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa21e59e740 1 --2- 172.21.15.2:0/369565274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475ab8a0 0x5622475b7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa21e59e740 1 --2- 172.21.15.2:0/369565274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475aaf80 0x5622475b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa21e59e740 1 -- 172.21.15.2:0/369565274 >> 172.21.15.2:0/369565274 conn(0x5622475a5980 msgr2=0x5622475a4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.674+0000 7fa21e59e740 1 -- 172.21.15.2:0/369565274 shutdown_connections 2024-03-15T13:04:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.675+0000 7fa21e59e740 1 -- 172.21.15.2:0/369565274 wait complete. 2024-03-15T13:04:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.675+0000 7fa21e59e740 1 -- start start 2024-03-15T13:04:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.675+0000 7fa21e59e740 1 Processor -- start 2024-03-15T13:04:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.675+0000 7fa21e59e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475aaf80 0x5622475bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.675+0000 7fa21e59e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475ab8a0 0x5622475bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.675+0000 7fa21e59e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475b8b60 0x5622475be480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.675+0000 7fa21e59e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622475b3740 con 0x5622475ab8a0 2024-03-15T13:04:43.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.675+0000 7fa21e59e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622475b38b0 con 0x5622475aaf80 2024-03-15T13:04:43.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.675+0000 7fa21e59e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562247624550 con 0x5622475b8b60 2024-03-15T13:04:43.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.675+0000 7fa21e59e740 10 client.0 _ll_register_callbacks cb 0x7fa204009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:43.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.675+0000 7fa218f62640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475ab8a0 0x5622475bdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.675+0000 7fa218f62640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475ab8a0 0x5622475bdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33246/0 (socket says 172.21.15.2:33246) 2024-03-15T13:04:43.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.675+0000 7fa218f62640 1 -- 172.21.15.2:0/3529665298 learned_addr learned my addr 172.21.15.2:0/3529665298 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.675+0000 7fa219763640 1 --2- 172.21.15.2:0/3529665298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475aaf80 0x5622475bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.675+0000 7fa219f64640 1 --2- 172.21.15.2:0/3529665298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475b8b60 0x5622475be480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa218f62640 1 -- 172.21.15.2:0/3529665298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475b8b60 msgr2=0x5622475be480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa218f62640 1 --2- 172.21.15.2:0/3529665298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475b8b60 0x5622475be480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa218f62640 1 -- 172.21.15.2:0/3529665298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475aaf80 msgr2=0x5622475bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa218f62640 1 --2- 172.21.15.2:0/3529665298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475aaf80 0x5622475bda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa218f62640 1 -- 172.21.15.2:0/3529665298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622474d5060 con 0x5622475ab8a0 2024-03-15T13:04:43.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa218f62640 1 --2- 172.21.15.2:0/3529665298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475ab8a0 0x5622475bdf40 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fa20402cdc0 tx=0x7fa2040048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d8c07b992a451e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa1fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:43.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa1fcff9640 1 -- 172.21.15.2:0/3529665298 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204048040 con 0x5622475ab8a0 2024-03-15T13:04:43.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa1fcff9640 1 -- 172.21.15.2:0/3529665298 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fa204043cf0 con 0x5622475ab8a0 2024-03-15T13:04:43.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa21e59e740 1 -- 172.21.15.2:0/3529665298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa210005020 con 0x5622475ab8a0 2024-03-15T13:04:43.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa1fcff9640 1 -- 172.21.15.2:0/3529665298 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2040044e0 con 0x5622475ab8a0 2024-03-15T13:04:43.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa21e59e740 10 client.17403 fetch_fsmap learned FSMap version 45 2024-03-15T13:04:43.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa21e59e740 1 -- 172.21.15.2:0/3529665298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7fa2141a4c00 con 0x5622475ab8a0 2024-03-15T13:04:43.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa1fcff9640 1 -- 172.21.15.2:0/3529665298 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa210005020 con 0x5622475ab8a0 2024-03-15T13:04:43.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa1fcff9640 1 -- 172.21.15.2:0/3529665298 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fa204004f70 con 0x5622475ab8a0 2024-03-15T13:04:43.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa21e59e740 10 client.17403 fetch_fsmap finished waiting for FSMap version 45 2024-03-15T13:04:43.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa21e59e740 10 client.17403 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:04:43.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa21e59e740 20 client.17403 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:43.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa21e59e740 1 --2- 172.21.15.2:0/3529665298 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x562247625ac0 0x562247627ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa21e59e740 4 client.17403 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.676+0000 7fa21e59e740 1 -- 172.21.15.2:0/3529665298 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56224729e890 con 0x562247625ac0 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa219763640 1 --2- 172.21.15.2:0/3529665298 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x562247625ac0 0x562247627ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa219763640 1 --2- 172.21.15.2:0/3529665298 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x562247625ac0 0x562247627ee0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b093c553a8b386f3 server_cookie=e12b171a234ab49e in_seq=0 out_seq=0 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa1fcff9640 10 client.17403 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa1fcff9640 1 -- 172.21.15.2:0/3529665298 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56224729e890 con 0x562247625ac0 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa1fcff9640 10 client.17403 handle_command_reply: tid=0 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa21e59e740 1 client.17403 shutdown 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa21e59e740 1 -- 172.21.15.2:0/3529665298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475ab8a0 msgr2=0x5622475bdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa21e59e740 1 --2- 172.21.15.2:0/3529665298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475ab8a0 0x5622475bdf40 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fa20402cdc0 tx=0x7fa2040048e0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa21e59e740 1 -- 172.21.15.2:0/3529665298 shutdown_connections 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa21e59e740 1 --2- 172.21.15.2:0/3529665298 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x562247625ac0 0x562247627ee0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa21e59e740 1 --2- 172.21.15.2:0/3529665298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5622475b8b60 0x5622475be480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa1fcff9640 0 client.17403 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa21e59e740 1 --2- 172.21.15.2:0/3529665298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622475ab8a0 0x5622475bdf40 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa21e59e740 1 --2- 172.21.15.2:0/3529665298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622475aaf80 0x5622475bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa21e59e740 1 -- 172.21.15.2:0/3529665298 >> 172.21.15.2:0/3529665298 conn(0x5622475a5980 msgr2=0x5622475a6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa21e59e740 1 -- 172.21.15.2:0/3529665298 shutdown_connections 2024-03-15T13:04:43.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.677+0000 7fa21e59e740 1 -- 172.21.15.2:0/3529665298 wait complete. 2024-03-15T13:04:43.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.678+0000 7fa21e59e740 20 client.17403 trim_cache size 0 max 16384 2024-03-15T13:04:43.669 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:04:43.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.678+0000 7fa21b9c7640 1 -- 172.21.15.2:0/2471580159 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2000a1c20 msgr2=0x7fa2000a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.678+0000 7fa21b9c7640 1 --2- 172.21.15.2:0/2471580159 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2000a1c20 0x7fa2000a40e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa204000c00 tx=0x7fa204005fd0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.678+0000 7fa21b9c7640 1 -- 172.21.15.2:0/2471580159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2141979c0 msgr2=0x7fa214197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.678+0000 7fa21b9c7640 1 --2- 172.21.15.2:0/2471580159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2141979c0 0x7fa214197e70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00eb10 tx=0x7fa20c00c2d0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.678+0000 7fa21b9c7640 1 -- 172.21.15.2:0/2471580159 shutdown_connections 2024-03-15T13:04:43.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.678+0000 7fa21b9c7640 1 --2- 172.21.15.2:0/2471580159 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2000a1c20 0x7fa2000a40e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.678+0000 7fa21b9c7640 1 --2- 172.21.15.2:0/2471580159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2141979c0 0x7fa214197e70 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.678+0000 7fa21b9c7640 1 --2- 172.21.15.2:0/2471580159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2141056a0 0x7fa21419dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.678+0000 7fa21b9c7640 1 --2- 172.21.15.2:0/2471580159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2141005d0 0x7fa21419d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.678+0000 7fa21b9c7640 1 -- 172.21.15.2:0/2471580159 >> 172.21.15.2:0/2471580159 conn(0x7fa2140774f0 msgr2=0x7fa2140ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.678+0000 7fa21b9c7640 1 -- 172.21.15.2:0/2471580159 shutdown_connections 2024-03-15T13:04:43.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.678+0000 7fa21b9c7640 1 -- 172.21.15.2:0/2471580159 wait complete. 2024-03-15T13:04:43.674 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:04:43.674 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:04:43.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.793+0000 7fc92a2c9640 1 -- 172.21.15.2:0/2096453384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9241014b0 msgr2=0x7fc924101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.793+0000 7fc92a2c9640 1 --2- 172.21.15.2:0/2096453384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9241014b0 0x7fc924101930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc918009890 tx=0x7fc91802bb70 comp rx=0 tx=0).stop 2024-03-15T13:04:43.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.793+0000 7fc92a2c9640 1 -- 172.21.15.2:0/2096453384 shutdown_connections 2024-03-15T13:04:43.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.793+0000 7fc92a2c9640 1 --2- 172.21.15.2:0/2096453384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc924101e70 0x7fc92410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.793+0000 7fc92a2c9640 1 --2- 172.21.15.2:0/2096453384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9241014b0 0x7fc924101930 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.793+0000 7fc92a2c9640 1 --2- 172.21.15.2:0/2096453384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc924107570 0x7fc924107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.793+0000 7fc92a2c9640 1 -- 172.21.15.2:0/2096453384 >> 172.21.15.2:0/2096453384 conn(0x7fc9240fcf20 msgr2=0x7fc9240ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.793+0000 7fc92a2c9640 1 -- 172.21.15.2:0/2096453384 shutdown_connections 2024-03-15T13:04:43.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.793+0000 7fc92a2c9640 1 -- 172.21.15.2:0/2096453384 wait complete. 2024-03-15T13:04:43.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.794+0000 7fc92a2c9640 1 Processor -- start 2024-03-15T13:04:43.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.794+0000 7fc92a2c9640 1 -- start start 2024-03-15T13:04:43.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.794+0000 7fc92a2c9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9241014b0 0x7fc92419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.794+0000 7fc92a2c9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc924101e70 0x7fc92419dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.794+0000 7fc92a2c9640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc924107570 0x7fc9241978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.794+0000 7fc92a2c9640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc924197f30 con 0x7fc9241014b0 2024-03-15T13:04:43.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.794+0000 7fc92a2c9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9241980a0 con 0x7fc924101e70 2024-03-15T13:04:43.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.794+0000 7fc92a2c9640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc924198260 con 0x7fc924107570 2024-03-15T13:04:43.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc923fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9241014b0 0x7fc92419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc923fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9241014b0 0x7fc92419d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33270/0 (socket says 172.21.15.2:33270) 2024-03-15T13:04:43.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc923fff640 1 -- 172.21.15.2:0/3043782662 learned_addr learned my addr 172.21.15.2:0/3043782662 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc9237fe640 1 --2- 172.21.15.2:0/3043782662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc924101e70 0x7fc92419dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc928866640 1 --2- 172.21.15.2:0/3043782662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc924107570 0x7fc9241978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc923fff640 1 -- 172.21.15.2:0/3043782662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc924107570 msgr2=0x7fc9241978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc923fff640 1 --2- 172.21.15.2:0/3043782662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc924107570 0x7fc9241978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc923fff640 1 -- 172.21.15.2:0/3043782662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc924101e70 msgr2=0x7fc92419dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc923fff640 1 --2- 172.21.15.2:0/3043782662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc924101e70 0x7fc92419dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc923fff640 1 -- 172.21.15.2:0/3043782662 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc918009490 con 0x7fc9241014b0 2024-03-15T13:04:43.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc9237fe640 1 --2- 172.21.15.2:0/3043782662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc924101e70 0x7fc92419dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:43.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc923fff640 1 --2- 172.21.15.2:0/3043782662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9241014b0 0x7fc92419d730 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc91400e9c0 tx=0x7fc91400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc9217fa640 1 -- 172.21.15.2:0/3043782662 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91400ccc0 con 0x7fc9241014b0 2024-03-15T13:04:43.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc92a2c9640 1 -- 172.21.15.2:0/3043782662 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc924198540 con 0x7fc9241014b0 2024-03-15T13:04:43.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc9217fa640 1 -- 172.21.15.2:0/3043782662 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc91400ce20 con 0x7fc9241014b0 2024-03-15T13:04:43.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc9217fa640 1 -- 172.21.15.2:0/3043782662 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914010430 con 0x7fc9241014b0 2024-03-15T13:04:43.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.795+0000 7fc92a2c9640 1 -- 172.21.15.2:0/3043782662 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9241a47d0 con 0x7fc9241014b0 2024-03-15T13:04:43.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc9217fa640 1 -- 172.21.15.2:0/3043782662 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fc914004f30 con 0x7fc9241014b0 2024-03-15T13:04:43.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc92cf87740 1 Processor -- start 2024-03-15T13:04:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc92cf87740 1 -- start start 2024-03-15T13:04:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc92cf87740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3c5250 0x55f3ff3c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc92cf87740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc92cf87740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3ce750 0x55f3ff3d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc92cf87740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ff380960 con 0x55f3ff3ce750 2024-03-15T13:04:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc92cf87740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ff334310 con 0x55f3ff3c5250 2024-03-15T13:04:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc92cf87740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ff331840 con 0x55f3ff3c5b70 2024-03-15T13:04:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc9217fa640 1 --2- 172.21.15.2:0/3043782662 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc90c0a1ba0 0x7fc90c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc9237fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3ce750 0x55f3ff3d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc9237fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3ce750 0x55f3ff3d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33278/0 (socket says 172.21.15.2:33278) 2024-03-15T13:04:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc923fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc9237fe640 1 -- 172.21.15.2:0/2378907033 learned_addr learned my addr 172.21.15.2:0/2378907033 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc928866640 1 --2- 172.21.15.2:0/2378907033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3c5250 0x55f3ff3c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc928866640 1 --2- 172.21.15.2:0/3043782662 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc90c0a1ba0 0x7fc90c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc9217fa640 1 -- 172.21.15.2:0/3043782662 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc914014070 con 0x7fc9241014b0 2024-03-15T13:04:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc9237fe640 1 -- 172.21.15.2:0/2378907033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 msgr2=0x55f3ff3ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.798+0000 7fc9237fe640 1 --2- 172.21.15.2:0/2378907033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.799+0000 7fc9237fe640 1 -- 172.21.15.2:0/2378907033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3c5250 msgr2=0x55f3ff3c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.799+0000 7fc9237fe640 1 --2- 172.21.15.2:0/2378907033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3c5250 0x55f3ff3c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.799+0000 7fc9237fe640 1 -- 172.21.15.2:0/2378907033 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3ff2ef060 con 0x55f3ff3ce750 2024-03-15T13:04:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.799+0000 7fc928866640 1 --2- 172.21.15.2:0/3043782662 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc90c0a1ba0 0x7fc90c0a4060 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc91c005c40 tx=0x7fc91c009500 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.799+0000 7fc9237fe640 1 --2- 172.21.15.2:0/2378907033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3ce750 0x55f3ff3d0b40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc91802c0b0 tx=0x7fc918005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9043b9bb5ba10239 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.799+0000 7fc90a7fc640 1 -- 172.21.15.2:0/2378907033 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918004230 con 0x55f3ff3ce750 2024-03-15T13:04:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.799+0000 7fc90a7fc640 1 -- 172.21.15.2:0/2378907033 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc918034910 con 0x55f3ff3ce750 2024-03-15T13:04:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.799+0000 7fc90a7fc640 1 -- 172.21.15.2:0/2378907033 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9180035f0 con 0x55f3ff3ce750 2024-03-15T13:04:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.800+0000 7fc92cf87740 1 -- 172.21.15.2:0/2378907033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3ce750 msgr2=0x55f3ff3d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.800+0000 7fc92cf87740 1 --2- 172.21.15.2:0/2378907033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3ce750 0x55f3ff3d0b40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc91802c0b0 tx=0x7fc918005b20 comp rx=0 tx=0).stop 2024-03-15T13:04:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.800+0000 7fc92cf87740 1 -- 172.21.15.2:0/2378907033 shutdown_connections 2024-03-15T13:04:43.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.800+0000 7fc92cf87740 1 --2- 172.21.15.2:0/2378907033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3ce750 0x55f3ff3d0b40 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.800+0000 7fc92cf87740 1 --2- 172.21.15.2:0/2378907033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.800+0000 7fc92cf87740 1 --2- 172.21.15.2:0/2378907033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3c5250 0x55f3ff3c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.800+0000 7fc92cf87740 1 -- 172.21.15.2:0/2378907033 >> 172.21.15.2:0/2378907033 conn(0x55f3ff3bf980 msgr2=0x55f3ff3bfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.800+0000 7fc92cf87740 1 -- 172.21.15.2:0/2378907033 shutdown_connections 2024-03-15T13:04:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.800+0000 7fc92cf87740 1 -- 172.21.15.2:0/2378907033 wait complete. 2024-03-15T13:04:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.800+0000 7fc92cf87740 1 -- start start 2024-03-15T13:04:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.800+0000 7fc92cf87740 1 Processor -- start 2024-03-15T13:04:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.800+0000 7fc92cf87740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3c5250 0x55f3ff3ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.800+0000 7fc92cf87740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc92cf87740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3ce750 0x55f3ff3cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc92cf87740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ff32d020 con 0x55f3ff3c5250 2024-03-15T13:04:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc92cf87740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ff000430 con 0x55f3ff3ce750 2024-03-15T13:04:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc92cf87740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3fefbce00 con 0x55f3ff3c5b70 2024-03-15T13:04:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc92cf87740 10 client.0 _ll_register_callbacks cb 0x55f3ff2d4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:43.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc923fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3c5250 0x55f3ff3ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc923fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3c5250 0x55f3ff3ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33294/0 (socket says 172.21.15.2:33294) 2024-03-15T13:04:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc923fff640 1 -- 172.21.15.2:0/2233987550 learned_addr learned my addr 172.21.15.2:0/2233987550 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc9237fe640 1 --2- 172.21.15.2:0/2233987550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc928866640 1 --2- 172.21.15.2:0/2233987550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3ce750 0x55f3ff3cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc923fff640 1 -- 172.21.15.2:0/2233987550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 msgr2=0x55f3ff3cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc923fff640 1 --2- 172.21.15.2:0/2233987550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc923fff640 1 -- 172.21.15.2:0/2233987550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3ce750 msgr2=0x55f3ff3cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc923fff640 1 --2- 172.21.15.2:0/2233987550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3ce750 0x55f3ff3cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc923fff640 1 -- 172.21.15.2:0/2233987550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92410ec30 con 0x55f3ff3c5250 2024-03-15T13:04:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc923fff640 1 --2- 172.21.15.2:0/2233987550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3c5250 0x55f3ff3ce090 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc91409a390 tx=0x7fc914096a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386b76dd4ce471b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc9037fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc9037fe640 1 -- 172.21.15.2:0/2233987550 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91406c040 con 0x55f3ff3c5250 2024-03-15T13:04:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc9037fe640 1 -- 172.21.15.2:0/2233987550 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc91409e7e0 con 0x55f3ff3c5250 2024-03-15T13:04:43.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc92cf87740 1 -- 172.21.15.2:0/2233987550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3ff02c7e0 con 0x55f3ff3c5250 2024-03-15T13:04:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc9037fe640 1 -- 172.21.15.2:0/2233987550 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9140a2e30 con 0x55f3ff3c5250 2024-03-15T13:04:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc92cf87740 10 client.17415 fetch_fsmap learned FSMap version 45 2024-03-15T13:04:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc92cf87740 1 -- 172.21.15.2:0/2233987550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55f3ff32b6a0 con 0x55f3ff3c5250 2024-03-15T13:04:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.801+0000 7fc9037fe640 1 -- 172.21.15.2:0/2233987550 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55f3ff02c7e0 con 0x55f3ff3c5250 2024-03-15T13:04:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.802+0000 7fc9037fe640 1 -- 172.21.15.2:0/2233987550 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fc91401f060 con 0x55f3ff3c5250 2024-03-15T13:04:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.802+0000 7fc92cf87740 10 client.17415 fetch_fsmap finished waiting for FSMap version 45 2024-03-15T13:04:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.802+0000 7fc92cf87740 10 client.17415 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:04:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.802+0000 7fc92cf87740 20 client.17415 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.802+0000 7fc92cf87740 1 --2- 172.21.15.2:0/2233987550 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f3ff3de5f0 0x55f3ff3e0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.802+0000 7fc92cf87740 4 client.17415 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.802+0000 7fc92cf87740 1 -- 172.21.15.2:0/2233987550 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3ff069720 con 0x55f3ff3de5f0 2024-03-15T13:04:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.802+0000 7fc9237fe640 1 --2- 172.21.15.2:0/2233987550 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f3ff3de5f0 0x55f3ff3e0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.802+0000 7fc9237fe640 1 --2- 172.21.15.2:0/2233987550 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f3ff3de5f0 0x55f3ff3e0a00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f264d99dac90097 server_cookie=76ae92ec23a6fe33 in_seq=0 out_seq=0 2024-03-15T13:04:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.802+0000 7fc9037fe640 10 client.17415 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:04:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.803+0000 7fc9037fe640 1 -- 172.21.15.2:0/2233987550 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f3ff069720 con 0x55f3ff3de5f0 2024-03-15T13:04:43.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.803+0000 7fc9037fe640 10 client.17415 handle_command_reply: tid=0 2024-03-15T13:04:43.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.803+0000 7fc92cf87740 1 client.17415 shutdown 2024-03-15T13:04:43.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.803+0000 7fc92cf87740 1 -- 172.21.15.2:0/2233987550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3c5250 msgr2=0x55f3ff3ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.803+0000 7fc92cf87740 1 --2- 172.21.15.2:0/2233987550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3c5250 0x55f3ff3ce090 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc91409a390 tx=0x7fc914096a60 comp rx=0 tx=0).stop 2024-03-15T13:04:43.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.803+0000 7fc92cf87740 1 -- 172.21.15.2:0/2233987550 shutdown_connections 2024-03-15T13:04:43.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.803+0000 7fc92cf87740 1 --2- 172.21.15.2:0/2233987550 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f3ff3de5f0 0x55f3ff3e0a00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.803+0000 7fc92cf87740 1 --2- 172.21.15.2:0/2233987550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3ce750 0x55f3ff3cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.803+0000 7fc92cf87740 1 --2- 172.21.15.2:0/2233987550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.803+0000 7fc92cf87740 1 --2- 172.21.15.2:0/2233987550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3c5250 0x55f3ff3ce090 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.803+0000 7fc9037fe640 0 client.17415 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:04:43.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.803+0000 7fc92cf87740 1 -- 172.21.15.2:0/2233987550 >> 172.21.15.2:0/2233987550 conn(0x55f3ff3bf980 msgr2=0x55f3ff3c16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.803+0000 7fc92cf87740 1 -- 172.21.15.2:0/2233987550 shutdown_connections 2024-03-15T13:04:43.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.803+0000 7fc92cf87740 1 -- 172.21.15.2:0/2233987550 wait complete. 2024-03-15T13:04:43.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.803+0000 7fc92cf87740 20 client.17415 trim_cache size 0 max 16384 2024-03-15T13:04:43.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc92cf87740 1 Processor -- start 2024-03-15T13:04:43.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc92cf87740 1 -- start start 2024-03-15T13:04:43.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc92cf87740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3c5250 0x55f3ff3cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc92cf87740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc92cf87740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3ce750 0x55f3ff3d9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc92cf87740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ff2b2130 con 0x55f3ff3c5250 2024-03-15T13:04:43.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc92cf87740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ff3bb0a0 con 0x55f3ff3ce750 2024-03-15T13:04:43.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc92cf87740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ff3cbcc0 con 0x55f3ff3c5b70 2024-03-15T13:04:43.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc923fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3c5250 0x55f3ff3cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc923fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3c5250 0x55f3ff3cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33300/0 (socket says 172.21.15.2:33300) 2024-03-15T13:04:43.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc923fff640 1 -- 172.21.15.2:0/495267584 learned_addr learned my addr 172.21.15.2:0/495267584 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc9237fe640 1 --2- 172.21.15.2:0/495267584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc928866640 1 --2- 172.21.15.2:0/495267584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3ce750 0x55f3ff3d9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc923fff640 1 -- 172.21.15.2:0/495267584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 msgr2=0x55f3ff3ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc923fff640 1 --2- 172.21.15.2:0/495267584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc923fff640 1 -- 172.21.15.2:0/495267584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3ce750 msgr2=0x55f3ff3d9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc923fff640 1 --2- 172.21.15.2:0/495267584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3ce750 0x55f3ff3d9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc923fff640 1 -- 172.21.15.2:0/495267584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3ff2c7c20 con 0x55f3ff3c5250 2024-03-15T13:04:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.816+0000 7fc923fff640 1 --2- 172.21.15.2:0/495267584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3c5250 0x55f3ff3cdb50 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc9140a2900 tx=0x7fc91400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bfde46fa37edc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.817+0000 7fc909ffb640 1 -- 172.21.15.2:0/495267584 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91409ad90 con 0x55f3ff3c5250 2024-03-15T13:04:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.817+0000 7fc909ffb640 1 -- 172.21.15.2:0/495267584 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc91409e7e0 con 0x55f3ff3c5250 2024-03-15T13:04:43.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.817+0000 7fc909ffb640 1 -- 172.21.15.2:0/495267584 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9140850b0 con 0x55f3ff3c5250 2024-03-15T13:04:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:43 smithi002 ceph-mon[32060]: pgmap v177: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:43 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2946891282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:04:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.817+0000 7fc92cf87740 1 -- 172.21.15.2:0/495267584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3c5250 msgr2=0x55f3ff3cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.817+0000 7fc92cf87740 1 --2- 172.21.15.2:0/495267584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3c5250 0x55f3ff3cdb50 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc9140a2900 tx=0x7fc91400cf90 comp rx=0 tx=0).stop 2024-03-15T13:04:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.817+0000 7fc92cf87740 1 -- 172.21.15.2:0/495267584 shutdown_connections 2024-03-15T13:04:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.817+0000 7fc92cf87740 1 --2- 172.21.15.2:0/495267584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3ce750 0x55f3ff3d9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.817+0000 7fc92cf87740 1 --2- 172.21.15.2:0/495267584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.817+0000 7fc92cf87740 1 --2- 172.21.15.2:0/495267584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3c5250 0x55f3ff3cdb50 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.817+0000 7fc92cf87740 1 -- 172.21.15.2:0/495267584 >> 172.21.15.2:0/495267584 conn(0x55f3ff3bf980 msgr2=0x55f3ff3c6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.817+0000 7fc92cf87740 1 -- 172.21.15.2:0/495267584 shutdown_connections 2024-03-15T13:04:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.817+0000 7fc92cf87740 1 -- 172.21.15.2:0/495267584 wait complete. 2024-03-15T13:04:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.818+0000 7fc92cf87740 1 -- start start 2024-03-15T13:04:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.818+0000 7fc92cf87740 1 Processor -- start 2024-03-15T13:04:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.818+0000 7fc92cf87740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3c5250 0x55f3ff3d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.818+0000 7fc92cf87740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.818+0000 7fc92cf87740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3ce750 0x55f3ff432f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.818+0000 7fc92cf87740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ff433450 con 0x55f3ff3ce750 2024-03-15T13:04:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.818+0000 7fc92cf87740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ff4335c0 con 0x55f3ff3c5250 2024-03-15T13:04:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.818+0000 7fc92cf87740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3ff433780 con 0x55f3ff3c5b70 2024-03-15T13:04:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.818+0000 7fc92cf87740 10 client.0 _ll_register_callbacks cb 0x7fc9140056f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.818+0000 7fc928866640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3ce750 0x55f3ff432f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.818+0000 7fc928866640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3ce750 0x55f3ff432f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33316/0 (socket says 172.21.15.2:33316) 2024-03-15T13:04:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.818+0000 7fc928866640 1 -- 172.21.15.2:0/1198739322 learned_addr learned my addr 172.21.15.2:0/1198739322 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.818+0000 7fc923fff640 1 --2- 172.21.15.2:0/1198739322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3c5250 0x55f3ff3d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.818+0000 7fc9237fe640 1 --2- 172.21.15.2:0/1198739322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc928866640 1 -- 172.21.15.2:0/1198739322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 msgr2=0x55f3ff3d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc928866640 1 --2- 172.21.15.2:0/1198739322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3d2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc928866640 1 -- 172.21.15.2:0/1198739322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3c5250 msgr2=0x55f3ff3d20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc928866640 1 --2- 172.21.15.2:0/1198739322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3c5250 0x55f3ff3d20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc928866640 1 -- 172.21.15.2:0/1198739322 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3ff32b6a0 con 0x55f3ff3ce750 2024-03-15T13:04:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc923fff640 1 --2- 172.21.15.2:0/1198739322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3c5250 0x55f3ff3d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc928866640 1 --2- 172.21.15.2:0/1198739322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3ce750 0x55f3ff432f10 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc91c011910 tx=0x7fc91c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b837e92bdc933f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc9037fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc9037fe640 1 -- 172.21.15.2:0/1198739322 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c025070 con 0x55f3ff3ce750 2024-03-15T13:04:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc9037fe640 1 -- 172.21.15.2:0/1198739322 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc91c004b20 con 0x55f3ff3ce750 2024-03-15T13:04:43.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc92cf87740 1 -- 172.21.15.2:0/1198739322 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc918034910 con 0x55f3ff3ce750 2024-03-15T13:04:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc9037fe640 1 -- 172.21.15.2:0/1198739322 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc91c004ce0 con 0x55f3ff3ce750 2024-03-15T13:04:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc92cf87740 10 client.17421 fetch_fsmap learned FSMap version 45 2024-03-15T13:04:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc92cf87740 1 -- 172.21.15.2:0/1198739322 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55f3ff2ed8c0 con 0x55f3ff3ce750 2024-03-15T13:04:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc9037fe640 1 -- 172.21.15.2:0/1198739322 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc918034910 con 0x55f3ff3ce750 2024-03-15T13:04:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc9037fe640 1 -- 172.21.15.2:0/1198739322 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fc91c011d20 con 0x55f3ff3ce750 2024-03-15T13:04:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc92cf87740 10 client.17421 fetch_fsmap finished waiting for FSMap version 45 2024-03-15T13:04:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc92cf87740 10 client.17421 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:04:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc92cf87740 20 client.17421 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc92cf87740 1 --2- 172.21.15.2:0/1198739322 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f3ff43b0b0 0x55f3ff43d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc92cf87740 4 client.17421 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:04:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.819+0000 7fc92cf87740 1 -- 172.21.15.2:0/1198739322 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f3ff0b8890 con 0x55f3ff43b0b0 2024-03-15T13:04:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc923fff640 1 --2- 172.21.15.2:0/1198739322 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f3ff43b0b0 0x55f3ff43d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc923fff640 1 --2- 172.21.15.2:0/1198739322 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f3ff43b0b0 0x55f3ff43d4d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df609f34836999f1 server_cookie=76dd863437b76ad9 in_seq=0 out_seq=0 2024-03-15T13:04:43.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc9037fe640 10 client.17421 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:04:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc9037fe640 1 -- 172.21.15.2:0/1198739322 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f3ff0b8890 con 0x55f3ff43b0b0 2024-03-15T13:04:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc9037fe640 10 client.17421 handle_command_reply: tid=0 2024-03-15T13:04:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc92cf87740 1 client.17421 shutdown 2024-03-15T13:04:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc92cf87740 1 -- 172.21.15.2:0/1198739322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3ce750 msgr2=0x55f3ff432f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc92cf87740 1 --2- 172.21.15.2:0/1198739322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3ce750 0x55f3ff432f10 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc91c011910 tx=0x7fc91c004a40 comp rx=0 tx=0).stop 2024-03-15T13:04:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc92cf87740 1 -- 172.21.15.2:0/1198739322 shutdown_connections 2024-03-15T13:04:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc92cf87740 1 --2- 172.21.15.2:0/1198739322 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f3ff43b0b0 0x55f3ff43d4d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc92cf87740 1 --2- 172.21.15.2:0/1198739322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3ff3ce750 0x55f3ff432f10 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc92cf87740 1 --2- 172.21.15.2:0/1198739322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3ff3c5b70 0x55f3ff3d2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc92cf87740 1 --2- 172.21.15.2:0/1198739322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ff3c5250 0x55f3ff3d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc9037fe640 0 client.17421 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:04:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc92cf87740 1 -- 172.21.15.2:0/1198739322 >> 172.21.15.2:0/1198739322 conn(0x55f3ff3bf980 msgr2=0x55f3ff3cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc92cf87740 1 -- 172.21.15.2:0/1198739322 shutdown_connections 2024-03-15T13:04:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.820+0000 7fc92cf87740 1 -- 172.21.15.2:0/1198739322 wait complete. 2024-03-15T13:04:43.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.821+0000 7fc92cf87740 20 client.17421 trim_cache size 0 max 16384 2024-03-15T13:04:43.813 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:04:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.821+0000 7fc92a2c9640 1 -- 172.21.15.2:0/3043782662 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc90c0a1ba0 msgr2=0x7fc90c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.821+0000 7fc92a2c9640 1 --2- 172.21.15.2:0/3043782662 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc90c0a1ba0 0x7fc90c0a4060 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc91c005c40 tx=0x7fc91c009500 comp rx=0 tx=0).stop 2024-03-15T13:04:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.821+0000 7fc92a2c9640 1 -- 172.21.15.2:0/3043782662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9241014b0 msgr2=0x7fc92419d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.821+0000 7fc92a2c9640 1 --2- 172.21.15.2:0/3043782662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9241014b0 0x7fc92419d730 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc91400e9c0 tx=0x7fc91400ee80 comp rx=0 tx=0).stop 2024-03-15T13:04:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.821+0000 7fc92a2c9640 1 -- 172.21.15.2:0/3043782662 shutdown_connections 2024-03-15T13:04:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.821+0000 7fc92a2c9640 1 --2- 172.21.15.2:0/3043782662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc924107570 0x7fc9241978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.821+0000 7fc92a2c9640 1 --2- 172.21.15.2:0/3043782662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc924101e70 0x7fc92419dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.821+0000 7fc92a2c9640 1 --2- 172.21.15.2:0/3043782662 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc90c0a1ba0 0x7fc90c0a4060 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.821+0000 7fc92a2c9640 1 --2- 172.21.15.2:0/3043782662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9241014b0 0x7fc92419d730 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.821+0000 7fc92a2c9640 1 -- 172.21.15.2:0/3043782662 >> 172.21.15.2:0/3043782662 conn(0x7fc9240fcf20 msgr2=0x7fc9240fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.821+0000 7fc92a2c9640 1 -- 172.21.15.2:0/3043782662 shutdown_connections 2024-03-15T13:04:43.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.821+0000 7fc92a2c9640 1 -- 172.21.15.2:0/3043782662 wait complete. 2024-03-15T13:04:43.817 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:04:43.817 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:04:43.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.936+0000 7fc910823640 1 -- 172.21.15.2:0/2553917062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc90c104b20 msgr2=0x7fc90c104f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.936+0000 7fc910823640 1 --2- 172.21.15.2:0/2553917062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc90c104b20 0x7fc90c104f00 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009900 tx=0x7fc8fc02bc30 comp rx=0 tx=0).stop 2024-03-15T13:04:43.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.936+0000 7fc910823640 1 -- 172.21.15.2:0/2553917062 shutdown_connections 2024-03-15T13:04:43.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.936+0000 7fc910823640 1 --2- 172.21.15.2:0/2553917062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90c100750 0x7fc90c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.936+0000 7fc910823640 1 --2- 172.21.15.2:0/2553917062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc90c0ffdb0 0x7fc90c100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.936+0000 7fc910823640 1 --2- 172.21.15.2:0/2553917062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc90c104b20 0x7fc90c104f00 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.936+0000 7fc910823640 1 -- 172.21.15.2:0/2553917062 >> 172.21.15.2:0/2553917062 conn(0x7fc90c075730 msgr2=0x7fc90c075b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.936+0000 7fc910823640 1 -- 172.21.15.2:0/2553917062 shutdown_connections 2024-03-15T13:04:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.936+0000 7fc910823640 1 -- 172.21.15.2:0/2553917062 wait complete. 2024-03-15T13:04:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc910823640 1 Processor -- start 2024-03-15T13:04:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc910823640 1 -- start start 2024-03-15T13:04:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc910823640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90c0ffdb0 0x7fc90c101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc910823640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc90c100750 0x7fc90c101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc910823640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc90c104b20 0x7fc90c102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc910823640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c1a5e70 con 0x7fc90c100750 2024-03-15T13:04:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc910823640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c1a5fe0 con 0x7fc90c104b20 2024-03-15T13:04:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc910823640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c1a61a0 con 0x7fc90c0ffdb0 2024-03-15T13:04:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc903fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc90c100750 0x7fc90c101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc90a59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90c0ffdb0 0x7fc90c101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc903fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc90c100750 0x7fc90c101d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33328/0 (socket says 172.21.15.2:33328) 2024-03-15T13:04:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc903fff640 1 -- 172.21.15.2:0/2534554928 learned_addr learned my addr 172.21.15.2:0/2534554928 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc90ad9d640 1 --2- 172.21.15.2:0/2534554928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc90c104b20 0x7fc90c102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc903fff640 1 -- 172.21.15.2:0/2534554928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90c0ffdb0 msgr2=0x7fc90c101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc903fff640 1 --2- 172.21.15.2:0/2534554928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90c0ffdb0 0x7fc90c101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc903fff640 1 -- 172.21.15.2:0/2534554928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc90c104b20 msgr2=0x7fc90c102290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc903fff640 1 --2- 172.21.15.2:0/2534554928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc90c104b20 0x7fc90c102290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.937+0000 7fc903fff640 1 -- 172.21.15.2:0/2534554928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc009560 con 0x7fc90c100750 2024-03-15T13:04:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.938+0000 7fc903fff640 1 --2- 172.21.15.2:0/2534554928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc90c100750 0x7fc90c101d50 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400e9f0 tx=0x7fc8f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.938+0000 7fc9037fe640 1 -- 172.21.15.2:0/2534554928 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4009700 con 0x7fc90c100750 2024-03-15T13:04:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.938+0000 7fc910823640 1 -- 172.21.15.2:0/2534554928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c1a6480 con 0x7fc90c100750 2024-03-15T13:04:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.938+0000 7fc9037fe640 1 -- 172.21.15.2:0/2534554928 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc8f400cd80 con 0x7fc90c100750 2024-03-15T13:04:43.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.938+0000 7fc9037fe640 1 -- 172.21.15.2:0/2534554928 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4018920 con 0x7fc90c100750 2024-03-15T13:04:43.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.938+0000 7fc910823640 1 -- 172.21.15.2:0/2534554928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c1a69e0 con 0x7fc90c100750 2024-03-15T13:04:43.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.940+0000 7fc9037fe640 1 -- 172.21.15.2:0/2534554928 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fc8f4018a80 con 0x7fc90c100750 2024-03-15T13:04:43.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc9133ff740 1 Processor -- start 2024-03-15T13:04:43.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc9133ff740 1 -- start start 2024-03-15T13:04:43.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc9133ff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7c250 0x55ee27c7c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc9133ff740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c7cb70 0x55ee27c85210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc9133ff740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c85750 0x55ee27c87b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc9133ff740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee27c37960 con 0x55ee27c85750 2024-03-15T13:04:43.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc9133ff740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee27beb310 con 0x55ee27c7cb70 2024-03-15T13:04:43.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc9133ff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee27be8840 con 0x55ee27c7c250 2024-03-15T13:04:43.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc9037fe640 1 --2- 172.21.15.2:0/2534554928 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc8f80a1ba0 0x7fc8f80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc903fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c85750 0x55ee27c87b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc90a59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c7cb70 0x55ee27c85210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc90ad9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7c250 0x55ee27c7c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc90a59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c7cb70 0x55ee27c85210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48666/0 (socket says 172.21.15.2:48666) 2024-03-15T13:04:43.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc90a59c640 1 -- 172.21.15.2:0/3759406375 learned_addr learned my addr 172.21.15.2:0/3759406375 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc903fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c85750 0x55ee27c87b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33332/0 (socket says 172.21.15.2:33332) 2024-03-15T13:04:43.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc90ad9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7c250 0x55ee27c7c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:34374/0 (socket says 172.21.15.2:34374) 2024-03-15T13:04:43.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc90ad9d640 1 --2- 172.21.15.2:0/2534554928 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc8f80a1ba0 0x7fc8f80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.941+0000 7fc9037fe640 1 -- 172.21.15.2:0/2534554928 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc8f4014070 con 0x7fc90c100750 2024-03-15T13:04:43.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.942+0000 7fc90a59c640 1 -- 172.21.15.2:0/3759406375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7c250 msgr2=0x55ee27c7c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.942+0000 7fc90a59c640 1 --2- 172.21.15.2:0/3759406375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7c250 0x55ee27c7c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.942+0000 7fc90a59c640 1 -- 172.21.15.2:0/3759406375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c85750 msgr2=0x55ee27c87b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.942+0000 7fc90a59c640 1 --2- 172.21.15.2:0/3759406375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c85750 0x55ee27c87b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.942+0000 7fc90a59c640 1 -- 172.21.15.2:0/3759406375 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee27ba6060 con 0x55ee27c7cb70 2024-03-15T13:04:43.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.942+0000 7fc903fff640 1 --2- 172.21.15.2:0/3759406375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c85750 0x55ee27c87b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:43.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.942+0000 7fc90ad9d640 1 --2- 172.21.15.2:0/2534554928 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc8f80a1ba0 0x7fc8f80a4060 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc90400aa60 tx=0x7fc90400a610 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.942+0000 7fc90a59c640 1 --2- 172.21.15.2:0/3759406375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c7cb70 0x55ee27c85210 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc036040 tx=0x7fc8fc02c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1967fcc38e907e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.942+0000 7fc900ff9640 1 -- 172.21.15.2:0/3759406375 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc004780 con 0x55ee27c7cb70 2024-03-15T13:04:43.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.942+0000 7fc900ff9640 1 -- 172.21.15.2:0/3759406375 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc8fc0048e0 con 0x55ee27c7cb70 2024-03-15T13:04:43.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.942+0000 7fc900ff9640 1 -- 172.21.15.2:0/3759406375 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc0037f0 con 0x55ee27c7cb70 2024-03-15T13:04:43.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.943+0000 7fc9133ff740 1 -- 172.21.15.2:0/3759406375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c7cb70 msgr2=0x55ee27c85210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.943+0000 7fc9133ff740 1 --2- 172.21.15.2:0/3759406375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c7cb70 0x55ee27c85210 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc036040 tx=0x7fc8fc02c6d0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.943+0000 7fc9133ff740 1 -- 172.21.15.2:0/3759406375 shutdown_connections 2024-03-15T13:04:43.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.943+0000 7fc9133ff740 1 --2- 172.21.15.2:0/3759406375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c85750 0x55ee27c87b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.943+0000 7fc9133ff740 1 --2- 172.21.15.2:0/3759406375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c7cb70 0x55ee27c85210 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.943+0000 7fc9133ff740 1 --2- 172.21.15.2:0/3759406375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7c250 0x55ee27c7c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.943+0000 7fc9133ff740 1 -- 172.21.15.2:0/3759406375 >> 172.21.15.2:0/3759406375 conn(0x55ee27c76980 msgr2=0x55ee27c76d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.943+0000 7fc9133ff740 1 -- 172.21.15.2:0/3759406375 shutdown_connections 2024-03-15T13:04:43.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.943+0000 7fc9133ff740 1 -- 172.21.15.2:0/3759406375 wait complete. 2024-03-15T13:04:43.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.943+0000 7fc9133ff740 1 -- start start 2024-03-15T13:04:43.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.943+0000 7fc9133ff740 1 Processor -- start 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc9133ff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7c250 0x55ee27c85080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc9133ff740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7cb70 0x55ee27c82410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc9133ff740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c85750 0x55ee27c82950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc9133ff740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee27be4020 con 0x55ee27c7cb70 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc9133ff740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee278b7430 con 0x55ee27c85750 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc9133ff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee27873e00 con 0x55ee27c7c250 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc9133ff740 10 client.0 _ll_register_callbacks cb 0x55ee27b8bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc903fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7cb70 0x55ee27c82410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc903fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7cb70 0x55ee27c82410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33338/0 (socket says 172.21.15.2:33338) 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc90a59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7c250 0x55ee27c85080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc903fff640 1 -- 172.21.15.2:0/4197685545 learned_addr learned my addr 172.21.15.2:0/4197685545 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc90ad9d640 1 --2- 172.21.15.2:0/4197685545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c85750 0x55ee27c82950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc903fff640 1 -- 172.21.15.2:0/4197685545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7c250 msgr2=0x55ee27c85080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc903fff640 1 --2- 172.21.15.2:0/4197685545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7c250 0x55ee27c85080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc903fff640 1 -- 172.21.15.2:0/4197685545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c85750 msgr2=0x55ee27c82950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc903fff640 1 --2- 172.21.15.2:0/4197685545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c85750 0x55ee27c82950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc903fff640 1 -- 172.21.15.2:0/4197685545 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c10d320 con 0x55ee27c7cb70 2024-03-15T13:04:43.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc903fff640 1 --2- 172.21.15.2:0/4197685545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7cb70 0x55ee27c82410 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40971d0 tx=0x7fc8f406a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1800c64555af3add server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc8f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:43.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc8f1ffb640 1 -- 172.21.15.2:0/4197685545 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40a2ef0 con 0x55ee27c7cb70 2024-03-15T13:04:43.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc8f1ffb640 1 -- 172.21.15.2:0/4197685545 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc8f40b6150 con 0x55ee27c7cb70 2024-03-15T13:04:43.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc9133ff740 1 -- 172.21.15.2:0/4197685545 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee278e37e0 con 0x55ee27c7cb70 2024-03-15T13:04:43.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc8f1ffb640 1 -- 172.21.15.2:0/4197685545 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8f40b6330 con 0x55ee27c7cb70 2024-03-15T13:04:43.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc9133ff740 10 client.17433 fetch_fsmap learned FSMap version 45 2024-03-15T13:04:43.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.944+0000 7fc9133ff740 1 -- 172.21.15.2:0/4197685545 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55ee27be26a0 con 0x55ee27c7cb70 2024-03-15T13:04:43.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.945+0000 7fc8f1ffb640 1 -- 172.21.15.2:0/4197685545 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55ee278e37e0 con 0x55ee27c7cb70 2024-03-15T13:04:43.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.945+0000 7fc8f1ffb640 1 -- 172.21.15.2:0/4197685545 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fc8f40ba0b0 con 0x55ee27c7cb70 2024-03-15T13:04:43.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.945+0000 7fc9133ff740 10 client.17433 fetch_fsmap finished waiting for FSMap version 45 2024-03-15T13:04:43.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.945+0000 7fc9133ff740 10 client.17433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:43.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.945+0000 7fc9133ff740 20 client.17433 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:43.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.945+0000 7fc9133ff740 1 --2- 172.21.15.2:0/4197685545 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ee27c957a0 0x55ee27c97bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.945+0000 7fc9133ff740 4 client.17433 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:43.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.945+0000 7fc9133ff740 1 -- 172.21.15.2:0/4197685545 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee27920720 con 0x55ee27c957a0 2024-03-15T13:04:43.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.945+0000 7fc90a59c640 1 --2- 172.21.15.2:0/4197685545 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ee27c957a0 0x55ee27c97bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.945+0000 7fc8f1ffb640 10 client.17433 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:43.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.945+0000 7fc90a59c640 1 --2- 172.21.15.2:0/4197685545 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ee27c957a0 0x55ee27c97bc0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ac8dae4e645ca0 server_cookie=169c5cc80cad82a8 in_seq=0 out_seq=0 2024-03-15T13:04:43.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.946+0000 7fc8f1ffb640 1 -- 172.21.15.2:0/4197685545 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ee27920720 con 0x55ee27c957a0 2024-03-15T13:04:43.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.946+0000 7fc8f1ffb640 10 client.17433 handle_command_reply: tid=0 2024-03-15T13:04:43.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.946+0000 7fc9133ff740 1 client.17433 shutdown 2024-03-15T13:04:43.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.946+0000 7fc9133ff740 1 -- 172.21.15.2:0/4197685545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7cb70 msgr2=0x55ee27c82410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.946+0000 7fc9133ff740 1 --2- 172.21.15.2:0/4197685545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7cb70 0x55ee27c82410 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40971d0 tx=0x7fc8f406a960 comp rx=0 tx=0).stop 2024-03-15T13:04:43.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.946+0000 7fc9133ff740 1 -- 172.21.15.2:0/4197685545 shutdown_connections 2024-03-15T13:04:43.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.946+0000 7fc9133ff740 1 --2- 172.21.15.2:0/4197685545 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ee27c957a0 0x55ee27c97bc0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.946+0000 7fc9133ff740 1 --2- 172.21.15.2:0/4197685545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c85750 0x55ee27c82950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.946+0000 7fc9133ff740 1 --2- 172.21.15.2:0/4197685545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7cb70 0x55ee27c82410 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.946+0000 7fc8f1ffb640 0 client.17433 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.946+0000 7fc9133ff740 1 --2- 172.21.15.2:0/4197685545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7c250 0x55ee27c85080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.946+0000 7fc9133ff740 1 -- 172.21.15.2:0/4197685545 >> 172.21.15.2:0/4197685545 conn(0x55ee27c76980 msgr2=0x55ee27c786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.946+0000 7fc9133ff740 1 -- 172.21.15.2:0/4197685545 shutdown_connections 2024-03-15T13:04:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.946+0000 7fc9133ff740 1 -- 172.21.15.2:0/4197685545 wait complete. 2024-03-15T13:04:43.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.946+0000 7fc9133ff740 20 client.17433 trim_cache size 0 max 16384 2024-03-15T13:04:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc9133ff740 1 Processor -- start 2024-03-15T13:04:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc9133ff740 1 -- start start 2024-03-15T13:04:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc9133ff740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7c250 0x55ee27c84b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc9133ff740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c7cb70 0x55ee27c85080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc9133ff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c85750 0x55ee27c90a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc9133ff740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee27b69130 con 0x55ee27c7c250 2024-03-15T13:04:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc9133ff740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee27c720b0 con 0x55ee27c7cb70 2024-03-15T13:04:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc9133ff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee27c77980 con 0x55ee27c85750 2024-03-15T13:04:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc90a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7c250 0x55ee27c84b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc90a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7c250 0x55ee27c84b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33352/0 (socket says 172.21.15.2:33352) 2024-03-15T13:04:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc90a59c640 1 -- 172.21.15.2:0/888889424 learned_addr learned my addr 172.21.15.2:0/888889424 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc90ad9d640 1 --2- 172.21.15.2:0/888889424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c85750 0x55ee27c90a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc903fff640 1 --2- 172.21.15.2:0/888889424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c7cb70 0x55ee27c85080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc90a59c640 1 -- 172.21.15.2:0/888889424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c85750 msgr2=0x55ee27c90a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc90a59c640 1 --2- 172.21.15.2:0/888889424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c85750 0x55ee27c90a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc90a59c640 1 -- 172.21.15.2:0/888889424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c7cb70 msgr2=0x55ee27c85080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc90a59c640 1 --2- 172.21.15.2:0/888889424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c7cb70 0x55ee27c85080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc90a59c640 1 -- 172.21.15.2:0/888889424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee27b7ec20 con 0x55ee27c7c250 2024-03-15T13:04:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc90a59c640 1 --2- 172.21.15.2:0/888889424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7c250 0x55ee27c84b40 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009900 tx=0x7fc8fc042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499b9feae9d2cd08 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc8f3fff640 1 -- 172.21.15.2:0/888889424 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc02ca90 con 0x55ee27c7c250 2024-03-15T13:04:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc8f3fff640 1 -- 172.21.15.2:0/888889424 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc8fc0522c0 con 0x55ee27c7c250 2024-03-15T13:04:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.959+0000 7fc8f3fff640 1 -- 172.21.15.2:0/888889424 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc052590 con 0x55ee27c7c250 2024-03-15T13:04:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.960+0000 7fc9133ff740 1 -- 172.21.15.2:0/888889424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7c250 msgr2=0x55ee27c84b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.960+0000 7fc9133ff740 1 --2- 172.21.15.2:0/888889424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7c250 0x55ee27c84b40 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009900 tx=0x7fc8fc042040 comp rx=0 tx=0).stop 2024-03-15T13:04:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.960+0000 7fc9133ff740 1 -- 172.21.15.2:0/888889424 shutdown_connections 2024-03-15T13:04:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.960+0000 7fc9133ff740 1 --2- 172.21.15.2:0/888889424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c85750 0x55ee27c90a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.960+0000 7fc9133ff740 1 --2- 172.21.15.2:0/888889424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c7cb70 0x55ee27c85080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.960+0000 7fc9133ff740 1 --2- 172.21.15.2:0/888889424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7c250 0x55ee27c84b40 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.960+0000 7fc9133ff740 1 -- 172.21.15.2:0/888889424 >> 172.21.15.2:0/888889424 conn(0x55ee27c76980 msgr2=0x55ee27c7d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.960+0000 7fc9133ff740 1 -- 172.21.15.2:0/888889424 shutdown_connections 2024-03-15T13:04:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.960+0000 7fc9133ff740 1 -- 172.21.15.2:0/888889424 wait complete. 2024-03-15T13:04:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.960+0000 7fc9133ff740 1 -- start start 2024-03-15T13:04:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.960+0000 7fc9133ff740 1 Processor -- start 2024-03-15T13:04:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc9133ff740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7c250 0x55ee27c8d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc9133ff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7cb70 0x55ee27c894d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc9133ff740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c85750 0x55ee27c89a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc9133ff740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee27c89f50 con 0x55ee27c7c250 2024-03-15T13:04:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc9133ff740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee27c8a090 con 0x55ee27c85750 2024-03-15T13:04:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc9133ff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee27c8a250 con 0x55ee27c7cb70 2024-03-15T13:04:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc9133ff740 10 client.0 _ll_register_callbacks cb 0x7fc8f409a660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc90a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7c250 0x55ee27c8d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc90a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7c250 0x55ee27c8d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33360/0 (socket says 172.21.15.2:33360) 2024-03-15T13:04:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc90a59c640 1 -- 172.21.15.2:0/3997910572 learned_addr learned my addr 172.21.15.2:0/3997910572 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc90ad9d640 1 --2- 172.21.15.2:0/3997910572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c85750 0x55ee27c89a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc903fff640 1 --2- 172.21.15.2:0/3997910572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7cb70 0x55ee27c894d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc90a59c640 1 -- 172.21.15.2:0/3997910572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7cb70 msgr2=0x55ee27c894d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc90a59c640 1 --2- 172.21.15.2:0/3997910572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7cb70 0x55ee27c894d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc90a59c640 1 -- 172.21.15.2:0/3997910572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c85750 msgr2=0x55ee27c89a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc90a59c640 1 --2- 172.21.15.2:0/3997910572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c85750 0x55ee27c89a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc90a59c640 1 -- 172.21.15.2:0/3997910572 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee27ba6060 con 0x55ee27c7c250 2024-03-15T13:04:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc90a59c640 1 --2- 172.21.15.2:0/3997910572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7c250 0x55ee27c8d150 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc000c00 tx=0x7fc8fc02c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4370bf3d3074ca68 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc8f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc8f1ffb640 1 -- 172.21.15.2:0/3997910572 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc02cca0 con 0x55ee27c7c250 2024-03-15T13:04:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc9133ff740 1 -- 172.21.15.2:0/3997910572 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8fc0048e0 con 0x55ee27c7c250 2024-03-15T13:04:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.961+0000 7fc8f1ffb640 1 -- 172.21.15.2:0/3997910572 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc8fc04f070 con 0x55ee27c7c250 2024-03-15T13:04:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.962+0000 7fc8f1ffb640 1 -- 172.21.15.2:0/3997910572 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8fc005650 con 0x55ee27c7c250 2024-03-15T13:04:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.962+0000 7fc9133ff740 10 client.17439 fetch_fsmap learned FSMap version 45 2024-03-15T13:04:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.962+0000 7fc9133ff740 1 -- 172.21.15.2:0/3997910572 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55ee27ba48c0 con 0x55ee27c7c250 2024-03-15T13:04:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.962+0000 7fc8f1ffb640 1 -- 172.21.15.2:0/3997910572 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc8fc0048e0 con 0x55ee27c7c250 2024-03-15T13:04:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.962+0000 7fc8f1ffb640 1 -- 172.21.15.2:0/3997910572 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) v1 ==== 7962+0+0 (secure 0 0 0) 0x7fc8fc03c050 con 0x55ee27c7c250 2024-03-15T13:04:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.962+0000 7fc9133ff740 10 client.17439 fetch_fsmap finished waiting for FSMap version 45 2024-03-15T13:04:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.962+0000 7fc9133ff740 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.962+0000 7fc9133ff740 20 client.17439 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.962+0000 7fc9133ff740 1 --2- 172.21.15.2:0/3997910572 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ee27cf2250 0x55ee27cf4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.962+0000 7fc9133ff740 4 client.17439 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:04:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.962+0000 7fc9133ff740 1 -- 172.21.15.2:0/3997910572 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ee2796f890 con 0x55ee27cf2250 2024-03-15T13:04:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.962+0000 7fc903fff640 1 --2- 172.21.15.2:0/3997910572 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ee27cf2250 0x55ee27cf4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc8f1ffb640 10 client.17439 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc903fff640 1 --2- 172.21.15.2:0/3997910572 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ee27cf2250 0x55ee27cf4670 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=355adddcebccf2e9 server_cookie=4d4a939d811999e8 in_seq=0 out_seq=0 2024-03-15T13:04:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc8f1ffb640 1 -- 172.21.15.2:0/3997910572 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ee2796f890 con 0x55ee27cf2250 2024-03-15T13:04:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc8f1ffb640 10 client.17439 handle_command_reply: tid=0 2024-03-15T13:04:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc9133ff740 1 client.17439 shutdown 2024-03-15T13:04:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc9133ff740 1 -- 172.21.15.2:0/3997910572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7c250 msgr2=0x55ee27c8d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc9133ff740 1 --2- 172.21.15.2:0/3997910572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7c250 0x55ee27c8d150 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc000c00 tx=0x7fc8fc02c330 comp rx=0 tx=0).stop 2024-03-15T13:04:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc9133ff740 1 -- 172.21.15.2:0/3997910572 shutdown_connections 2024-03-15T13:04:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc9133ff740 1 --2- 172.21.15.2:0/3997910572 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ee27cf2250 0x55ee27cf4670 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc9133ff740 1 --2- 172.21.15.2:0/3997910572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee27c85750 0x55ee27c89a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc9133ff740 1 --2- 172.21.15.2:0/3997910572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ee27c7cb70 0x55ee27c894d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc8f1ffb640 0 client.17439 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc9133ff740 1 --2- 172.21.15.2:0/3997910572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee27c7c250 0x55ee27c8d150 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc9133ff740 1 -- 172.21.15.2:0/3997910572 >> 172.21.15.2:0/3997910572 conn(0x55ee27c76980 msgr2=0x55ee27c81ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc9133ff740 1 -- 172.21.15.2:0/3997910572 shutdown_connections 2024-03-15T13:04:43.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc9133ff740 1 -- 172.21.15.2:0/3997910572 wait complete. 2024-03-15T13:04:43.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc9133ff740 20 client.17439 trim_cache size 0 max 16384 2024-03-15T13:04:43.957 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:04:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc910823640 1 -- 172.21.15.2:0/2534554928 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc8f80a1ba0 msgr2=0x7fc8f80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc910823640 1 --2- 172.21.15.2:0/2534554928 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc8f80a1ba0 0x7fc8f80a4060 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc90400aa60 tx=0x7fc90400a610 comp rx=0 tx=0).stop 2024-03-15T13:04:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc910823640 1 -- 172.21.15.2:0/2534554928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc90c100750 msgr2=0x7fc90c101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc910823640 1 --2- 172.21.15.2:0/2534554928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc90c100750 0x7fc90c101d50 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400e9f0 tx=0x7fc8f400eeb0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.963+0000 7fc910823640 1 -- 172.21.15.2:0/2534554928 shutdown_connections 2024-03-15T13:04:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.964+0000 7fc910823640 1 --2- 172.21.15.2:0/2534554928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc90c104b20 0x7fc90c102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.964+0000 7fc910823640 1 --2- 172.21.15.2:0/2534554928 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc8f80a1ba0 0x7fc8f80a4060 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.964+0000 7fc910823640 1 --2- 172.21.15.2:0/2534554928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc90c100750 0x7fc90c101d50 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.964+0000 7fc910823640 1 --2- 172.21.15.2:0/2534554928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90c0ffdb0 0x7fc90c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.964+0000 7fc910823640 1 -- 172.21.15.2:0/2534554928 >> 172.21.15.2:0/2534554928 conn(0x7fc90c075730 msgr2=0x7fc90c103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.964+0000 7fc910823640 1 -- 172.21.15.2:0/2534554928 shutdown_connections 2024-03-15T13:04:43.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:43.964+0000 7fc910823640 1 -- 172.21.15.2:0/2534554928 wait complete. 2024-03-15T13:04:44.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:43 smithi184 ceph-mon[35993]: pgmap v177: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:44.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:43 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2946891282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:04:44.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:43 smithi149 ceph-mon[35927]: pgmap v177: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:44.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:43 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2946891282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:04:44.960 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:04:44.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:04:44.961 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:04:45.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.094+0000 7fc3ab0da640 1 -- 172.21.15.2:0/4239411022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3a41056e0 msgr2=0x7fc3a410d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.094+0000 7fc3ab0da640 1 --2- 172.21.15.2:0/4239411022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3a41056e0 0x7fc3a410d2b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009cd0 tx=0x7fc39c02c0f0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.094+0000 7fc3ab0da640 1 -- 172.21.15.2:0/4239411022 shutdown_connections 2024-03-15T13:04:45.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.094+0000 7fc3ab0da640 1 --2- 172.21.15.2:0/4239411022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3a41056e0 0x7fc3a410d2b0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.094+0000 7fc3ab0da640 1 --2- 172.21.15.2:0/4239411022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3a4100fc0 0x7fc3a4105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.094+0000 7fc3ab0da640 1 --2- 172.21.15.2:0/4239411022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3a4100610 0x7fc3a41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.094+0000 7fc3ab0da640 1 -- 172.21.15.2:0/4239411022 >> 172.21.15.2:0/4239411022 conn(0x7fc3a4076770 msgr2=0x7fc3a4076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:45.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.094+0000 7fc3ab0da640 1 -- 172.21.15.2:0/4239411022 shutdown_connections 2024-03-15T13:04:45.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.095+0000 7fc3ab0da640 1 -- 172.21.15.2:0/4239411022 wait complete. 2024-03-15T13:04:45.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.095+0000 7fc3ab0da640 1 Processor -- start 2024-03-15T13:04:45.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.095+0000 7fc3ab0da640 1 -- start start 2024-03-15T13:04:45.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.095+0000 7fc3ab0da640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3a4100610 0x7fc3a4199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.095+0000 7fc3ab0da640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3a41056e0 0x7fc3a419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.095+0000 7fc3ab0da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3a419e600 0x7fc3a419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.095+0000 7fc3ab0da640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a419eff0 con 0x7fc3a41056e0 2024-03-15T13:04:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.095+0000 7fc3ab0da640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a419f160 con 0x7fc3a4100610 2024-03-15T13:04:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.095+0000 7fc3ab0da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a419f2d0 con 0x7fc3a419e600 2024-03-15T13:04:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.095+0000 7fc3a3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3a41056e0 0x7fc3a419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.095+0000 7fc3a8e76640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3a4100610 0x7fc3a4199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.095+0000 7fc3a3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3a41056e0 0x7fc3a419a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33390/0 (socket says 172.21.15.2:33390) 2024-03-15T13:04:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.095+0000 7fc3a3fff640 1 -- 172.21.15.2:0/4045486727 learned_addr learned my addr 172.21.15.2:0/4045486727 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.096+0000 7fc3a9677640 1 --2- 172.21.15.2:0/4045486727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3a419e600 0x7fc3a419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.096+0000 7fc3a3fff640 1 -- 172.21.15.2:0/4045486727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3a419e600 msgr2=0x7fc3a419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.096+0000 7fc3a3fff640 1 --2- 172.21.15.2:0/4045486727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3a419e600 0x7fc3a419eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.096+0000 7fc3a3fff640 1 -- 172.21.15.2:0/4045486727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3a4100610 msgr2=0x7fc3a4199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.096+0000 7fc3a3fff640 1 --2- 172.21.15.2:0/4045486727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3a4100610 0x7fc3a4199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.096+0000 7fc3a3fff640 1 -- 172.21.15.2:0/4045486727 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c009900 con 0x7fc3a41056e0 2024-03-15T13:04:45.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.096+0000 7fc3a3fff640 1 --2- 172.21.15.2:0/4045486727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3a41056e0 0x7fc3a419a280 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc39000e9c0 tx=0x7fc39000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:45.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.096+0000 7fc3a1ffb640 1 -- 172.21.15.2:0/4045486727 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc390009700 con 0x7fc3a41056e0 2024-03-15T13:04:45.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.096+0000 7fc3a1ffb640 1 -- 172.21.15.2:0/4045486727 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc39000cd80 con 0x7fc3a41056e0 2024-03-15T13:04:45.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.096+0000 7fc3ab0da640 1 -- 172.21.15.2:0/4045486727 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a41a68b0 con 0x7fc3a41056e0 2024-03-15T13:04:45.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.096+0000 7fc3a1ffb640 1 -- 172.21.15.2:0/4045486727 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc390010430 con 0x7fc3a41056e0 2024-03-15T13:04:45.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.096+0000 7fc3ab0da640 1 -- 172.21.15.2:0/4045486727 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a41a6d60 con 0x7fc3a41056e0 2024-03-15T13:04:45.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.098+0000 7fc3a1ffb640 1 -- 172.21.15.2:0/4045486727 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fc390004ff0 con 0x7fc3a41056e0 2024-03-15T13:04:45.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.099+0000 7fc3add8a740 1 Processor -- start 2024-03-15T13:04:45.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.099+0000 7fc3a1ffb640 1 --2- 172.21.15.2:0/4045486727 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc3940a1ba0 0x7fc3940a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.099+0000 7fc3add8a740 1 -- start start 2024-03-15T13:04:45.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.099+0000 7fc3a9677640 1 --2- 172.21.15.2:0/4045486727 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc3940a1ba0 0x7fc3940a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3a1ffb640 1 -- 172.21.15.2:0/4045486727 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc390014070 con 0x7fc3a41056e0 2024-03-15T13:04:45.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3add8a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae85504380 0x55ae85504760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3add8a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504d30 0x55ae8550d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3add8a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae8550d8b0 0x55ae8550fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3add8a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae85472f90 con 0x55ae85504380 2024-03-15T13:04:45.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3add8a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae854704c0 con 0x55ae8550d8b0 2024-03-15T13:04:45.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3add8a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8546bca0 con 0x55ae85504d30 2024-03-15T13:04:45.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3a8e76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae85504380 0x55ae85504760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3a8e76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae85504380 0x55ae85504760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33402/0 (socket says 172.21.15.2:33402) 2024-03-15T13:04:45.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3a8e76640 1 -- 172.21.15.2:0/1962766574 learned_addr learned my addr 172.21.15.2:0/1962766574 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:45.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3a3fff640 1 --2- 172.21.15.2:0/1962766574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504d30 0x55ae8550d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3a9677640 1 --2- 172.21.15.2:0/1962766574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae8550d8b0 0x55ae8550fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3a3fff640 1 -- 172.21.15.2:0/1962766574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae8550d8b0 msgr2=0x55ae8550fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3a9677640 1 --2- 172.21.15.2:0/4045486727 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc3940a1ba0 0x7fc3940a4060 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc39c005cc0 tx=0x7fc39c032ea0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:45.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3a3fff640 1 --2- 172.21.15.2:0/1962766574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae8550d8b0 0x55ae8550fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3a3fff640 1 -- 172.21.15.2:0/1962766574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae85504380 msgr2=0x55ae85504760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3a3fff640 1 --2- 172.21.15.2:0/1962766574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae85504380 0x55ae85504760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3a3fff640 1 -- 172.21.15.2:0/1962766574 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae8542e0b0 con 0x55ae85504d30 2024-03-15T13:04:45.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3a8e76640 1 --2- 172.21.15.2:0/1962766574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae85504380 0x55ae85504760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:45.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc3a3fff640 1 --2- 172.21.15.2:0/1962766574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504d30 0x55ae8550d2e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc3900cb280 tx=0x7fc390096b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f36b31543758f73 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:45.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.100+0000 7fc38effd640 1 -- 172.21.15.2:0/1962766574 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39009e8b0 con 0x55ae85504d30 2024-03-15T13:04:45.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.101+0000 7fc38effd640 1 -- 172.21.15.2:0/1962766574 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc3900a2f00 con 0x55ae85504d30 2024-03-15T13:04:45.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.101+0000 7fc38effd640 1 -- 172.21.15.2:0/1962766574 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3900cb690 con 0x55ae85504d30 2024-03-15T13:04:45.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.101+0000 7fc3add8a740 1 -- 172.21.15.2:0/1962766574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504d30 msgr2=0x55ae8550d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.101+0000 7fc3add8a740 1 --2- 172.21.15.2:0/1962766574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504d30 0x55ae8550d2e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc3900cb280 tx=0x7fc390096b30 comp rx=0 tx=0).stop 2024-03-15T13:04:45.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.101+0000 7fc3add8a740 1 -- 172.21.15.2:0/1962766574 shutdown_connections 2024-03-15T13:04:45.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.101+0000 7fc3add8a740 1 --2- 172.21.15.2:0/1962766574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae8550d8b0 0x55ae8550fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.101+0000 7fc3add8a740 1 --2- 172.21.15.2:0/1962766574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504d30 0x55ae8550d2e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.101+0000 7fc3add8a740 1 --2- 172.21.15.2:0/1962766574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae85504380 0x55ae85504760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.101+0000 7fc3add8a740 1 -- 172.21.15.2:0/1962766574 >> 172.21.15.2:0/1962766574 conn(0x55ae854fe840 msgr2=0x55ae854fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:45.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.101+0000 7fc3add8a740 1 -- 172.21.15.2:0/1962766574 shutdown_connections 2024-03-15T13:04:45.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.101+0000 7fc3add8a740 1 -- 172.21.15.2:0/1962766574 wait complete. 2024-03-15T13:04:45.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.102+0000 7fc3add8a740 1 -- start start 2024-03-15T13:04:45.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.102+0000 7fc3add8a740 1 Processor -- start 2024-03-15T13:04:45.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.102+0000 7fc3add8a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504380 0x55ae8550a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.102+0000 7fc3add8a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504d30 0x55ae8550ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.102+0000 7fc3add8a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae8550d8b0 0x55ae8550b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.102+0000 7fc3add8a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae85410f40 con 0x55ae8550d8b0 2024-03-15T13:04:45.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.102+0000 7fc3add8a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8513f4b0 con 0x55ae85504d30 2024-03-15T13:04:45.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.102+0000 7fc3add8a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae853f1670 con 0x55ae85504380 2024-03-15T13:04:45.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.102+0000 7fc3add8a740 10 client.0 _ll_register_callbacks cb 0x55ae8545ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:45.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.102+0000 7fc3a3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504d30 0x55ae8550ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.102+0000 7fc3a9677640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae8550d8b0 0x55ae8550b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.102+0000 7fc3a3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504d30 0x55ae8550ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48718/0 (socket says 172.21.15.2:48718) 2024-03-15T13:04:45.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.102+0000 7fc3a3fff640 1 -- 172.21.15.2:0/3123490047 learned_addr learned my addr 172.21.15.2:0/3123490047 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:45.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.102+0000 7fc3a8e76640 1 --2- 172.21.15.2:0/3123490047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504380 0x55ae8550a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc3a9677640 1 -- 172.21.15.2:0/3123490047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504380 msgr2=0x55ae8550a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc3a9677640 1 --2- 172.21.15.2:0/3123490047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504380 0x55ae8550a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc3a9677640 1 -- 172.21.15.2:0/3123490047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504d30 msgr2=0x55ae8550ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc3a9677640 1 --2- 172.21.15.2:0/3123490047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504d30 0x55ae8550ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc3a9677640 1 -- 172.21.15.2:0/3123490047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a41a6d60 con 0x55ae8550d8b0 2024-03-15T13:04:45.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc3a8e76640 1 --2- 172.21.15.2:0/3123490047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504380 0x55ae8550a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:45.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc3a3fff640 1 --2- 172.21.15.2:0/3123490047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504d30 0x55ae8550ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:45.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc3a9677640 1 --2- 172.21.15.2:0/3123490047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae8550d8b0 0x55ae8550b350 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc39c037040 tx=0x7fc39c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3184cc0ec4cf5f6e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:45.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc383fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:45.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc383fff640 1 -- 172.21.15.2:0/3123490047 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c032a90 con 0x55ae8550d8b0 2024-03-15T13:04:45.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc383fff640 1 -- 172.21.15.2:0/3123490047 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc39c003970 con 0x55ae8550d8b0 2024-03-15T13:04:45.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc3add8a740 1 -- 172.21.15.2:0/3123490047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae85445f40 con 0x55ae8550d8b0 2024-03-15T13:04:45.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc383fff640 1 -- 172.21.15.2:0/3123490047 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc39c003b30 con 0x55ae8550d8b0 2024-03-15T13:04:45.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc3add8a740 10 client.17451 fetch_fsmap learned FSMap version 46 2024-03-15T13:04:45.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc3add8a740 1 -- 172.21.15.2:0/3123490047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55ae8546a320 con 0x55ae8550d8b0 2024-03-15T13:04:45.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.103+0000 7fc383fff640 1 -- 172.21.15.2:0/3123490047 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55ae85445f40 con 0x55ae8550d8b0 2024-03-15T13:04:45.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.104+0000 7fc383fff640 1 -- 172.21.15.2:0/3123490047 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 46) v1 ==== 7958+0+0 (secure 0 0 0) 0x7fc39c041440 con 0x55ae8550d8b0 2024-03-15T13:04:45.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.104+0000 7fc3add8a740 10 client.17451 fetch_fsmap finished waiting for FSMap version 46 2024-03-15T13:04:45.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.104+0000 7fc3add8a740 10 client.17451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:45.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.104+0000 7fc3add8a740 20 client.17451 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:45.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.104+0000 7fc3add8a740 1 --2- 172.21.15.2:0/3123490047 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ae8551deb0 0x55ae855202a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.104+0000 7fc3add8a740 4 client.17451 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:45.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.104+0000 7fc3add8a740 1 -- 172.21.15.2:0/3123490047 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae85462310 con 0x55ae8551deb0 2024-03-15T13:04:45.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.104+0000 7fc3a8e76640 1 --2- 172.21.15.2:0/3123490047 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ae8551deb0 0x55ae855202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.104+0000 7fc383fff640 10 client.17451 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:45.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.104+0000 7fc3a8e76640 1 --2- 172.21.15.2:0/3123490047 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ae8551deb0 0x55ae855202a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14cdd37bd6b7ecd6 server_cookie=70d1a3e6637a1019 in_seq=0 out_seq=0 2024-03-15T13:04:45.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.105+0000 7fc383fff640 1 -- 172.21.15.2:0/3123490047 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ae85462310 con 0x55ae8551deb0 2024-03-15T13:04:45.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.105+0000 7fc383fff640 10 client.17451 handle_command_reply: tid=0 2024-03-15T13:04:45.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.105+0000 7fc3add8a740 1 client.17451 shutdown 2024-03-15T13:04:45.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.106+0000 7fc3add8a740 1 -- 172.21.15.2:0/3123490047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae8550d8b0 msgr2=0x55ae8550b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.106+0000 7fc3add8a740 1 --2- 172.21.15.2:0/3123490047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae8550d8b0 0x55ae8550b350 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc39c037040 tx=0x7fc39c02c0f0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.106+0000 7fc3add8a740 1 -- 172.21.15.2:0/3123490047 shutdown_connections 2024-03-15T13:04:45.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.106+0000 7fc3add8a740 1 --2- 172.21.15.2:0/3123490047 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ae8551deb0 0x55ae855202a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.106+0000 7fc3add8a740 1 --2- 172.21.15.2:0/3123490047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae8550d8b0 0x55ae8550b350 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.106+0000 7fc3add8a740 1 --2- 172.21.15.2:0/3123490047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504d30 0x55ae8550ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.106+0000 7fc383fff640 0 client.17451 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:45.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.106+0000 7fc3add8a740 1 --2- 172.21.15.2:0/3123490047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504380 0x55ae8550a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.106+0000 7fc3add8a740 1 -- 172.21.15.2:0/3123490047 >> 172.21.15.2:0/3123490047 conn(0x55ae854fe840 msgr2=0x55ae855007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:45.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.106+0000 7fc3add8a740 1 -- 172.21.15.2:0/3123490047 shutdown_connections 2024-03-15T13:04:45.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.106+0000 7fc3add8a740 1 -- 172.21.15.2:0/3123490047 wait complete. 2024-03-15T13:04:45.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.106+0000 7fc3add8a740 20 client.17451 trim_cache size 0 max 16384 2024-03-15T13:04:45.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.122+0000 7fc3add8a740 1 Processor -- start 2024-03-15T13:04:45.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.122+0000 7fc3add8a740 1 -- start start 2024-03-15T13:04:45.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.122+0000 7fc3add8a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504380 0x55ae8550d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.122+0000 7fc3add8a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504d30 0x55ae8550b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.122+0000 7fc3add8a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae8550d8b0 0x55ae85518db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.122+0000 7fc3add8a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae85445f40 con 0x55ae8550d8b0 2024-03-15T13:04:45.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.122+0000 7fc3add8a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae855036a0 con 0x55ae85504380 2024-03-15T13:04:45.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.122+0000 7fc3add8a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae85520900 con 0x55ae85504d30 2024-03-15T13:04:45.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.122+0000 7fc3a9677640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae8550d8b0 0x55ae85518db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.122+0000 7fc3a8e76640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504380 0x55ae8550d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.123+0000 7fc3a8e76640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504380 0x55ae8550d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48726/0 (socket says 172.21.15.2:48726) 2024-03-15T13:04:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.123+0000 7fc3a3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504d30 0x55ae8550b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.123+0000 7fc3a8e76640 1 -- 172.21.15.2:0/149461678 learned_addr learned my addr 172.21.15.2:0/149461678 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.123+0000 7fc3a9677640 1 -- 172.21.15.2:0/149461678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504d30 msgr2=0x55ae8550b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.123+0000 7fc3a9677640 1 --2- 172.21.15.2:0/149461678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504d30 0x55ae8550b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.123+0000 7fc3a9677640 1 -- 172.21.15.2:0/149461678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504380 msgr2=0x55ae8550d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.123+0000 7fc3a9677640 1 --2- 172.21.15.2:0/149461678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504380 0x55ae8550d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.123+0000 7fc3a9677640 1 -- 172.21.15.2:0/149461678 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae85406c00 con 0x55ae8550d8b0 2024-03-15T13:04:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.123+0000 7fc3a8e76640 1 --2- 172.21.15.2:0/149461678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504380 0x55ae8550d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.123+0000 7fc3a3fff640 1 --2- 172.21.15.2:0/149461678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504d30 0x55ae8550b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.123+0000 7fc3a9677640 1 --2- 172.21.15.2:0/149461678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae8550d8b0 0x55ae85518db0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc39c041770 tx=0x7fc39c0325e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58dcdc6ab4a3d9ae server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.123+0000 7fc38e7fc640 1 -- 172.21.15.2:0/149461678 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c059070 con 0x55ae8550d8b0 2024-03-15T13:04:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.123+0000 7fc38e7fc640 1 -- 172.21.15.2:0/149461678 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fc39c034d70 con 0x55ae8550d8b0 2024-03-15T13:04:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.123+0000 7fc38e7fc640 1 -- 172.21.15.2:0/149461678 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c032af0 con 0x55ae8550d8b0 2024-03-15T13:04:45.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.124+0000 7fc3add8a740 1 -- 172.21.15.2:0/149461678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae8550d8b0 msgr2=0x55ae85518db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.124+0000 7fc3add8a740 1 --2- 172.21.15.2:0/149461678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae8550d8b0 0x55ae85518db0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc39c041770 tx=0x7fc39c0325e0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.124+0000 7fc3add8a740 1 -- 172.21.15.2:0/149461678 shutdown_connections 2024-03-15T13:04:45.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.124+0000 7fc3add8a740 1 --2- 172.21.15.2:0/149461678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae8550d8b0 0x55ae85518db0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.124+0000 7fc3add8a740 1 --2- 172.21.15.2:0/149461678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae85504d30 0x55ae8550b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.124+0000 7fc3add8a740 1 --2- 172.21.15.2:0/149461678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504380 0x55ae8550d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.124+0000 7fc3add8a740 1 -- 172.21.15.2:0/149461678 >> 172.21.15.2:0/149461678 conn(0x55ae854fe840 msgr2=0x55ae85505130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:45.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.124+0000 7fc3add8a740 1 -- 172.21.15.2:0/149461678 shutdown_connections 2024-03-15T13:04:45.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.124+0000 7fc3add8a740 1 -- 172.21.15.2:0/149461678 wait complete. 2024-03-15T13:04:45.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.124+0000 7fc3add8a740 1 -- start start 2024-03-15T13:04:45.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.124+0000 7fc3add8a740 1 Processor -- start 2024-03-15T13:04:45.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.124+0000 7fc3add8a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504380 0x55ae855085a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3add8a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae85504d30 0x55ae85506f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3add8a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae8550d8b0 0x55ae85507670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3add8a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae85507d70 con 0x55ae85504d30 2024-03-15T13:04:45.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3add8a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae85578730 con 0x55ae85504380 2024-03-15T13:04:45.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3add8a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae85578870 con 0x55ae8550d8b0 2024-03-15T13:04:45.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3add8a740 10 client.0 _ll_register_callbacks cb 0x7fc398005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:45.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3a3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae85504d30 0x55ae85506f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3a8e76640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504380 0x55ae855085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3a8e76640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504380 0x55ae855085a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:48734/0 (socket says 172.21.15.2:48734) 2024-03-15T13:04:45.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3a8e76640 1 -- 172.21.15.2:0/3583512609 learned_addr learned my addr 172.21.15.2:0/3583512609 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:45.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3a9677640 1 --2- 172.21.15.2:0/3583512609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae8550d8b0 0x55ae85507670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3a3fff640 1 -- 172.21.15.2:0/3583512609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae8550d8b0 msgr2=0x55ae85507670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3a3fff640 1 --2- 172.21.15.2:0/3583512609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae8550d8b0 0x55ae85507670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3a3fff640 1 -- 172.21.15.2:0/3583512609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504380 msgr2=0x55ae855085a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3a3fff640 1 --2- 172.21.15.2:0/3583512609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504380 0x55ae855085a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3a3fff640 1 -- 172.21.15.2:0/3583512609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc398012040 con 0x55ae85504d30 2024-03-15T13:04:45.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3a9677640 1 --2- 172.21.15.2:0/3583512609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae8550d8b0 0x55ae85507670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:45.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3a3fff640 1 --2- 172.21.15.2:0/3583512609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae85504d30 0x55ae85506f50 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc39009a460 tx=0x7fc39009a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d5709f9254e826 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:45.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc383fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:45.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc383fff640 1 -- 172.21.15.2:0/3583512609 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39009a760 con 0x55ae85504d30 2024-03-15T13:04:45.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc383fff640 1 -- 172.21.15.2:0/3583512609 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fc3900a2a70 con 0x55ae85504d30 2024-03-15T13:04:45.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3add8a740 1 -- 172.21.15.2:0/3583512609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae8516b860 con 0x55ae85504d30 2024-03-15T13:04:45.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc383fff640 1 -- 172.21.15.2:0/3583512609 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3900a2c50 con 0x55ae85504d30 2024-03-15T13:04:45.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.125+0000 7fc3add8a740 10 client.17457 fetch_fsmap learned FSMap version 46 2024-03-15T13:04:45.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.126+0000 7fc3add8a740 1 -- 172.21.15.2:0/3583512609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55ae8542e0b0 con 0x55ae85504d30 2024-03-15T13:04:45.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.126+0000 7fc383fff640 1 -- 172.21.15.2:0/3583512609 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55ae8516b860 con 0x55ae85504d30 2024-03-15T13:04:45.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.126+0000 7fc383fff640 1 -- 172.21.15.2:0/3583512609 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 46) v1 ==== 7958+0+0 (secure 0 0 0) 0x7fc390081070 con 0x55ae85504d30 2024-03-15T13:04:45.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.126+0000 7fc3add8a740 10 client.17457 fetch_fsmap finished waiting for FSMap version 46 2024-03-15T13:04:45.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.126+0000 7fc3add8a740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:45.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.126+0000 7fc3add8a740 20 client.17457 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:45.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.126+0000 7fc3add8a740 1 --2- 172.21.15.2:0/3583512609 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ae85579d50 0x55ae8557c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.126+0000 7fc3add8a740 4 client.17457 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:04:45.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.126+0000 7fc3add8a740 1 -- 172.21.15.2:0/3583512609 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ae851bc410 con 0x55ae85579d50 2024-03-15T13:04:45.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.126+0000 7fc3a8e76640 1 --2- 172.21.15.2:0/3583512609 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ae85579d50 0x55ae8557c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.126+0000 7fc3a8e76640 1 --2- 172.21.15.2:0/3583512609 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ae85579d50 0x55ae8557c170 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc0185945fcf0c96 server_cookie=5ca29b7efc945554 in_seq=0 out_seq=0 2024-03-15T13:04:45.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.126+0000 7fc383fff640 10 client.17457 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:45.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.128+0000 7fc383fff640 1 -- 172.21.15.2:0/3583512609 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ae851bc410 con 0x55ae85579d50 2024-03-15T13:04:45.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.128+0000 7fc383fff640 10 client.17457 handle_command_reply: tid=0 2024-03-15T13:04:45.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.128+0000 7fc3add8a740 1 client.17457 shutdown 2024-03-15T13:04:45.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.128+0000 7fc3add8a740 1 -- 172.21.15.2:0/3583512609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae85504d30 msgr2=0x55ae85506f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.128+0000 7fc3add8a740 1 --2- 172.21.15.2:0/3583512609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae85504d30 0x55ae85506f50 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc39009a460 tx=0x7fc39009a600 comp rx=0 tx=0).stop 2024-03-15T13:04:45.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.128+0000 7fc3add8a740 1 -- 172.21.15.2:0/3583512609 shutdown_connections 2024-03-15T13:04:45.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.128+0000 7fc3add8a740 1 --2- 172.21.15.2:0/3583512609 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ae85579d50 0x55ae8557c170 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.128+0000 7fc3add8a740 1 --2- 172.21.15.2:0/3583512609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ae8550d8b0 0x55ae85507670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.128+0000 7fc3add8a740 1 --2- 172.21.15.2:0/3583512609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ae85504d30 0x55ae85506f50 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.128+0000 7fc3add8a740 1 --2- 172.21.15.2:0/3583512609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae85504380 0x55ae855085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.128+0000 7fc383fff640 0 client.17457 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:45.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.128+0000 7fc3add8a740 1 -- 172.21.15.2:0/3583512609 >> 172.21.15.2:0/3583512609 conn(0x55ae854fe840 msgr2=0x55ae854ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:45.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.128+0000 7fc3add8a740 1 -- 172.21.15.2:0/3583512609 shutdown_connections 2024-03-15T13:04:45.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.128+0000 7fc3add8a740 1 -- 172.21.15.2:0/3583512609 wait complete. 2024-03-15T13:04:45.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.128+0000 7fc3add8a740 20 client.17457 trim_cache size 0 max 16384 2024-03-15T13:04:45.122 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:04:45.122 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:04:45.122 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "9ff39ea8-23e5-4f0f-9fba-0129e54e4a3e", 2024-03-15T13:04:45.122 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:04:45.122 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:04:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.129+0000 7fc3ab0da640 1 -- 172.21.15.2:0/4045486727 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc3940a1ba0 msgr2=0x7fc3940a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.129+0000 7fc3ab0da640 1 --2- 172.21.15.2:0/4045486727 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc3940a1ba0 0x7fc3940a4060 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc39c005cc0 tx=0x7fc39c032ea0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.129+0000 7fc3ab0da640 1 -- 172.21.15.2:0/4045486727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3a41056e0 msgr2=0x7fc3a419a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.129+0000 7fc3ab0da640 1 --2- 172.21.15.2:0/4045486727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3a41056e0 0x7fc3a419a280 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc39000e9c0 tx=0x7fc39000ee80 comp rx=0 tx=0).stop 2024-03-15T13:04:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.129+0000 7fc3ab0da640 1 -- 172.21.15.2:0/4045486727 shutdown_connections 2024-03-15T13:04:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.129+0000 7fc3ab0da640 1 --2- 172.21.15.2:0/4045486727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3a419e600 0x7fc3a419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.129+0000 7fc3ab0da640 1 --2- 172.21.15.2:0/4045486727 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fc3940a1ba0 0x7fc3940a4060 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.129+0000 7fc3ab0da640 1 --2- 172.21.15.2:0/4045486727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3a41056e0 0x7fc3a419a280 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.129+0000 7fc3ab0da640 1 --2- 172.21.15.2:0/4045486727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3a4100610 0x7fc3a4199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.129+0000 7fc3ab0da640 1 -- 172.21.15.2:0/4045486727 >> 172.21.15.2:0/4045486727 conn(0x7fc3a4076770 msgr2=0x7fc3a4069fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.129+0000 7fc3ab0da640 1 -- 172.21.15.2:0/4045486727 shutdown_connections 2024-03-15T13:04:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.129+0000 7fc3ab0da640 1 -- 172.21.15.2:0/4045486727 wait complete. 2024-03-15T13:04:45.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:04:45.125 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:04:45.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.247+0000 7f600532c640 1 -- 172.21.15.2:0/2610415936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f600006a4c0 msgr2=0x7f600010e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.247+0000 7f600532c640 1 --2- 172.21.15.2:0/2610415936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f600006a4c0 0x7f600010e330 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009cd0 tx=0x7f5ff802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.247+0000 7f600532c640 1 -- 172.21.15.2:0/2610415936 shutdown_connections 2024-03-15T13:04:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.247+0000 7f600532c640 1 --2- 172.21.15.2:0/2610415936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f600006a4c0 0x7f600010e330 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.247+0000 7f600532c640 1 --2- 172.21.15.2:0/2610415936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6000069b20 0x7f6000069f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.247+0000 7f600532c640 1 --2- 172.21.15.2:0/2610415936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6000107640 0x7f6000107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.247+0000 7f600532c640 1 -- 172.21.15.2:0/2610415936 >> 172.21.15.2:0/2610415936 conn(0x7f6000077190 msgr2=0x7f60000775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.247+0000 7f600532c640 1 -- 172.21.15.2:0/2610415936 shutdown_connections 2024-03-15T13:04:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.247+0000 7f600532c640 1 -- 172.21.15.2:0/2610415936 wait complete. 2024-03-15T13:04:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f600532c640 1 Processor -- start 2024-03-15T13:04:45.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f600532c640 1 -- start start 2024-03-15T13:04:45.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f600532c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6000069b20 0x7f600019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f600532c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f600006a4c0 0x7f600019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f600532c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6000107640 0x7f6000197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f600532c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6000197f20 con 0x7f600006a4c0 2024-03-15T13:04:45.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f600532c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6000198090 con 0x7f6000107640 2024-03-15T13:04:45.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f600532c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6000198250 con 0x7f6000069b20 2024-03-15T13:04:45.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f5ffe7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f600006a4c0 0x7f600019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f5ffe7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f600006a4c0 0x7f600019dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33446/0 (socket says 172.21.15.2:33446) 2024-03-15T13:04:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f5ffe7fc640 1 -- 172.21.15.2:0/3710665722 learned_addr learned my addr 172.21.15.2:0/3710665722 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f5ffeffd640 1 --2- 172.21.15.2:0/3710665722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6000069b20 0x7f600019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f5fff7fe640 1 --2- 172.21.15.2:0/3710665722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6000107640 0x7f6000197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f5ffe7fc640 1 -- 172.21.15.2:0/3710665722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6000069b20 msgr2=0x7f600019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f5ffe7fc640 1 --2- 172.21.15.2:0/3710665722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6000069b20 0x7f600019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f5ffe7fc640 1 -- 172.21.15.2:0/3710665722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6000107640 msgr2=0x7f6000197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f5ffe7fc640 1 --2- 172.21.15.2:0/3710665722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6000107640 0x7f6000197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.248+0000 7f5ffe7fc640 1 -- 172.21.15.2:0/3710665722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff8009900 con 0x7f600006a4c0 2024-03-15T13:04:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.249+0000 7f5ffe7fc640 1 --2- 172.21.15.2:0/3710665722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f600006a4c0 0x7f600019dc00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400e9f0 tx=0x7f5ff400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.249+0000 7f5fe7fff640 1 -- 172.21.15.2:0/3710665722 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4009720 con 0x7f600006a4c0 2024-03-15T13:04:45.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.249+0000 7f600532c640 1 -- 172.21.15.2:0/3710665722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6000198530 con 0x7f600006a4c0 2024-03-15T13:04:45.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.249+0000 7f5fe7fff640 1 -- 172.21.15.2:0/3710665722 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5ff400cd80 con 0x7f600006a4c0 2024-03-15T13:04:45.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.249+0000 7f5fe7fff640 1 -- 172.21.15.2:0/3710665722 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4010690 con 0x7f600006a4c0 2024-03-15T13:04:45.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.249+0000 7f600532c640 1 -- 172.21.15.2:0/3710665722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60001a46f0 con 0x7f600006a4c0 2024-03-15T13:04:45.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.251+0000 7f5fe7fff640 1 -- 172.21.15.2:0/3710665722 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f5ff4004e90 con 0x7f600006a4c0 2024-03-15T13:04:45.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.251+0000 7f6007fe1740 1 Processor -- start 2024-03-15T13:04:45.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.251+0000 7f6007fe1740 1 -- start start 2024-03-15T13:04:45.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.251+0000 7f6007fe1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b0762b0 0x56528b076690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.251+0000 7f6007fe1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b076bd0 0x56528b07f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.251+0000 7f6007fe1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b07f7b0 0x56528b081ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.251+0000 7f5fe7fff640 1 --2- 172.21.15.2:0/3710665722 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5fe80a1ba0 0x7f5fe80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.251+0000 7f6007fe1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528afe5280 con 0x56528b076bd0 2024-03-15T13:04:45.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.251+0000 7f6007fe1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528afe27b0 con 0x56528b0762b0 2024-03-15T13:04:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.251+0000 7f6007fe1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528afddf90 con 0x56528b07f7b0 2024-03-15T13:04:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5fff7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b0762b0 0x56528b076690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5ffeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b076bd0 0x56528b07f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5fff7fe640 1 --2- 172.21.15.2:0/3710665722 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5fe80a1ba0 0x7f5fe80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5ffe7fc640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b07f7b0 0x56528b081ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5ffeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b076bd0 0x56528b07f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33460/0 (socket says 172.21.15.2:33460) 2024-03-15T13:04:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5ffeffd640 1 -- 172.21.15.2:0/3726508860 learned_addr learned my addr 172.21.15.2:0/3726508860 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5fe7fff640 1 -- 172.21.15.2:0/3710665722 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5ff4014070 con 0x7f600006a4c0 2024-03-15T13:04:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5ffeffd640 1 -- 172.21.15.2:0/3726508860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b07f7b0 msgr2=0x56528b081ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5ffeffd640 1 --2- 172.21.15.2:0/3726508860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b07f7b0 0x56528b081ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5ffeffd640 1 -- 172.21.15.2:0/3726508860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b0762b0 msgr2=0x56528b076690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5ffeffd640 1 --2- 172.21.15.2:0/3726508860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b0762b0 0x56528b076690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5ffeffd640 1 -- 172.21.15.2:0/3726508860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56528af9c570 con 0x56528b076bd0 2024-03-15T13:04:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5ffe7fc640 1 --2- 172.21.15.2:0/3726508860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b07f7b0 0x56528b081ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:45.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5ffeffd640 1 --2- 172.21.15.2:0/3726508860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b076bd0 0x56528b07f270 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009cf0 tx=0x7f5ff0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cbdd1537126aa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5fff7fe640 1 --2- 172.21.15.2:0/3710665722 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5fe80a1ba0 0x7f5fe80a4060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8005cc0 tx=0x7f5ff8005c50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5fe57fa640 1 -- 172.21.15.2:0/3726508860 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0009ea0 con 0x56528b076bd0 2024-03-15T13:04:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5fe57fa640 1 -- 172.21.15.2:0/3726508860 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5ff0004230 con 0x56528b076bd0 2024-03-15T13:04:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.252+0000 7f5fe57fa640 1 -- 172.21.15.2:0/3726508860 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00197e0 con 0x56528b076bd0 2024-03-15T13:04:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.253+0000 7f6007fe1740 1 -- 172.21.15.2:0/3726508860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b076bd0 msgr2=0x56528b07f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.253+0000 7f6007fe1740 1 --2- 172.21.15.2:0/3726508860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b076bd0 0x56528b07f270 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009cf0 tx=0x7f5ff0011810 comp rx=0 tx=0).stop 2024-03-15T13:04:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.253+0000 7f6007fe1740 1 -- 172.21.15.2:0/3726508860 shutdown_connections 2024-03-15T13:04:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.253+0000 7f6007fe1740 1 --2- 172.21.15.2:0/3726508860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b07f7b0 0x56528b081ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.253+0000 7f6007fe1740 1 --2- 172.21.15.2:0/3726508860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b076bd0 0x56528b07f270 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.253+0000 7f6007fe1740 1 --2- 172.21.15.2:0/3726508860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b0762b0 0x56528b076690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.253+0000 7f6007fe1740 1 -- 172.21.15.2:0/3726508860 >> 172.21.15.2:0/3726508860 conn(0x56528b070990 msgr2=0x56528b070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.253+0000 7f6007fe1740 1 -- 172.21.15.2:0/3726508860 shutdown_connections 2024-03-15T13:04:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.253+0000 7f6007fe1740 1 -- 172.21.15.2:0/3726508860 wait complete. 2024-03-15T13:04:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.253+0000 7f6007fe1740 1 -- start start 2024-03-15T13:04:45.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.253+0000 7f6007fe1740 1 Processor -- start 2024-03-15T13:04:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f6007fe1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b0762b0 0x56528b07f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f6007fe1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b076bd0 0x56528b07c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f6007fe1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b07f7b0 0x56528b07cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f6007fe1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528acb1430 con 0x56528b0762b0 2024-03-15T13:04:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f6007fe1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528ac6de00 con 0x56528b076bd0 2024-03-15T13:04:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f6007fe1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528af62fd0 con 0x56528b07f7b0 2024-03-15T13:04:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f6007fe1740 10 client.0 _ll_register_callbacks cb 0x56528afb8fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f5ffeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b0762b0 0x56528b07f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f5ffeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b0762b0 0x56528b07f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33476/0 (socket says 172.21.15.2:33476) 2024-03-15T13:04:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f5ffeffd640 1 -- 172.21.15.2:0/1097981399 learned_addr learned my addr 172.21.15.2:0/1097981399 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f5ffe7fc640 1 --2- 172.21.15.2:0/1097981399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b076bd0 0x56528b07c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f5fff7fe640 1 --2- 172.21.15.2:0/1097981399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b07f7b0 0x56528b07cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f5ffeffd640 1 -- 172.21.15.2:0/1097981399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b07f7b0 msgr2=0x56528b07cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f5ffeffd640 1 --2- 172.21.15.2:0/1097981399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b07f7b0 0x56528b07cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f5ffeffd640 1 -- 172.21.15.2:0/1097981399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b076bd0 msgr2=0x56528b07c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f5ffeffd640 1 --2- 172.21.15.2:0/1097981399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b076bd0 0x56528b07c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f5ffeffd640 1 -- 172.21.15.2:0/1097981399 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff0009950 con 0x56528b0762b0 2024-03-15T13:04:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f5ffeffd640 1 --2- 172.21.15.2:0/1097981399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b0762b0 0x56528b07f230 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0000c00 tx=0x7f5ff0005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5310235644c0d64e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f5fe2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f5fe2ff5640 1 -- 172.21.15.2:0/1097981399 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0005240 con 0x56528b0762b0 2024-03-15T13:04:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f5fe2ff5640 1 -- 172.21.15.2:0/1097981399 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5ff0003a90 con 0x56528b0762b0 2024-03-15T13:04:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.254+0000 7f6007fe1740 1 -- 172.21.15.2:0/1097981399 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56528acdd7e0 con 0x56528b0762b0 2024-03-15T13:04:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.255+0000 7f5fe2ff5640 1 -- 172.21.15.2:0/1097981399 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ff0009ea0 con 0x56528b0762b0 2024-03-15T13:04:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.255+0000 7f6007fe1740 10 client.17469 fetch_fsmap learned FSMap version 46 2024-03-15T13:04:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.255+0000 7f6007fe1740 1 -- 172.21.15.2:0/1097981399 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56528afdc610 con 0x56528b0762b0 2024-03-15T13:04:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.255+0000 7f5fe2ff5640 1 -- 172.21.15.2:0/1097981399 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x56528acdd7e0 con 0x56528b0762b0 2024-03-15T13:04:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.255+0000 7f5fe2ff5640 1 -- 172.21.15.2:0/1097981399 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 46) v1 ==== 7958+0+0 (secure 0 0 0) 0x7f5ff0021040 con 0x56528b0762b0 2024-03-15T13:04:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.255+0000 7f6007fe1740 10 client.17469 fetch_fsmap finished waiting for FSMap version 46 2024-03-15T13:04:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.255+0000 7f6007fe1740 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:45.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.255+0000 7f6007fe1740 20 client.17469 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:45.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.255+0000 7f6007fe1740 1 --2- 172.21.15.2:0/1097981399 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56528b08f990 0x56528b091da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.255+0000 7f6007fe1740 4 client.17469 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:04:45.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.255+0000 7f6007fe1740 1 -- 172.21.15.2:0/1097981399 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56528ad1a720 con 0x56528b08f990 2024-03-15T13:04:45.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.255+0000 7f5ffe7fc640 1 --2- 172.21.15.2:0/1097981399 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56528b08f990 0x56528b091da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.256+0000 7f5fe2ff5640 10 client.17469 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:45.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.256+0000 7f5ffe7fc640 1 --2- 172.21.15.2:0/1097981399 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56528b08f990 0x56528b091da0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0ed00ccebeeeb86 server_cookie=95865e877b390e96 in_seq=0 out_seq=0 2024-03-15T13:04:45.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.256+0000 7f5fe2ff5640 1 -- 172.21.15.2:0/1097981399 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56528ad1a720 con 0x56528b08f990 2024-03-15T13:04:45.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.256+0000 7f5fe2ff5640 10 client.17469 handle_command_reply: tid=0 2024-03-15T13:04:45.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.257+0000 7f6007fe1740 1 client.17469 shutdown 2024-03-15T13:04:45.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.257+0000 7f6007fe1740 1 -- 172.21.15.2:0/1097981399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b0762b0 msgr2=0x56528b07f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.257+0000 7f6007fe1740 1 --2- 172.21.15.2:0/1097981399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b0762b0 0x56528b07f230 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0000c00 tx=0x7f5ff0005d30 comp rx=0 tx=0).stop 2024-03-15T13:04:45.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.257+0000 7f6007fe1740 1 -- 172.21.15.2:0/1097981399 shutdown_connections 2024-03-15T13:04:45.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.257+0000 7f6007fe1740 1 --2- 172.21.15.2:0/1097981399 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56528b08f990 0x56528b091da0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.257+0000 7f6007fe1740 1 --2- 172.21.15.2:0/1097981399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b07f7b0 0x56528b07cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.257+0000 7f5fe2ff5640 0 client.17469 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:45.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.257+0000 7f6007fe1740 1 --2- 172.21.15.2:0/1097981399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b076bd0 0x56528b07c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.257+0000 7f6007fe1740 1 --2- 172.21.15.2:0/1097981399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b0762b0 0x56528b07f230 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.257+0000 7f6007fe1740 1 -- 172.21.15.2:0/1097981399 >> 172.21.15.2:0/1097981399 conn(0x56528b070990 msgr2=0x56528b072730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:45.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.257+0000 7f6007fe1740 1 -- 172.21.15.2:0/1097981399 shutdown_connections 2024-03-15T13:04:45.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.257+0000 7f6007fe1740 1 -- 172.21.15.2:0/1097981399 wait complete. 2024-03-15T13:04:45.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.257+0000 7f6007fe1740 20 client.17469 trim_cache size 0 max 16384 2024-03-15T13:04:45.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f6007fe1740 1 Processor -- start 2024-03-15T13:04:45.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f6007fe1740 1 -- start start 2024-03-15T13:04:45.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f6007fe1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b0762b0 0x56528b072460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f6007fe1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b076bd0 0x56528b07ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f6007fe1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b07f7b0 0x56528b07f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f6007fe1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528b077410 con 0x56528b07f7b0 2024-03-15T13:04:45.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f6007fe1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528b0719e0 con 0x56528b0762b0 2024-03-15T13:04:45.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f6007fe1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528b08bdd0 con 0x56528b076bd0 2024-03-15T13:04:45.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f5fff7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b07f7b0 0x56528b07f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f5fff7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b07f7b0 0x56528b07f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33484/0 (socket says 172.21.15.2:33484) 2024-03-15T13:04:45.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f5fff7fe640 1 -- 172.21.15.2:0/3052580367 learned_addr learned my addr 172.21.15.2:0/3052580367 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:45.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f5ffe7fc640 1 --2- 172.21.15.2:0/3052580367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b076bd0 0x56528b07ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f5ffeffd640 1 --2- 172.21.15.2:0/3052580367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b0762b0 0x56528b072460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f5fff7fe640 1 -- 172.21.15.2:0/3052580367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b076bd0 msgr2=0x56528b07ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f5fff7fe640 1 --2- 172.21.15.2:0/3052580367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b076bd0 0x56528b07ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f5fff7fe640 1 -- 172.21.15.2:0/3052580367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b0762b0 msgr2=0x56528b072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f5fff7fe640 1 --2- 172.21.15.2:0/3052580367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b0762b0 0x56528b072460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.273+0000 7f5fff7fe640 1 -- 172.21.15.2:0/3052580367 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56528af78ac0 con 0x56528b07f7b0 2024-03-15T13:04:45.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.274+0000 7f5fff7fe640 1 --2- 172.21.15.2:0/3052580367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b07f7b0 0x56528b07f230 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009cd0 tx=0x7f5ff8009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283ee2484ff8ed64 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.274+0000 7f5fe4ff9640 1 -- 172.21.15.2:0/3052580367 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8032680 con 0x56528b07f7b0 2024-03-15T13:04:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.274+0000 7f5fe4ff9640 1 -- 172.21.15.2:0/3052580367 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5ff8003e40 con 0x56528b07f7b0 2024-03-15T13:04:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.274+0000 7f5fe4ff9640 1 -- 172.21.15.2:0/3052580367 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8004610 con 0x56528b07f7b0 2024-03-15T13:04:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.274+0000 7f6007fe1740 1 -- 172.21.15.2:0/3052580367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b07f7b0 msgr2=0x56528b07f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.274+0000 7f6007fe1740 1 --2- 172.21.15.2:0/3052580367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b07f7b0 0x56528b07f230 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009cd0 tx=0x7f5ff8009820 comp rx=0 tx=0).stop 2024-03-15T13:04:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.274+0000 7f6007fe1740 1 -- 172.21.15.2:0/3052580367 shutdown_connections 2024-03-15T13:04:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.274+0000 7f6007fe1740 1 --2- 172.21.15.2:0/3052580367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b07f7b0 0x56528b07f230 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.274+0000 7f6007fe1740 1 --2- 172.21.15.2:0/3052580367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b076bd0 0x56528b07ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.274+0000 7f6007fe1740 1 --2- 172.21.15.2:0/3052580367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b0762b0 0x56528b072460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.274+0000 7f6007fe1740 1 -- 172.21.15.2:0/3052580367 >> 172.21.15.2:0/3052580367 conn(0x56528b070990 msgr2=0x56528b071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.274+0000 7f6007fe1740 1 -- 172.21.15.2:0/3052580367 shutdown_connections 2024-03-15T13:04:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.274+0000 7f6007fe1740 1 -- 172.21.15.2:0/3052580367 wait complete. 2024-03-15T13:04:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f6007fe1740 1 -- start start 2024-03-15T13:04:45.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f6007fe1740 1 Processor -- start 2024-03-15T13:04:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f6007fe1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b0762b0 0x56528b079290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f6007fe1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b076bd0 0x56528b0797d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f6007fe1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b07f7b0 0x56528b085660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f6007fe1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528b079ed0 con 0x56528b0762b0 2024-03-15T13:04:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f6007fe1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528b085ba0 con 0x56528b07f7b0 2024-03-15T13:04:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f6007fe1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528b085d60 con 0x56528b076bd0 2024-03-15T13:04:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f6007fe1740 10 client.0 _ll_register_callbacks cb 0x7f5ff0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:04:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f5ffeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b0762b0 0x56528b079290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f5ffeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b0762b0 0x56528b079290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33490/0 (socket says 172.21.15.2:33490) 2024-03-15T13:04:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f5ffeffd640 1 -- 172.21.15.2:0/953548283 learned_addr learned my addr 172.21.15.2:0/953548283 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:04:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f5fff7fe640 1 --2- 172.21.15.2:0/953548283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b07f7b0 0x56528b085660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f5ffe7fc640 1 --2- 172.21.15.2:0/953548283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b076bd0 0x56528b0797d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f5ffeffd640 1 -- 172.21.15.2:0/953548283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b076bd0 msgr2=0x56528b0797d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f5ffeffd640 1 --2- 172.21.15.2:0/953548283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b076bd0 0x56528b0797d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f5ffeffd640 1 -- 172.21.15.2:0/953548283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b07f7b0 msgr2=0x56528b085660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:04:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f5ffeffd640 1 --2- 172.21.15.2:0/953548283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b07f7b0 0x56528b085660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f5ffeffd640 1 -- 172.21.15.2:0/953548283 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff8048040 con 0x56528b0762b0 2024-03-15T13:04:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.275+0000 7f5fff7fe640 1 --2- 172.21.15.2:0/953548283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b07f7b0 0x56528b085660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:04:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f5ffeffd640 1 --2- 172.21.15.2:0/953548283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b0762b0 0x56528b079290 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0019900 tx=0x7f5ff0019930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40875fdded315750 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:04:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f5fe2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:04:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f5fe2ff5640 1 -- 172.21.15.2:0/953548283 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0009ea0 con 0x56528b0762b0 2024-03-15T13:04:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f5fe2ff5640 1 -- 172.21.15.2:0/953548283 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5ff0000e40 con 0x56528b0762b0 2024-03-15T13:04:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f6007fe1740 1 -- 172.21.15.2:0/953548283 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ff0004230 con 0x56528b0762b0 2024-03-15T13:04:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f5fe2ff5640 1 -- 172.21.15.2:0/953548283 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ff0009b70 con 0x56528b0762b0 2024-03-15T13:04:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f6007fe1740 10 client.17475 fetch_fsmap learned FSMap version 46 2024-03-15T13:04:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f6007fe1740 1 -- 172.21.15.2:0/953548283 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56528af9e6a0 con 0x56528b0762b0 2024-03-15T13:04:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f5fe2ff5640 1 -- 172.21.15.2:0/953548283 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5ff0004230 con 0x56528b0762b0 2024-03-15T13:04:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f5fe2ff5640 1 -- 172.21.15.2:0/953548283 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 46) v1 ==== 7958+0+0 (secure 0 0 0) 0x7f5ff0021030 con 0x56528b0762b0 2024-03-15T13:04:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f6007fe1740 10 client.17475 fetch_fsmap finished waiting for FSMap version 46 2024-03-15T13:04:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f6007fe1740 10 client.17475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:04:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f6007fe1740 20 client.17475 populate_metadata read hostname 'smithi002' 2024-03-15T13:04:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f6007fe1740 1 --2- 172.21.15.2:0/953548283 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56528b0ebcd0 0x56528b0ee0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:04:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f6007fe1740 4 client.17475 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:04:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.276+0000 7f6007fe1740 1 -- 172.21.15.2:0/953548283 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56528ad69890 con 0x56528b0ebcd0 2024-03-15T13:04:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f5ffe7fc640 1 --2- 172.21.15.2:0/953548283 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56528b0ebcd0 0x56528b0ee0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:04:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f5fe2ff5640 10 client.17475 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f5ffe7fc640 1 --2- 172.21.15.2:0/953548283 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56528b0ebcd0 0x56528b0ee0f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f673ef1090dc5b34 server_cookie=2cc3dec02b9b54b7 in_seq=0 out_seq=0 2024-03-15T13:04:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f5fe2ff5640 1 -- 172.21.15.2:0/953548283 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56528ad69890 con 0x56528b0ebcd0 2024-03-15T13:04:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f5fe2ff5640 10 client.17475 handle_command_reply: tid=0 2024-03-15T13:04:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f6007fe1740 1 client.17475 shutdown 2024-03-15T13:04:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f6007fe1740 1 -- 172.21.15.2:0/953548283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b0762b0 msgr2=0x56528b079290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f6007fe1740 1 --2- 172.21.15.2:0/953548283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b0762b0 0x56528b079290 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0019900 tx=0x7f5ff0019930 comp rx=0 tx=0).stop 2024-03-15T13:04:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f6007fe1740 1 -- 172.21.15.2:0/953548283 shutdown_connections 2024-03-15T13:04:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f6007fe1740 1 --2- 172.21.15.2:0/953548283 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x56528b0ebcd0 0x56528b0ee0f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f6007fe1740 1 --2- 172.21.15.2:0/953548283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56528b07f7b0 0x56528b085660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f6007fe1740 1 --2- 172.21.15.2:0/953548283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56528b076bd0 0x56528b0797d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f5fe2ff5640 0 client.17475 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:04:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f6007fe1740 1 --2- 172.21.15.2:0/953548283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56528b0762b0 0x56528b079290 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f6007fe1740 1 -- 172.21.15.2:0/953548283 >> 172.21.15.2:0/953548283 conn(0x56528b070990 msgr2=0x56528b06fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f6007fe1740 1 -- 172.21.15.2:0/953548283 shutdown_connections 2024-03-15T13:04:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f6007fe1740 1 -- 172.21.15.2:0/953548283 wait complete. 2024-03-15T13:04:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.277+0000 7f6007fe1740 20 client.17475 trim_cache size 0 max 16384 2024-03-15T13:04:45.271 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:04:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T13:04:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:04:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: "9ff39ea8-23e5-4f0f-9fba-0129e54e4a3e": { 2024-03-15T13:04:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:04:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "9ff39ea8-23e5-4f0f-9fba-0129e54e4a3e", 2024-03-15T13:04:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:04:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:04:45.271 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:04:45.271 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:04:45.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.278+0000 7f600532c640 1 -- 172.21.15.2:0/3710665722 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5fe80a1ba0 msgr2=0x7f5fe80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.278+0000 7f600532c640 1 --2- 172.21.15.2:0/3710665722 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5fe80a1ba0 0x7f5fe80a4060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8005cc0 tx=0x7f5ff8005c50 comp rx=0 tx=0).stop 2024-03-15T13:04:45.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.278+0000 7f600532c640 1 -- 172.21.15.2:0/3710665722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f600006a4c0 msgr2=0x7f600019dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:04:45.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.278+0000 7f600532c640 1 --2- 172.21.15.2:0/3710665722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f600006a4c0 0x7f600019dc00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400e9f0 tx=0x7f5ff400edf0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.278+0000 7f600532c640 1 -- 172.21.15.2:0/3710665722 shutdown_connections 2024-03-15T13:04:45.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.278+0000 7f600532c640 1 --2- 172.21.15.2:0/3710665722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6000107640 0x7f6000197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.278+0000 7f600532c640 1 --2- 172.21.15.2:0/3710665722 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5fe80a1ba0 0x7f5fe80a4060 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.278+0000 7f600532c640 1 --2- 172.21.15.2:0/3710665722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f600006a4c0 0x7f600019dc00 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.278+0000 7f600532c640 1 --2- 172.21.15.2:0/3710665722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6000069b20 0x7f600019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:04:45.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.278+0000 7f600532c640 1 -- 172.21.15.2:0/3710665722 >> 172.21.15.2:0/3710665722 conn(0x7f6000077190 msgr2=0x7f60000ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:04:45.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.278+0000 7f600532c640 1 -- 172.21.15.2:0/3710665722 shutdown_connections 2024-03-15T13:04:45.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:04:45.278+0000 7f600532c640 1 -- 172.21.15.2:0/3710665722 wait complete. 2024-03-15T13:04:45.274 INFO:tasks.cephfs.filesystem:scrub status for tag:9ff39ea8-23e5-4f0f-9fba-0129e54e4a3e - {'path': '/', 'tag': '9ff39ea8-23e5-4f0f-9fba-0129e54e4a3e', 'options': 'recursive,force'} 2024-03-15T13:04:46.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:45 smithi002 ceph-mon[32060]: pgmap v178: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:46.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:45 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:46.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:45 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:04:46.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:45 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:04:46.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:45 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:04:46.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:45 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:04:46.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:45 smithi184 ceph-mon[35993]: pgmap v178: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:46.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:45 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:46.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:45 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:04:46.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:45 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:04:46.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:45 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:04:46.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:45 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:04:46.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:45 smithi149 ceph-mon[35927]: pgmap v178: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:46.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:45 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:46.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:45 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:04:46.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:45 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:04:46.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:45 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:04:46.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:45 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:04:48.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:47 smithi002 ceph-mon[32060]: pgmap v179: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:04:48.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:47 smithi184 ceph-mon[35993]: pgmap v179: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:04:48.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:47 smithi149 ceph-mon[35927]: pgmap v179: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:04:50.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:49 smithi002 ceph-mon[32060]: pgmap v180: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:50.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:49 smithi184 ceph-mon[35993]: pgmap v180: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:50.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:49 smithi149 ceph-mon[35927]: pgmap v180: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:04:51.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:50 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:04:51.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:50 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:04:51.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:50 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:04:52.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:51 smithi002 ceph-mon[32060]: pgmap v181: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:04:52.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:51 smithi184 ceph-mon[35993]: pgmap v181: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:04:52.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:51 smithi149 ceph-mon[35927]: pgmap v181: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:04:54.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:53 smithi002 ceph-mon[32060]: pgmap v182: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:04:54.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:53 smithi184 ceph-mon[35993]: pgmap v182: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:04:54.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:53 smithi149 ceph-mon[35927]: pgmap v182: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:04:55.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:54 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:55.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:54 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:55.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:54 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:56.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:55 smithi002 ceph-mon[32060]: pgmap v183: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:04:56.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:55 smithi184 ceph-mon[35993]: pgmap v183: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:04:56.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:55 smithi149 ceph-mon[35927]: pgmap v183: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:04:58.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:57 smithi002 ceph-mon[32060]: pgmap v184: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:04:58.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:57 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:57 smithi184 ceph-mon[35993]: pgmap v184: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:04:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:57 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:04:58.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:57 smithi149 ceph-mon[35927]: pgmap v184: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:04:58.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:57 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:05:00.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:04:59 smithi002 ceph-mon[32060]: pgmap v185: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:05:00.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:04:59 smithi184 ceph-mon[35993]: pgmap v185: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:05:00.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:04:59 smithi149 ceph-mon[35927]: pgmap v185: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:05:02.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:01 smithi002 ceph-mon[32060]: pgmap v186: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:05:02.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:01 smithi184 ceph-mon[35993]: pgmap v186: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:05:02.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:01 smithi149 ceph-mon[35927]: pgmap v186: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:05:04.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:03 smithi002 ceph-mon[32060]: pgmap v187: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:04.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:03 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:05:04.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:03 smithi184 ceph-mon[35993]: pgmap v187: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:04.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:03 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:05:04.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:03 smithi149 ceph-mon[35927]: pgmap v187: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:04.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:03 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:05:05.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:04 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:05:05.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:04 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:05:05.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:04 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:05:05.959 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:05 smithi184 ceph-mon[35993]: pgmap v188: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:05.959 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:05 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:05:05.959 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:05 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:05:05.959 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:05 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:05:06.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:05 smithi002 ceph-mon[32060]: pgmap v188: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:06.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:05 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:05:06.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:05 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:05:06.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:05 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:05:06.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:05 smithi149 ceph-mon[35927]: pgmap v188: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:06.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:05 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:05:06.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:05 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:05:06.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:05 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:05:08.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:07 smithi002 ceph-mon[32060]: pgmap v189: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:05:08.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:07 smithi184 ceph-mon[35993]: pgmap v189: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:05:08.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:07 smithi149 ceph-mon[35927]: pgmap v189: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:05:10.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:09 smithi002 ceph-mon[32060]: pgmap v190: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:05:10.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:09 smithi184 ceph-mon[35993]: pgmap v190: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:05:10.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:09 smithi149 ceph-mon[35927]: pgmap v190: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:05:12.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:11 smithi002 ceph-mon[32060]: pgmap v191: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:05:12.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:11 smithi184 ceph-mon[35993]: pgmap v191: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:05:12.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:11 smithi149 ceph-mon[35927]: pgmap v191: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:05:14.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:13 smithi002 ceph-mon[32060]: pgmap v192: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:05:14.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:13 smithi002 ceph-mon[32060]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:05:14.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:13 smithi184 ceph-mon[35993]: pgmap v192: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:05:14.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:13 smithi184 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:05:14.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:13 smithi149 ceph-mon[35927]: pgmap v192: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:05:14.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:13 smithi149 ceph-mon[35927]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-15T13:05:15.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:05:15.276 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:05:15.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.408+0000 7f2e67f5e640 1 -- 172.21.15.2:0/3676794383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e60069b20 msgr2=0x7f2e60069fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.408+0000 7f2e67f5e640 1 --2- 172.21.15.2:0/3676794383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e60069b20 0x7f2e60069fa0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009930 tx=0x7f2e5002bcd0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.408+0000 7f2e67f5e640 1 -- 172.21.15.2:0/3676794383 shutdown_connections 2024-03-15T13:05:15.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.408+0000 7f2e67f5e640 1 --2- 172.21.15.2:0/3676794383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e6006a4e0 0x7f2e6010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.408+0000 7f2e67f5e640 1 --2- 172.21.15.2:0/3676794383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e60069b20 0x7f2e60069fa0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.408+0000 7f2e67f5e640 1 --2- 172.21.15.2:0/3676794383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e60107600 0x7f2e601079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.408+0000 7f2e67f5e640 1 -- 172.21.15.2:0/3676794383 >> 172.21.15.2:0/3676794383 conn(0x7f2e60077190 msgr2=0x7f2e600775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:15.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.408+0000 7f2e67f5e640 1 -- 172.21.15.2:0/3676794383 shutdown_connections 2024-03-15T13:05:15.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.408+0000 7f2e67f5e640 1 -- 172.21.15.2:0/3676794383 wait complete. 2024-03-15T13:05:15.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.409+0000 7f2e67f5e640 1 Processor -- start 2024-03-15T13:05:15.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.409+0000 7f2e67f5e640 1 -- start start 2024-03-15T13:05:15.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.409+0000 7f2e67f5e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e60069b20 0x7f2e6019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.409+0000 7f2e67f5e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e6006a4e0 0x7f2e6019fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.409+0000 7f2e67f5e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e60107600 0x7f2e60199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.409+0000 7f2e67f5e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6019a0e0 con 0x7f2e6006a4e0 2024-03-15T13:05:15.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.409+0000 7f2e67f5e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6019a250 con 0x7f2e60069b20 2024-03-15T13:05:15.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.409+0000 7f2e67f5e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6019a410 con 0x7f2e60107600 2024-03-15T13:05:15.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.409+0000 7f2e654f9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e6006a4e0 0x7f2e6019fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.409+0000 7f2e65cfa640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e60069b20 0x7f2e6019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.409+0000 7f2e654f9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e6006a4e0 0x7f2e6019fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57788/0 (socket says 172.21.15.2:57788) 2024-03-15T13:05:15.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.409+0000 7f2e654f9640 1 -- 172.21.15.2:0/3936614325 learned_addr learned my addr 172.21.15.2:0/3936614325 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:15.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.409+0000 7f2e664fb640 1 --2- 172.21.15.2:0/3936614325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e60107600 0x7f2e60199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.410+0000 7f2e654f9640 1 -- 172.21.15.2:0/3936614325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e60107600 msgr2=0x7f2e60199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.410+0000 7f2e654f9640 1 --2- 172.21.15.2:0/3936614325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e60107600 0x7f2e60199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.410+0000 7f2e654f9640 1 -- 172.21.15.2:0/3936614325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e60069b20 msgr2=0x7f2e6019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.410+0000 7f2e654f9640 1 --2- 172.21.15.2:0/3936614325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e60069b20 0x7f2e6019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.410+0000 7f2e654f9640 1 -- 172.21.15.2:0/3936614325 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e50009560 con 0x7f2e6006a4e0 2024-03-15T13:05:15.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.410+0000 7f2e654f9640 1 --2- 172.21.15.2:0/3936614325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e6006a4e0 0x7f2e6019fe20 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009900 tx=0x7f2e5002cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:15.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.410+0000 7f2e56ffd640 1 -- 172.21.15.2:0/3936614325 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5002ccb0 con 0x7f2e6006a4e0 2024-03-15T13:05:15.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.410+0000 7f2e56ffd640 1 -- 172.21.15.2:0/3936614325 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2e500054d0 con 0x7f2e6006a4e0 2024-03-15T13:05:15.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.410+0000 7f2e67f5e640 1 -- 172.21.15.2:0/3936614325 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6019a690 con 0x7f2e6006a4e0 2024-03-15T13:05:15.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.410+0000 7f2e56ffd640 1 -- 172.21.15.2:0/3936614325 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50003c90 con 0x7f2e6006a4e0 2024-03-15T13:05:15.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.410+0000 7f2e67f5e640 1 -- 172.21.15.2:0/3936614325 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e601a6980 con 0x7f2e6006a4e0 2024-03-15T13:05:15.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.412+0000 7f2e56ffd640 1 -- 172.21.15.2:0/3936614325 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f2e50005000 con 0x7f2e6006a4e0 2024-03-15T13:05:15.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e6ac18740 1 Processor -- start 2024-03-15T13:05:15.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e6ac18740 1 -- start start 2024-03-15T13:05:15.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e56ffd640 1 --2- 172.21.15.2:0/3936614325 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f2e4c0a1ba0 0x7f2e4c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e6ac18740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4472b0 0x55a39b447690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e6ac18740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b447bd0 0x55a39b450270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e6ac18740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b4507b0 0x55a39b452ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e6ac18740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39b3b6280 con 0x55a39b4472b0 2024-03-15T13:05:15.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e6ac18740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39b3b37b0 con 0x55a39b447bd0 2024-03-15T13:05:15.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e6ac18740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39b3aef90 con 0x55a39b4507b0 2024-03-15T13:05:15.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e56ffd640 1 -- 172.21.15.2:0/3936614325 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f2e500f0180 con 0x7f2e6006a4e0 2024-03-15T13:05:15.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e664fb640 1 --2- 172.21.15.2:0/3936614325 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f2e4c0a1ba0 0x7f2e4c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e65cfa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4472b0 0x55a39b447690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e65cfa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4472b0 0x55a39b447690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57800/0 (socket says 172.21.15.2:57800) 2024-03-15T13:05:15.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e65cfa640 1 -- 172.21.15.2:0/1031528931 learned_addr learned my addr 172.21.15.2:0/1031528931 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:15.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e654f9640 1 --2- 172.21.15.2:0/1031528931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b447bd0 0x55a39b450270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.413+0000 7f2e664fb640 1 --2- 172.21.15.2:0/1031528931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b4507b0 0x55a39b452ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.414+0000 7f2e654f9640 1 -- 172.21.15.2:0/1031528931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b4507b0 msgr2=0x55a39b452ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.414+0000 7f2e654f9640 1 --2- 172.21.15.2:0/1031528931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b4507b0 0x55a39b452ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.414+0000 7f2e654f9640 1 -- 172.21.15.2:0/1031528931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4472b0 msgr2=0x55a39b447690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.414+0000 7f2e654f9640 1 --2- 172.21.15.2:0/1031528931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4472b0 0x55a39b447690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.414+0000 7f2e654f9640 1 -- 172.21.15.2:0/1031528931 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a39b36d570 con 0x55a39b447bd0 2024-03-15T13:05:15.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.414+0000 7f2e65cfa640 1 --2- 172.21.15.2:0/1031528931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4472b0 0x55a39b447690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:15.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.414+0000 7f2e664fb640 1 --2- 172.21.15.2:0/3936614325 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f2e4c0a1ba0 0x7f2e4c0a4060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009fd0 tx=0x7f2e58013b80 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:15.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.414+0000 7f2e654f9640 1 --2- 172.21.15.2:0/1031528931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b447bd0 0x55a39b450270 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e500ed630 tx=0x7f2e50003730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6153d56988725913 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:15.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.414+0000 7f2e4bfff640 1 -- 172.21.15.2:0/1031528931 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e500c52b0 con 0x55a39b447bd0 2024-03-15T13:05:15.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.414+0000 7f2e4bfff640 1 -- 172.21.15.2:0/1031528931 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2e500052b0 con 0x55a39b447bd0 2024-03-15T13:05:15.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.414+0000 7f2e4bfff640 1 -- 172.21.15.2:0/1031528931 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e500b9330 con 0x55a39b447bd0 2024-03-15T13:05:15.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.415+0000 7f2e6ac18740 1 -- 172.21.15.2:0/1031528931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b447bd0 msgr2=0x55a39b450270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.415+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/1031528931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b447bd0 0x55a39b450270 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e500ed630 tx=0x7f2e50003730 comp rx=0 tx=0).stop 2024-03-15T13:05:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.415+0000 7f2e6ac18740 1 -- 172.21.15.2:0/1031528931 shutdown_connections 2024-03-15T13:05:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.415+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/1031528931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b4507b0 0x55a39b452ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.415+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/1031528931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b447bd0 0x55a39b450270 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.415+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/1031528931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4472b0 0x55a39b447690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.415+0000 7f2e6ac18740 1 -- 172.21.15.2:0/1031528931 >> 172.21.15.2:0/1031528931 conn(0x55a39b441990 msgr2=0x55a39b441d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.415+0000 7f2e6ac18740 1 -- 172.21.15.2:0/1031528931 shutdown_connections 2024-03-15T13:05:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.415+0000 7f2e6ac18740 1 -- 172.21.15.2:0/1031528931 wait complete. 2024-03-15T13:05:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.415+0000 7f2e6ac18740 1 -- start start 2024-03-15T13:05:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.415+0000 7f2e6ac18740 1 Processor -- start 2024-03-15T13:05:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e6ac18740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b4472b0 0x55a39b450250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e6ac18740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b447bd0 0x55a39b44d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e6ac18740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b4507b0 0x55a39b44dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e6ac18740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39b082430 con 0x55a39b447bd0 2024-03-15T13:05:15.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e6ac18740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39b03ee00 con 0x55a39b4472b0 2024-03-15T13:05:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e6ac18740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39b333fd0 con 0x55a39b4507b0 2024-03-15T13:05:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e6ac18740 10 client.0 _ll_register_callbacks cb 0x55a39b389fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e654f9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b447bd0 0x55a39b44d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e654f9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b447bd0 0x55a39b44d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57802/0 (socket says 172.21.15.2:57802) 2024-03-15T13:05:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e65cfa640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b4472b0 0x55a39b450250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e654f9640 1 -- 172.21.15.2:0/940192092 learned_addr learned my addr 172.21.15.2:0/940192092 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e664fb640 1 --2- 172.21.15.2:0/940192092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b4507b0 0x55a39b44dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e654f9640 1 -- 172.21.15.2:0/940192092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b4507b0 msgr2=0x55a39b44dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e654f9640 1 --2- 172.21.15.2:0/940192092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b4507b0 0x55a39b44dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e654f9640 1 -- 172.21.15.2:0/940192092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b4472b0 msgr2=0x55a39b450250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e654f9640 1 --2- 172.21.15.2:0/940192092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b4472b0 0x55a39b450250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e654f9640 1 -- 172.21.15.2:0/940192092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e601a6980 con 0x55a39b447bd0 2024-03-15T13:05:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e654f9640 1 --2- 172.21.15.2:0/940192092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b447bd0 0x55a39b44d5e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f2e50005ec0 tx=0x7f2e50007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53144155e14b40ad server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:15.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.416+0000 7f2e497fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:15.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.417+0000 7f2e497fa640 1 -- 172.21.15.2:0/940192092 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e500b92d0 con 0x55a39b447bd0 2024-03-15T13:05:15.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.417+0000 7f2e497fa640 1 -- 172.21.15.2:0/940192092 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f2e500bde40 con 0x55a39b447bd0 2024-03-15T13:05:15.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.417+0000 7f2e6ac18740 1 -- 172.21.15.2:0/940192092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a39b0ae7e0 con 0x55a39b447bd0 2024-03-15T13:05:15.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.417+0000 7f2e497fa640 1 -- 172.21.15.2:0/940192092 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e500c52b0 con 0x55a39b447bd0 2024-03-15T13:05:15.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.417+0000 7f2e6ac18740 10 client.17487 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:15.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.417+0000 7f2e6ac18740 1 -- 172.21.15.2:0/940192092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55a39b3ad610 con 0x55a39b447bd0 2024-03-15T13:05:15.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.417+0000 7f2e497fa640 1 -- 172.21.15.2:0/940192092 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55a39b0ae7e0 con 0x55a39b447bd0 2024-03-15T13:05:15.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.417+0000 7f2e497fa640 1 -- 172.21.15.2:0/940192092 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2e5003c070 con 0x55a39b447bd0 2024-03-15T13:05:15.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.417+0000 7f2e6ac18740 10 client.17487 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:15.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.417+0000 7f2e6ac18740 10 client.17487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:15.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.417+0000 7f2e6ac18740 20 client.17487 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:15.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.417+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/940192092 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a39b460310 0x55a39b462730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.417+0000 7f2e6ac18740 4 client.17487 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:15.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.417+0000 7f2e6ac18740 1 -- 172.21.15.2:0/940192092 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a39b0eb720 con 0x55a39b460310 2024-03-15T13:05:15.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.417+0000 7f2e65cfa640 1 --2- 172.21.15.2:0/940192092 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a39b460310 0x55a39b462730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.418+0000 7f2e65cfa640 1 --2- 172.21.15.2:0/940192092 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a39b460310 0x55a39b462730 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2634ca768eddd43e server_cookie=497ba1230132cf17 in_seq=0 out_seq=0 2024-03-15T13:05:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.418+0000 7f2e497fa640 10 client.17487 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.419+0000 7f2e497fa640 1 -- 172.21.15.2:0/940192092 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a39b0eb720 con 0x55a39b460310 2024-03-15T13:05:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.419+0000 7f2e497fa640 10 client.17487 handle_command_reply: tid=0 2024-03-15T13:05:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.419+0000 7f2e6ac18740 1 client.17487 shutdown 2024-03-15T13:05:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.419+0000 7f2e6ac18740 1 -- 172.21.15.2:0/940192092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b447bd0 msgr2=0x55a39b44d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.419+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/940192092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b447bd0 0x55a39b44d5e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f2e50005ec0 tx=0x7f2e50007740 comp rx=0 tx=0).stop 2024-03-15T13:05:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.419+0000 7f2e6ac18740 1 -- 172.21.15.2:0/940192092 shutdown_connections 2024-03-15T13:05:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.419+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/940192092 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a39b460310 0x55a39b462730 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.419+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/940192092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b4507b0 0x55a39b44dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.419+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/940192092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b447bd0 0x55a39b44d5e0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.419+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/940192092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b4472b0 0x55a39b450250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.419+0000 7f2e497fa640 0 client.17487 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.419+0000 7f2e6ac18740 1 -- 172.21.15.2:0/940192092 >> 172.21.15.2:0/940192092 conn(0x55a39b441990 msgr2=0x55a39b443730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:15.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.419+0000 7f2e6ac18740 1 -- 172.21.15.2:0/940192092 shutdown_connections 2024-03-15T13:05:15.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.419+0000 7f2e6ac18740 1 -- 172.21.15.2:0/940192092 wait complete. 2024-03-15T13:05:15.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.419+0000 7f2e6ac18740 20 client.17487 trim_cache size 0 max 16384 2024-03-15T13:05:15.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.435+0000 7f2e6ac18740 1 Processor -- start 2024-03-15T13:05:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.435+0000 7f2e6ac18740 1 -- start start 2024-03-15T13:05:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.435+0000 7f2e6ac18740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b4472b0 0x55a39b44c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.435+0000 7f2e6ac18740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b447bd0 0x55a39b44fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.435+0000 7f2e6ac18740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4507b0 0x55a39b450250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.435+0000 7f2e6ac18740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39b4429e0 con 0x55a39b4507b0 2024-03-15T13:05:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.435+0000 7f2e6ac18740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39b4465d0 con 0x55a39b4472b0 2024-03-15T13:05:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.435+0000 7f2e6ac18740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39b446f10 con 0x55a39b447bd0 2024-03-15T13:05:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.435+0000 7f2e664fb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4507b0 0x55a39b450250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.435+0000 7f2e664fb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4507b0 0x55a39b450250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57804/0 (socket says 172.21.15.2:57804) 2024-03-15T13:05:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.435+0000 7f2e664fb640 1 -- 172.21.15.2:0/2237561146 learned_addr learned my addr 172.21.15.2:0/2237561146 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.435+0000 7f2e65cfa640 1 --2- 172.21.15.2:0/2237561146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b4472b0 0x55a39b44c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.436+0000 7f2e654f9640 1 --2- 172.21.15.2:0/2237561146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b447bd0 0x55a39b44fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.436+0000 7f2e664fb640 1 -- 172.21.15.2:0/2237561146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b447bd0 msgr2=0x55a39b44fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.436+0000 7f2e664fb640 1 --2- 172.21.15.2:0/2237561146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b447bd0 0x55a39b44fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.436+0000 7f2e664fb640 1 -- 172.21.15.2:0/2237561146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b4472b0 msgr2=0x55a39b44c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.436+0000 7f2e664fb640 1 --2- 172.21.15.2:0/2237561146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b4472b0 0x55a39b44c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.436+0000 7f2e664fb640 1 -- 172.21.15.2:0/2237561146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a39b349ac0 con 0x55a39b4507b0 2024-03-15T13:05:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.436+0000 7f2e654f9640 1 --2- 172.21.15.2:0/2237561146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b447bd0 0x55a39b44fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:05:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.436+0000 7f2e664fb640 1 --2- 172.21.15.2:0/2237561146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4507b0 0x55a39b450250 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2e5800fb40 tx=0x7f2e58009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b018c80b158a3600 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.436+0000 7f2e4b7fe640 1 -- 172.21.15.2:0/2237561146 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e58003df0 con 0x55a39b4507b0 2024-03-15T13:05:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.436+0000 7f2e4b7fe640 1 -- 172.21.15.2:0/2237561146 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2e58025070 con 0x55a39b4507b0 2024-03-15T13:05:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.436+0000 7f2e4b7fe640 1 -- 172.21.15.2:0/2237561146 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e580108c0 con 0x55a39b4507b0 2024-03-15T13:05:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 -- 172.21.15.2:0/2237561146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4507b0 msgr2=0x55a39b450250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/2237561146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4507b0 0x55a39b450250 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2e5800fb40 tx=0x7f2e58009500 comp rx=0 tx=0).stop 2024-03-15T13:05:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 -- 172.21.15.2:0/2237561146 shutdown_connections 2024-03-15T13:05:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/2237561146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4507b0 0x55a39b450250 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/2237561146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b447bd0 0x55a39b44fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/2237561146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b4472b0 0x55a39b44c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 -- 172.21.15.2:0/2237561146 >> 172.21.15.2:0/2237561146 conn(0x55a39b441990 msgr2=0x55a39b44beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:15.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 -- 172.21.15.2:0/2237561146 shutdown_connections 2024-03-15T13:05:15.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 -- 172.21.15.2:0/2237561146 wait complete. 2024-03-15T13:05:15.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 -- start start 2024-03-15T13:05:15.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 Processor -- start 2024-03-15T13:05:15.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b4472b0 0x55a39b454020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b447bd0 0x55a39b454560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4507b0 0x55a39b4babc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39b4bb100 con 0x55a39b4507b0 2024-03-15T13:05:15.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39b4bb240 con 0x55a39b447bd0 2024-03-15T13:05:15.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39b4bb3d0 con 0x55a39b4472b0 2024-03-15T13:05:15.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.437+0000 7f2e6ac18740 10 client.0 _ll_register_callbacks cb 0x7f2e5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:15.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e664fb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4507b0 0x55a39b4babc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e664fb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4507b0 0x55a39b4babc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57818/0 (socket says 172.21.15.2:57818) 2024-03-15T13:05:15.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e664fb640 1 -- 172.21.15.2:0/736069871 learned_addr learned my addr 172.21.15.2:0/736069871 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:15.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e65cfa640 1 --2- 172.21.15.2:0/736069871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b4472b0 0x55a39b454020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e654f9640 1 --2- 172.21.15.2:0/736069871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b447bd0 0x55a39b454560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e664fb640 1 -- 172.21.15.2:0/736069871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b4472b0 msgr2=0x55a39b454020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e664fb640 1 --2- 172.21.15.2:0/736069871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b4472b0 0x55a39b454020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e664fb640 1 -- 172.21.15.2:0/736069871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b447bd0 msgr2=0x55a39b454560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e664fb640 1 --2- 172.21.15.2:0/736069871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b447bd0 0x55a39b454560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e664fb640 1 -- 172.21.15.2:0/736069871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e580122a0 con 0x55a39b4507b0 2024-03-15T13:05:15.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e664fb640 1 --2- 172.21.15.2:0/736069871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4507b0 0x55a39b4babc0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009b50 tx=0x7f2e58005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a1eba11a622c84 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:15.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e497fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:15.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e497fa640 1 -- 172.21.15.2:0/736069871 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e58010640 con 0x55a39b4507b0 2024-03-15T13:05:15.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e497fa640 1 -- 172.21.15.2:0/736069871 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f2e580107a0 con 0x55a39b4507b0 2024-03-15T13:05:15.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e6ac18740 1 -- 172.21.15.2:0/736069871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e500052b0 con 0x55a39b4507b0 2024-03-15T13:05:15.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e497fa640 1 -- 172.21.15.2:0/736069871 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e58010ab0 con 0x55a39b4507b0 2024-03-15T13:05:15.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e6ac18740 10 client.17493 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:15.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.438+0000 7f2e6ac18740 1 -- 172.21.15.2:0/736069871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55a39b36f6a0 con 0x55a39b4507b0 2024-03-15T13:05:15.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.439+0000 7f2e497fa640 1 -- 172.21.15.2:0/736069871 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2e500052b0 con 0x55a39b4507b0 2024-03-15T13:05:15.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.439+0000 7f2e497fa640 1 -- 172.21.15.2:0/736069871 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f2e58033480 con 0x55a39b4507b0 2024-03-15T13:05:15.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.439+0000 7f2e6ac18740 10 client.17493 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:15.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.439+0000 7f2e6ac18740 10 client.17493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:15.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.439+0000 7f2e6ac18740 20 client.17493 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:15.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.439+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/736069871 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a39b4bcae0 0x55a39b4bef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.439+0000 7f2e6ac18740 4 client.17493 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:05:15.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.439+0000 7f2e6ac18740 1 -- 172.21.15.2:0/736069871 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a39b13a890 con 0x55a39b4bcae0 2024-03-15T13:05:15.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.439+0000 7f2e65cfa640 1 --2- 172.21.15.2:0/736069871 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a39b4bcae0 0x55a39b4bef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.439+0000 7f2e65cfa640 1 --2- 172.21.15.2:0/736069871 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a39b4bcae0 0x55a39b4bef00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3441fe94e05b95a server_cookie=434801294b6c2d0 in_seq=0 out_seq=0 2024-03-15T13:05:15.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.439+0000 7f2e497fa640 10 client.17493 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:15.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.439+0000 7f2e497fa640 1 -- 172.21.15.2:0/736069871 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a39b13a890 con 0x55a39b4bcae0 2024-03-15T13:05:15.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.439+0000 7f2e497fa640 10 client.17493 handle_command_reply: tid=0 2024-03-15T13:05:15.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.439+0000 7f2e6ac18740 1 client.17493 shutdown 2024-03-15T13:05:15.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e6ac18740 1 -- 172.21.15.2:0/736069871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4507b0 msgr2=0x55a39b4babc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/736069871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4507b0 0x55a39b4babc0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009b50 tx=0x7f2e58005ae0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e6ac18740 1 -- 172.21.15.2:0/736069871 shutdown_connections 2024-03-15T13:05:15.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/736069871 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a39b4bcae0 0x55a39b4bef00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/736069871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a39b4507b0 0x55a39b4babc0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/736069871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a39b447bd0 0x55a39b454560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e6ac18740 1 --2- 172.21.15.2:0/736069871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a39b4472b0 0x55a39b454020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e497fa640 0 client.17493 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:15.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e6ac18740 1 -- 172.21.15.2:0/736069871 >> 172.21.15.2:0/736069871 conn(0x55a39b441990 msgr2=0x55a39b442d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:15.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e6ac18740 1 -- 172.21.15.2:0/736069871 shutdown_connections 2024-03-15T13:05:15.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e6ac18740 1 -- 172.21.15.2:0/736069871 wait complete. 2024-03-15T13:05:15.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e6ac18740 20 client.17493 trim_cache size 0 max 16384 2024-03-15T13:05:15.432 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:05:15.432 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:05:15.432 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:05:15.432 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:05:15.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e67f5e640 1 -- 172.21.15.2:0/3936614325 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f2e4c0a1ba0 msgr2=0x7f2e4c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e67f5e640 1 --2- 172.21.15.2:0/3936614325 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f2e4c0a1ba0 0x7f2e4c0a4060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009fd0 tx=0x7f2e58013b80 comp rx=0 tx=0).stop 2024-03-15T13:05:15.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e67f5e640 1 -- 172.21.15.2:0/3936614325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e6006a4e0 msgr2=0x7f2e6019fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e67f5e640 1 --2- 172.21.15.2:0/3936614325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e6006a4e0 0x7f2e6019fe20 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009900 tx=0x7f2e5002cac0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e67f5e640 1 -- 172.21.15.2:0/3936614325 shutdown_connections 2024-03-15T13:05:15.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e67f5e640 1 --2- 172.21.15.2:0/3936614325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e60107600 0x7f2e60199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e67f5e640 1 --2- 172.21.15.2:0/3936614325 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f2e4c0a1ba0 0x7f2e4c0a4060 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e67f5e640 1 --2- 172.21.15.2:0/3936614325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e6006a4e0 0x7f2e6019fe20 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e67f5e640 1 --2- 172.21.15.2:0/3936614325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e60069b20 0x7f2e6019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e67f5e640 1 -- 172.21.15.2:0/3936614325 >> 172.21.15.2:0/3936614325 conn(0x7f2e60077190 msgr2=0x7f2e6010c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:15.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e67f5e640 1 -- 172.21.15.2:0/3936614325 shutdown_connections 2024-03-15T13:05:15.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.440+0000 7f2e67f5e640 1 -- 172.21.15.2:0/3936614325 wait complete. 2024-03-15T13:05:15.436 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:05:15.437 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:05:15.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 -- 172.21.15.2:0/1439367598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3bc106490 msgr2=0x7fd3bc106870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 --2- 172.21.15.2:0/1439367598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3bc106490 0x7fd3bc106870 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009930 tx=0x7fd3b002bc30 comp rx=0 tx=0).stop 2024-03-15T13:05:15.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 -- 172.21.15.2:0/1439367598 shutdown_connections 2024-03-15T13:05:15.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 --2- 172.21.15.2:0/1439367598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3bc100db0 0x7fd3bc10d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 --2- 172.21.15.2:0/1439367598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3bc1003f0 0x7fd3bc100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 --2- 172.21.15.2:0/1439367598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3bc106490 0x7fd3bc106870 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 -- 172.21.15.2:0/1439367598 >> 172.21.15.2:0/1439367598 conn(0x7fd3bc0774f0 msgr2=0x7fd3bc077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:15.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 -- 172.21.15.2:0/1439367598 shutdown_connections 2024-03-15T13:05:15.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 -- 172.21.15.2:0/1439367598 wait complete. 2024-03-15T13:05:15.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 Processor -- start 2024-03-15T13:05:15.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 -- start start 2024-03-15T13:05:15.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3bc1003f0 0x7fd3bc19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3bc100db0 0x7fd3bc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3bc106490 0x7fd3bc196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc196f60 con 0x7fd3bc100db0 2024-03-15T13:05:15.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc1970d0 con 0x7fd3bc1003f0 2024-03-15T13:05:15.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3c2eae640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc197290 con 0x7fd3bc106490 2024-03-15T13:05:15.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3bbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3bc100db0 0x7fd3bc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.555+0000 7fd3bbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3bc100db0 0x7fd3bc19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57844/0 (socket says 172.21.15.2:57844) 2024-03-15T13:05:15.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.556+0000 7fd3bbfff640 1 -- 172.21.15.2:0/2432391206 learned_addr learned my addr 172.21.15.2:0/2432391206 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:15.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.556+0000 7fd3c144b640 1 --2- 172.21.15.2:0/2432391206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3bc106490 0x7fd3bc196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.556+0000 7fd3c0c4a640 1 --2- 172.21.15.2:0/2432391206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3bc1003f0 0x7fd3bc19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.556+0000 7fd3bbfff640 1 -- 172.21.15.2:0/2432391206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3bc106490 msgr2=0x7fd3bc196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.556+0000 7fd3bbfff640 1 --2- 172.21.15.2:0/2432391206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3bc106490 0x7fd3bc196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.556+0000 7fd3bbfff640 1 -- 172.21.15.2:0/2432391206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3bc1003f0 msgr2=0x7fd3bc19c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.556+0000 7fd3bbfff640 1 --2- 172.21.15.2:0/2432391206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3bc1003f0 0x7fd3bc19c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.556+0000 7fd3bbfff640 1 -- 172.21.15.2:0/2432391206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b4009840 con 0x7fd3bc100db0 2024-03-15T13:05:15.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.556+0000 7fd3bbfff640 1 --2- 172.21.15.2:0/2432391206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3bc100db0 0x7fd3bc19cca0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800e9f0 tx=0x7fd3a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:15.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.556+0000 7fd3b9ffb640 1 -- 172.21.15.2:0/2432391206 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8009700 con 0x7fd3bc100db0 2024-03-15T13:05:15.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.556+0000 7fd3b9ffb640 1 -- 172.21.15.2:0/2432391206 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd3a800ccc0 con 0x7fd3bc100db0 2024-03-15T13:05:15.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.556+0000 7fd3c2eae640 1 -- 172.21.15.2:0/2432391206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b0009560 con 0x7fd3bc100db0 2024-03-15T13:05:15.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.556+0000 7fd3b9ffb640 1 -- 172.21.15.2:0/2432391206 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8018820 con 0x7fd3bc100db0 2024-03-15T13:05:15.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.556+0000 7fd3c2eae640 1 -- 172.21.15.2:0/2432391206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3bc1a3620 con 0x7fd3bc100db0 2024-03-15T13:05:15.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.557+0000 7fd3c2eae640 1 -- 172.21.15.2:0/2432391206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3bc0682e0 con 0x7fd3bc100db0 2024-03-15T13:05:15.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.558+0000 7fd3b9ffb640 1 -- 172.21.15.2:0/2432391206 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fd3a8010430 con 0x7fd3bc100db0 2024-03-15T13:05:15.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.559+0000 7fd3b9ffb640 1 --2- 172.21.15.2:0/2432391206 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd3ac0a1ba0 0x7fd3ac0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.559+0000 7fd3c0c4a640 1 --2- 172.21.15.2:0/2432391206 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd3ac0a1ba0 0x7fd3ac0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.559+0000 7fd3b9ffb640 1 -- 172.21.15.2:0/2432391206 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fd3a8014070 con 0x7fd3bc100db0 2024-03-15T13:05:15.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.560+0000 7fd3c0c4a640 1 --2- 172.21.15.2:0/2432391206 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd3ac0a1ba0 0x7fd3ac0a4060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0000c00 tx=0x7fd3b0005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:15.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.564+0000 7fd3b9ffb640 1 -- 172.21.15.2:0/2432391206 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd3a8053280 con 0x7fd3bc100db0 2024-03-15T13:05:15.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.786+0000 7fd3c2eae640 1 -- 172.21.15.2:0/2432391206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3bc108960 con 0x7fd3bc100db0 2024-03-15T13:05:15.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.787+0000 7fd3b9ffb640 1 -- 172.21.15.2:0/2432391206 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+11453 (secure 0 0 0) 0x7fd3a8096ad0 con 0x7fd3bc100db0 2024-03-15T13:05:15.772 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:05:15.773 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:05:12.138845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24500,24464,24460,24434,24539]},"id":1}]} 2024-03-15T13:05:15.773 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 50 2024-03-15T13:05:15.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.789+0000 7fd3c2eae640 1 -- 172.21.15.2:0/2432391206 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd3ac0a1ba0 msgr2=0x7fd3ac0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.789+0000 7fd3c2eae640 1 --2- 172.21.15.2:0/2432391206 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd3ac0a1ba0 0x7fd3ac0a4060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0000c00 tx=0x7fd3b0005fd0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.790+0000 7fd3c2eae640 1 -- 172.21.15.2:0/2432391206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3bc100db0 msgr2=0x7fd3bc19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.790+0000 7fd3c2eae640 1 --2- 172.21.15.2:0/2432391206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3bc100db0 0x7fd3bc19cca0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800e9f0 tx=0x7fd3a800edf0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.790+0000 7fd3c2eae640 1 -- 172.21.15.2:0/2432391206 shutdown_connections 2024-03-15T13:05:15.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.790+0000 7fd3c2eae640 1 --2- 172.21.15.2:0/2432391206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3bc106490 0x7fd3bc196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.790+0000 7fd3c2eae640 1 --2- 172.21.15.2:0/2432391206 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd3ac0a1ba0 0x7fd3ac0a4060 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.790+0000 7fd3c2eae640 1 --2- 172.21.15.2:0/2432391206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd3bc100db0 0x7fd3bc19cca0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.790+0000 7fd3c2eae640 1 --2- 172.21.15.2:0/2432391206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3bc1003f0 0x7fd3bc19c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.790+0000 7fd3c2eae640 1 -- 172.21.15.2:0/2432391206 >> 172.21.15.2:0/2432391206 conn(0x7fd3bc0774f0 msgr2=0x7fd3bc102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:15.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.790+0000 7fd3c2eae640 1 -- 172.21.15.2:0/2432391206 shutdown_connections 2024-03-15T13:05:15.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.790+0000 7fd3c2eae640 1 -- 172.21.15.2:0/2432391206 wait complete. 2024-03-15T13:05:15.786 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:05:15.786 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:05:15.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.906+0000 7f04ab9b5640 1 -- 172.21.15.2:0/2792044219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a4100f80 msgr2=0x7f04a4104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.906+0000 7f04ab9b5640 1 --2- 172.21.15.2:0/2792044219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a4100f80 0x7f04a4104fc0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0494009930 tx=0x7f049402bc30 comp rx=0 tx=0).stop 2024-03-15T13:05:15.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.906+0000 7f04ab9b5640 1 -- 172.21.15.2:0/2792044219 shutdown_connections 2024-03-15T13:05:15.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.906+0000 7f04ab9b5640 1 --2- 172.21.15.2:0/2792044219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04a41056a0 0x7f04a410d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.906+0000 7f04ab9b5640 1 --2- 172.21.15.2:0/2792044219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a4100f80 0x7f04a4104fc0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.906+0000 7f04ab9b5640 1 --2- 172.21.15.2:0/2792044219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04a41005d0 0x7f04a41009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.906+0000 7f04ab9b5640 1 -- 172.21.15.2:0/2792044219 >> 172.21.15.2:0/2792044219 conn(0x7f04a40774f0 msgr2=0x7f04a4077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:15.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.906+0000 7f04ab9b5640 1 -- 172.21.15.2:0/2792044219 shutdown_connections 2024-03-15T13:05:15.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.906+0000 7f04ab9b5640 1 -- 172.21.15.2:0/2792044219 wait complete. 2024-03-15T13:05:15.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.906+0000 7f04ab9b5640 1 Processor -- start 2024-03-15T13:05:15.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.906+0000 7f04ab9b5640 1 -- start start 2024-03-15T13:05:15.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04ab9b5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04a41005d0 0x7f04a419d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04ab9b5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a41056a0 0x7f04a419dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04ab9b5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04a41979f0 0x7f04a4197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04ab9b5640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a41983e0 con 0x7f04a41056a0 2024-03-15T13:05:15.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04ab9b5640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a4198550 con 0x7f04a41979f0 2024-03-15T13:05:15.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04ab9b5640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a4198710 con 0x7f04a41005d0 2024-03-15T13:05:15.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04a8f50640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a41056a0 0x7f04a419dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04a8f50640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a41056a0 0x7f04a419dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57870/0 (socket says 172.21.15.2:57870) 2024-03-15T13:05:15.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04a8f50640 1 -- 172.21.15.2:0/2414247707 learned_addr learned my addr 172.21.15.2:0/2414247707 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:15.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04a9f52640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04a41979f0 0x7f04a4197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04a9751640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04a41005d0 0x7f04a419d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04a8f50640 1 -- 172.21.15.2:0/2414247707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04a41005d0 msgr2=0x7f04a419d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04a8f50640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04a41005d0 0x7f04a419d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04a8f50640 1 -- 172.21.15.2:0/2414247707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04a41979f0 msgr2=0x7f04a4197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04a8f50640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04a41979f0 0x7f04a4197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04a8f50640 1 -- 172.21.15.2:0/2414247707 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0494009560 con 0x7f04a41056a0 2024-03-15T13:05:15.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04a9f52640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04a41979f0 0x7f04a4197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:05:15.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04a9751640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04a41005d0 0x7f04a419d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:05:15.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04a8f50640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a41056a0 0x7f04a419dd80 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0494005ab0 tx=0x7f049402ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:15.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f049a7fc640 1 -- 172.21.15.2:0/2414247707 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049402cc30 con 0x7f04a41056a0 2024-03-15T13:05:15.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f049a7fc640 1 -- 172.21.15.2:0/2414247707 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0494005430 con 0x7f04a41056a0 2024-03-15T13:05:15.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04ab9b5640 1 -- 172.21.15.2:0/2414247707 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a41a4700 con 0x7f04a41056a0 2024-03-15T13:05:15.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f049a7fc640 1 -- 172.21.15.2:0/2414247707 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494003b80 con 0x7f04a41056a0 2024-03-15T13:05:15.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.907+0000 7f04ab9b5640 1 -- 172.21.15.2:0/2414247707 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a41a4bb0 con 0x7f04a41056a0 2024-03-15T13:05:15.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.910+0000 7f049a7fc640 1 -- 172.21.15.2:0/2414247707 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f04940055a0 con 0x7f04a41056a0 2024-03-15T13:05:15.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.910+0000 7f04ae593740 1 Processor -- start 2024-03-15T13:05:15.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.910+0000 7f04ae593740 1 -- start start 2024-03-15T13:05:15.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04ae593740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0060 0x56248c4f0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04ae593740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0a10 0x56248c4fd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04ae593740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4fdb60 0x56248c4fff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04ae593740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248c4ab960 con 0x56248c4f0060 2024-03-15T13:05:15.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04ae593740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248c45f310 con 0x56248c4fdb60 2024-03-15T13:05:15.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04ae593740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248c45c840 con 0x56248c4f0a10 2024-03-15T13:05:15.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04a9f52640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0060 0x56248c4f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04a8f50640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4fdb60 0x56248c4fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04a9f52640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0060 0x56248c4f0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57882/0 (socket says 172.21.15.2:57882) 2024-03-15T13:05:15.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04a9f52640 1 -- 172.21.15.2:0/3568343258 learned_addr learned my addr 172.21.15.2:0/3568343258 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:15.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04a9751640 1 --2- 172.21.15.2:0/3568343258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0a10 0x56248c4fd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f049a7fc640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f04900a1ba0 0x7f04900a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04a9f52640 1 -- 172.21.15.2:0/3568343258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0a10 msgr2=0x56248c4fd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04a9f52640 1 --2- 172.21.15.2:0/3568343258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0a10 0x56248c4fd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04a9f52640 1 -- 172.21.15.2:0/3568343258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4fdb60 msgr2=0x56248c4fff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04a9f52640 1 --2- 172.21.15.2:0/3568343258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4fdb60 0x56248c4fff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04a9f52640 1 -- 172.21.15.2:0/3568343258 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56248c41a060 con 0x56248c4f0060 2024-03-15T13:05:15.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04a9751640 1 --2- 172.21.15.2:0/3568343258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0a10 0x56248c4fd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:15.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f049a7fc640 1 -- 172.21.15.2:0/2414247707 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f04940efae0 con 0x7f04a41056a0 2024-03-15T13:05:15.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04a9f52640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f04900a1ba0 0x7f04900a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f04a9f52640 1 --2- 172.21.15.2:0/3568343258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0060 0x56248c4f0440 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f049c009fd0 tx=0x7f049c010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc69e62b2799596 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:15.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f048f7fe640 1 -- 172.21.15.2:0/3568343258 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c00e650 con 0x56248c4f0060 2024-03-15T13:05:15.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.911+0000 7f048f7fe640 1 -- 172.21.15.2:0/3568343258 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f049c00ee80 con 0x56248c4f0060 2024-03-15T13:05:15.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.912+0000 7f04a9f52640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f04900a1ba0 0x7f04900a4060 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f049c021000 tx=0x7f049c01f000 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:15.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.912+0000 7f048f7fe640 1 -- 172.21.15.2:0/3568343258 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c011e50 con 0x56248c4f0060 2024-03-15T13:05:15.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.912+0000 7f04ae593740 1 -- 172.21.15.2:0/3568343258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0060 msgr2=0x56248c4f0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.912+0000 7f04ae593740 1 --2- 172.21.15.2:0/3568343258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0060 0x56248c4f0440 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f049c009fd0 tx=0x7f049c010a00 comp rx=0 tx=0).stop 2024-03-15T13:05:15.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.912+0000 7f04ae593740 1 -- 172.21.15.2:0/3568343258 shutdown_connections 2024-03-15T13:05:15.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.912+0000 7f04ae593740 1 --2- 172.21.15.2:0/3568343258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4fdb60 0x56248c4fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.912+0000 7f04ae593740 1 --2- 172.21.15.2:0/3568343258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0a10 0x56248c4fd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.912+0000 7f04ae593740 1 --2- 172.21.15.2:0/3568343258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0060 0x56248c4f0440 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.912+0000 7f04ae593740 1 -- 172.21.15.2:0/3568343258 >> 172.21.15.2:0/3568343258 conn(0x56248c4ea980 msgr2=0x56248c4ead60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:15.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.912+0000 7f04ae593740 1 -- 172.21.15.2:0/3568343258 shutdown_connections 2024-03-15T13:05:15.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.912+0000 7f04ae593740 1 -- 172.21.15.2:0/3568343258 wait complete. 2024-03-15T13:05:15.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04ae593740 1 -- start start 2024-03-15T13:05:15.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04ae593740 1 Processor -- start 2024-03-15T13:05:15.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04ae593740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0060 0x56248c4fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04ae593740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4f0a10 0x56248c4fa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04ae593740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4fdb60 0x56248c4fae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04ae593740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248c458020 con 0x56248c4fdb60 2024-03-15T13:05:15.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04ae593740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248c12b430 con 0x56248c4f0a10 2024-03-15T13:05:15.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04ae593740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248c0e7e00 con 0x56248c4f0060 2024-03-15T13:05:15.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04ae593740 10 client.0 _ll_register_callbacks cb 0x56248c3ffcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:15.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04a9f52640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4fdb60 0x56248c4fae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04a9f52640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4fdb60 0x56248c4fae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57894/0 (socket says 172.21.15.2:57894) 2024-03-15T13:05:15.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04a9f52640 1 -- 172.21.15.2:0/4073236632 learned_addr learned my addr 172.21.15.2:0/4073236632 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:15.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04a9751640 1 --2- 172.21.15.2:0/4073236632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0060 0x56248c4fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04a8f50640 1 --2- 172.21.15.2:0/4073236632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4f0a10 0x56248c4fa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04a9f52640 1 -- 172.21.15.2:0/4073236632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0060 msgr2=0x56248c4fd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04a9f52640 1 --2- 172.21.15.2:0/4073236632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0060 0x56248c4fd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04a9f52640 1 -- 172.21.15.2:0/4073236632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4f0a10 msgr2=0x56248c4fa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04a9f52640 1 --2- 172.21.15.2:0/4073236632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4f0a10 0x56248c4fa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.913+0000 7f04a9f52640 1 -- 172.21.15.2:0/4073236632 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049c009c80 con 0x56248c4fdb60 2024-03-15T13:05:15.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f04a9f52640 1 --2- 172.21.15.2:0/4073236632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4fdb60 0x56248c4fae00 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f049c009e40 tx=0x7f049c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27458a2defd026e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:15.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f048cff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:15.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f048cff9640 1 -- 172.21.15.2:0/4073236632 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c011640 con 0x56248c4fdb60 2024-03-15T13:05:15.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f048cff9640 1 -- 172.21.15.2:0/4073236632 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f049c011e40 con 0x56248c4fdb60 2024-03-15T13:05:15.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f04ae593740 1 -- 172.21.15.2:0/4073236632 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56248c194720 con 0x56248c4fdb60 2024-03-15T13:05:15.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f048cff9640 1 -- 172.21.15.2:0/4073236632 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f049c0118f0 con 0x56248c4fdb60 2024-03-15T13:05:15.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f04ae593740 10 client.17511 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:15.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f04ae593740 1 -- 172.21.15.2:0/4073236632 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x56248c4566a0 con 0x56248c4fdb60 2024-03-15T13:05:15.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f048cff9640 1 -- 172.21.15.2:0/4073236632 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x56248c194720 con 0x56248c4fdb60 2024-03-15T13:05:15.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f048cff9640 1 -- 172.21.15.2:0/4073236632 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f049c02e070 con 0x56248c4fdb60 2024-03-15T13:05:15.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f04ae593740 10 client.17511 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:15.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f04ae593740 10 client.17511 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:05:15.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f04ae593740 20 client.17511 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:15.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f04ae593740 1 --2- 172.21.15.2:0/4073236632 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x56248c5120c0 0x56248c5144b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f04ae593740 4 client.17511 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:15.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.914+0000 7f04ae593740 1 -- 172.21.15.2:0/4073236632 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56248c1e3890 con 0x56248c5120c0 2024-03-15T13:05:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.915+0000 7f04a9751640 1 --2- 172.21.15.2:0/4073236632 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x56248c5120c0 0x56248c5144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.915+0000 7f048cff9640 10 client.17511 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:05:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.915+0000 7f04a9751640 1 --2- 172.21.15.2:0/4073236632 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x56248c5120c0 0x56248c5144b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f6b7eee9f0709cd9 server_cookie=c7b7bb9fc02237f4 in_seq=0 out_seq=0 2024-03-15T13:05:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.915+0000 7f048cff9640 1 -- 172.21.15.2:0/4073236632 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56248c1e3890 con 0x56248c5120c0 2024-03-15T13:05:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.915+0000 7f048cff9640 10 client.17511 handle_command_reply: tid=0 2024-03-15T13:05:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.916+0000 7f04ae593740 1 client.17511 shutdown 2024-03-15T13:05:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.916+0000 7f04ae593740 1 -- 172.21.15.2:0/4073236632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4fdb60 msgr2=0x56248c4fae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.916+0000 7f04ae593740 1 --2- 172.21.15.2:0/4073236632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4fdb60 0x56248c4fae00 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f049c009e40 tx=0x7f049c005c70 comp rx=0 tx=0).stop 2024-03-15T13:05:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.916+0000 7f04ae593740 1 -- 172.21.15.2:0/4073236632 shutdown_connections 2024-03-15T13:05:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.916+0000 7f04ae593740 1 --2- 172.21.15.2:0/4073236632 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x56248c5120c0 0x56248c5144b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.916+0000 7f048cff9640 0 client.17511 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:05:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.916+0000 7f04ae593740 1 --2- 172.21.15.2:0/4073236632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4fdb60 0x56248c4fae00 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.916+0000 7f04ae593740 1 --2- 172.21.15.2:0/4073236632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4f0a10 0x56248c4fa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.916+0000 7f04ae593740 1 --2- 172.21.15.2:0/4073236632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0060 0x56248c4fd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.916+0000 7f04ae593740 1 -- 172.21.15.2:0/4073236632 >> 172.21.15.2:0/4073236632 conn(0x56248c4ea980 msgr2=0x56248c4ec6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:15.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.916+0000 7f04ae593740 1 -- 172.21.15.2:0/4073236632 shutdown_connections 2024-03-15T13:05:15.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.916+0000 7f04ae593740 1 -- 172.21.15.2:0/4073236632 wait complete. 2024-03-15T13:05:15.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.916+0000 7f04ae593740 20 client.17511 trim_cache size 0 max 16384 2024-03-15T13:05:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.928+0000 7f04ae593740 1 Processor -- start 2024-03-15T13:05:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.928+0000 7f04ae593740 1 -- start start 2024-03-15T13:05:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.928+0000 7f04ae593740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0060 0x56248c4f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.928+0000 7f04ae593740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0a10 0x56248c4fcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.928+0000 7f04ae593740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4fdb60 0x56248c4f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.928+0000 7f04ae593740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248c4e6120 con 0x56248c4f0a10 2024-03-15T13:05:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.928+0000 7f04ae593740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248c3dd130 con 0x56248c4fdb60 2024-03-15T13:05:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.928+0000 7f04ae593740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248c4ef5b0 con 0x56248c4f0060 2024-03-15T13:05:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.929+0000 7f04a8f50640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0a10 0x56248c4fcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.929+0000 7f04a9751640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0060 0x56248c4f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.929+0000 7f04a8f50640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0a10 0x56248c4fcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57908/0 (socket says 172.21.15.2:57908) 2024-03-15T13:05:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.929+0000 7f04a8f50640 1 -- 172.21.15.2:0/1404732221 learned_addr learned my addr 172.21.15.2:0/1404732221 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.929+0000 7f04a9f52640 1 --2- 172.21.15.2:0/1404732221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4fdb60 0x56248c4f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.929+0000 7f04a8f50640 1 -- 172.21.15.2:0/1404732221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0060 msgr2=0x56248c4f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.929+0000 7f04a8f50640 1 --2- 172.21.15.2:0/1404732221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0060 0x56248c4f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.929+0000 7f04a8f50640 1 -- 172.21.15.2:0/1404732221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4fdb60 msgr2=0x56248c4f7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.929+0000 7f04a8f50640 1 --2- 172.21.15.2:0/1404732221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4fdb60 0x56248c4f7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.929+0000 7f04a8f50640 1 -- 172.21.15.2:0/1404732221 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56248c3f2c20 con 0x56248c4f0a10 2024-03-15T13:05:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.929+0000 7f04a8f50640 1 --2- 172.21.15.2:0/1404732221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0a10 0x56248c4fcff0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0494006010 tx=0x7f04940a7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe32a06762ce255 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.929+0000 7f048effd640 1 -- 172.21.15.2:0/1404732221 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04940bc010 con 0x56248c4f0a10 2024-03-15T13:05:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.929+0000 7f048effd640 1 -- 172.21.15.2:0/1404732221 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f04940bbe30 con 0x56248c4f0a10 2024-03-15T13:05:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.929+0000 7f048effd640 1 -- 172.21.15.2:0/1404732221 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049408fc80 con 0x56248c4f0a10 2024-03-15T13:05:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 -- 172.21.15.2:0/1404732221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0a10 msgr2=0x56248c4fcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 --2- 172.21.15.2:0/1404732221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0a10 0x56248c4fcff0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0494006010 tx=0x7f04940a7040 comp rx=0 tx=0).stop 2024-03-15T13:05:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 -- 172.21.15.2:0/1404732221 shutdown_connections 2024-03-15T13:05:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 --2- 172.21.15.2:0/1404732221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4fdb60 0x56248c4f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 --2- 172.21.15.2:0/1404732221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0a10 0x56248c4fcff0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 --2- 172.21.15.2:0/1404732221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0060 0x56248c4f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 -- 172.21.15.2:0/1404732221 >> 172.21.15.2:0/1404732221 conn(0x56248c4ea980 msgr2=0x56248c4e9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 -- 172.21.15.2:0/1404732221 shutdown_connections 2024-03-15T13:05:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 -- 172.21.15.2:0/1404732221 wait complete. 2024-03-15T13:05:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 -- start start 2024-03-15T13:05:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 Processor -- start 2024-03-15T13:05:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0060 0x56248c50a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0a10 0x56248c50ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4fdb60 0x56248c56c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248c50b420 con 0x56248c4f0060 2024-03-15T13:05:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248c56ce50 con 0x56248c4fdb60 2024-03-15T13:05:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.930+0000 7f04ae593740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248c56cf90 con 0x56248c4f0a10 2024-03-15T13:05:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f04a9751640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0060 0x56248c50a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f04ae593740 10 client.0 _ll_register_callbacks cb 0x7f04940b85a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f04a9751640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0060 0x56248c50a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57924/0 (socket says 172.21.15.2:57924) 2024-03-15T13:05:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f04a9751640 1 -- 172.21.15.2:0/4107744715 learned_addr learned my addr 172.21.15.2:0/4107744715 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f04a8f50640 1 --2- 172.21.15.2:0/4107744715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0a10 0x56248c50ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f04a9f52640 1 --2- 172.21.15.2:0/4107744715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4fdb60 0x56248c56c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f04a9751640 1 -- 172.21.15.2:0/4107744715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0a10 msgr2=0x56248c50ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f04a9751640 1 --2- 172.21.15.2:0/4107744715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0a10 0x56248c50ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f04a9751640 1 -- 172.21.15.2:0/4107744715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4fdb60 msgr2=0x56248c56c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f04a9751640 1 --2- 172.21.15.2:0/4107744715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4fdb60 0x56248c56c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f04a9751640 1 -- 172.21.15.2:0/4107744715 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a41a4bb0 con 0x56248c4f0060 2024-03-15T13:05:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f04a9751640 1 --2- 172.21.15.2:0/4107744715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0060 0x56248c50a830 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f04a4198b30 tx=0x7f04a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f051fcbf6e384a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f048cff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f048cff9640 1 -- 172.21.15.2:0/4107744715 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a0018040 con 0x56248c4f0060 2024-03-15T13:05:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f048cff9640 1 -- 172.21.15.2:0/4107744715 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f04a001e880 con 0x56248c4f0060 2024-03-15T13:05:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f04ae593740 1 -- 172.21.15.2:0/4107744715 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56248c1577e0 con 0x56248c4f0060 2024-03-15T13:05:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f048cff9640 1 -- 172.21.15.2:0/4107744715 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04a0003ee0 con 0x56248c4f0060 2024-03-15T13:05:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f04ae593740 10 client.17517 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.931+0000 7f04ae593740 1 -- 172.21.15.2:0/4107744715 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x56248c4188c0 con 0x56248c4f0060 2024-03-15T13:05:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.932+0000 7f048cff9640 1 -- 172.21.15.2:0/4107744715 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x56248c1577e0 con 0x56248c4f0060 2024-03-15T13:05:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.932+0000 7f048cff9640 1 -- 172.21.15.2:0/4107744715 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f04a0012c10 con 0x56248c4f0060 2024-03-15T13:05:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.932+0000 7f04ae593740 10 client.17517 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.932+0000 7f04ae593740 10 client.17517 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:05:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.932+0000 7f04ae593740 20 client.17517 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.932+0000 7f04ae593740 1 --2- 172.21.15.2:0/4107744715 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x56248c5120c0 0x56248c5707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.932+0000 7f04ae593740 4 client.17517 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:05:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.932+0000 7f04ae593740 1 -- 172.21.15.2:0/4107744715 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56248c103700 con 0x56248c5120c0 2024-03-15T13:05:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.932+0000 7f04a8f50640 1 --2- 172.21.15.2:0/4107744715 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x56248c5120c0 0x56248c5707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.932+0000 7f04a8f50640 1 --2- 172.21.15.2:0/4107744715 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x56248c5120c0 0x56248c5707d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d590b50bc87ce506 server_cookie=a6507787c06f9b74 in_seq=0 out_seq=0 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.932+0000 7f048cff9640 10 client.17517 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.932+0000 7f048cff9640 1 -- 172.21.15.2:0/4107744715 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56248c103700 con 0x56248c5120c0 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.932+0000 7f048cff9640 10 client.17517 handle_command_reply: tid=0 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.932+0000 7f04ae593740 1 client.17517 shutdown 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ae593740 1 -- 172.21.15.2:0/4107744715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0060 msgr2=0x56248c50a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ae593740 1 --2- 172.21.15.2:0/4107744715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0060 0x56248c50a830 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f04a4198b30 tx=0x7f04a0009e30 comp rx=0 tx=0).stop 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ae593740 1 -- 172.21.15.2:0/4107744715 shutdown_connections 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ae593740 1 --2- 172.21.15.2:0/4107744715 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x56248c5120c0 0x56248c5707d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ae593740 1 --2- 172.21.15.2:0/4107744715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56248c4fdb60 0x56248c56c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ae593740 1 --2- 172.21.15.2:0/4107744715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56248c4f0a10 0x56248c50ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ae593740 1 --2- 172.21.15.2:0/4107744715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248c4f0060 0x56248c50a830 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ae593740 1 -- 172.21.15.2:0/4107744715 >> 172.21.15.2:0/4107744715 conn(0x56248c4ea980 msgr2=0x56248c4f54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f048cff9640 0 client.17517 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ae593740 1 -- 172.21.15.2:0/4107744715 shutdown_connections 2024-03-15T13:05:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ae593740 1 -- 172.21.15.2:0/4107744715 wait complete. 2024-03-15T13:05:15.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ae593740 20 client.17517 trim_cache size 0 max 16384 2024-03-15T13:05:15.925 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:05:15.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ab9b5640 1 -- 172.21.15.2:0/2414247707 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f04900a1ba0 msgr2=0x7f04900a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ab9b5640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f04900a1ba0 0x7f04900a4060 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f049c021000 tx=0x7f049c01f000 comp rx=0 tx=0).stop 2024-03-15T13:05:15.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ab9b5640 1 -- 172.21.15.2:0/2414247707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a41056a0 msgr2=0x7f04a419dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:15.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ab9b5640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a41056a0 0x7f04a419dd80 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0494005ab0 tx=0x7f049402ca40 comp rx=0 tx=0).stop 2024-03-15T13:05:15.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ab9b5640 1 -- 172.21.15.2:0/2414247707 shutdown_connections 2024-03-15T13:05:15.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ab9b5640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04a41979f0 0x7f04a4197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ab9b5640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f04900a1ba0 0x7f04900a4060 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ab9b5640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04a41056a0 0x7f04a419dd80 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ab9b5640 1 --2- 172.21.15.2:0/2414247707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04a41005d0 0x7f04a419d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:15.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ab9b5640 1 -- 172.21.15.2:0/2414247707 >> 172.21.15.2:0/2414247707 conn(0x7f04a40774f0 msgr2=0x7f04a4068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:15.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ab9b5640 1 -- 172.21.15.2:0/2414247707 shutdown_connections 2024-03-15T13:05:15.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:15.933+0000 7f04ab9b5640 1 -- 172.21.15.2:0/2414247707 wait complete. 2024-03-15T13:05:15.929 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:05:15.929 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:05:15.952 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:15 smithi002 ceph-mon[32060]: pgmap v193: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:05:16.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.048+0000 7f8857ea7640 1 -- 172.21.15.2:0/3151209317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8850107560 msgr2=0x7f8850107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.048+0000 7f8857ea7640 1 --2- 172.21.15.2:0/3151209317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8850107560 0x7f8850107940 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8848009900 tx=0x7f884802bc30 comp rx=0 tx=0).stop 2024-03-15T13:05:16.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.048+0000 7f8857ea7640 1 -- 172.21.15.2:0/3151209317 shutdown_connections 2024-03-15T13:05:16.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.048+0000 7f8857ea7640 1 --2- 172.21.15.2:0/3151209317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8850101e60 0x7f885010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.048+0000 7f8857ea7640 1 --2- 172.21.15.2:0/3151209317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88501014a0 0x7f8850101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.048+0000 7f8857ea7640 1 --2- 172.21.15.2:0/3151209317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8850107560 0x7f8850107940 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.048+0000 7f8857ea7640 1 -- 172.21.15.2:0/3151209317 >> 172.21.15.2:0/3151209317 conn(0x7f88500788f0 msgr2=0x7f8850078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.048+0000 7f8857ea7640 1 -- 172.21.15.2:0/3151209317 shutdown_connections 2024-03-15T13:05:16.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.048+0000 7f8857ea7640 1 -- 172.21.15.2:0/3151209317 wait complete. 2024-03-15T13:05:16.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.049+0000 7f8857ea7640 1 Processor -- start 2024-03-15T13:05:16.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.049+0000 7f8857ea7640 1 -- start start 2024-03-15T13:05:16.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.049+0000 7f8857ea7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88501014a0 0x7f88500737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.049+0000 7f8857ea7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8850101e60 0x7f8850073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.049+0000 7f8857ea7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8850107560 0x7f8850074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.049+0000 7f8857ea7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88501a5e40 con 0x7f88501014a0 2024-03-15T13:05:16.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.049+0000 7f8857ea7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88501a5fb0 con 0x7f8850107560 2024-03-15T13:05:16.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.049+0000 7f8857ea7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88501a6170 con 0x7f8850101e60 2024-03-15T13:05:16.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.049+0000 7f8855c43640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88501014a0 0x7f88500737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.049+0000 7f8855c43640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88501014a0 0x7f88500737f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57948/0 (socket says 172.21.15.2:57948) 2024-03-15T13:05:16.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.049+0000 7f8855c43640 1 -- 172.21.15.2:0/1280346837 learned_addr learned my addr 172.21.15.2:0/1280346837 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.049+0000 7f8856444640 1 --2- 172.21.15.2:0/1280346837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8850107560 0x7f8850074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.049+0000 7f8855442640 1 --2- 172.21.15.2:0/1280346837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8850101e60 0x7f8850073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.050+0000 7f8855c43640 1 -- 172.21.15.2:0/1280346837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8850101e60 msgr2=0x7f8850073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.050+0000 7f8855c43640 1 --2- 172.21.15.2:0/1280346837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8850101e60 0x7f8850073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.050+0000 7f8855c43640 1 -- 172.21.15.2:0/1280346837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8850107560 msgr2=0x7f8850074270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.050+0000 7f8855c43640 1 --2- 172.21.15.2:0/1280346837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8850107560 0x7f8850074270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.050+0000 7f8855c43640 1 -- 172.21.15.2:0/1280346837 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8848009560 con 0x7f88501014a0 2024-03-15T13:05:16.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.050+0000 7f8856444640 1 --2- 172.21.15.2:0/1280346837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8850107560 0x7f8850074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:05:16.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.050+0000 7f8855c43640 1 --2- 172.21.15.2:0/1280346837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88501014a0 0x7f88500737f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8848009230 tx=0x7f884802cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.050+0000 7f884effd640 1 -- 172.21.15.2:0/1280346837 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848004250 con 0x7f88501014a0 2024-03-15T13:05:16.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.050+0000 7f884effd640 1 -- 172.21.15.2:0/1280346837 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f88480043b0 con 0x7f88501014a0 2024-03-15T13:05:16.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.050+0000 7f8857ea7640 1 -- 172.21.15.2:0/1280346837 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88501a63f0 con 0x7f88501014a0 2024-03-15T13:05:16.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.050+0000 7f884effd640 1 -- 172.21.15.2:0/1280346837 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848003bf0 con 0x7f88501014a0 2024-03-15T13:05:16.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.050+0000 7f8857ea7640 1 -- 172.21.15.2:0/1280346837 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88501a6900 con 0x7f88501014a0 2024-03-15T13:05:16.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.052+0000 7f884effd640 1 -- 172.21.15.2:0/1280346837 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f8848003d50 con 0x7f88501014a0 2024-03-15T13:05:16.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.052+0000 7f885ab66740 1 Processor -- start 2024-03-15T13:05:16.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.052+0000 7f885ab66740 1 -- start start 2024-03-15T13:05:16.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f885ab66740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3e250 0x560dc8c3e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f885ab66740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 0x560dc8c47210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f885ab66740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c47750 0x560dc8c49b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f885ab66740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8bf9960 con 0x560dc8c3e250 2024-03-15T13:05:16.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f885ab66740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8bad310 con 0x560dc8c47750 2024-03-15T13:05:16.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f885ab66740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8baa840 con 0x560dc8c3eb70 2024-03-15T13:05:16.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f8856444640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3e250 0x560dc8c3e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f8856444640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3e250 0x560dc8c3e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57964/0 (socket says 172.21.15.2:57964) 2024-03-15T13:05:16.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f8856444640 1 -- 172.21.15.2:0/629089711 learned_addr learned my addr 172.21.15.2:0/629089711 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f884effd640 1 --2- 172.21.15.2:0/1280346837 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f88380a1ba0 0x7f88380a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f8855c43640 1 --2- 172.21.15.2:0/629089711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 0x560dc8c47210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f8855442640 1 --2- 172.21.15.2:0/629089711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c47750 0x560dc8c49b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f8856444640 1 --2- 172.21.15.2:0/1280346837 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f88380a1ba0 0x7f88380a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f8855442640 1 -- 172.21.15.2:0/629089711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 msgr2=0x560dc8c47210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f8855442640 1 --2- 172.21.15.2:0/629089711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 0x560dc8c47210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f884effd640 1 -- 172.21.15.2:0/1280346837 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f88480efb10 con 0x7f88501014a0 2024-03-15T13:05:16.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f8855442640 1 -- 172.21.15.2:0/629089711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3e250 msgr2=0x560dc8c3e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f8855442640 1 --2- 172.21.15.2:0/629089711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3e250 0x560dc8c3e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f8855442640 1 -- 172.21.15.2:0/629089711 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dc8b68060 con 0x560dc8c47750 2024-03-15T13:05:16.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f8856444640 1 --2- 172.21.15.2:0/629089711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3e250 0x560dc8c3e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:16.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.053+0000 7f8855442640 1 --2- 172.21.15.2:0/629089711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c47750 0x560dc8c49b40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8844009bf0 tx=0x7f8844011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=796bb16334e8f60e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.054+0000 7f8856444640 1 --2- 172.21.15.2:0/1280346837 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f88380a1ba0 0x7f88380a4060 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8840009fd0 tx=0x7f8840004ab0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.054+0000 7f8837fff640 1 -- 172.21.15.2:0/629089711 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844004050 con 0x560dc8c47750 2024-03-15T13:05:16.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.054+0000 7f8837fff640 1 -- 172.21.15.2:0/629089711 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8844005020 con 0x560dc8c47750 2024-03-15T13:05:16.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.054+0000 7f8837fff640 1 -- 172.21.15.2:0/629089711 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844004cc0 con 0x560dc8c47750 2024-03-15T13:05:16.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.054+0000 7f885ab66740 1 -- 172.21.15.2:0/629089711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c47750 msgr2=0x560dc8c49b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.054+0000 7f885ab66740 1 --2- 172.21.15.2:0/629089711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c47750 0x560dc8c49b40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8844009bf0 tx=0x7f8844011810 comp rx=0 tx=0).stop 2024-03-15T13:05:16.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.054+0000 7f885ab66740 1 -- 172.21.15.2:0/629089711 shutdown_connections 2024-03-15T13:05:16.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.054+0000 7f885ab66740 1 --2- 172.21.15.2:0/629089711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c47750 0x560dc8c49b40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.054+0000 7f885ab66740 1 --2- 172.21.15.2:0/629089711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 0x560dc8c47210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.054+0000 7f885ab66740 1 --2- 172.21.15.2:0/629089711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3e250 0x560dc8c3e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.054+0000 7f885ab66740 1 -- 172.21.15.2:0/629089711 >> 172.21.15.2:0/629089711 conn(0x560dc8c38980 msgr2=0x560dc8c38d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.055+0000 7f885ab66740 1 -- 172.21.15.2:0/629089711 shutdown_connections 2024-03-15T13:05:16.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.055+0000 7f885ab66740 1 -- 172.21.15.2:0/629089711 wait complete. 2024-03-15T13:05:16.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.055+0000 7f885ab66740 1 -- start start 2024-03-15T13:05:16.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.055+0000 7f885ab66740 1 Processor -- start 2024-03-15T13:05:16.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.055+0000 7f885ab66740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c3e250 0x560dc8c47090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.055+0000 7f885ab66740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 0x560dc8c44420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.055+0000 7f885ab66740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c47750 0x560dc8c44960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.055+0000 7f885ab66740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8ba6020 con 0x560dc8c47750 2024-03-15T13:05:16.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.055+0000 7f885ab66740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8879430 con 0x560dc8c3e250 2024-03-15T13:05:16.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.055+0000 7f885ab66740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8835e00 con 0x560dc8c3eb70 2024-03-15T13:05:16.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.055+0000 7f885ab66740 10 client.0 _ll_register_callbacks cb 0x560dc8b4dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:16.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.055+0000 7f8856444640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c47750 0x560dc8c44960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f8855442640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 0x560dc8c44420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f8855c43640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c3e250 0x560dc8c47090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f8856444640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c47750 0x560dc8c44960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57980/0 (socket says 172.21.15.2:57980) 2024-03-15T13:05:16.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f8856444640 1 -- 172.21.15.2:0/2654511852 learned_addr learned my addr 172.21.15.2:0/2654511852 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f8855c43640 1 -- 172.21.15.2:0/2654511852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 msgr2=0x560dc8c44420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f8855c43640 1 --2- 172.21.15.2:0/2654511852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 0x560dc8c44420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f8855c43640 1 -- 172.21.15.2:0/2654511852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c47750 msgr2=0x560dc8c44960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f8855c43640 1 --2- 172.21.15.2:0/2654511852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c47750 0x560dc8c44960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f8855c43640 1 -- 172.21.15.2:0/2654511852 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8844009850 con 0x560dc8c3e250 2024-03-15T13:05:16.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f8856444640 1 --2- 172.21.15.2:0/2654511852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c47750 0x560dc8c44960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:16.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f8855442640 1 --2- 172.21.15.2:0/2654511852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 0x560dc8c44420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:16.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f8855c43640 1 --2- 172.21.15.2:0/2654511852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c3e250 0x560dc8c47090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f88480ed020 tx=0x7f884802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=587c27041bdbb484 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f88357fa640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:05:16.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f88357fa640 1 -- 172.21.15.2:0/2654511852 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88480c4ca0 con 0x560dc8c3e250 2024-03-15T13:05:16.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f88357fa640 1 -- 172.21.15.2:0/2654511852 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f88480bcc00 con 0x560dc8c3e250 2024-03-15T13:05:16.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f885ab66740 1 -- 172.21.15.2:0/2654511852 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560dc88a57e0 con 0x560dc8c3e250 2024-03-15T13:05:16.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f88357fa640 1 -- 172.21.15.2:0/2654511852 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88480c0650 con 0x560dc8c3e250 2024-03-15T13:05:16.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f885ab66740 10 client.27392 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:16.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.056+0000 7f885ab66740 1 -- 172.21.15.2:0/2654511852 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x560dc8ba46a0 con 0x560dc8c3e250 2024-03-15T13:05:16.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.057+0000 7f88357fa640 1 -- 172.21.15.2:0/2654511852 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x560dc88a57e0 con 0x560dc8c3e250 2024-03-15T13:05:16.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.057+0000 7f88357fa640 1 -- 172.21.15.2:0/2654511852 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f884803c070 con 0x560dc8c3e250 2024-03-15T13:05:16.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.057+0000 7f885ab66740 10 client.27392 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:16.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.057+0000 7f885ab66740 10 client.27392 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:05:16.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.057+0000 7f885ab66740 20 client.27392 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:16.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.057+0000 7f885ab66740 1 --2- 172.21.15.2:0/2654511852 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x560dc8c57560 0x560dc8c59970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.057+0000 7f885ab66740 4 client.27392 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:16.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.057+0000 7f885ab66740 1 -- 172.21.15.2:0/2654511852 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560dc88e2720 con 0x560dc8c57560 2024-03-15T13:05:16.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.057+0000 7f8855442640 1 --2- 172.21.15.2:0/2654511852 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x560dc8c57560 0x560dc8c59970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.057+0000 7f88357fa640 10 client.27392 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:05:16.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.057+0000 7f8855442640 1 --2- 172.21.15.2:0/2654511852 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x560dc8c57560 0x560dc8c59970 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fc2d830c2f3b1bec server_cookie=a14bd5f3ee6a2586 in_seq=0 out_seq=0 2024-03-15T13:05:16.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.058+0000 7f88357fa640 1 -- 172.21.15.2:0/2654511852 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560dc88e2720 con 0x560dc8c57560 2024-03-15T13:05:16.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.058+0000 7f88357fa640 10 client.27392 handle_command_reply: tid=0 2024-03-15T13:05:16.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.058+0000 7f885ab66740 1 client.27392 shutdown 2024-03-15T13:05:16.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.058+0000 7f885ab66740 1 -- 172.21.15.2:0/2654511852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c3e250 msgr2=0x560dc8c47090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.058+0000 7f885ab66740 1 --2- 172.21.15.2:0/2654511852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c3e250 0x560dc8c47090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f88480ed020 tx=0x7f884802bc30 comp rx=0 tx=0).stop 2024-03-15T13:05:16.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.058+0000 7f885ab66740 1 -- 172.21.15.2:0/2654511852 shutdown_connections 2024-03-15T13:05:16.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.058+0000 7f885ab66740 1 --2- 172.21.15.2:0/2654511852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c47750 0x560dc8c44960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.058+0000 7f885ab66740 1 --2- 172.21.15.2:0/2654511852 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x560dc8c57560 0x560dc8c59970 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.058+0000 7f885ab66740 1 --2- 172.21.15.2:0/2654511852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 0x560dc8c44420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.058+0000 7f885ab66740 1 --2- 172.21.15.2:0/2654511852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c3e250 0x560dc8c47090 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.058+0000 7f88357fa640 0 client.27392 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:05:16.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.058+0000 7f885ab66740 1 -- 172.21.15.2:0/2654511852 >> 172.21.15.2:0/2654511852 conn(0x560dc8c38980 msgr2=0x560dc8c3a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.058+0000 7f885ab66740 1 -- 172.21.15.2:0/2654511852 shutdown_connections 2024-03-15T13:05:16.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.058+0000 7f885ab66740 1 -- 172.21.15.2:0/2654511852 wait complete. 2024-03-15T13:05:16.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.058+0000 7f885ab66740 20 client.27392 trim_cache size 0 max 16384 2024-03-15T13:05:16.061 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:15 smithi184 ceph-mon[35993]: pgmap v193: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:05:16.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f885ab66740 1 Processor -- start 2024-03-15T13:05:16.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f885ab66740 1 -- start start 2024-03-15T13:05:16.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f885ab66740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c3e250 0x560dc8c372f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f885ab66740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3eb70 0x560dc8c46b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f885ab66740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c47750 0x560dc8c47090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f885ab66740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8b2b130 con 0x560dc8c3eb70 2024-03-15T13:05:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f885ab66740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8c57310 con 0x560dc8c3e250 2024-03-15T13:05:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f885ab66740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8c342b0 con 0x560dc8c47750 2024-03-15T13:05:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f8855442640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3eb70 0x560dc8c46b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f8855442640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3eb70 0x560dc8c46b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57982/0 (socket says 172.21.15.2:57982) 2024-03-15T13:05:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f8855442640 1 -- 172.21.15.2:0/4153616475 learned_addr learned my addr 172.21.15.2:0/4153616475 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f8855c43640 1 --2- 172.21.15.2:0/4153616475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c3e250 0x560dc8c372f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f8856444640 1 --2- 172.21.15.2:0/4153616475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c47750 0x560dc8c47090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f8855442640 1 -- 172.21.15.2:0/4153616475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c47750 msgr2=0x560dc8c47090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f8855442640 1 --2- 172.21.15.2:0/4153616475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c47750 0x560dc8c47090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f8855442640 1 -- 172.21.15.2:0/4153616475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c3e250 msgr2=0x560dc8c372f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f8855442640 1 --2- 172.21.15.2:0/4153616475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c3e250 0x560dc8c372f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.071+0000 7f8855442640 1 -- 172.21.15.2:0/4153616475 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dc8b40c20 con 0x560dc8c3eb70 2024-03-15T13:05:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.072+0000 7f8855442640 1 --2- 172.21.15.2:0/4153616475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3eb70 0x560dc8c46b50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8844009230 tx=0x7f8844005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f7f09647c9a71 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.072+0000 7f88377fe640 1 -- 172.21.15.2:0/4153616475 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884402a040 con 0x560dc8c3eb70 2024-03-15T13:05:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.072+0000 7f88377fe640 1 -- 172.21.15.2:0/4153616475 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8844009ec0 con 0x560dc8c3eb70 2024-03-15T13:05:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.072+0000 7f88377fe640 1 -- 172.21.15.2:0/4153616475 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844004e70 con 0x560dc8c3eb70 2024-03-15T13:05:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.072+0000 7f885ab66740 1 -- 172.21.15.2:0/4153616475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3eb70 msgr2=0x560dc8c46b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.072+0000 7f885ab66740 1 --2- 172.21.15.2:0/4153616475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3eb70 0x560dc8c46b50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8844009230 tx=0x7f8844005d30 comp rx=0 tx=0).stop 2024-03-15T13:05:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.072+0000 7f885ab66740 1 -- 172.21.15.2:0/4153616475 shutdown_connections 2024-03-15T13:05:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.072+0000 7f885ab66740 1 --2- 172.21.15.2:0/4153616475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c47750 0x560dc8c47090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.072+0000 7f885ab66740 1 --2- 172.21.15.2:0/4153616475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3eb70 0x560dc8c46b50 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.072+0000 7f885ab66740 1 --2- 172.21.15.2:0/4153616475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c3e250 0x560dc8c372f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.072+0000 7f885ab66740 1 -- 172.21.15.2:0/4153616475 >> 172.21.15.2:0/4153616475 conn(0x560dc8c38980 msgr2=0x560dc8c44540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.072+0000 7f885ab66740 1 -- 172.21.15.2:0/4153616475 shutdown_connections 2024-03-15T13:05:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.072+0000 7f885ab66740 1 -- 172.21.15.2:0/4153616475 wait complete. 2024-03-15T13:05:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.073+0000 7f885ab66740 1 -- start start 2024-03-15T13:05:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.073+0000 7f885ab66740 1 Processor -- start 2024-03-15T13:05:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.073+0000 7f885ab66740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3e250 0x560dc8c4f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.073+0000 7f885ab66740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 0x560dc8c41490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.073+0000 7f885ab66740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c47750 0x560dc8c419d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.073+0000 7f885ab66740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8c4f930 con 0x560dc8c3e250 2024-03-15T13:05:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.073+0000 7f885ab66740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8c41f10 con 0x560dc8c47750 2024-03-15T13:05:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.073+0000 7f885ab66740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc8c420d0 con 0x560dc8c3eb70 2024-03-15T13:05:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.073+0000 7f885ab66740 10 client.0 _ll_register_callbacks cb 0x7f8848004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.073+0000 7f8855c43640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3e250 0x560dc8c4f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.073+0000 7f8855c43640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3e250 0x560dc8c4f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57988/0 (socket says 172.21.15.2:57988) 2024-03-15T13:05:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.073+0000 7f8855c43640 1 -- 172.21.15.2:0/1192529095 learned_addr learned my addr 172.21.15.2:0/1192529095 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.073+0000 7f8856444640 1 --2- 172.21.15.2:0/1192529095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c47750 0x560dc8c419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.073+0000 7f8855442640 1 --2- 172.21.15.2:0/1192529095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 0x560dc8c41490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.073+0000 7f8855c43640 1 -- 172.21.15.2:0/1192529095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 msgr2=0x560dc8c41490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f8855c43640 1 --2- 172.21.15.2:0/1192529095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 0x560dc8c41490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f8855c43640 1 -- 172.21.15.2:0/1192529095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c47750 msgr2=0x560dc8c419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f8855c43640 1 --2- 172.21.15.2:0/1192529095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c47750 0x560dc8c419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f8855c43640 1 -- 172.21.15.2:0/1192529095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560dc8b68060 con 0x560dc8c3e250 2024-03-15T13:05:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f8855c43640 1 --2- 172.21.15.2:0/1192529095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3e250 0x560dc8c4f060 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f88480b8ed0 tx=0x7f88480baa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e355e14ca92100f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f88357fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f88357fa640 1 -- 172.21.15.2:0/1192529095 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88480bcc00 con 0x560dc8c3e250 2024-03-15T13:05:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f88357fa640 1 -- 172.21.15.2:0/1192529095 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f88480c4ca0 con 0x560dc8c3e250 2024-03-15T13:05:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f885ab66740 1 -- 172.21.15.2:0/1192529095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8844005020 con 0x560dc8c3e250 2024-03-15T13:05:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f88357fa640 1 -- 172.21.15.2:0/1192529095 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88480c0650 con 0x560dc8c3e250 2024-03-15T13:05:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f885ab66740 10 client.17535 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f885ab66740 1 -- 172.21.15.2:0/1192529095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x560dc8b668c0 con 0x560dc8c3e250 2024-03-15T13:05:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f88357fa640 1 -- 172.21.15.2:0/1192529095 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8844005020 con 0x560dc8c3e250 2024-03-15T13:05:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f88357fa640 1 -- 172.21.15.2:0/1192529095 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f884803c070 con 0x560dc8c3e250 2024-03-15T13:05:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f885ab66740 10 client.17535 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f885ab66740 10 client.17535 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:05:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f885ab66740 20 client.17535 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f885ab66740 1 --2- 172.21.15.2:0/1192529095 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x560dc8cb3d10 0x560dc8cb6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f885ab66740 4 client.17535 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:05:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.074+0000 7f885ab66740 1 -- 172.21.15.2:0/1192529095 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560dc8931890 con 0x560dc8cb3d10 2024-03-15T13:05:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f8855442640 1 --2- 172.21.15.2:0/1192529095 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x560dc8cb3d10 0x560dc8cb6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f8855442640 1 --2- 172.21.15.2:0/1192529095 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x560dc8cb3d10 0x560dc8cb6100 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=68c65862c891a7b0 server_cookie=ead77d7a2783764a in_seq=0 out_seq=0 2024-03-15T13:05:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f88357fa640 10 client.17535 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:05:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f88357fa640 1 -- 172.21.15.2:0/1192529095 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560dc8931890 con 0x560dc8cb3d10 2024-03-15T13:05:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f88357fa640 10 client.17535 handle_command_reply: tid=0 2024-03-15T13:05:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f885ab66740 1 client.17535 shutdown 2024-03-15T13:05:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f885ab66740 1 -- 172.21.15.2:0/1192529095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3e250 msgr2=0x560dc8c4f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f885ab66740 1 --2- 172.21.15.2:0/1192529095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3e250 0x560dc8c4f060 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f88480b8ed0 tx=0x7f88480baa10 comp rx=0 tx=0).stop 2024-03-15T13:05:16.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f885ab66740 1 -- 172.21.15.2:0/1192529095 shutdown_connections 2024-03-15T13:05:16.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f885ab66740 1 --2- 172.21.15.2:0/1192529095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dc8c47750 0x560dc8c419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f885ab66740 1 --2- 172.21.15.2:0/1192529095 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x560dc8cb3d10 0x560dc8cb6100 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f885ab66740 1 --2- 172.21.15.2:0/1192529095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560dc8c3eb70 0x560dc8c41490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f885ab66740 1 --2- 172.21.15.2:0/1192529095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560dc8c3e250 0x560dc8c4f060 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f88357fa640 0 client.17535 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:05:16.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f885ab66740 1 -- 172.21.15.2:0/1192529095 >> 172.21.15.2:0/1192529095 conn(0x560dc8c38980 msgr2=0x560dc8c3ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f885ab66740 1 -- 172.21.15.2:0/1192529095 shutdown_connections 2024-03-15T13:05:16.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f885ab66740 1 -- 172.21.15.2:0/1192529095 wait complete. 2024-03-15T13:05:16.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.075+0000 7f885ab66740 20 client.17535 trim_cache size 0 max 16384 2024-03-15T13:05:16.068 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:05:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.076+0000 7f8857ea7640 1 -- 172.21.15.2:0/1280346837 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f88380a1ba0 msgr2=0x7f88380a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.076+0000 7f8857ea7640 1 --2- 172.21.15.2:0/1280346837 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f88380a1ba0 0x7f88380a4060 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8840009fd0 tx=0x7f8840004ab0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.076+0000 7f8857ea7640 1 -- 172.21.15.2:0/1280346837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88501014a0 msgr2=0x7f88500737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.076+0000 7f8857ea7640 1 --2- 172.21.15.2:0/1280346837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88501014a0 0x7f88500737f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8848009230 tx=0x7f884802cac0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.076+0000 7f8857ea7640 1 -- 172.21.15.2:0/1280346837 shutdown_connections 2024-03-15T13:05:16.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.076+0000 7f8857ea7640 1 --2- 172.21.15.2:0/1280346837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8850107560 0x7f8850074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.076+0000 7f8857ea7640 1 --2- 172.21.15.2:0/1280346837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8850101e60 0x7f8850073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.076+0000 7f8857ea7640 1 --2- 172.21.15.2:0/1280346837 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f88380a1ba0 0x7f88380a4060 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.076+0000 7f8857ea7640 1 --2- 172.21.15.2:0/1280346837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88501014a0 0x7f88500737f0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.076+0000 7f8857ea7640 1 -- 172.21.15.2:0/1280346837 >> 172.21.15.2:0/1280346837 conn(0x7f88500788f0 msgr2=0x7f8850100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.076+0000 7f8857ea7640 1 -- 172.21.15.2:0/1280346837 shutdown_connections 2024-03-15T13:05:16.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.076+0000 7f8857ea7640 1 -- 172.21.15.2:0/1280346837 wait complete. 2024-03-15T13:05:16.071 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:05:16.072 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:05:16.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:15 smithi149 ceph-mon[35927]: pgmap v193: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-15T13:05:16.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 -- 172.21.15.2:0/3136663771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48f0100f80 msgr2=0x7f48f0104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 --2- 172.21.15.2:0/3136663771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48f0100f80 0x7f48f0104fc0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009930 tx=0x7f48dc02bc30 comp rx=0 tx=0).stop 2024-03-15T13:05:16.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 -- 172.21.15.2:0/3136663771 shutdown_connections 2024-03-15T13:05:16.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 --2- 172.21.15.2:0/3136663771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48f01056a0 0x7f48f010d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 --2- 172.21.15.2:0/3136663771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48f0100f80 0x7f48f0104fc0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 --2- 172.21.15.2:0/3136663771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48f01005d0 0x7f48f01009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 -- 172.21.15.2:0/3136663771 >> 172.21.15.2:0/3136663771 conn(0x7f48f00774f0 msgr2=0x7f48f0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 -- 172.21.15.2:0/3136663771 shutdown_connections 2024-03-15T13:05:16.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 -- 172.21.15.2:0/3136663771 wait complete. 2024-03-15T13:05:16.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 Processor -- start 2024-03-15T13:05:16.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 -- start start 2024-03-15T13:05:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48f01005d0 0x7f48f01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48f01056a0 0x7f48f0198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48f01994a0 0x7f48f01a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f01a40f0 con 0x7f48f01994a0 2024-03-15T13:05:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f01a4260 con 0x7f48f01005d0 2024-03-15T13:05:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.233+0000 7f48f6c9a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f01a4420 con 0x7f48f01056a0 2024-03-15T13:05:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48f5237640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48f01994a0 0x7f48f01a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48f5237640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48f01994a0 0x7f48f01a3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58010/0 (socket says 172.21.15.2:58010) 2024-03-15T13:05:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48effff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48f01056a0 0x7f48f0198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48f5237640 1 -- 172.21.15.2:0/2607598699 learned_addr learned my addr 172.21.15.2:0/2607598699 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48f4a36640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48f01005d0 0x7f48f01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48f5237640 1 -- 172.21.15.2:0/2607598699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48f01056a0 msgr2=0x7f48f0198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48f5237640 1 --2- 172.21.15.2:0/2607598699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48f01056a0 0x7f48f0198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48f5237640 1 -- 172.21.15.2:0/2607598699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48f01005d0 msgr2=0x7f48f01989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48f5237640 1 --2- 172.21.15.2:0/2607598699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48f01005d0 0x7f48f01989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48f5237640 1 -- 172.21.15.2:0/2607598699 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e8009840 con 0x7f48f01994a0 2024-03-15T13:05:16.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48effff640 1 --2- 172.21.15.2:0/2607598699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48f01056a0 0x7f48f0198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:16.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48f5237640 1 --2- 172.21.15.2:0/2607598699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48f01994a0 0x7f48f01a3a50 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f48e800efd0 tx=0x7f48e800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48edffb640 1 -- 172.21.15.2:0/2607598699 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8009b40 con 0x7f48f01994a0 2024-03-15T13:05:16.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48edffb640 1 -- 172.21.15.2:0/2607598699 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f48e8003b70 con 0x7f48f01994a0 2024-03-15T13:05:16.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48f6c9a640 1 -- 172.21.15.2:0/2607598699 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48dc009560 con 0x7f48f01994a0 2024-03-15T13:05:16.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48edffb640 1 -- 172.21.15.2:0/2607598699 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e80108f0 con 0x7f48f01994a0 2024-03-15T13:05:16.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.234+0000 7f48f6c9a640 1 -- 172.21.15.2:0/2607598699 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f01a4a00 con 0x7f48f01994a0 2024-03-15T13:05:16.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.237+0000 7f48edffb640 1 -- 172.21.15.2:0/2607598699 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f48e8003ce0 con 0x7f48f01994a0 2024-03-15T13:05:16.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.237+0000 7f48f9878740 1 Processor -- start 2024-03-15T13:05:16.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.237+0000 7f48f9878740 1 -- start start 2024-03-15T13:05:16.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.237+0000 7f48f9878740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670250 0x56230b670630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.237+0000 7f48f9878740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670b70 0x56230b679210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.237+0000 7f48f9878740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 0x56230b67bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.237+0000 7f48f9878740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230b62b960 con 0x56230b679750 2024-03-15T13:05:16.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.237+0000 7f48f9878740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230b5df310 con 0x56230b670b70 2024-03-15T13:05:16.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.237+0000 7f48f9878740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230b5dc840 con 0x56230b670250 2024-03-15T13:05:16.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48edffb640 1 --2- 172.21.15.2:0/2607598699 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f48e00a1ba0 0x7f48e00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48f4a36640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670b70 0x56230b679210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48effff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670250 0x56230b670630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48f4a36640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670b70 0x56230b679210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44032/0 (socket says 172.21.15.2:44032) 2024-03-15T13:05:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48f4a36640 1 -- 172.21.15.2:0/841251598 learned_addr learned my addr 172.21.15.2:0/841251598 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48effff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670250 0x56230b670630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37344/0 (socket says 172.21.15.2:37344) 2024-03-15T13:05:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48f5237640 1 --2- 172.21.15.2:0/2607598699 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f48e00a1ba0 0x7f48e00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48effff640 1 --2- 172.21.15.2:0/841251598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 0x56230b67bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48edffb640 1 -- 172.21.15.2:0/2607598699 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f48e8014070 con 0x7f48f01994a0 2024-03-15T13:05:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48f4a36640 1 -- 172.21.15.2:0/841251598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670250 msgr2=0x56230b670630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48f4a36640 1 --2- 172.21.15.2:0/841251598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670250 0x56230b670630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48f4a36640 1 -- 172.21.15.2:0/841251598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 msgr2=0x56230b67bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48f4a36640 1 --2- 172.21.15.2:0/841251598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 0x56230b67bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48f4a36640 1 -- 172.21.15.2:0/841251598 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56230b59a060 con 0x56230b670b70 2024-03-15T13:05:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48effff640 1 --2- 172.21.15.2:0/841251598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670250 0x56230b670630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48f4a36640 1 --2- 172.21.15.2:0/841251598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670b70 0x56230b679210 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009cf0 tx=0x7f48e4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c4ee5fc2e7f1f97 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48f5237640 1 --2- 172.21.15.2:0/2607598699 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f48e00a1ba0 0x7f48e00a4060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f48e809a680 tx=0x7f48e806cb80 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48daffd640 1 -- 172.21.15.2:0/841251598 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4009ea0 con 0x56230b670b70 2024-03-15T13:05:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48daffd640 1 -- 172.21.15.2:0/841251598 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f48e4004230 con 0x56230b670b70 2024-03-15T13:05:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.238+0000 7f48daffd640 1 -- 172.21.15.2:0/841251598 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4019750 con 0x56230b670b70 2024-03-15T13:05:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.239+0000 7f48f9878740 1 -- 172.21.15.2:0/841251598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670b70 msgr2=0x56230b679210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.239+0000 7f48f9878740 1 --2- 172.21.15.2:0/841251598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670b70 0x56230b679210 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009cf0 tx=0x7f48e4011810 comp rx=0 tx=0).stop 2024-03-15T13:05:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.239+0000 7f48f9878740 1 -- 172.21.15.2:0/841251598 shutdown_connections 2024-03-15T13:05:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.239+0000 7f48f9878740 1 --2- 172.21.15.2:0/841251598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 0x56230b67bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.239+0000 7f48f9878740 1 --2- 172.21.15.2:0/841251598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670b70 0x56230b679210 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.239+0000 7f48f9878740 1 --2- 172.21.15.2:0/841251598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670250 0x56230b670630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.239+0000 7f48f9878740 1 -- 172.21.15.2:0/841251598 >> 172.21.15.2:0/841251598 conn(0x56230b66a980 msgr2=0x56230b66ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.239+0000 7f48f9878740 1 -- 172.21.15.2:0/841251598 shutdown_connections 2024-03-15T13:05:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.239+0000 7f48f9878740 1 -- 172.21.15.2:0/841251598 wait complete. 2024-03-15T13:05:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.239+0000 7f48f9878740 1 -- start start 2024-03-15T13:05:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.239+0000 7f48f9878740 1 Processor -- start 2024-03-15T13:05:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f9878740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670250 0x56230b679040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f9878740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b670b70 0x56230b6763d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f9878740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b679750 0x56230b676910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f9878740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230b5d8020 con 0x56230b670b70 2024-03-15T13:05:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f9878740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230b2ab430 con 0x56230b670250 2024-03-15T13:05:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f9878740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230b267e00 con 0x56230b679750 2024-03-15T13:05:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f9878740 10 client.0 _ll_register_callbacks cb 0x56230b57fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f4a36640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b670b70 0x56230b6763d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f4a36640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b670b70 0x56230b6763d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58024/0 (socket says 172.21.15.2:58024) 2024-03-15T13:05:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f4a36640 1 -- 172.21.15.2:0/1999323133 learned_addr learned my addr 172.21.15.2:0/1999323133 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48effff640 1 --2- 172.21.15.2:0/1999323133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670250 0x56230b679040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48effff640 1 --2- 172.21.15.2:0/1999323133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b679750 0x56230b676910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f4a36640 1 -- 172.21.15.2:0/1999323133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b679750 msgr2=0x56230b676910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f4a36640 1 --2- 172.21.15.2:0/1999323133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b679750 0x56230b676910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f4a36640 1 -- 172.21.15.2:0/1999323133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670250 msgr2=0x56230b679040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f4a36640 1 --2- 172.21.15.2:0/1999323133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670250 0x56230b679040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f4a36640 1 -- 172.21.15.2:0/1999323133 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e4009950 con 0x56230b670b70 2024-03-15T13:05:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48f4a36640 1 --2- 172.21.15.2:0/1999323133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b670b70 0x56230b6763d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f48e4005bc0 tx=0x7f48e40041c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c3612631322526 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.240+0000 7f48cffff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.241+0000 7f48cffff640 1 -- 172.21.15.2:0/1999323133 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4005240 con 0x56230b670b70 2024-03-15T13:05:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.241+0000 7f48cffff640 1 -- 172.21.15.2:0/1999323133 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f48e4003a20 con 0x56230b670b70 2024-03-15T13:05:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.241+0000 7f48f9878740 1 -- 172.21.15.2:0/1999323133 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56230b2d77e0 con 0x56230b670b70 2024-03-15T13:05:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.241+0000 7f48cffff640 1 -- 172.21.15.2:0/1999323133 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48e4009ea0 con 0x56230b670b70 2024-03-15T13:05:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.241+0000 7f48f9878740 10 client.17547 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.241+0000 7f48f9878740 1 -- 172.21.15.2:0/1999323133 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x56230b5d66a0 con 0x56230b670b70 2024-03-15T13:05:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.241+0000 7f48cffff640 1 -- 172.21.15.2:0/1999323133 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x56230b2d77e0 con 0x56230b670b70 2024-03-15T13:05:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.241+0000 7f48cffff640 1 -- 172.21.15.2:0/1999323133 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f48e4021040 con 0x56230b670b70 2024-03-15T13:05:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.241+0000 7f48f9878740 10 client.17547 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.241+0000 7f48f9878740 10 client.17547 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:05:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.241+0000 7f48f9878740 20 client.17547 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.241+0000 7f48f9878740 1 --2- 172.21.15.2:0/1999323133 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56230b689770 0x56230b68bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.241+0000 7f48f9878740 4 client.17547 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.241+0000 7f48f9878740 1 -- 172.21.15.2:0/1999323133 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56230b314720 con 0x56230b689770 2024-03-15T13:05:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.241+0000 7f48effff640 1 --2- 172.21.15.2:0/1999323133 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56230b689770 0x56230b68bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.242+0000 7f48cffff640 10 client.17547 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:05:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.242+0000 7f48effff640 1 --2- 172.21.15.2:0/1999323133 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56230b689770 0x56230b68bb60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a410702556260f2c server_cookie=d2b7b0b06886cd27 in_seq=0 out_seq=0 2024-03-15T13:05:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.243+0000 7f48cffff640 1 -- 172.21.15.2:0/1999323133 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56230b314720 con 0x56230b689770 2024-03-15T13:05:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.243+0000 7f48cffff640 10 client.17547 handle_command_reply: tid=0 2024-03-15T13:05:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.243+0000 7f48f9878740 1 client.17547 shutdown 2024-03-15T13:05:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.243+0000 7f48f9878740 1 -- 172.21.15.2:0/1999323133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b670b70 msgr2=0x56230b6763d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.243+0000 7f48f9878740 1 --2- 172.21.15.2:0/1999323133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b670b70 0x56230b6763d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f48e4005bc0 tx=0x7f48e40041c0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.243+0000 7f48f9878740 1 -- 172.21.15.2:0/1999323133 shutdown_connections 2024-03-15T13:05:16.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.243+0000 7f48f9878740 1 --2- 172.21.15.2:0/1999323133 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56230b689770 0x56230b68bb60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.243+0000 7f48f9878740 1 --2- 172.21.15.2:0/1999323133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b679750 0x56230b676910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.243+0000 7f48f9878740 1 --2- 172.21.15.2:0/1999323133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b670b70 0x56230b6763d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.243+0000 7f48cffff640 0 client.17547 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:05:16.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.243+0000 7f48f9878740 1 --2- 172.21.15.2:0/1999323133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670250 0x56230b679040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.243+0000 7f48f9878740 1 -- 172.21.15.2:0/1999323133 >> 172.21.15.2:0/1999323133 conn(0x56230b66a980 msgr2=0x56230b66c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.243+0000 7f48f9878740 1 -- 172.21.15.2:0/1999323133 shutdown_connections 2024-03-15T13:05:16.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.243+0000 7f48f9878740 1 -- 172.21.15.2:0/1999323133 wait complete. 2024-03-15T13:05:16.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.243+0000 7f48f9878740 20 client.17547 trim_cache size 0 max 16384 2024-03-15T13:05:16.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48f9878740 1 Processor -- start 2024-03-15T13:05:16.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48f9878740 1 -- start start 2024-03-15T13:05:16.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48f9878740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670250 0x56230b678b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48f9878740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670b70 0x56230b679040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48f9878740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 0x56230b684ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48f9878740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230b55d130 con 0x56230b679750 2024-03-15T13:05:16.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48f9878740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230b66b2f0 con 0x56230b670250 2024-03-15T13:05:16.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48f9878740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230b66ba00 con 0x56230b670b70 2024-03-15T13:05:16.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48effff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 0x56230b684ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48effff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 0x56230b684ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58032/0 (socket says 172.21.15.2:58032) 2024-03-15T13:05:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48effff640 1 -- 172.21.15.2:0/3326057760 learned_addr learned my addr 172.21.15.2:0/3326057760 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48f4a36640 1 --2- 172.21.15.2:0/3326057760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670b70 0x56230b679040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48effff640 1 --2- 172.21.15.2:0/3326057760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670250 0x56230b678b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48f4a36640 1 -- 172.21.15.2:0/3326057760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670250 msgr2=0x56230b678b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48f4a36640 1 --2- 172.21.15.2:0/3326057760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670250 0x56230b678b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48f4a36640 1 -- 172.21.15.2:0/3326057760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 msgr2=0x56230b684ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48f4a36640 1 --2- 172.21.15.2:0/3326057760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 0x56230b684ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48f4a36640 1 -- 172.21.15.2:0/3326057760 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56230b572c20 con 0x56230b670b70 2024-03-15T13:05:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.256+0000 7f48f4a36640 1 --2- 172.21.15.2:0/3326057760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670b70 0x56230b679040 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009380 tx=0x7f48e4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f75c7a04d32fa450 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.257+0000 7f48da7fc640 1 -- 172.21.15.2:0/3326057760 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4003c20 con 0x56230b670b70 2024-03-15T13:05:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.257+0000 7f48da7fc640 1 -- 172.21.15.2:0/3326057760 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f48e4009ea0 con 0x56230b670b70 2024-03-15T13:05:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.257+0000 7f48da7fc640 1 -- 172.21.15.2:0/3326057760 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4004a20 con 0x56230b670b70 2024-03-15T13:05:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.257+0000 7f48f9878740 1 -- 172.21.15.2:0/3326057760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670b70 msgr2=0x56230b679040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.257+0000 7f48f9878740 1 --2- 172.21.15.2:0/3326057760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670b70 0x56230b679040 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009380 tx=0x7f48e4011810 comp rx=0 tx=0).stop 2024-03-15T13:05:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.257+0000 7f48f9878740 1 -- 172.21.15.2:0/3326057760 shutdown_connections 2024-03-15T13:05:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.257+0000 7f48f9878740 1 --2- 172.21.15.2:0/3326057760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 0x56230b684ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.257+0000 7f48f9878740 1 --2- 172.21.15.2:0/3326057760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670b70 0x56230b679040 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.257+0000 7f48f9878740 1 --2- 172.21.15.2:0/3326057760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670250 0x56230b678b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.257+0000 7f48f9878740 1 -- 172.21.15.2:0/3326057760 >> 172.21.15.2:0/3326057760 conn(0x56230b66a980 msgr2=0x56230b6764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.257+0000 7f48f9878740 1 -- 172.21.15.2:0/3326057760 shutdown_connections 2024-03-15T13:05:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.257+0000 7f48f9878740 1 -- 172.21.15.2:0/3326057760 wait complete. 2024-03-15T13:05:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.258+0000 7f48f9878740 1 -- start start 2024-03-15T13:05:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.258+0000 7f48f9878740 1 Processor -- start 2024-03-15T13:05:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.258+0000 7f48f9878740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670250 0x56230b67d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.258+0000 7f48f9878740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670b70 0x56230b67d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.258+0000 7f48f9878740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 0x56230b6e4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.258+0000 7f48f9878740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230b6e4550 con 0x56230b679750 2024-03-15T13:05:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.258+0000 7f48f9878740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230b6e4690 con 0x56230b670b70 2024-03-15T13:05:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.258+0000 7f48f9878740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230b6e47d0 con 0x56230b670250 2024-03-15T13:05:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.258+0000 7f48f9878740 10 client.0 _ll_register_callbacks cb 0x7f48dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.258+0000 7f48effff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670250 0x56230b67d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.258+0000 7f48f4a36640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670b70 0x56230b67d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.258+0000 7f48effff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 0x56230b6e4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.258+0000 7f48f4a36640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670b70 0x56230b67d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44054/0 (socket says 172.21.15.2:44054) 2024-03-15T13:05:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.258+0000 7f48effff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 0x56230b6e4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58048/0 (socket says 172.21.15.2:58048) 2024-03-15T13:05:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.258+0000 7f48f4a36640 1 -- 172.21.15.2:0/2351179450 learned_addr learned my addr 172.21.15.2:0/2351179450 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48f4a36640 1 -- 172.21.15.2:0/2351179450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670250 msgr2=0x56230b67d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48f4a36640 1 --2- 172.21.15.2:0/2351179450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670250 0x56230b67d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48f4a36640 1 -- 172.21.15.2:0/2351179450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 msgr2=0x56230b6e4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48f4a36640 1 --2- 172.21.15.2:0/2351179450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 0x56230b6e4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48f4a36640 1 -- 172.21.15.2:0/2351179450 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56230b5d66a0 con 0x56230b670b70 2024-03-15T13:05:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48effff640 1 --2- 172.21.15.2:0/2351179450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 0x56230b6e4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48f4a36640 1 --2- 172.21.15.2:0/2351179450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670b70 0x56230b67d6b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f48e4000c00 tx=0x7f48e4004620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b894c302facdb264 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48cffff640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:05:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48cffff640 1 -- 172.21.15.2:0/2351179450 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e40199d0 con 0x56230b670b70 2024-03-15T13:05:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48f9878740 1 -- 172.21.15.2:0/2351179450 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48e4004230 con 0x56230b670b70 2024-03-15T13:05:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48cffff640 1 -- 172.21.15.2:0/2351179450 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f48e4000e40 con 0x56230b670b70 2024-03-15T13:05:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48cffff640 1 -- 172.21.15.2:0/2351179450 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48e4019b30 con 0x56230b670b70 2024-03-15T13:05:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48f9878740 10 client.27416 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48f9878740 1 -- 172.21.15.2:0/2351179450 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x56230b5988c0 con 0x56230b670b70 2024-03-15T13:05:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48cffff640 1 -- 172.21.15.2:0/2351179450 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f48e4004230 con 0x56230b670b70 2024-03-15T13:05:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.259+0000 7f48cffff640 1 -- 172.21.15.2:0/2351179450 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f48e4021040 con 0x56230b670b70 2024-03-15T13:05:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 10 client.27416 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 10 client.27416 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 20 client.27416 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 1 --2- 172.21.15.2:0/2351179450 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56230b6e5ae0 0x56230b6e7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 4 client.27416 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 1 -- 172.21.15.2:0/2351179450 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56230b363890 con 0x56230b6e5ae0 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48effff640 1 --2- 172.21.15.2:0/2351179450 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56230b6e5ae0 0x56230b6e7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48cffff640 10 client.27416 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48effff640 1 --2- 172.21.15.2:0/2351179450 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56230b6e5ae0 0x56230b6e7f00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8594ca25c075f3e7 server_cookie=c9655f8717331118 in_seq=0 out_seq=0 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48cffff640 1 -- 172.21.15.2:0/2351179450 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56230b363890 con 0x56230b6e5ae0 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48cffff640 10 client.27416 handle_command_reply: tid=0 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 1 client.27416 shutdown 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 1 -- 172.21.15.2:0/2351179450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670b70 msgr2=0x56230b67d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 1 --2- 172.21.15.2:0/2351179450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670b70 0x56230b67d6b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f48e4000c00 tx=0x7f48e4004620 comp rx=0 tx=0).stop 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 1 -- 172.21.15.2:0/2351179450 shutdown_connections 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 1 --2- 172.21.15.2:0/2351179450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56230b679750 0x56230b6e4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 1 --2- 172.21.15.2:0/2351179450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56230b670b70 0x56230b67d6b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 1 --2- 172.21.15.2:0/2351179450 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56230b6e5ae0 0x56230b6e7f00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 1 --2- 172.21.15.2:0/2351179450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56230b670250 0x56230b67d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48cffff640 0 client.27416 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:05:16.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 1 -- 172.21.15.2:0/2351179450 >> 172.21.15.2:0/2351179450 conn(0x56230b66a980 msgr2=0x56230b670f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 1 -- 172.21.15.2:0/2351179450 shutdown_connections 2024-03-15T13:05:16.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.260+0000 7f48f9878740 1 -- 172.21.15.2:0/2351179450 wait complete. 2024-03-15T13:05:16.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.261+0000 7f48f9878740 20 client.27416 trim_cache size 0 max 16384 2024-03-15T13:05:16.253 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:05:16.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.261+0000 7f48f6c9a640 1 -- 172.21.15.2:0/2607598699 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f48e00a1ba0 msgr2=0x7f48e00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.261+0000 7f48f6c9a640 1 --2- 172.21.15.2:0/2607598699 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f48e00a1ba0 0x7f48e00a4060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f48e809a680 tx=0x7f48e806cb80 comp rx=0 tx=0).stop 2024-03-15T13:05:16.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.261+0000 7f48f6c9a640 1 -- 172.21.15.2:0/2607598699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48f01994a0 msgr2=0x7f48f01a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.261+0000 7f48f6c9a640 1 --2- 172.21.15.2:0/2607598699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48f01994a0 0x7f48f01a3a50 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f48e800efd0 tx=0x7f48e800c6a0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.261+0000 7f48f6c9a640 1 -- 172.21.15.2:0/2607598699 shutdown_connections 2024-03-15T13:05:16.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.261+0000 7f48f6c9a640 1 --2- 172.21.15.2:0/2607598699 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f48e00a1ba0 0x7f48e00a4060 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.261+0000 7f48f6c9a640 1 --2- 172.21.15.2:0/2607598699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48f01994a0 0x7f48f01a3a50 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.261+0000 7f48f6c9a640 1 --2- 172.21.15.2:0/2607598699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48f01056a0 0x7f48f0198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.261+0000 7f48f6c9a640 1 --2- 172.21.15.2:0/2607598699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48f01005d0 0x7f48f01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.261+0000 7f48f6c9a640 1 -- 172.21.15.2:0/2607598699 >> 172.21.15.2:0/2607598699 conn(0x7f48f00774f0 msgr2=0x7f48f00fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.261+0000 7f48f6c9a640 1 -- 172.21.15.2:0/2607598699 shutdown_connections 2024-03-15T13:05:16.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.261+0000 7f48f6c9a640 1 -- 172.21.15.2:0/2607598699 wait complete. 2024-03-15T13:05:16.257 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:05:16.257 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:05:16.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.377+0000 7fba0a847640 1 -- 172.21.15.2:0/3979656662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba04101e60 msgr2=0x7fba0410e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.377+0000 7fba0a847640 1 --2- 172.21.15.2:0/3979656662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba04101e60 0x7fba0410e320 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009ca0 tx=0x7fb9fc02c0f0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.377+0000 7fba0a847640 1 -- 172.21.15.2:0/3979656662 shutdown_connections 2024-03-15T13:05:16.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.377+0000 7fba0a847640 1 --2- 172.21.15.2:0/3979656662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba04101e60 0x7fba0410e320 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.377+0000 7fba0a847640 1 --2- 172.21.15.2:0/3979656662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba041014a0 0x7fba04101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.377+0000 7fba0a847640 1 --2- 172.21.15.2:0/3979656662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba04107560 0x7fba04107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.377+0000 7fba0a847640 1 -- 172.21.15.2:0/3979656662 >> 172.21.15.2:0/3979656662 conn(0x7fba040788f0 msgr2=0x7fba04078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.377+0000 7fba0a847640 1 -- 172.21.15.2:0/3979656662 shutdown_connections 2024-03-15T13:05:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.377+0000 7fba0a847640 1 -- 172.21.15.2:0/3979656662 wait complete. 2024-03-15T13:05:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba0a847640 1 Processor -- start 2024-03-15T13:05:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba0a847640 1 -- start start 2024-03-15T13:05:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba0a847640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041014a0 0x7fba0419d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba0a847640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba04101e60 0x7fba0419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba0a847640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba04107560 0x7fba04197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba0a847640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba04197ef0 con 0x7fba041014a0 2024-03-15T13:05:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba0a847640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba04198060 con 0x7fba04107560 2024-03-15T13:05:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba0a847640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba04198220 con 0x7fba04101e60 2024-03-15T13:05:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba03fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041014a0 0x7fba0419d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba03fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041014a0 0x7fba0419d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58064/0 (socket says 172.21.15.2:58064) 2024-03-15T13:05:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba03fff640 1 -- 172.21.15.2:0/2253364216 learned_addr learned my addr 172.21.15.2:0/2253364216 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba037fe640 1 --2- 172.21.15.2:0/2253364216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba04101e60 0x7fba0419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba08de4640 1 --2- 172.21.15.2:0/2253364216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba04107560 0x7fba04197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba037fe640 1 -- 172.21.15.2:0/2253364216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba04107560 msgr2=0x7fba04197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba037fe640 1 --2- 172.21.15.2:0/2253364216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba04107560 0x7fba04197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba037fe640 1 -- 172.21.15.2:0/2253364216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041014a0 msgr2=0x7fba0419d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba037fe640 1 --2- 172.21.15.2:0/2253364216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041014a0 0x7fba0419d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba037fe640 1 -- 172.21.15.2:0/2253364216 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc009900 con 0x7fba04101e60 2024-03-15T13:05:16.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.378+0000 7fba03fff640 1 --2- 172.21.15.2:0/2253364216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041014a0 0x7fba0419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:16.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.379+0000 7fba037fe640 1 --2- 172.21.15.2:0/2253364216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba04101e60 0x7fba0419dc20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800e9f0 tx=0x7fb9f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.379+0000 7fba017fa640 1 -- 172.21.15.2:0/2253364216 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800ccc0 con 0x7fba04101e60 2024-03-15T13:05:16.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.379+0000 7fba017fa640 1 -- 172.21.15.2:0/2253364216 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb9f800ce20 con 0x7fba04101e60 2024-03-15T13:05:16.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.379+0000 7fba017fa640 1 -- 172.21.15.2:0/2253364216 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8004a00 con 0x7fba04101e60 2024-03-15T13:05:16.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.379+0000 7fba0a847640 1 -- 172.21.15.2:0/2253364216 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba04198500 con 0x7fba04101e60 2024-03-15T13:05:16.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.379+0000 7fba0a847640 1 -- 172.21.15.2:0/2253364216 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba041a4710 con 0x7fba04101e60 2024-03-15T13:05:16.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.381+0000 7fba017fa640 1 -- 172.21.15.2:0/2253364216 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fb9f8004b60 con 0x7fba04101e60 2024-03-15T13:05:16.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.381+0000 7fba0d427740 1 Processor -- start 2024-03-15T13:05:16.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.381+0000 7fba0d427740 1 -- start start 2024-03-15T13:05:16.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba0d427740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb90995630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba0d427740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb90995b70 0x55bb9099e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba0d427740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb9099e750 0x55bb909a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba0d427740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb90950960 con 0x55bb9099e750 2024-03-15T13:05:16.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba0d427740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb90904310 con 0x55bb90995b70 2024-03-15T13:05:16.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba0d427740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb90901840 con 0x55bb90995250 2024-03-15T13:05:16.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba037fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb9099e750 0x55bb909a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba03fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb90995b70 0x55bb9099e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba017fa640 1 --2- 172.21.15.2:0/2253364216 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb9ec0a1ba0 0x7fb9ec0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba08de4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb90995630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba037fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb9099e750 0x55bb909a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58066/0 (socket says 172.21.15.2:58066) 2024-03-15T13:05:16.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba037fe640 1 -- 172.21.15.2:0/2492167360 learned_addr learned my addr 172.21.15.2:0/2492167360 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba08de4640 1 --2- 172.21.15.2:0/2253364216 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb9ec0a1ba0 0x7fb9ec0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba017fa640 1 -- 172.21.15.2:0/2253364216 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fb9f8014070 con 0x7fba04101e60 2024-03-15T13:05:16.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba03fff640 1 -- 172.21.15.2:0/2492167360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 msgr2=0x55bb90995630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba03fff640 1 --2- 172.21.15.2:0/2492167360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb90995630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba03fff640 1 -- 172.21.15.2:0/2492167360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb9099e750 msgr2=0x55bb909a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba03fff640 1 --2- 172.21.15.2:0/2492167360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb9099e750 0x55bb909a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba03fff640 1 -- 172.21.15.2:0/2492167360 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb908bf060 con 0x55bb90995b70 2024-03-15T13:05:16.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba03fff640 1 --2- 172.21.15.2:0/2492167360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb90995b70 0x55bb9099e210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40114f0 tx=0x7fb9f40119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a667e94fc66fd75c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fba08de4640 1 --2- 172.21.15.2:0/2253364216 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb9ec0a1ba0 0x7fb9ec0a4060 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc004af0 tx=0x7fb9fc02c0f0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.382+0000 7fb9ea7fc640 1 -- 172.21.15.2:0/2492167360 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4004030 con 0x55bb90995b70 2024-03-15T13:05:16.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.383+0000 7fb9ea7fc640 1 -- 172.21.15.2:0/2492167360 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb9f40035f0 con 0x55bb90995b70 2024-03-15T13:05:16.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.383+0000 7fb9ea7fc640 1 -- 172.21.15.2:0/2492167360 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f400f8f0 con 0x55bb90995b70 2024-03-15T13:05:16.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.383+0000 7fba0d427740 1 -- 172.21.15.2:0/2492167360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb90995b70 msgr2=0x55bb9099e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.383+0000 7fba0d427740 1 --2- 172.21.15.2:0/2492167360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb90995b70 0x55bb9099e210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40114f0 tx=0x7fb9f40119b0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.383+0000 7fba0d427740 1 -- 172.21.15.2:0/2492167360 shutdown_connections 2024-03-15T13:05:16.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.383+0000 7fba0d427740 1 --2- 172.21.15.2:0/2492167360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb9099e750 0x55bb909a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.383+0000 7fba0d427740 1 --2- 172.21.15.2:0/2492167360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb90995b70 0x55bb9099e210 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.383+0000 7fba0d427740 1 --2- 172.21.15.2:0/2492167360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb90995630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.383+0000 7fba0d427740 1 -- 172.21.15.2:0/2492167360 >> 172.21.15.2:0/2492167360 conn(0x55bb9098f980 msgr2=0x55bb9098fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.383+0000 7fba0d427740 1 -- 172.21.15.2:0/2492167360 shutdown_connections 2024-03-15T13:05:16.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.383+0000 7fba0d427740 1 -- 172.21.15.2:0/2492167360 wait complete. 2024-03-15T13:05:16.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.383+0000 7fba0d427740 1 -- start start 2024-03-15T13:05:16.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba0d427740 1 Processor -- start 2024-03-15T13:05:16.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba0d427740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb9099e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba0d427740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb90995b70 0x55bb9099b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba0d427740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb9099e750 0x55bb9099b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba0d427740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb908fd020 con 0x55bb9099e750 2024-03-15T13:05:16.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba0d427740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb905d0430 con 0x55bb90995b70 2024-03-15T13:05:16.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba0d427740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9058ce00 con 0x55bb90995250 2024-03-15T13:05:16.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba0d427740 10 client.0 _ll_register_callbacks cb 0x55bb908a4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:16.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba08de4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb9099e750 0x55bb9099b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba08de4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb9099e750 0x55bb9099b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58068/0 (socket says 172.21.15.2:58068) 2024-03-15T13:05:16.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba08de4640 1 -- 172.21.15.2:0/941179497 learned_addr learned my addr 172.21.15.2:0/941179497 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba037fe640 1 --2- 172.21.15.2:0/941179497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb90995b70 0x55bb9099b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba03fff640 1 --2- 172.21.15.2:0/941179497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb9099e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba08de4640 1 -- 172.21.15.2:0/941179497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 msgr2=0x55bb9099e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba08de4640 1 --2- 172.21.15.2:0/941179497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb9099e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba08de4640 1 -- 172.21.15.2:0/941179497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb90995b70 msgr2=0x55bb9099b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba08de4640 1 --2- 172.21.15.2:0/941179497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb90995b70 0x55bb9099b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.384+0000 7fba08de4640 1 -- 172.21.15.2:0/941179497 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f4009cd0 con 0x55bb9099e750 2024-03-15T13:05:16.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fba08de4640 1 --2- 172.21.15.2:0/941179497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb9099e750 0x55bb9099b930 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc036000 tx=0x7fb9fc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13de8ab21f4d837c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fb9e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:16.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fb9e37fe640 1 -- 172.21.15.2:0/941179497 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc003b70 con 0x55bb9099e750 2024-03-15T13:05:16.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fb9e37fe640 1 -- 172.21.15.2:0/941179497 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fb9fc003cd0 con 0x55bb9099e750 2024-03-15T13:05:16.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fba0d427740 1 -- 172.21.15.2:0/941179497 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb905fc7e0 con 0x55bb9099e750 2024-03-15T13:05:16.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fb9e37fe640 1 -- 172.21.15.2:0/941179497 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9fc003e90 con 0x55bb9099e750 2024-03-15T13:05:16.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fba0d427740 10 client.17565 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:16.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fba0d427740 1 -- 172.21.15.2:0/941179497 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55bb908fb6a0 con 0x55bb9099e750 2024-03-15T13:05:16.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fb9e37fe640 1 -- 172.21.15.2:0/941179497 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55bb905fc7e0 con 0x55bb9099e750 2024-03-15T13:05:16.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fb9e37fe640 1 -- 172.21.15.2:0/941179497 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb9fc055060 con 0x55bb9099e750 2024-03-15T13:05:16.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fba0d427740 10 client.17565 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:16.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fba0d427740 10 client.17565 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:05:16.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fba0d427740 20 client.17565 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:16.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fba0d427740 1 --2- 172.21.15.2:0/941179497 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55bb909ae920 0x55bb909b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fba0d427740 4 client.17565 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:16.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.385+0000 7fba0d427740 1 -- 172.21.15.2:0/941179497 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb90639720 con 0x55bb909ae920 2024-03-15T13:05:16.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.386+0000 7fba03fff640 1 --2- 172.21.15.2:0/941179497 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55bb909ae920 0x55bb909b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.386+0000 7fb9e37fe640 10 client.17565 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:05:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.386+0000 7fba03fff640 1 --2- 172.21.15.2:0/941179497 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55bb909ae920 0x55bb909b0d40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77f683ff52cfa9a7 server_cookie=3156e81dbbe7abf in_seq=0 out_seq=0 2024-03-15T13:05:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.387+0000 7fb9e37fe640 1 -- 172.21.15.2:0/941179497 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bb90639720 con 0x55bb909ae920 2024-03-15T13:05:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.387+0000 7fb9e37fe640 10 client.17565 handle_command_reply: tid=0 2024-03-15T13:05:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.387+0000 7fba0d427740 1 client.17565 shutdown 2024-03-15T13:05:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.387+0000 7fba0d427740 1 -- 172.21.15.2:0/941179497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb9099e750 msgr2=0x55bb9099b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.387+0000 7fba0d427740 1 --2- 172.21.15.2:0/941179497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb9099e750 0x55bb9099b930 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc036000 tx=0x7fb9fc0095c0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.387+0000 7fba0d427740 1 -- 172.21.15.2:0/941179497 shutdown_connections 2024-03-15T13:05:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.387+0000 7fba0d427740 1 --2- 172.21.15.2:0/941179497 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55bb909ae920 0x55bb909b0d40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.387+0000 7fba0d427740 1 --2- 172.21.15.2:0/941179497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb9099e750 0x55bb9099b930 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.387+0000 7fba0d427740 1 --2- 172.21.15.2:0/941179497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb90995b70 0x55bb9099b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.387+0000 7fb9e37fe640 0 client.17565 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:05:16.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.387+0000 7fba0d427740 1 --2- 172.21.15.2:0/941179497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb9099e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.387+0000 7fba0d427740 1 -- 172.21.15.2:0/941179497 >> 172.21.15.2:0/941179497 conn(0x55bb9098f980 msgr2=0x55bb909916d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.387+0000 7fba0d427740 1 -- 172.21.15.2:0/941179497 shutdown_connections 2024-03-15T13:05:16.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.387+0000 7fba0d427740 1 -- 172.21.15.2:0/941179497 wait complete. 2024-03-15T13:05:16.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.387+0000 7fba0d427740 20 client.17565 trim_cache size 0 max 16384 2024-03-15T13:05:16.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.399+0000 7fba0d427740 1 Processor -- start 2024-03-15T13:05:16.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba0d427740 1 -- start start 2024-03-15T13:05:16.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba0d427740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb9099db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba0d427740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb90995b70 0x55bb9099e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba0d427740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb9099e750 0x55bb909a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba0d427740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb90882130 con 0x55bb90995b70 2024-03-15T13:05:16.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba0d427740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb909a0f30 con 0x55bb9099e750 2024-03-15T13:05:16.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba0d427740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9099bc90 con 0x55bb90995250 2024-03-15T13:05:16.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba037fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb90995b70 0x55bb9099e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba037fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb90995b70 0x55bb9099e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58070/0 (socket says 172.21.15.2:58070) 2024-03-15T13:05:16.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba037fe640 1 -- 172.21.15.2:0/2735027476 learned_addr learned my addr 172.21.15.2:0/2735027476 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba03fff640 1 --2- 172.21.15.2:0/2735027476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb9099db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba08de4640 1 --2- 172.21.15.2:0/2735027476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb9099e750 0x55bb909a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba037fe640 1 -- 172.21.15.2:0/2735027476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 msgr2=0x55bb9099db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba037fe640 1 --2- 172.21.15.2:0/2735027476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb9099db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba037fe640 1 -- 172.21.15.2:0/2735027476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb9099e750 msgr2=0x55bb909a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba037fe640 1 --2- 172.21.15.2:0/2735027476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb9099e750 0x55bb909a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba037fe640 1 -- 172.21.15.2:0/2735027476 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb90897c20 con 0x55bb90995b70 2024-03-15T13:05:16.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fba037fe640 1 --2- 172.21.15.2:0/2735027476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb90995b70 0x55bb9099e060 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8004610 tx=0x7fb9f8085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd4a16bda125203 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fb9e9ffb640 1 -- 172.21.15.2:0/2735027476 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80cfea0 con 0x55bb90995b70 2024-03-15T13:05:16.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fb9e9ffb640 1 -- 172.21.15.2:0/2735027476 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb9f8099d50 con 0x55bb90995b70 2024-03-15T13:05:16.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.400+0000 7fb9e9ffb640 1 -- 172.21.15.2:0/2735027476 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8004e10 con 0x55bb90995b70 2024-03-15T13:05:16.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.401+0000 7fba0d427740 1 -- 172.21.15.2:0/2735027476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb90995b70 msgr2=0x55bb9099e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.401+0000 7fba0d427740 1 --2- 172.21.15.2:0/2735027476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb90995b70 0x55bb9099e060 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8004610 tx=0x7fb9f8085040 comp rx=0 tx=0).stop 2024-03-15T13:05:16.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.401+0000 7fba0d427740 1 -- 172.21.15.2:0/2735027476 shutdown_connections 2024-03-15T13:05:16.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.401+0000 7fba0d427740 1 --2- 172.21.15.2:0/2735027476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb9099e750 0x55bb909a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.401+0000 7fba0d427740 1 --2- 172.21.15.2:0/2735027476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb90995b70 0x55bb9099e060 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.401+0000 7fba0d427740 1 --2- 172.21.15.2:0/2735027476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb9099db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.401+0000 7fba0d427740 1 -- 172.21.15.2:0/2735027476 >> 172.21.15.2:0/2735027476 conn(0x55bb9098f980 msgr2=0x55bb90991610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.401+0000 7fba0d427740 1 -- 172.21.15.2:0/2735027476 shutdown_connections 2024-03-15T13:05:16.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.401+0000 7fba0d427740 1 -- 172.21.15.2:0/2735027476 wait complete. 2024-03-15T13:05:16.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.401+0000 7fba0d427740 1 -- start start 2024-03-15T13:05:16.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba0d427740 1 Processor -- start 2024-03-15T13:05:16.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba0d427740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb909a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba0d427740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb90995b70 0x55bb909a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba0d427740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb9099e750 0x55bb909a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba0d427740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb90a0a1e0 con 0x55bb90995b70 2024-03-15T13:05:16.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba0d427740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb90a0a320 con 0x55bb9099e750 2024-03-15T13:05:16.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba0d427740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb90a0a460 con 0x55bb90995250 2024-03-15T13:05:16.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba0d427740 10 client.0 _ll_register_callbacks cb 0x7fb9f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:16.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba037fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb90995b70 0x55bb909a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba037fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb90995b70 0x55bb909a2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58080/0 (socket says 172.21.15.2:58080) 2024-03-15T13:05:16.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba037fe640 1 -- 172.21.15.2:0/1892295324 learned_addr learned my addr 172.21.15.2:0/1892295324 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba03fff640 1 --2- 172.21.15.2:0/1892295324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb909a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba08de4640 1 --2- 172.21.15.2:0/1892295324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb9099e750 0x55bb909a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba037fe640 1 -- 172.21.15.2:0/1892295324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 msgr2=0x55bb909a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba037fe640 1 --2- 172.21.15.2:0/1892295324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb909a2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba037fe640 1 -- 172.21.15.2:0/1892295324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb9099e750 msgr2=0x55bb909a34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba037fe640 1 --2- 172.21.15.2:0/1892295324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb9099e750 0x55bb909a34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba037fe640 1 -- 172.21.15.2:0/1892295324 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba041a4710 con 0x55bb90995b70 2024-03-15T13:05:16.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba08de4640 1 --2- 172.21.15.2:0/1892295324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb9099e750 0x55bb909a34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:05:16.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba03fff640 1 --2- 172.21.15.2:0/1892295324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb909a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:05:16.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba037fe640 1 --2- 172.21.15.2:0/1892295324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb90995b70 0x55bb909a2fa0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8096dc0 tx=0x7fb9f80a2b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0623726bea0aefc server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fb9e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:16.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fb9e37fe640 1 -- 172.21.15.2:0/1892295324 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f809a3f0 con 0x55bb90995b70 2024-03-15T13:05:16.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fb9e37fe640 1 -- 172.21.15.2:0/1892295324 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fb9f809e540 con 0x55bb90995b70 2024-03-15T13:05:16.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.402+0000 7fba0d427740 1 -- 172.21.15.2:0/1892295324 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9f40035f0 con 0x55bb90995b70 2024-03-15T13:05:16.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.403+0000 7fb9e37fe640 1 -- 172.21.15.2:0/1892295324 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9f809aaf0 con 0x55bb90995b70 2024-03-15T13:05:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.403+0000 7fba0d427740 10 client.17571 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.403+0000 7fba0d427740 1 -- 172.21.15.2:0/1892295324 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55bb908bd8c0 con 0x55bb90995b70 2024-03-15T13:05:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.403+0000 7fb9e37fe640 1 -- 172.21.15.2:0/1892295324 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb9f40035f0 con 0x55bb90995b70 2024-03-15T13:05:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.403+0000 7fb9e37fe640 1 -- 172.21.15.2:0/1892295324 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fb9f80a2700 con 0x55bb90995b70 2024-03-15T13:05:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.403+0000 7fba0d427740 10 client.17571 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.403+0000 7fba0d427740 10 client.17571 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:05:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.403+0000 7fba0d427740 20 client.17571 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.403+0000 7fba0d427740 1 --2- 172.21.15.2:0/1892295324 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55bb90a0ba00 0x55bb90a0de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.403+0000 7fba0d427740 4 client.17571 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:05:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.403+0000 7fba0d427740 1 -- 172.21.15.2:0/1892295324 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bb90688890 con 0x55bb90a0ba00 2024-03-15T13:05:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.403+0000 7fba03fff640 1 --2- 172.21.15.2:0/1892295324 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55bb90a0ba00 0x55bb90a0de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.403+0000 7fba03fff640 1 --2- 172.21.15.2:0/1892295324 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55bb90a0ba00 0x55bb90a0de20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf512f037889c6a8 server_cookie=4ac83e1215a887c in_seq=0 out_seq=0 2024-03-15T13:05:16.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.403+0000 7fb9e37fe640 10 client.17571 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:05:16.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fb9e37fe640 1 -- 172.21.15.2:0/1892295324 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bb90688890 con 0x55bb90a0ba00 2024-03-15T13:05:16.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fb9e37fe640 10 client.17571 handle_command_reply: tid=0 2024-03-15T13:05:16.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0d427740 1 client.17571 shutdown 2024-03-15T13:05:16.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0d427740 1 -- 172.21.15.2:0/1892295324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb90995b70 msgr2=0x55bb909a2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0d427740 1 --2- 172.21.15.2:0/1892295324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb90995b70 0x55bb909a2fa0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8096dc0 tx=0x7fb9f80a2b90 comp rx=0 tx=0).stop 2024-03-15T13:05:16.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0d427740 1 -- 172.21.15.2:0/1892295324 shutdown_connections 2024-03-15T13:05:16.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0d427740 1 --2- 172.21.15.2:0/1892295324 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55bb90a0ba00 0x55bb90a0de20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0d427740 1 --2- 172.21.15.2:0/1892295324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb9099e750 0x55bb909a34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0d427740 1 --2- 172.21.15.2:0/1892295324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bb90995b70 0x55bb909a2fa0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fb9e37fe640 0 client.17571 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:05:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0d427740 1 --2- 172.21.15.2:0/1892295324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bb90995250 0x55bb909a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0d427740 1 -- 172.21.15.2:0/1892295324 >> 172.21.15.2:0/1892295324 conn(0x55bb9098f980 msgr2=0x55bb9099ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0d427740 1 -- 172.21.15.2:0/1892295324 shutdown_connections 2024-03-15T13:05:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0d427740 1 -- 172.21.15.2:0/1892295324 wait complete. 2024-03-15T13:05:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0d427740 20 client.17571 trim_cache size 0 max 16384 2024-03-15T13:05:16.399 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:05:16.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0a847640 1 -- 172.21.15.2:0/2253364216 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb9ec0a1ba0 msgr2=0x7fb9ec0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0a847640 1 --2- 172.21.15.2:0/2253364216 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb9ec0a1ba0 0x7fb9ec0a4060 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc004af0 tx=0x7fb9fc02c0f0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0a847640 1 -- 172.21.15.2:0/2253364216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba04101e60 msgr2=0x7fba0419dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0a847640 1 --2- 172.21.15.2:0/2253364216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba04101e60 0x7fba0419dc20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800e9f0 tx=0x7fb9f800eeb0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0a847640 1 -- 172.21.15.2:0/2253364216 shutdown_connections 2024-03-15T13:05:16.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0a847640 1 --2- 172.21.15.2:0/2253364216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba04107560 0x7fba04197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0a847640 1 --2- 172.21.15.2:0/2253364216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba04101e60 0x7fba0419dc20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0a847640 1 --2- 172.21.15.2:0/2253364216 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb9ec0a1ba0 0x7fb9ec0a4060 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.404+0000 7fba0a847640 1 --2- 172.21.15.2:0/2253364216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba041014a0 0x7fba0419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.405+0000 7fba0a847640 1 -- 172.21.15.2:0/2253364216 >> 172.21.15.2:0/2253364216 conn(0x7fba040788f0 msgr2=0x7fba040fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.405+0000 7fba0a847640 1 -- 172.21.15.2:0/2253364216 shutdown_connections 2024-03-15T13:05:16.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.405+0000 7fba0a847640 1 -- 172.21.15.2:0/2253364216 wait complete. 2024-03-15T13:05:16.402 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:05:16.402 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:05:16.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.522+0000 7fca77f6c640 1 -- 172.21.15.2:0/2780052515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca701053a0 msgr2=0x7fca70105780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.522+0000 7fca77f6c640 1 --2- 172.21.15.2:0/2780052515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca701053a0 0x7fca70105780 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fca68009900 tx=0x7fca6802bc80 comp rx=0 tx=0).stop 2024-03-15T13:05:16.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.522+0000 7fca77f6c640 1 -- 172.21.15.2:0/2780052515 shutdown_connections 2024-03-15T13:05:16.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.522+0000 7fca77f6c640 1 --2- 172.21.15.2:0/2780052515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca701014e0 0x7fca7010d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.522+0000 7fca77f6c640 1 --2- 172.21.15.2:0/2780052515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca70100b40 0x7fca70100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.522+0000 7fca77f6c640 1 --2- 172.21.15.2:0/2780052515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca701053a0 0x7fca70105780 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.522+0000 7fca77f6c640 1 -- 172.21.15.2:0/2780052515 >> 172.21.15.2:0/2780052515 conn(0x7fca700769b0 msgr2=0x7fca700fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.523+0000 7fca77f6c640 1 -- 172.21.15.2:0/2780052515 shutdown_connections 2024-03-15T13:05:16.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.523+0000 7fca77f6c640 1 -- 172.21.15.2:0/2780052515 wait complete. 2024-03-15T13:05:16.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.523+0000 7fca77f6c640 1 Processor -- start 2024-03-15T13:05:16.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.523+0000 7fca77f6c640 1 -- start start 2024-03-15T13:05:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.523+0000 7fca77f6c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca70100b40 0x7fca701989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.523+0000 7fca77f6c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca701014e0 0x7fca70198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.523+0000 7fca77f6c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca701053a0 0x7fca701a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.523+0000 7fca77f6c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca701997b0 con 0x7fca701053a0 2024-03-15T13:05:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.523+0000 7fca77f6c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca701a3fb0 con 0x7fca701014e0 2024-03-15T13:05:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.523+0000 7fca77f6c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca701a40f0 con 0x7fca70100b40 2024-03-15T13:05:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.523+0000 7fca76509640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca701053a0 0x7fca701a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.523+0000 7fca76509640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca701053a0 0x7fca701a3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58102/0 (socket says 172.21.15.2:58102) 2024-03-15T13:05:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.523+0000 7fca76509640 1 -- 172.21.15.2:0/2156135653 learned_addr learned my addr 172.21.15.2:0/2156135653 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.523+0000 7fca75d08640 1 --2- 172.21.15.2:0/2156135653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca70100b40 0x7fca701989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.523+0000 7fca75507640 1 --2- 172.21.15.2:0/2156135653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca701014e0 0x7fca70198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.524+0000 7fca76509640 1 -- 172.21.15.2:0/2156135653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca70100b40 msgr2=0x7fca701989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.524+0000 7fca76509640 1 --2- 172.21.15.2:0/2156135653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca70100b40 0x7fca701989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.524+0000 7fca76509640 1 -- 172.21.15.2:0/2156135653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca701014e0 msgr2=0x7fca70198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.524+0000 7fca76509640 1 --2- 172.21.15.2:0/2156135653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca701014e0 0x7fca70198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.524+0000 7fca76509640 1 -- 172.21.15.2:0/2156135653 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca68009560 con 0x7fca701053a0 2024-03-15T13:05:16.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.524+0000 7fca76509640 1 --2- 172.21.15.2:0/2156135653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca701053a0 0x7fca701a3a70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fca6000eb40 tx=0x7fca6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.524+0000 7fca6effd640 1 -- 172.21.15.2:0/2156135653 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6000ceb0 con 0x7fca701053a0 2024-03-15T13:05:16.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.524+0000 7fca6effd640 1 -- 172.21.15.2:0/2156135653 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fca60005020 con 0x7fca701053a0 2024-03-15T13:05:16.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.524+0000 7fca77f6c640 1 -- 172.21.15.2:0/2156135653 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca701a4350 con 0x7fca701053a0 2024-03-15T13:05:16.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.524+0000 7fca6effd640 1 -- 172.21.15.2:0/2156135653 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca60010670 con 0x7fca701053a0 2024-03-15T13:05:16.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.524+0000 7fca77f6c640 1 -- 172.21.15.2:0/2156135653 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca701a48b0 con 0x7fca701053a0 2024-03-15T13:05:16.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.526+0000 7fca6effd640 1 -- 172.21.15.2:0/2156135653 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fca60005190 con 0x7fca701053a0 2024-03-15T13:05:16.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.526+0000 7fca7ac25740 1 Processor -- start 2024-03-15T13:05:16.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca7ac25740 1 -- start start 2024-03-15T13:05:16.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca7ac25740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 0x55c2b94a2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca7ac25740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 0x55c2b94ab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca7ac25740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94ab750 0x55c2b94adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca7ac25740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b945d960 con 0x55c2b94ab750 2024-03-15T13:05:16.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca7ac25740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b9411310 con 0x55c2b94a2250 2024-03-15T13:05:16.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca7ac25740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b940e840 con 0x55c2b94a2b70 2024-03-15T13:05:16.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca6effd640 1 --2- 172.21.15.2:0/2156135653 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fca580a1c20 0x7fca580a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca75507640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 0x55c2b94a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca76509640 1 --2- 172.21.15.2:0/2156135653 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fca580a1c20 0x7fca580a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca75d08640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 0x55c2b94ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca75507640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 0x55c2b94a2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44152/0 (socket says 172.21.15.2:44152) 2024-03-15T13:05:16.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca75507640 1 -- 172.21.15.2:0/2860246506 learned_addr learned my addr 172.21.15.2:0/2860246506 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca75507640 1 --2- 172.21.15.2:0/2860246506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94ab750 0x55c2b94adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca6effd640 1 -- 172.21.15.2:0/2156135653 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fca60014070 con 0x7fca701053a0 2024-03-15T13:05:16.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca75507640 1 -- 172.21.15.2:0/2860246506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 msgr2=0x55c2b94ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca75507640 1 --2- 172.21.15.2:0/2860246506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 0x55c2b94ab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca75507640 1 -- 172.21.15.2:0/2860246506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94ab750 msgr2=0x55c2b94adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca75507640 1 --2- 172.21.15.2:0/2860246506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94ab750 0x55c2b94adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.527+0000 7fca75507640 1 -- 172.21.15.2:0/2860246506 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2b93cc060 con 0x55c2b94a2250 2024-03-15T13:05:16.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.528+0000 7fca75507640 1 --2- 172.21.15.2:0/2860246506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 0x55c2b94a2630 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fca640058c0 tx=0x7fca64005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a30ae523b49eae server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.528+0000 7fca76509640 1 --2- 172.21.15.2:0/2156135653 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fca580a1c20 0x7fca580a40e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fca600a2c10 tx=0x7fca6006cba0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.528+0000 7fca57fff640 1 -- 172.21.15.2:0/2860246506 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64004030 con 0x55c2b94a2250 2024-03-15T13:05:16.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.528+0000 7fca57fff640 1 -- 172.21.15.2:0/2860246506 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fca64005020 con 0x55c2b94a2250 2024-03-15T13:05:16.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.528+0000 7fca57fff640 1 -- 172.21.15.2:0/2860246506 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64004ca0 con 0x55c2b94a2250 2024-03-15T13:05:16.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 -- 172.21.15.2:0/2860246506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 msgr2=0x55c2b94a2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 --2- 172.21.15.2:0/2860246506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 0x55c2b94a2630 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fca640058c0 tx=0x7fca64005850 comp rx=0 tx=0).stop 2024-03-15T13:05:16.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 -- 172.21.15.2:0/2860246506 shutdown_connections 2024-03-15T13:05:16.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 --2- 172.21.15.2:0/2860246506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94ab750 0x55c2b94adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 --2- 172.21.15.2:0/2860246506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 0x55c2b94ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 --2- 172.21.15.2:0/2860246506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 0x55c2b94a2630 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 -- 172.21.15.2:0/2860246506 >> 172.21.15.2:0/2860246506 conn(0x55c2b949c980 msgr2=0x55c2b949cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 -- 172.21.15.2:0/2860246506 shutdown_connections 2024-03-15T13:05:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 -- 172.21.15.2:0/2860246506 wait complete. 2024-03-15T13:05:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 -- start start 2024-03-15T13:05:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 Processor -- start 2024-03-15T13:05:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 0x55c2b94ab110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94a2b70 0x55c2b94a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94ab750 0x55c2b94a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b940a020 con 0x55c2b94a2b70 2024-03-15T13:05:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b90dd430 con 0x55c2b94a2250 2024-03-15T13:05:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b9099e00 con 0x55c2b94ab750 2024-03-15T13:05:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.529+0000 7fca7ac25740 10 client.0 _ll_register_callbacks cb 0x55c2b93b1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca75d08640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94a2b70 0x55c2b94a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca75d08640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94a2b70 0x55c2b94a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58128/0 (socket says 172.21.15.2:58128) 2024-03-15T13:05:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca75d08640 1 -- 172.21.15.2:0/1408930041 learned_addr learned my addr 172.21.15.2:0/1408930041 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca75507640 1 --2- 172.21.15.2:0/1408930041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 0x55c2b94ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca75507640 1 --2- 172.21.15.2:0/1408930041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94ab750 0x55c2b94a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca75d08640 1 -- 172.21.15.2:0/1408930041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94ab750 msgr2=0x55c2b94a8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca75d08640 1 --2- 172.21.15.2:0/1408930041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94ab750 0x55c2b94a8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca75d08640 1 -- 172.21.15.2:0/1408930041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 msgr2=0x55c2b94ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca75d08640 1 --2- 172.21.15.2:0/1408930041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 0x55c2b94ab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca75d08640 1 -- 172.21.15.2:0/1408930041 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca64019040 con 0x55c2b94a2b70 2024-03-15T13:05:16.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca75d08640 1 --2- 172.21.15.2:0/1408930041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94a2b70 0x55c2b94a84a0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fca68036040 tx=0x7fca6802ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c313cabb848a0d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca557fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:16.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca557fa640 1 -- 172.21.15.2:0/1408930041 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68004040 con 0x55c2b94a2b70 2024-03-15T13:05:16.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca557fa640 1 -- 172.21.15.2:0/1408930041 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fca680041a0 con 0x55c2b94a2b70 2024-03-15T13:05:16.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca7ac25740 1 -- 172.21.15.2:0/1408930041 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2b91097e0 con 0x55c2b94a2b70 2024-03-15T13:05:16.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca557fa640 1 -- 172.21.15.2:0/1408930041 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca68004360 con 0x55c2b94a2b70 2024-03-15T13:05:16.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca7ac25740 10 client.17583 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.530+0000 7fca7ac25740 1 -- 172.21.15.2:0/1408930041 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55c2b94086a0 con 0x55c2b94a2b70 2024-03-15T13:05:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.531+0000 7fca557fa640 1 -- 172.21.15.2:0/1408930041 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fca68052080 con 0x55c2b94a2b70 2024-03-15T13:05:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.531+0000 7fca557fa640 1 -- 172.21.15.2:0/1408930041 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fca6803c040 con 0x55c2b94a2b70 2024-03-15T13:05:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.531+0000 7fca7ac25740 10 client.17583 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.531+0000 7fca7ac25740 10 client.17583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.531+0000 7fca7ac25740 20 client.17583 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.531+0000 7fca7ac25740 1 --2- 172.21.15.2:0/1408930041 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2b94bb880 0x55c2b94bdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.531+0000 7fca7ac25740 4 client.17583 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.531+0000 7fca7ac25740 1 -- 172.21.15.2:0/1408930041 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2b9146720 con 0x55c2b94bb880 2024-03-15T13:05:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.531+0000 7fca75507640 1 --2- 172.21.15.2:0/1408930041 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2b94bb880 0x55c2b94bdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.531+0000 7fca557fa640 10 client.17583 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.531+0000 7fca75507640 1 --2- 172.21.15.2:0/1408930041 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2b94bb880 0x55c2b94bdca0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e42db181570e0e7a server_cookie=874fbce16ab65406 in_seq=0 out_seq=0 2024-03-15T13:05:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.532+0000 7fca557fa640 1 -- 172.21.15.2:0/1408930041 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c2b9146720 con 0x55c2b94bb880 2024-03-15T13:05:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.532+0000 7fca557fa640 10 client.17583 handle_command_reply: tid=0 2024-03-15T13:05:16.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.532+0000 7fca7ac25740 1 client.17583 shutdown 2024-03-15T13:05:16.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.532+0000 7fca7ac25740 1 -- 172.21.15.2:0/1408930041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94a2b70 msgr2=0x55c2b94a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.532+0000 7fca7ac25740 1 --2- 172.21.15.2:0/1408930041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94a2b70 0x55c2b94a84a0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fca68036040 tx=0x7fca6802ca30 comp rx=0 tx=0).stop 2024-03-15T13:05:16.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.533+0000 7fca7ac25740 1 -- 172.21.15.2:0/1408930041 shutdown_connections 2024-03-15T13:05:16.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.533+0000 7fca7ac25740 1 --2- 172.21.15.2:0/1408930041 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2b94bb880 0x55c2b94bdca0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.533+0000 7fca7ac25740 1 --2- 172.21.15.2:0/1408930041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94ab750 0x55c2b94a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.533+0000 7fca7ac25740 1 --2- 172.21.15.2:0/1408930041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94a2b70 0x55c2b94a84a0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.533+0000 7fca7ac25740 1 --2- 172.21.15.2:0/1408930041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 0x55c2b94ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.533+0000 7fca557fa640 0 client.17583 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:16.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.533+0000 7fca7ac25740 1 -- 172.21.15.2:0/1408930041 >> 172.21.15.2:0/1408930041 conn(0x55c2b949c980 msgr2=0x55c2b949e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.533+0000 7fca7ac25740 1 -- 172.21.15.2:0/1408930041 shutdown_connections 2024-03-15T13:05:16.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.533+0000 7fca7ac25740 1 -- 172.21.15.2:0/1408930041 wait complete. 2024-03-15T13:05:16.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.533+0000 7fca7ac25740 20 client.17583 trim_cache size 0 max 16384 2024-03-15T13:05:16.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.545+0000 7fca7ac25740 1 Processor -- start 2024-03-15T13:05:16.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.545+0000 7fca7ac25740 1 -- start start 2024-03-15T13:05:16.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.545+0000 7fca7ac25740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94a2250 0x55c2b949ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.545+0000 7fca7ac25740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 0x55c2b94aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.545+0000 7fca7ac25740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94ab750 0x55c2b94ab110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.545+0000 7fca7ac25740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b938f130 con 0x55c2b94a2250 2024-03-15T13:05:16.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.545+0000 7fca7ac25740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b94a15d0 con 0x55c2b94ab750 2024-03-15T13:05:16.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.545+0000 7fca7ac25740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b94be300 con 0x55c2b94a2b70 2024-03-15T13:05:16.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.545+0000 7fca75507640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94a2250 0x55c2b949ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.545+0000 7fca75507640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94a2250 0x55c2b949ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58132/0 (socket says 172.21.15.2:58132) 2024-03-15T13:05:16.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.545+0000 7fca75507640 1 -- 172.21.15.2:0/51567854 learned_addr learned my addr 172.21.15.2:0/51567854 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.545+0000 7fca75d08640 1 --2- 172.21.15.2:0/51567854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 0x55c2b94aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.545+0000 7fca75507640 1 --2- 172.21.15.2:0/51567854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94ab750 0x55c2b94ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.546+0000 7fca75507640 1 -- 172.21.15.2:0/51567854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 msgr2=0x55c2b94aabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.546+0000 7fca75507640 1 --2- 172.21.15.2:0/51567854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 0x55c2b94aabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.546+0000 7fca75507640 1 -- 172.21.15.2:0/51567854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94ab750 msgr2=0x55c2b94ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.546+0000 7fca75507640 1 --2- 172.21.15.2:0/51567854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94ab750 0x55c2b94ab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.546+0000 7fca75507640 1 -- 172.21.15.2:0/51567854 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2b93a4c20 con 0x55c2b94a2250 2024-03-15T13:05:16.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.546+0000 7fca75507640 1 --2- 172.21.15.2:0/51567854 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94a2250 0x55c2b949ba00 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fca640117f0 tx=0x7fca64004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41a09aa627cd74f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.546+0000 7fca577fe640 1 -- 172.21.15.2:0/51567854 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64011250 con 0x55c2b94a2250 2024-03-15T13:05:16.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.546+0000 7fca577fe640 1 -- 172.21.15.2:0/51567854 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fca64005660 con 0x55c2b94a2250 2024-03-15T13:05:16.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.546+0000 7fca577fe640 1 -- 172.21.15.2:0/51567854 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64032d80 con 0x55c2b94a2250 2024-03-15T13:05:16.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.546+0000 7fca7ac25740 1 -- 172.21.15.2:0/51567854 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94a2250 msgr2=0x55c2b949ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.546+0000 7fca7ac25740 1 --2- 172.21.15.2:0/51567854 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94a2250 0x55c2b949ba00 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fca640117f0 tx=0x7fca64004840 comp rx=0 tx=0).stop 2024-03-15T13:05:16.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.546+0000 7fca7ac25740 1 -- 172.21.15.2:0/51567854 shutdown_connections 2024-03-15T13:05:16.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.546+0000 7fca7ac25740 1 --2- 172.21.15.2:0/51567854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94ab750 0x55c2b94ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.546+0000 7fca7ac25740 1 --2- 172.21.15.2:0/51567854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 0x55c2b94aabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.546+0000 7fca7ac25740 1 --2- 172.21.15.2:0/51567854 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94a2250 0x55c2b949ba00 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca7ac25740 1 -- 172.21.15.2:0/51567854 >> 172.21.15.2:0/51567854 conn(0x55c2b949c980 msgr2=0x55c2b949b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca7ac25740 1 -- 172.21.15.2:0/51567854 shutdown_connections 2024-03-15T13:05:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca7ac25740 1 -- 172.21.15.2:0/51567854 wait complete. 2024-03-15T13:05:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca7ac25740 1 -- start start 2024-03-15T13:05:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca7ac25740 1 Processor -- start 2024-03-15T13:05:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca7ac25740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 0x55c2b950f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca7ac25740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 0x55c2b950f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca7ac25740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94ab750 0x55c2b94a5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca7ac25740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b94a59b0 con 0x55c2b94ab750 2024-03-15T13:05:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca7ac25740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b94a5b20 con 0x55c2b94a2250 2024-03-15T13:05:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca7ac25740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2b94a5ce0 con 0x55c2b94a2b70 2024-03-15T13:05:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca7ac25740 10 client.0 _ll_register_callbacks cb 0x7fca68009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca75507640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94ab750 0x55c2b94a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca75507640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94ab750 0x55c2b94a5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58136/0 (socket says 172.21.15.2:58136) 2024-03-15T13:05:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca75507640 1 -- 172.21.15.2:0/2536116163 learned_addr learned my addr 172.21.15.2:0/2536116163 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca75d08640 1 --2- 172.21.15.2:0/2536116163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 0x55c2b950f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.547+0000 7fca75507640 1 --2- 172.21.15.2:0/2536116163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 0x55c2b950f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.548+0000 7fca75507640 1 -- 172.21.15.2:0/2536116163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 msgr2=0x55c2b950f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.548+0000 7fca75507640 1 --2- 172.21.15.2:0/2536116163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 0x55c2b950f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.548+0000 7fca75507640 1 -- 172.21.15.2:0/2536116163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 msgr2=0x55c2b950f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.548+0000 7fca75507640 1 --2- 172.21.15.2:0/2536116163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 0x55c2b950f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.548+0000 7fca75507640 1 -- 172.21.15.2:0/2536116163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2b93cc060 con 0x55c2b94ab750 2024-03-15T13:05:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.548+0000 7fca75d08640 1 --2- 172.21.15.2:0/2536116163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 0x55c2b950f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.548+0000 7fca75507640 1 --2- 172.21.15.2:0/2536116163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94ab750 0x55c2b94a5420 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x55c2b94b08b0 tx=0x7fca64032960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22791b226fe4e47c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.548+0000 7fca557fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.548+0000 7fca557fa640 1 -- 172.21.15.2:0/2536116163 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64005390 con 0x55c2b94ab750 2024-03-15T13:05:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.548+0000 7fca557fa640 1 -- 172.21.15.2:0/2536116163 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fca640114d0 con 0x55c2b94ab750 2024-03-15T13:05:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.548+0000 7fca7ac25740 1 -- 172.21.15.2:0/2536116163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2b94980f0 con 0x55c2b94ab750 2024-03-15T13:05:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.548+0000 7fca557fa640 1 -- 172.21.15.2:0/2536116163 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca640054f0 con 0x55c2b94ab750 2024-03-15T13:05:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.548+0000 7fca7ac25740 10 client.17589 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.548+0000 7fca7ac25740 1 -- 172.21.15.2:0/2536116163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55c2b93ca8c0 con 0x55c2b94ab750 2024-03-15T13:05:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.548+0000 7fca557fa640 1 -- 172.21.15.2:0/2536116163 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55c2b94980f0 con 0x55c2b94ab750 2024-03-15T13:05:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.549+0000 7fca557fa640 1 -- 172.21.15.2:0/2536116163 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fca6401d070 con 0x55c2b94ab750 2024-03-15T13:05:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.549+0000 7fca7ac25740 10 client.17589 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.549+0000 7fca7ac25740 10 client.17589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.549+0000 7fca7ac25740 20 client.17589 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.549+0000 7fca7ac25740 1 --2- 172.21.15.2:0/2536116163 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2b95180b0 0x55c2b951a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.549+0000 7fca7ac25740 4 client.17589 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:05:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.549+0000 7fca7ac25740 1 -- 172.21.15.2:0/2536116163 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fca64005020 con 0x55c2b95180b0 2024-03-15T13:05:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.549+0000 7fca75d08640 1 --2- 172.21.15.2:0/2536116163 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2b95180b0 0x55c2b951a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.549+0000 7fca557fa640 10 client.17589 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.549+0000 7fca75d08640 1 --2- 172.21.15.2:0/2536116163 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2b95180b0 0x55c2b951a4d0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82890eee5e446c3c server_cookie=ba718a227e8461e8 in_seq=0 out_seq=0 2024-03-15T13:05:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.549+0000 7fca557fa640 1 -- 172.21.15.2:0/2536116163 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fca64005020 con 0x55c2b95180b0 2024-03-15T13:05:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.549+0000 7fca557fa640 10 client.17589 handle_command_reply: tid=0 2024-03-15T13:05:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.549+0000 7fca7ac25740 1 client.17589 shutdown 2024-03-15T13:05:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.549+0000 7fca7ac25740 1 -- 172.21.15.2:0/2536116163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94ab750 msgr2=0x55c2b94a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.549+0000 7fca7ac25740 1 --2- 172.21.15.2:0/2536116163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94ab750 0x55c2b94a5420 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x55c2b94b08b0 tx=0x7fca64032960 comp rx=0 tx=0).stop 2024-03-15T13:05:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca7ac25740 1 -- 172.21.15.2:0/2536116163 shutdown_connections 2024-03-15T13:05:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca7ac25740 1 --2- 172.21.15.2:0/2536116163 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55c2b95180b0 0x55c2b951a4d0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca7ac25740 1 --2- 172.21.15.2:0/2536116163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c2b94ab750 0x55c2b94a5420 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca557fa640 0 client.17589 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca7ac25740 1 --2- 172.21.15.2:0/2536116163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c2b94a2b70 0x55c2b950f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca7ac25740 1 --2- 172.21.15.2:0/2536116163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c2b94a2250 0x55c2b950f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca7ac25740 1 -- 172.21.15.2:0/2536116163 >> 172.21.15.2:0/2536116163 conn(0x55c2b949c980 msgr2=0x55c2b949bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca7ac25740 1 -- 172.21.15.2:0/2536116163 shutdown_connections 2024-03-15T13:05:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca7ac25740 1 -- 172.21.15.2:0/2536116163 wait complete. 2024-03-15T13:05:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca7ac25740 20 client.17589 trim_cache size 0 max 16384 2024-03-15T13:05:16.542 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:05:16.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca77f6c640 1 -- 172.21.15.2:0/2156135653 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fca580a1c20 msgr2=0x7fca580a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca77f6c640 1 --2- 172.21.15.2:0/2156135653 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fca580a1c20 0x7fca580a40e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fca600a2c10 tx=0x7fca6006cba0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca77f6c640 1 -- 172.21.15.2:0/2156135653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca701053a0 msgr2=0x7fca701a3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:16.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca77f6c640 1 --2- 172.21.15.2:0/2156135653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca701053a0 0x7fca701a3a70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fca6000eb40 tx=0x7fca6000c2d0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca77f6c640 1 -- 172.21.15.2:0/2156135653 shutdown_connections 2024-03-15T13:05:16.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca77f6c640 1 --2- 172.21.15.2:0/2156135653 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fca580a1c20 0x7fca580a40e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca77f6c640 1 --2- 172.21.15.2:0/2156135653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca701053a0 0x7fca701a3a70 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca77f6c640 1 --2- 172.21.15.2:0/2156135653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca701014e0 0x7fca70198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca77f6c640 1 --2- 172.21.15.2:0/2156135653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca70100b40 0x7fca701989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:16.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca77f6c640 1 -- 172.21.15.2:0/2156135653 >> 172.21.15.2:0/2156135653 conn(0x7fca700769b0 msgr2=0x7fca701033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:16.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca77f6c640 1 -- 172.21.15.2:0/2156135653 shutdown_connections 2024-03-15T13:05:16.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:16.550+0000 7fca77f6c640 1 -- 172.21.15.2:0/2156135653 wait complete. 2024-03-15T13:05:16.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:16 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2432391206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:05:17.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:16 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2432391206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:05:17.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:16 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2432391206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:05:17.547 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:05:17.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:05:17.547 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:05:17.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.681+0000 7f657add4640 1 -- 172.21.15.2:0/293468869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65741064b0 msgr2=0x7f6574106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.681+0000 7f657add4640 1 --2- 172.21.15.2:0/293468869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65741064b0 0x7f6574106890 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f6568009930 tx=0x7f656802bc30 comp rx=0 tx=0).stop 2024-03-15T13:05:17.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.681+0000 7f657add4640 1 -- 172.21.15.2:0/293468869 shutdown_connections 2024-03-15T13:05:17.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.681+0000 7f657add4640 1 --2- 172.21.15.2:0/293468869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6574100dd0 0x7f657410d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.681+0000 7f657add4640 1 --2- 172.21.15.2:0/293468869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6574100410 0x7f6574100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.681+0000 7f657add4640 1 --2- 172.21.15.2:0/293468869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65741064b0 0x7f6574106890 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.681+0000 7f657add4640 1 -- 172.21.15.2:0/293468869 >> 172.21.15.2:0/293468869 conn(0x7f6574077550 msgr2=0x7f6574077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:17.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.681+0000 7f657add4640 1 -- 172.21.15.2:0/293468869 shutdown_connections 2024-03-15T13:05:17.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.681+0000 7f657add4640 1 -- 172.21.15.2:0/293468869 wait complete. 2024-03-15T13:05:17.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f657add4640 1 Processor -- start 2024-03-15T13:05:17.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f657add4640 1 -- start start 2024-03-15T13:05:17.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f657add4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6574100410 0x7f657419d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f657add4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6574100dd0 0x7f657419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f657add4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65741064b0 0x7f6574197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f657add4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6574198060 con 0x7f6574100410 2024-03-15T13:05:17.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f657add4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65741981d0 con 0x7f65741064b0 2024-03-15T13:05:17.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f657add4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6574198390 con 0x7f6574100dd0 2024-03-15T13:05:17.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f6578b70640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6574100410 0x7f657419d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f6578b70640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6574100410 0x7f657419d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58148/0 (socket says 172.21.15.2:58148) 2024-03-15T13:05:17.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f6578b70640 1 -- 172.21.15.2:0/3825366463 learned_addr learned my addr 172.21.15.2:0/3825366463 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:17.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f6579371640 1 --2- 172.21.15.2:0/3825366463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65741064b0 0x7f6574197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f6573fff640 1 --2- 172.21.15.2:0/3825366463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6574100dd0 0x7f657419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f6578b70640 1 -- 172.21.15.2:0/3825366463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6574100dd0 msgr2=0x7f657419dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f6578b70640 1 --2- 172.21.15.2:0/3825366463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6574100dd0 0x7f657419dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f6578b70640 1 -- 172.21.15.2:0/3825366463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65741064b0 msgr2=0x7f6574197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f6578b70640 1 --2- 172.21.15.2:0/3825366463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65741064b0 0x7f6574197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.682+0000 7f6578b70640 1 -- 172.21.15.2:0/3825366463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6568009560 con 0x7f6574100410 2024-03-15T13:05:17.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.683+0000 7f6578b70640 1 --2- 172.21.15.2:0/3825366463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6574100410 0x7f657419d860 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6568009900 tx=0x7f656802ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:17.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.683+0000 7f6571ffb640 1 -- 172.21.15.2:0/3825366463 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656802cc10 con 0x7f6574100410 2024-03-15T13:05:17.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.683+0000 7f6571ffb640 1 -- 172.21.15.2:0/3825366463 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f65680054d0 con 0x7f6574100410 2024-03-15T13:05:17.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.683+0000 7f6571ffb640 1 -- 172.21.15.2:0/3825366463 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568003c90 con 0x7f6574100410 2024-03-15T13:05:17.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.683+0000 7f657add4640 1 -- 172.21.15.2:0/3825366463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6574198610 con 0x7f6574100410 2024-03-15T13:05:17.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.683+0000 7f657add4640 1 -- 172.21.15.2:0/3825366463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65741a4900 con 0x7f6574100410 2024-03-15T13:05:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.685+0000 7f6571ffb640 1 -- 172.21.15.2:0/3825366463 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f6568005000 con 0x7f6574100410 2024-03-15T13:05:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f657d9a8740 1 Processor -- start 2024-03-15T13:05:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f657d9a8740 1 -- start start 2024-03-15T13:05:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f6571ffb640 1 --2- 172.21.15.2:0/3825366463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f65640a1ba0 0x7f65640a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f657d9a8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e31e0 0x555f1c5e35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f657d9a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5e3b90 0x555f1c5ec6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f657d9a8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5eccf0 0x555f1c5ed170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f657d9a8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1c551f90 con 0x555f1c5e31e0 2024-03-15T13:05:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f657d9a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1c54f4c0 con 0x555f1c5e3b90 2024-03-15T13:05:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f657d9a8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1c54aca0 con 0x555f1c5eccf0 2024-03-15T13:05:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f6579371640 1 --2- 172.21.15.2:0/3825366463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f65640a1ba0 0x7f65640a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f6571ffb640 1 -- 172.21.15.2:0/3825366463 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f65680efba0 con 0x7f6574100410 2024-03-15T13:05:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f6578b70640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e31e0 0x555f1c5e35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f6573fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5e3b90 0x555f1c5ec6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f6578b70640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e31e0 0x555f1c5e35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58156/0 (socket says 172.21.15.2:58156) 2024-03-15T13:05:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f6578b70640 1 -- 172.21.15.2:0/2898920064 learned_addr learned my addr 172.21.15.2:0/2898920064 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f6579371640 1 --2- 172.21.15.2:0/2898920064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5eccf0 0x555f1c5ed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f6578b70640 1 -- 172.21.15.2:0/2898920064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5eccf0 msgr2=0x555f1c5ed170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f6578b70640 1 --2- 172.21.15.2:0/2898920064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5eccf0 0x555f1c5ed170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f6578b70640 1 -- 172.21.15.2:0/2898920064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5e3b90 msgr2=0x555f1c5ec6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f6578b70640 1 --2- 172.21.15.2:0/2898920064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5e3b90 0x555f1c5ec6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f6578b70640 1 -- 172.21.15.2:0/2898920064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f1c50d0b0 con 0x555f1c5e31e0 2024-03-15T13:05:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f6573fff640 1 --2- 172.21.15.2:0/2898920064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5e3b90 0x555f1c5ec6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.686+0000 7f6579371640 1 --2- 172.21.15.2:0/3825366463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f65640a1ba0 0x7f65640a4060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f656c009fd0 tx=0x7f656c013b60 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:17.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.687+0000 7f6578b70640 1 --2- 172.21.15.2:0/2898920064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e31e0 0x555f1c5e35c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f65680ed0b0 tx=0x7f65680944d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7903af065da2afc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:17.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.687+0000 7f655effd640 1 -- 172.21.15.2:0/2898920064 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568099bb0 con 0x555f1c5e31e0 2024-03-15T13:05:17.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.687+0000 7f655effd640 1 -- 172.21.15.2:0/2898920064 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f65680035f0 con 0x555f1c5e31e0 2024-03-15T13:05:17.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.687+0000 7f655effd640 1 -- 172.21.15.2:0/2898920064 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65680c4cd0 con 0x555f1c5e31e0 2024-03-15T13:05:17.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.687+0000 7f657d9a8740 1 -- 172.21.15.2:0/2898920064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e31e0 msgr2=0x555f1c5e35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.687+0000 7f657d9a8740 1 --2- 172.21.15.2:0/2898920064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e31e0 0x555f1c5e35c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f65680ed0b0 tx=0x7f65680944d0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.687+0000 7f657d9a8740 1 -- 172.21.15.2:0/2898920064 shutdown_connections 2024-03-15T13:05:17.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.687+0000 7f657d9a8740 1 --2- 172.21.15.2:0/2898920064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5eccf0 0x555f1c5ed170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.687+0000 7f657d9a8740 1 --2- 172.21.15.2:0/2898920064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5e3b90 0x555f1c5ec6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.687+0000 7f657d9a8740 1 --2- 172.21.15.2:0/2898920064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e31e0 0x555f1c5e35c0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.687+0000 7f657d9a8740 1 -- 172.21.15.2:0/2898920064 >> 172.21.15.2:0/2898920064 conn(0x555f1c5dd840 msgr2=0x555f1c5ddc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:17.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.687+0000 7f657d9a8740 1 -- 172.21.15.2:0/2898920064 shutdown_connections 2024-03-15T13:05:17.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.688+0000 7f657d9a8740 1 -- 172.21.15.2:0/2898920064 wait complete. 2024-03-15T13:05:17.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.688+0000 7f657d9a8740 1 -- start start 2024-03-15T13:05:17.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.688+0000 7f657d9a8740 1 Processor -- start 2024-03-15T13:05:17.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.688+0000 7f657d9a8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5f0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.688+0000 7f657d9a8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e3b90 0x555f1c5f0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.688+0000 7f657d9a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5eccf0 0x555f1c5f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.688+0000 7f657d9a8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1c4eff40 con 0x555f1c5e3b90 2024-03-15T13:05:17.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.688+0000 7f657d9a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1c21e4b0 con 0x555f1c5eccf0 2024-03-15T13:05:17.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.688+0000 7f657d9a8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1c4d0670 con 0x555f1c5e31e0 2024-03-15T13:05:17.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.688+0000 7f657d9a8740 10 client.0 _ll_register_callbacks cb 0x555f1c53df70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:17.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.688+0000 7f6573fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e3b90 0x555f1c5f0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6573fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e3b90 0x555f1c5f0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58158/0 (socket says 172.21.15.2:58158) 2024-03-15T13:05:17.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6573fff640 1 -- 172.21.15.2:0/3146397914 learned_addr learned my addr 172.21.15.2:0/3146397914 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:17.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6578b70640 1 --2- 172.21.15.2:0/3146397914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5f0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6579371640 1 --2- 172.21.15.2:0/3146397914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5eccf0 0x555f1c5f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6573fff640 1 -- 172.21.15.2:0/3146397914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 msgr2=0x555f1c5f0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6573fff640 1 --2- 172.21.15.2:0/3146397914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5f0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6573fff640 1 -- 172.21.15.2:0/3146397914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5eccf0 msgr2=0x555f1c5f11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6573fff640 1 --2- 172.21.15.2:0/3146397914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5eccf0 0x555f1c5f11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6573fff640 1 -- 172.21.15.2:0/3146397914 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65741a4900 con 0x555f1c5e3b90 2024-03-15T13:05:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6578b70640 1 --2- 172.21.15.2:0/3146397914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5f0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6579371640 1 --2- 172.21.15.2:0/3146397914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5eccf0 0x555f1c5f11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6573fff640 1 --2- 172.21.15.2:0/3146397914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e3b90 0x555f1c5f0c80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f6560012730 tx=0x7f6560012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed291efe98dc9b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6553fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6553fff640 1 -- 172.21.15.2:0/3146397914 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560017070 con 0x555f1c5e3b90 2024-03-15T13:05:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6553fff640 1 -- 172.21.15.2:0/3146397914 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f6560004030 con 0x555f1c5e3b90 2024-03-15T13:05:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f657d9a8740 1 -- 172.21.15.2:0/3146397914 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f1c524f40 con 0x555f1c5e3b90 2024-03-15T13:05:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f6553fff640 1 -- 172.21.15.2:0/3146397914 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65600041f0 con 0x555f1c5e3b90 2024-03-15T13:05:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f657d9a8740 10 client.17601 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.689+0000 7f657d9a8740 1 -- 172.21.15.2:0/3146397914 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x555f1c549320 con 0x555f1c5e3b90 2024-03-15T13:05:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.690+0000 7f6553fff640 1 -- 172.21.15.2:0/3146397914 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x555f1c524f40 con 0x555f1c5e3b90 2024-03-15T13:05:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.690+0000 7f6553fff640 1 -- 172.21.15.2:0/3146397914 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f65600039a0 con 0x555f1c5e3b90 2024-03-15T13:05:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.690+0000 7f657d9a8740 10 client.17601 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.690+0000 7f657d9a8740 10 client.17601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.690+0000 7f657d9a8740 20 client.17601 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.690+0000 7f657d9a8740 1 --2- 172.21.15.2:0/3146397914 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x555f1c601220 0x555f1c603640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.690+0000 7f657d9a8740 4 client.17601 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.690+0000 7f657d9a8740 1 -- 172.21.15.2:0/3146397914 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f1c541310 con 0x555f1c601220 2024-03-15T13:05:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.690+0000 7f6578b70640 1 --2- 172.21.15.2:0/3146397914 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x555f1c601220 0x555f1c603640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.690+0000 7f6553fff640 10 client.17601 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.690+0000 7f6578b70640 1 --2- 172.21.15.2:0/3146397914 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x555f1c601220 0x555f1c603640 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e02ff8f35a5b503 server_cookie=a61ce92fcfb16dcf in_seq=0 out_seq=0 2024-03-15T13:05:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.691+0000 7f6553fff640 1 -- 172.21.15.2:0/3146397914 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555f1c541310 con 0x555f1c601220 2024-03-15T13:05:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.691+0000 7f6553fff640 10 client.17601 handle_command_reply: tid=0 2024-03-15T13:05:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.691+0000 7f657d9a8740 1 client.17601 shutdown 2024-03-15T13:05:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.692+0000 7f657d9a8740 1 -- 172.21.15.2:0/3146397914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e3b90 msgr2=0x555f1c5f0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.692+0000 7f657d9a8740 1 --2- 172.21.15.2:0/3146397914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e3b90 0x555f1c5f0c80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f6560012730 tx=0x7f6560012bf0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.692+0000 7f657d9a8740 1 -- 172.21.15.2:0/3146397914 shutdown_connections 2024-03-15T13:05:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.692+0000 7f657d9a8740 1 --2- 172.21.15.2:0/3146397914 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x555f1c601220 0x555f1c603640 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.692+0000 7f657d9a8740 1 --2- 172.21.15.2:0/3146397914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5eccf0 0x555f1c5f11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.692+0000 7f6553fff640 0 client.17601 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.692+0000 7f657d9a8740 1 --2- 172.21.15.2:0/3146397914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e3b90 0x555f1c5f0c80 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.692+0000 7f657d9a8740 1 --2- 172.21.15.2:0/3146397914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5f0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.692+0000 7f657d9a8740 1 -- 172.21.15.2:0/3146397914 >> 172.21.15.2:0/3146397914 conn(0x555f1c5dd840 msgr2=0x555f1c5df7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.692+0000 7f657d9a8740 1 -- 172.21.15.2:0/3146397914 shutdown_connections 2024-03-15T13:05:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.692+0000 7f657d9a8740 1 -- 172.21.15.2:0/3146397914 wait complete. 2024-03-15T13:05:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.692+0000 7f657d9a8740 20 client.17601 trim_cache size 0 max 16384 2024-03-15T13:05:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.708+0000 7f657d9a8740 1 Processor -- start 2024-03-15T13:05:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.708+0000 7f657d9a8740 1 -- start start 2024-03-15T13:05:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.708+0000 7f657d9a8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.708+0000 7f657d9a8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e3b90 0x555f1c5ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.708+0000 7f657d9a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5eccf0 0x555f1c5f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.708+0000 7f657d9a8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1c5f1a90 con 0x555f1c5e3b90 2024-03-15T13:05:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.708+0000 7f657d9a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1c5d7cf0 con 0x555f1c5eccf0 2024-03-15T13:05:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.708+0000 7f657d9a8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1c5e2670 con 0x555f1c5e31e0 2024-03-15T13:05:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.708+0000 7f6573fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e3b90 0x555f1c5ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.708+0000 7f6578b70640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.708+0000 7f6579371640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5eccf0 0x555f1c5f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.708+0000 7f6578b70640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5f2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37526/0 (socket says 172.21.15.2:37526) 2024-03-15T13:05:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.708+0000 7f6579371640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5eccf0 0x555f1c5f0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:44224/0 (socket says 172.21.15.2:44224) 2024-03-15T13:05:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.708+0000 7f6578b70640 1 -- 172.21.15.2:0/12234699 learned_addr learned my addr 172.21.15.2:0/12234699 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.708+0000 7f6573fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e3b90 0x555f1c5ee0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58166/0 (socket says 172.21.15.2:58166) 2024-03-15T13:05:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.709+0000 7f6578b70640 1 -- 172.21.15.2:0/12234699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5eccf0 msgr2=0x555f1c5f0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.709+0000 7f6578b70640 1 --2- 172.21.15.2:0/12234699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5eccf0 0x555f1c5f0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.709+0000 7f6578b70640 1 -- 172.21.15.2:0/12234699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e3b90 msgr2=0x555f1c5ee0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.709+0000 7f6578b70640 1 --2- 172.21.15.2:0/12234699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e3b90 0x555f1c5ee0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.709+0000 7f6578b70640 1 -- 172.21.15.2:0/12234699 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f1c4e5c00 con 0x555f1c5e31e0 2024-03-15T13:05:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.709+0000 7f6578b70640 1 --2- 172.21.15.2:0/12234699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5f2e70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f65680bc230 tx=0x7f65680dc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfef6c403f9dc256 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.709+0000 7f655e7fc640 1 -- 172.21.15.2:0/12234699 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65680bc530 con 0x555f1c5e31e0 2024-03-15T13:05:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.709+0000 7f655e7fc640 1 -- 172.21.15.2:0/12234699 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f65680bcc30 con 0x555f1c5e31e0 2024-03-15T13:05:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.709+0000 7f655e7fc640 1 -- 172.21.15.2:0/12234699 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65680a7180 con 0x555f1c5e31e0 2024-03-15T13:05:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.710+0000 7f657d9a8740 1 -- 172.21.15.2:0/12234699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 msgr2=0x555f1c5f2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.710+0000 7f657d9a8740 1 --2- 172.21.15.2:0/12234699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5f2e70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f65680bc230 tx=0x7f65680dc040 comp rx=0 tx=0).stop 2024-03-15T13:05:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.710+0000 7f657d9a8740 1 -- 172.21.15.2:0/12234699 shutdown_connections 2024-03-15T13:05:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.710+0000 7f657d9a8740 1 --2- 172.21.15.2:0/12234699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5eccf0 0x555f1c5f0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.710+0000 7f657d9a8740 1 --2- 172.21.15.2:0/12234699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5e3b90 0x555f1c5ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.710+0000 7f657d9a8740 1 --2- 172.21.15.2:0/12234699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5f2e70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.710+0000 7f657d9a8740 1 -- 172.21.15.2:0/12234699 >> 172.21.15.2:0/12234699 conn(0x555f1c5dd840 msgr2=0x555f1c5dc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.710+0000 7f657d9a8740 1 -- 172.21.15.2:0/12234699 shutdown_connections 2024-03-15T13:05:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.710+0000 7f657d9a8740 1 -- 172.21.15.2:0/12234699 wait complete. 2024-03-15T13:05:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.710+0000 7f657d9a8740 1 -- start start 2024-03-15T13:05:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.710+0000 7f657d9a8740 1 Processor -- start 2024-03-15T13:05:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.710+0000 7f657d9a8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5fb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f657d9a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5e3b90 0x555f1c5fb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f657d9a8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5eccf0 0x555f1c5fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f657d9a8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1c65bc70 con 0x555f1c5eccf0 2024-03-15T13:05:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f657d9a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1c65bde0 con 0x555f1c5e3b90 2024-03-15T13:05:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f657d9a8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1c65bfa0 con 0x555f1c5e31e0 2024-03-15T13:05:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f657d9a8740 10 client.0 _ll_register_callbacks cb 0x7f6560009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6579371640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5eccf0 0x555f1c5fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6578b70640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5fb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6578b70640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5fb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37528/0 (socket says 172.21.15.2:37528) 2024-03-15T13:05:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6573fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5e3b90 0x555f1c5fb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6579371640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5eccf0 0x555f1c5fbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58170/0 (socket says 172.21.15.2:58170) 2024-03-15T13:05:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6578b70640 1 -- 172.21.15.2:0/1984809467 learned_addr learned my addr 172.21.15.2:0/1984809467 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6579371640 1 -- 172.21.15.2:0/1984809467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 msgr2=0x555f1c5fb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6579371640 1 --2- 172.21.15.2:0/1984809467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5fb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6579371640 1 -- 172.21.15.2:0/1984809467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5e3b90 msgr2=0x555f1c5fb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6579371640 1 --2- 172.21.15.2:0/1984809467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5e3b90 0x555f1c5fb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6579371640 1 -- 172.21.15.2:0/1984809467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f1c50d0b0 con 0x555f1c5eccf0 2024-03-15T13:05:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6579371640 1 --2- 172.21.15.2:0/1984809467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5eccf0 0x555f1c5fbf20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f656c009bb0 tx=0x7f656c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb1896764a37691 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6553fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6553fff640 1 -- 172.21.15.2:0/1984809467 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c018050 con 0x555f1c5eccf0 2024-03-15T13:05:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f657d9a8740 1 -- 172.21.15.2:0/1984809467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f1c570900 con 0x555f1c5eccf0 2024-03-15T13:05:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6553fff640 1 -- 172.21.15.2:0/1984809467 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f656c004b20 con 0x555f1c5eccf0 2024-03-15T13:05:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f6553fff640 1 -- 172.21.15.2:0/1984809467 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f656c026070 con 0x555f1c5eccf0 2024-03-15T13:05:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f657d9a8740 10 client.17607 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.711+0000 7f657d9a8740 1 -- 172.21.15.2:0/1984809467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x555f1c50b910 con 0x555f1c5eccf0 2024-03-15T13:05:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.712+0000 7f6553fff640 1 -- 172.21.15.2:0/1984809467 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x555f1c570900 con 0x555f1c5eccf0 2024-03-15T13:05:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.712+0000 7f6553fff640 1 -- 172.21.15.2:0/1984809467 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f656c031080 con 0x555f1c5eccf0 2024-03-15T13:05:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.712+0000 7f657d9a8740 10 client.17607 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.712+0000 7f657d9a8740 10 client.17607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.712+0000 7f657d9a8740 20 client.17607 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.712+0000 7f657d9a8740 1 --2- 172.21.15.2:0/1984809467 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x555f1c65d860 0x555f1c65fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.712+0000 7f657d9a8740 4 client.17607 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:05:17.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.712+0000 7f657d9a8740 1 -- 172.21.15.2:0/1984809467 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555f1c24a860 con 0x555f1c65d860 2024-03-15T13:05:17.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.712+0000 7f6578b70640 1 --2- 172.21.15.2:0/1984809467 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x555f1c65d860 0x555f1c65fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.712+0000 7f6578b70640 1 --2- 172.21.15.2:0/1984809467 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x555f1c65d860 0x555f1c65fc80 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29e922f4ea48281c server_cookie=fc84d32d767928cf in_seq=0 out_seq=0 2024-03-15T13:05:17.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.712+0000 7f6553fff640 10 client.17607 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.714+0000 7f6553fff640 1 -- 172.21.15.2:0/1984809467 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555f1c24a860 con 0x555f1c65d860 2024-03-15T13:05:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.714+0000 7f6553fff640 10 client.17607 handle_command_reply: tid=0 2024-03-15T13:05:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.714+0000 7f657d9a8740 1 client.17607 shutdown 2024-03-15T13:05:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.714+0000 7f657d9a8740 1 -- 172.21.15.2:0/1984809467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5eccf0 msgr2=0x555f1c5fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.714+0000 7f657d9a8740 1 --2- 172.21.15.2:0/1984809467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5eccf0 0x555f1c5fbf20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f656c009bb0 tx=0x7f656c004ab0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.714+0000 7f657d9a8740 1 -- 172.21.15.2:0/1984809467 shutdown_connections 2024-03-15T13:05:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.714+0000 7f657d9a8740 1 --2- 172.21.15.2:0/1984809467 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x555f1c65d860 0x555f1c65fc80 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.714+0000 7f6553fff640 0 client.17607 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.714+0000 7f657d9a8740 1 --2- 172.21.15.2:0/1984809467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f1c5eccf0 0x555f1c5fbf20 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.714+0000 7f657d9a8740 1 --2- 172.21.15.2:0/1984809467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f1c5e3b90 0x555f1c5fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.714+0000 7f657d9a8740 1 --2- 172.21.15.2:0/1984809467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555f1c5e31e0 0x555f1c5fb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.714+0000 7f657d9a8740 1 -- 172.21.15.2:0/1984809467 >> 172.21.15.2:0/1984809467 conn(0x555f1c5dd840 msgr2=0x555f1c5dc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:17.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.714+0000 7f657d9a8740 1 -- 172.21.15.2:0/1984809467 shutdown_connections 2024-03-15T13:05:17.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.714+0000 7f657d9a8740 1 -- 172.21.15.2:0/1984809467 wait complete. 2024-03-15T13:05:17.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.714+0000 7f657d9a8740 20 client.17607 trim_cache size 0 max 16384 2024-03-15T13:05:17.705 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:05:17.705 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:05:17.705 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "f07a78fe-2ff4-449c-aceb-07e2ffc888b5", 2024-03-15T13:05:17.705 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:05:17.706 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:05:17.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.715+0000 7f657add4640 1 -- 172.21.15.2:0/3825366463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f65640a1ba0 msgr2=0x7f65640a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.715+0000 7f657add4640 1 --2- 172.21.15.2:0/3825366463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f65640a1ba0 0x7f65640a4060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f656c009fd0 tx=0x7f656c013b60 comp rx=0 tx=0).stop 2024-03-15T13:05:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.715+0000 7f657add4640 1 -- 172.21.15.2:0/3825366463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6574100410 msgr2=0x7f657419d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.715+0000 7f657add4640 1 --2- 172.21.15.2:0/3825366463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6574100410 0x7f657419d860 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6568009900 tx=0x7f656802ca20 comp rx=0 tx=0).stop 2024-03-15T13:05:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.715+0000 7f657add4640 1 -- 172.21.15.2:0/3825366463 shutdown_connections 2024-03-15T13:05:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.715+0000 7f657add4640 1 --2- 172.21.15.2:0/3825366463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65741064b0 0x7f6574197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.715+0000 7f657add4640 1 --2- 172.21.15.2:0/3825366463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6574100dd0 0x7f657419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.715+0000 7f657add4640 1 --2- 172.21.15.2:0/3825366463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f65640a1ba0 0x7f65640a4060 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.715+0000 7f657add4640 1 --2- 172.21.15.2:0/3825366463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6574100410 0x7f657419d860 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.715+0000 7f657add4640 1 -- 172.21.15.2:0/3825366463 >> 172.21.15.2:0/3825366463 conn(0x7f6574077550 msgr2=0x7f6574069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.715+0000 7f657add4640 1 -- 172.21.15.2:0/3825366463 shutdown_connections 2024-03-15T13:05:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.715+0000 7f657add4640 1 -- 172.21.15.2:0/3825366463 wait complete. 2024-03-15T13:05:17.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:05:17.711 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:05:17.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.830+0000 7f1b80bf4640 1 -- 172.21.15.2:0/340873491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b7c100f90 msgr2=0x7f1b7c104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.830+0000 7f1b80bf4640 1 --2- 172.21.15.2:0/340873491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b7c100f90 0x7f1b7c104fd0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009930 tx=0x7f1b6402bc30 comp rx=0 tx=0).stop 2024-03-15T13:05:17.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.831+0000 7f1b80bf4640 1 -- 172.21.15.2:0/340873491 shutdown_connections 2024-03-15T13:05:17.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.831+0000 7f1b80bf4640 1 --2- 172.21.15.2:0/340873491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b7c1056b0 0x7f1b7c10d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.831+0000 7f1b80bf4640 1 --2- 172.21.15.2:0/340873491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b7c100f90 0x7f1b7c104fd0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.831+0000 7f1b80bf4640 1 --2- 172.21.15.2:0/340873491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c1005e0 0x7f1b7c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.831+0000 7f1b80bf4640 1 -- 172.21.15.2:0/340873491 >> 172.21.15.2:0/340873491 conn(0x7f1b7c0775a0 msgr2=0x7f1b7c0779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:17.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.831+0000 7f1b80bf4640 1 -- 172.21.15.2:0/340873491 shutdown_connections 2024-03-15T13:05:17.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.831+0000 7f1b80bf4640 1 -- 172.21.15.2:0/340873491 wait complete. 2024-03-15T13:05:17.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.831+0000 7f1b80bf4640 1 Processor -- start 2024-03-15T13:05:17.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.831+0000 7f1b80bf4640 1 -- start start 2024-03-15T13:05:17.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b80bf4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b7c1005e0 0x7f1b7c19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b80bf4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c1056b0 0x7f1b7c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b80bf4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b7c196860 0x7f1b7c196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b80bf4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c197250 con 0x7f1b7c1005e0 2024-03-15T13:05:17.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b80bf4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c1973c0 con 0x7f1b7c196860 2024-03-15T13:05:17.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b80bf4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c197580 con 0x7f1b7c1056b0 2024-03-15T13:05:17.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b7a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b7c1005e0 0x7f1b7c19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b7a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b7c1005e0 0x7f1b7c19c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58182/0 (socket says 172.21.15.2:58182) 2024-03-15T13:05:17.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b7a59c640 1 -- 172.21.15.2:0/894053812 learned_addr learned my addr 172.21.15.2:0/894053812 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:17.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b7ad9d640 1 --2- 172.21.15.2:0/894053812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b7c196860 0x7f1b7c196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b79d9b640 1 --2- 172.21.15.2:0/894053812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c1056b0 0x7f1b7c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b7a59c640 1 -- 172.21.15.2:0/894053812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c1056b0 msgr2=0x7f1b7c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b7a59c640 1 --2- 172.21.15.2:0/894053812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c1056b0 0x7f1b7c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b7a59c640 1 -- 172.21.15.2:0/894053812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b7c196860 msgr2=0x7f1b7c196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b7a59c640 1 --2- 172.21.15.2:0/894053812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b7c196860 0x7f1b7c196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b7a59c640 1 -- 172.21.15.2:0/894053812 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b64009560 con 0x7f1b7c1005e0 2024-03-15T13:05:17.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b7a59c640 1 --2- 172.21.15.2:0/894053812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b7c1005e0 0x7f1b7c19c6b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400e9f0 tx=0x7f1b7400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:17.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b737fe640 1 -- 172.21.15.2:0/894053812 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b74009720 con 0x7f1b7c1005e0 2024-03-15T13:05:17.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b737fe640 1 -- 172.21.15.2:0/894053812 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1b7400cd80 con 0x7f1b7c1005e0 2024-03-15T13:05:17.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b80bf4640 1 -- 172.21.15.2:0/894053812 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7c1a3570 con 0x7f1b7c1005e0 2024-03-15T13:05:17.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.832+0000 7f1b737fe640 1 -- 172.21.15.2:0/894053812 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b74010690 con 0x7f1b7c1005e0 2024-03-15T13:05:17.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.833+0000 7f1b80bf4640 1 -- 172.21.15.2:0/894053812 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7c1a3a00 con 0x7f1b7c1005e0 2024-03-15T13:05:17.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.835+0000 7f1b737fe640 1 -- 172.21.15.2:0/894053812 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f1b74004e90 con 0x7f1b7c1005e0 2024-03-15T13:05:17.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.835+0000 7f1b837d3740 1 Processor -- start 2024-03-15T13:05:17.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.835+0000 7f1b837d3740 1 -- start start 2024-03-15T13:05:17.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.835+0000 7f1b837d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 0x5579380ee690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.835+0000 7f1b837d3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380f7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.835+0000 7f1b837d3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380f77b0 0x5579380f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b837d3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793805d280 con 0x5579380eebd0 2024-03-15T13:05:17.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b837d3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793805a7b0 con 0x5579380f77b0 2024-03-15T13:05:17.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b837d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557938055f90 con 0x5579380ee2b0 2024-03-15T13:05:17.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b737fe640 1 --2- 172.21.15.2:0/894053812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1b680a1ba0 0x7f1b680a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b7a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b7ad9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 0x5579380ee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b79d9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380f77b0 0x5579380f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b7ad9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 0x5579380ee690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37546/0 (socket says 172.21.15.2:37546) 2024-03-15T13:05:17.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b7a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380f7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58194/0 (socket says 172.21.15.2:58194) 2024-03-15T13:05:17.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b7ad9d640 1 -- 172.21.15.2:0/2188549526 learned_addr learned my addr 172.21.15.2:0/2188549526 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:17.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b7ad9d640 1 --2- 172.21.15.2:0/894053812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1b680a1ba0 0x7f1b680a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b737fe640 1 -- 172.21.15.2:0/894053812 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f1b74014070 con 0x7f1b7c1005e0 2024-03-15T13:05:17.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b79d9b640 1 -- 172.21.15.2:0/2188549526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 msgr2=0x5579380ee690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b79d9b640 1 --2- 172.21.15.2:0/2188549526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 0x5579380ee690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b79d9b640 1 -- 172.21.15.2:0/2188549526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 msgr2=0x5579380f7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b79d9b640 1 --2- 172.21.15.2:0/2188549526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380f7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b79d9b640 1 -- 172.21.15.2:0/2188549526 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557938014570 con 0x5579380f77b0 2024-03-15T13:05:17.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b7ad9d640 1 --2- 172.21.15.2:0/2188549526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 0x5579380ee690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:17.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b79d9b640 1 --2- 172.21.15.2:0/2188549526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380f77b0 0x5579380f9ba0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1b64036040 tx=0x7f1b6402c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c710efdb35abea9 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:17.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.836+0000 7f1b7ad9d640 1 --2- 172.21.15.2:0/894053812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1b680a1ba0 0x7f1b680a4060 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0049d0 tx=0x7f1b6c004ab0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:17.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.837+0000 7f1b70ff9640 1 -- 172.21.15.2:0/2188549526 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b64004780 con 0x5579380f77b0 2024-03-15T13:05:17.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.837+0000 7f1b70ff9640 1 -- 172.21.15.2:0/2188549526 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1b640048e0 con 0x5579380f77b0 2024-03-15T13:05:17.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.837+0000 7f1b70ff9640 1 -- 172.21.15.2:0/2188549526 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b64003850 con 0x5579380f77b0 2024-03-15T13:05:17.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.837+0000 7f1b837d3740 1 -- 172.21.15.2:0/2188549526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380f77b0 msgr2=0x5579380f9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.837+0000 7f1b837d3740 1 --2- 172.21.15.2:0/2188549526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380f77b0 0x5579380f9ba0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1b64036040 tx=0x7f1b6402c5d0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.837+0000 7f1b837d3740 1 -- 172.21.15.2:0/2188549526 shutdown_connections 2024-03-15T13:05:17.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.837+0000 7f1b837d3740 1 --2- 172.21.15.2:0/2188549526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380f77b0 0x5579380f9ba0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.837+0000 7f1b837d3740 1 --2- 172.21.15.2:0/2188549526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.837+0000 7f1b837d3740 1 --2- 172.21.15.2:0/2188549526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 0x5579380ee690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.837+0000 7f1b837d3740 1 -- 172.21.15.2:0/2188549526 >> 172.21.15.2:0/2188549526 conn(0x5579380e8990 msgr2=0x5579380e8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:17.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.837+0000 7f1b837d3740 1 -- 172.21.15.2:0/2188549526 shutdown_connections 2024-03-15T13:05:17.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.837+0000 7f1b837d3740 1 -- 172.21.15.2:0/2188549526 wait complete. 2024-03-15T13:05:17.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b837d3740 1 -- start start 2024-03-15T13:05:17.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b837d3740 1 Processor -- start 2024-03-15T13:05:17.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b837d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 0x5579380fa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b837d3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380eebd0 0x5579380fab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b837d3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380f77b0 0x5579380fb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b837d3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557937d29430 con 0x5579380f77b0 2024-03-15T13:05:17.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b837d3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557937ce5e00 con 0x5579380eebd0 2024-03-15T13:05:17.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b837d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557937fdafd0 con 0x5579380ee2b0 2024-03-15T13:05:17.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b837d3740 10 client.0 _ll_register_callbacks cb 0x557938030fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:17.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b7ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380f77b0 0x5579380fb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b79d9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380eebd0 0x5579380fab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b7ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380f77b0 0x5579380fb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58204/0 (socket says 172.21.15.2:58204) 2024-03-15T13:05:17.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b7ad9d640 1 -- 172.21.15.2:0/428176719 learned_addr learned my addr 172.21.15.2:0/428176719 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:17.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b7a59c640 1 --2- 172.21.15.2:0/428176719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 0x5579380fa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b79d9b640 1 -- 172.21.15.2:0/428176719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 msgr2=0x5579380fa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b79d9b640 1 --2- 172.21.15.2:0/428176719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 0x5579380fa650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b79d9b640 1 -- 172.21.15.2:0/428176719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380f77b0 msgr2=0x5579380fb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b79d9b640 1 --2- 172.21.15.2:0/428176719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380f77b0 0x5579380fb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b79d9b640 1 -- 172.21.15.2:0/428176719 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7c10da90 con 0x5579380eebd0 2024-03-15T13:05:17.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b7a59c640 1 --2- 172.21.15.2:0/428176719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 0x5579380fa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:17.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.838+0000 7f1b7ad9d640 1 --2- 172.21.15.2:0/428176719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380f77b0 0x5579380fb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:17.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.839+0000 7f1b79d9b640 1 --2- 172.21.15.2:0/428176719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380eebd0 0x5579380fab90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1b64036040 tx=0x7f1b6402cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78690eb190e4d790 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:17.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.839+0000 7f1b61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:05:17.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.839+0000 7f1b61ffb640 1 -- 172.21.15.2:0/428176719 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b64004780 con 0x5579380eebd0 2024-03-15T13:05:17.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.839+0000 7f1b837d3740 1 -- 172.21.15.2:0/428176719 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557937d557e0 con 0x5579380eebd0 2024-03-15T13:05:17.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.839+0000 7f1b61ffb640 1 -- 172.21.15.2:0/428176719 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f1b6404d070 con 0x5579380eebd0 2024-03-15T13:05:17.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.839+0000 7f1b61ffb640 1 -- 172.21.15.2:0/428176719 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b640051f0 con 0x5579380eebd0 2024-03-15T13:05:17.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.839+0000 7f1b837d3740 10 client.27482 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:17.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.839+0000 7f1b837d3740 1 -- 172.21.15.2:0/428176719 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x557938054610 con 0x5579380eebd0 2024-03-15T13:05:17.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.839+0000 7f1b61ffb640 1 -- 172.21.15.2:0/428176719 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1b64059080 con 0x5579380eebd0 2024-03-15T13:05:17.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.839+0000 7f1b61ffb640 1 -- 172.21.15.2:0/428176719 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1b6403c050 con 0x5579380eebd0 2024-03-15T13:05:17.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.839+0000 7f1b837d3740 10 client.27482 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:17.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.839+0000 7f1b837d3740 10 client.27482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:17.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.839+0000 7f1b837d3740 20 client.27482 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:17.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.839+0000 7f1b837d3740 1 --2- 172.21.15.2:0/428176719 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x557938109770 0x55793810bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.840+0000 7f1b837d3740 4 client.27482 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:17.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.840+0000 7f1b837d3740 1 -- 172.21.15.2:0/428176719 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557937d92720 con 0x557938109770 2024-03-15T13:05:17.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.840+0000 7f1b7a59c640 1 --2- 172.21.15.2:0/428176719 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x557938109770 0x55793810bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.840+0000 7f1b7a59c640 1 --2- 172.21.15.2:0/428176719 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x557938109770 0x55793810bb90 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ccfaa1a3814315f server_cookie=e2c043806e21bdb9 in_seq=0 out_seq=0 2024-03-15T13:05:17.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.840+0000 7f1b61ffb640 10 client.27482 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:17.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.841+0000 7f1b61ffb640 1 -- 172.21.15.2:0/428176719 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557937d92720 con 0x557938109770 2024-03-15T13:05:17.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.841+0000 7f1b61ffb640 10 client.27482 handle_command_reply: tid=0 2024-03-15T13:05:17.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.841+0000 7f1b837d3740 1 client.27482 shutdown 2024-03-15T13:05:17.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.841+0000 7f1b837d3740 1 -- 172.21.15.2:0/428176719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380eebd0 msgr2=0x5579380fab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.841+0000 7f1b837d3740 1 --2- 172.21.15.2:0/428176719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380eebd0 0x5579380fab90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1b64036040 tx=0x7f1b6402cc40 comp rx=0 tx=0).stop 2024-03-15T13:05:17.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.841+0000 7f1b837d3740 1 -- 172.21.15.2:0/428176719 shutdown_connections 2024-03-15T13:05:17.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.841+0000 7f1b837d3740 1 --2- 172.21.15.2:0/428176719 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x557938109770 0x55793810bb90 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.841+0000 7f1b837d3740 1 --2- 172.21.15.2:0/428176719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380f77b0 0x5579380fb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.841+0000 7f1b837d3740 1 --2- 172.21.15.2:0/428176719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380eebd0 0x5579380fab90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.841+0000 7f1b837d3740 1 --2- 172.21.15.2:0/428176719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 0x5579380fa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.841+0000 7f1b61ffb640 0 client.27482 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:17.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.841+0000 7f1b837d3740 1 -- 172.21.15.2:0/428176719 >> 172.21.15.2:0/428176719 conn(0x5579380e8990 msgr2=0x5579380ea730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:17.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.841+0000 7f1b837d3740 1 -- 172.21.15.2:0/428176719 shutdown_connections 2024-03-15T13:05:17.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.841+0000 7f1b837d3740 1 -- 172.21.15.2:0/428176719 wait complete. 2024-03-15T13:05:17.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.841+0000 7f1b837d3740 20 client.27482 trim_cache size 0 max 16384 2024-03-15T13:05:17.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.857+0000 7f1b837d3740 1 Processor -- start 2024-03-15T13:05:17.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.857+0000 7f1b837d3740 1 -- start start 2024-03-15T13:05:17.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.857+0000 7f1b837d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380e8990 0x557938104ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.857+0000 7f1b837d3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380ee2b0 0x557938105020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.857+0000 7f1b837d3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380f1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.857+0000 7f1b837d3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579380edf10 con 0x5579380eebd0 2024-03-15T13:05:17.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.857+0000 7f1b837d3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579380e81d0 con 0x5579380ee2b0 2024-03-15T13:05:17.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.857+0000 7f1b837d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579380e99e0 con 0x5579380e8990 2024-03-15T13:05:17.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.858+0000 7f1b7ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.858+0000 7f1b7ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380f1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58218/0 (socket says 172.21.15.2:58218) 2024-03-15T13:05:17.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.858+0000 7f1b7ad9d640 1 -- 172.21.15.2:0/2443691856 learned_addr learned my addr 172.21.15.2:0/2443691856 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:17.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.858+0000 7f1b79d9b640 1 --2- 172.21.15.2:0/2443691856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380ee2b0 0x557938105020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.858+0000 7f1b7a59c640 1 --2- 172.21.15.2:0/2443691856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380e8990 0x557938104ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.858+0000 7f1b7ad9d640 1 -- 172.21.15.2:0/2443691856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380e8990 msgr2=0x557938104ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.858+0000 7f1b7ad9d640 1 --2- 172.21.15.2:0/2443691856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380e8990 0x557938104ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.858+0000 7f1b7ad9d640 1 -- 172.21.15.2:0/2443691856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380ee2b0 msgr2=0x557938105020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.858+0000 7f1b7ad9d640 1 --2- 172.21.15.2:0/2443691856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380ee2b0 0x557938105020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.858+0000 7f1b7ad9d640 1 -- 172.21.15.2:0/2443691856 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557937ff0ac0 con 0x5579380eebd0 2024-03-15T13:05:17.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.858+0000 7f1b7ad9d640 1 --2- 172.21.15.2:0/2443691856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380f1800 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00fad0 tx=0x7f1b6c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cb5b80e69e3efb server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:17.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.858+0000 7f1b63fff640 1 -- 172.21.15.2:0/2443691856 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c023ce0 con 0x5579380eebd0 2024-03-15T13:05:17.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.858+0000 7f1b63fff640 1 -- 172.21.15.2:0/2443691856 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1b6c023e40 con 0x5579380eebd0 2024-03-15T13:05:17.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.858+0000 7f1b63fff640 1 -- 172.21.15.2:0/2443691856 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c02b720 con 0x5579380eebd0 2024-03-15T13:05:17.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 -- 172.21.15.2:0/2443691856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 msgr2=0x5579380f1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 --2- 172.21.15.2:0/2443691856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380f1800 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00fad0 tx=0x7f1b6c005bd0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 -- 172.21.15.2:0/2443691856 shutdown_connections 2024-03-15T13:05:17.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 --2- 172.21.15.2:0/2443691856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380f1800 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 --2- 172.21.15.2:0/2443691856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380ee2b0 0x557938105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 --2- 172.21.15.2:0/2443691856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380e8990 0x557938104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 -- 172.21.15.2:0/2443691856 >> 172.21.15.2:0/2443691856 conn(0x5579380f9ec0 msgr2=0x5579380e8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:17.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 -- 172.21.15.2:0/2443691856 shutdown_connections 2024-03-15T13:05:17.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 -- 172.21.15.2:0/2443691856 wait complete. 2024-03-15T13:05:17.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 -- start start 2024-03-15T13:05:17.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 Processor -- start 2024-03-15T13:05:17.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380e8990 0x5579380fba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 0x5579380fbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557938165050 con 0x5579380eebd0 2024-03-15T13:05:17.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557938165190 con 0x5579380e8990 2024-03-15T13:05:17.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579381652d0 con 0x5579380ee2b0 2024-03-15T13:05:17.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b837d3740 10 client.0 _ll_register_callbacks cb 0x7f1b74005780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:17.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b7ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.859+0000 7f1b7ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380fc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58220/0 (socket says 172.21.15.2:58220) 2024-03-15T13:05:17.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b7a59c640 1 --2- 172.21.15.2:0/2000258131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380e8990 0x5579380fba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b7ad9d640 1 -- 172.21.15.2:0/2000258131 learned_addr learned my addr 172.21.15.2:0/2000258131 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:17.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b79d9b640 1 --2- 172.21.15.2:0/2000258131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 0x5579380fbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b7ad9d640 1 -- 172.21.15.2:0/2000258131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 msgr2=0x5579380fbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b7ad9d640 1 --2- 172.21.15.2:0/2000258131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 0x5579380fbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b7ad9d640 1 -- 172.21.15.2:0/2000258131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380e8990 msgr2=0x5579380fba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:17.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b7ad9d640 1 --2- 172.21.15.2:0/2000258131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380e8990 0x5579380fba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b7ad9d640 1 -- 172.21.15.2:0/2000258131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6c0136d0 con 0x5579380eebd0 2024-03-15T13:05:17.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b7a59c640 1 --2- 172.21.15.2:0/2000258131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380e8990 0x5579380fba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:17.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b7ad9d640 1 --2- 172.21.15.2:0/2000258131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380fc4b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c005c40 tx=0x7f1b6c013860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f307df214acd5b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:17.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:17.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b61ffb640 1 -- 172.21.15.2:0/2000258131 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c010dd0 con 0x5579380eebd0 2024-03-15T13:05:17.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b61ffb640 1 -- 172.21.15.2:0/2000258131 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f1b6c010970 con 0x5579380eebd0 2024-03-15T13:05:17.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b837d3740 1 -- 172.21.15.2:0/2000258131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b640048e0 con 0x5579380eebd0 2024-03-15T13:05:17.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b61ffb640 1 -- 172.21.15.2:0/2000258131 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b6c010b30 con 0x5579380eebd0 2024-03-15T13:05:17.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b837d3740 10 client.17625 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:17.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b837d3740 1 -- 172.21.15.2:0/2000258131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5579380166a0 con 0x5579380eebd0 2024-03-15T13:05:17.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.860+0000 7f1b61ffb640 1 -- 172.21.15.2:0/2000258131 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1b640048e0 con 0x5579380eebd0 2024-03-15T13:05:17.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.861+0000 7f1b61ffb640 1 -- 172.21.15.2:0/2000258131 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1b6c034590 con 0x5579380eebd0 2024-03-15T13:05:17.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.861+0000 7f1b837d3740 10 client.17625 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:17.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.861+0000 7f1b837d3740 10 client.17625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:17.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.861+0000 7f1b837d3740 20 client.17625 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:17.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.861+0000 7f1b837d3740 1 --2- 172.21.15.2:0/2000258131 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5579380f77b0 0x557938168aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:17.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.861+0000 7f1b837d3740 4 client.17625 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:05:17.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.861+0000 7f1b837d3740 1 -- 172.21.15.2:0/2000258131 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557937de1890 con 0x5579380f77b0 2024-03-15T13:05:17.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.861+0000 7f1b7a59c640 1 --2- 172.21.15.2:0/2000258131 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5579380f77b0 0x557938168aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:17.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.861+0000 7f1b7a59c640 1 --2- 172.21.15.2:0/2000258131 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5579380f77b0 0x557938168aa0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=308c81a9eef82e3d server_cookie=e21b01db2dc5545e in_seq=0 out_seq=0 2024-03-15T13:05:17.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.861+0000 7f1b61ffb640 10 client.17625 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:17.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.861+0000 7f1b61ffb640 1 -- 172.21.15.2:0/2000258131 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557937de1890 con 0x5579380f77b0 2024-03-15T13:05:17.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.861+0000 7f1b61ffb640 10 client.17625 handle_command_reply: tid=0 2024-03-15T13:05:17.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.861+0000 7f1b837d3740 1 client.17625 shutdown 2024-03-15T13:05:17.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.861+0000 7f1b837d3740 1 -- 172.21.15.2:0/2000258131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 msgr2=0x5579380fc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.861+0000 7f1b837d3740 1 --2- 172.21.15.2:0/2000258131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380fc4b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c005c40 tx=0x7f1b6c013860 comp rx=0 tx=0).stop 2024-03-15T13:05:17.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b837d3740 1 -- 172.21.15.2:0/2000258131 shutdown_connections 2024-03-15T13:05:17.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b837d3740 1 --2- 172.21.15.2:0/2000258131 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5579380f77b0 0x557938168aa0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b837d3740 1 --2- 172.21.15.2:0/2000258131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579380eebd0 0x5579380fc4b0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b837d3740 1 --2- 172.21.15.2:0/2000258131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5579380ee2b0 0x5579380fbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b61ffb640 0 client.17625 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:17.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b837d3740 1 --2- 172.21.15.2:0/2000258131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579380e8990 0x5579380fba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b837d3740 1 -- 172.21.15.2:0/2000258131 >> 172.21.15.2:0/2000258131 conn(0x5579380f9ec0 msgr2=0x5579380e71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:17.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b837d3740 1 -- 172.21.15.2:0/2000258131 shutdown_connections 2024-03-15T13:05:17.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b837d3740 1 -- 172.21.15.2:0/2000258131 wait complete. 2024-03-15T13:05:17.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b837d3740 20 client.17625 trim_cache size 0 max 16384 2024-03-15T13:05:17.855 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:05:17.856 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T13:05:17.856 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:05:17.856 INFO:teuthology.orchestra.run.smithi002.stdout: "f07a78fe-2ff4-449c-aceb-07e2ffc888b5": { 2024-03-15T13:05:17.976 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:05:17.976 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "f07a78fe-2ff4-449c-aceb-07e2ffc888b5", 2024-03-15T13:05:17.976 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:05:17.976 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:05:17.976 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:05:17.977 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:05:17.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b80bf4640 1 -- 172.21.15.2:0/894053812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1b680a1ba0 msgr2=0x7f1b680a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b80bf4640 1 --2- 172.21.15.2:0/894053812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1b680a1ba0 0x7f1b680a4060 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0049d0 tx=0x7f1b6c004ab0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b80bf4640 1 -- 172.21.15.2:0/894053812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b7c1005e0 msgr2=0x7f1b7c19c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:17.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b80bf4640 1 --2- 172.21.15.2:0/894053812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b7c1005e0 0x7f1b7c19c6b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400e9f0 tx=0x7f1b7400edf0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b80bf4640 1 -- 172.21.15.2:0/894053812 shutdown_connections 2024-03-15T13:05:17.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b80bf4640 1 --2- 172.21.15.2:0/894053812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b7c196860 0x7f1b7c196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b80bf4640 1 --2- 172.21.15.2:0/894053812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b7c1056b0 0x7f1b7c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b80bf4640 1 --2- 172.21.15.2:0/894053812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1b680a1ba0 0x7f1b680a4060 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b80bf4640 1 --2- 172.21.15.2:0/894053812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b7c1005e0 0x7f1b7c19c6b0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:17.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b80bf4640 1 -- 172.21.15.2:0/894053812 >> 172.21.15.2:0/894053812 conn(0x7f1b7c0775a0 msgr2=0x7f1b7c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:17.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b80bf4640 1 -- 172.21.15.2:0/894053812 shutdown_connections 2024-03-15T13:05:17.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:17.862+0000 7f1b80bf4640 1 -- 172.21.15.2:0/894053812 wait complete. 2024-03-15T13:05:17.979 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:17 smithi002 ceph-mon[32060]: pgmap v194: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:05:17.980 INFO:tasks.cephfs.filesystem:scrub status for tag:f07a78fe-2ff4-449c-aceb-07e2ffc888b5 - {'path': '/', 'tag': 'f07a78fe-2ff4-449c-aceb-07e2ffc888b5', 'options': 'recursive,force'} 2024-03-15T13:05:18.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:17 smithi184 ceph-mon[35993]: pgmap v194: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:05:18.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:17 smithi149 ceph-mon[35927]: pgmap v194: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-15T13:05:19.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:18 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:05:19.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:18 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:05:19.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:18 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:05:19.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:18 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:05:19.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:18 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:05:19.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:18 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:05:19.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:18 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:05:19.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:18 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:05:19.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:18 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:05:19.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:18 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:05:19.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:18 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:05:19.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:18 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:05:20.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:19 smithi002 ceph-mon[32060]: pgmap v195: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-15T13:05:20.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:19 smithi184 ceph-mon[35993]: pgmap v195: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-15T13:05:20.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:19 smithi149 ceph-mon[35927]: pgmap v195: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-15T13:05:21.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:20 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:05:21.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:20 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:05:21.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:20 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:05:22.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:21 smithi002 ceph-mon[32060]: pgmap v196: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-15T13:05:22.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:21 smithi184 ceph-mon[35993]: pgmap v196: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-15T13:05:22.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:21 smithi149 ceph-mon[35927]: pgmap v196: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-15T13:05:24.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:23 smithi002 ceph-mon[32060]: pgmap v197: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:05:24.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:23 smithi184 ceph-mon[35993]: pgmap v197: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:05:24.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:23 smithi149 ceph-mon[35927]: pgmap v197: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:05:26.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:25 smithi002 ceph-mon[32060]: pgmap v198: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:05:26.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:25 smithi184 ceph-mon[35993]: pgmap v198: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:05:26.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:25 smithi149 ceph-mon[35927]: pgmap v198: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:05:28.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:27 smithi002 ceph-mon[32060]: pgmap v199: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:05:28.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:27 smithi184 ceph-mon[35993]: pgmap v199: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:05:28.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:27 smithi149 ceph-mon[35927]: pgmap v199: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:05:30.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:29 smithi002 ceph-mon[32060]: pgmap v200: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:05:30.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:29 smithi184 ceph-mon[35993]: pgmap v200: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:05:30.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:29 smithi149 ceph-mon[35927]: pgmap v200: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:05:32.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:31 smithi002 ceph-mon[32060]: pgmap v201: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:05:32.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:31 smithi184 ceph-mon[35993]: pgmap v201: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:05:32.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:31 smithi149 ceph-mon[35927]: pgmap v201: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:05:34.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:33 smithi002 ceph-mon[32060]: pgmap v202: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:34.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:33 smithi184 ceph-mon[35993]: pgmap v202: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:34.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:33 smithi149 ceph-mon[35927]: pgmap v202: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:36.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:35 smithi002 ceph-mon[32060]: pgmap v203: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:36.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:35 smithi184 ceph-mon[35993]: pgmap v203: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:36.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:35 smithi149 ceph-mon[35927]: pgmap v203: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:38.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:37 smithi002 ceph-mon[32060]: pgmap v204: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:05:38.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:37 smithi184 ceph-mon[35993]: pgmap v204: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:05:38.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:37 smithi149 ceph-mon[35927]: pgmap v204: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:05:40.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:39 smithi002 ceph-mon[32060]: pgmap v205: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:40.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:39 smithi184 ceph-mon[35993]: pgmap v205: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:40.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:39 smithi149 ceph-mon[35927]: pgmap v205: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:42.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:41 smithi002 ceph-mon[32060]: pgmap v206: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:05:42.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:41 smithi184 ceph-mon[35993]: pgmap v206: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:05:42.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:41 smithi149 ceph-mon[35927]: pgmap v206: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:05:44.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:43 smithi002 ceph-mon[32060]: pgmap v207: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:44.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:43 smithi184 ceph-mon[35993]: pgmap v207: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:44.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:43 smithi149 ceph-mon[35927]: pgmap v207: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:46.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:45 smithi002 ceph-mon[32060]: pgmap v208: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:46.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:45 smithi184 ceph-mon[35993]: pgmap v208: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:46.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:45 smithi149 ceph-mon[35927]: pgmap v208: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:47.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:05:47.981 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:05:48.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:47 smithi002 ceph-mon[32060]: pgmap v209: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:05:48.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:47 smithi184 ceph-mon[35993]: pgmap v209: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:05:48.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.113+0000 7f2137a9a640 1 -- 172.21.15.2:0/1846963061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2130069b60 msgr2=0x7f2130069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.113+0000 7f2137a9a640 1 --2- 172.21.15.2:0/1846963061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2130069b60 0x7f2130069fc0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f2120009900 tx=0x7f212002bc30 comp rx=0 tx=0).stop 2024-03-15T13:05:48.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.113+0000 7f2137a9a640 1 -- 172.21.15.2:0/1846963061 shutdown_connections 2024-03-15T13:05:48.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.113+0000 7f2137a9a640 1 --2- 172.21.15.2:0/1846963061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f213006a500 0x7f213010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.113+0000 7f2137a9a640 1 --2- 172.21.15.2:0/1846963061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2130069b60 0x7f2130069fc0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.113+0000 7f2137a9a640 1 --2- 172.21.15.2:0/1846963061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2130107570 0x7f2130107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.113+0000 7f2137a9a640 1 -- 172.21.15.2:0/1846963061 >> 172.21.15.2:0/1846963061 conn(0x7f21300771d0 msgr2=0x7f21300775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.113+0000 7f2137a9a640 1 -- 172.21.15.2:0/1846963061 shutdown_connections 2024-03-15T13:05:48.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.113+0000 7f2137a9a640 1 -- 172.21.15.2:0/1846963061 wait complete. 2024-03-15T13:05:48.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.114+0000 7f2137a9a640 1 Processor -- start 2024-03-15T13:05:48.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.114+0000 7f2137a9a640 1 -- start start 2024-03-15T13:05:48.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.114+0000 7f2137a9a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2130069b60 0x7f213019d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.114+0000 7f2137a9a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f213006a500 0x7f213019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.114+0000 7f2137a9a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2130107570 0x7f2130197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.114+0000 7f2137a9a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2130197f90 con 0x7f2130107570 2024-03-15T13:05:48.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.114+0000 7f2137a9a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2130198100 con 0x7f2130069b60 2024-03-15T13:05:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.114+0000 7f2137a9a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21301982c0 con 0x7f213006a500 2024-03-15T13:05:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.114+0000 7f2136037640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2130107570 0x7f2130197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.114+0000 7f2135836640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2130069b60 0x7f213019d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.114+0000 7f2136037640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2130107570 0x7f2130197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53042/0 (socket says 172.21.15.2:53042) 2024-03-15T13:05:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.114+0000 7f2136037640 1 -- 172.21.15.2:0/3124910788 learned_addr learned my addr 172.21.15.2:0/3124910788 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.114+0000 7f2135035640 1 --2- 172.21.15.2:0/3124910788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f213006a500 0x7f213019dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.115+0000 7f2136037640 1 -- 172.21.15.2:0/3124910788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f213006a500 msgr2=0x7f213019dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.115+0000 7f2136037640 1 --2- 172.21.15.2:0/3124910788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f213006a500 0x7f213019dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.115+0000 7f2136037640 1 -- 172.21.15.2:0/3124910788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2130069b60 msgr2=0x7f213019d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.115+0000 7f2136037640 1 --2- 172.21.15.2:0/3124910788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2130069b60 0x7f213019d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.115+0000 7f2136037640 1 -- 172.21.15.2:0/3124910788 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2120009560 con 0x7f2130107570 2024-03-15T13:05:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.115+0000 7f2136037640 1 --2- 172.21.15.2:0/3124910788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2130107570 0x7f2130197940 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f212800eb10 tx=0x7f212800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.115+0000 7f2126ffd640 1 -- 172.21.15.2:0/3124910788 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2128004b20 con 0x7f2130107570 2024-03-15T13:05:48.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.115+0000 7f2126ffd640 1 -- 172.21.15.2:0/3124910788 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2128004c80 con 0x7f2130107570 2024-03-15T13:05:48.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.115+0000 7f2137a9a640 1 -- 172.21.15.2:0/3124910788 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21301985a0 con 0x7f2130107570 2024-03-15T13:05:48.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.115+0000 7f2126ffd640 1 -- 172.21.15.2:0/3124910788 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2128010690 con 0x7f2130107570 2024-03-15T13:05:48.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.115+0000 7f2137a9a640 1 -- 172.21.15.2:0/3124910788 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21301a4830 con 0x7f2130107570 2024-03-15T13:05:48.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.117+0000 7f2126ffd640 1 -- 172.21.15.2:0/3124910788 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f2128004df0 con 0x7f2130107570 2024-03-15T13:05:48.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f213a67c740 1 Processor -- start 2024-03-15T13:05:48.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f213a67c740 1 -- start start 2024-03-15T13:05:48.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f213a67c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b66f60 0x55e617b67340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f213a67c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b67880 0x55e617b74680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f2126ffd640 1 --2- 172.21.15.2:0/3124910788 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f211c0a1ba0 0x7f211c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f213a67c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b74bc0 0x55e617b76fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f213a67c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e617ad6280 con 0x55e617b67880 2024-03-15T13:05:48.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f213a67c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e617ad37b0 con 0x55e617b74bc0 2024-03-15T13:05:48.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f213a67c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e617acef90 con 0x55e617b66f60 2024-03-15T13:05:48.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f2135836640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b67880 0x55e617b74680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f2136037640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b74bc0 0x55e617b76fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f2135035640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b66f60 0x55e617b67340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f2135836640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b67880 0x55e617b74680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53048/0 (socket says 172.21.15.2:53048) 2024-03-15T13:05:48.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f2135836640 1 -- 172.21.15.2:0/77395434 learned_addr learned my addr 172.21.15.2:0/77395434 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f2126ffd640 1 -- 172.21.15.2:0/3124910788 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f2128014070 con 0x7f2130107570 2024-03-15T13:05:48.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.118+0000 7f2135035640 1 --2- 172.21.15.2:0/3124910788 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f211c0a1ba0 0x7f211c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.119+0000 7f2135836640 1 -- 172.21.15.2:0/77395434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b66f60 msgr2=0x55e617b67340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.119+0000 7f2135836640 1 --2- 172.21.15.2:0/77395434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b66f60 0x55e617b67340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.119+0000 7f2135836640 1 -- 172.21.15.2:0/77395434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b74bc0 msgr2=0x55e617b76fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.119+0000 7f2135836640 1 --2- 172.21.15.2:0/77395434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b74bc0 0x55e617b76fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.119+0000 7f2135836640 1 -- 172.21.15.2:0/77395434 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e617a8d570 con 0x55e617b67880 2024-03-15T13:05:48.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.119+0000 7f2136037640 1 --2- 172.21.15.2:0/77395434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b74bc0 0x55e617b76fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:48.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.119+0000 7f2135035640 1 --2- 172.21.15.2:0/77395434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b66f60 0x55e617b67340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:48.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.119+0000 7f2135836640 1 --2- 172.21.15.2:0/77395434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b67880 0x55e617b74680 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f212c009c10 tx=0x7f212c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50f47f5a8413af8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.119+0000 7f2135035640 1 --2- 172.21.15.2:0/3124910788 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f211c0a1ba0 0x7f211c0a4060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2120005ab0 tx=0x7f2120005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.119+0000 7f211bfff640 1 -- 172.21.15.2:0/77395434 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c004230 con 0x55e617b67880 2024-03-15T13:05:48.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.119+0000 7f211bfff640 1 -- 172.21.15.2:0/77395434 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f212c009e80 con 0x55e617b67880 2024-03-15T13:05:48.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.119+0000 7f211bfff640 1 -- 172.21.15.2:0/77395434 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c004ca0 con 0x55e617b67880 2024-03-15T13:05:48.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.120+0000 7f213a67c740 1 -- 172.21.15.2:0/77395434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b67880 msgr2=0x55e617b74680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.120+0000 7f213a67c740 1 --2- 172.21.15.2:0/77395434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b67880 0x55e617b74680 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f212c009c10 tx=0x7f212c011810 comp rx=0 tx=0).stop 2024-03-15T13:05:48.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.120+0000 7f213a67c740 1 -- 172.21.15.2:0/77395434 shutdown_connections 2024-03-15T13:05:48.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.120+0000 7f213a67c740 1 --2- 172.21.15.2:0/77395434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b74bc0 0x55e617b76fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.120+0000 7f213a67c740 1 --2- 172.21.15.2:0/77395434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b67880 0x55e617b74680 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.120+0000 7f213a67c740 1 --2- 172.21.15.2:0/77395434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b66f60 0x55e617b67340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.120+0000 7f213a67c740 1 -- 172.21.15.2:0/77395434 >> 172.21.15.2:0/77395434 conn(0x55e617b61990 msgr2=0x55e617b61d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.120+0000 7f213a67c740 1 -- 172.21.15.2:0/77395434 shutdown_connections 2024-03-15T13:05:48.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.120+0000 7f213a67c740 1 -- 172.21.15.2:0/77395434 wait complete. 2024-03-15T13:05:48.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.120+0000 7f213a67c740 1 -- start start 2024-03-15T13:05:48.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.120+0000 7f213a67c740 1 Processor -- start 2024-03-15T13:05:48.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f213a67c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b74560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f213a67c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b67880 0x55e617b718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f213a67c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b74bc0 0x55e617b71e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f213a67c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6177a2430 con 0x55e617b66f60 2024-03-15T13:05:48.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f213a67c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61775ee00 con 0x55e617b67880 2024-03-15T13:05:48.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f213a67c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e617a53fd0 con 0x55e617b74bc0 2024-03-15T13:05:48.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f213a67c740 10 client.0 _ll_register_callbacks cb 0x55e617aa9fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:48.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f2135836640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b74560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f2135836640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b74560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53050/0 (socket says 172.21.15.2:53050) 2024-03-15T13:05:48.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f2135836640 1 -- 172.21.15.2:0/2592046895 learned_addr learned my addr 172.21.15.2:0/2592046895 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f2136037640 1 --2- 172.21.15.2:0/2592046895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b74bc0 0x55e617b71e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f2135035640 1 --2- 172.21.15.2:0/2592046895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b67880 0x55e617b718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f2135836640 1 -- 172.21.15.2:0/2592046895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b74bc0 msgr2=0x55e617b71e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f2135836640 1 --2- 172.21.15.2:0/2592046895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b74bc0 0x55e617b71e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f2135836640 1 -- 172.21.15.2:0/2592046895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b67880 msgr2=0x55e617b718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f2135836640 1 --2- 172.21.15.2:0/2592046895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b67880 0x55e617b718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.121+0000 7f2135836640 1 -- 172.21.15.2:0/2592046895 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212c009870 con 0x55e617b66f60 2024-03-15T13:05:48.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f2135836640 1 --2- 172.21.15.2:0/2592046895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b74560 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f212c0049a0 tx=0x7f212c004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c67168e566dcce server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f21197fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:48.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f21197fa640 1 -- 172.21.15.2:0/2592046895 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c004340 con 0x55e617b66f60 2024-03-15T13:05:48.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f21197fa640 1 -- 172.21.15.2:0/2592046895 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f212c0044a0 con 0x55e617b66f60 2024-03-15T13:05:48.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f213a67c740 1 -- 172.21.15.2:0/2592046895 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e61780b720 con 0x55e617b66f60 2024-03-15T13:05:48.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f21197fa640 1 -- 172.21.15.2:0/2592046895 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f212c005020 con 0x55e617b66f60 2024-03-15T13:05:48.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f213a67c740 10 client.17637 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:48.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f213a67c740 1 -- 172.21.15.2:0/2592046895 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55e617acd610 con 0x55e617b66f60 2024-03-15T13:05:48.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f21197fa640 1 -- 172.21.15.2:0/2592046895 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e61780b720 con 0x55e617b66f60 2024-03-15T13:05:48.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f21197fa640 1 -- 172.21.15.2:0/2592046895 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f212c023410 con 0x55e617b66f60 2024-03-15T13:05:48.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f213a67c740 10 client.17637 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:48.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f213a67c740 10 client.17637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:48.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f213a67c740 20 client.17637 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:48.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f213a67c740 1 --2- 172.21.15.2:0/2592046895 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e617b84d90 0x55e617b87180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f213a67c740 4 client.17637 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:48.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.122+0000 7f213a67c740 1 -- 172.21.15.2:0/2592046895 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e61785a890 con 0x55e617b84d90 2024-03-15T13:05:48.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.123+0000 7f2135035640 1 --2- 172.21.15.2:0/2592046895 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e617b84d90 0x55e617b87180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.123+0000 7f21197fa640 10 client.17637 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:48.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.123+0000 7f2135035640 1 --2- 172.21.15.2:0/2592046895 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e617b84d90 0x55e617b87180 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ac2cee0cc2e01ad server_cookie=c07ba2a067f8382d in_seq=0 out_seq=0 2024-03-15T13:05:48.114 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:47 smithi149 ceph-mon[35927]: pgmap v209: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:05:48.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.124+0000 7f21197fa640 1 -- 172.21.15.2:0/2592046895 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e61785a890 con 0x55e617b84d90 2024-03-15T13:05:48.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.124+0000 7f21197fa640 10 client.17637 handle_command_reply: tid=0 2024-03-15T13:05:48.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.124+0000 7f213a67c740 1 client.17637 shutdown 2024-03-15T13:05:48.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.124+0000 7f213a67c740 1 -- 172.21.15.2:0/2592046895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 msgr2=0x55e617b74560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.124+0000 7f213a67c740 1 --2- 172.21.15.2:0/2592046895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b74560 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f212c0049a0 tx=0x7f212c004f10 comp rx=0 tx=0).stop 2024-03-15T13:05:48.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.124+0000 7f213a67c740 1 -- 172.21.15.2:0/2592046895 shutdown_connections 2024-03-15T13:05:48.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.124+0000 7f213a67c740 1 --2- 172.21.15.2:0/2592046895 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e617b84d90 0x55e617b87180 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.124+0000 7f213a67c740 1 --2- 172.21.15.2:0/2592046895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b74bc0 0x55e617b71e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.124+0000 7f213a67c740 1 --2- 172.21.15.2:0/2592046895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b67880 0x55e617b718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.124+0000 7f21197fa640 0 client.17637 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:48.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.124+0000 7f213a67c740 1 --2- 172.21.15.2:0/2592046895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b74560 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.124+0000 7f213a67c740 1 -- 172.21.15.2:0/2592046895 >> 172.21.15.2:0/2592046895 conn(0x55e617b61990 msgr2=0x55e617b63730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.124+0000 7f213a67c740 1 -- 172.21.15.2:0/2592046895 shutdown_connections 2024-03-15T13:05:48.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.124+0000 7f213a67c740 1 -- 172.21.15.2:0/2592046895 wait complete. 2024-03-15T13:05:48.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.124+0000 7f213a67c740 20 client.17637 trim_cache size 0 max 16384 2024-03-15T13:05:48.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f213a67c740 1 Processor -- start 2024-03-15T13:05:48.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f213a67c740 1 -- start start 2024-03-15T13:05:48.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f213a67c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b71890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f213a67c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b67880 0x55e617b74020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f213a67c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b74bc0 0x55e617b74560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f213a67c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e617b818d0 con 0x55e617b66f60 2024-03-15T13:05:48.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f213a67c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e617b62590 con 0x55e617b74bc0 2024-03-15T13:05:48.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f213a67c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e617b62a30 con 0x55e617b67880 2024-03-15T13:05:48.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f2135836640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b71890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f2135836640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b71890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53052/0 (socket says 172.21.15.2:53052) 2024-03-15T13:05:48.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f2135836640 1 -- 172.21.15.2:0/1567447929 learned_addr learned my addr 172.21.15.2:0/1567447929 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f2136037640 1 --2- 172.21.15.2:0/1567447929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b74bc0 0x55e617b74560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f2135035640 1 --2- 172.21.15.2:0/1567447929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b67880 0x55e617b74020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f2135836640 1 -- 172.21.15.2:0/1567447929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b67880 msgr2=0x55e617b74020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f2135836640 1 --2- 172.21.15.2:0/1567447929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b67880 0x55e617b74020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f2135836640 1 -- 172.21.15.2:0/1567447929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b74bc0 msgr2=0x55e617b74560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f2135836640 1 --2- 172.21.15.2:0/1567447929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b74bc0 0x55e617b74560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f2135836640 1 -- 172.21.15.2:0/1567447929 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e617a69ac0 con 0x55e617b66f60 2024-03-15T13:05:48.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.141+0000 7f2135836640 1 --2- 172.21.15.2:0/1567447929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b71890 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f212c001eb0 tx=0x7f212c012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cf75bed73bdff5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.142+0000 7f211b7fe640 1 -- 172.21.15.2:0/1567447929 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c011dc0 con 0x55e617b66f60 2024-03-15T13:05:48.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.142+0000 7f211b7fe640 1 -- 172.21.15.2:0/1567447929 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f212c004510 con 0x55e617b66f60 2024-03-15T13:05:48.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.142+0000 7f211b7fe640 1 -- 172.21.15.2:0/1567447929 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c005300 con 0x55e617b66f60 2024-03-15T13:05:48.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.142+0000 7f213a67c740 1 -- 172.21.15.2:0/1567447929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 msgr2=0x55e617b71890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.142+0000 7f213a67c740 1 --2- 172.21.15.2:0/1567447929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b71890 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f212c001eb0 tx=0x7f212c012f90 comp rx=0 tx=0).stop 2024-03-15T13:05:48.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.142+0000 7f213a67c740 1 -- 172.21.15.2:0/1567447929 shutdown_connections 2024-03-15T13:05:48.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.142+0000 7f213a67c740 1 --2- 172.21.15.2:0/1567447929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b74bc0 0x55e617b74560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.142+0000 7f213a67c740 1 --2- 172.21.15.2:0/1567447929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b67880 0x55e617b74020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.142+0000 7f213a67c740 1 --2- 172.21.15.2:0/1567447929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b71890 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.142+0000 7f213a67c740 1 -- 172.21.15.2:0/1567447929 >> 172.21.15.2:0/1567447929 conn(0x55e617b61990 msgr2=0x55e617b71180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.142+0000 7f213a67c740 1 -- 172.21.15.2:0/1567447929 shutdown_connections 2024-03-15T13:05:48.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.142+0000 7f213a67c740 1 -- 172.21.15.2:0/1567447929 wait complete. 2024-03-15T13:05:48.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.142+0000 7f213a67c740 1 -- start start 2024-03-15T13:05:48.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f213a67c740 1 Processor -- start 2024-03-15T13:05:48.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f213a67c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b75cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f213a67c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b67880 0x55e617b761f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f213a67c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b74bc0 0x55e617b7adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f213a67c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e617b7b300 con 0x55e617b66f60 2024-03-15T13:05:48.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f213a67c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e617b7b470 con 0x55e617b67880 2024-03-15T13:05:48.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f213a67c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e617b7b630 con 0x55e617b74bc0 2024-03-15T13:05:48.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f213a67c740 10 client.0 _ll_register_callbacks cb 0x7f2128005b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:48.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f2135836640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b75cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f2135836640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b75cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53054/0 (socket says 172.21.15.2:53054) 2024-03-15T13:05:48.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f2135836640 1 -- 172.21.15.2:0/3469334288 learned_addr learned my addr 172.21.15.2:0/3469334288 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f2136037640 1 --2- 172.21.15.2:0/3469334288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b74bc0 0x55e617b7adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f2135035640 1 --2- 172.21.15.2:0/3469334288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b67880 0x55e617b761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f2135836640 1 -- 172.21.15.2:0/3469334288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b74bc0 msgr2=0x55e617b7adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f2135836640 1 --2- 172.21.15.2:0/3469334288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b74bc0 0x55e617b7adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f2135836640 1 -- 172.21.15.2:0/3469334288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b67880 msgr2=0x55e617b761f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f2135836640 1 --2- 172.21.15.2:0/3469334288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b67880 0x55e617b761f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f2135836640 1 -- 172.21.15.2:0/3469334288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e617acd610 con 0x55e617b66f60 2024-03-15T13:05:48.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.143+0000 7f2136037640 1 --2- 172.21.15.2:0/3469334288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b74bc0 0x55e617b7adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:48.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f2135836640 1 --2- 172.21.15.2:0/3469334288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b75cb0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f212c006010 tx=0x7f212c023c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a258f2e4722989a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f21197fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:48.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f21197fa640 1 -- 172.21.15.2:0/3469334288 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c004240 con 0x55e617b66f60 2024-03-15T13:05:48.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f21197fa640 1 -- 172.21.15.2:0/3469334288 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f212c000e40 con 0x55e617b66f60 2024-03-15T13:05:48.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f213a67c740 1 -- 172.21.15.2:0/3469334288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6177ce7e0 con 0x55e617b66f60 2024-03-15T13:05:48.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f21197fa640 1 -- 172.21.15.2:0/3469334288 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f212c004ea0 con 0x55e617b66f60 2024-03-15T13:05:48.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f213a67c740 10 client.17643 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:48.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f213a67c740 1 -- 172.21.15.2:0/3469334288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55e617a8f6a0 con 0x55e617b66f60 2024-03-15T13:05:48.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f21197fa640 1 -- 172.21.15.2:0/3469334288 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e6177ce7e0 con 0x55e617b66f60 2024-03-15T13:05:48.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f21197fa640 1 -- 172.21.15.2:0/3469334288 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f212c003820 con 0x55e617b66f60 2024-03-15T13:05:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f213a67c740 10 client.17643 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f213a67c740 10 client.17643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f213a67c740 20 client.17643 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f213a67c740 1 --2- 172.21.15.2:0/3469334288 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e617be1260 0x55e617be3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f213a67c740 4 client.17643 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:05:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.144+0000 7f213a67c740 1 -- 172.21.15.2:0/3469334288 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e6177bd4e0 con 0x55e617be1260 2024-03-15T13:05:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f2135035640 1 --2- 172.21.15.2:0/3469334288 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e617be1260 0x55e617be3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f2135035640 1 --2- 172.21.15.2:0/3469334288 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e617be1260 0x55e617be3650 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfc02731c19cbc55 server_cookie=6f77a88573d4c1ac in_seq=0 out_seq=0 2024-03-15T13:05:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f21197fa640 10 client.17643 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f21197fa640 1 -- 172.21.15.2:0/3469334288 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e6177bd4e0 con 0x55e617be1260 2024-03-15T13:05:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f21197fa640 10 client.17643 handle_command_reply: tid=0 2024-03-15T13:05:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f213a67c740 1 client.17643 shutdown 2024-03-15T13:05:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f213a67c740 1 -- 172.21.15.2:0/3469334288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 msgr2=0x55e617b75cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f213a67c740 1 --2- 172.21.15.2:0/3469334288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b75cb0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f212c006010 tx=0x7f212c023c10 comp rx=0 tx=0).stop 2024-03-15T13:05:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f213a67c740 1 -- 172.21.15.2:0/3469334288 shutdown_connections 2024-03-15T13:05:48.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f213a67c740 1 --2- 172.21.15.2:0/3469334288 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e617be1260 0x55e617be3650 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f213a67c740 1 --2- 172.21.15.2:0/3469334288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e617b74bc0 0x55e617b7adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f213a67c740 1 --2- 172.21.15.2:0/3469334288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e617b67880 0x55e617b761f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f213a67c740 1 --2- 172.21.15.2:0/3469334288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e617b66f60 0x55e617b75cb0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f21197fa640 0 client.17643 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:48.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f213a67c740 1 -- 172.21.15.2:0/3469334288 >> 172.21.15.2:0/3469334288 conn(0x55e617b61990 msgr2=0x55e617b636c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f213a67c740 1 -- 172.21.15.2:0/3469334288 shutdown_connections 2024-03-15T13:05:48.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f213a67c740 1 -- 172.21.15.2:0/3469334288 wait complete. 2024-03-15T13:05:48.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.145+0000 7f213a67c740 20 client.17643 trim_cache size 0 max 16384 2024-03-15T13:05:48.137 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:05:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:05:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:05:48.138 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:05:48.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.146+0000 7f2137a9a640 1 -- 172.21.15.2:0/3124910788 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f211c0a1ba0 msgr2=0x7f211c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.146+0000 7f2137a9a640 1 --2- 172.21.15.2:0/3124910788 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f211c0a1ba0 0x7f211c0a4060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2120005ab0 tx=0x7f2120005fd0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.146+0000 7f2137a9a640 1 -- 172.21.15.2:0/3124910788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2130107570 msgr2=0x7f2130197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.146+0000 7f2137a9a640 1 --2- 172.21.15.2:0/3124910788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2130107570 0x7f2130197940 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f212800eb10 tx=0x7f212800c2d0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.146+0000 7f2137a9a640 1 -- 172.21.15.2:0/3124910788 shutdown_connections 2024-03-15T13:05:48.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.146+0000 7f2137a9a640 1 --2- 172.21.15.2:0/3124910788 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f211c0a1ba0 0x7f211c0a4060 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.146+0000 7f2137a9a640 1 --2- 172.21.15.2:0/3124910788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2130107570 0x7f2130197940 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.146+0000 7f2137a9a640 1 --2- 172.21.15.2:0/3124910788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f213006a500 0x7f213019dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.146+0000 7f2137a9a640 1 --2- 172.21.15.2:0/3124910788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2130069b60 0x7f213019d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.146+0000 7f2137a9a640 1 -- 172.21.15.2:0/3124910788 >> 172.21.15.2:0/3124910788 conn(0x7f21300771d0 msgr2=0x7f21300fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.146+0000 7f2137a9a640 1 -- 172.21.15.2:0/3124910788 shutdown_connections 2024-03-15T13:05:48.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.146+0000 7f2137a9a640 1 -- 172.21.15.2:0/3124910788 wait complete. 2024-03-15T13:05:48.142 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:05:48.142 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:05:48.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.300+0000 7f3619e15640 1 -- 172.21.15.2:0/2476609488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3614102760 msgr2=0x7f3614102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.300+0000 7f3619e15640 1 --2- 172.21.15.2:0/2476609488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3614102760 0x7f3614102b40 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3604009900 tx=0x7f360402bc30 comp rx=0 tx=0).stop 2024-03-15T13:05:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.301+0000 7f3619e15640 1 -- 172.21.15.2:0/2476609488 shutdown_connections 2024-03-15T13:05:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.301+0000 7f3619e15640 1 --2- 172.21.15.2:0/2476609488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f361410bb30 0x7f361410dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.301+0000 7f3619e15640 1 --2- 172.21.15.2:0/2476609488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3614103080 0x7f361410b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.301+0000 7f3619e15640 1 --2- 172.21.15.2:0/2476609488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3614102760 0x7f3614102b40 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.301+0000 7f3619e15640 1 -- 172.21.15.2:0/2476609488 >> 172.21.15.2:0/2476609488 conn(0x7f36140774f0 msgr2=0x7f3614077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.301+0000 7f3619e15640 1 -- 172.21.15.2:0/2476609488 shutdown_connections 2024-03-15T13:05:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.301+0000 7f3619e15640 1 -- 172.21.15.2:0/2476609488 wait complete. 2024-03-15T13:05:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.301+0000 7f3619e15640 1 Processor -- start 2024-03-15T13:05:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.301+0000 7f3619e15640 1 -- start start 2024-03-15T13:05:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.301+0000 7f3619e15640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3614102760 0x7f361419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.301+0000 7f3619e15640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3614103080 0x7f361419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3619e15640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f361410bb30 0x7f3614197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3619e15640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3614197fd0 con 0x7f361410bb30 2024-03-15T13:05:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3619e15640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3614198140 con 0x7f3614102760 2024-03-15T13:05:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3619e15640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3614198300 con 0x7f3614103080 2024-03-15T13:05:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3613fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f361410bb30 0x7f3614197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3613fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f361410bb30 0x7f3614197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53078/0 (socket says 172.21.15.2:53078) 2024-03-15T13:05:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3613fff640 1 -- 172.21.15.2:0/3859965373 learned_addr learned my addr 172.21.15.2:0/3859965373 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3612ffd640 1 --2- 172.21.15.2:0/3859965373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3614103080 0x7f361419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f36137fe640 1 --2- 172.21.15.2:0/3859965373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3614102760 0x7f361419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3613fff640 1 -- 172.21.15.2:0/3859965373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3614103080 msgr2=0x7f361419dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3613fff640 1 --2- 172.21.15.2:0/3859965373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3614103080 0x7f361419dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3613fff640 1 -- 172.21.15.2:0/3859965373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3614102760 msgr2=0x7f361419d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3613fff640 1 --2- 172.21.15.2:0/3859965373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3614102760 0x7f361419d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3613fff640 1 -- 172.21.15.2:0/3859965373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3604009560 con 0x7f361410bb30 2024-03-15T13:05:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3613fff640 1 --2- 172.21.15.2:0/3859965373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f361410bb30 0x7f3614197980 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f360c00eb10 tx=0x7f360c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3610ff9640 1 -- 172.21.15.2:0/3859965373 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c004b20 con 0x7f361410bb30 2024-03-15T13:05:48.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3610ff9640 1 -- 172.21.15.2:0/3859965373 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f360c004c80 con 0x7f361410bb30 2024-03-15T13:05:48.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3619e15640 1 -- 172.21.15.2:0/3859965373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36141985e0 con 0x7f361410bb30 2024-03-15T13:05:48.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3610ff9640 1 -- 172.21.15.2:0/3859965373 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c010690 con 0x7f361410bb30 2024-03-15T13:05:48.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.302+0000 7f3619e15640 1 -- 172.21.15.2:0/3859965373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36141a4870 con 0x7f361410bb30 2024-03-15T13:05:48.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.303+0000 7f3619e15640 1 -- 172.21.15.2:0/3859965373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36140682e0 con 0x7f361410bb30 2024-03-15T13:05:48.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.304+0000 7f3610ff9640 1 -- 172.21.15.2:0/3859965373 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f360c004df0 con 0x7f361410bb30 2024-03-15T13:05:48.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.305+0000 7f3610ff9640 1 --2- 172.21.15.2:0/3859965373 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f35fc0a1ba0 0x7f35fc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.305+0000 7f36137fe640 1 --2- 172.21.15.2:0/3859965373 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f35fc0a1ba0 0x7f35fc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.305+0000 7f3610ff9640 1 -- 172.21.15.2:0/3859965373 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f360c014070 con 0x7f361410bb30 2024-03-15T13:05:48.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.305+0000 7f36137fe640 1 --2- 172.21.15.2:0/3859965373 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f35fc0a1ba0 0x7f35fc0a4060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3604005ab0 tx=0x7f3604005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.309+0000 7f3610ff9640 1 -- 172.21.15.2:0/3859965373 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f360c096d90 con 0x7f361410bb30 2024-03-15T13:05:48.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.535+0000 7f3619e15640 1 -- 172.21.15.2:0/3859965373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3614198eb0 con 0x7f361410bb30 2024-03-15T13:05:48.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.536+0000 7f3610ff9640 1 -- 172.21.15.2:0/3859965373 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+11453 (secure 0 0 0) 0x7f360c09a450 con 0x7f361410bb30 2024-03-15T13:05:48.521 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:05:48.521 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:05:12.138845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24500,24464,24460,24434,24539]},"id":1}]} 2024-03-15T13:05:48.521 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 50 2024-03-15T13:05:48.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.538+0000 7f3619e15640 1 -- 172.21.15.2:0/3859965373 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f35fc0a1ba0 msgr2=0x7f35fc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.538+0000 7f3619e15640 1 --2- 172.21.15.2:0/3859965373 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f35fc0a1ba0 0x7f35fc0a4060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3604005ab0 tx=0x7f3604005fd0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.538+0000 7f3619e15640 1 -- 172.21.15.2:0/3859965373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f361410bb30 msgr2=0x7f3614197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.538+0000 7f3619e15640 1 --2- 172.21.15.2:0/3859965373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f361410bb30 0x7f3614197980 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f360c00eb10 tx=0x7f360c00c2d0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.538+0000 7f3619e15640 1 -- 172.21.15.2:0/3859965373 shutdown_connections 2024-03-15T13:05:48.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.538+0000 7f3619e15640 1 --2- 172.21.15.2:0/3859965373 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f35fc0a1ba0 0x7f35fc0a4060 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.538+0000 7f3619e15640 1 --2- 172.21.15.2:0/3859965373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f361410bb30 0x7f3614197980 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.538+0000 7f3619e15640 1 --2- 172.21.15.2:0/3859965373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3614103080 0x7f361419dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.538+0000 7f3619e15640 1 --2- 172.21.15.2:0/3859965373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3614102760 0x7f361419d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.538+0000 7f3619e15640 1 -- 172.21.15.2:0/3859965373 >> 172.21.15.2:0/3859965373 conn(0x7f36140774f0 msgr2=0x7f36140fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.539+0000 7f3619e15640 1 -- 172.21.15.2:0/3859965373 shutdown_connections 2024-03-15T13:05:48.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.539+0000 7f3619e15640 1 -- 172.21.15.2:0/3859965373 wait complete. 2024-03-15T13:05:48.534 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:05:48.535 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:05:48.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.655+0000 7f0bcc13a640 1 -- 172.21.15.2:0/2249685286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bc4100420 msgr2=0x7f0bc410e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.655+0000 7f0bcc13a640 1 --2- 172.21.15.2:0/2249685286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bc4100420 0x7f0bc410e3b0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009ca0 tx=0x7f0bbc02c0f0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.655+0000 7f0bcc13a640 1 -- 172.21.15.2:0/2249685286 shutdown_connections 2024-03-15T13:05:48.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.655+0000 7f0bcc13a640 1 --2- 172.21.15.2:0/2249685286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bc4100420 0x7f0bc410e3b0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.655+0000 7f0bcc13a640 1 --2- 172.21.15.2:0/2249685286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bc40ffa60 0x7f0bc40ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.655+0000 7f0bcc13a640 1 --2- 172.21.15.2:0/2249685286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc41075f0 0x7f0bc41079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.655+0000 7f0bcc13a640 1 -- 172.21.15.2:0/2249685286 >> 172.21.15.2:0/2249685286 conn(0x7f0bc4076a20 msgr2=0x7f0bc40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.655+0000 7f0bcc13a640 1 -- 172.21.15.2:0/2249685286 shutdown_connections 2024-03-15T13:05:48.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.655+0000 7f0bcc13a640 1 -- 172.21.15.2:0/2249685286 wait complete. 2024-03-15T13:05:48.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bcc13a640 1 Processor -- start 2024-03-15T13:05:48.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bcc13a640 1 -- start start 2024-03-15T13:05:48.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bcc13a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bc40ffa60 0x7f0bc4100c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bcc13a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bc4100420 0x7f0bc4101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bcc13a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc41075f0 0x7f0bc41016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bcc13a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc41a5f10 con 0x7f0bc4100420 2024-03-15T13:05:48.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bcc13a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc41a6080 con 0x7f0bc41075f0 2024-03-15T13:05:48.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bcc13a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc41a6240 con 0x7f0bc40ffa60 2024-03-15T13:05:48.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bc96d5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bc4100420 0x7f0bc4101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bc96d5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bc4100420 0x7f0bc4101180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53102/0 (socket says 172.21.15.2:53102) 2024-03-15T13:05:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bc96d5640 1 -- 172.21.15.2:0/67378689 learned_addr learned my addr 172.21.15.2:0/67378689 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bc9ed6640 1 --2- 172.21.15.2:0/67378689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bc40ffa60 0x7f0bc4100c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bca6d7640 1 --2- 172.21.15.2:0/67378689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc41075f0 0x7f0bc41016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bc96d5640 1 -- 172.21.15.2:0/67378689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bc40ffa60 msgr2=0x7f0bc4100c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bc96d5640 1 --2- 172.21.15.2:0/67378689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bc40ffa60 0x7f0bc4100c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bc96d5640 1 -- 172.21.15.2:0/67378689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc41075f0 msgr2=0x7f0bc41016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bc96d5640 1 --2- 172.21.15.2:0/67378689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc41075f0 0x7f0bc41016c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.656+0000 7f0bc96d5640 1 -- 172.21.15.2:0/67378689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bbc009900 con 0x7f0bc4100420 2024-03-15T13:05:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.657+0000 7f0bc96d5640 1 --2- 172.21.15.2:0/67378689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bc4100420 0x7f0bc4101180 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400e9f0 tx=0x7f0bb400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.657+0000 7f0bbaffd640 1 -- 172.21.15.2:0/67378689 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4009720 con 0x7f0bc4100420 2024-03-15T13:05:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.657+0000 7f0bbaffd640 1 -- 172.21.15.2:0/67378689 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0bb400cd80 con 0x7f0bc4100420 2024-03-15T13:05:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.657+0000 7f0bcc13a640 1 -- 172.21.15.2:0/67378689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc41a6520 con 0x7f0bc4100420 2024-03-15T13:05:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.657+0000 7f0bbaffd640 1 -- 172.21.15.2:0/67378689 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4010690 con 0x7f0bc4100420 2024-03-15T13:05:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.657+0000 7f0bcc13a640 1 -- 172.21.15.2:0/67378689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc41a69e0 con 0x7f0bc4100420 2024-03-15T13:05:48.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.659+0000 7f0bbaffd640 1 -- 172.21.15.2:0/67378689 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f0bb4004e70 con 0x7f0bc4100420 2024-03-15T13:05:48.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.659+0000 7f0bcedeb740 1 Processor -- start 2024-03-15T13:05:48.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bcedeb740 1 -- start start 2024-03-15T13:05:48.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bcedeb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6250 0x561122fa6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bcedeb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122fa6b70 0x561122faf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bcedeb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122faf750 0x561122fb1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bcedeb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561122f61960 con 0x561122fa6b70 2024-03-15T13:05:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bcedeb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561122f15310 con 0x561122fa6250 2024-03-15T13:05:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bcedeb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561122f12840 con 0x561122faf750 2024-03-15T13:05:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bbaffd640 1 --2- 172.21.15.2:0/67378689 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f0bb00a1ba0 0x7f0bb00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bc9ed6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122fa6b70 0x561122faf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bc9ed6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122fa6b70 0x561122faf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53110/0 (socket says 172.21.15.2:53110) 2024-03-15T13:05:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bc9ed6640 1 -- 172.21.15.2:0/4150302819 learned_addr learned my addr 172.21.15.2:0/4150302819 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bca6d7640 1 --2- 172.21.15.2:0/4150302819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6250 0x561122fa6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bc96d5640 1 --2- 172.21.15.2:0/4150302819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122faf750 0x561122fb1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bca6d7640 1 --2- 172.21.15.2:0/67378689 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f0bb00a1ba0 0x7f0bb00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bbaffd640 1 -- 172.21.15.2:0/67378689 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f0bb4014070 con 0x7f0bc4100420 2024-03-15T13:05:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bc9ed6640 1 -- 172.21.15.2:0/4150302819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122faf750 msgr2=0x561122fb1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bc9ed6640 1 --2- 172.21.15.2:0/4150302819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122faf750 0x561122fb1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bc9ed6640 1 -- 172.21.15.2:0/4150302819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6250 msgr2=0x561122fa6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bc9ed6640 1 --2- 172.21.15.2:0/4150302819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6250 0x561122fa6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bc9ed6640 1 -- 172.21.15.2:0/4150302819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561122ed0060 con 0x561122fa6b70 2024-03-15T13:05:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bca6d7640 1 --2- 172.21.15.2:0/67378689 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f0bb00a1ba0 0x7f0bb00a4060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc004af0 tx=0x7f0bbc005c50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.660+0000 7f0bc9ed6640 1 --2- 172.21.15.2:0/4150302819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122fa6b70 0x561122faf210 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009c70 tx=0x7f0bc0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ccfba786079589a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.661+0000 7f0baffff640 1 -- 172.21.15.2:0/4150302819 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0004230 con 0x561122fa6b70 2024-03-15T13:05:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.661+0000 7f0baffff640 1 -- 172.21.15.2:0/4150302819 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0bc0004390 con 0x561122fa6b70 2024-03-15T13:05:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.661+0000 7f0baffff640 1 -- 172.21.15.2:0/4150302819 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0019760 con 0x561122fa6b70 2024-03-15T13:05:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.661+0000 7f0bcedeb740 1 -- 172.21.15.2:0/4150302819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122fa6b70 msgr2=0x561122faf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.661+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/4150302819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122fa6b70 0x561122faf210 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009c70 tx=0x7f0bc0011810 comp rx=0 tx=0).stop 2024-03-15T13:05:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.661+0000 7f0bcedeb740 1 -- 172.21.15.2:0/4150302819 shutdown_connections 2024-03-15T13:05:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.661+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/4150302819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122faf750 0x561122fb1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.661+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/4150302819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122fa6b70 0x561122faf210 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.661+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/4150302819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6250 0x561122fa6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.661+0000 7f0bcedeb740 1 -- 172.21.15.2:0/4150302819 >> 172.21.15.2:0/4150302819 conn(0x561122fa0980 msgr2=0x561122fa0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.661+0000 7f0bcedeb740 1 -- 172.21.15.2:0/4150302819 shutdown_connections 2024-03-15T13:05:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.662+0000 7f0bcedeb740 1 -- 172.21.15.2:0/4150302819 wait complete. 2024-03-15T13:05:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.662+0000 7f0bcedeb740 1 -- start start 2024-03-15T13:05:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.662+0000 7f0bcedeb740 1 Processor -- start 2024-03-15T13:05:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.662+0000 7f0bcedeb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6250 0x561122faf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.662+0000 7f0bcedeb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6b70 0x561122fac480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.662+0000 7f0bcedeb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122faf750 0x561122faca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.662+0000 7f0bcedeb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561122f0e020 con 0x561122faf750 2024-03-15T13:05:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.662+0000 7f0bcedeb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561122be1430 con 0x561122fa6b70 2024-03-15T13:05:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.662+0000 7f0bcedeb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561122b9de00 con 0x561122fa6250 2024-03-15T13:05:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.662+0000 7f0bcedeb740 10 client.0 _ll_register_callbacks cb 0x561122eb5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.662+0000 7f0bca6d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122faf750 0x561122faca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.662+0000 7f0bca6d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122faf750 0x561122faca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53122/0 (socket says 172.21.15.2:53122) 2024-03-15T13:05:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.662+0000 7f0bc96d5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6b70 0x561122fac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.662+0000 7f0bca6d7640 1 -- 172.21.15.2:0/2579979241 learned_addr learned my addr 172.21.15.2:0/2579979241 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.662+0000 7f0bc9ed6640 1 --2- 172.21.15.2:0/2579979241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6250 0x561122faf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.663+0000 7f0bca6d7640 1 -- 172.21.15.2:0/2579979241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6250 msgr2=0x561122faf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.663+0000 7f0bca6d7640 1 --2- 172.21.15.2:0/2579979241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6250 0x561122faf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.663+0000 7f0bca6d7640 1 -- 172.21.15.2:0/2579979241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6b70 msgr2=0x561122fac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.663+0000 7f0bca6d7640 1 --2- 172.21.15.2:0/2579979241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6b70 0x561122fac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.663+0000 7f0bca6d7640 1 -- 172.21.15.2:0/2579979241 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc00098d0 con 0x561122faf750 2024-03-15T13:05:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.663+0000 7f0bca6d7640 1 --2- 172.21.15.2:0/2579979241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122faf750 0x561122faca20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009ca0 tx=0x7f0bbc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c55c2e973b3374 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.663+0000 7f0bad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:48.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.663+0000 7f0bad7fa640 1 -- 172.21.15.2:0/2579979241 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc003b70 con 0x561122faf750 2024-03-15T13:05:48.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.663+0000 7f0bad7fa640 1 -- 172.21.15.2:0/2579979241 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f0bbc003cd0 con 0x561122faf750 2024-03-15T13:05:48.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.663+0000 7f0bcedeb740 1 -- 172.21.15.2:0/2579979241 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561122c0d7e0 con 0x561122faf750 2024-03-15T13:05:48.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.663+0000 7f0bad7fa640 1 -- 172.21.15.2:0/2579979241 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bbc003e90 con 0x561122faf750 2024-03-15T13:05:48.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.663+0000 7f0bcedeb740 10 client.17661 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:48.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.663+0000 7f0bcedeb740 1 -- 172.21.15.2:0/2579979241 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x561122f0c6a0 con 0x561122faf750 2024-03-15T13:05:48.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.663+0000 7f0bad7fa640 1 -- 172.21.15.2:0/2579979241 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x561122c0d7e0 con 0x561122faf750 2024-03-15T13:05:48.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.664+0000 7f0bad7fa640 1 -- 172.21.15.2:0/2579979241 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0bbc04c070 con 0x561122faf750 2024-03-15T13:05:48.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.664+0000 7f0bcedeb740 10 client.17661 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:48.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.664+0000 7f0bcedeb740 10 client.17661 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:05:48.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.664+0000 7f0bcedeb740 20 client.17661 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:48.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.664+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/2579979241 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x561122fbf8b0 0x561122fc1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.664+0000 7f0bcedeb740 4 client.17661 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:48.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.664+0000 7f0bcedeb740 1 -- 172.21.15.2:0/2579979241 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561122c4a720 con 0x561122fbf8b0 2024-03-15T13:05:48.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.664+0000 7f0bc9ed6640 1 --2- 172.21.15.2:0/2579979241 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x561122fbf8b0 0x561122fc1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.664+0000 7f0bc9ed6640 1 --2- 172.21.15.2:0/2579979241 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x561122fbf8b0 0x561122fc1cd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a6b3fb4849b2b05 server_cookie=cea9089c3c47325 in_seq=0 out_seq=0 2024-03-15T13:05:48.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.664+0000 7f0bad7fa640 10 client.17661 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:05:48.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.665+0000 7f0bad7fa640 1 -- 172.21.15.2:0/2579979241 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561122c4a720 con 0x561122fbf8b0 2024-03-15T13:05:48.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.665+0000 7f0bad7fa640 10 client.17661 handle_command_reply: tid=0 2024-03-15T13:05:48.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.665+0000 7f0bcedeb740 1 client.17661 shutdown 2024-03-15T13:05:48.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.665+0000 7f0bcedeb740 1 -- 172.21.15.2:0/2579979241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122faf750 msgr2=0x561122faca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.665+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/2579979241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122faf750 0x561122faca20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009ca0 tx=0x7f0bbc0095c0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.665+0000 7f0bcedeb740 1 -- 172.21.15.2:0/2579979241 shutdown_connections 2024-03-15T13:05:48.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.665+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/2579979241 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x561122fbf8b0 0x561122fc1cd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.665+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/2579979241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122faf750 0x561122faca20 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.665+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/2579979241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6b70 0x561122fac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.665+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/2579979241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6250 0x561122faf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.665+0000 7f0bad7fa640 0 client.17661 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:05:48.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.665+0000 7f0bcedeb740 1 -- 172.21.15.2:0/2579979241 >> 172.21.15.2:0/2579979241 conn(0x561122fa0980 msgr2=0x561122fa26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.665+0000 7f0bcedeb740 1 -- 172.21.15.2:0/2579979241 shutdown_connections 2024-03-15T13:05:48.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.665+0000 7f0bcedeb740 1 -- 172.21.15.2:0/2579979241 wait complete. 2024-03-15T13:05:48.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.665+0000 7f0bcedeb740 20 client.17661 trim_cache size 0 max 16384 2024-03-15T13:05:48.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bcedeb740 1 Processor -- start 2024-03-15T13:05:48.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bcedeb740 1 -- start start 2024-03-15T13:05:48.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bcedeb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122fa6250 0x561122faebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bcedeb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6b70 0x561122faf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bcedeb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122faf750 0x561122fbaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bcedeb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561122e93130 con 0x561122fa6250 2024-03-15T13:05:48.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bcedeb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561122faa9d0 con 0x561122faf750 2024-03-15T13:05:48.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bcedeb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561122fc75c0 con 0x561122fa6b70 2024-03-15T13:05:48.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bc9ed6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122fa6250 0x561122faebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bc9ed6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122fa6250 0x561122faebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53132/0 (socket says 172.21.15.2:53132) 2024-03-15T13:05:48.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bc9ed6640 1 -- 172.21.15.2:0/158543932 learned_addr learned my addr 172.21.15.2:0/158543932 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bca6d7640 1 --2- 172.21.15.2:0/158543932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122faf750 0x561122fbaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bc96d5640 1 --2- 172.21.15.2:0/158543932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6b70 0x561122faf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bc9ed6640 1 -- 172.21.15.2:0/158543932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6b70 msgr2=0x561122faf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bc9ed6640 1 --2- 172.21.15.2:0/158543932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6b70 0x561122faf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bc9ed6640 1 -- 172.21.15.2:0/158543932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122faf750 msgr2=0x561122fbaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bc9ed6640 1 --2- 172.21.15.2:0/158543932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122faf750 0x561122fbaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bc9ed6640 1 -- 172.21.15.2:0/158543932 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561122ea8c20 con 0x561122fa6250 2024-03-15T13:05:48.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.678+0000 7f0bc9ed6640 1 --2- 172.21.15.2:0/158543932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122fa6250 0x561122faebb0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0003f00 tx=0x7f0bc0003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01e7d10a2d2edd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.679+0000 7f0baf7fe640 1 -- 172.21.15.2:0/158543932 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0009e20 con 0x561122fa6250 2024-03-15T13:05:48.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.679+0000 7f0baf7fe640 1 -- 172.21.15.2:0/158543932 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0bc0003ac0 con 0x561122fa6250 2024-03-15T13:05:48.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.679+0000 7f0baf7fe640 1 -- 172.21.15.2:0/158543932 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc001a680 con 0x561122fa6250 2024-03-15T13:05:48.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.679+0000 7f0bcedeb740 1 -- 172.21.15.2:0/158543932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122fa6250 msgr2=0x561122faebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.679+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/158543932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122fa6250 0x561122faebb0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0003f00 tx=0x7f0bc0003f30 comp rx=0 tx=0).stop 2024-03-15T13:05:48.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.679+0000 7f0bcedeb740 1 -- 172.21.15.2:0/158543932 shutdown_connections 2024-03-15T13:05:48.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.679+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/158543932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122faf750 0x561122fbaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.679+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/158543932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6b70 0x561122faf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.679+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/158543932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122fa6250 0x561122faebb0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.679+0000 7f0bcedeb740 1 -- 172.21.15.2:0/158543932 >> 172.21.15.2:0/158543932 conn(0x561122fa0980 msgr2=0x561122f9fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.679+0000 7f0bcedeb740 1 -- 172.21.15.2:0/158543932 shutdown_connections 2024-03-15T13:05:48.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.679+0000 7f0bcedeb740 1 -- 172.21.15.2:0/158543932 wait complete. 2024-03-15T13:05:48.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bcedeb740 1 -- start start 2024-03-15T13:05:48.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bcedeb740 1 Processor -- start 2024-03-15T13:05:48.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bcedeb740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6250 0x561122fb3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bcedeb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6b70 0x561122fb37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bcedeb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122faf750 0x5611230141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bcedeb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561123014700 con 0x561122faf750 2024-03-15T13:05:48.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bcedeb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561123014870 con 0x561122fa6b70 2024-03-15T13:05:48.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bcedeb740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561123014a30 con 0x561122fa6250 2024-03-15T13:05:48.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bcedeb740 10 client.0 _ll_register_callbacks cb 0x7f0bb4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:48.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bca6d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122faf750 0x5611230141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bc9ed6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6250 0x561122fb3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bc96d5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6b70 0x561122fb37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bca6d7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122faf750 0x5611230141c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53140/0 (socket says 172.21.15.2:53140) 2024-03-15T13:05:48.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bc96d5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6b70 0x561122fb37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:58692/0 (socket says 172.21.15.2:58692) 2024-03-15T13:05:48.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bc9ed6640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6250 0x561122fb3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:33098/0 (socket says 172.21.15.2:33098) 2024-03-15T13:05:48.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bca6d7640 1 -- 172.21.15.2:0/995357362 learned_addr learned my addr 172.21.15.2:0/995357362 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bca6d7640 1 -- 172.21.15.2:0/995357362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6250 msgr2=0x561122fb3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bca6d7640 1 --2- 172.21.15.2:0/995357362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6250 0x561122fb3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bca6d7640 1 -- 172.21.15.2:0/995357362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6b70 msgr2=0x561122fb37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bca6d7640 1 --2- 172.21.15.2:0/995357362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6b70 0x561122fb37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bca6d7640 1 -- 172.21.15.2:0/995357362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc001e040 con 0x561122faf750 2024-03-15T13:05:48.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.680+0000 7f0bc9ed6640 1 --2- 172.21.15.2:0/995357362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6250 0x561122fb3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:48.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bca6d7640 1 --2- 172.21.15.2:0/995357362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122faf750 0x5611230141c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009ca0 tx=0x7f0bbc050d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8e7be87e9ba1f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bad7fa640 1 -- 172.21.15.2:0/995357362 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc003d80 con 0x561122faf750 2024-03-15T13:05:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bad7fa640 1 -- 172.21.15.2:0/995357362 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f0bbc004040 con 0x561122faf750 2024-03-15T13:05:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bcedeb740 1 -- 172.21.15.2:0/995357362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bc0004390 con 0x561122faf750 2024-03-15T13:05:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bad7fa640 1 -- 172.21.15.2:0/995357362 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bbc032eb0 con 0x561122faf750 2024-03-15T13:05:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bcedeb740 10 client.17667 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bcedeb740 1 -- 172.21.15.2:0/995357362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x561122ece8c0 con 0x561122faf750 2024-03-15T13:05:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bad7fa640 1 -- 172.21.15.2:0/995357362 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0bc0004390 con 0x561122faf750 2024-03-15T13:05:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bad7fa640 1 -- 172.21.15.2:0/995357362 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f0bbc057040 con 0x561122faf750 2024-03-15T13:05:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bcedeb740 10 client.17667 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bcedeb740 10 client.17667 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:05:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bcedeb740 20 client.17667 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/995357362 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x56112301bf20 0x56112301e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bcedeb740 4 client.17667 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:05:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.681+0000 7f0bcedeb740 1 -- 172.21.15.2:0/995357362 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561122c99890 con 0x56112301bf20 2024-03-15T13:05:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bc9ed6640 1 --2- 172.21.15.2:0/995357362 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x56112301bf20 0x56112301e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bad7fa640 10 client.17667 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:05:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bc9ed6640 1 --2- 172.21.15.2:0/995357362 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x56112301bf20 0x56112301e340 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c0e7fcf0f9818561 server_cookie=2859b22c101ab168 in_seq=0 out_seq=0 2024-03-15T13:05:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bad7fa640 1 -- 172.21.15.2:0/995357362 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561122c99890 con 0x56112301bf20 2024-03-15T13:05:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bad7fa640 10 client.17667 handle_command_reply: tid=0 2024-03-15T13:05:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bcedeb740 1 client.17667 shutdown 2024-03-15T13:05:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bcedeb740 1 -- 172.21.15.2:0/995357362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122faf750 msgr2=0x5611230141c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/995357362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122faf750 0x5611230141c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009ca0 tx=0x7f0bbc050d80 comp rx=0 tx=0).stop 2024-03-15T13:05:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bcedeb740 1 -- 172.21.15.2:0/995357362 shutdown_connections 2024-03-15T13:05:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/995357362 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x56112301bf20 0x56112301e340 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/995357362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561122faf750 0x5611230141c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/995357362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561122fa6b70 0x561122fb37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bad7fa640 0 client.17667 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:05:48.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bcedeb740 1 --2- 172.21.15.2:0/995357362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561122fa6250 0x561122fb3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bcedeb740 1 -- 172.21.15.2:0/995357362 >> 172.21.15.2:0/995357362 conn(0x561122fa0980 msgr2=0x561122fa7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bcedeb740 1 -- 172.21.15.2:0/995357362 shutdown_connections 2024-03-15T13:05:48.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bcedeb740 1 -- 172.21.15.2:0/995357362 wait complete. 2024-03-15T13:05:48.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.682+0000 7f0bcedeb740 20 client.17667 trim_cache size 0 max 16384 2024-03-15T13:05:48.675 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:05:48.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.683+0000 7f0bcc13a640 1 -- 172.21.15.2:0/67378689 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f0bb00a1ba0 msgr2=0x7f0bb00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.683+0000 7f0bcc13a640 1 --2- 172.21.15.2:0/67378689 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f0bb00a1ba0 0x7f0bb00a4060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc004af0 tx=0x7f0bbc005c50 comp rx=0 tx=0).stop 2024-03-15T13:05:48.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.683+0000 7f0bcc13a640 1 -- 172.21.15.2:0/67378689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bc4100420 msgr2=0x7f0bc4101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.683+0000 7f0bcc13a640 1 --2- 172.21.15.2:0/67378689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bc4100420 0x7f0bc4101180 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400e9f0 tx=0x7f0bb400edf0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.683+0000 7f0bcc13a640 1 -- 172.21.15.2:0/67378689 shutdown_connections 2024-03-15T13:05:48.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.683+0000 7f0bcc13a640 1 --2- 172.21.15.2:0/67378689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bc41075f0 0x7f0bc41016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.683+0000 7f0bcc13a640 1 --2- 172.21.15.2:0/67378689 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f0bb00a1ba0 0x7f0bb00a4060 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.683+0000 7f0bcc13a640 1 --2- 172.21.15.2:0/67378689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0bc4100420 0x7f0bc4101180 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.683+0000 7f0bcc13a640 1 --2- 172.21.15.2:0/67378689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bc40ffa60 0x7f0bc4100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.683+0000 7f0bcc13a640 1 -- 172.21.15.2:0/67378689 >> 172.21.15.2:0/67378689 conn(0x7f0bc4076a20 msgr2=0x7f0bc40fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.683+0000 7f0bcc13a640 1 -- 172.21.15.2:0/67378689 shutdown_connections 2024-03-15T13:05:48.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.683+0000 7f0bcc13a640 1 -- 172.21.15.2:0/67378689 wait complete. 2024-03-15T13:05:48.678 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:05:48.678 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:05:48.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.798+0000 7fdbbf3ae640 1 -- 172.21.15.2:0/1184252286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbb80ffa60 msgr2=0x7fdbb80ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.798+0000 7fdbbf3ae640 1 --2- 172.21.15.2:0/1184252286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbb80ffa60 0x7fdbb80ffee0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009900 tx=0x7fdbac02bc30 comp rx=0 tx=0).stop 2024-03-15T13:05:48.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.798+0000 7fdbbf3ae640 1 -- 172.21.15.2:0/1184252286 shutdown_connections 2024-03-15T13:05:48.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.798+0000 7fdbbf3ae640 1 --2- 172.21.15.2:0/1184252286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbb8100420 0x7fdbb810e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.798+0000 7fdbbf3ae640 1 --2- 172.21.15.2:0/1184252286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbb80ffa60 0x7fdbb80ffee0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.798+0000 7fdbbf3ae640 1 --2- 172.21.15.2:0/1184252286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbb81075f0 0x7fdbb81079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.798+0000 7fdbbf3ae640 1 -- 172.21.15.2:0/1184252286 >> 172.21.15.2:0/1184252286 conn(0x7fdbb8076a20 msgr2=0x7fdbb80fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.798+0000 7fdbbf3ae640 1 -- 172.21.15.2:0/1184252286 shutdown_connections 2024-03-15T13:05:48.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.798+0000 7fdbbf3ae640 1 -- 172.21.15.2:0/1184252286 wait complete. 2024-03-15T13:05:48.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbf3ae640 1 Processor -- start 2024-03-15T13:05:48.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbf3ae640 1 -- start start 2024-03-15T13:05:48.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbf3ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbb80ffa60 0x7fdbb819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbf3ae640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbb8100420 0x7fdbb819dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbf3ae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbb81075f0 0x7fdbb8197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbf3ae640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb8197f90 con 0x7fdbb81075f0 2024-03-15T13:05:48.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbf3ae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb8198100 con 0x7fdbb80ffa60 2024-03-15T13:05:48.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbf3ae640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb81982c0 con 0x7fdbb8100420 2024-03-15T13:05:48.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbd94b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbb81075f0 0x7fdbb8197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbd94b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbb81075f0 0x7fdbb8197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53160/0 (socket says 172.21.15.2:53160) 2024-03-15T13:05:48.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbd94b640 1 -- 172.21.15.2:0/1319978097 learned_addr learned my addr 172.21.15.2:0/1319978097 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbd14a640 1 --2- 172.21.15.2:0/1319978097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbb80ffa60 0x7fdbb819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbc949640 1 --2- 172.21.15.2:0/1319978097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbb8100420 0x7fdbb819dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbd94b640 1 -- 172.21.15.2:0/1319978097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbb8100420 msgr2=0x7fdbb819dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbd94b640 1 --2- 172.21.15.2:0/1319978097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbb8100420 0x7fdbb819dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbd94b640 1 -- 172.21.15.2:0/1319978097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbb80ffa60 msgr2=0x7fdbb819d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbd94b640 1 --2- 172.21.15.2:0/1319978097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbb80ffa60 0x7fdbb819d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbd94b640 1 -- 172.21.15.2:0/1319978097 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac009560 con 0x7fdbb81075f0 2024-03-15T13:05:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.799+0000 7fdbbd94b640 1 --2- 172.21.15.2:0/1319978097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbb81075f0 0x7fdbb8197940 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fdba800e970 tx=0x7fdba800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.800+0000 7fdbb67fc640 1 -- 172.21.15.2:0/1319978097 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8009720 con 0x7fdbb81075f0 2024-03-15T13:05:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.800+0000 7fdbbf3ae640 1 -- 172.21.15.2:0/1319978097 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb81985a0 con 0x7fdbb81075f0 2024-03-15T13:05:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.800+0000 7fdbb67fc640 1 -- 172.21.15.2:0/1319978097 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdba80054f0 con 0x7fdbb81075f0 2024-03-15T13:05:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.800+0000 7fdbb67fc640 1 -- 172.21.15.2:0/1319978097 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8010430 con 0x7fdbb81075f0 2024-03-15T13:05:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.800+0000 7fdbbf3ae640 1 -- 172.21.15.2:0/1319978097 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb81a4830 con 0x7fdbb81075f0 2024-03-15T13:05:48.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbb67fc640 1 -- 172.21.15.2:0/1319978097 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fdba8004f90 con 0x7fdbb81075f0 2024-03-15T13:05:48.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbc1f81740 1 Processor -- start 2024-03-15T13:05:48.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbc1f81740 1 -- start start 2024-03-15T13:05:48.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbc1f81740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb31250 0x56350eb31630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbc1f81740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb3a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbc1f81740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb3cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbc1f81740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350eaec960 con 0x56350eb3a750 2024-03-15T13:05:48.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbc1f81740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350eaa0310 con 0x56350eb31250 2024-03-15T13:05:48.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbc1f81740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350ea9d840 con 0x56350eb31b70 2024-03-15T13:05:48.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbb67fc640 1 --2- 172.21.15.2:0/1319978097 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdba00a1ba0 0x7fdba00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbbc949640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb3cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbbd14a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb3a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbbc949640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb3cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53174/0 (socket says 172.21.15.2:53174) 2024-03-15T13:05:48.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbbc949640 1 -- 172.21.15.2:0/1538869778 learned_addr learned my addr 172.21.15.2:0/1538869778 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbbc949640 1 --2- 172.21.15.2:0/1538869778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb31250 0x56350eb31630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbbd94b640 1 --2- 172.21.15.2:0/1319978097 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdba00a1ba0 0x7fdba00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbb67fc640 1 -- 172.21.15.2:0/1319978097 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fdba8014070 con 0x7fdbb81075f0 2024-03-15T13:05:48.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbbd14a640 1 -- 172.21.15.2:0/1538869778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb31250 msgr2=0x56350eb31630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbbd14a640 1 --2- 172.21.15.2:0/1538869778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb31250 0x56350eb31630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbbd14a640 1 -- 172.21.15.2:0/1538869778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 msgr2=0x56350eb3cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbbd14a640 1 --2- 172.21.15.2:0/1538869778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb3cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.803+0000 7fdbbd14a640 1 -- 172.21.15.2:0/1538869778 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56350ea5b060 con 0x56350eb31b70 2024-03-15T13:05:48.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.804+0000 7fdbbd14a640 1 --2- 172.21.15.2:0/1538869778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb3a210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0005fd0 tx=0x7fdbb00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b75c022a15b6e769 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.804+0000 7fdbbd94b640 1 --2- 172.21.15.2:0/1319978097 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdba00a1ba0 0x7fdba00a4060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdba809a510 tx=0x7fdba8083f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.804+0000 7fdb9f7fe640 1 -- 172.21.15.2:0/1538869778 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0003c30 con 0x56350eb31b70 2024-03-15T13:05:48.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.804+0000 7fdb9f7fe640 1 -- 172.21.15.2:0/1538869778 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdbb0003d90 con 0x56350eb31b70 2024-03-15T13:05:48.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.804+0000 7fdb9f7fe640 1 -- 172.21.15.2:0/1538869778 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0019a90 con 0x56350eb31b70 2024-03-15T13:05:48.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.805+0000 7fdbc1f81740 1 -- 172.21.15.2:0/1538869778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 msgr2=0x56350eb3a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.805+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/1538869778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb3a210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0005fd0 tx=0x7fdbb00118d0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.805+0000 7fdbc1f81740 1 -- 172.21.15.2:0/1538869778 shutdown_connections 2024-03-15T13:05:48.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.805+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/1538869778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb3cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.805+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/1538869778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb3a210 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.805+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/1538869778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb31250 0x56350eb31630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.805+0000 7fdbc1f81740 1 -- 172.21.15.2:0/1538869778 >> 172.21.15.2:0/1538869778 conn(0x56350eb2b980 msgr2=0x56350eb2bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.805+0000 7fdbc1f81740 1 -- 172.21.15.2:0/1538869778 shutdown_connections 2024-03-15T13:05:48.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.805+0000 7fdbc1f81740 1 -- 172.21.15.2:0/1538869778 wait complete. 2024-03-15T13:05:48.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.805+0000 7fdbc1f81740 1 -- start start 2024-03-15T13:05:48.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.805+0000 7fdbc1f81740 1 Processor -- start 2024-03-15T13:05:48.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.805+0000 7fdbc1f81740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb31250 0x56350eb3a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbc1f81740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb37410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbc1f81740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb37950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbc1f81740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350ea99020 con 0x56350eb3a750 2024-03-15T13:05:48.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbc1f81740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350e76c430 con 0x56350eb31250 2024-03-15T13:05:48.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbc1f81740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350e728e00 con 0x56350eb31b70 2024-03-15T13:05:48.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbc1f81740 10 client.0 _ll_register_callbacks cb 0x56350ea40cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:48.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbbc949640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb37950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbbc949640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb37950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53188/0 (socket says 172.21.15.2:53188) 2024-03-15T13:05:48.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbbc949640 1 -- 172.21.15.2:0/2336052960 learned_addr learned my addr 172.21.15.2:0/2336052960 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbbd14a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb37410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbbc949640 1 --2- 172.21.15.2:0/2336052960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb31250 0x56350eb3a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbbc949640 1 -- 172.21.15.2:0/2336052960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 msgr2=0x56350eb37410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbbc949640 1 --2- 172.21.15.2:0/2336052960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb37410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbbc949640 1 -- 172.21.15.2:0/2336052960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb31250 msgr2=0x56350eb3a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbbc949640 1 --2- 172.21.15.2:0/2336052960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb31250 0x56350eb3a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbbc949640 1 -- 172.21.15.2:0/2336052960 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb0009c00 con 0x56350eb3a750 2024-03-15T13:05:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbbd14a640 1 --2- 172.21.15.2:0/2336052960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb37410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbbc949640 1 --2- 172.21.15.2:0/2336052960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb37950 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fdbac035000 tx=0x7fdbac02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7dc256a2ccac803 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdb9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdb9cff9640 1 -- 172.21.15.2:0/2336052960 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac004360 con 0x56350eb3a750 2024-03-15T13:05:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdb9cff9640 1 -- 172.21.15.2:0/2336052960 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fdbac0044c0 con 0x56350eb3a750 2024-03-15T13:05:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.806+0000 7fdbc1f81740 1 -- 172.21.15.2:0/2336052960 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56350e7987e0 con 0x56350eb3a750 2024-03-15T13:05:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.807+0000 7fdb9cff9640 1 -- 172.21.15.2:0/2336052960 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbac004de0 con 0x56350eb3a750 2024-03-15T13:05:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.807+0000 7fdbc1f81740 10 client.17679 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:48.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.807+0000 7fdbc1f81740 1 -- 172.21.15.2:0/2336052960 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x56350ea976a0 con 0x56350eb3a750 2024-03-15T13:05:48.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.807+0000 7fdb9cff9640 1 -- 172.21.15.2:0/2336052960 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x56350e7987e0 con 0x56350eb3a750 2024-03-15T13:05:48.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.807+0000 7fdb9cff9640 1 -- 172.21.15.2:0/2336052960 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdbac003a00 con 0x56350eb3a750 2024-03-15T13:05:48.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.807+0000 7fdbc1f81740 10 client.17679 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:48.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.807+0000 7fdbc1f81740 10 client.17679 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:05:48.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.807+0000 7fdbc1f81740 20 client.17679 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:48.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.807+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/2336052960 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x56350eb4a7f0 0x56350eb4cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.807+0000 7fdbc1f81740 4 client.17679 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:48.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.807+0000 7fdbc1f81740 1 -- 172.21.15.2:0/2336052960 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56350e7d5720 con 0x56350eb4a7f0 2024-03-15T13:05:48.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.807+0000 7fdbbd14a640 1 --2- 172.21.15.2:0/2336052960 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x56350eb4a7f0 0x56350eb4cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.808+0000 7fdbbd14a640 1 --2- 172.21.15.2:0/2336052960 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x56350eb4a7f0 0x56350eb4cbe0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d6f6d21af0dff9cb server_cookie=d092266313e7280c in_seq=0 out_seq=0 2024-03-15T13:05:48.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.808+0000 7fdb9cff9640 10 client.17679 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:05:48.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.808+0000 7fdb9cff9640 1 -- 172.21.15.2:0/2336052960 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56350e7d5720 con 0x56350eb4a7f0 2024-03-15T13:05:48.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.808+0000 7fdb9cff9640 10 client.17679 handle_command_reply: tid=0 2024-03-15T13:05:48.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.808+0000 7fdbc1f81740 1 client.17679 shutdown 2024-03-15T13:05:48.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.808+0000 7fdbc1f81740 1 -- 172.21.15.2:0/2336052960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 msgr2=0x56350eb37950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.808+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/2336052960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb37950 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fdbac035000 tx=0x7fdbac02bc30 comp rx=0 tx=0).stop 2024-03-15T13:05:48.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.808+0000 7fdbc1f81740 1 -- 172.21.15.2:0/2336052960 shutdown_connections 2024-03-15T13:05:48.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.808+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/2336052960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb37950 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.808+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/2336052960 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x56350eb4a7f0 0x56350eb4cbe0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.808+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/2336052960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb37410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.808+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/2336052960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb31250 0x56350eb3a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.808+0000 7fdb9cff9640 0 client.17679 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:05:48.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.808+0000 7fdbc1f81740 1 -- 172.21.15.2:0/2336052960 >> 172.21.15.2:0/2336052960 conn(0x56350eb2b980 msgr2=0x56350eb2d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.809+0000 7fdbc1f81740 1 -- 172.21.15.2:0/2336052960 shutdown_connections 2024-03-15T13:05:48.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.809+0000 7fdbc1f81740 1 -- 172.21.15.2:0/2336052960 wait complete. 2024-03-15T13:05:48.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.809+0000 7fdbc1f81740 20 client.17679 trim_cache size 0 max 16384 2024-03-15T13:05:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:48 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3859965373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:05:48.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.821+0000 7fdbc1f81740 1 Processor -- start 2024-03-15T13:05:48.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.821+0000 7fdbc1f81740 1 -- start start 2024-03-15T13:05:48.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.821+0000 7fdbc1f81740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb31250 0x56350eb39b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.821+0000 7fdbc1f81740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb3a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.821+0000 7fdbc1f81740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb32660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.821+0000 7fdbc1f81740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350ea1e130 con 0x56350eb3a750 2024-03-15T13:05:48.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.821+0000 7fdbc1f81740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350eb37f50 con 0x56350eb31250 2024-03-15T13:05:48.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.821+0000 7fdbc1f81740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350eb37200 con 0x56350eb31b70 2024-03-15T13:05:48.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.821+0000 7fdbbc949640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb32660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.821+0000 7fdbbc949640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb32660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53204/0 (socket says 172.21.15.2:53204) 2024-03-15T13:05:48.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.821+0000 7fdbbd14a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb3a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.821+0000 7fdbbc949640 1 -- 172.21.15.2:0/2053449826 learned_addr learned my addr 172.21.15.2:0/2053449826 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.821+0000 7fdbbc949640 1 --2- 172.21.15.2:0/2053449826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb31250 0x56350eb39b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.822+0000 7fdbbd14a640 1 -- 172.21.15.2:0/2053449826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb31250 msgr2=0x56350eb39b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.822+0000 7fdbbd14a640 1 --2- 172.21.15.2:0/2053449826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb31250 0x56350eb39b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.822+0000 7fdbbd14a640 1 -- 172.21.15.2:0/2053449826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 msgr2=0x56350eb32660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.822+0000 7fdbbd14a640 1 --2- 172.21.15.2:0/2053449826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb32660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.822+0000 7fdbbd14a640 1 -- 172.21.15.2:0/2053449826 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56350ea33c20 con 0x56350eb31b70 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.822+0000 7fdbbc949640 1 --2- 172.21.15.2:0/2053449826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb32660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.822+0000 7fdbbd14a640 1 --2- 172.21.15.2:0/2053449826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb3a080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0000f80 tx=0x7fdbb0019710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b94e9f3ecc43cf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.822+0000 7fdb9effd640 1 -- 172.21.15.2:0/2053449826 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0004b60 con 0x56350eb31b70 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.822+0000 7fdb9effd640 1 -- 172.21.15.2:0/2053449826 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdbb0019c50 con 0x56350eb31b70 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.822+0000 7fdb9effd640 1 -- 172.21.15.2:0/2053449826 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb002c350 con 0x56350eb31b70 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 -- 172.21.15.2:0/2053449826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 msgr2=0x56350eb3a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/2053449826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb3a080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0000f80 tx=0x7fdbb0019710 comp rx=0 tx=0).stop 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 -- 172.21.15.2:0/2053449826 shutdown_connections 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/2053449826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb3a750 0x56350eb32660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/2053449826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb3a080 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/2053449826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb31250 0x56350eb39b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 -- 172.21.15.2:0/2053449826 >> 172.21.15.2:0/2053449826 conn(0x56350eb2b980 msgr2=0x56350eb31f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 -- 172.21.15.2:0/2053449826 shutdown_connections 2024-03-15T13:05:48.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 -- 172.21.15.2:0/2053449826 wait complete. 2024-03-15T13:05:48.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 -- start start 2024-03-15T13:05:48.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 Processor -- start 2024-03-15T13:05:48.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb31250 0x56350eb9e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb9e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb3a750 0x56350eb3e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350eb3ea60 con 0x56350eb31250 2024-03-15T13:05:48.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350eb3ebd0 con 0x56350eb3a750 2024-03-15T13:05:48.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.823+0000 7fdbc1f81740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56350eb3ed90 con 0x56350eb31b70 2024-03-15T13:05:48.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbc1f81740 10 client.0 _ll_register_callbacks cb 0x7fdbac005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:48.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbbc949640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb31250 0x56350eb9e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbbd14a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb9e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbbd14a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb9e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:33174/0 (socket says 172.21.15.2:33174) 2024-03-15T13:05:48.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbbc949640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb31250 0x56350eb9e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53214/0 (socket says 172.21.15.2:53214) 2024-03-15T13:05:48.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbbd14a640 1 -- 172.21.15.2:0/207935829 learned_addr learned my addr 172.21.15.2:0/207935829 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbbc949640 1 --2- 172.21.15.2:0/207935829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb3a750 0x56350eb3e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbbd14a640 1 -- 172.21.15.2:0/207935829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb3a750 msgr2=0x56350eb3e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbbd14a640 1 --2- 172.21.15.2:0/207935829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb3a750 0x56350eb3e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbbd14a640 1 -- 172.21.15.2:0/207935829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb31250 msgr2=0x56350eb9e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbbd14a640 1 --2- 172.21.15.2:0/207935829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb31250 0x56350eb9e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbbd14a640 1 -- 172.21.15.2:0/207935829 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56350ea976a0 con 0x56350eb31b70 2024-03-15T13:05:48.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbbc949640 1 --2- 172.21.15.2:0/207935829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb3a750 0x56350eb3e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:05:48.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbbd14a640 1 --2- 172.21.15.2:0/207935829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb9e910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0003d60 tx=0x7fdbb0004b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d89568518f954ab5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdb9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:05:48.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdb9cff9640 1 -- 172.21.15.2:0/207935829 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb002c350 con 0x56350eb31b70 2024-03-15T13:05:48.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdb9cff9640 1 -- 172.21.15.2:0/207935829 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fdbb002c4b0 con 0x56350eb31b70 2024-03-15T13:05:48.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbc1f81740 1 -- 172.21.15.2:0/207935829 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbb0003d90 con 0x56350eb31b70 2024-03-15T13:05:48.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdb9cff9640 1 -- 172.21.15.2:0/207935829 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbb002c670 con 0x56350eb31b70 2024-03-15T13:05:48.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbc1f81740 10 client.27478 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:48.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.824+0000 7fdbc1f81740 1 -- 172.21.15.2:0/207935829 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x56350ea5b060 con 0x56350eb31b70 2024-03-15T13:05:48.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.825+0000 7fdb9cff9640 1 -- 172.21.15.2:0/207935829 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdbb0003d90 con 0x56350eb31b70 2024-03-15T13:05:48.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.825+0000 7fdb9cff9640 1 -- 172.21.15.2:0/207935829 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fdbb0040070 con 0x56350eb31b70 2024-03-15T13:05:48.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.825+0000 7fdbc1f81740 10 client.27478 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:48.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.825+0000 7fdbc1f81740 10 client.27478 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:05:48.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.825+0000 7fdbc1f81740 20 client.27478 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:48.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.825+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/207935829 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x56350eba7050 0x56350eba9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.825+0000 7fdbc1f81740 4 client.27478 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:05:48.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.825+0000 7fdbc1f81740 1 -- 172.21.15.2:0/207935829 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56350e824890 con 0x56350eba7050 2024-03-15T13:05:48.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.825+0000 7fdbbc949640 1 --2- 172.21.15.2:0/207935829 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x56350eba7050 0x56350eba9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.825+0000 7fdbbc949640 1 --2- 172.21.15.2:0/207935829 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x56350eba7050 0x56350eba9470 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e956a52f49968e11 server_cookie=cfa15589936b7bb6 in_seq=0 out_seq=0 2024-03-15T13:05:48.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.825+0000 7fdb9cff9640 10 client.27478 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:05:48.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.825+0000 7fdb9cff9640 1 -- 172.21.15.2:0/207935829 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56350e824890 con 0x56350eba7050 2024-03-15T13:05:48.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.825+0000 7fdb9cff9640 10 client.27478 handle_command_reply: tid=0 2024-03-15T13:05:48.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.825+0000 7fdbc1f81740 1 client.27478 shutdown 2024-03-15T13:05:48.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbc1f81740 1 -- 172.21.15.2:0/207935829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 msgr2=0x56350eb9e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/207935829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb9e910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0003d60 tx=0x7fdbb0004b80 comp rx=0 tx=0).stop 2024-03-15T13:05:48.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbc1f81740 1 -- 172.21.15.2:0/207935829 shutdown_connections 2024-03-15T13:05:48.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/207935829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56350eb3a750 0x56350eb3e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/207935829 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x56350eba7050 0x56350eba9470 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/207935829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56350eb31b70 0x56350eb9e910 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbc1f81740 1 --2- 172.21.15.2:0/207935829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56350eb31250 0x56350eb9e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdb9cff9640 0 client.27478 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:05:48.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbc1f81740 1 -- 172.21.15.2:0/207935829 >> 172.21.15.2:0/207935829 conn(0x56350eb2b980 msgr2=0x56350eb31f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbc1f81740 1 -- 172.21.15.2:0/207935829 shutdown_connections 2024-03-15T13:05:48.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbc1f81740 1 -- 172.21.15.2:0/207935829 wait complete. 2024-03-15T13:05:48.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbc1f81740 20 client.27478 trim_cache size 0 max 16384 2024-03-15T13:05:48.817 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:05:48.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbbf3ae640 1 -- 172.21.15.2:0/1319978097 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdba00a1ba0 msgr2=0x7fdba00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbbf3ae640 1 --2- 172.21.15.2:0/1319978097 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdba00a1ba0 0x7fdba00a4060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdba809a510 tx=0x7fdba8083f90 comp rx=0 tx=0).stop 2024-03-15T13:05:48.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbbf3ae640 1 -- 172.21.15.2:0/1319978097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbb81075f0 msgr2=0x7fdbb8197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbbf3ae640 1 --2- 172.21.15.2:0/1319978097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbb81075f0 0x7fdbb8197940 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fdba800e970 tx=0x7fdba800ed70 comp rx=0 tx=0).stop 2024-03-15T13:05:48.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbbf3ae640 1 -- 172.21.15.2:0/1319978097 shutdown_connections 2024-03-15T13:05:48.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbbf3ae640 1 --2- 172.21.15.2:0/1319978097 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdba00a1ba0 0x7fdba00a4060 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbbf3ae640 1 --2- 172.21.15.2:0/1319978097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbb81075f0 0x7fdbb8197940 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbbf3ae640 1 --2- 172.21.15.2:0/1319978097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbb8100420 0x7fdbb819dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbbf3ae640 1 --2- 172.21.15.2:0/1319978097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbb80ffa60 0x7fdbb819d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbbf3ae640 1 -- 172.21.15.2:0/1319978097 >> 172.21.15.2:0/1319978097 conn(0x7fdbb8076a20 msgr2=0x7fdbb80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbbf3ae640 1 -- 172.21.15.2:0/1319978097 shutdown_connections 2024-03-15T13:05:48.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.826+0000 7fdbbf3ae640 1 -- 172.21.15.2:0/1319978097 wait complete. 2024-03-15T13:05:48.822 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:05:48.822 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:05:48.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.942+0000 7f5afd9ac640 1 -- 172.21.15.2:0/1958899334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5af8100f80 msgr2=0x7f5af8104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.942+0000 7f5afd9ac640 1 --2- 172.21.15.2:0/1958899334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5af8100f80 0x7f5af8104fc0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009930 tx=0x7f5ae002bc30 comp rx=0 tx=0).stop 2024-03-15T13:05:48.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.942+0000 7f5afd9ac640 1 -- 172.21.15.2:0/1958899334 shutdown_connections 2024-03-15T13:05:48.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.942+0000 7f5afd9ac640 1 --2- 172.21.15.2:0/1958899334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af81056a0 0x7f5af810d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.942+0000 7f5afd9ac640 1 --2- 172.21.15.2:0/1958899334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5af8100f80 0x7f5af8104fc0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.942+0000 7f5afd9ac640 1 --2- 172.21.15.2:0/1958899334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5af81005d0 0x7f5af81009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.942+0000 7f5afd9ac640 1 -- 172.21.15.2:0/1958899334 >> 172.21.15.2:0/1958899334 conn(0x7f5af80774f0 msgr2=0x7f5af8077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.942+0000 7f5afd9ac640 1 -- 172.21.15.2:0/1958899334 shutdown_connections 2024-03-15T13:05:48.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.942+0000 7f5afd9ac640 1 -- 172.21.15.2:0/1958899334 wait complete. 2024-03-15T13:05:48.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5afd9ac640 1 Processor -- start 2024-03-15T13:05:48.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5afd9ac640 1 -- start start 2024-03-15T13:05:48.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5afd9ac640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5af81005d0 0x7f5af819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5afd9ac640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af81056a0 0x7f5af819dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5afd9ac640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5af8197a40 0x7f5af8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5afd9ac640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af81984f0 con 0x7f5af81005d0 2024-03-15T13:05:48.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5afd9ac640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af8198660 con 0x7f5af81056a0 2024-03-15T13:05:48.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5afd9ac640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af8198820 con 0x7f5af8197a40 2024-03-15T13:05:48.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5af6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5af81005d0 0x7f5af819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5af6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5af81005d0 0x7f5af819d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53222/0 (socket says 172.21.15.2:53222) 2024-03-15T13:05:48.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5af6ffd640 1 -- 172.21.15.2:0/493295525 learned_addr learned my addr 172.21.15.2:0/493295525 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5aee7fc640 1 --2- 172.21.15.2:0/493295525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af81056a0 0x7f5af819dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5af77fe640 1 --2- 172.21.15.2:0/493295525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5af8197a40 0x7f5af8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5af6ffd640 1 -- 172.21.15.2:0/493295525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5af8197a40 msgr2=0x7f5af8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5af6ffd640 1 --2- 172.21.15.2:0/493295525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5af8197a40 0x7f5af8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5af6ffd640 1 -- 172.21.15.2:0/493295525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af81056a0 msgr2=0x7f5af819dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5af6ffd640 1 --2- 172.21.15.2:0/493295525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af81056a0 0x7f5af819dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.943+0000 7f5af6ffd640 1 -- 172.21.15.2:0/493295525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af0009840 con 0x7f5af81005d0 2024-03-15T13:05:48.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.944+0000 7f5af6ffd640 1 --2- 172.21.15.2:0/493295525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5af81005d0 0x7f5af819d7f0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800e9f0 tx=0x7f5ae800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.944+0000 7f5afc9aa640 1 -- 172.21.15.2:0/493295525 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8009700 con 0x7f5af81005d0 2024-03-15T13:05:48.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.944+0000 7f5afd9ac640 1 -- 172.21.15.2:0/493295525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae0009560 con 0x7f5af81005d0 2024-03-15T13:05:48.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.944+0000 7f5afc9aa640 1 -- 172.21.15.2:0/493295525 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5ae800ccc0 con 0x7f5af81005d0 2024-03-15T13:05:48.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.944+0000 7f5afc9aa640 1 -- 172.21.15.2:0/493295525 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8018820 con 0x7f5af81005d0 2024-03-15T13:05:48.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.944+0000 7f5afd9ac640 1 -- 172.21.15.2:0/493295525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af81a4a50 con 0x7f5af81005d0 2024-03-15T13:05:48.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.946+0000 7f5afc9aa640 1 -- 172.21.15.2:0/493295525 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f5ae8010430 con 0x7f5af81005d0 2024-03-15T13:05:48.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.946+0000 7f5b00587740 1 Processor -- start 2024-03-15T13:05:48.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.946+0000 7f5b00587740 1 -- start start 2024-03-15T13:05:48.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.946+0000 7f5afc9aa640 1 --2- 172.21.15.2:0/493295525 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ae40a1ba0 0x7f5ae40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.946+0000 7f5b00587740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb0ef00 0x55b2acb0f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.946+0000 7f5b00587740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0f820 0x55b2acb1c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.946+0000 7f5b00587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb1cb60 0x55b2acb1ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.946+0000 7f5b00587740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2acaca960 con 0x55b2acb0ef00 2024-03-15T13:05:48.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.946+0000 7f5b00587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2aca7e310 con 0x55b2acb1cb60 2024-03-15T13:05:48.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.946+0000 7f5b00587740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2aca7b840 con 0x55b2acb0f820 2024-03-15T13:05:48.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.946+0000 7f5af77fe640 1 --2- 172.21.15.2:0/493295525 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ae40a1ba0 0x7f5ae40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.946+0000 7f5af6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb0ef00 0x55b2acb0f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5af6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb0ef00 0x55b2acb0f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53230/0 (socket says 172.21.15.2:53230) 2024-03-15T13:05:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5af6ffd640 1 -- 172.21.15.2:0/1962293488 learned_addr learned my addr 172.21.15.2:0/1962293488 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5aee7fc640 1 --2- 172.21.15.2:0/1962293488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0f820 0x55b2acb1c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5af77fe640 1 --2- 172.21.15.2:0/1962293488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb1cb60 0x55b2acb1ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5afc9aa640 1 -- 172.21.15.2:0/493295525 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5ae8014070 con 0x7f5af81005d0 2024-03-15T13:05:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5af6ffd640 1 -- 172.21.15.2:0/1962293488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0f820 msgr2=0x55b2acb1c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5af6ffd640 1 --2- 172.21.15.2:0/1962293488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0f820 0x55b2acb1c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5af6ffd640 1 -- 172.21.15.2:0/1962293488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb1cb60 msgr2=0x55b2acb1ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5af6ffd640 1 --2- 172.21.15.2:0/1962293488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb1cb60 0x55b2acb1ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5af6ffd640 1 -- 172.21.15.2:0/1962293488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2aca39060 con 0x55b2acb0ef00 2024-03-15T13:05:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5aee7fc640 1 --2- 172.21.15.2:0/1962293488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0f820 0x55b2acb1c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5af77fe640 1 --2- 172.21.15.2:0/493295525 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ae40a1ba0 0x7f5ae40a4060 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005fd0 tx=0x7f5af0013970 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5af6ffd640 1 --2- 172.21.15.2:0/1962293488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb0ef00 0x55b2acb0f2e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8096510 tx=0x7f5ae8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa809a55185fb87 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5aedffb640 1 -- 172.21.15.2:0/1962293488 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae80ad260 con 0x55b2acb0ef00 2024-03-15T13:05:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5aedffb640 1 -- 172.21.15.2:0/1962293488 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5ae80ad080 con 0x55b2acb0ef00 2024-03-15T13:05:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.947+0000 7f5aedffb640 1 -- 172.21.15.2:0/1962293488 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae809a830 con 0x55b2acb0ef00 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.948+0000 7f5b00587740 1 -- 172.21.15.2:0/1962293488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb0ef00 msgr2=0x55b2acb0f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.948+0000 7f5b00587740 1 --2- 172.21.15.2:0/1962293488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb0ef00 0x55b2acb0f2e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8096510 tx=0x7f5ae8017000 comp rx=0 tx=0).stop 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.948+0000 7f5b00587740 1 -- 172.21.15.2:0/1962293488 shutdown_connections 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.948+0000 7f5b00587740 1 --2- 172.21.15.2:0/1962293488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb1cb60 0x55b2acb1ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.948+0000 7f5b00587740 1 --2- 172.21.15.2:0/1962293488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0f820 0x55b2acb1c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.948+0000 7f5b00587740 1 --2- 172.21.15.2:0/1962293488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb0ef00 0x55b2acb0f2e0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.948+0000 7f5b00587740 1 -- 172.21.15.2:0/1962293488 >> 172.21.15.2:0/1962293488 conn(0x55b2acb09980 msgr2=0x55b2acb09d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.948+0000 7f5b00587740 1 -- 172.21.15.2:0/1962293488 shutdown_connections 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.948+0000 7f5b00587740 1 -- 172.21.15.2:0/1962293488 wait complete. 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.948+0000 7f5b00587740 1 -- start start 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5b00587740 1 Processor -- start 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5b00587740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0ef00 0x55b2acb1c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5b00587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0f820 0x55b2acb19860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5b00587740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb19dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5b00587740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2aca77020 con 0x55b2acb1cb60 2024-03-15T13:05:48.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5b00587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ac74a430 con 0x55b2acb0f820 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5b00587740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ac706e00 con 0x55b2acb0ef00 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5b00587740 10 client.0 _ll_register_callbacks cb 0x55b2aca1ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5af77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb19dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5af6ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0ef00 0x55b2acb1c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5af77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb19dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53240/0 (socket says 172.21.15.2:53240) 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5af77fe640 1 -- 172.21.15.2:0/4144207557 learned_addr learned my addr 172.21.15.2:0/4144207557 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5aee7fc640 1 --2- 172.21.15.2:0/4144207557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0f820 0x55b2acb19860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5af77fe640 1 -- 172.21.15.2:0/4144207557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0ef00 msgr2=0x55b2acb1c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5af77fe640 1 --2- 172.21.15.2:0/4144207557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0ef00 0x55b2acb1c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5af77fe640 1 -- 172.21.15.2:0/4144207557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0f820 msgr2=0x55b2acb19860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5af77fe640 1 --2- 172.21.15.2:0/4144207557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0f820 0x55b2acb19860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5af77fe640 1 -- 172.21.15.2:0/4144207557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af81a4a50 con 0x55b2acb1cb60 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5af6ffd640 1 --2- 172.21.15.2:0/4144207557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0ef00 0x55b2acb1c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5af77fe640 1 --2- 172.21.15.2:0/4144207557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb19dc0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5af0003bf0 tx=0x7f5af0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7470fba5f9706fab server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.949+0000 7f5ad6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:48.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.950+0000 7f5ad6ffd640 1 -- 172.21.15.2:0/4144207557 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af00046f0 con 0x55b2acb1cb60 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.950+0000 7f5ad6ffd640 1 -- 172.21.15.2:0/4144207557 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5af0021410 con 0x55b2acb1cb60 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.950+0000 7f5b00587740 1 -- 172.21.15.2:0/4144207557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2ac7767e0 con 0x55b2acb1cb60 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.950+0000 7f5ad6ffd640 1 -- 172.21.15.2:0/4144207557 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5af0004850 con 0x55b2acb1cb60 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.950+0000 7f5b00587740 10 client.17697 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.950+0000 7f5b00587740 1 -- 172.21.15.2:0/4144207557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55b2aca756a0 con 0x55b2acb1cb60 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.950+0000 7f5ad6ffd640 1 -- 172.21.15.2:0/4144207557 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55b2ac7767e0 con 0x55b2acb1cb60 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.950+0000 7f5ad6ffd640 1 -- 172.21.15.2:0/4144207557 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5af002a790 con 0x55b2acb1cb60 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.950+0000 7f5b00587740 10 client.17697 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.950+0000 7f5b00587740 10 client.17697 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.950+0000 7f5b00587740 20 client.17697 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.950+0000 7f5b00587740 1 --2- 172.21.15.2:0/4144207557 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b2acb2ccc0 0x55b2acb2f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.950+0000 7f5b00587740 4 client.17697 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.950+0000 7f5b00587740 1 -- 172.21.15.2:0/4144207557 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2ac7b3720 con 0x55b2acb2ccc0 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.950+0000 7f5af6ffd640 1 --2- 172.21.15.2:0/4144207557 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b2acb2ccc0 0x55b2acb2f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.951+0000 7f5af6ffd640 1 --2- 172.21.15.2:0/4144207557 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b2acb2ccc0 0x55b2acb2f0e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fee3113f5f7a5271 server_cookie=38611438f969c09b in_seq=0 out_seq=0 2024-03-15T13:05:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.951+0000 7f5ad6ffd640 10 client.17697 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:05:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.951+0000 7f5ad6ffd640 1 -- 172.21.15.2:0/4144207557 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b2ac7b3720 con 0x55b2acb2ccc0 2024-03-15T13:05:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.951+0000 7f5ad6ffd640 10 client.17697 handle_command_reply: tid=0 2024-03-15T13:05:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.951+0000 7f5b00587740 1 client.17697 shutdown 2024-03-15T13:05:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.952+0000 7f5b00587740 1 -- 172.21.15.2:0/4144207557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 msgr2=0x55b2acb19dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.952+0000 7f5b00587740 1 --2- 172.21.15.2:0/4144207557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb19dc0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5af0003bf0 tx=0x7f5af0004ab0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.952+0000 7f5b00587740 1 -- 172.21.15.2:0/4144207557 shutdown_connections 2024-03-15T13:05:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.952+0000 7f5b00587740 1 --2- 172.21.15.2:0/4144207557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb19dc0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.952+0000 7f5b00587740 1 --2- 172.21.15.2:0/4144207557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0f820 0x55b2acb19860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.952+0000 7f5b00587740 1 --2- 172.21.15.2:0/4144207557 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b2acb2ccc0 0x55b2acb2f0e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.952+0000 7f5b00587740 1 --2- 172.21.15.2:0/4144207557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0ef00 0x55b2acb1c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.952+0000 7f5ad6ffd640 0 client.17697 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:05:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.952+0000 7f5b00587740 1 -- 172.21.15.2:0/4144207557 >> 172.21.15.2:0/4144207557 conn(0x55b2acb09980 msgr2=0x55b2acb0b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.952+0000 7f5b00587740 1 -- 172.21.15.2:0/4144207557 shutdown_connections 2024-03-15T13:05:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.952+0000 7f5b00587740 1 -- 172.21.15.2:0/4144207557 wait complete. 2024-03-15T13:05:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.952+0000 7f5b00587740 20 client.17697 trim_cache size 0 max 16384 2024-03-15T13:05:48.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.964+0000 7f5b00587740 1 Processor -- start 2024-03-15T13:05:48.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.964+0000 7f5b00587740 1 -- start start 2024-03-15T13:05:48.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.964+0000 7f5b00587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0ef00 0x55b2acb145c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.964+0000 7f5b00587740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0f820 0x55b2acb1bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.964+0000 7f5b00587740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb1c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.964+0000 7f5b00587740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ac9fc130 con 0x55b2acb1cb60 2024-03-15T13:05:48.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.964+0000 7f5b00587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2acb14b00 con 0x55b2acb0ef00 2024-03-15T13:05:48.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.964+0000 7f5b00587740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2acb2f740 con 0x55b2acb0f820 2024-03-15T13:05:48.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.964+0000 7f5af77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb1c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.964+0000 7f5af6ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0ef00 0x55b2acb145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.964+0000 7f5af77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb1c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53248/0 (socket says 172.21.15.2:53248) 2024-03-15T13:05:48.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.964+0000 7f5af77fe640 1 -- 172.21.15.2:0/2278135001 learned_addr learned my addr 172.21.15.2:0/2278135001 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.964+0000 7f5aee7fc640 1 --2- 172.21.15.2:0/2278135001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0f820 0x55b2acb1bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5af6ffd640 1 -- 172.21.15.2:0/2278135001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0f820 msgr2=0x55b2acb1bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5af6ffd640 1 --2- 172.21.15.2:0/2278135001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0f820 0x55b2acb1bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5af6ffd640 1 -- 172.21.15.2:0/2278135001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 msgr2=0x55b2acb1c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5af6ffd640 1 --2- 172.21.15.2:0/2278135001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb1c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5af6ffd640 1 -- 172.21.15.2:0/2278135001 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2aca11c20 con 0x55b2acb0ef00 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5af77fe640 1 --2- 172.21.15.2:0/2278135001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb1c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5af6ffd640 1 --2- 172.21.15.2:0/2278135001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0ef00 0x55b2acb145c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55b2acb1f570 tx=0x7f5ae80cd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e98791f5e0e4d4ca server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5aed7fa640 1 -- 172.21.15.2:0/2278135001 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae809a190 con 0x55b2acb0ef00 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5aed7fa640 1 -- 172.21.15.2:0/2278135001 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5ae809af30 con 0x55b2acb0ef00 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5aed7fa640 1 -- 172.21.15.2:0/2278135001 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8096ea0 con 0x55b2acb0ef00 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5b00587740 1 -- 172.21.15.2:0/2278135001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0ef00 msgr2=0x55b2acb145c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5b00587740 1 --2- 172.21.15.2:0/2278135001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0ef00 0x55b2acb145c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55b2acb1f570 tx=0x7f5ae80cd040 comp rx=0 tx=0).stop 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5b00587740 1 -- 172.21.15.2:0/2278135001 shutdown_connections 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5b00587740 1 --2- 172.21.15.2:0/2278135001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb1c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5b00587740 1 --2- 172.21.15.2:0/2278135001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0f820 0x55b2acb1bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5b00587740 1 --2- 172.21.15.2:0/2278135001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0ef00 0x55b2acb145c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.965+0000 7f5b00587740 1 -- 172.21.15.2:0/2278135001 >> 172.21.15.2:0/2278135001 conn(0x55b2acb09980 msgr2=0x55b2acb0b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5b00587740 1 -- 172.21.15.2:0/2278135001 shutdown_connections 2024-03-15T13:05:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5b00587740 1 -- 172.21.15.2:0/2278135001 wait complete. 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5b00587740 1 -- start start 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5b00587740 1 Processor -- start 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5b00587740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0ef00 0x55b2acb204f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5b00587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0f820 0x55b2acb20a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5b00587740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb87530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5b00587740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2acb21100 con 0x55b2acb1cb60 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5b00587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2acb87a70 con 0x55b2acb0f820 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5b00587740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2acb87bb0 con 0x55b2acb0ef00 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5b00587740 10 client.0 _ll_register_callbacks cb 0x7f5af0009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5af77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb87530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5af77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb87530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53254/0 (socket says 172.21.15.2:53254) 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5af77fe640 1 -- 172.21.15.2:0/2307041154 learned_addr learned my addr 172.21.15.2:0/2307041154 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5aee7fc640 1 --2- 172.21.15.2:0/2307041154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0f820 0x55b2acb20a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.966+0000 7f5af6ffd640 1 --2- 172.21.15.2:0/2307041154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0ef00 0x55b2acb204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.967+0000 7f5af77fe640 1 -- 172.21.15.2:0/2307041154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0ef00 msgr2=0x55b2acb204f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.967+0000 7f5af77fe640 1 --2- 172.21.15.2:0/2307041154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0ef00 0x55b2acb204f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.967+0000 7f5af77fe640 1 -- 172.21.15.2:0/2307041154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0f820 msgr2=0x55b2acb20a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:48.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.967+0000 7f5af77fe640 1 --2- 172.21.15.2:0/2307041154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0f820 0x55b2acb20a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.967+0000 7f5af77fe640 1 -- 172.21.15.2:0/2307041154 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2aca756a0 con 0x55b2acb1cb60 2024-03-15T13:05:48.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.967+0000 7f5af77fe640 1 --2- 172.21.15.2:0/2307041154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb87530 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5af002ac40 tx=0x7f5af00137b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d06f2af92367f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:48.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.967+0000 7f5ad6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:48.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.967+0000 7f5ad6ffd640 1 -- 172.21.15.2:0/2307041154 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0004900 con 0x55b2acb1cb60 2024-03-15T13:05:48.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.967+0000 7f5b00587740 1 -- 172.21.15.2:0/2307041154 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ae80ad080 con 0x55b2acb1cb60 2024-03-15T13:05:48.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.967+0000 7f5ad6ffd640 1 -- 172.21.15.2:0/2307041154 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f5af002a900 con 0x55b2acb1cb60 2024-03-15T13:05:48.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.967+0000 7f5ad6ffd640 1 -- 172.21.15.2:0/2307041154 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5af0038440 con 0x55b2acb1cb60 2024-03-15T13:05:48.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.967+0000 7f5b00587740 10 client.17703 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:48.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.967+0000 7f5b00587740 1 -- 172.21.15.2:0/2307041154 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55b2aca39060 con 0x55b2acb1cb60 2024-03-15T13:05:48.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.967+0000 7f5ad6ffd640 1 -- 172.21.15.2:0/2307041154 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5ae80ad080 con 0x55b2acb1cb60 2024-03-15T13:05:48.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.967+0000 7f5ad6ffd640 1 -- 172.21.15.2:0/2307041154 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5af0029770 con 0x55b2acb1cb60 2024-03-15T13:05:48.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5b00587740 10 client.17703 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:48.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5b00587740 10 client.17703 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5b00587740 20 client.17703 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5b00587740 1 --2- 172.21.15.2:0/2307041154 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b2acb89340 0x55b2acb8b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5b00587740 4 client.17703 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5b00587740 1 -- 172.21.15.2:0/2307041154 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b2ac802890 con 0x55b2acb89340 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5af6ffd640 1 --2- 172.21.15.2:0/2307041154 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b2acb89340 0x55b2acb8b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5af6ffd640 1 --2- 172.21.15.2:0/2307041154 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b2acb89340 0x55b2acb8b760 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a2723e436aeba71 server_cookie=a3f0d33d9fb83da1 in_seq=0 out_seq=0 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5ad6ffd640 10 client.17703 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5ad6ffd640 1 -- 172.21.15.2:0/2307041154 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b2ac802890 con 0x55b2acb89340 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5ad6ffd640 10 client.17703 handle_command_reply: tid=0 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5b00587740 1 client.17703 shutdown 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5b00587740 1 -- 172.21.15.2:0/2307041154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 msgr2=0x55b2acb87530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5b00587740 1 --2- 172.21.15.2:0/2307041154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb87530 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5af002ac40 tx=0x7f5af00137b0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5b00587740 1 -- 172.21.15.2:0/2307041154 shutdown_connections 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5b00587740 1 --2- 172.21.15.2:0/2307041154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2acb1cb60 0x55b2acb87530 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5b00587740 1 --2- 172.21.15.2:0/2307041154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2acb0f820 0x55b2acb20a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5b00587740 1 --2- 172.21.15.2:0/2307041154 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b2acb89340 0x55b2acb8b760 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5b00587740 1 --2- 172.21.15.2:0/2307041154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b2acb0ef00 0x55b2acb204f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5b00587740 1 -- 172.21.15.2:0/2307041154 >> 172.21.15.2:0/2307041154 conn(0x55b2acb09980 msgr2=0x55b2acb082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.968+0000 7f5ad6ffd640 0 client.17703 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:05:48.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.969+0000 7f5b00587740 1 -- 172.21.15.2:0/2307041154 shutdown_connections 2024-03-15T13:05:48.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.969+0000 7f5b00587740 1 -- 172.21.15.2:0/2307041154 wait complete. 2024-03-15T13:05:48.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.969+0000 7f5b00587740 20 client.17703 trim_cache size 0 max 16384 2024-03-15T13:05:48.959 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:05:48.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.969+0000 7f5afd9ac640 1 -- 172.21.15.2:0/493295525 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ae40a1ba0 msgr2=0x7f5ae40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.969+0000 7f5afd9ac640 1 --2- 172.21.15.2:0/493295525 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ae40a1ba0 0x7f5ae40a4060 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005fd0 tx=0x7f5af0013970 comp rx=0 tx=0).stop 2024-03-15T13:05:48.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.969+0000 7f5afd9ac640 1 -- 172.21.15.2:0/493295525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5af81005d0 msgr2=0x7f5af819d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:48.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.969+0000 7f5afd9ac640 1 --2- 172.21.15.2:0/493295525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5af81005d0 0x7f5af819d7f0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800e9f0 tx=0x7f5ae800edf0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.969+0000 7f5afd9ac640 1 -- 172.21.15.2:0/493295525 shutdown_connections 2024-03-15T13:05:48.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.969+0000 7f5afd9ac640 1 --2- 172.21.15.2:0/493295525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5af8197a40 0x7f5af8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.969+0000 7f5afd9ac640 1 --2- 172.21.15.2:0/493295525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af81056a0 0x7f5af819dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.969+0000 7f5afd9ac640 1 --2- 172.21.15.2:0/493295525 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ae40a1ba0 0x7f5ae40a4060 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.969+0000 7f5afd9ac640 1 --2- 172.21.15.2:0/493295525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5af81005d0 0x7f5af819d7f0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:48.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.969+0000 7f5afd9ac640 1 -- 172.21.15.2:0/493295525 >> 172.21.15.2:0/493295525 conn(0x7f5af80774f0 msgr2=0x7f5af8104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:48.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.969+0000 7f5afd9ac640 1 -- 172.21.15.2:0/493295525 shutdown_connections 2024-03-15T13:05:48.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:48.969+0000 7f5afd9ac640 1 -- 172.21.15.2:0/493295525 wait complete. 2024-03-15T13:05:48.965 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:05:48.965 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:05:49.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:48 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3859965373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:05:49.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.084+0000 7fe0a330d640 1 -- 172.21.15.2:0/2668238545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe09c106530 msgr2=0x7fe09c106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.084+0000 7fe0a330d640 1 --2- 172.21.15.2:0/2668238545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe09c106530 0x7fe09c106910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe094009860 tx=0x7fe09402bb70 comp rx=0 tx=0).stop 2024-03-15T13:05:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.085+0000 7fe0a330d640 1 -- 172.21.15.2:0/2668238545 shutdown_connections 2024-03-15T13:05:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.085+0000 7fe0a330d640 1 --2- 172.21.15.2:0/2668238545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe09c101020 0x7fe09c10d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.085+0000 7fe0a330d640 1 --2- 172.21.15.2:0/2668238545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe09c100660 0x7fe09c100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.085+0000 7fe0a330d640 1 --2- 172.21.15.2:0/2668238545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe09c106530 0x7fe09c106910 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.085+0000 7fe0a330d640 1 -- 172.21.15.2:0/2668238545 >> 172.21.15.2:0/2668238545 conn(0x7fe09c0772e0 msgr2=0x7fe09c0776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.085+0000 7fe0a330d640 1 -- 172.21.15.2:0/2668238545 shutdown_connections 2024-03-15T13:05:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.085+0000 7fe0a330d640 1 -- 172.21.15.2:0/2668238545 wait complete. 2024-03-15T13:05:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.085+0000 7fe0a330d640 1 Processor -- start 2024-03-15T13:05:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.085+0000 7fe0a330d640 1 -- start start 2024-03-15T13:05:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.085+0000 7fe0a330d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe09c100660 0x7fe09c102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.085+0000 7fe0a330d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe09c101020 0x7fe09c1025d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.085+0000 7fe0a330d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe09c106530 0x7fe09c102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.085+0000 7fe0a330d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c1a5e30 con 0x7fe09c106530 2024-03-15T13:05:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.085+0000 7fe0a330d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c1a5fa0 con 0x7fe09c100660 2024-03-15T13:05:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.085+0000 7fe0a330d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c1a6160 con 0x7fe09c101020 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe0a08a8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe09c101020 0x7fe09c1025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe0a18aa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe09c106530 0x7fe09c102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe0a08a8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe09c101020 0x7fe09c1025d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:33246/0 (socket says 172.21.15.2:33246) 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe0a10a9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe09c100660 0x7fe09c102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe0a08a8640 1 -- 172.21.15.2:0/4100528028 learned_addr learned my addr 172.21.15.2:0/4100528028 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe0a08a8640 1 -- 172.21.15.2:0/4100528028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe09c100660 msgr2=0x7fe09c102090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe0a08a8640 1 --2- 172.21.15.2:0/4100528028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe09c100660 0x7fe09c102090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe0a08a8640 1 -- 172.21.15.2:0/4100528028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe09c106530 msgr2=0x7fe09c102b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe0a08a8640 1 --2- 172.21.15.2:0/4100528028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe09c106530 0x7fe09c102b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe0a08a8640 1 -- 172.21.15.2:0/4100528028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe090009560 con 0x7fe09c101020 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe0a18aa640 1 --2- 172.21.15.2:0/4100528028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe09c106530 0x7fe09c102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe0a08a8640 1 --2- 172.21.15.2:0/4100528028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe09c101020 0x7fe09c1025d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe09000e9c0 tx=0x7fe09000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe09a7fc640 1 -- 172.21.15.2:0/4100528028 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09000cde0 con 0x7fe09c101020 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe0a330d640 1 -- 172.21.15.2:0/4100528028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe094009490 con 0x7fe09c101020 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe09a7fc640 1 -- 172.21.15.2:0/4100528028 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe0900035f0 con 0x7fe09c101020 2024-03-15T13:05:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe09a7fc640 1 -- 172.21.15.2:0/4100528028 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe090010670 con 0x7fe09c101020 2024-03-15T13:05:49.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.086+0000 7fe0a330d640 1 -- 172.21.15.2:0/4100528028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c1a67c0 con 0x7fe09c101020 2024-03-15T13:05:49.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe09a7fc640 1 -- 172.21.15.2:0/4100528028 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fe090003760 con 0x7fe09c101020 2024-03-15T13:05:49.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a5fc4740 1 Processor -- start 2024-03-15T13:05:49.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a5fc4740 1 -- start start 2024-03-15T13:05:49.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a5fc4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab601250 0x557cab601630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a5fc4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab60a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a5fc4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab60a750 0x557cab60cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a5fc4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cab5bc960 con 0x557cab601250 2024-03-15T13:05:49.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a5fc4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cab570310 con 0x557cab601b70 2024-03-15T13:05:49.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a5fc4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cab56d840 con 0x557cab60a750 2024-03-15T13:05:49.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a18aa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab601250 0x557cab601630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a18aa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab601250 0x557cab601630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53284/0 (socket says 172.21.15.2:53284) 2024-03-15T13:05:49.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a18aa640 1 -- 172.21.15.2:0/796637747 learned_addr learned my addr 172.21.15.2:0/796637747 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:49.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a10a9640 1 --2- 172.21.15.2:0/796637747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab60a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a08a8640 1 --2- 172.21.15.2:0/796637747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab60a750 0x557cab60cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe09a7fc640 1 --2- 172.21.15.2:0/4100528028 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe0840a1ba0 0x7fe0840a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a10a9640 1 -- 172.21.15.2:0/796637747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab60a750 msgr2=0x557cab60cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a10a9640 1 --2- 172.21.15.2:0/796637747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab60a750 0x557cab60cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a10a9640 1 -- 172.21.15.2:0/796637747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab601250 msgr2=0x557cab601630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a10a9640 1 --2- 172.21.15.2:0/796637747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab601250 0x557cab601630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a10a9640 1 -- 172.21.15.2:0/796637747 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cab52b060 con 0x557cab601b70 2024-03-15T13:05:49.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a18aa640 1 --2- 172.21.15.2:0/796637747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab601250 0x557cab601630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:49.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a08a8640 1 --2- 172.21.15.2:0/796637747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab60a750 0x557cab60cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:49.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.089+0000 7fe0a18aa640 1 --2- 172.21.15.2:0/4100528028 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe0840a1ba0 0x7fe0840a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.090+0000 7fe09a7fc640 1 -- 172.21.15.2:0/4100528028 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fe090014070 con 0x7fe09c101020 2024-03-15T13:05:49.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.090+0000 7fe0a10a9640 1 --2- 172.21.15.2:0/796637747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab60a210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe09402c180 tx=0x7fe0940323c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a402cd36d90864 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:49.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.090+0000 7fe0837fe640 1 -- 172.21.15.2:0/796637747 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094004030 con 0x557cab601b70 2024-03-15T13:05:49.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.090+0000 7fe0a18aa640 1 --2- 172.21.15.2:0/4100528028 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe0840a1ba0 0x7fe0840a4060 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009fd0 tx=0x7fe08c004a40 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:49.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.090+0000 7fe0837fe640 1 -- 172.21.15.2:0/796637747 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe094034440 con 0x557cab601b70 2024-03-15T13:05:49.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.090+0000 7fe0837fe640 1 -- 172.21.15.2:0/796637747 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09402c980 con 0x557cab601b70 2024-03-15T13:05:49.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/796637747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 msgr2=0x557cab60a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/796637747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab60a210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe09402c180 tx=0x7fe0940323c0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/796637747 shutdown_connections 2024-03-15T13:05:49.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/796637747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab60a750 0x557cab60cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/796637747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab60a210 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/796637747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab601250 0x557cab601630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/796637747 >> 172.21.15.2:0/796637747 conn(0x557cab5fb980 msgr2=0x557cab5fbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:49.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/796637747 shutdown_connections 2024-03-15T13:05:49.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/796637747 wait complete. 2024-03-15T13:05:49.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 -- start start 2024-03-15T13:05:49.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 Processor -- start 2024-03-15T13:05:49.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab601250 0x557cab60a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab607440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab60a750 0x557cab6079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cab569020 con 0x557cab60a750 2024-03-15T13:05:49.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cab23c430 con 0x557cab601b70 2024-03-15T13:05:49.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cab1f8e00 con 0x557cab601250 2024-03-15T13:05:49.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.091+0000 7fe0a5fc4740 10 client.0 _ll_register_callbacks cb 0x557cab510cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:49.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe0a18aa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab60a750 0x557cab6079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe0a18aa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab60a750 0x557cab6079a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53288/0 (socket says 172.21.15.2:53288) 2024-03-15T13:05:49.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe0a08a8640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab607440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe0a18aa640 1 -- 172.21.15.2:0/2015643504 learned_addr learned my addr 172.21.15.2:0/2015643504 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:49.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe0a10a9640 1 --2- 172.21.15.2:0/2015643504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab601250 0x557cab60a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe0a18aa640 1 -- 172.21.15.2:0/2015643504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab601250 msgr2=0x557cab60a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe0a18aa640 1 --2- 172.21.15.2:0/2015643504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab601250 0x557cab60a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe0a18aa640 1 -- 172.21.15.2:0/2015643504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 msgr2=0x557cab607440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe0a18aa640 1 --2- 172.21.15.2:0/2015643504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab607440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe0a18aa640 1 -- 172.21.15.2:0/2015643504 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c10daf0 con 0x557cab60a750 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe0a18aa640 1 --2- 172.21.15.2:0/2015643504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab60a750 0x557cab6079a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe08c01c6a0 tx=0x7fe08c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22db6cc4dc16a55a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe080ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe080ff9640 1 -- 172.21.15.2:0/2015643504 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c00e7c0 con 0x557cab60a750 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/2015643504 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cab2687e0 con 0x557cab60a750 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe080ff9640 1 -- 172.21.15.2:0/2015643504 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fe08c00e920 con 0x557cab60a750 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe080ff9640 1 -- 172.21.15.2:0/2015643504 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe08c00eb20 con 0x557cab60a750 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe0a5fc4740 10 client.17715 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.092+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/2015643504 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x557cab5676a0 con 0x557cab60a750 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.093+0000 7fe080ff9640 1 -- 172.21.15.2:0/2015643504 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x557cab2687e0 con 0x557cab60a750 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.093+0000 7fe080ff9640 1 -- 172.21.15.2:0/2015643504 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe08c0263a0 con 0x557cab60a750 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.093+0000 7fe0a5fc4740 10 client.17715 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.093+0000 7fe0a5fc4740 10 client.17715 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.093+0000 7fe0a5fc4740 20 client.17715 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.093+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/2015643504 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557cab61a7a0 0x557cab61cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.093+0000 7fe0a5fc4740 4 client.17715 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:49.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.093+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/2015643504 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557cab2a5720 con 0x557cab61a7a0 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.093+0000 7fe0a10a9640 1 --2- 172.21.15.2:0/2015643504 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557cab61a7a0 0x557cab61cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.093+0000 7fe080ff9640 10 client.17715 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.093+0000 7fe0a10a9640 1 --2- 172.21.15.2:0/2015643504 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557cab61a7a0 0x557cab61cbc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=778049e030e4ff38 server_cookie=7891e3ff81d96a46 in_seq=0 out_seq=0 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.094+0000 7fe080ff9640 1 -- 172.21.15.2:0/2015643504 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557cab2a5720 con 0x557cab61a7a0 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.094+0000 7fe080ff9640 10 client.17715 handle_command_reply: tid=0 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.094+0000 7fe0a5fc4740 1 client.17715 shutdown 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.094+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/2015643504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab60a750 msgr2=0x557cab6079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.094+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/2015643504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab60a750 0x557cab6079a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe08c01c6a0 tx=0x7fe08c004ab0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.094+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/2015643504 shutdown_connections 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.094+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/2015643504 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557cab61a7a0 0x557cab61cbc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.094+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/2015643504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab60a750 0x557cab6079a0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.094+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/2015643504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab607440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.094+0000 7fe080ff9640 0 client.17715 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.094+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/2015643504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab601250 0x557cab60a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.094+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/2015643504 >> 172.21.15.2:0/2015643504 conn(0x557cab5fb980 msgr2=0x557cab5fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:49.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.094+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/2015643504 shutdown_connections 2024-03-15T13:05:49.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.094+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/2015643504 wait complete. 2024-03-15T13:05:49.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.094+0000 7fe0a5fc4740 20 client.17715 trim_cache size 0 max 16384 2024-03-15T13:05:49.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a5fc4740 1 Processor -- start 2024-03-15T13:05:49.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a5fc4740 1 -- start start 2024-03-15T13:05:49.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a5fc4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab601250 0x557cab5faa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a5fc4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab609b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a5fc4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab60a750 0x557cab60a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a5fc4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cab4ee130 con 0x557cab60a750 2024-03-15T13:05:49.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a5fc4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cab60cba0 con 0x557cab601b70 2024-03-15T13:05:49.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a5fc4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cab5f7300 con 0x557cab601250 2024-03-15T13:05:49.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a18aa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab60a750 0x557cab60a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a18aa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab60a750 0x557cab60a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53294/0 (socket says 172.21.15.2:53294) 2024-03-15T13:05:49.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a18aa640 1 -- 172.21.15.2:0/1485567419 learned_addr learned my addr 172.21.15.2:0/1485567419 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:49.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a10a9640 1 --2- 172.21.15.2:0/1485567419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab601250 0x557cab5faa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a08a8640 1 --2- 172.21.15.2:0/1485567419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab609b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a18aa640 1 -- 172.21.15.2:0/1485567419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab601250 msgr2=0x557cab5faa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a18aa640 1 --2- 172.21.15.2:0/1485567419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab601250 0x557cab5faa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a18aa640 1 -- 172.21.15.2:0/1485567419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 msgr2=0x557cab609b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a18aa640 1 --2- 172.21.15.2:0/1485567419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab609b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a18aa640 1 -- 172.21.15.2:0/1485567419 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cab503c20 con 0x557cab60a750 2024-03-15T13:05:49.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a08a8640 1 --2- 172.21.15.2:0/1485567419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab609b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:05:49.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.107+0000 7fe0a18aa640 1 --2- 172.21.15.2:0/1485567419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab60a750 0x557cab60a0b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0267b0 tx=0x7fe08c005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45006ee32b124a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:49.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.108+0000 7fe082ffd640 1 -- 172.21.15.2:0/1485567419 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c00fe50 con 0x557cab60a750 2024-03-15T13:05:49.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.108+0000 7fe082ffd640 1 -- 172.21.15.2:0/1485567419 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe08c015040 con 0x557cab60a750 2024-03-15T13:05:49.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.108+0000 7fe082ffd640 1 -- 172.21.15.2:0/1485567419 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c00ecf0 con 0x557cab60a750 2024-03-15T13:05:49.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.108+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/1485567419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab60a750 msgr2=0x557cab60a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.108+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/1485567419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab60a750 0x557cab60a0b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0267b0 tx=0x7fe08c005f30 comp rx=0 tx=0).stop 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.108+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/1485567419 shutdown_connections 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.108+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/1485567419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab60a750 0x557cab60a0b0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.108+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/1485567419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab609b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.108+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/1485567419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab601250 0x557cab5faa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.108+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/1485567419 >> 172.21.15.2:0/1485567419 conn(0x557cab5fb980 msgr2=0x557cab5fa2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.108+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/1485567419 shutdown_connections 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.108+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/1485567419 wait complete. 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.108+0000 7fe0a5fc4740 1 -- start start 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a5fc4740 1 Processor -- start 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a5fc4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab601250 0x557cab612130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a5fc4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab60e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a5fc4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab60a750 0x557cab60e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a5fc4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cab60efb0 con 0x557cab601250 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a5fc4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cab60f120 con 0x557cab601b70 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a5fc4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cab60f2e0 con 0x557cab60a750 2024-03-15T13:05:49.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a5fc4740 10 client.0 _ll_register_callbacks cb 0x7fe08c011880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a10a9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab601250 0x557cab612130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a10a9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab601250 0x557cab612130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53304/0 (socket says 172.21.15.2:53304) 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a10a9640 1 -- 172.21.15.2:0/3216742207 learned_addr learned my addr 172.21.15.2:0/3216742207 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a08a8640 1 --2- 172.21.15.2:0/3216742207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab60e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a18aa640 1 --2- 172.21.15.2:0/3216742207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab60a750 0x557cab60e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a10a9640 1 -- 172.21.15.2:0/3216742207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab60a750 msgr2=0x557cab60e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a10a9640 1 --2- 172.21.15.2:0/3216742207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab60a750 0x557cab60e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a10a9640 1 -- 172.21.15.2:0/3216742207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 msgr2=0x557cab60e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a10a9640 1 --2- 172.21.15.2:0/3216742207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab60e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a10a9640 1 -- 172.21.15.2:0/3216742207 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08c01ce70 con 0x557cab601250 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a18aa640 1 --2- 172.21.15.2:0/3216742207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab60a750 0x557cab60e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe0a10a9640 1 --2- 172.21.15.2:0/3216742207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab601250 0x557cab612130 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe094000c00 tx=0x7fe094005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581e7fd8943e1f11 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.109+0000 7fe080ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.110+0000 7fe080ff9640 1 -- 172.21.15.2:0/3216742207 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094004240 con 0x557cab601250 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.110+0000 7fe080ff9640 1 -- 172.21.15.2:0/3216742207 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7fe09404e070 con 0x557cab601250 2024-03-15T13:05:49.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.110+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/3216742207 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cab5f70f0 con 0x557cab601250 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.110+0000 7fe080ff9640 1 -- 172.21.15.2:0/3216742207 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0940043a0 con 0x557cab601250 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.110+0000 7fe0a5fc4740 10 client.17721 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.110+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/3216742207 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x557cab5298c0 con 0x557cab601250 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.110+0000 7fe080ff9640 1 -- 172.21.15.2:0/3216742207 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x557cab5f70f0 con 0x557cab601250 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.110+0000 7fe080ff9640 1 -- 172.21.15.2:0/3216742207 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe094052790 con 0x557cab601250 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.110+0000 7fe0a5fc4740 10 client.17721 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.110+0000 7fe0a5fc4740 10 client.17721 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.110+0000 7fe0a5fc4740 20 client.17721 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.110+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/3216742207 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557cab677350 0x557cab679770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.110+0000 7fe0a5fc4740 4 client.17721 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.110+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/3216742207 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fe094034440 con 0x557cab677350 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.110+0000 7fe0a08a8640 1 --2- 172.21.15.2:0/3216742207 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557cab677350 0x557cab679770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe0a08a8640 1 --2- 172.21.15.2:0/3216742207 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557cab677350 0x557cab679770 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e6869ce5854fab0 server_cookie=51640c16391c1a71 in_seq=0 out_seq=0 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe080ff9640 10 client.17721 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe080ff9640 1 -- 172.21.15.2:0/3216742207 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fe094034440 con 0x557cab677350 2024-03-15T13:05:49.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe080ff9640 10 client.17721 handle_command_reply: tid=0 2024-03-15T13:05:49.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe0a5fc4740 1 client.17721 shutdown 2024-03-15T13:05:49.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/3216742207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab601250 msgr2=0x557cab612130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/3216742207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab601250 0x557cab612130 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe094000c00 tx=0x7fe094005fd0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/3216742207 shutdown_connections 2024-03-15T13:05:49.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/3216742207 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x557cab677350 0x557cab679770 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/3216742207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cab60a750 0x557cab60e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/3216742207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cab601b70 0x557cab60e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe080ff9640 0 client.17721 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:05:49.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe0a5fc4740 1 --2- 172.21.15.2:0/3216742207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557cab601250 0x557cab612130 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/3216742207 >> 172.21.15.2:0/3216742207 conn(0x557cab5fb980 msgr2=0x557cab5fab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:49.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/3216742207 shutdown_connections 2024-03-15T13:05:49.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe0a5fc4740 1 -- 172.21.15.2:0/3216742207 wait complete. 2024-03-15T13:05:49.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.111+0000 7fe0a5fc4740 20 client.17721 trim_cache size 0 max 16384 2024-03-15T13:05:49.103 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:05:49.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.112+0000 7fe0a330d640 1 -- 172.21.15.2:0/4100528028 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe0840a1ba0 msgr2=0x7fe0840a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.112+0000 7fe0a330d640 1 --2- 172.21.15.2:0/4100528028 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe0840a1ba0 0x7fe0840a4060 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009fd0 tx=0x7fe08c004a40 comp rx=0 tx=0).stop 2024-03-15T13:05:49.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.112+0000 7fe0a330d640 1 -- 172.21.15.2:0/4100528028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe09c101020 msgr2=0x7fe09c1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.112+0000 7fe0a330d640 1 --2- 172.21.15.2:0/4100528028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe09c101020 0x7fe09c1025d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe09000e9c0 tx=0x7fe09000ee80 comp rx=0 tx=0).stop 2024-03-15T13:05:49.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.112+0000 7fe0a330d640 1 -- 172.21.15.2:0/4100528028 shutdown_connections 2024-03-15T13:05:49.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.112+0000 7fe0a330d640 1 --2- 172.21.15.2:0/4100528028 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe0840a1ba0 0x7fe0840a4060 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.112+0000 7fe0a330d640 1 --2- 172.21.15.2:0/4100528028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe09c106530 0x7fe09c102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.112+0000 7fe0a330d640 1 --2- 172.21.15.2:0/4100528028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe09c101020 0x7fe09c1025d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.112+0000 7fe0a330d640 1 --2- 172.21.15.2:0/4100528028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe09c100660 0x7fe09c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.112+0000 7fe0a330d640 1 -- 172.21.15.2:0/4100528028 >> 172.21.15.2:0/4100528028 conn(0x7fe09c0772e0 msgr2=0x7fe09c069c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:49.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.112+0000 7fe0a330d640 1 -- 172.21.15.2:0/4100528028 shutdown_connections 2024-03-15T13:05:49.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.112+0000 7fe0a330d640 1 -- 172.21.15.2:0/4100528028 wait complete. 2024-03-15T13:05:49.107 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:05:49.107 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:05:49.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:48 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3859965373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:05:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.229+0000 7ff146d9a640 1 -- 172.21.15.2:0/3850668792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1401005d0 msgr2=0x7ff1401009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.229+0000 7ff146d9a640 1 --2- 172.21.15.2:0/3850668792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1401005d0 0x7ff1401009b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff134009900 tx=0x7ff13402bc30 comp rx=0 tx=0).stop 2024-03-15T13:05:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.229+0000 7ff146d9a640 1 -- 172.21.15.2:0/3850668792 shutdown_connections 2024-03-15T13:05:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.229+0000 7ff146d9a640 1 --2- 172.21.15.2:0/3850668792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1401056a0 0x7ff14010d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.229+0000 7ff146d9a640 1 --2- 172.21.15.2:0/3850668792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff140100f80 0x7ff140104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.229+0000 7ff146d9a640 1 --2- 172.21.15.2:0/3850668792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1401005d0 0x7ff1401009b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.229+0000 7ff146d9a640 1 -- 172.21.15.2:0/3850668792 >> 172.21.15.2:0/3850668792 conn(0x7ff1400774f0 msgr2=0x7ff140077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.229+0000 7ff146d9a640 1 -- 172.21.15.2:0/3850668792 shutdown_connections 2024-03-15T13:05:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.229+0000 7ff146d9a640 1 -- 172.21.15.2:0/3850668792 wait complete. 2024-03-15T13:05:49.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.229+0000 7ff146d9a640 1 Processor -- start 2024-03-15T13:05:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.229+0000 7ff146d9a640 1 -- start start 2024-03-15T13:05:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff146d9a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1401005d0 0x7ff140199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff146d9a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1401056a0 0x7ff14019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff146d9a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff14019e5f0 0x7ff14019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff146d9a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14019efe0 con 0x7ff14019e5f0 2024-03-15T13:05:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff146d9a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14019f150 con 0x7ff1401056a0 2024-03-15T13:05:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff146d9a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14019f2c0 con 0x7ff1401005d0 2024-03-15T13:05:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff145337640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff14019e5f0 0x7ff14019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff144b36640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1401005d0 0x7ff140199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff145337640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff14019e5f0 0x7ff14019eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53332/0 (socket says 172.21.15.2:53332) 2024-03-15T13:05:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff144b36640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1401005d0 0x7ff140199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:33294/0 (socket says 172.21.15.2:33294) 2024-03-15T13:05:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff145337640 1 -- 172.21.15.2:0/2482135111 learned_addr learned my addr 172.21.15.2:0/2482135111 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff13ffff640 1 --2- 172.21.15.2:0/2482135111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1401056a0 0x7ff14019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff145337640 1 -- 172.21.15.2:0/2482135111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1401005d0 msgr2=0x7ff140199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff145337640 1 --2- 172.21.15.2:0/2482135111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1401005d0 0x7ff140199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff145337640 1 -- 172.21.15.2:0/2482135111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1401056a0 msgr2=0x7ff14019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff145337640 1 --2- 172.21.15.2:0/2482135111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1401056a0 0x7ff14019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff145337640 1 -- 172.21.15.2:0/2482135111 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff134009560 con 0x7ff14019e5f0 2024-03-15T13:05:49.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff145337640 1 --2- 172.21.15.2:0/2482135111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff14019e5f0 0x7ff14019eaa0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff13800eb10 tx=0x7ff13800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:49.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff13dffb640 1 -- 172.21.15.2:0/2482135111 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138004b20 con 0x7ff14019e5f0 2024-03-15T13:05:49.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.230+0000 7ff146d9a640 1 -- 172.21.15.2:0/2482135111 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1401a68a0 con 0x7ff14019e5f0 2024-03-15T13:05:49.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.231+0000 7ff13dffb640 1 -- 172.21.15.2:0/2482135111 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff138004c80 con 0x7ff14019e5f0 2024-03-15T13:05:49.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.231+0000 7ff13dffb640 1 -- 172.21.15.2:0/2482135111 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138018630 con 0x7ff14019e5f0 2024-03-15T13:05:49.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.231+0000 7ff146d9a640 1 -- 172.21.15.2:0/2482135111 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1401a6d50 con 0x7ff14019e5f0 2024-03-15T13:05:49.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.233+0000 7ff13dffb640 1 -- 172.21.15.2:0/2482135111 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7ff138018850 con 0x7ff14019e5f0 2024-03-15T13:05:49.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.233+0000 7ff149978740 1 Processor -- start 2024-03-15T13:05:49.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.233+0000 7ff149978740 1 -- start start 2024-03-15T13:05:49.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff13dffb640 1 --2- 172.21.15.2:0/2482135111 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff1300a1ba0 0x7ff1300a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff149978740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 0x5638d89f7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff149978740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d8a00210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff149978740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 0x5638d8a02b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff149978740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d89b2960 con 0x5638d89f7b70 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff149978740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d8966310 con 0x5638d8a00750 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff149978740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d8963840 con 0x5638d89f7250 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff13ffff640 1 --2- 172.21.15.2:0/2482135111 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff1300a1ba0 0x7ff1300a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff144b36640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 0x5638d89f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff145337640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 0x5638d8a02b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff13dffb640 1 -- 172.21.15.2:0/2482135111 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7ff138014070 con 0x7ff14019e5f0 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff144b36640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 0x5638d89f7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:33308/0 (socket says 172.21.15.2:33308) 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff144b36640 1 -- 172.21.15.2:0/1114553186 learned_addr learned my addr 172.21.15.2:0/1114553186 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff13ffff640 1 --2- 172.21.15.2:0/1114553186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d8a00210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff144b36640 1 -- 172.21.15.2:0/1114553186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 msgr2=0x5638d8a02b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff144b36640 1 --2- 172.21.15.2:0/1114553186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 0x5638d8a02b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff144b36640 1 -- 172.21.15.2:0/1114553186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 msgr2=0x5638d8a00210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff144b36640 1 --2- 172.21.15.2:0/1114553186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d8a00210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff144b36640 1 -- 172.21.15.2:0/1114553186 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638d8921060 con 0x5638d89f7250 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff145337640 1 --2- 172.21.15.2:0/1114553186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 0x5638d8a02b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:49.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff13ffff640 1 --2- 172.21.15.2:0/2482135111 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff1300a1ba0 0x7ff1300a4060 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff1400fdf10 tx=0x7ff12c009b60 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:49.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.234+0000 7ff144b36640 1 --2- 172.21.15.2:0/1114553186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 0x5638d89f7630 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff134000c00 tx=0x7ff13402c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eb48109fe37351a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:49.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.235+0000 7ff12affd640 1 -- 172.21.15.2:0/1114553186 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff134004780 con 0x5638d89f7250 2024-03-15T13:05:49.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.235+0000 7ff12affd640 1 -- 172.21.15.2:0/1114553186 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff1340048e0 con 0x5638d89f7250 2024-03-15T13:05:49.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.235+0000 7ff12affd640 1 -- 172.21.15.2:0/1114553186 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff134005040 con 0x5638d89f7250 2024-03-15T13:05:49.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.235+0000 7ff149978740 1 -- 172.21.15.2:0/1114553186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 msgr2=0x5638d89f7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.235+0000 7ff149978740 1 --2- 172.21.15.2:0/1114553186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 0x5638d89f7630 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff134000c00 tx=0x7ff13402c670 comp rx=0 tx=0).stop 2024-03-15T13:05:49.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.235+0000 7ff149978740 1 -- 172.21.15.2:0/1114553186 shutdown_connections 2024-03-15T13:05:49.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.235+0000 7ff149978740 1 --2- 172.21.15.2:0/1114553186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 0x5638d8a02b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.235+0000 7ff149978740 1 --2- 172.21.15.2:0/1114553186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d8a00210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.235+0000 7ff149978740 1 --2- 172.21.15.2:0/1114553186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 0x5638d89f7630 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.235+0000 7ff149978740 1 -- 172.21.15.2:0/1114553186 >> 172.21.15.2:0/1114553186 conn(0x5638d89f1980 msgr2=0x5638d89f1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:49.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.235+0000 7ff149978740 1 -- 172.21.15.2:0/1114553186 shutdown_connections 2024-03-15T13:05:49.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.235+0000 7ff149978740 1 -- 172.21.15.2:0/1114553186 wait complete. 2024-03-15T13:05:49.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.236+0000 7ff149978740 1 -- start start 2024-03-15T13:05:49.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.236+0000 7ff149978740 1 Processor -- start 2024-03-15T13:05:49.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.236+0000 7ff149978740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 0x5638d8a00130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.236+0000 7ff149978740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d89fd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.236+0000 7ff149978740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 0x5638d89fdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.236+0000 7ff149978740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d895f020 con 0x5638d89f7b70 2024-03-15T13:05:49.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.236+0000 7ff149978740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d8632430 con 0x5638d8a00750 2024-03-15T13:05:49.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.236+0000 7ff149978740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d85eee00 con 0x5638d89f7250 2024-03-15T13:05:49.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.236+0000 7ff149978740 10 client.0 _ll_register_callbacks cb 0x5638d8906cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:49.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.236+0000 7ff13ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d89fd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.236+0000 7ff144b36640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 0x5638d8a00130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.236+0000 7ff13ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d89fd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53362/0 (socket says 172.21.15.2:53362) 2024-03-15T13:05:49.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.236+0000 7ff13ffff640 1 -- 172.21.15.2:0/1931387220 learned_addr learned my addr 172.21.15.2:0/1931387220 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:49.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.236+0000 7ff145337640 1 --2- 172.21.15.2:0/1931387220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 0x5638d89fdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.237+0000 7ff13ffff640 1 -- 172.21.15.2:0/1931387220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 msgr2=0x5638d8a00130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.237+0000 7ff13ffff640 1 --2- 172.21.15.2:0/1931387220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 0x5638d8a00130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.237+0000 7ff13ffff640 1 -- 172.21.15.2:0/1931387220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 msgr2=0x5638d89fdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.237+0000 7ff13ffff640 1 --2- 172.21.15.2:0/1931387220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 0x5638d89fdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.237+0000 7ff13ffff640 1 -- 172.21.15.2:0/1931387220 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14010da80 con 0x5638d89f7b70 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.237+0000 7ff144b36640 1 --2- 172.21.15.2:0/1931387220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 0x5638d8a00130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.237+0000 7ff13ffff640 1 --2- 172.21.15.2:0/1931387220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d89fd4c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff12c011f10 tx=0x7ff12c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c495dd1f837c63be server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.237+0000 7ff11ffff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.237+0000 7ff11ffff640 1 -- 172.21.15.2:0/1931387220 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c016040 con 0x5638d89f7b70 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.237+0000 7ff11ffff640 1 -- 172.21.15.2:0/1931387220 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7ff12c005020 con 0x5638d89f7b70 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.237+0000 7ff149978740 1 -- 172.21.15.2:0/1931387220 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638d865e7e0 con 0x5638d89f7b70 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.237+0000 7ff11ffff640 1 -- 172.21.15.2:0/1931387220 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff12c00fea0 con 0x5638d89f7b70 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.237+0000 7ff149978740 10 client.17733 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.237+0000 7ff149978740 1 -- 172.21.15.2:0/1931387220 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5638d895d6a0 con 0x5638d89f7b70 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.237+0000 7ff11ffff640 1 -- 172.21.15.2:0/1931387220 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5638d865e7e0 con 0x5638d89f7b70 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.238+0000 7ff11ffff640 1 -- 172.21.15.2:0/1931387220 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff12c026070 con 0x5638d89f7b70 2024-03-15T13:05:49.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.238+0000 7ff149978740 10 client.17733 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:49.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.238+0000 7ff149978740 10 client.17733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:49.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.238+0000 7ff149978740 20 client.17733 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:49.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.238+0000 7ff149978740 1 --2- 172.21.15.2:0/1931387220 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5638d8a10930 0x5638d8a12d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.238+0000 7ff149978740 4 client.17733 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:49.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.238+0000 7ff149978740 1 -- 172.21.15.2:0/1931387220 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638d869b720 con 0x5638d8a10930 2024-03-15T13:05:49.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.238+0000 7ff144b36640 1 --2- 172.21.15.2:0/1931387220 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5638d8a10930 0x5638d8a12d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.238+0000 7ff11ffff640 10 client.17733 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:49.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.238+0000 7ff144b36640 1 --2- 172.21.15.2:0/1931387220 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5638d8a10930 0x5638d8a12d50 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2f2bbfef061a00d server_cookie=41a7aa33469095af in_seq=0 out_seq=0 2024-03-15T13:05:49.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.239+0000 7ff11ffff640 1 -- 172.21.15.2:0/1931387220 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5638d869b720 con 0x5638d8a10930 2024-03-15T13:05:49.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.239+0000 7ff11ffff640 10 client.17733 handle_command_reply: tid=0 2024-03-15T13:05:49.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.239+0000 7ff149978740 1 client.17733 shutdown 2024-03-15T13:05:49.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.239+0000 7ff149978740 1 -- 172.21.15.2:0/1931387220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 msgr2=0x5638d89fd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.239+0000 7ff149978740 1 --2- 172.21.15.2:0/1931387220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d89fd4c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff12c011f10 tx=0x7ff12c01d000 comp rx=0 tx=0).stop 2024-03-15T13:05:49.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.239+0000 7ff149978740 1 -- 172.21.15.2:0/1931387220 shutdown_connections 2024-03-15T13:05:49.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.239+0000 7ff149978740 1 --2- 172.21.15.2:0/1931387220 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5638d8a10930 0x5638d8a12d50 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.239+0000 7ff149978740 1 --2- 172.21.15.2:0/1931387220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 0x5638d89fdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.239+0000 7ff149978740 1 --2- 172.21.15.2:0/1931387220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d89fd4c0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.239+0000 7ff11ffff640 0 client.17733 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:49.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.239+0000 7ff149978740 1 --2- 172.21.15.2:0/1931387220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 0x5638d8a00130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.239+0000 7ff149978740 1 -- 172.21.15.2:0/1931387220 >> 172.21.15.2:0/1931387220 conn(0x5638d89f1980 msgr2=0x5638d89f36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:49.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.240+0000 7ff149978740 1 -- 172.21.15.2:0/1931387220 shutdown_connections 2024-03-15T13:05:49.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.240+0000 7ff149978740 1 -- 172.21.15.2:0/1931387220 wait complete. 2024-03-15T13:05:49.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.240+0000 7ff149978740 20 client.17733 trim_cache size 0 max 16384 2024-03-15T13:05:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.252+0000 7ff149978740 1 Processor -- start 2024-03-15T13:05:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.252+0000 7ff149978740 1 -- start start 2024-03-15T13:05:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.252+0000 7ff149978740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 0x5638d89ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.252+0000 7ff149978740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d8a00130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.252+0000 7ff149978740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 0x5638d8a0ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.252+0000 7ff149978740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d8a0cd50 con 0x5638d89f7b70 2024-03-15T13:05:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.252+0000 7ff149978740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d88e4130 con 0x5638d8a00750 2024-03-15T13:05:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.252+0000 7ff149978740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d89e3a40 con 0x5638d89f7250 2024-03-15T13:05:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.252+0000 7ff13ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d8a00130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.252+0000 7ff13ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d8a00130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53364/0 (socket says 172.21.15.2:53364) 2024-03-15T13:05:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.252+0000 7ff13ffff640 1 -- 172.21.15.2:0/681589181 learned_addr learned my addr 172.21.15.2:0/681589181 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.252+0000 7ff144b36640 1 --2- 172.21.15.2:0/681589181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 0x5638d89ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.252+0000 7ff145337640 1 --2- 172.21.15.2:0/681589181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 0x5638d8a0ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.253+0000 7ff13ffff640 1 -- 172.21.15.2:0/681589181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 msgr2=0x5638d89ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.253+0000 7ff13ffff640 1 --2- 172.21.15.2:0/681589181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 0x5638d89ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.253+0000 7ff13ffff640 1 -- 172.21.15.2:0/681589181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 msgr2=0x5638d8a0ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.253+0000 7ff13ffff640 1 --2- 172.21.15.2:0/681589181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 0x5638d8a0ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.253+0000 7ff13ffff640 1 -- 172.21.15.2:0/681589181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638d88f9c20 con 0x5638d89f7b70 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.253+0000 7ff13ffff640 1 --2- 172.21.15.2:0/681589181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d8a00130 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00f800 tx=0x7ff12c011400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4c28e8bcf52722 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.253+0000 7ff12a7fc640 1 -- 172.21.15.2:0/681589181 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c016070 con 0x5638d89f7b70 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.253+0000 7ff12a7fc640 1 -- 172.21.15.2:0/681589181 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff12c00fea0 con 0x5638d89f7b70 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.253+0000 7ff12a7fc640 1 -- 172.21.15.2:0/681589181 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c004f50 con 0x5638d89f7b70 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.253+0000 7ff149978740 1 -- 172.21.15.2:0/681589181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 msgr2=0x5638d8a00130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.253+0000 7ff149978740 1 --2- 172.21.15.2:0/681589181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d8a00130 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00f800 tx=0x7ff12c011400 comp rx=0 tx=0).stop 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 1 -- 172.21.15.2:0/681589181 shutdown_connections 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 1 --2- 172.21.15.2:0/681589181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d8a00750 0x5638d8a0ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 1 --2- 172.21.15.2:0/681589181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d8a00130 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 1 --2- 172.21.15.2:0/681589181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d89f7250 0x5638d89ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 1 -- 172.21.15.2:0/681589181 >> 172.21.15.2:0/681589181 conn(0x5638d89f1980 msgr2=0x5638d89f0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 1 -- 172.21.15.2:0/681589181 shutdown_connections 2024-03-15T13:05:49.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 1 -- 172.21.15.2:0/681589181 wait complete. 2024-03-15T13:05:49.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 1 -- start start 2024-03-15T13:05:49.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 1 Processor -- start 2024-03-15T13:05:49.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d89f7250 0x5638d89fa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d89fa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d8a00750 0x5638d8a6b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d89fade0 con 0x5638d89f7b70 2024-03-15T13:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d8a6b6e0 con 0x5638d89f7250 2024-03-15T13:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d8a6b850 con 0x5638d8a00750 2024-03-15T13:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff149978740 10 client.0 _ll_register_callbacks cb 0x7ff12c011350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff13ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d89fa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff144b36640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d89f7250 0x5638d89fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff13ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d89fa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53374/0 (socket says 172.21.15.2:53374) 2024-03-15T13:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff13ffff640 1 -- 172.21.15.2:0/183435687 learned_addr learned my addr 172.21.15.2:0/183435687 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.254+0000 7ff145337640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d8a00750 0x5638d8a6b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.255+0000 7ff13ffff640 1 -- 172.21.15.2:0/183435687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d8a00750 msgr2=0x5638d8a6b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.255+0000 7ff13ffff640 1 --2- 172.21.15.2:0/183435687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d8a00750 0x5638d8a6b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.255+0000 7ff13ffff640 1 -- 172.21.15.2:0/183435687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d89f7250 msgr2=0x5638d89fa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:49.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.255+0000 7ff13ffff640 1 --2- 172.21.15.2:0/183435687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d89f7250 0x5638d89fa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.255+0000 7ff13ffff640 1 -- 172.21.15.2:0/183435687 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638d895d6a0 con 0x5638d89f7b70 2024-03-15T13:05:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.255+0000 7ff13ffff640 1 --2- 172.21.15.2:0/183435687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d89fa690 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff12c011f10 tx=0x7ff12c0048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ad52c34b766b79 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.255+0000 7ff11ffff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.255+0000 7ff11ffff640 1 -- 172.21.15.2:0/183435687 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c016040 con 0x5638d89f7b70 2024-03-15T13:05:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.255+0000 7ff11ffff640 1 -- 172.21.15.2:0/183435687 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7ff12c01f670 con 0x5638d89f7b70 2024-03-15T13:05:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.255+0000 7ff149978740 1 -- 172.21.15.2:0/183435687 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1340048e0 con 0x5638d89f7b70 2024-03-15T13:05:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.255+0000 7ff11ffff640 1 -- 172.21.15.2:0/183435687 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff12c0097a0 con 0x5638d89f7b70 2024-03-15T13:05:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.255+0000 7ff149978740 10 client.17739 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.255+0000 7ff149978740 1 -- 172.21.15.2:0/183435687 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5638d891f8c0 con 0x5638d89f7b70 2024-03-15T13:05:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.255+0000 7ff11ffff640 1 -- 172.21.15.2:0/183435687 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff1340048e0 con 0x5638d89f7b70 2024-03-15T13:05:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff11ffff640 1 -- 172.21.15.2:0/183435687 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff12c026050 con 0x5638d89f7b70 2024-03-15T13:05:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff149978740 10 client.17739 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff149978740 10 client.17739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff149978740 20 client.17739 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:49.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff149978740 1 --2- 172.21.15.2:0/183435687 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5638d8a6cac0 0x5638d8a6eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff149978740 4 client.17739 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:05:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff149978740 1 -- 172.21.15.2:0/183435687 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5638d86ea890 con 0x5638d8a6cac0 2024-03-15T13:05:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff144b36640 1 --2- 172.21.15.2:0/183435687 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5638d8a6cac0 0x5638d8a6eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff144b36640 1 --2- 172.21.15.2:0/183435687 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5638d8a6cac0 0x5638d8a6eee0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c65a31807d6a6c98 server_cookie=caf9d8ec863d1a8e in_seq=0 out_seq=0 2024-03-15T13:05:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff11ffff640 10 client.17739 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff11ffff640 1 -- 172.21.15.2:0/183435687 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5638d86ea890 con 0x5638d8a6cac0 2024-03-15T13:05:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff11ffff640 10 client.17739 handle_command_reply: tid=0 2024-03-15T13:05:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff149978740 1 client.17739 shutdown 2024-03-15T13:05:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff149978740 1 -- 172.21.15.2:0/183435687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 msgr2=0x5638d89fa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff149978740 1 --2- 172.21.15.2:0/183435687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d89fa690 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff12c011f10 tx=0x7ff12c0048d0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff149978740 1 -- 172.21.15.2:0/183435687 shutdown_connections 2024-03-15T13:05:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff149978740 1 --2- 172.21.15.2:0/183435687 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5638d8a6cac0 0x5638d8a6eee0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff149978740 1 --2- 172.21.15.2:0/183435687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5638d8a00750 0x5638d8a6b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff149978740 1 --2- 172.21.15.2:0/183435687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5638d89f7b70 0x5638d89fa690 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff149978740 1 --2- 172.21.15.2:0/183435687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5638d89f7250 0x5638d89fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff11ffff640 0 client.17739 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:49.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.256+0000 7ff149978740 1 -- 172.21.15.2:0/183435687 >> 172.21.15.2:0/183435687 conn(0x5638d89f1980 msgr2=0x5638d89f02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:49.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.257+0000 7ff149978740 1 -- 172.21.15.2:0/183435687 shutdown_connections 2024-03-15T13:05:49.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.257+0000 7ff149978740 1 -- 172.21.15.2:0/183435687 wait complete. 2024-03-15T13:05:49.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.257+0000 7ff149978740 20 client.17739 trim_cache size 0 max 16384 2024-03-15T13:05:49.247 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:05:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.257+0000 7ff146d9a640 1 -- 172.21.15.2:0/2482135111 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff1300a1ba0 msgr2=0x7ff1300a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.257+0000 7ff146d9a640 1 --2- 172.21.15.2:0/2482135111 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff1300a1ba0 0x7ff1300a4060 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff1400fdf10 tx=0x7ff12c009b60 comp rx=0 tx=0).stop 2024-03-15T13:05:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.257+0000 7ff146d9a640 1 -- 172.21.15.2:0/2482135111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff14019e5f0 msgr2=0x7ff14019eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.257+0000 7ff146d9a640 1 --2- 172.21.15.2:0/2482135111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff14019e5f0 0x7ff14019eaa0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff13800eb10 tx=0x7ff13800c2d0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.257+0000 7ff146d9a640 1 -- 172.21.15.2:0/2482135111 shutdown_connections 2024-03-15T13:05:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.257+0000 7ff146d9a640 1 --2- 172.21.15.2:0/2482135111 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff1300a1ba0 0x7ff1300a4060 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.257+0000 7ff146d9a640 1 --2- 172.21.15.2:0/2482135111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff14019e5f0 0x7ff14019eaa0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.257+0000 7ff146d9a640 1 --2- 172.21.15.2:0/2482135111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1401056a0 0x7ff14019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.257+0000 7ff146d9a640 1 --2- 172.21.15.2:0/2482135111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1401005d0 0x7ff140199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:49.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.257+0000 7ff146d9a640 1 -- 172.21.15.2:0/2482135111 >> 172.21.15.2:0/2482135111 conn(0x7ff1400774f0 msgr2=0x7ff1400feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.257+0000 7ff146d9a640 1 -- 172.21.15.2:0/2482135111 shutdown_connections 2024-03-15T13:05:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:49.257+0000 7ff146d9a640 1 -- 172.21.15.2:0/2482135111 wait complete. 2024-03-15T13:05:50.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:49 smithi002 ceph-mon[32060]: pgmap v210: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:50.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:49 smithi184 ceph-mon[35993]: pgmap v210: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:50.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:49 smithi149 ceph-mon[35927]: pgmap v210: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:50.253 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:05:50.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:05:50.254 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:05:50.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.388+0000 7f67c0a39640 1 -- 172.21.15.2:0/309039860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67bc10bb30 msgr2=0x7f67bc10dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.388+0000 7f67c0a39640 1 --2- 172.21.15.2:0/309039860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67bc10bb30 0x7f67bc10dfa0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009ca0 tx=0x7f67b402c180 comp rx=0 tx=0).stop 2024-03-15T13:05:50.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.388+0000 7f67c0a39640 1 -- 172.21.15.2:0/309039860 shutdown_connections 2024-03-15T13:05:50.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.388+0000 7f67c0a39640 1 --2- 172.21.15.2:0/309039860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67bc10bb30 0x7f67bc10dfa0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.388+0000 7f67c0a39640 1 --2- 172.21.15.2:0/309039860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67bc103080 0x7f67bc10b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.388+0000 7f67c0a39640 1 --2- 172.21.15.2:0/309039860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67bc102760 0x7f67bc102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.388+0000 7f67c0a39640 1 -- 172.21.15.2:0/309039860 >> 172.21.15.2:0/309039860 conn(0x7f67bc077550 msgr2=0x7f67bc077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:50.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.388+0000 7f67c0a39640 1 -- 172.21.15.2:0/309039860 shutdown_connections 2024-03-15T13:05:50.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.388+0000 7f67c0a39640 1 -- 172.21.15.2:0/309039860 wait complete. 2024-03-15T13:05:50.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.389+0000 7f67c0a39640 1 Processor -- start 2024-03-15T13:05:50.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.389+0000 7f67c0a39640 1 -- start start 2024-03-15T13:05:50.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.389+0000 7f67c0a39640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67bc102760 0x7f67bc1003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.389+0000 7f67c0a39640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67bc103080 0x7f67bc1008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.389+0000 7f67c0a39640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67bc10bb30 0x7f67bc100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.389+0000 7f67c0a39640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc1a5f20 con 0x7f67bc103080 2024-03-15T13:05:50.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.389+0000 7f67c0a39640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc1a6090 con 0x7f67bc102760 2024-03-15T13:05:50.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.389+0000 7f67c0a39640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc1a6250 con 0x7f67bc10bb30 2024-03-15T13:05:50.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.389+0000 7f67b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67bc103080 0x7f67bc1008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.389+0000 7f67ba59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67bc102760 0x7f67bc1003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.389+0000 7f67bad9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67bc10bb30 0x7f67bc100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.389+0000 7f67b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67bc103080 0x7f67bc1008f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53394/0 (socket says 172.21.15.2:53394) 2024-03-15T13:05:50.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.389+0000 7f67b3fff640 1 -- 172.21.15.2:0/3167962117 learned_addr learned my addr 172.21.15.2:0/3167962117 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:50.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.390+0000 7f67ba59c640 1 -- 172.21.15.2:0/3167962117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67bc10bb30 msgr2=0x7f67bc100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.390+0000 7f67ba59c640 1 --2- 172.21.15.2:0/3167962117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67bc10bb30 0x7f67bc100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.390+0000 7f67ba59c640 1 -- 172.21.15.2:0/3167962117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67bc103080 msgr2=0x7f67bc1008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.390+0000 7f67ba59c640 1 --2- 172.21.15.2:0/3167962117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67bc103080 0x7f67bc1008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.390+0000 7f67ba59c640 1 -- 172.21.15.2:0/3167962117 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b4009900 con 0x7f67bc102760 2024-03-15T13:05:50.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.390+0000 7f67b3fff640 1 --2- 172.21.15.2:0/3167962117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67bc103080 0x7f67bc1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:50.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.390+0000 7f67ba59c640 1 --2- 172.21.15.2:0/3167962117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67bc102760 0x7f67bc1003b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00e9f0 tx=0x7f67ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:50.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.390+0000 7f67b37fe640 1 -- 172.21.15.2:0/3167962117 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00ccc0 con 0x7f67bc102760 2024-03-15T13:05:50.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.390+0000 7f67c0a39640 1 -- 172.21.15.2:0/3167962117 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67bc1a6530 con 0x7f67bc102760 2024-03-15T13:05:50.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.390+0000 7f67b37fe640 1 -- 172.21.15.2:0/3167962117 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f67ac00ce20 con 0x7f67bc102760 2024-03-15T13:05:50.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.390+0000 7f67b37fe640 1 -- 172.21.15.2:0/3167962117 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac0049a0 con 0x7f67bc102760 2024-03-15T13:05:50.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.390+0000 7f67c0a39640 1 -- 172.21.15.2:0/3167962117 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67bc1a6a40 con 0x7f67bc102760 2024-03-15T13:05:50.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.392+0000 7f67b37fe640 1 -- 172.21.15.2:0/3167962117 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f67ac004b00 con 0x7f67bc102760 2024-03-15T13:05:50.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.393+0000 7f67c3618740 1 Processor -- start 2024-03-15T13:05:50.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.393+0000 7f67c3618740 1 -- start start 2024-03-15T13:05:50.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.393+0000 7f67c3618740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 0x55ed40a84760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.393+0000 7f67b37fe640 1 --2- 172.21.15.2:0/3167962117 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f67a80a1ba0 0x7f67a80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.393+0000 7f67c3618740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a84d30 0x55ed40a8d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.393+0000 7f67c3618740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a8d8b0 0x55ed40a8fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.393+0000 7f67c3618740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed409f2f90 con 0x55ed40a84d30 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.393+0000 7f67c3618740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed409f04c0 con 0x55ed40a8d8b0 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.393+0000 7f67c3618740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed409ebca0 con 0x55ed40a84380 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.393+0000 7f67b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a84d30 0x55ed40a8d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.393+0000 7f67ba59c640 1 --2- 172.21.15.2:0/3167962117 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f67a80a1ba0 0x7f67a80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.393+0000 7f67b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a84d30 0x55ed40a8d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53408/0 (socket says 172.21.15.2:53408) 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.394+0000 7f67b3fff640 1 -- 172.21.15.2:0/2023075181 learned_addr learned my addr 172.21.15.2:0/2023075181 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.394+0000 7f67bad9d640 1 --2- 172.21.15.2:0/2023075181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 0x55ed40a84760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.394+0000 7f67bad9d640 1 --2- 172.21.15.2:0/2023075181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a8d8b0 0x55ed40a8fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.394+0000 7f67b37fe640 1 -- 172.21.15.2:0/3167962117 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f67ac014070 con 0x7f67bc102760 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.394+0000 7f67b3fff640 1 -- 172.21.15.2:0/2023075181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 msgr2=0x55ed40a84760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.394+0000 7f67b3fff640 1 --2- 172.21.15.2:0/2023075181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 0x55ed40a84760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.394+0000 7f67b3fff640 1 -- 172.21.15.2:0/2023075181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a8d8b0 msgr2=0x55ed40a8fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.394+0000 7f67b3fff640 1 --2- 172.21.15.2:0/2023075181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a8d8b0 0x55ed40a8fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.394+0000 7f67b3fff640 1 -- 172.21.15.2:0/2023075181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed409ae0b0 con 0x55ed40a84d30 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.394+0000 7f67bad9d640 1 --2- 172.21.15.2:0/2023075181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 0x55ed40a84760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.394+0000 7f67ba59c640 1 --2- 172.21.15.2:0/3167962117 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f67a80a1ba0 0x7f67a80a4060 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f67ac09a040 tx=0x7f67ac00cf90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:50.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.394+0000 7f67b3fff640 1 --2- 172.21.15.2:0/2023075181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a84d30 0x55ed40a8d2e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f67a4011510 tx=0x7f67a40119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9519ca2ecbece312 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:50.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.394+0000 7f67b0ff9640 1 -- 172.21.15.2:0/2023075181 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a400f5c0 con 0x55ed40a84d30 2024-03-15T13:05:50.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.394+0000 7f67b0ff9640 1 -- 172.21.15.2:0/2023075181 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f67a400fbf0 con 0x55ed40a84d30 2024-03-15T13:05:50.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.394+0000 7f67b0ff9640 1 -- 172.21.15.2:0/2023075181 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4019cd0 con 0x55ed40a84d30 2024-03-15T13:05:50.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.395+0000 7f67c3618740 1 -- 172.21.15.2:0/2023075181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a84d30 msgr2=0x55ed40a8d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.395+0000 7f67c3618740 1 --2- 172.21.15.2:0/2023075181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a84d30 0x55ed40a8d2e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f67a4011510 tx=0x7f67a40119d0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.395+0000 7f67c3618740 1 -- 172.21.15.2:0/2023075181 shutdown_connections 2024-03-15T13:05:50.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.395+0000 7f67c3618740 1 --2- 172.21.15.2:0/2023075181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a8d8b0 0x55ed40a8fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.395+0000 7f67c3618740 1 --2- 172.21.15.2:0/2023075181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a84d30 0x55ed40a8d2e0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.395+0000 7f67c3618740 1 --2- 172.21.15.2:0/2023075181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 0x55ed40a84760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.395+0000 7f67c3618740 1 -- 172.21.15.2:0/2023075181 >> 172.21.15.2:0/2023075181 conn(0x55ed40a7e840 msgr2=0x55ed40a7ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:50.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.395+0000 7f67c3618740 1 -- 172.21.15.2:0/2023075181 shutdown_connections 2024-03-15T13:05:50.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.395+0000 7f67c3618740 1 -- 172.21.15.2:0/2023075181 wait complete. 2024-03-15T13:05:50.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.395+0000 7f67c3618740 1 -- start start 2024-03-15T13:05:50.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67c3618740 1 Processor -- start 2024-03-15T13:05:50.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67c3618740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a84380 0x55ed40a8a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67c3618740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84d30 0x55ed40a8add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67c3618740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a8d8b0 0x55ed40a8b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67c3618740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed40990f40 con 0x55ed40a8d8b0 2024-03-15T13:05:50.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67c3618740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed406bf4b0 con 0x55ed40a84380 2024-03-15T13:05:50.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67c3618740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed40971670 con 0x55ed40a84d30 2024-03-15T13:05:50.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67c3618740 10 client.0 _ll_register_callbacks cb 0x55ed409def70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:50.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67bad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a8d8b0 0x55ed40a8b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67bad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a8d8b0 0x55ed40a8b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53424/0 (socket says 172.21.15.2:53424) 2024-03-15T13:05:50.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67bad9d640 1 -- 172.21.15.2:0/3120443040 learned_addr learned my addr 172.21.15.2:0/3120443040 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:50.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67bad9d640 1 --2- 172.21.15.2:0/3120443040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a84380 0x55ed40a8a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67b3fff640 1 --2- 172.21.15.2:0/3120443040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84d30 0x55ed40a8add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67bad9d640 1 -- 172.21.15.2:0/3120443040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84d30 msgr2=0x55ed40a8add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67bad9d640 1 --2- 172.21.15.2:0/3120443040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84d30 0x55ed40a8add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67bad9d640 1 -- 172.21.15.2:0/3120443040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a84380 msgr2=0x55ed40a8a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67bad9d640 1 --2- 172.21.15.2:0/3120443040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a84380 0x55ed40a8a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67bad9d640 1 -- 172.21.15.2:0/3120443040 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a4009d10 con 0x55ed40a8d8b0 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67bad9d640 1 --2- 172.21.15.2:0/3120443040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a8d8b0 0x55ed40a8b310 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f67b4006010 tx=0x7f67b40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f90aadbe773c43 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.396+0000 7f67a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.397+0000 7f67a1ffb640 1 -- 172.21.15.2:0/3120443040 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4048030 con 0x55ed40a8d8b0 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.397+0000 7f67a1ffb640 1 -- 172.21.15.2:0/3120443040 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f67b4032b70 con 0x55ed40a8d8b0 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.397+0000 7f67c3618740 1 -- 172.21.15.2:0/3120443040 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed409c5f40 con 0x55ed40a8d8b0 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.397+0000 7f67a1ffb640 1 -- 172.21.15.2:0/3120443040 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67b4032d30 con 0x55ed40a8d8b0 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.397+0000 7f67c3618740 10 client.17751 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.397+0000 7f67c3618740 1 -- 172.21.15.2:0/3120443040 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55ed409ea320 con 0x55ed40a8d8b0 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.397+0000 7f67a1ffb640 1 -- 172.21.15.2:0/3120443040 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55ed409c5f40 con 0x55ed40a8d8b0 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.397+0000 7f67a1ffb640 1 -- 172.21.15.2:0/3120443040 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f67b4042420 con 0x55ed40a8d8b0 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.397+0000 7f67c3618740 10 client.17751 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.397+0000 7f67c3618740 10 client.17751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.397+0000 7f67c3618740 20 client.17751 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.397+0000 7f67c3618740 1 --2- 172.21.15.2:0/3120443040 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ed40a9de10 0x55ed40aa0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.397+0000 7f67c3618740 4 client.17751 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:50.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.397+0000 7f67c3618740 1 -- 172.21.15.2:0/3120443040 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed409e2310 con 0x55ed40a9de10 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.397+0000 7f67b3fff640 1 --2- 172.21.15.2:0/3120443040 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ed40a9de10 0x55ed40aa0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.398+0000 7f67a1ffb640 10 client.17751 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.398+0000 7f67b3fff640 1 --2- 172.21.15.2:0/3120443040 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ed40a9de10 0x55ed40aa0230 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1317043c4bd28439 server_cookie=86907352d0c7e99 in_seq=0 out_seq=0 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.398+0000 7f67a1ffb640 1 -- 172.21.15.2:0/3120443040 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ed409e2310 con 0x55ed40a9de10 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.398+0000 7f67a1ffb640 10 client.17751 handle_command_reply: tid=0 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.398+0000 7f67c3618740 1 client.17751 shutdown 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.399+0000 7f67c3618740 1 -- 172.21.15.2:0/3120443040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a8d8b0 msgr2=0x55ed40a8b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.399+0000 7f67c3618740 1 --2- 172.21.15.2:0/3120443040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a8d8b0 0x55ed40a8b310 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f67b4006010 tx=0x7f67b40095c0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.399+0000 7f67c3618740 1 -- 172.21.15.2:0/3120443040 shutdown_connections 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.399+0000 7f67c3618740 1 --2- 172.21.15.2:0/3120443040 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ed40a9de10 0x55ed40aa0230 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.399+0000 7f67c3618740 1 --2- 172.21.15.2:0/3120443040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a8d8b0 0x55ed40a8b310 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.399+0000 7f67c3618740 1 --2- 172.21.15.2:0/3120443040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84d30 0x55ed40a8add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.399+0000 7f67a1ffb640 0 client.17751 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.399+0000 7f67c3618740 1 --2- 172.21.15.2:0/3120443040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a84380 0x55ed40a8a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.399+0000 7f67c3618740 1 -- 172.21.15.2:0/3120443040 >> 172.21.15.2:0/3120443040 conn(0x55ed40a7e840 msgr2=0x55ed40a807a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.399+0000 7f67c3618740 1 -- 172.21.15.2:0/3120443040 shutdown_connections 2024-03-15T13:05:50.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.399+0000 7f67c3618740 1 -- 172.21.15.2:0/3120443040 wait complete. 2024-03-15T13:05:50.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.399+0000 7f67c3618740 20 client.17751 trim_cache size 0 max 16384 2024-03-15T13:05:50.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.415+0000 7f67c3618740 1 Processor -- start 2024-03-15T13:05:50.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.415+0000 7f67c3618740 1 -- start start 2024-03-15T13:05:50.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.415+0000 7f67c3618740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 0x55ed40a8cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.415+0000 7f67c3618740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a84d30 0x55ed40a98de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.415+0000 7f67c3618740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a8d8b0 0x55ed40a99320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.415+0000 7f67c3618740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed40a861e0 con 0x55ed40a84d30 2024-03-15T13:05:50.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.415+0000 7f67c3618740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed40a7e3d0 con 0x55ed40a8d8b0 2024-03-15T13:05:50.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.415+0000 7f67c3618740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed40aa0890 con 0x55ed40a84380 2024-03-15T13:05:50.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.415+0000 7f67b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a84d30 0x55ed40a98de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.415+0000 7f67bad9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 0x55ed40a8cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.415+0000 7f67bad9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a8d8b0 0x55ed40a99320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.415+0000 7f67b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a84d30 0x55ed40a98de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53436/0 (socket says 172.21.15.2:53436) 2024-03-15T13:05:50.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.415+0000 7f67b3fff640 1 -- 172.21.15.2:0/159763629 learned_addr learned my addr 172.21.15.2:0/159763629 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:50.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.416+0000 7f67b3fff640 1 -- 172.21.15.2:0/159763629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 msgr2=0x55ed40a8cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.416+0000 7f67b3fff640 1 --2- 172.21.15.2:0/159763629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 0x55ed40a8cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.416+0000 7f67b3fff640 1 -- 172.21.15.2:0/159763629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a8d8b0 msgr2=0x55ed40a99320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.416+0000 7f67b3fff640 1 --2- 172.21.15.2:0/159763629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a8d8b0 0x55ed40a99320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.416+0000 7f67b3fff640 1 -- 172.21.15.2:0/159763629 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed40986c00 con 0x55ed40a84d30 2024-03-15T13:05:50.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.416+0000 7f67bad9d640 1 --2- 172.21.15.2:0/159763629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a8d8b0 0x55ed40a99320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:50.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.416+0000 7f67b3fff640 1 --2- 172.21.15.2:0/159763629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a84d30 0x55ed40a98de0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f67a4011510 tx=0x7f67a4005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9582f9053d4c30ff server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:50.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.416+0000 7f67a3fff640 1 -- 172.21.15.2:0/159763629 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4019410 con 0x55ed40a84d30 2024-03-15T13:05:50.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.416+0000 7f67a3fff640 1 -- 172.21.15.2:0/159763629 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f67a4019570 con 0x55ed40a84d30 2024-03-15T13:05:50.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.416+0000 7f67a3fff640 1 -- 172.21.15.2:0/159763629 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4019b20 con 0x55ed40a84d30 2024-03-15T13:05:50.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 -- 172.21.15.2:0/159763629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a84d30 msgr2=0x55ed40a98de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 --2- 172.21.15.2:0/159763629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a84d30 0x55ed40a98de0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f67a4011510 tx=0x7f67a4005d30 comp rx=0 tx=0).stop 2024-03-15T13:05:50.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 -- 172.21.15.2:0/159763629 shutdown_connections 2024-03-15T13:05:50.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 --2- 172.21.15.2:0/159763629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a8d8b0 0x55ed40a99320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 --2- 172.21.15.2:0/159763629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a84d30 0x55ed40a98de0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 --2- 172.21.15.2:0/159763629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 0x55ed40a8cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 -- 172.21.15.2:0/159763629 >> 172.21.15.2:0/159763629 conn(0x55ed40a7e840 msgr2=0x55ed40a8b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:50.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 -- 172.21.15.2:0/159763629 shutdown_connections 2024-03-15T13:05:50.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 -- 172.21.15.2:0/159763629 wait complete. 2024-03-15T13:05:50.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 -- start start 2024-03-15T13:05:50.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 Processor -- start 2024-03-15T13:05:50.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 0x55ed40a955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a84d30 0x55ed40a91970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a8d8b0 0x55ed40a91eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed40a92580 con 0x55ed40a8d8b0 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed40a926f0 con 0x55ed40a84d30 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed40af9690 con 0x55ed40a84380 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67c3618740 10 client.0 _ll_register_callbacks cb 0x7f67b4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67bad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a8d8b0 0x55ed40a91eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67bad9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 0x55ed40a955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67b3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a84d30 0x55ed40a91970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67b3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a84d30 0x55ed40a91970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:58914/0 (socket says 172.21.15.2:58914) 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67bad9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 0x55ed40a955f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:33378/0 (socket says 172.21.15.2:33378) 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.417+0000 7f67b3fff640 1 -- 172.21.15.2:0/138876267 learned_addr learned my addr 172.21.15.2:0/138876267 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.418+0000 7f67b3fff640 1 -- 172.21.15.2:0/138876267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 msgr2=0x55ed40a955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.418+0000 7f67b3fff640 1 --2- 172.21.15.2:0/138876267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 0x55ed40a955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.418+0000 7f67b3fff640 1 -- 172.21.15.2:0/138876267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a8d8b0 msgr2=0x55ed40a91eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.418+0000 7f67b3fff640 1 --2- 172.21.15.2:0/138876267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a8d8b0 0x55ed40a91eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.418+0000 7f67b3fff640 1 -- 172.21.15.2:0/138876267 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed409ea320 con 0x55ed40a84d30 2024-03-15T13:05:50.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.418+0000 7f67bad9d640 1 --2- 172.21.15.2:0/138876267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 0x55ed40a955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:50.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.418+0000 7f67b3fff640 1 --2- 172.21.15.2:0/138876267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a84d30 0x55ed40a91970 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f67a40059f0 tx=0x7f67a401ac00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e340ef376b320623 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:50.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.418+0000 7f67a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:05:50.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.418+0000 7f67a1ffb640 1 -- 172.21.15.2:0/138876267 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4011070 con 0x55ed40a84d30 2024-03-15T13:05:50.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.418+0000 7f67a1ffb640 1 -- 172.21.15.2:0/138876267 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f67a40111d0 con 0x55ed40a84d30 2024-03-15T13:05:50.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.418+0000 7f67c3618740 1 -- 172.21.15.2:0/138876267 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed40a11900 con 0x55ed40a84d30 2024-03-15T13:05:50.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.418+0000 7f67a1ffb640 1 -- 172.21.15.2:0/138876267 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67a4004020 con 0x55ed40a84d30 2024-03-15T13:05:50.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.418+0000 7f67c3618740 10 client.27620 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:50.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.418+0000 7f67c3618740 1 -- 172.21.15.2:0/138876267 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55ed409ae0b0 con 0x55ed40a84d30 2024-03-15T13:05:50.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.419+0000 7f67a1ffb640 1 -- 172.21.15.2:0/138876267 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55ed40a11900 con 0x55ed40a84d30 2024-03-15T13:05:50.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.419+0000 7f67a1ffb640 1 -- 172.21.15.2:0/138876267 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f67a4036070 con 0x55ed40a84d30 2024-03-15T13:05:50.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.419+0000 7f67c3618740 10 client.27620 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:50.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.419+0000 7f67c3618740 10 client.27620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:50.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.419+0000 7f67c3618740 20 client.27620 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:50.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.419+0000 7f67c3618740 1 --2- 172.21.15.2:0/138876267 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ed40afaa20 0x55ed40afce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.419+0000 7f67c3618740 4 client.27620 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:05:50.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.419+0000 7f67c3618740 1 -- 172.21.15.2:0/138876267 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ed406eb860 con 0x55ed40afaa20 2024-03-15T13:05:50.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.419+0000 7f67bad9d640 1 --2- 172.21.15.2:0/138876267 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ed40afaa20 0x55ed40afce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.419+0000 7f67a1ffb640 10 client.27620 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:50.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.419+0000 7f67bad9d640 1 --2- 172.21.15.2:0/138876267 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ed40afaa20 0x55ed40afce40 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e73be231fc12b7c7 server_cookie=6f786541d49fc5aa in_seq=0 out_seq=0 2024-03-15T13:05:50.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.420+0000 7f67a1ffb640 1 -- 172.21.15.2:0/138876267 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ed406eb860 con 0x55ed40afaa20 2024-03-15T13:05:50.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.420+0000 7f67a1ffb640 10 client.27620 handle_command_reply: tid=0 2024-03-15T13:05:50.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.420+0000 7f67c3618740 1 client.27620 shutdown 2024-03-15T13:05:50.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.420+0000 7f67c3618740 1 -- 172.21.15.2:0/138876267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a84d30 msgr2=0x55ed40a91970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.420+0000 7f67c3618740 1 --2- 172.21.15.2:0/138876267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a84d30 0x55ed40a91970 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f67a40059f0 tx=0x7f67a401ac00 comp rx=0 tx=0).stop 2024-03-15T13:05:50.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c3618740 1 -- 172.21.15.2:0/138876267 shutdown_connections 2024-03-15T13:05:50.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c3618740 1 --2- 172.21.15.2:0/138876267 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ed40afaa20 0x55ed40afce40 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c3618740 1 --2- 172.21.15.2:0/138876267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ed40a8d8b0 0x55ed40a91eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c3618740 1 --2- 172.21.15.2:0/138876267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed40a84d30 0x55ed40a91970 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c3618740 1 --2- 172.21.15.2:0/138876267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed40a84380 0x55ed40a955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c3618740 1 -- 172.21.15.2:0/138876267 >> 172.21.15.2:0/138876267 conn(0x55ed40a7e840 msgr2=0x55ed40a7d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:50.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67a1ffb640 0 client.27620 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:50.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c3618740 1 -- 172.21.15.2:0/138876267 shutdown_connections 2024-03-15T13:05:50.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c3618740 1 -- 172.21.15.2:0/138876267 wait complete. 2024-03-15T13:05:50.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c3618740 20 client.27620 trim_cache size 0 max 16384 2024-03-15T13:05:50.411 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:05:50.411 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:05:50.411 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "eb1302c7-b63c-4428-9409-829a5468426b", 2024-03-15T13:05:50.411 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:05:50.411 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:05:50.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c0a39640 1 -- 172.21.15.2:0/3167962117 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f67a80a1ba0 msgr2=0x7f67a80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c0a39640 1 --2- 172.21.15.2:0/3167962117 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f67a80a1ba0 0x7f67a80a4060 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f67ac09a040 tx=0x7f67ac00cf90 comp rx=0 tx=0).stop 2024-03-15T13:05:50.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c0a39640 1 -- 172.21.15.2:0/3167962117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67bc102760 msgr2=0x7f67bc1003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c0a39640 1 --2- 172.21.15.2:0/3167962117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67bc102760 0x7f67bc1003b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00e9f0 tx=0x7f67ac00eeb0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c0a39640 1 -- 172.21.15.2:0/3167962117 shutdown_connections 2024-03-15T13:05:50.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c0a39640 1 --2- 172.21.15.2:0/3167962117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67bc10bb30 0x7f67bc100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c0a39640 1 --2- 172.21.15.2:0/3167962117 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f67a80a1ba0 0x7f67a80a4060 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c0a39640 1 --2- 172.21.15.2:0/3167962117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67bc103080 0x7f67bc1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c0a39640 1 --2- 172.21.15.2:0/3167962117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67bc102760 0x7f67bc1003b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c0a39640 1 -- 172.21.15.2:0/3167962117 >> 172.21.15.2:0/3167962117 conn(0x7f67bc077550 msgr2=0x7f67bc0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:50.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.421+0000 7f67c0a39640 1 -- 172.21.15.2:0/3167962117 shutdown_connections 2024-03-15T13:05:50.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.422+0000 7f67c0a39640 1 -- 172.21.15.2:0/3167962117 wait complete. 2024-03-15T13:05:50.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:05:50.417 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:05:50.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.537+0000 7f858b4ec640 1 -- 172.21.15.2:0/3715805442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8584107640 msgr2=0x7f8584107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.537+0000 7f858b4ec640 1 --2- 172.21.15.2:0/3715805442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8584107640 0x7f8584107a20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8580009930 tx=0x7f858002bc30 comp rx=0 tx=0).stop 2024-03-15T13:05:50.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.537+0000 7f858b4ec640 1 -- 172.21.15.2:0/3715805442 shutdown_connections 2024-03-15T13:05:50.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.537+0000 7f858b4ec640 1 --2- 172.21.15.2:0/3715805442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f858406a500 0x7f858410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.537+0000 7f858b4ec640 1 --2- 172.21.15.2:0/3715805442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8584069b60 0x7f8584069fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.537+0000 7f858b4ec640 1 --2- 172.21.15.2:0/3715805442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8584107640 0x7f8584107a20 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.537+0000 7f858b4ec640 1 -- 172.21.15.2:0/3715805442 >> 172.21.15.2:0/3715805442 conn(0x7f8584077190 msgr2=0x7f85840775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:50.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.537+0000 7f858b4ec640 1 -- 172.21.15.2:0/3715805442 shutdown_connections 2024-03-15T13:05:50.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.537+0000 7f858b4ec640 1 -- 172.21.15.2:0/3715805442 wait complete. 2024-03-15T13:05:50.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f858b4ec640 1 Processor -- start 2024-03-15T13:05:50.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f858b4ec640 1 -- start start 2024-03-15T13:05:50.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f858b4ec640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8584069b60 0x7f8584100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f858b4ec640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f858406a500 0x7f8584100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f858b4ec640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8584107640 0x7f8584101190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f858b4ec640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85841a5e20 con 0x7f8584107640 2024-03-15T13:05:50.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f858b4ec640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85841a5f90 con 0x7f858406a500 2024-03-15T13:05:50.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f858b4ec640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85841a6150 con 0x7f8584069b60 2024-03-15T13:05:50.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f8589a89640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8584107640 0x7f8584101190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f8589a89640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8584107640 0x7f8584101190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53466/0 (socket says 172.21.15.2:53466) 2024-03-15T13:05:50.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f8589288640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8584069b60 0x7f8584100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f8589a89640 1 -- 172.21.15.2:0/1943063195 learned_addr learned my addr 172.21.15.2:0/1943063195 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:50.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f8588a87640 1 --2- 172.21.15.2:0/1943063195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f858406a500 0x7f8584100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f8589a89640 1 -- 172.21.15.2:0/1943063195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8584069b60 msgr2=0x7f8584100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f8589a89640 1 --2- 172.21.15.2:0/1943063195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8584069b60 0x7f8584100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f8589a89640 1 -- 172.21.15.2:0/1943063195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f858406a500 msgr2=0x7f8584100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f8589a89640 1 --2- 172.21.15.2:0/1943063195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f858406a500 0x7f8584100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f8589a89640 1 -- 172.21.15.2:0/1943063195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8580009560 con 0x7f8584107640 2024-03-15T13:05:50.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.538+0000 7f8589a89640 1 --2- 172.21.15.2:0/1943063195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8584107640 0x7f8584101190 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f857c00eb10 tx=0x7f857c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:50.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.539+0000 7f857a7fc640 1 -- 172.21.15.2:0/1943063195 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c004b20 con 0x7f8584107640 2024-03-15T13:05:50.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.539+0000 7f857a7fc640 1 -- 172.21.15.2:0/1943063195 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f857c004c80 con 0x7f8584107640 2024-03-15T13:05:50.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.539+0000 7f857a7fc640 1 -- 172.21.15.2:0/1943063195 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c010690 con 0x7f8584107640 2024-03-15T13:05:50.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.539+0000 7f858b4ec640 1 -- 172.21.15.2:0/1943063195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85841a6430 con 0x7f8584107640 2024-03-15T13:05:50.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.539+0000 7f858b4ec640 1 -- 172.21.15.2:0/1943063195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85841a68f0 con 0x7f8584107640 2024-03-15T13:05:50.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.541+0000 7f857a7fc640 1 -- 172.21.15.2:0/1943063195 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f857c004df0 con 0x7f8584107640 2024-03-15T13:05:50.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.541+0000 7f858e19c740 1 Processor -- start 2024-03-15T13:05:50.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.541+0000 7f858e19c740 1 -- start start 2024-03-15T13:05:50.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f858e19c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbec690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f858e19c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbf5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f857a7fc640 1 --2- 172.21.15.2:0/1943063195 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f85700a1ba0 0x7f85700a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f858e19c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f858e19c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdcb5b280 con 0x55bfdcbf57b0 2024-03-15T13:05:50.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f858e19c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdcb587b0 con 0x55bfdcbecbd0 2024-03-15T13:05:50.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f858e19c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdcb53f90 con 0x55bfdcbec2b0 2024-03-15T13:05:50.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f8589288640 1 --2- 172.21.15.2:0/1943063195 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f85700a1ba0 0x7f85700a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f8588a87640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbec690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f8589a89640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f8588a87640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbec690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:33414/0 (socket says 172.21.15.2:33414) 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f8588a87640 1 -- 172.21.15.2:0/984968400 learned_addr learned my addr 172.21.15.2:0/984968400 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f857a7fc640 1 -- 172.21.15.2:0/1943063195 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f857c014070 con 0x7f8584107640 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f8588a87640 1 --2- 172.21.15.2:0/984968400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbf5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f8588a87640 1 -- 172.21.15.2:0/984968400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbecbd0 msgr2=0x55bfdcbf5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f8588a87640 1 --2- 172.21.15.2:0/984968400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbf5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f8588a87640 1 -- 172.21.15.2:0/984968400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 msgr2=0x55bfdcbf7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f8588a87640 1 --2- 172.21.15.2:0/984968400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f8588a87640 1 -- 172.21.15.2:0/984968400 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfdcb12570 con 0x55bfdcbec2b0 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f8589a89640 1 --2- 172.21.15.2:0/984968400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f8589288640 1 --2- 172.21.15.2:0/1943063195 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f85700a1ba0 0x7f85700a4060 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8580036040 tx=0x7f8580000bc0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f8588a87640 1 --2- 172.21.15.2:0/984968400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbec690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f85740117e0 tx=0x7f85740097d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a69f648cba89ae37 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.542+0000 7f856f7fe640 1 -- 172.21.15.2:0/984968400 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574004030 con 0x55bfdcbec2b0 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.543+0000 7f856f7fe640 1 -- 172.21.15.2:0/984968400 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f857400f7c0 con 0x55bfdcbec2b0 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.543+0000 7f856f7fe640 1 -- 172.21.15.2:0/984968400 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574003a10 con 0x55bfdcbec2b0 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.543+0000 7f858e19c740 1 -- 172.21.15.2:0/984968400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbec2b0 msgr2=0x55bfdcbec690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.543+0000 7f858e19c740 1 --2- 172.21.15.2:0/984968400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbec690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f85740117e0 tx=0x7f85740097d0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.543+0000 7f858e19c740 1 -- 172.21.15.2:0/984968400 shutdown_connections 2024-03-15T13:05:50.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.543+0000 7f858e19c740 1 --2- 172.21.15.2:0/984968400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.543+0000 7f858e19c740 1 --2- 172.21.15.2:0/984968400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbf5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.543+0000 7f858e19c740 1 --2- 172.21.15.2:0/984968400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbec690 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.543+0000 7f858e19c740 1 -- 172.21.15.2:0/984968400 >> 172.21.15.2:0/984968400 conn(0x55bfdcbe6990 msgr2=0x55bfdcbe6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:50.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.543+0000 7f858e19c740 1 -- 172.21.15.2:0/984968400 shutdown_connections 2024-03-15T13:05:50.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.543+0000 7f858e19c740 1 -- 172.21.15.2:0/984968400 wait complete. 2024-03-15T13:05:50.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f858e19c740 1 -- start start 2024-03-15T13:05:50.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f858e19c740 1 Processor -- start 2024-03-15T13:05:50.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f858e19c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbf5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f858e19c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbf25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f858e19c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f858e19c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdc827430 con 0x55bfdcbf57b0 2024-03-15T13:05:50.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f858e19c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdc7e3e00 con 0x55bfdcbec2b0 2024-03-15T13:05:50.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f858e19c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdcad8fd0 con 0x55bfdcbecbd0 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f858e19c740 10 client.0 _ll_register_callbacks cb 0x55bfdcb2efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f8589a89640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f8588a87640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbf25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f8589288640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbf5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f8588a87640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbf25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:33424/0 (socket says 172.21.15.2:33424) 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f8589288640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbf5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:58962/0 (socket says 172.21.15.2:58962) 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f8589a89640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53496/0 (socket says 172.21.15.2:53496) 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f8589288640 1 -- 172.21.15.2:0/3603205154 learned_addr learned my addr 172.21.15.2:0/3603205154 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f8589a89640 1 -- 172.21.15.2:0/3603205154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbecbd0 msgr2=0x55bfdcbf25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f8589a89640 1 --2- 172.21.15.2:0/3603205154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbf25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f8589a89640 1 -- 172.21.15.2:0/3603205154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbec2b0 msgr2=0x55bfdcbf5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f8589a89640 1 --2- 172.21.15.2:0/3603205154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbf5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.544+0000 7f8589a89640 1 -- 172.21.15.2:0/3603205154 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8574019040 con 0x55bfdcbf57b0 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f8589a89640 1 --2- 172.21.15.2:0/3603205154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf2cd0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f857c0cb490 tx=0x7f857c01da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a512595b9b0a5d76 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f856cff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:50.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f856cff9640 1 -- 172.21.15.2:0/3603205154 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c00acd0 con 0x55bfdcbf57b0 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f856cff9640 1 -- 172.21.15.2:0/3603205154 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f857c09b070 con 0x55bfdcbf57b0 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f858e19c740 1 -- 172.21.15.2:0/3603205154 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfdc8537e0 con 0x55bfdcbf57b0 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f856cff9640 1 -- 172.21.15.2:0/3603205154 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f857c0acfe0 con 0x55bfdcbf57b0 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f858e19c740 10 client.17769 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f858e19c740 1 -- 172.21.15.2:0/3603205154 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55bfdcb52610 con 0x55bfdcbf57b0 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f856cff9640 1 -- 172.21.15.2:0/3603205154 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55bfdc8537e0 con 0x55bfdcbf57b0 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f856cff9640 1 -- 172.21.15.2:0/3603205154 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f857c01f060 con 0x55bfdcbf57b0 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f858e19c740 10 client.17769 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f858e19c740 10 client.17769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f858e19c740 20 client.17769 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f858e19c740 1 --2- 172.21.15.2:0/3603205154 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bfdcc059b0 0x55bfdcc07da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f858e19c740 4 client.17769 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.545+0000 7f858e19c740 1 -- 172.21.15.2:0/3603205154 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfdc890720 con 0x55bfdcc059b0 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.546+0000 7f8589288640 1 --2- 172.21.15.2:0/3603205154 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bfdcc059b0 0x55bfdcc07da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.546+0000 7f856cff9640 10 client.17769 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:50.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.546+0000 7f8589288640 1 --2- 172.21.15.2:0/3603205154 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bfdcc059b0 0x55bfdcc07da0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71ad8e2fabef2514 server_cookie=3b0725fa3880e9c2 in_seq=0 out_seq=0 2024-03-15T13:05:50.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.547+0000 7f856cff9640 1 -- 172.21.15.2:0/3603205154 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bfdc890720 con 0x55bfdcc059b0 2024-03-15T13:05:50.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.547+0000 7f856cff9640 10 client.17769 handle_command_reply: tid=0 2024-03-15T13:05:50.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.547+0000 7f858e19c740 1 client.17769 shutdown 2024-03-15T13:05:50.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.547+0000 7f858e19c740 1 -- 172.21.15.2:0/3603205154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 msgr2=0x55bfdcbf2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.547+0000 7f858e19c740 1 --2- 172.21.15.2:0/3603205154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf2cd0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f857c0cb490 tx=0x7f857c01da50 comp rx=0 tx=0).stop 2024-03-15T13:05:50.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.547+0000 7f858e19c740 1 -- 172.21.15.2:0/3603205154 shutdown_connections 2024-03-15T13:05:50.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.547+0000 7f858e19c740 1 --2- 172.21.15.2:0/3603205154 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bfdcc059b0 0x55bfdcc07da0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.547+0000 7f858e19c740 1 --2- 172.21.15.2:0/3603205154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf2cd0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.547+0000 7f858e19c740 1 --2- 172.21.15.2:0/3603205154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbf25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.547+0000 7f856cff9640 0 client.17769 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:50.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.547+0000 7f858e19c740 1 --2- 172.21.15.2:0/3603205154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbf5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.547+0000 7f858e19c740 1 -- 172.21.15.2:0/3603205154 >> 172.21.15.2:0/3603205154 conn(0x55bfdcbe6990 msgr2=0x55bfdcbe8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:50.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.547+0000 7f858e19c740 1 -- 172.21.15.2:0/3603205154 shutdown_connections 2024-03-15T13:05:50.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.547+0000 7f858e19c740 1 -- 172.21.15.2:0/3603205154 wait complete. 2024-03-15T13:05:50.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.547+0000 7f858e19c740 20 client.17769 trim_cache size 0 max 16384 2024-03-15T13:05:50.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.563+0000 7f858e19c740 1 Processor -- start 2024-03-15T13:05:50.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.563+0000 7f858e19c740 1 -- start start 2024-03-15T13:05:50.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.563+0000 7f858e19c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbf2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.563+0000 7f858e19c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbf4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.563+0000 7f858e19c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.563+0000 7f858e19c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdcc08430 con 0x55bfdcbf57b0 2024-03-15T13:05:50.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.563+0000 7f858e19c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdcbebf10 con 0x55bfdcbecbd0 2024-03-15T13:05:50.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.563+0000 7f858e19c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdcbe7a10 con 0x55bfdcbec2b0 2024-03-15T13:05:50.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.563+0000 7f8589a89640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.563+0000 7f8588a87640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbf4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.563+0000 7f8589a89640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53500/0 (socket says 172.21.15.2:53500) 2024-03-15T13:05:50.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.563+0000 7f8589288640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbf2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.563+0000 7f8589a89640 1 -- 172.21.15.2:0/500984667 learned_addr learned my addr 172.21.15.2:0/500984667 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:50.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f8589a89640 1 -- 172.21.15.2:0/500984667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbec2b0 msgr2=0x55bfdcbf2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f8589a89640 1 --2- 172.21.15.2:0/500984667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbf2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f8589a89640 1 -- 172.21.15.2:0/500984667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbecbd0 msgr2=0x55bfdcbf4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f8589a89640 1 --2- 172.21.15.2:0/500984667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbf4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f8589a89640 1 -- 172.21.15.2:0/500984667 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfdcaeeac0 con 0x55bfdcbf57b0 2024-03-15T13:05:50.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f8589a89640 1 --2- 172.21.15.2:0/500984667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf5250 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f857c0049d0 tx=0x7f857c009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3002a5e13d0bd server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:50.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f856effd640 1 -- 172.21.15.2:0/500984667 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c01b360 con 0x55bfdcbf57b0 2024-03-15T13:05:50.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f856effd640 1 -- 172.21.15.2:0/500984667 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f857c09a4b0 con 0x55bfdcbf57b0 2024-03-15T13:05:50.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f856effd640 1 -- 172.21.15.2:0/500984667 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c0cbbe0 con 0x55bfdcbf57b0 2024-03-15T13:05:50.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f858e19c740 1 -- 172.21.15.2:0/500984667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 msgr2=0x55bfdcbf5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f858e19c740 1 --2- 172.21.15.2:0/500984667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf5250 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f857c0049d0 tx=0x7f857c009a70 comp rx=0 tx=0).stop 2024-03-15T13:05:50.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f858e19c740 1 -- 172.21.15.2:0/500984667 shutdown_connections 2024-03-15T13:05:50.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f858e19c740 1 --2- 172.21.15.2:0/500984667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf5250 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f858e19c740 1 --2- 172.21.15.2:0/500984667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbf4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f858e19c740 1 --2- 172.21.15.2:0/500984667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbf2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.564+0000 7f858e19c740 1 -- 172.21.15.2:0/500984667 >> 172.21.15.2:0/500984667 conn(0x55bfdcbe6990 msgr2=0x55bfdcbf1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:50.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f858e19c740 1 -- 172.21.15.2:0/500984667 shutdown_connections 2024-03-15T13:05:50.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f858e19c740 1 -- 172.21.15.2:0/500984667 wait complete. 2024-03-15T13:05:50.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f858e19c740 1 -- start start 2024-03-15T13:05:50.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f858e19c740 1 Processor -- start 2024-03-15T13:05:50.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f858e19c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbfd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f858e19c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbfd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f858e19c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f858e19c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdcbf71c0 con 0x55bfdcbf57b0 2024-03-15T13:05:50.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f858e19c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdcbf7330 con 0x55bfdcbec2b0 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f858e19c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfdcbf74f0 con 0x55bfdcbecbd0 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f858e19c740 10 client.0 _ll_register_callbacks cb 0x7f857c014f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f8589a89640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f8588a87640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbfd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f8589a89640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53514/0 (socket says 172.21.15.2:53514) 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f8589a89640 1 -- 172.21.15.2:0/2554806538 learned_addr learned my addr 172.21.15.2:0/2554806538 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.565+0000 7f8589288640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbfd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f8589a89640 1 -- 172.21.15.2:0/2554806538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbecbd0 msgr2=0x55bfdcbfd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f8589a89640 1 --2- 172.21.15.2:0/2554806538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbfd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f8589a89640 1 -- 172.21.15.2:0/2554806538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbec2b0 msgr2=0x55bfdcbfd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f8589a89640 1 --2- 172.21.15.2:0/2554806538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbfd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f8589a89640 1 -- 172.21.15.2:0/2554806538 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfdcb52610 con 0x55bfdcbf57b0 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f8589a89640 1 --2- 172.21.15.2:0/2554806538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf6af0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f857c0ae010 tx=0x7f857c0cb920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b09bc5b4c35af86 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f856cff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f856cff9640 1 -- 172.21.15.2:0/2554806538 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c09eac0 con 0x55bfdcbf57b0 2024-03-15T13:05:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f856cff9640 1 -- 172.21.15.2:0/2554806538 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9364+0+0 (secure 0 0 0) 0x7f857c09b070 con 0x55bfdcbf57b0 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f858e19c740 1 -- 172.21.15.2:0/2554806538 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f857400f7c0 con 0x55bfdcbf57b0 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f856cff9640 1 -- 172.21.15.2:0/2554806538 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f857c085870 con 0x55bfdcbf57b0 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f858e19c740 10 client.17775 fetch_fsmap learned FSMap version 50 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f858e19c740 1 -- 172.21.15.2:0/2554806538 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55bfdcb146a0 con 0x55bfdcbf57b0 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f856cff9640 1 -- 172.21.15.2:0/2554806538 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f857400f7c0 con 0x55bfdcbf57b0 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f856cff9640 1 -- 172.21.15.2:0/2554806538 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f857c01f070 con 0x55bfdcbf57b0 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f858e19c740 10 client.17775 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f858e19c740 10 client.17775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f858e19c740 20 client.17775 populate_metadata read hostname 'smithi002' 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.566+0000 7f858e19c740 1 --2- 172.21.15.2:0/2554806538 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bfdcc62000 0x55bfdcc643f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f858e19c740 4 client.17775 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f858e19c740 1 -- 172.21.15.2:0/2554806538 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bfdc8df890 con 0x55bfdcc62000 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f8589288640 1 --2- 172.21.15.2:0/2554806538 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bfdcc62000 0x55bfdcc643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f8589288640 1 --2- 172.21.15.2:0/2554806538 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bfdcc62000 0x55bfdcc643f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f035fa0e0e933a94 server_cookie=f9f6bc9303d58e07 in_seq=0 out_seq=0 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f856cff9640 10 client.17775 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:50.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f856cff9640 1 -- 172.21.15.2:0/2554806538 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bfdc8df890 con 0x55bfdcc62000 2024-03-15T13:05:50.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f856cff9640 10 client.17775 handle_command_reply: tid=0 2024-03-15T13:05:50.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f858e19c740 1 client.17775 shutdown 2024-03-15T13:05:50.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f858e19c740 1 -- 172.21.15.2:0/2554806538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 msgr2=0x55bfdcbf6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f858e19c740 1 --2- 172.21.15.2:0/2554806538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf6af0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f857c0ae010 tx=0x7f857c0cb920 comp rx=0 tx=0).stop 2024-03-15T13:05:50.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f858e19c740 1 -- 172.21.15.2:0/2554806538 shutdown_connections 2024-03-15T13:05:50.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f858e19c740 1 --2- 172.21.15.2:0/2554806538 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bfdcc62000 0x55bfdcc643f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f858e19c740 1 --2- 172.21.15.2:0/2554806538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bfdcbf57b0 0x55bfdcbf6af0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f858e19c740 1 --2- 172.21.15.2:0/2554806538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfdcbecbd0 0x55bfdcbfd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f858e19c740 1 --2- 172.21.15.2:0/2554806538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfdcbec2b0 0x55bfdcbfd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f856cff9640 0 client.17775 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:05:50.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.567+0000 7f858e19c740 1 -- 172.21.15.2:0/2554806538 >> 172.21.15.2:0/2554806538 conn(0x55bfdcbe6990 msgr2=0x55bfdcbe5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:50.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.568+0000 7f858e19c740 1 -- 172.21.15.2:0/2554806538 shutdown_connections 2024-03-15T13:05:50.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.568+0000 7f858e19c740 1 -- 172.21.15.2:0/2554806538 wait complete. 2024-03-15T13:05:50.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.568+0000 7f858e19c740 20 client.17775 trim_cache size 0 max 16384 2024-03-15T13:05:50.558 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:05:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T13:05:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:05:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "eb1302c7-b63c-4428-9409-829a5468426b": { 2024-03-15T13:05:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:05:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "eb1302c7-b63c-4428-9409-829a5468426b", 2024-03-15T13:05:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:05:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:05:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:05:50.558 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:05:50.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.568+0000 7f858b4ec640 1 -- 172.21.15.2:0/1943063195 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f85700a1ba0 msgr2=0x7f85700a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.568+0000 7f858b4ec640 1 --2- 172.21.15.2:0/1943063195 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f85700a1ba0 0x7f85700a4060 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8580036040 tx=0x7f8580000bc0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.568+0000 7f858b4ec640 1 -- 172.21.15.2:0/1943063195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8584107640 msgr2=0x7f8584101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:05:50.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.568+0000 7f858b4ec640 1 --2- 172.21.15.2:0/1943063195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8584107640 0x7f8584101190 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f857c00eb10 tx=0x7f857c00c2d0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.568+0000 7f858b4ec640 1 -- 172.21.15.2:0/1943063195 shutdown_connections 2024-03-15T13:05:50.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.568+0000 7f858b4ec640 1 --2- 172.21.15.2:0/1943063195 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f85700a1ba0 0x7f85700a4060 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.568+0000 7f858b4ec640 1 --2- 172.21.15.2:0/1943063195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8584107640 0x7f8584101190 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.568+0000 7f858b4ec640 1 --2- 172.21.15.2:0/1943063195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f858406a500 0x7f8584100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.568+0000 7f858b4ec640 1 --2- 172.21.15.2:0/1943063195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8584069b60 0x7f8584100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:05:50.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.568+0000 7f858b4ec640 1 -- 172.21.15.2:0/1943063195 >> 172.21.15.2:0/1943063195 conn(0x7f8584077190 msgr2=0x7f85840ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:05:50.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.568+0000 7f858b4ec640 1 -- 172.21.15.2:0/1943063195 shutdown_connections 2024-03-15T13:05:50.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:05:50.568+0000 7f858b4ec640 1 -- 172.21.15.2:0/1943063195 wait complete. 2024-03-15T13:05:50.563 INFO:tasks.cephfs.filesystem:scrub status for tag:eb1302c7-b63c-4428-9409-829a5468426b - {'path': '/', 'tag': 'eb1302c7-b63c-4428-9409-829a5468426b', 'options': 'recursive,force'} 2024-03-15T13:05:52.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:51 smithi002 ceph-mon[32060]: pgmap v211: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:05:52.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:51 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:05:52.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:51 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:05:52.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:51 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:05:52.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:51 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:05:52.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:51 smithi184 ceph-mon[35993]: pgmap v211: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:05:52.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:51 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:05:52.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:51 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:05:52.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:51 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:05:52.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:51 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:05:52.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:51 smithi149 ceph-mon[35927]: pgmap v211: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:05:52.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:51 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:05:52.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:51 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:05:52.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:51 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:05:52.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:51 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:05:54.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:53 smithi002 ceph-mon[32060]: pgmap v212: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:54.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:53 smithi184 ceph-mon[35993]: pgmap v212: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:54.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:53 smithi149 ceph-mon[35927]: pgmap v212: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:56.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:55 smithi002 ceph-mon[32060]: pgmap v213: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:56.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:55 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:05:56.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:55 smithi184 ceph-mon[35993]: pgmap v213: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:56.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:55 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:05:56.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:55 smithi149 ceph-mon[35927]: pgmap v213: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:05:56.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:55 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:05:58.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:57 smithi002 ceph-mon[32060]: pgmap v214: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:05:58.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:57 smithi184 ceph-mon[35993]: pgmap v214: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:05:58.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:57 smithi149 ceph-mon[35927]: pgmap v214: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:06:00.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:05:59 smithi002 ceph-mon[32060]: pgmap v215: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:06:00.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:05:59 smithi184 ceph-mon[35993]: pgmap v215: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:06:00.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:05:59 smithi149 ceph-mon[35927]: pgmap v215: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:06:02.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:01 smithi002 ceph-mon[32060]: pgmap v216: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:06:02.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:01 smithi184 ceph-mon[35993]: pgmap v216: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:06:02.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:01 smithi149 ceph-mon[35927]: pgmap v216: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:06:04.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:03 smithi002 ceph-mon[32060]: pgmap v217: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:06:04.087 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:03 smithi184 ceph-mon[35993]: pgmap v217: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:06:04.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:03 smithi149 ceph-mon[35927]: pgmap v217: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:06:06.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:05 smithi002 ceph-mon[32060]: pgmap v218: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:06:06.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:05 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:06:06.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:05 smithi184 ceph-mon[35993]: pgmap v218: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:06:06.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:05 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:06:06.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:05 smithi149 ceph-mon[35927]: pgmap v218: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:06:06.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:05 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:06:07.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:06 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:06:07.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:06 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:06:07.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:06 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:06:07.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:06 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:06:07.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:06 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:06:07.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:06 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:06:07.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:06 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:06:07.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:06 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:06:07.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:06 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:06:08.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:07 smithi002 ceph-mon[32060]: pgmap v219: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-03-15T13:06:08.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:07 smithi184 ceph-mon[35993]: pgmap v219: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-03-15T13:06:08.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:07 smithi149 ceph-mon[35927]: pgmap v219: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-03-15T13:06:10.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:09 smithi002 ceph-mon[32060]: pgmap v220: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-03-15T13:06:10.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:09 smithi184 ceph-mon[35993]: pgmap v220: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-03-15T13:06:10.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:09 smithi149 ceph-mon[35927]: pgmap v220: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-03-15T13:06:11.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:10 smithi002 ceph-mon[32060]: osdmap e93: 12 total, 12 up, 12 in 2024-03-15T13:06:11.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:10 smithi184 ceph-mon[35993]: osdmap e93: 12 total, 12 up, 12 in 2024-03-15T13:06:11.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:10 smithi149 ceph-mon[35927]: osdmap e93: 12 total, 12 up, 12 in 2024-03-15T13:06:12.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:11 smithi002 ceph-mon[32060]: pgmap v222: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-15T13:06:12.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:11 smithi184 ceph-mon[35993]: pgmap v222: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-15T13:06:12.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:11 smithi149 ceph-mon[35927]: pgmap v222: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-15T13:06:14.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:13 smithi002 ceph-mon[32060]: pgmap v223: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-15T13:06:14.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:13 smithi184 ceph-mon[35993]: pgmap v223: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-15T13:06:14.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:13 smithi149 ceph-mon[35927]: pgmap v223: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-15T13:06:16.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:15 smithi002 ceph-mon[32060]: pgmap v224: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-15T13:06:16.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:15 smithi184 ceph-mon[35993]: pgmap v224: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-15T13:06:16.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:15 smithi149 ceph-mon[35927]: pgmap v224: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-03-15T13:06:18.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:17 smithi184 ceph-mon[35993]: pgmap v225: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:06:18.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:17 smithi149 ceph-mon[35927]: pgmap v225: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:06:18.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:17 smithi002 ceph-mon[32060]: pgmap v225: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:06:20.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:19 smithi184 ceph-mon[35993]: pgmap v226: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:06:20.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:19 smithi149 ceph-mon[35927]: pgmap v226: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:06:20.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:19 smithi002 ceph-mon[32060]: pgmap v226: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:06:20.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:06:20.564 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:06:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.693+0000 7f5655555640 1 -- 172.21.15.2:0/1165805331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5650069b20 msgr2=0x7f5650069fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.693+0000 7f5655555640 1 --2- 172.21.15.2:0/1165805331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5650069b20 0x7f5650069fa0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5644009930 tx=0x7f564402bcd0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.693+0000 7f5655555640 1 -- 172.21.15.2:0/1165805331 shutdown_connections 2024-03-15T13:06:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.693+0000 7f5655555640 1 --2- 172.21.15.2:0/1165805331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f565006a4e0 0x7f565010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.693+0000 7f5655555640 1 --2- 172.21.15.2:0/1165805331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5650069b20 0x7f5650069fa0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.693+0000 7f5655555640 1 --2- 172.21.15.2:0/1165805331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5650107600 0x7f56501079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.693+0000 7f5655555640 1 -- 172.21.15.2:0/1165805331 >> 172.21.15.2:0/1165805331 conn(0x7f5650077190 msgr2=0x7f56500775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.693+0000 7f5655555640 1 -- 172.21.15.2:0/1165805331 shutdown_connections 2024-03-15T13:06:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.693+0000 7f5655555640 1 -- 172.21.15.2:0/1165805331 wait complete. 2024-03-15T13:06:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.694+0000 7f5655555640 1 Processor -- start 2024-03-15T13:06:20.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.694+0000 7f5655555640 1 -- start start 2024-03-15T13:06:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.694+0000 7f5655555640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5650069b20 0x7f5650100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.694+0000 7f5655555640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f565006a4e0 0x7f5650100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.694+0000 7f5655555640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5650107600 0x7f5650101190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.694+0000 7f5655555640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56501a5f50 con 0x7f5650107600 2024-03-15T13:06:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.694+0000 7f5655555640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56501a60c0 con 0x7f565006a4e0 2024-03-15T13:06:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.694+0000 7f5655555640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56501a6280 con 0x7f5650069b20 2024-03-15T13:06:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f564f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5650107600 0x7f5650101190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f564effd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5650069b20 0x7f5650100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f564f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5650107600 0x7f5650101190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33898/0 (socket says 172.21.15.2:33898) 2024-03-15T13:06:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f564f7fe640 1 -- 172.21.15.2:0/3034099182 learned_addr learned my addr 172.21.15.2:0/3034099182 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f564e7fc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f565006a4e0 0x7f5650100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f564f7fe640 1 -- 172.21.15.2:0/3034099182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5650069b20 msgr2=0x7f5650100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f564f7fe640 1 --2- 172.21.15.2:0/3034099182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5650069b20 0x7f5650100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f564f7fe640 1 -- 172.21.15.2:0/3034099182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f565006a4e0 msgr2=0x7f5650100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f564f7fe640 1 --2- 172.21.15.2:0/3034099182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f565006a4e0 0x7f5650100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f564f7fe640 1 -- 172.21.15.2:0/3034099182 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5648009490 con 0x7f5650107600 2024-03-15T13:06:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f564effd640 1 --2- 172.21.15.2:0/3034099182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5650069b20 0x7f5650100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f564e7fc640 1 --2- 172.21.15.2:0/3034099182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f565006a4e0 0x7f5650100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f564f7fe640 1 --2- 172.21.15.2:0/3034099182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5650107600 0x7f5650101190 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f564800e9b0 tx=0x7f564800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f5637fff640 1 -- 172.21.15.2:0/3034099182 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564800cd80 con 0x7f5650107600 2024-03-15T13:06:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f5637fff640 1 -- 172.21.15.2:0/3034099182 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f56480035f0 con 0x7f5650107600 2024-03-15T13:06:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f5637fff640 1 -- 172.21.15.2:0/3034099182 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564800cee0 con 0x7f5650107600 2024-03-15T13:06:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f5655555640 1 -- 172.21.15.2:0/3034099182 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5644009560 con 0x7f5650107600 2024-03-15T13:06:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.695+0000 7f5655555640 1 -- 172.21.15.2:0/3034099182 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56501a68b0 con 0x7f5650107600 2024-03-15T13:06:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.698+0000 7f5637fff640 1 -- 172.21.15.2:0/3034099182 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f5648003760 con 0x7f5650107600 2024-03-15T13:06:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.698+0000 7f5658205740 1 Processor -- start 2024-03-15T13:06:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.698+0000 7f5658205740 1 -- start start 2024-03-15T13:06:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.698+0000 7f5658205740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f659690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.698+0000 7f5658205740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f659bd0 0x55f47f662270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.698+0000 7f5658205740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f6627b0 0x55f47f664ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.698+0000 7f5658205740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f47f5c8280 con 0x55f47f6627b0 2024-03-15T13:06:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.698+0000 7f5658205740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f47f5c57b0 con 0x55f47f659bd0 2024-03-15T13:06:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.698+0000 7f5658205740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f47f5c0f90 con 0x55f47f6592b0 2024-03-15T13:06:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f5637fff640 1 --2- 172.21.15.2:0/3034099182 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f56380a1c20 0x7f56380a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f564e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f6627b0 0x55f47f664ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f564effd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f659bd0 0x55f47f662270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f564e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f6627b0 0x55f47f664ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33908/0 (socket says 172.21.15.2:33908) 2024-03-15T13:06:20.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f564e7fc640 1 -- 172.21.15.2:0/1823425001 learned_addr learned my addr 172.21.15.2:0/1823425001 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:20.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f564effd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f659bd0 0x55f47f662270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:54432/0 (socket says 172.21.15.2:54432) 2024-03-15T13:06:20.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f564e7fc640 1 --2- 172.21.15.2:0/1823425001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f659690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f564f7fe640 1 --2- 172.21.15.2:0/3034099182 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f56380a1c20 0x7f56380a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f564effd640 1 -- 172.21.15.2:0/1823425001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 msgr2=0x55f47f659690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:20.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f564effd640 1 --2- 172.21.15.2:0/1823425001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f659690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f5637fff640 1 -- 172.21.15.2:0/3034099182 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f5648014070 con 0x7f5650107600 2024-03-15T13:06:20.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f564effd640 1 -- 172.21.15.2:0/1823425001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f6627b0 msgr2=0x55f47f664ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:20.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f564effd640 1 --2- 172.21.15.2:0/1823425001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f6627b0 0x55f47f664ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f564effd640 1 -- 172.21.15.2:0/1823425001 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f47f57f570 con 0x55f47f659bd0 2024-03-15T13:06:20.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f564effd640 1 --2- 172.21.15.2:0/1823425001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f659bd0 0x55f47f662270 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5640005fd0 tx=0x7f5640011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f58b932566ab17e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:20.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f564f7fe640 1 --2- 172.21.15.2:0/3034099182 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f56380a1c20 0x7f56380a40e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f564809a520 tx=0x7f56480ba040 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f56357fa640 1 -- 172.21.15.2:0/1823425001 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640004d20 con 0x55f47f659bd0 2024-03-15T13:06:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f56357fa640 1 -- 172.21.15.2:0/1823425001 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5640004e80 con 0x55f47f659bd0 2024-03-15T13:06:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.699+0000 7f56357fa640 1 -- 172.21.15.2:0/1823425001 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56400043d0 con 0x55f47f659bd0 2024-03-15T13:06:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.700+0000 7f5658205740 1 -- 172.21.15.2:0/1823425001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f659bd0 msgr2=0x55f47f662270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.700+0000 7f5658205740 1 --2- 172.21.15.2:0/1823425001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f659bd0 0x55f47f662270 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5640005fd0 tx=0x7f5640011c70 comp rx=0 tx=0).stop 2024-03-15T13:06:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.700+0000 7f5658205740 1 -- 172.21.15.2:0/1823425001 shutdown_connections 2024-03-15T13:06:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.700+0000 7f5658205740 1 --2- 172.21.15.2:0/1823425001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f6627b0 0x55f47f664ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.700+0000 7f5658205740 1 --2- 172.21.15.2:0/1823425001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f659bd0 0x55f47f662270 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.700+0000 7f5658205740 1 --2- 172.21.15.2:0/1823425001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f659690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.700+0000 7f5658205740 1 -- 172.21.15.2:0/1823425001 >> 172.21.15.2:0/1823425001 conn(0x55f47f653990 msgr2=0x55f47f653d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.700+0000 7f5658205740 1 -- 172.21.15.2:0/1823425001 shutdown_connections 2024-03-15T13:06:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.700+0000 7f5658205740 1 -- 172.21.15.2:0/1823425001 wait complete. 2024-03-15T13:06:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.700+0000 7f5658205740 1 -- start start 2024-03-15T13:06:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f5658205740 1 Processor -- start 2024-03-15T13:06:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f5658205740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f662180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f5658205740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f659bd0 0x55f47f65f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f5658205740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f6627b0 0x55f47f65fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f5658205740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f47f294430 con 0x55f47f6627b0 2024-03-15T13:06:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f5658205740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f47f250e00 con 0x55f47f659bd0 2024-03-15T13:06:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f5658205740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f47f545fd0 con 0x55f47f6592b0 2024-03-15T13:06:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f5658205740 10 client.0 _ll_register_callbacks cb 0x55f47f59bfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f564e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f6627b0 0x55f47f65fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f564e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f6627b0 0x55f47f65fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33910/0 (socket says 172.21.15.2:33910) 2024-03-15T13:06:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f564e7fc640 1 -- 172.21.15.2:0/3390094762 learned_addr learned my addr 172.21.15.2:0/3390094762 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f564effd640 1 --2- 172.21.15.2:0/3390094762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f659bd0 0x55f47f65f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f564e7fc640 1 --2- 172.21.15.2:0/3390094762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f662180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f564e7fc640 1 -- 172.21.15.2:0/3390094762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 msgr2=0x55f47f662180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f564e7fc640 1 --2- 172.21.15.2:0/3390094762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f662180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f564e7fc640 1 -- 172.21.15.2:0/3390094762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f659bd0 msgr2=0x55f47f65f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:20.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f564e7fc640 1 --2- 172.21.15.2:0/3390094762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f659bd0 0x55f47f65f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f564e7fc640 1 -- 172.21.15.2:0/3390094762 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5640011440 con 0x55f47f6627b0 2024-03-15T13:06:20.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f564effd640 1 --2- 172.21.15.2:0/3390094762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f659bd0 0x55f47f65f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:20.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f564e7fc640 1 --2- 172.21.15.2:0/3390094762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f6627b0 0x55f47f65fb20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f564402c210 tx=0x7f564402c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766a1b05868161cc server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:20.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.701+0000 7f5632ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:20.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.702+0000 7f5632ff5640 1 -- 172.21.15.2:0/3390094762 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644005060 con 0x55f47f6627b0 2024-03-15T13:06:20.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.702+0000 7f5632ff5640 1 -- 172.21.15.2:0/3390094762 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f564404c060 con 0x55f47f6627b0 2024-03-15T13:06:20.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.702+0000 7f5658205740 1 -- 172.21.15.2:0/3390094762 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f47f2fd720 con 0x55f47f6627b0 2024-03-15T13:06:20.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.702+0000 7f5632ff5640 1 -- 172.21.15.2:0/3390094762 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56440051c0 con 0x55f47f6627b0 2024-03-15T13:06:20.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.702+0000 7f5658205740 10 client.17787 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:20.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.702+0000 7f5658205740 1 -- 172.21.15.2:0/3390094762 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55f47f5bf610 con 0x55f47f6627b0 2024-03-15T13:06:20.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.702+0000 7f5632ff5640 1 -- 172.21.15.2:0/3390094762 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55f47f2fd720 con 0x55f47f6627b0 2024-03-15T13:06:20.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.702+0000 7f5632ff5640 1 -- 172.21.15.2:0/3390094762 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5644043d20 con 0x55f47f6627b0 2024-03-15T13:06:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.702+0000 7f5658205740 10 client.17787 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.702+0000 7f5658205740 10 client.17787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.702+0000 7f5658205740 20 client.17787 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.702+0000 7f5658205740 1 --2- 172.21.15.2:0/3390094762 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f47f672840 0x55f47f674c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.702+0000 7f5658205740 4 client.17787 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.702+0000 7f5658205740 1 -- 172.21.15.2:0/3390094762 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f47f34c890 con 0x55f47f672840 2024-03-15T13:06:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.702+0000 7f564effd640 1 --2- 172.21.15.2:0/3390094762 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f47f672840 0x55f47f674c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.703+0000 7f564effd640 1 --2- 172.21.15.2:0/3390094762 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f47f672840 0x55f47f674c60 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2898a6b4d96f3a9d server_cookie=26bf91043a95a233 in_seq=0 out_seq=0 2024-03-15T13:06:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.703+0000 7f5632ff5640 10 client.17787 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.703+0000 7f5632ff5640 1 -- 172.21.15.2:0/3390094762 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f47f34c890 con 0x55f47f672840 2024-03-15T13:06:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.703+0000 7f5632ff5640 10 client.17787 handle_command_reply: tid=0 2024-03-15T13:06:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.703+0000 7f5658205740 1 client.17787 shutdown 2024-03-15T13:06:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.704+0000 7f5658205740 1 -- 172.21.15.2:0/3390094762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f6627b0 msgr2=0x55f47f65fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:20.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.704+0000 7f5658205740 1 --2- 172.21.15.2:0/3390094762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f6627b0 0x55f47f65fb20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f564402c210 tx=0x7f564402c670 comp rx=0 tx=0).stop 2024-03-15T13:06:20.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.704+0000 7f5658205740 1 -- 172.21.15.2:0/3390094762 shutdown_connections 2024-03-15T13:06:20.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.704+0000 7f5658205740 1 --2- 172.21.15.2:0/3390094762 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f47f672840 0x55f47f674c60 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.704+0000 7f5658205740 1 --2- 172.21.15.2:0/3390094762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f6627b0 0x55f47f65fb20 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.704+0000 7f5658205740 1 --2- 172.21.15.2:0/3390094762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f659bd0 0x55f47f65f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.704+0000 7f5632ff5640 0 client.17787 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:20.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.704+0000 7f5658205740 1 --2- 172.21.15.2:0/3390094762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f662180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.704+0000 7f5658205740 1 -- 172.21.15.2:0/3390094762 >> 172.21.15.2:0/3390094762 conn(0x55f47f653990 msgr2=0x55f47f655730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:20.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.704+0000 7f5658205740 1 -- 172.21.15.2:0/3390094762 shutdown_connections 2024-03-15T13:06:20.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.704+0000 7f5658205740 1 -- 172.21.15.2:0/3390094762 wait complete. 2024-03-15T13:06:20.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.704+0000 7f5658205740 20 client.17787 trim_cache size 0 max 16384 2024-03-15T13:06:20.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f5658205740 1 Processor -- start 2024-03-15T13:06:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f5658205740 1 -- start start 2024-03-15T13:06:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f5658205740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f65a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f5658205740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f659bd0 0x55f47f6528d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f5658205740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f6627b0 0x55f47f661c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f5658205740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f47f65ad90 con 0x55f47f659bd0 2024-03-15T13:06:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f5658205740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f47f672640 con 0x55f47f6627b0 2024-03-15T13:06:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f5658205740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f47f658f10 con 0x55f47f6592b0 2024-03-15T13:06:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f564effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f659bd0 0x55f47f6528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f564effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f659bd0 0x55f47f6528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33916/0 (socket says 172.21.15.2:33916) 2024-03-15T13:06:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f564effd640 1 -- 172.21.15.2:0/1242586993 learned_addr learned my addr 172.21.15.2:0/1242586993 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f564e7fc640 1 --2- 172.21.15.2:0/1242586993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f65a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f564e7fc640 1 --2- 172.21.15.2:0/1242586993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f6627b0 0x55f47f661c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f564effd640 1 -- 172.21.15.2:0/1242586993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 msgr2=0x55f47f65a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f564effd640 1 --2- 172.21.15.2:0/1242586993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f65a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f564effd640 1 -- 172.21.15.2:0/1242586993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f6627b0 msgr2=0x55f47f661c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f564effd640 1 --2- 172.21.15.2:0/1242586993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f6627b0 0x55f47f661c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f564effd640 1 -- 172.21.15.2:0/1242586993 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f47f55bac0 con 0x55f47f659bd0 2024-03-15T13:06:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.720+0000 7f564e7fc640 1 --2- 172.21.15.2:0/1242586993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f6627b0 0x55f47f661c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:06:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.721+0000 7f564effd640 1 --2- 172.21.15.2:0/1242586993 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f659bd0 0x55f47f6528d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f564000fcb0 tx=0x7f5640011710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05d5dc8e1fe0b43 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.721+0000 7f5634ff9640 1 -- 172.21.15.2:0/1242586993 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640003e00 con 0x55f47f659bd0 2024-03-15T13:06:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.721+0000 7f5634ff9640 1 -- 172.21.15.2:0/1242586993 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f564000fce0 con 0x55f47f659bd0 2024-03-15T13:06:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.721+0000 7f5634ff9640 1 -- 172.21.15.2:0/1242586993 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640005430 con 0x55f47f659bd0 2024-03-15T13:06:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.721+0000 7f5658205740 1 -- 172.21.15.2:0/1242586993 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f659bd0 msgr2=0x55f47f6528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.721+0000 7f5658205740 1 --2- 172.21.15.2:0/1242586993 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f659bd0 0x55f47f6528d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f564000fcb0 tx=0x7f5640011710 comp rx=0 tx=0).stop 2024-03-15T13:06:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.721+0000 7f5658205740 1 -- 172.21.15.2:0/1242586993 shutdown_connections 2024-03-15T13:06:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.721+0000 7f5658205740 1 --2- 172.21.15.2:0/1242586993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f6627b0 0x55f47f661c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.721+0000 7f5658205740 1 --2- 172.21.15.2:0/1242586993 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f659bd0 0x55f47f6528d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.721+0000 7f5658205740 1 --2- 172.21.15.2:0/1242586993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f65a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.721+0000 7f5658205740 1 -- 172.21.15.2:0/1242586993 >> 172.21.15.2:0/1242586993 conn(0x55f47f653990 msgr2=0x55f47f6521c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:20.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.721+0000 7f5658205740 1 -- 172.21.15.2:0/1242586993 shutdown_connections 2024-03-15T13:06:20.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.721+0000 7f5658205740 1 -- 172.21.15.2:0/1242586993 wait complete. 2024-03-15T13:06:20.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f5658205740 1 -- start start 2024-03-15T13:06:20.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f5658205740 1 Processor -- start 2024-03-15T13:06:20.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f5658205740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f665820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f5658205740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f659bd0 0x55f47f665d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f5658205740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f6627b0 0x55f47f6ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f5658205740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f47f666460 con 0x55f47f659bd0 2024-03-15T13:06:20.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f5658205740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f47f6ce650 con 0x55f47f6627b0 2024-03-15T13:06:20.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f5658205740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f47f6ce790 con 0x55f47f6592b0 2024-03-15T13:06:20.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f5658205740 10 client.0 _ll_register_callbacks cb 0x7f5644004b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:20.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f564effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f659bd0 0x55f47f665d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f564effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f659bd0 0x55f47f665d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33932/0 (socket says 172.21.15.2:33932) 2024-03-15T13:06:20.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f564effd640 1 -- 172.21.15.2:0/812292809 learned_addr learned my addr 172.21.15.2:0/812292809 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:20.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f564e7fc640 1 --2- 172.21.15.2:0/812292809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f665820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f564e7fc640 1 --2- 172.21.15.2:0/812292809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f6627b0 0x55f47f6ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f564effd640 1 -- 172.21.15.2:0/812292809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 msgr2=0x55f47f665820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:20.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f564effd640 1 --2- 172.21.15.2:0/812292809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f665820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f564effd640 1 -- 172.21.15.2:0/812292809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f6627b0 msgr2=0x55f47f6ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:20.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f564effd640 1 --2- 172.21.15.2:0/812292809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f6627b0 0x55f47f6ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.722+0000 7f564effd640 1 -- 172.21.15.2:0/812292809 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564001d040 con 0x55f47f659bd0 2024-03-15T13:06:20.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f564effd640 1 --2- 172.21.15.2:0/812292809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f659bd0 0x55f47f665d60 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f564002a000 tx=0x7f5640009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c2013808b6b54d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:20.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f5632ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:20.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f5632ff5640 1 -- 172.21.15.2:0/812292809 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564002aa40 con 0x55f47f659bd0 2024-03-15T13:06:20.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f5632ff5640 1 -- 172.21.15.2:0/812292809 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f564002aba0 con 0x55f47f659bd0 2024-03-15T13:06:20.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f5658205740 1 -- 172.21.15.2:0/812292809 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f47f2c07e0 con 0x55f47f659bd0 2024-03-15T13:06:20.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f5632ff5640 1 -- 172.21.15.2:0/812292809 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f564002ad60 con 0x55f47f659bd0 2024-03-15T13:06:20.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f5658205740 10 client.17793 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:20.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f5658205740 1 -- 172.21.15.2:0/812292809 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55f47f5816a0 con 0x55f47f659bd0 2024-03-15T13:06:20.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f5632ff5640 1 -- 172.21.15.2:0/812292809 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55f47f2c07e0 con 0x55f47f659bd0 2024-03-15T13:06:20.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f5632ff5640 1 -- 172.21.15.2:0/812292809 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5640032070 con 0x55f47f659bd0 2024-03-15T13:06:20.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f5658205740 10 client.17793 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:20.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f5658205740 10 client.17793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:20.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f5658205740 20 client.17793 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:20.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f5658205740 1 --2- 172.21.15.2:0/812292809 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f47f6cfb30 0x55f47f6d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f5658205740 4 client.17793 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:06:20.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f5658205740 1 -- 172.21.15.2:0/812292809 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f47f2af4e0 con 0x55f47f6cfb30 2024-03-15T13:06:20.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.723+0000 7f564e7fc640 1 --2- 172.21.15.2:0/812292809 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f47f6cfb30 0x55f47f6d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f564e7fc640 1 --2- 172.21.15.2:0/812292809 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f47f6cfb30 0x55f47f6d1f50 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54f27e30a4aca940 server_cookie=f5ab902dce63056e in_seq=0 out_seq=0 2024-03-15T13:06:20.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5632ff5640 10 client.17793 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:20.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5632ff5640 1 -- 172.21.15.2:0/812292809 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f47f2af4e0 con 0x55f47f6cfb30 2024-03-15T13:06:20.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5632ff5640 10 client.17793 handle_command_reply: tid=0 2024-03-15T13:06:20.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5658205740 1 client.17793 shutdown 2024-03-15T13:06:20.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5658205740 1 -- 172.21.15.2:0/812292809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f659bd0 msgr2=0x55f47f665d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:20.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5658205740 1 --2- 172.21.15.2:0/812292809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f659bd0 0x55f47f665d60 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f564002a000 tx=0x7f5640009570 comp rx=0 tx=0).stop 2024-03-15T13:06:20.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5658205740 1 -- 172.21.15.2:0/812292809 shutdown_connections 2024-03-15T13:06:20.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5658205740 1 --2- 172.21.15.2:0/812292809 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55f47f6cfb30 0x55f47f6d1f50 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5658205740 1 --2- 172.21.15.2:0/812292809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f47f6627b0 0x55f47f6ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5658205740 1 --2- 172.21.15.2:0/812292809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f47f659bd0 0x55f47f665d60 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5658205740 1 --2- 172.21.15.2:0/812292809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f47f6592b0 0x55f47f665820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5632ff5640 0 client.17793 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:20.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5658205740 1 -- 172.21.15.2:0/812292809 >> 172.21.15.2:0/812292809 conn(0x55f47f653990 msgr2=0x55f47f652a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:20.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5658205740 1 -- 172.21.15.2:0/812292809 shutdown_connections 2024-03-15T13:06:20.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5658205740 1 -- 172.21.15.2:0/812292809 wait complete. 2024-03-15T13:06:20.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.724+0000 7f5658205740 20 client.17793 trim_cache size 0 max 16384 2024-03-15T13:06:20.717 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:06:20.717 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:06:20.717 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:06:20.717 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:06:20.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.725+0000 7f5655555640 1 -- 172.21.15.2:0/3034099182 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f56380a1c20 msgr2=0x7f56380a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:20.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.725+0000 7f5655555640 1 --2- 172.21.15.2:0/3034099182 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f56380a1c20 0x7f56380a40e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f564809a520 tx=0x7f56480ba040 comp rx=0 tx=0).stop 2024-03-15T13:06:20.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.725+0000 7f5655555640 1 -- 172.21.15.2:0/3034099182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5650107600 msgr2=0x7f5650101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:20.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.725+0000 7f5655555640 1 --2- 172.21.15.2:0/3034099182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5650107600 0x7f5650101190 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f564800e9b0 tx=0x7f564800ee70 comp rx=0 tx=0).stop 2024-03-15T13:06:20.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.725+0000 7f5655555640 1 -- 172.21.15.2:0/3034099182 shutdown_connections 2024-03-15T13:06:20.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.725+0000 7f5655555640 1 --2- 172.21.15.2:0/3034099182 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f56380a1c20 0x7f56380a40e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.725+0000 7f5655555640 1 --2- 172.21.15.2:0/3034099182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5650107600 0x7f5650101190 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.725+0000 7f5655555640 1 --2- 172.21.15.2:0/3034099182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f565006a4e0 0x7f5650100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.725+0000 7f5655555640 1 --2- 172.21.15.2:0/3034099182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5650069b20 0x7f5650100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.725+0000 7f5655555640 1 -- 172.21.15.2:0/3034099182 >> 172.21.15.2:0/3034099182 conn(0x7f5650077190 msgr2=0x7f56500ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:20.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.725+0000 7f5655555640 1 -- 172.21.15.2:0/3034099182 shutdown_connections 2024-03-15T13:06:20.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.725+0000 7f5655555640 1 -- 172.21.15.2:0/3034099182 wait complete. 2024-03-15T13:06:20.720 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:06:20.720 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:06:20.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.839+0000 7f97194f4640 1 -- 172.21.15.2:0/1390753370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97141003f0 msgr2=0x7f9714100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:20.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.839+0000 7f97194f4640 1 --2- 172.21.15.2:0/1390753370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97141003f0 0x7f9714100870 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f9708009930 tx=0x7f970802bc30 comp rx=0 tx=0).stop 2024-03-15T13:06:20.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.839+0000 7f97194f4640 1 -- 172.21.15.2:0/1390753370 shutdown_connections 2024-03-15T13:06:20.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.839+0000 7f97194f4640 1 --2- 172.21.15.2:0/1390753370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9714100db0 0x7f971410d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.839+0000 7f97194f4640 1 --2- 172.21.15.2:0/1390753370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97141003f0 0x7f9714100870 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.839+0000 7f97194f4640 1 --2- 172.21.15.2:0/1390753370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9714106490 0x7f9714106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.839+0000 7f97194f4640 1 -- 172.21.15.2:0/1390753370 >> 172.21.15.2:0/1390753370 conn(0x7f97140774f0 msgr2=0x7f9714077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:20.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.839+0000 7f97194f4640 1 -- 172.21.15.2:0/1390753370 shutdown_connections 2024-03-15T13:06:20.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.839+0000 7f97194f4640 1 -- 172.21.15.2:0/1390753370 wait complete. 2024-03-15T13:06:20.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.840+0000 7f97194f4640 1 Processor -- start 2024-03-15T13:06:20.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.840+0000 7f97194f4640 1 -- start start 2024-03-15T13:06:20.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.840+0000 7f97194f4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97141003f0 0x7f971419d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.840+0000 7f97194f4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9714100db0 0x7f971419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.840+0000 7f97194f4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9714106490 0x7f9714197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.840+0000 7f97194f4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9714197f60 con 0x7f9714106490 2024-03-15T13:06:20.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.840+0000 7f97194f4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97141980d0 con 0x7f9714100db0 2024-03-15T13:06:20.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.840+0000 7f97194f4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9714198290 con 0x7f97141003f0 2024-03-15T13:06:20.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.840+0000 7f97137fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9714106490 0x7f9714197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.840+0000 7f97127fc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9714100db0 0x7f971419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.840+0000 7f97137fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9714106490 0x7f9714197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33944/0 (socket says 172.21.15.2:33944) 2024-03-15T13:06:20.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.840+0000 7f9712ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97141003f0 0x7f971419d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.840+0000 7f97137fe640 1 -- 172.21.15.2:0/3696823561 learned_addr learned my addr 172.21.15.2:0/3696823561 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:20.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.841+0000 7f97137fe640 1 -- 172.21.15.2:0/3696823561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97141003f0 msgr2=0x7f971419d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:20.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.841+0000 7f97137fe640 1 --2- 172.21.15.2:0/3696823561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97141003f0 0x7f971419d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.841+0000 7f97137fe640 1 -- 172.21.15.2:0/3696823561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9714100db0 msgr2=0x7f971419dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:20.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.841+0000 7f97137fe640 1 --2- 172.21.15.2:0/3696823561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9714100db0 0x7f971419dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:20.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.841+0000 7f97137fe640 1 -- 172.21.15.2:0/3696823561 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9704009840 con 0x7f9714106490 2024-03-15T13:06:20.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.841+0000 7f97127fc640 1 --2- 172.21.15.2:0/3696823561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9714100db0 0x7f971419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:20.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.841+0000 7f9712ffd640 1 --2- 172.21.15.2:0/3696823561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97141003f0 0x7f971419d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:20.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.841+0000 7f97137fe640 1 --2- 172.21.15.2:0/3696823561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9714106490 0x7f9714197910 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f970c00e970 tx=0x7f970c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:20.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.841+0000 7f96fbfff640 1 -- 172.21.15.2:0/3696823561 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c009720 con 0x7f9714106490 2024-03-15T13:06:20.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.841+0000 7f96fbfff640 1 -- 172.21.15.2:0/3696823561 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f970c0054f0 con 0x7f9714106490 2024-03-15T13:06:20.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.841+0000 7f97194f4640 1 -- 172.21.15.2:0/3696823561 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9708009560 con 0x7f9714106490 2024-03-15T13:06:20.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.841+0000 7f96fbfff640 1 -- 172.21.15.2:0/3696823561 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c010430 con 0x7f9714106490 2024-03-15T13:06:20.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.841+0000 7f97194f4640 1 -- 172.21.15.2:0/3696823561 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97141a4620 con 0x7f9714106490 2024-03-15T13:06:20.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.842+0000 7f97194f4640 1 -- 172.21.15.2:0/3696823561 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9714101b30 con 0x7f9714106490 2024-03-15T13:06:20.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.843+0000 7f96fbfff640 1 -- 172.21.15.2:0/3696823561 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f970c004f70 con 0x7f9714106490 2024-03-15T13:06:20.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.844+0000 7f96fbfff640 1 --2- 172.21.15.2:0/3696823561 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f96fc0a1ba0 0x7f96fc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:20.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.844+0000 7f96fbfff640 1 -- 172.21.15.2:0/3696823561 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f970c014070 con 0x7f9714106490 2024-03-15T13:06:20.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.844+0000 7f9712ffd640 1 --2- 172.21.15.2:0/3696823561 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f96fc0a1ba0 0x7f96fc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:20.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.844+0000 7f9712ffd640 1 --2- 172.21.15.2:0/3696823561 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f96fc0a1ba0 0x7f96fc0a4060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9704009fd0 tx=0x7f9704009500 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:20.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:20.848+0000 7f96fbfff640 1 -- 172.21.15.2:0/3696823561 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f970c096ac0 con 0x7f9714106490 2024-03-15T13:06:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.073+0000 7f97194f4640 1 -- 172.21.15.2:0/3696823561 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97140682e0 con 0x7f9714106490 2024-03-15T13:06:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.074+0000 7f96fbfff640 1 -- 172.21.15.2:0/3696823561 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+11453 (secure 0 0 0) 0x7f970c09a180 con 0x7f9714106490 2024-03-15T13:06:21.059 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:06:21.059 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:05:12.138845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24500,24464,24460,24434,24539]},"id":1}]} 2024-03-15T13:06:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 50 2024-03-15T13:06:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.076+0000 7f97194f4640 1 -- 172.21.15.2:0/3696823561 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f96fc0a1ba0 msgr2=0x7f96fc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.076+0000 7f97194f4640 1 --2- 172.21.15.2:0/3696823561 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f96fc0a1ba0 0x7f96fc0a4060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9704009fd0 tx=0x7f9704009500 comp rx=0 tx=0).stop 2024-03-15T13:06:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.076+0000 7f97194f4640 1 -- 172.21.15.2:0/3696823561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9714106490 msgr2=0x7f9714197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.076+0000 7f97194f4640 1 --2- 172.21.15.2:0/3696823561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9714106490 0x7f9714197910 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f970c00e970 tx=0x7f970c00ed70 comp rx=0 tx=0).stop 2024-03-15T13:06:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.076+0000 7f97194f4640 1 -- 172.21.15.2:0/3696823561 shutdown_connections 2024-03-15T13:06:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.076+0000 7f97194f4640 1 --2- 172.21.15.2:0/3696823561 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f96fc0a1ba0 0x7f96fc0a4060 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.076+0000 7f97194f4640 1 --2- 172.21.15.2:0/3696823561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9714106490 0x7f9714197910 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.076+0000 7f97194f4640 1 --2- 172.21.15.2:0/3696823561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9714100db0 0x7f971419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.076+0000 7f97194f4640 1 --2- 172.21.15.2:0/3696823561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97141003f0 0x7f971419d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.076+0000 7f97194f4640 1 -- 172.21.15.2:0/3696823561 >> 172.21.15.2:0/3696823561 conn(0x7f97140774f0 msgr2=0x7f9714068930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.076+0000 7f97194f4640 1 -- 172.21.15.2:0/3696823561 shutdown_connections 2024-03-15T13:06:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.076+0000 7f97194f4640 1 -- 172.21.15.2:0/3696823561 wait complete. 2024-03-15T13:06:21.072 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:06:21.072 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:06:21.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.192+0000 7fba6f72c640 1 -- 172.21.15.2:0/1723265560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba68107560 msgr2=0x7fba68107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.192+0000 7fba6f72c640 1 --2- 172.21.15.2:0/1723265560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba68107560 0x7fba68107940 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fba58009860 tx=0x7fba5802bb70 comp rx=0 tx=0).stop 2024-03-15T13:06:21.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.192+0000 7fba6f72c640 1 -- 172.21.15.2:0/1723265560 shutdown_connections 2024-03-15T13:06:21.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.192+0000 7fba6f72c640 1 --2- 172.21.15.2:0/1723265560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba68101e60 0x7fba6810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.192+0000 7fba6f72c640 1 --2- 172.21.15.2:0/1723265560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba681014a0 0x7fba68101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.192+0000 7fba6f72c640 1 --2- 172.21.15.2:0/1723265560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba68107560 0x7fba68107940 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.192+0000 7fba6f72c640 1 -- 172.21.15.2:0/1723265560 >> 172.21.15.2:0/1723265560 conn(0x7fba680788f0 msgr2=0x7fba68078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.192+0000 7fba6f72c640 1 -- 172.21.15.2:0/1723265560 shutdown_connections 2024-03-15T13:06:21.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.192+0000 7fba6f72c640 1 -- 172.21.15.2:0/1723265560 wait complete. 2024-03-15T13:06:21.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.193+0000 7fba6f72c640 1 Processor -- start 2024-03-15T13:06:21.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.193+0000 7fba6f72c640 1 -- start start 2024-03-15T13:06:21.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.193+0000 7fba6f72c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba681014a0 0x7fba6819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.193+0000 7fba6f72c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba68101e60 0x7fba6819dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.193+0000 7fba6f72c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba68107560 0x7fba68197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.193+0000 7fba6f72c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba68197ee0 con 0x7fba68101e60 2024-03-15T13:06:21.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.193+0000 7fba6f72c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba68198050 con 0x7fba68107560 2024-03-15T13:06:21.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.193+0000 7fba6f72c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba68198210 con 0x7fba681014a0 2024-03-15T13:06:21.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.193+0000 7fba6ccc7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba68101e60 0x7fba6819dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.193+0000 7fba6ccc7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba68101e60 0x7fba6819dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33958/0 (socket says 172.21.15.2:33958) 2024-03-15T13:06:21.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.193+0000 7fba6ccc7640 1 -- 172.21.15.2:0/3011130937 learned_addr learned my addr 172.21.15.2:0/3011130937 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.193+0000 7fba6d4c8640 1 --2- 172.21.15.2:0/3011130937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba681014a0 0x7fba6819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.193+0000 7fba6dcc9640 1 --2- 172.21.15.2:0/3011130937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba68107560 0x7fba68197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.194+0000 7fba6ccc7640 1 -- 172.21.15.2:0/3011130937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba681014a0 msgr2=0x7fba6819d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.194+0000 7fba6ccc7640 1 --2- 172.21.15.2:0/3011130937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba681014a0 0x7fba6819d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.194+0000 7fba6ccc7640 1 -- 172.21.15.2:0/3011130937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba68107560 msgr2=0x7fba68197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.194+0000 7fba6ccc7640 1 --2- 172.21.15.2:0/3011130937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba68107560 0x7fba68197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.194+0000 7fba6ccc7640 1 -- 172.21.15.2:0/3011130937 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba58009490 con 0x7fba68101e60 2024-03-15T13:06:21.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.194+0000 7fba6ccc7640 1 --2- 172.21.15.2:0/3011130937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba68101e60 0x7fba6819dc20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fba6400e9f0 tx=0x7fba6400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.194+0000 7fba5e7fc640 1 -- 172.21.15.2:0/3011130937 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64009720 con 0x7fba68101e60 2024-03-15T13:06:21.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.194+0000 7fba6f72c640 1 -- 172.21.15.2:0/3011130937 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba681984f0 con 0x7fba68101e60 2024-03-15T13:06:21.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.194+0000 7fba5e7fc640 1 -- 172.21.15.2:0/3011130937 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fba6400cd80 con 0x7fba68101e60 2024-03-15T13:06:21.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.194+0000 7fba5e7fc640 1 -- 172.21.15.2:0/3011130937 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64010690 con 0x7fba68101e60 2024-03-15T13:06:21.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.194+0000 7fba6f72c640 1 -- 172.21.15.2:0/3011130937 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba681a4780 con 0x7fba68101e60 2024-03-15T13:06:21.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.197+0000 7fba5e7fc640 1 -- 172.21.15.2:0/3011130937 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fba64004e90 con 0x7fba68101e60 2024-03-15T13:06:21.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.197+0000 7fba723e7740 1 Processor -- start 2024-03-15T13:06:21.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.197+0000 7fba723e7740 1 -- start start 2024-03-15T13:06:21.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.197+0000 7fba723e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9209250 0x5637a9209630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.197+0000 7fba723e7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209b70 0x5637a9212210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.197+0000 7fba723e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9212750 0x5637a9214b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.197+0000 7fba723e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637a91c4960 con 0x5637a9209250 2024-03-15T13:06:21.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.197+0000 7fba723e7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637a9178310 con 0x5637a9209b70 2024-03-15T13:06:21.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.197+0000 7fba723e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637a9175840 con 0x5637a9212750 2024-03-15T13:06:21.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.197+0000 7fba6dcc9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9209250 0x5637a9209630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.197+0000 7fba5e7fc640 1 --2- 172.21.15.2:0/3011130937 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fba540a1ba0 0x7fba540a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.197+0000 7fba6dcc9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9209250 0x5637a9209630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33966/0 (socket says 172.21.15.2:33966) 2024-03-15T13:06:21.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.197+0000 7fba6dcc9640 1 -- 172.21.15.2:0/2784612651 learned_addr learned my addr 172.21.15.2:0/2784612651 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.197+0000 7fba6d4c8640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209b70 0x5637a9212210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.197+0000 7fba6ccc7640 1 --2- 172.21.15.2:0/2784612651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9212750 0x5637a9214b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.198+0000 7fba6dcc9640 1 --2- 172.21.15.2:0/3011130937 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fba540a1ba0 0x7fba540a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.198+0000 7fba5e7fc640 1 -- 172.21.15.2:0/3011130937 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fba64014070 con 0x7fba68101e60 2024-03-15T13:06:21.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.198+0000 7fba6d4c8640 1 -- 172.21.15.2:0/2784612651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9212750 msgr2=0x5637a9214b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.198+0000 7fba6d4c8640 1 --2- 172.21.15.2:0/2784612651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9212750 0x5637a9214b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.198+0000 7fba6d4c8640 1 -- 172.21.15.2:0/2784612651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9209250 msgr2=0x5637a9209630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.198+0000 7fba6d4c8640 1 --2- 172.21.15.2:0/2784612651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9209250 0x5637a9209630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.198+0000 7fba6d4c8640 1 -- 172.21.15.2:0/2784612651 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637a9133060 con 0x5637a9209b70 2024-03-15T13:06:21.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.198+0000 7fba6ccc7640 1 --2- 172.21.15.2:0/2784612651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9212750 0x5637a9214b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:06:21.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.198+0000 7fba6d4c8640 1 --2- 172.21.15.2:0/2784612651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209b70 0x5637a9212210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fba5802c180 tx=0x7fba580323c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fad0efb36b16958 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.198+0000 7fba6dcc9640 1 --2- 172.21.15.2:0/3011130937 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fba540a1ba0 0x7fba540a4060 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fba60005c40 tx=0x7fba60004ab0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.198+0000 7fba537fe640 1 -- 172.21.15.2:0/2784612651 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba58004030 con 0x5637a9209b70 2024-03-15T13:06:21.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.198+0000 7fba537fe640 1 -- 172.21.15.2:0/2784612651 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fba58034440 con 0x5637a9209b70 2024-03-15T13:06:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.198+0000 7fba537fe640 1 -- 172.21.15.2:0/2784612651 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5802c980 con 0x5637a9209b70 2024-03-15T13:06:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.199+0000 7fba723e7740 1 -- 172.21.15.2:0/2784612651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209b70 msgr2=0x5637a9212210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.199+0000 7fba723e7740 1 --2- 172.21.15.2:0/2784612651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209b70 0x5637a9212210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fba5802c180 tx=0x7fba580323c0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.199+0000 7fba723e7740 1 -- 172.21.15.2:0/2784612651 shutdown_connections 2024-03-15T13:06:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.199+0000 7fba723e7740 1 --2- 172.21.15.2:0/2784612651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9212750 0x5637a9214b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.199+0000 7fba723e7740 1 --2- 172.21.15.2:0/2784612651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209b70 0x5637a9212210 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.199+0000 7fba723e7740 1 --2- 172.21.15.2:0/2784612651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9209250 0x5637a9209630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.199+0000 7fba723e7740 1 -- 172.21.15.2:0/2784612651 >> 172.21.15.2:0/2784612651 conn(0x5637a9203980 msgr2=0x5637a9203d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.199+0000 7fba723e7740 1 -- 172.21.15.2:0/2784612651 shutdown_connections 2024-03-15T13:06:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.199+0000 7fba723e7740 1 -- 172.21.15.2:0/2784612651 wait complete. 2024-03-15T13:06:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.199+0000 7fba723e7740 1 -- start start 2024-03-15T13:06:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.199+0000 7fba723e7740 1 Processor -- start 2024-03-15T13:06:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba723e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9209250 0x5637a9212040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba723e7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209b70 0x5637a920f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba723e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9212750 0x5637a920f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba723e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637a9171020 con 0x5637a9212750 2024-03-15T13:06:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba723e7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637a8e44430 con 0x5637a9209b70 2024-03-15T13:06:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba723e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637a8e00e00 con 0x5637a9209250 2024-03-15T13:06:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba723e7740 10 client.0 _ll_register_callbacks cb 0x5637a9118cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba6dcc9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9212750 0x5637a920f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba6ccc7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209b70 0x5637a920f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba6dcc9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9212750 0x5637a920f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33980/0 (socket says 172.21.15.2:33980) 2024-03-15T13:06:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba6d4c8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9209250 0x5637a9212040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba6dcc9640 1 -- 172.21.15.2:0/76368977 learned_addr learned my addr 172.21.15.2:0/76368977 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba6dcc9640 1 -- 172.21.15.2:0/76368977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9209250 msgr2=0x5637a9212040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba6dcc9640 1 --2- 172.21.15.2:0/76368977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9209250 0x5637a9212040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba6dcc9640 1 -- 172.21.15.2:0/76368977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209b70 msgr2=0x5637a920f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba6dcc9640 1 --2- 172.21.15.2:0/76368977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209b70 0x5637a920f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba6dcc9640 1 -- 172.21.15.2:0/76368977 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6810ec20 con 0x5637a9212750 2024-03-15T13:06:21.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba6ccc7640 1 --2- 172.21.15.2:0/76368977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209b70 0x5637a920f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:21.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba6dcc9640 1 --2- 172.21.15.2:0/76368977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9212750 0x5637a920f910 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fba6000e990 tx=0x7fba60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7782de6a83d9539 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:21.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba50ff9640 1 -- 172.21.15.2:0/76368977 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60023070 con 0x5637a9212750 2024-03-15T13:06:21.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.200+0000 7fba50ff9640 1 -- 172.21.15.2:0/76368977 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fba6001eab0 con 0x5637a9212750 2024-03-15T13:06:21.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.201+0000 7fba723e7740 1 -- 172.21.15.2:0/76368977 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637a8e707e0 con 0x5637a9212750 2024-03-15T13:06:21.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.201+0000 7fba50ff9640 1 -- 172.21.15.2:0/76368977 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba6001ec70 con 0x5637a9212750 2024-03-15T13:06:21.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.201+0000 7fba723e7740 10 client.17811 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:21.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.201+0000 7fba723e7740 1 -- 172.21.15.2:0/76368977 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5637a916f6a0 con 0x5637a9212750 2024-03-15T13:06:21.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.201+0000 7fba50ff9640 1 -- 172.21.15.2:0/76368977 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5637a8e707e0 con 0x5637a9212750 2024-03-15T13:06:21.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.201+0000 7fba50ff9640 1 -- 172.21.15.2:0/76368977 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fba600044d0 con 0x5637a9212750 2024-03-15T13:06:21.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.201+0000 7fba723e7740 10 client.17811 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:21.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.201+0000 7fba723e7740 10 client.17811 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:06:21.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.201+0000 7fba723e7740 20 client.17811 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:21.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.201+0000 7fba723e7740 1 --2- 172.21.15.2:0/76368977 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5637a9222860 0x5637a9224c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.201+0000 7fba723e7740 4 client.17811 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:21.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.201+0000 7fba723e7740 1 -- 172.21.15.2:0/76368977 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637a8ead720 con 0x5637a9222860 2024-03-15T13:06:21.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.201+0000 7fba6d4c8640 1 --2- 172.21.15.2:0/76368977 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5637a9222860 0x5637a9224c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.202+0000 7fba50ff9640 10 client.17811 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:06:21.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.202+0000 7fba6d4c8640 1 --2- 172.21.15.2:0/76368977 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5637a9222860 0x5637a9224c50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cecdda619e44694d server_cookie=5466707dd12e5b98 in_seq=0 out_seq=0 2024-03-15T13:06:21.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.202+0000 7fba50ff9640 1 -- 172.21.15.2:0/76368977 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5637a8ead720 con 0x5637a9222860 2024-03-15T13:06:21.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.202+0000 7fba50ff9640 10 client.17811 handle_command_reply: tid=0 2024-03-15T13:06:21.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.202+0000 7fba723e7740 1 client.17811 shutdown 2024-03-15T13:06:21.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.202+0000 7fba723e7740 1 -- 172.21.15.2:0/76368977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9212750 msgr2=0x5637a920f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.202+0000 7fba723e7740 1 --2- 172.21.15.2:0/76368977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9212750 0x5637a920f910 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fba6000e990 tx=0x7fba60004a40 comp rx=0 tx=0).stop 2024-03-15T13:06:21.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.203+0000 7fba723e7740 1 -- 172.21.15.2:0/76368977 shutdown_connections 2024-03-15T13:06:21.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.203+0000 7fba723e7740 1 --2- 172.21.15.2:0/76368977 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5637a9222860 0x5637a9224c50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.203+0000 7fba723e7740 1 --2- 172.21.15.2:0/76368977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9212750 0x5637a920f910 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.203+0000 7fba723e7740 1 --2- 172.21.15.2:0/76368977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209b70 0x5637a920f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.203+0000 7fba723e7740 1 --2- 172.21.15.2:0/76368977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9209250 0x5637a9212040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.203+0000 7fba723e7740 1 -- 172.21.15.2:0/76368977 >> 172.21.15.2:0/76368977 conn(0x5637a9203980 msgr2=0x5637a92056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.203+0000 7fba50ff9640 0 client.17811 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:06:21.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.203+0000 7fba723e7740 1 -- 172.21.15.2:0/76368977 shutdown_connections 2024-03-15T13:06:21.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.203+0000 7fba723e7740 1 -- 172.21.15.2:0/76368977 wait complete. 2024-03-15T13:06:21.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.203+0000 7fba723e7740 20 client.17811 trim_cache size 0 max 16384 2024-03-15T13:06:21.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.215+0000 7fba723e7740 1 Processor -- start 2024-03-15T13:06:21.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.215+0000 7fba723e7740 1 -- start start 2024-03-15T13:06:21.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.215+0000 7fba723e7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209250 0x5637a920ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.215+0000 7fba723e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9209b70 0x5637a9211b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.215+0000 7fba723e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9212750 0x5637a9212040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.215+0000 7fba723e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637a90f6130 con 0x5637a9209b70 2024-03-15T13:06:21.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.215+0000 7fba723e7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637a9204550 con 0x5637a9209250 2024-03-15T13:06:21.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.215+0000 7fba723e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637a9204980 con 0x5637a9212750 2024-03-15T13:06:21.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.215+0000 7fba6ccc7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9209b70 0x5637a9211b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.215+0000 7fba6ccc7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9209b70 0x5637a9211b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33990/0 (socket says 172.21.15.2:33990) 2024-03-15T13:06:21.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.215+0000 7fba6ccc7640 1 -- 172.21.15.2:0/1918380588 learned_addr learned my addr 172.21.15.2:0/1918380588 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.215+0000 7fba6d4c8640 1 --2- 172.21.15.2:0/1918380588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209250 0x5637a920ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.215+0000 7fba6dcc9640 1 --2- 172.21.15.2:0/1918380588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9212750 0x5637a9212040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba6ccc7640 1 -- 172.21.15.2:0/1918380588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9212750 msgr2=0x5637a9212040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba6ccc7640 1 --2- 172.21.15.2:0/1918380588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9212750 0x5637a9212040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba6ccc7640 1 -- 172.21.15.2:0/1918380588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209250 msgr2=0x5637a920ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba6ccc7640 1 --2- 172.21.15.2:0/1918380588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209250 0x5637a920ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba6ccc7640 1 -- 172.21.15.2:0/1918380588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637a910bc20 con 0x5637a9209b70 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba6d4c8640 1 --2- 172.21.15.2:0/1918380588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209250 0x5637a920ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba6ccc7640 1 --2- 172.21.15.2:0/1918380588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9209b70 0x5637a9211b00 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fba6400e9c0 tx=0x7fba6406daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b05aa3415df3de server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba52ffd640 1 -- 172.21.15.2:0/1918380588 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6409ae80 con 0x5637a9209b70 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba52ffd640 1 -- 172.21.15.2:0/1918380588 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fba640cf070 con 0x5637a9209b70 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba52ffd640 1 -- 172.21.15.2:0/1918380588 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba640a2a90 con 0x5637a9209b70 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba723e7740 1 -- 172.21.15.2:0/1918380588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9209b70 msgr2=0x5637a9211b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba723e7740 1 --2- 172.21.15.2:0/1918380588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9209b70 0x5637a9211b00 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fba6400e9c0 tx=0x7fba6406daf0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba723e7740 1 -- 172.21.15.2:0/1918380588 shutdown_connections 2024-03-15T13:06:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba723e7740 1 --2- 172.21.15.2:0/1918380588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9212750 0x5637a9212040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba723e7740 1 --2- 172.21.15.2:0/1918380588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9209b70 0x5637a9211b00 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba723e7740 1 --2- 172.21.15.2:0/1918380588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209250 0x5637a920ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.216+0000 7fba723e7740 1 -- 172.21.15.2:0/1918380588 >> 172.21.15.2:0/1918380588 conn(0x5637a9203980 msgr2=0x5637a920f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba723e7740 1 -- 172.21.15.2:0/1918380588 shutdown_connections 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba723e7740 1 -- 172.21.15.2:0/1918380588 wait complete. 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba723e7740 1 -- start start 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba723e7740 1 Processor -- start 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba723e7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209250 0x5637a921a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba723e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9209b70 0x5637a92165b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba723e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9212750 0x5637a9216af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba723e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637a92170b0 con 0x5637a9212750 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba723e7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637a9217220 con 0x5637a9209250 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba723e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637a92173e0 con 0x5637a9209b70 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba723e7740 10 client.0 _ll_register_callbacks cb 0x7fba6000e510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba6dcc9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9212750 0x5637a9216af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba6d4c8640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209250 0x5637a921a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba6dcc9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9212750 0x5637a9216af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33996/0 (socket says 172.21.15.2:33996) 2024-03-15T13:06:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba6ccc7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9209b70 0x5637a92165b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba6d4c8640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209250 0x5637a921a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:54484/0 (socket says 172.21.15.2:54484) 2024-03-15T13:06:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.217+0000 7fba6dcc9640 1 -- 172.21.15.2:0/4293249400 learned_addr learned my addr 172.21.15.2:0/4293249400 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba6dcc9640 1 -- 172.21.15.2:0/4293249400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9209b70 msgr2=0x5637a92165b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba6dcc9640 1 --2- 172.21.15.2:0/4293249400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9209b70 0x5637a92165b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba6dcc9640 1 -- 172.21.15.2:0/4293249400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209250 msgr2=0x5637a921a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba6dcc9640 1 --2- 172.21.15.2:0/4293249400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209250 0x5637a921a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba6dcc9640 1 -- 172.21.15.2:0/4293249400 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637a9133060 con 0x5637a9212750 2024-03-15T13:06:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba6ccc7640 1 --2- 172.21.15.2:0/4293249400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9209b70 0x5637a92165b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba6d4c8640 1 --2- 172.21.15.2:0/4293249400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209250 0x5637a921a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba6dcc9640 1 --2- 172.21.15.2:0/4293249400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9212750 0x5637a9216af0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fba6000d7b0 tx=0x7fba60028c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2041bc09f639dba1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba50ff9640 1 -- 172.21.15.2:0/4293249400 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60023050 con 0x5637a9212750 2024-03-15T13:06:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba50ff9640 1 -- 172.21.15.2:0/4293249400 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fba60004b20 con 0x5637a9212750 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba723e7740 1 -- 172.21.15.2:0/4293249400 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba58034440 con 0x5637a9212750 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba50ff9640 1 -- 172.21.15.2:0/4293249400 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba6000db90 con 0x5637a9212750 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba723e7740 10 client.17817 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba723e7740 1 -- 172.21.15.2:0/4293249400 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7fba681a4780 con 0x5637a9212750 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.218+0000 7fba50ff9640 1 -- 172.21.15.2:0/4293249400 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fba58034440 con 0x5637a9212750 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba50ff9640 1 -- 172.21.15.2:0/4293249400 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fba600054f0 con 0x5637a9212750 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba723e7740 10 client.17817 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba723e7740 10 client.17817 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba723e7740 20 client.17817 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba723e7740 1 --2- 172.21.15.2:0/4293249400 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5637a927f700 0x5637a9281b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba723e7740 4 client.17817 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba723e7740 1 -- 172.21.15.2:0/4293249400 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5637a8efc890 con 0x5637a927f700 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba6d4c8640 1 --2- 172.21.15.2:0/4293249400 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5637a927f700 0x5637a9281b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba6d4c8640 1 --2- 172.21.15.2:0/4293249400 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5637a927f700 0x5637a9281b20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=35bfb038a0969443 server_cookie=9b14d4d87cf4c7f in_seq=0 out_seq=0 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba50ff9640 10 client.17817 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:06:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba50ff9640 1 -- 172.21.15.2:0/4293249400 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5637a8efc890 con 0x5637a927f700 2024-03-15T13:06:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba50ff9640 10 client.17817 handle_command_reply: tid=0 2024-03-15T13:06:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba723e7740 1 client.17817 shutdown 2024-03-15T13:06:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba723e7740 1 -- 172.21.15.2:0/4293249400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9212750 msgr2=0x5637a9216af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba723e7740 1 --2- 172.21.15.2:0/4293249400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9212750 0x5637a9216af0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fba6000d7b0 tx=0x7fba60028c60 comp rx=0 tx=0).stop 2024-03-15T13:06:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba723e7740 1 -- 172.21.15.2:0/4293249400 shutdown_connections 2024-03-15T13:06:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba723e7740 1 --2- 172.21.15.2:0/4293249400 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x5637a927f700 0x5637a9281b20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba723e7740 1 --2- 172.21.15.2:0/4293249400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5637a9212750 0x5637a9216af0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba723e7740 1 --2- 172.21.15.2:0/4293249400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5637a9209b70 0x5637a92165b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba50ff9640 0 client.17817 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:06:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba723e7740 1 --2- 172.21.15.2:0/4293249400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637a9209250 0x5637a921a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.219+0000 7fba723e7740 1 -- 172.21.15.2:0/4293249400 >> 172.21.15.2:0/4293249400 conn(0x5637a9203980 msgr2=0x5637a9204b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.220+0000 7fba723e7740 1 -- 172.21.15.2:0/4293249400 shutdown_connections 2024-03-15T13:06:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.220+0000 7fba723e7740 1 -- 172.21.15.2:0/4293249400 wait complete. 2024-03-15T13:06:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.220+0000 7fba723e7740 20 client.17817 trim_cache size 0 max 16384 2024-03-15T13:06:21.212 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:06:21.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.220+0000 7fba6f72c640 1 -- 172.21.15.2:0/3011130937 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fba540a1ba0 msgr2=0x7fba540a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.220+0000 7fba6f72c640 1 --2- 172.21.15.2:0/3011130937 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fba540a1ba0 0x7fba540a4060 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fba60005c40 tx=0x7fba60004ab0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.220+0000 7fba6f72c640 1 -- 172.21.15.2:0/3011130937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba68101e60 msgr2=0x7fba6819dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.220+0000 7fba6f72c640 1 --2- 172.21.15.2:0/3011130937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba68101e60 0x7fba6819dc20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fba6400e9f0 tx=0x7fba6400edf0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.220+0000 7fba6f72c640 1 -- 172.21.15.2:0/3011130937 shutdown_connections 2024-03-15T13:06:21.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.220+0000 7fba6f72c640 1 --2- 172.21.15.2:0/3011130937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba68107560 0x7fba68197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.220+0000 7fba6f72c640 1 --2- 172.21.15.2:0/3011130937 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fba540a1ba0 0x7fba540a4060 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.220+0000 7fba6f72c640 1 --2- 172.21.15.2:0/3011130937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba68101e60 0x7fba6819dc20 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.220+0000 7fba6f72c640 1 --2- 172.21.15.2:0/3011130937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba681014a0 0x7fba6819d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.220+0000 7fba6f72c640 1 -- 172.21.15.2:0/3011130937 >> 172.21.15.2:0/3011130937 conn(0x7fba680788f0 msgr2=0x7fba680fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.220+0000 7fba6f72c640 1 -- 172.21.15.2:0/3011130937 shutdown_connections 2024-03-15T13:06:21.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.220+0000 7fba6f72c640 1 -- 172.21.15.2:0/3011130937 wait complete. 2024-03-15T13:06:21.217 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:06:21.217 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:06:21.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.336+0000 7fa303948640 1 -- 172.21.15.2:0/79848940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2fc0690b0 msgr2=0x7fa2fc10e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.336+0000 7fa303948640 1 --2- 172.21.15.2:0/79848940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2fc0690b0 0x7fa2fc10e320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009930 tx=0x7fa2ec02bc30 comp rx=0 tx=0).stop 2024-03-15T13:06:21.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa303948640 1 -- 172.21.15.2:0/79848940 shutdown_connections 2024-03-15T13:06:21.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa303948640 1 --2- 172.21.15.2:0/79848940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2fc0690b0 0x7fa2fc10e320 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa303948640 1 --2- 172.21.15.2:0/79848940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc0686f0 0x7fa2fc068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa303948640 1 --2- 172.21.15.2:0/79848940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2fc1074f0 0x7fa2fc1078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa303948640 1 -- 172.21.15.2:0/79848940 >> 172.21.15.2:0/79848940 conn(0x7fa2fc0774f0 msgr2=0x7fa2fc077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa303948640 1 -- 172.21.15.2:0/79848940 shutdown_connections 2024-03-15T13:06:21.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa303948640 1 -- 172.21.15.2:0/79848940 wait complete. 2024-03-15T13:06:21.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa303948640 1 Processor -- start 2024-03-15T13:06:21.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa303948640 1 -- start start 2024-03-15T13:06:21.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa303948640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2fc0686f0 0x7fa2fc19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa303948640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc0690b0 0x7fa2fc19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa303948640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2fc1074f0 0x7fa2fc1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa303948640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc197f40 con 0x7fa2fc0690b0 2024-03-15T13:06:21.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa303948640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc1980b0 con 0x7fa2fc1074f0 2024-03-15T13:06:21.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa303948640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc198270 con 0x7fa2fc0686f0 2024-03-15T13:06:21.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa300ee3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc0690b0 0x7fa2fc19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa300ee3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc0690b0 0x7fa2fc19dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34014/0 (socket says 172.21.15.2:34014) 2024-03-15T13:06:21.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.337+0000 7fa300ee3640 1 -- 172.21.15.2:0/3129221621 learned_addr learned my addr 172.21.15.2:0/3129221621 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.338+0000 7fa301ee5640 1 --2- 172.21.15.2:0/3129221621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2fc1074f0 0x7fa2fc1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.338+0000 7fa3016e4640 1 --2- 172.21.15.2:0/3129221621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2fc0686f0 0x7fa2fc19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.338+0000 7fa300ee3640 1 -- 172.21.15.2:0/3129221621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2fc0686f0 msgr2=0x7fa2fc19d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.338+0000 7fa300ee3640 1 --2- 172.21.15.2:0/3129221621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2fc0686f0 0x7fa2fc19d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.338+0000 7fa300ee3640 1 -- 172.21.15.2:0/3129221621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2fc1074f0 msgr2=0x7fa2fc1978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.338+0000 7fa300ee3640 1 --2- 172.21.15.2:0/3129221621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2fc1074f0 0x7fa2fc1978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.338+0000 7fa300ee3640 1 -- 172.21.15.2:0/3129221621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f0009490 con 0x7fa2fc0690b0 2024-03-15T13:06:21.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.338+0000 7fa300ee3640 1 --2- 172.21.15.2:0/3129221621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc0690b0 0x7fa2fc19dc80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000ea60 tx=0x7fa2f000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.338+0000 7fa2fa7fc640 1 -- 172.21.15.2:0/3129221621 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f000cde0 con 0x7fa2fc0690b0 2024-03-15T13:06:21.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.338+0000 7fa2fa7fc640 1 -- 172.21.15.2:0/3129221621 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa2f00035f0 con 0x7fa2fc0690b0 2024-03-15T13:06:21.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.338+0000 7fa303948640 1 -- 172.21.15.2:0/3129221621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ec009560 con 0x7fa2fc0690b0 2024-03-15T13:06:21.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.338+0000 7fa2fa7fc640 1 -- 172.21.15.2:0/3129221621 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0010690 con 0x7fa2fc0690b0 2024-03-15T13:06:21.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.338+0000 7fa303948640 1 -- 172.21.15.2:0/3129221621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2fc1a4600 con 0x7fa2fc0690b0 2024-03-15T13:06:21.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.340+0000 7fa2fa7fc640 1 -- 172.21.15.2:0/3129221621 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fa2f0003760 con 0x7fa2fc0690b0 2024-03-15T13:06:21.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa306603740 1 Processor -- start 2024-03-15T13:06:21.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa306603740 1 -- start start 2024-03-15T13:06:21.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa306603740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6ca250 0x5593fd6ca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa306603740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6cab70 0x5593fd6d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa306603740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6d3750 0x5593fd6d5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa306603740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fd685960 con 0x5593fd6cab70 2024-03-15T13:06:21.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa306603740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fd639310 con 0x5593fd6d3750 2024-03-15T13:06:21.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa306603740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fd636840 con 0x5593fd6ca250 2024-03-15T13:06:21.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa2fa7fc640 1 --2- 172.21.15.2:0/3129221621 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2e40a1ba0 0x7fa2e40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa3016e4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6cab70 0x5593fd6d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa301ee5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6ca250 0x5593fd6ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa3016e4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6cab70 0x5593fd6d3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34026/0 (socket says 172.21.15.2:34026) 2024-03-15T13:06:21.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa301ee5640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6ca250 0x5593fd6ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:51766/0 (socket says 172.21.15.2:51766) 2024-03-15T13:06:21.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa3016e4640 1 -- 172.21.15.2:0/2525896259 learned_addr learned my addr 172.21.15.2:0/2525896259 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa301ee5640 1 --2- 172.21.15.2:0/3129221621 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2e40a1ba0 0x7fa2e40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa300ee3640 1 --2- 172.21.15.2:0/2525896259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6d3750 0x5593fd6d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.341+0000 7fa2fa7fc640 1 -- 172.21.15.2:0/3129221621 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa2f0014070 con 0x7fa2fc0690b0 2024-03-15T13:06:21.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.342+0000 7fa3016e4640 1 -- 172.21.15.2:0/2525896259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6ca250 msgr2=0x5593fd6ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.342+0000 7fa3016e4640 1 --2- 172.21.15.2:0/2525896259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6ca250 0x5593fd6ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.342+0000 7fa3016e4640 1 -- 172.21.15.2:0/2525896259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6d3750 msgr2=0x5593fd6d5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.342+0000 7fa3016e4640 1 --2- 172.21.15.2:0/2525896259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6d3750 0x5593fd6d5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.342+0000 7fa3016e4640 1 -- 172.21.15.2:0/2525896259 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593fd5f4060 con 0x5593fd6cab70 2024-03-15T13:06:21.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.342+0000 7fa301ee5640 1 --2- 172.21.15.2:0/2525896259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6ca250 0x5593fd6ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:21.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.342+0000 7fa3016e4640 1 --2- 172.21.15.2:0/2525896259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6cab70 0x5593fd6d3210 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009fb0 tx=0x7fa2f40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab85e93d1b125fc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.342+0000 7fa301ee5640 1 --2- 172.21.15.2:0/3129221621 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2e40a1ba0 0x7fa2e40a4060 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec005940 tx=0x7fa2ec0058d0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.342+0000 7fa2e37fe640 1 -- 172.21.15.2:0/2525896259 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4004d20 con 0x5593fd6cab70 2024-03-15T13:06:21.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.342+0000 7fa2e37fe640 1 -- 172.21.15.2:0/2525896259 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa2f4004e80 con 0x5593fd6cab70 2024-03-15T13:06:21.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.342+0000 7fa2e37fe640 1 -- 172.21.15.2:0/2525896259 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4005220 con 0x5593fd6cab70 2024-03-15T13:06:21.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 -- 172.21.15.2:0/2525896259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6cab70 msgr2=0x5593fd6d3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 --2- 172.21.15.2:0/2525896259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6cab70 0x5593fd6d3210 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009fb0 tx=0x7fa2f40118d0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 -- 172.21.15.2:0/2525896259 shutdown_connections 2024-03-15T13:06:21.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 --2- 172.21.15.2:0/2525896259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6d3750 0x5593fd6d5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 --2- 172.21.15.2:0/2525896259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6cab70 0x5593fd6d3210 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 --2- 172.21.15.2:0/2525896259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6ca250 0x5593fd6ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 -- 172.21.15.2:0/2525896259 >> 172.21.15.2:0/2525896259 conn(0x5593fd6c4980 msgr2=0x5593fd6c4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 -- 172.21.15.2:0/2525896259 shutdown_connections 2024-03-15T13:06:21.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 -- 172.21.15.2:0/2525896259 wait complete. 2024-03-15T13:06:21.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 -- start start 2024-03-15T13:06:21.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 Processor -- start 2024-03-15T13:06:21.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6ca250 0x5593fd6d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6cab70 0x5593fd6d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6d3750 0x5593fd6d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fd632020 con 0x5593fd6cab70 2024-03-15T13:06:21.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fd305430 con 0x5593fd6d3750 2024-03-15T13:06:21.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.343+0000 7fa306603740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fd2c1e00 con 0x5593fd6ca250 2024-03-15T13:06:21.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa306603740 10 client.0 _ll_register_callbacks cb 0x5593fd5d9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:21.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa300ee3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6cab70 0x5593fd6d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa3016e4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6ca250 0x5593fd6d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa301ee5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6d3750 0x5593fd6d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa300ee3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6cab70 0x5593fd6d0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34028/0 (socket says 172.21.15.2:34028) 2024-03-15T13:06:21.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa300ee3640 1 -- 172.21.15.2:0/2746297155 learned_addr learned my addr 172.21.15.2:0/2746297155 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa300ee3640 1 -- 172.21.15.2:0/2746297155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6ca250 msgr2=0x5593fd6d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa300ee3640 1 --2- 172.21.15.2:0/2746297155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6ca250 0x5593fd6d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa300ee3640 1 -- 172.21.15.2:0/2746297155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6d3750 msgr2=0x5593fd6d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa300ee3640 1 --2- 172.21.15.2:0/2746297155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6d3750 0x5593fd6d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa300ee3640 1 -- 172.21.15.2:0/2746297155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f4009be0 con 0x5593fd6cab70 2024-03-15T13:06:21.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa300ee3640 1 --2- 172.21.15.2:0/2746297155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6cab70 0x5593fd6d0430 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0003f10 tx=0x7fa2f0015fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5cbcc1d1693938 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa2e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:21.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa2e0ff9640 1 -- 172.21.15.2:0/2746297155 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0003b60 con 0x5593fd6cab70 2024-03-15T13:06:21.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa2e0ff9640 1 -- 172.21.15.2:0/2746297155 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa2f009af70 con 0x5593fd6cab70 2024-03-15T13:06:21.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa306603740 1 -- 172.21.15.2:0/2746297155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593fd36e720 con 0x5593fd6cab70 2024-03-15T13:06:21.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa2e0ff9640 1 -- 172.21.15.2:0/2746297155 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2f009e9c0 con 0x5593fd6cab70 2024-03-15T13:06:21.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa306603740 10 client.17829 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:21.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa306603740 1 -- 172.21.15.2:0/2746297155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5593fd6306a0 con 0x5593fd6cab70 2024-03-15T13:06:21.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.344+0000 7fa2e0ff9640 1 -- 172.21.15.2:0/2746297155 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5593fd36e720 con 0x5593fd6cab70 2024-03-15T13:06:21.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.345+0000 7fa2e0ff9640 1 -- 172.21.15.2:0/2746297155 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa2f00b3200 con 0x5593fd6cab70 2024-03-15T13:06:21.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.345+0000 7fa306603740 10 client.17829 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:21.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.345+0000 7fa306603740 10 client.17829 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:06:21.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.345+0000 7fa306603740 20 client.17829 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:21.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.345+0000 7fa306603740 1 --2- 172.21.15.2:0/2746297155 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5593fd6e37d0 0x5593fd6e5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.345+0000 7fa306603740 4 client.17829 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:21.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.345+0000 7fa306603740 1 -- 172.21.15.2:0/2746297155 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593fd3bd890 con 0x5593fd6e37d0 2024-03-15T13:06:21.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.345+0000 7fa3016e4640 1 --2- 172.21.15.2:0/2746297155 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5593fd6e37d0 0x5593fd6e5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.345+0000 7fa2e0ff9640 10 client.17829 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:06:21.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.345+0000 7fa3016e4640 1 --2- 172.21.15.2:0/2746297155 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5593fd6e37d0 0x5593fd6e5bc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=42540f3f301aefd8 server_cookie=1afdb4fea0f52c0d in_seq=0 out_seq=0 2024-03-15T13:06:21.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.346+0000 7fa2e0ff9640 1 -- 172.21.15.2:0/2746297155 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5593fd3bd890 con 0x5593fd6e37d0 2024-03-15T13:06:21.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.346+0000 7fa2e0ff9640 10 client.17829 handle_command_reply: tid=0 2024-03-15T13:06:21.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.346+0000 7fa306603740 1 client.17829 shutdown 2024-03-15T13:06:21.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.346+0000 7fa306603740 1 -- 172.21.15.2:0/2746297155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6cab70 msgr2=0x5593fd6d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.346+0000 7fa306603740 1 --2- 172.21.15.2:0/2746297155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6cab70 0x5593fd6d0430 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0003f10 tx=0x7fa2f0015fc0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.346+0000 7fa306603740 1 -- 172.21.15.2:0/2746297155 shutdown_connections 2024-03-15T13:06:21.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.346+0000 7fa306603740 1 --2- 172.21.15.2:0/2746297155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6d3750 0x5593fd6d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.346+0000 7fa306603740 1 --2- 172.21.15.2:0/2746297155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6cab70 0x5593fd6d0430 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.346+0000 7fa306603740 1 --2- 172.21.15.2:0/2746297155 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5593fd6e37d0 0x5593fd6e5bc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.346+0000 7fa306603740 1 --2- 172.21.15.2:0/2746297155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6ca250 0x5593fd6d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.346+0000 7fa2e0ff9640 0 client.17829 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:06:21.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.346+0000 7fa306603740 1 -- 172.21.15.2:0/2746297155 >> 172.21.15.2:0/2746297155 conn(0x5593fd6c4980 msgr2=0x5593fd6c66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.346+0000 7fa306603740 1 -- 172.21.15.2:0/2746297155 shutdown_connections 2024-03-15T13:06:21.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.346+0000 7fa306603740 1 -- 172.21.15.2:0/2746297155 wait complete. 2024-03-15T13:06:21.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.346+0000 7fa306603740 20 client.17829 trim_cache size 0 max 16384 2024-03-15T13:06:21.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa306603740 1 Processor -- start 2024-03-15T13:06:21.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa306603740 1 -- start start 2024-03-15T13:06:21.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa306603740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6ca250 0x5593fd6c6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa306603740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6cab70 0x5593fd6d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa306603740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6d3750 0x5593fd6d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa306603740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fd5b7130 con 0x5593fd6ca250 2024-03-15T13:06:21.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa306603740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fd6b6a40 con 0x5593fd6cab70 2024-03-15T13:06:21.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa306603740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fd6c55a0 con 0x5593fd6d3750 2024-03-15T13:06:21.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa3016e4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6ca250 0x5593fd6c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa3016e4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6ca250 0x5593fd6c6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34034/0 (socket says 172.21.15.2:34034) 2024-03-15T13:06:21.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa3016e4640 1 -- 172.21.15.2:0/720308032 learned_addr learned my addr 172.21.15.2:0/720308032 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa300ee3640 1 --2- 172.21.15.2:0/720308032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6cab70 0x5593fd6d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa301ee5640 1 --2- 172.21.15.2:0/720308032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6d3750 0x5593fd6d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa3016e4640 1 -- 172.21.15.2:0/720308032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6d3750 msgr2=0x5593fd6d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa3016e4640 1 --2- 172.21.15.2:0/720308032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6d3750 0x5593fd6d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa3016e4640 1 -- 172.21.15.2:0/720308032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6cab70 msgr2=0x5593fd6d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa3016e4640 1 --2- 172.21.15.2:0/720308032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6cab70 0x5593fd6d2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa3016e4640 1 -- 172.21.15.2:0/720308032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593fd5ccc20 con 0x5593fd6ca250 2024-03-15T13:06:21.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.359+0000 7fa3016e4640 1 --2- 172.21.15.2:0/720308032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6ca250 0x5593fd6c6880 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009fb0 tx=0x7fa2f4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb5356aa6b8a809 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.360+0000 7fa2e2ffd640 1 -- 172.21.15.2:0/720308032 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4028040 con 0x5593fd6ca250 2024-03-15T13:06:21.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.360+0000 7fa2e2ffd640 1 -- 172.21.15.2:0/720308032 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa2f4011e00 con 0x5593fd6ca250 2024-03-15T13:06:21.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.360+0000 7fa2e2ffd640 1 -- 172.21.15.2:0/720308032 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4003ff0 con 0x5593fd6ca250 2024-03-15T13:06:21.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.360+0000 7fa306603740 1 -- 172.21.15.2:0/720308032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6ca250 msgr2=0x5593fd6c6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.360+0000 7fa306603740 1 --2- 172.21.15.2:0/720308032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6ca250 0x5593fd6c6880 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009fb0 tx=0x7fa2f4005bd0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.360+0000 7fa306603740 1 -- 172.21.15.2:0/720308032 shutdown_connections 2024-03-15T13:06:21.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.360+0000 7fa306603740 1 --2- 172.21.15.2:0/720308032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6d3750 0x5593fd6d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.360+0000 7fa306603740 1 --2- 172.21.15.2:0/720308032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6cab70 0x5593fd6d2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.360+0000 7fa306603740 1 --2- 172.21.15.2:0/720308032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6ca250 0x5593fd6c6880 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.360+0000 7fa306603740 1 -- 172.21.15.2:0/720308032 >> 172.21.15.2:0/720308032 conn(0x5593fd6c4980 msgr2=0x5593fd6d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.360+0000 7fa306603740 1 -- 172.21.15.2:0/720308032 shutdown_connections 2024-03-15T13:06:21.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.360+0000 7fa306603740 1 -- 172.21.15.2:0/720308032 wait complete. 2024-03-15T13:06:21.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.360+0000 7fa306603740 1 -- start start 2024-03-15T13:06:21.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa306603740 1 Processor -- start 2024-03-15T13:06:21.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa306603740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6ca250 0x5593fd6db120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa306603740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6cab70 0x5593fd6d74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa306603740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6d3750 0x5593fd6d79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa306603740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fd6d7f20 con 0x5593fd6ca250 2024-03-15T13:06:21.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa306603740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fd6d8090 con 0x5593fd6d3750 2024-03-15T13:06:21.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa306603740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fd6d8250 con 0x5593fd6cab70 2024-03-15T13:06:21.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa306603740 10 client.0 _ll_register_callbacks cb 0x7fa2f0003a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:21.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa3016e4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6ca250 0x5593fd6db120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa3016e4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6ca250 0x5593fd6db120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34040/0 (socket says 172.21.15.2:34040) 2024-03-15T13:06:21.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa3016e4640 1 -- 172.21.15.2:0/2287925594 learned_addr learned my addr 172.21.15.2:0/2287925594 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa300ee3640 1 --2- 172.21.15.2:0/2287925594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6cab70 0x5593fd6d74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa301ee5640 1 --2- 172.21.15.2:0/2287925594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6d3750 0x5593fd6d79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa3016e4640 1 -- 172.21.15.2:0/2287925594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6cab70 msgr2=0x5593fd6d74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa3016e4640 1 --2- 172.21.15.2:0/2287925594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6cab70 0x5593fd6d74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa3016e4640 1 -- 172.21.15.2:0/2287925594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6d3750 msgr2=0x5593fd6d79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa3016e4640 1 --2- 172.21.15.2:0/2287925594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6d3750 0x5593fd6d79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa3016e4640 1 -- 172.21.15.2:0/2287925594 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593fd5f4060 con 0x5593fd6ca250 2024-03-15T13:06:21.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.361+0000 7fa300ee3640 1 --2- 172.21.15.2:0/2287925594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6cab70 0x5593fd6d74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:21.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa3016e4640 1 --2- 172.21.15.2:0/2287925594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6ca250 0x5593fd6db120 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4011910 tx=0x7fa2f4011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816c0520afdb6d31 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa2e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:21.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa2e0ff9640 1 -- 172.21.15.2:0/2287925594 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4028040 con 0x5593fd6ca250 2024-03-15T13:06:21.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa2e0ff9640 1 -- 172.21.15.2:0/2287925594 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa2f40039f0 con 0x5593fd6ca250 2024-03-15T13:06:21.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa306603740 1 -- 172.21.15.2:0/2287925594 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593fd2dd700 con 0x5593fd6ca250 2024-03-15T13:06:21.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa2e0ff9640 1 -- 172.21.15.2:0/2287925594 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2f4004280 con 0x5593fd6ca250 2024-03-15T13:06:21.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa306603740 10 client.17835 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:21.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa306603740 1 -- 172.21.15.2:0/2287925594 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5593fd5f28c0 con 0x5593fd6ca250 2024-03-15T13:06:21.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa2e0ff9640 1 -- 172.21.15.2:0/2287925594 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5593fd2dd700 con 0x5593fd6ca250 2024-03-15T13:06:21.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa2e0ff9640 1 -- 172.21.15.2:0/2287925594 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa2f403d070 con 0x5593fd6ca250 2024-03-15T13:06:21.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa306603740 10 client.17835 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:21.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa306603740 10 client.17835 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:06:21.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa306603740 20 client.17835 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:21.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa306603740 1 --2- 172.21.15.2:0/2287925594 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5593fd7401d0 0x5593fd7425f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa306603740 4 client.17835 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:06:21.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa306603740 1 -- 172.21.15.2:0/2287925594 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5593fd742d10 con 0x5593fd7401d0 2024-03-15T13:06:21.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.362+0000 7fa300ee3640 1 --2- 172.21.15.2:0/2287925594 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5593fd7401d0 0x5593fd7425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa300ee3640 1 --2- 172.21.15.2:0/2287925594 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5593fd7401d0 0x5593fd7425f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=de01ff283228ef6c server_cookie=ddb39977fd763896 in_seq=0 out_seq=0 2024-03-15T13:06:21.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa2e0ff9640 10 client.17835 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:06:21.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa2e0ff9640 1 -- 172.21.15.2:0/2287925594 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5593fd742d10 con 0x5593fd7401d0 2024-03-15T13:06:21.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa2e0ff9640 10 client.17835 handle_command_reply: tid=0 2024-03-15T13:06:21.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa306603740 1 client.17835 shutdown 2024-03-15T13:06:21.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa306603740 1 -- 172.21.15.2:0/2287925594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6ca250 msgr2=0x5593fd6db120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa306603740 1 --2- 172.21.15.2:0/2287925594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6ca250 0x5593fd6db120 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4011910 tx=0x7fa2f4011dd0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa306603740 1 -- 172.21.15.2:0/2287925594 shutdown_connections 2024-03-15T13:06:21.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa306603740 1 --2- 172.21.15.2:0/2287925594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5593fd6d3750 0x5593fd6d79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa306603740 1 --2- 172.21.15.2:0/2287925594 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x5593fd7401d0 0x5593fd7425f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa306603740 1 --2- 172.21.15.2:0/2287925594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fd6cab70 0x5593fd6d74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa306603740 1 --2- 172.21.15.2:0/2287925594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5593fd6ca250 0x5593fd6db120 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa2e0ff9640 0 client.17835 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:06:21.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa306603740 1 -- 172.21.15.2:0/2287925594 >> 172.21.15.2:0/2287925594 conn(0x5593fd6c4980 msgr2=0x5593fd6d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa306603740 1 -- 172.21.15.2:0/2287925594 shutdown_connections 2024-03-15T13:06:21.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa306603740 1 -- 172.21.15.2:0/2287925594 wait complete. 2024-03-15T13:06:21.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.363+0000 7fa306603740 20 client.17835 trim_cache size 0 max 16384 2024-03-15T13:06:21.356 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:06:21.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.364+0000 7fa303948640 1 -- 172.21.15.2:0/3129221621 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2e40a1ba0 msgr2=0x7fa2e40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.364+0000 7fa303948640 1 --2- 172.21.15.2:0/3129221621 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2e40a1ba0 0x7fa2e40a4060 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec005940 tx=0x7fa2ec0058d0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.364+0000 7fa303948640 1 -- 172.21.15.2:0/3129221621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc0690b0 msgr2=0x7fa2fc19dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.364+0000 7fa303948640 1 --2- 172.21.15.2:0/3129221621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc0690b0 0x7fa2fc19dc80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000ea60 tx=0x7fa2f000ef20 comp rx=0 tx=0).stop 2024-03-15T13:06:21.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.364+0000 7fa303948640 1 -- 172.21.15.2:0/3129221621 shutdown_connections 2024-03-15T13:06:21.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.364+0000 7fa303948640 1 --2- 172.21.15.2:0/3129221621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2fc1074f0 0x7fa2fc1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.364+0000 7fa303948640 1 --2- 172.21.15.2:0/3129221621 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa2e40a1ba0 0x7fa2e40a4060 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.364+0000 7fa303948640 1 --2- 172.21.15.2:0/3129221621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2fc0690b0 0x7fa2fc19dc80 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.364+0000 7fa303948640 1 --2- 172.21.15.2:0/3129221621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2fc0686f0 0x7fa2fc19d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.364+0000 7fa303948640 1 -- 172.21.15.2:0/3129221621 >> 172.21.15.2:0/3129221621 conn(0x7fa2fc0774f0 msgr2=0x7fa2fc0feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.364+0000 7fa303948640 1 -- 172.21.15.2:0/3129221621 shutdown_connections 2024-03-15T13:06:21.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.364+0000 7fa303948640 1 -- 172.21.15.2:0/3129221621 wait complete. 2024-03-15T13:06:21.359 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:06:21.359 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:06:21.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.479+0000 7f9b34b9a640 1 -- 172.21.15.2:0/2380722603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b30100f80 msgr2=0x7f9b30104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.479+0000 7f9b34b9a640 1 --2- 172.21.15.2:0/2380722603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b30100f80 0x7f9b30104fc0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9b18009930 tx=0x7f9b1802bc30 comp rx=0 tx=0).stop 2024-03-15T13:06:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.479+0000 7f9b34b9a640 1 -- 172.21.15.2:0/2380722603 shutdown_connections 2024-03-15T13:06:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.479+0000 7f9b34b9a640 1 --2- 172.21.15.2:0/2380722603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b301056a0 0x7f9b3010d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.479+0000 7f9b34b9a640 1 --2- 172.21.15.2:0/2380722603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b30100f80 0x7f9b30104fc0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.479+0000 7f9b34b9a640 1 --2- 172.21.15.2:0/2380722603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b301005d0 0x7f9b301009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.479+0000 7f9b34b9a640 1 -- 172.21.15.2:0/2380722603 >> 172.21.15.2:0/2380722603 conn(0x7f9b300774f0 msgr2=0x7f9b30077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.480+0000 7f9b34b9a640 1 -- 172.21.15.2:0/2380722603 shutdown_connections 2024-03-15T13:06:21.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.480+0000 7f9b34b9a640 1 -- 172.21.15.2:0/2380722603 wait complete. 2024-03-15T13:06:21.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.480+0000 7f9b34b9a640 1 Processor -- start 2024-03-15T13:06:21.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.480+0000 7f9b34b9a640 1 -- start start 2024-03-15T13:06:21.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.480+0000 7f9b34b9a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b301005d0 0x7f9b3019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.480+0000 7f9b34b9a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b301056a0 0x7f9b3019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.480+0000 7f9b34b9a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b30196850 0x7f9b30196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.480+0000 7f9b34b9a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b30197300 con 0x7f9b301005d0 2024-03-15T13:06:21.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.480+0000 7f9b34b9a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b30197470 con 0x7f9b30196850 2024-03-15T13:06:21.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.480+0000 7f9b34b9a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b301975e0 con 0x7f9b301056a0 2024-03-15T13:06:21.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b2e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b301005d0 0x7f9b3019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b2e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b301005d0 0x7f9b3019c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34068/0 (socket says 172.21.15.2:34068) 2024-03-15T13:06:21.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b2ed9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b30196850 0x7f9b30196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b2dd9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b301056a0 0x7f9b3019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b2e59c640 1 -- 172.21.15.2:0/103970577 learned_addr learned my addr 172.21.15.2:0/103970577 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b2e59c640 1 -- 172.21.15.2:0/103970577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b301056a0 msgr2=0x7f9b3019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b2e59c640 1 --2- 172.21.15.2:0/103970577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b301056a0 0x7f9b3019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b2e59c640 1 -- 172.21.15.2:0/103970577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b30196850 msgr2=0x7f9b30196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b2e59c640 1 --2- 172.21.15.2:0/103970577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b30196850 0x7f9b30196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b2e59c640 1 -- 172.21.15.2:0/103970577 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b18009560 con 0x7f9b301005d0 2024-03-15T13:06:21.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b2e59c640 1 --2- 172.21.15.2:0/103970577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b301005d0 0x7f9b3019c6e0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800e9f0 tx=0x7f9b2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b277fe640 1 -- 172.21.15.2:0/103970577 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28009700 con 0x7f9b301005d0 2024-03-15T13:06:21.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b277fe640 1 -- 172.21.15.2:0/103970577 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9b2800cd80 con 0x7f9b301005d0 2024-03-15T13:06:21.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b34b9a640 1 -- 172.21.15.2:0/103970577 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b301a3590 con 0x7f9b301005d0 2024-03-15T13:06:21.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b277fe640 1 -- 172.21.15.2:0/103970577 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28018920 con 0x7f9b301005d0 2024-03-15T13:06:21.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.481+0000 7f9b34b9a640 1 -- 172.21.15.2:0/103970577 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b301a3a20 con 0x7f9b301005d0 2024-03-15T13:06:21.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b277fe640 1 -- 172.21.15.2:0/103970577 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f9b28018a80 con 0x7f9b301005d0 2024-03-15T13:06:21.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b3777b740 1 Processor -- start 2024-03-15T13:06:21.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b3777b740 1 -- start start 2024-03-15T13:06:21.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b3777b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f8084a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b3777b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f8084ab70 0x563f80853210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b3777b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f80853750 0x563f80855b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b3777b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f80805960 con 0x563f80853750 2024-03-15T13:06:21.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b3777b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f807b9310 con 0x563f8084a250 2024-03-15T13:06:21.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b3777b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f807b6840 con 0x563f8084ab70 2024-03-15T13:06:21.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b2dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f80853750 0x563f80855b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b2ed9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f8084a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b2dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f80853750 0x563f80855b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34084/0 (socket says 172.21.15.2:34084) 2024-03-15T13:06:21.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b2dd9b640 1 -- 172.21.15.2:0/2220597633 learned_addr learned my addr 172.21.15.2:0/2220597633 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b2e59c640 1 --2- 172.21.15.2:0/2220597633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f8084ab70 0x563f80853210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b277fe640 1 --2- 172.21.15.2:0/103970577 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9b1c0a3db0 0x7f9b1c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.484+0000 7f9b2ed9d640 1 --2- 172.21.15.2:0/103970577 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9b1c0a3db0 0x7f9b1c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.485+0000 7f9b2dd9b640 1 -- 172.21.15.2:0/2220597633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f8084ab70 msgr2=0x563f80853210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.485+0000 7f9b2dd9b640 1 --2- 172.21.15.2:0/2220597633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f8084ab70 0x563f80853210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.485+0000 7f9b2dd9b640 1 -- 172.21.15.2:0/2220597633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 msgr2=0x563f8084a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.485+0000 7f9b2dd9b640 1 --2- 172.21.15.2:0/2220597633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f8084a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.485+0000 7f9b277fe640 1 -- 172.21.15.2:0/103970577 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f9b28014070 con 0x7f9b301005d0 2024-03-15T13:06:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.485+0000 7f9b2dd9b640 1 -- 172.21.15.2:0/2220597633 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f80774060 con 0x563f80853750 2024-03-15T13:06:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.485+0000 7f9b2dd9b640 1 --2- 172.21.15.2:0/2220597633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f80853750 0x563f80855b40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9b18000c00 tx=0x7f9b1802c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9af1ff98a1bd16 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.485+0000 7f9b2ed9d640 1 --2- 172.21.15.2:0/103970577 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9b1c0a3db0 0x7f9b1c0a6270 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009fd0 tx=0x7f9b20004ab0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.485+0000 7f9b24ff9640 1 -- 172.21.15.2:0/2220597633 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b18004740 con 0x563f80853750 2024-03-15T13:06:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.485+0000 7f9b24ff9640 1 -- 172.21.15.2:0/2220597633 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9b180048a0 con 0x563f80853750 2024-03-15T13:06:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.485+0000 7f9b24ff9640 1 -- 172.21.15.2:0/2220597633 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b18003850 con 0x563f80853750 2024-03-15T13:06:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 -- 172.21.15.2:0/2220597633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f80853750 msgr2=0x563f80855b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 --2- 172.21.15.2:0/2220597633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f80853750 0x563f80855b40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9b18000c00 tx=0x7f9b1802c660 comp rx=0 tx=0).stop 2024-03-15T13:06:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 -- 172.21.15.2:0/2220597633 shutdown_connections 2024-03-15T13:06:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 --2- 172.21.15.2:0/2220597633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f80853750 0x563f80855b40 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 --2- 172.21.15.2:0/2220597633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f8084ab70 0x563f80853210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 --2- 172.21.15.2:0/2220597633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f8084a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 -- 172.21.15.2:0/2220597633 >> 172.21.15.2:0/2220597633 conn(0x563f80844980 msgr2=0x563f80844d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 -- 172.21.15.2:0/2220597633 shutdown_connections 2024-03-15T13:06:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 -- 172.21.15.2:0/2220597633 wait complete. 2024-03-15T13:06:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 -- start start 2024-03-15T13:06:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 Processor -- start 2024-03-15T13:06:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f808530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f8084ab70 0x563f80850470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f80853750 0x563f808509f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f807b2020 con 0x563f8084ab70 2024-03-15T13:06:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f80485430 con 0x563f8084a250 2024-03-15T13:06:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.486+0000 7f9b3777b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f80441e00 con 0x563f80853750 2024-03-15T13:06:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b3777b740 10 client.0 _ll_register_callbacks cb 0x563f80759cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b2dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f8084ab70 0x563f80850470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b2e59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f808530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b2ed9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f80853750 0x563f808509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b2dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f8084ab70 0x563f80850470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34090/0 (socket says 172.21.15.2:34090) 2024-03-15T13:06:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b2dd9b640 1 -- 172.21.15.2:0/2990241986 learned_addr learned my addr 172.21.15.2:0/2990241986 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b2e59c640 1 -- 172.21.15.2:0/2990241986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f80853750 msgr2=0x563f808509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b2e59c640 1 --2- 172.21.15.2:0/2990241986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f80853750 0x563f808509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b2e59c640 1 -- 172.21.15.2:0/2990241986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f8084ab70 msgr2=0x563f80850470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b2e59c640 1 --2- 172.21.15.2:0/2990241986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f8084ab70 0x563f80850470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b2e59c640 1 -- 172.21.15.2:0/2990241986 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3010da80 con 0x563f8084a250 2024-03-15T13:06:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b2dd9b640 1 --2- 172.21.15.2:0/2990241986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f8084ab70 0x563f80850470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b2e59c640 1 --2- 172.21.15.2:0/2990241986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f808530e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9b280cb270 tx=0x7f9b280d1360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c22776c45146ff8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:06:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b15ffb640 1 -- 172.21.15.2:0/2990241986 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2809a800 con 0x563f8084a250 2024-03-15T13:06:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b15ffb640 1 -- 172.21.15.2:0/2990241986 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f9b2809e8a0 con 0x563f8084a250 2024-03-15T13:06:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b3777b740 1 -- 172.21.15.2:0/2990241986 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f804b17e0 con 0x563f8084a250 2024-03-15T13:06:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b15ffb640 1 -- 172.21.15.2:0/2990241986 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b280a2ef0 con 0x563f8084a250 2024-03-15T13:06:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b3777b740 10 client.27710 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.487+0000 7f9b3777b740 1 -- 172.21.15.2:0/2990241986 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x563f807b06a0 con 0x563f8084a250 2024-03-15T13:06:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.488+0000 7f9b15ffb640 1 -- 172.21.15.2:0/2990241986 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x563f804b17e0 con 0x563f8084a250 2024-03-15T13:06:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.488+0000 7f9b15ffb640 1 -- 172.21.15.2:0/2990241986 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9b280d1020 con 0x563f8084a250 2024-03-15T13:06:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.488+0000 7f9b3777b740 10 client.27710 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.488+0000 7f9b3777b740 10 client.27710 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:06:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.488+0000 7f9b3777b740 20 client.27710 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.488+0000 7f9b3777b740 1 --2- 172.21.15.2:0/2990241986 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563f80863850 0x563f80865c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.488+0000 7f9b3777b740 4 client.27710 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.488+0000 7f9b3777b740 1 -- 172.21.15.2:0/2990241986 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f804ee720 con 0x563f80863850 2024-03-15T13:06:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.488+0000 7f9b2dd9b640 1 --2- 172.21.15.2:0/2990241986 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563f80863850 0x563f80865c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.488+0000 7f9b2dd9b640 1 --2- 172.21.15.2:0/2990241986 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563f80863850 0x563f80865c70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20b2d1d7751251fc server_cookie=d4c572a08abeaea2 in_seq=0 out_seq=0 2024-03-15T13:06:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.488+0000 7f9b15ffb640 10 client.27710 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:06:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.489+0000 7f9b15ffb640 1 -- 172.21.15.2:0/2990241986 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563f804ee720 con 0x563f80863850 2024-03-15T13:06:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.489+0000 7f9b15ffb640 10 client.27710 handle_command_reply: tid=0 2024-03-15T13:06:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.489+0000 7f9b3777b740 1 client.27710 shutdown 2024-03-15T13:06:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.489+0000 7f9b3777b740 1 -- 172.21.15.2:0/2990241986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 msgr2=0x563f808530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.489+0000 7f9b3777b740 1 --2- 172.21.15.2:0/2990241986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f808530e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9b280cb270 tx=0x7f9b280d1360 comp rx=0 tx=0).stop 2024-03-15T13:06:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.489+0000 7f9b3777b740 1 -- 172.21.15.2:0/2990241986 shutdown_connections 2024-03-15T13:06:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.489+0000 7f9b3777b740 1 --2- 172.21.15.2:0/2990241986 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563f80863850 0x563f80865c70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.489+0000 7f9b3777b740 1 --2- 172.21.15.2:0/2990241986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f80853750 0x563f808509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.489+0000 7f9b3777b740 1 --2- 172.21.15.2:0/2990241986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f8084ab70 0x563f80850470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.489+0000 7f9b3777b740 1 --2- 172.21.15.2:0/2990241986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f808530e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.489+0000 7f9b15ffb640 0 client.27710 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:06:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.489+0000 7f9b3777b740 1 -- 172.21.15.2:0/2990241986 >> 172.21.15.2:0/2990241986 conn(0x563f80844980 msgr2=0x563f808466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.489+0000 7f9b3777b740 1 -- 172.21.15.2:0/2990241986 shutdown_connections 2024-03-15T13:06:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.489+0000 7f9b3777b740 1 -- 172.21.15.2:0/2990241986 wait complete. 2024-03-15T13:06:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.489+0000 7f9b3777b740 20 client.27710 trim_cache size 0 max 16384 2024-03-15T13:06:21.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b3777b740 1 Processor -- start 2024-03-15T13:06:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b3777b740 1 -- start start 2024-03-15T13:06:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b3777b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f80852ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b3777b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f8084ab70 0x563f808530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b3777b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f80853750 0x563f8085ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b3777b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f80737130 con 0x563f8084ab70 2024-03-15T13:06:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b3777b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f804ee720 con 0x563f8084a250 2024-03-15T13:06:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b3777b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f80836a40 con 0x563f80853750 2024-03-15T13:06:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b2dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f8084ab70 0x563f808530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b2dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f8084ab70 0x563f808530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34092/0 (socket says 172.21.15.2:34092) 2024-03-15T13:06:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b2dd9b640 1 -- 172.21.15.2:0/1490722014 learned_addr learned my addr 172.21.15.2:0/1490722014 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b2e59c640 1 --2- 172.21.15.2:0/1490722014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f80852ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b2ed9d640 1 --2- 172.21.15.2:0/1490722014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f80853750 0x563f8085ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b2dd9b640 1 -- 172.21.15.2:0/1490722014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f80853750 msgr2=0x563f8085ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b2dd9b640 1 --2- 172.21.15.2:0/1490722014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f80853750 0x563f8085ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b2dd9b640 1 -- 172.21.15.2:0/1490722014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 msgr2=0x563f80852ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b2dd9b640 1 --2- 172.21.15.2:0/1490722014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f80852ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b2dd9b640 1 -- 172.21.15.2:0/1490722014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f8074cc20 con 0x563f8084ab70 2024-03-15T13:06:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.502+0000 7f9b2dd9b640 1 --2- 172.21.15.2:0/1490722014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f8084ab70 0x563f808530e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f9b18005ab0 tx=0x7f9b18042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73773bf9645e11a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.503+0000 7f9b17fff640 1 -- 172.21.15.2:0/1490722014 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b18004a10 con 0x563f8084ab70 2024-03-15T13:06:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.503+0000 7f9b17fff640 1 -- 172.21.15.2:0/1490722014 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9b180522d0 con 0x563f8084ab70 2024-03-15T13:06:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.503+0000 7f9b17fff640 1 -- 172.21.15.2:0/1490722014 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b180346a0 con 0x563f8084ab70 2024-03-15T13:06:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.503+0000 7f9b3777b740 1 -- 172.21.15.2:0/1490722014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f8084ab70 msgr2=0x563f808530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.503+0000 7f9b3777b740 1 --2- 172.21.15.2:0/1490722014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f8084ab70 0x563f808530e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f9b18005ab0 tx=0x7f9b18042040 comp rx=0 tx=0).stop 2024-03-15T13:06:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.503+0000 7f9b3777b740 1 -- 172.21.15.2:0/1490722014 shutdown_connections 2024-03-15T13:06:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.503+0000 7f9b3777b740 1 --2- 172.21.15.2:0/1490722014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f80853750 0x563f8085ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.503+0000 7f9b3777b740 1 --2- 172.21.15.2:0/1490722014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f8084ab70 0x563f808530e0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.503+0000 7f9b3777b740 1 --2- 172.21.15.2:0/1490722014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f80852ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.503+0000 7f9b3777b740 1 -- 172.21.15.2:0/1490722014 >> 172.21.15.2:0/1490722014 conn(0x563f80844980 msgr2=0x563f80843b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.503+0000 7f9b3777b740 1 -- 172.21.15.2:0/1490722014 shutdown_connections 2024-03-15T13:06:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.503+0000 7f9b3777b740 1 -- 172.21.15.2:0/1490722014 wait complete. 2024-03-15T13:06:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.503+0000 7f9b3777b740 1 -- start start 2024-03-15T13:06:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.503+0000 7f9b3777b740 1 Processor -- start 2024-03-15T13:06:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b3777b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f8085b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b3777b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f8084ab70 0x563f8084d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b3777b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f80853750 0x563f8084da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b3777b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8085b8d0 con 0x563f80853750 2024-03-15T13:06:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b3777b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8084dfd0 con 0x563f8084a250 2024-03-15T13:06:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b3777b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8084e190 con 0x563f8084ab70 2024-03-15T13:06:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b3777b740 10 client.0 _ll_register_callbacks cb 0x7f9b2800cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b2ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f80853750 0x563f8084da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b2e59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f8085b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b2ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f80853750 0x563f8084da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34106/0 (socket says 172.21.15.2:34106) 2024-03-15T13:06:21.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b2ed9d640 1 -- 172.21.15.2:0/243147386 learned_addr learned my addr 172.21.15.2:0/243147386 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b2e59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f8085b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:54574/0 (socket says 172.21.15.2:54574) 2024-03-15T13:06:21.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b2dd9b640 1 --2- 172.21.15.2:0/243147386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f8084ab70 0x563f8084d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b2ed9d640 1 -- 172.21.15.2:0/243147386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f8084ab70 msgr2=0x563f8084d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b2ed9d640 1 --2- 172.21.15.2:0/243147386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f8084ab70 0x563f8084d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b2ed9d640 1 -- 172.21.15.2:0/243147386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 msgr2=0x563f8085b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b2ed9d640 1 --2- 172.21.15.2:0/243147386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f8085b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b2ed9d640 1 -- 172.21.15.2:0/243147386 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f80774060 con 0x563f80853750 2024-03-15T13:06:21.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b2e59c640 1 --2- 172.21.15.2:0/243147386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f8085b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:21.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b2ed9d640 1 --2- 172.21.15.2:0/243147386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f80853750 0x563f8084da90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f9b20003bf0 tx=0x7f9b20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17d70f55f12b8a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:21.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b15ffb640 1 -- 172.21.15.2:0/243147386 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20027070 con 0x563f80853750 2024-03-15T13:06:21.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b15ffb640 1 -- 172.21.15.2:0/243147386 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f9b20004250 con 0x563f80853750 2024-03-15T13:06:21.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.504+0000 7f9b3777b740 1 -- 172.21.15.2:0/243147386 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f80850840 con 0x563f80853750 2024-03-15T13:06:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.505+0000 7f9b15ffb640 1 -- 172.21.15.2:0/243147386 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b20004410 con 0x563f80853750 2024-03-15T13:06:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.505+0000 7f9b3777b740 10 client.17853 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.505+0000 7f9b3777b740 1 -- 172.21.15.2:0/243147386 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x563f807728c0 con 0x563f80853750 2024-03-15T13:06:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.505+0000 7f9b15ffb640 1 -- 172.21.15.2:0/243147386 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x563f80850840 con 0x563f80853750 2024-03-15T13:06:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.505+0000 7f9b15ffb640 1 -- 172.21.15.2:0/243147386 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f9b20021440 con 0x563f80853750 2024-03-15T13:06:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.505+0000 7f9b3777b740 10 client.17853 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.505+0000 7f9b3777b740 10 client.17853 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:06:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.505+0000 7f9b3777b740 20 client.17853 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.505+0000 7f9b3777b740 1 --2- 172.21.15.2:0/243147386 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563f808c0600 0x563f808c2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.505+0000 7f9b3777b740 4 client.17853 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:06:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.505+0000 7f9b3777b740 1 -- 172.21.15.2:0/243147386 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f9b180048a0 con 0x563f808c0600 2024-03-15T13:06:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.505+0000 7f9b2e59c640 1 --2- 172.21.15.2:0/243147386 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563f808c0600 0x563f808c2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b15ffb640 10 client.17853 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:06:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b2e59c640 1 --2- 172.21.15.2:0/243147386 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563f808c0600 0x563f808c2a20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c6c3f1abb96849e server_cookie=697947df8cd7d98a in_seq=0 out_seq=0 2024-03-15T13:06:21.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b15ffb640 1 -- 172.21.15.2:0/243147386 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f9b180048a0 con 0x563f808c0600 2024-03-15T13:06:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b15ffb640 10 client.17853 handle_command_reply: tid=0 2024-03-15T13:06:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b3777b740 1 client.17853 shutdown 2024-03-15T13:06:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b3777b740 1 -- 172.21.15.2:0/243147386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f80853750 msgr2=0x563f8084da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b3777b740 1 --2- 172.21.15.2:0/243147386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f80853750 0x563f8084da90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f9b20003bf0 tx=0x7f9b20004a40 comp rx=0 tx=0).stop 2024-03-15T13:06:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b3777b740 1 -- 172.21.15.2:0/243147386 shutdown_connections 2024-03-15T13:06:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b3777b740 1 --2- 172.21.15.2:0/243147386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f80853750 0x563f8084da90 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b3777b740 1 --2- 172.21.15.2:0/243147386 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x563f808c0600 0x563f808c2a20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b3777b740 1 --2- 172.21.15.2:0/243147386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f8084ab70 0x563f8084d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b3777b740 1 --2- 172.21.15.2:0/243147386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f8084a250 0x563f8085b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b15ffb640 0 client.17853 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:06:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b3777b740 1 -- 172.21.15.2:0/243147386 >> 172.21.15.2:0/243147386 conn(0x563f80844980 msgr2=0x563f80845d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b3777b740 1 -- 172.21.15.2:0/243147386 shutdown_connections 2024-03-15T13:06:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b3777b740 1 -- 172.21.15.2:0/243147386 wait complete. 2024-03-15T13:06:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b3777b740 20 client.17853 trim_cache size 0 max 16384 2024-03-15T13:06:21.498 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:06:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b34b9a640 1 -- 172.21.15.2:0/103970577 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9b1c0a3db0 msgr2=0x7f9b1c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b34b9a640 1 --2- 172.21.15.2:0/103970577 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9b1c0a3db0 0x7f9b1c0a6270 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009fd0 tx=0x7f9b20004ab0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b34b9a640 1 -- 172.21.15.2:0/103970577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b301005d0 msgr2=0x7f9b3019c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b34b9a640 1 --2- 172.21.15.2:0/103970577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b301005d0 0x7f9b3019c6e0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800e9f0 tx=0x7f9b2800eeb0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b34b9a640 1 -- 172.21.15.2:0/103970577 shutdown_connections 2024-03-15T13:06:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.506+0000 7f9b34b9a640 1 --2- 172.21.15.2:0/103970577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b30196850 0x7f9b30196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.507+0000 7f9b34b9a640 1 --2- 172.21.15.2:0/103970577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b301056a0 0x7f9b3019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.507+0000 7f9b34b9a640 1 --2- 172.21.15.2:0/103970577 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9b1c0a3db0 0x7f9b1c0a6270 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.507+0000 7f9b34b9a640 1 --2- 172.21.15.2:0/103970577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b301005d0 0x7f9b3019c6e0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.507+0000 7f9b34b9a640 1 -- 172.21.15.2:0/103970577 >> 172.21.15.2:0/103970577 conn(0x7f9b300774f0 msgr2=0x7f9b300fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.507+0000 7f9b34b9a640 1 -- 172.21.15.2:0/103970577 shutdown_connections 2024-03-15T13:06:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.507+0000 7f9b34b9a640 1 -- 172.21.15.2:0/103970577 wait complete. 2024-03-15T13:06:21.502 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:06:21.502 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:06:21.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.622+0000 7f2817777640 1 -- 172.21.15.2:0/2494195526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28101056a0 msgr2=0x7f281010d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.622+0000 7f2817777640 1 --2- 172.21.15.2:0/2494195526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28101056a0 0x7f281010d270 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2808009cd0 tx=0x7f280802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.622+0000 7f2817777640 1 -- 172.21.15.2:0/2494195526 shutdown_connections 2024-03-15T13:06:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.622+0000 7f2817777640 1 --2- 172.21.15.2:0/2494195526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28101056a0 0x7f281010d270 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.622+0000 7f2817777640 1 --2- 172.21.15.2:0/2494195526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2810100f80 0x7f2810104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.622+0000 7f2817777640 1 --2- 172.21.15.2:0/2494195526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28101005d0 0x7f28101009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.622+0000 7f2817777640 1 -- 172.21.15.2:0/2494195526 >> 172.21.15.2:0/2494195526 conn(0x7f28100774f0 msgr2=0x7f2810077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.622+0000 7f2817777640 1 -- 172.21.15.2:0/2494195526 shutdown_connections 2024-03-15T13:06:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.622+0000 7f2817777640 1 -- 172.21.15.2:0/2494195526 wait complete. 2024-03-15T13:06:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.622+0000 7f2817777640 1 Processor -- start 2024-03-15T13:06:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.622+0000 7f2817777640 1 -- start start 2024-03-15T13:06:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.622+0000 7f2817777640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28101005d0 0x7f281019d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.622+0000 7f2817777640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28101056a0 0x7f281019dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.622+0000 7f2817777640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2810197930 0x7f2810197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.622+0000 7f2817777640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2810198320 con 0x7f28101056a0 2024-03-15T13:06:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f2817777640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2810198490 con 0x7f2810197930 2024-03-15T13:06:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f2817777640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2810198650 con 0x7f28101005d0 2024-03-15T13:06:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f2814d12640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28101056a0 0x7f281019dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f2814d12640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28101056a0 0x7f281019dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34122/0 (socket says 172.21.15.2:34122) 2024-03-15T13:06:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f2814d12640 1 -- 172.21.15.2:0/3664810868 learned_addr learned my addr 172.21.15.2:0/3664810868 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f2815513640 1 --2- 172.21.15.2:0/3664810868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28101005d0 0x7f281019d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f2815d14640 1 --2- 172.21.15.2:0/3664810868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2810197930 0x7f2810197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f2814d12640 1 -- 172.21.15.2:0/3664810868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28101005d0 msgr2=0x7f281019d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f2814d12640 1 --2- 172.21.15.2:0/3664810868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28101005d0 0x7f281019d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f2814d12640 1 -- 172.21.15.2:0/3664810868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2810197930 msgr2=0x7f2810197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f2814d12640 1 --2- 172.21.15.2:0/3664810868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2810197930 0x7f2810197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f2814d12640 1 -- 172.21.15.2:0/3664810868 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2808009900 con 0x7f28101056a0 2024-03-15T13:06:21.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f2814d12640 1 --2- 172.21.15.2:0/3664810868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28101056a0 0x7f281019dcc0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f280000e9c0 tx=0x7f280000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f28067fc640 1 -- 172.21.15.2:0/3664810868 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800009700 con 0x7f28101056a0 2024-03-15T13:06:21.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f2817777640 1 -- 172.21.15.2:0/3664810868 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28101a4640 con 0x7f28101056a0 2024-03-15T13:06:21.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f28067fc640 1 -- 172.21.15.2:0/3664810868 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f280000cd80 con 0x7f28101056a0 2024-03-15T13:06:21.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f28067fc640 1 -- 172.21.15.2:0/3664810868 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800010430 con 0x7f28101056a0 2024-03-15T13:06:21.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.623+0000 7f2817777640 1 -- 172.21.15.2:0/3664810868 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28101a4b50 con 0x7f28101056a0 2024-03-15T13:06:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.625+0000 7f28067fc640 1 -- 172.21.15.2:0/3664810868 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f2800004ff0 con 0x7f28101056a0 2024-03-15T13:06:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f281a42b740 1 Processor -- start 2024-03-15T13:06:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f281a42b740 1 -- start start 2024-03-15T13:06:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f281a42b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfc4250 0x55be0dfc4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f281a42b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4b70 0x55be0dfcd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f28067fc640 1 --2- 172.21.15.2:0/3664810868 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f27fc0a1ba0 0x7f27fc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f281a42b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfcd750 0x55be0dfcfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f281a42b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0df7f960 con 0x55be0dfc4b70 2024-03-15T13:06:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f281a42b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0df33310 con 0x55be0dfcd750 2024-03-15T13:06:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f281a42b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0df30840 con 0x55be0dfc4250 2024-03-15T13:06:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f2814d12640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4b70 0x55be0dfcd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f2815513640 1 --2- 172.21.15.2:0/3664810868 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f27fc0a1ba0 0x7f27fc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f2815d14640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfcd750 0x55be0dfcfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f2814d12640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4b70 0x55be0dfcd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34138/0 (socket says 172.21.15.2:34138) 2024-03-15T13:06:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f2814d12640 1 -- 172.21.15.2:0/1979893679 learned_addr learned my addr 172.21.15.2:0/1979893679 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f2815d14640 1 --2- 172.21.15.2:0/1979893679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfc4250 0x55be0dfc4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.626+0000 7f28067fc640 1 -- 172.21.15.2:0/3664810868 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f2800014070 con 0x7f28101056a0 2024-03-15T13:06:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.627+0000 7f2815d14640 1 -- 172.21.15.2:0/1979893679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfc4250 msgr2=0x55be0dfc4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.627+0000 7f2815d14640 1 --2- 172.21.15.2:0/1979893679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfc4250 0x55be0dfc4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.627+0000 7f2815d14640 1 -- 172.21.15.2:0/1979893679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4b70 msgr2=0x55be0dfcd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.627+0000 7f2815d14640 1 --2- 172.21.15.2:0/1979893679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4b70 0x55be0dfcd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.627+0000 7f2815d14640 1 -- 172.21.15.2:0/1979893679 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be0deee060 con 0x55be0dfcd750 2024-03-15T13:06:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.627+0000 7f2814d12640 1 --2- 172.21.15.2:0/1979893679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4b70 0x55be0dfcd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.627+0000 7f2815513640 1 --2- 172.21.15.2:0/3664810868 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f27fc0a1ba0 0x7f27fc0a4060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2810198a70 tx=0x7f280c005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.627+0000 7f2815d14640 1 --2- 172.21.15.2:0/1979893679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfcd750 0x55be0dfcfb40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2808037040 tx=0x7f2808009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d65016a2eec4817 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.627+0000 7f27fb7fe640 1 -- 172.21.15.2:0/1979893679 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28080346d0 con 0x55be0dfcd750 2024-03-15T13:06:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.627+0000 7f27fb7fe640 1 -- 172.21.15.2:0/1979893679 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2808032680 con 0x55be0dfcd750 2024-03-15T13:06:21.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.627+0000 7f27fb7fe640 1 -- 172.21.15.2:0/1979893679 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808003d00 con 0x55be0dfcd750 2024-03-15T13:06:21.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.628+0000 7f281a42b740 1 -- 172.21.15.2:0/1979893679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfcd750 msgr2=0x55be0dfcfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.628+0000 7f281a42b740 1 --2- 172.21.15.2:0/1979893679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfcd750 0x55be0dfcfb40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2808037040 tx=0x7f2808009360 comp rx=0 tx=0).stop 2024-03-15T13:06:21.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.628+0000 7f281a42b740 1 -- 172.21.15.2:0/1979893679 shutdown_connections 2024-03-15T13:06:21.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.628+0000 7f281a42b740 1 --2- 172.21.15.2:0/1979893679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfcd750 0x55be0dfcfb40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.628+0000 7f281a42b740 1 --2- 172.21.15.2:0/1979893679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4b70 0x55be0dfcd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.628+0000 7f281a42b740 1 --2- 172.21.15.2:0/1979893679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfc4250 0x55be0dfc4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.628+0000 7f281a42b740 1 -- 172.21.15.2:0/1979893679 >> 172.21.15.2:0/1979893679 conn(0x55be0dfbe980 msgr2=0x55be0dfbed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.628+0000 7f281a42b740 1 -- 172.21.15.2:0/1979893679 shutdown_connections 2024-03-15T13:06:21.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.628+0000 7f281a42b740 1 -- 172.21.15.2:0/1979893679 wait complete. 2024-03-15T13:06:21.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.628+0000 7f281a42b740 1 -- start start 2024-03-15T13:06:21.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.628+0000 7f281a42b740 1 Processor -- start 2024-03-15T13:06:21.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f281a42b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfc4250 0x55be0dfcd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f281a42b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4b70 0x55be0dfca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f281a42b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfcd750 0x55be0dfca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f281a42b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0df2c020 con 0x55be0dfc4b70 2024-03-15T13:06:21.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f281a42b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0dbff430 con 0x55be0dfc4250 2024-03-15T13:06:21.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f281a42b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0dbbbe00 con 0x55be0dfcd750 2024-03-15T13:06:21.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f281a42b740 10 client.0 _ll_register_callbacks cb 0x55be0ded3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:21.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f2815513640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfc4250 0x55be0dfcd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f2814d12640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4b70 0x55be0dfca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f2815513640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfc4250 0x55be0dfcd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:54608/0 (socket says 172.21.15.2:54608) 2024-03-15T13:06:21.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f2814d12640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4b70 0x55be0dfca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34152/0 (socket says 172.21.15.2:34152) 2024-03-15T13:06:21.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f2815513640 1 -- 172.21.15.2:0/2074230243 learned_addr learned my addr 172.21.15.2:0/2074230243 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f2815d14640 1 --2- 172.21.15.2:0/2074230243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfcd750 0x55be0dfca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f2814d12640 1 -- 172.21.15.2:0/2074230243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfcd750 msgr2=0x55be0dfca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f2814d12640 1 --2- 172.21.15.2:0/2074230243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfcd750 0x55be0dfca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f2814d12640 1 -- 172.21.15.2:0/2074230243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfc4250 msgr2=0x55be0dfcd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f2814d12640 1 --2- 172.21.15.2:0/2074230243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfc4250 0x55be0dfcd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f2814d12640 1 -- 172.21.15.2:0/2074230243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2810068650 con 0x55be0dfc4b70 2024-03-15T13:06:21.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f2815513640 1 --2- 172.21.15.2:0/2074230243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfc4250 0x55be0dfcd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:21.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f2814d12640 1 --2- 172.21.15.2:0/2074230243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4b70 0x55be0dfca450 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f280009a6f0 tx=0x7f2800065f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2e236ab579205 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f27f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:21.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f27f8ff9640 1 -- 172.21.15.2:0/2074230243 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280009ae60 con 0x55be0dfc4b70 2024-03-15T13:06:21.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f27f8ff9640 1 -- 172.21.15.2:0/2074230243 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f280009a760 con 0x55be0dfc4b70 2024-03-15T13:06:21.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.629+0000 7f281a42b740 1 -- 172.21.15.2:0/2074230243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be0dc2b7e0 con 0x55be0dfc4b70 2024-03-15T13:06:21.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.630+0000 7f27f8ff9640 1 -- 172.21.15.2:0/2074230243 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f280009e8b0 con 0x55be0dfc4b70 2024-03-15T13:06:21.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.630+0000 7f281a42b740 10 client.17865 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:21.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.630+0000 7f281a42b740 1 -- 172.21.15.2:0/2074230243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55be0df2a6a0 con 0x55be0dfc4b70 2024-03-15T13:06:21.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.630+0000 7f27f8ff9640 1 -- 172.21.15.2:0/2074230243 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55be0dc2b7e0 con 0x55be0dfc4b70 2024-03-15T13:06:21.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.630+0000 7f27f8ff9640 1 -- 172.21.15.2:0/2074230243 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f280001f060 con 0x55be0dfc4b70 2024-03-15T13:06:21.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.630+0000 7f281a42b740 10 client.17865 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:21.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.630+0000 7f281a42b740 10 client.17865 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:06:21.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.630+0000 7f281a42b740 20 client.17865 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:21.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.630+0000 7f281a42b740 1 --2- 172.21.15.2:0/2074230243 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55be0dfdd7d0 0x55be0dfdfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.630+0000 7f281a42b740 4 client.17865 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:21.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.630+0000 7f281a42b740 1 -- 172.21.15.2:0/2074230243 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be0dc68720 con 0x55be0dfdd7d0 2024-03-15T13:06:21.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.630+0000 7f2815513640 1 --2- 172.21.15.2:0/2074230243 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55be0dfdd7d0 0x55be0dfdfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.631+0000 7f2815513640 1 --2- 172.21.15.2:0/2074230243 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55be0dfdd7d0 0x55be0dfdfbc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=caaf87dde9474091 server_cookie=d896345578364fde in_seq=0 out_seq=0 2024-03-15T13:06:21.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.631+0000 7f27f8ff9640 10 client.17865 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:06:21.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.631+0000 7f27f8ff9640 1 -- 172.21.15.2:0/2074230243 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55be0dc68720 con 0x55be0dfdd7d0 2024-03-15T13:06:21.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.631+0000 7f27f8ff9640 10 client.17865 handle_command_reply: tid=0 2024-03-15T13:06:21.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.632+0000 7f281a42b740 1 client.17865 shutdown 2024-03-15T13:06:21.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.632+0000 7f281a42b740 1 -- 172.21.15.2:0/2074230243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4b70 msgr2=0x55be0dfca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.632+0000 7f281a42b740 1 --2- 172.21.15.2:0/2074230243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4b70 0x55be0dfca450 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f280009a6f0 tx=0x7f2800065f40 comp rx=0 tx=0).stop 2024-03-15T13:06:21.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.632+0000 7f281a42b740 1 -- 172.21.15.2:0/2074230243 shutdown_connections 2024-03-15T13:06:21.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.632+0000 7f281a42b740 1 --2- 172.21.15.2:0/2074230243 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55be0dfdd7d0 0x55be0dfdfbc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.632+0000 7f281a42b740 1 --2- 172.21.15.2:0/2074230243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfcd750 0x55be0dfca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.632+0000 7f281a42b740 1 --2- 172.21.15.2:0/2074230243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4b70 0x55be0dfca450 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.632+0000 7f27f8ff9640 0 client.17865 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:06:21.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.632+0000 7f281a42b740 1 --2- 172.21.15.2:0/2074230243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfc4250 0x55be0dfcd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.632+0000 7f281a42b740 1 -- 172.21.15.2:0/2074230243 >> 172.21.15.2:0/2074230243 conn(0x55be0dfbe980 msgr2=0x55be0dfc06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.632+0000 7f281a42b740 1 -- 172.21.15.2:0/2074230243 shutdown_connections 2024-03-15T13:06:21.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.632+0000 7f281a42b740 1 -- 172.21.15.2:0/2074230243 wait complete. 2024-03-15T13:06:21.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.632+0000 7f281a42b740 20 client.17865 trim_cache size 0 max 16384 2024-03-15T13:06:21.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.644+0000 7f281a42b740 1 Processor -- start 2024-03-15T13:06:21.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.644+0000 7f281a42b740 1 -- start start 2024-03-15T13:06:21.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.644+0000 7f281a42b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4250 0x55be0dfc5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.644+0000 7f281a42b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfc4b70 0x55be0dfccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.644+0000 7f281a42b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfcd750 0x55be0dfcd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.644+0000 7f281a42b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0deb1130 con 0x55be0dfc4250 2024-03-15T13:06:21.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.644+0000 7f281a42b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0dfda2d0 con 0x55be0dfcd750 2024-03-15T13:06:21.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.644+0000 7f281a42b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0dfc39f0 con 0x55be0dfc4b70 2024-03-15T13:06:21.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.645+0000 7f2815513640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4250 0x55be0dfc5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.645+0000 7f2815513640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4250 0x55be0dfc5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34168/0 (socket says 172.21.15.2:34168) 2024-03-15T13:06:21.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.645+0000 7f2815513640 1 -- 172.21.15.2:0/2819843058 learned_addr learned my addr 172.21.15.2:0/2819843058 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.645+0000 7f2815d14640 1 --2- 172.21.15.2:0/2819843058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfcd750 0x55be0dfcd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.645+0000 7f2814d12640 1 --2- 172.21.15.2:0/2819843058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfc4b70 0x55be0dfccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.645+0000 7f2815513640 1 -- 172.21.15.2:0/2819843058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfc4b70 msgr2=0x55be0dfccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.645+0000 7f2815513640 1 --2- 172.21.15.2:0/2819843058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfc4b70 0x55be0dfccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.645+0000 7f2815513640 1 -- 172.21.15.2:0/2819843058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfcd750 msgr2=0x55be0dfcd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.645+0000 7f2815513640 1 --2- 172.21.15.2:0/2819843058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfcd750 0x55be0dfcd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.645+0000 7f2815513640 1 -- 172.21.15.2:0/2819843058 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be0dec6c20 con 0x55be0dfc4250 2024-03-15T13:06:21.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.645+0000 7f2815513640 1 --2- 172.21.15.2:0/2819843058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4250 0x55be0dfc5b60 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f280c0118d0 tx=0x7f280c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f976b5e1db8d410 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.645+0000 7f27faffd640 1 -- 172.21.15.2:0/2819843058 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c0272d0 con 0x55be0dfc4250 2024-03-15T13:06:21.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.645+0000 7f27faffd640 1 -- 172.21.15.2:0/2819843058 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f280c025040 con 0x55be0dfc4250 2024-03-15T13:06:21.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.645+0000 7f27faffd640 1 -- 172.21.15.2:0/2819843058 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c027c20 con 0x55be0dfc4250 2024-03-15T13:06:21.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 -- 172.21.15.2:0/2819843058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4250 msgr2=0x55be0dfc5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 --2- 172.21.15.2:0/2819843058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4250 0x55be0dfc5b60 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f280c0118d0 tx=0x7f280c014000 comp rx=0 tx=0).stop 2024-03-15T13:06:21.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 -- 172.21.15.2:0/2819843058 shutdown_connections 2024-03-15T13:06:21.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 --2- 172.21.15.2:0/2819843058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfcd750 0x55be0dfcd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 --2- 172.21.15.2:0/2819843058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfc4b70 0x55be0dfccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 --2- 172.21.15.2:0/2819843058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4250 0x55be0dfc5b60 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 -- 172.21.15.2:0/2819843058 >> 172.21.15.2:0/2819843058 conn(0x55be0dfbe980 msgr2=0x55be0dfca570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 -- 172.21.15.2:0/2819843058 shutdown_connections 2024-03-15T13:06:21.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 -- 172.21.15.2:0/2819843058 wait complete. 2024-03-15T13:06:21.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 -- start start 2024-03-15T13:06:21.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 Processor -- start 2024-03-15T13:06:21.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4250 0x55be0dfd52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfc4b70 0x55be0dfd16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfcd750 0x55be0dfd1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0dfd2290 con 0x55be0dfc4250 2024-03-15T13:06:21.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0dfd2400 con 0x55be0dfc4b70 2024-03-15T13:06:21.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.646+0000 7f281a42b740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0e039020 con 0x55be0dfcd750 2024-03-15T13:06:21.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f281a42b740 10 client.0 _ll_register_callbacks cb 0x7f28000cb220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:21.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f2815513640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4250 0x55be0dfd52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f2815513640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4250 0x55be0dfd52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34172/0 (socket says 172.21.15.2:34172) 2024-03-15T13:06:21.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f2815513640 1 -- 172.21.15.2:0/97577232 learned_addr learned my addr 172.21.15.2:0/97577232 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f2815d14640 1 --2- 172.21.15.2:0/97577232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfcd750 0x55be0dfd1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f2814d12640 1 --2- 172.21.15.2:0/97577232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfc4b70 0x55be0dfd16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f2815513640 1 -- 172.21.15.2:0/97577232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfcd750 msgr2=0x55be0dfd1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f2815513640 1 --2- 172.21.15.2:0/97577232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfcd750 0x55be0dfd1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f2815513640 1 -- 172.21.15.2:0/97577232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfc4b70 msgr2=0x55be0dfd16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f2815513640 1 --2- 172.21.15.2:0/97577232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfc4b70 0x55be0dfd16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f2815513640 1 -- 172.21.15.2:0/97577232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280c017070 con 0x55be0dfc4250 2024-03-15T13:06:21.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f2815513640 1 --2- 172.21.15.2:0/97577232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4250 0x55be0dfd52e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f280c0097d0 tx=0x7f280c003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52044abb677ce76d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f27f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:21.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f27f8ff9640 1 -- 172.21.15.2:0/97577232 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c005500 con 0x55be0dfc4250 2024-03-15T13:06:21.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f27f8ff9640 1 -- 172.21.15.2:0/97577232 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f280c01e7f0 con 0x55be0dfc4250 2024-03-15T13:06:21.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f281a42b740 1 -- 172.21.15.2:0/97577232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2808032680 con 0x55be0dfc4250 2024-03-15T13:06:21.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f27f8ff9640 1 -- 172.21.15.2:0/97577232 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f280c01e9b0 con 0x55be0dfc4250 2024-03-15T13:06:21.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f281a42b740 10 client.17871 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:21.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.647+0000 7f281a42b740 1 -- 172.21.15.2:0/97577232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55be0deec8c0 con 0x55be0dfc4250 2024-03-15T13:06:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.648+0000 7f27f8ff9640 1 -- 172.21.15.2:0/97577232 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f2808032680 con 0x55be0dfc4250 2024-03-15T13:06:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.648+0000 7f27f8ff9640 1 -- 172.21.15.2:0/97577232 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f280c01a040 con 0x55be0dfc4250 2024-03-15T13:06:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.648+0000 7f281a42b740 10 client.17871 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.648+0000 7f281a42b740 10 client.17871 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:06:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.648+0000 7f281a42b740 20 client.17871 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.648+0000 7f281a42b740 1 --2- 172.21.15.2:0/97577232 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55be0e03a5a0 0x55be0e03c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.648+0000 7f281a42b740 4 client.17871 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:06:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.648+0000 7f281a42b740 1 -- 172.21.15.2:0/97577232 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be0dcb7890 con 0x55be0e03a5a0 2024-03-15T13:06:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.648+0000 7f2814d12640 1 --2- 172.21.15.2:0/97577232 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55be0e03a5a0 0x55be0e03c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.648+0000 7f27f8ff9640 10 client.17871 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:06:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.648+0000 7f2814d12640 1 --2- 172.21.15.2:0/97577232 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55be0e03a5a0 0x55be0e03c9c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5279f5e2403f22e server_cookie=febb515afccaecbb in_seq=0 out_seq=0 2024-03-15T13:06:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f27f8ff9640 1 -- 172.21.15.2:0/97577232 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be0dcb7890 con 0x55be0e03a5a0 2024-03-15T13:06:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f27f8ff9640 10 client.17871 handle_command_reply: tid=0 2024-03-15T13:06:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f281a42b740 1 client.17871 shutdown 2024-03-15T13:06:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f281a42b740 1 -- 172.21.15.2:0/97577232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4250 msgr2=0x55be0dfd52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f281a42b740 1 --2- 172.21.15.2:0/97577232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4250 0x55be0dfd52e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f280c0097d0 tx=0x7f280c003de0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f281a42b740 1 -- 172.21.15.2:0/97577232 shutdown_connections 2024-03-15T13:06:21.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f281a42b740 1 --2- 172.21.15.2:0/97577232 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55be0e03a5a0 0x55be0e03c9c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f281a42b740 1 --2- 172.21.15.2:0/97577232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be0dfcd750 0x55be0dfd1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f27f8ff9640 0 client.17871 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:06:21.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f281a42b740 1 --2- 172.21.15.2:0/97577232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be0dfc4b70 0x55be0dfd16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f281a42b740 1 --2- 172.21.15.2:0/97577232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be0dfc4250 0x55be0dfd52e0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f281a42b740 1 -- 172.21.15.2:0/97577232 >> 172.21.15.2:0/97577232 conn(0x55be0dfbe980 msgr2=0x55be0dfbfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f281a42b740 1 -- 172.21.15.2:0/97577232 shutdown_connections 2024-03-15T13:06:21.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f281a42b740 1 -- 172.21.15.2:0/97577232 wait complete. 2024-03-15T13:06:21.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f281a42b740 20 client.17871 trim_cache size 0 max 16384 2024-03-15T13:06:21.642 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:06:21.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f2817777640 1 -- 172.21.15.2:0/3664810868 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f27fc0a1ba0 msgr2=0x7f27fc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f2817777640 1 --2- 172.21.15.2:0/3664810868 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f27fc0a1ba0 0x7f27fc0a4060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2810198a70 tx=0x7f280c005f90 comp rx=0 tx=0).stop 2024-03-15T13:06:21.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f2817777640 1 -- 172.21.15.2:0/3664810868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28101056a0 msgr2=0x7f281019dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f2817777640 1 --2- 172.21.15.2:0/3664810868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28101056a0 0x7f281019dcc0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f280000e9c0 tx=0x7f280000ee80 comp rx=0 tx=0).stop 2024-03-15T13:06:21.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f2817777640 1 -- 172.21.15.2:0/3664810868 shutdown_connections 2024-03-15T13:06:21.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f2817777640 1 --2- 172.21.15.2:0/3664810868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2810197930 0x7f2810197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f2817777640 1 --2- 172.21.15.2:0/3664810868 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f27fc0a1ba0 0x7f27fc0a4060 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f2817777640 1 --2- 172.21.15.2:0/3664810868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28101056a0 0x7f281019dcc0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f2817777640 1 --2- 172.21.15.2:0/3664810868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f28101005d0 0x7f281019d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.649+0000 7f2817777640 1 -- 172.21.15.2:0/3664810868 >> 172.21.15.2:0/3664810868 conn(0x7f28100774f0 msgr2=0x7f28100fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.650+0000 7f2817777640 1 -- 172.21.15.2:0/3664810868 shutdown_connections 2024-03-15T13:06:21.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.650+0000 7f2817777640 1 -- 172.21.15.2:0/3664810868 wait complete. 2024-03-15T13:06:21.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:21 smithi002 ceph-mon[32060]: pgmap v227: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 202 B/s wr, 15 op/s 2024-03-15T13:06:21.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:21 smithi002 ceph-mon[32060]: osdmap e94: 12 total, 12 up, 12 in 2024-03-15T13:06:21.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:21 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3696823561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:06:21.645 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:06:21.645 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:06:21.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.765+0000 7f7fef023640 1 -- 172.21.15.2:0/3984775436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe81064b0 msgr2=0x7f7fe8106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.765+0000 7f7fef023640 1 --2- 172.21.15.2:0/3984775436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe81064b0 0x7f7fe8106890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009930 tx=0x7f7fdc02bc20 comp rx=0 tx=0).stop 2024-03-15T13:06:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.765+0000 7f7fef023640 1 -- 172.21.15.2:0/3984775436 shutdown_connections 2024-03-15T13:06:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.765+0000 7f7fef023640 1 --2- 172.21.15.2:0/3984775436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe8100dd0 0x7f7fe810d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.765+0000 7f7fef023640 1 --2- 172.21.15.2:0/3984775436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fe8100410 0x7f7fe8100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.765+0000 7f7fef023640 1 --2- 172.21.15.2:0/3984775436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe81064b0 0x7f7fe8106890 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.765+0000 7f7fef023640 1 -- 172.21.15.2:0/3984775436 >> 172.21.15.2:0/3984775436 conn(0x7f7fe8077550 msgr2=0x7f7fe8077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.765+0000 7f7fef023640 1 -- 172.21.15.2:0/3984775436 shutdown_connections 2024-03-15T13:06:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.765+0000 7f7fef023640 1 -- 172.21.15.2:0/3984775436 wait complete. 2024-03-15T13:06:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.765+0000 7f7fef023640 1 Processor -- start 2024-03-15T13:06:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.765+0000 7f7fef023640 1 -- start start 2024-03-15T13:06:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.765+0000 7f7fef023640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe8100410 0x7f7fe81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.765+0000 7f7fef023640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fe8100dd0 0x7f7fe8198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.765+0000 7f7fef023640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe81064b0 0x7f7fe81a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.765+0000 7f7fef023640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8199790 con 0x7f7fe81064b0 2024-03-15T13:06:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fef023640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe81a3f90 con 0x7f7fe8100dd0 2024-03-15T13:06:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fef023640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe81a40d0 con 0x7f7fe8100410 2024-03-15T13:06:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fed5c0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe81064b0 0x7f7fe81a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fecdbf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe8100410 0x7f7fe81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fecdbf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe8100410 0x7f7fe81989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:51928/0 (socket says 172.21.15.2:51928) 2024-03-15T13:06:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fecdbf640 1 -- 172.21.15.2:0/1990326584 learned_addr learned my addr 172.21.15.2:0/1990326584 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fe7fff640 1 --2- 172.21.15.2:0/1990326584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fe8100dd0 0x7f7fe8198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fed5c0640 1 -- 172.21.15.2:0/1990326584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe8100410 msgr2=0x7f7fe81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fed5c0640 1 --2- 172.21.15.2:0/1990326584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe8100410 0x7f7fe81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fed5c0640 1 -- 172.21.15.2:0/1990326584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fe8100dd0 msgr2=0x7f7fe8198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fed5c0640 1 --2- 172.21.15.2:0/1990326584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fe8100dd0 0x7f7fe8198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fed5c0640 1 -- 172.21.15.2:0/1990326584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc009560 con 0x7f7fe81064b0 2024-03-15T13:06:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fecdbf640 1 --2- 172.21.15.2:0/1990326584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe8100410 0x7f7fe81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fed5c0640 1 --2- 172.21.15.2:0/1990326584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe81064b0 0x7f7fe81a3a50 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000ea80 tx=0x7f7fe000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fe5ffb640 1 -- 172.21.15.2:0/1990326584 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000ce40 con 0x7f7fe81064b0 2024-03-15T13:06:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fef023640 1 -- 172.21.15.2:0/1990326584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe81a4330 con 0x7f7fe81064b0 2024-03-15T13:06:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fe5ffb640 1 -- 172.21.15.2:0/1990326584 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7fe0005020 con 0x7f7fe81064b0 2024-03-15T13:06:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.766+0000 7f7fe5ffb640 1 -- 172.21.15.2:0/1990326584 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0010670 con 0x7f7fe81064b0 2024-03-15T13:06:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.767+0000 7f7fef023640 1 -- 172.21.15.2:0/1990326584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe81a47f0 con 0x7f7fe81064b0 2024-03-15T13:06:21.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7fe5ffb640 1 -- 172.21.15.2:0/1990326584 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f7fe0005190 con 0x7f7fe81064b0 2024-03-15T13:06:21.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7ff1c02740 1 Processor -- start 2024-03-15T13:06:21.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7ff1c02740 1 -- start start 2024-03-15T13:06:21.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7fe5ffb640 1 --2- 172.21.15.2:0/1990326584 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7fd80a1ba0 0x7f7fd80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7ff1c02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb1f00 0x55bc68cb22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7ff1c02740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb2820 0x55bc68cbf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7ff1c02740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 0x55bc68cc1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7ff1c02740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc68c6d960 con 0x55bc68cb2820 2024-03-15T13:06:21.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7ff1c02740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc68c21310 con 0x55bc68cbfb60 2024-03-15T13:06:21.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7ff1c02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc68c1e840 con 0x55bc68cb1f00 2024-03-15T13:06:21.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7fe5ffb640 1 -- 172.21.15.2:0/1990326584 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7fe0014070 con 0x7f7fe81064b0 2024-03-15T13:06:21.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7fe7fff640 1 --2- 172.21.15.2:0/1990326584 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7fd80a1ba0 0x7f7fd80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7fe7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb2820 0x55bc68cbf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7fed5c0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 0x55bc68cc1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7fe7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb2820 0x55bc68cbf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34190/0 (socket says 172.21.15.2:34190) 2024-03-15T13:06:21.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7fe7fff640 1 -- 172.21.15.2:0/3593595001 learned_addr learned my addr 172.21.15.2:0/3593595001 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.769+0000 7f7fecdbf640 1 --2- 172.21.15.2:0/3593595001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb1f00 0x55bc68cb22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.770+0000 7f7fed5c0640 1 -- 172.21.15.2:0/3593595001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb1f00 msgr2=0x55bc68cb22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.770+0000 7f7fed5c0640 1 --2- 172.21.15.2:0/3593595001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb1f00 0x55bc68cb22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.770+0000 7f7fed5c0640 1 -- 172.21.15.2:0/3593595001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb2820 msgr2=0x55bc68cbf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.770+0000 7f7fed5c0640 1 --2- 172.21.15.2:0/3593595001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb2820 0x55bc68cbf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.770+0000 7f7fed5c0640 1 -- 172.21.15.2:0/3593595001 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc68bdc060 con 0x55bc68cbfb60 2024-03-15T13:06:21.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.770+0000 7f7fecdbf640 1 --2- 172.21.15.2:0/3593595001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb1f00 0x55bc68cb22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:21.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.770+0000 7f7fe7fff640 1 --2- 172.21.15.2:0/3593595001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb2820 0x55bc68cbf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:21.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.770+0000 7f7fe7fff640 1 --2- 172.21.15.2:0/1990326584 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7fd80a1ba0 0x7f7fd80a4060 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40119d0 tx=0x7f7fd4009c00 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.770+0000 7f7fed5c0640 1 --2- 172.21.15.2:0/3593595001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 0x55bc68cc1f50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00cb560 tx=0x7f7fe009a460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc7df0ce6ea415ba server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.770+0000 7f7fd2ffd640 1 -- 172.21.15.2:0/3593595001 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe009b140 con 0x55bc68cbfb60 2024-03-15T13:06:21.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.770+0000 7f7fd2ffd640 1 -- 172.21.15.2:0/3593595001 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7fe009aa40 con 0x55bc68cbfb60 2024-03-15T13:06:21.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.770+0000 7f7fd2ffd640 1 -- 172.21.15.2:0/3593595001 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe00a2d50 con 0x55bc68cbfb60 2024-03-15T13:06:21.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.771+0000 7f7ff1c02740 1 -- 172.21.15.2:0/3593595001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 msgr2=0x55bc68cc1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.771+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/3593595001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 0x55bc68cc1f50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00cb560 tx=0x7f7fe009a460 comp rx=0 tx=0).stop 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.771+0000 7f7ff1c02740 1 -- 172.21.15.2:0/3593595001 shutdown_connections 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.771+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/3593595001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 0x55bc68cc1f50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.771+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/3593595001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb2820 0x55bc68cbf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.771+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/3593595001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb1f00 0x55bc68cb22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.771+0000 7f7ff1c02740 1 -- 172.21.15.2:0/3593595001 >> 172.21.15.2:0/3593595001 conn(0x55bc68cac980 msgr2=0x55bc68cacd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.771+0000 7f7ff1c02740 1 -- 172.21.15.2:0/3593595001 shutdown_connections 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.771+0000 7f7ff1c02740 1 -- 172.21.15.2:0/3593595001 wait complete. 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.771+0000 7f7ff1c02740 1 -- start start 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.771+0000 7f7ff1c02740 1 Processor -- start 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7ff1c02740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb1f00 0x55bc68cbf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7ff1c02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb2820 0x55bc68cbc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7ff1c02740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 0x55bc68cbccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7ff1c02740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc68c1a020 con 0x55bc68cb1f00 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7ff1c02740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc688ed430 con 0x55bc68cbfb60 2024-03-15T13:06:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7ff1c02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc688a9e00 con 0x55bc68cb2820 2024-03-15T13:06:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7ff1c02740 10 client.0 _ll_register_callbacks cb 0x55bc68bc1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7fecdbf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb1f00 0x55bc68cbf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7fecdbf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb1f00 0x55bc68cbf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34196/0 (socket says 172.21.15.2:34196) 2024-03-15T13:06:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7fecdbf640 1 -- 172.21.15.2:0/3176945654 learned_addr learned my addr 172.21.15.2:0/3176945654 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7fed5c0640 1 --2- 172.21.15.2:0/3176945654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 0x55bc68cbccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7fe7fff640 1 --2- 172.21.15.2:0/3176945654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb2820 0x55bc68cbc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7fecdbf640 1 -- 172.21.15.2:0/3176945654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb2820 msgr2=0x55bc68cbc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7fecdbf640 1 --2- 172.21.15.2:0/3176945654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb2820 0x55bc68cbc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7fecdbf640 1 -- 172.21.15.2:0/3176945654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 msgr2=0x55bc68cbccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7fecdbf640 1 --2- 172.21.15.2:0/3176945654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 0x55bc68cbccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7fecdbf640 1 -- 172.21.15.2:0/3176945654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe81a47f0 con 0x55bc68cb1f00 2024-03-15T13:06:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7fecdbf640 1 --2- 172.21.15.2:0/3176945654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb1f00 0x55bc68cbf400 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc036040 tx=0x7f7fdc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b1e8c5178cd333 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7fc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7fc7fff640 1 -- 172.21.15.2:0/3176945654 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc004a70 con 0x55bc68cb1f00 2024-03-15T13:06:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7fc7fff640 1 -- 172.21.15.2:0/3176945654 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7fdc046070 con 0x55bc68cb1f00 2024-03-15T13:06:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.772+0000 7f7ff1c02740 1 -- 172.21.15.2:0/3176945654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc689197e0 con 0x55bc68cb1f00 2024-03-15T13:06:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.773+0000 7f7fc7fff640 1 -- 172.21.15.2:0/3176945654 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fdc004de0 con 0x55bc68cb1f00 2024-03-15T13:06:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.773+0000 7f7ff1c02740 10 client.17883 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.773+0000 7f7ff1c02740 1 -- 172.21.15.2:0/3176945654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55bc68c186a0 con 0x55bc68cb1f00 2024-03-15T13:06:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.773+0000 7f7fc7fff640 1 -- 172.21.15.2:0/3176945654 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7fdc053020 con 0x55bc68cb1f00 2024-03-15T13:06:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.773+0000 7f7fc7fff640 1 -- 172.21.15.2:0/3176945654 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7fdc03c040 con 0x55bc68cb1f00 2024-03-15T13:06:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.773+0000 7f7ff1c02740 10 client.17883 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.773+0000 7f7ff1c02740 10 client.17883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.773+0000 7f7ff1c02740 20 client.17883 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.773+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/3176945654 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bc68ccfc80 0x55bc68cd20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.773+0000 7f7ff1c02740 4 client.17883 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.773+0000 7f7ff1c02740 1 -- 172.21.15.2:0/3176945654 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc68956720 con 0x55bc68ccfc80 2024-03-15T13:06:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.773+0000 7f7fe7fff640 1 --2- 172.21.15.2:0/3176945654 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bc68ccfc80 0x55bc68cd20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.773+0000 7f7fc7fff640 10 client.17883 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.773+0000 7f7fe7fff640 1 --2- 172.21.15.2:0/3176945654 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bc68ccfc80 0x55bc68cd20a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f1018899737d289 server_cookie=21c188d5caf77423 in_seq=0 out_seq=0 2024-03-15T13:06:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.774+0000 7f7fc7fff640 1 -- 172.21.15.2:0/3176945654 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bc68956720 con 0x55bc68ccfc80 2024-03-15T13:06:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.774+0000 7f7fc7fff640 10 client.17883 handle_command_reply: tid=0 2024-03-15T13:06:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.774+0000 7f7ff1c02740 1 client.17883 shutdown 2024-03-15T13:06:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.774+0000 7f7ff1c02740 1 -- 172.21.15.2:0/3176945654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb1f00 msgr2=0x55bc68cbf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.774+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/3176945654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb1f00 0x55bc68cbf400 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc036040 tx=0x7f7fdc004270 comp rx=0 tx=0).stop 2024-03-15T13:06:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.774+0000 7f7ff1c02740 1 -- 172.21.15.2:0/3176945654 shutdown_connections 2024-03-15T13:06:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.774+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/3176945654 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bc68ccfc80 0x55bc68cd20a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.774+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/3176945654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 0x55bc68cbccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.774+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/3176945654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb2820 0x55bc68cbc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.774+0000 7f7fc7fff640 0 client.17883 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.774+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/3176945654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb1f00 0x55bc68cbf400 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.774+0000 7f7ff1c02740 1 -- 172.21.15.2:0/3176945654 >> 172.21.15.2:0/3176945654 conn(0x55bc68cac980 msgr2=0x55bc68cae6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.774+0000 7f7ff1c02740 1 -- 172.21.15.2:0/3176945654 shutdown_connections 2024-03-15T13:06:21.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.774+0000 7f7ff1c02740 1 -- 172.21.15.2:0/3176945654 wait complete. 2024-03-15T13:06:21.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.775+0000 7f7ff1c02740 20 client.17883 trim_cache size 0 max 16384 2024-03-15T13:06:21.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7ff1c02740 1 Processor -- start 2024-03-15T13:06:21.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7ff1c02740 1 -- start start 2024-03-15T13:06:21.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7ff1c02740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cb1f00 0x55bc68cbeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7ff1c02740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb2820 0x55bc68cbf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7ff1c02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cbfb60 0x55bc68ccaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7ff1c02740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc68b9f130 con 0x55bc68cb2820 2024-03-15T13:06:21.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7ff1c02740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc68cadbc0 con 0x55bc68cb1f00 2024-03-15T13:06:21.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7ff1c02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc68cb15f0 con 0x55bc68cbfb60 2024-03-15T13:06:21.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7fe7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb2820 0x55bc68cbf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7fe7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb2820 0x55bc68cbf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34202/0 (socket says 172.21.15.2:34202) 2024-03-15T13:06:21.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7fe7fff640 1 -- 172.21.15.2:0/1950572383 learned_addr learned my addr 172.21.15.2:0/1950572383 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7fecdbf640 1 --2- 172.21.15.2:0/1950572383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cb1f00 0x55bc68cbeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7fed5c0640 1 --2- 172.21.15.2:0/1950572383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cbfb60 0x55bc68ccaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7fe7fff640 1 -- 172.21.15.2:0/1950572383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cbfb60 msgr2=0x55bc68ccaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7fe7fff640 1 --2- 172.21.15.2:0/1950572383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cbfb60 0x55bc68ccaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7fe7fff640 1 -- 172.21.15.2:0/1950572383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cb1f00 msgr2=0x55bc68cbeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7fe7fff640 1 --2- 172.21.15.2:0/1950572383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cb1f00 0x55bc68cbeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.787+0000 7f7fe7fff640 1 -- 172.21.15.2:0/1950572383 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc68bb4c20 con 0x55bc68cb2820 2024-03-15T13:06:21.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.788+0000 7f7fe7fff640 1 --2- 172.21.15.2:0/1950572383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb2820 0x55bc68cbf400 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4000c00 tx=0x7f7fd4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8151f5505275e2d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.788+0000 7f7fd27fc640 1 -- 172.21.15.2:0/1950572383 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4016040 con 0x55bc68cb2820 2024-03-15T13:06:21.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.788+0000 7f7fd27fc640 1 -- 172.21.15.2:0/1950572383 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7fd400fe50 con 0x55bc68cb2820 2024-03-15T13:06:21.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.788+0000 7f7fd27fc640 1 -- 172.21.15.2:0/1950572383 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4004b20 con 0x55bc68cb2820 2024-03-15T13:06:21.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.788+0000 7f7ff1c02740 1 -- 172.21.15.2:0/1950572383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb2820 msgr2=0x55bc68cbf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.788+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/1950572383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb2820 0x55bc68cbf400 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4000c00 tx=0x7f7fd4021000 comp rx=0 tx=0).stop 2024-03-15T13:06:21.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.788+0000 7f7ff1c02740 1 -- 172.21.15.2:0/1950572383 shutdown_connections 2024-03-15T13:06:21.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.788+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/1950572383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cbfb60 0x55bc68ccaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.788+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/1950572383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb2820 0x55bc68cbf400 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.788+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/1950572383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cb1f00 0x55bc68cbeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.788+0000 7f7ff1c02740 1 -- 172.21.15.2:0/1950572383 >> 172.21.15.2:0/1950572383 conn(0x55bc68cac980 msgr2=0x55bc68cabb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.788+0000 7f7ff1c02740 1 -- 172.21.15.2:0/1950572383 shutdown_connections 2024-03-15T13:06:21.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.788+0000 7f7ff1c02740 1 -- 172.21.15.2:0/1950572383 wait complete. 2024-03-15T13:06:21.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7ff1c02740 1 -- start start 2024-03-15T13:06:21.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7ff1c02740 1 Processor -- start 2024-03-15T13:06:21.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7ff1c02740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb1f00 0x55bc68cba420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7ff1c02740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb2820 0x55bc68cc4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7ff1c02740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 0x55bc68cc4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7ff1c02740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc68cc55d0 con 0x55bc68cb1f00 2024-03-15T13:06:21.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7ff1c02740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc68cc5740 con 0x55bc68cbfb60 2024-03-15T13:06:21.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7ff1c02740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc68cc5900 con 0x55bc68cb2820 2024-03-15T13:06:21.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7ff1c02740 10 client.0 _ll_register_callbacks cb 0x7f7fdc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:21.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7fecdbf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb1f00 0x55bc68cba420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7fecdbf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb1f00 0x55bc68cba420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34208/0 (socket says 172.21.15.2:34208) 2024-03-15T13:06:21.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7fecdbf640 1 -- 172.21.15.2:0/2884015225 learned_addr learned my addr 172.21.15.2:0/2884015225 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:21.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7fe7fff640 1 --2- 172.21.15.2:0/2884015225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb2820 0x55bc68cc4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7fed5c0640 1 --2- 172.21.15.2:0/2884015225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 0x55bc68cc4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7fecdbf640 1 -- 172.21.15.2:0/2884015225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb2820 msgr2=0x55bc68cc4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7fecdbf640 1 --2- 172.21.15.2:0/2884015225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb2820 0x55bc68cc4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7fecdbf640 1 -- 172.21.15.2:0/2884015225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 msgr2=0x55bc68cc4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:21.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7fecdbf640 1 --2- 172.21.15.2:0/2884015225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 0x55bc68cc4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.789+0000 7f7fecdbf640 1 -- 172.21.15.2:0/2884015225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc68bda8c0 con 0x55bc68cb1f00 2024-03-15T13:06:21.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7fecdbf640 1 --2- 172.21.15.2:0/2884015225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb1f00 0x55bc68cba420 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc003a00 tx=0x7f7fdc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435dc2687396f768 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:21.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7fc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:21.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7fc7fff640 1 -- 172.21.15.2:0/2884015225 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc05b070 con 0x55bc68cb1f00 2024-03-15T13:06:21.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7fc7fff640 1 -- 172.21.15.2:0/2884015225 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7fdc046040 con 0x55bc68cb1f00 2024-03-15T13:06:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7ff1c02740 1 -- 172.21.15.2:0/2884015225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fe009aa40 con 0x55bc68cb1f00 2024-03-15T13:06:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7fc7fff640 1 -- 172.21.15.2:0/2884015225 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fdc005d50 con 0x55bc68cb1f00 2024-03-15T13:06:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7ff1c02740 10 client.17889 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7ff1c02740 1 -- 172.21.15.2:0/2884015225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55bc68bc1050 con 0x55bc68cb1f00 2024-03-15T13:06:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7fc7fff640 1 -- 172.21.15.2:0/2884015225 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7fe009aa40 con 0x55bc68cb1f00 2024-03-15T13:06:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7fc7fff640 1 -- 172.21.15.2:0/2884015225 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7fdc05e020 con 0x55bc68cb1f00 2024-03-15T13:06:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7ff1c02740 10 client.17889 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7ff1c02740 10 client.17889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7ff1c02740 20 client.17889 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/2884015225 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bc68d2c920 0x55bc68d2ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7ff1c02740 4 client.17889 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:06:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.790+0000 7f7ff1c02740 1 -- 172.21.15.2:0/2884015225 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bc689a5890 con 0x55bc68d2c920 2024-03-15T13:06:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7fe7fff640 1 --2- 172.21.15.2:0/2884015225 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bc68d2c920 0x55bc68d2ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7fc7fff640 10 client.17889 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7fe7fff640 1 --2- 172.21.15.2:0/2884015225 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bc68d2c920 0x55bc68d2ed40 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a477900abbf6b9 server_cookie=ffd432b2894c194c in_seq=0 out_seq=0 2024-03-15T13:06:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7fc7fff640 1 -- 172.21.15.2:0/2884015225 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bc689a5890 con 0x55bc68d2c920 2024-03-15T13:06:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7fc7fff640 10 client.17889 handle_command_reply: tid=0 2024-03-15T13:06:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7ff1c02740 1 client.17889 shutdown 2024-03-15T13:06:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7ff1c02740 1 -- 172.21.15.2:0/2884015225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb1f00 msgr2=0x55bc68cba420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/2884015225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb1f00 0x55bc68cba420 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc003a00 tx=0x7f7fdc005850 comp rx=0 tx=0).stop 2024-03-15T13:06:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7ff1c02740 1 -- 172.21.15.2:0/2884015225 shutdown_connections 2024-03-15T13:06:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/2884015225 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55bc68d2c920 0x55bc68d2ed40 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/2884015225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc68cbfb60 0x55bc68cc4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7fc7fff640 0 client.17889 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/2884015225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bc68cb2820 0x55bc68cc4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7ff1c02740 1 --2- 172.21.15.2:0/2884015225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bc68cb1f00 0x55bc68cba420 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7ff1c02740 1 -- 172.21.15.2:0/2884015225 >> 172.21.15.2:0/2884015225 conn(0x55bc68cac980 msgr2=0x55bc68cab2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7ff1c02740 1 -- 172.21.15.2:0/2884015225 shutdown_connections 2024-03-15T13:06:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7ff1c02740 1 -- 172.21.15.2:0/2884015225 wait complete. 2024-03-15T13:06:21.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.791+0000 7f7ff1c02740 20 client.17889 trim_cache size 0 max 16384 2024-03-15T13:06:21.783 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:06:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.792+0000 7f7fef023640 1 -- 172.21.15.2:0/1990326584 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7fd80a1ba0 msgr2=0x7f7fd80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.792+0000 7f7fef023640 1 --2- 172.21.15.2:0/1990326584 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7fd80a1ba0 0x7f7fd80a4060 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40119d0 tx=0x7f7fd4009c00 comp rx=0 tx=0).stop 2024-03-15T13:06:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.792+0000 7f7fef023640 1 -- 172.21.15.2:0/1990326584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe81064b0 msgr2=0x7f7fe81a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.792+0000 7f7fef023640 1 --2- 172.21.15.2:0/1990326584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe81064b0 0x7f7fe81a3a50 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000ea80 tx=0x7f7fe000ef40 comp rx=0 tx=0).stop 2024-03-15T13:06:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.792+0000 7f7fef023640 1 -- 172.21.15.2:0/1990326584 shutdown_connections 2024-03-15T13:06:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.792+0000 7f7fef023640 1 --2- 172.21.15.2:0/1990326584 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7fd80a1ba0 0x7f7fd80a4060 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.792+0000 7f7fef023640 1 --2- 172.21.15.2:0/1990326584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fe81064b0 0x7f7fe81a3a50 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.792+0000 7f7fef023640 1 --2- 172.21.15.2:0/1990326584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fe8100dd0 0x7f7fe8198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.792+0000 7f7fef023640 1 --2- 172.21.15.2:0/1990326584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fe8100410 0x7f7fe81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.792+0000 7f7fef023640 1 -- 172.21.15.2:0/1990326584 >> 172.21.15.2:0/1990326584 conn(0x7f7fe8077550 msgr2=0x7f7fe8103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.792+0000 7f7fef023640 1 -- 172.21.15.2:0/1990326584 shutdown_connections 2024-03-15T13:06:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:21.792+0000 7f7fef023640 1 -- 172.21.15.2:0/1990326584 wait complete. 2024-03-15T13:06:21.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:21 smithi184 ceph-mon[35993]: pgmap v227: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 202 B/s wr, 15 op/s 2024-03-15T13:06:21.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:21 smithi184 ceph-mon[35993]: osdmap e94: 12 total, 12 up, 12 in 2024-03-15T13:06:21.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:21 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3696823561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:06:21.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:21 smithi149 ceph-mon[35927]: pgmap v227: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 202 B/s wr, 15 op/s 2024-03-15T13:06:21.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:21 smithi149 ceph-mon[35927]: osdmap e94: 12 total, 12 up, 12 in 2024-03-15T13:06:21.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:21 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3696823561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:06:22.788 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:06:22.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:06:22.788 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:06:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.922+0000 7fbf45c78640 1 -- 172.21.15.2:0/916806015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf40107570 msgr2=0x7fbf40107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.922+0000 7fbf45c78640 1 --2- 172.21.15.2:0/916806015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf40107570 0x7fbf40107950 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009900 tx=0x7fbf3002bc30 comp rx=0 tx=0).stop 2024-03-15T13:06:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.922+0000 7fbf45c78640 1 -- 172.21.15.2:0/916806015 shutdown_connections 2024-03-15T13:06:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.922+0000 7fbf45c78640 1 --2- 172.21.15.2:0/916806015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf40101e70 0x7fbf4010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.922+0000 7fbf45c78640 1 --2- 172.21.15.2:0/916806015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf401014b0 0x7fbf40101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.922+0000 7fbf45c78640 1 --2- 172.21.15.2:0/916806015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf40107570 0x7fbf40107950 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.922+0000 7fbf45c78640 1 -- 172.21.15.2:0/916806015 >> 172.21.15.2:0/916806015 conn(0x7fbf400fcf20 msgr2=0x7fbf400ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.922+0000 7fbf45c78640 1 -- 172.21.15.2:0/916806015 shutdown_connections 2024-03-15T13:06:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.923+0000 7fbf45c78640 1 -- 172.21.15.2:0/916806015 wait complete. 2024-03-15T13:06:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.923+0000 7fbf45c78640 1 Processor -- start 2024-03-15T13:06:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.923+0000 7fbf45c78640 1 -- start start 2024-03-15T13:06:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.923+0000 7fbf45c78640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf401014b0 0x7fbf40199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.923+0000 7fbf45c78640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf40101e70 0x7fbf4019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.923+0000 7fbf45c78640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf40107570 0x7fbf4019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.923+0000 7fbf45c78640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4019ec00 con 0x7fbf401014b0 2024-03-15T13:06:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.923+0000 7fbf45c78640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4019ed70 con 0x7fbf40107570 2024-03-15T13:06:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.923+0000 7fbf45c78640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4019ef30 con 0x7fbf40101e70 2024-03-15T13:06:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf3f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf401014b0 0x7fbf40199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf3f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf401014b0 0x7fbf40199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34228/0 (socket says 172.21.15.2:34228) 2024-03-15T13:06:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf3f7fe640 1 -- 172.21.15.2:0/3278643613 learned_addr learned my addr 172.21.15.2:0/3278643613 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf3ffff640 1 --2- 172.21.15.2:0/3278643613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf40107570 0x7fbf4019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf3effd640 1 --2- 172.21.15.2:0/3278643613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf40101e70 0x7fbf4019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf3f7fe640 1 -- 172.21.15.2:0/3278643613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf40101e70 msgr2=0x7fbf4019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf3f7fe640 1 --2- 172.21.15.2:0/3278643613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf40101e70 0x7fbf4019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf3f7fe640 1 -- 172.21.15.2:0/3278643613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf40107570 msgr2=0x7fbf4019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf3f7fe640 1 --2- 172.21.15.2:0/3278643613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf40107570 0x7fbf4019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf3f7fe640 1 -- 172.21.15.2:0/3278643613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf30009560 con 0x7fbf401014b0 2024-03-15T13:06:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf3f7fe640 1 --2- 172.21.15.2:0/3278643613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf401014b0 0x7fbf40199d40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009230 tx=0x7fbf3002cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf3cff9640 1 -- 172.21.15.2:0/3278643613 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30004250 con 0x7fbf401014b0 2024-03-15T13:06:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf45c78640 1 -- 172.21.15.2:0/3278643613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4019f1b0 con 0x7fbf401014b0 2024-03-15T13:06:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf3cff9640 1 -- 172.21.15.2:0/3278643613 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbf300043b0 con 0x7fbf401014b0 2024-03-15T13:06:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf3cff9640 1 -- 172.21.15.2:0/3278643613 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30003bf0 con 0x7fbf401014b0 2024-03-15T13:06:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.924+0000 7fbf45c78640 1 -- 172.21.15.2:0/3278643613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf401a6a90 con 0x7fbf401014b0 2024-03-15T13:06:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.927+0000 7fbf3cff9640 1 -- 172.21.15.2:0/3278643613 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fbf30003d50 con 0x7fbf401014b0 2024-03-15T13:06:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.927+0000 7fbf4884c740 1 Processor -- start 2024-03-15T13:06:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.927+0000 7fbf4884c740 1 -- start start 2024-03-15T13:06:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.927+0000 7fbf3cff9640 1 --2- 172.21.15.2:0/3278643613 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbf280a1ba0 0x7fbf280a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.927+0000 7fbf4884c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0f0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.927+0000 7fbf4884c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f0d30 0x55604f0f92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf4884c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 0x55604f0fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf4884c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604f05ef90 con 0x55604f0f0380 2024-03-15T13:06:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf3ffff640 1 --2- 172.21.15.2:0/3278643613 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbf280a1ba0 0x7fbf280a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf4884c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604f05c4c0 con 0x55604f0f0d30 2024-03-15T13:06:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf4884c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604f057ca0 con 0x55604f0f98b0 2024-03-15T13:06:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf3cff9640 1 -- 172.21.15.2:0/3278643613 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fbf300efaa0 con 0x7fbf401014b0 2024-03-15T13:06:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf3f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf3f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0f0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34240/0 (socket says 172.21.15.2:34240) 2024-03-15T13:06:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf3f7fe640 1 -- 172.21.15.2:0/3571004826 learned_addr learned my addr 172.21.15.2:0/3571004826 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf3effd640 1 --2- 172.21.15.2:0/3571004826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f0d30 0x55604f0f92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf3ffff640 1 --2- 172.21.15.2:0/3571004826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 0x55604f0fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf3f7fe640 1 -- 172.21.15.2:0/3571004826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 msgr2=0x55604f0fbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf3f7fe640 1 --2- 172.21.15.2:0/3571004826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 0x55604f0fbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf3f7fe640 1 -- 172.21.15.2:0/3571004826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f0d30 msgr2=0x55604f0f92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf3f7fe640 1 --2- 172.21.15.2:0/3571004826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f0d30 0x55604f0f92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf3f7fe640 1 -- 172.21.15.2:0/3571004826 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55604f01a0b0 con 0x55604f0f0380 2024-03-15T13:06:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf3ffff640 1 --2- 172.21.15.2:0/3278643613 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbf280a1ba0 0x7fbf280a4060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbf38005fd0 tx=0x7fbf38013ae0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf3f7fe640 1 --2- 172.21.15.2:0/3571004826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0f0760 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fbf30005940 tx=0x7fbf300b86e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6cfd5d11287664 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf25ffb640 1 -- 172.21.15.2:0/3571004826 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300bbf80 con 0x55604f0f0380 2024-03-15T13:06:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf25ffb640 1 -- 172.21.15.2:0/3571004826 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbf300bbda0 con 0x55604f0f0380 2024-03-15T13:06:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.928+0000 7fbf25ffb640 1 -- 172.21.15.2:0/3571004826 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300b8ab0 con 0x55604f0f0380 2024-03-15T13:06:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.929+0000 7fbf4884c740 1 -- 172.21.15.2:0/3571004826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 msgr2=0x55604f0f0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.929+0000 7fbf4884c740 1 --2- 172.21.15.2:0/3571004826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0f0760 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fbf30005940 tx=0x7fbf300b86e0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.929+0000 7fbf4884c740 1 -- 172.21.15.2:0/3571004826 shutdown_connections 2024-03-15T13:06:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.929+0000 7fbf4884c740 1 --2- 172.21.15.2:0/3571004826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 0x55604f0fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.929+0000 7fbf4884c740 1 --2- 172.21.15.2:0/3571004826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f0d30 0x55604f0f92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.929+0000 7fbf4884c740 1 --2- 172.21.15.2:0/3571004826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0f0760 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.929+0000 7fbf4884c740 1 -- 172.21.15.2:0/3571004826 >> 172.21.15.2:0/3571004826 conn(0x55604f0ea840 msgr2=0x55604f0eac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.929+0000 7fbf4884c740 1 -- 172.21.15.2:0/3571004826 shutdown_connections 2024-03-15T13:06:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.929+0000 7fbf4884c740 1 -- 172.21.15.2:0/3571004826 wait complete. 2024-03-15T13:06:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.929+0000 7fbf4884c740 1 -- start start 2024-03-15T13:06:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf4884c740 1 Processor -- start 2024-03-15T13:06:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf4884c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf4884c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f0d30 0x55604f0f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf4884c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 0x55604f0f7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf4884c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604effcf40 con 0x55604f0f0380 2024-03-15T13:06:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf4884c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604ed2b4b0 con 0x55604f0f0d30 2024-03-15T13:06:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf4884c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604efdd670 con 0x55604f0f98b0 2024-03-15T13:06:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf4884c740 10 client.0 _ll_register_callbacks cb 0x55604f04af70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf3f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf3f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34248/0 (socket says 172.21.15.2:34248) 2024-03-15T13:06:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf3f7fe640 1 -- 172.21.15.2:0/1810183987 learned_addr learned my addr 172.21.15.2:0/1810183987 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf3effd640 1 --2- 172.21.15.2:0/1810183987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f0d30 0x55604f0f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf3ffff640 1 --2- 172.21.15.2:0/1810183987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 0x55604f0f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf3ffff640 1 -- 172.21.15.2:0/1810183987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f0d30 msgr2=0x55604f0f6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf3ffff640 1 --2- 172.21.15.2:0/1810183987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f0d30 0x55604f0f6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf3ffff640 1 -- 172.21.15.2:0/1810183987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 msgr2=0x55604f0f66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf3ffff640 1 --2- 172.21.15.2:0/1810183987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0f66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf3ffff640 1 -- 172.21.15.2:0/1810183987 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf401a6a90 con 0x55604f0f98b0 2024-03-15T13:06:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.930+0000 7fbf3f7fe640 1 --2- 172.21.15.2:0/1810183987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf3ffff640 1 --2- 172.21.15.2:0/1810183987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 0x55604f0f7150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbf38003d20 tx=0x7fbf38004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b9fd352a558a81c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf1effd640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:06:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf1effd640 1 -- 172.21.15.2:0/1810183987 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38004250 con 0x55604f0f98b0 2024-03-15T13:06:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf4884c740 1 -- 172.21.15.2:0/1810183987 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55604f031f40 con 0x55604f0f98b0 2024-03-15T13:06:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf1effd640 1 -- 172.21.15.2:0/1810183987 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fbf380043b0 con 0x55604f0f98b0 2024-03-15T13:06:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf1effd640 1 -- 172.21.15.2:0/1810183987 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf38004570 con 0x55604f0f98b0 2024-03-15T13:06:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf4884c740 10 client.27694 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf4884c740 1 -- 172.21.15.2:0/1810183987 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55604f056320 con 0x55604f0f98b0 2024-03-15T13:06:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf1effd640 1 -- 172.21.15.2:0/1810183987 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55604f031f40 con 0x55604f0f98b0 2024-03-15T13:06:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf1effd640 1 -- 172.21.15.2:0/1810183987 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbf38020420 con 0x55604f0f98b0 2024-03-15T13:06:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf4884c740 10 client.27694 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf4884c740 10 client.27694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf4884c740 20 client.27694 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf4884c740 1 --2- 172.21.15.2:0/1810183987 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55604f109c50 0x55604f10c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf4884c740 4 client.27694 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.931+0000 7fbf4884c740 1 -- 172.21.15.2:0/1810183987 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55604f04e310 con 0x55604f109c50 2024-03-15T13:06:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.932+0000 7fbf3f7fe640 1 --2- 172.21.15.2:0/1810183987 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55604f109c50 0x55604f10c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.932+0000 7fbf3f7fe640 1 --2- 172.21.15.2:0/1810183987 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55604f109c50 0x55604f10c070 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c575f4c272daf1 server_cookie=19a03e23e81ebe58 in_seq=0 out_seq=0 2024-03-15T13:06:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.932+0000 7fbf1effd640 10 client.27694 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.933+0000 7fbf1effd640 1 -- 172.21.15.2:0/1810183987 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55604f04e310 con 0x55604f109c50 2024-03-15T13:06:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.933+0000 7fbf1effd640 10 client.27694 handle_command_reply: tid=0 2024-03-15T13:06:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.933+0000 7fbf4884c740 1 client.27694 shutdown 2024-03-15T13:06:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.933+0000 7fbf4884c740 1 -- 172.21.15.2:0/1810183987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 msgr2=0x55604f0f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.933+0000 7fbf4884c740 1 --2- 172.21.15.2:0/1810183987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 0x55604f0f7150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbf38003d20 tx=0x7fbf38004ab0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.933+0000 7fbf4884c740 1 -- 172.21.15.2:0/1810183987 shutdown_connections 2024-03-15T13:06:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.933+0000 7fbf4884c740 1 --2- 172.21.15.2:0/1810183987 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55604f109c50 0x55604f10c070 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.933+0000 7fbf4884c740 1 --2- 172.21.15.2:0/1810183987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 0x55604f0f7150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.933+0000 7fbf4884c740 1 --2- 172.21.15.2:0/1810183987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f0d30 0x55604f0f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.933+0000 7fbf4884c740 1 --2- 172.21.15.2:0/1810183987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.933+0000 7fbf1effd640 0 client.27694 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.933+0000 7fbf4884c740 1 -- 172.21.15.2:0/1810183987 >> 172.21.15.2:0/1810183987 conn(0x55604f0ea840 msgr2=0x55604f0ec7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.933+0000 7fbf4884c740 1 -- 172.21.15.2:0/1810183987 shutdown_connections 2024-03-15T13:06:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.933+0000 7fbf4884c740 1 -- 172.21.15.2:0/1810183987 wait complete. 2024-03-15T13:06:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.933+0000 7fbf4884c740 20 client.27694 trim_cache size 0 max 16384 2024-03-15T13:06:22.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf4884c740 1 Processor -- start 2024-03-15T13:06:22.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf4884c740 1 -- start start 2024-03-15T13:06:22.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf4884c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f0380 0x55604f0f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf4884c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0d30 0x55604f0f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf4884c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 0x55604f0f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf4884c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604f0ef770 con 0x55604f0f0d30 2024-03-15T13:06:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf4884c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604f0e61b0 con 0x55604f0f0380 2024-03-15T13:06:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf4884c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604f0ea060 con 0x55604f0f98b0 2024-03-15T13:06:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf3effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0d30 0x55604f0f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf3effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0d30 0x55604f0f1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34264/0 (socket says 172.21.15.2:34264) 2024-03-15T13:06:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf3effd640 1 -- 172.21.15.2:0/217532442 learned_addr learned my addr 172.21.15.2:0/217532442 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf3ffff640 1 --2- 172.21.15.2:0/217532442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 0x55604f0f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf3f7fe640 1 --2- 172.21.15.2:0/217532442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f0380 0x55604f0f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf3effd640 1 -- 172.21.15.2:0/217532442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 msgr2=0x55604f0f6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf3effd640 1 --2- 172.21.15.2:0/217532442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 0x55604f0f6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf3effd640 1 -- 172.21.15.2:0/217532442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f0380 msgr2=0x55604f0f8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf3effd640 1 --2- 172.21.15.2:0/217532442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f0380 0x55604f0f8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf3effd640 1 -- 172.21.15.2:0/217532442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55604eff2c00 con 0x55604f0f0d30 2024-03-15T13:06:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf3ffff640 1 --2- 172.21.15.2:0/217532442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 0x55604f0f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.950+0000 7fbf3effd640 1 --2- 172.21.15.2:0/217532442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0d30 0x55604f0f1e20 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fbf4010e080 tx=0x7fbf34005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2defec480d43d73b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.951+0000 7fbf257fa640 1 -- 172.21.15.2:0/217532442 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34017070 con 0x55604f0f0d30 2024-03-15T13:06:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.951+0000 7fbf257fa640 1 -- 172.21.15.2:0/217532442 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbf34018040 con 0x55604f0f0d30 2024-03-15T13:06:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.951+0000 7fbf257fa640 1 -- 172.21.15.2:0/217532442 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34003e80 con 0x55604f0f0d30 2024-03-15T13:06:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.951+0000 7fbf4884c740 1 -- 172.21.15.2:0/217532442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0d30 msgr2=0x55604f0f1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.951+0000 7fbf4884c740 1 --2- 172.21.15.2:0/217532442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0d30 0x55604f0f1e20 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fbf4010e080 tx=0x7fbf34005f90 comp rx=0 tx=0).stop 2024-03-15T13:06:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.951+0000 7fbf4884c740 1 -- 172.21.15.2:0/217532442 shutdown_connections 2024-03-15T13:06:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.951+0000 7fbf4884c740 1 --2- 172.21.15.2:0/217532442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f98b0 0x55604f0f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.951+0000 7fbf4884c740 1 --2- 172.21.15.2:0/217532442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0d30 0x55604f0f1e20 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.951+0000 7fbf4884c740 1 --2- 172.21.15.2:0/217532442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f0380 0x55604f0f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.951+0000 7fbf4884c740 1 -- 172.21.15.2:0/217532442 >> 172.21.15.2:0/217532442 conn(0x55604f0ea840 msgr2=0x55604f0ebe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.951+0000 7fbf4884c740 1 -- 172.21.15.2:0/217532442 shutdown_connections 2024-03-15T13:06:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.951+0000 7fbf4884c740 1 -- 172.21.15.2:0/217532442 wait complete. 2024-03-15T13:06:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.951+0000 7fbf4884c740 1 -- start start 2024-03-15T13:06:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf4884c740 1 Processor -- start 2024-03-15T13:06:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf4884c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf4884c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f0d30 0x55604f0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf4884c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f98b0 0x55604f0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf4884c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604f164510 con 0x55604f0f0380 2024-03-15T13:06:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf4884c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604f164680 con 0x55604f0f98b0 2024-03-15T13:06:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf4884c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55604f164840 con 0x55604f0f0d30 2024-03-15T13:06:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf4884c740 10 client.0 _ll_register_callbacks cb 0x7fbf38003a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf3f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf3f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34270/0 (socket says 172.21.15.2:34270) 2024-03-15T13:06:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf3f7fe640 1 -- 172.21.15.2:0/744764233 learned_addr learned my addr 172.21.15.2:0/744764233 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf3effd640 1 --2- 172.21.15.2:0/744764233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f0d30 0x55604f0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf3ffff640 1 --2- 172.21.15.2:0/744764233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f98b0 0x55604f0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf3f7fe640 1 -- 172.21.15.2:0/744764233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f0d30 msgr2=0x55604f0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf3f7fe640 1 --2- 172.21.15.2:0/744764233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f0d30 0x55604f0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf3f7fe640 1 -- 172.21.15.2:0/744764233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f98b0 msgr2=0x55604f0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf3f7fe640 1 --2- 172.21.15.2:0/744764233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f98b0 0x55604f0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.952+0000 7fbf3f7fe640 1 -- 172.21.15.2:0/744764233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55604f056320 con 0x55604f0f0380 2024-03-15T13:06:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf3f7fe640 1 --2- 172.21.15.2:0/744764233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0fcc80 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fbf300b88a0 tx=0x7fbf300b8810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d044c8956a3b60 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf1effd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf1effd640 1 -- 172.21.15.2:0/744764233 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300bc210 con 0x55604f0f0380 2024-03-15T13:06:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf1effd640 1 -- 172.21.15.2:0/744764233 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fbf300bcb40 con 0x55604f0f0380 2024-03-15T13:06:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf4884c740 1 -- 172.21.15.2:0/744764233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55604f07d900 con 0x55604f0f0380 2024-03-15T13:06:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf1effd640 1 -- 172.21.15.2:0/744764233 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf300bbf80 con 0x55604f0f0380 2024-03-15T13:06:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf4884c740 10 client.17907 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf4884c740 1 -- 172.21.15.2:0/744764233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7fbf34009b10 con 0x55604f0f0380 2024-03-15T13:06:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf1effd640 1 -- 172.21.15.2:0/744764233 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55604f07d900 con 0x55604f0f0380 2024-03-15T13:06:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf1effd640 1 -- 172.21.15.2:0/744764233 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbf3003c070 con 0x55604f0f0380 2024-03-15T13:06:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf4884c740 10 client.17907 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf4884c740 10 client.17907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf4884c740 20 client.17907 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf4884c740 1 --2- 172.21.15.2:0/744764233 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55604f165cb0 0x55604f1680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf4884c740 4 client.17907 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:06:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf4884c740 1 -- 172.21.15.2:0/744764233 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55604ed57860 con 0x55604f165cb0 2024-03-15T13:06:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.953+0000 7fbf3effd640 1 --2- 172.21.15.2:0/744764233 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55604f165cb0 0x55604f1680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.954+0000 7fbf3effd640 1 --2- 172.21.15.2:0/744764233 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55604f165cb0 0x55604f1680d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f2092810aad4e server_cookie=d209a739275fe969 in_seq=0 out_seq=0 2024-03-15T13:06:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.954+0000 7fbf1effd640 10 client.17907 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.955+0000 7fbf1effd640 1 -- 172.21.15.2:0/744764233 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55604ed57860 con 0x55604f165cb0 2024-03-15T13:06:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.955+0000 7fbf1effd640 10 client.17907 handle_command_reply: tid=0 2024-03-15T13:06:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.955+0000 7fbf4884c740 1 client.17907 shutdown 2024-03-15T13:06:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.955+0000 7fbf4884c740 1 -- 172.21.15.2:0/744764233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 msgr2=0x55604f0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.955+0000 7fbf4884c740 1 --2- 172.21.15.2:0/744764233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0fcc80 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fbf300b88a0 tx=0x7fbf300b8810 comp rx=0 tx=0).stop 2024-03-15T13:06:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.955+0000 7fbf4884c740 1 -- 172.21.15.2:0/744764233 shutdown_connections 2024-03-15T13:06:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.955+0000 7fbf4884c740 1 --2- 172.21.15.2:0/744764233 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55604f165cb0 0x55604f1680d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.955+0000 7fbf4884c740 1 --2- 172.21.15.2:0/744764233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55604f0f98b0 0x55604f0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.955+0000 7fbf4884c740 1 --2- 172.21.15.2:0/744764233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55604f0f0d30 0x55604f0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.955+0000 7fbf1effd640 0 client.17907 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:22.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.955+0000 7fbf4884c740 1 --2- 172.21.15.2:0/744764233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55604f0f0380 0x55604f0fcc80 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.955+0000 7fbf4884c740 1 -- 172.21.15.2:0/744764233 >> 172.21.15.2:0/744764233 conn(0x55604f0ea840 msgr2=0x55604f0ebd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:22.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.955+0000 7fbf4884c740 1 -- 172.21.15.2:0/744764233 shutdown_connections 2024-03-15T13:06:22.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.955+0000 7fbf4884c740 1 -- 172.21.15.2:0/744764233 wait complete. 2024-03-15T13:06:22.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.955+0000 7fbf4884c740 20 client.17907 trim_cache size 0 max 16384 2024-03-15T13:06:22.947 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:06:22.947 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:06:22.947 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "93f85f19-f2e1-4b20-8a5b-a54aeac326f6", 2024-03-15T13:06:22.947 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:06:22.947 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:06:22.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.956+0000 7fbf45c78640 1 -- 172.21.15.2:0/3278643613 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbf280a1ba0 msgr2=0x7fbf280a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:22.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.956+0000 7fbf45c78640 1 --2- 172.21.15.2:0/3278643613 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbf280a1ba0 0x7fbf280a4060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbf38005fd0 tx=0x7fbf38013ae0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.956+0000 7fbf45c78640 1 -- 172.21.15.2:0/3278643613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf401014b0 msgr2=0x7fbf40199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:22.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.956+0000 7fbf45c78640 1 --2- 172.21.15.2:0/3278643613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf401014b0 0x7fbf40199d40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009230 tx=0x7fbf3002cac0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.956+0000 7fbf45c78640 1 -- 172.21.15.2:0/3278643613 shutdown_connections 2024-03-15T13:06:22.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.956+0000 7fbf45c78640 1 --2- 172.21.15.2:0/3278643613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf40107570 0x7fbf4019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.956+0000 7fbf45c78640 1 --2- 172.21.15.2:0/3278643613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf40101e70 0x7fbf4019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.956+0000 7fbf45c78640 1 --2- 172.21.15.2:0/3278643613 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbf280a1ba0 0x7fbf280a4060 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.956+0000 7fbf45c78640 1 --2- 172.21.15.2:0/3278643613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf401014b0 0x7fbf40199d40 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:22.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.956+0000 7fbf45c78640 1 -- 172.21.15.2:0/3278643613 >> 172.21.15.2:0/3278643613 conn(0x7fbf400fcf20 msgr2=0x7fbf400fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:22.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.956+0000 7fbf45c78640 1 -- 172.21.15.2:0/3278643613 shutdown_connections 2024-03-15T13:06:22.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:22.956+0000 7fbf45c78640 1 -- 172.21.15.2:0/3278643613 wait complete. 2024-03-15T13:06:22.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:06:22.952 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:06:23.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.072+0000 7f1f672d4640 1 -- 172.21.15.2:0/3256024247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f60101000 msgr2=0x7f1f6010d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:23.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.072+0000 7f1f672d4640 1 --2- 172.21.15.2:0/3256024247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f60101000 0x7f1f6010d200 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009ca0 tx=0x7f1f5002c140 comp rx=0 tx=0).stop 2024-03-15T13:06:23.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.072+0000 7f1f672d4640 1 -- 172.21.15.2:0/3256024247 shutdown_connections 2024-03-15T13:06:23.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.072+0000 7f1f672d4640 1 --2- 172.21.15.2:0/3256024247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f60101000 0x7f1f6010d200 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.072+0000 7f1f672d4640 1 --2- 172.21.15.2:0/3256024247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f60100640 0x7f1f60100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.072+0000 7f1f672d4640 1 --2- 172.21.15.2:0/3256024247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f601064d0 0x7f1f601068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.072+0000 7f1f672d4640 1 -- 172.21.15.2:0/3256024247 >> 172.21.15.2:0/3256024247 conn(0x7f1f600771d0 msgr2=0x7f1f600775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:23.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.072+0000 7f1f672d4640 1 -- 172.21.15.2:0/3256024247 shutdown_connections 2024-03-15T13:06:23.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.072+0000 7f1f672d4640 1 -- 172.21.15.2:0/3256024247 wait complete. 2024-03-15T13:06:23.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f672d4640 1 Processor -- start 2024-03-15T13:06:23.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f672d4640 1 -- start start 2024-03-15T13:06:23.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f672d4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f60100640 0x7f1f6019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f672d4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f60101000 0x7f1f6019dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f672d4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f601064d0 0x7f1f60197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f672d4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f60197ee0 con 0x7f1f601064d0 2024-03-15T13:06:23.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f672d4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f60198050 con 0x7f1f60101000 2024-03-15T13:06:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f672d4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f60198210 con 0x7f1f60100640 2024-03-15T13:06:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f65871640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f601064d0 0x7f1f60197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f65871640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f601064d0 0x7f1f60197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34300/0 (socket says 172.21.15.2:34300) 2024-03-15T13:06:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f65871640 1 -- 172.21.15.2:0/4069901609 learned_addr learned my addr 172.21.15.2:0/4069901609 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f6486f640 1 --2- 172.21.15.2:0/4069901609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f60101000 0x7f1f6019dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f65070640 1 --2- 172.21.15.2:0/4069901609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f60100640 0x7f1f6019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f65871640 1 -- 172.21.15.2:0/4069901609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f60100640 msgr2=0x7f1f6019d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f65871640 1 --2- 172.21.15.2:0/4069901609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f60100640 0x7f1f6019d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f65871640 1 -- 172.21.15.2:0/4069901609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f60101000 msgr2=0x7f1f6019dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f65871640 1 --2- 172.21.15.2:0/4069901609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f60101000 0x7f1f6019dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f65871640 1 -- 172.21.15.2:0/4069901609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f50009900 con 0x7f1f601064d0 2024-03-15T13:06:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.073+0000 7f1f65871640 1 --2- 172.21.15.2:0/4069901609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f601064d0 0x7f1f60197890 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f1f50005cc0 tx=0x7f1f50034a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.074+0000 7f1f5e7fc640 1 -- 172.21.15.2:0/4069901609 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50032880 con 0x7f1f601064d0 2024-03-15T13:06:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.074+0000 7f1f5e7fc640 1 -- 172.21.15.2:0/4069901609 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1f50003e40 con 0x7f1f601064d0 2024-03-15T13:06:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.074+0000 7f1f672d4640 1 -- 172.21.15.2:0/4069901609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f60198490 con 0x7f1f601064d0 2024-03-15T13:06:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.074+0000 7f1f5e7fc640 1 -- 172.21.15.2:0/4069901609 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50040450 con 0x7f1f601064d0 2024-03-15T13:06:23.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.074+0000 7f1f672d4640 1 -- 172.21.15.2:0/4069901609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f601a4780 con 0x7f1f601064d0 2024-03-15T13:06:23.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f5e7fc640 1 -- 172.21.15.2:0/4069901609 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f1f5000a040 con 0x7f1f601064d0 2024-03-15T13:06:23.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f69f90740 1 Processor -- start 2024-03-15T13:06:23.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f69f90740 1 -- start start 2024-03-15T13:06:23.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f69f90740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e42b0 0x563f721e4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f69f90740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721e4bd0 0x563f721ed270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f69f90740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721ed7b0 0x563f721efba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f69f90740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f72153280 con 0x563f721e42b0 2024-03-15T13:06:23.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f69f90740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f721507b0 con 0x563f721e4bd0 2024-03-15T13:06:23.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f69f90740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7214bf90 con 0x563f721ed7b0 2024-03-15T13:06:23.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f6486f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e42b0 0x563f721e4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f6486f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e42b0 0x563f721e4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34306/0 (socket says 172.21.15.2:34306) 2024-03-15T13:06:23.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f6486f640 1 -- 172.21.15.2:0/440782875 learned_addr learned my addr 172.21.15.2:0/440782875 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:23.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f65070640 1 --2- 172.21.15.2:0/440782875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721e4bd0 0x563f721ed270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f5e7fc640 1 --2- 172.21.15.2:0/4069901609 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1f480a1ba0 0x7f1f480a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f6486f640 1 --2- 172.21.15.2:0/440782875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721ed7b0 0x563f721efba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f65871640 1 --2- 172.21.15.2:0/4069901609 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1f480a1ba0 0x7f1f480a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f65070640 1 -- 172.21.15.2:0/440782875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721ed7b0 msgr2=0x563f721efba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f65070640 1 --2- 172.21.15.2:0/440782875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721ed7b0 0x563f721efba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f65070640 1 -- 172.21.15.2:0/440782875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e42b0 msgr2=0x563f721e4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f65070640 1 --2- 172.21.15.2:0/440782875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e42b0 0x563f721e4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f65070640 1 -- 172.21.15.2:0/440782875 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f7210a570 con 0x563f721e4bd0 2024-03-15T13:06:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f5e7fc640 1 -- 172.21.15.2:0/4069901609 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f1f500f07c0 con 0x7f1f601064d0 2024-03-15T13:06:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.077+0000 7f1f6486f640 1 --2- 172.21.15.2:0/440782875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e42b0 0x563f721e4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.078+0000 7f1f65070640 1 --2- 172.21.15.2:0/440782875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721e4bd0 0x563f721ed270 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009bf0 tx=0x7f1f58011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de5232f6d3da94ef server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.078+0000 7f1f65871640 1 --2- 172.21.15.2:0/4069901609 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1f480a1ba0 0x7f1f480a4060 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f500edd30 tx=0x7f1f500b2830 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.078+0000 7f1f477fe640 1 -- 172.21.15.2:0/440782875 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58004050 con 0x563f721e4bd0 2024-03-15T13:06:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.078+0000 7f1f477fe640 1 -- 172.21.15.2:0/440782875 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1f58005020 con 0x563f721e4bd0 2024-03-15T13:06:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.078+0000 7f1f477fe640 1 -- 172.21.15.2:0/440782875 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58004d00 con 0x563f721e4bd0 2024-03-15T13:06:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 -- 172.21.15.2:0/440782875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721e4bd0 msgr2=0x563f721ed270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:23.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 --2- 172.21.15.2:0/440782875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721e4bd0 0x563f721ed270 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009bf0 tx=0x7f1f58011810 comp rx=0 tx=0).stop 2024-03-15T13:06:23.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 -- 172.21.15.2:0/440782875 shutdown_connections 2024-03-15T13:06:23.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 --2- 172.21.15.2:0/440782875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721ed7b0 0x563f721efba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 --2- 172.21.15.2:0/440782875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721e4bd0 0x563f721ed270 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 --2- 172.21.15.2:0/440782875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e42b0 0x563f721e4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 -- 172.21.15.2:0/440782875 >> 172.21.15.2:0/440782875 conn(0x563f721de990 msgr2=0x563f721ded70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:23.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 -- 172.21.15.2:0/440782875 shutdown_connections 2024-03-15T13:06:23.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 -- 172.21.15.2:0/440782875 wait complete. 2024-03-15T13:06:23.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 -- start start 2024-03-15T13:06:23.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 Processor -- start 2024-03-15T13:06:23.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721e42b0 0x563f721ed250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e4bd0 0x563f721ea5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721ed7b0 0x563f721eacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f71e1f430 con 0x563f721e4bd0 2024-03-15T13:06:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f71ddbe00 con 0x563f721ed7b0 2024-03-15T13:06:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f720d0fd0 con 0x563f721e42b0 2024-03-15T13:06:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.079+0000 7f1f69f90740 10 client.0 _ll_register_callbacks cb 0x563f72126fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f65070640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e4bd0 0x563f721ea5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f6486f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721e42b0 0x563f721ed250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f65070640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e4bd0 0x563f721ea5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34312/0 (socket says 172.21.15.2:34312) 2024-03-15T13:06:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f65070640 1 -- 172.21.15.2:0/1035044519 learned_addr learned my addr 172.21.15.2:0/1035044519 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f6486f640 1 --2- 172.21.15.2:0/1035044519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721ed7b0 0x563f721eacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f65070640 1 -- 172.21.15.2:0/1035044519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721e42b0 msgr2=0x563f721ed250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:23.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f65070640 1 --2- 172.21.15.2:0/1035044519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721e42b0 0x563f721ed250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f65070640 1 -- 172.21.15.2:0/1035044519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721ed7b0 msgr2=0x563f721eacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:23.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f65070640 1 --2- 172.21.15.2:0/1035044519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721ed7b0 0x563f721eacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f65070640 1 -- 172.21.15.2:0/1035044519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f58009850 con 0x563f721e4bd0 2024-03-15T13:06:23.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f6486f640 1 --2- 172.21.15.2:0/1035044519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721e42b0 0x563f721ed250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:23.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f65070640 1 --2- 172.21.15.2:0/1035044519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e4bd0 0x563f721ea5e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f1f580057c0 tx=0x7f1f58004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62384fde35275d2a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:23.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:23.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f44ff9640 1 -- 172.21.15.2:0/1035044519 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f580055a0 con 0x563f721e4bd0 2024-03-15T13:06:23.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f44ff9640 1 -- 172.21.15.2:0/1035044519 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f1f58004050 con 0x563f721e4bd0 2024-03-15T13:06:23.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f69f90740 1 -- 172.21.15.2:0/1035044519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f71e4b7e0 con 0x563f721e4bd0 2024-03-15T13:06:23.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f44ff9640 1 -- 172.21.15.2:0/1035044519 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f58004210 con 0x563f721e4bd0 2024-03-15T13:06:23.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f69f90740 10 client.17919 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.080+0000 7f1f69f90740 1 -- 172.21.15.2:0/1035044519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x563f7214a610 con 0x563f721e4bd0 2024-03-15T13:06:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.081+0000 7f1f44ff9640 1 -- 172.21.15.2:0/1035044519 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x563f71e4b7e0 con 0x563f721e4bd0 2024-03-15T13:06:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.081+0000 7f1f44ff9640 1 -- 172.21.15.2:0/1035044519 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1f58021040 con 0x563f721e4bd0 2024-03-15T13:06:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.081+0000 7f1f69f90740 10 client.17919 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.081+0000 7f1f69f90740 10 client.17919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.081+0000 7f1f69f90740 20 client.17919 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.081+0000 7f1f69f90740 1 --2- 172.21.15.2:0/1035044519 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563f721fd9a0 0x563f721ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.081+0000 7f1f69f90740 4 client.17919 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.081+0000 7f1f69f90740 1 -- 172.21.15.2:0/1035044519 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f71e88720 con 0x563f721fd9a0 2024-03-15T13:06:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.081+0000 7f1f6486f640 1 --2- 172.21.15.2:0/1035044519 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563f721fd9a0 0x563f721ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.081+0000 7f1f44ff9640 10 client.17919 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.081+0000 7f1f6486f640 1 --2- 172.21.15.2:0/1035044519 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563f721fd9a0 0x563f721ffdc0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a9e4e8133f16014 server_cookie=7a38f52379e9cc60 in_seq=0 out_seq=0 2024-03-15T13:06:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.082+0000 7f1f44ff9640 1 -- 172.21.15.2:0/1035044519 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563f71e88720 con 0x563f721fd9a0 2024-03-15T13:06:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.082+0000 7f1f44ff9640 10 client.17919 handle_command_reply: tid=0 2024-03-15T13:06:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.082+0000 7f1f69f90740 1 client.17919 shutdown 2024-03-15T13:06:23.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.083+0000 7f1f69f90740 1 -- 172.21.15.2:0/1035044519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e4bd0 msgr2=0x563f721ea5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:23.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.083+0000 7f1f69f90740 1 --2- 172.21.15.2:0/1035044519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e4bd0 0x563f721ea5e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f1f580057c0 tx=0x7f1f58004f50 comp rx=0 tx=0).stop 2024-03-15T13:06:23.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.083+0000 7f1f69f90740 1 -- 172.21.15.2:0/1035044519 shutdown_connections 2024-03-15T13:06:23.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.083+0000 7f1f69f90740 1 --2- 172.21.15.2:0/1035044519 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563f721fd9a0 0x563f721ffdc0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.083+0000 7f1f69f90740 1 --2- 172.21.15.2:0/1035044519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721ed7b0 0x563f721eacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.083+0000 7f1f69f90740 1 --2- 172.21.15.2:0/1035044519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e4bd0 0x563f721ea5e0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.083+0000 7f1f69f90740 1 --2- 172.21.15.2:0/1035044519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721e42b0 0x563f721ed250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.083+0000 7f1f44ff9640 0 client.17919 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:23.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.083+0000 7f1f69f90740 1 -- 172.21.15.2:0/1035044519 >> 172.21.15.2:0/1035044519 conn(0x563f721de990 msgr2=0x563f721e0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:23.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.083+0000 7f1f69f90740 1 -- 172.21.15.2:0/1035044519 shutdown_connections 2024-03-15T13:06:23.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.083+0000 7f1f69f90740 1 -- 172.21.15.2:0/1035044519 wait complete. 2024-03-15T13:06:23.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.083+0000 7f1f69f90740 20 client.17919 trim_cache size 0 max 16384 2024-03-15T13:06:23.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f69f90740 1 Processor -- start 2024-03-15T13:06:23.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f69f90740 1 -- start start 2024-03-15T13:06:23.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f69f90740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721e42b0 0x563f721ecd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f69f90740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e4bd0 0x563f721ed250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f69f90740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721ed7b0 0x563f721f8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f69f90740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f721f0170 con 0x563f721e4bd0 2024-03-15T13:06:23.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f69f90740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f721e3f20 con 0x563f721ed7b0 2024-03-15T13:06:23.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f69f90740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f721e5410 con 0x563f721e42b0 2024-03-15T13:06:23.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f65070640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e4bd0 0x563f721ed250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f65070640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e4bd0 0x563f721ed250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34318/0 (socket says 172.21.15.2:34318) 2024-03-15T13:06:23.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f65070640 1 -- 172.21.15.2:0/3622211532 learned_addr learned my addr 172.21.15.2:0/3622211532 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:23.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f6486f640 1 --2- 172.21.15.2:0/3622211532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721e42b0 0x563f721ecd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f6486f640 1 --2- 172.21.15.2:0/3622211532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721ed7b0 0x563f721f8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f65070640 1 -- 172.21.15.2:0/3622211532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721e42b0 msgr2=0x563f721ecd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:23.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f65070640 1 --2- 172.21.15.2:0/3622211532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721e42b0 0x563f721ecd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f65070640 1 -- 172.21.15.2:0/3622211532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721ed7b0 msgr2=0x563f721f8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:23.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f65070640 1 --2- 172.21.15.2:0/3622211532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721ed7b0 0x563f721f8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f65070640 1 -- 172.21.15.2:0/3622211532 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f720e6ac0 con 0x563f721e4bd0 2024-03-15T13:06:23.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.099+0000 7f1f65070640 1 --2- 172.21.15.2:0/3622211532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e4bd0 0x563f721ed250 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009820 tx=0x7f1f58005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c828611a86066d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:23.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.100+0000 7f1f46ffd640 1 -- 172.21.15.2:0/3622211532 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58009ee0 con 0x563f721e4bd0 2024-03-15T13:06:23.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.100+0000 7f1f46ffd640 1 -- 172.21.15.2:0/3622211532 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1f580048d0 con 0x563f721e4bd0 2024-03-15T13:06:23.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.100+0000 7f1f46ffd640 1 -- 172.21.15.2:0/3622211532 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f580196c0 con 0x563f721e4bd0 2024-03-15T13:06:23.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.100+0000 7f1f69f90740 1 -- 172.21.15.2:0/3622211532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e4bd0 msgr2=0x563f721ed250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:23.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.100+0000 7f1f69f90740 1 --2- 172.21.15.2:0/3622211532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e4bd0 0x563f721ed250 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009820 tx=0x7f1f58005bb0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.100+0000 7f1f69f90740 1 -- 172.21.15.2:0/3622211532 shutdown_connections 2024-03-15T13:06:23.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.100+0000 7f1f69f90740 1 --2- 172.21.15.2:0/3622211532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721ed7b0 0x563f721f8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.100+0000 7f1f69f90740 1 --2- 172.21.15.2:0/3622211532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e4bd0 0x563f721ed250 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.100+0000 7f1f69f90740 1 --2- 172.21.15.2:0/3622211532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721e42b0 0x563f721ecd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.100+0000 7f1f69f90740 1 -- 172.21.15.2:0/3622211532 >> 172.21.15.2:0/3622211532 conn(0x563f721de990 msgr2=0x563f721e58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:23.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.100+0000 7f1f69f90740 1 -- 172.21.15.2:0/3622211532 shutdown_connections 2024-03-15T13:06:23.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.100+0000 7f1f69f90740 1 -- 172.21.15.2:0/3622211532 wait complete. 2024-03-15T13:06:23.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f69f90740 1 -- start start 2024-03-15T13:06:23.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f69f90740 1 Processor -- start 2024-03-15T13:06:23.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f69f90740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e42b0 0x563f721ef690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f69f90740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721e4bd0 0x563f721edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f69f90740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721ed7b0 0x563f721ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f69f90740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f721ee8d0 con 0x563f721e42b0 2024-03-15T13:06:23.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f69f90740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f721eea40 con 0x563f721e4bd0 2024-03-15T13:06:23.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f69f90740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f721f4670 con 0x563f721ed7b0 2024-03-15T13:06:23.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f69f90740 10 client.0 _ll_register_callbacks cb 0x7f1f540097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:23.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f6486f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e42b0 0x563f721ef690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f6486f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e42b0 0x563f721ef690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34324/0 (socket says 172.21.15.2:34324) 2024-03-15T13:06:23.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f6486f640 1 -- 172.21.15.2:0/3496449219 learned_addr learned my addr 172.21.15.2:0/3496449219 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:23.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f65070640 1 --2- 172.21.15.2:0/3496449219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721e4bd0 0x563f721edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f6486f640 1 --2- 172.21.15.2:0/3496449219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721ed7b0 0x563f721ee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f6486f640 1 -- 172.21.15.2:0/3496449219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721ed7b0 msgr2=0x563f721ee1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:23.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f6486f640 1 --2- 172.21.15.2:0/3496449219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721ed7b0 0x563f721ee1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f6486f640 1 -- 172.21.15.2:0/3496449219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721e4bd0 msgr2=0x563f721edc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:23.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f6486f640 1 --2- 172.21.15.2:0/3496449219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721e4bd0 0x563f721edc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f6486f640 1 -- 172.21.15.2:0/3496449219 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f7214a610 con 0x563f721e42b0 2024-03-15T13:06:23.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.101+0000 7f1f65070640 1 --2- 172.21.15.2:0/3496449219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721e4bd0 0x563f721edc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:23.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f6486f640 1 --2- 172.21.15.2:0/3496449219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e42b0 0x563f721ef690 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1f54000c00 tx=0x7f1f58004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662e1b49a14c30cd server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:23.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:23.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f44ff9640 1 -- 172.21.15.2:0/3496449219 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54018040 con 0x563f721e42b0 2024-03-15T13:06:23.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f69f90740 1 -- 172.21.15.2:0/3496449219 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f58005020 con 0x563f721e42b0 2024-03-15T13:06:23.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f44ff9640 1 -- 172.21.15.2:0/3496449219 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f1f540042f0 con 0x563f721e42b0 2024-03-15T13:06:23.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f44ff9640 1 -- 172.21.15.2:0/3496449219 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f5401f040 con 0x563f721e42b0 2024-03-15T13:06:23.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f69f90740 10 client.17925 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:23.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f69f90740 1 -- 172.21.15.2:0/3496449219 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x563f7210c6a0 con 0x563f721e42b0 2024-03-15T13:06:23.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f44ff9640 1 -- 172.21.15.2:0/3496449219 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1f58005020 con 0x563f721e42b0 2024-03-15T13:06:23.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f44ff9640 1 -- 172.21.15.2:0/3496449219 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1f54025070 con 0x563f721e42b0 2024-03-15T13:06:23.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f69f90740 10 client.17925 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:23.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f69f90740 10 client.17925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:23.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f69f90740 20 client.17925 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:23.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f69f90740 1 --2- 172.21.15.2:0/3496449219 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563f72259960 0x563f7225bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:23.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f69f90740 4 client.17925 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:06:23.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.102+0000 7f1f69f90740 1 -- 172.21.15.2:0/3496449219 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563f71ed7890 con 0x563f72259960 2024-03-15T13:06:23.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f65070640 1 --2- 172.21.15.2:0/3496449219 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563f72259960 0x563f7225bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:23.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f65070640 1 --2- 172.21.15.2:0/3496449219 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563f72259960 0x563f7225bd50 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c314d5278d1c2c83 server_cookie=79816f89db7fa0c5 in_seq=0 out_seq=0 2024-03-15T13:06:23.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f44ff9640 10 client.17925 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:23.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f44ff9640 1 -- 172.21.15.2:0/3496449219 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563f71ed7890 con 0x563f72259960 2024-03-15T13:06:23.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f44ff9640 10 client.17925 handle_command_reply: tid=0 2024-03-15T13:06:23.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f69f90740 1 client.17925 shutdown 2024-03-15T13:06:23.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f69f90740 1 -- 172.21.15.2:0/3496449219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e42b0 msgr2=0x563f721ef690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:23.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f69f90740 1 --2- 172.21.15.2:0/3496449219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e42b0 0x563f721ef690 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1f54000c00 tx=0x7f1f58004db0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f69f90740 1 -- 172.21.15.2:0/3496449219 shutdown_connections 2024-03-15T13:06:23.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f69f90740 1 --2- 172.21.15.2:0/3496449219 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x563f72259960 0x563f7225bd50 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f69f90740 1 --2- 172.21.15.2:0/3496449219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563f721ed7b0 0x563f721ee1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f69f90740 1 --2- 172.21.15.2:0/3496449219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f721e4bd0 0x563f721edc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f44ff9640 0 client.17925 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:23.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f69f90740 1 --2- 172.21.15.2:0/3496449219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f721e42b0 0x563f721ef690 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f69f90740 1 -- 172.21.15.2:0/3496449219 >> 172.21.15.2:0/3496449219 conn(0x563f721de990 msgr2=0x563f721e9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:23.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f69f90740 1 -- 172.21.15.2:0/3496449219 shutdown_connections 2024-03-15T13:06:23.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f69f90740 1 -- 172.21.15.2:0/3496449219 wait complete. 2024-03-15T13:06:23.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.103+0000 7f1f69f90740 20 client.17925 trim_cache size 0 max 16384 2024-03-15T13:06:23.097 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:06:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T13:06:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:06:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: "93f85f19-f2e1-4b20-8a5b-a54aeac326f6": { 2024-03-15T13:06:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:06:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "93f85f19-f2e1-4b20-8a5b-a54aeac326f6", 2024-03-15T13:06:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:06:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:06:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:06:23.097 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:06:23.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.104+0000 7f1f672d4640 1 -- 172.21.15.2:0/4069901609 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1f480a1ba0 msgr2=0x7f1f480a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:23.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.104+0000 7f1f672d4640 1 --2- 172.21.15.2:0/4069901609 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1f480a1ba0 0x7f1f480a4060 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f500edd30 tx=0x7f1f500b2830 comp rx=0 tx=0).stop 2024-03-15T13:06:23.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.104+0000 7f1f672d4640 1 -- 172.21.15.2:0/4069901609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f601064d0 msgr2=0x7f1f60197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:23.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.104+0000 7f1f672d4640 1 --2- 172.21.15.2:0/4069901609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f601064d0 0x7f1f60197890 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f1f50005cc0 tx=0x7f1f50034a80 comp rx=0 tx=0).stop 2024-03-15T13:06:23.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.104+0000 7f1f672d4640 1 -- 172.21.15.2:0/4069901609 shutdown_connections 2024-03-15T13:06:23.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.104+0000 7f1f672d4640 1 --2- 172.21.15.2:0/4069901609 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1f480a1ba0 0x7f1f480a4060 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.104+0000 7f1f672d4640 1 --2- 172.21.15.2:0/4069901609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f601064d0 0x7f1f60197890 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.104+0000 7f1f672d4640 1 --2- 172.21.15.2:0/4069901609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f60101000 0x7f1f6019dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.104+0000 7f1f672d4640 1 --2- 172.21.15.2:0/4069901609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f60100640 0x7f1f6019d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:23.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.104+0000 7f1f672d4640 1 -- 172.21.15.2:0/4069901609 >> 172.21.15.2:0/4069901609 conn(0x7f1f600771d0 msgr2=0x7f1f600ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:23.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.104+0000 7f1f672d4640 1 -- 172.21.15.2:0/4069901609 shutdown_connections 2024-03-15T13:06:23.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:23.104+0000 7f1f672d4640 1 -- 172.21.15.2:0/4069901609 wait complete. 2024-03-15T13:06:23.100 INFO:tasks.cephfs.filesystem:scrub status for tag:93f85f19-f2e1-4b20-8a5b-a54aeac326f6 - {'path': '/', 'tag': '93f85f19-f2e1-4b20-8a5b-a54aeac326f6', 'options': 'recursive,force'} 2024-03-15T13:06:23.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:23 smithi002 ceph-mon[32060]: pgmap v229: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:06:23.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:23 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:06:23.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:23 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:06:23.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:23 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:06:23.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:23 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:06:23.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:23 smithi184 ceph-mon[35993]: pgmap v229: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:06:23.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:23 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:06:23.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:23 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:06:23.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:23 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:06:23.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:23 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:06:23.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:23 smithi149 ceph-mon[35927]: pgmap v229: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:06:23.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:23 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:06:23.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:23 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:06:23.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:23 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:06:23.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:23 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:06:25.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:25 smithi002 ceph-mon[32060]: pgmap v230: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:06:25.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:25 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:06:25.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:25 smithi184 ceph-mon[35993]: pgmap v230: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:06:25.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:25 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:06:25.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:25 smithi149 ceph-mon[35927]: pgmap v230: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:06:25.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:25 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:06:27.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:27 smithi002 ceph-mon[32060]: pgmap v231: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-15T13:06:27.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:27 smithi184 ceph-mon[35993]: pgmap v231: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-15T13:06:27.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:27 smithi149 ceph-mon[35927]: pgmap v231: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-15T13:06:29.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:29 smithi002 ceph-mon[32060]: pgmap v232: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-15T13:06:29.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:29 smithi184 ceph-mon[35993]: pgmap v232: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-15T13:06:29.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:29 smithi149 ceph-mon[35927]: pgmap v232: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-15T13:06:31.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:31 smithi002 ceph-mon[32060]: pgmap v233: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-15T13:06:31.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:31 smithi184 ceph-mon[35993]: pgmap v233: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-15T13:06:31.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:31 smithi149 ceph-mon[35927]: pgmap v233: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-15T13:06:33.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:33 smithi002 ceph-mon[32060]: pgmap v234: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-15T13:06:33.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:33 smithi184 ceph-mon[35993]: pgmap v234: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-15T13:06:33.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:33 smithi149 ceph-mon[35927]: pgmap v234: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-15T13:06:35.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:35 smithi002 ceph-mon[32060]: pgmap v235: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:06:35.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:35 smithi184 ceph-mon[35993]: pgmap v235: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:06:35.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:35 smithi149 ceph-mon[35927]: pgmap v235: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:06:37.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:37 smithi002 ceph-mon[32060]: pgmap v236: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:06:37.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:37 smithi184 ceph-mon[35993]: pgmap v236: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:06:37.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:37 smithi149 ceph-mon[35927]: pgmap v236: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:06:39.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:39 smithi002 ceph-mon[32060]: pgmap v237: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:39.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:39 smithi184 ceph-mon[35993]: pgmap v237: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:39.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:39 smithi149 ceph-mon[35927]: pgmap v237: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:41 smithi002 ceph-mon[32060]: pgmap v238: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:06:41.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:41 smithi184 ceph-mon[35993]: pgmap v238: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:06:41.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:41 smithi149 ceph-mon[35927]: pgmap v238: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:06:43.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:43 smithi002 ceph-mon[32060]: pgmap v239: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:43.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:43 smithi184 ceph-mon[35993]: pgmap v239: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:43.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:43 smithi149 ceph-mon[35927]: pgmap v239: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:45.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:45 smithi002 ceph-mon[32060]: pgmap v240: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:45.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:45 smithi184 ceph-mon[35993]: pgmap v240: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:45.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:45 smithi149 ceph-mon[35927]: pgmap v240: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:47.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:47 smithi002 ceph-mon[32060]: pgmap v241: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:06:47.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:47 smithi184 ceph-mon[35993]: pgmap v241: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:06:47.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:47 smithi149 ceph-mon[35927]: pgmap v241: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:06:49.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:49 smithi002 ceph-mon[32060]: pgmap v242: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:49.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:49 smithi184 ceph-mon[35993]: pgmap v242: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:49.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:49 smithi149 ceph-mon[35927]: pgmap v242: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:51.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:51 smithi002 ceph-mon[32060]: pgmap v243: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:06:51.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:51 smithi184 ceph-mon[35993]: pgmap v243: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:06:51.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:51 smithi149 ceph-mon[35927]: pgmap v243: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:06:53.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:06:53.101 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:06:53.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.235+0000 7efd18d81640 1 -- 172.21.15.2:0/2694719270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd14107510 msgr2=0x7efd141078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.235+0000 7efd18d81640 1 --2- 172.21.15.2:0/2694719270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd14107510 0x7efd141078f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7efd04009900 tx=0x7efd0402bc30 comp rx=0 tx=0).stop 2024-03-15T13:06:53.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.235+0000 7efd18d81640 1 -- 172.21.15.2:0/2694719270 shutdown_connections 2024-03-15T13:06:53.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.235+0000 7efd18d81640 1 --2- 172.21.15.2:0/2694719270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd1406a520 0x7efd1410e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.235+0000 7efd18d81640 1 --2- 172.21.15.2:0/2694719270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd14069b80 0x7efd14069fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.235+0000 7efd18d81640 1 --2- 172.21.15.2:0/2694719270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd14107510 0x7efd141078f0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.235+0000 7efd18d81640 1 -- 172.21.15.2:0/2694719270 >> 172.21.15.2:0/2694719270 conn(0x7efd14077180 msgr2=0x7efd14077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.235+0000 7efd18d81640 1 -- 172.21.15.2:0/2694719270 shutdown_connections 2024-03-15T13:06:53.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.235+0000 7efd18d81640 1 -- 172.21.15.2:0/2694719270 wait complete. 2024-03-15T13:06:53.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.236+0000 7efd18d81640 1 Processor -- start 2024-03-15T13:06:53.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.236+0000 7efd18d81640 1 -- start start 2024-03-15T13:06:53.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.236+0000 7efd18d81640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd14069b80 0x7efd1419f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.236+0000 7efd18d81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1406a520 0x7efd1419fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.236+0000 7efd18d81640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd14107510 0x7efd14199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.236+0000 7efd18d81640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1419a130 con 0x7efd1406a520 2024-03-15T13:06:53.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.236+0000 7efd18d81640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1419a2a0 con 0x7efd14069b80 2024-03-15T13:06:53.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.236+0000 7efd18d81640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1419a460 con 0x7efd14107510 2024-03-15T13:06:53.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd09d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1406a520 0x7efd1419fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd09d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1406a520 0x7efd1419fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51590/0 (socket says 172.21.15.2:51590) 2024-03-15T13:06:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd09d9b640 1 -- 172.21.15.2:0/4223453448 learned_addr learned my addr 172.21.15.2:0/4223453448 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd12d9d640 1 --2- 172.21.15.2:0/4223453448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd14107510 0x7efd14199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd1259c640 1 --2- 172.21.15.2:0/4223453448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd14069b80 0x7efd1419f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd09d9b640 1 -- 172.21.15.2:0/4223453448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd14107510 msgr2=0x7efd14199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd09d9b640 1 --2- 172.21.15.2:0/4223453448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd14107510 0x7efd14199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd09d9b640 1 -- 172.21.15.2:0/4223453448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd14069b80 msgr2=0x7efd1419f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd09d9b640 1 --2- 172.21.15.2:0/4223453448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd14069b80 0x7efd1419f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd09d9b640 1 -- 172.21.15.2:0/4223453448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd04009560 con 0x7efd1406a520 2024-03-15T13:06:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd09d9b640 1 --2- 172.21.15.2:0/4223453448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1406a520 0x7efd1419fe70 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00e9f0 tx=0x7efcfc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd0bfff640 1 -- 172.21.15.2:0/4223453448 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc009700 con 0x7efd1406a520 2024-03-15T13:06:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd18d81640 1 -- 172.21.15.2:0/4223453448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1419a740 con 0x7efd1406a520 2024-03-15T13:06:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd0bfff640 1 -- 172.21.15.2:0/4223453448 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efcfc00cd80 con 0x7efd1406a520 2024-03-15T13:06:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd0bfff640 1 -- 172.21.15.2:0/4223453448 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc018920 con 0x7efd1406a520 2024-03-15T13:06:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.237+0000 7efd18d81640 1 -- 172.21.15.2:0/4223453448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd141a69d0 con 0x7efd1406a520 2024-03-15T13:06:53.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.239+0000 7efd0bfff640 1 -- 172.21.15.2:0/4223453448 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7efcfc018a80 con 0x7efd1406a520 2024-03-15T13:06:53.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.240+0000 7efd1ba39740 1 Processor -- start 2024-03-15T13:06:53.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.240+0000 7efd0bfff640 1 --2- 172.21.15.2:0/4223453448 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efd000a1ba0 0x7efd000a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.240+0000 7efd12d9d640 1 --2- 172.21.15.2:0/4223453448 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efd000a1ba0 0x7efd000a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.240+0000 7efd0bfff640 1 -- 172.21.15.2:0/4223453448 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7efcfc014070 con 0x7efd1406a520 2024-03-15T13:06:53.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.240+0000 7efd1ba39740 1 -- start start 2024-03-15T13:06:53.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.240+0000 7efd1ba39740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db16b2b0 0x5597db16b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.240+0000 7efd1ba39740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16bbd0 0x5597db174270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.240+0000 7efd1ba39740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db1747b0 0x5597db176ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd1ba39740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597db0da280 con 0x5597db1747b0 2024-03-15T13:06:53.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd1ba39740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597db0d77b0 con 0x5597db16b2b0 2024-03-15T13:06:53.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd1ba39740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597db0d2f90 con 0x5597db16bbd0 2024-03-15T13:06:53.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd12d9d640 1 --2- 172.21.15.2:0/4223453448 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efd000a1ba0 0x7efd000a4060 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005fd0 tx=0x7efd0c009500 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd1259c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db16b2b0 0x5597db16b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd09d9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16bbd0 0x5597db174270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd12d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db1747b0 0x5597db176ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd1259c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db16b2b0 0x5597db16b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:32974/0 (socket says 172.21.15.2:32974) 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd1259c640 1 -- 172.21.15.2:0/2987100814 learned_addr learned my addr 172.21.15.2:0/2987100814 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd1259c640 1 -- 172.21.15.2:0/2987100814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16bbd0 msgr2=0x5597db174270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd1259c640 1 --2- 172.21.15.2:0/2987100814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16bbd0 0x5597db174270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd1259c640 1 -- 172.21.15.2:0/2987100814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db1747b0 msgr2=0x5597db176ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd1259c640 1 --2- 172.21.15.2:0/2987100814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db1747b0 0x5597db176ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd1259c640 1 -- 172.21.15.2:0/2987100814 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597db091570 con 0x5597db16b2b0 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd1259c640 1 --2- 172.21.15.2:0/2987100814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db16b2b0 0x5597db16b690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efd04036040 tx=0x7efd0402c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2922152f54b1da13 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd08d99640 1 -- 172.21.15.2:0/2987100814 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04004780 con 0x5597db16b2b0 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd08d99640 1 -- 172.21.15.2:0/2987100814 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efd040048e0 con 0x5597db16b2b0 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.241+0000 7efd08d99640 1 -- 172.21.15.2:0/2987100814 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04003830 con 0x5597db16b2b0 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.242+0000 7efd1ba39740 1 -- 172.21.15.2:0/2987100814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db16b2b0 msgr2=0x5597db16b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.242+0000 7efd1ba39740 1 --2- 172.21.15.2:0/2987100814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db16b2b0 0x5597db16b690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efd04036040 tx=0x7efd0402c6d0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.242+0000 7efd1ba39740 1 -- 172.21.15.2:0/2987100814 shutdown_connections 2024-03-15T13:06:53.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.242+0000 7efd1ba39740 1 --2- 172.21.15.2:0/2987100814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db1747b0 0x5597db176ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.242+0000 7efd1ba39740 1 --2- 172.21.15.2:0/2987100814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16bbd0 0x5597db174270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.242+0000 7efd1ba39740 1 --2- 172.21.15.2:0/2987100814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db16b2b0 0x5597db16b690 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.242+0000 7efd1ba39740 1 -- 172.21.15.2:0/2987100814 >> 172.21.15.2:0/2987100814 conn(0x5597db165990 msgr2=0x5597db165d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.242+0000 7efd1ba39740 1 -- 172.21.15.2:0/2987100814 shutdown_connections 2024-03-15T13:06:53.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.242+0000 7efd1ba39740 1 -- 172.21.15.2:0/2987100814 wait complete. 2024-03-15T13:06:53.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1ba39740 1 -- start start 2024-03-15T13:06:53.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1ba39740 1 Processor -- start 2024-03-15T13:06:53.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1ba39740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16b2b0 0x5597db174170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1ba39740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16bbd0 0x5597db171500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1ba39740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db1747b0 0x5597db171b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1ba39740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597dada6430 con 0x5597db16b2b0 2024-03-15T13:06:53.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1ba39740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597dad62e00 con 0x5597db1747b0 2024-03-15T13:06:53.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1ba39740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597db057fd0 con 0x5597db16bbd0 2024-03-15T13:06:53.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1ba39740 10 client.0 _ll_register_callbacks cb 0x5597db0adfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:53.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16b2b0 0x5597db174170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16b2b0 0x5597db174170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51614/0 (socket says 172.21.15.2:51614) 2024-03-15T13:06:53.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1259c640 1 -- 172.21.15.2:0/3703765164 learned_addr learned my addr 172.21.15.2:0/3703765164 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd09d9b640 1 --2- 172.21.15.2:0/3703765164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16bbd0 0x5597db171500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd12d9d640 1 --2- 172.21.15.2:0/3703765164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db1747b0 0x5597db171b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1259c640 1 -- 172.21.15.2:0/3703765164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16bbd0 msgr2=0x5597db171500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1259c640 1 --2- 172.21.15.2:0/3703765164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16bbd0 0x5597db171500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1259c640 1 -- 172.21.15.2:0/3703765164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db1747b0 msgr2=0x5597db171b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1259c640 1 --2- 172.21.15.2:0/3703765164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db1747b0 0x5597db171b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1259c640 1 -- 172.21.15.2:0/3703765164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1410ebd0 con 0x5597db16b2b0 2024-03-15T13:06:53.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efd1259c640 1 --2- 172.21.15.2:0/3703765164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16b2b0 0x5597db174170 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7efd0402c170 tx=0x7efd0402c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cd16048d8f6d58 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.243+0000 7efcf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:53.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.244+0000 7efcf9ffb640 1 -- 172.21.15.2:0/3703765164 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd040051f0 con 0x5597db16b2b0 2024-03-15T13:06:53.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.244+0000 7efcf9ffb640 1 -- 172.21.15.2:0/3703765164 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7efd0404d070 con 0x5597db16b2b0 2024-03-15T13:06:53.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.244+0000 7efd1ba39740 1 -- 172.21.15.2:0/3703765164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597dadd27e0 con 0x5597db16b2b0 2024-03-15T13:06:53.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.244+0000 7efcf9ffb640 1 -- 172.21.15.2:0/3703765164 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd04004ad0 con 0x5597db16b2b0 2024-03-15T13:06:53.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.244+0000 7efd1ba39740 10 client.17937 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:53.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.244+0000 7efd1ba39740 1 -- 172.21.15.2:0/3703765164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5597db0d1610 con 0x5597db16b2b0 2024-03-15T13:06:53.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.244+0000 7efcf9ffb640 1 -- 172.21.15.2:0/3703765164 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5597dadd27e0 con 0x5597db16b2b0 2024-03-15T13:06:53.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.244+0000 7efcf9ffb640 1 -- 172.21.15.2:0/3703765164 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efd0403c050 con 0x5597db16b2b0 2024-03-15T13:06:53.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.244+0000 7efd1ba39740 10 client.17937 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:53.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.244+0000 7efd1ba39740 10 client.17937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:53.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.244+0000 7efd1ba39740 20 client.17937 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:53.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.244+0000 7efd1ba39740 1 --2- 172.21.15.2:0/3703765164 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5597db184590 0x5597db186980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.244+0000 7efd1ba39740 4 client.17937 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:53.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.244+0000 7efd1ba39740 1 -- 172.21.15.2:0/3703765164 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597dae0f720 con 0x5597db184590 2024-03-15T13:06:53.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.244+0000 7efd09d9b640 1 --2- 172.21.15.2:0/3703765164 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5597db184590 0x5597db186980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.245+0000 7efd09d9b640 1 --2- 172.21.15.2:0/3703765164 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5597db184590 0x5597db186980 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b7175aa7e03a72e server_cookie=7ee8e2e05b6c40d1 in_seq=0 out_seq=0 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.245+0000 7efcf9ffb640 10 client.17937 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.245+0000 7efcf9ffb640 1 -- 172.21.15.2:0/3703765164 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7efcfc0a94b0 con 0x5597db184590 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.245+0000 7efcf9ffb640 10 client.17937 handle_command_reply: tid=0 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.245+0000 7efd1ba39740 1 client.17937 shutdown 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.246+0000 7efd1ba39740 1 -- 172.21.15.2:0/3703765164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16b2b0 msgr2=0x5597db174170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.246+0000 7efd1ba39740 1 --2- 172.21.15.2:0/3703765164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16b2b0 0x5597db174170 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7efd0402c170 tx=0x7efd0402c3d0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.246+0000 7efd1ba39740 1 -- 172.21.15.2:0/3703765164 shutdown_connections 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.246+0000 7efd1ba39740 1 --2- 172.21.15.2:0/3703765164 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5597db184590 0x5597db186980 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.246+0000 7efd1ba39740 1 --2- 172.21.15.2:0/3703765164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db1747b0 0x5597db171b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.246+0000 7efcf9ffb640 0 client.17937 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.246+0000 7efd1ba39740 1 --2- 172.21.15.2:0/3703765164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16bbd0 0x5597db171500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.246+0000 7efd1ba39740 1 --2- 172.21.15.2:0/3703765164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16b2b0 0x5597db174170 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.246+0000 7efd1ba39740 1 -- 172.21.15.2:0/3703765164 >> 172.21.15.2:0/3703765164 conn(0x5597db165990 msgr2=0x5597db167730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.246+0000 7efd1ba39740 1 -- 172.21.15.2:0/3703765164 shutdown_connections 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.246+0000 7efd1ba39740 1 -- 172.21.15.2:0/3703765164 wait complete. 2024-03-15T13:06:53.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.246+0000 7efd1ba39740 20 client.17937 trim_cache size 0 max 16384 2024-03-15T13:06:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1ba39740 1 Processor -- start 2024-03-15T13:06:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1ba39740 1 -- start start 2024-03-15T13:06:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1ba39740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16b2b0 0x5597db173c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1ba39740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16bbd0 0x5597db174170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1ba39740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db1747b0 0x5597db17fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1ba39740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597db166560 con 0x5597db16b2b0 2024-03-15T13:06:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1ba39740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597db15fe20 con 0x5597db1747b0 2024-03-15T13:06:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1ba39740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597db16fa50 con 0x5597db16bbd0 2024-03-15T13:06:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16b2b0 0x5597db173c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16b2b0 0x5597db173c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51616/0 (socket says 172.21.15.2:51616) 2024-03-15T13:06:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1259c640 1 -- 172.21.15.2:0/2268899792 learned_addr learned my addr 172.21.15.2:0/2268899792 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd12d9d640 1 --2- 172.21.15.2:0/2268899792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db1747b0 0x5597db17fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd09d9b640 1 --2- 172.21.15.2:0/2268899792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16bbd0 0x5597db174170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1259c640 1 -- 172.21.15.2:0/2268899792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16bbd0 msgr2=0x5597db174170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1259c640 1 --2- 172.21.15.2:0/2268899792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16bbd0 0x5597db174170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1259c640 1 -- 172.21.15.2:0/2268899792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db1747b0 msgr2=0x5597db17fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1259c640 1 --2- 172.21.15.2:0/2268899792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db1747b0 0x5597db17fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1259c640 1 -- 172.21.15.2:0/2268899792 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597db06dac0 con 0x5597db16b2b0 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efd1259c640 1 --2- 172.21.15.2:0/2268899792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16b2b0 0x5597db173c30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7efd04000c00 tx=0x7efd04005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e2d0712c3bc885 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.262+0000 7efcfbfff640 1 -- 172.21.15.2:0/2268899792 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04043070 con 0x5597db16b2b0 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.263+0000 7efcfbfff640 1 -- 172.21.15.2:0/2268899792 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efd04034cf0 con 0x5597db16b2b0 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.263+0000 7efcfbfff640 1 -- 172.21.15.2:0/2268899792 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04005470 con 0x5597db16b2b0 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.263+0000 7efd1ba39740 1 -- 172.21.15.2:0/2268899792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16b2b0 msgr2=0x5597db173c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.263+0000 7efd1ba39740 1 --2- 172.21.15.2:0/2268899792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16b2b0 0x5597db173c30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7efd04000c00 tx=0x7efd04005bc0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.263+0000 7efd1ba39740 1 -- 172.21.15.2:0/2268899792 shutdown_connections 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.263+0000 7efd1ba39740 1 --2- 172.21.15.2:0/2268899792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db1747b0 0x5597db17fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.263+0000 7efd1ba39740 1 --2- 172.21.15.2:0/2268899792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16bbd0 0x5597db174170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.263+0000 7efd1ba39740 1 --2- 172.21.15.2:0/2268899792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16b2b0 0x5597db173c30 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.263+0000 7efd1ba39740 1 -- 172.21.15.2:0/2268899792 >> 172.21.15.2:0/2268899792 conn(0x5597db165990 msgr2=0x5597db164a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.263+0000 7efd1ba39740 1 -- 172.21.15.2:0/2268899792 shutdown_connections 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.263+0000 7efd1ba39740 1 -- 172.21.15.2:0/2268899792 wait complete. 2024-03-15T13:06:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.263+0000 7efd1ba39740 1 -- start start 2024-03-15T13:06:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd1ba39740 1 Processor -- start 2024-03-15T13:06:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd1ba39740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16b2b0 0x5597db175250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd1ba39740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16bbd0 0x5597db175790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd1ba39740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db1747b0 0x5597db17b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd1ba39740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597db175f10 con 0x5597db16bbd0 2024-03-15T13:06:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd1ba39740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597db176080 con 0x5597db1747b0 2024-03-15T13:06:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd1ba39740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597db17bc30 con 0x5597db16b2b0 2024-03-15T13:06:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd1ba39740 10 client.0 _ll_register_callbacks cb 0x7efd0c003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd09d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16bbd0 0x5597db175790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd1259c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16b2b0 0x5597db175250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd09d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16bbd0 0x5597db175790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51622/0 (socket says 172.21.15.2:51622) 2024-03-15T13:06:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd1259c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16b2b0 0x5597db175250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:52356/0 (socket says 172.21.15.2:52356) 2024-03-15T13:06:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd09d9b640 1 -- 172.21.15.2:0/1482073053 learned_addr learned my addr 172.21.15.2:0/1482073053 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd12d9d640 1 --2- 172.21.15.2:0/1482073053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db1747b0 0x5597db17b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd09d9b640 1 -- 172.21.15.2:0/1482073053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16b2b0 msgr2=0x5597db175250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd09d9b640 1 --2- 172.21.15.2:0/1482073053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16b2b0 0x5597db175250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd09d9b640 1 -- 172.21.15.2:0/1482073053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db1747b0 msgr2=0x5597db17b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd09d9b640 1 --2- 172.21.15.2:0/1482073053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db1747b0 0x5597db17b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd09d9b640 1 -- 172.21.15.2:0/1482073053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd04056040 con 0x5597db16bbd0 2024-03-15T13:06:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd1259c640 1 --2- 172.21.15.2:0/1482073053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16b2b0 0x5597db175250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efd09d9b640 1 --2- 172.21.15.2:0/1482073053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16bbd0 0x5597db175790 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0cb2d0 tx=0x7efcfc0ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda0ddf482180c0f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efcf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efcf9ffb640 1 -- 172.21.15.2:0/1482073053 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc0b3170 con 0x5597db16bbd0 2024-03-15T13:06:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.264+0000 7efcf9ffb640 1 -- 172.21.15.2:0/1482073053 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7efcfc0cb4f0 con 0x5597db16bbd0 2024-03-15T13:06:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.265+0000 7efd1ba39740 1 -- 172.21.15.2:0/1482073053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd040048e0 con 0x5597db16bbd0 2024-03-15T13:06:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.265+0000 7efcf9ffb640 1 -- 172.21.15.2:0/1482073053 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcfc0cb6b0 con 0x5597db16bbd0 2024-03-15T13:06:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.265+0000 7efd1ba39740 10 client.17943 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.265+0000 7efd1ba39740 1 -- 172.21.15.2:0/1482073053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5597db0936a0 con 0x5597db16bbd0 2024-03-15T13:06:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.265+0000 7efcf9ffb640 1 -- 172.21.15.2:0/1482073053 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efcfc0a4030 con 0x5597db16bbd0 2024-03-15T13:06:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.265+0000 7efcf9ffb640 1 -- 172.21.15.2:0/1482073053 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efcfc0ae260 con 0x5597db16bbd0 2024-03-15T13:06:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.265+0000 7efd1ba39740 10 client.17943 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.265+0000 7efd1ba39740 10 client.17943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.265+0000 7efd1ba39740 20 client.17943 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.265+0000 7efd1ba39740 1 --2- 172.21.15.2:0/1482073053 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5597db1e0b60 0x5597db1e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.265+0000 7efd1ba39740 4 client.17943 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:06:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.265+0000 7efd1ba39740 1 -- 172.21.15.2:0/1482073053 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5597dae5e890 con 0x5597db1e0b60 2024-03-15T13:06:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.265+0000 7efd1259c640 1 --2- 172.21.15.2:0/1482073053 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5597db1e0b60 0x5597db1e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efcf9ffb640 10 client.17943 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efd1259c640 1 --2- 172.21.15.2:0/1482073053 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5597db1e0b60 0x5597db1e2f80 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41f10cb1692d8866 server_cookie=fc029ce2af0d5741 in_seq=0 out_seq=0 2024-03-15T13:06:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efcf9ffb640 1 -- 172.21.15.2:0/1482073053 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5597dae5e890 con 0x5597db1e0b60 2024-03-15T13:06:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efcf9ffb640 10 client.17943 handle_command_reply: tid=0 2024-03-15T13:06:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efd1ba39740 1 client.17943 shutdown 2024-03-15T13:06:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efd1ba39740 1 -- 172.21.15.2:0/1482073053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16bbd0 msgr2=0x5597db175790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efd1ba39740 1 --2- 172.21.15.2:0/1482073053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16bbd0 0x5597db175790 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0cb2d0 tx=0x7efcfc0ba040 comp rx=0 tx=0).stop 2024-03-15T13:06:53.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efd1ba39740 1 -- 172.21.15.2:0/1482073053 shutdown_connections 2024-03-15T13:06:53.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efd1ba39740 1 --2- 172.21.15.2:0/1482073053 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5597db1e0b60 0x5597db1e2f80 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efd1ba39740 1 --2- 172.21.15.2:0/1482073053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597db1747b0 0x5597db17b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efcf9ffb640 0 client.17943 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:53.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efd1ba39740 1 --2- 172.21.15.2:0/1482073053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5597db16bbd0 0x5597db175790 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efd1ba39740 1 --2- 172.21.15.2:0/1482073053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597db16b2b0 0x5597db175250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efd1ba39740 1 -- 172.21.15.2:0/1482073053 >> 172.21.15.2:0/1482073053 conn(0x5597db165990 msgr2=0x5597db166ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efd1ba39740 1 -- 172.21.15.2:0/1482073053 shutdown_connections 2024-03-15T13:06:53.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efd1ba39740 1 -- 172.21.15.2:0/1482073053 wait complete. 2024-03-15T13:06:53.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efd1ba39740 20 client.17943 trim_cache size 0 max 16384 2024-03-15T13:06:53.257 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:06:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:06:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:06:53.258 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:06:53.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.266+0000 7efd18d81640 1 -- 172.21.15.2:0/4223453448 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efd000a1ba0 msgr2=0x7efd000a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.267+0000 7efd18d81640 1 --2- 172.21.15.2:0/4223453448 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efd000a1ba0 0x7efd000a4060 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005fd0 tx=0x7efd0c009500 comp rx=0 tx=0).stop 2024-03-15T13:06:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.267+0000 7efd18d81640 1 -- 172.21.15.2:0/4223453448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1406a520 msgr2=0x7efd1419fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.267+0000 7efd18d81640 1 --2- 172.21.15.2:0/4223453448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1406a520 0x7efd1419fe70 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00e9f0 tx=0x7efcfc00eeb0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.267+0000 7efd18d81640 1 -- 172.21.15.2:0/4223453448 shutdown_connections 2024-03-15T13:06:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.267+0000 7efd18d81640 1 --2- 172.21.15.2:0/4223453448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd14107510 0x7efd14199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.267+0000 7efd18d81640 1 --2- 172.21.15.2:0/4223453448 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efd000a1ba0 0x7efd000a4060 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.267+0000 7efd18d81640 1 --2- 172.21.15.2:0/4223453448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1406a520 0x7efd1419fe70 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.267+0000 7efd18d81640 1 --2- 172.21.15.2:0/4223453448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd14069b80 0x7efd1419f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.267+0000 7efd18d81640 1 -- 172.21.15.2:0/4223453448 >> 172.21.15.2:0/4223453448 conn(0x7efd14077180 msgr2=0x7efd1410c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.267+0000 7efd18d81640 1 -- 172.21.15.2:0/4223453448 shutdown_connections 2024-03-15T13:06:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.267+0000 7efd18d81640 1 -- 172.21.15.2:0/4223453448 wait complete. 2024-03-15T13:06:53.262 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:06:53.262 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:06:53.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.381+0000 7fd558c8f640 1 -- 172.21.15.2:0/3543691263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd554100fd0 msgr2=0x7fd554105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.381+0000 7fd558c8f640 1 --2- 172.21.15.2:0/3543691263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd554100fd0 0x7fd554105010 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009860 tx=0x7fd53c02bb70 comp rx=0 tx=0).stop 2024-03-15T13:06:53.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.381+0000 7fd558c8f640 1 -- 172.21.15.2:0/3543691263 shutdown_connections 2024-03-15T13:06:53.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.381+0000 7fd558c8f640 1 --2- 172.21.15.2:0/3543691263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5541056f0 0x7fd55410d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.381+0000 7fd558c8f640 1 --2- 172.21.15.2:0/3543691263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd554100fd0 0x7fd554105010 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.381+0000 7fd558c8f640 1 --2- 172.21.15.2:0/3543691263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd554100620 0x7fd554100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.381+0000 7fd558c8f640 1 -- 172.21.15.2:0/3543691263 >> 172.21.15.2:0/3543691263 conn(0x7fd554077490 msgr2=0x7fd5540778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.381+0000 7fd558c8f640 1 -- 172.21.15.2:0/3543691263 shutdown_connections 2024-03-15T13:06:53.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.381+0000 7fd558c8f640 1 -- 172.21.15.2:0/3543691263 wait complete. 2024-03-15T13:06:53.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd558c8f640 1 Processor -- start 2024-03-15T13:06:53.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd558c8f640 1 -- start start 2024-03-15T13:06:53.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd558c8f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd554100620 0x7fd554198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd558c8f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5541056f0 0x7fd554198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd558c8f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd554199450 0x7fd5541a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd558c8f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5541a40a0 con 0x7fd554100620 2024-03-15T13:06:53.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd558c8f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5541a4210 con 0x7fd5541056f0 2024-03-15T13:06:53.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd558c8f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5541a43d0 con 0x7fd554199450 2024-03-15T13:06:53.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd55259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd554100620 0x7fd554198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd55259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd554100620 0x7fd554198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51642/0 (socket says 172.21.15.2:51642) 2024-03-15T13:06:53.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd55259c640 1 -- 172.21.15.2:0/2635350467 learned_addr learned my addr 172.21.15.2:0/2635350467 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd551d9b640 1 --2- 172.21.15.2:0/2635350467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5541056f0 0x7fd554198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd552d9d640 1 --2- 172.21.15.2:0/2635350467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd554199450 0x7fd5541a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd55259c640 1 -- 172.21.15.2:0/2635350467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd554199450 msgr2=0x7fd5541a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd55259c640 1 --2- 172.21.15.2:0/2635350467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd554199450 0x7fd5541a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd55259c640 1 -- 172.21.15.2:0/2635350467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5541056f0 msgr2=0x7fd554198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd55259c640 1 --2- 172.21.15.2:0/2635350467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5541056f0 0x7fd554198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.382+0000 7fd55259c640 1 -- 172.21.15.2:0/2635350467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c009560 con 0x7fd554100620 2024-03-15T13:06:53.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.383+0000 7fd55259c640 1 --2- 172.21.15.2:0/2635350467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd554100620 0x7fd554198980 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00e9c0 tx=0x7fd54c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.383+0000 7fd54b7fe640 1 -- 172.21.15.2:0/2635350467 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c00cde0 con 0x7fd554100620 2024-03-15T13:06:53.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.383+0000 7fd54b7fe640 1 -- 172.21.15.2:0/2635350467 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd54c0035f0 con 0x7fd554100620 2024-03-15T13:06:53.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.383+0000 7fd558c8f640 1 -- 172.21.15.2:0/2635350467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53c009490 con 0x7fd554100620 2024-03-15T13:06:53.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.383+0000 7fd54b7fe640 1 -- 172.21.15.2:0/2635350467 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c010670 con 0x7fd554100620 2024-03-15T13:06:53.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.383+0000 7fd558c8f640 1 -- 172.21.15.2:0/2635350467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5541a4a00 con 0x7fd554100620 2024-03-15T13:06:53.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.384+0000 7fd558c8f640 1 -- 172.21.15.2:0/2635350467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd554068730 con 0x7fd554100620 2024-03-15T13:06:53.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.385+0000 7fd54b7fe640 1 -- 172.21.15.2:0/2635350467 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fd54c003760 con 0x7fd554100620 2024-03-15T13:06:53.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.385+0000 7fd54b7fe640 1 --2- 172.21.15.2:0/2635350467 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd5400a1ba0 0x7fd5400a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.386+0000 7fd551d9b640 1 --2- 172.21.15.2:0/2635350467 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd5400a1ba0 0x7fd5400a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.386+0000 7fd54b7fe640 1 -- 172.21.15.2:0/2635350467 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd54c014070 con 0x7fd554100620 2024-03-15T13:06:53.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.386+0000 7fd551d9b640 1 --2- 172.21.15.2:0/2635350467 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd5400a1ba0 0x7fd5400a4060 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd53c005940 tx=0x7fd53c0058d0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.390+0000 7fd54b7fe640 1 -- 172.21.15.2:0/2635350467 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd54c096ae0 con 0x7fd554100620 2024-03-15T13:06:53.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.610+0000 7fd558c8f640 1 -- 172.21.15.2:0/2635350467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd55406a110 con 0x7fd554100620 2024-03-15T13:06:53.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.611+0000 7fd54b7fe640 1 -- 172.21.15.2:0/2635350467 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+11453 (secure 0 0 0) 0x7fd54c09a1a0 con 0x7fd554100620 2024-03-15T13:06:53.596 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:06:53.596 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:05:12.138845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24500,24464,24460,24434,24539]},"id":1}]} 2024-03-15T13:06:53.596 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 50 2024-03-15T13:06:53.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.613+0000 7fd558c8f640 1 -- 172.21.15.2:0/2635350467 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd5400a1ba0 msgr2=0x7fd5400a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.613+0000 7fd558c8f640 1 --2- 172.21.15.2:0/2635350467 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd5400a1ba0 0x7fd5400a4060 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd53c005940 tx=0x7fd53c0058d0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.613+0000 7fd558c8f640 1 -- 172.21.15.2:0/2635350467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd554100620 msgr2=0x7fd554198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.613+0000 7fd558c8f640 1 --2- 172.21.15.2:0/2635350467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd554100620 0x7fd554198980 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00e9c0 tx=0x7fd54c00ee80 comp rx=0 tx=0).stop 2024-03-15T13:06:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.613+0000 7fd558c8f640 1 -- 172.21.15.2:0/2635350467 shutdown_connections 2024-03-15T13:06:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.614+0000 7fd558c8f640 1 --2- 172.21.15.2:0/2635350467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd554199450 0x7fd5541a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.614+0000 7fd558c8f640 1 --2- 172.21.15.2:0/2635350467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5541056f0 0x7fd554198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.614+0000 7fd558c8f640 1 --2- 172.21.15.2:0/2635350467 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd5400a1ba0 0x7fd5400a4060 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.614+0000 7fd558c8f640 1 --2- 172.21.15.2:0/2635350467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd554100620 0x7fd554198980 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.614+0000 7fd558c8f640 1 -- 172.21.15.2:0/2635350467 >> 172.21.15.2:0/2635350467 conn(0x7fd554077490 msgr2=0x7fd5540ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.614+0000 7fd558c8f640 1 -- 172.21.15.2:0/2635350467 shutdown_connections 2024-03-15T13:06:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.614+0000 7fd558c8f640 1 -- 172.21.15.2:0/2635350467 wait complete. 2024-03-15T13:06:53.609 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:06:53.609 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:06:53.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.729+0000 7f6c99435640 1 -- 172.21.15.2:0/3102242040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c941056e0 msgr2=0x7f6c9410d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.729+0000 7f6c99435640 1 --2- 172.21.15.2:0/3102242040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c941056e0 0x7f6c9410d2b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009cd0 tx=0x7f6c8c02c0f0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.729+0000 7f6c99435640 1 -- 172.21.15.2:0/3102242040 shutdown_connections 2024-03-15T13:06:53.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.729+0000 7f6c99435640 1 --2- 172.21.15.2:0/3102242040 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c941056e0 0x7f6c9410d2b0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.729+0000 7f6c99435640 1 --2- 172.21.15.2:0/3102242040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c94100fc0 0x7f6c94105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.729+0000 7f6c99435640 1 --2- 172.21.15.2:0/3102242040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c94100610 0x7f6c941009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.729+0000 7f6c99435640 1 -- 172.21.15.2:0/3102242040 >> 172.21.15.2:0/3102242040 conn(0x7f6c94076770 msgr2=0x7f6c94076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.729+0000 7f6c99435640 1 -- 172.21.15.2:0/3102242040 shutdown_connections 2024-03-15T13:06:53.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.729+0000 7f6c99435640 1 -- 172.21.15.2:0/3102242040 wait complete. 2024-03-15T13:06:53.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c99435640 1 Processor -- start 2024-03-15T13:06:53.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c99435640 1 -- start start 2024-03-15T13:06:53.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c99435640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c94100610 0x7f6c9419d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c99435640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c941056e0 0x7f6c9419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c99435640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c94197930 0x7f6c94197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c99435640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c94198320 con 0x7f6c94197930 2024-03-15T13:06:53.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c99435640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c94198490 con 0x7f6c941056e0 2024-03-15T13:06:53.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c99435640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c94198650 con 0x7f6c94100610 2024-03-15T13:06:53.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c937fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c94197930 0x7f6c94197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c92ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c94100610 0x7f6c9419d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c937fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c94197930 0x7f6c94197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51652/0 (socket says 172.21.15.2:51652) 2024-03-15T13:06:53.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c937fe640 1 -- 172.21.15.2:0/1776177991 learned_addr learned my addr 172.21.15.2:0/1776177991 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c8bfff640 1 --2- 172.21.15.2:0/1776177991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c941056e0 0x7f6c9419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c937fe640 1 -- 172.21.15.2:0/1776177991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c94100610 msgr2=0x7f6c9419d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c937fe640 1 --2- 172.21.15.2:0/1776177991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c94100610 0x7f6c9419d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c937fe640 1 -- 172.21.15.2:0/1776177991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c941056e0 msgr2=0x7f6c9419dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c937fe640 1 --2- 172.21.15.2:0/1776177991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c941056e0 0x7f6c9419dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c937fe640 1 -- 172.21.15.2:0/1776177991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c009900 con 0x7f6c94197930 2024-03-15T13:06:53.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c92ffd640 1 --2- 172.21.15.2:0/1776177991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c94100610 0x7f6c9419d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:53.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.730+0000 7f6c937fe640 1 --2- 172.21.15.2:0/1776177991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c94197930 0x7f6c94197de0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c005cc0 tx=0x7f6c8c034a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.731+0000 7f6c90ff9640 1 -- 172.21.15.2:0/1776177991 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c032880 con 0x7f6c94197930 2024-03-15T13:06:53.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.731+0000 7f6c90ff9640 1 -- 172.21.15.2:0/1776177991 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6c8c034e80 con 0x7f6c94197930 2024-03-15T13:06:53.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.731+0000 7f6c99435640 1 -- 172.21.15.2:0/1776177991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c941a4640 con 0x7f6c94197930 2024-03-15T13:06:53.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.731+0000 7f6c90ff9640 1 -- 172.21.15.2:0/1776177991 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0047c0 con 0x7f6c94197930 2024-03-15T13:06:53.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.731+0000 7f6c99435640 1 -- 172.21.15.2:0/1776177991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c941a4af0 con 0x7f6c94197930 2024-03-15T13:06:53.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.733+0000 7f6c90ff9640 1 -- 172.21.15.2:0/1776177991 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f6c8c00a040 con 0x7f6c94197930 2024-03-15T13:06:53.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.733+0000 7f6c9c00e740 1 Processor -- start 2024-03-15T13:06:53.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.733+0000 7f6c9c00e740 1 -- start start 2024-03-15T13:06:53.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.733+0000 7f6c90ff9640 1 --2- 172.21.15.2:0/1776177991 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f6c800a1c20 0x7f6c800a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c8bfff640 1 --2- 172.21.15.2:0/1776177991 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f6c800a1c20 0x7f6c800a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c90ff9640 1 -- 172.21.15.2:0/1776177991 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f6c8c0f0610 con 0x7f6c94197930 2024-03-15T13:06:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c9c00e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ec250 0x563c397ec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c9c00e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ecb70 0x563c397f5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c9c00e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397f5750 0x563c397f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c9c00e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c397a7960 con 0x563c397ecb70 2024-03-15T13:06:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c9c00e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3975b310 con 0x563c397f5750 2024-03-15T13:06:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c9c00e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c39758840 con 0x563c397ec250 2024-03-15T13:06:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ecb70 0x563c397f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ecb70 0x563c397f5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51666/0 (socket says 172.21.15.2:51666) 2024-03-15T13:06:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c8bfff640 1 -- 172.21.15.2:0/3942028751 learned_addr learned my addr 172.21.15.2:0/3942028751 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c937fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397f5750 0x563c397f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c92ffd640 1 --2- 172.21.15.2:0/3942028751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ec250 0x563c397ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c937fe640 1 -- 172.21.15.2:0/3942028751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ec250 msgr2=0x563c397ec630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c937fe640 1 --2- 172.21.15.2:0/3942028751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ec250 0x563c397ec630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c937fe640 1 -- 172.21.15.2:0/3942028751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ecb70 msgr2=0x563c397f5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c937fe640 1 --2- 172.21.15.2:0/3942028751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ecb70 0x563c397f5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c937fe640 1 -- 172.21.15.2:0/3942028751 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c39716060 con 0x563c397f5750 2024-03-15T13:06:53.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c8bfff640 1 --2- 172.21.15.2:0/1776177991 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f6c800a1c20 0x7f6c800a40e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009fb0 tx=0x7f6c7c013830 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c92ffd640 1 --2- 172.21.15.2:0/3942028751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ec250 0x563c397ec630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:53.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c937fe640 1 --2- 172.21.15.2:0/3942028751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397f5750 0x563c397f7b40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0edad0 tx=0x7f6c8c0bcaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38320aee5a0ff162 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.734+0000 7f6c897fa640 1 -- 172.21.15.2:0/3942028751 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0bcfb0 con 0x563c397f5750 2024-03-15T13:06:53.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.735+0000 7f6c897fa640 1 -- 172.21.15.2:0/3942028751 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6c8c0c1100 con 0x563c397f5750 2024-03-15T13:06:53.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.735+0000 7f6c897fa640 1 -- 172.21.15.2:0/3942028751 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0bd210 con 0x563c397f5750 2024-03-15T13:06:53.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.735+0000 7f6c9c00e740 1 -- 172.21.15.2:0/3942028751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397f5750 msgr2=0x563c397f7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.735+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/3942028751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397f5750 0x563c397f7b40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0edad0 tx=0x7f6c8c0bcaf0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.735+0000 7f6c9c00e740 1 -- 172.21.15.2:0/3942028751 shutdown_connections 2024-03-15T13:06:53.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.735+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/3942028751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397f5750 0x563c397f7b40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.735+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/3942028751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ecb70 0x563c397f5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.735+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/3942028751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ec250 0x563c397ec630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.735+0000 7f6c9c00e740 1 -- 172.21.15.2:0/3942028751 >> 172.21.15.2:0/3942028751 conn(0x563c397e6980 msgr2=0x563c397e6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.735+0000 7f6c9c00e740 1 -- 172.21.15.2:0/3942028751 shutdown_connections 2024-03-15T13:06:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.735+0000 7f6c9c00e740 1 -- 172.21.15.2:0/3942028751 wait complete. 2024-03-15T13:06:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c9c00e740 1 -- start start 2024-03-15T13:06:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c9c00e740 1 Processor -- start 2024-03-15T13:06:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c9c00e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ec250 0x563c397f5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c9c00e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ecb70 0x563c397f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c9c00e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397f5750 0x563c397f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c9c00e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c39754020 con 0x563c397ec250 2024-03-15T13:06:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c9c00e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c39427430 con 0x563c397f5750 2024-03-15T13:06:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c9c00e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c393e3e00 con 0x563c397ecb70 2024-03-15T13:06:53.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c9c00e740 10 client.0 _ll_register_callbacks cb 0x563c396fbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c92ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ec250 0x563c397f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c92ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ec250 0x563c397f5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51674/0 (socket says 172.21.15.2:51674) 2024-03-15T13:06:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c92ffd640 1 -- 172.21.15.2:0/164082904 learned_addr learned my addr 172.21.15.2:0/164082904 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c937fe640 1 --2- 172.21.15.2:0/164082904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397f5750 0x563c397f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c8bfff640 1 --2- 172.21.15.2:0/164082904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ecb70 0x563c397f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c92ffd640 1 -- 172.21.15.2:0/164082904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ecb70 msgr2=0x563c397f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c92ffd640 1 --2- 172.21.15.2:0/164082904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ecb70 0x563c397f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c92ffd640 1 -- 172.21.15.2:0/164082904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397f5750 msgr2=0x563c397f2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c92ffd640 1 --2- 172.21.15.2:0/164082904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397f5750 0x563c397f2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c92ffd640 1 -- 172.21.15.2:0/164082904 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c941a4af0 con 0x563c397ec250 2024-03-15T13:06:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c92ffd640 1 --2- 172.21.15.2:0/164082904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ec250 0x563c397f5130 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f6c84012d20 tx=0x7f6c8400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea54aca5110e43 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.736+0000 7f6c7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.737+0000 7f6c7a7fc640 1 -- 172.21.15.2:0/164082904 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84017070 con 0x563c397ec250 2024-03-15T13:06:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.737+0000 7f6c7a7fc640 1 -- 172.21.15.2:0/164082904 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f6c84012110 con 0x563c397ec250 2024-03-15T13:06:53.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.737+0000 7f6c9c00e740 1 -- 172.21.15.2:0/164082904 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c394537e0 con 0x563c397ec250 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.737+0000 7f6c7a7fc640 1 -- 172.21.15.2:0/164082904 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c84004050 con 0x563c397ec250 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.737+0000 7f6c9c00e740 10 client.17961 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.737+0000 7f6c9c00e740 1 -- 172.21.15.2:0/164082904 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x563c397526a0 con 0x563c397ec250 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.737+0000 7f6c7a7fc640 1 -- 172.21.15.2:0/164082904 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x563c394537e0 con 0x563c397ec250 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.737+0000 7f6c7a7fc640 1 -- 172.21.15.2:0/164082904 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6c84010430 con 0x563c397ec250 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.737+0000 7f6c9c00e740 10 client.17961 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.737+0000 7f6c9c00e740 10 client.17961 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.737+0000 7f6c9c00e740 20 client.17961 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.737+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/164082904 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563c39805760 0x563c39807b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.737+0000 7f6c9c00e740 4 client.17961 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.737+0000 7f6c9c00e740 1 -- 172.21.15.2:0/164082904 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c39490720 con 0x563c39805760 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.737+0000 7f6c8bfff640 1 --2- 172.21.15.2:0/164082904 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563c39805760 0x563c39807b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.738+0000 7f6c7a7fc640 10 client.17961 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.738+0000 7f6c8bfff640 1 --2- 172.21.15.2:0/164082904 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563c39805760 0x563c39807b80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3010f5009246c83f server_cookie=214fd1aaf95d8040 in_seq=0 out_seq=0 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.739+0000 7f6c7a7fc640 1 -- 172.21.15.2:0/164082904 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563c39490720 con 0x563c39805760 2024-03-15T13:06:53.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.739+0000 7f6c7a7fc640 10 client.17961 handle_command_reply: tid=0 2024-03-15T13:06:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.739+0000 7f6c9c00e740 1 client.17961 shutdown 2024-03-15T13:06:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.739+0000 7f6c9c00e740 1 -- 172.21.15.2:0/164082904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ec250 msgr2=0x563c397f5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.739+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/164082904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ec250 0x563c397f5130 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f6c84012d20 tx=0x7f6c8400f790 comp rx=0 tx=0).stop 2024-03-15T13:06:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.739+0000 7f6c9c00e740 1 -- 172.21.15.2:0/164082904 shutdown_connections 2024-03-15T13:06:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.739+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/164082904 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563c39805760 0x563c39807b80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.739+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/164082904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397f5750 0x563c397f2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.739+0000 7f6c7a7fc640 0 client.17961 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:06:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.739+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/164082904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ecb70 0x563c397f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.739+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/164082904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ec250 0x563c397f5130 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.739+0000 7f6c9c00e740 1 -- 172.21.15.2:0/164082904 >> 172.21.15.2:0/164082904 conn(0x563c397e6980 msgr2=0x563c397e86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.739+0000 7f6c9c00e740 1 -- 172.21.15.2:0/164082904 shutdown_connections 2024-03-15T13:06:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.739+0000 7f6c9c00e740 1 -- 172.21.15.2:0/164082904 wait complete. 2024-03-15T13:06:53.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.739+0000 7f6c9c00e740 20 client.17961 trim_cache size 0 max 16384 2024-03-15T13:06:53.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.751+0000 7f6c9c00e740 1 Processor -- start 2024-03-15T13:06:53.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.751+0000 7f6c9c00e740 1 -- start start 2024-03-15T13:06:53.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c9c00e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ec250 0x563c397e5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c9c00e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397ecb70 0x563c397f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c9c00e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397f5750 0x563c397f5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c9c00e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c396d9130 con 0x563c397ec250 2024-03-15T13:06:53.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c9c00e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c397ec970 con 0x563c397ecb70 2024-03-15T13:06:53.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c9c00e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c397e22b0 con 0x563c397f5750 2024-03-15T13:06:53.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c92ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ec250 0x563c397e5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c92ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ec250 0x563c397e5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51676/0 (socket says 172.21.15.2:51676) 2024-03-15T13:06:53.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c92ffd640 1 -- 172.21.15.2:0/2574819487 learned_addr learned my addr 172.21.15.2:0/2574819487 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c8bfff640 1 --2- 172.21.15.2:0/2574819487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397ecb70 0x563c397f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c937fe640 1 --2- 172.21.15.2:0/2574819487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397f5750 0x563c397f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c92ffd640 1 -- 172.21.15.2:0/2574819487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397f5750 msgr2=0x563c397f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c92ffd640 1 --2- 172.21.15.2:0/2574819487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397f5750 0x563c397f5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c92ffd640 1 -- 172.21.15.2:0/2574819487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397ecb70 msgr2=0x563c397f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c92ffd640 1 --2- 172.21.15.2:0/2574819487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397ecb70 0x563c397f4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c92ffd640 1 -- 172.21.15.2:0/2574819487 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c396eec20 con 0x563c397ec250 2024-03-15T13:06:53.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c92ffd640 1 --2- 172.21.15.2:0/2574819487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ec250 0x563c397e5a00 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f6c84000c00 tx=0x7f6c84005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f6cbc751a3410 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.752+0000 7f6c88ff9640 1 -- 172.21.15.2:0/2574819487 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84017070 con 0x563c397ec250 2024-03-15T13:06:53.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.753+0000 7f6c88ff9640 1 -- 172.21.15.2:0/2574819487 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6c84005580 con 0x563c397ec250 2024-03-15T13:06:53.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.753+0000 7f6c88ff9640 1 -- 172.21.15.2:0/2574819487 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c840035f0 con 0x563c397ec250 2024-03-15T13:06:53.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.753+0000 7f6c9c00e740 1 -- 172.21.15.2:0/2574819487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ec250 msgr2=0x563c397e5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.753+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/2574819487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ec250 0x563c397e5a00 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f6c84000c00 tx=0x7f6c84005870 comp rx=0 tx=0).stop 2024-03-15T13:06:53.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.753+0000 7f6c9c00e740 1 -- 172.21.15.2:0/2574819487 shutdown_connections 2024-03-15T13:06:53.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.753+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/2574819487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397f5750 0x563c397f5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.753+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/2574819487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397ecb70 0x563c397f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.753+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/2574819487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397ec250 0x563c397e5a00 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.753+0000 7f6c9c00e740 1 -- 172.21.15.2:0/2574819487 >> 172.21.15.2:0/2574819487 conn(0x563c397e6980 msgr2=0x563c397e52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.753+0000 7f6c9c00e740 1 -- 172.21.15.2:0/2574819487 shutdown_connections 2024-03-15T13:06:53.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.753+0000 7f6c9c00e740 1 -- 172.21.15.2:0/2574819487 wait complete. 2024-03-15T13:06:53.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.753+0000 7f6c9c00e740 1 -- start start 2024-03-15T13:06:53.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c9c00e740 1 Processor -- start 2024-03-15T13:06:53.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c9c00e740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ec250 0x563c397ef420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c9c00e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397ecb70 0x563c397ef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c9c00e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397f5750 0x563c3985a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c9c00e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3985a550 con 0x563c397f5750 2024-03-15T13:06:53.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c9c00e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3985a6c0 con 0x563c397ecb70 2024-03-15T13:06:53.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c9c00e740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3985a880 con 0x563c397ec250 2024-03-15T13:06:53.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c9c00e740 10 client.0 _ll_register_callbacks cb 0x7f6c84009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:53.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c937fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397f5750 0x563c3985a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397ecb70 0x563c397ef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c937fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397f5750 0x563c3985a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51678/0 (socket says 172.21.15.2:51678) 2024-03-15T13:06:53.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c937fe640 1 -- 172.21.15.2:0/3821335309 learned_addr learned my addr 172.21.15.2:0/3821335309 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c92ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ec250 0x563c397ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c937fe640 1 -- 172.21.15.2:0/3821335309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ec250 msgr2=0x563c397ef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c937fe640 1 --2- 172.21.15.2:0/3821335309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ec250 0x563c397ef420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c937fe640 1 -- 172.21.15.2:0/3821335309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397ecb70 msgr2=0x563c397ef960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c937fe640 1 --2- 172.21.15.2:0/3821335309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397ecb70 0x563c397ef960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c937fe640 1 -- 172.21.15.2:0/3821335309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c84021040 con 0x563c397f5750 2024-03-15T13:06:53.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c92ffd640 1 --2- 172.21.15.2:0/3821335309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ec250 0x563c397ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:53.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c937fe640 1 --2- 172.21.15.2:0/3821335309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397f5750 0x563c3985a010 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0bcf40 tx=0x7f6c8c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb601c19f82cce9e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.754+0000 7f6c7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:53.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.755+0000 7f6c7a7fc640 1 -- 172.21.15.2:0/3821335309 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0bcfb0 con 0x563c397f5750 2024-03-15T13:06:53.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.755+0000 7f6c7a7fc640 1 -- 172.21.15.2:0/3821335309 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f6c8c0c5750 con 0x563c397f5750 2024-03-15T13:06:53.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.755+0000 7f6c9c00e740 1 -- 172.21.15.2:0/3821335309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c397e20f0 con 0x563c397f5750 2024-03-15T13:06:53.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.755+0000 7f6c7a7fc640 1 -- 172.21.15.2:0/3821335309 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c8c0b9620 con 0x563c397f5750 2024-03-15T13:06:53.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.755+0000 7f6c9c00e740 10 client.17967 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:53.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.755+0000 7f6c9c00e740 1 -- 172.21.15.2:0/3821335309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x563c397148c0 con 0x563c397f5750 2024-03-15T13:06:53.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.755+0000 7f6c7a7fc640 1 -- 172.21.15.2:0/3821335309 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6c8c09f080 con 0x563c397f5750 2024-03-15T13:06:53.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.755+0000 7f6c7a7fc640 1 -- 172.21.15.2:0/3821335309 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6c8c0c52c0 con 0x563c397f5750 2024-03-15T13:06:53.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.755+0000 7f6c9c00e740 10 client.17967 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:53.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.755+0000 7f6c9c00e740 10 client.17967 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:06:53.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.755+0000 7f6c9c00e740 20 client.17967 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:53.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.755+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/3821335309 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563c39861ce0 0x563c39864100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.755+0000 7f6c9c00e740 4 client.17967 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:06:53.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.755+0000 7f6c9c00e740 1 -- 172.21.15.2:0/3821335309 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f6c8c0c1100 con 0x563c39861ce0 2024-03-15T13:06:53.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.755+0000 7f6c92ffd640 1 --2- 172.21.15.2:0/3821335309 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563c39861ce0 0x563c39864100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c92ffd640 1 --2- 172.21.15.2:0/3821335309 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563c39861ce0 0x563c39864100 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=98f042c727a45d2 server_cookie=bafce944f87c89f0 in_seq=0 out_seq=0 2024-03-15T13:06:53.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c7a7fc640 10 client.17967 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:06:53.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c7a7fc640 1 -- 172.21.15.2:0/3821335309 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f6c8c0c1100 con 0x563c39861ce0 2024-03-15T13:06:53.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c7a7fc640 10 client.17967 handle_command_reply: tid=0 2024-03-15T13:06:53.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c9c00e740 1 client.17967 shutdown 2024-03-15T13:06:53.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c9c00e740 1 -- 172.21.15.2:0/3821335309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397f5750 msgr2=0x563c3985a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/3821335309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397f5750 0x563c3985a010 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0bcf40 tx=0x7f6c8c0095c0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c9c00e740 1 -- 172.21.15.2:0/3821335309 shutdown_connections 2024-03-15T13:06:53.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/3821335309 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563c39861ce0 0x563c39864100 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/3821335309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c397f5750 0x563c3985a010 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c7a7fc640 0 client.17967 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:06:53.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/3821335309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c397ecb70 0x563c397ef960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c9c00e740 1 --2- 172.21.15.2:0/3821335309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c397ec250 0x563c397ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c9c00e740 1 -- 172.21.15.2:0/3821335309 >> 172.21.15.2:0/3821335309 conn(0x563c397e6980 msgr2=0x563c397ed920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c9c00e740 1 -- 172.21.15.2:0/3821335309 shutdown_connections 2024-03-15T13:06:53.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c9c00e740 1 -- 172.21.15.2:0/3821335309 wait complete. 2024-03-15T13:06:53.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c9c00e740 20 client.17967 trim_cache size 0 max 16384 2024-03-15T13:06:53.748 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:06:53.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c99435640 1 -- 172.21.15.2:0/1776177991 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f6c800a1c20 msgr2=0x7f6c800a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c99435640 1 --2- 172.21.15.2:0/1776177991 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f6c800a1c20 0x7f6c800a40e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009fb0 tx=0x7f6c7c013830 comp rx=0 tx=0).stop 2024-03-15T13:06:53.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.756+0000 7f6c99435640 1 -- 172.21.15.2:0/1776177991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c94197930 msgr2=0x7f6c94197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.757+0000 7f6c99435640 1 --2- 172.21.15.2:0/1776177991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c94197930 0x7f6c94197de0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c005cc0 tx=0x7f6c8c034a80 comp rx=0 tx=0).stop 2024-03-15T13:06:53.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.757+0000 7f6c99435640 1 -- 172.21.15.2:0/1776177991 shutdown_connections 2024-03-15T13:06:53.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.757+0000 7f6c99435640 1 --2- 172.21.15.2:0/1776177991 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f6c800a1c20 0x7f6c800a40e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.757+0000 7f6c99435640 1 --2- 172.21.15.2:0/1776177991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c94197930 0x7f6c94197de0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.757+0000 7f6c99435640 1 --2- 172.21.15.2:0/1776177991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c941056e0 0x7f6c9419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.757+0000 7f6c99435640 1 --2- 172.21.15.2:0/1776177991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c94100610 0x7f6c9419d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.757+0000 7f6c99435640 1 -- 172.21.15.2:0/1776177991 >> 172.21.15.2:0/1776177991 conn(0x7f6c94076770 msgr2=0x7f6c940fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.757+0000 7f6c99435640 1 -- 172.21.15.2:0/1776177991 shutdown_connections 2024-03-15T13:06:53.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.757+0000 7f6c99435640 1 -- 172.21.15.2:0/1776177991 wait complete. 2024-03-15T13:06:53.752 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:06:53.752 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:06:53.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:53 smithi002 ceph-mon[32060]: pgmap v244: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:53.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:53 smithi184 ceph-mon[35993]: pgmap v244: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:53.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.872+0000 7fefb4f6c640 1 -- 172.21.15.2:0/1073789920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefb0101e60 msgr2=0x7fefb010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.872+0000 7fefb4f6c640 1 --2- 172.21.15.2:0/1073789920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefb0101e60 0x7fefb010e320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009c10 tx=0x7fefa802bef0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.872+0000 7fefb4f6c640 1 -- 172.21.15.2:0/1073789920 shutdown_connections 2024-03-15T13:06:53.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.872+0000 7fefb4f6c640 1 --2- 172.21.15.2:0/1073789920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefb0101e60 0x7fefb010e320 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.872+0000 7fefb4f6c640 1 --2- 172.21.15.2:0/1073789920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefb01014a0 0x7fefb0101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.872+0000 7fefb4f6c640 1 --2- 172.21.15.2:0/1073789920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb0107560 0x7fefb0107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.872+0000 7fefb4f6c640 1 -- 172.21.15.2:0/1073789920 >> 172.21.15.2:0/1073789920 conn(0x7fefb00788f0 msgr2=0x7fefb0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.872+0000 7fefb4f6c640 1 -- 172.21.15.2:0/1073789920 shutdown_connections 2024-03-15T13:06:53.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.872+0000 7fefb4f6c640 1 -- 172.21.15.2:0/1073789920 wait complete. 2024-03-15T13:06:53.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefb4f6c640 1 Processor -- start 2024-03-15T13:06:53.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefb4f6c640 1 -- start start 2024-03-15T13:06:53.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefb4f6c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefb01014a0 0x7fefb00737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefb4f6c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefb0101e60 0x7fefb0073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefb4f6c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb0107560 0x7fefb0074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefb4f6c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb01a5e00 con 0x7fefb0107560 2024-03-15T13:06:53.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefb4f6c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb01a5f70 con 0x7fefb01014a0 2024-03-15T13:06:53.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefb4f6c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb01a6130 con 0x7fefb0101e60 2024-03-15T13:06:53.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefaed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb0107560 0x7fefb0074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefaed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb0107560 0x7fefb0074270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51682/0 (socket says 172.21.15.2:51682) 2024-03-15T13:06:53.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefaed9d640 1 -- 172.21.15.2:0/4209429393 learned_addr learned my addr 172.21.15.2:0/4209429393 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefa7fff640 1 --2- 172.21.15.2:0/4209429393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefb0101e60 0x7fefb0073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefae59c640 1 --2- 172.21.15.2:0/4209429393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefb01014a0 0x7fefb00737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefaed9d640 1 -- 172.21.15.2:0/4209429393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefb0101e60 msgr2=0x7fefb0073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefaed9d640 1 --2- 172.21.15.2:0/4209429393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefb0101e60 0x7fefb0073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefaed9d640 1 -- 172.21.15.2:0/4209429393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefb01014a0 msgr2=0x7fefb00737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefaed9d640 1 --2- 172.21.15.2:0/4209429393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefb01014a0 0x7fefb00737f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.873+0000 7fefaed9d640 1 -- 172.21.15.2:0/4209429393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa0009560 con 0x7fefb0107560 2024-03-15T13:06:53.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.874+0000 7fefaed9d640 1 --2- 172.21.15.2:0/4209429393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb0107560 0x7fefb0074270 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fefa80335f0 tx=0x7fefa8033620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.874+0000 7fefa77fe640 1 -- 172.21.15.2:0/4209429393 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa8033840 con 0x7fefb0107560 2024-03-15T13:06:53.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.874+0000 7fefa77fe640 1 -- 172.21.15.2:0/4209429393 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fefa8003b80 con 0x7fefb0107560 2024-03-15T13:06:53.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.874+0000 7fefb4f6c640 1 -- 172.21.15.2:0/4209429393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa8009840 con 0x7fefb0107560 2024-03-15T13:06:53.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.874+0000 7fefa77fe640 1 -- 172.21.15.2:0/4209429393 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa8034890 con 0x7fefb0107560 2024-03-15T13:06:53.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.874+0000 7fefb4f6c640 1 -- 172.21.15.2:0/4209429393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb01a6730 con 0x7fefb0107560 2024-03-15T13:06:53.862 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:53 smithi149 ceph-mon[35927]: pgmap v244: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:53.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.876+0000 7fefa77fe640 1 -- 172.21.15.2:0/4209429393 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fefa8003cf0 con 0x7fefb0107560 2024-03-15T13:06:53.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.876+0000 7fefb7c28740 1 Processor -- start 2024-03-15T13:06:53.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.876+0000 7fefb7c28740 1 -- start start 2024-03-15T13:06:53.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.876+0000 7fefb7c28740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803250 0x55d899803630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefb7c28740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803b70 0x55d89980c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefa77fe640 1 --2- 172.21.15.2:0/4209429393 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fef9c0a1ba0 0x7fef9c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefb7c28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d89980c750 0x55d89980eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefb7c28740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8997be960 con 0x55d899803250 2024-03-15T13:06:53.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefb7c28740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d899772310 con 0x55d899803b70 2024-03-15T13:06:53.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefb7c28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89976f840 con 0x55d89980c750 2024-03-15T13:06:53.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefaed9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d89980c750 0x55d89980eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefae59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803b70 0x55d89980c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefa7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803250 0x55d899803630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefaed9d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d89980c750 0x55d89980eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:52424/0 (socket says 172.21.15.2:52424) 2024-03-15T13:06:53.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefaed9d640 1 -- 172.21.15.2:0/3294692685 learned_addr learned my addr 172.21.15.2:0/3294692685 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefa77fe640 1 -- 172.21.15.2:0/4209429393 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fefa80f0800 con 0x7fefb0107560 2024-03-15T13:06:53.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefa7fff640 1 --2- 172.21.15.2:0/4209429393 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fef9c0a1ba0 0x7fef9c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefaed9d640 1 -- 172.21.15.2:0/3294692685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803b70 msgr2=0x55d89980c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefaed9d640 1 --2- 172.21.15.2:0/3294692685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803b70 0x55d89980c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefaed9d640 1 -- 172.21.15.2:0/3294692685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803250 msgr2=0x55d899803630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefaed9d640 1 --2- 172.21.15.2:0/3294692685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803250 0x55d899803630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefaed9d640 1 -- 172.21.15.2:0/3294692685 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d89972d060 con 0x55d89980c750 2024-03-15T13:06:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefae59c640 1 --2- 172.21.15.2:0/3294692685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803b70 0x55d89980c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefa7fff640 1 --2- 172.21.15.2:0/4209429393 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fef9c0a1ba0 0x7fef9c0a4060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fef980058c0 tx=0x7fef98009800 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefaed9d640 1 --2- 172.21.15.2:0/3294692685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d89980c750 0x55d89980eb40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fefa80b8e20 tx=0x7fefa80bcaa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44350cac5cd2fa47 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefa4ff9640 1 -- 172.21.15.2:0/3294692685 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa80ba970 con 0x55d89980c750 2024-03-15T13:06:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefa4ff9640 1 -- 172.21.15.2:0/3294692685 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fefa80ba790 con 0x55d89980c750 2024-03-15T13:06:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.877+0000 7fefa4ff9640 1 -- 172.21.15.2:0/3294692685 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa80c1290 con 0x55d89980c750 2024-03-15T13:06:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.878+0000 7fefb7c28740 1 -- 172.21.15.2:0/3294692685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d89980c750 msgr2=0x55d89980eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.878+0000 7fefb7c28740 1 --2- 172.21.15.2:0/3294692685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d89980c750 0x55d89980eb40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fefa80b8e20 tx=0x7fefa80bcaa0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.878+0000 7fefb7c28740 1 -- 172.21.15.2:0/3294692685 shutdown_connections 2024-03-15T13:06:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.878+0000 7fefb7c28740 1 --2- 172.21.15.2:0/3294692685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d89980c750 0x55d89980eb40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.878+0000 7fefb7c28740 1 --2- 172.21.15.2:0/3294692685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803b70 0x55d89980c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.878+0000 7fefb7c28740 1 --2- 172.21.15.2:0/3294692685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803250 0x55d899803630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.878+0000 7fefb7c28740 1 -- 172.21.15.2:0/3294692685 >> 172.21.15.2:0/3294692685 conn(0x55d8997fd980 msgr2=0x55d8997fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.878+0000 7fefb7c28740 1 -- 172.21.15.2:0/3294692685 shutdown_connections 2024-03-15T13:06:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.878+0000 7fefb7c28740 1 -- 172.21.15.2:0/3294692685 wait complete. 2024-03-15T13:06:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefb7c28740 1 -- start start 2024-03-15T13:06:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefb7c28740 1 Processor -- start 2024-03-15T13:06:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefb7c28740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803250 0x55d89980c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefb7c28740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803b70 0x55d899809440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefb7c28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d89980c750 0x55d8998099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefb7c28740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89976b020 con 0x55d899803b70 2024-03-15T13:06:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefb7c28740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d89943e430 con 0x55d899803250 2024-03-15T13:06:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefb7c28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8993fae00 con 0x55d89980c750 2024-03-15T13:06:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefb7c28740 10 client.0 _ll_register_callbacks cb 0x55d899712cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefa7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803b70 0x55d899809440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefa7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803b70 0x55d899809440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51688/0 (socket says 172.21.15.2:51688) 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefa7fff640 1 -- 172.21.15.2:0/1060442170 learned_addr learned my addr 172.21.15.2:0/1060442170 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefae59c640 1 --2- 172.21.15.2:0/1060442170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803250 0x55d89980c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefaed9d640 1 --2- 172.21.15.2:0/1060442170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d89980c750 0x55d8998099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefa7fff640 1 -- 172.21.15.2:0/1060442170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d89980c750 msgr2=0x55d8998099a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefa7fff640 1 --2- 172.21.15.2:0/1060442170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d89980c750 0x55d8998099a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefa7fff640 1 -- 172.21.15.2:0/1060442170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803250 msgr2=0x55d89980c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefa7fff640 1 --2- 172.21.15.2:0/1060442170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803250 0x55d89980c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefa7fff640 1 -- 172.21.15.2:0/1060442170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb01a6730 con 0x55d899803b70 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefaed9d640 1 --2- 172.21.15.2:0/1060442170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d89980c750 0x55d8998099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fefa7fff640 1 --2- 172.21.15.2:0/1060442170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803b70 0x55d899809440 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fef9800fee0 tx=0x7fef9801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4591f1034a8581d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.879+0000 7fef95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.880+0000 7fef95ffb640 1 -- 172.21.15.2:0/1060442170 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef98004220 con 0x55d899803b70 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.880+0000 7fef95ffb640 1 -- 172.21.15.2:0/1060442170 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fef98004380 con 0x55d899803b70 2024-03-15T13:06:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.880+0000 7fefb7c28740 1 -- 172.21.15.2:0/1060442170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d89946a7e0 con 0x55d899803b70 2024-03-15T13:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.880+0000 7fef95ffb640 1 -- 172.21.15.2:0/1060442170 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef98005020 con 0x55d899803b70 2024-03-15T13:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.880+0000 7fefb7c28740 10 client.17979 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.880+0000 7fefb7c28740 1 -- 172.21.15.2:0/1060442170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55d8997696a0 con 0x55d899803b70 2024-03-15T13:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.880+0000 7fef95ffb640 1 -- 172.21.15.2:0/1060442170 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55d89946a7e0 con 0x55d899803b70 2024-03-15T13:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.880+0000 7fef95ffb640 1 -- 172.21.15.2:0/1060442170 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fef98004b50 con 0x55d899803b70 2024-03-15T13:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.880+0000 7fefb7c28740 10 client.17979 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.880+0000 7fefb7c28740 10 client.17979 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.880+0000 7fefb7c28740 20 client.17979 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.880+0000 7fefb7c28740 1 --2- 172.21.15.2:0/1060442170 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55d89981c550 0x55d89981e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.880+0000 7fefb7c28740 4 client.17979 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.880+0000 7fefb7c28740 1 -- 172.21.15.2:0/1060442170 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8994a7720 con 0x55d89981c550 2024-03-15T13:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.880+0000 7fefae59c640 1 --2- 172.21.15.2:0/1060442170 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55d89981c550 0x55d89981e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.881+0000 7fefae59c640 1 --2- 172.21.15.2:0/1060442170 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55d89981c550 0x55d89981e970 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d4f4506693c71c0e server_cookie=983cec35ae7e5150 in_seq=0 out_seq=0 2024-03-15T13:06:53.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.881+0000 7fef95ffb640 10 client.17979 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.881+0000 7fef95ffb640 1 -- 172.21.15.2:0/1060442170 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d8994a7720 con 0x55d89981c550 2024-03-15T13:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.881+0000 7fef95ffb640 10 client.17979 handle_command_reply: tid=0 2024-03-15T13:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.882+0000 7fefb7c28740 1 client.17979 shutdown 2024-03-15T13:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.882+0000 7fefb7c28740 1 -- 172.21.15.2:0/1060442170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803b70 msgr2=0x55d899809440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.882+0000 7fefb7c28740 1 --2- 172.21.15.2:0/1060442170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803b70 0x55d899809440 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fef9800fee0 tx=0x7fef9801d000 comp rx=0 tx=0).stop 2024-03-15T13:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.882+0000 7fefb7c28740 1 -- 172.21.15.2:0/1060442170 shutdown_connections 2024-03-15T13:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.882+0000 7fefb7c28740 1 --2- 172.21.15.2:0/1060442170 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55d89981c550 0x55d89981e970 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.882+0000 7fefb7c28740 1 --2- 172.21.15.2:0/1060442170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d89980c750 0x55d8998099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.882+0000 7fefb7c28740 1 --2- 172.21.15.2:0/1060442170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803b70 0x55d899809440 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.882+0000 7fef95ffb640 0 client.17979 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.882+0000 7fefb7c28740 1 --2- 172.21.15.2:0/1060442170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803250 0x55d89980c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.882+0000 7fefb7c28740 1 -- 172.21.15.2:0/1060442170 >> 172.21.15.2:0/1060442170 conn(0x55d8997fd980 msgr2=0x55d8997ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.882+0000 7fefb7c28740 1 -- 172.21.15.2:0/1060442170 shutdown_connections 2024-03-15T13:06:53.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.882+0000 7fefb7c28740 1 -- 172.21.15.2:0/1060442170 wait complete. 2024-03-15T13:06:53.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.882+0000 7fefb7c28740 20 client.17979 trim_cache size 0 max 16384 2024-03-15T13:06:53.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.894+0000 7fefb7c28740 1 Processor -- start 2024-03-15T13:06:53.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.894+0000 7fefb7c28740 1 -- start start 2024-03-15T13:06:53.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.894+0000 7fefb7c28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d899803250 0x55d899804b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.894+0000 7fefb7c28740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803b70 0x55d89980bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.894+0000 7fefb7c28740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d89980c750 0x55d89980c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fefb7c28740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8996f0130 con 0x55d899803b70 2024-03-15T13:06:53.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fefb7c28740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8997fe980 con 0x55d89980c750 2024-03-15T13:06:53.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fefb7c28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d899802c90 con 0x55d899803250 2024-03-15T13:06:53.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fefa7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803b70 0x55d89980bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fefa7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803b70 0x55d89980bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51690/0 (socket says 172.21.15.2:51690) 2024-03-15T13:06:53.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fefa7fff640 1 -- 172.21.15.2:0/2095937562 learned_addr learned my addr 172.21.15.2:0/2095937562 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fefaed9d640 1 --2- 172.21.15.2:0/2095937562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d89980c750 0x55d89980c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fefae59c640 1 --2- 172.21.15.2:0/2095937562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d899803250 0x55d899804b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fefa7fff640 1 -- 172.21.15.2:0/2095937562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d899803250 msgr2=0x55d899804b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fefa7fff640 1 --2- 172.21.15.2:0/2095937562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d899803250 0x55d899804b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fefa7fff640 1 -- 172.21.15.2:0/2095937562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d89980c750 msgr2=0x55d89980c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fefa7fff640 1 --2- 172.21.15.2:0/2095937562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d89980c750 0x55d89980c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fefa7fff640 1 -- 172.21.15.2:0/2095937562 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d899705c20 con 0x55d899803b70 2024-03-15T13:06:53.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fefa7fff640 1 --2- 172.21.15.2:0/2095937562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803b70 0x55d89980bb70 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fef98004ee0 tx=0x7fef98009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229c3460eff52c31 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fef97fff640 1 -- 172.21.15.2:0/2095937562 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef980038e0 con 0x55d899803b70 2024-03-15T13:06:53.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fef97fff640 1 -- 172.21.15.2:0/2095937562 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fef9800eb60 con 0x55d899803b70 2024-03-15T13:06:53.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.895+0000 7fef97fff640 1 -- 172.21.15.2:0/2095937562 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef980053d0 con 0x55d899803b70 2024-03-15T13:06:53.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 -- 172.21.15.2:0/2095937562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803b70 msgr2=0x55d89980bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 --2- 172.21.15.2:0/2095937562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803b70 0x55d89980bb70 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fef98004ee0 tx=0x7fef98009640 comp rx=0 tx=0).stop 2024-03-15T13:06:53.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 -- 172.21.15.2:0/2095937562 shutdown_connections 2024-03-15T13:06:53.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 --2- 172.21.15.2:0/2095937562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d89980c750 0x55d89980c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 --2- 172.21.15.2:0/2095937562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d899803b70 0x55d89980bb70 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 --2- 172.21.15.2:0/2095937562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d899803250 0x55d899804b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 -- 172.21.15.2:0/2095937562 >> 172.21.15.2:0/2095937562 conn(0x55d8997fd980 msgr2=0x55d899809560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 -- 172.21.15.2:0/2095937562 shutdown_connections 2024-03-15T13:06:53.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 -- 172.21.15.2:0/2095937562 wait complete. 2024-03-15T13:06:53.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 -- start start 2024-03-15T13:06:53.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 Processor -- start 2024-03-15T13:06:53.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d899803250 0x55d899810800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803b70 0x55d899810d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d89980c750 0x55d899811280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d899877e10 con 0x55d89980c750 2024-03-15T13:06:53.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d899877f50 con 0x55d899803b70 2024-03-15T13:06:53.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d899878090 con 0x55d899803250 2024-03-15T13:06:53.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefb7c28740 10 client.0 _ll_register_callbacks cb 0x7fef98009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:53.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefaed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d89980c750 0x55d899811280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefae59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d899803250 0x55d899810800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefaed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d89980c750 0x55d899811280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51698/0 (socket says 172.21.15.2:51698) 2024-03-15T13:06:53.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.896+0000 7fefaed9d640 1 -- 172.21.15.2:0/3224667025 learned_addr learned my addr 172.21.15.2:0/3224667025 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:53.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fefa7fff640 1 --2- 172.21.15.2:0/3224667025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803b70 0x55d899810d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fefae59c640 1 -- 172.21.15.2:0/3224667025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803b70 msgr2=0x55d899810d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fefae59c640 1 --2- 172.21.15.2:0/3224667025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803b70 0x55d899810d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fefae59c640 1 -- 172.21.15.2:0/3224667025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d89980c750 msgr2=0x55d899811280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:53.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fefae59c640 1 --2- 172.21.15.2:0/3224667025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d89980c750 0x55d899811280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fefae59c640 1 -- 172.21.15.2:0/3224667025 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8997696a0 con 0x55d899803250 2024-03-15T13:06:53.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fefaed9d640 1 --2- 172.21.15.2:0/3224667025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d89980c750 0x55d899811280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:53.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fefae59c640 1 --2- 172.21.15.2:0/3224667025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d899803250 0x55d899810800 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55d89981f180 tx=0x7fefa0011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f37b275e66e568f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:53.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fef95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:06:53.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fef95ffb640 1 -- 172.21.15.2:0/3224667025 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0004220 con 0x55d899803250 2024-03-15T13:06:53.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fef95ffb640 1 -- 172.21.15.2:0/3224667025 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fefa0004380 con 0x55d899803250 2024-03-15T13:06:53.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fefb7c28740 1 -- 172.21.15.2:0/3224667025 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefa80ba790 con 0x55d899803250 2024-03-15T13:06:53.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fef95ffb640 1 -- 172.21.15.2:0/3224667025 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefa000fa00 con 0x55d899803250 2024-03-15T13:06:53.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fefb7c28740 10 client.27778 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:53.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fefb7c28740 1 -- 172.21.15.2:0/3224667025 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55d89972b8c0 con 0x55d899803250 2024-03-15T13:06:53.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.897+0000 7fef95ffb640 1 -- 172.21.15.2:0/3224667025 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fefa80ba790 con 0x55d899803250 2024-03-15T13:06:53.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fef95ffb640 1 -- 172.21.15.2:0/3224667025 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fefa001d040 con 0x55d899803250 2024-03-15T13:06:53.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefb7c28740 10 client.27778 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:53.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefb7c28740 10 client.27778 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:06:53.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefb7c28740 20 client.27778 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefb7c28740 1 --2- 172.21.15.2:0/3224667025 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55d899879890 0x55d89987bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefb7c28740 4 client.27778 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefb7c28740 1 -- 172.21.15.2:0/3224667025 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d8994f6890 con 0x55d899879890 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefa7fff640 1 --2- 172.21.15.2:0/3224667025 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55d899879890 0x55d89987bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefa7fff640 1 --2- 172.21.15.2:0/3224667025 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55d899879890 0x55d89987bcb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c889c79178f682c0 server_cookie=1a7d84966321b3d4 in_seq=0 out_seq=0 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fef95ffb640 10 client.27778 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fef95ffb640 1 -- 172.21.15.2:0/3224667025 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d8994f6890 con 0x55d899879890 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fef95ffb640 10 client.27778 handle_command_reply: tid=0 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefb7c28740 1 client.27778 shutdown 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefb7c28740 1 -- 172.21.15.2:0/3224667025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d899803250 msgr2=0x55d899810800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefb7c28740 1 --2- 172.21.15.2:0/3224667025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d899803250 0x55d899810800 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55d89981f180 tx=0x7fefa0011bf0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefb7c28740 1 -- 172.21.15.2:0/3224667025 shutdown_connections 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefb7c28740 1 --2- 172.21.15.2:0/3224667025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d89980c750 0x55d899811280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefb7c28740 1 --2- 172.21.15.2:0/3224667025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d899803b70 0x55d899810d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefb7c28740 1 --2- 172.21.15.2:0/3224667025 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55d899879890 0x55d89987bcb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.898+0000 7fefb7c28740 1 --2- 172.21.15.2:0/3224667025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d899803250 0x55d899810800 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fef95ffb640 0 client.27778 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:06:53.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb7c28740 1 -- 172.21.15.2:0/3224667025 >> 172.21.15.2:0/3224667025 conn(0x55d8997fd980 msgr2=0x55d8997feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb7c28740 1 -- 172.21.15.2:0/3224667025 shutdown_connections 2024-03-15T13:06:53.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb7c28740 1 -- 172.21.15.2:0/3224667025 wait complete. 2024-03-15T13:06:53.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb7c28740 20 client.27778 trim_cache size 0 max 16384 2024-03-15T13:06:53.890 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:06:53.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb4f6c640 1 -- 172.21.15.2:0/4209429393 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fef9c0a1ba0 msgr2=0x7fef9c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb4f6c640 1 --2- 172.21.15.2:0/4209429393 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fef9c0a1ba0 0x7fef9c0a4060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fef980058c0 tx=0x7fef98009800 comp rx=0 tx=0).stop 2024-03-15T13:06:53.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb4f6c640 1 -- 172.21.15.2:0/4209429393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb0107560 msgr2=0x7fefb0074270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:53.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb4f6c640 1 --2- 172.21.15.2:0/4209429393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb0107560 0x7fefb0074270 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fefa80335f0 tx=0x7fefa8033620 comp rx=0 tx=0).stop 2024-03-15T13:06:53.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb4f6c640 1 -- 172.21.15.2:0/4209429393 shutdown_connections 2024-03-15T13:06:53.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb4f6c640 1 --2- 172.21.15.2:0/4209429393 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fef9c0a1ba0 0x7fef9c0a4060 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb4f6c640 1 --2- 172.21.15.2:0/4209429393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefb0107560 0x7fefb0074270 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb4f6c640 1 --2- 172.21.15.2:0/4209429393 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefb0101e60 0x7fefb0073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb4f6c640 1 --2- 172.21.15.2:0/4209429393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefb01014a0 0x7fefb00737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:53.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb4f6c640 1 -- 172.21.15.2:0/4209429393 >> 172.21.15.2:0/4209429393 conn(0x7fefb00788f0 msgr2=0x7fefb0100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:53.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb4f6c640 1 -- 172.21.15.2:0/4209429393 shutdown_connections 2024-03-15T13:06:53.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:53.899+0000 7fefb4f6c640 1 -- 172.21.15.2:0/4209429393 wait complete. 2024-03-15T13:06:53.894 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:06:53.895 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:06:54.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.015+0000 7f4181e95640 1 -- 172.21.15.2:0/1849131658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f417c100dd0 msgr2=0x7f417c10d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.015+0000 7f4181e95640 1 --2- 172.21.15.2:0/1849131658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f417c100dd0 0x7f417c10d2e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f4174009cd0 tx=0x7f417402c0f0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.015+0000 7f4181e95640 1 -- 172.21.15.2:0/1849131658 shutdown_connections 2024-03-15T13:06:54.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.015+0000 7f4181e95640 1 --2- 172.21.15.2:0/1849131658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f417c100dd0 0x7f417c10d2e0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.015+0000 7f4181e95640 1 --2- 172.21.15.2:0/1849131658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f417c100410 0x7f417c100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.015+0000 7f4181e95640 1 --2- 172.21.15.2:0/1849131658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f417c1064b0 0x7f417c106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.015+0000 7f4181e95640 1 -- 172.21.15.2:0/1849131658 >> 172.21.15.2:0/1849131658 conn(0x7f417c077550 msgr2=0x7f417c077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.015+0000 7f4181e95640 1 -- 172.21.15.2:0/1849131658 shutdown_connections 2024-03-15T13:06:54.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.015+0000 7f4181e95640 1 -- 172.21.15.2:0/1849131658 wait complete. 2024-03-15T13:06:54.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f4181e95640 1 Processor -- start 2024-03-15T13:06:54.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f4181e95640 1 -- start start 2024-03-15T13:06:54.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f4181e95640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f417c100410 0x7f417c19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f4181e95640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f417c100dd0 0x7f417c19dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f4181e95640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f417c1064b0 0x7f417c197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f4181e95640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c197fd0 con 0x7f417c100dd0 2024-03-15T13:06:54.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f4181e95640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c198140 con 0x7f417c100410 2024-03-15T13:06:54.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f4181e95640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c198300 con 0x7f417c1064b0 2024-03-15T13:06:54.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f417affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f417c100dd0 0x7f417c19dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f417affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f417c100dd0 0x7f417c19dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51716/0 (socket says 172.21.15.2:51716) 2024-03-15T13:06:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f417affd640 1 -- 172.21.15.2:0/3678457697 learned_addr learned my addr 172.21.15.2:0/3678457697 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f417b7fe640 1 --2- 172.21.15.2:0/3678457697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f417c100410 0x7f417c19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f417bfff640 1 --2- 172.21.15.2:0/3678457697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f417c1064b0 0x7f417c197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f417affd640 1 -- 172.21.15.2:0/3678457697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f417c1064b0 msgr2=0x7f417c197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f417affd640 1 --2- 172.21.15.2:0/3678457697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f417c1064b0 0x7f417c197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f417affd640 1 -- 172.21.15.2:0/3678457697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f417c100410 msgr2=0x7f417c19d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.016+0000 7f417affd640 1 --2- 172.21.15.2:0/3678457697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f417c100410 0x7f417c19d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.017+0000 7f417affd640 1 -- 172.21.15.2:0/3678457697 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4174009900 con 0x7f417c100dd0 2024-03-15T13:06:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.017+0000 7f417affd640 1 --2- 172.21.15.2:0/3678457697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f417c100dd0 0x7f417c19dd10 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f417000e9f0 tx=0x7f417000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.017+0000 7f4178ff9640 1 -- 172.21.15.2:0/3678457697 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170009720 con 0x7f417c100dd0 2024-03-15T13:06:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.017+0000 7f4178ff9640 1 -- 172.21.15.2:0/3678457697 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f417000cd80 con 0x7f417c100dd0 2024-03-15T13:06:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.017+0000 7f4181e95640 1 -- 172.21.15.2:0/3678457697 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417c1985e0 con 0x7f417c100dd0 2024-03-15T13:06:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.017+0000 7f4178ff9640 1 -- 172.21.15.2:0/3678457697 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170010690 con 0x7f417c100dd0 2024-03-15T13:06:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.017+0000 7f4181e95640 1 -- 172.21.15.2:0/3678457697 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417c1a4870 con 0x7f417c100dd0 2024-03-15T13:06:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.019+0000 7f4178ff9640 1 -- 172.21.15.2:0/3678457697 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f4170004e70 con 0x7f417c100dd0 2024-03-15T13:06:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.019+0000 7f4184a70740 1 Processor -- start 2024-03-15T13:06:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.019+0000 7f4184a70740 1 -- start start 2024-03-15T13:06:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.019+0000 7f4184a70740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eec0f00 0x56417eec12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.019+0000 7f4184a70740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 0x56417eece620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.019+0000 7f4184a70740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eeceb60 0x56417eed0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.019+0000 7f4184a70740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56417ee7c960 con 0x56417eeceb60 2024-03-15T13:06:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.019+0000 7f4184a70740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56417ee30310 con 0x56417eec0f00 2024-03-15T13:06:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.019+0000 7f4184a70740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56417ee2d840 con 0x56417eec1820 2024-03-15T13:06:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f4178ff9640 1 --2- 172.21.15.2:0/3678457697 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f41640a1ba0 0x7f41640a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f417affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eeceb60 0x56417eed0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f417b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 0x56417eece620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f417b7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 0x56417eece620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:52488/0 (socket says 172.21.15.2:52488) 2024-03-15T13:06:54.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f417affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eeceb60 0x56417eed0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51730/0 (socket says 172.21.15.2:51730) 2024-03-15T13:06:54.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f417b7fe640 1 -- 172.21.15.2:0/3261730795 learned_addr learned my addr 172.21.15.2:0/3261730795 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:54.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f417bfff640 1 --2- 172.21.15.2:0/3261730795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eec0f00 0x56417eec12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f417bfff640 1 --2- 172.21.15.2:0/3678457697 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f41640a1ba0 0x7f41640a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f417b7fe640 1 -- 172.21.15.2:0/3261730795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eec0f00 msgr2=0x56417eec12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f417b7fe640 1 --2- 172.21.15.2:0/3261730795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eec0f00 0x56417eec12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f417b7fe640 1 -- 172.21.15.2:0/3261730795 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eeceb60 msgr2=0x56417eed0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f417b7fe640 1 --2- 172.21.15.2:0/3261730795 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eeceb60 0x56417eed0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f4178ff9640 1 -- 172.21.15.2:0/3678457697 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f4170014070 con 0x7f417c100dd0 2024-03-15T13:06:54.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f417b7fe640 1 -- 172.21.15.2:0/3261730795 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56417edeb060 con 0x56417eec1820 2024-03-15T13:06:54.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f417b7fe640 1 --2- 172.21.15.2:0/3261730795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 0x56417eece620 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f416c009c10 tx=0x7f416c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad5e96c60481a642 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f417bfff640 1 --2- 172.21.15.2:0/3678457697 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f41640a1ba0 0x7f41640a4060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4174005cc0 tx=0x7f4174005c50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f4161ffb640 1 -- 172.21.15.2:0/3261730795 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c004230 con 0x56417eec1820 2024-03-15T13:06:54.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f4161ffb640 1 -- 172.21.15.2:0/3261730795 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f416c009e80 con 0x56417eec1820 2024-03-15T13:06:54.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.020+0000 7f4161ffb640 1 -- 172.21.15.2:0/3261730795 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c004ca0 con 0x56417eec1820 2024-03-15T13:06:54.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.021+0000 7f4184a70740 1 -- 172.21.15.2:0/3261730795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 msgr2=0x56417eece620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.021+0000 7f4184a70740 1 --2- 172.21.15.2:0/3261730795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 0x56417eece620 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f416c009c10 tx=0x7f416c011810 comp rx=0 tx=0).stop 2024-03-15T13:06:54.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.021+0000 7f4184a70740 1 -- 172.21.15.2:0/3261730795 shutdown_connections 2024-03-15T13:06:54.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.021+0000 7f4184a70740 1 --2- 172.21.15.2:0/3261730795 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eeceb60 0x56417eed0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.021+0000 7f4184a70740 1 --2- 172.21.15.2:0/3261730795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 0x56417eece620 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.021+0000 7f4184a70740 1 --2- 172.21.15.2:0/3261730795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eec0f00 0x56417eec12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.021+0000 7f4184a70740 1 -- 172.21.15.2:0/3261730795 >> 172.21.15.2:0/3261730795 conn(0x56417eebb980 msgr2=0x56417eebbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.021+0000 7f4184a70740 1 -- 172.21.15.2:0/3261730795 shutdown_connections 2024-03-15T13:06:54.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.021+0000 7f4184a70740 1 -- 172.21.15.2:0/3261730795 wait complete. 2024-03-15T13:06:54.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.021+0000 7f4184a70740 1 -- start start 2024-03-15T13:06:54.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f4184a70740 1 Processor -- start 2024-03-15T13:06:54.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f4184a70740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eec0f00 0x56417eece3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f4184a70740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 0x56417eecb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f4184a70740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eeceb60 0x56417eecbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f4184a70740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56417ee29020 con 0x56417eeceb60 2024-03-15T13:06:54.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f4184a70740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56417eafc430 con 0x56417eec0f00 2024-03-15T13:06:54.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f4184a70740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56417eab8e00 con 0x56417eec1820 2024-03-15T13:06:54.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f4184a70740 10 client.0 _ll_register_callbacks cb 0x56417edd0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:54.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f417bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eeceb60 0x56417eecbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f417bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eeceb60 0x56417eecbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51746/0 (socket says 172.21.15.2:51746) 2024-03-15T13:06:54.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f417b7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eec0f00 0x56417eece3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f417affd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 0x56417eecb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f417bfff640 1 -- 172.21.15.2:0/1531979731 learned_addr learned my addr 172.21.15.2:0/1531979731 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:54.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f417bfff640 1 -- 172.21.15.2:0/1531979731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 msgr2=0x56417eecb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f417bfff640 1 --2- 172.21.15.2:0/1531979731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 0x56417eecb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f417bfff640 1 -- 172.21.15.2:0/1531979731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eec0f00 msgr2=0x56417eece3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f417bfff640 1 --2- 172.21.15.2:0/1531979731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eec0f00 0x56417eece3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f417bfff640 1 -- 172.21.15.2:0/1531979731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416c009870 con 0x56417eeceb60 2024-03-15T13:06:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f417bfff640 1 --2- 172.21.15.2:0/1531979731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eeceb60 0x56417eecbc70 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4174009cd0 tx=0x7f41740095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b140d568e9f52e5c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.022+0000 7f415affd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.023+0000 7f415affd640 1 -- 172.21.15.2:0/1531979731 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174032950 con 0x56417eeceb60 2024-03-15T13:06:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.023+0000 7f4184a70740 1 -- 172.21.15.2:0/1531979731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56417eb65720 con 0x56417eeceb60 2024-03-15T13:06:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.023+0000 7f415affd640 1 -- 172.21.15.2:0/1531979731 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f4174049040 con 0x56417eeceb60 2024-03-15T13:06:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.023+0000 7f415affd640 1 -- 172.21.15.2:0/1531979731 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4174032ab0 con 0x56417eeceb60 2024-03-15T13:06:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.023+0000 7f4184a70740 10 client.17997 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.023+0000 7f4184a70740 1 -- 172.21.15.2:0/1531979731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x56417ee276a0 con 0x56417eeceb60 2024-03-15T13:06:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.023+0000 7f415affd640 1 -- 172.21.15.2:0/1531979731 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x56417eb65720 con 0x56417eeceb60 2024-03-15T13:06:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.023+0000 7f415affd640 1 -- 172.21.15.2:0/1531979731 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f4174004660 con 0x56417eeceb60 2024-03-15T13:06:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.023+0000 7f4184a70740 10 client.17997 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.023+0000 7f4184a70740 10 client.17997 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:06:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.023+0000 7f4184a70740 20 client.17997 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.023+0000 7f4184a70740 1 --2- 172.21.15.2:0/1531979731 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56417eedec00 0x56417eee1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.023+0000 7f4184a70740 4 client.17997 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.023+0000 7f4184a70740 1 -- 172.21.15.2:0/1531979731 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56417ebb4890 con 0x56417eedec00 2024-03-15T13:06:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.023+0000 7f417b7fe640 1 --2- 172.21.15.2:0/1531979731 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56417eedec00 0x56417eee1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.024+0000 7f415affd640 10 client.17997 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:06:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.024+0000 7f417b7fe640 1 --2- 172.21.15.2:0/1531979731 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56417eedec00 0x56417eee1020 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ffcb8f571b4b1d server_cookie=165a164e0f74b520 in_seq=0 out_seq=0 2024-03-15T13:06:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.024+0000 7f415affd640 1 -- 172.21.15.2:0/1531979731 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56417ebb4890 con 0x56417eedec00 2024-03-15T13:06:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.024+0000 7f415affd640 10 client.17997 handle_command_reply: tid=0 2024-03-15T13:06:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.025+0000 7f4184a70740 1 client.17997 shutdown 2024-03-15T13:06:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.025+0000 7f4184a70740 1 -- 172.21.15.2:0/1531979731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eeceb60 msgr2=0x56417eecbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.025+0000 7f4184a70740 1 --2- 172.21.15.2:0/1531979731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eeceb60 0x56417eecbc70 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4174009cd0 tx=0x7f41740095c0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.025+0000 7f4184a70740 1 -- 172.21.15.2:0/1531979731 shutdown_connections 2024-03-15T13:06:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.025+0000 7f4184a70740 1 --2- 172.21.15.2:0/1531979731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eeceb60 0x56417eecbc70 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.025+0000 7f4184a70740 1 --2- 172.21.15.2:0/1531979731 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56417eedec00 0x56417eee1020 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.025+0000 7f4184a70740 1 --2- 172.21.15.2:0/1531979731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 0x56417eecb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.025+0000 7f4184a70740 1 --2- 172.21.15.2:0/1531979731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eec0f00 0x56417eece3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.025+0000 7f415affd640 0 client.17997 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:06:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.025+0000 7f4184a70740 1 -- 172.21.15.2:0/1531979731 >> 172.21.15.2:0/1531979731 conn(0x56417eebb980 msgr2=0x56417eebd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.025+0000 7f4184a70740 1 -- 172.21.15.2:0/1531979731 shutdown_connections 2024-03-15T13:06:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.025+0000 7f4184a70740 1 -- 172.21.15.2:0/1531979731 wait complete. 2024-03-15T13:06:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.025+0000 7f4184a70740 20 client.17997 trim_cache size 0 max 16384 2024-03-15T13:06:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.037+0000 7f4184a70740 1 Processor -- start 2024-03-15T13:06:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.037+0000 7f4184a70740 1 -- start start 2024-03-15T13:06:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.037+0000 7f4184a70740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eec0f00 0x56417eebd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.037+0000 7f4184a70740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eec1820 0x56417eebaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.037+0000 7f4184a70740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eeceb60 0x56417eecde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.037+0000 7f4184a70740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56417edae130 con 0x56417eec1820 2024-03-15T13:06:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.037+0000 7f4184a70740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56417eeb75c0 con 0x56417eec0f00 2024-03-15T13:06:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.037+0000 7f4184a70740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56417eebc980 con 0x56417eeceb60 2024-03-15T13:06:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.037+0000 7f417affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eec1820 0x56417eebaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.038+0000 7f417b7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eec0f00 0x56417eebd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.038+0000 7f417affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eec1820 0x56417eebaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51756/0 (socket says 172.21.15.2:51756) 2024-03-15T13:06:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.038+0000 7f417affd640 1 -- 172.21.15.2:0/1513725587 learned_addr learned my addr 172.21.15.2:0/1513725587 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.038+0000 7f417bfff640 1 --2- 172.21.15.2:0/1513725587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eeceb60 0x56417eecde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.038+0000 7f417affd640 1 -- 172.21.15.2:0/1513725587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eeceb60 msgr2=0x56417eecde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.038+0000 7f417affd640 1 --2- 172.21.15.2:0/1513725587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eeceb60 0x56417eecde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.038+0000 7f417affd640 1 -- 172.21.15.2:0/1513725587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eec0f00 msgr2=0x56417eebd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.038+0000 7f417affd640 1 --2- 172.21.15.2:0/1513725587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eec0f00 0x56417eebd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.038+0000 7f417affd640 1 -- 172.21.15.2:0/1513725587 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56417edc3c20 con 0x56417eec1820 2024-03-15T13:06:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.038+0000 7f417affd640 1 --2- 172.21.15.2:0/1513725587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eec1820 0x56417eebaa00 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f417009a4e0 tx=0x7f4170085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f551492d820f659 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.038+0000 7f41617fa640 1 -- 172.21.15.2:0/1513725587 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417009a140 con 0x56417eec1820 2024-03-15T13:06:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.038+0000 7f41617fa640 1 -- 172.21.15.2:0/1513725587 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f417009bb10 con 0x56417eec1820 2024-03-15T13:06:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.038+0000 7f41617fa640 1 -- 172.21.15.2:0/1513725587 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41700b3340 con 0x56417eec1820 2024-03-15T13:06:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 -- 172.21.15.2:0/1513725587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eec1820 msgr2=0x56417eebaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 --2- 172.21.15.2:0/1513725587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eec1820 0x56417eebaa00 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f417009a4e0 tx=0x7f4170085040 comp rx=0 tx=0).stop 2024-03-15T13:06:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 -- 172.21.15.2:0/1513725587 shutdown_connections 2024-03-15T13:06:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 --2- 172.21.15.2:0/1513725587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eeceb60 0x56417eecde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 --2- 172.21.15.2:0/1513725587 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eec1820 0x56417eebaa00 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 --2- 172.21.15.2:0/1513725587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eec0f00 0x56417eebd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 -- 172.21.15.2:0/1513725587 >> 172.21.15.2:0/1513725587 conn(0x56417eebb980 msgr2=0x56417eeba2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 -- 172.21.15.2:0/1513725587 shutdown_connections 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 -- 172.21.15.2:0/1513725587 wait complete. 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 -- start start 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 Processor -- start 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eec0f00 0x56417eed3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 0x56417eed3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eeceb60 0x56417eed3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56417ef3a560 con 0x56417eec0f00 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56417ef3a6d0 con 0x56417eeceb60 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56417ef3a890 con 0x56417eec1820 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f4184a70740 10 client.0 _ll_register_callbacks cb 0x7f4170005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.039+0000 7f417b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eec0f00 0x56417eed3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f417b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eec0f00 0x56417eed3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51772/0 (socket says 172.21.15.2:51772) 2024-03-15T13:06:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f417b7fe640 1 -- 172.21.15.2:0/122689215 learned_addr learned my addr 172.21.15.2:0/122689215 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f417bfff640 1 --2- 172.21.15.2:0/122689215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eeceb60 0x56417eed3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f417affd640 1 --2- 172.21.15.2:0/122689215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 0x56417eed3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f417b7fe640 1 -- 172.21.15.2:0/122689215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 msgr2=0x56417eed3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f417b7fe640 1 --2- 172.21.15.2:0/122689215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 0x56417eed3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f417b7fe640 1 -- 172.21.15.2:0/122689215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eeceb60 msgr2=0x56417eed3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f417b7fe640 1 --2- 172.21.15.2:0/122689215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eeceb60 0x56417eed3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f417b7fe640 1 -- 172.21.15.2:0/122689215 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417c1a4870 con 0x56417eec0f00 2024-03-15T13:06:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f417b7fe640 1 --2- 172.21.15.2:0/122689215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eec0f00 0x56417eed3050 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f416c000c00 tx=0x7f416c0097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907ddf092d86735b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f415affd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f415affd640 1 -- 172.21.15.2:0/122689215 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c029040 con 0x56417eec0f00 2024-03-15T13:06:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f415affd640 1 -- 172.21.15.2:0/122689215 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f416c004550 con 0x56417eec0f00 2024-03-15T13:06:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f4184a70740 1 -- 172.21.15.2:0/122689215 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56417eb287e0 con 0x56417eec0f00 2024-03-15T13:06:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f415affd640 1 -- 172.21.15.2:0/122689215 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f416c0035f0 con 0x56417eec0f00 2024-03-15T13:06:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f4184a70740 10 client.18003 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f4184a70740 1 -- 172.21.15.2:0/122689215 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x56417ede98c0 con 0x56417eec0f00 2024-03-15T13:06:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.040+0000 7f415affd640 1 -- 172.21.15.2:0/122689215 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x56417eb287e0 con 0x56417eec0f00 2024-03-15T13:06:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.041+0000 7f415affd640 1 -- 172.21.15.2:0/122689215 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f416c024620 con 0x56417eec0f00 2024-03-15T13:06:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.041+0000 7f4184a70740 10 client.18003 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.041+0000 7f4184a70740 10 client.18003 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:06:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.041+0000 7f4184a70740 20 client.18003 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.041+0000 7f4184a70740 1 --2- 172.21.15.2:0/122689215 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56417ef3c050 0x56417ef3e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.041+0000 7f4184a70740 4 client.18003 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:06:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.041+0000 7f4184a70740 1 -- 172.21.15.2:0/122689215 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56417ead4700 con 0x56417ef3c050 2024-03-15T13:06:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.041+0000 7f417affd640 1 --2- 172.21.15.2:0/122689215 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56417ef3c050 0x56417ef3e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.041+0000 7f415affd640 10 client.18003 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:06:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.041+0000 7f417affd640 1 --2- 172.21.15.2:0/122689215 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56417ef3c050 0x56417ef3e470 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f61d1e8a0b35762 server_cookie=2c5aca3f47bd6401 in_seq=0 out_seq=0 2024-03-15T13:06:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.041+0000 7f415affd640 1 -- 172.21.15.2:0/122689215 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56417ead4700 con 0x56417ef3c050 2024-03-15T13:06:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.041+0000 7f415affd640 10 client.18003 handle_command_reply: tid=0 2024-03-15T13:06:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.041+0000 7f4184a70740 1 client.18003 shutdown 2024-03-15T13:06:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.041+0000 7f4184a70740 1 -- 172.21.15.2:0/122689215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eec0f00 msgr2=0x56417eed3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.041+0000 7f4184a70740 1 --2- 172.21.15.2:0/122689215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eec0f00 0x56417eed3050 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f416c000c00 tx=0x7f416c0097e0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4184a70740 1 -- 172.21.15.2:0/122689215 shutdown_connections 2024-03-15T13:06:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4184a70740 1 --2- 172.21.15.2:0/122689215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56417eeceb60 0x56417eed3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4184a70740 1 --2- 172.21.15.2:0/122689215 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x56417ef3c050 0x56417ef3e470 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4184a70740 1 --2- 172.21.15.2:0/122689215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56417eec1820 0x56417eed3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4184a70740 1 --2- 172.21.15.2:0/122689215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56417eec0f00 0x56417eed3050 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f415affd640 0 client.18003 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:06:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4184a70740 1 -- 172.21.15.2:0/122689215 >> 172.21.15.2:0/122689215 conn(0x56417eebb980 msgr2=0x56417eebab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4184a70740 1 -- 172.21.15.2:0/122689215 shutdown_connections 2024-03-15T13:06:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4184a70740 1 -- 172.21.15.2:0/122689215 wait complete. 2024-03-15T13:06:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4184a70740 20 client.18003 trim_cache size 0 max 16384 2024-03-15T13:06:54.033 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:06:54.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4181e95640 1 -- 172.21.15.2:0/3678457697 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f41640a1ba0 msgr2=0x7f41640a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4181e95640 1 --2- 172.21.15.2:0/3678457697 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f41640a1ba0 0x7f41640a4060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4174005cc0 tx=0x7f4174005c50 comp rx=0 tx=0).stop 2024-03-15T13:06:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4181e95640 1 -- 172.21.15.2:0/3678457697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f417c100dd0 msgr2=0x7f417c19dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4181e95640 1 --2- 172.21.15.2:0/3678457697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f417c100dd0 0x7f417c19dd10 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f417000e9f0 tx=0x7f417000edf0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4181e95640 1 -- 172.21.15.2:0/3678457697 shutdown_connections 2024-03-15T13:06:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4181e95640 1 --2- 172.21.15.2:0/3678457697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f417c1064b0 0x7f417c197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4181e95640 1 --2- 172.21.15.2:0/3678457697 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f41640a1ba0 0x7f41640a4060 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4181e95640 1 --2- 172.21.15.2:0/3678457697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f417c100dd0 0x7f417c19dd10 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4181e95640 1 --2- 172.21.15.2:0/3678457697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f417c100410 0x7f417c19d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4181e95640 1 -- 172.21.15.2:0/3678457697 >> 172.21.15.2:0/3678457697 conn(0x7f417c077550 msgr2=0x7f417c069e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4181e95640 1 -- 172.21.15.2:0/3678457697 shutdown_connections 2024-03-15T13:06:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.042+0000 7f4181e95640 1 -- 172.21.15.2:0/3678457697 wait complete. 2024-03-15T13:06:54.037 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:06:54.038 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:06:54.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.157+0000 7f1759e4d640 1 -- 172.21.15.2:0/502686316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1754107570 msgr2=0x7f1754107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.157+0000 7f1759e4d640 1 --2- 172.21.15.2:0/502686316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1754107570 0x7f1754107950 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f1744009900 tx=0x7f174402bc30 comp rx=0 tx=0).stop 2024-03-15T13:06:54.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.157+0000 7f1759e4d640 1 -- 172.21.15.2:0/502686316 shutdown_connections 2024-03-15T13:06:54.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.157+0000 7f1759e4d640 1 --2- 172.21.15.2:0/502686316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1754101e70 0x7f175410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.157+0000 7f1759e4d640 1 --2- 172.21.15.2:0/502686316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17541014b0 0x7f1754101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.157+0000 7f1759e4d640 1 --2- 172.21.15.2:0/502686316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1754107570 0x7f1754107950 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.157+0000 7f1759e4d640 1 -- 172.21.15.2:0/502686316 >> 172.21.15.2:0/502686316 conn(0x7f17540fcf20 msgr2=0x7f17540ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.157+0000 7f1759e4d640 1 -- 172.21.15.2:0/502686316 shutdown_connections 2024-03-15T13:06:54.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.157+0000 7f1759e4d640 1 -- 172.21.15.2:0/502686316 wait complete. 2024-03-15T13:06:54.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.158+0000 7f1759e4d640 1 Processor -- start 2024-03-15T13:06:54.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.158+0000 7f1759e4d640 1 -- start start 2024-03-15T13:06:54.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.158+0000 7f1759e4d640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17541014b0 0x7f175419d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.158+0000 7f1759e4d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1754101e70 0x7f175419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.158+0000 7f1759e4d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1754107570 0x7f1754197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.158+0000 7f1759e4d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1754197fd0 con 0x7f1754101e70 2024-03-15T13:06:54.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.158+0000 7f1759e4d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1754198140 con 0x7f1754107570 2024-03-15T13:06:54.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.158+0000 7f1759e4d640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1754198300 con 0x7f17541014b0 2024-03-15T13:06:54.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.158+0000 7f1752ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1754101e70 0x7f175419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.158+0000 7f17537fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17541014b0 0x7f175419d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.158+0000 7f17537fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17541014b0 0x7f175419d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:52536/0 (socket says 172.21.15.2:52536) 2024-03-15T13:06:54.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.158+0000 7f1752ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1754101e70 0x7f175419dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51790/0 (socket says 172.21.15.2:51790) 2024-03-15T13:06:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.158+0000 7f17537fe640 1 -- 172.21.15.2:0/2642655483 learned_addr learned my addr 172.21.15.2:0/2642655483 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.158+0000 7f1753fff640 1 --2- 172.21.15.2:0/2642655483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1754107570 0x7f1754197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.159+0000 7f17537fe640 1 -- 172.21.15.2:0/2642655483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1754107570 msgr2=0x7f1754197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.159+0000 7f17537fe640 1 --2- 172.21.15.2:0/2642655483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1754107570 0x7f1754197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.159+0000 7f17537fe640 1 -- 172.21.15.2:0/2642655483 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1754101e70 msgr2=0x7f175419dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.159+0000 7f17537fe640 1 --2- 172.21.15.2:0/2642655483 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1754101e70 0x7f175419dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.159+0000 7f17537fe640 1 -- 172.21.15.2:0/2642655483 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1744009560 con 0x7f17541014b0 2024-03-15T13:06:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.159+0000 7f1752ffd640 1 --2- 172.21.15.2:0/2642655483 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1754101e70 0x7f175419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.159+0000 7f17537fe640 1 --2- 172.21.15.2:0/2642655483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17541014b0 0x7f175419d780 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1744009230 tx=0x7f174402cac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.159+0000 7f1750ff9640 1 -- 172.21.15.2:0/2642655483 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1744004250 con 0x7f17541014b0 2024-03-15T13:06:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.159+0000 7f1750ff9640 1 -- 172.21.15.2:0/2642655483 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f17440043b0 con 0x7f17541014b0 2024-03-15T13:06:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.159+0000 7f1759e4d640 1 -- 172.21.15.2:0/2642655483 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1754198580 con 0x7f17541014b0 2024-03-15T13:06:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.159+0000 7f1750ff9640 1 -- 172.21.15.2:0/2642655483 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1744003bf0 con 0x7f17541014b0 2024-03-15T13:06:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.159+0000 7f1759e4d640 1 -- 172.21.15.2:0/2642655483 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17541a4820 con 0x7f17541014b0 2024-03-15T13:06:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.161+0000 7f175ca20740 1 Processor -- start 2024-03-15T13:06:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.161+0000 7f175ca20740 1 -- start start 2024-03-15T13:06:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.161+0000 7f1750ff9640 1 -- 172.21.15.2:0/2642655483 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f1744003d50 con 0x7f17541014b0 2024-03-15T13:06:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f175ca20740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9d250 0x55f951d9d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f175ca20740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951d9db70 0x55f951da6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f175ca20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951da6750 0x55f951da8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f175ca20740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f951d58960 con 0x55f951d9d250 2024-03-15T13:06:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f175ca20740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f951d0c310 con 0x55f951d9db70 2024-03-15T13:06:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f175ca20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f951d09840 con 0x55f951da6750 2024-03-15T13:06:54.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f1753fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9d250 0x55f951d9d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f1753fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9d250 0x55f951d9d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51802/0 (socket says 172.21.15.2:51802) 2024-03-15T13:06:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f1753fff640 1 -- 172.21.15.2:0/2907463646 learned_addr learned my addr 172.21.15.2:0/2907463646 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f17537fe640 1 --2- 172.21.15.2:0/2907463646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951d9db70 0x55f951da6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f1752ffd640 1 --2- 172.21.15.2:0/2907463646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951da6750 0x55f951da8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f1753fff640 1 -- 172.21.15.2:0/2907463646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951da6750 msgr2=0x55f951da8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f1753fff640 1 --2- 172.21.15.2:0/2907463646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951da6750 0x55f951da8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f1753fff640 1 -- 172.21.15.2:0/2907463646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951d9db70 msgr2=0x55f951da6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f1753fff640 1 --2- 172.21.15.2:0/2907463646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951d9db70 0x55f951da6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f1753fff640 1 -- 172.21.15.2:0/2907463646 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f951cc7060 con 0x55f951d9d250 2024-03-15T13:06:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f1750ff9640 1 --2- 172.21.15.2:0/2642655483 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f173c0a1ba0 0x7f173c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f1753fff640 1 --2- 172.21.15.2:0/2907463646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9d250 0x55f951d9d630 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f174c009f50 tx=0x7f174c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4266a357374206db server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f17537fe640 1 --2- 172.21.15.2:0/2642655483 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f173c0a1ba0 0x7f173c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f1739ffb640 1 -- 172.21.15.2:0/2907463646 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c004d20 con 0x55f951d9d250 2024-03-15T13:06:54.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f1750ff9640 1 -- 172.21.15.2:0/2642655483 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f17440efaf0 con 0x7f17541014b0 2024-03-15T13:06:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f1739ffb640 1 -- 172.21.15.2:0/2907463646 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f174c004e80 con 0x55f951d9d250 2024-03-15T13:06:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.162+0000 7f1739ffb640 1 -- 172.21.15.2:0/2907463646 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c005240 con 0x55f951d9d250 2024-03-15T13:06:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.163+0000 7f175ca20740 1 -- 172.21.15.2:0/2907463646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9d250 msgr2=0x55f951d9d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.163+0000 7f17537fe640 1 --2- 172.21.15.2:0/2642655483 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f173c0a1ba0 0x7f173c0a4060 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f17440ed060 tx=0x7f17440a7040 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.163+0000 7f175ca20740 1 --2- 172.21.15.2:0/2907463646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9d250 0x55f951d9d630 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f174c009f50 tx=0x7f174c0118d0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.163+0000 7f175ca20740 1 -- 172.21.15.2:0/2907463646 shutdown_connections 2024-03-15T13:06:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.163+0000 7f175ca20740 1 --2- 172.21.15.2:0/2907463646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951da6750 0x55f951da8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.163+0000 7f175ca20740 1 --2- 172.21.15.2:0/2907463646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951d9db70 0x55f951da6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.163+0000 7f175ca20740 1 --2- 172.21.15.2:0/2907463646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9d250 0x55f951d9d630 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.163+0000 7f175ca20740 1 -- 172.21.15.2:0/2907463646 >> 172.21.15.2:0/2907463646 conn(0x55f951d97980 msgr2=0x55f951d97d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.163+0000 7f175ca20740 1 -- 172.21.15.2:0/2907463646 shutdown_connections 2024-03-15T13:06:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.163+0000 7f175ca20740 1 -- 172.21.15.2:0/2907463646 wait complete. 2024-03-15T13:06:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.163+0000 7f175ca20740 1 -- start start 2024-03-15T13:06:54.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f175ca20740 1 Processor -- start 2024-03-15T13:06:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f175ca20740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9d250 0x55f951da6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f175ca20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9db70 0x55f951da33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f175ca20740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951da6750 0x55f951da38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f175ca20740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f951d05020 con 0x55f951d9d250 2024-03-15T13:06:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f175ca20740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9519d8430 con 0x55f951da6750 2024-03-15T13:06:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f175ca20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f951994e00 con 0x55f951d9db70 2024-03-15T13:06:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f175ca20740 10 client.0 _ll_register_callbacks cb 0x55f951caccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f1753fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9d250 0x55f951da6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f1753fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9d250 0x55f951da6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51808/0 (socket says 172.21.15.2:51808) 2024-03-15T13:06:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f1753fff640 1 -- 172.21.15.2:0/4203551969 learned_addr learned my addr 172.21.15.2:0/4203551969 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f1752ffd640 1 --2- 172.21.15.2:0/4203551969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9db70 0x55f951da33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f1753fff640 1 --2- 172.21.15.2:0/4203551969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951da6750 0x55f951da38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f1753fff640 1 -- 172.21.15.2:0/4203551969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9db70 msgr2=0x55f951da33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f1753fff640 1 --2- 172.21.15.2:0/4203551969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9db70 0x55f951da33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f1753fff640 1 -- 172.21.15.2:0/4203551969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951da6750 msgr2=0x55f951da38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f1753fff640 1 --2- 172.21.15.2:0/4203551969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951da6750 0x55f951da38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f1753fff640 1 -- 172.21.15.2:0/4203551969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c009c00 con 0x55f951d9d250 2024-03-15T13:06:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f1752ffd640 1 --2- 172.21.15.2:0/4203551969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9db70 0x55f951da33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:06:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f1753fff640 1 --2- 172.21.15.2:0/4203551969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9d250 0x55f951da6010 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f174c009250 tx=0x7f174c0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e9698ce67e1f7c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f1732ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.164+0000 7f1732ffd640 1 -- 172.21.15.2:0/4203551969 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c005000 con 0x55f951d9d250 2024-03-15T13:06:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.165+0000 7f1732ffd640 1 -- 172.21.15.2:0/4203551969 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f174c005160 con 0x55f951d9d250 2024-03-15T13:06:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.165+0000 7f175ca20740 1 -- 172.21.15.2:0/4203551969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f951a41720 con 0x55f951d9d250 2024-03-15T13:06:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.165+0000 7f1732ffd640 1 -- 172.21.15.2:0/4203551969 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f174c005320 con 0x55f951d9d250 2024-03-15T13:06:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.165+0000 7f175ca20740 10 client.18015 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.165+0000 7f175ca20740 1 -- 172.21.15.2:0/4203551969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55f951d036a0 con 0x55f951d9d250 2024-03-15T13:06:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.165+0000 7f1732ffd640 1 -- 172.21.15.2:0/4203551969 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f174c036080 con 0x55f951d9d250 2024-03-15T13:06:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.165+0000 7f1732ffd640 1 -- 172.21.15.2:0/4203551969 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f174c025a80 con 0x55f951d9d250 2024-03-15T13:06:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.165+0000 7f175ca20740 10 client.18015 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.165+0000 7f175ca20740 10 client.18015 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:06:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.165+0000 7f175ca20740 20 client.18015 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.165+0000 7f175ca20740 1 --2- 172.21.15.2:0/4203551969 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f951db67c0 0x55f951db8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.165+0000 7f175ca20740 4 client.18015 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.165+0000 7f175ca20740 1 -- 172.21.15.2:0/4203551969 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f951a90890 con 0x55f951db67c0 2024-03-15T13:06:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.165+0000 7f1752ffd640 1 --2- 172.21.15.2:0/4203551969 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f951db67c0 0x55f951db8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.166+0000 7f1732ffd640 10 client.18015 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:06:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.166+0000 7f1752ffd640 1 --2- 172.21.15.2:0/4203551969 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f951db67c0 0x55f951db8be0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=550813053612ade9 server_cookie=e3ea55994b8dc525 in_seq=0 out_seq=0 2024-03-15T13:06:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.166+0000 7f1732ffd640 1 -- 172.21.15.2:0/4203551969 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f951a90890 con 0x55f951db67c0 2024-03-15T13:06:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.166+0000 7f1732ffd640 10 client.18015 handle_command_reply: tid=0 2024-03-15T13:06:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.166+0000 7f175ca20740 1 client.18015 shutdown 2024-03-15T13:06:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.166+0000 7f175ca20740 1 -- 172.21.15.2:0/4203551969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9d250 msgr2=0x55f951da6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.166+0000 7f175ca20740 1 --2- 172.21.15.2:0/4203551969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9d250 0x55f951da6010 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f174c009250 tx=0x7f174c0046a0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.166+0000 7f175ca20740 1 -- 172.21.15.2:0/4203551969 shutdown_connections 2024-03-15T13:06:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.166+0000 7f175ca20740 1 --2- 172.21.15.2:0/4203551969 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f951db67c0 0x55f951db8be0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.166+0000 7f175ca20740 1 --2- 172.21.15.2:0/4203551969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951da6750 0x55f951da38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.167+0000 7f1732ffd640 0 client.18015 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:06:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.167+0000 7f175ca20740 1 --2- 172.21.15.2:0/4203551969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9db70 0x55f951da33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.167+0000 7f175ca20740 1 --2- 172.21.15.2:0/4203551969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9d250 0x55f951da6010 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.167+0000 7f175ca20740 1 -- 172.21.15.2:0/4203551969 >> 172.21.15.2:0/4203551969 conn(0x55f951d97980 msgr2=0x55f951d996d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.167+0000 7f175ca20740 1 -- 172.21.15.2:0/4203551969 shutdown_connections 2024-03-15T13:06:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.167+0000 7f175ca20740 1 -- 172.21.15.2:0/4203551969 wait complete. 2024-03-15T13:06:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.167+0000 7f175ca20740 20 client.18015 trim_cache size 0 max 16384 2024-03-15T13:06:54.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.179+0000 7f175ca20740 1 Processor -- start 2024-03-15T13:06:54.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.179+0000 7f175ca20740 1 -- start start 2024-03-15T13:06:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.179+0000 7f175ca20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9d250 0x55f951d92a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.179+0000 7f175ca20740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9db70 0x55f951da5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.179+0000 7f175ca20740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951da6750 0x55f951da6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.179+0000 7f175ca20740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f951c8a130 con 0x55f951d9db70 2024-03-15T13:06:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.179+0000 7f175ca20740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f951da3190 con 0x55f951da6750 2024-03-15T13:06:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.179+0000 7f175ca20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f951d9c610 con 0x55f951d9d250 2024-03-15T13:06:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.180+0000 7f1753fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9d250 0x55f951d92a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.180+0000 7f1752ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9db70 0x55f951da5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.180+0000 7f1753fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9d250 0x55f951d92a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:52564/0 (socket says 172.21.15.2:52564) 2024-03-15T13:06:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.180+0000 7f1753fff640 1 -- 172.21.15.2:0/1739732710 learned_addr learned my addr 172.21.15.2:0/1739732710 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.180+0000 7f1753fff640 1 --2- 172.21.15.2:0/1739732710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951da6750 0x55f951da6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.180+0000 7f1753fff640 1 -- 172.21.15.2:0/1739732710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951da6750 msgr2=0x55f951da6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.180+0000 7f1753fff640 1 --2- 172.21.15.2:0/1739732710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951da6750 0x55f951da6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.180+0000 7f1753fff640 1 -- 172.21.15.2:0/1739732710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9db70 msgr2=0x55f951da5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.180+0000 7f1753fff640 1 --2- 172.21.15.2:0/1739732710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9db70 0x55f951da5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.180+0000 7f1753fff640 1 -- 172.21.15.2:0/1739732710 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f951c9fc20 con 0x55f951d9d250 2024-03-15T13:06:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.180+0000 7f1752ffd640 1 --2- 172.21.15.2:0/1739732710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9db70 0x55f951da5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.180+0000 7f1753fff640 1 --2- 172.21.15.2:0/1739732710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9d250 0x55f951d92a10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f174c011ed0 tx=0x7f174c014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d6e5312f6671c55 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.180+0000 7f17397fa640 1 -- 172.21.15.2:0/1739732710 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c003dc0 con 0x55f951d9d250 2024-03-15T13:06:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.180+0000 7f17397fa640 1 -- 172.21.15.2:0/1739732710 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f174c03f070 con 0x55f951d9d250 2024-03-15T13:06:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.180+0000 7f17397fa640 1 -- 172.21.15.2:0/1739732710 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c0256b0 con 0x55f951d9d250 2024-03-15T13:06:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 -- 172.21.15.2:0/1739732710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9d250 msgr2=0x55f951d92a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 --2- 172.21.15.2:0/1739732710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9d250 0x55f951d92a10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f174c011ed0 tx=0x7f174c014040 comp rx=0 tx=0).stop 2024-03-15T13:06:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 -- 172.21.15.2:0/1739732710 shutdown_connections 2024-03-15T13:06:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 --2- 172.21.15.2:0/1739732710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951da6750 0x55f951da6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 --2- 172.21.15.2:0/1739732710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951d9db70 0x55f951da5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 --2- 172.21.15.2:0/1739732710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9d250 0x55f951d92a10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 -- 172.21.15.2:0/1739732710 >> 172.21.15.2:0/1739732710 conn(0x55f951d97980 msgr2=0x55f951d96b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 -- 172.21.15.2:0/1739732710 shutdown_connections 2024-03-15T13:06:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 -- 172.21.15.2:0/1739732710 wait complete. 2024-03-15T13:06:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 -- start start 2024-03-15T13:06:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 Processor -- start 2024-03-15T13:06:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9d250 0x55f951daa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951d9db70 0x55f951daa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951da6750 0x55f951dad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f951dadef0 con 0x55f951da6750 2024-03-15T13:06:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f951dae060 con 0x55f951d9db70 2024-03-15T13:06:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.181+0000 7f175ca20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f951dae220 con 0x55f951d9d250 2024-03-15T13:06:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f175ca20740 10 client.0 _ll_register_callbacks cb 0x7f17440f34d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1753fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951da6750 0x55f951dad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1753fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951da6750 0x55f951dad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51822/0 (socket says 172.21.15.2:51822) 2024-03-15T13:06:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1753fff640 1 -- 172.21.15.2:0/2827667068 learned_addr learned my addr 172.21.15.2:0/2827667068 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:54.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1752ffd640 1 --2- 172.21.15.2:0/2827667068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951d9db70 0x55f951daa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1753fff640 1 --2- 172.21.15.2:0/2827667068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9d250 0x55f951daa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1753fff640 1 -- 172.21.15.2:0/2827667068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9d250 msgr2=0x55f951daa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1753fff640 1 --2- 172.21.15.2:0/2827667068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9d250 0x55f951daa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1753fff640 1 -- 172.21.15.2:0/2827667068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951d9db70 msgr2=0x55f951daa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1753fff640 1 --2- 172.21.15.2:0/2827667068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951d9db70 0x55f951daa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1753fff640 1 -- 172.21.15.2:0/2827667068 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1748020770 con 0x55f951da6750 2024-03-15T13:06:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1752ffd640 1 --2- 172.21.15.2:0/2827667068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951d9db70 0x55f951daa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1753fff640 1 --2- 172.21.15.2:0/2827667068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951da6750 0x55f951dad9b0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f174c009b90 tx=0x7f174c005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5576804c5915e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1732ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1732ffd640 1 -- 172.21.15.2:0/2827667068 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c0277d0 con 0x55f951da6750 2024-03-15T13:06:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1732ffd640 1 -- 172.21.15.2:0/2827667068 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f174c039030 con 0x55f951da6750 2024-03-15T13:06:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f175ca20740 1 -- 172.21.15.2:0/2827667068 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f951d930f0 con 0x55f951da6750 2024-03-15T13:06:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f1732ffd640 1 -- 172.21.15.2:0/2827667068 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f174c027930 con 0x55f951da6750 2024-03-15T13:06:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f175ca20740 10 client.18021 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:54.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.182+0000 7f175ca20740 1 -- 172.21.15.2:0/2827667068 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55f951d036a0 con 0x55f951da6750 2024-03-15T13:06:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.183+0000 7f1732ffd640 1 -- 172.21.15.2:0/2827667068 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55f951d930f0 con 0x55f951da6750 2024-03-15T13:06:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.183+0000 7f1732ffd640 1 -- 172.21.15.2:0/2827667068 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f174c030440 con 0x55f951da6750 2024-03-15T13:06:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.183+0000 7f175ca20740 10 client.18021 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.183+0000 7f175ca20740 10 client.18021 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:06:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.183+0000 7f175ca20740 20 client.18021 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.183+0000 7f175ca20740 1 --2- 172.21.15.2:0/2827667068 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f951db67c0 0x55f951e14d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.183+0000 7f175ca20740 4 client.18021 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:06:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.183+0000 7f175ca20740 1 -- 172.21.15.2:0/2827667068 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f951a047e0 con 0x55f951db67c0 2024-03-15T13:06:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.183+0000 7f1752ffd640 1 --2- 172.21.15.2:0/2827667068 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f951db67c0 0x55f951e14d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.183+0000 7f1752ffd640 1 --2- 172.21.15.2:0/2827667068 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f951db67c0 0x55f951e14d00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb7cd4f333114875 server_cookie=b7078a39c3ec6bb3 in_seq=0 out_seq=0 2024-03-15T13:06:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.183+0000 7f1732ffd640 10 client.18021 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:06:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.183+0000 7f1732ffd640 1 -- 172.21.15.2:0/2827667068 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f951a047e0 con 0x55f951db67c0 2024-03-15T13:06:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.183+0000 7f1732ffd640 10 client.18021 handle_command_reply: tid=0 2024-03-15T13:06:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f175ca20740 1 client.18021 shutdown 2024-03-15T13:06:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f175ca20740 1 -- 172.21.15.2:0/2827667068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951da6750 msgr2=0x55f951dad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f175ca20740 1 --2- 172.21.15.2:0/2827667068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951da6750 0x55f951dad9b0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f174c009b90 tx=0x7f174c005cf0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f175ca20740 1 -- 172.21.15.2:0/2827667068 shutdown_connections 2024-03-15T13:06:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f175ca20740 1 --2- 172.21.15.2:0/2827667068 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55f951db67c0 0x55f951e14d00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f175ca20740 1 --2- 172.21.15.2:0/2827667068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f951da6750 0x55f951dad9b0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f1732ffd640 0 client.18021 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:06:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f175ca20740 1 --2- 172.21.15.2:0/2827667068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f951d9db70 0x55f951daa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f175ca20740 1 --2- 172.21.15.2:0/2827667068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f951d9d250 0x55f951daa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f175ca20740 1 -- 172.21.15.2:0/2827667068 >> 172.21.15.2:0/2827667068 conn(0x55f951d97980 msgr2=0x55f951d9df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f175ca20740 1 -- 172.21.15.2:0/2827667068 shutdown_connections 2024-03-15T13:06:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f175ca20740 1 -- 172.21.15.2:0/2827667068 wait complete. 2024-03-15T13:06:54.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f175ca20740 20 client.18021 trim_cache size 0 max 16384 2024-03-15T13:06:54.176 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:06:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f1759e4d640 1 -- 172.21.15.2:0/2642655483 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f173c0a1ba0 msgr2=0x7f173c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f1759e4d640 1 --2- 172.21.15.2:0/2642655483 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f173c0a1ba0 0x7f173c0a4060 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f17440ed060 tx=0x7f17440a7040 comp rx=0 tx=0).stop 2024-03-15T13:06:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f1759e4d640 1 -- 172.21.15.2:0/2642655483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17541014b0 msgr2=0x7f175419d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f1759e4d640 1 --2- 172.21.15.2:0/2642655483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17541014b0 0x7f175419d780 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1744009230 tx=0x7f174402cac0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f1759e4d640 1 -- 172.21.15.2:0/2642655483 shutdown_connections 2024-03-15T13:06:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f1759e4d640 1 --2- 172.21.15.2:0/2642655483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1754107570 0x7f1754197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f1759e4d640 1 --2- 172.21.15.2:0/2642655483 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f173c0a1ba0 0x7f173c0a4060 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f1759e4d640 1 --2- 172.21.15.2:0/2642655483 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1754101e70 0x7f175419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f1759e4d640 1 --2- 172.21.15.2:0/2642655483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17541014b0 0x7f175419d780 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f1759e4d640 1 -- 172.21.15.2:0/2642655483 >> 172.21.15.2:0/2642655483 conn(0x7f17540fcf20 msgr2=0x7f17540fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.184+0000 7f1759e4d640 1 -- 172.21.15.2:0/2642655483 shutdown_connections 2024-03-15T13:06:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.185+0000 7f1759e4d640 1 -- 172.21.15.2:0/2642655483 wait complete. 2024-03-15T13:06:54.180 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:06:54.180 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:06:54.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.300+0000 7f2716a07640 1 -- 172.21.15.2:0/4234047326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27101014a0 msgr2=0x7f2710101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.300+0000 7f2716a07640 1 --2- 172.21.15.2:0/4234047326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27101014a0 0x7f2710101920 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2704009930 tx=0x7f270402bc30 comp rx=0 tx=0).stop 2024-03-15T13:06:54.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.300+0000 7f2716a07640 1 -- 172.21.15.2:0/4234047326 shutdown_connections 2024-03-15T13:06:54.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.300+0000 7f2716a07640 1 --2- 172.21.15.2:0/4234047326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2710101e60 0x7f271010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.300+0000 7f2716a07640 1 --2- 172.21.15.2:0/4234047326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27101014a0 0x7f2710101920 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.300+0000 7f2716a07640 1 --2- 172.21.15.2:0/4234047326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2710107560 0x7f2710107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.300+0000 7f2716a07640 1 -- 172.21.15.2:0/4234047326 >> 172.21.15.2:0/4234047326 conn(0x7f27100788f0 msgr2=0x7f2710078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.300+0000 7f2716a07640 1 -- 172.21.15.2:0/4234047326 shutdown_connections 2024-03-15T13:06:54.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.300+0000 7f2716a07640 1 -- 172.21.15.2:0/4234047326 wait complete. 2024-03-15T13:06:54.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f2716a07640 1 Processor -- start 2024-03-15T13:06:54.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f2716a07640 1 -- start start 2024-03-15T13:06:54.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f2716a07640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27101014a0 0x7f271019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f2716a07640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2710101e60 0x7f271019dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f2716a07640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2710107560 0x7f27101978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f2716a07640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710197f10 con 0x7f27101014a0 2024-03-15T13:06:54.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f2716a07640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710198080 con 0x7f2710101e60 2024-03-15T13:06:54.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f2716a07640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710198240 con 0x7f2710107560 2024-03-15T13:06:54.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f270ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27101014a0 0x7f271019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f270ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27101014a0 0x7f271019d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51834/0 (socket says 172.21.15.2:51834) 2024-03-15T13:06:54.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f270ffff640 1 -- 172.21.15.2:0/3617292153 learned_addr learned my addr 172.21.15.2:0/3617292153 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:54.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f270f7fe640 1 --2- 172.21.15.2:0/3617292153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2710101e60 0x7f271019dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f2714fa4640 1 --2- 172.21.15.2:0/3617292153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2710107560 0x7f27101978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f270ffff640 1 -- 172.21.15.2:0/3617292153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2710107560 msgr2=0x7f27101978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f270ffff640 1 --2- 172.21.15.2:0/3617292153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2710107560 0x7f27101978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f270ffff640 1 -- 172.21.15.2:0/3617292153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2710101e60 msgr2=0x7f271019dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f270ffff640 1 --2- 172.21.15.2:0/3617292153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2710101e60 0x7f271019dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.301+0000 7f270ffff640 1 -- 172.21.15.2:0/3617292153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2704009560 con 0x7f27101014a0 2024-03-15T13:06:54.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.302+0000 7f270ffff640 1 --2- 172.21.15.2:0/3617292153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27101014a0 0x7f271019d710 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f270000e9f0 tx=0x7f270000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.302+0000 7f270d7fa640 1 -- 172.21.15.2:0/3617292153 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700009720 con 0x7f27101014a0 2024-03-15T13:06:54.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.302+0000 7f2716a07640 1 -- 172.21.15.2:0/3617292153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2710198520 con 0x7f27101014a0 2024-03-15T13:06:54.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.302+0000 7f270d7fa640 1 -- 172.21.15.2:0/3617292153 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f270000cd80 con 0x7f27101014a0 2024-03-15T13:06:54.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.302+0000 7f270d7fa640 1 -- 172.21.15.2:0/3617292153 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700010690 con 0x7f27101014a0 2024-03-15T13:06:54.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.302+0000 7f2716a07640 1 -- 172.21.15.2:0/3617292153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27101a47b0 con 0x7f27101014a0 2024-03-15T13:06:54.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f27195e7740 1 Processor -- start 2024-03-15T13:06:54.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f27195e7740 1 -- start start 2024-03-15T13:06:54.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f270d7fa640 1 -- 172.21.15.2:0/3617292153 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f2700004e90 con 0x7f27101014a0 2024-03-15T13:06:54.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f27195e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9a250 0x55a54ca9a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f27195e7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9ab70 0x55a54caa3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f27195e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54caa3750 0x55a54caa5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f27195e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54ca55960 con 0x55a54ca9a250 2024-03-15T13:06:54.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f27195e7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54ca09310 con 0x55a54ca9ab70 2024-03-15T13:06:54.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f27195e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54ca06840 con 0x55a54caa3750 2024-03-15T13:06:54.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f2714fa4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9a250 0x55a54ca9a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f270ffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9ab70 0x55a54caa3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f2714fa4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9a250 0x55a54ca9a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51848/0 (socket says 172.21.15.2:51848) 2024-03-15T13:06:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f2714fa4640 1 -- 172.21.15.2:0/3716751339 learned_addr learned my addr 172.21.15.2:0/3716751339 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f270f7fe640 1 --2- 172.21.15.2:0/3716751339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54caa3750 0x55a54caa5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f270ffff640 1 -- 172.21.15.2:0/3716751339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54caa3750 msgr2=0x55a54caa5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f270ffff640 1 --2- 172.21.15.2:0/3716751339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54caa3750 0x55a54caa5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f270ffff640 1 -- 172.21.15.2:0/3716751339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9a250 msgr2=0x55a54ca9a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f270ffff640 1 --2- 172.21.15.2:0/3716751339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9a250 0x55a54ca9a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f270ffff640 1 -- 172.21.15.2:0/3716751339 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a54c9c4060 con 0x55a54ca9ab70 2024-03-15T13:06:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f2714fa4640 1 --2- 172.21.15.2:0/3716751339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9a250 0x55a54ca9a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.305+0000 7f270d7fa640 1 --2- 172.21.15.2:0/3617292153 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f26f80a1ba0 0x7f26f80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.306+0000 7f2714fa4640 1 --2- 172.21.15.2:0/3617292153 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f26f80a1ba0 0x7f26f80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.306+0000 7f270ffff640 1 --2- 172.21.15.2:0/3716751339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9ab70 0x55a54caa3210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f27000cb2a0 tx=0x7f27000d1e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0768f8cadbe2ce7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.306+0000 7f270d7fa640 1 -- 172.21.15.2:0/3617292153 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f2700014070 con 0x7f27101014a0 2024-03-15T13:06:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.306+0000 7f26f67fc640 1 -- 172.21.15.2:0/3716751339 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700096a20 con 0x55a54ca9ab70 2024-03-15T13:06:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.306+0000 7f26f67fc640 1 -- 172.21.15.2:0/3716751339 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f270009a550 con 0x55a54ca9ab70 2024-03-15T13:06:54.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.306+0000 7f26f67fc640 1 -- 172.21.15.2:0/3716751339 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270009ae80 con 0x55a54ca9ab70 2024-03-15T13:06:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 -- 172.21.15.2:0/3716751339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9ab70 msgr2=0x55a54caa3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f2714fa4640 1 --2- 172.21.15.2:0/3617292153 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f26f80a1ba0 0x7f26f80a4060 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2708011890 tx=0x7f2708009500 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 --2- 172.21.15.2:0/3716751339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9ab70 0x55a54caa3210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f27000cb2a0 tx=0x7f27000d1e30 comp rx=0 tx=0).stop 2024-03-15T13:06:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 -- 172.21.15.2:0/3716751339 shutdown_connections 2024-03-15T13:06:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 --2- 172.21.15.2:0/3716751339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54caa3750 0x55a54caa5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 --2- 172.21.15.2:0/3716751339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9ab70 0x55a54caa3210 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 --2- 172.21.15.2:0/3716751339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9a250 0x55a54ca9a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 -- 172.21.15.2:0/3716751339 >> 172.21.15.2:0/3716751339 conn(0x55a54ca94980 msgr2=0x55a54ca94d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 -- 172.21.15.2:0/3716751339 shutdown_connections 2024-03-15T13:06:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 -- 172.21.15.2:0/3716751339 wait complete. 2024-03-15T13:06:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 -- start start 2024-03-15T13:06:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 Processor -- start 2024-03-15T13:06:54.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54ca9a250 0x55a54caa30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9ab70 0x55a54caa0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54caa3750 0x55a54caa09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54ca02020 con 0x55a54caa3750 2024-03-15T13:06:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54c6d5430 con 0x55a54ca9ab70 2024-03-15T13:06:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.307+0000 7f27195e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54c691e00 con 0x55a54ca9a250 2024-03-15T13:06:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f27195e7740 10 client.0 _ll_register_callbacks cb 0x55a54c9a9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f2714fa4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54caa3750 0x55a54caa09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f270ffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54ca9a250 0x55a54caa30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f270f7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9ab70 0x55a54caa0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f2714fa4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54caa3750 0x55a54caa09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51864/0 (socket says 172.21.15.2:51864) 2024-03-15T13:06:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f270ffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54ca9a250 0x55a54caa30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:52596/0 (socket says 172.21.15.2:52596) 2024-03-15T13:06:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f2714fa4640 1 -- 172.21.15.2:0/2054395173 learned_addr learned my addr 172.21.15.2:0/2054395173 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f2714fa4640 1 -- 172.21.15.2:0/2054395173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54ca9a250 msgr2=0x55a54caa30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f2714fa4640 1 --2- 172.21.15.2:0/2054395173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54ca9a250 0x55a54caa30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f2714fa4640 1 -- 172.21.15.2:0/2054395173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9ab70 msgr2=0x55a54caa0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f2714fa4640 1 --2- 172.21.15.2:0/2054395173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9ab70 0x55a54caa0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f2714fa4640 1 -- 172.21.15.2:0/2054395173 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27101a47b0 con 0x55a54caa3750 2024-03-15T13:06:54.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f2714fa4640 1 --2- 172.21.15.2:0/2054395173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54caa3750 0x55a54caa09f0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f270800fbe0 tx=0x7f2708004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed861226444931 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f26ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:54.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f26ef7fe640 1 -- 172.21.15.2:0/2054395173 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708022070 con 0x55a54caa3750 2024-03-15T13:06:54.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f26ef7fe640 1 -- 172.21.15.2:0/2054395173 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f2708004b20 con 0x55a54caa3750 2024-03-15T13:06:54.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f27195e7740 1 -- 172.21.15.2:0/2054395173 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a54c7017e0 con 0x55a54caa3750 2024-03-15T13:06:54.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f26ef7fe640 1 -- 172.21.15.2:0/2054395173 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2708016070 con 0x55a54caa3750 2024-03-15T13:06:54.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f27195e7740 10 client.18033 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:54.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.308+0000 7f27195e7740 1 -- 172.21.15.2:0/2054395173 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55a54ca006a0 con 0x55a54caa3750 2024-03-15T13:06:54.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.309+0000 7f26ef7fe640 1 -- 172.21.15.2:0/2054395173 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55a54c7017e0 con 0x55a54caa3750 2024-03-15T13:06:54.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.309+0000 7f26ef7fe640 1 -- 172.21.15.2:0/2054395173 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f270801dbe0 con 0x55a54caa3750 2024-03-15T13:06:54.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.309+0000 7f27195e7740 10 client.18033 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:54.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.309+0000 7f27195e7740 10 client.18033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:54.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.309+0000 7f27195e7740 20 client.18033 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:54.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.309+0000 7f27195e7740 1 --2- 172.21.15.2:0/2054395173 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a54cab3940 0x55a54cab5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.309+0000 7f27195e7740 4 client.18033 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:54.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.309+0000 7f27195e7740 1 -- 172.21.15.2:0/2054395173 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a54c73e720 con 0x55a54cab3940 2024-03-15T13:06:54.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.309+0000 7f270ffff640 1 --2- 172.21.15.2:0/2054395173 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a54cab3940 0x55a54cab5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.309+0000 7f270ffff640 1 --2- 172.21.15.2:0/2054395173 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a54cab3940 0x55a54cab5d60 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd86842ede89b64e server_cookie=f8424cc7b0785a9a in_seq=0 out_seq=0 2024-03-15T13:06:54.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.309+0000 7f26ef7fe640 10 client.18033 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:54.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.310+0000 7f26ef7fe640 1 -- 172.21.15.2:0/2054395173 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a54c73e720 con 0x55a54cab3940 2024-03-15T13:06:54.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.310+0000 7f26ef7fe640 10 client.18033 handle_command_reply: tid=0 2024-03-15T13:06:54.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.310+0000 7f27195e7740 1 client.18033 shutdown 2024-03-15T13:06:54.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.311+0000 7f27195e7740 1 -- 172.21.15.2:0/2054395173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54caa3750 msgr2=0x55a54caa09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.311+0000 7f27195e7740 1 --2- 172.21.15.2:0/2054395173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54caa3750 0x55a54caa09f0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f270800fbe0 tx=0x7f2708004ab0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.311+0000 7f27195e7740 1 -- 172.21.15.2:0/2054395173 shutdown_connections 2024-03-15T13:06:54.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.311+0000 7f27195e7740 1 --2- 172.21.15.2:0/2054395173 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a54cab3940 0x55a54cab5d60 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.311+0000 7f27195e7740 1 --2- 172.21.15.2:0/2054395173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54caa3750 0x55a54caa09f0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.311+0000 7f27195e7740 1 --2- 172.21.15.2:0/2054395173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9ab70 0x55a54caa0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.311+0000 7f26ef7fe640 0 client.18033 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:54.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.311+0000 7f27195e7740 1 --2- 172.21.15.2:0/2054395173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54ca9a250 0x55a54caa30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.311+0000 7f27195e7740 1 -- 172.21.15.2:0/2054395173 >> 172.21.15.2:0/2054395173 conn(0x55a54ca94980 msgr2=0x55a54ca966d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.311+0000 7f27195e7740 1 -- 172.21.15.2:0/2054395173 shutdown_connections 2024-03-15T13:06:54.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.311+0000 7f27195e7740 1 -- 172.21.15.2:0/2054395173 wait complete. 2024-03-15T13:06:54.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.311+0000 7f27195e7740 20 client.18033 trim_cache size 0 max 16384 2024-03-15T13:06:54.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f27195e7740 1 Processor -- start 2024-03-15T13:06:54.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f27195e7740 1 -- start start 2024-03-15T13:06:54.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f27195e7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9a250 0x55a54ca93a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f27195e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9ab70 0x55a54caa2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f27195e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54caa3750 0x55a54caa30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f27195e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54c987130 con 0x55a54ca9ab70 2024-03-15T13:06:54.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f27195e7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54ca95bb0 con 0x55a54ca9a250 2024-03-15T13:06:54.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f27195e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54caa0ff0 con 0x55a54caa3750 2024-03-15T13:06:54.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f270f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9ab70 0x55a54caa2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f270ffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9a250 0x55a54ca93a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f2714fa4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54caa3750 0x55a54caa30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f270ffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9a250 0x55a54ca93a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:33284/0 (socket says 172.21.15.2:33284) 2024-03-15T13:06:54.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f270f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9ab70 0x55a54caa2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51880/0 (socket says 172.21.15.2:51880) 2024-03-15T13:06:54.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f270ffff640 1 -- 172.21.15.2:0/2885724233 learned_addr learned my addr 172.21.15.2:0/2885724233 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:54.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f270ffff640 1 -- 172.21.15.2:0/2885724233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54caa3750 msgr2=0x55a54caa30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.323+0000 7f270ffff640 1 --2- 172.21.15.2:0/2885724233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54caa3750 0x55a54caa30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.324+0000 7f270ffff640 1 -- 172.21.15.2:0/2885724233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9ab70 msgr2=0x55a54caa2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.324+0000 7f270ffff640 1 --2- 172.21.15.2:0/2885724233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9ab70 0x55a54caa2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.324+0000 7f270ffff640 1 -- 172.21.15.2:0/2885724233 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a54c99cc20 con 0x55a54ca9a250 2024-03-15T13:06:54.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.324+0000 7f270f7fe640 1 --2- 172.21.15.2:0/2885724233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9ab70 0x55a54caa2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:54.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.324+0000 7f270ffff640 1 --2- 172.21.15.2:0/2885724233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9a250 0x55a54ca93a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2700009380 tx=0x7f2700009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9703c45b1493b9c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.324+0000 7f26f5ffb640 1 -- 172.21.15.2:0/2885724233 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27000d1350 con 0x55a54ca9a250 2024-03-15T13:06:54.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.324+0000 7f26f5ffb640 1 -- 172.21.15.2:0/2885724233 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f270009a2c0 con 0x55a54ca9a250 2024-03-15T13:06:54.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.324+0000 7f26f5ffb640 1 -- 172.21.15.2:0/2885724233 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270009a810 con 0x55a54ca9a250 2024-03-15T13:06:54.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.324+0000 7f27195e7740 1 -- 172.21.15.2:0/2885724233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9a250 msgr2=0x55a54ca93a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.324+0000 7f27195e7740 1 --2- 172.21.15.2:0/2885724233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9a250 0x55a54ca93a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2700009380 tx=0x7f2700009310 comp rx=0 tx=0).stop 2024-03-15T13:06:54.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.324+0000 7f27195e7740 1 -- 172.21.15.2:0/2885724233 shutdown_connections 2024-03-15T13:06:54.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.324+0000 7f27195e7740 1 --2- 172.21.15.2:0/2885724233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54caa3750 0x55a54caa30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.324+0000 7f27195e7740 1 --2- 172.21.15.2:0/2885724233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9ab70 0x55a54caa2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.324+0000 7f27195e7740 1 --2- 172.21.15.2:0/2885724233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54ca9a250 0x55a54ca93a00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.324+0000 7f27195e7740 1 -- 172.21.15.2:0/2885724233 >> 172.21.15.2:0/2885724233 conn(0x55a54ca94980 msgr2=0x55a54ca932f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f27195e7740 1 -- 172.21.15.2:0/2885724233 shutdown_connections 2024-03-15T13:06:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f27195e7740 1 -- 172.21.15.2:0/2885724233 wait complete. 2024-03-15T13:06:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f27195e7740 1 -- start start 2024-03-15T13:06:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f27195e7740 1 Processor -- start 2024-03-15T13:06:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f27195e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9a250 0x55a54cb08500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f27195e7740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54ca9ab70 0x55a54cb08a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f27195e7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54caa3750 0x55a54caa71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f27195e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54caa7780 con 0x55a54ca9a250 2024-03-15T13:06:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f27195e7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54caa78f0 con 0x55a54caa3750 2024-03-15T13:06:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f27195e7740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54caa7ab0 con 0x55a54ca9ab70 2024-03-15T13:06:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f27195e7740 10 client.0 _ll_register_callbacks cb 0x7f2708011e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f270ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9a250 0x55a54cb08500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f270ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9a250 0x55a54cb08500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51882/0 (socket says 172.21.15.2:51882) 2024-03-15T13:06:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f270ffff640 1 -- 172.21.15.2:0/1759989229 learned_addr learned my addr 172.21.15.2:0/1759989229 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f2714fa4640 1 --2- 172.21.15.2:0/1759989229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54caa3750 0x55a54caa71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.325+0000 7f270f7fe640 1 --2- 172.21.15.2:0/1759989229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54ca9ab70 0x55a54cb08a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f270ffff640 1 -- 172.21.15.2:0/1759989229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54ca9ab70 msgr2=0x55a54cb08a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f270ffff640 1 --2- 172.21.15.2:0/1759989229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54ca9ab70 0x55a54cb08a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f270ffff640 1 -- 172.21.15.2:0/1759989229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54caa3750 msgr2=0x55a54caa71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f270ffff640 1 --2- 172.21.15.2:0/1759989229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54caa3750 0x55a54caa71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f270ffff640 1 -- 172.21.15.2:0/1759989229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271010ec20 con 0x55a54ca9a250 2024-03-15T13:06:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f270ffff640 1 --2- 172.21.15.2:0/1759989229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9a250 0x55a54cb08500 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2700098a30 tx=0x7f2700004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8171cc1026537cf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f26ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f26ef7fe640 1 -- 172.21.15.2:0/1759989229 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27000d1350 con 0x55a54ca9a250 2024-03-15T13:06:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f26ef7fe640 1 -- 172.21.15.2:0/1759989229 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f270009ae80 con 0x55a54ca9a250 2024-03-15T13:06:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f27195e7740 1 -- 172.21.15.2:0/1759989229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a54ca900f0 con 0x55a54ca9a250 2024-03-15T13:06:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f26ef7fe640 1 -- 172.21.15.2:0/1759989229 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f270009a7f0 con 0x55a54ca9a250 2024-03-15T13:06:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f27195e7740 10 client.18039 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f27195e7740 1 -- 172.21.15.2:0/1759989229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55a54c9c4060 con 0x55a54ca9a250 2024-03-15T13:06:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f26ef7fe640 1 -- 172.21.15.2:0/1759989229 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55a54ca900f0 con 0x55a54ca9a250 2024-03-15T13:06:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f26ef7fe640 1 -- 172.21.15.2:0/1759989229 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f270001f060 con 0x55a54ca9a250 2024-03-15T13:06:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f27195e7740 10 client.18039 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f27195e7740 10 client.18039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f27195e7740 20 client.18039 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f27195e7740 1 --2- 172.21.15.2:0/1759989229 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a54cb0ff10 0x55a54cb12300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f27195e7740 4 client.18039 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:06:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.326+0000 7f27195e7740 1 -- 172.21.15.2:0/1759989229 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f270009a550 con 0x55a54cb0ff10 2024-03-15T13:06:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f270f7fe640 1 --2- 172.21.15.2:0/1759989229 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a54cb0ff10 0x55a54cb12300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f270f7fe640 1 --2- 172.21.15.2:0/1759989229 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a54cb0ff10 0x55a54cb12300 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=816c7c146cdc6a14 server_cookie=ee49a3cd901f4876 in_seq=0 out_seq=0 2024-03-15T13:06:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f26ef7fe640 10 client.18039 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f26ef7fe640 1 -- 172.21.15.2:0/1759989229 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f270009a550 con 0x55a54cb0ff10 2024-03-15T13:06:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f26ef7fe640 10 client.18039 handle_command_reply: tid=0 2024-03-15T13:06:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f27195e7740 1 client.18039 shutdown 2024-03-15T13:06:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f27195e7740 1 -- 172.21.15.2:0/1759989229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9a250 msgr2=0x55a54cb08500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f27195e7740 1 --2- 172.21.15.2:0/1759989229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9a250 0x55a54cb08500 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2700098a30 tx=0x7f2700004960 comp rx=0 tx=0).stop 2024-03-15T13:06:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f27195e7740 1 -- 172.21.15.2:0/1759989229 shutdown_connections 2024-03-15T13:06:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f27195e7740 1 --2- 172.21.15.2:0/1759989229 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a54cb0ff10 0x55a54cb12300 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f27195e7740 1 --2- 172.21.15.2:0/1759989229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54caa3750 0x55a54caa71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f27195e7740 1 --2- 172.21.15.2:0/1759989229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a54ca9ab70 0x55a54cb08a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f26ef7fe640 0 client.18039 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f27195e7740 1 --2- 172.21.15.2:0/1759989229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a54ca9a250 0x55a54cb08500 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f27195e7740 1 -- 172.21.15.2:0/1759989229 >> 172.21.15.2:0/1759989229 conn(0x55a54ca94980 msgr2=0x55a54ca93b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f27195e7740 1 -- 172.21.15.2:0/1759989229 shutdown_connections 2024-03-15T13:06:54.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f27195e7740 1 -- 172.21.15.2:0/1759989229 wait complete. 2024-03-15T13:06:54.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.327+0000 7f27195e7740 20 client.18039 trim_cache size 0 max 16384 2024-03-15T13:06:54.320 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:06:54.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.328+0000 7f2716a07640 1 -- 172.21.15.2:0/3617292153 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f26f80a1ba0 msgr2=0x7f26f80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.328+0000 7f2716a07640 1 --2- 172.21.15.2:0/3617292153 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f26f80a1ba0 0x7f26f80a4060 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2708011890 tx=0x7f2708009500 comp rx=0 tx=0).stop 2024-03-15T13:06:54.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.328+0000 7f2716a07640 1 -- 172.21.15.2:0/3617292153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27101014a0 msgr2=0x7f271019d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:54.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.328+0000 7f2716a07640 1 --2- 172.21.15.2:0/3617292153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27101014a0 0x7f271019d710 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f270000e9f0 tx=0x7f270000edf0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.328+0000 7f2716a07640 1 -- 172.21.15.2:0/3617292153 shutdown_connections 2024-03-15T13:06:54.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.328+0000 7f2716a07640 1 --2- 172.21.15.2:0/3617292153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2710107560 0x7f27101978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.328+0000 7f2716a07640 1 --2- 172.21.15.2:0/3617292153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2710101e60 0x7f271019dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.328+0000 7f2716a07640 1 --2- 172.21.15.2:0/3617292153 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f26f80a1ba0 0x7f26f80a4060 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.328+0000 7f2716a07640 1 --2- 172.21.15.2:0/3617292153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27101014a0 0x7f271019d710 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:54.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.328+0000 7f2716a07640 1 -- 172.21.15.2:0/3617292153 >> 172.21.15.2:0/3617292153 conn(0x7f27100788f0 msgr2=0x7f2710100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:54.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.328+0000 7f2716a07640 1 -- 172.21.15.2:0/3617292153 shutdown_connections 2024-03-15T13:06:54.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:54.328+0000 7f2716a07640 1 -- 172.21.15.2:0/3617292153 wait complete. 2024-03-15T13:06:54.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:54 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2635350467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:06:54.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:54 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2635350467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:06:54.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:54 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2635350467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:06:55.324 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:06:55.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:06:55.324 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:06:55.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.457+0000 7f41a5091640 1 -- 172.21.15.2:0/1487878716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41a01014a0 msgr2=0x7f41a0101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.457+0000 7f41a5091640 1 --2- 172.21.15.2:0/1487878716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41a01014a0 0x7f41a0101920 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f4194009900 tx=0x7f419402bc30 comp rx=0 tx=0).stop 2024-03-15T13:06:55.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.457+0000 7f41a5091640 1 -- 172.21.15.2:0/1487878716 shutdown_connections 2024-03-15T13:06:55.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.457+0000 7f41a5091640 1 --2- 172.21.15.2:0/1487878716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41a0101e60 0x7f41a010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.457+0000 7f41a5091640 1 --2- 172.21.15.2:0/1487878716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41a01014a0 0x7f41a0101920 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.457+0000 7f41a5091640 1 --2- 172.21.15.2:0/1487878716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41a0107560 0x7f41a0107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.457+0000 7f41a5091640 1 -- 172.21.15.2:0/1487878716 >> 172.21.15.2:0/1487878716 conn(0x7f41a00788f0 msgr2=0x7f41a0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:55.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.457+0000 7f41a5091640 1 -- 172.21.15.2:0/1487878716 shutdown_connections 2024-03-15T13:06:55.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.457+0000 7f41a5091640 1 -- 172.21.15.2:0/1487878716 wait complete. 2024-03-15T13:06:55.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.458+0000 7f41a5091640 1 Processor -- start 2024-03-15T13:06:55.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.458+0000 7f41a5091640 1 -- start start 2024-03-15T13:06:55.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.458+0000 7f41a5091640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41a01014a0 0x7f41a019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.458+0000 7f41a5091640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41a0101e60 0x7f41a019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.458+0000 7f41a5091640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41a0107560 0x7f41a0197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.458+0000 7f41a5091640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a0197ec0 con 0x7f41a0107560 2024-03-15T13:06:55.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.458+0000 7f41a5091640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a0198030 con 0x7f41a0101e60 2024-03-15T13:06:55.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.458+0000 7f41a5091640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a01981f0 con 0x7f41a01014a0 2024-03-15T13:06:55.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.458+0000 7f419f59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41a0107560 0x7f41a0197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.458+0000 7f419f59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41a0107560 0x7f41a0197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51902/0 (socket says 172.21.15.2:51902) 2024-03-15T13:06:55.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.458+0000 7f419f59e640 1 -- 172.21.15.2:0/550255127 learned_addr learned my addr 172.21.15.2:0/550255127 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:55.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.458+0000 7f419ed9d640 1 --2- 172.21.15.2:0/550255127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41a01014a0 0x7f41a019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.458+0000 7f419e59c640 1 --2- 172.21.15.2:0/550255127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41a0101e60 0x7f41a019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.459+0000 7f419f59e640 1 -- 172.21.15.2:0/550255127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41a01014a0 msgr2=0x7f41a019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.459+0000 7f419f59e640 1 --2- 172.21.15.2:0/550255127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41a01014a0 0x7f41a019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.459+0000 7f419f59e640 1 -- 172.21.15.2:0/550255127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41a0101e60 msgr2=0x7f41a019dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.459+0000 7f419f59e640 1 --2- 172.21.15.2:0/550255127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41a0101e60 0x7f41a019dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.459+0000 7f419f59e640 1 -- 172.21.15.2:0/550255127 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4194009560 con 0x7f41a0107560 2024-03-15T13:06:55.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.459+0000 7f419f59e640 1 --2- 172.21.15.2:0/550255127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41a0107560 0x7f41a0197870 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f419800eb10 tx=0x7f419800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:55.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.459+0000 7f4187fff640 1 -- 172.21.15.2:0/550255127 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198004b20 con 0x7f41a0107560 2024-03-15T13:06:55.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.459+0000 7f4187fff640 1 -- 172.21.15.2:0/550255127 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4198004c80 con 0x7f41a0107560 2024-03-15T13:06:55.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.459+0000 7f41a5091640 1 -- 172.21.15.2:0/550255127 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a01984d0 con 0x7f41a0107560 2024-03-15T13:06:55.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.459+0000 7f4187fff640 1 -- 172.21.15.2:0/550255127 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198010690 con 0x7f41a0107560 2024-03-15T13:06:55.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.459+0000 7f41a5091640 1 -- 172.21.15.2:0/550255127 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a01a4760 con 0x7f41a0107560 2024-03-15T13:06:55.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.461+0000 7f4187fff640 1 -- 172.21.15.2:0/550255127 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f4198004df0 con 0x7f41a0107560 2024-03-15T13:06:55.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f41a7c6a740 1 Processor -- start 2024-03-15T13:06:55.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f41a7c6a740 1 -- start start 2024-03-15T13:06:55.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f4187fff640 1 --2- 172.21.15.2:0/550255127 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f41880a1ba0 0x7f41880a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f41a7c6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 0x55e810333640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f41a7c6a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333c10 0x55e81033c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f41a7c6a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e81033cd10 0x55e81033d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f41a7c6a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8102a1f90 con 0x55e81033cd10 2024-03-15T13:06:55.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f41a7c6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81029f4c0 con 0x55e810333260 2024-03-15T13:06:55.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f41a7c6a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81029aca0 con 0x55e810333c10 2024-03-15T13:06:55.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f419e59c640 1 --2- 172.21.15.2:0/550255127 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f41880a1ba0 0x7f41880a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f419ed9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 0x55e810333640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f419f59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e81033cd10 0x55e81033d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f419ed9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 0x55e810333640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:33314/0 (socket says 172.21.15.2:33314) 2024-03-15T13:06:55.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f419ed9d640 1 -- 172.21.15.2:0/3501515679 learned_addr learned my addr 172.21.15.2:0/3501515679 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:55.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f4187fff640 1 -- 172.21.15.2:0/550255127 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f4198014070 con 0x7f41a0107560 2024-03-15T13:06:55.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.462+0000 7f419e59c640 1 --2- 172.21.15.2:0/3501515679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333c10 0x55e81033c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.463+0000 7f419f59e640 1 -- 172.21.15.2:0/3501515679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333c10 msgr2=0x55e81033c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.463+0000 7f419f59e640 1 --2- 172.21.15.2:0/3501515679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333c10 0x55e81033c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.463+0000 7f419f59e640 1 -- 172.21.15.2:0/3501515679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 msgr2=0x55e810333640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.463+0000 7f419f59e640 1 --2- 172.21.15.2:0/3501515679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 0x55e810333640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.463+0000 7f419f59e640 1 -- 172.21.15.2:0/3501515679 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e81025d0b0 con 0x55e81033cd10 2024-03-15T13:06:55.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.463+0000 7f419ed9d640 1 --2- 172.21.15.2:0/3501515679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 0x55e810333640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:55.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.463+0000 7f419e59c640 1 --2- 172.21.15.2:0/550255127 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f41880a1ba0 0x7f41880a4060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4194005ab0 tx=0x7f41940046c0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:55.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.463+0000 7f419f59e640 1 --2- 172.21.15.2:0/3501515679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e81033cd10 0x55e81033d170 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f41980cb630 tx=0x7f419801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7bf66c34ef833b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:55.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.463+0000 7f41857fa640 1 -- 172.21.15.2:0/3501515679 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419809a470 con 0x55e81033cd10 2024-03-15T13:06:55.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.463+0000 7f41857fa640 1 -- 172.21.15.2:0/3501515679 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f41980b3840 con 0x55e81033cd10 2024-03-15T13:06:55.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.463+0000 7f41857fa640 1 -- 172.21.15.2:0/3501515679 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198097180 con 0x55e81033cd10 2024-03-15T13:06:55.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 -- 172.21.15.2:0/3501515679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e81033cd10 msgr2=0x55e81033d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/3501515679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e81033cd10 0x55e81033d170 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f41980cb630 tx=0x7f419801d000 comp rx=0 tx=0).stop 2024-03-15T13:06:55.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 -- 172.21.15.2:0/3501515679 shutdown_connections 2024-03-15T13:06:55.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/3501515679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e81033cd10 0x55e81033d170 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/3501515679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333c10 0x55e81033c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/3501515679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 0x55e810333640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 -- 172.21.15.2:0/3501515679 >> 172.21.15.2:0/3501515679 conn(0x55e81032d840 msgr2=0x55e81032dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:55.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 -- 172.21.15.2:0/3501515679 shutdown_connections 2024-03-15T13:06:55.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 -- 172.21.15.2:0/3501515679 wait complete. 2024-03-15T13:06:55.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 -- start start 2024-03-15T13:06:55.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 Processor -- start 2024-03-15T13:06:55.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 0x55e810340810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333c10 0x55e810340d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e81033cd10 0x55e810341290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81023ff40 con 0x55e81033cd10 2024-03-15T13:06:55.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e80ff6e4b0 con 0x55e810333260 2024-03-15T13:06:55.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e810220670 con 0x55e810333c10 2024-03-15T13:06:55.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f41a7c6a740 10 client.0 _ll_register_callbacks cb 0x55e81028df70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:55.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.464+0000 7f419f59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e81033cd10 0x55e810341290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f419e59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333c10 0x55e810340d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f419f59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e81033cd10 0x55e810341290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51932/0 (socket says 172.21.15.2:51932) 2024-03-15T13:06:55.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f419f59e640 1 -- 172.21.15.2:0/1625458218 learned_addr learned my addr 172.21.15.2:0/1625458218 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:55.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f419ed9d640 1 --2- 172.21.15.2:0/1625458218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 0x55e810340810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f419f59e640 1 -- 172.21.15.2:0/1625458218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333c10 msgr2=0x55e810340d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f419f59e640 1 --2- 172.21.15.2:0/1625458218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333c10 0x55e810340d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f419f59e640 1 -- 172.21.15.2:0/1625458218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 msgr2=0x55e810340810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f419f59e640 1 --2- 172.21.15.2:0/1625458218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 0x55e810340810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f419f59e640 1 -- 172.21.15.2:0/1625458218 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a01a4760 con 0x55e81033cd10 2024-03-15T13:06:55.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f419e59c640 1 --2- 172.21.15.2:0/1625458218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333c10 0x55e810340d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:55.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f419f59e640 1 --2- 172.21.15.2:0/1625458218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e81033cd10 0x55e810341290 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f419809a810 tx=0x7f41980a3340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f592848ef63c461 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:55.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f4182ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:55.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f4182ff5640 1 -- 172.21.15.2:0/1625458218 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198094970 con 0x55e81033cd10 2024-03-15T13:06:55.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f4182ff5640 1 -- 172.21.15.2:0/1625458218 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f419809be40 con 0x55e81033cd10 2024-03-15T13:06:55.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f41a7c6a740 1 -- 172.21.15.2:0/1625458218 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e810274f40 con 0x55e81033cd10 2024-03-15T13:06:55.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f4182ff5640 1 -- 172.21.15.2:0/1625458218 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f419809a470 con 0x55e81033cd10 2024-03-15T13:06:55.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f41a7c6a740 10 client.18051 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:55.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f41a7c6a740 1 -- 172.21.15.2:0/1625458218 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55e810299320 con 0x55e81033cd10 2024-03-15T13:06:55.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.465+0000 7f4182ff5640 1 -- 172.21.15.2:0/1625458218 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e810274f40 con 0x55e81033cd10 2024-03-15T13:06:55.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.466+0000 7f4182ff5640 1 -- 172.21.15.2:0/1625458218 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f419801f060 con 0x55e81033cd10 2024-03-15T13:06:55.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.466+0000 7f41a7c6a740 10 client.18051 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:55.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.466+0000 7f41a7c6a740 10 client.18051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:55.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.466+0000 7f41a7c6a740 20 client.18051 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:55.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.466+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/1625458218 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e810350dc0 0x55e8103531e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.466+0000 7f41a7c6a740 4 client.18051 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:55.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.466+0000 7f41a7c6a740 1 -- 172.21.15.2:0/1625458218 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e810291310 con 0x55e810350dc0 2024-03-15T13:06:55.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.466+0000 7f419ed9d640 1 --2- 172.21.15.2:0/1625458218 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e810350dc0 0x55e8103531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.466+0000 7f419ed9d640 1 --2- 172.21.15.2:0/1625458218 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e810350dc0 0x55e8103531e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=585ba7c1f158254f server_cookie=265fcbc89cf92c1e in_seq=0 out_seq=0 2024-03-15T13:06:55.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.466+0000 7f4182ff5640 10 client.18051 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:55.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.467+0000 7f4182ff5640 1 -- 172.21.15.2:0/1625458218 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e810291310 con 0x55e810350dc0 2024-03-15T13:06:55.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.467+0000 7f4182ff5640 10 client.18051 handle_command_reply: tid=0 2024-03-15T13:06:55.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.467+0000 7f41a7c6a740 1 client.18051 shutdown 2024-03-15T13:06:55.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.467+0000 7f41a7c6a740 1 -- 172.21.15.2:0/1625458218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e81033cd10 msgr2=0x55e810341290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.467+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/1625458218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e81033cd10 0x55e810341290 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f419809a810 tx=0x7f41980a3340 comp rx=0 tx=0).stop 2024-03-15T13:06:55.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.467+0000 7f41a7c6a740 1 -- 172.21.15.2:0/1625458218 shutdown_connections 2024-03-15T13:06:55.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.467+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/1625458218 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e810350dc0 0x55e8103531e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.467+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/1625458218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e81033cd10 0x55e810341290 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.467+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/1625458218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333c10 0x55e810340d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.467+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/1625458218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 0x55e810340810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.467+0000 7f4182ff5640 0 client.18051 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:55.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.467+0000 7f41a7c6a740 1 -- 172.21.15.2:0/1625458218 >> 172.21.15.2:0/1625458218 conn(0x55e81032d840 msgr2=0x55e81032f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:55.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.468+0000 7f41a7c6a740 1 -- 172.21.15.2:0/1625458218 shutdown_connections 2024-03-15T13:06:55.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.468+0000 7f41a7c6a740 1 -- 172.21.15.2:0/1625458218 wait complete. 2024-03-15T13:06:55.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.468+0000 7f41a7c6a740 20 client.18051 trim_cache size 0 max 16384 2024-03-15T13:06:55.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f41a7c6a740 1 Processor -- start 2024-03-15T13:06:55.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f41a7c6a740 1 -- start start 2024-03-15T13:06:55.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f41a7c6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 0x55e8103410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f41a7c6a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e810333c10 0x55e81033e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f41a7c6a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81033cd10 0x55e810342640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f41a7c6a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8103432e0 con 0x55e810333c10 2024-03-15T13:06:55.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f41a7c6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e810353840 con 0x55e810333260 2024-03-15T13:06:55.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f41a7c6a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e810332460 con 0x55e81033cd10 2024-03-15T13:06:55.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f419e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e810333c10 0x55e81033e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f419ed9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 0x55e8103410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f419e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e810333c10 0x55e81033e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51936/0 (socket says 172.21.15.2:51936) 2024-03-15T13:06:55.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f419e59c640 1 -- 172.21.15.2:0/2656903939 learned_addr learned my addr 172.21.15.2:0/2656903939 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f419f59e640 1 --2- 172.21.15.2:0/2656903939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81033cd10 0x55e810342640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f419e59c640 1 -- 172.21.15.2:0/2656903939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81033cd10 msgr2=0x55e810342640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f419e59c640 1 --2- 172.21.15.2:0/2656903939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81033cd10 0x55e810342640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f419e59c640 1 -- 172.21.15.2:0/2656903939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 msgr2=0x55e8103410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f419e59c640 1 --2- 172.21.15.2:0/2656903939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 0x55e8103410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.484+0000 7f419e59c640 1 -- 172.21.15.2:0/2656903939 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e810235c00 con 0x55e810333c10 2024-03-15T13:06:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.485+0000 7f419e59c640 1 --2- 172.21.15.2:0/2656903939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e810333c10 0x55e81033e100 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4194009900 tx=0x7f419404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50491ce69eeda714 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.485+0000 7f4184ff9640 1 -- 172.21.15.2:0/2656903939 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194048070 con 0x55e810333c10 2024-03-15T13:06:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.485+0000 7f4184ff9640 1 -- 172.21.15.2:0/2656903939 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f41940048d0 con 0x55e810333c10 2024-03-15T13:06:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.485+0000 7f4184ff9640 1 -- 172.21.15.2:0/2656903939 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194004f60 con 0x55e810333c10 2024-03-15T13:06:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.485+0000 7f41a7c6a740 1 -- 172.21.15.2:0/2656903939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e810333c10 msgr2=0x55e81033e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.485+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/2656903939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e810333c10 0x55e81033e100 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4194009900 tx=0x7f419404c000 comp rx=0 tx=0).stop 2024-03-15T13:06:55.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.485+0000 7f41a7c6a740 1 -- 172.21.15.2:0/2656903939 shutdown_connections 2024-03-15T13:06:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.485+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/2656903939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81033cd10 0x55e810342640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.485+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/2656903939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e810333c10 0x55e81033e100 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.485+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/2656903939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e810333260 0x55e8103410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.485+0000 7f41a7c6a740 1 -- 172.21.15.2:0/2656903939 >> 172.21.15.2:0/2656903939 conn(0x55e81032d840 msgr2=0x55e810341f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.485+0000 7f41a7c6a740 1 -- 172.21.15.2:0/2656903939 shutdown_connections 2024-03-15T13:06:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.485+0000 7f41a7c6a740 1 -- 172.21.15.2:0/2656903939 wait complete. 2024-03-15T13:06:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f41a7c6a740 1 -- start start 2024-03-15T13:06:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f41a7c6a740 1 Processor -- start 2024-03-15T13:06:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f41a7c6a740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333260 0x55e810345d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f41a7c6a740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e810333c10 0x55e8103462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f41a7c6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e81033cd10 0x55e8103467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f41a7c6a740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81034cb10 con 0x55e810333c10 2024-03-15T13:06:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f41a7c6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81034cc80 con 0x55e81033cd10 2024-03-15T13:06:55.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f41a7c6a740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8103ac680 con 0x55e810333260 2024-03-15T13:06:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f41a7c6a740 10 client.0 _ll_register_callbacks cb 0x7f41980b6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f419e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e810333c10 0x55e8103462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f419e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e810333c10 0x55e8103462a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51944/0 (socket says 172.21.15.2:51944) 2024-03-15T13:06:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f419e59c640 1 -- 172.21.15.2:0/624645868 learned_addr learned my addr 172.21.15.2:0/624645868 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f419ed9d640 1 --2- 172.21.15.2:0/624645868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333260 0x55e810345d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f419f59e640 1 --2- 172.21.15.2:0/624645868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e81033cd10 0x55e8103467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f419e59c640 1 -- 172.21.15.2:0/624645868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333260 msgr2=0x55e810345d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f419e59c640 1 --2- 172.21.15.2:0/624645868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333260 0x55e810345d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f419e59c640 1 -- 172.21.15.2:0/624645868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e81033cd10 msgr2=0x55e8103467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f419e59c640 1 --2- 172.21.15.2:0/624645868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e81033cd10 0x55e8103467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.486+0000 7f419e59c640 1 -- 172.21.15.2:0/624645868 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a010ec20 con 0x55e810333c10 2024-03-15T13:06:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f419e59c640 1 --2- 172.21.15.2:0/624645868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e810333c10 0x55e8103462a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f419402c170 tx=0x7f4194003940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680a23aac33041bf server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f4182ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:55.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f4182ff5640 1 -- 172.21.15.2:0/624645868 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194048040 con 0x55e810333c10 2024-03-15T13:06:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f4182ff5640 1 -- 172.21.15.2:0/624645868 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f4194003c30 con 0x55e810333c10 2024-03-15T13:06:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f41a7c6a740 1 -- 172.21.15.2:0/624645868 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8102c0900 con 0x55e810333c10 2024-03-15T13:06:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f4182ff5640 1 -- 172.21.15.2:0/624645868 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4194004a40 con 0x55e810333c10 2024-03-15T13:06:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f41a7c6a740 10 client.18057 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f41a7c6a740 1 -- 172.21.15.2:0/624645868 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55e81025b910 con 0x55e810333c10 2024-03-15T13:06:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f4182ff5640 1 -- 172.21.15.2:0/624645868 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e8102c0900 con 0x55e810333c10 2024-03-15T13:06:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f4182ff5640 1 -- 172.21.15.2:0/624645868 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f419403c040 con 0x55e810333c10 2024-03-15T13:06:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f41a7c6a740 10 client.18057 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f41a7c6a740 10 client.18057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f41a7c6a740 20 client.18057 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/624645868 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e8103adaa0 0x55e8103afec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f41a7c6a740 4 client.18057 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:06:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.487+0000 7f41a7c6a740 1 -- 172.21.15.2:0/624645868 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e80ff9a860 con 0x55e8103adaa0 2024-03-15T13:06:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.488+0000 7f419ed9d640 1 --2- 172.21.15.2:0/624645868 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e8103adaa0 0x55e8103afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.488+0000 7f4182ff5640 10 client.18057 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:55.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.488+0000 7f419ed9d640 1 --2- 172.21.15.2:0/624645868 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e8103adaa0 0x55e8103afec0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=278bad82e82adcbb server_cookie=3e28896a3609fb58 in_seq=0 out_seq=0 2024-03-15T13:06:55.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.489+0000 7f4182ff5640 1 -- 172.21.15.2:0/624645868 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e80ff9a860 con 0x55e8103adaa0 2024-03-15T13:06:55.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.489+0000 7f4182ff5640 10 client.18057 handle_command_reply: tid=0 2024-03-15T13:06:55.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.489+0000 7f41a7c6a740 1 client.18057 shutdown 2024-03-15T13:06:55.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.489+0000 7f41a7c6a740 1 -- 172.21.15.2:0/624645868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e810333c10 msgr2=0x55e8103462a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.489+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/624645868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e810333c10 0x55e8103462a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f419402c170 tx=0x7f4194003940 comp rx=0 tx=0).stop 2024-03-15T13:06:55.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.489+0000 7f41a7c6a740 1 -- 172.21.15.2:0/624645868 shutdown_connections 2024-03-15T13:06:55.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.489+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/624645868 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55e8103adaa0 0x55e8103afec0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.489+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/624645868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e81033cd10 0x55e8103467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.489+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/624645868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e810333c10 0x55e8103462a0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.489+0000 7f41a7c6a740 1 --2- 172.21.15.2:0/624645868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e810333260 0x55e810345d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.489+0000 7f41a7c6a740 1 -- 172.21.15.2:0/624645868 >> 172.21.15.2:0/624645868 conn(0x55e81032d840 msgr2=0x55e81032c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:55.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.489+0000 7f4182ff5640 0 client.18057 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:55.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.489+0000 7f41a7c6a740 1 -- 172.21.15.2:0/624645868 shutdown_connections 2024-03-15T13:06:55.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.489+0000 7f41a7c6a740 1 -- 172.21.15.2:0/624645868 wait complete. 2024-03-15T13:06:55.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.489+0000 7f41a7c6a740 20 client.18057 trim_cache size 0 max 16384 2024-03-15T13:06:55.480 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:06:55.480 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:06:55.480 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "b025d0ef-24a5-4a81-a6a1-cf34ceb83c92", 2024-03-15T13:06:55.480 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:06:55.480 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:06:55.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.490+0000 7f41a5091640 1 -- 172.21.15.2:0/550255127 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f41880a1ba0 msgr2=0x7f41880a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.490+0000 7f41a5091640 1 --2- 172.21.15.2:0/550255127 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f41880a1ba0 0x7f41880a4060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4194005ab0 tx=0x7f41940046c0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.490+0000 7f41a5091640 1 -- 172.21.15.2:0/550255127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41a0107560 msgr2=0x7f41a0197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.490+0000 7f41a5091640 1 --2- 172.21.15.2:0/550255127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41a0107560 0x7f41a0197870 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f419800eb10 tx=0x7f419800c2d0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.490+0000 7f41a5091640 1 -- 172.21.15.2:0/550255127 shutdown_connections 2024-03-15T13:06:55.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.490+0000 7f41a5091640 1 --2- 172.21.15.2:0/550255127 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f41880a1ba0 0x7f41880a4060 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.490+0000 7f41a5091640 1 --2- 172.21.15.2:0/550255127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41a0107560 0x7f41a0197870 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.490+0000 7f41a5091640 1 --2- 172.21.15.2:0/550255127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41a0101e60 0x7f41a019dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.490+0000 7f41a5091640 1 --2- 172.21.15.2:0/550255127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41a01014a0 0x7f41a019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.490+0000 7f41a5091640 1 -- 172.21.15.2:0/550255127 >> 172.21.15.2:0/550255127 conn(0x7f41a00788f0 msgr2=0x7f41a0100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:55.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.490+0000 7f41a5091640 1 -- 172.21.15.2:0/550255127 shutdown_connections 2024-03-15T13:06:55.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.490+0000 7f41a5091640 1 -- 172.21.15.2:0/550255127 wait complete. 2024-03-15T13:06:55.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:06:55.486 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:06:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.606+0000 7f5189a0f640 1 -- 172.21.15.2:0/3179316564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5184101e20 msgr2=0x7f518410e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.606+0000 7f5189a0f640 1 --2- 172.21.15.2:0/3179316564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5184101e20 0x7f518410e330 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f517c009cd0 tx=0x7f517c02c0f0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.606+0000 7f5189a0f640 1 -- 172.21.15.2:0/3179316564 shutdown_connections 2024-03-15T13:06:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.606+0000 7f5189a0f640 1 --2- 172.21.15.2:0/3179316564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5184101e20 0x7f518410e330 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.606+0000 7f5189a0f640 1 --2- 172.21.15.2:0/3179316564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5184101460 0x7f51841018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.606+0000 7f5189a0f640 1 --2- 172.21.15.2:0/3179316564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5184107500 0x7f51841078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.606+0000 7f5189a0f640 1 -- 172.21.15.2:0/3179316564 >> 172.21.15.2:0/3179316564 conn(0x7f51840fcf90 msgr2=0x7f51840ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:55.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.606+0000 7f5189a0f640 1 -- 172.21.15.2:0/3179316564 shutdown_connections 2024-03-15T13:06:55.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.606+0000 7f5189a0f640 1 -- 172.21.15.2:0/3179316564 wait complete. 2024-03-15T13:06:55.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.606+0000 7f5189a0f640 1 Processor -- start 2024-03-15T13:06:55.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.606+0000 7f5189a0f640 1 -- start start 2024-03-15T13:06:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f5189a0f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5184101460 0x7f518419d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f5189a0f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5184101e20 0x7f518419dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f5189a0f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5184107500 0x7f5184197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f5189a0f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5184197f50 con 0x7f5184101e20 2024-03-15T13:06:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f5189a0f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51841980c0 con 0x7f5184107500 2024-03-15T13:06:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f5189a0f640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5184198280 con 0x7f5184101460 2024-03-15T13:06:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f517a7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5184101e20 0x7f518419dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f517a7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5184101e20 0x7f518419dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51964/0 (socket says 172.21.15.2:51964) 2024-03-15T13:06:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f517a7fc640 1 -- 172.21.15.2:0/3426300771 learned_addr learned my addr 172.21.15.2:0/3426300771 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f5182ffd640 1 --2- 172.21.15.2:0/3426300771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5184101460 0x7f518419d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f51837fe640 1 --2- 172.21.15.2:0/3426300771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5184107500 0x7f5184197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f517a7fc640 1 -- 172.21.15.2:0/3426300771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5184101460 msgr2=0x7f518419d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f517a7fc640 1 --2- 172.21.15.2:0/3426300771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5184101460 0x7f518419d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f517a7fc640 1 -- 172.21.15.2:0/3426300771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5184107500 msgr2=0x7f5184197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f517a7fc640 1 --2- 172.21.15.2:0/3426300771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5184107500 0x7f5184197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f517a7fc640 1 -- 172.21.15.2:0/3426300771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c009900 con 0x7f5184101e20 2024-03-15T13:06:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.607+0000 7f517a7fc640 1 --2- 172.21.15.2:0/3426300771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5184101e20 0x7f518419dc90 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f516c00e9f0 tx=0x7f516c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.608+0000 7f5188a0d640 1 -- 172.21.15.2:0/3426300771 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c009700 con 0x7f5184101e20 2024-03-15T13:06:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.608+0000 7f5188a0d640 1 -- 172.21.15.2:0/3426300771 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f516c00cd80 con 0x7f5184101e20 2024-03-15T13:06:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.608+0000 7f5189a0f640 1 -- 172.21.15.2:0/3426300771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5184198560 con 0x7f5184101e20 2024-03-15T13:06:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.608+0000 7f5188a0d640 1 -- 172.21.15.2:0/3426300771 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c018920 con 0x7f5184101e20 2024-03-15T13:06:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.608+0000 7f5189a0f640 1 -- 172.21.15.2:0/3426300771 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51841a47f0 con 0x7f5184101e20 2024-03-15T13:06:55.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.610+0000 7f5188a0d640 1 -- 172.21.15.2:0/3426300771 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f516c018a80 con 0x7f5184101e20 2024-03-15T13:06:55.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.610+0000 7f518c5e0740 1 Processor -- start 2024-03-15T13:06:55.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.610+0000 7f518c5e0740 1 -- start start 2024-03-15T13:06:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.610+0000 7f518c5e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd258332b0 0x55cd25833690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.610+0000 7f518c5e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 0x55cd2583c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.610+0000 7f518c5e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2583c7b0 0x55cd2583eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.610+0000 7f518c5e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd257a2280 con 0x55cd2583c7b0 2024-03-15T13:06:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.610+0000 7f518c5e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd2579f7b0 con 0x55cd258332b0 2024-03-15T13:06:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f518c5e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd2579af90 con 0x55cd25833bd0 2024-03-15T13:06:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f5188a0d640 1 --2- 172.21.15.2:0/3426300771 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f51700a1ba0 0x7f51700a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f517a7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2583c7b0 0x55cd2583eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f5182ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 0x55cd2583c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f51837fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd258332b0 0x55cd25833690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f51837fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd258332b0 0x55cd25833690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:33376/0 (socket says 172.21.15.2:33376) 2024-03-15T13:06:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f5182ffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 0x55cd2583c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:52702/0 (socket says 172.21.15.2:52702) 2024-03-15T13:06:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f51837fe640 1 -- 172.21.15.2:0/836557970 learned_addr learned my addr 172.21.15.2:0/836557970 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f51837fe640 1 --2- 172.21.15.2:0/3426300771 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f51700a1ba0 0x7f51700a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f5182ffd640 1 -- 172.21.15.2:0/836557970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd258332b0 msgr2=0x55cd25833690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f5182ffd640 1 --2- 172.21.15.2:0/836557970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd258332b0 0x55cd25833690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f5188a0d640 1 -- 172.21.15.2:0/3426300771 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f516c014070 con 0x7f5184101e20 2024-03-15T13:06:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f5182ffd640 1 -- 172.21.15.2:0/836557970 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2583c7b0 msgr2=0x55cd2583eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f5182ffd640 1 --2- 172.21.15.2:0/836557970 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2583c7b0 0x55cd2583eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f5182ffd640 1 -- 172.21.15.2:0/836557970 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd25759570 con 0x55cd25833bd0 2024-03-15T13:06:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f51837fe640 1 --2- 172.21.15.2:0/3426300771 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f51700a1ba0 0x7f51700a4060 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f517c009430 tx=0x7f517c005c50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f5182ffd640 1 --2- 172.21.15.2:0/836557970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 0x55cd2583c270 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5174013ba0 tx=0x7f5174009b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17630ec3f330ea4c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f5179ffb640 1 -- 172.21.15.2:0/836557970 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174018070 con 0x55cd25833bd0 2024-03-15T13:06:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f5179ffb640 1 -- 172.21.15.2:0/836557970 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5174009e20 con 0x55cd25833bd0 2024-03-15T13:06:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.611+0000 7f5179ffb640 1 -- 172.21.15.2:0/836557970 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174003a10 con 0x55cd25833bd0 2024-03-15T13:06:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.612+0000 7f518c5e0740 1 -- 172.21.15.2:0/836557970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 msgr2=0x55cd2583c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.612+0000 7f518c5e0740 1 --2- 172.21.15.2:0/836557970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 0x55cd2583c270 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5174013ba0 tx=0x7f5174009b70 comp rx=0 tx=0).stop 2024-03-15T13:06:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.612+0000 7f518c5e0740 1 -- 172.21.15.2:0/836557970 shutdown_connections 2024-03-15T13:06:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.612+0000 7f518c5e0740 1 --2- 172.21.15.2:0/836557970 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2583c7b0 0x55cd2583eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.612+0000 7f518c5e0740 1 --2- 172.21.15.2:0/836557970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 0x55cd2583c270 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.612+0000 7f518c5e0740 1 --2- 172.21.15.2:0/836557970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd258332b0 0x55cd25833690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.612+0000 7f518c5e0740 1 -- 172.21.15.2:0/836557970 >> 172.21.15.2:0/836557970 conn(0x55cd2582d990 msgr2=0x55cd2582dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.612+0000 7f518c5e0740 1 -- 172.21.15.2:0/836557970 shutdown_connections 2024-03-15T13:06:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.612+0000 7f518c5e0740 1 -- 172.21.15.2:0/836557970 wait complete. 2024-03-15T13:06:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.612+0000 7f518c5e0740 1 -- start start 2024-03-15T13:06:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f518c5e0740 1 Processor -- start 2024-03-15T13:06:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f518c5e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd258332b0 0x55cd2583c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f518c5e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 0x55cd25839530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f518c5e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd2583c7b0 0x55cd25839b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f518c5e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd2546e430 con 0x55cd258332b0 2024-03-15T13:06:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f518c5e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd2542ae00 con 0x55cd2583c7b0 2024-03-15T13:06:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f518c5e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd2571ffd0 con 0x55cd25833bd0 2024-03-15T13:06:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f518c5e0740 10 client.0 _ll_register_callbacks cb 0x55cd25775fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f5182ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd258332b0 0x55cd2583c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f51837fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd2583c7b0 0x55cd25839b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f5182ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd258332b0 0x55cd2583c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51980/0 (socket says 172.21.15.2:51980) 2024-03-15T13:06:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f5182ffd640 1 -- 172.21.15.2:0/1409232545 learned_addr learned my addr 172.21.15.2:0/1409232545 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f517a7fc640 1 --2- 172.21.15.2:0/1409232545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 0x55cd25839530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f51837fe640 1 -- 172.21.15.2:0/1409232545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 msgr2=0x55cd25839530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f51837fe640 1 --2- 172.21.15.2:0/1409232545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 0x55cd25839530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f51837fe640 1 -- 172.21.15.2:0/1409232545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd258332b0 msgr2=0x55cd2583c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f51837fe640 1 --2- 172.21.15.2:0/1409232545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd258332b0 0x55cd2583c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f51837fe640 1 -- 172.21.15.2:0/1409232545 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5174013880 con 0x55cd2583c7b0 2024-03-15T13:06:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f5182ffd640 1 --2- 172.21.15.2:0/1409232545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd258332b0 0x55cd2583c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.613+0000 7f517a7fc640 1 --2- 172.21.15.2:0/1409232545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 0x55cd25839530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f51837fe640 1 --2- 172.21.15.2:0/1409232545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd2583c7b0 0x55cd25839b40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f517c02c700 tx=0x7f517c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9532e3082ae51ce5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f5162ffd640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:06:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f5162ffd640 1 -- 172.21.15.2:0/1409232545 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c003b70 con 0x55cd2583c7b0 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f518c5e0740 1 -- 172.21.15.2:0/1409232545 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd2549a7e0 con 0x55cd2583c7b0 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f5162ffd640 1 -- 172.21.15.2:0/1409232545 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f517c003cd0 con 0x55cd2583c7b0 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f5162ffd640 1 -- 172.21.15.2:0/1409232545 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f517c003e90 con 0x55cd2583c7b0 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f518c5e0740 10 client.27932 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f518c5e0740 1 -- 172.21.15.2:0/1409232545 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55cd25799610 con 0x55cd2583c7b0 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f5162ffd640 1 -- 172.21.15.2:0/1409232545 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55cd2549a7e0 con 0x55cd2583c7b0 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f5162ffd640 1 -- 172.21.15.2:0/1409232545 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f517c04c070 con 0x55cd2583c7b0 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f518c5e0740 10 client.27932 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f518c5e0740 10 client.27932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f518c5e0740 20 client.27932 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f518c5e0740 1 --2- 172.21.15.2:0/1409232545 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cd2584c960 0x55cd2584ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f518c5e0740 4 client.27932 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.614+0000 7f518c5e0740 1 -- 172.21.15.2:0/1409232545 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd254d7720 con 0x55cd2584c960 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.615+0000 7f5182ffd640 1 --2- 172.21.15.2:0/1409232545 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cd2584c960 0x55cd2584ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.615+0000 7f5182ffd640 1 --2- 172.21.15.2:0/1409232545 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cd2584c960 0x55cd2584ed50 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb0ca5b38a8117a server_cookie=6780cf11d53381f8 in_seq=0 out_seq=0 2024-03-15T13:06:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.615+0000 7f5162ffd640 10 client.27932 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.616+0000 7f5162ffd640 1 -- 172.21.15.2:0/1409232545 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cd254d7720 con 0x55cd2584c960 2024-03-15T13:06:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.616+0000 7f5162ffd640 10 client.27932 handle_command_reply: tid=0 2024-03-15T13:06:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.616+0000 7f518c5e0740 1 client.27932 shutdown 2024-03-15T13:06:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.616+0000 7f518c5e0740 1 -- 172.21.15.2:0/1409232545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd2583c7b0 msgr2=0x55cd25839b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.616+0000 7f518c5e0740 1 --2- 172.21.15.2:0/1409232545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd2583c7b0 0x55cd25839b40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f517c02c700 tx=0x7f517c0095c0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.616+0000 7f518c5e0740 1 -- 172.21.15.2:0/1409232545 shutdown_connections 2024-03-15T13:06:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.616+0000 7f518c5e0740 1 --2- 172.21.15.2:0/1409232545 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cd2584c960 0x55cd2584ed50 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.616+0000 7f518c5e0740 1 --2- 172.21.15.2:0/1409232545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd2583c7b0 0x55cd25839b40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.616+0000 7f518c5e0740 1 --2- 172.21.15.2:0/1409232545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 0x55cd25839530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.616+0000 7f5162ffd640 0 client.27932 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.616+0000 7f518c5e0740 1 --2- 172.21.15.2:0/1409232545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd258332b0 0x55cd2583c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.616+0000 7f518c5e0740 1 -- 172.21.15.2:0/1409232545 >> 172.21.15.2:0/1409232545 conn(0x55cd2582d990 msgr2=0x55cd2582f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.616+0000 7f518c5e0740 1 -- 172.21.15.2:0/1409232545 shutdown_connections 2024-03-15T13:06:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.616+0000 7f518c5e0740 1 -- 172.21.15.2:0/1409232545 wait complete. 2024-03-15T13:06:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.616+0000 7f518c5e0740 20 client.27932 trim_cache size 0 max 16384 2024-03-15T13:06:55.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.632+0000 7f518c5e0740 1 Processor -- start 2024-03-15T13:06:55.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.632+0000 7f518c5e0740 1 -- start start 2024-03-15T13:06:55.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f518c5e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd258332b0 0x55cd25839a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f518c5e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 0x55cd2583bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f518c5e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd2583c7b0 0x55cd2583c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f518c5e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd2582ea60 con 0x55cd258332b0 2024-03-15T13:06:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f518c5e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd2582e7c0 con 0x55cd2583c7b0 2024-03-15T13:06:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f518c5e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd2584f3b0 con 0x55cd25833bd0 2024-03-15T13:06:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f5182ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd258332b0 0x55cd25839a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f5182ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd258332b0 0x55cd25839a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51984/0 (socket says 172.21.15.2:51984) 2024-03-15T13:06:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f5182ffd640 1 -- 172.21.15.2:0/981070045 learned_addr learned my addr 172.21.15.2:0/981070045 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f517a7fc640 1 --2- 172.21.15.2:0/981070045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 0x55cd2583bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f51837fe640 1 --2- 172.21.15.2:0/981070045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd2583c7b0 0x55cd2583c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f5182ffd640 1 -- 172.21.15.2:0/981070045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 msgr2=0x55cd2583bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f5182ffd640 1 --2- 172.21.15.2:0/981070045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 0x55cd2583bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f5182ffd640 1 -- 172.21.15.2:0/981070045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd2583c7b0 msgr2=0x55cd2583c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f5182ffd640 1 --2- 172.21.15.2:0/981070045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd2583c7b0 0x55cd2583c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f5182ffd640 1 -- 172.21.15.2:0/981070045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd25735ac0 con 0x55cd258332b0 2024-03-15T13:06:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f51837fe640 1 --2- 172.21.15.2:0/981070045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd2583c7b0 0x55cd2583c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:06:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f5182ffd640 1 --2- 172.21.15.2:0/981070045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd258332b0 0x55cd25839a80 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5174000c00 tx=0x7f5174013b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e32874e95ff202 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f51797fa640 1 -- 172.21.15.2:0/981070045 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174018040 con 0x55cd258332b0 2024-03-15T13:06:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f51797fa640 1 -- 172.21.15.2:0/981070045 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5174019040 con 0x55cd258332b0 2024-03-15T13:06:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.633+0000 7f51797fa640 1 -- 172.21.15.2:0/981070045 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174003720 con 0x55cd258332b0 2024-03-15T13:06:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.634+0000 7f518c5e0740 1 -- 172.21.15.2:0/981070045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd258332b0 msgr2=0x55cd25839a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.634+0000 7f518c5e0740 1 --2- 172.21.15.2:0/981070045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd258332b0 0x55cd25839a80 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5174000c00 tx=0x7f5174013b50 comp rx=0 tx=0).stop 2024-03-15T13:06:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.634+0000 7f518c5e0740 1 -- 172.21.15.2:0/981070045 shutdown_connections 2024-03-15T13:06:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.634+0000 7f518c5e0740 1 --2- 172.21.15.2:0/981070045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd2583c7b0 0x55cd2583c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.634+0000 7f518c5e0740 1 --2- 172.21.15.2:0/981070045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd25833bd0 0x55cd2583bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.634+0000 7f518c5e0740 1 --2- 172.21.15.2:0/981070045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd258332b0 0x55cd25839a80 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.634+0000 7f518c5e0740 1 -- 172.21.15.2:0/981070045 >> 172.21.15.2:0/981070045 conn(0x55cd2582d990 msgr2=0x55cd2582ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.634+0000 7f518c5e0740 1 -- 172.21.15.2:0/981070045 shutdown_connections 2024-03-15T13:06:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.634+0000 7f518c5e0740 1 -- 172.21.15.2:0/981070045 wait complete. 2024-03-15T13:06:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.634+0000 7f518c5e0740 1 -- start start 2024-03-15T13:06:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f518c5e0740 1 Processor -- start 2024-03-15T13:06:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f518c5e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd258332b0 0x55cd2583d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f518c5e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd25833bd0 0x55cd2583d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f518c5e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2583c7b0 0x55cd25843670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f518c5e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd2583de30 con 0x55cd2583c7b0 2024-03-15T13:06:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f518c5e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd25843bb0 con 0x55cd25833bd0 2024-03-15T13:06:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f518c5e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd25843d70 con 0x55cd258332b0 2024-03-15T13:06:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f518c5e0740 10 client.0 _ll_register_callbacks cb 0x7f5174005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:06:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f51837fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2583c7b0 0x55cd25843670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f51837fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2583c7b0 0x55cd25843670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51994/0 (socket says 172.21.15.2:51994) 2024-03-15T13:06:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f51837fe640 1 -- 172.21.15.2:0/389349065 learned_addr learned my addr 172.21.15.2:0/389349065 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:06:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f5182ffd640 1 --2- 172.21.15.2:0/389349065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd258332b0 0x55cd2583d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f517a7fc640 1 --2- 172.21.15.2:0/389349065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd25833bd0 0x55cd2583d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f51837fe640 1 -- 172.21.15.2:0/389349065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd258332b0 msgr2=0x55cd2583d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f51837fe640 1 --2- 172.21.15.2:0/389349065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd258332b0 0x55cd2583d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f51837fe640 1 -- 172.21.15.2:0/389349065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd25833bd0 msgr2=0x55cd2583d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:06:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f51837fe640 1 --2- 172.21.15.2:0/389349065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd25833bd0 0x55cd2583d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f51837fe640 1 -- 172.21.15.2:0/389349065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd25799610 con 0x55cd2583c7b0 2024-03-15T13:06:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f51837fe640 1 --2- 172.21.15.2:0/389349065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2583c7b0 0x55cd25843670 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f517c034790 tx=0x7f517c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cba7459f770719b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:06:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f5162ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:06:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f5162ffd640 1 -- 172.21.15.2:0/389349065 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c04fdc0 con 0x55cd2583c7b0 2024-03-15T13:06:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f5162ffd640 1 -- 172.21.15.2:0/389349065 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f517c003970 con 0x55cd2583c7b0 2024-03-15T13:06:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.635+0000 7f518c5e0740 1 -- 172.21.15.2:0/389349065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5174009e20 con 0x55cd2583c7b0 2024-03-15T13:06:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.636+0000 7f5162ffd640 1 -- 172.21.15.2:0/389349065 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f517c04f730 con 0x55cd2583c7b0 2024-03-15T13:06:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.636+0000 7f518c5e0740 10 client.18075 fetch_fsmap learned FSMap version 50 2024-03-15T13:06:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.636+0000 7f518c5e0740 1 -- 172.21.15.2:0/389349065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55cd25759570 con 0x55cd2583c7b0 2024-03-15T13:06:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.636+0000 7f5162ffd640 1 -- 172.21.15.2:0/389349065 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5174009e20 con 0x55cd2583c7b0 2024-03-15T13:06:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.636+0000 7f5162ffd640 1 -- 172.21.15.2:0/389349065 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f517c057040 con 0x55cd2583c7b0 2024-03-15T13:06:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.636+0000 7f518c5e0740 10 client.18075 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:06:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.636+0000 7f518c5e0740 10 client.18075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:06:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.636+0000 7f518c5e0740 20 client.18075 populate_metadata read hostname 'smithi002' 2024-03-15T13:06:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.636+0000 7f518c5e0740 1 --2- 172.21.15.2:0/389349065 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cd258a8a30 0x55cd258aae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:06:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.636+0000 7f518c5e0740 4 client.18075 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:06:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.636+0000 7f518c5e0740 1 -- 172.21.15.2:0/389349065 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cd25526890 con 0x55cd258a8a30 2024-03-15T13:06:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.636+0000 7f5182ffd640 1 --2- 172.21.15.2:0/389349065 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cd258a8a30 0x55cd258aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:06:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f5162ffd640 10 client.18075 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f5182ffd640 1 --2- 172.21.15.2:0/389349065 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cd258a8a30 0x55cd258aae50 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7032dcae75ddcc7 server_cookie=1cd2192f579ef572 in_seq=0 out_seq=0 2024-03-15T13:06:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f5162ffd640 1 -- 172.21.15.2:0/389349065 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cd25526890 con 0x55cd258a8a30 2024-03-15T13:06:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f5162ffd640 10 client.18075 handle_command_reply: tid=0 2024-03-15T13:06:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f518c5e0740 1 client.18075 shutdown 2024-03-15T13:06:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f518c5e0740 1 -- 172.21.15.2:0/389349065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2583c7b0 msgr2=0x55cd25843670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f518c5e0740 1 --2- 172.21.15.2:0/389349065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2583c7b0 0x55cd25843670 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f517c034790 tx=0x7f517c009800 comp rx=0 tx=0).stop 2024-03-15T13:06:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f518c5e0740 1 -- 172.21.15.2:0/389349065 shutdown_connections 2024-03-15T13:06:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f518c5e0740 1 --2- 172.21.15.2:0/389349065 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55cd258a8a30 0x55cd258aae50 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f518c5e0740 1 --2- 172.21.15.2:0/389349065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cd2583c7b0 0x55cd25843670 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f518c5e0740 1 --2- 172.21.15.2:0/389349065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd25833bd0 0x55cd2583d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f5162ffd640 0 client.18075 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:06:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f518c5e0740 1 --2- 172.21.15.2:0/389349065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd258332b0 0x55cd2583d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f518c5e0740 1 -- 172.21.15.2:0/389349065 >> 172.21.15.2:0/389349065 conn(0x55cd2582d990 msgr2=0x55cd2582ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f518c5e0740 1 -- 172.21.15.2:0/389349065 shutdown_connections 2024-03-15T13:06:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f518c5e0740 1 -- 172.21.15.2:0/389349065 wait complete. 2024-03-15T13:06:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.637+0000 7f518c5e0740 20 client.18075 trim_cache size 0 max 16384 2024-03-15T13:06:55.629 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:06:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T13:06:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:06:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "b025d0ef-24a5-4a81-a6a1-cf34ceb83c92": { 2024-03-15T13:06:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:06:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "b025d0ef-24a5-4a81-a6a1-cf34ceb83c92", 2024-03-15T13:06:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:06:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:06:55.630 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:06:55.630 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:06:55.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.638+0000 7f5189a0f640 1 -- 172.21.15.2:0/3426300771 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f51700a1ba0 msgr2=0x7f51700a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.638+0000 7f5189a0f640 1 --2- 172.21.15.2:0/3426300771 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f51700a1ba0 0x7f51700a4060 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f517c009430 tx=0x7f517c005c50 comp rx=0 tx=0).stop 2024-03-15T13:06:55.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.638+0000 7f5189a0f640 1 -- 172.21.15.2:0/3426300771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5184101e20 msgr2=0x7f518419dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:06:55.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.638+0000 7f5189a0f640 1 --2- 172.21.15.2:0/3426300771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5184101e20 0x7f518419dc90 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f516c00e9f0 tx=0x7f516c00eeb0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.638+0000 7f5189a0f640 1 -- 172.21.15.2:0/3426300771 shutdown_connections 2024-03-15T13:06:55.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.638+0000 7f5189a0f640 1 --2- 172.21.15.2:0/3426300771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5184107500 0x7f5184197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.638+0000 7f5189a0f640 1 --2- 172.21.15.2:0/3426300771 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f51700a1ba0 0x7f51700a4060 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.638+0000 7f5189a0f640 1 --2- 172.21.15.2:0/3426300771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5184101e20 0x7f518419dc90 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.638+0000 7f5189a0f640 1 --2- 172.21.15.2:0/3426300771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5184101460 0x7f518419d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:06:55.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.638+0000 7f5189a0f640 1 -- 172.21.15.2:0/3426300771 >> 172.21.15.2:0/3426300771 conn(0x7f51840fcf90 msgr2=0x7f51840fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:06:55.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.638+0000 7f5189a0f640 1 -- 172.21.15.2:0/3426300771 shutdown_connections 2024-03-15T13:06:55.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:06:55.638+0000 7f5189a0f640 1 -- 172.21.15.2:0/3426300771 wait complete. 2024-03-15T13:06:55.632 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:55 smithi002 ceph-mon[32060]: pgmap v245: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:55.633 INFO:tasks.cephfs.filesystem:scrub status for tag:b025d0ef-24a5-4a81-a6a1-cf34ceb83c92 - {'path': '/', 'tag': 'b025d0ef-24a5-4a81-a6a1-cf34ceb83c92', 'options': 'recursive,force'} 2024-03-15T13:06:55.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:55 smithi184 ceph-mon[35993]: pgmap v245: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:55.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:55 smithi149 ceph-mon[35927]: pgmap v245: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:56.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:56 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:06:56.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:56 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:06:56.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:56 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:06:56.808 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:56 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:06:56.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:56 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:06:56.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:56 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:06:56.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:56 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:06:56.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:56 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:06:57.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:56 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:06:57.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:56 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:06:57.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:56 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:06:57.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:56 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:06:57.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:57 smithi184 ceph-mon[35993]: pgmap v246: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:06:57.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:57 smithi149 ceph-mon[35927]: pgmap v246: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:06:58.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:57 smithi002 ceph-mon[32060]: pgmap v246: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:06:59.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:06:59 smithi184 ceph-mon[35993]: pgmap v247: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:06:59.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:06:59 smithi149 ceph-mon[35927]: pgmap v247: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:00.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:06:59 smithi002 ceph-mon[32060]: pgmap v247: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:00.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:00 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:07:01.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:00 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:07:01.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:00 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:07:01.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:01 smithi149 ceph-mon[35927]: pgmap v248: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:07:02.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:01 smithi002 ceph-mon[32060]: pgmap v248: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:07:02.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:01 smithi184 ceph-mon[35993]: pgmap v248: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:07:03.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:03 smithi149 ceph-mon[35927]: pgmap v249: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:07:04.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:03 smithi002 ceph-mon[32060]: pgmap v249: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:07:04.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:03 smithi184 ceph-mon[35993]: pgmap v249: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:07:05.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:05 smithi149 ceph-mon[35927]: pgmap v250: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:07:05.859 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:05 smithi184 ceph-mon[35993]: pgmap v250: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:07:06.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:05 smithi002 ceph-mon[32060]: pgmap v250: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:07:06.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:06 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:07:07.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:06 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:07:07.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:06 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:07:07.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:07 smithi149 ceph-mon[35927]: pgmap v251: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-15T13:07:07.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:07 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:07:07.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:07 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:07:07.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:07 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:07:08.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:07 smithi002 ceph-mon[32060]: pgmap v251: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-15T13:07:08.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:07 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:07:08.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:07 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:07:08.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:07 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:07:08.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:07 smithi184 ceph-mon[35993]: pgmap v251: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-15T13:07:08.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:07 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:07:08.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:07 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:07:08.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:07 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:07:09.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:09 smithi149 ceph-mon[35927]: pgmap v252: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-15T13:07:10.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:09 smithi002 ceph-mon[32060]: pgmap v252: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-15T13:07:10.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:09 smithi184 ceph-mon[35993]: pgmap v252: 129 pgs: 129 active+clean; 266 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-15T13:07:10.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:10 smithi149 ceph-mon[35927]: osdmap e95: 12 total, 12 up, 12 in 2024-03-15T13:07:11.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:10 smithi002 ceph-mon[32060]: osdmap e95: 12 total, 12 up, 12 in 2024-03-15T13:07:11.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:10 smithi184 ceph-mon[35993]: osdmap e95: 12 total, 12 up, 12 in 2024-03-15T13:07:11.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:11 smithi149 ceph-mon[35927]: pgmap v254: 129 pgs: 21 active+clean+snaptrim, 58 active+clean, 50 active+clean+snaptrim_wait; 264 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:07:12.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:11 smithi002 ceph-mon[32060]: pgmap v254: 129 pgs: 21 active+clean+snaptrim, 58 active+clean, 50 active+clean+snaptrim_wait; 264 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:07:12.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:11 smithi184 ceph-mon[35993]: pgmap v254: 129 pgs: 21 active+clean+snaptrim, 58 active+clean, 50 active+clean+snaptrim_wait; 264 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:07:13.854 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:13 smithi002 ceph-mon[32060]: pgmap v255: 129 pgs: 21 active+clean+snaptrim, 58 active+clean, 50 active+clean+snaptrim_wait; 264 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:07:13.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:13 smithi149 ceph-mon[35927]: pgmap v255: 129 pgs: 21 active+clean+snaptrim, 58 active+clean, 50 active+clean+snaptrim_wait; 264 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:07:14.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:13 smithi184 ceph-mon[35993]: pgmap v255: 129 pgs: 21 active+clean+snaptrim, 58 active+clean, 50 active+clean+snaptrim_wait; 264 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:07:15.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:15 smithi149 ceph-mon[35927]: pgmap v256: 129 pgs: 21 active+clean+snaptrim, 58 active+clean, 50 active+clean+snaptrim_wait; 264 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:07:16.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:15 smithi002 ceph-mon[32060]: pgmap v256: 129 pgs: 21 active+clean+snaptrim, 58 active+clean, 50 active+clean+snaptrim_wait; 264 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:07:16.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:15 smithi184 ceph-mon[35993]: pgmap v256: 129 pgs: 21 active+clean+snaptrim, 58 active+clean, 50 active+clean+snaptrim_wait; 264 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:07:17.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:17 smithi149 ceph-mon[35927]: pgmap v257: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:07:18.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:17 smithi002 ceph-mon[32060]: pgmap v257: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:07:18.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:17 smithi184 ceph-mon[35993]: pgmap v257: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:07:19.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:19 smithi149 ceph-mon[35927]: pgmap v258: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:07:20.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:19 smithi002 ceph-mon[32060]: pgmap v258: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:07:20.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:19 smithi184 ceph-mon[35993]: pgmap v258: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:07:21.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:21 smithi002 ceph-mon[32060]: pgmap v259: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 198 B/s wr, 14 op/s 2024-03-15T13:07:21.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:21 smithi002 ceph-mon[32060]: osdmap e96: 12 total, 12 up, 12 in 2024-03-15T13:07:21.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:21 smithi184 ceph-mon[35993]: pgmap v259: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 198 B/s wr, 14 op/s 2024-03-15T13:07:21.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:21 smithi184 ceph-mon[35993]: osdmap e96: 12 total, 12 up, 12 in 2024-03-15T13:07:21.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:21 smithi149 ceph-mon[35927]: pgmap v259: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 198 B/s wr, 14 op/s 2024-03-15T13:07:21.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:21 smithi149 ceph-mon[35927]: osdmap e96: 12 total, 12 up, 12 in 2024-03-15T13:07:23.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:23 smithi002 ceph-mon[32060]: pgmap v261: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:07:23.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:23 smithi184 ceph-mon[35993]: pgmap v261: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:07:23.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:23 smithi149 ceph-mon[35927]: pgmap v261: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:07:25.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:07:25.634 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:07:25.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.766+0000 7f665f623640 1 -- 172.21.15.2:0/3194590996 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6658100640 msgr2=0x7f6658100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.766+0000 7f665f623640 1 --2- 172.21.15.2:0/3194590996 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6658100640 0x7f6658100aa0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6648009900 tx=0x7f664802bc30 comp rx=0 tx=0).stop 2024-03-15T13:07:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.766+0000 7f665f623640 1 -- 172.21.15.2:0/3194590996 shutdown_connections 2024-03-15T13:07:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.766+0000 7f665f623640 1 --2- 172.21.15.2:0/3194590996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6658100fe0 0x7f665810d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.766+0000 7f665f623640 1 --2- 172.21.15.2:0/3194590996 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6658100640 0x7f6658100aa0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.766+0000 7f665f623640 1 --2- 172.21.15.2:0/3194590996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6658106420 0x7f6658106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.766+0000 7f665f623640 1 -- 172.21.15.2:0/3194590996 >> 172.21.15.2:0/3194590996 conn(0x7f6658077190 msgr2=0x7f66580775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.766+0000 7f665f623640 1 -- 172.21.15.2:0/3194590996 shutdown_connections 2024-03-15T13:07:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.766+0000 7f665f623640 1 -- 172.21.15.2:0/3194590996 wait complete. 2024-03-15T13:07:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.767+0000 7f665f623640 1 Processor -- start 2024-03-15T13:07:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.767+0000 7f665f623640 1 -- start start 2024-03-15T13:07:25.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.767+0000 7f665f623640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6658100640 0x7f6658199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.767+0000 7f665f623640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6658100fe0 0x7f665819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.767+0000 7f665f623640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6658106420 0x7f665819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.767+0000 7f665f623640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665819ec30 con 0x7f6658106420 2024-03-15T13:07:25.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.767+0000 7f665f623640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665819eda0 con 0x7f6658100fe0 2024-03-15T13:07:25.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.767+0000 7f665f623640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665819ef60 con 0x7f6658100640 2024-03-15T13:07:25.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.767+0000 7f665dbc0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6658106420 0x7f665819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.767+0000 7f665d3bf640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6658100640 0x7f6658199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.767+0000 7f665dbc0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6658106420 0x7f665819e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49294/0 (socket says 172.21.15.2:49294) 2024-03-15T13:07:25.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.767+0000 7f665dbc0640 1 -- 172.21.15.2:0/3177958555 learned_addr learned my addr 172.21.15.2:0/3177958555 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:25.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.767+0000 7f665cbbe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6658100fe0 0x7f665819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.768+0000 7f665dbc0640 1 -- 172.21.15.2:0/3177958555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6658100640 msgr2=0x7f6658199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:25.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.768+0000 7f665dbc0640 1 --2- 172.21.15.2:0/3177958555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6658100640 0x7f6658199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.768+0000 7f665dbc0640 1 -- 172.21.15.2:0/3177958555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6658100fe0 msgr2=0x7f665819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:25.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.768+0000 7f665dbc0640 1 --2- 172.21.15.2:0/3177958555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6658100fe0 0x7f665819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.768+0000 7f665dbc0640 1 -- 172.21.15.2:0/3177958555 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6648009560 con 0x7f6658106420 2024-03-15T13:07:25.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.768+0000 7f665d3bf640 1 --2- 172.21.15.2:0/3177958555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6658100640 0x7f6658199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:25.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.768+0000 7f665cbbe640 1 --2- 172.21.15.2:0/3177958555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6658100fe0 0x7f665819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:25.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.768+0000 7f665dbc0640 1 --2- 172.21.15.2:0/3177958555 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6658106420 0x7f665819e630 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f665000eb10 tx=0x7f665000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:25.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.768+0000 7f664e7fc640 1 -- 172.21.15.2:0/3177958555 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6650004b20 con 0x7f6658106420 2024-03-15T13:07:25.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.768+0000 7f664e7fc640 1 -- 172.21.15.2:0/3177958555 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6650004c80 con 0x7f6658106420 2024-03-15T13:07:25.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.768+0000 7f665f623640 1 -- 172.21.15.2:0/3177958555 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665819f240 con 0x7f6658106420 2024-03-15T13:07:25.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.768+0000 7f664e7fc640 1 -- 172.21.15.2:0/3177958555 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6650010690 con 0x7f6658106420 2024-03-15T13:07:25.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.768+0000 7f665f623640 1 -- 172.21.15.2:0/3177958555 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66581a6ac0 con 0x7f6658106420 2024-03-15T13:07:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.771+0000 7f664e7fc640 1 -- 172.21.15.2:0/3177958555 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f6650004df0 con 0x7f6658106420 2024-03-15T13:07:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.771+0000 7f66621fe740 1 Processor -- start 2024-03-15T13:07:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.771+0000 7f66621fe740 1 -- start start 2024-03-15T13:07:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.771+0000 7f66621fe740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac972a2b0 0x560ac972a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.771+0000 7f66621fe740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972abd0 0x560ac9733270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f66621fe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac97337b0 0x560ac9735ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f66621fe740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac9699280 con 0x560ac972abd0 2024-03-15T13:07:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f66621fe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac96967b0 con 0x560ac97337b0 2024-03-15T13:07:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f66621fe740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac9691f90 con 0x560ac972a2b0 2024-03-15T13:07:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f665d3bf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972abd0 0x560ac9733270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f665cbbe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac972a2b0 0x560ac972a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f665d3bf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972abd0 0x560ac9733270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49296/0 (socket says 172.21.15.2:49296) 2024-03-15T13:07:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f665d3bf640 1 -- 172.21.15.2:0/2230665112 learned_addr learned my addr 172.21.15.2:0/2230665112 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f665cbbe640 1 --2- 172.21.15.2:0/2230665112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac97337b0 0x560ac9735ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f664e7fc640 1 --2- 172.21.15.2:0/3177958555 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f66440a1ba0 0x7f66440a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f665d3bf640 1 -- 172.21.15.2:0/2230665112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac972a2b0 msgr2=0x560ac972a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f665d3bf640 1 --2- 172.21.15.2:0/2230665112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac972a2b0 0x560ac972a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f665d3bf640 1 -- 172.21.15.2:0/2230665112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac97337b0 msgr2=0x560ac9735ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f665d3bf640 1 --2- 172.21.15.2:0/2230665112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac97337b0 0x560ac9735ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f665d3bf640 1 -- 172.21.15.2:0/2230665112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ac9650570 con 0x560ac972abd0 2024-03-15T13:07:25.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f665cbbe640 1 --2- 172.21.15.2:0/2230665112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac972a2b0 0x560ac972a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f665dbc0640 1 --2- 172.21.15.2:0/3177958555 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f66440a1ba0 0x7f66440a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f664e7fc640 1 -- 172.21.15.2:0/3177958555 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f6650014070 con 0x7f6658106420 2024-03-15T13:07:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f665d3bf640 1 --2- 172.21.15.2:0/2230665112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972abd0 0x560ac9733270 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f66540114f0 tx=0x7f66540119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ea57878e8b6cec server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.772+0000 7f66437fe640 1 -- 172.21.15.2:0/2230665112 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654011250 con 0x560ac972abd0 2024-03-15T13:07:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.773+0000 7f66437fe640 1 -- 172.21.15.2:0/2230665112 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f665400fa90 con 0x560ac972abd0 2024-03-15T13:07:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.773+0000 7f66437fe640 1 -- 172.21.15.2:0/2230665112 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654019a10 con 0x560ac972abd0 2024-03-15T13:07:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.773+0000 7f665dbc0640 1 --2- 172.21.15.2:0/3177958555 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f66440a1ba0 0x7f66440a4060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6650096e60 tx=0x7f6650084010 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.773+0000 7f66621fe740 1 -- 172.21.15.2:0/2230665112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972abd0 msgr2=0x560ac9733270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.773+0000 7f66621fe740 1 --2- 172.21.15.2:0/2230665112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972abd0 0x560ac9733270 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f66540114f0 tx=0x7f66540119b0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.773+0000 7f66621fe740 1 -- 172.21.15.2:0/2230665112 shutdown_connections 2024-03-15T13:07:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.773+0000 7f66621fe740 1 --2- 172.21.15.2:0/2230665112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac97337b0 0x560ac9735ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.773+0000 7f66621fe740 1 --2- 172.21.15.2:0/2230665112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972abd0 0x560ac9733270 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.773+0000 7f66621fe740 1 --2- 172.21.15.2:0/2230665112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac972a2b0 0x560ac972a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.773+0000 7f66621fe740 1 -- 172.21.15.2:0/2230665112 >> 172.21.15.2:0/2230665112 conn(0x560ac9724990 msgr2=0x560ac9724d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.773+0000 7f66621fe740 1 -- 172.21.15.2:0/2230665112 shutdown_connections 2024-03-15T13:07:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.773+0000 7f66621fe740 1 -- 172.21.15.2:0/2230665112 wait complete. 2024-03-15T13:07:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.774+0000 7f66621fe740 1 -- start start 2024-03-15T13:07:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.774+0000 7f66621fe740 1 Processor -- start 2024-03-15T13:07:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.774+0000 7f66621fe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972a2b0 0x560ac9730610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.774+0000 7f66621fe740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972abd0 0x560ac9730b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.774+0000 7f66621fe740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac97337b0 0x560ac9731090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.774+0000 7f66621fe740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac9365430 con 0x560ac972abd0 2024-03-15T13:07:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.774+0000 7f66621fe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac9321e00 con 0x560ac972a2b0 2024-03-15T13:07:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.774+0000 7f66621fe740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac9616fd0 con 0x560ac97337b0 2024-03-15T13:07:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.774+0000 7f66621fe740 10 client.0 _ll_register_callbacks cb 0x560ac966cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.774+0000 7f665d3bf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972abd0 0x560ac9730b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.774+0000 7f665cbbe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972a2b0 0x560ac9730610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.774+0000 7f665d3bf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972abd0 0x560ac9730b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49298/0 (socket says 172.21.15.2:49298) 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.774+0000 7f665d3bf640 1 -- 172.21.15.2:0/2471258125 learned_addr learned my addr 172.21.15.2:0/2471258125 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.774+0000 7f665cbbe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972a2b0 0x560ac9730610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:33526/0 (socket says 172.21.15.2:33526) 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.774+0000 7f665cbbe640 1 --2- 172.21.15.2:0/2471258125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac97337b0 0x560ac9731090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.775+0000 7f665d3bf640 1 -- 172.21.15.2:0/2471258125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac97337b0 msgr2=0x560ac9731090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.775+0000 7f665d3bf640 1 --2- 172.21.15.2:0/2471258125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac97337b0 0x560ac9731090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.775+0000 7f665d3bf640 1 -- 172.21.15.2:0/2471258125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972a2b0 msgr2=0x560ac9730610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.775+0000 7f665d3bf640 1 --2- 172.21.15.2:0/2471258125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972a2b0 0x560ac9730610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.775+0000 7f665d3bf640 1 -- 172.21.15.2:0/2471258125 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6654009cc0 con 0x560ac972abd0 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.775+0000 7f665d3bf640 1 --2- 172.21.15.2:0/2471258125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972abd0 0x560ac9730b50 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f66540114c0 tx=0x7f6654011420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b0a20f53fe85e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.775+0000 7f6640ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.775+0000 7f6640ff9640 1 -- 172.21.15.2:0/2471258125 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665400f7c0 con 0x560ac972abd0 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.775+0000 7f6640ff9640 1 -- 172.21.15.2:0/2471258125 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f6654004020 con 0x560ac972abd0 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.775+0000 7f66621fe740 1 -- 172.21.15.2:0/2471258125 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ac93917e0 con 0x560ac972abd0 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.775+0000 7f6640ff9640 1 -- 172.21.15.2:0/2471258125 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f665400f920 con 0x560ac972abd0 2024-03-15T13:07:25.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.775+0000 7f66621fe740 10 client.18087 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.775+0000 7f66621fe740 1 -- 172.21.15.2:0/2471258125 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x560ac9690610 con 0x560ac972abd0 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.775+0000 7f6640ff9640 1 -- 172.21.15.2:0/2471258125 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x560ac93917e0 con 0x560ac972abd0 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.776+0000 7f6640ff9640 1 -- 172.21.15.2:0/2471258125 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f6654019950 con 0x560ac972abd0 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.776+0000 7f66621fe740 10 client.18087 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.776+0000 7f66621fe740 10 client.18087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.776+0000 7f66621fe740 20 client.18087 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.776+0000 7f66621fe740 1 --2- 172.21.15.2:0/2471258125 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x560ac97438a0 0x560ac9745cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.776+0000 7f66621fe740 4 client.18087 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.776+0000 7f66621fe740 1 -- 172.21.15.2:0/2471258125 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ac93ce720 con 0x560ac97438a0 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.776+0000 7f665cbbe640 1 --2- 172.21.15.2:0/2471258125 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x560ac97438a0 0x560ac9745cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.776+0000 7f665cbbe640 1 --2- 172.21.15.2:0/2471258125 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x560ac97438a0 0x560ac9745cc0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39d7a38311020b7f server_cookie=6e1a87dbe73292c7 in_seq=0 out_seq=0 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.776+0000 7f6640ff9640 10 client.18087 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.777+0000 7f6640ff9640 1 -- 172.21.15.2:0/2471258125 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560ac93ce720 con 0x560ac97438a0 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.777+0000 7f6640ff9640 10 client.18087 handle_command_reply: tid=0 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.777+0000 7f66621fe740 1 client.18087 shutdown 2024-03-15T13:07:25.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.777+0000 7f66621fe740 1 -- 172.21.15.2:0/2471258125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972abd0 msgr2=0x560ac9730b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:25.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.777+0000 7f66621fe740 1 --2- 172.21.15.2:0/2471258125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972abd0 0x560ac9730b50 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f66540114c0 tx=0x7f6654011420 comp rx=0 tx=0).stop 2024-03-15T13:07:25.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.777+0000 7f66621fe740 1 -- 172.21.15.2:0/2471258125 shutdown_connections 2024-03-15T13:07:25.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.777+0000 7f66621fe740 1 --2- 172.21.15.2:0/2471258125 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x560ac97438a0 0x560ac9745cc0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.777+0000 7f66621fe740 1 --2- 172.21.15.2:0/2471258125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac97337b0 0x560ac9731090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.777+0000 7f66621fe740 1 --2- 172.21.15.2:0/2471258125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972abd0 0x560ac9730b50 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.777+0000 7f66621fe740 1 --2- 172.21.15.2:0/2471258125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972a2b0 0x560ac9730610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.777+0000 7f6640ff9640 0 client.18087 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:25.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.777+0000 7f66621fe740 1 -- 172.21.15.2:0/2471258125 >> 172.21.15.2:0/2471258125 conn(0x560ac9724990 msgr2=0x560ac9726730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:25.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.777+0000 7f66621fe740 1 -- 172.21.15.2:0/2471258125 shutdown_connections 2024-03-15T13:07:25.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.777+0000 7f66621fe740 1 -- 172.21.15.2:0/2471258125 wait complete. 2024-03-15T13:07:25.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.777+0000 7f66621fe740 20 client.18087 trim_cache size 0 max 16384 2024-03-15T13:07:25.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.793+0000 7f66621fe740 1 Processor -- start 2024-03-15T13:07:25.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f66621fe740 1 -- start start 2024-03-15T13:07:25.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f66621fe740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972a2b0 0x560ac9732d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f66621fe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972abd0 0x560ac9730eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f66621fe740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac97337b0 0x560ac973eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f66621fe740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac9725780 con 0x560ac972a2b0 2024-03-15T13:07:25.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f66621fe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac972b410 con 0x560ac972abd0 2024-03-15T13:07:25.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f66621fe740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac972eba0 con 0x560ac97337b0 2024-03-15T13:07:25.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f665cbbe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972a2b0 0x560ac9732d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f665cbbe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972a2b0 0x560ac9732d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49312/0 (socket says 172.21.15.2:49312) 2024-03-15T13:07:25.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f665cbbe640 1 -- 172.21.15.2:0/979381723 learned_addr learned my addr 172.21.15.2:0/979381723 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:25.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f665d3bf640 1 --2- 172.21.15.2:0/979381723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972abd0 0x560ac9730eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f665cbbe640 1 --2- 172.21.15.2:0/979381723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac97337b0 0x560ac973eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f665cbbe640 1 -- 172.21.15.2:0/979381723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac97337b0 msgr2=0x560ac973eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:25.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f665cbbe640 1 --2- 172.21.15.2:0/979381723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac97337b0 0x560ac973eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f665cbbe640 1 -- 172.21.15.2:0/979381723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972abd0 msgr2=0x560ac9730eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f665cbbe640 1 --2- 172.21.15.2:0/979381723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972abd0 0x560ac9730eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f665cbbe640 1 -- 172.21.15.2:0/979381723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ac962cac0 con 0x560ac972a2b0 2024-03-15T13:07:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.794+0000 7f665cbbe640 1 --2- 172.21.15.2:0/979381723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972a2b0 0x560ac9732d40 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f664802caa0 tx=0x7f6648005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23786bdd6001ec90 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.795+0000 7f6642ffd640 1 -- 172.21.15.2:0/979381723 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648048040 con 0x560ac972a2b0 2024-03-15T13:07:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.795+0000 7f6642ffd640 1 -- 172.21.15.2:0/979381723 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6648057070 con 0x560ac972a2b0 2024-03-15T13:07:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.795+0000 7f6642ffd640 1 -- 172.21.15.2:0/979381723 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648051970 con 0x560ac972a2b0 2024-03-15T13:07:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.795+0000 7f66621fe740 1 -- 172.21.15.2:0/979381723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972a2b0 msgr2=0x560ac9732d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.795+0000 7f66621fe740 1 --2- 172.21.15.2:0/979381723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972a2b0 0x560ac9732d40 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f664802caa0 tx=0x7f6648005fd0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.795+0000 7f66621fe740 1 -- 172.21.15.2:0/979381723 shutdown_connections 2024-03-15T13:07:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.795+0000 7f66621fe740 1 --2- 172.21.15.2:0/979381723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac97337b0 0x560ac973eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.795+0000 7f66621fe740 1 --2- 172.21.15.2:0/979381723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972abd0 0x560ac9730eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.795+0000 7f66621fe740 1 --2- 172.21.15.2:0/979381723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac972a2b0 0x560ac9732d40 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.795+0000 7f66621fe740 1 -- 172.21.15.2:0/979381723 >> 172.21.15.2:0/979381723 conn(0x560ac9724990 msgr2=0x560ac9723a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.795+0000 7f66621fe740 1 -- 172.21.15.2:0/979381723 shutdown_connections 2024-03-15T13:07:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.795+0000 7f66621fe740 1 -- 172.21.15.2:0/979381723 wait complete. 2024-03-15T13:07:25.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f66621fe740 1 -- start start 2024-03-15T13:07:25.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f66621fe740 1 Processor -- start 2024-03-15T13:07:25.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f66621fe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972a2b0 0x560ac9739c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f66621fe740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac972abd0 0x560ac973a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f66621fe740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac97337b0 0x560ac972d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f66621fe740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac972daf0 con 0x560ac97337b0 2024-03-15T13:07:25.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f66621fe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac972dc60 con 0x560ac972a2b0 2024-03-15T13:07:25.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f66621fe740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ac972de20 con 0x560ac972abd0 2024-03-15T13:07:25.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f66621fe740 10 client.0 _ll_register_callbacks cb 0x7f665401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:25.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f665cbbe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac97337b0 0x560ac972d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f665cbbe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac97337b0 0x560ac972d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49328/0 (socket says 172.21.15.2:49328) 2024-03-15T13:07:25.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f665cbbe640 1 -- 172.21.15.2:0/3968831408 learned_addr learned my addr 172.21.15.2:0/3968831408 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:25.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f665d3bf640 1 --2- 172.21.15.2:0/3968831408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac972abd0 0x560ac973a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f665cbbe640 1 --2- 172.21.15.2:0/3968831408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972a2b0 0x560ac9739c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f665cbbe640 1 -- 172.21.15.2:0/3968831408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac972abd0 msgr2=0x560ac973a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:25.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f665cbbe640 1 --2- 172.21.15.2:0/3968831408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac972abd0 0x560ac973a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f665cbbe640 1 -- 172.21.15.2:0/3968831408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972a2b0 msgr2=0x560ac9739c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:25.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f665cbbe640 1 --2- 172.21.15.2:0/3968831408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972a2b0 0x560ac9739c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.796+0000 7f665cbbe640 1 -- 172.21.15.2:0/3968831408 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6648030070 con 0x560ac97337b0 2024-03-15T13:07:25.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f665cbbe640 1 --2- 172.21.15.2:0/3968831408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac97337b0 0x560ac972d3f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f6648004c40 tx=0x7f6648005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56901d5b44afae57 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:25.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f6640ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:25.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f6640ff9640 1 -- 172.21.15.2:0/3968831408 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648048040 con 0x560ac97337b0 2024-03-15T13:07:25.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f6640ff9640 1 -- 172.21.15.2:0/3968831408 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f6648063060 con 0x560ac97337b0 2024-03-15T13:07:25.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f66621fe740 1 -- 172.21.15.2:0/3968831408 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f665400fa90 con 0x560ac97337b0 2024-03-15T13:07:25.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f6640ff9640 1 -- 172.21.15.2:0/3968831408 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66480053c0 con 0x560ac97337b0 2024-03-15T13:07:25.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f66621fe740 10 client.18093 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:25.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f66621fe740 1 -- 172.21.15.2:0/3968831408 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x560ac96526a0 con 0x560ac97337b0 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f6640ff9640 1 -- 172.21.15.2:0/3968831408 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6648069080 con 0x560ac97337b0 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f6640ff9640 1 -- 172.21.15.2:0/3968831408 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f664803c040 con 0x560ac97337b0 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f66621fe740 10 client.18093 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f66621fe740 10 client.18093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f66621fe740 20 client.18093 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f66621fe740 1 --2- 172.21.15.2:0/3968831408 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x560ac979ff70 0x560ac97a2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f66621fe740 4 client.18093 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f66621fe740 1 -- 172.21.15.2:0/3968831408 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560ac941d890 con 0x560ac979ff70 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.797+0000 7f665d3bf640 1 --2- 172.21.15.2:0/3968831408 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x560ac979ff70 0x560ac97a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f665d3bf640 1 --2- 172.21.15.2:0/3968831408 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x560ac979ff70 0x560ac97a2390 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66a831a2be063204 server_cookie=31e1eb09869483b6 in_seq=0 out_seq=0 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f6640ff9640 10 client.18093 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f6640ff9640 1 -- 172.21.15.2:0/3968831408 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560ac941d890 con 0x560ac979ff70 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f6640ff9640 10 client.18093 handle_command_reply: tid=0 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f66621fe740 1 client.18093 shutdown 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f66621fe740 1 -- 172.21.15.2:0/3968831408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac97337b0 msgr2=0x560ac972d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f66621fe740 1 --2- 172.21.15.2:0/3968831408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac97337b0 0x560ac972d3f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f6648004c40 tx=0x7f6648005e50 comp rx=0 tx=0).stop 2024-03-15T13:07:25.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f66621fe740 1 -- 172.21.15.2:0/3968831408 shutdown_connections 2024-03-15T13:07:25.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f66621fe740 1 --2- 172.21.15.2:0/3968831408 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x560ac979ff70 0x560ac97a2390 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f66621fe740 1 --2- 172.21.15.2:0/3968831408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560ac97337b0 0x560ac972d3f0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f6640ff9640 0 client.18093 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:25.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f66621fe740 1 --2- 172.21.15.2:0/3968831408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560ac972abd0 0x560ac973a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f66621fe740 1 --2- 172.21.15.2:0/3968831408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ac972a2b0 0x560ac9739c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f66621fe740 1 -- 172.21.15.2:0/3968831408 >> 172.21.15.2:0/3968831408 conn(0x560ac9724990 msgr2=0x560ac9725b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:25.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f66621fe740 1 -- 172.21.15.2:0/3968831408 shutdown_connections 2024-03-15T13:07:25.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f66621fe740 1 -- 172.21.15.2:0/3968831408 wait complete. 2024-03-15T13:07:25.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.798+0000 7f66621fe740 20 client.18093 trim_cache size 0 max 16384 2024-03-15T13:07:25.789 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:07:25.790 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:07:25.790 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:07:25.790 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:07:25.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.799+0000 7f665f623640 1 -- 172.21.15.2:0/3177958555 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f66440a1ba0 msgr2=0x7f66440a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:25.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.799+0000 7f665f623640 1 --2- 172.21.15.2:0/3177958555 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f66440a1ba0 0x7f66440a4060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6650096e60 tx=0x7f6650084010 comp rx=0 tx=0).stop 2024-03-15T13:07:25.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.799+0000 7f665f623640 1 -- 172.21.15.2:0/3177958555 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6658106420 msgr2=0x7f665819e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:25.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.799+0000 7f665f623640 1 --2- 172.21.15.2:0/3177958555 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6658106420 0x7f665819e630 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f665000eb10 tx=0x7f665000c2d0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.799+0000 7f665f623640 1 -- 172.21.15.2:0/3177958555 shutdown_connections 2024-03-15T13:07:25.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.799+0000 7f665f623640 1 --2- 172.21.15.2:0/3177958555 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f66440a1ba0 0x7f66440a4060 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.799+0000 7f665f623640 1 --2- 172.21.15.2:0/3177958555 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6658106420 0x7f665819e630 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.799+0000 7f665f623640 1 --2- 172.21.15.2:0/3177958555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6658100fe0 0x7f665819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.799+0000 7f665f623640 1 --2- 172.21.15.2:0/3177958555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6658100640 0x7f6658199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.799+0000 7f665f623640 1 -- 172.21.15.2:0/3177958555 >> 172.21.15.2:0/3177958555 conn(0x7f6658077190 msgr2=0x7f66580ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:25.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.799+0000 7f665f623640 1 -- 172.21.15.2:0/3177958555 shutdown_connections 2024-03-15T13:07:25.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.799+0000 7f665f623640 1 -- 172.21.15.2:0/3177958555 wait complete. 2024-03-15T13:07:25.793 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:25 smithi002 ceph-mon[32060]: pgmap v262: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:07:25.794 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:07:25.794 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:07:25.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:25 smithi184 ceph-mon[35993]: pgmap v262: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:07:25.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:25 smithi149 ceph-mon[35927]: pgmap v262: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:07:25.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.912+0000 7f15c2a07640 1 -- 172.21.15.2:0/1147829773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15bc10dd70 msgr2=0x7f15bc110160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:25.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.912+0000 7f15c2a07640 1 --2- 172.21.15.2:0/1147829773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15bc10dd70 0x7f15bc110160 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009be0 tx=0x7f15b402bf80 comp rx=0 tx=0).stop 2024-03-15T13:07:25.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.912+0000 7f15c2a07640 1 -- 172.21.15.2:0/1147829773 shutdown_connections 2024-03-15T13:07:25.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.912+0000 7f15c2a07640 1 --2- 172.21.15.2:0/1147829773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15bc10dd70 0x7f15bc110160 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.912+0000 7f15c2a07640 1 --2- 172.21.15.2:0/1147829773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15bc0741c0 0x7f15bc10d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.912+0000 7f15c2a07640 1 --2- 172.21.15.2:0/1147829773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15bc0738a0 0x7f15bc073c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.912+0000 7f15c2a07640 1 -- 172.21.15.2:0/1147829773 >> 172.21.15.2:0/1147829773 conn(0x7f15bc0fcf60 msgr2=0x7f15bc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:25.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.912+0000 7f15c2a07640 1 -- 172.21.15.2:0/1147829773 shutdown_connections 2024-03-15T13:07:25.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.912+0000 7f15c2a07640 1 -- 172.21.15.2:0/1147829773 wait complete. 2024-03-15T13:07:25.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.913+0000 7f15c2a07640 1 Processor -- start 2024-03-15T13:07:25.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.913+0000 7f15c2a07640 1 -- start start 2024-03-15T13:07:25.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.913+0000 7f15c2a07640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15bc0738a0 0x7f15bc199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.913+0000 7f15c2a07640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15bc0741c0 0x7f15bc19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.913+0000 7f15c2a07640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15bc10dd70 0x7f15bc19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.913+0000 7f15c2a07640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc19ec20 con 0x7f15bc10dd70 2024-03-15T13:07:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.913+0000 7f15c2a07640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc19ed90 con 0x7f15bc0741c0 2024-03-15T13:07:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.913+0000 7f15c2a07640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc19ef50 con 0x7f15bc0738a0 2024-03-15T13:07:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.913+0000 7f15c0fa4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15bc10dd70 0x7f15bc19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.913+0000 7f15c0fa4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15bc10dd70 0x7f15bc19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49352/0 (socket says 172.21.15.2:49352) 2024-03-15T13:07:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.913+0000 7f15c0fa4640 1 -- 172.21.15.2:0/4169352840 learned_addr learned my addr 172.21.15.2:0/4169352840 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.913+0000 7f15bbfff640 1 --2- 172.21.15.2:0/4169352840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15bc0738a0 0x7f15bc199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.913+0000 7f15bb7fe640 1 --2- 172.21.15.2:0/4169352840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15bc0741c0 0x7f15bc19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.914+0000 7f15c0fa4640 1 -- 172.21.15.2:0/4169352840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15bc0738a0 msgr2=0x7f15bc199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:25.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.914+0000 7f15c0fa4640 1 --2- 172.21.15.2:0/4169352840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15bc0738a0 0x7f15bc199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.914+0000 7f15c0fa4640 1 -- 172.21.15.2:0/4169352840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15bc0741c0 msgr2=0x7f15bc19a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.914+0000 7f15c0fa4640 1 --2- 172.21.15.2:0/4169352840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15bc0741c0 0x7f15bc19a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.914+0000 7f15c0fa4640 1 -- 172.21.15.2:0/4169352840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b4009840 con 0x7f15bc10dd70 2024-03-15T13:07:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.914+0000 7f15c0fa4640 1 --2- 172.21.15.2:0/4169352840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15bc10dd70 0x7f15bc19e620 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f15b4033420 tx=0x7f15b40336b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.914+0000 7f15b97fa640 1 -- 172.21.15.2:0/4169352840 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b40338a0 con 0x7f15bc10dd70 2024-03-15T13:07:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.914+0000 7f15c2a07640 1 -- 172.21.15.2:0/4169352840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15bc19f1d0 con 0x7f15bc10dd70 2024-03-15T13:07:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.914+0000 7f15b97fa640 1 -- 172.21.15.2:0/4169352840 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f15b4003b80 con 0x7f15bc10dd70 2024-03-15T13:07:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.914+0000 7f15b97fa640 1 -- 172.21.15.2:0/4169352840 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b403e540 con 0x7f15bc10dd70 2024-03-15T13:07:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.914+0000 7f15c2a07640 1 -- 172.21.15.2:0/4169352840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15bc1a6ab0 con 0x7f15bc10dd70 2024-03-15T13:07:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.915+0000 7f15c2a07640 1 -- 172.21.15.2:0/4169352840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15bc074f50 con 0x7f15bc10dd70 2024-03-15T13:07:25.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.916+0000 7f15b97fa640 1 -- 172.21.15.2:0/4169352840 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f15b4003cf0 con 0x7f15bc10dd70 2024-03-15T13:07:25.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.917+0000 7f15b97fa640 1 --2- 172.21.15.2:0/4169352840 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f15a40a1ba0 0x7f15a40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:25.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.917+0000 7f15bbfff640 1 --2- 172.21.15.2:0/4169352840 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f15a40a1ba0 0x7f15a40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:25.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.917+0000 7f15b97fa640 1 -- 172.21.15.2:0/4169352840 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f15b40f0660 con 0x7f15bc10dd70 2024-03-15T13:07:25.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.917+0000 7f15bbfff640 1 --2- 172.21.15.2:0/4169352840 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f15a40a1ba0 0x7f15a40a4060 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f15ac005fd0 tx=0x7f15ac009260 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:25.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:25.921+0000 7f15b97fa640 1 -- 172.21.15.2:0/4169352840 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f15b40b92a0 con 0x7f15bc10dd70 2024-03-15T13:07:26.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.142+0000 7f15c2a07640 1 -- 172.21.15.2:0/4169352840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15bc108ef0 con 0x7f15bc10dd70 2024-03-15T13:07:26.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.143+0000 7f15b97fa640 1 -- 172.21.15.2:0/4169352840 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+11453 (secure 0 0 0) 0x7f15b40bc960 con 0x7f15bc10dd70 2024-03-15T13:07:26.128 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:07:26.128 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:05:12.138845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24500,24464,24460,24434,24539]},"id":1}]} 2024-03-15T13:07:26.128 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 50 2024-03-15T13:07:26.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.145+0000 7f15c2a07640 1 -- 172.21.15.2:0/4169352840 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f15a40a1ba0 msgr2=0x7f15a40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.145+0000 7f15c2a07640 1 --2- 172.21.15.2:0/4169352840 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f15a40a1ba0 0x7f15a40a4060 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f15ac005fd0 tx=0x7f15ac009260 comp rx=0 tx=0).stop 2024-03-15T13:07:26.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.145+0000 7f15c2a07640 1 -- 172.21.15.2:0/4169352840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15bc10dd70 msgr2=0x7f15bc19e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.145+0000 7f15c2a07640 1 --2- 172.21.15.2:0/4169352840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15bc10dd70 0x7f15bc19e620 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f15b4033420 tx=0x7f15b40336b0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.145+0000 7f15c2a07640 1 -- 172.21.15.2:0/4169352840 shutdown_connections 2024-03-15T13:07:26.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.145+0000 7f15c2a07640 1 --2- 172.21.15.2:0/4169352840 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f15a40a1ba0 0x7f15a40a4060 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.145+0000 7f15c2a07640 1 --2- 172.21.15.2:0/4169352840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15bc10dd70 0x7f15bc19e620 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.145+0000 7f15c2a07640 1 --2- 172.21.15.2:0/4169352840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15bc0741c0 0x7f15bc19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.145+0000 7f15c2a07640 1 --2- 172.21.15.2:0/4169352840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15bc0738a0 0x7f15bc199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.145+0000 7f15c2a07640 1 -- 172.21.15.2:0/4169352840 >> 172.21.15.2:0/4169352840 conn(0x7f15bc0fcf60 msgr2=0x7f15bc0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.145+0000 7f15c2a07640 1 -- 172.21.15.2:0/4169352840 shutdown_connections 2024-03-15T13:07:26.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.145+0000 7f15c2a07640 1 -- 172.21.15.2:0/4169352840 wait complete. 2024-03-15T13:07:26.141 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:07:26.141 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:07:26.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.261+0000 7f186f6ae640 1 -- 172.21.15.2:0/2122228247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18681074f0 msgr2=0x7f18681078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.261+0000 7f186f6ae640 1 --2- 172.21.15.2:0/2122228247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18681074f0 0x7f18681078d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1860009890 tx=0x7f186002bb70 comp rx=0 tx=0).stop 2024-03-15T13:07:26.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.261+0000 7f186f6ae640 1 -- 172.21.15.2:0/2122228247 shutdown_connections 2024-03-15T13:07:26.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.261+0000 7f186f6ae640 1 --2- 172.21.15.2:0/2122228247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18680690b0 0x7f186810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.261+0000 7f186f6ae640 1 --2- 172.21.15.2:0/2122228247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18680686f0 0x7f1868068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.261+0000 7f186f6ae640 1 --2- 172.21.15.2:0/2122228247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18681074f0 0x7f18681078d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.261+0000 7f186f6ae640 1 -- 172.21.15.2:0/2122228247 >> 172.21.15.2:0/2122228247 conn(0x7f18680774f0 msgr2=0x7f1868077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.261+0000 7f186f6ae640 1 -- 172.21.15.2:0/2122228247 shutdown_connections 2024-03-15T13:07:26.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.261+0000 7f186f6ae640 1 -- 172.21.15.2:0/2122228247 wait complete. 2024-03-15T13:07:26.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.261+0000 7f186f6ae640 1 Processor -- start 2024-03-15T13:07:26.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.261+0000 7f186f6ae640 1 -- start start 2024-03-15T13:07:26.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.261+0000 7f186f6ae640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18680686f0 0x7f18681003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186f6ae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18680690b0 0x7f18681008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186f6ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18681074f0 0x7f1868100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186f6ae640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18681a5ea0 con 0x7f18680690b0 2024-03-15T13:07:26.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186f6ae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18681a6010 con 0x7f18681074f0 2024-03-15T13:07:26.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186f6ae640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18681a61d0 con 0x7f18680686f0 2024-03-15T13:07:26.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186cc49640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18680690b0 0x7f18681008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186cc49640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18680690b0 0x7f18681008f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49368/0 (socket says 172.21.15.2:49368) 2024-03-15T13:07:26.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186cc49640 1 -- 172.21.15.2:0/4041920469 learned_addr learned my addr 172.21.15.2:0/4041920469 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186d44a640 1 --2- 172.21.15.2:0/4041920469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18680686f0 0x7f18681003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186dc4b640 1 --2- 172.21.15.2:0/4041920469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18681074f0 0x7f1868100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186cc49640 1 -- 172.21.15.2:0/4041920469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18680686f0 msgr2=0x7f18681003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186cc49640 1 --2- 172.21.15.2:0/4041920469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18680686f0 0x7f18681003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186cc49640 1 -- 172.21.15.2:0/4041920469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18681074f0 msgr2=0x7f1868100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186cc49640 1 --2- 172.21.15.2:0/4041920469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18681074f0 0x7f1868100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186cc49640 1 -- 172.21.15.2:0/4041920469 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c009560 con 0x7f18680690b0 2024-03-15T13:07:26.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186cc49640 1 --2- 172.21.15.2:0/4041920469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18680690b0 0x7f18681008f0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f185c00e9c0 tx=0x7f185c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f18667fc640 1 -- 172.21.15.2:0/4041920469 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00cde0 con 0x7f18680690b0 2024-03-15T13:07:26.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186f6ae640 1 -- 172.21.15.2:0/4041920469 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1860009490 con 0x7f18680690b0 2024-03-15T13:07:26.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f18667fc640 1 -- 172.21.15.2:0/4041920469 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f185c0035f0 con 0x7f18680690b0 2024-03-15T13:07:26.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f18667fc640 1 -- 172.21.15.2:0/4041920469 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c010670 con 0x7f18680690b0 2024-03-15T13:07:26.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.262+0000 7f186f6ae640 1 -- 172.21.15.2:0/4041920469 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18681a6830 con 0x7f18680690b0 2024-03-15T13:07:26.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.264+0000 7f18667fc640 1 -- 172.21.15.2:0/4041920469 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f185c003760 con 0x7f18680690b0 2024-03-15T13:07:26.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f187236c740 1 Processor -- start 2024-03-15T13:07:26.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f187236c740 1 -- start start 2024-03-15T13:07:26.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f187236c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeeda250 0x55d3eeeda630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f187236c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeedab70 0x55d3eeee3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f187236c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeee3750 0x55d3eeee5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f187236c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3eee95960 con 0x55d3eeeda250 2024-03-15T13:07:26.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f187236c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3eee49310 con 0x55d3eeee3750 2024-03-15T13:07:26.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f187236c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3eee46840 con 0x55d3eeedab70 2024-03-15T13:07:26.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f18667fc640 1 --2- 172.21.15.2:0/4041920469 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f18500a1ba0 0x7f18500a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f186dc4b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeeda250 0x55d3eeeda630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f186d44a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeedab70 0x55d3eeee3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f186cc49640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeee3750 0x55d3eeee5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f186dc4b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeeda250 0x55d3eeeda630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49376/0 (socket says 172.21.15.2:49376) 2024-03-15T13:07:26.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f186dc4b640 1 -- 172.21.15.2:0/4289191739 learned_addr learned my addr 172.21.15.2:0/4289191739 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f186d44a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeedab70 0x55d3eeee3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37012/0 (socket says 172.21.15.2:37012) 2024-03-15T13:07:26.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f186dc4b640 1 --2- 172.21.15.2:0/4041920469 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f18500a1ba0 0x7f18500a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.265+0000 7f18667fc640 1 -- 172.21.15.2:0/4041920469 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f185c014070 con 0x7f18680690b0 2024-03-15T13:07:26.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.266+0000 7f186d44a640 1 -- 172.21.15.2:0/4289191739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeee3750 msgr2=0x55d3eeee5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.266+0000 7f186d44a640 1 --2- 172.21.15.2:0/4289191739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeee3750 0x55d3eeee5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.266+0000 7f186d44a640 1 -- 172.21.15.2:0/4289191739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeeda250 msgr2=0x55d3eeeda630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.266+0000 7f186d44a640 1 --2- 172.21.15.2:0/4289191739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeeda250 0x55d3eeeda630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.266+0000 7f186d44a640 1 -- 172.21.15.2:0/4289191739 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3eee04060 con 0x55d3eeedab70 2024-03-15T13:07:26.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.266+0000 7f186cc49640 1 --2- 172.21.15.2:0/4289191739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeee3750 0x55d3eeee5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:26.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.266+0000 7f186dc4b640 1 --2- 172.21.15.2:0/4289191739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeeda250 0x55d3eeeda630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:26.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.266+0000 7f186d44a640 1 --2- 172.21.15.2:0/4289191739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeedab70 0x55d3eeee3210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f186002c0b0 tx=0x7f18600323c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67e1b392862392a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.266+0000 7f186dc4b640 1 --2- 172.21.15.2:0/4041920469 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f18500a1ba0 0x7f18500a4060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f18580099e0 tx=0x7f1858009500 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.266+0000 7f184f7fe640 1 -- 172.21.15.2:0/4289191739 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1860034e80 con 0x55d3eeedab70 2024-03-15T13:07:26.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.266+0000 7f184f7fe640 1 -- 172.21.15.2:0/4289191739 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1860004500 con 0x55d3eeedab70 2024-03-15T13:07:26.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.266+0000 7f184f7fe640 1 -- 172.21.15.2:0/4289191739 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186002c910 con 0x55d3eeedab70 2024-03-15T13:07:26.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.267+0000 7f187236c740 1 -- 172.21.15.2:0/4289191739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeedab70 msgr2=0x55d3eeee3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.267+0000 7f187236c740 1 --2- 172.21.15.2:0/4289191739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeedab70 0x55d3eeee3210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f186002c0b0 tx=0x7f18600323c0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.267+0000 7f187236c740 1 -- 172.21.15.2:0/4289191739 shutdown_connections 2024-03-15T13:07:26.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.267+0000 7f187236c740 1 --2- 172.21.15.2:0/4289191739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeee3750 0x55d3eeee5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.267+0000 7f187236c740 1 --2- 172.21.15.2:0/4289191739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeedab70 0x55d3eeee3210 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.267+0000 7f187236c740 1 --2- 172.21.15.2:0/4289191739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeeda250 0x55d3eeeda630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.267+0000 7f187236c740 1 -- 172.21.15.2:0/4289191739 >> 172.21.15.2:0/4289191739 conn(0x55d3eeed4980 msgr2=0x55d3eeed4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.267+0000 7f187236c740 1 -- 172.21.15.2:0/4289191739 shutdown_connections 2024-03-15T13:07:26.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.267+0000 7f187236c740 1 -- 172.21.15.2:0/4289191739 wait complete. 2024-03-15T13:07:26.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.267+0000 7f187236c740 1 -- start start 2024-03-15T13:07:26.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.267+0000 7f187236c740 1 Processor -- start 2024-03-15T13:07:26.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f187236c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeeda250 0x55d3eeee3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f187236c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeedab70 0x55d3eeee04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f187236c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeee3750 0x55d3eeee0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f187236c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3eee42020 con 0x55d3eeedab70 2024-03-15T13:07:26.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f187236c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3eeb15430 con 0x55d3eeee3750 2024-03-15T13:07:26.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f187236c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3eead1e00 con 0x55d3eeeda250 2024-03-15T13:07:26.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f187236c740 10 client.0 _ll_register_callbacks cb 0x55d3eede9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:26.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f186cc49640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeedab70 0x55d3eeee04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f186d44a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeeda250 0x55d3eeee3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f186dc4b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeee3750 0x55d3eeee0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f186cc49640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeedab70 0x55d3eeee04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49384/0 (socket says 172.21.15.2:49384) 2024-03-15T13:07:26.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f186d44a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeeda250 0x55d3eeee3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37016/0 (socket says 172.21.15.2:37016) 2024-03-15T13:07:26.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f186cc49640 1 -- 172.21.15.2:0/1795523962 learned_addr learned my addr 172.21.15.2:0/1795523962 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f186dc4b640 1 -- 172.21.15.2:0/1795523962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeeda250 msgr2=0x55d3eeee3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f186dc4b640 1 --2- 172.21.15.2:0/1795523962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeeda250 0x55d3eeee3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f186dc4b640 1 -- 172.21.15.2:0/1795523962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeedab70 msgr2=0x55d3eeee04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f186dc4b640 1 --2- 172.21.15.2:0/1795523962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeedab70 0x55d3eeee04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f186dc4b640 1 -- 172.21.15.2:0/1795523962 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186810ec20 con 0x55d3eeee3750 2024-03-15T13:07:26.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f186dc4b640 1 --2- 172.21.15.2:0/1795523962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeee3750 0x55d3eeee0ab0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f185800fc20 tx=0x7f1858004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50a2db50a54ef014 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f184cff9640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:07:26.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f184cff9640 1 -- 172.21.15.2:0/1795523962 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858011e10 con 0x55d3eeee3750 2024-03-15T13:07:26.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f184cff9640 1 -- 172.21.15.2:0/1795523962 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f18580047b0 con 0x55d3eeee3750 2024-03-15T13:07:26.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.268+0000 7f187236c740 1 -- 172.21.15.2:0/1795523962 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3eeb7e720 con 0x55d3eeee3750 2024-03-15T13:07:26.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.269+0000 7f184cff9640 1 -- 172.21.15.2:0/1795523962 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1858004b20 con 0x55d3eeee3750 2024-03-15T13:07:26.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.269+0000 7f187236c740 10 client.27974 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:26.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.269+0000 7f187236c740 1 -- 172.21.15.2:0/1795523962 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55d3eee406a0 con 0x55d3eeee3750 2024-03-15T13:07:26.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.269+0000 7f184cff9640 1 -- 172.21.15.2:0/1795523962 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55d3eeb7e720 con 0x55d3eeee3750 2024-03-15T13:07:26.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.269+0000 7f184cff9640 1 -- 172.21.15.2:0/1795523962 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f185801f460 con 0x55d3eeee3750 2024-03-15T13:07:26.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.269+0000 7f187236c740 10 client.27974 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:26.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.269+0000 7f187236c740 10 client.27974 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:07:26.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.269+0000 7f187236c740 20 client.27974 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:26.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.269+0000 7f187236c740 1 --2- 172.21.15.2:0/1795523962 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55d3eeef3920 0x55d3eeef5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.269+0000 7f187236c740 4 client.27974 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:07:26.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.269+0000 7f187236c740 1 -- 172.21.15.2:0/1795523962 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3eebcd890 con 0x55d3eeef3920 2024-03-15T13:07:26.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.269+0000 7f186d44a640 1 --2- 172.21.15.2:0/1795523962 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55d3eeef3920 0x55d3eeef5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.270+0000 7f184cff9640 10 client.27974 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:07:26.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.270+0000 7f186d44a640 1 --2- 172.21.15.2:0/1795523962 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55d3eeef3920 0x55d3eeef5d10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=337e1dd26bb80e77 server_cookie=1d97cbee3b3c5e73 in_seq=0 out_seq=0 2024-03-15T13:07:26.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.271+0000 7f184cff9640 1 -- 172.21.15.2:0/1795523962 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d3eebcd890 con 0x55d3eeef3920 2024-03-15T13:07:26.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.271+0000 7f184cff9640 10 client.27974 handle_command_reply: tid=0 2024-03-15T13:07:26.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.271+0000 7f187236c740 1 client.27974 shutdown 2024-03-15T13:07:26.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.271+0000 7f187236c740 1 -- 172.21.15.2:0/1795523962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeee3750 msgr2=0x55d3eeee0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.271+0000 7f187236c740 1 --2- 172.21.15.2:0/1795523962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeee3750 0x55d3eeee0ab0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f185800fc20 tx=0x7f1858004a40 comp rx=0 tx=0).stop 2024-03-15T13:07:26.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.271+0000 7f187236c740 1 -- 172.21.15.2:0/1795523962 shutdown_connections 2024-03-15T13:07:26.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.271+0000 7f187236c740 1 --2- 172.21.15.2:0/1795523962 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55d3eeef3920 0x55d3eeef5d10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.271+0000 7f187236c740 1 --2- 172.21.15.2:0/1795523962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeee3750 0x55d3eeee0ab0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.271+0000 7f187236c740 1 --2- 172.21.15.2:0/1795523962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeedab70 0x55d3eeee04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.271+0000 7f187236c740 1 --2- 172.21.15.2:0/1795523962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeeda250 0x55d3eeee3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.271+0000 7f184cff9640 0 client.27974 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:07:26.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.271+0000 7f187236c740 1 -- 172.21.15.2:0/1795523962 >> 172.21.15.2:0/1795523962 conn(0x55d3eeed4980 msgr2=0x55d3eeed66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.271+0000 7f187236c740 1 -- 172.21.15.2:0/1795523962 shutdown_connections 2024-03-15T13:07:26.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.271+0000 7f187236c740 1 -- 172.21.15.2:0/1795523962 wait complete. 2024-03-15T13:07:26.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.271+0000 7f187236c740 20 client.27974 trim_cache size 0 max 16384 2024-03-15T13:07:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.283+0000 7f187236c740 1 Processor -- start 2024-03-15T13:07:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.283+0000 7f187236c740 1 -- start start 2024-03-15T13:07:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f187236c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeeda250 0x55d3eeee2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f187236c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeedab70 0x55d3eeee3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f187236c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeee3750 0x55d3eeeeea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f187236c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3eedc7130 con 0x55d3eeee3750 2024-03-15T13:07:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f187236c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3eeed05c0 con 0x55d3eeeda250 2024-03-15T13:07:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f187236c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3eeed9eb0 con 0x55d3eeedab70 2024-03-15T13:07:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f186d44a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeeda250 0x55d3eeee2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f186dc4b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeee3750 0x55d3eeeeea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f186dc4b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeee3750 0x55d3eeeeea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49394/0 (socket says 172.21.15.2:49394) 2024-03-15T13:07:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f186d44a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeeda250 0x55d3eeee2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:33602/0 (socket says 172.21.15.2:33602) 2024-03-15T13:07:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f186dc4b640 1 -- 172.21.15.2:0/3326659684 learned_addr learned my addr 172.21.15.2:0/3326659684 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f186cc49640 1 --2- 172.21.15.2:0/3326659684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeedab70 0x55d3eeee3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f186dc4b640 1 -- 172.21.15.2:0/3326659684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeedab70 msgr2=0x55d3eeee3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f186dc4b640 1 --2- 172.21.15.2:0/3326659684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeedab70 0x55d3eeee3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f186dc4b640 1 -- 172.21.15.2:0/3326659684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeeda250 msgr2=0x55d3eeee2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f186dc4b640 1 --2- 172.21.15.2:0/3326659684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeeda250 0x55d3eeee2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f186dc4b640 1 -- 172.21.15.2:0/3326659684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3eeddcc20 con 0x55d3eeee3750 2024-03-15T13:07:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f186dc4b640 1 --2- 172.21.15.2:0/3326659684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeee3750 0x55d3eeeeea70 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f18580095b0 tx=0x7f1858009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9968adc8e96bafa server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f184effd640 1 -- 172.21.15.2:0/3326659684 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858004420 con 0x55d3eeee3750 2024-03-15T13:07:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f184effd640 1 -- 172.21.15.2:0/3326659684 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f18580047b0 con 0x55d3eeee3750 2024-03-15T13:07:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.284+0000 7f184effd640 1 -- 172.21.15.2:0/3326659684 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18580379a0 con 0x55d3eeee3750 2024-03-15T13:07:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.285+0000 7f187236c740 1 -- 172.21.15.2:0/3326659684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeee3750 msgr2=0x55d3eeeeea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.285+0000 7f187236c740 1 --2- 172.21.15.2:0/3326659684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeee3750 0x55d3eeeeea70 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f18580095b0 tx=0x7f1858009630 comp rx=0 tx=0).stop 2024-03-15T13:07:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.285+0000 7f187236c740 1 -- 172.21.15.2:0/3326659684 shutdown_connections 2024-03-15T13:07:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.285+0000 7f187236c740 1 --2- 172.21.15.2:0/3326659684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeee3750 0x55d3eeeeea70 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.285+0000 7f187236c740 1 --2- 172.21.15.2:0/3326659684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeedab70 0x55d3eeee3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.285+0000 7f187236c740 1 --2- 172.21.15.2:0/3326659684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeeda250 0x55d3eeee2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.285+0000 7f187236c740 1 -- 172.21.15.2:0/3326659684 >> 172.21.15.2:0/3326659684 conn(0x55d3eeed4980 msgr2=0x55d3eeed3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.285+0000 7f187236c740 1 -- 172.21.15.2:0/3326659684 shutdown_connections 2024-03-15T13:07:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.285+0000 7f187236c740 1 -- 172.21.15.2:0/3326659684 wait complete. 2024-03-15T13:07:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.285+0000 7f187236c740 1 -- start start 2024-03-15T13:07:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.285+0000 7f187236c740 1 Processor -- start 2024-03-15T13:07:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f187236c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeeda250 0x55d3eeedcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f187236c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeedab70 0x55d3eeedd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f187236c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeee3750 0x55d3eeedd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f187236c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3eef4e1a0 con 0x55d3eeeda250 2024-03-15T13:07:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f187236c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3eef4e310 con 0x55d3eeedab70 2024-03-15T13:07:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f187236c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3eef4e4d0 con 0x55d3eeee3750 2024-03-15T13:07:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f187236c740 10 client.0 _ll_register_callbacks cb 0x7f1858019040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f186d44a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeeda250 0x55d3eeedcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f186d44a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeeda250 0x55d3eeedcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49398/0 (socket says 172.21.15.2:49398) 2024-03-15T13:07:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f186d44a640 1 -- 172.21.15.2:0/2351991643 learned_addr learned my addr 172.21.15.2:0/2351991643 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f186cc49640 1 --2- 172.21.15.2:0/2351991643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeedab70 0x55d3eeedd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f186dc4b640 1 --2- 172.21.15.2:0/2351991643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeee3750 0x55d3eeedd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f186d44a640 1 -- 172.21.15.2:0/2351991643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeee3750 msgr2=0x55d3eeedd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f186d44a640 1 --2- 172.21.15.2:0/2351991643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeee3750 0x55d3eeedd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f186d44a640 1 -- 172.21.15.2:0/2351991643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeedab70 msgr2=0x55d3eeedd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f186d44a640 1 --2- 172.21.15.2:0/2351991643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeedab70 0x55d3eeedd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f186d44a640 1 -- 172.21.15.2:0/2351991643 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3eee406a0 con 0x55d3eeeda250 2024-03-15T13:07:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f186d44a640 1 --2- 172.21.15.2:0/2351991643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeeda250 0x55d3eeedcd10 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f1860000c00 tx=0x7f1860005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698a8ae3672044c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f184cff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f184cff9640 1 -- 172.21.15.2:0/2351991643 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1860034940 con 0x55d3eeeda250 2024-03-15T13:07:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f184cff9640 1 -- 172.21.15.2:0/2351991643 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f18600548c0 con 0x55d3eeeda250 2024-03-15T13:07:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.286+0000 7f187236c740 1 -- 172.21.15.2:0/2351991643 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3eeb417e0 con 0x55d3eeeda250 2024-03-15T13:07:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.287+0000 7f184cff9640 1 -- 172.21.15.2:0/2351991643 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1860054a80 con 0x55d3eeeda250 2024-03-15T13:07:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.287+0000 7f187236c740 10 client.18117 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.287+0000 7f187236c740 1 -- 172.21.15.2:0/2351991643 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55d3eee028c0 con 0x55d3eeeda250 2024-03-15T13:07:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.287+0000 7f184cff9640 1 -- 172.21.15.2:0/2351991643 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55d3eeb417e0 con 0x55d3eeeda250 2024-03-15T13:07:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.287+0000 7f184cff9640 1 -- 172.21.15.2:0/2351991643 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1860032600 con 0x55d3eeeda250 2024-03-15T13:07:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.287+0000 7f187236c740 10 client.18117 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.287+0000 7f187236c740 10 client.18117 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:07:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.287+0000 7f187236c740 20 client.18117 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.287+0000 7f187236c740 1 --2- 172.21.15.2:0/2351991643 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55d3eef4f900 0x55d3eef51cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.287+0000 7f187236c740 4 client.18117 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:07:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.287+0000 7f187236c740 1 -- 172.21.15.2:0/2351991643 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d3eeaed700 con 0x55d3eef4f900 2024-03-15T13:07:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.287+0000 7f186cc49640 1 --2- 172.21.15.2:0/2351991643 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55d3eef4f900 0x55d3eef51cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f184cff9640 10 client.18117 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:07:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f186cc49640 1 --2- 172.21.15.2:0/2351991643 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55d3eef4f900 0x55d3eef51cf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3d4cbe75fda63249 server_cookie=d0e5c17f7471941f in_seq=0 out_seq=0 2024-03-15T13:07:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f184cff9640 1 -- 172.21.15.2:0/2351991643 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d3eeaed700 con 0x55d3eef4f900 2024-03-15T13:07:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f184cff9640 10 client.18117 handle_command_reply: tid=0 2024-03-15T13:07:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f187236c740 1 client.18117 shutdown 2024-03-15T13:07:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f187236c740 1 -- 172.21.15.2:0/2351991643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeeda250 msgr2=0x55d3eeedcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f187236c740 1 --2- 172.21.15.2:0/2351991643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeeda250 0x55d3eeedcd10 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f1860000c00 tx=0x7f1860005b20 comp rx=0 tx=0).stop 2024-03-15T13:07:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f187236c740 1 -- 172.21.15.2:0/2351991643 shutdown_connections 2024-03-15T13:07:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f187236c740 1 --2- 172.21.15.2:0/2351991643 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55d3eef4f900 0x55d3eef51cf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f187236c740 1 --2- 172.21.15.2:0/2351991643 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d3eeee3750 0x55d3eeedd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f184cff9640 0 client.18117 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:07:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f187236c740 1 --2- 172.21.15.2:0/2351991643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3eeedab70 0x55d3eeedd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f187236c740 1 --2- 172.21.15.2:0/2351991643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d3eeeda250 0x55d3eeedcd10 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f187236c740 1 -- 172.21.15.2:0/2351991643 >> 172.21.15.2:0/2351991643 conn(0x55d3eeed4980 msgr2=0x55d3eeedb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f187236c740 1 -- 172.21.15.2:0/2351991643 shutdown_connections 2024-03-15T13:07:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f187236c740 1 -- 172.21.15.2:0/2351991643 wait complete. 2024-03-15T13:07:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.288+0000 7f187236c740 20 client.18117 trim_cache size 0 max 16384 2024-03-15T13:07:26.281 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:07:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.289+0000 7f186f6ae640 1 -- 172.21.15.2:0/4041920469 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f18500a1ba0 msgr2=0x7f18500a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.289+0000 7f186f6ae640 1 --2- 172.21.15.2:0/4041920469 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f18500a1ba0 0x7f18500a4060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f18580099e0 tx=0x7f1858009500 comp rx=0 tx=0).stop 2024-03-15T13:07:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.289+0000 7f186f6ae640 1 -- 172.21.15.2:0/4041920469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18680690b0 msgr2=0x7f18681008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.289+0000 7f186f6ae640 1 --2- 172.21.15.2:0/4041920469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18680690b0 0x7f18681008f0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f185c00e9c0 tx=0x7f185c00ee80 comp rx=0 tx=0).stop 2024-03-15T13:07:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.289+0000 7f186f6ae640 1 -- 172.21.15.2:0/4041920469 shutdown_connections 2024-03-15T13:07:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.289+0000 7f186f6ae640 1 --2- 172.21.15.2:0/4041920469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18681074f0 0x7f1868100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.289+0000 7f186f6ae640 1 --2- 172.21.15.2:0/4041920469 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f18500a1ba0 0x7f18500a4060 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.289+0000 7f186f6ae640 1 --2- 172.21.15.2:0/4041920469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18680690b0 0x7f18681008f0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.289+0000 7f186f6ae640 1 --2- 172.21.15.2:0/4041920469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18680686f0 0x7f18681003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.289+0000 7f186f6ae640 1 -- 172.21.15.2:0/4041920469 >> 172.21.15.2:0/4041920469 conn(0x7f18680774f0 msgr2=0x7f18680fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.289+0000 7f186f6ae640 1 -- 172.21.15.2:0/4041920469 shutdown_connections 2024-03-15T13:07:26.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.289+0000 7f186f6ae640 1 -- 172.21.15.2:0/4041920469 wait complete. 2024-03-15T13:07:26.284 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:07:26.284 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:07:26.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.404+0000 7f8256956640 1 -- 172.21.15.2:0/305212920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8250100b40 msgr2=0x7f8250100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.404+0000 7f8256956640 1 --2- 172.21.15.2:0/305212920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8250100b40 0x7f8250100fa0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f8244009900 tx=0x7f824402bc80 comp rx=0 tx=0).stop 2024-03-15T13:07:26.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.404+0000 7f8256956640 1 -- 172.21.15.2:0/305212920 shutdown_connections 2024-03-15T13:07:26.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.404+0000 7f8256956640 1 --2- 172.21.15.2:0/305212920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82501014e0 0x7f825010d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.404+0000 7f8256956640 1 --2- 172.21.15.2:0/305212920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8250100b40 0x7f8250100fa0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.404+0000 7f8256956640 1 --2- 172.21.15.2:0/305212920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82501053a0 0x7f8250105780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.404+0000 7f8256956640 1 -- 172.21.15.2:0/305212920 >> 172.21.15.2:0/305212920 conn(0x7f82500769b0 msgr2=0x7f82500fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.404+0000 7f8256956640 1 -- 172.21.15.2:0/305212920 shutdown_connections 2024-03-15T13:07:26.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.404+0000 7f8256956640 1 -- 172.21.15.2:0/305212920 wait complete. 2024-03-15T13:07:26.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.404+0000 7f8256956640 1 Processor -- start 2024-03-15T13:07:26.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.404+0000 7f8256956640 1 -- start start 2024-03-15T13:07:26.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8256956640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8250100b40 0x7f825019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8256956640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82501014e0 0x7f825019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8256956640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501053a0 0x7f82501968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8256956640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8250196f00 con 0x7f82501053a0 2024-03-15T13:07:26.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8256956640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8250197070 con 0x7f8250100b40 2024-03-15T13:07:26.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8256956640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8250197230 con 0x7f82501014e0 2024-03-15T13:07:26.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8254ef3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501053a0 0x7f82501968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f824ffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8250100b40 0x7f825019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8254ef3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501053a0 0x7f82501968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49412/0 (socket says 172.21.15.2:49412) 2024-03-15T13:07:26.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8254ef3640 1 -- 172.21.15.2:0/4196696301 learned_addr learned my addr 172.21.15.2:0/4196696301 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f824f7fe640 1 --2- 172.21.15.2:0/4196696301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82501014e0 0x7f825019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8254ef3640 1 -- 172.21.15.2:0/4196696301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82501014e0 msgr2=0x7f825019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8254ef3640 1 --2- 172.21.15.2:0/4196696301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82501014e0 0x7f825019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8254ef3640 1 -- 172.21.15.2:0/4196696301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8250100b40 msgr2=0x7f825019c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8254ef3640 1 --2- 172.21.15.2:0/4196696301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8250100b40 0x7f825019c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8254ef3640 1 -- 172.21.15.2:0/4196696301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8240009490 con 0x7f82501053a0 2024-03-15T13:07:26.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f824ffff640 1 --2- 172.21.15.2:0/4196696301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8250100b40 0x7f825019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:26.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8254ef3640 1 --2- 172.21.15.2:0/4196696301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501053a0 0x7f82501968b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f824800eb10 tx=0x7f824800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f824d7fa640 1 -- 172.21.15.2:0/4196696301 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8248004b20 con 0x7f82501053a0 2024-03-15T13:07:26.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f824d7fa640 1 -- 172.21.15.2:0/4196696301 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8248004c80 con 0x7f82501053a0 2024-03-15T13:07:26.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8256956640 1 -- 172.21.15.2:0/4196696301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8244009560 con 0x7f82501053a0 2024-03-15T13:07:26.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f824d7fa640 1 -- 172.21.15.2:0/4196696301 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8248010690 con 0x7f82501053a0 2024-03-15T13:07:26.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.405+0000 7f8256956640 1 -- 172.21.15.2:0/4196696301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82501a35c0 con 0x7f82501053a0 2024-03-15T13:07:26.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f824d7fa640 1 -- 172.21.15.2:0/4196696301 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f8248004df0 con 0x7f82501053a0 2024-03-15T13:07:26.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f8259611740 1 Processor -- start 2024-03-15T13:07:26.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f8259611740 1 -- start start 2024-03-15T13:07:26.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f8259611740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee533250 0x556dee533630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f8259611740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee533b70 0x556dee53c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f8259611740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee53c750 0x556dee53eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f8259611740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dee4ee960 con 0x556dee53c750 2024-03-15T13:07:26.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f8259611740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dee4a2310 con 0x556dee533b70 2024-03-15T13:07:26.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f8259611740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dee49f840 con 0x556dee533250 2024-03-15T13:07:26.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f824d7fa640 1 --2- 172.21.15.2:0/4196696301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f82380a1ba0 0x7f82380a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f824f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee53c750 0x556dee53eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f824f7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee533250 0x556dee533630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f824ffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee533b70 0x556dee53c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f824f7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee533250 0x556dee533630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:37072/0 (socket says 172.21.15.2:37072) 2024-03-15T13:07:26.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f824f7fe640 1 -- 172.21.15.2:0/2225092396 learned_addr learned my addr 172.21.15.2:0/2225092396 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.408+0000 7f8254ef3640 1 --2- 172.21.15.2:0/4196696301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f82380a1ba0 0x7f82380a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.409+0000 7f824d7fa640 1 -- 172.21.15.2:0/4196696301 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f8248014070 con 0x7f82501053a0 2024-03-15T13:07:26.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.409+0000 7f824f7fe640 1 -- 172.21.15.2:0/2225092396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee533b70 msgr2=0x556dee53c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.409+0000 7f824f7fe640 1 --2- 172.21.15.2:0/2225092396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee533b70 0x556dee53c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.409+0000 7f824f7fe640 1 -- 172.21.15.2:0/2225092396 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee53c750 msgr2=0x556dee53eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.409+0000 7f824f7fe640 1 --2- 172.21.15.2:0/2225092396 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee53c750 0x556dee53eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.409+0000 7f824f7fe640 1 -- 172.21.15.2:0/2225092396 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dee45d060 con 0x556dee533250 2024-03-15T13:07:26.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.409+0000 7f8254ef3640 1 --2- 172.21.15.2:0/4196696301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f82380a1ba0 0x7f82380a4060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f82480cb5d0 tx=0x7f82480661e0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.409+0000 7f824f7fe640 1 --2- 172.21.15.2:0/2225092396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee533250 0x556dee533630 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8244009900 tx=0x7f8244005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba6f90e4fb67dd9f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.409+0000 7f82367fc640 1 -- 172.21.15.2:0/2225092396 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824402c800 con 0x556dee533250 2024-03-15T13:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.409+0000 7f82367fc640 1 -- 172.21.15.2:0/2225092396 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8244004500 con 0x556dee533250 2024-03-15T13:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.409+0000 7f82367fc640 1 -- 172.21.15.2:0/2225092396 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244003bb0 con 0x556dee533250 2024-03-15T13:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.410+0000 7f8259611740 1 -- 172.21.15.2:0/2225092396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee533250 msgr2=0x556dee533630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.410+0000 7f8259611740 1 --2- 172.21.15.2:0/2225092396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee533250 0x556dee533630 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8244009900 tx=0x7f8244005fd0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.410+0000 7f8259611740 1 -- 172.21.15.2:0/2225092396 shutdown_connections 2024-03-15T13:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.410+0000 7f8259611740 1 --2- 172.21.15.2:0/2225092396 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee53c750 0x556dee53eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.410+0000 7f8259611740 1 --2- 172.21.15.2:0/2225092396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee533b70 0x556dee53c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.410+0000 7f8259611740 1 --2- 172.21.15.2:0/2225092396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee533250 0x556dee533630 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.410+0000 7f8259611740 1 -- 172.21.15.2:0/2225092396 >> 172.21.15.2:0/2225092396 conn(0x556dee52d980 msgr2=0x556dee52dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.410+0000 7f8259611740 1 -- 172.21.15.2:0/2225092396 shutdown_connections 2024-03-15T13:07:26.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.410+0000 7f8259611740 1 -- 172.21.15.2:0/2225092396 wait complete. 2024-03-15T13:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.410+0000 7f8259611740 1 -- start start 2024-03-15T13:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.410+0000 7f8259611740 1 Processor -- start 2024-03-15T13:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f8259611740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee533250 0x556dee53c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f8259611740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533b70 0x556dee5394a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f8259611740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee53c750 0x556dee539ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f8259611740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dee49b020 con 0x556dee533b70 2024-03-15T13:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f8259611740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dee16e430 con 0x556dee53c750 2024-03-15T13:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f8259611740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dee12ae00 con 0x556dee533250 2024-03-15T13:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f8259611740 10 client.0 _ll_register_callbacks cb 0x556dee442cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f824ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533b70 0x556dee5394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f824ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533b70 0x556dee5394a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49430/0 (socket says 172.21.15.2:49430) 2024-03-15T13:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f824ffff640 1 -- 172.21.15.2:0/986164508 learned_addr learned my addr 172.21.15.2:0/986164508 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f824f7fe640 1 --2- 172.21.15.2:0/986164508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee533250 0x556dee53c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f824f7fe640 1 --2- 172.21.15.2:0/986164508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee53c750 0x556dee539ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f824ffff640 1 -- 172.21.15.2:0/986164508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee533250 msgr2=0x556dee53c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f824ffff640 1 --2- 172.21.15.2:0/986164508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee533250 0x556dee53c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f824ffff640 1 -- 172.21.15.2:0/986164508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee53c750 msgr2=0x556dee539ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f824ffff640 1 --2- 172.21.15.2:0/986164508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee53c750 0x556dee539ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f824ffff640 1 -- 172.21.15.2:0/986164508 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825010dbc0 con 0x556dee533b70 2024-03-15T13:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f824ffff640 1 --2- 172.21.15.2:0/986164508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533b70 0x556dee5394a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f8240012d20 tx=0x7f824000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a33cfebf67de99 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f822f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f822f7fe640 1 -- 172.21.15.2:0/986164508 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240017070 con 0x556dee533b70 2024-03-15T13:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f8259611740 1 -- 172.21.15.2:0/986164508 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556dee1d7720 con 0x556dee533b70 2024-03-15T13:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.411+0000 7f822f7fe640 1 -- 172.21.15.2:0/986164508 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f8240004030 con 0x556dee533b70 2024-03-15T13:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.412+0000 7f822f7fe640 1 -- 172.21.15.2:0/986164508 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f824000fe60 con 0x556dee533b70 2024-03-15T13:07:26.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.412+0000 7f8259611740 10 client.18129 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.412+0000 7f8259611740 1 -- 172.21.15.2:0/986164508 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x556dee4996a0 con 0x556dee533b70 2024-03-15T13:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.412+0000 7f822f7fe640 1 -- 172.21.15.2:0/986164508 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x556dee1d7720 con 0x556dee533b70 2024-03-15T13:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.412+0000 7f822f7fe640 1 -- 172.21.15.2:0/986164508 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8240010430 con 0x556dee533b70 2024-03-15T13:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.412+0000 7f8259611740 10 client.18129 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.412+0000 7f8259611740 10 client.18129 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.412+0000 7f8259611740 20 client.18129 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.412+0000 7f8259611740 1 --2- 172.21.15.2:0/986164508 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x556dee54eb30 0x556dee550f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.412+0000 7f8259611740 4 client.18129 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.412+0000 7f8259611740 1 -- 172.21.15.2:0/986164508 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556dee226890 con 0x556dee54eb30 2024-03-15T13:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.412+0000 7f824f7fe640 1 --2- 172.21.15.2:0/986164508 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x556dee54eb30 0x556dee550f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.413+0000 7f824f7fe640 1 --2- 172.21.15.2:0/986164508 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x556dee54eb30 0x556dee550f50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=189dddfb663a381a server_cookie=c56ffea361b05f20 in_seq=0 out_seq=0 2024-03-15T13:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.413+0000 7f822f7fe640 10 client.18129 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.414+0000 7f822f7fe640 1 -- 172.21.15.2:0/986164508 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f824405b4b0 con 0x556dee54eb30 2024-03-15T13:07:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.414+0000 7f822f7fe640 10 client.18129 handle_command_reply: tid=0 2024-03-15T13:07:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.414+0000 7f8259611740 1 client.18129 shutdown 2024-03-15T13:07:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.414+0000 7f8259611740 1 -- 172.21.15.2:0/986164508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533b70 msgr2=0x556dee5394a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.414+0000 7f8259611740 1 --2- 172.21.15.2:0/986164508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533b70 0x556dee5394a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f8240012d20 tx=0x7f824000f790 comp rx=0 tx=0).stop 2024-03-15T13:07:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.414+0000 7f8259611740 1 -- 172.21.15.2:0/986164508 shutdown_connections 2024-03-15T13:07:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.414+0000 7f8259611740 1 --2- 172.21.15.2:0/986164508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee53c750 0x556dee539ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.414+0000 7f8259611740 1 --2- 172.21.15.2:0/986164508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533b70 0x556dee5394a0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.414+0000 7f8259611740 1 --2- 172.21.15.2:0/986164508 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x556dee54eb30 0x556dee550f50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.414+0000 7f8259611740 1 --2- 172.21.15.2:0/986164508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee533250 0x556dee53c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.414+0000 7f822f7fe640 0 client.18129 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:07:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.414+0000 7f8259611740 1 -- 172.21.15.2:0/986164508 >> 172.21.15.2:0/986164508 conn(0x556dee52d980 msgr2=0x556dee52f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.414+0000 7f8259611740 1 -- 172.21.15.2:0/986164508 shutdown_connections 2024-03-15T13:07:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.414+0000 7f8259611740 1 -- 172.21.15.2:0/986164508 wait complete. 2024-03-15T13:07:26.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.414+0000 7f8259611740 20 client.18129 trim_cache size 0 max 16384 2024-03-15T13:07:26.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.426+0000 7f8259611740 1 Processor -- start 2024-03-15T13:07:26.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.426+0000 7f8259611740 1 -- start start 2024-03-15T13:07:26.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.426+0000 7f8259611740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533250 0x556dee53bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.426+0000 7f8259611740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee533b70 0x556dee53c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.426+0000 7f8259611740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee53c750 0x556dee549c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.426+0000 7f8259611740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dee420130 con 0x556dee533250 2024-03-15T13:07:26.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.426+0000 7f8259611740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dee532cc0 con 0x556dee533b70 2024-03-15T13:07:26.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.426+0000 7f8259611740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dee5292b0 con 0x556dee53c750 2024-03-15T13:07:26.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.427+0000 7f824f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533250 0x556dee53bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.427+0000 7f824f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533250 0x556dee53bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49438/0 (socket says 172.21.15.2:49438) 2024-03-15T13:07:26.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.427+0000 7f824f7fe640 1 -- 172.21.15.2:0/533849834 learned_addr learned my addr 172.21.15.2:0/533849834 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.427+0000 7f824ffff640 1 --2- 172.21.15.2:0/533849834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee533b70 0x556dee53c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.427+0000 7f824f7fe640 1 --2- 172.21.15.2:0/533849834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee53c750 0x556dee549c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.427+0000 7f824f7fe640 1 -- 172.21.15.2:0/533849834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee53c750 msgr2=0x556dee549c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.427+0000 7f824f7fe640 1 --2- 172.21.15.2:0/533849834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee53c750 0x556dee549c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.427+0000 7f824f7fe640 1 -- 172.21.15.2:0/533849834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee533b70 msgr2=0x556dee53c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.427+0000 7f824f7fe640 1 --2- 172.21.15.2:0/533849834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee533b70 0x556dee53c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.427+0000 7f824f7fe640 1 -- 172.21.15.2:0/533849834 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dee435c20 con 0x556dee533250 2024-03-15T13:07:26.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.427+0000 7f824f7fe640 1 --2- 172.21.15.2:0/533849834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533250 0x556dee53bbd0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8244005ab0 tx=0x7f8244000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a9dee55c2b9dba server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.427+0000 7f8235ffb640 1 -- 172.21.15.2:0/533849834 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824402cc00 con 0x556dee533250 2024-03-15T13:07:26.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.427+0000 7f8235ffb640 1 -- 172.21.15.2:0/533849834 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8244056040 con 0x556dee533250 2024-03-15T13:07:26.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.427+0000 7f8235ffb640 1 -- 172.21.15.2:0/533849834 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82440346a0 con 0x556dee533250 2024-03-15T13:07:26.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.428+0000 7f8259611740 1 -- 172.21.15.2:0/533849834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533250 msgr2=0x556dee53bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.428+0000 7f8259611740 1 --2- 172.21.15.2:0/533849834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533250 0x556dee53bbd0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8244005ab0 tx=0x7f8244000bc0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.428+0000 7f8259611740 1 -- 172.21.15.2:0/533849834 shutdown_connections 2024-03-15T13:07:26.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.428+0000 7f8259611740 1 --2- 172.21.15.2:0/533849834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee53c750 0x556dee549c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.428+0000 7f8259611740 1 --2- 172.21.15.2:0/533849834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee533b70 0x556dee53c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.428+0000 7f8259611740 1 --2- 172.21.15.2:0/533849834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533250 0x556dee53bbd0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.428+0000 7f8259611740 1 -- 172.21.15.2:0/533849834 >> 172.21.15.2:0/533849834 conn(0x556dee52d980 msgr2=0x556dee52cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.428+0000 7f8259611740 1 -- 172.21.15.2:0/533849834 shutdown_connections 2024-03-15T13:07:26.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.428+0000 7f8259611740 1 -- 172.21.15.2:0/533849834 wait complete. 2024-03-15T13:07:26.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.428+0000 7f8259611740 1 -- start start 2024-03-15T13:07:26.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.428+0000 7f8259611740 1 Processor -- start 2024-03-15T13:07:26.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f8259611740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee533250 0x556dee535d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f8259611740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533b70 0x556dee536250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f8259611740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee53c750 0x556dee536790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f8259611740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dee5a93e0 con 0x556dee533b70 2024-03-15T13:07:26.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f8259611740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dee5a9520 con 0x556dee533250 2024-03-15T13:07:26.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f8259611740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dee5a96e0 con 0x556dee53c750 2024-03-15T13:07:26.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f8259611740 10 client.0 _ll_register_callbacks cb 0x7f8240009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:26.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f824ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533b70 0x556dee536250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f824ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533b70 0x556dee536250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49452/0 (socket says 172.21.15.2:49452) 2024-03-15T13:07:26.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f824ffff640 1 -- 172.21.15.2:0/3694440445 learned_addr learned my addr 172.21.15.2:0/3694440445 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f824f7fe640 1 --2- 172.21.15.2:0/3694440445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee533250 0x556dee535d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f824f7fe640 1 --2- 172.21.15.2:0/3694440445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee53c750 0x556dee536790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f824ffff640 1 -- 172.21.15.2:0/3694440445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee53c750 msgr2=0x556dee536790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f824ffff640 1 --2- 172.21.15.2:0/3694440445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee53c750 0x556dee536790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f824ffff640 1 -- 172.21.15.2:0/3694440445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee533250 msgr2=0x556dee535d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f824ffff640 1 --2- 172.21.15.2:0/3694440445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee533250 0x556dee535d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f824ffff640 1 -- 172.21.15.2:0/3694440445 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556dee45d060 con 0x556dee533b70 2024-03-15T13:07:26.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f824f7fe640 1 --2- 172.21.15.2:0/3694440445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee53c750 0x556dee536790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:07:26.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f824ffff640 1 --2- 172.21.15.2:0/3694440445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533b70 0x556dee536250 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x556dee53aa70 tx=0x7f8240010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8827b8e62f1b7bb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f822f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:26.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f822f7fe640 1 -- 172.21.15.2:0/3694440445 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240017050 con 0x556dee533b70 2024-03-15T13:07:26.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f822f7fe640 1 -- 172.21.15.2:0/3694440445 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f8240000e40 con 0x556dee533b70 2024-03-15T13:07:26.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.429+0000 7f8259611740 1 -- 172.21.15.2:0/3694440445 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556dee19a7e0 con 0x556dee533b70 2024-03-15T13:07:26.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.430+0000 7f822f7fe640 1 -- 172.21.15.2:0/3694440445 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f824000fe60 con 0x556dee533b70 2024-03-15T13:07:26.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.430+0000 7f8259611740 10 client.18135 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:26.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.430+0000 7f8259611740 1 -- 172.21.15.2:0/3694440445 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x556dee45b8c0 con 0x556dee533b70 2024-03-15T13:07:26.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.430+0000 7f822f7fe640 1 -- 172.21.15.2:0/3694440445 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x556dee19a7e0 con 0x556dee533b70 2024-03-15T13:07:26.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.430+0000 7f822f7fe640 1 -- 172.21.15.2:0/3694440445 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8240003ac0 con 0x556dee533b70 2024-03-15T13:07:26.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.430+0000 7f8259611740 10 client.18135 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:26.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.430+0000 7f8259611740 10 client.18135 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:07:26.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.430+0000 7f8259611740 20 client.18135 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:26.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.430+0000 7f8259611740 1 --2- 172.21.15.2:0/3694440445 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x556dee5aaf20 0x556dee5ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.430+0000 7f8259611740 4 client.18135 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:07:26.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.430+0000 7f8259611740 1 -- 172.21.15.2:0/3694440445 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556dee146700 con 0x556dee5aaf20 2024-03-15T13:07:26.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.430+0000 7f824f7fe640 1 --2- 172.21.15.2:0/3694440445 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x556dee5aaf20 0x556dee5ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f822f7fe640 10 client.18135 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:07:26.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f824f7fe640 1 --2- 172.21.15.2:0/3694440445 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x556dee5aaf20 0x556dee5ad340 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f74b0fa3e4b74134 server_cookie=994198835e03c942 in_seq=0 out_seq=0 2024-03-15T13:07:26.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f822f7fe640 1 -- 172.21.15.2:0/3694440445 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556dee146700 con 0x556dee5aaf20 2024-03-15T13:07:26.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f822f7fe640 10 client.18135 handle_command_reply: tid=0 2024-03-15T13:07:26.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8259611740 1 client.18135 shutdown 2024-03-15T13:07:26.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8259611740 1 -- 172.21.15.2:0/3694440445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533b70 msgr2=0x556dee536250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8259611740 1 --2- 172.21.15.2:0/3694440445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533b70 0x556dee536250 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x556dee53aa70 tx=0x7f8240010660 comp rx=0 tx=0).stop 2024-03-15T13:07:26.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8259611740 1 -- 172.21.15.2:0/3694440445 shutdown_connections 2024-03-15T13:07:26.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8259611740 1 --2- 172.21.15.2:0/3694440445 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x556dee5aaf20 0x556dee5ad340 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8259611740 1 --2- 172.21.15.2:0/3694440445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556dee53c750 0x556dee536790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8259611740 1 --2- 172.21.15.2:0/3694440445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556dee533b70 0x556dee536250 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f822f7fe640 0 client.18135 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:07:26.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8259611740 1 --2- 172.21.15.2:0/3694440445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dee533250 0x556dee535d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8259611740 1 -- 172.21.15.2:0/3694440445 >> 172.21.15.2:0/3694440445 conn(0x556dee52d980 msgr2=0x556dee52ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8259611740 1 -- 172.21.15.2:0/3694440445 shutdown_connections 2024-03-15T13:07:26.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8259611740 1 -- 172.21.15.2:0/3694440445 wait complete. 2024-03-15T13:07:26.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8259611740 20 client.18135 trim_cache size 0 max 16384 2024-03-15T13:07:26.424 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:07:26.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8256956640 1 -- 172.21.15.2:0/4196696301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f82380a1ba0 msgr2=0x7f82380a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8256956640 1 --2- 172.21.15.2:0/4196696301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f82380a1ba0 0x7f82380a4060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f82480cb5d0 tx=0x7f82480661e0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8256956640 1 -- 172.21.15.2:0/4196696301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501053a0 msgr2=0x7f82501968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.431+0000 7f8256956640 1 --2- 172.21.15.2:0/4196696301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501053a0 0x7f82501968b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f824800eb10 tx=0x7f824800c2d0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.432+0000 7f8256956640 1 -- 172.21.15.2:0/4196696301 shutdown_connections 2024-03-15T13:07:26.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.432+0000 7f8256956640 1 --2- 172.21.15.2:0/4196696301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f82380a1ba0 0x7f82380a4060 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.432+0000 7f8256956640 1 --2- 172.21.15.2:0/4196696301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82501053a0 0x7f82501968b0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.432+0000 7f8256956640 1 --2- 172.21.15.2:0/4196696301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82501014e0 0x7f825019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.432+0000 7f8256956640 1 --2- 172.21.15.2:0/4196696301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8250100b40 0x7f825019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.432+0000 7f8256956640 1 -- 172.21.15.2:0/4196696301 >> 172.21.15.2:0/4196696301 conn(0x7f82500769b0 msgr2=0x7f8250103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.432+0000 7f8256956640 1 -- 172.21.15.2:0/4196696301 shutdown_connections 2024-03-15T13:07:26.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.432+0000 7f8256956640 1 -- 172.21.15.2:0/4196696301 wait complete. 2024-03-15T13:07:26.427 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:07:26.427 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:07:26.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.547+0000 7f932e888640 1 -- 172.21.15.2:0/2806371823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9328100f80 msgr2=0x7f9328104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.547+0000 7f932e888640 1 --2- 172.21.15.2:0/2806371823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9328100f80 0x7f9328104fc0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f9318009930 tx=0x7f931802bc30 comp rx=0 tx=0).stop 2024-03-15T13:07:26.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.547+0000 7f932e888640 1 -- 172.21.15.2:0/2806371823 shutdown_connections 2024-03-15T13:07:26.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.547+0000 7f932e888640 1 --2- 172.21.15.2:0/2806371823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93281056a0 0x7f932810d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.547+0000 7f932e888640 1 --2- 172.21.15.2:0/2806371823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9328100f80 0x7f9328104fc0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.547+0000 7f932e888640 1 --2- 172.21.15.2:0/2806371823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93281005d0 0x7f93281009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.547+0000 7f932e888640 1 -- 172.21.15.2:0/2806371823 >> 172.21.15.2:0/2806371823 conn(0x7f93280774f0 msgr2=0x7f9328077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.547+0000 7f932e888640 1 -- 172.21.15.2:0/2806371823 shutdown_connections 2024-03-15T13:07:26.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.547+0000 7f932e888640 1 -- 172.21.15.2:0/2806371823 wait complete. 2024-03-15T13:07:26.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.547+0000 7f932e888640 1 Processor -- start 2024-03-15T13:07:26.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.547+0000 7f932e888640 1 -- start start 2024-03-15T13:07:26.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932e888640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93281005d0 0x7f9328199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932e888640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93281056a0 0x7f932819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932e888640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f932819e660 0x7f932819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932e888640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932819f050 con 0x7f932819e660 2024-03-15T13:07:26.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932e888640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932819f1c0 con 0x7f93281056a0 2024-03-15T13:07:26.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932e888640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932819f330 con 0x7f93281005d0 2024-03-15T13:07:26.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932ce25640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f932819e660 0x7f932819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f9327fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93281005d0 0x7f9328199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932ce25640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f932819e660 0x7f932819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49480/0 (socket says 172.21.15.2:49480) 2024-03-15T13:07:26.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932ce25640 1 -- 172.21.15.2:0/440919668 learned_addr learned my addr 172.21.15.2:0/440919668 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f93277fe640 1 --2- 172.21.15.2:0/440919668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93281056a0 0x7f932819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932ce25640 1 -- 172.21.15.2:0/440919668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93281005d0 msgr2=0x7f9328199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932ce25640 1 --2- 172.21.15.2:0/440919668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93281005d0 0x7f9328199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932ce25640 1 -- 172.21.15.2:0/440919668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93281056a0 msgr2=0x7f932819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932ce25640 1 --2- 172.21.15.2:0/440919668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93281056a0 0x7f932819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932ce25640 1 -- 172.21.15.2:0/440919668 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9318009560 con 0x7f932819e660 2024-03-15T13:07:26.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932ce25640 1 --2- 172.21.15.2:0/440919668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f932819e660 0x7f932819eb10 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f932000eb10 tx=0x7f932000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f93257fa640 1 -- 172.21.15.2:0/440919668 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320004b20 con 0x7f932819e660 2024-03-15T13:07:26.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932e888640 1 -- 172.21.15.2:0/440919668 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93281a6910 con 0x7f932819e660 2024-03-15T13:07:26.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f93257fa640 1 -- 172.21.15.2:0/440919668 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9320004c80 con 0x7f932819e660 2024-03-15T13:07:26.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f93257fa640 1 -- 172.21.15.2:0/440919668 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320010690 con 0x7f932819e660 2024-03-15T13:07:26.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.548+0000 7f932e888640 1 -- 172.21.15.2:0/440919668 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93281a6d40 con 0x7f932819e660 2024-03-15T13:07:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.551+0000 7f93257fa640 1 -- 172.21.15.2:0/440919668 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f9320004df0 con 0x7f932819e660 2024-03-15T13:07:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.551+0000 7f933146d740 1 Processor -- start 2024-03-15T13:07:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.551+0000 7f933146d740 1 -- start start 2024-03-15T13:07:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.551+0000 7f933146d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ee7250 0x557d86ee7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.551+0000 7f933146d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7b70 0x557d86ef0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.551+0000 7f933146d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ef0750 0x557d86ef2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.551+0000 7f933146d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d86ea2960 con 0x557d86ef0750 2024-03-15T13:07:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.551+0000 7f933146d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d86e56310 con 0x557d86ee7b70 2024-03-15T13:07:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.551+0000 7f933146d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d86e53840 con 0x557d86ee7250 2024-03-15T13:07:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.551+0000 7f93257fa640 1 --2- 172.21.15.2:0/440919668 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f93100a1ba0 0x7f93100a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.551+0000 7f93277fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ef0750 0x557d86ef2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.551+0000 7f932ce25640 1 --2- 172.21.15.2:0/440919668 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f93100a1ba0 0x7f93100a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.551+0000 7f93277fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ef0750 0x557d86ef2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49496/0 (socket says 172.21.15.2:49496) 2024-03-15T13:07:26.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.551+0000 7f9327fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7b70 0x557d86ef0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.551+0000 7f93277fe640 1 -- 172.21.15.2:0/2354725797 learned_addr learned my addr 172.21.15.2:0/2354725797 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.552+0000 7f93257fa640 1 -- 172.21.15.2:0/440919668 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f9320014070 con 0x7f932819e660 2024-03-15T13:07:26.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.552+0000 7f93277fe640 1 --2- 172.21.15.2:0/2354725797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ee7250 0x557d86ee7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.552+0000 7f9327fff640 1 -- 172.21.15.2:0/2354725797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ee7250 msgr2=0x557d86ee7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.552+0000 7f9327fff640 1 --2- 172.21.15.2:0/2354725797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ee7250 0x557d86ee7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.552+0000 7f9327fff640 1 -- 172.21.15.2:0/2354725797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ef0750 msgr2=0x557d86ef2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.552+0000 7f9327fff640 1 --2- 172.21.15.2:0/2354725797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ef0750 0x557d86ef2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.552+0000 7f9327fff640 1 -- 172.21.15.2:0/2354725797 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d86e11060 con 0x557d86ee7b70 2024-03-15T13:07:26.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.552+0000 7f93277fe640 1 --2- 172.21.15.2:0/2354725797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ef0750 0x557d86ef2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:26.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.552+0000 7f932ce25640 1 --2- 172.21.15.2:0/440919668 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f93100a1ba0 0x7f93100a4060 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f93200cb540 tx=0x7f932006cb70 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.552+0000 7f9327fff640 1 --2- 172.21.15.2:0/2354725797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7b70 0x557d86ef0210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f931c009c00 tx=0x7f931c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92de31ba27b0384 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.552+0000 7f930e7fc640 1 -- 172.21.15.2:0/2354725797 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c004230 con 0x557d86ee7b70 2024-03-15T13:07:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.552+0000 7f930e7fc640 1 -- 172.21.15.2:0/2354725797 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f931c009e70 con 0x557d86ee7b70 2024-03-15T13:07:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.552+0000 7f930e7fc640 1 -- 172.21.15.2:0/2354725797 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c004c60 con 0x557d86ee7b70 2024-03-15T13:07:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.553+0000 7f933146d740 1 -- 172.21.15.2:0/2354725797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7b70 msgr2=0x557d86ef0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.553+0000 7f933146d740 1 --2- 172.21.15.2:0/2354725797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7b70 0x557d86ef0210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f931c009c00 tx=0x7f931c011810 comp rx=0 tx=0).stop 2024-03-15T13:07:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.553+0000 7f933146d740 1 -- 172.21.15.2:0/2354725797 shutdown_connections 2024-03-15T13:07:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.553+0000 7f933146d740 1 --2- 172.21.15.2:0/2354725797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ef0750 0x557d86ef2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.553+0000 7f933146d740 1 --2- 172.21.15.2:0/2354725797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7b70 0x557d86ef0210 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.553+0000 7f933146d740 1 --2- 172.21.15.2:0/2354725797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ee7250 0x557d86ee7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.553+0000 7f933146d740 1 -- 172.21.15.2:0/2354725797 >> 172.21.15.2:0/2354725797 conn(0x557d86ee1980 msgr2=0x557d86ee1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.553+0000 7f933146d740 1 -- 172.21.15.2:0/2354725797 shutdown_connections 2024-03-15T13:07:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.553+0000 7f933146d740 1 -- 172.21.15.2:0/2354725797 wait complete. 2024-03-15T13:07:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.553+0000 7f933146d740 1 -- start start 2024-03-15T13:07:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f933146d740 1 Processor -- start 2024-03-15T13:07:26.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f933146d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7250 0x557d86ef0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f933146d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7b70 0x557d86eed4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f933146d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ef0750 0x557d86eedae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f933146d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d86e4f020 con 0x557d86ee7250 2024-03-15T13:07:26.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f933146d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d86b22430 con 0x557d86ee7b70 2024-03-15T13:07:26.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f933146d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d86adee00 con 0x557d86ef0750 2024-03-15T13:07:26.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f933146d740 10 client.0 _ll_register_callbacks cb 0x557d86df6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:26.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f93277fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7250 0x557d86ef0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f93277fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7250 0x557d86ef0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49498/0 (socket says 172.21.15.2:49498) 2024-03-15T13:07:26.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f93277fe640 1 -- 172.21.15.2:0/25621124 learned_addr learned my addr 172.21.15.2:0/25621124 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f9327fff640 1 --2- 172.21.15.2:0/25621124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7b70 0x557d86eed4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f93277fe640 1 --2- 172.21.15.2:0/25621124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ef0750 0x557d86eedae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f93277fe640 1 -- 172.21.15.2:0/25621124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ef0750 msgr2=0x557d86eedae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f93277fe640 1 --2- 172.21.15.2:0/25621124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ef0750 0x557d86eedae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f93277fe640 1 -- 172.21.15.2:0/25621124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7b70 msgr2=0x557d86eed4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f93277fe640 1 --2- 172.21.15.2:0/25621124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7b70 0x557d86eed4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f93277fe640 1 -- 172.21.15.2:0/25621124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c009830 con 0x557d86ee7250 2024-03-15T13:07:26.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f93277fe640 1 --2- 172.21.15.2:0/25621124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7250 0x557d86ef0140 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9318009230 tx=0x7f931802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eda7e6879d9ce12 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.554+0000 7f93077fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:26.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.555+0000 7f93077fe640 1 -- 172.21.15.2:0/25621124 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318004040 con 0x557d86ee7250 2024-03-15T13:07:26.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.555+0000 7f93077fe640 1 -- 172.21.15.2:0/25621124 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f93180041a0 con 0x557d86ee7250 2024-03-15T13:07:26.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.555+0000 7f933146d740 1 -- 172.21.15.2:0/25621124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d86b4e7e0 con 0x557d86ee7250 2024-03-15T13:07:26.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.555+0000 7f93077fe640 1 -- 172.21.15.2:0/25621124 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9318004360 con 0x557d86ee7250 2024-03-15T13:07:26.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.555+0000 7f933146d740 10 client.18147 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:26.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.555+0000 7f933146d740 1 -- 172.21.15.2:0/25621124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x557d86e4d6a0 con 0x557d86ee7250 2024-03-15T13:07:26.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.555+0000 7f93077fe640 1 -- 172.21.15.2:0/25621124 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x557d86b4e7e0 con 0x557d86ee7250 2024-03-15T13:07:26.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.555+0000 7f93077fe640 1 -- 172.21.15.2:0/25621124 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f931804d070 con 0x557d86ee7250 2024-03-15T13:07:26.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.555+0000 7f933146d740 10 client.18147 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:26.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.555+0000 7f933146d740 10 client.18147 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:07:26.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.555+0000 7f933146d740 20 client.18147 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:26.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.555+0000 7f933146d740 1 --2- 172.21.15.2:0/25621124 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557d86f00800 0x557d86f02bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.555+0000 7f933146d740 4 client.18147 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:07:26.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.555+0000 7f933146d740 1 -- 172.21.15.2:0/25621124 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d86b8b720 con 0x557d86f00800 2024-03-15T13:07:26.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.555+0000 7f9327fff640 1 --2- 172.21.15.2:0/25621124 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557d86f00800 0x557d86f02bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.556+0000 7f9327fff640 1 --2- 172.21.15.2:0/25621124 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557d86f00800 0x557d86f02bf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3e9cd6005b03313 server_cookie=ce1c2d9bff7f6700 in_seq=0 out_seq=0 2024-03-15T13:07:26.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.556+0000 7f93077fe640 10 client.18147 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:07:26.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.557+0000 7f93077fe640 1 -- 172.21.15.2:0/25621124 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557d86b8b720 con 0x557d86f00800 2024-03-15T13:07:26.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.557+0000 7f93077fe640 10 client.18147 handle_command_reply: tid=0 2024-03-15T13:07:26.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.557+0000 7f933146d740 1 client.18147 shutdown 2024-03-15T13:07:26.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.557+0000 7f933146d740 1 -- 172.21.15.2:0/25621124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7250 msgr2=0x557d86ef0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.557+0000 7f933146d740 1 --2- 172.21.15.2:0/25621124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7250 0x557d86ef0140 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9318009230 tx=0x7f931802bc30 comp rx=0 tx=0).stop 2024-03-15T13:07:26.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.557+0000 7f933146d740 1 -- 172.21.15.2:0/25621124 shutdown_connections 2024-03-15T13:07:26.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.557+0000 7f933146d740 1 --2- 172.21.15.2:0/25621124 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557d86f00800 0x557d86f02bf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.557+0000 7f933146d740 1 --2- 172.21.15.2:0/25621124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ef0750 0x557d86eedae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.557+0000 7f933146d740 1 --2- 172.21.15.2:0/25621124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7b70 0x557d86eed4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.557+0000 7f933146d740 1 --2- 172.21.15.2:0/25621124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7250 0x557d86ef0140 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.557+0000 7f933146d740 1 -- 172.21.15.2:0/25621124 >> 172.21.15.2:0/25621124 conn(0x557d86ee1980 msgr2=0x557d86ee36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.557+0000 7f93077fe640 0 client.18147 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:07:26.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.557+0000 7f933146d740 1 -- 172.21.15.2:0/25621124 shutdown_connections 2024-03-15T13:07:26.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.557+0000 7f933146d740 1 -- 172.21.15.2:0/25621124 wait complete. 2024-03-15T13:07:26.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.557+0000 7f933146d740 20 client.18147 trim_cache size 0 max 16384 2024-03-15T13:07:26.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.569+0000 7f933146d740 1 Processor -- start 2024-03-15T13:07:26.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.569+0000 7f933146d740 1 -- start start 2024-03-15T13:07:26.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.569+0000 7f933146d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7250 0x557d86ee0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.569+0000 7f933146d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7b70 0x557d86eefc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f933146d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ef0750 0x557d86ef0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f933146d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d86efcdb0 con 0x557d86ee7b70 2024-03-15T13:07:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f933146d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d86dd4130 con 0x557d86ee7250 2024-03-15T13:07:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f933146d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d86b4e7e0 con 0x557d86ef0750 2024-03-15T13:07:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f9327fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7b70 0x557d86eefc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f9327fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7b70 0x557d86eefc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49504/0 (socket says 172.21.15.2:49504) 2024-03-15T13:07:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f9327fff640 1 -- 172.21.15.2:0/2360010833 learned_addr learned my addr 172.21.15.2:0/2360010833 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f93277fe640 1 --2- 172.21.15.2:0/2360010833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7250 0x557d86ee0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f93277fe640 1 --2- 172.21.15.2:0/2360010833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ef0750 0x557d86ef0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f9327fff640 1 -- 172.21.15.2:0/2360010833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ef0750 msgr2=0x557d86ef0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f9327fff640 1 --2- 172.21.15.2:0/2360010833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ef0750 0x557d86ef0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f9327fff640 1 -- 172.21.15.2:0/2360010833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7250 msgr2=0x557d86ee0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f9327fff640 1 --2- 172.21.15.2:0/2360010833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7250 0x557d86ee0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f9327fff640 1 -- 172.21.15.2:0/2360010833 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d86de9c20 con 0x557d86ee7b70 2024-03-15T13:07:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f93277fe640 1 --2- 172.21.15.2:0/2360010833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7250 0x557d86ee0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:07:26.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f9327fff640 1 --2- 172.21.15.2:0/2360010833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7b70 0x557d86eefc00 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f931c009710 tx=0x7f931c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcb8a2b908b876b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f930dffb640 1 -- 172.21.15.2:0/2360010833 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c004f20 con 0x557d86ee7b70 2024-03-15T13:07:26.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f930dffb640 1 -- 172.21.15.2:0/2360010833 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f931c004680 con 0x557d86ee7b70 2024-03-15T13:07:26.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.570+0000 7f930dffb640 1 -- 172.21.15.2:0/2360010833 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c02c970 con 0x557d86ee7b70 2024-03-15T13:07:26.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 -- 172.21.15.2:0/2360010833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7b70 msgr2=0x557d86eefc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 --2- 172.21.15.2:0/2360010833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7b70 0x557d86eefc00 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f931c009710 tx=0x7f931c0096a0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 -- 172.21.15.2:0/2360010833 shutdown_connections 2024-03-15T13:07:26.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 --2- 172.21.15.2:0/2360010833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ef0750 0x557d86ef0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 --2- 172.21.15.2:0/2360010833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7b70 0x557d86eefc00 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 --2- 172.21.15.2:0/2360010833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ee7250 0x557d86ee0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 -- 172.21.15.2:0/2360010833 >> 172.21.15.2:0/2360010833 conn(0x557d86ee1980 msgr2=0x557d86ee02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 -- 172.21.15.2:0/2360010833 shutdown_connections 2024-03-15T13:07:26.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 -- 172.21.15.2:0/2360010833 wait complete. 2024-03-15T13:07:26.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 -- start start 2024-03-15T13:07:26.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 Processor -- start 2024-03-15T13:07:26.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7250 0x557d86ef4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ee7b70 0x557d86ef45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ef0750 0x557d86f5b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d86ef4d40 con 0x557d86ee7250 2024-03-15T13:07:26.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d86f5b5c0 con 0x557d86ef0750 2024-03-15T13:07:26.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.571+0000 7f933146d740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d86f5b780 con 0x557d86ee7b70 2024-03-15T13:07:26.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f933146d740 10 client.0 _ll_register_callbacks cb 0x7f9318004bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:26.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f93277fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7250 0x557d86ef4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f93277fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7250 0x557d86ef4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49520/0 (socket says 172.21.15.2:49520) 2024-03-15T13:07:26.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f93277fe640 1 -- 172.21.15.2:0/588967251 learned_addr learned my addr 172.21.15.2:0/588967251 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f9327fff640 1 --2- 172.21.15.2:0/588967251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ee7b70 0x557d86ef45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f93277fe640 1 --2- 172.21.15.2:0/588967251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ef0750 0x557d86f5b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f93277fe640 1 -- 172.21.15.2:0/588967251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ee7b70 msgr2=0x557d86ef45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f93277fe640 1 --2- 172.21.15.2:0/588967251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ee7b70 0x557d86ef45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f93277fe640 1 -- 172.21.15.2:0/588967251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ef0750 msgr2=0x557d86f5b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f93277fe640 1 --2- 172.21.15.2:0/588967251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ef0750 0x557d86f5b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f93277fe640 1 -- 172.21.15.2:0/588967251 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c01d040 con 0x557d86ee7250 2024-03-15T13:07:26.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f93277fe640 1 --2- 172.21.15.2:0/588967251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7250 0x557d86ef4090 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9318009230 tx=0x7f931802c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cf5fea2407c1be server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f93077fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:26.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f93077fe640 1 -- 172.21.15.2:0/588967251 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318004240 con 0x557d86ee7250 2024-03-15T13:07:26.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f93077fe640 1 -- 172.21.15.2:0/588967251 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f93180043a0 con 0x557d86ee7250 2024-03-15T13:07:26.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f933146d740 1 -- 172.21.15.2:0/588967251 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f931c009e70 con 0x557d86ee7250 2024-03-15T13:07:26.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f93077fe640 1 -- 172.21.15.2:0/588967251 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93180035f0 con 0x557d86ee7250 2024-03-15T13:07:26.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f933146d740 10 client.18153 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:26.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f933146d740 1 -- 172.21.15.2:0/588967251 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x557d86e0f8c0 con 0x557d86ee7250 2024-03-15T13:07:26.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.572+0000 7f93077fe640 1 -- 172.21.15.2:0/588967251 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f931c009e70 con 0x557d86ee7250 2024-03-15T13:07:26.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.573+0000 7f93077fe640 1 -- 172.21.15.2:0/588967251 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f931804d070 con 0x557d86ee7250 2024-03-15T13:07:26.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.573+0000 7f933146d740 10 client.18153 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:26.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.573+0000 7f933146d740 10 client.18153 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:07:26.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.573+0000 7f933146d740 20 client.18153 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:26.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.573+0000 7f933146d740 1 --2- 172.21.15.2:0/588967251 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557d86f5cd60 0x557d86f5f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.573+0000 7f933146d740 4 client.18153 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:07:26.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.573+0000 7f933146d740 1 -- 172.21.15.2:0/588967251 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d86bda890 con 0x557d86f5cd60 2024-03-15T13:07:26.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.573+0000 7f9327fff640 1 --2- 172.21.15.2:0/588967251 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557d86f5cd60 0x557d86f5f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.573+0000 7f93077fe640 10 client.18153 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:07:26.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.573+0000 7f9327fff640 1 --2- 172.21.15.2:0/588967251 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557d86f5cd60 0x557d86f5f150 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d25847d6fa4b3166 server_cookie=bced1312ed4d0b79 in_seq=0 out_seq=0 2024-03-15T13:07:26.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.573+0000 7f93077fe640 1 -- 172.21.15.2:0/588967251 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d86bda890 con 0x557d86f5cd60 2024-03-15T13:07:26.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.573+0000 7f93077fe640 10 client.18153 handle_command_reply: tid=0 2024-03-15T13:07:26.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.573+0000 7f933146d740 1 client.18153 shutdown 2024-03-15T13:07:26.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f933146d740 1 -- 172.21.15.2:0/588967251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7250 msgr2=0x557d86ef4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f933146d740 1 --2- 172.21.15.2:0/588967251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7250 0x557d86ef4090 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9318009230 tx=0x7f931802c6d0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f933146d740 1 -- 172.21.15.2:0/588967251 shutdown_connections 2024-03-15T13:07:26.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f933146d740 1 --2- 172.21.15.2:0/588967251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d86ef0750 0x557d86f5b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f933146d740 1 --2- 172.21.15.2:0/588967251 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x557d86f5cd60 0x557d86f5f150 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f933146d740 1 --2- 172.21.15.2:0/588967251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557d86ee7b70 0x557d86ef45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f933146d740 1 --2- 172.21.15.2:0/588967251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557d86ee7250 0x557d86ef4090 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f933146d740 1 -- 172.21.15.2:0/588967251 >> 172.21.15.2:0/588967251 conn(0x557d86ee1980 msgr2=0x557d86ee0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f93077fe640 0 client.18153 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:07:26.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f933146d740 1 -- 172.21.15.2:0/588967251 shutdown_connections 2024-03-15T13:07:26.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f933146d740 1 -- 172.21.15.2:0/588967251 wait complete. 2024-03-15T13:07:26.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f933146d740 20 client.18153 trim_cache size 0 max 16384 2024-03-15T13:07:26.565 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:07:26.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f932e888640 1 -- 172.21.15.2:0/440919668 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f93100a1ba0 msgr2=0x7f93100a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f932e888640 1 --2- 172.21.15.2:0/440919668 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f93100a1ba0 0x7f93100a4060 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f93200cb540 tx=0x7f932006cb70 comp rx=0 tx=0).stop 2024-03-15T13:07:26.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f932e888640 1 -- 172.21.15.2:0/440919668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f932819e660 msgr2=0x7f932819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f932e888640 1 --2- 172.21.15.2:0/440919668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f932819e660 0x7f932819eb10 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f932000eb10 tx=0x7f932000c2d0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f932e888640 1 -- 172.21.15.2:0/440919668 shutdown_connections 2024-03-15T13:07:26.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f932e888640 1 --2- 172.21.15.2:0/440919668 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f93100a1ba0 0x7f93100a4060 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f932e888640 1 --2- 172.21.15.2:0/440919668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f932819e660 0x7f932819eb10 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f932e888640 1 --2- 172.21.15.2:0/440919668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93281056a0 0x7f932819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f932e888640 1 --2- 172.21.15.2:0/440919668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93281005d0 0x7f9328199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f932e888640 1 -- 172.21.15.2:0/440919668 >> 172.21.15.2:0/440919668 conn(0x7f93280774f0 msgr2=0x7f93280686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f932e888640 1 -- 172.21.15.2:0/440919668 shutdown_connections 2024-03-15T13:07:26.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.574+0000 7f932e888640 1 -- 172.21.15.2:0/440919668 wait complete. 2024-03-15T13:07:26.569 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:07:26.570 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:07:26.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.689+0000 7fe7fd13e640 1 -- 172.21.15.2:0/1879028245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7f810bb30 msgr2=0x7fe7f810dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.689+0000 7fe7fd13e640 1 --2- 172.21.15.2:0/1879028245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7f810bb30 0x7fe7f810dfa0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009900 tx=0x7fe7f002bcd0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.689+0000 7fe7fd13e640 1 -- 172.21.15.2:0/1879028245 shutdown_connections 2024-03-15T13:07:26.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.689+0000 7fe7fd13e640 1 --2- 172.21.15.2:0/1879028245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7f810bb30 0x7fe7f810dfa0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.689+0000 7fe7fd13e640 1 --2- 172.21.15.2:0/1879028245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7f8103080 0x7fe7f810b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.689+0000 7fe7fd13e640 1 --2- 172.21.15.2:0/1879028245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7f8102760 0x7fe7f8102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.689+0000 7fe7fd13e640 1 -- 172.21.15.2:0/1879028245 >> 172.21.15.2:0/1879028245 conn(0x7fe7f8077550 msgr2=0x7fe7f8077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.689+0000 7fe7fd13e640 1 -- 172.21.15.2:0/1879028245 shutdown_connections 2024-03-15T13:07:26.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.689+0000 7fe7fd13e640 1 -- 172.21.15.2:0/1879028245 wait complete. 2024-03-15T13:07:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.690+0000 7fe7fd13e640 1 Processor -- start 2024-03-15T13:07:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.690+0000 7fe7fd13e640 1 -- start start 2024-03-15T13:07:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.690+0000 7fe7fd13e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7f8102760 0x7fe7f819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.690+0000 7fe7fd13e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7f8103080 0x7fe7f819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.690+0000 7fe7fd13e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7f810bb30 0x7fe7f8197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.690+0000 7fe7fd13e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f8197fc0 con 0x7fe7f8103080 2024-03-15T13:07:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.690+0000 7fe7fd13e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f8198130 con 0x7fe7f810bb30 2024-03-15T13:07:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.690+0000 7fe7fd13e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f81982f0 con 0x7fe7f8102760 2024-03-15T13:07:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.690+0000 7fe7f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7f8103080 0x7fe7f819dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.690+0000 7fe7f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7f8103080 0x7fe7f819dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49540/0 (socket says 172.21.15.2:49540) 2024-03-15T13:07:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.690+0000 7fe7f659c640 1 -- 172.21.15.2:0/2371962366 learned_addr learned my addr 172.21.15.2:0/2371962366 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.691+0000 7fe7f6d9d640 1 --2- 172.21.15.2:0/2371962366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7f8102760 0x7fe7f819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.691+0000 7fe7f759e640 1 --2- 172.21.15.2:0/2371962366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7f810bb30 0x7fe7f8197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.691+0000 7fe7f659c640 1 -- 172.21.15.2:0/2371962366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7f8102760 msgr2=0x7fe7f819d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.691+0000 7fe7f659c640 1 --2- 172.21.15.2:0/2371962366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7f8102760 0x7fe7f819d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.691+0000 7fe7f659c640 1 -- 172.21.15.2:0/2371962366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7f810bb30 msgr2=0x7fe7f8197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.691+0000 7fe7f659c640 1 --2- 172.21.15.2:0/2371962366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7f810bb30 0x7fe7f8197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.691+0000 7fe7f659c640 1 -- 172.21.15.2:0/2371962366 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e8009840 con 0x7fe7f8103080 2024-03-15T13:07:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.691+0000 7fe7f659c640 1 --2- 172.21.15.2:0/2371962366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7f8103080 0x7fe7f819dd00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000e9f0 tx=0x7fe7e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.691+0000 7fe7effff640 1 -- 172.21.15.2:0/2371962366 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0009720 con 0x7fe7f8103080 2024-03-15T13:07:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.691+0000 7fe7fd13e640 1 -- 172.21.15.2:0/2371962366 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f0009560 con 0x7fe7f8103080 2024-03-15T13:07:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.691+0000 7fe7effff640 1 -- 172.21.15.2:0/2371962366 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe7e000cd80 con 0x7fe7f8103080 2024-03-15T13:07:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.691+0000 7fe7effff640 1 -- 172.21.15.2:0/2371962366 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0010690 con 0x7fe7f8103080 2024-03-15T13:07:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.691+0000 7fe7fd13e640 1 -- 172.21.15.2:0/2371962366 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f81a4680 con 0x7fe7f8103080 2024-03-15T13:07:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.693+0000 7fe7effff640 1 -- 172.21.15.2:0/2371962366 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fe7e0004e90 con 0x7fe7f8103080 2024-03-15T13:07:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7ffdf6740 1 Processor -- start 2024-03-15T13:07:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7ffdf6740 1 -- start start 2024-03-15T13:07:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7effff640 1 --2- 172.21.15.2:0/2371962366 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe7e40a1ba0 0x7fe7e40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7f759e640 1 --2- 172.21.15.2:0/2371962366 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe7e40a1ba0 0x7fe7e40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7effff640 1 -- 172.21.15.2:0/2371962366 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe7e0014070 con 0x7fe7f8103080 2024-03-15T13:07:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7ffdf6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6e2250 0x55a6bb6e2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7ffdf6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6eb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7ffdf6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6eb750 0x55a6bb6edb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7ffdf6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bb69d960 con 0x55a6bb6e2b70 2024-03-15T13:07:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7ffdf6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bb651310 con 0x55a6bb6e2250 2024-03-15T13:07:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7ffdf6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bb64e840 con 0x55a6bb6eb750 2024-03-15T13:07:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6eb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7f6d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6e2250 0x55a6bb6e2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6eb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49546/0 (socket says 172.21.15.2:49546) 2024-03-15T13:07:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7f659c640 1 -- 172.21.15.2:0/3580083017 learned_addr learned my addr 172.21.15.2:0/3580083017 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.694+0000 7fe7f759e640 1 --2- 172.21.15.2:0/3580083017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6eb750 0x55a6bb6edb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.695+0000 7fe7f659c640 1 -- 172.21.15.2:0/3580083017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6eb750 msgr2=0x55a6bb6edb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.695+0000 7fe7f659c640 1 --2- 172.21.15.2:0/3580083017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6eb750 0x55a6bb6edb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.695+0000 7fe7f659c640 1 -- 172.21.15.2:0/3580083017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6e2250 msgr2=0x55a6bb6e2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.695+0000 7fe7f659c640 1 --2- 172.21.15.2:0/3580083017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6e2250 0x55a6bb6e2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.695+0000 7fe7f659c640 1 -- 172.21.15.2:0/3580083017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6bb60c060 con 0x55a6bb6e2b70 2024-03-15T13:07:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.695+0000 7fe7f6d9d640 1 --2- 172.21.15.2:0/3580083017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6e2250 0x55a6bb6e2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.695+0000 7fe7f759e640 1 --2- 172.21.15.2:0/2371962366 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe7e40a1ba0 0x7fe7e40a4060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0005ab0 tx=0x7fe7f00046c0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.695+0000 7fe7f659c640 1 --2- 172.21.15.2:0/3580083017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6eb210 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00a2960 tx=0x7fe7e009a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ca7f196196971a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.695+0000 7fe7ed7fa640 1 -- 172.21.15.2:0/3580083017 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e00b3210 con 0x55a6bb6e2b70 2024-03-15T13:07:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.695+0000 7fe7ed7fa640 1 -- 172.21.15.2:0/3580083017 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe7e009e840 con 0x55a6bb6e2b70 2024-03-15T13:07:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.695+0000 7fe7ed7fa640 1 -- 172.21.15.2:0/3580083017 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e00ba0d0 con 0x55a6bb6e2b70 2024-03-15T13:07:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3580083017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 msgr2=0x55a6bb6eb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/3580083017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6eb210 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00a2960 tx=0x7fe7e009a050 comp rx=0 tx=0).stop 2024-03-15T13:07:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3580083017 shutdown_connections 2024-03-15T13:07:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/3580083017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6eb750 0x55a6bb6edb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/3580083017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6eb210 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/3580083017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6e2250 0x55a6bb6e2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3580083017 >> 172.21.15.2:0/3580083017 conn(0x55a6bb6dc980 msgr2=0x55a6bb6dcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3580083017 shutdown_connections 2024-03-15T13:07:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3580083017 wait complete. 2024-03-15T13:07:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 -- start start 2024-03-15T13:07:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 Processor -- start 2024-03-15T13:07:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6e2250 0x55a6bb6eb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6e84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 0x55a6bb6e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bb64a020 con 0x55a6bb6e2b70 2024-03-15T13:07:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bb31d430 con 0x55a6bb6eb750 2024-03-15T13:07:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.696+0000 7fe7ffdf6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bb2d9e00 con 0x55a6bb6e2250 2024-03-15T13:07:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7ffdf6740 10 client.0 _ll_register_callbacks cb 0x55a6bb5f1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6e84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6e84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49560/0 (socket says 172.21.15.2:49560) 2024-03-15T13:07:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7f659c640 1 -- 172.21.15.2:0/1074620242 learned_addr learned my addr 172.21.15.2:0/1074620242 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7f6d9d640 1 --2- 172.21.15.2:0/1074620242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6e2250 0x55a6bb6eb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7f759e640 1 --2- 172.21.15.2:0/1074620242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 0x55a6bb6e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7f659c640 1 -- 172.21.15.2:0/1074620242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6e2250 msgr2=0x55a6bb6eb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7f659c640 1 --2- 172.21.15.2:0/1074620242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6e2250 0x55a6bb6eb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7f659c640 1 -- 172.21.15.2:0/1074620242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 msgr2=0x55a6bb6e8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7f659c640 1 --2- 172.21.15.2:0/1074620242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 0x55a6bb6e8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7f659c640 1 -- 172.21.15.2:0/1074620242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e0083e70 con 0x55a6bb6e2b70 2024-03-15T13:07:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7f659c640 1 --2- 172.21.15.2:0/1074620242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6e84d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0096b00 tx=0x7fe7e0004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c71de831deaec0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7de7fc640 1 -- 172.21.15.2:0/1074620242 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e00b3210 con 0x55a6bb6e2b70 2024-03-15T13:07:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7de7fc640 1 -- 172.21.15.2:0/1074620242 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe7e00ba2e0 con 0x55a6bb6e2b70 2024-03-15T13:07:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/1074620242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6bb3497e0 con 0x55a6bb6e2b70 2024-03-15T13:07:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7de7fc640 1 -- 172.21.15.2:0/1074620242 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7e0098790 con 0x55a6bb6e2b70 2024-03-15T13:07:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7ffdf6740 10 client.18165 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.697+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/1074620242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55a6bb6486a0 con 0x55a6bb6e2b70 2024-03-15T13:07:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.698+0000 7fe7de7fc640 1 -- 172.21.15.2:0/1074620242 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55a6bb3497e0 con 0x55a6bb6e2b70 2024-03-15T13:07:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.698+0000 7fe7de7fc640 1 -- 172.21.15.2:0/1074620242 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe7e001f060 con 0x55a6bb6e2b70 2024-03-15T13:07:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.698+0000 7fe7ffdf6740 10 client.18165 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.698+0000 7fe7ffdf6740 10 client.18165 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:07:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.698+0000 7fe7ffdf6740 20 client.18165 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.698+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/1074620242 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55a6bb6fb5b0 0x55a6bb6fd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.698+0000 7fe7ffdf6740 4 client.18165 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:07:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.698+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/1074620242 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6bb386720 con 0x55a6bb6fb5b0 2024-03-15T13:07:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.698+0000 7fe7f6d9d640 1 --2- 172.21.15.2:0/1074620242 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55a6bb6fb5b0 0x55a6bb6fd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.698+0000 7fe7f6d9d640 1 --2- 172.21.15.2:0/1074620242 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55a6bb6fb5b0 0x55a6bb6fd9d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a78b79d8cdadbb21 server_cookie=12e4222d336f3d09 in_seq=0 out_seq=0 2024-03-15T13:07:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.698+0000 7fe7de7fc640 10 client.18165 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:07:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.699+0000 7fe7de7fc640 1 -- 172.21.15.2:0/1074620242 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a6bb386720 con 0x55a6bb6fb5b0 2024-03-15T13:07:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.699+0000 7fe7de7fc640 10 client.18165 handle_command_reply: tid=0 2024-03-15T13:07:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.699+0000 7fe7ffdf6740 1 client.18165 shutdown 2024-03-15T13:07:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.699+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/1074620242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 msgr2=0x55a6bb6e84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.699+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/1074620242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6e84d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0096b00 tx=0x7fe7e0004920 comp rx=0 tx=0).stop 2024-03-15T13:07:26.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.699+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/1074620242 shutdown_connections 2024-03-15T13:07:26.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.699+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/1074620242 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55a6bb6fb5b0 0x55a6bb6fd9d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.699+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/1074620242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 0x55a6bb6e8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.699+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/1074620242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6e84d0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.699+0000 7fe7de7fc640 0 client.18165 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:07:26.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.699+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/1074620242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6e2250 0x55a6bb6eb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.699+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/1074620242 >> 172.21.15.2:0/1074620242 conn(0x55a6bb6dc980 msgr2=0x55a6bb6de6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.700+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/1074620242 shutdown_connections 2024-03-15T13:07:26.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.700+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/1074620242 wait complete. 2024-03-15T13:07:26.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.700+0000 7fe7ffdf6740 20 client.18165 trim_cache size 0 max 16384 2024-03-15T13:07:26.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.712+0000 7fe7ffdf6740 1 Processor -- start 2024-03-15T13:07:26.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.712+0000 7fe7ffdf6740 1 -- start start 2024-03-15T13:07:26.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.712+0000 7fe7ffdf6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6e2250 0x55a6bb6eac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.712+0000 7fe7ffdf6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6eb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.712+0000 7fe7ffdf6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 0x55a6bb6f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.712+0000 7fe7ffdf6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bb5cf130 con 0x55a6bb6e2b70 2024-03-15T13:07:26.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.712+0000 7fe7ffdf6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bb6e40a0 con 0x55a6bb6eb750 2024-03-15T13:07:26.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.712+0000 7fe7ffdf6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bb6dc6e0 con 0x55a6bb6e2250 2024-03-15T13:07:26.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.712+0000 7fe7f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6eb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.712+0000 7fe7f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6eb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49572/0 (socket says 172.21.15.2:49572) 2024-03-15T13:07:26.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.712+0000 7fe7f659c640 1 -- 172.21.15.2:0/3516850992 learned_addr learned my addr 172.21.15.2:0/3516850992 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.712+0000 7fe7f6d9d640 1 --2- 172.21.15.2:0/3516850992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6e2250 0x55a6bb6eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.712+0000 7fe7f759e640 1 --2- 172.21.15.2:0/3516850992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 0x55a6bb6f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.713+0000 7fe7f659c640 1 -- 172.21.15.2:0/3516850992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6e2250 msgr2=0x55a6bb6eac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.713+0000 7fe7f659c640 1 --2- 172.21.15.2:0/3516850992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6e2250 0x55a6bb6eac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.713+0000 7fe7f659c640 1 -- 172.21.15.2:0/3516850992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 msgr2=0x55a6bb6f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.713+0000 7fe7f659c640 1 --2- 172.21.15.2:0/3516850992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 0x55a6bb6f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.713+0000 7fe7f659c640 1 -- 172.21.15.2:0/3516850992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6bb5e4c20 con 0x55a6bb6e2b70 2024-03-15T13:07:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.713+0000 7fe7f659c640 1 --2- 172.21.15.2:0/3516850992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6eb140 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe7e009a090 tx=0x7fe7e009a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d317a6221eedc1b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.713+0000 7fe7ecff9640 1 -- 172.21.15.2:0/3516850992 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e009adf0 con 0x55a6bb6e2b70 2024-03-15T13:07:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.713+0000 7fe7ecff9640 1 -- 172.21.15.2:0/3516850992 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe7e00b3210 con 0x55a6bb6e2b70 2024-03-15T13:07:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.713+0000 7fe7ecff9640 1 -- 172.21.15.2:0/3516850992 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e00967b0 con 0x55a6bb6e2b70 2024-03-15T13:07:26.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3516850992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 msgr2=0x55a6bb6eb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/3516850992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6eb140 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe7e009a090 tx=0x7fe7e009a870 comp rx=0 tx=0).stop 2024-03-15T13:07:26.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3516850992 shutdown_connections 2024-03-15T13:07:26.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/3516850992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 0x55a6bb6f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/3516850992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6eb140 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/3516850992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6e2250 0x55a6bb6eac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3516850992 >> 172.21.15.2:0/3516850992 conn(0x55a6bb6dc980 msgr2=0x55a6bb6dbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3516850992 shutdown_connections 2024-03-15T13:07:26.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3516850992 wait complete. 2024-03-15T13:07:26.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 -- start start 2024-03-15T13:07:26.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 Processor -- start 2024-03-15T13:07:26.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6e2250 0x55a6bb6f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 0x55a6bb6ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bb6ef760 con 0x55a6bb6e2b70 2024-03-15T13:07:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bb6ef8d0 con 0x55a6bb6eb750 2024-03-15T13:07:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bb6efa90 con 0x55a6bb6e2250 2024-03-15T13:07:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7ffdf6740 10 client.0 _ll_register_callbacks cb 0x7fe7e00b1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6f31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49580/0 (socket says 172.21.15.2:49580) 2024-03-15T13:07:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7f759e640 1 --2- 172.21.15.2:0/3458358525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 0x55a6bb6ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7f659c640 1 -- 172.21.15.2:0/3458358525 learned_addr learned my addr 172.21.15.2:0/3458358525 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.714+0000 7fe7f6d9d640 1 --2- 172.21.15.2:0/3458358525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6e2250 0x55a6bb6f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7f659c640 1 -- 172.21.15.2:0/3458358525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6e2250 msgr2=0x55a6bb6f2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7f659c640 1 --2- 172.21.15.2:0/3458358525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6e2250 0x55a6bb6f2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7f659c640 1 -- 172.21.15.2:0/3458358525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 msgr2=0x55a6bb6ef120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7f659c640 1 --2- 172.21.15.2:0/3458358525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 0x55a6bb6ef120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7f659c640 1 -- 172.21.15.2:0/3458358525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6bb6486a0 con 0x55a6bb6e2b70 2024-03-15T13:07:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7f759e640 1 --2- 172.21.15.2:0/3458358525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 0x55a6bb6ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7f659c640 1 --2- 172.21.15.2:0/3458358525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6f31c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000c6a0 tx=0x7fe7e00962d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4051fc148d4db7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7de7fc640 1 -- 172.21.15.2:0/3458358525 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e009adf0 con 0x55a6bb6e2b70 2024-03-15T13:07:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7de7fc640 1 -- 172.21.15.2:0/3458358525 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe7e00ba2e0 con 0x55a6bb6e2b70 2024-03-15T13:07:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3458358525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7e009e840 con 0x55a6bb6e2b70 2024-03-15T13:07:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7de7fc640 1 -- 172.21.15.2:0/3458358525 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7e0085350 con 0x55a6bb6e2b70 2024-03-15T13:07:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7ffdf6740 10 client.18171 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3458358525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55a6bb60a8c0 con 0x55a6bb6e2b70 2024-03-15T13:07:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7de7fc640 1 -- 172.21.15.2:0/3458358525 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe7e009e840 con 0x55a6bb6e2b70 2024-03-15T13:07:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7de7fc640 1 -- 172.21.15.2:0/3458358525 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe7e001f030 con 0x55a6bb6e2b70 2024-03-15T13:07:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7ffdf6740 10 client.18171 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.715+0000 7fe7ffdf6740 10 client.18171 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:07:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7ffdf6740 20 client.18171 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/3458358525 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55a6bb75bea0 0x55a6bb75e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7ffdf6740 4 client.18171 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:07:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3458358525 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a6bb3d5890 con 0x55a6bb75bea0 2024-03-15T13:07:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7f6d9d640 1 --2- 172.21.15.2:0/3458358525 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55a6bb75bea0 0x55a6bb75e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7f6d9d640 1 --2- 172.21.15.2:0/3458358525 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55a6bb75bea0 0x55a6bb75e290 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5fde6b568774f04 server_cookie=97243fda3d29ab3d in_seq=0 out_seq=0 2024-03-15T13:07:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7de7fc640 10 client.18171 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:07:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7de7fc640 1 -- 172.21.15.2:0/3458358525 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a6bb3d5890 con 0x55a6bb75bea0 2024-03-15T13:07:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7de7fc640 10 client.18171 handle_command_reply: tid=0 2024-03-15T13:07:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7ffdf6740 1 client.18171 shutdown 2024-03-15T13:07:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3458358525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 msgr2=0x55a6bb6f31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/3458358525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6f31c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000c6a0 tx=0x7fe7e00962d0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3458358525 shutdown_connections 2024-03-15T13:07:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/3458358525 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55a6bb75bea0 0x55a6bb75e290 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/3458358525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a6bb6eb750 0x55a6bb6ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/3458358525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a6bb6e2b70 0x55a6bb6f31c0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7de7fc640 0 client.18171 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:07:26.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7ffdf6740 1 --2- 172.21.15.2:0/3458358525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a6bb6e2250 0x55a6bb6f2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3458358525 >> 172.21.15.2:0/3458358525 conn(0x55a6bb6dc980 msgr2=0x55a6bb6db2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3458358525 shutdown_connections 2024-03-15T13:07:26.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.716+0000 7fe7ffdf6740 1 -- 172.21.15.2:0/3458358525 wait complete. 2024-03-15T13:07:26.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.717+0000 7fe7ffdf6740 20 client.18171 trim_cache size 0 max 16384 2024-03-15T13:07:26.708 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:07:26.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.717+0000 7fe7fd13e640 1 -- 172.21.15.2:0/2371962366 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe7e40a1ba0 msgr2=0x7fe7e40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.717+0000 7fe7fd13e640 1 --2- 172.21.15.2:0/2371962366 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe7e40a1ba0 0x7fe7e40a4060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0005ab0 tx=0x7fe7f00046c0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.717+0000 7fe7fd13e640 1 -- 172.21.15.2:0/2371962366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7f8103080 msgr2=0x7fe7f819dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.717+0000 7fe7fd13e640 1 --2- 172.21.15.2:0/2371962366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7f8103080 0x7fe7f819dd00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000e9f0 tx=0x7fe7e000edf0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.717+0000 7fe7fd13e640 1 -- 172.21.15.2:0/2371962366 shutdown_connections 2024-03-15T13:07:26.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.717+0000 7fe7fd13e640 1 --2- 172.21.15.2:0/2371962366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7f810bb30 0x7fe7f8197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.717+0000 7fe7fd13e640 1 --2- 172.21.15.2:0/2371962366 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe7e40a1ba0 0x7fe7e40a4060 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.717+0000 7fe7fd13e640 1 --2- 172.21.15.2:0/2371962366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7f8103080 0x7fe7f819dd00 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.717+0000 7fe7fd13e640 1 --2- 172.21.15.2:0/2371962366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7f8102760 0x7fe7f819d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.717+0000 7fe7fd13e640 1 -- 172.21.15.2:0/2371962366 >> 172.21.15.2:0/2371962366 conn(0x7fe7f8077550 msgr2=0x7fe7f80fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.717+0000 7fe7fd13e640 1 -- 172.21.15.2:0/2371962366 shutdown_connections 2024-03-15T13:07:26.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.717+0000 7fe7fd13e640 1 -- 172.21.15.2:0/2371962366 wait complete. 2024-03-15T13:07:26.711 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:26 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/4169352840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:07:26.712 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:07:26.712 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:07:26.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:26 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/4169352840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:07:26.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.832+0000 7fedc5e64640 1 -- 172.21.15.2:0/2165076986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc0107560 msgr2=0x7fedc0107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.832+0000 7fedc5e64640 1 --2- 172.21.15.2:0/2165076986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc0107560 0x7fedc0107940 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009900 tx=0x7fedb002bc30 comp rx=0 tx=0).stop 2024-03-15T13:07:26.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.832+0000 7fedc5e64640 1 -- 172.21.15.2:0/2165076986 shutdown_connections 2024-03-15T13:07:26.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.832+0000 7fedc5e64640 1 --2- 172.21.15.2:0/2165076986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedc0101e60 0x7fedc010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.832+0000 7fedc5e64640 1 --2- 172.21.15.2:0/2165076986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedc01014a0 0x7fedc0101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.832+0000 7fedc5e64640 1 --2- 172.21.15.2:0/2165076986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc0107560 0x7fedc0107940 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.832+0000 7fedc5e64640 1 -- 172.21.15.2:0/2165076986 >> 172.21.15.2:0/2165076986 conn(0x7fedc00788f0 msgr2=0x7fedc0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.832+0000 7fedc5e64640 1 -- 172.21.15.2:0/2165076986 shutdown_connections 2024-03-15T13:07:26.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.832+0000 7fedc5e64640 1 -- 172.21.15.2:0/2165076986 wait complete. 2024-03-15T13:07:26.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.833+0000 7fedc5e64640 1 Processor -- start 2024-03-15T13:07:26.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.833+0000 7fedc5e64640 1 -- start start 2024-03-15T13:07:26.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.833+0000 7fedc5e64640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedc01014a0 0x7fedc019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.833+0000 7fedc5e64640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc0101e60 0x7fedc019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.833+0000 7fedc5e64640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedc0107560 0x7fedc0197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.833+0000 7fedc5e64640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc0197ec0 con 0x7fedc0101e60 2024-03-15T13:07:26.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.833+0000 7fedc5e64640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc0198030 con 0x7fedc0107560 2024-03-15T13:07:26.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.833+0000 7fedc5e64640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc01981f0 con 0x7fedc01014a0 2024-03-15T13:07:26.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.833+0000 7fedbeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc0101e60 0x7fedc019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.834+0000 7fedbffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedc0107560 0x7fedc0197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.833+0000 7fedbeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc0101e60 0x7fedc019dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49592/0 (socket says 172.21.15.2:49592) 2024-03-15T13:07:26.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.834+0000 7fedbeffd640 1 -- 172.21.15.2:0/583949987 learned_addr learned my addr 172.21.15.2:0/583949987 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.834+0000 7fedbf7fe640 1 --2- 172.21.15.2:0/583949987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedc01014a0 0x7fedc019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.834+0000 7fedbeffd640 1 -- 172.21.15.2:0/583949987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedc01014a0 msgr2=0x7fedc019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.834+0000 7fedbeffd640 1 --2- 172.21.15.2:0/583949987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedc01014a0 0x7fedc019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.834+0000 7fedbeffd640 1 -- 172.21.15.2:0/583949987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedc0107560 msgr2=0x7fedc0197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.834+0000 7fedbeffd640 1 --2- 172.21.15.2:0/583949987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedc0107560 0x7fedc0197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.834+0000 7fedbeffd640 1 -- 172.21.15.2:0/583949987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb0009560 con 0x7fedc0101e60 2024-03-15T13:07:26.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.834+0000 7fedbeffd640 1 --2- 172.21.15.2:0/583949987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc0101e60 0x7fedc019dc00 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fedb400e9f0 tx=0x7fedb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.834+0000 7fedbcff9640 1 -- 172.21.15.2:0/583949987 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4009700 con 0x7fedc0101e60 2024-03-15T13:07:26.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.834+0000 7fedbcff9640 1 -- 172.21.15.2:0/583949987 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fedb400cd80 con 0x7fedc0101e60 2024-03-15T13:07:26.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.834+0000 7fedc5e64640 1 -- 172.21.15.2:0/583949987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc01984d0 con 0x7fedc0101e60 2024-03-15T13:07:26.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.834+0000 7fedbcff9640 1 -- 172.21.15.2:0/583949987 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4018920 con 0x7fedc0101e60 2024-03-15T13:07:26.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.834+0000 7fedc5e64640 1 -- 172.21.15.2:0/583949987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc01a4760 con 0x7fedc0101e60 2024-03-15T13:07:26.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.837+0000 7fedc8a44740 1 Processor -- start 2024-03-15T13:07:26.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.837+0000 7fedc8a44740 1 -- start start 2024-03-15T13:07:26.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.837+0000 7fedbcff9640 1 -- 172.21.15.2:0/583949987 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fedb4018a80 con 0x7fedc0101e60 2024-03-15T13:07:26.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.837+0000 7fedc8a44740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34250 0x55d122c34630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.837+0000 7fedc8a44740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c34b70 0x55d122c3d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.837+0000 7fedc8a44740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c3fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.837+0000 7fedc8a44740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d122bef960 con 0x55d122c3d750 2024-03-15T13:07:26.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.837+0000 7fedc8a44740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d122ba3310 con 0x55d122c34b70 2024-03-15T13:07:26.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.837+0000 7fedc8a44740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d122ba0840 con 0x55d122c34250 2024-03-15T13:07:26.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.837+0000 7fedbeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c3fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.837+0000 7fedbffff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34250 0x55d122c34630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.837+0000 7fedbeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c3fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49604/0 (socket says 172.21.15.2:49604) 2024-03-15T13:07:26.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.837+0000 7fedbeffd640 1 -- 172.21.15.2:0/147181528 learned_addr learned my addr 172.21.15.2:0/147181528 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.837+0000 7fedbf7fe640 1 --2- 172.21.15.2:0/147181528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c34b70 0x55d122c3d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.838+0000 7fedbcff9640 1 --2- 172.21.15.2:0/583949987 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feda80a1ba0 0x7feda80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.838+0000 7fedbeffd640 1 -- 172.21.15.2:0/147181528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34250 msgr2=0x55d122c34630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.838+0000 7fedbeffd640 1 --2- 172.21.15.2:0/147181528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34250 0x55d122c34630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.838+0000 7fedbeffd640 1 -- 172.21.15.2:0/147181528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c34b70 msgr2=0x55d122c3d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.838+0000 7fedbeffd640 1 --2- 172.21.15.2:0/147181528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c34b70 0x55d122c3d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.838+0000 7fedbeffd640 1 -- 172.21.15.2:0/147181528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d122b5e060 con 0x55d122c3d750 2024-03-15T13:07:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.838+0000 7fedbffff640 1 --2- 172.21.15.2:0/583949987 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feda80a1ba0 0x7feda80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.838+0000 7fedbcff9640 1 -- 172.21.15.2:0/583949987 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fedb4014070 con 0x7fedc0101e60 2024-03-15T13:07:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.838+0000 7fedbeffd640 1 --2- 172.21.15.2:0/147181528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c3fb40 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fedb40cb1a0 tx=0x7fedb406a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a924c094b3396a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.838+0000 7feda5ffb640 1 -- 172.21.15.2:0/147181528 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb40cf070 con 0x55d122c3d750 2024-03-15T13:07:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.838+0000 7feda5ffb640 1 -- 172.21.15.2:0/147181528 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fedb409a590 con 0x55d122c3d750 2024-03-15T13:07:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.838+0000 7feda5ffb640 1 -- 172.21.15.2:0/147181528 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb406e030 con 0x55d122c3d750 2024-03-15T13:07:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.839+0000 7fedc8a44740 1 -- 172.21.15.2:0/147181528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 msgr2=0x55d122c3fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.839+0000 7fedc8a44740 1 --2- 172.21.15.2:0/147181528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c3fb40 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fedb40cb1a0 tx=0x7fedb406a920 comp rx=0 tx=0).stop 2024-03-15T13:07:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.839+0000 7fedbffff640 1 --2- 172.21.15.2:0/583949987 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feda80a1ba0 0x7feda80a4060 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fedb8005fd0 tx=0x7fedb8004a40 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.839+0000 7fedc8a44740 1 -- 172.21.15.2:0/147181528 shutdown_connections 2024-03-15T13:07:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.839+0000 7fedc8a44740 1 --2- 172.21.15.2:0/147181528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c3fb40 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.839+0000 7fedc8a44740 1 --2- 172.21.15.2:0/147181528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c34b70 0x55d122c3d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.839+0000 7fedc8a44740 1 --2- 172.21.15.2:0/147181528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34250 0x55d122c34630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.839+0000 7fedc8a44740 1 -- 172.21.15.2:0/147181528 >> 172.21.15.2:0/147181528 conn(0x55d122c2e980 msgr2=0x55d122c2ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.839+0000 7fedc8a44740 1 -- 172.21.15.2:0/147181528 shutdown_connections 2024-03-15T13:07:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.839+0000 7fedc8a44740 1 -- 172.21.15.2:0/147181528 wait complete. 2024-03-15T13:07:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.839+0000 7fedc8a44740 1 -- start start 2024-03-15T13:07:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.839+0000 7fedc8a44740 1 Processor -- start 2024-03-15T13:07:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedc8a44740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c34250 0x55d122c3d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedc8a44740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34b70 0x55d122c3a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedc8a44740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c3d750 0x55d122c3a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedc8a44740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d122b9c020 con 0x55d122c34250 2024-03-15T13:07:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedc8a44740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d12286f430 con 0x55d122c3d750 2024-03-15T13:07:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedc8a44740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d12282be00 con 0x55d122c34b70 2024-03-15T13:07:26.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedc8a44740 10 client.0 _ll_register_callbacks cb 0x55d122b43cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:26.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedbf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c34250 0x55d122c3d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedbf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c34250 0x55d122c3d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49608/0 (socket says 172.21.15.2:49608) 2024-03-15T13:07:26.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedbf7fe640 1 -- 172.21.15.2:0/2807024838 learned_addr learned my addr 172.21.15.2:0/2807024838 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedbeffd640 1 --2- 172.21.15.2:0/2807024838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34b70 0x55d122c3a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedbffff640 1 --2- 172.21.15.2:0/2807024838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c3d750 0x55d122c3a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedbf7fe640 1 -- 172.21.15.2:0/2807024838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34b70 msgr2=0x55d122c3a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedbf7fe640 1 --2- 172.21.15.2:0/2807024838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34b70 0x55d122c3a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedbf7fe640 1 -- 172.21.15.2:0/2807024838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c3d750 msgr2=0x55d122c3a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedbf7fe640 1 --2- 172.21.15.2:0/2807024838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c3d750 0x55d122c3a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedbf7fe640 1 -- 172.21.15.2:0/2807024838 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc01a4760 con 0x55d122c34250 2024-03-15T13:07:26.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fedbf7fe640 1 --2- 172.21.15.2:0/2807024838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c34250 0x55d122c3d000 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fedb0036040 tx=0x7fedb002ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb354f1904ec30c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.840+0000 7fed9effd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:26.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.841+0000 7fed9effd640 1 -- 172.21.15.2:0/2807024838 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0004040 con 0x55d122c34250 2024-03-15T13:07:26.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.841+0000 7fed9effd640 1 -- 172.21.15.2:0/2807024838 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fedb00041a0 con 0x55d122c34250 2024-03-15T13:07:26.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.841+0000 7fedc8a44740 1 -- 172.21.15.2:0/2807024838 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d12289b7e0 con 0x55d122c34250 2024-03-15T13:07:26.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.841+0000 7fed9effd640 1 -- 172.21.15.2:0/2807024838 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedb0004360 con 0x55d122c34250 2024-03-15T13:07:26.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.841+0000 7fedc8a44740 10 client.18183 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:26.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.841+0000 7fedc8a44740 1 -- 172.21.15.2:0/2807024838 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55d122b9a6a0 con 0x55d122c34250 2024-03-15T13:07:26.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.841+0000 7fed9effd640 1 -- 172.21.15.2:0/2807024838 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55d12289b7e0 con 0x55d122c34250 2024-03-15T13:07:26.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.841+0000 7fed9effd640 1 -- 172.21.15.2:0/2807024838 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fedb003c040 con 0x55d122c34250 2024-03-15T13:07:26.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.841+0000 7fedc8a44740 10 client.18183 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:26.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.841+0000 7fedc8a44740 10 client.18183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:07:26.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.841+0000 7fedc8a44740 20 client.18183 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:26.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.841+0000 7fedc8a44740 1 --2- 172.21.15.2:0/2807024838 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d122c4d740 0x55d122c4fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.841+0000 7fedc8a44740 4 client.18183 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:07:26.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.841+0000 7fedc8a44740 1 -- 172.21.15.2:0/2807024838 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1228d8720 con 0x55d122c4d740 2024-03-15T13:07:26.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.841+0000 7fedbeffd640 1 --2- 172.21.15.2:0/2807024838 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d122c4d740 0x55d122c4fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.842+0000 7fed9effd640 10 client.18183 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:26.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.842+0000 7fedbeffd640 1 --2- 172.21.15.2:0/2807024838 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d122c4d740 0x55d122c4fb30 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=121e7ccdd6add1c9 server_cookie=6c9dce85a64dbabf in_seq=0 out_seq=0 2024-03-15T13:07:26.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.843+0000 7fed9effd640 1 -- 172.21.15.2:0/2807024838 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d1228d8720 con 0x55d122c4d740 2024-03-15T13:07:26.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.843+0000 7fed9effd640 10 client.18183 handle_command_reply: tid=0 2024-03-15T13:07:26.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.843+0000 7fedc8a44740 1 client.18183 shutdown 2024-03-15T13:07:26.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.843+0000 7fedc8a44740 1 -- 172.21.15.2:0/2807024838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c34250 msgr2=0x55d122c3d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.843+0000 7fedc8a44740 1 --2- 172.21.15.2:0/2807024838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c34250 0x55d122c3d000 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fedb0036040 tx=0x7fedb002ca40 comp rx=0 tx=0).stop 2024-03-15T13:07:26.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.843+0000 7fedc8a44740 1 -- 172.21.15.2:0/2807024838 shutdown_connections 2024-03-15T13:07:26.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.843+0000 7fedc8a44740 1 --2- 172.21.15.2:0/2807024838 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d122c4d740 0x55d122c4fb30 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.843+0000 7fedc8a44740 1 --2- 172.21.15.2:0/2807024838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c3d750 0x55d122c3a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.843+0000 7fed9effd640 0 client.18183 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:26.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.843+0000 7fedc8a44740 1 --2- 172.21.15.2:0/2807024838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34b70 0x55d122c3a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.843+0000 7fedc8a44740 1 --2- 172.21.15.2:0/2807024838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c34250 0x55d122c3d000 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.843+0000 7fedc8a44740 1 -- 172.21.15.2:0/2807024838 >> 172.21.15.2:0/2807024838 conn(0x55d122c2e980 msgr2=0x55d122c306d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.843+0000 7fedc8a44740 1 -- 172.21.15.2:0/2807024838 shutdown_connections 2024-03-15T13:07:26.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.843+0000 7fedc8a44740 1 -- 172.21.15.2:0/2807024838 wait complete. 2024-03-15T13:07:26.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.843+0000 7fedc8a44740 20 client.18183 trim_cache size 0 max 16384 2024-03-15T13:07:26.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedc8a44740 1 Processor -- start 2024-03-15T13:07:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedc8a44740 1 -- start start 2024-03-15T13:07:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedc8a44740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c34250 0x55d122c2d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedc8a44740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34b70 0x55d122c3cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedc8a44740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c3d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedc8a44740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d122b21130 con 0x55d122c3d750 2024-03-15T13:07:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedc8a44740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d122c2a2b0 con 0x55d122c34250 2024-03-15T13:07:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedc8a44740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d122c2e7b0 con 0x55d122c34b70 2024-03-15T13:07:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedbffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c3d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedbffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c3d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49622/0 (socket says 172.21.15.2:49622) 2024-03-15T13:07:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedbffff640 1 -- 172.21.15.2:0/3324254098 learned_addr learned my addr 172.21.15.2:0/3324254098 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedbeffd640 1 --2- 172.21.15.2:0/3324254098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34b70 0x55d122c3cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedbf7fe640 1 --2- 172.21.15.2:0/3324254098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c34250 0x55d122c2d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedbffff640 1 -- 172.21.15.2:0/3324254098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34b70 msgr2=0x55d122c3cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedbffff640 1 --2- 172.21.15.2:0/3324254098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34b70 0x55d122c3cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedbffff640 1 -- 172.21.15.2:0/3324254098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c34250 msgr2=0x55d122c2d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedbffff640 1 --2- 172.21.15.2:0/3324254098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c34250 0x55d122c2d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedbffff640 1 -- 172.21.15.2:0/3324254098 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d122b36c20 con 0x55d122c3d750 2024-03-15T13:07:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7fedbffff640 1 --2- 172.21.15.2:0/3324254098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c3d000 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fedb8020740 tx=0x7fedb8013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b23b2471c8e89d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7feda57fa640 1 -- 172.21.15.2:0/3324254098 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8025040 con 0x55d122c3d750 2024-03-15T13:07:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.856+0000 7feda57fa640 1 -- 172.21.15.2:0/3324254098 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fedb8017070 con 0x55d122c3d750 2024-03-15T13:07:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.857+0000 7feda57fa640 1 -- 172.21.15.2:0/3324254098 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb802ad70 con 0x55d122c3d750 2024-03-15T13:07:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.857+0000 7fedc8a44740 1 -- 172.21.15.2:0/3324254098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 msgr2=0x55d122c3d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.857+0000 7fedc8a44740 1 --2- 172.21.15.2:0/3324254098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c3d000 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fedb8020740 tx=0x7fedb8013030 comp rx=0 tx=0).stop 2024-03-15T13:07:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.857+0000 7fedc8a44740 1 -- 172.21.15.2:0/3324254098 shutdown_connections 2024-03-15T13:07:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.857+0000 7fedc8a44740 1 --2- 172.21.15.2:0/3324254098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c3d000 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.857+0000 7fedc8a44740 1 --2- 172.21.15.2:0/3324254098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34b70 0x55d122c3cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.857+0000 7fedc8a44740 1 --2- 172.21.15.2:0/3324254098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c34250 0x55d122c2d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.857+0000 7fedc8a44740 1 -- 172.21.15.2:0/3324254098 >> 172.21.15.2:0/3324254098 conn(0x55d122c2e980 msgr2=0x55d122c35920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.857+0000 7fedc8a44740 1 -- 172.21.15.2:0/3324254098 shutdown_connections 2024-03-15T13:07:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.857+0000 7fedc8a44740 1 -- 172.21.15.2:0/3324254098 wait complete. 2024-03-15T13:07:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.857+0000 7fedc8a44740 1 -- start start 2024-03-15T13:07:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedc8a44740 1 Processor -- start 2024-03-15T13:07:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedc8a44740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34250 0x55d122c38080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedc8a44740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c34b70 0x55d122c42520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedc8a44740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c42a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedc8a44740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d122c43130 con 0x55d122c3d750 2024-03-15T13:07:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedc8a44740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d122c432a0 con 0x55d122c34b70 2024-03-15T13:07:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedc8a44740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d122ca8ff0 con 0x55d122c34250 2024-03-15T13:07:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedc8a44740 10 client.0 _ll_register_callbacks cb 0x7fedb0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:26.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedbffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c42a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedbffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c42a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49632/0 (socket says 172.21.15.2:49632) 2024-03-15T13:07:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedbf7fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34250 0x55d122c38080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedbffff640 1 -- 172.21.15.2:0/1000278605 learned_addr learned my addr 172.21.15.2:0/1000278605 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedbeffd640 1 --2- 172.21.15.2:0/1000278605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c34b70 0x55d122c42520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedbffff640 1 -- 172.21.15.2:0/1000278605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34250 msgr2=0x55d122c38080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedbffff640 1 --2- 172.21.15.2:0/1000278605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34250 0x55d122c38080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedbffff640 1 -- 172.21.15.2:0/1000278605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c34b70 msgr2=0x55d122c42520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedbffff640 1 --2- 172.21.15.2:0/1000278605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c34b70 0x55d122c42520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedbffff640 1 -- 172.21.15.2:0/1000278605 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb8020450 con 0x55d122c3d750 2024-03-15T13:07:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fedbffff640 1 --2- 172.21.15.2:0/1000278605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c42a60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fedb80044d0 tx=0x7fedb8004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd71466979c9caa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.858+0000 7fed9effd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.859+0000 7fed9effd640 1 -- 172.21.15.2:0/1000278605 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8025030 con 0x55d122c3d750 2024-03-15T13:07:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.859+0000 7fed9effd640 1 -- 172.21.15.2:0/1000278605 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fedb802a970 con 0x55d122c3d750 2024-03-15T13:07:26.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.859+0000 7fedc8a44740 1 -- 172.21.15.2:0/1000278605 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedb409a590 con 0x55d122c3d750 2024-03-15T13:07:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.859+0000 7fed9effd640 1 -- 172.21.15.2:0/1000278605 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedb802ab30 con 0x55d122c3d750 2024-03-15T13:07:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.859+0000 7fedc8a44740 10 client.18189 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.859+0000 7fedc8a44740 1 -- 172.21.15.2:0/1000278605 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55d122b5c8c0 con 0x55d122c3d750 2024-03-15T13:07:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.859+0000 7fed9effd640 1 -- 172.21.15.2:0/1000278605 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fedb409a590 con 0x55d122c3d750 2024-03-15T13:07:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.859+0000 7fed9effd640 1 -- 172.21.15.2:0/1000278605 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fedb8032040 con 0x55d122c3d750 2024-03-15T13:07:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.859+0000 7fedc8a44740 10 client.18189 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.859+0000 7fedc8a44740 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:07:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.859+0000 7fedc8a44740 20 client.18189 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.859+0000 7fedc8a44740 1 --2- 172.21.15.2:0/1000278605 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d122caa670 0x55d122caca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.859+0000 7fedc8a44740 4 client.18189 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:07:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.859+0000 7fedc8a44740 1 -- 172.21.15.2:0/1000278605 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d122927890 con 0x55d122caa670 2024-03-15T13:07:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.859+0000 7fedbf7fe640 1 --2- 172.21.15.2:0/1000278605 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d122caa670 0x55d122caca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fed9effd640 10 client.18189 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fedbf7fe640 1 --2- 172.21.15.2:0/1000278605 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d122caa670 0x55d122caca90 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c364513ed0a98f7 server_cookie=2b79cfcd668ce9cc in_seq=0 out_seq=0 2024-03-15T13:07:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fed9effd640 1 -- 172.21.15.2:0/1000278605 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d122927890 con 0x55d122caa670 2024-03-15T13:07:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fed9effd640 10 client.18189 handle_command_reply: tid=0 2024-03-15T13:07:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fedc8a44740 1 client.18189 shutdown 2024-03-15T13:07:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fedc8a44740 1 -- 172.21.15.2:0/1000278605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 msgr2=0x55d122c42a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fedc8a44740 1 --2- 172.21.15.2:0/1000278605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c42a60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fedb80044d0 tx=0x7fedb8004860 comp rx=0 tx=0).stop 2024-03-15T13:07:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fedc8a44740 1 -- 172.21.15.2:0/1000278605 shutdown_connections 2024-03-15T13:07:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fedc8a44740 1 --2- 172.21.15.2:0/1000278605 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d122caa670 0x55d122caca90 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fedc8a44740 1 --2- 172.21.15.2:0/1000278605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d122c3d750 0x55d122c42a60 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fed9effd640 0 client.18189 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fedc8a44740 1 --2- 172.21.15.2:0/1000278605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d122c34b70 0x55d122c42520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fedc8a44740 1 --2- 172.21.15.2:0/1000278605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d122c34250 0x55d122c38080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fedc8a44740 1 -- 172.21.15.2:0/1000278605 >> 172.21.15.2:0/1000278605 conn(0x55d122c2e980 msgr2=0x55d122c2faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fedc8a44740 1 -- 172.21.15.2:0/1000278605 shutdown_connections 2024-03-15T13:07:26.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fedc8a44740 1 -- 172.21.15.2:0/1000278605 wait complete. 2024-03-15T13:07:26.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.860+0000 7fedc8a44740 20 client.18189 trim_cache size 0 max 16384 2024-03-15T13:07:26.852 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:07:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.861+0000 7fedc5e64640 1 -- 172.21.15.2:0/583949987 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feda80a1ba0 msgr2=0x7feda80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.861+0000 7fedc5e64640 1 --2- 172.21.15.2:0/583949987 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feda80a1ba0 0x7feda80a4060 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fedb8005fd0 tx=0x7fedb8004a40 comp rx=0 tx=0).stop 2024-03-15T13:07:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.861+0000 7fedc5e64640 1 -- 172.21.15.2:0/583949987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc0101e60 msgr2=0x7fedc019dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.861+0000 7fedc5e64640 1 --2- 172.21.15.2:0/583949987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc0101e60 0x7fedc019dc00 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fedb400e9f0 tx=0x7fedb400eeb0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.861+0000 7fedc5e64640 1 -- 172.21.15.2:0/583949987 shutdown_connections 2024-03-15T13:07:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.861+0000 7fedc5e64640 1 --2- 172.21.15.2:0/583949987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedc0107560 0x7fedc0197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.861+0000 7fedc5e64640 1 --2- 172.21.15.2:0/583949987 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feda80a1ba0 0x7feda80a4060 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.861+0000 7fedc5e64640 1 --2- 172.21.15.2:0/583949987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc0101e60 0x7fedc019dc00 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.861+0000 7fedc5e64640 1 --2- 172.21.15.2:0/583949987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedc01014a0 0x7fedc019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.861+0000 7fedc5e64640 1 -- 172.21.15.2:0/583949987 >> 172.21.15.2:0/583949987 conn(0x7fedc00788f0 msgr2=0x7fedc0100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:26.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.861+0000 7fedc5e64640 1 -- 172.21.15.2:0/583949987 shutdown_connections 2024-03-15T13:07:26.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:26.861+0000 7fedc5e64640 1 -- 172.21.15.2:0/583949987 wait complete. 2024-03-15T13:07:26.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:26 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/4169352840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:07:27.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:27 smithi002 ceph-mon[32060]: pgmap v263: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-15T13:07:27.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:27 smithi184 ceph-mon[35993]: pgmap v263: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-15T13:07:27.856 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:07:27.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:07:27.857 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:07:27.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:27 smithi149 ceph-mon[35927]: pgmap v263: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-15T13:07:27.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.991+0000 7f689c1ac640 1 -- 172.21.15.2:0/2809232508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6894103080 msgr2=0x7f689410b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:27.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.991+0000 7f689c1ac640 1 --2- 172.21.15.2:0/2809232508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6894103080 0x7f689410b560 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f6884009900 tx=0x7f688402bcd0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.991+0000 7f689c1ac640 1 -- 172.21.15.2:0/2809232508 shutdown_connections 2024-03-15T13:07:27.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.991+0000 7f689c1ac640 1 --2- 172.21.15.2:0/2809232508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f689410bb30 0x7f689410dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.991+0000 7f689c1ac640 1 --2- 172.21.15.2:0/2809232508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6894103080 0x7f689410b560 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.991+0000 7f689c1ac640 1 --2- 172.21.15.2:0/2809232508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6894102760 0x7f6894102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.991+0000 7f689c1ac640 1 -- 172.21.15.2:0/2809232508 >> 172.21.15.2:0/2809232508 conn(0x7f6894077550 msgr2=0x7f6894077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:27.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.991+0000 7f689c1ac640 1 -- 172.21.15.2:0/2809232508 shutdown_connections 2024-03-15T13:07:27.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.991+0000 7f689c1ac640 1 -- 172.21.15.2:0/2809232508 wait complete. 2024-03-15T13:07:27.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.991+0000 7f689c1ac640 1 Processor -- start 2024-03-15T13:07:27.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.991+0000 7f689c1ac640 1 -- start start 2024-03-15T13:07:27.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f689c1ac640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6894102760 0x7f689419d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:27.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f689c1ac640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6894103080 0x7f689419ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:27.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f689c1ac640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f689410bb30 0x7f6894197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:27.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f689c1ac640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6894198080 con 0x7f6894103080 2024-03-15T13:07:27.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f689c1ac640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68941981f0 con 0x7f6894102760 2024-03-15T13:07:27.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f689c1ac640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68941983b0 con 0x7f689410bb30 2024-03-15T13:07:27.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f6899747640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6894103080 0x7f689419ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:27.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f6899747640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6894103080 0x7f689419ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60354/0 (socket says 172.21.15.2:60354) 2024-03-15T13:07:27.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f6899747640 1 -- 172.21.15.2:0/83459812 learned_addr learned my addr 172.21.15.2:0/83459812 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:27.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f6899f48640 1 --2- 172.21.15.2:0/83459812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6894102760 0x7f689419d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:27.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f689a749640 1 --2- 172.21.15.2:0/83459812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f689410bb30 0x7f6894197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:27.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f6899747640 1 -- 172.21.15.2:0/83459812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f689410bb30 msgr2=0x7f6894197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:27.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f6899747640 1 --2- 172.21.15.2:0/83459812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f689410bb30 0x7f6894197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f6899747640 1 -- 172.21.15.2:0/83459812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6894102760 msgr2=0x7f689419d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:27.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f6899747640 1 --2- 172.21.15.2:0/83459812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6894102760 0x7f689419d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f6899747640 1 -- 172.21.15.2:0/83459812 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6884009560 con 0x7f6894103080 2024-03-15T13:07:27.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f6899747640 1 --2- 172.21.15.2:0/83459812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6894103080 0x7f689419ddc0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f6884005940 tx=0x7f688402cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:27.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f688affd640 1 -- 172.21.15.2:0/83459812 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688402cd30 con 0x7f6894103080 2024-03-15T13:07:27.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f689c1ac640 1 -- 172.21.15.2:0/83459812 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6894198630 con 0x7f6894103080 2024-03-15T13:07:27.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f688affd640 1 -- 172.21.15.2:0/83459812 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f68840054f0 con 0x7f6894103080 2024-03-15T13:07:27.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.992+0000 7f688affd640 1 -- 172.21.15.2:0/83459812 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884003d00 con 0x7f6894103080 2024-03-15T13:07:27.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.993+0000 7f689c1ac640 1 -- 172.21.15.2:0/83459812 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68941a4920 con 0x7f6894103080 2024-03-15T13:07:27.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.995+0000 7f688affd640 1 -- 172.21.15.2:0/83459812 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f6884005020 con 0x7f6894103080 2024-03-15T13:07:27.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.995+0000 7f689ed85740 1 Processor -- start 2024-03-15T13:07:27.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.995+0000 7f689ed85740 1 -- start start 2024-03-15T13:07:27.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.995+0000 7f688affd640 1 --2- 172.21.15.2:0/83459812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f68800a1ba0 0x7f68800a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:27.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.995+0000 7f689ed85740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a341e0 0x5614d8a345c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:27.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.995+0000 7f689ed85740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a34b90 0x5614d8a3d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:27.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.995+0000 7f689ed85740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a3dcf0 0x5614d8a3e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:27.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.995+0000 7f689ed85740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d89a2f90 con 0x5614d8a341e0 2024-03-15T13:07:27.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.995+0000 7f689ed85740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d89a04c0 con 0x5614d8a34b90 2024-03-15T13:07:27.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.995+0000 7f689ed85740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d899bca0 con 0x5614d8a3dcf0 2024-03-15T13:07:27.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f689a749640 1 --2- 172.21.15.2:0/83459812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f68800a1ba0 0x7f68800a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:27.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f688affd640 1 -- 172.21.15.2:0/83459812 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f68840f0140 con 0x7f6894103080 2024-03-15T13:07:27.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f6899f48640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a341e0 0x5614d8a345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:27.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f6899f48640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a341e0 0x5614d8a345c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60358/0 (socket says 172.21.15.2:60358) 2024-03-15T13:07:27.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f6899f48640 1 -- 172.21.15.2:0/3042475530 learned_addr learned my addr 172.21.15.2:0/3042475530 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:27.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f6899747640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a34b90 0x5614d8a3d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:27.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f689a749640 1 --2- 172.21.15.2:0/3042475530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a3dcf0 0x5614d8a3e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:27.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f6899f48640 1 -- 172.21.15.2:0/3042475530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a3dcf0 msgr2=0x5614d8a3e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:27.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f6899f48640 1 --2- 172.21.15.2:0/3042475530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a3dcf0 0x5614d8a3e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f6899f48640 1 -- 172.21.15.2:0/3042475530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a34b90 msgr2=0x5614d8a3d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:27.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f6899f48640 1 --2- 172.21.15.2:0/3042475530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a34b90 0x5614d8a3d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f6899f48640 1 -- 172.21.15.2:0/3042475530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614d895e0b0 con 0x5614d8a341e0 2024-03-15T13:07:27.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f6899747640 1 --2- 172.21.15.2:0/3042475530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a34b90 0x5614d8a3d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:27.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f689a749640 1 --2- 172.21.15.2:0/83459812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f68800a1ba0 0x7f68800a4060 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f688c009fd0 tx=0x7f688c009de0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:27.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f6899f48640 1 --2- 172.21.15.2:0/3042475530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a341e0 0x5614d8a345c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f689000bcb0 tx=0x7f68900107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fb442802849300 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:27.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f687ffff640 1 -- 172.21.15.2:0/3042475530 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890010ed0 con 0x5614d8a341e0 2024-03-15T13:07:27.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f687ffff640 1 -- 172.21.15.2:0/3042475530 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6890015070 con 0x5614d8a341e0 2024-03-15T13:07:27.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.996+0000 7f687ffff640 1 -- 172.21.15.2:0/3042475530 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68900119d0 con 0x5614d8a341e0 2024-03-15T13:07:27.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.997+0000 7f689ed85740 1 -- 172.21.15.2:0/3042475530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a341e0 msgr2=0x5614d8a345c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:27.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.997+0000 7f689ed85740 1 --2- 172.21.15.2:0/3042475530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a341e0 0x5614d8a345c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f689000bcb0 tx=0x7f68900107a0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.997+0000 7f689ed85740 1 -- 172.21.15.2:0/3042475530 shutdown_connections 2024-03-15T13:07:27.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.997+0000 7f689ed85740 1 --2- 172.21.15.2:0/3042475530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a3dcf0 0x5614d8a3e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.997+0000 7f689ed85740 1 --2- 172.21.15.2:0/3042475530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a34b90 0x5614d8a3d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.997+0000 7f689ed85740 1 --2- 172.21.15.2:0/3042475530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a341e0 0x5614d8a345c0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.997+0000 7f689ed85740 1 -- 172.21.15.2:0/3042475530 >> 172.21.15.2:0/3042475530 conn(0x5614d8a2e840 msgr2=0x5614d8a2ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:27.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.997+0000 7f689ed85740 1 -- 172.21.15.2:0/3042475530 shutdown_connections 2024-03-15T13:07:27.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.997+0000 7f689ed85740 1 -- 172.21.15.2:0/3042475530 wait complete. 2024-03-15T13:07:27.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.997+0000 7f689ed85740 1 -- start start 2024-03-15T13:07:27.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689ed85740 1 Processor -- start 2024-03-15T13:07:27.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689ed85740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a341e0 0x5614d8a417e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:27.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689ed85740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a34b90 0x5614d8a41d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:27.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689ed85740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a42260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:27.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689ed85740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d8940f40 con 0x5614d8a3dcf0 2024-03-15T13:07:27.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689ed85740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d866f4b0 con 0x5614d8a341e0 2024-03-15T13:07:27.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689ed85740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d8921670 con 0x5614d8a34b90 2024-03-15T13:07:27.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689ed85740 10 client.0 _ll_register_callbacks cb 0x5614d898ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:27.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689a749640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a42260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:27.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689a749640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a42260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60362/0 (socket says 172.21.15.2:60362) 2024-03-15T13:07:27.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689a749640 1 -- 172.21.15.2:0/3476815827 learned_addr learned my addr 172.21.15.2:0/3476815827 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:27.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f6899f48640 1 --2- 172.21.15.2:0/3476815827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a341e0 0x5614d8a417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:27.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f6899747640 1 --2- 172.21.15.2:0/3476815827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a34b90 0x5614d8a41d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:27.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689a749640 1 -- 172.21.15.2:0/3476815827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a34b90 msgr2=0x5614d8a41d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:27.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689a749640 1 --2- 172.21.15.2:0/3476815827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a34b90 0x5614d8a41d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689a749640 1 -- 172.21.15.2:0/3476815827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a341e0 msgr2=0x5614d8a417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:27.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689a749640 1 --2- 172.21.15.2:0/3476815827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a341e0 0x5614d8a417e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.998+0000 7f689a749640 1 -- 172.21.15.2:0/3476815827 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689000b8e0 con 0x5614d8a3dcf0 2024-03-15T13:07:27.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f689a749640 1 --2- 172.21.15.2:0/3476815827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a42260 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f688c003bf0 tx=0x7f688c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2d14bfa0a6af7e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:27.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f687d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:27.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f687d7fa640 1 -- 172.21.15.2:0/3476815827 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c004260 con 0x5614d8a3dcf0 2024-03-15T13:07:27.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f687d7fa640 1 -- 172.21.15.2:0/3476815827 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f688c0043c0 con 0x5614d8a3dcf0 2024-03-15T13:07:27.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f689ed85740 1 -- 172.21.15.2:0/3476815827 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614d8975f40 con 0x5614d8a3dcf0 2024-03-15T13:07:27.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f687d7fa640 1 -- 172.21.15.2:0/3476815827 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f688c004580 con 0x5614d8a3dcf0 2024-03-15T13:07:27.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f689ed85740 10 client.18201 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:27.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f689ed85740 1 -- 172.21.15.2:0/3476815827 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5614d899a320 con 0x5614d8a3dcf0 2024-03-15T13:07:27.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f687d7fa640 1 -- 172.21.15.2:0/3476815827 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5614d8975f40 con 0x5614d8a3dcf0 2024-03-15T13:07:27.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f687d7fa640 1 -- 172.21.15.2:0/3476815827 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f688c021440 con 0x5614d8a3dcf0 2024-03-15T13:07:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f689ed85740 10 client.18201 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f689ed85740 10 client.18201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:07:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f689ed85740 20 client.18201 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f689ed85740 1 --2- 172.21.15.2:0/3476815827 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5614d8a52340 0x5614d8a54760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f689ed85740 4 client.18201 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:07:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:27.999+0000 7f689ed85740 1 -- 172.21.15.2:0/3476815827 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614d8992310 con 0x5614d8a52340 2024-03-15T13:07:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.000+0000 7f6899f48640 1 --2- 172.21.15.2:0/3476815827 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5614d8a52340 0x5614d8a54760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.000+0000 7f6899f48640 1 --2- 172.21.15.2:0/3476815827 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5614d8a52340 0x5614d8a54760 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9610877cd61e869b server_cookie=cf5722bb85f961de in_seq=0 out_seq=0 2024-03-15T13:07:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.000+0000 7f687d7fa640 10 client.18201 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.001+0000 7f687d7fa640 1 -- 172.21.15.2:0/3476815827 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5614d8992310 con 0x5614d8a52340 2024-03-15T13:07:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.001+0000 7f687d7fa640 10 client.18201 handle_command_reply: tid=0 2024-03-15T13:07:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.001+0000 7f689ed85740 1 client.18201 shutdown 2024-03-15T13:07:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.001+0000 7f689ed85740 1 -- 172.21.15.2:0/3476815827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 msgr2=0x5614d8a42260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.001+0000 7f689ed85740 1 --2- 172.21.15.2:0/3476815827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a42260 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f688c003bf0 tx=0x7f688c004ab0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.001+0000 7f689ed85740 1 -- 172.21.15.2:0/3476815827 shutdown_connections 2024-03-15T13:07:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.001+0000 7f689ed85740 1 --2- 172.21.15.2:0/3476815827 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5614d8a52340 0x5614d8a54760 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.001+0000 7f689ed85740 1 --2- 172.21.15.2:0/3476815827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a42260 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.001+0000 7f689ed85740 1 --2- 172.21.15.2:0/3476815827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a34b90 0x5614d8a41d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.001+0000 7f689ed85740 1 --2- 172.21.15.2:0/3476815827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a341e0 0x5614d8a417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.001+0000 7f687d7fa640 0 client.18201 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.001+0000 7f689ed85740 1 -- 172.21.15.2:0/3476815827 >> 172.21.15.2:0/3476815827 conn(0x5614d8a2e840 msgr2=0x5614d8a307a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.001+0000 7f689ed85740 1 -- 172.21.15.2:0/3476815827 shutdown_connections 2024-03-15T13:07:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.001+0000 7f689ed85740 1 -- 172.21.15.2:0/3476815827 wait complete. 2024-03-15T13:07:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.001+0000 7f689ed85740 20 client.18201 trim_cache size 0 max 16384 2024-03-15T13:07:28.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.017+0000 7f689ed85740 1 Processor -- start 2024-03-15T13:07:28.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.017+0000 7f689ed85740 1 -- start start 2024-03-15T13:07:28.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f689ed85740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a341e0 0x5614d8a43f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f689ed85740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a34b90 0x5614d8a42080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f689ed85740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a3f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f689ed85740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d8a3bf70 con 0x5614d8a3dcf0 2024-03-15T13:07:28.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f689ed85740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d8a336d0 con 0x5614d8a34b90 2024-03-15T13:07:28.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f689ed85740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d8a3dad0 con 0x5614d8a341e0 2024-03-15T13:07:28.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f689a749640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a3f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f6899747640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a34b90 0x5614d8a42080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f6899f48640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a341e0 0x5614d8a43f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f689a749640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a3f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60364/0 (socket says 172.21.15.2:60364) 2024-03-15T13:07:28.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f689a749640 1 -- 172.21.15.2:0/1901363805 learned_addr learned my addr 172.21.15.2:0/1901363805 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:28.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f6899747640 1 -- 172.21.15.2:0/1901363805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a341e0 msgr2=0x5614d8a43f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:28.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f6899747640 1 --2- 172.21.15.2:0/1901363805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a341e0 0x5614d8a43f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f6899747640 1 -- 172.21.15.2:0/1901363805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 msgr2=0x5614d8a3f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:28.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f6899747640 1 --2- 172.21.15.2:0/1901363805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a3f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f6899747640 1 -- 172.21.15.2:0/1901363805 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614d8a3c0b0 con 0x5614d8a34b90 2024-03-15T13:07:28.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f689a749640 1 --2- 172.21.15.2:0/1901363805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a3f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:28.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f6899747640 1 --2- 172.21.15.2:0/1901363805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a34b90 0x5614d8a42080 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f68840ed600 tx=0x7f68840bc500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=922fb548ab92ef3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:28.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.018+0000 7f687f7fe640 1 -- 172.21.15.2:0/1901363805 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68840044e0 con 0x5614d8a34b90 2024-03-15T13:07:28.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.019+0000 7f687f7fe640 1 -- 172.21.15.2:0/1901363805 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f688402ce90 con 0x5614d8a34b90 2024-03-15T13:07:28.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.019+0000 7f687f7fe640 1 -- 172.21.15.2:0/1901363805 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68840c0cc0 con 0x5614d8a34b90 2024-03-15T13:07:28.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.019+0000 7f689ed85740 1 -- 172.21.15.2:0/1901363805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a34b90 msgr2=0x5614d8a42080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:28.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.019+0000 7f689ed85740 1 --2- 172.21.15.2:0/1901363805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a34b90 0x5614d8a42080 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f68840ed600 tx=0x7f68840bc500 comp rx=0 tx=0).stop 2024-03-15T13:07:28.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.019+0000 7f689ed85740 1 -- 172.21.15.2:0/1901363805 shutdown_connections 2024-03-15T13:07:28.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.019+0000 7f689ed85740 1 --2- 172.21.15.2:0/1901363805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a3f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.019+0000 7f689ed85740 1 --2- 172.21.15.2:0/1901363805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a34b90 0x5614d8a42080 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.019+0000 7f689ed85740 1 --2- 172.21.15.2:0/1901363805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a341e0 0x5614d8a43f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.019+0000 7f689ed85740 1 -- 172.21.15.2:0/1901363805 >> 172.21.15.2:0/1901363805 conn(0x5614d8a2e840 msgr2=0x5614d8a2d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:28.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.019+0000 7f689ed85740 1 -- 172.21.15.2:0/1901363805 shutdown_connections 2024-03-15T13:07:28.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.019+0000 7f689ed85740 1 -- 172.21.15.2:0/1901363805 wait complete. 2024-03-15T13:07:28.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.019+0000 7f689ed85740 1 -- start start 2024-03-15T13:07:28.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.019+0000 7f689ed85740 1 Processor -- start 2024-03-15T13:07:28.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689ed85740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a341e0 0x5614d8a49950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689ed85740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a34b90 0x5614d8a4cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689ed85740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a4d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689ed85740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d8a4d920 con 0x5614d8a3dcf0 2024-03-15T13:07:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689ed85740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d8a4da90 con 0x5614d8a341e0 2024-03-15T13:07:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689ed85740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d8a4dc50 con 0x5614d8a34b90 2024-03-15T13:07:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689ed85740 10 client.0 _ll_register_callbacks cb 0x7f688c009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689a749640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a4d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689a749640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a4d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60372/0 (socket says 172.21.15.2:60372) 2024-03-15T13:07:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689a749640 1 -- 172.21.15.2:0/2095490245 learned_addr learned my addr 172.21.15.2:0/2095490245 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f6899747640 1 --2- 172.21.15.2:0/2095490245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a34b90 0x5614d8a4cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f6899f48640 1 --2- 172.21.15.2:0/2095490245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a341e0 0x5614d8a49950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689a749640 1 -- 172.21.15.2:0/2095490245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a34b90 msgr2=0x5614d8a4cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689a749640 1 --2- 172.21.15.2:0/2095490245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a34b90 0x5614d8a4cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689a749640 1 -- 172.21.15.2:0/2095490245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a341e0 msgr2=0x5614d8a49950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689a749640 1 --2- 172.21.15.2:0/2095490245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a341e0 0x5614d8a49950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689a749640 1 -- 172.21.15.2:0/2095490245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614d899a320 con 0x5614d8a3dcf0 2024-03-15T13:07:28.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f689a749640 1 --2- 172.21.15.2:0/2095490245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a4d3e0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f688c021770 tx=0x7f688c013a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160564161fc68899 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.020+0000 7f687d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.021+0000 7f687d7fa640 1 -- 172.21.15.2:0/2095490245 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c013400 con 0x5614d8a3dcf0 2024-03-15T13:07:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.021+0000 7f687d7fa640 1 -- 172.21.15.2:0/2095490245 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f688c0047b0 con 0x5614d8a3dcf0 2024-03-15T13:07:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.021+0000 7f689ed85740 1 -- 172.21.15.2:0/2095490245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614d89c1900 con 0x5614d8a3dcf0 2024-03-15T13:07:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.021+0000 7f687d7fa640 1 -- 172.21.15.2:0/2095490245 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f688c038450 con 0x5614d8a3dcf0 2024-03-15T13:07:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.021+0000 7f689ed85740 10 client.18207 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.021+0000 7f689ed85740 1 -- 172.21.15.2:0/2095490245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f6894198630 con 0x5614d8a3dcf0 2024-03-15T13:07:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.021+0000 7f687d7fa640 1 -- 172.21.15.2:0/2095490245 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5614d89c1900 con 0x5614d8a3dcf0 2024-03-15T13:07:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.021+0000 7f687d7fa640 1 -- 172.21.15.2:0/2095490245 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f688c0124e0 con 0x5614d8a3dcf0 2024-03-15T13:07:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.021+0000 7f689ed85740 10 client.18207 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.021+0000 7f689ed85740 10 client.18207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:07:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.021+0000 7f689ed85740 20 client.18207 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:28.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.021+0000 7f689ed85740 1 --2- 172.21.15.2:0/2095490245 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5614d8aaf0b0 0x5614d8ab14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.021+0000 7f689ed85740 4 client.18207 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:07:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.021+0000 7f689ed85740 1 -- 172.21.15.2:0/2095490245 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5614d869b860 con 0x5614d8aaf0b0 2024-03-15T13:07:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.021+0000 7f6899f48640 1 --2- 172.21.15.2:0/2095490245 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5614d8aaf0b0 0x5614d8ab14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.022+0000 7f687d7fa640 10 client.18207 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.022+0000 7f6899f48640 1 --2- 172.21.15.2:0/2095490245 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5614d8aaf0b0 0x5614d8ab14d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f2713d08079cdb server_cookie=b07c2bc90e4f4552 in_seq=0 out_seq=0 2024-03-15T13:07:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.023+0000 7f687d7fa640 1 -- 172.21.15.2:0/2095490245 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5614d869b860 con 0x5614d8aaf0b0 2024-03-15T13:07:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.023+0000 7f687d7fa640 10 client.18207 handle_command_reply: tid=0 2024-03-15T13:07:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.023+0000 7f689ed85740 1 client.18207 shutdown 2024-03-15T13:07:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.023+0000 7f689ed85740 1 -- 172.21.15.2:0/2095490245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 msgr2=0x5614d8a4d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.023+0000 7f689ed85740 1 --2- 172.21.15.2:0/2095490245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a4d3e0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f688c021770 tx=0x7f688c013a30 comp rx=0 tx=0).stop 2024-03-15T13:07:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.023+0000 7f689ed85740 1 -- 172.21.15.2:0/2095490245 shutdown_connections 2024-03-15T13:07:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.023+0000 7f689ed85740 1 --2- 172.21.15.2:0/2095490245 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5614d8aaf0b0 0x5614d8ab14d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.023+0000 7f689ed85740 1 --2- 172.21.15.2:0/2095490245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614d8a3dcf0 0x5614d8a4d3e0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.023+0000 7f689ed85740 1 --2- 172.21.15.2:0/2095490245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5614d8a34b90 0x5614d8a4cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.023+0000 7f687d7fa640 0 client.18207 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:28.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.023+0000 7f689ed85740 1 --2- 172.21.15.2:0/2095490245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614d8a341e0 0x5614d8a49950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.023+0000 7f689ed85740 1 -- 172.21.15.2:0/2095490245 >> 172.21.15.2:0/2095490245 conn(0x5614d8a2e840 msgr2=0x5614d8a2d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:28.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.023+0000 7f689ed85740 1 -- 172.21.15.2:0/2095490245 shutdown_connections 2024-03-15T13:07:28.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.023+0000 7f689ed85740 1 -- 172.21.15.2:0/2095490245 wait complete. 2024-03-15T13:07:28.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.024+0000 7f689ed85740 20 client.18207 trim_cache size 0 max 16384 2024-03-15T13:07:28.015 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:07:28.015 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:07:28.016 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "0fc12026-188f-46fa-87b9-baa531d6e6af", 2024-03-15T13:07:28.016 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:07:28.016 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:07:28.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.024+0000 7f689c1ac640 1 -- 172.21.15.2:0/83459812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f68800a1ba0 msgr2=0x7f68800a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:28.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.024+0000 7f689c1ac640 1 --2- 172.21.15.2:0/83459812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f68800a1ba0 0x7f68800a4060 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f688c009fd0 tx=0x7f688c009de0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.024+0000 7f689c1ac640 1 -- 172.21.15.2:0/83459812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6894103080 msgr2=0x7f689419ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:28.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.024+0000 7f689c1ac640 1 --2- 172.21.15.2:0/83459812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6894103080 0x7f689419ddc0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f6884005940 tx=0x7f688402cb40 comp rx=0 tx=0).stop 2024-03-15T13:07:28.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.024+0000 7f689c1ac640 1 -- 172.21.15.2:0/83459812 shutdown_connections 2024-03-15T13:07:28.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.024+0000 7f689c1ac640 1 --2- 172.21.15.2:0/83459812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f689410bb30 0x7f6894197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.024+0000 7f689c1ac640 1 --2- 172.21.15.2:0/83459812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f68800a1ba0 0x7f68800a4060 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.024+0000 7f689c1ac640 1 --2- 172.21.15.2:0/83459812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6894103080 0x7f689419ddc0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.024+0000 7f689c1ac640 1 --2- 172.21.15.2:0/83459812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6894102760 0x7f689419d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.024+0000 7f689c1ac640 1 -- 172.21.15.2:0/83459812 >> 172.21.15.2:0/83459812 conn(0x7f6894077550 msgr2=0x7f68940fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:28.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.024+0000 7f689c1ac640 1 -- 172.21.15.2:0/83459812 shutdown_connections 2024-03-15T13:07:28.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.024+0000 7f689c1ac640 1 -- 172.21.15.2:0/83459812 wait complete. 2024-03-15T13:07:28.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:07:28.020 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:07:28.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.140+0000 7f04e0c47640 1 -- 172.21.15.2:0/1524182888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04dc107500 msgr2=0x7f04dc1078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:28.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.140+0000 7f04e0c47640 1 --2- 172.21.15.2:0/1524182888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04dc107500 0x7f04dc1078e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009930 tx=0x7f04cc02bc30 comp rx=0 tx=0).stop 2024-03-15T13:07:28.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.140+0000 7f04e0c47640 1 -- 172.21.15.2:0/1524182888 shutdown_connections 2024-03-15T13:07:28.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.140+0000 7f04e0c47640 1 --2- 172.21.15.2:0/1524182888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04dc101e20 0x7f04dc10e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.140+0000 7f04e0c47640 1 --2- 172.21.15.2:0/1524182888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04dc101460 0x7f04dc1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.140+0000 7f04e0c47640 1 --2- 172.21.15.2:0/1524182888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04dc107500 0x7f04dc1078e0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.140+0000 7f04e0c47640 1 -- 172.21.15.2:0/1524182888 >> 172.21.15.2:0/1524182888 conn(0x7f04dc0fcf90 msgr2=0x7f04dc0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:28.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.140+0000 7f04e0c47640 1 -- 172.21.15.2:0/1524182888 shutdown_connections 2024-03-15T13:07:28.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.140+0000 7f04e0c47640 1 -- 172.21.15.2:0/1524182888 wait complete. 2024-03-15T13:07:28.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04e0c47640 1 Processor -- start 2024-03-15T13:07:28.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04e0c47640 1 -- start start 2024-03-15T13:07:28.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04e0c47640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04dc101460 0x7f04dc0737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04e0c47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04dc101e20 0x7f04dc073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04e0c47640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04dc107500 0x7f04dc074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04e0c47640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc1a5f30 con 0x7f04dc101460 2024-03-15T13:07:28.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04e0c47640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc1a60a0 con 0x7f04dc107500 2024-03-15T13:07:28.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04e0c47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc1a6260 con 0x7f04dc101e20 2024-03-15T13:07:28.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04da59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04dc101460 0x7f04dc0737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04da59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04dc101460 0x7f04dc0737f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60390/0 (socket says 172.21.15.2:60390) 2024-03-15T13:07:28.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04da59c640 1 -- 172.21.15.2:0/4041905663 learned_addr learned my addr 172.21.15.2:0/4041905663 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:28.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04dad9d640 1 --2- 172.21.15.2:0/4041905663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04dc107500 0x7f04dc074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04d9d9b640 1 --2- 172.21.15.2:0/4041905663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04dc101e20 0x7f04dc073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04da59c640 1 -- 172.21.15.2:0/4041905663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04dc101e20 msgr2=0x7f04dc073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:28.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04da59c640 1 --2- 172.21.15.2:0/4041905663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04dc101e20 0x7f04dc073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04da59c640 1 -- 172.21.15.2:0/4041905663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04dc107500 msgr2=0x7f04dc074270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:28.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04da59c640 1 --2- 172.21.15.2:0/4041905663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04dc107500 0x7f04dc074270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.141+0000 7f04da59c640 1 -- 172.21.15.2:0/4041905663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d4009840 con 0x7f04dc101460 2024-03-15T13:07:28.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.142+0000 7f04da59c640 1 --2- 172.21.15.2:0/4041905663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04dc101460 0x7f04dc0737f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009900 tx=0x7f04cc02ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:28.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.142+0000 7f04cb7fe640 1 -- 172.21.15.2:0/4041905663 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc02cc10 con 0x7f04dc101460 2024-03-15T13:07:28.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.142+0000 7f04cb7fe640 1 -- 172.21.15.2:0/4041905663 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f04cc0054d0 con 0x7f04dc101460 2024-03-15T13:07:28.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.142+0000 7f04cb7fe640 1 -- 172.21.15.2:0/4041905663 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc003b80 con 0x7f04dc101460 2024-03-15T13:07:28.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.142+0000 7f04e0c47640 1 -- 172.21.15.2:0/4041905663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04cc009560 con 0x7f04dc101460 2024-03-15T13:07:28.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.142+0000 7f04e0c47640 1 -- 172.21.15.2:0/4041905663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04dc1a6860 con 0x7f04dc101460 2024-03-15T13:07:28.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.144+0000 7f04cb7fe640 1 -- 172.21.15.2:0/4041905663 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f04cc003ce0 con 0x7f04dc101460 2024-03-15T13:07:28.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.144+0000 7f04e3827740 1 Processor -- start 2024-03-15T13:07:28.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04e3827740 1 -- start start 2024-03-15T13:07:28.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04e3827740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096f2b0 0x556e3096f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04e3827740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e30978270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04e3827740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 0x556e3097aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04e3827740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e308de280 con 0x556e3096fbd0 2024-03-15T13:07:28.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04e3827740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e308db7b0 con 0x556e3096f2b0 2024-03-15T13:07:28.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04e3827740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e308d6f90 con 0x556e309787b0 2024-03-15T13:07:28.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04cb7fe640 1 --2- 172.21.15.2:0/4041905663 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f04c40a1ba0 0x7f04c40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04da59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e30978270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04d9d9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 0x556e3097aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04da59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e30978270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60398/0 (socket says 172.21.15.2:60398) 2024-03-15T13:07:28.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04dad9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096f2b0 0x556e3096f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04da59c640 1 -- 172.21.15.2:0/738295568 learned_addr learned my addr 172.21.15.2:0/738295568 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:28.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04dad9d640 1 --2- 172.21.15.2:0/4041905663 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f04c40a1ba0 0x7f04c40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04cb7fe640 1 -- 172.21.15.2:0/4041905663 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f04cc0efb70 con 0x7f04dc101460 2024-03-15T13:07:28.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04d9d9b640 1 -- 172.21.15.2:0/738295568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096f2b0 msgr2=0x556e3096f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:28.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04d9d9b640 1 --2- 172.21.15.2:0/738295568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096f2b0 0x556e3096f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04d9d9b640 1 -- 172.21.15.2:0/738295568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 msgr2=0x556e30978270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:28.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04d9d9b640 1 --2- 172.21.15.2:0/738295568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e30978270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.145+0000 7f04d9d9b640 1 -- 172.21.15.2:0/738295568 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e30895570 con 0x556e309787b0 2024-03-15T13:07:28.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.146+0000 7f04dad9d640 1 --2- 172.21.15.2:0/4041905663 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f04c40a1ba0 0x7f04c40a4060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005fd0 tx=0x7f04d4009500 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:28.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.146+0000 7f04d9d9b640 1 --2- 172.21.15.2:0/738295568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 0x556e3097aba0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009bf0 tx=0x7f04d0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31c8ba381b51328d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:28.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.146+0000 7f04c8ff9640 1 -- 172.21.15.2:0/738295568 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d0004050 con 0x556e309787b0 2024-03-15T13:07:28.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.146+0000 7f04c8ff9640 1 -- 172.21.15.2:0/738295568 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f04d0005020 con 0x556e309787b0 2024-03-15T13:07:28.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.146+0000 7f04c8ff9640 1 -- 172.21.15.2:0/738295568 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d0004c60 con 0x556e309787b0 2024-03-15T13:07:28.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.146+0000 7f04e3827740 1 -- 172.21.15.2:0/738295568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 msgr2=0x556e3097aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:28.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.146+0000 7f04e3827740 1 --2- 172.21.15.2:0/738295568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 0x556e3097aba0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009bf0 tx=0x7f04d0011810 comp rx=0 tx=0).stop 2024-03-15T13:07:28.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 1 -- 172.21.15.2:0/738295568 shutdown_connections 2024-03-15T13:07:28.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 1 --2- 172.21.15.2:0/738295568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 0x556e3097aba0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 1 --2- 172.21.15.2:0/738295568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e30978270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 1 --2- 172.21.15.2:0/738295568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096f2b0 0x556e3096f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 1 -- 172.21.15.2:0/738295568 >> 172.21.15.2:0/738295568 conn(0x556e30969990 msgr2=0x556e30969d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:28.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 1 -- 172.21.15.2:0/738295568 shutdown_connections 2024-03-15T13:07:28.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 1 -- 172.21.15.2:0/738295568 wait complete. 2024-03-15T13:07:28.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 1 -- start start 2024-03-15T13:07:28.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 1 Processor -- start 2024-03-15T13:07:28.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096f2b0 0x556e30978210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e309755a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 0x556e30975cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e305aa430 con 0x556e3096fbd0 2024-03-15T13:07:28.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e30566e00 con 0x556e3096f2b0 2024-03-15T13:07:28.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3085bfd0 con 0x556e309787b0 2024-03-15T13:07:28.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04e3827740 10 client.0 _ll_register_callbacks cb 0x556e308b1fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:28.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04d9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e309755a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04d9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e309755a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60404/0 (socket says 172.21.15.2:60404) 2024-03-15T13:07:28.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.147+0000 7f04d9d9b640 1 -- 172.21.15.2:0/3928469042 learned_addr learned my addr 172.21.15.2:0/3928469042 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:28.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04dad9d640 1 --2- 172.21.15.2:0/3928469042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 0x556e30975cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04da59c640 1 --2- 172.21.15.2:0/3928469042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096f2b0 0x556e30978210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04d9d9b640 1 -- 172.21.15.2:0/3928469042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 msgr2=0x556e30975cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:28.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04d9d9b640 1 --2- 172.21.15.2:0/3928469042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 0x556e30975cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04d9d9b640 1 -- 172.21.15.2:0/3928469042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096f2b0 msgr2=0x556e30978210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:28.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04d9d9b640 1 --2- 172.21.15.2:0/3928469042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096f2b0 0x556e30978210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04d9d9b640 1 -- 172.21.15.2:0/3928469042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d0009850 con 0x556e3096fbd0 2024-03-15T13:07:28.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04d9d9b640 1 --2- 172.21.15.2:0/3928469042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e309755a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f04d0000c00 tx=0x7f04d0009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a299bd49b0396d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:28.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:28.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04c1ffb640 1 -- 172.21.15.2:0/3928469042 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d0004e30 con 0x556e3096fbd0 2024-03-15T13:07:28.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04c1ffb640 1 -- 172.21.15.2:0/3928469042 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f04d0004050 con 0x556e3096fbd0 2024-03-15T13:07:28.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04e3827740 1 -- 172.21.15.2:0/3928469042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e305d67e0 con 0x556e3096fbd0 2024-03-15T13:07:28.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04c1ffb640 1 -- 172.21.15.2:0/3928469042 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04d0004250 con 0x556e3096fbd0 2024-03-15T13:07:28.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04e3827740 10 client.18219 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:28.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04e3827740 1 -- 172.21.15.2:0/3928469042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x556e308d5610 con 0x556e3096fbd0 2024-03-15T13:07:28.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.148+0000 7f04c1ffb640 1 -- 172.21.15.2:0/3928469042 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x556e305d67e0 con 0x556e3096fbd0 2024-03-15T13:07:28.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.149+0000 7f04c1ffb640 1 -- 172.21.15.2:0/3928469042 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f04d0021040 con 0x556e3096fbd0 2024-03-15T13:07:28.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.149+0000 7f04e3827740 10 client.18219 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:28.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.149+0000 7f04e3827740 10 client.18219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:07:28.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.149+0000 7f04e3827740 20 client.18219 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:28.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.149+0000 7f04e3827740 1 --2- 172.21.15.2:0/3928469042 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556e30988930 0x556e3098ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.149+0000 7f04e3827740 4 client.18219 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:07:28.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.149+0000 7f04e3827740 1 -- 172.21.15.2:0/3928469042 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e30613720 con 0x556e30988930 2024-03-15T13:07:28.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.149+0000 7f04da59c640 1 --2- 172.21.15.2:0/3928469042 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556e30988930 0x556e3098ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.149+0000 7f04da59c640 1 --2- 172.21.15.2:0/3928469042 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556e30988930 0x556e3098ad50 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=763ed59dd57b5b35 server_cookie=3b943d76944e69d0 in_seq=0 out_seq=0 2024-03-15T13:07:28.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.149+0000 7f04c1ffb640 10 client.18219 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:28.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.150+0000 7f04c1ffb640 1 -- 172.21.15.2:0/3928469042 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556e30613720 con 0x556e30988930 2024-03-15T13:07:28.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.150+0000 7f04c1ffb640 10 client.18219 handle_command_reply: tid=0 2024-03-15T13:07:28.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.150+0000 7f04e3827740 1 client.18219 shutdown 2024-03-15T13:07:28.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.150+0000 7f04e3827740 1 -- 172.21.15.2:0/3928469042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 msgr2=0x556e309755a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:28.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.150+0000 7f04e3827740 1 --2- 172.21.15.2:0/3928469042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e309755a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f04d0000c00 tx=0x7f04d0009fd0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.150+0000 7f04e3827740 1 -- 172.21.15.2:0/3928469042 shutdown_connections 2024-03-15T13:07:28.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.150+0000 7f04e3827740 1 --2- 172.21.15.2:0/3928469042 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556e30988930 0x556e3098ad50 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.150+0000 7f04e3827740 1 --2- 172.21.15.2:0/3928469042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 0x556e30975cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.150+0000 7f04e3827740 1 --2- 172.21.15.2:0/3928469042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e309755a0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.150+0000 7f04c1ffb640 0 client.18219 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:28.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.150+0000 7f04e3827740 1 --2- 172.21.15.2:0/3928469042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096f2b0 0x556e30978210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.150+0000 7f04e3827740 1 -- 172.21.15.2:0/3928469042 >> 172.21.15.2:0/3928469042 conn(0x556e30969990 msgr2=0x556e3096b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:28.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.150+0000 7f04e3827740 1 -- 172.21.15.2:0/3928469042 shutdown_connections 2024-03-15T13:07:28.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.151+0000 7f04e3827740 1 -- 172.21.15.2:0/3928469042 wait complete. 2024-03-15T13:07:28.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.151+0000 7f04e3827740 20 client.18219 trim_cache size 0 max 16384 2024-03-15T13:07:28.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04e3827740 1 Processor -- start 2024-03-15T13:07:28.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04e3827740 1 -- start start 2024-03-15T13:07:28.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04e3827740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e3096f2b0 0x556e30975540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04e3827740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e30977cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04e3827740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e309787b0 0x556e30978210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04e3827740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3097b1c0 con 0x556e3096fbd0 2024-03-15T13:07:28.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04e3827740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3096aa10 con 0x556e309787b0 2024-03-15T13:07:28.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04e3827740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3096a7c0 con 0x556e3096f2b0 2024-03-15T13:07:28.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04d9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e30977cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04d9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e30977cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60410/0 (socket says 172.21.15.2:60410) 2024-03-15T13:07:28.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04d9d9b640 1 -- 172.21.15.2:0/2033253746 learned_addr learned my addr 172.21.15.2:0/2033253746 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:28.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04da59c640 1 --2- 172.21.15.2:0/2033253746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e3096f2b0 0x556e30975540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04dad9d640 1 --2- 172.21.15.2:0/2033253746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e309787b0 0x556e30978210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04d9d9b640 1 -- 172.21.15.2:0/2033253746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e3096f2b0 msgr2=0x556e30975540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:28.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04d9d9b640 1 --2- 172.21.15.2:0/2033253746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e3096f2b0 0x556e30975540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04d9d9b640 1 -- 172.21.15.2:0/2033253746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e309787b0 msgr2=0x556e30978210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:28.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04d9d9b640 1 --2- 172.21.15.2:0/2033253746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e309787b0 0x556e30978210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04d9d9b640 1 -- 172.21.15.2:0/2033253746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e30871ac0 con 0x556e3096fbd0 2024-03-15T13:07:28.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04dad9d640 1 --2- 172.21.15.2:0/2033253746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e309787b0 0x556e30978210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:07:28.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.167+0000 7f04d9d9b640 1 --2- 172.21.15.2:0/2033253746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e30977cd0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f04d0005e00 tx=0x7f04d0009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2047c6050f1470db server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:28.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.168+0000 7f04c3fff640 1 -- 172.21.15.2:0/2033253746 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d0005600 con 0x556e3096fbd0 2024-03-15T13:07:28.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.168+0000 7f04c3fff640 1 -- 172.21.15.2:0/2033253746 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f04d0004c30 con 0x556e3096fbd0 2024-03-15T13:07:28.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.168+0000 7f04c3fff640 1 -- 172.21.15.2:0/2033253746 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d002e6a0 con 0x556e3096fbd0 2024-03-15T13:07:28.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.168+0000 7f04e3827740 1 -- 172.21.15.2:0/2033253746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 msgr2=0x556e30977cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:28.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.168+0000 7f04e3827740 1 --2- 172.21.15.2:0/2033253746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e30977cd0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f04d0005e00 tx=0x7f04d0009a70 comp rx=0 tx=0).stop 2024-03-15T13:07:28.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.168+0000 7f04e3827740 1 -- 172.21.15.2:0/2033253746 shutdown_connections 2024-03-15T13:07:28.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.168+0000 7f04e3827740 1 --2- 172.21.15.2:0/2033253746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e309787b0 0x556e30978210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.168+0000 7f04e3827740 1 --2- 172.21.15.2:0/2033253746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096fbd0 0x556e30977cd0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.168+0000 7f04e3827740 1 --2- 172.21.15.2:0/2033253746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e3096f2b0 0x556e30975540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.168+0000 7f04e3827740 1 -- 172.21.15.2:0/2033253746 >> 172.21.15.2:0/2033253746 conn(0x556e30969990 msgr2=0x556e30974e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:28.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.168+0000 7f04e3827740 1 -- 172.21.15.2:0/2033253746 shutdown_connections 2024-03-15T13:07:28.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.168+0000 7f04e3827740 1 -- 172.21.15.2:0/2033253746 wait complete. 2024-03-15T13:07:28.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.168+0000 7f04e3827740 1 -- start start 2024-03-15T13:07:28.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04e3827740 1 Processor -- start 2024-03-15T13:07:28.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04e3827740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096f2b0 0x556e30978c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04e3827740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096fbd0 0x556e309791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04e3827740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 0x556e30979710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04e3827740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3097a7a0 con 0x556e3096f2b0 2024-03-15T13:07:28.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04e3827740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3097a910 con 0x556e3096fbd0 2024-03-15T13:07:28.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04e3827740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e3097f670 con 0x556e309787b0 2024-03-15T13:07:28.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04e3827740 10 client.0 _ll_register_callbacks cb 0x7f04cc0b8040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:28.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04da59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096f2b0 0x556e30978c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04da59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096f2b0 0x556e30978c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60412/0 (socket says 172.21.15.2:60412) 2024-03-15T13:07:28.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04da59c640 1 -- 172.21.15.2:0/2779992814 learned_addr learned my addr 172.21.15.2:0/2779992814 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:28.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04dad9d640 1 --2- 172.21.15.2:0/2779992814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 0x556e30979710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04d9d9b640 1 --2- 172.21.15.2:0/2779992814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096fbd0 0x556e309791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04da59c640 1 -- 172.21.15.2:0/2779992814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 msgr2=0x556e30979710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:28.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04da59c640 1 --2- 172.21.15.2:0/2779992814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 0x556e30979710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04da59c640 1 -- 172.21.15.2:0/2779992814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096fbd0 msgr2=0x556e309791d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:28.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04da59c640 1 --2- 172.21.15.2:0/2779992814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096fbd0 0x556e309791d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04da59c640 1 -- 172.21.15.2:0/2779992814 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e308d5610 con 0x556e3096f2b0 2024-03-15T13:07:28.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04d9d9b640 1 --2- 172.21.15.2:0/2779992814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096fbd0 0x556e309791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:07:28.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04da59c640 1 --2- 172.21.15.2:0/2779992814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096f2b0 0x556e30978c90 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f04cc02ce10 tx=0x7f04cc0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f61d44121eefff server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:28.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:28.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.169+0000 7f04c1ffb640 1 -- 172.21.15.2:0/2779992814 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc0bbe70 con 0x556e3096f2b0 2024-03-15T13:07:28.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.170+0000 7f04c1ffb640 1 -- 172.21.15.2:0/2779992814 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f04cc0bcc10 con 0x556e3096f2b0 2024-03-15T13:07:28.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.170+0000 7f04e3827740 1 -- 172.21.15.2:0/2779992814 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04d0005020 con 0x556e3096f2b0 2024-03-15T13:07:28.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.170+0000 7f04c1ffb640 1 -- 172.21.15.2:0/2779992814 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04cc0bc510 con 0x556e3096f2b0 2024-03-15T13:07:28.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.170+0000 7f04e3827740 10 client.18225 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:28.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.170+0000 7f04e3827740 1 -- 172.21.15.2:0/2779992814 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x556e308976a0 con 0x556e3096f2b0 2024-03-15T13:07:28.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.170+0000 7f04c1ffb640 1 -- 172.21.15.2:0/2779992814 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f04d0005020 con 0x556e3096f2b0 2024-03-15T13:07:28.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.170+0000 7f04c1ffb640 1 -- 172.21.15.2:0/2779992814 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f04cc03c070 con 0x556e3096f2b0 2024-03-15T13:07:28.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.170+0000 7f04e3827740 10 client.18225 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:28.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.170+0000 7f04e3827740 10 client.18225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:07:28.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.170+0000 7f04e3827740 20 client.18225 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:28.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.170+0000 7f04e3827740 1 --2- 172.21.15.2:0/2779992814 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556e309e46a0 0x556e309e6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:28.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.170+0000 7f04e3827740 4 client.18225 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:07:28.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.170+0000 7f04e3827740 1 -- 172.21.15.2:0/2779992814 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556e30662890 con 0x556e309e46a0 2024-03-15T13:07:28.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.170+0000 7f04d9d9b640 1 --2- 172.21.15.2:0/2779992814 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556e309e46a0 0x556e309e6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:28.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04c1ffb640 10 client.18225 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:28.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04d9d9b640 1 --2- 172.21.15.2:0/2779992814 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556e309e46a0 0x556e309e6ac0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c9109cb39f1bf58 server_cookie=77c1673315b8b0d3 in_seq=0 out_seq=0 2024-03-15T13:07:28.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04c1ffb640 1 -- 172.21.15.2:0/2779992814 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556e30662890 con 0x556e309e46a0 2024-03-15T13:07:28.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04c1ffb640 10 client.18225 handle_command_reply: tid=0 2024-03-15T13:07:28.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04e3827740 1 client.18225 shutdown 2024-03-15T13:07:28.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04e3827740 1 -- 172.21.15.2:0/2779992814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096f2b0 msgr2=0x556e30978c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:28.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04e3827740 1 --2- 172.21.15.2:0/2779992814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096f2b0 0x556e30978c90 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f04cc02ce10 tx=0x7f04cc0dc040 comp rx=0 tx=0).stop 2024-03-15T13:07:28.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04e3827740 1 -- 172.21.15.2:0/2779992814 shutdown_connections 2024-03-15T13:07:28.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04e3827740 1 --2- 172.21.15.2:0/2779992814 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556e309e46a0 0x556e309e6ac0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04e3827740 1 --2- 172.21.15.2:0/2779992814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e309787b0 0x556e30979710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04e3827740 1 --2- 172.21.15.2:0/2779992814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e3096fbd0 0x556e309791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04e3827740 1 --2- 172.21.15.2:0/2779992814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e3096f2b0 0x556e30978c90 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04c1ffb640 0 client.18225 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:28.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04e3827740 1 -- 172.21.15.2:0/2779992814 >> 172.21.15.2:0/2779992814 conn(0x556e30969990 msgr2=0x556e3096b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:28.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04e3827740 1 -- 172.21.15.2:0/2779992814 shutdown_connections 2024-03-15T13:07:28.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04e3827740 1 -- 172.21.15.2:0/2779992814 wait complete. 2024-03-15T13:07:28.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.171+0000 7f04e3827740 20 client.18225 trim_cache size 0 max 16384 2024-03-15T13:07:28.164 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:07:28.165 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T13:07:28.165 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:07:28.165 INFO:teuthology.orchestra.run.smithi002.stdout: "0fc12026-188f-46fa-87b9-baa531d6e6af": { 2024-03-15T13:07:28.165 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:07:28.165 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "0fc12026-188f-46fa-87b9-baa531d6e6af", 2024-03-15T13:07:28.165 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:07:28.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:07:28.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:07:28.165 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:07:28.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.172+0000 7f04e0c47640 1 -- 172.21.15.2:0/4041905663 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f04c40a1ba0 msgr2=0x7f04c40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:28.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.172+0000 7f04e0c47640 1 --2- 172.21.15.2:0/4041905663 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f04c40a1ba0 0x7f04c40a4060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005fd0 tx=0x7f04d4009500 comp rx=0 tx=0).stop 2024-03-15T13:07:28.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.172+0000 7f04e0c47640 1 -- 172.21.15.2:0/4041905663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04dc101460 msgr2=0x7f04dc0737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:28.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.172+0000 7f04e0c47640 1 --2- 172.21.15.2:0/4041905663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04dc101460 0x7f04dc0737f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009900 tx=0x7f04cc02ca20 comp rx=0 tx=0).stop 2024-03-15T13:07:28.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.172+0000 7f04e0c47640 1 -- 172.21.15.2:0/4041905663 shutdown_connections 2024-03-15T13:07:28.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.172+0000 7f04e0c47640 1 --2- 172.21.15.2:0/4041905663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04dc107500 0x7f04dc074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.172+0000 7f04e0c47640 1 --2- 172.21.15.2:0/4041905663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04dc101e20 0x7f04dc073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.172+0000 7f04e0c47640 1 --2- 172.21.15.2:0/4041905663 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f04c40a1ba0 0x7f04c40a4060 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.172+0000 7f04e0c47640 1 --2- 172.21.15.2:0/4041905663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04dc101460 0x7f04dc0737f0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:28.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.172+0000 7f04e0c47640 1 -- 172.21.15.2:0/4041905663 >> 172.21.15.2:0/4041905663 conn(0x7f04dc0fcf90 msgr2=0x7f04dc0fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:28.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.172+0000 7f04e0c47640 1 -- 172.21.15.2:0/4041905663 shutdown_connections 2024-03-15T13:07:28.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:28.172+0000 7f04e0c47640 1 -- 172.21.15.2:0/4041905663 wait complete. 2024-03-15T13:07:28.168 INFO:tasks.cephfs.filesystem:scrub status for tag:0fc12026-188f-46fa-87b9-baa531d6e6af - {'path': '/', 'tag': '0fc12026-188f-46fa-87b9-baa531d6e6af', 'options': 'recursive,force'} 2024-03-15T13:07:28.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:28 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:07:28.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:28 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:07:28.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:28 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:07:28.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:28 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:07:28.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:28 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:07:28.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:28 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:07:28.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:28 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:07:28.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:28 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:07:28.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:28 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:07:28.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:28 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:07:28.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:28 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:07:28.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:28 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:07:29.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:29 smithi002 ceph-mon[32060]: pgmap v264: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-15T13:07:29.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:29 smithi184 ceph-mon[35993]: pgmap v264: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-15T13:07:29.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:29 smithi149 ceph-mon[35927]: pgmap v264: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-15T13:07:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:30 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:07:30.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:30 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:07:30.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:30 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:07:31.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:31 smithi002 ceph-mon[32060]: pgmap v265: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-15T13:07:31.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:31 smithi184 ceph-mon[35993]: pgmap v265: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-15T13:07:31.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:31 smithi149 ceph-mon[35927]: pgmap v265: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-15T13:07:33.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:33 smithi002 ceph-mon[32060]: pgmap v266: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-15T13:07:33.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:33 smithi184 ceph-mon[35993]: pgmap v266: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-15T13:07:33.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:33 smithi149 ceph-mon[35927]: pgmap v266: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-15T13:07:35.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:35 smithi002 ceph-mon[32060]: pgmap v267: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:07:35.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:35 smithi184 ceph-mon[35993]: pgmap v267: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:07:35.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:35 smithi149 ceph-mon[35927]: pgmap v267: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:07:37.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:37 smithi002 ceph-mon[32060]: pgmap v268: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:07:37.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:37 smithi184 ceph-mon[35993]: pgmap v268: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:07:37.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:37 smithi149 ceph-mon[35927]: pgmap v268: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:07:39.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:39 smithi184 ceph-mon[35993]: pgmap v269: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:07:39.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:39 smithi149 ceph-mon[35927]: pgmap v269: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:07:40.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:39 smithi002 ceph-mon[32060]: pgmap v269: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:07:41.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:41 smithi184 ceph-mon[35993]: pgmap v270: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:07:41.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:41 smithi149 ceph-mon[35927]: pgmap v270: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:07:42.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:41 smithi002 ceph-mon[32060]: pgmap v270: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:07:43.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:43 smithi184 ceph-mon[35993]: pgmap v271: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:43.852 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:43 smithi002 ceph-mon[32060]: pgmap v271: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:43.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:43 smithi149 ceph-mon[35927]: pgmap v271: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:45.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:45 smithi149 ceph-mon[35927]: pgmap v272: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:46.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:45 smithi002 ceph-mon[32060]: pgmap v272: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:46.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:45 smithi184 ceph-mon[35993]: pgmap v272: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:47.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:47 smithi149 ceph-mon[35927]: pgmap v273: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:07:48.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:47 smithi002 ceph-mon[32060]: pgmap v273: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:07:48.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:47 smithi184 ceph-mon[35993]: pgmap v273: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:07:49.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:49 smithi149 ceph-mon[35927]: pgmap v274: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:50.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:49 smithi002 ceph-mon[32060]: pgmap v274: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:50.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:49 smithi184 ceph-mon[35993]: pgmap v274: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:51.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:51 smithi149 ceph-mon[35927]: pgmap v275: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:07:52.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:51 smithi002 ceph-mon[32060]: pgmap v275: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:07:52.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:51 smithi184 ceph-mon[35993]: pgmap v275: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:07:53.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:53 smithi149 ceph-mon[35927]: pgmap v276: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:54.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:53 smithi002 ceph-mon[32060]: pgmap v276: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:54.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:53 smithi184 ceph-mon[35993]: pgmap v276: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:55.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:55 smithi149 ceph-mon[35927]: pgmap v277: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:56.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:55 smithi002 ceph-mon[32060]: pgmap v277: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:56.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:55 smithi184 ceph-mon[35993]: pgmap v277: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:57.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:57 smithi149 ceph-mon[35927]: pgmap v278: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:07:58.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:57 smithi002 ceph-mon[32060]: pgmap v278: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:07:58.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:57 smithi184 ceph-mon[35993]: pgmap v278: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:07:58.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:07:58.169 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:07:58.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.301+0000 7f9805033640 1 -- 172.21.15.2:0/1469658402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9800069b20 msgr2=0x7f9800069fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.302+0000 7f9805033640 1 --2- 172.21.15.2:0/1469658402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9800069b20 0x7f9800069fa0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009930 tx=0x7f97f402bcd0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.302+0000 7f9805033640 1 -- 172.21.15.2:0/1469658402 shutdown_connections 2024-03-15T13:07:58.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.302+0000 7f9805033640 1 --2- 172.21.15.2:0/1469658402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f980006a4e0 0x7f980010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.302+0000 7f9805033640 1 --2- 172.21.15.2:0/1469658402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9800069b20 0x7f9800069fa0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.302+0000 7f9805033640 1 --2- 172.21.15.2:0/1469658402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9800107600 0x7f98001079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.302+0000 7f9805033640 1 -- 172.21.15.2:0/1469658402 >> 172.21.15.2:0/1469658402 conn(0x7f9800077190 msgr2=0x7f98000775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.302+0000 7f9805033640 1 -- 172.21.15.2:0/1469658402 shutdown_connections 2024-03-15T13:07:58.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.302+0000 7f9805033640 1 -- 172.21.15.2:0/1469658402 wait complete. 2024-03-15T13:07:58.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.302+0000 7f9805033640 1 Processor -- start 2024-03-15T13:07:58.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f9805033640 1 -- start start 2024-03-15T13:07:58.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f9805033640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9800069b20 0x7f980019d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f9805033640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f980006a4e0 0x7f980019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f9805033640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9800107600 0x7f9800197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f9805033640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9800197fe0 con 0x7f9800069b20 2024-03-15T13:07:58.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f9805033640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9800198150 con 0x7f9800107600 2024-03-15T13:07:58.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f9805033640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9800198310 con 0x7f980006a4e0 2024-03-15T13:07:58.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9800069b20 0x7f980019d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9800069b20 0x7f980019d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60116/0 (socket says 172.21.15.2:60116) 2024-03-15T13:07:58.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f97fed9d640 1 -- 172.21.15.2:0/103485350 learned_addr learned my addr 172.21.15.2:0/103485350 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f97fe59c640 1 --2- 172.21.15.2:0/103485350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f980006a4e0 0x7f980019dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f97ff59e640 1 --2- 172.21.15.2:0/103485350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9800107600 0x7f9800197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f97fed9d640 1 -- 172.21.15.2:0/103485350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f980006a4e0 msgr2=0x7f980019dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f97fed9d640 1 --2- 172.21.15.2:0/103485350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f980006a4e0 0x7f980019dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f97fed9d640 1 -- 172.21.15.2:0/103485350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9800107600 msgr2=0x7f9800197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f97fed9d640 1 --2- 172.21.15.2:0/103485350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9800107600 0x7f9800197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f97fed9d640 1 -- 172.21.15.2:0/103485350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f4009560 con 0x7f9800069b20 2024-03-15T13:07:58.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.303+0000 7f97fed9d640 1 --2- 172.21.15.2:0/103485350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9800069b20 0x7f980019d790 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f97f000e9f0 tx=0x7f97f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.304+0000 7f97e7fff640 1 -- 172.21.15.2:0/103485350 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0009720 con 0x7f9800069b20 2024-03-15T13:07:58.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.304+0000 7f9805033640 1 -- 172.21.15.2:0/103485350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98001985f0 con 0x7f9800069b20 2024-03-15T13:07:58.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.304+0000 7f97e7fff640 1 -- 172.21.15.2:0/103485350 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f97f000cd80 con 0x7f9800069b20 2024-03-15T13:07:58.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.304+0000 7f97e7fff640 1 -- 172.21.15.2:0/103485350 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0010690 con 0x7f9800069b20 2024-03-15T13:07:58.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.304+0000 7f9805033640 1 -- 172.21.15.2:0/103485350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98001a4830 con 0x7f9800069b20 2024-03-15T13:07:58.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.306+0000 7f97e7fff640 1 -- 172.21.15.2:0/103485350 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f97f0004e90 con 0x7f9800069b20 2024-03-15T13:07:58.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.306+0000 7f9807c0c740 1 Processor -- start 2024-03-15T13:07:58.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.306+0000 7f9807c0c740 1 -- start start 2024-03-15T13:07:58.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.306+0000 7f9807c0c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a2a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.306+0000 7f9807c0c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a2a900 0x556626a37680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.306+0000 7f9807c0c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a37bc0 0x556626a39fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.306+0000 7f9807c0c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556626999280 con 0x556626a29fe0 2024-03-15T13:07:58.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.306+0000 7f9807c0c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566269967b0 con 0x556626a2a900 2024-03-15T13:07:58.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.306+0000 7f9807c0c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556626991f90 con 0x556626a37bc0 2024-03-15T13:07:58.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.306+0000 7f97e7fff640 1 --2- 172.21.15.2:0/103485350 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f97e80a1ba0 0x7f97e80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.306+0000 7f97ff59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a2a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.306+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a2a900 0x556626a37680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.306+0000 7f97fe59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a37bc0 0x556626a39fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.306+0000 7f97fe59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a37bc0 0x556626a39fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:35018/0 (socket says 172.21.15.2:35018) 2024-03-15T13:07:58.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.306+0000 7f97ff59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a2a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60118/0 (socket says 172.21.15.2:60118) 2024-03-15T13:07:58.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.307+0000 7f97fe59c640 1 -- 172.21.15.2:0/2065928491 learned_addr learned my addr 172.21.15.2:0/2065928491 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.307+0000 7f97ff59e640 1 --2- 172.21.15.2:0/103485350 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f97e80a1ba0 0x7f97e80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.307+0000 7f97e7fff640 1 -- 172.21.15.2:0/103485350 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f97f0014070 con 0x7f9800069b20 2024-03-15T13:07:58.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.307+0000 7f97fe59c640 1 -- 172.21.15.2:0/2065928491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a2a900 msgr2=0x556626a37680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.307+0000 7f97fe59c640 1 --2- 172.21.15.2:0/2065928491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a2a900 0x556626a37680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.307+0000 7f97fe59c640 1 -- 172.21.15.2:0/2065928491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 msgr2=0x556626a2a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.307+0000 7f97fe59c640 1 --2- 172.21.15.2:0/2065928491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a2a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.307+0000 7f97fe59c640 1 -- 172.21.15.2:0/2065928491 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556626950570 con 0x556626a37bc0 2024-03-15T13:07:58.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.307+0000 7f97fed9d640 1 --2- 172.21.15.2:0/2065928491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a2a900 0x556626a37680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:58.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.307+0000 7f97ff59e640 1 --2- 172.21.15.2:0/103485350 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f97e80a1ba0 0x7f97e80a4060 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f97f8005c40 tx=0x7f97f8009500 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.307+0000 7f97fe59c640 1 --2- 172.21.15.2:0/2065928491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a37bc0 0x556626a39fb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009230 tx=0x7f97f402c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c8808435b8e5386 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.307+0000 7f97e57fa640 1 -- 172.21.15.2:0/2065928491 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f40046c0 con 0x556626a37bc0 2024-03-15T13:07:58.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.307+0000 7f97e57fa640 1 -- 172.21.15.2:0/2065928491 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f97f4004820 con 0x556626a37bc0 2024-03-15T13:07:58.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.307+0000 7f97e57fa640 1 -- 172.21.15.2:0/2065928491 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f4005040 con 0x556626a37bc0 2024-03-15T13:07:58.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.308+0000 7f9807c0c740 1 -- 172.21.15.2:0/2065928491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a37bc0 msgr2=0x556626a39fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.308+0000 7f9807c0c740 1 --2- 172.21.15.2:0/2065928491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a37bc0 0x556626a39fb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009230 tx=0x7f97f402c670 comp rx=0 tx=0).stop 2024-03-15T13:07:58.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.308+0000 7f9807c0c740 1 -- 172.21.15.2:0/2065928491 shutdown_connections 2024-03-15T13:07:58.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.308+0000 7f9807c0c740 1 --2- 172.21.15.2:0/2065928491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a37bc0 0x556626a39fb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.308+0000 7f9807c0c740 1 --2- 172.21.15.2:0/2065928491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a2a900 0x556626a37680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.308+0000 7f9807c0c740 1 --2- 172.21.15.2:0/2065928491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a2a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.308+0000 7f9807c0c740 1 -- 172.21.15.2:0/2065928491 >> 172.21.15.2:0/2065928491 conn(0x556626a24990 msgr2=0x556626a24d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.308+0000 7f9807c0c740 1 -- 172.21.15.2:0/2065928491 shutdown_connections 2024-03-15T13:07:58.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.308+0000 7f9807c0c740 1 -- 172.21.15.2:0/2065928491 wait complete. 2024-03-15T13:07:58.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.308+0000 7f9807c0c740 1 -- start start 2024-03-15T13:07:58.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.308+0000 7f9807c0c740 1 Processor -- start 2024-03-15T13:07:58.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f9807c0c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a29fe0 0x556626a37620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f9807c0c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a2a900 0x556626a349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f9807c0c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 0x556626a350d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f9807c0c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556626665430 con 0x556626a2a900 2024-03-15T13:07:58.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f9807c0c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556626621e00 con 0x556626a37bc0 2024-03-15T13:07:58.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f9807c0c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556626916fd0 con 0x556626a29fe0 2024-03-15T13:07:58.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f9807c0c740 10 client.0 _ll_register_callbacks cb 0x55662696cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:58.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f97fe59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a2a900 0x556626a349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f97fe59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a2a900 0x556626a349b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60126/0 (socket says 172.21.15.2:60126) 2024-03-15T13:07:58.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f97fe59c640 1 -- 172.21.15.2:0/3307816037 learned_addr learned my addr 172.21.15.2:0/3307816037 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f97ff59e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 0x556626a350d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f97fed9d640 1 --2- 172.21.15.2:0/3307816037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a29fe0 0x556626a37620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f97fe59c640 1 -- 172.21.15.2:0/3307816037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a29fe0 msgr2=0x556626a37620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f97fe59c640 1 --2- 172.21.15.2:0/3307816037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a29fe0 0x556626a37620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f97fe59c640 1 -- 172.21.15.2:0/3307816037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 msgr2=0x556626a350d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f97fe59c640 1 --2- 172.21.15.2:0/3307816037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 0x556626a350d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f97fe59c640 1 -- 172.21.15.2:0/3307816037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980010eba0 con 0x556626a2a900 2024-03-15T13:07:58.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f97ff59e640 1 --2- 172.21.15.2:0/3307816037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 0x556626a350d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:58.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f97fe59c640 1 --2- 172.21.15.2:0/3307816037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a2a900 0x556626a349b0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009900 tx=0x7f97f4004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553f1e9970e8c2e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.309+0000 7f97e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:58.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.310+0000 7f97e2ff5640 1 -- 172.21.15.2:0/3307816037 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f40051f0 con 0x556626a2a900 2024-03-15T13:07:58.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.310+0000 7f97e2ff5640 1 -- 172.21.15.2:0/3307816037 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f97f4005350 con 0x556626a2a900 2024-03-15T13:07:58.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.310+0000 7f9807c0c740 1 -- 172.21.15.2:0/3307816037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566266917e0 con 0x556626a2a900 2024-03-15T13:07:58.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.310+0000 7f97e2ff5640 1 -- 172.21.15.2:0/3307816037 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97f40044e0 con 0x556626a2a900 2024-03-15T13:07:58.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.310+0000 7f9807c0c740 10 client.18237 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:58.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.310+0000 7f9807c0c740 1 -- 172.21.15.2:0/3307816037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x556626990610 con 0x556626a2a900 2024-03-15T13:07:58.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.310+0000 7f97e2ff5640 1 -- 172.21.15.2:0/3307816037 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5566266917e0 con 0x556626a2a900 2024-03-15T13:07:58.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.310+0000 7f97e2ff5640 1 -- 172.21.15.2:0/3307816037 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f97f4052030 con 0x556626a2a900 2024-03-15T13:07:58.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.310+0000 7f9807c0c740 10 client.18237 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:58.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.310+0000 7f9807c0c740 10 client.18237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:07:58.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.310+0000 7f9807c0c740 20 client.18237 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:58.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.310+0000 7f9807c0c740 1 --2- 172.21.15.2:0/3307816037 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556626a479e0 0x556626a49e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.310+0000 7f9807c0c740 4 client.18237 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:07:58.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.310+0000 7f9807c0c740 1 -- 172.21.15.2:0/3307816037 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566266ce720 con 0x556626a479e0 2024-03-15T13:07:58.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.310+0000 7f97fed9d640 1 --2- 172.21.15.2:0/3307816037 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556626a479e0 0x556626a49e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.311+0000 7f97fed9d640 1 --2- 172.21.15.2:0/3307816037 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556626a479e0 0x556626a49e00 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3beb9c80e332010d server_cookie=8086a3920ca295e in_seq=0 out_seq=0 2024-03-15T13:07:58.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.311+0000 7f97e2ff5640 10 client.18237 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:58.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.312+0000 7f97e2ff5640 1 -- 172.21.15.2:0/3307816037 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5566266ce720 con 0x556626a479e0 2024-03-15T13:07:58.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.312+0000 7f97e2ff5640 10 client.18237 handle_command_reply: tid=0 2024-03-15T13:07:58.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.312+0000 7f9807c0c740 1 client.18237 shutdown 2024-03-15T13:07:58.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.312+0000 7f9807c0c740 1 -- 172.21.15.2:0/3307816037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a2a900 msgr2=0x556626a349b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.312+0000 7f9807c0c740 1 --2- 172.21.15.2:0/3307816037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a2a900 0x556626a349b0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009900 tx=0x7f97f4004bc0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.312+0000 7f9807c0c740 1 -- 172.21.15.2:0/3307816037 shutdown_connections 2024-03-15T13:07:58.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.312+0000 7f9807c0c740 1 --2- 172.21.15.2:0/3307816037 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556626a479e0 0x556626a49e00 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.312+0000 7f9807c0c740 1 --2- 172.21.15.2:0/3307816037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 0x556626a350d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.312+0000 7f9807c0c740 1 --2- 172.21.15.2:0/3307816037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a2a900 0x556626a349b0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.312+0000 7f97e2ff5640 0 client.18237 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:58.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.312+0000 7f9807c0c740 1 --2- 172.21.15.2:0/3307816037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a29fe0 0x556626a37620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.312+0000 7f9807c0c740 1 -- 172.21.15.2:0/3307816037 >> 172.21.15.2:0/3307816037 conn(0x556626a24990 msgr2=0x556626a26730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.312+0000 7f9807c0c740 1 -- 172.21.15.2:0/3307816037 shutdown_connections 2024-03-15T13:07:58.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.312+0000 7f9807c0c740 1 -- 172.21.15.2:0/3307816037 wait complete. 2024-03-15T13:07:58.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.312+0000 7f9807c0c740 20 client.18237 trim_cache size 0 max 16384 2024-03-15T13:07:58.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.328+0000 7f9807c0c740 1 Processor -- start 2024-03-15T13:07:58.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.328+0000 7f9807c0c740 1 -- start start 2024-03-15T13:07:58.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.328+0000 7f9807c0c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a34950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f9807c0c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a2a900 0x556626a370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f9807c0c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 0x556626a37620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f9807c0c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556626a200f0 con 0x556626a29fe0 2024-03-15T13:07:58.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f9807c0c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556626a25c10 con 0x556626a37bc0 2024-03-15T13:07:58.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f9807c0c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556626a3a580 con 0x556626a2a900 2024-03-15T13:07:58.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a34950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a34950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60136/0 (socket says 172.21.15.2:60136) 2024-03-15T13:07:58.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f97ff59e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 0x556626a37620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f97fed9d640 1 -- 172.21.15.2:0/2335655633 learned_addr learned my addr 172.21.15.2:0/2335655633 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f97fe59c640 1 --2- 172.21.15.2:0/2335655633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a2a900 0x556626a370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f97fed9d640 1 -- 172.21.15.2:0/2335655633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a2a900 msgr2=0x556626a370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f97fed9d640 1 --2- 172.21.15.2:0/2335655633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a2a900 0x556626a370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f97fed9d640 1 -- 172.21.15.2:0/2335655633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 msgr2=0x556626a37620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f97fed9d640 1 --2- 172.21.15.2:0/2335655633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 0x556626a37620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f97fed9d640 1 -- 172.21.15.2:0/2335655633 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55662692cac0 con 0x556626a29fe0 2024-03-15T13:07:58.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f97ff59e640 1 --2- 172.21.15.2:0/2335655633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 0x556626a37620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:58.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f97fed9d640 1 --2- 172.21.15.2:0/2335655633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a34950 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f97f00cb290 tx=0x7f97f00ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913d6052640a6fdf server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f97e4ff9640 1 -- 172.21.15.2:0/2335655633 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f009a710 con 0x556626a29fe0 2024-03-15T13:07:58.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f97e4ff9640 1 -- 172.21.15.2:0/2335655633 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f97f0083e90 con 0x556626a29fe0 2024-03-15T13:07:58.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.329+0000 7f97e4ff9640 1 -- 172.21.15.2:0/2335655633 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0098a10 con 0x556626a29fe0 2024-03-15T13:07:58.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.330+0000 7f9807c0c740 1 -- 172.21.15.2:0/2335655633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 msgr2=0x556626a34950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.330+0000 7f9807c0c740 1 --2- 172.21.15.2:0/2335655633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a34950 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f97f00cb290 tx=0x7f97f00ba040 comp rx=0 tx=0).stop 2024-03-15T13:07:58.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.330+0000 7f9807c0c740 1 -- 172.21.15.2:0/2335655633 shutdown_connections 2024-03-15T13:07:58.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.330+0000 7f9807c0c740 1 --2- 172.21.15.2:0/2335655633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 0x556626a37620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.330+0000 7f9807c0c740 1 --2- 172.21.15.2:0/2335655633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a2a900 0x556626a370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.330+0000 7f9807c0c740 1 --2- 172.21.15.2:0/2335655633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a34950 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.330+0000 7f9807c0c740 1 -- 172.21.15.2:0/2335655633 >> 172.21.15.2:0/2335655633 conn(0x556626a24990 msgr2=0x556626a34240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.330+0000 7f9807c0c740 1 -- 172.21.15.2:0/2335655633 shutdown_connections 2024-03-15T13:07:58.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.330+0000 7f9807c0c740 1 -- 172.21.15.2:0/2335655633 wait complete. 2024-03-15T13:07:58.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.330+0000 7f9807c0c740 1 -- start start 2024-03-15T13:07:58.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.330+0000 7f9807c0c740 1 Processor -- start 2024-03-15T13:07:58.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f9807c0c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a3f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f9807c0c740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a2a900 0x556626a31ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f9807c0c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 0x556626a32000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f9807c0c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556626a32540 con 0x556626a29fe0 2024-03-15T13:07:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f9807c0c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556626a326b0 con 0x556626a37bc0 2024-03-15T13:07:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f9807c0c740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556626a32870 con 0x556626a2a900 2024-03-15T13:07:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f9807c0c740 10 client.0 _ll_register_callbacks cb 0x7f97f0005780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a3f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f97fe59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a2a900 0x556626a31ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f97fe59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a2a900 0x556626a31ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:35036/0 (socket says 172.21.15.2:35036) 2024-03-15T13:07:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f97fed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a3f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60148/0 (socket says 172.21.15.2:60148) 2024-03-15T13:07:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f97fe59c640 1 -- 172.21.15.2:0/3384385592 learned_addr learned my addr 172.21.15.2:0/3384385592 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f97ff59e640 1 --2- 172.21.15.2:0/3384385592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 0x556626a32000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f97fe59c640 1 -- 172.21.15.2:0/3384385592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 msgr2=0x556626a32000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f97fe59c640 1 --2- 172.21.15.2:0/3384385592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 0x556626a32000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f97fe59c640 1 -- 172.21.15.2:0/3384385592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 msgr2=0x556626a3f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f97fe59c640 1 --2- 172.21.15.2:0/3384385592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a3f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f97fe59c640 1 -- 172.21.15.2:0/3384385592 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f8019040 con 0x556626a2a900 2024-03-15T13:07:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f97fed9d640 1 --2- 172.21.15.2:0/3384385592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a3f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f97fe59c640 1 --2- 172.21.15.2:0/3384385592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a2a900 0x556626a31ac0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f97f4001eb0 tx=0x7f97f4005220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaa66d0bfdeb65b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f97e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:07:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.331+0000 7f97e2ff5640 1 -- 172.21.15.2:0/3384385592 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f4005590 con 0x556626a2a900 2024-03-15T13:07:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.332+0000 7f9807c0c740 1 -- 172.21.15.2:0/3384385592 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97f4004820 con 0x556626a2a900 2024-03-15T13:07:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.332+0000 7f97e2ff5640 1 -- 172.21.15.2:0/3384385592 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f97f404a450 con 0x556626a2a900 2024-03-15T13:07:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.332+0000 7f97e2ff5640 1 -- 172.21.15.2:0/3384385592 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97f402c550 con 0x556626a2a900 2024-03-15T13:07:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.332+0000 7f9807c0c740 10 client.28036 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.332+0000 7f9807c0c740 1 -- 172.21.15.2:0/3384385592 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f98001985f0 con 0x556626a2a900 2024-03-15T13:07:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.332+0000 7f97e2ff5640 1 -- 172.21.15.2:0/3384385592 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f97f4004820 con 0x556626a2a900 2024-03-15T13:07:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.332+0000 7f97e2ff5640 1 -- 172.21.15.2:0/3384385592 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f97f4052040 con 0x556626a2a900 2024-03-15T13:07:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.332+0000 7f9807c0c740 10 client.28036 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.332+0000 7f9807c0c740 10 client.28036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:07:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.332+0000 7f9807c0c740 20 client.28036 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.332+0000 7f9807c0c740 1 --2- 172.21.15.2:0/3384385592 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556626aa4220 0x556626aa6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.332+0000 7f9807c0c740 4 client.28036 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:07:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.332+0000 7f9807c0c740 1 -- 172.21.15.2:0/3384385592 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55662671d890 con 0x556626aa4220 2024-03-15T13:07:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.332+0000 7f97fed9d640 1 --2- 172.21.15.2:0/3384385592 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556626aa4220 0x556626aa6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f97e2ff5640 10 client.28036 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f97fed9d640 1 --2- 172.21.15.2:0/3384385592 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556626aa4220 0x556626aa6610 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36cfbb31a4d76dfa server_cookie=bbad0343c8b18b5b in_seq=0 out_seq=0 2024-03-15T13:07:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f97e2ff5640 1 -- 172.21.15.2:0/3384385592 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55662671d890 con 0x556626aa4220 2024-03-15T13:07:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f97e2ff5640 10 client.28036 handle_command_reply: tid=0 2024-03-15T13:07:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f9807c0c740 1 client.28036 shutdown 2024-03-15T13:07:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f9807c0c740 1 -- 172.21.15.2:0/3384385592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a2a900 msgr2=0x556626a31ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f9807c0c740 1 --2- 172.21.15.2:0/3384385592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a2a900 0x556626a31ac0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f97f4001eb0 tx=0x7f97f4005220 comp rx=0 tx=0).stop 2024-03-15T13:07:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f9807c0c740 1 -- 172.21.15.2:0/3384385592 shutdown_connections 2024-03-15T13:07:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f9807c0c740 1 --2- 172.21.15.2:0/3384385592 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x556626aa4220 0x556626aa6610 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f9807c0c740 1 --2- 172.21.15.2:0/3384385592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556626a37bc0 0x556626a32000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f9807c0c740 1 --2- 172.21.15.2:0/3384385592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556626a2a900 0x556626a31ac0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f9807c0c740 1 --2- 172.21.15.2:0/3384385592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556626a29fe0 0x556626a3f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f97e2ff5640 0 client.28036 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f9807c0c740 1 -- 172.21.15.2:0/3384385592 >> 172.21.15.2:0/3384385592 conn(0x556626a24990 msgr2=0x556626a25da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f9807c0c740 1 -- 172.21.15.2:0/3384385592 shutdown_connections 2024-03-15T13:07:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f9807c0c740 1 -- 172.21.15.2:0/3384385592 wait complete. 2024-03-15T13:07:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.333+0000 7f9807c0c740 20 client.28036 trim_cache size 0 max 16384 2024-03-15T13:07:58.326 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:07:58.326 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:07:58.326 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:07:58.326 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:07:58.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.334+0000 7f9805033640 1 -- 172.21.15.2:0/103485350 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f97e80a1ba0 msgr2=0x7f97e80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.334+0000 7f9805033640 1 --2- 172.21.15.2:0/103485350 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f97e80a1ba0 0x7f97e80a4060 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f97f8005c40 tx=0x7f97f8009500 comp rx=0 tx=0).stop 2024-03-15T13:07:58.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.334+0000 7f9805033640 1 -- 172.21.15.2:0/103485350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9800069b20 msgr2=0x7f980019d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.334+0000 7f9805033640 1 --2- 172.21.15.2:0/103485350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9800069b20 0x7f980019d790 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f97f000e9f0 tx=0x7f97f000edf0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.334+0000 7f9805033640 1 -- 172.21.15.2:0/103485350 shutdown_connections 2024-03-15T13:07:58.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.334+0000 7f9805033640 1 --2- 172.21.15.2:0/103485350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9800107600 0x7f9800197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.334+0000 7f9805033640 1 --2- 172.21.15.2:0/103485350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f980006a4e0 0x7f980019dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.334+0000 7f9805033640 1 --2- 172.21.15.2:0/103485350 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f97e80a1ba0 0x7f97e80a4060 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.334+0000 7f9805033640 1 --2- 172.21.15.2:0/103485350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9800069b20 0x7f980019d790 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.334+0000 7f9805033640 1 -- 172.21.15.2:0/103485350 >> 172.21.15.2:0/103485350 conn(0x7f9800077190 msgr2=0x7f98000ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.334+0000 7f9805033640 1 -- 172.21.15.2:0/103485350 shutdown_connections 2024-03-15T13:07:58.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.334+0000 7f9805033640 1 -- 172.21.15.2:0/103485350 wait complete. 2024-03-15T13:07:58.329 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:07:58.329 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:07:58.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.448+0000 7ff923c23640 1 -- 172.21.15.2:0/3093114326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff91c100620 msgr2=0x7ff91c100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.448+0000 7ff923c23640 1 --2- 172.21.15.2:0/3093114326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff91c100620 0x7ff91c100a00 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff918009930 tx=0x7ff91802bc30 comp rx=0 tx=0).stop 2024-03-15T13:07:58.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.448+0000 7ff923c23640 1 -- 172.21.15.2:0/3093114326 shutdown_connections 2024-03-15T13:07:58.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.448+0000 7ff923c23640 1 --2- 172.21.15.2:0/3093114326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff91c1056f0 0x7ff91c10d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.448+0000 7ff923c23640 1 --2- 172.21.15.2:0/3093114326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff91c100fd0 0x7ff91c105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.448+0000 7ff923c23640 1 --2- 172.21.15.2:0/3093114326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff91c100620 0x7ff91c100a00 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.448+0000 7ff923c23640 1 -- 172.21.15.2:0/3093114326 >> 172.21.15.2:0/3093114326 conn(0x7ff91c077490 msgr2=0x7ff91c0778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.448+0000 7ff923c23640 1 -- 172.21.15.2:0/3093114326 shutdown_connections 2024-03-15T13:07:58.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.448+0000 7ff923c23640 1 -- 172.21.15.2:0/3093114326 wait complete. 2024-03-15T13:07:58.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.448+0000 7ff923c23640 1 Processor -- start 2024-03-15T13:07:58.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.448+0000 7ff923c23640 1 -- start start 2024-03-15T13:07:58.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff923c23640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff91c100620 0x7ff91c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff923c23640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff91c1056f0 0x7ff91c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff923c23640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff91c1968a0 0x7ff91c196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff923c23640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c197290 con 0x7ff91c1056f0 2024-03-15T13:07:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff923c23640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c197400 con 0x7ff91c100620 2024-03-15T13:07:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff923c23640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c1975c0 con 0x7ff91c1968a0 2024-03-15T13:07:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff9211be640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff91c1056f0 0x7ff91c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff9211be640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff91c1056f0 0x7ff91c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60172/0 (socket says 172.21.15.2:60172) 2024-03-15T13:07:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff9211be640 1 -- 172.21.15.2:0/3996213136 learned_addr learned my addr 172.21.15.2:0/3996213136 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff9219bf640 1 --2- 172.21.15.2:0/3996213136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff91c100620 0x7ff91c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff9221c0640 1 --2- 172.21.15.2:0/3996213136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff91c1968a0 0x7ff91c196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff9211be640 1 -- 172.21.15.2:0/3996213136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff91c1968a0 msgr2=0x7ff91c196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff9211be640 1 --2- 172.21.15.2:0/3996213136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff91c1968a0 0x7ff91c196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff9211be640 1 -- 172.21.15.2:0/3996213136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff91c100620 msgr2=0x7ff91c19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff9211be640 1 --2- 172.21.15.2:0/3996213136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff91c100620 0x7ff91c19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff9211be640 1 -- 172.21.15.2:0/3996213136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c009490 con 0x7ff91c1056f0 2024-03-15T13:07:58.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff9219bf640 1 --2- 172.21.15.2:0/3996213136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff91c100620 0x7ff91c19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:58.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff9211be640 1 --2- 172.21.15.2:0/3996213136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff91c1056f0 0x7ff91c19cc30 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00e9a0 tx=0x7ff90c00eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff912ffd640 1 -- 172.21.15.2:0/3996213136 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c00cd80 con 0x7ff91c1056f0 2024-03-15T13:07:58.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff912ffd640 1 -- 172.21.15.2:0/3996213136 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff90c0035f0 con 0x7ff91c1056f0 2024-03-15T13:07:58.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff923c23640 1 -- 172.21.15.2:0/3996213136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff914009840 con 0x7ff91c1056f0 2024-03-15T13:07:58.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.449+0000 7ff912ffd640 1 -- 172.21.15.2:0/3996213136 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c010690 con 0x7ff91c1056f0 2024-03-15T13:07:58.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.450+0000 7ff923c23640 1 -- 172.21.15.2:0/3996213136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff918009560 con 0x7ff91c1056f0 2024-03-15T13:07:58.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.450+0000 7ff923c23640 1 -- 172.21.15.2:0/3996213136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91c068730 con 0x7ff91c1056f0 2024-03-15T13:07:58.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.452+0000 7ff912ffd640 1 -- 172.21.15.2:0/3996213136 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7ff90c003760 con 0x7ff91c1056f0 2024-03-15T13:07:58.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.452+0000 7ff912ffd640 1 --2- 172.21.15.2:0/3996213136 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff9080a1c20 0x7ff9080a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.452+0000 7ff9219bf640 1 --2- 172.21.15.2:0/3996213136 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff9080a1c20 0x7ff9080a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.452+0000 7ff912ffd640 1 -- 172.21.15.2:0/3996213136 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ff90c014070 con 0x7ff91c1056f0 2024-03-15T13:07:58.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.453+0000 7ff9219bf640 1 --2- 172.21.15.2:0/3996213136 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff9080a1c20 0x7ff9080a40e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff918000c00 tx=0x7ff918005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.456+0000 7ff912ffd640 1 -- 172.21.15.2:0/3996213136 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff90c096c10 con 0x7ff91c1056f0 2024-03-15T13:07:58.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.674+0000 7ff923c23640 1 -- 172.21.15.2:0/3996213136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff91c197dd0 con 0x7ff91c1056f0 2024-03-15T13:07:58.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.674+0000 7ff912ffd640 1 -- 172.21.15.2:0/3996213136 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+11453 (secure 0 0 0) 0x7ff90c09a2d0 con 0x7ff91c1056f0 2024-03-15T13:07:58.659 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:07:58.660 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:05:12.138845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24500,24464,24460,24434,24539]},"id":1}]} 2024-03-15T13:07:58.660 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 50 2024-03-15T13:07:58.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.676+0000 7ff923c23640 1 -- 172.21.15.2:0/3996213136 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff9080a1c20 msgr2=0x7ff9080a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.676+0000 7ff923c23640 1 --2- 172.21.15.2:0/3996213136 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff9080a1c20 0x7ff9080a40e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff918000c00 tx=0x7ff918005fd0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.676+0000 7ff923c23640 1 -- 172.21.15.2:0/3996213136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff91c1056f0 msgr2=0x7ff91c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.676+0000 7ff923c23640 1 --2- 172.21.15.2:0/3996213136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff91c1056f0 0x7ff91c19cc30 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00e9a0 tx=0x7ff90c00eda0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.676+0000 7ff923c23640 1 -- 172.21.15.2:0/3996213136 shutdown_connections 2024-03-15T13:07:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.676+0000 7ff923c23640 1 --2- 172.21.15.2:0/3996213136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff91c1968a0 0x7ff91c196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.676+0000 7ff923c23640 1 --2- 172.21.15.2:0/3996213136 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff9080a1c20 0x7ff9080a40e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.676+0000 7ff923c23640 1 --2- 172.21.15.2:0/3996213136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff91c1056f0 0x7ff91c19cc30 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.677+0000 7ff923c23640 1 --2- 172.21.15.2:0/3996213136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff91c100620 0x7ff91c19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.677+0000 7ff923c23640 1 -- 172.21.15.2:0/3996213136 >> 172.21.15.2:0/3996213136 conn(0x7ff91c077490 msgr2=0x7ff91c1068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.677+0000 7ff923c23640 1 -- 172.21.15.2:0/3996213136 shutdown_connections 2024-03-15T13:07:58.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.677+0000 7ff923c23640 1 -- 172.21.15.2:0/3996213136 wait complete. 2024-03-15T13:07:58.672 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:07:58.672 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:07:58.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.793+0000 7fa81cf2c640 1 -- 172.21.15.2:0/4140673944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8181056e0 msgr2=0x7fa81810d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.793+0000 7fa81cf2c640 1 --2- 172.21.15.2:0/4140673944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8181056e0 0x7fa81810d2b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa808009cd0 tx=0x7fa80802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.793+0000 7fa81cf2c640 1 -- 172.21.15.2:0/4140673944 shutdown_connections 2024-03-15T13:07:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.793+0000 7fa81cf2c640 1 --2- 172.21.15.2:0/4140673944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8181056e0 0x7fa81810d2b0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.793+0000 7fa81cf2c640 1 --2- 172.21.15.2:0/4140673944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa818100fc0 0x7fa818105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.793+0000 7fa81cf2c640 1 --2- 172.21.15.2:0/4140673944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa818100610 0x7fa8181009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.793+0000 7fa81cf2c640 1 -- 172.21.15.2:0/4140673944 >> 172.21.15.2:0/4140673944 conn(0x7fa818076770 msgr2=0x7fa818076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.793+0000 7fa81cf2c640 1 -- 172.21.15.2:0/4140673944 shutdown_connections 2024-03-15T13:07:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.793+0000 7fa81cf2c640 1 -- 172.21.15.2:0/4140673944 wait complete. 2024-03-15T13:07:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa81cf2c640 1 Processor -- start 2024-03-15T13:07:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa81cf2c640 1 -- start start 2024-03-15T13:07:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa81cf2c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa818100610 0x7fa81819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa81cf2c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8181056e0 0x7fa81819dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa81cf2c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa818197990 0x7fa818197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa81cf2c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa818198380 con 0x7fa8181056e0 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa81cf2c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8181984f0 con 0x7fa818100610 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa81cf2c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8181986b0 con 0x7fa818197990 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa815d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8181056e0 0x7fa81819dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa81659c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa818100610 0x7fa81819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa815d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8181056e0 0x7fa81819dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60182/0 (socket says 172.21.15.2:60182) 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa815d9b640 1 -- 172.21.15.2:0/4095880919 learned_addr learned my addr 172.21.15.2:0/4095880919 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa816d9d640 1 --2- 172.21.15.2:0/4095880919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa818197990 0x7fa818197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa815d9b640 1 -- 172.21.15.2:0/4095880919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa818197990 msgr2=0x7fa818197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa815d9b640 1 --2- 172.21.15.2:0/4095880919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa818197990 0x7fa818197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa815d9b640 1 -- 172.21.15.2:0/4095880919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa818100610 msgr2=0x7fa81819d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa815d9b640 1 --2- 172.21.15.2:0/4095880919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa818100610 0x7fa81819d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.794+0000 7fa815d9b640 1 -- 172.21.15.2:0/4095880919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa808009900 con 0x7fa8181056e0 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.795+0000 7fa815d9b640 1 --2- 172.21.15.2:0/4095880919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8181056e0 0x7fa81819dcd0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa80000e9c0 tx=0x7fa80000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.795+0000 7fa80f7fe640 1 -- 172.21.15.2:0/4095880919 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800009700 con 0x7fa8181056e0 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.795+0000 7fa80f7fe640 1 -- 172.21.15.2:0/4095880919 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa80000cd80 con 0x7fa8181056e0 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.795+0000 7fa81cf2c640 1 -- 172.21.15.2:0/4095880919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8181a4650 con 0x7fa8181056e0 2024-03-15T13:07:58.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.795+0000 7fa80f7fe640 1 -- 172.21.15.2:0/4095880919 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800010430 con 0x7fa8181056e0 2024-03-15T13:07:58.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.795+0000 7fa81cf2c640 1 -- 172.21.15.2:0/4095880919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8181a4b60 con 0x7fa8181056e0 2024-03-15T13:07:58.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.797+0000 7fa80f7fe640 1 -- 172.21.15.2:0/4095880919 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fa800004ff0 con 0x7fa8181056e0 2024-03-15T13:07:58.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.797+0000 7fa81fbe2740 1 Processor -- start 2024-03-15T13:07:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa81fbe2740 1 -- start start 2024-03-15T13:07:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa81fbe2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1c250 0x55c650b1c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa81fbe2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1cb70 0x55c650b25210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa81fbe2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b27b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa81fbe2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c650ad7960 con 0x55c650b25750 2024-03-15T13:07:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa81fbe2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c650a8b310 con 0x55c650b1c250 2024-03-15T13:07:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa81fbe2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c650a88840 con 0x55c650b1cb70 2024-03-15T13:07:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa80f7fe640 1 --2- 172.21.15.2:0/4095880919 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa8040a1ba0 0x7fa8040a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa815d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b27b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa816d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1c250 0x55c650b1c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa81659c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1cb70 0x55c650b25210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa81659c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1cb70 0x55c650b25210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:35086/0 (socket says 172.21.15.2:35086) 2024-03-15T13:07:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa81659c640 1 -- 172.21.15.2:0/3534024051 learned_addr learned my addr 172.21.15.2:0/3534024051 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa816d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1c250 0x55c650b1c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:58306/0 (socket says 172.21.15.2:58306) 2024-03-15T13:07:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa816d9d640 1 --2- 172.21.15.2:0/4095880919 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa8040a1ba0 0x7fa8040a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa80f7fe640 1 -- 172.21.15.2:0/4095880919 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa800014070 con 0x7fa8181056e0 2024-03-15T13:07:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa81659c640 1 -- 172.21.15.2:0/3534024051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1c250 msgr2=0x55c650b1c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa81659c640 1 --2- 172.21.15.2:0/3534024051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1c250 0x55c650b1c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa81659c640 1 -- 172.21.15.2:0/3534024051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 msgr2=0x55c650b27b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa81659c640 1 --2- 172.21.15.2:0/3534024051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b27b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.798+0000 7fa81659c640 1 -- 172.21.15.2:0/3534024051 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c650a46060 con 0x55c650b1cb70 2024-03-15T13:07:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.799+0000 7fa816d9d640 1 --2- 172.21.15.2:0/4095880919 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa8040a1ba0 0x7fa8040a4060 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa808005cc0 tx=0x7fa808005c50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.799+0000 7fa81659c640 1 --2- 172.21.15.2:0/3534024051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1cb70 0x55c650b25210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa810009ca0 tx=0x7fa810011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39bff76fed1c94e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.799+0000 7fa80cff9640 1 -- 172.21.15.2:0/3534024051 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810005220 con 0x55c650b1cb70 2024-03-15T13:07:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.799+0000 7fa80cff9640 1 -- 172.21.15.2:0/3534024051 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa810004c70 con 0x55c650b1cb70 2024-03-15T13:07:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.799+0000 7fa80cff9640 1 -- 172.21.15.2:0/3534024051 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810019740 con 0x55c650b1cb70 2024-03-15T13:07:58.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.799+0000 7fa81fbe2740 1 -- 172.21.15.2:0/3534024051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1cb70 msgr2=0x55c650b25210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.799+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/3534024051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1cb70 0x55c650b25210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa810009ca0 tx=0x7fa810011810 comp rx=0 tx=0).stop 2024-03-15T13:07:58.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.799+0000 7fa81fbe2740 1 -- 172.21.15.2:0/3534024051 shutdown_connections 2024-03-15T13:07:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.799+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/3534024051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b27b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.799+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/3534024051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1cb70 0x55c650b25210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.799+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/3534024051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1c250 0x55c650b1c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.799+0000 7fa81fbe2740 1 -- 172.21.15.2:0/3534024051 >> 172.21.15.2:0/3534024051 conn(0x55c650b16980 msgr2=0x55c650b16d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa81fbe2740 1 -- 172.21.15.2:0/3534024051 shutdown_connections 2024-03-15T13:07:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa81fbe2740 1 -- 172.21.15.2:0/3534024051 wait complete. 2024-03-15T13:07:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa81fbe2740 1 -- start start 2024-03-15T13:07:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa81fbe2740 1 Processor -- start 2024-03-15T13:07:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa81fbe2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1c250 0x55c650b24fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa81fbe2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 0x55c650b22370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa81fbe2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b228b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa81fbe2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c650a84020 con 0x55c650b25750 2024-03-15T13:07:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa81fbe2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c650757430 con 0x55c650b1cb70 2024-03-15T13:07:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa81fbe2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c650713e00 con 0x55c650b1c250 2024-03-15T13:07:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa81fbe2740 10 client.0 _ll_register_callbacks cb 0x55c650a2bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa816d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa816d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b228b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60210/0 (socket says 172.21.15.2:60210) 2024-03-15T13:07:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa81659c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1c250 0x55c650b24fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa816d9d640 1 -- 172.21.15.2:0/2983269242 learned_addr learned my addr 172.21.15.2:0/2983269242 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.800+0000 7fa815d9b640 1 --2- 172.21.15.2:0/2983269242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 0x55c650b22370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.801+0000 7fa816d9d640 1 -- 172.21.15.2:0/2983269242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1c250 msgr2=0x55c650b24fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.801+0000 7fa816d9d640 1 --2- 172.21.15.2:0/2983269242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1c250 0x55c650b24fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.801+0000 7fa816d9d640 1 -- 172.21.15.2:0/2983269242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 msgr2=0x55c650b22370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.801+0000 7fa816d9d640 1 --2- 172.21.15.2:0/2983269242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 0x55c650b22370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.801+0000 7fa816d9d640 1 -- 172.21.15.2:0/2983269242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8100098d0 con 0x55c650b25750 2024-03-15T13:07:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.801+0000 7fa816d9d640 1 --2- 172.21.15.2:0/2983269242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b228b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa808009cd0 tx=0x7fa80802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19583db1fb14471a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.801+0000 7fa7fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.801+0000 7fa7fdffb640 1 -- 172.21.15.2:0/2983269242 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808032a90 con 0x55c650b25750 2024-03-15T13:07:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.801+0000 7fa7fdffb640 1 -- 172.21.15.2:0/2983269242 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa808003970 con 0x55c650b25750 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.801+0000 7fa81fbe2740 1 -- 172.21.15.2:0/2983269242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6507837e0 con 0x55c650b25750 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.801+0000 7fa7fdffb640 1 -- 172.21.15.2:0/2983269242 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa808003b30 con 0x55c650b25750 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.801+0000 7fa81fbe2740 10 client.18261 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.801+0000 7fa81fbe2740 1 -- 172.21.15.2:0/2983269242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55c650a826a0 con 0x55c650b25750 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.801+0000 7fa7fdffb640 1 -- 172.21.15.2:0/2983269242 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55c6507837e0 con 0x55c650b25750 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.802+0000 7fa7fdffb640 1 -- 172.21.15.2:0/2983269242 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa80804c070 con 0x55c650b25750 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.802+0000 7fa81fbe2740 10 client.18261 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.802+0000 7fa81fbe2740 10 client.18261 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.802+0000 7fa81fbe2740 20 client.18261 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.802+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/2983269242 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c650b35780 0x55c650b37ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.802+0000 7fa81fbe2740 4 client.18261 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.802+0000 7fa81fbe2740 1 -- 172.21.15.2:0/2983269242 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6507c0720 con 0x55c650b35780 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.802+0000 7fa81659c640 1 --2- 172.21.15.2:0/2983269242 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c650b35780 0x55c650b37ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.802+0000 7fa7fdffb640 10 client.18261 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.802+0000 7fa81659c640 1 --2- 172.21.15.2:0/2983269242 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c650b35780 0x55c650b37ba0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bfaa01d36616d52 server_cookie=9455a301b32d8847 in_seq=0 out_seq=0 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.803+0000 7fa7fdffb640 1 -- 172.21.15.2:0/2983269242 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c6507c0720 con 0x55c650b35780 2024-03-15T13:07:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.803+0000 7fa7fdffb640 10 client.18261 handle_command_reply: tid=0 2024-03-15T13:07:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.803+0000 7fa81fbe2740 1 client.18261 shutdown 2024-03-15T13:07:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.803+0000 7fa81fbe2740 1 -- 172.21.15.2:0/2983269242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 msgr2=0x55c650b228b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.803+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/2983269242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b228b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa808009cd0 tx=0x7fa80802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.803+0000 7fa81fbe2740 1 -- 172.21.15.2:0/2983269242 shutdown_connections 2024-03-15T13:07:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.803+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/2983269242 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c650b35780 0x55c650b37ba0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.803+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/2983269242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b228b0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.803+0000 7fa7fdffb640 0 client.18261 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:07:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.803+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/2983269242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 0x55c650b22370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.803+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/2983269242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1c250 0x55c650b24fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.803+0000 7fa81fbe2740 1 -- 172.21.15.2:0/2983269242 >> 172.21.15.2:0/2983269242 conn(0x55c650b16980 msgr2=0x55c650b186d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.803+0000 7fa81fbe2740 1 -- 172.21.15.2:0/2983269242 shutdown_connections 2024-03-15T13:07:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.803+0000 7fa81fbe2740 1 -- 172.21.15.2:0/2983269242 wait complete. 2024-03-15T13:07:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.803+0000 7fa81fbe2740 20 client.18261 trim_cache size 0 max 16384 2024-03-15T13:07:58.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa81fbe2740 1 Processor -- start 2024-03-15T13:07:58.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa81fbe2740 1 -- start start 2024-03-15T13:07:58.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa81fbe2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1c250 0x55c650b152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa81fbe2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 0x55c650b24aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa81fbe2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b24fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa81fbe2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c650a09130 con 0x55c650b25750 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa81fbe2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c650b22eb0 con 0x55c650b1cb70 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa81fbe2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c650b167b0 con 0x55c650b1c250 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa816d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b24fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa816d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b24fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60212/0 (socket says 172.21.15.2:60212) 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa81659c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1c250 0x55c650b152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa816d9d640 1 -- 172.21.15.2:0/2710791422 learned_addr learned my addr 172.21.15.2:0/2710791422 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa815d9b640 1 --2- 172.21.15.2:0/2710791422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 0x55c650b24aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa816d9d640 1 -- 172.21.15.2:0/2710791422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1c250 msgr2=0x55c650b152f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa816d9d640 1 --2- 172.21.15.2:0/2710791422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1c250 0x55c650b152f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa816d9d640 1 -- 172.21.15.2:0/2710791422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 msgr2=0x55c650b24aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa816d9d640 1 --2- 172.21.15.2:0/2710791422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 0x55c650b24aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.816+0000 7fa816d9d640 1 -- 172.21.15.2:0/2710791422 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c650a1ec20 con 0x55c650b25750 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.817+0000 7fa816d9d640 1 --2- 172.21.15.2:0/2710791422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b24fe0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa808005d70 tx=0x7fa808042810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80bdbef04fc4e2f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.817+0000 7fa7fffff640 1 -- 172.21.15.2:0/2710791422 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808032a90 con 0x55c650b25750 2024-03-15T13:07:58.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.817+0000 7fa7fffff640 1 -- 172.21.15.2:0/2710791422 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa808049040 con 0x55c650b25750 2024-03-15T13:07:58.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.817+0000 7fa7fffff640 1 -- 172.21.15.2:0/2710791422 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808003d50 con 0x55c650b25750 2024-03-15T13:07:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.817+0000 7fa81fbe2740 1 -- 172.21.15.2:0/2710791422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 msgr2=0x55c650b24fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.817+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/2710791422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b24fe0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa808005d70 tx=0x7fa808042810 comp rx=0 tx=0).stop 2024-03-15T13:07:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.817+0000 7fa81fbe2740 1 -- 172.21.15.2:0/2710791422 shutdown_connections 2024-03-15T13:07:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.817+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/2710791422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b25750 0x55c650b24fe0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.817+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/2710791422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 0x55c650b24aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.817+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/2710791422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b1c250 0x55c650b152f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.817+0000 7fa81fbe2740 1 -- 172.21.15.2:0/2710791422 >> 172.21.15.2:0/2710791422 conn(0x55c650b16980 msgr2=0x55c650b1d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.817+0000 7fa81fbe2740 1 -- 172.21.15.2:0/2710791422 shutdown_connections 2024-03-15T13:07:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.817+0000 7fa81fbe2740 1 -- 172.21.15.2:0/2710791422 wait complete. 2024-03-15T13:07:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81fbe2740 1 -- start start 2024-03-15T13:07:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81fbe2740 1 Processor -- start 2024-03-15T13:07:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81fbe2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b1c250 0x55c650b2d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81fbe2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 0x55c650b29620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81fbe2740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b25750 0x55c650b29b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81fbe2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c650b2a120 con 0x55c650b1c250 2024-03-15T13:07:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81fbe2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c650b2a290 con 0x55c650b1cb70 2024-03-15T13:07:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81fbe2740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c650b2a450 con 0x55c650b25750 2024-03-15T13:07:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81fbe2740 10 client.0 _ll_register_callbacks cb 0x7fa810005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b1c250 0x55c650b2d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b1c250 0x55c650b2d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60214/0 (socket says 172.21.15.2:60214) 2024-03-15T13:07:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81659c640 1 -- 172.21.15.2:0/1099906214 learned_addr learned my addr 172.21.15.2:0/1099906214 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa815d9b640 1 --2- 172.21.15.2:0/1099906214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 0x55c650b29620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa816d9d640 1 --2- 172.21.15.2:0/1099906214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b25750 0x55c650b29b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81659c640 1 -- 172.21.15.2:0/1099906214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b25750 msgr2=0x55c650b29b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81659c640 1 --2- 172.21.15.2:0/1099906214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b25750 0x55c650b29b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81659c640 1 -- 172.21.15.2:0/1099906214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 msgr2=0x55c650b29620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81659c640 1 --2- 172.21.15.2:0/1099906214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 0x55c650b29620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81659c640 1 -- 172.21.15.2:0/1099906214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c650a826a0 con 0x55c650b1c250 2024-03-15T13:07:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa815d9b640 1 --2- 172.21.15.2:0/1099906214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 0x55c650b29620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:07:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa81659c640 1 --2- 172.21.15.2:0/1099906214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b1c250 0x55c650b2d2a0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa810009ca0 tx=0x7fa810005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5881a427ec7b4cf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.818+0000 7fa7fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.819+0000 7fa7fdffb640 1 -- 172.21.15.2:0/1099906214 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81002d350 con 0x55c650b1c250 2024-03-15T13:07:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.819+0000 7fa7fdffb640 1 -- 172.21.15.2:0/1099906214 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa810004040 con 0x55c650b1c250 2024-03-15T13:07:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.819+0000 7fa81fbe2740 1 -- 172.21.15.2:0/1099906214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c650b120f0 con 0x55c650b1c250 2024-03-15T13:07:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.819+0000 7fa7fdffb640 1 -- 172.21.15.2:0/1099906214 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa810011d40 con 0x55c650b1c250 2024-03-15T13:07:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.819+0000 7fa81fbe2740 10 client.18267 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.819+0000 7fa81fbe2740 1 -- 172.21.15.2:0/1099906214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55c650a448c0 con 0x55c650b1c250 2024-03-15T13:07:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.819+0000 7fa7fdffb640 1 -- 172.21.15.2:0/1099906214 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55c650b120f0 con 0x55c650b1c250 2024-03-15T13:07:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.819+0000 7fa7fdffb640 1 -- 172.21.15.2:0/1099906214 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fa81002d070 con 0x55c650b1c250 2024-03-15T13:07:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.819+0000 7fa81fbe2740 10 client.18267 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.819+0000 7fa81fbe2740 10 client.18267 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:07:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.819+0000 7fa81fbe2740 20 client.18267 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.819+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/1099906214 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c650b926b0 0x55c650b94ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.819+0000 7fa81fbe2740 4 client.18267 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.819+0000 7fa81fbe2740 1 -- 172.21.15.2:0/1099906214 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fa810004c70 con 0x55c650b926b0 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.819+0000 7fa815d9b640 1 --2- 172.21.15.2:0/1099906214 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c650b926b0 0x55c650b94ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa815d9b640 1 --2- 172.21.15.2:0/1099906214 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c650b926b0 0x55c650b94ad0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6ca0ea43be65f9ae server_cookie=841a5353a84223af in_seq=0 out_seq=0 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa7fdffb640 10 client.18267 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa7fdffb640 1 -- 172.21.15.2:0/1099906214 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fa810004c70 con 0x55c650b926b0 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa7fdffb640 10 client.18267 handle_command_reply: tid=0 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa81fbe2740 1 client.18267 shutdown 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa81fbe2740 1 -- 172.21.15.2:0/1099906214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b1c250 msgr2=0x55c650b2d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/1099906214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b1c250 0x55c650b2d2a0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa810009ca0 tx=0x7fa810005d30 comp rx=0 tx=0).stop 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa81fbe2740 1 -- 172.21.15.2:0/1099906214 shutdown_connections 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/1099906214 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55c650b926b0 0x55c650b94ad0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/1099906214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c650b25750 0x55c650b29b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/1099906214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c650b1cb70 0x55c650b29620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa7fdffb640 0 client.18267 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:07:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa81fbe2740 1 --2- 172.21.15.2:0/1099906214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c650b1c250 0x55c650b2d2a0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa81fbe2740 1 -- 172.21.15.2:0/1099906214 >> 172.21.15.2:0/1099906214 conn(0x55c650b16980 msgr2=0x55c650b1cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa81fbe2740 1 -- 172.21.15.2:0/1099906214 shutdown_connections 2024-03-15T13:07:58.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa81fbe2740 1 -- 172.21.15.2:0/1099906214 wait complete. 2024-03-15T13:07:58.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.820+0000 7fa81fbe2740 20 client.18267 trim_cache size 0 max 16384 2024-03-15T13:07:58.811 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:07:58.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.821+0000 7fa81cf2c640 1 -- 172.21.15.2:0/4095880919 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa8040a1ba0 msgr2=0x7fa8040a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.821+0000 7fa81cf2c640 1 --2- 172.21.15.2:0/4095880919 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa8040a1ba0 0x7fa8040a4060 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa808005cc0 tx=0x7fa808005c50 comp rx=0 tx=0).stop 2024-03-15T13:07:58.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.821+0000 7fa81cf2c640 1 -- 172.21.15.2:0/4095880919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8181056e0 msgr2=0x7fa81819dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.821+0000 7fa81cf2c640 1 --2- 172.21.15.2:0/4095880919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8181056e0 0x7fa81819dcd0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa80000e9c0 tx=0x7fa80000ee80 comp rx=0 tx=0).stop 2024-03-15T13:07:58.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.821+0000 7fa81cf2c640 1 -- 172.21.15.2:0/4095880919 shutdown_connections 2024-03-15T13:07:58.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.821+0000 7fa81cf2c640 1 --2- 172.21.15.2:0/4095880919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa818197990 0x7fa818197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.821+0000 7fa81cf2c640 1 --2- 172.21.15.2:0/4095880919 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa8040a1ba0 0x7fa8040a4060 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.821+0000 7fa81cf2c640 1 --2- 172.21.15.2:0/4095880919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8181056e0 0x7fa81819dcd0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.821+0000 7fa81cf2c640 1 --2- 172.21.15.2:0/4095880919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa818100610 0x7fa81819d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.821+0000 7fa81cf2c640 1 -- 172.21.15.2:0/4095880919 >> 172.21.15.2:0/4095880919 conn(0x7fa818076770 msgr2=0x7fa8180fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.821+0000 7fa81cf2c640 1 -- 172.21.15.2:0/4095880919 shutdown_connections 2024-03-15T13:07:58.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.821+0000 7fa81cf2c640 1 -- 172.21.15.2:0/4095880919 wait complete. 2024-03-15T13:07:58.816 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:07:58.816 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:07:58.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.936+0000 7ff049eeb640 1 -- 172.21.15.2:0/2024504297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0440ffdb0 msgr2=0x7ff044100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.936+0000 7ff049eeb640 1 --2- 172.21.15.2:0/2024504297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0440ffdb0 0x7ff044100230 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff038009900 tx=0x7ff03802bc30 comp rx=0 tx=0).stop 2024-03-15T13:07:58.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.936+0000 7ff049eeb640 1 -- 172.21.15.2:0/2024504297 shutdown_connections 2024-03-15T13:07:58.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.936+0000 7ff049eeb640 1 --2- 172.21.15.2:0/2024504297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff044100770 0x7ff04410e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.936+0000 7ff049eeb640 1 --2- 172.21.15.2:0/2024504297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0440ffdb0 0x7ff044100230 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.936+0000 7ff049eeb640 1 --2- 172.21.15.2:0/2024504297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0441076f0 0x7ff044107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.936+0000 7ff049eeb640 1 -- 172.21.15.2:0/2024504297 >> 172.21.15.2:0/2024504297 conn(0x7ff044075730 msgr2=0x7ff044075b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.936+0000 7ff049eeb640 1 -- 172.21.15.2:0/2024504297 shutdown_connections 2024-03-15T13:07:58.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.936+0000 7ff049eeb640 1 -- 172.21.15.2:0/2024504297 wait complete. 2024-03-15T13:07:58.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.937+0000 7ff049eeb640 1 Processor -- start 2024-03-15T13:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.937+0000 7ff049eeb640 1 -- start start 2024-03-15T13:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.937+0000 7ff049eeb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0440ffdb0 0x7ff04419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.937+0000 7ff049eeb640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff044100770 0x7ff04419dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.937+0000 7ff049eeb640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0441076f0 0x7ff044197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.937+0000 7ff049eeb640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff044197fc0 con 0x7ff0440ffdb0 2024-03-15T13:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.937+0000 7ff049eeb640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff044198130 con 0x7ff044100770 2024-03-15T13:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.937+0000 7ff049eeb640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0441982f0 con 0x7ff0441076f0 2024-03-15T13:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.937+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0440ffdb0 0x7ff04419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.937+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0440ffdb0 0x7ff04419d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60218/0 (socket says 172.21.15.2:60218) 2024-03-15T13:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.937+0000 7ff0437fe640 1 -- 172.21.15.2:0/137140494 learned_addr learned my addr 172.21.15.2:0/137140494 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.937+0000 7ff042ffd640 1 --2- 172.21.15.2:0/137140494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff044100770 0x7ff04419dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.937+0000 7ff043fff640 1 --2- 172.21.15.2:0/137140494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0441076f0 0x7ff044197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.938+0000 7ff0437fe640 1 -- 172.21.15.2:0/137140494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0441076f0 msgr2=0x7ff044197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.938+0000 7ff0437fe640 1 --2- 172.21.15.2:0/137140494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0441076f0 0x7ff044197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.938+0000 7ff0437fe640 1 -- 172.21.15.2:0/137140494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff044100770 msgr2=0x7ff04419dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.938+0000 7ff0437fe640 1 --2- 172.21.15.2:0/137140494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff044100770 0x7ff04419dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.938+0000 7ff0437fe640 1 -- 172.21.15.2:0/137140494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff038009560 con 0x7ff0440ffdb0 2024-03-15T13:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.938+0000 7ff0437fe640 1 --2- 172.21.15.2:0/137140494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0440ffdb0 0x7ff04419d7c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7ff03400e9f0 tx=0x7ff03400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.938+0000 7ff040ff9640 1 -- 172.21.15.2:0/137140494 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034009720 con 0x7ff0440ffdb0 2024-03-15T13:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.938+0000 7ff049eeb640 1 -- 172.21.15.2:0/137140494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0441985d0 con 0x7ff0440ffdb0 2024-03-15T13:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.938+0000 7ff040ff9640 1 -- 172.21.15.2:0/137140494 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff03400cd80 con 0x7ff0440ffdb0 2024-03-15T13:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.938+0000 7ff040ff9640 1 -- 172.21.15.2:0/137140494 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034010690 con 0x7ff0440ffdb0 2024-03-15T13:07:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.938+0000 7ff049eeb640 1 -- 172.21.15.2:0/137140494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0441a4860 con 0x7ff0440ffdb0 2024-03-15T13:07:58.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.940+0000 7ff040ff9640 1 -- 172.21.15.2:0/137140494 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7ff034004e90 con 0x7ff0440ffdb0 2024-03-15T13:07:58.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff04cb9f740 1 Processor -- start 2024-03-15T13:07:58.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff04cb9f740 1 -- start start 2024-03-15T13:07:58.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff040ff9640 1 --2- 172.21.15.2:0/137140494 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff02c0a1ba0 0x7ff02c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff04cb9f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 0x559ee8b0f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff04cb9f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 0x559ee8b18210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff04cb9f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b18750 0x559ee8b1ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff04cb9f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee8aca960 con 0x559ee8b18750 2024-03-15T13:07:58.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff04cb9f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee8a7e310 con 0x559ee8b0fb70 2024-03-15T13:07:58.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff04cb9f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee8a7b840 con 0x559ee8b0f250 2024-03-15T13:07:58.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff043fff640 1 --2- 172.21.15.2:0/137140494 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff02c0a1ba0 0x7ff02c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff040ff9640 1 -- 172.21.15.2:0/137140494 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ff034014070 con 0x7ff0440ffdb0 2024-03-15T13:07:58.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff042ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 0x559ee8b18210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 0x559ee8b0f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff043fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b18750 0x559ee8b1ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff042ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 0x559ee8b18210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:58340/0 (socket says 172.21.15.2:58340) 2024-03-15T13:07:58.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff042ffd640 1 -- 172.21.15.2:0/1831598697 learned_addr learned my addr 172.21.15.2:0/1831598697 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 0x559ee8b0f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:35126/0 (socket says 172.21.15.2:35126) 2024-03-15T13:07:58.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff042ffd640 1 -- 172.21.15.2:0/1831598697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 msgr2=0x559ee8b0f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff042ffd640 1 --2- 172.21.15.2:0/1831598697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 0x559ee8b0f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff042ffd640 1 -- 172.21.15.2:0/1831598697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b18750 msgr2=0x559ee8b1ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff042ffd640 1 --2- 172.21.15.2:0/1831598697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b18750 0x559ee8b1ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff042ffd640 1 -- 172.21.15.2:0/1831598697 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ee8a39060 con 0x559ee8b0fb70 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff043fff640 1 --2- 172.21.15.2:0/137140494 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff02c0a1ba0 0x7ff02c0a4060 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff03c005fd0 tx=0x7ff03c009e60 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.941+0000 7ff042ffd640 1 --2- 172.21.15.2:0/1831598697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 0x559ee8b18210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff038036040 tx=0x7ff03802c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78e5bfdc83990f7e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.942+0000 7ff029ffb640 1 -- 172.21.15.2:0/1831598697 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038004780 con 0x559ee8b0fb70 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.942+0000 7ff029ffb640 1 -- 172.21.15.2:0/1831598697 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff0380048e0 con 0x559ee8b0fb70 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.942+0000 7ff029ffb640 1 -- 172.21.15.2:0/1831598697 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038003830 con 0x559ee8b0fb70 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.942+0000 7ff04cb9f740 1 -- 172.21.15.2:0/1831598697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 msgr2=0x559ee8b18210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.942+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/1831598697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 0x559ee8b18210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff038036040 tx=0x7ff03802c6d0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.942+0000 7ff04cb9f740 1 -- 172.21.15.2:0/1831598697 shutdown_connections 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.942+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/1831598697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b18750 0x559ee8b1ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.942+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/1831598697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 0x559ee8b18210 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.942+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/1831598697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 0x559ee8b0f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.942+0000 7ff04cb9f740 1 -- 172.21.15.2:0/1831598697 >> 172.21.15.2:0/1831598697 conn(0x559ee8b09980 msgr2=0x559ee8b09d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.942+0000 7ff04cb9f740 1 -- 172.21.15.2:0/1831598697 shutdown_connections 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.942+0000 7ff04cb9f740 1 -- 172.21.15.2:0/1831598697 wait complete. 2024-03-15T13:07:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff04cb9f740 1 -- start start 2024-03-15T13:07:58.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff04cb9f740 1 Processor -- start 2024-03-15T13:07:58.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff04cb9f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 0x559ee8b18010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff04cb9f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 0x559ee8b153a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff04cb9f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b18750 0x559ee8b158e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff04cb9f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee8a77020 con 0x559ee8b18750 2024-03-15T13:07:58.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff04cb9f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee874a430 con 0x559ee8b0fb70 2024-03-15T13:07:58.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff04cb9f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee8706e00 con 0x559ee8b0f250 2024-03-15T13:07:58.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff04cb9f740 10 client.0 _ll_register_callbacks cb 0x559ee8a1ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:58.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff043fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b18750 0x559ee8b158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff042ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 0x559ee8b153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff043fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b18750 0x559ee8b158e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60234/0 (socket says 172.21.15.2:60234) 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff043fff640 1 -- 172.21.15.2:0/2292390250 learned_addr learned my addr 172.21.15.2:0/2292390250 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff0437fe640 1 --2- 172.21.15.2:0/2292390250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 0x559ee8b18010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff043fff640 1 -- 172.21.15.2:0/2292390250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 msgr2=0x559ee8b18010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff043fff640 1 --2- 172.21.15.2:0/2292390250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 0x559ee8b18010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff043fff640 1 -- 172.21.15.2:0/2292390250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 msgr2=0x559ee8b153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff043fff640 1 --2- 172.21.15.2:0/2292390250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 0x559ee8b153a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff043fff640 1 -- 172.21.15.2:0/2292390250 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04410ec60 con 0x559ee8b18750 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.943+0000 7ff042ffd640 1 --2- 172.21.15.2:0/2292390250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 0x559ee8b153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff043fff640 1 --2- 172.21.15.2:0/2292390250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b18750 0x559ee8b158e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7ff03c013390 tx=0x7ff03c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08f1fa666946dfd server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff022ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff022ffd640 1 -- 172.21.15.2:0/2292390250 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c021410 con 0x559ee8b18750 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff022ffd640 1 -- 172.21.15.2:0/2292390250 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ff03c021570 con 0x559ee8b18750 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff04cb9f740 1 -- 172.21.15.2:0/2292390250 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ee87767e0 con 0x559ee8b18750 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff022ffd640 1 -- 172.21.15.2:0/2292390250 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff03c021730 con 0x559ee8b18750 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff04cb9f740 10 client.18279 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff04cb9f740 1 -- 172.21.15.2:0/2292390250 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x559ee8a756a0 con 0x559ee8b18750 2024-03-15T13:07:58.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff022ffd640 1 -- 172.21.15.2:0/2292390250 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x559ee87767e0 con 0x559ee8b18750 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff022ffd640 1 -- 172.21.15.2:0/2292390250 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff03c02bb80 con 0x559ee8b18750 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff04cb9f740 10 client.18279 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff04cb9f740 10 client.18279 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff04cb9f740 20 client.18279 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/2292390250 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559ee8b287b0 0x559ee8b2abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff04cb9f740 4 client.18279 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.944+0000 7ff04cb9f740 1 -- 172.21.15.2:0/2292390250 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ee87b3720 con 0x559ee8b287b0 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.945+0000 7ff0437fe640 1 --2- 172.21.15.2:0/2292390250 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559ee8b287b0 0x559ee8b2abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.945+0000 7ff022ffd640 10 client.18279 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.945+0000 7ff0437fe640 1 --2- 172.21.15.2:0/2292390250 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559ee8b287b0 0x559ee8b2abd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=76b51502fa1a5de0 server_cookie=d7631b8a79c142f5 in_seq=0 out_seq=0 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.946+0000 7ff022ffd640 1 -- 172.21.15.2:0/2292390250 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559ee87b3720 con 0x559ee8b287b0 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.946+0000 7ff022ffd640 10 client.18279 handle_command_reply: tid=0 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.946+0000 7ff04cb9f740 1 client.18279 shutdown 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.946+0000 7ff04cb9f740 1 -- 172.21.15.2:0/2292390250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b18750 msgr2=0x559ee8b158e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.946+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/2292390250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b18750 0x559ee8b158e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7ff03c013390 tx=0x7ff03c004ab0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.946+0000 7ff04cb9f740 1 -- 172.21.15.2:0/2292390250 shutdown_connections 2024-03-15T13:07:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.946+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/2292390250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b18750 0x559ee8b158e0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.946+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/2292390250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 0x559ee8b153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.946+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/2292390250 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559ee8b287b0 0x559ee8b2abd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.946+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/2292390250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 0x559ee8b18010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.946+0000 7ff022ffd640 0 client.18279 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:07:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.946+0000 7ff04cb9f740 1 -- 172.21.15.2:0/2292390250 >> 172.21.15.2:0/2292390250 conn(0x559ee8b09980 msgr2=0x559ee8b0b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.946+0000 7ff04cb9f740 1 -- 172.21.15.2:0/2292390250 shutdown_connections 2024-03-15T13:07:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.946+0000 7ff04cb9f740 1 -- 172.21.15.2:0/2292390250 wait complete. 2024-03-15T13:07:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.946+0000 7ff04cb9f740 20 client.18279 trim_cache size 0 max 16384 2024-03-15T13:07:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.958+0000 7ff04cb9f740 1 Processor -- start 2024-03-15T13:07:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.958+0000 7ff04cb9f740 1 -- start start 2024-03-15T13:07:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.958+0000 7ff04cb9f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b0f250 0x559ee8b17ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff04cb9f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 0x559ee8b18010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff04cb9f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b18750 0x559ee8b23aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff04cb9f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee89fc130 con 0x559ee8b0f250 2024-03-15T13:07:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff04cb9f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee8b055c0 con 0x559ee8b0fb70 2024-03-15T13:07:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff04cb9f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee8b16190 con 0x559ee8b18750 2024-03-15T13:07:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b0f250 0x559ee8b17ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b0f250 0x559ee8b17ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60246/0 (socket says 172.21.15.2:60246) 2024-03-15T13:07:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff0437fe640 1 -- 172.21.15.2:0/2011745781 learned_addr learned my addr 172.21.15.2:0/2011745781 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff042ffd640 1 --2- 172.21.15.2:0/2011745781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 0x559ee8b18010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff043fff640 1 --2- 172.21.15.2:0/2011745781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b18750 0x559ee8b23aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff0437fe640 1 -- 172.21.15.2:0/2011745781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b18750 msgr2=0x559ee8b23aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff0437fe640 1 --2- 172.21.15.2:0/2011745781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b18750 0x559ee8b23aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff0437fe640 1 -- 172.21.15.2:0/2011745781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 msgr2=0x559ee8b18010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff0437fe640 1 --2- 172.21.15.2:0/2011745781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 0x559ee8b18010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff0437fe640 1 -- 172.21.15.2:0/2011745781 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ee8a11c20 con 0x559ee8b0f250 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff0437fe640 1 --2- 172.21.15.2:0/2011745781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b0f250 0x559ee8b17ad0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff034096be0 tx=0x7ff034009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce9a04a6891e820 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff0297fa640 1 -- 172.21.15.2:0/2011745781 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03409e8d0 con 0x559ee8b0f250 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff0297fa640 1 -- 172.21.15.2:0/2011745781 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff0340a2f20 con 0x559ee8b0f250 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.959+0000 7ff0297fa640 1 -- 172.21.15.2:0/2011745781 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0340cb520 con 0x559ee8b0f250 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 -- 172.21.15.2:0/2011745781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b0f250 msgr2=0x559ee8b17ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/2011745781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b0f250 0x559ee8b17ad0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff034096be0 tx=0x7ff034009310 comp rx=0 tx=0).stop 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 -- 172.21.15.2:0/2011745781 shutdown_connections 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/2011745781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b18750 0x559ee8b23aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/2011745781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b0fb70 0x559ee8b18010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/2011745781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b0f250 0x559ee8b17ad0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 -- 172.21.15.2:0/2011745781 >> 172.21.15.2:0/2011745781 conn(0x559ee8b09980 msgr2=0x559ee8b0b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 -- 172.21.15.2:0/2011745781 shutdown_connections 2024-03-15T13:07:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 -- 172.21.15.2:0/2011745781 wait complete. 2024-03-15T13:07:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 -- start start 2024-03-15T13:07:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 Processor -- start 2024-03-15T13:07:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 0x559ee8b20280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b0fb70 0x559ee8b1c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b18750 0x559ee8b1cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee8b1d100 con 0x559ee8b0fb70 2024-03-15T13:07:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee8b1d270 con 0x559ee8b18750 2024-03-15T13:07:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.960+0000 7ff04cb9f740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ee8b1d430 con 0x559ee8b0f250 2024-03-15T13:07:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff04cb9f740 10 client.0 _ll_register_callbacks cb 0x7ff03c0135a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff042ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b0fb70 0x559ee8b1c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 0x559ee8b20280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff042ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b0fb70 0x559ee8b1c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60248/0 (socket says 172.21.15.2:60248) 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff042ffd640 1 -- 172.21.15.2:0/3110692878 learned_addr learned my addr 172.21.15.2:0/3110692878 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff043fff640 1 --2- 172.21.15.2:0/3110692878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b18750 0x559ee8b1cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff042ffd640 1 -- 172.21.15.2:0/3110692878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 msgr2=0x559ee8b20280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff042ffd640 1 --2- 172.21.15.2:0/3110692878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 0x559ee8b20280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff042ffd640 1 -- 172.21.15.2:0/3110692878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b18750 msgr2=0x559ee8b1cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff042ffd640 1 --2- 172.21.15.2:0/3110692878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b18750 0x559ee8b1cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff042ffd640 1 -- 172.21.15.2:0/3110692878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0441a4860 con 0x559ee8b0fb70 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff042ffd640 1 --2- 172.21.15.2:0/3110692878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b0fb70 0x559ee8b1c600 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7ff038005bc0 tx=0x7ff038004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a10c7853ef4707c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff022ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff022ffd640 1 -- 172.21.15.2:0/3110692878 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038004780 con 0x559ee8b0fb70 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff022ffd640 1 -- 172.21.15.2:0/3110692878 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ff038004fe0 con 0x559ee8b0fb70 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff04cb9f740 1 -- 172.21.15.2:0/3110692878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0380048e0 con 0x559ee8b0fb70 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff022ffd640 1 -- 172.21.15.2:0/3110692878 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0380051a0 con 0x559ee8b0fb70 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff04cb9f740 10 client.18285 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.961+0000 7ff04cb9f740 1 -- 172.21.15.2:0/3110692878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x559ee8a378c0 con 0x559ee8b0fb70 2024-03-15T13:07:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.962+0000 7ff022ffd640 1 -- 172.21.15.2:0/3110692878 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff0380048e0 con 0x559ee8b0fb70 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.962+0000 7ff022ffd640 1 -- 172.21.15.2:0/3110692878 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff03803c050 con 0x559ee8b0fb70 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.962+0000 7ff04cb9f740 10 client.18285 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.962+0000 7ff04cb9f740 10 client.18285 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.962+0000 7ff04cb9f740 20 client.18285 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.962+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/3110692878 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559ee8b85400 0x559ee8b87820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.962+0000 7ff04cb9f740 4 client.18285 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.962+0000 7ff04cb9f740 1 -- 172.21.15.2:0/3110692878 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559ee8802890 con 0x559ee8b85400 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.962+0000 7ff0437fe640 1 --2- 172.21.15.2:0/3110692878 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559ee8b85400 0x559ee8b87820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.962+0000 7ff022ffd640 10 client.18285 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.962+0000 7ff0437fe640 1 --2- 172.21.15.2:0/3110692878 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559ee8b85400 0x559ee8b87820 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=18859d4e761efeff server_cookie=4fdaadd97c2de5e1 in_seq=0 out_seq=0 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.962+0000 7ff022ffd640 1 -- 172.21.15.2:0/3110692878 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559ee8802890 con 0x559ee8b85400 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.962+0000 7ff022ffd640 10 client.18285 handle_command_reply: tid=0 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.962+0000 7ff04cb9f740 1 client.18285 shutdown 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff04cb9f740 1 -- 172.21.15.2:0/3110692878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b0fb70 msgr2=0x559ee8b1c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/3110692878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b0fb70 0x559ee8b1c600 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7ff038005bc0 tx=0x7ff038004610 comp rx=0 tx=0).stop 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff04cb9f740 1 -- 172.21.15.2:0/3110692878 shutdown_connections 2024-03-15T13:07:58.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/3110692878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ee8b18750 0x559ee8b1cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/3110692878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ee8b0fb70 0x559ee8b1c600 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/3110692878 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x559ee8b85400 0x559ee8b87820 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff04cb9f740 1 --2- 172.21.15.2:0/3110692878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ee8b0f250 0x559ee8b20280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff022ffd640 0 client.18285 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:07:58.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff04cb9f740 1 -- 172.21.15.2:0/3110692878 >> 172.21.15.2:0/3110692878 conn(0x559ee8b09980 msgr2=0x559ee8b10090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff04cb9f740 1 -- 172.21.15.2:0/3110692878 shutdown_connections 2024-03-15T13:07:58.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff04cb9f740 1 -- 172.21.15.2:0/3110692878 wait complete. 2024-03-15T13:07:58.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff04cb9f740 20 client.18285 trim_cache size 0 max 16384 2024-03-15T13:07:58.953 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:07:58.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff049eeb640 1 -- 172.21.15.2:0/137140494 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff02c0a1ba0 msgr2=0x7ff02c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff049eeb640 1 --2- 172.21.15.2:0/137140494 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff02c0a1ba0 0x7ff02c0a4060 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff03c005fd0 tx=0x7ff03c009e60 comp rx=0 tx=0).stop 2024-03-15T13:07:58.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff049eeb640 1 -- 172.21.15.2:0/137140494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0440ffdb0 msgr2=0x7ff04419d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:58.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff049eeb640 1 --2- 172.21.15.2:0/137140494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0440ffdb0 0x7ff04419d7c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7ff03400e9f0 tx=0x7ff03400edf0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff049eeb640 1 -- 172.21.15.2:0/137140494 shutdown_connections 2024-03-15T13:07:58.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff049eeb640 1 --2- 172.21.15.2:0/137140494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0441076f0 0x7ff044197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff049eeb640 1 --2- 172.21.15.2:0/137140494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff044100770 0x7ff04419dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff049eeb640 1 --2- 172.21.15.2:0/137140494 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff02c0a1ba0 0x7ff02c0a4060 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff049eeb640 1 --2- 172.21.15.2:0/137140494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0440ffdb0 0x7ff04419d7c0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:58.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff049eeb640 1 -- 172.21.15.2:0/137140494 >> 172.21.15.2:0/137140494 conn(0x7ff044075730 msgr2=0x7ff0440fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:58.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff049eeb640 1 -- 172.21.15.2:0/137140494 shutdown_connections 2024-03-15T13:07:58.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:58.963+0000 7ff049eeb640 1 -- 172.21.15.2:0/137140494 wait complete. 2024-03-15T13:07:58.959 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:07:58.959 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:07:59.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 -- 172.21.15.2:0/1182387742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1394100e90 msgr2=0x7f1394104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 --2- 172.21.15.2:0/1182387742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1394100e90 0x7f1394104ed0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f1384009930 tx=0x7f138402bc30 comp rx=0 tx=0).stop 2024-03-15T13:07:59.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 -- 172.21.15.2:0/1182387742 shutdown_connections 2024-03-15T13:07:59.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 --2- 172.21.15.2:0/1182387742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13941055b0 0x7f139410d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 --2- 172.21.15.2:0/1182387742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1394100e90 0x7f1394104ed0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 --2- 172.21.15.2:0/1182387742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13941004e0 0x7f13941008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 -- 172.21.15.2:0/1182387742 >> 172.21.15.2:0/1182387742 conn(0x7f13940fbd20 msgr2=0x7f13940fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 -- 172.21.15.2:0/1182387742 shutdown_connections 2024-03-15T13:07:59.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 -- 172.21.15.2:0/1182387742 wait complete. 2024-03-15T13:07:59.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 Processor -- start 2024-03-15T13:07:59.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 -- start start 2024-03-15T13:07:59.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13941004e0 0x7f139419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13941055b0 0x7f139419dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1394197880 0x7f1394197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1394198270 con 0x7f1394197880 2024-03-15T13:07:59.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13941983e0 con 0x7f13941004e0 2024-03-15T13:07:59.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.079+0000 7f139b1a2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13941985a0 con 0x7f13941055b0 2024-03-15T13:07:59.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f139a9a1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1394197880 0x7f1394197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f139a9a1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1394197880 0x7f1394197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60266/0 (socket says 172.21.15.2:60266) 2024-03-15T13:07:59.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f139a9a1640 1 -- 172.21.15.2:0/2136559569 learned_addr learned my addr 172.21.15.2:0/2136559569 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f139a1a0640 1 --2- 172.21.15.2:0/2136559569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13941004e0 0x7f139419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f139999f640 1 --2- 172.21.15.2:0/2136559569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13941055b0 0x7f139419dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f139a9a1640 1 -- 172.21.15.2:0/2136559569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13941055b0 msgr2=0x7f139419dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f139a9a1640 1 --2- 172.21.15.2:0/2136559569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13941055b0 0x7f139419dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f139a9a1640 1 -- 172.21.15.2:0/2136559569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13941004e0 msgr2=0x7f139419d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f139a9a1640 1 --2- 172.21.15.2:0/2136559569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13941004e0 0x7f139419d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f139a9a1640 1 -- 172.21.15.2:0/2136559569 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1384009560 con 0x7f1394197880 2024-03-15T13:07:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f139a9a1640 1 --2- 172.21.15.2:0/2136559569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1394197880 0x7f1394197d30 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f138c00eb10 tx=0x7f138c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f138b7fe640 1 -- 172.21.15.2:0/2136559569 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c004b20 con 0x7f1394197880 2024-03-15T13:07:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f138b7fe640 1 -- 172.21.15.2:0/2136559569 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f138c004c80 con 0x7f1394197880 2024-03-15T13:07:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f139b1a2640 1 -- 172.21.15.2:0/2136559569 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13941a4590 con 0x7f1394197880 2024-03-15T13:07:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f138b7fe640 1 -- 172.21.15.2:0/2136559569 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c010690 con 0x7f1394197880 2024-03-15T13:07:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.080+0000 7f139b1a2640 1 -- 172.21.15.2:0/2136559569 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13941a4ac0 con 0x7f1394197880 2024-03-15T13:07:59.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.083+0000 7f139efd9740 1 Processor -- start 2024-03-15T13:07:59.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.083+0000 7f138b7fe640 1 -- 172.21.15.2:0/2136559569 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f138c004df0 con 0x7f1394197880 2024-03-15T13:07:59.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.083+0000 7f139efd9740 1 -- start start 2024-03-15T13:07:59.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139efd9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd497440 0x5612fd497820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139efd9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497d60 0x5612fd4a0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139efd9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd4a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139efd9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fd452960 con 0x5612fd4a0940 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139efd9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fd406310 con 0x5612fd497440 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139efd9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fd403840 con 0x5612fd497d60 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139a1a0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497d60 0x5612fd4a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139999f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd4a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139a1a0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497d60 0x5612fd4a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:35176/0 (socket says 172.21.15.2:35176) 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139a1a0640 1 -- 172.21.15.2:0/1068338196 learned_addr learned my addr 172.21.15.2:0/1068338196 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139999f640 1 --2- 172.21.15.2:0/1068338196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd497440 0x5612fd497820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f138b7fe640 1 --2- 172.21.15.2:0/2136559569 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f13800a1c20 0x7f13800a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139a9a1640 1 --2- 172.21.15.2:0/2136559569 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f13800a1c20 0x7f13800a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f138b7fe640 1 -- 172.21.15.2:0/2136559569 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f138c014070 con 0x7f1394197880 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139a1a0640 1 -- 172.21.15.2:0/1068338196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd497440 msgr2=0x5612fd497820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139a1a0640 1 --2- 172.21.15.2:0/1068338196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd497440 0x5612fd497820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139a1a0640 1 -- 172.21.15.2:0/1068338196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 msgr2=0x5612fd4a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139a1a0640 1 --2- 172.21.15.2:0/1068338196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd4a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139a1a0640 1 -- 172.21.15.2:0/1068338196 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612fd3c1060 con 0x5612fd497d60 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.084+0000 7f139999f640 1 --2- 172.21.15.2:0/1068338196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd4a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.085+0000 7f139a1a0640 1 --2- 172.21.15.2:0/1068338196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497d60 0x5612fd4a0400 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1390009c90 tx=0x7f1390011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aa4c01101e99b7d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.085+0000 7f139a9a1640 1 --2- 172.21.15.2:0/2136559569 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f13800a1c20 0x7f13800a40e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f138c097410 tx=0x7f138c06cc40 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.085+0000 7f1388bfe640 1 -- 172.21.15.2:0/1068338196 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390009e40 con 0x5612fd497d60 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.085+0000 7f1388bfe640 1 -- 172.21.15.2:0/1068338196 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f13900054f0 con 0x5612fd497d60 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.085+0000 7f1388bfe640 1 -- 172.21.15.2:0/1068338196 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390019610 con 0x5612fd497d60 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.085+0000 7f139efd9740 1 -- 172.21.15.2:0/1068338196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497d60 msgr2=0x5612fd4a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.085+0000 7f139efd9740 1 --2- 172.21.15.2:0/1068338196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497d60 0x5612fd4a0400 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1390009c90 tx=0x7f1390011810 comp rx=0 tx=0).stop 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 1 -- 172.21.15.2:0/1068338196 shutdown_connections 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 1 --2- 172.21.15.2:0/1068338196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd4a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 1 --2- 172.21.15.2:0/1068338196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497d60 0x5612fd4a0400 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 1 --2- 172.21.15.2:0/1068338196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd497440 0x5612fd497820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 1 -- 172.21.15.2:0/1068338196 >> 172.21.15.2:0/1068338196 conn(0x5612fd491b70 msgr2=0x5612fd491f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 1 -- 172.21.15.2:0/1068338196 shutdown_connections 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 1 -- 172.21.15.2:0/1068338196 wait complete. 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 1 -- start start 2024-03-15T13:07:59.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 1 Processor -- start 2024-03-15T13:07:59.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497440 0x5612fd4a0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd497d60 0x5612fd49d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd49db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fd3ff020 con 0x5612fd4a0940 2024-03-15T13:07:59.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fd0d2430 con 0x5612fd497d60 2024-03-15T13:07:59.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fd08ee00 con 0x5612fd497440 2024-03-15T13:07:59.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139efd9740 10 client.0 _ll_register_callbacks cb 0x5612fd3a6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:59.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139999f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd49db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139999f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd49db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60276/0 (socket says 172.21.15.2:60276) 2024-03-15T13:07:59.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139999f640 1 -- 172.21.15.2:0/3964425012 learned_addr learned my addr 172.21.15.2:0/3964425012 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:59.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139999f640 1 --2- 172.21.15.2:0/3964425012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497440 0x5612fd4a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.086+0000 7f139a1a0640 1 --2- 172.21.15.2:0/3964425012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd497d60 0x5612fd49d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.087+0000 7f139999f640 1 -- 172.21.15.2:0/3964425012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497440 msgr2=0x5612fd4a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.087+0000 7f139999f640 1 --2- 172.21.15.2:0/3964425012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497440 0x5612fd4a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.087+0000 7f139999f640 1 -- 172.21.15.2:0/3964425012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd497d60 msgr2=0x5612fd49d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.087+0000 7f139999f640 1 --2- 172.21.15.2:0/3964425012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd497d60 0x5612fd49d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.087+0000 7f139999f640 1 -- 172.21.15.2:0/3964425012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13900098f0 con 0x5612fd4a0940 2024-03-15T13:07:59.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.087+0000 7f139999f640 1 --2- 172.21.15.2:0/3964425012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd49db80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1384005de0 tx=0x7f138402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9d0c04cc8d5ea8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.087+0000 7f137dffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:59.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.087+0000 7f137dffb640 1 -- 172.21.15.2:0/3964425012 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138402cc20 con 0x5612fd4a0940 2024-03-15T13:07:59.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.087+0000 7f137dffb640 1 -- 172.21.15.2:0/3964425012 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f1384004480 con 0x5612fd4a0940 2024-03-15T13:07:59.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.087+0000 7f139efd9740 1 -- 172.21.15.2:0/3964425012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612fd0fe7e0 con 0x5612fd4a0940 2024-03-15T13:07:59.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.087+0000 7f137dffb640 1 -- 172.21.15.2:0/3964425012 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1384004de0 con 0x5612fd4a0940 2024-03-15T13:07:59.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.087+0000 7f139efd9740 10 client.18297 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:59.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.087+0000 7f139efd9740 1 -- 172.21.15.2:0/3964425012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5612fd3fd6a0 con 0x5612fd4a0940 2024-03-15T13:07:59.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.087+0000 7f137dffb640 1 -- 172.21.15.2:0/3964425012 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5612fd0fe7e0 con 0x5612fd4a0940 2024-03-15T13:07:59.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.088+0000 7f137dffb640 1 -- 172.21.15.2:0/3964425012 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f13840039d0 con 0x5612fd4a0940 2024-03-15T13:07:59.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.088+0000 7f139efd9740 10 client.18297 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:59.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.088+0000 7f139efd9740 10 client.18297 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:07:59.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.088+0000 7f139efd9740 20 client.18297 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:59.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.088+0000 7f139efd9740 1 --2- 172.21.15.2:0/3964425012 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5612fd4b09b0 0x5612fd4b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.088+0000 7f139efd9740 4 client.18297 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:07:59.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.088+0000 7f139efd9740 1 -- 172.21.15.2:0/3964425012 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612fd13b720 con 0x5612fd4b09b0 2024-03-15T13:07:59.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.088+0000 7f139a1a0640 1 --2- 172.21.15.2:0/3964425012 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5612fd4b09b0 0x5612fd4b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.088+0000 7f137dffb640 10 client.18297 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:07:59.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.088+0000 7f139a1a0640 1 --2- 172.21.15.2:0/3964425012 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5612fd4b09b0 0x5612fd4b2da0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6abf20b728fecfa3 server_cookie=41fa8adf477d8bbc in_seq=0 out_seq=0 2024-03-15T13:07:59.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.089+0000 7f137dffb640 1 -- 172.21.15.2:0/3964425012 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5612fd13b720 con 0x5612fd4b09b0 2024-03-15T13:07:59.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.089+0000 7f137dffb640 10 client.18297 handle_command_reply: tid=0 2024-03-15T13:07:59.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.089+0000 7f139efd9740 1 client.18297 shutdown 2024-03-15T13:07:59.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.089+0000 7f139efd9740 1 -- 172.21.15.2:0/3964425012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 msgr2=0x5612fd49db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.089+0000 7f139efd9740 1 --2- 172.21.15.2:0/3964425012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd49db80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1384005de0 tx=0x7f138402bc30 comp rx=0 tx=0).stop 2024-03-15T13:07:59.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.089+0000 7f139efd9740 1 -- 172.21.15.2:0/3964425012 shutdown_connections 2024-03-15T13:07:59.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.089+0000 7f139efd9740 1 --2- 172.21.15.2:0/3964425012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd49db80 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.089+0000 7f139efd9740 1 --2- 172.21.15.2:0/3964425012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd497d60 0x5612fd49d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.089+0000 7f139efd9740 1 --2- 172.21.15.2:0/3964425012 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5612fd4b09b0 0x5612fd4b2da0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.089+0000 7f139efd9740 1 --2- 172.21.15.2:0/3964425012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497440 0x5612fd4a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.089+0000 7f137dffb640 0 client.18297 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:07:59.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.089+0000 7f139efd9740 1 -- 172.21.15.2:0/3964425012 >> 172.21.15.2:0/3964425012 conn(0x5612fd491b70 msgr2=0x5612fd4938c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.089+0000 7f139efd9740 1 -- 172.21.15.2:0/3964425012 shutdown_connections 2024-03-15T13:07:59.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.089+0000 7f139efd9740 1 -- 172.21.15.2:0/3964425012 wait complete. 2024-03-15T13:07:59.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.089+0000 7f139efd9740 20 client.18297 trim_cache size 0 max 16384 2024-03-15T13:07:59.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139efd9740 1 Processor -- start 2024-03-15T13:07:59.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139efd9740 1 -- start start 2024-03-15T13:07:59.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139efd9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497440 0x5612fd49fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139efd9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd497d60 0x5612fd4a0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139efd9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd4abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139efd9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fd384130 con 0x5612fd4a0940 2024-03-15T13:07:59.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139efd9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fd13b720 con 0x5612fd497d60 2024-03-15T13:07:59.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139efd9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fd49e180 con 0x5612fd497440 2024-03-15T13:07:59.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139999f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd4abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139a1a0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd497d60 0x5612fd4a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139999f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd4abcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60288/0 (socket says 172.21.15.2:60288) 2024-03-15T13:07:59.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139999f640 1 -- 172.21.15.2:0/951431050 learned_addr learned my addr 172.21.15.2:0/951431050 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:59.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139999f640 1 --2- 172.21.15.2:0/951431050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497440 0x5612fd49fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139999f640 1 -- 172.21.15.2:0/951431050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497440 msgr2=0x5612fd49fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139999f640 1 --2- 172.21.15.2:0/951431050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497440 0x5612fd49fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139999f640 1 -- 172.21.15.2:0/951431050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd497d60 msgr2=0x5612fd4a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139999f640 1 --2- 172.21.15.2:0/951431050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd497d60 0x5612fd4a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139999f640 1 -- 172.21.15.2:0/951431050 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612fd399c20 con 0x5612fd4a0940 2024-03-15T13:07:59.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.102+0000 7f139999f640 1 --2- 172.21.15.2:0/951431050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd4abcc0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f1384001eb0 tx=0x7f1384005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ead67b41a32bf64 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.103+0000 7f137ffff640 1 -- 172.21.15.2:0/951431050 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384003ed0 con 0x5612fd4a0940 2024-03-15T13:07:59.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.103+0000 7f137ffff640 1 -- 172.21.15.2:0/951431050 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1384049070 con 0x5612fd4a0940 2024-03-15T13:07:59.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.103+0000 7f137ffff640 1 -- 172.21.15.2:0/951431050 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384043990 con 0x5612fd4a0940 2024-03-15T13:07:59.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.103+0000 7f139efd9740 1 -- 172.21.15.2:0/951431050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 msgr2=0x5612fd4abcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.103+0000 7f139efd9740 1 --2- 172.21.15.2:0/951431050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd4abcc0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f1384001eb0 tx=0x7f1384005fd0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.103+0000 7f139efd9740 1 -- 172.21.15.2:0/951431050 shutdown_connections 2024-03-15T13:07:59.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.103+0000 7f139efd9740 1 --2- 172.21.15.2:0/951431050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd4a0940 0x5612fd4abcc0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.103+0000 7f139efd9740 1 --2- 172.21.15.2:0/951431050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd497d60 0x5612fd4a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.103+0000 7f139efd9740 1 --2- 172.21.15.2:0/951431050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497440 0x5612fd49fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.103+0000 7f139efd9740 1 -- 172.21.15.2:0/951431050 >> 172.21.15.2:0/951431050 conn(0x5612fd491b70 msgr2=0x5612fd493830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.103+0000 7f139efd9740 1 -- 172.21.15.2:0/951431050 shutdown_connections 2024-03-15T13:07:59.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.103+0000 7f139efd9740 1 -- 172.21.15.2:0/951431050 wait complete. 2024-03-15T13:07:59.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.103+0000 7f139efd9740 1 -- start start 2024-03-15T13:07:59.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.103+0000 7f139efd9740 1 Processor -- start 2024-03-15T13:07:59.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139efd9740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497440 0x5612fd4a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139efd9740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd497d60 0x5612fd4a4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139efd9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd4a0940 0x5612fd4a4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139efd9740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fd50b3c0 con 0x5612fd497d60 2024-03-15T13:07:59.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139efd9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fd50b530 con 0x5612fd4a0940 2024-03-15T13:07:59.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139efd9740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fd50b6f0 con 0x5612fd497440 2024-03-15T13:07:59.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139efd9740 10 client.0 _ll_register_callbacks cb 0x7f1384005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:59.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139a1a0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd497d60 0x5612fd4a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139a1a0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd497d60 0x5612fd4a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60290/0 (socket says 172.21.15.2:60290) 2024-03-15T13:07:59.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139a1a0640 1 -- 172.21.15.2:0/2703122673 learned_addr learned my addr 172.21.15.2:0/2703122673 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:59.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139999f640 1 --2- 172.21.15.2:0/2703122673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497440 0x5612fd4a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139999f640 1 --2- 172.21.15.2:0/2703122673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd4a0940 0x5612fd4a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139a1a0640 1 -- 172.21.15.2:0/2703122673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497440 msgr2=0x5612fd4a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139a1a0640 1 --2- 172.21.15.2:0/2703122673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497440 0x5612fd4a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139a1a0640 1 -- 172.21.15.2:0/2703122673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd4a0940 msgr2=0x5612fd4a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139a1a0640 1 --2- 172.21.15.2:0/2703122673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd4a0940 0x5612fd4a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139a1a0640 1 -- 172.21.15.2:0/2703122673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612fd3fd6a0 con 0x5612fd497d60 2024-03-15T13:07:59.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139999f640 1 --2- 172.21.15.2:0/2703122673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd4a0940 0x5612fd4a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:07:59.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139a1a0640 1 --2- 172.21.15.2:0/2703122673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd497d60 0x5612fd4a4420 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f1390011d10 tx=0x7f1390009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309f574a97bb1035 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f137dffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f137dffb640 1 -- 172.21.15.2:0/2703122673 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139002c280 con 0x5612fd497d60 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f137dffb640 1 -- 172.21.15.2:0/2703122673 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f1390027040 con 0x5612fd497d60 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.104+0000 7f139efd9740 1 -- 172.21.15.2:0/2703122673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612fd49d450 con 0x5612fd497d60 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.105+0000 7f137dffb640 1 -- 172.21.15.2:0/2703122673 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1390004740 con 0x5612fd497d60 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.105+0000 7f139efd9740 10 client.18303 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.105+0000 7f139efd9740 1 -- 172.21.15.2:0/2703122673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5612fd3bf8c0 con 0x5612fd497d60 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.105+0000 7f137dffb640 1 -- 172.21.15.2:0/2703122673 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5612fd49d450 con 0x5612fd497d60 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.105+0000 7f137dffb640 1 -- 172.21.15.2:0/2703122673 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f139002c630 con 0x5612fd497d60 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.105+0000 7f139efd9740 10 client.18303 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.105+0000 7f139efd9740 10 client.18303 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.105+0000 7f139efd9740 20 client.18303 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.105+0000 7f139efd9740 1 --2- 172.21.15.2:0/2703122673 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5612fd50ce60 0x5612fd50f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.105+0000 7f139efd9740 4 client.18303 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.105+0000 7f139efd9740 1 -- 172.21.15.2:0/2703122673 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f13900054f0 con 0x5612fd50ce60 2024-03-15T13:07:59.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.105+0000 7f139999f640 1 --2- 172.21.15.2:0/2703122673 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5612fd50ce60 0x5612fd50f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.105+0000 7f137dffb640 10 client.18303 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.105+0000 7f139999f640 1 --2- 172.21.15.2:0/2703122673 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5612fd50ce60 0x5612fd50f250 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5acc70355c8533b3 server_cookie=5b607a2a4868741d in_seq=0 out_seq=0 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f137dffb640 1 -- 172.21.15.2:0/2703122673 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f13900054f0 con 0x5612fd50ce60 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f137dffb640 10 client.18303 handle_command_reply: tid=0 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139efd9740 1 client.18303 shutdown 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139efd9740 1 -- 172.21.15.2:0/2703122673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd497d60 msgr2=0x5612fd4a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139efd9740 1 --2- 172.21.15.2:0/2703122673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd497d60 0x5612fd4a4420 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f1390011d10 tx=0x7f1390009640 comp rx=0 tx=0).stop 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139efd9740 1 -- 172.21.15.2:0/2703122673 shutdown_connections 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139efd9740 1 --2- 172.21.15.2:0/2703122673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612fd4a0940 0x5612fd4a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139efd9740 1 --2- 172.21.15.2:0/2703122673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612fd497d60 0x5612fd4a4420 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139efd9740 1 --2- 172.21.15.2:0/2703122673 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5612fd50ce60 0x5612fd50f250 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139efd9740 1 --2- 172.21.15.2:0/2703122673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612fd497440 0x5612fd4a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f137dffb640 0 client.18303 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139efd9740 1 -- 172.21.15.2:0/2703122673 >> 172.21.15.2:0/2703122673 conn(0x5612fd491b70 msgr2=0x5612fd490440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139efd9740 1 -- 172.21.15.2:0/2703122673 shutdown_connections 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139efd9740 1 -- 172.21.15.2:0/2703122673 wait complete. 2024-03-15T13:07:59.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139efd9740 20 client.18303 trim_cache size 0 max 16384 2024-03-15T13:07:59.097 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:07:59.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139b1a2640 1 -- 172.21.15.2:0/2136559569 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f13800a1c20 msgr2=0x7f13800a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139b1a2640 1 --2- 172.21.15.2:0/2136559569 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f13800a1c20 0x7f13800a40e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f138c097410 tx=0x7f138c06cc40 comp rx=0 tx=0).stop 2024-03-15T13:07:59.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139b1a2640 1 -- 172.21.15.2:0/2136559569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1394197880 msgr2=0x7f1394197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139b1a2640 1 --2- 172.21.15.2:0/2136559569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1394197880 0x7f1394197d30 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f138c00eb10 tx=0x7f138c00c2d0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139b1a2640 1 -- 172.21.15.2:0/2136559569 shutdown_connections 2024-03-15T13:07:59.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139b1a2640 1 --2- 172.21.15.2:0/2136559569 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f13800a1c20 0x7f13800a40e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139b1a2640 1 --2- 172.21.15.2:0/2136559569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1394197880 0x7f1394197d30 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139b1a2640 1 --2- 172.21.15.2:0/2136559569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13941055b0 0x7f139419dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139b1a2640 1 --2- 172.21.15.2:0/2136559569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13941004e0 0x7f139419d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.106+0000 7f139b1a2640 1 -- 172.21.15.2:0/2136559569 >> 172.21.15.2:0/2136559569 conn(0x7f13940fbd20 msgr2=0x7f13940fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.107+0000 7f139b1a2640 1 -- 172.21.15.2:0/2136559569 shutdown_connections 2024-03-15T13:07:59.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.107+0000 7f139b1a2640 1 -- 172.21.15.2:0/2136559569 wait complete. 2024-03-15T13:07:59.101 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:07:59.101 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:07:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.221+0000 7f1b2f9ac640 1 -- 172.21.15.2:0/2852469568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b28102700 msgr2=0x7f1b28102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.221+0000 7f1b2f9ac640 1 --2- 172.21.15.2:0/2852469568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b28102700 0x7f1b28102ae0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009900 tx=0x7f1b2402bcd0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.221+0000 7f1b2f9ac640 1 -- 172.21.15.2:0/2852469568 shutdown_connections 2024-03-15T13:07:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.221+0000 7f1b2f9ac640 1 --2- 172.21.15.2:0/2852469568 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b2810bad0 0x7f1b2810df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.221+0000 7f1b2f9ac640 1 --2- 172.21.15.2:0/2852469568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b28103020 0x7f1b2810b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.221+0000 7f1b2f9ac640 1 --2- 172.21.15.2:0/2852469568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b28102700 0x7f1b28102ae0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.221+0000 7f1b2f9ac640 1 -- 172.21.15.2:0/2852469568 >> 172.21.15.2:0/2852469568 conn(0x7f1b28076770 msgr2=0x7f1b28076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.221+0000 7f1b2f9ac640 1 -- 172.21.15.2:0/2852469568 shutdown_connections 2024-03-15T13:07:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.221+0000 7f1b2f9ac640 1 -- 172.21.15.2:0/2852469568 wait complete. 2024-03-15T13:07:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.222+0000 7f1b2f9ac640 1 Processor -- start 2024-03-15T13:07:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.222+0000 7f1b2f9ac640 1 -- start start 2024-03-15T13:07:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.222+0000 7f1b2f9ac640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b28102700 0x7f1b28100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.222+0000 7f1b2f9ac640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b28103020 0x7f1b28100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.222+0000 7f1b2f9ac640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b2810bad0 0x7f1b28100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.222+0000 7f1b2f9ac640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b281a5ec0 con 0x7f1b28103020 2024-03-15T13:07:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.222+0000 7f1b2f9ac640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b281a6030 con 0x7f1b2810bad0 2024-03-15T13:07:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.222+0000 7f1b2f9ac640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b281a61f0 con 0x7f1b28102700 2024-03-15T13:07:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.222+0000 7f1b2cf47640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b28103020 0x7f1b28100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.222+0000 7f1b2cf47640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b28103020 0x7f1b28100890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60312/0 (socket says 172.21.15.2:60312) 2024-03-15T13:07:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.222+0000 7f1b2cf47640 1 -- 172.21.15.2:0/1752933172 learned_addr learned my addr 172.21.15.2:0/1752933172 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.222+0000 7f1b2d748640 1 --2- 172.21.15.2:0/1752933172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b28102700 0x7f1b28100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.222+0000 7f1b2df49640 1 --2- 172.21.15.2:0/1752933172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b2810bad0 0x7f1b28100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.223+0000 7f1b2cf47640 1 -- 172.21.15.2:0/1752933172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b28102700 msgr2=0x7f1b28100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.223+0000 7f1b2cf47640 1 --2- 172.21.15.2:0/1752933172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b28102700 0x7f1b28100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.223+0000 7f1b2cf47640 1 -- 172.21.15.2:0/1752933172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b2810bad0 msgr2=0x7f1b28100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.223+0000 7f1b2cf47640 1 --2- 172.21.15.2:0/1752933172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b2810bad0 0x7f1b28100dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.223+0000 7f1b2cf47640 1 -- 172.21.15.2:0/1752933172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b24009560 con 0x7f1b28103020 2024-03-15T13:07:59.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.223+0000 7f1b2df49640 1 --2- 172.21.15.2:0/1752933172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b2810bad0 0x7f1b28100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:07:59.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.223+0000 7f1b2cf47640 1 --2- 172.21.15.2:0/1752933172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b28103020 0x7f1b28100890 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800e9f0 tx=0x7f1b1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.223+0000 7f1b1e7fc640 1 -- 172.21.15.2:0/1752933172 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b18009700 con 0x7f1b28103020 2024-03-15T13:07:59.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.223+0000 7f1b1e7fc640 1 -- 172.21.15.2:0/1752933172 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1b1800cd80 con 0x7f1b28103020 2024-03-15T13:07:59.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.223+0000 7f1b2f9ac640 1 -- 172.21.15.2:0/1752933172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b281a64d0 con 0x7f1b28103020 2024-03-15T13:07:59.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.223+0000 7f1b1e7fc640 1 -- 172.21.15.2:0/1752933172 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b18018920 con 0x7f1b28103020 2024-03-15T13:07:59.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.223+0000 7f1b2f9ac640 1 -- 172.21.15.2:0/1752933172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b281a69e0 con 0x7f1b28103020 2024-03-15T13:07:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.225+0000 7f1b1e7fc640 1 -- 172.21.15.2:0/1752933172 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f1b18018a80 con 0x7f1b28103020 2024-03-15T13:07:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.225+0000 7f1b32587740 1 Processor -- start 2024-03-15T13:07:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.225+0000 7f1b32587740 1 -- start start 2024-03-15T13:07:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b1e7fc640 1 --2- 172.21.15.2:0/1752933172 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1b140a1bd0 0x7f1b140a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b32587740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8250 0x5566767d8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b32587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8b70 0x5566767e1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b32587740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 0x5566767e3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b32587740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556676793960 con 0x5566767e1750 2024-03-15T13:07:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b32587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556676747310 con 0x5566767d8b70 2024-03-15T13:07:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b32587740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556676744840 con 0x5566767d8250 2024-03-15T13:07:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b1e7fc640 1 -- 172.21.15.2:0/1752933172 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f1b18014070 con 0x7f1b28103020 2024-03-15T13:07:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b2df49640 1 --2- 172.21.15.2:0/1752933172 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1b140a1bd0 0x7f1b140a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b2cf47640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8b70 0x5566767e1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b2d748640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8250 0x5566767d8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b2cf47640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8b70 0x5566767e1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:58442/0 (socket says 172.21.15.2:58442) 2024-03-15T13:07:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b2d748640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8250 0x5566767d8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:35226/0 (socket says 172.21.15.2:35226) 2024-03-15T13:07:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b2cf47640 1 -- 172.21.15.2:0/1970565006 learned_addr learned my addr 172.21.15.2:0/1970565006 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b2df49640 1 --2- 172.21.15.2:0/1970565006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 0x5566767e3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b2cf47640 1 -- 172.21.15.2:0/1970565006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8250 msgr2=0x5566767d8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b2cf47640 1 --2- 172.21.15.2:0/1970565006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8250 0x5566767d8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b2cf47640 1 -- 172.21.15.2:0/1970565006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 msgr2=0x5566767e3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b2cf47640 1 --2- 172.21.15.2:0/1970565006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 0x5566767e3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b2cf47640 1 -- 172.21.15.2:0/1970565006 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556676702060 con 0x5566767d8b70 2024-03-15T13:07:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b2df49640 1 --2- 172.21.15.2:0/1752933172 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1b140a1bd0 0x7f1b140a4090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009270 tx=0x7f1b20009e00 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.226+0000 7f1b2cf47640 1 --2- 172.21.15.2:0/1970565006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8b70 0x5566767e1210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1b18096c30 tx=0x7f1b1809a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ab4e6b4ca5b365 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.227+0000 7f1b137fe640 1 -- 172.21.15.2:0/1970565006 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1809e920 con 0x5566767d8b70 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.227+0000 7f1b137fe640 1 -- 172.21.15.2:0/1970565006 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1b180a2f70 con 0x5566767d8b70 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.227+0000 7f1b137fe640 1 -- 172.21.15.2:0/1970565006 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b18099c20 con 0x5566767d8b70 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.227+0000 7f1b32587740 1 -- 172.21.15.2:0/1970565006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8b70 msgr2=0x5566767e1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.227+0000 7f1b32587740 1 --2- 172.21.15.2:0/1970565006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8b70 0x5566767e1210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1b18096c30 tx=0x7f1b1809a1f0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.227+0000 7f1b32587740 1 -- 172.21.15.2:0/1970565006 shutdown_connections 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.227+0000 7f1b32587740 1 --2- 172.21.15.2:0/1970565006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 0x5566767e3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.227+0000 7f1b32587740 1 --2- 172.21.15.2:0/1970565006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8b70 0x5566767e1210 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.227+0000 7f1b32587740 1 --2- 172.21.15.2:0/1970565006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8250 0x5566767d8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.227+0000 7f1b32587740 1 -- 172.21.15.2:0/1970565006 >> 172.21.15.2:0/1970565006 conn(0x5566767d2980 msgr2=0x5566767d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.227+0000 7f1b32587740 1 -- 172.21.15.2:0/1970565006 shutdown_connections 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.227+0000 7f1b32587740 1 -- 172.21.15.2:0/1970565006 wait complete. 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.228+0000 7f1b32587740 1 -- start start 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.228+0000 7f1b32587740 1 Processor -- start 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.228+0000 7f1b32587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8250 0x5566767e11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.228+0000 7f1b32587740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8b70 0x5566767de4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.228+0000 7f1b32587740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 0x5566767debc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.228+0000 7f1b32587740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556676740020 con 0x5566767e1750 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.228+0000 7f1b32587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556676413430 con 0x5566767d8250 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.228+0000 7f1b32587740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566763cfe00 con 0x5566767d8b70 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.228+0000 7f1b32587740 10 client.0 _ll_register_callbacks cb 0x5566766e7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.228+0000 7f1b2df49640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 0x5566767debc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.228+0000 7f1b2cf47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8b70 0x5566767de4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.228+0000 7f1b2df49640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 0x5566767debc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60336/0 (socket says 172.21.15.2:60336) 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.228+0000 7f1b2df49640 1 -- 172.21.15.2:0/1570359563 learned_addr learned my addr 172.21.15.2:0/1570359563 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.228+0000 7f1b2d748640 1 --2- 172.21.15.2:0/1570359563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8250 0x5566767e11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b2df49640 1 -- 172.21.15.2:0/1570359563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8b70 msgr2=0x5566767de4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b2df49640 1 --2- 172.21.15.2:0/1570359563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8b70 0x5566767de4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b2df49640 1 -- 172.21.15.2:0/1570359563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8250 msgr2=0x5566767e11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b2df49640 1 --2- 172.21.15.2:0/1570359563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8250 0x5566767e11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b2df49640 1 -- 172.21.15.2:0/1570359563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b281a69e0 con 0x5566767e1750 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b2cf47640 1 --2- 172.21.15.2:0/1570359563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8b70 0x5566767de4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:59.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b2df49640 1 --2- 172.21.15.2:0/1570359563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 0x5566767debc0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f1b20012780 tx=0x7f1b20009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ed35a36f058979 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b10ff9640 1 -- 172.21.15.2:0/1570359563 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b20009a20 con 0x5566767e1750 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b10ff9640 1 -- 172.21.15.2:0/1570359563 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f1b20026070 con 0x5566767e1750 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b32587740 1 -- 172.21.15.2:0/1570359563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55667643f7e0 con 0x5566767e1750 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b10ff9640 1 -- 172.21.15.2:0/1570359563 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b20020620 con 0x5566767e1750 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b32587740 10 client.18315 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b32587740 1 -- 172.21.15.2:0/1570359563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55667673e6a0 con 0x5566767e1750 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b10ff9640 1 -- 172.21.15.2:0/1570359563 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55667643f7e0 con 0x5566767e1750 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.229+0000 7f1b10ff9640 1 -- 172.21.15.2:0/1570359563 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1b20029840 con 0x5566767e1750 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.230+0000 7f1b32587740 10 client.18315 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.230+0000 7f1b32587740 10 client.18315 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.230+0000 7f1b32587740 20 client.18315 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.230+0000 7f1b32587740 1 --2- 172.21.15.2:0/1570359563 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5566767f1970 0x5566767f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.230+0000 7f1b32587740 4 client.18315 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.230+0000 7f1b32587740 1 -- 172.21.15.2:0/1570359563 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55667647c720 con 0x5566767f1970 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.230+0000 7f1b2d748640 1 --2- 172.21.15.2:0/1570359563 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5566767f1970 0x5566767f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.230+0000 7f1b10ff9640 10 client.18315 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.230+0000 7f1b2d748640 1 --2- 172.21.15.2:0/1570359563 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5566767f1970 0x5566767f3d90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b950c27be5066b2 server_cookie=52895cff264cb9f8 in_seq=0 out_seq=0 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.231+0000 7f1b10ff9640 1 -- 172.21.15.2:0/1570359563 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55667647c720 con 0x5566767f1970 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.231+0000 7f1b10ff9640 10 client.18315 handle_command_reply: tid=0 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.231+0000 7f1b32587740 1 client.18315 shutdown 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.231+0000 7f1b32587740 1 -- 172.21.15.2:0/1570359563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 msgr2=0x5566767debc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.231+0000 7f1b32587740 1 --2- 172.21.15.2:0/1570359563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 0x5566767debc0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f1b20012780 tx=0x7f1b20009500 comp rx=0 tx=0).stop 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.231+0000 7f1b32587740 1 -- 172.21.15.2:0/1570359563 shutdown_connections 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.231+0000 7f1b32587740 1 --2- 172.21.15.2:0/1570359563 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x5566767f1970 0x5566767f3d90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.231+0000 7f1b32587740 1 --2- 172.21.15.2:0/1570359563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 0x5566767debc0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.231+0000 7f1b10ff9640 0 client.18315 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.231+0000 7f1b32587740 1 --2- 172.21.15.2:0/1570359563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8b70 0x5566767de4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.231+0000 7f1b32587740 1 --2- 172.21.15.2:0/1570359563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8250 0x5566767e11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.231+0000 7f1b32587740 1 -- 172.21.15.2:0/1570359563 >> 172.21.15.2:0/1570359563 conn(0x5566767d2980 msgr2=0x5566767d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.231+0000 7f1b32587740 1 -- 172.21.15.2:0/1570359563 shutdown_connections 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.231+0000 7f1b32587740 1 -- 172.21.15.2:0/1570359563 wait complete. 2024-03-15T13:07:59.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.231+0000 7f1b32587740 20 client.18315 trim_cache size 0 max 16384 2024-03-15T13:07:59.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.243+0000 7f1b32587740 1 Processor -- start 2024-03-15T13:07:59.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.243+0000 7f1b32587740 1 -- start start 2024-03-15T13:07:59.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b32587740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8250 0x5566767d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b32587740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767d8b70 0x5566767e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b32587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767e1750 0x5566767ecad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b32587740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566766c5130 con 0x5566767d8b70 2024-03-15T13:07:59.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b32587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566767c4a40 con 0x5566767e1750 2024-03-15T13:07:59.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b32587740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566767df210 con 0x5566767d8250 2024-03-15T13:07:59.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b2cf47640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767d8b70 0x5566767e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b2cf47640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767d8b70 0x5566767e0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60346/0 (socket says 172.21.15.2:60346) 2024-03-15T13:07:59.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b2cf47640 1 -- 172.21.15.2:0/2161685731 learned_addr learned my addr 172.21.15.2:0/2161685731 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:59.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b2d748640 1 --2- 172.21.15.2:0/2161685731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8250 0x5566767d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b2df49640 1 --2- 172.21.15.2:0/2161685731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767e1750 0x5566767ecad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b2cf47640 1 -- 172.21.15.2:0/2161685731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8250 msgr2=0x5566767d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b2cf47640 1 --2- 172.21.15.2:0/2161685731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8250 0x5566767d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b2cf47640 1 -- 172.21.15.2:0/2161685731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767e1750 msgr2=0x5566767ecad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b2cf47640 1 --2- 172.21.15.2:0/2161685731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767e1750 0x5566767ecad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b2cf47640 1 -- 172.21.15.2:0/2161685731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566766dac20 con 0x5566767d8b70 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b2cf47640 1 --2- 172.21.15.2:0/2161685731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767d8b70 0x5566767e0ce0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009230 tx=0x7f1b18000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01fa09b3857135d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b12ffd640 1 -- 172.21.15.2:0/2161685731 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1809e920 con 0x5566767d8b70 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b12ffd640 1 -- 172.21.15.2:0/2161685731 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1b18010be0 con 0x5566767d8b70 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.244+0000 7f1b12ffd640 1 -- 172.21.15.2:0/2161685731 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b18085310 con 0x5566767d8b70 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.245+0000 7f1b32587740 1 -- 172.21.15.2:0/2161685731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767d8b70 msgr2=0x5566767e0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.245+0000 7f1b32587740 1 --2- 172.21.15.2:0/2161685731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767d8b70 0x5566767e0ce0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009230 tx=0x7f1b18000bc0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.245+0000 7f1b32587740 1 -- 172.21.15.2:0/2161685731 shutdown_connections 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.245+0000 7f1b32587740 1 --2- 172.21.15.2:0/2161685731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767e1750 0x5566767ecad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.245+0000 7f1b32587740 1 --2- 172.21.15.2:0/2161685731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767d8b70 0x5566767e0ce0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.245+0000 7f1b32587740 1 --2- 172.21.15.2:0/2161685731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8250 0x5566767d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.245+0000 7f1b32587740 1 -- 172.21.15.2:0/2161685731 >> 172.21.15.2:0/2161685731 conn(0x5566767d2980 msgr2=0x5566767d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.245+0000 7f1b32587740 1 -- 172.21.15.2:0/2161685731 shutdown_connections 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.245+0000 7f1b32587740 1 -- 172.21.15.2:0/2161685731 wait complete. 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.245+0000 7f1b32587740 1 -- start start 2024-03-15T13:07:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.245+0000 7f1b32587740 1 Processor -- start 2024-03-15T13:07:59.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b32587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8250 0x5566767e4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b32587740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8b70 0x5566767e51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b32587740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 0x5566767e56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b32587740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55667684c340 con 0x5566767e1750 2024-03-15T13:07:59.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b32587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55667684c4b0 con 0x5566767d8250 2024-03-15T13:07:59.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b32587740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55667684c670 con 0x5566767d8b70 2024-03-15T13:07:59.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b32587740 10 client.0 _ll_register_callbacks cb 0x7f1b20010f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:59.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b2df49640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 0x5566767e56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b2d748640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8250 0x5566767e4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b2cf47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8b70 0x5566767e51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b2d748640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8250 0x5566767e4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:58462/0 (socket says 172.21.15.2:58462) 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b2d748640 1 -- 172.21.15.2:0/2077526527 learned_addr learned my addr 172.21.15.2:0/2077526527 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b2df49640 1 -- 172.21.15.2:0/2077526527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8b70 msgr2=0x5566767e51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b2df49640 1 --2- 172.21.15.2:0/2077526527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8b70 0x5566767e51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b2df49640 1 -- 172.21.15.2:0/2077526527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8250 msgr2=0x5566767e4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b2df49640 1 --2- 172.21.15.2:0/2077526527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8250 0x5566767e4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b2df49640 1 -- 172.21.15.2:0/2077526527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b180ba040 con 0x5566767e1750 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b2cf47640 1 --2- 172.21.15.2:0/2077526527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8b70 0x5566767e51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b2d748640 1 --2- 172.21.15.2:0/2077526527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8250 0x5566767e4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b2df49640 1 --2- 172.21.15.2:0/2077526527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 0x5566767e56f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1b20000f80 tx=0x7f1b20010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b7a685214f8cbe server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b10ff9640 1 -- 172.21.15.2:0/2077526527 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2003c070 con 0x5566767e1750 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b10ff9640 1 -- 172.21.15.2:0/2077526527 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f1b20026040 con 0x5566767e1750 2024-03-15T13:07:59.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b32587740 1 -- 172.21.15.2:0/2077526527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b180a2f70 con 0x5566767e1750 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.246+0000 7f1b10ff9640 1 -- 172.21.15.2:0/2077526527 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b20009900 con 0x5566767e1750 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.247+0000 7f1b32587740 10 client.18321 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.247+0000 7f1b32587740 1 -- 172.21.15.2:0/2077526527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5566767008c0 con 0x5566767e1750 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.247+0000 7f1b10ff9640 1 -- 172.21.15.2:0/2077526527 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f1b180a2f70 con 0x5566767e1750 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.247+0000 7f1b10ff9640 1 -- 172.21.15.2:0/2077526527 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f1b2003f020 con 0x5566767e1750 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.247+0000 7f1b32587740 10 client.18321 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.247+0000 7f1b32587740 10 client.18321 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.247+0000 7f1b32587740 20 client.18321 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.247+0000 7f1b32587740 1 --2- 172.21.15.2:0/2077526527 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55667684dd60 0x556676850180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.247+0000 7f1b32587740 4 client.18321 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.247+0000 7f1b32587740 1 -- 172.21.15.2:0/2077526527 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5566764cb890 con 0x55667684dd60 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.247+0000 7f1b2d748640 1 --2- 172.21.15.2:0/2077526527 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55667684dd60 0x556676850180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.247+0000 7f1b2d748640 1 --2- 172.21.15.2:0/2077526527 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55667684dd60 0x556676850180 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73c5b4dfa2c5ca server_cookie=db00f76acce6b710 in_seq=0 out_seq=0 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.247+0000 7f1b10ff9640 10 client.18321 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b10ff9640 1 -- 172.21.15.2:0/2077526527 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5566764cb890 con 0x55667684dd60 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b10ff9640 10 client.18321 handle_command_reply: tid=0 2024-03-15T13:07:59.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b32587740 1 client.18321 shutdown 2024-03-15T13:07:59.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b32587740 1 -- 172.21.15.2:0/2077526527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 msgr2=0x5566767e56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b32587740 1 --2- 172.21.15.2:0/2077526527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 0x5566767e56f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1b20000f80 tx=0x7f1b20010d60 comp rx=0 tx=0).stop 2024-03-15T13:07:59.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b32587740 1 -- 172.21.15.2:0/2077526527 shutdown_connections 2024-03-15T13:07:59.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b32587740 1 --2- 172.21.15.2:0/2077526527 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55667684dd60 0x556676850180 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b32587740 1 --2- 172.21.15.2:0/2077526527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566767e1750 0x5566767e56f0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b32587740 1 --2- 172.21.15.2:0/2077526527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566767d8b70 0x5566767e51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b10ff9640 0 client.18321 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:07:59.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b32587740 1 --2- 172.21.15.2:0/2077526527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566767d8250 0x5566767e4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b32587740 1 -- 172.21.15.2:0/2077526527 >> 172.21.15.2:0/2077526527 conn(0x5566767d2980 msgr2=0x5566767d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b32587740 1 -- 172.21.15.2:0/2077526527 shutdown_connections 2024-03-15T13:07:59.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b32587740 1 -- 172.21.15.2:0/2077526527 wait complete. 2024-03-15T13:07:59.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b32587740 20 client.18321 trim_cache size 0 max 16384 2024-03-15T13:07:59.239 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:07:59.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b2f9ac640 1 -- 172.21.15.2:0/1752933172 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1b140a1bd0 msgr2=0x7f1b140a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b2f9ac640 1 --2- 172.21.15.2:0/1752933172 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1b140a1bd0 0x7f1b140a4090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009270 tx=0x7f1b20009e00 comp rx=0 tx=0).stop 2024-03-15T13:07:59.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b2f9ac640 1 -- 172.21.15.2:0/1752933172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b28103020 msgr2=0x7f1b28100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b2f9ac640 1 --2- 172.21.15.2:0/1752933172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b28103020 0x7f1b28100890 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800e9f0 tx=0x7f1b1800eeb0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b2f9ac640 1 -- 172.21.15.2:0/1752933172 shutdown_connections 2024-03-15T13:07:59.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b2f9ac640 1 --2- 172.21.15.2:0/1752933172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b2810bad0 0x7f1b28100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b2f9ac640 1 --2- 172.21.15.2:0/1752933172 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f1b140a1bd0 0x7f1b140a4090 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b2f9ac640 1 --2- 172.21.15.2:0/1752933172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1b28103020 0x7f1b28100890 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b2f9ac640 1 --2- 172.21.15.2:0/1752933172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b28102700 0x7f1b28100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b2f9ac640 1 -- 172.21.15.2:0/1752933172 >> 172.21.15.2:0/1752933172 conn(0x7f1b28076770 msgr2=0x7f1b280fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b2f9ac640 1 -- 172.21.15.2:0/1752933172 shutdown_connections 2024-03-15T13:07:59.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.248+0000 7f1b2f9ac640 1 -- 172.21.15.2:0/1752933172 wait complete. 2024-03-15T13:07:59.243 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:07:59.243 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:07:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.363+0000 7fd894aa7640 1 -- 172.21.15.2:0/1197968403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8901014a0 msgr2=0x7fd890101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.363+0000 7fd894aa7640 1 --2- 172.21.15.2:0/1197968403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8901014a0 0x7fd890101920 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd884009900 tx=0x7fd88402bc30 comp rx=0 tx=0).stop 2024-03-15T13:07:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.363+0000 7fd894aa7640 1 -- 172.21.15.2:0/1197968403 shutdown_connections 2024-03-15T13:07:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.363+0000 7fd894aa7640 1 --2- 172.21.15.2:0/1197968403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890101e60 0x7fd89010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.363+0000 7fd894aa7640 1 --2- 172.21.15.2:0/1197968403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8901014a0 0x7fd890101920 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.363+0000 7fd894aa7640 1 --2- 172.21.15.2:0/1197968403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd890107560 0x7fd890107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.363+0000 7fd894aa7640 1 -- 172.21.15.2:0/1197968403 >> 172.21.15.2:0/1197968403 conn(0x7fd8900fcf90 msgr2=0x7fd8900ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.363+0000 7fd894aa7640 1 -- 172.21.15.2:0/1197968403 shutdown_connections 2024-03-15T13:07:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.363+0000 7fd894aa7640 1 -- 172.21.15.2:0/1197968403 wait complete. 2024-03-15T13:07:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd894aa7640 1 Processor -- start 2024-03-15T13:07:59.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd894aa7640 1 -- start start 2024-03-15T13:07:59.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd894aa7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8901014a0 0x7fd89019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd894aa7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd890101e60 0x7fd89019dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd894aa7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890107560 0x7fd8901978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd894aa7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd890197f20 con 0x7fd890101e60 2024-03-15T13:07:59.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd894aa7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd890198090 con 0x7fd890107560 2024-03-15T13:07:59.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd894aa7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd890198250 con 0x7fd8901014a0 2024-03-15T13:07:59.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd88dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd890101e60 0x7fd89019dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd88dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd890101e60 0x7fd89019dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60378/0 (socket says 172.21.15.2:60378) 2024-03-15T13:07:59.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd88dd9b640 1 -- 172.21.15.2:0/1801840050 learned_addr learned my addr 172.21.15.2:0/1801840050 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:59.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd88ed9d640 1 --2- 172.21.15.2:0/1801840050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890107560 0x7fd8901978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd88e59c640 1 --2- 172.21.15.2:0/1801840050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8901014a0 0x7fd89019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd88dd9b640 1 -- 172.21.15.2:0/1801840050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8901014a0 msgr2=0x7fd89019d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd88dd9b640 1 --2- 172.21.15.2:0/1801840050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8901014a0 0x7fd89019d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd88dd9b640 1 -- 172.21.15.2:0/1801840050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890107560 msgr2=0x7fd8901978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd88dd9b640 1 --2- 172.21.15.2:0/1801840050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890107560 0x7fd8901978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd88dd9b640 1 -- 172.21.15.2:0/1801840050 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd884009560 con 0x7fd890101e60 2024-03-15T13:07:59.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.364+0000 7fd88dd9b640 1 --2- 172.21.15.2:0/1801840050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd890101e60 0x7fd89019dc60 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fd884009230 tx=0x7fd88402cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.365+0000 7fd87f7fe640 1 -- 172.21.15.2:0/1801840050 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884004250 con 0x7fd890101e60 2024-03-15T13:07:59.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.365+0000 7fd894aa7640 1 -- 172.21.15.2:0/1801840050 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8901984d0 con 0x7fd890101e60 2024-03-15T13:07:59.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.365+0000 7fd87f7fe640 1 -- 172.21.15.2:0/1801840050 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd8840043b0 con 0x7fd890101e60 2024-03-15T13:07:59.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.365+0000 7fd87f7fe640 1 -- 172.21.15.2:0/1801840050 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884003bf0 con 0x7fd890101e60 2024-03-15T13:07:59.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.365+0000 7fd894aa7640 1 -- 172.21.15.2:0/1801840050 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8901a47c0 con 0x7fd890101e60 2024-03-15T13:07:59.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.367+0000 7fd87f7fe640 1 -- 172.21.15.2:0/1801840050 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fd884003d50 con 0x7fd890101e60 2024-03-15T13:07:59.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.367+0000 7fd897766740 1 Processor -- start 2024-03-15T13:07:59.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd897766740 1 -- start start 2024-03-15T13:07:59.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd87f7fe640 1 --2- 172.21.15.2:0/1801840050 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd8780a1ba0 0x7fd8780a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd897766740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211250 0x55be3b211630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd897766740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 0x55be3b21a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd897766740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b21a750 0x55be3b21cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd897766740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3b1cc960 con 0x55be3b211250 2024-03-15T13:07:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd897766740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3b180310 con 0x55be3b211b70 2024-03-15T13:07:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd897766740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3b17d840 con 0x55be3b21a750 2024-03-15T13:07:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd88ed9d640 1 --2- 172.21.15.2:0/1801840050 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd8780a1ba0 0x7fd8780a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd88e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211250 0x55be3b211630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd87f7fe640 1 -- 172.21.15.2:0/1801840050 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd8840efb10 con 0x7fd890101e60 2024-03-15T13:07:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd88e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211250 0x55be3b211630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60388/0 (socket says 172.21.15.2:60388) 2024-03-15T13:07:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd88e59c640 1 -- 172.21.15.2:0/457534424 learned_addr learned my addr 172.21.15.2:0/457534424 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd88dd9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 0x55be3b21a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd88ed9d640 1 --2- 172.21.15.2:0/457534424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b21a750 0x55be3b21cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd88e59c640 1 -- 172.21.15.2:0/457534424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b21a750 msgr2=0x55be3b21cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd88e59c640 1 --2- 172.21.15.2:0/457534424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b21a750 0x55be3b21cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd88e59c640 1 -- 172.21.15.2:0/457534424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 msgr2=0x55be3b21a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd88e59c640 1 --2- 172.21.15.2:0/457534424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 0x55be3b21a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd88e59c640 1 -- 172.21.15.2:0/457534424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be3b13b060 con 0x55be3b211250 2024-03-15T13:07:59.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.368+0000 7fd88dd9b640 1 --2- 172.21.15.2:0/457534424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 0x55be3b21a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:59.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.369+0000 7fd88ed9d640 1 --2- 172.21.15.2:0/1801840050 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd8780a1ba0 0x7fd8780a4060 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd888009fd0 tx=0x7fd888013b80 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.369+0000 7fd88e59c640 1 --2- 172.21.15.2:0/457534424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211250 0x55be3b211630 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fd880009c20 tx=0x7fd880011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c48a0f6b7629d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.369+0000 7fd87cff9640 1 -- 172.21.15.2:0/457534424 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880005220 con 0x55be3b211250 2024-03-15T13:07:59.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.369+0000 7fd87cff9640 1 -- 172.21.15.2:0/457534424 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd880009e90 con 0x55be3b211250 2024-03-15T13:07:59.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.369+0000 7fd87cff9640 1 -- 172.21.15.2:0/457534424 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880019690 con 0x55be3b211250 2024-03-15T13:07:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.369+0000 7fd897766740 1 -- 172.21.15.2:0/457534424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211250 msgr2=0x55be3b211630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.369+0000 7fd897766740 1 --2- 172.21.15.2:0/457534424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211250 0x55be3b211630 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fd880009c20 tx=0x7fd880011810 comp rx=0 tx=0).stop 2024-03-15T13:07:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 1 -- 172.21.15.2:0/457534424 shutdown_connections 2024-03-15T13:07:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 1 --2- 172.21.15.2:0/457534424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b21a750 0x55be3b21cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 1 --2- 172.21.15.2:0/457534424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 0x55be3b21a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 1 --2- 172.21.15.2:0/457534424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211250 0x55be3b211630 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 1 -- 172.21.15.2:0/457534424 >> 172.21.15.2:0/457534424 conn(0x55be3b20b980 msgr2=0x55be3b20bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 1 -- 172.21.15.2:0/457534424 shutdown_connections 2024-03-15T13:07:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 1 -- 172.21.15.2:0/457534424 wait complete. 2024-03-15T13:07:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 1 -- start start 2024-03-15T13:07:59.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 1 Processor -- start 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211250 0x55be3b21a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 0x55be3b217430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b21a750 0x55be3b217990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3b179020 con 0x55be3b211250 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3ae4c430 con 0x55be3b211b70 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3ae08e00 con 0x55be3b21a750 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd897766740 10 client.0 _ll_register_callbacks cb 0x55be3b120cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd88e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211250 0x55be3b21a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd88e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211250 0x55be3b21a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60392/0 (socket says 172.21.15.2:60392) 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd88e59c640 1 -- 172.21.15.2:0/504676886 learned_addr learned my addr 172.21.15.2:0/504676886 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd88dd9b640 1 --2- 172.21.15.2:0/504676886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 0x55be3b217430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.370+0000 7fd88ed9d640 1 --2- 172.21.15.2:0/504676886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b21a750 0x55be3b217990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.371+0000 7fd88e59c640 1 -- 172.21.15.2:0/504676886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b21a750 msgr2=0x55be3b217990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.371+0000 7fd88e59c640 1 --2- 172.21.15.2:0/504676886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b21a750 0x55be3b217990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.371+0000 7fd88e59c640 1 -- 172.21.15.2:0/504676886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 msgr2=0x55be3b217430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.371+0000 7fd88e59c640 1 --2- 172.21.15.2:0/504676886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 0x55be3b217430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.371+0000 7fd88e59c640 1 -- 172.21.15.2:0/504676886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8800098d0 con 0x55be3b211250 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.371+0000 7fd88e59c640 1 --2- 172.21.15.2:0/504676886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211250 0x55be3b21a0a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd88001b040 tx=0x7fd880004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef1f90f6466a707 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.371+0000 7fd875ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.371+0000 7fd875ffb640 1 -- 172.21.15.2:0/504676886 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880023430 con 0x55be3b211250 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.371+0000 7fd875ffb640 1 -- 172.21.15.2:0/504676886 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd880023590 con 0x55be3b211250 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.371+0000 7fd897766740 1 -- 172.21.15.2:0/504676886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be3ae787e0 con 0x55be3b211250 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.371+0000 7fd875ffb640 1 -- 172.21.15.2:0/504676886 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd880023750 con 0x55be3b211250 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.371+0000 7fd897766740 10 client.18333 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.371+0000 7fd897766740 1 -- 172.21.15.2:0/504676886 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55be3b1776a0 con 0x55be3b211250 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.371+0000 7fd875ffb640 1 -- 172.21.15.2:0/504676886 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55be3ae787e0 con 0x55be3b211250 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.372+0000 7fd875ffb640 1 -- 172.21.15.2:0/504676886 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd880021040 con 0x55be3b211250 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.372+0000 7fd897766740 10 client.18333 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.372+0000 7fd897766740 10 client.18333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.372+0000 7fd897766740 20 client.18333 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.372+0000 7fd897766740 1 --2- 172.21.15.2:0/504676886 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55be3b22a800 0x55be3b22cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.372+0000 7fd897766740 4 client.18333 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.372+0000 7fd897766740 1 -- 172.21.15.2:0/504676886 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be3aeb5720 con 0x55be3b22a800 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.372+0000 7fd88dd9b640 1 --2- 172.21.15.2:0/504676886 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55be3b22a800 0x55be3b22cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.372+0000 7fd875ffb640 10 client.18333 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.372+0000 7fd88dd9b640 1 --2- 172.21.15.2:0/504676886 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55be3b22a800 0x55be3b22cbf0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1185a97bf65dc578 server_cookie=9ff8f1952197c55a in_seq=0 out_seq=0 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.373+0000 7fd875ffb640 1 -- 172.21.15.2:0/504676886 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55be3aeb5720 con 0x55be3b22a800 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.373+0000 7fd875ffb640 10 client.18333 handle_command_reply: tid=0 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.373+0000 7fd897766740 1 client.18333 shutdown 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.373+0000 7fd897766740 1 -- 172.21.15.2:0/504676886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211250 msgr2=0x55be3b21a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.373+0000 7fd897766740 1 --2- 172.21.15.2:0/504676886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211250 0x55be3b21a0a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd88001b040 tx=0x7fd880004970 comp rx=0 tx=0).stop 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.373+0000 7fd897766740 1 -- 172.21.15.2:0/504676886 shutdown_connections 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.373+0000 7fd897766740 1 --2- 172.21.15.2:0/504676886 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55be3b22a800 0x55be3b22cbf0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.373+0000 7fd897766740 1 --2- 172.21.15.2:0/504676886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b21a750 0x55be3b217990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.373+0000 7fd897766740 1 --2- 172.21.15.2:0/504676886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 0x55be3b217430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.373+0000 7fd897766740 1 --2- 172.21.15.2:0/504676886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211250 0x55be3b21a0a0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.373+0000 7fd875ffb640 0 client.18333 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:59.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.373+0000 7fd897766740 1 -- 172.21.15.2:0/504676886 >> 172.21.15.2:0/504676886 conn(0x55be3b20b980 msgr2=0x55be3b20d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.373+0000 7fd897766740 1 -- 172.21.15.2:0/504676886 shutdown_connections 2024-03-15T13:07:59.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.373+0000 7fd897766740 1 -- 172.21.15.2:0/504676886 wait complete. 2024-03-15T13:07:59.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.373+0000 7fd897766740 20 client.18333 trim_cache size 0 max 16384 2024-03-15T13:07:59.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.385+0000 7fd897766740 1 Processor -- start 2024-03-15T13:07:59.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.385+0000 7fd897766740 1 -- start start 2024-03-15T13:07:59.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd897766740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b211250 0x55be3b219b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd897766740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211b70 0x55be3b21a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd897766740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b21a750 0x55be3b225ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd897766740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3b0fe130 con 0x55be3b211b70 2024-03-15T13:07:59.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd897766740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3b21d110 con 0x55be3b21a750 2024-03-15T13:07:59.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd897766740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3b210640 con 0x55be3b211250 2024-03-15T13:07:59.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd88dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211b70 0x55be3b21a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd88dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211b70 0x55be3b21a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60406/0 (socket says 172.21.15.2:60406) 2024-03-15T13:07:59.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd88dd9b640 1 -- 172.21.15.2:0/3177323712 learned_addr learned my addr 172.21.15.2:0/3177323712 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:59.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd88e59c640 1 --2- 172.21.15.2:0/3177323712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b211250 0x55be3b219b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd88ed9d640 1 --2- 172.21.15.2:0/3177323712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b21a750 0x55be3b225ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd88dd9b640 1 -- 172.21.15.2:0/3177323712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b211250 msgr2=0x55be3b219b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd88dd9b640 1 --2- 172.21.15.2:0/3177323712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b211250 0x55be3b219b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd88dd9b640 1 -- 172.21.15.2:0/3177323712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b21a750 msgr2=0x55be3b225ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd88dd9b640 1 --2- 172.21.15.2:0/3177323712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b21a750 0x55be3b225ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd88dd9b640 1 -- 172.21.15.2:0/3177323712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be3b113c20 con 0x55be3b211b70 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd88e59c640 1 --2- 172.21.15.2:0/3177323712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b211250 0x55be3b219b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd88dd9b640 1 --2- 172.21.15.2:0/3177323712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211b70 0x55be3b21a0a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fd88402ce10 tx=0x7fd8840dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7ca3d761e111ee server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd877fff640 1 -- 172.21.15.2:0/3177323712 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8840bbf90 con 0x55be3b211b70 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd877fff640 1 -- 172.21.15.2:0/3177323712 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd8840bbdb0 con 0x55be3b211b70 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.386+0000 7fd877fff640 1 -- 172.21.15.2:0/3177323712 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8840b8ac0 con 0x55be3b211b70 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.387+0000 7fd897766740 1 -- 172.21.15.2:0/3177323712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211b70 msgr2=0x55be3b21a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.387+0000 7fd897766740 1 --2- 172.21.15.2:0/3177323712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211b70 0x55be3b21a0a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fd88402ce10 tx=0x7fd8840dc040 comp rx=0 tx=0).stop 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.387+0000 7fd897766740 1 -- 172.21.15.2:0/3177323712 shutdown_connections 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.387+0000 7fd897766740 1 --2- 172.21.15.2:0/3177323712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b21a750 0x55be3b225ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.387+0000 7fd897766740 1 --2- 172.21.15.2:0/3177323712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b211b70 0x55be3b21a0a0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.387+0000 7fd897766740 1 --2- 172.21.15.2:0/3177323712 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b211250 0x55be3b219b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.387+0000 7fd897766740 1 -- 172.21.15.2:0/3177323712 >> 172.21.15.2:0/3177323712 conn(0x55be3b20b980 msgr2=0x55be3b20ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.387+0000 7fd897766740 1 -- 172.21.15.2:0/3177323712 shutdown_connections 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.387+0000 7fd897766740 1 -- 172.21.15.2:0/3177323712 wait complete. 2024-03-15T13:07:59.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.387+0000 7fd897766740 1 -- start start 2024-03-15T13:07:59.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.387+0000 7fd897766740 1 Processor -- start 2024-03-15T13:07:59.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd897766740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b211250 0x55be3b21e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd897766740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 0x55be3b21e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd897766740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b21a750 0x55be3b2850c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd897766740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3b285600 con 0x55be3b21a750 2024-03-15T13:07:59.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd897766740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3b285740 con 0x55be3b211b70 2024-03-15T13:07:59.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd897766740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3b285880 con 0x55be3b211250 2024-03-15T13:07:59.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd897766740 10 client.0 _ll_register_callbacks cb 0x7fd884004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:07:59.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd88e59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b211250 0x55be3b21e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd88dd9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 0x55be3b21e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd88ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b21a750 0x55be3b2850c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd88e59c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b211250 0x55be3b21e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:35314/0 (socket says 172.21.15.2:35314) 2024-03-15T13:07:59.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd88dd9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 0x55be3b21e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:58502/0 (socket says 172.21.15.2:58502) 2024-03-15T13:07:59.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd88e59c640 1 -- 172.21.15.2:0/1840099271 learned_addr learned my addr 172.21.15.2:0/1840099271 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:07:59.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd88ed9d640 1 --2- 172.21.15.2:0/1840099271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b21a750 0x55be3b2850c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60414/0 (socket says 172.21.15.2:60414) 2024-03-15T13:07:59.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd88e59c640 1 -- 172.21.15.2:0/1840099271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 msgr2=0x55be3b21e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd88e59c640 1 --2- 172.21.15.2:0/1840099271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 0x55be3b21e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd88e59c640 1 -- 172.21.15.2:0/1840099271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b21a750 msgr2=0x55be3b2850c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:07:59.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd88e59c640 1 --2- 172.21.15.2:0/1840099271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b21a750 0x55be3b2850c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd88e59c640 1 -- 172.21.15.2:0/1840099271 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be3b1776a0 con 0x55be3b211250 2024-03-15T13:07:59.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd88ed9d640 1 --2- 172.21.15.2:0/1840099271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b21a750 0x55be3b2850c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:07:59.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd88e59c640 1 --2- 172.21.15.2:0/1840099271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b211250 0x55be3b21e1c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd8800199e0 tx=0x7fd880005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b9287fa33f244e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:07:59.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd875ffb640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:07:59.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd875ffb640 1 -- 172.21.15.2:0/1840099271 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880009af0 con 0x55be3b211250 2024-03-15T13:07:59.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd897766740 1 -- 172.21.15.2:0/1840099271 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd880009e90 con 0x55be3b211250 2024-03-15T13:07:59.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.388+0000 7fd875ffb640 1 -- 172.21.15.2:0/1840099271 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd880000e40 con 0x55be3b211250 2024-03-15T13:07:59.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.389+0000 7fd875ffb640 1 -- 172.21.15.2:0/1840099271 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd880019dd0 con 0x55be3b211250 2024-03-15T13:07:59.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.389+0000 7fd897766740 10 client.28132 fetch_fsmap learned FSMap version 50 2024-03-15T13:07:59.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.389+0000 7fd897766740 1 -- 172.21.15.2:0/1840099271 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7fd8901984d0 con 0x55be3b211250 2024-03-15T13:07:59.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.389+0000 7fd875ffb640 1 -- 172.21.15.2:0/1840099271 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd880009e90 con 0x55be3b211250 2024-03-15T13:07:59.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.389+0000 7fd875ffb640 1 -- 172.21.15.2:0/1840099271 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd880041070 con 0x55be3b211250 2024-03-15T13:07:59.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.389+0000 7fd897766740 10 client.28132 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:07:59.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.389+0000 7fd897766740 10 client.28132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:07:59.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.389+0000 7fd897766740 20 client.28132 populate_metadata read hostname 'smithi002' 2024-03-15T13:07:59.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.389+0000 7fd897766740 1 --2- 172.21.15.2:0/1840099271 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55be3b286c80 0x55be3b2890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:07:59.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.389+0000 7fd897766740 4 client.28132 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:07:59.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.389+0000 7fd897766740 1 -- 172.21.15.2:0/1840099271 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be3af04890 con 0x55be3b286c80 2024-03-15T13:07:59.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.389+0000 7fd88dd9b640 1 --2- 172.21.15.2:0/1840099271 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55be3b286c80 0x55be3b2890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.389+0000 7fd875ffb640 10 client.28132 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.389+0000 7fd88dd9b640 1 --2- 172.21.15.2:0/1840099271 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55be3b286c80 0x55be3b2890a0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=709965cf0b721694 server_cookie=a996d9ceb7125c56 in_seq=0 out_seq=0 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd875ffb640 1 -- 172.21.15.2:0/1840099271 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be3af04890 con 0x55be3b286c80 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd875ffb640 10 client.28132 handle_command_reply: tid=0 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd897766740 1 client.28132 shutdown 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd897766740 1 -- 172.21.15.2:0/1840099271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b211250 msgr2=0x55be3b21e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd897766740 1 --2- 172.21.15.2:0/1840099271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b211250 0x55be3b21e1c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd8800199e0 tx=0x7fd880005da0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd897766740 1 -- 172.21.15.2:0/1840099271 shutdown_connections 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd897766740 1 --2- 172.21.15.2:0/1840099271 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55be3b286c80 0x55be3b2890a0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd897766740 1 --2- 172.21.15.2:0/1840099271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55be3b21a750 0x55be3b2850c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd875ffb640 0 client.28132 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd897766740 1 --2- 172.21.15.2:0/1840099271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be3b211b70 0x55be3b21e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd897766740 1 --2- 172.21.15.2:0/1840099271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be3b211250 0x55be3b21e1c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd897766740 1 -- 172.21.15.2:0/1840099271 >> 172.21.15.2:0/1840099271 conn(0x55be3b20b980 msgr2=0x55be3b211f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd897766740 1 -- 172.21.15.2:0/1840099271 shutdown_connections 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd897766740 1 -- 172.21.15.2:0/1840099271 wait complete. 2024-03-15T13:07:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd897766740 20 client.28132 trim_cache size 0 max 16384 2024-03-15T13:07:59.381 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:07:59.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd894aa7640 1 -- 172.21.15.2:0/1801840050 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd8780a1ba0 msgr2=0x7fd8780a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd894aa7640 1 --2- 172.21.15.2:0/1801840050 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd8780a1ba0 0x7fd8780a4060 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd888009fd0 tx=0x7fd888013b80 comp rx=0 tx=0).stop 2024-03-15T13:07:59.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd894aa7640 1 -- 172.21.15.2:0/1801840050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd890101e60 msgr2=0x7fd89019dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:07:59.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd894aa7640 1 --2- 172.21.15.2:0/1801840050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd890101e60 0x7fd89019dc60 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fd884009230 tx=0x7fd88402cac0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd894aa7640 1 -- 172.21.15.2:0/1801840050 shutdown_connections 2024-03-15T13:07:59.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd894aa7640 1 --2- 172.21.15.2:0/1801840050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd890107560 0x7fd8901978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd894aa7640 1 --2- 172.21.15.2:0/1801840050 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd8780a1ba0 0x7fd8780a4060 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd894aa7640 1 --2- 172.21.15.2:0/1801840050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd890101e60 0x7fd89019dc60 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd894aa7640 1 --2- 172.21.15.2:0/1801840050 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8901014a0 0x7fd89019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:07:59.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.390+0000 7fd894aa7640 1 -- 172.21.15.2:0/1801840050 >> 172.21.15.2:0/1801840050 conn(0x7fd8900fcf90 msgr2=0x7fd8900fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:07:59.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.391+0000 7fd894aa7640 1 -- 172.21.15.2:0/1801840050 shutdown_connections 2024-03-15T13:07:59.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:07:59.391+0000 7fd894aa7640 1 -- 172.21.15.2:0/1801840050 wait complete. 2024-03-15T13:07:59.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:59 smithi149 ceph-mon[35927]: pgmap v279: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:07:59.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:07:59 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3996213136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:08:00.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:59 smithi002 ceph-mon[32060]: pgmap v279: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:00.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:07:59 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3996213136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:08:00.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:59 smithi184 ceph-mon[35993]: pgmap v279: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:00.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:07:59 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3996213136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:08:00.386 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:08:00.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:08:00.386 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:08:00.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.521+0000 7fef12823640 1 -- 172.21.15.2:0/2145622635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef0c107570 msgr2=0x7fef0c107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.521+0000 7fef12823640 1 --2- 172.21.15.2:0/2145622635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef0c107570 0x7fef0c107950 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7feefc009900 tx=0x7feefc02bc30 comp rx=0 tx=0).stop 2024-03-15T13:08:00.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.521+0000 7fef12823640 1 -- 172.21.15.2:0/2145622635 shutdown_connections 2024-03-15T13:08:00.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.521+0000 7fef12823640 1 --2- 172.21.15.2:0/2145622635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef0c101e70 0x7fef0c10e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.521+0000 7fef12823640 1 --2- 172.21.15.2:0/2145622635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef0c1014b0 0x7fef0c101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.521+0000 7fef12823640 1 --2- 172.21.15.2:0/2145622635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef0c107570 0x7fef0c107950 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.521+0000 7fef12823640 1 -- 172.21.15.2:0/2145622635 >> 172.21.15.2:0/2145622635 conn(0x7fef0c0fcf20 msgr2=0x7fef0c0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:00.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.521+0000 7fef12823640 1 -- 172.21.15.2:0/2145622635 shutdown_connections 2024-03-15T13:08:00.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.521+0000 7fef12823640 1 -- 172.21.15.2:0/2145622635 wait complete. 2024-03-15T13:08:00.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.521+0000 7fef12823640 1 Processor -- start 2024-03-15T13:08:00.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef12823640 1 -- start start 2024-03-15T13:08:00.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef12823640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef0c1014b0 0x7fef0c0737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef12823640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef0c101e70 0x7fef0c073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef12823640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef0c107570 0x7fef0c074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef12823640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c1a5ef0 con 0x7fef0c1014b0 2024-03-15T13:08:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef12823640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c1a6060 con 0x7fef0c101e70 2024-03-15T13:08:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef12823640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c1a6220 con 0x7fef0c107570 2024-03-15T13:08:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef0c1014b0 0x7fef0c0737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef0c1014b0 0x7fef0c0737f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60432/0 (socket says 172.21.15.2:60432) 2024-03-15T13:08:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef0bfff640 1 -- 172.21.15.2:0/1969071851 learned_addr learned my addr 172.21.15.2:0/1969071851 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef0b7fe640 1 --2- 172.21.15.2:0/1969071851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef0c101e70 0x7fef0c073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef10dc0640 1 --2- 172.21.15.2:0/1969071851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef0c107570 0x7fef0c074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef0bfff640 1 -- 172.21.15.2:0/1969071851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef0c107570 msgr2=0x7fef0c074270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef0bfff640 1 --2- 172.21.15.2:0/1969071851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef0c107570 0x7fef0c074270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef0bfff640 1 -- 172.21.15.2:0/1969071851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef0c101e70 msgr2=0x7fef0c073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef0bfff640 1 --2- 172.21.15.2:0/1969071851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef0c101e70 0x7fef0c073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.522+0000 7fef0bfff640 1 -- 172.21.15.2:0/1969071851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc009560 con 0x7fef0c1014b0 2024-03-15T13:08:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.523+0000 7fef0bfff640 1 --2- 172.21.15.2:0/1969071851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef0c1014b0 0x7fef0c0737f0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7feefc009230 tx=0x7feefc02cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.523+0000 7fef097fa640 1 -- 172.21.15.2:0/1969071851 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc004250 con 0x7fef0c1014b0 2024-03-15T13:08:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.523+0000 7fef097fa640 1 -- 172.21.15.2:0/1969071851 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7feefc0043b0 con 0x7fef0c1014b0 2024-03-15T13:08:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.523+0000 7fef12823640 1 -- 172.21.15.2:0/1969071851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0c1a64a0 con 0x7fef0c1014b0 2024-03-15T13:08:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.523+0000 7fef097fa640 1 -- 172.21.15.2:0/1969071851 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc003bf0 con 0x7fef0c1014b0 2024-03-15T13:08:00.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.523+0000 7fef12823640 1 -- 172.21.15.2:0/1969071851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0c1a69b0 con 0x7fef0c1014b0 2024-03-15T13:08:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.525+0000 7fef097fa640 1 -- 172.21.15.2:0/1969071851 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7feefc003d50 con 0x7fef0c1014b0 2024-03-15T13:08:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.525+0000 7fef15404740 1 Processor -- start 2024-03-15T13:08:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.525+0000 7fef15404740 1 -- start start 2024-03-15T13:08:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.525+0000 7fef097fa640 1 --2- 172.21.15.2:0/1969071851 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feef40a1ba0 0x7feef40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.525+0000 7fef15404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 0x564f4d41c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.525+0000 7fef15404740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41cd30 0x564f4d4252e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.525+0000 7fef15404740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d4258b0 0x564f4d427d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.525+0000 7fef15404740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4d38af90 con 0x564f4d41cd30 2024-03-15T13:08:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.525+0000 7fef15404740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4d3884c0 con 0x564f4d41c380 2024-03-15T13:08:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.525+0000 7fef15404740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4d383ca0 con 0x564f4d4258b0 2024-03-15T13:08:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7fef0b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41cd30 0x564f4d4252e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 0x564f4d41c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7fef0b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41cd30 0x564f4d4252e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60442/0 (socket says 172.21.15.2:60442) 2024-03-15T13:08:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7fef0b7fe640 1 -- 172.21.15.2:0/3513649613 learned_addr learned my addr 172.21.15.2:0/3513649613 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7fef10dc0640 1 --2- 172.21.15.2:0/1969071851 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feef40a1ba0 0x7feef40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7fef097fa640 1 -- 172.21.15.2:0/1969071851 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7feefc0efb10 con 0x7fef0c1014b0 2024-03-15T13:08:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7fef10dc0640 1 --2- 172.21.15.2:0/3513649613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d4258b0 0x564f4d427d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7fef0b7fe640 1 -- 172.21.15.2:0/3513649613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d4258b0 msgr2=0x564f4d427d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7fef0b7fe640 1 --2- 172.21.15.2:0/3513649613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d4258b0 0x564f4d427d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7fef0b7fe640 1 -- 172.21.15.2:0/3513649613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 msgr2=0x564f4d41c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7fef0b7fe640 1 --2- 172.21.15.2:0/3513649613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 0x564f4d41c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7fef0b7fe640 1 -- 172.21.15.2:0/3513649613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f4d3460b0 con 0x564f4d41cd30 2024-03-15T13:08:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7fef0bfff640 1 --2- 172.21.15.2:0/3513649613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 0x564f4d41c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7fef10dc0640 1 --2- 172.21.15.2:0/1969071851 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feef40a1ba0 0x7feef40a4060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef04005fd0 tx=0x7fef04009b70 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7fef0b7fe640 1 --2- 172.21.15.2:0/3513649613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41cd30 0x564f4d4252e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fef00009ca0 tx=0x7fef00011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4610af064a63f833 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7feef27fc640 1 -- 172.21.15.2:0/3513649613 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00019630 con 0x564f4d41cd30 2024-03-15T13:08:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7feef27fc640 1 -- 172.21.15.2:0/3513649613 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fef00009e50 con 0x564f4d41cd30 2024-03-15T13:08:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.526+0000 7feef27fc640 1 -- 172.21.15.2:0/3513649613 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0000ea30 con 0x564f4d41cd30 2024-03-15T13:08:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.527+0000 7fef15404740 1 -- 172.21.15.2:0/3513649613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41cd30 msgr2=0x564f4d4252e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.527+0000 7fef15404740 1 --2- 172.21.15.2:0/3513649613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41cd30 0x564f4d4252e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fef00009ca0 tx=0x7fef00011810 comp rx=0 tx=0).stop 2024-03-15T13:08:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.527+0000 7fef15404740 1 -- 172.21.15.2:0/3513649613 shutdown_connections 2024-03-15T13:08:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.527+0000 7fef15404740 1 --2- 172.21.15.2:0/3513649613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d4258b0 0x564f4d427d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.527+0000 7fef15404740 1 --2- 172.21.15.2:0/3513649613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41cd30 0x564f4d4252e0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.527+0000 7fef15404740 1 --2- 172.21.15.2:0/3513649613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 0x564f4d41c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.527+0000 7fef15404740 1 -- 172.21.15.2:0/3513649613 >> 172.21.15.2:0/3513649613 conn(0x564f4d416840 msgr2=0x564f4d416c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:00.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.527+0000 7fef15404740 1 -- 172.21.15.2:0/3513649613 shutdown_connections 2024-03-15T13:08:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.527+0000 7fef15404740 1 -- 172.21.15.2:0/3513649613 wait complete. 2024-03-15T13:08:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.527+0000 7fef15404740 1 -- start start 2024-03-15T13:08:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.527+0000 7fef15404740 1 Processor -- start 2024-03-15T13:08:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef15404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 0x564f4d422810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef15404740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41cd30 0x564f4d422d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef15404740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d4258b0 0x564f4d423290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef15404740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4d328f40 con 0x564f4d41cd30 2024-03-15T13:08:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef15404740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4d0574b0 con 0x564f4d41c380 2024-03-15T13:08:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef15404740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4d309670 con 0x564f4d4258b0 2024-03-15T13:08:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef15404740 10 client.0 _ll_register_callbacks cb 0x564f4d376f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef0b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41cd30 0x564f4d422d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 0x564f4d422810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef0b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41cd30 0x564f4d422d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60454/0 (socket says 172.21.15.2:60454) 2024-03-15T13:08:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef0b7fe640 1 -- 172.21.15.2:0/2048797745 learned_addr learned my addr 172.21.15.2:0/2048797745 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef10dc0640 1 --2- 172.21.15.2:0/2048797745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d4258b0 0x564f4d423290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef0b7fe640 1 -- 172.21.15.2:0/2048797745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d4258b0 msgr2=0x564f4d423290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef0b7fe640 1 --2- 172.21.15.2:0/2048797745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d4258b0 0x564f4d423290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef0b7fe640 1 -- 172.21.15.2:0/2048797745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 msgr2=0x564f4d422810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef0b7fe640 1 --2- 172.21.15.2:0/2048797745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 0x564f4d422810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef0b7fe640 1 -- 172.21.15.2:0/2048797745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef000098d0 con 0x564f4d41cd30 2024-03-15T13:08:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef0bfff640 1 --2- 172.21.15.2:0/2048797745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 0x564f4d422810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7fef0b7fe640 1 --2- 172.21.15.2:0/2048797745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41cd30 0x564f4d422d50 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fef0001c040 tx=0x7fef00005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9cf5128cc80f4b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.528+0000 7feeeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.529+0000 7feeeb7fe640 1 -- 172.21.15.2:0/2048797745 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00004220 con 0x564f4d41cd30 2024-03-15T13:08:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.529+0000 7feeeb7fe640 1 -- 172.21.15.2:0/2048797745 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fef00004380 con 0x564f4d41cd30 2024-03-15T13:08:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.529+0000 7fef15404740 1 -- 172.21.15.2:0/2048797745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f4d35df40 con 0x564f4d41cd30 2024-03-15T13:08:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.529+0000 7feeeb7fe640 1 -- 172.21.15.2:0/2048797745 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef0000e7b0 con 0x564f4d41cd30 2024-03-15T13:08:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.529+0000 7fef15404740 10 client.18351 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.529+0000 7fef15404740 1 -- 172.21.15.2:0/2048797745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x564f4d382320 con 0x564f4d41cd30 2024-03-15T13:08:00.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.529+0000 7feeeb7fe640 1 -- 172.21.15.2:0/2048797745 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x564f4d35df40 con 0x564f4d41cd30 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.529+0000 7feeeb7fe640 1 -- 172.21.15.2:0/2048797745 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fef0002b020 con 0x564f4d41cd30 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.529+0000 7fef15404740 10 client.18351 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.529+0000 7fef15404740 10 client.18351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.529+0000 7fef15404740 20 client.18351 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.529+0000 7fef15404740 1 --2- 172.21.15.2:0/2048797745 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564f4d435db0 0x564f4d4381f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.529+0000 7fef15404740 4 client.18351 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.529+0000 7fef15404740 1 -- 172.21.15.2:0/2048797745 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f4d37a310 con 0x564f4d435db0 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.529+0000 7fef0bfff640 1 --2- 172.21.15.2:0/2048797745 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564f4d435db0 0x564f4d4381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.530+0000 7fef0bfff640 1 --2- 172.21.15.2:0/2048797745 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564f4d435db0 0x564f4d4381f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45a8a477eeea40bd server_cookie=c0d60f74c7cc6b77 in_seq=0 out_seq=0 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.530+0000 7feeeb7fe640 10 client.18351 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.531+0000 7feeeb7fe640 1 -- 172.21.15.2:0/2048797745 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564f4d37a310 con 0x564f4d435db0 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.531+0000 7feeeb7fe640 10 client.18351 handle_command_reply: tid=0 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.531+0000 7fef15404740 1 client.18351 shutdown 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.531+0000 7fef15404740 1 -- 172.21.15.2:0/2048797745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41cd30 msgr2=0x564f4d422d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.531+0000 7fef15404740 1 --2- 172.21.15.2:0/2048797745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41cd30 0x564f4d422d50 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fef0001c040 tx=0x7fef00005720 comp rx=0 tx=0).stop 2024-03-15T13:08:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.531+0000 7fef15404740 1 -- 172.21.15.2:0/2048797745 shutdown_connections 2024-03-15T13:08:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.531+0000 7fef15404740 1 --2- 172.21.15.2:0/2048797745 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564f4d435db0 0x564f4d4381f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.531+0000 7fef15404740 1 --2- 172.21.15.2:0/2048797745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d4258b0 0x564f4d423290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.531+0000 7feeeb7fe640 0 client.18351 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.531+0000 7fef15404740 1 --2- 172.21.15.2:0/2048797745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41cd30 0x564f4d422d50 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.531+0000 7fef15404740 1 --2- 172.21.15.2:0/2048797745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 0x564f4d422810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.531+0000 7fef15404740 1 -- 172.21.15.2:0/2048797745 >> 172.21.15.2:0/2048797745 conn(0x564f4d416840 msgr2=0x564f4d4187a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.531+0000 7fef15404740 1 -- 172.21.15.2:0/2048797745 shutdown_connections 2024-03-15T13:08:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.531+0000 7fef15404740 1 -- 172.21.15.2:0/2048797745 wait complete. 2024-03-15T13:08:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.531+0000 7fef15404740 20 client.18351 trim_cache size 0 max 16384 2024-03-15T13:08:00.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.547+0000 7fef15404740 1 Processor -- start 2024-03-15T13:08:00.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.547+0000 7fef15404740 1 -- start start 2024-03-15T13:08:00.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.547+0000 7fef15404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 0x564f4d424f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.547+0000 7fef15404740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d41cd30 0x564f4d4230b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.547+0000 7fef15404740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d4258b0 0x564f4d430de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.547+0000 7fef15404740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4d41d670 con 0x564f4d4258b0 2024-03-15T13:08:00.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.547+0000 7fef15404740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4d417410 con 0x564f4d41c380 2024-03-15T13:08:00.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.547+0000 7fef15404740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4d4175d0 con 0x564f4d41cd30 2024-03-15T13:08:00.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.548+0000 7fef10dc0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d4258b0 0x564f4d430de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.548+0000 7fef10dc0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d4258b0 0x564f4d430de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60470/0 (socket says 172.21.15.2:60470) 2024-03-15T13:08:00.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.548+0000 7fef10dc0640 1 -- 172.21.15.2:0/1432649278 learned_addr learned my addr 172.21.15.2:0/1432649278 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:00.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.548+0000 7fef0bfff640 1 --2- 172.21.15.2:0/1432649278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 0x564f4d424f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.548+0000 7fef0b7fe640 1 --2- 172.21.15.2:0/1432649278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d41cd30 0x564f4d4230b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.548+0000 7fef10dc0640 1 -- 172.21.15.2:0/1432649278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d41cd30 msgr2=0x564f4d4230b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.548+0000 7fef10dc0640 1 --2- 172.21.15.2:0/1432649278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d41cd30 0x564f4d4230b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.548+0000 7fef10dc0640 1 -- 172.21.15.2:0/1432649278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 msgr2=0x564f4d424f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.548+0000 7fef10dc0640 1 --2- 172.21.15.2:0/1432649278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 0x564f4d424f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.548+0000 7fef10dc0640 1 -- 172.21.15.2:0/1432649278 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f4d31ec00 con 0x564f4d4258b0 2024-03-15T13:08:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.548+0000 7fef10dc0640 1 --2- 172.21.15.2:0/1432649278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d4258b0 0x564f4d430de0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fef0400feb0 tx=0x7fef04009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a318a8eac39f954 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.548+0000 7feef1ffb640 1 -- 172.21.15.2:0/1432649278 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04003e00 con 0x564f4d4258b0 2024-03-15T13:08:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.548+0000 7feef1ffb640 1 -- 172.21.15.2:0/1432649278 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fef04004550 con 0x564f4d4258b0 2024-03-15T13:08:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.548+0000 7feef1ffb640 1 -- 172.21.15.2:0/1432649278 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04021690 con 0x564f4d4258b0 2024-03-15T13:08:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.549+0000 7fef15404740 1 -- 172.21.15.2:0/1432649278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d4258b0 msgr2=0x564f4d430de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.549+0000 7fef15404740 1 --2- 172.21.15.2:0/1432649278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d4258b0 0x564f4d430de0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fef0400feb0 tx=0x7fef04009500 comp rx=0 tx=0).stop 2024-03-15T13:08:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.549+0000 7fef15404740 1 -- 172.21.15.2:0/1432649278 shutdown_connections 2024-03-15T13:08:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.549+0000 7fef15404740 1 --2- 172.21.15.2:0/1432649278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d4258b0 0x564f4d430de0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.549+0000 7fef15404740 1 --2- 172.21.15.2:0/1432649278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d41cd30 0x564f4d4230b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.549+0000 7fef15404740 1 --2- 172.21.15.2:0/1432649278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d41c380 0x564f4d424f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.549+0000 7fef15404740 1 -- 172.21.15.2:0/1432649278 >> 172.21.15.2:0/1432649278 conn(0x564f4d416840 msgr2=0x564f4d4158e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.549+0000 7fef15404740 1 -- 172.21.15.2:0/1432649278 shutdown_connections 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.549+0000 7fef15404740 1 -- 172.21.15.2:0/1432649278 wait complete. 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.549+0000 7fef15404740 1 -- start start 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.549+0000 7fef15404740 1 Processor -- start 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.549+0000 7fef15404740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41c380 0x564f4d42d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef15404740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d41cd30 0x564f4d41f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef15404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d4258b0 0x564f4d41ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef15404740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4d420440 con 0x564f4d41c380 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef15404740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4d4205b0 con 0x564f4d4258b0 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef15404740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4d420770 con 0x564f4d41cd30 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef15404740 10 client.0 _ll_register_callbacks cb 0x7fef0001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41c380 0x564f4d42d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41c380 0x564f4d42d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60484/0 (socket says 172.21.15.2:60484) 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef0bfff640 1 -- 172.21.15.2:0/2161301582 learned_addr learned my addr 172.21.15.2:0/2161301582 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef10dc0640 1 --2- 172.21.15.2:0/2161301582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d4258b0 0x564f4d41ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef0b7fe640 1 --2- 172.21.15.2:0/2161301582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d41cd30 0x564f4d41f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef0bfff640 1 -- 172.21.15.2:0/2161301582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d41cd30 msgr2=0x564f4d41f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef0bfff640 1 --2- 172.21.15.2:0/2161301582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d41cd30 0x564f4d41f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef0bfff640 1 -- 172.21.15.2:0/2161301582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d4258b0 msgr2=0x564f4d41ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef0bfff640 1 --2- 172.21.15.2:0/2161301582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d4258b0 0x564f4d41ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef0bfff640 1 -- 172.21.15.2:0/2161301582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef04016030 con 0x564f4d41c380 2024-03-15T13:08:00.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef0bfff640 1 --2- 172.21.15.2:0/2161301582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41c380 0x564f4d42d4b0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7feefc000c00 tx=0x7feefc034f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fcf36781afbd61 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:00.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7feeeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:00.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7feeeb7fe640 1 -- 172.21.15.2:0/2161301582 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc0c4ea0 con 0x564f4d41c380 2024-03-15T13:08:00.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7feeeb7fe640 1 -- 172.21.15.2:0/2161301582 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7feefc0bcbb0 con 0x564f4d41c380 2024-03-15T13:08:00.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef15404740 1 -- 172.21.15.2:0/2161301582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f4d3a9900 con 0x564f4d41c380 2024-03-15T13:08:00.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7feeeb7fe640 1 -- 172.21.15.2:0/2161301582 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7feefc0a70b0 con 0x564f4d41c380 2024-03-15T13:08:00.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef15404740 10 client.18357 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:00.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.550+0000 7fef15404740 1 -- 172.21.15.2:0/2161301582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x564f4d344910 con 0x564f4d41c380 2024-03-15T13:08:00.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.551+0000 7feeeb7fe640 1 -- 172.21.15.2:0/2161301582 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x564f4d3a9900 con 0x564f4d41c380 2024-03-15T13:08:00.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.551+0000 7feeeb7fe640 1 -- 172.21.15.2:0/2161301582 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7feefc03c070 con 0x564f4d41c380 2024-03-15T13:08:00.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.551+0000 7fef15404740 10 client.18357 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:00.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.551+0000 7fef15404740 10 client.18357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:08:00.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.551+0000 7fef15404740 20 client.18357 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:00.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.551+0000 7fef15404740 1 --2- 172.21.15.2:0/2161301582 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564f4d492a00 0x564f4d494df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.551+0000 7fef15404740 4 client.18357 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:08:00.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.551+0000 7fef15404740 1 -- 172.21.15.2:0/2161301582 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564f4d083860 con 0x564f4d492a00 2024-03-15T13:08:00.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.551+0000 7fef0b7fe640 1 --2- 172.21.15.2:0/2161301582 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564f4d492a00 0x564f4d494df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.551+0000 7fef0b7fe640 1 --2- 172.21.15.2:0/2161301582 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564f4d492a00 0x564f4d494df0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8213ef3bcd30eaab server_cookie=f968820312c0efd5 in_seq=0 out_seq=0 2024-03-15T13:08:00.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.551+0000 7feeeb7fe640 10 client.18357 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:00.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.552+0000 7feeeb7fe640 1 -- 172.21.15.2:0/2161301582 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564f4d083860 con 0x564f4d492a00 2024-03-15T13:08:00.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.552+0000 7feeeb7fe640 10 client.18357 handle_command_reply: tid=0 2024-03-15T13:08:00.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.552+0000 7fef15404740 1 client.18357 shutdown 2024-03-15T13:08:00.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7fef15404740 1 -- 172.21.15.2:0/2161301582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41c380 msgr2=0x564f4d42d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7fef15404740 1 --2- 172.21.15.2:0/2161301582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41c380 0x564f4d42d4b0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7feefc000c00 tx=0x7feefc034f90 comp rx=0 tx=0).stop 2024-03-15T13:08:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7fef15404740 1 -- 172.21.15.2:0/2161301582 shutdown_connections 2024-03-15T13:08:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7fef15404740 1 --2- 172.21.15.2:0/2161301582 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564f4d492a00 0x564f4d494df0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7fef15404740 1 --2- 172.21.15.2:0/2161301582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f4d4258b0 0x564f4d41ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7fef15404740 1 --2- 172.21.15.2:0/2161301582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f4d41cd30 0x564f4d41f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7fef15404740 1 --2- 172.21.15.2:0/2161301582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f4d41c380 0x564f4d42d4b0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7fef15404740 1 -- 172.21.15.2:0/2161301582 >> 172.21.15.2:0/2161301582 conn(0x564f4d416840 msgr2=0x564f4d43cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7feeeb7fe640 0 client.18357 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7fef15404740 1 -- 172.21.15.2:0/2161301582 shutdown_connections 2024-03-15T13:08:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7fef15404740 1 -- 172.21.15.2:0/2161301582 wait complete. 2024-03-15T13:08:00.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7fef15404740 20 client.18357 trim_cache size 0 max 16384 2024-03-15T13:08:00.543 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:08:00.543 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:08:00.543 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "9c62f729-e7c5-441d-801e-9fc75f9d3938", 2024-03-15T13:08:00.544 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:08:00.544 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:08:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7fef12823640 1 -- 172.21.15.2:0/1969071851 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feef40a1ba0 msgr2=0x7feef40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7fef12823640 1 --2- 172.21.15.2:0/1969071851 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feef40a1ba0 0x7feef40a4060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef04005fd0 tx=0x7fef04009b70 comp rx=0 tx=0).stop 2024-03-15T13:08:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7fef12823640 1 -- 172.21.15.2:0/1969071851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef0c1014b0 msgr2=0x7fef0c0737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.553+0000 7fef12823640 1 --2- 172.21.15.2:0/1969071851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef0c1014b0 0x7fef0c0737f0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7feefc009230 tx=0x7feefc02cac0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.554+0000 7fef12823640 1 -- 172.21.15.2:0/1969071851 shutdown_connections 2024-03-15T13:08:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.554+0000 7fef12823640 1 --2- 172.21.15.2:0/1969071851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef0c107570 0x7fef0c074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.554+0000 7fef12823640 1 --2- 172.21.15.2:0/1969071851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef0c101e70 0x7fef0c073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.554+0000 7fef12823640 1 --2- 172.21.15.2:0/1969071851 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7feef40a1ba0 0x7feef40a4060 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.554+0000 7fef12823640 1 --2- 172.21.15.2:0/1969071851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef0c1014b0 0x7fef0c0737f0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.554+0000 7fef12823640 1 -- 172.21.15.2:0/1969071851 >> 172.21.15.2:0/1969071851 conn(0x7fef0c0fcf20 msgr2=0x7fef0c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.554+0000 7fef12823640 1 -- 172.21.15.2:0/1969071851 shutdown_connections 2024-03-15T13:08:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.554+0000 7fef12823640 1 -- 172.21.15.2:0/1969071851 wait complete. 2024-03-15T13:08:00.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:08:00.549 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:08:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.668+0000 7fe80b8a7640 1 -- 172.21.15.2:0/142576142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe804102710 msgr2=0x7fe804102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.668+0000 7fe80b8a7640 1 --2- 172.21.15.2:0/142576142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe804102710 0x7fe804102af0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc009900 tx=0x7fe7fc02bcd0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.668+0000 7fe80b8a7640 1 -- 172.21.15.2:0/142576142 shutdown_connections 2024-03-15T13:08:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.668+0000 7fe80b8a7640 1 --2- 172.21.15.2:0/142576142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe80410ba50 0x7fe80410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.668+0000 7fe80b8a7640 1 --2- 172.21.15.2:0/142576142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe804103030 0x7fe80410b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.668+0000 7fe80b8a7640 1 --2- 172.21.15.2:0/142576142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe804102710 0x7fe804102af0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.668+0000 7fe80b8a7640 1 -- 172.21.15.2:0/142576142 >> 172.21.15.2:0/142576142 conn(0x7fe8040775a0 msgr2=0x7fe8040779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.669+0000 7fe80b8a7640 1 -- 172.21.15.2:0/142576142 shutdown_connections 2024-03-15T13:08:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.669+0000 7fe80b8a7640 1 -- 172.21.15.2:0/142576142 wait complete. 2024-03-15T13:08:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.669+0000 7fe80b8a7640 1 Processor -- start 2024-03-15T13:08:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.669+0000 7fe80b8a7640 1 -- start start 2024-03-15T13:08:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.669+0000 7fe80b8a7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe804102710 0x7fe80419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.669+0000 7fe80b8a7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe804103030 0x7fe80419dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.669+0000 7fe80b8a7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe80410ba50 0x7fe8041978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.669+0000 7fe80b8a7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe804197f30 con 0x7fe804103030 2024-03-15T13:08:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.669+0000 7fe80b8a7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8041980a0 con 0x7fe804102710 2024-03-15T13:08:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.669+0000 7fe80b8a7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe804198260 con 0x7fe80410ba50 2024-03-15T13:08:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.669+0000 7fe808e42640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe804103030 0x7fe80419dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.669+0000 7fe808e42640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe804103030 0x7fe80419dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60504/0 (socket says 172.21.15.2:60504) 2024-03-15T13:08:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.669+0000 7fe808e42640 1 -- 172.21.15.2:0/2938296798 learned_addr learned my addr 172.21.15.2:0/2938296798 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.669+0000 7fe809643640 1 --2- 172.21.15.2:0/2938296798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe804102710 0x7fe80419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.670+0000 7fe809e44640 1 --2- 172.21.15.2:0/2938296798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe80410ba50 0x7fe8041978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.670+0000 7fe808e42640 1 -- 172.21.15.2:0/2938296798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe80410ba50 msgr2=0x7fe8041978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.670+0000 7fe808e42640 1 --2- 172.21.15.2:0/2938296798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe80410ba50 0x7fe8041978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.670+0000 7fe808e42640 1 -- 172.21.15.2:0/2938296798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe804102710 msgr2=0x7fe80419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.670+0000 7fe808e42640 1 --2- 172.21.15.2:0/2938296798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe804102710 0x7fe80419d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.670+0000 7fe808e42640 1 -- 172.21.15.2:0/2938296798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc009560 con 0x7fe804103030 2024-03-15T13:08:00.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.670+0000 7fe809643640 1 --2- 172.21.15.2:0/2938296798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe804102710 0x7fe80419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:00.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.670+0000 7fe808e42640 1 --2- 172.21.15.2:0/2938296798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe804103030 0x7fe80419dc70 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400e9f0 tx=0x7fe7f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:00.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.670+0000 7fe7fa7fc640 1 -- 172.21.15.2:0/2938296798 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4009720 con 0x7fe804103030 2024-03-15T13:08:00.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.670+0000 7fe80b8a7640 1 -- 172.21.15.2:0/2938296798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe804198540 con 0x7fe804103030 2024-03-15T13:08:00.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.670+0000 7fe7fa7fc640 1 -- 172.21.15.2:0/2938296798 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe7f400cd80 con 0x7fe804103030 2024-03-15T13:08:00.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.670+0000 7fe7fa7fc640 1 -- 172.21.15.2:0/2938296798 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4010690 con 0x7fe804103030 2024-03-15T13:08:00.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.670+0000 7fe80b8a7640 1 -- 172.21.15.2:0/2938296798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8041a47d0 con 0x7fe804103030 2024-03-15T13:08:00.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.673+0000 7fe80e567740 1 Processor -- start 2024-03-15T13:08:00.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.673+0000 7fe80e567740 1 -- start start 2024-03-15T13:08:00.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.673+0000 7fe7fa7fc640 1 -- 172.21.15.2:0/2938296798 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fe7f4004e90 con 0x7fe804103030 2024-03-15T13:08:00.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.673+0000 7fe80e567740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fa2b0 0x5561f42fa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.673+0000 7fe80e567740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fabd0 0x5561f4303270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.673+0000 7fe80e567740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f43037b0 0x5561f4305ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.673+0000 7fe80e567740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f4269280 con 0x5561f42fa2b0 2024-03-15T13:08:00.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.673+0000 7fe80e567740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f42667b0 con 0x5561f43037b0 2024-03-15T13:08:00.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.673+0000 7fe80e567740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f4261f90 con 0x5561f42fabd0 2024-03-15T13:08:00.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe809e44640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fa2b0 0x5561f42fa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe809e44640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fa2b0 0x5561f42fa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60518/0 (socket says 172.21.15.2:60518) 2024-03-15T13:08:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe809e44640 1 -- 172.21.15.2:0/3726136287 learned_addr learned my addr 172.21.15.2:0/3726136287 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe809643640 1 --2- 172.21.15.2:0/3726136287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fabd0 0x5561f4303270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe808e42640 1 --2- 172.21.15.2:0/3726136287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f43037b0 0x5561f4305ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe809e44640 1 -- 172.21.15.2:0/3726136287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fabd0 msgr2=0x5561f4303270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe809e44640 1 --2- 172.21.15.2:0/3726136287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fabd0 0x5561f4303270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe809e44640 1 -- 172.21.15.2:0/3726136287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f43037b0 msgr2=0x5561f4305ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe809e44640 1 --2- 172.21.15.2:0/3726136287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f43037b0 0x5561f4305ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe809e44640 1 -- 172.21.15.2:0/3726136287 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561f4220570 con 0x5561f42fa2b0 2024-03-15T13:08:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe7fa7fc640 1 --2- 172.21.15.2:0/2938296798 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe7f00a1ba0 0x7fe7f00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe809e44640 1 --2- 172.21.15.2:0/3726136287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fa2b0 0x5561f42fa690 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe800009f50 tx=0x7fe8000118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7221fad461567a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe809643640 1 --2- 172.21.15.2:0/2938296798 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe7f00a1ba0 0x7fe7f00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe7fa7fc640 1 -- 172.21.15.2:0/2938296798 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe7f4014070 con 0x7fe804103030 2024-03-15T13:08:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe7ef7fe640 1 -- 172.21.15.2:0/3726136287 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800004d20 con 0x5561f42fa2b0 2024-03-15T13:08:00.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe7ef7fe640 1 -- 172.21.15.2:0/3726136287 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe800004e80 con 0x5561f42fa2b0 2024-03-15T13:08:00.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.674+0000 7fe7ef7fe640 1 -- 172.21.15.2:0/3726136287 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800005280 con 0x5561f42fa2b0 2024-03-15T13:08:00.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.675+0000 7fe80e567740 1 -- 172.21.15.2:0/3726136287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fa2b0 msgr2=0x5561f42fa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.675+0000 7fe809643640 1 --2- 172.21.15.2:0/2938296798 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe7f00a1ba0 0x7fe7f00a4060 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc005ab0 tx=0x7fe7fc005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:00.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.675+0000 7fe80e567740 1 --2- 172.21.15.2:0/3726136287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fa2b0 0x5561f42fa690 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe800009f50 tx=0x7fe8000118d0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.675+0000 7fe80e567740 1 -- 172.21.15.2:0/3726136287 shutdown_connections 2024-03-15T13:08:00.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.675+0000 7fe80e567740 1 --2- 172.21.15.2:0/3726136287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f43037b0 0x5561f4305ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.675+0000 7fe80e567740 1 --2- 172.21.15.2:0/3726136287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fabd0 0x5561f4303270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.675+0000 7fe80e567740 1 --2- 172.21.15.2:0/3726136287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fa2b0 0x5561f42fa690 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.675+0000 7fe80e567740 1 -- 172.21.15.2:0/3726136287 >> 172.21.15.2:0/3726136287 conn(0x5561f42f4990 msgr2=0x5561f42f4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:00.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.675+0000 7fe80e567740 1 -- 172.21.15.2:0/3726136287 shutdown_connections 2024-03-15T13:08:00.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.675+0000 7fe80e567740 1 -- 172.21.15.2:0/3726136287 wait complete. 2024-03-15T13:08:00.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.675+0000 7fe80e567740 1 -- start start 2024-03-15T13:08:00.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe80e567740 1 Processor -- start 2024-03-15T13:08:00.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe80e567740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fa2b0 0x5561f4303270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe80e567740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f42fabd0 0x5561f4300600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe80e567740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f43037b0 0x5561f4300d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe80e567740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f3f35430 con 0x5561f43037b0 2024-03-15T13:08:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe80e567740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f3ef1e00 con 0x5561f42fabd0 2024-03-15T13:08:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe80e567740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f41e6fd0 con 0x5561f42fa2b0 2024-03-15T13:08:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe80e567740 10 client.0 _ll_register_callbacks cb 0x5561f423cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe809e44640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f43037b0 0x5561f4300d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe809e44640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f43037b0 0x5561f4300d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60526/0 (socket says 172.21.15.2:60526) 2024-03-15T13:08:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe809e44640 1 -- 172.21.15.2:0/1270353929 learned_addr learned my addr 172.21.15.2:0/1270353929 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe809643640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fa2b0 0x5561f4303270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe808e42640 1 --2- 172.21.15.2:0/1270353929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f42fabd0 0x5561f4300600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe809e44640 1 -- 172.21.15.2:0/1270353929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fa2b0 msgr2=0x5561f4303270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe809e44640 1 --2- 172.21.15.2:0/1270353929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fa2b0 0x5561f4303270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe809e44640 1 -- 172.21.15.2:0/1270353929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f42fabd0 msgr2=0x5561f4300600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe809e44640 1 --2- 172.21.15.2:0/1270353929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f42fabd0 0x5561f4300600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.676+0000 7fe809e44640 1 -- 172.21.15.2:0/1270353929 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe800009c00 con 0x5561f43037b0 2024-03-15T13:08:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe809e44640 1 --2- 172.21.15.2:0/1270353929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f43037b0 0x5561f4300d40 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe800009f20 tx=0x7fe800004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504dd1f008edd858 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe7ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe7ecff9640 1 -- 172.21.15.2:0/1270353929 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800003e40 con 0x5561f43037b0 2024-03-15T13:08:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe7ecff9640 1 -- 172.21.15.2:0/1270353929 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe800005020 con 0x5561f43037b0 2024-03-15T13:08:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe80e567740 1 -- 172.21.15.2:0/1270353929 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561f3f9e720 con 0x5561f43037b0 2024-03-15T13:08:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe7ecff9640 1 -- 172.21.15.2:0/1270353929 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8000051e0 con 0x5561f43037b0 2024-03-15T13:08:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe80e567740 10 client.18369 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe80e567740 1 -- 172.21.15.2:0/1270353929 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5561f4260610 con 0x5561f43037b0 2024-03-15T13:08:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe7ecff9640 1 -- 172.21.15.2:0/1270353929 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5561f3f9e720 con 0x5561f43037b0 2024-03-15T13:08:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe7ecff9640 1 -- 172.21.15.2:0/1270353929 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe80000f8d0 con 0x5561f43037b0 2024-03-15T13:08:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe80e567740 10 client.18369 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe80e567740 10 client.18369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:08:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe80e567740 20 client.18369 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:00.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe80e567740 1 --2- 172.21.15.2:0/1270353929 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5561f4313950 0x5561f4315d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe80e567740 4 client.18369 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:08:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe80e567740 1 -- 172.21.15.2:0/1270353929 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561f3fed890 con 0x5561f4313950 2024-03-15T13:08:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.677+0000 7fe809643640 1 --2- 172.21.15.2:0/1270353929 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5561f4313950 0x5561f4315d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.678+0000 7fe809643640 1 --2- 172.21.15.2:0/1270353929 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5561f4313950 0x5561f4315d70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcba26a4cf986979 server_cookie=417bb4ef3b59e3da in_seq=0 out_seq=0 2024-03-15T13:08:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.678+0000 7fe7ecff9640 10 client.18369 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.678+0000 7fe7ecff9640 1 -- 172.21.15.2:0/1270353929 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5561f3fed890 con 0x5561f4313950 2024-03-15T13:08:00.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.678+0000 7fe7ecff9640 10 client.18369 handle_command_reply: tid=0 2024-03-15T13:08:00.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.678+0000 7fe80e567740 1 client.18369 shutdown 2024-03-15T13:08:00.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.679+0000 7fe80e567740 1 -- 172.21.15.2:0/1270353929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f43037b0 msgr2=0x5561f4300d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.679+0000 7fe80e567740 1 --2- 172.21.15.2:0/1270353929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f43037b0 0x5561f4300d40 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe800009f20 tx=0x7fe800004cf0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.679+0000 7fe80e567740 1 -- 172.21.15.2:0/1270353929 shutdown_connections 2024-03-15T13:08:00.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.679+0000 7fe80e567740 1 --2- 172.21.15.2:0/1270353929 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5561f4313950 0x5561f4315d70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.679+0000 7fe80e567740 1 --2- 172.21.15.2:0/1270353929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f43037b0 0x5561f4300d40 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.679+0000 7fe80e567740 1 --2- 172.21.15.2:0/1270353929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f42fabd0 0x5561f4300600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.679+0000 7fe7ecff9640 0 client.18369 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:00.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.679+0000 7fe80e567740 1 --2- 172.21.15.2:0/1270353929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fa2b0 0x5561f4303270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.679+0000 7fe80e567740 1 -- 172.21.15.2:0/1270353929 >> 172.21.15.2:0/1270353929 conn(0x5561f42f4990 msgr2=0x5561f42f6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:00.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.679+0000 7fe80e567740 1 -- 172.21.15.2:0/1270353929 shutdown_connections 2024-03-15T13:08:00.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.679+0000 7fe80e567740 1 -- 172.21.15.2:0/1270353929 wait complete. 2024-03-15T13:08:00.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.679+0000 7fe80e567740 20 client.18369 trim_cache size 0 max 16384 2024-03-15T13:08:00.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.695+0000 7fe80e567740 1 Processor -- start 2024-03-15T13:08:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.695+0000 7fe80e567740 1 -- start start 2024-03-15T13:08:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.695+0000 7fe80e567740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fa2b0 0x5561f43005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.695+0000 7fe80e567740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f42fabd0 0x5561f4302d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.695+0000 7fe80e567740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f43037b0 0x5561f4303270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.695+0000 7fe80e567740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f42f95a0 con 0x5561f42fa2b0 2024-03-15T13:08:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.695+0000 7fe80e567740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f4306170 con 0x5561f42fabd0 2024-03-15T13:08:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.695+0000 7fe80e567740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f42f59e0 con 0x5561f43037b0 2024-03-15T13:08:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.695+0000 7fe809643640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fa2b0 0x5561f43005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.695+0000 7fe809643640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fa2b0 0x5561f43005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60540/0 (socket says 172.21.15.2:60540) 2024-03-15T13:08:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.695+0000 7fe809643640 1 -- 172.21.15.2:0/3490066324 learned_addr learned my addr 172.21.15.2:0/3490066324 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.695+0000 7fe808e42640 1 --2- 172.21.15.2:0/3490066324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f42fabd0 0x5561f4302d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.695+0000 7fe809e44640 1 --2- 172.21.15.2:0/3490066324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f43037b0 0x5561f4303270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.696+0000 7fe809643640 1 -- 172.21.15.2:0/3490066324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f43037b0 msgr2=0x5561f4303270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.696+0000 7fe809643640 1 --2- 172.21.15.2:0/3490066324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f43037b0 0x5561f4303270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.696+0000 7fe809643640 1 -- 172.21.15.2:0/3490066324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f42fabd0 msgr2=0x5561f4302d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.696+0000 7fe809643640 1 --2- 172.21.15.2:0/3490066324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f42fabd0 0x5561f4302d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.696+0000 7fe809643640 1 -- 172.21.15.2:0/3490066324 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561f41fcac0 con 0x5561f42fa2b0 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.696+0000 7fe809643640 1 --2- 172.21.15.2:0/3490066324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fa2b0 0x5561f43005a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc036040 tx=0x7fe7fc042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a2f1c8eed4c96e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.696+0000 7fe7eeffd640 1 -- 172.21.15.2:0/3490066324 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc051040 con 0x5561f42fa2b0 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.696+0000 7fe7eeffd640 1 -- 172.21.15.2:0/3490066324 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe7fc044070 con 0x5561f42fa2b0 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.696+0000 7fe7eeffd640 1 -- 172.21.15.2:0/3490066324 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc058e00 con 0x5561f42fa2b0 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.696+0000 7fe80e567740 1 -- 172.21.15.2:0/3490066324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fa2b0 msgr2=0x5561f43005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.696+0000 7fe80e567740 1 --2- 172.21.15.2:0/3490066324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fa2b0 0x5561f43005a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc036040 tx=0x7fe7fc042000 comp rx=0 tx=0).stop 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.696+0000 7fe80e567740 1 -- 172.21.15.2:0/3490066324 shutdown_connections 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.696+0000 7fe80e567740 1 --2- 172.21.15.2:0/3490066324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f43037b0 0x5561f4303270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.696+0000 7fe80e567740 1 --2- 172.21.15.2:0/3490066324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f42fabd0 0x5561f4302d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.696+0000 7fe80e567740 1 --2- 172.21.15.2:0/3490066324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fa2b0 0x5561f43005a0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe80e567740 1 -- 172.21.15.2:0/3490066324 >> 172.21.15.2:0/3490066324 conn(0x5561f42f4990 msgr2=0x5561f42ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:00.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe80e567740 1 -- 172.21.15.2:0/3490066324 shutdown_connections 2024-03-15T13:08:00.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe80e567740 1 -- 172.21.15.2:0/3490066324 wait complete. 2024-03-15T13:08:00.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe80e567740 1 -- start start 2024-03-15T13:08:00.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe80e567740 1 Processor -- start 2024-03-15T13:08:00.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe80e567740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fa2b0 0x5561f430b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe80e567740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fabd0 0x5561f4307690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe80e567740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f43037b0 0x5561f4307bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe80e567740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f4308190 con 0x5561f42fabd0 2024-03-15T13:08:00.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe80e567740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f4308300 con 0x5561f43037b0 2024-03-15T13:08:00.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe80e567740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f43084c0 con 0x5561f42fa2b0 2024-03-15T13:08:00.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe80e567740 10 client.0 _ll_register_callbacks cb 0x7fe7f40d1710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:00.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe808e42640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fabd0 0x5561f4307690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe808e42640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fabd0 0x5561f4307690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60544/0 (socket says 172.21.15.2:60544) 2024-03-15T13:08:00.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe808e42640 1 -- 172.21.15.2:0/2347959938 learned_addr learned my addr 172.21.15.2:0/2347959938 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:00.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe809643640 1 --2- 172.21.15.2:0/2347959938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fa2b0 0x5561f430b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.697+0000 7fe809e44640 1 --2- 172.21.15.2:0/2347959938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f43037b0 0x5561f4307bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe808e42640 1 -- 172.21.15.2:0/2347959938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fa2b0 msgr2=0x5561f430b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe808e42640 1 --2- 172.21.15.2:0/2347959938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fa2b0 0x5561f430b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe808e42640 1 -- 172.21.15.2:0/2347959938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f43037b0 msgr2=0x5561f4307bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:00.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe808e42640 1 --2- 172.21.15.2:0/2347959938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f43037b0 0x5561f4307bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe808e42640 1 -- 172.21.15.2:0/2347959938 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80410eb70 con 0x5561f42fabd0 2024-03-15T13:08:00.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe809643640 1 --2- 172.21.15.2:0/2347959938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fa2b0 0x5561f430b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:00.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe808e42640 1 --2- 172.21.15.2:0/2347959938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fabd0 0x5561f4307690 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe7f409a710 tx=0x7fe7f4096ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a332e1983628b287 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:00.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe7ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:00.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe7ecff9640 1 -- 172.21.15.2:0/2347959938 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f409a0e0 con 0x5561f42fabd0 2024-03-15T13:08:00.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe80e567740 1 -- 172.21.15.2:0/2347959938 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561f3f617e0 con 0x5561f42fabd0 2024-03-15T13:08:00.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe7ecff9640 1 -- 172.21.15.2:0/2347959938 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe7f409ae80 con 0x5561f42fabd0 2024-03-15T13:08:00.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe7ecff9640 1 -- 172.21.15.2:0/2347959938 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7f409a780 con 0x5561f42fabd0 2024-03-15T13:08:00.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe80e567740 10 client.18375 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:00.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe80e567740 1 -- 172.21.15.2:0/2347959938 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5561f42226a0 con 0x5561f42fabd0 2024-03-15T13:08:00.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe7ecff9640 1 -- 172.21.15.2:0/2347959938 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5561f3f617e0 con 0x5561f42fabd0 2024-03-15T13:08:00.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.698+0000 7fe7ecff9640 1 -- 172.21.15.2:0/2347959938 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe7f401f060 con 0x5561f42fabd0 2024-03-15T13:08:00.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 10 client.18375 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:00.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 10 client.18375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:08:00.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 20 client.18375 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:00.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 1 --2- 172.21.15.2:0/2347959938 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5561f43708c0 0x5561f4372cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:00.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 4 client.18375 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:08:00.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 1 -- 172.21.15.2:0/2347959938 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5561f3f504e0 con 0x5561f43708c0 2024-03-15T13:08:00.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe809643640 1 --2- 172.21.15.2:0/2347959938 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5561f43708c0 0x5561f4372cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:00.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe809643640 1 --2- 172.21.15.2:0/2347959938 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5561f43708c0 0x5561f4372cb0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38c19b557a7464bb server_cookie=9300b47bda9243e in_seq=0 out_seq=0 2024-03-15T13:08:00.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe7ecff9640 10 client.18375 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:00.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe7ecff9640 1 -- 172.21.15.2:0/2347959938 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5561f3f504e0 con 0x5561f43708c0 2024-03-15T13:08:00.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe7ecff9640 10 client.18375 handle_command_reply: tid=0 2024-03-15T13:08:00.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 1 client.18375 shutdown 2024-03-15T13:08:00.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 1 -- 172.21.15.2:0/2347959938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fabd0 msgr2=0x5561f4307690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 1 --2- 172.21.15.2:0/2347959938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fabd0 0x5561f4307690 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe7f409a710 tx=0x7fe7f4096ae0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 1 -- 172.21.15.2:0/2347959938 shutdown_connections 2024-03-15T13:08:00.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 1 --2- 172.21.15.2:0/2347959938 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x5561f43708c0 0x5561f4372cb0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 1 --2- 172.21.15.2:0/2347959938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561f43037b0 0x5561f4307bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 1 --2- 172.21.15.2:0/2347959938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5561f42fabd0 0x5561f4307690 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 1 --2- 172.21.15.2:0/2347959938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f42fa2b0 0x5561f430b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe7ecff9640 0 client.18375 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:00.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 1 -- 172.21.15.2:0/2347959938 >> 172.21.15.2:0/2347959938 conn(0x5561f42f4990 msgr2=0x5561f42f31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:00.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 1 -- 172.21.15.2:0/2347959938 shutdown_connections 2024-03-15T13:08:00.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.699+0000 7fe80e567740 1 -- 172.21.15.2:0/2347959938 wait complete. 2024-03-15T13:08:00.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.700+0000 7fe80e567740 20 client.18375 trim_cache size 0 max 16384 2024-03-15T13:08:00.691 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:08:00.692 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T13:08:00.692 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:08:00.692 INFO:teuthology.orchestra.run.smithi002.stdout: "9c62f729-e7c5-441d-801e-9fc75f9d3938": { 2024-03-15T13:08:00.692 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:08:00.692 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "9c62f729-e7c5-441d-801e-9fc75f9d3938", 2024-03-15T13:08:00.692 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:08:00.692 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:08:00.692 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:08:00.692 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:08:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.700+0000 7fe80b8a7640 1 -- 172.21.15.2:0/2938296798 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe7f00a1ba0 msgr2=0x7fe7f00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.700+0000 7fe80b8a7640 1 --2- 172.21.15.2:0/2938296798 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe7f00a1ba0 0x7fe7f00a4060 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc005ab0 tx=0x7fe7fc005fd0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.700+0000 7fe80b8a7640 1 -- 172.21.15.2:0/2938296798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe804103030 msgr2=0x7fe80419dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.700+0000 7fe80b8a7640 1 --2- 172.21.15.2:0/2938296798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe804103030 0x7fe80419dc70 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400e9f0 tx=0x7fe7f400edf0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.700+0000 7fe80b8a7640 1 -- 172.21.15.2:0/2938296798 shutdown_connections 2024-03-15T13:08:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.700+0000 7fe80b8a7640 1 --2- 172.21.15.2:0/2938296798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe80410ba50 0x7fe8041978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.700+0000 7fe80b8a7640 1 --2- 172.21.15.2:0/2938296798 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe7f00a1ba0 0x7fe7f00a4060 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.700+0000 7fe80b8a7640 1 --2- 172.21.15.2:0/2938296798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe804103030 0x7fe80419dc70 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.700+0000 7fe80b8a7640 1 --2- 172.21.15.2:0/2938296798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe804102710 0x7fe80419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.700+0000 7fe80b8a7640 1 -- 172.21.15.2:0/2938296798 >> 172.21.15.2:0/2938296798 conn(0x7fe8040775a0 msgr2=0x7fe8040fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:00.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.700+0000 7fe80b8a7640 1 -- 172.21.15.2:0/2938296798 shutdown_connections 2024-03-15T13:08:00.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:00.700+0000 7fe80b8a7640 1 -- 172.21.15.2:0/2938296798 wait complete. 2024-03-15T13:08:00.695 INFO:tasks.cephfs.filesystem:scrub status for tag:9c62f729-e7c5-441d-801e-9fc75f9d3938 - {'path': '/', 'tag': '9c62f729-e7c5-441d-801e-9fc75f9d3938', 'options': 'recursive,force'} 2024-03-15T13:08:01.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:01 smithi149 ceph-mon[35927]: pgmap v280: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:08:01.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:01 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:08:01.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:01 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:08:01.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:01 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:08:01.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:01 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:08:02.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:01 smithi002 ceph-mon[32060]: pgmap v280: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:08:02.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:01 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:08:02.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:01 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:08:02.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:01 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:08:02.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:01 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:08:02.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:01 smithi184 ceph-mon[35993]: pgmap v280: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:08:02.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:01 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:08:02.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:01 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:08:02.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:01 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:08:02.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:01 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:08:03.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:03 smithi149 ceph-mon[35927]: pgmap v281: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:04.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:03 smithi002 ceph-mon[32060]: pgmap v281: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:04.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:03 smithi184 ceph-mon[35993]: pgmap v281: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:05.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:05 smithi149 ceph-mon[35927]: pgmap v282: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:05.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:05 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:08:05.859 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:05 smithi184 ceph-mon[35993]: pgmap v282: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:05.859 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:05 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:08:06.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:05 smithi002 ceph-mon[32060]: pgmap v282: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:06.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:05 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:08:07.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:07 smithi149 ceph-mon[35927]: pgmap v283: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-15T13:08:07.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:07 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:08:08.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:07 smithi002 ceph-mon[32060]: pgmap v283: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-15T13:08:08.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:07 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:08:08.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:07 smithi184 ceph-mon[35993]: pgmap v283: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-03-15T13:08:08.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:07 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:08:08.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:08 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:08:08.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:08 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:08:08.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:08 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:08:09.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:08 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:08:09.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:08 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:08:09.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:08 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:08:09.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:08 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:08:09.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:08 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:08:09.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:08 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:08:09.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:09 smithi149 ceph-mon[35927]: pgmap v284: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-15T13:08:10.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:09 smithi002 ceph-mon[32060]: pgmap v284: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-15T13:08:10.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:09 smithi184 ceph-mon[35993]: pgmap v284: 129 pgs: 129 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-03-15T13:08:11.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:10 smithi002 ceph-mon[32060]: osdmap e97: 12 total, 12 up, 12 in 2024-03-15T13:08:11.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:10 smithi002 ceph-mon[32060]: pgmap v286: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-03-15T13:08:11.307 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:10 smithi184 ceph-mon[35993]: osdmap e97: 12 total, 12 up, 12 in 2024-03-15T13:08:11.307 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:10 smithi184 ceph-mon[35993]: pgmap v286: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-03-15T13:08:11.358 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:10 smithi149 ceph-mon[35927]: osdmap e97: 12 total, 12 up, 12 in 2024-03-15T13:08:11.358 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:10 smithi149 ceph-mon[35927]: pgmap v286: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-03-15T13:08:13.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:13 smithi002 ceph-mon[32060]: pgmap v287: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-03-15T13:08:13.307 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:13 smithi184 ceph-mon[35993]: pgmap v287: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-03-15T13:08:13.358 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:13 smithi149 ceph-mon[35927]: pgmap v287: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-03-15T13:08:15.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:15 smithi002 ceph-mon[32060]: pgmap v288: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-03-15T13:08:15.307 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:15 smithi184 ceph-mon[35993]: pgmap v288: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-03-15T13:08:15.358 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:15 smithi149 ceph-mon[35927]: pgmap v288: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 247 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-03-15T13:08:17.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:17 smithi002 ceph-mon[32060]: pgmap v289: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:08:17.307 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:17 smithi184 ceph-mon[35993]: pgmap v289: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:08:17.358 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:17 smithi149 ceph-mon[35927]: pgmap v289: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:08:19.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:19 smithi002 ceph-mon[32060]: pgmap v290: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:08:19.307 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:19 smithi184 ceph-mon[35993]: pgmap v290: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:08:19.358 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:19 smithi149 ceph-mon[35927]: pgmap v290: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-03-15T13:08:21.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:21 smithi002 ceph-mon[32060]: pgmap v291: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 101 B/s wr, 11 op/s 2024-03-15T13:08:21.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:21 smithi002 ceph-mon[32060]: osdmap e98: 12 total, 12 up, 12 in 2024-03-15T13:08:21.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:21 smithi184 ceph-mon[35993]: pgmap v291: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 101 B/s wr, 11 op/s 2024-03-15T13:08:21.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:21 smithi184 ceph-mon[35993]: osdmap e98: 12 total, 12 up, 12 in 2024-03-15T13:08:21.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:21 smithi149 ceph-mon[35927]: pgmap v291: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 101 B/s wr, 11 op/s 2024-03-15T13:08:21.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:21 smithi149 ceph-mon[35927]: osdmap e98: 12 total, 12 up, 12 in 2024-03-15T13:08:23.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:23 smithi002 ceph-mon[32060]: pgmap v293: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:08:23.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:23 smithi184 ceph-mon[35993]: pgmap v293: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:08:23.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:23 smithi149 ceph-mon[35927]: pgmap v293: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:08:25.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:25 smithi002 ceph-mon[32060]: pgmap v294: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:08:25.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:25 smithi184 ceph-mon[35993]: pgmap v294: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:08:25.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:25 smithi149 ceph-mon[35927]: pgmap v294: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:08:27.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:27 smithi002 ceph-mon[32060]: pgmap v295: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-15T13:08:27.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:27 smithi184 ceph-mon[35993]: pgmap v295: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-15T13:08:27.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:27 smithi149 ceph-mon[35927]: pgmap v295: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-15T13:08:29.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:29 smithi002 ceph-mon[32060]: pgmap v296: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-15T13:08:29.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:29 smithi184 ceph-mon[35993]: pgmap v296: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-15T13:08:29.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:29 smithi149 ceph-mon[35927]: pgmap v296: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-15T13:08:30.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:08:30.696 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:08:30.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.828+0000 7f6700d5e640 1 -- 172.21.15.2:0/296856523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66fc1005e0 msgr2=0x7f66fc1009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:30.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.828+0000 7f6700d5e640 1 --2- 172.21.15.2:0/296856523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66fc1005e0 0x7f66fc1009c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009930 tx=0x7f66ec02bc30 comp rx=0 tx=0).stop 2024-03-15T13:08:30.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.828+0000 7f6700d5e640 1 -- 172.21.15.2:0/296856523 shutdown_connections 2024-03-15T13:08:30.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.828+0000 7f6700d5e640 1 --2- 172.21.15.2:0/296856523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc1056b0 0x7f66fc10d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.828+0000 7f6700d5e640 1 --2- 172.21.15.2:0/296856523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66fc100f90 0x7f66fc104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.828+0000 7f6700d5e640 1 --2- 172.21.15.2:0/296856523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66fc1005e0 0x7f66fc1009c0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.828+0000 7f6700d5e640 1 -- 172.21.15.2:0/296856523 >> 172.21.15.2:0/296856523 conn(0x7f66fc0775a0 msgr2=0x7f66fc0779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.828+0000 7f6700d5e640 1 -- 172.21.15.2:0/296856523 shutdown_connections 2024-03-15T13:08:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.828+0000 7f6700d5e640 1 -- 172.21.15.2:0/296856523 wait complete. 2024-03-15T13:08:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f6700d5e640 1 Processor -- start 2024-03-15T13:08:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f6700d5e640 1 -- start start 2024-03-15T13:08:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f6700d5e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66fc1005e0 0x7f66fc102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f6700d5e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc1056b0 0x7f66fc1025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f6700d5e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66fc102af0 0x7f66fc1a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f6700d5e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1a6290 con 0x7f66fc102af0 2024-03-15T13:08:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f6700d5e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1a6400 con 0x7f66fc1005e0 2024-03-15T13:08:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f6700d5e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1a65c0 con 0x7f66fc1056b0 2024-03-15T13:08:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f66fad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66fc102af0 0x7f66fc1a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f66fad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66fc102af0 0x7f66fc1a5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57346/0 (socket says 172.21.15.2:57346) 2024-03-15T13:08:30.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f66fad9d640 1 -- 172.21.15.2:0/1995196312 learned_addr learned my addr 172.21.15.2:0/1995196312 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:30.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f66fa59c640 1 --2- 172.21.15.2:0/1995196312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66fc1005e0 0x7f66fc102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f66f3fff640 1 --2- 172.21.15.2:0/1995196312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc1056b0 0x7f66fc1025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f66fad9d640 1 -- 172.21.15.2:0/1995196312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc1056b0 msgr2=0x7f66fc1025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:30.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f66fad9d640 1 --2- 172.21.15.2:0/1995196312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc1056b0 0x7f66fc1025b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.829+0000 7f66fad9d640 1 -- 172.21.15.2:0/1995196312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66fc1005e0 msgr2=0x7f66fc102070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:30.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.830+0000 7f66fad9d640 1 --2- 172.21.15.2:0/1995196312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66fc1005e0 0x7f66fc102070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.830+0000 7f66fad9d640 1 -- 172.21.15.2:0/1995196312 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ec009560 con 0x7f66fc102af0 2024-03-15T13:08:30.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.830+0000 7f66fa59c640 1 --2- 172.21.15.2:0/1995196312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66fc1005e0 0x7f66fc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:30.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.830+0000 7f66fad9d640 1 --2- 172.21.15.2:0/1995196312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66fc102af0 0x7f66fc1a5cc0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f66f400eb10 tx=0x7f66f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:30.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.830+0000 7f66f37fe640 1 -- 172.21.15.2:0/1995196312 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4004b20 con 0x7f66fc102af0 2024-03-15T13:08:30.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.830+0000 7f66f37fe640 1 -- 172.21.15.2:0/1995196312 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f66f4004c80 con 0x7f66fc102af0 2024-03-15T13:08:30.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.830+0000 7f6700d5e640 1 -- 172.21.15.2:0/1995196312 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66fc1a68a0 con 0x7f66fc102af0 2024-03-15T13:08:30.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.830+0000 7f66f37fe640 1 -- 172.21.15.2:0/1995196312 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4010690 con 0x7f66fc102af0 2024-03-15T13:08:30.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.830+0000 7f6700d5e640 1 -- 172.21.15.2:0/1995196312 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc1a6db0 con 0x7f66fc102af0 2024-03-15T13:08:30.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.832+0000 7f66f37fe640 1 -- 172.21.15.2:0/1995196312 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f66f4004df0 con 0x7f66fc102af0 2024-03-15T13:08:30.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.832+0000 7f6703a19740 1 Processor -- start 2024-03-15T13:08:30.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f6703a19740 1 -- start start 2024-03-15T13:08:30.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f6703a19740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 0x55aa84226690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f6703a19740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa84226bd0 0x55aa8422f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f6703a19740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa8422f7b0 0x55aa84231ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f6703a19740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa84195280 con 0x55aa8422f7b0 2024-03-15T13:08:30.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f6703a19740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa841927b0 con 0x55aa84226bd0 2024-03-15T13:08:30.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f6703a19740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8418df90 con 0x55aa842262b0 2024-03-15T13:08:30.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f66f3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa8422f7b0 0x55aa84231ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f66f37fe640 1 --2- 172.21.15.2:0/1995196312 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f66e80a1bd0 0x7f66e80a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f66f3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa8422f7b0 0x55aa84231ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57352/0 (socket says 172.21.15.2:57352) 2024-03-15T13:08:30.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f66f3fff640 1 -- 172.21.15.2:0/936626109 learned_addr learned my addr 172.21.15.2:0/936626109 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:30.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f66fa59c640 1 --2- 172.21.15.2:0/936626109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa84226bd0 0x55aa8422f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f66f3fff640 1 --2- 172.21.15.2:0/936626109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 0x55aa84226690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f66fad9d640 1 --2- 172.21.15.2:0/1995196312 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f66e80a1bd0 0x7f66e80a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f66fa59c640 1 -- 172.21.15.2:0/936626109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 msgr2=0x55aa84226690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f66fa59c640 1 --2- 172.21.15.2:0/936626109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 0x55aa84226690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f66fa59c640 1 -- 172.21.15.2:0/936626109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa8422f7b0 msgr2=0x55aa84231ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f66fa59c640 1 --2- 172.21.15.2:0/936626109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa8422f7b0 0x55aa84231ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f66f37fe640 1 -- 172.21.15.2:0/1995196312 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f66f4014070 con 0x7f66fc102af0 2024-03-15T13:08:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.833+0000 7f66fa59c640 1 -- 172.21.15.2:0/936626109 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa8414c570 con 0x55aa84226bd0 2024-03-15T13:08:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.834+0000 7f66fa59c640 1 --2- 172.21.15.2:0/936626109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa84226bd0 0x55aa8422f270 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f66ec02c170 tx=0x7f66ec004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e406ecb8f526d483 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.834+0000 7f66fad9d640 1 --2- 172.21.15.2:0/1995196312 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f66e80a1bd0 0x7f66e80a4090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f66f40cb620 tx=0x7f66f409e040 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.834+0000 7f66f0ff9640 1 -- 172.21.15.2:0/936626109 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec02c5d0 con 0x55aa84226bd0 2024-03-15T13:08:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.834+0000 7f66f0ff9640 1 -- 172.21.15.2:0/936626109 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f66ec004030 con 0x55aa84226bd0 2024-03-15T13:08:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.834+0000 7f66f0ff9640 1 -- 172.21.15.2:0/936626109 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec003a20 con 0x55aa84226bd0 2024-03-15T13:08:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.835+0000 7f6703a19740 1 -- 172.21.15.2:0/936626109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa84226bd0 msgr2=0x55aa8422f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.835+0000 7f6703a19740 1 --2- 172.21.15.2:0/936626109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa84226bd0 0x55aa8422f270 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f66ec02c170 tx=0x7f66ec004ab0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.835+0000 7f6703a19740 1 -- 172.21.15.2:0/936626109 shutdown_connections 2024-03-15T13:08:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.835+0000 7f6703a19740 1 --2- 172.21.15.2:0/936626109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa8422f7b0 0x55aa84231ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.835+0000 7f6703a19740 1 --2- 172.21.15.2:0/936626109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa84226bd0 0x55aa8422f270 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.835+0000 7f6703a19740 1 --2- 172.21.15.2:0/936626109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 0x55aa84226690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.835+0000 7f6703a19740 1 -- 172.21.15.2:0/936626109 >> 172.21.15.2:0/936626109 conn(0x55aa84220990 msgr2=0x55aa84220d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.835+0000 7f6703a19740 1 -- 172.21.15.2:0/936626109 shutdown_connections 2024-03-15T13:08:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.835+0000 7f6703a19740 1 -- 172.21.15.2:0/936626109 wait complete. 2024-03-15T13:08:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.835+0000 7f6703a19740 1 -- start start 2024-03-15T13:08:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.835+0000 7f6703a19740 1 Processor -- start 2024-03-15T13:08:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f6703a19740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 0x55aa8422f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f6703a19740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa84226bd0 0x55aa8422c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f6703a19740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa8422f7b0 0x55aa8422cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f6703a19740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa83e61430 con 0x55aa8422f7b0 2024-03-15T13:08:30.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f6703a19740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa83e1de00 con 0x55aa84226bd0 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f6703a19740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa84112fd0 con 0x55aa842262b0 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f6703a19740 10 client.0 _ll_register_callbacks cb 0x55aa84168fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f66f3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa8422f7b0 0x55aa8422cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f66f3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 0x55aa8422f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f66f3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 0x55aa8422f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:39866/0 (socket says 172.21.15.2:39866) 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f66f3fff640 1 -- 172.21.15.2:0/2912019337 learned_addr learned my addr 172.21.15.2:0/2912019337 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f66fa59c640 1 --2- 172.21.15.2:0/2912019337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa84226bd0 0x55aa8422c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f66f3fff640 1 -- 172.21.15.2:0/2912019337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa84226bd0 msgr2=0x55aa8422c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f66f3fff640 1 --2- 172.21.15.2:0/2912019337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa84226bd0 0x55aa8422c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f66f3fff640 1 -- 172.21.15.2:0/2912019337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa8422f7b0 msgr2=0x55aa8422cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f66f3fff640 1 --2- 172.21.15.2:0/2912019337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa8422f7b0 0x55aa8422cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f66f3fff640 1 -- 172.21.15.2:0/2912019337 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc10da90 con 0x55aa842262b0 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f66f3fff640 1 --2- 172.21.15.2:0/2912019337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 0x55aa8422f180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f66e400f430 tx=0x7f66e4005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=381c60b0bdaf4305 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f66e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f66e1ffb640 1 -- 172.21.15.2:0/2912019337 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e400fe70 con 0x55aa842262b0 2024-03-15T13:08:30.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f66e1ffb640 1 -- 172.21.15.2:0/2912019337 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f66e4004030 con 0x55aa842262b0 2024-03-15T13:08:30.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f6703a19740 1 -- 172.21.15.2:0/2912019337 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa83e8d7e0 con 0x55aa842262b0 2024-03-15T13:08:30.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.836+0000 7f66e1ffb640 1 -- 172.21.15.2:0/2912019337 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66e40041f0 con 0x55aa842262b0 2024-03-15T13:08:30.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.837+0000 7f6703a19740 10 client.28180 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:30.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.837+0000 7f6703a19740 1 -- 172.21.15.2:0/2912019337 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55aa8418c610 con 0x55aa842262b0 2024-03-15T13:08:30.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.837+0000 7f66e1ffb640 1 -- 172.21.15.2:0/2912019337 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55aa83e8d7e0 con 0x55aa842262b0 2024-03-15T13:08:30.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.837+0000 7f66e1ffb640 1 -- 172.21.15.2:0/2912019337 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f66e4004960 con 0x55aa842262b0 2024-03-15T13:08:30.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.837+0000 7f6703a19740 10 client.28180 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:30.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.837+0000 7f6703a19740 10 client.28180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:08:30.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.837+0000 7f6703a19740 20 client.28180 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:30.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.837+0000 7f6703a19740 1 --2- 172.21.15.2:0/2912019337 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55aa8423f890 0x55aa84241cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.837+0000 7f6703a19740 4 client.28180 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:08:30.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.837+0000 7f6703a19740 1 -- 172.21.15.2:0/2912019337 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa83eca720 con 0x55aa8423f890 2024-03-15T13:08:30.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.837+0000 7f66fa59c640 1 --2- 172.21.15.2:0/2912019337 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55aa8423f890 0x55aa84241cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.837+0000 7f66fa59c640 1 --2- 172.21.15.2:0/2912019337 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55aa8423f890 0x55aa84241cb0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f67e7ac3f98ac6b server_cookie=50234a1c392118e4 in_seq=0 out_seq=0 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.837+0000 7f66e1ffb640 10 client.28180 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.838+0000 7f66e1ffb640 1 -- 172.21.15.2:0/2912019337 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55aa83eca720 con 0x55aa8423f890 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.838+0000 7f66e1ffb640 10 client.28180 handle_command_reply: tid=0 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.838+0000 7f6703a19740 1 client.28180 shutdown 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.839+0000 7f6703a19740 1 -- 172.21.15.2:0/2912019337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 msgr2=0x55aa8422f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.839+0000 7f6703a19740 1 --2- 172.21.15.2:0/2912019337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 0x55aa8422f180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f66e400f430 tx=0x7f66e4005850 comp rx=0 tx=0).stop 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.839+0000 7f6703a19740 1 -- 172.21.15.2:0/2912019337 shutdown_connections 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.839+0000 7f6703a19740 1 --2- 172.21.15.2:0/2912019337 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55aa8423f890 0x55aa84241cb0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.839+0000 7f6703a19740 1 --2- 172.21.15.2:0/2912019337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa8422f7b0 0x55aa8422cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.839+0000 7f6703a19740 1 --2- 172.21.15.2:0/2912019337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa84226bd0 0x55aa8422c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.839+0000 7f6703a19740 1 --2- 172.21.15.2:0/2912019337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 0x55aa8422f180 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.839+0000 7f66e1ffb640 0 client.28180 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.839+0000 7f6703a19740 1 -- 172.21.15.2:0/2912019337 >> 172.21.15.2:0/2912019337 conn(0x55aa84220990 msgr2=0x55aa84222730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.839+0000 7f6703a19740 1 -- 172.21.15.2:0/2912019337 shutdown_connections 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.839+0000 7f6703a19740 1 -- 172.21.15.2:0/2912019337 wait complete. 2024-03-15T13:08:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.839+0000 7f6703a19740 20 client.28180 trim_cache size 0 max 16384 2024-03-15T13:08:30.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.855+0000 7f6703a19740 1 Processor -- start 2024-03-15T13:08:30.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.855+0000 7f6703a19740 1 -- start start 2024-03-15T13:08:30.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.855+0000 7f6703a19740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 0x55aa8422ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.855+0000 7f6703a19740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa84226bd0 0x55aa8422f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.855+0000 7f6703a19740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa8422f7b0 0x55aa8423aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.855+0000 7f6703a19740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8421c100 con 0x55aa84226bd0 2024-03-15T13:08:30.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.855+0000 7f6703a19740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa84221a10 con 0x55aa8422f7b0 2024-03-15T13:08:30.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.855+0000 7f6703a19740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8422d4c0 con 0x55aa842262b0 2024-03-15T13:08:30.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.855+0000 7f66fa59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa84226bd0 0x55aa8422f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.855+0000 7f66fa59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa84226bd0 0x55aa8422f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57376/0 (socket says 172.21.15.2:57376) 2024-03-15T13:08:30.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.855+0000 7f66fa59c640 1 -- 172.21.15.2:0/2018700797 learned_addr learned my addr 172.21.15.2:0/2018700797 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:30.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.855+0000 7f66f3fff640 1 --2- 172.21.15.2:0/2018700797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 0x55aa8422ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.855+0000 7f66f3fff640 1 --2- 172.21.15.2:0/2018700797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa8422f7b0 0x55aa8423aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f66fa59c640 1 -- 172.21.15.2:0/2018700797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 msgr2=0x55aa8422ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:30.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f66fa59c640 1 --2- 172.21.15.2:0/2018700797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 0x55aa8422ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f66fa59c640 1 -- 172.21.15.2:0/2018700797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa8422f7b0 msgr2=0x55aa8423aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:30.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f66fa59c640 1 --2- 172.21.15.2:0/2018700797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa8422f7b0 0x55aa8423aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f66fa59c640 1 -- 172.21.15.2:0/2018700797 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa84128ac0 con 0x55aa84226bd0 2024-03-15T13:08:30.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f66f3fff640 1 --2- 172.21.15.2:0/2018700797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa8422f7b0 0x55aa8423aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:08:30.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f66fa59c640 1 --2- 172.21.15.2:0/2018700797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa84226bd0 0x55aa8422f180 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f66ec02bc70 tx=0x7f66ec042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d580257cca3d389f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f66e3fff640 1 -- 172.21.15.2:0/2018700797 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0522c0 con 0x55aa84226bd0 2024-03-15T13:08:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f66e3fff640 1 -- 172.21.15.2:0/2018700797 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f66ec004de0 con 0x55aa84226bd0 2024-03-15T13:08:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f66e3fff640 1 -- 172.21.15.2:0/2018700797 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec003850 con 0x55aa84226bd0 2024-03-15T13:08:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f6703a19740 1 -- 172.21.15.2:0/2018700797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa84226bd0 msgr2=0x55aa8422f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f6703a19740 1 --2- 172.21.15.2:0/2018700797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa84226bd0 0x55aa8422f180 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f66ec02bc70 tx=0x7f66ec042040 comp rx=0 tx=0).stop 2024-03-15T13:08:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f6703a19740 1 -- 172.21.15.2:0/2018700797 shutdown_connections 2024-03-15T13:08:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f6703a19740 1 --2- 172.21.15.2:0/2018700797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa8422f7b0 0x55aa8423aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f6703a19740 1 --2- 172.21.15.2:0/2018700797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa84226bd0 0x55aa8422f180 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f6703a19740 1 --2- 172.21.15.2:0/2018700797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa842262b0 0x55aa8422ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f6703a19740 1 -- 172.21.15.2:0/2018700797 >> 172.21.15.2:0/2018700797 conn(0x55aa84220990 msgr2=0x55aa84227850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f6703a19740 1 -- 172.21.15.2:0/2018700797 shutdown_connections 2024-03-15T13:08:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.856+0000 7f6703a19740 1 -- 172.21.15.2:0/2018700797 wait complete. 2024-03-15T13:08:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f6703a19740 1 -- start start 2024-03-15T13:08:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f6703a19740 1 Processor -- start 2024-03-15T13:08:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f6703a19740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa842262b0 0x55aa8422fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f6703a19740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa84226bd0 0x55aa84230190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f6703a19740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa8422f7b0 0x55aa842306d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f6703a19740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa84231760 con 0x55aa84226bd0 2024-03-15T13:08:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f6703a19740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa842318d0 con 0x55aa842262b0 2024-03-15T13:08:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f6703a19740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa84236610 con 0x55aa8422f7b0 2024-03-15T13:08:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f6703a19740 10 client.0 _ll_register_callbacks cb 0x7f66e4012eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f66fa59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa84226bd0 0x55aa84230190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f66f3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa842262b0 0x55aa8422fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f66fa59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa84226bd0 0x55aa84230190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57380/0 (socket says 172.21.15.2:57380) 2024-03-15T13:08:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f66fa59c640 1 -- 172.21.15.2:0/1859161204 learned_addr learned my addr 172.21.15.2:0/1859161204 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f66f3fff640 1 --2- 172.21.15.2:0/1859161204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa8422f7b0 0x55aa842306d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f66fa59c640 1 -- 172.21.15.2:0/1859161204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa8422f7b0 msgr2=0x55aa842306d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f66fa59c640 1 --2- 172.21.15.2:0/1859161204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa8422f7b0 0x55aa842306d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f66fa59c640 1 -- 172.21.15.2:0/1859161204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa842262b0 msgr2=0x55aa8422fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f66fa59c640 1 --2- 172.21.15.2:0/1859161204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa842262b0 0x55aa8422fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f66fa59c640 1 -- 172.21.15.2:0/1859161204 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa8414c570 con 0x55aa84226bd0 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.857+0000 7f66f3fff640 1 --2- 172.21.15.2:0/1859161204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa842262b0 0x55aa8422fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f66fa59c640 1 --2- 172.21.15.2:0/1859161204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa84226bd0 0x55aa84230190 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f66ec000c00 tx=0x7f66ec004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d978aac5a02eb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f66e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f66e1ffb640 1 -- 172.21.15.2:0/1859161204 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec004f50 con 0x55aa84226bd0 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f66e1ffb640 1 -- 172.21.15.2:0/1859161204 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f66ec04f070 con 0x55aa84226bd0 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f6703a19740 1 -- 172.21.15.2:0/1859161204 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa8422c340 con 0x55aa84226bd0 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f66e1ffb640 1 -- 172.21.15.2:0/1859161204 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66ec0050b0 con 0x55aa84226bd0 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f6703a19740 10 client.18393 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f6703a19740 1 -- 172.21.15.2:0/1859161204 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55aa8414e6a0 con 0x55aa84226bd0 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f66e1ffb640 1 -- 172.21.15.2:0/1859161204 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55aa8422c340 con 0x55aa84226bd0 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f66e1ffb640 1 -- 172.21.15.2:0/1859161204 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f66ec03c050 con 0x55aa84226bd0 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f6703a19740 10 client.18393 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f6703a19740 10 client.18393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:08:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f6703a19740 20 client.18393 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f6703a19740 1 --2- 172.21.15.2:0/1859161204 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55aa8429b580 0x55aa8429d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f6703a19740 4 client.18393 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:08:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.858+0000 7f6703a19740 1 -- 172.21.15.2:0/1859161204 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f66ec004030 con 0x55aa8429b580 2024-03-15T13:08:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f66f3fff640 1 --2- 172.21.15.2:0/1859161204 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55aa8429b580 0x55aa8429d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f66f3fff640 1 --2- 172.21.15.2:0/1859161204 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55aa8429b580 0x55aa8429d9a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ae7dfa54e7a6eb8 server_cookie=5bb571ad5ba00952 in_seq=0 out_seq=0 2024-03-15T13:08:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f66e1ffb640 10 client.18393 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f66e1ffb640 1 -- 172.21.15.2:0/1859161204 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f66ec004030 con 0x55aa8429b580 2024-03-15T13:08:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f66e1ffb640 10 client.18393 handle_command_reply: tid=0 2024-03-15T13:08:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f6703a19740 1 client.18393 shutdown 2024-03-15T13:08:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f6703a19740 1 -- 172.21.15.2:0/1859161204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa84226bd0 msgr2=0x55aa84230190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f6703a19740 1 --2- 172.21.15.2:0/1859161204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa84226bd0 0x55aa84230190 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f66ec000c00 tx=0x7f66ec004470 comp rx=0 tx=0).stop 2024-03-15T13:08:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f6703a19740 1 -- 172.21.15.2:0/1859161204 shutdown_connections 2024-03-15T13:08:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f6703a19740 1 --2- 172.21.15.2:0/1859161204 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55aa8429b580 0x55aa8429d9a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f6703a19740 1 --2- 172.21.15.2:0/1859161204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa8422f7b0 0x55aa842306d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f6703a19740 1 --2- 172.21.15.2:0/1859161204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa84226bd0 0x55aa84230190 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f66e1ffb640 0 client.18393 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:30.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f6703a19740 1 --2- 172.21.15.2:0/1859161204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa842262b0 0x55aa8422fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f6703a19740 1 -- 172.21.15.2:0/1859161204 >> 172.21.15.2:0/1859161204 conn(0x55aa84220990 msgr2=0x55aa8422ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:30.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f6703a19740 1 -- 172.21.15.2:0/1859161204 shutdown_connections 2024-03-15T13:08:30.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.859+0000 7f6703a19740 1 -- 172.21.15.2:0/1859161204 wait complete. 2024-03-15T13:08:30.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.860+0000 7f6703a19740 20 client.18393 trim_cache size 0 max 16384 2024-03-15T13:08:30.850 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:08:30.850 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:08:30.850 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:08:30.851 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:08:30.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.860+0000 7f6700d5e640 1 -- 172.21.15.2:0/1995196312 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f66e80a1bd0 msgr2=0x7f66e80a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:30.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.860+0000 7f6700d5e640 1 --2- 172.21.15.2:0/1995196312 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f66e80a1bd0 0x7f66e80a4090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f66f40cb620 tx=0x7f66f409e040 comp rx=0 tx=0).stop 2024-03-15T13:08:30.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.860+0000 7f6700d5e640 1 -- 172.21.15.2:0/1995196312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66fc102af0 msgr2=0x7f66fc1a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:30.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.860+0000 7f6700d5e640 1 --2- 172.21.15.2:0/1995196312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66fc102af0 0x7f66fc1a5cc0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f66f400eb10 tx=0x7f66f400c2d0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.860+0000 7f6700d5e640 1 -- 172.21.15.2:0/1995196312 shutdown_connections 2024-03-15T13:08:30.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.860+0000 7f6700d5e640 1 --2- 172.21.15.2:0/1995196312 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f66e80a1bd0 0x7f66e80a4090 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.860+0000 7f6700d5e640 1 --2- 172.21.15.2:0/1995196312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66fc102af0 0x7f66fc1a5cc0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.860+0000 7f6700d5e640 1 --2- 172.21.15.2:0/1995196312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc1056b0 0x7f66fc1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.860+0000 7f6700d5e640 1 --2- 172.21.15.2:0/1995196312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66fc1005e0 0x7f66fc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.860+0000 7f6700d5e640 1 -- 172.21.15.2:0/1995196312 >> 172.21.15.2:0/1995196312 conn(0x7f66fc0775a0 msgr2=0x7f66fc068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:30.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.860+0000 7f6700d5e640 1 -- 172.21.15.2:0/1995196312 shutdown_connections 2024-03-15T13:08:30.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.860+0000 7f6700d5e640 1 -- 172.21.15.2:0/1995196312 wait complete. 2024-03-15T13:08:30.855 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:08:30.856 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:08:30.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.974+0000 7f9a325a1640 1 -- 172.21.15.2:0/1751576514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a2c103080 msgr2=0x7f9a2c10b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:30.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.974+0000 7f9a325a1640 1 --2- 172.21.15.2:0/1751576514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a2c103080 0x7f9a2c10b560 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009900 tx=0x7f9a2002bc30 comp rx=0 tx=0).stop 2024-03-15T13:08:30.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.974+0000 7f9a325a1640 1 -- 172.21.15.2:0/1751576514 shutdown_connections 2024-03-15T13:08:30.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.974+0000 7f9a325a1640 1 --2- 172.21.15.2:0/1751576514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a2c10bb30 0x7f9a2c10dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.974+0000 7f9a325a1640 1 --2- 172.21.15.2:0/1751576514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a2c103080 0x7f9a2c10b560 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.974+0000 7f9a325a1640 1 --2- 172.21.15.2:0/1751576514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a2c102760 0x7f9a2c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.974+0000 7f9a325a1640 1 -- 172.21.15.2:0/1751576514 >> 172.21.15.2:0/1751576514 conn(0x7f9a2c0774f0 msgr2=0x7f9a2c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:30.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.974+0000 7f9a325a1640 1 -- 172.21.15.2:0/1751576514 shutdown_connections 2024-03-15T13:08:30.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.974+0000 7f9a325a1640 1 -- 172.21.15.2:0/1751576514 wait complete. 2024-03-15T13:08:30.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.975+0000 7f9a325a1640 1 Processor -- start 2024-03-15T13:08:30.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.975+0000 7f9a325a1640 1 -- start start 2024-03-15T13:08:30.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.975+0000 7f9a325a1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a2c102760 0x7f9a2c19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.975+0000 7f9a325a1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a2c103080 0x7f9a2c19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.975+0000 7f9a325a1640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a2c10bb30 0x7f9a2c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.975+0000 7f9a325a1640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c198010 con 0x7f9a2c103080 2024-03-15T13:08:30.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.975+0000 7f9a325a1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c198180 con 0x7f9a2c102760 2024-03-15T13:08:30.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.975+0000 7f9a325a1640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c198340 con 0x7f9a2c10bb30 2024-03-15T13:08:30.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.975+0000 7f9a2b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a2c103080 0x7f9a2c19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.975+0000 7f9a2b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a2c103080 0x7f9a2c19dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57400/0 (socket says 172.21.15.2:57400) 2024-03-15T13:08:30.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.975+0000 7f9a2b7fe640 1 -- 172.21.15.2:0/2025951463 learned_addr learned my addr 172.21.15.2:0/2025951463 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:30.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.975+0000 7f9a2bfff640 1 --2- 172.21.15.2:0/2025951463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a2c102760 0x7f9a2c19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.975+0000 7f9a30b3e640 1 --2- 172.21.15.2:0/2025951463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a2c10bb30 0x7f9a2c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.976+0000 7f9a2b7fe640 1 -- 172.21.15.2:0/2025951463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a2c10bb30 msgr2=0x7f9a2c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:30.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.976+0000 7f9a2b7fe640 1 --2- 172.21.15.2:0/2025951463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a2c10bb30 0x7f9a2c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.976+0000 7f9a2b7fe640 1 -- 172.21.15.2:0/2025951463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a2c102760 msgr2=0x7f9a2c19d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:30.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.976+0000 7f9a2b7fe640 1 --2- 172.21.15.2:0/2025951463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a2c102760 0x7f9a2c19d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:30.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.976+0000 7f9a2b7fe640 1 -- 172.21.15.2:0/2025951463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a20009560 con 0x7f9a2c103080 2024-03-15T13:08:30.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.976+0000 7f9a2b7fe640 1 --2- 172.21.15.2:0/2025951463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a2c103080 0x7f9a2c19dd50 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009230 tx=0x7f9a2002cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:30.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.976+0000 7f9a297fa640 1 -- 172.21.15.2:0/2025951463 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a20004250 con 0x7f9a2c103080 2024-03-15T13:08:30.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.976+0000 7f9a325a1640 1 -- 172.21.15.2:0/2025951463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2c1985c0 con 0x7f9a2c103080 2024-03-15T13:08:30.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.976+0000 7f9a297fa640 1 -- 172.21.15.2:0/2025951463 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9a200043b0 con 0x7f9a2c103080 2024-03-15T13:08:30.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.976+0000 7f9a297fa640 1 -- 172.21.15.2:0/2025951463 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a20003bf0 con 0x7f9a2c103080 2024-03-15T13:08:30.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.976+0000 7f9a325a1640 1 -- 172.21.15.2:0/2025951463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2c1a48b0 con 0x7f9a2c103080 2024-03-15T13:08:30.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.977+0000 7f9a325a1640 1 -- 172.21.15.2:0/2025951463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a2c0682e0 con 0x7f9a2c103080 2024-03-15T13:08:30.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.978+0000 7f9a297fa640 1 -- 172.21.15.2:0/2025951463 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f9a20003d50 con 0x7f9a2c103080 2024-03-15T13:08:30.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.979+0000 7f9a297fa640 1 --2- 172.21.15.2:0/2025951463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9a140a1ba0 0x7f9a140a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:30.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.979+0000 7f9a2bfff640 1 --2- 172.21.15.2:0/2025951463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9a140a1ba0 0x7f9a140a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:30.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.979+0000 7f9a297fa640 1 -- 172.21.15.2:0/2025951463 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f9a200efa90 con 0x7f9a2c103080 2024-03-15T13:08:30.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.979+0000 7f9a2bfff640 1 --2- 172.21.15.2:0/2025951463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9a140a1ba0 0x7f9a140a4060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c198b00 tx=0x7f9a1c005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:30.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:30.983+0000 7f9a297fa640 1 -- 172.21.15.2:0/2025951463 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9a200b86d0 con 0x7f9a2c103080 2024-03-15T13:08:31.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.203+0000 7f9a325a1640 1 -- 172.21.15.2:0/2025951463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a2c106c20 con 0x7f9a2c103080 2024-03-15T13:08:31.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.203+0000 7f9a297fa640 1 -- 172.21.15.2:0/2025951463 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+11453 (secure 0 0 0) 0x7f9a200bbd90 con 0x7f9a2c103080 2024-03-15T13:08:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:08:31.188 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:05:12.138845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24500,24464,24460,24434,24539]},"id":1}]} 2024-03-15T13:08:31.189 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 50 2024-03-15T13:08:31.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.205+0000 7f9a325a1640 1 -- 172.21.15.2:0/2025951463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9a140a1ba0 msgr2=0x7f9a140a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.205+0000 7f9a325a1640 1 --2- 172.21.15.2:0/2025951463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9a140a1ba0 0x7f9a140a4060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c198b00 tx=0x7f9a1c005f90 comp rx=0 tx=0).stop 2024-03-15T13:08:31.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.205+0000 7f9a325a1640 1 -- 172.21.15.2:0/2025951463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a2c103080 msgr2=0x7f9a2c19dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.205+0000 7f9a325a1640 1 --2- 172.21.15.2:0/2025951463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a2c103080 0x7f9a2c19dd50 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009230 tx=0x7f9a2002cac0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.205+0000 7f9a325a1640 1 -- 172.21.15.2:0/2025951463 shutdown_connections 2024-03-15T13:08:31.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.205+0000 7f9a325a1640 1 --2- 172.21.15.2:0/2025951463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a2c10bb30 0x7f9a2c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.205+0000 7f9a325a1640 1 --2- 172.21.15.2:0/2025951463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9a140a1ba0 0x7f9a140a4060 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.205+0000 7f9a325a1640 1 --2- 172.21.15.2:0/2025951463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a2c103080 0x7f9a2c19dd50 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.205+0000 7f9a325a1640 1 --2- 172.21.15.2:0/2025951463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a2c102760 0x7f9a2c19d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.205+0000 7f9a325a1640 1 -- 172.21.15.2:0/2025951463 >> 172.21.15.2:0/2025951463 conn(0x7f9a2c0774f0 msgr2=0x7f9a2c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.206+0000 7f9a325a1640 1 -- 172.21.15.2:0/2025951463 shutdown_connections 2024-03-15T13:08:31.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.206+0000 7f9a325a1640 1 -- 172.21.15.2:0/2025951463 wait complete. 2024-03-15T13:08:31.201 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:08:31.201 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:08:31.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.321+0000 7fbbc4325640 1 -- 172.21.15.2:0/1452886645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbbc103050 msgr2=0x7fbbbc10b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.321+0000 7fbbc4325640 1 --2- 172.21.15.2:0/1452886645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbbc103050 0x7fbbbc10b530 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009900 tx=0x7fbbac02bcd0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.321+0000 7fbbc4325640 1 -- 172.21.15.2:0/1452886645 shutdown_connections 2024-03-15T13:08:31.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.321+0000 7fbbc4325640 1 --2- 172.21.15.2:0/1452886645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbbc10bb00 0x7fbbbc10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.321+0000 7fbbc4325640 1 --2- 172.21.15.2:0/1452886645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbbc103050 0x7fbbbc10b530 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.321+0000 7fbbc4325640 1 --2- 172.21.15.2:0/1452886645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbbc102730 0x7fbbbc102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.321+0000 7fbbc4325640 1 -- 172.21.15.2:0/1452886645 >> 172.21.15.2:0/1452886645 conn(0x7fbbbc076770 msgr2=0x7fbbbc076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.321+0000 7fbbc4325640 1 -- 172.21.15.2:0/1452886645 shutdown_connections 2024-03-15T13:08:31.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.321+0000 7fbbc4325640 1 -- 172.21.15.2:0/1452886645 wait complete. 2024-03-15T13:08:31.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc4325640 1 Processor -- start 2024-03-15T13:08:31.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc4325640 1 -- start start 2024-03-15T13:08:31.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc4325640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbbc102730 0x7fbbbc19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc4325640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbbc103050 0x7fbbbc19dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc4325640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbbc10bb00 0x7fbbbc1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc4325640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc197ff0 con 0x7fbbbc103050 2024-03-15T13:08:31.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc4325640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc198160 con 0x7fbbbc10bb00 2024-03-15T13:08:31.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc4325640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc198320 con 0x7fbbbc102730 2024-03-15T13:08:31.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc18c0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbbc103050 0x7fbbbc19dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc18c0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbbc103050 0x7fbbbc19dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57428/0 (socket says 172.21.15.2:57428) 2024-03-15T13:08:31.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc18c0640 1 -- 172.21.15.2:0/3667124611 learned_addr learned my addr 172.21.15.2:0/3667124611 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc28c2640 1 --2- 172.21.15.2:0/3667124611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbbc10bb00 0x7fbbbc1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc20c1640 1 --2- 172.21.15.2:0/3667124611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbbc102730 0x7fbbbc19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc18c0640 1 -- 172.21.15.2:0/3667124611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbbc102730 msgr2=0x7fbbbc19d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc18c0640 1 --2- 172.21.15.2:0/3667124611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbbc102730 0x7fbbbc19d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc18c0640 1 -- 172.21.15.2:0/3667124611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbbc10bb00 msgr2=0x7fbbbc1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc18c0640 1 --2- 172.21.15.2:0/3667124611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbbc10bb00 0x7fbbbc1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc18c0640 1 -- 172.21.15.2:0/3667124611 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac009560 con 0x7fbbbc103050 2024-03-15T13:08:31.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbc18c0640 1 --2- 172.21.15.2:0/3667124611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbbc103050 0x7fbbbc19dd30 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009230 tx=0x7fbbac02cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.322+0000 7fbbb37fe640 1 -- 172.21.15.2:0/3667124611 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac004250 con 0x7fbbbc103050 2024-03-15T13:08:31.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.323+0000 7fbbb37fe640 1 -- 172.21.15.2:0/3667124611 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbbac0043b0 con 0x7fbbbc103050 2024-03-15T13:08:31.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.323+0000 7fbbc4325640 1 -- 172.21.15.2:0/3667124611 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbbc1985a0 con 0x7fbbbc103050 2024-03-15T13:08:31.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.323+0000 7fbbb37fe640 1 -- 172.21.15.2:0/3667124611 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac003bf0 con 0x7fbbbc103050 2024-03-15T13:08:31.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.323+0000 7fbbc4325640 1 -- 172.21.15.2:0/3667124611 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbbc1a4890 con 0x7fbbbc103050 2024-03-15T13:08:31.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.325+0000 7fbbb37fe640 1 -- 172.21.15.2:0/3667124611 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fbbac003d50 con 0x7fbbbc103050 2024-03-15T13:08:31.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.325+0000 7fbbc6fd8740 1 Processor -- start 2024-03-15T13:08:31.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.325+0000 7fbbc6fd8740 1 -- start start 2024-03-15T13:08:31.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.325+0000 7fbbb37fe640 1 --2- 172.21.15.2:0/3667124611 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbba80a1ba0 0x7fbba80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc6fd8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea19992e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc6fd8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea1999820 0x563ea19a6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc6fd8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea19a6b60 0x563ea19a8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc6fd8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea1954960 con 0x563ea19a6b60 2024-03-15T13:08:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc6fd8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea1908310 con 0x563ea1998f00 2024-03-15T13:08:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc6fd8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea1905840 con 0x563ea1999820 2024-03-15T13:08:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc28c2640 1 --2- 172.21.15.2:0/3667124611 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbba80a1ba0 0x7fbba80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbb37fe640 1 -- 172.21.15.2:0/3667124611 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fbbac0f0110 con 0x7fbbbc103050 2024-03-15T13:08:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc20c1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea19992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc28c2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea19a6b60 0x563ea19a8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc18c0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea1999820 0x563ea19a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc20c1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea19992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:43532/0 (socket says 172.21.15.2:43532) 2024-03-15T13:08:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc20c1640 1 -- 172.21.15.2:0/25707735 learned_addr learned my addr 172.21.15.2:0/25707735 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc20c1640 1 -- 172.21.15.2:0/25707735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea1999820 msgr2=0x563ea19a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc20c1640 1 --2- 172.21.15.2:0/25707735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea1999820 0x563ea19a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc20c1640 1 -- 172.21.15.2:0/25707735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea19a6b60 msgr2=0x563ea19a8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc20c1640 1 --2- 172.21.15.2:0/25707735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea19a6b60 0x563ea19a8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc20c1640 1 -- 172.21.15.2:0/25707735 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ea18c3060 con 0x563ea1998f00 2024-03-15T13:08:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc28c2640 1 --2- 172.21.15.2:0/3667124611 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbba80a1ba0 0x7fbba80a4060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc06aba0 tx=0x7fbbb4009e90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbc20c1640 1 --2- 172.21.15.2:0/25707735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea19992e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009ca0 tx=0x7fbbb8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f19c8648eff9f612 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbb0bfe640 1 -- 172.21.15.2:0/25707735 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8009e50 con 0x563ea1998f00 2024-03-15T13:08:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbb0bfe640 1 -- 172.21.15.2:0/25707735 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbbb80054f0 con 0x563ea1998f00 2024-03-15T13:08:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.326+0000 7fbbb0bfe640 1 -- 172.21.15.2:0/25707735 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8019670 con 0x563ea1998f00 2024-03-15T13:08:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.327+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/25707735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 msgr2=0x563ea19992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.327+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/25707735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea19992e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009ca0 tx=0x7fbbb8011810 comp rx=0 tx=0).stop 2024-03-15T13:08:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.327+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/25707735 shutdown_connections 2024-03-15T13:08:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.327+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/25707735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea19a6b60 0x563ea19a8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.327+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/25707735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea1999820 0x563ea19a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.327+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/25707735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea19992e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.327+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/25707735 >> 172.21.15.2:0/25707735 conn(0x563ea1993980 msgr2=0x563ea1993d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.327+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/25707735 shutdown_connections 2024-03-15T13:08:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.327+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/25707735 wait complete. 2024-03-15T13:08:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc6fd8740 1 -- start start 2024-03-15T13:08:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc6fd8740 1 Processor -- start 2024-03-15T13:08:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc6fd8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea19a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc6fd8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea1999820 0x563ea19a3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc6fd8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea19a6b60 0x563ea19a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc6fd8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea1901020 con 0x563ea1999820 2024-03-15T13:08:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc6fd8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea15d4430 con 0x563ea1998f00 2024-03-15T13:08:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc6fd8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea1590e00 con 0x563ea19a6b60 2024-03-15T13:08:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc6fd8740 10 client.0 _ll_register_callbacks cb 0x563ea18a8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc18c0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea1999820 0x563ea19a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc20c1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea19a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc20c1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea19a63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:43536/0 (socket says 172.21.15.2:43536) 2024-03-15T13:08:31.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc20c1640 1 -- 172.21.15.2:0/1357154287 learned_addr learned my addr 172.21.15.2:0/1357154287 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc28c2640 1 --2- 172.21.15.2:0/1357154287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea19a6b60 0x563ea19a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc18c0640 1 -- 172.21.15.2:0/1357154287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea19a6b60 msgr2=0x563ea19a3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc18c0640 1 --2- 172.21.15.2:0/1357154287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea19a6b60 0x563ea19a3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc18c0640 1 -- 172.21.15.2:0/1357154287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 msgr2=0x563ea19a63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc18c0640 1 --2- 172.21.15.2:0/1357154287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea19a63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc18c0640 1 -- 172.21.15.2:0/1357154287 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb80098d0 con 0x563ea1999820 2024-03-15T13:08:31.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbbc18c0640 1 --2- 172.21.15.2:0/1357154287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea1999820 0x563ea19a3760 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0ed5d0 tx=0x7fbbac0bc6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f50b819db8ae74a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.328+0000 7fbba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:31.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.329+0000 7fbba5ffb640 1 -- 172.21.15.2:0/1357154287 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0bcab0 con 0x563ea1999820 2024-03-15T13:08:31.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.329+0000 7fbba5ffb640 1 -- 172.21.15.2:0/1357154287 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fbbac0c0c00 con 0x563ea1999820 2024-03-15T13:08:31.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.329+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/1357154287 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ea16007e0 con 0x563ea1999820 2024-03-15T13:08:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.329+0000 7fbba5ffb640 1 -- 172.21.15.2:0/1357154287 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbac0c5250 con 0x563ea1999820 2024-03-15T13:08:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.329+0000 7fbbc6fd8740 10 client.18411 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.329+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/1357154287 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x563ea18ff6a0 con 0x563ea1999820 2024-03-15T13:08:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.329+0000 7fbba5ffb640 1 -- 172.21.15.2:0/1357154287 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x563ea16007e0 con 0x563ea1999820 2024-03-15T13:08:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.329+0000 7fbba5ffb640 1 -- 172.21.15.2:0/1357154287 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbbac03c070 con 0x563ea1999820 2024-03-15T13:08:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.329+0000 7fbbc6fd8740 10 client.18411 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.329+0000 7fbbc6fd8740 10 client.18411 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:08:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.329+0000 7fbbc6fd8740 20 client.18411 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.329+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/1357154287 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563ea19b6920 0x563ea19b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.329+0000 7fbbc6fd8740 4 client.18411 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:08:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.329+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/1357154287 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ea163d720 con 0x563ea19b6920 2024-03-15T13:08:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.329+0000 7fbbc20c1640 1 --2- 172.21.15.2:0/1357154287 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563ea19b6920 0x563ea19b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.330+0000 7fbba5ffb640 10 client.18411 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:08:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.330+0000 7fbbc20c1640 1 --2- 172.21.15.2:0/1357154287 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563ea19b6920 0x563ea19b8d30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e144037eeb767744 server_cookie=3cf9ceeaf518d2c6 in_seq=0 out_seq=0 2024-03-15T13:08:31.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.330+0000 7fbba5ffb640 1 -- 172.21.15.2:0/1357154287 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563ea163d720 con 0x563ea19b6920 2024-03-15T13:08:31.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.330+0000 7fbba5ffb640 10 client.18411 handle_command_reply: tid=0 2024-03-15T13:08:31.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.330+0000 7fbbc6fd8740 1 client.18411 shutdown 2024-03-15T13:08:31.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.330+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/1357154287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea1999820 msgr2=0x563ea19a3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.330+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/1357154287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea1999820 0x563ea19a3760 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0ed5d0 tx=0x7fbbac0bc6b0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.330+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/1357154287 shutdown_connections 2024-03-15T13:08:31.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.330+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/1357154287 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563ea19b6920 0x563ea19b8d30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.330+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/1357154287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea19a6b60 0x563ea19a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.330+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/1357154287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea1999820 0x563ea19a3760 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.330+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/1357154287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea19a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.330+0000 7fbba5ffb640 0 client.18411 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:08:31.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.330+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/1357154287 >> 172.21.15.2:0/1357154287 conn(0x563ea1993980 msgr2=0x563ea19956d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.331+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/1357154287 shutdown_connections 2024-03-15T13:08:31.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.331+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/1357154287 wait complete. 2024-03-15T13:08:31.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.331+0000 7fbbc6fd8740 20 client.18411 trim_cache size 0 max 16384 2024-03-15T13:08:31.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.343+0000 7fbbc6fd8740 1 Processor -- start 2024-03-15T13:08:31.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.343+0000 7fbbc6fd8740 1 -- start start 2024-03-15T13:08:31.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.343+0000 7fbbc6fd8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea1992dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.343+0000 7fbbc6fd8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea1999820 0x563ea19a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.343+0000 7fbbc6fd8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea19a6b60 0x563ea19a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.343+0000 7fbbc6fd8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea1886130 con 0x563ea19a6b60 2024-03-15T13:08:31.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.343+0000 7fbbc6fd8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea198f2b0 con 0x563ea1998f00 2024-03-15T13:08:31.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.343+0000 7fbbc6fd8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea19b6730 con 0x563ea1999820 2024-03-15T13:08:31.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.343+0000 7fbbc28c2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea19a6b60 0x563ea19a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.343+0000 7fbbc20c1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea1992dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.343+0000 7fbbc28c2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea19a6b60 0x563ea19a63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57460/0 (socket says 172.21.15.2:57460) 2024-03-15T13:08:31.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.343+0000 7fbbc28c2640 1 -- 172.21.15.2:0/1209540226 learned_addr learned my addr 172.21.15.2:0/1209540226 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.343+0000 7fbbc18c0640 1 --2- 172.21.15.2:0/1209540226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea1999820 0x563ea19a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbbc28c2640 1 -- 172.21.15.2:0/1209540226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea1999820 msgr2=0x563ea19a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbbc28c2640 1 --2- 172.21.15.2:0/1209540226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea1999820 0x563ea19a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbbc28c2640 1 -- 172.21.15.2:0/1209540226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 msgr2=0x563ea1992dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbbc28c2640 1 --2- 172.21.15.2:0/1209540226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea1992dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbbc28c2640 1 -- 172.21.15.2:0/1209540226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ea189bc20 con 0x563ea19a6b60 2024-03-15T13:08:31.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbbc28c2640 1 --2- 172.21.15.2:0/1209540226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea19a6b60 0x563ea19a63d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4011820 tx=0x7fbbb4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb5d75ff9ca26b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbba7fff640 1 -- 172.21.15.2:0/1209540226 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4016050 con 0x563ea19a6b60 2024-03-15T13:08:31.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbba7fff640 1 -- 172.21.15.2:0/1209540226 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbbb4004500 con 0x563ea19a6b60 2024-03-15T13:08:31.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbba7fff640 1 -- 172.21.15.2:0/1209540226 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb401fb20 con 0x563ea19a6b60 2024-03-15T13:08:31.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/1209540226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea19a6b60 msgr2=0x563ea19a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/1209540226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea19a6b60 0x563ea19a63d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4011820 tx=0x7fbbb4021000 comp rx=0 tx=0).stop 2024-03-15T13:08:31.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/1209540226 shutdown_connections 2024-03-15T13:08:31.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/1209540226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea19a6b60 0x563ea19a63d0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/1209540226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea1999820 0x563ea19a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/1209540226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea1992dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/1209540226 >> 172.21.15.2:0/1209540226 conn(0x563ea1993980 msgr2=0x563ea19a3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/1209540226 shutdown_connections 2024-03-15T13:08:31.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.344+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/1209540226 wait complete. 2024-03-15T13:08:31.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc6fd8740 1 -- start start 2024-03-15T13:08:31.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc6fd8740 1 Processor -- start 2024-03-15T13:08:31.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc6fd8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea19ae550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc6fd8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea1999820 0x563ea19aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc6fd8740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea19a6b60 0x563ea19aae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc6fd8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea19aee20 con 0x563ea1999820 2024-03-15T13:08:31.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc6fd8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea19ab350 con 0x563ea1998f00 2024-03-15T13:08:31.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc6fd8740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea19ab510 con 0x563ea19a6b60 2024-03-15T13:08:31.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc6fd8740 10 client.0 _ll_register_callbacks cb 0x7fbbac0431e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc18c0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea1999820 0x563ea19aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc18c0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea1999820 0x563ea19aa8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57470/0 (socket says 172.21.15.2:57470) 2024-03-15T13:08:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc18c0640 1 -- 172.21.15.2:0/174106442 learned_addr learned my addr 172.21.15.2:0/174106442 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc28c2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea19a6b60 0x563ea19aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc20c1640 1 --2- 172.21.15.2:0/174106442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea19ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc18c0640 1 -- 172.21.15.2:0/174106442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea19a6b60 msgr2=0x563ea19aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc18c0640 1 --2- 172.21.15.2:0/174106442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea19a6b60 0x563ea19aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc18c0640 1 -- 172.21.15.2:0/174106442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 msgr2=0x563ea19ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc18c0640 1 --2- 172.21.15.2:0/174106442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea19ae550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc18c0640 1 -- 172.21.15.2:0/174106442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb4019040 con 0x563ea1999820 2024-03-15T13:08:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbbc18c0640 1 --2- 172.21.15.2:0/174106442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea1999820 0x563ea19aa8d0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0b9430 tx=0x7fbbac0bc720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415a46ec9664cf7f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.345+0000 7fbba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:31.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.346+0000 7fbba5ffb640 1 -- 172.21.15.2:0/174106442 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac04db80 con 0x563ea1999820 2024-03-15T13:08:31.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.346+0000 7fbba5ffb640 1 -- 172.21.15.2:0/174106442 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fbbac0c0c00 con 0x563ea1999820 2024-03-15T13:08:31.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.346+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/174106442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbb80054f0 con 0x563ea1999820 2024-03-15T13:08:31.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.346+0000 7fbba5ffb640 1 -- 172.21.15.2:0/174106442 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbac09f070 con 0x563ea1999820 2024-03-15T13:08:31.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.346+0000 7fbbc6fd8740 10 client.18417 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:31.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.346+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/174106442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x563ea18c18c0 con 0x563ea1999820 2024-03-15T13:08:31.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.346+0000 7fbba5ffb640 1 -- 172.21.15.2:0/174106442 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbbb80054f0 con 0x563ea1999820 2024-03-15T13:08:31.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.346+0000 7fbba5ffb640 1 -- 172.21.15.2:0/174106442 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fbbac03c040 con 0x563ea1999820 2024-03-15T13:08:31.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.346+0000 7fbbc6fd8740 10 client.18417 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:31.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.346+0000 7fbbc6fd8740 10 client.18417 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:08:31.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.346+0000 7fbbc6fd8740 20 client.18417 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:31.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.346+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/174106442 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563ea1a13080 0x563ea1a154a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.346+0000 7fbbc6fd8740 4 client.18417 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:08:31.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.346+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/174106442 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563ea168c890 con 0x563ea1a13080 2024-03-15T13:08:31.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.346+0000 7fbbc20c1640 1 --2- 172.21.15.2:0/174106442 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563ea1a13080 0x563ea1a154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbba5ffb640 10 client.18417 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:08:31.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbbc20c1640 1 --2- 172.21.15.2:0/174106442 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563ea1a13080 0x563ea1a154a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ad5ecc029fceff3f server_cookie=1f8dd0b930cda424 in_seq=0 out_seq=0 2024-03-15T13:08:31.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbba5ffb640 1 -- 172.21.15.2:0/174106442 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563ea168c890 con 0x563ea1a13080 2024-03-15T13:08:31.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbba5ffb640 10 client.18417 handle_command_reply: tid=0 2024-03-15T13:08:31.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbbc6fd8740 1 client.18417 shutdown 2024-03-15T13:08:31.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/174106442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea1999820 msgr2=0x563ea19aa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/174106442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea1999820 0x563ea19aa8d0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0b9430 tx=0x7fbbac0bc720 comp rx=0 tx=0).stop 2024-03-15T13:08:31.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/174106442 shutdown_connections 2024-03-15T13:08:31.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/174106442 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x563ea1a13080 0x563ea1a154a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/174106442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ea19a6b60 0x563ea19aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/174106442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563ea1999820 0x563ea19aa8d0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbba5ffb640 0 client.18417 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:08:31.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbbc6fd8740 1 --2- 172.21.15.2:0/174106442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ea1998f00 0x563ea19ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/174106442 >> 172.21.15.2:0/174106442 conn(0x563ea1993980 msgr2=0x563ea19bd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/174106442 shutdown_connections 2024-03-15T13:08:31.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbbc6fd8740 1 -- 172.21.15.2:0/174106442 wait complete. 2024-03-15T13:08:31.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.347+0000 7fbbc6fd8740 20 client.18417 trim_cache size 0 max 16384 2024-03-15T13:08:31.339 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:08:31.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.348+0000 7fbbc4325640 1 -- 172.21.15.2:0/3667124611 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbba80a1ba0 msgr2=0x7fbba80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.348+0000 7fbbc4325640 1 --2- 172.21.15.2:0/3667124611 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbba80a1ba0 0x7fbba80a4060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc06aba0 tx=0x7fbbb4009e90 comp rx=0 tx=0).stop 2024-03-15T13:08:31.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.348+0000 7fbbc4325640 1 -- 172.21.15.2:0/3667124611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbbc103050 msgr2=0x7fbbbc19dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.348+0000 7fbbc4325640 1 --2- 172.21.15.2:0/3667124611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbbc103050 0x7fbbbc19dd30 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009230 tx=0x7fbbac02cb60 comp rx=0 tx=0).stop 2024-03-15T13:08:31.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.348+0000 7fbbc4325640 1 -- 172.21.15.2:0/3667124611 shutdown_connections 2024-03-15T13:08:31.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.348+0000 7fbbc4325640 1 --2- 172.21.15.2:0/3667124611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbbc10bb00 0x7fbbbc1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.348+0000 7fbbc4325640 1 --2- 172.21.15.2:0/3667124611 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbba80a1ba0 0x7fbba80a4060 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.348+0000 7fbbc4325640 1 --2- 172.21.15.2:0/3667124611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbbc103050 0x7fbbbc19dd30 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.348+0000 7fbbc4325640 1 --2- 172.21.15.2:0/3667124611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbbc102730 0x7fbbbc19d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.348+0000 7fbbc4325640 1 -- 172.21.15.2:0/3667124611 >> 172.21.15.2:0/3667124611 conn(0x7fbbbc076770 msgr2=0x7fbbbc0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.348+0000 7fbbc4325640 1 -- 172.21.15.2:0/3667124611 shutdown_connections 2024-03-15T13:08:31.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.348+0000 7fbbc4325640 1 -- 172.21.15.2:0/3667124611 wait complete. 2024-03-15T13:08:31.343 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:08:31.343 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:08:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.463+0000 7f03d7bf4640 1 -- 172.21.15.2:0/539043077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03d0100b40 msgr2=0x7f03d0100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.463+0000 7f03d7bf4640 1 --2- 172.21.15.2:0/539043077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03d0100b40 0x7f03d0100fa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009930 tx=0x7f03c002bc30 comp rx=0 tx=0).stop 2024-03-15T13:08:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.463+0000 7f03d7bf4640 1 -- 172.21.15.2:0/539043077 shutdown_connections 2024-03-15T13:08:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.463+0000 7f03d7bf4640 1 --2- 172.21.15.2:0/539043077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03d01014e0 0x7f03d010d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.463+0000 7f03d7bf4640 1 --2- 172.21.15.2:0/539043077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03d0100b40 0x7f03d0100fa0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.463+0000 7f03d7bf4640 1 --2- 172.21.15.2:0/539043077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d01053a0 0x7f03d0105780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.463+0000 7f03d7bf4640 1 -- 172.21.15.2:0/539043077 >> 172.21.15.2:0/539043077 conn(0x7f03d00769b0 msgr2=0x7f03d00fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.463+0000 7f03d7bf4640 1 -- 172.21.15.2:0/539043077 shutdown_connections 2024-03-15T13:08:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.463+0000 7f03d7bf4640 1 -- 172.21.15.2:0/539043077 wait complete. 2024-03-15T13:08:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d7bf4640 1 Processor -- start 2024-03-15T13:08:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d7bf4640 1 -- start start 2024-03-15T13:08:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d7bf4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0100b40 0x7f03d019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d7bf4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03d01014e0 0x7f03d019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d7bf4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03d01053a0 0x7f03d01968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d7bf4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d0196f00 con 0x7f03d0100b40 2024-03-15T13:08:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d7bf4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d0197070 con 0x7f03d01053a0 2024-03-15T13:08:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d7bf4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d0197230 con 0x7f03d01014e0 2024-03-15T13:08:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d5990640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0100b40 0x7f03d019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d5990640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0100b40 0x7f03d019c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57498/0 (socket says 172.21.15.2:57498) 2024-03-15T13:08:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d5990640 1 -- 172.21.15.2:0/1681139944 learned_addr learned my addr 172.21.15.2:0/1681139944 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d518f640 1 --2- 172.21.15.2:0/1681139944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03d01014e0 0x7f03d019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d6191640 1 --2- 172.21.15.2:0/1681139944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03d01053a0 0x7f03d01968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d5990640 1 -- 172.21.15.2:0/1681139944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03d01014e0 msgr2=0x7f03d019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d5990640 1 --2- 172.21.15.2:0/1681139944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03d01014e0 0x7f03d019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d5990640 1 -- 172.21.15.2:0/1681139944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03d01053a0 msgr2=0x7f03d01968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d5990640 1 --2- 172.21.15.2:0/1681139944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03d01053a0 0x7f03d01968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d5990640 1 -- 172.21.15.2:0/1681139944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c8009840 con 0x7f03d0100b40 2024-03-15T13:08:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.464+0000 7f03d5990640 1 --2- 172.21.15.2:0/1681139944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0100b40 0x7f03d019c700 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00e9f0 tx=0x7f03cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.465+0000 7f03c6ffd640 1 -- 172.21.15.2:0/1681139944 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc009700 con 0x7f03d0100b40 2024-03-15T13:08:31.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.465+0000 7f03d7bf4640 1 -- 172.21.15.2:0/1681139944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c0009560 con 0x7f03d0100b40 2024-03-15T13:08:31.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.465+0000 7f03c6ffd640 1 -- 172.21.15.2:0/1681139944 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f03cc00ccc0 con 0x7f03d0100b40 2024-03-15T13:08:31.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.465+0000 7f03c6ffd640 1 -- 172.21.15.2:0/1681139944 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc018820 con 0x7f03d0100b40 2024-03-15T13:08:31.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.465+0000 7f03d7bf4640 1 -- 172.21.15.2:0/1681139944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d01a35c0 con 0x7f03d0100b40 2024-03-15T13:08:31.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.468+0000 7f03c6ffd640 1 -- 172.21.15.2:0/1681139944 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f03cc010430 con 0x7f03d0100b40 2024-03-15T13:08:31.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.468+0000 7f03da7ce740 1 Processor -- start 2024-03-15T13:08:31.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.468+0000 7f03da7ce740 1 -- start start 2024-03-15T13:08:31.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.468+0000 7f03da7ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860250 0x55c38e860630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.468+0000 7f03da7ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 0x55c38e869210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.468+0000 7f03da7ce740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e869750 0x55c38e86bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.468+0000 7f03da7ce740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e81b960 con 0x55c38e869750 2024-03-15T13:08:31.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.468+0000 7f03da7ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e7cf310 con 0x55c38e860b70 2024-03-15T13:08:31.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.468+0000 7f03da7ce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e7cc840 con 0x55c38e860250 2024-03-15T13:08:31.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.468+0000 7f03d518f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e869750 0x55c38e86bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.468+0000 7f03d5990640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 0x55c38e869210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.468+0000 7f03d518f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e869750 0x55c38e86bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57512/0 (socket says 172.21.15.2:57512) 2024-03-15T13:08:31.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.468+0000 7f03d518f640 1 -- 172.21.15.2:0/4101817494 learned_addr learned my addr 172.21.15.2:0/4101817494 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.468+0000 7f03d6191640 1 --2- 172.21.15.2:0/4101817494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860250 0x55c38e860630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.468+0000 7f03c6ffd640 1 --2- 172.21.15.2:0/1681139944 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f03bc0a1ba0 0x7f03bc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.469+0000 7f03d6191640 1 --2- 172.21.15.2:0/1681139944 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f03bc0a1ba0 0x7f03bc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.469+0000 7f03d518f640 1 -- 172.21.15.2:0/4101817494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860250 msgr2=0x55c38e860630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.469+0000 7f03d518f640 1 --2- 172.21.15.2:0/4101817494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860250 0x55c38e860630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.469+0000 7f03d518f640 1 -- 172.21.15.2:0/4101817494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 msgr2=0x55c38e869210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.469+0000 7f03d518f640 1 --2- 172.21.15.2:0/4101817494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 0x55c38e869210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.469+0000 7f03d518f640 1 -- 172.21.15.2:0/4101817494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c38e78a060 con 0x55c38e869750 2024-03-15T13:08:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.469+0000 7f03d5990640 1 --2- 172.21.15.2:0/4101817494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 0x55c38e869210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.469+0000 7f03c6ffd640 1 -- 172.21.15.2:0/1681139944 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f03cc014070 con 0x7f03d0100b40 2024-03-15T13:08:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.469+0000 7f03d518f640 1 --2- 172.21.15.2:0/4101817494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e869750 0x55c38e86bb40 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f03c0005ab0 tx=0x7f03c002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663ca5bd96599264 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.469+0000 7f03d6191640 1 --2- 172.21.15.2:0/1681139944 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f03bc0a1ba0 0x7f03bc0a4060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009fd0 tx=0x7f03c8004ab0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.469+0000 7f03bbfff640 1 -- 172.21.15.2:0/4101817494 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0004780 con 0x55c38e869750 2024-03-15T13:08:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.469+0000 7f03bbfff640 1 -- 172.21.15.2:0/4101817494 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f03c00048e0 con 0x55c38e869750 2024-03-15T13:08:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.469+0000 7f03bbfff640 1 -- 172.21.15.2:0/4101817494 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0003850 con 0x55c38e869750 2024-03-15T13:08:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.470+0000 7f03da7ce740 1 -- 172.21.15.2:0/4101817494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e869750 msgr2=0x55c38e86bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.470+0000 7f03da7ce740 1 --2- 172.21.15.2:0/4101817494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e869750 0x55c38e86bb40 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f03c0005ab0 tx=0x7f03c002c5d0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.470+0000 7f03da7ce740 1 -- 172.21.15.2:0/4101817494 shutdown_connections 2024-03-15T13:08:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.470+0000 7f03da7ce740 1 --2- 172.21.15.2:0/4101817494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e869750 0x55c38e86bb40 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.470+0000 7f03da7ce740 1 --2- 172.21.15.2:0/4101817494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 0x55c38e869210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.470+0000 7f03da7ce740 1 --2- 172.21.15.2:0/4101817494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860250 0x55c38e860630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.470+0000 7f03da7ce740 1 -- 172.21.15.2:0/4101817494 >> 172.21.15.2:0/4101817494 conn(0x55c38e85a980 msgr2=0x55c38e85ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.470+0000 7f03da7ce740 1 -- 172.21.15.2:0/4101817494 shutdown_connections 2024-03-15T13:08:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.470+0000 7f03da7ce740 1 -- 172.21.15.2:0/4101817494 wait complete. 2024-03-15T13:08:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.470+0000 7f03da7ce740 1 -- start start 2024-03-15T13:08:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.470+0000 7f03da7ce740 1 Processor -- start 2024-03-15T13:08:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03da7ce740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e860250 0x55c38e8690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03da7ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 0x55c38e866470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03da7ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e869750 0x55c38e8669f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03da7ce740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e7c8020 con 0x55c38e860250 2024-03-15T13:08:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03da7ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e49b430 con 0x55c38e860b70 2024-03-15T13:08:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03da7ce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e457e00 con 0x55c38e869750 2024-03-15T13:08:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03da7ce740 10 client.0 _ll_register_callbacks cb 0x55c38e76fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03d5990640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e860250 0x55c38e8690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03d5990640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e860250 0x55c38e8690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57514/0 (socket says 172.21.15.2:57514) 2024-03-15T13:08:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03d5990640 1 -- 172.21.15.2:0/2253587853 learned_addr learned my addr 172.21.15.2:0/2253587853 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03d6191640 1 --2- 172.21.15.2:0/2253587853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e869750 0x55c38e8669f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03d518f640 1 --2- 172.21.15.2:0/2253587853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 0x55c38e866470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03d5990640 1 -- 172.21.15.2:0/2253587853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e869750 msgr2=0x55c38e8669f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03d5990640 1 --2- 172.21.15.2:0/2253587853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e869750 0x55c38e8669f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03d5990640 1 -- 172.21.15.2:0/2253587853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 msgr2=0x55c38e866470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03d5990640 1 --2- 172.21.15.2:0/2253587853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 0x55c38e866470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03d5990640 1 -- 172.21.15.2:0/2253587853 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d010dbc0 con 0x55c38e860250 2024-03-15T13:08:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03d6191640 1 --2- 172.21.15.2:0/2253587853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e869750 0x55c38e8669f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:08:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03d518f640 1 --2- 172.21.15.2:0/2253587853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 0x55c38e866470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:08:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03d5990640 1 --2- 172.21.15.2:0/2253587853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e860250 0x55c38e8690e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f03cc096c90 tx=0x7f03cc09a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0463c324543a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03b97fa640 1 -- 172.21.15.2:0/2253587853 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc0108e0 con 0x55c38e860250 2024-03-15T13:08:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03b97fa640 1 -- 172.21.15.2:0/2253587853 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f03cc09af30 con 0x55c38e860250 2024-03-15T13:08:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.471+0000 7f03da7ce740 1 -- 172.21.15.2:0/2253587853 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c38e4c77e0 con 0x55c38e860250 2024-03-15T13:08:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.472+0000 7f03b97fa640 1 -- 172.21.15.2:0/2253587853 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03cc0b3320 con 0x55c38e860250 2024-03-15T13:08:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.472+0000 7f03da7ce740 10 client.18429 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:31.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.472+0000 7f03da7ce740 1 -- 172.21.15.2:0/2253587853 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55c38e7c66a0 con 0x55c38e860250 2024-03-15T13:08:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.472+0000 7f03b97fa640 1 -- 172.21.15.2:0/2253587853 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55c38e4c77e0 con 0x55c38e860250 2024-03-15T13:08:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.472+0000 7f03b97fa640 1 -- 172.21.15.2:0/2253587853 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f03cc0b3040 con 0x55c38e860250 2024-03-15T13:08:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.472+0000 7f03da7ce740 10 client.18429 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.472+0000 7f03da7ce740 10 client.18429 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:08:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.472+0000 7f03da7ce740 20 client.18429 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.472+0000 7f03da7ce740 1 --2- 172.21.15.2:0/2253587853 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55c38e8796b0 0x55c38e87baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.472+0000 7f03da7ce740 4 client.18429 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:08:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.472+0000 7f03da7ce740 1 -- 172.21.15.2:0/2253587853 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c38e504720 con 0x55c38e8796b0 2024-03-15T13:08:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.472+0000 7f03d518f640 1 --2- 172.21.15.2:0/2253587853 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55c38e8796b0 0x55c38e87baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.473+0000 7f03b97fa640 10 client.18429 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:08:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.473+0000 7f03d518f640 1 --2- 172.21.15.2:0/2253587853 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55c38e8796b0 0x55c38e87baa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7245b0d131a1ccdf server_cookie=716bf1eb0600eb84 in_seq=0 out_seq=0 2024-03-15T13:08:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.473+0000 7f03b97fa640 1 -- 172.21.15.2:0/2253587853 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c38e504720 con 0x55c38e8796b0 2024-03-15T13:08:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.473+0000 7f03b97fa640 10 client.18429 handle_command_reply: tid=0 2024-03-15T13:08:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.473+0000 7f03da7ce740 1 client.18429 shutdown 2024-03-15T13:08:31.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.473+0000 7f03da7ce740 1 -- 172.21.15.2:0/2253587853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e860250 msgr2=0x55c38e8690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.473+0000 7f03da7ce740 1 --2- 172.21.15.2:0/2253587853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e860250 0x55c38e8690e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f03cc096c90 tx=0x7f03cc09a370 comp rx=0 tx=0).stop 2024-03-15T13:08:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.473+0000 7f03da7ce740 1 -- 172.21.15.2:0/2253587853 shutdown_connections 2024-03-15T13:08:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.473+0000 7f03da7ce740 1 --2- 172.21.15.2:0/2253587853 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55c38e8796b0 0x55c38e87baa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.473+0000 7f03da7ce740 1 --2- 172.21.15.2:0/2253587853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e869750 0x55c38e8669f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.473+0000 7f03b97fa640 0 client.18429 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:08:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.473+0000 7f03da7ce740 1 --2- 172.21.15.2:0/2253587853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 0x55c38e866470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.473+0000 7f03da7ce740 1 --2- 172.21.15.2:0/2253587853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e860250 0x55c38e8690e0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.473+0000 7f03da7ce740 1 -- 172.21.15.2:0/2253587853 >> 172.21.15.2:0/2253587853 conn(0x55c38e85a980 msgr2=0x55c38e85c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.474+0000 7f03da7ce740 1 -- 172.21.15.2:0/2253587853 shutdown_connections 2024-03-15T13:08:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.474+0000 7f03da7ce740 1 -- 172.21.15.2:0/2253587853 wait complete. 2024-03-15T13:08:31.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.474+0000 7f03da7ce740 20 client.18429 trim_cache size 0 max 16384 2024-03-15T13:08:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03da7ce740 1 Processor -- start 2024-03-15T13:08:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03da7ce740 1 -- start start 2024-03-15T13:08:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03da7ce740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e860250 0x55c38e868ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03da7ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860b70 0x55c38e8690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03da7ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e869750 0x55c38e874a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03da7ce740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e74d130 con 0x55c38e860250 2024-03-15T13:08:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03da7ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e85fc90 con 0x55c38e869750 2024-03-15T13:08:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03da7ce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e85f600 con 0x55c38e860b70 2024-03-15T13:08:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03d5990640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e860250 0x55c38e868ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03d5990640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e860250 0x55c38e868ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57520/0 (socket says 172.21.15.2:57520) 2024-03-15T13:08:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03d5990640 1 -- 172.21.15.2:0/3112175475 learned_addr learned my addr 172.21.15.2:0/3112175475 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03d518f640 1 --2- 172.21.15.2:0/3112175475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860b70 0x55c38e8690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03d6191640 1 --2- 172.21.15.2:0/3112175475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e869750 0x55c38e874a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03d5990640 1 -- 172.21.15.2:0/3112175475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860b70 msgr2=0x55c38e8690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03d5990640 1 --2- 172.21.15.2:0/3112175475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860b70 0x55c38e8690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03d5990640 1 -- 172.21.15.2:0/3112175475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e869750 msgr2=0x55c38e874a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03d5990640 1 --2- 172.21.15.2:0/3112175475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e869750 0x55c38e874a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03d5990640 1 -- 172.21.15.2:0/3112175475 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c38e762c20 con 0x55c38e860250 2024-03-15T13:08:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.486+0000 7f03d6191640 1 --2- 172.21.15.2:0/3112175475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e869750 0x55c38e874a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:08:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.487+0000 7f03d5990640 1 --2- 172.21.15.2:0/3112175475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e860250 0x55c38e868ba0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00e9c0 tx=0x7f03cc09a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc947c2efe7bc7ce server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.487+0000 7f03bb7fe640 1 -- 172.21.15.2:0/3112175475 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc09a190 con 0x55c38e860250 2024-03-15T13:08:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.487+0000 7f03bb7fe640 1 -- 172.21.15.2:0/3112175475 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f03cc09af30 con 0x55c38e860250 2024-03-15T13:08:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.487+0000 7f03bb7fe640 1 -- 172.21.15.2:0/3112175475 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc09a830 con 0x55c38e860250 2024-03-15T13:08:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.487+0000 7f03da7ce740 1 -- 172.21.15.2:0/3112175475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e860250 msgr2=0x55c38e868ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.487+0000 7f03da7ce740 1 --2- 172.21.15.2:0/3112175475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e860250 0x55c38e868ba0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00e9c0 tx=0x7f03cc09a4a0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.487+0000 7f03da7ce740 1 -- 172.21.15.2:0/3112175475 shutdown_connections 2024-03-15T13:08:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.487+0000 7f03da7ce740 1 --2- 172.21.15.2:0/3112175475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e869750 0x55c38e874a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.487+0000 7f03da7ce740 1 --2- 172.21.15.2:0/3112175475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860b70 0x55c38e8690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.487+0000 7f03da7ce740 1 --2- 172.21.15.2:0/3112175475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e860250 0x55c38e868ba0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.487+0000 7f03da7ce740 1 -- 172.21.15.2:0/3112175475 >> 172.21.15.2:0/3112175475 conn(0x55c38e85a980 msgr2=0x55c38e859b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.487+0000 7f03da7ce740 1 -- 172.21.15.2:0/3112175475 shutdown_connections 2024-03-15T13:08:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.487+0000 7f03da7ce740 1 -- 172.21.15.2:0/3112175475 wait complete. 2024-03-15T13:08:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03da7ce740 1 -- start start 2024-03-15T13:08:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03da7ce740 1 Processor -- start 2024-03-15T13:08:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03da7ce740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860250 0x55c38e86d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03da7ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 0x55c38e86dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03da7ce740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e869750 0x55c38e8cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03da7ce740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e8cd5a0 con 0x55c38e869750 2024-03-15T13:08:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03da7ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e8cd710 con 0x55c38e860b70 2024-03-15T13:08:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03da7ce740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e8cd8d0 con 0x55c38e860250 2024-03-15T13:08:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03da7ce740 10 client.0 _ll_register_callbacks cb 0x7f03cc010c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03d6191640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e869750 0x55c38e8cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03d6191640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e869750 0x55c38e8cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57530/0 (socket says 172.21.15.2:57530) 2024-03-15T13:08:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03d518f640 1 --2- 172.21.15.2:0/2583845955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 0x55c38e86dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03d5990640 1 --2- 172.21.15.2:0/2583845955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860250 0x55c38e86d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03d6191640 1 -- 172.21.15.2:0/2583845955 learned_addr learned my addr 172.21.15.2:0/2583845955 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03d6191640 1 -- 172.21.15.2:0/2583845955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860250 msgr2=0x55c38e86d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03d6191640 1 --2- 172.21.15.2:0/2583845955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860250 0x55c38e86d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03d6191640 1 -- 172.21.15.2:0/2583845955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 msgr2=0x55c38e86dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03d6191640 1 --2- 172.21.15.2:0/2583845955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 0x55c38e86dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03d6191640 1 -- 172.21.15.2:0/2583845955 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c38e7c66a0 con 0x55c38e869750 2024-03-15T13:08:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03d5990640 1 --2- 172.21.15.2:0/2583845955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860250 0x55c38e86d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.488+0000 7f03d518f640 1 --2- 172.21.15.2:0/2583845955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 0x55c38e86dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03d6191640 1 --2- 172.21.15.2:0/2583845955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e869750 0x55c38e8cced0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f03c800fd80 tx=0x7f03c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bd71fe6618e770 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03b97fa640 1 -- 172.21.15.2:0/2583845955 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c80047d0 con 0x55c38e869750 2024-03-15T13:08:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03b97fa640 1 -- 172.21.15.2:0/2583845955 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f03c8004b20 con 0x55c38e869750 2024-03-15T13:08:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03da7ce740 1 -- 172.21.15.2:0/2583845955 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03c00048e0 con 0x55c38e869750 2024-03-15T13:08:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03b97fa640 1 -- 172.21.15.2:0/2583845955 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03c8003ec0 con 0x55c38e869750 2024-03-15T13:08:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03da7ce740 10 client.18435 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03da7ce740 1 -- 172.21.15.2:0/2583845955 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55c38e7888c0 con 0x55c38e869750 2024-03-15T13:08:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03b97fa640 1 -- 172.21.15.2:0/2583845955 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f03c00048e0 con 0x55c38e869750 2024-03-15T13:08:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03b97fa640 1 -- 172.21.15.2:0/2583845955 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f03c8022440 con 0x55c38e869750 2024-03-15T13:08:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03da7ce740 10 client.18435 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03da7ce740 10 client.18435 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:08:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03da7ce740 20 client.18435 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03da7ce740 1 --2- 172.21.15.2:0/2583845955 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55c38e8d6220 0x55c38e8d8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03da7ce740 4 client.18435 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:08:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.489+0000 7f03da7ce740 1 -- 172.21.15.2:0/2583845955 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c38e553890 con 0x55c38e8d6220 2024-03-15T13:08:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03d5990640 1 --2- 172.21.15.2:0/2583845955 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55c38e8d6220 0x55c38e8d8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03b97fa640 10 client.18435 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:08:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03d5990640 1 --2- 172.21.15.2:0/2583845955 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55c38e8d6220 0x55c38e8d8640 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e44ebfd64af1b22e server_cookie=8185db3d6fe3e708 in_seq=0 out_seq=0 2024-03-15T13:08:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03b97fa640 1 -- 172.21.15.2:0/2583845955 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c38e553890 con 0x55c38e8d6220 2024-03-15T13:08:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03b97fa640 10 client.18435 handle_command_reply: tid=0 2024-03-15T13:08:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03da7ce740 1 client.18435 shutdown 2024-03-15T13:08:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03da7ce740 1 -- 172.21.15.2:0/2583845955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e869750 msgr2=0x55c38e8cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03da7ce740 1 --2- 172.21.15.2:0/2583845955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e869750 0x55c38e8cced0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f03c800fd80 tx=0x7f03c8004a40 comp rx=0 tx=0).stop 2024-03-15T13:08:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03da7ce740 1 -- 172.21.15.2:0/2583845955 shutdown_connections 2024-03-15T13:08:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03da7ce740 1 --2- 172.21.15.2:0/2583845955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c38e869750 0x55c38e8cced0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03da7ce740 1 --2- 172.21.15.2:0/2583845955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c38e860b70 0x55c38e86dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03da7ce740 1 --2- 172.21.15.2:0/2583845955 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55c38e8d6220 0x55c38e8d8640 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03da7ce740 1 --2- 172.21.15.2:0/2583845955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c38e860250 0x55c38e86d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03b97fa640 0 client.18435 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:08:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03da7ce740 1 -- 172.21.15.2:0/2583845955 >> 172.21.15.2:0/2583845955 conn(0x55c38e85a980 msgr2=0x55c38e861090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03da7ce740 1 -- 172.21.15.2:0/2583845955 shutdown_connections 2024-03-15T13:08:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03da7ce740 1 -- 172.21.15.2:0/2583845955 wait complete. 2024-03-15T13:08:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.490+0000 7f03da7ce740 20 client.18435 trim_cache size 0 max 16384 2024-03-15T13:08:31.483 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:08:31.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.491+0000 7f03d7bf4640 1 -- 172.21.15.2:0/1681139944 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f03bc0a1ba0 msgr2=0x7f03bc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.491+0000 7f03d7bf4640 1 --2- 172.21.15.2:0/1681139944 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f03bc0a1ba0 0x7f03bc0a4060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009fd0 tx=0x7f03c8004ab0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.491+0000 7f03d7bf4640 1 -- 172.21.15.2:0/1681139944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0100b40 msgr2=0x7f03d019c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.491+0000 7f03d7bf4640 1 --2- 172.21.15.2:0/1681139944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0100b40 0x7f03d019c700 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00e9f0 tx=0x7f03cc00edf0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.491+0000 7f03d7bf4640 1 -- 172.21.15.2:0/1681139944 shutdown_connections 2024-03-15T13:08:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.491+0000 7f03d7bf4640 1 --2- 172.21.15.2:0/1681139944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03d01053a0 0x7f03d01968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.491+0000 7f03d7bf4640 1 --2- 172.21.15.2:0/1681139944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03d01014e0 0x7f03d019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.491+0000 7f03d7bf4640 1 --2- 172.21.15.2:0/1681139944 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f03bc0a1ba0 0x7f03bc0a4060 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.491+0000 7f03d7bf4640 1 --2- 172.21.15.2:0/1681139944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03d0100b40 0x7f03d019c700 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.491+0000 7f03d7bf4640 1 -- 172.21.15.2:0/1681139944 >> 172.21.15.2:0/1681139944 conn(0x7f03d00769b0 msgr2=0x7f03d0103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.491+0000 7f03d7bf4640 1 -- 172.21.15.2:0/1681139944 shutdown_connections 2024-03-15T13:08:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.491+0000 7f03d7bf4640 1 -- 172.21.15.2:0/1681139944 wait complete. 2024-03-15T13:08:31.486 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:08:31.486 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:08:31.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.607+0000 7fe6cc415640 1 -- 172.21.15.2:0/3035472553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c41074f0 msgr2=0x7fe6c41078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.607+0000 7fe6cc415640 1 --2- 172.21.15.2:0/3035472553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c41074f0 0x7fe6c41078d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009930 tx=0x7fe6bc02bc30 comp rx=0 tx=0).stop 2024-03-15T13:08:31.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.608+0000 7fe6cc415640 1 -- 172.21.15.2:0/3035472553 shutdown_connections 2024-03-15T13:08:31.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.608+0000 7fe6cc415640 1 --2- 172.21.15.2:0/3035472553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6c40690b0 0x7fe6c410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.608+0000 7fe6cc415640 1 --2- 172.21.15.2:0/3035472553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6c40686f0 0x7fe6c4068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.608+0000 7fe6cc415640 1 --2- 172.21.15.2:0/3035472553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c41074f0 0x7fe6c41078d0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.608+0000 7fe6cc415640 1 -- 172.21.15.2:0/3035472553 >> 172.21.15.2:0/3035472553 conn(0x7fe6c40774f0 msgr2=0x7fe6c4077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.608+0000 7fe6cc415640 1 -- 172.21.15.2:0/3035472553 shutdown_connections 2024-03-15T13:08:31.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.608+0000 7fe6cc415640 1 -- 172.21.15.2:0/3035472553 wait complete. 2024-03-15T13:08:31.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.608+0000 7fe6cc415640 1 Processor -- start 2024-03-15T13:08:31.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.608+0000 7fe6cc415640 1 -- start start 2024-03-15T13:08:31.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.608+0000 7fe6cc415640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6c40686f0 0x7fe6c419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.608+0000 7fe6cc415640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6c40690b0 0x7fe6c419dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.608+0000 7fe6cc415640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c41074f0 0x7fe6c41978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.608+0000 7fe6cc415640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c4197f40 con 0x7fe6c41074f0 2024-03-15T13:08:31.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.608+0000 7fe6cc415640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c41980b0 con 0x7fe6c40686f0 2024-03-15T13:08:31.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.608+0000 7fe6cc415640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c4198270 con 0x7fe6c40690b0 2024-03-15T13:08:31.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6ca9b2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c41074f0 0x7fe6c41978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6ca1b1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6c40686f0 0x7fe6c419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6ca9b2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c41074f0 0x7fe6c41978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57546/0 (socket says 172.21.15.2:57546) 2024-03-15T13:08:31.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6ca1b1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6c40686f0 0x7fe6c419d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:43634/0 (socket says 172.21.15.2:43634) 2024-03-15T13:08:31.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6ca9b2640 1 -- 172.21.15.2:0/288853346 learned_addr learned my addr 172.21.15.2:0/288853346 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/288853346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6c40690b0 0x7fe6c419dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6ca9b2640 1 -- 172.21.15.2:0/288853346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6c40690b0 msgr2=0x7fe6c419dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6ca9b2640 1 --2- 172.21.15.2:0/288853346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6c40690b0 0x7fe6c419dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6ca9b2640 1 -- 172.21.15.2:0/288853346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6c40686f0 msgr2=0x7fe6c419d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6ca9b2640 1 --2- 172.21.15.2:0/288853346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6c40686f0 0x7fe6c419d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6ca9b2640 1 -- 172.21.15.2:0/288853346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b4009840 con 0x7fe6c41074f0 2024-03-15T13:08:31.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6ca1b1640 1 --2- 172.21.15.2:0/288853346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6c40686f0 0x7fe6c419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:31.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6ca9b2640 1 --2- 172.21.15.2:0/288853346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c41074f0 0x7fe6c41978f0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400efd0 tx=0x7fe6b400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6c37fe640 1 -- 172.21.15.2:0/288853346 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4009b40 con 0x7fe6c41074f0 2024-03-15T13:08:31.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6c37fe640 1 -- 172.21.15.2:0/288853346 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe6b4003b70 con 0x7fe6c41074f0 2024-03-15T13:08:31.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6cc415640 1 -- 172.21.15.2:0/288853346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6bc009560 con 0x7fe6c41074f0 2024-03-15T13:08:31.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6c37fe640 1 -- 172.21.15.2:0/288853346 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40108f0 con 0x7fe6c41074f0 2024-03-15T13:08:31.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.609+0000 7fe6cc415640 1 -- 172.21.15.2:0/288853346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c41a4600 con 0x7fe6c41074f0 2024-03-15T13:08:31.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.612+0000 7fe6c37fe640 1 -- 172.21.15.2:0/288853346 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fe6b4003ce0 con 0x7fe6c41074f0 2024-03-15T13:08:31.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.612+0000 7fe6ceff4740 1 Processor -- start 2024-03-15T13:08:31.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.612+0000 7fe6ceff4740 1 -- start start 2024-03-15T13:08:31.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6ceff4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b225716f80 0x55b225717360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6ceff4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2257178a0 0x55b225724620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6ceff4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225724b60 0x55b225726f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6ceff4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2256d2960 con 0x55b225716f80 2024-03-15T13:08:31.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6ceff4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b225686310 con 0x55b2257178a0 2024-03-15T13:08:31.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6ceff4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b225683840 con 0x55b225724b60 2024-03-15T13:08:31.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6c99b0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b225716f80 0x55b225717360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6c99b0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b225716f80 0x55b225717360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57550/0 (socket says 172.21.15.2:57550) 2024-03-15T13:08:31.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6ca1b1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2257178a0 0x55b225724620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6c99b0640 1 -- 172.21.15.2:0/1950436246 learned_addr learned my addr 172.21.15.2:0/1950436246 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/1950436246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225724b60 0x55b225726f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6c37fe640 1 --2- 172.21.15.2:0/288853346 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe6ac0a1ba0 0x7fe6ac0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6c99b0640 1 -- 172.21.15.2:0/1950436246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225724b60 msgr2=0x55b225726f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/1950436246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225724b60 0x55b225726f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6c99b0640 1 -- 172.21.15.2:0/1950436246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2257178a0 msgr2=0x55b225724620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/1950436246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2257178a0 0x55b225724620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6c99b0640 1 -- 172.21.15.2:0/1950436246 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b225641060 con 0x55b225716f80 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6ca1b1640 1 --2- 172.21.15.2:0/1950436246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2257178a0 0x55b225724620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6ca9b2640 1 --2- 172.21.15.2:0/288853346 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe6ac0a1ba0 0x7fe6ac0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/1950436246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b225716f80 0x55b225717360 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009fc0 tx=0x7fe6b8009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16edd87c56c08a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.613+0000 7fe6c37fe640 1 -- 172.21.15.2:0/288853346 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe6b4014070 con 0x7fe6c41074f0 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.614+0000 7fe6c0bfe640 1 -- 172.21.15.2:0/1950436246 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8004050 con 0x55b225716f80 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.614+0000 7fe6c0bfe640 1 -- 172.21.15.2:0/1950436246 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe6b8005020 con 0x55b225716f80 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.614+0000 7fe6c0bfe640 1 -- 172.21.15.2:0/1950436246 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8004b00 con 0x55b225716f80 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.614+0000 7fe6ceff4740 1 -- 172.21.15.2:0/1950436246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b225716f80 msgr2=0x55b225717360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.614+0000 7fe6ca9b2640 1 --2- 172.21.15.2:0/288853346 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe6ac0a1ba0 0x7fe6ac0a4060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40cb5d0 tx=0x7fe6b406cc00 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.614+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/1950436246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b225716f80 0x55b225717360 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009fc0 tx=0x7fe6b8009830 comp rx=0 tx=0).stop 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.614+0000 7fe6ceff4740 1 -- 172.21.15.2:0/1950436246 shutdown_connections 2024-03-15T13:08:31.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.614+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/1950436246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225724b60 0x55b225726f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.614+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/1950436246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2257178a0 0x55b225724620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.614+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/1950436246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b225716f80 0x55b225717360 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.614+0000 7fe6ceff4740 1 -- 172.21.15.2:0/1950436246 >> 172.21.15.2:0/1950436246 conn(0x55b225711980 msgr2=0x55b225711d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.614+0000 7fe6ceff4740 1 -- 172.21.15.2:0/1950436246 shutdown_connections 2024-03-15T13:08:31.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.614+0000 7fe6ceff4740 1 -- 172.21.15.2:0/1950436246 wait complete. 2024-03-15T13:08:31.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ceff4740 1 -- start start 2024-03-15T13:08:31.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ceff4740 1 Processor -- start 2024-03-15T13:08:31.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ceff4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225716f80 0x55b2257244e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ceff4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2257178a0 0x55b225721870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ceff4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b225724b60 0x55b225721df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ceff4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b22567f020 con 0x55b2257178a0 2024-03-15T13:08:31.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ceff4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b225352430 con 0x55b225724b60 2024-03-15T13:08:31.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ceff4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b22530ee00 con 0x55b225716f80 2024-03-15T13:08:31.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ceff4740 10 client.0 _ll_register_callbacks cb 0x55b225626cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:31.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ca1b1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2257178a0 0x55b225721870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ca1b1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2257178a0 0x55b225721870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57562/0 (socket says 172.21.15.2:57562) 2024-03-15T13:08:31.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ca1b1640 1 -- 172.21.15.2:0/3223859526 learned_addr learned my addr 172.21.15.2:0/3223859526 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/3223859526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225716f80 0x55b2257244e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/3223859526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b225724b60 0x55b225721df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ca1b1640 1 -- 172.21.15.2:0/3223859526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225716f80 msgr2=0x55b2257244e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ca1b1640 1 --2- 172.21.15.2:0/3223859526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225716f80 0x55b2257244e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ca1b1640 1 -- 172.21.15.2:0/3223859526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b225724b60 msgr2=0x55b225721df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.615+0000 7fe6ca1b1640 1 --2- 172.21.15.2:0/3223859526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b225724b60 0x55b225721df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6ca1b1640 1 -- 172.21.15.2:0/3223859526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b8009d10 con 0x55b2257178a0 2024-03-15T13:08:31.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6ca1b1640 1 --2- 172.21.15.2:0/3223859526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2257178a0 0x55b225721870 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009230 tx=0x7fe6bc004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0feee945dca5b4b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:31.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6a9ffb640 1 -- 172.21.15.2:0/3223859526 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc004310 con 0x55b2257178a0 2024-03-15T13:08:31.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6a9ffb640 1 -- 172.21.15.2:0/3223859526 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe6bc004470 con 0x55b2257178a0 2024-03-15T13:08:31.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6ceff4740 1 -- 172.21.15.2:0/3223859526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b22537e7e0 con 0x55b2257178a0 2024-03-15T13:08:31.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6a9ffb640 1 -- 172.21.15.2:0/3223859526 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6bc0035f0 con 0x55b2257178a0 2024-03-15T13:08:31.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6ceff4740 10 client.18447 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:31.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6ceff4740 1 -- 172.21.15.2:0/3223859526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55b22567d6a0 con 0x55b2257178a0 2024-03-15T13:08:31.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6a9ffb640 1 -- 172.21.15.2:0/3223859526 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55b22537e7e0 con 0x55b2257178a0 2024-03-15T13:08:31.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6a9ffb640 1 -- 172.21.15.2:0/3223859526 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe6bc03c040 con 0x55b2257178a0 2024-03-15T13:08:31.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6ceff4740 10 client.18447 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:31.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6ceff4740 10 client.18447 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:08:31.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6ceff4740 20 client.18447 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:31.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/3223859526 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b225734ca0 0x55b2257370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6ceff4740 4 client.18447 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:08:31.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.616+0000 7fe6ceff4740 1 -- 172.21.15.2:0/3223859526 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2253bb720 con 0x55b225734ca0 2024-03-15T13:08:31.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.617+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/3223859526 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b225734ca0 0x55b2257370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.617+0000 7fe6a9ffb640 10 client.18447 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:08:31.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.617+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/3223859526 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b225734ca0 0x55b2257370c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63cc418cde9786b9 server_cookie=51b870ab00f96cda in_seq=0 out_seq=0 2024-03-15T13:08:31.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.618+0000 7fe6a9ffb640 1 -- 172.21.15.2:0/3223859526 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b2253bb720 con 0x55b225734ca0 2024-03-15T13:08:31.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.618+0000 7fe6a9ffb640 10 client.18447 handle_command_reply: tid=0 2024-03-15T13:08:31.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.618+0000 7fe6ceff4740 1 client.18447 shutdown 2024-03-15T13:08:31.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.618+0000 7fe6ceff4740 1 -- 172.21.15.2:0/3223859526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2257178a0 msgr2=0x55b225721870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.618+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/3223859526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2257178a0 0x55b225721870 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009230 tx=0x7fe6bc004b00 comp rx=0 tx=0).stop 2024-03-15T13:08:31.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.618+0000 7fe6ceff4740 1 -- 172.21.15.2:0/3223859526 shutdown_connections 2024-03-15T13:08:31.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.618+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/3223859526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b225724b60 0x55b225721df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.618+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/3223859526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2257178a0 0x55b225721870 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.618+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/3223859526 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b225734ca0 0x55b2257370c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.618+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/3223859526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225716f80 0x55b2257244e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.618+0000 7fe6ceff4740 1 -- 172.21.15.2:0/3223859526 >> 172.21.15.2:0/3223859526 conn(0x55b225711980 msgr2=0x55b2257136d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.618+0000 7fe6a9ffb640 0 client.18447 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:08:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.618+0000 7fe6ceff4740 1 -- 172.21.15.2:0/3223859526 shutdown_connections 2024-03-15T13:08:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.618+0000 7fe6ceff4740 1 -- 172.21.15.2:0/3223859526 wait complete. 2024-03-15T13:08:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.618+0000 7fe6ceff4740 20 client.18447 trim_cache size 0 max 16384 2024-03-15T13:08:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.630+0000 7fe6ceff4740 1 Processor -- start 2024-03-15T13:08:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.630+0000 7fe6ceff4740 1 -- start start 2024-03-15T13:08:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.630+0000 7fe6ceff4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225716f80 0x55b225723fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.630+0000 7fe6ceff4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2257178a0 0x55b2257244e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6ceff4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b225724b60 0x55b22572fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6ceff4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b225604130 con 0x55b2257178a0 2024-03-15T13:08:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6ceff4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b225716bb0 con 0x55b225724b60 2024-03-15T13:08:31.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6ceff4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b22570d2b0 con 0x55b225716f80 2024-03-15T13:08:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6ca1b1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2257178a0 0x55b2257244e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6c99b0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225716f80 0x55b225723fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6c99b0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225716f80 0x55b225723fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:40058/0 (socket says 172.21.15.2:40058) 2024-03-15T13:08:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6c99b0640 1 -- 172.21.15.2:0/2224784307 learned_addr learned my addr 172.21.15.2:0/2224784307 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/2224784307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b225724b60 0x55b22572fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6c99b0640 1 -- 172.21.15.2:0/2224784307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b225724b60 msgr2=0x55b22572fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/2224784307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b225724b60 0x55b22572fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6c99b0640 1 -- 172.21.15.2:0/2224784307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2257178a0 msgr2=0x55b2257244e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/2224784307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2257178a0 0x55b2257244e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6c99b0640 1 -- 172.21.15.2:0/2224784307 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b225619c20 con 0x55b225716f80 2024-03-15T13:08:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6ca1b1640 1 --2- 172.21.15.2:0/2224784307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2257178a0 0x55b2257244e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/2224784307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225716f80 0x55b225723fa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55b225737300 tx=0x7fe6b8009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7064011fa9a73b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6abfff640 1 -- 172.21.15.2:0/2224784307 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8005690 con 0x55b225716f80 2024-03-15T13:08:31.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6abfff640 1 -- 172.21.15.2:0/2224784307 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe6b8004500 con 0x55b225716f80 2024-03-15T13:08:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.631+0000 7fe6abfff640 1 -- 172.21.15.2:0/2224784307 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8004e60 con 0x55b225716f80 2024-03-15T13:08:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 -- 172.21.15.2:0/2224784307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225716f80 msgr2=0x55b225723fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/2224784307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225716f80 0x55b225723fa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55b225737300 tx=0x7fe6b8009830 comp rx=0 tx=0).stop 2024-03-15T13:08:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 -- 172.21.15.2:0/2224784307 shutdown_connections 2024-03-15T13:08:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/2224784307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b225724b60 0x55b22572fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/2224784307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2257178a0 0x55b2257244e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/2224784307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225716f80 0x55b225723fa0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 -- 172.21.15.2:0/2224784307 >> 172.21.15.2:0/2224784307 conn(0x55b225711980 msgr2=0x55b22571ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 -- 172.21.15.2:0/2224784307 shutdown_connections 2024-03-15T13:08:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 -- 172.21.15.2:0/2224784307 wait complete. 2024-03-15T13:08:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 -- start start 2024-03-15T13:08:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 Processor -- start 2024-03-15T13:08:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b225716f80 0x55b22572c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2257178a0 0x55b22571e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225724b60 0x55b22571ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b22572cd50 con 0x55b225716f80 2024-03-15T13:08:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b22571f360 con 0x55b2257178a0 2024-03-15T13:08:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b22571f520 con 0x55b225724b60 2024-03-15T13:08:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.632+0000 7fe6ceff4740 10 client.0 _ll_register_callbacks cb 0x7fe6bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6c99b0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b225716f80 0x55b22572c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6c99b0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b225716f80 0x55b22572c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57584/0 (socket says 172.21.15.2:57584) 2024-03-15T13:08:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6c99b0640 1 -- 172.21.15.2:0/2174977671 learned_addr learned my addr 172.21.15.2:0/2174977671 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6ca1b1640 1 --2- 172.21.15.2:0/2174977671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2257178a0 0x55b22571e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/2174977671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225724b60 0x55b22571ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6c99b0640 1 -- 172.21.15.2:0/2174977671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225724b60 msgr2=0x55b22571ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/2174977671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225724b60 0x55b22571ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6c99b0640 1 -- 172.21.15.2:0/2174977671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2257178a0 msgr2=0x55b22571e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/2174977671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2257178a0 0x55b22571e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6c99b0640 1 -- 172.21.15.2:0/2174977671 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b22567d6a0 con 0x55b225716f80 2024-03-15T13:08:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6c99b0640 1 --2- 172.21.15.2:0/2174977671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b225716f80 0x55b22572c480 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8006010 tx=0x7fe6b8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ddd458a9caafed server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6a9ffb640 1 -- 172.21.15.2:0/2174977671 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8004be0 con 0x55b225716f80 2024-03-15T13:08:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6a9ffb640 1 -- 172.21.15.2:0/2174977671 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe6b8004d40 con 0x55b225716f80 2024-03-15T13:08:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6ceff4740 1 -- 172.21.15.2:0/2174977671 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6b8005020 con 0x55b225716f80 2024-03-15T13:08:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6a9ffb640 1 -- 172.21.15.2:0/2174977671 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6b8005330 con 0x55b225716f80 2024-03-15T13:08:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6ceff4740 10 client.18453 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6ceff4740 1 -- 172.21.15.2:0/2174977671 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55b225641060 con 0x55b225716f80 2024-03-15T13:08:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.633+0000 7fe6a9ffb640 1 -- 172.21.15.2:0/2174977671 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe6b8005020 con 0x55b225716f80 2024-03-15T13:08:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6a9ffb640 1 -- 172.21.15.2:0/2174977671 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fe6b80328e0 con 0x55b225716f80 2024-03-15T13:08:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6ceff4740 10 client.18453 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6ceff4740 10 client.18453 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:08:31.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6ceff4740 20 client.18453 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/2174977671 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b225791420 0x55b225793810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6ceff4740 4 client.18453 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:08:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6ceff4740 1 -- 172.21.15.2:0/2174977671 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b22540a890 con 0x55b225791420 2024-03-15T13:08:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6ca1b1640 1 --2- 172.21.15.2:0/2174977671 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b225791420 0x55b225793810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6a9ffb640 10 client.18453 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:08:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6ca1b1640 1 --2- 172.21.15.2:0/2174977671 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b225791420 0x55b225793810 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67e04b704d71d12b server_cookie=1a765731b4d42740 in_seq=0 out_seq=0 2024-03-15T13:08:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6a9ffb640 1 -- 172.21.15.2:0/2174977671 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b22540a890 con 0x55b225791420 2024-03-15T13:08:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6a9ffb640 10 client.18453 handle_command_reply: tid=0 2024-03-15T13:08:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6ceff4740 1 client.18453 shutdown 2024-03-15T13:08:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6ceff4740 1 -- 172.21.15.2:0/2174977671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b225716f80 msgr2=0x55b22572c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/2174977671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b225716f80 0x55b22572c480 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8006010 tx=0x7fe6b8005850 comp rx=0 tx=0).stop 2024-03-15T13:08:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6ceff4740 1 -- 172.21.15.2:0/2174977671 shutdown_connections 2024-03-15T13:08:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.634+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/2174977671 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x55b225791420 0x55b225793810 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/2174977671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b225724b60 0x55b22571ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/2174977671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2257178a0 0x55b22571e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6ceff4740 1 --2- 172.21.15.2:0/2174977671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b225716f80 0x55b22572c480 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6a9ffb640 0 client.18453 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:08:31.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6ceff4740 1 -- 172.21.15.2:0/2174977671 >> 172.21.15.2:0/2174977671 conn(0x55b225711980 msgr2=0x55b22571c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6ceff4740 1 -- 172.21.15.2:0/2174977671 shutdown_connections 2024-03-15T13:08:31.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6ceff4740 1 -- 172.21.15.2:0/2174977671 wait complete. 2024-03-15T13:08:31.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6ceff4740 20 client.18453 trim_cache size 0 max 16384 2024-03-15T13:08:31.626 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:08:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6cc415640 1 -- 172.21.15.2:0/288853346 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe6ac0a1ba0 msgr2=0x7fe6ac0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6cc415640 1 --2- 172.21.15.2:0/288853346 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe6ac0a1ba0 0x7fe6ac0a4060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40cb5d0 tx=0x7fe6b406cc00 comp rx=0 tx=0).stop 2024-03-15T13:08:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6cc415640 1 -- 172.21.15.2:0/288853346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c41074f0 msgr2=0x7fe6c41978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6cc415640 1 --2- 172.21.15.2:0/288853346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c41074f0 0x7fe6c41978f0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400efd0 tx=0x7fe6b400c6a0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6cc415640 1 -- 172.21.15.2:0/288853346 shutdown_connections 2024-03-15T13:08:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6cc415640 1 --2- 172.21.15.2:0/288853346 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe6ac0a1ba0 0x7fe6ac0a4060 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6cc415640 1 --2- 172.21.15.2:0/288853346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6c41074f0 0x7fe6c41978f0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6cc415640 1 --2- 172.21.15.2:0/288853346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6c40690b0 0x7fe6c419dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6cc415640 1 --2- 172.21.15.2:0/288853346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6c40686f0 0x7fe6c419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6cc415640 1 -- 172.21.15.2:0/288853346 >> 172.21.15.2:0/288853346 conn(0x7fe6c40774f0 msgr2=0x7fe6c40ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6cc415640 1 -- 172.21.15.2:0/288853346 shutdown_connections 2024-03-15T13:08:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.635+0000 7fe6cc415640 1 -- 172.21.15.2:0/288853346 wait complete. 2024-03-15T13:08:31.630 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:08:31.630 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:08:31.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.750+0000 7fd9edb48640 1 -- 172.21.15.2:0/2946327564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9e81014a0 msgr2=0x7fd9e8101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.750+0000 7fd9edb48640 1 --2- 172.21.15.2:0/2946327564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9e81014a0 0x7fd9e8101920 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009900 tx=0x7fd9dc02bc30 comp rx=0 tx=0).stop 2024-03-15T13:08:31.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.750+0000 7fd9edb48640 1 -- 172.21.15.2:0/2946327564 shutdown_connections 2024-03-15T13:08:31.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.750+0000 7fd9edb48640 1 --2- 172.21.15.2:0/2946327564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9e8101e60 0x7fd9e810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.750+0000 7fd9edb48640 1 --2- 172.21.15.2:0/2946327564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9e81014a0 0x7fd9e8101920 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.750+0000 7fd9edb48640 1 --2- 172.21.15.2:0/2946327564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9e8107560 0x7fd9e8107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.750+0000 7fd9edb48640 1 -- 172.21.15.2:0/2946327564 >> 172.21.15.2:0/2946327564 conn(0x7fd9e80788f0 msgr2=0x7fd9e8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.750+0000 7fd9edb48640 1 -- 172.21.15.2:0/2946327564 shutdown_connections 2024-03-15T13:08:31.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.750+0000 7fd9edb48640 1 -- 172.21.15.2:0/2946327564 wait complete. 2024-03-15T13:08:31.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.750+0000 7fd9edb48640 1 Processor -- start 2024-03-15T13:08:31.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.750+0000 7fd9edb48640 1 -- start start 2024-03-15T13:08:31.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9edb48640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9e81014a0 0x7fd9e819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9edb48640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9e8101e60 0x7fd9e819dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9edb48640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9e8107560 0x7fd9e8197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9edb48640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e8197ee0 con 0x7fd9e81014a0 2024-03-15T13:08:31.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9edb48640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e8198050 con 0x7fd9e8101e60 2024-03-15T13:08:31.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9edb48640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e8198210 con 0x7fd9e8107560 2024-03-15T13:08:31.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9e77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9e81014a0 0x7fd9e819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9e77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9e81014a0 0x7fd9e819d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57612/0 (socket says 172.21.15.2:57612) 2024-03-15T13:08:31.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9e77fe640 1 -- 172.21.15.2:0/3671881094 learned_addr learned my addr 172.21.15.2:0/3671881094 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9e6ffd640 1 --2- 172.21.15.2:0/3671881094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9e8101e60 0x7fd9e819dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9e7fff640 1 --2- 172.21.15.2:0/3671881094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9e8107560 0x7fd9e8197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9e77fe640 1 -- 172.21.15.2:0/3671881094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9e8107560 msgr2=0x7fd9e8197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9e77fe640 1 --2- 172.21.15.2:0/3671881094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9e8107560 0x7fd9e8197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9e77fe640 1 -- 172.21.15.2:0/3671881094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9e8101e60 msgr2=0x7fd9e819dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9e77fe640 1 --2- 172.21.15.2:0/3671881094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9e8101e60 0x7fd9e819dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9e77fe640 1 -- 172.21.15.2:0/3671881094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc009560 con 0x7fd9e81014a0 2024-03-15T13:08:31.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9e77fe640 1 --2- 172.21.15.2:0/3671881094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9e81014a0 0x7fd9e819d6e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800e9f0 tx=0x7fd9d800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9e4ff9640 1 -- 172.21.15.2:0/3671881094 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8009720 con 0x7fd9e81014a0 2024-03-15T13:08:31.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9edb48640 1 -- 172.21.15.2:0/3671881094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e81984f0 con 0x7fd9e81014a0 2024-03-15T13:08:31.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9e4ff9640 1 -- 172.21.15.2:0/3671881094 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd9d800cd80 con 0x7fd9e81014a0 2024-03-15T13:08:31.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.751+0000 7fd9e4ff9640 1 -- 172.21.15.2:0/3671881094 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8010690 con 0x7fd9e81014a0 2024-03-15T13:08:31.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.752+0000 7fd9edb48640 1 -- 172.21.15.2:0/3671881094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e81a4780 con 0x7fd9e81014a0 2024-03-15T13:08:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.754+0000 7fd9e4ff9640 1 -- 172.21.15.2:0/3671881094 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fd9d8004e90 con 0x7fd9e81014a0 2024-03-15T13:08:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.754+0000 7fd9f07ff740 1 Processor -- start 2024-03-15T13:08:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.754+0000 7fd9f07ff740 1 -- start start 2024-03-15T13:08:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.754+0000 7fd9f07ff740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e522781630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.754+0000 7fd9f07ff740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 0x55e52278a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.754+0000 7fd9f07ff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e52278a750 0x55e52278cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.754+0000 7fd9f07ff740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52273c960 con 0x55e522781250 2024-03-15T13:08:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.754+0000 7fd9f07ff740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5226f0310 con 0x55e522781b70 2024-03-15T13:08:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.754+0000 7fd9f07ff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5226ed840 con 0x55e52278a750 2024-03-15T13:08:31.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.754+0000 7fd9e7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e522781630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.754+0000 7fd9e7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e522781630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57626/0 (socket says 172.21.15.2:57626) 2024-03-15T13:08:31.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.754+0000 7fd9e7fff640 1 -- 172.21.15.2:0/98999577 learned_addr learned my addr 172.21.15.2:0/98999577 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.754+0000 7fd9e77fe640 1 --2- 172.21.15.2:0/98999577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 0x55e52278a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.754+0000 7fd9e4ff9640 1 --2- 172.21.15.2:0/3671881094 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd9d00a1ba0 0x7fd9d00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.754+0000 7fd9e6ffd640 1 --2- 172.21.15.2:0/98999577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e52278a750 0x55e52278cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.755+0000 7fd9e7fff640 1 --2- 172.21.15.2:0/3671881094 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd9d00a1ba0 0x7fd9d00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.755+0000 7fd9e7fff640 1 -- 172.21.15.2:0/98999577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e52278a750 msgr2=0x55e52278cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.755+0000 7fd9e7fff640 1 --2- 172.21.15.2:0/98999577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e52278a750 0x55e52278cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.755+0000 7fd9e7fff640 1 -- 172.21.15.2:0/98999577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 msgr2=0x55e52278a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.755+0000 7fd9e4ff9640 1 -- 172.21.15.2:0/3671881094 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd9d8014070 con 0x7fd9e81014a0 2024-03-15T13:08:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.755+0000 7fd9e7fff640 1 --2- 172.21.15.2:0/98999577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 0x55e52278a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.755+0000 7fd9e7fff640 1 -- 172.21.15.2:0/98999577 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5226ab060 con 0x55e522781250 2024-03-15T13:08:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.755+0000 7fd9e7fff640 1 --2- 172.21.15.2:0/98999577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e522781630 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0005c40 tx=0x7fd9e000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01881747157d197 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.755+0000 7fd9cdffb640 1 -- 172.21.15.2:0/98999577 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0003ec0 con 0x55e522781250 2024-03-15T13:08:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.755+0000 7fd9cdffb640 1 -- 172.21.15.2:0/98999577 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd9e0013170 con 0x55e522781250 2024-03-15T13:08:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.755+0000 7fd9e7fff640 1 --2- 172.21.15.2:0/3671881094 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd9d00a1ba0 0x7fd9d00a4060 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0023000 tx=0x7fd9e0021000 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.755+0000 7fd9cdffb640 1 -- 172.21.15.2:0/98999577 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0010960 con 0x55e522781250 2024-03-15T13:08:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.756+0000 7fd9f07ff740 1 -- 172.21.15.2:0/98999577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 msgr2=0x55e522781630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.756+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/98999577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e522781630 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0005c40 tx=0x7fd9e000f3c0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.756+0000 7fd9f07ff740 1 -- 172.21.15.2:0/98999577 shutdown_connections 2024-03-15T13:08:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.756+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/98999577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e52278a750 0x55e52278cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.756+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/98999577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 0x55e52278a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.756+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/98999577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e522781630 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.756+0000 7fd9f07ff740 1 -- 172.21.15.2:0/98999577 >> 172.21.15.2:0/98999577 conn(0x55e52277b980 msgr2=0x55e52277bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.756+0000 7fd9f07ff740 1 -- 172.21.15.2:0/98999577 shutdown_connections 2024-03-15T13:08:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.756+0000 7fd9f07ff740 1 -- 172.21.15.2:0/98999577 wait complete. 2024-03-15T13:08:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.756+0000 7fd9f07ff740 1 -- start start 2024-03-15T13:08:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.756+0000 7fd9f07ff740 1 Processor -- start 2024-03-15T13:08:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9f07ff740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e52278a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9f07ff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e522781b70 0x55e522787420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9f07ff740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e52278a750 0x55e522787960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9f07ff740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5226e9020 con 0x55e522781250 2024-03-15T13:08:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9f07ff740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5223bc430 con 0x55e52278a750 2024-03-15T13:08:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9f07ff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e522378e00 con 0x55e522781b70 2024-03-15T13:08:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9f07ff740 10 client.0 _ll_register_callbacks cb 0x55e522690cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9e77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e52278a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9e77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e52278a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57630/0 (socket says 172.21.15.2:57630) 2024-03-15T13:08:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9e77fe640 1 -- 172.21.15.2:0/1469999814 learned_addr learned my addr 172.21.15.2:0/1469999814 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9e7fff640 1 --2- 172.21.15.2:0/1469999814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e52278a750 0x55e522787960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9e6ffd640 1 --2- 172.21.15.2:0/1469999814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e522781b70 0x55e522787420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9e77fe640 1 -- 172.21.15.2:0/1469999814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e522781b70 msgr2=0x55e522787420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9e77fe640 1 --2- 172.21.15.2:0/1469999814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e522781b70 0x55e522787420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9e77fe640 1 -- 172.21.15.2:0/1469999814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e52278a750 msgr2=0x55e522787960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9e77fe640 1 --2- 172.21.15.2:0/1469999814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e52278a750 0x55e522787960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9e77fe640 1 -- 172.21.15.2:0/1469999814 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e0013700 con 0x55e522781250 2024-03-15T13:08:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9e77fe640 1 --2- 172.21.15.2:0/1469999814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e52278a090 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800e9c0 tx=0x7fd9d809a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb462fd44cf779f server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9c6ffd640 1 -- 172.21.15.2:0/1469999814 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d809e8d0 con 0x55e522781250 2024-03-15T13:08:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9c6ffd640 1 -- 172.21.15.2:0/1469999814 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd9d80a2f20 con 0x55e522781250 2024-03-15T13:08:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.757+0000 7fd9f07ff740 1 -- 172.21.15.2:0/1469999814 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5223e87e0 con 0x55e522781250 2024-03-15T13:08:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.758+0000 7fd9c6ffd640 1 -- 172.21.15.2:0/1469999814 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9d809a780 con 0x55e522781250 2024-03-15T13:08:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.758+0000 7fd9f07ff740 10 client.18465 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.758+0000 7fd9f07ff740 1 -- 172.21.15.2:0/1469999814 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55e5226e76a0 con 0x55e522781250 2024-03-15T13:08:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.758+0000 7fd9c6ffd640 1 -- 172.21.15.2:0/1469999814 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55e5223e87e0 con 0x55e522781250 2024-03-15T13:08:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.758+0000 7fd9c6ffd640 1 -- 172.21.15.2:0/1469999814 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd9d801f060 con 0x55e522781250 2024-03-15T13:08:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.758+0000 7fd9f07ff740 10 client.18465 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.758+0000 7fd9f07ff740 10 client.18465 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:08:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.758+0000 7fd9f07ff740 20 client.18465 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.758+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/1469999814 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55e52279a820 0x55e52279cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.758+0000 7fd9f07ff740 4 client.18465 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:08:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.758+0000 7fd9f07ff740 1 -- 172.21.15.2:0/1469999814 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e522425720 con 0x55e52279a820 2024-03-15T13:08:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.758+0000 7fd9e6ffd640 1 --2- 172.21.15.2:0/1469999814 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55e52279a820 0x55e52279cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.759+0000 7fd9e6ffd640 1 --2- 172.21.15.2:0/1469999814 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55e52279a820 0x55e52279cc10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c9e05d3a2aa90e2 server_cookie=509bfaa11e582ed2 in_seq=0 out_seq=0 2024-03-15T13:08:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.759+0000 7fd9c6ffd640 10 client.18465 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:08:31.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.759+0000 7fd9c6ffd640 1 -- 172.21.15.2:0/1469999814 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e522425720 con 0x55e52279a820 2024-03-15T13:08:31.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.759+0000 7fd9c6ffd640 10 client.18465 handle_command_reply: tid=0 2024-03-15T13:08:31.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.759+0000 7fd9f07ff740 1 client.18465 shutdown 2024-03-15T13:08:31.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.759+0000 7fd9f07ff740 1 -- 172.21.15.2:0/1469999814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 msgr2=0x55e52278a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.759+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/1469999814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e52278a090 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800e9c0 tx=0x7fd9d809a3f0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.759+0000 7fd9f07ff740 1 -- 172.21.15.2:0/1469999814 shutdown_connections 2024-03-15T13:08:31.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.759+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/1469999814 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55e52279a820 0x55e52279cc10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.759+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/1469999814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e52278a750 0x55e522787960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.759+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/1469999814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e522781b70 0x55e522787420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.759+0000 7fd9c6ffd640 0 client.18465 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:08:31.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.759+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/1469999814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e52278a090 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.759+0000 7fd9f07ff740 1 -- 172.21.15.2:0/1469999814 >> 172.21.15.2:0/1469999814 conn(0x55e52277b980 msgr2=0x55e52277d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.759+0000 7fd9f07ff740 1 -- 172.21.15.2:0/1469999814 shutdown_connections 2024-03-15T13:08:31.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.760+0000 7fd9f07ff740 1 -- 172.21.15.2:0/1469999814 wait complete. 2024-03-15T13:08:31.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.760+0000 7fd9f07ff740 20 client.18465 trim_cache size 0 max 16384 2024-03-15T13:08:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9f07ff740 1 Processor -- start 2024-03-15T13:08:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9f07ff740 1 -- start start 2024-03-15T13:08:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9f07ff740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e522789b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9f07ff740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 0x55e52278a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9f07ff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e52278a750 0x55e522795ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9f07ff740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52266e130 con 0x55e522781250 2024-03-15T13:08:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9f07ff740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e522787d20 con 0x55e522781b70 2024-03-15T13:08:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9f07ff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52279a650 con 0x55e52278a750 2024-03-15T13:08:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9e77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e522789b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9e77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e522789b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57644/0 (socket says 172.21.15.2:57644) 2024-03-15T13:08:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9e77fe640 1 -- 172.21.15.2:0/2804790402 learned_addr learned my addr 172.21.15.2:0/2804790402 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9e6ffd640 1 --2- 172.21.15.2:0/2804790402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 0x55e52278a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9e7fff640 1 --2- 172.21.15.2:0/2804790402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e52278a750 0x55e522795ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9e77fe640 1 -- 172.21.15.2:0/2804790402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e52278a750 msgr2=0x55e522795ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9e77fe640 1 --2- 172.21.15.2:0/2804790402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e52278a750 0x55e522795ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9e77fe640 1 -- 172.21.15.2:0/2804790402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 msgr2=0x55e52278a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9e77fe640 1 --2- 172.21.15.2:0/2804790402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 0x55e52278a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9e77fe640 1 -- 172.21.15.2:0/2804790402 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e522683c20 con 0x55e522781250 2024-03-15T13:08:31.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.772+0000 7fd9e6ffd640 1 --2- 172.21.15.2:0/2804790402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 0x55e52278a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:08:31.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.773+0000 7fd9e77fe640 1 --2- 172.21.15.2:0/2804790402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e522789b50 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80a29f0 tx=0x7fd9d80cb390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48538743a65df4cc server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.773+0000 7fd9cd7fa640 1 -- 172.21.15.2:0/2804790402 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8096a20 con 0x55e522781250 2024-03-15T13:08:31.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.773+0000 7fd9cd7fa640 1 -- 172.21.15.2:0/2804790402 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd9d8098d40 con 0x55e522781250 2024-03-15T13:08:31.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.773+0000 7fd9cd7fa640 1 -- 172.21.15.2:0/2804790402 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80cb520 con 0x55e522781250 2024-03-15T13:08:31.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.773+0000 7fd9f07ff740 1 -- 172.21.15.2:0/2804790402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 msgr2=0x55e522789b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.773+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/2804790402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e522789b50 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80a29f0 tx=0x7fd9d80cb390 comp rx=0 tx=0).stop 2024-03-15T13:08:31.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.773+0000 7fd9f07ff740 1 -- 172.21.15.2:0/2804790402 shutdown_connections 2024-03-15T13:08:31.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.773+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/2804790402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e52278a750 0x55e522795ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.773+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/2804790402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 0x55e52278a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.773+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/2804790402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e522781250 0x55e522789b50 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.773+0000 7fd9f07ff740 1 -- 172.21.15.2:0/2804790402 >> 172.21.15.2:0/2804790402 conn(0x55e52277b980 msgr2=0x55e522782920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.773+0000 7fd9f07ff740 1 -- 172.21.15.2:0/2804790402 shutdown_connections 2024-03-15T13:08:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.773+0000 7fd9f07ff740 1 -- 172.21.15.2:0/2804790402 wait complete. 2024-03-15T13:08:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9f07ff740 1 -- start start 2024-03-15T13:08:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9f07ff740 1 Processor -- start 2024-03-15T13:08:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9f07ff740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e522781250 0x55e522792140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9f07ff740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 0x55e5227845b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9f07ff740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e52278a750 0x55e522784af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9f07ff740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e522785030 con 0x55e52278a750 2024-03-15T13:08:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9f07ff740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5227851a0 con 0x55e522781b70 2024-03-15T13:08:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9f07ff740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e522785360 con 0x55e522781250 2024-03-15T13:08:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9f07ff740 10 client.0 _ll_register_callbacks cb 0x7fd9d8005780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9e7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e52278a750 0x55e522784af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9e7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e52278a750 0x55e522784af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57646/0 (socket says 172.21.15.2:57646) 2024-03-15T13:08:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9e7fff640 1 -- 172.21.15.2:0/2321619284 learned_addr learned my addr 172.21.15.2:0/2321619284 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9e77fe640 1 --2- 172.21.15.2:0/2321619284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e522781250 0x55e522792140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9e6ffd640 1 --2- 172.21.15.2:0/2321619284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 0x55e5227845b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9e7fff640 1 -- 172.21.15.2:0/2321619284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e522781250 msgr2=0x55e522792140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9e7fff640 1 --2- 172.21.15.2:0/2321619284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e522781250 0x55e522792140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9e7fff640 1 -- 172.21.15.2:0/2321619284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 msgr2=0x55e5227845b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.774+0000 7fd9e7fff640 1 --2- 172.21.15.2:0/2321619284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 0x55e5227845b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9e7fff640 1 -- 172.21.15.2:0/2321619284 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5226e76a0 con 0x55e52278a750 2024-03-15T13:08:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9e7fff640 1 --2- 172.21.15.2:0/2321619284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e52278a750 0x55e522784af0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0005fa0 tx=0x7fd9e0009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4941273d87806b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9c6ffd640 1 -- 172.21.15.2:0/2321619284 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e002c030 con 0x55e52278a750 2024-03-15T13:08:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9c6ffd640 1 -- 172.21.15.2:0/2321619284 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fd9e0010930 con 0x55e52278a750 2024-03-15T13:08:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9f07ff740 1 -- 172.21.15.2:0/2321619284 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9e0013170 con 0x55e52278a750 2024-03-15T13:08:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9c6ffd640 1 -- 172.21.15.2:0/2321619284 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9e0003ec0 con 0x55e52278a750 2024-03-15T13:08:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9f07ff740 10 client.18471 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9f07ff740 1 -- 172.21.15.2:0/2321619284 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55e5226a98c0 con 0x55e52278a750 2024-03-15T13:08:31.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9c6ffd640 1 -- 172.21.15.2:0/2321619284 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fd9e003b080 con 0x55e52278a750 2024-03-15T13:08:31.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9c6ffd640 1 -- 172.21.15.2:0/2321619284 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7fd9e0005640 con 0x55e52278a750 2024-03-15T13:08:31.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9f07ff740 10 client.18471 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:31.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9f07ff740 10 client.18471 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:08:31.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9f07ff740 20 client.18471 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:31.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/2321619284 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55e5227f74b0 0x55e5227f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9f07ff740 4 client.18471 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:08:31.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.775+0000 7fd9f07ff740 1 -- 172.21.15.2:0/2321619284 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e522474890 con 0x55e5227f74b0 2024-03-15T13:08:31.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9e77fe640 1 --2- 172.21.15.2:0/2321619284 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55e5227f74b0 0x55e5227f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9c6ffd640 10 client.18471 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:08:31.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9e77fe640 1 --2- 172.21.15.2:0/2321619284 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55e5227f74b0 0x55e5227f98d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bde6b267f1fd9a0b server_cookie=5f84f830a77239be in_seq=0 out_seq=0 2024-03-15T13:08:31.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9c6ffd640 1 -- 172.21.15.2:0/2321619284 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e522474890 con 0x55e5227f74b0 2024-03-15T13:08:31.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9c6ffd640 10 client.18471 handle_command_reply: tid=0 2024-03-15T13:08:31.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9f07ff740 1 client.18471 shutdown 2024-03-15T13:08:31.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9f07ff740 1 -- 172.21.15.2:0/2321619284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e52278a750 msgr2=0x55e522784af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/2321619284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e52278a750 0x55e522784af0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0005fa0 tx=0x7fd9e0009510 comp rx=0 tx=0).stop 2024-03-15T13:08:31.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9f07ff740 1 -- 172.21.15.2:0/2321619284 shutdown_connections 2024-03-15T13:08:31.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/2321619284 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55e5227f74b0 0x55e5227f98d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/2321619284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e52278a750 0x55e522784af0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/2321619284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e522781b70 0x55e5227845b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9f07ff740 1 --2- 172.21.15.2:0/2321619284 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e522781250 0x55e522792140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9c6ffd640 0 client.18471 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:08:31.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9f07ff740 1 -- 172.21.15.2:0/2321619284 >> 172.21.15.2:0/2321619284 conn(0x55e52277b980 msgr2=0x55e52277cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9f07ff740 1 -- 172.21.15.2:0/2321619284 shutdown_connections 2024-03-15T13:08:31.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9f07ff740 1 -- 172.21.15.2:0/2321619284 wait complete. 2024-03-15T13:08:31.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.776+0000 7fd9f07ff740 20 client.18471 trim_cache size 0 max 16384 2024-03-15T13:08:31.769 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:08:31.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.777+0000 7fd9edb48640 1 -- 172.21.15.2:0/3671881094 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd9d00a1ba0 msgr2=0x7fd9d00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.777+0000 7fd9edb48640 1 --2- 172.21.15.2:0/3671881094 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd9d00a1ba0 0x7fd9d00a4060 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0023000 tx=0x7fd9e0021000 comp rx=0 tx=0).stop 2024-03-15T13:08:31.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.777+0000 7fd9edb48640 1 -- 172.21.15.2:0/3671881094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9e81014a0 msgr2=0x7fd9e819d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.777+0000 7fd9edb48640 1 --2- 172.21.15.2:0/3671881094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9e81014a0 0x7fd9e819d6e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800e9f0 tx=0x7fd9d800edf0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.777+0000 7fd9edb48640 1 -- 172.21.15.2:0/3671881094 shutdown_connections 2024-03-15T13:08:31.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.777+0000 7fd9edb48640 1 --2- 172.21.15.2:0/3671881094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9e8107560 0x7fd9e8197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.777+0000 7fd9edb48640 1 --2- 172.21.15.2:0/3671881094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9e8101e60 0x7fd9e819dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.777+0000 7fd9edb48640 1 --2- 172.21.15.2:0/3671881094 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fd9d00a1ba0 0x7fd9d00a4060 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.777+0000 7fd9edb48640 1 --2- 172.21.15.2:0/3671881094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9e81014a0 0x7fd9e819d6e0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.777+0000 7fd9edb48640 1 -- 172.21.15.2:0/3671881094 >> 172.21.15.2:0/3671881094 conn(0x7fd9e80788f0 msgr2=0x7fd9e80fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.777+0000 7fd9edb48640 1 -- 172.21.15.2:0/3671881094 shutdown_connections 2024-03-15T13:08:31.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.777+0000 7fd9edb48640 1 -- 172.21.15.2:0/3671881094 wait complete. 2024-03-15T13:08:31.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:31 smithi002 ceph-mon[32060]: pgmap v297: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-15T13:08:31.771 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:31 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2025951463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:08:31.772 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:08:31.772 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:08:31.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:31 smithi184 ceph-mon[35993]: pgmap v297: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-15T13:08:31.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:31 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2025951463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:08:31.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:31 smithi149 ceph-mon[35927]: pgmap v297: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-15T13:08:31.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:31 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2025951463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:08:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.893+0000 7f8b4e7c7640 1 -- 172.21.15.2:0/3514184176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b48100dd0 msgr2=0x7f8b4810d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.893+0000 7f8b4e7c7640 1 --2- 172.21.15.2:0/3514184176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b48100dd0 0x7f8b4810d2e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009cd0 tx=0x7f8b4002c0f0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.893+0000 7f8b4e7c7640 1 -- 172.21.15.2:0/3514184176 shutdown_connections 2024-03-15T13:08:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.893+0000 7f8b4e7c7640 1 --2- 172.21.15.2:0/3514184176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b48100dd0 0x7f8b4810d2e0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.893+0000 7f8b4e7c7640 1 --2- 172.21.15.2:0/3514184176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b48100410 0x7f8b48100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.893+0000 7f8b4e7c7640 1 --2- 172.21.15.2:0/3514184176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b481064b0 0x7f8b48106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.893+0000 7f8b4e7c7640 1 -- 172.21.15.2:0/3514184176 >> 172.21.15.2:0/3514184176 conn(0x7f8b48077550 msgr2=0x7f8b48077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.893+0000 7f8b4e7c7640 1 -- 172.21.15.2:0/3514184176 shutdown_connections 2024-03-15T13:08:31.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.893+0000 7f8b4e7c7640 1 -- 172.21.15.2:0/3514184176 wait complete. 2024-03-15T13:08:31.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b4e7c7640 1 Processor -- start 2024-03-15T13:08:31.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b4e7c7640 1 -- start start 2024-03-15T13:08:31.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b4e7c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b48100410 0x7f8b4819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b4e7c7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b48100dd0 0x7f8b4819dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b4e7c7640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b481064b0 0x7f8b48197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b4e7c7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b48197fd0 con 0x7f8b48100410 2024-03-15T13:08:31.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b4e7c7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b48198140 con 0x7f8b48100dd0 2024-03-15T13:08:31.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b4e7c7640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b48198300 con 0x7f8b481064b0 2024-03-15T13:08:31.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b48100410 0x7f8b4819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b48100410 0x7f8b4819d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57660/0 (socket says 172.21.15.2:57660) 2024-03-15T13:08:31.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b47fff640 1 -- 172.21.15.2:0/1008205348 learned_addr learned my addr 172.21.15.2:0/1008205348 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b477fe640 1 --2- 172.21.15.2:0/1008205348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b48100dd0 0x7f8b4819dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b4cd64640 1 --2- 172.21.15.2:0/1008205348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b481064b0 0x7f8b48197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b47fff640 1 -- 172.21.15.2:0/1008205348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b481064b0 msgr2=0x7f8b48197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b47fff640 1 --2- 172.21.15.2:0/1008205348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b481064b0 0x7f8b48197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b47fff640 1 -- 172.21.15.2:0/1008205348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b48100dd0 msgr2=0x7f8b4819dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b47fff640 1 --2- 172.21.15.2:0/1008205348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b48100dd0 0x7f8b4819dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.894+0000 7f8b47fff640 1 -- 172.21.15.2:0/1008205348 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40009900 con 0x7f8b48100410 2024-03-15T13:08:31.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.895+0000 7f8b47fff640 1 --2- 172.21.15.2:0/1008205348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b48100410 0x7f8b4819d7d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800e9f0 tx=0x7f8b3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.895+0000 7f8b457fa640 1 -- 172.21.15.2:0/1008205348 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38009700 con 0x7f8b48100410 2024-03-15T13:08:31.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.895+0000 7f8b4e7c7640 1 -- 172.21.15.2:0/1008205348 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b481985e0 con 0x7f8b48100410 2024-03-15T13:08:31.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.895+0000 7f8b457fa640 1 -- 172.21.15.2:0/1008205348 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8b3800cd80 con 0x7f8b48100410 2024-03-15T13:08:31.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.895+0000 7f8b457fa640 1 -- 172.21.15.2:0/1008205348 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38018920 con 0x7f8b48100410 2024-03-15T13:08:31.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.895+0000 7f8b4e7c7640 1 -- 172.21.15.2:0/1008205348 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b481a4870 con 0x7f8b48100410 2024-03-15T13:08:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.897+0000 7f8b457fa640 1 -- 172.21.15.2:0/1008205348 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f8b38018a80 con 0x7f8b48100410 2024-03-15T13:08:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.897+0000 7f8b513a5740 1 Processor -- start 2024-03-15T13:08:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.897+0000 7f8b513a5740 1 -- start start 2024-03-15T13:08:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.897+0000 7f8b457fa640 1 --2- 172.21.15.2:0/1008205348 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8b300a1ba0 0x7f8b300a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.897+0000 7f8b513a5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 0x55deb0b79630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.897+0000 7f8b513a5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0b82210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.897+0000 7f8b513a5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b84b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.897+0000 7f8b513a5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb0b34960 con 0x55deb0b79b70 2024-03-15T13:08:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.897+0000 7f8b513a5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb0ae8310 con 0x55deb0b79250 2024-03-15T13:08:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.897+0000 7f8b513a5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb0ae5840 con 0x55deb0b82750 2024-03-15T13:08:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b4cd64640 1 --2- 172.21.15.2:0/1008205348 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8b300a1ba0 0x7f8b300a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b477fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0b82210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 0x55deb0b79630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b477fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0b82210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57666/0 (socket says 172.21.15.2:57666) 2024-03-15T13:08:31.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b477fe640 1 -- 172.21.15.2:0/2659414859 learned_addr learned my addr 172.21.15.2:0/2659414859 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b4cd64640 1 --2- 172.21.15.2:0/2659414859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b84b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b457fa640 1 -- 172.21.15.2:0/1008205348 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f8b38014070 con 0x7f8b48100410 2024-03-15T13:08:31.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b477fe640 1 -- 172.21.15.2:0/2659414859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 msgr2=0x55deb0b84b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b477fe640 1 --2- 172.21.15.2:0/2659414859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b84b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b477fe640 1 -- 172.21.15.2:0/2659414859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 msgr2=0x55deb0b79630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b477fe640 1 --2- 172.21.15.2:0/2659414859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 0x55deb0b79630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b477fe640 1 -- 172.21.15.2:0/2659414859 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55deb0aa3060 con 0x55deb0b79b70 2024-03-15T13:08:31.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b4cd64640 1 --2- 172.21.15.2:0/1008205348 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8b300a1ba0 0x7f8b300a4060 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8b40005cc0 tx=0x7f8b40009820 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b477fe640 1 --2- 172.21.15.2:0/2659414859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0b82210 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009c60 tx=0x7f8b3c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16458e54082380e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b2e7fc640 1 -- 172.21.15.2:0/2659414859 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c004230 con 0x55deb0b79b70 2024-03-15T13:08:31.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b2e7fc640 1 -- 172.21.15.2:0/2659414859 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8b3c004390 con 0x55deb0b79b70 2024-03-15T13:08:31.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.898+0000 7f8b2e7fc640 1 -- 172.21.15.2:0/2659414859 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0196f0 con 0x55deb0b79b70 2024-03-15T13:08:31.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.899+0000 7f8b513a5740 1 -- 172.21.15.2:0/2659414859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 msgr2=0x55deb0b82210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.899+0000 7f8b513a5740 1 --2- 172.21.15.2:0/2659414859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0b82210 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009c60 tx=0x7f8b3c011810 comp rx=0 tx=0).stop 2024-03-15T13:08:31.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.899+0000 7f8b513a5740 1 -- 172.21.15.2:0/2659414859 shutdown_connections 2024-03-15T13:08:31.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.899+0000 7f8b513a5740 1 --2- 172.21.15.2:0/2659414859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b84b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.899+0000 7f8b513a5740 1 --2- 172.21.15.2:0/2659414859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0b82210 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.899+0000 7f8b513a5740 1 --2- 172.21.15.2:0/2659414859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 0x55deb0b79630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.899+0000 7f8b513a5740 1 -- 172.21.15.2:0/2659414859 >> 172.21.15.2:0/2659414859 conn(0x55deb0b73980 msgr2=0x55deb0b73d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.899+0000 7f8b513a5740 1 -- 172.21.15.2:0/2659414859 shutdown_connections 2024-03-15T13:08:31.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.899+0000 7f8b513a5740 1 -- 172.21.15.2:0/2659414859 wait complete. 2024-03-15T13:08:31.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.899+0000 7f8b513a5740 1 -- start start 2024-03-15T13:08:31.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b513a5740 1 Processor -- start 2024-03-15T13:08:31.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b513a5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79250 0x55deb0b820f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b513a5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79b70 0x55deb0b7f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b513a5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b7fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b513a5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb0ae1020 con 0x55deb0b79250 2024-03-15T13:08:31.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b513a5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb07b4430 con 0x55deb0b79b70 2024-03-15T13:08:31.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b513a5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb0770e00 con 0x55deb0b82750 2024-03-15T13:08:31.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b513a5740 10 client.0 _ll_register_callbacks cb 0x55deb0a88cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:31.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79250 0x55deb0b820f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79250 0x55deb0b820f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57668/0 (socket says 172.21.15.2:57668) 2024-03-15T13:08:31.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b47fff640 1 -- 172.21.15.2:0/1780050581 learned_addr learned my addr 172.21.15.2:0/1780050581 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b477fe640 1 --2- 172.21.15.2:0/1780050581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79b70 0x55deb0b7f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b4cd64640 1 --2- 172.21.15.2:0/1780050581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b7fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b47fff640 1 -- 172.21.15.2:0/1780050581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 msgr2=0x55deb0b7fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b47fff640 1 --2- 172.21.15.2:0/1780050581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b7fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b47fff640 1 -- 172.21.15.2:0/1780050581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79b70 msgr2=0x55deb0b7f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b47fff640 1 --2- 172.21.15.2:0/1780050581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79b70 0x55deb0b7f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b47fff640 1 -- 172.21.15.2:0/1780050581 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3c009890 con 0x55deb0b79250 2024-03-15T13:08:31.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b47fff640 1 --2- 172.21.15.2:0/1780050581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79250 0x55deb0b820f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f8b38096bf0 tx=0x7f8b3801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbb5c57586cdb50 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.900+0000 7f8b277fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:31.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.901+0000 7f8b277fe640 1 -- 172.21.15.2:0/1780050581 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b380acd50 con 0x55deb0b79250 2024-03-15T13:08:31.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.901+0000 7f8b277fe640 1 -- 172.21.15.2:0/1780050581 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f8b380acb20 con 0x55deb0b79250 2024-03-15T13:08:31.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.901+0000 7f8b513a5740 1 -- 172.21.15.2:0/1780050581 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55deb07e07e0 con 0x55deb0b79250 2024-03-15T13:08:31.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.901+0000 7f8b277fe640 1 -- 172.21.15.2:0/1780050581 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b3809a790 con 0x55deb0b79250 2024-03-15T13:08:31.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.901+0000 7f8b513a5740 10 client.18483 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:31.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.901+0000 7f8b513a5740 1 -- 172.21.15.2:0/1780050581 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55deb0adf6a0 con 0x55deb0b79250 2024-03-15T13:08:31.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.901+0000 7f8b277fe640 1 -- 172.21.15.2:0/1780050581 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55deb07e07e0 con 0x55deb0b79250 2024-03-15T13:08:31.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.901+0000 7f8b277fe640 1 -- 172.21.15.2:0/1780050581 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8b380cb480 con 0x55deb0b79250 2024-03-15T13:08:31.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.901+0000 7f8b513a5740 10 client.18483 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:31.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.901+0000 7f8b513a5740 10 client.18483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:08:31.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.901+0000 7f8b513a5740 20 client.18483 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:31.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.901+0000 7f8b513a5740 1 --2- 172.21.15.2:0/1780050581 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55deb0b92870 0x55deb0b94c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.901+0000 7f8b513a5740 4 client.18483 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:08:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.901+0000 7f8b513a5740 1 -- 172.21.15.2:0/1780050581 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55deb081d720 con 0x55deb0b92870 2024-03-15T13:08:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.901+0000 7f8b477fe640 1 --2- 172.21.15.2:0/1780050581 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55deb0b92870 0x55deb0b94c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.902+0000 7f8b277fe640 10 client.18483 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.902+0000 7f8b477fe640 1 --2- 172.21.15.2:0/1780050581 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55deb0b92870 0x55deb0b94c60 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f10983a78bb0a0 server_cookie=97dbb2efd214f0b0 in_seq=0 out_seq=0 2024-03-15T13:08:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.902+0000 7f8b277fe640 1 -- 172.21.15.2:0/1780050581 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55deb081d720 con 0x55deb0b92870 2024-03-15T13:08:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.902+0000 7f8b277fe640 10 client.18483 handle_command_reply: tid=0 2024-03-15T13:08:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.902+0000 7f8b513a5740 1 client.18483 shutdown 2024-03-15T13:08:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.903+0000 7f8b513a5740 1 -- 172.21.15.2:0/1780050581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79250 msgr2=0x55deb0b820f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.903+0000 7f8b513a5740 1 --2- 172.21.15.2:0/1780050581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79250 0x55deb0b820f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f8b38096bf0 tx=0x7f8b3801f000 comp rx=0 tx=0).stop 2024-03-15T13:08:31.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.903+0000 7f8b513a5740 1 -- 172.21.15.2:0/1780050581 shutdown_connections 2024-03-15T13:08:31.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.903+0000 7f8b513a5740 1 --2- 172.21.15.2:0/1780050581 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55deb0b92870 0x55deb0b94c60 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.903+0000 7f8b513a5740 1 --2- 172.21.15.2:0/1780050581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b7fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.903+0000 7f8b277fe640 0 client.18483 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:31.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.903+0000 7f8b513a5740 1 --2- 172.21.15.2:0/1780050581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79b70 0x55deb0b7f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.903+0000 7f8b513a5740 1 --2- 172.21.15.2:0/1780050581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79250 0x55deb0b820f0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.903+0000 7f8b513a5740 1 -- 172.21.15.2:0/1780050581 >> 172.21.15.2:0/1780050581 conn(0x55deb0b73980 msgr2=0x55deb0b756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.903+0000 7f8b513a5740 1 -- 172.21.15.2:0/1780050581 shutdown_connections 2024-03-15T13:08:31.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.903+0000 7f8b513a5740 1 -- 172.21.15.2:0/1780050581 wait complete. 2024-03-15T13:08:31.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.903+0000 7f8b513a5740 20 client.18483 trim_cache size 0 max 16384 2024-03-15T13:08:31.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.915+0000 7f8b513a5740 1 Processor -- start 2024-03-15T13:08:31.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.915+0000 7f8b513a5740 1 -- start start 2024-03-15T13:08:31.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.915+0000 7f8b513a5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 0x55deb0b81bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.915+0000 7f8b513a5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0b820f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.915+0000 7f8b513a5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b8dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.915+0000 7f8b513a5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb0a66130 con 0x55deb0b79b70 2024-03-15T13:08:31.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.915+0000 7f8b513a5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb0b926d0 con 0x55deb0b79250 2024-03-15T13:08:31.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.915+0000 7f8b513a5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb0b7da50 con 0x55deb0b82750 2024-03-15T13:08:31.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.915+0000 7f8b477fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0b820f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.916+0000 7f8b477fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0b820f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57680/0 (socket says 172.21.15.2:57680) 2024-03-15T13:08:31.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.916+0000 7f8b477fe640 1 -- 172.21.15.2:0/3105006366 learned_addr learned my addr 172.21.15.2:0/3105006366 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.916+0000 7f8b47fff640 1 --2- 172.21.15.2:0/3105006366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 0x55deb0b81bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.916+0000 7f8b4cd64640 1 --2- 172.21.15.2:0/3105006366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b8dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.916+0000 7f8b477fe640 1 -- 172.21.15.2:0/3105006366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 msgr2=0x55deb0b8dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.916+0000 7f8b477fe640 1 --2- 172.21.15.2:0/3105006366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b8dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.916+0000 7f8b477fe640 1 -- 172.21.15.2:0/3105006366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 msgr2=0x55deb0b81bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.916+0000 7f8b477fe640 1 --2- 172.21.15.2:0/3105006366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 0x55deb0b81bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.916+0000 7f8b477fe640 1 -- 172.21.15.2:0/3105006366 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55deb0a7bc20 con 0x55deb0b79b70 2024-03-15T13:08:31.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.916+0000 7f8b4cd64640 1 --2- 172.21.15.2:0/3105006366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b8dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:08:31.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.916+0000 7f8b47fff640 1 --2- 172.21.15.2:0/3105006366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 0x55deb0b81bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:08:31.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.916+0000 7f8b477fe640 1 --2- 172.21.15.2:0/3105006366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0b820f0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009710 tx=0x7f8b3c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f3cfe4b1de5978 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.916+0000 7f8b2dffb640 1 -- 172.21.15.2:0/3105006366 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0052c0 con 0x55deb0b79b70 2024-03-15T13:08:31.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.916+0000 7f8b2dffb640 1 -- 172.21.15.2:0/3105006366 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8b3c009e10 con 0x55deb0b79b70 2024-03-15T13:08:31.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.916+0000 7f8b2dffb640 1 -- 172.21.15.2:0/3105006366 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c004760 con 0x55deb0b79b70 2024-03-15T13:08:31.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 -- 172.21.15.2:0/3105006366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 msgr2=0x55deb0b820f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 --2- 172.21.15.2:0/3105006366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0b820f0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009710 tx=0x7f8b3c009b80 comp rx=0 tx=0).stop 2024-03-15T13:08:31.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 -- 172.21.15.2:0/3105006366 shutdown_connections 2024-03-15T13:08:31.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 --2- 172.21.15.2:0/3105006366 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b8dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 --2- 172.21.15.2:0/3105006366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0b820f0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 --2- 172.21.15.2:0/3105006366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 0x55deb0b81bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 -- 172.21.15.2:0/3105006366 >> 172.21.15.2:0/3105006366 conn(0x55deb0b73980 msgr2=0x55deb0b72b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 -- 172.21.15.2:0/3105006366 shutdown_connections 2024-03-15T13:08:31.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 -- 172.21.15.2:0/3105006366 wait complete. 2024-03-15T13:08:31.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 -- start start 2024-03-15T13:08:31.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 Processor -- start 2024-03-15T13:08:31.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 0x55deb0be6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0be6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b862e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb0b868a0 con 0x55deb0b79b70 2024-03-15T13:08:31.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb0b86a10 con 0x55deb0b79250 2024-03-15T13:08:31.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb0b86bd0 con 0x55deb0b82750 2024-03-15T13:08:31.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b513a5740 10 client.0 _ll_register_callbacks cb 0x7f8b3800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:31.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b477fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0be6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b477fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0be6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57692/0 (socket says 172.21.15.2:57692) 2024-03-15T13:08:31.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.917+0000 7f8b477fe640 1 -- 172.21.15.2:0/3337654929 learned_addr learned my addr 172.21.15.2:0/3337654929 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:31.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b47fff640 1 --2- 172.21.15.2:0/3337654929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 0x55deb0be6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b4cd64640 1 --2- 172.21.15.2:0/3337654929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b862e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b477fe640 1 -- 172.21.15.2:0/3337654929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 msgr2=0x55deb0b862e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b477fe640 1 --2- 172.21.15.2:0/3337654929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b862e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b477fe640 1 -- 172.21.15.2:0/3337654929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 msgr2=0x55deb0be6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:31.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b477fe640 1 --2- 172.21.15.2:0/3337654929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 0x55deb0be6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b477fe640 1 -- 172.21.15.2:0/3337654929 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55deb0aa3060 con 0x55deb0b79b70 2024-03-15T13:08:31.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b477fe640 1 --2- 172.21.15.2:0/3337654929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0be6950 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c003d30 tx=0x7f8b3c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c101266d610238f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:31.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b277fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:31.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b277fe640 1 -- 172.21.15.2:0/3337654929 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c038070 con 0x55deb0b79b70 2024-03-15T13:08:31.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b277fe640 1 -- 172.21.15.2:0/3337654929 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f8b3c003800 con 0x55deb0b79b70 2024-03-15T13:08:31.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b513a5740 1 -- 172.21.15.2:0/3337654929 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b3c004390 con 0x55deb0b79b70 2024-03-15T13:08:31.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b277fe640 1 -- 172.21.15.2:0/3337654929 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b3c003e90 con 0x55deb0b79b70 2024-03-15T13:08:31.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b513a5740 10 client.18489 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:31.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b513a5740 1 -- 172.21.15.2:0/3337654929 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55deb0aa18c0 con 0x55deb0b79b70 2024-03-15T13:08:31.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.918+0000 7f8b277fe640 1 -- 172.21.15.2:0/3337654929 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8b3c004390 con 0x55deb0b79b70 2024-03-15T13:08:31.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.919+0000 7f8b277fe640 1 -- 172.21.15.2:0/3337654929 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f8b3c03b030 con 0x55deb0b79b70 2024-03-15T13:08:31.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.919+0000 7f8b513a5740 10 client.18489 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:31.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.919+0000 7f8b513a5740 10 client.18489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:08:31.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.919+0000 7f8b513a5740 20 client.18489 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:31.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.919+0000 7f8b513a5740 1 --2- 172.21.15.2:0/3337654929 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55deb0bef230 0x55deb0bf1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:31.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.919+0000 7f8b513a5740 4 client.18489 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:08:31.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.919+0000 7f8b513a5740 1 -- 172.21.15.2:0/3337654929 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55deb086c890 con 0x55deb0bef230 2024-03-15T13:08:31.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.919+0000 7f8b47fff640 1 --2- 172.21.15.2:0/3337654929 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55deb0bef230 0x55deb0bf1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:31.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.919+0000 7f8b277fe640 10 client.18489 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:31.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.919+0000 7f8b47fff640 1 --2- 172.21.15.2:0/3337654929 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55deb0bef230 0x55deb0bf1650 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=338d679742e33079 server_cookie=844579c54967552a in_seq=0 out_seq=0 2024-03-15T13:08:31.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.919+0000 7f8b277fe640 1 -- 172.21.15.2:0/3337654929 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55deb086c890 con 0x55deb0bef230 2024-03-15T13:08:31.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.919+0000 7f8b277fe640 10 client.18489 handle_command_reply: tid=0 2024-03-15T13:08:31.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.919+0000 7f8b513a5740 1 client.18489 shutdown 2024-03-15T13:08:31.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b513a5740 1 -- 172.21.15.2:0/3337654929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 msgr2=0x55deb0be6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b513a5740 1 --2- 172.21.15.2:0/3337654929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0be6950 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c003d30 tx=0x7f8b3c009780 comp rx=0 tx=0).stop 2024-03-15T13:08:31.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b513a5740 1 -- 172.21.15.2:0/3337654929 shutdown_connections 2024-03-15T13:08:31.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b513a5740 1 --2- 172.21.15.2:0/3337654929 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55deb0bef230 0x55deb0bf1650 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b513a5740 1 --2- 172.21.15.2:0/3337654929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55deb0b82750 0x55deb0b862e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b513a5740 1 --2- 172.21.15.2:0/3337654929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55deb0b79b70 0x55deb0be6950 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b277fe640 0 client.18489 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:31.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b513a5740 1 --2- 172.21.15.2:0/3337654929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55deb0b79250 0x55deb0be6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b513a5740 1 -- 172.21.15.2:0/3337654929 >> 172.21.15.2:0/3337654929 conn(0x55deb0b73980 msgr2=0x55deb0b7a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b513a5740 1 -- 172.21.15.2:0/3337654929 shutdown_connections 2024-03-15T13:08:31.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b513a5740 1 -- 172.21.15.2:0/3337654929 wait complete. 2024-03-15T13:08:31.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b513a5740 20 client.18489 trim_cache size 0 max 16384 2024-03-15T13:08:31.911 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:08:31.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b4e7c7640 1 -- 172.21.15.2:0/1008205348 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8b300a1ba0 msgr2=0x7f8b300a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b4e7c7640 1 --2- 172.21.15.2:0/1008205348 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8b300a1ba0 0x7f8b300a4060 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8b40005cc0 tx=0x7f8b40009820 comp rx=0 tx=0).stop 2024-03-15T13:08:31.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b4e7c7640 1 -- 172.21.15.2:0/1008205348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b48100410 msgr2=0x7f8b4819d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:31.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b4e7c7640 1 --2- 172.21.15.2:0/1008205348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b48100410 0x7f8b4819d7d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800e9f0 tx=0x7f8b3800eeb0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b4e7c7640 1 -- 172.21.15.2:0/1008205348 shutdown_connections 2024-03-15T13:08:31.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b4e7c7640 1 --2- 172.21.15.2:0/1008205348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b481064b0 0x7f8b48197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b4e7c7640 1 --2- 172.21.15.2:0/1008205348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b48100dd0 0x7f8b4819dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b4e7c7640 1 --2- 172.21.15.2:0/1008205348 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8b300a1ba0 0x7f8b300a4060 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b4e7c7640 1 --2- 172.21.15.2:0/1008205348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b48100410 0x7f8b4819d7d0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:31.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b4e7c7640 1 -- 172.21.15.2:0/1008205348 >> 172.21.15.2:0/1008205348 conn(0x7f8b48077550 msgr2=0x7f8b48069e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:31.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b4e7c7640 1 -- 172.21.15.2:0/1008205348 shutdown_connections 2024-03-15T13:08:31.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:31.920+0000 7f8b4e7c7640 1 -- 172.21.15.2:0/1008205348 wait complete. 2024-03-15T13:08:32.917 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-15T13:08:32.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:08:32.917 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-15T13:08:33.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.052+0000 7ff064a23640 1 -- 172.21.15.2:0/3227316683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0601074f0 msgr2=0x7ff0601078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.052+0000 7ff064a23640 1 --2- 172.21.15.2:0/3227316683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0601074f0 0x7ff0601078d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ff050009cd0 tx=0x7ff05002c0f0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.052+0000 7ff064a23640 1 -- 172.21.15.2:0/3227316683 shutdown_connections 2024-03-15T13:08:33.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.052+0000 7ff064a23640 1 --2- 172.21.15.2:0/3227316683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0600690b0 0x7ff06010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.052+0000 7ff064a23640 1 --2- 172.21.15.2:0/3227316683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0600686f0 0x7ff060068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.052+0000 7ff064a23640 1 --2- 172.21.15.2:0/3227316683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0601074f0 0x7ff0601078d0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.052+0000 7ff064a23640 1 -- 172.21.15.2:0/3227316683 >> 172.21.15.2:0/3227316683 conn(0x7ff0600774f0 msgr2=0x7ff060077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:33.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.052+0000 7ff064a23640 1 -- 172.21.15.2:0/3227316683 shutdown_connections 2024-03-15T13:08:33.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.052+0000 7ff064a23640 1 -- 172.21.15.2:0/3227316683 wait complete. 2024-03-15T13:08:33.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff064a23640 1 Processor -- start 2024-03-15T13:08:33.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff064a23640 1 -- start start 2024-03-15T13:08:33.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff064a23640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0600686f0 0x7ff0601003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff064a23640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0600690b0 0x7ff0601008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff064a23640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0601074f0 0x7ff060100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff064a23640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0601a5e80 con 0x7ff0601074f0 2024-03-15T13:08:33.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff064a23640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0601a5ff0 con 0x7ff0600690b0 2024-03-15T13:08:33.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff064a23640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0601a61b0 con 0x7ff0600686f0 2024-03-15T13:08:33.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff05ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0601074f0 0x7ff060100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff05ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0601074f0 0x7ff060100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57710/0 (socket says 172.21.15.2:57710) 2024-03-15T13:08:33.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff05ed9d640 1 -- 172.21.15.2:0/1609627220 learned_addr learned my addr 172.21.15.2:0/1609627220 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:33.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff05e59c640 1 --2- 172.21.15.2:0/1609627220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0600686f0 0x7ff0601003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff05dd9b640 1 --2- 172.21.15.2:0/1609627220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0600690b0 0x7ff0601008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff05ed9d640 1 -- 172.21.15.2:0/1609627220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0600686f0 msgr2=0x7ff0601003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff05ed9d640 1 --2- 172.21.15.2:0/1609627220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0600686f0 0x7ff0601003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff05ed9d640 1 -- 172.21.15.2:0/1609627220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0600690b0 msgr2=0x7ff0601008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff05ed9d640 1 --2- 172.21.15.2:0/1609627220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0600690b0 0x7ff0601008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff05ed9d640 1 -- 172.21.15.2:0/1609627220 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff050009900 con 0x7ff0601074f0 2024-03-15T13:08:33.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.053+0000 7ff05ed9d640 1 --2- 172.21.15.2:0/1609627220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0601074f0 0x7ff060100e30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ff05800e970 tx=0x7ff05800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:33.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.054+0000 7ff0577fe640 1 -- 172.21.15.2:0/1609627220 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058009720 con 0x7ff0601074f0 2024-03-15T13:08:33.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.054+0000 7ff0577fe640 1 -- 172.21.15.2:0/1609627220 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff0580054f0 con 0x7ff0601074f0 2024-03-15T13:08:33.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.054+0000 7ff064a23640 1 -- 172.21.15.2:0/1609627220 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0601a6490 con 0x7ff0601074f0 2024-03-15T13:08:33.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.054+0000 7ff0577fe640 1 -- 172.21.15.2:0/1609627220 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058010430 con 0x7ff0601074f0 2024-03-15T13:08:33.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.054+0000 7ff064a23640 1 -- 172.21.15.2:0/1609627220 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0601a69a0 con 0x7ff0601074f0 2024-03-15T13:08:33.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.057+0000 7ff0577fe640 1 -- 172.21.15.2:0/1609627220 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7ff058004f90 con 0x7ff0601074f0 2024-03-15T13:08:33.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.057+0000 7ff067603740 1 Processor -- start 2024-03-15T13:08:33.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.057+0000 7ff067603740 1 -- start start 2024-03-15T13:08:33.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.057+0000 7ff067603740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b590380 0x55ac6b590760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.057+0000 7ff067603740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b5992e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.057+0000 7ff067603740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b5998b0 0x55ac6b59bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.057+0000 7ff067603740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6b4fef90 con 0x55ac6b590d30 2024-03-15T13:08:33.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.057+0000 7ff067603740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6b4fc4c0 con 0x55ac6b5998b0 2024-03-15T13:08:33.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.057+0000 7ff067603740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6b4f7ca0 con 0x55ac6b590380 2024-03-15T13:08:33.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.057+0000 7ff05e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b5992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.057+0000 7ff05e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b5992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57722/0 (socket says 172.21.15.2:57722) 2024-03-15T13:08:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.057+0000 7ff05e59c640 1 -- 172.21.15.2:0/383280754 learned_addr learned my addr 172.21.15.2:0/383280754 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.057+0000 7ff05dd9b640 1 --2- 172.21.15.2:0/383280754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b590380 0x55ac6b590760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.057+0000 7ff05dd9b640 1 --2- 172.21.15.2:0/383280754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b5998b0 0x55ac6b59bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.057+0000 7ff0577fe640 1 --2- 172.21.15.2:0/1609627220 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff04c0a1ba0 0x7ff04c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.058+0000 7ff05e59c640 1 -- 172.21.15.2:0/383280754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b590380 msgr2=0x55ac6b590760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.058+0000 7ff05e59c640 1 --2- 172.21.15.2:0/383280754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b590380 0x55ac6b590760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.058+0000 7ff05e59c640 1 -- 172.21.15.2:0/383280754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b5998b0 msgr2=0x55ac6b59bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.058+0000 7ff05e59c640 1 --2- 172.21.15.2:0/383280754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b5998b0 0x55ac6b59bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.058+0000 7ff05e59c640 1 -- 172.21.15.2:0/383280754 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac6b4ba0b0 con 0x55ac6b590d30 2024-03-15T13:08:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.058+0000 7ff05dd9b640 1 --2- 172.21.15.2:0/383280754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b590380 0x55ac6b590760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.058+0000 7ff05ed9d640 1 --2- 172.21.15.2:0/1609627220 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff04c0a1ba0 0x7ff04c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.058+0000 7ff0577fe640 1 -- 172.21.15.2:0/1609627220 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ff058014070 con 0x7ff0601074f0 2024-03-15T13:08:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.058+0000 7ff05e59c640 1 --2- 172.21.15.2:0/383280754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b5992e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ff050004af0 tx=0x7ff050005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34007666c4b9a76a server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.058+0000 7ff054ff9640 1 -- 172.21.15.2:0/383280754 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050034440 con 0x55ac6b590d30 2024-03-15T13:08:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.058+0000 7ff054ff9640 1 -- 172.21.15.2:0/383280754 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff0500345a0 con 0x55ac6b590d30 2024-03-15T13:08:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.058+0000 7ff054ff9640 1 -- 172.21.15.2:0/383280754 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050003b70 con 0x55ac6b590d30 2024-03-15T13:08:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.059+0000 7ff067603740 1 -- 172.21.15.2:0/383280754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 msgr2=0x55ac6b5992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.059+0000 7ff067603740 1 --2- 172.21.15.2:0/383280754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b5992e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ff050004af0 tx=0x7ff050005f70 comp rx=0 tx=0).stop 2024-03-15T13:08:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.059+0000 7ff05ed9d640 1 --2- 172.21.15.2:0/1609627220 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff04c0a1ba0 0x7ff04c0a4060 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff05809a740 tx=0x7ff058085520 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.059+0000 7ff067603740 1 -- 172.21.15.2:0/383280754 shutdown_connections 2024-03-15T13:08:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.059+0000 7ff067603740 1 --2- 172.21.15.2:0/383280754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b5998b0 0x55ac6b59bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.059+0000 7ff067603740 1 --2- 172.21.15.2:0/383280754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b5992e0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.059+0000 7ff067603740 1 --2- 172.21.15.2:0/383280754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b590380 0x55ac6b590760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.059+0000 7ff067603740 1 -- 172.21.15.2:0/383280754 >> 172.21.15.2:0/383280754 conn(0x55ac6b58a840 msgr2=0x55ac6b58ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.059+0000 7ff067603740 1 -- 172.21.15.2:0/383280754 shutdown_connections 2024-03-15T13:08:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.059+0000 7ff067603740 1 -- 172.21.15.2:0/383280754 wait complete. 2024-03-15T13:08:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.059+0000 7ff067603740 1 -- start start 2024-03-15T13:08:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.059+0000 7ff067603740 1 Processor -- start 2024-03-15T13:08:33.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff067603740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b590380 0x55ac6b596870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff067603740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590d30 0x55ac6b596db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff067603740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b5998b0 0x55ac6b5972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff067603740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6b49cf40 con 0x55ac6b5998b0 2024-03-15T13:08:33.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff067603740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6b1cb4b0 con 0x55ac6b590d30 2024-03-15T13:08:33.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff067603740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6b47d670 con 0x55ac6b590380 2024-03-15T13:08:33.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff067603740 10 client.0 _ll_register_callbacks cb 0x55ac6b4eaf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:33.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff05dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b5998b0 0x55ac6b5972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff05e59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590d30 0x55ac6b596db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff05dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b5998b0 0x55ac6b5972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57738/0 (socket says 172.21.15.2:57738) 2024-03-15T13:08:33.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff05dd9b640 1 -- 172.21.15.2:0/131580729 learned_addr learned my addr 172.21.15.2:0/131580729 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:33.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff05dd9b640 1 --2- 172.21.15.2:0/131580729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b590380 0x55ac6b596870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff05e59c640 1 -- 172.21.15.2:0/131580729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b590380 msgr2=0x55ac6b596870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff05e59c640 1 --2- 172.21.15.2:0/131580729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b590380 0x55ac6b596870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff05e59c640 1 -- 172.21.15.2:0/131580729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b5998b0 msgr2=0x55ac6b5972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff05e59c640 1 --2- 172.21.15.2:0/131580729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b5998b0 0x55ac6b5972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff05e59c640 1 -- 172.21.15.2:0/131580729 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06010ec20 con 0x55ac6b590d30 2024-03-15T13:08:33.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff05dd9b640 1 --2- 172.21.15.2:0/131580729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b5998b0 0x55ac6b5972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:33.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff05e59c640 1 --2- 172.21.15.2:0/131580729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590d30 0x55ac6b596db0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55ac6b58bac0 tx=0x7ff050032910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b703f408ed07ee85 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:33.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.060+0000 7ff045ffb640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:08:33.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.061+0000 7ff045ffb640 1 -- 172.21.15.2:0/131580729 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050053070 con 0x55ac6b590d30 2024-03-15T13:08:33.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.061+0000 7ff045ffb640 1 -- 172.21.15.2:0/131580729 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ff050034650 con 0x55ac6b590d30 2024-03-15T13:08:33.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.061+0000 7ff067603740 1 -- 172.21.15.2:0/131580729 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac6b4d1f40 con 0x55ac6b590d30 2024-03-15T13:08:33.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.061+0000 7ff045ffb640 1 -- 172.21.15.2:0/131580729 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff050032b20 con 0x55ac6b590d30 2024-03-15T13:08:33.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.061+0000 7ff067603740 10 client.28364 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:33.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.061+0000 7ff067603740 1 -- 172.21.15.2:0/131580729 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55ac6b4f6320 con 0x55ac6b590d30 2024-03-15T13:08:33.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.061+0000 7ff045ffb640 1 -- 172.21.15.2:0/131580729 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55ac6b4d1f40 con 0x55ac6b590d30 2024-03-15T13:08:33.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.061+0000 7ff045ffb640 1 -- 172.21.15.2:0/131580729 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff050056020 con 0x55ac6b590d30 2024-03-15T13:08:33.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.061+0000 7ff067603740 10 client.28364 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:33.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.061+0000 7ff067603740 10 client.28364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:08:33.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.061+0000 7ff067603740 20 client.28364 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:33.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.061+0000 7ff067603740 1 --2- 172.21.15.2:0/131580729 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ac6b5a9aa0 0x55ac6b5abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.061+0000 7ff067603740 4 client.28364 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:08:33.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.061+0000 7ff067603740 1 -- 172.21.15.2:0/131580729 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac6b4ee310 con 0x55ac6b5a9aa0 2024-03-15T13:08:33.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.061+0000 7ff05dd9b640 1 --2- 172.21.15.2:0/131580729 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ac6b5a9aa0 0x55ac6b5abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.062+0000 7ff05dd9b640 1 --2- 172.21.15.2:0/131580729 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ac6b5a9aa0 0x55ac6b5abe90 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3806b82ca23100c7 server_cookie=eefc5a36371f7ce2 in_seq=0 out_seq=0 2024-03-15T13:08:33.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.062+0000 7ff045ffb640 10 client.28364 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:33.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.063+0000 7ff045ffb640 1 -- 172.21.15.2:0/131580729 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ac6b4ee310 con 0x55ac6b5a9aa0 2024-03-15T13:08:33.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.063+0000 7ff045ffb640 10 client.28364 handle_command_reply: tid=0 2024-03-15T13:08:33.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.063+0000 7ff067603740 1 client.28364 shutdown 2024-03-15T13:08:33.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.063+0000 7ff067603740 1 -- 172.21.15.2:0/131580729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590d30 msgr2=0x55ac6b596db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.063+0000 7ff067603740 1 --2- 172.21.15.2:0/131580729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590d30 0x55ac6b596db0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55ac6b58bac0 tx=0x7ff050032910 comp rx=0 tx=0).stop 2024-03-15T13:08:33.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.063+0000 7ff067603740 1 -- 172.21.15.2:0/131580729 shutdown_connections 2024-03-15T13:08:33.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.063+0000 7ff067603740 1 --2- 172.21.15.2:0/131580729 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ac6b5a9aa0 0x55ac6b5abe90 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.063+0000 7ff067603740 1 --2- 172.21.15.2:0/131580729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b5998b0 0x55ac6b5972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.063+0000 7ff045ffb640 0 client.28364 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:33.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.063+0000 7ff067603740 1 --2- 172.21.15.2:0/131580729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590d30 0x55ac6b596db0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.063+0000 7ff067603740 1 --2- 172.21.15.2:0/131580729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b590380 0x55ac6b596870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.063+0000 7ff067603740 1 -- 172.21.15.2:0/131580729 >> 172.21.15.2:0/131580729 conn(0x55ac6b58a840 msgr2=0x55ac6b58c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:33.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.063+0000 7ff067603740 1 -- 172.21.15.2:0/131580729 shutdown_connections 2024-03-15T13:08:33.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.063+0000 7ff067603740 1 -- 172.21.15.2:0/131580729 wait complete. 2024-03-15T13:08:33.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.063+0000 7ff067603740 20 client.28364 trim_cache size 0 max 16384 2024-03-15T13:08:33.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.079+0000 7ff067603740 1 Processor -- start 2024-03-15T13:08:33.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.079+0000 7ff067603740 1 -- start start 2024-03-15T13:08:33.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.079+0000 7ff067603740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b58a840 0x55ac6b598fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.079+0000 7ff067603740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590380 0x55ac6b596f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.079+0000 7ff067603740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b5a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.079+0000 7ff067603740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6b58a6d0 con 0x55ac6b590d30 2024-03-15T13:08:33.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.079+0000 7ff067603740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6b586110 con 0x55ac6b590380 2024-03-15T13:08:33.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.079+0000 7ff067603740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6b58a060 con 0x55ac6b58a840 2024-03-15T13:08:33.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.080+0000 7ff05dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b5a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.080+0000 7ff05e59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590380 0x55ac6b596f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.080+0000 7ff05e59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590380 0x55ac6b596f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:43820/0 (socket says 172.21.15.2:43820) 2024-03-15T13:08:33.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.080+0000 7ff05dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b5a4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57744/0 (socket says 172.21.15.2:57744) 2024-03-15T13:08:33.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.080+0000 7ff05e59c640 1 -- 172.21.15.2:0/1624869111 learned_addr learned my addr 172.21.15.2:0/1624869111 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:33.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.080+0000 7ff05dd9b640 1 --2- 172.21.15.2:0/1624869111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b58a840 0x55ac6b598fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.080+0000 7ff05e59c640 1 -- 172.21.15.2:0/1624869111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b58a840 msgr2=0x55ac6b598fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.080+0000 7ff05e59c640 1 --2- 172.21.15.2:0/1624869111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b58a840 0x55ac6b598fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.080+0000 7ff05e59c640 1 -- 172.21.15.2:0/1624869111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 msgr2=0x55ac6b5a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.080+0000 7ff05e59c640 1 --2- 172.21.15.2:0/1624869111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b5a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.080+0000 7ff05e59c640 1 -- 172.21.15.2:0/1624869111 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac6b492c00 con 0x55ac6b590380 2024-03-15T13:08:33.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.080+0000 7ff05e59c640 1 --2- 172.21.15.2:0/1624869111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590380 0x55ac6b596f40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff050004af0 tx=0x7ff050000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=654a668859faeb74 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:33.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.080+0000 7ff047fff640 1 -- 172.21.15.2:0/1624869111 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050053070 con 0x55ac6b590380 2024-03-15T13:08:33.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.080+0000 7ff047fff640 1 -- 172.21.15.2:0/1624869111 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff05002c7f0 con 0x55ac6b590380 2024-03-15T13:08:33.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.080+0000 7ff047fff640 1 -- 172.21.15.2:0/1624869111 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050004880 con 0x55ac6b590380 2024-03-15T13:08:33.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 -- 172.21.15.2:0/1624869111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590380 msgr2=0x55ac6b596f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 --2- 172.21.15.2:0/1624869111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590380 0x55ac6b596f40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff050004af0 tx=0x7ff050000f40 comp rx=0 tx=0).stop 2024-03-15T13:08:33.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 -- 172.21.15.2:0/1624869111 shutdown_connections 2024-03-15T13:08:33.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 --2- 172.21.15.2:0/1624869111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b5a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 --2- 172.21.15.2:0/1624869111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590380 0x55ac6b596f40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 --2- 172.21.15.2:0/1624869111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b58a840 0x55ac6b598fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 -- 172.21.15.2:0/1624869111 >> 172.21.15.2:0/1624869111 conn(0x55ac6b591480 msgr2=0x55ac6b596830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:33.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 -- 172.21.15.2:0/1624869111 shutdown_connections 2024-03-15T13:08:33.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 -- 172.21.15.2:0/1624869111 wait complete. 2024-03-15T13:08:33.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 -- start start 2024-03-15T13:08:33.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 Processor -- start 2024-03-15T13:08:33.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b58a840 0x55ac6b5a1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590380 0x55ac6b5a1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b593700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6b593dd0 con 0x55ac6b590d30 2024-03-15T13:08:33.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6b593f40 con 0x55ac6b590380 2024-03-15T13:08:33.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6b594100 con 0x55ac6b58a840 2024-03-15T13:08:33.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.081+0000 7ff067603740 10 client.0 _ll_register_callbacks cb 0x7ff048009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:33.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff05dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b593700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff05e59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590380 0x55ac6b5a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff05e59c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590380 0x55ac6b5a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:43826/0 (socket says 172.21.15.2:43826) 2024-03-15T13:08:33.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff05e59c640 1 -- 172.21.15.2:0/3405606244 learned_addr learned my addr 172.21.15.2:0/3405606244 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:33.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff05dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b593700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57758/0 (socket says 172.21.15.2:57758) 2024-03-15T13:08:33.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff05dd9b640 1 --2- 172.21.15.2:0/3405606244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b58a840 0x55ac6b5a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff05e59c640 1 -- 172.21.15.2:0/3405606244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b58a840 msgr2=0x55ac6b5a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff05e59c640 1 --2- 172.21.15.2:0/3405606244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b58a840 0x55ac6b5a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff05e59c640 1 -- 172.21.15.2:0/3405606244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 msgr2=0x55ac6b593700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff05e59c640 1 --2- 172.21.15.2:0/3405606244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b593700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff05e59c640 1 -- 172.21.15.2:0/3405606244 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac6b4f6320 con 0x55ac6b590380 2024-03-15T13:08:33.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff05e59c640 1 --2- 172.21.15.2:0/3405606244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590380 0x55ac6b5a1560 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff050006010 tx=0x7ff050032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=231457d30cd30346 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:33.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff045ffb640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-03-15T13:08:33.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff045ffb640 1 -- 172.21.15.2:0/3405606244 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050053040 con 0x55ac6b590380 2024-03-15T13:08:33.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff067603740 1 -- 172.21.15.2:0/3405606244 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac6b51d900 con 0x55ac6b590380 2024-03-15T13:08:33.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff045ffb640 1 -- 172.21.15.2:0/3405606244 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ff050032da0 con 0x55ac6b590380 2024-03-15T13:08:33.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff045ffb640 1 -- 172.21.15.2:0/3405606244 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff05002c7f0 con 0x55ac6b590380 2024-03-15T13:08:33.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff067603740 10 client.28370 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:33.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.082+0000 7ff067603740 1 -- 172.21.15.2:0/3405606244 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55ac6b4b8910 con 0x55ac6b590380 2024-03-15T13:08:33.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.083+0000 7ff045ffb640 1 -- 172.21.15.2:0/3405606244 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55ac6b51d900 con 0x55ac6b590380 2024-03-15T13:08:33.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.083+0000 7ff045ffb640 1 -- 172.21.15.2:0/3405606244 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7ff050056070 con 0x55ac6b590380 2024-03-15T13:08:33.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.083+0000 7ff067603740 10 client.28370 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:33.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.083+0000 7ff067603740 10 client.28370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:08:33.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.083+0000 7ff067603740 20 client.28370 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:33.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.083+0000 7ff067603740 1 --2- 172.21.15.2:0/3405606244 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ac6b5998b0 0x55ac6b6081c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.083+0000 7ff067603740 4 client.28370 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-15T13:08:33.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.083+0000 7ff067603740 1 -- 172.21.15.2:0/3405606244 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ac6b1f7860 con 0x55ac6b5998b0 2024-03-15T13:08:33.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.083+0000 7ff05dd9b640 1 --2- 172.21.15.2:0/3405606244 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ac6b5998b0 0x55ac6b6081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.083+0000 7ff05dd9b640 1 --2- 172.21.15.2:0/3405606244 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ac6b5998b0 0x55ac6b6081c0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efd7f6c5d7b86f64 server_cookie=e5a52997e60a49fc in_seq=0 out_seq=0 2024-03-15T13:08:33.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.083+0000 7ff045ffb640 10 client.28370 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:33.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.084+0000 7ff045ffb640 1 -- 172.21.15.2:0/3405606244 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ac6b1f7860 con 0x55ac6b5998b0 2024-03-15T13:08:33.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.084+0000 7ff045ffb640 10 client.28370 handle_command_reply: tid=0 2024-03-15T13:08:33.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.084+0000 7ff067603740 1 client.28370 shutdown 2024-03-15T13:08:33.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff067603740 1 -- 172.21.15.2:0/3405606244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590380 msgr2=0x55ac6b5a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff067603740 1 --2- 172.21.15.2:0/3405606244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590380 0x55ac6b5a1560 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff050006010 tx=0x7ff050032b20 comp rx=0 tx=0).stop 2024-03-15T13:08:33.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff067603740 1 -- 172.21.15.2:0/3405606244 shutdown_connections 2024-03-15T13:08:33.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff067603740 1 --2- 172.21.15.2:0/3405606244 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55ac6b5998b0 0x55ac6b6081c0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff067603740 1 --2- 172.21.15.2:0/3405606244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ac6b590d30 0x55ac6b593700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff067603740 1 --2- 172.21.15.2:0/3405606244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac6b590380 0x55ac6b5a1560 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff045ffb640 0 client.28370 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:33.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff067603740 1 --2- 172.21.15.2:0/3405606244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ac6b58a840 0x55ac6b5a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff067603740 1 -- 172.21.15.2:0/3405606244 >> 172.21.15.2:0/3405606244 conn(0x55ac6b591480 msgr2=0x55ac6b5898e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:33.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff067603740 1 -- 172.21.15.2:0/3405606244 shutdown_connections 2024-03-15T13:08:33.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff067603740 1 -- 172.21.15.2:0/3405606244 wait complete. 2024-03-15T13:08:33.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff067603740 20 client.28370 trim_cache size 0 max 16384 2024-03-15T13:08:33.076 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:08:33.077 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2024-03-15T13:08:33.077 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "92217670-cfe3-480e-a014-f8e4f90b7a78", 2024-03-15T13:08:33.077 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2024-03-15T13:08:33.077 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:08:33.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff064a23640 1 -- 172.21.15.2:0/1609627220 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff04c0a1ba0 msgr2=0x7ff04c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff064a23640 1 --2- 172.21.15.2:0/1609627220 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff04c0a1ba0 0x7ff04c0a4060 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff05809a740 tx=0x7ff058085520 comp rx=0 tx=0).stop 2024-03-15T13:08:33.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff064a23640 1 -- 172.21.15.2:0/1609627220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0601074f0 msgr2=0x7ff060100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff064a23640 1 --2- 172.21.15.2:0/1609627220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0601074f0 0x7ff060100e30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ff05800e970 tx=0x7ff05800ed70 comp rx=0 tx=0).stop 2024-03-15T13:08:33.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff064a23640 1 -- 172.21.15.2:0/1609627220 shutdown_connections 2024-03-15T13:08:33.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff064a23640 1 --2- 172.21.15.2:0/1609627220 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff04c0a1ba0 0x7ff04c0a4060 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff064a23640 1 --2- 172.21.15.2:0/1609627220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0601074f0 0x7ff060100e30 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff064a23640 1 --2- 172.21.15.2:0/1609627220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0600690b0 0x7ff0601008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff064a23640 1 --2- 172.21.15.2:0/1609627220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0600686f0 0x7ff0601003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff064a23640 1 -- 172.21.15.2:0/1609627220 >> 172.21.15.2:0/1609627220 conn(0x7ff0600774f0 msgr2=0x7ff0600ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:33.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff064a23640 1 -- 172.21.15.2:0/1609627220 shutdown_connections 2024-03-15T13:08:33.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.085+0000 7ff064a23640 1 -- 172.21.15.2:0/1609627220 wait complete. 2024-03-15T13:08:33.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:08:33.081 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:08:33.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.202+0000 7f87f3eff640 1 -- 172.21.15.2:0/3521645599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87ec101e20 msgr2=0x7f87ec10e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.202+0000 7f87f3eff640 1 --2- 172.21.15.2:0/3521645599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87ec101e20 0x7f87ec10e330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009ca0 tx=0x7f87e402c0f0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.202+0000 7f87f3eff640 1 -- 172.21.15.2:0/3521645599 shutdown_connections 2024-03-15T13:08:33.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.202+0000 7f87f3eff640 1 --2- 172.21.15.2:0/3521645599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87ec101e20 0x7f87ec10e330 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.202+0000 7f87f3eff640 1 --2- 172.21.15.2:0/3521645599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87ec101460 0x7f87ec1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.202+0000 7f87f3eff640 1 --2- 172.21.15.2:0/3521645599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87ec107500 0x7f87ec1078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.202+0000 7f87f3eff640 1 -- 172.21.15.2:0/3521645599 >> 172.21.15.2:0/3521645599 conn(0x7f87ec0fcf90 msgr2=0x7f87ec0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:33.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.202+0000 7f87f3eff640 1 -- 172.21.15.2:0/3521645599 shutdown_connections 2024-03-15T13:08:33.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.202+0000 7f87f3eff640 1 -- 172.21.15.2:0/3521645599 wait complete. 2024-03-15T13:08:33.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f3eff640 1 Processor -- start 2024-03-15T13:08:33.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f3eff640 1 -- start start 2024-03-15T13:08:33.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f3eff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87ec101460 0x7f87ec197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f3eff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87ec101e20 0x7f87ec1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f3eff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87ec107500 0x7f87ec19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f3eff640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec19ca50 con 0x7f87ec101e20 2024-03-15T13:08:33.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f3eff640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec19cbc0 con 0x7f87ec101460 2024-03-15T13:08:33.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f3eff640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec19cd80 con 0x7f87ec107500 2024-03-15T13:08:33.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f149a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87ec101e20 0x7f87ec1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f149a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87ec101e20 0x7f87ec1980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57764/0 (socket says 172.21.15.2:57764) 2024-03-15T13:08:33.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f149a640 1 -- 172.21.15.2:0/381086535 learned_addr learned my addr 172.21.15.2:0/381086535 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:33.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f1c9b640 1 --2- 172.21.15.2:0/381086535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87ec101460 0x7f87ec197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f249c640 1 --2- 172.21.15.2:0/381086535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87ec107500 0x7f87ec19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f149a640 1 -- 172.21.15.2:0/381086535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87ec107500 msgr2=0x7f87ec19c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f149a640 1 --2- 172.21.15.2:0/381086535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87ec107500 0x7f87ec19c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f149a640 1 -- 172.21.15.2:0/381086535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87ec101460 msgr2=0x7f87ec197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f149a640 1 --2- 172.21.15.2:0/381086535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87ec101460 0x7f87ec197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.203+0000 7f87f149a640 1 -- 172.21.15.2:0/381086535 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e8009490 con 0x7f87ec101e20 2024-03-15T13:08:33.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.204+0000 7f87f149a640 1 --2- 172.21.15.2:0/381086535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87ec101e20 0x7f87ec1980a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00e9f0 tx=0x7f87dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:33.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.204+0000 7f87e2ffd640 1 -- 172.21.15.2:0/381086535 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc009700 con 0x7f87ec101e20 2024-03-15T13:08:33.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.204+0000 7f87f3eff640 1 -- 172.21.15.2:0/381086535 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e4009900 con 0x7f87ec101e20 2024-03-15T13:08:33.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.204+0000 7f87e2ffd640 1 -- 172.21.15.2:0/381086535 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f87dc00cd80 con 0x7f87ec101e20 2024-03-15T13:08:33.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.204+0000 7f87e2ffd640 1 -- 172.21.15.2:0/381086535 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc018920 con 0x7f87ec101e20 2024-03-15T13:08:33.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.204+0000 7f87f3eff640 1 -- 172.21.15.2:0/381086535 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec1a4700 con 0x7f87ec101e20 2024-03-15T13:08:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.206+0000 7f87e2ffd640 1 -- 172.21.15.2:0/381086535 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f87dc018a80 con 0x7f87ec101e20 2024-03-15T13:08:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.206+0000 7f87f6bb5740 1 Processor -- start 2024-03-15T13:08:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f6bb5740 1 -- start start 2024-03-15T13:08:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f6bb5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c703c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f6bb5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c703c900 0x55d1c7049680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f6bb5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c704bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f6bb5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c6fab280 con 0x55d1c7049bc0 2024-03-15T13:08:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f6bb5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c6fa87b0 con 0x55d1c703bfe0 2024-03-15T13:08:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f6bb5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c6fa3f90 con 0x55d1c703c900 2024-03-15T13:08:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87e2ffd640 1 --2- 172.21.15.2:0/381086535 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f87d80a1ba0 0x7f87d80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f149a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c704bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f249c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c703c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f1c9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c703c900 0x55d1c7049680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f249c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c703c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:43850/0 (socket says 172.21.15.2:43850) 2024-03-15T13:08:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f249c640 1 -- 172.21.15.2:0/3364110373 learned_addr learned my addr 172.21.15.2:0/3364110373 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f249c640 1 --2- 172.21.15.2:0/381086535 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f87d80a1ba0 0x7f87d80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87e2ffd640 1 -- 172.21.15.2:0/381086535 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f87dc014070 con 0x7f87ec101e20 2024-03-15T13:08:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f149a640 1 -- 172.21.15.2:0/3364110373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c703c900 msgr2=0x55d1c7049680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f149a640 1 --2- 172.21.15.2:0/3364110373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c703c900 0x55d1c7049680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f149a640 1 -- 172.21.15.2:0/3364110373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 msgr2=0x55d1c703c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f149a640 1 --2- 172.21.15.2:0/3364110373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c703c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f149a640 1 -- 172.21.15.2:0/3364110373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1c6f62570 con 0x55d1c7049bc0 2024-03-15T13:08:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.207+0000 7f87f249c640 1 --2- 172.21.15.2:0/3364110373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c703c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.208+0000 7f87f149a640 1 --2- 172.21.15.2:0/3364110373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c704bfb0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0cb270 tx=0x7f87dc06db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb63ce637daaffb server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.208+0000 7f87f249c640 1 --2- 172.21.15.2:0/381086535 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f87d80a1ba0 0x7f87d80a4060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009430 tx=0x7f87e4005c50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.208+0000 7f87d7fff640 1 -- 172.21.15.2:0/3364110373 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc098fb0 con 0x55d1c7049bc0 2024-03-15T13:08:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.208+0000 7f87d7fff640 1 -- 172.21.15.2:0/3364110373 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f87dc098dc0 con 0x55d1c7049bc0 2024-03-15T13:08:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.208+0000 7f87d7fff640 1 -- 172.21.15.2:0/3364110373 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc096e70 con 0x55d1c7049bc0 2024-03-15T13:08:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.208+0000 7f87f6bb5740 1 -- 172.21.15.2:0/3364110373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 msgr2=0x55d1c704bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.208+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/3364110373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c704bfb0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0cb270 tx=0x7f87dc06db70 comp rx=0 tx=0).stop 2024-03-15T13:08:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.208+0000 7f87f6bb5740 1 -- 172.21.15.2:0/3364110373 shutdown_connections 2024-03-15T13:08:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.208+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/3364110373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c704bfb0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.208+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/3364110373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c703c900 0x55d1c7049680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.208+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/3364110373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c703c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.208+0000 7f87f6bb5740 1 -- 172.21.15.2:0/3364110373 >> 172.21.15.2:0/3364110373 conn(0x55d1c7036990 msgr2=0x55d1c7036d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f6bb5740 1 -- 172.21.15.2:0/3364110373 shutdown_connections 2024-03-15T13:08:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f6bb5740 1 -- 172.21.15.2:0/3364110373 wait complete. 2024-03-15T13:08:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f6bb5740 1 -- start start 2024-03-15T13:08:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f6bb5740 1 Processor -- start 2024-03-15T13:08:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f6bb5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c703bfe0 0x55d1c7049660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f6bb5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703c900 0x55d1c70469f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f6bb5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c7049bc0 0x55d1c70470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f6bb5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c6c77430 con 0x55d1c703bfe0 2024-03-15T13:08:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f6bb5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c6c33e00 con 0x55d1c703c900 2024-03-15T13:08:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f6bb5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c6f28fd0 con 0x55d1c7049bc0 2024-03-15T13:08:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f6bb5740 10 client.0 _ll_register_callbacks cb 0x55d1c6f7efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f1c9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c703bfe0 0x55d1c7049660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f1c9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c703bfe0 0x55d1c7049660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57774/0 (socket says 172.21.15.2:57774) 2024-03-15T13:08:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f1c9b640 1 -- 172.21.15.2:0/2197954764 learned_addr learned my addr 172.21.15.2:0/2197954764 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f249c640 1 --2- 172.21.15.2:0/2197954764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c7049bc0 0x55d1c70470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.209+0000 7f87f149a640 1 --2- 172.21.15.2:0/2197954764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703c900 0x55d1c70469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.210+0000 7f87f1c9b640 1 -- 172.21.15.2:0/2197954764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c7049bc0 msgr2=0x55d1c70470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.210+0000 7f87f1c9b640 1 --2- 172.21.15.2:0/2197954764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c7049bc0 0x55d1c70470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.210+0000 7f87f1c9b640 1 -- 172.21.15.2:0/2197954764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703c900 msgr2=0x55d1c70469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.210+0000 7f87f1c9b640 1 --2- 172.21.15.2:0/2197954764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703c900 0x55d1c70469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.210+0000 7f87f1c9b640 1 -- 172.21.15.2:0/2197954764 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec1a4700 con 0x55d1c703bfe0 2024-03-15T13:08:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.210+0000 7f87f1c9b640 1 --2- 172.21.15.2:0/2197954764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c703bfe0 0x55d1c7049660 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f87e80128d0 tx=0x7f87e8012d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64323422a732687e server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:33.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.210+0000 7f87d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:33.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.210+0000 7f87d57fa640 1 -- 172.21.15.2:0/2197954764 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8017070 con 0x55d1c703bfe0 2024-03-15T13:08:33.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.210+0000 7f87d57fa640 1 -- 172.21.15.2:0/2197954764 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f87e8004030 con 0x55d1c703bfe0 2024-03-15T13:08:33.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.210+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2197954764 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1c6ca37e0 con 0x55d1c703bfe0 2024-03-15T13:08:33.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.210+0000 7f87d57fa640 1 -- 172.21.15.2:0/2197954764 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87e80122e0 con 0x55d1c703bfe0 2024-03-15T13:08:33.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.210+0000 7f87f6bb5740 10 client.18519 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:33.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.210+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2197954764 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55d1c6fa2610 con 0x55d1c703bfe0 2024-03-15T13:08:33.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.210+0000 7f87d57fa640 1 -- 172.21.15.2:0/2197954764 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55d1c6ca37e0 con 0x55d1c703bfe0 2024-03-15T13:08:33.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.211+0000 7f87d57fa640 1 -- 172.21.15.2:0/2197954764 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f87e8003850 con 0x55d1c703bfe0 2024-03-15T13:08:33.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.211+0000 7f87f6bb5740 10 client.18519 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:33.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.211+0000 7f87f6bb5740 10 client.18519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:08:33.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.211+0000 7f87f6bb5740 20 client.18519 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:33.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.211+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/2197954764 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d1c7059eb0 0x55d1c705c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.211+0000 7f87f6bb5740 4 client.18519 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:08:33.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.211+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2197954764 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1c6ce0720 con 0x55d1c7059eb0 2024-03-15T13:08:33.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.211+0000 7f87f149a640 1 --2- 172.21.15.2:0/2197954764 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d1c7059eb0 0x55d1c705c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.211+0000 7f87d57fa640 10 client.18519 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:33.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.211+0000 7f87f149a640 1 --2- 172.21.15.2:0/2197954764 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d1c7059eb0 0x55d1c705c2f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bed23d2b3e15c200 server_cookie=6db629856de909df in_seq=0 out_seq=0 2024-03-15T13:08:33.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.212+0000 7f87d57fa640 1 -- 172.21.15.2:0/2197954764 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d1c6ce0720 con 0x55d1c7059eb0 2024-03-15T13:08:33.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.212+0000 7f87d57fa640 10 client.18519 handle_command_reply: tid=0 2024-03-15T13:08:33.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.212+0000 7f87f6bb5740 1 client.18519 shutdown 2024-03-15T13:08:33.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.212+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2197954764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c703bfe0 msgr2=0x55d1c7049660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.212+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/2197954764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c703bfe0 0x55d1c7049660 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f87e80128d0 tx=0x7f87e8012d90 comp rx=0 tx=0).stop 2024-03-15T13:08:33.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.212+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2197954764 shutdown_connections 2024-03-15T13:08:33.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.212+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/2197954764 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d1c7059eb0 0x55d1c705c2f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.212+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/2197954764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c7049bc0 0x55d1c70470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.212+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/2197954764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703c900 0x55d1c70469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.212+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/2197954764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c703bfe0 0x55d1c7049660 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.212+0000 7f87d57fa640 0 client.18519 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:33.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.212+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2197954764 >> 172.21.15.2:0/2197954764 conn(0x55d1c7036990 msgr2=0x55d1c7038730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:33.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.212+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2197954764 shutdown_connections 2024-03-15T13:08:33.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.212+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2197954764 wait complete. 2024-03-15T13:08:33.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.213+0000 7f87f6bb5740 20 client.18519 trim_cache size 0 max 16384 2024-03-15T13:08:33.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.228+0000 7f87f6bb5740 1 Processor -- start 2024-03-15T13:08:33.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.228+0000 7f87f6bb5740 1 -- start start 2024-03-15T13:08:33.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f6bb5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c70358d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f6bb5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c703c900 0x55d1c7049120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f6bb5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c7049660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f6bb5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c70568c0 con 0x55d1c7049bc0 2024-03-15T13:08:33.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f6bb5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c7030e90 con 0x55d1c703bfe0 2024-03-15T13:08:33.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f6bb5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c703bc40 con 0x55d1c703c900 2024-03-15T13:08:33.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f249c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c7049660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f1c9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c70358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f249c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c7049660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57790/0 (socket says 172.21.15.2:57790) 2024-03-15T13:08:33.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f249c640 1 -- 172.21.15.2:0/4294686219 learned_addr learned my addr 172.21.15.2:0/4294686219 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:33.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f149a640 1 --2- 172.21.15.2:0/4294686219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c703c900 0x55d1c7049120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f249c640 1 -- 172.21.15.2:0/4294686219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c703c900 msgr2=0x55d1c7049120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f249c640 1 --2- 172.21.15.2:0/4294686219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c703c900 0x55d1c7049120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f249c640 1 -- 172.21.15.2:0/4294686219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 msgr2=0x55d1c70358d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f249c640 1 --2- 172.21.15.2:0/4294686219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c70358d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f249c640 1 -- 172.21.15.2:0/4294686219 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1c6f3eac0 con 0x55d1c7049bc0 2024-03-15T13:08:33.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f1c9b640 1 --2- 172.21.15.2:0/4294686219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c70358d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:33.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87f249c640 1 --2- 172.21.15.2:0/4294686219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c7049660 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009ca0 tx=0x7f87e4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b2ec77aaba29fc server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:33.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87d77fe640 1 -- 172.21.15.2:0/4294686219 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4054070 con 0x55d1c7049bc0 2024-03-15T13:08:33.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87d77fe640 1 -- 172.21.15.2:0/4294686219 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f87e4049050 con 0x55d1c7049bc0 2024-03-15T13:08:33.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.229+0000 7f87d77fe640 1 -- 172.21.15.2:0/4294686219 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4003bd0 con 0x55d1c7049bc0 2024-03-15T13:08:33.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.230+0000 7f87f6bb5740 1 -- 172.21.15.2:0/4294686219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 msgr2=0x55d1c7049660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.230+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/4294686219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c7049660 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009ca0 tx=0x7f87e4009360 comp rx=0 tx=0).stop 2024-03-15T13:08:33.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.230+0000 7f87f6bb5740 1 -- 172.21.15.2:0/4294686219 shutdown_connections 2024-03-15T13:08:33.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.230+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/4294686219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c7049660 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.230+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/4294686219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c703c900 0x55d1c7049120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.230+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/4294686219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c70358d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.230+0000 7f87f6bb5740 1 -- 172.21.15.2:0/4294686219 >> 172.21.15.2:0/4294686219 conn(0x55d1c7036990 msgr2=0x55d1c70351c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:33.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.230+0000 7f87f6bb5740 1 -- 172.21.15.2:0/4294686219 shutdown_connections 2024-03-15T13:08:33.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.230+0000 7f87f6bb5740 1 -- 172.21.15.2:0/4294686219 wait complete. 2024-03-15T13:08:33.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.230+0000 7f87f6bb5740 1 -- start start 2024-03-15T13:08:33.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.230+0000 7f87f6bb5740 1 Processor -- start 2024-03-15T13:08:33.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f6bb5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c70512f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f6bb5740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c703c900 0x55d1c7051830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f6bb5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c7043980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f6bb5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c7044080 con 0x55d1c7049bc0 2024-03-15T13:08:33.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f6bb5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c70441f0 con 0x55d1c703bfe0 2024-03-15T13:08:33.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f6bb5740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c70443b0 con 0x55d1c703c900 2024-03-15T13:08:33.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f6bb5740 10 client.0 _ll_register_callbacks cb 0x7f87e80099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:08:33.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f249c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c7043980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f249c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c7043980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57796/0 (socket says 172.21.15.2:57796) 2024-03-15T13:08:33.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f1c9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c70512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f249c640 1 -- 172.21.15.2:0/2077149407 learned_addr learned my addr 172.21.15.2:0/2077149407 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:33.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f149a640 1 --2- 172.21.15.2:0/2077149407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c703c900 0x55d1c7051830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f249c640 1 -- 172.21.15.2:0/2077149407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c703c900 msgr2=0x55d1c7051830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f249c640 1 --2- 172.21.15.2:0/2077149407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c703c900 0x55d1c7051830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f249c640 1 -- 172.21.15.2:0/2077149407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 msgr2=0x55d1c70512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:08:33.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f249c640 1 --2- 172.21.15.2:0/2077149407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c70512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f249c640 1 -- 172.21.15.2:0/2077149407 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1c6fa2610 con 0x55d1c7049bc0 2024-03-15T13:08:33.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f1c9b640 1 --2- 172.21.15.2:0/2077149407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c70512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:08:33.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f249c640 1 --2- 172.21.15.2:0/2077149407 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c7043980 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f87e4037040 tx=0x7f87e4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0375f72a5fd782 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:33.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:08:33.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87d57fa640 1 -- 172.21.15.2:0/2077149407 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4054040 con 0x55d1c7049bc0 2024-03-15T13:08:33.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2077149407 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87dc098dc0 con 0x55d1c7049bc0 2024-03-15T13:08:33.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.231+0000 7f87d57fa640 1 -- 172.21.15.2:0/2077149407 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f87e40328c0 con 0x55d1c7049bc0 2024-03-15T13:08:33.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.232+0000 7f87d57fa640 1 -- 172.21.15.2:0/2077149407 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87e4032a80 con 0x55d1c7049bc0 2024-03-15T13:08:33.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.232+0000 7f87f6bb5740 10 client.18525 fetch_fsmap learned FSMap version 50 2024-03-15T13:08:33.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.232+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2077149407 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f87e4030070 con 0x55d1c7049bc0 2024-03-15T13:08:33.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.232+0000 7f87d57fa640 1 -- 172.21.15.2:0/2077149407 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f87dc098dc0 con 0x55d1c7049bc0 2024-03-15T13:08:33.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.232+0000 7f87d57fa640 1 -- 172.21.15.2:0/2077149407 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f87e4050420 con 0x55d1c7049bc0 2024-03-15T13:08:33.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.232+0000 7f87f6bb5740 10 client.18525 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:08:33.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.232+0000 7f87f6bb5740 10 client.18525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:08:33.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.232+0000 7f87f6bb5740 20 client.18525 populate_metadata read hostname 'smithi002' 2024-03-15T13:08:33.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.232+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/2077149407 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d1c70b67e0 0x55d1c70b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:33.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.232+0000 7f87f6bb5740 4 client.18525 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:08:33.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.232+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2077149407 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d1c6d2f890 con 0x55d1c70b67e0 2024-03-15T13:08:33.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.232+0000 7f87f1c9b640 1 --2- 172.21.15.2:0/2077149407 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d1c70b67e0 0x55d1c70b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:33.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f1c9b640 1 --2- 172.21.15.2:0/2077149407 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d1c70b67e0 0x55d1c70b8c00 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21e5880540f48683 server_cookie=666ea201b8e34a3f in_seq=0 out_seq=0 2024-03-15T13:08:33.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87d57fa640 10 client.18525 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:33.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87d57fa640 1 -- 172.21.15.2:0/2077149407 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d1c6d2f890 con 0x55d1c70b67e0 2024-03-15T13:08:33.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87d57fa640 10 client.18525 handle_command_reply: tid=0 2024-03-15T13:08:33.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f6bb5740 1 client.18525 shutdown 2024-03-15T13:08:33.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2077149407 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 msgr2=0x55d1c7043980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/2077149407 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c7043980 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f87e4037040 tx=0x7f87e4009560 comp rx=0 tx=0).stop 2024-03-15T13:08:33.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2077149407 shutdown_connections 2024-03-15T13:08:33.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/2077149407 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55d1c70b67e0 0x55d1c70b8c00 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/2077149407 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1c7049bc0 0x55d1c7043980 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87d57fa640 0 client.18525 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:08:33.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/2077149407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1c703c900 0x55d1c7051830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f6bb5740 1 --2- 172.21.15.2:0/2077149407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1c703bfe0 0x55d1c70512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2077149407 >> 172.21.15.2:0/2077149407 conn(0x55d1c7036990 msgr2=0x55d1c7035a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:33.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2077149407 shutdown_connections 2024-03-15T13:08:33.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f6bb5740 1 -- 172.21.15.2:0/2077149407 wait complete. 2024-03-15T13:08:33.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f6bb5740 20 client.18525 trim_cache size 0 max 16384 2024-03-15T13:08:33.224 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:08:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-15T13:08:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2024-03-15T13:08:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "92217670-cfe3-480e-a014-f8e4f90b7a78": { 2024-03-15T13:08:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2024-03-15T13:08:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "92217670-cfe3-480e-a014-f8e4f90b7a78", 2024-03-15T13:08:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2024-03-15T13:08:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:08:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:08:33.225 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:08:33.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f3eff640 1 -- 172.21.15.2:0/381086535 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f87d80a1ba0 msgr2=0x7f87d80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f3eff640 1 --2- 172.21.15.2:0/381086535 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f87d80a1ba0 0x7f87d80a4060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009430 tx=0x7f87e4005c50 comp rx=0 tx=0).stop 2024-03-15T13:08:33.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f3eff640 1 -- 172.21.15.2:0/381086535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87ec101e20 msgr2=0x7f87ec1980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:33.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.233+0000 7f87f3eff640 1 --2- 172.21.15.2:0/381086535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87ec101e20 0x7f87ec1980a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00e9f0 tx=0x7f87dc00eeb0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.234+0000 7f87f3eff640 1 -- 172.21.15.2:0/381086535 shutdown_connections 2024-03-15T13:08:33.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.234+0000 7f87f3eff640 1 --2- 172.21.15.2:0/381086535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87ec107500 0x7f87ec19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.234+0000 7f87f3eff640 1 --2- 172.21.15.2:0/381086535 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f87d80a1ba0 0x7f87d80a4060 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.234+0000 7f87f3eff640 1 --2- 172.21.15.2:0/381086535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87ec101e20 0x7f87ec1980a0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.234+0000 7f87f3eff640 1 --2- 172.21.15.2:0/381086535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87ec101460 0x7f87ec197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:33.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.234+0000 7f87f3eff640 1 -- 172.21.15.2:0/381086535 >> 172.21.15.2:0/381086535 conn(0x7f87ec0fcf90 msgr2=0x7f87ec0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:33.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.234+0000 7f87f3eff640 1 -- 172.21.15.2:0/381086535 shutdown_connections 2024-03-15T13:08:33.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:33.234+0000 7f87f3eff640 1 -- 172.21.15.2:0/381086535 wait complete. 2024-03-15T13:08:33.228 INFO:tasks.cephfs.filesystem:scrub status for tag:92217670-cfe3-480e-a014-f8e4f90b7a78 - {'path': '/', 'tag': '92217670-cfe3-480e-a014-f8e4f90b7a78', 'options': 'recursive,force'} 2024-03-15T13:08:33.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:33 smithi002 ceph-mon[32060]: pgmap v298: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:08:33.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:33 smithi002 ceph-mon[32060]: scrub queued for path: / 2024-03-15T13:08:33.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:33 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:08:33.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:33 smithi002 ceph-mon[32060]: scrub summary: active paths [/] 2024-03-15T13:08:33.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:33 smithi002 ceph-mon[32060]: scrub summary: idle+waiting paths [/] 2024-03-15T13:08:33.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:33 smithi184 ceph-mon[35993]: pgmap v298: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:08:33.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:33 smithi184 ceph-mon[35993]: scrub queued for path: / 2024-03-15T13:08:33.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:33 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:08:33.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:33 smithi184 ceph-mon[35993]: scrub summary: active paths [/] 2024-03-15T13:08:33.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:33 smithi184 ceph-mon[35993]: scrub summary: idle+waiting paths [/] 2024-03-15T13:08:33.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:33 smithi149 ceph-mon[35927]: pgmap v298: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:08:33.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:33 smithi149 ceph-mon[35927]: scrub queued for path: / 2024-03-15T13:08:33.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:33 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:08:33.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:33 smithi149 ceph-mon[35927]: scrub summary: active paths [/] 2024-03-15T13:08:33.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:33 smithi149 ceph-mon[35927]: scrub summary: idle+waiting paths [/] 2024-03-15T13:08:35.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:35 smithi002 ceph-mon[32060]: pgmap v299: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-15T13:08:35.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:35 smithi002 ceph-mon[32060]: scrub summary: idle 2024-03-15T13:08:35.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:35 smithi184 ceph-mon[35993]: pgmap v299: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-15T13:08:35.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:35 smithi184 ceph-mon[35993]: scrub summary: idle 2024-03-15T13:08:35.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:35 smithi149 ceph-mon[35927]: pgmap v299: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-15T13:08:35.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:35 smithi149 ceph-mon[35927]: scrub summary: idle 2024-03-15T13:08:37.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:37 smithi002 ceph-mon[32060]: pgmap v300: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-03-15T13:08:37.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:37 smithi184 ceph-mon[35993]: pgmap v300: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-03-15T13:08:37.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:37 smithi149 ceph-mon[35927]: pgmap v300: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-03-15T13:08:39.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:39 smithi184 ceph-mon[35993]: pgmap v301: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:39.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:39 smithi149 ceph-mon[35927]: pgmap v301: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:40.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:39 smithi002 ceph-mon[32060]: pgmap v301: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:41.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:41 smithi184 ceph-mon[35993]: pgmap v302: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-03-15T13:08:41.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:41 smithi149 ceph-mon[35927]: pgmap v302: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-03-15T13:08:42.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:41 smithi002 ceph-mon[32060]: pgmap v302: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-03-15T13:08:43.853 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:43 smithi002 ceph-mon[32060]: pgmap v303: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:08:43.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:43 smithi149 ceph-mon[35927]: pgmap v303: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:08:44.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:43 smithi184 ceph-mon[35993]: pgmap v303: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:08:45.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:45 smithi149 ceph-mon[35927]: pgmap v304: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:08:46.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:45 smithi002 ceph-mon[32060]: pgmap v304: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:08:46.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:45 smithi184 ceph-mon[35993]: pgmap v304: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-15T13:08:47.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:47 smithi149 ceph-mon[35927]: pgmap v305: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:08:48.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:47 smithi002 ceph-mon[32060]: pgmap v305: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:08:48.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:47 smithi184 ceph-mon[35993]: pgmap v305: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-15T13:08:49.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:49 smithi149 ceph-mon[35927]: pgmap v306: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:50.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:49 smithi002 ceph-mon[32060]: pgmap v306: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:50.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:49 smithi184 ceph-mon[35993]: pgmap v306: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:51.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:51 smithi149 ceph-mon[35927]: pgmap v307: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:08:52.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:51 smithi002 ceph-mon[32060]: pgmap v307: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:08:52.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:51 smithi184 ceph-mon[35993]: pgmap v307: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:08:53.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:53 smithi149 ceph-mon[35927]: pgmap v308: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:54.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:53 smithi002 ceph-mon[32060]: pgmap v308: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:54.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:53 smithi184 ceph-mon[35993]: pgmap v308: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:55.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:55 smithi149 ceph-mon[35927]: pgmap v309: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:56.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:55 smithi002 ceph-mon[32060]: pgmap v309: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:56.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:55 smithi184 ceph-mon[35993]: pgmap v309: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:08:57.032 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-15T13:08:57.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.163+0000 7ffb3af77640 1 -- 172.21.15.2:0/539241161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb34100640 msgr2=0x7ffb34100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:57.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.163+0000 7ffb3af77640 1 --2- 172.21.15.2:0/539241161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb34100640 0x7ffb34100ac0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009900 tx=0x7ffb2802bc30 comp rx=0 tx=0).stop 2024-03-15T13:08:57.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.163+0000 7ffb3af77640 1 -- 172.21.15.2:0/539241161 shutdown_connections 2024-03-15T13:08:57.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.163+0000 7ffb3af77640 1 --2- 172.21.15.2:0/539241161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb34101000 0x7ffb3410d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.163+0000 7ffb3af77640 1 --2- 172.21.15.2:0/539241161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb34100640 0x7ffb34100ac0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.163+0000 7ffb3af77640 1 --2- 172.21.15.2:0/539241161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb34105390 0x7ffb34105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.163+0000 7ffb3af77640 1 -- 172.21.15.2:0/539241161 >> 172.21.15.2:0/539241161 conn(0x7ffb34077240 msgr2=0x7ffb34077650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:57.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.163+0000 7ffb3af77640 1 -- 172.21.15.2:0/539241161 shutdown_connections 2024-03-15T13:08:57.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.163+0000 7ffb3af77640 1 -- 172.21.15.2:0/539241161 wait complete. 2024-03-15T13:08:57.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.163+0000 7ffb3af77640 1 Processor -- start 2024-03-15T13:08:57.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.163+0000 7ffb3af77640 1 -- start start 2024-03-15T13:08:57.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb3af77640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb34100640 0x7ffb3419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:57.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb3af77640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb34101000 0x7ffb3419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:57.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb3af77640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb34105390 0x7ffb34197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:57.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb3af77640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb34197fd0 con 0x7ffb34100640 2024-03-15T13:08:57.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb3af77640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb34198140 con 0x7ffb34101000 2024-03-15T13:08:57.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb3af77640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb34198300 con 0x7ffb34105390 2024-03-15T13:08:57.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb38d13640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb34100640 0x7ffb3419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:57.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb38d13640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb34100640 0x7ffb3419d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46006/0 (socket says 172.21.15.2:46006) 2024-03-15T13:08:57.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb38d13640 1 -- 172.21.15.2:0/1427210879 learned_addr learned my addr 172.21.15.2:0/1427210879 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:57.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb33fff640 1 --2- 172.21.15.2:0/1427210879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb34101000 0x7ffb3419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:57.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb39514640 1 --2- 172.21.15.2:0/1427210879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb34105390 0x7ffb34197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:57.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb38d13640 1 -- 172.21.15.2:0/1427210879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb34105390 msgr2=0x7ffb34197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:57.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb38d13640 1 --2- 172.21.15.2:0/1427210879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb34105390 0x7ffb34197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb38d13640 1 -- 172.21.15.2:0/1427210879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb34101000 msgr2=0x7ffb3419dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:57.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb38d13640 1 --2- 172.21.15.2:0/1427210879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb34101000 0x7ffb3419dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb38d13640 1 -- 172.21.15.2:0/1427210879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb28009560 con 0x7ffb34100640 2024-03-15T13:08:57.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb39514640 1 --2- 172.21.15.2:0/1427210879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb34105390 0x7ffb34197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:08:57.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb38d13640 1 --2- 172.21.15.2:0/1427210879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb34100640 0x7ffb3419d7d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400e9f0 tx=0x7ffb2400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:57.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb31ffb640 1 -- 172.21.15.2:0/1427210879 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24009720 con 0x7ffb34100640 2024-03-15T13:08:57.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb31ffb640 1 -- 172.21.15.2:0/1427210879 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ffb2400cd80 con 0x7ffb34100640 2024-03-15T13:08:57.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb31ffb640 1 -- 172.21.15.2:0/1427210879 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24010690 con 0x7ffb34100640 2024-03-15T13:08:57.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.164+0000 7ffb3af77640 1 -- 172.21.15.2:0/1427210879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb341985e0 con 0x7ffb34100640 2024-03-15T13:08:57.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.165+0000 7ffb3af77640 1 -- 172.21.15.2:0/1427210879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb341a4870 con 0x7ffb34100640 2024-03-15T13:08:57.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.166+0000 7ffb3af77640 1 -- 172.21.15.2:0/1427210879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3404f250 con 0x7ffb34100640 2024-03-15T13:08:57.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.167+0000 7ffb31ffb640 1 -- 172.21.15.2:0/1427210879 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7ffb24004e90 con 0x7ffb34100640 2024-03-15T13:08:57.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.167+0000 7ffb31ffb640 1 --2- 172.21.15.2:0/1427210879 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ffb1c0a1cc0 0x7ffb1c0a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:57.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.167+0000 7ffb31ffb640 1 -- 172.21.15.2:0/1427210879 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7ffb24014070 con 0x7ffb34100640 2024-03-15T13:08:57.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.167+0000 7ffb33fff640 1 --2- 172.21.15.2:0/1427210879 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ffb1c0a1cc0 0x7ffb1c0a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:57.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.168+0000 7ffb33fff640 1 --2- 172.21.15.2:0/1427210879 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ffb1c0a1cc0 0x7ffb1c0a4180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffb28005ab0 tx=0x7ffb28005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:57.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.172+0000 7ffb31ffb640 1 -- 172.21.15.2:0/1427210879 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ffb24096a70 con 0x7ffb34100640 2024-03-15T13:08:57.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.377+0000 7ffb3af77640 1 -- 172.21.15.2:0/1427210879 --> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb34110600 con 0x7ffb1c0a1cc0 2024-03-15T13:08:57.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.384+0000 7ffb31ffb640 1 -- 172.21.15.2:0/1427210879 <== mgr.15741 v2:172.21.15.2:6800/1550147071 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7ffb34110600 con 0x7ffb1c0a1cc0 2024-03-15T13:08:57.368 INFO:teuthology.orchestra.run.smithi002.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-15T00:00:00", "created": "2024-03-15T12:58:54", "first": "2024-03-15T12:59:00", "last": "2024-03-15T13:08:00", "last_pruned": "2024-03-15T13:08:00", "created_count": 10, "pruned_count": 3, "active": true} 2024-03-15T13:08:57.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.386+0000 7ffb3af77640 1 -- 172.21.15.2:0/1427210879 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ffb1c0a1cc0 msgr2=0x7ffb1c0a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:57.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.386+0000 7ffb3af77640 1 --2- 172.21.15.2:0/1427210879 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ffb1c0a1cc0 0x7ffb1c0a4180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffb28005ab0 tx=0x7ffb28005fd0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.386+0000 7ffb3af77640 1 -- 172.21.15.2:0/1427210879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb34100640 msgr2=0x7ffb3419d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:57.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.386+0000 7ffb3af77640 1 --2- 172.21.15.2:0/1427210879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb34100640 0x7ffb3419d7d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400e9f0 tx=0x7ffb2400edf0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.386+0000 7ffb3af77640 1 -- 172.21.15.2:0/1427210879 shutdown_connections 2024-03-15T13:08:57.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.386+0000 7ffb3af77640 1 --2- 172.21.15.2:0/1427210879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb34105390 0x7ffb34197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.386+0000 7ffb3af77640 1 --2- 172.21.15.2:0/1427210879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb34101000 0x7ffb3419dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.386+0000 7ffb3af77640 1 --2- 172.21.15.2:0/1427210879 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ffb1c0a1cc0 0x7ffb1c0a4180 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.386+0000 7ffb3af77640 1 --2- 172.21.15.2:0/1427210879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb34100640 0x7ffb3419d7d0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.386+0000 7ffb3af77640 1 -- 172.21.15.2:0/1427210879 >> 172.21.15.2:0/1427210879 conn(0x7ffb34077240 msgr2=0x7ffb34104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:57.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.386+0000 7ffb3af77640 1 -- 172.21.15.2:0/1427210879 shutdown_connections 2024-03-15T13:08:57.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.386+0000 7ffb3af77640 1 -- 172.21.15.2:0/1427210879 wait complete. 2024-03-15T13:08:57.381 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-15T13:08:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.502+0000 7f6733f55640 1 -- 172.21.15.2:0/1141011187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f672c105700 msgr2=0x7f672c10d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.502+0000 7f6733f55640 1 --2- 172.21.15.2:0/1141011187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f672c105700 0x7f672c10d2d0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f671c009cd0 tx=0x7f671c02c0f0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.502+0000 7f6733f55640 1 -- 172.21.15.2:0/1141011187 shutdown_connections 2024-03-15T13:08:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.502+0000 7f6733f55640 1 --2- 172.21.15.2:0/1141011187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f672c105700 0x7f672c10d2d0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.502+0000 7f6733f55640 1 --2- 172.21.15.2:0/1141011187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f672c100fe0 0x7f672c105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.502+0000 7f6733f55640 1 --2- 172.21.15.2:0/1141011187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f672c100630 0x7f672c100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.502+0000 7f6733f55640 1 -- 172.21.15.2:0/1141011187 >> 172.21.15.2:0/1141011187 conn(0x7f672c077520 msgr2=0x7f672c077930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.502+0000 7f6733f55640 1 -- 172.21.15.2:0/1141011187 shutdown_connections 2024-03-15T13:08:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.502+0000 7f6733f55640 1 -- 172.21.15.2:0/1141011187 wait complete. 2024-03-15T13:08:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.503+0000 7f6733f55640 1 Processor -- start 2024-03-15T13:08:57.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.503+0000 7f6733f55640 1 -- start start 2024-03-15T13:08:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.503+0000 7f6733f55640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f672c100630 0x7f672c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.503+0000 7f6733f55640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f672c105700 0x7f672c19dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.503+0000 7f6733f55640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f672c197850 0x7f672c197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.503+0000 7f6733f55640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c198300 con 0x7f672c105700 2024-03-15T13:08:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.503+0000 7f6733f55640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c198470 con 0x7f672c100630 2024-03-15T13:08:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.503+0000 7f6733f55640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c1985e0 con 0x7f672c197850 2024-03-15T13:08:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.503+0000 7f67314f0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f672c105700 0x7f672c19dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.503+0000 7f67314f0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f672c105700 0x7f672c19dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46036/0 (socket says 172.21.15.2:46036) 2024-03-15T13:08:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.503+0000 7f67314f0640 1 -- 172.21.15.2:0/3824291306 learned_addr learned my addr 172.21.15.2:0/3824291306 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:08:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.503+0000 7f6731cf1640 1 --2- 172.21.15.2:0/3824291306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f672c100630 0x7f672c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.503+0000 7f67324f2640 1 --2- 172.21.15.2:0/3824291306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f672c197850 0x7f672c197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.504+0000 7f67314f0640 1 -- 172.21.15.2:0/3824291306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f672c197850 msgr2=0x7f672c197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.504+0000 7f67314f0640 1 --2- 172.21.15.2:0/3824291306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f672c197850 0x7f672c197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.504+0000 7f67314f0640 1 -- 172.21.15.2:0/3824291306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f672c100630 msgr2=0x7f672c19d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:57.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.504+0000 7f67314f0640 1 --2- 172.21.15.2:0/3824291306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f672c100630 0x7f672c19d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.504+0000 7f67314f0640 1 -- 172.21.15.2:0/3824291306 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c009900 con 0x7f672c105700 2024-03-15T13:08:57.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.504+0000 7f67314f0640 1 --2- 172.21.15.2:0/3824291306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f672c105700 0x7f672c19dc20 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f6720005fd0 tx=0x7f672000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:57.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.504+0000 7f672affd640 1 -- 172.21.15.2:0/3824291306 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720009c30 con 0x7f672c105700 2024-03-15T13:08:57.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.504+0000 7f672affd640 1 -- 172.21.15.2:0/3824291306 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f6720009d90 con 0x7f672c105700 2024-03-15T13:08:57.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.504+0000 7f6733f55640 1 -- 172.21.15.2:0/3824291306 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672c1a4590 con 0x7f672c105700 2024-03-15T13:08:57.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.504+0000 7f672affd640 1 -- 172.21.15.2:0/3824291306 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720010460 con 0x7f672c105700 2024-03-15T13:08:57.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.504+0000 7f6733f55640 1 -- 172.21.15.2:0/3824291306 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672c1a4aa0 con 0x7f672c105700 2024-03-15T13:08:57.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.505+0000 7f6733f55640 1 -- 172.21.15.2:0/3824291306 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672c101b00 con 0x7f672c105700 2024-03-15T13:08:57.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.506+0000 7f672affd640 1 -- 172.21.15.2:0/3824291306 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f6720003ef0 con 0x7f672c105700 2024-03-15T13:08:57.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.507+0000 7f672affd640 1 --2- 172.21.15.2:0/3824291306 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f67140a1cc0 0x7f67140a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:08:57.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.507+0000 7f6731cf1640 1 --2- 172.21.15.2:0/3824291306 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f67140a1cc0 0x7f67140a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:08:57.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.507+0000 7f672affd640 1 -- 172.21.15.2:0/3824291306 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f67200cd670 con 0x7f672c105700 2024-03-15T13:08:57.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.507+0000 7f6731cf1640 1 --2- 172.21.15.2:0/3824291306 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f67140a1cc0 0x7f67140a4180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6724009bd0 tx=0x7f6724005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:08:57.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.511+0000 7f672affd640 1 -- 172.21.15.2:0/3824291306 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f67200961a0 con 0x7f672c105700 2024-03-15T13:08:57.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.717+0000 7f6733f55640 1 -- 172.21.15.2:0/3824291306 --> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f672c1010f0 con 0x7f67140a1cc0 2024-03-15T13:08:57.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.723+0000 7f672affd640 1 -- 172.21.15.2:0/3824291306 <== mgr.15741 v2:172.21.15.2:6800/1550147071 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f672c1010f0 con 0x7f67140a1cc0 2024-03-15T13:08:57.708 INFO:teuthology.orchestra.run.smithi002.stdout:/ 1m 6m3h 2024-03-15T13:08:57.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.726+0000 7f6733f55640 1 -- 172.21.15.2:0/3824291306 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f67140a1cc0 msgr2=0x7f67140a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:57.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.726+0000 7f6733f55640 1 --2- 172.21.15.2:0/3824291306 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f67140a1cc0 0x7f67140a4180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6724009bd0 tx=0x7f6724005f90 comp rx=0 tx=0).stop 2024-03-15T13:08:57.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.726+0000 7f6733f55640 1 -- 172.21.15.2:0/3824291306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f672c105700 msgr2=0x7f672c19dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:08:57.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.726+0000 7f6733f55640 1 --2- 172.21.15.2:0/3824291306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f672c105700 0x7f672c19dc20 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f6720005fd0 tx=0x7f672000dde0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.726+0000 7f6733f55640 1 -- 172.21.15.2:0/3824291306 shutdown_connections 2024-03-15T13:08:57.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.726+0000 7f6733f55640 1 --2- 172.21.15.2:0/3824291306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f672c197850 0x7f672c197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.726+0000 7f6733f55640 1 --2- 172.21.15.2:0/3824291306 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f67140a1cc0 0x7f67140a4180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.726+0000 7f6733f55640 1 --2- 172.21.15.2:0/3824291306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f672c105700 0x7f672c19dc20 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.726+0000 7f6733f55640 1 --2- 172.21.15.2:0/3824291306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f672c100630 0x7f672c19d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:08:57.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.726+0000 7f6733f55640 1 -- 172.21.15.2:0/3824291306 >> 172.21.15.2:0/3824291306 conn(0x7f672c077520 msgr2=0x7f672c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:08:57.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.726+0000 7f6733f55640 1 -- 172.21.15.2:0/3824291306 shutdown_connections 2024-03-15T13:08:57.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:08:57.726+0000 7f6733f55640 1 -- 172.21.15.2:0/3824291306 wait complete. 2024-03-15T13:08:57.719 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:57 smithi002 ceph-mon[32060]: pgmap v310: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:08:57.721 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-15T13:08:57.732 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-15T13:08:57.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:57 smithi149 ceph-mon[35927]: pgmap v310: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:08:58.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:57 smithi184 ceph-mon[35993]: pgmap v310: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:08:58.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:58 smithi149 ceph-mon[35927]: from='client.18531 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T13:08:58.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:58 smithi149 ceph-mon[35927]: from='client.18537 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T13:08:59.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:58 smithi002 ceph-mon[32060]: from='client.18531 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T13:08:59.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:58 smithi002 ceph-mon[32060]: from='client.18537 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T13:08:59.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:58 smithi184 ceph-mon[35993]: from='client.18531 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T13:08:59.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:58 smithi184 ceph-mon[35993]: from='client.18537 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-15T13:08:59.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:08:59 smithi149 ceph-mon[35927]: pgmap v311: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:00.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:08:59 smithi002 ceph-mon[32060]: pgmap v311: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:00.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:08:59 smithi184 ceph-mon[35993]: pgmap v311: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:01.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:01 smithi149 ceph-mon[35927]: pgmap v312: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-15T13:09:02.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:01 smithi002 ceph-mon[32060]: pgmap v312: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-15T13:09:02.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:01 smithi184 ceph-mon[35993]: pgmap v312: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-15T13:09:03.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-15T13:09:03.229 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-15T13:09:03.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.362+0000 7f5ec6b25640 1 -- 172.21.15.2:0/4026003116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec0069b60 msgr2=0x7f5ec0069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.362+0000 7f5ec6b25640 1 --2- 172.21.15.2:0/4026003116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec0069b60 0x7f5ec0069fc0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009900 tx=0x7f5eb402bc30 comp rx=0 tx=0).stop 2024-03-15T13:09:03.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.363+0000 7f5ec6b25640 1 -- 172.21.15.2:0/4026003116 shutdown_connections 2024-03-15T13:09:03.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.363+0000 7f5ec6b25640 1 --2- 172.21.15.2:0/4026003116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec006a500 0x7f5ec010e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.363+0000 7f5ec6b25640 1 --2- 172.21.15.2:0/4026003116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec0069b60 0x7f5ec0069fc0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.363+0000 7f5ec6b25640 1 --2- 172.21.15.2:0/4026003116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec0107510 0x7f5ec01078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.363+0000 7f5ec6b25640 1 -- 172.21.15.2:0/4026003116 >> 172.21.15.2:0/4026003116 conn(0x7f5ec0077190 msgr2=0x7f5ec00775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.363+0000 7f5ec6b25640 1 -- 172.21.15.2:0/4026003116 shutdown_connections 2024-03-15T13:09:03.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.363+0000 7f5ec6b25640 1 -- 172.21.15.2:0/4026003116 wait complete. 2024-03-15T13:09:03.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.363+0000 7f5ec6b25640 1 Processor -- start 2024-03-15T13:09:03.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.363+0000 7f5ec6b25640 1 -- start start 2024-03-15T13:09:03.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.363+0000 7f5ec6b25640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec0069b60 0x7f5ec019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.363+0000 7f5ec6b25640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec006a500 0x7f5ec019dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.363+0000 7f5ec6b25640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec0107510 0x7f5ec0197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.363+0000 7f5ec6b25640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec0197f50 con 0x7f5ec006a500 2024-03-15T13:09:03.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.363+0000 7f5ec6b25640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec01980c0 con 0x7f5ec0107510 2024-03-15T13:09:03.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.363+0000 7f5ec6b25640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec0198280 con 0x7f5ec0069b60 2024-03-15T13:09:03.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.364+0000 7f5ebffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec006a500 0x7f5ec019dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.364+0000 7f5ebffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec006a500 0x7f5ec019dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57828/0 (socket says 172.21.15.2:57828) 2024-03-15T13:09:03.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.364+0000 7f5ebffff640 1 -- 172.21.15.2:0/1843766295 learned_addr learned my addr 172.21.15.2:0/1843766295 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:03.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.364+0000 7f5ec48c1640 1 --2- 172.21.15.2:0/1843766295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec0069b60 0x7f5ec019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.364+0000 7f5ec50c2640 1 --2- 172.21.15.2:0/1843766295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec0107510 0x7f5ec0197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.364+0000 7f5ebffff640 1 -- 172.21.15.2:0/1843766295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec0069b60 msgr2=0x7f5ec019d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.364+0000 7f5ebffff640 1 --2- 172.21.15.2:0/1843766295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec0069b60 0x7f5ec019d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.364+0000 7f5ebffff640 1 -- 172.21.15.2:0/1843766295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec0107510 msgr2=0x7f5ec0197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.364+0000 7f5ebffff640 1 --2- 172.21.15.2:0/1843766295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec0107510 0x7f5ec0197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.364+0000 7f5ebffff640 1 -- 172.21.15.2:0/1843766295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb4009560 con 0x7f5ec006a500 2024-03-15T13:09:03.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.364+0000 7f5ebffff640 1 --2- 172.21.15.2:0/1843766295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec006a500 0x7f5ec019dc90 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009230 tx=0x7f5eb402cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.364+0000 7f5ebdffb640 1 -- 172.21.15.2:0/1843766295 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4004250 con 0x7f5ec006a500 2024-03-15T13:09:03.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.364+0000 7f5ebdffb640 1 -- 172.21.15.2:0/1843766295 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5eb40043b0 con 0x7f5ec006a500 2024-03-15T13:09:03.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.364+0000 7f5ec6b25640 1 -- 172.21.15.2:0/1843766295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec0198500 con 0x7f5ec006a500 2024-03-15T13:09:03.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.364+0000 7f5ebdffb640 1 -- 172.21.15.2:0/1843766295 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4003bf0 con 0x7f5ec006a500 2024-03-15T13:09:03.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.365+0000 7f5ec6b25640 1 -- 172.21.15.2:0/1843766295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec01a47f0 con 0x7f5ec006a500 2024-03-15T13:09:03.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.367+0000 7f5ebdffb640 1 -- 172.21.15.2:0/1843766295 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f5eb4003d50 con 0x7f5ec006a500 2024-03-15T13:09:03.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.367+0000 7f5ec97dd740 1 Processor -- start 2024-03-15T13:09:03.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.367+0000 7f5ec97dd740 1 -- start start 2024-03-15T13:09:03.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ebdffb640 1 --2- 172.21.15.2:0/1843766295 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ea80a1c20 0x7f5ea80a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ec97dd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7ffffb2b0 0x55a7ffffb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ec97dd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a800004270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ec97dd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8000047b0 0x55a800006ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ec97dd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7fff6a280 con 0x55a7ffffbbd0 2024-03-15T13:09:03.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ec97dd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7fff677b0 con 0x55a7ffffb2b0 2024-03-15T13:09:03.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ec97dd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7fff62f90 con 0x55a8000047b0 2024-03-15T13:09:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ec50c2640 1 --2- 172.21.15.2:0/1843766295 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ea80a1c20 0x7f5ea80a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ebffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a800004270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ebdffb640 1 -- 172.21.15.2:0/1843766295 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f5eb40efac0 con 0x7f5ec006a500 2024-03-15T13:09:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ec48c1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7ffffb2b0 0x55a7ffffb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ebffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a800004270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57836/0 (socket says 172.21.15.2:57836) 2024-03-15T13:09:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ebffff640 1 -- 172.21.15.2:0/2038731620 learned_addr learned my addr 172.21.15.2:0/2038731620 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ec48c1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7ffffb2b0 0x55a7ffffb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:49330/0 (socket says 172.21.15.2:49330) 2024-03-15T13:09:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ec50c2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8000047b0 0x55a800006ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ebffff640 1 -- 172.21.15.2:0/2038731620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8000047b0 msgr2=0x55a800006ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ebffff640 1 --2- 172.21.15.2:0/2038731620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8000047b0 0x55a800006ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ebffff640 1 -- 172.21.15.2:0/2038731620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7ffffb2b0 msgr2=0x55a7ffffb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ebffff640 1 --2- 172.21.15.2:0/2038731620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7ffffb2b0 0x55a7ffffb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ebffff640 1 -- 172.21.15.2:0/2038731620 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7fff21570 con 0x55a7ffffbbd0 2024-03-15T13:09:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ec48c1640 1 --2- 172.21.15.2:0/2038731620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7ffffb2b0 0x55a7ffffb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:03.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ec50c2640 1 --2- 172.21.15.2:0/1843766295 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ea80a1c20 0x7f5ea80a40e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8005fd0 tx=0x7f5eb8009e00 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.368+0000 7f5ebffff640 1 --2- 172.21.15.2:0/2038731620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a800004270 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40b8dd0 tx=0x7f5eb402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0056e41892dba3 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.369+0000 7f5ea6ffd640 1 -- 172.21.15.2:0/2038731620 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40bbd60 con 0x55a7ffffbbd0 2024-03-15T13:09:03.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.369+0000 7f5ea6ffd640 1 -- 172.21.15.2:0/2038731620 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5eb40ac5b0 con 0x55a7ffffbbd0 2024-03-15T13:09:03.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.369+0000 7f5ea6ffd640 1 -- 172.21.15.2:0/2038731620 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4005610 con 0x55a7ffffbbd0 2024-03-15T13:09:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.369+0000 7f5ec97dd740 1 -- 172.21.15.2:0/2038731620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 msgr2=0x55a800004270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.369+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/2038731620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a800004270 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40b8dd0 tx=0x7f5eb402bc30 comp rx=0 tx=0).stop 2024-03-15T13:09:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.369+0000 7f5ec97dd740 1 -- 172.21.15.2:0/2038731620 shutdown_connections 2024-03-15T13:09:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.369+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/2038731620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8000047b0 0x55a800006ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.369+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/2038731620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a800004270 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.369+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/2038731620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7ffffb2b0 0x55a7ffffb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.369+0000 7f5ec97dd740 1 -- 172.21.15.2:0/2038731620 >> 172.21.15.2:0/2038731620 conn(0x55a7ffff5990 msgr2=0x55a7ffff5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.369+0000 7f5ec97dd740 1 -- 172.21.15.2:0/2038731620 shutdown_connections 2024-03-15T13:09:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.369+0000 7f5ec97dd740 1 -- 172.21.15.2:0/2038731620 wait complete. 2024-03-15T13:09:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.370+0000 7f5ec97dd740 1 -- start start 2024-03-15T13:09:03.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.370+0000 7f5ec97dd740 1 Processor -- start 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.370+0000 7f5ec97dd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7ffffb2b0 0x55a800004250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.370+0000 7f5ec97dd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a8000015e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.370+0000 7f5ec97dd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8000047b0 0x55a800001cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.370+0000 7f5ec97dd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ffc36430 con 0x55a7ffffbbd0 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.370+0000 7f5ec97dd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ffbf2e00 con 0x55a8000047b0 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.370+0000 7f5ec97dd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ffee7fd0 con 0x55a7ffffb2b0 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.370+0000 7f5ec97dd740 10 client.0 _ll_register_callbacks cb 0x55a7fff3dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.370+0000 7f5ebffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a8000015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.370+0000 7f5ebffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a8000015e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57844/0 (socket says 172.21.15.2:57844) 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.370+0000 7f5ebffff640 1 -- 172.21.15.2:0/2240359803 learned_addr learned my addr 172.21.15.2:0/2240359803 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.370+0000 7f5ec48c1640 1 --2- 172.21.15.2:0/2240359803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7ffffb2b0 0x55a800004250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.371+0000 7f5ec50c2640 1 --2- 172.21.15.2:0/2240359803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8000047b0 0x55a800001cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.371+0000 7f5ebffff640 1 -- 172.21.15.2:0/2240359803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7ffffb2b0 msgr2=0x55a800004250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.371+0000 7f5ebffff640 1 --2- 172.21.15.2:0/2240359803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7ffffb2b0 0x55a800004250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.371+0000 7f5ebffff640 1 -- 172.21.15.2:0/2240359803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8000047b0 msgr2=0x55a800001cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.371+0000 7f5ebffff640 1 --2- 172.21.15.2:0/2240359803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8000047b0 0x55a800001cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.371+0000 7f5ebffff640 1 -- 172.21.15.2:0/2240359803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec01a47f0 con 0x55a7ffffbbd0 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.371+0000 7f5ebffff640 1 --2- 172.21.15.2:0/2240359803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a8000015e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40bc390 tx=0x7f5eb40bc000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15b1cfdc42fd312 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.371+0000 7f5e9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.371+0000 7f5e9ffff640 1 -- 172.21.15.2:0/2240359803 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40c0370 con 0x55a7ffffbbd0 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.371+0000 7f5e9ffff640 1 -- 172.21.15.2:0/2240359803 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f5eb40bc400 con 0x55a7ffffbbd0 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.371+0000 7f5ec97dd740 1 -- 172.21.15.2:0/2240359803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7ffc627e0 con 0x55a7ffffbbd0 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.371+0000 7f5e9ffff640 1 -- 172.21.15.2:0/2240359803 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5eb40bbd60 con 0x55a7ffffbbd0 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.371+0000 7f5ec97dd740 10 client.18549 fetch_fsmap learned FSMap version 50 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.371+0000 7f5ec97dd740 1 -- 172.21.15.2:0/2240359803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55a7fff61610 con 0x55a7ffffbbd0 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.371+0000 7f5e9ffff640 1 -- 172.21.15.2:0/2240359803 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55a7ffc627e0 con 0x55a7ffffbbd0 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.372+0000 7f5e9ffff640 1 -- 172.21.15.2:0/2240359803 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5eb403c070 con 0x55a7ffffbbd0 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.372+0000 7f5ec97dd740 10 client.18549 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.372+0000 7f5ec97dd740 10 client.18549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.372+0000 7f5ec97dd740 20 client.18549 populate_metadata read hostname 'smithi002' 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.372+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/2240359803 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a800014360 0x55a800016750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.372+0000 7f5ec97dd740 4 client.18549 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.372+0000 7f5ec97dd740 1 -- 172.21.15.2:0/2240359803 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7ffc9f720 con 0x55a800014360 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.372+0000 7f5ec48c1640 1 --2- 172.21.15.2:0/2240359803 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a800014360 0x55a800016750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.372+0000 7f5ec48c1640 1 --2- 172.21.15.2:0/2240359803 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a800014360 0x55a800016750 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7fcf43ba01d1ddb server_cookie=a355b1499080907a in_seq=0 out_seq=0 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.372+0000 7f5e9ffff640 10 client.18549 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.373+0000 7f5e9ffff640 1 -- 172.21.15.2:0/2240359803 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a7ffc9f720 con 0x55a800014360 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.373+0000 7f5e9ffff640 10 client.18549 handle_command_reply: tid=0 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.373+0000 7f5ec97dd740 1 client.18549 shutdown 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.373+0000 7f5ec97dd740 1 -- 172.21.15.2:0/2240359803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 msgr2=0x55a8000015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.373+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/2240359803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a8000015e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40bc390 tx=0x7f5eb40bc000 comp rx=0 tx=0).stop 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.373+0000 7f5ec97dd740 1 -- 172.21.15.2:0/2240359803 shutdown_connections 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.373+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/2240359803 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a800014360 0x55a800016750 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.373+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/2240359803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8000047b0 0x55a800001cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.373+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/2240359803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a8000015e0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.373+0000 7f5e9ffff640 0 client.18549 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.373+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/2240359803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7ffffb2b0 0x55a800004250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.373+0000 7f5ec97dd740 1 -- 172.21.15.2:0/2240359803 >> 172.21.15.2:0/2240359803 conn(0x55a7ffff5990 msgr2=0x55a7ffff7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.373+0000 7f5ec97dd740 1 -- 172.21.15.2:0/2240359803 shutdown_connections 2024-03-15T13:09:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.373+0000 7f5ec97dd740 1 -- 172.21.15.2:0/2240359803 wait complete. 2024-03-15T13:09:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.373+0000 7f5ec97dd740 20 client.18549 trim_cache size 0 max 16384 2024-03-15T13:09:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.389+0000 7f5ec97dd740 1 Processor -- start 2024-03-15T13:09:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.389+0000 7f5ec97dd740 1 -- start start 2024-03-15T13:09:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.389+0000 7f5ec97dd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7ffffb2b0 0x55a800003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.389+0000 7f5ec97dd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a800004250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ec97dd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8000047b0 0x55a80000f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ec97dd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ffff69e0 con 0x55a7ffffbbd0 2024-03-15T13:09:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ec97dd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7fffefe20 con 0x55a7ffffb2b0 2024-03-15T13:09:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ec97dd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ffffaf10 con 0x55a8000047b0 2024-03-15T13:09:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ebffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a800004250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ec48c1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7ffffb2b0 0x55a800003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ebffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a800004250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57852/0 (socket says 172.21.15.2:57852) 2024-03-15T13:09:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ebffff640 1 -- 172.21.15.2:0/1002783875 learned_addr learned my addr 172.21.15.2:0/1002783875 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ec50c2640 1 --2- 172.21.15.2:0/1002783875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8000047b0 0x55a80000f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ebffff640 1 -- 172.21.15.2:0/1002783875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8000047b0 msgr2=0x55a80000f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ebffff640 1 --2- 172.21.15.2:0/1002783875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8000047b0 0x55a80000f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ebffff640 1 -- 172.21.15.2:0/1002783875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7ffffb2b0 msgr2=0x55a800003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ebffff640 1 --2- 172.21.15.2:0/1002783875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7ffffb2b0 0x55a800003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ebffff640 1 -- 172.21.15.2:0/1002783875 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7ffefdac0 con 0x55a7ffffbbd0 2024-03-15T13:09:03.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ebffff640 1 --2- 172.21.15.2:0/1002783875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a800004250 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40b8dd0 tx=0x7f5eb4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994d3820d40d8555 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ea67fc640 1 -- 172.21.15.2:0/1002783875 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40bbd60 con 0x55a7ffffbbd0 2024-03-15T13:09:03.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ea67fc640 1 -- 172.21.15.2:0/1002783875 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5eb40c0370 con 0x55a7ffffbbd0 2024-03-15T13:09:03.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.390+0000 7f5ea67fc640 1 -- 172.21.15.2:0/1002783875 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40ed090 con 0x55a7ffffbbd0 2024-03-15T13:09:03.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.391+0000 7f5ec97dd740 1 -- 172.21.15.2:0/1002783875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 msgr2=0x55a800004250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.391+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/1002783875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a800004250 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40b8dd0 tx=0x7f5eb4005fd0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.391+0000 7f5ec97dd740 1 -- 172.21.15.2:0/1002783875 shutdown_connections 2024-03-15T13:09:03.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.391+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/1002783875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8000047b0 0x55a80000f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.391+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/1002783875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a800004250 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.391+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/1002783875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a7ffffb2b0 0x55a800003d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.391+0000 7f5ec97dd740 1 -- 172.21.15.2:0/1002783875 >> 172.21.15.2:0/1002783875 conn(0x55a7ffff5990 msgr2=0x55a800000610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.391+0000 7f5ec97dd740 1 -- 172.21.15.2:0/1002783875 shutdown_connections 2024-03-15T13:09:03.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.391+0000 7f5ec97dd740 1 -- 172.21.15.2:0/1002783875 wait complete. 2024-03-15T13:09:03.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.391+0000 7f5ec97dd740 1 -- start start 2024-03-15T13:09:03.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.391+0000 7f5ec97dd740 1 Processor -- start 2024-03-15T13:09:03.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ec97dd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7ffffb2b0 0x55a80000bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ec97dd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a8000059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ec97dd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8000047b0 0x55a800005ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ec97dd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a800006420 con 0x55a7ffffbbd0 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ec97dd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a800006590 con 0x55a8000047b0 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ec97dd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a800006750 con 0x55a7ffffb2b0 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ec97dd740 10 client.0 _ll_register_callbacks cb 0x7f5eb0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ebffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a8000059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ebffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a8000059a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57864/0 (socket says 172.21.15.2:57864) 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ebffff640 1 -- 172.21.15.2:0/297404459 learned_addr learned my addr 172.21.15.2:0/297404459 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ec48c1640 1 --2- 172.21.15.2:0/297404459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7ffffb2b0 0x55a80000bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ec50c2640 1 --2- 172.21.15.2:0/297404459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8000047b0 0x55a800005ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ebffff640 1 -- 172.21.15.2:0/297404459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7ffffb2b0 msgr2=0x55a80000bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ebffff640 1 --2- 172.21.15.2:0/297404459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7ffffb2b0 0x55a80000bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ebffff640 1 -- 172.21.15.2:0/297404459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8000047b0 msgr2=0x55a800005ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ebffff640 1 --2- 172.21.15.2:0/297404459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8000047b0 0x55a800005ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ebffff640 1 -- 172.21.15.2:0/297404459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7fff61610 con 0x55a7ffffbbd0 2024-03-15T13:09:03.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ebffff640 1 --2- 172.21.15.2:0/297404459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a8000059a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40bc390 tx=0x7f5eb40bc070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768f0b6e78349b4c server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5e9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:09:03.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5e9ffff640 1 -- 172.21.15.2:0/297404459 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40bbd60 con 0x55a7ffffbbd0 2024-03-15T13:09:03.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5e9ffff640 1 -- 172.21.15.2:0/297404459 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f5eb40c0550 con 0x55a7ffffbbd0 2024-03-15T13:09:03.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ec97dd740 1 -- 172.21.15.2:0/297404459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7ffcee890 con 0x55a7ffffbbd0 2024-03-15T13:09:03.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5e9ffff640 1 -- 172.21.15.2:0/297404459 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5eb40c0370 con 0x55a7ffffbbd0 2024-03-15T13:09:03.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ec97dd740 10 client.18555 fetch_fsmap learned FSMap version 50 2024-03-15T13:09:03.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.392+0000 7f5ec97dd740 1 -- 172.21.15.2:0/297404459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55a7fff236a0 con 0x55a7ffffbbd0 2024-03-15T13:09:03.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.393+0000 7f5e9ffff640 1 -- 172.21.15.2:0/297404459 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55a7ffcee890 con 0x55a7ffffbbd0 2024-03-15T13:09:03.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.393+0000 7f5e9ffff640 1 -- 172.21.15.2:0/297404459 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5eb403c070 con 0x55a7ffffbbd0 2024-03-15T13:09:03.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.393+0000 7f5ec97dd740 10 client.18555 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:09:03.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.393+0000 7f5ec97dd740 10 client.18555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:09:03.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.393+0000 7f5ec97dd740 20 client.18555 populate_metadata read hostname 'smithi002' 2024-03-15T13:09:03.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.393+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/297404459 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a800070b20 0x55a800072f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.393+0000 7f5ec97dd740 4 client.18555 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.393+0000 7f5ec97dd740 1 -- 172.21.15.2:0/297404459 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a7ffc514e0 con 0x55a800070b20 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.393+0000 7f5ec48c1640 1 --2- 172.21.15.2:0/297404459 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a800070b20 0x55a800072f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.393+0000 7f5ec48c1640 1 --2- 172.21.15.2:0/297404459 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a800070b20 0x55a800072f10 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bad8d874f98a44e server_cookie=509482bd49d61c88 in_seq=0 out_seq=0 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.393+0000 7f5e9ffff640 10 client.18555 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5e9ffff640 1 -- 172.21.15.2:0/297404459 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a7ffc514e0 con 0x55a800070b20 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5e9ffff640 10 client.18555 handle_command_reply: tid=0 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec97dd740 1 client.18555 shutdown 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec97dd740 1 -- 172.21.15.2:0/297404459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 msgr2=0x55a8000059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/297404459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a8000059a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40bc390 tx=0x7f5eb40bc070 comp rx=0 tx=0).stop 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec97dd740 1 -- 172.21.15.2:0/297404459 shutdown_connections 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/297404459 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x55a800070b20 0x55a800072f10 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/297404459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8000047b0 0x55a800005ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/297404459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7ffffbbd0 0x55a8000059a0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5e9ffff640 0 client.18555 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec97dd740 1 --2- 172.21.15.2:0/297404459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7ffffb2b0 0x55a80000bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec97dd740 1 -- 172.21.15.2:0/297404459 >> 172.21.15.2:0/297404459 conn(0x55a7ffff5990 msgr2=0x55a7ffff41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec97dd740 1 -- 172.21.15.2:0/297404459 shutdown_connections 2024-03-15T13:09:03.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec97dd740 1 -- 172.21.15.2:0/297404459 wait complete. 2024-03-15T13:09:03.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec97dd740 20 client.18555 trim_cache size 0 max 16384 2024-03-15T13:09:03.384 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:09:03.384 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2024-03-15T13:09:03.384 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2024-03-15T13:09:03.384 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:09:03.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec6b25640 1 -- 172.21.15.2:0/1843766295 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ea80a1c20 msgr2=0x7f5ea80a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec6b25640 1 --2- 172.21.15.2:0/1843766295 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ea80a1c20 0x7f5ea80a40e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8005fd0 tx=0x7f5eb8009e00 comp rx=0 tx=0).stop 2024-03-15T13:09:03.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec6b25640 1 -- 172.21.15.2:0/1843766295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec006a500 msgr2=0x7f5ec019dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.394+0000 7f5ec6b25640 1 --2- 172.21.15.2:0/1843766295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec006a500 0x7f5ec019dc90 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009230 tx=0x7f5eb402cac0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.395+0000 7f5ec6b25640 1 -- 172.21.15.2:0/1843766295 shutdown_connections 2024-03-15T13:09:03.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.395+0000 7f5ec6b25640 1 --2- 172.21.15.2:0/1843766295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ec0107510 0x7f5ec0197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.395+0000 7f5ec6b25640 1 --2- 172.21.15.2:0/1843766295 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5ea80a1c20 0x7f5ea80a40e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.395+0000 7f5ec6b25640 1 --2- 172.21.15.2:0/1843766295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ec006a500 0x7f5ec019dc90 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.395+0000 7f5ec6b25640 1 --2- 172.21.15.2:0/1843766295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ec0069b60 0x7f5ec019d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.395+0000 7f5ec6b25640 1 -- 172.21.15.2:0/1843766295 >> 172.21.15.2:0/1843766295 conn(0x7f5ec0077190 msgr2=0x7f5ec00ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.395+0000 7f5ec6b25640 1 -- 172.21.15.2:0/1843766295 shutdown_connections 2024-03-15T13:09:03.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.395+0000 7f5ec6b25640 1 -- 172.21.15.2:0/1843766295 wait complete. 2024-03-15T13:09:03.389 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-15T13:09:03.390 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:09:03.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.508+0000 7f859d32c640 1 -- 172.21.15.2:0/3395009194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f859810bb00 msgr2=0x7f859810df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.508+0000 7f859d32c640 1 --2- 172.21.15.2:0/3395009194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f859810bb00 0x7f859810df30 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f8590009ca0 tx=0x7f859002c180 comp rx=0 tx=0).stop 2024-03-15T13:09:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.508+0000 7f859d32c640 1 -- 172.21.15.2:0/3395009194 shutdown_connections 2024-03-15T13:09:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.508+0000 7f859d32c640 1 --2- 172.21.15.2:0/3395009194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f859810bb00 0x7f859810df30 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.508+0000 7f859d32c640 1 --2- 172.21.15.2:0/3395009194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8598103050 0x7f859810b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.508+0000 7f859d32c640 1 --2- 172.21.15.2:0/3395009194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8598102730 0x7f8598102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.508+0000 7f859d32c640 1 -- 172.21.15.2:0/3395009194 >> 172.21.15.2:0/3395009194 conn(0x7f8598076770 msgr2=0x7f8598076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.508+0000 7f859d32c640 1 -- 172.21.15.2:0/3395009194 shutdown_connections 2024-03-15T13:09:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.508+0000 7f859d32c640 1 -- 172.21.15.2:0/3395009194 wait complete. 2024-03-15T13:09:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f859d32c640 1 Processor -- start 2024-03-15T13:09:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f859d32c640 1 -- start start 2024-03-15T13:09:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f859d32c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8598102730 0x7f859819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f859d32c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8598103050 0x7f859819dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f859d32c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859810bb00 0x7f8598197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f859d32c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8598197f90 con 0x7f8598102730 2024-03-15T13:09:03.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f859d32c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8598198100 con 0x7f8598103050 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f859d32c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85981982c0 con 0x7f859810bb00 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f8596ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8598102730 0x7f859819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f8596ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8598102730 0x7f859819d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57892/0 (socket says 172.21.15.2:57892) 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f8596ffd640 1 -- 172.21.15.2:0/1174511665 learned_addr learned my addr 172.21.15.2:0/1174511665 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f858ffff640 1 --2- 172.21.15.2:0/1174511665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8598103050 0x7f859819dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f85977fe640 1 --2- 172.21.15.2:0/1174511665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859810bb00 0x7f8598197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f8596ffd640 1 -- 172.21.15.2:0/1174511665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859810bb00 msgr2=0x7f8598197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f8596ffd640 1 --2- 172.21.15.2:0/1174511665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859810bb00 0x7f8598197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f8596ffd640 1 -- 172.21.15.2:0/1174511665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8598103050 msgr2=0x7f859819dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f8596ffd640 1 --2- 172.21.15.2:0/1174511665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8598103050 0x7f859819dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.509+0000 7f8596ffd640 1 -- 172.21.15.2:0/1174511665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8590009900 con 0x7f8598102730 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.510+0000 7f8596ffd640 1 --2- 172.21.15.2:0/1174511665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8598102730 0x7f859819d790 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f858800e9f0 tx=0x7f858800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.510+0000 7f8594ff9640 1 -- 172.21.15.2:0/1174511665 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588009720 con 0x7f8598102730 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.510+0000 7f8594ff9640 1 -- 172.21.15.2:0/1174511665 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f858800cd80 con 0x7f8598102730 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.510+0000 7f859d32c640 1 -- 172.21.15.2:0/1174511665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85981985a0 con 0x7f8598102730 2024-03-15T13:09:03.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.510+0000 7f8594ff9640 1 -- 172.21.15.2:0/1174511665 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588010690 con 0x7f8598102730 2024-03-15T13:09:03.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.510+0000 7f859d32c640 1 -- 172.21.15.2:0/1174511665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85981a4830 con 0x7f8598102730 2024-03-15T13:09:03.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.511+0000 7f859d32c640 1 -- 172.21.15.2:0/1174511665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859806a380 con 0x7f8598102730 2024-03-15T13:09:03.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.512+0000 7f8594ff9640 1 -- 172.21.15.2:0/1174511665 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f85880107f0 con 0x7f8598102730 2024-03-15T13:09:03.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.512+0000 7f8594ff9640 1 --2- 172.21.15.2:0/1174511665 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f85840a1c20 0x7f85840a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.513+0000 7f858ffff640 1 --2- 172.21.15.2:0/1174511665 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f85840a1c20 0x7f85840a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.513+0000 7f8594ff9640 1 -- 172.21.15.2:0/1174511665 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f8588014070 con 0x7f8598102730 2024-03-15T13:09:03.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.513+0000 7f858ffff640 1 --2- 172.21.15.2:0/1174511665 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f85840a1c20 0x7f85840a40e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f85980ffbd0 tx=0x7f8580005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.516+0000 7f8594ff9640 1 -- 172.21.15.2:0/1174511665 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8588096bc0 con 0x7f8598102730 2024-03-15T13:09:03.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.738+0000 7f859d32c640 1 -- 172.21.15.2:0/1174511665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8598106c80 con 0x7f8598102730 2024-03-15T13:09:03.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.739+0000 7f8594ff9640 1 -- 172.21.15.2:0/1174511665 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+11453 (secure 0 0 0) 0x7f8598106c80 con 0x7f8598102730 2024-03-15T13:09:03.726 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:09:03.727 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:05:12.138845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24500,24464,24460,24434,24539]},"id":1}]} 2024-03-15T13:09:03.727 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 50 2024-03-15T13:09:03.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.741+0000 7f859d32c640 1 -- 172.21.15.2:0/1174511665 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f85840a1c20 msgr2=0x7f85840a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.741+0000 7f859d32c640 1 --2- 172.21.15.2:0/1174511665 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f85840a1c20 0x7f85840a40e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f85980ffbd0 tx=0x7f8580005f90 comp rx=0 tx=0).stop 2024-03-15T13:09:03.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.741+0000 7f859d32c640 1 -- 172.21.15.2:0/1174511665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8598102730 msgr2=0x7f859819d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.741+0000 7f859d32c640 1 --2- 172.21.15.2:0/1174511665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8598102730 0x7f859819d790 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f858800e9f0 tx=0x7f858800edf0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.741+0000 7f859d32c640 1 -- 172.21.15.2:0/1174511665 shutdown_connections 2024-03-15T13:09:03.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.741+0000 7f859d32c640 1 --2- 172.21.15.2:0/1174511665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f859810bb00 0x7f8598197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.741+0000 7f859d32c640 1 --2- 172.21.15.2:0/1174511665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8598103050 0x7f859819dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.741+0000 7f859d32c640 1 --2- 172.21.15.2:0/1174511665 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f85840a1c20 0x7f85840a40e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.741+0000 7f859d32c640 1 --2- 172.21.15.2:0/1174511665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8598102730 0x7f859819d790 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.741+0000 7f859d32c640 1 -- 172.21.15.2:0/1174511665 >> 172.21.15.2:0/1174511665 conn(0x7f8598076770 msgr2=0x7f85980fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.741+0000 7f859d32c640 1 -- 172.21.15.2:0/1174511665 shutdown_connections 2024-03-15T13:09:03.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.741+0000 7f859d32c640 1 -- 172.21.15.2:0/1174511665 wait complete. 2024-03-15T13:09:03.736 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:09:03.736 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-15T13:09:03.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.856+0000 7f7fa6c15640 1 -- 172.21.15.2:0/3115269938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa01056a0 msgr2=0x7f7fa010d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.856+0000 7f7fa6c15640 1 --2- 172.21.15.2:0/3115269938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa01056a0 0x7f7fa010d270 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009cd0 tx=0x7f7f9802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.856+0000 7f7fa6c15640 1 -- 172.21.15.2:0/3115269938 shutdown_connections 2024-03-15T13:09:03.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.856+0000 7f7fa6c15640 1 --2- 172.21.15.2:0/3115269938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa01056a0 0x7f7fa010d270 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.856+0000 7f7fa6c15640 1 --2- 172.21.15.2:0/3115269938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fa0100f80 0x7f7fa0104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.856+0000 7f7fa6c15640 1 --2- 172.21.15.2:0/3115269938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fa01005d0 0x7f7fa01009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.856+0000 7f7fa6c15640 1 -- 172.21.15.2:0/3115269938 >> 172.21.15.2:0/3115269938 conn(0x7f7fa00774f0 msgr2=0x7f7fa0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.856+0000 7f7fa6c15640 1 -- 172.21.15.2:0/3115269938 shutdown_connections 2024-03-15T13:09:03.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.856+0000 7f7fa6c15640 1 -- 172.21.15.2:0/3115269938 wait complete. 2024-03-15T13:09:03.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.856+0000 7f7fa6c15640 1 Processor -- start 2024-03-15T13:09:03.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.856+0000 7f7fa6c15640 1 -- start start 2024-03-15T13:09:03.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa6c15640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fa01005d0 0x7f7fa0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa6c15640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fa01056a0 0x7f7fa0198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa6c15640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa0199440 0x7f7fa01a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa6c15640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01a4090 con 0x7f7fa0199440 2024-03-15T13:09:03.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa6c15640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01a4200 con 0x7f7fa01056a0 2024-03-15T13:09:03.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa6c15640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01a43c0 con 0x7f7fa01005d0 2024-03-15T13:09:03.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa51b2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa0199440 0x7f7fa01a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa51b2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa0199440 0x7f7fa01a39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57912/0 (socket says 172.21.15.2:57912) 2024-03-15T13:09:03.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa51b2640 1 -- 172.21.15.2:0/1965661894 learned_addr learned my addr 172.21.15.2:0/1965661894 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:03.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7f9ffff640 1 --2- 172.21.15.2:0/1965661894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fa01056a0 0x7f7fa0198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa49b1640 1 --2- 172.21.15.2:0/1965661894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fa01005d0 0x7f7fa0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa51b2640 1 -- 172.21.15.2:0/1965661894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fa01005d0 msgr2=0x7f7fa0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa51b2640 1 --2- 172.21.15.2:0/1965661894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fa01005d0 0x7f7fa0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa51b2640 1 -- 172.21.15.2:0/1965661894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fa01056a0 msgr2=0x7f7fa0198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa51b2640 1 --2- 172.21.15.2:0/1965661894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fa01056a0 0x7f7fa0198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa51b2640 1 -- 172.21.15.2:0/1965661894 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f98009900 con 0x7f7fa0199440 2024-03-15T13:09:03.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7f9ffff640 1 --2- 172.21.15.2:0/1965661894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fa01056a0 0x7f7fa0198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:03.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7fa51b2640 1 --2- 172.21.15.2:0/1965661894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa0199440 0x7f7fa01a39f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7f98005cc0 tx=0x7f7f98034a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.857+0000 7f7f9dffb640 1 -- 172.21.15.2:0/1965661894 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98032880 con 0x7f7fa0199440 2024-03-15T13:09:03.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.858+0000 7f7fa6c15640 1 -- 172.21.15.2:0/1965661894 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa01a4640 con 0x7f7fa0199440 2024-03-15T13:09:03.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.858+0000 7f7f9dffb640 1 -- 172.21.15.2:0/1965661894 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7f98034e80 con 0x7f7fa0199440 2024-03-15T13:09:03.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.858+0000 7f7f9dffb640 1 -- 172.21.15.2:0/1965661894 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f980047c0 con 0x7f7fa0199440 2024-03-15T13:09:03.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.858+0000 7f7fa6c15640 1 -- 172.21.15.2:0/1965661894 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa01a4b40 con 0x7f7fa0199440 2024-03-15T13:09:03.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.860+0000 7f7f9dffb640 1 -- 172.21.15.2:0/1965661894 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f7f9800a040 con 0x7f7fa0199440 2024-03-15T13:09:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.860+0000 7f7fa97fc740 1 Processor -- start 2024-03-15T13:09:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.860+0000 7f7fa97fc740 1 -- start start 2024-03-15T13:09:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7fa97fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 0x55cbec61e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7fa97fc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61eb70 0x55cbec627210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7f9dffb640 1 --2- 172.21.15.2:0/1965661894 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7f900a1ba0 0x7f7f900a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7fa97fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec627750 0x55cbec629b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7fa97fc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec5d9960 con 0x55cbec61eb70 2024-03-15T13:09:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7fa97fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec58d310 con 0x55cbec61e250 2024-03-15T13:09:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7fa97fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec58a840 con 0x55cbec627750 2024-03-15T13:09:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7f9ffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 0x55cbec61e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7fa49b1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61eb70 0x55cbec627210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7f9ffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 0x55cbec61e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:49410/0 (socket says 172.21.15.2:49410) 2024-03-15T13:09:03.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7fa51b2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec627750 0x55cbec629b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7f9ffff640 1 -- 172.21.15.2:0/1334464304 learned_addr learned my addr 172.21.15.2:0/1334464304 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7f9ffff640 1 --2- 172.21.15.2:0/1965661894 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7f900a1ba0 0x7f7f900a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7f9dffb640 1 -- 172.21.15.2:0/1965661894 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7f980f0690 con 0x7f7fa0199440 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7f9ffff640 1 -- 172.21.15.2:0/1334464304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec627750 msgr2=0x55cbec629b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7f9ffff640 1 --2- 172.21.15.2:0/1334464304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec627750 0x55cbec629b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7f9ffff640 1 -- 172.21.15.2:0/1334464304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61eb70 msgr2=0x55cbec627210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7f9ffff640 1 --2- 172.21.15.2:0/1334464304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61eb70 0x55cbec627210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7f9ffff640 1 -- 172.21.15.2:0/1334464304 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbec548060 con 0x55cbec61e250 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7fa49b1640 1 --2- 172.21.15.2:0/1334464304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61eb70 0x55cbec627210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7fa51b2640 1 --2- 172.21.15.2:0/1334464304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec627750 0x55cbec629b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7f9ffff640 1 --2- 172.21.15.2:0/1334464304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 0x55cbec61e630 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c011df0 tx=0x7f7f8c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb3847e2cb117262 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7f9ffff640 1 --2- 172.21.15.2:0/1965661894 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7f900a1ba0 0x7f7f900a4060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00fc90 tx=0x7f7f8c018000 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.861+0000 7f7f8affd640 1 -- 172.21.15.2:0/1334464304 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00fea0 con 0x55cbec61e250 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.862+0000 7f7f8affd640 1 -- 172.21.15.2:0/1334464304 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7f8c004500 con 0x55cbec61e250 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.862+0000 7f7f8affd640 1 -- 172.21.15.2:0/1334464304 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c003850 con 0x55cbec61e250 2024-03-15T13:09:03.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.862+0000 7f7fa97fc740 1 -- 172.21.15.2:0/1334464304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 msgr2=0x55cbec61e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.862+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/1334464304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 0x55cbec61e630 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c011df0 tx=0x7f7f8c00f790 comp rx=0 tx=0).stop 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.862+0000 7f7fa97fc740 1 -- 172.21.15.2:0/1334464304 shutdown_connections 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.862+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/1334464304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec627750 0x55cbec629b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.862+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/1334464304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61eb70 0x55cbec627210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.862+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/1334464304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 0x55cbec61e630 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.862+0000 7f7fa97fc740 1 -- 172.21.15.2:0/1334464304 >> 172.21.15.2:0/1334464304 conn(0x55cbec618980 msgr2=0x55cbec618d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.862+0000 7f7fa97fc740 1 -- 172.21.15.2:0/1334464304 shutdown_connections 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.862+0000 7f7fa97fc740 1 -- 172.21.15.2:0/1334464304 wait complete. 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.863+0000 7f7fa97fc740 1 -- start start 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.863+0000 7f7fa97fc740 1 Processor -- start 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.863+0000 7f7fa97fc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61e250 0x55cbec627110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.863+0000 7f7fa97fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec61eb70 0x55cbec6244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.863+0000 7f7fa97fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec627750 0x55cbec624ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.863+0000 7f7fa97fc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec586020 con 0x55cbec61e250 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.863+0000 7f7fa97fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec259430 con 0x55cbec627750 2024-03-15T13:09:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.863+0000 7f7fa97fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec215e00 con 0x55cbec61eb70 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.863+0000 7f7fa97fc740 10 client.0 _ll_register_callbacks cb 0x55cbec52dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.863+0000 7f7fa49b1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61e250 0x55cbec627110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.863+0000 7f7fa49b1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61e250 0x55cbec627110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57918/0 (socket says 172.21.15.2:57918) 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.863+0000 7f7fa49b1640 1 -- 172.21.15.2:0/2595216460 learned_addr learned my addr 172.21.15.2:0/2595216460 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.863+0000 7f7f9ffff640 1 --2- 172.21.15.2:0/2595216460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec61eb70 0x55cbec6244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.863+0000 7f7fa51b2640 1 --2- 172.21.15.2:0/2595216460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec627750 0x55cbec624ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7fa49b1640 1 -- 172.21.15.2:0/2595216460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec61eb70 msgr2=0x55cbec6244a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7fa49b1640 1 --2- 172.21.15.2:0/2595216460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec61eb70 0x55cbec6244a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7fa49b1640 1 -- 172.21.15.2:0/2595216460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec627750 msgr2=0x55cbec624ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7fa49b1640 1 --2- 172.21.15.2:0/2595216460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec627750 0x55cbec624ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7fa49b1640 1 -- 172.21.15.2:0/2595216460 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c019040 con 0x55cbec61e250 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7fa49b1640 1 --2- 172.21.15.2:0/2595216460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61e250 0x55cbec627110 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f7f94012740 tx=0x7f7f94012b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f496b485d1fe0ac server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7f7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7f7ffff640 1 -- 172.21.15.2:0/2595216460 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94017070 con 0x55cbec61e250 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7f7ffff640 1 -- 172.21.15.2:0/2595216460 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7f94004050 con 0x55cbec61e250 2024-03-15T13:09:03.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7fa97fc740 1 -- 172.21.15.2:0/2595216460 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbec2c2720 con 0x55cbec61e250 2024-03-15T13:09:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7f7ffff640 1 -- 172.21.15.2:0/2595216460 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f94004210 con 0x55cbec61e250 2024-03-15T13:09:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7fa97fc740 10 client.18573 fetch_fsmap learned FSMap version 50 2024-03-15T13:09:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7fa97fc740 1 -- 172.21.15.2:0/2595216460 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55cbec5846a0 con 0x55cbec61e250 2024-03-15T13:09:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7f7ffff640 1 -- 172.21.15.2:0/2595216460 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55cbec2c2720 con 0x55cbec61e250 2024-03-15T13:09:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7f7ffff640 1 -- 172.21.15.2:0/2595216460 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7f940039f0 con 0x55cbec61e250 2024-03-15T13:09:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7fa97fc740 10 client.18573 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:09:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7fa97fc740 10 client.18573 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:09:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7fa97fc740 20 client.18573 populate_metadata read hostname 'smithi002' 2024-03-15T13:09:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/2595216460 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55cbec637920 0x55cbec639d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7fa97fc740 4 client.18573 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:09:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.864+0000 7f7fa97fc740 1 -- 172.21.15.2:0/2595216460 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbec311890 con 0x55cbec637920 2024-03-15T13:09:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.865+0000 7f7f9ffff640 1 --2- 172.21.15.2:0/2595216460 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55cbec637920 0x55cbec639d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.865+0000 7f7f7ffff640 10 client.18573 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:09:03.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.865+0000 7f7f9ffff640 1 --2- 172.21.15.2:0/2595216460 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55cbec637920 0x55cbec639d40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7424a5a6535dd502 server_cookie=52a28cdb111223aa in_seq=0 out_seq=0 2024-03-15T13:09:03.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.866+0000 7f7f7ffff640 1 -- 172.21.15.2:0/2595216460 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cbec311890 con 0x55cbec637920 2024-03-15T13:09:03.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.866+0000 7f7f7ffff640 10 client.18573 handle_command_reply: tid=0 2024-03-15T13:09:03.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.866+0000 7f7fa97fc740 1 client.18573 shutdown 2024-03-15T13:09:03.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.866+0000 7f7fa97fc740 1 -- 172.21.15.2:0/2595216460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61e250 msgr2=0x55cbec627110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.866+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/2595216460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61e250 0x55cbec627110 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f7f94012740 tx=0x7f7f94012b40 comp rx=0 tx=0).stop 2024-03-15T13:09:03.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.866+0000 7f7fa97fc740 1 -- 172.21.15.2:0/2595216460 shutdown_connections 2024-03-15T13:09:03.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.866+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/2595216460 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55cbec637920 0x55cbec639d40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.866+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/2595216460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec627750 0x55cbec624ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.866+0000 7f7f7ffff640 0 client.18573 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:09:03.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.866+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/2595216460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec61eb70 0x55cbec6244a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.866+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/2595216460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61e250 0x55cbec627110 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.866+0000 7f7fa97fc740 1 -- 172.21.15.2:0/2595216460 >> 172.21.15.2:0/2595216460 conn(0x55cbec618980 msgr2=0x55cbec61a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.866+0000 7f7fa97fc740 1 -- 172.21.15.2:0/2595216460 shutdown_connections 2024-03-15T13:09:03.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.866+0000 7f7fa97fc740 1 -- 172.21.15.2:0/2595216460 wait complete. 2024-03-15T13:09:03.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.866+0000 7f7fa97fc740 20 client.18573 trim_cache size 0 max 16384 2024-03-15T13:09:03.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:03 smithi149 ceph-mon[35927]: pgmap v313: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-15T13:09:03.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7fa97fc740 1 Processor -- start 2024-03-15T13:09:03.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7fa97fc740 1 -- start start 2024-03-15T13:09:03.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7fa97fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 0x55cbec617a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7fa97fc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61eb70 0x55cbec626bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7fa97fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec627750 0x55cbec627110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7fa97fc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec50b130 con 0x55cbec61eb70 2024-03-15T13:09:03.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7fa97fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec625850 con 0x55cbec61e250 2024-03-15T13:09:03.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7fa97fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec6200a0 con 0x55cbec627750 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7f9ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61eb70 0x55cbec626bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7f9ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61eb70 0x55cbec626bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57922/0 (socket says 172.21.15.2:57922) 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7f9ffff640 1 -- 172.21.15.2:0/3010197141 learned_addr learned my addr 172.21.15.2:0/3010197141 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7fa49b1640 1 --2- 172.21.15.2:0/3010197141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 0x55cbec617a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7fa51b2640 1 --2- 172.21.15.2:0/3010197141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec627750 0x55cbec627110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7f9ffff640 1 -- 172.21.15.2:0/3010197141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec627750 msgr2=0x55cbec627110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7f9ffff640 1 --2- 172.21.15.2:0/3010197141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec627750 0x55cbec627110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7f9ffff640 1 -- 172.21.15.2:0/3010197141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 msgr2=0x55cbec617a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7f9ffff640 1 --2- 172.21.15.2:0/3010197141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 0x55cbec617a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7f9ffff640 1 -- 172.21.15.2:0/3010197141 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbec520c20 con 0x55cbec61eb70 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7f9ffff640 1 --2- 172.21.15.2:0/3010197141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61eb70 0x55cbec626bd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c0112e0 tx=0x7f7f8c011310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34c09d3bd05187 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7f8a7fc640 1 -- 172.21.15.2:0/3010197141 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c02b040 con 0x55cbec61eb70 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.879+0000 7f7f8a7fc640 1 -- 172.21.15.2:0/3010197141 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7f8c0110c0 con 0x55cbec61eb70 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.880+0000 7f7f8a7fc640 1 -- 172.21.15.2:0/3010197141 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00edc0 con 0x55cbec61eb70 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.880+0000 7f7fa97fc740 1 -- 172.21.15.2:0/3010197141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61eb70 msgr2=0x55cbec626bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.880+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/3010197141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61eb70 0x55cbec626bd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c0112e0 tx=0x7f7f8c011310 comp rx=0 tx=0).stop 2024-03-15T13:09:03.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.880+0000 7f7fa97fc740 1 -- 172.21.15.2:0/3010197141 shutdown_connections 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.880+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/3010197141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec627750 0x55cbec627110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.880+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/3010197141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec61eb70 0x55cbec626bd0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.880+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/3010197141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 0x55cbec617a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.880+0000 7f7fa97fc740 1 -- 172.21.15.2:0/3010197141 >> 172.21.15.2:0/3010197141 conn(0x55cbec618980 msgr2=0x55cbec6172f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.880+0000 7f7fa97fc740 1 -- 172.21.15.2:0/3010197141 shutdown_connections 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.880+0000 7f7fa97fc740 1 -- 172.21.15.2:0/3010197141 wait complete. 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.880+0000 7f7fa97fc740 1 -- start start 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.880+0000 7f7fa97fc740 1 Processor -- start 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa97fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 0x55cbec62b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa97fc740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec61eb70 0x55cbec62b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa97fc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec627750 0x55cbec68c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa97fc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec68c6c0 con 0x55cbec627750 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa97fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec68c830 con 0x55cbec61e250 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa97fc740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbec68c9f0 con 0x55cbec61eb70 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa97fc740 10 client.0 _ll_register_callbacks cb 0x7f7f94009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:09:03.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa51b2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec627750 0x55cbec68c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa51b2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec627750 0x55cbec68c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57934/0 (socket says 172.21.15.2:57934) 2024-03-15T13:09:03.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa51b2640 1 -- 172.21.15.2:0/1183328541 learned_addr learned my addr 172.21.15.2:0/1183328541 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:03.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa49b1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 0x55cbec62b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7f9ffff640 1 --2- 172.21.15.2:0/1183328541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec61eb70 0x55cbec62b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa51b2640 1 -- 172.21.15.2:0/1183328541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec61eb70 msgr2=0x55cbec62b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa51b2640 1 --2- 172.21.15.2:0/1183328541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec61eb70 0x55cbec62b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa51b2640 1 -- 172.21.15.2:0/1183328541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 msgr2=0x55cbec62b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa51b2640 1 --2- 172.21.15.2:0/1183328541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 0x55cbec62b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa51b2640 1 -- 172.21.15.2:0/1183328541 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbec548060 con 0x55cbec627750 2024-03-15T13:09:03.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7fa51b2640 1 --2- 172.21.15.2:0/1183328541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec627750 0x55cbec68c180 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7f980bcfc0 tx=0x7f7f9803c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4a7c1321074cca server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7f7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:09:03.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.881+0000 7f7f7ffff640 1 -- 172.21.15.2:0/1183328541 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f980bd730 con 0x55cbec627750 2024-03-15T13:09:03.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.882+0000 7f7f7ffff640 1 -- 172.21.15.2:0/1183328541 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7f980ef070 con 0x55cbec627750 2024-03-15T13:09:03.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.882+0000 7f7fa97fc740 1 -- 172.21.15.2:0/1183328541 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbec2857e0 con 0x55cbec627750 2024-03-15T13:09:03.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.882+0000 7f7f7ffff640 1 -- 172.21.15.2:0/1183328541 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f980bd030 con 0x55cbec627750 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.882+0000 7f7fa97fc740 10 client.18579 fetch_fsmap learned FSMap version 50 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.882+0000 7f7fa97fc740 1 -- 172.21.15.2:0/1183328541 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55cbec5468c0 con 0x55cbec627750 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.882+0000 7f7f7ffff640 1 -- 172.21.15.2:0/1183328541 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55cbec2857e0 con 0x55cbec627750 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.882+0000 7f7f7ffff640 1 -- 172.21.15.2:0/1183328541 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f7f980b96a0 con 0x55cbec627750 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.882+0000 7f7fa97fc740 10 client.18579 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.882+0000 7f7fa97fc740 10 client.18579 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.882+0000 7f7fa97fc740 20 client.18579 populate_metadata read hostname 'smithi002' 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.882+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/1183328541 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55cbec694060 0x55cbec696450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.882+0000 7f7fa97fc740 4 client.18579 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.882+0000 7f7fa97fc740 1 -- 172.21.15.2:0/1183328541 --> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cbec231700 con 0x55cbec694060 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.882+0000 7f7fa49b1640 1 --2- 172.21.15.2:0/1183328541 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55cbec694060 0x55cbec696450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7f7ffff640 10 client.18579 ms_handle_connect on v2:172.21.15.149:6832/2587208478 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7fa49b1640 1 --2- 172.21.15.2:0/1183328541 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55cbec694060 0x55cbec696450 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4d619ec45277c846 server_cookie=157da5a30214a1d0 in_seq=0 out_seq=0 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7f7ffff640 1 -- 172.21.15.2:0/1183328541 <== mds.4 v2:172.21.15.149:6832/2587208478 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cbec231700 con 0x55cbec694060 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7f7ffff640 10 client.18579 handle_command_reply: tid=0 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7fa97fc740 1 client.18579 shutdown 2024-03-15T13:09:03.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7fa97fc740 1 -- 172.21.15.2:0/1183328541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec627750 msgr2=0x55cbec68c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/1183328541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec627750 0x55cbec68c180 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7f980bcfc0 tx=0x7f7f9803c6f0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7fa97fc740 1 -- 172.21.15.2:0/1183328541 shutdown_connections 2024-03-15T13:09:03.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/1183328541 >> [v2:172.21.15.149:6832/2587208478,v1:172.21.15.149:6833/2587208478] conn(0x55cbec694060 0x55cbec696450 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/1183328541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cbec627750 0x55cbec68c180 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/1183328541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cbec61eb70 0x55cbec62b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7f7ffff640 0 client.18579 ms_handle_reset on v2:172.21.15.149:6832/2587208478 2024-03-15T13:09:03.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7fa97fc740 1 --2- 172.21.15.2:0/1183328541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cbec61e250 0x55cbec62b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7fa97fc740 1 -- 172.21.15.2:0/1183328541 >> 172.21.15.2:0/1183328541 conn(0x55cbec618980 msgr2=0x55cbec61f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7fa97fc740 1 -- 172.21.15.2:0/1183328541 shutdown_connections 2024-03-15T13:09:03.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7fa97fc740 1 -- 172.21.15.2:0/1183328541 wait complete. 2024-03-15T13:09:03.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.883+0000 7f7fa97fc740 20 client.18579 trim_cache size 0 max 16384 2024-03-15T13:09:03.873 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:09:03.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.884+0000 7f7fa6c15640 1 -- 172.21.15.2:0/1965661894 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7f900a1ba0 msgr2=0x7f7f900a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.884+0000 7f7fa6c15640 1 --2- 172.21.15.2:0/1965661894 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7f900a1ba0 0x7f7f900a4060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00fc90 tx=0x7f7f8c018000 comp rx=0 tx=0).stop 2024-03-15T13:09:03.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.884+0000 7f7fa6c15640 1 -- 172.21.15.2:0/1965661894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa0199440 msgr2=0x7f7fa01a39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.884+0000 7f7fa6c15640 1 --2- 172.21.15.2:0/1965661894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa0199440 0x7f7fa01a39f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7f98005cc0 tx=0x7f7f98034a80 comp rx=0 tx=0).stop 2024-03-15T13:09:03.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.884+0000 7f7fa6c15640 1 -- 172.21.15.2:0/1965661894 shutdown_connections 2024-03-15T13:09:03.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.884+0000 7f7fa6c15640 1 --2- 172.21.15.2:0/1965661894 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7f900a1ba0 0x7f7f900a4060 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.884+0000 7f7fa6c15640 1 --2- 172.21.15.2:0/1965661894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa0199440 0x7f7fa01a39f0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.884+0000 7f7fa6c15640 1 --2- 172.21.15.2:0/1965661894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fa01056a0 0x7f7fa0198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.884+0000 7f7fa6c15640 1 --2- 172.21.15.2:0/1965661894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fa01005d0 0x7f7fa0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.884+0000 7f7fa6c15640 1 -- 172.21.15.2:0/1965661894 >> 172.21.15.2:0/1965661894 conn(0x7f7fa00774f0 msgr2=0x7f7fa00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.884+0000 7f7fa6c15640 1 -- 172.21.15.2:0/1965661894 shutdown_connections 2024-03-15T13:09:03.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.884+0000 7f7fa6c15640 1 -- 172.21.15.2:0/1965661894 wait complete. 2024-03-15T13:09:03.877 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:03 smithi002 ceph-mon[32060]: pgmap v313: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-15T13:09:03.878 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:09:03.879 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-15T13:09:03.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.999+0000 7f33b7172640 1 -- 172.21.15.2:0/3661141539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b01056a0 msgr2=0x7f33b010d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.999+0000 7f33b7172640 1 --2- 172.21.15.2:0/3661141539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b01056a0 0x7f33b010d270 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009cd0 tx=0x7f33a802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.999+0000 7f33b7172640 1 -- 172.21.15.2:0/3661141539 shutdown_connections 2024-03-15T13:09:03.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.999+0000 7f33b7172640 1 --2- 172.21.15.2:0/3661141539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b01056a0 0x7f33b010d270 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.999+0000 7f33b7172640 1 --2- 172.21.15.2:0/3661141539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b0100f80 0x7f33b0104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.999+0000 7f33b7172640 1 --2- 172.21.15.2:0/3661141539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33b01005d0 0x7f33b01009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.999+0000 7f33b7172640 1 -- 172.21.15.2:0/3661141539 >> 172.21.15.2:0/3661141539 conn(0x7f33b00774f0 msgr2=0x7f33b0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.999+0000 7f33b7172640 1 -- 172.21.15.2:0/3661141539 shutdown_connections 2024-03-15T13:09:03.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:03.999+0000 7f33b7172640 1 -- 172.21.15.2:0/3661141539 wait complete. 2024-03-15T13:09:03.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b7172640 1 Processor -- start 2024-03-15T13:09:03.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b7172640 1 -- start start 2024-03-15T13:09:03.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b7172640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b01005d0 0x7f33b019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b7172640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b01056a0 0x7f33b019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b7172640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33b01978f0 0x7f33b0197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b7172640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b01982e0 con 0x7f33b01005d0 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b7172640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b0198450 con 0x7f33b01978f0 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b7172640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b0198610 con 0x7f33b01056a0 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b4f0e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b01005d0 0x7f33b019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b4f0e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b01005d0 0x7f33b019d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57948/0 (socket says 172.21.15.2:57948) 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b4f0e640 1 -- 172.21.15.2:0/3434342388 learned_addr learned my addr 172.21.15.2:0/3434342388 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b570f640 1 --2- 172.21.15.2:0/3434342388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33b01978f0 0x7f33b0197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33affff640 1 --2- 172.21.15.2:0/3434342388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b01056a0 0x7f33b019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b4f0e640 1 -- 172.21.15.2:0/3434342388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b01056a0 msgr2=0x7f33b019dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b4f0e640 1 --2- 172.21.15.2:0/3434342388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b01056a0 0x7f33b019dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b4f0e640 1 -- 172.21.15.2:0/3434342388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33b01978f0 msgr2=0x7f33b0197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b4f0e640 1 --2- 172.21.15.2:0/3434342388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33b01978f0 0x7f33b0197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.000+0000 7f33b4f0e640 1 -- 172.21.15.2:0/3434342388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a8009900 con 0x7f33b01005d0 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.001+0000 7f33b4f0e640 1 --2- 172.21.15.2:0/3434342388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b01005d0 0x7f33b019d740 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f33a000e9f0 tx=0x7f33a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.001+0000 7f33adffb640 1 -- 172.21.15.2:0/3434342388 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0009700 con 0x7f33b01005d0 2024-03-15T13:09:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.001+0000 7f33adffb640 1 -- 172.21.15.2:0/3434342388 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f33a000cd80 con 0x7f33b01005d0 2024-03-15T13:09:03.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.001+0000 7f33adffb640 1 -- 172.21.15.2:0/3434342388 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0018920 con 0x7f33b01005d0 2024-03-15T13:09:03.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.001+0000 7f33b7172640 1 -- 172.21.15.2:0/3434342388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b01a4600 con 0x7f33b01005d0 2024-03-15T13:09:03.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.001+0000 7f33b7172640 1 -- 172.21.15.2:0/3434342388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b01a4b10 con 0x7f33b01005d0 2024-03-15T13:09:03.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.003+0000 7f33adffb640 1 -- 172.21.15.2:0/3434342388 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f33a0018a80 con 0x7f33b01005d0 2024-03-15T13:09:03.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.003+0000 7f33b9e20740 1 Processor -- start 2024-03-15T13:09:03.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.003+0000 7f33b9e20740 1 -- start start 2024-03-15T13:09:03.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33adffb640 1 --2- 172.21.15.2:0/3434342388 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f339c0a1ba0 0x7f339c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33b9e20740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f12a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33b9e20740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f137620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33b9e20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 0x55dc3f139f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33b9e20740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3f0e5960 con 0x55dc3f12a820 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33b9e20740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3f099310 con 0x55dc3f129f00 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33b9e20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3f096840 con 0x55dc3f137b60 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33affff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f137620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33b570f640 1 --2- 172.21.15.2:0/3434342388 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f339c0a1ba0 0x7f339c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33b4f0e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f12a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33affff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f137620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57952/0 (socket says 172.21.15.2:57952) 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33affff640 1 -- 172.21.15.2:0/2501480635 learned_addr learned my addr 172.21.15.2:0/2501480635 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33adffb640 1 -- 172.21.15.2:0/3434342388 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f33a0014070 con 0x7f33b01005d0 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33b570f640 1 --2- 172.21.15.2:0/2501480635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 0x55dc3f139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33affff640 1 -- 172.21.15.2:0/2501480635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 msgr2=0x55dc3f139f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33affff640 1 --2- 172.21.15.2:0/2501480635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 0x55dc3f139f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33affff640 1 -- 172.21.15.2:0/2501480635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 msgr2=0x55dc3f12a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33affff640 1 --2- 172.21.15.2:0/2501480635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f12a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33affff640 1 -- 172.21.15.2:0/2501480635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc3f054060 con 0x55dc3f12a820 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33b570f640 1 --2- 172.21.15.2:0/2501480635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 0x55dc3f139f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33b4f0e640 1 --2- 172.21.15.2:0/2501480635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f12a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33affff640 1 --2- 172.21.15.2:0/2501480635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f137620 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f3398009c00 tx=0x7f3398011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1e3856eb94a4f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33b570f640 1 --2- 172.21.15.2:0/3434342388 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f339c0a1ba0 0x7f339c0a4060 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f33a8005cc0 tx=0x7f33a8009820 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33a6ffd640 1 -- 172.21.15.2:0/2501480635 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398004050 con 0x55dc3f12a820 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33a6ffd640 1 -- 172.21.15.2:0/2501480635 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3398005020 con 0x55dc3f12a820 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.004+0000 7f33a6ffd640 1 -- 172.21.15.2:0/2501480635 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398004c40 con 0x55dc3f12a820 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.005+0000 7f33b9e20740 1 -- 172.21.15.2:0/2501480635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 msgr2=0x55dc3f137620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.005+0000 7f33b9e20740 1 --2- 172.21.15.2:0/2501480635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f137620 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f3398009c00 tx=0x7f3398011810 comp rx=0 tx=0).stop 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.005+0000 7f33b9e20740 1 -- 172.21.15.2:0/2501480635 shutdown_connections 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.005+0000 7f33b9e20740 1 --2- 172.21.15.2:0/2501480635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 0x55dc3f139f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.005+0000 7f33b9e20740 1 --2- 172.21.15.2:0/2501480635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f137620 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.005+0000 7f33b9e20740 1 --2- 172.21.15.2:0/2501480635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f12a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.005+0000 7f33b9e20740 1 -- 172.21.15.2:0/2501480635 >> 172.21.15.2:0/2501480635 conn(0x55dc3f124980 msgr2=0x55dc3f124d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.005+0000 7f33b9e20740 1 -- 172.21.15.2:0/2501480635 shutdown_connections 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.005+0000 7f33b9e20740 1 -- 172.21.15.2:0/2501480635 wait complete. 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.005+0000 7f33b9e20740 1 -- start start 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33b9e20740 1 Processor -- start 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33b9e20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f129f00 0x55dc3f137420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33b9e20740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f1347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33b9e20740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f137b60 0x55dc3f134cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33b9e20740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3f092020 con 0x55dc3f12a820 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33b9e20740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3ed65430 con 0x55dc3f137b60 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33b9e20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3ed21e00 con 0x55dc3f129f00 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33b9e20740 10 client.0 _ll_register_callbacks cb 0x55dc3f039cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33affff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f1347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33affff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f1347b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57960/0 (socket says 172.21.15.2:57960) 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33affff640 1 -- 172.21.15.2:0/1803309620 learned_addr learned my addr 172.21.15.2:0/1803309620 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33b570f640 1 --2- 172.21.15.2:0/1803309620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f137b60 0x55dc3f134cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33b4f0e640 1 --2- 172.21.15.2:0/1803309620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f129f00 0x55dc3f137420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33affff640 1 -- 172.21.15.2:0/1803309620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f129f00 msgr2=0x55dc3f137420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33affff640 1 --2- 172.21.15.2:0/1803309620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f129f00 0x55dc3f137420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33affff640 1 -- 172.21.15.2:0/1803309620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f137b60 msgr2=0x55dc3f134cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:03.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33affff640 1 --2- 172.21.15.2:0/1803309620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f137b60 0x55dc3f134cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.006+0000 7f33affff640 1 -- 172.21.15.2:0/1803309620 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3398009830 con 0x55dc3f12a820 2024-03-15T13:09:03.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f33affff640 1 --2- 172.21.15.2:0/1803309620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f1347b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f33980057c0 tx=0x7f33980056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6a055acd638b7d server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:03.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f338ffff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:09:03.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f338ffff640 1 -- 172.21.15.2:0/1803309620 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398004e30 con 0x55dc3f12a820 2024-03-15T13:09:03.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f338ffff640 1 -- 172.21.15.2:0/1803309620 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f3398004050 con 0x55dc3f12a820 2024-03-15T13:09:03.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f33b9e20740 1 -- 172.21.15.2:0/1803309620 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc3ed917e0 con 0x55dc3f12a820 2024-03-15T13:09:03.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f338ffff640 1 -- 172.21.15.2:0/1803309620 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3398004250 con 0x55dc3f12a820 2024-03-15T13:09:03.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f33b9e20740 10 client.18591 fetch_fsmap learned FSMap version 50 2024-03-15T13:09:03.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f33b9e20740 1 -- 172.21.15.2:0/1803309620 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55dc3f0906a0 con 0x55dc3f12a820 2024-03-15T13:09:03.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f338ffff640 1 -- 172.21.15.2:0/1803309620 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55dc3ed917e0 con 0x55dc3f12a820 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f338ffff640 1 -- 172.21.15.2:0/1803309620 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3398021040 con 0x55dc3f12a820 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f33b9e20740 10 client.18591 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f33b9e20740 10 client.18591 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f33b9e20740 20 client.18591 populate_metadata read hostname 'smithi002' 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f33b9e20740 1 --2- 172.21.15.2:0/1803309620 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55dc3f147bb0 0x55dc3f149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f33b9e20740 4 client.18591 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.007+0000 7f33b9e20740 1 -- 172.21.15.2:0/1803309620 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc3edce720 con 0x55dc3f147bb0 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.008+0000 7f33b4f0e640 1 --2- 172.21.15.2:0/1803309620 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55dc3f147bb0 0x55dc3f149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.008+0000 7f33b4f0e640 1 --2- 172.21.15.2:0/1803309620 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55dc3f147bb0 0x55dc3f149fd0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=66387aeaf33b7a07 server_cookie=86bff0602b7ef183 in_seq=0 out_seq=0 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.008+0000 7f338ffff640 10 client.18591 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.008+0000 7f338ffff640 1 -- 172.21.15.2:0/1803309620 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f33a00a94b0 con 0x55dc3f147bb0 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.008+0000 7f338ffff640 10 client.18591 handle_command_reply: tid=0 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.009+0000 7f33b9e20740 1 client.18591 shutdown 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.009+0000 7f33b9e20740 1 -- 172.21.15.2:0/1803309620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 msgr2=0x55dc3f1347b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.009+0000 7f33b9e20740 1 --2- 172.21.15.2:0/1803309620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f1347b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f33980057c0 tx=0x7f33980056f0 comp rx=0 tx=0).stop 2024-03-15T13:09:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.009+0000 7f33b9e20740 1 -- 172.21.15.2:0/1803309620 shutdown_connections 2024-03-15T13:09:04.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.009+0000 7f33b9e20740 1 --2- 172.21.15.2:0/1803309620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f137b60 0x55dc3f134cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.009+0000 7f33b9e20740 1 --2- 172.21.15.2:0/1803309620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f1347b0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.009+0000 7f33b9e20740 1 --2- 172.21.15.2:0/1803309620 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55dc3f147bb0 0x55dc3f149fd0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.009+0000 7f33b9e20740 1 --2- 172.21.15.2:0/1803309620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f129f00 0x55dc3f137420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.009+0000 7f338ffff640 0 client.18591 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:09:04.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.009+0000 7f33b9e20740 1 -- 172.21.15.2:0/1803309620 >> 172.21.15.2:0/1803309620 conn(0x55dc3f124980 msgr2=0x55dc3f1266d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.009+0000 7f33b9e20740 1 -- 172.21.15.2:0/1803309620 shutdown_connections 2024-03-15T13:09:04.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.009+0000 7f33b9e20740 1 -- 172.21.15.2:0/1803309620 wait complete. 2024-03-15T13:09:04.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.009+0000 7f33b9e20740 20 client.18591 trim_cache size 0 max 16384 2024-03-15T13:09:04.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.021+0000 7f33b9e20740 1 Processor -- start 2024-03-15T13:09:04.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.021+0000 7f33b9e20740 1 -- start start 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.021+0000 7f33b9e20740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f12f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.021+0000 7f33b9e20740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f136ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.021+0000 7f33b9e20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 0x55dc3f137420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.021+0000 7f33b9e20740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3f017130 con 0x55dc3f12a820 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.021+0000 7f33b9e20740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3f135050 con 0x55dc3f129f00 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.021+0000 7f33b9e20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3f1244b0 con 0x55dc3f137b60 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.022+0000 7f33affff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f136ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.022+0000 7f33b4f0e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f12f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.022+0000 7f33b4f0e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f12f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:49474/0 (socket says 172.21.15.2:49474) 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.022+0000 7f33b4f0e640 1 -- 172.21.15.2:0/2514031130 learned_addr learned my addr 172.21.15.2:0/2514031130 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.022+0000 7f33b570f640 1 --2- 172.21.15.2:0/2514031130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 0x55dc3f137420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.022+0000 7f33affff640 1 -- 172.21.15.2:0/2514031130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 msgr2=0x55dc3f137420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.022+0000 7f33affff640 1 --2- 172.21.15.2:0/2514031130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 0x55dc3f137420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.022+0000 7f33affff640 1 -- 172.21.15.2:0/2514031130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 msgr2=0x55dc3f12f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.022+0000 7f33affff640 1 --2- 172.21.15.2:0/2514031130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f12f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.022+0000 7f33affff640 1 -- 172.21.15.2:0/2514031130 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc3f02cc20 con 0x55dc3f12a820 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.022+0000 7f33b4f0e640 1 --2- 172.21.15.2:0/2514031130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f12f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.022+0000 7f33affff640 1 --2- 172.21.15.2:0/2514031130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f136ee0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f3398005790 tx=0x7f3398005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0a9ed3e858c6c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.022+0000 7f33a67fc640 1 -- 172.21.15.2:0/2514031130 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398023430 con 0x55dc3f12a820 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.022+0000 7f33a67fc640 1 -- 172.21.15.2:0/2514031130 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3398011d10 con 0x55dc3f12a820 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.022+0000 7f33a67fc640 1 -- 172.21.15.2:0/2514031130 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398023be0 con 0x55dc3f12a820 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 -- 172.21.15.2:0/2514031130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 msgr2=0x55dc3f136ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 --2- 172.21.15.2:0/2514031130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f136ee0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f3398005790 tx=0x7f3398005510 comp rx=0 tx=0).stop 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 -- 172.21.15.2:0/2514031130 shutdown_connections 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 --2- 172.21.15.2:0/2514031130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 0x55dc3f137420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 --2- 172.21.15.2:0/2514031130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f136ee0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 --2- 172.21.15.2:0/2514031130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f12f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 -- 172.21.15.2:0/2514031130 >> 172.21.15.2:0/2514031130 conn(0x55dc3f124980 msgr2=0x55dc3f123b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 -- 172.21.15.2:0/2514031130 shutdown_connections 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 -- 172.21.15.2:0/2514031130 wait complete. 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 -- start start 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 Processor -- start 2024-03-15T13:09:04.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f13ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f13bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 0x55dc3f19b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3f19bb20 con 0x55dc3f12a820 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3f19bc90 con 0x55dc3f129f00 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.023+0000 7f33b9e20740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3f19be50 con 0x55dc3f137b60 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33b9e20740 10 client.0 _ll_register_callbacks cb 0x7f33a0096aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33affff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f13bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33b4f0e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f13ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33b570f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 0x55dc3f19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33b4f0e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f13ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:49484/0 (socket says 172.21.15.2:49484) 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33b570f640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 0x55dc3f19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:58926/0 (socket says 172.21.15.2:58926) 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33affff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f13bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57978/0 (socket says 172.21.15.2:57978) 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33b4f0e640 1 -- 172.21.15.2:0/1225400756 learned_addr learned my addr 172.21.15.2:0/1225400756 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33affff640 1 -- 172.21.15.2:0/1225400756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 msgr2=0x55dc3f19b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33affff640 1 --2- 172.21.15.2:0/1225400756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 0x55dc3f19b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33affff640 1 -- 172.21.15.2:0/1225400756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 msgr2=0x55dc3f13ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33affff640 1 --2- 172.21.15.2:0/1225400756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f13ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33affff640 1 -- 172.21.15.2:0/1225400756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b01a4600 con 0x55dc3f12a820 2024-03-15T13:09:04.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33b570f640 1 --2- 172.21.15.2:0/1225400756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 0x55dc3f19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:04.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33affff640 1 --2- 172.21.15.2:0/1225400756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f13bf70 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x55dc3f132f90 tx=0x7f3398009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954075032085858 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f338ffff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:09:04.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f338ffff640 1 -- 172.21.15.2:0/1225400756 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398023430 con 0x55dc3f12a820 2024-03-15T13:09:04.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f33b9e20740 1 -- 172.21.15.2:0/1225400756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3398005020 con 0x55dc3f12a820 2024-03-15T13:09:04.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f338ffff640 1 -- 172.21.15.2:0/1225400756 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f3398000e40 con 0x55dc3f12a820 2024-03-15T13:09:04.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.024+0000 7f338ffff640 1 -- 172.21.15.2:0/1225400756 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3398023b70 con 0x55dc3f12a820 2024-03-15T13:09:04.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.025+0000 7f33b9e20740 10 client.18597 fetch_fsmap learned FSMap version 50 2024-03-15T13:09:04.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.025+0000 7f33b9e20740 1 -- 172.21.15.2:0/1225400756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55dc3f0906a0 con 0x55dc3f12a820 2024-03-15T13:09:04.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.025+0000 7f338ffff640 1 -- 172.21.15.2:0/1225400756 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3398005020 con 0x55dc3f12a820 2024-03-15T13:09:04.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.025+0000 7f338ffff640 1 -- 172.21.15.2:0/1225400756 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3398019430 con 0x55dc3f12a820 2024-03-15T13:09:04.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.025+0000 7f33b9e20740 10 client.18597 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:09:04.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.025+0000 7f33b9e20740 10 client.18597 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-15T13:09:04.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.025+0000 7f33b9e20740 20 client.18597 populate_metadata read hostname 'smithi002' 2024-03-15T13:09:04.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.025+0000 7f33b9e20740 1 --2- 172.21.15.2:0/1225400756 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55dc3f1a4270 0x55dc3f1a6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.025+0000 7f33b9e20740 4 client.18597 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:09:04.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.025+0000 7f33b9e20740 1 -- 172.21.15.2:0/1225400756 --> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dc3ee1d890 con 0x55dc3f1a4270 2024-03-15T13:09:04.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.025+0000 7f33b4f0e640 1 --2- 172.21.15.2:0/1225400756 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55dc3f1a4270 0x55dc3f1a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.025+0000 7f33b4f0e640 1 --2- 172.21.15.2:0/1225400756 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55dc3f1a4270 0x55dc3f1a6690 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd3a6e31594aa239 server_cookie=7b276eff47349aed in_seq=0 out_seq=0 2024-03-15T13:09:04.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.025+0000 7f338ffff640 10 client.18597 ms_handle_connect on v2:172.21.15.184:6834/1255256953 2024-03-15T13:09:04.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f338ffff640 1 -- 172.21.15.2:0/1225400756 <== mds.3 v2:172.21.15.184:6834/1255256953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dc3ee1d890 con 0x55dc3f1a4270 2024-03-15T13:09:04.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f338ffff640 10 client.18597 handle_command_reply: tid=0 2024-03-15T13:09:04.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b9e20740 1 client.18597 shutdown 2024-03-15T13:09:04.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b9e20740 1 -- 172.21.15.2:0/1225400756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 msgr2=0x55dc3f13bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b9e20740 1 --2- 172.21.15.2:0/1225400756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f13bf70 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x55dc3f132f90 tx=0x7f3398009f90 comp rx=0 tx=0).stop 2024-03-15T13:09:04.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b9e20740 1 -- 172.21.15.2:0/1225400756 shutdown_connections 2024-03-15T13:09:04.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b9e20740 1 --2- 172.21.15.2:0/1225400756 >> [v2:172.21.15.184:6834/1255256953,v1:172.21.15.184:6835/1255256953] conn(0x55dc3f1a4270 0x55dc3f1a6690 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b9e20740 1 --2- 172.21.15.2:0/1225400756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dc3f137b60 0x55dc3f19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b9e20740 1 --2- 172.21.15.2:0/1225400756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc3f12a820 0x55dc3f13bf70 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f338ffff640 0 client.18597 ms_handle_reset on v2:172.21.15.184:6834/1255256953 2024-03-15T13:09:04.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b9e20740 1 --2- 172.21.15.2:0/1225400756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3f129f00 0x55dc3f13ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b9e20740 1 -- 172.21.15.2:0/1225400756 >> 172.21.15.2:0/1225400756 conn(0x55dc3f124980 msgr2=0x55dc3f134040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b9e20740 1 -- 172.21.15.2:0/1225400756 shutdown_connections 2024-03-15T13:09:04.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b9e20740 1 -- 172.21.15.2:0/1225400756 wait complete. 2024-03-15T13:09:04.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b9e20740 20 client.18597 trim_cache size 0 max 16384 2024-03-15T13:09:04.016 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:09:04.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b7172640 1 -- 172.21.15.2:0/3434342388 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f339c0a1ba0 msgr2=0x7f339c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b7172640 1 --2- 172.21.15.2:0/3434342388 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f339c0a1ba0 0x7f339c0a4060 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f33a8005cc0 tx=0x7f33a8009820 comp rx=0 tx=0).stop 2024-03-15T13:09:04.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b7172640 1 -- 172.21.15.2:0/3434342388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b01005d0 msgr2=0x7f33b019d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.026+0000 7f33b7172640 1 --2- 172.21.15.2:0/3434342388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b01005d0 0x7f33b019d740 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f33a000e9f0 tx=0x7f33a000eeb0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.027+0000 7f33b7172640 1 -- 172.21.15.2:0/3434342388 shutdown_connections 2024-03-15T13:09:04.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.027+0000 7f33b7172640 1 --2- 172.21.15.2:0/3434342388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33b01978f0 0x7f33b0197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.027+0000 7f33b7172640 1 --2- 172.21.15.2:0/3434342388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b01056a0 0x7f33b019dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.027+0000 7f33b7172640 1 --2- 172.21.15.2:0/3434342388 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f339c0a1ba0 0x7f339c0a4060 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.027+0000 7f33b7172640 1 --2- 172.21.15.2:0/3434342388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b01005d0 0x7f33b019d740 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.027+0000 7f33b7172640 1 -- 172.21.15.2:0/3434342388 >> 172.21.15.2:0/3434342388 conn(0x7f33b00774f0 msgr2=0x7f33b00fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.027+0000 7f33b7172640 1 -- 172.21.15.2:0/3434342388 shutdown_connections 2024-03-15T13:09:04.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.027+0000 7f33b7172640 1 -- 172.21.15.2:0/3434342388 wait complete. 2024-03-15T13:09:04.022 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:09:04.022 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-15T13:09:04.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:03 smithi184 ceph-mon[35993]: pgmap v313: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-15T13:09:04.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.142+0000 7f3fb1402640 1 -- 172.21.15.2:0/1104101581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac1055c0 msgr2=0x7f3fac10d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.142+0000 7f3fb1402640 1 --2- 172.21.15.2:0/1104101581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac1055c0 0x7f3fac10d190 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009cd0 tx=0x7f3fa402c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.142+0000 7f3fb1402640 1 -- 172.21.15.2:0/1104101581 shutdown_connections 2024-03-15T13:09:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.142+0000 7f3fb1402640 1 --2- 172.21.15.2:0/1104101581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac1055c0 0x7f3fac10d190 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.142+0000 7f3fb1402640 1 --2- 172.21.15.2:0/1104101581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fac100ea0 0x7f3fac104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.142+0000 7f3fb1402640 1 --2- 172.21.15.2:0/1104101581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fac1004f0 0x7f3fac1008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.142+0000 7f3fb1402640 1 -- 172.21.15.2:0/1104101581 >> 172.21.15.2:0/1104101581 conn(0x7f3fac0fbd50 msgr2=0x7f3fac0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.142+0000 7f3fb1402640 1 -- 172.21.15.2:0/1104101581 shutdown_connections 2024-03-15T13:09:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.142+0000 7f3fb1402640 1 -- 172.21.15.2:0/1104101581 wait complete. 2024-03-15T13:09:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fb1402640 1 Processor -- start 2024-03-15T13:09:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fb1402640 1 -- start start 2024-03-15T13:09:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fb1402640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fac1004f0 0x7f3fac10da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fb1402640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fac1055c0 0x7f3fac10dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fb1402640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac10e4f0 0x7f3fac1a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fb1402640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac1a6190 con 0x7f3fac10e4f0 2024-03-15T13:09:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fb1402640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac1a6300 con 0x7f3fac1055c0 2024-03-15T13:09:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fb1402640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac1a64c0 con 0x7f3fac1004f0 2024-03-15T13:09:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fab7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac10e4f0 0x7f3fac1a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fab7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac10e4f0 0x7f3fac1a5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57990/0 (socket says 172.21.15.2:57990) 2024-03-15T13:09:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3faaffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fac1004f0 0x7f3fac10da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fab7fe640 1 -- 172.21.15.2:0/625985812 learned_addr learned my addr 172.21.15.2:0/625985812 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fa3fff640 1 --2- 172.21.15.2:0/625985812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fac1055c0 0x7f3fac10dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fab7fe640 1 -- 172.21.15.2:0/625985812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fac1004f0 msgr2=0x7f3fac10da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fab7fe640 1 --2- 172.21.15.2:0/625985812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fac1004f0 0x7f3fac10da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fab7fe640 1 -- 172.21.15.2:0/625985812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fac1055c0 msgr2=0x7f3fac10dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fab7fe640 1 --2- 172.21.15.2:0/625985812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fac1055c0 0x7f3fac10dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.143+0000 7f3fab7fe640 1 -- 172.21.15.2:0/625985812 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9c009490 con 0x7f3fac10e4f0 2024-03-15T13:09:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.144+0000 7f3fab7fe640 1 --2- 172.21.15.2:0/625985812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac10e4f0 0x7f3fac1a5c20 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4005cc0 tx=0x7f3fa4034a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.144+0000 7f3fa8ff9640 1 -- 172.21.15.2:0/625985812 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4032880 con 0x7f3fac10e4f0 2024-03-15T13:09:04.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.144+0000 7f3fa8ff9640 1 -- 172.21.15.2:0/625985812 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3fa4034e80 con 0x7f3fac10e4f0 2024-03-15T13:09:04.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.144+0000 7f3fb1402640 1 -- 172.21.15.2:0/625985812 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa4009900 con 0x7f3fac10e4f0 2024-03-15T13:09:04.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.144+0000 7f3fa8ff9640 1 -- 172.21.15.2:0/625985812 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40047c0 con 0x7f3fac10e4f0 2024-03-15T13:09:04.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.144+0000 7f3fb1402640 1 -- 172.21.15.2:0/625985812 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac1a6ab0 con 0x7f3fac10e4f0 2024-03-15T13:09:04.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.146+0000 7f3fa8ff9640 1 -- 172.21.15.2:0/625985812 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f3fa400a040 con 0x7f3fac10e4f0 2024-03-15T13:09:04.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.146+0000 7f3fb3fdd740 1 Processor -- start 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.146+0000 7f3fb3fdd740 1 -- start start 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fb3fdd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439d440 0x5584a439d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fa8ff9640 1 --2- 172.21.15.2:0/625985812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3f980a1c20 0x7f3f980a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fb3fdd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a439dd60 0x5584a43a6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fb3fdd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a43a6940 0x5584a43a8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fb3fdd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a4358960 con 0x5584a439dd60 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fb3fdd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a430c310 con 0x5584a43a6940 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fb3fdd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a4309840 con 0x5584a439d440 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3faaffd640 1 --2- 172.21.15.2:0/625985812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3f980a1c20 0x7f3f980a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fab7fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a43a6940 0x5584a43a8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fa3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439d440 0x5584a439d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fa3fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439d440 0x5584a439d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.2:58946/0 (socket says 172.21.15.2:58946) 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fa3fff640 1 -- 172.21.15.2:0/1213358535 learned_addr learned my addr 172.21.15.2:0/1213358535 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fa3fff640 1 --2- 172.21.15.2:0/1213358535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a439dd60 0x5584a43a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fa8ff9640 1 -- 172.21.15.2:0/625985812 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f3fa40f0670 con 0x7f3fac10e4f0 2024-03-15T13:09:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fa3fff640 1 -- 172.21.15.2:0/1213358535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a43a6940 msgr2=0x5584a43a8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fa3fff640 1 --2- 172.21.15.2:0/1213358535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a43a6940 0x5584a43a8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fa3fff640 1 -- 172.21.15.2:0/1213358535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a439dd60 msgr2=0x5584a43a6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fa3fff640 1 --2- 172.21.15.2:0/1213358535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a439dd60 0x5584a43a6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fa3fff640 1 -- 172.21.15.2:0/1213358535 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584a42c7060 con 0x5584a439d440 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fab7fe640 1 --2- 172.21.15.2:0/1213358535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a43a6940 0x5584a43a8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3faaffd640 1 --2- 172.21.15.2:0/625985812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3f980a1c20 0x7f3f980a40e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009d60 tx=0x7f3f9c005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.147+0000 7f3fa3fff640 1 --2- 172.21.15.2:0/1213358535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439d440 0x5584a439d820 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009fd0 tx=0x7f3f940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f631e468b25d9d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.148+0000 7f3fa17fa640 1 -- 172.21.15.2:0/1213358535 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94004050 con 0x5584a439d440 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.148+0000 7f3fa17fa640 1 -- 172.21.15.2:0/1213358535 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3f9400f7c0 con 0x5584a439d440 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.148+0000 7f3fa17fa640 1 -- 172.21.15.2:0/1213358535 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94003980 con 0x5584a439d440 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.148+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/1213358535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439d440 msgr2=0x5584a439d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.148+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/1213358535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439d440 0x5584a439d820 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009fd0 tx=0x7f3f940046c0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.148+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/1213358535 shutdown_connections 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.148+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/1213358535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a43a6940 0x5584a43a8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.148+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/1213358535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a439dd60 0x5584a43a6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.148+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/1213358535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439d440 0x5584a439d820 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.148+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/1213358535 >> 172.21.15.2:0/1213358535 conn(0x5584a4397b70 msgr2=0x5584a4397f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.148+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/1213358535 shutdown_connections 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.148+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/1213358535 wait complete. 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.149+0000 7f3fb3fdd740 1 -- start start 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.149+0000 7f3fb3fdd740 1 Processor -- start 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.149+0000 7f3fb3fdd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a439d440 0x5584a43a6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.149+0000 7f3fb3fdd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a439dd60 0x5584a43a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.149+0000 7f3fb3fdd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a43a6940 0x5584a43a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.149+0000 7f3fb3fdd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a4305020 con 0x5584a439dd60 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.149+0000 7f3fb3fdd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a3fd8430 con 0x5584a439d440 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.149+0000 7f3fb3fdd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a3f94e00 con 0x5584a43a6940 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.149+0000 7f3fb3fdd740 10 client.0 _ll_register_callbacks cb 0x5584a42accb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.149+0000 7f3fa3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a439dd60 0x5584a43a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.149+0000 7f3faaffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a439d440 0x5584a43a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.149+0000 7f3fa3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a439dd60 0x5584a43a36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57998/0 (socket says 172.21.15.2:57998) 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.149+0000 7f3fa3fff640 1 -- 172.21.15.2:0/3472249916 learned_addr learned my addr 172.21.15.2:0/3472249916 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.149+0000 7f3fab7fe640 1 --2- 172.21.15.2:0/3472249916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a43a6940 0x5584a43a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.149+0000 7f3fa3fff640 1 -- 172.21.15.2:0/3472249916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a43a6940 msgr2=0x5584a43a3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3fa3fff640 1 --2- 172.21.15.2:0/3472249916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a43a6940 0x5584a43a3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3fa3fff640 1 -- 172.21.15.2:0/3472249916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a439d440 msgr2=0x5584a43a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3fa3fff640 1 --2- 172.21.15.2:0/3472249916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a439d440 0x5584a43a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3fa3fff640 1 -- 172.21.15.2:0/3472249916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f94019040 con 0x5584a439dd60 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3fa3fff640 1 --2- 172.21.15.2:0/3472249916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a439dd60 0x5584a43a36b0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009660 tx=0x7f3f9400fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d8980efe6bf54b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3f927fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3f927fc640 1 -- 172.21.15.2:0/3472249916 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94011260 con 0x5584a439dd60 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3f927fc640 1 -- 172.21.15.2:0/3472249916 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f3f94005040 con 0x5584a439dd60 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/3472249916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584a40047e0 con 0x5584a439dd60 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3f927fc640 1 -- 172.21.15.2:0/3472249916 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f94005200 con 0x5584a439dd60 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3fb3fdd740 10 client.18609 fetch_fsmap learned FSMap version 50 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/3472249916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5584a43036a0 con 0x5584a439dd60 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3f927fc640 1 -- 172.21.15.2:0/3472249916 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x5584a40047e0 con 0x5584a439dd60 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3f927fc640 1 -- 172.21.15.2:0/3472249916 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3f94010430 con 0x5584a439dd60 2024-03-15T13:09:04.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3fb3fdd740 10 client.18609 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3fb3fdd740 10 client.18609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3fb3fdd740 20 client.18609 populate_metadata read hostname 'smithi002' 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/3472249916 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5584a43b6980 0x5584a43b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3fb3fdd740 4 client.18609 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.150+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/3472249916 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584a4041720 con 0x5584a43b6980 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.151+0000 7f3faaffd640 1 --2- 172.21.15.2:0/3472249916 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5584a43b6980 0x5584a43b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.151+0000 7f3faaffd640 1 --2- 172.21.15.2:0/3472249916 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5584a43b6980 0x5584a43b8da0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81aa49f59f145203 server_cookie=9fe122066b82acc in_seq=0 out_seq=0 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.151+0000 7f3f927fc640 10 client.18609 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.152+0000 7f3f927fc640 1 -- 172.21.15.2:0/3472249916 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5584a4041720 con 0x5584a43b6980 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.152+0000 7f3f927fc640 10 client.18609 handle_command_reply: tid=0 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.152+0000 7f3fb3fdd740 1 client.18609 shutdown 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.152+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/3472249916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a439dd60 msgr2=0x5584a43a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.152+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/3472249916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a439dd60 0x5584a43a36b0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009660 tx=0x7f3f9400fe00 comp rx=0 tx=0).stop 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.152+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/3472249916 shutdown_connections 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.152+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/3472249916 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5584a43b6980 0x5584a43b8da0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.152+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/3472249916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a43a6940 0x5584a43a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.152+0000 7f3f927fc640 0 client.18609 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:09:04.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.152+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/3472249916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a439dd60 0x5584a43a36b0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.152+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/3472249916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a439d440 0x5584a43a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.152+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/3472249916 >> 172.21.15.2:0/3472249916 conn(0x5584a4397b70 msgr2=0x5584a43998c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.152+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/3472249916 shutdown_connections 2024-03-15T13:09:04.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.152+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/3472249916 wait complete. 2024-03-15T13:09:04.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.152+0000 7f3fb3fdd740 20 client.18609 trim_cache size 0 max 16384 2024-03-15T13:09:04.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.165+0000 7f3fb3fdd740 1 Processor -- start 2024-03-15T13:09:04.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.165+0000 7f3fb3fdd740 1 -- start start 2024-03-15T13:09:04.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.165+0000 7f3fb3fdd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439d440 0x5584a43a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.165+0000 7f3fb3fdd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a439dd60 0x5584a43a6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.165+0000 7f3fb3fdd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a43a6940 0x5584a43b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.165+0000 7f3fb3fdd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a43b2fa0 con 0x5584a43a6940 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.165+0000 7f3fb3fdd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a428a130 con 0x5584a439dd60 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.165+0000 7f3fb3fdd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a43a9580 con 0x5584a439d440 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.166+0000 7f3fab7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a43a6940 0x5584a43b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.166+0000 7f3fab7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a43a6940 0x5584a43b1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58006/0 (socket says 172.21.15.2:58006) 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.166+0000 7f3faaffd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439d440 0x5584a43a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.166+0000 7f3fab7fe640 1 -- 172.21.15.2:0/52150060 learned_addr learned my addr 172.21.15.2:0/52150060 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.166+0000 7f3fa3fff640 1 --2- 172.21.15.2:0/52150060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a439dd60 0x5584a43a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.166+0000 7f3fab7fe640 1 -- 172.21.15.2:0/52150060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439d440 msgr2=0x5584a43a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.166+0000 7f3fab7fe640 1 --2- 172.21.15.2:0/52150060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439d440 0x5584a43a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.166+0000 7f3fab7fe640 1 -- 172.21.15.2:0/52150060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a439dd60 msgr2=0x5584a43a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.166+0000 7f3fab7fe640 1 --2- 172.21.15.2:0/52150060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a439dd60 0x5584a43a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.166+0000 7f3fab7fe640 1 -- 172.21.15.2:0/52150060 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584a429fc20 con 0x5584a43a6940 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.166+0000 7f3fab7fe640 1 --2- 172.21.15.2:0/52150060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a43a6940 0x5584a43b1cc0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4033000 tx=0x7f3fa40b9310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff62ba2157fe8a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.166+0000 7f3fa0ff9640 1 -- 172.21.15.2:0/52150060 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40bd6b0 con 0x5584a43a6940 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.166+0000 7f3fa0ff9640 1 -- 172.21.15.2:0/52150060 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3fa40c6610 con 0x5584a43a6940 2024-03-15T13:09:04.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.166+0000 7f3fa0ff9640 1 -- 172.21.15.2:0/52150060 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40c57e0 con 0x5584a43a6940 2024-03-15T13:09:04.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.167+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/52150060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a43a6940 msgr2=0x5584a43b1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.167+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/52150060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a43a6940 0x5584a43b1cc0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4033000 tx=0x7f3fa40b9310 comp rx=0 tx=0).stop 2024-03-15T13:09:04.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.167+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/52150060 shutdown_connections 2024-03-15T13:09:04.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.167+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/52150060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a43a6940 0x5584a43b1cc0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.167+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/52150060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a439dd60 0x5584a43a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.167+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/52150060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439d440 0x5584a43a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.167+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/52150060 >> 172.21.15.2:0/52150060 conn(0x5584a4397b70 msgr2=0x5584a4396cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.167+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/52150060 shutdown_connections 2024-03-15T13:09:04.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.167+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/52150060 wait complete. 2024-03-15T13:09:04.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.167+0000 7f3fb3fdd740 1 -- start start 2024-03-15T13:09:04.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.167+0000 7f3fb3fdd740 1 Processor -- start 2024-03-15T13:09:04.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.167+0000 7f3fb3fdd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a439d440 0x5584a43ade90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.167+0000 7f3fb3fdd740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439dd60 0x5584a43ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fb3fdd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a43a6940 0x5584a43aa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fb3fdd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a43aaa00 con 0x5584a43a6940 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fb3fdd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a43aab70 con 0x5584a439d440 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fb3fdd740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a43aad30 con 0x5584a439dd60 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fb3fdd740 10 client.0 _ll_register_callbacks cb 0x7f3f94009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fab7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a43a6940 0x5584a43aa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fab7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a43a6940 0x5584a43aa330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58014/0 (socket says 172.21.15.2:58014) 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fab7fe640 1 -- 172.21.15.2:0/714547474 learned_addr learned my addr 172.21.15.2:0/714547474 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3faaffd640 1 --2- 172.21.15.2:0/714547474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a439d440 0x5584a43ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fa3fff640 1 --2- 172.21.15.2:0/714547474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439dd60 0x5584a43ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fab7fe640 1 -- 172.21.15.2:0/714547474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439dd60 msgr2=0x5584a43ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fab7fe640 1 --2- 172.21.15.2:0/714547474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439dd60 0x5584a43ae3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fab7fe640 1 -- 172.21.15.2:0/714547474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a439d440 msgr2=0x5584a43ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fab7fe640 1 --2- 172.21.15.2:0/714547474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a439d440 0x5584a43ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fab7fe640 1 -- 172.21.15.2:0/714547474 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa4009900 con 0x5584a43a6940 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fab7fe640 1 --2- 172.21.15.2:0/714547474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a43a6940 0x5584a43aa330 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40b9410 tx=0x7f3fa40b8c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7158b2aa27125fa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3f927fc640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:09:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3f927fc640 1 -- 172.21.15.2:0/714547474 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40ec070 con 0x5584a43a6940 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3f927fc640 1 -- 172.21.15.2:0/714547474 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f3fa40bd940 con 0x5584a43a6940 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/714547474 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f9400f7c0 con 0x5584a43a6940 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3f927fc640 1 -- 172.21.15.2:0/714547474 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fa40ca1d0 con 0x5584a43a6940 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fb3fdd740 10 client.18615 fetch_fsmap learned FSMap version 50 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/714547474 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5584a42c58c0 con 0x5584a43a6940 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.168+0000 7f3f927fc640 1 -- 172.21.15.2:0/714547474 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3f9400f7c0 con 0x5584a43a6940 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.169+0000 7f3f927fc640 1 -- 172.21.15.2:0/714547474 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f3fa40edc40 con 0x5584a43a6940 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.169+0000 7f3fb3fdd740 10 client.18615 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.169+0000 7f3fb3fdd740 10 client.18615 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.169+0000 7f3fb3fdd740 20 client.18615 populate_metadata read hostname 'smithi002' 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.169+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/714547474 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5584a4412f00 0x5584a44152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.169+0000 7f3fb3fdd740 4 client.18615 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.169+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/714547474 --> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5584a4090890 con 0x5584a4412f00 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.169+0000 7f3faaffd640 1 --2- 172.21.15.2:0/714547474 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5584a4412f00 0x5584a44152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.169+0000 7f3faaffd640 1 --2- 172.21.15.2:0/714547474 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5584a4412f00 0x5584a44152f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6895d60467d5de53 server_cookie=c9af6e479227d689 in_seq=0 out_seq=0 2024-03-15T13:09:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.169+0000 7f3f927fc640 10 client.18615 ms_handle_connect on v2:172.21.15.149:6834/694548319 2024-03-15T13:09:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.169+0000 7f3f927fc640 1 -- 172.21.15.2:0/714547474 <== mds.2 v2:172.21.15.149:6834/694548319 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5584a4090890 con 0x5584a4412f00 2024-03-15T13:09:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.169+0000 7f3f927fc640 10 client.18615 handle_command_reply: tid=0 2024-03-15T13:09:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.169+0000 7f3fb3fdd740 1 client.18615 shutdown 2024-03-15T13:09:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/714547474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a43a6940 msgr2=0x5584a43aa330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/714547474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a43a6940 0x5584a43aa330 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40b9410 tx=0x7f3fa40b8c50 comp rx=0 tx=0).stop 2024-03-15T13:09:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/714547474 shutdown_connections 2024-03-15T13:09:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/714547474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5584a43a6940 0x5584a43aa330 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/714547474 >> [v2:172.21.15.149:6834/694548319,v1:172.21.15.149:6835/694548319] conn(0x5584a4412f00 0x5584a44152f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/714547474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584a439dd60 0x5584a43ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3f927fc640 0 client.18615 ms_handle_reset on v2:172.21.15.149:6834/694548319 2024-03-15T13:09:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb3fdd740 1 --2- 172.21.15.2:0/714547474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584a439d440 0x5584a43ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/714547474 >> 172.21.15.2:0/714547474 conn(0x5584a4397b70 msgr2=0x5584a439e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/714547474 shutdown_connections 2024-03-15T13:09:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb3fdd740 1 -- 172.21.15.2:0/714547474 wait complete. 2024-03-15T13:09:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb3fdd740 20 client.18615 trim_cache size 0 max 16384 2024-03-15T13:09:04.160 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:09:04.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb1402640 1 -- 172.21.15.2:0/625985812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3f980a1c20 msgr2=0x7f3f980a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb1402640 1 --2- 172.21.15.2:0/625985812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3f980a1c20 0x7f3f980a40e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009d60 tx=0x7f3f9c005f90 comp rx=0 tx=0).stop 2024-03-15T13:09:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb1402640 1 -- 172.21.15.2:0/625985812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac10e4f0 msgr2=0x7f3fac1a5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb1402640 1 --2- 172.21.15.2:0/625985812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac10e4f0 0x7f3fac1a5c20 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4005cc0 tx=0x7f3fa4034a80 comp rx=0 tx=0).stop 2024-03-15T13:09:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb1402640 1 -- 172.21.15.2:0/625985812 shutdown_connections 2024-03-15T13:09:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb1402640 1 --2- 172.21.15.2:0/625985812 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3f980a1c20 0x7f3f980a40e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb1402640 1 --2- 172.21.15.2:0/625985812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac10e4f0 0x7f3fac1a5c20 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb1402640 1 --2- 172.21.15.2:0/625985812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fac1055c0 0x7f3fac10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb1402640 1 --2- 172.21.15.2:0/625985812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fac1004f0 0x7f3fac10da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb1402640 1 -- 172.21.15.2:0/625985812 >> 172.21.15.2:0/625985812 conn(0x7f3fac0fbd50 msgr2=0x7f3fac0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb1402640 1 -- 172.21.15.2:0/625985812 shutdown_connections 2024-03-15T13:09:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.170+0000 7f3fb1402640 1 -- 172.21.15.2:0/625985812 wait complete. 2024-03-15T13:09:04.165 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:09:04.165 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-15T13:09:04.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.285+0000 7efed4809640 1 -- 172.21.15.2:0/1075859823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efed01056a0 msgr2=0x7efed010d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.285+0000 7efed4809640 1 --2- 172.21.15.2:0/1075859823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efed01056a0 0x7efed010d270 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7efec0009cd0 tx=0x7efec002c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.286+0000 7efed4809640 1 -- 172.21.15.2:0/1075859823 shutdown_connections 2024-03-15T13:09:04.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.286+0000 7efed4809640 1 --2- 172.21.15.2:0/1075859823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efed01056a0 0x7efed010d270 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.286+0000 7efed4809640 1 --2- 172.21.15.2:0/1075859823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efed0100f80 0x7efed0104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.286+0000 7efed4809640 1 --2- 172.21.15.2:0/1075859823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efed01005d0 0x7efed01009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.286+0000 7efed4809640 1 -- 172.21.15.2:0/1075859823 >> 172.21.15.2:0/1075859823 conn(0x7efed00774f0 msgr2=0x7efed0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.286+0000 7efed4809640 1 -- 172.21.15.2:0/1075859823 shutdown_connections 2024-03-15T13:09:04.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.286+0000 7efed4809640 1 -- 172.21.15.2:0/1075859823 wait complete. 2024-03-15T13:09:04.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.286+0000 7efed4809640 1 Processor -- start 2024-03-15T13:09:04.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.286+0000 7efed4809640 1 -- start start 2024-03-15T13:09:04.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efed4809640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efed01005d0 0x7efed01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efed4809640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efed01056a0 0x7efed0198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efed4809640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efed0199410 0x7efed01a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efed4809640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed01a4060 con 0x7efed01005d0 2024-03-15T13:09:04.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efed4809640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed01a41d0 con 0x7efed01056a0 2024-03-15T13:09:04.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efed4809640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed01a4390 con 0x7efed0199410 2024-03-15T13:09:04.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efece59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efed01005d0 0x7efed01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efece59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efed01005d0 0x7efed01988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58030/0 (socket says 172.21.15.2:58030) 2024-03-15T13:09:04.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efece59c640 1 -- 172.21.15.2:0/2945783286 learned_addr learned my addr 172.21.15.2:0/2945783286 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efecdd9b640 1 --2- 172.21.15.2:0/2945783286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efed01056a0 0x7efed0198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efeced9d640 1 --2- 172.21.15.2:0/2945783286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efed0199410 0x7efed01a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efece59c640 1 -- 172.21.15.2:0/2945783286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efed0199410 msgr2=0x7efed01a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efece59c640 1 --2- 172.21.15.2:0/2945783286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efed0199410 0x7efed01a3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efece59c640 1 -- 172.21.15.2:0/2945783286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efed01056a0 msgr2=0x7efed0198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efece59c640 1 --2- 172.21.15.2:0/2945783286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efed01056a0 0x7efed0198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efece59c640 1 -- 172.21.15.2:0/2945783286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec0009900 con 0x7efed01005d0 2024-03-15T13:09:04.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efecdd9b640 1 --2- 172.21.15.2:0/2945783286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efed01056a0 0x7efed0198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:04.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efece59c640 1 --2- 172.21.15.2:0/2945783286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efed01005d0 0x7efed01988f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7efec800e9f0 tx=0x7efec800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efec77fe640 1 -- 172.21.15.2:0/2945783286 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec8009700 con 0x7efed01005d0 2024-03-15T13:09:04.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efed4809640 1 -- 172.21.15.2:0/2945783286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed01a4670 con 0x7efed01005d0 2024-03-15T13:09:04.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efec77fe640 1 -- 172.21.15.2:0/2945783286 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efec800cd80 con 0x7efed01005d0 2024-03-15T13:09:04.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.287+0000 7efec77fe640 1 -- 172.21.15.2:0/2945783286 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec8018920 con 0x7efed01005d0 2024-03-15T13:09:04.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.288+0000 7efed4809640 1 -- 172.21.15.2:0/2945783286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed01a4b80 con 0x7efed01005d0 2024-03-15T13:09:04.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.290+0000 7efec77fe640 1 -- 172.21.15.2:0/2945783286 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7efec8018a80 con 0x7efed01005d0 2024-03-15T13:09:04.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.290+0000 7efed73e1740 1 Processor -- start 2024-03-15T13:09:04.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.290+0000 7efed73e1740 1 -- start start 2024-03-15T13:09:04.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.290+0000 7efed73e1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23daf250 0x55ab23daf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.290+0000 7efed73e1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23dafb70 0x55ab23db8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.290+0000 7efed73e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23dbab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.290+0000 7efed73e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab23d6a960 con 0x55ab23db8750 2024-03-15T13:09:04.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.290+0000 7efed73e1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab23d1e310 con 0x55ab23daf250 2024-03-15T13:09:04.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.290+0000 7efed73e1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab23d1b840 con 0x55ab23dafb70 2024-03-15T13:09:04.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.290+0000 7efec77fe640 1 --2- 172.21.15.2:0/2945783286 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efebc0a1ba0 0x7efebc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.290+0000 7efeced9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23daf250 0x55ab23daf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.290+0000 7efecdd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23dbab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.290+0000 7efeced9d640 1 --2- 172.21.15.2:0/2945783286 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efebc0a1ba0 0x7efebc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.290+0000 7efecdd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23dbab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58038/0 (socket says 172.21.15.2:58038) 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.290+0000 7efecdd9b640 1 -- 172.21.15.2:0/2182001698 learned_addr learned my addr 172.21.15.2:0/2182001698 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.291+0000 7efece59c640 1 --2- 172.21.15.2:0/2182001698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23dafb70 0x55ab23db8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.291+0000 7efec77fe640 1 -- 172.21.15.2:0/2945783286 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7efec8014070 con 0x7efed01005d0 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.291+0000 7efecdd9b640 1 -- 172.21.15.2:0/2182001698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23dafb70 msgr2=0x55ab23db8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.291+0000 7efecdd9b640 1 --2- 172.21.15.2:0/2182001698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23dafb70 0x55ab23db8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.291+0000 7efecdd9b640 1 -- 172.21.15.2:0/2182001698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23daf250 msgr2=0x55ab23daf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.291+0000 7efecdd9b640 1 --2- 172.21.15.2:0/2182001698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23daf250 0x55ab23daf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.291+0000 7efecdd9b640 1 -- 172.21.15.2:0/2182001698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab23cd9060 con 0x55ab23db8750 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.291+0000 7efeced9d640 1 --2- 172.21.15.2:0/2182001698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23daf250 0x55ab23daf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.291+0000 7efecdd9b640 1 --2- 172.21.15.2:0/2182001698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23dbab40 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009fd0 tx=0x7efeb80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509712541a0da18b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.291+0000 7efeced9d640 1 --2- 172.21.15.2:0/2945783286 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efebc0a1ba0 0x7efebc0a4060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efec0005cc0 tx=0x7efec0005c50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.291+0000 7efec4ff9640 1 -- 172.21.15.2:0/2182001698 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb8005020 con 0x55ab23db8750 2024-03-15T13:09:04.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.291+0000 7efec4ff9640 1 -- 172.21.15.2:0/2182001698 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efeb8005650 con 0x55ab23db8750 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.291+0000 7efec4ff9640 1 -- 172.21.15.2:0/2182001698 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb800f7e0 con 0x55ab23db8750 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.292+0000 7efed73e1740 1 -- 172.21.15.2:0/2182001698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 msgr2=0x55ab23dbab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.292+0000 7efed73e1740 1 --2- 172.21.15.2:0/2182001698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23dbab40 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009fd0 tx=0x7efeb80118d0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.292+0000 7efed73e1740 1 -- 172.21.15.2:0/2182001698 shutdown_connections 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.292+0000 7efed73e1740 1 --2- 172.21.15.2:0/2182001698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23dbab40 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.292+0000 7efed73e1740 1 --2- 172.21.15.2:0/2182001698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23dafb70 0x55ab23db8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.292+0000 7efed73e1740 1 --2- 172.21.15.2:0/2182001698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23daf250 0x55ab23daf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.292+0000 7efed73e1740 1 -- 172.21.15.2:0/2182001698 >> 172.21.15.2:0/2182001698 conn(0x55ab23da9980 msgr2=0x55ab23da9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.292+0000 7efed73e1740 1 -- 172.21.15.2:0/2182001698 shutdown_connections 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.292+0000 7efed73e1740 1 -- 172.21.15.2:0/2182001698 wait complete. 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.292+0000 7efed73e1740 1 -- start start 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.292+0000 7efed73e1740 1 Processor -- start 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efed73e1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23daf250 0x55ab23db80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efed73e1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23dafb70 0x55ab23db5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efed73e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23db5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efed73e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab23d17020 con 0x55ab23db8750 2024-03-15T13:09:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efed73e1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab239ea430 con 0x55ab23dafb70 2024-03-15T13:09:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efed73e1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab239a6e00 con 0x55ab23daf250 2024-03-15T13:09:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efed73e1740 10 client.0 _ll_register_callbacks cb 0x55ab23cbecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:09:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efeced9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23db5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efeced9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23db5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58052/0 (socket says 172.21.15.2:58052) 2024-03-15T13:09:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efeced9d640 1 -- 172.21.15.2:0/1055158298 learned_addr learned my addr 172.21.15.2:0/1055158298 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efecdd9b640 1 --2- 172.21.15.2:0/1055158298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23dafb70 0x55ab23db5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efece59c640 1 --2- 172.21.15.2:0/1055158298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23daf250 0x55ab23db80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efeced9d640 1 -- 172.21.15.2:0/1055158298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23daf250 msgr2=0x55ab23db80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efeced9d640 1 --2- 172.21.15.2:0/1055158298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23daf250 0x55ab23db80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efeced9d640 1 -- 172.21.15.2:0/1055158298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23dafb70 msgr2=0x55ab23db5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efeced9d640 1 --2- 172.21.15.2:0/1055158298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23dafb70 0x55ab23db5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efeced9d640 1 -- 172.21.15.2:0/1055158298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb8009c00 con 0x55ab23db8750 2024-03-15T13:09:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efeced9d640 1 --2- 172.21.15.2:0/1055158298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23db5990 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7efec0036000 tx=0x7efec002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c4a38e348c9f7b server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efeb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efeb5ffb640 1 -- 172.21.15.2:0/1055158298 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0003970 con 0x55ab23db8750 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efeb5ffb640 1 -- 172.21.15.2:0/1055158298 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7efec0003ad0 con 0x55ab23db8750 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.293+0000 7efed73e1740 1 -- 172.21.15.2:0/1055158298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab23a167e0 con 0x55ab23db8750 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.294+0000 7efeb5ffb640 1 -- 172.21.15.2:0/1055158298 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7efec0003c90 con 0x55ab23db8750 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.294+0000 7efed73e1740 10 client.18627 fetch_fsmap learned FSMap version 50 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.294+0000 7efed73e1740 1 -- 172.21.15.2:0/1055158298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55ab23d156a0 con 0x55ab23db8750 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.294+0000 7efeb5ffb640 1 -- 172.21.15.2:0/1055158298 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x55ab23a167e0 con 0x55ab23db8750 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.294+0000 7efeb5ffb640 1 -- 172.21.15.2:0/1055158298 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efec004c070 con 0x55ab23db8750 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.294+0000 7efed73e1740 10 client.18627 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.294+0000 7efed73e1740 10 client.18627 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.294+0000 7efed73e1740 20 client.18627 populate_metadata read hostname 'smithi002' 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.294+0000 7efed73e1740 1 --2- 172.21.15.2:0/1055158298 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55ab23dc88c0 0x55ab23dcacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.294+0000 7efed73e1740 4 client.18627 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.294+0000 7efed73e1740 1 -- 172.21.15.2:0/1055158298 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab23a53720 con 0x55ab23dc88c0 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.294+0000 7efece59c640 1 --2- 172.21.15.2:0/1055158298 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55ab23dc88c0 0x55ab23dcacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efeb5ffb640 10 client.18627 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:09:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efece59c640 1 --2- 172.21.15.2:0/1055158298 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55ab23dc88c0 0x55ab23dcacb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=42487a0bbf687469 server_cookie=231a72782e3925fe in_seq=0 out_seq=0 2024-03-15T13:09:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efeb5ffb640 1 -- 172.21.15.2:0/1055158298 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7efec80a94b0 con 0x55ab23dc88c0 2024-03-15T13:09:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efeb5ffb640 10 client.18627 handle_command_reply: tid=0 2024-03-15T13:09:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efed73e1740 1 client.18627 shutdown 2024-03-15T13:09:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efed73e1740 1 -- 172.21.15.2:0/1055158298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 msgr2=0x55ab23db5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efed73e1740 1 --2- 172.21.15.2:0/1055158298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23db5990 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7efec0036000 tx=0x7efec002c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efed73e1740 1 -- 172.21.15.2:0/1055158298 shutdown_connections 2024-03-15T13:09:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efed73e1740 1 --2- 172.21.15.2:0/1055158298 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55ab23dc88c0 0x55ab23dcacb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efed73e1740 1 --2- 172.21.15.2:0/1055158298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23db5990 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efeb5ffb640 0 client.18627 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:09:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efed73e1740 1 --2- 172.21.15.2:0/1055158298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23dafb70 0x55ab23db5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efed73e1740 1 --2- 172.21.15.2:0/1055158298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23daf250 0x55ab23db80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efed73e1740 1 -- 172.21.15.2:0/1055158298 >> 172.21.15.2:0/1055158298 conn(0x55ab23da9980 msgr2=0x55ab23dab6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efed73e1740 1 -- 172.21.15.2:0/1055158298 shutdown_connections 2024-03-15T13:09:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efed73e1740 1 -- 172.21.15.2:0/1055158298 wait complete. 2024-03-15T13:09:04.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.295+0000 7efed73e1740 20 client.18627 trim_cache size 0 max 16384 2024-03-15T13:09:04.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efed73e1740 1 Processor -- start 2024-03-15T13:09:04.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efed73e1740 1 -- start start 2024-03-15T13:09:04.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efed73e1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23daf250 0x55ab23db5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efed73e1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23dafb70 0x55ab23db7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efed73e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23db80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efed73e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab23c9c130 con 0x55ab23db8750 2024-03-15T13:09:04.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efed73e1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab23daec90 con 0x55ab23dafb70 2024-03-15T13:09:04.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efed73e1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab23da55c0 con 0x55ab23daf250 2024-03-15T13:09:04.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efeced9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23db80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efeced9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23db80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58066/0 (socket says 172.21.15.2:58066) 2024-03-15T13:09:04.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efeced9d640 1 -- 172.21.15.2:0/2514702635 learned_addr learned my addr 172.21.15.2:0/2514702635 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efece59c640 1 --2- 172.21.15.2:0/2514702635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23daf250 0x55ab23db5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efecdd9b640 1 --2- 172.21.15.2:0/2514702635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23dafb70 0x55ab23db7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efeced9d640 1 -- 172.21.15.2:0/2514702635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23daf250 msgr2=0x55ab23db5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efeced9d640 1 --2- 172.21.15.2:0/2514702635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23daf250 0x55ab23db5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efeced9d640 1 -- 172.21.15.2:0/2514702635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23dafb70 msgr2=0x55ab23db7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efeced9d640 1 --2- 172.21.15.2:0/2514702635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23dafb70 0x55ab23db7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efeced9d640 1 -- 172.21.15.2:0/2514702635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab23cb1c20 con 0x55ab23db8750 2024-03-15T13:09:04.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.308+0000 7efeced9d640 1 --2- 172.21.15.2:0/2514702635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23db80a0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7efec0037040 tx=0x7efec0009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6366ea4fe5a817af server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.309+0000 7efeb7fff640 1 -- 172.21.15.2:0/2514702635 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0032490 con 0x55ab23db8750 2024-03-15T13:09:04.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.309+0000 7efeb7fff640 1 -- 172.21.15.2:0/2514702635 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efec0049040 con 0x55ab23db8750 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.309+0000 7efeb7fff640 1 -- 172.21.15.2:0/2514702635 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0003bd0 con 0x55ab23db8750 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.309+0000 7efed73e1740 1 -- 172.21.15.2:0/2514702635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 msgr2=0x55ab23db80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.309+0000 7efed73e1740 1 --2- 172.21.15.2:0/2514702635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23db80a0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7efec0037040 tx=0x7efec0009820 comp rx=0 tx=0).stop 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.309+0000 7efed73e1740 1 -- 172.21.15.2:0/2514702635 shutdown_connections 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.309+0000 7efed73e1740 1 --2- 172.21.15.2:0/2514702635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23db8750 0x55ab23db80a0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.309+0000 7efed73e1740 1 --2- 172.21.15.2:0/2514702635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23dafb70 0x55ab23db7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.309+0000 7efed73e1740 1 --2- 172.21.15.2:0/2514702635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23daf250 0x55ab23db5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.309+0000 7efed73e1740 1 -- 172.21.15.2:0/2514702635 >> 172.21.15.2:0/2514702635 conn(0x55ab23da9980 msgr2=0x55ab23da8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.309+0000 7efed73e1740 1 -- 172.21.15.2:0/2514702635 shutdown_connections 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.309+0000 7efed73e1740 1 -- 172.21.15.2:0/2514702635 wait complete. 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.309+0000 7efed73e1740 1 -- start start 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efed73e1740 1 Processor -- start 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efed73e1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23daf250 0x55ab23db22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efed73e1740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23dafb70 0x55ab23db2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efed73e1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23db8750 0x55ab23e23160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efed73e1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab23e236a0 con 0x55ab23daf250 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efed73e1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab23e237e0 con 0x55ab23db8750 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efed73e1740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab23e23950 con 0x55ab23dafb70 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efed73e1740 10 client.0 _ll_register_callbacks cb 0x7efec8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efece59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23daf250 0x55ab23db22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efece59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23daf250 0x55ab23db22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58068/0 (socket says 172.21.15.2:58068) 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efecdd9b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23dafb70 0x55ab23db2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efece59c640 1 -- 172.21.15.2:0/2550825798 learned_addr learned my addr 172.21.15.2:0/2550825798 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efeced9d640 1 --2- 172.21.15.2:0/2550825798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23db8750 0x55ab23e23160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efece59c640 1 -- 172.21.15.2:0/2550825798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23dafb70 msgr2=0x55ab23db2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efece59c640 1 --2- 172.21.15.2:0/2550825798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23dafb70 0x55ab23db2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efece59c640 1 -- 172.21.15.2:0/2550825798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23db8750 msgr2=0x55ab23e23160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efece59c640 1 --2- 172.21.15.2:0/2550825798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23db8750 0x55ab23e23160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efece59c640 1 -- 172.21.15.2:0/2550825798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab23d156a0 con 0x55ab23daf250 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efece59c640 1 --2- 172.21.15.2:0/2550825798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23daf250 0x55ab23db22d0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7efec800c6a0 tx=0x7efec801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be85aac7792e6d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efeb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:09:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.310+0000 7efeb5ffb640 1 -- 172.21.15.2:0/2550825798 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec80b2070 con 0x55ab23daf250 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.311+0000 7efeb5ffb640 1 -- 172.21.15.2:0/2550825798 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7efec80b3260 con 0x55ab23daf250 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.311+0000 7efed73e1740 1 -- 172.21.15.2:0/2550825798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efeb8005650 con 0x55ab23daf250 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.311+0000 7efeb5ffb640 1 -- 172.21.15.2:0/2550825798 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7efec8010be0 con 0x55ab23daf250 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.311+0000 7efed73e1740 10 client.18633 fetch_fsmap learned FSMap version 50 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.311+0000 7efed73e1740 1 -- 172.21.15.2:0/2550825798 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55ab23cd78c0 con 0x55ab23daf250 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.311+0000 7efeb5ffb640 1 -- 172.21.15.2:0/2550825798 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efeb8005650 con 0x55ab23daf250 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.311+0000 7efeb5ffb640 1 -- 172.21.15.2:0/2550825798 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7efec80a7020 con 0x55ab23daf250 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.311+0000 7efed73e1740 10 client.18633 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.311+0000 7efed73e1740 10 client.18633 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.311+0000 7efed73e1740 20 client.18633 populate_metadata read hostname 'smithi002' 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.311+0000 7efed73e1740 1 --2- 172.21.15.2:0/2550825798 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55ab23e250b0 0x55ab23e274d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.311+0000 7efed73e1740 4 client.18633 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.311+0000 7efed73e1740 1 -- 172.21.15.2:0/2550825798 --> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ab23aa2890 con 0x55ab23e250b0 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.311+0000 7efecdd9b640 1 --2- 172.21.15.2:0/2550825798 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55ab23e250b0 0x55ab23e274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efeb5ffb640 10 client.18633 ms_handle_connect on v2:172.21.15.149:6836/1813851428 2024-03-15T13:09:04.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efecdd9b640 1 --2- 172.21.15.2:0/2550825798 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55ab23e250b0 0x55ab23e274d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa64d82edbc9468f server_cookie=5559899ffd500665 in_seq=0 out_seq=0 2024-03-15T13:09:04.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efeb5ffb640 1 -- 172.21.15.2:0/2550825798 <== mds.1 v2:172.21.15.149:6836/1813851428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ab23aa2890 con 0x55ab23e250b0 2024-03-15T13:09:04.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efeb5ffb640 10 client.18633 handle_command_reply: tid=0 2024-03-15T13:09:04.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed73e1740 1 client.18633 shutdown 2024-03-15T13:09:04.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed73e1740 1 -- 172.21.15.2:0/2550825798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23daf250 msgr2=0x55ab23db22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed73e1740 1 --2- 172.21.15.2:0/2550825798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23daf250 0x55ab23db22d0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7efec800c6a0 tx=0x7efec801ccc0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed73e1740 1 -- 172.21.15.2:0/2550825798 shutdown_connections 2024-03-15T13:09:04.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed73e1740 1 --2- 172.21.15.2:0/2550825798 >> [v2:172.21.15.149:6836/1813851428,v1:172.21.15.149:6837/1813851428] conn(0x55ab23e250b0 0x55ab23e274d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed73e1740 1 --2- 172.21.15.2:0/2550825798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab23db8750 0x55ab23e23160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed73e1740 1 --2- 172.21.15.2:0/2550825798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab23dafb70 0x55ab23db2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed73e1740 1 --2- 172.21.15.2:0/2550825798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ab23daf250 0x55ab23db22d0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efeb5ffb640 0 client.18633 ms_handle_reset on v2:172.21.15.149:6836/1813851428 2024-03-15T13:09:04.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed73e1740 1 -- 172.21.15.2:0/2550825798 >> 172.21.15.2:0/2550825798 conn(0x55ab23da9980 msgr2=0x55ab23daff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed73e1740 1 -- 172.21.15.2:0/2550825798 shutdown_connections 2024-03-15T13:09:04.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed73e1740 1 -- 172.21.15.2:0/2550825798 wait complete. 2024-03-15T13:09:04.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed73e1740 20 client.18633 trim_cache size 0 max 16384 2024-03-15T13:09:04.303 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:09:04.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed4809640 1 -- 172.21.15.2:0/2945783286 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efebc0a1ba0 msgr2=0x7efebc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed4809640 1 --2- 172.21.15.2:0/2945783286 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efebc0a1ba0 0x7efebc0a4060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efec0005cc0 tx=0x7efec0005c50 comp rx=0 tx=0).stop 2024-03-15T13:09:04.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed4809640 1 -- 172.21.15.2:0/2945783286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efed01005d0 msgr2=0x7efed01988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.312+0000 7efed4809640 1 --2- 172.21.15.2:0/2945783286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efed01005d0 0x7efed01988f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7efec800e9f0 tx=0x7efec800eeb0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.313+0000 7efed4809640 1 -- 172.21.15.2:0/2945783286 shutdown_connections 2024-03-15T13:09:04.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.313+0000 7efed4809640 1 --2- 172.21.15.2:0/2945783286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efed0199410 0x7efed01a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.313+0000 7efed4809640 1 --2- 172.21.15.2:0/2945783286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efed01056a0 0x7efed0198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.313+0000 7efed4809640 1 --2- 172.21.15.2:0/2945783286 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efebc0a1ba0 0x7efebc0a4060 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.313+0000 7efed4809640 1 --2- 172.21.15.2:0/2945783286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efed01005d0 0x7efed01988f0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.313+0000 7efed4809640 1 -- 172.21.15.2:0/2945783286 >> 172.21.15.2:0/2945783286 conn(0x7efed00774f0 msgr2=0x7efed00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.313+0000 7efed4809640 1 -- 172.21.15.2:0/2945783286 shutdown_connections 2024-03-15T13:09:04.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.313+0000 7efed4809640 1 -- 172.21.15.2:0/2945783286 wait complete. 2024-03-15T13:09:04.307 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-15T13:09:04.307 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-15T13:09:04.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.427+0000 7f561ab2e640 1 -- 172.21.15.2:0/3480298938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56141005d0 msgr2=0x7f56141009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.427+0000 7f561ab2e640 1 --2- 172.21.15.2:0/3480298938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56141005d0 0x7f56141009b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f5608009930 tx=0x7f560802bc30 comp rx=0 tx=0).stop 2024-03-15T13:09:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.427+0000 7f561ab2e640 1 -- 172.21.15.2:0/3480298938 shutdown_connections 2024-03-15T13:09:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.427+0000 7f561ab2e640 1 --2- 172.21.15.2:0/3480298938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56141056a0 0x7f561410d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.427+0000 7f561ab2e640 1 --2- 172.21.15.2:0/3480298938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5614100f80 0x7f5614104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.427+0000 7f561ab2e640 1 --2- 172.21.15.2:0/3480298938 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56141005d0 0x7f56141009b0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.427+0000 7f561ab2e640 1 -- 172.21.15.2:0/3480298938 >> 172.21.15.2:0/3480298938 conn(0x7f56140774f0 msgr2=0x7f5614077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.427+0000 7f561ab2e640 1 -- 172.21.15.2:0/3480298938 shutdown_connections 2024-03-15T13:09:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.427+0000 7f561ab2e640 1 -- 172.21.15.2:0/3480298938 wait complete. 2024-03-15T13:09:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f561ab2e640 1 Processor -- start 2024-03-15T13:09:04.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f561ab2e640 1 -- start start 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f561ab2e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56141005d0 0x7f561419d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f561ab2e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56141056a0 0x7f561419dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f561ab2e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56141979c0 0x7f5614197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f561ab2e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56141983b0 con 0x7f56141056a0 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f561ab2e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5614198520 con 0x7f56141979c0 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f561ab2e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56141986e0 con 0x7f56141005d0 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f5613fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56141056a0 0x7f561419dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f5613fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56141056a0 0x7f561419dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58094/0 (socket says 172.21.15.2:58094) 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f5613fff640 1 -- 172.21.15.2:0/4227245753 learned_addr learned my addr 172.21.15.2:0/4227245753 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f56188ca640 1 --2- 172.21.15.2:0/4227245753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56141005d0 0x7f561419d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f56190cb640 1 --2- 172.21.15.2:0/4227245753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56141979c0 0x7f5614197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f5613fff640 1 -- 172.21.15.2:0/4227245753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56141005d0 msgr2=0x7f561419d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f5613fff640 1 --2- 172.21.15.2:0/4227245753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56141005d0 0x7f561419d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f5613fff640 1 -- 172.21.15.2:0/4227245753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56141979c0 msgr2=0x7f5614197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f5613fff640 1 --2- 172.21.15.2:0/4227245753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56141979c0 0x7f5614197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f5613fff640 1 -- 172.21.15.2:0/4227245753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5608009560 con 0x7f56141056a0 2024-03-15T13:09:04.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.428+0000 7f5613fff640 1 --2- 172.21.15.2:0/4227245753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56141056a0 0x7f561419dd50 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f560000e9f0 tx=0x7f560000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.429+0000 7f5611ffb640 1 -- 172.21.15.2:0/4227245753 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56000096c0 con 0x7f56141056a0 2024-03-15T13:09:04.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.429+0000 7f5611ffb640 1 -- 172.21.15.2:0/4227245753 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f56000054f0 con 0x7f56141056a0 2024-03-15T13:09:04.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.429+0000 7f561ab2e640 1 -- 172.21.15.2:0/4227245753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56141a46d0 con 0x7f56141056a0 2024-03-15T13:09:04.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.429+0000 7f5611ffb640 1 -- 172.21.15.2:0/4227245753 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600010430 con 0x7f56141056a0 2024-03-15T13:09:04.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.429+0000 7f561ab2e640 1 -- 172.21.15.2:0/4227245753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56141a4bb0 con 0x7f56141056a0 2024-03-15T13:09:04.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.431+0000 7f5611ffb640 1 -- 172.21.15.2:0/4227245753 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f5600005000 con 0x7f56141056a0 2024-03-15T13:09:04.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.431+0000 7f561d7e0740 1 Processor -- start 2024-03-15T13:09:04.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.431+0000 7f561d7e0740 1 -- start start 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.431+0000 7f5611ffb640 1 --2- 172.21.15.2:0/4227245753 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f56040a1c20 0x7f56040a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f561d7e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7adda250 0x564a7adda630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f561d7e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7addab70 0x564a7ade3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f561d7e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 0x564a7ade5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f5611ffb640 1 -- 172.21.15.2:0/4227245753 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f5600014070 con 0x7f56141056a0 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f56190cb640 1 --2- 172.21.15.2:0/4227245753 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f56040a1c20 0x7f56040a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f561d7e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ad95960 con 0x564a7adda250 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f561d7e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ad49310 con 0x564a7ade3750 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f561d7e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ad46840 con 0x564a7addab70 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f56188ca640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7adda250 0x564a7adda630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f5613fff640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7addab70 0x564a7ade3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f56188ca640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7adda250 0x564a7adda630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58108/0 (socket says 172.21.15.2:58108) 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f56188ca640 1 -- 172.21.15.2:0/3837228207 learned_addr learned my addr 172.21.15.2:0/3837228207 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f56190cb640 1 --2- 172.21.15.2:0/3837228207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 0x564a7ade5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f5613fff640 1 -- 172.21.15.2:0/3837228207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 msgr2=0x564a7ade5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f5613fff640 1 --2- 172.21.15.2:0/3837228207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 0x564a7ade5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f5613fff640 1 -- 172.21.15.2:0/3837228207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7adda250 msgr2=0x564a7adda630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f5613fff640 1 --2- 172.21.15.2:0/3837228207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7adda250 0x564a7adda630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f5613fff640 1 -- 172.21.15.2:0/3837228207 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a7ad04060 con 0x564a7addab70 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f56190cb640 1 --2- 172.21.15.2:0/4227245753 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f56040a1c20 0x7f56040a40e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f560c009fd0 tx=0x7f560c013b00 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f5613fff640 1 --2- 172.21.15.2:0/3837228207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7addab70 0x564a7ade3210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f56000cb240 tx=0x7f560009a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b8fb8f6634e5f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f55feffd640 1 -- 172.21.15.2:0/3837228207 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600083ea0 con 0x564a7addab70 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f55feffd640 1 -- 172.21.15.2:0/3837228207 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f56000b8e90 con 0x564a7addab70 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.432+0000 7f55feffd640 1 -- 172.21.15.2:0/3837228207 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600098a20 con 0x564a7addab70 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.433+0000 7f561d7e0740 1 -- 172.21.15.2:0/3837228207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7addab70 msgr2=0x564a7ade3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.433+0000 7f561d7e0740 1 --2- 172.21.15.2:0/3837228207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7addab70 0x564a7ade3210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f56000cb240 tx=0x7f560009a140 comp rx=0 tx=0).stop 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.433+0000 7f561d7e0740 1 -- 172.21.15.2:0/3837228207 shutdown_connections 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.433+0000 7f561d7e0740 1 --2- 172.21.15.2:0/3837228207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 0x564a7ade5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.433+0000 7f561d7e0740 1 --2- 172.21.15.2:0/3837228207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7addab70 0x564a7ade3210 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.433+0000 7f561d7e0740 1 --2- 172.21.15.2:0/3837228207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7adda250 0x564a7adda630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.433+0000 7f561d7e0740 1 -- 172.21.15.2:0/3837228207 >> 172.21.15.2:0/3837228207 conn(0x564a7add4980 msgr2=0x564a7add4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.433+0000 7f561d7e0740 1 -- 172.21.15.2:0/3837228207 shutdown_connections 2024-03-15T13:09:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.433+0000 7f561d7e0740 1 -- 172.21.15.2:0/3837228207 wait complete. 2024-03-15T13:09:04.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.433+0000 7f561d7e0740 1 -- start start 2024-03-15T13:09:04.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f561d7e0740 1 Processor -- start 2024-03-15T13:09:04.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f561d7e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7adda250 0x564a7aded430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f561d7e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7addab70 0x564a7aded970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f561d7e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7ade3750 0x564a7ade7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f561d7e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ad42020 con 0x564a7addab70 2024-03-15T13:09:04.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f561d7e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7aa15430 con 0x564a7adda250 2024-03-15T13:09:04.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f561d7e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7a9d1e00 con 0x564a7ade3750 2024-03-15T13:09:04.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f561d7e0740 10 client.0 _ll_register_callbacks cb 0x564a7ace9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:09:04.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f5613fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7addab70 0x564a7aded970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f5613fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7addab70 0x564a7aded970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58120/0 (socket says 172.21.15.2:58120) 2024-03-15T13:09:04.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f5613fff640 1 -- 172.21.15.2:0/2798175087 learned_addr learned my addr 172.21.15.2:0/2798175087 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f56188ca640 1 --2- 172.21.15.2:0/2798175087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7adda250 0x564a7aded430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f56190cb640 1 --2- 172.21.15.2:0/2798175087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7ade3750 0x564a7ade7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f5613fff640 1 -- 172.21.15.2:0/2798175087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7ade3750 msgr2=0x564a7ade7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f5613fff640 1 --2- 172.21.15.2:0/2798175087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7ade3750 0x564a7ade7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f5613fff640 1 -- 172.21.15.2:0/2798175087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7adda250 msgr2=0x564a7aded430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f5613fff640 1 --2- 172.21.15.2:0/2798175087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7adda250 0x564a7aded430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f5613fff640 1 -- 172.21.15.2:0/2798175087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56141a4bb0 con 0x564a7addab70 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f5613fff640 1 --2- 172.21.15.2:0/2798175087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7addab70 0x564a7aded970 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f560009a6b0 tx=0x7f560009a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb9953e2f07a3e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.434+0000 7f55f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.435+0000 7f55f3fff640 1 -- 172.21.15.2:0/2798175087 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600083ea0 con 0x564a7addab70 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.435+0000 7f55f3fff640 1 -- 172.21.15.2:0/2798175087 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f560009e870 con 0x564a7addab70 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.435+0000 7f561d7e0740 1 -- 172.21.15.2:0/2798175087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a7aa417e0 con 0x564a7addab70 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.435+0000 7f55f3fff640 1 -- 172.21.15.2:0/2798175087 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56000b3210 con 0x564a7addab70 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.435+0000 7f561d7e0740 10 client.18645 fetch_fsmap learned FSMap version 50 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.435+0000 7f561d7e0740 1 -- 172.21.15.2:0/2798175087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x564a7ad406a0 con 0x564a7addab70 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.435+0000 7f55f3fff640 1 -- 172.21.15.2:0/2798175087 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x564a7aa417e0 con 0x564a7addab70 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.435+0000 7f55f3fff640 1 -- 172.21.15.2:0/2798175087 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5600080020 con 0x564a7addab70 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.435+0000 7f561d7e0740 10 client.18645 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:09:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.435+0000 7f561d7e0740 10 client.18645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:09:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.435+0000 7f561d7e0740 20 client.18645 populate_metadata read hostname 'smithi002' 2024-03-15T13:09:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.435+0000 7f561d7e0740 1 --2- 172.21.15.2:0/2798175087 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564a7adf79e0 0x564a7adf9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.435+0000 7f561d7e0740 4 client.18645 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-15T13:09:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.435+0000 7f561d7e0740 1 -- 172.21.15.2:0/2798175087 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a7aa7e720 con 0x564a7adf79e0 2024-03-15T13:09:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.435+0000 7f56188ca640 1 --2- 172.21.15.2:0/2798175087 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564a7adf79e0 0x564a7adf9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.436+0000 7f55f3fff640 10 client.18645 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:09:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.436+0000 7f56188ca640 1 --2- 172.21.15.2:0/2798175087 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564a7adf79e0 0x564a7adf9e20 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63f165cfeae24b3d server_cookie=59830183e9a65e92 in_seq=0 out_seq=0 2024-03-15T13:09:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.436+0000 7f55f3fff640 1 -- 172.21.15.2:0/2798175087 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564a7aa7e720 con 0x564a7adf79e0 2024-03-15T13:09:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.436+0000 7f55f3fff640 10 client.18645 handle_command_reply: tid=0 2024-03-15T13:09:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.436+0000 7f561d7e0740 1 client.18645 shutdown 2024-03-15T13:09:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.437+0000 7f561d7e0740 1 -- 172.21.15.2:0/2798175087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7addab70 msgr2=0x564a7aded970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.437+0000 7f561d7e0740 1 --2- 172.21.15.2:0/2798175087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7addab70 0x564a7aded970 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f560009a6b0 tx=0x7f560009a1b0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.437+0000 7f561d7e0740 1 -- 172.21.15.2:0/2798175087 shutdown_connections 2024-03-15T13:09:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.437+0000 7f561d7e0740 1 --2- 172.21.15.2:0/2798175087 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564a7adf79e0 0x564a7adf9e20 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.437+0000 7f561d7e0740 1 --2- 172.21.15.2:0/2798175087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7ade3750 0x564a7ade7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.437+0000 7f561d7e0740 1 --2- 172.21.15.2:0/2798175087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7addab70 0x564a7aded970 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.437+0000 7f55f3fff640 0 client.18645 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:09:04.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.437+0000 7f561d7e0740 1 --2- 172.21.15.2:0/2798175087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7adda250 0x564a7aded430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.437+0000 7f561d7e0740 1 -- 172.21.15.2:0/2798175087 >> 172.21.15.2:0/2798175087 conn(0x564a7add4980 msgr2=0x564a7add66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.437+0000 7f561d7e0740 1 -- 172.21.15.2:0/2798175087 shutdown_connections 2024-03-15T13:09:04.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.437+0000 7f561d7e0740 1 -- 172.21.15.2:0/2798175087 wait complete. 2024-03-15T13:09:04.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.437+0000 7f561d7e0740 20 client.18645 trim_cache size 0 max 16384 2024-03-15T13:09:04.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f561d7e0740 1 Processor -- start 2024-03-15T13:09:04.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f561d7e0740 1 -- start start 2024-03-15T13:09:04.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f561d7e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7adda250 0x564a7ade1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f561d7e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7addab70 0x564a7ade2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f561d7e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 0x564a7adecf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f561d7e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7acc7130 con 0x564a7adda250 2024-03-15T13:09:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f561d7e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7adf7830 con 0x564a7ade3750 2024-03-15T13:09:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f561d7e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7aa417e0 con 0x564a7addab70 2024-03-15T13:09:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f56188ca640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7adda250 0x564a7ade1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f56188ca640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7adda250 0x564a7ade1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58134/0 (socket says 172.21.15.2:58134) 2024-03-15T13:09:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f56188ca640 1 -- 172.21.15.2:0/1796735786 learned_addr learned my addr 172.21.15.2:0/1796735786 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f5613fff640 1 --2- 172.21.15.2:0/1796735786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7addab70 0x564a7ade2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f56190cb640 1 --2- 172.21.15.2:0/1796735786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 0x564a7adecf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f56188ca640 1 -- 172.21.15.2:0/1796735786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7addab70 msgr2=0x564a7ade2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f56188ca640 1 --2- 172.21.15.2:0/1796735786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7addab70 0x564a7ade2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f56188ca640 1 -- 172.21.15.2:0/1796735786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 msgr2=0x564a7adecf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.449+0000 7f56188ca640 1 --2- 172.21.15.2:0/1796735786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 0x564a7adecf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.450+0000 7f56188ca640 1 -- 172.21.15.2:0/1796735786 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a7acdcc20 con 0x564a7adda250 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.450+0000 7f56190cb640 1 --2- 172.21.15.2:0/1796735786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 0x564a7adecf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.450+0000 7f56188ca640 1 --2- 172.21.15.2:0/1796735786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7adda250 0x564a7ade1bc0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f5608009900 tx=0x7f5608005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161a88797781f7f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.450+0000 7f55fe7fc640 1 -- 172.21.15.2:0/1796735786 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560804a330 con 0x564a7adda250 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.450+0000 7f55fe7fc640 1 -- 172.21.15.2:0/1796735786 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5608004510 con 0x564a7adda250 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.450+0000 7f55fe7fc640 1 -- 172.21.15.2:0/1796735786 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560804a860 con 0x564a7adda250 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.450+0000 7f561d7e0740 1 -- 172.21.15.2:0/1796735786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7adda250 msgr2=0x564a7ade1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.450+0000 7f561d7e0740 1 --2- 172.21.15.2:0/1796735786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7adda250 0x564a7ade1bc0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f5608009900 tx=0x7f5608005fd0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.450+0000 7f561d7e0740 1 -- 172.21.15.2:0/1796735786 shutdown_connections 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.450+0000 7f561d7e0740 1 --2- 172.21.15.2:0/1796735786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 0x564a7adecf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.450+0000 7f561d7e0740 1 --2- 172.21.15.2:0/1796735786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7addab70 0x564a7ade2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.450+0000 7f561d7e0740 1 --2- 172.21.15.2:0/1796735786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7adda250 0x564a7ade1bc0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.450+0000 7f561d7e0740 1 -- 172.21.15.2:0/1796735786 >> 172.21.15.2:0/1796735786 conn(0x564a7add4980 msgr2=0x564a7adcfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.450+0000 7f561d7e0740 1 -- 172.21.15.2:0/1796735786 shutdown_connections 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.451+0000 7f561d7e0740 1 -- 172.21.15.2:0/1796735786 wait complete. 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.451+0000 7f561d7e0740 1 -- start start 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.451+0000 7f561d7e0740 1 Processor -- start 2024-03-15T13:09:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.451+0000 7f561d7e0740 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7adda250 0x564a7adf0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.451+0000 7f561d7e0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7addab70 0x564a7adf1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.451+0000 7f561d7e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 0x564a7ae4c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.451+0000 7f561d7e0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ae4c8f0 con 0x564a7addab70 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.451+0000 7f561d7e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ae4ca60 con 0x564a7ade3750 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.451+0000 7f561d7e0740 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ae4cc20 con 0x564a7adda250 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.451+0000 7f561d7e0740 10 client.0 _ll_register_callbacks cb 0x7f56000b1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.451+0000 7f5613fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7addab70 0x564a7adf1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.451+0000 7f5613fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7addab70 0x564a7adf1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58144/0 (socket says 172.21.15.2:58144) 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.451+0000 7f5613fff640 1 -- 172.21.15.2:0/2080193599 learned_addr learned my addr 172.21.15.2:0/2080193599 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.451+0000 7f56188ca640 1 --2- 172.21.15.2:0/2080193599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7adda250 0x564a7adf0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.451+0000 7f56190cb640 1 --2- 172.21.15.2:0/2080193599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 0x564a7ae4c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f5613fff640 1 -- 172.21.15.2:0/2080193599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7adda250 msgr2=0x564a7adf0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f5613fff640 1 --2- 172.21.15.2:0/2080193599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7adda250 0x564a7adf0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f5613fff640 1 -- 172.21.15.2:0/2080193599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 msgr2=0x564a7ae4c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f5613fff640 1 --2- 172.21.15.2:0/2080193599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 0x564a7ae4c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f5613fff640 1 -- 172.21.15.2:0/2080193599 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561410da80 con 0x564a7addab70 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f5613fff640 1 --2- 172.21.15.2:0/2080193599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7addab70 0x564a7adf1480 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f560000e9c0 tx=0x7f5600004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800c6acbde3f5036 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f55f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f55f3fff640 1 -- 172.21.15.2:0/2080193599 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600083ea0 con 0x564a7addab70 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f55f3fff640 1 -- 172.21.15.2:0/2080193599 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f56000ba0d0 con 0x564a7addab70 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f561d7e0740 1 -- 172.21.15.2:0/2080193599 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56000b8e90 con 0x564a7addab70 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f55f3fff640 1 -- 172.21.15.2:0/2080193599 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f560009a260 con 0x564a7addab70 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f561d7e0740 10 client.18651 fetch_fsmap learned FSMap version 50 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f561d7e0740 1 -- 172.21.15.2:0/2080193599 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x564a7ad028c0 con 0x564a7addab70 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f55f3fff640 1 -- 172.21.15.2:0/2080193599 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f56000b8e90 con 0x564a7addab70 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f55f3fff640 1 -- 172.21.15.2:0/2080193599 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) v1 ==== 7942+0+0 (secure 0 0 0) 0x7f5600080070 con 0x564a7addab70 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f561d7e0740 10 client.18651 fetch_fsmap finished waiting for FSMap version 50 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f561d7e0740 10 client.18651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f561d7e0740 20 client.18651 populate_metadata read hostname 'smithi002' 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f561d7e0740 1 --2- 172.21.15.2:0/2080193599 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564a7ae53f60 0x564a7ae56350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f561d7e0740 4 client.18651 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.452+0000 7f561d7e0740 1 -- 172.21.15.2:0/2080193599 --> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564a7aacd890 con 0x564a7ae53f60 2024-03-15T13:09:04.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.453+0000 7f56188ca640 1 --2- 172.21.15.2:0/2080193599 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564a7ae53f60 0x564a7ae56350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:04.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.453+0000 7f56188ca640 1 --2- 172.21.15.2:0/2080193599 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564a7ae53f60 0x564a7ae56350 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca5802f98700d53 server_cookie=bf7f05e15abce94d in_seq=0 out_seq=0 2024-03-15T13:09:04.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.453+0000 7f55f3fff640 10 client.18651 ms_handle_connect on v2:172.21.15.149:6838/1715135128 2024-03-15T13:09:04.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.453+0000 7f55f3fff640 1 -- 172.21.15.2:0/2080193599 <== mds.0 v2:172.21.15.149:6838/1715135128 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564a7aacd890 con 0x564a7ae53f60 2024-03-15T13:09:04.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.453+0000 7f55f3fff640 10 client.18651 handle_command_reply: tid=0 2024-03-15T13:09:04.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.453+0000 7f561d7e0740 1 client.18651 shutdown 2024-03-15T13:09:04.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561d7e0740 1 -- 172.21.15.2:0/2080193599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7addab70 msgr2=0x564a7adf1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561d7e0740 1 --2- 172.21.15.2:0/2080193599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7addab70 0x564a7adf1480 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f560000e9c0 tx=0x7f5600004a30 comp rx=0 tx=0).stop 2024-03-15T13:09:04.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561d7e0740 1 -- 172.21.15.2:0/2080193599 shutdown_connections 2024-03-15T13:09:04.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561d7e0740 1 --2- 172.21.15.2:0/2080193599 >> [v2:172.21.15.149:6838/1715135128,v1:172.21.15.149:6839/1715135128] conn(0x564a7ae53f60 0x564a7ae56350 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561d7e0740 1 --2- 172.21.15.2:0/2080193599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a7ade3750 0x564a7ae4c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561d7e0740 1 --2- 172.21.15.2:0/2080193599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a7addab70 0x564a7adf1480 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f55f3fff640 0 client.18651 ms_handle_reset on v2:172.21.15.149:6838/1715135128 2024-03-15T13:09:04.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561d7e0740 1 --2- 172.21.15.2:0/2080193599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a7adda250 0x564a7adf0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561d7e0740 1 -- 172.21.15.2:0/2080193599 >> 172.21.15.2:0/2080193599 conn(0x564a7add4980 msgr2=0x564a7adecf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561d7e0740 1 -- 172.21.15.2:0/2080193599 shutdown_connections 2024-03-15T13:09:04.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561d7e0740 1 -- 172.21.15.2:0/2080193599 wait complete. 2024-03-15T13:09:04.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561d7e0740 20 client.18651 trim_cache size 0 max 16384 2024-03-15T13:09:04.444 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2024-03-15T13:09:04.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561ab2e640 1 -- 172.21.15.2:0/4227245753 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f56040a1c20 msgr2=0x7f56040a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561ab2e640 1 --2- 172.21.15.2:0/4227245753 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f56040a1c20 0x7f56040a40e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f560c009fd0 tx=0x7f560c013b00 comp rx=0 tx=0).stop 2024-03-15T13:09:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561ab2e640 1 -- 172.21.15.2:0/4227245753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56141056a0 msgr2=0x7f561419dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561ab2e640 1 --2- 172.21.15.2:0/4227245753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56141056a0 0x7f561419dd50 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f560000e9f0 tx=0x7f560000eeb0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561ab2e640 1 -- 172.21.15.2:0/4227245753 shutdown_connections 2024-03-15T13:09:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561ab2e640 1 --2- 172.21.15.2:0/4227245753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56141979c0 0x7f5614197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561ab2e640 1 --2- 172.21.15.2:0/4227245753 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f56040a1c20 0x7f56040a40e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561ab2e640 1 --2- 172.21.15.2:0/4227245753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56141056a0 0x7f561419dd50 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561ab2e640 1 --2- 172.21.15.2:0/4227245753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56141005d0 0x7f561419d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.454+0000 7f561ab2e640 1 -- 172.21.15.2:0/4227245753 >> 172.21.15.2:0/4227245753 conn(0x7f56140774f0 msgr2=0x7f56140689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.455+0000 7f561ab2e640 1 -- 172.21.15.2:0/4227245753 shutdown_connections 2024-03-15T13:09:04.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:04.455+0000 7f561ab2e640 1 -- 172.21.15.2:0/4227245753 wait complete. 2024-03-15T13:09:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:04 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1174511665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:09:04.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:04 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1174511665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:09:05.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:04 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1174511665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:09:05.450 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-15T13:09:05.450 INFO:tasks.fwd_scrub:done joining 2024-03-15T13:09:05.450 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-15T13:09:05.461 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T13:09:05.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.591+0000 7f08d73a0640 1 -- 172.21.15.2:0/2981498462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08d0100db0 msgr2=0x7f08d010d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:05.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.591+0000 7f08d73a0640 1 --2- 172.21.15.2:0/2981498462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08d0100db0 0x7f08d010d2c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009cd0 tx=0x7f08c802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.591+0000 7f08d73a0640 1 -- 172.21.15.2:0/2981498462 shutdown_connections 2024-03-15T13:09:05.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.591+0000 7f08d73a0640 1 --2- 172.21.15.2:0/2981498462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08d0100db0 0x7f08d010d2c0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.591+0000 7f08d73a0640 1 --2- 172.21.15.2:0/2981498462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08d01003f0 0x7f08d0100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.591+0000 7f08d73a0640 1 --2- 172.21.15.2:0/2981498462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08d0106490 0x7f08d0106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.591+0000 7f08d73a0640 1 -- 172.21.15.2:0/2981498462 >> 172.21.15.2:0/2981498462 conn(0x7f08d00774f0 msgr2=0x7f08d0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.591+0000 7f08d73a0640 1 -- 172.21.15.2:0/2981498462 shutdown_connections 2024-03-15T13:09:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.591+0000 7f08d73a0640 1 -- 172.21.15.2:0/2981498462 wait complete. 2024-03-15T13:09:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.591+0000 7f08d73a0640 1 Processor -- start 2024-03-15T13:09:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.591+0000 7f08d73a0640 1 -- start start 2024-03-15T13:09:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d73a0640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08d01003f0 0x7f08d0196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d73a0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08d0100db0 0x7f08d0196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d73a0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08d0106490 0x7f08d01971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d73a0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d01a5e90 con 0x7f08d0106490 2024-03-15T13:09:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d73a0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d01a6000 con 0x7f08d0100db0 2024-03-15T13:09:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d73a0640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d01a61c0 con 0x7f08d01003f0 2024-03-15T13:09:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d593d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08d0106490 0x7f08d01971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d493b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08d0100db0 0x7f08d0196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d593d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08d0106490 0x7f08d01971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58158/0 (socket says 172.21.15.2:58158) 2024-03-15T13:09:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d593d640 1 -- 172.21.15.2:0/1192206786 learned_addr learned my addr 172.21.15.2:0/1192206786 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d513c640 1 --2- 172.21.15.2:0/1192206786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08d01003f0 0x7f08d0196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d593d640 1 -- 172.21.15.2:0/1192206786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08d01003f0 msgr2=0x7f08d0196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d593d640 1 --2- 172.21.15.2:0/1192206786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08d01003f0 0x7f08d0196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d593d640 1 -- 172.21.15.2:0/1192206786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08d0100db0 msgr2=0x7f08d0196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d593d640 1 --2- 172.21.15.2:0/1192206786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08d0100db0 0x7f08d0196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d593d640 1 -- 172.21.15.2:0/1192206786 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c8009900 con 0x7f08d0106490 2024-03-15T13:09:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d493b640 1 --2- 172.21.15.2:0/1192206786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08d0100db0 0x7f08d0196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08d593d640 1 --2- 172.21.15.2:0/1192206786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08d0106490 0x7f08d01971d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f08c8005cc0 tx=0x7f08c8034a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.592+0000 7f08c67fc640 1 -- 172.21.15.2:0/1192206786 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c8032880 con 0x7f08d0106490 2024-03-15T13:09:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.593+0000 7f08d73a0640 1 -- 172.21.15.2:0/1192206786 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d01a6440 con 0x7f08d0106490 2024-03-15T13:09:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.593+0000 7f08c67fc640 1 -- 172.21.15.2:0/1192206786 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f08c8034e80 con 0x7f08d0106490 2024-03-15T13:09:05.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.593+0000 7f08c67fc640 1 -- 172.21.15.2:0/1192206786 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c80047c0 con 0x7f08d0106490 2024-03-15T13:09:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.593+0000 7f08d73a0640 1 -- 172.21.15.2:0/1192206786 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d01a6940 con 0x7f08d0106490 2024-03-15T13:09:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.593+0000 7f08d73a0640 1 -- 172.21.15.2:0/1192206786 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d00682e0 con 0x7f08d0106490 2024-03-15T13:09:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.595+0000 7f08c67fc640 1 -- 172.21.15.2:0/1192206786 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f08c800a040 con 0x7f08d0106490 2024-03-15T13:09:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.595+0000 7f08c67fc640 1 --2- 172.21.15.2:0/1192206786 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f08bc0a1ba0 0x7f08bc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.596+0000 7f08c67fc640 1 -- 172.21.15.2:0/1192206786 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f08c80f0ac0 con 0x7f08d0106490 2024-03-15T13:09:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.596+0000 7f08d513c640 1 --2- 172.21.15.2:0/1192206786 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f08bc0a1ba0 0x7f08bc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.596+0000 7f08d513c640 1 --2- 172.21.15.2:0/1192206786 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f08bc0a1ba0 0x7f08bc0a4060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009b50 tx=0x7f08cc005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:05.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.599+0000 7f08c67fc640 1 -- 172.21.15.2:0/1192206786 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f08c80b9700 con 0x7f08d0106490 2024-03-15T13:09:05.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.783+0000 7f08d73a0640 1 -- 172.21.15.2:0/1192206786 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08d0108980 con 0x7f08d0106490 2024-03-15T13:09:05.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.784+0000 7f08c67fc640 1 -- 172.21.15.2:0/1192206786 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+25137 (secure 0 0 0) 0x7f08c80bcdc0 con 0x7f08d0106490 2024-03-15T13:09:05.771 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:09:05.771 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":98,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T13:08:20.544787+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":14,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":14,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/343748862":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/1602727020":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/883225361":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/3173750072":"2024-03-16T12:58:53.781001+0000","172.21.15.2:6800/343748862":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/2211760175":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/4030567035":"2024-03-16T12:58:53.781001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":14,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":14,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T13:09:05.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.786+0000 7f08d73a0640 1 -- 172.21.15.2:0/1192206786 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f08bc0a1ba0 msgr2=0x7f08bc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:05.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.786+0000 7f08d73a0640 1 --2- 172.21.15.2:0/1192206786 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f08bc0a1ba0 0x7f08bc0a4060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009b50 tx=0x7f08cc005f90 comp rx=0 tx=0).stop 2024-03-15T13:09:05.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.786+0000 7f08d73a0640 1 -- 172.21.15.2:0/1192206786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08d0106490 msgr2=0x7f08d01971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:05.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.786+0000 7f08d73a0640 1 --2- 172.21.15.2:0/1192206786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08d0106490 0x7f08d01971d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f08c8005cc0 tx=0x7f08c8034a80 comp rx=0 tx=0).stop 2024-03-15T13:09:05.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.787+0000 7f08d73a0640 1 -- 172.21.15.2:0/1192206786 shutdown_connections 2024-03-15T13:09:05.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.787+0000 7f08d73a0640 1 --2- 172.21.15.2:0/1192206786 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f08bc0a1ba0 0x7f08bc0a4060 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.787+0000 7f08d73a0640 1 --2- 172.21.15.2:0/1192206786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08d0106490 0x7f08d01971d0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.787+0000 7f08d73a0640 1 --2- 172.21.15.2:0/1192206786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08d0100db0 0x7f08d0196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.787+0000 7f08d73a0640 1 --2- 172.21.15.2:0/1192206786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08d01003f0 0x7f08d0196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.787+0000 7f08d73a0640 1 -- 172.21.15.2:0/1192206786 >> 172.21.15.2:0/1192206786 conn(0x7f08d00774f0 msgr2=0x7f08d0102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:05.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.787+0000 7f08d73a0640 1 -- 172.21.15.2:0/1192206786 shutdown_connections 2024-03-15T13:09:05.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.787+0000 7f08d73a0640 1 -- 172.21.15.2:0/1192206786 wait complete. 2024-03-15T13:09:05.780 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:05 smithi002 ceph-mon[32060]: pgmap v314: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-15T13:09:05.782 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-15T12:51:47.004843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-15T12:56:18.559882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-15T12:56:19.751767+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-15T12:56:23.664879+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-15T13:09:05.782 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-15T13:09:05.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:05 smithi149 ceph-mon[35927]: pgmap v314: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-15T13:09:05.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.903+0000 7f495012c640 1 -- 172.21.15.2:0/850950334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948101e60 msgr2=0x7f494810e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:05.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.903+0000 7f495012c640 1 --2- 172.21.15.2:0/850950334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948101e60 0x7f494810e320 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f4940009ca0 tx=0x7f494002c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.903+0000 7f495012c640 1 -- 172.21.15.2:0/850950334 shutdown_connections 2024-03-15T13:09:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.903+0000 7f495012c640 1 --2- 172.21.15.2:0/850950334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948101e60 0x7f494810e320 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.903+0000 7f495012c640 1 --2- 172.21.15.2:0/850950334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49481014a0 0x7f4948101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.903+0000 7f495012c640 1 --2- 172.21.15.2:0/850950334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4948107560 0x7f4948107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.903+0000 7f495012c640 1 -- 172.21.15.2:0/850950334 >> 172.21.15.2:0/850950334 conn(0x7f49480788f0 msgr2=0x7f4948078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.903+0000 7f495012c640 1 -- 172.21.15.2:0/850950334 shutdown_connections 2024-03-15T13:09:05.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.903+0000 7f495012c640 1 -- 172.21.15.2:0/850950334 wait complete. 2024-03-15T13:09:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f495012c640 1 Processor -- start 2024-03-15T13:09:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f495012c640 1 -- start start 2024-03-15T13:09:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f495012c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49481014a0 0x7f49480737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:05.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f495012c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4948101e60 0x7f4948073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f495012c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948107560 0x7f4948074270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f495012c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49481a5e00 con 0x7f4948107560 2024-03-15T13:09:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f495012c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49481a5f70 con 0x7f4948101e60 2024-03-15T13:09:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f495012c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49481a6130 con 0x7f49481014a0 2024-03-15T13:09:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f494e6c9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948107560 0x7f4948074270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f494dec8640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49481014a0 0x7f49480737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f494e6c9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948107560 0x7f4948074270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58174/0 (socket says 172.21.15.2:58174) 2024-03-15T13:09:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f494e6c9640 1 -- 172.21.15.2:0/3780449994 learned_addr learned my addr 172.21.15.2:0/3780449994 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f494d6c7640 1 --2- 172.21.15.2:0/3780449994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4948101e60 0x7f4948073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f494e6c9640 1 -- 172.21.15.2:0/3780449994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49481014a0 msgr2=0x7f49480737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f494e6c9640 1 --2- 172.21.15.2:0/3780449994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49481014a0 0x7f49480737f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f494e6c9640 1 -- 172.21.15.2:0/3780449994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4948101e60 msgr2=0x7f4948073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:05.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f494e6c9640 1 --2- 172.21.15.2:0/3780449994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4948101e60 0x7f4948073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:05.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f494e6c9640 1 -- 172.21.15.2:0/3780449994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4940009900 con 0x7f4948107560 2024-03-15T13:09:05.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.904+0000 7f494e6c9640 1 --2- 172.21.15.2:0/3780449994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948107560 0x7f4948074270 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f4940005cc0 tx=0x7f49400325f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:05.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.905+0000 7f493effd640 1 -- 172.21.15.2:0/3780449994 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49400328b0 con 0x7f4948107560 2024-03-15T13:09:05.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.905+0000 7f493effd640 1 -- 172.21.15.2:0/3780449994 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f4940003e40 con 0x7f4948107560 2024-03-15T13:09:05.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.905+0000 7f495012c640 1 -- 172.21.15.2:0/3780449994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49481a63b0 con 0x7f4948107560 2024-03-15T13:09:05.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.905+0000 7f493effd640 1 -- 172.21.15.2:0/3780449994 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940040450 con 0x7f4948107560 2024-03-15T13:09:05.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.905+0000 7f495012c640 1 -- 172.21.15.2:0/3780449994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49481a68b0 con 0x7f4948107560 2024-03-15T13:09:05.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.906+0000 7f495012c640 1 -- 172.21.15.2:0/3780449994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4948068730 con 0x7f4948107560 2024-03-15T13:09:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.907+0000 7f493effd640 1 -- 172.21.15.2:0/3780449994 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f494000a040 con 0x7f4948107560 2024-03-15T13:09:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.907+0000 7f493effd640 1 --2- 172.21.15.2:0/3780449994 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f49340a1c20 0x7f49340a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.908+0000 7f494dec8640 1 --2- 172.21.15.2:0/3780449994 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f49340a1c20 0x7f49340a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.908+0000 7f493effd640 1 -- 172.21.15.2:0/3780449994 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f49400f0f80 con 0x7f4948107560 2024-03-15T13:09:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.908+0000 7f494dec8640 1 --2- 172.21.15.2:0/3780449994 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f49340a1c20 0x7f49340a40e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4944009b50 tx=0x7f4944005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:05.911+0000 7f493effd640 1 -- 172.21.15.2:0/3780449994 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f49400b9ce0 con 0x7f4948107560 2024-03-15T13:09:05.958 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:05 smithi184 ceph-mon[35993]: pgmap v314: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-15T13:09:06.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.093+0000 7f495012c640 1 -- 172.21.15.2:0/3780449994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4948109970 con 0x7f4948107560 2024-03-15T13:09:06.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.094+0000 7f493effd640 1 -- 172.21.15.2:0/3780449994 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49400bd3a0 con 0x7f4948107560 2024-03-15T13:09:06.078 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-15T13:09:06.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.096+0000 7f495012c640 1 -- 172.21.15.2:0/3780449994 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f49340a1c20 msgr2=0x7f49340a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:06.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.096+0000 7f495012c640 1 --2- 172.21.15.2:0/3780449994 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f49340a1c20 0x7f49340a40e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4944009b50 tx=0x7f4944005f90 comp rx=0 tx=0).stop 2024-03-15T13:09:06.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.096+0000 7f495012c640 1 -- 172.21.15.2:0/3780449994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948107560 msgr2=0x7f4948074270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:06.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.096+0000 7f495012c640 1 --2- 172.21.15.2:0/3780449994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948107560 0x7f4948074270 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f4940005cc0 tx=0x7f49400325f0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.096+0000 7f495012c640 1 -- 172.21.15.2:0/3780449994 shutdown_connections 2024-03-15T13:09:06.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.096+0000 7f495012c640 1 --2- 172.21.15.2:0/3780449994 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f49340a1c20 0x7f49340a40e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.096+0000 7f495012c640 1 --2- 172.21.15.2:0/3780449994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4948107560 0x7f4948074270 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.096+0000 7f495012c640 1 --2- 172.21.15.2:0/3780449994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4948101e60 0x7f4948073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.096+0000 7f495012c640 1 --2- 172.21.15.2:0/3780449994 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49481014a0 0x7f49480737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.096+0000 7f495012c640 1 -- 172.21.15.2:0/3780449994 >> 172.21.15.2:0/3780449994 conn(0x7f49480788f0 msgr2=0x7f49480fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:06.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.096+0000 7f495012c640 1 -- 172.21.15.2:0/3780449994 shutdown_connections 2024-03-15T13:09:06.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.096+0000 7f495012c640 1 -- 172.21.15.2:0/3780449994 wait complete. 2024-03-15T13:09:06.091 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-15T13:09:06.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.212+0000 7fb8eb156640 1 -- 172.21.15.2:0/4275073264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e41005d0 msgr2=0x7fb8e41009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:06.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.212+0000 7fb8eb156640 1 --2- 172.21.15.2:0/4275073264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e41005d0 0x7fb8e41009b0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009930 tx=0x7fb8d802bc30 comp rx=0 tx=0).stop 2024-03-15T13:09:06.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.212+0000 7fb8eb156640 1 -- 172.21.15.2:0/4275073264 shutdown_connections 2024-03-15T13:09:06.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.212+0000 7fb8eb156640 1 --2- 172.21.15.2:0/4275073264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8e41056a0 0x7fb8e410d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.212+0000 7fb8eb156640 1 --2- 172.21.15.2:0/4275073264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8e4100f80 0x7fb8e4104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.212+0000 7fb8eb156640 1 --2- 172.21.15.2:0/4275073264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e41005d0 0x7fb8e41009b0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.212+0000 7fb8eb156640 1 -- 172.21.15.2:0/4275073264 >> 172.21.15.2:0/4275073264 conn(0x7fb8e40774f0 msgr2=0x7fb8e4077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:06.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.212+0000 7fb8eb156640 1 -- 172.21.15.2:0/4275073264 shutdown_connections 2024-03-15T13:09:06.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.212+0000 7fb8eb156640 1 -- 172.21.15.2:0/4275073264 wait complete. 2024-03-15T13:09:06.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.212+0000 7fb8eb156640 1 Processor -- start 2024-03-15T13:09:06.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.212+0000 7fb8eb156640 1 -- start start 2024-03-15T13:09:06.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.212+0000 7fb8eb156640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8e41005d0 0x7fb8e4199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:06.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.212+0000 7fb8eb156640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8e41056a0 0x7fb8e419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:06.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.212+0000 7fb8eb156640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e419e700 0x7fb8e419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:06.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8eb156640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e419f0f0 con 0x7fb8e419e700 2024-03-15T13:09:06.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8eb156640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e419f260 con 0x7fb8e41005d0 2024-03-15T13:09:06.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8eb156640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e419f3d0 con 0x7fb8e41056a0 2024-03-15T13:09:06.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8e96f3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e419e700 0x7fb8e419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:06.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8e96f3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e419e700 0x7fb8e419ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58196/0 (socket says 172.21.15.2:58196) 2024-03-15T13:09:06.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8e96f3640 1 -- 172.21.15.2:0/3473197463 learned_addr learned my addr 172.21.15.2:0/3473197463 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:06.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8e8ef2640 1 --2- 172.21.15.2:0/3473197463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8e41005d0 0x7fb8e4199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:06.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8e3fff640 1 --2- 172.21.15.2:0/3473197463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8e41056a0 0x7fb8e419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:06.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8e96f3640 1 -- 172.21.15.2:0/3473197463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8e41056a0 msgr2=0x7fb8e419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:06.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8e96f3640 1 --2- 172.21.15.2:0/3473197463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8e41056a0 0x7fb8e419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8e96f3640 1 -- 172.21.15.2:0/3473197463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8e41005d0 msgr2=0x7fb8e4199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:06.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8e96f3640 1 --2- 172.21.15.2:0/3473197463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8e41005d0 0x7fb8e4199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8e96f3640 1 -- 172.21.15.2:0/3473197463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d8009560 con 0x7fb8e419e700 2024-03-15T13:09:06.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8e96f3640 1 --2- 172.21.15.2:0/3473197463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e419e700 0x7fb8e419ebb0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00eb10 tx=0x7fb8dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:06.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8e1ffb640 1 -- 172.21.15.2:0/3473197463 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc004b20 con 0x7fb8e419e700 2024-03-15T13:09:06.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8e1ffb640 1 -- 172.21.15.2:0/3473197463 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fb8dc004c80 con 0x7fb8e419e700 2024-03-15T13:09:06.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8e1ffb640 1 -- 172.21.15.2:0/3473197463 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc010690 con 0x7fb8e419e700 2024-03-15T13:09:06.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8eb156640 1 -- 172.21.15.2:0/3473197463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e41a69b0 con 0x7fb8e419e700 2024-03-15T13:09:06.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.213+0000 7fb8eb156640 1 -- 172.21.15.2:0/3473197463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e41a6e60 con 0x7fb8e419e700 2024-03-15T13:09:06.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.214+0000 7fb8eb156640 1 -- 172.21.15.2:0/3473197463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e40682e0 con 0x7fb8e419e700 2024-03-15T13:09:06.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.215+0000 7fb8e1ffb640 1 -- 172.21.15.2:0/3473197463 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fb8dc004df0 con 0x7fb8e419e700 2024-03-15T13:09:06.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.216+0000 7fb8e1ffb640 1 --2- 172.21.15.2:0/3473197463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb8d40a1c20 0x7fb8d40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:06.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.216+0000 7fb8e1ffb640 1 -- 172.21.15.2:0/3473197463 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fb8dc014070 con 0x7fb8e419e700 2024-03-15T13:09:06.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.216+0000 7fb8e8ef2640 1 --2- 172.21.15.2:0/3473197463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb8d40a1c20 0x7fb8d40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:06.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.216+0000 7fb8e8ef2640 1 --2- 172.21.15.2:0/3473197463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb8d40a1c20 0x7fb8d40a40e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8000c00 tx=0x7fb8d8005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:06.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.220+0000 7fb8e1ffb640 1 -- 172.21.15.2:0/3473197463 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb8dc096b80 con 0x7fb8e419e700 2024-03-15T13:09:06.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.400+0000 7fb8eb156640 1 -- 172.21.15.2:0/3473197463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8e4100fb0 con 0x7fb8e419e700 2024-03-15T13:09:06.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.400+0000 7fb8e1ffb640 1 -- 172.21.15.2:0/3473197463 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb8dc09a240 con 0x7fb8e419e700 2024-03-15T13:09:06.385 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T13:09:06.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.402+0000 7fb8eb156640 1 -- 172.21.15.2:0/3473197463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb8d40a1c20 msgr2=0x7fb8d40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:06.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.402+0000 7fb8eb156640 1 --2- 172.21.15.2:0/3473197463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb8d40a1c20 0x7fb8d40a40e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8000c00 tx=0x7fb8d8005fd0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.402+0000 7fb8eb156640 1 -- 172.21.15.2:0/3473197463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e419e700 msgr2=0x7fb8e419ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:06.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.402+0000 7fb8eb156640 1 --2- 172.21.15.2:0/3473197463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e419e700 0x7fb8e419ebb0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00eb10 tx=0x7fb8dc00c2d0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.402+0000 7fb8eb156640 1 -- 172.21.15.2:0/3473197463 shutdown_connections 2024-03-15T13:09:06.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.402+0000 7fb8eb156640 1 --2- 172.21.15.2:0/3473197463 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fb8d40a1c20 0x7fb8d40a40e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.402+0000 7fb8eb156640 1 --2- 172.21.15.2:0/3473197463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e419e700 0x7fb8e419ebb0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.402+0000 7fb8eb156640 1 --2- 172.21.15.2:0/3473197463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8e41056a0 0x7fb8e419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.402+0000 7fb8eb156640 1 --2- 172.21.15.2:0/3473197463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8e41005d0 0x7fb8e4199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.402+0000 7fb8eb156640 1 -- 172.21.15.2:0/3473197463 >> 172.21.15.2:0/3473197463 conn(0x7fb8e40774f0 msgr2=0x7fb8e40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:06.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.403+0000 7fb8eb156640 1 -- 172.21.15.2:0/3473197463 shutdown_connections 2024-03-15T13:09:06.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.403+0000 7fb8eb156640 1 -- 172.21.15.2:0/3473197463 wait complete. 2024-03-15T13:09:06.398 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-15T13:09:06.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.519+0000 7fe63966b640 1 -- 172.21.15.2:0/3649769575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe634104b20 msgr2=0x7fe634104f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:06.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.519+0000 7fe63966b640 1 --2- 172.21.15.2:0/3649769575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe634104b20 0x7fe634104f00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe624009890 tx=0x7fe62402bb70 comp rx=0 tx=0).stop 2024-03-15T13:09:06.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.519+0000 7fe63966b640 1 -- 172.21.15.2:0/3649769575 shutdown_connections 2024-03-15T13:09:06.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.519+0000 7fe63966b640 1 --2- 172.21.15.2:0/3649769575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe634100750 0x7fe63410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.519+0000 7fe63966b640 1 --2- 172.21.15.2:0/3649769575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6340ffdb0 0x7fe634100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.519+0000 7fe63966b640 1 --2- 172.21.15.2:0/3649769575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe634104b20 0x7fe634104f00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.519+0000 7fe63966b640 1 -- 172.21.15.2:0/3649769575 >> 172.21.15.2:0/3649769575 conn(0x7fe634075730 msgr2=0x7fe634075b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:06.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.519+0000 7fe63966b640 1 -- 172.21.15.2:0/3649769575 shutdown_connections 2024-03-15T13:09:06.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.519+0000 7fe63966b640 1 -- 172.21.15.2:0/3649769575 wait complete. 2024-03-15T13:09:06.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe63966b640 1 Processor -- start 2024-03-15T13:09:06.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe63966b640 1 -- start start 2024-03-15T13:09:06.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe63966b640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6340ffdb0 0x7fe634101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:06.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe63966b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe634100750 0x7fe634101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe63966b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe634104b20 0x7fe634102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe63966b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6341a5ed0 con 0x7fe634104b20 2024-03-15T13:09:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe63966b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6341a6040 con 0x7fe634100750 2024-03-15T13:09:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe63966b640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6341a6200 con 0x7fe6340ffdb0 2024-03-15T13:09:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe6337fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe634104b20 0x7fe634102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe6337fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe634104b20 0x7fe634102290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58220/0 (socket says 172.21.15.2:58220) 2024-03-15T13:09:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe6337fe640 1 -- 172.21.15.2:0/2885887196 learned_addr learned my addr 172.21.15.2:0/2885887196 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe632ffd640 1 --2- 172.21.15.2:0/2885887196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6340ffdb0 0x7fe634101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe62bfff640 1 --2- 172.21.15.2:0/2885887196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe634100750 0x7fe634101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe6337fe640 1 -- 172.21.15.2:0/2885887196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6340ffdb0 msgr2=0x7fe634101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe6337fe640 1 --2- 172.21.15.2:0/2885887196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6340ffdb0 0x7fe634101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe6337fe640 1 -- 172.21.15.2:0/2885887196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe634100750 msgr2=0x7fe634101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe6337fe640 1 --2- 172.21.15.2:0/2885887196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe634100750 0x7fe634101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe6337fe640 1 -- 172.21.15.2:0/2885887196 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c009560 con 0x7fe634104b20 2024-03-15T13:09:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.520+0000 7fe6337fe640 1 --2- 172.21.15.2:0/2885887196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe634104b20 0x7fe634102290 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00ec60 tx=0x7fe62c00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:06.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.521+0000 7fe630ff9640 1 -- 172.21.15.2:0/2885887196 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c004d20 con 0x7fe634104b20 2024-03-15T13:09:06.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.521+0000 7fe63966b640 1 -- 172.21.15.2:0/2885887196 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe624009490 con 0x7fe634104b20 2024-03-15T13:09:06.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.521+0000 7fe630ff9640 1 -- 172.21.15.2:0/2885887196 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fe62c004e80 con 0x7fe634104b20 2024-03-15T13:09:06.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.521+0000 7fe630ff9640 1 -- 172.21.15.2:0/2885887196 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c0055e0 con 0x7fe634104b20 2024-03-15T13:09:06.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.521+0000 7fe63966b640 1 -- 172.21.15.2:0/2885887196 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6341a68b0 con 0x7fe634104b20 2024-03-15T13:09:06.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.522+0000 7fe63966b640 1 -- 172.21.15.2:0/2885887196 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe634068730 con 0x7fe634104b20 2024-03-15T13:09:06.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.523+0000 7fe630ff9640 1 -- 172.21.15.2:0/2885887196 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fe62c00ed50 con 0x7fe634104b20 2024-03-15T13:09:06.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.523+0000 7fe630ff9640 1 --2- 172.21.15.2:0/2885887196 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe6200a1ba0 0x7fe6200a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:06.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.523+0000 7fe632ffd640 1 --2- 172.21.15.2:0/2885887196 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe6200a1ba0 0x7fe6200a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:06.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.524+0000 7fe630ff9640 1 -- 172.21.15.2:0/2885887196 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fe62c0cdc70 con 0x7fe634104b20 2024-03-15T13:09:06.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.524+0000 7fe632ffd640 1 --2- 172.21.15.2:0/2885887196 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe6200a1ba0 0x7fe6200a4060 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe624000c00 tx=0x7fe6240058b0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:06.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.528+0000 7fe630ff9640 1 -- 172.21.15.2:0/2885887196 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe62c0967a0 con 0x7fe634104b20 2024-03-15T13:09:06.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.711+0000 7fe63966b640 1 -- 172.21.15.2:0/2885887196 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe63410a080 con 0x7fe634104b20 2024-03-15T13:09:06.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.711+0000 7fe630ff9640 1 -- 172.21.15.2:0/2885887196 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe62c099e60 con 0x7fe634104b20 2024-03-15T13:09:06.696 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-03-15T13:09:06.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.713+0000 7fe63966b640 1 -- 172.21.15.2:0/2885887196 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe6200a1ba0 msgr2=0x7fe6200a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:06.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.713+0000 7fe63966b640 1 --2- 172.21.15.2:0/2885887196 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe6200a1ba0 0x7fe6200a4060 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe624000c00 tx=0x7fe6240058b0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.713+0000 7fe63966b640 1 -- 172.21.15.2:0/2885887196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe634104b20 msgr2=0x7fe634102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:06.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.713+0000 7fe63966b640 1 --2- 172.21.15.2:0/2885887196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe634104b20 0x7fe634102290 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00ec60 tx=0x7fe62c00c7a0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.713+0000 7fe63966b640 1 -- 172.21.15.2:0/2885887196 shutdown_connections 2024-03-15T13:09:06.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.713+0000 7fe63966b640 1 --2- 172.21.15.2:0/2885887196 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fe6200a1ba0 0x7fe6200a4060 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.713+0000 7fe63966b640 1 --2- 172.21.15.2:0/2885887196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe634104b20 0x7fe634102290 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.713+0000 7fe63966b640 1 --2- 172.21.15.2:0/2885887196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe634100750 0x7fe634101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.713+0000 7fe63966b640 1 --2- 172.21.15.2:0/2885887196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6340ffdb0 0x7fe634101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.713+0000 7fe63966b640 1 -- 172.21.15.2:0/2885887196 >> 172.21.15.2:0/2885887196 conn(0x7fe634075730 msgr2=0x7fe6340fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:06.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.713+0000 7fe63966b640 1 -- 172.21.15.2:0/2885887196 shutdown_connections 2024-03-15T13:09:06.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.713+0000 7fe63966b640 1 -- 172.21.15.2:0/2885887196 wait complete. 2024-03-15T13:09:06.707 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:06 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1192206786' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T13:09:06.707 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:06 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3780449994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T13:09:06.707 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:06 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3473197463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T13:09:06.708 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-15T13:09:06.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.829+0000 7fedda04e640 1 -- 172.21.15.2:0/857244585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd41056e0 msgr2=0x7fedd410d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:06.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.829+0000 7fedda04e640 1 --2- 172.21.15.2:0/857244585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd41056e0 0x7fedd410d2b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009cd0 tx=0x7fedcc02c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.829+0000 7fedda04e640 1 -- 172.21.15.2:0/857244585 shutdown_connections 2024-03-15T13:09:06.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.829+0000 7fedda04e640 1 --2- 172.21.15.2:0/857244585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd41056e0 0x7fedd410d2b0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.829+0000 7fedda04e640 1 --2- 172.21.15.2:0/857244585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedd4100fc0 0x7fedd4105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.829+0000 7fedda04e640 1 --2- 172.21.15.2:0/857244585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedd4100610 0x7fedd41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.829+0000 7fedda04e640 1 -- 172.21.15.2:0/857244585 >> 172.21.15.2:0/857244585 conn(0x7fedd4076770 msgr2=0x7fedd4076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:06.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.829+0000 7fedda04e640 1 -- 172.21.15.2:0/857244585 shutdown_connections 2024-03-15T13:09:06.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.829+0000 7fedda04e640 1 -- 172.21.15.2:0/857244585 wait complete. 2024-03-15T13:09:06.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.829+0000 7fedda04e640 1 Processor -- start 2024-03-15T13:09:06.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedda04e640 1 -- start start 2024-03-15T13:09:06.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedda04e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd4100610 0x7fedd419d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:06.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedda04e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedd41056e0 0x7fedd419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedda04e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedd4197990 0x7fedd4197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedda04e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd4198380 con 0x7fedd4100610 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedda04e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd41984f0 con 0x7fedd41056e0 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedda04e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd41986b0 con 0x7fedd4197990 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedd37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd4100610 0x7fedd419d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedd37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd4100610 0x7fedd419d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58242/0 (socket says 172.21.15.2:58242) 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedd37fe640 1 -- 172.21.15.2:0/55841790 learned_addr learned my addr 172.21.15.2:0/55841790 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedd3fff640 1 --2- 172.21.15.2:0/55841790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedd4197990 0x7fedd4197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedd2ffd640 1 --2- 172.21.15.2:0/55841790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedd41056e0 0x7fedd419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedd37fe640 1 -- 172.21.15.2:0/55841790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedd4197990 msgr2=0x7fedd4197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedd37fe640 1 --2- 172.21.15.2:0/55841790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedd4197990 0x7fedd4197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedd37fe640 1 -- 172.21.15.2:0/55841790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedd41056e0 msgr2=0x7fedd419dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedd37fe640 1 --2- 172.21.15.2:0/55841790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedd41056e0 0x7fedd419dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedd37fe640 1 -- 172.21.15.2:0/55841790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc009900 con 0x7fedd4100610 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.830+0000 7fedd37fe640 1 --2- 172.21.15.2:0/55841790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd4100610 0x7fedd419d7e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fedc400e9f0 tx=0x7fedc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.831+0000 7fedd0ff9640 1 -- 172.21.15.2:0/55841790 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4009700 con 0x7fedd4100610 2024-03-15T13:09:06.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.831+0000 7fedd0ff9640 1 -- 172.21.15.2:0/55841790 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fedc400cd80 con 0x7fedd4100610 2024-03-15T13:09:06.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.831+0000 7fedda04e640 1 -- 172.21.15.2:0/55841790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd41a46a0 con 0x7fedd4100610 2024-03-15T13:09:06.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.831+0000 7fedd0ff9640 1 -- 172.21.15.2:0/55841790 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4018920 con 0x7fedd4100610 2024-03-15T13:09:06.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.831+0000 7fedda04e640 1 -- 172.21.15.2:0/55841790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd41a4bb0 con 0x7fedd4100610 2024-03-15T13:09:06.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.832+0000 7fedda04e640 1 -- 172.21.15.2:0/55841790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd4101ae0 con 0x7fedd4100610 2024-03-15T13:09:06.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.833+0000 7fedd0ff9640 1 -- 172.21.15.2:0/55841790 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fedc4018a80 con 0x7fedd4100610 2024-03-15T13:09:06.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.833+0000 7fedd0ff9640 1 --2- 172.21.15.2:0/55841790 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fedbc0a1ba0 0x7fedbc0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:06.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.833+0000 7fedd2ffd640 1 --2- 172.21.15.2:0/55841790 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fedbc0a1ba0 0x7fedbc0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:06.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.833+0000 7fedd0ff9640 1 -- 172.21.15.2:0/55841790 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fedc4014070 con 0x7fedd4100610 2024-03-15T13:09:06.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.834+0000 7fedd2ffd640 1 --2- 172.21.15.2:0/55841790 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fedbc0a1ba0 0x7fedbc0a4060 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fedd40ffc50 tx=0x7fedc8005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:06.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:06.838+0000 7fedd0ff9640 1 -- 172.21.15.2:0/55841790 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fedc4053280 con 0x7fedd4100610 2024-03-15T13:09:06.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:06 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1192206786' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T13:09:06.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:06 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3780449994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T13:09:06.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:06 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3473197463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T13:09:07.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.022+0000 7fedda04e640 1 -- 172.21.15.2:0/55841790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fedd4108a00 con 0x7fedd4100610 2024-03-15T13:09:07.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.022+0000 7fedd0ff9640 1 -- 172.21.15.2:0/55841790 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v98) v1 ==== 103+0+11 (secure 0 0 0) 0x7fedd4108a00 con 0x7fedd4100610 2024-03-15T13:09:07.007 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T13:09:07.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.024+0000 7fedda04e640 1 -- 172.21.15.2:0/55841790 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fedbc0a1ba0 msgr2=0x7fedbc0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:07.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.024+0000 7fedda04e640 1 --2- 172.21.15.2:0/55841790 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fedbc0a1ba0 0x7fedbc0a4060 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fedd40ffc50 tx=0x7fedc8005f90 comp rx=0 tx=0).stop 2024-03-15T13:09:07.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.024+0000 7fedda04e640 1 -- 172.21.15.2:0/55841790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd4100610 msgr2=0x7fedd419d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:07.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.024+0000 7fedda04e640 1 --2- 172.21.15.2:0/55841790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd4100610 0x7fedd419d7e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fedc400e9f0 tx=0x7fedc400eeb0 comp rx=0 tx=0).stop 2024-03-15T13:09:07.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.024+0000 7fedda04e640 1 -- 172.21.15.2:0/55841790 shutdown_connections 2024-03-15T13:09:07.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.024+0000 7fedda04e640 1 --2- 172.21.15.2:0/55841790 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedd4197990 0x7fedd4197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:07.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.024+0000 7fedda04e640 1 --2- 172.21.15.2:0/55841790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedd41056e0 0x7fedd419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:07.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.024+0000 7fedda04e640 1 --2- 172.21.15.2:0/55841790 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fedbc0a1ba0 0x7fedbc0a4060 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:07.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.024+0000 7fedda04e640 1 --2- 172.21.15.2:0/55841790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd4100610 0x7fedd419d7e0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:07.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.024+0000 7fedda04e640 1 -- 172.21.15.2:0/55841790 >> 172.21.15.2:0/55841790 conn(0x7fedd4076770 msgr2=0x7fedd40fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:07.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.024+0000 7fedda04e640 1 -- 172.21.15.2:0/55841790 shutdown_connections 2024-03-15T13:09:07.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.024+0000 7fedda04e640 1 -- 172.21.15.2:0/55841790 wait complete. 2024-03-15T13:09:07.019 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-15T13:09:07.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:06 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1192206786' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T13:09:07.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:06 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3780449994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T13:09:07.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:06 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3473197463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T13:09:07.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.137+0000 7f07a2fe2640 1 -- 172.21.15.2:0/3622452028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f079c100fd0 msgr2=0x7f079c105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.137+0000 7f07a2fe2640 1 --2- 172.21.15.2:0/3622452028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f079c100fd0 0x7f079c105010 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f078c009930 tx=0x7f078c02bc30 comp rx=0 tx=0).stop 2024-03-15T13:09:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.137+0000 7f07a2fe2640 1 -- 172.21.15.2:0/3622452028 shutdown_connections 2024-03-15T13:09:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.137+0000 7f07a2fe2640 1 --2- 172.21.15.2:0/3622452028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f079c1056f0 0x7f079c10d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.137+0000 7f07a2fe2640 1 --2- 172.21.15.2:0/3622452028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f079c100fd0 0x7f079c105010 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.137+0000 7f07a2fe2640 1 --2- 172.21.15.2:0/3622452028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f079c100620 0x7f079c100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.137+0000 7f07a2fe2640 1 -- 172.21.15.2:0/3622452028 >> 172.21.15.2:0/3622452028 conn(0x7f079c077490 msgr2=0x7f079c0778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.137+0000 7f07a2fe2640 1 -- 172.21.15.2:0/3622452028 shutdown_connections 2024-03-15T13:09:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.137+0000 7f07a2fe2640 1 -- 172.21.15.2:0/3622452028 wait complete. 2024-03-15T13:09:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a2fe2640 1 Processor -- start 2024-03-15T13:09:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a2fe2640 1 -- start start 2024-03-15T13:09:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a2fe2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f079c100620 0x7f079c19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a2fe2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f079c1056f0 0x7f079c19ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a2fe2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f079c197a20 0x7f079c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a2fe2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c198410 con 0x7f079c197a20 2024-03-15T13:09:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a2fe2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c198580 con 0x7f079c1056f0 2024-03-15T13:09:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a2fe2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c198740 con 0x7f079c100620 2024-03-15T13:09:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a157f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f079c197a20 0x7f079c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a157f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f079c197a20 0x7f079c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58264/0 (socket says 172.21.15.2:58264) 2024-03-15T13:09:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a157f640 1 -- 172.21.15.2:0/519220489 learned_addr learned my addr 172.21.15.2:0/519220489 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f0793fff640 1 --2- 172.21.15.2:0/519220489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f079c1056f0 0x7f079c19ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a0d7e640 1 --2- 172.21.15.2:0/519220489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f079c100620 0x7f079c19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a157f640 1 -- 172.21.15.2:0/519220489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f079c100620 msgr2=0x7f079c19d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a157f640 1 --2- 172.21.15.2:0/519220489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f079c100620 0x7f079c19d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a157f640 1 -- 172.21.15.2:0/519220489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f079c1056f0 msgr2=0x7f079c19ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a157f640 1 --2- 172.21.15.2:0/519220489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f079c1056f0 0x7f079c19ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a157f640 1 -- 172.21.15.2:0/519220489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c009560 con 0x7f079c197a20 2024-03-15T13:09:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.138+0000 7f07a157f640 1 --2- 172.21.15.2:0/519220489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f079c197a20 0x7f079c197ed0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f079400eb10 tx=0x7f079400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:07.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.139+0000 7f0791ffb640 1 -- 172.21.15.2:0/519220489 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794004b20 con 0x7f079c197a20 2024-03-15T13:09:07.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.139+0000 7f0791ffb640 1 -- 172.21.15.2:0/519220489 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f0794004c80 con 0x7f079c197a20 2024-03-15T13:09:07.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.139+0000 7f07a2fe2640 1 -- 172.21.15.2:0/519220489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079c1a4730 con 0x7f079c197a20 2024-03-15T13:09:07.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.139+0000 7f0791ffb640 1 -- 172.21.15.2:0/519220489 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794010690 con 0x7f079c197a20 2024-03-15T13:09:07.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.139+0000 7f07a2fe2640 1 -- 172.21.15.2:0/519220489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c1a4bc0 con 0x7f079c197a20 2024-03-15T13:09:07.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.140+0000 7f07a2fe2640 1 -- 172.21.15.2:0/519220489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079c068730 con 0x7f079c197a20 2024-03-15T13:09:07.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.141+0000 7f0791ffb640 1 -- 172.21.15.2:0/519220489 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f0794004df0 con 0x7f079c197a20 2024-03-15T13:09:07.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.141+0000 7f0791ffb640 1 --2- 172.21.15.2:0/519220489 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f07840a1ba0 0x7f07840a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:07.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.141+0000 7f0791ffb640 1 -- 172.21.15.2:0/519220489 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f0794014070 con 0x7f079c197a20 2024-03-15T13:09:07.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.141+0000 7f07a0d7e640 1 --2- 172.21.15.2:0/519220489 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f07840a1ba0 0x7f07840a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:07.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.142+0000 7f07a0d7e640 1 --2- 172.21.15.2:0/519220489 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f07840a1ba0 0x7f07840a4060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f079c198b60 tx=0x7f0798005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:07.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.146+0000 7f0791ffb640 1 -- 172.21.15.2:0/519220489 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0794096c10 con 0x7f079c197a20 2024-03-15T13:09:07.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.376+0000 7f07a2fe2640 1 -- 172.21.15.2:0/519220489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f079c100fd0 con 0x7f079c197a20 2024-03-15T13:09:07.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.385+0000 7f0791ffb640 1 -- 172.21.15.2:0/519220489 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+734990 (secure 0 0 0) 0x7f079409a2d0 con 0x7f079c197a20 2024-03-15T13:09:07.379 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:09:07.379 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:09:07.379 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-03-15T13:09:07.379 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.379 INFO:teuthology.orchestra.run.smithi002.stdout: "active_gid": 15741, 2024-03-15T13:09:07.379 INFO:teuthology.orchestra.run.smithi002.stdout: "active_name": "x", 2024-03-15T13:09:07.379 INFO:teuthology.orchestra.run.smithi002.stdout: "active_addrs": { 2024-03-15T13:09:07.380 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-03-15T13:09:07.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.380 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-03-15T13:09:07.380 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-03-15T13:09:07.380 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1550147071 2024-03-15T13:09:07.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.380 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-03-15T13:09:07.380 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-03-15T13:09:07.380 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1550147071 2024-03-15T13:09:07.380 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.380 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-03-15T13:09:07.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.380 INFO:teuthology.orchestra.run.smithi002.stdout: "active_addr": "172.21.15.2:6801/1550147071", 2024-03-15T13:09:07.381 INFO:teuthology.orchestra.run.smithi002.stdout: "active_change": "2024-03-15T12:58:53.781334+0000", 2024-03-15T13:09:07.381 INFO:teuthology.orchestra.run.smithi002.stdout: "active_mgr_features": 4540701547738038271, 2024-03-15T13:09:07.381 INFO:teuthology.orchestra.run.smithi002.stdout: "available": true, 2024-03-15T13:09:07.381 INFO:teuthology.orchestra.run.smithi002.stdout: "standbys": [ 2024-03-15T13:09:07.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.381 INFO:teuthology.orchestra.run.smithi002.stdout: "gid": 25534, 2024-03-15T13:09:07.381 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "z", 2024-03-15T13:09:07.381 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_features": 4540701547738038271, 2024-03-15T13:09:07.381 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2024-03-15T13:09:07.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.381 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2024-03-15T13:09:07.381 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.381 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.382 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.382 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-03-15T13:09:07.382 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-03-15T13:09:07.382 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.382 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.382 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.382 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-03-15T13:09:07.382 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.382 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.382 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.382 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2024-03-15T13:09:07.382 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.383 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.383 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.383 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.383 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.383 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.383 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.383 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.383 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.383 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.383 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.383 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.383 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.383 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.384 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.384 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.384 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.384 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.384 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.384 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.384 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.384 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.384 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.384 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.384 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.385 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.385 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.385 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.385 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.385 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.385 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.385 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.385 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.385 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.385 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.385 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.386 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.386 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.386 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.386 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.386 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.386 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.386 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.386 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.386 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.386 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.386 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.387 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.387 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.387 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.387 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.387 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.387 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.387 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.387 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.387 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.387 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.387 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.388 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.388 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.388 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.388 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.388 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.388 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.388 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.388 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.388 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.388 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.388 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.388 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2024-03-15T13:09:07.388 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2024-03-15T13:09:07.388 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.389 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.389 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.389 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.389 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.389 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.389 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2024-03-15T13:09:07.389 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.389 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.389 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.389 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2024-03-15T13:09:07.389 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2024-03-15T13:09:07.389 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.389 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.390 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.390 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2024-03-15T13:09:07.390 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.390 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.390 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.390 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2024-03-15T13:09:07.390 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.390 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.390 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.390 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2024-03-15T13:09:07.390 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2024-03-15T13:09:07.390 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.390 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.391 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.391 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.391 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.391 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.391 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.391 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2024-03-15T13:09:07.391 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.391 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.391 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.391 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2024-03-15T13:09:07.391 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2024-03-15T13:09:07.391 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.391 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.391 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.392 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.392 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.392 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.392 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.392 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2024-03-15T13:09:07.392 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.392 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.392 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.392 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2024-03-15T13:09:07.392 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2024-03-15T13:09:07.392 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.392 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.392 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.393 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.394 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.394 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.394 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2024-03-15T13:09:07.394 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.394 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.394 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.394 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2024-03-15T13:09:07.394 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2024-03-15T13:09:07.394 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.394 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.394 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.394 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.395 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.395 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.395 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.395 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-15T13:09:07.395 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.395 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.395 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.395 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2024-03-15T13:09:07.395 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2024-03-15T13:09:07.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.395 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.395 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.395 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.395 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.396 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.396 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.396 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2024-03-15T13:09:07.396 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.396 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.396 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.396 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.396 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.396 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2024-03-15T13:09:07.396 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.396 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.396 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2024-03-15T13:09:07.397 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2024-03-15T13:09:07.398 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.398 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.398 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.398 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2024-03-15T13:09:07.398 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.398 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.398 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.398 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2024-03-15T13:09:07.398 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-15T13:09:07.398 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.398 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.398 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2024-03-15T13:09:07.398 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2024-03-15T13:09:07.399 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-03-15T13:09:07.399 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.399 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.399 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-03-15T13:09:07.399 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-03-15T13:09:07.399 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-03-15T13:09:07.399 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.399 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-15T13:09:07.399 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-15T13:09:07.399 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.399 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.399 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2024-03-15T13:09:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.401 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.402 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2024-03-15T13:09:07.402 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2024-03-15T13:09:07.402 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2024-03-15T13:09:07.402 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.402 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2024-03-15T13:09:07.402 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-15T13:09:07.402 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.402 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.402 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2024-03-15T13:09:07.402 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2024-03-15T13:09:07.402 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.402 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.402 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.402 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-03-15T13:09:07.403 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-03-15T13:09:07.404 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.404 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-15T13:09:07.404 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-15T13:09:07.404 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.404 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.404 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.404 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.404 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.404 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.404 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.404 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.405 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.405 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.405 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.405 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.405 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.405 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.405 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.405 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.405 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.405 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.406 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.406 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.406 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.406 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.406 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.406 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.406 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.406 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.406 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.407 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.407 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.407 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.407 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.407 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.407 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.407 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.407 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.407 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.407 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.407 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.407 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.408 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.408 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.408 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.408 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.408 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.408 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.408 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.408 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.408 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.408 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.408 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.408 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.408 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.408 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.409 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.409 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.409 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.409 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.409 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.409 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.409 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.409 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.409 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.409 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.409 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.409 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.409 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.409 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2024-03-15T13:09:07.410 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2024-03-15T13:09:07.410 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:07.410 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.410 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.410 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-03-15T13:09:07.410 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.410 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.410 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.410 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-15T13:09:07.410 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.410 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.410 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: "read", 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap", 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap-read" 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.411 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.412 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.413 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2024-03-15T13:09:07.414 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-03-15T13:09:07.415 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.415 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.415 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:07.415 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.415 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.415 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.415 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-15T13:09:07.415 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.415 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.415 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.415 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.415 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.416 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.418 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2024-03-15T13:09:07.418 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2024-03-15T13:09:07.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.418 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.418 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.418 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2024-03-15T13:09:07.418 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.418 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.418 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.418 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-15T13:09:07.418 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.418 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.418 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.419 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2024-03-15T13:09:07.419 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2024-03-15T13:09:07.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.419 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.419 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.419 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.419 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.419 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.419 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.419 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-15T13:09:07.419 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-15T13:09:07.419 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.419 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.419 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2024-03-15T13:09:07.420 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm_log_destination": { 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm_log_destination", 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.421 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.422 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.422 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.422 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.422 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.422 INFO:teuthology.orchestra.run.smithi002.stdout: "file", 2024-03-15T13:09:07.422 INFO:teuthology.orchestra.run.smithi002.stdout: "file,syslog", 2024-03-15T13:09:07.422 INFO:teuthology.orchestra.run.smithi002.stdout: "syslog" 2024-03-15T13:09:07.422 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.422 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-15T13:09:07.422 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.422 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.422 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.422 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2024-03-15T13:09:07.422 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.423 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.424 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.425 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-03-15T13:09:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2024-03-15T13:09:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2024-03-15T13:09:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-15T13:09:07.426 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.427 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-03-15T13:09:07.428 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2024-03-15T13:09:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2024-03-15T13:09:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-15T13:09:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2024-03-15T13:09:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.429 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2024-03-15T13:09:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2024-03-15T13:09:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-15T13:09:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2024-03-15T13:09:07.430 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2024-03-15T13:09:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2024-03-15T13:09:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-15T13:09:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2024-03-15T13:09:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.432 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2024-03-15T13:09:07.433 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2024-03-15T13:09:07.433 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.433 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.433 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.433 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-15T13:09:07.433 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.433 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.433 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.433 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2024-03-15T13:09:07.433 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.433 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.433 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.433 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2024-03-15T13:09:07.433 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2024-03-15T13:09:07.434 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.434 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.434 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.434 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-15T13:09:07.434 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.434 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.434 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.434 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2024-03-15T13:09:07.434 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.434 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.434 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.434 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2024-03-15T13:09:07.434 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2024-03-15T13:09:07.434 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.435 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.435 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.435 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-15T13:09:07.435 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.435 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.435 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.435 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-03-15T13:09:07.435 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.435 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.435 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.435 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_nvmeof": { 2024-03-15T13:09:07.435 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_nvmeof", 2024-03-15T13:09:07.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.435 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Nvme-of container image", 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-15T13:09:07.436 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.437 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2024-03-15T13:09:07.438 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.440 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2024-03-15T13:09:07.440 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2024-03-15T13:09:07.440 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.440 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.440 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.440 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-03-15T13:09:07.440 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.440 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.440 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.440 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-15T13:09:07.440 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.440 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.440 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.441 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2024-03-15T13:09:07.441 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2024-03-15T13:09:07.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.441 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.441 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.441 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2024-03-15T13:09:07.441 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.441 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.441 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.441 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-15T13:09:07.441 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.441 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.441 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.441 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2024-03-15T13:09:07.442 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2024-03-15T13:09:07.442 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.442 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.442 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.442 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2024-03-15T13:09:07.442 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.442 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.442 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.442 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-15T13:09:07.442 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.442 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.442 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2024-03-15T13:09:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.444 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-03-15T13:09:07.445 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: "hw_monitoring": { 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hw_monitoring", 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.446 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: "inventory_list_all": { 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "inventory_list_all", 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-15T13:09:07.447 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.448 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.448 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.448 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.448 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.448 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.448 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.448 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.448 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.448 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.448 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.448 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.449 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.449 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.449 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.449 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.449 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.449 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.449 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.449 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.449 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.449 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2024-03-15T13:09:07.449 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2024-03-15T13:09:07.449 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.449 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.449 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.450 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.451 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.451 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.451 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-15T13:09:07.451 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.451 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.451 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.451 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.451 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.451 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.451 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.451 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.451 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.451 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.453 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.454 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.454 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.454 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-15T13:09:07.454 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.454 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.454 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.454 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-15T13:09:07.454 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-15T13:09:07.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.454 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.454 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.454 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2024-03-15T13:09:07.454 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.454 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.455 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.455 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-15T13:09:07.455 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.455 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.455 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.455 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2024-03-15T13:09:07.455 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2024-03-15T13:09:07.455 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.455 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.455 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.455 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:07.455 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.456 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.456 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.456 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2024-03-15T13:09:07.456 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.456 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.456 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.456 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2024-03-15T13:09:07.456 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2024-03-15T13:09:07.456 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.456 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.456 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.456 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:07.456 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.456 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-15T13:09:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2024-03-15T13:09:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2024-03-15T13:09:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.458 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2024-03-15T13:09:07.458 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.458 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.458 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.458 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-03-15T13:09:07.458 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-03-15T13:09:07.458 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.458 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.458 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.458 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2024-03-15T13:09:07.458 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.458 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.458 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.458 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: "oob_default_addr": { 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "oob_default_addr", 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "169.254.1.1", 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.459 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.461 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2024-03-15T13:09:07.462 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.463 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.464 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.465 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.465 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-15T13:09:07.465 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.465 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.465 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.465 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2024-03-15T13:09:07.465 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2024-03-15T13:09:07.465 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.465 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.465 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.465 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2024-03-15T13:09:07.465 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.465 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.465 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.466 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2024-03-15T13:09:07.466 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.466 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.466 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.466 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2024-03-15T13:09:07.466 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2024-03-15T13:09:07.466 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.466 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.466 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.466 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.466 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.466 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.466 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.466 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-15T13:09:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2024-03-15T13:09:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2024-03-15T13:09:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-15T13:09:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.468 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.468 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2024-03-15T13:09:07.468 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2024-03-15T13:09:07.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.468 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.468 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.468 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.468 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.468 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.468 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-15T13:09:07.468 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.470 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2024-03-15T13:09:07.470 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2024-03-15T13:09:07.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.470 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.470 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.470 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.470 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.470 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.470 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.470 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-15T13:09:07.470 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.470 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.470 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.470 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2024-03-15T13:09:07.471 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2024-03-15T13:09:07.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.471 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.471 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.471 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.471 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.471 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.471 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.471 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-15T13:09:07.471 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.471 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.471 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.471 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.471 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.472 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.473 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.474 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.474 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.474 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.474 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.474 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.474 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.474 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.474 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.474 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.474 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.474 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.476 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.476 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.476 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.476 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.476 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.476 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.476 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.476 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.476 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.476 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.476 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.476 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2024-03-15T13:09:07.477 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2024-03-15T13:09:07.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.478 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.478 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.478 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2024-03-15T13:09:07.478 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.478 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.478 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.478 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-15T13:09:07.478 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.478 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.478 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.479 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2024-03-15T13:09:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-15T13:09:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.480 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.481 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-15T13:09:07.481 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-15T13:09:07.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.481 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.481 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.481 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.481 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.481 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.481 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.481 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.481 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.481 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.482 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2024-03-15T13:09:07.482 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2024-03-15T13:09:07.482 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.482 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.482 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.482 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.482 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.482 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.482 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.482 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.482 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.482 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.482 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.482 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-15T13:09:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.484 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.484 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.484 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.484 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.484 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.484 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.484 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.484 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.484 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.484 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.484 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-15T13:09:07.484 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-15T13:09:07.484 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.484 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-15T13:09:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-15T13:09:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.486 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.486 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.486 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.486 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.486 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.486 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.486 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.486 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.486 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-15T13:09:07.486 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-15T13:09:07.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.486 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.487 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-15T13:09:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-15T13:09:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.488 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.489 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.489 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.489 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.489 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.489 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.489 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.489 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-15T13:09:07.489 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-15T13:09:07.489 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.489 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.489 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.489 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.489 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.489 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.490 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.490 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.490 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.490 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.490 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.490 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-15T13:09:07.490 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-15T13:09:07.490 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.490 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.490 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.490 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.490 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.490 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-15T13:09:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-15T13:09:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.492 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.492 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.492 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.492 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2024-03-15T13:09:07.492 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-15T13:09:07.492 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.492 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.492 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.492 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-03-15T13:09:07.492 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.492 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.492 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.492 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.493 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.494 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.494 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2024-03-15T13:09:07.494 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2024-03-15T13:09:07.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.494 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.494 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.494 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.494 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.494 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.494 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.494 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.494 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.494 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.495 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.495 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2024-03-15T13:09:07.495 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-15T13:09:07.495 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.495 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.495 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.495 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-03-15T13:09:07.495 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.495 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.495 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.495 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.495 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.495 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.497 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-15T13:09:07.497 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-15T13:09:07.497 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.497 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.497 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.497 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.497 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.497 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.497 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.497 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.497 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.497 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.497 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-15T13:09:07.498 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-15T13:09:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2024-03-15T13:09:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-15T13:09:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.500 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.500 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.500 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.500 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.500 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.500 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.500 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.500 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.500 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.500 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.500 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-15T13:09:07.500 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-15T13:09:07.500 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.501 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.502 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.503 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.503 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.503 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.503 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.503 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.503 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-15T13:09:07.503 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-15T13:09:07.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.503 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.503 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.503 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.503 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.503 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.504 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.504 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.504 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.504 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.504 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.504 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-15T13:09:07.504 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-15T13:09:07.504 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.504 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.504 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.504 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.504 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.504 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.505 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-15T13:09:07.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.510 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:07.510 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.510 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.510 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.510 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.510 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.510 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.510 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.510 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-15T13:09:07.510 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-15T13:09:07.510 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.510 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.511 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.511 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2024-03-15T13:09:07.511 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.511 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.511 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.511 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.511 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.511 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.511 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.511 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-15T13:09:07.511 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-15T13:09:07.511 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.511 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.511 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.512 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.513 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.514 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.514 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.514 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.514 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2024-03-15T13:09:07.514 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-15T13:09:07.514 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.514 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.514 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.514 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.514 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.514 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.514 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.514 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.514 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-15T13:09:07.516 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.518 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-03-15T13:09:07.518 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.518 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.518 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.518 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.518 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.518 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.518 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.518 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2024-03-15T13:09:07.518 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2024-03-15T13:09:07.518 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.518 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.518 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.518 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.519 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.519 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.519 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.519 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.519 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.519 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.519 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.519 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2024-03-15T13:09:07.519 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2024-03-15T13:09:07.519 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.519 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.519 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.519 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.519 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.520 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.521 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.522 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-03-15T13:09:07.522 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-03-15T13:09:07.522 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.522 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.522 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.522 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.522 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.522 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.522 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.522 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.522 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.522 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.524 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.524 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.524 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.524 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.524 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.524 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.524 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.524 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.524 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.524 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.524 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.524 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.524 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.525 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.526 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.526 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.526 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.526 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.526 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.526 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.526 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.526 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.526 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.526 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.526 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.526 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.526 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.526 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.527 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.527 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.527 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.527 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.527 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.527 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.527 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.527 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.527 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.527 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2024-03-15T13:09:07.527 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2024-03-15T13:09:07.527 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.527 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.528 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.528 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.528 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.528 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.528 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.528 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2024-03-15T13:09:07.528 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.528 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.528 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.528 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2024-03-15T13:09:07.528 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2024-03-15T13:09:07.528 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.528 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.528 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.529 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.529 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.529 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.529 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.529 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.529 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.529 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.529 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.529 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-03-15T13:09:07.529 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-03-15T13:09:07.529 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.529 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.529 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-03-15T13:09:07.530 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.530 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.530 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.530 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.530 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.530 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.530 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.530 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-03-15T13:09:07.530 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-03-15T13:09:07.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.530 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.530 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.530 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2024-03-15T13:09:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-03-15T13:09:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-03-15T13:09:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.532 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.532 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.532 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.532 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.532 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.532 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.532 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2024-03-15T13:09:07.532 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2024-03-15T13:09:07.532 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.532 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.532 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.532 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2024-03-15T13:09:07.532 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.533 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.533 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.533 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.533 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.533 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.533 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.533 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-03-15T13:09:07.533 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-03-15T13:09:07.533 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.533 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.533 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.533 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2024-03-15T13:09:07.534 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.534 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.534 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.534 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.534 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2024-03-15T13:09:07.534 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.534 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.534 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.534 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.534 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.534 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-03-15T13:09:07.534 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-03-15T13:09:07.534 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.535 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.535 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.535 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-03-15T13:09:07.535 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-03-15T13:09:07.535 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-03-15T13:09:07.535 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.535 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.535 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.535 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.535 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.535 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2024-03-15T13:09:07.535 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2024-03-15T13:09:07.535 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.535 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.536 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.536 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.536 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.536 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.536 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.536 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.536 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.536 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.536 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.536 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-03-15T13:09:07.536 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.537 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.537 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.537 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2024-03-15T13:09:07.537 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2024-03-15T13:09:07.537 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.537 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.537 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.537 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.537 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.537 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.537 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2024-03-15T13:09:07.537 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.537 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.537 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.538 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.538 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.538 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.538 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.538 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.538 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.538 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.538 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.538 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.538 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.538 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.538 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.538 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.539 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.539 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.539 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.539 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.539 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.539 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.539 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.539 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.539 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.539 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.539 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.539 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.539 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.541 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.543 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.544 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.544 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-15T13:09:07.544 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.544 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.544 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.544 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2024-03-15T13:09:07.544 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2024-03-15T13:09:07.544 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.544 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.544 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.544 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2024-03-15T13:09:07.544 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.544 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.545 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.545 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2024-03-15T13:09:07.545 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.545 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.545 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.545 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2024-03-15T13:09:07.545 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2024-03-15T13:09:07.545 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.545 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.545 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.545 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-03-15T13:09:07.545 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.545 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.545 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.546 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-15T13:09:07.546 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.546 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.546 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.546 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2024-03-15T13:09:07.546 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2024-03-15T13:09:07.546 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.546 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.546 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.546 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.546 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.546 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.547 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2024-03-15T13:09:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2024-03-15T13:09:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2024-03-15T13:09:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2024-03-15T13:09:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-15T13:09:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.549 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.550 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.551 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.551 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.551 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.551 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.551 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.551 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.551 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.551 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.551 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.551 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.551 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.551 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.552 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.552 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.552 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.552 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.552 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.552 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.552 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.552 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.552 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.552 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.552 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.552 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.552 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.553 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-03-15T13:09:07.555 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.557 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2024-03-15T13:09:07.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.559 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.560 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2024-03-15T13:09:07.560 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2024-03-15T13:09:07.560 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.560 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.560 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-15T13:09:07.560 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.560 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.560 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.560 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.560 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.560 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.560 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.560 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.560 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.561 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.562 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.563 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.564 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.564 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.564 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.564 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.564 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.564 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.564 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.564 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.564 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.564 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.564 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.564 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.565 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2024-03-15T13:09:07.565 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2024-03-15T13:09:07.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.565 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.565 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.565 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.565 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.565 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.565 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.565 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2024-03-15T13:09:07.565 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.565 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.565 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.565 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2024-03-15T13:09:07.566 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2024-03-15T13:09:07.566 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.566 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.566 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.566 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2024-03-15T13:09:07.566 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.566 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.566 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.566 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2024-03-15T13:09:07.566 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.566 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.566 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.566 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2024-03-15T13:09:07.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.568 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.569 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.569 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.569 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.569 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.569 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2024-03-15T13:09:07.569 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.569 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.569 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.569 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2024-03-15T13:09:07.569 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2024-03-15T13:09:07.569 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.569 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.569 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.570 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2024-03-15T13:09:07.570 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.570 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.570 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.570 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-15T13:09:07.570 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.570 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.570 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.570 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2024-03-15T13:09:07.570 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.571 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.571 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.571 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.571 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.571 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.571 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.571 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.571 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.571 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.571 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.571 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.571 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.571 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.571 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.572 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.572 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.572 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.572 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.572 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.572 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.572 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.572 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.572 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.572 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.572 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.573 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.573 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.573 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.573 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.573 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.573 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.573 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.573 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.573 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.573 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.573 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.573 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.573 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.573 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.574 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.574 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.574 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.574 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.574 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.574 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.574 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.574 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.574 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.574 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.574 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.574 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.574 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.574 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.574 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.576 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2024-03-15T13:09:07.576 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.576 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.576 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.576 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.576 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.576 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.576 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.576 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.576 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.577 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.577 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.577 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.577 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.577 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.577 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.577 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.577 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.577 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.577 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.577 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.577 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.577 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.577 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.578 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.578 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.578 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.578 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.578 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.578 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.578 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.578 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.578 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.578 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.578 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.578 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.579 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.580 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.581 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.582 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-15T13:09:07.582 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.582 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.582 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.582 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2024-03-15T13:09:07.582 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2024-03-15T13:09:07.582 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.582 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.582 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.582 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:07.582 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2024-03-15T13:09:07.582 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.582 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.582 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.583 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.584 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.586 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.587 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.587 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.587 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.587 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.587 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.587 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.587 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.587 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.587 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2024-03-15T13:09:07.587 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.587 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.588 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.589 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2024-03-15T13:09:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2024-03-15T13:09:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2024-03-15T13:09:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2024-03-15T13:09:07.596 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2024-03-15T13:09:07.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.596 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.596 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.596 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.596 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.596 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.596 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.596 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2024-03-15T13:09:07.596 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.596 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.596 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.596 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2024-03-15T13:09:07.596 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2024-03-15T13:09:07.597 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.598 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.599 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.600 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.601 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.602 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.603 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.604 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.605 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.605 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.605 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.605 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.605 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.605 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.605 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.605 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.605 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.605 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.605 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.606 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.607 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.607 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.607 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.607 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.607 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.607 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.607 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2024-03-15T13:09:07.607 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.607 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.607 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.608 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.609 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.610 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.612 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.613 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.614 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.614 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.614 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.614 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.614 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.614 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.614 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.614 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.614 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.614 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.614 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.614 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.614 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.614 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.615 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-03-15T13:09:07.617 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.619 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.620 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.621 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.622 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.622 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.622 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.622 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.622 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.622 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.622 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.622 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.622 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.622 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.622 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.622 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.622 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.622 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.626 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.627 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.628 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.629 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.630 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-03-15T13:09:07.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.633 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.634 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.635 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.636 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.637 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.639 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.640 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-03-15T13:09:07.641 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2024-03-15T13:09:07.642 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.643 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.644 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.645 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.646 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.647 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.647 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.647 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.647 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.647 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.647 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.647 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.647 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.647 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.647 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.647 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.647 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.647 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.647 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.647 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.648 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.648 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.648 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.648 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.648 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.648 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.648 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.648 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.648 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.648 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.649 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2024-03-15T13:09:07.649 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2024-03-15T13:09:07.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.649 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.649 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.649 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.649 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.649 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.649 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.649 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.649 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.649 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.649 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.650 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-15T13:09:07.650 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-15T13:09:07.650 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.650 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.650 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.650 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2024-03-15T13:09:07.650 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.650 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.650 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.650 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.650 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.650 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.650 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-03-15T13:09:07.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.652 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.653 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.654 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2024-03-15T13:09:07.655 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2024-03-15T13:09:07.655 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.655 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.655 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.655 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.655 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.655 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-03-15T13:09:07.655 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-03-15T13:09:07.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.655 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.655 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-03-15T13:09:07.655 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-03-15T13:09:07.656 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-03-15T13:09:07.656 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.656 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.656 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.656 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.656 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.656 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-03-15T13:09:07.656 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.656 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.656 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.656 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.657 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.659 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.660 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.660 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.660 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.660 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.660 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.660 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.660 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.660 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.660 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.660 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.660 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.660 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.661 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.661 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.661 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.661 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2024-03-15T13:09:07.661 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2024-03-15T13:09:07.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.661 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.661 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:07.661 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.661 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.661 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.661 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.661 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.662 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.662 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.662 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2024-03-15T13:09:07.662 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2024-03-15T13:09:07.662 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.662 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.662 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.662 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.662 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.662 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.662 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.662 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.662 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.662 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.664 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.666 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.667 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.667 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.667 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.667 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.667 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.667 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.667 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.667 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.667 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.667 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.667 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.667 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.667 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.668 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.668 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.668 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.668 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.668 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.668 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.668 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.668 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.668 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.668 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.668 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.668 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.668 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.668 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.669 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.670 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.671 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.672 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.674 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.675 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.676 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.676 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.676 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.676 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.676 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.676 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.676 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.676 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.676 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.676 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.676 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.677 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2024-03-15T13:09:07.677 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.677 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.677 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.677 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.677 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.677 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.677 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.677 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.677 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.677 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.677 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.677 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.678 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.679 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.680 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.682 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.683 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.685 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.686 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.687 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2024-03-15T13:09:07.688 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2024-03-15T13:09:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2024-03-15T13:09:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2024-03-15T13:09:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.690 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.691 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.692 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.693 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2024-03-15T13:09:07.694 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.696 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.696 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.696 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.696 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.696 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.696 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.696 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.696 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.696 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2024-03-15T13:09:07.696 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2024-03-15T13:09:07.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.696 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2024-03-15T13:09:07.698 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2024-03-15T13:09:07.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.699 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.699 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.699 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.699 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.699 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.699 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2024-03-15T13:09:07.699 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.699 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.699 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.699 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.699 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.700 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.700 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.700 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.700 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.700 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.700 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.700 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.700 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.700 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.700 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.700 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.700 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.700 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.700 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.701 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.701 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.701 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.701 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.701 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.701 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.701 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.701 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.701 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.701 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.701 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.701 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.702 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.702 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.702 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.702 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.702 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.702 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.702 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.702 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.702 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.702 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.702 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.702 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.702 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.703 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.706 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.706 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.706 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.706 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.706 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.706 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.706 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.706 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.706 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.706 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.706 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.706 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.706 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.707 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.708 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.708 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.708 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.708 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.708 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.708 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.708 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.708 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.708 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.708 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.708 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.708 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.708 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.708 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.710 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.711 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.713 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2024-03-15T13:09:07.714 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.714 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.714 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.714 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2024-03-15T13:09:07.714 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2024-03-15T13:09:07.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.714 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.714 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.714 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-15T13:09:07.714 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.714 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.714 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.714 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.714 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.714 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.716 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.717 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.718 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.718 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.718 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.718 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.718 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.718 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.718 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.718 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.718 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.718 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.718 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.719 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.719 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.719 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.719 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.719 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.719 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.719 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.719 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.719 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.719 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.719 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.719 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.719 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.720 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.720 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.720 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.720 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.720 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.720 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.720 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.720 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.720 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.720 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2024-03-15T13:09:07.720 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.721 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2024-03-15T13:09:07.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.723 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.724 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.725 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.726 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.727 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.728 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.730 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.731 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.732 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.732 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.732 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.732 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.732 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.732 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.732 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.732 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.732 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.732 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.732 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.732 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.732 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.733 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.736 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2024-03-15T13:09:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2024-03-15T13:09:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.738 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.739 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.739 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.739 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.739 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.739 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.739 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.739 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.739 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2024-03-15T13:09:07.739 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.739 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.739 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.739 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.740 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.741 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.741 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.741 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.741 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.741 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.741 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.741 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.741 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.741 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.741 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.741 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.741 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.741 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.741 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.742 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.742 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.742 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.742 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.742 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.742 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.742 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.742 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.742 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.742 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.742 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.742 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.745 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.745 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.745 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.745 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.745 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.745 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.745 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.745 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.745 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.745 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.745 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.745 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.746 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.746 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.746 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.746 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.746 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.746 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.746 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.746 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.746 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.746 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.746 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.746 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.747 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.748 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.748 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.748 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.748 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.748 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.748 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.748 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.748 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.748 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.748 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.748 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.748 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.748 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.749 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: "periodic_async_work": { 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "periodic_async_work", 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.750 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Periodically check for async work", 2024-03-15T13:09:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2024-03-15T13:09:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2024-03-15T13:09:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-03-15T13:09:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.752 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-15T13:09:07.752 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.752 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.752 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.752 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2024-03-15T13:09:07.752 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.752 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.752 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.752 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2024-03-15T13:09:07.752 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2024-03-15T13:09:07.753 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-03-15T13:09:07.753 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.753 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.753 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2024-03-15T13:09:07.753 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.753 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.753 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.753 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.753 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.753 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.753 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.753 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.754 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.755 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.756 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.757 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.757 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.757 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.757 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.757 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.757 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.757 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.757 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.757 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.757 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.757 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.757 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.757 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.757 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.758 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.758 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.758 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.758 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.758 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.758 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.758 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.758 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.758 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.758 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.758 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.758 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.758 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.758 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.759 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2024-03-15T13:09:07.760 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.761 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: "gid": 25613, 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "y", 2024-03-15T13:09:07.762 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_features": 4540701547738038271, 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2024-03-15T13:09:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.764 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.765 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.766 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.767 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.768 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.769 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.770 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.771 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-15T13:09:07.773 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.774 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.775 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-15T13:09:07.776 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.777 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2024-03-15T13:09:07.778 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2024-03-15T13:09:07.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-03-15T13:09:07.778 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.778 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.778 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2024-03-15T13:09:07.778 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-03-15T13:09:07.778 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2024-03-15T13:09:07.778 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.778 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-15T13:09:07.778 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.778 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.778 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.778 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2024-03-15T13:09:07.778 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.779 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.780 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-15T13:09:07.781 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.782 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.783 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.784 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.785 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.786 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.787 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-03-15T13:09:07.787 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-03-15T13:09:07.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.787 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.787 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2024-03-15T13:09:07.787 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.787 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.787 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.787 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2024-03-15T13:09:07.787 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2024-03-15T13:09:07.787 INFO:teuthology.orchestra.run.smithi002.stdout: "read", 2024-03-15T13:09:07.787 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap", 2024-03-15T13:09:07.787 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap-read" 2024-03-15T13:09:07.788 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.788 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2024-03-15T13:09:07.788 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.788 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.788 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.788 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2024-03-15T13:09:07.788 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2024-03-15T13:09:07.788 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.788 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.788 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.788 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.788 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.788 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.789 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-15T13:09:07.790 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.791 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.791 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2024-03-15T13:09:07.791 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2024-03-15T13:09:07.791 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-03-15T13:09:07.791 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.791 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.791 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:07.791 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.791 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.791 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.791 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-15T13:09:07.791 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.791 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.791 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.792 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.793 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.793 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-15T13:09:07.793 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.793 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.793 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.793 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2024-03-15T13:09:07.793 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2024-03-15T13:09:07.793 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.793 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.793 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.793 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2024-03-15T13:09:07.793 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.793 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.793 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.794 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-15T13:09:07.794 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.794 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.794 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.794 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2024-03-15T13:09:07.794 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2024-03-15T13:09:07.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.794 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.794 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2024-03-15T13:09:07.794 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.794 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.794 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.794 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-15T13:09:07.795 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.795 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.795 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.795 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2024-03-15T13:09:07.795 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2024-03-15T13:09:07.795 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.795 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.795 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.795 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.795 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.795 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.795 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.795 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-15T13:09:07.795 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-15T13:09:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2024-03-15T13:09:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2024-03-15T13:09:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-03-15T13:09:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-15T13:09:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.797 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.797 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2024-03-15T13:09:07.797 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2024-03-15T13:09:07.797 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:07.797 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.797 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.797 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2024-03-15T13:09:07.797 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.797 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.797 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.797 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-15T13:09:07.797 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.797 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.797 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.798 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm_log_destination": { 2024-03-15T13:09:07.798 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm_log_destination", 2024-03-15T13:09:07.798 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.798 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.798 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.798 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.798 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.798 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.798 INFO:teuthology.orchestra.run.smithi002.stdout: "file", 2024-03-15T13:09:07.798 INFO:teuthology.orchestra.run.smithi002.stdout: "file,syslog", 2024-03-15T13:09:07.798 INFO:teuthology.orchestra.run.smithi002.stdout: "syslog" 2024-03-15T13:09:07.798 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.801 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2024-03-15T13:09:07.801 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2024-03-15T13:09:07.801 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.801 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.801 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.801 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.801 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.801 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.801 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.801 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-15T13:09:07.801 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.801 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.801 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.801 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.802 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-15T13:09:07.803 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-03-15T13:09:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.805 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.805 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.805 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2024-03-15T13:09:07.805 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2024-03-15T13:09:07.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.805 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.805 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-15T13:09:07.805 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.805 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.805 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.805 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2024-03-15T13:09:07.805 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.805 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.806 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.806 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2024-03-15T13:09:07.806 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2024-03-15T13:09:07.806 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.806 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.806 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.806 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-15T13:09:07.806 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.806 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.806 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.806 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2024-03-15T13:09:07.806 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.806 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.806 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2024-03-15T13:09:07.807 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2024-03-15T13:09:07.808 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.808 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.808 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.808 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-15T13:09:07.808 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.808 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.808 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.808 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2024-03-15T13:09:07.808 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.808 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.808 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.808 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2024-03-15T13:09:07.808 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2024-03-15T13:09:07.808 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.809 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.809 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-15T13:09:07.809 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.809 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.809 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.809 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2024-03-15T13:09:07.809 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.809 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.809 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.809 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2024-03-15T13:09:07.809 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2024-03-15T13:09:07.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.809 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.810 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.810 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-15T13:09:07.810 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.810 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.810 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.810 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2024-03-15T13:09:07.810 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.810 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.810 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.810 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2024-03-15T13:09:07.810 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2024-03-15T13:09:07.810 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.810 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.810 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_nvmeof": { 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_nvmeof", 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-15T13:09:07.811 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Nvme-of container image", 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.812 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.813 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.814 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.814 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2024-03-15T13:09:07.814 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2024-03-15T13:09:07.814 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.814 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.814 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.814 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-15T13:09:07.814 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.814 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.814 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.814 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2024-03-15T13:09:07.814 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.814 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2024-03-15T13:09:07.816 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.817 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2024-03-15T13:09:07.818 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.818 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.818 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.818 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-15T13:09:07.818 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.818 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.818 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.818 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2024-03-15T13:09:07.818 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2024-03-15T13:09:07.818 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.818 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.818 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.818 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2024-03-15T13:09:07.818 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.820 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: "hw_monitoring": { 2024-03-15T13:09:07.821 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hw_monitoring", 2024-03-15T13:09:07.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.822 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.822 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.822 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.822 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.822 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.822 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.822 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-15T13:09:07.822 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.822 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.822 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.822 INFO:teuthology.orchestra.run.smithi002.stdout: "inventory_list_all": { 2024-03-15T13:09:07.822 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "inventory_list_all", 2024-03-15T13:09:07.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.823 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.825 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.826 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.827 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.827 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.827 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.827 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.827 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.827 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.827 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.827 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.827 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.827 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.827 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.827 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.827 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.827 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-15T13:09:07.828 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2024-03-15T13:09:07.829 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.830 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.830 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.830 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-15T13:09:07.830 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.830 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.830 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.830 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2024-03-15T13:09:07.830 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2024-03-15T13:09:07.830 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.830 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.830 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.830 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:07.830 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.830 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-15T13:09:07.831 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.833 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.834 INFO:teuthology.orchestra.run.smithi002.stdout: "oob_default_addr": { 2024-03-15T13:09:07.834 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "oob_default_addr", 2024-03-15T13:09:07.834 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.834 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.834 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.834 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "169.254.1.1", 2024-03-15T13:09:07.834 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.834 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.834 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.834 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-15T13:09:07.834 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.834 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.834 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.835 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2024-03-15T13:09:07.835 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2024-03-15T13:09:07.835 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.835 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.835 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.835 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-15T13:09:07.835 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.835 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.835 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.835 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-15T13:09:07.835 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.835 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.835 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.835 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.836 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.837 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.838 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.839 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.840 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2024-03-15T13:09:07.840 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2024-03-15T13:09:07.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.840 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.840 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2024-03-15T13:09:07.840 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.840 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.840 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.840 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2024-03-15T13:09:07.840 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.840 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.840 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.841 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2024-03-15T13:09:07.841 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2024-03-15T13:09:07.841 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.841 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.841 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.841 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.841 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.841 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.841 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.841 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-15T13:09:07.841 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.841 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.841 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2024-03-15T13:09:07.842 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2024-03-15T13:09:07.843 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.843 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.843 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.843 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.843 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.843 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.843 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.843 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-15T13:09:07.843 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.843 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.843 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.843 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2024-03-15T13:09:07.843 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2024-03-15T13:09:07.843 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.844 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.845 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.845 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.845 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.845 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.845 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-15T13:09:07.845 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.845 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.845 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.845 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2024-03-15T13:09:07.845 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2024-03-15T13:09:07.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.845 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.845 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.846 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.846 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.846 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.846 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.846 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-15T13:09:07.846 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.846 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.846 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.846 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.846 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.846 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2024-03-15T13:09:07.846 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.846 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.847 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.848 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.850 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.851 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.851 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.851 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.851 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.851 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.851 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2024-03-15T13:09:07.851 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2024-03-15T13:09:07.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.851 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.851 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2024-03-15T13:09:07.851 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.852 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.852 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.852 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-15T13:09:07.852 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.852 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.852 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.852 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2024-03-15T13:09:07.852 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2024-03-15T13:09:07.852 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.852 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.852 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.852 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2024-03-15T13:09:07.852 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.853 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.853 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.853 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-15T13:09:07.853 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.853 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.853 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.853 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2024-03-15T13:09:07.853 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.853 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.853 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.853 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-15T13:09:07.854 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-15T13:09:07.854 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.854 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.854 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.854 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:07.854 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.854 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.854 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.854 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.854 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.854 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.854 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.855 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2024-03-15T13:09:07.855 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-15T13:09:07.855 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.855 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.855 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.855 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.855 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.855 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.855 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.855 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.855 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.855 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.855 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.856 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-15T13:09:07.856 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-15T13:09:07.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.856 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.856 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.856 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.856 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.856 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.856 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.856 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.856 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.856 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.856 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.857 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2024-03-15T13:09:07.857 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2024-03-15T13:09:07.857 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.857 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.857 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.857 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.857 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.857 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.857 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.857 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.857 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.857 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.857 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.858 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-15T13:09:07.858 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-15T13:09:07.858 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.858 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.858 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.858 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.858 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.858 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.858 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.858 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.858 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.858 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.858 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.859 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2024-03-15T13:09:07.859 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-15T13:09:07.859 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.859 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.859 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.859 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.859 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.859 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.859 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.859 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.859 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.859 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.859 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.859 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-15T13:09:07.860 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-15T13:09:07.860 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.860 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.860 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.860 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.860 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.860 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.860 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.860 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.860 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.860 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.860 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.860 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-15T13:09:07.860 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.861 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.862 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.863 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.864 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2024-03-15T13:09:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.867 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.868 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.868 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.868 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.868 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.868 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.868 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.868 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.868 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.868 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.868 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2024-03-15T13:09:07.868 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2024-03-15T13:09:07.868 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.868 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.868 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-03-15T13:09:07.869 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.870 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.870 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.870 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.870 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.870 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.870 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.870 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-15T13:09:07.870 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-15T13:09:07.870 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.870 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.870 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.870 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.870 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.870 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.871 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.872 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2024-03-15T13:09:07.873 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.874 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.875 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.876 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.876 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.876 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.876 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.876 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.876 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.876 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.876 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-15T13:09:07.876 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-15T13:09:07.876 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.876 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.876 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.876 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.876 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.877 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.878 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.878 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.878 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.878 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-15T13:09:07.878 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-15T13:09:07.878 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.878 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.878 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.878 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.878 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.878 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.878 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.878 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.878 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.879 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.880 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.881 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.882 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.882 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.882 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.882 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.882 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.882 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.882 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.882 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.882 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.882 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-15T13:09:07.882 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-15T13:09:07.882 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.882 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.883 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.883 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-15T13:09:07.883 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.883 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.883 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.883 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.883 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.883 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.883 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.883 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-15T13:09:07.883 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-15T13:09:07.883 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.883 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.884 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.884 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:07.884 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.884 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.884 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.884 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.884 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.884 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.884 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.884 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-15T13:09:07.884 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-15T13:09:07.884 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.884 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.885 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.886 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.887 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.888 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.889 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.889 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2024-03-15T13:09:07.889 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-15T13:09:07.889 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.889 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.889 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.889 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.889 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.889 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.889 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.889 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.889 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.889 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.889 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.890 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-15T13:09:07.890 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-15T13:09:07.890 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.890 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.890 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.890 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-03-15T13:09:07.890 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.890 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.890 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.890 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.890 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.890 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.890 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-15T13:09:07.891 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-15T13:09:07.892 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.892 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.892 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.892 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-03-15T13:09:07.892 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.892 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.892 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.892 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.892 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.892 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.892 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.892 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2024-03-15T13:09:07.892 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2024-03-15T13:09:07.892 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.893 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.894 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.895 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2024-03-15T13:09:07.895 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.895 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.895 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.895 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2024-03-15T13:09:07.895 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2024-03-15T13:09:07.895 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.895 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.895 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.895 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2024-03-15T13:09:07.895 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.895 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.895 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.895 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.896 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.896 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.896 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.896 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-03-15T13:09:07.896 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-03-15T13:09:07.896 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.896 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.896 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.896 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.896 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.896 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.896 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.896 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.896 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.897 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.897 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.897 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.897 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.897 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.897 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.897 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.897 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.897 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.897 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.897 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.897 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.897 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.898 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.898 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.898 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.898 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.898 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.898 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.898 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.898 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.898 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.898 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.898 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.898 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.898 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.898 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.899 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.900 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.901 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.902 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.903 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.904 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.904 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.904 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.904 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.904 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-03-15T13:09:07.904 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-03-15T13:09:07.904 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.904 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.904 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.904 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2024-03-15T13:09:07.904 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.904 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.904 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.904 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.905 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.906 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.906 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2024-03-15T13:09:07.906 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2024-03-15T13:09:07.906 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.906 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.906 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.906 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2024-03-15T13:09:07.906 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.906 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.906 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.906 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.906 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.906 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.906 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.907 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.909 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2024-03-15T13:09:07.909 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2024-03-15T13:09:07.909 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.909 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.909 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.909 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.909 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.909 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.909 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.909 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.909 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.909 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.909 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.910 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.910 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-03-15T13:09:07.910 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.910 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.910 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.910 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2024-03-15T13:09:07.910 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2024-03-15T13:09:07.910 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.910 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.910 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.910 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.910 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.910 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.911 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.912 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.912 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.912 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.912 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.912 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.912 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.912 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.912 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.912 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.912 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.912 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.912 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.912 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.912 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.913 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.913 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.913 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.913 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.913 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.913 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.913 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.913 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.913 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.913 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.913 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.913 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.913 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.913 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.914 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.914 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.914 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.914 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.914 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.914 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.914 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.914 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.914 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.914 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.914 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.914 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.914 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.914 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.915 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.915 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.915 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.915 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.915 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.915 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.915 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.915 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.915 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.915 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.915 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.915 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.915 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.916 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2024-03-15T13:09:07.916 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2024-03-15T13:09:07.916 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.916 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.916 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.916 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2024-03-15T13:09:07.916 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.916 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.916 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.916 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-15T13:09:07.916 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.916 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.916 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.917 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2024-03-15T13:09:07.917 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2024-03-15T13:09:07.917 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.917 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.917 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.917 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2024-03-15T13:09:07.917 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.917 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.917 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.917 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-15T13:09:07.917 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.917 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.917 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.917 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.918 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.919 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.920 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-15T13:09:07.921 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2024-03-15T13:09:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.924 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.924 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.924 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.924 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.924 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.924 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.924 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.924 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.924 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.924 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.924 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.924 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.924 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.925 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.925 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.925 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.925 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.925 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.925 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.925 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.925 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.925 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.925 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.925 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.925 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.925 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.925 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.926 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.926 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.926 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.926 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.926 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.926 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.926 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.926 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.926 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.926 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.926 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.926 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-15T13:09:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.931 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.931 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.931 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2024-03-15T13:09:07.931 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2024-03-15T13:09:07.931 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.931 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.931 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.931 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2024-03-15T13:09:07.931 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.931 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.931 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.931 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-15T13:09:07.931 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.932 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.932 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.932 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2024-03-15T13:09:07.932 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2024-03-15T13:09:07.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.932 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.932 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.932 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.932 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.932 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.932 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.932 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2024-03-15T13:09:07.933 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.933 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.933 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.933 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-03-15T13:09:07.933 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-03-15T13:09:07.933 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:07.933 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.933 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.933 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2024-03-15T13:09:07.933 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2024-03-15T13:09:07.933 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.933 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.933 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-15T13:09:07.933 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.934 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.934 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.934 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.934 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.934 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.934 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.934 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.934 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.934 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.934 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.934 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.934 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.935 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.935 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.935 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.935 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.935 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.935 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.935 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.935 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.935 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.935 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.935 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.935 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.935 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.936 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.936 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.936 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.936 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.936 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.936 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.936 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.936 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.936 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.936 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.936 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.936 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.936 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.936 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.937 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.937 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.937 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.937 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.937 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.937 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.937 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.937 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.937 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.937 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.937 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.937 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.937 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.938 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.938 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.938 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.938 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.938 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.938 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.938 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.938 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.938 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.938 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.938 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.939 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.939 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.939 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.939 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2024-03-15T13:09:07.939 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2024-03-15T13:09:07.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.939 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.939 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.939 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.939 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.939 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.939 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.939 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2024-03-15T13:09:07.940 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.940 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.940 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.940 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2024-03-15T13:09:07.940 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2024-03-15T13:09:07.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.940 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.940 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.940 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2024-03-15T13:09:07.940 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.940 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.940 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.940 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2024-03-15T13:09:07.940 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.941 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.941 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.941 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-03-15T13:09:07.941 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-03-15T13:09:07.941 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.941 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.941 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.941 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2024-03-15T13:09:07.941 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.941 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.941 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.941 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-15T13:09:07.941 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.942 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.942 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.942 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2024-03-15T13:09:07.942 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2024-03-15T13:09:07.942 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.942 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.942 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.942 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-03-15T13:09:07.942 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-03-15T13:09:07.942 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2024-03-15T13:09:07.942 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.942 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-15T13:09:07.942 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.942 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.943 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.943 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2024-03-15T13:09:07.943 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2024-03-15T13:09:07.943 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.943 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.943 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.943 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.943 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.943 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.943 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.943 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2024-03-15T13:09:07.943 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.943 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.943 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.946 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.946 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.946 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.946 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.946 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.946 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.946 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.946 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.946 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.946 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.946 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.946 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.946 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.947 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.947 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.947 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.947 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.947 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.947 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.947 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.947 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.947 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.947 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.947 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.947 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.947 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.947 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.948 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.949 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.949 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.949 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.949 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.949 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.949 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.949 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.949 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.949 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.949 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.949 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.949 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.949 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.950 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2024-03-15T13:09:07.950 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.950 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.950 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.950 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.950 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.950 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.950 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.950 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.950 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.950 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.950 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.950 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.950 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.950 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.951 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.953 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.953 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.953 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.953 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.953 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.953 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.953 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.953 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.953 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.953 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.953 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.953 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.953 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.954 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.954 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.954 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.954 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.954 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.954 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.954 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.954 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.954 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.954 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.954 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.954 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.954 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.954 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.955 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-15T13:09:07.956 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.956 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.956 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.956 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2024-03-15T13:09:07.956 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2024-03-15T13:09:07.956 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.956 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.956 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.956 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:07.956 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2024-03-15T13:09:07.956 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.956 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.956 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-15T13:09:07.956 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.957 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.957 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.957 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.957 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.957 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.957 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.957 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.957 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.957 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.957 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.957 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.957 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.957 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.957 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.958 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.958 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.958 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.958 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.958 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.958 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.958 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.958 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.958 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.958 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.958 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.958 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.958 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.959 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.959 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.959 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.959 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.959 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.959 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.959 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.959 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.959 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.959 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.959 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.959 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.959 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.961 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.962 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.963 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.964 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.965 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.966 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.966 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.966 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.966 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.966 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.966 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.966 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.966 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.966 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.966 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.966 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.966 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.966 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.967 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.967 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.967 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.967 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.967 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.967 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.967 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.967 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.967 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.967 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.967 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2024-03-15T13:09:07.967 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2024-03-15T13:09:07.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.967 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2024-03-15T13:09:07.968 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2024-03-15T13:09:07.970 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.970 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.970 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.970 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2024-03-15T13:09:07.970 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2024-03-15T13:09:07.970 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.970 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.970 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.970 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.970 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.970 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.970 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.970 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-15T13:09:07.971 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.972 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.973 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.973 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.973 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.973 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.973 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.973 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.973 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.973 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.973 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.973 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.973 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.973 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.973 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.973 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.974 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.974 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.974 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.974 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.974 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.974 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.974 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.974 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.974 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.974 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.974 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.974 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.974 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.975 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.975 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.975 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.975 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.975 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.975 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.975 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.975 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.975 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.975 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.975 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.975 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.975 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.976 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.977 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.978 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.978 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.978 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.978 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.978 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.978 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.978 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.978 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.978 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.978 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.978 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.978 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.978 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.978 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.981 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.982 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2024-03-15T13:09:07.982 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.982 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.982 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.982 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.982 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.982 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.982 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.982 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.982 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.982 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.982 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.982 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.983 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.984 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.985 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.986 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.986 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.986 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.986 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.986 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.986 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.986 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.986 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.986 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.986 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.986 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.986 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-03-15T13:09:07.986 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.988 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.988 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.988 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.988 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.988 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.988 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.988 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.988 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.988 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.988 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.988 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.989 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.990 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.990 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.990 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.990 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.990 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.990 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.990 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.990 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.990 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.990 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.990 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.990 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.990 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.991 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.992 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.992 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.992 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.992 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.992 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.992 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.992 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.992 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.992 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2024-03-15T13:09:07.992 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-03-15T13:09:07.992 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.992 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.992 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.992 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2024-03-15T13:09:07.993 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.994 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.994 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.994 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.994 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.994 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.994 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.994 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.994 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.994 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.994 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.994 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.994 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.994 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.996 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.997 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.997 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:07.997 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:07.997 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:07.997 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:07.997 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:07.997 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.997 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.997 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.997 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.997 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:07.997 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:07.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:07.997 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.998 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.998 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:07.998 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.998 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.998 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:07.998 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:07.998 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:07.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:07.998 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:07.998 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.998 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:07.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:07.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:07.998 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2024-03-15T13:09:07.999 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:07.999 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:07.999 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:07.999 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:07.999 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:07.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:07.999 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:07.999 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:07.999 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:07.999 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:07.999 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:07.999 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:07.999 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:07.999 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:07.999 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.000 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.000 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.000 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.000 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.000 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.000 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.000 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.000 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.000 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.000 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.000 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.000 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.000 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.000 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.002 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.002 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.002 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.002 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.002 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.002 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.002 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.002 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.002 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.002 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.002 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.002 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.002 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.003 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.004 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.005 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.005 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.005 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.005 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.005 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.005 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.005 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.005 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.005 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.005 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.005 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.005 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.007 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.008 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.009 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.010 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.011 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.012 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.014 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.015 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.016 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2024-03-15T13:09:08.017 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.017 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.017 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.017 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2024-03-15T13:09:08.017 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.017 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.017 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.017 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-03-15T13:09:08.017 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-03-15T13:09:08.017 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.017 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.017 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.017 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-03-15T13:09:08.017 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.018 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.018 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.018 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2024-03-15T13:09:08.018 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.018 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.018 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.018 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2024-03-15T13:09:08.018 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.018 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.018 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.019 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2024-03-15T13:09:08.019 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2024-03-15T13:09:08.019 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.019 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.019 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.019 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.019 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.019 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.019 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.019 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.019 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.019 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.019 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.019 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2024-03-15T13:09:08.020 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2024-03-15T13:09:08.020 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.020 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.020 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.020 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.020 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.020 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.020 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.020 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-15T13:09:08.020 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-15T13:09:08.020 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.020 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.020 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.021 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.021 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.021 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.021 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.021 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.021 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.021 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.021 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.021 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.021 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.021 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.021 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.021 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.021 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.022 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.022 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.022 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.022 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.022 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.022 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.022 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.022 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.022 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.022 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.022 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.022 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.022 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.023 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.023 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.023 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.023 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.023 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.023 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.023 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.023 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.023 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.023 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.023 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.023 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.023 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.023 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.024 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.024 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.024 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.024 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.024 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.024 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.024 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.024 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.024 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.024 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.024 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.024 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.025 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.025 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.025 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.025 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.025 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.025 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.025 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.025 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.025 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.025 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.025 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.025 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.025 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2024-03-15T13:09:08.026 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2024-03-15T13:09:08.026 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.026 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.026 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.026 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.026 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.026 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.026 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.026 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.026 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.026 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.026 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.026 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-15T13:09:08.027 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-15T13:09:08.027 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.027 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.027 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.027 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2024-03-15T13:09:08.027 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.027 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.027 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.027 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.027 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.027 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.027 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.027 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2024-03-15T13:09:08.027 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2024-03-15T13:09:08.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:08.028 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.028 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2024-03-15T13:09:08.028 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.028 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.028 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.028 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.028 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.028 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.028 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.028 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-03-15T13:09:08.028 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.029 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.030 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.030 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2024-03-15T13:09:08.030 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.030 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.030 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.030 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-15T13:09:08.030 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.030 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.030 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.030 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2024-03-15T13:09:08.030 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2024-03-15T13:09:08.030 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.030 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.031 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.031 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2024-03-15T13:09:08.031 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.031 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.031 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.031 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.031 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.031 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.031 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.031 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-03-15T13:09:08.031 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-03-15T13:09:08.031 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.031 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.031 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.032 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.033 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.034 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.035 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.035 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.035 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.035 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.035 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.035 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.035 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.035 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.035 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.035 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.035 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.035 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.035 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.036 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.036 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.036 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.036 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.036 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.036 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.036 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.036 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.036 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.036 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.036 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.036 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.036 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.036 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.037 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:08.038 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.039 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.040 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.040 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.040 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.040 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2024-03-15T13:09:08.040 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2024-03-15T13:09:08.040 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.040 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.040 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.040 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.040 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.040 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.040 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.040 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.040 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.041 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.042 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.043 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.044 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.045 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.046 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.047 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.048 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.049 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.050 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.051 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.054 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.055 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.056 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.056 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.056 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.056 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.056 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.056 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.056 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.056 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.056 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.056 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.056 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.056 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.056 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.058 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.058 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.058 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.058 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.058 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.058 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.058 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.058 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.058 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.058 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.058 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.058 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.058 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.059 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2024-03-15T13:09:08.059 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2024-03-15T13:09:08.059 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.059 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.059 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.059 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2024-03-15T13:09:08.059 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.059 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.059 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.059 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-15T13:09:08.059 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.059 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.059 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.060 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2024-03-15T13:09:08.060 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.060 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.060 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.060 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.060 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.060 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.060 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.060 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.060 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.060 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.060 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.060 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.061 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.061 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.061 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.061 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.061 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.061 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.061 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.061 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.061 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.061 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.061 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.061 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.061 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.061 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.062 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.062 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.062 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.062 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.062 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.062 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.062 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.062 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.062 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.062 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.062 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.062 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.062 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.062 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2024-03-15T13:09:08.064 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2024-03-15T13:09:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2024-03-15T13:09:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.066 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.066 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2024-03-15T13:09:08.066 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.066 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.066 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.066 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.066 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.066 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.066 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.066 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2024-03-15T13:09:08.066 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2024-03-15T13:09:08.066 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.067 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.068 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.069 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.069 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.069 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.069 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.069 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.069 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.069 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2024-03-15T13:09:08.069 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2024-03-15T13:09:08.069 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.069 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.069 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.069 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.070 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.070 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.070 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.070 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.070 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.070 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.070 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2024-03-15T13:09:08.070 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2024-03-15T13:09:08.070 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.070 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.070 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.070 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.070 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.070 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2024-03-15T13:09:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2024-03-15T13:09:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.071 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.072 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.072 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.072 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.072 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.072 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2024-03-15T13:09:08.072 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2024-03-15T13:09:08.072 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.072 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.072 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.072 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.072 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-03-15T13:09:08.072 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2024-03-15T13:09:08.072 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.073 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.074 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.074 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.074 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2024-03-15T13:09:08.074 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2024-03-15T13:09:08.074 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.074 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.074 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.074 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.074 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.074 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.074 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.074 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.074 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.075 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.075 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.075 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2024-03-15T13:09:08.075 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2024-03-15T13:09:08.075 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.075 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.075 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.075 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.075 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.075 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.075 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.075 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.075 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.075 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.076 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.076 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.076 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.076 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-03-15T13:09:08.076 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.076 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.076 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.076 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2024-03-15T13:09:08.076 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2024-03-15T13:09:08.076 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.076 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.076 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.076 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.077 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.077 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.077 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.077 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2024-03-15T13:09:08.077 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.077 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.077 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.077 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2024-03-15T13:09:08.077 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2024-03-15T13:09:08.077 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.077 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.077 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.077 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.078 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.078 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.078 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.078 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2024-03-15T13:09:08.078 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.078 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.078 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.078 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.078 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.078 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.078 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.078 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.078 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.079 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.079 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.079 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.079 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.079 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.079 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.079 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.079 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.079 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.079 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.079 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.079 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.079 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.079 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.080 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.082 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.082 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.082 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.082 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.082 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.082 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.082 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.082 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.082 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.082 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.082 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.082 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.082 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.082 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.083 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.084 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.085 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.086 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.087 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2024-03-15T13:09:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.088 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.090 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.091 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.091 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.091 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.091 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.091 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.091 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.091 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.091 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.091 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.091 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.091 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.091 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.092 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.092 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.092 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.092 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.092 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.092 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.092 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.092 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.092 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.092 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.092 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.092 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.092 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2024-03-15T13:09:08.093 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.095 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.096 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.097 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.097 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.097 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.097 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.097 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.097 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.097 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.097 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.097 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.097 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.097 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.097 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.097 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.097 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.098 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.098 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.098 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.098 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.098 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.098 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.098 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.098 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.098 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.098 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.098 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.098 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.098 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.098 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.099 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.100 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.100 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.100 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.100 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.100 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.100 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.100 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2024-03-15T13:09:08.100 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.100 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.100 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.100 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2024-03-15T13:09:08.100 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.102 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.102 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.102 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.102 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.102 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-15T13:09:08.102 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.102 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.102 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.102 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2024-03-15T13:09:08.102 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2024-03-15T13:09:08.102 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.102 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.102 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.103 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.103 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.103 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.103 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.103 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-15T13:09:08.103 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.103 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.103 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.103 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2024-03-15T13:09:08.103 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2024-03-15T13:09:08.103 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.103 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.103 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.103 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.104 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.105 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-03-15T13:09:08.107 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.108 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.109 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.110 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.110 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.110 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.110 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.110 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2024-03-15T13:09:08.110 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2024-03-15T13:09:08.110 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.110 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.110 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.110 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.110 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.110 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.110 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.110 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.111 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.112 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.112 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.112 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.112 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.112 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.112 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.112 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.112 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.112 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.112 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.112 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.112 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.112 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.112 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.113 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.113 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.113 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.113 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.113 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.113 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.113 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.113 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.113 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.113 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.113 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.113 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.113 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.113 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.114 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.114 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.114 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.114 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.114 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.114 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.114 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.114 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.114 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.114 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.114 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.114 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.114 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.115 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.115 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.115 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.115 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.115 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.115 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.115 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.115 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.115 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.115 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.115 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.115 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.115 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.116 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.116 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.116 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.116 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.116 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.116 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.116 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.116 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.116 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.116 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.116 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.116 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.116 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2024-03-15T13:09:08.116 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.117 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.118 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.119 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.120 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.121 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.122 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.122 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.122 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.122 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.122 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.122 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.122 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.122 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.122 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.122 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.122 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.122 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.123 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.123 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.123 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.123 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.123 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.123 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.123 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.123 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.123 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.123 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.123 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.123 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.123 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.124 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.124 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.124 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.124 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.124 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.124 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.124 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-03-15T13:09:08.124 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.124 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.125 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.125 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.125 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.125 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.125 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.125 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.125 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.125 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.125 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.125 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.125 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.125 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.125 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.125 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.126 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.126 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.126 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.126 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.126 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.126 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.126 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.126 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.126 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.126 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.126 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.127 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.127 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.127 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.127 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.127 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.127 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.127 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.127 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.127 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.127 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.127 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.127 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.127 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.128 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.128 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.128 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.128 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.128 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.128 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.128 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.128 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.128 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.128 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.128 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.128 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.128 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.128 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2024-03-15T13:09:08.129 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2024-03-15T13:09:08.130 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.130 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.130 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.130 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2024-03-15T13:09:08.130 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.130 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.130 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.130 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-15T13:09:08.130 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.130 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.130 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.130 INFO:teuthology.orchestra.run.smithi002.stdout: "periodic_async_work": { 2024-03-15T13:09:08.131 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "periodic_async_work", 2024-03-15T13:09:08.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.131 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.131 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.131 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.131 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.131 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.131 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.131 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Periodically check for async work", 2024-03-15T13:09:08.131 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.131 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.131 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.131 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2024-03-15T13:09:08.131 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2024-03-15T13:09:08.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.132 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.132 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.132 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-03-15T13:09:08.132 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.132 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.132 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.132 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-15T13:09:08.132 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.132 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.132 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.132 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.132 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.133 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.134 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.134 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.134 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2024-03-15T13:09:08.134 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2024-03-15T13:09:08.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.134 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.134 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.134 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.134 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.134 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.134 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.134 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.134 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.134 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.135 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.135 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-03-15T13:09:08.135 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-03-15T13:09:08.135 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.135 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.135 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.135 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-03-15T13:09:08.135 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.135 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.135 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.135 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.135 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.135 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.135 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.136 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.136 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.136 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.136 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.136 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.136 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.136 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.136 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.136 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.136 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.136 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.136 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.136 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.137 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.137 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.137 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.137 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.137 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.137 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.137 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.137 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.137 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.137 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.137 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.137 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.137 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.137 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.138 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.139 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.139 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.139 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.139 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.139 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.139 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.139 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.139 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.139 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.139 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.139 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.139 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.139 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2024-03-15T13:09:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2024-03-15T13:09:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2024-03-15T13:09:08.141 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.142 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.143 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.143 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.143 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.143 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.143 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-03-15T13:09:08.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.143 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.143 INFO:teuthology.orchestra.run.smithi002.stdout: "modules": [ 2024-03-15T13:09:08.143 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-03-15T13:09:08.143 INFO:teuthology.orchestra.run.smithi002.stdout: "dashboard", 2024-03-15T13:09:08.143 INFO:teuthology.orchestra.run.smithi002.stdout: "iostat", 2024-03-15T13:09:08.143 INFO:teuthology.orchestra.run.smithi002.stdout: "nfs", 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: "restful", 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_schedule" 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-03-15T13:09:08.144 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.145 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.145 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.145 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2024-03-15T13:09:08.145 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.145 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.145 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.145 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.145 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.145 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.145 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.145 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.145 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.145 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.146 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.147 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2024-03-15T13:09:08.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.150 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.151 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2024-03-15T13:09:08.152 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2024-03-15T13:09:08.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.155 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2024-03-15T13:09:08.156 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2024-03-15T13:09:08.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.157 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.157 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.157 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.157 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.157 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.157 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.157 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2024-03-15T13:09:08.157 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.157 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.157 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.157 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2024-03-15T13:09:08.157 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2024-03-15T13:09:08.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.158 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.158 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.158 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2024-03-15T13:09:08.158 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.158 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.158 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.158 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2024-03-15T13:09:08.158 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-15T13:09:08.158 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.158 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.158 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2024-03-15T13:09:08.158 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2024-03-15T13:09:08.158 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.159 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2024-03-15T13:09:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.161 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2024-03-15T13:09:08.162 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-15T13:09:08.162 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.162 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.162 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2024-03-15T13:09:08.162 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2024-03-15T13:09:08.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.162 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.162 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.162 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2024-03-15T13:09:08.162 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.162 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.162 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.162 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-15T13:09:08.163 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.164 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.164 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.164 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.164 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.164 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.164 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.164 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.164 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.164 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.164 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.164 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.164 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.164 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.165 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.166 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.166 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.166 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.166 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.166 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.166 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.166 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.166 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.166 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.166 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.166 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.166 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.166 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.167 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-03-15T13:09:08.168 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.169 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: "read", 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap", 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap-read" 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.170 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.171 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-15T13:09:08.173 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-15T13:09:08.173 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.173 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.173 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2024-03-15T13:09:08.173 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2024-03-15T13:09:08.173 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-03-15T13:09:08.173 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.173 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.173 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:08.173 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.173 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.173 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.173 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-15T13:09:08.173 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-03-15T13:09:08.174 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.175 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.175 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.175 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-15T13:09:08.175 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.175 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.175 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.175 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2024-03-15T13:09:08.175 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2024-03-15T13:09:08.175 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.175 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.175 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.175 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2024-03-15T13:09:08.175 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.176 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.176 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.176 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-15T13:09:08.176 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.176 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.176 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.176 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2024-03-15T13:09:08.176 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2024-03-15T13:09:08.176 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.176 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.176 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.176 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2024-03-15T13:09:08.176 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.177 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.177 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.177 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-15T13:09:08.177 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.177 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.177 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.177 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2024-03-15T13:09:08.177 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2024-03-15T13:09:08.177 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.177 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.177 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.177 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.177 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.178 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-15T13:09:08.179 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.179 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.179 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.179 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2024-03-15T13:09:08.179 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2024-03-15T13:09:08.179 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:08.179 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.179 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.179 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2024-03-15T13:09:08.179 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.179 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.179 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.179 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-15T13:09:08.179 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.180 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.180 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.180 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm_log_destination": { 2024-03-15T13:09:08.180 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm_log_destination", 2024-03-15T13:09:08.180 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.180 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.180 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.180 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.180 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.180 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.180 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.180 INFO:teuthology.orchestra.run.smithi002.stdout: "file", 2024-03-15T13:09:08.180 INFO:teuthology.orchestra.run.smithi002.stdout: "file,syslog", 2024-03-15T13:09:08.181 INFO:teuthology.orchestra.run.smithi002.stdout: "syslog" 2024-03-15T13:09:08.181 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.181 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-15T13:09:08.181 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.181 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.181 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.181 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2024-03-15T13:09:08.181 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2024-03-15T13:09:08.181 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.181 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.181 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.181 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.181 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.182 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.182 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.182 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-15T13:09:08.182 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.182 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.182 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.182 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2024-03-15T13:09:08.182 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2024-03-15T13:09:08.182 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.182 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.182 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.182 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.182 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.182 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.183 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.184 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.185 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2024-03-15T13:09:08.186 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2024-03-15T13:09:08.186 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.186 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.186 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.186 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-15T13:09:08.186 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.186 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.186 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.186 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2024-03-15T13:09:08.186 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.186 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.186 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.187 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2024-03-15T13:09:08.187 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2024-03-15T13:09:08.187 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.187 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.187 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.187 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-15T13:09:08.187 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.187 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.187 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.187 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-03-15T13:09:08.187 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.187 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.187 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.188 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2024-03-15T13:09:08.188 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2024-03-15T13:09:08.188 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.188 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.188 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.188 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-15T13:09:08.188 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.188 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.188 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.188 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2024-03-15T13:09:08.188 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.188 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.189 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2024-03-15T13:09:08.189 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2024-03-15T13:09:08.189 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.189 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.189 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.189 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-15T13:09:08.189 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.189 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.189 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.189 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2024-03-15T13:09:08.189 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.189 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.189 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.190 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2024-03-15T13:09:08.190 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2024-03-15T13:09:08.190 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.190 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.190 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.190 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-15T13:09:08.190 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.190 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.190 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.190 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2024-03-15T13:09:08.190 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.190 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.190 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.191 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2024-03-15T13:09:08.191 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2024-03-15T13:09:08.191 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.191 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.191 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.191 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-15T13:09:08.191 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.191 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.191 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.191 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2024-03-15T13:09:08.191 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.191 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.191 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.192 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2024-03-15T13:09:08.192 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2024-03-15T13:09:08.192 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.192 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.192 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-15T13:09:08.192 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.192 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.192 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.192 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2024-03-15T13:09:08.192 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.192 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.192 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2024-03-15T13:09:08.193 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2024-03-15T13:09:08.194 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.194 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.194 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.194 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-15T13:09:08.194 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.194 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.194 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.194 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-03-15T13:09:08.194 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.194 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.194 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_nvmeof": { 2024-03-15T13:09:08.194 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_nvmeof", 2024-03-15T13:09:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-15T13:09:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Nvme-of container image", 2024-03-15T13:09:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2024-03-15T13:09:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2024-03-15T13:09:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.196 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.196 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.196 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-15T13:09:08.196 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.196 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.196 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.196 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-03-15T13:09:08.196 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.196 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.196 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.196 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2024-03-15T13:09:08.196 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2024-03-15T13:09:08.197 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.197 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.197 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.197 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-15T13:09:08.197 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.197 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.197 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.197 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2024-03-15T13:09:08.197 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.197 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.197 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.197 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2024-03-15T13:09:08.197 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2024-03-15T13:09:08.198 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.198 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.198 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.198 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-15T13:09:08.198 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.198 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.198 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.198 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2024-03-15T13:09:08.198 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.198 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.198 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.198 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2024-03-15T13:09:08.198 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2024-03-15T13:09:08.199 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.199 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.199 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.199 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.199 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.199 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.199 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.199 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2024-03-15T13:09:08.199 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.199 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.199 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.199 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2024-03-15T13:09:08.199 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2024-03-15T13:09:08.199 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.200 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.201 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2024-03-15T13:09:08.201 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.201 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.201 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.201 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-15T13:09:08.201 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.201 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.201 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.201 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2024-03-15T13:09:08.201 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2024-03-15T13:09:08.201 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.201 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.201 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.201 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2024-03-15T13:09:08.202 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.202 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.202 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.202 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-15T13:09:08.202 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.202 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.202 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.202 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2024-03-15T13:09:08.202 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2024-03-15T13:09:08.202 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.202 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.202 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.202 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2024-03-15T13:09:08.202 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.203 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.204 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.205 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.205 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2024-03-15T13:09:08.205 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2024-03-15T13:09:08.205 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.205 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.205 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.205 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-03-15T13:09:08.205 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.205 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.205 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.205 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2024-03-15T13:09:08.205 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.205 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: "hw_monitoring": { 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hw_monitoring", 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: "inventory_list_all": { 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "inventory_list_all", 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.207 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.208 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.209 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.209 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2024-03-15T13:09:08.209 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2024-03-15T13:09:08.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.209 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.209 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.209 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.209 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.209 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.209 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.209 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-15T13:09:08.209 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.209 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.209 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.210 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.211 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.213 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-15T13:09:08.213 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-15T13:09:08.213 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.213 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.213 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.213 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.213 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.213 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.213 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.213 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-15T13:09:08.213 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.213 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.213 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.214 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-15T13:09:08.214 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-15T13:09:08.214 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.214 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.214 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.214 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2024-03-15T13:09:08.214 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.214 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.214 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.214 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-15T13:09:08.214 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.214 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.214 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2024-03-15T13:09:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2024-03-15T13:09:08.216 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.216 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.216 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.216 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:08.216 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.216 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.216 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.216 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-15T13:09:08.216 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.216 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.216 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.216 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2024-03-15T13:09:08.216 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.217 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: "oob_default_addr": { 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "oob_default_addr", 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.218 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.219 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "169.254.1.1", 2024-03-15T13:09:08.219 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.219 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.219 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.219 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-15T13:09:08.219 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.219 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.219 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.219 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2024-03-15T13:09:08.219 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2024-03-15T13:09:08.219 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.219 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.219 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.220 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-15T13:09:08.220 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.220 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.220 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.220 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-15T13:09:08.220 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.220 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.220 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.220 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2024-03-15T13:09:08.220 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2024-03-15T13:09:08.220 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.220 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.221 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.221 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.221 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.221 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.221 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.221 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-15T13:09:08.221 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.221 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.221 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.221 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2024-03-15T13:09:08.221 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2024-03-15T13:09:08.221 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.221 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.222 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.222 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.222 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.222 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.222 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.222 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-15T13:09:08.222 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.222 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.222 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.223 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2024-03-15T13:09:08.223 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2024-03-15T13:09:08.223 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.223 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.223 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.223 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.223 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.223 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.223 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.223 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-15T13:09:08.223 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.223 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.223 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.224 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2024-03-15T13:09:08.224 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2024-03-15T13:09:08.224 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.224 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.224 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.224 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.224 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.224 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.224 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.224 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-15T13:09:08.224 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.224 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2024-03-15T13:09:08.226 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.227 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.228 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.228 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.228 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.228 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.228 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.228 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-15T13:09:08.228 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.228 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.228 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.228 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2024-03-15T13:09:08.228 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2024-03-15T13:09:08.228 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.228 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-15T13:09:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2024-03-15T13:09:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2024-03-15T13:09:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.230 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.230 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.230 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.230 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.230 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2024-03-15T13:09:08.230 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.230 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.230 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.230 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2024-03-15T13:09:08.230 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2024-03-15T13:09:08.230 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.230 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.230 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.230 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.231 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.231 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.231 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.231 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-15T13:09:08.231 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.231 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.231 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.231 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2024-03-15T13:09:08.231 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2024-03-15T13:09:08.231 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.231 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.231 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.232 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.232 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.232 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.232 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-15T13:09:08.232 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.232 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.232 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.232 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.232 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.232 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2024-03-15T13:09:08.232 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.232 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.233 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.234 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.235 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.236 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.236 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.236 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.236 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.236 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.236 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.236 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.236 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.236 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.236 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.236 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.236 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.237 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.237 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.237 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.237 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.237 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.237 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.237 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.237 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.237 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2024-03-15T13:09:08.237 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2024-03-15T13:09:08.237 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.237 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.237 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2024-03-15T13:09:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-15T13:09:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2024-03-15T13:09:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2024-03-15T13:09:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.239 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.240 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.241 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.242 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.242 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.242 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.242 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.242 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.242 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.242 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2024-03-15T13:09:08.242 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2024-03-15T13:09:08.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.242 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.242 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.242 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.242 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.243 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.244 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-15T13:09:08.245 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.246 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.248 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.248 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.248 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.248 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.248 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.248 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-15T13:09:08.248 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-15T13:09:08.248 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.248 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.248 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.248 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.248 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.248 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.248 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.249 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.249 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.249 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.249 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.249 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-15T13:09:08.249 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-15T13:09:08.249 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.249 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.249 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.249 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.249 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.249 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.249 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.250 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.251 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.251 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.251 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-15T13:09:08.251 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-15T13:09:08.251 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.251 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.251 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.251 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.251 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.251 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.251 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.251 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.251 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.251 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.252 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.252 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2024-03-15T13:09:08.252 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-15T13:09:08.252 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.252 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.252 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.252 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-03-15T13:09:08.252 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.252 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.252 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.252 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.252 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.252 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.252 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.253 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2024-03-15T13:09:08.254 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2024-03-15T13:09:08.254 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.254 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.254 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.254 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.254 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.254 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.254 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.254 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.254 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.254 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.255 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2024-03-15T13:09:08.255 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-15T13:09:08.255 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.255 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.255 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.255 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-03-15T13:09:08.255 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.255 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.255 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.255 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.255 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.256 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.256 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.256 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-15T13:09:08.256 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-15T13:09:08.256 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.256 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.256 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.256 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.256 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.256 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.256 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.257 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.258 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.258 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.258 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-15T13:09:08.258 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-15T13:09:08.258 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.258 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.258 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.258 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.258 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.258 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.258 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.258 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.258 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.259 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.259 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.259 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-15T13:09:08.259 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-15T13:09:08.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.259 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.259 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.259 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.259 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.259 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.259 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.259 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.259 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.261 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-15T13:09:08.261 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-15T13:09:08.261 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.261 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.261 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.261 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.261 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.261 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.261 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.261 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.261 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.261 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.261 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.262 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-15T13:09:08.262 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-15T13:09:08.262 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.262 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.262 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.262 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.262 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.262 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.262 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.262 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.262 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.262 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.263 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.263 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-15T13:09:08.263 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-15T13:09:08.263 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.263 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.263 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.263 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.263 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.263 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.263 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.263 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.263 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.264 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.264 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.264 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-15T13:09:08.264 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-15T13:09:08.264 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.264 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.264 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.264 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.264 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.264 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.264 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.265 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.265 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.265 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.265 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.265 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-15T13:09:08.265 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-15T13:09:08.265 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.265 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.265 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.265 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.265 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.265 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.265 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.266 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.266 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.266 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-15T13:09:08.266 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-15T13:09:08.266 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.266 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.266 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.266 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.266 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.266 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.267 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.267 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.267 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.267 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.267 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.267 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-15T13:09:08.267 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-15T13:09:08.267 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.268 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.268 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.268 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.268 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.268 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.268 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.268 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.268 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.268 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.268 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.269 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-15T13:09:08.269 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-15T13:09:08.269 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.269 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.269 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.269 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.269 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.269 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.269 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.269 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.269 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.269 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.269 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.270 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2024-03-15T13:09:08.270 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-15T13:09:08.270 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.270 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.270 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.270 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.270 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.270 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.270 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.270 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.270 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.270 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.270 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.271 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-15T13:09:08.271 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-15T13:09:08.271 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.271 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.271 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.271 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-15T13:09:08.271 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.271 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.271 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.271 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.271 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.271 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.271 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.272 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-15T13:09:08.272 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-15T13:09:08.272 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.272 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.272 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.272 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:08.272 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.272 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.272 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.272 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.272 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.272 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.272 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.273 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-15T13:09:08.273 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-15T13:09:08.273 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.273 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.273 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.273 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2024-03-15T13:09:08.273 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.273 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.273 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.273 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.273 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.273 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.273 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.274 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-15T13:09:08.274 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-15T13:09:08.274 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.274 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.274 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.274 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2024-03-15T13:09:08.274 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.274 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.274 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.274 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.274 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.274 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.275 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.275 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2024-03-15T13:09:08.275 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-15T13:09:08.275 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.275 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.275 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.275 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.276 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.276 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.276 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.276 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.276 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.276 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.276 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.277 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-15T13:09:08.277 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-15T13:09:08.277 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.277 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.277 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.277 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-03-15T13:09:08.277 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.278 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.278 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.278 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.278 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.278 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.278 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.278 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2024-03-15T13:09:08.278 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-15T13:09:08.278 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.278 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.279 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.279 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.279 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.279 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.279 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.279 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.279 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.279 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.279 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2024-03-15T13:09:08.279 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-15T13:09:08.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.280 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.280 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.280 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.280 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.280 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.280 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.280 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.280 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.280 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.280 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.280 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-15T13:09:08.281 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-15T13:09:08.281 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.281 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.281 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.281 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-03-15T13:09:08.281 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.281 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.281 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.281 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.281 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.281 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.281 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.282 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-15T13:09:08.282 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-15T13:09:08.282 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.282 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.282 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.282 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:08.282 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.282 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.282 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.282 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.282 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.283 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.283 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.283 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-15T13:09:08.283 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-15T13:09:08.283 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.283 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.283 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.283 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-03-15T13:09:08.283 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.283 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.284 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.284 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.284 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.284 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.284 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.284 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2024-03-15T13:09:08.284 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2024-03-15T13:09:08.284 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.284 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.284 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.284 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.284 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.284 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.285 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.285 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.285 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.285 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.285 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.285 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2024-03-15T13:09:08.285 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2024-03-15T13:09:08.285 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.285 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.286 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.286 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.286 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.286 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.286 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.286 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.286 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.286 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.286 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2024-03-15T13:09:08.287 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2024-03-15T13:09:08.287 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.287 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.287 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.287 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.287 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.287 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.287 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.287 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2024-03-15T13:09:08.287 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.287 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.288 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.288 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2024-03-15T13:09:08.288 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2024-03-15T13:09:08.288 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.288 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.288 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.288 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2024-03-15T13:09:08.288 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.288 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.288 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.288 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.289 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.289 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.289 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.289 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-03-15T13:09:08.289 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-03-15T13:09:08.289 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.289 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.289 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.289 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.290 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.290 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.290 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.290 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.290 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.290 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.290 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.290 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.290 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.290 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.291 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.291 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.291 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.291 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.291 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.291 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.291 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.291 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.291 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.291 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.291 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.291 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.291 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.292 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.292 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.292 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.292 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.292 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.292 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.292 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.292 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.292 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.292 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.292 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.292 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.293 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.293 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.293 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.293 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.293 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.293 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.293 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.293 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.293 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.293 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.293 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.293 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.294 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.294 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.294 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.294 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.294 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.294 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.294 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.294 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.294 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.294 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.294 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.294 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.295 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.295 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.295 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.295 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.295 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.295 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.295 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.295 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.295 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.295 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.295 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.295 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.296 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2024-03-15T13:09:08.296 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2024-03-15T13:09:08.296 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.296 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.296 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.296 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.296 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.296 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.296 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.296 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2024-03-15T13:09:08.296 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.296 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.296 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.297 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2024-03-15T13:09:08.297 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2024-03-15T13:09:08.297 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.297 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.297 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.297 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.297 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.297 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.297 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.297 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.297 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.297 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.297 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.298 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-03-15T13:09:08.298 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-03-15T13:09:08.298 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.298 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.298 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.298 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-03-15T13:09:08.298 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.298 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.298 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.298 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.298 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.298 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.298 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.299 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-03-15T13:09:08.299 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-03-15T13:09:08.299 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.299 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.299 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2024-03-15T13:09:08.299 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.299 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.299 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.299 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.300 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.300 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.300 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.300 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-03-15T13:09:08.300 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-03-15T13:09:08.300 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.300 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.300 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.300 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.300 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.300 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.301 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.301 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.301 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.301 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.301 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.301 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2024-03-15T13:09:08.301 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2024-03-15T13:09:08.301 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.301 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.301 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.301 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2024-03-15T13:09:08.301 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.301 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.302 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.302 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.302 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.302 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.302 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.302 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-03-15T13:09:08.302 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-03-15T13:09:08.302 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.302 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.302 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.303 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2024-03-15T13:09:08.303 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.303 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.303 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.303 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.303 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2024-03-15T13:09:08.303 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.303 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.303 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.303 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.304 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.304 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-03-15T13:09:08.304 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-03-15T13:09:08.304 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.304 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.304 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.304 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-03-15T13:09:08.304 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-03-15T13:09:08.304 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-03-15T13:09:08.305 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.305 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.305 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.305 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.305 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.305 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2024-03-15T13:09:08.305 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2024-03-15T13:09:08.305 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.305 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.305 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.305 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.306 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.306 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.306 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.306 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.306 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.306 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.306 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.306 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.306 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.306 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-03-15T13:09:08.307 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.307 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.307 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.307 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2024-03-15T13:09:08.307 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2024-03-15T13:09:08.307 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.307 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.307 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.307 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.307 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.307 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.307 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.307 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2024-03-15T13:09:08.307 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.308 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.308 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.308 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.308 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.308 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.308 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.308 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.308 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.308 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.308 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.308 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.308 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.308 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.309 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.309 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.309 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.309 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.309 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.309 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.309 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.309 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.309 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.309 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.309 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.310 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.310 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.310 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.310 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.310 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.310 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.310 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.310 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.310 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.310 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.310 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.310 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.310 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.310 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.311 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.311 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.311 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.311 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.311 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.311 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.311 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.311 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.311 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.311 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.311 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.311 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.311 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.312 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.312 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.312 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.312 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.312 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.312 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.312 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.312 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.312 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.312 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.312 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.312 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.312 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.313 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.313 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2024-03-15T13:09:08.313 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2024-03-15T13:09:08.313 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.313 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.313 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.313 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2024-03-15T13:09:08.313 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.313 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.313 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.313 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-15T13:09:08.313 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.313 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.313 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.314 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2024-03-15T13:09:08.314 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2024-03-15T13:09:08.314 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.314 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.314 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.314 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2024-03-15T13:09:08.314 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.314 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.314 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.314 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-15T13:09:08.314 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.315 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.315 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2024-03-15T13:09:08.315 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2024-03-15T13:09:08.315 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.315 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.315 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.315 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2024-03-15T13:09:08.315 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.315 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.315 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.315 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2024-03-15T13:09:08.315 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.316 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.316 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2024-03-15T13:09:08.316 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2024-03-15T13:09:08.316 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.316 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.316 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.316 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-03-15T13:09:08.316 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.316 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.316 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.316 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-15T13:09:08.316 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.316 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.316 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.317 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2024-03-15T13:09:08.317 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2024-03-15T13:09:08.317 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.317 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.317 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.317 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.317 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.317 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.317 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.317 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-15T13:09:08.317 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.317 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.317 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-03-15T13:09:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-03-15T13:09:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-03-15T13:09:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2024-03-15T13:09:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.318 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2024-03-15T13:09:08.319 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2024-03-15T13:09:08.319 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.319 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.319 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.319 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2024-03-15T13:09:08.319 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.319 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.319 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.319 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-15T13:09:08.319 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.319 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.319 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.319 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.319 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.320 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2024-03-15T13:09:08.320 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.320 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.320 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.320 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.320 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.320 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.320 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.320 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.320 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.320 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.320 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.321 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.321 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.321 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.321 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.321 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.321 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.321 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.321 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.321 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.321 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.321 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.321 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.321 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.322 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.322 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.322 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.322 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.322 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.322 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.322 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.322 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.322 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.322 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.322 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.322 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.322 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.323 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.324 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.324 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.324 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.324 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.324 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.324 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.324 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.324 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.324 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.324 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.324 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.324 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.324 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.325 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.325 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.325 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2024-03-15T13:09:08.325 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2024-03-15T13:09:08.325 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.325 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.325 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.325 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-03-15T13:09:08.325 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.325 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.325 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.325 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.325 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.325 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.326 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.326 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2024-03-15T13:09:08.326 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2024-03-15T13:09:08.326 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.326 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.326 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.326 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2024-03-15T13:09:08.326 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.326 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.326 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.326 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.326 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-03-15T13:09:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-03-15T13:09:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-03-15T13:09:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2024-03-15T13:09:08.328 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.329 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.329 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.329 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-15T13:09:08.329 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.329 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.329 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.329 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2024-03-15T13:09:08.329 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2024-03-15T13:09:08.329 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.329 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.329 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.329 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2024-03-15T13:09:08.329 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.329 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.330 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.330 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-15T13:09:08.330 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.330 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.330 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.330 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2024-03-15T13:09:08.330 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2024-03-15T13:09:08.330 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.330 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.330 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.330 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.330 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.330 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.331 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.331 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2024-03-15T13:09:08.331 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.331 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.331 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.331 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-03-15T13:09:08.331 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-03-15T13:09:08.331 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.331 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.331 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.331 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2024-03-15T13:09:08.331 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2024-03-15T13:09:08.331 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.331 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.332 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-15T13:09:08.332 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.332 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.332 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.332 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.332 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.332 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.332 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.332 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.332 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.332 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.332 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.333 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.333 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.333 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.333 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.333 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.333 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.333 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.333 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.333 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.333 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.333 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.333 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.334 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.334 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.334 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.334 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.334 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.334 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.334 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.334 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.334 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.334 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.334 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.335 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.335 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.335 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.335 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.335 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.335 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.335 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.335 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.335 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.335 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.335 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.336 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.336 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.336 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.336 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.336 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.336 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.336 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.336 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.336 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.336 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.336 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.336 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.337 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.337 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.337 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.337 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.337 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.337 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.337 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.337 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.337 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.337 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2024-03-15T13:09:08.337 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2024-03-15T13:09:08.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.337 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.338 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.338 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.338 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.338 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.338 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.338 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2024-03-15T13:09:08.338 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.338 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.338 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.338 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2024-03-15T13:09:08.338 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2024-03-15T13:09:08.338 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.338 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.339 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.339 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2024-03-15T13:09:08.339 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.339 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.339 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.339 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2024-03-15T13:09:08.339 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.339 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.339 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.339 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-03-15T13:09:08.339 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-03-15T13:09:08.339 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.339 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.339 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.340 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2024-03-15T13:09:08.340 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.340 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.340 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.340 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-15T13:09:08.340 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.340 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.340 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.340 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2024-03-15T13:09:08.340 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2024-03-15T13:09:08.340 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.340 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.340 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.340 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-03-15T13:09:08.341 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-03-15T13:09:08.341 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2024-03-15T13:09:08.341 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.341 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-15T13:09:08.341 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.341 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.341 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.341 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2024-03-15T13:09:08.341 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2024-03-15T13:09:08.341 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.341 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.341 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.341 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.341 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.342 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.342 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.342 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2024-03-15T13:09:08.342 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.342 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.342 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.342 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2024-03-15T13:09:08.342 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2024-03-15T13:09:08.342 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.342 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.342 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.342 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2024-03-15T13:09:08.342 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.343 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.343 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.343 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-15T13:09:08.343 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.343 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.343 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.343 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.343 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.343 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2024-03-15T13:09:08.343 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.343 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.343 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.344 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.344 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.344 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.344 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.344 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.344 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.344 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.344 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.344 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.344 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.344 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.344 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.344 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.344 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.344 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.346 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.346 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.346 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.346 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.346 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.346 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.346 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.346 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.346 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.346 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.346 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.346 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.346 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.347 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.347 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.347 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.347 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.347 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.347 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.347 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.347 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.347 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.347 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.347 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.347 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.347 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.347 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.348 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.348 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.348 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.348 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.348 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.348 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.348 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.348 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.348 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.348 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2024-03-15T13:09:08.348 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.349 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.349 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.349 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.349 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.349 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.349 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.349 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.349 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.349 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.349 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.349 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.349 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.349 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.349 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.349 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.350 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.350 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.350 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.350 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.350 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.350 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.350 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.351 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.351 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.351 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.351 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.351 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.351 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.351 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.351 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.352 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.352 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.352 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.352 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.352 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.352 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.352 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.352 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.352 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.352 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.352 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.352 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.352 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.353 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.353 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.353 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.353 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.353 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.353 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.353 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.353 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.353 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.353 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.353 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.353 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.353 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.354 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.354 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.354 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.354 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.354 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.354 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.354 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.354 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.354 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.354 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.354 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.354 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.355 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2024-03-15T13:09:08.355 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.355 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.355 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.355 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2024-03-15T13:09:08.355 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2024-03-15T13:09:08.355 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.355 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.355 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2024-03-15T13:09:08.355 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.355 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.356 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.356 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-15T13:09:08.356 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.356 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.356 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.356 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2024-03-15T13:09:08.356 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2024-03-15T13:09:08.356 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.356 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.356 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.356 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:08.356 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2024-03-15T13:09:08.356 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.357 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.357 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-15T13:09:08.357 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.357 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.357 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.357 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.357 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.357 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.357 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.357 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.357 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.357 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.358 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.358 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.358 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.358 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.358 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.358 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.358 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.358 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.358 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.358 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.358 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.358 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.358 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.359 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.360 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.360 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.360 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.360 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.360 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.360 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.360 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.360 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.360 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.360 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.360 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.360 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.360 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.360 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.360 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.361 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.361 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.361 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.361 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.361 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.361 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.361 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.361 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.361 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.361 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.361 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.361 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.361 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.361 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2024-03-15T13:09:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2024-03-15T13:09:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2024-03-15T13:09:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.362 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.363 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.363 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2024-03-15T13:09:08.363 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.363 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.363 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.363 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2024-03-15T13:09:08.363 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-15T13:09:08.363 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.363 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.363 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.363 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.363 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.363 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.364 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.364 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.364 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.364 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.364 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.364 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.364 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.364 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.364 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.364 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.364 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.364 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.364 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.365 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.365 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.365 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.365 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.365 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.365 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.365 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.365 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.365 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.365 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.365 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.365 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.365 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.365 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2024-03-15T13:09:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2024-03-15T13:09:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2024-03-15T13:09:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-15T13:09:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2024-03-15T13:09:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2024-03-15T13:09:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2024-03-15T13:09:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.370 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.370 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2024-03-15T13:09:08.370 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.370 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.370 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.370 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2024-03-15T13:09:08.370 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2024-03-15T13:09:08.370 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.370 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.370 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.371 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2024-03-15T13:09:08.371 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.371 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.371 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.371 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2024-03-15T13:09:08.371 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.371 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.371 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.371 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2024-03-15T13:09:08.371 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2024-03-15T13:09:08.372 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.372 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.372 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.372 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.372 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.372 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.372 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.372 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2024-03-15T13:09:08.372 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.372 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.372 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.372 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2024-03-15T13:09:08.372 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2024-03-15T13:09:08.373 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.373 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.373 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.373 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2024-03-15T13:09:08.373 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.373 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.373 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.373 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-15T13:09:08.373 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-15T13:09:08.373 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.373 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.373 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.373 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.374 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2024-03-15T13:09:08.374 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.374 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.374 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.374 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.374 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.374 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.374 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.374 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.374 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.374 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.375 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.375 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.375 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.375 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.375 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.375 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.375 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.375 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.375 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.375 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.375 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.375 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.375 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.376 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.376 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.376 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.376 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.376 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.376 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.376 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.376 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.376 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.376 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.376 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.376 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.376 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.377 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.377 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.377 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.377 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.377 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.377 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.377 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.377 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.377 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.377 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.377 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.377 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.377 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.377 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.378 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.378 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.378 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.378 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.378 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.378 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.378 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.378 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.378 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.378 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.378 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.378 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.379 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.379 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.379 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.379 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.379 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.379 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.379 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.379 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.379 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2024-03-15T13:09:08.379 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.379 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.380 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.380 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.380 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.380 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.380 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.380 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.380 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.380 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.380 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.380 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.380 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.380 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.380 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.380 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.380 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.381 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.381 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.381 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.381 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.381 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.381 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.381 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.381 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.381 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.381 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.381 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.381 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.381 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.381 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.382 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.382 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.382 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.382 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.382 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.382 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.382 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.382 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.382 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.382 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.382 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.382 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.382 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.382 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.383 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.383 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.383 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.383 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.383 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.383 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.383 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.383 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.383 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.383 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.383 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.383 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.383 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.383 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.384 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.384 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.384 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.384 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.384 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.384 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.384 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.384 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.384 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.384 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.384 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.384 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.384 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2024-03-15T13:09:08.385 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.385 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.385 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.385 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.385 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.385 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.385 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.385 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.385 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.385 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.385 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.385 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.385 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.385 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.385 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.386 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.386 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.386 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.386 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.386 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.386 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.386 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.386 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.386 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.386 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.386 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.386 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.387 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.387 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.387 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.387 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.387 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.387 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.387 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.387 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.387 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.387 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.387 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.387 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.388 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.388 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.388 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.388 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.388 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.388 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.388 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.388 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.388 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.388 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.388 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.388 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.388 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.388 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.388 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.389 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.389 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.389 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.389 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.389 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.389 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.389 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.389 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.389 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.389 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.389 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.389 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.389 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.390 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.390 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.390 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-03-15T13:09:08.390 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.390 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.390 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.390 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2024-03-15T13:09:08.390 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2024-03-15T13:09:08.390 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.390 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.390 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.390 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.391 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.392 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.392 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.392 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.392 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.392 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.392 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.392 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.392 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.392 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.392 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.392 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.392 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.392 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.392 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.393 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.393 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.393 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.393 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.393 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.393 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.393 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.393 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.393 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.393 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.393 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.394 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.394 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.394 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.394 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.394 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.394 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.394 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.394 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.394 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.394 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.394 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.394 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.394 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.394 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.394 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.395 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.395 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.395 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.395 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.395 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.395 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.395 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.395 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.395 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.395 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.395 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.396 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.396 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.396 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.396 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2024-03-15T13:09:08.396 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-03-15T13:09:08.396 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.396 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.396 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.396 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.396 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.396 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.396 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.397 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-03-15T13:09:08.397 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2024-03-15T13:09:08.397 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2024-03-15T13:09:08.397 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.397 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2024-03-15T13:09:08.397 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.397 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.397 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.397 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.397 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.397 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2024-03-15T13:09:08.398 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.398 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.398 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.398 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.398 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.398 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.398 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.398 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.398 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.399 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.399 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.399 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.399 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.399 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.399 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.399 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.399 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.399 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.399 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.399 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.400 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.400 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.400 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.400 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.400 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.400 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.400 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.401 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.401 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.401 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.401 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.401 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.401 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.402 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.402 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.402 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.402 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.402 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.402 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.402 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.402 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.402 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.403 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.403 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.403 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.403 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.403 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.403 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.403 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.403 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.403 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.403 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.404 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.404 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.404 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.404 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.404 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.404 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.404 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.404 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.404 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.404 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.405 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.405 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.405 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.405 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.405 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.405 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2024-03-15T13:09:08.405 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.405 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.405 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.405 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.406 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.406 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.406 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.406 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.406 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.406 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.406 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.406 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.407 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.407 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.407 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.407 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.407 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.407 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.407 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.407 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.407 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.408 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.408 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.408 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.408 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.408 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.408 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.408 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.409 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.409 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.409 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.409 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.409 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.409 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.409 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.409 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.409 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.409 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.409 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.410 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.410 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.410 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.410 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.410 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.410 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.410 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.410 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.411 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.411 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.411 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.411 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.411 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.411 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.411 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.411 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.411 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.412 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.412 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.412 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.412 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.412 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.412 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.412 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.412 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.412 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.412 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.412 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.413 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.413 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2024-03-15T13:09:08.413 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.413 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.413 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.413 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.413 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.413 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.413 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.413 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.414 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.414 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.414 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.414 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.414 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.414 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.414 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.414 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.414 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.414 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.414 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.414 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.415 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.415 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.415 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.415 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.415 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.415 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.415 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.415 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.416 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.416 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.416 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.416 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.416 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.416 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.416 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.416 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.416 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.416 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.416 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.416 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.416 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.417 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.417 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.417 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.417 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.417 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.417 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.417 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.417 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.417 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.417 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.417 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.417 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.417 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.418 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.418 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.418 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.418 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.418 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.418 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.418 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.418 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.419 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.419 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.419 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.419 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.419 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.419 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-03-15T13:09:08.419 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-03-15T13:09:08.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.419 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.419 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.419 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-03-15T13:09:08.420 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.420 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.420 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.420 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.420 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.420 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.420 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.420 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2024-03-15T13:09:08.420 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2024-03-15T13:09:08.421 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:08.421 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.421 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.421 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-03-15T13:09:08.421 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2024-03-15T13:09:08.421 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.421 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.421 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2024-03-15T13:09:08.421 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-15T13:09:08.421 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.421 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.422 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2024-03-15T13:09:08.422 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.422 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.422 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.422 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2024-03-15T13:09:08.422 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2024-03-15T13:09:08.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.422 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.422 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.423 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.423 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.423 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.423 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.423 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-15T13:09:08.423 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.423 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.423 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.423 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-03-15T13:09:08.423 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-03-15T13:09:08.423 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.423 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.423 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.423 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.424 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.424 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.424 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.424 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.424 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.424 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.424 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.424 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.424 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.424 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.424 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.424 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.425 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.425 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.425 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.425 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.425 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.425 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.425 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.425 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.425 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.425 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.425 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.425 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.425 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.425 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.426 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.426 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.426 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.426 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.426 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.426 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.426 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.426 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.426 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.426 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.426 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.426 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.427 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.427 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.427 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.427 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.427 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.427 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.427 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.427 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.427 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.427 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.427 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.428 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.428 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.428 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.428 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.428 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.428 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.428 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.428 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.428 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.428 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.428 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.428 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.429 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.429 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.429 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.429 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.429 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.429 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.429 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.429 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.429 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.429 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.429 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2024-03-15T13:09:08.429 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2024-03-15T13:09:08.430 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.430 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.430 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2024-03-15T13:09:08.430 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.430 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.430 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.430 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2024-03-15T13:09:08.430 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.430 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.430 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.431 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-03-15T13:09:08.431 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-03-15T13:09:08.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.431 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.431 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.431 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-03-15T13:09:08.431 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.431 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.431 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.431 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2024-03-15T13:09:08.431 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.432 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.432 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.432 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.432 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2024-03-15T13:09:08.432 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.432 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.432 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.432 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2024-03-15T13:09:08.432 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2024-03-15T13:09:08.432 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.433 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.433 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.433 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.433 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.433 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.433 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.433 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.433 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.433 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.433 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.434 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2024-03-15T13:09:08.434 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2024-03-15T13:09:08.434 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.434 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.434 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.434 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.434 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.434 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.434 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.434 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-15T13:09:08.434 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-15T13:09:08.434 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.435 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.435 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.435 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.437 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.437 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.438 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.438 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.438 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.438 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.438 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.438 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.438 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.438 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.438 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.439 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.439 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.439 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.439 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.439 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.439 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.439 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.439 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.439 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.439 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.439 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.440 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.440 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.440 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.440 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.440 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.440 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.440 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.440 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.440 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.441 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.441 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.441 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.441 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.441 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.441 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.441 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.441 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.442 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.442 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.442 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.442 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.442 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.442 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.442 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.442 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.443 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.443 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.443 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.443 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.443 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.443 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.443 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.443 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.444 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.444 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.444 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.444 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.444 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2024-03-15T13:09:08.444 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2024-03-15T13:09:08.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.444 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.445 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.445 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.445 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.445 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.445 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.445 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.445 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.445 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.446 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.446 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-15T13:09:08.446 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-15T13:09:08.446 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.446 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.446 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.446 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2024-03-15T13:09:08.446 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.447 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.447 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.447 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.447 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.447 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.447 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.447 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2024-03-15T13:09:08.447 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2024-03-15T13:09:08.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:08.447 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.448 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2024-03-15T13:09:08.448 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.448 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.448 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.448 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.448 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.448 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.448 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.449 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-03-15T13:09:08.449 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-03-15T13:09:08.449 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.449 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.449 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.449 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-03-15T13:09:08.449 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.449 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.449 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.449 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-15T13:09:08.450 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.450 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.450 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.450 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-03-15T13:09:08.450 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-03-15T13:09:08.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.450 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.450 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.450 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2024-03-15T13:09:08.450 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.451 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.451 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.451 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-15T13:09:08.451 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.451 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.451 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.451 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2024-03-15T13:09:08.451 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2024-03-15T13:09:08.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.452 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.452 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.452 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2024-03-15T13:09:08.452 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.452 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.452 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.452 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.452 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.452 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.452 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.453 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-03-15T13:09:08.453 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-03-15T13:09:08.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.453 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.453 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.453 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2024-03-15T13:09:08.453 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.453 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.453 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.453 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2024-03-15T13:09:08.454 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2024-03-15T13:09:08.454 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.454 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.454 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.454 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.454 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.454 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-03-15T13:09:08.454 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-03-15T13:09:08.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.454 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.454 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.455 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-03-15T13:09:08.455 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-03-15T13:09:08.455 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-03-15T13:09:08.455 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.455 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.455 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.455 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.455 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.455 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.455 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.455 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-03-15T13:09:08.455 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.455 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.456 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.456 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.456 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.456 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.456 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.456 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.456 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.456 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.456 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.456 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.456 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.456 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.456 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.456 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.456 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.457 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.457 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.457 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.457 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.457 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.457 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.457 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.457 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.457 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.457 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.457 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.457 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.458 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.458 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.458 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.458 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.458 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.458 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.458 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.458 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.458 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.458 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.458 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.458 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.458 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.458 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.459 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.459 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.459 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.459 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.459 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.459 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.459 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.459 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.459 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.459 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.459 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.459 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.459 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.460 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.460 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.460 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.460 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.460 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.460 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.460 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.460 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.460 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.460 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.460 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2024-03-15T13:09:08.460 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2024-03-15T13:09:08.460 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.462 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.462 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.462 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.462 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.462 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.462 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.462 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.462 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.462 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2024-03-15T13:09:08.462 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2024-03-15T13:09:08.462 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.462 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.462 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.463 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.463 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.463 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.463 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.463 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.463 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.463 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.463 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.464 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2024-03-15T13:09:08.464 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.464 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.464 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.464 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2024-03-15T13:09:08.464 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2024-03-15T13:09:08.464 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.464 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.464 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.464 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.464 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.464 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.464 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.464 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.464 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.465 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.465 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.465 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-03-15T13:09:08.465 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-03-15T13:09:08.465 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.465 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.465 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.465 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.465 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.465 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.465 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.465 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.465 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.466 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.466 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.466 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.466 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.466 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.466 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.466 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.466 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.466 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.466 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.466 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.466 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.466 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.466 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.467 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.467 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.467 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.467 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.467 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.467 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.467 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.467 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.467 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.467 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.467 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.468 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.468 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.468 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.468 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.468 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.468 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.468 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.468 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.468 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.468 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.468 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.469 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.469 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.469 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.469 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.469 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.469 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.469 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.469 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.469 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.469 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.469 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.469 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.469 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.469 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.470 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.470 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.470 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.470 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.470 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.470 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.470 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.470 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.470 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.470 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.470 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.470 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.471 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-03-15T13:09:08.471 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-03-15T13:09:08.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.471 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.471 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.471 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.471 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.471 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.471 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.471 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.471 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.471 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.471 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.471 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-03-15T13:09:08.472 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-03-15T13:09:08.472 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.472 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.472 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.472 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.472 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.472 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.472 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.472 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.472 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.472 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.472 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.473 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2024-03-15T13:09:08.473 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.473 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.473 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.473 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.473 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.473 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.473 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.473 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.473 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.473 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.474 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.474 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.474 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.474 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.474 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.474 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.474 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.474 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.474 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.474 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.474 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.474 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.475 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.475 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.475 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.475 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.475 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.475 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.475 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.475 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.475 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.475 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.475 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.476 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.476 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.476 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.476 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.476 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.476 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.476 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.476 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.476 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.476 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.476 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.476 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.477 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.477 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.477 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.477 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.477 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.477 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.477 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.477 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.477 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.477 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.477 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.477 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.478 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.478 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.478 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.478 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.478 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.478 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.478 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.478 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.478 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.478 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2024-03-15T13:09:08.479 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.479 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.479 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.479 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.479 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.479 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.479 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.479 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.479 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.479 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.479 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.479 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.479 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.479 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.480 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.480 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.480 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.480 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.480 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.480 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.480 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.480 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.480 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.480 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.480 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.480 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.480 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.480 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.482 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.482 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.482 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.482 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.482 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.482 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.482 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.482 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.482 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.482 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.482 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.482 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.482 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.482 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.483 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.483 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.483 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.483 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.483 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.483 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.483 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.483 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.483 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2024-03-15T13:09:08.483 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2024-03-15T13:09:08.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.484 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.484 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.484 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2024-03-15T13:09:08.484 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.484 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.484 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.484 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-15T13:09:08.484 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.484 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.484 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.484 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.484 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.485 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2024-03-15T13:09:08.485 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.485 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.485 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.485 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.485 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.485 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.485 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.485 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.485 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.485 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.485 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.485 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.486 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.486 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.486 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.486 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.486 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.486 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.486 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.486 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.486 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.486 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.486 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.487 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.487 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.487 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.487 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.487 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.487 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.487 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.487 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.487 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.487 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.487 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.487 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.487 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.488 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.488 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.488 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.488 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.488 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.488 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.488 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.488 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.488 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.488 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.488 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.488 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.488 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.488 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.489 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.489 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.489 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.489 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.489 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.489 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.489 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.489 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.489 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.489 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.489 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.489 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.489 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.490 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.490 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.490 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2024-03-15T13:09:08.490 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2024-03-15T13:09:08.490 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.490 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.490 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.490 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.490 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.490 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.490 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.490 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.490 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.490 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.491 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.491 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2024-03-15T13:09:08.491 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2024-03-15T13:09:08.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.491 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.491 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.491 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2024-03-15T13:09:08.491 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.491 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.491 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.491 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.491 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.491 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.491 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.492 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2024-03-15T13:09:08.492 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2024-03-15T13:09:08.492 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.492 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.492 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.492 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.492 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.492 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.492 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.492 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.492 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.492 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.493 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2024-03-15T13:09:08.493 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2024-03-15T13:09:08.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.493 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.493 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.493 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.493 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.493 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.493 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.493 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.493 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.493 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.494 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.494 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2024-03-15T13:09:08.494 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2024-03-15T13:09:08.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-03-15T13:09:08.494 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.494 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.494 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.494 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.494 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.494 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.494 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.494 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.494 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.494 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.495 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2024-03-15T13:09:08.495 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2024-03-15T13:09:08.495 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.495 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.495 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.495 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.495 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.495 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.495 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.495 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.495 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.495 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.495 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.496 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2024-03-15T13:09:08.496 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2024-03-15T13:09:08.496 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.496 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.496 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.496 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.496 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.496 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.496 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.496 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.496 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.496 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.496 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2024-03-15T13:09:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2024-03-15T13:09:08.498 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.498 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.498 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.498 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.498 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-03-15T13:09:08.498 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2024-03-15T13:09:08.498 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.498 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.498 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.498 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.498 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.499 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2024-03-15T13:09:08.499 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2024-03-15T13:09:08.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.499 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.499 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.499 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.499 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.499 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.499 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.499 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.499 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.499 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.499 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.500 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2024-03-15T13:09:08.500 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2024-03-15T13:09:08.500 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.500 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.500 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.500 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.500 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.500 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.500 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.500 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.500 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.500 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.500 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.501 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2024-03-15T13:09:08.501 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2024-03-15T13:09:08.501 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.501 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.501 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.501 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.501 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.501 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.501 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.501 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.501 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.501 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.502 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.502 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.502 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.502 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-03-15T13:09:08.502 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.502 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.502 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.502 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2024-03-15T13:09:08.502 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2024-03-15T13:09:08.502 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.502 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.503 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.503 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.503 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.503 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.503 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.503 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2024-03-15T13:09:08.503 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.503 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.503 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.503 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2024-03-15T13:09:08.503 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2024-03-15T13:09:08.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.503 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.504 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.504 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.504 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.504 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.504 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.504 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2024-03-15T13:09:08.504 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.504 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.504 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.504 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.504 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.504 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.504 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.505 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.505 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.505 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.505 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.505 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.505 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.505 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.505 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.505 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.505 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.505 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.505 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.505 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.505 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.505 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.506 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.506 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.506 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.506 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.506 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.506 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.506 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.506 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.506 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.506 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.506 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.506 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.507 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.507 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.507 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.507 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.507 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.507 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.507 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.507 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.507 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.507 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.507 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.507 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.508 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.508 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.508 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.508 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.508 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.508 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.508 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.508 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.508 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.508 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.508 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.509 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.509 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.509 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.509 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.509 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.509 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.509 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.509 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.509 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.509 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.509 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.510 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2024-03-15T13:09:08.510 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.510 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.510 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.510 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.510 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.510 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.510 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.510 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.510 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.510 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.510 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.510 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.512 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.512 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.512 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.512 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.512 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.512 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.512 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.512 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.512 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.512 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.512 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.513 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.513 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.513 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.513 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.513 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.513 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.513 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.513 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.513 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.513 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.513 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.513 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.513 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.513 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.513 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.514 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.514 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.514 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.514 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.514 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.514 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.514 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.514 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.514 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.514 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.514 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.514 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.514 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.515 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.515 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.515 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2024-03-15T13:09:08.515 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.515 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.515 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.515 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.515 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.515 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.515 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.516 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.516 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.516 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.516 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.516 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.516 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.516 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.516 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.516 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.516 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.516 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.516 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.516 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.516 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.518 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.518 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.518 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.518 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.518 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.518 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.518 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.518 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.518 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.518 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.518 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.518 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.518 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.519 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.519 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.519 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.519 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.519 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.519 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.519 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.519 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.519 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.519 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.519 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.519 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.520 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2024-03-15T13:09:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2024-03-15T13:09:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-15T13:09:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.522 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-03-15T13:09:08.522 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-03-15T13:09:08.522 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.522 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.522 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.522 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2024-03-15T13:09:08.522 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.522 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.522 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.522 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.522 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.522 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.522 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.523 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.523 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.523 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.523 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.523 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.523 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.523 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.523 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.523 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.523 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.523 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.523 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.523 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.523 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.523 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.524 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.524 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.524 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.524 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.524 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.524 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.524 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.524 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.524 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.524 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.524 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.524 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.524 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.524 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.525 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.525 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.525 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.525 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.525 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.525 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.525 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.525 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.525 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.525 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.525 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.525 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.525 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.525 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.526 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.526 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.526 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.526 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.526 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.526 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.526 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.526 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.526 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.526 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.526 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.526 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.528 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.529 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2024-03-15T13:09:08.530 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2024-03-15T13:09:08.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.530 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.530 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.530 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-03-15T13:09:08.530 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.530 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.530 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.530 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-15T13:09:08.530 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.530 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.530 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.531 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2024-03-15T13:09:08.531 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2024-03-15T13:09:08.531 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.531 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.531 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.531 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.531 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.531 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.531 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.531 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-15T13:09:08.531 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.531 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.531 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.532 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2024-03-15T13:09:08.532 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2024-03-15T13:09:08.532 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.532 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.532 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.532 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.532 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.532 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.532 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.532 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-15T13:09:08.532 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.532 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.532 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.533 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2024-03-15T13:09:08.533 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2024-03-15T13:09:08.533 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.533 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.533 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.533 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.533 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.533 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.533 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.533 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.533 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.533 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.533 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.534 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2024-03-15T13:09:08.534 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2024-03-15T13:09:08.534 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.534 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.534 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.534 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.534 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.534 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.534 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.534 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.534 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.534 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.534 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.535 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-03-15T13:09:08.536 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.538 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2024-03-15T13:09:08.538 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.538 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.538 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.538 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.538 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.538 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.538 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.538 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2024-03-15T13:09:08.538 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2024-03-15T13:09:08.538 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.538 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.540 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.541 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.541 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.541 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.541 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.541 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.541 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.541 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.541 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.541 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.541 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.541 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.541 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.541 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.542 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.542 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.542 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.542 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.542 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.542 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.542 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.542 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.542 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.542 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.542 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.543 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.543 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.543 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.543 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.543 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.543 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.543 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.543 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.543 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.543 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.543 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.543 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.543 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.543 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.544 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.544 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.544 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.544 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.544 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.544 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.544 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.544 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.544 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.544 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.544 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.544 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.544 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.544 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.545 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.545 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.545 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2024-03-15T13:09:08.545 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2024-03-15T13:09:08.545 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.545 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.545 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.545 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.545 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.545 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.545 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.545 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.545 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.546 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.546 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.546 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2024-03-15T13:09:08.546 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2024-03-15T13:09:08.546 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.546 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.546 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.546 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.546 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.546 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.546 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.546 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.546 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.547 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.547 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.547 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2024-03-15T13:09:08.547 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2024-03-15T13:09:08.547 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.547 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.547 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.547 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-15T13:09:08.547 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.547 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.547 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.547 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.547 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.548 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.549 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.550 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.550 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.550 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.550 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.550 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.550 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.550 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.550 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.550 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.550 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.550 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.550 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.550 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.551 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.552 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.552 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.552 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.552 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.552 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.552 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.552 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.552 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.552 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.552 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.552 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.552 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.553 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.553 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.553 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.553 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-03-15T13:09:08.553 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.553 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.553 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.553 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.553 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.553 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.553 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.553 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.554 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.554 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.554 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.554 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.554 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.554 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.554 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.554 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.554 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.554 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.554 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.554 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.554 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.554 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.555 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.555 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.555 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.555 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.555 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.555 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.555 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.555 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.555 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.555 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.555 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.555 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.555 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.556 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.556 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.556 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.556 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.556 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.556 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.556 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.556 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.556 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.556 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.556 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.556 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.556 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.556 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.556 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.557 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.557 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.557 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.557 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.557 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.557 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.557 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.557 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.557 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.557 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.557 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.557 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.557 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.557 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.558 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.558 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.558 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.558 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.558 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2024-03-15T13:09:08.558 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2024-03-15T13:09:08.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.558 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.558 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2024-03-15T13:09:08.558 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.558 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.558 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.559 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-15T13:09:08.559 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.559 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.559 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.559 INFO:teuthology.orchestra.run.smithi002.stdout: "periodic_async_work": { 2024-03-15T13:09:08.559 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "periodic_async_work", 2024-03-15T13:09:08.559 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.559 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.559 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.559 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.559 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.559 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.560 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.560 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Periodically check for async work", 2024-03-15T13:09:08.560 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.560 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.560 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.560 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2024-03-15T13:09:08.560 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2024-03-15T13:09:08.560 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.560 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.560 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.560 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-03-15T13:09:08.560 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.560 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-15T13:09:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2024-03-15T13:09:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-03-15T13:09:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-03-15T13:09:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-03-15T13:09:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2024-03-15T13:09:08.562 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2024-03-15T13:09:08.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-03-15T13:09:08.562 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.562 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.562 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2024-03-15T13:09:08.562 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.562 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.562 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.562 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.562 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.562 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.562 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.563 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2024-03-15T13:09:08.563 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2024-03-15T13:09:08.563 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.563 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.563 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.563 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.563 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.563 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.563 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.563 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.563 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.563 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.564 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.565 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.565 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.565 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.565 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.565 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.565 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.565 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.565 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.565 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.565 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.565 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.565 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.565 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.565 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.565 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.567 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.568 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2024-03-15T13:09:08.569 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2024-03-15T13:09:08.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-03-15T13:09:08.570 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.570 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2024-03-15T13:09:08.570 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.570 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.570 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.570 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.570 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.570 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.570 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.570 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2024-03-15T13:09:08.570 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.571 INFO:teuthology.orchestra.run.smithi002.stdout: "services": { 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: "dashboard": "https://172.21.15.2:8443/" 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: "always_on_modules": { 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: "octopus": [ 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.572 INFO:teuthology.orchestra.run.smithi002.stdout: "pacific": [ 2024-03-15T13:09:08.573 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-03-15T13:09:08.573 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-15T13:09:08.573 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-03-15T13:09:08.573 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-03-15T13:09:08.573 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-03-15T13:09:08.573 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-03-15T13:09:08.573 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-03-15T13:09:08.573 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-03-15T13:09:08.573 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-03-15T13:09:08.573 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-03-15T13:09:08.573 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.573 INFO:teuthology.orchestra.run.smithi002.stdout: "quincy": [ 2024-03-15T13:09:08.573 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-03-15T13:09:08.574 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-15T13:09:08.574 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-03-15T13:09:08.574 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-03-15T13:09:08.574 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-03-15T13:09:08.574 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-03-15T13:09:08.574 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-03-15T13:09:08.574 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-03-15T13:09:08.574 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-03-15T13:09:08.574 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-03-15T13:09:08.574 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.574 INFO:teuthology.orchestra.run.smithi002.stdout: "reef": [ 2024-03-15T13:09:08.574 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-03-15T13:09:08.574 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-15T13:09:08.574 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-03-15T13:09:08.575 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-03-15T13:09:08.575 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-03-15T13:09:08.575 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-03-15T13:09:08.575 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-03-15T13:09:08.575 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-03-15T13:09:08.575 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-03-15T13:09:08.575 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-03-15T13:09:08.575 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-15T13:09:08.575 INFO:teuthology.orchestra.run.smithi002.stdout: "squid": [ 2024-03-15T13:09:08.575 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-03-15T13:09:08.575 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-15T13:09:08.575 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-03-15T13:09:08.575 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-03-15T13:09:08.575 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-03-15T13:09:08.575 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: "last_failure_osd_epoch": 92, 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: "active_clients": [ 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 683945830 2024-03-15T13:09:08.576 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.577 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-03-15T13:09:08.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.577 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "libcephsqlite", 2024-03-15T13:09:08.577 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-03-15T13:09:08.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.577 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-03-15T13:09:08.577 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-03-15T13:09:08.577 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2975296973 2024-03-15T13:09:08.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.577 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-03-15T13:09:08.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.577 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-03-15T13:09:08.577 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-03-15T13:09:08.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.578 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-03-15T13:09:08.578 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-03-15T13:09:08.578 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3575560203 2024-03-15T13:09:08.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.578 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-03-15T13:09:08.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.578 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-03-15T13:09:08.578 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-03-15T13:09:08.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.578 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-03-15T13:09:08.578 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 832605498 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2787359583 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.579 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-03-15T13:09:08.580 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-03-15T13:09:08.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-03-15T13:09:08.580 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-03-15T13:09:08.580 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-03-15T13:09:08.580 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2112932763 2024-03-15T13:09:08.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.580 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-03-15T13:09:08.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:08.580 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-03-15T13:09:08.580 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:09:08.581 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:07 smithi184 ceph-mon[35993]: pgmap v315: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-03-15T13:09:08.582 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:07 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2885887196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T13:09:08.582 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:07 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/55841790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T13:09:08.582 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:07 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/519220489' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-15T13:09:08.582 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:07 smithi149 ceph-mon[35927]: pgmap v315: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-03-15T13:09:08.582 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:07 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2885887196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T13:09:08.582 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:07 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/55841790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T13:09:08.582 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:07 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/519220489' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-15T13:09:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.390+0000 7f07a2fe2640 1 -- 172.21.15.2:0/519220489 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f07840a1ba0 msgr2=0x7f07840a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.390+0000 7f07a2fe2640 1 --2- 172.21.15.2:0/519220489 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f07840a1ba0 0x7f07840a4060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f079c198b60 tx=0x7f0798005f90 comp rx=0 tx=0).stop 2024-03-15T13:09:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.390+0000 7f07a2fe2640 1 -- 172.21.15.2:0/519220489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f079c197a20 msgr2=0x7f079c197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.390+0000 7f07a2fe2640 1 --2- 172.21.15.2:0/519220489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f079c197a20 0x7f079c197ed0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f079400eb10 tx=0x7f079400c2d0 comp rx=0 tx=0).stop 2024-03-15T13:09:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.390+0000 7f07a2fe2640 1 -- 172.21.15.2:0/519220489 shutdown_connections 2024-03-15T13:09:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.390+0000 7f07a2fe2640 1 --2- 172.21.15.2:0/519220489 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f07840a1ba0 0x7f07840a4060 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.390+0000 7f07a2fe2640 1 --2- 172.21.15.2:0/519220489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f079c197a20 0x7f079c197ed0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.390+0000 7f07a2fe2640 1 --2- 172.21.15.2:0/519220489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f079c1056f0 0x7f079c19ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.390+0000 7f07a2fe2640 1 --2- 172.21.15.2:0/519220489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f079c100620 0x7f079c19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.390+0000 7f07a2fe2640 1 -- 172.21.15.2:0/519220489 >> 172.21.15.2:0/519220489 conn(0x7f079c077490 msgr2=0x7f079c104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.390+0000 7f07a2fe2640 1 -- 172.21.15.2:0/519220489 shutdown_connections 2024-03-15T13:09:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:07.390+0000 7f07a2fe2640 1 -- 172.21.15.2:0/519220489 wait complete. 2024-03-15T13:09:08.589 DEBUG:tasks.check_counter:Getting stats from a 2024-03-15T13:09:08.589 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-15T13:09:08.589 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:07 smithi002 ceph-mon[32060]: pgmap v315: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-03-15T13:09:08.589 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:07 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2885887196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T13:09:08.590 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:07 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/55841790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T13:09:08.590 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:07 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/519220489' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-15T13:09:08.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:08 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:09:08.866 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:08 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:09:08.878 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:08 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-15T13:09:09.342 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T13:09:09.734 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:09 smithi002 ceph-mon[32060]: pgmap v316: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-03-15T13:09:10.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:09 smithi184 ceph-mon[35993]: pgmap v316: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-03-15T13:09:10.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:09 smithi149 ceph-mon[35927]: pgmap v316: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-03-15T13:09:10.962 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:10 smithi002 ceph-mon[32060]: osdmap e99: 12 total, 12 up, 12 in 2024-03-15T13:09:10.962 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:10 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:10.962 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:10 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:10.962 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:10 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:10.962 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:10 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:10.962 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:10 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:10.962 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:10 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:10.962 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:10 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:10.962 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:10 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:11.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:10 smithi184 ceph-mon[35993]: osdmap e99: 12 total, 12 up, 12 in 2024-03-15T13:09:11.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:10 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:11.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:10 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:11.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:10 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:11.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:10 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:11.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:10 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:11.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:10 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:11.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:10 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:11.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:10 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:11.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:10 smithi149 ceph-mon[35927]: osdmap e99: 12 total, 12 up, 12 in 2024-03-15T13:09:11.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:10 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:11.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:10 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:11.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:10 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:11.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:10 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:11.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:10 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:11.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:10 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:11.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:10 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:11.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:10 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:11.245 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:09:11.245 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-03-15T13:09:11.245 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 1089, 2024-03-15T13:09:11.245 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 1091, 2024-03-15T13:09:11.245 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 4534215, 2024-03-15T13:09:11.245 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 338680, 2024-03-15T13:09:11.245 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 6, 2024-03-15T13:09:11.245 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 2, 2024-03-15T13:09:11.245 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.102047983, 2024-03-15T13:09:11.245 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.034014264, 2024-03-15T13:09:11.246 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.027648813, 2024-03-15T13:09:11.246 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.029955834, 2024-03-15T13:09:11.246 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:11.246 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 1089, 2024-03-15T13:09:11.246 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.041809629, 2024-03-15T13:09:11.246 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000038392 2024-03-15T13:09:11.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.246 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:11.246 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.246 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.246 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.246 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-15T13:09:11.247 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2024-03-15T13:09:11.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.247 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-03-15T13:09:11.247 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 930, 2024-03-15T13:09:11.247 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 895, 2024-03-15T13:09:11.247 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 5370998, 2024-03-15T13:09:11.247 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 336424, 2024-03-15T13:09:11.247 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-03-15T13:09:11.247 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 3, 2024-03-15T13:09:11.247 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.102249347, 2024-03-15T13:09:11.247 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.033318900, 2024-03-15T13:09:11.247 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.053512216, 2024-03-15T13:09:11.247 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.020766103, 2024-03-15T13:09:11.247 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:11.247 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 895, 2024-03-15T13:09:11.248 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.034395370, 2024-03-15T13:09:11.248 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000038430 2024-03-15T13:09:11.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.248 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:11.248 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.248 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.248 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.248 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 1732528, 2024-03-15T13:09:11.248 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 128608 2024-03-15T13:09:11.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.248 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-03-15T13:09:11.248 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 453, 2024-03-15T13:09:11.248 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 612, 2024-03-15T13:09:11.248 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 111288, 2024-03-15T13:09:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 579973, 2024-03-15T13:09:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 7, 2024-03-15T13:09:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 5, 2024-03-15T13:09:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.056282609, 2024-03-15T13:09:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.024403176, 2024-03-15T13:09:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.011430050, 2024-03-15T13:09:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.014797783, 2024-03-15T13:09:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 609, 2024-03-15T13:09:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.028850189, 2024-03-15T13:09:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000047373 2024-03-15T13:09:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.249 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.250 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.250 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-15T13:09:11.250 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 439712 2024-03-15T13:09:11.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.250 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-03-15T13:09:11.250 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 1, 2024-03-15T13:09:11.250 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-03-15T13:09:11.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.250 INFO:teuthology.orchestra.run.smithi002.stdout: "finisher-MDSRank": { 2024-03-15T13:09:11.250 INFO:teuthology.orchestra.run.smithi002.stdout: "queue_len": 0, 2024-03-15T13:09:11.250 INFO:teuthology.orchestra.run.smithi002.stdout: "complete_latency": { 2024-03-15T13:09:11.250 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 3675, 2024-03-15T13:09:11.250 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.092872601, 2024-03-15T13:09:11.250 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000025271 2024-03-15T13:09:11.251 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:11.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.251 INFO:teuthology.orchestra.run.smithi002.stdout: "finisher-PurgeQueue": { 2024-03-15T13:09:11.251 INFO:teuthology.orchestra.run.smithi002.stdout: "queue_len": 0, 2024-03-15T13:09:11.251 INFO:teuthology.orchestra.run.smithi002.stdout: "complete_latency": { 2024-03-15T13:09:11.251 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.251 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.251 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.251 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:11.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.251 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-15T13:09:11.251 INFO:teuthology.orchestra.run.smithi002.stdout: "request": 0, 2024-03-15T13:09:11.251 INFO:teuthology.orchestra.run.smithi002.stdout: "reply": 0, 2024-03-15T13:09:11.251 INFO:teuthology.orchestra.run.smithi002.stdout: "reply_latency": { 2024-03-15T13:09:11.251 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: "slow_reply": 0, 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: "forward": 0, 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_fetch_complete": 0, 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_fetch_keys": 0, 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_commit": 0, 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_split": 0, 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_merge": 0, 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes": 1332, 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_top": 0, 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_bottom": 1332, 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_pin_tail": 0, 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_pinned": 1332, 2024-03-15T13:09:11.252 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_expired": 0, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_with_caps": 0, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": 0, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "subtrees": 1, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse": 0, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_hit": 0, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_forward": 0, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_discover": 0, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_dir_fetch": 0, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_remote_ino": 0, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_lock": 0, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "load_cent": 0, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "q": 0, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "exported": 0, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "exported_inodes": 0, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "imported": 0, 2024-03-15T13:09:11.253 INFO:teuthology.orchestra.run.smithi002.stdout: "imported_inodes": 0, 2024-03-15T13:09:11.254 INFO:teuthology.orchestra.run.smithi002.stdout: "openino_dir_fetch": 0, 2024-03-15T13:09:11.254 INFO:teuthology.orchestra.run.smithi002.stdout: "openino_backtrace_fetch": 0, 2024-03-15T13:09:11.254 INFO:teuthology.orchestra.run.smithi002.stdout: "openino_peer_discover": 0, 2024-03-15T13:09:11.254 INFO:teuthology.orchestra.run.smithi002.stdout: "root_rfiles": 0, 2024-03-15T13:09:11.254 INFO:teuthology.orchestra.run.smithi002.stdout: "root_rbytes": 0, 2024-03-15T13:09:11.254 INFO:teuthology.orchestra.run.smithi002.stdout: "root_rsnaps": 0, 2024-03-15T13:09:11.254 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_backtrace_fetch": 0, 2024-03-15T13:09:11.254 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_set_tag": 0, 2024-03-15T13:09:11.254 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_backtrace_repaired": 0, 2024-03-15T13:09:11.254 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_inotable_repaired": 0, 2024-03-15T13:09:11.254 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_dir_inodes": 0, 2024-03-15T13:09:11.254 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_dir_base_inodes": 0, 2024-03-15T13:09:11.254 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_dirfrag_rstats": 0, 2024-03-15T13:09:11.254 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_file_inodes": 0, 2024-03-15T13:09:11.255 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_inode_file_caps": 0, 2024-03-15T13:09:11.255 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_cap_op_revoke": 0, 2024-03-15T13:09:11.255 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_cap_op_grant": 0, 2024-03-15T13:09:11.255 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_cap_op_trunc": 0, 2024-03-15T13:09:11.255 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-15T13:09:11.255 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-15T13:09:11.255 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_client_caps": 0, 2024-03-15T13:09:11.255 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_client_caps_dirty": 0, 2024-03-15T13:09:11.255 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_client_cap_release": 0, 2024-03-15T13:09:11.255 INFO:teuthology.orchestra.run.smithi002.stdout: "process_request_cap_release": 0 2024-03-15T13:09:11.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.255 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache": { 2024-03-15T13:09:11.255 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_update": 0, 2024-03-15T13:09:11.255 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_update_receipt": 0, 2024-03-15T13:09:11.255 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_try_discover": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_send_discover": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_handle_discover": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "num_strays": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "num_strays_delayed": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "num_strays_enqueuing": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "strays_created": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "strays_enqueued": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "strays_reintegrated": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "strays_migrated": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "num_recovering_processing": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "num_recovering_enqueued": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "num_recovering_prioritized": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_started": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_completed": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_enqueue_scrub": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_exportdir": 0, 2024-03-15T13:09:11.256 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_flush": 0, 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_fragmentdir": 0, 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_fragstats": 0, 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_inodestats": 0 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log": { 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: "evlrg": 0, 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: "evadd": 0, 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: "evex": 0, 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: "evtrm": 0, 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: "ev": 3577, 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: "evexg": 0, 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: "evexd": 0, 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: "segadd": 0, 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: "segex": 0, 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: "segtrm": 0, 2024-03-15T13:09:11.257 INFO:teuthology.orchestra.run.smithi002.stdout: "seg": 5, 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "segmjr": 2, 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "segexg": 0, 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "segexd": 0, 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "expos": 4194304, 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "wrpos": 11854882, 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "rdpos": 11852096, 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "jlat": { 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "replayed": 3577 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_mem": { 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 213, 2024-03-15T13:09:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "ino+": 1650, 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "ino-": 1437, 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "dir": 18, 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "dir+": 145, 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "dir-": 127, 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "dn": 1332, 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "dn+": 2284, 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "dn-": 952, 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "cap": 0, 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "cap+": 0, 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "cap-": 0, 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "rss": 53664, 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "heap": 215420 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_server": { 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "dispatch_client_request": 0, 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "dispatch_server_request": 0, 2024-03-15T13:09:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_client_request": 0, 2024-03-15T13:09:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_client_session": 0, 2024-03-15T13:09:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_peer_request": 0, 2024-03-15T13:09:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "req_create_latency": { 2024-03-15T13:09:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "req_getattr_latency": { 2024-03-15T13:09:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "req_getfilelock_latency": { 2024-03-15T13:09:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "req_link_latency": { 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookup_latency": { 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookuphash_latency": { 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookupino_latency": { 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookupname_latency": { 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookupparent_latency": { 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookupsnap_latency": { 2024-03-15T13:09:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lssnap_latency": { 2024-03-15T13:09:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "req_mkdir_latency": { 2024-03-15T13:09:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "req_mknod_latency": { 2024-03-15T13:09:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "req_mksnap_latency": { 2024-03-15T13:09:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "req_open_latency": { 2024-03-15T13:09:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "req_readdir_latency": { 2024-03-15T13:09:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "req_rename_latency": { 2024-03-15T13:09:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "req_renamesnap_latency": { 2024-03-15T13:09:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "req_snapdiff_latency": { 2024-03-15T13:09:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "req_rmdir_latency": { 2024-03-15T13:09:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "req_rmsnap_latency": { 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "req_rmxattr_latency": { 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "req_setattr_latency": { 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "req_setdirlayout_latency": { 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "req_setfilelock_latency": { 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "req_setlayout_latency": { 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "req_setxattr_latency": { 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "req_symlink_latency": { 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "req_unlink_latency": { 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "cap_revoke_eviction": 0, 2024-03-15T13:09:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "cap_acquisition_throttle": 0, 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "req_getvxattr_latency": { 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_sessions": { 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "session_count": 1, 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "session_add": 4, 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "session_remove": 3, 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "sessions_open": 3, 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "sessions_stale": 0, 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "total_load": 0, 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "average_load": 0, 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "avg_session_uptime": 7343, 2024-03-15T13:09:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "mdthresh_evicted": 0 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-03-15T13:09:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-03-15T13:09:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 198274, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 58, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 9448, 2024-03-15T13:09:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 197, 2024-03-15T13:09:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-03-15T13:09:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-03-15T13:09:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-03-15T13:09:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-03-15T13:09:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 1416549, 2024-03-15T13:09:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 12299, 2024-03-15T13:09:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-03-15T13:09:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-03-15T13:09:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-03-15T13:09:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-03-15T13:09:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter": { 2024-03-15T13:09:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "op_active": 0, 2024-03-15T13:09:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "op_laggy": 0, 2024-03-15T13:09:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "op_send": 2208, 2024-03-15T13:09:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "op_send_bytes": 21, 2024-03-15T13:09:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "op_resend": 3, 2024-03-15T13:09:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "op_reply": 2208, 2024-03-15T13:09:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "op_latency": { 2024-03-15T13:09:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 2208, 2024-03-15T13:09:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 1.052975458, 2024-03-15T13:09:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000476891 2024-03-15T13:09:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "op_inflight": 0, 2024-03-15T13:09:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "oplen_avg": { 2024-03-15T13:09:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 2208, 2024-03-15T13:09:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 2209 2024-03-15T13:09:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "op": 2208, 2024-03-15T13:09:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "op_r": 2208, 2024-03-15T13:09:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "op_w": 0, 2024-03-15T13:09:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "op_rmw": 0, 2024-03-15T13:09:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "op_pg": 0, 2024-03-15T13:09:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_stat": 1381, 2024-03-15T13:09:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_create": 0, 2024-03-15T13:09:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_read": 825, 2024-03-15T13:09:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_write": 0, 2024-03-15T13:09:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_writefull": 0, 2024-03-15T13:09:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_writesame": 0, 2024-03-15T13:09:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_append": 0, 2024-03-15T13:09:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_zero": 0, 2024-03-15T13:09:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_truncate": 0, 2024-03-15T13:09:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_delete": 0, 2024-03-15T13:09:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_mapext": 0, 2024-03-15T13:09:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_sparse_read": 0, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_clonerange": 0, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_getxattr": 1, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_setxattr": 0, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_cmpxattr": 0, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_rmxattr": 0, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_resetxattrs": 0, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_call": 0, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_watch": 0, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_notify": 0, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_src_cmpxattr": 0, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_pgls": 0, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_pgls_filter": 0, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_other": 0, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "linger_active": 0, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "linger_send": 0, 2024-03-15T13:09:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "linger_resend": 0, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "linger_ping": 0, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "poolop_active": 0, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "poolop_send": 0, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "poolop_resend": 0, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "poolstat_active": 0, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "poolstat_send": 0, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "poolstat_resend": 0, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "statfs_active": 0, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "statfs_send": 0, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "statfs_resend": 0, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "command_active": 0, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "command_send": 0, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "command_resend": 0, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "map_epoch": 99, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "map_full": 0, 2024-03-15T13:09:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "map_inc": 10, 2024-03-15T13:09:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_sessions": 6, 2024-03-15T13:09:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_session_open": 6, 2024-03-15T13:09:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_session_close": 0, 2024-03-15T13:09:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_laggy": 0, 2024-03-15T13:09:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_wr": 0, 2024-03-15T13:09:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_rd": 2, 2024-03-15T13:09:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_del": 0 2024-03-15T13:09:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "oft": { 2024-03-15T13:09:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_total_objs": 0, 2024-03-15T13:09:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_total_kv_pairs": 0, 2024-03-15T13:09:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_total_updates": 0, 2024-03-15T13:09:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_total_removes": 0 2024-03-15T13:09:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "osd-slow-ops": { 2024-03-15T13:09:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "slow_ops_count": 0 2024-03-15T13:09:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "purge_queue": { 2024-03-15T13:09:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executing_ops": 0, 2024-03-15T13:09:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executing_ops_high_water": 0, 2024-03-15T13:09:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executing": 0, 2024-03-15T13:09:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executing_high_water": 0, 2024-03-15T13:09:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executed_ops": 0, 2024-03-15T13:09:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executed": 0, 2024-03-15T13:09:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_item_in_journal": 0 2024-03-15T13:09:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-15T13:09:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-03-15T13:09:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 2468, 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 9778289, 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 2468, 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 2468, 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 9778289, 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-objecter_bytes": { 2024-03-15T13:09:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-03-15T13:09:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-03-15T13:09:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-03-15T13:09:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 0, 2024-03-15T13:09:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 0, 2024-03-15T13:09:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 2208, 2024-03-15T13:09:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 7660583, 2024-03-15T13:09:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 2208, 2024-03-15T13:09:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 7660583, 2024-03-15T13:09:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-03-15T13:09:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-objecter_ops": { 2024-03-15T13:09:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-03-15T13:09:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 1024, 2024-03-15T13:09:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-03-15T13:09:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 0, 2024-03-15T13:09:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 0, 2024-03-15T13:09:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 2208, 2024-03-15T13:09:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 2208, 2024-03-15T13:09:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 2208, 2024-03-15T13:09:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 2208, 2024-03-15T13:09:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-03-15T13:09:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-write_buf_throttle": { 2024-03-15T13:09:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-03-15T13:09:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 3758096384, 2024-03-15T13:09:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-03-15T13:09:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 0, 2024-03-15T13:09:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 0, 2024-03-15T13:09:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-03-15T13:09:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-03-15T13:09:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 0, 2024-03-15T13:09:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 0, 2024-03-15T13:09:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-03-15T13:09:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-write_buf_throttle-0x55e15de5e960": { 2024-03-15T13:09:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-03-15T13:09:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 3758096384, 2024-03-15T13:09:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-03-15T13:09:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 0, 2024-03-15T13:09:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 0, 2024-03-15T13:09:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-03-15T13:09:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-03-15T13:09:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 0, 2024-03-15T13:09:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 0, 2024-03-15T13:09:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-03-15T13:09:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:11.286 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:09:11.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:11 smithi002 ceph-mon[32060]: pgmap v318: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:09:11.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:11 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi184 to 176.2M 2024-03-15T13:09:11.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:11 smithi002 ceph-mon[32060]: Unable to set osd_memory_target on smithi184 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-03-15T13:09:11.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:11 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:12.051 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-15T13:09:12.051 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-15T13:09:12.052 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-03-15T13:09:12.052 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-15T13:09:12.052 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-15T13:09:12.052 DEBUG:tasks.check_counter:Getting stats from d 2024-03-15T13:09:12.052 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-15T13:09:12.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:11 smithi184 ceph-mon[35993]: pgmap v318: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:09:12.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:11 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi184 to 176.2M 2024-03-15T13:09:12.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:11 smithi184 ceph-mon[35993]: Unable to set osd_memory_target on smithi184 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-03-15T13:09:12.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:11 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:12.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:11 smithi149 ceph-mon[35927]: pgmap v318: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:09:12.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:11 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi184 to 176.2M 2024-03-15T13:09:12.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:11 smithi149 ceph-mon[35927]: Unable to set osd_memory_target on smithi184 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-03-15T13:09:12.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:11 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:12.319 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T13:09:13.464 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:13 smithi002 ceph-mon[32060]: pgmap v319: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:09:13.465 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:13 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:13.465 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:13 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:13.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:13 smithi184 ceph-mon[35993]: pgmap v319: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:09:13.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:13 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:13.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:13 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:13.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:13 smithi149 ceph-mon[35927]: pgmap v319: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:09:13.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:13 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:13.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:13 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:14.083 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:09:14.083 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-03-15T13:09:14.083 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2024-03-15T13:09:14.083 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2024-03-15T13:09:14.083 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2024-03-15T13:09:14.083 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2024-03-15T13:09:14.083 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-03-15T13:09:14.083 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2024-03-15T13:09:14.083 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.001065714, 2024-03-15T13:09:14.083 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2024-03-15T13:09:14.084 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000181064, 2024-03-15T13:09:14.084 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-15T13:09:14.084 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:14.084 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:14.084 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:14.084 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:14.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:14.084 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:14.084 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:14.084 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:14.084 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:14.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:14.084 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-15T13:09:14.084 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2024-03-15T13:09:14.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:14.085 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-03-15T13:09:14.085 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 6, 2024-03-15T13:09:14.085 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 166, 2024-03-15T13:09:14.085 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 3312, 2024-03-15T13:09:14.085 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 184080, 2024-03-15T13:09:14.085 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 5, 2024-03-15T13:09:14.085 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 3, 2024-03-15T13:09:14.085 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.013412729, 2024-03-15T13:09:14.085 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.008803777, 2024-03-15T13:09:14.085 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.002351237, 2024-03-15T13:09:14.085 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-15T13:09:14.085 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:14.085 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 164, 2024-03-15T13:09:14.086 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.014999510, 2024-03-15T13:09:14.086 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000091460 2024-03-15T13:09:14.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:14.086 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:14.086 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:14.086 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:14.086 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:14.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:14.086 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-15T13:09:14.086 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 184080 2024-03-15T13:09:14.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:14.086 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-03-15T13:09:14.086 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 227, 2024-03-15T13:09:14.086 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 216, 2024-03-15T13:09:14.087 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 1565120, 2024-03-15T13:09:14.087 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 129312, 2024-03-15T13:09:14.087 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2024-03-15T13:09:14.087 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2024-03-15T13:09:14.087 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.045442229, 2024-03-15T13:09:14.087 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.014387316, 2024-03-15T13:09:14.087 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.036288987, 2024-03-15T13:09:14.087 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.004202787, 2024-03-15T13:09:14.087 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:14.087 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 215, 2024-03-15T13:09:14.087 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.010161596, 2024-03-15T13:09:14.087 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000047263 2024-03-15T13:09:14.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:14.087 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:14.087 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:14.088 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:14.088 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:14.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:14.088 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 1565120, 2024-03-15T13:09:14.088 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 129312 2024-03-15T13:09:14.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:14.088 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-03-15T13:09:14.088 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2024-03-15T13:09:14.088 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-03-15T13:09:14.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:14.088 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-03-15T13:09:14.088 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-03-15T13:09:14.088 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-03-15T13:09:14.088 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-03-15T13:09:14.089 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-03-15T13:09:14.089 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-03-15T13:09:14.089 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-03-15T13:09:14.089 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-15T13:09:14.089 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-03-15T13:09:14.089 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-15T13:09:14.089 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-03-15T13:09:14.089 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-03-15T13:09:14.089 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-03-15T13:09:14.089 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-15T13:09:14.089 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-03-15T13:09:14.089 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-03-15T13:09:14.089 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-03-15T13:09:14.089 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-03-15T13:09:14.089 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-03-15T13:09:14.090 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-15T13:09:14.090 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-03-15T13:09:14.090 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-15T13:09:14.090 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-03-15T13:09:14.090 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-03-15T13:09:14.090 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-03-15T13:09:14.090 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-03-15T13:09:14.090 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-03-15T13:09:14.090 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-15T13:09:14.090 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-03-15T13:09:14.090 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-03-15T13:09:14.090 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-03-15T13:09:14.091 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-03-15T13:09:14.091 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-03-15T13:09:14.091 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-03-15T13:09:14.091 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-03-15T13:09:14.091 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-03-15T13:09:14.091 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-03-15T13:09:14.091 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148714, 2024-03-15T13:09:14.091 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 30, 2024-03-15T13:09:14.091 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-03-15T13:09:14.091 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-03-15T13:09:14.091 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-03-15T13:09:14.091 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-03-15T13:09:14.091 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-03-15T13:09:14.091 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-03-15T13:09:14.091 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-15T13:09:14.092 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 229, 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 1533131, 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 229, 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 229, 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 1533131, 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:14.093 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:14.094 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:09:14.536 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:14 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:14.536 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:14 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:14.536 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:14 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:14.536 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:14 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:14.536 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:14 smithi002 ceph-mon[32060]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-03-15T13:09:14.536 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:14 smithi002 ceph-mon[32060]: Unable to set osd_memory_target on smithi002 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-15T13:09:14.536 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:14 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:09:14.536 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:14 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:09:14.536 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:14 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:14.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:14 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:14.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:14 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:14.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:14 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:14.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:14 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:14.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:14 smithi184 ceph-mon[35993]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-03-15T13:09:14.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:14 smithi184 ceph-mon[35993]: Unable to set osd_memory_target on smithi002 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-15T13:09:14.558 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:14 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:09:14.558 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:14 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:09:14.558 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:14 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:14.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:14 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:14.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:14 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:14.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:14 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:14.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:14 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-15T13:09:14.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:14 smithi149 ceph-mon[35927]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-03-15T13:09:14.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:14 smithi149 ceph-mon[35927]: Unable to set osd_memory_target on smithi002 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-15T13:09:14.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:14 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-15T13:09:14.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:14 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-15T13:09:14.609 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:14 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' 2024-03-15T13:09:14.885 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-15T13:09:14.885 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-15T13:09:14.886 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-15T13:09:14.886 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-15T13:09:14.886 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-15T13:09:14.886 DEBUG:tasks.check_counter:Getting stats from g 2024-03-15T13:09:14.886 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-15T13:09:15.136 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T13:09:15.515 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:15 smithi002 ceph-mon[32060]: pgmap v320: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:09:15.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:15 smithi184 ceph-mon[35993]: pgmap v320: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:09:15.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:15 smithi149 ceph-mon[35927]: pgmap v320: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-03-15T13:09:16.630 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:16 smithi002 ceph-mon[32060]: osdmap e100: 12 total, 12 up, 12 in 2024-03-15T13:09:16.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:16 smithi184 ceph-mon[35993]: osdmap e100: 12 total, 12 up, 12 in 2024-03-15T13:09:16.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:16 smithi149 ceph-mon[35927]: osdmap e100: 12 total, 12 up, 12 in 2024-03-15T13:09:17.014 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:09:17.014 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-03-15T13:09:17.014 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 754, 2024-03-15T13:09:17.014 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 756, 2024-03-15T13:09:17.014 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 253289, 2024-03-15T13:09:17.014 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 234495, 2024-03-15T13:09:17.014 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 6, 2024-03-15T13:09:17.014 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 2, 2024-03-15T13:09:17.014 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.082763498, 2024-03-15T13:09:17.017 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.035148412, 2024-03-15T13:09:17.017 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.017460850, 2024-03-15T13:09:17.017 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.021762594, 2024-03-15T13:09:17.017 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:17.017 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 754, 2024-03-15T13:09:17.017 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.036954961, 2024-03-15T13:09:17.018 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000049011 2024-03-15T13:09:17.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.018 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:17.018 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.018 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.018 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.018 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-15T13:09:17.018 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2024-03-15T13:09:17.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.018 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-03-15T13:09:17.018 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 1722, 2024-03-15T13:09:17.018 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 1687, 2024-03-15T13:09:17.019 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 4938526, 2024-03-15T13:09:17.019 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 583529, 2024-03-15T13:09:17.019 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-03-15T13:09:17.019 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 3, 2024-03-15T13:09:17.019 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.133990188, 2024-03-15T13:09:17.019 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.039533881, 2024-03-15T13:09:17.019 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.062851671, 2024-03-15T13:09:17.019 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.037627216, 2024-03-15T13:09:17.019 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:17.019 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 1687, 2024-03-15T13:09:17.019 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.048629338, 2024-03-15T13:09:17.019 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000028825 2024-03-15T13:09:17.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.019 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:17.019 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.020 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.020 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.020 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 1733856, 2024-03-15T13:09:17.020 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 130656 2024-03-15T13:09:17.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.020 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-03-15T13:09:17.020 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 5, 2024-03-15T13:09:17.020 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 165, 2024-03-15T13:09:17.020 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 1668, 2024-03-15T13:09:17.020 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 442432, 2024-03-15T13:09:17.020 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 7, 2024-03-15T13:09:17.020 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 5, 2024-03-15T13:09:17.020 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.013435799, 2024-03-15T13:09:17.020 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.008542635, 2024-03-15T13:09:17.021 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.002856003, 2024-03-15T13:09:17.021 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000061657, 2024-03-15T13:09:17.021 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:17.021 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 162, 2024-03-15T13:09:17.021 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.012653053, 2024-03-15T13:09:17.021 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000078105 2024-03-15T13:09:17.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.021 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:17.021 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.021 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.021 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.021 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-15T13:09:17.021 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 441488 2024-03-15T13:09:17.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.022 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-03-15T13:09:17.022 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 1, 2024-03-15T13:09:17.022 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-03-15T13:09:17.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.022 INFO:teuthology.orchestra.run.smithi002.stdout: "finisher-MDSRank": { 2024-03-15T13:09:17.022 INFO:teuthology.orchestra.run.smithi002.stdout: "queue_len": 0, 2024-03-15T13:09:17.022 INFO:teuthology.orchestra.run.smithi002.stdout: "complete_latency": { 2024-03-15T13:09:17.022 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 3745, 2024-03-15T13:09:17.022 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.090041775, 2024-03-15T13:09:17.022 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000024043 2024-03-15T13:09:17.022 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:17.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.023 INFO:teuthology.orchestra.run.smithi002.stdout: "finisher-PurgeQueue": { 2024-03-15T13:09:17.023 INFO:teuthology.orchestra.run.smithi002.stdout: "queue_len": 0, 2024-03-15T13:09:17.023 INFO:teuthology.orchestra.run.smithi002.stdout: "complete_latency": { 2024-03-15T13:09:17.023 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.023 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.023 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:17.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.023 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-15T13:09:17.023 INFO:teuthology.orchestra.run.smithi002.stdout: "request": 0, 2024-03-15T13:09:17.023 INFO:teuthology.orchestra.run.smithi002.stdout: "reply": 0, 2024-03-15T13:09:17.023 INFO:teuthology.orchestra.run.smithi002.stdout: "reply_latency": { 2024-03-15T13:09:17.023 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.023 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.023 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.024 INFO:teuthology.orchestra.run.smithi002.stdout: "slow_reply": 0, 2024-03-15T13:09:17.024 INFO:teuthology.orchestra.run.smithi002.stdout: "forward": 0, 2024-03-15T13:09:17.024 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_fetch_complete": 0, 2024-03-15T13:09:17.024 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_fetch_keys": 0, 2024-03-15T13:09:17.024 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_commit": 0, 2024-03-15T13:09:17.024 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_split": 0, 2024-03-15T13:09:17.024 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_merge": 0, 2024-03-15T13:09:17.024 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes": 659, 2024-03-15T13:09:17.024 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_top": 0, 2024-03-15T13:09:17.024 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_bottom": 659, 2024-03-15T13:09:17.024 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_pin_tail": 0, 2024-03-15T13:09:17.024 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_pinned": 659, 2024-03-15T13:09:17.024 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_expired": 0, 2024-03-15T13:09:17.024 INFO:teuthology.orchestra.run.smithi002.stdout: "inodes_with_caps": 0, 2024-03-15T13:09:17.025 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": 0, 2024-03-15T13:09:17.025 INFO:teuthology.orchestra.run.smithi002.stdout: "subtrees": 1, 2024-03-15T13:09:17.025 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse": 0, 2024-03-15T13:09:17.025 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_hit": 0, 2024-03-15T13:09:17.025 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_forward": 0, 2024-03-15T13:09:17.025 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_discover": 0, 2024-03-15T13:09:17.025 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_dir_fetch": 0, 2024-03-15T13:09:17.025 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_remote_ino": 0, 2024-03-15T13:09:17.025 INFO:teuthology.orchestra.run.smithi002.stdout: "traverse_lock": 0, 2024-03-15T13:09:17.025 INFO:teuthology.orchestra.run.smithi002.stdout: "load_cent": 0, 2024-03-15T13:09:17.025 INFO:teuthology.orchestra.run.smithi002.stdout: "q": 0, 2024-03-15T13:09:17.025 INFO:teuthology.orchestra.run.smithi002.stdout: "exported": 0, 2024-03-15T13:09:17.025 INFO:teuthology.orchestra.run.smithi002.stdout: "exported_inodes": 0, 2024-03-15T13:09:17.025 INFO:teuthology.orchestra.run.smithi002.stdout: "imported": 0, 2024-03-15T13:09:17.025 INFO:teuthology.orchestra.run.smithi002.stdout: "imported_inodes": 0, 2024-03-15T13:09:17.026 INFO:teuthology.orchestra.run.smithi002.stdout: "openino_dir_fetch": 0, 2024-03-15T13:09:17.026 INFO:teuthology.orchestra.run.smithi002.stdout: "openino_backtrace_fetch": 0, 2024-03-15T13:09:17.026 INFO:teuthology.orchestra.run.smithi002.stdout: "openino_peer_discover": 0, 2024-03-15T13:09:17.026 INFO:teuthology.orchestra.run.smithi002.stdout: "root_rfiles": 0, 2024-03-15T13:09:17.026 INFO:teuthology.orchestra.run.smithi002.stdout: "root_rbytes": 0, 2024-03-15T13:09:17.026 INFO:teuthology.orchestra.run.smithi002.stdout: "root_rsnaps": 0, 2024-03-15T13:09:17.026 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_backtrace_fetch": 0, 2024-03-15T13:09:17.026 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_set_tag": 0, 2024-03-15T13:09:17.026 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_backtrace_repaired": 0, 2024-03-15T13:09:17.026 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_inotable_repaired": 0, 2024-03-15T13:09:17.026 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_dir_inodes": 0, 2024-03-15T13:09:17.026 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_dir_base_inodes": 0, 2024-03-15T13:09:17.026 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_dirfrag_rstats": 0, 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_file_inodes": 0, 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_inode_file_caps": 0, 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_cap_op_revoke": 0, 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_cap_op_grant": 0, 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_cap_op_trunc": 0, 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_client_caps": 0, 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_client_caps_dirty": 0, 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_client_cap_release": 0, 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: "process_request_cap_release": 0 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache": { 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_update": 0, 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_update_receipt": 0, 2024-03-15T13:09:17.027 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_try_discover": 0, 2024-03-15T13:09:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_send_discover": 0, 2024-03-15T13:09:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: "dir_handle_discover": 0, 2024-03-15T13:09:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: "num_strays": 0, 2024-03-15T13:09:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: "num_strays_delayed": 0, 2024-03-15T13:09:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: "num_strays_enqueuing": 0, 2024-03-15T13:09:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: "strays_created": 0, 2024-03-15T13:09:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: "strays_enqueued": 0, 2024-03-15T13:09:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: "strays_reintegrated": 0, 2024-03-15T13:09:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: "strays_migrated": 0, 2024-03-15T13:09:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: "num_recovering_processing": 0, 2024-03-15T13:09:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: "num_recovering_enqueued": 0, 2024-03-15T13:09:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: "num_recovering_prioritized": 0, 2024-03-15T13:09:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_started": 0, 2024-03-15T13:09:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_completed": 0, 2024-03-15T13:09:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_enqueue_scrub": 0, 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_exportdir": 0, 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_flush": 0, 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_fragmentdir": 0, 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_fragstats": 0, 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: "ireq_inodestats": 0 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log": { 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: "evlrg": 0, 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: "evadd": 0, 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: "evex": 0, 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: "evtrm": 0, 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: "ev": 1382, 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: "evexg": 0, 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: "evexd": 0, 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: "segadd": 0, 2024-03-15T13:09:17.029 INFO:teuthology.orchestra.run.smithi002.stdout: "segex": 0, 2024-03-15T13:09:17.030 INFO:teuthology.orchestra.run.smithi002.stdout: "segtrm": 0, 2024-03-15T13:09:17.030 INFO:teuthology.orchestra.run.smithi002.stdout: "seg": 3, 2024-03-15T13:09:17.030 INFO:teuthology.orchestra.run.smithi002.stdout: "segmjr": 2, 2024-03-15T13:09:17.030 INFO:teuthology.orchestra.run.smithi002.stdout: "segexg": 0, 2024-03-15T13:09:17.030 INFO:teuthology.orchestra.run.smithi002.stdout: "segexd": 0, 2024-03-15T13:09:17.030 INFO:teuthology.orchestra.run.smithi002.stdout: "expos": 4194304, 2024-03-15T13:09:17.030 INFO:teuthology.orchestra.run.smithi002.stdout: "wrpos": 7028686, 2024-03-15T13:09:17.030 INFO:teuthology.orchestra.run.smithi002.stdout: "rdpos": 7026125, 2024-03-15T13:09:17.030 INFO:teuthology.orchestra.run.smithi002.stdout: "jlat": { 2024-03-15T13:09:17.030 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.030 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.030 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.030 INFO:teuthology.orchestra.run.smithi002.stdout: "replayed": 1382 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_mem": { 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: "ino": 212, 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: "ino+": 815, 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: "ino-": 603, 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: "dir": 17, 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: "dir+": 92, 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: "dir-": 75, 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: "dn": 659, 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: "dn+": 1154, 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: "dn-": 495, 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: "cap": 0, 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: "cap+": 0, 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: "cap-": 0, 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rss": 52340, 2024-03-15T13:09:17.031 INFO:teuthology.orchestra.run.smithi002.stdout: "heap": 215420 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_server": { 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: "dispatch_client_request": 0, 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: "dispatch_server_request": 0, 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_client_request": 0, 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_client_session": 0, 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: "handle_peer_request": 0, 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: "req_create_latency": { 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: "req_getattr_latency": { 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.032 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.033 INFO:teuthology.orchestra.run.smithi002.stdout: "req_getfilelock_latency": { 2024-03-15T13:09:17.033 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.033 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.033 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.033 INFO:teuthology.orchestra.run.smithi002.stdout: "req_link_latency": { 2024-03-15T13:09:17.033 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.033 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.033 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.033 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookup_latency": { 2024-03-15T13:09:17.033 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.033 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.033 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.034 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookuphash_latency": { 2024-03-15T13:09:17.034 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.034 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.034 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.034 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookupino_latency": { 2024-03-15T13:09:17.034 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.034 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.034 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.034 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookupname_latency": { 2024-03-15T13:09:17.034 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.034 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookupparent_latency": { 2024-03-15T13:09:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lookupsnap_latency": { 2024-03-15T13:09:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: "req_lssnap_latency": { 2024-03-15T13:09:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.036 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.036 INFO:teuthology.orchestra.run.smithi002.stdout: "req_mkdir_latency": { 2024-03-15T13:09:17.036 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.036 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.036 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.037 INFO:teuthology.orchestra.run.smithi002.stdout: "req_mknod_latency": { 2024-03-15T13:09:17.037 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.037 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.037 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.037 INFO:teuthology.orchestra.run.smithi002.stdout: "req_mksnap_latency": { 2024-03-15T13:09:17.037 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.037 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.037 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.037 INFO:teuthology.orchestra.run.smithi002.stdout: "req_open_latency": { 2024-03-15T13:09:17.037 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.037 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.037 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.038 INFO:teuthology.orchestra.run.smithi002.stdout: "req_readdir_latency": { 2024-03-15T13:09:17.038 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.038 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.038 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.038 INFO:teuthology.orchestra.run.smithi002.stdout: "req_rename_latency": { 2024-03-15T13:09:17.038 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.038 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.038 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.038 INFO:teuthology.orchestra.run.smithi002.stdout: "req_renamesnap_latency": { 2024-03-15T13:09:17.038 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.039 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.039 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.039 INFO:teuthology.orchestra.run.smithi002.stdout: "req_snapdiff_latency": { 2024-03-15T13:09:17.039 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.039 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.039 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.039 INFO:teuthology.orchestra.run.smithi002.stdout: "req_rmdir_latency": { 2024-03-15T13:09:17.039 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.039 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.039 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.039 INFO:teuthology.orchestra.run.smithi002.stdout: "req_rmsnap_latency": { 2024-03-15T13:09:17.039 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: "req_rmxattr_latency": { 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: "req_setattr_latency": { 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: "req_setdirlayout_latency": { 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.040 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: "req_setfilelock_latency": { 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: "req_setlayout_latency": { 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: "req_setxattr_latency": { 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.041 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.042 INFO:teuthology.orchestra.run.smithi002.stdout: "req_symlink_latency": { 2024-03-15T13:09:17.042 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.042 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.042 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.042 INFO:teuthology.orchestra.run.smithi002.stdout: "req_unlink_latency": { 2024-03-15T13:09:17.042 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.042 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.042 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.042 INFO:teuthology.orchestra.run.smithi002.stdout: "cap_revoke_eviction": 0, 2024-03-15T13:09:17.042 INFO:teuthology.orchestra.run.smithi002.stdout: "cap_acquisition_throttle": 0, 2024-03-15T13:09:17.042 INFO:teuthology.orchestra.run.smithi002.stdout: "req_getvxattr_latency": { 2024-03-15T13:09:17.042 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_sessions": { 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: "session_count": 2, 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: "session_add": 4, 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: "session_remove": 2, 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: "sessions_open": 3, 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: "sessions_stale": 0, 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: "total_load": 0, 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: "average_load": 0, 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: "avg_session_uptime": 3984, 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: "mdthresh_evicted": 0 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-03-15T13:09:17.044 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-03-15T13:09:17.044 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-03-15T13:09:17.044 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-03-15T13:09:17.044 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-03-15T13:09:17.044 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-03-15T13:09:17.044 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-03-15T13:09:17.044 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-15T13:09:17.044 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-03-15T13:09:17.044 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-15T13:09:17.044 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-03-15T13:09:17.044 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-03-15T13:09:17.044 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-03-15T13:09:17.044 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-15T13:09:17.044 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-03-15T13:09:17.044 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-03-15T13:09:17.045 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-03-15T13:09:17.046 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-03-15T13:09:17.046 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-03-15T13:09:17.046 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-03-15T13:09:17.046 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-03-15T13:09:17.046 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-03-15T13:09:17.046 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 197473, 2024-03-15T13:09:17.046 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 55, 2024-03-15T13:09:17.046 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-03-15T13:09:17.046 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-03-15T13:09:17.046 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-03-15T13:09:17.046 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-03-15T13:09:17.046 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-03-15T13:09:17.046 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-03-15T13:09:17.046 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-03-15T13:09:17.046 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 9688, 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 203, 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 936333, 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 9369, 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter": { 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: "op_active": 0, 2024-03-15T13:09:17.047 INFO:teuthology.orchestra.run.smithi002.stdout: "op_laggy": 0, 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: "op_send": 2213, 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: "op_send_bytes": 21, 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: "op_resend": 3, 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: "op_reply": 2213, 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: "op_latency": { 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 2213, 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.976977226, 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000441471 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: "op_inflight": 0, 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: "oplen_avg": { 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 2213, 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 2214 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: "op": 2213, 2024-03-15T13:09:17.048 INFO:teuthology.orchestra.run.smithi002.stdout: "op_r": 2213, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "op_w": 0, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "op_rmw": 0, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "op_pg": 0, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_stat": 1393, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_create": 0, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_read": 818, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_write": 0, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_writefull": 0, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_writesame": 0, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_append": 0, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_zero": 0, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_truncate": 0, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_delete": 0, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_mapext": 0, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_sparse_read": 0, 2024-03-15T13:09:17.049 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_clonerange": 0, 2024-03-15T13:09:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_getxattr": 1, 2024-03-15T13:09:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_setxattr": 0, 2024-03-15T13:09:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_cmpxattr": 0, 2024-03-15T13:09:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_rmxattr": 0, 2024-03-15T13:09:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_resetxattrs": 0, 2024-03-15T13:09:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_call": 0, 2024-03-15T13:09:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_watch": 0, 2024-03-15T13:09:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_notify": 0, 2024-03-15T13:09:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_src_cmpxattr": 0, 2024-03-15T13:09:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_pgls": 0, 2024-03-15T13:09:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_pgls_filter": 0, 2024-03-15T13:09:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "osdop_other": 0, 2024-03-15T13:09:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "linger_active": 0, 2024-03-15T13:09:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "linger_send": 0, 2024-03-15T13:09:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "linger_resend": 0, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "linger_ping": 0, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "poolop_active": 0, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "poolop_send": 0, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "poolop_resend": 0, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "poolstat_active": 0, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "poolstat_send": 0, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "poolstat_resend": 0, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "statfs_active": 0, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "statfs_send": 0, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "statfs_resend": 0, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "command_active": 0, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "command_send": 0, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "command_resend": 0, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "map_epoch": 100, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "map_full": 0, 2024-03-15T13:09:17.051 INFO:teuthology.orchestra.run.smithi002.stdout: "map_inc": 11, 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_sessions": 6, 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_session_open": 6, 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_session_close": 0, 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_laggy": 0, 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_wr": 0, 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_rd": 2, 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_del": 0 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: "oft": { 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_total_objs": 0, 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_total_kv_pairs": 0, 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_total_updates": 0, 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: "omap_total_removes": 0 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: "osd-slow-ops": { 2024-03-15T13:09:17.052 INFO:teuthology.orchestra.run.smithi002.stdout: "slow_ops_count": 0 2024-03-15T13:09:17.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.053 INFO:teuthology.orchestra.run.smithi002.stdout: "purge_queue": { 2024-03-15T13:09:17.053 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executing_ops": 0, 2024-03-15T13:09:17.053 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executing_ops_high_water": 0, 2024-03-15T13:09:17.053 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executing": 0, 2024-03-15T13:09:17.053 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executing_high_water": 0, 2024-03-15T13:09:17.053 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executed_ops": 0, 2024-03-15T13:09:17.053 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_executed": 0, 2024-03-15T13:09:17.053 INFO:teuthology.orchestra.run.smithi002.stdout: "pq_item_in_journal": 0 2024-03-15T13:09:17.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.053 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-15T13:09:17.053 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-03-15T13:09:17.053 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-03-15T13:09:17.053 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-03-15T13:09:17.053 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 2477, 2024-03-15T13:09:17.054 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 4954259, 2024-03-15T13:09:17.054 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:17.054 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 2477, 2024-03-15T13:09:17.054 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-03-15T13:09:17.054 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-03-15T13:09:17.054 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 2477, 2024-03-15T13:09:17.054 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 4954259, 2024-03-15T13:09:17.054 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-03-15T13:09:17.054 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.054 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.054 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.054 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:17.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.054 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-objecter_bytes": { 2024-03-15T13:09:17.054 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-03-15T13:09:17.055 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-03-15T13:09:17.055 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-03-15T13:09:17.055 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 0, 2024-03-15T13:09:17.055 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 0, 2024-03-15T13:09:17.055 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:17.055 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:17.055 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 2213, 2024-03-15T13:09:17.055 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 2834387, 2024-03-15T13:09:17.055 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 2213, 2024-03-15T13:09:17.055 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 2834387, 2024-03-15T13:09:17.055 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-03-15T13:09:17.055 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.055 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.055 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:17.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.056 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-objecter_ops": { 2024-03-15T13:09:17.056 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-03-15T13:09:17.056 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 1024, 2024-03-15T13:09:17.056 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-03-15T13:09:17.056 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 0, 2024-03-15T13:09:17.056 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 0, 2024-03-15T13:09:17.056 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:17.056 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:17.056 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 2213, 2024-03-15T13:09:17.056 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 2213, 2024-03-15T13:09:17.056 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 2213, 2024-03-15T13:09:17.056 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 2213, 2024-03-15T13:09:17.056 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-03-15T13:09:17.056 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-write_buf_throttle": { 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 3758096384, 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 0, 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 0, 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 0, 2024-03-15T13:09:17.057 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 0, 2024-03-15T13:09:17.058 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-03-15T13:09:17.058 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.058 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.058 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.058 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:17.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:17.058 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-write_buf_throttle-0x564a02d26960": { 2024-03-15T13:09:17.058 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-03-15T13:09:17.058 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 3758096384, 2024-03-15T13:09:17.058 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-03-15T13:09:17.058 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 0, 2024-03-15T13:09:17.058 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 0, 2024-03-15T13:09:17.058 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:17.058 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:17.058 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-03-15T13:09:17.059 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-03-15T13:09:17.059 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 0, 2024-03-15T13:09:17.059 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 0, 2024-03-15T13:09:17.059 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-03-15T13:09:17.059 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:17.059 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:17.059 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:17.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:17.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:17.059 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:09:17.764 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:17 smithi002 ceph-mon[32060]: pgmap v322: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-15T13:09:17.789 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-15T13:09:17.789 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-15T13:09:17.789 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-03-15T13:09:17.789 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-03-15T13:09:17.789 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-03-15T13:09:17.790 DEBUG:tasks.check_counter:Getting stats from j 2024-03-15T13:09:17.790 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-15T13:09:17.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:17 smithi184 ceph-mon[35993]: pgmap v322: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-15T13:09:17.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:17 smithi149 ceph-mon[35927]: pgmap v322: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-15T13:09:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.a/config 2024-03-15T13:09:19.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:19 smithi002 ceph-mon[32060]: pgmap v323: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 245 B/s wr, 12 op/s 2024-03-15T13:09:19.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:19 smithi184 ceph-mon[35993]: pgmap v323: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 245 B/s wr, 12 op/s 2024-03-15T13:09:19.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:19 smithi149 ceph-mon[35927]: pgmap v323: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 245 B/s wr, 12 op/s 2024-03-15T13:09:20.026 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-15T13:09:20.026 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-03-15T13:09:20.026 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 233, 2024-03-15T13:09:20.026 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 220, 2024-03-15T13:09:20.026 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 1568656, 2024-03-15T13:09:20.026 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 131008, 2024-03-15T13:09:20.026 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-03-15T13:09:20.026 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2024-03-15T13:09:20.026 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.047168355, 2024-03-15T13:09:20.026 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.014609329, 2024-03-15T13:09:20.026 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.037095481, 2024-03-15T13:09:20.027 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.004418272, 2024-03-15T13:09:20.027 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:20.027 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 220, 2024-03-15T13:09:20.027 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.010620220, 2024-03-15T13:09:20.027 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000048273 2024-03-15T13:09:20.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:20.027 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:20.027 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:20.027 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:20.027 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:20.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:20.027 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 1568656, 2024-03-15T13:09:20.027 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 131008 2024-03-15T13:09:20.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:20.027 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 2, 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 167, 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 320, 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 184160, 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 5, 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 3, 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.013974712, 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.009954238, 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.001918930, 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 164, 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.013325287, 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000081251 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:20.029 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:20.029 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:20.029 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:20.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:20.029 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-15T13:09:20.029 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 184160 2024-03-15T13:09:20.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:20.029 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-03-15T13:09:20.029 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2024-03-15T13:09:20.029 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2024-03-15T13:09:20.029 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2024-03-15T13:09:20.029 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2024-03-15T13:09:20.029 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2024-03-15T13:09:20.029 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2024-03-15T13:09:20.030 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.000470373, 2024-03-15T13:09:20.030 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2024-03-15T13:09:20.030 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000407484, 2024-03-15T13:09:20.030 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-15T13:09:20.030 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:20.030 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:20.030 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:20.030 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:20.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:20.030 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:20.030 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:20.030 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:20.030 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:20.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:20.030 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-15T13:09:20.031 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2024-03-15T13:09:20.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:20.031 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-03-15T13:09:20.031 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2024-03-15T13:09:20.031 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-03-15T13:09:20.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:20.031 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-03-15T13:09:20.031 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-03-15T13:09:20.031 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-03-15T13:09:20.031 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-03-15T13:09:20.031 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-03-15T13:09:20.031 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-03-15T13:09:20.031 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-03-15T13:09:20.031 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-03-15T13:09:20.032 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-03-15T13:09:20.033 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-03-15T13:09:20.033 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-03-15T13:09:20.033 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-03-15T13:09:20.033 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-15T13:09:20.033 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-03-15T13:09:20.033 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-03-15T13:09:20.033 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-03-15T13:09:20.033 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-03-15T13:09:20.033 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-03-15T13:09:20.033 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-03-15T13:09:20.033 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-03-15T13:09:20.033 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-03-15T13:09:20.033 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-03-15T13:09:20.033 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148754, 2024-03-15T13:09:20.033 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 31, 2024-03-15T13:09:20.034 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-03-15T13:09:20.034 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-03-15T13:09:20.034 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-03-15T13:09:20.034 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-03-15T13:09:20.034 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-03-15T13:09:20.034 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-03-15T13:09:20.034 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-03-15T13:09:20.034 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-03-15T13:09:20.034 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2024-03-15T13:09:20.034 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2024-03-15T13:09:20.034 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-03-15T13:09:20.034 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-03-15T13:09:20.034 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-03-15T13:09:20.034 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-03-15T13:09:20.035 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2024-03-15T13:09:20.035 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2024-03-15T13:09:20.035 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-03-15T13:09:20.035 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-03-15T13:09:20.035 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-03-15T13:09:20.035 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-03-15T13:09:20.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-15T13:09:20.035 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-15T13:09:20.035 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-03-15T13:09:20.035 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-03-15T13:09:20.035 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-03-15T13:09:20.036 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 231, 2024-03-15T13:09:20.036 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 1533391, 2024-03-15T13:09:20.036 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:20.036 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 231, 2024-03-15T13:09:20.036 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-03-15T13:09:20.036 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-03-15T13:09:20.036 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 231, 2024-03-15T13:09:20.036 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 1533391, 2024-03-15T13:09:20.036 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-03-15T13:09:20.036 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-03-15T13:09:20.036 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-03-15T13:09:20.036 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-03-15T13:09:20.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:20.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-15T13:09:20.037 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-15T13:09:20.695 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-15T13:09:20.695 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-15T13:09:20.695 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-15T13:09:20.695 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-15T13:09:20.695 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-15T13:09:20.695 DEBUG:tasks.check_counter:Getting stats from b 2024-03-15T13:09:20.695 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-15T13:09:21.013 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.b/config 2024-03-15T13:09:21.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:21 smithi184 ceph-mon[35993]: pgmap v324: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:09:21.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:21 smithi149 ceph-mon[35927]: pgmap v324: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:09:22.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:21 smithi002 ceph-mon[32060]: pgmap v324: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:09:22.954 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-15T13:09:22.954 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-15T13:09:22.954 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 16686, 2024-03-15T13:09:22.954 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 17446, 2024-03-15T13:09:22.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 5062771, 2024-03-15T13:09:22.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 51256299, 2024-03-15T13:09:22.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 15, 2024-03-15T13:09:22.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-03-15T13:09:22.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 1.803035052, 2024-03-15T13:09:22.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.664914522, 2024-03-15T13:09:22.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.549664888, 2024-03-15T13:09:22.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.500305004, 2024-03-15T13:09:22.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:22.955 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 17450, 2024-03-15T13:09:22.955 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 1.447762869, 2024-03-15T13:09:22.955 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000082966 2024-03-15T13:09:22.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:22.956 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 12618, 2024-03-15T13:09:22.956 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.002404263, 2024-03-15T13:09:22.956 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000190 2024-03-15T13:09:22.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 329344, 2024-03-15T13:09:22.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 99408 2024-03-15T13:09:22.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.956 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-15T13:09:22.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 15428, 2024-03-15T13:09:22.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 15935, 2024-03-15T13:09:22.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 5203790, 2024-03-15T13:09:22.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 49781410, 2024-03-15T13:09:22.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 160, 2024-03-15T13:09:22.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 10, 2024-03-15T13:09:22.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 1.875029004, 2024-03-15T13:09:22.957 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.723820514, 2024-03-15T13:09:22.957 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.479708046, 2024-03-15T13:09:22.957 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.551351594, 2024-03-15T13:09:22.957 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:22.957 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 15932, 2024-03-15T13:09:22.957 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 3.139394894, 2024-03-15T13:09:22.957 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000197049 2024-03-15T13:09:22.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.957 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:22.957 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4690, 2024-03-15T13:09:22.957 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001614895, 2024-03-15T13:09:22.957 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000344 2024-03-15T13:09:22.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.958 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 43312, 2024-03-15T13:09:22.958 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 142624 2024-03-15T13:09:22.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.958 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-15T13:09:22.958 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 59601, 2024-03-15T13:09:22.958 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 98068, 2024-03-15T13:09:22.958 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 22149223, 2024-03-15T13:09:22.958 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 72073358, 2024-03-15T13:09:22.958 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 20, 2024-03-15T13:09:22.958 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 9, 2024-03-15T13:09:22.958 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 4.881059692, 2024-03-15T13:09:22.958 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 2.121756371, 2024-03-15T13:09:22.958 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 4.475005831, 2024-03-15T13:09:22.958 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.482142647, 2024-03-15T13:09:22.959 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:22.959 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 98062, 2024-03-15T13:09:22.959 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 4.053965310, 2024-03-15T13:09:22.959 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000041340 2024-03-15T13:09:22.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.959 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:22.959 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 125701, 2024-03-15T13:09:22.959 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.022901167, 2024-03-15T13:09:22.959 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000182 2024-03-15T13:09:22.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.959 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1838176, 2024-03-15T13:09:22.959 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 505344 2024-03-15T13:09:22.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.960 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-15T13:09:22.960 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-15T13:09:22.960 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-15T13:09:22.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.960 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-15T13:09:22.960 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-15T13:09:22.960 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-15T13:09:22.960 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 9134, 2024-03-15T13:09:22.960 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 39.854552677, 2024-03-15T13:09:22.960 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.004363318 2024-03-15T13:09:22.960 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:22.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.960 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-15T13:09:22.960 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-15T13:09:22.960 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-15T13:09:22.961 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 16472, 2024-03-15T13:09:22.961 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 4.407268481, 2024-03-15T13:09:22.961 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000267561 2024-03-15T13:09:22.961 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:22.961 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.961 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-15T13:09:22.961 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 24262, 2024-03-15T13:09:22.961 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 23787, 2024-03-15T13:09:22.961 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-15T13:09:22.961 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 23787, 2024-03-15T13:09:22.961 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 78.629873658, 2024-03-15T13:09:22.961 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.003305581 2024-03-15T13:09:22.961 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.962 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-15T13:09:22.962 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 475, 2024-03-15T13:09:22.962 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-15T13:09:22.962 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-15T13:09:22.962 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 174, 2024-03-15T13:09:22.962 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 18, 2024-03-15T13:09:22.962 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-03-15T13:09:22.962 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 15848, 2024-03-15T13:09:22.962 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 228, 2024-03-15T13:09:22.962 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 15064, 2024-03-15T13:09:22.962 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 556, 2024-03-15T13:09:22.962 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 15520, 2024-03-15T13:09:22.962 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 953, 2024-03-15T13:09:22.962 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 3, 2024-03-15T13:09:22.962 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 3, 2024-03-15T13:09:22.963 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 18, 2024-03-15T13:09:22.963 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 25996, 2024-03-15T13:09:22.963 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 24856, 2024-03-15T13:09:22.963 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 337, 2024-03-15T13:09:22.963 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 7, 2024-03-15T13:09:22.963 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-15T13:09:22.963 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-15T13:09:22.963 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-03-15T13:09:22.963 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-03-15T13:09:22.963 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-15T13:09:22.963 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 59, 2024-03-15T13:09:22.963 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 936, 2024-03-15T13:09:22.963 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 57, 2024-03-15T13:09:22.963 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 1034, 2024-03-15T13:09:22.964 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-15T13:09:22.964 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 10279, 2024-03-15T13:09:22.964 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-15T13:09:22.964 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 4, 2024-03-15T13:09:22.964 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 234, 2024-03-15T13:09:22.964 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 7, 2024-03-15T13:09:22.964 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 10279, 2024-03-15T13:09:22.964 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-15T13:09:22.964 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-15T13:09:22.964 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-15T13:09:22.964 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 960, 2024-03-15T13:09:22.964 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 19, 2024-03-15T13:09:22.964 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 941, 2024-03-15T13:09:22.964 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 9319, 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 3315, 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 22585, 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 22125, 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 12434, 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 83, 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 32135 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-03-15T13:09:22.965 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 14, 2024-03-15T13:09:22.966 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 70, 2024-03-15T13:09:22.966 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 598, 2024-03-15T13:09:22.966 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-15T13:09:22.966 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-15T13:09:22.966 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 10368, 2024-03-15T13:09:22.966 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 9770, 2024-03-15T13:09:22.966 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-15T13:09:22.966 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-15T13:09:22.966 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-15T13:09:22.966 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-15T13:09:22.966 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-15T13:09:22.966 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-15T13:09:22.966 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-15T13:09:22.966 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 19, 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 59, 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 18, 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 48215, 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 48215, 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 67, 2024-03-15T13:09:22.967 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-03-15T13:09:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-03-15T13:09:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 67, 2024-03-15T13:09:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 5, 2024-03-15T13:09:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-15T13:09:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-03-15T13:09:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-03-15T13:09:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 122085710, 2024-03-15T13:09:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-03-15T13:09:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-15T13:09:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2364, 2024-03-15T13:09:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 44.895919525, 2024-03-15T13:09:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.018991505 2024-03-15T13:09:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-03-15T13:09:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.969 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-15T13:09:22.969 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 7620, 2024-03-15T13:09:22.969 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 18350, 2024-03-15T13:09:22.969 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 10730, 2024-03-15T13:09:22.969 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 736, 2024-03-15T13:09:22.969 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 797, 2024-03-15T13:09:22.969 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 61, 2024-03-15T13:09:22.969 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 15848, 2024-03-15T13:09:22.969 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 26584, 2024-03-15T13:09:22.969 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 10736, 2024-03-15T13:09:22.969 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 3, 2024-03-15T13:09:22.969 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 11320, 2024-03-15T13:09:22.969 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 11317, 2024-03-15T13:09:22.969 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 124012, 2024-03-15T13:09:22.969 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 231804 2024-03-15T13:09:22.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.970 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-15T13:09:22.970 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 29177, 2024-03-15T13:09:22.970 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 622, 2024-03-15T13:09:22.970 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 24262, 2024-03-15T13:09:22.970 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 92, 2024-03-15T13:09:22.970 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 1376, 2024-03-15T13:09:22.970 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-15T13:09:22.970 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 10680, 2024-03-15T13:09:22.970 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 22.880832639, 2024-03-15T13:09:22.970 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002142400 2024-03-15T13:09:22.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.970 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-15T13:09:22.970 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 144, 2024-03-15T13:09:22.970 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.063099383, 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000438190 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 902, 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 9.255403613, 2024-03-15T13:09:22.971 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.010260979 2024-03-15T13:09:22.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.972 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-15T13:09:22.972 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.972 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.972 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.972 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-15T13:09:22.972 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.972 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.972 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.972 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-15T13:09:22.972 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.972 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.973 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.973 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-15T13:09:22.973 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.973 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.973 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.973 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-15T13:09:22.973 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.973 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.973 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.973 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-15T13:09:22.974 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 11, 2024-03-15T13:09:22.974 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.003892837, 2024-03-15T13:09:22.974 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000353894 2024-03-15T13:09:22.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.974 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-15T13:09:22.974 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 623, 2024-03-15T13:09:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 2.362844590, 2024-03-15T13:09:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.003792687 2024-03-15T13:09:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-15T13:09:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-15T13:09:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 11, 2024-03-15T13:09:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.437803694, 2024-03-15T13:09:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.039800335 2024-03-15T13:09:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-15T13:09:22.975 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2024-03-15T13:09:22.976 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000468459, 2024-03-15T13:09:22.976 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000468459 2024-03-15T13:09:22.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.976 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-15T13:09:22.976 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 992, 2024-03-15T13:09:22.976 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 2.111270149, 2024-03-15T13:09:22.976 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002128296 2024-03-15T13:09:22.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.976 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-15T13:09:22.976 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-03-15T13:09:22.976 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.002915203, 2024-03-15T13:09:22.976 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001457601 2024-03-15T13:09:22.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.976 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-15T13:09:22.976 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.977 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.977 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.977 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-15T13:09:22.977 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.977 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.977 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.977 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-15T13:09:22.977 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 601, 2024-03-15T13:09:22.977 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 20.556589507, 2024-03-15T13:09:22.977 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.034203975 2024-03-15T13:09:22.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.977 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-15T13:09:22.977 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4, 2024-03-15T13:09:22.978 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.017741235, 2024-03-15T13:09:22.978 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.004435308 2024-03-15T13:09:22.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.978 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-15T13:09:22.978 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.978 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.978 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.978 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-15T13:09:22.978 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 40, 2024-03-15T13:09:22.978 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.166468156, 2024-03-15T13:09:22.978 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.004161703 2024-03-15T13:09:22.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.979 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-15T13:09:22.979 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.979 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.979 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.979 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-15T13:09:22.979 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.979 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.979 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.979 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-15T13:09:22.979 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.979 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.979 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.980 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-15T13:09:22.980 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 9, 2024-03-15T13:09:22.980 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.022889809, 2024-03-15T13:09:22.980 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002543312 2024-03-15T13:09:22.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.980 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-15T13:09:22.980 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.980 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.980 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.980 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-15T13:09:22.980 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 9767, 2024-03-15T13:09:22.980 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 20.747654384, 2024-03-15T13:09:22.980 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002124260 2024-03-15T13:09:22.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.981 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-15T13:09:22.981 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-15T13:09:22.981 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-15T13:09:22.981 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.981 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.981 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.981 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:22.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.981 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-15T13:09:22.981 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 2, 2024-03-15T13:09:22.981 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 6, 2024-03-15T13:09:22.981 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 4, 2024-03-15T13:09:22.981 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 2, 2024-03-15T13:09:22.981 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-15T13:09:22.981 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 159, 2024-03-15T13:09:22.982 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 79, 2024-03-15T13:09:22.982 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 631, 2024-03-15T13:09:22.982 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-15T13:09:22.982 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.982 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-15T13:09:22.982 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-15T13:09:22.982 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-15T13:09:22.982 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-15T13:09:22.982 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-15T13:09:22.982 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-15T13:09:22.982 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-15T13:09:22.982 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-15T13:09:22.982 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-15T13:09:22.982 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-15T13:09:22.983 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-15T13:09:22.983 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-15T13:09:22.983 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-15T13:09:22.983 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-15T13:09:22.983 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-15T13:09:22.983 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-15T13:09:22.983 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-15T13:09:22.983 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-15T13:09:22.983 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-15T13:09:22.983 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-15T13:09:22.983 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-15T13:09:22.983 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-15T13:09:22.983 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-15T13:09:22.983 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-15T13:09:22.983 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-15T13:09:22.984 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-15T13:09:22.984 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-15T13:09:22.984 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-15T13:09:22.984 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-15T13:09:22.984 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-15T13:09:22.984 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-15T13:09:22.984 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-15T13:09:22.984 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-15T13:09:22.984 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-15T13:09:22.984 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-15T13:09:22.984 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-15T13:09:22.984 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-15T13:09:22.984 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 5093447, 2024-03-15T13:09:22.984 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 1320, 2024-03-15T13:09:22.985 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-03-15T13:09:22.985 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-03-15T13:09:22.985 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-15T13:09:22.985 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-15T13:09:22.985 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-15T13:09:22.985 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-15T13:09:22.985 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-15T13:09:22.985 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-15T13:09:22.985 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 9448, 2024-03-15T13:09:22.985 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 197, 2024-03-15T13:09:22.985 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-15T13:09:22.985 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-15T13:09:22.985 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-15T13:09:22.985 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-15T13:09:22.985 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 28910011, 2024-03-15T13:09:22.986 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 335345, 2024-03-15T13:09:22.986 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-15T13:09:22.986 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-15T13:09:22.986 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-15T13:09:22.986 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-15T13:09:22.986 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.986 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-15T13:09:22.986 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-15T13:09:22.986 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-15T13:09:22.986 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 33617, 2024-03-15T13:09:22.986 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 122855870, 2024-03-15T13:09:22.986 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 31, 2024-03-15T13:09:22.986 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 33617, 2024-03-15T13:09:22.986 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-15T13:09:22.986 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 33617, 2024-03-15T13:09:22.987 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 113.292623909, 2024-03-15T13:09:22.987 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.003370099 2024-03-15T13:09:22.987 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.987 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-15T13:09:22.987 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-15T13:09:22.987 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 33617, 2024-03-15T13:09:22.987 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 34056 2024-03-15T13:09:22.987 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.987 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 33617, 2024-03-15T13:09:22.987 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 10511, 2024-03-15T13:09:22.987 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 23106, 2024-03-15T13:09:22.987 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-15T13:09:22.987 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-15T13:09:22.987 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 0, 2024-03-15T13:09:22.988 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 11, 2024-03-15T13:09:22.988 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 213, 2024-03-15T13:09:22.988 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 2558, 2024-03-15T13:09:22.988 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 99, 2024-03-15T13:09:22.988 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-15T13:09:22.988 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-15T13:09:22.988 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-15T13:09:22.988 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 18, 2024-03-15T13:09:22.988 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 20101, 2024-03-15T13:09:22.988 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-15T13:09:22.988 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-15T13:09:22.988 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-15T13:09:22.988 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 10298, 2024-03-15T13:09:22.988 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 22, 2024-03-15T13:09:22.988 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-15T13:09:22.989 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-15T13:09:22.989 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-15T13:09:22.989 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-15T13:09:22.989 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-15T13:09:22.989 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-15T13:09:22.989 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-15T13:09:22.989 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-15T13:09:22.989 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-15T13:09:22.989 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-15T13:09:22.989 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-15T13:09:22.989 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-15T13:09:22.989 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-15T13:09:22.989 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-15T13:09:22.989 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-15T13:09:22.989 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-15T13:09:22.990 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-15T13:09:22.990 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-15T13:09:22.990 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-15T13:09:22.990 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-15T13:09:22.990 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-15T13:09:22.990 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-15T13:09:22.990 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-15T13:09:22.990 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-15T13:09:22.990 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-15T13:09:22.990 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-15T13:09:22.990 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 100, 2024-03-15T13:09:22.990 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-15T13:09:22.990 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 16, 2024-03-15T13:09:22.990 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 12, 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 12, 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 605, 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 131 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-15T13:09:22.991 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.992 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-15T13:09:22.992 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-15T13:09:22.992 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 21, 2024-03-15T13:09:22.992 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-15T13:09:22.992 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 5, 2024-03-15T13:09:22.992 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 19561, 2024-03-15T13:09:22.992 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 9770, 2024-03-15T13:09:22.992 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-15T13:09:22.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.992 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-15T13:09:22.992 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:22.992 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-15T13:09:22.992 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:22.992 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 91711, 2024-03-15T13:09:22.992 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 24096790, 2024-03-15T13:09:22.993 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:22.993 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 91711, 2024-03-15T13:09:22.993 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-15T13:09:22.993 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-15T13:09:22.993 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 91711, 2024-03-15T13:09:22.993 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 24096790, 2024-03-15T13:09:22.993 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:22.993 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.993 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.993 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.993 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:22.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.993 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-15T13:09:22.993 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:22.993 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-15T13:09:22.994 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:22.994 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-15T13:09:22.994 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-15T13:09:22.994 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:22.994 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:22.994 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 33617, 2024-03-15T13:09:22.994 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 123972768, 2024-03-15T13:09:22.994 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 33617, 2024-03-15T13:09:22.994 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 123972768, 2024-03-15T13:09:22.994 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:22.994 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.994 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.994 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.995 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:22.995 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.995 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-15T13:09:22.995 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:22.995 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-15T13:09:22.995 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:22.995 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-15T13:09:22.995 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-15T13:09:22.995 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:22.995 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:22.995 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 33617, 2024-03-15T13:09:22.995 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 33617, 2024-03-15T13:09:22.995 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 33617, 2024-03-15T13:09:22.995 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 33617, 2024-03-15T13:09:22.995 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:22.996 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.996 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.996 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.996 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:22.996 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.996 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-15T13:09:22.996 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:22.996 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-15T13:09:22.996 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:22.996 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 9770, 2024-03-15T13:09:22.996 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 1116898, 2024-03-15T13:09:22.996 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:22.996 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 9770, 2024-03-15T13:09:22.996 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-15T13:09:22.997 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-15T13:09:22.997 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 194, 2024-03-15T13:09:22.997 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1116898, 2024-03-15T13:09:22.997 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:22.997 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.997 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.997 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.997 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:22.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:22.997 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55b2d82ba6e0": { 2024-03-15T13:09:22.997 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:22.997 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-15T13:09:22.997 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:22.997 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 48215, 2024-03-15T13:09:22.998 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 117891406, 2024-03-15T13:09:22.998 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:22.998 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 48215, 2024-03-15T13:09:22.998 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-15T13:09:22.998 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-15T13:09:22.998 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 2364, 2024-03-15T13:09:22.998 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 117891406, 2024-03-15T13:09:22.998 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:22.998 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:22.998 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:22.998 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:22.998 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:22.998 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:22.998 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-15T13:09:23.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:23 smithi149 ceph-mon[35927]: pgmap v325: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:09:23.802 INFO:tasks.check_counter:Daemon mds.b mds.exported=59 2024-03-15T13:09:23.803 INFO:tasks.check_counter:Daemon mds.b mds.imported=57 2024-03-15T13:09:23.803 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-03-15T13:09:23.803 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-03-15T13:09:23.803 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-03-15T13:09:23.803 DEBUG:tasks.check_counter:Getting stats from e 2024-03-15T13:09:23.803 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-15T13:09:23.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:23 smithi184 ceph-mon[35993]: pgmap v325: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:09:24.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:23 smithi002 ceph-mon[32060]: pgmap v325: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:09:24.144 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.b/config 2024-03-15T13:09:25.598 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:25 smithi149 ceph-mon[35927]: pgmap v326: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:09:26.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:25 smithi002 ceph-mon[32060]: pgmap v326: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:09:26.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:25 smithi184 ceph-mon[35993]: pgmap v326: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-03-15T13:09:26.061 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-15T13:09:26.061 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-15T13:09:26.061 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 749, 2024-03-15T13:09:26.062 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 975, 2024-03-15T13:09:26.062 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 219754, 2024-03-15T13:09:26.062 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 1688673, 2024-03-15T13:09:26.062 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 53, 2024-03-15T13:09:26.062 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2024-03-15T13:09:26.062 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.064837741, 2024-03-15T13:09:26.062 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.019515362, 2024-03-15T13:09:26.062 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.034358845, 2024-03-15T13:09:26.062 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.007681359, 2024-03-15T13:09:26.062 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:26.062 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 973, 2024-03-15T13:09:26.062 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.089280746, 2024-03-15T13:09:26.062 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000091758 2024-03-15T13:09:26.062 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.063 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:26.063 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 965, 2024-03-15T13:09:26.063 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000165820, 2024-03-15T13:09:26.063 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000171 2024-03-15T13:09:26.063 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.063 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1584, 2024-03-15T13:09:26.063 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 1440 2024-03-15T13:09:26.063 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.063 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-15T13:09:26.063 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 278, 2024-03-15T13:09:26.063 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 296, 2024-03-15T13:09:26.063 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 111372, 2024-03-15T13:09:26.063 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 264576, 2024-03-15T13:09:26.063 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 13, 2024-03-15T13:09:26.064 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 6, 2024-03-15T13:09:26.064 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.021716897, 2024-03-15T13:09:26.064 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.006398480, 2024-03-15T13:09:26.064 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.010853302, 2024-03-15T13:09:26.064 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.004187107, 2024-03-15T13:09:26.064 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:26.064 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 294, 2024-03-15T13:09:26.064 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.020172221, 2024-03-15T13:09:26.064 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000068612 2024-03-15T13:09:26.064 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.064 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:26.064 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 191, 2024-03-15T13:09:26.064 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000028446, 2024-03-15T13:09:26.064 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000148 2024-03-15T13:09:26.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.065 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 6688, 2024-03-15T13:09:26.065 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 138688 2024-03-15T13:09:26.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.065 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-15T13:09:26.065 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 2977, 2024-03-15T13:09:26.065 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 2513, 2024-03-15T13:09:26.065 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 2775008, 2024-03-15T13:09:26.065 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 1183380, 2024-03-15T13:09:26.065 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 7, 2024-03-15T13:09:26.065 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 6, 2024-03-15T13:09:26.065 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.220942993, 2024-03-15T13:09:26.065 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.094501165, 2024-03-15T13:09:26.065 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.156005287, 2024-03-15T13:09:26.066 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.009500101, 2024-03-15T13:09:26.066 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:26.066 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2512, 2024-03-15T13:09:26.066 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.089676762, 2024-03-15T13:09:26.066 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000035699 2024-03-15T13:09:26.066 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.066 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:26.066 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4411, 2024-03-15T13:09:26.066 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000799521, 2024-03-15T13:09:26.066 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000181 2024-03-15T13:09:26.066 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.066 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1763744, 2024-03-15T13:09:26.066 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 458944 2024-03-15T13:09:26.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.067 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-15T13:09:26.067 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-15T13:09:26.067 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-15T13:09:26.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.067 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-15T13:09:26.067 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-15T13:09:26.067 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-15T13:09:26.067 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 490, 2024-03-15T13:09:26.067 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.189483954, 2024-03-15T13:09:26.067 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000386701 2024-03-15T13:09:26.067 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:26.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.067 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-15T13:09:26.068 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-15T13:09:26.068 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-15T13:09:26.068 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 236, 2024-03-15T13:09:26.068 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.012236459, 2024-03-15T13:09:26.068 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000051849 2024-03-15T13:09:26.068 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:26.068 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.068 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-15T13:09:26.068 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 188, 2024-03-15T13:09:26.068 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 187, 2024-03-15T13:09:26.068 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-15T13:09:26.068 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 187, 2024-03-15T13:09:26.068 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.415302420, 2024-03-15T13:09:26.068 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002220868 2024-03-15T13:09:26.068 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.069 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-15T13:09:26.069 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 1, 2024-03-15T13:09:26.069 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-15T13:09:26.069 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-15T13:09:26.069 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 11, 2024-03-15T13:09:26.069 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-03-15T13:09:26.069 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-03-15T13:09:26.069 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 28, 2024-03-15T13:09:26.069 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 2, 2024-03-15T13:09:26.069 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 9, 2024-03-15T13:09:26.069 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 17, 2024-03-15T13:09:26.069 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 20, 2024-03-15T13:09:26.069 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 131, 2024-03-15T13:09:26.069 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 1, 2024-03-15T13:09:26.069 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 1, 2024-03-15T13:09:26.070 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 4, 2024-03-15T13:09:26.070 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 507, 2024-03-15T13:09:26.070 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 246, 2024-03-15T13:09:26.070 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-03-15T13:09:26.070 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 14, 2024-03-15T13:09:26.070 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-15T13:09:26.070 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-15T13:09:26.070 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-03-15T13:09:26.070 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-03-15T13:09:26.070 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-15T13:09:26.070 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 10, 2024-03-15T13:09:26.070 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 76, 2024-03-15T13:09:26.070 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 10, 2024-03-15T13:09:26.070 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 41, 2024-03-15T13:09:26.071 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-15T13:09:26.071 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 73, 2024-03-15T13:09:26.071 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-15T13:09:26.071 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 4, 2024-03-15T13:09:26.071 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 234, 2024-03-15T13:09:26.071 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 7, 2024-03-15T13:09:26.071 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 73, 2024-03-15T13:09:26.071 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-15T13:09:26.071 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-15T13:09:26.071 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-15T13:09:26.071 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 1, 2024-03-15T13:09:26.071 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-03-15T13:09:26.071 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 1, 2024-03-15T13:09:26.071 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 72, 2024-03-15T13:09:26.071 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-03-15T13:09:26.072 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 26, 2024-03-15T13:09:26.072 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 138, 2024-03-15T13:09:26.072 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-15T13:09:26.072 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-15T13:09:26.072 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-15T13:09:26.072 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 194, 2024-03-15T13:09:26.072 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 86, 2024-03-15T13:09:26.072 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 20, 2024-03-15T13:09:26.072 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 197 2024-03-15T13:09:26.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.072 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-15T13:09:26.072 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-03-15T13:09:26.072 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-03-15T13:09:26.072 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-03-15T13:09:26.072 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 16, 2024-03-15T13:09:26.073 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-03-15T13:09:26.073 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 1, 2024-03-15T13:09:26.073 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-15T13:09:26.073 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-15T13:09:26.073 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 79, 2024-03-15T13:09:26.073 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 78, 2024-03-15T13:09:26.073 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-15T13:09:26.073 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-15T13:09:26.073 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-15T13:09:26.073 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-15T13:09:26.073 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-15T13:09:26.073 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-15T13:09:26.073 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-15T13:09:26.073 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-03-15T13:09:26.074 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 11, 2024-03-15T13:09:26.074 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-15T13:09:26.074 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-03-15T13:09:26.074 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-15T13:09:26.074 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-15T13:09:26.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.074 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-15T13:09:26.074 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-15T13:09:26.074 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 447, 2024-03-15T13:09:26.074 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-03-15T13:09:26.074 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-03-15T13:09:26.074 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 447, 2024-03-15T13:09:26.074 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-15T13:09:26.074 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-03-15T13:09:26.074 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 2, 2024-03-15T13:09:26.075 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-03-15T13:09:26.075 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-03-15T13:09:26.075 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 2, 2024-03-15T13:09:26.075 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-03-15T13:09:26.075 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-15T13:09:26.075 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-03-15T13:09:26.075 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-03-15T13:09:26.075 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 5014111, 2024-03-15T13:09:26.075 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-03-15T13:09:26.075 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-15T13:09:26.075 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 160, 2024-03-15T13:09:26.075 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.390256152, 2024-03-15T13:09:26.075 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002439100 2024-03-15T13:09:26.075 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.076 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-03-15T13:09:26.076 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.076 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-15T13:09:26.076 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 30, 2024-03-15T13:09:26.076 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 179, 2024-03-15T13:09:26.076 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 149, 2024-03-15T13:09:26.076 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 26, 2024-03-15T13:09:26.076 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 71, 2024-03-15T13:09:26.076 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 45, 2024-03-15T13:09:26.076 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 28, 2024-03-15T13:09:26.076 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 237, 2024-03-15T13:09:26.076 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 209, 2024-03-15T13:09:26.077 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 1, 2024-03-15T13:09:26.077 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 94, 2024-03-15T13:09:26.077 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 93, 2024-03-15T13:09:26.077 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 55380, 2024-03-15T13:09:26.077 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 223612 2024-03-15T13:09:26.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.077 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-15T13:09:26.077 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 487, 2024-03-15T13:09:26.077 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 11, 2024-03-15T13:09:26.077 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 188, 2024-03-15T13:09:26.077 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 24, 2024-03-15T13:09:26.077 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 114, 2024-03-15T13:09:26.077 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-15T13:09:26.077 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 38, 2024-03-15T13:09:26.077 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.102076225, 2024-03-15T13:09:26.078 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002686216 2024-03-15T13:09:26.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.078 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-15T13:09:26.078 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 3, 2024-03-15T13:09:26.078 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001184936, 2024-03-15T13:09:26.078 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000394978 2024-03-15T13:09:26.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.078 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-15T13:09:26.078 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.078 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.078 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.078 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-15T13:09:26.078 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.078 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.079 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.079 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-15T13:09:26.079 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 48, 2024-03-15T13:09:26.079 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.037622373, 2024-03-15T13:09:26.079 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000783799 2024-03-15T13:09:26.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.079 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-15T13:09:26.079 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.079 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.079 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.079 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-15T13:09:26.079 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.079 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-15T13:09:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-15T13:09:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-15T13:09:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.081 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.081 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.081 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-15T13:09:26.081 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.081 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.081 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.081 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.081 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-15T13:09:26.081 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.081 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.082 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.082 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-15T13:09:26.082 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.082 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.082 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.082 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-15T13:09:26.082 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.082 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.082 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.082 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-15T13:09:26.083 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.083 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.083 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.083 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-15T13:09:26.083 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 19, 2024-03-15T13:09:26.083 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.011511325, 2024-03-15T13:09:26.083 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000605859 2024-03-15T13:09:26.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.083 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-15T13:09:26.083 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.083 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.083 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.083 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-15T13:09:26.084 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.084 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.084 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.084 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-15T13:09:26.084 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.084 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.084 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.084 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-15T13:09:26.084 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2024-03-15T13:09:26.084 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.013817798, 2024-03-15T13:09:26.084 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.013817798 2024-03-15T13:09:26.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.085 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-15T13:09:26.085 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.085 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.085 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.085 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-15T13:09:26.085 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.085 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.085 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.085 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-15T13:09:26.085 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.085 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.085 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.086 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-15T13:09:26.086 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.086 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.086 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.086 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.086 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-15T13:09:26.086 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.086 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.086 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.086 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.086 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-15T13:09:26.086 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.086 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.086 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.087 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-15T13:09:26.087 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.087 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.087 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.087 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-15T13:09:26.087 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.087 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.087 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.087 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-15T13:09:26.087 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 78, 2024-03-15T13:09:26.087 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.249089763, 2024-03-15T13:09:26.087 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.003193458 2024-03-15T13:09:26.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.088 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-15T13:09:26.088 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-15T13:09:26.088 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-15T13:09:26.088 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.088 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.088 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.088 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:26.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.088 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-15T13:09:26.088 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-03-15T13:09:26.088 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 5, 2024-03-15T13:09:26.088 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 4, 2024-03-15T13:09:26.088 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-03-15T13:09:26.088 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 88, 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 88, 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 648, 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-15T13:09:26.089 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-15T13:09:26.090 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-15T13:09:26.090 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-15T13:09:26.090 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-15T13:09:26.090 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-15T13:09:26.090 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-15T13:09:26.090 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-15T13:09:26.090 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-15T13:09:26.090 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-15T13:09:26.090 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-15T13:09:26.090 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-15T13:09:26.090 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-15T13:09:26.090 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-15T13:09:26.090 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-15T13:09:26.090 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-15T13:09:26.090 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 346796, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 123, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-03-15T13:09:26.091 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-15T13:09:26.092 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-15T13:09:26.092 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-15T13:09:26.092 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-15T13:09:26.092 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-15T13:09:26.092 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-15T13:09:26.092 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 9448, 2024-03-15T13:09:26.092 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 197, 2024-03-15T13:09:26.092 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-15T13:09:26.092 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-15T13:09:26.092 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-15T13:09:26.092 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-15T13:09:26.092 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 148400, 2024-03-15T13:09:26.092 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 699, 2024-03-15T13:09:26.092 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-15T13:09:26.092 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 565, 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 856645, 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 33, 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 565, 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 565, 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 1.410970406, 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002497292 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.093 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-15T13:09:26.094 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-15T13:09:26.094 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 565, 2024-03-15T13:09:26.094 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 654 2024-03-15T13:09:26.094 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.094 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 565, 2024-03-15T13:09:26.094 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 101, 2024-03-15T13:09:26.094 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 464, 2024-03-15T13:09:26.094 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-15T13:09:26.094 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-15T13:09:26.094 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 0, 2024-03-15T13:09:26.094 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 10, 2024-03-15T13:09:26.094 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 28, 2024-03-15T13:09:26.094 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 188, 2024-03-15T13:09:26.094 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 33, 2024-03-15T13:09:26.095 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-15T13:09:26.095 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-15T13:09:26.095 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-15T13:09:26.095 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-03-15T13:09:26.095 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 168, 2024-03-15T13:09:26.095 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-15T13:09:26.095 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-15T13:09:26.095 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-15T13:09:26.095 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 73, 2024-03-15T13:09:26.095 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 20, 2024-03-15T13:09:26.095 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-15T13:09:26.095 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-15T13:09:26.095 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-15T13:09:26.095 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-15T13:09:26.095 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-15T13:09:26.096 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-15T13:09:26.096 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-15T13:09:26.096 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-15T13:09:26.096 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-15T13:09:26.096 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-15T13:09:26.096 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-15T13:09:26.096 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-15T13:09:26.096 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-15T13:09:26.096 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-15T13:09:26.096 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-15T13:09:26.096 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-15T13:09:26.096 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-15T13:09:26.096 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-15T13:09:26.096 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-15T13:09:26.096 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-15T13:09:26.097 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-15T13:09:26.097 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-15T13:09:26.097 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-15T13:09:26.097 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-15T13:09:26.097 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-15T13:09:26.097 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-15T13:09:26.097 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 100, 2024-03-15T13:09:26.097 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-15T13:09:26.097 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 11, 2024-03-15T13:09:26.097 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 12, 2024-03-15T13:09:26.097 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 12, 2024-03-15T13:09:26.097 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-03-15T13:09:26.097 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-15T13:09:26.097 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 95, 2024-03-15T13:09:26.097 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-03-15T13:09:26.098 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 39 2024-03-15T13:09:26.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.098 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-15T13:09:26.098 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-03-15T13:09:26.098 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-15T13:09:26.098 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-15T13:09:26.098 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-15T13:09:26.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.098 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-15T13:09:26.098 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-15T13:09:26.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.098 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-15T13:09:26.098 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-15T13:09:26.098 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 10, 2024-03-15T13:09:26.098 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-15T13:09:26.099 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 5, 2024-03-15T13:09:26.099 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 156, 2024-03-15T13:09:26.099 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 78, 2024-03-15T13:09:26.099 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-15T13:09:26.099 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.099 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-15T13:09:26.099 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:26.099 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-15T13:09:26.099 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:26.099 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 4000, 2024-03-15T13:09:26.099 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 2726694, 2024-03-15T13:09:26.099 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:26.099 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 4000, 2024-03-15T13:09:26.099 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-15T13:09:26.099 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 4000, 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 2726694, 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:26.100 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:26.101 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 565, 2024-03-15T13:09:26.101 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 865467, 2024-03-15T13:09:26.101 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 565, 2024-03-15T13:09:26.101 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 865467, 2024-03-15T13:09:26.101 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:26.101 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.101 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.101 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.101 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:26.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.101 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-15T13:09:26.101 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:26.101 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-15T13:09:26.101 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:26.101 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-15T13:09:26.102 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-15T13:09:26.102 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:26.102 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:26.102 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 565, 2024-03-15T13:09:26.102 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 565, 2024-03-15T13:09:26.102 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 565, 2024-03-15T13:09:26.102 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 565, 2024-03-15T13:09:26.102 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:26.102 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.102 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.102 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.102 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:26.102 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.102 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-15T13:09:26.102 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:26.103 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-15T13:09:26.103 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:26.103 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 78, 2024-03-15T13:09:26.103 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 8822, 2024-03-15T13:09:26.103 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:26.103 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 78, 2024-03-15T13:09:26.103 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-15T13:09:26.103 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-15T13:09:26.103 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 28, 2024-03-15T13:09:26.103 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 8822, 2024-03-15T13:09:26.103 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:26.103 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.103 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.103 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.104 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:26.104 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:26.104 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x5626982166e0": { 2024-03-15T13:09:26.104 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:26.104 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-15T13:09:26.104 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:26.104 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 447, 2024-03-15T13:09:26.104 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 819807, 2024-03-15T13:09:26.104 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:26.104 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 447, 2024-03-15T13:09:26.104 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-15T13:09:26.104 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-15T13:09:26.104 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 160, 2024-03-15T13:09:26.104 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 819807, 2024-03-15T13:09:26.104 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:26.105 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:26.105 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:26.105 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:26.105 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:26.105 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:26.105 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-15T13:09:26.821 INFO:tasks.check_counter:Daemon mds.e mds.exported=10 2024-03-15T13:09:26.821 INFO:tasks.check_counter:Daemon mds.e mds.imported=10 2024-03-15T13:09:26.821 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=7 2024-03-15T13:09:26.821 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-15T13:09:26.821 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-15T13:09:26.821 DEBUG:tasks.check_counter:Getting stats from h 2024-03-15T13:09:26.821 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-15T13:09:27.136 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.b/config 2024-03-15T13:09:27.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:27 smithi149 ceph-mon[35927]: pgmap v327: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-15T13:09:28.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:27 smithi002 ceph-mon[32060]: pgmap v327: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-15T13:09:28.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:27 smithi184 ceph-mon[35993]: pgmap v327: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-15T13:09:29.048 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-15T13:09:29.048 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-15T13:09:29.048 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 2677, 2024-03-15T13:09:29.048 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 2230, 2024-03-15T13:09:29.048 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 1049818, 2024-03-15T13:09:29.049 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 2757128, 2024-03-15T13:09:29.049 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 39, 2024-03-15T13:09:29.049 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 7, 2024-03-15T13:09:29.049 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.190247482, 2024-03-15T13:09:29.049 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.082104843, 2024-03-15T13:09:29.049 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.118296613, 2024-03-15T13:09:29.049 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.009998178, 2024-03-15T13:09:29.049 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:29.049 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2228, 2024-03-15T13:09:29.049 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.088571088, 2024-03-15T13:09:29.049 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000039753 2024-03-15T13:09:29.049 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.049 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:29.049 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4076, 2024-03-15T13:09:29.050 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000745122, 2024-03-15T13:09:29.050 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000182 2024-03-15T13:09:29.050 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.050 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-15T13:09:29.050 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 312736 2024-03-15T13:09:29.050 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.050 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-15T13:09:29.050 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 643, 2024-03-15T13:09:29.050 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1090, 2024-03-15T13:09:29.050 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 205858, 2024-03-15T13:09:29.050 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 701068, 2024-03-15T13:09:29.050 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 16, 2024-03-15T13:09:29.050 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 5, 2024-03-15T13:09:29.050 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.049351143, 2024-03-15T13:09:29.050 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.020045721, 2024-03-15T13:09:29.051 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.028067601, 2024-03-15T13:09:29.051 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.002239108, 2024-03-15T13:09:29.051 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:29.051 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1087, 2024-03-15T13:09:29.051 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.060418784, 2024-03-15T13:09:29.051 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000055583 2024-03-15T13:09:29.051 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.051 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:29.051 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1335, 2024-03-15T13:09:29.051 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000254923, 2024-03-15T13:09:29.051 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000190 2024-03-15T13:09:29.051 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.051 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 2816, 2024-03-15T13:09:29.051 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 139616 2024-03-15T13:09:29.052 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.052 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-15T13:09:29.052 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 681, 2024-03-15T13:09:29.052 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 642, 2024-03-15T13:09:29.052 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 1880180, 2024-03-15T13:09:29.052 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 391861, 2024-03-15T13:09:29.052 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 16, 2024-03-15T13:09:29.052 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 6, 2024-03-15T13:09:29.052 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.066081536, 2024-03-15T13:09:29.052 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.015843425, 2024-03-15T13:09:29.052 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.053502537, 2024-03-15T13:09:29.052 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.012216750, 2024-03-15T13:09:29.052 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:29.052 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 642, 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.042724065, 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000066548 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 17, 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000002503, 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000147 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1772304, 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 147440 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-15T13:09:29.053 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-15T13:09:29.054 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-15T13:09:29.054 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 492, 2024-03-15T13:09:29.054 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.242103492, 2024-03-15T13:09:29.054 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000492080 2024-03-15T13:09:29.054 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:29.054 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.054 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-15T13:09:29.054 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-15T13:09:29.054 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-15T13:09:29.054 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 345, 2024-03-15T13:09:29.054 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.021864899, 2024-03-15T13:09:29.054 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000063376 2024-03-15T13:09:29.054 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:29.054 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.054 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-15T13:09:29.055 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 282, 2024-03-15T13:09:29.055 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 282, 2024-03-15T13:09:29.055 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-15T13:09:29.055 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 282, 2024-03-15T13:09:29.055 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.453771787, 2024-03-15T13:09:29.055 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001609119 2024-03-15T13:09:29.055 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.055 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-15T13:09:29.055 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-03-15T13:09:29.055 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-15T13:09:29.055 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-15T13:09:29.055 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 11, 2024-03-15T13:09:29.055 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-03-15T13:09:29.055 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-03-15T13:09:29.055 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 35, 2024-03-15T13:09:29.056 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 3, 2024-03-15T13:09:29.056 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 2, 2024-03-15T13:09:29.056 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 30, 2024-03-15T13:09:29.056 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 30, 2024-03-15T13:09:29.056 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 256, 2024-03-15T13:09:29.056 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 2, 2024-03-15T13:09:29.056 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 2, 2024-03-15T13:09:29.056 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 3, 2024-03-15T13:09:29.056 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 594, 2024-03-15T13:09:29.056 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 397, 2024-03-15T13:09:29.056 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-03-15T13:09:29.056 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 15, 2024-03-15T13:09:29.056 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-15T13:09:29.056 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-15T13:09:29.056 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 12, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 202, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 12, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 109, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 111, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 4, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 234, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 7, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 111, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-15T13:09:29.057 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 2, 2024-03-15T13:09:29.058 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-03-15T13:09:29.058 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 2, 2024-03-15T13:09:29.058 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 109, 2024-03-15T13:09:29.058 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-03-15T13:09:29.058 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 32, 2024-03-15T13:09:29.058 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 175, 2024-03-15T13:09:29.058 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-15T13:09:29.058 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-15T13:09:29.058 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-15T13:09:29.058 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 216, 2024-03-15T13:09:29.058 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 133, 2024-03-15T13:09:29.058 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 25, 2024-03-15T13:09:29.058 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 290 2024-03-15T13:09:29.058 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.059 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-15T13:09:29.059 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-03-15T13:09:29.059 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-03-15T13:09:29.059 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-03-15T13:09:29.059 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 14, 2024-03-15T13:09:29.059 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-03-15T13:09:29.059 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 2, 2024-03-15T13:09:29.059 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-15T13:09:29.059 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-15T13:09:29.059 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 133, 2024-03-15T13:09:29.059 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 131, 2024-03-15T13:09:29.059 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-15T13:09:29.059 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-15T13:09:29.059 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-15T13:09:29.059 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-15T13:09:29.060 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-15T13:09:29.060 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-15T13:09:29.060 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-15T13:09:29.060 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-03-15T13:09:29.060 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 12, 2024-03-15T13:09:29.060 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-15T13:09:29.060 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-03-15T13:09:29.060 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-15T13:09:29.060 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-15T13:09:29.060 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.060 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-15T13:09:29.060 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-15T13:09:29.060 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 586, 2024-03-15T13:09:29.060 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-03-15T13:09:29.061 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-03-15T13:09:29.061 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 586, 2024-03-15T13:09:29.061 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-15T13:09:29.061 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-03-15T13:09:29.061 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 2, 2024-03-15T13:09:29.061 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-03-15T13:09:29.061 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-03-15T13:09:29.061 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 2, 2024-03-15T13:09:29.061 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-03-15T13:09:29.061 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-15T13:09:29.061 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-03-15T13:09:29.061 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-03-15T13:09:29.061 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 5435243, 2024-03-15T13:09:29.061 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-03-15T13:09:29.061 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-15T13:09:29.062 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 147, 2024-03-15T13:09:29.062 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.305979965, 2024-03-15T13:09:29.062 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002081496 2024-03-15T13:09:29.062 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.062 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-03-15T13:09:29.062 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.062 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-15T13:09:29.062 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 31, 2024-03-15T13:09:29.062 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 298, 2024-03-15T13:09:29.062 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 267, 2024-03-15T13:09:29.062 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 22, 2024-03-15T13:09:29.062 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 76, 2024-03-15T13:09:29.062 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 54, 2024-03-15T13:09:29.062 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 35, 2024-03-15T13:09:29.063 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 422, 2024-03-15T13:09:29.063 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 387, 2024-03-15T13:09:29.063 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 2, 2024-03-15T13:09:29.063 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 147, 2024-03-15T13:09:29.063 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 145, 2024-03-15T13:09:29.063 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 55740, 2024-03-15T13:09:29.063 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 223612 2024-03-15T13:09:29.063 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.063 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-15T13:09:29.063 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 476, 2024-03-15T13:09:29.063 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 12, 2024-03-15T13:09:29.063 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 282, 2024-03-15T13:09:29.063 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 26, 2024-03-15T13:09:29.063 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 97, 2024-03-15T13:09:29.063 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 24, 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.056520428, 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002355017 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.002420328, 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001210164 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-15T13:09:29.064 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.065 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.065 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.065 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-15T13:09:29.065 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 102, 2024-03-15T13:09:29.065 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.058903765, 2024-03-15T13:09:29.065 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000577487 2024-03-15T13:09:29.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.065 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-15T13:09:29.065 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.065 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.065 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.065 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-15T13:09:29.065 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.066 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.066 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.066 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.066 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-15T13:09:29.066 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.066 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.066 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.066 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.066 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-15T13:09:29.066 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.066 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.066 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.066 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.066 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-15T13:09:29.067 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.067 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.067 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.067 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-15T13:09:29.067 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.067 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.067 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.067 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-15T13:09:29.068 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.068 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.068 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.068 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.068 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-15T13:09:29.068 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.068 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.068 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.068 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.068 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-15T13:09:29.069 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.069 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.069 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.069 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.069 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-15T13:09:29.069 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.069 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.069 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.069 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.069 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-15T13:09:29.069 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 21, 2024-03-15T13:09:29.069 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.013433519, 2024-03-15T13:09:29.069 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000639691 2024-03-15T13:09:29.069 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.070 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-15T13:09:29.070 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.070 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.070 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.070 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.070 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-15T13:09:29.070 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.070 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.070 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.070 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.070 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-15T13:09:29.070 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.070 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.070 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.039699106, 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.019849553 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.071 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-15T13:09:29.072 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.072 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.072 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.072 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-15T13:09:29.072 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.072 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.072 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.072 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-15T13:09:29.072 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.072 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.072 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.073 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-15T13:09:29.073 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.073 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.073 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.073 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.073 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-15T13:09:29.073 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.073 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.073 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.073 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.073 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-15T13:09:29.073 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.073 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.073 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.074 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-15T13:09:29.074 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 131, 2024-03-15T13:09:29.074 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.282794641, 2024-03-15T13:09:29.074 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002158737 2024-03-15T13:09:29.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.074 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-15T13:09:29.074 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-15T13:09:29.074 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-15T13:09:29.074 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.074 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.074 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.074 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:29.075 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.075 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-15T13:09:29.075 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 2, 2024-03-15T13:09:29.075 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 4, 2024-03-15T13:09:29.075 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 2, 2024-03-15T13:09:29.075 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 2, 2024-03-15T13:09:29.075 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-15T13:09:29.075 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 196, 2024-03-15T13:09:29.075 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 98, 2024-03-15T13:09:29.075 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 639, 2024-03-15T13:09:29.075 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-15T13:09:29.075 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.075 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-15T13:09:29.075 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-15T13:09:29.075 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-15T13:09:29.076 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-15T13:09:29.076 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-15T13:09:29.076 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-15T13:09:29.076 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-15T13:09:29.076 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-15T13:09:29.076 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-15T13:09:29.076 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-15T13:09:29.076 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-15T13:09:29.076 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-15T13:09:29.076 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-15T13:09:29.076 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-15T13:09:29.076 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-15T13:09:29.076 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-15T13:09:29.076 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-15T13:09:29.077 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-15T13:09:29.077 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-15T13:09:29.077 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-15T13:09:29.077 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-15T13:09:29.077 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-15T13:09:29.077 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-15T13:09:29.077 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-15T13:09:29.077 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-15T13:09:29.077 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-15T13:09:29.077 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-15T13:09:29.077 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-15T13:09:29.077 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-15T13:09:29.077 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-15T13:09:29.077 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-15T13:09:29.077 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-15T13:09:29.078 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-15T13:09:29.078 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-15T13:09:29.078 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-15T13:09:29.078 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-15T13:09:29.078 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-15T13:09:29.078 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 300969, 2024-03-15T13:09:29.078 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 123, 2024-03-15T13:09:29.078 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-03-15T13:09:29.078 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-03-15T13:09:29.078 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-15T13:09:29.078 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-15T13:09:29.078 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-15T13:09:29.078 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-15T13:09:29.079 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-15T13:09:29.079 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-15T13:09:29.079 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 9448, 2024-03-15T13:09:29.079 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 197, 2024-03-15T13:09:29.079 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-15T13:09:29.079 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-15T13:09:29.079 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-15T13:09:29.079 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-15T13:09:29.079 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 141967, 2024-03-15T13:09:29.079 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 1001, 2024-03-15T13:09:29.079 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-15T13:09:29.079 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-15T13:09:29.079 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-15T13:09:29.079 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-15T13:09:29.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.080 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-15T13:09:29.080 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-15T13:09:29.080 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-15T13:09:29.080 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 715, 2024-03-15T13:09:29.080 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 1289676, 2024-03-15T13:09:29.080 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 33, 2024-03-15T13:09:29.080 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 715, 2024-03-15T13:09:29.080 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-15T13:09:29.080 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 715, 2024-03-15T13:09:29.080 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 1.826961721, 2024-03-15T13:09:29.080 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002555191 2024-03-15T13:09:29.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.080 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-15T13:09:29.080 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-15T13:09:29.080 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 715, 2024-03-15T13:09:29.081 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 809 2024-03-15T13:09:29.081 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.081 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 715, 2024-03-15T13:09:29.081 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 144, 2024-03-15T13:09:29.081 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 571, 2024-03-15T13:09:29.081 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-15T13:09:29.081 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-15T13:09:29.081 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 0, 2024-03-15T13:09:29.081 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 10, 2024-03-15T13:09:29.081 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 33, 2024-03-15T13:09:29.081 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 180, 2024-03-15T13:09:29.081 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 35, 2024-03-15T13:09:29.081 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-15T13:09:29.081 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 274, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 111, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 20, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-15T13:09:29.082 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-15T13:09:29.083 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 100, 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 11, 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 12, 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 12, 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 105, 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 41 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-15T13:09:29.084 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 10, 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 5, 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 262, 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 131, 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:29.085 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 3997, 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 2756734, 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 3997, 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 3997, 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 2756734, 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:29.086 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 715, 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 1304811, 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 715, 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1304811, 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-15T13:09:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 715, 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 715, 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 715, 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 715, 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 131, 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 15135, 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 131, 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 33, 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 15135, 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:29.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:29.090 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x555bbe5ec6e0": { 2024-03-15T13:09:29.090 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:29.090 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-15T13:09:29.090 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:29.090 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 586, 2024-03-15T13:09:29.090 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 1240939, 2024-03-15T13:09:29.090 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:29.090 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 586, 2024-03-15T13:09:29.090 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-15T13:09:29.090 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-15T13:09:29.090 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 147, 2024-03-15T13:09:29.090 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1240939, 2024-03-15T13:09:29.091 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:29.091 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:29.091 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:29.091 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:29.091 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:29.091 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:29.091 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-15T13:09:29.798 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:29 smithi149 ceph-mon[35927]: pgmap v328: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:29.830 INFO:tasks.check_counter:Daemon mds.h mds.exported=12 2024-03-15T13:09:29.830 INFO:tasks.check_counter:Daemon mds.h mds.imported=12 2024-03-15T13:09:29.830 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-03-15T13:09:29.831 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-15T13:09:29.831 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-15T13:09:29.831 DEBUG:tasks.check_counter:Getting stats from k 2024-03-15T13:09:29.831 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-15T13:09:30.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:29 smithi002 ceph-mon[32060]: pgmap v328: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:30.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:29 smithi184 ceph-mon[35993]: pgmap v328: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:30.152 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.b/config 2024-03-15T13:09:31.713 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:31 smithi149 ceph-mon[35927]: pgmap v329: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:09:32.024 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-15T13:09:32.024 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-15T13:09:32.025 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 4228, 2024-03-15T13:09:32.025 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 3575, 2024-03-15T13:09:32.025 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 3569668, 2024-03-15T13:09:32.025 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 4471365, 2024-03-15T13:09:32.025 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 13, 2024-03-15T13:09:32.025 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 5, 2024-03-15T13:09:32.025 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.322409890, 2024-03-15T13:09:32.025 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.141953197, 2024-03-15T13:09:32.025 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.206343579, 2024-03-15T13:09:32.025 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.020781165, 2024-03-15T13:09:32.025 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:32.025 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 3575, 2024-03-15T13:09:32.025 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.141473943, 2024-03-15T13:09:32.025 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000039573 2024-03-15T13:09:32.026 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.026 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:32.026 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 6094, 2024-03-15T13:09:32.026 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001083139, 2024-03-15T13:09:32.026 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000177 2024-03-15T13:09:32.026 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.026 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1756128, 2024-03-15T13:09:32.026 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 161472 2024-03-15T13:09:32.026 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.026 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-15T13:09:32.026 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1814, 2024-03-15T13:09:32.026 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 2847, 2024-03-15T13:09:32.026 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 585348, 2024-03-15T13:09:32.026 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 1849479, 2024-03-15T13:09:32.027 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 50, 2024-03-15T13:09:32.027 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 8, 2024-03-15T13:09:32.027 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.143470739, 2024-03-15T13:09:32.027 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.051909243, 2024-03-15T13:09:32.027 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.082218635, 2024-03-15T13:09:32.027 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.008300098, 2024-03-15T13:09:32.027 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:32.027 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2844, 2024-03-15T13:09:32.027 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.170142154, 2024-03-15T13:09:32.027 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000059824 2024-03-15T13:09:32.027 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.027 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:32.027 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 3532, 2024-03-15T13:09:32.027 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000671228, 2024-03-15T13:09:32.028 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000190 2024-03-15T13:09:32.028 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.028 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-03-15T13:09:32.028 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 134656 2024-03-15T13:09:32.028 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.028 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-15T13:09:32.028 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 401, 2024-03-15T13:09:32.028 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 531, 2024-03-15T13:09:32.028 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 105117, 2024-03-15T13:09:32.028 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 496772, 2024-03-15T13:09:32.028 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 9, 2024-03-15T13:09:32.028 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 6, 2024-03-15T13:09:32.028 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.039016782, 2024-03-15T13:09:32.028 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.014424846, 2024-03-15T13:09:32.029 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.009624364, 2024-03-15T13:09:32.029 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.011805909, 2024-03-15T13:09:32.029 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:32.029 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 529, 2024-03-15T13:09:32.029 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.060462388, 2024-03-15T13:09:32.029 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000114295 2024-03-15T13:09:32.029 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.029 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:32.029 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 16, 2024-03-15T13:09:32.029 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000002182, 2024-03-15T13:09:32.029 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000136 2024-03-15T13:09:32.029 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.029 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 2512, 2024-03-15T13:09:32.029 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 317792 2024-03-15T13:09:32.030 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.030 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-15T13:09:32.030 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-15T13:09:32.030 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-15T13:09:32.030 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.030 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-15T13:09:32.030 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-15T13:09:32.030 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-15T13:09:32.030 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 867, 2024-03-15T13:09:32.030 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.624117474, 2024-03-15T13:09:32.030 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000719858 2024-03-15T13:09:32.030 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:32.030 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.030 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-15T13:09:32.031 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-15T13:09:32.031 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-15T13:09:32.031 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 511, 2024-03-15T13:09:32.031 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.069193078, 2024-03-15T13:09:32.031 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000135407 2024-03-15T13:09:32.031 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:32.031 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.031 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-15T13:09:32.031 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 718, 2024-03-15T13:09:32.031 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 601, 2024-03-15T13:09:32.031 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-15T13:09:32.031 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 601, 2024-03-15T13:09:32.031 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 1.334142135, 2024-03-15T13:09:32.031 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002219870 2024-03-15T13:09:32.032 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.032 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-15T13:09:32.032 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 117, 2024-03-15T13:09:32.032 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-15T13:09:32.032 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-15T13:09:32.032 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 11, 2024-03-15T13:09:32.032 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-03-15T13:09:32.032 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-03-15T13:09:32.032 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 421, 2024-03-15T13:09:32.032 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 2, 2024-03-15T13:09:32.032 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 360, 2024-03-15T13:09:32.032 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 59, 2024-03-15T13:09:32.032 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 417, 2024-03-15T13:09:32.032 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 495, 2024-03-15T13:09:32.033 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 1, 2024-03-15T13:09:32.033 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 1, 2024-03-15T13:09:32.033 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 3, 2024-03-15T13:09:32.033 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 1515, 2024-03-15T13:09:32.033 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 834, 2024-03-15T13:09:32.033 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 98, 2024-03-15T13:09:32.033 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 15, 2024-03-15T13:09:32.033 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-15T13:09:32.033 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-15T13:09:32.033 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-03-15T13:09:32.033 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-03-15T13:09:32.033 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-15T13:09:32.033 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 23, 2024-03-15T13:09:32.033 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 432, 2024-03-15T13:09:32.034 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 24, 2024-03-15T13:09:32.034 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 682, 2024-03-15T13:09:32.034 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-15T13:09:32.034 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 168, 2024-03-15T13:09:32.034 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-15T13:09:32.034 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 4, 2024-03-15T13:09:32.034 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 234, 2024-03-15T13:09:32.034 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 7, 2024-03-15T13:09:32.034 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 168, 2024-03-15T13:09:32.034 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-15T13:09:32.034 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-15T13:09:32.034 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-15T13:09:32.034 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 7, 2024-03-15T13:09:32.035 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-03-15T13:09:32.035 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 7, 2024-03-15T13:09:32.035 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 161, 2024-03-15T13:09:32.035 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 4, 2024-03-15T13:09:32.035 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 34, 2024-03-15T13:09:32.035 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 385, 2024-03-15T13:09:32.035 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-15T13:09:32.035 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-15T13:09:32.035 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-15T13:09:32.035 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 707, 2024-03-15T13:09:32.035 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 456, 2024-03-15T13:09:32.035 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 23, 2024-03-15T13:09:32.035 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 624 2024-03-15T13:09:32.035 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.035 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 18, 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 15, 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 7, 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 257, 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 250, 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-15T13:09:32.036 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-15T13:09:32.037 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-15T13:09:32.037 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-03-15T13:09:32.037 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 24, 2024-03-15T13:09:32.037 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-15T13:09:32.037 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-03-15T13:09:32.037 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-15T13:09:32.037 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-15T13:09:32.037 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.037 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-15T13:09:32.037 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-15T13:09:32.037 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 1382, 2024-03-15T13:09:32.037 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-03-15T13:09:32.037 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-03-15T13:09:32.037 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 1382, 2024-03-15T13:09:32.037 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-15T13:09:32.038 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-03-15T13:09:32.038 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 3, 2024-03-15T13:09:32.038 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-03-15T13:09:32.038 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-03-15T13:09:32.038 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 3, 2024-03-15T13:09:32.038 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-03-15T13:09:32.038 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-15T13:09:32.038 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-03-15T13:09:32.038 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-03-15T13:09:32.038 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 7028686, 2024-03-15T13:09:32.038 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-03-15T13:09:32.038 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-15T13:09:32.038 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 328, 2024-03-15T13:09:32.038 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.689750391, 2024-03-15T13:09:32.039 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002102897 2024-03-15T13:09:32.039 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.039 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-03-15T13:09:32.039 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.039 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-15T13:09:32.039 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 225, 2024-03-15T13:09:32.039 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 812, 2024-03-15T13:09:32.039 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 587, 2024-03-15T13:09:32.039 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 31, 2024-03-15T13:09:32.039 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 88, 2024-03-15T13:09:32.039 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 57, 2024-03-15T13:09:32.039 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 421, 2024-03-15T13:09:32.039 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 1166, 2024-03-15T13:09:32.039 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 745, 2024-03-15T13:09:32.040 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 1, 2024-03-15T13:09:32.040 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 444, 2024-03-15T13:09:32.040 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 443, 2024-03-15T13:09:32.040 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 59008, 2024-03-15T13:09:32.040 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 223612 2024-03-15T13:09:32.040 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.040 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-15T13:09:32.040 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 1333, 2024-03-15T13:09:32.040 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 36, 2024-03-15T13:09:32.040 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 718, 2024-03-15T13:09:32.040 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 35, 2024-03-15T13:09:32.040 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 282, 2024-03-15T13:09:32.040 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-15T13:09:32.040 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 106, 2024-03-15T13:09:32.040 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.358007389, 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.003377428 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 5, 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001816591, 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000363318 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.041 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.042 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.042 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-15T13:09:32.042 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 203, 2024-03-15T13:09:32.042 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.251183990, 2024-03-15T13:09:32.042 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001237359 2024-03-15T13:09:32.042 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.042 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-15T13:09:32.042 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.042 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.042 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.042 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.042 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-15T13:09:32.042 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.043 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.043 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.043 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.043 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-15T13:09:32.043 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.043 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.043 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.043 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.043 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-15T13:09:32.043 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.043 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.043 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.043 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.043 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-15T13:09:32.043 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.044 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.044 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.044 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.044 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-15T13:09:32.044 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.044 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.044 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.044 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.045 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-15T13:09:32.045 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.045 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.045 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.045 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.045 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-15T13:09:32.045 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.045 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.045 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.045 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.045 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-15T13:09:32.045 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.045 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.045 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.045 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.046 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-15T13:09:32.046 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.046 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.046 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.046 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.046 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-15T13:09:32.046 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 29, 2024-03-15T13:09:32.046 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.026491345, 2024-03-15T13:09:32.046 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000913494 2024-03-15T13:09:32.046 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.046 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-15T13:09:32.046 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.046 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.046 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 7, 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.123594486, 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.017656355 2024-03-15T13:09:32.047 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000890561, 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000890561 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.048 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-15T13:09:32.049 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.049 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.049 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.049 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.049 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-15T13:09:32.049 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.049 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.049 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.049 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.049 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-15T13:09:32.049 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.049 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.049 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.049 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.049 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-15T13:09:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-15T13:09:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-15T13:09:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 250, 2024-03-15T13:09:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.572157773, 2024-03-15T13:09:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002288631 2024-03-15T13:09:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.050 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 2, 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 4, 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 2, 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 2, 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 290, 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 145, 2024-03-15T13:09:32.051 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 642, 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-15T13:09:32.052 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-15T13:09:32.053 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-15T13:09:32.054 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-15T13:09:32.054 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-15T13:09:32.054 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-15T13:09:32.054 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-15T13:09:32.054 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-15T13:09:32.054 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-15T13:09:32.054 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-15T13:09:32.054 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 347345, 2024-03-15T13:09:32.054 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 130, 2024-03-15T13:09:32.054 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-03-15T13:09:32.054 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-03-15T13:09:32.054 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-15T13:09:32.054 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-15T13:09:32.055 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-15T13:09:32.055 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-15T13:09:32.055 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-15T13:09:32.055 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-15T13:09:32.055 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 9448, 2024-03-15T13:09:32.055 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 197, 2024-03-15T13:09:32.055 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-15T13:09:32.055 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-15T13:09:32.055 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-15T13:09:32.055 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-15T13:09:32.055 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 834413, 2024-03-15T13:09:32.055 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 9551, 2024-03-15T13:09:32.055 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-15T13:09:32.055 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-15T13:09:32.055 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 1196, 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 2923183, 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 33, 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 1196, 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1196, 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 3.070935618, 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002567671 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-15T13:09:32.056 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1196, 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 1293 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 1196, 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 201, 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 995, 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 0, 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 10, 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 33, 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 361, 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 37, 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-15T13:09:32.057 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-03-15T13:09:32.058 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 512, 2024-03-15T13:09:32.058 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-15T13:09:32.058 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-15T13:09:32.058 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-15T13:09:32.058 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 168, 2024-03-15T13:09:32.058 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 20, 2024-03-15T13:09:32.058 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-15T13:09:32.058 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-15T13:09:32.058 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-15T13:09:32.058 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-15T13:09:32.058 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-15T13:09:32.058 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-15T13:09:32.058 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-15T13:09:32.058 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-15T13:09:32.059 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 100, 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 11, 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 12, 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 12, 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 107, 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 45 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-15T13:09:32.060 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 10, 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 5, 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 500, 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 250, 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:32.061 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-15T13:09:32.062 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:32.062 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 6439, 2024-03-15T13:09:32.062 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 3660712, 2024-03-15T13:09:32.062 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:32.062 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 6439, 2024-03-15T13:09:32.062 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-15T13:09:32.062 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-15T13:09:32.062 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 6439, 2024-03-15T13:09:32.062 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 3660712, 2024-03-15T13:09:32.062 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:32.062 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.062 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.062 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.062 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:32.063 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.063 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-15T13:09:32.063 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:32.063 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-15T13:09:32.063 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:32.063 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-15T13:09:32.063 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-15T13:09:32.063 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:32.063 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:32.063 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 1196, 2024-03-15T13:09:32.063 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 2951577, 2024-03-15T13:09:32.063 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1196, 2024-03-15T13:09:32.063 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 2951577, 2024-03-15T13:09:32.063 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:32.064 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.064 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.064 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.064 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:32.064 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.064 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-15T13:09:32.064 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:32.064 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-15T13:09:32.064 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:32.064 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-15T13:09:32.064 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-15T13:09:32.064 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:32.064 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:32.064 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 1196, 2024-03-15T13:09:32.064 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 1196, 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1196, 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1196, 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 250, 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 28394, 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:32.065 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 250, 2024-03-15T13:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-15T13:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-15T13:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 33, 2024-03-15T13:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 28394, 2024-03-15T13:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-15T13:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x560e30aa26e0": { 2024-03-15T13:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-15T13:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-15T13:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-15T13:09:32.066 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 1382, 2024-03-15T13:09:32.067 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 2834382, 2024-03-15T13:09:32.067 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:32.067 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 1382, 2024-03-15T13:09:32.067 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-15T13:09:32.067 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-15T13:09:32.067 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 328, 2024-03-15T13:09:32.067 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 2834382, 2024-03-15T13:09:32.067 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-15T13:09:32.067 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-15T13:09:32.067 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-15T13:09:32.067 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-15T13:09:32.067 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:32.067 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-15T13:09:32.067 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-15T13:09:32.068 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:31 smithi184 ceph-mon[35993]: pgmap v329: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:09:32.068 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:31 smithi002 ceph-mon[32060]: pgmap v329: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:09:32.809 INFO:tasks.check_counter:Daemon mds.k mds.exported=23 2024-03-15T13:09:32.809 INFO:tasks.check_counter:Daemon mds.k mds.imported=24 2024-03-15T13:09:32.809 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=7 2024-03-15T13:09:32.809 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-03-15T13:09:32.810 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-03-15T13:09:32.810 DEBUG:tasks.check_counter:Getting stats from c 2024-03-15T13:09:32.810 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-15T13:09:33.064 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T13:09:33.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:33 smithi184 ceph-mon[35993]: pgmap v330: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:33.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:33 smithi149 ceph-mon[35927]: pgmap v330: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:34.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:33 smithi002 ceph-mon[32060]: pgmap v330: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:34.975 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-03-15T13:09:34.975 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-03-15T13:09:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 1049, 2024-03-15T13:09:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1009, 2024-03-15T13:09:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 2020576, 2024-03-15T13:09:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 374666, 2024-03-15T13:09:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 5, 2024-03-15T13:09:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 1, 2024-03-15T13:09:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.093188815, 2024-03-15T13:09:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.024751518, 2024-03-15T13:09:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.050744738, 2024-03-15T13:09:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.024194219, 2024-03-15T13:09:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1008, 2024-03-15T13:09:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.030817562, 2024-03-15T13:09:34.976 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000030572 2024-03-15T13:09:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 1760512, 2024-03-15T13:09:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 133952 2024-03-15T13:09:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-03-15T13:09:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 4, 2024-03-15T13:09:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 170, 2024-03-15T13:09:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 876, 2024-03-15T13:09:34.977 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 451228, 2024-03-15T13:09:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 6, 2024-03-15T13:09:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 4, 2024-03-15T13:09:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.008092391, 2024-03-15T13:09:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.004584395, 2024-03-15T13:09:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.002052223, 2024-03-15T13:09:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000076621, 2024-03-15T13:09:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 167, 2024-03-15T13:09:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.012677119, 2024-03-15T13:09:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000075910 2024-03-15T13:09:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:34.978 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-15T13:09:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 450544 2024-03-15T13:09:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-03-15T13:09:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 1497, 2024-03-15T13:09:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1497, 2024-03-15T13:09:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 1624238, 2024-03-15T13:09:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 464778, 2024-03-15T13:09:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 4, 2024-03-15T13:09:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 3, 2024-03-15T13:09:34.979 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.075863437, 2024-03-15T13:09:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.022478021, 2024-03-15T13:09:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.020014218, 2024-03-15T13:09:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.028727481, 2024-03-15T13:09:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1496, 2024-03-15T13:09:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.035544126, 2024-03-15T13:09:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000023759 2024-03-15T13:09:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-15T13:09:34.980 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 448 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3870, 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.072398523, 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000018707 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-03-15T13:09:34.981 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-03-15T13:09:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-03-15T13:09:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 0, 2024-03-15T13:09:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 0, 2024-03-15T13:09:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-03-15T13:09:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.982 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-03-15T13:09:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 0, 2024-03-15T13:09:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-03-15T13:09:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-03-15T13:09:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 0, 2024-03-15T13:09:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-03-15T13:09:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-03-15T13:09:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 150, 2024-03-15T13:09:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 0, 2024-03-15T13:09:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 150, 2024-03-15T13:09:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-03-15T13:09:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 150, 2024-03-15T13:09:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 1, 2024-03-15T13:09:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2024-03-15T13:09:34.983 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2024-03-15T13:09:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 1, 2024-03-15T13:09:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 0, 2024-03-15T13:09:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 0, 2024-03-15T13:09:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2024-03-15T13:09:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 0, 2024-03-15T13:09:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-03-15T13:09:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-03-15T13:09:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-03-15T13:09:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2024-03-15T13:09:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-03-15T13:09:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-03-15T13:09:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-03-15T13:09:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-03-15T13:09:34.984 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-03-15T13:09:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-03-15T13:09:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-03-15T13:09:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-03-15T13:09:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 0, 2024-03-15T13:09:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 0, 2024-03-15T13:09:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2024-03-15T13:09:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-03-15T13:09:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-03-15T13:09:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-03-15T13:09:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-03-15T13:09:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-03-15T13:09:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-03-15T13:09:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-03-15T13:09:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-03-15T13:09:34.985 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-03-15T13:09:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2024-03-15T13:09:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-03-15T13:09:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-03-15T13:09:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-15T13:09:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-15T13:09:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2024-03-15T13:09:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-03-15T13:09:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-03-15T13:09:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2024-03-15T13:09:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-03-15T13:09:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-03-15T13:09:34.986 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 0, 2024-03-15T13:09:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 0, 2024-03-15T13:09:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 0, 2024-03-15T13:09:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-03-15T13:09:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-03-15T13:09:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-03-15T13:09:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-03-15T13:09:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-03-15T13:09:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-03-15T13:09:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-03-15T13:09:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-03-15T13:09:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-03-15T13:09:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-03-15T13:09:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-03-15T13:09:34.987 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-03-15T13:09:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-03-15T13:09:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-03-15T13:09:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-03-15T13:09:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-03-15T13:09:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-03-15T13:09:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-03-15T13:09:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2024-03-15T13:09:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-03-15T13:09:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-03-15T13:09:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 0, 2024-03-15T13:09:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-03-15T13:09:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-03-15T13:09:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 586, 2024-03-15T13:09:34.988 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-03-15T13:09:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-03-15T13:09:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 0, 2024-03-15T13:09:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-03-15T13:09:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-03-15T13:09:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 2, 2024-03-15T13:09:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2024-03-15T13:09:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-03-15T13:09:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-03-15T13:09:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-03-15T13:09:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 5435243, 2024-03-15T13:09:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 5432682, 2024-03-15T13:09:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-03-15T13:09:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.989 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 586 2024-03-15T13:09:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-03-15T13:09:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 14, 2024-03-15T13:09:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 303, 2024-03-15T13:09:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 289, 2024-03-15T13:09:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 4, 2024-03-15T13:09:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 83, 2024-03-15T13:09:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 79, 2024-03-15T13:09:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 150, 2024-03-15T13:09:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 412, 2024-03-15T13:09:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 262, 2024-03-15T13:09:34.990 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2024-03-15T13:09:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 0, 2024-03-15T13:09:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 0, 2024-03-15T13:09:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 51036, 2024-03-15T13:09:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 215420 2024-03-15T13:09:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-03-15T13:09:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 0, 2024-03-15T13:09:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-03-15T13:09:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 0, 2024-03-15T13:09:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 0, 2024-03-15T13:09:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 0, 2024-03-15T13:09:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-03-15T13:09:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.991 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-03-15T13:09:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-03-15T13:09:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-03-15T13:09:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.992 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.993 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-03-15T13:09:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-03-15T13:09:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.994 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-03-15T13:09:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-03-15T13:09:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.998 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-03-15T13:09:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-03-15T13:09:34.999 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.000 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-03-15T13:09:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-03-15T13:09:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-03-15T13:09:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.001 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-03-15T13:09:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-03-15T13:09:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-03-15T13:09:35.002 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-03-15T13:09:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-03-15T13:09:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-03-15T13:09:35.003 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-03-15T13:09:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-03-15T13:09:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.004 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-03-15T13:09:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-03-15T13:09:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-03-15T13:09:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-03-15T13:09:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-03-15T13:09:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.005 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-03-15T13:09:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 2, 2024-03-15T13:09:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 4, 2024-03-15T13:09:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 2, 2024-03-15T13:09:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 4, 2024-03-15T13:09:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-03-15T13:09:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 0, 2024-03-15T13:09:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 0, 2024-03-15T13:09:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 3012, 2024-03-15T13:09:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-03-15T13:09:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.006 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-03-15T13:09:35.007 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-03-15T13:09:35.008 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 181330, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 48, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 9448, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 197, 2024-03-15T13:09:35.009 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-03-15T13:09:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-03-15T13:09:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-03-15T13:09:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-03-15T13:09:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 147359, 2024-03-15T13:09:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 1132, 2024-03-15T13:09:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-03-15T13:09:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-03-15T13:09:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-03-15T13:09:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-03-15T13:09:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-03-15T13:09:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-03-15T13:09:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-03-15T13:09:35.010 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 2269, 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 21, 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 3, 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 2269, 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2269, 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 1.052977259, 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000464071 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2269, 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 2270 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 2269, 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 2269, 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 0, 2024-03-15T13:09:35.011 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 1432, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 0, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 835, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 0, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 0, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 0, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 1, 2024-03-15T13:09:35.012 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 0, 2024-03-15T13:09:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-03-15T13:09:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-03-15T13:09:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-03-15T13:09:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-03-15T13:09:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-03-15T13:09:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-03-15T13:09:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-03-15T13:09:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-03-15T13:09:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-03-15T13:09:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-03-15T13:09:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-03-15T13:09:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-03-15T13:09:35.013 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-03-15T13:09:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-03-15T13:09:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-03-15T13:09:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-03-15T13:09:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-03-15T13:09:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-03-15T13:09:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-03-15T13:09:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-03-15T13:09:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-03-15T13:09:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-03-15T13:09:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-03-15T13:09:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-03-15T13:09:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-03-15T13:09:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-03-15T13:09:35.014 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 100, 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 11, 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 4, 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 4, 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 0, 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 0, 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 2, 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 0 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 0, 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-03-15T13:09:35.015 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2024-03-15T13:09:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2024-03-15T13:09:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-03-15T13:09:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-03-15T13:09:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-03-15T13:09:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-03-15T13:09:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-03-15T13:09:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-03-15T13:09:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-03-15T13:09:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-03-15T13:09:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-15T13:09:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:35.016 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 2546, 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 3399641, 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 2546, 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2546, 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 3399641, 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.017 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-03-15T13:09:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-03-15T13:09:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-03-15T13:09:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-03-15T13:09:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 2269, 2024-03-15T13:09:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 1240944, 2024-03-15T13:09:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2269, 2024-03-15T13:09:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 1240944, 2024-03-15T13:09:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.018 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-03-15T13:09:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-03-15T13:09:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-03-15T13:09:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-03-15T13:09:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 2269, 2024-03-15T13:09:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 2269, 2024-03-15T13:09:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2269, 2024-03-15T13:09:35.019 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 2269, 2024-03-15T13:09:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-03-15T13:09:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-03-15T13:09:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-03-15T13:09:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-03-15T13:09:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:35.020 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x55ef8786e960": { 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-03-15T13:09:35.021 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-03-15T13:09:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-03-15T13:09:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-03-15T13:09:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-03-15T13:09:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:35.022 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:35.022 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-03-15T13:09:35.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:35 smithi184 ceph-mon[35993]: pgmap v331: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:35.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:35 smithi149 ceph-mon[35927]: pgmap v331: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:35.864 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-15T13:09:35.864 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-15T13:09:35.864 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-03-15T13:09:35.864 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-15T13:09:35.864 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-15T13:09:35.864 DEBUG:tasks.check_counter:Getting stats from f 2024-03-15T13:09:35.865 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-15T13:09:36.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:35 smithi002 ceph-mon[32060]: pgmap v331: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:36.120 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T13:09:37.825 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:37 smithi184 ceph-mon[35993]: pgmap v332: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:09:37.825 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-03-15T13:09:37.825 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-03-15T13:09:37.825 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 4988, 2024-03-15T13:09:37.826 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 7655, 2024-03-15T13:09:37.826 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 1930286, 2024-03-15T13:09:37.826 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 8328048, 2024-03-15T13:09:37.826 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 25, 2024-03-15T13:09:37.826 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 4, 2024-03-15T13:09:37.826 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.307859931, 2024-03-15T13:09:37.826 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.126839738, 2024-03-15T13:09:37.826 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.158402380, 2024-03-15T13:09:37.826 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.033704172, 2024-03-15T13:09:37.826 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:37.826 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 7654, 2024-03-15T13:09:37.826 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.177137313, 2024-03-15T13:09:37.826 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000023143 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 8786, 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.001373695, 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000156 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 448 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 1530, 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1525, 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 2091351, 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 4346713, 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 37, 2024-03-15T13:09:37.827 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 10, 2024-03-15T13:09:37.828 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.136917295, 2024-03-15T13:09:37.828 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.042170491, 2024-03-15T13:09:37.828 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.070559712, 2024-03-15T13:09:37.828 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.032392881, 2024-03-15T13:09:37.828 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:37.828 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1524, 2024-03-15T13:09:37.828 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.072611027, 2024-03-15T13:09:37.828 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000047645 2024-03-15T13:09:37.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.828 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:37.828 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 204, 2024-03-15T13:09:37.828 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000026762, 2024-03-15T13:09:37.828 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000131 2024-03-15T13:09:37.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.828 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 1759088, 2024-03-15T13:09:37.829 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 153760 2024-03-15T13:09:37.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.829 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-03-15T13:09:37.829 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 5293, 2024-03-15T13:09:37.829 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 4686, 2024-03-15T13:09:37.829 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 2158616, 2024-03-15T13:09:37.829 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 2034204, 2024-03-15T13:09:37.829 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 11, 2024-03-15T13:09:37.829 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 6, 2024-03-15T13:09:37.829 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.271284946, 2024-03-15T13:09:37.829 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.107924941, 2024-03-15T13:09:37.829 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.160795904, 2024-03-15T13:09:37.829 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.026446815, 2024-03-15T13:09:37.829 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:37.829 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 4683, 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.120851213, 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000025806 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 7595, 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.001028128, 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000135 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 450352 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.830 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-03-15T13:09:37.831 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-03-15T13:09:37.831 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-03-15T13:09:37.831 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1581, 2024-03-15T13:09:37.831 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 1.352253310, 2024-03-15T13:09:37.831 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000855315 2024-03-15T13:09:37.831 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:37.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.831 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-03-15T13:09:37.831 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-03-15T13:09:37.831 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-03-15T13:09:37.831 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1872, 2024-03-15T13:09:37.831 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.153683202, 2024-03-15T13:09:37.831 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000082095 2024-03-15T13:09:37.831 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:37.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 1755, 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 1739, 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1739, 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 2.484139612, 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001428487 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 16, 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 11, 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-03-15T13:09:37.832 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 431, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 4, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 3, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 424, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 424, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 943, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 1, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 1, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 4, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 2964, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 2218, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 10, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 34, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2024-03-15T13:09:37.833 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-03-15T13:09:37.834 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 31, 2024-03-15T13:09:37.834 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 851, 2024-03-15T13:09:37.834 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 32, 2024-03-15T13:09:37.834 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 631, 2024-03-15T13:09:37.834 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-03-15T13:09:37.834 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 890, 2024-03-15T13:09:37.834 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-03-15T13:09:37.834 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 4, 2024-03-15T13:09:37.834 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 234, 2024-03-15T13:09:37.834 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 7, 2024-03-15T13:09:37.834 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 890, 2024-03-15T13:09:37.834 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-03-15T13:09:37.834 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-03-15T13:09:37.834 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-03-15T13:09:37.835 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 9, 2024-03-15T13:09:37.835 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-03-15T13:09:37.835 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 9, 2024-03-15T13:09:37.835 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 881, 2024-03-15T13:09:37.835 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 4, 2024-03-15T13:09:37.835 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 261, 2024-03-15T13:09:37.835 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 1311, 2024-03-15T13:09:37.835 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-03-15T13:09:37.835 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-15T13:09:37.835 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-15T13:09:37.835 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 1718, 2024-03-15T13:09:37.835 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 839, 2024-03-15T13:09:37.835 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 35, 2024-03-15T13:09:37.835 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 2152 2024-03-15T13:09:37.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 0, 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 0, 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 33, 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 10, 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 6, 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 924, 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 918, 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-03-15T13:09:37.836 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 31, 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 3577, 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 3577, 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-03-15T13:09:37.837 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-03-15T13:09:37.838 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 5, 2024-03-15T13:09:37.838 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-03-15T13:09:37.838 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-03-15T13:09:37.838 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 5, 2024-03-15T13:09:37.838 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2024-03-15T13:09:37.838 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-03-15T13:09:37.838 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-03-15T13:09:37.838 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-03-15T13:09:37.838 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 11854882, 2024-03-15T13:09:37.838 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 0, 2024-03-15T13:09:37.838 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-03-15T13:09:37.838 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 409, 2024-03-15T13:09:37.838 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.780476520, 2024-03-15T13:09:37.838 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001908255 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 0 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 232, 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 1627, 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 1395, 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 38, 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 117, 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 79, 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 431, 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 2292, 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 1861, 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 1, 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 951, 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 950, 2024-03-15T13:09:37.839 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 65748, 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 223612 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 2602, 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 33, 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 1755, 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 21, 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 338, 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 300, 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.386046189, 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001286820 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2, 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.001961948, 2024-03-15T13:09:37.840 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000980974 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 450, 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.263014848, 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000584477 2024-03-15T13:09:37.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.842 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-03-15T13:09:37.842 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.842 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.842 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.842 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-03-15T13:09:37.842 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.842 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.842 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.842 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-03-15T13:09:37.842 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.842 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.842 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.842 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.843 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-03-15T13:09:37.843 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.843 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.843 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.843 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-03-15T13:09:37.843 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.843 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.843 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.843 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-03-15T13:09:37.843 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.843 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.843 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.844 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-03-15T13:09:37.844 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.844 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.844 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.844 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.844 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-03-15T13:09:37.844 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.844 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 62, 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.099588933, 2024-03-15T13:09:37.845 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001606273 2024-03-15T13:09:37.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.846 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-03-15T13:09:37.846 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.846 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.846 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.846 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-03-15T13:09:37.846 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.846 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.846 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.846 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-03-15T13:09:37.846 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.846 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 6, 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.201071297, 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.033511882 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.847 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1, 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000815511, 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000815511 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.848 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-03-15T13:09:37.849 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.849 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.849 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.849 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-03-15T13:09:37.849 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.849 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.849 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.849 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-03-15T13:09:37.849 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.849 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.849 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.849 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.849 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-03-15T13:09:37.850 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 918, 2024-03-15T13:09:37.850 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 1.531640886, 2024-03-15T13:09:37.850 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001668454 2024-03-15T13:09:37.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.850 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-03-15T13:09:37.850 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-03-15T13:09:37.850 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-03-15T13:09:37.850 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.850 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.850 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.850 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:37.850 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.850 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-03-15T13:09:37.850 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 1, 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 4, 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 3, 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 1, 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 1132, 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 1132, 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 661, 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-03-15T13:09:37.851 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-03-15T13:09:37.852 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-15T13:09:37.852 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-03-15T13:09:37.852 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-15T13:09:37.852 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-03-15T13:09:37.852 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-03-15T13:09:37.852 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-03-15T13:09:37.852 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-15T13:09:37.852 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-03-15T13:09:37.852 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-03-15T13:09:37.852 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-03-15T13:09:37.852 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-03-15T13:09:37.852 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-03-15T13:09:37.852 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-15T13:09:37.852 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-03-15T13:09:37.852 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-03-15T13:09:37.853 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 354731, 2024-03-15T13:09:37.854 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 134, 2024-03-15T13:09:37.854 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-03-15T13:09:37.854 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-03-15T13:09:37.854 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-03-15T13:09:37.854 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-03-15T13:09:37.854 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-03-15T13:09:37.854 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-03-15T13:09:37.854 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-03-15T13:09:37.854 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-03-15T13:09:37.854 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 9448, 2024-03-15T13:09:37.854 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 197, 2024-03-15T13:09:37.854 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-03-15T13:09:37.854 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-03-15T13:09:37.854 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-03-15T13:09:37.855 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-03-15T13:09:37.855 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 879282, 2024-03-15T13:09:37.855 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 10401, 2024-03-15T13:09:37.855 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-03-15T13:09:37.855 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-03-15T13:09:37.855 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-03-15T13:09:37.855 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-03-15T13:09:37.855 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.855 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-03-15T13:09:37.855 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-03-15T13:09:37.857 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-03-15T13:09:37.857 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 3423, 2024-03-15T13:09:37.857 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 7852738, 2024-03-15T13:09:37.857 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 33, 2024-03-15T13:09:37.858 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 3423, 2024-03-15T13:09:37.858 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-03-15T13:09:37.858 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3423, 2024-03-15T13:09:37.858 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 9.009803351, 2024-03-15T13:09:37.858 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.002632136 2024-03-15T13:09:37.858 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.858 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-03-15T13:09:37.858 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-03-15T13:09:37.858 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3423, 2024-03-15T13:09:37.858 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 3552 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 3423, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 952, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 2471, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 0, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 10, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 62, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 471, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 48, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 1849, 2024-03-15T13:09:37.859 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-03-15T13:09:37.860 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-03-15T13:09:37.860 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-03-15T13:09:37.860 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 890, 2024-03-15T13:09:37.860 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 20, 2024-03-15T13:09:37.860 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-03-15T13:09:37.860 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-03-15T13:09:37.860 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-03-15T13:09:37.860 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-03-15T13:09:37.860 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-03-15T13:09:37.860 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-03-15T13:09:37.860 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-03-15T13:09:37.860 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-03-15T13:09:37.860 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-03-15T13:09:37.860 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-03-15T13:09:37.860 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-03-15T13:09:37.861 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-03-15T13:09:37.861 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-03-15T13:09:37.861 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-03-15T13:09:37.861 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-03-15T13:09:37.861 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-03-15T13:09:37.861 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-03-15T13:09:37.861 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-03-15T13:09:37.861 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-03-15T13:09:37.861 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-03-15T13:09:37.861 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-03-15T13:09:37.861 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-03-15T13:09:37.861 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-03-15T13:09:37.861 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-03-15T13:09:37.861 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-03-15T13:09:37.861 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 100, 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 11, 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 12, 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 12, 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 0, 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 144, 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 0, 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 58 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 1, 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-03-15T13:09:37.862 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 10, 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 5, 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 1836, 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 918, 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:37.863 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-03-15T13:09:37.864 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:37.864 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 11807, 2024-03-15T13:09:37.864 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 5098455, 2024-03-15T13:09:37.864 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:37.864 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 11807, 2024-03-15T13:09:37.864 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-03-15T13:09:37.864 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-03-15T13:09:37.864 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 11807, 2024-03-15T13:09:37.864 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 5098455, 2024-03-15T13:09:37.864 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:37.864 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.864 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.864 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.864 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:37.864 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 3423, 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 7957744, 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3423, 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 7957744, 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.865 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 3423, 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 3423, 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3423, 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 3423, 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:37.866 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 918, 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 105006, 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 918, 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 62, 2024-03-15T13:09:37.867 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 105006, 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x558e394566e0": { 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 3577, 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 7660578, 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 3577, 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-03-15T13:09:37.868 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-03-15T13:09:37.869 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 409, 2024-03-15T13:09:37.869 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 7660578, 2024-03-15T13:09:37.869 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:37.869 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:37.869 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:37.869 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:37.869 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:37.869 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:37.869 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-03-15T13:09:37.870 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:37 smithi149 ceph-mon[35927]: pgmap v332: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:09:38.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:37 smithi002 ceph-mon[32060]: pgmap v332: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:09:38.598 INFO:tasks.check_counter:Daemon mds.f mds.exported=31 2024-03-15T13:09:38.599 INFO:tasks.check_counter:Daemon mds.f mds.imported=32 2024-03-15T13:09:38.599 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=7 2024-03-15T13:09:38.599 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-15T13:09:38.599 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-15T13:09:38.599 DEBUG:tasks.check_counter:Getting stats from i 2024-03-15T13:09:38.599 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-15T13:09:38.847 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T13:09:39.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:39 smithi149 ceph-mon[35927]: pgmap v333: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:40.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:39 smithi002 ceph-mon[32060]: pgmap v333: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:40.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:39 smithi184 ceph-mon[35993]: pgmap v333: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:40.735 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-03-15T13:09:40.736 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-03-15T13:09:40.736 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 2103, 2024-03-15T13:09:40.736 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 2105, 2024-03-15T13:09:40.736 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 51384749, 2024-03-15T13:09:40.736 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 654043, 2024-03-15T13:09:40.736 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 8, 2024-03-15T13:09:40.736 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 4, 2024-03-15T13:09:40.736 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.143413688, 2024-03-15T13:09:40.736 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.035133003, 2024-03-15T13:09:40.736 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.055639951, 2024-03-15T13:09:40.736 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.042034623, 2024-03-15T13:09:40.736 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:40.736 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2103, 2024-03-15T13:09:40.736 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.048207774, 2024-03-15T13:09:40.736 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000022923 2024-03-15T13:09:40.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.737 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:40.737 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.737 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.737 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.737 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-15T13:09:40.737 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 0 2024-03-15T13:09:40.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.737 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-03-15T13:09:40.737 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 438, 2024-03-15T13:09:40.737 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 398, 2024-03-15T13:09:40.737 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 39556930, 2024-03-15T13:09:40.737 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 184560, 2024-03-15T13:09:40.738 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 5, 2024-03-15T13:09:40.738 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 4, 2024-03-15T13:09:40.738 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.062309673, 2024-03-15T13:09:40.738 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.012215340, 2024-03-15T13:09:40.738 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.055206594, 2024-03-15T13:09:40.738 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.006703130, 2024-03-15T13:09:40.738 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:40.738 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 397, 2024-03-15T13:09:40.738 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.014846248, 2024-03-15T13:09:40.738 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000037396 2024-03-15T13:09:40.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.738 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:40.738 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.739 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.739 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.739 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 1760672, 2024-03-15T13:09:40.739 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 134112 2024-03-15T13:09:40.739 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.739 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-03-15T13:09:40.739 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 374, 2024-03-15T13:09:40.739 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 538, 2024-03-15T13:09:40.739 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 29458894, 2024-03-15T13:09:40.739 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 567041, 2024-03-15T13:09:40.739 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 8, 2024-03-15T13:09:40.739 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 6, 2024-03-15T13:09:40.739 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.045172296, 2024-03-15T13:09:40.739 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.012031269, 2024-03-15T13:09:40.740 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.021439629, 2024-03-15T13:09:40.740 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.008318385, 2024-03-15T13:09:40.740 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:40.740 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 536, 2024-03-15T13:09:40.740 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.017733286, 2024-03-15T13:09:40.740 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000033084 2024-03-15T13:09:40.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.740 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:40.740 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.740 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.740 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.740 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.740 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-15T13:09:40.741 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 452592 2024-03-15T13:09:40.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.741 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-03-15T13:09:40.741 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-03-15T13:09:40.741 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-03-15T13:09:40.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.741 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-03-15T13:09:40.741 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-03-15T13:09:40.741 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-03-15T13:09:40.741 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 4309, 2024-03-15T13:09:40.741 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.078593675, 2024-03-15T13:09:40.741 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000018239 2024-03-15T13:09:40.741 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:40.741 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.742 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-03-15T13:09:40.742 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-03-15T13:09:40.742 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-03-15T13:09:40.742 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.742 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.742 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.742 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:40.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.742 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-03-15T13:09:40.742 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 0, 2024-03-15T13:09:40.742 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 0, 2024-03-15T13:09:40.742 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-03-15T13:09:40.742 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.743 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.743 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.743 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.743 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-03-15T13:09:40.743 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 0, 2024-03-15T13:09:40.743 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-03-15T13:09:40.743 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-03-15T13:09:40.743 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 0, 2024-03-15T13:09:40.743 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-03-15T13:09:40.743 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-03-15T13:09:40.743 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 25616, 2024-03-15T13:09:40.743 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 0, 2024-03-15T13:09:40.743 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 25616, 2024-03-15T13:09:40.743 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-03-15T13:09:40.743 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 25616, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 0, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 2, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 0, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 0, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 0, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-03-15T13:09:40.744 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-03-15T13:09:40.745 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-03-15T13:09:40.745 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-03-15T13:09:40.745 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-03-15T13:09:40.745 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 4, 2024-03-15T13:09:40.745 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 234, 2024-03-15T13:09:40.745 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 7, 2024-03-15T13:09:40.745 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-03-15T13:09:40.745 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-03-15T13:09:40.745 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-03-15T13:09:40.745 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-03-15T13:09:40.745 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-03-15T13:09:40.745 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-03-15T13:09:40.745 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-03-15T13:09:40.745 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-03-15T13:09:40.745 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-03-15T13:09:40.746 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2024-03-15T13:09:40.746 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-03-15T13:09:40.746 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-03-15T13:09:40.746 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-15T13:09:40.746 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-15T13:09:40.746 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2024-03-15T13:09:40.746 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-03-15T13:09:40.746 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-03-15T13:09:40.746 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2024-03-15T13:09:40.746 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.746 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-03-15T13:09:40.746 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-03-15T13:09:40.746 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 0, 2024-03-15T13:09:40.746 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-03-15T13:09:40.747 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 0, 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 48215, 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 0, 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-03-15T13:09:40.748 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 67, 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 5, 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 122085710, 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 122085574, 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 48215 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-03-15T13:09:40.749 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 7615, 2024-03-15T13:09:40.750 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 18322, 2024-03-15T13:09:40.750 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 10707, 2024-03-15T13:09:40.750 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 728, 2024-03-15T13:09:40.750 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 784, 2024-03-15T13:09:40.750 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 56, 2024-03-15T13:09:40.750 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 25616, 2024-03-15T13:09:40.750 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 26570, 2024-03-15T13:09:40.750 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 954, 2024-03-15T13:09:40.750 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2024-03-15T13:09:40.750 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 0, 2024-03-15T13:09:40.750 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 0, 2024-03-15T13:09:40.750 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 94108, 2024-03-15T13:09:40.750 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 215420 2024-03-15T13:09:40.750 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.750 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-03-15T13:09:40.751 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 0, 2024-03-15T13:09:40.751 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-03-15T13:09:40.751 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 0, 2024-03-15T13:09:40.751 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 0, 2024-03-15T13:09:40.751 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 0, 2024-03-15T13:09:40.751 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-03-15T13:09:40.751 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.751 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.751 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.751 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.751 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-03-15T13:09:40.751 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.751 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.751 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.751 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-03-15T13:09:40.752 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.753 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.753 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.753 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.753 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-03-15T13:09:40.753 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.753 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.753 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.753 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.753 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-03-15T13:09:40.753 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.753 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.753 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.753 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.754 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-03-15T13:09:40.754 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.754 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.754 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.754 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.754 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-03-15T13:09:40.754 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.754 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.754 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.754 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.754 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-03-15T13:09:40.754 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.754 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.754 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.755 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.755 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-03-15T13:09:40.755 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.755 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.755 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.755 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.755 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-03-15T13:09:40.755 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.756 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.756 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.756 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.756 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-03-15T13:09:40.756 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.756 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.756 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.756 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.756 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-03-15T13:09:40.756 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.756 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.756 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.756 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.756 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-03-15T13:09:40.756 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.757 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.758 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.758 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.758 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-03-15T13:09:40.758 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.758 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.758 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.758 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.758 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-03-15T13:09:40.758 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.758 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.758 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.758 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.758 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-03-15T13:09:40.758 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.759 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.760 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.760 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-03-15T13:09:40.760 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.760 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.760 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.760 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.760 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-03-15T13:09:40.760 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.760 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.760 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.760 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.760 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-03-15T13:09:40.760 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.761 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.761 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.761 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.761 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-03-15T13:09:40.761 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.761 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.761 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.761 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.761 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-03-15T13:09:40.761 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-03-15T13:09:40.761 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-03-15T13:09:40.761 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.761 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.761 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.761 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:40.762 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.762 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-03-15T13:09:40.762 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 2, 2024-03-15T13:09:40.762 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 6, 2024-03-15T13:09:40.762 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 4, 2024-03-15T13:09:40.762 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 3, 2024-03-15T13:09:40.762 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-03-15T13:09:40.762 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 0, 2024-03-15T13:09:40.762 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 0, 2024-03-15T13:09:40.762 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 4181, 2024-03-15T13:09:40.762 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-03-15T13:09:40.762 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.762 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-03-15T13:09:40.763 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-03-15T13:09:40.763 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-03-15T13:09:40.763 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-03-15T13:09:40.763 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-03-15T13:09:40.763 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-03-15T13:09:40.763 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-03-15T13:09:40.763 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-15T13:09:40.763 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-03-15T13:09:40.763 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-15T13:09:40.763 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-03-15T13:09:40.763 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-03-15T13:09:40.763 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-03-15T13:09:40.763 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-15T13:09:40.763 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-03-15T13:09:40.763 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-03-15T13:09:40.764 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-03-15T13:09:40.764 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-03-15T13:09:40.764 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-03-15T13:09:40.764 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-15T13:09:40.764 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-03-15T13:09:40.764 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-15T13:09:40.764 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-03-15T13:09:40.764 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-03-15T13:09:40.764 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-03-15T13:09:40.764 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-03-15T13:09:40.764 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-03-15T13:09:40.764 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-15T13:09:40.764 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-03-15T13:09:40.764 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-03-15T13:09:40.765 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-03-15T13:09:40.765 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-03-15T13:09:40.765 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-03-15T13:09:40.765 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-03-15T13:09:40.765 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-03-15T13:09:40.765 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-03-15T13:09:40.765 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-03-15T13:09:40.765 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 231361, 2024-03-15T13:09:40.765 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 75, 2024-03-15T13:09:40.765 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-03-15T13:09:40.765 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-03-15T13:09:40.765 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-03-15T13:09:40.765 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-03-15T13:09:40.765 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-03-15T13:09:40.765 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-03-15T13:09:40.766 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-03-15T13:09:40.766 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-03-15T13:09:40.766 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 9448, 2024-03-15T13:09:40.766 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 197, 2024-03-15T13:09:40.766 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-03-15T13:09:40.766 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-03-15T13:09:40.766 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-03-15T13:09:40.766 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-03-15T13:09:40.766 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 35303215, 2024-03-15T13:09:40.766 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 318756, 2024-03-15T13:09:40.766 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-03-15T13:09:40.766 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-03-15T13:09:40.766 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-03-15T13:09:40.766 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-03-15T13:09:40.766 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.767 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-03-15T13:09:40.767 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-03-15T13:09:40.767 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-03-15T13:09:40.767 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 2633, 2024-03-15T13:09:40.767 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 26, 2024-03-15T13:09:40.767 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 4, 2024-03-15T13:09:40.767 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 2633, 2024-03-15T13:09:40.767 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-03-15T13:09:40.767 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2633, 2024-03-15T13:09:40.767 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 1.210973795, 2024-03-15T13:09:40.767 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000459921 2024-03-15T13:09:40.767 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.767 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-03-15T13:09:40.767 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-03-15T13:09:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2633, 2024-03-15T13:09:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 2634 2024-03-15T13:09:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 2633, 2024-03-15T13:09:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 2633, 2024-03-15T13:09:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 0, 2024-03-15T13:09:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-03-15T13:09:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-03-15T13:09:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 1661, 2024-03-15T13:09:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 0, 2024-03-15T13:09:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 969, 2024-03-15T13:09:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 0, 2024-03-15T13:09:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 0, 2024-03-15T13:09:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-03-15T13:09:40.768 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-03-15T13:09:40.769 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-03-15T13:09:40.769 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-03-15T13:09:40.769 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 0, 2024-03-15T13:09:40.769 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-03-15T13:09:40.769 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-03-15T13:09:40.769 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-03-15T13:09:40.769 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 2, 2024-03-15T13:09:40.769 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 0, 2024-03-15T13:09:40.769 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-03-15T13:09:40.769 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-03-15T13:09:40.769 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-03-15T13:09:40.769 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-03-15T13:09:40.769 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-03-15T13:09:40.770 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-03-15T13:09:40.770 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-03-15T13:09:40.770 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-03-15T13:09:40.770 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-03-15T13:09:40.770 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-03-15T13:09:40.770 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-03-15T13:09:40.770 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-03-15T13:09:40.770 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-03-15T13:09:40.770 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-03-15T13:09:40.770 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-03-15T13:09:40.770 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-03-15T13:09:40.770 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-03-15T13:09:40.770 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-03-15T13:09:40.770 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-03-15T13:09:40.771 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-03-15T13:09:40.771 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-03-15T13:09:40.771 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-03-15T13:09:40.771 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-03-15T13:09:40.771 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-03-15T13:09:40.771 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-03-15T13:09:40.771 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-03-15T13:09:40.771 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 100, 2024-03-15T13:09:40.771 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-03-15T13:09:40.771 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 11, 2024-03-15T13:09:40.771 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 10, 2024-03-15T13:09:40.771 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 10, 2024-03-15T13:09:40.771 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 0, 2024-03-15T13:09:40.771 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-03-15T13:09:40.771 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 0, 2024-03-15T13:09:40.772 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 2, 2024-03-15T13:09:40.772 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 0 2024-03-15T13:09:40.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.772 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-03-15T13:09:40.772 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 0, 2024-03-15T13:09:40.772 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-03-15T13:09:40.772 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-03-15T13:09:40.772 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-03-15T13:09:40.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.772 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2024-03-15T13:09:40.772 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2024-03-15T13:09:40.772 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 2911, 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 120121628, 2024-03-15T13:09:40.773 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 2911, 2024-03-15T13:09:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-03-15T13:09:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-03-15T13:09:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2911, 2024-03-15T13:09:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 120121628, 2024-03-15T13:09:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-03-15T13:09:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-03-15T13:09:40.774 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-03-15T13:09:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-03-15T13:09:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 2633, 2024-03-15T13:09:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 117902007, 2024-03-15T13:09:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2633, 2024-03-15T13:09:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 117902007, 2024-03-15T13:09:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.775 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-03-15T13:09:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-03-15T13:09:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-03-15T13:09:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-03-15T13:09:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 2633, 2024-03-15T13:09:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 2633, 2024-03-15T13:09:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2633, 2024-03-15T13:09:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 2633, 2024-03-15T13:09:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.776 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-03-15T13:09:40.777 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x55d52f488960": { 2024-03-15T13:09:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-03-15T13:09:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-03-15T13:09:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-03-15T13:09:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:40.778 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-03-15T13:09:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-03-15T13:09:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-03-15T13:09:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-03-15T13:09:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:40.779 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:40.779 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-03-15T13:09:41.644 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-15T13:09:41.644 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-15T13:09:41.644 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-03-15T13:09:41.644 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-15T13:09:41.644 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-15T13:09:41.645 DEBUG:tasks.check_counter:Getting stats from l 2024-03-15T13:09:41.645 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b shell --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-15T13:09:41.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:41 smithi149 ceph-mon[35927]: pgmap v334: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:09:41.904 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/mon.c/config 2024-03-15T13:09:42.030 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:41 smithi184 ceph-mon[35993]: pgmap v334: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:09:42.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:41 smithi002 ceph-mon[32060]: pgmap v334: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:09:43.681 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:43 smithi184 ceph-mon[35993]: pgmap v335: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:43.705 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-03-15T13:09:43.705 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-03-15T13:09:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 276, 2024-03-15T13:09:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 243, 2024-03-15T13:09:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 1735234, 2024-03-15T13:09:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 138637, 2024-03-15T13:09:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 5, 2024-03-15T13:09:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 1, 2024-03-15T13:09:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.038138652, 2024-03-15T13:09:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.009093733, 2024-03-15T13:09:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.036255917, 2024-03-15T13:09:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.003917143, 2024-03-15T13:09:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 242, 2024-03-15T13:09:43.706 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.007826505, 2024-03-15T13:09:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000032340 2024-03-15T13:09:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 1734400, 2024-03-15T13:09:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 138320 2024-03-15T13:09:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-03-15T13:09:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 722, 2024-03-15T13:09:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 890, 2024-03-15T13:09:43.707 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 179470, 2024-03-15T13:09:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 672777, 2024-03-15T13:09:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 7, 2024-03-15T13:09:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 5, 2024-03-15T13:09:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.054114424, 2024-03-15T13:09:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.017051868, 2024-03-15T13:09:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.012514534, 2024-03-15T13:09:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.019081892, 2024-03-15T13:09:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 888, 2024-03-15T13:09:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.027342500, 2024-03-15T13:09:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000030791 2024-03-15T13:09:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:43.708 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-15T13:09:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 450096 2024-03-15T13:09:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-03-15T13:09:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 1611, 2024-03-15T13:09:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1613, 2024-03-15T13:09:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 1298892, 2024-03-15T13:09:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 501079, 2024-03-15T13:09:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 4, 2024-03-15T13:09:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 3, 2024-03-15T13:09:43.709 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.093385811, 2024-03-15T13:09:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.027752899, 2024-03-15T13:09:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.025449096, 2024-03-15T13:09:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.032233843, 2024-03-15T13:09:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-03-15T13:09:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1611, 2024-03-15T13:09:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.037167041, 2024-03-15T13:09:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000023070 2024-03-15T13:09:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-03-15T13:09:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.710 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-15T13:09:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 0 2024-03-15T13:09:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-03-15T13:09:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-03-15T13:09:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-03-15T13:09:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-03-15T13:09:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-03-15T13:09:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-03-15T13:09:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3808, 2024-03-15T13:09:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.069596153, 2024-03-15T13:09:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000018276 2024-03-15T13:09:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.711 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-03-15T13:09:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-03-15T13:09:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-03-15T13:09:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-03-15T13:09:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 0, 2024-03-15T13:09:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 0, 2024-03-15T13:09:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-03-15T13:09:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.712 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-03-15T13:09:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 0, 2024-03-15T13:09:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-03-15T13:09:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-03-15T13:09:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 0, 2024-03-15T13:09:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-03-15T13:09:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-03-15T13:09:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 86, 2024-03-15T13:09:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 0, 2024-03-15T13:09:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 86, 2024-03-15T13:09:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-03-15T13:09:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 86, 2024-03-15T13:09:43.713 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 0, 2024-03-15T13:09:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2024-03-15T13:09:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2024-03-15T13:09:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 1, 2024-03-15T13:09:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 0, 2024-03-15T13:09:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 0, 2024-03-15T13:09:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2024-03-15T13:09:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 0, 2024-03-15T13:09:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-03-15T13:09:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-03-15T13:09:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-03-15T13:09:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2024-03-15T13:09:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-03-15T13:09:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-03-15T13:09:43.714 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-03-15T13:09:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-03-15T13:09:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-03-15T13:09:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-03-15T13:09:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-03-15T13:09:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-03-15T13:09:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 0, 2024-03-15T13:09:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 0, 2024-03-15T13:09:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2024-03-15T13:09:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-03-15T13:09:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-03-15T13:09:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-03-15T13:09:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-03-15T13:09:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-03-15T13:09:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-03-15T13:09:43.715 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-03-15T13:09:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-03-15T13:09:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-03-15T13:09:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2024-03-15T13:09:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-03-15T13:09:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-03-15T13:09:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-15T13:09:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-15T13:09:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2024-03-15T13:09:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-03-15T13:09:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-03-15T13:09:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2024-03-15T13:09:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-03-15T13:09:43.716 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-03-15T13:09:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 0, 2024-03-15T13:09:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 0, 2024-03-15T13:09:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 0, 2024-03-15T13:09:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-03-15T13:09:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-03-15T13:09:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-03-15T13:09:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-03-15T13:09:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-03-15T13:09:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-03-15T13:09:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-03-15T13:09:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-03-15T13:09:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-03-15T13:09:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-03-15T13:09:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-03-15T13:09:43.717 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-03-15T13:09:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-03-15T13:09:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-03-15T13:09:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-03-15T13:09:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-03-15T13:09:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-03-15T13:09:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-03-15T13:09:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2024-03-15T13:09:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-03-15T13:09:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-03-15T13:09:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 0, 2024-03-15T13:09:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-03-15T13:09:43.718 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-03-15T13:09:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 447, 2024-03-15T13:09:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-03-15T13:09:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-03-15T13:09:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 0, 2024-03-15T13:09:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-03-15T13:09:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-03-15T13:09:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 2, 2024-03-15T13:09:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2024-03-15T13:09:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-03-15T13:09:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-03-15T13:09:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-03-15T13:09:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 5014111, 2024-03-15T13:09:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 5013512, 2024-03-15T13:09:43.719 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-03-15T13:09:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 447 2024-03-15T13:09:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-03-15T13:09:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 8, 2024-03-15T13:09:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 176, 2024-03-15T13:09:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 168, 2024-03-15T13:09:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 3, 2024-03-15T13:09:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 70, 2024-03-15T13:09:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 67, 2024-03-15T13:09:43.720 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 86, 2024-03-15T13:09:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 216, 2024-03-15T13:09:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 130, 2024-03-15T13:09:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2024-03-15T13:09:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 0, 2024-03-15T13:09:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 0, 2024-03-15T13:09:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 51220, 2024-03-15T13:09:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 215420 2024-03-15T13:09:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-03-15T13:09:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 0, 2024-03-15T13:09:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-03-15T13:09:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 0, 2024-03-15T13:09:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 0, 2024-03-15T13:09:43.721 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 0, 2024-03-15T13:09:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-03-15T13:09:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-03-15T13:09:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-03-15T13:09:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.722 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-03-15T13:09:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-03-15T13:09:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-03-15T13:09:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.723 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.724 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-03-15T13:09:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-03-15T13:09:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-03-15T13:09:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.725 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-03-15T13:09:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-03-15T13:09:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.727 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.728 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-03-15T13:09:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-03-15T13:09:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-03-15T13:09:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.729 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-03-15T13:09:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-03-15T13:09:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-03-15T13:09:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.730 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-03-15T13:09:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-03-15T13:09:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-03-15T13:09:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-03-15T13:09:43.731 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-03-15T13:09:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-03-15T13:09:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 1, 2024-03-15T13:09:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 5, 2024-03-15T13:09:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 4, 2024-03-15T13:09:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 3, 2024-03-15T13:09:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-03-15T13:09:43.732 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 0, 2024-03-15T13:09:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 0, 2024-03-15T13:09:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 7718, 2024-03-15T13:09:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-03-15T13:09:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-03-15T13:09:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-03-15T13:09:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-03-15T13:09:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-03-15T13:09:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-03-15T13:09:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-03-15T13:09:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-03-15T13:09:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-15T13:09:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-03-15T13:09:43.733 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-15T13:09:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-03-15T13:09:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-03-15T13:09:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-03-15T13:09:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-15T13:09:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-03-15T13:09:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-03-15T13:09:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-03-15T13:09:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-03-15T13:09:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-03-15T13:09:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-15T13:09:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-03-15T13:09:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-15T13:09:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-03-15T13:09:43.734 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-03-15T13:09:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-03-15T13:09:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-03-15T13:09:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-03-15T13:09:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-15T13:09:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-03-15T13:09:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-03-15T13:09:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-03-15T13:09:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-03-15T13:09:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-03-15T13:09:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-03-15T13:09:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-03-15T13:09:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-03-15T13:09:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-03-15T13:09:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 190161, 2024-03-15T13:09:43.735 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 52, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 9448, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 197, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 87531, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 544, 2024-03-15T13:09:43.736 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-03-15T13:09:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-03-15T13:09:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-03-15T13:09:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-03-15T13:09:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-03-15T13:09:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-03-15T13:09:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-03-15T13:09:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 2328, 2024-03-15T13:09:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 21, 2024-03-15T13:09:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 3, 2024-03-15T13:09:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 2328, 2024-03-15T13:09:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-03-15T13:09:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2328, 2024-03-15T13:09:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 1.129975630, 2024-03-15T13:09:43.737 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000485384 2024-03-15T13:09:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-03-15T13:09:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-03-15T13:09:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2328, 2024-03-15T13:09:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 2329 2024-03-15T13:09:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 2328, 2024-03-15T13:09:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 2328, 2024-03-15T13:09:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 0, 2024-03-15T13:09:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-03-15T13:09:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-03-15T13:09:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 1494, 2024-03-15T13:09:43.738 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 0, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 832, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 0, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 0, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 0, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 1, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 0, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-03-15T13:09:43.739 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-03-15T13:09:43.740 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 100, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 11, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 5, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 5, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 0, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 0, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 2, 2024-03-15T13:09:43.741 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 0 2024-03-15T13:09:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-03-15T13:09:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 0, 2024-03-15T13:09:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-03-15T13:09:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-03-15T13:09:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-03-15T13:09:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2024-03-15T13:09:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2024-03-15T13:09:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-03-15T13:09:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-03-15T13:09:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-03-15T13:09:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-03-15T13:09:43.742 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 2605, 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 2962176, 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 2605, 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2605, 2024-03-15T13:09:43.743 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 2962176, 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 2328, 2024-03-15T13:09:43.744 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 819812, 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2328, 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 819812, 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:43.745 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 2328, 2024-03-15T13:09:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 2328, 2024-03-15T13:09:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2328, 2024-03-15T13:09:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 2328, 2024-03-15T13:09:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-03-15T13:09:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-03-15T13:09:43.746 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-03-15T13:09:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-03-15T13:09:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-03-15T13:09:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-03-15T13:09:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-03-15T13:09:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-03-15T13:09:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-03-15T13:09:43.747 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x55936f2fe960": { 2024-03-15T13:09:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-03-15T13:09:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-03-15T13:09:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-03-15T13:09:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-03-15T13:09:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-03-15T13:09:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-03-15T13:09:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-03-15T13:09:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-03-15T13:09:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-03-15T13:09:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-03-15T13:09:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-03-15T13:09:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-03-15T13:09:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-03-15T13:09:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-03-15T13:09:43.748 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-03-15T13:09:43.749 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:43.749 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-03-15T13:09:43.749 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-03-15T13:09:43.853 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:43 smithi002 ceph-mon[32060]: pgmap v335: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:43.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:43 smithi149 ceph-mon[35927]: pgmap v335: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:44.522 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-15T13:09:44.522 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-15T13:09:44.522 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-03-15T13:09:44.522 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-03-15T13:09:44.522 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-03-15T13:09:44.523 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-15T13:09:44.535 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-15T13:09:44.535 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T13:09:44.535 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-15T13:09:44.552 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T13:09:44.552 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-15T13:09:44.608 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T13:09:44.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.770+0000 7f5e3876c640 1 -- 172.21.15.2:0/3023778936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e30100db0 msgr2=0x7f5e3010d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:44.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.770+0000 7f5e3876c640 1 --2- 172.21.15.2:0/3023778936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e30100db0 0x7f5e3010d2c0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009cd0 tx=0x7f5e2802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:44.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e3876c640 1 -- 172.21.15.2:0/3023778936 shutdown_connections 2024-03-15T13:09:44.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e3876c640 1 --2- 172.21.15.2:0/3023778936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e30100db0 0x7f5e3010d2c0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:44.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e3876c640 1 --2- 172.21.15.2:0/3023778936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e301003f0 0x7f5e30100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:44.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e3876c640 1 --2- 172.21.15.2:0/3023778936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e30106490 0x7f5e30106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:44.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e3876c640 1 -- 172.21.15.2:0/3023778936 >> 172.21.15.2:0/3023778936 conn(0x7f5e300774f0 msgr2=0x7f5e30077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:44.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e3876c640 1 -- 172.21.15.2:0/3023778936 shutdown_connections 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e3876c640 1 -- 172.21.15.2:0/3023778936 wait complete. 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e3876c640 1 Processor -- start 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e3876c640 1 -- start start 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e3876c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e301003f0 0x7f5e3019d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e3876c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e30100db0 0x7f5e3019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e3876c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e30106490 0x7f5e30197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e3876c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e30197f90 con 0x7f5e30100db0 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e3876c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e30198100 con 0x7f5e30106490 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e3876c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e301982c0 con 0x7f5e301003f0 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e35d07640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e30100db0 0x7f5e3019dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e35d07640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e30100db0 0x7f5e3019dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44080/0 (socket says 172.21.15.2:44080) 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.771+0000 7f5e35d07640 1 -- 172.21.15.2:0/1967135374 learned_addr learned my addr 172.21.15.2:0/1967135374 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.772+0000 7f5e36d09640 1 --2- 172.21.15.2:0/1967135374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e30106490 0x7f5e30197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.772+0000 7f5e36508640 1 --2- 172.21.15.2:0/1967135374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e301003f0 0x7f5e3019d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.772+0000 7f5e35d07640 1 -- 172.21.15.2:0/1967135374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e301003f0 msgr2=0x7f5e3019d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:44.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.772+0000 7f5e35d07640 1 --2- 172.21.15.2:0/1967135374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e301003f0 0x7f5e3019d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.772+0000 7f5e35d07640 1 -- 172.21.15.2:0/1967135374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e30106490 msgr2=0x7f5e30197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.772+0000 7f5e35d07640 1 --2- 172.21.15.2:0/1967135374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e30106490 0x7f5e30197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.772+0000 7f5e35d07640 1 -- 172.21.15.2:0/1967135374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e28009900 con 0x7f5e30100db0 2024-03-15T13:09:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.772+0000 7f5e35d07640 1 --2- 172.21.15.2:0/1967135374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e30100db0 0x7f5e3019dcd0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000e9f0 tx=0x7f5e2000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.772+0000 7f5e277fe640 1 -- 172.21.15.2:0/1967135374 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20009720 con 0x7f5e30100db0 2024-03-15T13:09:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.772+0000 7f5e3876c640 1 -- 172.21.15.2:0/1967135374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e301985a0 con 0x7f5e30100db0 2024-03-15T13:09:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.772+0000 7f5e277fe640 1 -- 172.21.15.2:0/1967135374 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5e2000cd80 con 0x7f5e30100db0 2024-03-15T13:09:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.772+0000 7f5e277fe640 1 -- 172.21.15.2:0/1967135374 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20010690 con 0x7f5e30100db0 2024-03-15T13:09:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.772+0000 7f5e3876c640 1 -- 172.21.15.2:0/1967135374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e301a4830 con 0x7f5e30100db0 2024-03-15T13:09:44.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.773+0000 7f5e3876c640 1 -- 172.21.15.2:0/1967135374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e300682e0 con 0x7f5e30100db0 2024-03-15T13:09:44.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.775+0000 7f5e277fe640 1 -- 172.21.15.2:0/1967135374 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f5e200107f0 con 0x7f5e30100db0 2024-03-15T13:09:44.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.775+0000 7f5e277fe640 1 --2- 172.21.15.2:0/1967135374 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5e1c0a1ba0 0x7f5e1c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:44.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.775+0000 7f5e36508640 1 --2- 172.21.15.2:0/1967135374 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5e1c0a1ba0 0x7f5e1c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:44.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.775+0000 7f5e277fe640 1 -- 172.21.15.2:0/1967135374 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f5e20014070 con 0x7f5e30100db0 2024-03-15T13:09:44.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.776+0000 7f5e36508640 1 --2- 172.21.15.2:0/1967135374 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5e1c0a1ba0 0x7f5e1c0a4060 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5e30198a80 tx=0x7f5e2c005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:44.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.779+0000 7f5e277fe640 1 -- 172.21.15.2:0/1967135374 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5e20096c40 con 0x7f5e30100db0 2024-03-15T13:09:44.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.965+0000 7f5e3876c640 1 -- 172.21.15.2:0/1967135374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e30198e70 con 0x7f5e30100db0 2024-03-15T13:09:44.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.967+0000 7f5e277fe640 1 -- 172.21.15.2:0/1967135374 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+25138 (secure 0 0 0) 0x7f5e2009a300 con 0x7f5e30100db0 2024-03-15T13:09:44.951 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:09:44.952 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":100,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T13:09:15.557407+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/343748862":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/1602727020":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/883225361":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/3173750072":"2024-03-16T12:58:53.781001+0000","172.21.15.2:6800/343748862":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/2211760175":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/4030567035":"2024-03-16T12:58:53.781001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T13:09:44.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.969+0000 7f5e3876c640 1 -- 172.21.15.2:0/1967135374 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5e1c0a1ba0 msgr2=0x7f5e1c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:44.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.969+0000 7f5e3876c640 1 --2- 172.21.15.2:0/1967135374 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5e1c0a1ba0 0x7f5e1c0a4060 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5e30198a80 tx=0x7f5e2c005f90 comp rx=0 tx=0).stop 2024-03-15T13:09:44.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.969+0000 7f5e3876c640 1 -- 172.21.15.2:0/1967135374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e30100db0 msgr2=0x7f5e3019dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:44.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.969+0000 7f5e3876c640 1 --2- 172.21.15.2:0/1967135374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e30100db0 0x7f5e3019dcd0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000e9f0 tx=0x7f5e2000edf0 comp rx=0 tx=0).stop 2024-03-15T13:09:44.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.969+0000 7f5e3876c640 1 -- 172.21.15.2:0/1967135374 shutdown_connections 2024-03-15T13:09:44.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.969+0000 7f5e3876c640 1 --2- 172.21.15.2:0/1967135374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e30106490 0x7f5e30197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:44.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.969+0000 7f5e3876c640 1 --2- 172.21.15.2:0/1967135374 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f5e1c0a1ba0 0x7f5e1c0a4060 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:44.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.969+0000 7f5e3876c640 1 --2- 172.21.15.2:0/1967135374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e30100db0 0x7f5e3019dcd0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:44.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.969+0000 7f5e3876c640 1 --2- 172.21.15.2:0/1967135374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e301003f0 0x7f5e3019d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:44.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.969+0000 7f5e3876c640 1 -- 172.21.15.2:0/1967135374 >> 172.21.15.2:0/1967135374 conn(0x7f5e300774f0 msgr2=0x7f5e30068930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:44.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.969+0000 7f5e3876c640 1 -- 172.21.15.2:0/1967135374 shutdown_connections 2024-03-15T13:09:44.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:44.969+0000 7f5e3876c640 1 -- 172.21.15.2:0/1967135374 wait complete. 2024-03-15T13:09:44.964 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-15T12:51:47.004843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-15T12:56:18.559882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-15T12:56:19.751767+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-15T12:56:23.664879+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-15T13:09:44.965 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-15T13:09:45.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.087+0000 7f9842380640 1 -- 172.21.15.2:0/72924980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f983c1074f0 msgr2=0x7f983c1078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:45.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.087+0000 7f9842380640 1 --2- 172.21.15.2:0/72924980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f983c1074f0 0x7f983c1078d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f982c009930 tx=0x7f982c02bc30 comp rx=0 tx=0).stop 2024-03-15T13:09:45.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.087+0000 7f9842380640 1 -- 172.21.15.2:0/72924980 shutdown_connections 2024-03-15T13:09:45.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.087+0000 7f9842380640 1 --2- 172.21.15.2:0/72924980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f983c0690b0 0x7f983c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.087+0000 7f9842380640 1 --2- 172.21.15.2:0/72924980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f983c0686f0 0x7f983c068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.087+0000 7f9842380640 1 --2- 172.21.15.2:0/72924980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f983c1074f0 0x7f983c1078d0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.087+0000 7f9842380640 1 -- 172.21.15.2:0/72924980 >> 172.21.15.2:0/72924980 conn(0x7f983c0774f0 msgr2=0x7f983c077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:45.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.087+0000 7f9842380640 1 -- 172.21.15.2:0/72924980 shutdown_connections 2024-03-15T13:09:45.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.087+0000 7f9842380640 1 -- 172.21.15.2:0/72924980 wait complete. 2024-03-15T13:09:45.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.087+0000 7f9842380640 1 Processor -- start 2024-03-15T13:09:45.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.087+0000 7f9842380640 1 -- start start 2024-03-15T13:09:45.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f9842380640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f983c0686f0 0x7f983c19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:45.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f9842380640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f983c0690b0 0x7f983c19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:45.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f9842380640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f983c1074f0 0x7f983c1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:45.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f9842380640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c197f40 con 0x7f983c0690b0 2024-03-15T13:09:45.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f9842380640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c1980b0 con 0x7f983c0686f0 2024-03-15T13:09:45.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f9842380640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c198270 con 0x7f983c1074f0 2024-03-15T13:09:45.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f983b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f983c0690b0 0x7f983c19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:45.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f983b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f983c0690b0 0x7f983c19dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44108/0 (socket says 172.21.15.2:44108) 2024-03-15T13:09:45.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f983b7fe640 1 -- 172.21.15.2:0/2547124273 learned_addr learned my addr 172.21.15.2:0/2547124273 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:45.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f984091d640 1 --2- 172.21.15.2:0/2547124273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f983c1074f0 0x7f983c1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:45.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f983bfff640 1 --2- 172.21.15.2:0/2547124273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f983c0686f0 0x7f983c19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:45.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f983b7fe640 1 -- 172.21.15.2:0/2547124273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f983c1074f0 msgr2=0x7f983c1978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:45.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f983b7fe640 1 --2- 172.21.15.2:0/2547124273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f983c1074f0 0x7f983c1978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f983b7fe640 1 -- 172.21.15.2:0/2547124273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f983c0686f0 msgr2=0x7f983c19d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:45.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f983b7fe640 1 --2- 172.21.15.2:0/2547124273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f983c0686f0 0x7f983c19d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f983b7fe640 1 -- 172.21.15.2:0/2547124273 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c009560 con 0x7f983c0690b0 2024-03-15T13:09:45.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f983b7fe640 1 --2- 172.21.15.2:0/2547124273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f983c0690b0 0x7f983c19dc80 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f982400e9f0 tx=0x7f982400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:45.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f98397fa640 1 -- 172.21.15.2:0/2547124273 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824009700 con 0x7f983c0690b0 2024-03-15T13:09:45.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.088+0000 7f98397fa640 1 -- 172.21.15.2:0/2547124273 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f982400cd80 con 0x7f983c0690b0 2024-03-15T13:09:45.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.089+0000 7f9842380640 1 -- 172.21.15.2:0/2547124273 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983c198550 con 0x7f983c0690b0 2024-03-15T13:09:45.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.089+0000 7f98397fa640 1 -- 172.21.15.2:0/2547124273 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824018920 con 0x7f983c0690b0 2024-03-15T13:09:45.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.089+0000 7f9842380640 1 -- 172.21.15.2:0/2547124273 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983c1a47e0 con 0x7f983c0690b0 2024-03-15T13:09:45.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.090+0000 7f9842380640 1 -- 172.21.15.2:0/2547124273 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983c0682e0 con 0x7f983c0690b0 2024-03-15T13:09:45.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.091+0000 7f98397fa640 1 -- 172.21.15.2:0/2547124273 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f9824018a80 con 0x7f983c0690b0 2024-03-15T13:09:45.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.091+0000 7f98397fa640 1 --2- 172.21.15.2:0/2547124273 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f98280a1ba0 0x7f98280a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:45.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.091+0000 7f98397fa640 1 -- 172.21.15.2:0/2547124273 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f9824014070 con 0x7f983c0690b0 2024-03-15T13:09:45.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.091+0000 7f983bfff640 1 --2- 172.21.15.2:0/2547124273 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f98280a1ba0 0x7f98280a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:45.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.092+0000 7f983bfff640 1 --2- 172.21.15.2:0/2547124273 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f98280a1ba0 0x7f98280a4060 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f982c000c00 tx=0x7f982c005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:45.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.096+0000 7f98397fa640 1 -- 172.21.15.2:0/2547124273 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9824053280 con 0x7f983c0690b0 2024-03-15T13:09:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.283+0000 7f9842380640 1 -- 172.21.15.2:0/2547124273 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f983c109970 con 0x7f983c0690b0 2024-03-15T13:09:45.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.284+0000 7f98397fa640 1 -- 172.21.15.2:0/2547124273 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9824096a60 con 0x7f983c0690b0 2024-03-15T13:09:45.268 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-15T13:09:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.286+0000 7f9842380640 1 -- 172.21.15.2:0/2547124273 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f98280a1ba0 msgr2=0x7f98280a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.286+0000 7f9842380640 1 --2- 172.21.15.2:0/2547124273 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f98280a1ba0 0x7f98280a4060 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f982c000c00 tx=0x7f982c005fd0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.286+0000 7f9842380640 1 -- 172.21.15.2:0/2547124273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f983c0690b0 msgr2=0x7f983c19dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:45.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.286+0000 7f9842380640 1 --2- 172.21.15.2:0/2547124273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f983c0690b0 0x7f983c19dc80 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f982400e9f0 tx=0x7f982400eeb0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.286+0000 7f9842380640 1 -- 172.21.15.2:0/2547124273 shutdown_connections 2024-03-15T13:09:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.286+0000 7f9842380640 1 --2- 172.21.15.2:0/2547124273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f983c1074f0 0x7f983c1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.286+0000 7f9842380640 1 --2- 172.21.15.2:0/2547124273 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f98280a1ba0 0x7f98280a4060 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.286+0000 7f9842380640 1 --2- 172.21.15.2:0/2547124273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f983c0690b0 0x7f983c19dc80 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.286+0000 7f9842380640 1 --2- 172.21.15.2:0/2547124273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f983c0686f0 0x7f983c19d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.286+0000 7f9842380640 1 -- 172.21.15.2:0/2547124273 >> 172.21.15.2:0/2547124273 conn(0x7f983c0774f0 msgr2=0x7f983c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.286+0000 7f9842380640 1 -- 172.21.15.2:0/2547124273 shutdown_connections 2024-03-15T13:09:45.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.286+0000 7f9842380640 1 -- 172.21.15.2:0/2547124273 wait complete. 2024-03-15T13:09:45.281 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-15T13:09:45.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.403+0000 7f8ae0465640 1 -- 172.21.15.2:0/651260828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ad80741e0 msgr2=0x7f8ad810d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:45.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.403+0000 7f8ae0465640 1 --2- 172.21.15.2:0/651260828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ad80741e0 0x7f8ad810d7e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009900 tx=0x7f8ac802bcd0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.403+0000 7f8ae0465640 1 -- 172.21.15.2:0/651260828 shutdown_connections 2024-03-15T13:09:45.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.403+0000 7f8ae0465640 1 --2- 172.21.15.2:0/651260828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ad810dd20 0x7f8ad8110130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.403+0000 7f8ae0465640 1 --2- 172.21.15.2:0/651260828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ad80741e0 0x7f8ad810d7e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.403+0000 7f8ae0465640 1 --2- 172.21.15.2:0/651260828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ad80738c0 0x7f8ad8073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.403+0000 7f8ae0465640 1 -- 172.21.15.2:0/651260828 >> 172.21.15.2:0/651260828 conn(0x7f8ad80788f0 msgr2=0x7f8ad8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:45.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.403+0000 7f8ae0465640 1 -- 172.21.15.2:0/651260828 shutdown_connections 2024-03-15T13:09:45.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.403+0000 7f8ae0465640 1 -- 172.21.15.2:0/651260828 wait complete. 2024-03-15T13:09:45.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.403+0000 7f8ae0465640 1 Processor -- start 2024-03-15T13:09:45.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.403+0000 7f8ae0465640 1 -- start start 2024-03-15T13:09:45.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8ae0465640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ad80738c0 0x7f8ad8199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:45.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8ae0465640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ad80741e0 0x7f8ad819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:45.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8ae0465640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ad810dd20 0x7f8ad819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:45.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8ae0465640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad819ec30 con 0x7f8ad810dd20 2024-03-15T13:09:45.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8ae0465640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad819eda0 con 0x7f8ad80738c0 2024-03-15T13:09:45.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8ae0465640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad819ef60 con 0x7f8ad80741e0 2024-03-15T13:09:45.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8adea02640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ad810dd20 0x7f8ad819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:45.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8ade201640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ad80738c0 0x7f8ad8199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:45.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8adea02640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ad810dd20 0x7f8ad819e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44124/0 (socket says 172.21.15.2:44124) 2024-03-15T13:09:45.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8adea02640 1 -- 172.21.15.2:0/2567251440 learned_addr learned my addr 172.21.15.2:0/2567251440 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:45.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8ade201640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ad80738c0 0x7f8ad8199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.2:52638/0 (socket says 172.21.15.2:52638) 2024-03-15T13:09:45.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8adda00640 1 --2- 172.21.15.2:0/2567251440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ad80741e0 0x7f8ad819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:45.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8adea02640 1 -- 172.21.15.2:0/2567251440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ad80741e0 msgr2=0x7f8ad819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:45.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8adea02640 1 --2- 172.21.15.2:0/2567251440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ad80741e0 0x7f8ad819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8adea02640 1 -- 172.21.15.2:0/2567251440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ad80738c0 msgr2=0x7f8ad8199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:45.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8adea02640 1 --2- 172.21.15.2:0/2567251440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ad80738c0 0x7f8ad8199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8adea02640 1 -- 172.21.15.2:0/2567251440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac8009560 con 0x7f8ad810dd20 2024-03-15T13:09:45.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8adea02640 1 --2- 172.21.15.2:0/2567251440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ad810dd20 0x7f8ad819e630 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000eb10 tx=0x7f8ad000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:45.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8acf7fe640 1 -- 172.21.15.2:0/2567251440 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad000ce80 con 0x7f8ad810dd20 2024-03-15T13:09:45.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8ae0465640 1 -- 172.21.15.2:0/2567251440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad819f240 con 0x7f8ad810dd20 2024-03-15T13:09:45.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8acf7fe640 1 -- 172.21.15.2:0/2567251440 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8ad0005020 con 0x7f8ad810dd20 2024-03-15T13:09:45.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.404+0000 7f8acf7fe640 1 -- 172.21.15.2:0/2567251440 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0010720 con 0x7f8ad810dd20 2024-03-15T13:09:45.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.405+0000 7f8ae0465640 1 -- 172.21.15.2:0/2567251440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad81a6ac0 con 0x7f8ad810dd20 2024-03-15T13:09:45.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.405+0000 7f8ae0465640 1 -- 172.21.15.2:0/2567251440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad8068730 con 0x7f8ad810dd20 2024-03-15T13:09:45.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.407+0000 7f8acf7fe640 1 -- 172.21.15.2:0/2567251440 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f8ad0005190 con 0x7f8ad810dd20 2024-03-15T13:09:45.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.407+0000 7f8acf7fe640 1 --2- 172.21.15.2:0/2567251440 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8ac40a1ba0 0x7f8ac40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:45.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.407+0000 7f8acf7fe640 1 -- 172.21.15.2:0/2567251440 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f8ad0014070 con 0x7f8ad810dd20 2024-03-15T13:09:45.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.407+0000 7f8ade201640 1 --2- 172.21.15.2:0/2567251440 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8ac40a1ba0 0x7f8ac40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:45.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.408+0000 7f8ade201640 1 --2- 172.21.15.2:0/2567251440 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8ac40a1ba0 0x7f8ac40a4060 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009bd0 tx=0x7f8ad4005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:45.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.411+0000 7f8acf7fe640 1 -- 172.21.15.2:0/2567251440 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8ad0096d90 con 0x7f8ad810dd20 2024-03-15T13:09:45.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.593+0000 7f8ae0465640 1 -- 172.21.15.2:0/2567251440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8ad819b0c0 con 0x7f8ad810dd20 2024-03-15T13:09:45.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.594+0000 7f8acf7fe640 1 -- 172.21.15.2:0/2567251440 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8ad009a450 con 0x7f8ad810dd20 2024-03-15T13:09:45.578 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T13:09:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.596+0000 7f8ae0465640 1 -- 172.21.15.2:0/2567251440 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8ac40a1ba0 msgr2=0x7f8ac40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.596+0000 7f8ae0465640 1 --2- 172.21.15.2:0/2567251440 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8ac40a1ba0 0x7f8ac40a4060 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009bd0 tx=0x7f8ad4005f90 comp rx=0 tx=0).stop 2024-03-15T13:09:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.596+0000 7f8ae0465640 1 -- 172.21.15.2:0/2567251440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ad810dd20 msgr2=0x7f8ad819e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.596+0000 7f8ae0465640 1 --2- 172.21.15.2:0/2567251440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ad810dd20 0x7f8ad819e630 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000eb10 tx=0x7f8ad000c2d0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.596+0000 7f8ae0465640 1 -- 172.21.15.2:0/2567251440 shutdown_connections 2024-03-15T13:09:45.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.596+0000 7f8ae0465640 1 --2- 172.21.15.2:0/2567251440 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f8ac40a1ba0 0x7f8ac40a4060 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.596+0000 7f8ae0465640 1 --2- 172.21.15.2:0/2567251440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ad810dd20 0x7f8ad819e630 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.596+0000 7f8ae0465640 1 --2- 172.21.15.2:0/2567251440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ad80741e0 0x7f8ad819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.596+0000 7f8ae0465640 1 --2- 172.21.15.2:0/2567251440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ad80738c0 0x7f8ad8199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.596+0000 7f8ae0465640 1 -- 172.21.15.2:0/2567251440 >> 172.21.15.2:0/2567251440 conn(0x7f8ad80788f0 msgr2=0x7f8ad80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:45.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.596+0000 7f8ae0465640 1 -- 172.21.15.2:0/2567251440 shutdown_connections 2024-03-15T13:09:45.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.596+0000 7f8ae0465640 1 -- 172.21.15.2:0/2567251440 wait complete. 2024-03-15T13:09:45.591 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-15T13:09:45.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.712+0000 7fa9ef222640 1 -- 172.21.15.2:0/2944583056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8103080 msgr2=0x7fa9e810b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:45.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.712+0000 7fa9ef222640 1 --2- 172.21.15.2:0/2944583056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8103080 0x7fa9e810b560 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009900 tx=0x7fa9d402bcd0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.712+0000 7fa9ef222640 1 -- 172.21.15.2:0/2944583056 shutdown_connections 2024-03-15T13:09:45.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.712+0000 7fa9ef222640 1 --2- 172.21.15.2:0/2944583056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e810bb30 0x7fa9e810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.712+0000 7fa9ef222640 1 --2- 172.21.15.2:0/2944583056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8103080 0x7fa9e810b560 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.712+0000 7fa9ef222640 1 --2- 172.21.15.2:0/2944583056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e8102760 0x7fa9e8102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.712+0000 7fa9ef222640 1 -- 172.21.15.2:0/2944583056 >> 172.21.15.2:0/2944583056 conn(0x7fa9e8077550 msgr2=0x7fa9e8077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:45.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.712+0000 7fa9ef222640 1 -- 172.21.15.2:0/2944583056 shutdown_connections 2024-03-15T13:09:45.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.712+0000 7fa9ef222640 1 -- 172.21.15.2:0/2944583056 wait complete. 2024-03-15T13:09:45.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.712+0000 7fa9ef222640 1 Processor -- start 2024-03-15T13:09:45.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.712+0000 7fa9ef222640 1 -- start start 2024-03-15T13:09:45.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9ef222640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e8102760 0x7fa9e819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9ef222640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8103080 0x7fa9e819dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9ef222640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e810bb30 0x7fa9e81979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9ef222640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e8197ff0 con 0x7fa9e8103080 2024-03-15T13:09:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9ef222640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e8198160 con 0x7fa9e8102760 2024-03-15T13:09:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9ef222640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e8198320 con 0x7fa9e810bb30 2024-03-15T13:09:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9e7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8103080 0x7fa9e819dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9e7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8103080 0x7fa9e819dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44140/0 (socket says 172.21.15.2:44140) 2024-03-15T13:09:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9e7fff640 1 -- 172.21.15.2:0/3059811444 learned_addr learned my addr 172.21.15.2:0/3059811444 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9ecfbe640 1 --2- 172.21.15.2:0/3059811444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e8102760 0x7fa9e819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9ed7bf640 1 --2- 172.21.15.2:0/3059811444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e810bb30 0x7fa9e81979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9e7fff640 1 -- 172.21.15.2:0/3059811444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e810bb30 msgr2=0x7fa9e81979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9e7fff640 1 --2- 172.21.15.2:0/3059811444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e810bb30 0x7fa9e81979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9e7fff640 1 -- 172.21.15.2:0/3059811444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e8102760 msgr2=0x7fa9e819d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9e7fff640 1 --2- 172.21.15.2:0/3059811444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e8102760 0x7fa9e819d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9e7fff640 1 -- 172.21.15.2:0/3059811444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d4009560 con 0x7fa9e8103080 2024-03-15T13:09:45.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9e7fff640 1 --2- 172.21.15.2:0/3059811444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8103080 0x7fa9e819dd30 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4000c00 tx=0x7fa9d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:45.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9e5ffb640 1 -- 172.21.15.2:0/3059811444 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40044e0 con 0x7fa9e8103080 2024-03-15T13:09:45.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.713+0000 7fa9e5ffb640 1 -- 172.21.15.2:0/3059811444 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa9d4005000 con 0x7fa9e8103080 2024-03-15T13:09:45.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.714+0000 7fa9ef222640 1 -- 172.21.15.2:0/3059811444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e81985a0 con 0x7fa9e8103080 2024-03-15T13:09:45.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.714+0000 7fa9e5ffb640 1 -- 172.21.15.2:0/3059811444 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4003ab0 con 0x7fa9e8103080 2024-03-15T13:09:45.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.714+0000 7fa9ef222640 1 -- 172.21.15.2:0/3059811444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e81a4890 con 0x7fa9e8103080 2024-03-15T13:09:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.715+0000 7fa9ef222640 1 -- 172.21.15.2:0/3059811444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e8198ed0 con 0x7fa9e8103080 2024-03-15T13:09:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.716+0000 7fa9e5ffb640 1 -- 172.21.15.2:0/3059811444 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fa9d4005170 con 0x7fa9e8103080 2024-03-15T13:09:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.716+0000 7fa9e5ffb640 1 --2- 172.21.15.2:0/3059811444 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa9d80a1ba0 0x7fa9d80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.717+0000 7fa9ecfbe640 1 --2- 172.21.15.2:0/3059811444 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa9d80a1ba0 0x7fa9d80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.717+0000 7fa9e5ffb640 1 -- 172.21.15.2:0/3059811444 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fa9d40efff0 con 0x7fa9e8103080 2024-03-15T13:09:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.717+0000 7fa9ecfbe640 1 --2- 172.21.15.2:0/3059811444 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa9d80a1ba0 0x7fa9d80a4060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8198ae0 tx=0x7fa9dc005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:45.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.720+0000 7fa9e5ffb640 1 -- 172.21.15.2:0/3059811444 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa9d40b8cf0 con 0x7fa9e8103080 2024-03-15T13:09:45.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:45 smithi149 ceph-mon[35927]: pgmap v336: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:45.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:45 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1967135374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T13:09:45.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:45 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2547124273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T13:09:45.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:45 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2567251440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T13:09:45.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.904+0000 7fa9ef222640 1 -- 172.21.15.2:0/3059811444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9e8106c20 con 0x7fa9e8103080 2024-03-15T13:09:45.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.904+0000 7fa9e5ffb640 1 -- 172.21.15.2:0/3059811444 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa9d4042020 con 0x7fa9e8103080 2024-03-15T13:09:45.889 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-03-15T13:09:45.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.906+0000 7fa9ef222640 1 -- 172.21.15.2:0/3059811444 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa9d80a1ba0 msgr2=0x7fa9d80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:45.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.906+0000 7fa9ef222640 1 --2- 172.21.15.2:0/3059811444 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa9d80a1ba0 0x7fa9d80a4060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8198ae0 tx=0x7fa9dc005f90 comp rx=0 tx=0).stop 2024-03-15T13:09:45.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.906+0000 7fa9ef222640 1 -- 172.21.15.2:0/3059811444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8103080 msgr2=0x7fa9e819dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:45.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.906+0000 7fa9ef222640 1 --2- 172.21.15.2:0/3059811444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8103080 0x7fa9e819dd30 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4000c00 tx=0x7fa9d40042f0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.907+0000 7fa9ef222640 1 -- 172.21.15.2:0/3059811444 shutdown_connections 2024-03-15T13:09:45.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.907+0000 7fa9ef222640 1 --2- 172.21.15.2:0/3059811444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9e810bb30 0x7fa9e81979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.907+0000 7fa9ef222640 1 --2- 172.21.15.2:0/3059811444 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa9d80a1ba0 0x7fa9d80a4060 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.907+0000 7fa9ef222640 1 --2- 172.21.15.2:0/3059811444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8103080 0x7fa9e819dd30 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.907+0000 7fa9ef222640 1 --2- 172.21.15.2:0/3059811444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e8102760 0x7fa9e819d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:45.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.907+0000 7fa9ef222640 1 -- 172.21.15.2:0/3059811444 >> 172.21.15.2:0/3059811444 conn(0x7fa9e8077550 msgr2=0x7fa9e80fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:45.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.907+0000 7fa9ef222640 1 -- 172.21.15.2:0/3059811444 shutdown_connections 2024-03-15T13:09:45.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:45.907+0000 7fa9ef222640 1 -- 172.21.15.2:0/3059811444 wait complete. 2024-03-15T13:09:45.900 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:45 smithi002 ceph-mon[32060]: pgmap v336: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:45.900 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:45 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1967135374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T13:09:45.900 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:45 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2547124273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T13:09:45.900 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:45 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2567251440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T13:09:45.901 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-15T13:09:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.023+0000 7f576fc83640 1 -- 172.21.15.2:0/3931874006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57681005d0 msgr2=0x7f57681009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.023+0000 7f576fc83640 1 --2- 172.21.15.2:0/3931874006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57681005d0 0x7f57681009b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f5764009cd0 tx=0x7f576402c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.023+0000 7f576fc83640 1 -- 172.21.15.2:0/3931874006 shutdown_connections 2024-03-15T13:09:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.023+0000 7f576fc83640 1 --2- 172.21.15.2:0/3931874006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57681056a0 0x7f576810d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.023+0000 7f576fc83640 1 --2- 172.21.15.2:0/3931874006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5768100f80 0x7f5768104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.023+0000 7f576fc83640 1 --2- 172.21.15.2:0/3931874006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57681005d0 0x7f57681009b0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.023+0000 7f576fc83640 1 -- 172.21.15.2:0/3931874006 >> 172.21.15.2:0/3931874006 conn(0x7f57680774f0 msgr2=0x7f5768077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.023+0000 7f576fc83640 1 -- 172.21.15.2:0/3931874006 shutdown_connections 2024-03-15T13:09:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.023+0000 7f576fc83640 1 -- 172.21.15.2:0/3931874006 wait complete. 2024-03-15T13:09:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.023+0000 7f576fc83640 1 Processor -- start 2024-03-15T13:09:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.023+0000 7f576fc83640 1 -- start start 2024-03-15T13:09:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576fc83640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57681005d0 0x7f576819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576fc83640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57681056a0 0x7f576819dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576fc83640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5768197960 0x7f5768197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576fc83640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5768198350 con 0x7f57681005d0 2024-03-15T13:09:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576fc83640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57681984c0 con 0x7f5768197960 2024-03-15T13:09:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576fc83640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5768198680 con 0x7f57681056a0 2024-03-15T13:09:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576da1f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57681005d0 0x7f576819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576da1f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57681005d0 0x7f576819d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44156/0 (socket says 172.21.15.2:44156) 2024-03-15T13:09:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576da1f640 1 -- 172.21.15.2:0/4076503616 learned_addr learned my addr 172.21.15.2:0/4076503616 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576d21e640 1 --2- 172.21.15.2:0/4076503616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57681056a0 0x7f576819dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576e220640 1 --2- 172.21.15.2:0/4076503616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5768197960 0x7f5768197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576da1f640 1 -- 172.21.15.2:0/4076503616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57681056a0 msgr2=0x7f576819dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576da1f640 1 --2- 172.21.15.2:0/4076503616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57681056a0 0x7f576819dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576da1f640 1 -- 172.21.15.2:0/4076503616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5768197960 msgr2=0x7f5768197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576da1f640 1 --2- 172.21.15.2:0/4076503616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5768197960 0x7f5768197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576da1f640 1 -- 172.21.15.2:0/4076503616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5758009490 con 0x7f57681005d0 2024-03-15T13:09:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576da1f640 1 --2- 172.21.15.2:0/4076503616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57681005d0 0x7f576819d7b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f5764005cc0 tx=0x7f5764032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f575effd640 1 -- 172.21.15.2:0/4076503616 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764032880 con 0x7f57681005d0 2024-03-15T13:09:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f575effd640 1 -- 172.21.15.2:0/4076503616 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f5764003e40 con 0x7f57681005d0 2024-03-15T13:09:46.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576fc83640 1 -- 172.21.15.2:0/4076503616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5764009900 con 0x7f57681005d0 2024-03-15T13:09:46.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f575effd640 1 -- 172.21.15.2:0/4076503616 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764040450 con 0x7f57681005d0 2024-03-15T13:09:46.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.024+0000 7f576fc83640 1 -- 172.21.15.2:0/4076503616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57681a4960 con 0x7f57681005d0 2024-03-15T13:09:46.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.026+0000 7f576fc83640 1 -- 172.21.15.2:0/4076503616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57680682e0 con 0x7f57681005d0 2024-03-15T13:09:46.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.027+0000 7f575effd640 1 -- 172.21.15.2:0/4076503616 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f576400a040 con 0x7f57681005d0 2024-03-15T13:09:46.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.027+0000 7f575effd640 1 --2- 172.21.15.2:0/4076503616 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f57540a1ba0 0x7f57540a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:46.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.027+0000 7f576d21e640 1 --2- 172.21.15.2:0/4076503616 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f57540a1ba0 0x7f57540a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:46.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.028+0000 7f575effd640 1 -- 172.21.15.2:0/4076503616 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f57640f0750 con 0x7f57681005d0 2024-03-15T13:09:46.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.028+0000 7f576d21e640 1 --2- 172.21.15.2:0/4076503616 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f57540a1ba0 0x7f57540a4060 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f57680fde40 tx=0x7f5758005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:46.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.032+0000 7f575effd640 1 -- 172.21.15.2:0/4076503616 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f57640b9420 con 0x7f57681005d0 2024-03-15T13:09:46.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:45 smithi184 ceph-mon[35993]: pgmap v336: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:46.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:45 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1967135374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T13:09:46.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:45 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2547124273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T13:09:46.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:45 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2567251440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T13:09:46.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.212+0000 7f576fc83640 1 -- 172.21.15.2:0/4076503616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5768100f80 con 0x7f57681005d0 2024-03-15T13:09:46.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.213+0000 7f575effd640 1 -- 172.21.15.2:0/4076503616 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v100) v1 ==== 103+0+11 (secure 0 0 0) 0x7f57640bcae0 con 0x7f57681005d0 2024-03-15T13:09:46.197 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T13:09:46.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.215+0000 7f576fc83640 1 -- 172.21.15.2:0/4076503616 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f57540a1ba0 msgr2=0x7f57540a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:46.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.215+0000 7f576fc83640 1 --2- 172.21.15.2:0/4076503616 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f57540a1ba0 0x7f57540a4060 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f57680fde40 tx=0x7f5758005f90 comp rx=0 tx=0).stop 2024-03-15T13:09:46.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.215+0000 7f576fc83640 1 -- 172.21.15.2:0/4076503616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57681005d0 msgr2=0x7f576819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:46.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.215+0000 7f576fc83640 1 --2- 172.21.15.2:0/4076503616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57681005d0 0x7f576819d7b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f5764005cc0 tx=0x7f5764032fc0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.215+0000 7f576fc83640 1 -- 172.21.15.2:0/4076503616 shutdown_connections 2024-03-15T13:09:46.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.215+0000 7f576fc83640 1 --2- 172.21.15.2:0/4076503616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5768197960 0x7f5768197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.215+0000 7f576fc83640 1 --2- 172.21.15.2:0/4076503616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57681056a0 0x7f576819dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.215+0000 7f576fc83640 1 --2- 172.21.15.2:0/4076503616 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f57540a1ba0 0x7f57540a4060 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.215+0000 7f576fc83640 1 --2- 172.21.15.2:0/4076503616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57681005d0 0x7f576819d7b0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.215+0000 7f576fc83640 1 -- 172.21.15.2:0/4076503616 >> 172.21.15.2:0/4076503616 conn(0x7f57680774f0 msgr2=0x7f57680689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:46.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.215+0000 7f576fc83640 1 -- 172.21.15.2:0/4076503616 shutdown_connections 2024-03-15T13:09:46.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.215+0000 7f576fc83640 1 -- 172.21.15.2:0/4076503616 wait complete. 2024-03-15T13:09:46.210 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-15T13:09:46.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.328+0000 7f43a4f56640 1 -- 172.21.15.2:0/4062782215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43a01056a0 msgr2=0x7f43a010d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:46.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.328+0000 7f43a4f56640 1 --2- 172.21.15.2:0/4062782215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43a01056a0 0x7f43a010d270 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f4398009cd0 tx=0x7f439802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.329+0000 7f43a4f56640 1 -- 172.21.15.2:0/4062782215 shutdown_connections 2024-03-15T13:09:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.329+0000 7f43a4f56640 1 --2- 172.21.15.2:0/4062782215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43a01056a0 0x7f43a010d270 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.329+0000 7f43a4f56640 1 --2- 172.21.15.2:0/4062782215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43a0100f80 0x7f43a0104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.329+0000 7f43a4f56640 1 --2- 172.21.15.2:0/4062782215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a01005d0 0x7f43a01009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.329+0000 7f43a4f56640 1 -- 172.21.15.2:0/4062782215 >> 172.21.15.2:0/4062782215 conn(0x7f43a00774f0 msgr2=0x7f43a0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.329+0000 7f43a4f56640 1 -- 172.21.15.2:0/4062782215 shutdown_connections 2024-03-15T13:09:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.329+0000 7f43a4f56640 1 -- 172.21.15.2:0/4062782215 wait complete. 2024-03-15T13:09:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.329+0000 7f43a4f56640 1 Processor -- start 2024-03-15T13:09:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.329+0000 7f43a4f56640 1 -- start start 2024-03-15T13:09:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f43a4f56640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43a01005d0 0x7f43a019d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f43a4f56640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a01056a0 0x7f43a019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f43a4f56640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43a0197940 0x7f43a0197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f43a4f56640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a0198330 con 0x7f43a01005d0 2024-03-15T13:09:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f43a4f56640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a01984a0 con 0x7f43a01056a0 2024-03-15T13:09:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f43a4f56640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a0198660 con 0x7f43a0197940 2024-03-15T13:09:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f439e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43a01005d0 0x7f43a019d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f439e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43a01005d0 0x7f43a019d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44174/0 (socket says 172.21.15.2:44174) 2024-03-15T13:09:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f439e59c640 1 -- 172.21.15.2:0/2709409718 learned_addr learned my addr 172.21.15.2:0/2709409718 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f439dd9b640 1 --2- 172.21.15.2:0/2709409718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a01056a0 0x7f43a019dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f439ed9d640 1 --2- 172.21.15.2:0/2709409718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43a0197940 0x7f43a0197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f439e59c640 1 -- 172.21.15.2:0/2709409718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43a0197940 msgr2=0x7f43a0197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f439e59c640 1 --2- 172.21.15.2:0/2709409718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43a0197940 0x7f43a0197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f439e59c640 1 -- 172.21.15.2:0/2709409718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a01056a0 msgr2=0x7f43a019dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f439e59c640 1 --2- 172.21.15.2:0/2709409718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a01056a0 0x7f43a019dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f439e59c640 1 -- 172.21.15.2:0/2709409718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4390009490 con 0x7f43a01005d0 2024-03-15T13:09:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f439e59c640 1 --2- 172.21.15.2:0/2709409718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43a01005d0 0x7f43a019d790 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f438800e9f0 tx=0x7f438800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f43977fe640 1 -- 172.21.15.2:0/2709409718 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4388009700 con 0x7f43a01005d0 2024-03-15T13:09:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f43977fe640 1 -- 172.21.15.2:0/2709409718 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f438800cd80 con 0x7f43a01005d0 2024-03-15T13:09:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f43977fe640 1 -- 172.21.15.2:0/2709409718 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4388018920 con 0x7f43a01005d0 2024-03-15T13:09:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f43a4f56640 1 -- 172.21.15.2:0/2709409718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4398009900 con 0x7f43a01005d0 2024-03-15T13:09:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.330+0000 7f43a4f56640 1 -- 172.21.15.2:0/2709409718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a01a49d0 con 0x7f43a01005d0 2024-03-15T13:09:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.331+0000 7f43a4f56640 1 -- 172.21.15.2:0/2709409718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a00682e0 con 0x7f43a01005d0 2024-03-15T13:09:46.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.332+0000 7f43977fe640 1 -- 172.21.15.2:0/2709409718 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f4388018a80 con 0x7f43a01005d0 2024-03-15T13:09:46.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.333+0000 7f43977fe640 1 --2- 172.21.15.2:0/2709409718 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f438c0a1ba0 0x7f438c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:46.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.333+0000 7f439dd9b640 1 --2- 172.21.15.2:0/2709409718 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f438c0a1ba0 0x7f438c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:46.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.333+0000 7f43977fe640 1 -- 172.21.15.2:0/2709409718 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f4388014070 con 0x7f43a01005d0 2024-03-15T13:09:46.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.333+0000 7f439dd9b640 1 --2- 172.21.15.2:0/2709409718 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f438c0a1ba0 0x7f438c0a4060 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f43a0069710 tx=0x7f4390005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:46.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.338+0000 7f43977fe640 1 -- 172.21.15.2:0/2709409718 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4388053280 con 0x7f43a01005d0 2024-03-15T13:09:46.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.532+0000 7f43a4f56640 1 -- 172.21.15.2:0/2709409718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f43a0100fb0 con 0x7f43a01005d0 2024-03-15T13:09:46.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.532+0000 7f43977fe640 1 -- 172.21.15.2:0/2709409718 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v100) v1 ==== 81+0+981 (secure 0 0 0) 0x7f4388096aa0 con 0x7f43a01005d0 2024-03-15T13:09:46.517 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/343748862 2024-03-16T12:58:53.781001+0000 2024-03-15T13:09:46.517 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3673716509 2024-03-16T12:49:18.658825+0000 2024-03-15T13:09:46.517 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3111414481 2024-03-16T12:48:51.791072+0000 2024-03-15T13:09:46.517 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3111414481 2024-03-16T12:48:51.791072+0000 2024-03-15T13:09:46.517 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3546094985 2024-03-16T12:48:51.791072+0000 2024-03-15T13:09:46.517 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1464336909 2024-03-16T12:48:51.791072+0000 2024-03-15T13:09:46.517 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/84615744 2024-03-16T12:48:51.791072+0000 2024-03-15T13:09:46.517 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/656434628 2024-03-16T12:49:18.658825+0000 2024-03-15T13:09:46.518 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3269509343 2024-03-16T12:49:18.658825+0000 2024-03-15T13:09:46.518 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3269509343 2024-03-16T12:49:18.658825+0000 2024-03-15T13:09:46.518 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3349070582 2024-03-16T12:49:18.658825+0000 2024-03-15T13:09:46.518 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1602727020 2024-03-16T12:58:53.781001+0000 2024-03-15T13:09:46.518 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/883225361 2024-03-16T12:58:53.781001+0000 2024-03-15T13:09:46.518 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3173750072 2024-03-16T12:58:53.781001+0000 2024-03-15T13:09:46.518 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/343748862 2024-03-16T12:58:53.781001+0000 2024-03-15T13:09:46.518 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2211760175 2024-03-16T12:58:53.781001+0000 2024-03-15T13:09:46.518 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/4030567035 2024-03-16T12:58:53.781001+0000 2024-03-15T13:09:46.519 INFO:teuthology.orchestra.run.smithi002.stderr:listed 17 entries 2024-03-15T13:09:46.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.534+0000 7f43a4f56640 1 -- 172.21.15.2:0/2709409718 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f438c0a1ba0 msgr2=0x7f438c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:46.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.534+0000 7f43a4f56640 1 --2- 172.21.15.2:0/2709409718 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f438c0a1ba0 0x7f438c0a4060 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f43a0069710 tx=0x7f4390005f90 comp rx=0 tx=0).stop 2024-03-15T13:09:46.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.534+0000 7f43a4f56640 1 -- 172.21.15.2:0/2709409718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43a01005d0 msgr2=0x7f43a019d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:46.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.534+0000 7f43a4f56640 1 --2- 172.21.15.2:0/2709409718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43a01005d0 0x7f43a019d790 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f438800e9f0 tx=0x7f438800eeb0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.534+0000 7f43a4f56640 1 -- 172.21.15.2:0/2709409718 shutdown_connections 2024-03-15T13:09:46.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.534+0000 7f43a4f56640 1 --2- 172.21.15.2:0/2709409718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43a0197940 0x7f43a0197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.534+0000 7f43a4f56640 1 --2- 172.21.15.2:0/2709409718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43a01056a0 0x7f43a019dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.534+0000 7f43a4f56640 1 --2- 172.21.15.2:0/2709409718 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f438c0a1ba0 0x7f438c0a4060 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.534+0000 7f43a4f56640 1 --2- 172.21.15.2:0/2709409718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43a01005d0 0x7f43a019d790 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.534+0000 7f43a4f56640 1 -- 172.21.15.2:0/2709409718 >> 172.21.15.2:0/2709409718 conn(0x7f43a00774f0 msgr2=0x7f43a01042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:46.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.534+0000 7f43a4f56640 1 -- 172.21.15.2:0/2709409718 shutdown_connections 2024-03-15T13:09:46.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.534+0000 7f43a4f56640 1 -- 172.21.15.2:0/2709409718 wait complete. 2024-03-15T13:09:46.529 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-15T13:09:46.529 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-15T13:09:46.545 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T13:09:46.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.704+0000 7f4484c47640 1 -- 172.21.15.2:0/497487252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4480107560 msgr2=0x7f4480107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:46.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.704+0000 7f4484c47640 1 --2- 172.21.15.2:0/497487252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4480107560 0x7f4480107940 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f4474009900 tx=0x7f447402bc30 comp rx=0 tx=0).stop 2024-03-15T13:09:46.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.704+0000 7f4484c47640 1 -- 172.21.15.2:0/497487252 shutdown_connections 2024-03-15T13:09:46.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.704+0000 7f4484c47640 1 --2- 172.21.15.2:0/497487252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4480101e60 0x7f448010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.704+0000 7f4484c47640 1 --2- 172.21.15.2:0/497487252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44801014a0 0x7f4480101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.704+0000 7f4484c47640 1 --2- 172.21.15.2:0/497487252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4480107560 0x7f4480107940 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.704+0000 7f4484c47640 1 -- 172.21.15.2:0/497487252 >> 172.21.15.2:0/497487252 conn(0x7f44800fcf90 msgr2=0x7f44800ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.704+0000 7f4484c47640 1 -- 172.21.15.2:0/497487252 shutdown_connections 2024-03-15T13:09:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.704+0000 7f4484c47640 1 -- 172.21.15.2:0/497487252 wait complete. 2024-03-15T13:09:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.704+0000 7f4484c47640 1 Processor -- start 2024-03-15T13:09:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.704+0000 7f4484c47640 1 -- start start 2024-03-15T13:09:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f4484c47640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44801014a0 0x7f448019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f4484c47640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4480101e60 0x7f448019dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f4484c47640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4480107560 0x7f4480197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f4484c47640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4480197ee0 con 0x7f4480101e60 2024-03-15T13:09:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f4484c47640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4480198050 con 0x7f44801014a0 2024-03-15T13:09:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f4484c47640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4480198210 con 0x7f4480107560 2024-03-15T13:09:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f447dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4480101e60 0x7f448019dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f447dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4480101e60 0x7f448019dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44200/0 (socket says 172.21.15.2:44200) 2024-03-15T13:09:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f447dd9b640 1 -- 172.21.15.2:0/1673595147 learned_addr learned my addr 172.21.15.2:0/1673595147 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f447e59c640 1 --2- 172.21.15.2:0/1673595147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44801014a0 0x7f448019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:46.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f447ed9d640 1 --2- 172.21.15.2:0/1673595147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4480107560 0x7f4480197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:46.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f447dd9b640 1 -- 172.21.15.2:0/1673595147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4480107560 msgr2=0x7f4480197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:46.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f447dd9b640 1 --2- 172.21.15.2:0/1673595147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4480107560 0x7f4480197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f447dd9b640 1 -- 172.21.15.2:0/1673595147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44801014a0 msgr2=0x7f448019d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:46.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f447dd9b640 1 --2- 172.21.15.2:0/1673595147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44801014a0 0x7f448019d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f447dd9b640 1 -- 172.21.15.2:0/1673595147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4474009560 con 0x7f4480101e60 2024-03-15T13:09:46.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f447dd9b640 1 --2- 172.21.15.2:0/1673595147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4480101e60 0x7f448019dc20 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f447000e9f0 tx=0x7f447000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:46.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f446f7fe640 1 -- 172.21.15.2:0/1673595147 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470009700 con 0x7f4480101e60 2024-03-15T13:09:46.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f446f7fe640 1 -- 172.21.15.2:0/1673595147 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f447000cd80 con 0x7f4480101e60 2024-03-15T13:09:46.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f4484c47640 1 -- 172.21.15.2:0/1673595147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44801984f0 con 0x7f4480101e60 2024-03-15T13:09:46.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f446f7fe640 1 -- 172.21.15.2:0/1673595147 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470018920 con 0x7f4480101e60 2024-03-15T13:09:46.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.705+0000 7f4484c47640 1 -- 172.21.15.2:0/1673595147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44801a4780 con 0x7f4480101e60 2024-03-15T13:09:46.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.706+0000 7f4484c47640 1 -- 172.21.15.2:0/1673595147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4480068730 con 0x7f4480101e60 2024-03-15T13:09:46.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.707+0000 7f446f7fe640 1 -- 172.21.15.2:0/1673595147 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f4470018a80 con 0x7f4480101e60 2024-03-15T13:09:46.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.708+0000 7f446f7fe640 1 --2- 172.21.15.2:0/1673595147 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f44680a1ba0 0x7f44680a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:46.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.708+0000 7f447e59c640 1 --2- 172.21.15.2:0/1673595147 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f44680a1ba0 0x7f44680a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:46.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.708+0000 7f446f7fe640 1 -- 172.21.15.2:0/1673595147 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f4470014070 con 0x7f4480101e60 2024-03-15T13:09:46.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.709+0000 7f447e59c640 1 --2- 172.21.15.2:0/1673595147 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f44680a1ba0 0x7f44680a4060 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4474005ab0 tx=0x7f4474005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:46.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.712+0000 7f446f7fe640 1 -- 172.21.15.2:0/1673595147 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4470053280 con 0x7f4480101e60 2024-03-15T13:09:46.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:46 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3059811444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T13:09:46.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:46 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/4076503616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T13:09:46.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:46 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2709409718' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-15T13:09:46.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:46 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3059811444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T13:09:46.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:46 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/4076503616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T13:09:46.807 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:46 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2709409718' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-15T13:09:46.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:46 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3059811444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T13:09:46.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:46 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/4076503616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T13:09:46.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:46 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2709409718' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-15T13:09:46.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.894+0000 7f4484c47640 1 -- 172.21.15.2:0/1673595147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44801115f0 con 0x7f4480101e60 2024-03-15T13:09:46.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.896+0000 7f446f7fe640 1 -- 172.21.15.2:0/1673595147 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+25138 (secure 0 0 0) 0x7f4470096a60 con 0x7f4480101e60 2024-03-15T13:09:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:09:46.880 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":100,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T13:09:15.557407+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/343748862":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/1602727020":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/883225361":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/3173750072":"2024-03-16T12:58:53.781001+0000","172.21.15.2:6800/343748862":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/2211760175":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/4030567035":"2024-03-16T12:58:53.781001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T13:09:46.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.897+0000 7f4484c47640 1 -- 172.21.15.2:0/1673595147 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f44680a1ba0 msgr2=0x7f44680a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:46.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.897+0000 7f4484c47640 1 --2- 172.21.15.2:0/1673595147 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f44680a1ba0 0x7f44680a4060 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4474005ab0 tx=0x7f4474005fd0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.897+0000 7f4484c47640 1 -- 172.21.15.2:0/1673595147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4480101e60 msgr2=0x7f448019dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:46.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.897+0000 7f4484c47640 1 --2- 172.21.15.2:0/1673595147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4480101e60 0x7f448019dc20 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f447000e9f0 tx=0x7f447000eeb0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.897+0000 7f4484c47640 1 -- 172.21.15.2:0/1673595147 shutdown_connections 2024-03-15T13:09:46.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.897+0000 7f4484c47640 1 --2- 172.21.15.2:0/1673595147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4480107560 0x7f4480197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.897+0000 7f4484c47640 1 --2- 172.21.15.2:0/1673595147 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f44680a1ba0 0x7f44680a4060 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.897+0000 7f4484c47640 1 --2- 172.21.15.2:0/1673595147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4480101e60 0x7f448019dc20 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.897+0000 7f4484c47640 1 --2- 172.21.15.2:0/1673595147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44801014a0 0x7f448019d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:46.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.897+0000 7f4484c47640 1 -- 172.21.15.2:0/1673595147 >> 172.21.15.2:0/1673595147 conn(0x7f44800fcf90 msgr2=0x7f44800fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:46.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.898+0000 7f4484c47640 1 -- 172.21.15.2:0/1673595147 shutdown_connections 2024-03-15T13:09:46.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:46.898+0000 7f4484c47640 1 -- 172.21.15.2:0/1673595147 wait complete. 2024-03-15T13:09:46.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-15T12:51:47.004843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-15T12:56:18.559882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-15T12:56:19.751767+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-15T12:56:23.664879+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-15T13:09:46.893 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-15T13:09:47.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.014+0000 7f7fa59f4640 1 -- 172.21.15.2:0/2138498246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa01005d0 msgr2=0x7f7fa01009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.014+0000 7f7fa59f4640 1 --2- 172.21.15.2:0/2138498246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa01005d0 0x7f7fa01009b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009cd0 tx=0x7f7f9002c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.014+0000 7f7fa59f4640 1 -- 172.21.15.2:0/2138498246 shutdown_connections 2024-03-15T13:09:47.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.014+0000 7f7fa59f4640 1 --2- 172.21.15.2:0/2138498246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fa01056a0 0x7f7fa010d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.014+0000 7f7fa59f4640 1 --2- 172.21.15.2:0/2138498246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fa0100f80 0x7f7fa0104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.014+0000 7f7fa59f4640 1 --2- 172.21.15.2:0/2138498246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa01005d0 0x7f7fa01009b0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.014+0000 7f7fa59f4640 1 -- 172.21.15.2:0/2138498246 >> 172.21.15.2:0/2138498246 conn(0x7f7fa00774f0 msgr2=0x7f7fa0077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:47.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.014+0000 7f7fa59f4640 1 -- 172.21.15.2:0/2138498246 shutdown_connections 2024-03-15T13:09:47.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.014+0000 7f7fa59f4640 1 -- 172.21.15.2:0/2138498246 wait complete. 2024-03-15T13:09:47.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.015+0000 7f7fa59f4640 1 Processor -- start 2024-03-15T13:09:47.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.015+0000 7f7fa59f4640 1 -- start start 2024-03-15T13:09:47.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.015+0000 7f7fa59f4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fa01005d0 0x7f7fa01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.015+0000 7f7fa59f4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fa01056a0 0x7f7fa0198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.015+0000 7f7fa59f4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa0199480 0x7f7fa01a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.015+0000 7f7fa59f4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01a4100 con 0x7f7fa0199480 2024-03-15T13:09:47.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.015+0000 7f7fa59f4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01a4270 con 0x7f7fa01005d0 2024-03-15T13:09:47.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.015+0000 7f7fa59f4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01a4430 con 0x7f7fa01056a0 2024-03-15T13:09:47.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.015+0000 7f7f9f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa0199480 0x7f7fa01a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.015+0000 7f7f9f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa0199480 0x7f7fa01a3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44216/0 (socket says 172.21.15.2:44216) 2024-03-15T13:09:47.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.015+0000 7f7f9f7fe640 1 -- 172.21.15.2:0/4065414547 learned_addr learned my addr 172.21.15.2:0/4065414547 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:47.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.015+0000 7f7f9effd640 1 --2- 172.21.15.2:0/4065414547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fa01005d0 0x7f7fa01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.015+0000 7f7f9e7fc640 1 --2- 172.21.15.2:0/4065414547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fa01056a0 0x7f7fa0198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.016+0000 7f7f9f7fe640 1 -- 172.21.15.2:0/4065414547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fa01056a0 msgr2=0x7f7fa0198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.016+0000 7f7f9f7fe640 1 --2- 172.21.15.2:0/4065414547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fa01056a0 0x7f7fa0198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.016+0000 7f7f9f7fe640 1 -- 172.21.15.2:0/4065414547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fa01005d0 msgr2=0x7f7fa01989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.016+0000 7f7f9f7fe640 1 --2- 172.21.15.2:0/4065414547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fa01005d0 0x7f7fa01989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.016+0000 7f7f9f7fe640 1 -- 172.21.15.2:0/4065414547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f88009490 con 0x7f7fa0199480 2024-03-15T13:09:47.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.016+0000 7f7f9f7fe640 1 --2- 172.21.15.2:0/4065414547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa0199480 0x7f7fa01a3a30 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800e9b0 tx=0x7f7f9800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:47.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.016+0000 7f7fa49f2640 1 -- 172.21.15.2:0/4065414547 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9800cd80 con 0x7f7fa0199480 2024-03-15T13:09:47.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.016+0000 7f7fa59f4640 1 -- 172.21.15.2:0/4065414547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f98009490 con 0x7f7fa0199480 2024-03-15T13:09:47.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.016+0000 7f7fa49f2640 1 -- 172.21.15.2:0/4065414547 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f7f980035f0 con 0x7f7fa0199480 2024-03-15T13:09:47.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.016+0000 7f7fa49f2640 1 -- 172.21.15.2:0/4065414547 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9800cee0 con 0x7f7fa0199480 2024-03-15T13:09:47.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.016+0000 7f7fa59f4640 1 -- 172.21.15.2:0/4065414547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f90009900 con 0x7f7fa0199480 2024-03-15T13:09:47.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.017+0000 7f7fa59f4640 1 -- 172.21.15.2:0/4065414547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa00682e0 con 0x7f7fa0199480 2024-03-15T13:09:47.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.018+0000 7f7fa49f2640 1 -- 172.21.15.2:0/4065414547 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f7f98003760 con 0x7f7fa0199480 2024-03-15T13:09:47.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.019+0000 7f7fa49f2640 1 --2- 172.21.15.2:0/4065414547 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7f8c0a1ba0 0x7f7f8c0a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.019+0000 7f7f9effd640 1 --2- 172.21.15.2:0/4065414547 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7f8c0a1ba0 0x7f7f8c0a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.019+0000 7f7fa49f2640 1 -- 172.21.15.2:0/4065414547 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f7f98014070 con 0x7f7fa0199480 2024-03-15T13:09:47.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.019+0000 7f7f9effd640 1 --2- 172.21.15.2:0/4065414547 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7f8c0a1ba0 0x7f7f8c0a4060 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7f90005cc0 tx=0x7f7f90005c50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:47.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.022+0000 7f7fa49f2640 1 -- 172.21.15.2:0/4065414547 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7f98096c30 con 0x7f7fa0199480 2024-03-15T13:09:47.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.202+0000 7f7fa59f4640 1 -- 172.21.15.2:0/4065414547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7fa0068c60 con 0x7f7fa0199480 2024-03-15T13:09:47.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.203+0000 7f7fa49f2640 1 -- 172.21.15.2:0/4065414547 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7f9809a2f0 con 0x7f7fa0199480 2024-03-15T13:09:47.187 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-15T13:09:47.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.204+0000 7f7fa59f4640 1 -- 172.21.15.2:0/4065414547 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7f8c0a1ba0 msgr2=0x7f7f8c0a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.204+0000 7f7fa59f4640 1 --2- 172.21.15.2:0/4065414547 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7f8c0a1ba0 0x7f7f8c0a4060 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7f90005cc0 tx=0x7f7f90005c50 comp rx=0 tx=0).stop 2024-03-15T13:09:47.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.204+0000 7f7fa59f4640 1 -- 172.21.15.2:0/4065414547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa0199480 msgr2=0x7f7fa01a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.204+0000 7f7fa59f4640 1 --2- 172.21.15.2:0/4065414547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa0199480 0x7f7fa01a3a30 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800e9b0 tx=0x7f7f9800ee70 comp rx=0 tx=0).stop 2024-03-15T13:09:47.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.205+0000 7f7fa59f4640 1 -- 172.21.15.2:0/4065414547 shutdown_connections 2024-03-15T13:09:47.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.205+0000 7f7fa59f4640 1 --2- 172.21.15.2:0/4065414547 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f7f8c0a1ba0 0x7f7f8c0a4060 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.205+0000 7f7fa59f4640 1 --2- 172.21.15.2:0/4065414547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7fa0199480 0x7f7fa01a3a30 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.205+0000 7f7fa59f4640 1 --2- 172.21.15.2:0/4065414547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fa01056a0 0x7f7fa0198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.205+0000 7f7fa59f4640 1 --2- 172.21.15.2:0/4065414547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fa01005d0 0x7f7fa01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.205+0000 7f7fa59f4640 1 -- 172.21.15.2:0/4065414547 >> 172.21.15.2:0/4065414547 conn(0x7f7fa00774f0 msgr2=0x7f7fa00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:47.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.205+0000 7f7fa59f4640 1 -- 172.21.15.2:0/4065414547 shutdown_connections 2024-03-15T13:09:47.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.205+0000 7f7fa59f4640 1 -- 172.21.15.2:0/4065414547 wait complete. 2024-03-15T13:09:47.200 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-15T13:09:47.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.321+0000 7fdb6fad4640 1 -- 172.21.15.2:0/492770820 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb68102760 msgr2=0x7fdb68102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.321+0000 7fdb6fad4640 1 --2- 172.21.15.2:0/492770820 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb68102760 0x7fdb68102b40 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009ca0 tx=0x7fdb5802c180 comp rx=0 tx=0).stop 2024-03-15T13:09:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.321+0000 7fdb6fad4640 1 -- 172.21.15.2:0/492770820 shutdown_connections 2024-03-15T13:09:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.321+0000 7fdb6fad4640 1 --2- 172.21.15.2:0/492770820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb6810bb30 0x7fdb6810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.321+0000 7fdb6fad4640 1 --2- 172.21.15.2:0/492770820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb68103080 0x7fdb6810b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.321+0000 7fdb6fad4640 1 --2- 172.21.15.2:0/492770820 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb68102760 0x7fdb68102b40 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.321+0000 7fdb6fad4640 1 -- 172.21.15.2:0/492770820 >> 172.21.15.2:0/492770820 conn(0x7fdb68077550 msgr2=0x7fdb68077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.321+0000 7fdb6fad4640 1 -- 172.21.15.2:0/492770820 shutdown_connections 2024-03-15T13:09:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.321+0000 7fdb6fad4640 1 -- 172.21.15.2:0/492770820 wait complete. 2024-03-15T13:09:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.321+0000 7fdb6fad4640 1 Processor -- start 2024-03-15T13:09:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.321+0000 7fdb6fad4640 1 -- start start 2024-03-15T13:09:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6fad4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb68102760 0x7fdb6819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6fad4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb68103080 0x7fdb6819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6fad4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb6810bb30 0x7fdb68197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6fad4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb68197fe0 con 0x7fdb6810bb30 2024-03-15T13:09:47.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6fad4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb68198150 con 0x7fdb68102760 2024-03-15T13:09:47.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6fad4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb68198310 con 0x7fdb68103080 2024-03-15T13:09:47.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6e071640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb6810bb30 0x7fdb68197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6e071640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb6810bb30 0x7fdb68197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44232/0 (socket says 172.21.15.2:44232) 2024-03-15T13:09:47.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6e071640 1 -- 172.21.15.2:0/3132760754 learned_addr learned my addr 172.21.15.2:0/3132760754 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:47.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6d870640 1 --2- 172.21.15.2:0/3132760754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb68102760 0x7fdb6819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6d06f640 1 --2- 172.21.15.2:0/3132760754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb68103080 0x7fdb6819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6e071640 1 -- 172.21.15.2:0/3132760754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb68103080 msgr2=0x7fdb6819dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6e071640 1 --2- 172.21.15.2:0/3132760754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb68103080 0x7fdb6819dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6e071640 1 -- 172.21.15.2:0/3132760754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb68102760 msgr2=0x7fdb6819d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6e071640 1 --2- 172.21.15.2:0/3132760754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb68102760 0x7fdb6819d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6e071640 1 -- 172.21.15.2:0/3132760754 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb60009490 con 0x7fdb6810bb30 2024-03-15T13:09:47.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6d06f640 1 --2- 172.21.15.2:0/3132760754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb68103080 0x7fdb6819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:47.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6e071640 1 --2- 172.21.15.2:0/3132760754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb6810bb30 0x7fdb68197990 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000e9b0 tx=0x7fdb6000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:47.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb66ffd640 1 -- 172.21.15.2:0/3132760754 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6000cd80 con 0x7fdb6810bb30 2024-03-15T13:09:47.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb6fad4640 1 -- 172.21.15.2:0/3132760754 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb58009900 con 0x7fdb6810bb30 2024-03-15T13:09:47.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb66ffd640 1 -- 172.21.15.2:0/3132760754 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdb600035f0 con 0x7fdb6810bb30 2024-03-15T13:09:47.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.322+0000 7fdb66ffd640 1 -- 172.21.15.2:0/3132760754 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6000cee0 con 0x7fdb6810bb30 2024-03-15T13:09:47.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.323+0000 7fdb6fad4640 1 -- 172.21.15.2:0/3132760754 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb681a46a0 con 0x7fdb6810bb30 2024-03-15T13:09:47.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.323+0000 7fdb6fad4640 1 -- 172.21.15.2:0/3132760754 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb68068730 con 0x7fdb6810bb30 2024-03-15T13:09:47.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.325+0000 7fdb66ffd640 1 -- 172.21.15.2:0/3132760754 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fdb60003760 con 0x7fdb6810bb30 2024-03-15T13:09:47.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.325+0000 7fdb66ffd640 1 --2- 172.21.15.2:0/3132760754 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdb500a1ba0 0x7fdb500a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.325+0000 7fdb6d870640 1 --2- 172.21.15.2:0/3132760754 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdb500a1ba0 0x7fdb500a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.325+0000 7fdb66ffd640 1 -- 172.21.15.2:0/3132760754 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fdb60014070 con 0x7fdb6810bb30 2024-03-15T13:09:47.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.326+0000 7fdb6d870640 1 --2- 172.21.15.2:0/3132760754 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdb500a1ba0 0x7fdb500a4060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdb58004af0 tx=0x7fdb58005c50 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:47.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.329+0000 7fdb66ffd640 1 -- 172.21.15.2:0/3132760754 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdb60096bf0 con 0x7fdb6810bb30 2024-03-15T13:09:47.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.511+0000 7fdb6fad4640 1 -- 172.21.15.2:0/3132760754 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb68106c20 con 0x7fdb6810bb30 2024-03-15T13:09:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.511+0000 7fdb66ffd640 1 -- 172.21.15.2:0/3132760754 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb6009a2b0 con 0x7fdb6810bb30 2024-03-15T13:09:47.496 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T13:09:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.513+0000 7fdb6fad4640 1 -- 172.21.15.2:0/3132760754 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdb500a1ba0 msgr2=0x7fdb500a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.513+0000 7fdb6fad4640 1 --2- 172.21.15.2:0/3132760754 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdb500a1ba0 0x7fdb500a4060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdb58004af0 tx=0x7fdb58005c50 comp rx=0 tx=0).stop 2024-03-15T13:09:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.513+0000 7fdb6fad4640 1 -- 172.21.15.2:0/3132760754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb6810bb30 msgr2=0x7fdb68197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.513+0000 7fdb6fad4640 1 --2- 172.21.15.2:0/3132760754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb6810bb30 0x7fdb68197990 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000e9b0 tx=0x7fdb6000ee70 comp rx=0 tx=0).stop 2024-03-15T13:09:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.513+0000 7fdb6fad4640 1 -- 172.21.15.2:0/3132760754 shutdown_connections 2024-03-15T13:09:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.513+0000 7fdb6fad4640 1 --2- 172.21.15.2:0/3132760754 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdb500a1ba0 0x7fdb500a4060 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.513+0000 7fdb6fad4640 1 --2- 172.21.15.2:0/3132760754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb6810bb30 0x7fdb68197990 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.513+0000 7fdb6fad4640 1 --2- 172.21.15.2:0/3132760754 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb68103080 0x7fdb6819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.513+0000 7fdb6fad4640 1 --2- 172.21.15.2:0/3132760754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb68102760 0x7fdb6819d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.513+0000 7fdb6fad4640 1 -- 172.21.15.2:0/3132760754 >> 172.21.15.2:0/3132760754 conn(0x7fdb68077550 msgr2=0x7fdb680fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:47.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.513+0000 7fdb6fad4640 1 -- 172.21.15.2:0/3132760754 shutdown_connections 2024-03-15T13:09:47.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.513+0000 7fdb6fad4640 1 -- 172.21.15.2:0/3132760754 wait complete. 2024-03-15T13:09:47.508 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-15T13:09:47.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.630+0000 7f9cbdc9a640 1 -- 172.21.15.2:0/1021003895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb810bb30 msgr2=0x7f9cb810dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.630+0000 7f9cbdc9a640 1 --2- 172.21.15.2:0/1021003895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb810bb30 0x7f9cb810dfa0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009ca0 tx=0x7f9cb002c180 comp rx=0 tx=0).stop 2024-03-15T13:09:47.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.630+0000 7f9cbdc9a640 1 -- 172.21.15.2:0/1021003895 shutdown_connections 2024-03-15T13:09:47.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.630+0000 7f9cbdc9a640 1 --2- 172.21.15.2:0/1021003895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb810bb30 0x7f9cb810dfa0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.630+0000 7f9cbdc9a640 1 --2- 172.21.15.2:0/1021003895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cb8103080 0x7f9cb810b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.630+0000 7f9cbdc9a640 1 --2- 172.21.15.2:0/1021003895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cb8102760 0x7f9cb8102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.630+0000 7f9cbdc9a640 1 -- 172.21.15.2:0/1021003895 >> 172.21.15.2:0/1021003895 conn(0x7f9cb8077550 msgr2=0x7f9cb8077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:47.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.630+0000 7f9cbdc9a640 1 -- 172.21.15.2:0/1021003895 shutdown_connections 2024-03-15T13:09:47.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.630+0000 7f9cbdc9a640 1 -- 172.21.15.2:0/1021003895 wait complete. 2024-03-15T13:09:47.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.630+0000 7f9cbdc9a640 1 Processor -- start 2024-03-15T13:09:47.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.630+0000 7f9cbdc9a640 1 -- start start 2024-03-15T13:09:47.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cbdc9a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cb8102760 0x7f9cb819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cbdc9a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb8103080 0x7f9cb819dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cbdc9a640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cb810bb30 0x7f9cb8197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cbdc9a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb8197f60 con 0x7f9cb8103080 2024-03-15T13:09:47.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cbdc9a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb81980d0 con 0x7f9cb8102760 2024-03-15T13:09:47.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cbdc9a640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb8198290 con 0x7f9cb810bb30 2024-03-15T13:09:47.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cb6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb8103080 0x7f9cb819dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cb6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb8103080 0x7f9cb819dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44246/0 (socket says 172.21.15.2:44246) 2024-03-15T13:09:47.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cb6ffd640 1 -- 172.21.15.2:0/995588301 learned_addr learned my addr 172.21.15.2:0/995588301 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:47.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cb77fe640 1 --2- 172.21.15.2:0/995588301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cb8102760 0x7f9cb819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cb7fff640 1 --2- 172.21.15.2:0/995588301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cb810bb30 0x7f9cb8197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cb6ffd640 1 -- 172.21.15.2:0/995588301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cb810bb30 msgr2=0x7f9cb8197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cb6ffd640 1 --2- 172.21.15.2:0/995588301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cb810bb30 0x7f9cb8197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cb6ffd640 1 -- 172.21.15.2:0/995588301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cb8102760 msgr2=0x7f9cb819d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cb6ffd640 1 --2- 172.21.15.2:0/995588301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cb8102760 0x7f9cb819d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cb6ffd640 1 -- 172.21.15.2:0/995588301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac009490 con 0x7f9cb8103080 2024-03-15T13:09:47.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cb77fe640 1 --2- 172.21.15.2:0/995588301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cb8102760 0x7f9cb819d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:47.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cb6ffd640 1 --2- 172.21.15.2:0/995588301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb8103080 0x7f9cb819dca0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00ea60 tx=0x7f9cac00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:47.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cb4ff9640 1 -- 172.21.15.2:0/995588301 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac00cde0 con 0x7f9cb8103080 2024-03-15T13:09:47.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cb4ff9640 1 -- 172.21.15.2:0/995588301 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f9cac0035f0 con 0x7f9cb8103080 2024-03-15T13:09:47.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cbdc9a640 1 -- 172.21.15.2:0/995588301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb0009900 con 0x7f9cb8103080 2024-03-15T13:09:47.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cb4ff9640 1 -- 172.21.15.2:0/995588301 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac010690 con 0x7f9cb8103080 2024-03-15T13:09:47.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.631+0000 7f9cbdc9a640 1 -- 172.21.15.2:0/995588301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb81a4620 con 0x7f9cb8103080 2024-03-15T13:09:47.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.632+0000 7f9cbdc9a640 1 -- 172.21.15.2:0/995588301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb8068730 con 0x7f9cb8103080 2024-03-15T13:09:47.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.633+0000 7f9cb4ff9640 1 -- 172.21.15.2:0/995588301 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f9cac003760 con 0x7f9cb8103080 2024-03-15T13:09:47.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.634+0000 7f9cb4ff9640 1 --2- 172.21.15.2:0/995588301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9ca00a1ba0 0x7f9ca00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.634+0000 7f9cb77fe640 1 --2- 172.21.15.2:0/995588301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9ca00a1ba0 0x7f9ca00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.634+0000 7f9cb4ff9640 1 -- 172.21.15.2:0/995588301 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f9cac014070 con 0x7f9cb8103080 2024-03-15T13:09:47.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.634+0000 7f9cb77fe640 1 --2- 172.21.15.2:0/995588301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9ca00a1ba0 0x7f9ca00a4060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8198a50 tx=0x7f9ca8005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:47.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.638+0000 7f9cb4ff9640 1 -- 172.21.15.2:0/995588301 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9cac096c40 con 0x7f9cb8103080 2024-03-15T13:09:47.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:47 smithi002 ceph-mon[32060]: pgmap v337: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:09:47.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:47 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1673595147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T13:09:47.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:47 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/4065414547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T13:09:47.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:47 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3132760754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T13:09:47.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.821+0000 7f9cbdc9a640 1 -- 172.21.15.2:0/995588301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9cb81116e0 con 0x7f9cb8103080 2024-03-15T13:09:47.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.821+0000 7f9cb4ff9640 1 -- 172.21.15.2:0/995588301 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9cac09a300 con 0x7f9cb8103080 2024-03-15T13:09:47.806 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-03-15T13:09:47.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.823+0000 7f9cbdc9a640 1 -- 172.21.15.2:0/995588301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9ca00a1ba0 msgr2=0x7f9ca00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.823+0000 7f9cbdc9a640 1 --2- 172.21.15.2:0/995588301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9ca00a1ba0 0x7f9ca00a4060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8198a50 tx=0x7f9ca8005f90 comp rx=0 tx=0).stop 2024-03-15T13:09:47.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.823+0000 7f9cbdc9a640 1 -- 172.21.15.2:0/995588301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb8103080 msgr2=0x7f9cb819dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.823+0000 7f9cbdc9a640 1 --2- 172.21.15.2:0/995588301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb8103080 0x7f9cb819dca0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00ea60 tx=0x7f9cac00ef20 comp rx=0 tx=0).stop 2024-03-15T13:09:47.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.824+0000 7f9cbdc9a640 1 -- 172.21.15.2:0/995588301 shutdown_connections 2024-03-15T13:09:47.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.824+0000 7f9cbdc9a640 1 --2- 172.21.15.2:0/995588301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9cb810bb30 0x7f9cb8197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.824+0000 7f9cbdc9a640 1 --2- 172.21.15.2:0/995588301 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f9ca00a1ba0 0x7f9ca00a4060 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.824+0000 7f9cbdc9a640 1 --2- 172.21.15.2:0/995588301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb8103080 0x7f9cb819dca0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.824+0000 7f9cbdc9a640 1 --2- 172.21.15.2:0/995588301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cb8102760 0x7f9cb819d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.824+0000 7f9cbdc9a640 1 -- 172.21.15.2:0/995588301 >> 172.21.15.2:0/995588301 conn(0x7f9cb8077550 msgr2=0x7f9cb80fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:47.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.824+0000 7f9cbdc9a640 1 -- 172.21.15.2:0/995588301 shutdown_connections 2024-03-15T13:09:47.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.824+0000 7f9cbdc9a640 1 -- 172.21.15.2:0/995588301 wait complete. 2024-03-15T13:09:47.818 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-15T13:09:47.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:47 smithi149 ceph-mon[35927]: pgmap v337: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:09:47.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:47 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1673595147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T13:09:47.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:47 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/4065414547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T13:09:47.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:47 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3132760754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T13:09:47.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.939+0000 7f891504e640 1 -- 172.21.15.2:0/2023856509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89101005d0 msgr2=0x7f89101009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.939+0000 7f891504e640 1 --2- 172.21.15.2:0/2023856509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89101005d0 0x7f89101009b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f8908009930 tx=0x7f890802bc30 comp rx=0 tx=0).stop 2024-03-15T13:09:47.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.939+0000 7f891504e640 1 -- 172.21.15.2:0/2023856509 shutdown_connections 2024-03-15T13:09:47.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.939+0000 7f891504e640 1 --2- 172.21.15.2:0/2023856509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89101056a0 0x7f891010d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.939+0000 7f891504e640 1 --2- 172.21.15.2:0/2023856509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8910100f80 0x7f8910104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.939+0000 7f891504e640 1 --2- 172.21.15.2:0/2023856509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89101005d0 0x7f89101009b0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.939+0000 7f891504e640 1 -- 172.21.15.2:0/2023856509 >> 172.21.15.2:0/2023856509 conn(0x7f89100774f0 msgr2=0x7f8910077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:47.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.939+0000 7f891504e640 1 -- 172.21.15.2:0/2023856509 shutdown_connections 2024-03-15T13:09:47.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.939+0000 7f891504e640 1 -- 172.21.15.2:0/2023856509 wait complete. 2024-03-15T13:09:47.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.940+0000 7f891504e640 1 Processor -- start 2024-03-15T13:09:47.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.940+0000 7f891504e640 1 -- start start 2024-03-15T13:09:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.940+0000 7f891504e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89101005d0 0x7f8910102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.940+0000 7f891504e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89101056a0 0x7f89101025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.940+0000 7f891504e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8910102ae0 0x7f89101a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.940+0000 7f891504e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89101a6340 con 0x7f89101056a0 2024-03-15T13:09:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.940+0000 7f891504e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89101a64b0 con 0x7f8910102ae0 2024-03-15T13:09:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.940+0000 7f891504e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89101a6670 con 0x7f89101005d0 2024-03-15T13:09:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f890e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89101056a0 0x7f89101025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f890f59e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8910102ae0 0x7f89101a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f890e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89101056a0 0x7f89101025a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57756/0 (socket says 172.21.15.2:57756) 2024-03-15T13:09:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f890e59c640 1 -- 172.21.15.2:0/487604275 learned_addr learned my addr 172.21.15.2:0/487604275 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:47.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f890ed9d640 1 --2- 172.21.15.2:0/487604275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89101005d0 0x7f8910102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f890f59e640 1 -- 172.21.15.2:0/487604275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89101005d0 msgr2=0x7f8910102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f890f59e640 1 --2- 172.21.15.2:0/487604275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89101005d0 0x7f8910102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f890f59e640 1 -- 172.21.15.2:0/487604275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89101056a0 msgr2=0x7f89101025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f890f59e640 1 --2- 172.21.15.2:0/487604275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89101056a0 0x7f89101025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f890f59e640 1 -- 172.21.15.2:0/487604275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8908009560 con 0x7f8910102ae0 2024-03-15T13:09:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f890e59c640 1 --2- 172.21.15.2:0/487604275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89101056a0 0x7f89101025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f890ed9d640 1 --2- 172.21.15.2:0/487604275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89101005d0 0x7f8910102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f890f59e640 1 --2- 172.21.15.2:0/487604275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8910102ae0 0x7f89101a5d70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8900009fd0 tx=0x7f890000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f88f7fff640 1 -- 172.21.15.2:0/487604275 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890000ce20 con 0x7f8910102ae0 2024-03-15T13:09:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f891504e640 1 -- 172.21.15.2:0/487604275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89101a6950 con 0x7f8910102ae0 2024-03-15T13:09:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f88f7fff640 1 -- 172.21.15.2:0/487604275 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f8900005020 con 0x7f8910102ae0 2024-03-15T13:09:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f88f7fff640 1 -- 172.21.15.2:0/487604275 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8900010690 con 0x7f8910102ae0 2024-03-15T13:09:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.941+0000 7f891504e640 1 -- 172.21.15.2:0/487604275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89101a6e60 con 0x7f8910102ae0 2024-03-15T13:09:47.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.942+0000 7f891504e640 1 -- 172.21.15.2:0/487604275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89100682e0 con 0x7f8910102ae0 2024-03-15T13:09:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.944+0000 7f88f7fff640 1 -- 172.21.15.2:0/487604275 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f8900005570 con 0x7f8910102ae0 2024-03-15T13:09:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.944+0000 7f88f7fff640 1 --2- 172.21.15.2:0/487604275 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f88f80a1bd0 0x7f88f80a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.944+0000 7f890ed9d640 1 --2- 172.21.15.2:0/487604275 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f88f80a1bd0 0x7f88f80a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.944+0000 7f88f7fff640 1 -- 172.21.15.2:0/487604275 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f8900014070 con 0x7f8910102ae0 2024-03-15T13:09:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.945+0000 7f890ed9d640 1 --2- 172.21.15.2:0/487604275 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f88f80a1bd0 0x7f88f80a4090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8908000c00 tx=0x7f8908005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:47.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:47.953+0000 7f88f7fff640 1 -- 172.21.15.2:0/487604275 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8900096d40 con 0x7f8910102ae0 2024-03-15T13:09:48.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:47 smithi184 ceph-mon[35993]: pgmap v337: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-15T13:09:48.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:47 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1673595147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T13:09:48.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:47 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/4065414547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-15T13:09:48.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:47 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3132760754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-15T13:09:48.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.133+0000 7f891504e640 1 -- 172.21.15.2:0/487604275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f89101098e0 con 0x7f8910102ae0 2024-03-15T13:09:48.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.134+0000 7f88f7fff640 1 -- 172.21.15.2:0/487604275 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v100) v1 ==== 103+0+11 (secure 0 0 0) 0x7f890009a400 con 0x7f8910102ae0 2024-03-15T13:09:48.118 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-15T13:09:48.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.136+0000 7f891504e640 1 -- 172.21.15.2:0/487604275 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f88f80a1bd0 msgr2=0x7f88f80a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:48.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.136+0000 7f891504e640 1 --2- 172.21.15.2:0/487604275 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f88f80a1bd0 0x7f88f80a4090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8908000c00 tx=0x7f8908005fd0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.136+0000 7f891504e640 1 -- 172.21.15.2:0/487604275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8910102ae0 msgr2=0x7f89101a5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:48.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.136+0000 7f891504e640 1 --2- 172.21.15.2:0/487604275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8910102ae0 0x7f89101a5d70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8900009fd0 tx=0x7f890000ef10 comp rx=0 tx=0).stop 2024-03-15T13:09:48.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.136+0000 7f891504e640 1 -- 172.21.15.2:0/487604275 shutdown_connections 2024-03-15T13:09:48.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.136+0000 7f891504e640 1 --2- 172.21.15.2:0/487604275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8910102ae0 0x7f89101a5d70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.136+0000 7f891504e640 1 --2- 172.21.15.2:0/487604275 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f88f80a1bd0 0x7f88f80a4090 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.136+0000 7f891504e640 1 --2- 172.21.15.2:0/487604275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89101056a0 0x7f89101025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.136+0000 7f891504e640 1 --2- 172.21.15.2:0/487604275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89101005d0 0x7f8910102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.136+0000 7f891504e640 1 -- 172.21.15.2:0/487604275 >> 172.21.15.2:0/487604275 conn(0x7f89100774f0 msgr2=0x7f89100689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:48.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.136+0000 7f891504e640 1 -- 172.21.15.2:0/487604275 shutdown_connections 2024-03-15T13:09:48.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.136+0000 7f891504e640 1 -- 172.21.15.2:0/487604275 wait complete. 2024-03-15T13:09:48.131 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-15T13:09:48.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.251+0000 7f511d3f4640 1 -- 172.21.15.2:0/3089796334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118100dd0 msgr2=0x7f511810d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:48.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.251+0000 7f511d3f4640 1 --2- 172.21.15.2:0/3089796334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118100dd0 0x7f511810d2e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f5110009cd0 tx=0x7f511002c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.251+0000 7f511d3f4640 1 -- 172.21.15.2:0/3089796334 shutdown_connections 2024-03-15T13:09:48.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.251+0000 7f511d3f4640 1 --2- 172.21.15.2:0/3089796334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118100dd0 0x7f511810d2e0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.251+0000 7f511d3f4640 1 --2- 172.21.15.2:0/3089796334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5118100410 0x7f5118100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.251+0000 7f511d3f4640 1 --2- 172.21.15.2:0/3089796334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51181064b0 0x7f5118106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.251+0000 7f511d3f4640 1 -- 172.21.15.2:0/3089796334 >> 172.21.15.2:0/3089796334 conn(0x7f5118077550 msgr2=0x7f5118077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:48.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.251+0000 7f511d3f4640 1 -- 172.21.15.2:0/3089796334 shutdown_connections 2024-03-15T13:09:48.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.251+0000 7f511d3f4640 1 -- 172.21.15.2:0/3089796334 wait complete. 2024-03-15T13:09:48.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.251+0000 7f511d3f4640 1 Processor -- start 2024-03-15T13:09:48.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.251+0000 7f511d3f4640 1 -- start start 2024-03-15T13:09:48.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f511d3f4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118100410 0x7f511819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:48.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f511d3f4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5118100dd0 0x7f511819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:48.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f511d3f4640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51181064b0 0x7f5118197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:48.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f511d3f4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118197fe0 con 0x7f5118100410 2024-03-15T13:09:48.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f511d3f4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118198150 con 0x7f5118100dd0 2024-03-15T13:09:48.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f511d3f4640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118198310 con 0x7f51181064b0 2024-03-15T13:09:48.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f5116ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118100410 0x7f511819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:48.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f5116ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118100410 0x7f511819d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57776/0 (socket says 172.21.15.2:57776) 2024-03-15T13:09:48.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f51167fc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5118100dd0 0x7f511819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:48.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f5116ffd640 1 -- 172.21.15.2:0/4080990213 learned_addr learned my addr 172.21.15.2:0/4080990213 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:48.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f51177fe640 1 --2- 172.21.15.2:0/4080990213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51181064b0 0x7f5118197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:48.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f5116ffd640 1 -- 172.21.15.2:0/4080990213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51181064b0 msgr2=0x7f5118197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:48.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f5116ffd640 1 --2- 172.21.15.2:0/4080990213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51181064b0 0x7f5118197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f5116ffd640 1 -- 172.21.15.2:0/4080990213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5118100dd0 msgr2=0x7f511819dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:48.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f5116ffd640 1 --2- 172.21.15.2:0/4080990213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5118100dd0 0x7f511819dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f5116ffd640 1 -- 172.21.15.2:0/4080990213 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5110009900 con 0x7f5118100410 2024-03-15T13:09:48.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f51167fc640 1 --2- 172.21.15.2:0/4080990213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5118100dd0 0x7f511819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:48.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f5116ffd640 1 --2- 172.21.15.2:0/4080990213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118100410 0x7f511819d7e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f510800e9f0 tx=0x7f510800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:48.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f50fffff640 1 -- 172.21.15.2:0/4080990213 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108009720 con 0x7f5118100410 2024-03-15T13:09:48.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.252+0000 7f50fffff640 1 -- 172.21.15.2:0/4080990213 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f510800cd80 con 0x7f5118100410 2024-03-15T13:09:48.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.253+0000 7f511d3f4640 1 -- 172.21.15.2:0/4080990213 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51181985f0 con 0x7f5118100410 2024-03-15T13:09:48.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.253+0000 7f50fffff640 1 -- 172.21.15.2:0/4080990213 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108010690 con 0x7f5118100410 2024-03-15T13:09:48.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.253+0000 7f511d3f4640 1 -- 172.21.15.2:0/4080990213 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51181a4880 con 0x7f5118100410 2024-03-15T13:09:48.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.253+0000 7f511d3f4640 1 -- 172.21.15.2:0/4080990213 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5118068730 con 0x7f5118100410 2024-03-15T13:09:48.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.254+0000 7f50fffff640 1 -- 172.21.15.2:0/4080990213 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f5108004e90 con 0x7f5118100410 2024-03-15T13:09:48.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.255+0000 7f50fffff640 1 --2- 172.21.15.2:0/4080990213 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f51000a1ba0 0x7f51000a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:48.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.255+0000 7f51167fc640 1 --2- 172.21.15.2:0/4080990213 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f51000a1ba0 0x7f51000a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:48.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.255+0000 7f50fffff640 1 -- 172.21.15.2:0/4080990213 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f5108014070 con 0x7f5118100410 2024-03-15T13:09:48.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.256+0000 7f51167fc640 1 --2- 172.21.15.2:0/4080990213 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f51000a1ba0 0x7f51000a4060 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5118077ba0 tx=0x7f510c005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:48.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.260+0000 7f50fffff640 1 -- 172.21.15.2:0/4080990213 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5108096990 con 0x7f5118100410 2024-03-15T13:09:48.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.455+0000 7f511d3f4640 1 -- 172.21.15.2:0/4080990213 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5118198ec0 con 0x7f5118100410 2024-03-15T13:09:48.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.455+0000 7f50fffff640 1 -- 172.21.15.2:0/4080990213 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v100) v1 ==== 81+0+981 (secure 0 0 0) 0x7f51080b8e60 con 0x7f5118100410 2024-03-15T13:09:48.440 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/343748862 2024-03-16T12:58:53.781001+0000 2024-03-15T13:09:48.440 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3673716509 2024-03-16T12:49:18.658825+0000 2024-03-15T13:09:48.440 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3111414481 2024-03-16T12:48:51.791072+0000 2024-03-15T13:09:48.440 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3111414481 2024-03-16T12:48:51.791072+0000 2024-03-15T13:09:48.440 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3546094985 2024-03-16T12:48:51.791072+0000 2024-03-15T13:09:48.440 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1464336909 2024-03-16T12:48:51.791072+0000 2024-03-15T13:09:48.440 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/84615744 2024-03-16T12:48:51.791072+0000 2024-03-15T13:09:48.440 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/656434628 2024-03-16T12:49:18.658825+0000 2024-03-15T13:09:48.440 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3269509343 2024-03-16T12:49:18.658825+0000 2024-03-15T13:09:48.440 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3269509343 2024-03-16T12:49:18.658825+0000 2024-03-15T13:09:48.440 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3349070582 2024-03-16T12:49:18.658825+0000 2024-03-15T13:09:48.440 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1602727020 2024-03-16T12:58:53.781001+0000 2024-03-15T13:09:48.440 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/883225361 2024-03-16T12:58:53.781001+0000 2024-03-15T13:09:48.440 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3173750072 2024-03-16T12:58:53.781001+0000 2024-03-15T13:09:48.440 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/343748862 2024-03-16T12:58:53.781001+0000 2024-03-15T13:09:48.441 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2211760175 2024-03-16T12:58:53.781001+0000 2024-03-15T13:09:48.441 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/4030567035 2024-03-16T12:58:53.781001+0000 2024-03-15T13:09:48.441 INFO:teuthology.orchestra.run.smithi002.stderr:listed 17 entries 2024-03-15T13:09:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.457+0000 7f511d3f4640 1 -- 172.21.15.2:0/4080990213 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f51000a1ba0 msgr2=0x7f51000a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.457+0000 7f511d3f4640 1 --2- 172.21.15.2:0/4080990213 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f51000a1ba0 0x7f51000a4060 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5118077ba0 tx=0x7f510c005f90 comp rx=0 tx=0).stop 2024-03-15T13:09:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.457+0000 7f511d3f4640 1 -- 172.21.15.2:0/4080990213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118100410 msgr2=0x7f511819d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.457+0000 7f511d3f4640 1 --2- 172.21.15.2:0/4080990213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118100410 0x7f511819d7e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f510800e9f0 tx=0x7f510800edf0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.457+0000 7f511d3f4640 1 -- 172.21.15.2:0/4080990213 shutdown_connections 2024-03-15T13:09:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.457+0000 7f511d3f4640 1 --2- 172.21.15.2:0/4080990213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51181064b0 0x7f5118197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.457+0000 7f511d3f4640 1 --2- 172.21.15.2:0/4080990213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5118100dd0 0x7f511819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.457+0000 7f511d3f4640 1 --2- 172.21.15.2:0/4080990213 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f51000a1ba0 0x7f51000a4060 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.457+0000 7f511d3f4640 1 --2- 172.21.15.2:0/4080990213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5118100410 0x7f511819d7e0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.457+0000 7f511d3f4640 1 -- 172.21.15.2:0/4080990213 >> 172.21.15.2:0/4080990213 conn(0x7f5118077550 msgr2=0x7f5118069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.457+0000 7f511d3f4640 1 -- 172.21.15.2:0/4080990213 shutdown_connections 2024-03-15T13:09:48.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:48.457+0000 7f511d3f4640 1 -- 172.21.15.2:0/4080990213 wait complete. 2024-03-15T13:09:48.452 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2024-03-15T13:09:48.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:09:48.452 DEBUG:teuthology.orchestra.run.smithi002:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-15T13:09:48.479 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi002.stderr:ceph-fuse[86457]: fuse finished with error 0 and tester_r 0 2024-03-15T13:09:48.479 INFO:teuthology.orchestra.run:waiting for 300 2024-03-15T13:09:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:48 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/995588301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T13:09:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:48 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/487604275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T13:09:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:48 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/4080990213' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-15T13:09:48.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:48 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/995588301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T13:09:48.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:48 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/487604275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T13:09:48.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:48 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/4080990213' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-15T13:09:49.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:48 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/995588301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-15T13:09:49.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:48 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/487604275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-15T13:09:49.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:48 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/4080990213' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-15T13:09:49.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:49 smithi149 ceph-mon[35927]: pgmap v338: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:50.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:49 smithi002 ceph-mon[32060]: pgmap v338: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:50.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:49 smithi184 ceph-mon[35993]: pgmap v338: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-15T13:09:51.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:51 smithi149 ceph-mon[35927]: pgmap v339: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-03-15T13:09:52.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:51 smithi002 ceph-mon[32060]: pgmap v339: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-03-15T13:09:52.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:51 smithi184 ceph-mon[35993]: pgmap v339: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-03-15T13:09:53.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:53 smithi149 ceph-mon[35927]: pgmap v340: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-15T13:09:54.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:53 smithi002 ceph-mon[32060]: pgmap v340: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-15T13:09:54.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:53 smithi184 ceph-mon[35993]: pgmap v340: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-15T13:09:54.581 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi002.front.sepia.ceph.com 2024-03-15T13:09:54.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:09:54.582 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-15T13:09:54.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:09:54.599 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-03-15T13:09:54.656 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-15T13:09:54.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:09:54.657 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-15T13:09:54.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-15T13:09:54.727 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2024-03-15T13:09:54.821 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-15T13:09:54.832 INFO:tasks.cephadm:Teardown begin 2024-03-15T13:09:54.833 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-15T13:09:54.833 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:09:54.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.955+0000 7fbf606c2640 1 -- 172.21.15.2:0/4253523128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58100740 msgr2=0x7fbf58100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:54.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.955+0000 7fbf606c2640 1 --2- 172.21.15.2:0/4253523128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58100740 0x7fbf58100bc0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009930 tx=0x7fbf4802bc30 comp rx=0 tx=0).stop 2024-03-15T13:09:54.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.955+0000 7fbf606c2640 1 -- 172.21.15.2:0/4253523128 shutdown_connections 2024-03-15T13:09:54.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.955+0000 7fbf606c2640 1 --2- 172.21.15.2:0/4253523128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf58101100 0x7fbf5810d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:54.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.955+0000 7fbf606c2640 1 --2- 172.21.15.2:0/4253523128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58100740 0x7fbf58100bc0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:54.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.955+0000 7fbf606c2640 1 --2- 172.21.15.2:0/4253523128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf581053d0 0x7fbf581057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:54.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.955+0000 7fbf606c2640 1 -- 172.21.15.2:0/4253523128 >> 172.21.15.2:0/4253523128 conn(0x7fbf580761f0 msgr2=0x7fbf58076600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:54.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.955+0000 7fbf606c2640 1 -- 172.21.15.2:0/4253523128 shutdown_connections 2024-03-15T13:09:54.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.955+0000 7fbf606c2640 1 -- 172.21.15.2:0/4253523128 wait complete. 2024-03-15T13:09:54.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.956+0000 7fbf606c2640 1 Processor -- start 2024-03-15T13:09:54.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.956+0000 7fbf606c2640 1 -- start start 2024-03-15T13:09:54.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.956+0000 7fbf606c2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58100740 0x7fbf5810dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:54.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.956+0000 7fbf606c2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf58101100 0x7fbf5810e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:54.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.956+0000 7fbf606c2640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf581053d0 0x7fbf5810e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:54.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.956+0000 7fbf606c2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf581a5f00 con 0x7fbf58100740 2024-03-15T13:09:54.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.956+0000 7fbf606c2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf581a6070 con 0x7fbf58101100 2024-03-15T13:09:54.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.956+0000 7fbf606c2640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf581a6230 con 0x7fbf581053d0 2024-03-15T13:09:54.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.956+0000 7fbf5e45e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58100740 0x7fbf5810dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:54.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.956+0000 7fbf5e45e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58100740 0x7fbf5810dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57788/0 (socket says 172.21.15.2:57788) 2024-03-15T13:09:54.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.956+0000 7fbf5e45e640 1 -- 172.21.15.2:0/2855400766 learned_addr learned my addr 172.21.15.2:0/2855400766 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:54.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.957+0000 7fbf5dc5d640 1 --2- 172.21.15.2:0/2855400766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf58101100 0x7fbf5810e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:54.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.957+0000 7fbf5ec5f640 1 --2- 172.21.15.2:0/2855400766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf581053d0 0x7fbf5810e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:54.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.957+0000 7fbf5e45e640 1 -- 172.21.15.2:0/2855400766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf581053d0 msgr2=0x7fbf5810e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:54.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.957+0000 7fbf5e45e640 1 --2- 172.21.15.2:0/2855400766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf581053d0 0x7fbf5810e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:54.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.957+0000 7fbf5e45e640 1 -- 172.21.15.2:0/2855400766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf58101100 msgr2=0x7fbf5810e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:54.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.957+0000 7fbf5e45e640 1 --2- 172.21.15.2:0/2855400766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf58101100 0x7fbf5810e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:54.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.957+0000 7fbf5e45e640 1 -- 172.21.15.2:0/2855400766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf48009560 con 0x7fbf58100740 2024-03-15T13:09:54.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.957+0000 7fbf5e45e640 1 --2- 172.21.15.2:0/2855400766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58100740 0x7fbf5810dbf0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400e9f0 tx=0x7fbf5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:54.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.957+0000 7fbf4f7fe640 1 -- 172.21.15.2:0/2855400766 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54009700 con 0x7fbf58100740 2024-03-15T13:09:54.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.957+0000 7fbf4f7fe640 1 -- 172.21.15.2:0/2855400766 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fbf5400cd80 con 0x7fbf58100740 2024-03-15T13:09:54.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.957+0000 7fbf606c2640 1 -- 172.21.15.2:0/2855400766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf581a6510 con 0x7fbf58100740 2024-03-15T13:09:54.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.957+0000 7fbf4f7fe640 1 -- 172.21.15.2:0/2855400766 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54018920 con 0x7fbf58100740 2024-03-15T13:09:54.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.957+0000 7fbf606c2640 1 -- 172.21.15.2:0/2855400766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf581a69f0 con 0x7fbf58100740 2024-03-15T13:09:54.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.958+0000 7fbf606c2640 1 -- 172.21.15.2:0/2855400766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf58101b10 con 0x7fbf58100740 2024-03-15T13:09:54.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.960+0000 7fbf4f7fe640 1 -- 172.21.15.2:0/2855400766 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fbf54018a80 con 0x7fbf58100740 2024-03-15T13:09:54.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.960+0000 7fbf4f7fe640 1 --2- 172.21.15.2:0/2855400766 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbf440a1ba0 0x7fbf440a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:54.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.960+0000 7fbf4f7fe640 1 -- 172.21.15.2:0/2855400766 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fbf54014070 con 0x7fbf58100740 2024-03-15T13:09:54.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.960+0000 7fbf5dc5d640 1 --2- 172.21.15.2:0/2855400766 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbf440a1ba0 0x7fbf440a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:54.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.961+0000 7fbf5dc5d640 1 --2- 172.21.15.2:0/2855400766 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbf440a1ba0 0x7fbf440a4060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbf48000c00 tx=0x7fbf48005fd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:54.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:54.964+0000 7fbf4f7fe640 1 -- 172.21.15.2:0/2855400766 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbf54096ed0 con 0x7fbf58100740 2024-03-15T13:09:55.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.187+0000 7fbf606c2640 1 -- 172.21.15.2:0/2855400766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf58108870 con 0x7fbf58100740 2024-03-15T13:09:55.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.188+0000 7fbf4f7fe640 1 -- 172.21.15.2:0/2855400766 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+11453 (secure 0 0 0) 0x7fbf5409a590 con 0x7fbf58100740 2024-03-15T13:09:55.172 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:09:55.173 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:05:12.138845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24500,24464,24460,24434,24539]},"id":1}]} 2024-03-15T13:09:55.173 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 50 2024-03-15T13:09:55.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.190+0000 7fbf606c2640 1 -- 172.21.15.2:0/2855400766 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbf440a1ba0 msgr2=0x7fbf440a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:55.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.190+0000 7fbf606c2640 1 --2- 172.21.15.2:0/2855400766 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbf440a1ba0 0x7fbf440a4060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbf48000c00 tx=0x7fbf48005fd0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.190+0000 7fbf606c2640 1 -- 172.21.15.2:0/2855400766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58100740 msgr2=0x7fbf5810dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:55.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.190+0000 7fbf606c2640 1 --2- 172.21.15.2:0/2855400766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58100740 0x7fbf5810dbf0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400e9f0 tx=0x7fbf5400eeb0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.190+0000 7fbf606c2640 1 -- 172.21.15.2:0/2855400766 shutdown_connections 2024-03-15T13:09:55.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.190+0000 7fbf606c2640 1 --2- 172.21.15.2:0/2855400766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf581053d0 0x7fbf5810e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.190+0000 7fbf606c2640 1 --2- 172.21.15.2:0/2855400766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf58101100 0x7fbf5810e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.190+0000 7fbf606c2640 1 --2- 172.21.15.2:0/2855400766 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fbf440a1ba0 0x7fbf440a4060 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.190+0000 7fbf606c2640 1 --2- 172.21.15.2:0/2855400766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58100740 0x7fbf5810dbf0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.190+0000 7fbf606c2640 1 -- 172.21.15.2:0/2855400766 >> 172.21.15.2:0/2855400766 conn(0x7fbf580761f0 msgr2=0x7fbf580fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:55.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.190+0000 7fbf606c2640 1 -- 172.21.15.2:0/2855400766 shutdown_connections 2024-03-15T13:09:55.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.190+0000 7fbf606c2640 1 -- 172.21.15.2:0/2855400766 wait complete. 2024-03-15T13:09:55.186 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-15T13:09:55.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.305+0000 7f29e296c640 1 -- 172.21.15.2:0/1493143403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29dc102780 msgr2=0x7f29dc102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:55.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.305+0000 7f29e296c640 1 --2- 172.21.15.2:0/1493143403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29dc102780 0x7f29dc102b60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009860 tx=0x7f29d002bc00 comp rx=0 tx=0).stop 2024-03-15T13:09:55.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.305+0000 7f29e296c640 1 -- 172.21.15.2:0/1493143403 shutdown_connections 2024-03-15T13:09:55.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.305+0000 7f29e296c640 1 --2- 172.21.15.2:0/1493143403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29dc10bb50 0x7f29dc10dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.305+0000 7f29e296c640 1 --2- 172.21.15.2:0/1493143403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29dc1030a0 0x7f29dc10b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.305+0000 7f29e296c640 1 --2- 172.21.15.2:0/1493143403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29dc102780 0x7f29dc102b60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.305+0000 7f29e296c640 1 -- 172.21.15.2:0/1493143403 >> 172.21.15.2:0/1493143403 conn(0x7f29dc077550 msgr2=0x7f29dc077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:55.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.305+0000 7f29e296c640 1 -- 172.21.15.2:0/1493143403 shutdown_connections 2024-03-15T13:09:55.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.305+0000 7f29e296c640 1 -- 172.21.15.2:0/1493143403 wait complete. 2024-03-15T13:09:55.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.305+0000 7f29e296c640 1 Processor -- start 2024-03-15T13:09:55.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.305+0000 7f29e296c640 1 -- start start 2024-03-15T13:09:55.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29e296c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29dc102780 0x7f29dc19d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29e296c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29dc1030a0 0x7f29dc19dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29e296c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29dc10bb50 0x7f29dc1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29e296c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc198020 con 0x7f29dc10bb50 2024-03-15T13:09:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29e296c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc198190 con 0x7f29dc1030a0 2024-03-15T13:09:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29e296c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc198350 con 0x7f29dc102780 2024-03-15T13:09:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29e0f09640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29dc10bb50 0x7f29dc1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29e0f09640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29dc10bb50 0x7f29dc1979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57806/0 (socket says 172.21.15.2:57806) 2024-03-15T13:09:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29e0f09640 1 -- 172.21.15.2:0/1660270095 learned_addr learned my addr 172.21.15.2:0/1660270095 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29dbfff640 1 --2- 172.21.15.2:0/1660270095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29dc102780 0x7f29dc19d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29db7fe640 1 --2- 172.21.15.2:0/1660270095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29dc1030a0 0x7f29dc19dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:55.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29e0f09640 1 -- 172.21.15.2:0/1660270095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29dc102780 msgr2=0x7f29dc19d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:55.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29e0f09640 1 --2- 172.21.15.2:0/1660270095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29dc102780 0x7f29dc19d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29e0f09640 1 -- 172.21.15.2:0/1660270095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29dc1030a0 msgr2=0x7f29dc19dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:55.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29e0f09640 1 --2- 172.21.15.2:0/1660270095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29dc1030a0 0x7f29dc19dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29e0f09640 1 -- 172.21.15.2:0/1660270095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d4009900 con 0x7f29dc10bb50 2024-03-15T13:09:55.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29dbfff640 1 --2- 172.21.15.2:0/1660270095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29dc102780 0x7f29dc19d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-15T13:09:55.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29e0f09640 1 --2- 172.21.15.2:0/1660270095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29dc10bb50 0x7f29dc1979d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f29d400efc0 tx=0x7f29d400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:55.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.306+0000 7f29d97fa640 1 -- 172.21.15.2:0/1660270095 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4003970 con 0x7f29dc10bb50 2024-03-15T13:09:55.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.307+0000 7f29e296c640 1 -- 172.21.15.2:0/1660270095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d0009490 con 0x7f29dc10bb50 2024-03-15T13:09:55.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.307+0000 7f29d97fa640 1 -- 172.21.15.2:0/1660270095 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f29d4004020 con 0x7f29dc10bb50 2024-03-15T13:09:55.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.307+0000 7f29d97fa640 1 -- 172.21.15.2:0/1660270095 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d400b9b0 con 0x7f29dc10bb50 2024-03-15T13:09:55.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.307+0000 7f29e296c640 1 -- 172.21.15.2:0/1660270095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc1a46e0 con 0x7f29dc10bb50 2024-03-15T13:09:55.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.308+0000 7f29e296c640 1 -- 172.21.15.2:0/1660270095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29dc068730 con 0x7f29dc10bb50 2024-03-15T13:09:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.309+0000 7f29d97fa640 1 -- 172.21.15.2:0/1660270095 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f29d400a040 con 0x7f29dc10bb50 2024-03-15T13:09:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.309+0000 7f29d97fa640 1 --2- 172.21.15.2:0/1660270095 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f29c40a1ba0 0x7f29c40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.309+0000 7f29dbfff640 1 --2- 172.21.15.2:0/1660270095 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f29c40a1ba0 0x7f29c40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.309+0000 7f29d97fa640 1 -- 172.21.15.2:0/1660270095 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f29d40cdaf0 con 0x7f29dc10bb50 2024-03-15T13:09:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.310+0000 7f29dbfff640 1 --2- 172.21.15.2:0/1660270095 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f29c40a1ba0 0x7f29c40a4060 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009830 tx=0x7f29d00058d0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:55.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.314+0000 7f29d97fa640 1 -- 172.21.15.2:0/1660270095 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f29d4096620 con 0x7f29dc10bb50 2024-03-15T13:09:55.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.532+0000 7f29e296c640 1 -- 172.21.15.2:0/1660270095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29dc106c40 con 0x7f29dc10bb50 2024-03-15T13:09:55.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.533+0000 7f29d97fa640 1 -- 172.21.15.2:0/1660270095 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+11453 (secure 0 0 0) 0x7f29d4099ce0 con 0x7f29dc10bb50 2024-03-15T13:09:55.517 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:09:55.518 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/245353007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":245353007},{"type":"v1","addr":"172.21.15.2:6835","nonce":245353007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1406254318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":1406254318},{"type":"v1","addr":"172.21.15.2:6837","nonce":1406254318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-15T12:56:20.559317+0000","modified":"2024-03-15T13:05:12.138845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24539,"mds_1":24500,"mds_2":24464,"mds_3":24460,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6839/2513297233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":2513297233},{"type":"v1","addr":"172.21.15.2:6839","nonce":2513297233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.2:6841/3747539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3747539896},{"type":"v1","addr":"172.21.15.2:6841","nonce":3747539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/730881833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":730881833},{"type":"v1","addr":"172.21.15.184:6833","nonce":730881833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24434":{"gid":24434,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.149:6833/2587208478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2587208478},{"type":"v1","addr":"172.21.15.149:6833","nonce":2587208478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.184:6835/1255256953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":1255256953},{"type":"v1","addr":"172.21.15.184:6835","nonce":1255256953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.149:6835/694548319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":694548319},{"type":"v1","addr":"172.21.15.149:6835","nonce":694548319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24472":{"gid":24472,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3698954880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3698954880},{"type":"v1","addr":"172.21.15.184:6837","nonce":3698954880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6839/1170793322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":1170793322},{"type":"v1","addr":"172.21.15.184:6839","nonce":1170793322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6837/1813851428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1813851428},{"type":"v1","addr":"172.21.15.149:6837","nonce":1813851428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24539":{"gid":24539,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/1715135128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1715135128},{"type":"v1","addr":"172.21.15.149:6839","nonce":1715135128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24539,"qdb_cluster":[24500,24464,24460,24434,24539]},"id":1}]} 2024-03-15T13:09:55.518 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 50 2024-03-15T13:09:55.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.535+0000 7f29e296c640 1 -- 172.21.15.2:0/1660270095 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f29c40a1ba0 msgr2=0x7f29c40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:55.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.535+0000 7f29e296c640 1 --2- 172.21.15.2:0/1660270095 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f29c40a1ba0 0x7f29c40a4060 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009830 tx=0x7f29d00058d0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.535+0000 7f29e296c640 1 -- 172.21.15.2:0/1660270095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29dc10bb50 msgr2=0x7f29dc1979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:55.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.535+0000 7f29e296c640 1 --2- 172.21.15.2:0/1660270095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29dc10bb50 0x7f29dc1979d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f29d400efc0 tx=0x7f29d400c800 comp rx=0 tx=0).stop 2024-03-15T13:09:55.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.535+0000 7f29e296c640 1 -- 172.21.15.2:0/1660270095 shutdown_connections 2024-03-15T13:09:55.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.535+0000 7f29e296c640 1 --2- 172.21.15.2:0/1660270095 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f29c40a1ba0 0x7f29c40a4060 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.535+0000 7f29e296c640 1 --2- 172.21.15.2:0/1660270095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29dc10bb50 0x7f29dc1979d0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.535+0000 7f29e296c640 1 --2- 172.21.15.2:0/1660270095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29dc1030a0 0x7f29dc19dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.535+0000 7f29e296c640 1 --2- 172.21.15.2:0/1660270095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29dc102780 0x7f29dc19d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.535+0000 7f29e296c640 1 -- 172.21.15.2:0/1660270095 >> 172.21.15.2:0/1660270095 conn(0x7f29dc077550 msgr2=0x7f29dc0fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:55.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.535+0000 7f29e296c640 1 -- 172.21.15.2:0/1660270095 shutdown_connections 2024-03-15T13:09:55.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.535+0000 7f29e296c640 1 -- 172.21.15.2:0/1660270095 wait complete. 2024-03-15T13:09:55.531 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-15T13:09:55.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.651+0000 7f3c71355640 1 -- 172.21.15.2:0/2480024966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c106880 msgr2=0x7f3c6c10e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:55.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.651+0000 7f3c71355640 1 --2- 172.21.15.2:0/2480024966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c106880 0x7f3c6c10e400 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009ca0 tx=0x7f3c6402c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.651+0000 7f3c71355640 1 -- 172.21.15.2:0/2480024966 shutdown_connections 2024-03-15T13:09:55.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.651+0000 7f3c71355640 1 --2- 172.21.15.2:0/2480024966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c106880 0x7f3c6c10e400 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.651+0000 7f3c71355640 1 --2- 172.21.15.2:0/2480024966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c6c06a7e0 0x7f3c6c106150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.651+0000 7f3c71355640 1 --2- 172.21.15.2:0/2480024966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c069e30 0x7f3c6c06a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.651+0000 7f3c71355640 1 -- 172.21.15.2:0/2480024966 >> 172.21.15.2:0/2480024966 conn(0x7f3c6c077490 msgr2=0x7f3c6c0778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:55.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.651+0000 7f3c71355640 1 -- 172.21.15.2:0/2480024966 shutdown_connections 2024-03-15T13:09:55.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.651+0000 7f3c71355640 1 -- 172.21.15.2:0/2480024966 wait complete. 2024-03-15T13:09:55.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.652+0000 7f3c71355640 1 Processor -- start 2024-03-15T13:09:55.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.652+0000 7f3c71355640 1 -- start start 2024-03-15T13:09:55.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.652+0000 7f3c71355640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c069e30 0x7f3c6c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:55.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.652+0000 7f3c71355640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c106880 0x7f3c6c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:55.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.652+0000 7f3c71355640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c6c1978c0 0x7f3c6c197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:55.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.652+0000 7f3c71355640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c1982b0 con 0x7f3c6c106880 2024-03-15T13:09:55.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.652+0000 7f3c71355640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c198420 con 0x7f3c6c069e30 2024-03-15T13:09:55.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.652+0000 7f3c71355640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c1985e0 con 0x7f3c6c1978c0 2024-03-15T13:09:55.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.652+0000 7f3c627fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c106880 0x7f3c6c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:55.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.652+0000 7f3c6affd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c069e30 0x7f3c6c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:55.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.652+0000 7f3c627fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c106880 0x7f3c6c19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57832/0 (socket says 172.21.15.2:57832) 2024-03-15T13:09:55.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.652+0000 7f3c627fc640 1 -- 172.21.15.2:0/374731139 learned_addr learned my addr 172.21.15.2:0/374731139 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:55.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.652+0000 7f3c6b7fe640 1 --2- 172.21.15.2:0/374731139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c6c1978c0 0x7f3c6c197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:55.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.653+0000 7f3c627fc640 1 -- 172.21.15.2:0/374731139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c6c1978c0 msgr2=0x7f3c6c197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:55.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.653+0000 7f3c627fc640 1 --2- 172.21.15.2:0/374731139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c6c1978c0 0x7f3c6c197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.653+0000 7f3c627fc640 1 -- 172.21.15.2:0/374731139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c069e30 msgr2=0x7f3c6c19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:55.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.653+0000 7f3c627fc640 1 --2- 172.21.15.2:0/374731139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c069e30 0x7f3c6c19d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.653+0000 7f3c627fc640 1 -- 172.21.15.2:0/374731139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c64009900 con 0x7f3c6c106880 2024-03-15T13:09:55.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.653+0000 7f3c627fc640 1 --2- 172.21.15.2:0/374731139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c106880 0x7f3c6c19dc00 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400e9f0 tx=0x7f3c5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:55.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.653+0000 7f3c68ff9640 1 -- 172.21.15.2:0/374731139 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c540096c0 con 0x7f3c6c106880 2024-03-15T13:09:55.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.653+0000 7f3c71355640 1 -- 172.21.15.2:0/374731139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6c1a4580 con 0x7f3c6c106880 2024-03-15T13:09:55.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.653+0000 7f3c68ff9640 1 -- 172.21.15.2:0/374731139 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f3c540054f0 con 0x7f3c6c106880 2024-03-15T13:09:55.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.653+0000 7f3c68ff9640 1 -- 172.21.15.2:0/374731139 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54010430 con 0x7f3c6c106880 2024-03-15T13:09:55.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.653+0000 7f3c71355640 1 -- 172.21.15.2:0/374731139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6c1a4a40 con 0x7f3c6c106880 2024-03-15T13:09:55.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.654+0000 7f3c71355640 1 -- 172.21.15.2:0/374731139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6c068730 con 0x7f3c6c106880 2024-03-15T13:09:55.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.655+0000 7f3c68ff9640 1 -- 172.21.15.2:0/374731139 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f3c54005000 con 0x7f3c6c106880 2024-03-15T13:09:55.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.656+0000 7f3c68ff9640 1 --2- 172.21.15.2:0/374731139 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3c580a1ba0 0x7f3c580a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:55.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.656+0000 7f3c6affd640 1 --2- 172.21.15.2:0/374731139 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3c580a1ba0 0x7f3c580a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:55.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.656+0000 7f3c68ff9640 1 -- 172.21.15.2:0/374731139 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7f3c54014070 con 0x7f3c6c106880 2024-03-15T13:09:55.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.656+0000 7f3c6affd640 1 --2- 172.21.15.2:0/374731139 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3c580a1ba0 0x7f3c580a4060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c106350 tx=0x7f3c5c005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:55.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.660+0000 7f3c68ff9640 1 -- 172.21.15.2:0/374731139 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3c54096c40 con 0x7f3c6c106880 2024-03-15T13:09:55.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:55 smithi002 ceph-mon[32060]: pgmap v341: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-15T13:09:55.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:55 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2855400766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:09:55.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:55 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/1660270095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:09:55.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.842+0000 7f3c71355640 1 -- 172.21.15.2:0/374731139 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c6c198da0 con 0x7f3c6c106880 2024-03-15T13:09:55.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.844+0000 7f3c68ff9640 1 -- 172.21.15.2:0/374731139 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+25138 (secure 0 0 0) 0x7f3c5409a300 con 0x7f3c6c106880 2024-03-15T13:09:55.828 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-15T13:09:55.829 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":100,"fsid":"2767718a-e2ca-11ee-95c9-87774f69a715","created":"2024-03-15T12:48:20.095922+0000","modified":"2024-03-15T13:09:15.557407+0000","last_up_change":"2024-03-15T12:55:30.130291+0000","last_in_change":"2024-03-15T12:55:10.382179+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-15T12:51:47.004843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-15T12:56:18.559882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-15T12:56:19.751767+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-15T12:56:23.664879+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f9c5bb62-482c-45e1-b63b-b487ca609054","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6803","nonce":2123099355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6805","nonce":2123099355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6809","nonce":2123099355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2123099355},{"type":"v1","addr":"172.21.15.2:6807","nonce":2123099355}]},"public_addr":"172.21.15.2:6803/2123099355","cluster_addr":"172.21.15.2:6805/2123099355","heartbeat_back_addr":"172.21.15.2:6809/2123099355","heartbeat_front_addr":"172.21.15.2:6807/2123099355","state":["exists","up"]},{"osd":1,"uuid":"5182e55d-f2cf-40be-acff-0534c2bce7ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6801","nonce":3295500987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6803","nonce":3295500987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6807","nonce":3295500987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3295500987},{"type":"v1","addr":"172.21.15.149:6805","nonce":3295500987}]},"public_addr":"172.21.15.149:6801/3295500987","cluster_addr":"172.21.15.149:6803/3295500987","heartbeat_back_addr":"172.21.15.149:6807/3295500987","heartbeat_front_addr":"172.21.15.149:6805/3295500987","state":["exists","up"]},{"osd":2,"uuid":"8db2a1c3-c04d-4b78-ad1c-e587fad5ce88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6801","nonce":1645410344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6803","nonce":1645410344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6807","nonce":1645410344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":1645410344},{"type":"v1","addr":"172.21.15.184:6805","nonce":1645410344}]},"public_addr":"172.21.15.184:6801/1645410344","cluster_addr":"172.21.15.184:6803/1645410344","heartbeat_back_addr":"172.21.15.184:6807/1645410344","heartbeat_front_addr":"172.21.15.184:6805/1645410344","state":["exists","up"]},{"osd":3,"uuid":"b2438d20-597b-4510-8cb7-3f0ce89067a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6811","nonce":12565302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6813","nonce":12565302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6817","nonce":12565302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":12565302},{"type":"v1","addr":"172.21.15.2:6815","nonce":12565302}]},"public_addr":"172.21.15.2:6811/12565302","cluster_addr":"172.21.15.2:6813/12565302","heartbeat_back_addr":"172.21.15.2:6817/12565302","heartbeat_front_addr":"172.21.15.2:6815/12565302","state":["exists","up"]},{"osd":4,"uuid":"65f6e3fc-1a8e-4180-9f3f-7a81b98de439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6809","nonce":1790191535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6811","nonce":1790191535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6815","nonce":1790191535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1790191535},{"type":"v1","addr":"172.21.15.149:6813","nonce":1790191535}]},"public_addr":"172.21.15.149:6809/1790191535","cluster_addr":"172.21.15.149:6811/1790191535","heartbeat_back_addr":"172.21.15.149:6815/1790191535","heartbeat_front_addr":"172.21.15.149:6813/1790191535","state":["exists","up"]},{"osd":5,"uuid":"e43c485c-25d6-41d5-8b9b-e22e4cc46727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6809","nonce":1659458624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6811","nonce":1659458624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6815","nonce":1659458624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":1659458624},{"type":"v1","addr":"172.21.15.184:6813","nonce":1659458624}]},"public_addr":"172.21.15.184:6809/1659458624","cluster_addr":"172.21.15.184:6811/1659458624","heartbeat_back_addr":"172.21.15.184:6815/1659458624","heartbeat_front_addr":"172.21.15.184:6813/1659458624","state":["exists","up"]},{"osd":6,"uuid":"0966e5aa-2b44-4517-a8ea-a4e3f8ac4b27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6819","nonce":4103461638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6821","nonce":4103461638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6825","nonce":4103461638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":4103461638},{"type":"v1","addr":"172.21.15.2:6823","nonce":4103461638}]},"public_addr":"172.21.15.2:6819/4103461638","cluster_addr":"172.21.15.2:6821/4103461638","heartbeat_back_addr":"172.21.15.2:6825/4103461638","heartbeat_front_addr":"172.21.15.2:6823/4103461638","state":["exists","up"]},{"osd":7,"uuid":"997ac305-e8f7-4ae6-b9ea-a491dfad807a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6817","nonce":4241660371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6819","nonce":4241660371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6823","nonce":4241660371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":4241660371},{"type":"v1","addr":"172.21.15.149:6821","nonce":4241660371}]},"public_addr":"172.21.15.149:6817/4241660371","cluster_addr":"172.21.15.149:6819/4241660371","heartbeat_back_addr":"172.21.15.149:6823/4241660371","heartbeat_front_addr":"172.21.15.149:6821/4241660371","state":["exists","up"]},{"osd":8,"uuid":"99a26e37-2f65-4775-80d0-e8388462928f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6817","nonce":2462722579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6819","nonce":2462722579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6823","nonce":2462722579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":2462722579},{"type":"v1","addr":"172.21.15.184:6821","nonce":2462722579}]},"public_addr":"172.21.15.184:6817/2462722579","cluster_addr":"172.21.15.184:6819/2462722579","heartbeat_back_addr":"172.21.15.184:6823/2462722579","heartbeat_front_addr":"172.21.15.184:6821/2462722579","state":["exists","up"]},{"osd":9,"uuid":"8b52d0d0-43fa-41ac-a775-722ff5480e45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6827","nonce":49148413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6829","nonce":49148413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6833","nonce":49148413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":49148413},{"type":"v1","addr":"172.21.15.2:6831","nonce":49148413}]},"public_addr":"172.21.15.2:6827/49148413","cluster_addr":"172.21.15.2:6829/49148413","heartbeat_back_addr":"172.21.15.2:6833/49148413","heartbeat_front_addr":"172.21.15.2:6831/49148413","state":["exists","up"]},{"osd":10,"uuid":"792088a5-75ff-44b2-b1b0-09a7a0b84554","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6825","nonce":432128202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6827","nonce":432128202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6831","nonce":432128202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":432128202},{"type":"v1","addr":"172.21.15.149:6829","nonce":432128202}]},"public_addr":"172.21.15.149:6825/432128202","cluster_addr":"172.21.15.149:6827/432128202","heartbeat_back_addr":"172.21.15.149:6831/432128202","heartbeat_front_addr":"172.21.15.149:6829/432128202","state":["exists","up"]},{"osd":11,"uuid":"5a9260e1-292e-4694-863b-b6710b18feb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6825","nonce":1184158252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6827","nonce":1184158252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6831","nonce":1184158252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1184158252},{"type":"v1","addr":"172.21.15.184:6829","nonce":1184158252}]},"public_addr":"172.21.15.184:6825/1184158252","cluster_addr":"172.21.15.184:6827/1184158252","heartbeat_back_addr":"172.21.15.184:6831/1184158252","heartbeat_front_addr":"172.21.15.184:6829/1184158252","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:50:55.024545+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:18.678508+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:51:42.498622+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:06.823507+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:31.180662+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:52:56.110011+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:21.527769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:53:46.658402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:11.701297+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:54:37.755848+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:02.973188+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-15T12:55:28.379430+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:6801/343748862":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/3673716509":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:6800/3111414481":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/3546094985":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/1464336909":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/84615744":"2024-03-16T12:48:51.791072+0000","172.21.15.2:0/656434628":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6800/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:6801/3269509343":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/3349070582":"2024-03-16T12:49:18.658825+0000","172.21.15.2:0/1602727020":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/883225361":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/3173750072":"2024-03-16T12:58:53.781001+0000","172.21.15.2:6800/343748862":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/2211760175":"2024-03-16T12:58:53.781001+0000","172.21.15.2:0/4030567035":"2024-03-16T12:58:53.781001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-15T13:09:55.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.846+0000 7f3c71355640 1 -- 172.21.15.2:0/374731139 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3c580a1ba0 msgr2=0x7f3c580a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:55.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.846+0000 7f3c71355640 1 --2- 172.21.15.2:0/374731139 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3c580a1ba0 0x7f3c580a4060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c106350 tx=0x7f3c5c005f90 comp rx=0 tx=0).stop 2024-03-15T13:09:55.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.846+0000 7f3c71355640 1 -- 172.21.15.2:0/374731139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c106880 msgr2=0x7f3c6c19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:55.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.846+0000 7f3c71355640 1 --2- 172.21.15.2:0/374731139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c106880 0x7f3c6c19dc00 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400e9f0 tx=0x7f3c5400eeb0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.846+0000 7f3c71355640 1 -- 172.21.15.2:0/374731139 shutdown_connections 2024-03-15T13:09:55.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.846+0000 7f3c71355640 1 --2- 172.21.15.2:0/374731139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c6c1978c0 0x7f3c6c197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.846+0000 7f3c71355640 1 --2- 172.21.15.2:0/374731139 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f3c580a1ba0 0x7f3c580a4060 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.846+0000 7f3c71355640 1 --2- 172.21.15.2:0/374731139 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c6c106880 0x7f3c6c19dc00 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.846+0000 7f3c71355640 1 --2- 172.21.15.2:0/374731139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c069e30 0x7f3c6c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.846+0000 7f3c71355640 1 -- 172.21.15.2:0/374731139 >> 172.21.15.2:0/374731139 conn(0x7f3c6c077490 msgr2=0x7f3c6c105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:55.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.846+0000 7f3c71355640 1 -- 172.21.15.2:0/374731139 shutdown_connections 2024-03-15T13:09:55.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.846+0000 7f3c71355640 1 -- 172.21.15.2:0/374731139 wait complete. 2024-03-15T13:09:55.845 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-15T13:09:55.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:55 smithi149 ceph-mon[35927]: pgmap v341: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-15T13:09:55.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:55 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2855400766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:09:55.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:55 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/1660270095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:09:55.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.964+0000 7fdcfc977640 1 -- 172.21.15.2:0/96813376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcf81056a0 msgr2=0x7fdcf810d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:55.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.964+0000 7fdcfc977640 1 --2- 172.21.15.2:0/96813376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcf81056a0 0x7fdcf810d270 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009cd0 tx=0x7fdce802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.964+0000 7fdcfc977640 1 -- 172.21.15.2:0/96813376 shutdown_connections 2024-03-15T13:09:55.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.964+0000 7fdcfc977640 1 --2- 172.21.15.2:0/96813376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcf81056a0 0x7fdcf810d270 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.964+0000 7fdcfc977640 1 --2- 172.21.15.2:0/96813376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcf8100f80 0x7fdcf8104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.964+0000 7fdcfc977640 1 --2- 172.21.15.2:0/96813376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf81005d0 0x7fdcf81009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.964+0000 7fdcfc977640 1 -- 172.21.15.2:0/96813376 >> 172.21.15.2:0/96813376 conn(0x7fdcf80774f0 msgr2=0x7fdcf8077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:55.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.964+0000 7fdcfc977640 1 -- 172.21.15.2:0/96813376 shutdown_connections 2024-03-15T13:09:55.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.964+0000 7fdcfc977640 1 -- 172.21.15.2:0/96813376 wait complete. 2024-03-15T13:09:55.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcfc977640 1 Processor -- start 2024-03-15T13:09:55.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcfc977640 1 -- start start 2024-03-15T13:09:55.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcfc977640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcf81005d0 0x7fdcf819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:55.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcfc977640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf81056a0 0x7fdcf819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:55.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcfc977640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcf8196810 0x7fdcf8196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:55.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcfc977640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf81972c0 con 0x7fdcf81005d0 2024-03-15T13:09:55.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcfc977640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf8197430 con 0x7fdcf81056a0 2024-03-15T13:09:55.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcfc977640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf81975a0 con 0x7fdcf8196810 2024-03-15T13:09:55.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcf659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcf81005d0 0x7fdcf819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:55.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcf659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcf81005d0 0x7fdcf819c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57858/0 (socket says 172.21.15.2:57858) 2024-03-15T13:09:55.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcf659c640 1 -- 172.21.15.2:0/2882781436 learned_addr learned my addr 172.21.15.2:0/2882781436 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:55.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcf5d9b640 1 --2- 172.21.15.2:0/2882781436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf81056a0 0x7fdcf819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:55.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcf6d9d640 1 --2- 172.21.15.2:0/2882781436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcf8196810 0x7fdcf8196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:55.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcf659c640 1 -- 172.21.15.2:0/2882781436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcf8196810 msgr2=0x7fdcf8196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:55.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcf659c640 1 --2- 172.21.15.2:0/2882781436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcf8196810 0x7fdcf8196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcf659c640 1 -- 172.21.15.2:0/2882781436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf81056a0 msgr2=0x7fdcf819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:55.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcf659c640 1 --2- 172.21.15.2:0/2882781436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf81056a0 0x7fdcf819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:55.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.965+0000 7fdcf659c640 1 -- 172.21.15.2:0/2882781436 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce8009900 con 0x7fdcf81005d0 2024-03-15T13:09:55.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.966+0000 7fdcf659c640 1 --2- 172.21.15.2:0/2882781436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcf81005d0 0x7fdcf819c6a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000e9f0 tx=0x7fdcf000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:55.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.966+0000 7fdcef7fe640 1 -- 172.21.15.2:0/2882781436 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0009700 con 0x7fdcf81005d0 2024-03-15T13:09:55.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.966+0000 7fdcef7fe640 1 -- 172.21.15.2:0/2882781436 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fdcf000cd80 con 0x7fdcf81005d0 2024-03-15T13:09:55.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.966+0000 7fdcef7fe640 1 -- 172.21.15.2:0/2882781436 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0018920 con 0x7fdcf81005d0 2024-03-15T13:09:55.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.966+0000 7fdcfc977640 1 -- 172.21.15.2:0/2882781436 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf81a3550 con 0x7fdcf81005d0 2024-03-15T13:09:55.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.966+0000 7fdcfc977640 1 -- 172.21.15.2:0/2882781436 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf81a39e0 con 0x7fdcf81005d0 2024-03-15T13:09:55.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.967+0000 7fdcfc977640 1 -- 172.21.15.2:0/2882781436 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf8101aa0 con 0x7fdcf81005d0 2024-03-15T13:09:55.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.968+0000 7fdcef7fe640 1 -- 172.21.15.2:0/2882781436 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fdcf0018a80 con 0x7fdcf81005d0 2024-03-15T13:09:55.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.968+0000 7fdcef7fe640 1 --2- 172.21.15.2:0/2882781436 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdce40a1c20 0x7fdce40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:55.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.968+0000 7fdcef7fe640 1 -- 172.21.15.2:0/2882781436 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9452+0+0 (secure 0 0 0) 0x7fdcf0014070 con 0x7fdcf81005d0 2024-03-15T13:09:55.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.968+0000 7fdcf5d9b640 1 --2- 172.21.15.2:0/2882781436 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdce40a1c20 0x7fdce40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:55.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.969+0000 7fdcf5d9b640 1 --2- 172.21.15.2:0/2882781436 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdce40a1c20 0x7fdce40a40e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80ffe90 tx=0x7fdce0005f90 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:55.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:55.973+0000 7fdcef7fe640 1 -- 172.21.15.2:0/2882781436 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdcf0053280 con 0x7fdcf81005d0 2024-03-15T13:09:56.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:55 smithi184 ceph-mon[35993]: pgmap v341: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-15T13:09:56.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:55 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2855400766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:09:56.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:55 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/1660270095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-15T13:09:56.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.196+0000 7fdcfc977640 1 -- 172.21.15.2:0/2882781436 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdcf8108930 con 0x7fdcf81005d0 2024-03-15T13:09:56.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.198+0000 7fdcef7fe640 1 -- 172.21.15.2:0/2882781436 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdcf0096a20 con 0x7fdcf81005d0 2024-03-15T13:09:56.182 INFO:teuthology.orchestra.run.smithi002.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-15T13:09:56.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.202+0000 7fdcfc977640 1 -- 172.21.15.2:0/2882781436 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdce40a1c20 msgr2=0x7fdce40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:56.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.202+0000 7fdcfc977640 1 --2- 172.21.15.2:0/2882781436 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdce40a1c20 0x7fdce40a40e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80ffe90 tx=0x7fdce0005f90 comp rx=0 tx=0).stop 2024-03-15T13:09:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.202+0000 7fdcfc977640 1 -- 172.21.15.2:0/2882781436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcf81005d0 msgr2=0x7fdcf819c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.202+0000 7fdcfc977640 1 --2- 172.21.15.2:0/2882781436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcf81005d0 0x7fdcf819c6a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000e9f0 tx=0x7fdcf000eeb0 comp rx=0 tx=0).stop 2024-03-15T13:09:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.202+0000 7fdcfc977640 1 -- 172.21.15.2:0/2882781436 shutdown_connections 2024-03-15T13:09:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.202+0000 7fdcfc977640 1 --2- 172.21.15.2:0/2882781436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcf8196810 0x7fdcf8196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.202+0000 7fdcfc977640 1 --2- 172.21.15.2:0/2882781436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf81056a0 0x7fdcf819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.202+0000 7fdcfc977640 1 --2- 172.21.15.2:0/2882781436 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fdce40a1c20 0x7fdce40a40e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.202+0000 7fdcfc977640 1 --2- 172.21.15.2:0/2882781436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcf81005d0 0x7fdcf819c6a0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.202+0000 7fdcfc977640 1 -- 172.21.15.2:0/2882781436 >> 172.21.15.2:0/2882781436 conn(0x7fdcf80774f0 msgr2=0x7fdcf80fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.202+0000 7fdcfc977640 1 -- 172.21.15.2:0/2882781436 shutdown_connections 2024-03-15T13:09:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.202+0000 7fdcfc977640 1 -- 172.21.15.2:0/2882781436 wait complete. 2024-03-15T13:09:56.202 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-15T13:09:56.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.333+0000 7fa20f8da640 1 -- 172.21.15.2:0/440357771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa208107500 msgr2=0x7fa2081078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:56.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.333+0000 7fa20f8da640 1 --2- 172.21.15.2:0/440357771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa208107500 0x7fa2081078e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa204009930 tx=0x7fa20402bc20 comp rx=0 tx=0).stop 2024-03-15T13:09:56.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.333+0000 7fa20f8da640 1 -- 172.21.15.2:0/440357771 shutdown_connections 2024-03-15T13:09:56.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.333+0000 7fa20f8da640 1 --2- 172.21.15.2:0/440357771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa208069090 0x7fa20810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:56.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.333+0000 7fa20f8da640 1 --2- 172.21.15.2:0/440357771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2080686d0 0x7fa208068b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:56.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.333+0000 7fa20f8da640 1 --2- 172.21.15.2:0/440357771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa208107500 0x7fa2081078e0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:56.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.333+0000 7fa20f8da640 1 -- 172.21.15.2:0/440357771 >> 172.21.15.2:0/440357771 conn(0x7fa2080774f0 msgr2=0x7fa208077900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:56.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.333+0000 7fa20f8da640 1 -- 172.21.15.2:0/440357771 shutdown_connections 2024-03-15T13:09:56.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.333+0000 7fa20f8da640 1 -- 172.21.15.2:0/440357771 wait complete. 2024-03-15T13:09:56.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.334+0000 7fa20f8da640 1 Processor -- start 2024-03-15T13:09:56.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.334+0000 7fa20f8da640 1 -- start start 2024-03-15T13:09:56.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.334+0000 7fa20f8da640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2080686d0 0x7fa2081003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:56.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.334+0000 7fa20f8da640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa208069090 0x7fa2081008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:56.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.334+0000 7fa20f8da640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa208107500 0x7fa208100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:56.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.334+0000 7fa20f8da640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2081a5f50 con 0x7fa208107500 2024-03-15T13:09:56.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.334+0000 7fa20f8da640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2081a60c0 con 0x7fa2080686d0 2024-03-15T13:09:56.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.334+0000 7fa20f8da640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2081a6280 con 0x7fa208069090 2024-03-15T13:09:56.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.334+0000 7fa20de77640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa208107500 0x7fa208100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:56.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.334+0000 7fa20de77640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa208107500 0x7fa208100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57906/0 (socket says 172.21.15.2:57906) 2024-03-15T13:09:56.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.334+0000 7fa20de77640 1 -- 172.21.15.2:0/3867571391 learned_addr learned my addr 172.21.15.2:0/3867571391 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:56.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.334+0000 7fa20ce75640 1 --2- 172.21.15.2:0/3867571391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa208069090 0x7fa2081008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:56.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.334+0000 7fa20d676640 1 --2- 172.21.15.2:0/3867571391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2080686d0 0x7fa2081003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:56.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.335+0000 7fa20de77640 1 -- 172.21.15.2:0/3867571391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa208069090 msgr2=0x7fa2081008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:56.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.335+0000 7fa20de77640 1 --2- 172.21.15.2:0/3867571391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa208069090 0x7fa2081008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:56.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.335+0000 7fa20de77640 1 -- 172.21.15.2:0/3867571391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2080686d0 msgr2=0x7fa2081003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:56.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.335+0000 7fa20de77640 1 --2- 172.21.15.2:0/3867571391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2080686d0 0x7fa2081003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:56.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.335+0000 7fa20de77640 1 -- 172.21.15.2:0/3867571391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204009560 con 0x7fa208107500 2024-03-15T13:09:56.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.335+0000 7fa20de77640 1 --2- 172.21.15.2:0/3867571391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa208107500 0x7fa208100e30 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa200009fd0 tx=0x7fa20000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:56.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.335+0000 7fa1fe7fc640 1 -- 172.21.15.2:0/3867571391 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20000bea0 con 0x7fa208107500 2024-03-15T13:09:56.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.335+0000 7fa20f8da640 1 -- 172.21.15.2:0/3867571391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2081a6560 con 0x7fa208107500 2024-03-15T13:09:56.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.335+0000 7fa1fe7fc640 1 -- 172.21.15.2:0/3867571391 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7fa200009940 con 0x7fa208107500 2024-03-15T13:09:56.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.335+0000 7fa1fe7fc640 1 -- 172.21.15.2:0/3867571391 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2000106c0 con 0x7fa208107500 2024-03-15T13:09:56.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.335+0000 7fa20f8da640 1 -- 172.21.15.2:0/3867571391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2081a6a20 con 0x7fa208107500 2024-03-15T13:09:56.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.336+0000 7fa20f8da640 1 -- 172.21.15.2:0/3867571391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2080682e0 con 0x7fa208107500 2024-03-15T13:09:56.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.337+0000 7fa1fe7fc640 1 -- 172.21.15.2:0/3867571391 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7fa200010820 con 0x7fa208107500 2024-03-15T13:09:56.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.338+0000 7fa1fe7fc640 1 --2- 172.21.15.2:0/3867571391 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa1f40a1ba0 0x7fa1f40a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:56.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.338+0000 7fa20d676640 1 --2- 172.21.15.2:0/3867571391 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa1f40a1ba0 0x7fa1f40a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:56.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.338+0000 7fa1fe7fc640 1 -- 172.21.15.2:0/3867571391 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9794+0+0 (secure 0 0 0) 0x7fa2000cd780 con 0x7fa208107500 2024-03-15T13:09:56.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.339+0000 7fa20d676640 1 --2- 172.21.15.2:0/3867571391 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa1f40a1ba0 0x7fa1f40a4060 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa204006010 tx=0x7fa204000bc0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:56.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.343+0000 7fa1fe7fc640 1 -- 172.21.15.2:0/3867571391 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa2000960f0 con 0x7fa208107500 2024-03-15T13:09:56.483 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-a[32056]: 2024-03-15T13:09:56.197+0000 7fdfac1c1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-15T13:09:56.483 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-a[74141]: ignoring --setuser ceph since I am not root 2024-03-15T13:09:56.483 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-a[74141]: ignoring --setgroup ceph since I am not root 2024-03-15T13:09:56.483 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-mds[74147]: -- 172.21.15.2:0/1766186166 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556026b0c1a0 con 0x556026ae2800 2024-03-15T13:09:56.484 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-a[74141]: starting mds.a at 2024-03-15T13:09:56.484 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-g[73785]: ignoring --setuser ceph since I am not root 2024-03-15T13:09:56.484 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-g[73785]: ignoring --setgroup ceph since I am not root 2024-03-15T13:09:56.484 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-mds[73789]: -- 172.21.15.2:0/3731655478 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5625cc0981a0 con 0x5625cc06e800 2024-03-15T13:09:56.484 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-g[73785]: starting mds.g at 2024-03-15T13:09:56.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:56.571+0000 7fa20f8da640 1 -- 172.21.15.2:0/3867571391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa2081099d0 con 0x7fa208107500 2024-03-15T13:09:56.557 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-f[66359]: ignoring --setuser ceph since I am not root 2024-03-15T13:09:56.558 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-f[66359]: ignoring --setgroup ceph since I am not root 2024-03-15T13:09:56.558 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-mds[66363]: -- 172.21.15.184:0/2573536368 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561f3379e1a0 con 0x561f33774800 2024-03-15T13:09:56.558 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-f[66359]: starting mds.f at 2024-03-15T13:09:56.558 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-l[66050]: ignoring --setuser ceph since I am not root 2024-03-15T13:09:56.558 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-l[66050]: ignoring --setgroup ceph since I am not root 2024-03-15T13:09:56.558 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-mds[66054]: -- 172.21.15.184:0/2353761262 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563225b461a0 con 0x563225b1d000 2024-03-15T13:09:56.558 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-l[66050]: starting mds.l at 2024-03-15T13:09:56.559 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-i[66981]: ignoring --setuser ceph since I am not root 2024-03-15T13:09:56.559 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-i[66981]: ignoring --setgroup ceph since I am not root 2024-03-15T13:09:56.559 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-mds[66985]: -- 172.21.15.184:0/549190990 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55783d61a1a0 con 0x55783d5f0c00 2024-03-15T13:09:56.559 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-i[66981]: starting mds.i at 2024-03-15T13:09:56.559 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-c[66664]: ignoring --setuser ceph since I am not root 2024-03-15T13:09:56.559 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-c[66664]: ignoring --setgroup ceph since I am not root 2024-03-15T13:09:56.559 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-mds[66668]: -- 172.21.15.184:0/3544457145 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564baf8d41a0 con 0x564baf8aa800 2024-03-15T13:09:56.559 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-c[66664]: starting mds.c at 2024-03-15T13:09:56.608 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-e[65228]: ignoring --setuser ceph since I am not root 2024-03-15T13:09:56.608 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-e[65228]: ignoring --setgroup ceph since I am not root 2024-03-15T13:09:56.608 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-mds[65232]: -- 172.21.15.149:0/2495013285 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55de286a61a0 con 0x55de2867d000 2024-03-15T13:09:56.608 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-e[65228]: starting mds.e at 2024-03-15T13:09:56.609 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-b[65861]: ignoring --setuser ceph since I am not root 2024-03-15T13:09:56.609 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-b[65861]: ignoring --setgroup ceph since I am not root 2024-03-15T13:09:56.609 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-mds[65865]: -- 172.21.15.149:0/967695042 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561b164ca1a0 con 0x561b164a0800 2024-03-15T13:09:56.609 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-b[65861]: starting mds.b at 2024-03-15T13:09:56.609 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-h[65554]: ignoring --setuser ceph since I am not root 2024-03-15T13:09:56.609 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-h[65554]: ignoring --setgroup ceph since I am not root 2024-03-15T13:09:56.609 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-mds[65558]: -- 172.21.15.149:0/1730953073 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562a3c5181a0 con 0x562a3c4ee800 2024-03-15T13:09:56.609 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-h[65554]: starting mds.h at 2024-03-15T13:09:56.610 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-k[64909]: ignoring --setuser ceph since I am not root 2024-03-15T13:09:56.610 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-k[64909]: ignoring --setgroup ceph since I am not root 2024-03-15T13:09:56.610 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-mds[64913]: -- 172.21.15.149:0/2765680759 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560b136341a0 con 0x560b1360a800 2024-03-15T13:09:56.610 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-k[64909]: starting mds.k at 2024-03-15T13:09:56.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/374731139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T13:09:56.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2882781436' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-15T13:09:56.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-mon[32060]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-15T13:09:56.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-mon[32060]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-15T13:09:56.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2882781436' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-15T13:09:56.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-mon[32060]: osdmap e101: 12 total, 12 up, 12 in 2024-03-15T13:09:56.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-mon[32060]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-15T13:09:56.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:56 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3867571391' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-15T13:09:57.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/374731139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T13:09:57.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2882781436' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-15T13:09:57.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-mon[35993]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-15T13:09:57.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-mon[35993]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-15T13:09:57.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2882781436' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-15T13:09:57.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-mon[35993]: osdmap e101: 12 total, 12 up, 12 in 2024-03-15T13:09:57.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-mon[35993]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-15T13:09:57.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:56 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3867571391' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-15T13:09:57.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/374731139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-15T13:09:57.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2882781436' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-15T13:09:57.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-mon[35927]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-15T13:09:57.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-mon[35927]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-15T13:09:57.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2882781436' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-15T13:09:57.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-mon[35927]: osdmap e101: 12 total, 12 up, 12 in 2024-03-15T13:09:57.108 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-mon[35927]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-15T13:09:57.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:56 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3867571391' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-15T13:09:57.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.199+0000 7fa1fe7fc640 1 -- 172.21.15.2:0/3867571391 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa2000997b0 con 0x7fa208107500 2024-03-15T13:09:57.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.202+0000 7fa20f8da640 1 -- 172.21.15.2:0/3867571391 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa1f40a1ba0 msgr2=0x7fa1f40a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:57.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.202+0000 7fa20f8da640 1 --2- 172.21.15.2:0/3867571391 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa1f40a1ba0 0x7fa1f40a4060 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa204006010 tx=0x7fa204000bc0 comp rx=0 tx=0).stop 2024-03-15T13:09:57.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.202+0000 7fa20f8da640 1 -- 172.21.15.2:0/3867571391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa208107500 msgr2=0x7fa208100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:57.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.202+0000 7fa20f8da640 1 --2- 172.21.15.2:0/3867571391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa208107500 0x7fa208100e30 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa200009fd0 tx=0x7fa20000b260 comp rx=0 tx=0).stop 2024-03-15T13:09:57.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.202+0000 7fa20f8da640 1 -- 172.21.15.2:0/3867571391 shutdown_connections 2024-03-15T13:09:57.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.202+0000 7fa20f8da640 1 --2- 172.21.15.2:0/3867571391 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7fa1f40a1ba0 0x7fa1f40a4060 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:57.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.202+0000 7fa20f8da640 1 --2- 172.21.15.2:0/3867571391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa208107500 0x7fa208100e30 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:57.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.202+0000 7fa20f8da640 1 --2- 172.21.15.2:0/3867571391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa208069090 0x7fa2081008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:57.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.202+0000 7fa20f8da640 1 --2- 172.21.15.2:0/3867571391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2080686d0 0x7fa2081003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:57.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.202+0000 7fa20f8da640 1 -- 172.21.15.2:0/3867571391 >> 172.21.15.2:0/3867571391 conn(0x7fa2080774f0 msgr2=0x7fa2080fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:57.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.202+0000 7fa20f8da640 1 -- 172.21.15.2:0/3867571391 shutdown_connections 2024-03-15T13:09:57.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.202+0000 7fa20f8da640 1 -- 172.21.15.2:0/3867571391 wait complete. 2024-03-15T13:09:57.201 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-15T13:09:57.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.326+0000 7efddee0e640 1 -- 172.21.15.2:0/818986382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdd81014b0 msgr2=0x7efdd8101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:57.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.326+0000 7efddee0e640 1 --2- 172.21.15.2:0/818986382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdd81014b0 0x7efdd8101930 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009ca0 tx=0x7efdc802c0f0 comp rx=0 tx=0).stop 2024-03-15T13:09:57.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.326+0000 7efddee0e640 1 -- 172.21.15.2:0/818986382 shutdown_connections 2024-03-15T13:09:57.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.326+0000 7efddee0e640 1 --2- 172.21.15.2:0/818986382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd8101e70 0x7efdd810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:57.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.326+0000 7efddee0e640 1 --2- 172.21.15.2:0/818986382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdd81014b0 0x7efdd8101930 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:57.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.326+0000 7efddee0e640 1 --2- 172.21.15.2:0/818986382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd8107570 0x7efdd8107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.326+0000 7efddee0e640 1 -- 172.21.15.2:0/818986382 >> 172.21.15.2:0/818986382 conn(0x7efdd80fcf20 msgr2=0x7efdd80ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.327+0000 7efddee0e640 1 -- 172.21.15.2:0/818986382 shutdown_connections 2024-03-15T13:09:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.327+0000 7efddee0e640 1 -- 172.21.15.2:0/818986382 wait complete. 2024-03-15T13:09:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.327+0000 7efddee0e640 1 Processor -- start 2024-03-15T13:09:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.327+0000 7efddee0e640 1 -- start start 2024-03-15T13:09:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.327+0000 7efddee0e640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd81014b0 0x7efdd819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.327+0000 7efddee0e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd8101e70 0x7efdd819dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.327+0000 7efddee0e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdd8107570 0x7efdd8197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.327+0000 7efddee0e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd8197f90 con 0x7efdd8107570 2024-03-15T13:09:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.327+0000 7efddee0e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd8198100 con 0x7efdd8101e70 2024-03-15T13:09:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.327+0000 7efddee0e640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd81982c0 con 0x7efdd81014b0 2024-03-15T13:09:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.327+0000 7efddd3ab640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdd8107570 0x7efdd8197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:57.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.327+0000 7efddd3ab640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdd8107570 0x7efdd8197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57932/0 (socket says 172.21.15.2:57932) 2024-03-15T13:09:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.327+0000 7efddd3ab640 1 -- 172.21.15.2:0/3292616245 learned_addr learned my addr 172.21.15.2:0/3292616245 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.327+0000 7efdd7fff640 1 --2- 172.21.15.2:0/3292616245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd8101e70 0x7efdd819dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.328+0000 7efddd3ab640 1 -- 172.21.15.2:0/3292616245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd81014b0 msgr2=0x7efdd819d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.328+0000 7efddcbaa640 1 --2- 172.21.15.2:0/3292616245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd81014b0 0x7efdd819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.328+0000 7efddd3ab640 1 --2- 172.21.15.2:0/3292616245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd81014b0 0x7efdd819d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.328+0000 7efddd3ab640 1 -- 172.21.15.2:0/3292616245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd8101e70 msgr2=0x7efdd819dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.328+0000 7efddd3ab640 1 --2- 172.21.15.2:0/3292616245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd8101e70 0x7efdd819dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.328+0000 7efddd3ab640 1 -- 172.21.15.2:0/3292616245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc8009900 con 0x7efdd8107570 2024-03-15T13:09:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.328+0000 7efddcbaa640 1 --2- 172.21.15.2:0/3292616245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd81014b0 0x7efdd819d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T13:09:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.328+0000 7efddd3ab640 1 --2- 172.21.15.2:0/3292616245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdd8107570 0x7efdd8197940 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7efdd000d750 tx=0x7efdd000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.328+0000 7efdd5ffb640 1 -- 172.21.15.2:0/3292616245 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0009bd0 con 0x7efdd8107570 2024-03-15T13:09:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.328+0000 7efdd5ffb640 1 -- 172.21.15.2:0/3292616245 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7efdd000de60 con 0x7efdd8107570 2024-03-15T13:09:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.328+0000 7efddee0e640 1 -- 172.21.15.2:0/3292616245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd81985a0 con 0x7efdd8107570 2024-03-15T13:09:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.328+0000 7efdd5ffb640 1 -- 172.21.15.2:0/3292616245 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0004e50 con 0x7efdd8107570 2024-03-15T13:09:57.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.328+0000 7efddee0e640 1 -- 172.21.15.2:0/3292616245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd81a47e0 con 0x7efdd8107570 2024-03-15T13:09:57.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.329+0000 7efddee0e640 1 -- 172.21.15.2:0/3292616245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd80682e0 con 0x7efdd8107570 2024-03-15T13:09:57.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.330+0000 7efdd5ffb640 1 -- 172.21.15.2:0/3292616245 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7efdd0009d30 con 0x7efdd8107570 2024-03-15T13:09:57.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.331+0000 7efdd5ffb640 1 --2- 172.21.15.2:0/3292616245 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efdc40a1c20 0x7efdc40a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:57.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.331+0000 7efdd5ffb640 1 -- 172.21.15.2:0/3292616245 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9794+0+0 (secure 0 0 0) 0x7efdd00cd640 con 0x7efdd8107570 2024-03-15T13:09:57.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.331+0000 7efddcbaa640 1 --2- 172.21.15.2:0/3292616245 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efdc40a1c20 0x7efdc40a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:57.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.331+0000 7efddcbaa640 1 --2- 172.21.15.2:0/3292616245 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efdc40a1c20 0x7efdc40a40e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efdd8106610 tx=0x7efdc0005e70 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:57.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.335+0000 7efdd5ffb640 1 -- 172.21.15.2:0/3292616245 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efdd0096010 con 0x7efdd8107570 2024-03-15T13:09:57.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:57.519+0000 7efddee0e640 1 -- 172.21.15.2:0/3292616245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efdd8109980 con 0x7efdd8107570 2024-03-15T13:09:57.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: pgmap v342: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-03-15T13:09:57.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-15T13:09:57.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-15T13:09:57.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: Cluster is now healthy 2024-03-15T13:09:57.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3867571391' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-15T13:09:57.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.184:6834/45498782,v1:172.21.15.184:6835/45498782] up:boot 2024-03-15T13:09:57.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.184:6832/2550447787,v1:172.21.15.184:6833/2550447787] up:boot 2024-03-15T13:09:57.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.2:6840/3613700889,v1:172.21.15.2:6841/3613700889] up:boot 2024-03-15T13:09:57.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.149:6834/347531372,v1:172.21.15.149:6838/347531372] up:boot 2024-03-15T13:09:57.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.149:6837/541103655,v1:172.21.15.149:6839/541103655] up:boot 2024-03-15T13:09:57.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.184:6838/2553778472,v1:172.21.15.184:6839/2553778472] up:boot 2024-03-15T13:09:57.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.2:6838/3715593713,v1:172.21.15.2:6839/3715593713] up:boot 2024-03-15T13:09:57.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.149:6833/3037238813,v1:172.21.15.149:6836/3037238813] up:boot 2024-03-15T13:09:57.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.149:6832/3922373083,v1:172.21.15.149:6835/3922373083] up:boot 2024-03-15T13:09:57.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: mds.? [v2:172.21.15.184:6836/4218790582,v1:172.21.15.184:6837/4218790582] up:boot 2024-03-15T13:09:57.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: fsmap 12 up:standby 2024-03-15T13:09:57.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-15T13:09:57.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-15T13:09:57.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-15T13:09:57.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-15T13:09:57.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-15T13:09:57.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-15T13:09:57.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-15T13:09:57.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-15T13:09:57.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-15T13:09:57.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-15T13:09:57.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:57 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3292616245' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-15T13:09:58.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: pgmap v342: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-03-15T13:09:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-15T13:09:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-15T13:09:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: Cluster is now healthy 2024-03-15T13:09:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3867571391' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-15T13:09:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.184:6834/45498782,v1:172.21.15.184:6835/45498782] up:boot 2024-03-15T13:09:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.184:6832/2550447787,v1:172.21.15.184:6833/2550447787] up:boot 2024-03-15T13:09:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.2:6840/3613700889,v1:172.21.15.2:6841/3613700889] up:boot 2024-03-15T13:09:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.149:6834/347531372,v1:172.21.15.149:6838/347531372] up:boot 2024-03-15T13:09:58.058 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.149:6837/541103655,v1:172.21.15.149:6839/541103655] up:boot 2024-03-15T13:09:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.184:6838/2553778472,v1:172.21.15.184:6839/2553778472] up:boot 2024-03-15T13:09:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.2:6838/3715593713,v1:172.21.15.2:6839/3715593713] up:boot 2024-03-15T13:09:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.149:6833/3037238813,v1:172.21.15.149:6836/3037238813] up:boot 2024-03-15T13:09:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.149:6832/3922373083,v1:172.21.15.149:6835/3922373083] up:boot 2024-03-15T13:09:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: mds.? [v2:172.21.15.184:6836/4218790582,v1:172.21.15.184:6837/4218790582] up:boot 2024-03-15T13:09:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: fsmap 12 up:standby 2024-03-15T13:09:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-15T13:09:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-15T13:09:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-15T13:09:58.059 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-15T13:09:58.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-15T13:09:58.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-15T13:09:58.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-15T13:09:58.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-15T13:09:58.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-15T13:09:58.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-15T13:09:58.060 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:57 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3292616245' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-15T13:09:58.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: pgmap v342: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-03-15T13:09:58.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-15T13:09:58.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-15T13:09:58.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: Cluster is now healthy 2024-03-15T13:09:58.109 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3867571391' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-15T13:09:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.184:6834/45498782,v1:172.21.15.184:6835/45498782] up:boot 2024-03-15T13:09:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.184:6832/2550447787,v1:172.21.15.184:6833/2550447787] up:boot 2024-03-15T13:09:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.2:6840/3613700889,v1:172.21.15.2:6841/3613700889] up:boot 2024-03-15T13:09:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.149:6834/347531372,v1:172.21.15.149:6838/347531372] up:boot 2024-03-15T13:09:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.149:6837/541103655,v1:172.21.15.149:6839/541103655] up:boot 2024-03-15T13:09:58.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.184:6838/2553778472,v1:172.21.15.184:6839/2553778472] up:boot 2024-03-15T13:09:58.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.2:6838/3715593713,v1:172.21.15.2:6839/3715593713] up:boot 2024-03-15T13:09:58.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.149:6833/3037238813,v1:172.21.15.149:6836/3037238813] up:boot 2024-03-15T13:09:58.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.149:6832/3922373083,v1:172.21.15.149:6835/3922373083] up:boot 2024-03-15T13:09:58.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: mds.? [v2:172.21.15.184:6836/4218790582,v1:172.21.15.184:6837/4218790582] up:boot 2024-03-15T13:09:58.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: fsmap 12 up:standby 2024-03-15T13:09:58.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-15T13:09:58.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-15T13:09:58.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-15T13:09:58.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-15T13:09:58.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-15T13:09:58.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-15T13:09:58.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-15T13:09:58.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-15T13:09:58.112 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-15T13:09:58.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: from='mgr.15741 172.21.15.2:0/4088238005' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-15T13:09:58.113 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:57 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3292616245' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-15T13:09:58.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.201+0000 7efdd5ffb640 1 -- 172.21.15.2:0/3292616245 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) v1 ==== 181+0+0 (secure 0 0 0) 0x7efdd00200d0 con 0x7efdd8107570 2024-03-15T13:09:58.186 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' removed 2024-03-15T13:09:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.205+0000 7efddee0e640 1 -- 172.21.15.2:0/3292616245 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efdc40a1c20 msgr2=0x7efdc40a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.205+0000 7efddee0e640 1 --2- 172.21.15.2:0/3292616245 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efdc40a1c20 0x7efdc40a40e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efdd8106610 tx=0x7efdc0005e70 comp rx=0 tx=0).stop 2024-03-15T13:09:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.205+0000 7efddee0e640 1 -- 172.21.15.2:0/3292616245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdd8107570 msgr2=0x7efdd8197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.205+0000 7efddee0e640 1 --2- 172.21.15.2:0/3292616245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdd8107570 0x7efdd8197940 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7efdd000d750 tx=0x7efdd000dc10 comp rx=0 tx=0).stop 2024-03-15T13:09:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.205+0000 7efddee0e640 1 -- 172.21.15.2:0/3292616245 shutdown_connections 2024-03-15T13:09:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.205+0000 7efddee0e640 1 --2- 172.21.15.2:0/3292616245 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7efdc40a1c20 0x7efdc40a40e0 secure :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efdd8106610 tx=0x7efdc0005e70 comp rx=0 tx=0).stop 2024-03-15T13:09:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.205+0000 7efddee0e640 1 --2- 172.21.15.2:0/3292616245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdd8107570 0x7efdd8197940 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.205+0000 7efddee0e640 1 --2- 172.21.15.2:0/3292616245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd8101e70 0x7efdd819dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.205+0000 7efddee0e640 1 --2- 172.21.15.2:0/3292616245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efdd81014b0 0x7efdd819d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.205+0000 7efddee0e640 1 -- 172.21.15.2:0/3292616245 >> 172.21.15.2:0/3292616245 conn(0x7efdd80fcf20 msgr2=0x7efdd80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.206+0000 7efddee0e640 1 -- 172.21.15.2:0/3292616245 shutdown_connections 2024-03-15T13:09:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.206+0000 7efddee0e640 1 -- 172.21.15.2:0/3292616245 wait complete. 2024-03-15T13:09:58.206 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-15T13:09:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.335+0000 7f74d71fd640 1 -- 172.21.15.2:0/5591604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d0102760 msgr2=0x7f74d0102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.335+0000 7f74d71fd640 1 --2- 172.21.15.2:0/5591604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d0102760 0x7f74d0102b40 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009900 tx=0x7f74cc02bcd0 comp rx=0 tx=0).stop 2024-03-15T13:09:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.335+0000 7f74d71fd640 1 -- 172.21.15.2:0/5591604 shutdown_connections 2024-03-15T13:09:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.335+0000 7f74d71fd640 1 --2- 172.21.15.2:0/5591604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74d010bb30 0x7f74d010dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.335+0000 7f74d71fd640 1 --2- 172.21.15.2:0/5591604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d0103080 0x7f74d010b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.335+0000 7f74d71fd640 1 --2- 172.21.15.2:0/5591604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d0102760 0x7f74d0102b40 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.335+0000 7f74d71fd640 1 -- 172.21.15.2:0/5591604 >> 172.21.15.2:0/5591604 conn(0x7f74d0077550 msgr2=0x7f74d0077960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:58.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.335+0000 7f74d71fd640 1 -- 172.21.15.2:0/5591604 shutdown_connections 2024-03-15T13:09:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.335+0000 7f74d71fd640 1 -- 172.21.15.2:0/5591604 wait complete. 2024-03-15T13:09:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74d71fd640 1 Processor -- start 2024-03-15T13:09:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74d71fd640 1 -- start start 2024-03-15T13:09:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74d71fd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74d0102760 0x7f74d01003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74d71fd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d0103080 0x7f74d01008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74d71fd640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d010bb30 0x7f74d0100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74d71fd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d01a5f10 con 0x7f74d0103080 2024-03-15T13:09:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74d71fd640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d01a6080 con 0x7f74d0102760 2024-03-15T13:09:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74d71fd640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d01a6240 con 0x7f74d010bb30 2024-03-15T13:09:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d0103080 0x7f74d01008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d0103080 0x7f74d01008f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54192/0 (socket says 172.21.15.2:54192) 2024-03-15T13:09:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74c7fff640 1 -- 172.21.15.2:0/2021982863 learned_addr learned my addr 172.21.15.2:0/2021982863 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74d4f99640 1 --2- 172.21.15.2:0/2021982863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74d0102760 0x7f74d01003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:58.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74d579a640 1 --2- 172.21.15.2:0/2021982863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d010bb30 0x7f74d0100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74c7fff640 1 -- 172.21.15.2:0/2021982863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d010bb30 msgr2=0x7f74d0100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74c7fff640 1 --2- 172.21.15.2:0/2021982863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d010bb30 0x7f74d0100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74c7fff640 1 -- 172.21.15.2:0/2021982863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74d0102760 msgr2=0x7f74d01003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74c7fff640 1 --2- 172.21.15.2:0/2021982863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74d0102760 0x7f74d01003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74c7fff640 1 -- 172.21.15.2:0/2021982863 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc009560 con 0x7f74d0103080 2024-03-15T13:09:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74d579a640 1 --2- 172.21.15.2:0/2021982863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d010bb30 0x7f74d0100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-15T13:09:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.336+0000 7f74c7fff640 1 --2- 172.21.15.2:0/2021982863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d0103080 0x7f74d01008f0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f74c000d770 tx=0x7f74c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.337+0000 7f74c5ffb640 1 -- 172.21.15.2:0/2021982863 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0009c30 con 0x7f74d0103080 2024-03-15T13:09:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.337+0000 7f74c5ffb640 1 -- 172.21.15.2:0/2021982863 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7f74c000de80 con 0x7f74d0103080 2024-03-15T13:09:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.337+0000 7f74d71fd640 1 -- 172.21.15.2:0/2021982863 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d01a6520 con 0x7f74d0103080 2024-03-15T13:09:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.337+0000 7f74c5ffb640 1 -- 172.21.15.2:0/2021982863 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0004e50 con 0x7f74d0103080 2024-03-15T13:09:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.337+0000 7f74d71fd640 1 -- 172.21.15.2:0/2021982863 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d01a69e0 con 0x7f74d0103080 2024-03-15T13:09:58.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.338+0000 7f74d71fd640 1 -- 172.21.15.2:0/2021982863 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d0068730 con 0x7f74d0103080 2024-03-15T13:09:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.339+0000 7f74c5ffb640 1 -- 172.21.15.2:0/2021982863 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7f74c0005000 con 0x7f74d0103080 2024-03-15T13:09:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.339+0000 7f74c5ffb640 1 --2- 172.21.15.2:0/2021982863 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f74b80a1ba0 0x7f74b80a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.339+0000 7f74c5ffb640 1 -- 172.21.15.2:0/2021982863 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9298+0+0 (secure 0 0 0) 0x7f74c00cd5c0 con 0x7f74d0103080 2024-03-15T13:09:58.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.339+0000 7f74d4f99640 1 --2- 172.21.15.2:0/2021982863 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f74b80a1ba0 0x7f74b80a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:58.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.340+0000 7f74d4f99640 1 --2- 172.21.15.2:0/2021982863 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f74b80a1ba0 0x7f74b80a4060 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f74cc005ab0 tx=0x7f74cc005a20 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:58.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.344+0000 7f74c5ffb640 1 -- 172.21.15.2:0/2021982863 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f74c0096180 con 0x7f74d0103080 2024-03-15T13:09:58.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:58.527+0000 7f74d71fd640 1 -- 172.21.15.2:0/2021982863 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f74d0106c20 con 0x7f74d0103080 2024-03-15T13:09:59.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.210+0000 7f74c5ffb640 1 -- 172.21.15.2:0/2021982863 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) v1 ==== 169+0+0 (secure 0 0 0) 0x7f74c00200d0 con 0x7f74d0103080 2024-03-15T13:09:59.195 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' removed 2024-03-15T13:09:59.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.214+0000 7f74d71fd640 1 -- 172.21.15.2:0/2021982863 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f74b80a1ba0 msgr2=0x7f74b80a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:59.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.214+0000 7f74d71fd640 1 --2- 172.21.15.2:0/2021982863 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f74b80a1ba0 0x7f74b80a4060 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f74cc005ab0 tx=0x7f74cc005a20 comp rx=0 tx=0).stop 2024-03-15T13:09:59.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.214+0000 7f74d71fd640 1 -- 172.21.15.2:0/2021982863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d0103080 msgr2=0x7f74d01008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:59.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.214+0000 7f74d71fd640 1 --2- 172.21.15.2:0/2021982863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d0103080 0x7f74d01008f0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f74c000d770 tx=0x7f74c000dc30 comp rx=0 tx=0).stop 2024-03-15T13:09:59.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.215+0000 7f74d71fd640 1 -- 172.21.15.2:0/2021982863 shutdown_connections 2024-03-15T13:09:59.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.215+0000 7f74d71fd640 1 --2- 172.21.15.2:0/2021982863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74d010bb30 0x7f74d0100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:59.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.215+0000 7f74d71fd640 1 --2- 172.21.15.2:0/2021982863 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7f74b80a1ba0 0x7f74b80a4060 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:59.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.215+0000 7f74d71fd640 1 --2- 172.21.15.2:0/2021982863 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d0103080 0x7f74d01008f0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:59.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.215+0000 7f74d71fd640 1 --2- 172.21.15.2:0/2021982863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74d0102760 0x7f74d01003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:59.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.215+0000 7f74d71fd640 1 -- 172.21.15.2:0/2021982863 >> 172.21.15.2:0/2021982863 conn(0x7f74d0077550 msgr2=0x7f74d00fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:59.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.215+0000 7f74d71fd640 1 -- 172.21.15.2:0/2021982863 shutdown_connections 2024-03-15T13:09:59.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.216+0000 7f74d71fd640 1 -- 172.21.15.2:0/2021982863 wait complete. 2024-03-15T13:09:59.219 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-15T13:09:59.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.347+0000 7ff0bef2c640 1 -- 172.21.15.2:0/4201340364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0b81014b0 msgr2=0x7ff0b8101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:59.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.347+0000 7ff0bef2c640 1 --2- 172.21.15.2:0/4201340364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0b81014b0 0x7ff0b8101930 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009900 tx=0x7ff0ac02bc30 comp rx=0 tx=0).stop 2024-03-15T13:09:59.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.348+0000 7ff0bef2c640 1 -- 172.21.15.2:0/4201340364 shutdown_connections 2024-03-15T13:09:59.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.348+0000 7ff0bef2c640 1 --2- 172.21.15.2:0/4201340364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0b8101e70 0x7ff0b810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:59.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.348+0000 7ff0bef2c640 1 --2- 172.21.15.2:0/4201340364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0b81014b0 0x7ff0b8101930 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:59.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.348+0000 7ff0bef2c640 1 --2- 172.21.15.2:0/4201340364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b8107570 0x7ff0b8107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:59.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.348+0000 7ff0bef2c640 1 -- 172.21.15.2:0/4201340364 >> 172.21.15.2:0/4201340364 conn(0x7ff0b80fcf80 msgr2=0x7ff0b80ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:09:59.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.348+0000 7ff0bef2c640 1 -- 172.21.15.2:0/4201340364 shutdown_connections 2024-03-15T13:09:59.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.348+0000 7ff0bef2c640 1 -- 172.21.15.2:0/4201340364 wait complete. 2024-03-15T13:09:59.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.348+0000 7ff0bef2c640 1 Processor -- start 2024-03-15T13:09:59.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.348+0000 7ff0bef2c640 1 -- start start 2024-03-15T13:09:59.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.348+0000 7ff0bef2c640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b81014b0 0x7ff0b819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:59.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.348+0000 7ff0bef2c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0b8101e70 0x7ff0b819dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:59.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.348+0000 7ff0bef2c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0b8107570 0x7ff0b81978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:59.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.348+0000 7ff0bef2c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8197f10 con 0x7ff0b8107570 2024-03-15T13:09:59.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.348+0000 7ff0bef2c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8198080 con 0x7ff0b8101e70 2024-03-15T13:09:59.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.348+0000 7ff0bef2c640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8198240 con 0x7ff0b81014b0 2024-03-15T13:09:59.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0bd4c9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0b8107570 0x7ff0b81978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:59.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0bd4c9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0b8107570 0x7ff0b81978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54210/0 (socket says 172.21.15.2:54210) 2024-03-15T13:09:59.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0bd4c9640 1 -- 172.21.15.2:0/2202133903 learned_addr learned my addr 172.21.15.2:0/2202133903 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-15T13:09:59.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0b7fff640 1 --2- 172.21.15.2:0/2202133903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0b8101e70 0x7ff0b819dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:59.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0bccc8640 1 --2- 172.21.15.2:0/2202133903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b81014b0 0x7ff0b819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:59.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0bd4c9640 1 -- 172.21.15.2:0/2202133903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b81014b0 msgr2=0x7ff0b819d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:59.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0bd4c9640 1 --2- 172.21.15.2:0/2202133903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b81014b0 0x7ff0b819d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:59.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0bd4c9640 1 -- 172.21.15.2:0/2202133903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0b8101e70 msgr2=0x7ff0b819dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:09:59.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0bd4c9640 1 --2- 172.21.15.2:0/2202133903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0b8101e70 0x7ff0b819dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:09:59.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0bd4c9640 1 -- 172.21.15.2:0/2202133903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac009560 con 0x7ff0b8107570 2024-03-15T13:09:59.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0bd4c9640 1 --2- 172.21.15.2:0/2202133903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0b8107570 0x7ff0b81978c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800d750 tx=0x7ff0a800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:59.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0b5ffb640 1 -- 172.21.15.2:0/2202133903 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8009bd0 con 0x7ff0b8107570 2024-03-15T13:09:59.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0bef2c640 1 -- 172.21.15.2:0/2202133903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b8198520 con 0x7ff0b8107570 2024-03-15T13:09:59.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0b5ffb640 1 -- 172.21.15.2:0/2202133903 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1136+0+0 (secure 0 0 0) 0x7ff0a800de60 con 0x7ff0b8107570 2024-03-15T13:09:59.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0b5ffb640 1 -- 172.21.15.2:0/2202133903 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8004e50 con 0x7ff0b8107570 2024-03-15T13:09:59.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.349+0000 7ff0bef2c640 1 -- 172.21.15.2:0/2202133903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b81a47b0 con 0x7ff0b8107570 2024-03-15T13:09:59.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.350+0000 7ff0bef2c640 1 -- 172.21.15.2:0/2202133903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b80682e0 con 0x7ff0b8107570 2024-03-15T13:09:59.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.351+0000 7ff0b5ffb640 1 -- 172.21.15.2:0/2202133903 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 136409+0+0 (secure 0 0 0) 0x7ff0a8009d30 con 0x7ff0b8107570 2024-03-15T13:09:59.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.352+0000 7ff0b5ffb640 1 --2- 172.21.15.2:0/2202133903 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff0a00a1ba0 0x7ff0a00a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-15T13:09:59.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.352+0000 7ff0bccc8640 1 --2- 172.21.15.2:0/2202133903 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff0a00a1ba0 0x7ff0a00a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-15T13:09:59.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.352+0000 7ff0b5ffb640 1 -- 172.21.15.2:0/2202133903 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8797+0+0 (secure 0 0 0) 0x7ff0a80cd3a0 con 0x7ff0b8107570 2024-03-15T13:09:59.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.352+0000 7ff0bccc8640 1 --2- 172.21.15.2:0/2202133903 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff0a00a1ba0 0x7ff0a00a4060 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0005c40 tx=0x7ff0b0005bd0 comp rx=0 tx=0).ready entity=mgr.15741 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-15T13:09:59.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.357+0000 7ff0b5ffb640 1 -- 172.21.15.2:0/2202133903 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff0a8096100 con 0x7ff0b8107570 2024-03-15T13:09:59.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:09:59.538+0000 7ff0bef2c640 1 -- 172.21.15.2:0/2202133903 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0b81099c0 con 0x7ff0b8107570 2024-03-15T13:09:59.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:59 smithi002 ceph-mon[32060]: pgmap v344: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-15T13:09:59.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:59 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/3292616245' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-15T13:09:59.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:59 smithi002 ceph-mon[32060]: osdmap e102: 12 total, 12 up, 12 in 2024-03-15T13:09:59.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:09:59 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2021982863' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-15T13:09:59.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:59 smithi184 ceph-mon[35993]: pgmap v344: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-15T13:09:59.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:59 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/3292616245' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-15T13:09:59.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:59 smithi184 ceph-mon[35993]: osdmap e102: 12 total, 12 up, 12 in 2024-03-15T13:09:59.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:09:59 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2021982863' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-15T13:09:59.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:59 smithi149 ceph-mon[35927]: pgmap v344: 129 pgs: 129 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-15T13:09:59.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:59 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/3292616245' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-15T13:09:59.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:59 smithi149 ceph-mon[35927]: osdmap e102: 12 total, 12 up, 12 in 2024-03-15T13:09:59.609 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:09:59 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2021982863' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-15T13:10:00.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:10:00.216+0000 7ff0b5ffb640 1 -- 172.21.15.2:0/2202133903 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v104) v1 ==== 178+0+0 (secure 0 0 0) 0x7ff0a80200d0 con 0x7ff0b8107570 2024-03-15T13:10:00.200 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data_ec' removed 2024-03-15T13:10:00.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:10:00.219+0000 7ff0bef2c640 1 -- 172.21.15.2:0/2202133903 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff0a00a1ba0 msgr2=0x7ff0a00a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:10:00.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:10:00.219+0000 7ff0bef2c640 1 --2- 172.21.15.2:0/2202133903 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff0a00a1ba0 0x7ff0a00a4060 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0005c40 tx=0x7ff0b0005bd0 comp rx=0 tx=0).stop 2024-03-15T13:10:00.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:10:00.219+0000 7ff0bef2c640 1 -- 172.21.15.2:0/2202133903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0b8107570 msgr2=0x7ff0b81978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-15T13:10:00.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:10:00.220+0000 7ff0bef2c640 1 --2- 172.21.15.2:0/2202133903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0b8107570 0x7ff0b81978c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800d750 tx=0x7ff0a800dc10 comp rx=0 tx=0).stop 2024-03-15T13:10:00.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:10:00.220+0000 7ff0bef2c640 1 -- 172.21.15.2:0/2202133903 shutdown_connections 2024-03-15T13:10:00.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:10:00.220+0000 7ff0bef2c640 1 --2- 172.21.15.2:0/2202133903 >> [v2:172.21.15.2:6800/1550147071,v1:172.21.15.2:6801/1550147071] conn(0x7ff0a00a1ba0 0x7ff0a00a4060 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:10:00.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:10:00.220+0000 7ff0bef2c640 1 --2- 172.21.15.2:0/2202133903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0b8107570 0x7ff0b81978c0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:10:00.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:10:00.220+0000 7ff0bef2c640 1 --2- 172.21.15.2:0/2202133903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0b8101e70 0x7ff0b819dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:10:00.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:10:00.220+0000 7ff0bef2c640 1 --2- 172.21.15.2:0/2202133903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0b81014b0 0x7ff0b819d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-15T13:10:00.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:10:00.220+0000 7ff0bef2c640 1 -- 172.21.15.2:0/2202133903 >> 172.21.15.2:0/2202133903 conn(0x7ff0b80fcf80 msgr2=0x7ff0b80fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-15T13:10:00.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:10:00.220+0000 7ff0bef2c640 1 -- 172.21.15.2:0/2202133903 shutdown_connections 2024-03-15T13:10:00.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-15T13:10:00.220+0000 7ff0bef2c640 1 -- 172.21.15.2:0/2202133903 wait complete. 2024-03-15T13:10:00.224 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-15T13:10:00.258 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-15T13:10:00.289 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-15T13:10:00.322 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-15T13:10:00.322 DEBUG:teuthology.orchestra.run.smithi002:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-15T13:10:00.338 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-15T13:10:00.353 DEBUG:teuthology.orchestra.run.smithi184:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-15T13:10:00.378 INFO:tasks.cephadm:Stopping all daemons... 2024-03-15T13:10:00.378 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-15T13:10:00.378 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mon.a 2024-03-15T13:10:00.485 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:10:00 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2021982863' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-15T13:10:00.485 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:10:00 smithi002 ceph-mon[32060]: osdmap e103: 12 total, 12 up, 12 in 2024-03-15T13:10:00.486 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:10:00 smithi002 ceph-mon[32060]: from='client.? 172.21.15.2:0/2202133903' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-15T13:10:00.486 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:10:00 smithi002 ceph-mon[32060]: overall HEALTH_OK 2024-03-15T13:10:00.486 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:10:00 smithi002 systemd[1]: Stopping Ceph mon.a for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:10:00.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:10:00 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2021982863' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-15T13:10:00.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:10:00 smithi184 ceph-mon[35993]: osdmap e103: 12 total, 12 up, 12 in 2024-03-15T13:10:00.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:10:00 smithi184 ceph-mon[35993]: from='client.? 172.21.15.2:0/2202133903' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-15T13:10:00.557 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:10:00 smithi184 ceph-mon[35993]: overall HEALTH_OK 2024-03-15T13:10:00.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:10:00 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2021982863' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-15T13:10:00.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:10:00 smithi149 ceph-mon[35927]: osdmap e103: 12 total, 12 up, 12 in 2024-03-15T13:10:00.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:10:00 smithi149 ceph-mon[35927]: from='client.? 172.21.15.2:0/2202133903' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-15T13:10:00.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:10:00 smithi149 ceph-mon[35927]: overall HEALTH_OK 2024-03-15T13:10:00.801 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 13:10:00 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T13:10:00.500+0000 7fa904848700 -1 client.15768: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-03-15T13:10:00.801 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 13:10:00 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T13:10:00.501+0000 7fa904848700 -1 client.15768: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-03-15T13:10:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:10:00 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-a[32056]: 2024-03-15T13:10:00.693+0000 7fdfb19cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:10:00.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:10:00 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-a[32056]: 2024-03-15T13:10:00.693+0000 7fdfb19cc700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-15T13:10:00.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:10:00 smithi002 podman[104490]: 2024-03-15 13:10:00.714958745 +0000 UTC m=+0.134946752 container died 8198c386c33e32c3cc1a0a058ec5e97bcdd512ca11e79b0c592e011fcb23501c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-15T13:10:01.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:10:01 smithi002 podman[104490]: 2024-03-15 13:10:01.078194512 +0000 UTC m=+0.498182454 container cleanup 8198c386c33e32c3cc1a0a058ec5e97bcdd512ca11e79b0c592e011fcb23501c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-a, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-15T13:10:01.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Mar 15 13:10:01 smithi002 bash[104490]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-a 2024-03-15T13:10:01.503 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mon.a.service' 2024-03-15T13:10:01.547 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:10:01.547 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-15T13:10:01.547 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-15T13:10:01.547 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mon.b 2024-03-15T13:10:01.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:10:01 smithi149 systemd[1]: Stopping Ceph mon.b for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:10:02.130 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:10:01 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-b[35915]: 2024-03-15T13:10:01.952+0000 7ff77ae0c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:10:02.130 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:10:01 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-b[35915]: 2024-03-15T13:10:01.952+0000 7ff77ae0c700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-15T13:10:02.460 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:10:02 smithi149 podman[71257]: 2024-03-15 13:10:02.217404237 +0000 UTC m=+0.464401451 container died 1f242cf58ee4b96d06a60358d54c2932d61ddb5156d2dde17c02b950a0668b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-b, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-15T13:10:02.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:10:02 smithi149 podman[71257]: 2024-03-15 13:10:02.578052742 +0000 UTC m=+0.825049955 container cleanup 1f242cf58ee4b96d06a60358d54c2932d61ddb5156d2dde17c02b950a0668b65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-15T13:10:02.846 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 15 13:10:02 smithi149 bash[71257]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-b 2024-03-15T13:10:03.010 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mon.b.service' 2024-03-15T13:10:03.058 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:10:03.058 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-15T13:10:03.058 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-15T13:10:03.058 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mon.c 2024-03-15T13:10:03.383 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:10:03 smithi184 systemd[1]: Stopping Ceph mon.c for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:10:03.741 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:10:03 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-c[35967]: 2024-03-15T13:10:03.397+0000 7f4a5075d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:10:03.742 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:10:03 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-c[35967]: 2024-03-15T13:10:03.397+0000 7f4a5075d700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-15T13:10:03.742 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:10:03 smithi184 podman[72694]: 2024-03-15 13:10:03.459533523 +0000 UTC m=+0.180589131 container died 535014bd6255a2b3967fbf12cc203e451c87f2fee475a57a8f8a2bc9c54f27cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-15T13:10:04.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:10:03 smithi184 podman[72694]: 2024-03-15 13:10:03.849211589 +0000 UTC m=+0.570267202 container cleanup 535014bd6255a2b3967fbf12cc203e451c87f2fee475a57a8f8a2bc9c54f27cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d) 2024-03-15T13:10:04.057 INFO:journalctl@ceph.mon.c.smithi184.stdout:Mar 15 13:10:03 smithi184 bash[72694]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mon-c 2024-03-15T13:10:04.249 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mon.c.service' 2024-03-15T13:10:04.295 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:10:04.295 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-15T13:10:04.295 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-15T13:10:04.295 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.x 2024-03-15T13:10:04.609 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 13:10:04 smithi002 systemd[1]: Stopping Ceph mgr.x for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:10:04.860 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 13:10:04 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T13:10:04.624+0000 7fa99f80a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:10:04.860 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 13:10:04 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x[32294]: 2024-03-15T13:10:04.624+0000 7fa99f80a700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-15T13:10:04.860 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 13:10:04 smithi002 podman[104698]: 2024-03-15 13:10:04.656560363 +0000 UTC m=+0.148942769 container died 52c2b5b203c7614e0fc2f11fb21673dcbd10afc47d5e374a51dd406d58a5f067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-15T13:10:05.287 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 13:10:05 smithi002 podman[104698]: 2024-03-15 13:10:05.059881752 +0000 UTC m=+0.552264157 container cleanup 52c2b5b203c7614e0fc2f11fb21673dcbd10afc47d5e374a51dd406d58a5f067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-15T13:10:05.287 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Mar 15 13:10:05 smithi002 bash[104698]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-x 2024-03-15T13:10:05.419 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.x.service' 2024-03-15T13:10:05.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:10:05.457 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-15T13:10:05.457 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-15T13:10:05.457 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.y 2024-03-15T13:10:05.796 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 13:10:05 smithi149 systemd[1]: Stopping Ceph mgr.y for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:10:06.092 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 13:10:05 smithi149 podman[71393]: 2024-03-15 13:10:05.811595226 +0000 UTC m=+0.145349137 container died 1004fc3320ccecc18f7bed77f8d2229e4d9ff9b01628180f09cadbb9d87c0c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T13:10:06.358 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 13:10:06 smithi149 podman[71393]: 2024-03-15 13:10:06.200261895 +0000 UTC m=+0.534015806 container cleanup 1004fc3320ccecc18f7bed77f8d2229e4d9ff9b01628180f09cadbb9d87c0c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-15T13:10:06.358 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 15 13:10:06 smithi149 bash[71393]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-y 2024-03-15T13:10:06.652 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.y.service' 2024-03-15T13:10:06.689 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:10:06.689 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-15T13:10:06.689 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-15T13:10:06.689 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.z 2024-03-15T13:10:07.026 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 13:10:06 smithi184 systemd[1]: Stopping Ceph mgr.z for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:10:07.307 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 13:10:07 smithi184 podman[72830]: 2024-03-15 13:10:07.041771665 +0000 UTC m=+0.151178862 container died de6c34942d948b567abe1c09450e3dda095e14d3e2497dfbc6476c92a34a8f2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-03-15T13:10:07.682 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 13:10:07 smithi184 podman[72830]: 2024-03-15 13:10:07.463516008 +0000 UTC m=+0.572923186 container cleanup de6c34942d948b567abe1c09450e3dda095e14d3e2497dfbc6476c92a34a8f2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-15T13:10:07.682 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 13:10:07 smithi184 bash[72830]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z 2024-03-15T13:10:07.923 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.z.service' 2024-03-15T13:10:07.947 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 13:10:07 smithi184 podman[72844]: 2024-03-15 13:10:07.698580681 +0000 UTC m=+0.649792563 container remove de6c34942d948b567abe1c09450e3dda095e14d3e2497dfbc6476c92a34a8f2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mgr-z, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T13:10:07.947 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 13:10:07 smithi184 systemd[1]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-15T13:10:07.947 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 13:10:07 smithi184 systemd[1]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.z.service: Failed with result 'exit-code'. 2024-03-15T13:10:07.947 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 13:10:07 smithi184 systemd[1]: Stopped Ceph mgr.z for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T13:10:07.948 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Mar 15 13:10:07 smithi184 systemd[1]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mgr.z.service: Consumed 19.367s CPU time. 2024-03-15T13:10:07.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:10:07.963 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-15T13:10:07.963 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-15T13:10:07.963 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.0 2024-03-15T13:10:08.290 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 13:10:08 smithi002 systemd[1]: Stopping Ceph osd.0 for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:10:08.551 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 13:10:08 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-0[46898]: 2024-03-15T13:10:08.305+0000 7f71b4242700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:10:08.551 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 13:10:08 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-0[46898]: 2024-03-15T13:10:08.305+0000 7f71b4242700 -1 osd.0 104 *** Got signal Terminated *** 2024-03-15T13:10:08.551 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 13:10:08 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-0[46898]: 2024-03-15T13:10:08.305+0000 7f71b4242700 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-15T13:10:13.891 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 13:10:13 smithi002 podman[104821]: 2024-03-15 13:10:13.656376378 +0000 UTC m=+5.484973192 container died 604d60c59f2c9e3720b381756048124c6e7f69f04852081ef1819b99fa81cc96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-03-15T13:10:14.284 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 13:10:14 smithi002 podman[104821]: 2024-03-15 13:10:14.006751715 +0000 UTC m=+5.835348538 container cleanup 604d60c59f2c9e3720b381756048124c6e7f69f04852081ef1819b99fa81cc96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-15T13:10:14.284 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 13:10:14 smithi002 bash[104821]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-0 2024-03-15T13:10:14.551 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 13:10:14 smithi002 podman[104838]: 2024-03-15 13:10:14.300153274 +0000 UTC m=+0.641694932 container remove 604d60c59f2c9e3720b381756048124c6e7f69f04852081ef1819b99fa81cc96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-15T13:10:14.929 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 13:10:14 smithi002 podman[104928]: 2024-03-15 13:10:14.531554785 +0000 UTC m=+0.018666151 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T13:10:14.929 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 13:10:14 smithi002 podman[104928]: 2024-03-15 13:10:14.659316845 +0000 UTC m=+0.146428198 container create 12da37d98833af2840a317de3ebd427e878a3174cb6c1fd81cdecc6e090ee25b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-15T13:10:14.929 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 13:10:14 smithi002 podman[104928]: 2024-03-15 13:10:14.945368509 +0000 UTC m=+0.432479863 container init 12da37d98833af2840a317de3ebd427e878a3174cb6c1fd81cdecc6e090ee25b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-15T13:10:15.231 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 13:10:14 smithi002 podman[104928]: 2024-03-15 13:10:14.948789476 +0000 UTC m=+0.435900833 container start 12da37d98833af2840a317de3ebd427e878a3174cb6c1fd81cdecc6e090ee25b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-15T13:10:15.231 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 13:10:15 smithi002 podman[104928]: 2024-03-15 13:10:15.045514522 +0000 UTC m=+0.532625879 container attach 12da37d98833af2840a317de3ebd427e878a3174cb6c1fd81cdecc6e090ee25b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-15T13:10:15.510 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 15 13:10:15 smithi002 podman[104928]: 2024-03-15 13:10:15.247260565 +0000 UTC m=+0.734371919 container died 12da37d98833af2840a317de3ebd427e878a3174cb6c1fd81cdecc6e090ee25b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-0-deactivate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-15T13:10:15.910 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.0.service' 2024-03-15T13:10:15.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:10:15.951 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-15T13:10:15.952 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-15T13:10:15.952 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.1 2024-03-15T13:10:16.263 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 13:10:16 smithi149 systemd[1]: Stopping Ceph osd.1 for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:10:16.608 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 13:10:16 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-1[40532]: 2024-03-15T13:10:16.278+0000 7f6179b3c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:10:16.608 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 13:10:16 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-1[40532]: 2024-03-15T13:10:16.278+0000 7f6179b3c700 -1 osd.1 104 *** Got signal Terminated *** 2024-03-15T13:10:16.608 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 13:10:16 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-1[40532]: 2024-03-15T13:10:16.278+0000 7f6179b3c700 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-15T13:10:21.946 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 13:10:21 smithi149 podman[71515]: 2024-03-15 13:10:21.680809058 +0000 UTC m=+5.520250779 container died b85f013e2c6318348bfd1f1bd95e7e2e8e35e5bf800854b51fbcae2d7edc51e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-15T13:10:22.306 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 13:10:22 smithi149 podman[71515]: 2024-03-15 13:10:22.07086379 +0000 UTC m=+5.910305507 container cleanup b85f013e2c6318348bfd1f1bd95e7e2e8e35e5bf800854b51fbcae2d7edc51e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-1, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d) 2024-03-15T13:10:22.306 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 13:10:22 smithi149 bash[71515]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-1 2024-03-15T13:10:22.607 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 13:10:22 smithi149 podman[71530]: 2024-03-15 13:10:22.322505794 +0000 UTC m=+0.642677209 container remove b85f013e2c6318348bfd1f1bd95e7e2e8e35e5bf800854b51fbcae2d7edc51e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-1, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T13:10:22.858 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 13:10:22 smithi149 podman[71621]: 2024-03-15 13:10:22.623454536 +0000 UTC m=+0.102439596 container create 09e5a4cc220f6e22b509ae9e8375a6132c3d7464296bca1c7e241ac169960f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-15T13:10:22.858 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 13:10:22 smithi149 podman[71621]: 2024-03-15 13:10:22.539322265 +0000 UTC m=+0.018307340 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T13:10:22.858 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 13:10:22 smithi149 podman[71621]: 2024-03-15 13:10:22.850946169 +0000 UTC m=+0.329931230 container init 09e5a4cc220f6e22b509ae9e8375a6132c3d7464296bca1c7e241ac169960f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d) 2024-03-15T13:10:22.858 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 13:10:22 smithi149 podman[71621]: 2024-03-15 13:10:22.85393649 +0000 UTC m=+0.332921549 container start 09e5a4cc220f6e22b509ae9e8375a6132c3d7464296bca1c7e241ac169960f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-15T13:10:23.134 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 13:10:22 smithi149 podman[71621]: 2024-03-15 13:10:22.95120455 +0000 UTC m=+0.430189610 container attach 09e5a4cc220f6e22b509ae9e8375a6132c3d7464296bca1c7e241ac169960f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-15T13:10:23.409 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 15 13:10:23 smithi149 podman[71621]: 2024-03-15 13:10:23.151806949 +0000 UTC m=+0.630792015 container died 09e5a4cc220f6e22b509ae9e8375a6132c3d7464296bca1c7e241ac169960f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-1-deactivate, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-15T13:10:23.745 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.1.service' 2024-03-15T13:10:23.785 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:10:23.786 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-15T13:10:23.786 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-15T13:10:23.786 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.2 2024-03-15T13:10:24.093 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 13:10:23 smithi184 systemd[1]: Stopping Ceph osd.2 for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:10:24.557 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 13:10:24 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-2[41183]: 2024-03-15T13:10:24.108+0000 7f1e0df09700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:10:24.557 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 13:10:24 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-2[41183]: 2024-03-15T13:10:24.108+0000 7f1e0df09700 -1 osd.2 104 *** Got signal Terminated *** 2024-03-15T13:10:24.557 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 13:10:24 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-2[41183]: 2024-03-15T13:10:24.108+0000 7f1e0df09700 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-15T13:10:29.901 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 13:10:29 smithi184 podman[72952]: 2024-03-15 13:10:29.631001896 +0000 UTC m=+5.637238301 container died 672a332c1998fd6c81aace7a1d02cfa7c374d5de8db4f486ed806ba6b7ee6f4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-15T13:10:30.285 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 13:10:30 smithi184 podman[72952]: 2024-03-15 13:10:30.025931296 +0000 UTC m=+6.032167748 container cleanup 672a332c1998fd6c81aace7a1d02cfa7c374d5de8db4f486ed806ba6b7ee6f4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-15T13:10:30.285 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 13:10:30 smithi184 bash[72952]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-2 2024-03-15T13:10:30.557 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 13:10:30 smithi184 podman[72969]: 2024-03-15 13:10:30.302195129 +0000 UTC m=+0.667629181 container remove 672a332c1998fd6c81aace7a1d02cfa7c374d5de8db4f486ed806ba6b7ee6f4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T13:10:30.891 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 13:10:30 smithi184 podman[73062]: 2024-03-15 13:10:30.54762944 +0000 UTC m=+0.017401692 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T13:10:30.891 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 13:10:30 smithi184 podman[73062]: 2024-03-15 13:10:30.653281055 +0000 UTC m=+0.123053302 container create 422e400a27d5a432e67cd7c09d57fa06d2ec80e082076c39d20e9243301c9d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, CEPH_POINT_RELEASE=) 2024-03-15T13:10:31.181 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 13:10:30 smithi184 podman[73062]: 2024-03-15 13:10:30.907719 +0000 UTC m=+0.377491254 container init 422e400a27d5a432e67cd7c09d57fa06d2ec80e082076c39d20e9243301c9d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-15T13:10:31.181 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 13:10:30 smithi184 podman[73062]: 2024-03-15 13:10:30.911258507 +0000 UTC m=+0.381030757 container start 422e400a27d5a432e67cd7c09d57fa06d2ec80e082076c39d20e9243301c9d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-15T13:10:31.181 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 13:10:31 smithi184 podman[73062]: 2024-03-15 13:10:31.014415197 +0000 UTC m=+0.484187462 container attach 422e400a27d5a432e67cd7c09d57fa06d2ec80e082076c39d20e9243301c9d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T13:10:31.459 INFO:journalctl@ceph.osd.2.smithi184.stdout:Mar 15 13:10:31 smithi184 podman[73062]: 2024-03-15 13:10:31.197580644 +0000 UTC m=+0.667352896 container died 422e400a27d5a432e67cd7c09d57fa06d2ec80e082076c39d20e9243301c9d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-15T13:10:31.900 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.2.service' 2024-03-15T13:10:31.940 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:10:31.941 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-15T13:10:31.941 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-15T13:10:31.941 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.3 2024-03-15T13:10:32.261 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 13:10:32 smithi002 systemd[1]: Stopping Ceph osd.3 for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:10:32.551 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 13:10:32 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-3[53699]: 2024-03-15T13:10:32.277+0000 7fbb2fa26700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:10:32.551 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 13:10:32 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-3[53699]: 2024-03-15T13:10:32.277+0000 7fbb2fa26700 -1 osd.3 104 *** Got signal Terminated *** 2024-03-15T13:10:32.551 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 13:10:32 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-3[53699]: 2024-03-15T13:10:32.277+0000 7fbb2fa26700 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-15T13:10:34.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:34 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:34.499+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:35.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:35 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:35.375+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:35.807 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:35 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:35.531+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:35.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:35 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:35.477+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:36.802 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:36 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:36.394+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:36.807 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:36 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:36.564+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:36.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:36 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:36.498+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:37.751 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:37 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:37.423+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:37.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:37 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:37.489+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:38.036 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 13:10:37 smithi002 podman[105048]: 2024-03-15 13:10:37.766954409 +0000 UTC m=+5.615253573 container died 412edb00b7da0b9777dcb0d035732a81c52ee17ee477d26d11156cb95aaf4910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-3, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-15T13:10:38.057 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:37 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:37.606+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:38.301 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 13:10:38 smithi002 podman[105048]: 2024-03-15 13:10:38.152807739 +0000 UTC m=+6.001106902 container cleanup 412edb00b7da0b9777dcb0d035732a81c52ee17ee477d26d11156cb95aaf4910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-15T13:10:38.301 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 13:10:38 smithi002 bash[105048]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-3 2024-03-15T13:10:38.754 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 13:10:38 smithi002 podman[105065]: 2024-03-15 13:10:38.41272273 +0000 UTC m=+0.639316870 container remove 412edb00b7da0b9777dcb0d035732a81c52ee17ee477d26d11156cb95aaf4910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-15T13:10:38.755 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:38 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:38.431+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:38.829 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:38 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:38.592+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:38.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:38 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:38.450+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:39.041 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 13:10:38 smithi002 podman[105153]: 2024-03-15 13:10:38.674287137 +0000 UTC m=+0.017801816 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T13:10:39.041 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 13:10:38 smithi002 podman[105153]: 2024-03-15 13:10:38.788653044 +0000 UTC m=+0.132167743 container create 726b2307ed4b858f2223a2bf12692ddda5e82518b83a7c5c766f6487c3908a2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5) 2024-03-15T13:10:39.301 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 13:10:39 smithi002 podman[105153]: 2024-03-15 13:10:39.057850586 +0000 UTC m=+0.401365246 container init 726b2307ed4b858f2223a2bf12692ddda5e82518b83a7c5c766f6487c3908a2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-15T13:10:39.301 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 13:10:39 smithi002 podman[105153]: 2024-03-15 13:10:39.061477087 +0000 UTC m=+0.404991749 container start 726b2307ed4b858f2223a2bf12692ddda5e82518b83a7c5c766f6487c3908a2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-15T13:10:39.301 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 13:10:39 smithi002 podman[105153]: 2024-03-15 13:10:39.157925375 +0000 UTC m=+0.501440043 container attach 726b2307ed4b858f2223a2bf12692ddda5e82518b83a7c5c766f6487c3908a2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-15T13:10:39.307 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:38 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:38.844+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:39.653 INFO:journalctl@ceph.osd.3.smithi002.stdout:Mar 15 13:10:39 smithi002 podman[105153]: 2024-03-15 13:10:39.35903579 +0000 UTC m=+0.702550456 container died 726b2307ed4b858f2223a2bf12692ddda5e82518b83a7c5c766f6487c3908a2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-squid-aba3c1d, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-15T13:10:39.653 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:39 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:39.447+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:39.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:39 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:39.459+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:39.963 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.3.service' 2024-03-15T13:10:40.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:10:40.003 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-15T13:10:40.003 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-15T13:10:40.004 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.4 2024-03-15T13:10:40.057 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:39 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:39.623+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:40.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:39 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:39.816+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:40.308 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 13:10:40 smithi149 systemd[1]: Stopping Ceph osd.4 for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:10:40.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:40 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:40.438+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:40.608 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:40 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:40.471+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:40.608 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 13:10:40 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-4[47022]: 2024-03-15T13:10:40.322+0000 7faf2e3f9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:10:40.608 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 13:10:40 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-4[47022]: 2024-03-15T13:10:40.322+0000 7faf2e3f9700 -1 osd.4 104 *** Got signal Terminated *** 2024-03-15T13:10:40.608 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 13:10:40 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-4[47022]: 2024-03-15T13:10:40.322+0000 7faf2e3f9700 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-15T13:10:41.057 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:40 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:40.585+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:41.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:40 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:40.819+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:41.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:41 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:41.471+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:41.807 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:41 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:41.796+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:41.807 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:41 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:41.548+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:41.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:41 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:41.477+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:42.763 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:42 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:42.502+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:42.763 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:42 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:42.374+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:42.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:42 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:42.427+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:42.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:42 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:42.513+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:43.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:42 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:42.779+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:43.702 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:43 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:43.436+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:43.702 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:43 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:43.436+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:43.807 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:43 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:43.457+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:43.807 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:43 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:43.329+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:43.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:43 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:43.562+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:44.269 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:43 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:43.827+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:44.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:44 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:44.065+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:44.557 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:44 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:44.437+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:44.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:44 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:44.286+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:44.713 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:44 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:44.415+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:44.713 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:44 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:44.415+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:44.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:44 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:44.563+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:45.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:44 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:44.791+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:45.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:44 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:44.791+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:10:45.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:45 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:45.053+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:45.359 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 13:10:44 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-4[47022]: 2024-03-15T13:10:44.930+0000 7faf2a9ed700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.619350+0000 front 2024-03-15T13:10:20.619320+0000 (oldest deadline 2024-03-15T13:10:44.719520+0000) 2024-03-15T13:10:45.557 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:45 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:45.476+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:45.558 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:45 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:45.271+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:45.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:45 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:45.431+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:45.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:45 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:45.431+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:45.856 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:45 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:45.571+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:46.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:45 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:45.744+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:46.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:45 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:45.744+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:10:46.108 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 13:10:45 smithi149 podman[71741]: 2024-03-15 13:10:45.872595985 +0000 UTC m=+5.666005802 container died 1e1e54d8bd58b18951929d1c913bd1fd70880f0da92d1754bfc12d65c10b3555 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-4, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-15T13:10:46.108 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:46 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:46.092+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:46.500 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 13:10:46 smithi149 podman[71741]: 2024-03-15 13:10:46.257319654 +0000 UTC m=+6.050729470 container cleanup 1e1e54d8bd58b18951929d1c913bd1fd70880f0da92d1754bfc12d65c10b3555 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-4, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2) 2024-03-15T13:10:46.500 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 13:10:46 smithi149 bash[71741]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-4 2024-03-15T13:10:46.557 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:46 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:46.467+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:46.557 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:46 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:46.467+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.959539+0000 front 2024-03-15T13:10:20.959411+0000 (oldest deadline 2024-03-15T13:10:45.659509+0000) 2024-03-15T13:10:46.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:46 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:46.254+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:46.793 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 13:10:46 smithi149 podman[71756]: 2024-03-15 13:10:46.516943954 +0000 UTC m=+0.641335665 container remove 1e1e54d8bd58b18951929d1c913bd1fd70880f0da92d1754bfc12d65c10b3555 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-4, org.label-schema.build-date=20240307, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-03-15T13:10:46.793 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:46 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:46.601+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:46.793 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:46 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:46.601+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:10:46.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:46 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:46.429+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:46.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:46 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:46.429+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:47.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:46 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:46.756+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:47.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:46 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:46.756+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:10:47.070 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 13:10:46 smithi149 podman[71848]: 2024-03-15 13:10:46.712306886 +0000 UTC m=+0.018025702 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T13:10:47.070 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 13:10:46 smithi149 podman[71848]: 2024-03-15 13:10:46.851580723 +0000 UTC m=+0.157299534 container create 7ce86a64747ba9b1f15ca05e1f6b8c3285ea700f356f3a574bf3a4047a23696e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-4-deactivate, RELEASE=wip-lusov-qdb-squid-aba3c1d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-15T13:10:47.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:46 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:46.981+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.859481+0000 front 2024-03-15T13:10:20.859509+0000 (oldest deadline 2024-03-15T13:10:46.759702+0000) 2024-03-15T13:10:47.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:47 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:47.086+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:47.359 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 13:10:47 smithi149 podman[71848]: 2024-03-15 13:10:47.112244637 +0000 UTC m=+0.417963450 container init 7ce86a64747ba9b1f15ca05e1f6b8c3285ea700f356f3a574bf3a4047a23696e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-15T13:10:47.359 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 13:10:47 smithi149 podman[71848]: 2024-03-15 13:10:47.1153669 +0000 UTC m=+0.421085712 container start 7ce86a64747ba9b1f15ca05e1f6b8c3285ea700f356f3a574bf3a4047a23696e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-15T13:10:47.359 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 13:10:47 smithi149 podman[71848]: 2024-03-15 13:10:47.204034831 +0000 UTC m=+0.509753658 container attach 7ce86a64747ba9b1f15ca05e1f6b8c3285ea700f356f3a574bf3a4047a23696e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-15T13:10:47.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:47 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:47.212+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:47.557 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:47 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:47.446+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:47.557 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:47 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:47.446+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.959539+0000 front 2024-03-15T13:10:20.959411+0000 (oldest deadline 2024-03-15T13:10:45.659509+0000) 2024-03-15T13:10:47.691 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:47 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:47.595+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:47.691 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:47 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:47.595+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:10:47.691 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 15 13:10:47 smithi149 podman[71848]: 2024-03-15 13:10:47.40172956 +0000 UTC m=+0.707448376 container died 7ce86a64747ba9b1f15ca05e1f6b8c3285ea700f356f3a574bf3a4047a23696e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-15T13:10:47.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:47 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:47.398+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:47.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:47 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:47.398+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:47.977 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.4.service' 2024-03-15T13:10:48.020 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:10:48.020 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-15T13:10:48.020 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-15T13:10:48.020 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.5 2024-03-15T13:10:48.049 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:47 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:47.802+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:48.049 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:47 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:47.802+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:10:48.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:47 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:47.984+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.859481+0000 front 2024-03-15T13:10:20.859509+0000 (oldest deadline 2024-03-15T13:10:46.759702+0000) 2024-03-15T13:10:48.307 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:48 smithi184 systemd[1]: Stopping Ceph osd.5 for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:10:48.307 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:48.218+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:48.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:48 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:48.091+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:48.749 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:48.329+0000 7ff7d2410700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:10:48.749 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:48.329+0000 7ff7d2410700 -1 osd.5 104 *** Got signal Terminated *** 2024-03-15T13:10:48.749 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:48.329+0000 7ff7d2410700 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-15T13:10:48.749 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:48.453+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:48.750 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:48.453+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.959539+0000 front 2024-03-15T13:10:20.959411+0000 (oldest deadline 2024-03-15T13:10:45.659509+0000) 2024-03-15T13:10:48.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:48.418+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:48.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:48.418+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:48.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:48 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:48.593+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:48.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:48 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:48.593+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:10:49.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:48.765+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:49.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:48 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:48.765+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:10:49.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:48.942+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.859481+0000 front 2024-03-15T13:10:20.859509+0000 (oldest deadline 2024-03-15T13:10:46.759702+0000) 2024-03-15T13:10:49.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:49 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:49.079+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:49.557 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:49 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:49.415+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:49.557 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:49 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:49.415+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.959539+0000 front 2024-03-15T13:10:20.959411+0000 (oldest deadline 2024-03-15T13:10:45.659509+0000) 2024-03-15T13:10:49.558 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:49 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:49.250+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:49.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:49 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:49.405+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:49.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:49 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:49.405+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:49.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:49 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:49.612+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:49.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:49 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:49.612+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:10:50.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:49 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:49.809+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:50.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:49 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:49.809+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:10:50.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:49 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:49.954+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.859481+0000 front 2024-03-15T13:10:20.859509+0000 (oldest deadline 2024-03-15T13:10:46.759702+0000) 2024-03-15T13:10:50.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:49 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:49.954+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.760354+0000 front 2024-03-15T13:10:26.760283+0000 (oldest deadline 2024-03-15T13:10:49.660247+0000) 2024-03-15T13:10:50.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:50 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:50.085+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:50.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:50 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:50.231+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:50.558 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:50 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:50.457+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:50.558 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:50 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:50.457+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.959539+0000 front 2024-03-15T13:10:20.959411+0000 (oldest deadline 2024-03-15T13:10:45.659509+0000) 2024-03-15T13:10:50.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:50 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:50.449+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:50.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:50 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:50.449+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:50.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:50 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:50.449+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:10:50.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:50 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:50.599+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:50.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:50 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:50.599+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:10:50.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:50 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:50.599+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:10:51.177 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:50 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:50.835+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:51.178 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:50 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:50.835+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:10:51.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:50 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:50.912+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.859481+0000 front 2024-03-15T13:10:20.859509+0000 (oldest deadline 2024-03-15T13:10:46.759702+0000) 2024-03-15T13:10:51.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:50 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:50.912+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.760354+0000 front 2024-03-15T13:10:26.760283+0000 (oldest deadline 2024-03-15T13:10:49.660247+0000) 2024-03-15T13:10:51.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:51 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:51.059+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:51.489 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:51 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:51.192+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:51.778 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:51 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:51.505+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:51.778 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:51 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:51.505+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.959539+0000 front 2024-03-15T13:10:20.959411+0000 (oldest deadline 2024-03-15T13:10:45.659509+0000) 2024-03-15T13:10:51.778 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:51 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:51.505+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:25.659915+0000 front 2024-03-15T13:10:25.659969+0000 (oldest deadline 2024-03-15T13:10:50.959993+0000) 2024-03-15T13:10:51.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:51 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:51.483+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:51.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:51 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:51.483+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:51.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:51 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:51.483+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:10:51.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:51 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:51.595+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:51.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:51 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:51.595+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:10:51.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:51 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:51.595+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:10:52.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:51 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:51.793+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:52.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:51 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:51.793+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:10:52.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:51 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:51.793+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:10:52.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:51 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:51.957+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.859481+0000 front 2024-03-15T13:10:20.859509+0000 (oldest deadline 2024-03-15T13:10:46.759702+0000) 2024-03-15T13:10:52.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:51 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:51.957+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.760354+0000 front 2024-03-15T13:10:26.760283+0000 (oldest deadline 2024-03-15T13:10:49.660247+0000) 2024-03-15T13:10:52.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:52 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:52.095+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:52.520 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:52 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:52.195+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:52.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:52 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:52.445+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:52.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:52 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:52.445+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:52.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:52 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:52.445+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:10:52.807 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:52 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:52.758+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:52.807 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:52 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:52.758+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:10:52.807 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:52 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:52.758+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:10:52.807 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:52 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:52.535+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:09.857866+0000 front 2024-03-15T13:10:09.857946+0000 (oldest deadline 2024-03-15T13:10:35.157642+0000) 2024-03-15T13:10:52.807 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:52 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:52.535+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.959539+0000 front 2024-03-15T13:10:20.959411+0000 (oldest deadline 2024-03-15T13:10:45.659509+0000) 2024-03-15T13:10:52.807 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:52 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5[47671]: 2024-03-15T13:10:52.535+0000 7ff7cea04700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:25.659915+0000 front 2024-03-15T13:10:25.659969+0000 (oldest deadline 2024-03-15T13:10:50.959993+0000) 2024-03-15T13:10:53.101 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:52 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:52.628+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:53.101 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:52 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:52.628+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:10:53.101 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:52 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:52.628+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:10:53.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:52 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:52.977+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.859481+0000 front 2024-03-15T13:10:20.859509+0000 (oldest deadline 2024-03-15T13:10:46.759702+0000) 2024-03-15T13:10:53.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:52 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:52.977+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.760354+0000 front 2024-03-15T13:10:26.760283+0000 (oldest deadline 2024-03-15T13:10:49.660247+0000) 2024-03-15T13:10:53.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:53 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:53.116+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:53.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:53 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:53.116+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:10:53.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:53 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:53.239+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:53.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:53 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:53.448+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:53.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:53 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:53.448+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:53.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:53 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:53.448+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:10:54.057 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:53 smithi184 podman[73182]: 2024-03-15 13:10:53.807480188 +0000 UTC m=+5.584387678 container died 4cb29c7f652fc13ab02656941ea49db79600f85cf1b650dec9ace4f0442f2bb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-15T13:10:54.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:53 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:53.741+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:54.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:53 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:53.741+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:10:54.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:53 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:53.741+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:10:54.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:53 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:53.638+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:54.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:53 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:53.638+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:10:54.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:53 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:53.638+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:10:54.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:53 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:53.931+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.859481+0000 front 2024-03-15T13:10:20.859509+0000 (oldest deadline 2024-03-15T13:10:46.759702+0000) 2024-03-15T13:10:54.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:53 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:53.931+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.760354+0000 front 2024-03-15T13:10:26.760283+0000 (oldest deadline 2024-03-15T13:10:49.660247+0000) 2024-03-15T13:10:54.558 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:54 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:54.280+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:54.558 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:54 smithi184 podman[73182]: 2024-03-15 13:10:54.254603101 +0000 UTC m=+6.031510592 container cleanup 4cb29c7f652fc13ab02656941ea49db79600f85cf1b650dec9ace4f0442f2bb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-15T13:10:54.558 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:54 smithi184 bash[73182]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5 2024-03-15T13:10:54.608 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:54 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:54.137+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:54.608 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:54 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:54.137+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:10:54.764 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:54 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:54.465+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:54.764 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:54 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:54.465+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:54.764 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:54 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:54.465+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:10:54.807 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:54 smithi184 podman[73201]: 2024-03-15 13:10:54.57285681 +0000 UTC m=+0.765405240 container remove 4cb29c7f652fc13ab02656941ea49db79600f85cf1b650dec9ace4f0442f2bb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-15T13:10:54.807 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:54 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:54.693+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:54.807 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:54 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:54.693+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:10:54.808 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:54 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:54.693+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:10:55.051 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:54 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:54.905+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.859481+0000 front 2024-03-15T13:10:20.859509+0000 (oldest deadline 2024-03-15T13:10:46.759702+0000) 2024-03-15T13:10:55.051 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:54 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:54.905+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.760354+0000 front 2024-03-15T13:10:26.760283+0000 (oldest deadline 2024-03-15T13:10:49.660247+0000) 2024-03-15T13:10:55.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:54 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:54.632+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:55.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:54 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:54.632+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:10:55.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:54 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:54.632+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:10:55.268 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:54 smithi184 podman[73294]: 2024-03-15 13:10:54.820454653 +0000 UTC m=+0.018400592 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T13:10:55.269 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:54 smithi184 podman[73294]: 2024-03-15 13:10:54.924009874 +0000 UTC m=+0.121955846 container create 31285c9c31058a696456523c2e10eb16e2963041833f87fffca1a93762c4359b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-15T13:10:55.269 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:55 smithi184 podman[73294]: 2024-03-15 13:10:55.152083102 +0000 UTC m=+0.350029042 container init 31285c9c31058a696456523c2e10eb16e2963041833f87fffca1a93762c4359b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-15T13:10:55.269 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:55 smithi184 podman[73294]: 2024-03-15 13:10:55.155571368 +0000 UTC m=+0.353517293 container start 31285c9c31058a696456523c2e10eb16e2963041833f87fffca1a93762c4359b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-15T13:10:55.269 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:55 smithi184 podman[73294]: 2024-03-15 13:10:55.284971861 +0000 UTC m=+0.482917787 container attach 31285c9c31058a696456523c2e10eb16e2963041833f87fffca1a93762c4359b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-15T13:10:55.557 INFO:journalctl@ceph.osd.5.smithi184.stdout:Mar 15 13:10:55 smithi184 podman[73294]: 2024-03-15 13:10:55.446069558 +0000 UTC m=+0.644015482 container died 31285c9c31058a696456523c2e10eb16e2963041833f87fffca1a93762c4359b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-15T13:10:55.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:55 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:55.316+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:55.558 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:55 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:55.316+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:10:55.570 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:55 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:55.136+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:55.570 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:55 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:55.136+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:10:55.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:55 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:55.423+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:55.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:55 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:55.423+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:55.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:55 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:55.423+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:10:55.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:55 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:55.586+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:55.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:55 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:55.586+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:10:55.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:55 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:55.586+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:10:56.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:55 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:55.672+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:56.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:55 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:55.672+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:10:56.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:55 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:55.672+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:10:56.145 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.5.service' 2024-03-15T13:10:56.231 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:10:56.231 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-15T13:10:56.231 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-15T13:10:56.231 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.6 2024-03-15T13:10:56.261 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:55 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:55.928+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.859481+0000 front 2024-03-15T13:10:20.859509+0000 (oldest deadline 2024-03-15T13:10:46.759702+0000) 2024-03-15T13:10:56.261 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:55 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:55.928+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.760354+0000 front 2024-03-15T13:10:26.760283+0000 (oldest deadline 2024-03-15T13:10:49.660247+0000) 2024-03-15T13:10:56.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:56.091+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:56.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:56.091+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:10:56.531 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:56 smithi002 systemd[1]: Stopping Ceph osd.6 for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:10:56.532 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:56.387+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:56.532 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:56.387+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:56.532 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:56.387+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:10:56.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:56.305+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:56.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:56.305+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:10:56.803 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:56.547+0000 7f7ef8501700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:10:56.803 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:56.547+0000 7f7ef8501700 -1 osd.6 104 *** Got signal Terminated *** 2024-03-15T13:10:56.804 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:56.547+0000 7f7ef8501700 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-15T13:10:56.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:56.596+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:56.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:56.596+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:10:56.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:56 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:56.596+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:10:57.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:56.665+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:57.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:56.665+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:10:57.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:56 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:56.665+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:10:57.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:56.930+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.859481+0000 front 2024-03-15T13:10:20.859509+0000 (oldest deadline 2024-03-15T13:10:46.759702+0000) 2024-03-15T13:10:57.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:56 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:56.930+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.760354+0000 front 2024-03-15T13:10:26.760283+0000 (oldest deadline 2024-03-15T13:10:49.660247+0000) 2024-03-15T13:10:57.547 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:57 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:57.131+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:57.547 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:57 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:57.131+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:10:57.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:57 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:57.274+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:57.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:57 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:57.274+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:10:57.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:57 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:57.390+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:57.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:57 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:57.390+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:57.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:57 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:57.390+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:10:57.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:57 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:57.562+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:57.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:57 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:57.562+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:10:57.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:57 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:57.562+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:10:58.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:57 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:57.706+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:58.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:57 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:57.706+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:10:58.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:57 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:57.706+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:10:58.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:57 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:57.979+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.859481+0000 front 2024-03-15T13:10:20.859509+0000 (oldest deadline 2024-03-15T13:10:46.759702+0000) 2024-03-15T13:10:58.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:57 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:57.979+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.760354+0000 front 2024-03-15T13:10:26.760283+0000 (oldest deadline 2024-03-15T13:10:49.660247+0000) 2024-03-15T13:10:58.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:58 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:58.256+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:58.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:58 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:58.256+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:10:58.557 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:58 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:58.143+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:58.558 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:58 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:58.143+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:10:58.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:58 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:58.362+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:58.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:58 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:58.362+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:58.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:58 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:58.362+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:10:58.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:58 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:58.573+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:58.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:58 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:58.573+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:10:58.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:58 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:58.573+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:10:59.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:58 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:58.664+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:10:59.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:58 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:58.664+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:10:59.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:58 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:58.664+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:10:59.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:58 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:58.930+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.859481+0000 front 2024-03-15T13:10:20.859509+0000 (oldest deadline 2024-03-15T13:10:46.759702+0000) 2024-03-15T13:10:59.301 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:58 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:58.930+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.760354+0000 front 2024-03-15T13:10:26.760283+0000 (oldest deadline 2024-03-15T13:10:49.660247+0000) 2024-03-15T13:10:59.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:59 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:59.100+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:10:59.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:59 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:59.100+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:10:59.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:10:59 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:10:59.100+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.207265+0000 front 2024-03-15T13:10:36.207294+0000 (oldest deadline 2024-03-15T13:10:58.507305+0000) 2024-03-15T13:10:59.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:59 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:59.237+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:10:59.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:10:59 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:10:59.237+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:10:59.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:59 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:59.321+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:10:59.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:59 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:59.321+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:10:59.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:10:59 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:10:59.321+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:10:59.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:59 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:59.540+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:10:59.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:59 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:59.540+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:10:59.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:10:59 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:10:59.540+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:00.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:59 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:59.690+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:00.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:59 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:59.690+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:00.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:10:59 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:10:59.690+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:00.285 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:59 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:59.944+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.859481+0000 front 2024-03-15T13:10:20.859509+0000 (oldest deadline 2024-03-15T13:10:46.759702+0000) 2024-03-15T13:11:00.285 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:10:59 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:10:59.944+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.760354+0000 front 2024-03-15T13:10:26.760283+0000 (oldest deadline 2024-03-15T13:10:49.660247+0000) 2024-03-15T13:11:00.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:00 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:00.083+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:11:00.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:00 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:00.084+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:11:00.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:00 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:00.084+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.207265+0000 front 2024-03-15T13:10:36.207294+0000 (oldest deadline 2024-03-15T13:10:58.507305+0000) 2024-03-15T13:11:00.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:00 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:00.301+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:00.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:00 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:00.301+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:00.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:00 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:00.301+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:00.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:00 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:00.224+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:00.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:00 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:00.224+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:00.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:00 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:00.585+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:00.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:00 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:00.585+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:00.859 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:00 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:00.585+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:00.859 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:00 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:00.585+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:01.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:00 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:00.665+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:01.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:00 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:00.665+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:01.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:00 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:00.665+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:01.268 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:11:00 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:11:00.987+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.859481+0000 front 2024-03-15T13:10:20.859509+0000 (oldest deadline 2024-03-15T13:10:46.759702+0000) 2024-03-15T13:11:01.268 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:11:00 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:11:00.987+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.760354+0000 front 2024-03-15T13:10:26.760283+0000 (oldest deadline 2024-03-15T13:10:49.660247+0000) 2024-03-15T13:11:01.268 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:11:00 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6[60415]: 2024-03-15T13:11:00.987+0000 7f7ef0aed700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.663209+0000 front 2024-03-15T13:10:34.663241+0000 (oldest deadline 2024-03-15T13:10:59.963323+0000) 2024-03-15T13:11:01.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:01 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:01.054+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:11:01.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:01 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:01.054+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:11:01.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:01 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:01.054+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.207265+0000 front 2024-03-15T13:10:36.207294+0000 (oldest deadline 2024-03-15T13:10:58.507305+0000) 2024-03-15T13:11:01.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:01 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:01.284+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:01.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:01 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:01.284+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:01.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:01 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:01.284+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:01.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:01 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:01.284+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:01.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:01 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:01.224+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:01.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:01 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:01.224+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:01.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:01 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:01.572+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:01.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:01 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:01.572+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:01.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:01 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:01.572+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:01.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:01 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:01.572+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:02.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:01 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:01.684+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:02.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:01 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:01.684+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:02.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:01 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:01.684+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:02.282 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:11:02 smithi002 podman[105269]: 2024-03-15 13:11:02.009848841 +0000 UTC m=+5.564245384 container died 04addd96f0fabf967ac46912047ff3fa7cef88c3a2051bac16bf74697c62a995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-15T13:11:02.282 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:02 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:02.258+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:02.282 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:02 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:02.258+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:02.282 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:02 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:02.258+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:02.282 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:02 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:02.258+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:02.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:02 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:02.026+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:11:02.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:02 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:02.026+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:11:02.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:02 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:02.026+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.207265+0000 front 2024-03-15T13:10:36.207294+0000 (oldest deadline 2024-03-15T13:10:58.507305+0000) 2024-03-15T13:11:02.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:11:02 smithi002 podman[105269]: 2024-03-15 13:11:02.389268623 +0000 UTC m=+5.943665168 container cleanup 04addd96f0fabf967ac46912047ff3fa7cef88c3a2051bac16bf74697c62a995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d) 2024-03-15T13:11:02.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:11:02 smithi002 bash[105269]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6 2024-03-15T13:11:02.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:02 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:02.227+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:02.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:02 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:02.227+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:02.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:02 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:02.227+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:02.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:02 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:02.612+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:02.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:02 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:02.612+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:02.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:02 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:02.612+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:02.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:02 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:02.612+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:02.916 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:11:02 smithi002 podman[105287]: 2024-03-15 13:11:02.632555794 +0000 UTC m=+0.619860526 container remove 04addd96f0fabf967ac46912047ff3fa7cef88c3a2051bac16bf74697c62a995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-15T13:11:03.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:02 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:02.635+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:03.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:02 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:02.635+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:03.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:02 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:02.635+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:03.194 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:11:02 smithi002 podman[105377]: 2024-03-15 13:11:02.836239656 +0000 UTC m=+0.017499900 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T13:11:03.194 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:11:02 smithi002 podman[105377]: 2024-03-15 13:11:02.950129363 +0000 UTC m=+0.131389644 container create 81cb33d01f240043e932a140e862559dff0600f91bef665cea92324e4a19eba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-squid-aba3c1d) 2024-03-15T13:11:03.194 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:11:03 smithi002 podman[105377]: 2024-03-15 13:11:03.21073705 +0000 UTC m=+0.391997298 container init 81cb33d01f240043e932a140e862559dff0600f91bef665cea92324e4a19eba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-03-15T13:11:03.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:03 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:03.021+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:11:03.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:03 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:03.021+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:11:03.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:03 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:03.021+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.207265+0000 front 2024-03-15T13:10:36.207294+0000 (oldest deadline 2024-03-15T13:10:58.507305+0000) 2024-03-15T13:11:03.499 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:11:03 smithi002 podman[105377]: 2024-03-15 13:11:03.213944533 +0000 UTC m=+0.395204775 container start 81cb33d01f240043e932a140e862559dff0600f91bef665cea92324e4a19eba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-15T13:11:03.499 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:11:03 smithi002 podman[105377]: 2024-03-15 13:11:03.302517894 +0000 UTC m=+0.483778144 container attach 81cb33d01f240043e932a140e862559dff0600f91bef665cea92324e4a19eba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-15T13:11:03.500 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:03 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:03.299+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:03.500 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:03 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:03.299+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:03.500 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:03 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:03.299+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:03.500 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:03 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:03.299+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:03.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:03 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:03.249+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:03.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:03 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:03.249+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:03.558 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:03 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:03.249+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:03.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Mar 15 13:11:03 smithi002 podman[105377]: 2024-03-15 13:11:03.516348437 +0000 UTC m=+0.697608670 container died 81cb33d01f240043e932a140e862559dff0600f91bef665cea92324e4a19eba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-15T13:11:04.049 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:03 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:03.642+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:04.049 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:03 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:03.642+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:04.049 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:03 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:03.642+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:04.049 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:03 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:03.642+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:04.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:03 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:03.636+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:04.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:03 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:03.636+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:04.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:03 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:03.636+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:04.136 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.6.service' 2024-03-15T13:11:04.223 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:11:04.223 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-15T13:11:04.224 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-15T13:11:04.224 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.7 2024-03-15T13:11:04.331 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:04 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:04.065+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:11:04.331 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:04 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:04.065+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:11:04.332 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:04 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:04.065+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.207265+0000 front 2024-03-15T13:10:36.207294+0000 (oldest deadline 2024-03-15T13:10:58.507305+0000) 2024-03-15T13:11:04.332 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:04 smithi149 systemd[1]: Stopping Ceph osd.7 for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:11:04.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:04 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:04.321+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:04.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:04 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:04.321+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:04.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:04 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:04.321+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:04.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:04 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:04.321+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:04.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:04 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:04.261+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:04.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:04 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:04.261+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:04.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:04 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:04.261+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:04.597 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:04 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:04.551+0000 7f471f417700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:11:04.597 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:04 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:04.551+0000 7f471f417700 -1 osd.7 104 *** Got signal Terminated *** 2024-03-15T13:11:04.597 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:04 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:04.551+0000 7f471f417700 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-15T13:11:04.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:04 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:04.612+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:04.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:04 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:04.612+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:04.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:04 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:04.612+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:04.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:04 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:04.612+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:05.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:04 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:04.664+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:05.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:04 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:04.664+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:05.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:04 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:04.664+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:05.283 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:05 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:05.044+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:11:05.283 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:05 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:05.044+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:11:05.284 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:05 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:05.044+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.207265+0000 front 2024-03-15T13:10:36.207294+0000 (oldest deadline 2024-03-15T13:10:58.507305+0000) 2024-03-15T13:11:05.339 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:05 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:05.240+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:05.339 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:05 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:05.240+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:05.339 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:05 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:05.240+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:05.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:05 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:05.359+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:05.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:05 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:05.359+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:05.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:05 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:05.359+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:05.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:05 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:05.359+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:05.981 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:05 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:05.643+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:05.981 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:05 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:05.643+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:05.981 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:05 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:05.643+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:05.981 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:05 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:05.643+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:06.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:05 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:05.689+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:06.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:05 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:05.689+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:06.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:05 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:05.689+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:06.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:05 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:05.997+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:11:06.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:05 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:05.997+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:11:06.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:05 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:05.997+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.207265+0000 front 2024-03-15T13:10:36.207294+0000 (oldest deadline 2024-03-15T13:10:58.507305+0000) 2024-03-15T13:11:06.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:05 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:05.997+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:43.009173+0000 front 2024-03-15T13:10:43.009078+0000 (oldest deadline 2024-03-15T13:11:05.909199+0000) 2024-03-15T13:11:06.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:06 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:06.209+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:06.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:06 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:06.209+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:06.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:06 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:06.209+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:06.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:06 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:06.340+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:06.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:06 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:06.340+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:06.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:06 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:06.340+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:06.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:06 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:06.340+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:06.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:06 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:06.606+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:06.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:06 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:06.606+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:06.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:06 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:06.606+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:06.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:06 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:06.606+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:06.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:06 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:06.606+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:07.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:06 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:06.668+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:07.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:06 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:06.668+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:07.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:06 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:06.668+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:07.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:06 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:06.964+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:11:07.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:06 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:06.964+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:11:07.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:06 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:06.964+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.207265+0000 front 2024-03-15T13:10:36.207294+0000 (oldest deadline 2024-03-15T13:10:58.507305+0000) 2024-03-15T13:11:07.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:06 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:06.964+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:43.009173+0000 front 2024-03-15T13:10:43.009078+0000 (oldest deadline 2024-03-15T13:11:05.909199+0000) 2024-03-15T13:11:07.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:07 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:07.317+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:07.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:07 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:07.317+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:07.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:07 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:07.317+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:07.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:07 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:07.317+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:07.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:07 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:07.317+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:07.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:07 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:07.206+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:07.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:07 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:07.206+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:07.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:07 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:07.206+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:07.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:07 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:07.206+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.432317+0000 front 2024-03-15T13:10:41.432451+0000 (oldest deadline 2024-03-15T13:11:06.732251+0000) 2024-03-15T13:11:07.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:07 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:07.593+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:07.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:07 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:07.593+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:07.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:07 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:07.593+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:07.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:07 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:07.593+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:07.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:07 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:07.593+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:08.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:07 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:07.712+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:08.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:07 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:07.712+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:08.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:07 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:07.712+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:08.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:07 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:07.939+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:11:08.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:07 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:07.939+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:11:08.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:07 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:07.939+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.207265+0000 front 2024-03-15T13:10:36.207294+0000 (oldest deadline 2024-03-15T13:10:58.507305+0000) 2024-03-15T13:11:08.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:07 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:07.939+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:43.009173+0000 front 2024-03-15T13:10:43.009078+0000 (oldest deadline 2024-03-15T13:11:05.909199+0000) 2024-03-15T13:11:08.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:08 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:08.299+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:08.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:08 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:08.299+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:08.552 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:08 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:08.299+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:08.552 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:08 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:08.299+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:08.552 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:08 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:08.299+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:08.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:08 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:08.255+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:08.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:08 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:08.255+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:08.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:08 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:08.255+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:08.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:08 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:08.255+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.432317+0000 front 2024-03-15T13:10:41.432451+0000 (oldest deadline 2024-03-15T13:11:06.732251+0000) 2024-03-15T13:11:08.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:08 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:08.584+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:08.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:08 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:08.584+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:08.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:08 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:08.584+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:08.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:08 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:08.584+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:08.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:08 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:08.584+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:09.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:08 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:08.747+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:09.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:08 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:08.747+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:09.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:08 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:08.747+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:09.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:08 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:08.747+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:09.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:08 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:08.986+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:19.304295+0000 front 2024-03-15T13:10:19.304323+0000 (oldest deadline 2024-03-15T13:10:44.004488+0000) 2024-03-15T13:11:09.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:08 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:08.986+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:28.705822+0000 front 2024-03-15T13:10:28.705696+0000 (oldest deadline 2024-03-15T13:10:52.706209+0000) 2024-03-15T13:11:09.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:08 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:08.986+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.207265+0000 front 2024-03-15T13:10:36.207294+0000 (oldest deadline 2024-03-15T13:10:58.507305+0000) 2024-03-15T13:11:09.358 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:08 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7[53603]: 2024-03-15T13:11:08.986+0000 7f4717a03700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:43.009173+0000 front 2024-03-15T13:10:43.009078+0000 (oldest deadline 2024-03-15T13:11:05.909199+0000) 2024-03-15T13:11:09.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:09 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:09.267+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:09.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:09 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:09.267+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:09.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:09 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:09.267+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:09.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:09 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:09.267+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:09.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:09 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:09.267+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:09.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:09 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:09.230+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:09.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:09 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:09.230+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:09.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:09 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:09.230+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:09.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:09 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:09.230+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.432317+0000 front 2024-03-15T13:10:41.432451+0000 (oldest deadline 2024-03-15T13:11:06.732251+0000) 2024-03-15T13:11:09.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:09 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:09.575+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:09.859 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:09 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:09.575+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:09.859 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:09 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:09.575+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:09.859 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:09 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:09.575+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:09.859 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:09 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:09.575+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:10.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:09 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:09.724+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:10.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:09 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:09.724+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:10.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:09 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:09.724+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:10.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:09 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:09.724+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:10.211 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:09 smithi149 podman[71957]: 2024-03-15 13:11:09.948337836 +0000 UTC m=+5.510531666 container died e0d2e9aa7bcb6ca9529d8822a9d47d13cff9fe9bc313c735cf7d6137b569d21f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-15T13:11:10.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:10 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:10.220+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:10.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:10 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:10.220+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:10.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:10 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:10.220+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:10.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:10 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:10.220+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:10.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:10 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:10.220+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:10.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:10 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:10.262+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:10.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:10 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:10.262+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:10.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:10 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:10.262+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:10.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:10 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:10.262+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.432317+0000 front 2024-03-15T13:10:41.432451+0000 (oldest deadline 2024-03-15T13:11:06.732251+0000) 2024-03-15T13:11:10.577 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:10 smithi149 podman[71957]: 2024-03-15 13:11:10.318340294 +0000 UTC m=+5.880534124 container cleanup e0d2e9aa7bcb6ca9529d8822a9d47d13cff9fe9bc313c735cf7d6137b569d21f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d) 2024-03-15T13:11:10.577 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:10 smithi149 bash[71957]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7 2024-03-15T13:11:10.858 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:10 smithi149 podman[71985]: 2024-03-15 13:11:10.594483273 +0000 UTC m=+0.640891512 container remove e0d2e9aa7bcb6ca9529d8822a9d47d13cff9fe9bc313c735cf7d6137b569d21f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-15T13:11:10.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:10 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:10.604+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:10.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:10 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:10.604+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:10.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:10 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:10.604+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:10.858 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:10 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:10.604+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:10.859 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:10 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:10.604+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:11.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:10 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:10.738+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:11.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:10 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:10.738+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:11.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:10 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:10.738+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:11.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:10 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:10.738+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:11.181 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:10 smithi149 podman[72077]: 2024-03-15 13:11:10.836224018 +0000 UTC m=+0.018157758 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T13:11:11.181 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:10 smithi149 podman[72077]: 2024-03-15 13:11:10.962102165 +0000 UTC m=+0.144035893 container create ec9b15dfe2dd019661320b67ba7e8a7a9b49c576be5d944a8db19bac4f281a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-15T13:11:11.181 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:11 smithi149 podman[72077]: 2024-03-15 13:11:11.197725822 +0000 UTC m=+0.379659544 container init ec9b15dfe2dd019661320b67ba7e8a7a9b49c576be5d944a8db19bac4f281a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5, ceph=True) 2024-03-15T13:11:11.475 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:11 smithi149 podman[72077]: 2024-03-15 13:11:11.200872692 +0000 UTC m=+0.382806417 container start ec9b15dfe2dd019661320b67ba7e8a7a9b49c576be5d944a8db19bac4f281a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-15T13:11:11.475 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:11 smithi149 podman[72077]: 2024-03-15 13:11:11.289371296 +0000 UTC m=+0.471305058 container attach ec9b15dfe2dd019661320b67ba7e8a7a9b49c576be5d944a8db19bac4f281a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-15T13:11:11.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:11 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:11.232+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:11.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:11 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:11.232+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:11.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:11 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:11.232+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:11.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:11 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:11.232+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:11.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:11 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:11.232+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:11.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:11 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:11.268+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:11.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:11 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:11.268+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:11.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:11 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:11.268+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:11.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:11 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:11.268+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.432317+0000 front 2024-03-15T13:10:41.432451+0000 (oldest deadline 2024-03-15T13:11:06.732251+0000) 2024-03-15T13:11:11.784 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 15 13:11:11 smithi149 podman[72077]: 2024-03-15 13:11:11.492056911 +0000 UTC m=+0.673990638 container died ec9b15dfe2dd019661320b67ba7e8a7a9b49c576be5d944a8db19bac4f281a80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T13:11:11.785 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:11 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:11.622+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:11.785 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:11 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:11.622+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:11.785 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:11 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:11.622+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:11.785 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:11 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:11.622+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:11.785 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:11 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:11.622+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:12.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:11 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:11.760+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:12.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:11 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:11.760+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:12.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:11 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:11.760+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:12.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:11 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:11.760+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:12.088 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.7.service' 2024-03-15T13:11:12.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:11:12.131 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-15T13:11:12.131 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-15T13:11:12.132 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.8 2024-03-15T13:11:12.445 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:12 smithi184 systemd[1]: Stopping Ceph osd.8 for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:11:12.445 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:12 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:12.243+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:12.445 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:12 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:12.243+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:12.445 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:12 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:12.243+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:12.445 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:12 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:12.243+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.432317+0000 front 2024-03-15T13:10:41.432451+0000 (oldest deadline 2024-03-15T13:11:06.732251+0000) 2024-03-15T13:11:12.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:12 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:12.238+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:12.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:12 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:12.238+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:12.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:12 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:12.238+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:12.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:12 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:12.238+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:12.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:12 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:12.238+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:12.700 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:12 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:12.460+0000 7f259d36e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:11:12.700 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:12 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:12.460+0000 7f259d36e700 -1 osd.8 104 *** Got signal Terminated *** 2024-03-15T13:11:12.700 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:12 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:12.460+0000 7f259d36e700 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-15T13:11:13.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:12 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:12.715+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:13.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:12 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:12.715+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:13.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:12 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:12.715+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:13.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:12 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:12.715+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:13.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:12 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:12.663+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:13.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:12 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:12.663+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:13.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:12 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:12.663+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:13.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:12 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:12.663+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:13.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:12 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:12.663+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:13.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:13 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:13.253+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:13.553 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:13 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:13.253+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:13.553 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:13 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:13.253+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:13.553 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:13 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:13.253+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:13.553 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:13 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:13.253+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:13.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:13 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:13.275+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:13.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:13 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:13.275+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:13.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:13 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:13.275+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:13.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:13 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:13.275+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.432317+0000 front 2024-03-15T13:10:41.432451+0000 (oldest deadline 2024-03-15T13:11:06.732251+0000) 2024-03-15T13:11:14.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:13 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:13.691+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:14.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:13 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:13.691+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:14.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:13 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:13.691+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:14.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:13 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:13.691+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:14.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:13 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:13.662+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:14.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:13 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:13.662+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:14.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:13 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:13.662+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:14.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:13 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:13.662+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:14.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:13 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:13.662+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:14.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:14 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:14.263+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:14.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:14 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:14.263+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:14.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:14 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:14.263+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:14.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:14 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:14.263+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:14.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:14 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:14.263+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:14.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:14 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:14.263+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.742756+0000 front 2024-03-15T13:10:50.742636+0000 (oldest deadline 2024-03-15T13:11:14.242569+0000) 2024-03-15T13:11:14.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:14 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:14.239+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:14.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:14 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:14.239+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:14.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:14 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:14.239+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:14.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:14 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:14.239+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.432317+0000 front 2024-03-15T13:10:41.432451+0000 (oldest deadline 2024-03-15T13:11:06.732251+0000) 2024-03-15T13:11:15.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:14 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:14.688+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:15.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:14 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:14.688+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:15.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:14 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:14.688+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:15.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:14 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:14.688+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:15.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:14 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:14.690+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:15.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:14 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:14.690+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:15.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:14 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:14.690+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:15.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:14 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:14.690+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:15.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:14 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:14.690+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:15.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:14 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:14.690+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:15.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:15 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:15.299+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:15.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:15 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:15.299+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:15.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:15 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:15.299+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:15.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:15 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:15.299+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:15.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:15 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:15.299+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:15.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:15 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:15.299+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.742756+0000 front 2024-03-15T13:10:50.742636+0000 (oldest deadline 2024-03-15T13:11:14.242569+0000) 2024-03-15T13:11:15.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:15 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:15.269+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:15.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:15 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:15.269+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:15.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:15 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:15.269+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:15.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:15 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:15.269+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.432317+0000 front 2024-03-15T13:10:41.432451+0000 (oldest deadline 2024-03-15T13:11:06.732251+0000) 2024-03-15T13:11:16.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:15 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:15.706+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:16.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:15 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:15.706+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:16.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:15 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:15.706+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:16.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:15 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:15.706+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:16.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:15 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:15.707+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:16.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:15 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:15.707+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:16.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:15 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:15.707+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:16.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:15 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:15.707+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:16.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:15 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:15.707+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:16.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:15 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:15.707+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:16.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:16 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:16.278+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:16.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:16 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:16.278+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:16.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:16 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:16.278+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:16.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:16 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:16.278+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:16.552 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:16 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:16.278+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:16.552 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:16 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:16.278+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.742756+0000 front 2024-03-15T13:10:50.742636+0000 (oldest deadline 2024-03-15T13:11:14.242569+0000) 2024-03-15T13:11:16.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:16 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:16.301+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:16.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:16 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:16.301+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:16.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:16 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:16.301+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:16.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:16 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:16.301+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.432317+0000 front 2024-03-15T13:10:41.432451+0000 (oldest deadline 2024-03-15T13:11:06.732251+0000) 2024-03-15T13:11:16.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:16 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:16.301+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:53.034280+0000 front 2024-03-15T13:10:53.034418+0000 (oldest deadline 2024-03-15T13:11:15.834851+0000) 2024-03-15T13:11:17.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:16 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:16.684+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:17.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:16 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:16.684+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:17.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:16 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:16.684+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:17.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:16 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:16.684+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:17.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:16 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:16.741+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:17.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:16 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:16.741+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:17.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:16 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:16.741+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:17.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:16 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:16.741+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:17.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:16 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:16.741+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:17.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:16 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:16.741+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:17.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:17 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:17.316+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:17.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:17 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:17.316+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:17.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:17 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:17.316+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:17.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:17 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:17.316+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:17.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:17 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:17.316+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:17.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:17 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:17.316+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.742756+0000 front 2024-03-15T13:10:50.742636+0000 (oldest deadline 2024-03-15T13:11:14.242569+0000) 2024-03-15T13:11:17.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:17 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:17.263+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:16.328644+0000 front 2024-03-15T13:10:16.328588+0000 (oldest deadline 2024-03-15T13:10:42.228589+0000) 2024-03-15T13:11:17.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:17 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:17.263+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:24.430188+0000 front 2024-03-15T13:10:24.430217+0000 (oldest deadline 2024-03-15T13:10:54.430816+0000) 2024-03-15T13:11:17.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:17 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:17.263+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.431548+0000 front 2024-03-15T13:10:34.431497+0000 (oldest deadline 2024-03-15T13:11:01.431721+0000) 2024-03-15T13:11:17.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:17 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:17.263+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.432317+0000 front 2024-03-15T13:10:41.432451+0000 (oldest deadline 2024-03-15T13:11:06.732251+0000) 2024-03-15T13:11:17.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:17 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8[54649]: 2024-03-15T13:11:17.263+0000 7f2599962700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:53.034280+0000 front 2024-03-15T13:10:53.034418+0000 (oldest deadline 2024-03-15T13:11:15.834851+0000) 2024-03-15T13:11:17.943 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:17 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:17.699+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:17.943 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:17 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:17.699+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:17.943 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:17 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:17.699+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:17.943 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:17 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:17.699+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:17.943 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:17 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:17.699+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:18.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:17 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:17.765+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:18.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:17 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:17.765+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:18.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:17 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:17.765+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:18.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:17 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:17.765+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:18.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:17 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:17.765+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:18.109 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:17 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:17.765+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:18.220 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:17 smithi184 podman[73413]: 2024-03-15 13:11:17.959720417 +0000 UTC m=+5.614821688 container died 9fbfb93e36457fb788a0514ad3f96e9f2eb884b62386e0678195df3e74e10b11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-qdb-squid-aba3c1d) 2024-03-15T13:11:18.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:18 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:18.313+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:18.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:18 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:18.313+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:18.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:18 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:18.313+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:18.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:18 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:18.313+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:18.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:18 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:18.313+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:18.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:18 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:18.313+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.742756+0000 front 2024-03-15T13:10:50.742636+0000 (oldest deadline 2024-03-15T13:11:14.242569+0000) 2024-03-15T13:11:18.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:18 smithi184 podman[73413]: 2024-03-15 13:11:18.327171591 +0000 UTC m=+5.982272860 container cleanup 9fbfb93e36457fb788a0514ad3f96e9f2eb884b62386e0678195df3e74e10b11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-15T13:11:18.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:18 smithi184 bash[73413]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8 2024-03-15T13:11:18.936 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:18 smithi184 podman[73430]: 2024-03-15 13:11:18.611674442 +0000 UTC m=+0.653324838 container remove 9fbfb93e36457fb788a0514ad3f96e9f2eb884b62386e0678195df3e74e10b11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-lusov-qdb-squid-aba3c1d, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T13:11:18.936 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:18 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:18.714+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:18.937 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:18 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:18.714+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:18.937 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:18 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:18.714+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:18.937 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:18 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:18.714+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:18.937 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:18 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:18.714+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:19.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:18 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:18.738+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:19.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:18 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:18.738+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:19.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:18 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:18.738+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:19.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:18 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:18.738+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:19.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:18 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:18.738+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:19.109 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:18 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:18.738+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:19.256 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:18 smithi184 podman[73522]: 2024-03-15 13:11:18.857348839 +0000 UTC m=+0.018711501 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T13:11:19.256 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:18 smithi184 podman[73522]: 2024-03-15 13:11:18.979197953 +0000 UTC m=+0.140560611 container create 0d639efec5c950776b824e7c6b0b6ed3c319889ad46f55c4ee69fc9339fce58c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T13:11:19.256 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:19 smithi184 podman[73522]: 2024-03-15 13:11:19.198373931 +0000 UTC m=+0.359736589 container init 0d639efec5c950776b824e7c6b0b6ed3c319889ad46f55c4ee69fc9339fce58c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d) 2024-03-15T13:11:19.257 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:19 smithi184 podman[73522]: 2024-03-15 13:11:19.201466534 +0000 UTC m=+0.362829192 container start 0d639efec5c950776b824e7c6b0b6ed3c319889ad46f55c4ee69fc9339fce58c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-15T13:11:19.257 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:19 smithi184 podman[73522]: 2024-03-15 13:11:19.27314812 +0000 UTC m=+0.434510791 container attach 0d639efec5c950776b824e7c6b0b6ed3c319889ad46f55c4ee69fc9339fce58c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-03-15T13:11:19.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:19 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:19.302+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:19.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:19 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:19.302+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:19.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:19 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:19.302+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:19.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:19 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:19.302+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:19.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:19 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:19.302+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:19.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:19 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:19.302+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.742756+0000 front 2024-03-15T13:10:50.742636+0000 (oldest deadline 2024-03-15T13:11:14.242569+0000) 2024-03-15T13:11:19.557 INFO:journalctl@ceph.osd.8.smithi184.stdout:Mar 15 13:11:19 smithi184 podman[73522]: 2024-03-15 13:11:19.490365687 +0000 UTC m=+0.651728353 container died 0d639efec5c950776b824e7c6b0b6ed3c319889ad46f55c4ee69fc9339fce58c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-8-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-15T13:11:20.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:19 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:19.670+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:20.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:19 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:19.670+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:20.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:19 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:19.670+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:20.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:19 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:19.670+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:20.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:19 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:19.670+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:20.105 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.8.service' 2024-03-15T13:11:20.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:19 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:19.724+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:20.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:19 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:19.724+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:20.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:19 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:19.724+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:20.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:19 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:19.724+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:20.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:19 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:19.724+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:20.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:19 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:19.724+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:20.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:11:20.147 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-15T13:11:20.147 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-15T13:11:20.147 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.9 2024-03-15T13:11:20.446 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:20 smithi002 systemd[1]: Stopping Ceph osd.9 for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:11:20.446 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:20 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:20.338+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:20.446 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:20 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:20.338+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:20.447 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:20 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:20.338+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:20.447 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:20 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:20.338+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:20.447 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:20 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:20.338+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:20.447 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:20 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:20.338+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.742756+0000 front 2024-03-15T13:10:50.742636+0000 (oldest deadline 2024-03-15T13:11:14.242569+0000) 2024-03-15T13:11:20.447 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:20 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:20.461+0000 7fd12c708700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:11:20.447 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:20 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:20.461+0000 7fd12c708700 -1 osd.9 104 *** Got signal Terminated *** 2024-03-15T13:11:20.447 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:20 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:20.461+0000 7fd12c708700 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-15T13:11:21.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:20 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:20.699+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:21.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:20 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:20.699+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:21.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:20 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:20.699+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:21.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:20 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:20.699+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:21.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:20 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:20.699+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:21.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:20 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:20.773+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:21.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:20 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:20.773+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:21.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:20 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:20.773+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:21.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:20 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:20.773+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:21.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:20 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:20.773+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:21.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:20 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:20.773+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:21.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:21 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:21.361+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:21.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:21 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:21.361+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:21.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:21 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:21.361+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:21.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:21 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:21.361+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:21.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:21 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:21.361+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:21.802 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:21 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:21.361+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.742756+0000 front 2024-03-15T13:10:50.742636+0000 (oldest deadline 2024-03-15T13:11:14.242569+0000) 2024-03-15T13:11:22.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:21 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:21.653+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:22.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:21 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:21.653+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:22.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:21 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:21.653+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:22.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:21 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:21.653+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:22.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:21 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:21.653+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:22.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:21 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:21.754+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:22.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:21 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:21.754+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:22.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:21 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:21.754+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:22.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:21 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:21.754+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:22.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:21 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:21.754+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:22.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:21 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:21.754+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:22.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:22 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:22.412+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:22.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:22 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:22.412+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:22.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:22 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:22.412+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:22.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:22 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:22.412+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:22.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:22 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:22.412+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:22.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:22 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:22.412+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.742756+0000 front 2024-03-15T13:10:50.742636+0000 (oldest deadline 2024-03-15T13:11:14.242569+0000) 2024-03-15T13:11:23.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:22 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:22.661+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:23.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:22 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:22.661+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:23.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:22 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:22.661+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:23.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:22 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:22.661+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:23.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:22 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:22.661+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:23.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:22 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:22.714+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:23.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:22 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:22.714+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:23.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:22 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:22.714+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:23.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:22 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:22.714+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:23.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:22 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:22.714+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:23.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:22 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:22.714+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:23.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:23 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:23.401+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:23.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:23 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:23.401+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:23.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:23 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:23.401+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:23.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:23 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:23.401+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:23.802 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:23 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:23.401+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:23.802 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:23 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:23.402+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.742756+0000 front 2024-03-15T13:10:50.742636+0000 (oldest deadline 2024-03-15T13:11:14.242569+0000) 2024-03-15T13:11:24.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:23 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:23.658+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:24.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:23 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:23.658+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:24.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:23 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:23.658+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:24.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:23 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:23.658+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:24.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:23 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:23.658+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:24.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:23 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:23.742+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:24.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:23 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:23.742+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:24.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:23 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:23.742+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:24.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:23 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:23.742+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:24.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:23 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:23.742+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:24.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:23 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:23.742+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:24.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:24 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:24.384+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:24.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:24 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:24.384+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:24.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:24 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:24.384+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:24.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:24 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:24.384+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:24.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:24 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:24.384+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:24.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:24 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:24.384+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.742756+0000 front 2024-03-15T13:10:50.742636+0000 (oldest deadline 2024-03-15T13:11:14.242569+0000) 2024-03-15T13:11:25.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:24 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:24.686+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:25.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:24 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:24.686+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:25.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:24 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:24.686+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:25.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:24 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:24.686+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:25.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:24 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:24.686+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:25.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:24 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:24.686+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:25.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:24 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:24.741+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:25.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:24 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:24.741+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:25.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:24 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:24.741+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:25.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:24 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:24.741+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:25.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:24 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:24.741+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:25.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:24 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:24.741+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:25.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:25 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:25.395+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.936250+0000 front 2024-03-15T13:10:12.936224+0000 (oldest deadline 2024-03-15T13:10:35.236331+0000) 2024-03-15T13:11:25.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:25 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:25.395+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:18.637936+0000 front 2024-03-15T13:10:18.638243+0000 (oldest deadline 2024-03-15T13:10:43.337950+0000) 2024-03-15T13:11:25.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:25 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:25.395+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.839043+0000 front 2024-03-15T13:10:26.838958+0000 (oldest deadline 2024-03-15T13:10:49.739042+0000) 2024-03-15T13:11:25.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:25 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:25.395+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:36.140324+0000 front 2024-03-15T13:10:36.140382+0000 (oldest deadline 2024-03-15T13:11:00.840468+0000) 2024-03-15T13:11:25.802 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:25 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:25.395+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.941334+0000 front 2024-03-15T13:10:41.941368+0000 (oldest deadline 2024-03-15T13:11:06.641494+0000) 2024-03-15T13:11:25.802 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:25 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9[67317]: 2024-03-15T13:11:25.395+0000 7fd124cf4700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.742756+0000 front 2024-03-15T13:10:50.742636+0000 (oldest deadline 2024-03-15T13:11:14.242569+0000) 2024-03-15T13:11:26.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:25 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:25.674+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:26.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:25 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:25.674+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:26.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:25 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:25.674+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:26.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:25 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:25.674+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:26.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:25 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:25.674+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:26.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:25 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:25.674+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:26.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:25 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:25.708+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:26.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:25 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:25.708+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:26.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:25 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:25.708+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:26.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:25 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:25.708+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:26.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:25 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:25.708+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:26.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:25 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:25.708+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:26.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:25 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:25.708+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.795890+0000 front 2024-03-15T13:11:00.795963+0000 (oldest deadline 2024-03-15T13:11:25.495656+0000) 2024-03-15T13:11:26.278 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:26 smithi002 podman[105494]: 2024-03-15 13:11:26.005468436 +0000 UTC m=+5.644382157 container died fcd0926988c46e5d1c6cfae2511e111399216f8e03ec4c96b68f3e121e9db2a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-15T13:11:26.551 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:26 smithi002 podman[105494]: 2024-03-15 13:11:26.295369213 +0000 UTC m=+5.934282933 container cleanup fcd0926988c46e5d1c6cfae2511e111399216f8e03ec4c96b68f3e121e9db2a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-15T13:11:26.552 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:26 smithi002 bash[105494]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9 2024-03-15T13:11:26.552 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:26 smithi002 podman[105509]: 2024-03-15 13:11:26.513240069 +0000 UTC m=+0.505874515 container remove fcd0926988c46e5d1c6cfae2511e111399216f8e03ec4c96b68f3e121e9db2a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d) 2024-03-15T13:11:27.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:26 smithi002 podman[105601]: 2024-03-15 13:11:26.799536688 +0000 UTC m=+0.108673967 container create 8355b9a8cf9bfd5cec3edf4186dc91324895bf5cb48b66d834df6bc9f94b562f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-15T13:11:27.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:26 smithi002 podman[105601]: 2024-03-15 13:11:26.708962869 +0000 UTC m=+0.018100094 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T13:11:27.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:26 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:26.657+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:27.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:26 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:26.657+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:27.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:26 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:26.657+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:27.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:26 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:26.657+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:27.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:26 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:26.657+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:27.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:26 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:26.657+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:27.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:26 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:26.749+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:27.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:26 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:26.749+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:27.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:26 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:26.749+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:27.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:26 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:26.749+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:27.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:26 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:26.749+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:27.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:26 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:26.749+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:27.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:26 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:26.749+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.795890+0000 front 2024-03-15T13:11:00.795963+0000 (oldest deadline 2024-03-15T13:11:25.495656+0000) 2024-03-15T13:11:27.363 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:27 smithi002 podman[105601]: 2024-03-15 13:11:27.085103842 +0000 UTC m=+0.394241072 container init 8355b9a8cf9bfd5cec3edf4186dc91324895bf5cb48b66d834df6bc9f94b562f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9-deactivate, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-15T13:11:27.364 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:27 smithi002 podman[105601]: 2024-03-15 13:11:27.088350236 +0000 UTC m=+0.397487463 container start 8355b9a8cf9bfd5cec3edf4186dc91324895bf5cb48b66d834df6bc9f94b562f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-15T13:11:27.364 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:27 smithi002 podman[105601]: 2024-03-15 13:11:27.168315625 +0000 UTC m=+0.477452854 container attach 8355b9a8cf9bfd5cec3edf4186dc91324895bf5cb48b66d834df6bc9f94b562f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-15T13:11:27.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Mar 15 13:11:27 smithi002 podman[105601]: 2024-03-15 13:11:27.380705337 +0000 UTC m=+0.689842547 container died 8355b9a8cf9bfd5cec3edf4186dc91324895bf5cb48b66d834df6bc9f94b562f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-9-deactivate, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-15T13:11:27.878 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.9.service' 2024-03-15T13:11:27.918 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:11:27.918 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-15T13:11:27.918 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-15T13:11:27.919 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.10 2024-03-15T13:11:27.949 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:27 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:27.750+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:27.950 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:27 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:27.750+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:27.950 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:27 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:27.750+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:27.950 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:27 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:27.750+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:27.950 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:27 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:27.750+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:27.950 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:27 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:27.750+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:27.950 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:27 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:27.750+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.795890+0000 front 2024-03-15T13:11:00.795963+0000 (oldest deadline 2024-03-15T13:11:25.495656+0000) 2024-03-15T13:11:28.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:27 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:27.653+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:28.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:27 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:27.653+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:28.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:27 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:27.653+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:28.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:27 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:27.653+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:28.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:27 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:27.653+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:28.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:27 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:27.653+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:28.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:27 smithi149 systemd[1]: Stopping Ceph osd.10 for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:11:28.608 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:28 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:28.253+0000 7f6d680a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:11:28.608 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:28 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:28.253+0000 7f6d680a4700 -1 osd.10 104 *** Got signal Terminated *** 2024-03-15T13:11:28.608 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:28 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:28.253+0000 7f6d680a4700 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-15T13:11:29.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:28 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:28.684+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:29.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:28 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:28.684+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:29.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:28 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:28.684+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:29.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:28 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:28.684+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:29.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:28 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:28.684+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:29.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:28 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:28.684+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:29.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:28 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:28.735+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:29.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:28 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:28.735+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:29.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:28 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:28.735+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:29.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:28 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:28.735+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:29.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:28 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:28.735+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:29.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:28 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:28.735+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:29.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:28 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:28.735+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.795890+0000 front 2024-03-15T13:11:00.795963+0000 (oldest deadline 2024-03-15T13:11:25.495656+0000) 2024-03-15T13:11:30.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:29 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:29.676+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:30.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:29 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:29.676+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:30.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:29 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:29.676+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:30.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:29 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:29.676+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:30.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:29 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:29.676+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:30.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:29 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:29.676+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:30.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:29 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:29.725+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:30.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:29 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:29.725+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:30.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:29 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:29.725+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:30.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:29 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:29.725+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:30.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:29 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:29.725+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:30.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:29 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:29.725+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:30.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:29 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:29.725+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.795890+0000 front 2024-03-15T13:11:00.795963+0000 (oldest deadline 2024-03-15T13:11:25.495656+0000) 2024-03-15T13:11:31.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:30 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:30.700+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:31.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:30 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:30.700+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:31.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:30 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:30.700+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:31.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:30 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:30.700+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:31.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:30 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:30.700+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:31.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:30 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:30.700+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:31.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:30 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:30.700+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-15T13:11:07.693310+0000 front 2024-03-15T13:11:07.693203+0000 (oldest deadline 2024-03-15T13:11:29.993039+0000) 2024-03-15T13:11:31.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:30 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:30.748+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:31.109 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:30 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:30.748+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:31.109 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:30 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:30.748+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:31.109 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:30 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:30.748+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:31.109 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:30 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:30.748+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:31.109 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:30 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:30.748+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:31.109 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:30 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:30.748+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.795890+0000 front 2024-03-15T13:11:00.795963+0000 (oldest deadline 2024-03-15T13:11:25.495656+0000) 2024-03-15T13:11:32.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:31 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:31.709+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:32.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:31 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:31.709+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:32.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:31 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:31.709+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:32.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:31 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:31.709+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:32.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:31 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:31.709+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:32.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:31 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:31.709+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:32.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:31 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:31.709+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-15T13:11:07.693310+0000 front 2024-03-15T13:11:07.693203+0000 (oldest deadline 2024-03-15T13:11:29.993039+0000) 2024-03-15T13:11:32.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:31 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:31.738+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:32.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:31 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:31.738+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:32.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:31 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:31.738+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:32.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:31 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:31.738+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:32.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:31 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:31.738+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:32.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:31 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:31.738+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:32.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:31 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:31.738+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.795890+0000 front 2024-03-15T13:11:00.795963+0000 (oldest deadline 2024-03-15T13:11:25.495656+0000) 2024-03-15T13:11:33.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:32 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:32.711+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:33.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:32 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:32.711+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:33.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:32 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:32.711+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:33.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:32 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:32.711+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:33.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:32 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:32.711+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:33.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:32 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:32.711+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:33.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:32 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:32.711+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-15T13:11:07.693310+0000 front 2024-03-15T13:11:07.693203+0000 (oldest deadline 2024-03-15T13:11:29.993039+0000) 2024-03-15T13:11:33.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:32 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:32.699+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.786534+0000 front 2024-03-15T13:10:12.786521+0000 (oldest deadline 2024-03-15T13:10:33.886555+0000) 2024-03-15T13:11:33.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:32 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:32.699+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.888104+0000 front 2024-03-15T13:10:20.888127+0000 (oldest deadline 2024-03-15T13:10:46.188257+0000) 2024-03-15T13:11:33.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:32 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:32.699+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.689779+0000 front 2024-03-15T13:10:26.689643+0000 (oldest deadline 2024-03-15T13:10:50.189423+0000) 2024-03-15T13:11:33.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:32 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:32.699+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-03-15T13:10:34.890325+0000 front 2024-03-15T13:10:34.890387+0000 (oldest deadline 2024-03-15T13:11:00.190361+0000) 2024-03-15T13:11:33.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:32 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:32.699+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.891109+0000 front 2024-03-15T13:10:41.891129+0000 (oldest deadline 2024-03-15T13:11:05.991308+0000) 2024-03-15T13:11:33.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:32 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:32.699+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:50.592999+0000 front 2024-03-15T13:10:50.592923+0000 (oldest deadline 2024-03-15T13:11:14.092797+0000) 2024-03-15T13:11:33.108 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:32 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10[60208]: 2024-03-15T13:11:32.699+0000 7f6d60690700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.795890+0000 front 2024-03-15T13:11:00.795963+0000 (oldest deadline 2024-03-15T13:11:25.495656+0000) 2024-03-15T13:11:34.009 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:33 smithi149 podman[72183]: 2024-03-15 13:11:33.728928022 +0000 UTC m=+5.594740670 container died d525ac5997a00cc9a770710ffb9b4280ba12abad159a7c76727c580ffdfdb4ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-15T13:11:34.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:33 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:33.663+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:34.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:33 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:33.663+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:34.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:33 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:33.663+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:34.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:33 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:33.663+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:34.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:33 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:33.663+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:34.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:33 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:33.663+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:34.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:33 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:33.663+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-15T13:11:07.693310+0000 front 2024-03-15T13:11:07.693203+0000 (oldest deadline 2024-03-15T13:11:29.993039+0000) 2024-03-15T13:11:34.358 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:34 smithi149 podman[72183]: 2024-03-15 13:11:34.061162308 +0000 UTC m=+5.926974946 container cleanup d525ac5997a00cc9a770710ffb9b4280ba12abad159a7c76727c580ffdfdb4ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5) 2024-03-15T13:11:34.358 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:34 smithi149 bash[72183]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10 2024-03-15T13:11:34.358 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:34 smithi149 podman[72200]: 2024-03-15 13:11:34.253698978 +0000 UTC m=+0.518128186 container remove d525ac5997a00cc9a770710ffb9b4280ba12abad159a7c76727c580ffdfdb4ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-15T13:11:34.934 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:34 smithi149 podman[72294]: 2024-03-15 13:11:34.572150031 +0000 UTC m=+0.017622673 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T13:11:34.935 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:34 smithi149 podman[72294]: 2024-03-15 13:11:34.679804491 +0000 UTC m=+0.125277171 container create c4d0b91c4a6ac77fcd44e6a8520caf5dc1c416c34a22177361901761bcb0111c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T13:11:34.935 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:34 smithi149 podman[72294]: 2024-03-15 13:11:34.873542858 +0000 UTC m=+0.319015495 container init c4d0b91c4a6ac77fcd44e6a8520caf5dc1c416c34a22177361901761bcb0111c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-15T13:11:34.935 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:34 smithi149 podman[72294]: 2024-03-15 13:11:34.876870573 +0000 UTC m=+0.322343217 container start c4d0b91c4a6ac77fcd44e6a8520caf5dc1c416c34a22177361901761bcb0111c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T13:11:34.935 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:34 smithi149 podman[72294]: 2024-03-15 13:11:34.951376567 +0000 UTC m=+0.396849214 container attach c4d0b91c4a6ac77fcd44e6a8520caf5dc1c416c34a22177361901761bcb0111c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-15T13:11:35.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:34 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:34.654+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:35.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:34 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:34.654+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:35.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:34 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:34.654+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:35.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:34 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:34.654+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:35.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:34 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:34.654+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:35.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:34 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:34.654+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:35.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:34 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:34.654+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-15T13:11:07.693310+0000 front 2024-03-15T13:11:07.693203+0000 (oldest deadline 2024-03-15T13:11:29.993039+0000) 2024-03-15T13:11:35.358 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:35 smithi149 podman[72294]: 2024-03-15 13:11:35.16416548 +0000 UTC m=+0.609638118 container died c4d0b91c4a6ac77fcd44e6a8520caf5dc1c416c34a22177361901761bcb0111c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-15T13:11:35.675 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.10.service' 2024-03-15T13:11:35.741 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:35 smithi149 podman[72320]: 2024-03-15 13:11:35.668094785 +0000 UTC m=+0.498829171 container remove c4d0b91c4a6ac77fcd44e6a8520caf5dc1c416c34a22177361901761bcb0111c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-03-15T13:11:35.741 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:35 smithi149 systemd[1]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.10.service: Deactivated successfully. 2024-03-15T13:11:35.741 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:35 smithi149 systemd[1]: Stopped Ceph osd.10 for 2767718a-e2ca-11ee-95c9-87774f69a715. 2024-03-15T13:11:35.741 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 15 13:11:35 smithi149 systemd[1]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.10.service: Consumed 34.822s CPU time. 2024-03-15T13:11:35.756 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:11:35.756 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-15T13:11:35.756 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-15T13:11:35.756 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.11 2024-03-15T13:11:35.958 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:35 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:35.690+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:35.958 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:35 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:35.690+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:35.958 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:35 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:35.690+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:35.958 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:35 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:35.690+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:35.959 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:35 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:35.690+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:35.959 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:35 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:35.690+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:35.959 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:35 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:35.690+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-15T13:11:07.693310+0000 front 2024-03-15T13:11:07.693203+0000 (oldest deadline 2024-03-15T13:11:29.993039+0000) 2024-03-15T13:11:35.959 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:35 smithi184 systemd[1]: Stopping Ceph osd.11 for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:11:36.307 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:36 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:36.079+0000 7f3319978700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:11:36.307 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:36 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:36.079+0000 7f3319978700 -1 osd.11 104 *** Got signal Terminated *** 2024-03-15T13:11:36.307 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:36 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:36.079+0000 7f3319978700 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-15T13:11:37.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:36 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:36.685+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:37.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:36 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:36.685+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:37.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:36 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:36.685+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:37.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:36 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:36.685+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:37.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:36 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:36.685+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:37.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:36 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:36.685+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:37.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:36 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:36.685+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-15T13:11:07.693310+0000 front 2024-03-15T13:11:07.693203+0000 (oldest deadline 2024-03-15T13:11:29.993039+0000) 2024-03-15T13:11:38.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:37 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:37.714+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:38.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:37 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:37.714+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:38.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:37 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:37.714+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:38.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:37 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:37.714+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:38.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:37 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:37.714+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:38.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:37 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:37.714+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:38.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:37 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:37.714+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-15T13:11:07.693310+0000 front 2024-03-15T13:11:07.693203+0000 (oldest deadline 2024-03-15T13:11:29.993039+0000) 2024-03-15T13:11:39.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:38 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:38.721+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:39.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:38 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:38.721+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:39.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:38 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:38.721+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:39.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:38 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:38.721+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:39.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:38 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:38.721+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:39.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:38 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:38.721+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:39.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:38 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:38.721+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-15T13:11:07.693310+0000 front 2024-03-15T13:11:07.693203+0000 (oldest deadline 2024-03-15T13:11:29.993039+0000) 2024-03-15T13:11:40.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:39 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:39.674+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:40.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:39 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:39.674+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:40.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:39 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:39.674+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:40.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:39 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:39.674+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:40.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:39 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:39.674+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:40.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:39 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:39.674+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:40.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:39 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:39.674+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-15T13:11:07.693310+0000 front 2024-03-15T13:11:07.693203+0000 (oldest deadline 2024-03-15T13:11:29.993039+0000) 2024-03-15T13:11:41.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:40 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:40.651+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-15T13:10:12.785149+0000 front 2024-03-15T13:10:12.785247+0000 (oldest deadline 2024-03-15T13:10:38.085288+0000) 2024-03-15T13:11:41.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:40 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:40.651+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-15T13:10:20.386522+0000 front 2024-03-15T13:10:20.386510+0000 (oldest deadline 2024-03-15T13:10:43.886412+0000) 2024-03-15T13:11:41.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:40 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:40.651+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-03-15T13:10:26.187414+0000 front 2024-03-15T13:10:26.187451+0000 (oldest deadline 2024-03-15T13:10:50.887596+0000) 2024-03-15T13:11:41.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:40 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:40.651+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-15T13:10:41.989749+0000 front 2024-03-15T13:10:41.989616+0000 (oldest deadline 2024-03-15T13:11:07.889671+0000) 2024-03-15T13:11:41.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:40 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:40.651+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-03-15T13:10:52.491136+0000 front 2024-03-15T13:10:52.491019+0000 (oldest deadline 2024-03-15T13:11:17.191279+0000) 2024-03-15T13:11:41.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:40 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:40.651+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-03-15T13:11:00.092151+0000 front 2024-03-15T13:11:00.092170+0000 (oldest deadline 2024-03-15T13:11:24.192090+0000) 2024-03-15T13:11:41.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:40 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:40.651+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-15T13:11:07.693310+0000 front 2024-03-15T13:11:07.693203+0000 (oldest deadline 2024-03-15T13:11:29.993039+0000) 2024-03-15T13:11:41.058 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:40 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11[61236]: 2024-03-15T13:11:40.651+0000 7f3311f64700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.184:6820 osd.8 since back 2024-03-15T13:11:15.194396+0000 front 2024-03-15T13:11:15.194370+0000 (oldest deadline 2024-03-15T13:11:39.894359+0000) 2024-03-15T13:11:41.681 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:41 smithi184 podman[73637]: 2024-03-15 13:11:41.407106324 +0000 UTC m=+5.436232832 container died 7ffc6d74865b36374e19ef7beff642ce739f2423d821bce4dc1712c1c8c0f230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T13:11:42.048 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:41 smithi184 podman[73637]: 2024-03-15 13:11:41.796737646 +0000 UTC m=+5.825864156 container cleanup 7ffc6d74865b36374e19ef7beff642ce739f2423d821bce4dc1712c1c8c0f230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-squid-aba3c1d) 2024-03-15T13:11:42.048 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:41 smithi184 bash[73637]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11 2024-03-15T13:11:42.307 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:42 smithi184 podman[73654]: 2024-03-15 13:11:42.064384083 +0000 UTC m=+0.646034296 container remove 7ffc6d74865b36374e19ef7beff642ce739f2423d821bce4dc1712c1c8c0f230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-15T13:11:42.618 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:42 smithi184 podman[73743]: 2024-03-15 13:11:42.274224693 +0000 UTC m=+0.018789489 image pull 80ec0d3daaba7a52b05a2c1e9fce2fdb808f0661f4843706a56e0840fed82af0 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3 2024-03-15T13:11:42.618 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:42 smithi184 podman[73743]: 2024-03-15 13:11:42.373610582 +0000 UTC m=+0.118175378 container create 7fc6c775eaaea9f5c448842f3917ea388d37ef1d8a569608457554a47fd1f0df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-15T13:11:42.618 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:42 smithi184 podman[73743]: 2024-03-15 13:11:42.567934527 +0000 UTC m=+0.312499324 container init 7fc6c775eaaea9f5c448842f3917ea388d37ef1d8a569608457554a47fd1f0df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-15T13:11:42.618 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:42 smithi184 podman[73743]: 2024-03-15 13:11:42.571038941 +0000 UTC m=+0.315603745 container start 7fc6c775eaaea9f5c448842f3917ea388d37ef1d8a569608457554a47fd1f0df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-03-15T13:11:42.618 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:42 smithi184 podman[73743]: 2024-03-15 13:11:42.634355151 +0000 UTC m=+0.378919954 container attach 7fc6c775eaaea9f5c448842f3917ea388d37ef1d8a569608457554a47fd1f0df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-15T13:11:43.057 INFO:journalctl@ceph.osd.11.smithi184.stdout:Mar 15 13:11:42 smithi184 podman[73743]: 2024-03-15 13:11:42.863854993 +0000 UTC m=+0.608419793 container died 7fc6c775eaaea9f5c448842f3917ea388d37ef1d8a569608457554a47fd1f0df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True) 2024-03-15T13:11:43.458 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@osd.11.service' 2024-03-15T13:11:43.500 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:11:43.500 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-15T13:11:43.500 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-15T13:11:43.501 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.a 2024-03-15T13:11:43.799 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 13:11:43 smithi002 systemd[1]: Stopping Ceph mds.a for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:11:44.051 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 13:11:43 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-a[74141]: 2024-03-15T13:11:43.814+0000 7fa75b9ec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:11:44.051 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 13:11:43 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-a[74141]: 2024-03-15T13:11:43.814+0000 7fa75b9ec700 -1 mds.a *** got signal Terminated *** 2024-03-15T13:11:48.074 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 13:11:47 smithi002 podman[105719]: 2024-03-15 13:11:47.826591994 +0000 UTC m=+4.107465954 container died 8b4995460d1b5663a41702cde9d280171af47520d697d82bd3713c2ba24bae0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-15T13:11:48.416 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 13:11:48 smithi002 podman[105719]: 2024-03-15 13:11:48.189938772 +0000 UTC m=+4.470812740 container cleanup 8b4995460d1b5663a41702cde9d280171af47520d697d82bd3713c2ba24bae0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T13:11:48.416 INFO:journalctl@ceph.mds.a.smithi002.stdout:Mar 15 13:11:48 smithi002 bash[105719]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-a 2024-03-15T13:11:48.515 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.a.service' 2024-03-15T13:11:48.553 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:11:48.553 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-15T13:11:48.554 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-15T13:11:48.554 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.d 2024-03-15T13:11:48.801 INFO:journalctl@ceph.mds.d.smithi002.stdout:Mar 15 13:11:48 smithi002 systemd[1]: Stopping Ceph mds.d for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:11:49.301 INFO:journalctl@ceph.mds.d.smithi002.stdout:Mar 15 13:11:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-d[73190]: 2024-03-15T13:11:48.910+0000 7f17f0986700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:11:49.301 INFO:journalctl@ceph.mds.d.smithi002.stdout:Mar 15 13:11:48 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-d[73190]: 2024-03-15T13:11:48.910+0000 7f17f0986700 -1 mds.d *** got signal Terminated *** 2024-03-15T13:11:51.935 INFO:journalctl@ceph.mds.d.smithi002.stdout:Mar 15 13:11:51 smithi002 podman[105851]: 2024-03-15 13:11:51.633719336 +0000 UTC m=+2.844540026 container died 2ad74e4d29bb5f3d6b644f35b237330f4f04babd615a52e2df961e1ec061ae87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-d, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d) 2024-03-15T13:11:51.935 INFO:journalctl@ceph.mds.d.smithi002.stdout:Mar 15 13:11:51 smithi002 podman[105851]: 2024-03-15 13:11:51.951691981 +0000 UTC m=+3.162512674 container cleanup 2ad74e4d29bb5f3d6b644f35b237330f4f04babd615a52e2df961e1ec061ae87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-15T13:11:52.301 INFO:journalctl@ceph.mds.d.smithi002.stdout:Mar 15 13:11:51 smithi002 bash[105851]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-d 2024-03-15T13:11:52.301 INFO:journalctl@ceph.mds.d.smithi002.stdout:Mar 15 13:11:52 smithi002 podman[105865]: 2024-03-15 13:11:52.194519658 +0000 UTC m=+0.555326380 container remove 2ad74e4d29bb5f3d6b644f35b237330f4f04babd615a52e2df961e1ec061ae87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-d, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux ) 2024-03-15T13:11:52.413 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.d.service' 2024-03-15T13:11:52.448 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:11:52.448 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-15T13:11:52.448 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-15T13:11:52.448 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.g 2024-03-15T13:11:52.772 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 13:11:52 smithi002 systemd[1]: Stopping Ceph mds.g for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:11:53.051 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 13:11:52 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-g[73785]: 2024-03-15T13:11:52.788+0000 7fb118ed1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:11:53.051 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 13:11:52 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-g[73785]: 2024-03-15T13:11:52.788+0000 7fb118ed1700 -1 mds.g *** got signal Terminated *** 2024-03-15T13:11:56.551 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 13:11:56 smithi002 podman[105975]: 2024-03-15 13:11:56.274985061 +0000 UTC m=+3.592193225 container died 8b4c087eff7f86355f50c41890884c097498cce1d5145a58dd77c73d140ef5ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-g, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-squid-aba3c1d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-15T13:11:56.998 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 13:11:56 smithi002 podman[105975]: 2024-03-15 13:11:56.763750579 +0000 UTC m=+4.080958733 container cleanup 8b4c087eff7f86355f50c41890884c097498cce1d5145a58dd77c73d140ef5ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-g, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-squid-aba3c1d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-03-15T13:11:56.998 INFO:journalctl@ceph.mds.g.smithi002.stdout:Mar 15 13:11:56 smithi002 bash[105975]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-g 2024-03-15T13:11:57.181 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.g.service' 2024-03-15T13:11:57.214 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:11:57.215 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-15T13:11:57.215 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-15T13:11:57.215 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.j 2024-03-15T13:11:57.534 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 13:11:57 smithi002 systemd[1]: Stopping Ceph mds.j for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:11:57.801 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 13:11:57 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-j[72835]: 2024-03-15T13:11:57.550+0000 7f079362f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:11:57.801 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 13:11:57 smithi002 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-j[72835]: 2024-03-15T13:11:57.550+0000 7f079362f700 -1 mds.j *** got signal Terminated *** 2024-03-15T13:12:00.918 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 13:12:00 smithi002 podman[106097]: 2024-03-15 13:12:00.678826997 +0000 UTC m=+3.231637001 container died 8302642d79b05c6abb4bc8de3bc36f98f146dafad0d048ce62febc8b27411c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-j, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-15T13:12:01.168 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 13:12:01 smithi002 podman[106097]: 2024-03-15 13:12:01.017455433 +0000 UTC m=+3.570265436 container cleanup 8302642d79b05c6abb4bc8de3bc36f98f146dafad0d048ce62febc8b27411c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-j, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-15T13:12:01.168 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 13:12:01 smithi002 bash[106097]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-j 2024-03-15T13:12:01.426 INFO:journalctl@ceph.mds.j.smithi002.stdout:Mar 15 13:12:01 smithi002 podman[106111]: 2024-03-15 13:12:01.185148822 +0000 UTC m=+0.506206903 container remove 8302642d79b05c6abb4bc8de3bc36f98f146dafad0d048ce62febc8b27411c47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d) 2024-03-15T13:12:01.429 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.j.service' 2024-03-15T13:12:01.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:12:01.463 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-15T13:12:01.464 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-15T13:12:01.464 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.b 2024-03-15T13:12:01.769 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 13:12:01 smithi149 systemd[1]: Stopping Ceph mds.b for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:12:02.108 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 13:12:01 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-b[65861]: 2024-03-15T13:12:01.784+0000 7f7167530700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:12:02.108 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 13:12:01 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-b[65861]: 2024-03-15T13:12:01.784+0000 7f7167530700 -1 mds.b *** got signal Terminated *** 2024-03-15T13:12:04.543 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 13:12:04 smithi149 podman[72412]: 2024-03-15 13:12:04.2745357 +0000 UTC m=+2.600827280 container died bfb0b91488884145c0ece4321164f1c86145119d7bf7437b3f7b18d998865016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-b, org.label-schema.build-date=20240307, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-03-15T13:12:04.858 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 13:12:04 smithi149 podman[72412]: 2024-03-15 13:12:04.643129913 +0000 UTC m=+2.969421492 container cleanup bfb0b91488884145c0ece4321164f1c86145119d7bf7437b3f7b18d998865016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-15T13:12:04.858 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 15 13:12:04 smithi149 bash[72412]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-b 2024-03-15T13:12:05.044 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.b.service' 2024-03-15T13:12:05.081 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:12:05.081 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-15T13:12:05.081 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-15T13:12:05.081 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.e 2024-03-15T13:12:05.396 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 13:12:05 smithi149 systemd[1]: Stopping Ceph mds.e for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:12:05.858 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 13:12:05 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-e[65228]: 2024-03-15T13:12:05.412+0000 7f5350fe6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:12:05.858 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 13:12:05 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-e[65228]: 2024-03-15T13:12:05.412+0000 7f5350fe6700 -1 mds.e *** got signal Terminated *** 2024-03-15T13:12:08.522 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 13:12:08 smithi149 podman[72533]: 2024-03-15 13:12:08.280647525 +0000 UTC m=+2.961986870 container died 9b4a7928af3cb3beff5ce93c0f24365c6c1ef97be77a424f98802eed08cedc75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-15T13:12:08.781 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 13:12:08 smithi149 podman[72533]: 2024-03-15 13:12:08.596248557 +0000 UTC m=+3.277587903 container cleanup 9b4a7928af3cb3beff5ce93c0f24365c6c1ef97be77a424f98802eed08cedc75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-15T13:12:08.781 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 15 13:12:08 smithi149 bash[72533]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-e 2024-03-15T13:12:09.003 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.e.service' 2024-03-15T13:12:09.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:12:09.040 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-15T13:12:09.040 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-15T13:12:09.040 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.h 2024-03-15T13:12:09.350 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 13:12:09 smithi149 systemd[1]: Stopping Ceph mds.h for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:12:09.608 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 13:12:09 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-h[65554]: 2024-03-15T13:12:09.366+0000 7f059b67b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:12:09.608 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 13:12:09 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-h[65554]: 2024-03-15T13:12:09.366+0000 7f059b67b700 -1 mds.h *** got signal Terminated *** 2024-03-15T13:12:12.542 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 13:12:12 smithi149 podman[72667]: 2024-03-15 13:12:12.277803217 +0000 UTC m=+3.003944763 container died 0e291e77314ffa4e31e0eff7f0ec33f32b0936590b0af89f30288c0bea6d4492 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-h, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-15T13:12:12.858 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 13:12:12 smithi149 podman[72667]: 2024-03-15 13:12:12.633329404 +0000 UTC m=+3.359470952 container cleanup 0e291e77314ffa4e31e0eff7f0ec33f32b0936590b0af89f30288c0bea6d4492 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-h, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-15T13:12:12.858 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 15 13:12:12 smithi149 bash[72667]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-h 2024-03-15T13:12:13.072 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.h.service' 2024-03-15T13:12:13.108 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:12:13.108 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-15T13:12:13.108 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-15T13:12:13.108 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.k 2024-03-15T13:12:13.428 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 13:12:13 smithi149 systemd[1]: Stopping Ceph mds.k for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:12:13.858 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 13:12:13 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-k[64909]: 2024-03-15T13:12:13.444+0000 7f8a4c574700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:12:13.858 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 13:12:13 smithi149 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-k[64909]: 2024-03-15T13:12:13.444+0000 7f8a4c574700 -1 mds.k *** got signal Terminated *** 2024-03-15T13:12:16.520 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 13:12:16 smithi149 podman[72788]: 2024-03-15 13:12:16.281579393 +0000 UTC m=+2.939464558 container died 0103c35aed93e9382c6732c6aa0c11754deed9e116f5211c12e5d3557ba0e35c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-k, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-15T13:12:16.850 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 13:12:16 smithi149 podman[72788]: 2024-03-15 13:12:16.619716243 +0000 UTC m=+3.277601407 container cleanup 0103c35aed93e9382c6732c6aa0c11754deed9e116f5211c12e5d3557ba0e35c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-k, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-squid-aba3c1d) 2024-03-15T13:12:16.851 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 13:12:16 smithi149 bash[72788]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-k 2024-03-15T13:12:16.851 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 15 13:12:16 smithi149 podman[72802]: 2024-03-15 13:12:16.778927198 +0000 UTC m=+0.496662774 container remove 0103c35aed93e9382c6732c6aa0c11754deed9e116f5211c12e5d3557ba0e35c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-k, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True) 2024-03-15T13:12:16.982 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.k.service' 2024-03-15T13:12:17.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:12:17.016 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-15T13:12:17.016 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-15T13:12:17.016 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.c 2024-03-15T13:12:17.307 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 13:12:17 smithi184 systemd[1]: Stopping Ceph mds.c for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:12:17.582 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 13:12:17 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-c[66664]: 2024-03-15T13:12:17.345+0000 7fdff576b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:12:17.582 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 13:12:17 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-c[66664]: 2024-03-15T13:12:17.345+0000 7fdff576b700 -1 mds.c *** got signal Terminated *** 2024-03-15T13:12:20.529 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 13:12:20 smithi184 podman[73849]: 2024-03-15 13:12:20.278716405 +0000 UTC m=+3.041996030 container died eadf048c5df15920ca296c57d759acfada8611b7b6600c137fd5b485256b40af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-15T13:12:20.807 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 13:12:20 smithi184 podman[73849]: 2024-03-15 13:12:20.628252604 +0000 UTC m=+3.391532235 container cleanup eadf048c5df15920ca296c57d759acfada8611b7b6600c137fd5b485256b40af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T13:12:20.807 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 13:12:20 smithi184 bash[73849]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-c 2024-03-15T13:12:20.807 INFO:journalctl@ceph.mds.c.smithi184.stdout:Mar 15 13:12:20 smithi184 podman[73874]: 2024-03-15 13:12:20.771281508 +0000 UTC m=+0.487095719 container remove eadf048c5df15920ca296c57d759acfada8611b7b6600c137fd5b485256b40af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-c, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-15T13:12:20.955 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.c.service' 2024-03-15T13:12:20.993 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:12:20.994 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-15T13:12:20.994 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-15T13:12:20.994 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.f 2024-03-15T13:12:21.307 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 13:12:21 smithi184 systemd[1]: Stopping Ceph mds.f for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:12:21.807 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 13:12:21 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-f[66359]: 2024-03-15T13:12:21.331+0000 7f9da1de3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:12:21.807 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 13:12:21 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-f[66359]: 2024-03-15T13:12:21.331+0000 7f9da1de3700 -1 mds.f *** got signal Terminated *** 2024-03-15T13:12:24.541 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 13:12:24 smithi184 podman[73984]: 2024-03-15 13:12:24.282227375 +0000 UTC m=+3.051776508 container died eb3f5a7c69a9765d4823cb90b621c14027604fe7671636db9f799716cad44486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-squid-aba3c1d, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-03-15T13:12:24.807 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 13:12:24 smithi184 podman[73984]: 2024-03-15 13:12:24.640193606 +0000 UTC m=+3.409742733 container cleanup eb3f5a7c69a9765d4823cb90b621c14027604fe7671636db9f799716cad44486 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-f, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-squid-aba3c1d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-15T13:12:24.807 INFO:journalctl@ceph.mds.f.smithi184.stdout:Mar 15 13:12:24 smithi184 bash[73984]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-f 2024-03-15T13:12:25.053 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.f.service' 2024-03-15T13:12:25.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:12:25.088 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-15T13:12:25.088 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-15T13:12:25.088 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.i 2024-03-15T13:12:25.557 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 13:12:25 smithi184 systemd[1]: Stopping Ceph mds.i for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:12:25.557 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 13:12:25 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-i[66981]: 2024-03-15T13:12:25.417+0000 7f6e76041700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:12:25.557 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 13:12:25 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-i[66981]: 2024-03-15T13:12:25.417+0000 7f6e76041700 -1 mds.i *** got signal Terminated *** 2024-03-15T13:12:28.557 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 13:12:28 smithi184 podman[74105]: 2024-03-15 13:12:28.267187637 +0000 UTC m=+2.947935029 container died b78052221147e9aaf1615fe83b2b625a644608d6ee5633d01557d4f3a7e3a06d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-i, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-15T13:12:28.886 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 13:12:28 smithi184 podman[74105]: 2024-03-15 13:12:28.7020691 +0000 UTC m=+3.382816489 container cleanup b78052221147e9aaf1615fe83b2b625a644608d6ee5633d01557d4f3a7e3a06d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-squid-aba3c1d, GIT_CLEAN=True) 2024-03-15T13:12:28.886 INFO:journalctl@ceph.mds.i.smithi184.stdout:Mar 15 13:12:28 smithi184 bash[74105]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-i 2024-03-15T13:12:29.104 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.i.service' 2024-03-15T13:12:29.138 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:12:29.139 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-15T13:12:29.139 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-15T13:12:29.139 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.l 2024-03-15T13:12:29.447 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 13:12:29 smithi184 systemd[1]: Stopping Ceph mds.l for 2767718a-e2ca-11ee-95c9-87774f69a715... 2024-03-15T13:12:29.807 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 13:12:29 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-l[66050]: 2024-03-15T13:12:29.463+0000 7fd6cbf46700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-15T13:12:29.807 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 13:12:29 smithi184 ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-l[66050]: 2024-03-15T13:12:29.463+0000 7fd6cbf46700 -1 mds.l *** got signal Terminated *** 2024-03-15T13:12:32.497 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 13:12:32 smithi184 podman[74226]: 2024-03-15 13:12:32.262350213 +0000 UTC m=+2.884258511 container died a8956d8f1c6f7ff5e2a7e4a03088f14110e9b781a3607b34d1522aab1bb0004a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-l, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d) 2024-03-15T13:12:32.807 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 13:12:32 smithi184 podman[74226]: 2024-03-15 13:12:32.61370229 +0000 UTC m=+3.235610588 container cleanup a8956d8f1c6f7ff5e2a7e4a03088f14110e9b781a3607b34d1522aab1bb0004a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:264d2baad1e15edc24e723089ae314893f176dbcd8c8990ddeaead799c48d2b3, name=ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-l, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-lusov-qdb-squid-aba3c1d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-15T13:12:32.807 INFO:journalctl@ceph.mds.l.smithi184.stdout:Mar 15 13:12:32 smithi184 bash[74226]: ceph-2767718a-e2ca-11ee-95c9-87774f69a715-mds-l 2024-03-15T13:12:33.011 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2767718a-e2ca-11ee-95c9-87774f69a715@mds.l.service' 2024-03-15T13:12:33.044 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-15T13:12:33.044 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-15T13:12:33.044 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-15T13:12:33.240 INFO:teuthology.orchestra.run.smithi002.stdout:Deleting cluster with fsid: 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T13:12:35.643 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-15T13:12:35.835 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T13:12:38.237 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-15T13:12:38.429 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T13:12:40.691 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-15T13:12:40.726 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-15T13:12:40.761 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-15T13:12:40.790 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-15T13:12:40.791 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993/remote/smithi002/crash 2024-03-15T13:12:40.792 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/crash -- . 2024-03-15T13:12:40.823 INFO:teuthology.orchestra.run.smithi002.stderr:tar: /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-15T13:12:40.823 INFO:teuthology.orchestra.run.smithi002.stderr:tar: Error is not recoverable: exiting now 2024-03-15T13:12:40.825 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993/remote/smithi149/crash 2024-03-15T13:12:40.826 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/crash -- . 2024-03-15T13:12:40.871 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-15T13:12:40.872 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2024-03-15T13:12:40.873 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993/remote/smithi184/crash 2024-03-15T13:12:40.874 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/crash -- . 2024-03-15T13:12:40.912 INFO:teuthology.orchestra.run.smithi184.stderr:tar: /var/lib/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-15T13:12:40.913 INFO:teuthology.orchestra.run.smithi184.stderr:tar: Error is not recoverable: exiting now 2024-03-15T13:12:40.914 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-15T13:12:40.914 DEBUG:teuthology.orchestra.run.smithi002:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-15T13:12:40.954 INFO:teuthology.orchestra.run.smithi002.stdout:2024-03-15T12:50:17.678430+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-15T13:12:40.955 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-15T13:12:40.955 DEBUG:teuthology.orchestra.run.smithi002:> sudo egrep '\[SEC\]' /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-15T13:12:41.029 DEBUG:teuthology.orchestra.run.smithi002:> sudo egrep '\[ERR\]' /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-15T13:12:41.101 DEBUG:teuthology.orchestra.run.smithi002:> sudo egrep '\[WRN\]' /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-15T13:12:41.174 INFO:teuthology.orchestra.run.smithi002.stdout:2024-03-15T12:50:17.678430+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-15T13:12:41.174 INFO:tasks.cephadm:Compressing logs... 2024-03-15T13:12:41.174 DEBUG:teuthology.orchestra.run.smithi002:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-15T13:12:41.216 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-15T13:12:41.218 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-15T13:12:41.240 INFO:teuthology.orchestra.run.smithi002.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-15T13:12:41.240 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-15T13:12:41.240 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-15T13:12:41.240 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-15T13:12:41.241 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.6.log 2024-03-15T13:12:41.241 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mon.a.log 2024-03-15T13:12:41.242 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-15T13:12:41.242 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.6.log: /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.g.log 2024-03-15T13:12:41.243 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.j.log 2024-03-15T13:12:41.244 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.g.log: 92.7%gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.3.log 2024-03-15T13:12:41.244 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-15T13:12:41.244 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mgr.x.log 2024-03-15T13:12:41.245 INFO:teuthology.orchestra.run.smithi184.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-15T13:12:41.245 INFO:teuthology.orchestra.run.smithi184.stderr:: No such file or directory 2024-03-15T13:12:41.246 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.l.log 2024-03-15T13:12:41.246 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-15T13:12:41.246 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.i.log 2024-03-15T13:12:41.247 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.3.log: 89.9% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-15T13:12:41.247 INFO:teuthology.orchestra.run.smithi002.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-15T13:12:41.251 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.11.log 2024-03-15T13:12:41.251 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mgr.z.log 2024-03-15T13:12:41.251 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.8.log 2024-03-15T13:12:41.251 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mgr.z.log: gzip 92.4% -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.5.log 2024-03-15T13:12:41.251 INFO:teuthology.orchestra.run.smithi184.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-15T13:12:41.251 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.f.log 2024-03-15T13:12:41.251 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose --/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.5.log: /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-15T13:12:41.251 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.f.log: 90.0% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-15T13:12:41.251 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.log 2024-03-15T13:12:41.251 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-15T13:12:41.251 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.2.log 2024-03-15T13:12:41.255 INFO:teuthology.orchestra.run.smithi002.stderr: 94.6% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.j.log.gz 2024-03-15T13:12:41.255 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.0.log 2024-03-15T13:12:41.256 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-15T13:12:41.256 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.log 2024-03-15T13:12:41.256 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-15T13:12:41.256 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-15T13:12:41.256 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.log: 87.4% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-15T13:12:41.256 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.9.log 2024-03-15T13:12:41.257 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/cephadm.log 2024-03-15T13:12:41.258 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-03-15T13:12:41.258 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.10.log 2024-03-15T13:12:41.258 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-15T13:12:41.258 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.b.log 2024-03-15T13:12:41.258 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.4.log 2024-03-15T13:12:41.258 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.1.log 2024-03-15T13:12:41.258 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.k.log 2024-03-15T13:12:41.258 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.h.log 2024-03-15T13:12:41.258 INFO:teuthology.orchestra.run.smithi149.stderr:gzip/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.audit.log: -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.7.log 2024-03-15T13:12:41.258 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.b.log: /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.4.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-15T13:12:41.258 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.h.log: gzip/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.1.log: -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.e.log 2024-03-15T13:12:41.258 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.k.log: /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.7.log: 90.1% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-15T13:12:41.258 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mgr.y.log 2024-03-15T13:12:41.260 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.log: 87.5% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-15T13:12:41.260 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.c.log 2024-03-15T13:12:41.260 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.2.log: 91.9% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mgr.z.log.gz 2024-03-15T13:12:41.260 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-15T13:12:41.265 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mon.c.log 2024-03-15T13:12:41.266 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-15T13:12:41.269 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.d.log 2024-03-15T13:12:41.275 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.a.log 2024-03-15T13:12:41.280 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mgr.y.log: 91.8%gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.log 2024-03-15T13:12:41.280 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-15T13:12:41.281 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-15T13:12:41.287 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-15T13:12:41.287 INFO:teuthology.orchestra.run.smithi149.stderr: -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mgr.y.log.gz 2024-03-15T13:12:41.293 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.d.log.gz 2024-03-15T13:12:41.300 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mon.b.log 2024-03-15T13:12:41.302 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-volume.log: 87.5% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-15T13:12:41.305 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mon.c.log: 91.0% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-15T13:12:41.327 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-15T13:12:41.354 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.l.log.gz 2024-03-15T13:12:41.381 INFO:teuthology.orchestra.run.smithi184.stderr: 94.0% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.c.log.gz 2024-03-15T13:12:41.438 INFO:teuthology.orchestra.run.smithi002.stderr: 94.3% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.g.log.gz 2024-03-15T13:12:41.667 INFO:teuthology.orchestra.run.smithi002.stderr: 89.9% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mgr.x.log.gz 2024-03-15T13:12:41.706 INFO:teuthology.orchestra.run.smithi002.stderr: 94.8% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.a.log.gz 2024-03-15T13:12:41.769 INFO:teuthology.orchestra.run.smithi149.stderr: 93.5% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.e.log.gz 2024-03-15T13:12:41.817 INFO:teuthology.orchestra.run.smithi184.stderr: 92.0% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mon.c.log.gz 2024-03-15T13:12:41.905 INFO:teuthology.orchestra.run.smithi149.stderr: 93.7% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.h.log.gz 2024-03-15T13:12:41.933 INFO:teuthology.orchestra.run.smithi149.stderr: 92.2% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mon.b.log.gz 2024-03-15T13:12:42.539 INFO:teuthology.orchestra.run.smithi149.stderr: 94.1% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.k.log.gz 2024-03-15T13:12:42.564 INFO:teuthology.orchestra.run.smithi002.stderr: 90.6% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mon.a.log.gz 2024-03-15T13:12:44.856 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.f.log.gz 2024-03-15T13:12:45.056 INFO:teuthology.orchestra.run.smithi184.stderr: 95.6% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.i.log.gz 2024-03-15T13:12:46.657 INFO:teuthology.orchestra.run.smithi184.stderr: 91.6% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.8.log.gz 2024-03-15T13:12:47.230 INFO:teuthology.orchestra.run.smithi184.stderr: 91.8% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.11.log.gz 2024-03-15T13:12:48.425 INFO:teuthology.orchestra.run.smithi002.stderr: 91.8% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.0.log.gz 2024-03-15T13:12:48.755 INFO:teuthology.orchestra.run.smithi149.stderr: 91.7% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.7.log.gz 2024-03-15T13:12:49.251 INFO:teuthology.orchestra.run.smithi002.stderr: 91.7% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.6.log.gz 2024-03-15T13:12:49.467 INFO:teuthology.orchestra.run.smithi149.stderr: 92.0% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.10.log.gz 2024-03-15T13:12:49.608 INFO:teuthology.orchestra.run.smithi149.stderr: 92.0% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.1.log.gz 2024-03-15T13:12:49.945 INFO:teuthology.orchestra.run.smithi184.stderr: 91.7% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.2.log.gz 2024-03-15T13:12:49.963 INFO:teuthology.orchestra.run.smithi149.stderr: 91.5% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.4.log.gz 2024-03-15T13:12:50.942 INFO:teuthology.orchestra.run.smithi002.stderr: 91.6% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.3.log.gz 2024-03-15T13:12:51.010 INFO:teuthology.orchestra.run.smithi184.stderr: 91.5% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.5.log.gz 2024-03-15T13:12:51.011 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-03-15T13:12:51.012 INFO:teuthology.orchestra.run.smithi184.stderr:real 0m9.779s 2024-03-15T13:12:51.012 INFO:teuthology.orchestra.run.smithi184.stderr:user 0m36.811s 2024-03-15T13:12:51.012 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m1.120s 2024-03-15T13:12:52.650 INFO:teuthology.orchestra.run.smithi002.stderr: 91.8% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-osd.9.log.gz 2024-03-15T13:12:53.638 INFO:teuthology.orchestra.run.smithi002.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-15T13:12:53.640 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-03-15T13:12:53.640 INFO:teuthology.orchestra.run.smithi002.stderr:real 0m12.412s 2024-03-15T13:12:53.640 INFO:teuthology.orchestra.run.smithi002.stderr:user 0m49.297s 2024-03-15T13:12:53.640 INFO:teuthology.orchestra.run.smithi002.stderr:sys 0m1.404s 2024-03-15T13:13:15.486 INFO:teuthology.orchestra.run.smithi149.stderr: 93.3% -- replaced with /var/log/ceph/2767718a-e2ca-11ee-95c9-87774f69a715/ceph-mds.b.log.gz 2024-03-15T13:13:15.487 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-03-15T13:13:15.488 INFO:teuthology.orchestra.run.smithi149.stderr:real 0m34.254s 2024-03-15T13:13:15.488 INFO:teuthology.orchestra.run.smithi149.stderr:user 1m7.328s 2024-03-15T13:13:15.488 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m2.216s 2024-03-15T13:13:15.488 INFO:tasks.cephadm:Archiving logs... 2024-03-15T13:13:15.488 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993/remote/smithi002/log 2024-03-15T13:13:15.489 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-15T13:13:22.315 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993/remote/smithi149/log 2024-03-15T13:13:22.316 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-15T13:13:32.356 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993/remote/smithi184/log 2024-03-15T13:13:32.358 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-15T13:13:37.661 INFO:tasks.cephadm:Removing cluster... 2024-03-15T13:13:37.661 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 --force 2024-03-15T13:13:37.860 INFO:teuthology.orchestra.run.smithi002.stdout:Deleting cluster with fsid: 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T13:13:38.154 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 --force 2024-03-15T13:13:38.348 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T13:13:38.713 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2767718a-e2ca-11ee-95c9-87774f69a715 --force 2024-03-15T13:13:38.902 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: 2767718a-e2ca-11ee-95c9-87774f69a715 2024-03-15T13:13:39.203 INFO:tasks.cephadm:Removing cephadm ... 2024-03-15T13:13:39.203 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-15T13:13:39.221 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-15T13:13:39.237 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-15T13:13:39.253 INFO:tasks.cephadm:Teardown complete 2024-03-15T13:13:39.254 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-15T13:13:39.265 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-15T13:13:39.265 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-15T13:13:39.268 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-15T13:13:39.279 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-15T13:13:39.392 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-15T13:13:39.403 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-15T13:13:39.403 DEBUG:teuthology.orchestra.run.smithi002:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-15T13:13:39.405 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-15T13:13:39.408 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-15T13:13:39.419 INFO:teuthology.orchestra.run.smithi002.stderr:bash: line 1: ntpq: command not found 2024-03-15T13:13:39.420 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-03-15T13:13:39.421 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-03-15T13:13:39.423 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-15T13:13:39.423 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2024-03-15T13:13:39.424 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 9 +1353us[+1367us] +/- 109ms 2024-03-15T13:13:39.424 INFO:teuthology.orchestra.run.smithi002.stdout:^* hv02.front.sepia.ceph.com 3 6 377 8 -1511us[-1497us] +/- 76ms 2024-03-15T13:13:39.424 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +570us[ +584us] +/- 72ms 2024-03-15T13:13:39.424 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-15T13:13:39.425 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-15T13:13:39.425 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-03-15T13:13:39.425 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 +1354us[+1367us] +/- 109ms 2024-03-15T13:13:39.425 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv02.front.sepia.ceph.com 3 6 377 12 -1507us[-1494us] +/- 76ms 2024-03-15T13:13:39.426 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +573us[ +586us] +/- 72ms 2024-03-15T13:13:39.426 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-15T13:13:39.426 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-15T13:13:39.427 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-03-15T13:13:39.427 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 +1413us[+1418us] +/- 109ms 2024-03-15T13:13:39.427 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv02.front.sepia.ceph.com 3 6 377 12 -1451us[-1447us] +/- 76ms 2024-03-15T13:13:39.427 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +654us[ +654us] +/- 72ms 2024-03-15T13:13:39.427 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-15T13:13:39.427 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-15T13:13:39.438 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-15T13:13:39.438 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-15T13:13:39.451 DEBUG:teuthology.orchestra.run.smithi002:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-15T13:13:39.556 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-15T13:13:39.628 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-15T13:13:39.699 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-15T13:13:39.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-15T13:13:39.731 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-15T13:13:39.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-15T13:13:39.762 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-15T13:13:39.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-15T13:13:39.792 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-15T13:13:39.805 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-15T13:13:39.815 INFO:teuthology.task.internal:Duration was 1957.403579 seconds 2024-03-15T13:13:39.815 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-15T13:13:39.837 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-15T13:13:39.837 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-15T13:13:39.840 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-15T13:13:39.842 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-15T13:13:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-15T13:13:39.886 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-15T13:13:39.886 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-15T13:13:40.409 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-15T13:13:40.409 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi002.front.sepia.ceph.com 2024-03-15T13:13:40.409 DEBUG:teuthology.orchestra.run.smithi002:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-15T13:13:40.431 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-03-15T13:13:40.431 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-15T13:13:40.479 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2024-03-15T13:13:40.479 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-15T13:13:40.504 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-15T13:13:40.505 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-15T13:13:40.508 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-15T13:13:40.522 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-15T13:13:40.572 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-15T13:13:40.573 DEBUG:teuthology.orchestra.run.smithi002:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-15T13:13:40.578 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-15T13:13:40.592 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-15T13:13:40.985 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-15T13:13:40.997 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-15T13:13:40.997 DEBUG:teuthology.orchestra.run.smithi002:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-15T13:13:41.023 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-15T13:13:41.051 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-15T13:13:41.078 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-15T13:13:41.092 DEBUG:teuthology.orchestra.run.smithi002:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-15T13:13:41.094 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-15T13:13:41.096 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-15T13:13:41.117 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = core 2024-03-15T13:13:41.119 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-03-15T13:13:41.143 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2024-03-15T13:13:41.169 DEBUG:teuthology.orchestra.run.smithi002:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-15T13:13:41.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-15T13:13:41.186 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-15T13:13:41.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-15T13:13:41.200 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-15T13:13:41.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-15T13:13:41.223 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-15T13:13:41.235 INFO:teuthology.task.internal:Transferring archived files... 2024-03-15T13:13:41.236 DEBUG:teuthology.misc:Transferring archived files from smithi002:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993/remote/smithi002 2024-03-15T13:13:41.236 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-15T13:13:41.284 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993/remote/smithi149 2024-03-15T13:13:41.285 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-15T13:13:41.325 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993/remote/smithi184 2024-03-15T13:13:41.326 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-15T13:13:41.363 INFO:teuthology.task.internal:Removing archive directory... 2024-03-15T13:13:41.363 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-15T13:13:41.366 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-15T13:13:41.368 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-15T13:13:41.412 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-15T13:13:41.425 INFO:teuthology.task.internal:Not uploading archives. 2024-03-15T13:13:41.425 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-15T13:13:41.436 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-15T13:13:41.436 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-15T13:13:41.438 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-15T13:13:41.440 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-15T13:13:41.450 INFO:teuthology.orchestra.run.smithi002.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 15 13:13 /home/ubuntu/cephtest 2024-03-15T13:13:41.452 INFO:teuthology.orchestra.run.smithi149.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 15 13:13 /home/ubuntu/cephtest 2024-03-15T13:13:41.467 INFO:teuthology.orchestra.run.smithi184.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 15 13:13 /home/ubuntu/cephtest 2024-03-15T13:13:41.468 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-15T13:13:41.480 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-15T13:13:41.504 INFO:teuthology.nuke:Checking targets against current locks 2024-03-15T13:13:41.529 DEBUG:teuthology.nuke:shortname: smithi002 2024-03-15T13:13:41.529 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-15T13:13:41.551 DEBUG:teuthology.nuke:shortname: smithi149 2024-03-15T13:13:41.552 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-15T13:13:41.556 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-15 12:31:22.484210', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBABhPollIpRhiA6TCjJYoOStcnohLcq0SKbzqF8cdvZiE78+44hG40KL3rAizf6AxipHJBQT3p0J83/O8OIc8xQ='} 2024-03-15T13:13:41.572 DEBUG:teuthology.nuke:shortname: smithi184 2024-03-15T13:13:41.573 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-15T13:13:41.579 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-15 12:31:22.482281', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKjHaaL6R+WYZzUH2uBY+xOq0sK99jhOfHkoTIQulkLjcwuEqznoVQ+7uZai3AURfz7ONA9oorFRnVNPvJYpG6o='} 2024-03-15T13:13:41.595 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7601993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-15 12:31:22.486691', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG4pdgFTELesbcqVJl1lUCmcR/+y2/2gIX5pJK68t9x5P0B5uq40G7EtX1gxARnox/nBDZ/Yvjfb5ZWL5O6sPZo='} 2024-03-15T13:13:41.614 INFO:teuthology.orchestra.console.smithi002:Power off 2024-03-15T13:13:41.614 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-15T13:13:41.631 INFO:teuthology.orchestra.console.smithi149:Power off 2024-03-15T13:13:41.632 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-15T13:13:41.643 DEBUG:teuthology.orchestra.console.smithi002:power off output: Chassis Power Control: Down/Off 2024-03-15T13:13:41.643 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-15T13:13:41.655 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off 2024-03-15T13:13:41.655 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-15T13:13:41.674 INFO:teuthology.orchestra.console.smithi184:Power off 2024-03-15T13:13:41.674 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-15T13:13:41.683 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-03-15T13:13:41.684 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2024-03-15T13:13:41.696 DEBUG:teuthology.orchestra.console.smithi184:power off output: Chassis Power Control: Down/Off 2024-03-15T13:13:41.696 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-15T13:13:41.723 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2024-03-15T13:13:45.685 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-15T13:13:45.698 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-15T13:13:45.724 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-15T13:13:45.803 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-03-15T13:13:45.809 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2024-03-15T13:13:45.834 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2024-03-15T13:13:49.806 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-15T13:13:49.816 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-15T13:13:49.835 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-15T13:13:49.918 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2024-03-15T13:13:49.942 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2024-03-15T13:13:49.945 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is off 2024-03-15T13:13:50.041 INFO:teuthology.orchestra.console.smithi149:Power off completed 2024-03-15T13:13:50.043 INFO:teuthology.orchestra.console.smithi002:Power off completed 2024-03-15T13:13:50.045 INFO:teuthology.orchestra.console.smithi184:Power off completed 2024-03-15T13:13:50.150 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/norstats}} duration: 1957.4035789966583 failure_reason: '"2024-03-15T12:50:17.678430+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: scheduled_leonidus@teuthology success: false 2024-03-15T13:13:50.151 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-15T13:13:50.252 INFO:teuthology.run:FAIL